The advent of Large Language Models (LLMs) has raised concerns regarding their enormous carbon footprint, beginning with energy-intensive training and continuing through repeated inference.
The final training run for a model with 175 billion parameters can emit an estimated 550 metric tons of carbon dioxide. When accounting for the full life cycle, including hardware manufacturing and model development, the footprint is even more substantial. While training generates a significant portion of emissions, inference occurs consistently and on a large scale. Recent estimates suggest that inference can account for up to 90 per cent of a model’s total life cycle energy use.
This study investigates the potential of using fine-tuned Small Language Models (SLMs) as a sustainable alternative for predefined tasks. Findings demonstrate the viability of smaller models in mitigating the environmental impact of resource-heavy LLMs. Significant improvements have been observed in performance after fine-tuning for tasks such as sentiment analysis and content creation. For instance, the carbon emitted during these tasks can be up to 13,000 times lower per query when using an SLM compared to a large-scale model. Performance for content creation remains largely unchanged, yet the emissions are substantially reduced.
Using fine-tuned SLMs lowers both capital and operational expenditure, demonstrating that sustainable Green AI is not just ethically beneficial but also financially advantageous.
Despite fine-tuning, SLMs currently struggle to match LLMs for complex tasks such as chain-of-thought reasoning and functional code generation. These tasks typically require models with billions of parameters to achieve accuracy. This indicates that the answer to the energy dilemma is to fine-tune SLMs for the specific tasks where they excel, rather than attempting to replace large models entirely. This approach points toward a future of more energy-efficient AI deployment, where curated task-specific fine-tuning achieves strong performance with a drastically lower carbon footprint.
Firms must balance the benefits of AI adoption with strategies to mitigate environmental costs. This challenge extends beyond environmental impact to financial implications, which can be analysed using a Total Cost of Ownership (TCO) framework. Inference is not a one-time capital expense but rather a continuous operational cost depending on energy consumption. Deploying large models requires significant capital expenditure for clusters of expensive GPUs. Using fine-tuned SLMs lowers both capital and operational expenditure, demonstrating that sustainable Green AI is not just ethically beneficial but also financially advantageous. Future regulatory efforts should include mandates for providers to disclose the energy costs of inferences so that overall energy consumption may be accurately calculated.
The team at Academii are always happy to discuss all your training and education needs, help your organisation attract and train new talent, and build a resilient workforce. Please drop us a line here to know more.
