EcoThink: A Green Adaptive Inference Framework for Sustainable and Accessible Agents
About
As the Web transitions from static retrieval to generative interaction, the escalating environmental footprint of Large Language Models (LLMs) presents a critical sustainability challenge. Current paradigms indiscriminately apply computation-intensive strategies like Chain-of-Thought (CoT) to billions of daily queries, causing LLM overthinking, a redundancy that amplifies carbon emissions and operational barriers. This inefficiency directly undermines UN Sustainable Development Goals 13 (Climate Action) and 10 (Reduced Inequalities) by hindering equitable AI access in resource-constrained regions. To address this, we introduce EcoThink, an energy-aware adaptive inference framework designed to reconcile high-performance AI intelligence with environmental responsibility. EcoThink employs a lightweight, distillation-based router to dynamically assess query complexity, skipping unnecessary reasoning for factoid retrieval while reserving deep computation for complex logic. Extensive evaluations across 9 diverse benchmarks demonstrate that EcoThink reduces inference energy by 40.4% on average (up to 81.9% for web knowledge retrieval) without statistically significant performance loss. By mitigating algorithmic waste, EcoThink offers a scalable path toward a sustainable, inclusive, and energy-efficient generative AI Agent.
Related benchmarks
| Task | Dataset | Result | Rank | |
|---|---|---|---|---|
| Math | GSM8K | Accuracy0.945 | 206 | |
| Commonsense Reasoning | StrategyQA | Accuracy90.5 | 174 | |
| Commonsense Reasoning | ARC-C | Accuracy93.1 | 172 | |
| Truthfulness | TruthfulQA | Truthfulness Accuracy88.7 | 86 | |
| Reasoning | ARC-C | -- | 80 | |
| Math Reasoning | SVAMP | Accuracy92.8 | 40 | |
| Dialogue | MT-Bench | MT-Bench Score8.9 | 29 | |
| Question Answering | WebQuestions | WebQ Accuracy79.4 | 11 | |
| Web Knowledge Retrieval | HotpotQA | Accuracy87.6 | 7 | |
| Web Knowledge Retrieval | TriviaQA | Accuracy90.2 | 7 |