Microsoft Research and UC Berkeley have released TokenEconomizer, a free plugin that cuts AI coding agent costs by learning from earlier sessions in the same repository. On 277 SWE-Bench tasks, mean token cost fell 18.4% to 30.9% across five agent-model pairings, MarkTechPost’s write-up of the paper and ProPakistani’s cost breakdown reported. The plugin leaves model weights and agent code unchanged and runs beside Claude Code, Codex, GitHub Copilot CLI, and Pi.
Key Takeaways
- TokenEconomizer cut mean SWE-Bench cost 18.4% to 30.9% across five pairings.
- Pass@1 rose in four of five setups; Claude Code with Opus 4.8 fell from 74% to 73%.
- Removing unused tools saved the most on its own, at 13.2% to 17.7%.
- It is Apache 2.0, needs Python 3.10 or newer, and stores lessons locally.
Where Coding Agents Burn Tokens
Every turn re-bills the system prompt, every loaded tool schema, and the full history. A profile of 60 SWE-Bench tasks across six repositories, using Claude Code with Opus 4.8, found three recurring wastes.
- Repeated discovery: once five tasks precede the current one, 30% of tool calls repeat an earlier action.
- Unused tools: 23 schemas load every turn, 16 are never called, and three cover 98% of calls.
- Failure loops: 10 failed runs were 17% of tasks but 32% of total spend.
The project page states the failure split directly:
The 17% of tasks that fail take 32% of the spend
RTK, Serena, and Aider’s RepoMap each cut cost on Django and raised it on SymPy. None of those fixed helpers was cheaper on every repository tested.
Three Controls That Sit Outside the Model
TokenEconomizer treats the agent as a black box and keeps per-repository state from completed traces. Policy is set through the prompt, tool settings, and the event stream. It has no API key of its own; model calls route through the user’s agent and are counted in the reported cost. Learned state stays in ~/.tokeneconomizer.
An LLM proposer turns trace digests into one-line facts, such as a working test command. Regex detectors check later traces without a model call. A fact enters the prompt after three confirmations on tasks with non-overlapping edited files, and only if its need rate stays at or above 0.4. One contradiction counts as five confirmations, so a wrong fact retires quickly.
Tool pruning always keeps shell, read, write, and edit. Other tools are masked after three consecutive unused tasks, or when an LLM judge marks them irrelevant. The full catalog returns every five tasks for a re-audit. On Django with Claude Code, the first request on task 2 fell from 43,351 to 16,295 prompt tokens.
The failure-loop guard sets a turn budget at 1.25 times the 90th percentile of successful run lengths. An LLM judge must reach 0.7 confidence before calling the agent stuck. A stuck run gets one redirect, then a stop if the loop returns.
What SWE-Bench Says About AI Coding Agent Costs
The paper, arXiv 2610.22732, ran the same 277 tasks from six repositories, three seeds per setting, with controller calls in the bill. FourWeekMBA quoted the paper’s claim of lower cost “with comparable task success”. Pass@1 rose in four of five pairings.
Claude Code with Opus 4.8 slipped from 74% to 73% and still posted the largest cut, 30.9%. Codex with GPT-5.6 Sol saved 26.1%, ProPakistani reported from Table 1.
| Test | Mean cost change |
|---|---|
| SWE-Bench, five pairings | 18.4% to 30.9% lower |
| Claude Code with Opus 4.8 | 30.9% lower |
| That pairing plus RTK | 33.2% lower |
| Codex with GPT-5.6 Sol | 26.1% lower |
| Web-Bench, outside SWE-Bench | 16.0% to 21.2% lower |
| CredData, outside SWE-Bench | 16.5% to 19.6% lower |
| ReposVul audits, outside SWE-Bench | 6.4% to 9.3% lower |
Most repositories cost more at first, and all six tested projects were cheaper by the sixth task. Alone on Claude Code, tool pruning saved 13.2% to 17.7%, micro-skills 11.8% to 14.3%, and the loop guard 5.4% to 7.1%. Lower reasoning effort also cut spend, but Pass@1 fell 1.8 to 10.3 points.
Why a Learned Controller Beat Fixed Compressors
The plugin was cheaper in all 30 repository-setting cells. RTK raised cost in 12 of those cells, and Aider RepoMap in 15. It adjusts what it injects and hides for that repository, instead of applying one recipe to Django and SymPy alike.
Install Notes and the Production Gap
The release is Apache 2.0, needs Python 3.10 or newer, and runs without a GPU. On Claude Code, clone the repo, add it as a marketplace, then run claude plugin install tokeneconomizer@tokeneconomizer. Pruning and the loop guard are on by default; micro-skills start off, because the proposer adds model calls. The README says the plugin never handles credentials.
Teams comparing agent spend, including GitHub Copilot Fusion’s general availability, should not treat these author-run benchmarks as a production invoice. FourWeekMBA found no independent replication and no production-workload numbers in the pages it reviewed. For engineering managers, the useful split is harness versus model: the plugin does not make a token cheaper, it stops long-lived repositories from rebilling the same discovery, unused schemas, and stuck loops. A one-off audit in a fresh repo will not show the same curve.
