98% of my AI token spend was overhead. Here are the 5 changes that cut it

Last week I shared a measurement: in my agentic sessions, 85-98% of token spend was re-reading context.

My longest run consumed 16.2 million input tokens to process just 357 tokens of fresh user text.

This is the practical follow-up: what I changed.

1) Compact Mid-Session

History growth is multiplicative. In one 186-turn session, per-turn context climbed from roughly 23k to 127k tokens.

I now compact around turn 25-30. Estimated reduction on later turns: 60-70%.

This is the highest-impact lever.

2) Put Instruction Files on a Diet

Anything auto-loaded is re-read every turn.

My project files had drifted to 8k-15k tokens. I removed stale decisions, closed items, and long historical notes.

Current rule of thumb:

• Entry file under 150 lines
• Project files under 80 lines

3) End with a Handover, Not a Marathon Session

Long sessions compound cost non-linearly.

I end with a short handover file that captures decisions, open items, and file pointers. The next session starts clean at a smaller baseline instead of inheriting a bloated thread.

4) Split Entry Context from Detailed Context

Keep always-loaded context minimal.

Load detailed docs only when needed. Default-loaded context is expensive; on-demand context is comparatively cheap.

5) Route Small Tasks to Small Models

Simple transformations do not need frontier reasoning every time.

This is not the biggest savings source, but it is a reliable secondary reduction.

The Core Mechanism

In agentic workflows, you pay for what is loaded, not only for what you ask.

If you are optimizing costs, start by measuring your context behavior and session shape before tuning model choices.

Source post: https://www.linkedin.com/posts/wilcodetree_agenticai-ai-llm-activity-7471583252852654080-nXbm