UpShaqo
Intelligence desk
Agent Workflows Source-backed analysis

Anthropic Shrinks the Token Tax Slowing Down Coding Agents

A new Messages API feature stops long-running Claude agents from resending entire conversation histories on every turn, cutting median response wait times by roughly a third in heavy sessions.

UpShaqo Editorial IntelligenceNovember 3, 20266 min read
Intelligence standard

Independent UpShaqo analysis built from fresh, attributed sources. We explain the impact instead of repeating the announcement.

Read for leverage: focus on the workflow change, the customer problem, and the next action—not only the product announcement.

A coding agent that has been working for twenty minutes on a messy refactor doesn't just remember what it did five minutes ago — it has to be told, in full, every single time it speaks again. Every tool result, every file diff, every screenshot gets repackaged and resent to the model with each new request. By the time a session stretches past a few hundred thousand tokens, that resend becomes the bottleneck: the agent is waiting not because it's thinking hard, but because it's re-reading its own homework before it can answer.

Anthropic's fix, rolled out this week as a public beta in the Messages API, is almost embarrassingly simple in concept: stop making the agent repeat itself. Message threads let a client send the full conversation once, then transmit only what's new on each subsequent turn, along with the ID of the previous response. Anthropic holds the thread in memory — encrypted, never written to disk — for five minutes after the last request, after which the client has to resend the whole conversation to start fresh.

The Part That Actually Moves the Needle

The headline number is specific and verifiable: for requests between 500,000 and 1,000,000 tokens, median time to first token drops by about 35% with message threads enabled, according to Anthropic's own published figures. That's not a marketing abstraction — it's the measurable gap between a developer staring at a spinner and a developer getting the next line of code.

Crucially, this is a latency fix, not a cost fix. Claude still reads the entire conversation on every turn behind the scenes, so token usage and billing stay the same. What shrinks is the size of the request payload itself — the thing that has to travel over the wire and get parsed before the model can start generating. For short exchanges this barely matters. For a Claude Code session that's been running for an hour, it's the difference between a tool that feels instant and one that feels like dial-up.

Why This Problem Was Getting Worse, Not Better

The quiet irony here is that better agents created this bottleneck. As coding agents, computer-use agents, and document-heavy workflows got more capable, they also got chattier — hundreds of requests per session, each one dragging the full history behind it like an anchor. Anthropic names three workloads explicitly as the intended beneficiaries: long-running coding agents where every tool result adds to the conversation, computer-use agents where screenshots previously had to be re-uploaded on every turn, and document or image-heavy pipelines where payload size compounds fast. In each case, the pattern is the same — context keeps growing, and until now, so did the tax for carrying it.

A Scenario: The Catalog Video Pipeline

Consider a small e-commerce tooling team that wires a coding agent in Claude Code to Runway's developer plugin, which connects coding agents directly to Runway's API for generating product video from catalog data. The workflow: describe the feature — "a product video for every new item in the store" — and let the agent build it end-to-end, reading account details, selecting models through a Runway Model Router, and debugging failures as they surface.

That kind of session is exactly the shape message threads was built for. Each turn adds a tool call, a model response, an error trace, maybe a thumbnail reference. Without message threads, every one of those turns resends the accumulating history, and by turn two hundred the agent is dragging a payload heavy enough to noticeably slow down each response. With message threads on, the team sends only the incremental update each time. The agent still has full context — Claude is still reading everything — but the network round-trip shrinks, and the wait before the agent starts typing its next move drops in proportion to how much history has piled up.

For this team, the practical effect isn't a headline metric. It's that a long build-and-debug loop that used to involve visible lag between steps now feels closer to continuous. That matters more than it sounds: in agent-driven development, the friction that kills adoption usually isn't capability, it's the dead air between turns that makes a developer tab away and lose the thread themselves.

What Doesn't Change, and Where the Caution Belongs

Message threads is not a silver bullet, and Anthropic is specific about its limits. The benefit scales with how much context an agent is carrying — a short chatbot exchange won't notice the difference, because there's little history to avoid resending. The five-minute thread expiration also means genuinely idle agents lose the optimization and fall back to full resends, so workflows with long human-in-the-loop pauses won't see the same gains as continuously active sessions. And because this is a beta feature layered into the Messages API, teams integrating it into production pipelines should treat it the way they'd treat any beta: test latency improvements against their own workload shape rather than assuming Anthropic's 500k–1M token benchmark applies uniformly to shorter sessions.

Where Teams Can Use It Today

The feature is live now with the thread parameter on the native Claude Platform, and it's also available through Claude Platform on AWS, Claude in Microsoft Foundry, and Claude Models on Google Cloud. Standard token rates and existing rate limits still apply, and Anthropic notes it's already on by default for most Claude Code users — meaning many developers are likely benefiting from the latency drop without having touched a configuration setting.

What to Do With This

For teams running agents with long, tool-heavy sessions — coding assistants, computer-use agents, or anything chaining third-party API calls like Runway's — the practical next step is straightforward: check whether message threads is active in your integration, and if you're calling the Messages API directly rather than through Claude Code, explicitly add the thread parameter and measure time-to-first-token before and after on your actual workload. Because token usage is unchanged, there's no cost tradeoff to weigh — only an infrastructure decision about whether your client architecture can track thread IDs and handle the five-minute expiration cleanly. For anyone building on top of Claude Code today, the more interesting question is less "should we adopt this" and more "how much latency have we already been quietly losing to full-history resends without noticing."

Sources

#Claude API#Anthropic#Agent Latency#Claude Code#Message Threads#Developer Tools

Two doors. Pick one.

Hire the team.
Or become it.