What if switching to a stronger AI model mid-long-running-task made the outcome worse?

That’s one of the most important findings from a new paper on long-running AI agents: “The Handoff Tax.”

The researchers studied what happens when an AI agent switches models during a task - for example, moving from a cheaper model to a more capable one when the first model struggles.

The intuitive assumption is simple:

Start cheap. Escalate when needed. Save money.

But the results show it’s not that simple.

When a stronger model inherits the full reasoning trajectory of a weaker model, it can get anchored by the weaker model’s assumptions, dead ends, and mistakes.

In the GPT experiments, a raw handoff from the lower-capability model recovered just 36% of the quality gap to the stronger model. But when the researchers removed the previous reasoning history while keeping the code changes, quality recovery jumped to 84%.

That is a big difference. And the reverse was also true.

When a strong model handed work down to a cheaper model, preserving the strong model’s reasoning helped. Removing that context reduced quality.

The implication is powerful:

AI handoffs are asymmetric. Strong models may benefit from forgetting weak-model reasoning. Weak models may benefit from inheriting strong-model reasoning.

This changes how we should think about agent architecture. The question is no longer just:

“Which model should act next?”

It is also:

“What should the next model inherit?”

A few practical implications for enterprise AI:

  1. Don’t assume “more context” is always better.
  2. Preserve verified work products separately from speculative reasoning.
  3. Design different handoff strategies for escalation vs downshift. Treat handoff design as part of the agent policy, not as plumbing.

Example 1: Financial Research

A high-capability model performs the difficult analysis, identifies risks, and builds the core investment thesis. The task is then handed to a cheaper model to draft summaries, reports, and follow-up communications. Keeping the expert model’s reasoning gives the cheaper model a strong scaffold instead of forcing it to rediscover the analysis. Impact: retain quality while shifting lower-value execution to a lower-cost model.

Example 2: Cybersecurity Incident Response

A low-cost agent continuously triages alerts, gathers logs, runs routine checks etc When a complex attack is detected, the case is escalated to a more capable model. Instead of passing weak hypothesis the handoff preserves evidence, actions taken, and current system state. Impact: the expensive model starts with useful facts improving response quality while avoiding wasted mistakes by the lower cost model.

Ref: The Handoff Tax: Continuing Non-Native Trajectories in LLM Agents