What was studied
The July 24, 2026 preprint The Regression Tax compares nearly 6,000 runs across two office-automation benchmarks and three agent harness stacks. It defines a regression as a task solved without a skill but failed after adding one, separating that from residual failures that occur both ways.
Three regression modes
- Skill-description osmosis: a skill changes behavior merely by appearing in context, even when never invoked.
- Grounding displacement: the prescribed procedure overrides interpretation of the actual input.
- Verification displacement: following the procedure suppresses checks the agent would otherwise perform.
The authors report that stronger skills often lead by destroying fewer existing successes rather than creating many more new successes. Aggregate success rates hide that cost.
A better evaluation
Teams should record new gains, regressions, and persistent failures separately. Skills need explicit grounding to source inputs and an independent result check after the procedure. Regression sets should include tasks where the skill is not supposed to run, and tests should be repeated when the model or harness changes.
Limits
This is a preprint without peer review. Results from two benchmarks and three harness stacks may not generalize to every agent, and regression rates depend on model, skill design, and task distribution. The conclusion is not that skills are inherently bad, but that their net effect must be decomposed and measured.