Model-Size Agnostic Iteration¶
Type: Principle / finding
Referenced from: closed-loop-agent-control
Definition¶
Model-Size Agnostic Iteration is the principle that tooling quality and feedback loop completeness matter more than model capability past a certain threshold. A smaller, cheaper model operating inside a closed feedback loop can outperform a frontier model that relies on human-observed feedback.
This directly challenges the assumption that model quality is the primary determinant of agent performance.
The Evidence¶
Nikita M.'s experiment:
- Smaller model: DeepSeek V4 Flash (sub-agent model) with a closed feedback loop (3 tools)
- Stronger model (implied): Frontier model with human monitoring and direction-giving
- Result: The smaller model with the closed loop was "much much more effective"
The experiment isolated one variable: feedback loop completeness. Same task, same game engine, same coding agent — only the feedback mechanism differed.
Why It Works¶
| Factor | Frontier Model + Human Loop | Smaller Model + Tool Loop |
|---|---|---|
| Latency | Human observation → interpretation → articulation (seconds to minutes) | Tool call roundtrip (milliseconds) |
| Information fidelity | Lossy human description | Complete structured state |
| Temporal control | Human decides when to check | Agent decides when time advances |
| Iteration speed | Bounded by human attention | Bounded only by compute |
Implications for Agent Design¶
- Invest in tooling before model upgrades — a mediocre model with excellent tools beats a brilliant model with poor tools
- Close loops, don't add humans — human-in-the-loop is often a workaround for missing tools, not a feature
- Cost efficiency — smaller models with good loops may be more cost-effective than frontier models with human intermediation
- Threshold effect — there's likely a minimum model quality required; below it, no amount of tooling helps
Related Concepts¶
- closed-feedback-loop — the architectural principle this validates
- tool-mediated-game-state-control — the tooling that enabled this result
- time-as-knob — temporal control as the key loop mechanism
- sharpener — the broader principle of improving capability through tooling, not model size