Tool-Mediated Game State Control

Type: Tool pattern / mechanism
Referenced from: closed-loop-agent-control

Definition

A three-tool extension that provides a coding agent with a complete sensory-motor loop over a running game engine:

Tool Function Loop Role
getState Reads current game state via JSON HTTP endpoint embedded in the binary Perception — structured view of the world
adjustClock Advances the game clock so game logic runs in lockstep with agent reasoning Temporal control — decides when the world advances
sendInput Sends inputs to control the game (flap, shoot) Action — acts on the world

Why It's Different From Human-in-the-Loop

The tools form a complete sensory-motor loop where the agent is inside the loop, not observing from outside. The agent:
- Reads state directly (no screenshots, no human interpretation)
- Acts directly (no human articulating directions)
- Controls when the simulation advances (no waiting for human reaction)

This is fundamentally different from a human watching a screen and telling the agent what to do.

Implementation Context

  • Model: DeepSeek V4 Flash (opencode-go sub)
  • Game engine: Bevy (Rust)
  • Coding agent: Pi coding agent
  • Task: Build Flappy Bird, then add a space-invaders shooting mechanic
  • Scaffolding: Minimal; the Pi extension with three tools was the key addition

Key Insight

The adjustClock tool is the most novel element. By letting the agent control game time, it can:
- Advance frame-by-frame for precise debugging
- Skip ahead to test specific scenarios
- Pause to reason about state
- Replay interactions to verify fixes