Chapter 8: Memory & Context Management

Context is the most underrated challenge in AI automation. AI models process information through a context window—current models handle 100K+ tokens but context still has hard limits. When context overflows, older content gets compacted (summarized and compressed); the agent still works but some earlier details are gone. The critical rule: don't rely on conversation history for important state. Write it to files—files survive session restarts; conversation history doesn't. Sub-agents with long-running tasks can exhaust their context, causing repetitive responses, forgotten instructions, or incoherent outputs. The fix is breaking tasks into smaller chunks rather than asking for "everything about X" in one shot.

OpenClaw has a 5-layer memory system. Layer 1: SOUL.md is the agent's identity—tone, communication style, core principles, domain expertise, recurring rules. It's loaded at every session start and shapes all behavior. Written once, updated rarely, drives long-term patterns. Layer 2: MEMORY.md stores persistent cross-session facts—routing decisions, operator preferences, system configuration, known issues, escalation patterns. The agent writes to it when it learns something worth remembering; the operator edits it when context changes. Layer 3: Daily memory logs (memory/YYYY-MM-DD.md) record operational events—projects initiated, completions, failures and resolutions, gate passes/fails, operator decisions, notable findings. Layer 4: Project context files (projects/{id}/context.md) accumulate project-specific state updated as projects evolve, enabling sub-agents spawned months later to read history and resume without re-work. Layer 5: Structured data files (data/.json, CSVs) are read and written by processes, not directly by agents.

Context overflow is survivable if you've built recovery into the system. When it happens—signs include forgotten earlier instructions, repeated questions, [compacted: tool output removed] messages—write current state to a file immediately, then start a fresh session with "Continue project X. Read context.md for background." SOUL.md should include memory writing rules as explicit instructions: after any significant task, update MEMORY.md if you made a decision worth remembering, discovered something about the system, learned a preference, or solved a non-obvious problem. A weekly cron maintenance task reads all memory files from the past week, identifies patterns or learnings for MEMORY.md, and proposes updates—keeping the memory system current without requiring manual maintenance.

Key Patterns

• **Files over conversation history:** Write important state to files; conversation history doesn't survive session restarts
• **5-layer memory stack:** SOUL.md (identity) → MEMORY.md (persistent facts) → daily logs → project context → data files
• **Compact on overflow:** Break long tasks into smaller chunks; design every project with a context.md for recovery
• **Explicit memory writing rules in SOUL.md:** What to write to MEMORY.md and when
• **Weekly memory maintenance cron:** Automated review of memory files to keep MEMORY.md current

Related Concepts

• [[kelly-handbook-ch7-multi-agent]] for how multi-agent work writes to memory layers
• [[kelly-handbook-ch2-architecture]] for session architecture and context window mechanics
• [[kelly-research-transcripts]] for research-context memory patterns
• [[kelly-handbook-ch6-cron]] for the weekly memory maintenance cron job pattern