Memory System

The Memory System is the factory's persistence infrastructure — a dual-layer architecture combining narrative memory files with a semantic search layer that delivers sub-20ms lookups. Published as memory-system-v2 on ClawdHub, this system evolved from basic daily markdown logs into a high-performance memory architecture that enables the factory's autonomous operation by ensuring every session starts with relevant context from all prior work.

The problem the Memory System solves is context continuity: an autonomous builder that forgets what it did yesterday can't compound learnings. Without persistent memory, each session starts empty and the factory is condemned to repeat the same mistakes. The 5-layer memory system (see five-layer-memory) provides the structural layers, but the Memory System adds the critical capability of fast, semantically meaningful retrieval.

The dual-layer architecture consists of two integrated components:

Layer 1 — Narrative Memory (Files): The foundation is the established 5-layer system — SOUL.md (identity), MEMORY.md (learned facts), daily logs (memory/YYYY-MM-DD.md), project context (projects/{id}/context.md), and structured data (data/.json). These files are the source of truth. They are human-readable, narrative-formatted, and durable. Every significant decision, discovery, or pattern gets written here. The narrative format is intentional — LLM recall is associative, not indexed, so narrative content compounds more effectively than tables or structured data at the top layers.

Layer 2 — Semantic Index: A searchable index overlaid on the narrative files. When content is written to memory, it's also indexed in a semantic search engine (typically a local vector store or lightweight JSON-based index). When a query comes in — "what have we learned about App Store ASO?" — the semantic index returns relevant passages in under 20ms. The index is not the memory; it's an acceleration structure over the memory. The index can be rebuilt from the source files, but queries go through the index for speed.

The sub-20ms lookup time is a deliberate engineering constraint. At the speed the factory operates — spawning multiple agents in parallel, running overnight builds, processing dozens of opportunities — memory retrieval can't be a bottleneck. 20ms means the memory system adds negligible latency to any operation that needs to look up prior context. If a lookup took 500ms, agents would time out or context would stale before retrieval completed.

The Memory System is what makes the autonomous builder identity viable. When the Router (see kelly-router) wakes up each session, it loads SOUL.md and MEMORY.md (the small map) and is immediately operational with the full context of prior work. It doesn't need to re-learn the factory's operating procedures, the operator's preferences, or the current pipeline state — that's all in memory. The project context files load on demand when entering a project scope, giving targeted context without bloating the base session.

The semantic search capability also enables the learning loop (see learning-loop) to function at scale. When a new app build surfaces a pattern worth capturing, the Memory System can quickly identify related prior patterns across dozens of previous builds — "we've seen this competitive gap before; here's what we tried and what worked." This cross-build pattern matching is what makes the factory compounding rather than repetitive.

The evolution to memory-system-v2 was documented publicly: starting from basic daily markdown logs, the system upgraded to include semantic indexing when the factory's output volume made manual memory search untenable. At 85+ iOS apps built, the memory corpus was too large to search manually, and the 5-layer system's linear file scanning was too slow for real-time agent operations. The semantic layer solved both problems.