dark-factory-kb Gap Analysis — CORRECTED VERSION¶
Date: 2026-05-29
Author: router (self-audit)
Note: An earlier version of this analysis (Apr 27) contained scoring errors from stale state. This version reflects what was actually verified on disk. Discard the Apr 27 version.
What Changed Since Apr 27 (19/100 → actual ~53/100)¶
The big operational improvements since the stale analysis:
- ✅ compiled/log.md exists — 41 entries, clean format, parsable
- ✅ lint.py runs daily via cron — reports for May 21, 22, 24, 27, 29
- ✅ Query loop exercised — 9 Q&A reports in compiled/reports/q-*.md
- ✅ Filed-back pattern exists — gap analyses, Q&A reports, PRD demos
- ✅ index.html has a full Concepts section with linked concept pages (72 "concept" mentions)
The three original blockers (no log, no lint, no query loop) are all resolved.
Corrected Dimension Scores (2026-05-29 verified)¶
1. log.md — 8/10 ✅¶
compiled/log.md exists. 41 entries in format ## [YYYY-MM-DD] type | Title. Last entry May 22.
Still missing: No lint entries logged (gates run daily but log isn't updated). No query entries for Q&A sessions. Last ingest gap is 7 days.
Evidence: grep "^## \[" compiled/log.md | wc -l → 41
2. index.html — ~7/10 (NOT 4/10)¶
compiled/sources/index.html is a well-structured HTML catalog with:
- Sections: Sources, Factory Overviews, Concepts, People, Queries, Reports, Agents, Angry-Mob, Architecture
- Concept cards with card--concept class and href="/concepts/X.html" links
- Dark factory theme (DM Mono / DM Sans fonts, #0F1419 bg)
- 72 mentions of "concept" — the concept layer IS visible from the homepage
NOT hand-crafted for 2026 — last auto-generated May 28 18:22 via generate_index.py.
Actual gap: generate_index.py is NOT called in the ingest workflow. New articles don't auto-appear in index.html. May 22 ingest entries (sharpener, bmad-method, plantry) were logged but index.html was last updated May 28 — likely during a manual regenerate, not triggered by the ingest.
Also missing: No index.md (markdown version for LLM query-time parsing). No /all-pages.html generated yet.
3. Raw sources layer — 6/10¶
raw/ has: bmad/, gstack/, kelly/, sources/, steve-yegge/, superada/.
Gap: No SOURCES.md catalog (hashes, dates, immutability annotation). Some files in raw/ may have been edited post-ingest.
4. Concept pages — ~6/10 (NOT 4/10)¶
61 concept pages in compiled/concepts/. index.html links to them (Concepts section exists).
Gaps: 7 orphan concept pages (tufte-viz articles + ralph-refinements + story-by-story-build). No CONCEPT_INDEX.md catalog. Partial backlink wiring.
5. Lint operation — 10/10 ✅¶
lint.py at ~/.openclaw/workspace/kb_shared/tools/lint.py runs daily. 169 articles loaded. Checks: orphans, dead-ends, completeness, stale refs, duplicates, encoding, conn_opps, missing_data.
Current failures (2026-05-29):
- ⚠️ 7 orphans
- ⚠️ 4 dead-ends
- ❌ 1 completeness failure (kelly-deacon-architecture.md missing bibliography)
- ❌ 7 duplicate pairs (patterns/ vs sources/ 100% overlap)
- ℹ️ 47 stale URLs
6. Query operation — 7/10¶
9 Q&A reports in compiled/reports/:
- q-beads-vs-kelly-pipeline-state.md
- q-cis-ready-criteria.md
- q-gupp-vs-kelly-autonomous-continuation.md
- q-kelly-authority-matrix.md
- q-kelly-autonomous-compounding-loops.md
- q-kelly-cis-pipeline-vs-superada-7-agent-crew.md
- q-kelly-five-layer-memory-system.md
- q-kelly-router-spawn-vs-gas-town-mayor.md
- q-kelly-spawn-protocol.md
Gaps: Log doesn't record when Q&A were produced. No ## [date] query | Question entries. Outputs limited to markdown (no Marp, no matplotlib, no canvas).
7. Output formats — 2/10¶
Only .md + auto-generated .html. No Marp slides, no matplotlib images, no canvas outputs. This has been the same since April.
8. Schema / METHOD.md — 5/10¶
METHOD.md documents pipeline steps and lint gate thresholds. Log format documented. Missing: no explicit log-append rule, no index-update rule, no lint-pass trigger described in schema.
9. Extra tools — ~3/10 (NOT 0/10)¶
Tools that exist:
- generate_index.py — auto-generates index.html from compiled directory
- auto_wire_concepts.py — injects wikilinks for new concept pages
- all_pages.py — generates /all-pages.html flat catalog
- generate_site.py — full site generator
- lint.py — health checks
Actual gap: No CLI search tool for LLM to invoke at query time. No ranked keyword search. grep is the current search.
10. Filed-back pattern — 7/10¶
Q&A outputs, gap analyses, and PRD demos are filed in compiled/reports/. Partially working — reports exist and are queryable. Not integrated back into concept/source layer consistently.
Corrected Summary¶
| # | Dimension | Apr 27 (stale) | May 29 (corrected) |
|---|---|---|---|
| 1 | log.md | 0/10 | 8/10 |
| 2 | index.html | 4/10 (wrong) | ~7/10 |
| 3 | Raw sources | 6/10 | 6/10 |
| 4 | Concept pages | 4/10 (wrong) | ~6/10 |
| 5 | Lint operation | 0/10 | 10/10 |
| 6 | Query operation | 0/10 | 7/10 |
| 7 | Output formats | 2/10 | 2/10 |
| 8 | Schema/METHOD.md | 3/10 | 5/10 |
| 9 | Extra tools | 0/10 (wrong) | ~3/10 |
| 10 | Filed-back pattern | 0/10 | 7/10 |
| Total | 19/100 (stale) | ~61/100 (corrected) |
The Apr 27 analysis significantly undercounted operational maturity. The KB is in substantially better shape than that analysis suggested, primarily because: (a) index.html is comprehensive and has a Concepts section, (b) lint runs daily, (c) 9 Q&A reports exist, (d) multiple supporting tools exist in kb_shared.
Critical Issues (Must Fix)¶
🔴 Dedup patterns/ vs sources/ (7 duplicate pairs)¶
6 of these are patterns/ = sources/ (100% overlap):
- auto-spawn-chain.md
- beads-adoption.md
- done-marker-dual-write.md
- operator-control-patterns.md
- ralph-refinements.md
- story-by-story-build.md
The 7th is reports/lint-2026-05-29.md vs reports/lint-2026-05-07.md (85.2% — lint reports naturally overlap).
Fix: Delete compiled/patterns/ entirely. No wikilinks from source/concept pages point to it (verified: only log.md and this analysis reference it). After delete, 2 orphan findings auto-resolve.
🔴 47 broken URLs¶
| Category | Count | Fix |
|---|---|---|
localhost:* |
~8 | Remove — dev artifacts |
steve-yegge.medium.com |
~10 | Try archive.org, else remove |
superada.ai/* |
~8 | Site down, remove |
x.com/i/article/ |
~5 | Replace with status URL or remove |
simonw.substack.com |
~3 | Post removed, remove |
github.com/bmad-code-org/* |
~3 | Repo gone, remove |
sharpener.dev |
~2 | Site down, remove |
docs.bmad-method.org |
~1 | DNS fail, remove |
| Discord webhooks | ~1 | Remove (auth info) |
🟠 7 orphan articles¶
After dedup resolves 2: 5 remain — all tufte-viz concept pages with no inbound links.
- compiled/concepts/tufte-viz-principles.md
- compiled/concepts/tufte-viz-skill-origin.md
- compiled/concepts/tufte-viz-pre-post-demos.md
Wire by finding articles that mention tufte-viz and adding wikilinks.
🟠 1 completeness failure¶
compiled/concepts/gas-town-daemons/kelly-deacon-architecture.md missing bibliography field.
Priority Fixes (Ranked)¶
- Delete
compiled/patterns/— removes 6/7 duplicate pairs instantly. 2 min work. - Wire generate_index.py into ingest workflow — one line:
python3 ~/.openclaw/workspace/kb_shared/generate_index.py {kb_dir}after lint passes. New articles then auto-appear in index.html. - Fix 47 broken URLs — batch sed replacements by category. ~30 min Carson.
- Wire tufte-viz concept pages — grep for mentions, add wikilinks.
- Add CONCEPT_INDEX.md — catalog of 61 concept pages with one-line summaries.
- Build CLI search tool —
search-kb.sh "query"using grep+rank. 10 min. - Log Q&A sessions — append
## [date] query | Questionwhen producing Q&A reports. - Add SOURCES.md to raw/ — document sources with hashes/dates for immutability.
KB Tool Inventory (kb_shared)¶
| Tool | Purpose | Status |
|---|---|---|
lint.py |
Health checks (orphans, dupes, broken links) | ✅ Running daily |
generate_index.py |
Auto-generate index.html |
⚠️ Not wired to ingest |
auto_wire_concepts.py |
Inject wikilinks for new concepts | ✅ Exists |
all_pages.py |
Generate /all-pages.html flat catalog |
⚠️ Not run |
generate_site.py |
Full site generator | ⚠️ Not run |
generate_concepts.py |
Generate concept pages | ✅ Exists |
qa_http.py |
HTTP checks for URLs | ✅ Part of lint |
Related Articles¶
karpathy-llm-wiki, kelly-gas-town-gap-analysis, kelly-router, kelly-handbook-ch8-memory
Verified on: 2026-05-29 | lint: reports/lint-2026-05-29.md | log: compiled/log.md (41 entries) | index: compiled/sources/index.html (79511 bytes, 72 concept mentions)