A hands-on build. One agent, two Python scripts, a small dashboard.
Do you actually know
what your AI is doing?
Build a stack that puts
you back in control.
Track what works. Catch the pointless and the risky before they spread. One project becomes the home base for your whole AI strategy, learning your context and getting sharper every week.
01 · Why it matters
Your Stack Becomes Your Strategy
Not a list. The layer that lets you see what works, guard against pointless and risky AI, and steer the whole thing.
Most teams keep bolting on AI tool after AI tool and lose the plot. Nobody knows what is running, what is working, or what is quietly risky. This project is the fix. It is not a list. It is the layer that turns a pile of tools into a strategy you can see and steer.
You build, then you can actually see what earns its place. What gets used, what just sits there, what has quietly drifted out of sync.
Your stack becomes your guard against pointless and potentially malicious AI. You see every tool you run, so nothing creeps in unnoticed and spreads across your projects.
One place every AI decision runs from. Not a chatbot bolted on the side. The home base for your whole AI strategy.
It understands your context and what you have built, not what someone built once. It is not frozen. It gets sharper the more you run it.
This is the difference between using AI and running it. A list tells you what you have. This tells you what is working, what to cut, and where to go next. The three small pieces in this guide are how you get there.
02 · The one idea
You Curate. The Agent Maintains.
You bring the judgement. The agent keeps it current. Get that split right and the map gets better every week.
Here is the split that makes the whole thing work. You bring the judgement: what matters, why you picked a tool, how two things connect. The agent handles the upkeep: updating links, spotting what has drifted, keeping every page in sync. You curate. The agent maintains. Do it the other way round and the thing rots.
A script finds the facts. You add the meaning. The agent keeps it current.
This follows Andrej Karpathy's LLM-wiki pattern. A flat list of what you have is something a script can build on its own. But why you chose a tool, how two tools work together, what you decided and when, no script can ever know that. So you split the two. The cheap, factual layer regenerates from scratch every run. The layer that holds your judgement is written once and improved forever.
Reference: Karpathy's LLM-wiki gist, the pattern this whole build follows.
03 · The shape
Three Pieces, Three Questions
Manifest, wiki, dashboard. What do I have, how does it connect, show me. Each answers one.
Three pieces. Each answers a different question. This is a stripped-down version of a much larger system. The big one tracks hundreds of skills across dozens of projects and runs a vector layer. You need none of that to start. This is the spine.
Manifest
A script scans your projects and writes a list of every skill, MCP, plugin, and agent. Two files: ai-stack.md for you, inventory.json for the dashboard.
Wiki
A folder of linked markdown pages. You and your agent write them. A small engine builds the search index and the graph. This is the part that compounds.
Dashboard
A small Next.js app that reads those files and renders them in your browser. Three routes: Glance, Registry, Wiki.
Why the split matters
The manifest is a flat list of facts a script can find on its own. The wiki holds the things a script can never know: why you chose a tool, how two tools work together, what you decided and when. Keep them apart and the cheap, factual layer never overwrites the considered one.
New to skills, MCPs, and agents? Start with the agent skills guide, then come back and track them all here.
04 · The split
Sources, Wiki, Index
Three layers underneath. Read-only sources, the wiki as the truth, and a rebuildable index on top.
Underneath both the manifest and the wiki sit three layers. Getting this order right is what keeps the system honest: the expensive layer never gets clobbered by the cheap one.
Sources
Read-onlyYour actual projects, skills, and configs. The engine reads them and never edits them. The ground truth on disk.
Wiki
The truthMarkdown pages the agent writes. This is the source of truth for meaning. It cannot be regenerated, so it is the layer you protect.
Index
DerivedA SQLite database and a graph, built from the markdown. Never hand-edited, always rebuildable. Throw it away and rebuild it any time.
Sources flow up into the wiki. The wiki flows up into the index. Nothing flows back down. Edit a source or a wiki page, re-run the engine, and the index rebuilds itself.
Enjoying the guide?
Enter your details to unlock the remaining insights. We'll also send you the complete guide as a reference.
05 · Pillar one
The Manifest: What You Have
A script scans your projects and writes the list. Curate, scan, merge, render. The drift flag is the trick.
One Python script, standard library only. It is regenerated from scratch every run, so it is always a true picture of what is on disk right now. The loop has four steps.
Curate
A hand-written dictionary of the why the filesystem cannot know.
Scan
Walk your projects, collect facts: paths, names, content hashes.
Merge
Collapse duplicate copies of the same skill into one entry, flag drift.
Render + diff
Write the JSON, write the manual, append what changed to the changelog.
$ AI_STACK_SCAN_ROOT=~/projects python scripts/sync.py Synced: 12 skills, 5 orchestrators, 6 MCPs, 1 plugin.
What it writes (trimmed)
{ "counts": { "skills_unique": 12, "skills_drifted": 2, "mcps": 6, "plugins": 1 }, "skill_registry": [ { "name": "pdf-extractor", "usage_count": 3, "drift": true, "description": "Pull text and tables out of PDFs." } ] }
The trick: the skill registry
One logical skill can live in many folders. The registry collapses copies by name, keeps every location, and sets drift: true when the copies have different content hashes. That one flag tells you which skills have quietly diverged across your projects. The changelog then records what got added or removed on every run, so you have a dated history for free.
06 · Pillar two
The Wiki: How It Connects
Linked markdown pages, Karpathy-style. This is the part that compounds. Written once, improved forever.
This is the part that compounds. The manifest is rebuilt from scratch every run. The wiki is written once and improved forever. It is a folder of linked markdown pages, plain text on purpose, so it survives any tool change and any model can read it.
Four kinds of page
entity
One real thing. A skill, an MCP, a framework, a decision.
concept
An idea or pattern that is not a single thing. The manifest loop.
connection
How two or more things relate. Why the wiki and manifest are separate.
qa
A good answer to a past question, filed so you never re-derive it.
A fifth folder, candidates/, is a holding pen. New pages the agent is unsure about land there first and stay out of search until you promote them, so low-confidence drafts never pollute your answers.
Pages link to each other
Wikilinks connect pages, with typed edges that say how things relate.
[[pdf-extractor]] a plain link
[[pdf-extractor|depends-on]] a typed link
[[pdf-extractor|depends-on|the PDF tool]] typed, custom text
Edge types: relates-to, depends-on, deploys-into, owned-by, supersedes, contradicts, part-of, used-by. A link to a page that does not exist yet is fine. The engine records it and lint reports it as a broken link. That list is your backlog of pages still to write.
The contract
Your agent reads one file before it touches the wiki: wiki/WIKI.md. It is the rulebook. It states the three layers, the page types, the frontmatter, the link vocabulary, and the ingest, query, and lint operations. Good query answers get filed back as qa pages, so the wiki compounds every time you use it.
07 · Pillar three
The Dashboard: Show Me
A small Next.js app that reads the files off disk. One pattern, repeated. No database needed.
A small Next.js app. Server components read the files off disk and render them. No database, no API layer, no client-side fetching by default. The pattern to learn is one function: a service reads a file, an async page awaits it and renders. Everything else is a repeat of that.
/
Glance
The counts, plus the latest entries from the changelog. The one-screen status of your stack.
/registry
Registry
The inventory tables. Every skill, its copies, its drift flag, and what it does.
/wiki
Wiki
The page list, and one page per slug, with wikilinks rewritten into real links before rendering.
$ npx create-next-app@latest dashboard --ts --app --src-dir $ npm install react-markdown remark-gfm $ cd dashboard && npm run dev # http://localhost:3010
Reads the files, never a copy
The dashboard reads the same inventory.json and wiki markdown the engines write, straight from disk. Reload the page and it is current. Four runtime packages, plain CSS, no build step to remember. Grow large later and you can add an optional SQLite read mode that falls back to JSON when the database is not there.
08 · The layout
The Shape Of The Project
Every file has a home. Generated files are read-only. Sources stay sources.
The whole project, top to bottom. Two short Python scripts, a folder of markdown, and a small app. Notice the pattern: every file is either a source you edit or something GENERATED and read-only.
my-ai-stack/ ├── CLAUDE.md rules for your agent ├── ai-stack.md GENERATED human manual ├── scripts/ │ ├── sync.py the manifest engine │ └── wiki.py the wiki engine ├── data/ │ └── inventory.json GENERATED machine mirror ├── history/ │ └── changelog.md GENERATED dated diffs ├── wiki/ │ ├── WIKI.md the contract your agent reads first │ ├── index.md GENERATED page catalogue │ ├── ROUTING.md GENERATED branch map │ ├── wiki.db GENERATED search + graph index │ ├── entities/ one page per real thing │ ├── concepts/ ideas and patterns │ ├── connections/ how X relates to Y │ ├── qa/ answers worth keeping │ └── candidates/ drafts, not yet trusted └── dashboard/ Next.js app
Generated files are never hand-edited. To change them, you edit a source: the curated dictionary in sync.py, or a wiki markdown page. Then you re-run the engine. The project's own CLAUDE.md tells your agent exactly that, which is what makes the stack self-maintaining.
09 · The loop
It Stays Current On Its Own
Sync, ingest, reindex, lint, reload. Five steps a week, and the map never rots.
The whole point is that it stays current with almost no effort. Weekly, or whenever something changes, you run five steps. Most of it is one command.
python scripts/sync.py
Rescan, rewrite the manifest, record the diff in the changelog.
ask your agent to ingest
It reads WIKI.md, updates the pages a change touches, files new ones. Unsure ones land in candidates.
python scripts/wiki.py reindex
Rebuild the search index and the navigation files.
python scripts/wiki.py lint
Broken links, orphans, sparse pages. The broken-link list is your writing backlog.
reload the dashboard
Already current. It reads the files live, so there is nothing to deploy.
Automate the scan, keep ingest human
You can put the sync on a daily schedule with cron, a scheduled task, or your agent's own scheduler. Keep the wiki ingest human-triggered, though. The scan is mechanical, but synthesis is the part that needs your judgement. When you want to know how something connects, point the agent at the wiki, not your whole filesystem. It reads the index, picks a branch, and answers from the pages.
10 · Build it
The Mega Prompt
Paste this into Claude Code and it interviews you, plans, and builds the whole stack with you.
If you just want it built, do not copy files by hand. Paste the prompt below into Claude Code and it builds the whole thing with you: it interviews you, writes a plan, scaffolds every file, runs the first sync, and tells you how to start the dashboard. Same interview-then-plan-then-build flow this guide was written with.
You need three things
Claude Code
Or any coding agent that can run shell commands and edit files.
Python 3.9+
For the two engine scripts. Standard library only. No pip install.
Node 20+
For the dashboard. Four runtime packages, nothing exotic.
Open in an empty folder
Start a fresh Claude Code session where you want the project to live.
Turn on plan mode
Type /plan so the agent interviews you and plans before it writes a single file.
Paste, answer, approve
Paste the prompt, answer the questions, review the plan it writes, then approve.
The Build Your Own AI Stack mega prompt
You are going to scaffold a small, self-maintaining "AI stack" project in the
current directory. It has three parts:
1. A MANIFEST engine (Python): scans my projects for skills, MCP servers,
CLAUDE.md files and plugins, merges those facts with a hand-curated
catalogue, and writes data/inventory.json + ai-stack.md + a dated
history/changelog.md.
2. A WIKI (markdown + a Python engine): a Karpathy-style LLM wiki that maps
how things connect and why. Follow Karpathy's pattern exactly:
https://gist.github.com/karpathy/442a6bf555914893e9891c11519de94f
The markdown pages are the source of truth. A SQLite + FTS index and a
graph are derived and rebuildable. Single tier, fully local. No vector
database, no cloud, no multi-tenant scoping.
3. A DASHBOARD (Next.js, App Router, TypeScript): server components read the
files off disk and render them. Three routes: Glance, Registry, Wiki.
Markdown/JSON by default; an optional SQLite read mode for advanced users.
You are in plan mode. Do NOT write any files yet. Work in this order.
STEP 1 - INTERVIEW ME.
Use the AskUserQuestion tool to gather what you need. Group the questions into
one or two calls (max four questions per call). Ask:
- Project name and the directory to create it in (default: the current dir).
- The root directory (or directories) to scan for the manifest
(default: ~/projects).
- Which categories to track (multi-select): skills (SKILL.md), MCP servers,
CLAUDE.md orchestrators, plugins, sub-agents, scheduled tasks.
Default the first four on.
- Dashboard data mode: Markdown/JSON files (default) or SQLite (advanced,
needs Node 22+).
- Engine language: Python (default) or Node.
- Styling: plain CSS (default, zero dependencies) or minimal Tailwind.
- Add a daily scheduled sync? Yes or no.
For each question give a clear recommended default as the first option. If I
pick "Other", honour what I type.
STEP 2 - PLAN.
Once you have my answers, write a short implementation plan to the plan file:
the file tree you will create, the engine commands, and how to run the
dashboard. State plainly what you are leaving out (vector search, multi-tier
wikis, drift auto-repair, cost tracking) and that those are documented as an
upgrade path. Then call ExitPlanMode to ask me to approve.
STEP 3 - BUILD (after I approve).
Create this structure (adjust to my answers):
CLAUDE.md rules: manifest-first; generated files read-only;
re-sync after changes; pointer to wiki/WIKI.md.
README.md what it is and how to run it.
scripts/sync.py the manifest engine. Four stages: a CURATED dict
(the editorial "why"); scanners that walk the scan
root for the chosen categories and record path,
name, content hash and mtime; a registry that
collapses duplicate skills by name and flags drift
(more than one distinct content hash); a renderer
that writes data/inventory.json (machine) and
ai-stack.md (one section per category), plus a
history/changelog.md built by diffing a saved
fingerprint snapshot against the previous run.
Standard library only. Scan root from the
AI_STACK_SCAN_ROOT env var.
scripts/wiki.py the wiki engine. Standard library only (sqlite3 +
argparse). Subcommands: init (create wiki.db +
folders), reindex (scan markdown, parse
[[slug|edge|text]] wikilinks into a links table,
rebuild an FTS5 index, regenerate index.md and
ROUTING.md), lint (broken links, orphan pages,
sparse pages), graph (emit a dependency-free
graph.html + graph.json), log, promote (move a
candidates/ page to active). dst_slug is stored
even when the target page does not exist yet, so
lint can report broken links.
wiki/WIKI.md the contract the agent reads before touching the
wiki: the three layers (immutable sources ->
markdown pages -> derived index), the four page
types (entity, concept, connection, qa), the
frontmatter spec, the wikilink + edge vocabulary,
and the Ingest / Query / Lint operations. State
that good query answers get filed back as qa/
pages so the wiki compounds.
wiki/schema.sql pages, links, pages_fts (FTS5).
wiki/{entities,concepts,connections,qa,candidates}/ page folders. Seed each
content folder with one short example page that
links to the others, so reindex and the dashboard
have something to show.
wiki/{index.md,ROUTING.md,log.md} navigation + ops log (index.md and
ROUTING.md are regenerated by reindex).
dashboard/ Next.js (scaffold with create-next-app:
--ts --app --src-dir --import-alias "@/*", no
eslint, no tailwind unless I chose Tailwind).
Install react-markdown + remark-gfm. Dev port 3010.
A service layer reads the surrounding repo
(data/inventory.json and wiki/*.md) via the
filesystem in server components. Routes: / (Glance:
the counts plus the latest changelog entries),
/registry (the inventory tables), /wiki (page list)
and /wiki/[slug] (one page, with [[wikilinks]]
rewritten to /wiki/<slug> links before rendering).
Guard the slug param with a regex. If I chose
SQLite mode, add a db.service that reads a SQLite
snapshot when present and falls back to the JSON
file when not.
If I chose Node for the engines, port sync.py and wiki.py to Node .mjs scripts
with the same behaviour and commands. If I chose a daily sync, set it up with
the most local mechanism available (cron, a scheduled task, or the agent's own
scheduler) and document it in the README; keep the wiki ingest human-triggered.
GUARDRAILS.
- Generated files (ai-stack.md, data/inventory.json, wiki/index.md,
wiki/ROUTING.md, wiki/wiki.db) are read-only. To change them, edit the
source (the CURATED dict or the wiki markdown) and re-run the engine. Say
this in CLAUDE.md.
- Single-tier wiki. No vector database. No cloud services. No secrets.
- Keep dependencies minimal: Python standard library for the engines; for the
dashboard only next, react, react-dom, react-markdown, remark-gfm (plus
tailwind only if I asked for it).
- Write the wiki/WIKI.md contract first, then the example pages, so the engine
has something to index.
STEP 4 - FINISH.
After building, run: python scripts/sync.py, then python scripts/wiki.py init,
then python scripts/wiki.py reindex. Fix any errors. Then print a short summary:
the counts from the first sync, the wiki page count, and the exact commands to
start the dashboard (cd dashboard && npm install && npm run dev). Tell me to set
AI_STACK_ROOT if the dashboard is not directly inside the stack folder.Want to hand-build it?
The mega prompt is the fast path. The full written guide has every file explained and the complete template code for both engines and the dashboard, in case you want to understand or edit each piece by hand. The prompt builds it. The guide teaches it.
11 · Grow into it
What This Leaves Out
This is the small version on purpose. Here is what the full system adds, and when to bolt each one on.
This is the small version on purpose. The four-step loop and the Karpathy wiki are the whole foundation. Everything below is an add-on you can bolt on later without changing the core. So start small, then grow into it.
Vector search
Semantic search over the wiki, not just keywords.
Add it when Hundreds of pages and keyword search starts to miss things.
Multiple wiki tiers
Isolated wikis per client or project, each rolling a summary up to a parent.
Add it when You map work for clients and must never let one client's data bleed into another.
Drift auto-repair
A canonical source per skill, so drifted copies refresh instead of just being flagged.
Add it when You deploy the same skill into many projects and want them identical.
Cost tracking
Token and dollar accounting per agent run.
Add it when You are spending real money on API calls and need to see where.
A richer dashboard
Charts, a 3D graph, a command palette, semantic recall.
Add it when The basic three pages stop being enough.
The full system we run does all of this: hundreds of skills across dozens of projects, a vector layer, multi-tier client wikis, and a much bigger dashboard. It all started as the three small pieces in this guide. The spine is the same.
A map that maintains itself
A manifest that lists what you have. A wiki that holds how it connects and why. A dashboard that shows both. You curate and direct, the agent does the upkeep, and the map gets more useful every week instead of rotting. This is one organ. The full operating system wires every organ together.
Part of the AI Operating System handbook.