The second brain

In common with a lot of other personal AI setups, I keep knowledge as plain markdown in a set of local folders. This is also synced to a repo to make it easier to use on multiple machines. This post is about how I'm trying to keep it a curated knowledge base rather than a collection of files, why I didn't reach for a vector database, and where it's drifting on me.

A note on scope. This is the knowledge layer, the second brain, the substrate. How the runtime itself is wired, why runtime, substrate and tools are kept as separate swappable layers are all points I've discussed in the harness post. It is effectively part of RafeOS, but it's really part of my broader setup for the use of AI and agentic harnesses.

Three tiers

The knowledge substrate has three tiers, each with a clear job.

• Durable, curated knowledge: projects, decisions, reference material, the things future sessions should treat as settled. Interlinked, deliberately maintained, small enough to be read in full.
• Transient working files: session handovers, research notes, drafts, the running log of what a session did. Useful in the moment, mostly disposable after.
• Operational state: the live data (tasks, calendar, health figures, messages). This doesn't live in markdown at all. It sits behind the API, in its proper store, queried on demand.

That third tier is really a story of unification, integration, and query coherence. The knowledge story is the first two tiers, and specifically the line between them.

What’s actually in there

The durable tier isn’t one big document; it has a deliberate shape:

• About me — who I am, how I work, my preferences and standing context, so a session starts already knowing me rather than asking.
• Domains, with projects inside them — areas of life and work (home, health, finances, career, family), each holding its settled decisions, reference and notes. Projects live inside a domain rather than in one flat list, which is exactly why the maps matter.
• Maps — a few one-page indexes sitting over the folders: what exists, which domain it’s under, how far along it is. They’re how the agent (and I) get oriented before diving into the detail.

The transient tier is mostly organised by time and task rather than topic: dated session handovers in a fixed shape (what was done, files touched, what’s next), research notes and drafts, and an outputs area where generated files land, filed by domain. They earn their keep for a day or a week, even a month, and then mostly get deleted or archived.

Claude Cowork projects do overlap a little here. You can achieve some of the above using only a Cowork project folder (it's designed to bring in files after all). So it's accurate to say Cowork integrates and to some extent intermixes with the substrate.

Similarly, technical execution projects are ordinary Claude Code or Codex project folders and Git repos, with the CLAUDE.md, or equivalent, having a pointer to the shared substrate. The difference, compared to Cowork, is the transient tier tends to get folded into the claude Code project folder (it's implementation-first usage for me means this is more structured than it sounds). In practise, this means that Claude Code and Codex reference rather than integrate with the substrate.

The graduation rule

The key convention is what I call graduation, and it's one sentence: if a file holds decisions that future sessions should treat as authoritative, it graduates to the durable tier; if it's context for a single moment, it stays in the working tier.

A handover note from one session to the next is moment-context. It stays working, and it can be deleted without anyone mourning it. A research dump while I'm weighing two options is working. A draft is working. But the decision that comes out of that research ("we went with X, for these reasons, and here's what that rules out") graduates. It moves into durable knowledge, gets linked to the things it touches, and becomes something the agent can rely on next time instead of re-deriving.

Why I pre-compile instead of retrieving

The obvious way to give an agent access to a big pile of personal knowledge is retrieval: embed everything into a vector store, and at query time pull back the chunks that look relevant. RAG. It works, and for some problems it's the right tool.

For a working personal knowledge base I think other routes are worth exploring, and Andrej Karpathy put the reason better than I would in his "LLM Wiki" piece. Retrieval rediscovers your knowledge on every single query. Each question starts from a pile of raw documents and reconstructs an answer from scratch, at the mercy of whatever the similarity search happened to surface.

A curated wiki (set of documents) does the opposite. The thinking is done once, when knowledge graduates and gets written down properly, and from then on it's there to be read. The work of deciding what matters and how it connects has already happened, by hand or by a deliberate pipeline, before the agent ever opens the file. The agent reads it directly, the way you'd read a good internal wiki, rather than reassembling it from fragments every time.

I notice it most in the small things. A lot of the second brain is just well-documented context I’ve written down once, and a good deal of it is markdown pointers — a short note that points at the thing rather than restating it.

That's also where the everyday gap between tools shows up. Claude Code and Claude Cowork are noticeably stronger for me than plain Claude chat or ChatGPT, and the reason isn't the model. It's that those two are fully into this substrate. A harness wired into your context beats the out-of-the-box version every time. The more you write down properly, the less you have to say, and the better the answers get.

The trade-off is honest: pre-compiling costs effort up front, and it doesn't scale to millions of documents. But a personal knowledge base isn't millions of documents. It's a few hundred files that benefit enormously from being curated.

One set of files, two ways in

The substrate is plain markdown, which means I get a good human interface for free. I can opt to navigate and edit it in Obsidian: the graph view, backlinks, the local-first canon that Steph Ango (Obsidian's CEO) sums up as "File Over App", the idea that your knowledge should outlive any particular tool that displays it. Geoffrey Litt and the Ink & Switch crowd have been making the deeper version of this argument for years under "malleable software" and local-first: the file, not the app, is the durable thing.

Obsidian is a view, not the source of truth. I also sometimes use Typora for editing files. The agents interact with the substrate as plain files, reading, writing and moving them through pipelines. In other words the human gets a lovely interface; the files stay boringly readable by anything.

Memory and the wiki are not the same store

One distinction to note. There's the durable shared wiki I've been describing, and separately, each runtime has its own working memory: session-derived, runtime-specific, the stuff it picks up in the course of doing a job. This comes out of the box with Claude and ChatGPT and has grown increasingly sophisticated.

There's obvious crossover. Claude encourages the creation of CLAUDE.md and other runtime and project-specific files. I do use these and will also point them at the substrate.

The bit that isn't solved

A single shared substrate is lovely until more than one agent is writing to it at once, and RafeOS has more than one runtime. How separate agents share this memory without colliding is a real problem, but it's a runtime problem, not a knowledge-layer one, so I've made that argument where it belongs in the harness post rather than re-running it here. For the knowledge layer, the point stands: curate it, graduate into it, get the maintenance honest, and keep it in a format that will still open in ten years.

This post was drafted with Claude and edited by me — which is also, more or less, how the substrate it describes gets written.