An agentic harness on swappable layers

If you work with AI for any length of time, you end up with one or more versions of an agentic harness (or a personal AI system): typically combining a runtime (powered by a frontier model), a set of tools (connectors and integrations), and a shared substrate (personal knowledge management).

There's a lot of thinking out there on this and it's a fast moving space. I've borrowed from a lot of different patterns. RafeOS isn't a rival to any one; it sits across several of them at once. So this is "here's mine", not "here's the answer". What follows is the part I think is transferable: how it's put together so the pieces can be pulled out and replaced.

The bet, and the hedge against it

Right now, the bet for me is that a combination of Claude Cowork and Claude Code is the right harness setup for me. It's where I do the heavy, think-alongside-me work, and it's good enough that I reach for them daily.

The hedge is that I've tried hard not to build around it. Years of composable architecture teaches you one reflex above all others: don't couple the thing that changes fast to the thing that should outlast it. Agent runtimes are changing monthly. My data shouldn't have to.

So RafeOS is built with consideration for three independent layers:

• The runtime is the agent doing the reasoning. Anthropic-centric today and, in theory, replaceable.
• The substrate is where memory and notes live: plain markdown in a Git repo. A human can read it, and so can any agent that can read a file. There's nothing Claude-specific about a .md file.
• The tool platform is how the agent reaches my data: a plain HTTP API with predictable verbs (/v1/<module>/<verb>). Accessible directly, through the CLI, and via an MCP.

It's not quite this simple because Anthropic, OpenAI etc. have their own memory too (various guises, depending on harness), but the way they operate does encourage you to have your own to provide wider and longer-term context.

Has the bet actually held up?

A few months ago I'd have called this "untested insurance". It's now partially proven. RafeOS runs across more than one runtime: I use both Anthropic and OpenAI harnesses regularly, on multiple devices, and the separation has held (and both companies have been iterating very rapidly). I'm also experimenting with things like OpenClaw (see below) and Hermes and can get going faster on these thanks to shared substrate and tools.

The substrate and tools have also been through multiple iterations, and through that churn things have mostly carried on working. That's the bet doing its job: each layer has changed and they didn't drag the others with it.

What the substrate carries best is context across the two places I work. Planning and knowledge work happen in Claude Cowork (and sometimes in ChatGPT), building and execution in Claude Code or Codex. Everything sits on the same markdown substrate, so a decision reasoned through in one is already known to the other. I don't re-explain a project when I move from one to the other, and that continuity is quietly one of the most valuable properties of the whole system.

Running more than one runtime did surface a problem I haven't cracked: how do separate runtimes share memory without colliding? Two agents writing to the same markdown substrate is mostly fine until they both want to write at once, or one acts on a note the other just changed. Git gives me the audit trail and a way to merge after the fact, but it isn't a real-time coordination layer, and I haven't built one.

For now I manage it by discipline rather than design, careful to use each runtime for a different kind of work so collisions stay rare. But "managed by being careful" is a polite way of saying unsolved. It's the honest frontier of this setup, and the bit I'd most like to see someone else's good answer to.

On a related note, what's also somewhat unsettled is syncing the substrate across devices cleanly. Git helps (it's the closest thing I have to a shared source of truth several machines can pull from) but it isn't a finished answer and, in practice, I've ended up using harnesses less on remote servers than I originally thought I would (I do use them for always-on headless one-shot queries).

So: held under real conditions, not yet bulletproof.

Aside: OpenClaw experiments

There's more than one runtime in play. Besides the multiple flavours of Claude and ChatGPT, I've been trialling OpenClaw running on a remote server. I set it up mostly to understand how OpenClaw works, and right now the verdict is that it may be less useful for me than for someone starting fresh. The reason is faintly funny. OpenClaw's whole pitch is wiring a capable agent into your tools and your life, and I'd already done a lot of that. So much intelligence already lives in my RafeOS tools and APIs that the agent layer has less to add. Right now it's mainly teaching me about loops (how an always-on agent paces and re-checks itself) rather than something I lean on.

The next real test of it

The real test of "keep the runtime swappable" is still ahead of me, and it's a local LLM. I built RafeOS to allow for it from the start (a model on my own hardware is just another runtime behind the same tools and substrate) but so far I've done little more than experiment. The swap is designed for, not exercised. Running the whole thing against a local model in anger is something I haven't done yet.

I think that changes in the back half of 2026. There's roughly a nine-to-twelve-month lag between frontier capability and what runs locally, and the token economics keep pulling the same way. Enterprise is already shifting the distribution, and I'd expect personal use to follow. When it does, the layer separation stops being a tidy idea and becomes the thing that lets me move a real workload onto a local model without rebuilding around it.

Aside: where it should learn its own shortcuts

There's a deeper version of self-improvement RafeOS doesn't do yet: skill distillation. When I walk the agent through the same multi-step workflow for the fourth or fifth time, nothing notices the repetition and promotes it into a reusable, named capability. The loop is entirely manual. Mostly it's me spotting that we're doing the same dance, and prompting the co-creation of a skill or an equivalent.

Nous Research's Hermes is the system I'd point to as an interesting pattern: an agent that watches its own behaviour, distils recurring patterns into skills, and improves its memory without a human in the loop. That's the direction I expect a mature personal harness to head, the substrate learning its own shortcuts.

I also want to spend time looking at dreaming, more on that another time.

Why the seams are the point

Strip it back and the claim is small. Almost none of these pieces is novel on its own. Markdown stores, headless agent calls, a tool API, an agent with its own inbox all exist elsewhere, often done better. What I'd stand behind is the coherence: they sit on shared, swappable layers, so the whole thing bends with the field instead of being rebuilt every time the field moves.

Once again this is the composable-architecture lesson, applied to a personal system. You can't predict which runtime wins, or how pricing might have an impact, or which of today's tools or projects survives two years. So you don't bet on any of them. You keep the layers apart, make the seams boring, and let the parts you can't predict be the parts you can replace.

This was drafted with Claude and edited by me, fitting for a post about a system built the same way.