What is RafeOS?

I've written about why I built RafeOS: the thinking, the two questions I'm chasing, and what it's taught me. This post is the other half, and a more practical one. People keep asking the reasonable follow-up: yes, but what actually is it? So here it is, laid out in parts.

The short version: RafeOS is three things — a platform holding the tools and services that read and act on my data, a knowledge layer – the data and context the platform draws on, and a set of surfaces – the different ways I reach and interact with the platform and the knowledge. The caveat: I also tend to use it interchangeable to refer to my wider AI/agentic and self-hosted set up too.

So perhaps it's most accurate to say RafeOS is an ecosystem, combining what I built what I borrowed to create an agentic-first layer to help run my life.

The shape

The architecture (below) reads as the three things above: how I reach it, the platform itself, the data and context it draws on, and the borrowed foundations underneath. Solid is what I built; dashed is borrowed.

How it acts

The behaviour (below) is a flow: something triggers a response, it's handled with as much or as little autonomy as the job needs, and it has effects.

Three things can set it going: me (asking, glancing, replying), an agent (delegating to a sub-agent, handing off, or cross-checking one AI against another), or a loop or event (a schedule, a webhook).

Whatever fires it, the request is handled with only as much autonomy as it needs — from a plain passthrough that just shows data, through deterministic automation and fixed workflows, up to a full agent handed a goal that works out its own steps.

Then it has effects: read, respond, use a tool, create something, notify me, or act on something in the world — with a firm gate on anything outbound (messages to other people, real-world actions, anything touching money or secrets). The same platform runs human-only, as a hybrid of me and an agent, or fully autonomously. The human account of what that's actually like to live with is in the agentic-edge post.

How I reach it

There are multiple surfaces I use to reach RafeOS. Each suits a different moment and use case. It is the obvious expression of a deliberately interface agnostic intent.

Here's a quick run through:

App

Cockpit (a progressive web app, installed on Mac, iPad and iPhone) for the at-a-glance view: the day's shape, health, messages owed a reply, transport, home; and for human-first actions. It surfaces a summary and links out for depth rather than rebuilding the things it sits on.

CLI

rafeos command line for text based interaction from quick summary to quick capture (rafeos tasks add "…"). This is a quick-query interaction and is most useful when I'm already working in terminal.

Agentic chat

Through an MCP connector, a normal chat window — Claude, ChatGPT, Grok, and experimental local models via Ollama can reach the same tools. This is the quick-query door, especially useful on the go, with a pleasant UX and AI smarts.

Agentic harnesses

Claude Code, Claude Cowork and OpenClaw: the places I actually work. This is where the building and the heavier, longer-running jobs happen, the agent calling the tools directly as it goes (including work on RafeOS itself). This is the surface I use most.

Messaging

Signal, Telegram and WhatsApp: two-way, and a good way to reach me on the go or for when I'm away from a bigger screen. A scheduled briefing arrives as a message; I can ask something back.

The platform

At the centre is the thing I actually built: the platform (the engine, if you like). One modular service, rafeos-api, with a matching MCP umbrella (rafeos-mcp) and the rafeos CLI, plus a set of skills — packaged know-how an agent can invoke. Alongside my own tools sit connectors to third-party ones (the Anthropic-hosted tools, Home Assistant, other MCPs): borrowed, but reached the same way.

Between them the modules expose well over a hundred small tools, each doing one readable thing. The commands follow a deliberately boring pattern, so once you've seen one you can guess the next:

rafeos tasks add "call HMRC"
rafeos messaging search "restaurant booking"
rafeos health summary
rafeos transport tube-status

One detail worth calling out: every change is attributed — the store records whether it was me or an agent that made it. That's a small thing that does a lot of work; it's what makes heavy automation trustworthy rather than spooky. The architecture post (One API, one MCP, one CLI) covers why it's built this way; here the point is just that there's one platform, and everything else leans on it.

The modules are the obvious slices of life (with more being added).

• Tasks — a to-do store several agents and I all write to, with most of the admin handled automatically (deep dive).
• Gmail and Calendar — access to email and schedule, plus some wider services
• Messaging — WhatsApp, Signal and Telegram bridged into one private, searchable archive I own (deep dive).
• Reading — a daily digest built to surface what's worth my attention and suppress the rest (deep dive).
• Health — Oura, Withings and Apple Health unified into one view, with an interpretation, not just numbers (deep dive).
• Home — Connections to Home Assistant, Octopus, and other smart-home tools.
• ...and Briefing – AI and rules powered briefing that is composed using the modules above (deep dive).

Data & context

A platform is only as good as what it knows, and RafeOS draws on two kinds of thing — the line between them being who owns it.

What it owns. The substrate is plain markdown — the system's written record: knowledge, project state and decisions, session logs, the context files the agents read. The agents write to it directly or through pipelines; when needed I edit it by hand in Typora or Obsidian; Git syncs it across machines (there's a whole post on the substrate). Then there are also small SQLite databases for tasks, reading, health and messaging for more structured data.

What it pulls live. Everything else is borrowed data, fetched on demand and grouped roughly: productivity (email, calendar, files, CRM); home and environment (Home Assistant, screen time, the electricity tariff, weather); health (Oura, Withings, Apple Health); and reading and the odd extras. There are some local caches for performance and costs reasons too.

The memory inside ChatGPT and Claude is relevant here too, but it is gated or siloed to those services.

Foundations

None of this would exist without a lot of borrowed foundations, and it's worth being honest about that. The AI models are Anthropic's, OpenAI's, xAI’s, and experimental local ones via Ollama. The agentic harnesses are other people's software.

Underneath sits a stack of self-hosted, open-source services — the Matrix server and bridges that carry the messages, the RSS reader, Home Assistant, the metrics tools — running on my own hardware across a few machines on a private network.

What ties it together

If I had to keep one thing, it would probably be the briefing. It owns no data of its own; it reads all the other modules and tells me what they add up to: a short morning read-out of what's on, what needs me, and what's merely worth knowing, with the genuinely urgent floated to the top. It's the clearest expression of the whole system's goal - not more dashboards to check, but the right things brought to me.

Where to go from here

That's the map: an ecosystem I'm building and evolving, the ways I reach it, the data and memory it draws on, and the foundations it stands on.

The why is in the pillar; the depth is in the module posts linked above; and the current state of it all — what works, what's still alpha, what it costs — is threaded through those. Read this one first if you want to know what the parts are before you hear the arguments about them.