The floor

The format is the contract.

Everything else in brain-mcp — the database, the search index, the recall tools — is a convenience built on top of a directory of append-only files. If all of it burned, the directory would rebuild it. That is the only promise that matters, so it is the one written into the layout.

Where your agents keep transcripts on disk

These paths are not well documented anywhere, so here they are. Each one is a directory you can open right now, and each is governed by a retention policy you did not choose.

Agent	Transcript location	Retention
Claude Code	~/.claude/projects/<slug>/<session>.jsonl	`cleanupPeriodDays`, default 30, min 1. Deleted at startup.
Codex	~/.codex/sessions/<YYYY>/<MM>/<DD>/rollout-*.jsonl	No documented policy found. Observed retaining ~11 months on one machine.
Pi	~/.pi/agent/sessions/<slug>/<ts>_<uuid>.jsonl	No documented policy found.

Claude Code's retention is quoted from the official settings documentation. The other two rows say “no documented policy found” because that is what a search turned up — absence of a documented policy is not a promise of retention, and should not be read as one.

MEASURED2026-08-20 · Asia/Jerusalem · one machine

INSTRUMENTls -d ~/.claude/projects ~/.codex/sessions ~/.pi/agent/sessions && find each -type f -name '*.jsonl' | wc -l

~/.claude/projects 1,375 files oldest 2026-07-10 ~/.codex/sessions 360 files oldest 2025-09-17 (release-day observation) ~/.pi/agent/sessions present, .jsonl session files

UNMEASUREDWhether these paths are identical on Linux or Windows, whether they change between versions, and what any vendor intends to do with them in future. Verify on your own machine before relying on a path.

One home, one floor

The recorder writes into a single directory, overridable with BRAIN_HOME, created mode 0700.

~/.brain/
  spool/<lane>/              hooks + watchers write here first
                             (atomic tmp+rename)
  lake/<lane>/               THE FLOOR OF RECORD
                             <session>.jsonl  — append-only
                             <session>.g2.jsonl — rewrite = new generation,
                                                  the old one is kept
  manifest/manifest.jsonl    append-only, versioned lines {"v":1,...}
  offsets/<lane>/<session>   line count + prefix fingerprint
                             (this is how a rewrite gets detected)
  health/                    side-effect heartbeats — mtimes are the signal
  brain.duckdb               CACHE. Fully re-derivable from lake/ + manifest/

Spool, then lake

Hooks and watchers never write to the floor directly. They drop a file into the spool with an atomic tmp-and-rename, and the applier moves it down. A half-written chunk is never a half-written floor.

Rewrites open a generation

If an origin file is truncated or rewritten, the offsets fingerprint stops matching. The recorder does not overwrite — it opens <session>.g2.jsonl and keeps the old generation. Nothing on the floor is ever edited.

Every chunk gets a manifest line

One append-only line per chunk applied to the lake. The line carries a byte range, a line range, and the sha256 of exactly those bytes — which is what makes a citation checkable rather than decorative.

{"v":1,
 "lane":"cc","agent":"claude-code",
 "session":"<session-uuid>",
 "gen":1,
 "byte_from":184320,"byte_to":201154,
 "line_from":412,"line_to":437,
 "chunk_sha256":"9f2c…",
 "captured_at":"2026-08-20T09:14:22+00:00",
 "event_ts_first":"2026-08-20T09:13:58Z",
 "machine_id":"…","origin_path":"~/.claude/projects/…/….jsonl",
 "source":"hook"}

Field names are the real ones, written by the recorder. Values are illustrative.

The write order is the durability model

1Lake first. The bytes land on the floor before anything claims they exist.
2Manifest second. The claim about the bytes, with their hash.
3DuckDB last, and it is only a cache. A crash between any two steps re-applies idempotently: the lake append is guarded by a length check, the database inserts are primary-key idempotent.

The floor is schema-free. Views know the dialects.

Claude Code, Codex and Pi do not agree on what a message looks like, and there is no reason to believe next year's agent will agree with any of them. So the floor takes no position: it stores the vendor's bytes, unmodified, exactly as they were written.

Understanding is pushed up into dialect views — a view per agent that knows that vocabulary and projects it into a common shape. When a vendor changes their format, you write a new view. You do not migrate the floor, because the floor never interpreted anything in the first place.

A parser you can rewrite is a bug. A byte you discarded is permanent.

This was v1's sin

The first version of brain-mcp parsed each transcript on the way in, extracted the fields it thought were interesting, wrote those into a database, and did not keep the file. Every later question the parser had not anticipated was unanswerable, and every parser bug was retroactive and unfixable — the evidence needed to fix it had been thrown away at ingest. The recorder exists because parse-and-discard cannot be repaired after the fact. Keep the bytes; decide what they mean later, as many times as you like.

Backup means read it back

A copy operation that returned zero is not a backup — it is a command that did not error. The backup verb copies the floor and then re-hashes sampled files at the destination against the sha256 already recorded in the manifest. It samples rather than re-hashing everything, so a clean result is evidence, not proof.

The verify can fail. That is the point of running it — it does not report success and leave you to find out later.

The command is brain-mcp backup <dest>. The sync is pure-add and content-addressed, so it never deletes or rewrites anything at the destination.

Verifying a citation by hand

You do not have to trust the tool for this. A manifest line names a file, a byte range and a hash, so two standard commands settle it:

# the lines a citation points at
sed -n '412,437p' ~/.brain/lake/cc/<session>.jsonl

# the hash of exactly those bytes, vs chunk_sha256
sed -n '412,437p' ~/.brain/lake/cc/<session>.jsonl | shasum -a 256

If those two disagree with the manifest, the citation is wrong and you should stop believing the tool. That is the intended failure mode: checkable, and cheap to check.

How health is measured →The recall contract →

The schema, verbatim

Extracted from the installed package (2.0.0b3) by the stamp script — not hand-copied, so it cannot drift from the code.

CREATE SCHEMA IF NOT EXISTS floor;

CREATE TABLE IF NOT EXISTS floor.lanes (
  lane       TEXT PRIMARY KEY,     -- 'cc_transcript' | 'codex_rollout' | 'pi_session' ...
  agent      TEXT NOT NULL,        -- 'cc' | 'codex' | 'pi'
  mode       TEXT NOT NULL CHECK (mode IN ('hook','watch')),
  root_glob  TEXT NOT NULL,
  enabled    BOOLEAN NOT NULL DEFAULT true
);

-- one row per ORIGIN file ever seen (identity carries machine_id: cross-check M4)
CREATE TABLE IF NOT EXISTS floor.files (
  file_id      TEXT PRIMARY KEY,   -- sha256(machine_id:agent:realpath)[:32]
  machine_id   TEXT NOT NULL,
  agent        TEXT NOT NULL,
  lane         TEXT NOT NULL,
  abs_path     TEXT NOT NULL,
  session_hint TEXT,               -- uuid regex'd from filename
  first_seen   TIMESTAMPTZ NOT NULL DEFAULT now()
);

-- append-only observation log (gravity's witness idea, per visit)
CREATE TABLE IF NOT EXISTS floor.witnesses (
  file_id      TEXT NOT NULL,
  observed_at  TIMESTAMPTZ NOT NULL DEFAULT now(),
  size_bytes   BIGINT NOT NULL,
  mtime        TIMESTAMPTZ,
  growth       TEXT NOT NULL,      -- 'first' | 'append' | 'rewrite' | 'truncated' | 'unchanged'
  prefix_ok    BOOLEAN,
  new_lines    INTEGER NOT NULL DEFAULT 0
);

-- per-line rows. Duplicates preserved; dedup is derived-layer business (C3 resolution).
CREATE TABLE IF NOT EXISTS floor.raw_lines (
  file_id       TEXT NOT NULL,
  witness_gen   SMALLINT NOT NULL DEFAULT 1,   -- bumped on rewrite; OLD GENERATIONS KEPT
  byte_offset   BIGINT NOT NULL,
  line_no       INTEGER NOT NULL,              -- 1-based within (file, gen) — the citation line number
  raw_line      TEXT NOT NULL,                 -- utf-8; lake object is byte-exact if this was replaced
  line_sha256   TEXT NOT NULL,                 -- sha256 of ORIGINAL line bytes (no trailing newline)
  n_bytes       INTEGER NOT NULL,              -- original byte length incl. newline
  utf8_replaced BOOLEAN NOT NULL DEFAULT false,
  lane          TEXT NOT NULL,
  agent         TEXT NOT NULL,
  event_time    TIMESTAMPTZ,                   -- the line's OWN clock; nullable (not every line has one)
  captured_at   TIMESTAMPTZ NOT NULL,          -- NEVER null (the 89/89 lesson)
  PRIMARY KEY (file_id, witness_gen, byte_offset)
);
CREATE INDEX IF NOT EXISTS rl_evt  ON floor.raw_lines(event_time);
CREATE INDEX IF NOT EXISTS rl_lane ON floor.raw_lines(lane);

-- the sync-state v1 defined and never used, made real
CREATE TABLE IF NOT EXISTS floor.ingest_state (
  file_id       TEXT PRIMARY KEY,
  witness_gen   SMALLINT NOT NULL DEFAULT 1,
  last_size     BIGINT NOT NULL,
  last_offset   BIGINT NOT NULL,     -- byte offset AFTER the last complete consumed line
  last_line_no  INTEGER NOT NULL,
  prefix_sha256 TEXT NOT NULL,       -- sha256 of bytes [0, last_offset)
  updated_at    TIMESTAMPTZ NOT NULL DEFAULT now()
);

The dialect views, verbatim

The floor stays schema-free; each view alone knows one agent's vocabulary. Same extraction, same guarantee.

CREATE SCHEMA IF NOT EXISTS dialects;
CREATE SCHEMA IF NOT EXISTS derived;

-- latest-generation floor rows only
CREATE OR REPLACE VIEW dialects.v_live AS
SELECT r.* FROM floor.raw_lines r
JOIN floor.ingest_state s ON s.file_id = r.file_id AND s.witness_gen = r.witness_gen;

CREATE OR REPLACE VIEW dialects.v_cc AS
SELECT
  r.file_id, r.witness_gen, r.line_no, f.abs_path, f.session_hint AS session_id,
  r.event_time, r.captured_at,
  json_extract_string(r.raw_line, '$.type')       AS ev_type,
  json_extract_string(r.raw_line, '$.message.role')  AS role,
  json_extract_string(r.raw_line, '$.message.model') AS model,
  json_extract(r.raw_line, '$.message.content')   AS content,
  TRY_CAST(json_extract_string(r.raw_line, '$.isSidechain') AS BOOLEAN) AS is_sidechain,
  json_extract_string(r.raw_line, '$.cwd')        AS cwd
FROM dialects.v_live r JOIN floor.files f USING (file_id)
WHERE r.agent = 'cc' AND json_valid(r.raw_line);

CREATE OR REPLACE VIEW dialects.v_codex AS
SELECT
  r.file_id, r.witness_gen, r.line_no, f.abs_path, f.session_hint AS session_id,
  coalesce(r.event_time, TRY_CAST(json_extract_string(r.raw_line,'$.timestamp') AS TIMESTAMPTZ)) AS event_time,
  r.captured_at,
  json_extract_string(r.raw_line, '$.type')            AS envelope_type,
  json_extract_string(r.raw_line, '$.payload.type')    AS body_type,
  json_extract_string(r.raw_line, '$.payload.role')    AS role,
  json_extract(r.raw_line, '$.payload.content')        AS content
FROM dialects.v_live r JOIN floor.files f USING (file_id)
WHERE r.agent = 'codex' AND json_valid(r.raw_line);

CREATE OR REPLACE VIEW dialects.v_pi AS
SELECT
  q.file_id, q.witness_gen, q.line_no, q.abs_path, q.session_hint AS session_id,
  q.event_time, q.captured_at, q.ev_type,
  last_value(CASE WHEN q.ev_type='model_change'
             THEN json_extract_string(q.raw_line,'$.modelId') END IGNORE NULLS)
    OVER (PARTITION BY q.file_id, q.witness_gen ORDER BY q.line_no
          ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW) AS model,
  json_extract_string(q.raw_line, '$.message.role')    AS role,
  json_extract(q.raw_line, '$.message.content')        AS content
FROM (SELECT r.*, f.abs_path, f.session_hint,
             json_extract_string(r.raw_line,'$.type') AS ev_type
      FROM dialects.v_live r JOIN floor.files f USING (file_id)
      WHERE r.agent = 'pi' AND json_valid(r.raw_line)) q;

-- shared text extraction over the common block-array shape
CREATE OR REPLACE MACRO derived.text_of(content) AS
  CASE WHEN content IS NULL THEN NULL
       WHEN json_type(content) = 'VARCHAR' THEN json_extract_string(content, '$')
       WHEN json_type(content) = 'ARRAY' THEN
         array_to_string(list_transform(
           list_filter(json_extract(content,'$[*]'),
             x -> json_extract_string(x,'$.type') IN ('text','input_text','output_text')),
           x -> json_extract_string(x,'$.text')), chr(10))
       ELSE NULL END;

CREATE OR REPLACE VIEW derived.v_messages_all AS
SELECT 'cc:'||session_id||':'||witness_gen||':'||line_no AS msg_id,
       'claude-code' AS agent, session_id, role, model,
       derived.text_of(content) AS text, event_time, captured_at,
       file_id, witness_gen, line_no
FROM dialects.v_cc
WHERE ev_type IN ('user','assistant') AND NOT coalesce(is_sidechain, false)
  AND role IN ('user','assistant')
UNION ALL BY NAME
SELECT 'codex:'||session_id||':'||witness_gen||':'||line_no,
       'codex', session_id, role, NULL AS model,
       derived.text_of(content), event_time, captured_at, file_id, witness_gen, line_no
FROM dialects.v_codex
WHERE envelope_type='response_item' AND body_type='message' AND role IN ('user','assistant')
UNION ALL BY NAME
SELECT 'pi:'||session_id||':'||witness_gen||':'||line_no,
       'pi', session_id, role, model,
       derived.text_of(content), event_time, captured_at, file_id, witness_gen, line_no
FROM dialects.v_pi
WHERE ev_type='message' AND role IN ('user','assistant');

CREATE TABLE IF NOT EXISTS derived.messages AS
  SELECT * FROM derived.v_messages_all WITH NO DATA;