Architecture

How brain-mcp transforms raw conversations into queryable cognitive context.

Data Flow

brain-mcp follows a layered pipeline. Raw conversations flow through processing stages into multiple queryable stores, each optimized for different query types.

Your Conversations
(ChatGPT, Claude Desktop, Claude Code, Custom)
              │
         sync pipelines
         (Python scripts)
              │
              ▼
   all_conversations.parquet
   (unified format, immutable source)
              │
     ┌────────┼────────┐
     ▼        ▼        ▼
  LanceDB   DuckDB   Structured
  vectors   queries   summaries (v6)
  (768-dim)  (SQL)    (JSON)
     │        │        │
     └────────┼────────┘
              │
         MCP Server
         (25 tools)
              │
     ┌────────┼────────┐
     ▼        ▼        ▼
  Claude    Cursor   Windsurf
  Desktop            (any MCP client)

The Three Layers

Source Layer — Parquet

All conversations are stored as a single Parquet file. This is the immutable source of truth — never modified after write. Format: message_id, conversation_id, role, content, created, source, plus metadata. Queried via DuckDB for fast SQL.

Embedding Layer — LanceDB

User messages are embedded locally using nomic-embed-text-v1.5 (768-dimensional vectors). Stored in LanceDB — a local, serverless vector database. Enables semantic search: find conversations by concept, not just keyword. Only user messages are embedded; short messages (<10 chars) and noise patterns are filtered.

Summary Layer — v6 Structured Summaries

Each conversation is summarized into structured JSON: summary, decisions, open questions, breakthroughs, domain classification, thinking stage, and quotable phrases. These power the prosthetic tools — domain tracking, open thread detection, and thinking synthesis. Also embedded in LanceDB for semantic summary search.

Design Principles

🏠

100% Local

Embeddings run on your hardware (Apple Silicon MPS, CUDA, or CPU). LanceDB is a file — no database server. No cloud dependencies for core operations.

🔒

Immutable Source

Raw conversation data in Parquet is never modified. All derived layers (embeddings, summaries) can be deleted and regenerated from source.

🔌

Standard Protocol

MCP (Model Context Protocol) means any compatible client works out of the box. No custom integrations, no API wrappers, no SDK to learn.

⚡

Millisecond Queries

Most tools respond in under 50ms. The MCP server keeps models and database connections warm in memory for instant responses.

Technology Stack

Component	Technology	Purpose
MCP Server	`FastMCP (Python)`	Tool registration and stdio transport
SQL Engine	`DuckDB`	Fast analytical queries on Parquet files
Vector Store	`LanceDB`	Local vector search, no server needed
Embedding Model	`nomic-embed-text-v1.5`	768-dim vectors, runs locally
Data Format	`Apache Parquet`	Columnar storage, compact and fast
Sync Pipeline	`Python + cron`	Hourly/nightly data synchronization

Next: Data Sources

Learn about the domain system that organizes your thinking, or explore the prosthetic model.

PreviousInstallation NextData Sources