Skip to content
Novaterra

The memory model

A Memory is the atomic unit of everything your Twin knows about you:

{
  id, beingId,
  kind: 'fact' | 'episode' | 'preference' | 'relationship' | 'document' | 'insight',
  content: string,       // one self-contained, third-person sentence
  source: string,        // 'muse' | 'email:<threadId>' | 'file:<root>' | 'browser' | 'studio:<projectId>' | 'omni' ...
  importance: number,    // 0..1
  tags: string[],
  createdAt: ISODate,
  lastAccessed: ISODate | null,
}

Memories are always written as one plain sentence in third person — “Owner prefers dark roast over espresso”, not a transcript fragment — so they read the same whether they came from a conversation, an email, or a file summary.

Storage: SQLite + FTS5 + float blobs

Memories live in the memories table; a companion memories_fts virtual table (FTS5) is kept in sync by a trigger so text search works with zero extra code at write time. Embeddings, when an OpenRouter key is present, are stored separately in memory_embeddings as raw Float32Array buffers (toBlob/fromBlob in apps/api/src/twin/memory.ts) — small enough datasets that cosine similarity is computed in-process in plain JavaScript rather than through a vector database.

GET    /api/twin/memories?q=&kind=&limit=50   → Memory[]   (q present → hybrid recall, else newest-first)
POST   /api/twin/memories                     → Memory     { kind, content, source, importance, tags }
DELETE /api/twin/memories/:id                              → { ok: true }

Deduplication

Before a new memory is inserted, findDuplicate() normalises whitespace/case and checks the being’s last 400 memories for an exact match. A duplicate isn’t rejected or silently dropped — its importance is nudged up (capped at 1) and lastAccessed is refreshed, so saying the same true thing about yourself twice reinforces it instead of cluttering the store.

Hybrid recall

recallMemories(beingId, query, opts) fuses two signals:

  1. FTS5 rank — searchMemoryIds() runs the query against memories_fts, ranked ids only.
  2. Cosine similarity — if an OpenRouter key is present, the query is embedded (embed tier) and compared against every one of the being’s stored embeddings with a plain dot-product cosine.

Both are normalised to 0–1, fused 50/50, then blended with two more terms — importance (18% weight) and a recency curve that halves roughly every 45 days (12% weight) — so a highly important memory from months ago can still outrank a throwaway one from this morning, but all else equal, recent and reinforced memories win. The top results also get lastAccessed touched, so recall itself is part of what keeps a memory “alive”. Without an OpenRouter key, recall runs on FTS5 alone — degraded, never broken.

Embeddings are batched, not per-write

New memories don’t embed synchronously. queueEmbedding() adds the memory to an in-process map and schedules a flush 600ms later (flushEmbeddings()), so a burst of memories from one learning-loop pass turns into one /embeddings call instead of one per memory — directly serving the sprint’s budget rule (PLAN.md §7: “batch [embeddings]”). At most 96 pending memories are flushed per call; anything left over reschedules itself 200ms later.

Extraction: how free text becomes memories

extractMemories(text, source, beingId) is the cheap-tier structured call every ingestion path shares (learning from a Muse conversation, a file summary, an email thread, a Chrome history cluster). It returns:

{
  memories: Array<{ kind, content, importance, tags }>,  // max 12
  profile: { facts, goals, preferences, routines, tone? },
}

Models like to invent kinds (“goal”, “habit”) the contract doesn’t define — coerceKind() maps known aliases onto the real MemoryKind enum (goal/goals → fact, habit/routine → preference, person/family → relationship, event → episode, note/file → document, interest/idea/opinion → insight/preference) and falls back to fact for anything else, so a slightly-off model response never breaks the pipeline. Without an OpenRouter key, or on a model failure, extractSentences() degrades to plain sentence-splitting: memorable-looking sentences (12– 400 characters) become low-importance (0.4) episode memories instead of nothing at all.

Article III, in code

The constitution promises “read all of it, correct any of it, delete any of it” (see What Novaterra is). Concretely: GET /api/twin/memories returns everything with no pagination trick hiding older ones (up to limit, capped at 200 per call, newest first or by relevance); there’s no separate “edit” route because a memory is a sentence, not a form — the practical edit path is delete-and-re-teach through POST /api/twin/learn; and DELETE /api/twin/memories/:id removes both the memory row and its embedding row in the same call, so nothing lingers in the vector store after a memory is forgotten.

A concrete example

Terminal window
curl -s -X POST http://localhost:4000/api/twin/memories \
  -H 'content-type: application/json' -b cookies.txt \
  -d '{"kind":"preference","content":"Owner prefers dark roast over espresso.","source":"muse","importance":0.5,"tags":["coffee"]}'
{
  "id": "mem_9f2a...",
  "beingId": "b_owner...",
  "kind": "preference",
  "content": "Owner prefers dark roast over espresso.",
  "source": "muse",
  "importance": 0.5,
  "tags": ["coffee"],
  "createdAt": "2026-09-06T08:12:03.000Z",
  "lastAccessed": null
}
Terminal window
curl -s "http://localhost:4000/api/twin/memories?q=coffee&limit=5" -b cookies.txt

returns that same memory (and anything else coffee-related) ranked by the fused score above, not just a substring match.