studbook studbook/docs/spec/README.md
Status: draft spec. No implementation. `spec` rung. Nothing in §7 or §10 has been ruled. Do not build from this yet.

Studbook

Status: draft spec. No implementation. `spec` rung. Nothing in §7 or §10 has been ruled. Do not build from this yet.

The data layer for the [&] stack, built on the substrate the stack already owns instead of on a hosted Postgres it does not.

A studbook is a pedigree register: every animal in it is recorded with its ancestry, and a record whose parentage cannot be shown is not a record. That is the whole design goal here. A record in studbook carries where it came from, and a record that cannot show its provenance is refused rather than stored.

1. What is decided, and by whom

Decided by Travis, 2026-07-30:

  • The shared-Supabase route is abandoned. ampersand-supabase is archived, not

failed — the 35 migrations across 11 schemas still apply and the code running against them still runs. The route changed, not the verdict on the code.

  • The replacement is called studbook.

  • It lives in core, not substrate and not compose. Core is the layer that rests on

substrate and that compose consumes, and a database is exactly that shape.

Everything from §5 onward is derived from reading the substrate code or from published prior art, and is marked as such. It is a proposal. §10 lists what has to be ruled before any of it is built.

2. Why the Supabase route was abandoned

Not because it did not work. It worked locally. Three reasons it was the wrong shape for this stack:

  1. It could not carry a certificate. Core's job is that the dark factory closes as

one pass with a certificate at every step. Postgres stores a row. It does not store why the row is that row, and nothing in the migration layer could refuse a row whose provenance did not check out. The stack's central claim — that a divergence is refusable rather than silent — stopped at the database boundary.

  1. It was a dependency the stack does not control. live_deployed against a hosted

instance is partly a claim about somebody else's uptime.

  1. The substrate grew the capability underneath it. See §5.

3. What studbook is

A record store where the identity of a record is the hash of its content, and where a record's ancestry is part of what is hashed.

Three properties, in the order they matter:

  • Content-addressed. The key is the hash of the canonical bytes. Writing the same

content twice is one record, not two.

  • Verified on read. Every read re-derives the id from the bytes and refuses to return

a record that no longer matches its key. Bit-rot and tampering surface as a named refusal, never as plausible-looking bad data.

  • Pedigreed. A record names its parents by their ids. Because ids are content hashes,

a lineage cannot be edited after the fact without changing every id downstream of the edit.

4. Where this sits, and the three questions

substrate → core → compose.

Studbook may depend on substrate. It may not depend on any product in compose. Products in compose may consume it. If studbook reimplements something substrate already proves, that is a finding, not a shortcut.

The component test applies to core too:

  1. What does it consume? From substrate: content-addressed durable storage,

hash-verified reads, atomic write, exact-CAS, closure-verified bundles (§5).

  1. What one service does it offer? A queryable pedigree register — given a record,

its ancestry; given two states, what changed between them; given a claim, the evidence chain that produced it, with every link re-derivable.

  1. What does it connect to on either side? Beneath: TRVM/forge/. Above: KILN's

manifest and audit lineage first, then any compose lane that needs provenance it can show. It connects to nothing that has a user in it until §10.2 is answered.

5. What substrate already provides

Evidence, not assertion — running code with batteries behind it:

CapabilityWhereGuarantee
Content-addressed object storeTRVM/forge/wrl_store.py _ContentStorekey IS sha(bytes); put refuses mislabeled content (WRL_STORE_ID_MISMATCH)
Verified readwrl_store.py _get_bytesre-hashes every read; WRL_STORE_CORRUPT on mismatch, WRL_STORE_MISSING on absence
Durable writewrl_store.py _atomic_writetemp → flush+fsync → os.replace → dir fsync; a crash leaves a .tmp-* stub, never a torn record
Idempotent write_put_bytessame content → same id → same file; reorder- and label-equivalent inputs collapse to one file
Mutable named documentsTRVM/forge/wrl_project.pynamed docs over the immutable substrate
Optimistic concurrencywrl_project.pyper-document exact-CAS: WRL_PROJECT_STALE, WRL_PROJECT_EXISTS, WRL_PROJECT_MISSING
Verified export/importTRVM/forge/wrl_bundle.pyclosure (WRL_BUNDLE_UNRESOLVED), identity re-derivation (WRL_BUNDLE_IDENTITY), corruption (WRL_BUNDLE_CORRUPT)
Crash recoveryRecoveryJournalV1 (v0.6-0)separate, atomic, non-authoritative checkpoint; never auto-applied on reopen

That is values, references, per-document transactions, integrity-checked backup and restore, and a crash journal. A from-scratch storage engine would spend its first two milestones earning most of this. Studbook does not rebuild it.

6. What substrate does NOT provide

The honest half. Supabase was not only Postgres — it was PostgREST, Auth and row-level security. Dropping it drops all four.

  1. No query layer and no index. wrl_store.py says it outright: *"The stores hold NO

in-memory index."* ids() is a listdir and a sort. Everything is get-by-id. Any access pattern that is not "I already know the hash" has no answer today. This is the largest gap. §7.3 is a candidate answer to it.

  1. No authentication and no row-level security. Nothing in substrate has any concept

of a user, and content addressing is actively hostile to access control: if the key is the hash of the content, then knowing the content is knowing the key. Confidentiality has to come from somewhere else entirely. WebHost.Systems and FleetPrompt use Supabase Auth and RLS today, so until §10.2 is answered studbook is not their replacement.

  1. No network surface. The stores are library objects over an explicit filesystem

root. No server, no wire protocol, no pooling.

  1. No cross-record transaction. CAS is per-document. Two records that must move

together have no mechanism today.

  1. Keys are tied to the frozen identity ladder. The stores are keyed by sem-,

scen- and replay- prefixes. A general record type is new work — and the identity spine is frozen, so touching it is a ruling to be asked for, never a task to be done. See §7.2.

7. Proposed design — NOT RULED

7.1 The record

A studbook record is canonical bytes plus a declared ancestry, sealed together so that the ancestry is inside the hash rather than beside it. Sketch, not a schema:

  • content — canonical bytes, serialized by the same discipline wrl_canonical.py

already uses (sorted keys, exact types, no floats where an int will do).

  • parents — an ordered list of studbook ids. Order is part of the canonical form,

because a claim derived from A-then-B is not the claim derived from B-then-A.

  • kind — what sort of record this is, so a reader can refuse a record it does not

understand rather than half-read it.

The seal is the existing one: identity is the hash of the canonical bytes, a write of mismatched content is refused, and every read re-derives. That is _put_bytes and _get_bytes behaviour, unchanged.

Two consequences worth stating before they are discovered:

  • A record cannot be written before its parents. A store that admits a record naming

an absent parent has stopped being a pedigree register. The refusal already has a shape — wrl_bundle.py calls it WRL_BUNDLE_UNRESOLVED.

  • Editing history is not possible, only forking it. Changing any ancestor changes

every id beneath it. This is the property; it is not a limitation to be worked around.

7.2 Identity — what studbook must ask for

Studbook needs at least one new key prefix, because §6.5 says the three existing stores are keyed to the WRL ladder and a general record is not a sem-, scen- or replay-.

That is a ruling request, not a design decision, and it is listed as §10.5. This section exists to make the request precise rather than to pre-empt it:

  • one prefix, for the record itself;

  • no second rung for ancestry — ancestry is inside the record, so it is not a separate

identity;

  • nothing that requires re-deriving an existing sem-, scen- or replay- id.

If the answer is no, studbook lives inside an existing rung or does not exist. Either is a legitimate outcome and neither is a reason to widen the ladder quietly.

7.3 The index problem, and the candidate answer

Gap §6.1 is the one that decides whether studbook is weeks or years. Substrate has content addressing and no index. A B+tree gives an index and gives up content addressing as the storage primitive — which is the property the stack's whole claim rests on.

A prolly tree (probabilistic B-tree; invented by the Noms team, used as the storage engine under Dolt) is both. It is a block-oriented ordered search tree whose every node is named by the hash of its contents — a B-tree that is also a merkle DAG. It gives:

  • ordered key access and range scans — the missing index;

  • content addressing preserved at every level, so a divergence stays refusable all the way

down rather than only at the blob boundary;

  • structural sharing between versions — a shared subtree is stored once;

  • history independence — insertion order does not change the tree, so equal content has

equal identity, which is the property _put_bytes already enforces for blobs;

  • diff proportional to the size of the difference, by comparing subtree hashes rather

than walking records.

That last property is not an optimisation for a pedigree register. It is the register: what changed between these two states, and can you prove it, answered by arithmetic rather than by scanning.

Stated with its costs so this does not become an unexamined enthusiasm: chunk-boundary selection is a tuning problem, writes are copy-on-write with real write amplification, point-lookup latency is worse than a tuned B+tree, and the reference implementation represents years of work. This is a reading task before it is a build task.

7.4 Surface — studbook adds none in v0

Studbook is a library over substrate. It gets no new language surface until there is something running to describe.

If a surface is ever proposed, three constraints hold and are checkable today:

  • `~~>` is taken. It is the WRL route arrow — wrl_canonical.py:388 emits

route %s ~~%s~~> %s. ~~ is the mailbox operator.

  • `#` opens a comment. In the bootstrap surface wrl_ir.py:201 does

raw.split("#", 1)[0]; in WRL Core the comment character is ; (wrl_ir.py:423, :508, :522). So any trailing-# form — query#, snapshot# — lexes as a bare token followed by a comment. It cannot parse.

  • `->` is unused. It appears in no wrl_*.py. That makes it available, not chosen.

7.5 Names studbook must not reuse

  • Film. Forge ships Film v0.7: a per-epoch record of world observables, compared

byte-for-byte in every binding_run* battery. If studbook ever grows a replay or diagnostics artifact it needs its own name. Two artifacts sharing a load-bearing name inside one stack is a defect, not a convenience.

  • Any WRL_* refusal code already in use. Studbook's refusals get their own namespace so

a caller can tell which layer refused.

Records studbook owns should be versioned in the shape the tree already uses — forge.bundle.v1, forge.project.v2, forge.recovery.v1, forge.runtime_job.v1 — i.e. <owner>.<thing>.v<n>, owned by studbook.

8. First slice — suggested, not planned

The narrowest useful slice appears to be KILN's manifest and audit lineage. It is provenance-shaped, it has no user in it, it is already in core, and it is already being written.

Concretely: an ordered, content-addressed store of manifests where each names its parents, queryable by range, answering "what changed between these two runs" without a scan. That exercises §7.3 against real data, does not require answering §6.2, and either demonstrates the diff property or fails to — which is what a first slice is for.

This is a suggestion. §10.4 is the ruling.

9. Non-goals

  • Not a general-purpose database. If a product needs joins and ad-hoc queries over

user data, that is an argument for keeping Postgres for that product, and §6.1 says so honestly.

  • Not a SQLite competitor. No point-lookup parity target, no SQL compatibility

surface, no page cache, no GPU or NPU tier. A pedigree register does not need any of it, and adopting them would make every claim above unprovable for years.

  • Not a new identity rung without §10.5.

  • Not a reason to migrate anything today. Nothing moves off Supabase until studbook

can show it holds the same data with the same guarantees.

  • Not a rewrite of substrate storage. §5 is consumed, not reimplemented.

10. What has to be ruled before implementation starts

These are Travis's, not mine.

  1. What happens to the live data. fleet.* serves KILN and FleetPrompt today, and

kag.* (migrations 010–019) has been orphaned since the BendScript pivot on 2026-04-27. Archiving the lane did not archive the database. Studbook does not get to pretend that data is not there.

  1. Where confidentiality comes from, given §6.2. Until this is answered studbook

cannot host anything with a user in it — which today means it cannot replace what WebHost.Systems and FleetPrompt use Supabase for.

  1. Whether studbook is one store or two. Provenance records want content addressing.

User accounts want mutability and secrecy. Forcing both into one design is how this gets worse than what it replaces. §7.3 resolves the index question only for the provenance half.

  1. Scope of the first slice (§8).

  2. Whether the identity ladder admits a studbook prefix (§7.2), and if not, what

studbook is keyed by instead.

11. Prior art, and what each one settles

Named so the design is not re-derived from scratch, and so distinctiveness is not overclaimed.

  • CozoDB — embedded, transactional, relational-graph, recursive Datalog, with

per-relation time travel. Settles that embedded + recursive + historical is buildable and shipped; anything studbook claims there is not novel.

  • Dolt / Noms — SQL with a commit graph, branch, merge, cell-wise diff and lineage,

built on prolly trees. This is the closest existing system to what §7 describes, and it is where §7.3 comes from.

  • Feldera / DBSP — automatic incremental view maintenance for rich query languages

(VLDB '23 best paper). Settles that "keep derived results correct as the world changes" is a solved research problem with an implementation, and is not a reason on its own to build a new engine.

  • Datomic — immutable facts, transaction time, as-of queries. The model studbook's

record shape is closest to conceptually.

  • SQLite — the separation of pager, B-tree, WAL and VFS. Worth learning as a set of

boundaries even though §9 declines to build them.

What none of them have is a record whose identity is derived by the same frozen spine that seals the rest of this stack, so that a divergence is refusable across the boundary rather than only inside the database. That is the only distinctiveness studbook should claim, and it is the one that does not require out-building any of the systems above.

Open in the interactive atlas