docs(architecture): ADR-010/011 — I/O seams, GC-state home, vocabulary; pre-decomposition consistency round
ADR-010 (new): the Backend trait's I/O seams pinned before any code — 'get' returns a crate-internal read cursor (kv engines materialize; 'local' pread loop; pg-lo lo_get windows), 'put' has two named forms (whole-value kv / staged-put large); GC state is store-core-owned via a crate-internal contract-tested engine-state seam on the SQL-backed engines (backends never learn liveness — ADR-005 verbatim); the store core holds the joint entry+pin transaction (corrects ADR-009 §2's ownership statement); kv-tier fleet-validity rule added (mirror of ADR-008 §3); ops pin-token TTL race resolved (the token IS the pin; renewal rides have/re-put; pause-past-TTL falls to delete-then-recover); feature graph pinned (pg-lo does not imply postgres). ADR-011 (new): vocabulary pinned — tier (contract, exactly two) vs engine (concrete impl, exactly five); store instance / node / fleet split (fleet = pool-sharing, not node count — fixes requirements.md's self-contradiction with REQ-2); mem demoted from 'backend'/'testing tier' to the kv tier's third engine (contract-reference engine); fs tier renamed 'large' including the feature name (last free moment before code exists); constructor modes pinned (dual-tier default, kv-only, mem-only) plus the composite fleet-validity predicate. Consistency round across all specs and ADR-003/004/007/008/009: stale pg-lo passages resolved per ADR-009's supersession; ADR-008 owner 'node id' → store-instance id; threshold wording corrected (conservative edge, not midpoint, of the 128-256 KiB crossover zone); SweepReport shape specced; mem classification unified; 008/009 ADR files renamed to match the tier rename; README deferral-policy recap aligned (third category = decided-but-sequenced work, not a parking kind); POC crate list completed. Verification: two independent architecture review rounds (the first found 4 criticals — unpinned trait I/O shapes, missing fleet-state seam, node/fleet self-contradiction, pin-token/TTL conflict — all resolved; final round: zero criticals); all markdown links resolve; ADR tables complete (11 ADRs).
This commit is contained in:
1 parent
9bbe6f3ad9
commit
abf8c5ae9c
18 files changed
+1137
-336
No files matched your search
@@ -0,0 +1,225 @@
|
||||
# ADR-011: Vocabulary — tiers, engines, store instances, fleets; mem joins the kv tier; the fs tier is renamed `large`
|
||||
|
||||
## Status
|
||||
|
||||
Accepted
|
||||
|
||||
## Context
|
||||
|
||||
The engine story (ADR-007/008/009) grew the physical layer faster than
|
||||
its vocabulary grew, and the pre-decomposition review found the drift
|
||||
is now self-contradictory in the anchor document itself:
|
||||
|
||||
1. **Node vs store instance.** requirements.md defines **Node** as
|
||||
"one running store instance" — then REQ-2 states "a replicator node
|
||||
may run multiple store instances (a fleet)" and **Fleet** as "two
|
||||
or more nodes over one shared pool". Under the written definition,
|
||||
one process hosting two instances is simultaneously one node (by
|
||||
definition) and two nodes (by REQ-2), and the GC arbitration
|
||||
domain ("node is the GC arbitration domain") is undefined for the
|
||||
exact topology ADR-008 exists for. ADR-010 fixes pin ownership;
|
||||
the pin's *owner id* and the node/instance distinction it needs
|
||||
are still unpinned vocabulary.
|
||||
2. **The "fs" tier name is now wrong.** The tier was named when its
|
||||
only engine was a filesystem (ADR-003's `local` layout). After
|
||||
ADR-009, half its engines are postgres Large Objects — the name
|
||||
describes one engine's medium, not the tier. Worse, "fs" collides
|
||||
with the dispatch language: "large content goes to fs" never said
|
||||
*why* — the tier's actual routing criterion is content length
|
||||
(ADR-003's size-threshold dispatch). The docs already speak the
|
||||
true name informally: store-api.md says "small tier"/"large tier".
|
||||
3. **mem's classification drifts.** backends-and-dispatch.md calls it
|
||||
a "backend" in one place and a "testing/utility tier" in another;
|
||||
ADR-003 lists it under "shipped backends … plus a testing tier";
|
||||
ADR-004 calls it an ephemeral tier and carves it out of the
|
||||
two-backend count. In truth it is neither a tier nor a separate
|
||||
backend: it implements exactly the kv tier's contract (including
|
||||
`size`, trivially), it is the contract-reference engine every
|
||||
other engine's conformance tests mirror, and "engine" names what
|
||||
it is (ADR-007 established that an engine is a concrete
|
||||
implementation of one tier's contract — requirements.md already
|
||||
defines the word this way).
|
||||
4. **Tier and Backend were never formally defined** even though
|
||||
ADR-004's scope note ("the 'two' here counts tiers, not engines")
|
||||
leans on both words. Also undecided: whether the feature names
|
||||
follow the tier rename now or after code exists — the last free
|
||||
moment is now (zero code; after implementation the rename breaks
|
||||
crates.io/feature ergonomics for every consumer).
|
||||
|
||||
Deciding facts are in hand: REQ-2's fleet fact, the engine set's
|
||||
final shape (ADR-007/009), the dispatch mechanism (ADR-003), and
|
||||
ADR-010's GC-state-home table. This is a vocabulary-and-boundary
|
||||
decision, not an architecture change: ADR-004's tier≠engine note
|
||||
already established the shape; this ADR gives it final, consistent
|
||||
names.
|
||||
|
||||
Candidates weighed for the tier rename:
|
||||
|
||||
- **`lo`/`los`** — names the *postgres* engine (Large Objects), so
|
||||
"the lo tier's `local` engine" would be backwards. Rejected.
|
||||
- **`lfs`** — workable, echoes git-lfs specifically; but the tier
|
||||
also serves packfiles and any oversized workspace artifact, and the
|
||||
echo binds the crate's naming to one consumer's protocol shape.
|
||||
Rejected.
|
||||
- **`large`** — the dispatch-true name (size-threshold routing), the
|
||||
word the specs already use informally ("large tier"), medium-agnostic.
|
||||
**Chosen.** For symmetry in prose, the kv tier stays `kv` (its
|
||||
routing criterion — "small" — lives in the prose as "small blobs"
|
||||
and needs no rename: `kv` names the mechanism class, and `small`
|
||||
as a feature/tier name would read as an adjective; the pair's
|
||||
asymmetry is cosmetic, the *load-bearing* correction is that
|
||||
neither tier is named after a medium).
|
||||
|
||||
Feature-name mirroring: `fs` → `large`; `pg-lo` keeps its name (it
|
||||
names the engine's mechanism, accurate); `postgres` keeps its name
|
||||
(it names the kv engine); `mem` keeps its name; `kv` and `ops` keep
|
||||
their names.
|
||||
|
||||
## Decision
|
||||
|
||||
### 1. Vocabulary (the pinned definitions; requirements.md is the canonical copy)
|
||||
|
||||
- **Tier** — one *contract* (a `Backend` trait implementation
|
||||
boundary) serving one routing class of the pool. The crate ships
|
||||
exactly two tiers: **kv** (small content) and **large** (over
|
||||
the size-threshold content). Tier count (2) is ADR-004's count.
|
||||
- **Engine** — a concrete storage implementation of one tier's
|
||||
contract (the existing definition, unchanged). Engine count (5) is
|
||||
this ADR's count.
|
||||
- **Backend** — *the* contract (the `Backend` trait) plus its shipped
|
||||
engine set, i.e. the tier as shipped. "Backend" and "tier" name the
|
||||
same thing at different altitudes; the crate says "tier" for
|
||||
counting, "the `Backend` trait" for the contract. "Two backends"
|
||||
reads forever as "two tiers".
|
||||
- **Store instance** — one running instantiation of the store facade
|
||||
(one engine set — one or two engines per the constructor mode, §4 —
|
||||
plus dispatch and GC state). **The GC
|
||||
arbitration domain** for single-instance topologies; the pin-row
|
||||
owner id under fleet engines (ADR-010's `owner`; this corrects
|
||||
ADR-008's "node id" wording, which predates the instance/node
|
||||
split — a store instance is what owns pins).
|
||||
- **Node** — one running *process* (an embedder's binary). A node
|
||||
hosts **one or more store instances**. Node is the process/ops unit:
|
||||
alkcall connections, ACL grants, sweep cadence, and deployment
|
||||
identity attach to nodes; GC arbitration and pin ownership attach
|
||||
to store instances.
|
||||
- **Fleet** — **two or more store instances over one shared pool**
|
||||
(via a shared SQL engine). Deliberately not defined by node count:
|
||||
two instances in one process over one postgres pool are a fleet
|
||||
under this definition (and REQ-2's phrasing "a replicator node may
|
||||
run multiple store instances (a fleet)" reads true as written);
|
||||
two processes on two sqlite files are two pools, never a fleet.
|
||||
- **Deployment** — unchanged (one operator's whole system).
|
||||
|
||||
### 2. mem joins the kv tier as its third engine
|
||||
|
||||
`mem` is the **kv tier's ephemeral engine** (feature `mem`,
|
||||
default-off): a `BTreeMap`-shaped implementation of the kv contract,
|
||||
including `size` (trivially) and list; never fleet-valid (ADR-010 §3's
|
||||
rule and this ADR §4's constructor table); the **contract-reference
|
||||
engine** — every other engine's
|
||||
conformance tests mirror its suite. It is not a tier, not a third
|
||||
backend, and the ADR-004 count ("two") survives untouched because
|
||||
tier and engine were never the same count — this ADR is that
|
||||
distinction's final form.
|
||||
|
||||
### 3. The fs tier is renamed `large`
|
||||
|
||||
In every document, status line, feature name, and constructor
|
||||
parameter: tier `fs` → tier `large`; feature `fs` → feature `large`
|
||||
(default-on, as `fs` was); dispatch prose says "large tier"/"small
|
||||
tier" as store-api.md already does; the `local` engine's name is
|
||||
kept (its meaning — this node's own filesystem — stays the
|
||||
constructor's meaning even under the shared-media fleet posture,
|
||||
where it is explicitly *not* only-node-local; the constructor's
|
||||
explicit shared-media assertion, ADR-008 §3, is what disambiguates).
|
||||
|
||||
### 4. The constructor table (the load-bearing copy of the fleet-validity summary; ADR-010 §3 points here)
|
||||
|
||||
| Tier | Engine | Feature | Default | Fleet-valid |
|
||||
|---|---|---|---|---|
|
||||
| kv | sqlite | — (tier default) | yes (tier default-on) | no |
|
||||
| kv | postgres | `postgres` | no | **yes** |
|
||||
| kv | mem | `mem` | no | no |
|
||||
| large | local | — (tier default) | yes (tier default-on) | shared media only, explicit assertion |
|
||||
| large | pg-lo | `pg-lo` | no | **yes** |
|
||||
|
||||
Constructor modes (dispatch shapes) — pinned here (implied by
|
||||
ADR-008 §3's re-routing bullet and the mem tests, but never named):
|
||||
- **dual-tier** (the default): kv + large, size-threshold dispatch
|
||||
between them (ADR-003).
|
||||
- **kv-only** (ADR-008's "kv-tier-only dispatch", now named): a
|
||||
constructor mode with no large tier; over-threshold puts are a
|
||||
constructor-time error (the client-node re-routing posture handles
|
||||
them via the ops surface instead — its composition, not a store
|
||||
mode).
|
||||
- **mem-only** (test/embedder-ephemeral mode): the mem engine alone,
|
||||
no dispatch threshold in effect (everything is local and small);
|
||||
a testing posture, explicitly not a production story.
|
||||
- Single-tier SQL modes (postgres-only, pg-lo-only) are **not**
|
||||
constructor modes this crate ships: the two-tier shape is
|
||||
ADR-004's requirement (both tiers are load-bearing by measured
|
||||
economics); "postgres-only" as in *one SQL instance serving both
|
||||
tiers* already exists (kv=postgres + large=pg-lo — the ADR-009
|
||||
consolidation), which is the dual-tier mode over one engine.
|
||||
|
||||
### 5. Deferral-policy wording (README alignment)
|
||||
|
||||
The deferral policy's categories are: `deferred(scope)`,
|
||||
externally-owned questions, and decided-but-sequenced work — three
|
||||
names, defined in open-questions.md's header. README.md's recap says
|
||||
"two parking kinds" and is corrected to name all three (the third is
|
||||
not a *parking* kind — it is work, not a parked question — hence
|
||||
this ADR's wording here rather than a silent edit).
|
||||
|
||||
## Consequences
|
||||
|
||||
**Positive**
|
||||
|
||||
- The anchor document's self-contradiction (node/fleet) becomes a
|
||||
three-level vocabulary (store instance / node / deployment) whose
|
||||
levels map one-to-one onto the mechanisms (GC domain / process-ops
|
||||
unit / operator whole).
|
||||
- The fs→large rename removes the last medium-named tier; feature
|
||||
names, tier names, and dispatch prose now agree, before any code
|
||||
exists to break.
|
||||
- mem's classification is stated once, as the contract-reference kv
|
||||
engine; the "plus a testing tier" carve-outs dissolve.
|
||||
- The constructor table gives task decomposition a closed set: 5
|
||||
engine implementations, 3 constructor modes, 1 dispatch policy.
|
||||
|
||||
**Negative**
|
||||
|
||||
- Vocabulary churn in every doc that says "fs" (the rename is
|
||||
mechanical but touches all specs and ADR-003/004/007/008/009 status
|
||||
lines and reference sections). Zero code exists — this is the last
|
||||
free moment; after Phase 1 starts, feature renames break consumers.
|
||||
- "Backend"/"tier" as interchangeable words risks reader confusion
|
||||
with ADR-003's "Backend trait" — mitigated by the rule that the
|
||||
*trait* is always named with "trait" ("the `Backend` trait"), never
|
||||
bare "the backend".
|
||||
|
||||
**Neutral**
|
||||
|
||||
- No engine's contract, feature flag semantics, or constructor
|
||||
parameter changes; `mem` gains the word "engine" and loses the
|
||||
word "backend".
|
||||
- REQ-2's meaning is unchanged (the fleet definition is re-derived
|
||||
from the same fact, not re-fact'd).
|
||||
|
||||
## References
|
||||
|
||||
- ADR-003 (the trait contract; its "shipped backends" list is now
|
||||
read through this ADR's vocabulary), ADR-004 (tier≠engine — the
|
||||
scope note this ADR gives final names), ADR-007 (kv engines;
|
||||
constructor selection), ADR-008 (the second tier's engine candidate
|
||||
and the fleet topology; the vocabulary it assumed is now pinned),
|
||||
ADR-009 (pg-lo admission; the consolidation option whose
|
||||
constructor mode this ADR names dual-tier), [ADR-010](010-backend-io-seams-and-fleet-state-home.md)
|
||||
(the I/O seams and GC-state home this vocabulary must fit)
|
||||
- [requirements.md](../requirements.md) (the canonical vocabulary
|
||||
copy, REQ-1..4); [backends-and-dispatch.md](../backends-and-dispatch.md)
|
||||
(the tier/engine table's spec home)
|
||||
- `docs/research/poc-trait-dispatch-findings.md` finding 1 (the
|
||||
reference-impl role mem already played in the POCs);
|
||||
`poc-pglo-findings.md` (the gate the engine suite mirrors)
|
||||
Reference in new issue
Block a user