From 9bbe6f3ad9506d9bee48495baf26655940e0a20b Mon Sep 17 00:00:00 2001 From: "glm-5.3-flash" Date: Sat, 3 Oct 2026 04:30:31 +0000 Subject: [PATCH] =?UTF-8?q?docs(architecture):=20ADR-009=20=E2=80=94=20pg-?= =?UTF-8?q?lo=20admitted=20as=20the=20fs=20tier's=20second=20engine=20on?= =?UTF-8?q?=20POC=20#7?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - ADR-009 (new): postgres Large Objects ship as the fs tier's second engine (feature pg-lo, default-off), admitted on POC #7's passed gate (contract 10/10, durable put ~= fs at >=1 MiB, lo_get window gets, transactional LO lifecycle = zero crash orphans). Engine shape: companion table (lo_entries) is the contract authority; put = one tx (lo_create + lowrite + row + pin); get = descriptorless lo_get windows (lo_*64 discipline); named operator deltas (catalog space never returned / autovacuum inherited / sweep for the legacy bypass class only) and named performance deltas (cached-get 20-50x behind page-cache fs single-stream, ~700 MB/s aggregate at 16 readers = the fleet serving picture). - Ripple: backends-and-dispatch (pg-lo shipped; fleet locality contract gains the whole-fleet-pg-lo option), overview (layer map, feature table + pg-lo row, OQ -> shipped pointer), requirements (engine def + REQ-2 note updated), README (current-state + ADR table), store-api (invariant 1/6 engine lists), ADR-008 Status amendment (pg-lo posture superseded by ADR-009), open-questions (audit trail completed: decided -> sequenced -> completed). Verification: docs-only; all cross-references resolve (ADR/REQ/POC); register, findings, spec, and both engine-ADR status lines agree. --- docs/architecture/README.md | 21 ++- docs/architecture/backends-and-dispatch.md | 45 ++--- ...rait-size-probe-fleet-gc-and-fs-engines.md | 4 +- .../decisions/009-pg-lo-fs-tier-engine.md | 156 ++++++++++++++++++ docs/architecture/open-questions.md | 12 +- docs/architecture/overview.md | 12 +- docs/architecture/requirements.md | 8 +- docs/architecture/store-api.md | 7 +- 8 files changed, 226 insertions(+), 39 deletions(-) create mode 100644 docs/architecture/decisions/009-pg-lo-fs-tier-engine.md diff --git a/docs/architecture/README.md b/docs/architecture/README.md index 2eee107..02da1da 100644 --- a/docs/architecture/README.md +++ b/docs/architecture/README.md @@ -1,6 +1,6 @@ --- status: draft -last_updated: 2026-10-03 +last_updated: 2026-10-03 (ADR-009 — pg-lo admitted on POC #7) --- # alkblobs — Architecture @@ -47,6 +47,18 @@ there — not inherited as hedges: admission-gated on POC #7). It also adds [requirements.md](requirements.md) — the REQ anchor that ends per-session re-derivation of consumer facts. +- **ADR-009** (2026-10-03) admits `pg-lo` — postgres Large Objects as + the fs tier's second engine (feature `pg-lo`, default-off) — on + POC #7's passed gate (`docs/research/poc-pglo-findings.md`): + contract 10/10 exact-count tests, durable put ≈ fs durable put at + ≥1 MiB, descriptorless `lo_get` window gets, transactional LO + lifecycle (zero crash orphans), companion-table contract authority. + REQ-2's fleet consolidation option is now shipped: a fleet node can + run postgres-only. Named deltas: cached-get 20–50× behind + page-cache fs single-stream (aggregate serving ~700 MB/s at 16 + readers), catalog space reused-but-never-returned, autovacuum + inherited. The admission door (ADR-003, exercised by ADR-007 and + ADR-009) has now shipped both its tier-extending engines. ## Architecture Documents @@ -73,6 +85,7 @@ there — not inherited as hedges: | [006](decisions/006-verification-posture-and-transfer-encoding.md) | Whole-blob verification; transfer encoding excluded by scoping | Accepted | | [007](decisions/007-two-kv-engines-sqlite-and-postgres.md) | Two kv engines (sqlite + postgres) behind one trait; constructor-selected | Accepted | | [008](decisions/008-trait-size-probe-fleet-gc-and-fs-engines.md) | Trait `size` probe; fleet GC (DB-backed pins, advisory-locked sweeper); fs-tier engines | Accepted | +| [009](decisions/009-pg-lo-fs-tier-engine.md) | pg-lo — postgres Large Objects as the fs tier's second engine (admitted on POC #7) | Accepted | ## Open Questions @@ -84,8 +97,10 @@ alkfs intake; carried for visibility, gating nothing here). OQ-09 (second kv engine) resolved by ADR-007. The pg-lo fs-engine question resolved to *sequenced work*, not a parked question: REQ-2/ADR-008 named it the candidate fs-tier engine and POC #7 -(`docs/research/poc-pglo-spec.md`) is its admission evidence — a task, -not a wait. **Deferral +(`docs/research/poc-pglo-spec.md`) was its admission evidence — a +task, not a wait; that task completed 2026-10-03 +(`docs/research/poc-pglo-findings.md`, passed) and ADR-009 shipped +the engine. **Deferral policy (the "Schrödinger's code" rule):** a *decision this crate needs before shipping* may not be deferred on a dependency that is itself waiting for this crate to exist. Two parking kinds remain legitimate: diff --git a/docs/architecture/backends-and-dispatch.md b/docs/architecture/backends-and-dispatch.md index b23e2c8..86d545e 100644 --- a/docs/architecture/backends-and-dispatch.md +++ b/docs/architecture/backends-and-dispatch.md @@ -128,7 +128,7 @@ sweep-outcome CI gate, run against dockerized postgres under near-linearly to ~37k puts/s at 24 conns — the only engine whose throughput *increases* under concurrency. -### fs backend (feature `fs`, default-on); engines: `local` (default) + `pg-lo` (candidate) +### fs backend (feature `fs`, default-on); engines: `local` (default) + `pg-lo` (feature `pg-lo`, default-off) Large blobs. Like the kv tier (ADR-007), the fs tier is **one contract, multiple engines** — the medium is a per-node constructor choice @@ -142,24 +142,31 @@ for fleets): rename for the two-pass unknown-length path; pread-based range reads (POC #3 finding A2 — local range serving is sound, e.g. for packfiles). -- **`pg-lo` engine** (candidate, *not shipped*; ADR-008): postgres - Large Objects as the fs tier's storage — the same engine-behind- - one-trait move as ADR-007, one tier over. Admission is gated on - POC #7's measured evidence (passed 2026-10-03: - `docs/research/poc-pglo-findings.md` — durable put ≈ fs's durable - put, `lo_get` window gets, companion-table authority, clean - crash-orphan behavior) and its own engine ADR (see - `docs/research/poc-pglo-spec.md`). -- **Fleet locality contract (ADR-008):** over one shared pool, the fs - tier is either shared media (every node's `local` root on the same - fleet-shared media — soundness caveats and the required media - guarantees in ADR-008) or re-routed (one storage node serves pool - large-blob content via the ops surface; client nodes are - kv-tier-only over pool content). Mixed per-node-local fs tiers over - one pool is the partitioning failure this contract exists to - prevent — a documented deployment invariant the constructor cannot - fully prove but must be declared against (ADR-008 names the - enforceable seam and the detection symptom). +- **`pg-lo` engine** (shipped, feature `pg-lo`, default-off; ADR-009): + postgres Large Objects as the fs tier's storage — the same + engine-behind-one-trait move as ADR-007, one tier over, admitted on + POC #7's measured evidence (contract 10/10 exact-count tests; + durable put ≈ fs durable put at ≥1 MiB, beats it <1 MiB on the POC + disk; gets ride descriptorless `lo_get(oid, off, len)` windows over + the companion table as contract authority; LO creation + transactional — zero crash orphans). Named deltas: catalog space + reused-but-never-returned (monitor rel size; `VACUUM FULL` is the + operator's shrink path), autovacuum inherited, cached-get 20–50× + behind page-cache fs single-stream (~700 MB/s aggregate at 16 + readers — the fleet serving picture). +- **Fleet locality contract (ADR-008, extended by ADR-009):** over one + shared pool, the fs tier is one of: shared media (every node's + `local` root on the same fleet-shared media — soundness caveats and + the required media guarantees in ADR-008), re-routed (one storage + node serves pool large-blob content via the ops surface; client + nodes are kv-tier-only over pool content), **or `pg-lo` for the + whole fleet** (ADR-009 — one engine, no shared media, no re-routing; + pool content lives in the same pg instance the kv tier rides). + Mixed per-node-local fs tiers over one pool is the partitioning + failure this contract exists to prevent — a documented deployment + invariant the constructor cannot fully prove but must be declared + against (ADR-008 names the enforceable seam and the detection + symptom). ### mem backend (feature `mem`, default-off) diff --git a/docs/architecture/decisions/008-trait-size-probe-fleet-gc-and-fs-engines.md b/docs/architecture/decisions/008-trait-size-probe-fleet-gc-and-fs-engines.md index 86e57e5..3aefa5e 100644 --- a/docs/architecture/decisions/008-trait-size-probe-fleet-gc-and-fs-engines.md +++ b/docs/architecture/decisions/008-trait-size-probe-fleet-gc-and-fs-engines.md @@ -2,7 +2,9 @@ ## Status -Accepted +Accepted (the §3 pg-lo bullet's "candidate, not shipped" posture was +superseded 2026-10-03 by ADR-009, which admitted pg-lo on POC #7's +passed gate; every other part of this ADR stands as written) ## Context diff --git a/docs/architecture/decisions/009-pg-lo-fs-tier-engine.md b/docs/architecture/decisions/009-pg-lo-fs-tier-engine.md new file mode 100644 index 0000000..6ae30e2 --- /dev/null +++ b/docs/architecture/decisions/009-pg-lo-fs-tier-engine.md @@ -0,0 +1,156 @@ +# ADR-009: pg-lo — postgres Large Objects as the fs tier's second engine + +## Status + +Accepted (admission evidence: POC #7, passed 2026-10-03 — +`docs/research/poc-pglo-findings.md`; the ADR-008-named candidate's +gate) + +## Context + +ADR-008 named `pg-lo` as the candidate fs-tier engine for REQ-2's +fleet topology (multiple store instances sharing one pg pool, serving +large/beyond-threshold pool content) — expressly gated on the ADR-003 +admission door: measured evidence first, in the ADR-007 shape. POC #7 +ran that door (`poc-pglo-spec.md` → `poc-pglo-findings.md`, +findings C1–C7): a ~400-line `PgLoBackend` over the ADR-003/008 trait +contract (including `size`), 10/10 exact-count sweep-outcome contract +tests, on the POC #5 driver stack (tokio-postgres + deadpool-postgres +via SQL `lo_*` functions — no new dependency, as ADR-008 §Neutral +projected), against dockerized postgres:16. + +The gate's three legs all passed: + +- **Contract** — complete `list()` (= companion table == LO catalog + oids, after mixed ops), GC-participating delete, virgin-store + no-ops, CAS re-put, stage rollback, ranged reads; proven by the + exact-count sweep-outcome shape ADR-007's gate uses. +- **Performance** — durable puts 60–65 MB/s at ≥1 MiB, within 5–10% of + durable local fs at 16–128 MiB and *beating* it below 1 MiB on the + POC box (fsync-dominated); cached gets 70–180 MB/s single-stream, + ~700 MB/s aggregate at 16 readers — 20–50× behind page-cache fs + single-stream, named honestly (see Consequences). +- **Ops posture** — the tx-scoped-descriptor question resolved in + pg's favor (descriptorless `lo_get(oid, off, len)` windows: pool- + friendly, stateless, ranged reads never pay the tx cost); LO + catalog churn page-granular and reusing; LO creation transactional — + crash orphans structurally zero. + +## Decision + +**The fs tier ships `pg-lo` as its second engine — postgres Large +Objects — behind the same `Backend` trait, feature `pg-lo` (default- +off), constructor-selected per node.** REQ-2's deployment choice set +(ADR-008) extends from {shared media, re-routing} to include the +consolidation option as a shipped engine. + +1. **Engine shape (POC #7 C1 is the contract):** LOs hold the bytes; + the companion table `lo_entries (key bytea PK, loid oid, size + bigint, committed_at)` is the *contract authority* — `list()`, + `size()`, `has()`, CAS-existence are all table-level. Never trust + or read `pg_largeobject` for addressing (the C6 sweep validates + against the table only). +2. **Put = one transaction**: BEGIN → `lo_create` → chunked `lowrite` + (512 KiB statements — the measured knee; statement count, not + LOBLKSIZE, is what costs) → companion-row insert → COMMIT. + Pin-before-publish (ADR-005/008) rides this shape: entry row + pin + row commit together with the content. Unknown-length puts stage + into the LO and commit the row on completion; rollback discards + both. +3. **Get = `lo_get(oid, off, len)` windows by default** (POC #7 C4): + the fs-tier handle closes over the companion row (oid + length) — + immutable, so the row is authoritative — not over a tx-scoped + descriptor. Ranged reads are single stateless statements, + pool-friendly (acquire p99 1–6 ms ≤ pool size). The held-descriptor + tx posture exists as the fallback (measured: no advantage even for + sequential full reads). **Reads ride `lo_*` 64-bit variants + end-to-end** (`lo_lseek64`/int8 offsets; `lo_lseek`'s int4 cap is a + C7 trap the engine must not inherit). +4. **Delete = one tx** (`lo_unlink` + row delete) under the ADR-008 + delete-window arbitration (table-level; nothing fs-specific). +5. **Posture deltas, as operator requirements (the honest cost of the + engine, POC #7 C5/C6):** + - **LO catalog space is reused but never returned** — the engine's + inverted-fs property. The engine ADR requirement: rel-size + monitoring (`pg_total_relation_size('pg_largeobject')`) named in + ops docs; `VACUUM FULL`/`pg_repack` is the only shrink path — + an operator runbook line, never automatic. + - **Autovacuum inherited; keep it on.** `pg_largeobject` is an + ordinary catalog heap (autovacuum applies, measured active); a + deployment disabling autovacuum gets an operator-visible note. + - **Orphan-recovery sweep exists for the legacy/bypass class only** + (committed LOs without companion rows — mixed/legacy stores). The + crash story is structurally clean: LO lifecycle is transactional + in modern postgres; kill mid-tx leaves zero pages, zero + half-commits (cleaner than fs `local`'s stage-file residue, + which needs its own recovery sweep). +6. **Performance deltas, as named capacity facts (not hidden, not + solved):** single-stream cached get runs 20–50× behind page-cache + local fs at ≥1 MiB (server-side LO page-walk + copy ceiling, + ~65–70 MB/s per in-flight statement); the fleet serving picture + aggregates ~700 MB/s at 16 readers (server-side parallelism) — + which is REQ-2's relevant number — and exceeds-threshold single + clone streams get ~120 MB/s. Deployments whose large-blob serving + is dominated by single-stream page-cache-bound reads stay better + served by `local`-on-shared-media; pg-lo's case is the + consolidated fleet (one engine to operate) whose serving burden is + many-user aggregate. A third engine (S3-like) remains the + ADR-003 door; nothing here touches it. +7. **redb-class guardrail maintained:** this is the second engine the + admission door ships (ADR-007: kv; ADR-009: fs) — both on the same + measured-evidence shape; no third engine is opened by this one. + +## Consequences + +**Positive** + +- REQ-2's consolidation option is real and shipped: a fleet node runs + *postgres only* — kv (bytea) tier + fs (LO) tier from one engine, + no shared-media requirement, no re-routing topology, no consumer-ops + dependency for pool content. +- The engine adds ~400 lines of impl over an already-fixed contract — + the additive shape both ADR-007 and ADR-008 projected, now + evidenced. +- Crash hygiene is *better* than fs `local` (transactional LO + lifecycle vs. stage-file residue); the engine's only recovery sweep + targets a class (bypassed-LO) that healthy deployments never produce. + +**Negative** + +- Single-stream cached get is 20–50× behind local fs — a real, + named cost. Deployments with page-cache-friendly media and single- + dominant readers should stay on `local` engines. (Aggregate serving + — the fleet case — scales to ~700 MB/s on the POC box; per-box + numbers shift with cores.) +- LO catalog space is never returned (monitor + occasional + operator-run `VACUUM FULL`), the inverse of fs `local`'s + immediate-return. +- Feature/maintenance surface: the fs tier now carries two engines + (like the kv tier), each with posture deltas to keep honest + (ADR-007's negative, inherited). + +**Neutral** + +- The `Backend` trait is unchanged (ADR-008's `size` probe is the + last trait change — pg-lo fits the contract as it stands). +- Driver stack: unchanged (tokio-postgres + deadpool; SQL `lo_*` + functions, `postgres_large_object` crate stays avoided). +- The `pg-lo` feature defaults off; CI's `--all-features` gate adds + it to the sweep-outcome suite (dockerized postgres:16, same shape + as the `postgres` kv engine's gate). + +## References + +- `docs/research/poc-pglo-findings.md` (POC #7, findings C1–C7 — the + evidence base); `poc-pglo-spec.md` (the gate this ADR's legs + mirror) +- [requirements.md](../requirements.md) — REQ-2 (the fleet topology + this engine serves); REQ-1/3/4 unchanged postures +- ADR-003 (the trait contract + admission door exercised twice now), + ADR-004 (tier ≠ engine, fs tier edition), ADR-005/008 (the GC + invariants this engine participates in — table-level, nothing + fs-specific), ADR-007 (the engine-admission precedent and driver + stack) +- [backends-and-dispatch.md](../backends-and-dispatch.md) (the fs + engine surface this amends); [overview.md](../overview.md); + [store-api.md](../store-api.md) \ No newline at end of file diff --git a/docs/architecture/open-questions.md b/docs/architecture/open-questions.md index e7e302a..4f5f4e9 100644 --- a/docs/architecture/open-questions.md +++ b/docs/architecture/open-questions.md @@ -253,10 +253,14 @@ multi-instance-with-large-blobs topology a standing requirement nothing to defer *on*. It resolved to ADR-008 naming pg-lo as the candidate fs-tier engine + POC #7 (`docs/research/poc-pglo-spec.md`) as its admission evidence — sequenced work, per the deferral-policy -header's third category. An earlier draft of that OQ parked on -"alkgit's architecture process names the topology" — rendered moot the -same day by the REQ-2 fact; recorded here because the audit trail of -*why not parked* is part of this document's job. +header's third category. That work completed the same day: POC #7 +passed (`docs/research/poc-pglo-findings.md`, gate verdict passed +with named deltas) and ADR-009 shipped the engine. An earlier draft +of that OQ parked on "alkgit's architecture process names the +topology" — rendered moot by the REQ-2 fact before it was even +written; recorded here because the audit trail of *why not parked* is +part of this document's job, and because it is the worked example of +the header's third category (decided → sequenced → completed). OQ-07/OQ-08 are owned by other repos' processes and are not alkblobs tracker tasks (their outcome arrives *through* their owners, not diff --git a/docs/architecture/overview.md b/docs/architecture/overview.md index 40a06b1..3083698 100644 --- a/docs/architecture/overview.md +++ b/docs/architecture/overview.md @@ -70,7 +70,8 @@ or three times. │ │ ADR-007: kv = 2 engines; │ ▼ ADR-008: fs engines too) │ engine per node: sqlite (default) | postgres (feature "postgres") -│ fs engine per node: local (default) | pg-lo (candidate, POC #7) +│ fs engine per node: local (default) | pg-lo (feature "pg-lo", +│ default-off; ADR-009) transport: none, anywhere. alkgit/alkfs or the embedder dials; an alkcall Connection is handed in at the ops boundary. @@ -82,6 +83,7 @@ or three times. |---|---|---| | base crate (store core + backends) | `tokio`, `thiserror`, `parking_lot`, plus `rusqlite` behind the `kv` feature and nothing behind `fs`/`mem` | no alkcall, no serialization frameworks | | `postgres` feature (ADR-007) | adds `tokio-postgres` + `deadpool-postgres` behind the kv tier's second engine, default-off | depends on `kv`'s contract surface (the `Backend` trait definition is base-crate code); the engine's *impl* is additive. Same `Backend` contract, own sweep-safety CI gate under `--all-features` | +| `pg-lo` feature (ADR-009) | Large Objects as the fs tier's second engine, default-off | rides the same `tokio-postgres`/`deadpool-postgres` stack as the `postgres` kv engine (SQL `lo_*` functions, no new dependency); same admission/test-gate shape | | `ops` feature | adds `alkcall` (feature-gated, default-off) | JSON op specs + binary channels + `AccessControl`; ADR-001 | | consumer crates | depend on alkblobs (and on alkcall directly when they speak ops) | alkgit, alkfs | @@ -126,6 +128,7 @@ wasm-clean), never an in-process embedder (ADR-001 §Consequences). | [006](decisions/006-verification-posture-and-transfer-encoding.md) | Verification posture | whole-blob verification; chunk trees excluded by scoping | | [007](decisions/007-two-kv-engines-sqlite-and-postgres.md) | Two kv engines | sqlite default + postgres behind one trait; one relational engine + fs per node | | [008](decisions/008-trait-size-probe-fleet-gc-and-fs-engines.md) | `size` probe + fleet GC + fs engines | trait length probe; fleet pins/sweeps engine-backed; fs tier engine-selectable | +| [009](decisions/009-pg-lo-fs-tier-engine.md) | pg-lo fs engine | postgres Large Objects as the fs tier's second engine, admitted on POC #7; a fleet node can run postgres-only | ## Open Questions @@ -140,10 +143,9 @@ crate level: open-read requirement) - **OQ-10**: second kv engine — resolved by ADR-007 (sqlite + postgres shipped behind one trait; per-node constructor choice) -- **POC #7** (`docs/research/poc-pglo-spec.md`): pg Large Objects as - the fs tier's `pg-lo` engine — spec'd **sequenced work** (not a - parked question; open-questions.md's policy header distinguishes the - two): admission evidence for ADR-008's named candidate (REQ-2) +- **POC #7** (`docs/research/poc-pglo-findings.md`): pg Large Objects + as the fs tier's `pg-lo` engine — **passed** 2026-10-03; admitted as + a shipped engine by ADR-009 (REQ-2's consolidation option) ## References diff --git a/docs/architecture/requirements.md b/docs/architecture/requirements.md index db49cfc..a1e2772 100644 --- a/docs/architecture/requirements.md +++ b/docs/architecture/requirements.md @@ -1,6 +1,6 @@ --- status: draft -last_updated: 2026-10-03 +last_updated: 2026-10-03 (pg-lo admitted by ADR-009 on POC #7 — REQ-2's consolidation option is shipped) --- # Requirements @@ -16,8 +16,7 @@ the ADRs re-derive from it.** - **Engine** — a concrete storage implementation of one tier's contract: the kv tier's engines are sqlite and postgres (ADR-007); - the fs tier's engines are `local` (default) and `pg-lo` (candidate — - see REQ-2, ADR-008). + the fs tier's engines are `local` (default) and `pg-lo` (ADR-009). - **Pool** — the flat content-addressed namespace of one deployment (`hash → bytes` everywhere; ADR-005). - **Node** — one running store instance (one process embedder; its fs @@ -62,7 +61,8 @@ replicators; ACL/ownership above (alkcall layer). Consequences are derived in ADR-008: fleet GC (DB-backed pins, advisory-locked sweeper, SQL delete arbitration), the fs-tier engine concept with its locality contract, and the pg-lo candidate - engine (POC #7). + engine (POC #7 — **passed** 2026-10-03, admitted as a shipped + engine by ADR-009: a fleet node can consolidate on postgres-only). - **REQ-3 (traces: ADR-007, POC #5 B3/B4)** — concurrency scale-out: the fleet's hub-class nodes need put throughput that *increases* under concurrency (pg: near-linear to ~37k puts/s at 24 conns); diff --git a/docs/architecture/store-api.md b/docs/architecture/store-api.md index 83ce281..aa6bdc0 100644 --- a/docs/architecture/store-api.md +++ b/docs/architecture/store-api.md @@ -129,7 +129,8 @@ lesson generalizes to any kv engine). 1. Whole-blob put/get round-trips byte-identical under the canonical derivation, both tiers, both put paths, **and every shipped engine - of both tiers** (kv: sqlite + postgres; fs: `local`; ADR-007/008) + of both tiers** (kv: sqlite + postgres; fs: `local` + pg-lo; + ADR-007/008/009) (interop with real git remains the source of truth, per POC #1's lesson about hardcoded vectors). 2. Dedup: putting identical content twice (same or different path) is @@ -147,8 +148,8 @@ lesson generalizes to any kv engine). tests assert through sweep outcomes (POC #1 finding 2's lesson codified). 6. `size`/`stat` on any engine returns the entry length without - fetching content (asserted per-backend: kv probe, fs stat — - ADR-008). + fetching content (asserted per-backend: kv probe, fs stat, pg-lo + companion-row probe — ADR-008/009). ## Design Decisions