From 493516424a825ac30a90c4445b77f62b34616fe0 Mon Sep 17 00:00:00 2001 From: "glm-5.3-flash" Date: Thu, 1 Oct 2026 06:51:32 +0000 Subject: [PATCH] =?UTF-8?q?docs(research):=20POC=20#1=20passed=20=E2=80=94?= =?UTF-8?q?=20trait=20contract=20+=20digest=20+=20dispatch?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit POC code: /workspace/alkblobs-trait-poc (17 tests passing, clippy -D warnings + fmt clean; digests validated byte-exact against the real git hash-object CLI, both object formats) Findings (poc-trait-dispatch-findings.md, 8): - lean Backend trait (opaque byte keys, complete list(), put-path pinning) is sufficient; Key typed wrapper lives at the boundary - redb traps: list() key-vs-value destructure (silent until GC catches it — malformed list is as deadly as incomplete), virgin-db TableDoesNotExist on first read, no in-memory db (temp file used) - pinning: Pinned RAII guard + refcount map works; batch-scope pins and the sweep-vs-put race are named Phase 1 requirements (iroh DeleteSet = prior art) - git-blob preamble abstraction validated; external-git oids address pool entries with zero indirection (the OQ-BL-03 resolution, implementation-confirmed) - tagged key format ([algo byte][digest]) makes git-sha256 + git-sha1 coexist; sort-stable for sweep diffs - dual dispatch works as simply as hoped; routing deterministic - namespace tables: protect callback MUST share live state (the snapshot-install bug proves the API lesson: register_liveness_source shape, not install_into) - GC semantics verified: exact deletion counts, abort-callback, delete-then-recover byte-identical re-put phase-0.md: POC #1 marked passed, #2 absorbed (validated), #4 covered in miniature (concurrency half deferred to Phase 1 impl work); OQ-BL-02 base settled; OQ-BL-03 implementation-validated; OQ-BL-05 mechanism settled with findings folded in; plan updated (POC #3 is the last open POC) --- docs/research/phase-0.md | 159 +++++++----- docs/research/poc-trait-dispatch-findings.md | 246 +++++++++++++++++++ 2 files changed, 340 insertions(+), 65 deletions(-) create mode 100644 docs/research/poc-trait-dispatch-findings.md diff --git a/docs/research/phase-0.md b/docs/research/phase-0.md index 39b32f3..4f4b64f 100644 --- a/docs/research/phase-0.md +++ b/docs/research/phase-0.md @@ -1,10 +1,9 @@ --- status: draft -last_updated: 2026-10-01 (hash simplification round: canonical git-family -hash resolution — BLAKE3 demoted from a requirement to a conditional; -OQ-BL-03 rewritten, POC #2 absorbed into #1 and #3 given the -micro-benchmark pull-out; earlier rounds: GC/pooling round, -dedup/p2p consumer round, setup draft) +last_updated: 2026-10-02 (POC #1 executed and passed — see +poc-trait-dispatch-findings.md; POC #2 absorbed earlier; register +updated; earlier rounds: hash simplification, GC/pooling, dedup/p2p, +setup draft) --- # alkblobs — Phase 0 (Exploration) @@ -329,16 +328,21 @@ plan against. ### OQ-BL-02: Multi-backend dispatch — the appfile problem -Small blobs in kv/sqlite, large on fs fallback — the shape is agreed, -the mechanics are not: how does put/get pick a backend (size -thresholds? per-algorithm routing? per-namespace config?), how does it -work when a blob should migrate between backends, and what happens on a -get when the "wrong" backend was probed? alknet's probe hit exactly -this ("dealing with more than one backend at a time"); its findings -need re-verification against the current iroh-blobs before relying on -them. Expected shape: a backend trait + a dispatch layer, but the trait -shape is a one-way door once written — deserves a POC before -committing. +**Base settled by POC #1 (2026-10-02, `poc-trait-dispatch-findings.md` +findings 1/2/6):** the lean `Backend` trait contract works — opaque +byte keys (typed `Key` at the store-layer boundary only), +`has`/`get`/`put`/`delete`/`list`/`name`, `list()` complete by +contract (a malformed list is as deadly as an incomplete one — the +reedb trap in finding 2), and size-threshold dual dispatch (kv small / +fs large) is a thin layer with deterministic per-digest routing. The +"virgin store" read path must treat a missing table as absent/empty. + +Remaining (above-trait concerns, no further POC gate): migration +between backends policy; whether per-namespace backend config is ever +needed; the sqlite-vs-fs micro-benchmark (POC #3's pull-out) still +anchors the small-blob belief the dispatch leans on; streaming put/ +get shape rides POC #3's findings (`Vec` was the *contract* +question, not the performance question). ### OQ-BL-03: Hash abstraction — RESOLVED: canonical git-family hash @@ -409,6 +413,18 @@ consumer in sight (OQ-BL-01). The wire-format question reads OQ-BL-03 first — a wire ADR inherits whatever key-encoding mechanics this settles. +**Implementation-validated by POC #1 (2026-10-02):** the +domain-separated preamble abstraction carried both algorithms; +`git-sha256` and `git-sha1` coexist in one flat pool via the tagged +key encoding (`[algorithm byte][digest]`); external-git oids address +pool entries with no mapping layer (byte-exact vs `git hash-object`, +including the empty blob). Residual Phase 1 notes from the POC: the +algo-discriminant must remain part of the key itself (a per-store +single-algorithm config would not have survived the interop test); a +single leading tag byte is probably needless (currently one key +kind); fixed-length enum-tagged keys sort cleanly for the sweep's +live-set diffs. + ### OQ-BL-04: Verification and chunking story iroh-blobs' verification is bao outboard encoding over fixed-size @@ -506,19 +522,29 @@ GC/pooling round; the rudolfs collision):** abort-callback + pinning layers exist to prevent. **Still open (the residual mechanics):** namespace registry shape -(namespaced root tags vs separate reference tables), GC scheduling -(interval + injectable protect vs consumer-driven sweeps), and the pack -tension below. +(namespaced root tags vs separate reference tables — the POC built +in-memory reference tables; persistence of the tables is Phase 1 +mechanical work), GC scheduling (interval + injectable protect vs +consumer-driven sweeps), **concurrency** — the sweep-vs-put race is a +named Phase 1 requirement with iroh's DeleteSet/ProtectHandle +transactions as verified prior art (POC #1 finding 3; the POC worked +single-threaded), batch-scope pins for multi-put writes (finding 3), +and the callback-must-be-live-shared API lesson (finding 7 — prefer a +`register_liveness_source`-shaped seam over any install-copies-state +verb). Mark-and-sweep-with-protect-callback (lean (a) + pinning) is +now empirically the right mechanism; option (b) (store-native +manifest traversal) stays parked unless a real consumer's live-set +computation proves too heavy. **The pack tension (alkgit-shaped, constrains the store surface)** — git packfiles are pack-efficient but blob one object-store-per-pack (bad for cross-repo dedup); loose-per-object in the pooled store dedups -but is fs-inefficient at git's object volumes. Small git objects in the -kv backend largely dissolve this for the common case; the residual -question is what the store must support for large/packed content — -per-object granularity, range reads into packed blobs, or both -(interacts with OQ-BL-04's range-read question — likely read them -together). +but is fs-inefficient at git's object volumes. Small git +objects in the kv backend largely dissolve this for the common +case; the residual question is what the store must support for +large/packed content — per-object granularity, range reads into +packed blobs, or both (interacts with OQ-BL-04's range-read +question — likely read them together). ### OQ-BL-06: POC register (draft) @@ -529,50 +555,50 @@ is actually deciding: | # | What | Status | Where | |---|------|--------|-------| -| 1 | Backend-trait + dispatch shape (kv small / fs large); trait must include `list()` complete by contract (rudolfs anti-lesson) + temp-tag/pinning on the put path; the hash-enum abstraction with git-blob-sha-256's domain-separated preamble + git-sha-1 tolerance case | **Pending** — first POC | findings file TBD | -| 2 | ~~Multi-hash store~~ **Absorbed into #1** (2026-10-01 hash round): the canonical-hash resolution removed the "two families coexisting" question; what remains (the preamble abstraction + SHA-1 tolerance) is POC #1's trait work | **Absorbed** | — | +| 1 | Backend-trait + dispatch shape (kv small / fs large); trait must include `list()` complete by contract (rudolfs anti-lesson) + temp-tag/pinning on the put path; the hash-enum abstraction with git-blob-sha-256's domain-separated preamble + git-sha-1 tolerance case | **Passed 2026-10-02** — `poc-trait-dispatch-findings.md` (8 findings; code: `/workspace/alkblobs-trait-poc`) | findings landed | +| 2 | ~~Multi-hash store~~ **Absorbed into #1** (2026-10-01 hash round): the canonical-hash resolution removed the "two families coexisting" question; what remains (the preamble abstraction + SHA-1 tolerance) is POC #1's trait work | **Absorbed** (and validated by #1) | — | | 3 | Large-blob path (iroh-blobs store read under current checkout; fs fallback + range reads; streaming `LFSObject`-shaped put/get + `fanout` seam) + the small-object sqlite-vs-fs micro-benchmark (the dual-belief anchor) | **Pending** — reading-and-design POC + benchmark | findings file TBD | -| 4 | Pooled CAS + GC: namespace reference tables over the flat pool; mark-and-sweep with protect-callback + TempTag pinning; delete-then-recover semantics | **Pending** — after #1 (needs a backend to pool over) | findings file TBD | +| 4 | Pooled CAS + GC: namespace reference tables over the flat pool; mark-and-sweep with protect-callback + TempTag pinning; delete-then-recover semantics | **Covered in miniature by #1** (2026-10-02): namespace tables + sweep + recover all validated single-threaded; the concurrency half (sweep-vs-put race, batch-scope pins) is Phase 1 implementation work, not a POC gate | `poc-trait-dispatch-findings.md` findings 3/7/8 | -Sequencing note: #1 is the single opening worktree now (multi-hash -subsumed; the git-preamble hashing is part of its trait prove-out); +Sequencing note: #1 is done (single opening worktree; multi-hash +subsumed; git-preamble hashing validated byte-exact vs the git CLI); #3 is a reading-and-design POC against the current iroh-blobs checkout plus the alknet probe re-verification, with its micro-benchmark -pull-out; #4's GC half matters most — it is a correctness surface and -should be a worked example at minimum. The pack tension (OQ-BL-05) -rides #3's reading rather than earning its own POC yet. +pull-out; #4's single-threaded core is covered by #1 (see the +register row) and the residual is implementation work. The pack +tension (OQ-BL-05) rides #3's reading rather than earning its own POC +yet. -**What each POC is deciding (updated 2026-10-01, post-hash round):** +**What each POC is deciding (updated 2026-10-02, post-POC-#1):** -- **POC #1 decides the backend trait's minimum contract and the - canonical-hash key encoding** — and both are already sharpened by - the prior rounds: `list()` must be complete (rudolfs' S3 backend - punting on list is *why* it can never GC), the put path must - support not-yet-referenced pinning (TempTag-shaped RAII, iroh-blobs' - batch-scope pattern), and the digest/key abstraction must handle - git's domain-separated preamble (`blob \0`) for both SHA-256 - (canonical) and SHA-1 (tolerated) — i.e. `fn(content) → digest` - alone is insufficient, per OQ-BL-03's resolution. The trait and key - shapes are one-way doors; this POC is where they get evidence - rather than instinct. +- **POC #1 — DONE.** Decided the backend trait's minimum contract + (opaque byte keys, `list()` complete-by-contract, put-path + pinning) and validated the canonical-hash key encoding (git's + domain-separated preamble for both SHA-256 and SHA-1; external-git + oid interop byte-exact). Findings summary in the table row; full + detail in `poc-trait-dispatch-findings.md` (findings 1-8: opaque- + keys-at-the-boundary, the redb list/read-path traps, the pinning + shape + sweep-race caveat, the CLI-validated digest, the tagged key + format, dispatch simplicity, the callback-must-be-live-shared + lesson, GC semantics). - **POC #2 is no more** — the "SHA-256 + BLAKE3 coexisting" question evaporated when BLAKE3's only reason-for-being (iroh-blobs - inheritance) was rejected. Its residual content (the preamble - abstraction, the SHA-1 tolerance case) is POC #1's hardest part now. -- **POC #3 is the reading-and-design leg** — against the *current* - iroh-blobs checkout (the store eval) + re-verifying alknet's probe. - The **micro-benchmark pull-out** (2026-10-01, this round): the - "kv/sqlite is faster for small blobs" belief underlies the dual- - backend design and traces to *old* alknet research — worth anchoring - empirically (small objects, 1KB-64KB, both backends, random + - sequential) before the dispatch policy leans on it. The pack - tension (OQ-BL-05) is analyzed here without its own POC. -- **POC #4 is the GC worked example** — the correctness surface of - OQ-BL-05: namespace reference tables + mark-and-sweep with - protect-callback + TempTag pinning + delete-then-recover, exercised - until the failure modes (pin missed, sweep racing a write, live-set - miscomputed) are named rather than theoretical. Expected to be the - POC that actually constrains the trait in return. + inheritance) was rejected, and its residual content validated under + #1. +- **POC #3 is the next (and last) POC** — reading-and-design against + the *current* iroh-blobs checkout (the store eval) + re-verifying + alknet's probe + the **micro-benchmark pull-out** (the "kv/sqlite + is faster for small blobs" belief underlies the dual-backend design + and traces to *old* alknet research — anchor it: small objects, + 1KB-64KB, both backends, random + sequential). The pack tension + (OQ-BL-05) is analyzed here without its own POC. +- **POC #4's single-threaded core is covered by #1** (namespace + tables + mark-and-sweep + protect-callback abort + delete-then- + recover all validated; `poc-trait-dispatch-findings.md` findings + 3/7/8). The concurrent half (sweep-vs-put race, batch-scope pins, + incremental scheduling) is Phase 1 implementation work — the + mechanism choice it was gating is settled, and races are + construction details with iroh's DeleteSet as the named prior art. POC placement conventions (inherited from alksocks/alktunnels): a POC that needs code from this repo runs in a worktree/branch @@ -583,14 +609,17 @@ in `docs/research/` regardless of where the code lives. ## Phase 0 plan (next steps) -1. **Write `iroh-blobs-eval.md`** against the *current* checkout, - focused on `src/store` (kv + flat backends, bao, chunking) — the - conclusions inventory we may borrow, and the weld points we won't. +1. ~~Write `iroh-blobs-eval.md`~~ — still open, rides POC #3's + reading against the *current* checkout (kv + flat backends, bao, + chunking; the GC mechanics eval is already done — see + `poc-trait-dispatch-findings.md`'s context and phase-0's verified + GC section). 2. **Re-verify alknet's probe** (`alknet-blobs-external-store-probe.md`, `poc-summary.md`) against the current upstream; mark what carried - over. + over — rides POC #3 too. 3. **Read gix-odb** (`/workspace/git-oxide/gix-odb`) for the alkgit baseline: its storage layout, object model, and where a blob-store crate under/beside it earns its keep. -4. **Open POCs** per OQ-BL-06, in the order above. +4. **POC #3** (the last open POC): reading-and-design + the + sqlite-vs-fs micro-benchmark per OQ-BL-06. 5. Converge: recommended approach + final OQ register → Phase 1. \ No newline at end of file diff --git a/docs/research/poc-trait-dispatch-findings.md b/docs/research/poc-trait-dispatch-findings.md new file mode 100644 index 0000000..5d17840 --- /dev/null +++ b/docs/research/poc-trait-dispatch-findings.md @@ -0,0 +1,246 @@ +# POC: trait contract + digest abstraction + dispatch — findings + +> **POC register #1** (absorbing #2), per `docs/research/phase-0.md` OQ-BL-06. +> Code: standalone crate `/workspace/alkblobs-trait-poc` (findings land here +> regardless of where the code lives, per the established convention). +> Date: 2026-10-02. Status: **passed**, with findings that reshape +> OQ-BL-02, OQ-BL-03 (key mechanics), and OQ-BL-05 (GC seam). + +## What this POC set out to decide + +From the phase-0 register (as updated by the GC and hash rounds): + +1. **Backend trait's minimum contract** — can a lean `Backend` trait + (`has`/`get`/`put`/`delete`/`list`) carry everything the store layer + needs, with: + - `list()` complete by contract (the rudolfs anti-lesson — incomplete + listing is why its S3 backend can never GC); + - not-yet-referenced pinning on the put path (the TempTag/RAII shape)? +2. **The digest abstraction's hardest member** — git's oid derivation + (`"blob \0" + content`, per OQ-BL-03's canonical resolution): + does "algorithm as domain-separated derivation" work, can + `git-sha256` (canonical) and `git-sha1` (tolerated) coexist in one + flat pool, and does the pinned key encoding hold up? +3. **Dual-backend dispatch** — does the kv-small/fs-large appfile + dispatch hold together empirically, including the get-fall-through + case? +4. **GC mechanism** — namespace reference tables + mark-and-sweep with + a protect callback + RAII pinning, exercised to the delete- + then-recover semantics (POC #4's core, pulled forward because it + needed only pinning, which the trait work required anyway). + +## Result summary + +| Question | Verdict | +|----------|---------| +| Lean `Backend` trait sufficient? | **Yes** — with `Key→bytes` at the boundary (finding 1) | +| `list()` complete by contract | **Yes** — dual-backend `list()` union passed; redb key-vs-value trap found (finding 2) | +| RAII pinning shape | **Yes** — `Pinned` guard with refcount-on-pins; sweep honors + callback can abort (finding 3) | +| git-blob derivation as digest | **Validated end to end** — byte-exact vs `git hash-object` (finding 4) | +| sha256 + sha1 coexisting via tagged keys | **Yes** (finding 5) | +| Size-threshold dual dispatch | **Yes** — routing, fall-through get, miss path (finding 6) | +| Namespace tables + sweep liveness | **Yes with a callback-share lesson** (finding 7) | +| Delete-then-recover (re-put) | **Yes** — re-put lands byte-identical under the same oid | + +All 17 tests pass (`cargo test`), clippy `-D warnings` and `rustfmt` +clean. Digest validations run against the real `git hash-object` CLI +(git 2.43), not just hardcoded vectors. + +## Findings + +### Finding 1: the `Backend` trait should speak opaque bytes, not typed keys — Key→bytes at the boundary + +The `Backend` trait uses `&[u8]` keys everywhere; `Key` (the typed +digest wrapper) converts at the store-layer boundary via +`Key::as_bytes()`. This kept backends dumb (they cannot know what a +digest is) and the digest layer free to evolve. For the real crate this +suggests the seam as built: + +- `Backend` = opaque byte keys + byte values (+ `list`, `name`), + exactly what was built in the POC (`src/backend.rs`). +- typed keys live in the store/digest layer only. + +The alternative — making `Backend` generic over `K: Key` — was not +tried; with only one key type in play, generics would have bought +nothing and the opaque boundary is also the wire/persistence boundary +(backend files/rows must survive digest-layer evolution: a redb row or +fs filename has no schema migration story). Opaque bytes with a +self-describing tag byte (finding 5) is the durable choice. + +### Finding 2: redb's `list()` returns what you iterate — read the key, not the value; missing-table startup; and the "list is load-bearing" proof + +Three concrete redb behaviors worth recording for the real kv backend: + +1. **Key-vs-value trap (caught by the tests, not by review):** + `table.range(..)` yields `(key, value)` pairs; iterating with the + wrong destructure produced a `list()` of *values* (the content + bytes!). The failure showed up as a sweep that deleted 2 blobs when + 1 was expected (the live-check compared garbage keys, so the pinned + blob's "key" was found in the computed live set... of content + values). Lesson recorded: **`list()` correctness is only observable + through GC** — a wrong list() is silent until a sweep deletes the + wrong things, i.e. the rudolfs "incomplete list = broken GC" lesson + generalize: **a *malformed* list is equally deadly and equally + silent.** The test caught it only because deletion counts asserted + exactly. +2. **Fresh-database read path:** a redb database opened but never + written has *no tables* — `open_table` errors with + `TableDoesNotExist` on the first read. Read paths must tolerate the + missing-table error (treat as empty/absent) or the first `has()` + on a fresh store fails. In the POC, `open_read_table()` maps + `TableDoesNotExist → None` (and `list() → empty`). The real crate + will hit the same thing no matter the kv engine; the general shape + is "read paths on a virgin store must be no-ops, not errors." +3. **redb 2.x has no in-memory database** (iroh-blobs' + previous-versions kv memory store notwithstanding — that is a + different engine). The POC used `open_temp()` (temp dir file) for + kv tests. For the real crate: an in-process ephemeral kv would be + either `MemBackend` (the POC's `BTreeMap` one) or sqlite. + +### Finding 3: pinning (TempTag equivalent) works at the store layer, above backends + +The `Pinned` guard (`src/pin.rs`): put-path RAII that holds a per-key +refcount in the store's pin map. Sweep treats pin-map keys as part of +the live set. This proves the iroh-blobs `TempTag` *conclusion* without +their mechanism (no weak-pointer tag-drop plumbing): a boxed closure +dropped on guard release. What the POC did NOT yet build (and the real +crate must): + +- Batch-scope pins (iroh's `Batch::temp_tag`): a scoped lifetime that + protects a set of in-flight blobs and drops them together. The POC's + per-blob guard sufficed for single-put flows; multi-put batches + (manifest writes are exactly this) will want the scope form. +- The sweep-vs-put race: the POC's pins are a `parking_lot::Mutex` + map, and sweep reads it under the lock before listing — for a real + concurrent sweep (`tokio` task) the race window (a pin added between + "list" and "delete") is real; iroh solves it with their DeleteSet + transactions (`ProtectHandle`/protect-cancel). **Recorded as an open + implementation requirement for Phase 1**, not solved by this POC. + +### Finding 4: git-blob-sha-256 works — byte-exact against the git CLI + +`Digest::git_sha256(content)` (preamble `blob \0` + content, +SHA-256) matched `git hash-object --stdin --object-format=sha256` for +every test input: empty, short text, binary-with-NULs, 75 KB. Same for +`git-sha1` (git 2.43's default) at up to 130 KB. The domain-separated +preamble abstraction (an algorithm defines its own input preprocessing) +was sufficient — no `fn(content) → digest` bare-func limitation was +felt anywhere. Note the empirical gotcha: `--object-format=sha256` is +*not* a `git hash-object` flag in git 2.43 — it must go to +`git init --object-format=sha256` and the hash-object then runs inside +that repo (test harness detail, but cost an iteration to learn). + +**Corollary validated (`store_addresses_git_oids_directly` test):** an +oid produced by *external git* addresses the same pool entry our store +puts — `Digest::from_git_oid_hex(...)` round-trips into the store, the +put dedups against it, and the get returns the content. This is the +OQ-BL-03 "no mapping layer" resolution confirmed at the implementation +level, not just on paper. + +### Finding 5: tagged key format — one byte of tag + algorithm discriminant + +The POC's key encoding: `[tag=0x01][algorithm byte][digest bytes]`, +total 1+1+len. Notes for the real design: + +- The single tag byte is probably needless in the real crate (there is + currently exactly one key *kind*); the **algorithm byte is the + load-bearing part** — it makes `git-sha256` and `git-sha1` keys + disjoint in the same flat pool with zero mapping. Test + `key_format_is_tagged_and_length_checked` validated + length-checking rejects malformed keys (truncated, unknown + algorithm). +- Alternative considered but not built: variable-length + length-prefixed keys. With only two algorithms, the enum-tagged + fixed-length form is simpler and total-order stable (both key kinds + sort cleanly in a BTreeMap — required for the sweep's live-set + diffs). +- This encoding is inside the one-way door OQ-BL-03 flags; whatever + the real crate picks, **algorithm identity must be part of the key + itself** (a per-store single-algorithm config would not have + survived the finding-4 corollary: external-git oids and pooled + workspaces live in one table). + +### Finding 6: dual-dispatch works as simply as hoped + +`DualDispatch` (kv small / fs large, size threshold): routing by +`value.len()`, fall-through get (small miss → large), the +wrong-backend-miss case, and per-backend `backend_of()`. The appfile +dispatch is genuinely a thin layer over the trait — no surprises. One +structural note: since put routes by size, a *re-put* of the same +content will always route to the same backend (same length), so +dispatch is deterministic for a given digest; the "blob migrates +between backends" question (phase-0 OQ-BL-02) remains open but is +independent of the trait shape — nothing in the built contract +prevents a migrate-between-backends sweep later. + +The `list()` union across backends (dedup on key) passed with both +backends disjoint — completing the contract requirement that GC's +sweep sees the whole pool. + +### Finding 7: namespace tables + protect callback — the snapshot bug that proves the seam + +The `NamespaceTables` (`src/namespace.rs`): reference sets keyed by +namespace name, installed into a store as a protect callback. The first +implementation **cloned the tables into the callback** (a snapshot) — +registering roots *after* install was invisible to GC, and liveness +broke. The fix (share the `Arc>` handle, not a snapshot) is +trivial but the failure mode is the important finding: **the protect +callback must be live-shared, and consumers will trip on this if the +real crate's API implies "install" copies state.** API-shape lesson +for Phase 1: call the seam `register_liveness_source` (or have +`NamespaceTables` *hold* the store reference and register itself) +rather than `install_into` — the "install" verb implied copy +semantics that broke exactly like the test showed. + +With shared handles, the namespace tests pass cleanly: dropping a +namespace leaves the other namespace's blobs alive, the shared blob +survives both drops, and delete-then-recover re-puts +byte-identically — the OQ-BL-05 mechanics, validated in miniature. + +### Finding 8: GC semantics — sweep counts are exact, and liveness composition works + +The sweep composes liveness from three sources (pins, protect +callbacks, and the "nothing live" case) and the tests verify exact +deletion counts. The `protect_callback_can_abort_sweep` test validates +the iroh-blobs `ProtectOutcome::Abort` conclusion — a protection- +source failure aborts the run (typed error), nothing deleted. + +What the POC did not test (honest scope limits, all Phase 1 work): +concurrent sweep-vs-put (see finding 3), incremental/interval +scheduling (the POC sweeps are explicit calls), and cross-hash +registration (deferred by OQ-BL-03 pending any consumer asking). + +## Consequences for the phase-0 OQ register + +- **OQ-BL-02 (dispatch):** the lean `Backend` trait contract with + complete `list()` + opaque keys is validated as the base. Remaining: + migration-between-backends policy and (maybe) per-namespace backend + config — both above-trait concerns, no further POC gate. +- **OQ-BL-03 (hash):** the canonical git-family resolution is now + implementation-validated, including external-git oid interop. + Residual: only key-encoding mechanics for Phase 1 (finding 5 notes). +- **OQ-BL-05 (pooling/GC):** mark-and-sweep-with-protect-callback + validated as the mechanism; namespace-as-root-tables works. Phase 1 + ADR inputs now include: the callback-must-be-live-shared lesson + (finding 7), the sweep-vs-put race requirement from finding 3, and + the pack tension (unchanged — rides POC #3's reading). +- **POC #4's GC worked example is effectively done in miniature** by + this POC (namespace tables + sweep honored + recover semantics). + The register should mark #4 as covered by #1 with the concurrency + half still open as Phase 1 implementation work (not a POC gate). + +## POC quality notes + +- 17 tests total (5 lib-level, 12 integration), all passing; digest + vectors cross-checked against the real `git` CLI (2.43.0), + **not just hardcoded** — though the hardcoded sha256 vectors were + themselves initially wrong (transcription) and the CLI check is what + actually proved the implementation; a good reminder that known- + answer tests are only as trustworthy as their source of truth. +- clippy `-D warnings` clean, `rustfmt` clean. +- Deliberate scope limits: no async I/O in backends (sync under + `block_in_place` for the kv; fs is sync std), no streaming put/get + (the POC's `Vec` API is the *contract* question, not the + performance question — streaming shape rides POC #3's findings), + no persistence tests across process restarts (redb/fs both durable + engines; nothing store-layer-specific to prove). \ No newline at end of file