glm-5.3-flash
abf8c5ae9c
ADR-010 (new): the Backend trait's I/O seams pinned before any code — 'get' returns a crate-internal read cursor (kv engines materialize; 'local' pread loop; pg-lo lo_get windows), 'put' has two named forms (whole-value kv / staged-put large); GC state is store-core-owned via a crate-internal contract-tested engine-state seam on the SQL-backed engines (backends never learn liveness — ADR-005 verbatim); the store core holds the joint entry+pin transaction (corrects ADR-009 §2's ownership statement); kv-tier fleet-validity rule added (mirror of ADR-008 §3); ops pin-token TTL race resolved (the token IS the pin; renewal rides have/re-put; pause-past-TTL falls to delete-then-recover); feature graph pinned (pg-lo does not imply postgres). ADR-011 (new): vocabulary pinned — tier (contract, exactly two) vs engine (concrete impl, exactly five); store instance / node / fleet split (fleet = pool-sharing, not node count — fixes requirements.md's self-contradiction with REQ-2); mem demoted from 'backend'/'testing tier' to the kv tier's third engine (contract-reference engine); fs tier renamed 'large' including the feature name (last free moment before code exists); constructor modes pinned (dual-tier default, kv-only, mem-only) plus the composite fleet-validity predicate. Consistency round across all specs and ADR-003/004/007/008/009: stale pg-lo passages resolved per ADR-009's supersession; ADR-008 owner 'node id' → store-instance id; threshold wording corrected (conservative edge, not midpoint, of the 128-256 KiB crossover zone); SweepReport shape specced; mem classification unified; 008/009 ADR files renamed to match the tier rename; README deferral-policy recap aligned (third category = decided-but-sequenced work, not a parking kind); POC crate list completed. Verification: two independent architecture review rounds (the first found 4 criticals — unpinned trait I/O shapes, missing fleet-state seam, node/fleet self-contradiction, pin-token/TTL conflict — all resolved; final round: zero criticals); all markdown links resolve; ADR tables complete (11 ADRs).
125 lines
5.5 KiB
Markdown
125 lines
5.5 KiB
Markdown
# ADR-002: Canonical hash and key encoding
|
|
|
|
## Status
|
|
|
|
Accepted
|
|
|
|
## Context
|
|
|
|
The store keys every entry under a content hash. Phase 0 (OQ-BL-03)
|
|
resolved the hash *policy* — canonical git-blob-sha-256, tolerated
|
|
git-blob-sha-1, BLAKE3 demoted — after the inherited premise "iroh uses
|
|
BLAKE3, so we must" was rejected (convention 7: borrow conclusions, not
|
|
welds; nothing in the consumer set forces BLAKE3: alkgit forces git's
|
|
derivation by protocol definition, and every other consumer sees
|
|
addresses, not algorithms).
|
|
|
|
The residual was the **key encoding mechanics** — the byte layout of
|
|
keys inside backends. This is the crate's declared one-way door
|
|
(AGENTS convention 5): key bytes persist in kv rows and `local`-engine filenames,
|
|
with no schema-migration story; and this is the wire-format ADR that
|
|
must exist before the first consumer (convention 7's gate on publishing
|
|
wire-relevant decisions). POC #1 validated the mechanics empirically:
|
|
byte-exact `git hash-object` interop (finding 4, both algorithms,
|
|
external-oid round-trip into the pool), and tagged-key coexistence of
|
|
both algorithms in one flat pool (finding 5).
|
|
|
|
## Decision
|
|
|
|
### Hash policy (three classes)
|
|
|
|
1. **Canonical — `git-blob-sha-256`:** `H("blob <len>\0" + content)`,
|
|
SHA-256. The store's canonical algorithm; git objects are first-class
|
|
entries with zero indirection; one algorithm domain = one address
|
|
space, so non-git consumers dedup into the same pool by construction
|
|
(ADR-005's premise).
|
|
2. **Tolerated legacy — `git-blob-sha-1`:** same derivation, SHA-1 —
|
|
accepted for existing-repo interop (a protocol necessity; git's own
|
|
collision-hardened SHA-1 threat model inherited verbatim).
|
|
3. **Excluded — BLAKE3:** not present in this crate. It was an
|
|
iroh-blobs inheritance, not a requirement (its virtues — merkle-native
|
|
trees, raw throughput — are irrelevant here under the canonical
|
|
digest at our scales). If a chunk-tree transfer encoding is ever
|
|
adopted it is a *new* decision at a *new* layer (ADR-006 excludes it
|
|
from this crate's scope by scoping, not by hedge).
|
|
|
|
Security footnote (carried from OQ-BL-03): one algorithm across
|
|
consumers is safe here — SHA-256 has no practical cross-protocol
|
|
ambiguity at these input shapes; the preamble domain-separates the
|
|
derivation. SHA-1's caveat is git's accepted position, inherited.
|
|
|
|
### Key encoding (the one-way door's bytes)
|
|
|
|
Physical key layout inside all backends:
|
|
|
|
```
|
|
key := <algorithm byte> <digest bytes>
|
|
algorithm byte: 0x01 = git-blob-sha-256 (32-byte digest)
|
|
0x02 = git-blob-sha-1 (20-byte digest)
|
|
```
|
|
|
|
- **Algorithm identity is inside the key.** A per-store
|
|
single-algorithm configuration would not have survived POC #1's
|
|
external-oid interop test — external git oids and pooled workspace
|
|
content must be addressable from the same table.
|
|
- **The tag byte is dropped** (POC finding 5: exactly one key kind
|
|
exists; the algorithm byte is the load-bearing part). Honesty note:
|
|
the POC validated the *tagged* layout `[tag][algorithm][digest]`;
|
|
dropping the single tag byte is the decided simplification of that
|
|
layout (the finding's own reasoning — one key kind — is the evidence
|
|
the simplification rests on; the algorithm byte, the load-bearing
|
|
half, is the tested part).
|
|
- **Fixed length per algorithm** (32 or 20 bytes); total key lengths
|
|
33/21. Fixed-length enum-tagged keys sort cleanly in total order —
|
|
required for the sweep's live-set diffs (BTreeMap/range-scan walks).
|
|
- **Unknown-algorithm and truncated keys are rejects**, not guesses
|
|
(POC's length-checking test is the standard).
|
|
- **Backends are opaque to all of this:** they store `&[u8]` keys; only
|
|
the store core constructs/interprets typed keys (ADR-003).
|
|
|
|
Algorithm bytes are assigned from this ADR's registry; new algorithms
|
|
require a new ADR (another one-way-door byte).
|
|
|
|
### The preamble as abstraction
|
|
|
|
An algorithm = domain-separated derivation (its own input
|
|
preprocessing) + digest. The preamble `"blob <len>\0"` is not a
|
|
special case — POC #1's domain-separated-premise abstraction held with
|
|
no special-casing anywhere. Note the structural consequence: the length
|
|
is *inside* the hashed input, which is what forces the two put paths
|
|
(ADR-003) and caps verification at whole-blob (ADR-006) — both accepted
|
|
rather than worked around.
|
|
|
|
## Consequences
|
|
|
|
**Positive**
|
|
|
|
- Git oids (either format) address pool entries directly from external
|
|
producers — no mapping layer, byte-exact (tested against the git CLI,
|
|
not just vectors).
|
|
- One address space across all consumers; dedup is structural.
|
|
- Key bytes are self-describing; backend data survives digest-layer
|
|
evolution (opaque at the boundary).
|
|
|
|
**Negative**
|
|
|
|
- The encoding is frozen once a backend ships data (the door closes);
|
|
algorithm byte 0x00 is reserved and never assigned (a future encoding
|
|
revision would need a different key universe or a migration ADR).
|
|
- SHA-1 acceptance inherits git's threat model — documented, not
|
|
re-litigated here.
|
|
|
|
**Neutral**
|
|
|
|
- Variable-length length-prefixed keys were considered (POC finding 5)
|
|
and rejected for now: two algorithms make fixed-length simpler and
|
|
sort-stable. Extension to a third algorithm re-opens byte layout —
|
|
new ADR.
|
|
|
|
## References
|
|
|
|
- `docs/research/phase-0.md` OQ-BL-03 (resolution + encoding notes)
|
|
- `docs/research/poc-trait-dispatch-findings.md` findings 4/5
|
|
- [hashing-and-keys.md](../hashing-and-keys.md)
|
|
- ADR-003 (opaque byte keys; two put paths the preamble forces);
|
|
ADR-005 (one address space); ADR-006 (whole-blob verification) |