docs(architecture): Phase 1 spec tree — 7 specs, ADRs 001-006, OQ tracker
- overview/hashing/store-api/backends/gc/ops specs + README index, all draft status with YAML frontmatter and cross-referenced ADR/OQ - ADR-001: substrate posture (store layer alkcall-free = family conformance, not deviation) + ops placement decided (feature-gated module, no consumer-waited deferral) - ADR-002: canonical git-blob-sha-256 + key encoding — the declared wire-format ADR (one-way door) - ADR-003: backend contract, sqlite/fs/mem tiers, pure-function dispatch, no migration, unknown-length buffering semantics pinned - ADR-004: exactly two production backends; manifest/path-tree layers are consumer-side (the corrected two-vs-three-backends framing) - ADR-005: pooled CAS, three liveness sources, delete windows with pin-before-publish + delete-time arbitration (race closed), no ambient scheduling - ADR-006: whole-blob verification; chunk-tree encodings excluded by scoping, not deferred on a paused consumer - open-questions.md: OQ-BL-01..06 resolved with ADR cross-refs; OQ-07 /08 externally-owned, OQ-09 deferred(scope) with SDD tracker task - AGENTS.md: status updated to Phase 1, gix-odb path corrected Verification: cargo test / clippy -D warnings / fmt --check clean
This commit is contained in:
1 parent
7d9ac8a6ec
commit
4b5009d85a
16 files changed
+1953
-8
No files matched your search
@@ -0,0 +1,125 @@
|
||||
# ADR-002: Canonical hash and key encoding
|
||||
|
||||
## Status
|
||||
|
||||
Accepted
|
||||
|
||||
## Context
|
||||
|
||||
The store keys every entry under a content hash. Phase 0 (OQ-BL-03)
|
||||
resolved the hash *policy* — canonical git-blob-sha-256, tolerated
|
||||
git-blob-sha-1, BLAKE3 demoted — after the inherited premise "iroh uses
|
||||
BLAKE3, so we must" was rejected (convention 7: borrow conclusions, not
|
||||
welds; nothing in the consumer set forces BLAKE3: alkgit forces git's
|
||||
derivation by protocol definition, and every other consumer sees
|
||||
addresses, not algorithms).
|
||||
|
||||
The residual was the **key encoding mechanics** — the byte layout of
|
||||
keys inside backends. This is the crate's declared one-way door
|
||||
(AGENTS convention 5): key bytes persist in kv rows and fs filenames,
|
||||
with no schema-migration story; and this is the wire-format ADR that
|
||||
must exist before the first consumer (convention 7's gate on publishing
|
||||
wire-relevant decisions). POC #1 validated the mechanics empirically:
|
||||
byte-exact `git hash-object` interop (finding 4, both algorithms,
|
||||
external-oid round-trip into the pool), and tagged-key coexistence of
|
||||
both algorithms in one flat pool (finding 5).
|
||||
|
||||
## Decision
|
||||
|
||||
### Hash policy (three tiers)
|
||||
|
||||
1. **Canonical — `git-blob-sha-256`:** `H("blob <len>\0" + content)`,
|
||||
SHA-256. The store's canonical algorithm; git objects are first-class
|
||||
entries with zero indirection; one algorithm domain = one address
|
||||
space, so non-git consumers dedup into the same pool by construction
|
||||
(ADR-005's premise).
|
||||
2. **Tolerated legacy — `git-blob-sha-1`:** same derivation, SHA-1 —
|
||||
accepted for existing-repo interop (a protocol necessity; git's own
|
||||
collision-hardened SHA-1 threat model inherited verbatim).
|
||||
3. **Excluded — BLAKE3:** not present in this crate. It was an
|
||||
iroh-blobs inheritance, not a requirement (its virtues — merkle-native
|
||||
trees, raw throughput — are irrelevant here under the canonical
|
||||
digest at our scales). If a chunk-tree transfer encoding is ever
|
||||
adopted it is a *new* decision at a *new* layer (ADR-006 excludes it
|
||||
from this crate's scope by scoping, not by hedge).
|
||||
|
||||
Security footnote (carried from OQ-BL-03): one algorithm across
|
||||
consumers is safe here — SHA-256 has no practical cross-protocol
|
||||
ambiguity at these input shapes; the preamble domain-separates the
|
||||
derivation. SHA-1's caveat is git's accepted position, inherited.
|
||||
|
||||
### Key encoding (the one-way door's bytes)
|
||||
|
||||
Physical key layout inside all backends:
|
||||
|
||||
```
|
||||
key := <algorithm byte> <digest bytes>
|
||||
algorithm byte: 0x01 = git-blob-sha-256 (32-byte digest)
|
||||
0x02 = git-blob-sha-1 (20-byte digest)
|
||||
```
|
||||
|
||||
- **Algorithm identity is inside the key.** A per-store
|
||||
single-algorithm configuration would not have survived POC #1's
|
||||
external-oid interop test — external git oids and pooled workspace
|
||||
content must be addressable from the same table.
|
||||
- **The tag byte is dropped** (POC finding 5: exactly one key kind
|
||||
exists; the algorithm byte is the load-bearing part). Honesty note:
|
||||
the POC validated the *tagged* layout `[tag][algorithm][digest]`;
|
||||
dropping the single tag byte is the decided simplification of that
|
||||
layout (the finding's own reasoning — one key kind — is the evidence
|
||||
the simplification rests on; the algorithm byte, the load-bearing
|
||||
half, is the tested part).
|
||||
- **Fixed length per algorithm** (32 or 20 bytes); total key lengths
|
||||
33/21. Fixed-length enum-tagged keys sort cleanly in total order —
|
||||
required for the sweep's live-set diffs (BTreeMap/range-scan walks).
|
||||
- **Unknown-algorithm and truncated keys are rejects**, not guesses
|
||||
(POC's length-checking test is the standard).
|
||||
- **Backends are opaque to all of this:** they store `&[u8]` keys; only
|
||||
the store core constructs/interprets typed keys (ADR-003).
|
||||
|
||||
Algorithm bytes are assigned from this ADR's registry; new algorithms
|
||||
require a new ADR (another one-way-door byte).
|
||||
|
||||
### The preamble as abstraction
|
||||
|
||||
An algorithm = domain-separated derivation (its own input
|
||||
preprocessing) + digest. The preamble `"blob <len>\0"` is not a
|
||||
special case — POC #1's domain-separated-premise abstraction held with
|
||||
no special-casing anywhere. Note the structural consequence: the length
|
||||
is *inside* the hashed input, which is what forces the two put paths
|
||||
(ADR-003) and caps verification at whole-blob (ADR-006) — both accepted
|
||||
rather than worked around.
|
||||
|
||||
## Consequences
|
||||
|
||||
**Positive**
|
||||
|
||||
- Git oids (either format) address pool entries directly from external
|
||||
producers — no mapping layer, byte-exact (tested against the git CLI,
|
||||
not just vectors).
|
||||
- One address space across all consumers; dedup is structural.
|
||||
- Key bytes are self-describing; backend data survives digest-layer
|
||||
evolution (opaque at the boundary).
|
||||
|
||||
**Negative**
|
||||
|
||||
- The encoding is frozen once a backend ships data (the door closes);
|
||||
algorithm byte 0x00 is reserved and never assigned (a future encoding
|
||||
revision would need a different key universe or a migration ADR).
|
||||
- SHA-1 acceptance inherits git's threat model — documented, not
|
||||
re-litigated here.
|
||||
|
||||
**Neutral**
|
||||
|
||||
- Variable-length length-prefixed keys were considered (POC finding 5)
|
||||
and rejected for now: two algorithms make fixed-length simpler and
|
||||
sort-stable. Extension to a third algorithm re-opens byte layout —
|
||||
new ADR.
|
||||
|
||||
## References
|
||||
|
||||
- `docs/research/phase-0.md` OQ-BL-03 (resolution + encoding notes)
|
||||
- `docs/research/poc-trait-dispatch-findings.md` findings 4/5
|
||||
- [hashing-and-keys.md](../hashing-and-keys.md)
|
||||
- ADR-003 (opaque byte keys; two put paths the preamble forces);
|
||||
ADR-005 (one address space); ADR-006 (whole-blob verification)
|
||||
Reference in new issue
Block a user