alkblobs/docs/architecture/decisions/002-canonical-hash-and-key-encoding.md

# ADR-002: Canonical hash and key encoding

## Status

Accepted

## Context

The store keys every entry under a content hash. Phase 0 (OQ-BL-03)
resolved the hash *policy* — canonical git-blob-sha-256, tolerated
git-blob-sha-1, BLAKE3 demoted — after the inherited premise "iroh uses
BLAKE3, so we must" was rejected (convention 7: borrow conclusions, not
welds; nothing in the consumer set forces BLAKE3: alkgit forces git's
derivation by protocol definition, and every other consumer sees
addresses, not algorithms).

The residual was the **key encoding mechanics** — the byte layout of
keys inside backends. This is the crate's declared one-way door
(AGENTS convention 5): key bytes persist in kv rows and fs filenames,
with no schema-migration story; and this is the wire-format ADR that
must exist before the first consumer (convention 7's gate on publishing
wire-relevant decisions). POC #1 validated the mechanics empirically:
byte-exact `git hash-object` interop (finding 4, both algorithms,
external-oid round-trip into the pool), and tagged-key coexistence of
both algorithms in one flat pool (finding 5).

## Decision

### Hash policy (three tiers)

1. **Canonical — `git-blob-sha-256`:** `H("blob <len>\0" + content)`,
   SHA-256. The store's canonical algorithm; git objects are first-class
   entries with zero indirection; one algorithm domain = one address
   space, so non-git consumers dedup into the same pool by construction
   (ADR-005's premise).
2. **Tolerated legacy — `git-blob-sha-1`:** same derivation, SHA-1 —
   accepted for existing-repo interop (a protocol necessity; git's own
   collision-hardened SHA-1 threat model inherited verbatim).
3. **Excluded — BLAKE3:** not present in this crate. It was an
   iroh-blobs inheritance, not a requirement (its virtues — merkle-native
   trees, raw throughput — are irrelevant here under the canonical
   digest at our scales). If a chunk-tree transfer encoding is ever
   adopted it is a *new* decision at a *new* layer (ADR-006 excludes it
   from this crate's scope by scoping, not by hedge).

Security footnote (carried from OQ-BL-03): one algorithm across
consumers is safe here — SHA-256 has no practical cross-protocol
ambiguity at these input shapes; the preamble domain-separates the
derivation. SHA-1's caveat is git's accepted position, inherited.

### Key encoding (the one-way door's bytes)

Physical key layout inside all backends:

```
key := <algorithm byte> <digest bytes>
algorithm byte: 0x01 = git-blob-sha-256 (32-byte digest)
                0x02 = git-blob-sha-1   (20-byte digest)
```

- **Algorithm identity is inside the key.** A per-store
  single-algorithm configuration would not have survived POC #1's
  external-oid interop test — external git oids and pooled workspace
  content must be addressable from the same table.
- **The tag byte is dropped** (POC finding 5: exactly one key kind
  exists; the algorithm byte is the load-bearing part). Honesty note:
  the POC validated the *tagged* layout `[tag][algorithm][digest]`;
  dropping the single tag byte is the decided simplification of that
  layout (the finding's own reasoning — one key kind — is the evidence
  the simplification rests on; the algorithm byte, the load-bearing
  half, is the tested part).
- **Fixed length per algorithm** (32 or 20 bytes); total key lengths
  33/21. Fixed-length enum-tagged keys sort cleanly in total order —
  required for the sweep's live-set diffs (BTreeMap/range-scan walks).
- **Unknown-algorithm and truncated keys are rejects**, not guesses
  (POC's length-checking test is the standard).
- **Backends are opaque to all of this:** they store `&[u8]` keys; only
  the store core constructs/interprets typed keys (ADR-003).

Algorithm bytes are assigned from this ADR's registry; new algorithms
require a new ADR (another one-way-door byte).

### The preamble as abstraction

An algorithm = domain-separated derivation (its own input
preprocessing) + digest. The preamble `"blob <len>\0"` is not a
special case — POC #1's domain-separated-premise abstraction held with
no special-casing anywhere. Note the structural consequence: the length
is *inside* the hashed input, which is what forces the two put paths
(ADR-003) and caps verification at whole-blob (ADR-006) — both accepted
rather than worked around.

## Consequences

**Positive**

- Git oids (either format) address pool entries directly from external
  producers — no mapping layer, byte-exact (tested against the git CLI,
  not just vectors).
- One address space across all consumers; dedup is structural.
- Key bytes are self-describing; backend data survives digest-layer
  evolution (opaque at the boundary).

**Negative**

- The encoding is frozen once a backend ships data (the door closes);
  algorithm byte 0x00 is reserved and never assigned (a future encoding
  revision would need a different key universe or a migration ADR).
- SHA-1 acceptance inherits git's threat model — documented, not
  re-litigated here.

**Neutral**

- Variable-length length-prefixed keys were considered (POC finding 5)
  and rejected for now: two algorithms make fixed-length simpler and
  sort-stable. Extension to a third algorithm re-opens byte layout —
  new ADR.

## References

- `docs/research/phase-0.md` OQ-BL-03 (resolution + encoding notes)
- `docs/research/poc-trait-dispatch-findings.md` findings 4/5
- [hashing-and-keys.md](../hashing-and-keys.md)
- ADR-003 (opaque byte keys; two put paths the preamble forces);
  ADR-005 (one address space); ADR-006 (whole-blob verification)