alkblobs/docs/architecture/gc-and-namespaces.md

---
status: draft
last_updated: 2026-10-03 (ADR-010 state-home rule + ADR-011 instance/fleet vocabulary)
---

# Pooling, namespaces, and GC

## What this is

The layer that makes the store *pooled*: one content-addressed pool per
node, dedup across every consumer by construction, with garbage
collection that is safe against concurrent writes. The pool property is
the reason both demanding consumers exist (overview.md); GC is its
price, and this document specifies the mechanism the POCs validated
(ADR-005).

## The pooled CAS

- **One pool per deployment; a store instance is one GC arbitration
  domain** (the engine/vocabulary definitions live in requirements.md
  / ADR-011: a node — one process — may host several instances; the
  instance is where pins and arbitration live for single-instance
  topologies). Repos, workspaces, and appfile stores are all
  *sets of hash references* over the same CAS — never walled-off
  per-repo stores. Git's `alternates`/object-pool mechanism (GitLab
  object pools) is the same idea done awkwardly; the pool generalizes it
  to unrelated repos and to p2p replication. **A fleet (REQ-2) — two
  or more store instances sharing one pool — is the one topology where
  the GC mechanism below extends beyond one instance: ADR-008 pins,
  sweeper locking, and delete arbitration move into the shared engine;
  the invariants are unchanged. Where this crate's GC state (fleet
  rows and locks) physically lives, and who holds the joint
  entry+pin transaction, is ruled by ADR-010 §2: the store core owns
  all GC state; SQL-backed engines host it through a crate-internal,
  contract-tested seam; backends never learn liveness.**
- **One address space.** Under the canonical hash (ADR-002), the same
  file content has the same address whether it entered via a git oid or
  a workspace manifest — cross-consumer dedup is free by construction,
  not a feature.
- **Physically flat, logically namespaced** (the rudolfs inversion).
  The byte layer is `hash → bytes`, backends namespace-blind; the
  namespace is a *reference set* above the crate:

## Namespaces (logical, above the pool)

- A namespace is `namespace → set of root hashes` held by the consumer
  (git refs *are* the names; an LFS pointer file in a tree *is* the
  reference; an alkfs path tree *is* the manifest). The store never
  walks consumer-side manifests — it stays structure-blind.
- The store-side counterpart is the **liveness seam**: consumers
  register live-shared liveness sources (ADR-005, §Decision —
  `register_liveness_source`; the POC's copy-semantics failure finding 7
  is the pinning reason). Namespace tables may also exist purely
  in-memory for ephemeral consumers; nothing in the store persists
  namespaces — where roots live when a deployment has no SQL engine (no
  kv engine with rows) is a
  non-question by design: roots live with the consumer, whose references
  are as durable as they need to be.
- ACL mapping for network ops: the namespace is the resource
  (`resource_id_path` selects it) — see ops-surface.md.

## Mark-and-sweep GC (ADR-005)

Validated mechanism (POC #1 findings 3/7/8 — exact sweep counts, abort
semantics, delete-then-recover as byte-identical re-put):

- **Liveness sources (roots):**
  1. consumer-registered liveness sources (live-shared, above),
  2. put-path **pins** (RAII; batch scope available for multi-put
     writes — manifest writes are exactly this),
  3. the **protect callback** consulted before each sweep: it may add
     externally-known hashes or **abort the run** (`GcAborted`; a
     flaky protection source skips the sweep rather than risk
     deletion — iroh's `ProtectOutcome::Abort` conclusion, adopted).
- **Sweep:** enumerate the whole pool (backends' complete `list()`,
  ADR-003), compute the live set, batch-delete the dead
  (batch-sized, ~100/batch, iroh's proven shape). Delete-then-recover
  is a re-put — byte-identical under the same key (validated).
- **Traversal ownership: lean.** Liveness computation beyond "these
  roots exist" is the consumer's job, handed over via the seam. The
  store never learns manifest formats. If a real consumer's live-set
  computation proves too heavy, store-native traversal is a *new* ADR
  (documented constraint, not a parked hedge — the un-pause condition
  is an external fact: a measured consumer cost).

## Delete windows (the sweep-vs-put race)

The race (POC #1 finding 3): a pin added between sweep-"list" and
sweep-"delete" is a lost pin — a blob deleted while in flight. The
architecture resolves it as a **delete window** with three phases —
mark (enumerate + live-set), per-key arbitration under the
pin/liveness lock *at delete time*, batched commit — resting on two
ordering invariants: **put commits pin before publish**, and **deletes
arbitrate under the pin lock at delete time** (full protocol + proof:
ADR-005 §Sweep; iroh `delete_set.rs`'s ProtectHandle/protect-cancel
shape and its serialized-actor pattern are the re-borrowed prior art —
their serialized actor is exactly "arbitrate at delete time"; the
window batches the arbitration).

Test-asserted property: **a visible pool entry is never deleted while
liveness (pin or registered source) protects it, and an in-flight put
is never deleted by a sweep started before it committed.** Direct
`delete(key)` refuses pinned keys (typed error; same arbitration).

`has()`/`get()` during a window observe either the old or the new
state; there is no torn observation (the pool's entries are immutable;
existence flips atomically per key).

## GC scheduling and scope (ADR-005)

- **Explicit `sweep()`; embedder owns cadence.** No background timers
  in the crate. An embedder that wants interval sweeps adds them above
  (one call). Consumer-driven sweeps (after a manifest prune, say) are
  the same API.
- **Sweep scoping:** whole-pool is the base. Namespace-scoped sweeps
  run by computing a narrower live set from that namespace's liveness
  source; the pool remains shared (scoping is a liveness computation
  choice, not a physical partition).
- **Accounting** (per-namespace sizes): computed from reference sets by
  consumers; the store reports pool-level totals only (`list` +
  `stat`). Physical-layer accounting would re-introduce namespace
  physics — rejected with the per-namespace engine config
  (ADR-003 §Consequences).

## Design Decisions

| ADR | Decision | Summary |
|---|---|---|
| [005](decisions/005-pooled-cas-and-gc-mechanism.md) | Pooled CAS & GC | one pool, registered liveness, mark-and-sweep, delete windows, no ambient scheduling |
| [003](decisions/003-backend-contract-and-dispatch.md) | list() contract | complete lists are the GC substrate |
| [004](decisions/004-two-backends-no-third.md) | Structure-blindness | manifests/refs never sink into the store |
| [008](decisions/008-trait-size-probe-fleet-gc-and-large-engines.md) | Fleet GC extension | DB-backed pins, advisory-locked single sweeper, SQL delete arbitration — for shared-pool (fleet) topologies only; the in-process protocol is unchanged elsewhere |
| [010](decisions/010-backend-io-seams-and-fleet-state-home.md) | GC-state home | the store core owns all GC state; SQL engines host it via the contract-tested engine-state seam; pin-tx ownership ruled |

## Open Questions

None owned by this document. Shared-pool GC coordination within a
deployment is now in-crate (fleet mechanism: ADR-008). What remains a
replicator-policy-layer property (above the crate) is *cross-pool*
coordination — gossip sync between separately-pooled replicator nodes
deciding what each keeps — governed by the same seam; it raises a
question here only when a consumer *names a requirement*
(OQ-07/OQ-08 intake), which is re-entry via a new decision at a new
ADR, not a parked question: no decision here waits on a consumer, and
a future requirement does not create one retroactively.

## References

- `docs/research/phase-0.md` OQ-BL-05 (settled decisions + residuals)
- `docs/research/poc-trait-dispatch-findings.md` findings 3/7/8
- `docs/research/iroh-blobs-eval.md` — gc.rs / delete_set.rs reads
  (the re-borrowed conclusions)
- rudolfs — the physical-namespace anti-pattern (inverted)
- gix-odb — alternates/pool prior art for the p2p case
- ADR-005; [store-api.md](store-api.md) (Pin semantics, sweep API);
  [ops-surface.md](ops-surface.md) (namespace-as-resource ACL)