alkgit/docs/architecture/backend.md

status, last_updated

status	last_updated
stable	2026-10-23 (the wave-5 release gate — the trait family and the seam types are the core's published surface; the implementation track composes against it)

Backend traits: the storage seam

Re-baselined 2026-10-19 (ADR-019..023, the substrate round): the trait family and its pinned signatures stand unchanged; the implementation story beneath them is now substrate-first (alkstore + alkblobs — substrate.md). registry-file and the on-disk odb engine are gone; the feature model and the record schema changed accordingly. The seam decisions live in the ADRs; this doc is the decomposer-facing spec.

What this is

The payload side of the protocol crate: the traits a service deployer implements (or consumes via alkgit-substrate — the shipped composition, ADR-025) so the git protocol can talk to storage. Per ADR-010 this mirrors alktty's TtyBackend pattern — traits in the protocol crate, implementations excluded from it (now structurally: the composition is a sibling crate, ADR-025). Unlike alktunnels (no backend trait), git needs the seam: pack generation/ingestion is too heavy to hard-wire.

The trait family (ADR-010 sub-decision 4, ADR-011/012, ADR-018)

Five traits, kept small and orthogonal — no substrate-crate types appear anywhere in the protocol core, the traits, or the seam types; gix component types appear only at/above the seam (gix_hash::ObjectId in the seam signatures — ADR-018) and in the wire layer's toolkit (gix-protocol/packetline):

1. GitRegistry — repo id → RepoRecord (visibility, grants keyed on the stable logical identity id, action set {read, write, manage} — ADR-011, ADR-015). Async resolve; the authoritative mapping (ADR-008); resolution failure is indistinguishable from authorization failure (ADR-007). Authorization is alkgit-core's policy function evaluated on the record (data in the backend, policy in core — an embedder with a foreign permission system maps its ACL into grants at resolve time).
2. GitRegistryStore: GitRegistry — the write supertrait (put_repo/update_repo/remove_repo; alknet ADR-035's read/write split shape). The git/repo/* ops wrap it (ADR-012 §3). Grants ride the record, so grant mutation is update_repo (convenience wrappers additive later, two-way) — which is also the app's promotion path (roles/teams compile to flat grants upstream; ADR-015 §6) and the shape a future replication sync would write through (ADR-015 §8).
3. GitRefs — listing for advertisement (refs + peeled tags + symref targets, the ls-refs response data) and ref transactions (CAS apply for receive-pack, name validation per git ref rules + reserved-namespace deny-list).
4. GitPackGen — (&RepoRecord, wants, boundary_haves, limits, sink: Box<dyn io::Write + Send>) → streaming pack (names per the pinned signature below — ADR-018 §4), plus common_haves(repo, haves) -> recognized subset for the negotiation ack loop (a have is recognized iff it exists in the object store and is a commit — the honest boundary: never ack what we cannot subtract; ADR-014). Negotiation-agnostic generation. Missing objects abort with an error, never a broken pack (ADR-004).
5. GitPackIngest — client pack stream → validated updates + ingested objects. Thin-pack bases resolve from the object store (push sends thin packs by default — ADR-013 §5); objects land under pin-bridge protection until the ref CAS lands (ADR-020 §2 — the substrate composition; an impl without pins owns a different landing story). It prepares the validated ref updates; the transaction itself is applied by GitRefs (single CAS home — ingest validates, refs commits; one transaction per push is what makes atomic correct — ADR-013 §7). The prepare binding carries push_options: Option<PushOptions> (owned, the ADR-018 §4 pinned shape — ADR-013 §11, review 001 N-4): the parsed per-push metadata, present only when push-options was negotiated — None until the config gate opens; alkgit parses and forwards the option pairs verbatim, never interprets them. Budgeted (ADR-009 max pack size); the impl runs its blocking work off the async executor (concurrency model below).

Minimal-vs-full was the open sub-question; resolved as full family (the collapse-cost rationale lives with ADR-018's review trail) — each trait is one or two methods plus types, and the seams are kept independent so an implementer satisfies only what it has. alkgit-substrate implements all five (ADR-019/025); a downstream with foreign storage implements any subset and consumes only the protocol core itself (depends on alkgit, which structurally carries no substrate — ADR-025).

Pinned signatures (ADR-018; the freeze-inventory shapes)

Unchanged by the re-baseline — POC-O and POC-X consumed exactly these shapes with zero signature drift (ADR-019 §9). The registry pair is ADR-012 §1 as amended. The object-storage family (ADR-018 §4 — pinned there; this section is the mirror the decomposer implements from):

• GitRefs: list_refs(&RepoRecord) -> Vec<RefLine> (the full set — ref-prefix filtering is wire-side, client-driven per ls-refs grammar) and apply_updates(&RepoRecord, Vec<RefUpdate>, atomic: bool) -> Vec<RefOutcome> (one CAS transaction per call — the atomic-correctness home, ADR-013 §7; realized as the one-transaction row-level CAS in the substrate impl, ADR-019 §2).
• GitPackGen: generate(&RepoRecord, wants, boundary_haves, limits, sink: Box<dyn io::Write + Send>) (ADR-004's pipeline) and common_haves(&RepoRecord, haves) -> Vec<ObjectId> (the ADR-014 ack/done-round seam: a have is recognized iff it exists and is a commit — one seam predicate applied at both points, the ack rounds and the done-round boundary alike; the odb-free wire layer refines nothing, ADR-010 — the earlier "the is-commit refinement is the wire layer's ACK-line rule" wording was drift, fixed by the wave-2 gate). The replication have-aware trim rides the haves plumbing (ADR-023).
• GitPackIngest: prepare(&RepoRecord, updates: Vec<RefUpdate>, pack: Box<dyn io::Read + Send>, push_options: Option<PushOptions>, limits) -> PreparedPush (prepare-only; the CAS application lives in GitRefs::apply_updates — ADR-013 §6–7). The updates are the command-derived ref updates the wire layer parsed (expected = old, new = new); the impl validates them — the connectivity walk from the new tips (ADR-020 §2) — and returns them verbatim as PreparedPush.updates (verbatim round-trip in v1, no impl-side filtering — name validation and CAS stay wire-side/backend-CAS-side; the wave-3 prepare-seam amendment).
• The repo parameter is &RepoRecord (ADR-018 §2): the wire layer resolves once (ADR-007 step 2) and the impls key their storage on repo_id — refs are rows keyed by it, objects are store entries under its namespace/liveness domain (ADR-019; there is no path in the record — storage_root is gone per ADR-019 §8, and no impl opens anything from it).
• Shared types (ADR-018 §5): RefLine (oid: None = the unborn symref line, the N-5 rider's shape), RefUpdate (expected: None = create, new: None = delete), RefOutcome (the per-ref CAS result — an outcome, not an error; the reason string is the client-displayed ng text), PreparedPush (unpack: Result<(), String> + the updates; pack-level validation failures land as the unpack ng leg, not Err), PushOptions (the verbatim (key, value) pairs — ADR-013 §11 / N-4's additive pin).
• Object ids are gix_hash::ObjectId — always-on in the manifest; hash-format-parameterized internally, so OQ-05's policy change survives without a trait change.
• Errors: traits 1–2 → RegistryError (re-scoped below); traits 3–5 → StorageError (ADR-018 §7 — the missing-object abort, malformed input, and io variants; CAS and fsck outcomes are values, not errors).

Crate and feature model (ADR-025; supersedes ADR-021's graph)

The workspace has two published crates (ADR-025): the protocol core alkgit and the shipped composition alkgit-substrate (the seam above is the contract between them). Engine choice is an assembly-level constructor statement, never a feature (alkstore ADR-016's deployment honesty); features gate implementation code that needs dependencies. The placement table:

Crate :: feature	Gates	Deps	Default
alkgit :: sha256	hash-algorithm passthrough (OQ-05)	gix-hash/sha256	off
alkgit-substrate :: sqlite	the sqlite co-tenant registry/ref adapter (the concrete tables + connections)	alkstore-sqlite	on
alkgit-substrate :: postgres	the postgres twin	alkstore-postgres	off
alkgit-substrate :: lfs	the LFS liveness/pointer-lifecycle surface (ADR-022)	—	on (fail-safe default — the rationale is ADR-022 §6)

• Reactivity and the engine-generic composition are alkgit-substrate's base, not a feature (the placement rule, ADR-025): the reactive halves — announce, wake runner, stream consumer, reconcile — ride the crate's unconditional deps (alkstore core trait + alkblobs; ADR-019 §5's no-non-reactive-shipped-path ruling). An embedder takes reactivity by taking the crate, or implements the seam itself against alkgit without the substrate crate at all — the all-or-nothing posture, now visible in the dependency graph.
• The former substrate/sqlite/postgres features of alkgit are gone (ADR-025's relocation: composition base + engine twin features, both now on alkgit-substrate); the gix engine feature is dissolved (ADR-021 §1, carried): gix component crates are ordinary dependencies of both crates (the pack pipeline, hash, wire toolkit); no on-disk odb engine is shipped.
• The protocol core compiles and tests without any substrate dependency structurally (cargo test -p alkgit; no backend features exist to disable).
• Hash: sha1 pinned (the compile-time-rejected invariant from docs/research/gitoxide.md); sha256 passthrough feature on the core crate (OQ-05 policy unchanged; alkblobs' GitObjectSha256 registry byte is already registered — no renumbering later).
• Encodes the POC-2 prerequisites by construction: handle sharing (the substrate Find front is constructed once, per-session views over it), generation on blocking threads, O(counts) memory, missing-objects abort; ingestion per ADR-020 §2 (unpack-to-loose, pin bridge, owned connectivity walk).
• The published-crates rule: alkblobs 0.2, the alkstore family 0.1, alkcall 0.10, and alkgit itself (in alkgit-substrate's manifest) pinned by version — never path deps to sibling checkouts (the gitoxide reading-clone rule applies).

Concurrency model

• Implementations own their internal threading: the substrate impls run pack generation/ingestion on spawn_blocking with owned handles moved in (POC-2's shape, unchanged in effect). The sync Find / async store bridge is a two-context rule the impls own in one place: Handle::block_on from the blocking pipeline task; never from async contexts (block_in_place there — POC-O learned the panic the live way).
• parking_lot short-held locks; poisoned-lock recovery unwrap_or_else(|e| e.into_inner()) (convention 2). No panics in library code; no unwrap/expect outside tests.
• Trait execution model (review 001 A-6): all five traits are #[async_trait] — Send + Sync, dyn-compatible (E0038 forbids bare async fn), signatures per ADR-012 §1. GitRegistry::resolve is async (one call per session/request, before the first protocol byte — ADR-011 §6); DQ-6 resolved: new traits follow the same posture (ADR-019 §9). alkstore's hand-boxed BoxedFuture posture stays confined to alkstore's surface.
• The wire layer enforces the pipeline-concurrency budget itself (ADR-009's permit acquired around each GitPackGen/GitPackIngest call); the impls' spawn_blocking is invisible from the wire layer.

Public API surface

Per-crate crate-root re-exports (the alktty pattern; ADR-025's partition):

• alkgit: backend traits + types (including RepoRecord, AccessAction, the authorize policy function, the authorized-repo marker type (the door constructs it only after the ADR-007 check passes — session tuples carry it, transport.md session substrate layer), and the git/repo/* op spec+handler pairs), GitAdapter/register_openable (producer), GitSession (consumer — the ADR-017 typed client: ls_refs/fetch/push), session substrate types (ADR-005 — including the service selector and the authorized-repo marker), Limits, protocol error enums; the sha256 feature exports its hash-policy surface.
• alkgit-substrate: the composition family (substrate.md §public API is the member list) — the registry adapter trait pair + engine constructors under sqlite/postgres, the in-memory default adapter, the cached read provider, the reactor (announce/wake-runner/stream-consumer/reconcile), the five backend-trait implementations (the object-side trio included — the assembler constructs the backend from them), the substrate Find/odb front, the GC liveness sources, and the LFS surface under lfs.

The embedder-facing freeze point remains OQ-03 (per-crate after ADR-025's re-partition).

The two op kinds (ADR-012 §3, the classification)

alkgit ships both alkcall op kinds from one crate — the first family payload to do so (alktty/alktunnels: open ops only; alknet-docker, planned: call ops only):

• Open op (channels/git/sub via register_openable; open-op params {repo, service} — ADR-016) — the binary half: negotiation + service selector + ACL point (ADR-007; the service selects the authorize action, read for fetch / write for push), yields the duplex git session.
• Call ops (git/repo/{create,delete,update,get}) — the JSON half: thin OperationSpec+handler pairs over Arc<dyn GitRegistryStore>, Visibility::External, always-on (no substrate needed — the trait pair is the seam). create is gated by the static registry check (required_scopes: ["git:repo:create"]); delete/update/get carry an empty AccessControl and the handler evaluates the ADR-015 gate — scope git:admin OR the record's manage grant, via the authorize policy function, generic FORBIDDEN denial, unknown-repo ≡ unauthorized (ADR-008). Registered by the assembler wherever it wants them exposed. Managing records (op scopes) and git access (repo grants) are distinct sources — but the record is the single per-repo authz surface, and repo creation seeds the creator's {read, write, manage} grants (ADR-015).
• Deletion is a GC consequence (ADR-019 §6): delete removes the registry row; the refs rows are owned by the same removal; the repo's objects become liveness-uncovered and the substrate sweep reclaims them on its next pass. No synchronous object deletion exists.

Registry types and schemas (review 001 N-3; the freeze-inventory draft)

The serde shapes the ops handlers and the registry adapters both serialize — pinned here so the two workstreams cannot invent them independently. These are the N-3 schemas ADR-015 chained after the manage-grant shape: they capture the three-action grant shape (ADR-015), the opaque grant-key rule (ADR-015 §7 / OQ-16), and the unknown-repo ≡ unauthorized rule at the op layer (ADR-008). Draft until first publish (additive fields have been added before publish before — ADR-016's params, ADR-017's API); at publish they enter OQ-03's inventory as compat surface. Schema change this cycle: the storage_root field is gone (ADR-019 §8 — substrate-only storage owns no path; the record is identity/visibility/grants only; ADR-008's never-a-path rule holds a fortiori).

RepoRecord (serde shape)

{
  "repo_id": "alkdev/alkgit",          // the registry id (string, non-empty)
  "visibility": "public",              // "public" | "private" (lowercase)
  "grants": {                          // identity_id (opaque string, ADR-015 §7) → grant set
    "u-7731": ["read", "write", "manage"]
  }
}

• grants values are arrays of the action strings read / write / manage (unique, lowercase; manage implies the lower actions per ADR-015 §2's monotonicity — stored sets need not be closed, the policy function evaluates that).
• No path fields of any kind: the record never carries server-side storage locations (ADR-019 §8). Storage placement is the composition's constructor concern, keyed on repo_id.
• Additive-field rule: unknown fields are rejected, fail-closed, in op input schemas (additionalProperties: false — the same extension rule as ADR-016's open-op params; extensions replace, not accumulate, before publish).

RegistryError (the registry-family error set)

The error type of traits 1–2 (GitRegistry/GitRegistryStore) — the scope is the registry family, not the whole trait family (the object-storage traits carry StorageError, ADR-018 §7, after the "every failure the trait family can produce" claim was found to over-reach — review 002 R-2). Five variants — every failure the registry family can produce, no catch-all:

Variant	Meaning	Wire mapping
NotFound	repo id unresolved	collapses with Forbidden to the generic denial (unknown ≡ unauthorized, ADR-008; transport.md error taxonomy)
Forbidden	gate denied (op-layer or authorize)	generic FORBIDDEN — same wire shape as NotFound, never a variant leak
AlreadyExists	create with a taken id	op-level error; still NOT disclosed as an existence oracle to non-creators — create responses may carry it, resolve never returns it
Invalid	malformed record/field (store-side validation, e.g. empty repo id, unknown action string)	op-level error with the field name
Io(String)	backing-store failure (serialized as message — not std::io::Error, which is not serde-stable)	session/op error, substrate-appropriate

Io is the only stringly-typed variant; everything else is structural so ops can report precise JSON errors ({"error": "already_exists", …}) while the trait surface stays Result<_, RegistryError> per ADR-012 §1.

StorageError (ADR-018 §7) is the object-storage sibling — traits 3–5: ObjectMissing { oid } (the ADR-004 generation abort), Invalid(String) (malformed/backend input), Io(String) (same stability rule as RegistryError::Io). CAS-stale and fsck outcomes are deliberately not variants: they are RefOutcome/PreparedPush.unpack values (protocol reports the session survives), which is what keeps the no-catch-all claim true for both error types.

git/repo/* op JSON schemas (request/response)

Op	Request	Response	Errors
git/repo/create	{repo_id, visibility} — no placement fields (ADR-019 §8; the substrate composition keys storage on repo_id); grants seeded {read, write, manage} for the caller (ADR-015 §4); an explicit grants field is an admin-shape extension, NOT v1	{repo_id, visibility}	already_exists, invalid
git/repo/delete	{repo_id}	{}	unauthorized (the single wire code for unknown-repo ≡ not-authorized — N-2's collapse rule; io)
git/repo/update	{repo_id, visibility?, grants?} — both optional; PATCH semantics (omitted fields unchanged; present fields replaced wholesale — see the update-semantics note below); grant mutation is the only grant-write path (ADR-015 §3)	{repo_id, visibility, grants} (post-write record)	unauthorized (collapsed, N-2), invalid, io
git/repo/get	{repo_id} (single read; list is a v2 additive op)	{repo_id, visibility, grants} (no placement fields — ADR-019 §8)	unauthorized (collapsed, N-2), io

• Error codes are the wire code strings in the table (snake_case, one namespace git:repo:*), each carried in the op's ErrorDefinition (alkcall ADR-016 disclosure convention — the same pattern alktty's channel:open_failed uses). The unauthorized code is the collapsed denial: both RegistryError::NotFound and Forbidden map to it at the op-wire boundary (unknown ≡ unauthorized, ADR-008 + review 001 N-2), so no variant leaks an existence oracle; AlreadyExists is create-side only (the caller knows the id — discloseable without an oracle).
• Update semantics are PATCH (review 002 R-3 — the "full-record replace of the provided fields" qualifier was parseable either way, and the wrong reading is destructive: a visibility-only update clearing grants locks the grant-holder out): omitted fields are left unchanged, present fields are replaced wholesale (grants, when present, replaces the whole grant map — partial grant edits are read-modify-write at the caller). The response echoes the post-write record so the caller confirms what landed, including what was left unchanged.
• already_exists disclosure posture (review 002 R-6, recorded so a later agent does not "fix" it into an upstream-incompatible denial): a git:repo:create-scoped identity probing arbitrary ids can learn which exist — that is accepted. Create is a trusted, scope-gated, low-population surface; upstream gitea behaves the same; the position is that id-probing at create-scope is inside the trust boundary the scope already grants, while resolve-side disclosure (the fetch/push path) stays collapsed per ADR-008/N-2.
• update's response echoes the post-write record so grant edits confirm what landed (the last-writer-race posture is ADR-012 §Consequences, unchanged).
• All four op input schemas: additionalProperties: false (the alksocks {}-precedent — same fail-closed extension rule as ADR-016's open-op params).

Repo-id grammar (review 002 R-5)

repo_id is the registry key on every wire and op surface, so its grammar is pinned once here:

• Shape: one or two path segments, owner/name — lowercase alphanumerics and -/_, segments non-empty and at most 100 bytes total (the owner/name convention matches self-hosted git and the OQ-16 namespacing direction; a single segment is a valid legacy/ local shape).
• Wire handling (ADR-008 governs): the id is an opaque registry key — parsed against this grammar for rejection only, never decomposed (the owner segment confers nothing; the registry index is flat), never joined/normalized into paths.
• Error mapping: a grammar-invalid id is unauthorized on the serving paths (collapsed with unknown-repo, ADR-008 — the id shape leaks nothing) and invalid on the op paths (create/update — the caller needs the correction).
• Http mounting note (for doors.md consumers): two-segment ids need a two-placeholder route (/{owner}/{repo}/info/refs with single-segment ids allowed too) — the door crate pins its own route grammar when the alkhttp git/lfs features land; alkgit's contract is only "the full id string arrives resolution-ready." Route tables on alkhttp door surfaces are static shapes (no route fallback — the POC-L/alkhttp constraint), so multi-segment ids mean enumerated patterns.

Design Decisions

ADR	Decision	Summary
004	Pack pipeline	data::output gen / data::input ingestion
007	ACL first	registry returns rule inputs
008	Repo identity	wire names are registry ids (record carries no path — ADR-019 §8)
009	Budgets	limits flow into gen/ingest
010	Pure protocol crate	producer/consumer halves, no doors/binary; granularity superseded (ADR-025)
011	Per-repo authorization	grants in records, policy in core (amended: ADR-015)
012	Registry + ops	write trait, CRUD ops, feature split (backing default superseded by ADR-019; gate amended: ADR-015)
015	Manage grant + op gate	manage tier, admin-scope-OR-manage gate, create seeds manage
013	receive-pack	thin-pack ingestion, transaction CAS, report framing (amended to the substrate shapes: ADR-020)
014	Negotiation	ack loop, common_haves seam, no ready
016	Native session preamble	{repo, service} open-op params, request-line preamble, service in the tuple
017	Consumer half	GitSession typed client (construction shape simplified: ADR-023 §2)
018	Trait signatures + storage errors	signatures held with zero drift; StorageError for traits 3–5
019	Substrate re-baseline	storage on alkstore + alkblobs; registry-file scrapped; reactive-by-default; coherence contract
020	receive-pack duplex	packless delete pushes (bounded peek); unpack-to-loose ingestion; pin bridge
021	Feature graph	superseded by ADR-025 (graph re-partitioned)
025	Two-crate workspace	crate + feature model (the placement table), the seam as the inter-crate contract
022	LFS	media under GitObjectSha256; alkhttp lfs door; wired liveness
023	Replication v1	protocol-native via GitSession; mirror plane = optimization bucket

Open Questions

• OQ-03: publish freeze inventory (the op set enters it; the revised RepoRecord schema enters it).
• OQ-05: sha256 policy (deferred(scope)).
• OQ-17: relational registry schema migrations — resolved as ADR-024 (construction-run, version-stamped, additive-first; the LFS refcount column ships in the initial schema).
• OQ-19: implementation-phase wire validations (new — the unborn-HEAD rider and the common_haves substrate-odb validation).
• OQ-06 resolved by ADR-012 and now superseded by ADR-019 (registry backing is the substrate composition; registry-file is scrapped).
• OQ-04 resolved (ADR-013, amended: ADR-020); OQ-02 resolved (ADR-014).

References

• substrate.md (the shipped composition this seam fronts — reactive halves, coherence contract, GC chain, adapters)
• lfs.md (the LFS surface)
• docs/research/poc2-findings.md (generation pipeline + prerequisites), docs/research/poco-findings.md, docs/research/pocr-findings.md, docs/research/pocx-findings.md (the re-baseline evidence)
• docs/research/substrate-recomposition.md (the re-baseline doc)
• docs/research/gitoxide.md (API contract notes — normative for the gix component usage)
• alktty backend.rs (the trait template)
• alknet ADR-033/035 (the repo/adapter + read/write-split pattern)
• ADR-010 (the structural decision), ADR-025 (the crate model this doc's feature section implements), ADR-011/012/018 (the seam this doc specifies), ADR-019..023 (the re-baseline round)