alkgit/docs/architecture/backend.md

status, last_updated

status	last_updated
reviewed	2026-09-30

Backend traits: the storage seam

What this is

The payload side of the protocol crate: the traits a service deployer implements (or consumes via the feature-gated gix implementation) so the git protocol can talk to storage. Per ADR-010 this mirrors alktty's TtyBackend pattern — traits in-crate, real implementation behind a feature. Unlike alktunnels (no backend trait), git needs the seam: pack generation/ingestion is too heavy to hard-wire.

The trait family (ADR-010 sub-decision 4, ADR-011/012)

Five traits, kept small and orthogonal — the protocol crate never sees gix types:

1. GitRegistry — repo id → RepoRecord (storage root, visibility, grants keyed on the stable logical identity id, action set {read, write, manage} — ADR-011, ADR-015). Async resolve; the authoritative mapping (ADR-008); resolution failure is indistinguishable from authorization failure (ADR-007). Authorization is alkgit-core's policy function evaluated on the record (data in the backend, policy in core — an embedder with a foreign permission system maps its ACL into grants at resolve time).
2. GitRegistryStore: GitRegistry — the write supertrait (put_repo/update_repo/remove_repo; alknet ADR-035's read/write split shape). The git/repo/* ops wrap it (ADR-012 §3). Grants ride the record, so grant mutation is update_repo (convenience wrappers additive later, two-way) — which is also the app's promotion path (roles/teams compile to flat grants upstream; ADR-015 §6) and the shape a future replication sync would write through (ADR-015 §8).
3. GitRefs — listing for advertisement (refs + peeled tags + symref targets, the ls-refs response data) and ref transactions (CAS apply for receive-pack, name validation per git ref rules + reserved- namespace deny-list).
4. GitPackGen — (repo, wants, haves, limits) → streaming pack (io::Write consumer), plus common_haves(repo, haves) -> recognized subset for the negotiation ack loop (a have is recognized iff it exists in the object store and is a commit — the honest boundary: never ack what we cannot subtract; ADR-014). Negotiation-agnostic generation. Missing objects abort with an error, never a broken pack (ADR-004).
5. GitPackIngest — client pack stream → indexed pack + fsck/ connectivity report. Thin-pack bases resolve from the server's own odb (push sends thin packs by default — ADR-013 §5); the pack lands with a .keep guard; missing objects → unpack ng. It prepares the validated ref updates; the transaction itself is applied by GitRefs (single CAS home — ingest validates, refs commits; one transaction per push is what makes atomic correct — ADR-013 §7). The prepare binding carries push_options: Option<&PushOptions> (ADR-013 §11, review 001 N-4): the parsed per-push metadata, present only when push-options was negotiated — None until the config gate opens; alkgit parses and forwards the option pairs verbatim, never interprets them (their meaning is the caller's policy domain). Budgeted (ADR-009 max pack size); the impl runs its blocking work off the async executor (concurrency model below).

Minimal-vs-full was the open sub-question; resolved as full family — each trait is one or two methods plus types, and collapsing them (e.g. refs into the registry) would force one impl block per downstream where independent seams are cheaper to satisfy. The gix feature implements the three object-storage traits (3–5: GitRefs, GitPackGen, GitPackIngest); a downstream with its own object store implements 3–5 and reuses 1–2 (GitRegistry/GitRegistryStore), or none of it.

Pinned signatures (ADR-018; the freeze-inventory shapes)

The registry pair is ADR-012 §1 as amended. The object-storage family (ADR-018 §4 — pinned there; this section is the mirror the decomposer implements from):

• GitRefs: list_refs(&RepoRecord) -> Vec<RefLine> (the full set — ref-prefix filtering is wire-side, client-driven per ls-refs grammar) and apply_updates(&RepoRecord, Vec<RefUpdate>, atomic: bool) -> Vec<RefOutcome> (one CAS transaction per call — the atomic-correctness home, ADR-013 §7).
• GitPackGen: generate(&RepoRecord, wants, boundary_haves, limits, sink: Box<dyn io::Write + Send>) (ADR-004's pipeline) and common_haves(&RepoRecord, haves) -> Vec<ObjectId> (the ADR-014 ack/done-round seam; existence is the operative predicate here — the is-commit refinement is the wire layer's ACK-line rule).
• GitPackIngest: prepare(&RepoRecord, pack: Box<dyn io::Read + Send>, push_options: Option<PushOptions>, limits) -> PreparedPush (prepare-only; the CAS application lives in GitRefs::apply_updates — ADR-013 §6–7).
• The repo parameter is &RepoRecord (ADR-018 §2): the wire layer resolves once (ADR-007 step 2) and the record carries storage_root — the gix impl opens the odb from it (gix-discover + gix_odb::Store::at) with no second lookup and no handle type.
• Shared types (ADR-018 §5): RefLine (oid: None = the unborn symref line, the N-5 rider's shape), RefUpdate (expected: None = create, new: None = delete), RefOutcome (the per-ref CAS result — an outcome, not an error; the reason string is the client-displayed ng text), PreparedPush (unpack: Result<(), String> + the updates; pack-level validation failures land as the unpack ng leg, not Err), PushOptions (the verbatim (key, value) pairs — ADR-013 §11 / N-4's additive pin).
• Object ids are gix_hash::ObjectId — always-on in the manifest; hash-format-parameterized internally, so OQ-05's policy change survives without a trait change.
• Errors: traits 1–2 → RegistryError (re-scoped below); traits 3–5 → StorageError (ADR-018 §7 — the missing-object abort, malformed input, and io variants; CAS and fsck outcomes are values, not errors).

Feature model (ADR-012 §4, amends ADR-010's single-gix story)

Two independent seams, two default-on features:

Feature	Contents	gitoxide
gix	GitRefs/GitPackGen/GitPackIngest impls — the local-disk object-storage engine (odb/pack/ref/object/fsck components)	yes
registry-file	GitRegistry/GitRegistryStore default impl (per-repo record files + in-memory index, config-seeded, op-mutable, atomic writes)	no

• default-features = false — wire/protocol layer only (an embedder brings its own backends). Both features default-on (batteries included); they are independent seams — DB-registry + gix-engine and S3-storage + simple-registry combinations both exist downstream.
• The gix facade crate is not a dependency (ADR-012 §4): the backends drive gix-odb/gix-pack/gix-ref at the component level (POC-2's shape); repo opening from a registry-resolved root is gix-discover + gix-odb::Store::at. Verified at implementation.
• The registry-file store needs no gitoxide; persistence adapters (SQLite et al.) are future, separate, additive — ADR-012 §2.
• Hash: sha1 pinned (the compile-time-rejected invariant from docs/research/gitoxide.md); sha256 passthrough feature (OQ-05 policy unchanged).
• Encodes the POC-2 prerequisites by construction: odb handle sharing (Arc<Store> shared, per-session handles, prevent_pack_unload() + ignore_replacements = true), generation on blocking threads, O(counts) memory, missing-objects abort.
• Received-pack ingestion via gix-pack::Bundle::write_to_directory_eagerly (streaming-input, thin-base lookup) + gix-fsck + gix-ref transactions with PreviousValue::MustExistAndMatch CAS (ADR-004, ADR-013 — the full push state machine is decided there).

Concurrency model

• gix structures: parking_lot short-held locks; per-session handles and spawn_blocking live inside the gix impls (POC-2's shape: store shared, handle per session, generation on blocking threads).
• Poisoned locks: unwrap_or_else(|e| e.into_inner()) (convention 2).
• Trait execution model (review 001 A-6): all five traits are #[async_trait] — Send + Sync, dyn-compatible (E0038 forbids bare async fn), signatures per ADR-012 §1. GitRegistry::resolve is async (one call per session/request, before the first protocol byte — ADR-011 §6); no sync-hot-path constraint exists here, unlike alkcall's accept loop. The execution model the signatures imply:
  • The wire layer enforces the pipeline-concurrency budget itself (ADR-009's "max concurrent blocking pipeline tasks" — a permit acquired in the wire layer around each GitPackGen/GitPackIngest call; the admission point of "enforced at assembly/acceptance time"). The wire layer is backend-trait-only (ADR-010) and knows nothing of stores, handles, or threads — so the permit, not spawn_blocking, is its entire concurrency contract.
  • Implementations must not block the async executor and own their internal threading: the gix impls run pack generation/ingestion on spawn_blocking with the owned handle moved in inside the trait impl (POC-2's shape, restated at its true layer — it is the impl's internal structure, invisible from the wire layer).
  • The traits are async so an embedder whose storage is async (DB registries, network object stores) implements them natively; the gix impl's blocking work is an internal detail, not part of the seam.
  • Bridge to the signatures (ADR-018 §4): parameter ownership is the fork-prevention — the wire layer constructs the boxed Send + 'static stream ends and owned vecs inside the permit, and the impls move them into their internal spawn_blocking tasks. Nothing is borrowed across the seam.

Public API surface

Crate-root re-exports (the alktty pattern): backend traits + types (including RepoRecord, AccessAction, the authorize policy function, the authorized-repo marker type (the door constructs it only after the ADR-007 check passes — session tuples carry it, transport.md substrate layer), and the git/repo/* op spec+handler pairs), GitAdapter/register_openable (producer), GitSession (consumer — the ADR-017 typed client: ls_refs/fetch/push), substrate types, Limits, protocol error enums; feature types (GixBackend family under gix, the file registry under registry-file) exported under their features. The embedder-facing freeze point remains OQ-03 (narrowed: it is now this crate's own publish).

The two op kinds (ADR-012 §3, the classification)

alkgit ships both alkcall op kinds from one crate — the first family payload to do so (alktty/alktunnels: open ops only; alknet-docker, planned: call ops only):

• Open op (channels/git/sub via register_openable; open-op params {repo, service} — ADR-016) — the binary half: negotiation + service selector + ACL point (ADR-007; the service selects the authorize action, read for fetch / write for push), yields the duplex git session.
• Call ops (git/repo/{create,delete,update,get}) — the JSON half: thin OperationSpec+handler pairs over Arc<dyn GitRegistryStore>, Visibility::External, always-on (no gitoxide). create is gated by the static registry check (required_scopes: ["git:repo:create"]); delete/update/get carry an empty AccessControl and the handler evaluates the ADR-015 gate — scope git:admin OR the record's manage grant, via the authorize policy function, generic FORBIDDEN denial, unknown-repo ≡ unauthorized (ADR-008). Registered by the assembler wherever it wants them exposed. Managing records (op scopes) and git access (repo grants) are distinct sources — but the record is the single per-repo authz surface, and repo creation seeds the creator's {read, write, manage} grants (ADR-015).

Registry types and schemas (review 001 N-3; the freeze-inventory draft)

The serde shapes the ops handlers and the registry-file store both serialize — pinned here so the two workstreams cannot invent them independently. These are the N-3 schemas ADR-015 chained after the manage-grant shape: they capture the three-action grant shape (ADR-015), the opaque grant-key rule (ADR-015 §7 / OQ-16), and the unknown-repo ≡ unauthorized rule at the op layer (ADR-008). Draft until first publish (additive fields have been added before publish before — ADR-016's params, ADR-017's API); at publish they enter OQ-03's inventory as compat surface.

RepoRecord (serde shape)

{
  "repo_id": "alkdev/alkgit",          // the registry id (string, non-empty)
  "storage_root": "/var/lib/alkgit/…", // server-side path, never on the wire (ADR-008)
  "visibility": "public",              // "public" | "private" (lowercase)
  "grants": {                          // identity_id (opaque string, ADR-015 §7) → grant set
    "u-7731": ["read", "write", "manage"]
  }
}

• grants values are arrays of the action strings read / write / manage (unique, lowercase; manage implies the lower actions per ADR-015 §2's monotonicity — stored sets need not be closed, the policy function evaluates that).
• storage_root is the one field that must never round-trip through an op response (ops return record data, not paths — ADR-008's never-a- path rule; the get-op response schema omits it, the resolve trait surface keeps it). Keys are snake_case, no serde renames — the record file IS the serde shape (ADR-012 §2's on-disk form).
• Additive-field rule: unknown fields are rejected, fail-closed, both on-disk (an unknown field in a stored record file is a version skew the operator must see, not silently dropped) and in op input schemas (additionalProperties: false — the same extension rule as ADR-016's open-op params; extensions replace, not accumulate, before publish).

RegistryError (the registry-family error set)

The error type of traits 1–2 (GitRegistry/GitRegistryStore) — the scope is the registry family, not the whole trait family (the object-storage traits carry StorageError, ADR-018 §7, after the "every failure the trait family can produce" claim was found to over-reach — review 002 R-2). Five variants — every failure the registry family can produce, no catch-all:

Variant	Meaning	Wire mapping
NotFound	repo id unresolved	collapses with Forbidden to the generic denial (unknown ≡ unauthorized, ADR-008; transport.md error taxonomy)
Forbidden	gate denied (op-layer or authorize)	generic FORBIDDEN — same wire shape as NotFound, never a variant leak
AlreadyExists	create with a taken id	op-level error; still NOT disclosed as an existence oracle to non-creators — create responses may carry it, resolve never returns it
Invalid	malformed record/field (store-side validation, e.g. empty repo id, unknown action string)	op-level error with the field name
Io(String)	backing-store failure (serialized as message — not std::io::Error, which is not serde-stable)	session/op error, substrate-appropriate

Io is the only stringly-typed variant; everything else is structural so ops can report precise JSON errors ({"error": "already_exists", …}) while the trait surface stays Result<_, RegistryError> per ADR-012 §1.

StorageError (ADR-018 §7) is the object-storage sibling — traits 3–5: ObjectMissing { oid } (the ADR-004 generation abort), Invalid(String) (malformed/backend input), Io(String) (same stability rule as RegistryError::Io). CAS-stale and fsck outcomes are deliberately not variants: they are RefOutcome/PreparedPush.unpack values (protocol reports the session survives), which is what keeps the no-catch-all claim true for both error types.

git/repo/* op JSON schemas (request/response)

Op	Request	Response	Errors
git/repo/create	{repo_id, visibility} — storage root assigned by the store's naming rules (ADR-012 §2); grants seeded {read, write, manage} for the caller (ADR-015 §4); an explicit grants field is an admin-shape extension, NOT v1	{repo_id, visibility}	already_exists, invalid
git/repo/delete	{repo_id}	{}	unauthorized (the single wire code for unknown-repo ≡ not-authorized — N-2's collapse rule; io)
git/repo/update	{repo_id, visibility?, grants?} — both optional; PATCH semantics (omitted fields unchanged; present fields replaced wholesale — see the update-semantics note below); grant mutation is the only grant-write path (ADR-015 §3)	{repo_id, visibility, grants} (post-write record, no storage root)	unauthorized (collapsed, N-2), invalid, io
git/repo/get	{repo_id} (single read; list is a v2 additive op)	{repo_id, visibility, grants} (no storage root — ADR-008)	unauthorized (collapsed, N-2), io

• Error codes are the wire code strings in the table (snake_case, one namespace git:repo:*), each carried in the op's ErrorDefinition (alkcall ADR-016 disclosure convention — the same pattern alktty's channel:open_failed uses). The unauthorized code is the collapsed denial: both RegistryError::NotFound and Forbidden map to it at the op-wire boundary (unknown ≡ unauthorized, ADR-008 + review 001 N-2), so no variant leaks an existence oracle; AlreadyExists is create-side only (the caller knows the id — discloseable without an oracle).
• Update semantics are PATCH (review 002 R-3 — the "full-record replace of the provided fields" qualifier was parseable either way, and the wrong reading is destructive: a visibility-only update clearing grants locks the grant-holder out): omitted fields are left unchanged, present fields are replaced wholesale (grants, when present, replaces the whole grant map — partial grant edits are read-modify-write at the caller). The response echoes the post-write record so the caller confirms what landed, including what was left unchanged.
• already_exists disclosure posture (review 002 R-6, recorded so a later agent does not "fix" it into an upstream-incompatible denial): a git:repo:create-scoped identity probing arbitrary ids can learn which exist — that is accepted. Create is a trusted, scope-gated, low-population surface; upstream gitea behaves the same; the position is that id-probing at create-scope is inside the trust boundary the scope already grants, while resolve-side disclosure (the fetch/push path) stays collapsed per ADR-008/N-2.
• update's response echoes the post-write record so grant edits confirm what landed (the last-writer-race posture is ADR-012 §Consequences, unchanged).
• All four op input schemas: additionalProperties: false (the alksocks {}-precedent — same fail-closed extension rule as ADR-016's open-op params).

Repo-id grammar (review 002 R-5)

repo_id is the registry key on every wire and op surface, so its grammar is pinned once here:

• Shape: one or two path segments, owner/name — lowercase alphanumerics and -/_, segments non-empty and at most 100 bytes total (the owner/name convention matches self-hosted git and the OQ-16 namespacing direction; a single segment is a valid legacy/ local shape).
• Wire handling (ADR-008 governs): the id is an opaque registry key — parsed against this grammar for rejection only, never decomposed (the owner segment confers nothing; the registry index is flat), never joined/normalized into paths.
• Error mapping: a grammar-invalid id is unauthorized on the serving paths (collapsed with unknown-repo, ADR-008 — the id shape leaks nothing) and invalid on the op paths (create/update — the caller needs the correction).
• The registry-file store's on-disk naming derives from the id by percent-encoding the / (alkdev/alkgit → alkdev%2Falkgit under the roots dir): flat directory, no traversal, deterministic round-trip on reload. Implementation detail (ADR-012 §2's naming rules), stated here because the id grammar forces the choice.
• Http mounting note (for doors.md consumers): two-segment ids need a two-placeholder route (/{owner}/{repo}/info/refs with single-segment ids allowed too) — the door crate pins its own route grammar when the alkhttp git feature lands; alkgit's contract is only "the full id string arrives resolution-ready."

Design Decisions

ADR	Decision	Summary
004	Pack pipeline	data::output gen / data::input ingestion
007	ACL first	registry returns rule inputs
008	Repo identity	wire names are registry ids
009	Budgets	limits flow into gen/ingest
010	Pure protocol crate	traits in-crate, impls behind features
011	Per-repo authorization	grants in records, policy in core (amended: ADR-015)
012	Registry + ops	write trait, file default, CRUD ops, feature split (gate amended: ADR-015)
015	Manage grant + op gate	manage tier, admin-scope-OR-manage gate, create seeds manage
013	receive-pack	thin-pack ingestion, transaction CAS, report framing
014	Negotiation	ack loop, common_haves seam, no ready
016	Native session preamble	{repo, service} open-op params, request-line preamble, service in the tuple
017	Consumer half	GitSession typed client (ls_refs/fetch/push), custom alkcall Transport + gix-protocol, hand-rolled push
018	Trait signatures + storage errors	object-storage trait signatures pinned, StorageError for traits 3–5, &RepoRecord repo param (RegistryError re-scoped to traits 1–2)

Open Questions

• OQ-05: sha256 policy (deferred(scope)).
• OQ-03: publish-time API freeze inventory (the op set enters it).
• OQ-04 resolved (ADR-013 — ingestion composition bound: thin-base lookup, .keep landing, transaction CAS).
• OQ-02 resolved (ADR-014 — common_haves ack seam added to GitPackGen).

References

• docs/research/gitoxide.md (API contract notes — normative for the gix impl)
• docs/research/poc2-findings.md (generation pipeline + prerequisites)
• alktty backend.rs/local module (the trait + feature template)
• alknet ADR-033/035 (the repo/adapter + read/write-split pattern)
• ADR-010 (the structural decision), ADR-011/012 (the seam this doc specifies)