alkdev/alkgit

0 Fork 0

Files

T

glm-5.3-flash 8f73da5d12 docs(architecture): phase 1 bootstrap — specs, 9 ADRs, OQ tracker

Architecture documentation structure per sdd_process phase 1:

- README index (doc table, ADR table, lifecycle), overview with crate
  map, dependency rules, and security invariants
- Component specs: storage, transport, http, ssh, alkgitd (all draft)
- ADRs 001-009: crate decomposition, front-door-blind core, V2-first
  protocol, pack pipeline (data::output generation / data::input
  ingestion), session substrate types, http adapter composition
  (proposed, OQ-01), ACL-before-advertisement, registry-resolved repo
  identity, bounded-resources budgets
- open-questions.md: OQ-01..08 with two deferred(scope), one
  deferred(unclear), door-type definitions, blocker tracker tasks in
  tasks/architecture/
- v1 ssh-door decision recorded: russh terminates wire SSH in alkgitd;
  alkcall channels stay the internal substrate (OQ-03 partially
  resolved)

Two review rounds (fresh-context subagent): 4 critical + 17 warnings
fixed in round one; zero critical + 4 warnings + 5 suggestions fixed in
round two. All ADR/OQ cross-references verified resolving.

2026-09-21 03:55:33 +00:00

3.6 KiB

Raw Blame History

ADR-004: Pack pipeline — gitoxide `data::output` generation, `data::input` streaming ingestion

Status

Accepted

Context

Pack generation and ingestion are the two halves of fetch and receive-pack. POC-2 resolved the generation question empirically; the ingestion tool turned out to be mislabeled in the original research plan (the plan called it bundle::write — the correct tool is data::input streaming) and was corrected there.

Generation options were:

a. gix-pack::bundle::write into a temp dir, read back — wrong tool: it is the index-from-stream machinery (consumes an existing pack stream, mmaps to resolve deltas, always writes files). Not a generation path. b. gix-pack::data::output::bytes fed by an odb walk — validated live: streams to any io::Write with O(counts) memory (60k-object pack → 5.3 MB out, ~11 MB extra RSS), real git 2.43 clones fsck-clean.

Ingestion (receive-pack): client sends a possibly-thin pack stream; server must index it, fsck it, and apply ref updates via CAS.

Decision

Fetch-side generation is gitoxide's own pipeline, composed exactly as gitoxide-core/src/pack/create.rs does (POC-2's validated composition):

peel wants to commit tips
  → commit-ancestry walk (gix_traverse::commit::Simple, Parents::All)
      [+ non-commit tips as-is]
  → count::objects(_unthreaded, TreeContents)   [per-commit tree expansion]
  → entry::iter_from_counts                      [chunked deflate + pack-delta copy]
  → InOrderIter → bytes::FromEntriesIter (V2, sha1)
  → io::Write sink (sideband on the wire, or http response body)

Two-stage closure is mandatory: TreeContents does not follow commit parents; feeding tip commits alone produces incomplete packs (POC-2's course correction). The odb handle needs prevent_pack_unload() + ignore_replacements = true. gix_odb::Cache is not Sync: share Arc<Store>, build a handle per session, generate on blocking threads.

Missing objects mid-generation must abort the fetch (sideband error band), never emit a broken pack — check entry statistics (missing_objects > 0).

Receive-side ingestion parses the client pack stream (gix-pack::data::input with streaming-input), fscks it (gix-fsck connectivity), and applies ref updates via gix-ref transaction CAS. The exact push state machine (capability advertisement set, CAS timing, status report, http framing) is OQ-04's investigation target; the ingestion-tool choice above is decided.

Delta synthesis for loose objects is an optimization backlog item, not a v1 commitment: existing pack deltas copy through for free; loose objects ship as compressed bases. No upstream delta-encode API exists (gix-delta is apply-only).

Consequences

Fresh clones of loose-ish repos ship uncompressed bases (fine for v1; clients re-pack at rest); repos kept packed get pack-copy efficiency.
Memory stays O(counts); streaming under back pressure is proven on the http path (POC-3: flat RSS under a 650 KB/s reader).
Determinism: objects_unthreaded gives deterministic order; threaded count + InOrderIter is the scale path later.
alkgit-core exposes this as a type taking (odb handle, wants, haves) → streaming pack; negotiation-agnostic (POC-2 follow-up 1).

References

docs/research/poc2-findings.md (the whole basis), docs/research/poc3-findings.md (streaming proof)
docs/research/gitoxide.md §"Storage" (generation-pipeline notes)
docs/research/git-protocol.md §"Server-side pack generation"
ADR-005 (how the sink reaches the wire), ADR-009 (memory/time budgets)
transport.md §fetch, storage.md §packs

3.6 KiB Raw Blame History

ADR-004: Pack pipeline — gitoxide data::output generation, data::input streaming ingestion