glm-5.3-flash
86bf5a0cf0
Structural decision (OQ-09 resolved): alkgit follows the alktty/ alktunnels template — a single published protocol crate on alkcall channels, no binary, no front doors. - ADR-010 supersedes ADR-001 (crate decomposition) and ADR-006 (http router factory); both marked Superseded - Single crate at repo root: Cargo.toml with gix feature (default-on backend implementations; wire layer compiles without it — gix-hash always-on with sha1 per the compile-time-rejected invariant), crates/ workspace deleted, src/lib.rs stub in place - doors.md replaces http.md/ssh.md/alkgitd.md: alkhttp git-feature sequencing (after first publish), alkssh requirement (fixed-grammar exec dispatch), native alk/git path, downstream assembly - backend.md replaces storage.md: GitRegistry/GitRefs/GitPackGen/ GitPackIngest traits (ingest validates, refs commits — single CAS home), gix feature encodes POC-2 prerequisites - transport.md reframed for the single crate; backend traits replace hook traits in the public API - OQ-09 resolved (all five sub-decisions in ADR-010), OQ-01 resolved (subsumed), OQ-03 narrowed to publish-freeze, OQ-08 narrowed to registry identity + vault placement, OQ-07 rescoped to the gix feature's registry impl - vision.md v2: single-binary/monorepo framing corrected as init-agent artifact; POC checklist marked complete - AGENTS.md + .opencode agent specs updated to the new shape Verification: cargo build (default + no-default-features), cargo test --all-features, clippy --all-features -D warnings, fmt --check all pass. Third review round: zero critical, all warnings/suggestions addressed (GitPackGen signature amended in ADR-004, stale anchors fixed, ADR-006 body tense normalized, CAS split stated, vision residuals cleaned).
85 lines
3.9 KiB
Markdown
85 lines
3.9 KiB
Markdown
# ADR-004: Pack pipeline — gitoxide `data::output` generation, `data::input` streaming ingestion
|
|
|
|
## Status
|
|
Accepted
|
|
|
|
## Context
|
|
|
|
Pack generation and ingestion are the two halves of fetch and receive-pack.
|
|
POC-2 resolved the generation question empirically; the ingestion tool
|
|
turned out to be mislabeled in the original research plan (the plan called
|
|
it `bundle::write` — the correct tool is `data::input` streaming) and was
|
|
corrected there.
|
|
|
|
Generation options were:
|
|
|
|
a. `gix-pack::bundle::write` into a temp dir, read back — **wrong tool**:
|
|
it is the index-from-stream machinery (consumes an *existing* pack
|
|
stream, mmaps to resolve deltas, always writes files). Not a generation
|
|
path.
|
|
b. `gix-pack::data::output::bytes` fed by an odb walk — validated live:
|
|
streams to any `io::Write` with O(counts) memory (60k-object pack →
|
|
5.3 MB out, ~11 MB extra RSS), real git 2.43 clones fsck-clean.
|
|
|
|
Ingestion (receive-pack): client sends a possibly-thin pack stream; server
|
|
must index it, fsck it, and apply ref updates via CAS.
|
|
|
|
## Decision
|
|
|
|
**Fetch-side generation** is gitoxide's own pipeline, composed exactly as
|
|
`gitoxide-core/src/pack/create.rs` does (POC-2's validated composition):
|
|
|
|
```
|
|
peel wants to commit tips
|
|
→ commit-ancestry walk (gix_traverse::commit::Simple, Parents::All)
|
|
[+ non-commit tips as-is]
|
|
→ count::objects(_unthreaded, TreeContents) [per-commit tree expansion]
|
|
→ entry::iter_from_counts [chunked deflate + pack-delta copy]
|
|
→ InOrderIter → bytes::FromEntriesIter (V2, sha1)
|
|
→ io::Write sink (sideband on the wire, or http response body)
|
|
```
|
|
|
|
Two-stage closure is mandatory: `TreeContents` does **not** follow commit
|
|
parents; feeding tip commits alone produces incomplete packs (POC-2's
|
|
course correction). The odb handle needs `prevent_pack_unload()` +
|
|
`ignore_replacements = true`. `gix_odb::Cache` is not `Sync`: share
|
|
`Arc<Store>`, build a handle per session, generate on blocking threads.
|
|
|
|
Missing objects mid-generation must **abort** the fetch (sideband error
|
|
band), never emit a broken pack — check entry statistics
|
|
(`missing_objects > 0`).
|
|
|
|
**Receive-side ingestion** parses the client pack stream
|
|
(`gix-pack::data::input` with `streaming-input`), fscks it
|
|
(`gix-fsck` connectivity), and applies ref updates via `gix-ref`
|
|
transaction CAS. The exact push state machine (capability advertisement
|
|
set, CAS timing, status report, http framing) is OQ-04's investigation
|
|
target; the ingestion-tool choice above is decided.
|
|
|
|
**Delta synthesis** for loose objects is an optimization backlog item, not
|
|
a v1 commitment: existing pack deltas copy through for free; loose objects
|
|
ship as compressed bases. No upstream delta-encode API exists
|
|
(`gix-delta` is apply-only).
|
|
|
|
## Consequences
|
|
|
|
(2026-09-21 amendment, ADR-010: the crate layout changed — the type
|
|
below is now the `GitPackGen` backend trait in the single crate,
|
|
gix-free by signature `(repo, wants, haves, limits)`; backend.md is
|
|
authoritative for the trait shape. The pipeline itself is unchanged.)
|
|
|
|
- Fresh clones of loose-ish repos ship uncompressed bases (fine for v1;
|
|
clients re-pack at rest); repos kept packed get pack-copy efficiency.
|
|
- Memory stays O(counts); streaming under back pressure is proven on the
|
|
http path (POC-3: flat RSS under a 650 KB/s reader).
|
|
- Determinism: `objects_unthreaded` gives deterministic order; threaded
|
|
count + `InOrderIter` is the scale path later.
|
|
- The generation seam is negotiation-agnostic: boundary sets in → pack
|
|
out (POC-2 follow-up 1).
|
|
|
|
## References
|
|
- `docs/research/poc2-findings.md` (the whole basis), `docs/research/poc3-findings.md` (streaming proof)
|
|
- `docs/research/gitoxide.md` §"Storage" (generation-pipeline notes)
|
|
- `docs/research/git-protocol.md` §"Server-side pack generation"
|
|
- ADR-005 (how the sink reaches the wire), ADR-009 (memory/time budgets)
|
|
- transport.md §fetch, backend.md §"The trait family" (GitPackGen) |