# ADR-004: Pack pipeline — gitoxide `data::output` generation, `data::input` streaming ingestion

## Status
Accepted

## Context

Pack generation and ingestion are the two halves of fetch and receive-pack.
POC-2 resolved the generation question empirically; the ingestion tool
turned out to be mislabeled in the original research plan (the plan called
it `bundle::write` — the correct tool is `data::input` streaming) and was
corrected there.

Generation options were:

a. `gix-pack::bundle::write` into a temp dir, read back — **wrong tool**:
   it is the index-from-stream machinery (consumes an *existing* pack
   stream, mmaps to resolve deltas, always writes files). Not a generation
   path.
b. `gix-pack::data::output::bytes` fed by an odb walk — validated live:
   streams to any `io::Write` with O(counts) memory (60k-object pack →
   5.3 MB out, ~11 MB extra RSS), real git 2.43 clones fsck-clean.

Ingestion (receive-pack): client sends a possibly-thin pack stream; server
must index it, fsck it, and apply ref updates via CAS.

## Decision

**Fetch-side generation** is gitoxide's own pipeline, composed exactly as
`gitoxide-core/src/pack/create.rs` does (POC-2's validated composition):

```
peel wants to commit tips
  → commit-ancestry walk (gix_traverse::commit::Simple, Parents::All)
      [+ non-commit tips as-is]
  → count::objects(_unthreaded, TreeContents)   [per-commit tree expansion]
  → entry::iter_from_counts                      [chunked deflate + pack-delta copy]
  → InOrderIter → bytes::FromEntriesIter (V2, sha1)
  → io::Write sink (sideband on the wire, or http response body)
```

Two-stage closure is mandatory: `TreeContents` does **not** follow commit
parents; feeding tip commits alone produces incomplete packs (POC-2's
course correction). The odb handle needs `prevent_pack_unload()` +
`ignore_replacements = true`. `gix_odb::Cache` is not `Sync`: share
`Arc<Store>`, build a handle per session, generate on blocking threads.

Missing objects mid-generation must **abort** the fetch (sideband error
band), never emit a broken pack — check entry statistics
(`missing_objects > 0`).

**Receive-side ingestion** parses the client pack stream
(`gix-pack::data::input` with `streaming-input`), fscks it
(`gix-fsck` connectivity), and applies ref updates via `gix-ref`
transaction CAS. The exact push state machine (capability advertisement
set, CAS timing, status report, http framing) is OQ-04's investigation
target; the ingestion-tool choice above is decided.

**Delta synthesis** for loose objects is an optimization backlog item, not
a v1 commitment: existing pack deltas copy through for free; loose objects
ship as compressed bases. No upstream delta-encode API exists
(`gix-delta` is apply-only).

## Consequences

- Fresh clones of loose-ish repos ship uncompressed bases (fine for v1;
  clients re-pack at rest); repos kept packed get pack-copy efficiency.
- Memory stays O(counts); streaming under back pressure is proven on the
  http path (POC-3: flat RSS under a 650 KB/s reader).
- Determinism: `objects_unthreaded` gives deterministic order; threaded
  count + `InOrderIter` is the scale path later.
- `alkgit-core` exposes this as a type taking (odb handle, wants, haves) →
  streaming pack; negotiation-agnostic (POC-2 follow-up 1).

## References
- `docs/research/poc2-findings.md` (the whole basis), `docs/research/poc3-findings.md` (streaming proof)
- `docs/research/gitoxide.md` §"Storage" (generation-pipeline notes)
- `docs/research/git-protocol.md` §"Server-side pack generation"
- ADR-005 (how the sink reaches the wire), ADR-009 (memory/time budgets)
- transport.md §fetch, storage.md §packs