alkgit/docs/research/git-protocol.md

# Research: git smart protocol (server side) — what we must implement

**Status**: initial pass complete
**Date**: 2026-09-19
**Sources**: gitoxide source reading (client-side parsers as reference),
git http-protocol + packfile protocol specs (git-scm.com protocol docs),
observation of gitserver's shape as a sanity check that the surface is this
small (no code reused — MPL-2.0 incompatible with our MIT/Apache-2.0; see
`license-note.md`).

## TL;DR

The server-side surface is well-bounded: capability advertisement (V0/V1 and
V2 flavors), `ls-refs`, fetch negotiation (wants/haves → acks → pack),
receive-pack (pack ingestion + ref update CAS + status report), plus the
stateless HTTP framing and the SSH exec framing around them. gitoxide gives
us every primitive (pkt-line codec, pack parse/write, ref transactions,
fsck); what's missing is only the *orchestration*, which is ours to write —
and that's the layer where our auth/ACL model lives, which is exactly where
we want to own code.

## Protocol surface inventory

### V2 (the target; what modern git speaks first)

1. Client sends `GIT_PROTOCOL=version=2` (SSH: env line / http: header).
2. Server replies with capability advertisement: `version 2`, then
   capabilities as pkt-lines: `agent=...`, `ls-refs=...`, `fetch=...`
   (shallow, filter, sideband-64k, packfile-uris...), `object-format=sha1`.
   **Observed (POC-1)**: the advertisement is sent once per session;
   commands follow without re-advertisement.
3. `command=ls-refs` with args (peel, symrefs, ref-prefix) → server streams
   ref lines then flush.
4. `command=fetch` with args (want lines, have lines, done, thin-pack,
   no-progress, include-tag, shallow/deepen...) → server sends
   `acknowledgments` section (acked ids or `NAK`), then either
   `ready` + `packfile` section (pack streamed over sideband) if done, or
   waits for more haves.
   **Observed (POC-1, git 2.43)**: on the `done` path the response is just
   `packfile\n` + sideband chunks + flush — no `acknowledgments`, no
   `ready`. The acknowledgments section appears on multi-round negotiation
   (no `done`), not on the done path.

### git:// framing detail (observed, POC-1)

First pkt-line: `git-upload-pack <repo>\0host=<host>\0\0version=2\0` —
service + repo, NUL-separated extras, an empty segment, then
`version=2`. The repo name arrives in-band before any ref data; a server
must resolve it against the registry (and run ACL) before emitting the
advertisement.

### V0/V1 (fallback for old clients / http stateless)

- First response line: `"<capabilities> <null-octet> <ref> <obj-id>"` style
  advertisement with refs, then fetch loop: wants → haves (NAK/ACK) → pack
  over sideband. Stateless-http variant requires the client to POST a
  separate request for each round; `ack` state must be re-derived or the
  multi-round negotiation declined (we can require V2 for http and keep V0/V1
  for ssh only — OQ candidate).

### receive-pack (push; mostly version-independent)

- Client sends update requests (`<old> <new> <ref>`) + optional shallow lines
  + pack stream (possibly thin).
- Server: validate CAS per ref (old must match current unless zero-id create),
  fsck/connectivity the pack, apply ref transaction atomically, reply with
  `unpack <ok|ng>` + per-ref `ok|ng <ref> <reason>` lines.

### Framing per interface

- **SSH**: single exec request `git-upload-pack '<path>'` /
  `git-receive-pack '<path>'` / `git-upload-archive`; pkt-line on stdin/stdout;
  V2 via env var; sideband on fetch.
- **HTTP**: `GET /info/refs?service=<name>` (advertisement in
  `# service=git-upload-pack` preamble for smart clients),
  `POST /<name>` with pkt-line body; content-type
  `application/x-git-{upload,receive}-pack-*`; chunked streaming both ways;
  `Cache-Control: no-cache`.

## Server-side pack generation — the crux

`gitserver` (MPL-2.0, read-only reference) demonstrates the pragmatic path:
compute the pack from wants/haves and hand it to the response writer — they
use `gix` + hand-rolled protocol_v2 (742 LOC) and receive_pack (549 LOC) for
the whole protocol surface. We cannot copy that code, but its size confirms
the surface is small enough to own outright with gitoxide primitives:

- Pack generation options to POC:
  a. `gix-pack bundle::write` with an in-memory sink (verify it can stream,
     not just write files).
  b. `gix-pack::data::output::bytes` entries-to-bytes writer fed by an object
     walk over `gix-odb` (full control over want/have closure; no file
     intermediates).
  **Resolved (POC-2, `poc2-findings.md`)**: (b) wins — it is gitoxide's own
  generation pipeline (`count::objects` → `entry::iter_from_counts` →
  `bytes::FromEntriesIter`, as composed by `gix pack create`) and streams to
  any `io::Write` with O(counts) memory. (a) is not a generation tool: it
  indexes an *existing pack stream* (receive-pack's job) and always writes a
  temp file (it mmaps the pack to resolve deltas for the index).
- **The closure is two stages**: commit ancestry
  (`gix_traverse::commit::Simple`, `Parents::All`, tips peeled to commits)
  feeding `ObjectExpansion::TreeContents` (which expands each commit's own
  tree but does **not** follow parents). Wants alone through `TreeContents`
  produce an incomplete pack that `git index-pack --strict` rejects.
- **odb handle prerequisites** for generation:
  `prevent_pack_unload()` + `ignore_replacements = true` (asserts/corruption
  otherwise). `gix_odb::Cache` is not `Sync` (per-thread `RefCell` caches):
  share the `Arc<Store>`, build a handle per session, generate on blocking
  threads.
- **Delta behavior**: existing pack deltas are copied through
  (`from_pack_entry`); loose objects are written as compressed bases. No
  delta *synthesis* exists upstream (`gix-delta` is apply-only) — an
  optimization backlog item vs `git upload-pack`, not a correctness gap.
- Missing objects during generation do not abort by default (they become
  skipped "invalid" entries); the server must check entry statistics
  (`missing_objects`) and abort the fetch with a sideband error band instead
  of emitting a broken pack.
- Thin packs (delta against client haves) are an optimization — v0 can
  declare `thin-pack` unsupported initially; V2 fetch arg parsing must still
  accept/decline it gracefully.
- `filter` (partial clone) and `packfile-uris` can be declined in v1 of
  alkgit; advertised capabilities must be honest (never advertise what we
  don't serve — the gitea-class bug pattern is promising something and
  enforcing elsewhere).

## Negotiation policy (ours to define)

Minimal correct initial policy:
- V2: respond with full `acknowledgments` (common ids) each round; send pack
  on `done`.
- Enforce max rounds/haves budget per session (DoS bound), max pack size for
  receive, wall-clock limits for long negotiations.
- Shallow (` deepen/shallow lines) is a v2-later concern; reject with a
  clear pkt-line error initially (honest capability advertisement again).

## Security-relevant protocol notes

- `git-upload-archive` is a separate command — do not serve it initially.
- Path traversal: repo names on the wire (`git-upload-pack '~/x'` or
  `../`) must normalize to a registry lookup — never to a filesystem path
  from the wire. Repo IDs resolve to storage roots configured server-side.
- receive-pack ref names: validate against git ref rules (no `..`, no
  control chars, no `refs/heads/foo.lock` games) — `gix-ref` name parsing
  plus our own deny-list for e.g. `refs/` reserved namespaces.
- The advertisement phase must run ACL before emitting a single ref line
  (ref names leak info; private repos must not advertise anything).

## Open items → architecture OQs

- OQ: support V0/V1 at all on http (or V2-only http, V0/V1 ssh)?
- OQ: shallow clone support timeline.
- OQ: thin-pack on fetch (requires delta against client have set).
- Resolved: pack streaming composition (POC-2 decided: entries-to-bytes
  pipeline; see `poc2-findings.md`).
- OQ: delta synthesis for loose-object serving (optimization backlog).
- OQ: `object-format=sha256` support policy (feature flag exists; no real
  ecosystem need yet — default sha1, keep flag).