alkgit/docs/research/gitoxide.md

# Research: gitoxide (gix) as the git engine

**Status**: initial pass complete
**Date**: 2026-09-19
**Sources**: local clone at `/workspace/gitoxide` (repo HEAD `77c8cd956`,
tag-described as `gix-transport-v0.59.2-130-g77c8cd956`), crates.io API.

## TL;DR

gitoxide covers the storage layer (odb, packs, refs, objects, fsck) and gives
us the packetline codec, but it does **not** give us a git *server*. `gix-protocol`
and `gix-transport` are client-side (fetch/clone from the point of view of the
machine asking for objects). The server half of the smart protocol — capability
advertisement, `ls-refs` command dispatch, fetch negotiation on the serving side,
pack generation on demand, and receive-pack application with ref transaction and
update-hook semantics — has to be built by us. This is expected and acceptable:
it is exactly the narrow protocol-shim layer our security model wants to own
anyway (see `git-protocol.md`).

## Version alignment (important)

| crate | crates.io max stable | local clone | note |
|---|---|---|---|
| gix | 0.87.1 | 0.87.1 | in sync |
| gix-packetline | 0.22.2 | 0.22.2 | in sync |
| gix-transport | 0.59.2 | 0.59.2 | in sync |
| gix-protocol | 0.65.1 | 0.65.1 | in sync |
| gix-pack | 0.74.2 | 0.74.2 | in sync |
| gix-odb | 0.84.0 | 0.84.0 | in sync |
| gix-ref | 0.67.1 | 0.67.1 | in sync |
| gix-object | 0.64.1 | 0.64.1 | in sync |
| gix-hash | 0.26.2 | 0.26.2 | in sync |
| gix-fsck | 0.25.1 | 0.25.1 | in sync |

The published `0.87.1` (2026-08-24) matches the clone; the clone carries a
handful of unreleased commits on top. **Pin published crates.io versions** in
our manifests; treat the clone as a reading/reference aid, not a path
dependency. If an unreleased fix turns out to be required, that is an OQ for
architecture (gitoxide git-patch or `[patch]` section is the fallback).

## Hash algorithm gotcha (encountered live)

`gix` with `default-features = false` and no hash feature fails to compile:
`gix-hash` emits `compile_error!("Please set either the sha1 or the sha256
feature flag")`. The workspace manifest now pins `features = ["sha1"]`
throughout (gix, gix-pack, gix-odb, gix-object, gix-hash) and our crates carry
a matching `sha256` passthrough feature for the future. This is a
compile-time-rejected invariant, so the config can't drift silently.

## What we can use off the shelf

### Storage (strongest area — use as-is)
- `gix-odb` — object database with loose + packed stores, dynamic multi-index
  loading, `Handle` with caching; the `dynamic` store refreshes on mtime
  changes, which suits a long-running server.
  **API contract notes (POC-2, validated live):** handles used for pack
  generation need `prevent_pack_unload()` and `ignore_replacements = true`.
  `Cache` is deliberately **not `Sync`** (per-thread `RefCell` pack/object
  caches); share `Arc<Store>` across tasks and build a handle per session —
  generation belongs on blocking threads with the owned handle moved in.
- `gix-pack` — pack data reading (index file, multi-index, delta resolution)
  and pack *writing* (`bundle::write` → `write_to_directory` for
  index-from-stream). Pack writing produces index+pack into a directory;
  streaming a pack directly to a socket needs evaluation (POC candidate).
  **Generation pipeline (POC-2, validated live):** `data::output` (feature
  `generate`) is the on-demand generation path:
  `count::objects(_unthreaded)` (closure expansion; `TreeContents` expands
  each input commit's own tree but does NOT follow parents — feed the
  commit-ancestry walk through it) → `entry::iter_from_counts` (chunked
  deflate + pack-delta copy, back-pressured) → `InOrderIter` →
  `bytes::FromEntriesIter` (header/entries/trailer to any `io::Write`, V2
  only). Composition reference: `gitoxide-core/src/pack/create.rs`.
  `bundle::write_to_directory` is the *indexing* side (consume an existing
  pack stream; needs `streaming-input`) — receive-pack's tool, not fetch's.
  Deltas are copied from existing packs; no delta synthesis exists upstream
  (`gix-delta` is apply-only). Entry stats (`finalize()`) expose
  copied/recompressed/missing counts — use `missing_objects` to abort
  instead of emitting broken packs. Feature set for our lean build:
  `sha1` + `generate` + `parallel` (+ `streaming-input` for bundle write).
- `gix-ref` — ref store with `transaction` module (compare-and-swap semantics,
  reflog), which is what receive-pack needs for atomic ref updates.
- `gix-object` — object parsing/encoding (commit, tree, tag, blob).
- `gix-fsck` — connectivity checks for received packs.
- `gix-discover` — repository discovery / `.git` dir resolution.
- `gix` facade — repo opening, config, etc. For library-developer usage gitoxide
  recommends `default-features = false` + only the components needed; follow
  that to keep compile times down.

### Wire format (use as-is)
- `gix-packetline` — pkt-line encode/decode, both blocking and `futures-io`
  async (`async-io` feature). This is the one crate both the http and ssh
  interfaces need most; it is small and stable (0.22.x).
  **API contract notes (POC-1, validated live):** `async_io` codec is built
  on `futures_io` traits — bridging a tokio stream takes
  `tokio::io::split` + `tokio_util::compat::{compat, compat_write}` (two
  one-liners). `StreamingPeekableIter` treats `Flush` as a stop-delimiter:
  on flush, `read_line()` returns `None` (no line is yielded) and the
  iterator is inert until `.reset()` — a command loop must reset after
  every request or the next command is silently swallowed. Return type is
  `Option<io::Result<Result<PacketLineRef, decode::Error>>>` (three layers;
  wrap it in a session type).
- `gix-transport` — defines `Protocol` (V0/V1/V2), `client::MessageKind`, and
  the fetch-side abstractions. Only the *types* are reusable server-side;
  its transport implementations (client) are not what we need.

### Client side (exists, mostly irrelevant for us)
- `gix-protocol` — handshake, `ls-refs`, fetch negotiation *from the client
  side* (`fetch::Response::from_line_reader` parses server responses; the
  negotiate module builds request arguments). Useful as a *reference* for
  response shapes we must produce server-side, and possibly reused for
  alkgit-to-alkgit replication later. `gix-transport/src/lib.rs` exports only
  `pub mod client` — confirming no server-side half exists upstream.

## What gitoxide does NOT give us (we own this)

1. **Capability advertisement** (V0/V1 first-want line and V2 capability
   handshake) — must produce ourselves.
2. **Server-side command dispatch** — `ls-refs`, `fetch` V2 command parsing
   and the request→response state machine.
3. **Server-side negotiation** — evaluating client haves against our refs
   (ack/NAK logic, shallow handling on the serve side).
4. **On-demand pack generation for a fetch** — ~~pack writing exists
   (`gix-pack bundle::write`) but "stream a pack computed from a want/have set
   to a socket" composition needs a POC; worst case we walk objects ourselves
   via odb and feed `gix-pack::data::output::bytes` entries-to-bytes writers.~~
   **Resolved (POC-2)**: the `data::output` pipeline streams to any
   `io::Write`; we own only the orchestration (peel tips → commit walk →
   count → entries → bytes) and the missing-objects check. Delta *synthesis*
   for loose objects remains an upstream gap (optimization backlog).
5. **receive-pack** — parsing client pack stream (`gix-pack::data::input`
   can parse a pack from a reader), fsck, ref CAS updates via `gix-ref`
   transaction, reporting status (`unpack ok`/`ng` lines).

## License

MIT OR Apache-2.0 — same as ours. Clean to depend on and to read for
inspiration (attribution not required under either license, though NOTICE-type
courtesy is fine).