alkgit/docs/research/poc-1-findings.md

POC-1 findings: pkt-line over alkcall BiStream

Status: complete — proceed Date: 2026-09-20 Mode: standalone (/workspace/alkgit-poc1, disposable per pocs.md; this document is the deliverable on main)

Hypothesis (restated)

A full V2 fetch handshake (ls-refs + one fetch round) between real git CLI (as client, git clone/git fetch) and a Rust listener that bridges alkcall BiStream to gix-packetline's async codec can be completed.

Result

Confirmed. Real git 2.43 successfully completed a full protocol V2 conversation with a Rust server whose entire wire path is:

tokio TcpStream
  -> alkcall Connection::from_bidi(stream, alpn, addr)      [alkcall 0.8.0]
  -> ProtocolHandler::handle(conn, &auth)
  -> Connection::accept_bi() -> BiStream                    [joined, boxed dyn]
  -> tokio::io::split(stream) -> (ReadHalf, WriteHalf)
  -> tokio_util::compat::{compat, compat_write}             [tokio <-> futures-io]
  -> gix_packetline::async_io::{StreamingPeekableIter, Writer, encode}  [0.22.2]
  -> V2 server state machine (ours)

Validated against real git's parser (the strictest pkt-line validator available), all over the alkcall Connection/BiStream/ProtocolHandler surface — no simulation, no bypass:

Operation	Result
git ls-remote git://…/repo.git	all 5 refs + peeled tag + ^{} deref, clean exit
git clone (V2, full handshake)	clone completes, git fsck --strict passes
git fetch (incremental, refs/heads/dev:…)	completes, ref updated
git checkout v1.0 (annotated tag served)	checks out at the tagged commit
in-process selftest over tokio::io::duplex	advertisement → ls-refs → fetch → sideband pack, asserts pass

The fixture repo (built with the gix facade: 3 commits, 2 branches + feature branch, annotated tag, symbolic HEAD) round-trips byte-exact — the clone's object graph is identical to the fixture's.

Answered hypotheses

1. The flagged risk — futures-io vs tokio — is real and cheap to solve

gix-packetline 0.22.2's async-io feature is built on futures_io traits; alkcall's BiStream is a tokio AsyncRead + AsyncWrite. The bridge is two one-line trait calls from tokio-util's compat feature:

let (read_half, write_half) = tokio::io::split(stream);   // BiStream: Unpin, so plain split works
let reader = StreamingPeekableIter::new(read_half.compat(), &[PacketLineRef::Flush], false);
let writer = write_half.compat_write();

No hand-written adapter needed. BiStream being Unpin by construction (boxed dyn AsyncReadWrite + Unpin) means tokio::io::split(stream) works directly on it, exactly as ADR-009 intends. The composition boundary ("consume (identity, repo id, duplex stream, limits)") survives contact with the packetline codec.

2. Full V2 fetch handshake shape (byte-level, observed)

Captured ground truth from real git upload-pack via stdio pipes and GIT_TRACE_PACKET, then reproduced it:

• Advertisement (once per session, not per command): version 2, agent=…, capability lines, flush. git does not re-advertise between commands — my first implementation re-advertised in the command loop and the client hung. This is now encoded in the POC's state machine (advertise once → command loop).
• command=ls-refs: request is command line(s) + 0001 delimiter + args (peel, symrefs, ref-prefix …) + flush. Response is ref lines ("<oid> <ref>[ symref-target:…][ peeled:…]") + flush. ref-prefix filtering is client-driven; peel/symrefs are per-request flags.
• command=fetch with done: git 2.43's response is just packfile\n + sideband chunks (0001-framed band-1 data) + flush — no acknowledgments section and no ready when done is present. The git-scm protocol docs' grammar suggests an acknowledgments section; the real wire does not send one on the done path. (Recorded because git-protocol.md's inventory assumed the docs' shape.)
• git:// framing (used by the POC's tcp bridge): one pkt-line git-upload-pack <repo>\0host=<host>\0\0version=2\0 — the V2 request rides NUL-separated extras after an empty segment. Repo name is in-band here, so a real server resolves it against the registry before any ref line is emitted (ACL-before-advertisement invariant holds naturally).

3. gix-packetline StreamingPeekableIter has a non-obvious API contract

Two findings worth encoding in alkgit-transport's design:

1. Flush is a stop-delimiter, not a yielded line. Constructed with &[PacketLineRef::Flush], read_line() returns None when the peer's flush arrives; the line is not returned as PacketLineRef::Flush. After the stop, the iterator is inert until .reset(). Consequence: a V2 command loop must read_command() { …; reader.reset() } after every request — forgetting reset() silently swallows the next command (this was the POC's second hang).
2. read_line() returns Option<io::Result<Result<PacketLineRef, decode::Error>>> — three layers deep (None | io error | decode error | line). The POC normalized this into a small helper; the real crate should wrap it in a friendlier session type.

4. Honest capability advertisement is easy to get right

The POC advertises exactly ls-refs=unborn, fetch=wait-for-done, object-format=sha1 — and accepts/ignores thin-pack/ofs-delta/no-progress because the behavior (never emit deltas, never send progress) matches what those args would request. Declining shallow, filter, packfile-uris, object-info, server-option by not advertising them was accepted by real git without complaint. Real git sends agent= and object-format=sha1 on its command requests; parsing object-format from command requests (and rejecting mismatches) is a to-do for the real implementation.

What the POC deliberately does NOT settle (POC-2's job)

The fetch response pack is a framing vehicle: hand-rolled, no deltas, fully materialized in memory (Vec<u8>), whole-object zlib only. It is deliberately naive — POC-2 owns pack composition quality (delta compression, gix-pack bundle-write vs entries-to-bytes, streaming, memory behavior at 10k objects). One lesson already visible from here:

• The naive pack writer needs the full reachable closure (commit → tree → subtrees → blobs, tag → target). The first version walked only commit/tag edges and real git rejected the pack with did not receive expected object — object closure computation is non-optional and is exactly the want/have-set walk git-protocol.md describes. This strengthens the case for measuring POC-2's option (b) (gix-pack::data::output::bytes fed by an odb walk) early.

What it changes in the research docs

• alk-stack.md §"What to verify" item 1: resolved — BiStream ↔ pkt-line fit confirmed; the compat adapter is the only glue.
• git-protocol.md: add the observed done-path shape (no acknowledgments on done), the git:// first-request framing detail, and the advertise-once rule to the V2 section.
• gitoxide.md: gix-packetline async usage note (stop-delimiter + reset() contract) so the transport crate gets it right the first time.
• pocs.md: POC-1 outcome recorded.

Follow-ups for phase 1 (architecture input)

1. alkgit-transport session type should wrap the three-layer bridge (split → compat → packetline) so handlers never see futures-io types; the POC's serve_v2 is the seed of that type's interface.
2. Command-request parsing (command=… + delim + args + flush) and the reset() discipline belong in one type (RequestReader), not spread across handlers.
3. Sideband pack streaming (band-1 chunks + flush) is trivial over the same Writer; max chunk size (65000 < 65515) and a max-pack-size budget should be server-configurable (bounded-resources invariant).
4. read_request_line's repo-id extraction is the ACL hook: resolve repo ID → registry → storage root, AccessControl::check(peer) before the advertisement or any ref line is written (visible-surface = authorized-surface).

Verification transcript (2026-09-20)

$ cargo run -- bridge 9419 /tmp/opencode/poc1-run     # serves fixture
$ git -c protocol.version=2 clone git://127.0.0.1:9419/repo.git clone-final
    -> completes
$ cd clone-final && git fsck --strict                 # FSCK-OK
$ git checkout v1.0                                   # annotated tag works
$ git -c protocol.version=2 fetch origin refs/heads/dev:refs/heads/newdev
    -> * [new branch] dev -> newdev
$ git -c protocol.version=2 ls-remote …               # 5 refs + peeled

Traces captured with GIT_TRACE_PACKET=1 show the client receiving the POC's advertisement (agent=alkgit-poc1/0.1), ref advertisement with symref-target/peeled attributes, and the packfile section over sideband with flush termination.