docs(research): POC-1 complete — pkt-line over alkcall BiStream verified

Full V2 fetch handshake (ls-refs + fetch with done + sideband pack) runs
between real git 2.43 and a Rust listener over the alkcall
Connection -> BiStream -> tokio-util compat -> gix-packetline path.

Verification: git ls-remote, git clone (fsck --strict clean), incremental
git fetch, annotated-tag checkout, in-process duplex selftest. All over
alkcall 0.8 core types; the flagged futures-io risk resolved with a
two-line tokio-util compat adapter.

Findings: docs/research/poc-1-findings.md (deliverable; POC source stays
disposable at /workspace/alkgit-poc1 per the standalone POC mode).

Also records in the research docs:
- git-protocol.md: observed done-path shape (no acknowledgments section),
  git:// first-request framing, advertise-once rule
- gitoxide.md: gix-packetline async stop-delimiter + reset() contract
- alk-stack.md: POC-1 verification item marked resolved
- pocs.md: POC-1 status -> proceed
This commit is contained in:
glm-5.3-flash committed 2026-09-20 12:18:04 +00:00
1 parent b9ab5e4c9e
commit 665f701133
5 files changed
+209

No files matched your search

+3
View File
@@ -81,6 +81,9 @@ door produced the stream.
1. **BiStream ↔ pkt-line fit**: run a real `git clone` handshake over an
alkcall `Connection` using `gix-packetline` async codec end-to-end.
**Verified (POC-1, 2026-09-20)**: real git 2.43 clones/fetches
fsck-clean over `Connection → BiStream → tokio-util compat →
gix-packetline`. See `poc-1-findings.md`.
2. **HTTP smart protocol shape**: confirm alkhttp's streaming response body
can carry a pack (chunked) and that request bodies stream in without
buffering the whole POST in memory (receive-pack can be gigabytes).
+14
View File
@@ -27,6 +27,8 @@ we want to own code.
2. Server replies with capability advertisement: `version 2`, then
capabilities as pkt-lines: `agent=...`, `ls-refs=...`, `fetch=...`
(shallow, filter, sideband-64k, packfile-uris...), `object-format=sha1`.
**Observed (POC-1)**: the advertisement is sent once per session;
commands follow without re-advertisement.
3. `command=ls-refs` with args (peel, symrefs, ref-prefix) → server streams
ref lines then flush.
4. `command=fetch` with args (want lines, have lines, done, thin-pack,
@@ -34,6 +36,18 @@ we want to own code.
`acknowledgments` section (acked ids or `NAK`), then either
`ready` + `packfile` section (pack streamed over sideband) if done, or
waits for more haves.
**Observed (POC-1, git 2.43)**: on the `done` path the response is just
`packfile\n` + sideband chunks + flush — no `acknowledgments`, no
`ready`. The acknowledgments section appears on multi-round negotiation
(no `done`), not on the done path.
### git:// framing detail (observed, POC-1)
First pkt-line: `git-upload-pack <repo>\0host=<host>\0\0version=2\0` —
service + repo, NUL-separated extras, an empty segment, then
`version=2`. The repo name arrives in-band before any ref data; a server
must resolve it against the registry (and run ACL) before emitting the
advertisement.
### V0/V1 (fallback for old clients / http stateless)
+9
View File
@@ -70,6 +70,15 @@ compile-time-rejected invariant, so the config can't drift silently.
- `gix-packetline` — pkt-line encode/decode, both blocking and `futures-io`
async (`async-io` feature). This is the one crate both the http and ssh
interfaces need most; it is small and stable (0.22.x).
**API contract notes (POC-1, validated live):** `async_io` codec is built
on `futures_io` traits — bridging a tokio stream takes
`tokio::io::split` + `tokio_util::compat::{compat, compat_write}` (two
one-liners). `StreamingPeekableIter` treats `Flush` as a stop-delimiter:
on flush, `read_line()` returns `None` (no line is yielded) and the
iterator is inert until `.reset()` — a command loop must reset after
every request or the next command is silently swallowed. Return type is
`Option<io::Result<Result<PacketLineRef, decode::Error>>>` (three layers;
wrap it in a session type).
- `gix-transport` — defines `Protocol` (V0/V1/V2), `client::MessageKind`, and
the fetch-side abstractions. Only the *types* are reusable server-side;
its transport implementations (client) are not what we need.
+178
View File
@@ -0,0 +1,178 @@
# POC-1 findings: pkt-line over alkcall BiStream
**Status**: complete — **proceed**
**Date**: 2026-09-20
**Mode**: standalone (`/workspace/alkgit-poc1`, disposable per `pocs.md`; this
document is the deliverable on main)
## Hypothesis (restated)
A full V2 fetch handshake (`ls-refs` + one `fetch` round) between real `git`
CLI (as client, `git clone`/`git fetch`) and a Rust listener that bridges
alkcall `BiStream` to `gix-packetline`'s async codec can be completed.
## Result
**Confirmed.** Real `git` 2.43 successfully completed a full protocol V2
conversation with a Rust server whose entire wire path is:
```
tokio TcpStream
-> alkcall Connection::from_bidi(stream, alpn, addr) [alkcall 0.8.0]
-> ProtocolHandler::handle(conn, &auth)
-> Connection::accept_bi() -> BiStream [joined, boxed dyn]
-> tokio::io::split(stream) -> (ReadHalf, WriteHalf)
-> tokio_util::compat::{compat, compat_write} [tokio <-> futures-io]
-> gix_packetline::async_io::{StreamingPeekableIter, Writer, encode} [0.22.2]
-> V2 server state machine (ours)
```
Validated against real git's parser (the strictest pkt-line validator
available), all over the alkcall `Connection`/`BiStream`/`ProtocolHandler`
surface — no simulation, no bypass:
| Operation | Result |
|---|---|
| `git ls-remote git://…/repo.git` | all 5 refs + peeled tag + `^{}` deref, clean exit |
| `git clone` (V2, full handshake) | clone completes, `git fsck --strict` passes |
| `git fetch` (incremental, `refs/heads/dev:…`) | completes, ref updated |
| `git checkout v1.0` (annotated tag served) | checks out at the tagged commit |
| in-process `selftest` over `tokio::io::duplex` | advertisement → ls-refs → fetch → sideband pack, asserts pass |
The fixture repo (built with the `gix` facade: 3 commits, 2 branches +
feature branch, annotated tag, symbolic HEAD) round-trips byte-exact — the
clone's object graph is identical to the fixture's.
## Answered hypotheses
### 1. The flagged risk — `futures-io` vs tokio — is real and cheap to solve
`gix-packetline` 0.22.2's `async-io` feature is built on `futures_io`
traits; alkcall's `BiStream` is a tokio `AsyncRead + AsyncWrite`. The bridge
is two one-line trait calls from `tokio-util`'s `compat` feature:
```rust
let (read_half, write_half) = tokio::io::split(stream); // BiStream: Unpin, so plain split works
let reader = StreamingPeekableIter::new(read_half.compat(), &[PacketLineRef::Flush], false);
let writer = write_half.compat_write();
```
No hand-written adapter needed. `BiStream` being `Unpin` by construction
(boxed `dyn AsyncReadWrite + Unpin`) means `tokio::io::split(stream)` works
directly on it, exactly as ADR-009 intends. **The composition boundary
("consume (identity, repo id, duplex stream, limits)") survives contact with
the packetline codec.**
### 2. Full V2 fetch handshake shape (byte-level, observed)
Captured ground truth from real `git upload-pack` via stdio pipes and
`GIT_TRACE_PACKET`, then reproduced it:
- **Advertisement (once per session, not per command)**: `version 2`,
`agent=…`, capability lines, flush. git does **not** re-advertise between
commands — my first implementation re-advertised in the command loop and
the client hung. This is now encoded in the POC's state machine
(advertise once → command loop).
- **`command=ls-refs`**: request is command line(s) + `0001` delimiter +
args (`peel`, `symrefs`, `ref-prefix …`) + flush. Response is ref lines
(`"<oid> <ref>[ symref-target:…][ peeled:…]"`) + flush. `ref-prefix`
filtering is client-driven; `peel`/`symrefs` are per-request flags.
- **`command=fetch` with `done`**: git 2.43's response is just
`packfile\n` + sideband chunks (`0001`-framed band-1 data) + flush —
**no `acknowledgments` section and no `ready`** when `done` is present.
The git-scm protocol docs' grammar suggests an acknowledgments section;
the real wire does not send one on the `done` path. (Recorded because
`git-protocol.md`'s inventory assumed the docs' shape.)
- **`git://` framing** (used by the POC's tcp bridge): one pkt-line
`git-upload-pack <repo>\0host=<host>\0\0version=2\0` — the V2 request
rides NUL-separated extras after an empty segment. Repo name is in-band
here, so a real server resolves it against the registry before any ref
line is emitted (ACL-before-advertisement invariant holds naturally).
### 3. gix-packetline `StreamingPeekableIter` has a non-obvious API contract
Two findings worth encoding in `alkgit-transport`'s design:
1. **Flush is a stop-delimiter, not a yielded line.** Constructed with
`&[PacketLineRef::Flush]`, `read_line()` returns `None` when the peer's
flush arrives; the line is *not* returned as `PacketLineRef::Flush`.
After the stop, the iterator is inert until `.reset()`. Consequence: a
V2 command loop must `read_command() { …; reader.reset() }` after every
request — forgetting `reset()` silently swallows the next command
(this was the POC's second hang).
2. **`read_line()` returns `Option<io::Result<Result<PacketLineRef,
decode::Error>>>`** — three layers deep (`None` | io error | decode
error | line). The POC normalized this into a small helper; the real
crate should wrap it in a friendlier session type.
### 4. Honest capability advertisement is easy to get right
The POC advertises exactly `ls-refs=unborn`, `fetch=wait-for-done`,
`object-format=sha1` — and accepts/ignores `thin-pack`/`ofs-delta`/`no-progress`
because the behavior (never emit deltas, never send progress) matches what
those args would request. Declining `shallow`, `filter`, `packfile-uris`,
`object-info`, `server-option` by not advertising them was accepted by real
git without complaint. Real git sends `agent=` and `object-format=sha1` on
its command requests; parsing `object-format` from command requests (and
rejecting mismatches) is a to-do for the real implementation.
## What the POC deliberately does NOT settle (POC-2's job)
The `fetch` response pack is a **framing vehicle**: hand-rolled, no deltas,
fully materialized in memory (`Vec<u8>`), whole-object zlib only. It is
deliberately naive — POC-2 owns pack *composition* quality (delta
compression, `gix-pack` bundle-write vs entries-to-bytes, streaming,
memory behavior at 10k objects). One lesson already visible from here:
- The naive pack writer needs the **full reachable closure**
(commit → tree → subtrees → blobs, tag → target). The first version
walked only commit/tag edges and real git rejected the pack with
`did not receive expected object` — object closure computation is
non-optional and is exactly the want/have-set walk `git-protocol.md`
describes. This strengthens the case for measuring POC-2's option (b)
(`gix-pack::data::output::bytes` fed by an odb walk) early.
## What it changes in the research docs
- `alk-stack.md` §"What to verify" item 1: **resolved** — BiStream ↔
pkt-line fit confirmed; the compat adapter is the only glue.
- `git-protocol.md`: add the observed `done`-path shape (no
acknowledgments on `done`), the git:// first-request framing detail, and
the advertise-once rule to the V2 section.
- `gitoxide.md`: `gix-packetline` async usage note (stop-delimiter +
`reset()` contract) so the transport crate gets it right the first time.
- `pocs.md`: POC-1 outcome recorded.
## Follow-ups for phase 1 (architecture input)
1. `alkgit-transport` session type should wrap the three-layer bridge
(split → compat → packetline) so handlers never see `futures-io` types;
the POC's `serve_v2` is the seed of that type's interface.
2. Command-request parsing (`command=…` + delim + args + flush) and the
`reset()` discipline belong in one type (`RequestReader`), not spread
across handlers.
3. Sideband pack streaming (band-1 chunks + flush) is trivial over the
same `Writer`; `max chunk size` (65000 < 65515) and a max-pack-size
budget should be server-configurable (bounded-resources invariant).
4. `read_request_line`'s repo-id extraction is the ACL hook: resolve repo
ID → registry → storage root, `AccessControl::check(peer)` **before**
the advertisement or any ref line is written (visible-surface =
authorized-surface).
## Verification transcript (2026-09-20)
```
$ cargo run -- bridge 9419 /tmp/opencode/poc1-run # serves fixture
$ git -c protocol.version=2 clone git://127.0.0.1:9419/repo.git clone-final
-> completes
$ cd clone-final && git fsck --strict # FSCK-OK
$ git checkout v1.0 # annotated tag works
$ git -c protocol.version=2 fetch origin refs/heads/dev:refs/heads/newdev
-> * [new branch] dev -> newdev
$ git -c protocol.version=2 ls-remote … # 5 refs + peeled
```
Traces captured with `GIT_TRACE_PACKET=1` show the client receiving the
POC's advertisement (`agent=alkgit-poc1/0.1`), ref advertisement with
`symref-target`/`peeled` attributes, and the `packfile` section over
sideband with flush termination.
+5
View File
@@ -24,6 +24,11 @@ docs.
## POC-1: pkt-line over alkcall BiStream
**Status**: complete (2026-09-20) — **proceed**. See `poc-1-findings.md`.
Real git 2.43 clones/fetches/fsck-clean over the alkcall
`Connection → BiStream → tokio-util compat → gix-packetline` path; the
flagged futures-io risk resolved with a two-line compat adapter.
**Hypothesis**: a full V2 fetch handshake (`ls-refs` + one `fetch` round)
between real `git` CLI (as client, `git clone`/`git fetch`) and a Rust
listener that bridges alkcall `BiStream` to `gix-packetline`'s async codec