diff --git a/docs/architecture/README.md b/docs/architecture/README.md index 203132c..d9a5c06 100644 --- a/docs/architecture/README.md +++ b/docs/architecture/README.md @@ -1,6 +1,6 @@ --- status: draft -last_updated: 2026-09-21 +last_updated: 2026-09-25 --- # alkgit Architecture @@ -14,12 +14,14 @@ to a POC finding or research doc, or is flagged as an open question. ## Current State Phase 1, architecture committed to the pure-protocol-crate shape (ADR-010; -OQ-09 resolved). All docs below are `draft` except the superseded ADRs. -POC-1/2/3 validated the git protocol half end-to-end against real git -2.43. This cycle settled the auth/backend theme: per-repo authorization -(ADR-011, OQ-08), registry backing + write surface + CRUD ops (ADR-012, -OQ-06/OQ-07). The remaining design work is the receive-pack state machine -(OQ-04) and V2 multi-round negotiation (OQ-02). +OQ-09 resolved). POC-1/2/3 validated the git protocol half end-to-end +against real git 2.43. Previous cycles settled the auth/backend theme +(ADR-011, ADR-012). This cycle settled the wire surface against +real-client captures: the receive-pack push state machine (ADR-013, +OQ-04) and the V2 multi-round negotiation ack loop (ADR-014, OQ-02). All +wire-layer design is now capture-grounded; the remaining open questions +are the publish-freeze timing (OQ-03, a release decision) and sha256 +policy (OQ-05, deferred on ecosystem need). ## Architecture Documents @@ -47,14 +49,16 @@ OQ-06/OQ-07). The remaining design work is the receive-pack state machine | [010](decisions/010-pure-protocol-crate.md) | Pure protocol crate (alktty/alktunnels template) | Accepted | | [011](decisions/011-per-repo-authorization.md) | Per-repo authorization (grants in records, policy in core) | Accepted | | [012](decisions/012-registry-backing-and-ops.md) | Registry backing, write surface, CRUD ops, feature split | Accepted | +| [013](decisions/013-receive-pack-state-machine.md) | receive-pack state machine (V0-framed push, thin-pack, unpack-first CAS) | Accepted | +| [014](decisions/014-v2-negotiation-ack-loop.md) | V2 negotiation ack loop (no `ready`, wait-for-done stays) | Accepted | ## Open Questions All unresolved questions are tracked in [open-questions.md](open-questions.md) -with stable OQ-IDs, priorities, and cross-references. Highest-priority -open: OQ-04 (receive-pack validation). Also open: OQ-02 (multi-round -negotiation), OQ-03 (publish freeze inventory), OQ-05 (sha256, -deferred). +with stable OQ-IDs, priorities, and cross-references. Remaining: OQ-03 +(publish freeze inventory — partially resolved, blocked on first-publish +timing) and OQ-05 (sha256, deferred on ecosystem need). The wire-layer +questions (OQ-02, OQ-04) resolved this cycle with ADR-014/ADR-013. ## Document Lifecycle diff --git a/docs/architecture/backend.md b/docs/architecture/backend.md index cb127e4..29b8101 100644 --- a/docs/architecture/backend.md +++ b/docs/architecture/backend.md @@ -1,6 +1,6 @@ --- status: draft -last_updated: 2026-09-21 +last_updated: 2026-09-25 --- # Backend traits: the storage seam @@ -36,20 +36,28 @@ gix types: for receive-pack, name validation per git ref rules + reserved- namespace deny-list). 4. **`GitPackGen`** — (repo, wants, haves, limits) → streaming pack - (`io::Write` consumer). Negotiation-agnostic. Missing objects abort - with an error, never a broken pack (ADR-004). + (`io::Write` consumer), plus `common_haves(repo, haves) -> recognized + subset` for the negotiation ack loop (a have is recognized iff it + exists in the object store and is a commit — the honest boundary: + never ack what we cannot subtract; ADR-014). Negotiation-agnostic + generation. Missing objects abort with an error, never a broken pack + (ADR-004). 5. **`GitPackIngest`** — client pack stream → indexed pack + fsck/ - connectivity report. It *prepares* the validated ref updates; the - transaction itself is applied by `GitRefs` (single CAS home — ingest - validates, refs commits). Budgeted (ADR-009 max pack size); - blocking-thread friendly. + connectivity report. Thin-pack bases resolve from the server's own odb + (push sends thin packs by default — ADR-013 §5); the pack lands with a + `.keep` guard; missing objects → `unpack ng`. It *prepares* the + validated ref updates; the transaction itself is applied by `GitRefs` + (single CAS home — ingest validates, refs commits; one transaction per + push is what makes `atomic` correct — ADR-013 §7). Budgeted (ADR-009 + max pack size); blocking-thread friendly. Minimal-vs-full was the open sub-question; resolved as **full family** — -the four traits are each one or two methods plus types, and collapsing -them (e.g. refs into the registry) would force one impl block per -downstream where independent seams are cheaper to satisfy. The `gix` -feature implements all four; a downstream with its own object store -implements 3–4 and reuses 1–2, or none of it. +each trait is one or two methods plus types, and collapsing them (e.g. +refs into the registry) would force one impl block per downstream where +independent seams are cheaper to satisfy. The `gix` feature implements +the three object-storage traits (3–5: `GitRefs`, `GitPackGen`, +`GitPackIngest`); a downstream with its own object store implements 3–5 +and reuses 1–2 (`GitRegistry`/`GitRegistryStore`), or none of it. ## Feature model (ADR-012 §4, amends ADR-010's single-`gix` story) @@ -77,9 +85,10 @@ Two independent seams, two default-on features: (`Arc` shared, per-session handles, `prevent_pack_unload()` + `ignore_replacements = true`), generation on blocking threads, O(counts) memory, missing-objects abort. -- Received-pack ingestion via `gix-pack::data::input` (`streaming-input`) - + `gix-fsck` + `gix-ref` transactions (ADR-004). Validation against - real `git push` is OQ-04. +- Received-pack ingestion via `gix-pack::Bundle::write_to_directory_eagerly` + (`streaming-input`, thin-base lookup) + `gix-fsck` + `gix-ref` + transactions with `PreviousValue::MustExistAndMatch` CAS (ADR-004, ADR-013 + — the full push state machine is decided there). ## Concurrency model @@ -132,12 +141,17 @@ planned: call ops only): | [010](decisions/010-pure-protocol-crate.md) | Pure protocol crate | traits in-crate, impls behind features | | [011](decisions/011-per-repo-authorization.md) | Per-repo authorization | grants in records, policy in core | | [012](decisions/012-registry-backing-and-ops.md) | Registry + ops | write trait, file default, CRUD ops, feature split | +| [013](decisions/013-receive-pack-state-machine.md) | receive-pack | thin-pack ingestion, transaction CAS, report framing | +| [014](decisions/014-v2-negotiation-ack-loop.md) | Negotiation | ack loop, `common_haves` seam, no `ready` | ## Open Questions -- **OQ-04**: pack ingestion validation (deferred(unclear)). - **OQ-05**: sha256 policy (deferred(scope)). - **OQ-03**: publish-time API freeze inventory (the op set enters it). +- OQ-04 resolved (ADR-013 — ingestion composition bound: thin-base + lookup, `.keep` landing, transaction CAS). +- OQ-02 resolved (ADR-014 — `common_haves` ack seam added to + `GitPackGen`). ## References diff --git a/docs/architecture/decisions/003-protocol-v2-first.md b/docs/architecture/decisions/003-protocol-v2-first.md index 8761937..f60d58d 100644 --- a/docs/architecture/decisions/003-protocol-v2-first.md +++ b/docs/architecture/decisions/003-protocol-v2-first.md @@ -34,9 +34,10 @@ Decision drivers: exactly as observed in POC-1 (re-advertising between commands hangs real git). - Advertised capabilities are exactly what we serve: `ls-refs=unborn`, - `fetch=wait-for-done` (v1 fetch policy is full-closure pack on `done`; - multi-round negotiation, OQ-02, may add capability values when it - lands), `object-format=sha1` (+ sha256 under the feature flag, if and + `fetch=wait-for-done` (the full-closure pack on `done` policy + validated in POC-1/2; the multi-round ack loop, ADR-014, needs no + capability change — the acknowledgments section is grammar, not + capability), `object-format=sha1` (+ sha256 under the feature flag, if and when tested — OQ-05). Nothing unimplemented is advertised (shallow, filter, packfile-uris, object-info, server-option are *declined by omission*; POC-1 confirmed real git accepts this). @@ -46,23 +47,22 @@ Decision drivers: - HTTP requests protocol V2 only; ssh requests protocol V2 only (see below). -**V0/V1 policy: decided now as V2-only for v1.** Both doors -(http and ssh) speak V2; clients that cannot speak V2 get a clear pkt-line -error. Reversal is wire-visible (the advertised version set is a wire +**V0/V1 policy: decided now as V2-only for v1 — fetch only.** Both doors +(http and ssh) speak V2 for *fetch*; clients that cannot speak V2 get a +clear pkt-line error. Reversal is wire-visible (the advertised version set is a wire format), so treat this as effectively one-way once published — the decision is still made now, and revisiting it needs a new ADR plus a deprecation window for clients. The motivation stands: a concrete consumer needing V0/V1 (e.g. very old CI images) has not been identified. -The V0/V1 receive-pack shape (used by `git push` on http stateless framing -in older clients) is unaffected by this choice: receive-pack is mostly -version-independent (see transport.md). +**Receive-pack is exempt**: the push path is V0-framed by upstream design +on every client generation (no version negotiation exists on push at +all — ADR-013 §1), so the V2-only rule never touches `git push`. The +V0/V1 decline applies to the fetch path only. -Sequencing note (fixes the scope contradiction the review found): the v1 -fetch policy that ships first is full-closure-on-`done` (POC-validated). +Sequencing note: the v1 fetch policy that ships first is full-closure-on-`done` (POC-validated). Multi-round V2 negotiation (haves/acks without `done`) is in v1's *scope* -but lands after the done-path works end-to-end; its ack logic is OQ-02's -open question, and any capability-advertisement change it requires -happens then. +but lands after the done-path works end-to-end; its ack loop is now +decided (ADR-014) and needs no capability-advertisement change. ## Consequences diff --git a/docs/architecture/decisions/004-pack-pipeline.md b/docs/architecture/decisions/004-pack-pipeline.md index 288576f..e638c2d 100644 --- a/docs/architecture/decisions/004-pack-pipeline.md +++ b/docs/architecture/decisions/004-pack-pipeline.md @@ -52,9 +52,9 @@ band), never emit a broken pack — check entry statistics **Receive-side ingestion** parses the client pack stream (`gix-pack::data::input` with `streaming-input`), fscks it (`gix-fsck` connectivity), and applies ref updates via `gix-ref` -transaction CAS. The exact push state machine (capability advertisement -set, CAS timing, status report, http framing) is OQ-04's investigation -target; the ingestion-tool choice above is decided. +transaction CAS. The push state machine (capability advertisement +set, CAS timing, status report, http framing) is decided in ADR-013; +the ingestion-tool choice above is decided. **Delta synthesis** for loose objects is an optimization backlog item, not a v1 commitment: existing pack deltas copy through for free; loose objects diff --git a/docs/architecture/decisions/009-bounded-resources-budget.md b/docs/architecture/decisions/009-bounded-resources-budget.md index 58ef8ea..d180e4a 100644 --- a/docs/architecture/decisions/009-bounded-resources-budget.md +++ b/docs/architecture/decisions/009-bounded-resources-budget.md @@ -33,11 +33,11 @@ overrides are assembler config. | Budget | Applies to | Default direction | |---|---|---| | max negotiation rounds | fetch (V2, no `done`) | tens | -| max haves per round | fetch | thousands | +| max haves per round | fetch | thousands (the client's stateless doubling can legitimately reach 16384 — set the default at or above that) | | max pack size | receive-pack | config-bound (tens of MB v1) | | max request body | http POSTs (receive-pack especially) | same as max pack size | | session wall clock | all sessions (enforced by transport's session loop on every door — it is the one component all doors hand the session to; alkcall channel caps add a second bound where channels exist) | tens of minutes | -| max concurrent pack generations | server-wide (blocking-pool budget) | small count | +| max concurrent blocking pipeline tasks | server-wide (blocking-pool budget; covers pack generation and pack ingestion alike — both are `spawn_blocking` consumers) | small count | | sideband chunk size | fetch streaming | 65000 (fixed, per protocol) | | max advertisement refs | ls-refs response | config-bound | diff --git a/docs/architecture/decisions/013-receive-pack-state-machine.md b/docs/architecture/decisions/013-receive-pack-state-machine.md new file mode 100644 index 0000000..087596a --- /dev/null +++ b/docs/architecture/decisions/013-receive-pack-state-machine.md @@ -0,0 +1,231 @@ +# ADR-013: receive-pack (push) state machine + +## Status + +Accepted (resolves OQ-04) + +## Context + +OQ-04 (`deferred(unclear)`) held the receive-pack validation gap: the +pieces were decided (ADR-004 ingestion via `gix-pack::data::input` with +`streaming-input`; `gix-ref` transaction CAS; `gix-fsck` connectivity; +POC-3: request bodies stream) but the push state machine's shape was not. +The listed unknowns: capability advertisement set under the +honest-advertisement invariant, request-line parsing + shallow policy, +thin-pack acceptance, ingestion composition, CAS timing, status-report +shape, and the http framing surface (ADR-003's V2 decision covers fetch +only). + +Resolution method: a walkthrough against real `git push` captures +(git 2.43.0 over file://, git:// daemon, and smart-http against a +ground-truth mock receive-pack), raw stdio requests driven into real +`git receive-pack` for policy-error ground truth, and gitoxide source +verification of the ingestion composition. Captures are recorded in +`docs/research/push-captures.md`. + +## Decision + +**Receive-pack is V0-framed, by upstream design, and we serve it as such.** + +1. **Version surface**: receive-pack has no version negotiation at all — + `git push` never speaks V2 on the push path (observed: `protocol.v2` + clients push with the identical V0/V1 shape; no `version=2` exchange + exists). ADR-003's V2-first decision therefore governs fetch only; the + push side is V0-framed on every door. This is upstream's shape, not a + choice of ours, and it is wire-stable by the same token. + +2. **Capability advertisement** (per-door, honest per ADR-003): + - Smart-http GET: `# service=git-receive-pack` + flush, then a + **V0-shaped ref advertisement** — capabilities ride NUL-attached on + the FIRST ref line; remaining ref lines are bare; flush. The + `capabilities^{}` zero-id sentinel replaces the ref lines only when + the repo has no refs (git rejects a bare capability dump, and rejects + a trailing sentinel when refs exist). + - Duplex: the same ref advertisement without the service prefix; the + server speaks first. + - The served set is `report-status report-status-v2 delete-refs + side-band-64k atomic ofs-delta object-format=sha1` (+ `push-options` + under a config gate). **`quiet` is not advertised** (it is a + client-side preference whose only server duty is suppressing band-2 + progress chatter — we never send progress). `agent=alkgit/` + is advertised (same convention upstream uses; value is the crate + version). `shallow`/`allow-tip-sha1-in-want`-style fetch caps do + not appear here. + - Everything we do not serve (e.g. `push-cert`) is declined by omission. + - `report-status-v2` is served: the v1 report shape (no option + lines) IS valid v2 — option lines exist only when the server emits + them, and v1 emits none (push-options are metadata-only in v1). + Clients selecting v2 parse the same shape. + - ACL runs before the first ref line (ADR-007 unchanged): private + repos advertise nothing. + +3. **Request parsing** (one POST / one stream): + ``` + command lines ( , caps NUL-attached on line 1) + [shallow lines] → v1: rejected up front with a pkt-line error (see 4) + flush + [push-options section: bare pkt-lines + flush] (only if negotiated) + PACK stream (raw, to end of body / stream) + ``` + - Deletes: `old= new=zero-id`; creates: `old=zero-id`. + - An immediate flush (no commands) is a client-side nothing-to-do — + reply flush, no report. + - A pack is ALWAYS expected after the command flush (real receive-pack + errors with `unpack eof …` + all-refs `unpacker error` otherwise). + An empty pack (zero objects) is valid and reported as such. + +4. **Shallow on push: rejected for v1** — confirming the OQ's expected + resolution. `git push` from a shallow clone sends `shallow ` + lines before the commands. Real receive-pack accepts them when the + pack completes history and rejects with + `ng shallow update not allowed` otherwise. We serve neither: + shallow request lines are rejected up front with a clear pkt-line error + (mirroring fetch's decline-by-omission in ADR-003, keeping depth + semantics symmetric). Declining up front is cheaper than accepting + packs we cannot fsck: a shallow push's pack terminates on grafted + boundaries that our connectivity check cannot close, and + `receive.shallowUpdateDeepen`-style semantics are a scope decision for + a later ADR, not a v1 need. This is a two-way door: adding shallow + support later is additive (the shallow-request capability rides + upstream's existing grammar when implemented). + +5. **Thin packs: accepted, bases from the server odb.** `git push` sends + thin packs by default (`push.thin=true`, no capability involved): + captured packs exclude objects the client believes the server has and + fail `git index-pack --strict` standalone; `--fix-thin` completes them. + Ingestion therefore passes the server's own `gix_object::Find` handle + as `thin_pack_base_object_lookup` to `Bundle::write_to_directory` (the + upstream parameter exists for exactly this); the no-lookup + composition (`Option` — a lookup that can never resolve, + valid only for packs known self-contained, per poc2-findings) is not + used on the push path. Thin-pack base + availability is enforced by the lookup: a base the server lacks fails + ingestion (→ `unpack ng`), which is the correct honest outcome. + +6. **Ingestion composition** (ADR-004's tool choice, now bound to the + push machine): + - Stream the pack body into `Bundle::write_to_directory_eagerly` (= + `data::input` `streaming-input` with `LookupRefDeltaObjectsIter` when + bases resolve, `EntryDataMode::KeepAndCrc32`, `Mode::Verify`), + `directory` = the repo's pack dir: the outcome is a written pack + + index with a `.keep` file created before the pack lands (gc-safe + against the not-yet-applied refs) — gitoxide's own receive shape. + - Budgets (ADR-009): max pack size counts the streamed body + (413-mapped on http); wall clock and the blocking-pipeline + concurrency budget apply (the generation budget covers ingestion — + same `spawn_blocking` pool, amended in ADR-009's table). + - Ingestion runs on `spawn_blocking` (POC-2's shape: store shared, + handle per session). + - fsck/connectivity: `gix-fsck::Connectivity` over the odb handle + after indexing, for each new tip reachable from the pushed refs; + missing objects → `unpack ng `. + - The ingest trait (`GitPackIngest`) **prepares** validated updates; + the ref transaction itself is applied by `GitRefs` (single CAS home, + backend.md unchanged). + +7. **CAS timing: unpack-first, then per-ref checks** (the observed + upstream order, adopted): parse commands → read + index + fsck the + pack → per-ref validation (name, CAS, policy) → atomic rollback if + selected → report. Evidence: real receive-pack reports `unpack ok` + + `ng funny refname` / `deletion prohibited` / stale-old ng only + AFTER unpack; a missing pack fails everything at unpack. A pre-unpack + CAS pre-check is permitted as a fail-fast optimization (reject before + reading the body), but the protocol-correct order is as observed and + is what v1 implements. Per-ref CAS uses `gix-ref` transactions + (`Change::Update/Delete` with `PreviousValue::MustExistAndMatch` / + `MustNotExist`) — one transaction for the whole push, which is what + makes `atomic` correct. **The v1 update policy** (the "policy" leg of + the per-ref check): CAS on old-value per the request (zero-id create, + current-id update/delete) and nothing more — deletes are allowed + (the `delete-refs` capability is advertised) and non-fast-forward + updates are allowed (force-push is the client's explicit choice via + the advertised old value; the CAS contract is exact-match, and + upstream's `denyNonFastForwards`-style history checks are a + deployment concern, expressible later as config in the assembly + layer without a trait change — the reasons observed above come from + upstream's *config-dependent* policies, not protocol requirements). + A reason alkgit emits itself (`funny refname`, `shallow update not + allowed`, `atomic push failure`, CAS failures) needs no config. + +8. **Status report**: when the client selected `side-band-64k`, the + report is band-1 chunks whose payload is ITSELF pkt-line-framed: + `unpack ok|ng `, per-ref `ok ` / `ng `, + an INNER flush terminating the report section, then the OUTER flush. + Without sideband: bare pkt-lines to one flush. The report is + `unpack ok` even when some refs fail; pack-level failure is + `unpack ng ` + all-refs `unpacker error`. Observed per-ref + reasons from real receive-pack are forwarded verbatim (`shallow + update not allowed`, `deletion prohibited`, `non-fast-forward`, + `atomic push failure`, `funny refname`); CAS-stale failures carry a + server-chosen reason (upstream uses `stale info`-style text + internally; ours is free to pick — clients display it verbatim). + Human-readable diagnostics ride band-2. The + substrate owns the framing (a report sent unwrapped when sideband was + selected aborts real clients — observed `fatal: protocol error: bad + line length character`), so handlers cannot get it wrong. `quiet` + suppresses progress (which we never send), not errors — band-2 + diagnostics are always permitted. + +9. **Ref-name validation** (per-ref, after unpack): `gix_validate:: + reference::name` plus alkgit's deny-list — the git-protocol.md + security note, now pinned to the exact validation point and reason + string (`funny refname`). The deny-list is a small constant in the + transport layer (git-refname-invalid namespaces: names ending in + `.lock`, `refs/` reserved prefixes used by our own metadata — the + list lives with the validation code; exact contents are an + implementation-time detail following `git check-ref-format` rules, + not an architecture variable). + +10. **Atomic**: when `atomic` was negotiated and any ref fails, every + other ref reports `ng atomic push failure` (observed live). + Implemented by preparing the whole `gix-ref` transaction and committing + it once; per-ref results map from the transaction outcome. + +11. **Push-options**: negotiated (`push-options` advertised only when the + assembler enables it; default off in v1), request section parsed + (bare pkt-lines between the command flush and the pack) and surfaced + to the ingest/refs seam as per-push metadata. Rejection of specific + options is `ng `; the section's absence when not + negotiated must not be parsed as commands (the flush boundary is + authoritative). + +12. **Http framing for push** (POC-3 facts, now extended): small pushes + arrive Content-Length; large pushes arrive chunked, and the client + may send a 4-byte `0000` probe POST first, answered 200-empty. The + POST Content-Type is `application/x-git-receive-pack-request`; the + response is `application/x-git-receive-pack-result` (sideband-wrapped + report when negotiated), ending at a flush — the stateless substrate + (ADR-005) owns these rules unchanged. + +## Consequences + +- **Positive**: every wire shape in the push path is POC/capture-grounded + (not grammar-inferred); the ingestion composition binds ADR-004's tools + to concrete calls, with thin-pack handling first-class upstream + (`.keep`-guarded pack landing, base lookup); `atomic` falls out of the + single-transaction commit; the honest-advertisement invariant holds with + a small served set; OQ-04's seven unknowns all have decisions. +- **Negative**: shallow pushes are rejected (clients see a clear error, + not a silent decline) — a real capability gap vs `git receive-pack`, + accepted for v1 symmetry with fetch's shallow decline. `push-options` + default-off adds a config surface (additive, two-way). +- **Neutral**: the V0 framing of push coexists with the V2 fetch path in + the same session layer; the state machines differ per command, which + the substrate already models. CAS-before-unpack fail-fast remains an + optimization backlog item (tracked: + `tasks/architecture/oq-13-cas-failfast.md`). + +## References + +- `docs/research/push-captures.md` (the captures this decision is built on) +- `docs/research/poc3-findings.md` (request streaming, http framing facts) +- ADR-003 (V2-first — fetch only; honest advertisement everywhere), + ADR-004 (pack pipeline: ingestion tools), ADR-005 (substrate owns + framing), ADR-007 (ACL before advertisement), ADR-009 (budgets) +- gitoxide: `gix-pack` `Bundle::write_to_directory[_eagerly]` + (`thin_pack_base_object_lookup`, `.keep` handling), `data::input` + (`LookupRefDeltaObjectsIter`, `Mode::Verify`, `EntryDataMode`), `gix-ref` + transactions (`PreviousValue::MustExistAndMatch`), `gix-fsck` + (`Connectivity`), `gix-validate::reference::name` +- transport.md §receive-pack, backend.md §"The trait family" (GitPackIngest, + GitRefs), doors.md \ No newline at end of file diff --git a/docs/architecture/doors.md b/docs/architecture/doors.md index 8e5d951..55d74b7 100644 --- a/docs/architecture/doors.md +++ b/docs/architecture/doors.md @@ -1,6 +1,6 @@ --- status: draft -last_updated: 2026-09-21 +last_updated: 2026-09-25 --- # Doors: how alkgit is exposed @@ -56,11 +56,15 @@ reference implementation of exactly that mapping. **Framing facts the feature must honor** (all POC-3-validated, encoded in the substrate, not re-decided): responses end at flush (never `0002`); -flush is one-shot per response; flush-only POSTs are probes answered +top-level flush is one-shot per response (the receive-pack sideband +report's inner flush lives inside band-1 and is part of the report +framing — ADR-013 §8; the rule governs protocol-level sections only); +flush-only POSTs are probes answered 200-empty; request bodies stream (no accumulation); response bodies stream under back pressure (bounded mpsc → `Body::from_stream`); request bodies carry a budget (ADR-009 — alkhttp custom routes are unbounded by -default). +default). Push framing per ADR-013 §12 (probe/Content-Length/chunked, +request/result content types). ## alkssh (git-over-ssh, future) @@ -109,6 +113,8 @@ deployment's docs, not here. | [010](decisions/010-pure-protocol-crate.md) | Pure protocol crate | doors are family infrastructure | | [011](decisions/011-per-repo-authorization.md) | Per-repo authorization | `authorize` policy, grants in records | | [012](decisions/012-registry-backing-and-ops.md) | Registry + ops | op registration surface, two op kinds | +| [013](decisions/013-receive-pack-state-machine.md) | receive-pack | V0-framed push advertisement per door, report framing | +| [014](decisions/014-v2-negotiation-ack-loop.md) | Negotiation | one round per POST; ack section is per-round, stateless | ## Open Questions diff --git a/docs/architecture/open-questions.md b/docs/architecture/open-questions.md index 1450148..baeed07 100644 --- a/docs/architecture/open-questions.md +++ b/docs/architecture/open-questions.md @@ -1,6 +1,6 @@ --- status: draft -last_updated: 2026-09-21 +last_updated: 2026-09-25 --- # Open Questions @@ -23,10 +23,13 @@ when their impacts say so; it records how careful the resolution must be. | OQ | Status | Blocked on / investigation | |---|---|---| -| OQ-04 | deferred(unclear) | receive-pack walkthrough (capabilities, shallow, thin-pack, CAS timing) + push POC | | OQ-05 | deferred(scope) | ecosystem need for sha256 | | OQ-03 | partially resolved | first-publish timing (release decision; the freeze inventory lives in backend.md/transport.md) | +OQ-02 and OQ-04 resolved this cycle (ADR-014, ADR-013); OQ-08/06/07/09/01 +resolved in earlier cycles. No `open` or `deferred(unclear)` questions +remain. + ## Theme: composition / crate shapes ### OQ-09: Slim-crate model — doors as family infrastructure, git as a payload service @@ -71,55 +74,56 @@ when their impacts say so; it records how careful the resolution must be. - **Blocked on**: first-publish timing (a release decision, not an architecture question). The API surface inventory lives in [backend.md](backend.md) §public API and [transport.md](transport.md) - §public API. -- **Cross-references**: ADR-010, ADR-002, ADR-012, backend.md, transport.md + §public API. ADR-013/014 added the push/negotiation trait surface + (`GitPackIngest` binding, `GitPackGen::common_haves`) to the freeze + inventory. +- **Cross-references**: ADR-010, ADR-002, ADR-012, ADR-013, ADR-014, + backend.md, transport.md ## Theme: transport / protocol ### OQ-02: V2 multi-round negotiation (ack/NAK logic, `wait-for-done` retirement) - **Origin**: [transport.md], poc2-findings §"does NOT settle" -- **Status**: open -- **Priority**: medium (full-closure-on-`done` works; multi-round is an - efficiency feature, not correctness) -- **Impacts**: fetch efficiency on repos with large shared history; - capability advertisement text (`fetch=` value). -- **Resolution path**: design the ack loop (rounds budget per ADR-009) - when transport implementation begins; POC-2's generator is - negotiation-agnostic already. -- **Cross-references**: ADR-003, ADR-004, ADR-009, transport.md §fetch +- **Status**: **resolved** — ADR-014 (V2 negotiation ack loop). The + duplex-capture walkthrough against git 2.43.0 (ground-truth negotiation + mock, cross-checked against `fetch-pack.c`) resolved the grammar: no- + `done` rounds get `acknowledgments` (`ACK ` per recognized have, + `NAK` when none, flush — never `ready`, so FLUSH is always the + terminator); the `done` round generates closure(wants) − closure(haves) + with no cross-round server state (clients re-send wants + commons every + round); `wait-for-done` stays and the advertisement text is unchanged; + the ack check is a new backend-trait method (`common_haves`) keeping + the honest boundary at the seam. Captures: + `docs/research/negotiation-captures.md`. +- **Resolution**: [decisions/014-v2-negotiation-ack-loop.md] +- **Cross-references**: ADR-003, ADR-004, ADR-005, ADR-009, ADR-014, + transport.md §fetch, backend.md §"The trait family" (GitPackGen) ### OQ-04: receive-pack (push) — validation gap - **Origin**: [transport.md], poc3-findings §"does NOT settle" -- **Status**: deferred(unclear) -- **Door type**: two-way -- **Priority**: high -- **Impacts**: blocks receive-pack implementation tasks; push is the - always-authenticated half of the wire surface. -- **Investigation**: the pieces are decided (POC-2: pack ingestion via - `gix-pack::data::input` with `streaming-input` (ADR-004); `gix-ref` - transaction CAS; fsck via `gix-fsck`; POC-3: request bodies stream). The shape to - work through: the full push state machine — - (a) the receive-pack **capability advertisement set** - (`report-status`/`report-status-v2`, `delete-refs`, `push-options`, - `atomic`, `side-band-64k`, `object-format`) under the honest-advertisement - invariant (ADR-003); (b) request-line parsing - (` ` + shallow lines policy — expected resolution: - **reject shallow on push for v1**, mirroring fetch's decline-by-omission - in ADR-003, so depth semantics stay symmetric; confirm against real - `git push` behavior); - (c) thin-pack acceptance on push (client packs may be thin; accepting - implies base-object availability requirements); (d) pack ingestion - mid-stream; (e) CAS validation timing (before vs after pack index); - (f) status report (`unpack ok|ng` + per-ref lines); (g) the - receive-pack version/framing surface over http (which framing `git push` - uses against us; ADR-003's V2 decision covers fetch only). Method: - walkthrough against real `git push` captures, then a small POC if the - ingestion composition is not obvious from POC-2's findings. Tracker - task: `tasks/architecture/oq-04-receive-pack.md`. -- **Cross-references**: ADR-003, ADR-004, ADR-009, transport.md - §receive-pack, backend.md §"The trait family" (GitRefs), doors.md +- **Status**: **resolved** — ADR-013 (receive-pack state machine). The + walkthrough against real `git push` captures (git 2.43.0: file://, + git://, smart-http, plus raw stdio requests into real `git receive-pack`) + resolved every listed unknown: the push path is V0-framed by upstream + design (no version negotiation, ADR-003 governs fetch only); the + advertisement is a V0-shaped ref advertisement (caps on the first ref + line, `capabilities^{}` sentinel only for empty repos) with served set + `report-status(-v2) delete-refs side-band-64k atomic ofs-delta + object-format=sha1` (+ `push-options` config-gated); shallow request + lines are rejected up front for v1 (symmetric with fetch); thin packs + accepted with bases from the server odb (default client behavior, no + capability); ingestion = `Bundle::write_to_directory_eagerly` + + `gix-fsck` + one `gix-ref` transaction per push (`.keep`-guarded pack + landing); CAS timing is unpack-first-then-per-ref (observed upstream + order); status report is band-1 pkt-line-framed with inner flush when + sideband was selected. Captures: + `docs/research/push-captures.md`. +- **Resolution**: [decisions/013-receive-pack-state-machine.md] +- **Cross-references**: ADR-003, ADR-004, ADR-005, ADR-009, ADR-013, + transport.md §receive-pack, backend.md §"The trait family" (GitRefs, + GitPackIngest), doors.md ### OQ-05: sha256 support policy diff --git a/docs/architecture/overview.md b/docs/architecture/overview.md index 585b2b1..675d977 100644 --- a/docs/architecture/overview.md +++ b/docs/architecture/overview.md @@ -1,6 +1,6 @@ --- status: draft -last_updated: 2026-09-21 +last_updated: 2026-09-25 --- # Overview: alkgit @@ -70,8 +70,9 @@ same pattern. Doors live in the door crates — see [doors.md](doors.md). - Smart-http streaming both ways (POC-3 → the stateless substrate that alkhttp's future `git` feature maps onto; `docs/research/poc3-findings.md` §alkhttp fit is the mounting reference). -The full V2 fetch path against real git 2.43 is proven; receive-pack is -designed but not yet exercised (OQ-04). +The full V2 fetch path against real git 2.43 is proven; receive-pack and +multi-round negotiation are design-complete against real-client captures +(ADR-013, ADR-014 — `push-captures.md`, `negotiation-captures.md`). ## Design Decisions @@ -79,7 +80,7 @@ designed but not yet exercised (OQ-04). |---|---|---| | [001](decisions/001-crate-decomposition.md) | Crate decomposition | **superseded by ADR-010** | | [002](decisions/002-front-door-blind-core.md) | Session boundary | (identity, repo, stream, limits) — unchanged, load-bearing | -| [003](decisions/003-protocol-v2-first.md) | V2-first protocol | V2-only both doors; honest advertisement | +| [003](decisions/003-protocol-v2-first.md) | V2-first protocol | V2-only fetch both doors; push is V0-framed by upstream design (ADR-013); honest advertisement | | [004](decisions/004-pack-pipeline.md) | Pack pipeline | `data::output` gen / `data::input` ingestion | | [005](decisions/005-session-substrate-types.md) | Substrate types | duplex + stateless APIs over one state machine | | [006](decisions/006-http-adapter-composition.md) | HTTP adapter composition | **superseded by ADR-010** (mounting → alkhttp feature) | @@ -89,20 +90,20 @@ designed but not yet exercised (OQ-04). | [010](decisions/010-pure-protocol-crate.md) | Pure protocol crate | single crate, producer/consumer halves, no doors/binary | | [011](decisions/011-per-repo-authorization.md) | Per-repo authorization | grants in repo records, policy in core, vault-nil | | [012](decisions/012-registry-backing-and-ops.md) | Registry + ops | read/write split, file default, CRUD ops, feature split | +| [013](decisions/013-receive-pack-state-machine.md) | receive-pack | V0-framed push machine, thin-pack acceptance, unpack-first CAS | +| [014](decisions/014-v2-negotiation-ack-loop.md) | Negotiation | ack loop, `common_haves` seam, no `ready` | ## Open Questions Key questions tracked in [open-questions.md](open-questions.md): -- **OQ-04**: receive-pack (push) validation gap (high — the - always-authenticated half of the wire surface). -- **OQ-02**: V2 multi-round negotiation (medium — efficiency, not - correctness). -- **OQ-03**: publish-time API freeze inventory (the `git/repo/*` op - set enters it; ADR-012). +- **OQ-03**: publish-time API freeze inventory (the `git/repo/*` op set + and the trait family enter it; ADR-012, ADR-013/014's trait additions). - **OQ-05**: sha256 policy (deferred(scope), low). -Resolved this cycle: OQ-08 (ADR-011 — per-repo authorization, grants in -records, vault-nil), OQ-06 (ADR-012 — `registry-file` default, -persistence adapters additive), OQ-07 (ADR-012 — CRUD ops shipped -External with scope+ownership ACL). \ No newline at end of file +Resolved this cycle: OQ-04 (ADR-013 — receive-pack state machine, +capture-grounded), OQ-02 (ADR-014 — negotiation ack loop). Earlier: +OQ-08 (ADR-011 — per-repo authorization, grants in records, vault-nil), +OQ-06 (ADR-012 — `registry-file` default, persistence adapters +additive), OQ-07 (ADR-012 — CRUD ops shipped External with +scope+ownership ACL), OQ-09/OQ-01 (ADR-010 — pure protocol crate). \ No newline at end of file diff --git a/docs/architecture/transport.md b/docs/architecture/transport.md index ca25d17..8554631 100644 --- a/docs/architecture/transport.md +++ b/docs/architecture/transport.md @@ -1,6 +1,6 @@ --- status: draft -last_updated: 2026-09-21 +last_updated: 2026-09-25 --- # alkgit: Git Smart Protocol (wire layer) @@ -43,8 +43,8 @@ substrate property (per-request state), not a protocol fork. hangs on re-advertisement); per-request-set on http (stateless: the client re-sends the dump). - Honest capability list: exactly what we serve (`ls-refs=unborn`, - `fetch=wait-for-done` for the v1 done-path policy — ADR-003 pins the - values; OQ-02 may extend them when multi-round lands, + `fetch=wait-for-done` — the ack loop needs no capability change, the + acknowledgments section is grammar not capability (ADR-014); `object-format=sha1`). Unimplemented features are declined by omission (validated against real git, POC-1). `git-upload-archive` is not served (fixed refusal — ADR-008's never-execute rule, ssh analog in @@ -59,25 +59,58 @@ substrate property (per-request state), not a protocol fork. ### fetch - Parse wants/haves/done/args; object-format check (reject mismatches — - POC-1 to-do). -- Negotiation policy (v1 initial): full-closure pack on `done` - (POC-validated). Multi-round ack/NAK negotiation: **OQ-02**. + the object-format line is validated against the advertisement's + `object-format` on every command; ADR-013 pins the push-side check to + the same rule). +- Negotiation policy: the full ack loop (ADR-014) — no-`done` rounds + get an `acknowledgments` section (`ACK ` per recognized have via + the backend's `common_haves`, `NAK` when none, flush; never `ready`), + the `done` round generates closure(wants) − closure(haves) via + `GitPackGen`. No cross-round state on either substrate (the client + re-sends wants + commons each round — negotiation-captures.md). + Advertisement text is unchanged: `fetch=wait-for-done`. - Pack generation via `GitPackGen` (ADR-004), streamed over sideband on duplex / sideband-in-response on http; generation runs on `spawn_blocking` with the owned handle moved in (POC-2's shape: store shared, handle per session, generation on blocking threads). -- Round/haves budgets enforced here (ADR-009). +- Round/haves budgets enforced here (ADR-009); an empty resulting pack + (client already has everything) is a valid zero-object packfile. ### receive-pack (push) -- Parse update requests (` ` + shallow lines), ingest the - pack stream (POC-3-verified: request bodies stream), apply CAS - transactions, emit status report. The capability set, shallow policy, - CAS timing, and exact status-report shape are OQ-04's investigation - target (design intent: shape per OQ-04's resolution). -- Ingestion via `GitPackIngest` (ADR-004); validation pending — **OQ-04** - (includes the receive-pack version surface, which ADR-003's fetch-V2 - decision does not cover). +V0-framed by upstream design (no version negotiation on the push path — +ADR-013); shapes are capture-grounded (`push-captures.md`), not +grammar-inferred. + +- **Advertisement**: V0-shaped ref advertisement — caps NUL-attached on + the first ref line, `capabilities^{}` sentinel only for empty repos; + served set `report-status report-status-v2 delete-refs side-band-64k + atomic ofs-delta object-format=sha1` (+ `push-options` under config + gate); ACL before the first ref line (ADR-007). +- **Request**: command lines (` `), shallow lines + rejected up front for v1 (symmetric with fetch's decline, ADR-013 §4), + flush, optional push-options section, then the pack stream — which is + always expected (missing pack errors at unpack). An immediate flush is + a client-side nothing-to-do (reply flush, no report). +- **Ingestion** via `GitPackIngest` (ADR-004): thin packs accepted with + bases from the server odb (default client behavior, no capability); + `Bundle::write_to_directory_eagerly` with the repo's pack dir, `.keep` + guard; `gix-fsck` connectivity per new tip; missing objects → + `unpack ng`. +- **CAS timing**: unpack-first, then per-ref checks (name via + `gix_validate::reference::name` + reserved deny-list, CAS via `gix-ref` + transactions, policy), atomic rollback, report — the observed upstream + order (ADR-013 §7). One transaction per push. +- **Status report**: band-1 pkt-line-framed (`unpack ok|ng`, per-ref + `ok|ng `, inner flush, outer flush) when sideband was + selected; bare pkt-lines otherwise. Substrate owns the framing + (unwrapped reports abort real clients). +- **Http framing**: Content-Length or chunked request (probe POST + answered 200-empty above the client's postBuffer), Content-Type + `application/x-git-receive-pack-request` / `...-result`, response ends + at flush (ADR-005 unchanged). +- Push-options parsing and `atomic` rollback semantics per ADR-013 + §10–11. ### Error taxonomy @@ -104,26 +137,31 @@ Publish-freeze point: **OQ-03**. | ADR | Decision | Summary | |---|---|---| | [002](decisions/002-front-door-blind-core.md) | Session boundary | duplex + stateless entry points | -| [003](decisions/003-protocol-v2-first.md) | V2-first | honest advertisement, V0/V1 declined | +| [003](decisions/003-protocol-v2-first.md) | V2-first | honest advertisement; V0/V1 declined on fetch, push V0-framed (ADR-013) | | [004](decisions/004-pack-pipeline.md) | Pack pipeline | generation on blocking threads, O(counts) | | [005](decisions/005-session-substrate-types.md) | Substrate types | request reader, sideband sink, http framing rules | | [009](decisions/009-bounded-resources-budget.md) | Budgets | `Limits` in every session tuple | | [010](decisions/010-pure-protocol-crate.md) | Pure protocol crate | wire layer is backend-trait-only | +| [013](decisions/013-receive-pack-state-machine.md) | receive-pack | V0-framed push machine, thin-pack acceptance, unpack-first CAS | +| [014](decisions/014-v2-negotiation-ack-loop.md) | Negotiation | ack loop, no `ready`, wait-for-done stays | ## Open Questions -- **OQ-02**: V2 multi-round negotiation (open — efficiency, not - correctness). -- **OQ-04**: receive-pack state machine + validation (deferred(unclear)). - **OQ-03**: publish/API freeze (partially resolved — single-crate shape settled by ADR-010). - **OQ-05**: sha256 policy (deferred(scope)). +- OQ-02 resolved (ADR-014 — ack loop design). +- OQ-04 resolved (ADR-013 — receive-pack state machine). ## References - `docs/research/poc-1-findings.md`, `docs/research/poc2-findings.md`, `docs/research/poc3-findings.md` (the normative wire behavior — observed against real git, not docs' grammar) +- `docs/research/push-captures.md` (the push-path normative record — + ADR-013's basis) +- `docs/research/negotiation-captures.md` (the negotiation normative + record — ADR-014's basis) - `docs/research/git-protocol.md` (inventory + observed corrections) - `docs/research/gitoxide.md` §"Wire format" (packetline contracts) - alktty `wire.rs`/`session.rs`/`adapter.rs` (the template's half shapes) \ No newline at end of file diff --git a/docs/research/README.md b/docs/research/README.md index 4d23fe5..92cc568 100644 --- a/docs/research/README.md +++ b/docs/research/README.md @@ -4,12 +4,17 @@ Phase 0 (exploration) research. Feeds phase 1 (architecture). | Doc | Topic | Status | |---|---|---| -| [vision.md](vision.md) | Vision, guiding principles, non-goals, phase-0 checklist | draft v2 (amended 2026-09-21) +| [vision.md](vision.md) | Vision, guiding principles, non-goals, phase-0 checklist | draft v2 (amended 2026-09-21) | | [gitoxide.md](gitoxide.md) | gitoxide (gix) capability + version alignment | initial pass complete | | [alk-stack.md](alk-stack.md) | alk stack fit, integration surface, gitea-lesson constraints | initial pass complete | | [git-protocol.md](git-protocol.md) | Server-side git smart protocol inventory (what we own) | initial pass complete | | [reference-policy.md](reference-policy.md) | Licenses, reference projects, reuse policy | complete | | [pocs.md](pocs.md) | POC plan (what to validate before architecture commits) | planned | +| [poc-1-findings.md](poc-1-findings.md) | POC-1: pkt-line over alkcall BiStream (duplex producer) | complete — proceed | +| [poc2-findings.md](poc2-findings.md) | POC-2: server-side pack generation (streaming pipeline) | complete — proceed | +| [poc3-findings.md](poc3-findings.md) | POC-3: smart-http shape through alkhttp (stateless substrate) | complete — proceed | +| [push-captures.md](push-captures.md) | Normative receive-pack (push) wire record (real git captures) | complete — ADR-013 basis | +| [negotiation-captures.md](negotiation-captures.md) | Normative V2 negotiation record (real git captures + source) | complete — ADR-014 basis | ## Key findings so far diff --git a/docs/research/push-captures.md b/docs/research/push-captures.md new file mode 100644 index 0000000..1e6ee9a --- /dev/null +++ b/docs/research/push-captures.md @@ -0,0 +1,132 @@ +# Research: receive-pack (push) walkthrough captures + +**Status**: complete +**Date**: 2026-09-25 +**Client**: git 2.43.0 +**Method**: real `git push` (file://, git:// daemon, and smart-http against +a ground-truth mock receive-pack server validated against real git +behavior), plus raw stdio requests driven into real `git receive-pack` for +policy-error ground truth. This is the normative push-path wire record +that ADR-013 is built on. Scratch logs were in /tmp/opencode/oq04/ (not +kept); every fact below is restated in full here. + +## Advertisement (GET info/refs + duplex) + +- Smart-http GET: `# service=git-receive-pack` + flush, then a **V0-shaped + ref advertisement**: ` \0` on the FIRST ref line only, + remaining ref lines bare, flush. The `capabilities^{}` zero-id sentinel + line replaces the ref lines ONLY when the repo has no refs. + Content-Type: `application/x-git-receive-pack-advertisement`. +- Duplex (git daemon): same ref advertisement WITHOUT the service prefix; + server speaks first; no version negotiation at all — receive-pack is + V0/V1-framed even for modern git (protocol.version=2 does not apply). +- Real receive-pack caps: `report-status report-status-v2 delete-refs + side-band-64k quiet atomic ofs-delta object-format=sha1 agent=git/2.43.0` + (+ `push-options` when receive.advertisePushOptions=true). The client + selects a subset NUL-attached on its FIRST command line (observed: + `report-status-v2 side-band-64k quiet object-format=sha1 agent=...`; + note the leading space after the NUL). +- git 2.43 rejects a bare capability dump (no refs, no sentinel) and + rejects a trailing `capabilities^{}` line when refs exist + (`fatal: unexpected capabilities^{}`). + +## Request (POST git-receive-pack) + +- Body: command pkt-lines ` ` (caps NUL-attached on the + first line only) + flush + [push-options section: bare pkt-lines + + flush, only when push-options was negotiated] + PACK stream (raw, to + end of body). +- Deletes: old= new=zero-id. Creates: old=zero-id. +- Shallow clients push `shallow ` lines BEFORE the command lines + (captured from a --depth 1 clone push). Real receive-pack ACCEPTS them + (unpack ok, refs applied) when the pack completes history; rejects with + `ng shallow update not allowed` when the server would have to + graft (receive.shallowUpdateDeepen unset). +- HTTP framing: POST Content-Type + `application/x-git-receive-pack-request`; small bodies Content-Length; + large (above http.postBuffer, git 2.43 default 1 MB) → chunked, and a + 4-byte `0000` probe POST answered 200-empty (same shape as upload-pack, + POC-3). +- git:// framing: first pkt `git-receive-pack /\0host=\0`, then + the advertisement (server speaks first, no service prefix), then the + request on the stream. + +## Thin pack + +- `git push` sends THIN packs by default (push.thin=true): the pack + excludes objects the client believes the server already has (observed: + the unchanged parent tree and its blob excluded; the new commit, tree, + blob sent). NO `thin-pack` capability is involved on push (that string + is a fetch arg). Ingestion MUST resolve delta bases from the server's + own odb — the `Option` no-lookup composition is only valid when + the pack is known-not-thin. +- Verified: `git index-pack --strict --stdin` FAILS on the captured pack + ("did not receive expected object"); `--fix-thin` succeeds. + +## Status report + +- When side-band-64k was selected: the report is band-1 pkt-line chunks; + the band payload is ITSELF pkt-line-framed: `unpack ok|ng `, + then per-ref `ok ` / `ng `, terminated by an INNER + flush pkt inside the band; the response then ends with an OUTER flush. + Without sideband (file:// local path): report lines are bare pkt-lines + to a single flush. +- Observed per-ref ng reasons (real receive-pack): `shallow update not + allowed`, `deletion prohibited`, `non-fast-forward`, `atomic push + failure` (rollback of other refs). `unpack ok` even when some refs fail; + pack-level failure → `unpack ng ` (git source: the index-pack + error text; not live-captured). +- Atomic: captured `ng refs/heads/dev deletion prohibited` + `ng + refs/heads/main atomic push failure` in one report (dev was the real + failure; main was rolled back). +- Human-readable diagnostics ride band-2 (`remote: error: denying + non-fast-forward refs/heads/main (you should pull first)`). +- Client aborts on framing errors: a report sent WITHOUT sideband when the + client selected side-band-64k → `fatal: protocol error: bad line length + character` (observed live) — the report MUST be sideband-wrapped when + side-band-64k was selected. + +## CAS / race semantics + +- The advertisement IS the client's CAS check: a stale old-id that does + not appear in the advertisement → client refuses to send (non-FF or + fetch-first message, NO commands transmitted). +- Server-side CAS is still required: races between advertisement and + apply, and clients that send wrong old-ids deliberately. old must match + current (or zero-id create) per ref; else `ng `. +- Empty command list (immediate `0000`) = client-side nothing-to-do. + +## Version/framing surface + +- No version line anywhere in receive-pack: no `version=2` exchange on + push; the push path is V0/V1-framed regardless of protocol.version. +- report-status-v2 selected by the client when offered; response shape is + identical to v1 for plain pushes (v2 adds option lines only when + options exist). +## Raw stdio captures into real `git receive-pack` (hand-built requests) + +- Corrupt pack: `unpack protocol error (pack signature mismatch detected)` + + `ng unpacker error` — unpack runs BEFORE ref checks; a failed pack + reports per-ref 'unpacker error' (not the detailed reason). +- Truncated request: `unpack eof before pack header was fully read`. +- Invalid ref name with a VALID pack: `unpack ok` + `ng refs/heads/../evil + funny refname` — name validation is per-ref, after unpack, with reason + 'funny refname'. (Client-side git refuses such refnames itself; this is + the malicious-client path.) +- Advertisement confirmed on the raw path: first ref line carries the caps + NUL-attached; `unpack ng ` at pack level, per-ref lines after. + +## Pack-read ordering (definitive) + +Real receive-pack ALWAYS expects a pack after the command flush (missing +pack → `unpack eof before pack header was fully read` + all refs +`unpacker error`). Per-ref checks (CAS, name validation, policy) run AFTER +the pack is indexed — observed: stale-old ng, 'funny refname' ng, 'deletion +prohibited' ng all arrive only after `unpack ok`. Fail-fast CAS before +reading the pack is an optimization, not the protocol order. + +## quiet semantics + +The client selects `quiet` on its first command line in every capture; band-2 +error lines still arrived (`remote: error: denying non-fast-forward ...`). +quiet suppresses progress chatter, not error/diagnostic bands. diff --git a/tasks/architecture/oq-04-receive-pack.md b/tasks/architecture/oq-04-receive-pack.md index 9599743..a9f42ff 100644 --- a/tasks/architecture/oq-04-receive-pack.md +++ b/tasks/architecture/oq-04-receive-pack.md @@ -1,42 +1,45 @@ --- id: architecture/oq-04-receive-pack name: OQ-04 unblock — receive-pack walkthrough/POC arrives -status: pending +status: completed depends_on: [] scope: narrow risk: trivial impact: component level: research -tags: [external-trigger, deferred-oq] +tags: [external-trigger, deferred-oq, resolved-early] --- ## Description Tracker for OQ-04 (`deferred(unclear)`): receive-pack (push) design -validation. This task is not actionable work — it tracks whether the -investigation has arrived. When a receive-pack design walkthrough (against -real `git push` captures) or a validation push POC is scheduled, mark this -completed; OQ-04 transitions to `open`. The investigation scope is in -`docs/architecture/open-questions.md` (OQ-04): capability advertisement -set, request-line parsing + shallow policy, thin-pack acceptance, CAS -timing, status report, http framing. +validation. **Resolved early (2026-09-25, ADR-013)** — the investigation +was run in-session rather than waiting for scheduling: real `git push` +captures (git 2.43.0 over file://, git://, smart-http, plus raw stdio +requests into real `git receive-pack`) resolved the full push state +machine. OQ-04 is resolved by +`docs/architecture/decisions/013-receive-pack-state-machine.md`; captures +in `docs/research/push-captures.md`. ## Work None by default. On trigger: set OQ-04 to `open` in `docs/architecture/open-questions.md` and spawn the focused session that -works through the push state machine. +works through the push state machine. (Triggered and completed in the +2026-09-25 session.) ## Verification - OQ-04 status matches this task's state in - `docs/architecture/open-questions.md`. + `docs/architecture/open-questions.md` (resolved). ## Out of scope -- Actually designing/implementing receive-pack (that is the follow-on - session's work). +- Actually implementing receive-pack (phase 3 implementation work). ## Summary -> Filled on completion. \ No newline at end of file +Resolved early via ADR-013: the walkthrough ran in-session; all seven +unknowns (advertisement set, shallow policy, thin-pack acceptance, +ingestion composition, CAS timing, status report, version/framing +surface) have capture-grounded decisions. \ No newline at end of file diff --git a/tasks/architecture/oq-13-cas-failfast.md b/tasks/architecture/oq-13-cas-failfast.md new file mode 100644 index 0000000..2595288 --- /dev/null +++ b/tasks/architecture/oq-13-cas-failfast.md @@ -0,0 +1,45 @@ +--- +id: architecture/oq-13-cas-failfast +name: Backlog — CAS-before-unpack fail-fast pre-check on push +status: pending +depends_on: [] +scope: narrow +risk: low +impact: component +level: implementation +tags: [optimization-backlog] +--- + +## Description + +Optimization backlog item from ADR-013 (§7, §Consequences): a pre-unpack +CAS pre-check on receive-pack — reject commands with obviously-wrong +old-ids before reading the pack body, failing fast instead of ingesting +a pack whose refs would be rejected anyway. The protocol-correct order +(unpack-first, then per-ref checks) is what v1 implements; this is an +optimization, additive, and two-way (internal behavior, not wire shape — +the report shape is unchanged). + +## Work + +When receive-pack implementation is on the critical path for a +deployment with heavy stale-push traffic (measured, not hypothetical): +add the pre-check to the transport push state machine, after command +parse and before ingestion. Rejected refs report the same `ng +`; the pack body is then drained (or the connection closed per +the substrate's error rule). + +## Verification + +- All existing push tests still pass (real `git push` against the + transport). +- A stale-old push errors before the pack body is fully read (observable + via the body-budget counters or a truncated-body test). + +## Out of scope + +- Changing the report shape or the protocol order (ADR-013 §7 stands). + +## Summary + +> Filled on completion. \ No newline at end of file