From 31207057ffef40d8c7b43654592575deb748f7f9 Mon Sep 17 00:00:00 2001 From: "glm-5.3-flash" Date: Mon, 26 Oct 2026 06:09:38 +0000 Subject: [PATCH] =?UTF-8?q?docs(research):=20phase=200=20closed=20?= =?UTF-8?q?=E2=80=94=20pocs.md=20refreshed,=20reverse-runner=20pattern=20e?= =?UTF-8?q?xtracted?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - pocs.md: marked complete; corrected the stale 0.5x-era accept-defaults hypothesis wording (0.64.1: only auth_publickey_offered defaults to accept), stage D recorded as ran-in-full, completion gate updated with per-question resolution status - added client-side scope section: no dedicated POC needed (client API exercised throughout POC-1/POC-3) + the acceptor-default fact (russh client accept-and-drop for unhandled forwarded opens is ours to fix) - new reverse-runner.md: alkminer pattern extraction (concepts only, never code) — hub-dials-runner surface inventory, both tunnel legs, the OAI-proxy secret-hygiene shape, and the client-direction lessons (keepalive defaults, reconnect-restore, acceptor disposition) - vision.md: scope-question resolution addendum + whitespace typo fix --- docs/research/pocs.md | 71 ++++++++++++++++----- docs/research/reverse-runner.md | 109 ++++++++++++++++++++++++++++++++ docs/research/vision.md | 11 +++- 3 files changed, 173 insertions(+), 18 deletions(-) create mode 100644 docs/research/reverse-runner.md diff --git a/docs/research/pocs.md b/docs/research/pocs.md index 8847cdb..d96c559 100644 --- a/docs/research/pocs.md +++ b/docs/research/pocs.md @@ -1,8 +1,11 @@ # POCs — phase 0 validation plan -Status: Draft (phase 0 research). Deliverables of each POC are -findings docs (`docs/research/poc-N-*.md` when run); POC source is -disposable and never merges to main. Mode for all: **standalone** +Status: **Complete** (phase 0). All three POCs ran against the current +published russh (0.64.1) and passed; the findings docs +(`poc-1-auth-bridge.md`, `poc-2-vault-keys.md`, +`poc-3-mapping-bakeoff.md`) are canonical and supersede the planned +hypothesis wording below wherever they record corrections. POC source +is disposable and never merges to main. Mode for all: **standalone** (repos have no code to build on yet) — scratch cargo projects at `/workspace/` unless a POC explicitly builds on repo code (then branch mode per `docs/sdd_process.md`). @@ -38,9 +41,11 @@ keys; russh client available as a secondary client for - `auth_publickey_offered` disposition (accept-and-let-decide) behaves with OpenSSH's probe-then-sign flow. - Decline matrix: `auth_none`/`password`/`keyboard-interactive` - declined (default impls are *accept* — every default must be - overridden and proven); `Config::methods` advertises `publickey` - only. + declined (0.64.1 correction: the *default impls already reject* at + this russh — only `auth_publickey_offered` defaults to accept; + `poc-1-auth-bridge.md` §3. The family contract still requires + implementing every hook explicitly — the pin is a one-way door); + `Config::methods` advertises `publickey` only. - `ChannelOpenHandle` drop = auto-reject semantics; unfeatured channel-type declines observable by vanilla `ssh` (`ssh host` session-open refused, `-L` refused, clean diagnostics). @@ -127,11 +132,14 @@ forms). russh server loopback + vanilla clients; per stage: real `ChannelClient` vs local-dial stand-in). Whichever route the POC exercises informs the phase 1 ADR on the presence/absence of a serve-over-stream entry in alktunnels. -- **Stage D — session/pty mapping**, last and optional (a real PTY - backend needs alktty's `local` feature; the wire-level questions are - already largely answered by A/B/C): shell over - `drive_session` with the locally-constructed `NegotiateRequest`, - window-change and signal mapping. +- **Stage D — session/pty mapping**, final stage (ran in full): shell, + exec, window-change and signal mapping against a real PTY + (`LocalTtyBackend`, alktty `local`). The "optional" hedge in the + original plan was unnecessary — stage D completed 13/13 and produced + the load-bearing session-row corrections: the alktty bridge is a + **translation bridge** (chunk framing both directions, not a byte + pump) and the alktty exit chunk carries no signal identity + (`poc-3-mapping-bakeoff.md` §Stage D). **Validates:** the one-seam-per-mapping hypothesis; the parse-then- route dispatch; stderr semantics in pipe vs PTY mode; EOF/close/signal @@ -142,18 +150,47 @@ channels reassembly cap on the bridged route). fixups recorded; stage D may surface PTY-merge stderr surprises that reshape the request-level mapping rows. +## Client-side scope (post-POC addition) + +The three POCs are server-door-centric by design, but the crate has a +**client direction** of equal scope: a family app dials an *upstream* +vanilla-SSH server and consumes its surface (exec/pty channels, +`direct-tcpip` opens, the `tcpip_forward` + accepted-`forwarded-tcpip` +listener flow). No dedicated POC was needed: the POCs exercised the +client API throughout (POC-1's russh-client auth leg, POC-3 §D.3's +client-side request mechanics — `Channel::exec()`, the `wait()`-drain +trap, fire-and-forget `window_change`/`signal` before `into_stream()`). + +Two client-side facts phase 1 designs from: + +1. **The acceptor rule.** The only server-initiated channel type in the + pattern is `forwarded-tcpip`, and russh's client default for an + unhandled forwarded open is *accept-and-drop* — which is not the + family fail-closed posture. The client-side accepted-open + disposition (accept into a registered handler vs decline) is + alkssh's contract to define. +2. **A real consumer pattern exists**: the alkminer hub dialing rented + GPU containers (exec + job management, SFTP binary deploy, `-L` and + `-R` tunnel legs in both directions, keepalives tuned for NAT'd + hosts). Extracted (concepts only, never code) in + `reverse-runner.md`. + ## Sequencing and gates 1. POC-1 first — everything downstream presumes its auth plumbing. 2. POC-2 after POC-1 (reuses POC-1's server loopback). 3. POC-3 last; stage A gates phase 1's mapping ADR work — it is the path alkforge phase 0 consumes first. -4. Phase 0 completes when the findings docs exist and the OQ list in - `vision.md` §scope questions is either answered by findings or - carried into phase 1's `open-questions.md` as seeded OQs. POC-1/2/3 - findings docs now exist; scope question 5 (mapping-table details) - is answered per `poc-3-mapping-bakeoff.md` §hypothesis verdicts, - and the POC-seeded OQs are listed in each findings doc. +4. Phase 0 completed when the findings docs existed and the OQ list in + `vision.md` §scope questions was either answered by findings or + carried into phase 1's `open-questions.md` as seeded OQs. Done: + POC-1/2/3 findings docs exist; scope question 5 (mapping-table + details) is answered per `poc-3-mapping-bakeoff.md` §hypothesis + verdicts; scope question 4 (auth dispositions + client-side vault + keys) is answered per POC-1/POC-2; scope question 2 (russh pin + facts) is verified; questions 1 (the `alk/ssh` topology) and 3 + (feature naming) are decision territory carried as phase 1 OQs — + the seeded OQ list is in each findings doc. ## References diff --git a/docs/research/reverse-runner.md b/docs/research/reverse-runner.md new file mode 100644 index 0000000..bd54faa --- /dev/null +++ b/docs/research/reverse-runner.md @@ -0,0 +1,109 @@ +# Reverse runner — client-direction pattern extraction (alkminer) + +Status: phase 0 post-POC extraction. **Reading aid only — concepts and +wire shapes, never code** (the source is not open-sourced and stays +that way; per repo license hygiene, reference code is for reading and +facts). Source: +`/workspace/@alkminer/alkminer` — specifically +`crates/alkminer-host/src/remote/{ssh,exec,forward,manager,sftp}.rs` +and `src/oai/mod.rs`. Also old (russh 0.62 era) and POC-grade; its +anti-patterns are as instructive as its patterns, and are named below +as anti-patterns, not precedents. + +## The direction + +The reverse-runner pattern inverts the typical runner flow: the +**hub/coordinator is the SSH client** and dials out to vanilla `sshd` +on rented containers (vast.ai, runpod — any box that exposes SSH). The +runner exposes nothing but the wire; the hub is the active party and +pushes all work and all channel opens. + +Security shape (why this beat the alternatives in practice): + +- **No inbound surface on runners** — only `sshd` with one key. +- **Secrets never leave the hub** — the pattern's killer use was an + OAI-compatible proxy served *by the hub* (`oai/mod.rs`, alkminer + ADR-016): the instance points its LLM tooling at a localhost URL + that rides the tunnel to the hub's proxy, which injects the real + provider key. The runner authenticates to the proxy with a scoped + dev token — least privilege, no upstream-secret custody. +- **Reachback both ways, deliberately**: hub→runner services via `-L`, + runner→hub services via `-R` (the two legs below were both used, + sometimes chained: instance port ‑L→ hub port ‑L→ dev workstation). + +## Per-connection surface (what the hub actually drove) + +One authenticated SSH connection per instance carried: + +| Surface | alkminer use | alkssh client-direction translation | +|---|---|---| +| blocking `exec` | provisioning commands, health checks | session channel open + `exec()`; alktty pipe-mode seam when the runner is family-built | +| background jobs (`start_job`/`job_status`/`job_logs`/`stop_job`) | mining worker lifecycle; pid files on the runner, job rows in hub-side storage | assembly-level job supervision over exec channels; the alkminer pid-file approach is the ad-hoc analog of POC-3 §D's pipe-mode signal gap (alktty exit-chunk signal identity OQ) | +| SFTP | binary deploy to the instance | **future alkfs client half** — real consumer evidence that the `sftp` feature's deferred implementation needs both directions (alkforge CI artifact deploy is the consumer) | +| `-L` client-side tunnel | opencode dev tunnel (instance:4096 → hub) | client opens `direct-tcpip` per accepted local connection and pumps (POC-1's `into_stream` plumbing, reversed) | +| `-R` client-side tunnel | hub-service reach tunnel (instance → hub's HTTP/OAI proxy) | `tcpip_forward` global request (returns the bound port — no mpsc sink param, matching `russh.md`'s client inventory) then **accept the server-initiated `forwarded-tcpip` opens** and relay each to the hub-local target | + +## Client-direction facts that phase 1 designs from + +1. **The acceptor row is unavoidable and its disposition is ours.** russh's + client handler receives `server_channel_open_forwarded_tcpip` for + `-R` data. In alkminer's 0.62-era code (and unchanged in kind at + 0.64.1), an accepted forwarded channel with no handler installed is + **accept-and-drop** — a fail-open-ish default. alkssh's client + contract: only channel types/features the assembly registered are + served; otherwise decline cleanly. This is the client side of the + feature-graph symmetry rule; unlike the server auth defaults (which + 0.64.1 already has rejecting), the client acceptor default is + *ours to enforce*. +2. **The routing sink installs post-connect.** alkminer's shape: the + handler exposes a late-bound handler slot + (`install_forward_sink`) set after `connect` by the consumer that + owns the forward. The family translation is an assembly-injected + acceptor-handlers registry (per channel type / per feature), not a + static config — but the *late-install* dynamic is real: consumers + turn capabilities on/off over a live connection. +3. **Keepalives are load-bearing on cloud runners.** The hub needed + `keepalive_interval` 15 s + `keepalive_max` 3 (OpenSSH + `ServerAliveInterval=15`/`ServerAliveCountMax=3` parity) — with + russh's default (no keepalives), dead NAT'd connections sat + unnoticed until a health-check exec failed. Evidence for the + client-side bounded-resources defaults (the server side has + symmetric knobs; cloud edges need both). +4. **Reconnect-and-restore semantics.** Tunnel ports were persisted + (hub storage) and re-bound identically on reconnect; forwards were + tracked per connection with idempotent stop handles. This is + assembly policy (session/forward lifecycle), not wire contract — + kept in view so phase 1 doesn't accidentally bake reconnect policy + into the crate. +5. **Host-key pinning anti-pattern.** alkminer's `check_server_key` + returned accept-all (TOFU-without-the-F, 0.62 default posture + overridden to the permissive extreme for rental fleets). The family + rule (no TOFU; expected host fingerprints pinned at the assembly + layer) is right, and this code is the working exhibit of why. The + *convenience* need is real (fleets churn); the resolution is + pinning policy at the assembly, not accept-all in the crate. +6. **Per-runner key registration.** Keys were minted per instance and + registered hub-side (their provisioning flow, including a real + gitea client for dev workspaces) — the deployment precedent for + alkforge's per-runner `IdentityProvider` rows. + +## Not carried + +- No code extraction in either direction (concepts/wire shapes only). +- The job-supervision storage model (hub-side SQLite job ledger, pid + files on the runner) belongs to the assembly layer in a family + deployment; the crate's surface is channels and tunnels. +- No per-target scoping of proxied traffic existed (dev token proved + "a valid dev instance", nothing more) — a real least-privilege gap + the family model fixes via per-identity ACLs on the service side. + +## References + +- `pocs.md` §Client-side scope (the scope item this doc grounds) +- `mapping-table.md` (server-direction rows; POC-3 stage C is the + server side of the same bridge legs), `poc-1-auth-bridge.md` §7 + (the channel↔stream plumbing both leg styles reuse), + `poc-3-mapping-bakeoff.md` §D.3 (client-request mechanics) +- `auth-bridge.md` §Host keys and `check_server_key` (the pinning + rule this doc's anti-pattern motivates) +- alkminer ADR-016 (the OAI proxy pattern — secrets stay at the hub) \ No newline at end of file diff --git a/docs/research/vision.md b/docs/research/vision.md index 5f5c264..e9b00c0 100644 --- a/docs/research/vision.md +++ b/docs/research/vision.md @@ -50,7 +50,7 @@ not protocol invention. exactly one point per mapping. No channel-type knowledge flows upstream into the protocol crates. 5. **Identity is the family's, not SSH's.** SSH user names, `~/.ssh/` - conventions, and password auth are not the alk family's identity + conventions, and password auth are not the alk family's identity model. A verified key's fingerprint → `Identity` via alkcall's `IdentityProvider` is the only resolution path; secrets an alkssh *client* holds come from alkvault-backed sources — vault-encrypted @@ -102,6 +102,15 @@ phase 1's `open-questions.md`: the family seam, the parameter mapping, and the decline contract (see `mapping-table.md`); POC validation per `pocs.md`. +Resolution status (post-POC): question 4 is answered by +POC-1/POC-2 evidence, question 5 by POC-3 (three mapping-row +corrections recorded), and question 2's *facts* are verified (the pin +decision itself is phase 1 ADR territory). Questions 1 (the `alk/ssh` +topology) and 3 (feature naming) are phase 1 decisions, carried as +seeded OQs / ADR territory per `pocs.md` §Sequencing and gates. A +client-direction consumer pattern (hub-dials-runner) is extracted in +`reverse-runner.md` and shapes the phase 1 client surface. + ## References - family ADR-003 (`/workspace/@alkdev/alkfamily/docs/architecture/decisions/003-alkssh-russh-adapter.md`)