From 17e041e149cfbd9fce9f66df0eef12336308a138 Mon Sep 17 00:00:00 2001 From: "glm-5.3-flash" Date: Sat, 7 Nov 2026 13:39:39 +0000 Subject: [PATCH] docs: host-access pass closes the deployment walk residues - 52 repos (35 public + 17 private incl. alkfamily): both halves of ADR-002's scope sentence are already operational; new-repo default is private - visit patterns: web surface is crawler/scan noise (42k reqs, ~12.8k IPs over 2 days); real git transport is near-100% SSH - non-git SSH usage: none authorized in 72h; pure git door - key inventory: ~10 accounts, 12 ed25519 keys, no tokens/webhooks/ protected branches; DB-verified zero usage of issues/PRs/LFS/ actions across both halves - migration sizing corrected: ~158 MB git on disk (DB size column stale); backup locations remain a cutover-plan verification - OQ-FG-09 gains a live precedent: blob-heavy crawls burned gitea CPU until bounded --- docs/research/deployment-walk.md | 198 +++++++++++++++++++++++-------- docs/research/phase-0.md | 54 +++++---- 2 files changed, 183 insertions(+), 69 deletions(-) diff --git a/docs/research/deployment-walk.md b/docs/research/deployment-walk.md index 097b3f7..9a5b3c0 100644 --- a/docs/research/deployment-walk.md +++ b/docs/research/deployment-walk.md @@ -33,11 +33,11 @@ What was inspected, and how it was kept behavior-only: topology including the gitea container next to it. These are the repo's deploy examples rather than the live host's files, so they are treated as indicative topology, not ground truth (see sect 8). -- **Not inspected**: gitea source, the host's `app.ini` or its access - logs: this environment has no host shell, and the behavior-only rule - excludes the source in any case. Gitea's version (28.3.0) came from - the deployment's own public API endpoint, which is itself a walk - finding. +- **Not inspected at any stage**: gitea source (license posture, + AGENTS.md convention 13). The host-side pass was read-only over + logs, the gitea Postgres, and filesystem sizes; gitea's runtime + configuration was consumed through its documentation in the + operator docs rather than by reading its config file. Pacing note (rate-limit safety): every live request ran sequentially with small sleeps; total live traffic for the walk was under 100 @@ -57,15 +57,21 @@ requests and one 60 MB of mirror fetches. Nothing bursted. Two structural observations the census already supports: -1. **The "web view for the OSS projects, public clones for them" ADR - scope sentence undershoots present reality.** The deployment is not - "some OSS projects public": the entire org, all 35 repos, is - anonymously readable, through the web UI, the JSON API, and both - git transports (verified anonymous smart-protocol HTTP clone). The - existing public surface is *everything the forge serves*, web and - git alike. A per-repo public flag (phase-0 sect 5 draft posture) is - a strict refinement of a currently coarser de-facto org-wide - anonymous-read grant, not a departure. +1. **The ADR-002 model is already operational, and the API census only + showed its public half.** The deployment holds **52 repos: 35 + public ("web view + public clones for the OSS projects") and 17 + private ("SSH git transport for everything else")**, including + `alkdev/alkfamily`, the family's own coordination repo. New-repo + default on the deployment is private, and the anonymous API + enumerates only the public half (the private half is invisible to + the API), so the "org-wide anonymous read" observation applies to + the public 35: through the web UI, the JSON API, and both git + transports (verified anonymous smart-protocol HTTP clone). The + per-repo public flag (phase-0 sect 5 draft posture) is a strict + refinement of a currently coarser de-facto org-wide + anonymous-read grant on the public projects, not a departure; the + private half confirms the "everything else private" half of the + ADR sentence is load-bearing too. 2. **The anonymity is unauthenticated by construction.** An anonymous visitor gets repo listings, file trees, server version, user roster, and full clones. This is the operative peer model today, @@ -132,10 +138,12 @@ returns 404; the `has_wiki`/`has_issues` flags are on but empty: default-flag inertia), 0 stars/forks/watchers. The forge is used as a git host and a landing page. Nothing else. -## 4. Repo-content census (the usage evidence) +## 4. Repo-content census (the usage evidence: public half) Full data in the walk notes; summarized per repo (branch count -excludes `main`; commits counted across all refs): +excludes `main`; commits counted across all refs). This table covers +the 35 public repos from the first-pass API census; the private half +is covered in the host-access addendum (sect 10): | Repo | Br | Tags | Commits | Activity window | Notes | |---|---|---|---|---|---| @@ -224,6 +232,10 @@ inertia, or deferred. five agent identities, one key each: admin-provisioned, invite-to -work territory, exactly OQ-FG-04's expected small set. 8. No README is normal: the web view cannot assume one. +9. **The private half is load-bearing too**: SSH git transport with + per-identity write to private repos (17 of 52; `alkfamily` and + friends), where "everything else private" is already the daily + operative model (new-repo default private on the deployment). **Inertia (dropped in 0.1, with a migration note):** @@ -272,12 +284,14 @@ inertia, or deferred. phase-0 sect 5 remains right for the model; present reality is "one org, everyone in it writes `main` everywhere". Migration: six writable identities plus the human admin. -- **Migration shape (OQ-FG-07).** ~60 MB, 35 repos, 2607 commits, - 87 tags, two import histories: `mirror fetch + push` through the - protocol halves is plainly sufficient for all of it. Nothing needs - gitea-side export formats; account/key data regeneration is the - only non-git work (seven accounts, seven-plus keys, alkvault - references). +- **Migration shape (OQ-FG-07).** On-disk git data is ~158 MB + across all 52 repos (public ~60 MB, private ~100 MB; the private + repos are mostly small research/workspaces, same size class as the + public ones), plus 87+ public-half tags and import histories: + `mirror fetch + push` through the protocol halves is plainly + sufficient for all of it. Nothing needs gitea-side export formats; + account/key data regeneration is the only non-git work (about ten + accounts, twelve ed25519 keys, alkvault references). - **Anonymous exposure shrinks massively.** Today an anonymous peer sees the gitea web app, the JSON API, the user roster, and the exact server version. The 0.1 web view plus nameless git serving @@ -304,35 +318,28 @@ inertia, or deferred. ## 8. Not verifiable from outside (open verification items) -These need host access or user knowledge; they do not block the 0.1 -spec as walked, but each is a residual risk on its assumption: +These needed host access or user knowledge. Items 1, 3, 4, and 5 were +resolved by a same-day host-access pass (sect 10); items 2 and 6 +remain open: -1. **Visit/traffic patterns**: no logs available in this - environment. Which web pages are actually visited (repo pages vs - API vs explore), and how much web traffic the clone doors get, is - unmeasured. The web view sizing in sect 6 rests on the surface - being *exposed*, not on visit data. Measure at reverse-proxy - (`/var/log/reverse-proxy/access.log`) before freezing the 0.1 web - spec if there is any doubt. +1. ~~Visit/traffic patterns~~: resolved (sect 10.1). 2. **How the outside links in** (what URLs get pasted/linked externally: file permalinks? repo pages?): no observation path - from outside. If external links to blob lines exist, the web view - needs line-permalink stability sooner than 0.1 assumptions allow. -3. **Host-side configuration** (`app.ini` customizations beyond - defaults, whether issues/wiki were ever configured and abandoned - vs never configured): the flags-on-but-empty pattern suggests - never-set-up-then-inertia, unconfirmed. -4. **SSH key inventory** (how many keys per account, key material - age/algorithm) and **who else holds keys**: observable only at - the host; the account roster (7 accounts) came from the public - API instead. -5. **Non-git SSH usage** (tunnels/port-forwards via the SSH door): - the door's structure (gitea, forced git command, no shell) - strongly suggests none; nothing in the family repos or deploy - artifacts indicates a shell-capable channel. Confirm at cutover - so no silent dependency breaks. -6. **Backup layout** (the "three locations" from ADR-002's context), - so migration can verify data-integrity posture before cutover. + found, even from the host logs (no referer data). If external + links to blob lines exist, the web view needs line-permalink + stability sooner than 0.1 assumptions allow. +3. ~~Host-side configuration~~: resolved: documented posture in the + system docs and confirmed by the flags-on-but-empty pattern: + forge features (issues/wiki/PRs) were configured then simply never + used (0 rows DB-wide, see sect 10.4). +4. ~~SSH key inventory~~: resolved (sect 10.3). +5. ~~Non-git SSH usage~~: resolved: none authorized in the sampled + window; the SSH door is a pure git door (sect 10.2). +6. **Backup layout** (the "three locations" from ADR-002's context): + the dump tooling is documented in the system docs; the live + off-host copy layout was not verified from dev1. Verify copy + locations before the migration cutover so the cutover has a + verified restore point. ## 9. Provenance of numbers @@ -344,3 +351,98 @@ spec as walked, but each is a residual risk on its assumption: - Error-mode and no-shell behaviors: live SSH probes, same day. - Deployment topology: `reverse-proxy/deploy/docker-compose.yml` + `deploy/README.md` (repo examples, confirm at host). +- Everything in sect 10: host-side pass on dev1, same day + (reverse-proxy access logs, gitea container logs 72h window, + gitea Postgres read-only queries, on-disk `du`). + +## 10. Host-access addendum (dev1, 2026-11-07) + +A paced, read-only pass on the gitea host (system docs + logs + +gitea's Postgres) to close the sect 8 residues. Machine-identifying +details stay in the system docs; only behavior-relevant facts here. + +### 10.1 Visit/traffic patterns (reverse-proxy access logs, 5-day sample) + +The anonymous web surface is overwhelmingly machine traffic. In the +sample, roughly 42k requests (2-day sample) from ~12.8k unique IPs, +with the median IP contributing a couple of requests: distributed +crawler/scan noise. Concentration highlights: a handful of repeat +clients (one contributing ~2k requests: Google's crawlers account +for most of the identifiable remainder), a single external client +polling the issues pages of about six repos hundreds of times each +(despite all repos having zero issues: unchanged-page monitoring +behavior), ~440 anonymous `/api/v1/version` fingerprint probes, and +a `/user/login` stream that is scan noise. + +Classed by what is being fetched (GET/POST over 5 days): file +browse (~34k), assets (~24k), commit/history pages (~21k), +other-repo pages (~17k), issue/wiki page fetches (~8.5k, all +zero-content monitoring/scan), login/auth (~6.8k), the rest small. +By contrast, the actual git-over-HTTP transport (`.git/info/refs` + +`git-upload-pack`) shows only ~18 requests (and at least one +genuine external anonymous clone in the gitea logs), and the RSS +feeds get a steady small stream. **Conclusion: the web view's +real-visitor value is as the org's public frontend; human/agent +page volume is a rounding error next to crawler noise, and the git +doors are almost entirely SSH.** + +Operational lesson already learned live (recorded in the system +docs and worth alkforge sizing from): blob-heavy crawl paths +(`/raw`, `/blame`, corpus files) once caused sustained gitea CPU +burn until robots rules and firewall drops put it back to baseline. +OQ-FG-09's bounded-resource posture (per-IP rate limits, body/size +caps, cheap paths for blob serving) has a real incident precedent. + +### 10.2 SSH door (gitea container logs, 72h window) + +- Authorized SSH sessions decompose into pure git verbs: ~41 + `git-receive-pack` (write) and ~40 `git-upload-pack` (read) + authorizations, balanced read/write. Zero non-git authorized + sessions in the window: the door is a pure git door, confirming + the walk's structural read. +- Writes concentrate on the live repos (`alkgen`, `alkjs`, + `alkfamily`, `alkforge`, `alkssh`), i.e. exactly the repos with + visible recent activity; the rest of the write authorization log + is at most one action per repo. +- ~2k failed SSH auth attempts in 72h from rotating addresses: + background internet scan noise against port 22, absorbed by + fail2ban as designed. Not a target pattern, just the ambient load + the door always carries. + +### 10.3 SSH key inventory (gitea Postgres, read-only) + +- Accounts: 10 rows in the account table: the human admin + (`alkimiadev`, 2 keys), `bot` (admin, 0 keys), four named agent + identities (`glm-5.1/5.2`, `deepseek-v4-pro/flash`; one key each + except `glm-5.2` with 7), `glm-5.3-flash` (this session's key), + plus two accounts tied to private-repo owner identities and the + org row. All 12 registered keys are ed25519. No access tokens, no + webhooks, no deploy keys, no protected branches, no protected + tags: any of these would be forge features alkforge never needs to + adopt beyond its own model. +- The private half (17 repos, sect 2) is served entirely through + this same key/identity path; there is no separate private-door + mechanism to preserve. + +### 10.4 Feature-usage ground truth (gitea Postgres) + +DB-wide row counts settle any "maybe the private half uses it" +question: **0** issues, pulls, reactions, LFS objects/locks, access +tokens, webhooks, action runs/runners, protected branches/tags. +Releases: only gitea's auto-created tag rows (65 across 13 repos, +matching the tags the census already counted: no real release +objects with notes/attachments anywhere). The DB's per-repo size +column is stale post-migration and disagrees with on-disk reality +(correct figure: ~158 MB git on disk, sect 6). + +### 10.5 Confirmed host posture relevant to the migration + +- New-repo default is private; registration is disabled (admin + provisioning only), matching OQ-FG-04's expectations exactly. +- The gitea binary itself has been on a fast CVE-patch cadence + (three recent version upgrades to clear security batches), + consistent with ADR-002's standing-exposure motivation. +- No recurring on-host backup job exists (dump tooling documented in + the system runbooks instead): the sect 8.6 verification of the + off-host "three locations" stays open until the cutover plan + verifies a restore point. diff --git a/docs/research/phase-0.md b/docs/research/phase-0.md index 6604766..b5d9ddb 100644 --- a/docs/research/phase-0.md +++ b/docs/research/phase-0.md @@ -112,22 +112,35 @@ method, per-repo census, and three-bucket classification are in - The forge is used as a git host and a landing page, nothing else: 0 issues, 0 PRs, 0 wiki pages, 0 gitea releases, 0 stars/forks - org-wide across all history; no LFS (heaviest blob ~3.2 MB); no - submodules; no forge-side CI. Issues/PRs/wiki/releases are inertia. -- All 35 `alkdev` repos are anonymously readable today (web, JSON - API, HTTP clone, SSH read); the de-facto public-read grant is - org-wide, and the anonymous peer class is the deployment's - dominant consumer class (OQ-FG-03 confirmed operative). -- Development is single-branch (`main`) with direct pushes by five - per-agent accounts plus a human admin and a `bot`; no PR flow. - Tags are the release surface (87 semver tags, per-crate prefixed). - `typebox`/`typemap` carry full-history upstream imports. -- Migration (OQ-FG-07) is easy: ~60 MB total, 2607 commits, + org-wide across all history (DB-verified, both halves); no LFS + (heaviest blob ~3.2 MB); no submodules; no forge-side CI, tokens, + webhooks, or protected branches. Issues/PRs/wiki/releases are + inertia. +- The deployment holds 52 repos: 35 public (the OSS half: anonymous + web, API, HTTP clone, SSH read) and 17 private (the "SSH git for + everything else" half: `alkfamily` and the other non-OSS work, + served through the same per-identity key path). Both halves of + ADR-002's scope sentence are already operational reality. +- Development is single-branch (`main`) with direct pushes by per + -agent accounts plus a human admin and a `bot`; no PR flow; SSH is + near-100% of real git transport (only ~18 git-over-HTTP requests + in the 5-day sample, ~2k probe-scans absorbed by fail2ban in 72h). + Tags are the release surface (87 semver tags, per-crate + prefixed). `typebox`/`typemap` carry full-history upstream + imports. +- The web surface is dominated by crawler/scan noise (~42k requests + from ~12.8k IPs in 2 days; median IP ~2 requests); human/agent web + volume is negligible, so the minimal web view is correctly minimal. + A live precedent (blob-heavy crawls burning gitea CPU until + bounded) feeds OQ-FG-09's limits. +- Migration (OQ-FG-07) is easy: ~158 MB on disk across all 52 repos, mirror-fetch + push suffices; account/key regeneration is the only - non-git work (7 accounts). -- Residue that cannot be observed from outside (visit patterns, - external inbound links, host config, key inventory): recorded in - the walk doc's sect 8 with the measurement point for each. + non-git work (about ten accounts, twelve ed25519 keys). +- Residue: mostly closed. A host-access pass (read-only: logs, + Postgres, on-disk sizes) resolved the visit-pattern, host-config, + key-inventory, and non-git-SSH questions (walk doc's sect 8, sect + 10). Still open: external inbound-link conventions and the + off-host backup-location verification for the cutover plan. The original provisional table and rules below are preserved for traceability; the filled-in version superseding them is in the walk @@ -275,12 +288,11 @@ open-questions.md` in Phase 1. Order below is rough priority. Section 3's walk. Blocks the 0.1 feature freeze, the migration plan, and the web-view spec. Highest priority; everything downstream sizes -from it. **Resolution (2026-11-07): walked and answered modulo the -outside-observable residues in the walk doc's sect 8** (visit -patterns, external inbound links, host-side config, SSH key -inventory, non-git SSH confirmation, backup layout: none of which -blocks the 0.1 spec as walked). See -[`deployment-walk.md`](deployment-walk.md). +from it. **Resolution (2026-11-07): walked and answered.** An +outside-in behavior walk plus a read-only host-access pass covers +everything except external inbound-link conventions and the off-host +backup-location verification (both cutover-plan inputs, not spec +blocks); see [`deployment-walk.md`](deployment-walk.md) sect 8/10. ### OQ-FG-02: What is alkforge 0.1's binary/service shape?