From f5e0e8e19980e9eb7ed380aa084100767cb54d08 Mon Sep 17 00:00:00 2001 From: "glm-5.3-flash" Date: Sun, 11 Oct 2026 01:24:14 +0000 Subject: [PATCH] =?UTF-8?q?release-deployment-matrix:=20deployment=20matri?= =?UTF-8?q?x=20finalized=20as=20release=20text=20=E2=80=94=20every=20row?= =?UTF-8?q?=20verified=20as=20implemented=20(corrections:=20consumer-side?= =?UTF-8?q?=20FULL=20durability=20option=20was=20false=20=E2=80=94=20synch?= =?UTF-8?q?ronous=20hardwired=20NORMAL=20at=20bootstrap,=20no=20surface;?= =?UTF-8?q?=20both-engines=20->=20all-three=20queue=20stamps;=20max=5Fsize?= =?UTF-8?q?=20one-guard=20understated=20->=20three=20guards=20listed;=20NF?= =?UTF-8?q?S=20honesty=20re-cited=20to=20ADR-016;=20unsourced=20mem=20sub-?= =?UTF-8?q?millisecond=20latency=20->=20structural=20statement=20+=20hones?= =?UTF-8?q?t=20no-POC=20latency=20bullet),=20env-boundary=20grep=20re-run?= =?UTF-8?q?=20clean=20(harness-only)=20and=20pinned=20in=20Configuration?= =?UTF-8?q?=20boundary,=20N-1=20growth=20acceptance=20folded=20(positionin?= =?UTF-8?q?g=20pointer=20removed,=20D-16=20attributed,=20POC=20#1/#2/pgdia?= =?UTF-8?q?g/wave-6/alkblobs-path=20pointers=20removed),=20status=20draft?= =?UTF-8?q?=20->=20stable;=20task=20->=20completed=20with=20Notes=20+=20Su?= =?UTF-8?q?mmary?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- docs/architecture/deployment.md | 69 ++++++++++++++------- tasks/release-deployment-matrix.md | 97 +++++++++++++++++++++++++++--- 2 files changed, 135 insertions(+), 31 deletions(-) diff --git a/docs/architecture/deployment.md b/docs/architecture/deployment.md index 3d70471..164b5ba 100644 --- a/docs/architecture/deployment.md +++ b/docs/architecture/deployment.md @@ -1,7 +1,10 @@ --- -status: draft -last_updated: "2026-10-11 (the N-1 growth-posture acceptance — per-engine -record of the three unconstrained-growth paths, consciously accepted)" +status: stable +last_updated: "2026-10-11 (the deployment-matrix release final pass — +every row verified true as implemented across the three engines, the +env-variable boundary pinned, the growth-posture acceptance folded +into the release text, the fixed `synchronous = NORMAL` posture +corrected, internal task/wave pointers removed)" --- # Deployment @@ -19,8 +22,8 @@ document holds the facts. | Engine | Host posture | Notes | |---|---|---| -| SQLite | **single-machine**, file-backed | NFS two-writers unsupported (honker's honesty posture, inherited, [ADR-003](decisions/003-sqlite-driver.md)). Cross-process *on one host* is verified POC ground (`data_version` is cross-process by nature). | -| Postgres | **multi-host native** | Nothing assumes a shared host; POC #2 ran all-through-network (docker bridge) with the same properties ([ADR-004](decisions/004-postgres-driver.md)). | +| SQLite | **single-machine**, file-backed | NFS two-writers unsupported — a `.db` on network storage is not a supportable deployment posture, inherited from the upstream fork ([ADR-016](decisions/016-deployment-honesty.md), [ADR-003](decisions/003-sqlite-driver.md)). Cross-process *on one host* is verified territory (`data_version` is cross-process by nature). | +| Postgres | **multi-host native** | Nothing assumes a shared host; the engine ran fully across a network bridge (docker) with the same properties, measured ([ADR-004](decisions/004-postgres-driver.md)). | | Mem | **single-process, ephemeral** ([ADR-024](decisions/024-mem-engine.md) §2) | In-memory, per-instance; no cross-process story at all — two engine instances share nothing by design. No durability, by design: loss on process exit is the documented posture, and the dependency on `alkstore-mem` *is* the ephemerality declaration (honest for workloads that accept it). Never fleet-valid — the shared-pool predicate ([ADR-010](decisions/010-queue-semantics-depth.md) §3) fails structurally; there is no pool. Compile-clean on `wasm32-unknown-unknown` ([ADR-024](decisions/024-mem-engine.md) §5) — the family's member that runs in the sandbox. | The unified trait must not pretend SQLite is multi-host — and it does @@ -73,10 +76,9 @@ Options for OQ-08, with their outcome |---|---|---| | Pool | `max_size` per process | claims/queries via deadpool | | Listener | **+1** per LISTEN-ing process | non-pooled, dedicated; pooled connections cannot carry LISTEN (deadpool#360, test-pinned — [ADR-004](decisions/004-postgres-driver.md)) | -| — | — | Sizing rule: `max_size + 1` per process; verify end-to-end accounting (POC #2's pgdiag-st-4: pool 6 + 1 listener + probe conns = exact server-side count) | +| — | — | Sizing rule: `max_size + 1` per process; verify end-to-end accounting (`pg_stat_activity` shows the exact server-side count) | -Shared-server co-tenancy (the alkblobs ADR-008 precedent — -`/workspace/@alkdev/alkblobs/docs/architecture/decisions/` — consumer +Shared-server co-tenancy (the family's queue precedent — consumer tables co-tenant the pg instance) is supported and expected — the [naming / reserved namespace contract](core-contract.md#naming--reserved-namespace) protects reserved names; queue/stream/lock/schedule tables live in one @@ -120,7 +122,7 @@ deliberately leaves to the caller's constants; the [ADR-023](decisions/023-fourth-review-round.md) §2 domain table covers the trait-surface arguments, not these. This applies to the queue stamps (`QueueOpts`: `visibility_timeout_s`, -`dead_letter_retention_s`, `max_attempts`) on both engines: +`dead_letter_retention_s`, `max_attempts`) on all three engines: - Negative or zero `visibility_timeout_s` stamps claims whose deadline is already past — every claim is instantly reclaimable (the @@ -138,22 +140,42 @@ queue stamps (`QueueOpts`: `visibility_timeout_s`, shape) — repair the opts before enqueueing rather than repairing rows afterwards. -The same trusted-as-given shape applies to `PgOpts::max_size`'s -*positive-integer* contract — 0 is typed-failed at open, the one knob -with an engine-side guard; everything above has none. The -consumer-obligation shape here is the visibility-budgeting precedent +The same trusted-as-given shape applies to the engine opts' other +numeric fields — with the exceptions the code actually carries: +`PgOpts::max_size`'s *positive-integer* contract (`0` is typed-failed +at open, no round trip) and `SqliteOpts::poll_interval`'s +non-zero contract (`0` rejected at open); `SqliteOpts::max_readers` +clamps `0` up to one reader. The queue stamps above are the +unvalidated surface. The consumer-obligation shape here is the +visibility-budgeting precedent ([ADR-010](decisions/010-queue-semantics-depth.md) §2): the constant is the consumer's, the behavior of a mis-set one is documented, nothing runtime is fabricated. +## Configuration boundary + +Env reads are harness-only; production configuration is the +opts-constructor surface. No production path in any of the three +engine crates or the core consults the process environment — grep +re-verified across the workspace at release: every `env::var` read +lives in a test target (the pg test modules' DSN plumbing; test temp +files use `std::env::temp_dir()`, not configuration). Connection +settings, credentials, and knobs arrive exclusively through `open`'s +arguments / option structs (`PgOpts`, `SqliteOpts`, the mem +constructor), handed in by the consumer's own config layer — so no +credential (or any setting) can arrive implicitly. If a future +consumer-facing surface ever wants env conveniences, the family +pattern is caller-side: the app's config layer reads env and passes +the values into the opts. + ## Durability knobs | Engine | Knob | Shape | |---|---|---| -| SQLite | `synchronous` | WAL + `NORMAL` shipped ([ADR-003](decisions/003-sqlite-driver.md)); FULL is available consumer-side for stricter durability; commit fsyncs land at WAL checkpoints (the ~1000-commit spike cadence, POC #1) | +| SQLite | `synchronous` | not a knob — the engine hardwires `synchronous = NORMAL` at connection bootstrap (WAL shipped, [ADR-003](decisions/003-sqlite-driver.md)); commit fsyncs land at WAL checkpoints (POC-measured: checkpoint spikes roughly every ~1000 commits, hundreds of ms each); there is no consumer surface to raise the sync level (a `FULL` option would be a post-v1 engine-surface addition) | | SQLite | `poll_interval` | the watcher's `data_version` poll cadence — 1 ms shipping default ([ADR-023](decisions/023-fourth-review-round.md) §4, the measured-wake-latency posture), carried on `SqliteOpts`; the idle cost is ~1000 poll reads/sec/instance; raising the interval trades wake latency (interval-bound) for idle CPU — the tuning recipe for latency-tolerant deployments | | Postgres | `synchronous_commit` | per-session knob; `on` is ship config (p50 2.40 ms seam); `off` trades max-tail (40.9 ms) for slightly better p50 — measured, honest trade ([ADR-004](decisions/004-postgres-driver.md)); session-level SET mechanics POC-verified | -| Mem | — (none) | There is no durability knob and no durability: the honest row is the absence itself ([ADR-024](decisions/024-mem-engine.md) §2). Wake latency is sub-millisecond in-process; no tuning surface exists | +| Mem | — (none) | There is no durability knob and no durability: the honest row is the absence itself ([ADR-024](decisions/024-mem-engine.md) §2). Wake delivery is an in-process channel send at the commit boundary — no network or driver round-trip exists to budget; no tuning surface exists | These are engine-configuration concerns, *not* trait surface. What part of engine config is contract-level shape vs engine-crate docs is @@ -174,7 +196,7 @@ non-goal — adding it would be a scope change. | Growth path | SQLite | Postgres | Mem | |---|---|---|---| | Dead letters (`__alkstore_dead` / pg dead table / mem's dead table) | grows forever unless the consumer opts in and sweeps — accepted | same shape — accepted | grows in memory; erased at process exit; retention still sweep-on-call — accepted | -| Notify transport (`__alkstore_notifications`) | the engine's only unconstrained-growth-with-hygiene table — pruned to its newest 10,000 rows *at attach only* (D-16); an idle-but-listening process never prunes — accepted with the cadence caveat | **no table, no path** — `pg_notify` is transient transport; nothing persists (≤ 8000 bytes per notify, [ADR-004](decisions/004-postgres-driver.md)) | **no storage** — wakes deliver at the commit boundary to open receivers; per-subscriber feeds are unbounded in-memory, ending at engine drop | +| Notify transport (`__alkstore_notifications`) | the engine's only unconstrained-growth-with-hygiene table — pruned to its newest 10,000 rows *at attach only* (the at-attach cap, [ADR-010](decisions/010-queue-semantics-depth.md) §6); an idle-but-listening process never prunes — accepted with the cadence caveat | **no table, no path** — `pg_notify` is transient transport; nothing persists (≤ 8000 bytes per notify, [ADR-004](decisions/004-postgres-driver.md)) | **no storage** — wakes deliver at the commit boundary to open receivers; per-subscriber feeds are unbounded in-memory, ending at engine drop | | Stream log | durable-log posture by contract ([ADR-015](decisions/015-streams-depth.md)); `trim_to` is the consumer-invoked cap — accepted | same durable-log shape — accepted | in-memory event log, process-lifetime-bounded; `trim_to` available | The per-path rationale: @@ -189,8 +211,10 @@ The per-path rationale: wants dead-letter hygiene budgets the sweep (a `@every` schedule is the natural recipe). Growth is therefore fully opted-in twice: retention off by default, and no automatic cadence when on. -2. **Notifications** — SQLite-only growth. The D-16 at-attach cap - (oldest-first trim to 10,000 rows, every connection open) is the +2. **Notifications** — SQLite-only growth. The at-attach prune cap + (oldest-first trim to 10,000 rows, every connection open; the + substrate provenance register's D-16 port delta, + [ADR-010](decisions/010-queue-semantics-depth.md) §6) is the *only* prune site: there is no background sweeper and no consumer- visible prune operation (the table is engine-internal wake transport, kept off the contract by ADR-008 §4's disposition). @@ -216,9 +240,6 @@ log, and unconsumed wake feeds alike — is process-lifetime-bounded by design, ending at engine drop or process exit. Accepted structurally rather than via caps. -This record is the input the deployment-matrix final pass folds into -the document's release durability/growth text. - ## Toolchain / platform notes | Note | Engine | Affects | @@ -227,7 +248,7 @@ the document's release durability/growth text. | `bundled-sqlite` adds a C build (~10 s dev, cacheable) | SQLite | build/CI time | | `libsqlite3-sys` collision with sqlx today | any mixed-driver binary | structurally avoided ([ADR-001](decisions/001-crate-split.md) single-driver rule) | | Listener `application_name` set for diagnosability (kill-targetable) | Postgres | ops runbooks ([ADR-004](decisions/004-postgres-driver.md)) | -| `alkstore-mem` compiles clean on `wasm32-unknown-unknown` (wave-6 acceptance, [ADR-024](decisions/024-mem-engine.md) §5); tokio features limited to the wasm-supported subset (sync + time); current-thread-runtime-clean internals (the suite's mem column runs under that flavor) | Mem | wasm targets, sandboxes, downstream ffi/napi/python adapters — the only engine that can compile to wasm (rusqlite/tokio-postgres are structurally out) | +| `alkstore-mem` compiles clean on `wasm32-unknown-unknown` ([ADR-024](decisions/024-mem-engine.md) §5); tokio features limited to the wasm-supported subset (sync + time); current-thread-runtime-clean internals (the contract suite's mem column runs under that flavor) | Mem | wasm targets, sandboxes, downstream ffi/napi/python adapters — the only engine that can compile to wasm (rusqlite/tokio-postgres are structurally out) | ## Consumer-facing latency profile (indicative, POC-measured) @@ -239,6 +260,10 @@ From both POCs (single-box, relative shapes are the deliverable — - Wake: ~1.1–2.2 ms p50 both engines at default cadence. - Queue claim: LISTEN-driven 3–6 ms p50 (pg); poll-only interval-bound (32–50 ms at a 50 ms poll). +- Mem: no POC measured it (the engine postdates the POCs); its wake + and tx costs are in-process channel/lock operations, not the + seam/network classes above — see [engine-mem.md](engine-mem.md)'s + architecture for what it realizes instead. - Absolute numbers will differ per hardware/network; they set *expectations of order*, not SLAs — contract docs must not bake them in ([ADR-007](decisions/007-transactional-seam.md)'s pg-POC diff --git a/tasks/release-deployment-matrix.md b/tasks/release-deployment-matrix.md index afe6670..ea15665 100644 --- a/tasks/release-deployment-matrix.md +++ b/tasks/release-deployment-matrix.md @@ -1,7 +1,7 @@ --- id: release-deployment-matrix name: Deployment matrix final pass — all three engines, release text -status: pending +status: completed depends_on: [release-n1-growth-posture] scope: moderate risk: low @@ -47,15 +47,15 @@ own docs), so every row must be true as implemented, not as designed. ## Acceptance Criteria -- [ ] Every deployment.md row verified true as implemented across the +- [x] Every deployment.md row verified true as implemented across the three engines; corrections landed -- [ ] The env-boundary grep re-run (result recorded) + the boundary +- [x] The env-boundary grep re-run (result recorded) + the boundary sentence pinned in deployment.md -- [ ] N-1's acceptance folded into the growth/durability text -- [ ] The mem rows re-read as release text -- [ ] No internal-task/wave pointers remain in the release-facing text +- [x] N-1's acceptance folded into the growth/durability text +- [x] The mem rows re-read as release text +- [x] No internal-task/wave pointers remain in the release-facing text (POC-measured caveats excepted) -- [ ] Workspace gates green (docs-only expected) +- [x] Workspace gates green (docs-only expected) ## References @@ -69,8 +69,87 @@ own docs), so every row must be true as implemented, not as designed. ## Notes -> To be filled by implementation agent +- **Row-by-row verification was against code, not just the ADRs.** + Code-verified true: WAL + `synchronous = NORMAL` at connection + bootstrap (`substrate/schema.rs` DEFAULT_PRAGMAS); + `SqliteOpts::poll_interval` (1 ms default, zero rejected → + `Error::Database`), `max_readers` (0 clamps to 1); + `synchronous_commit` wired as `-c synchronous_commit=…` on every + pooled connection, default on; `NoTls` on all three pg connection + paths (pool, listener, forwarder reconnect); listener + `application_name` `{prefix}-{pid}-{seq}`; `PgOpts::max_size` 0 + rejected at open; the queue-stamp mis-set behaviors on all three + engines (pre-claim `attempts >= max_attempts` arm: sqlite + `queue_ops.rs:404`, pg `queue.rs:357`, mem `queue.rs:128`; + "max attempts exceeded" reason strings); D-16's + `NOTIFY_ATTACH_MAX_ROWS` = 10,000 oldest-first at-attach trim. +- **Corrections landed (the pass's real findings):** + 1. *"FULL is available consumer-side for stricter durability" was + false as implemented* — the engine hardwires the pragma at + bootstrap on every connection with no consumer surface; the row + now states the fixed posture and names a `FULL` option as a + post-v1 engine-surface addition. + 2. *"both engines" → all three* — the consumer-obligation queue-stamp + paragraph predated mem. + 3. *"`max_size`: the one knob with an engine-side guard" was + understated* — the trusted-as-given paragraph now lists the three + guards the code actually carries (`max_size` 0 rejected, + `poll_interval` 0 rejected, `max_readers` clamped) and re-pins + the queue stamps as the unvalidated surface. + 4. *The NFS honesty cited ADR-003 but ADR-016 carries it* — + citation fixed, phrasing de-internaled ("a `.db` on network + storage is not a supportable deployment posture"). + 5. *The mem durability row's "wake latency is sub-millisecond" had + no source anywhere in the repo* (no POC measured mem; no spec + states the number) — replaced with the structural, verifiable + statement (in-process channel send at the commit boundary, no + round-trip class to budget), and the latency profile gained a mem + bullet stating honestly that no POC measured it (which also + removed the "both POCs" three-engine dangling coverage). +- **Env-boundary re-check (review 003's method re-run):** grep across + the workspace — every `env::var` read lives in a test target + (10 pg test modules' DSN plumbing; `std::env::temp_dir()` for test + temp files only); production paths (core, all three engines, the + constructors) are env-free. Result unchanged from prior review. The + boundary is pinned in a new `## Configuration boundary` section + ("env reads are harness-only; production configuration is the + opts-constructor surface") plus the caller-side family pattern for + a future env convenience. No engine-spec placement added — the + engine specs' opts docs already state the constructor surface, and + the one sentence belongs in the matrix. +- **N-1 fold:** the growth-posture acceptance section stands as the + release durability/growth text; the "this record is the input the + deployment-matrix final pass folds in" positioning sentence was + removed (folded this pass), the bare "(D-16)" references were + attributed ("the substrate provenance register's D-16 port delta, + ADR-010 §6"), and the mem paragraph (process-lifetime-bounded by + design) retained verbatim from the N-1 record. +- **Internal pointers removed:** "POC #1"/"POC #2" numbered refs + (checkpoint cadence → "POC-measured"; through-network posture → + "measured" with ADR-004 citation), "POC #2's pgdiag-st-4" sizing + parenthetical → generic `pg_stat_activity` accounting note, the + alkblobs absolute workspace path in the co-tenancy paragraph, and + the "wave-6 acceptance" pointer in the wasm toolchain row. +- **Frontmatter:** status `draft` → `stable` (release text; matching + the engine specs' convention). POC-measured caveats and numbers + kept (verified against the POC findings docs — e.g. the SQLite + ~0.35 ms p50 matches the shipped spawn_blocking seam's 0.354 ms). ## Summary -> To be filled on completion \ No newline at end of file +Every deployment.md row was read against the three implemented +engines; all rows verified true as implemented except the five +corrections above (one honesty-critical false claim — the +consumer-side `FULL` durability option — plus stale two-engine +wording, an understated guard list, a mis-cited ADR, and an +unsourced mem latency number). The env-variable boundary grep was +re-run clean (harness-only) and pinned in a new Configuration +boundary section; N-1's growth-posture acceptance is folded into the +release growth text with the task-positioning pointer removed; the +mem rows hold true as release text; no internal task/wave pointers +remain in the release-facing text (POC-measured caveats kept). Docs +only: `deployment.md` + this task file. Workspace gates green +(`cargo build`, `cargo test` — 15 suites ok, `cargo clippy +--all-targets -- -D warnings`, `cargo fmt --check`, +`RUSTDOCFLAGS="-D warnings" cargo doc --no-deps`, fuzz corpus replay +ok). \ No newline at end of file