diff --git a/docs/architecture/deployment.md b/docs/architecture/deployment.md index adc4d7b..3d70471 100644 --- a/docs/architecture/deployment.md +++ b/docs/architecture/deployment.md @@ -1,7 +1,7 @@ --- status: draft -last_updated: "2026-10-10 (ADR-024 — the mem engine joins the matrix: -host row, connection budget, durability posture, wasm32 toolchain note)" +last_updated: "2026-10-11 (the N-1 growth-posture acceptance — per-engine +record of the three unconstrained-growth paths, consciously accepted)" --- # Deployment @@ -161,6 +161,64 @@ decided: constructors and option structs live in the engine crates; the contract is the trait the constructor returns ([ADR-008](decisions/008-contract-v1-pinning.md) §6). +## Growth postures (consciously accepted) + +Three engine-internal table/state families grow without internal +hygiene caps. This is the decision of record, made against the +*implemented* engines (not the design docs): all three are accepted +for v1, per engine, with the rationale below. None carries a v1 +engine-side cap beyond what is described; engine-side hygiene (a +background sweeper, an integrated sweep cadence) is a deliberate +non-goal — adding it would be a scope change. + +| Growth path | SQLite | Postgres | Mem | +|---|---|---|---| +| Dead letters (`__alkstore_dead` / pg dead table / mem's dead table) | grows forever unless the consumer opts in and sweeps — accepted | same shape — accepted | grows in memory; erased at process exit; retention still sweep-on-call — accepted | +| Notify transport (`__alkstore_notifications`) | the engine's only unconstrained-growth-with-hygiene table — pruned to its newest 10,000 rows *at attach only* (D-16); an idle-but-listening process never prunes — accepted with the cadence caveat | **no table, no path** — `pg_notify` is transient transport; nothing persists (≤ 8000 bytes per notify, [ADR-004](decisions/004-postgres-driver.md)) | **no storage** — wakes deliver at the commit boundary to open receivers; per-subscriber feeds are unbounded in-memory, ending at engine drop | +| Stream log | durable-log posture by contract ([ADR-015](decisions/015-streams-depth.md)); `trim_to` is the consumer-invoked cap — accepted | same durable-log shape — accepted | in-memory event log, process-lifetime-bounded; `trim_to` available | + +The per-path rationale: + +1. **Dead letters** — the retention stamp + (`QueueOpts::dead_letter_retention_s`) is opt-in; the default + `None` is documented and suite-tested as "dead rows live forever". + Accepted on top of that: + even with a retention set, its enforcement has exactly one trigger — + a caller-driven `sweep_expired()` call (the `Queue` trait method; + the engines run no background retention sweeper). A consumer who + wants dead-letter hygiene budgets the sweep (a `@every` schedule is + the natural recipe). Growth is therefore fully opted-in twice: + retention off by default, and no automatic cadence when on. +2. **Notifications** — SQLite-only growth. The D-16 at-attach cap + (oldest-first trim to 10,000 rows, every connection open) is the + *only* prune site: there is no background sweeper and no consumer- + visible prune operation (the table is engine-internal wake + transport, kept off the contract by ADR-008 §4's disposition). + An idle-but-listening process attaches never, so its + `__alkstore_notifications` table grows past the cap until the next + attach — a reconnect, a second `open` of the same file, or a + restart. Accepted for v1 with that cadence sharpened in the + [SQLite engine spec](engine-sqlite.md#notifications-table-hygiene) + (workloads with long-lived notify-heavy processes should budget an + attach event or size the table deliberately). +3. **Stream log** — growth is the contract's durable-log posture, not + an accident: events persist until deleted, replay works across + consumers, and the bounded-growth op is `trim_to` on the stream + handle, consumer-invoked by design ([ADR-015](decisions/015-streams-depth.md) + §5: the default is unbounded, documented squarely). Accepted as + contract posture; nothing engine-side to add. + +The mem engine's entries are postures new since the paths were first +flagged (its charter, [ADR-024](decisions/024-mem-engine.md), postdates +them): everything the durable engines keep in tables mem keeps in +guarded in-process state, so its growth — dead letters, the stream +log, and unconsumed wake feeds alike — is process-lifetime-bounded by +design, ending at engine drop or process exit. Accepted structurally +rather than via caps. + +This record is the input the deployment-matrix final pass folds into +the document's release durability/growth text. + ## Toolchain / platform notes | Note | Engine | Affects | diff --git a/docs/architecture/engine-sqlite.md b/docs/architecture/engine-sqlite.md index 121e5ee..c981dc9 100644 --- a/docs/architecture/engine-sqlite.md +++ b/docs/architecture/engine-sqlite.md @@ -1,6 +1,8 @@ --- status: stable -last_updated: "2026-10-10 (review-wave-5 gate: the verification backlog fully discharged in the version-stamped contract suite, green against this engine; spec flipped draft → stable)" +last_updated: "2026-10-11 (the notify prune-cadence note — the +notifications table's at-attach-only hygiene sharpened, accepted at +the growth-posture review)" --- # SQLite engine @@ -108,6 +110,29 @@ per-contract obligations live in the core spec and are not restated here. over mpsc — measured ~2× the spawn_blocking shape at p50 in POC #1), or a raised watcher cadence for idle CPU. Neither is the default. +### Notifications-table hygiene + +The `__alkstore_notifications` table (this engine's notify wake +transport) is pruned to its newest 10,000 rows, oldest-first — +the at-attach cap (ADR-010 §6, register D-16) — and the attach is +the **only** prune site: the prune runs at every connection open (the +substrate's bootstrap); there is no background sweeper, no +time-based retention, and no consumer-visible prune operation — the +table stayed off the contract surface by ADR-008 §4's disposition +(upstream's `prune_notifications*` helpers are not carried). + +The consequence a deployer should size honestly: an idle-but-listening +process — one that opens its connections at startup and holds them — +attaches never, so nothing trims the table; every committed notify +across the fleet lands a row and the table grows past the cap until +the next attach (a reconnect, a second `open` of the same file by any +process, or a restart). Attach-prune is hygiene at open, not retention +at runtime. Long-lived, notify-heavy single-process deployments should +either budget an attach event into their runbook or size the table +deliberately. Accepted with that caveat at the growth-posture review; +the per-engine record lives in [deployment.md](deployment.md)'s growth +postures. + ## What the fork question resolved The [quality read (OQ-06)](../research/quality-read-honker-core.md) — diff --git a/tasks/release-n1-growth-posture.md b/tasks/release-n1-growth-posture.md index d5fd578..de98359 100644 --- a/tasks/release-n1-growth-posture.md +++ b/tasks/release-n1-growth-posture.md @@ -1,7 +1,7 @@ --- id: release-n1-growth-posture name: N-1 growth-posture review — conscious acceptance (or hygiene) -status: pending +status: completed depends_on: [] scope: narrow risk: low @@ -50,16 +50,16 @@ design; say so in the record), and land the acceptance: ## Acceptance Criteria -- [ ] The three growth paths re-read as implemented (SQLite + pg + +- [x] The three growth paths re-read as implemented (SQLite + pg + mem); the review's characterization confirmed or corrected in the record -- [ ] The conscious-acceptance decision recorded per engine (or a +- [x] The conscious-acceptance decision recorded per engine (or a user-approved hygiene scope change, which would be a follow-up task) -- [ ] The notify prune-cadence note landed in the SQLite engine's docs -- [ ] The acceptance substance positioned for the deployment-matrix +- [x] The notify prune-cadence note landed in the SQLite engine's docs +- [x] The acceptance substance positioned for the deployment-matrix final pass to fold in -- [ ] Workspace gates green (docs-only expected) +- [x] Workspace gates green (docs-only expected) ## References @@ -72,8 +72,56 @@ design; say so in the record), and land the acceptance: ## Notes -> To be filled by implementation agent +- **Decision: conscious acceptance, all three paths, all three + engines.** No engine-side hygiene scope change proposed or wanted. +- **The review's characterization confirmed, with one sharpening per + path** (verified against code, not just docs): + - *Dead letters*: the characterization held, plus a sharpening the + review didn't state — retention enforcement has exactly **one + trigger**: a caller-driven `sweep_expired()` (`Queue` trait + method). Engines run no background retention sweeper (the pg + suite's comment says it outright: the sweep is "the only sweeper + dead rows ever have"). So growth is opted-in twice: retention + `None` by default, and no automatic cadence when it's set. + - *Notifications*: held — `NOTIFY_ATTACH_MAX_ROWS` = 10,000, + oldest-first, runs only at connection open (the substrate + bootstrap, `schema.rs`); the D-16 cap triggers **only** on a + fresh attach. Confirmed SQLite-only: pg has **no notifications + table** (`pg_notify` is transient, ≤ 8000 bytes; pg schema has no + such table). Upstream's `prune_notifications*` helpers were not + carried (ADR-008 §4's disposition), so attach is the only prune + site. + - *Stream*: held — no engine-side trim beyond the consumer-invoked + `trim_to` (ADR-015 §5's posture: default unbounded, documented + squarely). +- **The mem engine's postures (new since the review)**: its + notify/stream/dead growth is process-lifetime-bounded by design — + notify stores nothing (commit-boundary delivery to open receivers; + per-subscriber feeds are unbounded mpsc, ending at engine drop), + the stream log and dead table are in-memory state erased at process + exit. Accepted structurally rather than via caps; recorded in the + deployment.md acceptance. +- The acceptance record landed in `deployment.md` §"Growth postures + (consciously accepted)" with an explicit "this record is the input + the deployment-matrix final pass folds in" positioning; the notify + prune-cadence note landed in `engine-sqlite.md` §"Notifications- + table hygiene" (anchored heading — deployment.md links to it). No + internal task pointers in the release-facing text; the matrix pass + can fold the substance directly. ## Summary -> To be filled on completion \ No newline at end of file +Re-read the three growth paths against the implemented engines +(`alkstore-sqlite/src/substrate/schema.rs` (D-16 at-attach prune), +`queue_ops.rs` (`sweep_dead_retention` inside the savepoint guard, +sweep-on-call only), `alkstore-postgres/src/schema.rs` (no +notifications table), `queue.rs`/`stream.rs` sweep/trim shapes, and +`alkstore-mem/src/{notify,wakes,queue,stream}.rs`). Decision of record +(**all three accepted for v1, per engine**, no hygiene scope change): +landed in `docs/architecture/deployment.md` §"Growth postures +(consciously accepted)" (per-engine table + rationale, frontmatter +updated), and the notify prune-cadence note (attach is the only prune +site; idle-but-listening processes never prune) landed in +`docs/architecture/engine-sqlite.md` §"Notifications-table hygiene" +(frontmatter updated). Docs-only change; workspace gates green +(`cargo build`, `cargo test`, clippy `-D warnings`, fmt --check). \ No newline at end of file