diff --git a/docs/plans/implementation.md b/docs/plans/implementation.md index bb0dc72..5b55298 100644 --- a/docs/plans/implementation.md +++ b/docs/plans/implementation.md @@ -1,6 +1,6 @@ --- status: draft -last_updated: "2026-10-11 (wave 3 decomposed into tasks/)" +last_updated: "2026-10-12 (wave 4 decomposed into tasks/)" --- # alkblobs — Implementation plan @@ -51,7 +51,7 @@ ops payload surfaces; release readiness is last. | [1](#wave-1--foundations) | Crate/feature scaffold; hashing & keys (the one-way door); error model; Backend trait + I/O seams + engine-state seam shape; facade value types; conformance-suite convention | — | decomposed (2026-10-11) | | [2](#wave-2--the-store-works-dependency-free-mem--local) | mem + local engines; facade put/read paths + dispatch + constructor modes; in-process GC protocol; suite's first columns | wave 1 | decomposed (2026-10-11) | | [3](#wave-3--sqlite-engine) | sqlite kv engine (default): WAL posture, prepared statements, engine-state window family, suite column | wave 2 | decomposed (2026-10-11) | -| [4](#wave-4--postgres-kv-engine) | postgres kv engine: schema, pool, CAS rows, stable-cursor list, full engine-state clause families, suite column | wave 2 | pending | +| [4](#wave-4--postgres-kv-engine) | postgres kv engine: schema, pool, CAS rows, stable-cursor list, full engine-state clause families, suite column | wave 2 | decomposed (2026-10-12) | | [5](#wave-5--pg-lo-large-engine) | pg-lo large engine: LO staged put, descriptorless lo_get windows, companion rows, window-family seam clauses, suite column | wave 4 | pending | | [6](#wave-6--fleet-gc) | Fleet GC mechanism: DB-backed pins, joint entry+pin tx, sweep lease, SQL delete arbitration, fleet-validity predicate + constructor declarations | waves 4 + 5 | pending | | [7](#wave-7--ops-surface) | `ops` feature: alkcall-backed op family, ACL mapping, pin tokens + renewal, fanout seam | wave 6 | pending | diff --git a/tasks/engine-postgres-fleet.md b/tasks/engine-postgres-fleet.md new file mode 100644 index 0000000..8efc6ef --- /dev/null +++ b/tasks/engine-postgres-fleet.md @@ -0,0 +1,198 @@ +--- +id: engine-postgres-fleet +name: postgres engine-state clause families — the fleet host's full seam +status: pending +depends_on: [engine-postgres] +scope: moderate +risk: medium +impact: component +level: implementation +tags: [wave-4, engines, postgres, gc] +--- + +## Description + +Implement the engine-state companion seam's **full clause families** on +the postgres backend (`src/postgres.rs` or its seam module): the +`EngineWindowState` impl **and** the `EngineFleetState` impl per +ADR-012 §5's postgres row — every family **yes** (postgres is the +fleet host, ADR-012 §4). This is the seam implementation pattern +pg-lo's window family reuses (wave 5) and the fleet wave's mechanics +ride (wave 6). + +- **`EngineWindowState`** (the window family — active durably + single-node, fleet-active when declared): + - **`ensure_gc_schema`** — creates **`gc_pins` + `gc_window`** (the + fleet host's both-tables row; unconditionally — the applicability + row does not condition on the fleet declaration): + `gc_pins (key bytea, owner bytea, expiry bigint)` and + `gc_window (key bytea, visible_after bigint)` per ADR-010 §2's + schema. `owner` is the store-instance id (ADR-011's vocabulary); + the pin row's identity is `(key, owner)` — protection composes by + row. The embedder's liveness table is never created here (ADR-008's + schema-ownership rule — the sweep reads it and never owns it). + Called at store construction by the store core + (`store-postgres-construction` wires the call); the clause body is + engine-side. + - **`window_stage`** — stage candidates into the durable window, + **idempotent per `(key, deadline)`** (the upsert discipline — a + same-deadline re-stage is idempotent, a new-deadline re-stage + updates the row; the sqlite engine's recorded semantics). + - **`window_take(now)`** — take overdue candidates + (`visible_after <= now`, the `<=` boundary the seam doc pins): the + rows **leave the window table and return as keys** — one atomic + step (selected rows removed and returned together; a row is never + handed out twice). Empty when nothing is overdue; virgin store: + empty. + - **Timestamp encoding** (recorded design point): the sqlite + engine's signed i64 nanosecond count from the Unix epoch, carried + as `BIGINT` — the seam's crate-wide encoding (sub-second precision + for the test tunings, pre-epoch negatives, full i64 range); the + sqlite engine's `time_to_nanos`/`nanos_to_time` helpers are + reference, not a dependency (per-engine bodies, one encoding). +- **`EngineFleetState`** (the fleet family — the postgres host's + alone; **fleet activation is wave 6**: under `fleet: false` the + store core never calls these — the clause bodies exist and are + contract-tested per applicability): + - **`JointTx`** — the pg transaction the pooled client carries + (recorded design point: the concrete pooled-transaction type and + how it keeps its connection checked out for the tx's life; `Send` + per the trait bound). **Ownership rule, pinned in the signature**: + the store core holds the tx from `begin_joint_tx` until + `pin_commit` consumes it; the engine contributes mechanics and + never owns it (ADR-010 §2's ruling). + - **`begin_joint_tx`** — open the tx on a pooled connection, + uncommitted until `pin_commit`. + - **`joint_put`** — the entry half: the engine's put mechanics (the + `ON CONFLICT DO NOTHING` CAS clause) executed **inside the held + tx**, same first-writer-wins semantics as the public `put`. + - **`pin_commit(tx, entries)`** — the joint terminal step: the + pin-row inserts **and** the commit — publish and protection land + in one step (ADR-005's pin-before-publish, fleet form). On any + failure the entire tx aborts: no pin rows, no published entry, + zero residue — the stage-hygiene invariant's tx form. Consumes the + tx. + - **`pin_renew(key, owner)`** — `true` when a **live** (un-expired) + `(key, owner)` row existed and its expiry moved to this call's + successor time; an expired or absent row is never resurrected: + `false`, nothing updated (ADR-012 §2's expiry semantics verbatim). + **Named design point — the successor-time source**: the clause + signature carries no expiry (ADR-010 §2: the TTL lives with the + store's renewal schedule, the engine owns only the SQL) while the + row update needs the new expiry — decide the wiring consistent + with the seam doc's reading (the natural shape: the engine + computes `expiry = now + TTL` from the shipped pin-TTL constant — + ADR-008's default 60 s, constructor-tunable when wave 6's fleet + constructor lands — and the store owns the TTL/3 cadence, i.e. + *when* renewal is called) and record it. + - **`pin_check(key)`** — classify against un-expired pin rows: any + un-expired `(key, *)` row exists → `PinState::Protected`; else + `Unprotected`. The engine never decides deletion — it only + classifies. + - **`arbitrate(key)`** — the full delete-time arbitration (ADR-008): + the un-expired pin rows **plus the embedder's registered liveness + tables**, under the `SELECT … FOR UPDATE` finger-lock keyed at + delete time, folded into one answer — `true` = liveness protects + the key at this instant. The engine-side **registration of named + liveness-table references** is part of this body (ADR-012 §6.6: + the fleet form of `register_liveness_source` registers a named + table reference *in the shared engine*) — a crate-internal + engine-side mechanism the store core's fleet registration verb + forwards to (the facade verb itself is wave 6). The engine holds + table *references*, never liveness semantics; the referenced + tables are never created by the engine (the schema-ownership + rule). Contract tests use a test-owned embedder table. + - **`sweep_lock_acquire()` / `sweep_lock_release()`** — the sweep + lease's shipped provider: the **pg advisory lock** with ADR-013 + §2's named exclusive semantics — atomic test-and-set + (`pg_try_advisory_lock`), one sweeper per pool, `Ok(true)` = held, + `Ok(false)` = contention (the second sweeper's sweep aborts with + `GcAbortCause::SweeperLock` as report data — the store core's + reading, wave 6). **Named design points**: session-scoped vs + transaction-scoped (session-scoped — ADR-013 §2's holdership rule + requires holdership to end at owner death/session loss, which only + the session scope provides); the lock key (a fixed key, + schema-qualified or namespaced so co-tenant engines on one + instance never collide); and **acquire/release on the same pooled + connection** (advisory locks are per-session — the lease must hold + its client for its duration; the release-on-another-connection + hazard is the recorded pitfall). +- **Contract tests** per the applicability row (ADR-012 §5's table is + the contract-test matrix): the schema creates both tables (and the + embedder's liveness table never); stage idempotence per + `(key, deadline)` incl. the re-stage-at-new-deadline form; take + returns exactly the overdue set, removes it, leaves future rows; the + `<=` boundary; the joint tx atomicity (a forced failure → no pin + rows, no entry, zero residue); `pin_renew`'s live/expired/absent + semantics; `pin_check`'s two-variant classification; `arbitrate` + folding pin rows + a registered test-owned liveness table under the + finger-lock; the advisory lock's acquire/contention/release shape + and the session-loss reaping semantics (a dropped client's lock is + gone — testable by dropping the lease's connection). +- **Tests need a live server** — the alkstore harness convention: + env-driven (`ALKBLOBS_PG_*`), skip cleanly server-less, acceptance + requires green against the harness server (`pglo-poc`, :15432, db + `blobs`, `postgres`/`poc`). + +The store core drives these clauses in wave 6 (fleet mode); the +non-fleet store construction consumes `ensure_gc_schema` only +(`store-postgres-construction`). This task ships the clauses and their +engine-level contract tests. + +## Acceptance Criteria + +- [ ] The postgres backend implements `EngineWindowState` and + `EngineFleetState` completely — every clause family of ADR-012 + §5's postgres row +- [ ] `ensure_gc_schema` creates `gc_pins` + `gc_window`; the + embedder's liveness table is never created by the engine + (contract-tested) +- [ ] `window_stage` idempotent per `(key, deadline)`; `window_take` + returns exactly the overdue set (`<= now` boundary), removes it + atomically, leaves future rows; empty on none/virgin +- [ ] The joint tx atomicity holds: `pin_commit` publishes entry + + pin rows in one step; any failure aborts with zero residue + (contract-tested) +- [ ] `pin_renew` never resurrects an expired/absent row; + `pin_check` classifies per the two-variant `PinState`; + `arbitrate` folds pin rows + registered liveness-table + references under the `FOR UPDATE` finger-lock (contract-tested + with a test-owned embedder table) +- [ ] The advisory lock: atomic test-and-set, contention → + `Ok(false)`, release no-op-safe, session-loss reaping; the + same-connection acquire/release discipline recorded +- [ ] Named design points recorded: the `JointTx` concrete type, the + `pin_renew` successor-time source, the timestamp encoding, the + lock key + scope, the lease's connection holding +- [ ] Contract tests env-driven, skipping cleanly server-less; green + against the harness server at acceptance +- [ ] `cargo test --features postgres` green server-less (skipping); + `cargo clippy --all-targets -- -D warnings` and + `cargo fmt --check` green (default + `--features postgres`) + +## References + +- src/gc.rs (`EngineWindowState` + `EngineFleetState` — the seam + contracts and their clause docs) +- docs/architecture/decisions/010-backend-io-seams-and-fleet-state-home.md + §2 (the clause bodies; the tx-ownership ruling; the advisory lock) +- docs/architecture/decisions/012-pre-decomposition-consistency-rulings.md + §3 (fleet activation is a declaration), §4 (the GC-state host), §5 + (the applicability table — postgres all-yes), §6.6 (the liveness + registration's API home), §6.7 (the executor) +- docs/architecture/decisions/008-trait-size-probe-fleet-gc-and-large-engines.md + §2 (the fleet mechanism: pin rows, expiry, TTL 60 s, the advisory + lock, the staged window) +- docs/architecture/decisions/013-family-composition-reactive-boundary-and-feature-graph.md + §2 (the sweep lease's named exclusive semantics) +- src/sqlite.rs (the window family's reference implementation — same + semantics, different dialect) +- tasks/engine-postgres.md (the pool/schema/driver posture this rides) + +## Notes + +> To be filled by implementation agent + +## Summary + +> To be filled on completion \ No newline at end of file diff --git a/tasks/engine-postgres.md b/tasks/engine-postgres.md new file mode 100644 index 0000000..58916f8 --- /dev/null +++ b/tasks/engine-postgres.md @@ -0,0 +1,181 @@ +--- +id: engine-postgres +name: postgres kv engine — the fleet-capable engine +status: pending +depends_on: [] +scope: narrow +risk: medium +impact: component +level: implementation +tags: [wave-4, engines, postgres] +--- + +## Description + +Implement the `postgres` kv-tier engine (feature `postgres`, +default-off; ADR-007; REQ-2/REQ-3) against the wave-1 trait +(`src/backend.rs`) — the second SQL-backed engine and the fleet host +(ADR-012 §4). The sqlite engine (`src/sqlite.rs`) is the structural +twin: same contract, different driver cluster and posture deltas. + +- **Backend impl** (`src/postgres.rs`, behind + `#[cfg(feature = "postgres")]`): the seven methods plus + `put_staged`, opaque byte keys/values, `name() -> "postgres"`. + - **Schema bootstrap** — one engine-owned PostgreSQL schema + (recorded design point: the default schema name, the + quoted-identifier boundary for every identifier entering SQL text + — the alkstore `quote_identifier` pattern, `alkstore-postgres` + src/schema.rs) holding the POC #5 finding B5 row shape: + `blobs (key bytea PRIMARY KEY, value bytea NOT NULL, size bigint + NOT NULL) WITH (fillfactor = 90)`. Idempotent DDL + (`CREATE … IF NOT EXISTS`) at the constructor; no migration + machinery. The dedicated `size` column is the **metadata-only + `size` probe**: `SELECT size … WHERE key = $1` rides the PK index + and never loads the value blob (ADR-008's size discipline). + - **Pool** — deadpool-postgres over tokio-postgres + (`deadpool_postgres::Pool`); pool size configurable with a named + shipped default (recorded design point; the alkstore + `DEFAULT_MAX_SIZE = 8` precedent). **Connection pooling is + load-bearing, not an optimization** (POC #5 B2: fresh sessions + cost ~20 ms) — the module doc records the posture. + - **Per-connection prepared-statement discipline** — prepared + statements die with sessions (POC #5 B5 hit the "prepared + statement s_1 does not exist" error exactly when a pool-recycled + connection lost its statement); every engine statement rides + `prepare_cached` on the pooled client so the cache is + per-connection by construction. Recorded design point. + - **`synchronous_commit` parity knob** — configurable per engine + instance, wired on **every pooled connection** (the alkstore + POC-verified per-session connect-options mechanics, + `alkstore-postgres` src/opts.rs). Default `on` (the ship config); + `off` is the documented sqlite-NORMAL-alike latency trade (B5: + latency moves ~2-3× across settings). This is the parity knob + with sqlite's `synchronous=NORMAL`-WAL shipped tier — recorded in + the module doc as posture, not hidden. + - **CAS rows** — `INSERT … ON CONFLICT DO NOTHING` + first-writer-wins on both put forms; a re-put of an existing key + is a no-op, never an overwrite (ADR-007 §3: the sqlite + `INSERT OR IGNORE` mapping). + - `has` — an **EXISTS probe** (`SELECT EXISTS(…)`) — never a value + load. `get` — the **materializing cursor body** (kv family): the + whole value loaded once (bounded by tier policy), `len()` from the + row's size column, reads slice the materialized value composing + `backend.rs`'s `clamped`. `put_staged` — the **degenerate + in-memory stage** (ADR-010 §1, the sqlite engine's shape): + `append` extends the buffer, `commit` publishes via the same + `ON CONFLICT DO NOTHING` clause and self-discards on failure, + `discard` drops the buffer with zero residue. + - `delete` — per-key rows-affected discipline + (`Err(Error::Missing)` when absent). **Plus the crate-internal + batch-delete helper** — `DELETE … WHERE key = ANY($1)` over a + `bytea[]` parameter (ADR-007 §5's sweep-safety mechanics): the + sweep's batched delete form, contract-tested in-module; its + store-core consumer is wave 6's fleet sweep (scope-noted + `#![allow(dead_code)]` until then — the wave-1 seam-shape + discipline). + - `list` — **complete and well-formed as a stable cursor over a + vacuuming table** (ADR-007 §5): one statement's MVCC snapshot is + stable for its duration (ghost rows from concurrent vacuuming + never appear); the natural realization is the single + `SELECT key … ORDER BY key` statement with tokio-postgres's + row-streaming portal batches (bounded memory on huge pools) — the + exact fetch shape is a recorded design point. + - **Virgin-store posture** — the constructor creates the schema and + tables; the tolerance is the sqlite engine's recorded read-only + split (reads on a schema-less database map to absent/empty, writes + surface the engine error — never silently creating schema). The + test shape (a fixture connecting without running the bootstrap) + mirrors `src/sqlite.rs`'s `new_tableless` pattern. Recorded design + point. + - **Autovacuum/fillfactor posture** — stated in the module doc, not + silence (POC #5 B5: autovacuum is a real maintenance surface for a + churning CAS pool; fillfactor 90 measured; the ~40× small-tier + storage overhead accepted per B6 — documented, not hidden). The + B5 tuning numbers are POC-derived posture defaults (ADR-007 §6's + deferred-cost pattern): the first real pg deployment verifies them + against its media. +- **Natively async** — tokio-postgres is async: **no `spawn_blocking` + seam** (the sqlite engine's blocking discipline does not carry; the + pool is the concurrency posture). No blocking I/O on the async path; + no `unwrap`/`expect` outside tests; thiserror errors surface as + `Error::Io` via the engine's mapping discipline (`Missing` via the + rows-affected/virgin-store mapping). +- **Unit tests** in-module: the method contract (virgin-store no-ops + incl. the schema-less form, CAS re-put, Missing discipline, size + exactness + metadata-only), the staged-put happy path and both + hygiene forms (discard → zero residue; commit failure → + self-discard — a `#[cfg(test)]` fail-commit knob standing in for + media failure, the mem/sqlite pattern), list stability and + completeness, the batch-delete helper, the `synchronous_commit` + wiring. **Tests need a live server** — the alkstore harness + convention: connection settings ride the environment + (`ALKBLOBS_PG_HOST/PORT/USER/PASSWORD/DB`), tests skip cleanly + server-less, acceptance requires green against the server. The + harness server: dockerized `pglo-poc` (`postgres:16-alpine`) on + :15432, db `blobs`, user/password `postgres`/`poc` (verified live + 2026-10-12). + +The engine-state clause families (`EngineWindowState` + +`EngineFleetState`) are `engine-postgres-fleet` (separate task); the +constructor wiring is `store-postgres-construction`. This task ships +the engine and its in-module unit tests only. + +## Acceptance Criteria + +- [ ] `src/postgres.rs` implements `Backend` completely (seven methods + + `put_staged`) behind feature `postgres` (default-off), with + the materializing cursor body and the degenerate staging body +- [ ] One engine-owned schema (quoted-identifier boundary), the B5 row + shape with fillfactor 90, idempotent bootstrap at the + constructor; autovacuum/storage-overhead posture recorded in the + module doc +- [ ] deadpool pool with a named shipped default size; + per-connection `prepare_cached` discipline; the + `synchronous_commit` parity knob wired on every pooled + connection, configurable, default `on` +- [ ] CAS first-writer-wins on both put forms; re-put never overwrites +- [ ] `size` reads the dedicated size column via the PK index, never + the value blob (metadata-only, unit-tested) +- [ ] `list` complete and well-formed as a stable MVCC-snapshot cursor + (design point recorded); the batch-delete helper + (`key = ANY($1)`) contract-tested, scope-noted until wave 6 +- [ ] Stage hygiene holds in both forms (discard → zero residue; + failed commit → self-discard), unit-tested +- [ ] Virgin-store reads are no-ops including the schema-less form + (unit-tested); writes surface the error (the read-only split) +- [ ] Unit tests env-driven (`ALKBLOBS_PG_*`), skipping cleanly + server-less; green against the harness server at acceptance +- [ ] `cargo test` (default — postgres is default-off, the engine + compiles out) green; `cargo test --features postgres` green + server-less (skipping); `cargo clippy --all-targets -- -D + warnings` and `cargo fmt --check` green (default + + `--features postgres`) + +## References + +- docs/architecture/backends-and-dispatch.md (§kv tier — postgres + engine) +- docs/architecture/decisions/007-two-kv-engines-sqlite-and-postgres.md + (§4's B5 deltas as requirements; §5's sweep-safety proof; §6's + deferred-cost pattern) +- docs/architecture/decisions/003-backend-contract-and-dispatch.md + (the trait contract; the three numbered put rules' kv side) +- docs/architecture/decisions/010-backend-io-seams-and-fleet-state-home.md + §1 (the whole-value put's pg body; the materializing cursor) +- docs/research/poc-postgres-kv-findings.md (findings B1-B6 — the row + shape, the pooling discipline, the durability knob, the storage + overhead) +- /workspace/@alkdev/alkstore/alkstore-postgres (the sibling harness + patterns: src/opts.rs's PgOpts + per-session synchronous_commit + wiring, src/schema.rs's bootstrap + quote_identifier, the suite + factory's fresh-schema-per-open) +- src/backend.rs (the trait + seam contracts this implements) +- src/sqlite.rs (the structural twin's shapes this mirrors) + +## Notes + +> To be filled by implementation agent + +## Summary + +> To be filled on completion \ No newline at end of file diff --git a/tasks/review-wave-4.md b/tasks/review-wave-4.md new file mode 100644 index 0000000..bf3f8ff --- /dev/null +++ b/tasks/review-wave-4.md @@ -0,0 +1,114 @@ +--- +id: review-wave-4 +name: Wave 4 review gate — the postgres engine vs the contract +status: pending +depends_on: [suite-column-postgres] +scope: narrow +risk: low +impact: phase +level: review +tags: [wave-4, review] +--- + +## Description + +The wave-4 review gate — the engine-wave review shape +(docs/plans/implementation.md §Review gates: "engine-vs-contract +conformance"), before wave 5 decomposes. The gate reads the +implementation against the pinned ADR text line by line (the +wave-1/2/3 gates' method). + +Gate checks: + +- **Engine-vs-contract conformance**: the suite column green against + dockerized postgres (server-less runs skip cleanly — a skip never + substitutes for acceptance); the engine-state clause families + contract-tested per the applicability row (ADR-012 §5's postgres + row — every family yes); virgin-store semantics; the `size` + metadata-only discipline (the size-column probe never loads the + value blob). +- **The postgres posture vs ADR-007**: the B5 deltas carried as + requirements, not hedges — the `synchronous_commit` parity knob + (configurable, per-connection wiring, default `on`); pooled + connections with per-connection prepared-statement discipline + (statements die with sessions); autovacuum + fillfactor tuning + stated in the module doc; the ~40× small-tier storage overhead + accepted and documented; the B5 tuning numbers recorded as + POC-derived posture defaults (ADR-007 §6's deferred-cost pattern). +- **The sweep-safety proof vs ADR-007 §5**: `list()` complete and + well-formed as a stable MVCC-snapshot cursor over a vacuuming + table; the batch-delete helper (`key = ANY($1)`) contract-tested; + the exact-count sweep-outcome rows green against the server. +- **The seam families vs ADR-010 §2 / ADR-012 §4/§5**: the full + clause families on postgres (the fleet host) — `ensure_gc_schema` + (both tables, the embedder's liveness table never created), + `window_stage`/`window_take` (the sqlite semantics' dialect twin), + the joint tx ownership rule honored in the `JointTx` shape (the + store holds; the engine contributes), `pin_commit`'s atomicity + (publish + protection in one step, zero residue on failure), + `pin_renew`'s never-resurrect semantics, `pin_check`'s + classification-only discipline, `arbitrate`'s finger-lock fold, + the advisory lock's ADR-013 §2 lease semantics (atomic + test-and-set, session-scoped holdership, same-connection + release). **Fleet activation absent by construction** — no + `fleet:` parameter, no store-core fleet wiring (wave 6); the + clause bodies exist and are contract-tested per applicability. +- **The non-fleet posture vs ADR-012 §3**: the constructors are + implicitly `fleet: false` — the in-process protocol verbatim; the + recorded posture names wave 6's slotting-in as a non-event. +- **Async/blocking discipline**: tokio-postgres natively async — no + `spawn_blocking` seam (the sqlite discipline does not carry); no + blocking I/O on the async path; the pool is the concurrency + posture; no `unwrap`/`expect` outside tests. +- **Named design points audited as resolved** (each recorded in its + task's Notes): the schema name + quoted-identifier boundary; the + pool default size; the prepared-statement discipline; the DSN + + opts constructor shape; the virgin-store tolerance split; the + list-cursor fetch shape; the timestamp encoding; the `JointTx` + concrete type; the `pin_renew` successor-time source; the lock + key + scope; the lease's connection holding. +- **Family conventions**: no comments in code (doc comments on public + API fine), thiserror, tokio-only async, no unwrap/expect outside + tests, naming, one-module-per-file re-export shape, the + `#![allow(dead_code)]` scope notes matching their remaining scope + (the batch-delete helper's wave-6 note intact). +- **Gates green**: `cargo test` (default), `cargo test --features + mem`, `cargo test --features postgres` (server-less: skipping), + `cargo test --all-features`, `cargo check --no-default-features + --features postgres` (the pg-pure build), `cargo clippy + --all-targets -- -D warnings` (default + `--features postgres` + + `--all-features`), `cargo fmt --check`. + +Findings are fixed inline where cheap, or recorded as wave-5 +amendments where they shape the next decomposition. The gate's +completion is what unblocks wave 5's decomposition. + +## Acceptance Criteria + +- [ ] Every gate check above read and verified against the cited ADR + text (not from memory of it) +- [ ] Named design points each decided, recorded, and consistent with + the specs +- [ ] Findings fixed inline or recorded as wave-5 amendments with + owners +- [ ] All gates re-run green after any fixes +- [ ] Wave 5 decomposition unblocked (recorded in this task's Summary) + +## References + +- docs/plans/implementation.md (§Wave 4, §Review gates — engine wave + reviews) +- docs/architecture/decisions/007-two-kv-engines-sqlite-and-postgres.md +- docs/architecture/decisions/010-backend-io-seams-and-fleet-state-home.md §2 +- docs/architecture/decisions/012-pre-decomposition-consistency-rulings.md §3/§4/§5/§6.6/§6.7 +- docs/architecture/decisions/013-family-composition-reactive-boundary-and-feature-graph.md §2/§4 +- docs/architecture/store-api.md +- tasks/review-wave-3.md (the gate method; the structural twin's gate) + +## Notes + +> To be filled by the review agent + +## Summary + +> To be filled on completion \ No newline at end of file diff --git a/tasks/store-postgres-construction.md b/tasks/store-postgres-construction.md new file mode 100644 index 0000000..468b106 --- /dev/null +++ b/tasks/store-postgres-construction.md @@ -0,0 +1,129 @@ +--- +id: store-postgres-construction +name: postgres constructor wiring — the fleet-capable store shapes +status: pending +depends_on: [engine-postgres, engine-postgres-fleet] +scope: narrow +risk: low +impact: component +level: implementation +tags: [wave-4, store, postgres, construction] +--- + +## Description + +Slot postgres into the facade's engine representation — the slotting-in +pattern `src/store.rs`'s module doc pinned ("adding an enum variant and +a named constructor is slotting in"): + +- **`KvEngine::Postgres`** variant behind `#[cfg(feature = + "postgres")]` + the dispatch arms (`put`/`get`/`has`/`size`/ + `delete`/`list`). The kv cursor dispatch slots the postgres + materializing body into **`KvCursor`** (the recorded sub-enum design + point: "postgres (wave 4) slots one arm in here" — the store.rs doc + note is updated to the landed state). +- **Named constructors**, feature-gated on `postgres` (default-off — + they construct under `--features postgres`): + - **dual-tier** `kv = postgres + large = local` — the + fleet-capable-kv production shape (non-fleet here; see below); + fallible (config parse, connect, schema bootstrap, the local root + creation). Size-threshold dispatch at the constructor-tunable + threshold as in the sqlite shape. + - **kv-only(postgres)** — no large tier; `threshold` is the put-time + rejection point (ADR-012 §6.2; the rejection machinery is the put + seam's, already landed — only the constructor slots in). + - Both take the **connection settings + engine opts** (recorded + design point: the DSN form — the tokio-postgres `Config`-parseable + string, the alkstore `open(config, PgOpts)` pin adapted to the + named-constructor shape — and the opts shape carrying schema name, + pool size, `synchronous_commit`; engine-configuration concerns + ride the constructor parameters, never invented defaults) and the + **delete-window delay** parameter (`Duration`, default + `DEFAULT_DELETE_WINDOW_DELAY`, `0` the legal degenerate tuning — + the sqlite constructors' signature shape). +- **Non-fleet posture** — these constructors are implicitly + `fleet: false` (ADR-012 §3): the ADR-005 in-process protocol applies + verbatim regardless of engine; pins are in-process guards, the sweep + lease is the in-process lock, and none of the `EngineFleetState` + clauses run. **The `fleet: true` declaration is wave 6** (the + fleet-validity predicate + constructor declarations) — no fleet + parameter is added here; wave 6 slots it in without reshaping. + Recorded in the constructor docs so the wave-6 slotting is a + non-event. +- **`ensure_gc_schema` at construction** — the store-core call + (engine-neutral; the postgres arm creates `gc_pins` + `gc_window` + per the fleet host's applicability row). A construction failure is + `Err(Error::Io)` per the fallible-constructor pattern. +- **The one-window-host rule's store-core posture** — already recorded + (the durable window hosts on the kv engine when it is SQL-backed); + the postgres arm slots in under it with no reshaping. Verify the + recorded posture's wording covers postgres (it names "sqlite or + postgres" already — ADR-012 §5) and that the form branch keys off + the kv-engine enum, never a flag. +- **Stale scope notes updated** — the engine-representation doc's + "postgres (wave 4)" forward references, the `KvCursor` design-point + note, and any `#![allow(dead_code)]` scope notes whose scope changes + (the batch-delete helper's note stays — its consumer is still wave + 6). +- **Constructor unit tests**: the modes construct and introspect + correctly (`mode()`/`threshold()`), `ensure_gc_schema` ran (both GC + tables exist in the fresh schema), the window-delay default is the + pinned constant, kv-only rejection wiring unchanged, the dual-tier + shape round-trips end-to-end (a smoke put/get through both tiers). + Env-driven, skipping cleanly server-less (the engine tests' + convention). +- **The pg-pure build** — `cargo check --no-default-features + --features postgres` (and the tests target under it): ADR-013 §4's + supported non-default configuration first becomes constructible + here; verify it compiles clean. + +## Acceptance Criteria + +- [ ] `KvEngine::Postgres` slots in; all six kv ops dispatch; the kv + cursor body slots into `KvCursor` (the design-point note updated + to the landed state) +- [ ] `dual_tier_postgres_local` + `kv_only_postgres` constructors + exist behind feature `postgres`, take connection settings + + engine opts + threshold + window delay, fallible where I/O is + fallible +- [ ] `ensure_gc_schema` runs at construction (both GC tables in the + fresh schema); the window-delay shipped default is 5 min + (pinned constant) and constructor-tunable; `0` legal +- [ ] The non-fleet posture recorded (in-process protocol verbatim; + the `fleet: true` declaration is wave 6, slotting in without + reshaping) +- [ ] The one-window-host posture verified to cover postgres; the form + branch keys off the kv-engine enum +- [ ] The pg-pure build (`--no-default-features --features postgres`) + compiles clean +- [ ] `cargo test` (default) green; `cargo test --features postgres` + green server-less (skipping); `cargo clippy --all-targets -- -D + warnings` and `cargo fmt --check` green (default + + `--features postgres`) + +## References + +- src/store.rs (the engine-representation design point; the + constructor modes; the slotting-in pattern; the `KvCursor` + sub-enum design point) +- docs/architecture/decisions/011-vocabulary-tiers-engines-and-fleet-state.md + §4 (modes; the constructor table) +- docs/architecture/decisions/012-pre-decomposition-consistency-rulings.md + §3 (fleet activation is a declaration — wave 6), §5 (the + one-window-host rule), §6.3 (dual-tier shapes) +- docs/architecture/decisions/008-trait-size-probe-fleet-gc-and-large-engines.md + §2 (the window delay, default 5 min) +- docs/architecture/decisions/013-family-composition-reactive-boundary-and-feature-graph.md + §4 (the feature graph; the pg-pure build) +- tasks/engine-postgres.md, tasks/engine-postgres-fleet.md (the engine + + seam bodies this wires) +- tasks/store-sqlite-construction.md (the structural twin's recorded + design points) + +## Notes + +> To be filled by implementation agent + +## Summary + +> To be filled on completion \ No newline at end of file diff --git a/tasks/suite-column-postgres.md b/tasks/suite-column-postgres.md new file mode 100644 index 0000000..b1f0bf5 --- /dev/null +++ b/tasks/suite-column-postgres.md @@ -0,0 +1,128 @@ +--- +id: suite-column-postgres +name: Conformance suite — the postgres column (the SKIP-posture column) +status: pending +depends_on: [store-postgres-construction] +scope: moderate +risk: medium +impact: phase +level: implementation +tags: [wave-4, suite, postgres] +--- + +## Description + +Grow the **postgres column** — the suite's first SKIP-posture column +(README §SKIP posture: SQL-attached factories ride a server; when no +server is reachable the column **skips cleanly** — a skip is not a +failure — and the wave's acceptance requires green **against the +server**). The column runs under `--features postgres` (feature +`postgres`, default-off). + +- **Factories** (`tests/conformance/postgres_column.rs`): + - dual-tier `kv = postgres + large = local` at a column-declared + threshold and the **test tuning of the window delay (`0` — + execute-on-next-sweep)**; + - `kv-only(postgres)` at a column-declared threshold (the rejection + row's host; threshold materializable per the README's + configuration contract). + **The alkstore pg-factory pattern, reused** (`alkstore-postgres` + tests/contract_suite.rs): a **fresh engine-owned schema per + `open()`** — unique per call (seeded per tag/pid/time so two factory + instances never share a schema name, racing opens included), tracked + per-instance in a schema registry, teardown + `DROP SCHEMA IF EXISTS … CASCADE` over exactly the schemas the + instance minted (idempotent, owned — never the shared server's other + schemas; safe against an abandoned store: `CASCADE` removes + dependents server-side while the store's pooled connections fail + closed on next use). Panicking-row-safe — the sqlite column's + instance-tracks-its-own-site pattern over schemas instead of temp + dirs. +- **Connection settings ride the environment** — the alkstore harness + convention: `ALKBLOBS_PG_HOST/PORT/USER/PASSWORD/DB` (db defaulting + `postgres` if unset), a reachability probe gating the rows (the SKIP + posture's factory-owner responsibility). **The harness server**: + dockerized `pglo-poc` (`postgres:16-alpine`) on :15432, db `blobs`, + user/password `postgres`/`poc` — verified live 2026-10-12; the + container is long-running in the dev environment (the alkstore + harness's same server). +- **Mirror the form-independent reference rows** (README §The + reference row set): INV-1 round-trips (content matrix × both put + paths, in-body), INV-6 stat discipline, the `NoLivenessSources`/ + `ProtectFailed` abort shapes (form-independent — abort before + staging), and the kv-only rejection row (threshold-agnostic by + discovery, unchanged). +- **The durable-schedule sweep rows** — the sqlite column's family + (README §The schedule-relative reading for SQL-backed columns): + INV-2 dedup and INV-3 stage hygiene with counts read against the + staged schedule, the two-sweep exact-count row over the full + protection set (both tiers, the window observed), the + sweep-vs-put race across the window delay (execution-time + re-arbitration), the holder-stages-only lease shape, and INV-5 list + correctness through durable-schedule outcomes. +- **The dual-tier dispatch rows** mirror the sqlite column's family + (pure-function routing, threshold boundaries, fall-through per op) + with the placement oracle adapted by medium: kv entries live as rows + in the pg schema's `blobs` table, large entries under the local + root — the oracle distinguishes by medium (an independent + connection counting the schema's rows vs the root's committed + files). +- **Virgin-store row**: reads are no-ops on a fresh schema (pool, + both GC tables, and committed tree all empty) — the constructor + always ensures the schema (the engine's recorded posture), this row + being the facade form. +- **Structural factory tests** (the mem/sqlite columns' pattern): + factory isolation (fresh schema per open, nothing shared — two + opens never observe one another) and teardown-against-a-panicking-row + (schemas really dropped). +- **README amendments** (the convention's normative test-side home): + the postgres rows table in §Columns beyond the reference set; the + column plan's SKIP-posture note now naming a landed column; current + state (four columns; the first SKIP-posture column, green against + the harness server). + +## Acceptance Criteria + +- [ ] The postgres column green **against the harness server** + (`pglo-poc`, :15432): the mirrored form-independent reference + rows + the durable-schedule sweep rows + the dispatch rows + the + virgin-store row, every row ADR-stamped and traceable to the + backlog +- [ ] Server-less runs skip cleanly (the reachability probe; the + default-build gates stay green with the column skipped) +- [ ] Factory isolation + idempotent teardown hold (fresh schema per + open; `DROP SCHEMA … CASCADE` over exactly the minted schemas; + teardown safe against a panicking row), structurally exercised +- [ ] README amendments recorded (the postgres rows table; current + state) +- [ ] `cargo test` (default) green — the column compiles out; + `cargo test --features postgres` green server-less (skipping) + and green against the server at acceptance; + `cargo clippy --all-targets -- -D warnings` and + `cargo fmt --check` green (default + `--features postgres`) + +## References + +- tests/conformance/README.md (the conventions this grows against; + §SKIP posture, §Columns beyond the reference set, §The + schedule-relative reading for SQL-backed columns) +- tests/conformance/factory.rs (the factory contract this implements) +- tests/conformance/rows.rs (the reference row set being mirrored) +- tests/conformance/sqlite_column.rs (the SQL-backed column pattern — + the durable-schedule rows' first host) +- /workspace/@alkdev/alkstore/alkstore-postgres/tests/contract_suite.rs + (the pg-factory pattern reused: fresh schema per open, the schema + registry, the CASCADE teardown, the env-driven DSN) +- docs/architecture/store-api.md (§Invariants — the test gate) +- docs/architecture/decisions/012-pre-decomposition-consistency-rulings.md + §6.7 (the schedule the sweep rows assert against) +- docs/architecture/decisions/007-two-kv-engines-sqlite-and-postgres.md + §5 (the exact-count sweep-outcome gate against dockerized postgres) + +## Notes + +> To be filled by implementation agent + +## Summary + +> To be filled on completion \ No newline at end of file