docs(tasks): decompose wave 4 — the postgres kv engine, the fleet host's full seam, the first SKIP-posture column
Five tasks against the wave-3 contract surface as reviewed (no amendments from the gate): - engine-postgres: the fleet-capable kv engine — the B5 row shape (fillfactor 90) in one engine-owned schema, the deadpool pool with per-connection prepared-statement discipline, the synchronous_commit parity knob wired per-connection, ON CONFLICT DO NOTHING CAS, the stable MVCC-snapshot list cursor, the batch-delete helper (key = ANY($1), consumer wave 6), the read-only virgin-store split - engine-postgres-fleet: the engine-state seam's full clause families (ADR-012 §5's postgres row — every family yes): gc_pins + gc_window, the joint tx ownership rule in the JointTx shape, pin_commit atomicity, pin_renew/pin_check/arbitrate, the pg advisory lock as the sweep lease's shipped provider; fleet activation stays wave 6 - store-postgres-construction: KvEngine::Postgres slots in, the KvCursor arm, the dual-tier/kv-only constructors (DSN + opts + threshold + window delay), the non-fleet posture recorded, the pg-pure build verified - suite-column-postgres: the first SKIP-posture column — the alkstore pg-factory pattern reused (fresh schema per open, CASCADE teardown, env-driven ALKBLOBS_PG_* settings), durable-schedule sweep rows, acceptance green against the harness server (pglo-poc :15432, verified live) - review-wave-4: the engine-wave gate (engine-vs-contract conformance) Verification: dependency graph validated (all ids resolve, no cycles, topological order clean); docs-only change. Refs: docs/plans/implementation.md §Wave 4
This commit is contained in:
glm-5.3-flash committed
1 parent
611d30a3c4
commit
7ea6dd6bcb
6 files changed
+752
-2
No files matched your search
@@ -1,6 +1,6 @@
|
||||
---
|
||||
status: draft
|
||||
last_updated: "2026-10-11 (wave 3 decomposed into tasks/)"
|
||||
last_updated: "2026-10-12 (wave 4 decomposed into tasks/)"
|
||||
---
|
||||
|
||||
# alkblobs — Implementation plan
|
||||
@@ -51,7 +51,7 @@ ops payload surfaces; release readiness is last.
|
||||
| [1](#wave-1--foundations) | Crate/feature scaffold; hashing & keys (the one-way door); error model; Backend trait + I/O seams + engine-state seam shape; facade value types; conformance-suite convention | — | decomposed (2026-10-11) |
|
||||
| [2](#wave-2--the-store-works-dependency-free-mem--local) | mem + local engines; facade put/read paths + dispatch + constructor modes; in-process GC protocol; suite's first columns | wave 1 | decomposed (2026-10-11) |
|
||||
| [3](#wave-3--sqlite-engine) | sqlite kv engine (default): WAL posture, prepared statements, engine-state window family, suite column | wave 2 | decomposed (2026-10-11) |
|
||||
| [4](#wave-4--postgres-kv-engine) | postgres kv engine: schema, pool, CAS rows, stable-cursor list, full engine-state clause families, suite column | wave 2 | pending |
|
||||
| [4](#wave-4--postgres-kv-engine) | postgres kv engine: schema, pool, CAS rows, stable-cursor list, full engine-state clause families, suite column | wave 2 | decomposed (2026-10-12) |
|
||||
| [5](#wave-5--pg-lo-large-engine) | pg-lo large engine: LO staged put, descriptorless lo_get windows, companion rows, window-family seam clauses, suite column | wave 4 | pending |
|
||||
| [6](#wave-6--fleet-gc) | Fleet GC mechanism: DB-backed pins, joint entry+pin tx, sweep lease, SQL delete arbitration, fleet-validity predicate + constructor declarations | waves 4 + 5 | pending |
|
||||
| [7](#wave-7--ops-surface) | `ops` feature: alkcall-backed op family, ACL mapping, pin tokens + renewal, fanout seam | wave 6 | pending |
|
||||
|
||||
@@ -0,0 +1,198 @@
|
||||
---
|
||||
id: engine-postgres-fleet
|
||||
name: postgres engine-state clause families — the fleet host's full seam
|
||||
status: pending
|
||||
depends_on: [engine-postgres]
|
||||
scope: moderate
|
||||
risk: medium
|
||||
impact: component
|
||||
level: implementation
|
||||
tags: [wave-4, engines, postgres, gc]
|
||||
---
|
||||
|
||||
## Description
|
||||
|
||||
Implement the engine-state companion seam's **full clause families** on
|
||||
the postgres backend (`src/postgres.rs` or its seam module): the
|
||||
`EngineWindowState` impl **and** the `EngineFleetState` impl per
|
||||
ADR-012 §5's postgres row — every family **yes** (postgres is the
|
||||
fleet host, ADR-012 §4). This is the seam implementation pattern
|
||||
pg-lo's window family reuses (wave 5) and the fleet wave's mechanics
|
||||
ride (wave 6).
|
||||
|
||||
- **`EngineWindowState`** (the window family — active durably
|
||||
single-node, fleet-active when declared):
|
||||
- **`ensure_gc_schema`** — creates **`gc_pins` + `gc_window`** (the
|
||||
fleet host's both-tables row; unconditionally — the applicability
|
||||
row does not condition on the fleet declaration):
|
||||
`gc_pins (key bytea, owner bytea, expiry bigint)` and
|
||||
`gc_window (key bytea, visible_after bigint)` per ADR-010 §2's
|
||||
schema. `owner` is the store-instance id (ADR-011's vocabulary);
|
||||
the pin row's identity is `(key, owner)` — protection composes by
|
||||
row. The embedder's liveness table is never created here (ADR-008's
|
||||
schema-ownership rule — the sweep reads it and never owns it).
|
||||
Called at store construction by the store core
|
||||
(`store-postgres-construction` wires the call); the clause body is
|
||||
engine-side.
|
||||
- **`window_stage`** — stage candidates into the durable window,
|
||||
**idempotent per `(key, deadline)`** (the upsert discipline — a
|
||||
same-deadline re-stage is idempotent, a new-deadline re-stage
|
||||
updates the row; the sqlite engine's recorded semantics).
|
||||
- **`window_take(now)`** — take overdue candidates
|
||||
(`visible_after <= now`, the `<=` boundary the seam doc pins): the
|
||||
rows **leave the window table and return as keys** — one atomic
|
||||
step (selected rows removed and returned together; a row is never
|
||||
handed out twice). Empty when nothing is overdue; virgin store:
|
||||
empty.
|
||||
- **Timestamp encoding** (recorded design point): the sqlite
|
||||
engine's signed i64 nanosecond count from the Unix epoch, carried
|
||||
as `BIGINT` — the seam's crate-wide encoding (sub-second precision
|
||||
for the test tunings, pre-epoch negatives, full i64 range); the
|
||||
sqlite engine's `time_to_nanos`/`nanos_to_time` helpers are
|
||||
reference, not a dependency (per-engine bodies, one encoding).
|
||||
- **`EngineFleetState`** (the fleet family — the postgres host's
|
||||
alone; **fleet activation is wave 6**: under `fleet: false` the
|
||||
store core never calls these — the clause bodies exist and are
|
||||
contract-tested per applicability):
|
||||
- **`JointTx`** — the pg transaction the pooled client carries
|
||||
(recorded design point: the concrete pooled-transaction type and
|
||||
how it keeps its connection checked out for the tx's life; `Send`
|
||||
per the trait bound). **Ownership rule, pinned in the signature**:
|
||||
the store core holds the tx from `begin_joint_tx` until
|
||||
`pin_commit` consumes it; the engine contributes mechanics and
|
||||
never owns it (ADR-010 §2's ruling).
|
||||
- **`begin_joint_tx`** — open the tx on a pooled connection,
|
||||
uncommitted until `pin_commit`.
|
||||
- **`joint_put`** — the entry half: the engine's put mechanics (the
|
||||
`ON CONFLICT DO NOTHING` CAS clause) executed **inside the held
|
||||
tx**, same first-writer-wins semantics as the public `put`.
|
||||
- **`pin_commit(tx, entries)`** — the joint terminal step: the
|
||||
pin-row inserts **and** the commit — publish and protection land
|
||||
in one step (ADR-005's pin-before-publish, fleet form). On any
|
||||
failure the entire tx aborts: no pin rows, no published entry,
|
||||
zero residue — the stage-hygiene invariant's tx form. Consumes the
|
||||
tx.
|
||||
- **`pin_renew(key, owner)`** — `true` when a **live** (un-expired)
|
||||
`(key, owner)` row existed and its expiry moved to this call's
|
||||
successor time; an expired or absent row is never resurrected:
|
||||
`false`, nothing updated (ADR-012 §2's expiry semantics verbatim).
|
||||
**Named design point — the successor-time source**: the clause
|
||||
signature carries no expiry (ADR-010 §2: the TTL lives with the
|
||||
store's renewal schedule, the engine owns only the SQL) while the
|
||||
row update needs the new expiry — decide the wiring consistent
|
||||
with the seam doc's reading (the natural shape: the engine
|
||||
computes `expiry = now + TTL` from the shipped pin-TTL constant —
|
||||
ADR-008's default 60 s, constructor-tunable when wave 6's fleet
|
||||
constructor lands — and the store owns the TTL/3 cadence, i.e.
|
||||
*when* renewal is called) and record it.
|
||||
- **`pin_check(key)`** — classify against un-expired pin rows: any
|
||||
un-expired `(key, *)` row exists → `PinState::Protected`; else
|
||||
`Unprotected`. The engine never decides deletion — it only
|
||||
classifies.
|
||||
- **`arbitrate(key)`** — the full delete-time arbitration (ADR-008):
|
||||
the un-expired pin rows **plus the embedder's registered liveness
|
||||
tables**, under the `SELECT … FOR UPDATE` finger-lock keyed at
|
||||
delete time, folded into one answer — `true` = liveness protects
|
||||
the key at this instant. The engine-side **registration of named
|
||||
liveness-table references** is part of this body (ADR-012 §6.6:
|
||||
the fleet form of `register_liveness_source` registers a named
|
||||
table reference *in the shared engine*) — a crate-internal
|
||||
engine-side mechanism the store core's fleet registration verb
|
||||
forwards to (the facade verb itself is wave 6). The engine holds
|
||||
table *references*, never liveness semantics; the referenced
|
||||
tables are never created by the engine (the schema-ownership
|
||||
rule). Contract tests use a test-owned embedder table.
|
||||
- **`sweep_lock_acquire()` / `sweep_lock_release()`** — the sweep
|
||||
lease's shipped provider: the **pg advisory lock** with ADR-013
|
||||
§2's named exclusive semantics — atomic test-and-set
|
||||
(`pg_try_advisory_lock`), one sweeper per pool, `Ok(true)` = held,
|
||||
`Ok(false)` = contention (the second sweeper's sweep aborts with
|
||||
`GcAbortCause::SweeperLock` as report data — the store core's
|
||||
reading, wave 6). **Named design points**: session-scoped vs
|
||||
transaction-scoped (session-scoped — ADR-013 §2's holdership rule
|
||||
requires holdership to end at owner death/session loss, which only
|
||||
the session scope provides); the lock key (a fixed key,
|
||||
schema-qualified or namespaced so co-tenant engines on one
|
||||
instance never collide); and **acquire/release on the same pooled
|
||||
connection** (advisory locks are per-session — the lease must hold
|
||||
its client for its duration; the release-on-another-connection
|
||||
hazard is the recorded pitfall).
|
||||
- **Contract tests** per the applicability row (ADR-012 §5's table is
|
||||
the contract-test matrix): the schema creates both tables (and the
|
||||
embedder's liveness table never); stage idempotence per
|
||||
`(key, deadline)` incl. the re-stage-at-new-deadline form; take
|
||||
returns exactly the overdue set, removes it, leaves future rows; the
|
||||
`<=` boundary; the joint tx atomicity (a forced failure → no pin
|
||||
rows, no entry, zero residue); `pin_renew`'s live/expired/absent
|
||||
semantics; `pin_check`'s two-variant classification; `arbitrate`
|
||||
folding pin rows + a registered test-owned liveness table under the
|
||||
finger-lock; the advisory lock's acquire/contention/release shape
|
||||
and the session-loss reaping semantics (a dropped client's lock is
|
||||
gone — testable by dropping the lease's connection).
|
||||
- **Tests need a live server** — the alkstore harness convention:
|
||||
env-driven (`ALKBLOBS_PG_*`), skip cleanly server-less, acceptance
|
||||
requires green against the harness server (`pglo-poc`, :15432, db
|
||||
`blobs`, `postgres`/`poc`).
|
||||
|
||||
The store core drives these clauses in wave 6 (fleet mode); the
|
||||
non-fleet store construction consumes `ensure_gc_schema` only
|
||||
(`store-postgres-construction`). This task ships the clauses and their
|
||||
engine-level contract tests.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- [ ] The postgres backend implements `EngineWindowState` and
|
||||
`EngineFleetState` completely — every clause family of ADR-012
|
||||
§5's postgres row
|
||||
- [ ] `ensure_gc_schema` creates `gc_pins` + `gc_window`; the
|
||||
embedder's liveness table is never created by the engine
|
||||
(contract-tested)
|
||||
- [ ] `window_stage` idempotent per `(key, deadline)`; `window_take`
|
||||
returns exactly the overdue set (`<= now` boundary), removes it
|
||||
atomically, leaves future rows; empty on none/virgin
|
||||
- [ ] The joint tx atomicity holds: `pin_commit` publishes entry +
|
||||
pin rows in one step; any failure aborts with zero residue
|
||||
(contract-tested)
|
||||
- [ ] `pin_renew` never resurrects an expired/absent row;
|
||||
`pin_check` classifies per the two-variant `PinState`;
|
||||
`arbitrate` folds pin rows + registered liveness-table
|
||||
references under the `FOR UPDATE` finger-lock (contract-tested
|
||||
with a test-owned embedder table)
|
||||
- [ ] The advisory lock: atomic test-and-set, contention →
|
||||
`Ok(false)`, release no-op-safe, session-loss reaping; the
|
||||
same-connection acquire/release discipline recorded
|
||||
- [ ] Named design points recorded: the `JointTx` concrete type, the
|
||||
`pin_renew` successor-time source, the timestamp encoding, the
|
||||
lock key + scope, the lease's connection holding
|
||||
- [ ] Contract tests env-driven, skipping cleanly server-less; green
|
||||
against the harness server at acceptance
|
||||
- [ ] `cargo test --features postgres` green server-less (skipping);
|
||||
`cargo clippy --all-targets -- -D warnings` and
|
||||
`cargo fmt --check` green (default + `--features postgres`)
|
||||
|
||||
## References
|
||||
|
||||
- src/gc.rs (`EngineWindowState` + `EngineFleetState` — the seam
|
||||
contracts and their clause docs)
|
||||
- docs/architecture/decisions/010-backend-io-seams-and-fleet-state-home.md
|
||||
§2 (the clause bodies; the tx-ownership ruling; the advisory lock)
|
||||
- docs/architecture/decisions/012-pre-decomposition-consistency-rulings.md
|
||||
§3 (fleet activation is a declaration), §4 (the GC-state host), §5
|
||||
(the applicability table — postgres all-yes), §6.6 (the liveness
|
||||
registration's API home), §6.7 (the executor)
|
||||
- docs/architecture/decisions/008-trait-size-probe-fleet-gc-and-large-engines.md
|
||||
§2 (the fleet mechanism: pin rows, expiry, TTL 60 s, the advisory
|
||||
lock, the staged window)
|
||||
- docs/architecture/decisions/013-family-composition-reactive-boundary-and-feature-graph.md
|
||||
§2 (the sweep lease's named exclusive semantics)
|
||||
- src/sqlite.rs (the window family's reference implementation — same
|
||||
semantics, different dialect)
|
||||
- tasks/engine-postgres.md (the pool/schema/driver posture this rides)
|
||||
|
||||
## Notes
|
||||
|
||||
> To be filled by implementation agent
|
||||
|
||||
## Summary
|
||||
|
||||
> To be filled on completion
|
||||
@@ -0,0 +1,181 @@
|
||||
---
|
||||
id: engine-postgres
|
||||
name: postgres kv engine — the fleet-capable engine
|
||||
status: pending
|
||||
depends_on: []
|
||||
scope: narrow
|
||||
risk: medium
|
||||
impact: component
|
||||
level: implementation
|
||||
tags: [wave-4, engines, postgres]
|
||||
---
|
||||
|
||||
## Description
|
||||
|
||||
Implement the `postgres` kv-tier engine (feature `postgres`,
|
||||
default-off; ADR-007; REQ-2/REQ-3) against the wave-1 trait
|
||||
(`src/backend.rs`) — the second SQL-backed engine and the fleet host
|
||||
(ADR-012 §4). The sqlite engine (`src/sqlite.rs`) is the structural
|
||||
twin: same contract, different driver cluster and posture deltas.
|
||||
|
||||
- **Backend impl** (`src/postgres.rs`, behind
|
||||
`#[cfg(feature = "postgres")]`): the seven methods plus
|
||||
`put_staged`, opaque byte keys/values, `name() -> "postgres"`.
|
||||
- **Schema bootstrap** — one engine-owned PostgreSQL schema
|
||||
(recorded design point: the default schema name, the
|
||||
quoted-identifier boundary for every identifier entering SQL text
|
||||
— the alkstore `quote_identifier` pattern, `alkstore-postgres`
|
||||
src/schema.rs) holding the POC #5 finding B5 row shape:
|
||||
`blobs (key bytea PRIMARY KEY, value bytea NOT NULL, size bigint
|
||||
NOT NULL) WITH (fillfactor = 90)`. Idempotent DDL
|
||||
(`CREATE … IF NOT EXISTS`) at the constructor; no migration
|
||||
machinery. The dedicated `size` column is the **metadata-only
|
||||
`size` probe**: `SELECT size … WHERE key = $1` rides the PK index
|
||||
and never loads the value blob (ADR-008's size discipline).
|
||||
- **Pool** — deadpool-postgres over tokio-postgres
|
||||
(`deadpool_postgres::Pool`); pool size configurable with a named
|
||||
shipped default (recorded design point; the alkstore
|
||||
`DEFAULT_MAX_SIZE = 8` precedent). **Connection pooling is
|
||||
load-bearing, not an optimization** (POC #5 B2: fresh sessions
|
||||
cost ~20 ms) — the module doc records the posture.
|
||||
- **Per-connection prepared-statement discipline** — prepared
|
||||
statements die with sessions (POC #5 B5 hit the "prepared
|
||||
statement s_1 does not exist" error exactly when a pool-recycled
|
||||
connection lost its statement); every engine statement rides
|
||||
`prepare_cached` on the pooled client so the cache is
|
||||
per-connection by construction. Recorded design point.
|
||||
- **`synchronous_commit` parity knob** — configurable per engine
|
||||
instance, wired on **every pooled connection** (the alkstore
|
||||
POC-verified per-session connect-options mechanics,
|
||||
`alkstore-postgres` src/opts.rs). Default `on` (the ship config);
|
||||
`off` is the documented sqlite-NORMAL-alike latency trade (B5:
|
||||
latency moves ~2-3× across settings). This is the parity knob
|
||||
with sqlite's `synchronous=NORMAL`-WAL shipped tier — recorded in
|
||||
the module doc as posture, not hidden.
|
||||
- **CAS rows** — `INSERT … ON CONFLICT DO NOTHING`
|
||||
first-writer-wins on both put forms; a re-put of an existing key
|
||||
is a no-op, never an overwrite (ADR-007 §3: the sqlite
|
||||
`INSERT OR IGNORE` mapping).
|
||||
- `has` — an **EXISTS probe** (`SELECT EXISTS(…)`) — never a value
|
||||
load. `get` — the **materializing cursor body** (kv family): the
|
||||
whole value loaded once (bounded by tier policy), `len()` from the
|
||||
row's size column, reads slice the materialized value composing
|
||||
`backend.rs`'s `clamped`. `put_staged` — the **degenerate
|
||||
in-memory stage** (ADR-010 §1, the sqlite engine's shape):
|
||||
`append` extends the buffer, `commit` publishes via the same
|
||||
`ON CONFLICT DO NOTHING` clause and self-discards on failure,
|
||||
`discard` drops the buffer with zero residue.
|
||||
- `delete` — per-key rows-affected discipline
|
||||
(`Err(Error::Missing)` when absent). **Plus the crate-internal
|
||||
batch-delete helper** — `DELETE … WHERE key = ANY($1)` over a
|
||||
`bytea[]` parameter (ADR-007 §5's sweep-safety mechanics): the
|
||||
sweep's batched delete form, contract-tested in-module; its
|
||||
store-core consumer is wave 6's fleet sweep (scope-noted
|
||||
`#![allow(dead_code)]` until then — the wave-1 seam-shape
|
||||
discipline).
|
||||
- `list` — **complete and well-formed as a stable cursor over a
|
||||
vacuuming table** (ADR-007 §5): one statement's MVCC snapshot is
|
||||
stable for its duration (ghost rows from concurrent vacuuming
|
||||
never appear); the natural realization is the single
|
||||
`SELECT key … ORDER BY key` statement with tokio-postgres's
|
||||
row-streaming portal batches (bounded memory on huge pools) — the
|
||||
exact fetch shape is a recorded design point.
|
||||
- **Virgin-store posture** — the constructor creates the schema and
|
||||
tables; the tolerance is the sqlite engine's recorded read-only
|
||||
split (reads on a schema-less database map to absent/empty, writes
|
||||
surface the engine error — never silently creating schema). The
|
||||
test shape (a fixture connecting without running the bootstrap)
|
||||
mirrors `src/sqlite.rs`'s `new_tableless` pattern. Recorded design
|
||||
point.
|
||||
- **Autovacuum/fillfactor posture** — stated in the module doc, not
|
||||
silence (POC #5 B5: autovacuum is a real maintenance surface for a
|
||||
churning CAS pool; fillfactor 90 measured; the ~40× small-tier
|
||||
storage overhead accepted per B6 — documented, not hidden). The
|
||||
B5 tuning numbers are POC-derived posture defaults (ADR-007 §6's
|
||||
deferred-cost pattern): the first real pg deployment verifies them
|
||||
against its media.
|
||||
- **Natively async** — tokio-postgres is async: **no `spawn_blocking`
|
||||
seam** (the sqlite engine's blocking discipline does not carry; the
|
||||
pool is the concurrency posture). No blocking I/O on the async path;
|
||||
no `unwrap`/`expect` outside tests; thiserror errors surface as
|
||||
`Error::Io` via the engine's mapping discipline (`Missing` via the
|
||||
rows-affected/virgin-store mapping).
|
||||
- **Unit tests** in-module: the method contract (virgin-store no-ops
|
||||
incl. the schema-less form, CAS re-put, Missing discipline, size
|
||||
exactness + metadata-only), the staged-put happy path and both
|
||||
hygiene forms (discard → zero residue; commit failure →
|
||||
self-discard — a `#[cfg(test)]` fail-commit knob standing in for
|
||||
media failure, the mem/sqlite pattern), list stability and
|
||||
completeness, the batch-delete helper, the `synchronous_commit`
|
||||
wiring. **Tests need a live server** — the alkstore harness
|
||||
convention: connection settings ride the environment
|
||||
(`ALKBLOBS_PG_HOST/PORT/USER/PASSWORD/DB`), tests skip cleanly
|
||||
server-less, acceptance requires green against the server. The
|
||||
harness server: dockerized `pglo-poc` (`postgres:16-alpine`) on
|
||||
:15432, db `blobs`, user/password `postgres`/`poc` (verified live
|
||||
2026-10-12).
|
||||
|
||||
The engine-state clause families (`EngineWindowState` +
|
||||
`EngineFleetState`) are `engine-postgres-fleet` (separate task); the
|
||||
constructor wiring is `store-postgres-construction`. This task ships
|
||||
the engine and its in-module unit tests only.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- [ ] `src/postgres.rs` implements `Backend` completely (seven methods
|
||||
+ `put_staged`) behind feature `postgres` (default-off), with
|
||||
the materializing cursor body and the degenerate staging body
|
||||
- [ ] One engine-owned schema (quoted-identifier boundary), the B5 row
|
||||
shape with fillfactor 90, idempotent bootstrap at the
|
||||
constructor; autovacuum/storage-overhead posture recorded in the
|
||||
module doc
|
||||
- [ ] deadpool pool with a named shipped default size;
|
||||
per-connection `prepare_cached` discipline; the
|
||||
`synchronous_commit` parity knob wired on every pooled
|
||||
connection, configurable, default `on`
|
||||
- [ ] CAS first-writer-wins on both put forms; re-put never overwrites
|
||||
- [ ] `size` reads the dedicated size column via the PK index, never
|
||||
the value blob (metadata-only, unit-tested)
|
||||
- [ ] `list` complete and well-formed as a stable MVCC-snapshot cursor
|
||||
(design point recorded); the batch-delete helper
|
||||
(`key = ANY($1)`) contract-tested, scope-noted until wave 6
|
||||
- [ ] Stage hygiene holds in both forms (discard → zero residue;
|
||||
failed commit → self-discard), unit-tested
|
||||
- [ ] Virgin-store reads are no-ops including the schema-less form
|
||||
(unit-tested); writes surface the error (the read-only split)
|
||||
- [ ] Unit tests env-driven (`ALKBLOBS_PG_*`), skipping cleanly
|
||||
server-less; green against the harness server at acceptance
|
||||
- [ ] `cargo test` (default — postgres is default-off, the engine
|
||||
compiles out) green; `cargo test --features postgres` green
|
||||
server-less (skipping); `cargo clippy --all-targets -- -D
|
||||
warnings` and `cargo fmt --check` green (default +
|
||||
`--features postgres`)
|
||||
|
||||
## References
|
||||
|
||||
- docs/architecture/backends-and-dispatch.md (§kv tier — postgres
|
||||
engine)
|
||||
- docs/architecture/decisions/007-two-kv-engines-sqlite-and-postgres.md
|
||||
(§4's B5 deltas as requirements; §5's sweep-safety proof; §6's
|
||||
deferred-cost pattern)
|
||||
- docs/architecture/decisions/003-backend-contract-and-dispatch.md
|
||||
(the trait contract; the three numbered put rules' kv side)
|
||||
- docs/architecture/decisions/010-backend-io-seams-and-fleet-state-home.md
|
||||
§1 (the whole-value put's pg body; the materializing cursor)
|
||||
- docs/research/poc-postgres-kv-findings.md (findings B1-B6 — the row
|
||||
shape, the pooling discipline, the durability knob, the storage
|
||||
overhead)
|
||||
- /workspace/@alkdev/alkstore/alkstore-postgres (the sibling harness
|
||||
patterns: src/opts.rs's PgOpts + per-session synchronous_commit
|
||||
wiring, src/schema.rs's bootstrap + quote_identifier, the suite
|
||||
factory's fresh-schema-per-open)
|
||||
- src/backend.rs (the trait + seam contracts this implements)
|
||||
- src/sqlite.rs (the structural twin's shapes this mirrors)
|
||||
|
||||
## Notes
|
||||
|
||||
> To be filled by implementation agent
|
||||
|
||||
## Summary
|
||||
|
||||
> To be filled on completion
|
||||
@@ -0,0 +1,114 @@
|
||||
---
|
||||
id: review-wave-4
|
||||
name: Wave 4 review gate — the postgres engine vs the contract
|
||||
status: pending
|
||||
depends_on: [suite-column-postgres]
|
||||
scope: narrow
|
||||
risk: low
|
||||
impact: phase
|
||||
level: review
|
||||
tags: [wave-4, review]
|
||||
---
|
||||
|
||||
## Description
|
||||
|
||||
The wave-4 review gate — the engine-wave review shape
|
||||
(docs/plans/implementation.md §Review gates: "engine-vs-contract
|
||||
conformance"), before wave 5 decomposes. The gate reads the
|
||||
implementation against the pinned ADR text line by line (the
|
||||
wave-1/2/3 gates' method).
|
||||
|
||||
Gate checks:
|
||||
|
||||
- **Engine-vs-contract conformance**: the suite column green against
|
||||
dockerized postgres (server-less runs skip cleanly — a skip never
|
||||
substitutes for acceptance); the engine-state clause families
|
||||
contract-tested per the applicability row (ADR-012 §5's postgres
|
||||
row — every family yes); virgin-store semantics; the `size`
|
||||
metadata-only discipline (the size-column probe never loads the
|
||||
value blob).
|
||||
- **The postgres posture vs ADR-007**: the B5 deltas carried as
|
||||
requirements, not hedges — the `synchronous_commit` parity knob
|
||||
(configurable, per-connection wiring, default `on`); pooled
|
||||
connections with per-connection prepared-statement discipline
|
||||
(statements die with sessions); autovacuum + fillfactor tuning
|
||||
stated in the module doc; the ~40× small-tier storage overhead
|
||||
accepted and documented; the B5 tuning numbers recorded as
|
||||
POC-derived posture defaults (ADR-007 §6's deferred-cost pattern).
|
||||
- **The sweep-safety proof vs ADR-007 §5**: `list()` complete and
|
||||
well-formed as a stable MVCC-snapshot cursor over a vacuuming
|
||||
table; the batch-delete helper (`key = ANY($1)`) contract-tested;
|
||||
the exact-count sweep-outcome rows green against the server.
|
||||
- **The seam families vs ADR-010 §2 / ADR-012 §4/§5**: the full
|
||||
clause families on postgres (the fleet host) — `ensure_gc_schema`
|
||||
(both tables, the embedder's liveness table never created),
|
||||
`window_stage`/`window_take` (the sqlite semantics' dialect twin),
|
||||
the joint tx ownership rule honored in the `JointTx` shape (the
|
||||
store holds; the engine contributes), `pin_commit`'s atomicity
|
||||
(publish + protection in one step, zero residue on failure),
|
||||
`pin_renew`'s never-resurrect semantics, `pin_check`'s
|
||||
classification-only discipline, `arbitrate`'s finger-lock fold,
|
||||
the advisory lock's ADR-013 §2 lease semantics (atomic
|
||||
test-and-set, session-scoped holdership, same-connection
|
||||
release). **Fleet activation absent by construction** — no
|
||||
`fleet:` parameter, no store-core fleet wiring (wave 6); the
|
||||
clause bodies exist and are contract-tested per applicability.
|
||||
- **The non-fleet posture vs ADR-012 §3**: the constructors are
|
||||
implicitly `fleet: false` — the in-process protocol verbatim; the
|
||||
recorded posture names wave 6's slotting-in as a non-event.
|
||||
- **Async/blocking discipline**: tokio-postgres natively async — no
|
||||
`spawn_blocking` seam (the sqlite discipline does not carry); no
|
||||
blocking I/O on the async path; the pool is the concurrency
|
||||
posture; no `unwrap`/`expect` outside tests.
|
||||
- **Named design points audited as resolved** (each recorded in its
|
||||
task's Notes): the schema name + quoted-identifier boundary; the
|
||||
pool default size; the prepared-statement discipline; the DSN +
|
||||
opts constructor shape; the virgin-store tolerance split; the
|
||||
list-cursor fetch shape; the timestamp encoding; the `JointTx`
|
||||
concrete type; the `pin_renew` successor-time source; the lock
|
||||
key + scope; the lease's connection holding.
|
||||
- **Family conventions**: no comments in code (doc comments on public
|
||||
API fine), thiserror, tokio-only async, no unwrap/expect outside
|
||||
tests, naming, one-module-per-file re-export shape, the
|
||||
`#![allow(dead_code)]` scope notes matching their remaining scope
|
||||
(the batch-delete helper's wave-6 note intact).
|
||||
- **Gates green**: `cargo test` (default), `cargo test --features
|
||||
mem`, `cargo test --features postgres` (server-less: skipping),
|
||||
`cargo test --all-features`, `cargo check --no-default-features
|
||||
--features postgres` (the pg-pure build), `cargo clippy
|
||||
--all-targets -- -D warnings` (default + `--features postgres` +
|
||||
`--all-features`), `cargo fmt --check`.
|
||||
|
||||
Findings are fixed inline where cheap, or recorded as wave-5
|
||||
amendments where they shape the next decomposition. The gate's
|
||||
completion is what unblocks wave 5's decomposition.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- [ ] Every gate check above read and verified against the cited ADR
|
||||
text (not from memory of it)
|
||||
- [ ] Named design points each decided, recorded, and consistent with
|
||||
the specs
|
||||
- [ ] Findings fixed inline or recorded as wave-5 amendments with
|
||||
owners
|
||||
- [ ] All gates re-run green after any fixes
|
||||
- [ ] Wave 5 decomposition unblocked (recorded in this task's Summary)
|
||||
|
||||
## References
|
||||
|
||||
- docs/plans/implementation.md (§Wave 4, §Review gates — engine wave
|
||||
reviews)
|
||||
- docs/architecture/decisions/007-two-kv-engines-sqlite-and-postgres.md
|
||||
- docs/architecture/decisions/010-backend-io-seams-and-fleet-state-home.md §2
|
||||
- docs/architecture/decisions/012-pre-decomposition-consistency-rulings.md §3/§4/§5/§6.6/§6.7
|
||||
- docs/architecture/decisions/013-family-composition-reactive-boundary-and-feature-graph.md §2/§4
|
||||
- docs/architecture/store-api.md
|
||||
- tasks/review-wave-3.md (the gate method; the structural twin's gate)
|
||||
|
||||
## Notes
|
||||
|
||||
> To be filled by the review agent
|
||||
|
||||
## Summary
|
||||
|
||||
> To be filled on completion
|
||||
@@ -0,0 +1,129 @@
|
||||
---
|
||||
id: store-postgres-construction
|
||||
name: postgres constructor wiring — the fleet-capable store shapes
|
||||
status: pending
|
||||
depends_on: [engine-postgres, engine-postgres-fleet]
|
||||
scope: narrow
|
||||
risk: low
|
||||
impact: component
|
||||
level: implementation
|
||||
tags: [wave-4, store, postgres, construction]
|
||||
---
|
||||
|
||||
## Description
|
||||
|
||||
Slot postgres into the facade's engine representation — the slotting-in
|
||||
pattern `src/store.rs`'s module doc pinned ("adding an enum variant and
|
||||
a named constructor is slotting in"):
|
||||
|
||||
- **`KvEngine::Postgres`** variant behind `#[cfg(feature =
|
||||
"postgres")]` + the dispatch arms (`put`/`get`/`has`/`size`/
|
||||
`delete`/`list`). The kv cursor dispatch slots the postgres
|
||||
materializing body into **`KvCursor`** (the recorded sub-enum design
|
||||
point: "postgres (wave 4) slots one arm in here" — the store.rs doc
|
||||
note is updated to the landed state).
|
||||
- **Named constructors**, feature-gated on `postgres` (default-off —
|
||||
they construct under `--features postgres`):
|
||||
- **dual-tier** `kv = postgres + large = local` — the
|
||||
fleet-capable-kv production shape (non-fleet here; see below);
|
||||
fallible (config parse, connect, schema bootstrap, the local root
|
||||
creation). Size-threshold dispatch at the constructor-tunable
|
||||
threshold as in the sqlite shape.
|
||||
- **kv-only(postgres)** — no large tier; `threshold` is the put-time
|
||||
rejection point (ADR-012 §6.2; the rejection machinery is the put
|
||||
seam's, already landed — only the constructor slots in).
|
||||
- Both take the **connection settings + engine opts** (recorded
|
||||
design point: the DSN form — the tokio-postgres `Config`-parseable
|
||||
string, the alkstore `open(config, PgOpts)` pin adapted to the
|
||||
named-constructor shape — and the opts shape carrying schema name,
|
||||
pool size, `synchronous_commit`; engine-configuration concerns
|
||||
ride the constructor parameters, never invented defaults) and the
|
||||
**delete-window delay** parameter (`Duration`, default
|
||||
`DEFAULT_DELETE_WINDOW_DELAY`, `0` the legal degenerate tuning —
|
||||
the sqlite constructors' signature shape).
|
||||
- **Non-fleet posture** — these constructors are implicitly
|
||||
`fleet: false` (ADR-012 §3): the ADR-005 in-process protocol applies
|
||||
verbatim regardless of engine; pins are in-process guards, the sweep
|
||||
lease is the in-process lock, and none of the `EngineFleetState`
|
||||
clauses run. **The `fleet: true` declaration is wave 6** (the
|
||||
fleet-validity predicate + constructor declarations) — no fleet
|
||||
parameter is added here; wave 6 slots it in without reshaping.
|
||||
Recorded in the constructor docs so the wave-6 slotting is a
|
||||
non-event.
|
||||
- **`ensure_gc_schema` at construction** — the store-core call
|
||||
(engine-neutral; the postgres arm creates `gc_pins` + `gc_window`
|
||||
per the fleet host's applicability row). A construction failure is
|
||||
`Err(Error::Io)` per the fallible-constructor pattern.
|
||||
- **The one-window-host rule's store-core posture** — already recorded
|
||||
(the durable window hosts on the kv engine when it is SQL-backed);
|
||||
the postgres arm slots in under it with no reshaping. Verify the
|
||||
recorded posture's wording covers postgres (it names "sqlite or
|
||||
postgres" already — ADR-012 §5) and that the form branch keys off
|
||||
the kv-engine enum, never a flag.
|
||||
- **Stale scope notes updated** — the engine-representation doc's
|
||||
"postgres (wave 4)" forward references, the `KvCursor` design-point
|
||||
note, and any `#![allow(dead_code)]` scope notes whose scope changes
|
||||
(the batch-delete helper's note stays — its consumer is still wave
|
||||
6).
|
||||
- **Constructor unit tests**: the modes construct and introspect
|
||||
correctly (`mode()`/`threshold()`), `ensure_gc_schema` ran (both GC
|
||||
tables exist in the fresh schema), the window-delay default is the
|
||||
pinned constant, kv-only rejection wiring unchanged, the dual-tier
|
||||
shape round-trips end-to-end (a smoke put/get through both tiers).
|
||||
Env-driven, skipping cleanly server-less (the engine tests'
|
||||
convention).
|
||||
- **The pg-pure build** — `cargo check --no-default-features
|
||||
--features postgres` (and the tests target under it): ADR-013 §4's
|
||||
supported non-default configuration first becomes constructible
|
||||
here; verify it compiles clean.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- [ ] `KvEngine::Postgres` slots in; all six kv ops dispatch; the kv
|
||||
cursor body slots into `KvCursor` (the design-point note updated
|
||||
to the landed state)
|
||||
- [ ] `dual_tier_postgres_local` + `kv_only_postgres` constructors
|
||||
exist behind feature `postgres`, take connection settings +
|
||||
engine opts + threshold + window delay, fallible where I/O is
|
||||
fallible
|
||||
- [ ] `ensure_gc_schema` runs at construction (both GC tables in the
|
||||
fresh schema); the window-delay shipped default is 5 min
|
||||
(pinned constant) and constructor-tunable; `0` legal
|
||||
- [ ] The non-fleet posture recorded (in-process protocol verbatim;
|
||||
the `fleet: true` declaration is wave 6, slotting in without
|
||||
reshaping)
|
||||
- [ ] The one-window-host posture verified to cover postgres; the form
|
||||
branch keys off the kv-engine enum
|
||||
- [ ] The pg-pure build (`--no-default-features --features postgres`)
|
||||
compiles clean
|
||||
- [ ] `cargo test` (default) green; `cargo test --features postgres`
|
||||
green server-less (skipping); `cargo clippy --all-targets -- -D
|
||||
warnings` and `cargo fmt --check` green (default +
|
||||
`--features postgres`)
|
||||
|
||||
## References
|
||||
|
||||
- src/store.rs (the engine-representation design point; the
|
||||
constructor modes; the slotting-in pattern; the `KvCursor`
|
||||
sub-enum design point)
|
||||
- docs/architecture/decisions/011-vocabulary-tiers-engines-and-fleet-state.md
|
||||
§4 (modes; the constructor table)
|
||||
- docs/architecture/decisions/012-pre-decomposition-consistency-rulings.md
|
||||
§3 (fleet activation is a declaration — wave 6), §5 (the
|
||||
one-window-host rule), §6.3 (dual-tier shapes)
|
||||
- docs/architecture/decisions/008-trait-size-probe-fleet-gc-and-large-engines.md
|
||||
§2 (the window delay, default 5 min)
|
||||
- docs/architecture/decisions/013-family-composition-reactive-boundary-and-feature-graph.md
|
||||
§4 (the feature graph; the pg-pure build)
|
||||
- tasks/engine-postgres.md, tasks/engine-postgres-fleet.md (the engine
|
||||
+ seam bodies this wires)
|
||||
- tasks/store-sqlite-construction.md (the structural twin's recorded
|
||||
design points)
|
||||
|
||||
## Notes
|
||||
|
||||
> To be filled by implementation agent
|
||||
|
||||
## Summary
|
||||
|
||||
> To be filled on completion
|
||||
@@ -0,0 +1,128 @@
|
||||
---
|
||||
id: suite-column-postgres
|
||||
name: Conformance suite — the postgres column (the SKIP-posture column)
|
||||
status: pending
|
||||
depends_on: [store-postgres-construction]
|
||||
scope: moderate
|
||||
risk: medium
|
||||
impact: phase
|
||||
level: implementation
|
||||
tags: [wave-4, suite, postgres]
|
||||
---
|
||||
|
||||
## Description
|
||||
|
||||
Grow the **postgres column** — the suite's first SKIP-posture column
|
||||
(README §SKIP posture: SQL-attached factories ride a server; when no
|
||||
server is reachable the column **skips cleanly** — a skip is not a
|
||||
failure — and the wave's acceptance requires green **against the
|
||||
server**). The column runs under `--features postgres` (feature
|
||||
`postgres`, default-off).
|
||||
|
||||
- **Factories** (`tests/conformance/postgres_column.rs`):
|
||||
- dual-tier `kv = postgres + large = local` at a column-declared
|
||||
threshold and the **test tuning of the window delay (`0` —
|
||||
execute-on-next-sweep)**;
|
||||
- `kv-only(postgres)` at a column-declared threshold (the rejection
|
||||
row's host; threshold materializable per the README's
|
||||
configuration contract).
|
||||
**The alkstore pg-factory pattern, reused** (`alkstore-postgres`
|
||||
tests/contract_suite.rs): a **fresh engine-owned schema per
|
||||
`open()`** — unique per call (seeded per tag/pid/time so two factory
|
||||
instances never share a schema name, racing opens included), tracked
|
||||
per-instance in a schema registry, teardown
|
||||
`DROP SCHEMA IF EXISTS … CASCADE` over exactly the schemas the
|
||||
instance minted (idempotent, owned — never the shared server's other
|
||||
schemas; safe against an abandoned store: `CASCADE` removes
|
||||
dependents server-side while the store's pooled connections fail
|
||||
closed on next use). Panicking-row-safe — the sqlite column's
|
||||
instance-tracks-its-own-site pattern over schemas instead of temp
|
||||
dirs.
|
||||
- **Connection settings ride the environment** — the alkstore harness
|
||||
convention: `ALKBLOBS_PG_HOST/PORT/USER/PASSWORD/DB` (db defaulting
|
||||
`postgres` if unset), a reachability probe gating the rows (the SKIP
|
||||
posture's factory-owner responsibility). **The harness server**:
|
||||
dockerized `pglo-poc` (`postgres:16-alpine`) on :15432, db `blobs`,
|
||||
user/password `postgres`/`poc` — verified live 2026-10-12; the
|
||||
container is long-running in the dev environment (the alkstore
|
||||
harness's same server).
|
||||
- **Mirror the form-independent reference rows** (README §The
|
||||
reference row set): INV-1 round-trips (content matrix × both put
|
||||
paths, in-body), INV-6 stat discipline, the `NoLivenessSources`/
|
||||
`ProtectFailed` abort shapes (form-independent — abort before
|
||||
staging), and the kv-only rejection row (threshold-agnostic by
|
||||
discovery, unchanged).
|
||||
- **The durable-schedule sweep rows** — the sqlite column's family
|
||||
(README §The schedule-relative reading for SQL-backed columns):
|
||||
INV-2 dedup and INV-3 stage hygiene with counts read against the
|
||||
staged schedule, the two-sweep exact-count row over the full
|
||||
protection set (both tiers, the window observed), the
|
||||
sweep-vs-put race across the window delay (execution-time
|
||||
re-arbitration), the holder-stages-only lease shape, and INV-5 list
|
||||
correctness through durable-schedule outcomes.
|
||||
- **The dual-tier dispatch rows** mirror the sqlite column's family
|
||||
(pure-function routing, threshold boundaries, fall-through per op)
|
||||
with the placement oracle adapted by medium: kv entries live as rows
|
||||
in the pg schema's `blobs` table, large entries under the local
|
||||
root — the oracle distinguishes by medium (an independent
|
||||
connection counting the schema's rows vs the root's committed
|
||||
files).
|
||||
- **Virgin-store row**: reads are no-ops on a fresh schema (pool,
|
||||
both GC tables, and committed tree all empty) — the constructor
|
||||
always ensures the schema (the engine's recorded posture), this row
|
||||
being the facade form.
|
||||
- **Structural factory tests** (the mem/sqlite columns' pattern):
|
||||
factory isolation (fresh schema per open, nothing shared — two
|
||||
opens never observe one another) and teardown-against-a-panicking-row
|
||||
(schemas really dropped).
|
||||
- **README amendments** (the convention's normative test-side home):
|
||||
the postgres rows table in §Columns beyond the reference set; the
|
||||
column plan's SKIP-posture note now naming a landed column; current
|
||||
state (four columns; the first SKIP-posture column, green against
|
||||
the harness server).
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- [ ] The postgres column green **against the harness server**
|
||||
(`pglo-poc`, :15432): the mirrored form-independent reference
|
||||
rows + the durable-schedule sweep rows + the dispatch rows + the
|
||||
virgin-store row, every row ADR-stamped and traceable to the
|
||||
backlog
|
||||
- [ ] Server-less runs skip cleanly (the reachability probe; the
|
||||
default-build gates stay green with the column skipped)
|
||||
- [ ] Factory isolation + idempotent teardown hold (fresh schema per
|
||||
open; `DROP SCHEMA … CASCADE` over exactly the minted schemas;
|
||||
teardown safe against a panicking row), structurally exercised
|
||||
- [ ] README amendments recorded (the postgres rows table; current
|
||||
state)
|
||||
- [ ] `cargo test` (default) green — the column compiles out;
|
||||
`cargo test --features postgres` green server-less (skipping)
|
||||
and green against the server at acceptance;
|
||||
`cargo clippy --all-targets -- -D warnings` and
|
||||
`cargo fmt --check` green (default + `--features postgres`)
|
||||
|
||||
## References
|
||||
|
||||
- tests/conformance/README.md (the conventions this grows against;
|
||||
§SKIP posture, §Columns beyond the reference set, §The
|
||||
schedule-relative reading for SQL-backed columns)
|
||||
- tests/conformance/factory.rs (the factory contract this implements)
|
||||
- tests/conformance/rows.rs (the reference row set being mirrored)
|
||||
- tests/conformance/sqlite_column.rs (the SQL-backed column pattern —
|
||||
the durable-schedule rows' first host)
|
||||
- /workspace/@alkdev/alkstore/alkstore-postgres/tests/contract_suite.rs
|
||||
(the pg-factory pattern reused: fresh schema per open, the schema
|
||||
registry, the CASCADE teardown, the env-driven DSN)
|
||||
- docs/architecture/store-api.md (§Invariants — the test gate)
|
||||
- docs/architecture/decisions/012-pre-decomposition-consistency-rulings.md
|
||||
§6.7 (the schedule the sweep rows assert against)
|
||||
- docs/architecture/decisions/007-two-kv-engines-sqlite-and-postgres.md
|
||||
§5 (the exact-count sweep-outcome gate against dockerized postgres)
|
||||
|
||||
## Notes
|
||||
|
||||
> To be filled by implementation agent
|
||||
|
||||
## Summary
|
||||
|
||||
> To be filled on completion
|
||||
Reference in new issue
Block a user