diff --git a/.opencode/agents/code-reviewer.md b/.opencode/agents/code-reviewer.md index dfe3129..0825398 100644 --- a/.opencode/agents/code-reviewer.md +++ b/.opencode/agents/code-reviewer.md @@ -111,13 +111,15 @@ cargo fmt --check # Format check For this project, also verify: - No comments in code (per project convention) -- Error handling uses `anyhow::Result` (application) / `thiserror` (library) — no +- Error handling uses `thiserror` (library crate) — no panics in library code -- Feature flags are used correctly (`tls`, `iroh`, `acme`) — base crate compiles - lean +- Feature flags are used correctly — base crate compiles lean - Public API is well-documented with `///` doc comments where appropriate - Module structure follows Rust conventions (`mod.rs`, `lib.rs`) - No unnecessary `unwrap()` or `expect()` in library code +- For database code: SQL injection safety (parameterized queries only, no + string-built SQL), connection/pool handling under cancellation (no + panics poisoning a shared pool), and migration/DDL ordering #### E. Security diff --git a/.opencode/agents/coordinator.md b/.opencode/agents/coordinator.md index 2035066..2644676 100644 --- a/.opencode/agents/coordinator.md +++ b/.opencode/agents/coordinator.md @@ -191,7 +191,7 @@ also include: Example prompt template: ``` -You are an implementation specialist for the @alkdev/alkblobs project. +You are an implementation specialist for the @alkdev/alkstore project. Your task: {{task}} @@ -204,7 +204,7 @@ Your task: {{task}} 7. Push: git push origin $(git branch --show-current) 8. Notify: worktree({action: "notify", args: {message: "Task completed: {{task}}. ", level: "info"}}) -Key project constraints (@alkdev/alkblobs): +Key project constraints (@alkdev/alkstore): - Rust: use cargo build, cargo clippy, cargo fmt, cargo test - No comments in code - thiserror for error types diff --git a/.opencode/agents/implementation-specialist.md b/.opencode/agents/implementation-specialist.md index 1a3bb64..06df865 100644 --- a/.opencode/agents/implementation-specialist.md +++ b/.opencode/agents/implementation-specialist.md @@ -207,7 +207,9 @@ This is especially important for complex tasks that span many file operations. ## Project Conventions -Read `AGENTS.md` at project root for full details. Key rules: +Read `AGENTS.md` at project root for full details. Key rules (the +cross-crate, stable ones — crate-specific conventions live in `AGENTS.md` +and evolve with the project): 1. **No comments in code** — Per project convention. Doc comments (`///`, `//!`) are fine and expected on public API. @@ -217,55 +219,19 @@ Read `AGENTS.md` at project root for full details. Key rules: operation does not cascade to other operations. 3. **`tokio` is the async runtime** — all I/O is async. Use `tokio::sync` primitives (`oneshot`, `mpsc`) for request correlation and subscription - channels; `parking_lot` for short-held internal locks (`PendingRequestMap`). -4. **No secret material on the wire** — `call.requested`/`call.responded` - payloads and `OperationContext.metadata` carry no private keys, API keys, or - decrypted credentials. Outbound credentials flow through `Capabilities` - injected at the assembly layer → `HandlerRegistration.capabilities` → - `OperationContext.capabilities` → handler. -5. **No-env-vars invariant** — no handler reads outbound credentials from any - source other than `OperationContext.capabilities`. This is a spec-level - invariant, not a runtime convention. -6. **`OperationEnv` must remain a trait** — the trait-based design enables - registry layering (session overlays, connection overlays, peer-keyed - composition). Do not make it concrete or hardcode the global registry. -7. **Wire formats are stable** — `EventEnvelope` (`{ type, id, payload }` + - length-prefixed JSON framing) and the channels 8-byte chunk header - (`[channel_id:u32 BE][length:u32 BE][payload]`) are one-way doors. New event - types may be added; existing shapes must not change. -8. **Producer/consumer, not server/client** — both sides of a call or channels - connection can initiate. Use "producer"/"consumer" or "accept side"/"connect - side," not "server"/"client." -9. **Vendored core types** — `Connection`, `ProtocolHandler`, `BiStream`, - `BidiStreamSource`, `AuthContext`, `IdentityProvider`, `Identity`, - `AuthToken`, `Capabilities`, `OwnershipProvider`, `HandlerError`, - `StreamError` live in this crate. Do not add a separate `alkcore` dependency. - Keep them lean (no TLS, no transport coupling, no endpoint/accept-loop). -10. **BAST documents for wire formats** — every binary wire format carries a - BAST (Binary Abstract Syntax Tree) document as its machine-readable spec - (e.g. the channels chunk header's `docs/architecture/chunk-header.bast.json`, - embedded as `CHUNK_HEADER_BAST`). BAST is plain JSON — no dependency - required to author or consume it. The `alktype` crate compiles BAST into - readers/writers/validators; future codegen derives language-specific - implementations. Trivial or hot-path formats (chunk header, tty framing) - stay hand-rolled with the BAST doc as the contract; complex formats (sftp) - use the alktype engine or codegen. Do not roll your own offset map or - validator for complex formats. -11. **Feature flags** — transports may be feature-gated if the need arises. The - base crate should compile lean (no `quinn`, no `iroh` unless the feature is - on). Verify both `cargo test` (default) and `cargo test --all-features` pass - if features are added. -12. **Abort cascades to descendants** — `call.aborted` for a parent cascades to - all non-terminal descendants. Default `abort-dependents`; - `continue-running` opt-in. The composing handler decides the child's policy, - not the wire caller. -13. **Peer authorization via `AccessControl`** — a remote peer's call is - authorized by `AccessControl::check(peer_identity)`. No `remote_safe` flag, - no `trusted_peer` bypass. `Visibility::Internal` ops are never wire-callable. -14. **Naming conventions** — Rust standard: `snake_case` for functions/variables/ - modules, `PascalCase` for types/traits, `SCREAMING_SNAKE_CASE` for constants. -15. **Module structure** — one module per file under `src/`, re-exported from - `src/lib.rs`. Public API surface is `lib.rs` re-exports. + channels; `parking_lot` for short-held internal locks. +4. **Naming conventions** — Rust standard: `snake_case` for functions/variables/ + modules, `PascalCase` for types/traits, `SCREAMING_SNAKE_CASE` for constants. +5. **Module structure** — one module per file under `src/`, re-exported from + `src/lib.rs`. Public API surface is `lib.rs` re-exports. +6. **Feature flags** — optional dependencies (e.g. the postgres engine) may be + feature-gated if the need arises. The base crate should compile lean. + Verify both `cargo test` (default) and `cargo test --all-features` pass + if features are added. +7. **Decisions land in ADRs** — if you discover a decision the architecture + didn't make (an undocumented trade-off, an interface shape you must + choose), do not decide it silently: notify the coordinator (`level: + "blocking"` if it changes the API surface) so an ADR gets written. ## Key Principles diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 0000000..87f8b94 --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,121 @@ +# AGENTS.md + +Operating instructions for opencode agents working in this repo. opencode +auto-loads this file as instructions, overriding the built-in defaults for +this project. Custom agents in `.opencode/agents/` inherit these rules +unless their own prompts say otherwise. + +## Repo posture + +This repo is in **Phase 0 (Exploration)** per `docs/sdd_process.md`. +There is no crate yet — no `Cargo.toml`, no `src/`, no README (the README +gets written just before first publish, so it can be honest about what +shipped). What exists is `docs/research/phase-0.md` (the Phase 0 +document: vision, prior art, open questions, POC register), the SDD +process (`docs/sdd_process.md`), and the agent defs +(`.opencode/agents/`). Do not scaffold implementation code +from this posture unless a task or the user asks for a POC. + +Do not treat `docs/research/phase-0.md` as settled architecture — it is +a working research document whose open questions are genuinely open. +The hedging rules of `docs/sdd_process.md` (no circular deferrals, no +"resolved with escape hatches") still apply to it. + +## Git Workflow + +**Commit and push when reasonable.** When a change is complete and +verified, commit and push to `origin/main` without asking. This +overrides the built-in default of "only commit when explicitly asked." +In Phase 0 "verified" means the doc checks below — there is no build +gate yet. + +Exceptions — **do not** commit or push without asking: + +- The change is exploratory / speculative (you're not sure the user + wants it kept) +- The user is actively reviewing the diff and may ask for changes +- You'd be force-pushing, amending a pushed commit, creating an empty + commit, or skipping hooks + +Never commit secrets, keys, or credentials. If a commit fails or hooks +reject it, fix the issue and create a new commit — do not amend the +failed one. + +Git identity is preconfigured (`glm-5.3-flash `). Do +not change `git config`, skip hooks, or use `git commit -i`. + +## Phase 0 conventions + +1. **Findings land in `docs/research/`.** Research notes, POC + specifications, and POC findings all go there, named consistently + (`poc--spec.md` / `poc--findings.md` follows the + alkblobs precedent). A POC that needs this repo's code runs in a + worktree (`.worktrees/research//` per the SDD process); a + self-contained POC (like the alkblobs ones) runs as a standalone + crate in the global workspace. Findings land in this repo either + way. +2. **Open questions get OQ-ST-NN IDs** and live in the register in + `docs/research/phase-0.md` until Phase 1 promotes them to + `docs/architecture/open-questions.md`. Numbering is stable — never + renumber; append. +3. **Reference read-only checkouts by path and revision.** The external + references for this crate (`/workspace/honker`, + `/workspace/pgboss-rs`) are read-only reference checkouts of + third-party projects. Note the checkout state when your findings + depend on code specifics; licenses and provenance get recorded when + we adopt code, not while only reading. +4. **POC code stays out of this repo** until Phase 1 adopts it. A POC + crate proves the hypothesis and reports findings; it does not + pre-scaffold the real crate layout. +5. **No mock/main-repo documentation.** No README, no CHANGELOG, no + crate-level docs until the shape exists. Docs that describe a thing + that doesn't exist are exactly the kind of dishonest artifact we + write docs to avoid. +6. **No comments in code** (when POC crates get written) unless the + user explicitly asks. Doc comments (`///`, `//!`) are fine on public + API. Inline `//` comments only when a non-obvious safety/correctness + constraint would otherwise be missed, or the user asks. + +## Conventions that will apply once the crate exists + +These are family-standard and pre-committed; they'll be duplicated into +task prompts and tightened by Phase 1 ADRs: + +- Rust crate, `tokio` async runtime, `thiserror` error types, no panics + in library code, no `unwrap()`/`expect()` outside tests. +- No comments in code (see 6 above). +- Module-per-file under `src/`, re-exported from `src/lib.rs`; public + API surface is the lib re-exports. +- Optional dependencies (the postgres engine likely being the first) + are feature-gated; the base crate compiles lean; both + `cargo test --features ` and default-feature builds pass when + features exist. +- Categorical estimates (`scope`, `risk`, `impact`, `level`) in every + task file — per `docs/sdd_process.md` these are structurally + required, not optional metadata. + +## Verification Commands + +Phase 0 — none (no code yet). The doc conventions above are checked by +reading. When the crate appears, this section gets the standard Rust +gates (`cargo test`, `cargo clippy --all-targets -- -D warnings`, +`cargo fmt --check`, and the fuzz corpus replay if fuzzing is adopted — +the alksocks/alktty/alktunnels pattern) and becomes the merge gate the +coordinator runs. + +## Architecture Context + +- `docs/research/phase-0.md` — the Phase 0 document: vision, prior + art, open questions (OQ-ST-01..NN), the POC register, and (eventually) + the converged recommendation. Read it before any non-trivial work in + this repo. +- `docs/sdd_process.md` — the SDD process. Phase 0 in progress; + `docs/architecture/` does not exist yet. +- The SDD process and agent defs were copied from downstream projects + (alkblobs) in the workspace; project-specific stale references were + cleaned at Phase 0 setup (2026-10-03). If you find a section that + assumes another crate's machinery, fix it rather than working around + it — same discipline as stale TODOs. +- If a TODO or doc reference cites a design direction that a later ADR + or Phase 0 decision rejected, the reference is stale — remove it and + align; do not implement the rejected design. \ No newline at end of file diff --git a/docs/research/phase-0.md b/docs/research/phase-0.md new file mode 100644 index 0000000..d37cd2d --- /dev/null +++ b/docs/research/phase-0.md @@ -0,0 +1,359 @@ +--- +status: draft +last_updated: 2026-10-03 (initial draft from the setup discussion) +--- + +# alkstore — Phase 0 (Exploration) + +This document captures Phase 0 (Exploration) for the `alkstore` crate: +vision, guiding principles, prior art, and the open-question register +(OQ-ST-01..NN). Phase 0's objective per `docs/sdd_process.md`: *capture +vision and guiding principles; research options; validate approaches; +converge on a recommended approach.* This is the initial draft — none of +the questions below are settled, and there is no POC register yet. + +Context for why this crate starts now: **alkblobs** +(`/workspace/@alkdev/alkblobs` — spec + POCs only, paused mid-planning) +hit repeated circular hedging in its Phase 0, and a root cause was that +the storage substrate it deploys onto was itself disjoint and fuzzy — a +repo pattern with a default in-memory adapter across the alk* ecosystem, +cache-invalidation patches in hot paths, non-invalidated caches where +delay was tolerable, no single definition of "how does a change in the +database become visible to other processes/connections?" alkblobs paused +partly to let this crate answer that first. alkstore is the attempt to +make that substrate real once, so downstream stores don't re-derive it. + +## Vision and guiding principles + +**One sentence (draft):** one reactive store interface over SQLite and +Postgres — durable pub/sub notify, queues, streams, and the transactional +integration (write + enqueue in one transaction) that honker delivers on +SQLite — with the Postgres side building on the natively-available +machinery (`pg_notify`/`LISTEN`, and the pgboss job-queue schema family) +rather than emulating it. + +**The honker relationship.** `/workspace/honker` (read-only reference; +alpha-quality per its own README, MIT/Apache-2.0 dual) adds +Postgres-style `NOTIFY`/`LISTEN` semantics to SQLite without a broker: +durable at-least-once queues with retries/delay/priority/visibility +timeouts/dead-letter, durable streams with per-consumer offsets, +cron/`@every` scheduling, named locks, rate limits, transactional +outbox — all as INSERTs inside the caller's transaction, with the +cross-process wake delivered by a shared watcher that polls `PRAGMA +data_version` (default 1 ms → single-digit-ms delivery) and re-reads +indexed state after every wake. Its own Prior Art section names the +Postgres side: `pg_notify` gives fast triggers with no retry or +visibility semantics; pg-boss/Oban are the durable-layer gold standards +— "If you already run Postgres, use the Postgres tools." + +The goal is **not** "port honker to Postgres." The goal is the interface: +one store API whose consumer code (queues, streams, notify) looks the +same whether the backing engine is SQLite or Postgres, while each engine +uses its own native wake/delivery story under the hood. The honker docs +recommend pgboss + `pg_notify` for the Postgres equivalent — that +recommendation is the design brief for this crate's Postgres engine. + +**Consumer shape (from the paused alkblobs planning):** any crate that +today uses the ecosystem's repo-pattern + in-memory adapter should be +able to swap in an alkstore-backed engine and get durability + true +cross-process/cross-instance reactivity. That means the reactive surface +must compose with *client-side* caching: a subscriber that also holds a +cache can invalidate on notification instead of re-querying or re-polling +— the hot-path pattern the ecosystem already uses, given a real +invalidation source. + +Guiding principles: + +1. **One interface, two engines, native underneath.** The abstraction + layer unifies the consumer-visible features; the engines stay + dialects, not two emulations of one dialect. SQLite follows honker's + design (queue in the same file, same transaction, watcher-based + wake); Postgres follows pg-boss' design (schema-based job tables + + `pg_notify`-driven wake). A "lowest common denominator" unification + (both sides polling, both sides emulating LISTEN) is explicitly the + failure mode to avoid — it would re-create the fuzziness this crate + exists to remove. +2. **Transactional local-adjacency is the load-bearing property.** + Honker's core claim: enqueue/publish/notify in the same transaction + as the business write; rollback drops both. The unified surface must + preserve this on any engine, because the ecosystem's repo pattern + assumes it (a business write that loses its side-effect notification + is the dual-write problem honker names). +3. **Ownership of the whole stack.** honker is third-party; pgboss-rs + is third-party. Whether any of them are adopted, forked, or used as + schema/design reference only is a deliberate per-question decision — + not inherited by adjacency. The workspace precedent is the targeted + fork (alksocks' fast-socks5 extraction: adopt the design, own the + code, port to our conventions). +4. **Substrate-agnostic consumer API, engine-specific setup.** A + consumer opens a `Store` from a connection string / file path and + gets the same trait surface. Which engine is behind what can vary + (per-deployment config), but consumer code must not branch on + engine type. +5. **No panics, `tokio`, `thiserror`, lean base crate, feature-gated + optional engines** — family-standard, pre-committed (AGENTS.md). + +## The driver conflict (Phase 0's central tension) + +The immediate design fork, flagged by the user: + +- **The alkblobs POCs used `tokio-postgres` + `deadpool-postgres`** + (validated in `poc-postgres-kv-findings.md`, `poc-pglo-findings.md` — + including Large Objects work). +- **pgboss-rs (`/workspace/pgboss-rs`) uses `sqlx`** (`sqlx Postgres + runtime-tokio`). +- honker-core uses `rusqlite`. + +A single crate with both engines means a driver decision, and the +reactivity story is entangled with it: + +- **pgboss-rs currently has no `LISTEN`/`NOTIFY` at all** (verified + 2026-10-03 against the checkout — `src/` contains no LISTEN/NOTIFY + usage; consumption is `fetch_job` polling). The node original relies + on `pg-boss`'s own maintenance/polling; the port did not pick up a + push channel. So *even* "use pgboss for the queue" does not deliver + reactivity — LISTEN/NOTIFY wiring would be new work either way, and + the driver choice determines *whose* LISTEN plumbing (sqlx's + `PgListener` is built-in; tokio-postgres uses its `Connection` + notifications). +- **Honker's reactivity on SQLite is a watcher polling `PRAGMA + data_version`** — a fundamentally different mechanism from + LISTEN/NOTIFY. The unified reactive trait must abstract over both + without collapsing to the polling behavior of the weaker side. + +This tension is OQ-ST-04 below. It is *not* resolved by "pgboss is well +written so start there" — that is exactly the inherited-assumption +shape the SDD process flags. What pgboss-rs genuinely offers (schema +DDL, job states, retry semantics, the node-compatible API) is design +reference regardless of driver. + +## Prior art + +Notes below are from reading the checkouts on 2026-10-03; both external +projects are read-only references. Provenance/licensing gets recorded +per AGENTS.md §3 when code is adopted, not while only reading. + +### honker — the SQLite-side template + +`/workspace/honker` (SQLite extension + bindings). What matters for +this crate: + +- **The full feature set to match on Postgres** (its §What It Does): + notify/listen across processes, durable at-least-once queues + (retries, delayed jobs, priority, visibility timeouts, dead-letter + rows, result storage), durable streams with per-consumer offsets, + cron/`@every` scheduling, named locks, rate limits, transactional + outbox helpers. Deliberately excluded there: workflow DAGs, task + chains/chords, multi-writer replication, cross-machine locking — + scope line likely inherited, to be confirmed. +- **The wake mechanism** — `PRAGMA data_version` polling watcher + (default 1 ms; raise for idle CPU), re-read indexed state after + wake, overtriggering on purpose ("one indexed SELECT is cheap; a + missed wake is a correctness bug."). Optional kernel-events and WAL + shared-memory backends exist in source builds. +- **Single-machine honesty** — file-backed, one host; NFS-two-writers + explicitly not supported. This posture needs an explicit Postgres + counterpart (multi-host is Postgres' normal case, so the interface + must not bake SQLite's single-host assumption into the shared + surface). +- **The transactional enqueue shape** — every feature is an INSERT + inside the caller's transaction. This is the pattern the unified API + must keep visible and cheap. + +### pgboss-rs — the Postgres queue family reference + +`/workspace/pgboss-rs` (v0.1.0-rc6, MIT/Apache-2.0 dual). Ported from +node pg-boss: builder-based queue/job API, retry/delay/priority/ +singleton/dead-letter concepts, `sqlx` 0.8, schema-scoped DDL. + +- Verified gap (2026-10-03): **no LISTEN/NOTIFY** anywhere in `src/` + — consumption is polling `fetch_job`. Any push-reactivity is new + work, not an adoption freebie. +- Its value as reference: the pg-boss schema family (job states, + maintenance/dead-letter behavior) is battle-tested against real + Postgres semantics — worth borrowing *as design*, independent of the + driver decision. +- The node original (pg-boss) is the upstream of record for semantics + the port may have dropped; compare against it when adopting queue + semantics. + +### Honker's Postgres-side recommendation + +The honker README's own posture: if you run Postgres, use the Postgres +tools. `pg_notify` + pgboss is the recommended assembly. The design +brief: the queue machinery from the pg-boss family, the push semantics +from LISTEN/NOTIFY, the unified API shape from honker's Rust binding. + +### The alk* repo pattern — what this crate replaces + +The ecosystem's current shape: a repository trait with a default +in-memory adapter; cache-invalidation wiring in hot paths; uninvalidated +(non-reactive) caches where delay was acceptable; each project +composing these slightly differently. No persistence-backed reactive +substrate exists in the family — alkblobs was the first project to try +to plan against one, found it missing, and paused. This crate's reason +to exist is precisely that that substrate should exist once, well, +instead of per-project approximations. + +### alkcall — the substrate (not a dependency of the store layer) + +`/workspace/@alkdev/alkcall` (pure protocol crate, no transport). The +alk* crates (alktty, alktunnels, alksocks) are its consumers; a future +alkstore ops/protocol surface (if this crate ever exposes store access +over alkcall channels) rides the same substrate. Like the alkblobs +split (store layer stays substrate-free), the store layer here stays +alkcall-free; any networked surface is an ops module/sibling concern +and a separate decision. + +## Open Questions + +Register in `docs/research/phase-0.md`; IDs OQ-ST-NN (stable, append +only). Promotion target: Phase 1 `docs/architecture/open-questions.md`. + +### OQ-ST-01: Scope boundary — which honker features are in-scope? + +Honker's feature list (queues, streams, notify, scheduling, locks, rate +limits, outbox helpers) is large; per-feature scope decisions don't +exist yet. Also undetermined: the exclusion lines (honker deliberately +excludes DAGs, task chains/chords, multi-writer replication, +distributed locking). + +Not yet decidable without working through the concrete consumers (the +paused alkblobs crates and alkfs planning) — per-feature need hasn't +been articulated. Blocked on a consumer-driven inventory pass. + +### OQ-ST-02: Crate scope — one store crate, or reactive-core + engines? + +Options include: single crate with feature-gated engines (the alk* +feature-gate pattern); a core trait crate + per-engine crates; engine +crates consuming a thin core. The answer constrains the driver decision +(OQ-ST-04) and the base-crate-lean invariant. + +The use case isn't concrete yet (no engine code written, no consumer +wired). Deferred(scope) — concrete use case: the first engine +implementation would force this shape. + +### OQ-ST-03: Driver story — sqlx, tokio-postgres, or per-engine drivers? + +The named tension (§The driver conflict), restated as the decision: + +- **One driver across engines**: sqlx (both sqlite + postgres native + support, one API — but the alkblobs POC evidence is tokio-postgres) + vs tokio-postgres per-engine (sqlite story unclear — tokio-postgres + is pg-only; rusqlite is the sqlite native). +- **Per-engine drivers under a unified trait**: tokio-postgres + + deadpool-postgres (POC-validated in alkblobs findings) + rusqlite + (honker's choice, so honker's SQLite machinery ports cleanly). +- **Adopt/fork pgboss-rs**: brings sqlx along where the queue lives. + +Honest unknowns worth surfacing: does sqlx support SQLite +`data_version`/extension-style machinery equally well? Does a unified +trait over `(tokio-postgres, rusqlite)` pay more trait-fitting cost +than sqlx's single-API convenience costs elsewhere? What does +SQLITE_ENABLE/extension loading look like under sqlx vs rusqlite? + +Genuinely open; needs research rounds (library capabilities vs the +unified-trait shape) and possibly a POC. Not deferred — this is the +central Phase 0 research question. + +### OQ-ST-04: The reactive abstraction — what does the unified notify surface look like? + +The two engines' mechanisms are structurally different: SQLite = +watcher polling `PRAGMA data_version` (deliver on commit; no +server-side push exists), Postgres = LISTEN/NOTIFY (server push, +connection-bound, no retry/visibility semantics). The reactive trait +must have a shape both implement without one emulating the other's +weaknesses: + +- What is the subscription type (`channel`? stream of envelopes?) +- What is the delivery guarantee contract on each engine (honker's + wake-on-commit + re-read is *not* exactly-once — what does the trait + promise?) +- Does the trait absorb the enqueue+notify-in-one-transaction shape + (honker's core) — and how does that compose with Postgres + transaction-scoped LISTEN semantics? +- How does a caching subscriber (the ecosystem's hot-path pattern) + receive sufficient invalidation information (keys? table/channel + names? opaque wake + re-read contract?) + +Open; this is the second central research question, coupled to OQ-ST-03 +(the driver determines what LISTEN plumbing exists). + +### OQ-ST-05: Queue semantics — adopt, fork, or re-derive? + +If queues land in scope (OQ-ST-01), the pg-boss schema family is the +Postgres-side incumbent and honker's queue design is the SQLite-side +one. Options: adopt pgboss-rs as a dependency (new feature-gated +option); targeted-fork the relevant subsystem (alksocks precedent, +ported to our conventions); schema/design-reference only (re-derive on +our driver). Fork-vs-derive depends on how much of pgboss-rs is +queue-machinery vs driver-wiring (the sqlx coupling — OQ-ST-03) and on +our tolerance for the alpha-state rc port. + +Open; inputs: the OQ-ST-01 inventory + OQ-ST-03 resolution. + +### OQ-ST-06: Honker relationship — reference, fork, or vendor? + +Design-reference only (read, don't copy), targeted fork of +honker-core's engine machinery, or vendor the extension? Honker is +alpha-quality per its own README, MIT/Apache-2.0 dual-licensed, and +covers only the SQLite side — but it embodies exactly the watcher/ +transactional design this crate wants on SQLite. Fork-postures in this +workspace have precedent (alksocks' extraction) but have been for +*owning* a needed subset, not for adopting an alpha wholesale. + +Open; needs the license/provenance check (AGENTS.md §3) and a quality +assessment honker's watcher/transactional core. + +### OQ-ST-07: SQLite-side scope — loadable extension, embedded rusqlite, or both? + +Honker ships as a loadable extension usable by *any* SQLite client, +plus per-language bindings. This crate (a Rust library) may not need +the loadable-extension surface at all — embedding the engine machinery +in-process may be the whole story (the honker-core shape minus the +extension/binding packaging). Determines how much of honker is even +candidate material. + +Open; rides the first consumer-driven scope pass (OQ-ST-01). + +### OQ-ST-08: Multi-host / deployment posture + +Honker is explicitly single-machine (file-backed SQLite). Postgres is +natively multi-host. The unified surface must not pretend SQLite is +multi-host, but where does the honest boundary live — per-engine +capability flags? A documented deployment matrix? Does the trait need +to expose engine capabilities at all? + +Open; partially rides OQ-ST-04 (the trait's shape constrains where +capability differences can surface). + +## Phase 0 plan + +Iteration expected; this register grows as the consumer inventory and +research rounds land. Expected sequence (deliberately rough): + +1. Consumer-driven scope inventory (OQ-ST-01) — what the paused + alkblobs consumers and alkfs planning actually need from the reactive + store; walks back the per-feature scope decisions. +2. Research rounds on OQ-ST-03/04 (drivers + reactive shape) — library + capability matrices, then a POC if the unified-trait shape needs + validation (likely, given the structural mismatch noted in OQ-ST-04). +3. Ownership decisions (OQ-ST-05/06) — adopt/fork/derive per subsystem, + after the driver and shape questions narrow the option space. +4. Converge; Phase 1 opens with the ADR backlog this register becomes. + +## References + +- honker — `/workspace/honker` (git checkout; README + `honker-core/src/` + read 2026-10-03): the SQLite-side feature/wake template. +- pgboss-rs — `/workspace/pgboss-rs` (git checkout of + github.com/rustworthy/pgboss-rs, v0.1.0-rc6; read 2026-10-03, + LISTEN/NOTIFY-absence verified): the Postgres queue-family reference. +- honker's own prior-art section: pg_notify, pg-boss, Oban, Huey — the + external lineage this crate inherits from both sides. +- alkblobs — `/workspace/@alkdev/alkblobs` (spec+POCs, paused): the + paused planning this crate unblocks; its POC findings + (`poc-postgres-kv-findings.md`, `poc-pglo-findings.md`) are the + tokio-postgres evidence base. +- alkcall — `/workspace/@alkdev/alkcall`: the family substrate; + referenced for the store-layer-isolation principle only. \ No newline at end of file diff --git a/docs/sdd_process.md b/docs/sdd_process.md index 0d477a9..ec6fc20 100644 --- a/docs/sdd_process.md +++ b/docs/sdd_process.md @@ -2,7 +2,7 @@ ## Overview -This document defines the SDD process for the @alkdev/alkblobs package. It +This document defines the SDD process for the @alkdev/alkstore package. It leverages: - **OpenCode CLI** as the agent execution environment