phase 0 setup: agent defs cleaned, AGENTS.md, initial phase-0.md draft
- sdd_process.md + coordinator.md: stale @alkdev/alkblobs name from the copy fixed to @alkdev/alkstore - implementation-specialist.md: copied alkcall-family conventions (OperationEnv, vendored core types, BAST/wire formats) replaced with crate-neutral rules + an ADR-escalation rule - code-reviewer.md: stale tls/iroh/acme feature list removed; anyhow posture corrected; db-specific review checks added - AGENTS.md: phase-0 posture (no crate yet, no README, POC/discipline conventions, OQ-ST-NN register, reference checkouts) - docs/research/phase-0.md: initial draft — vision, prior art (honker, pgboss-rs read incl. verified pgboss-rs LISTEN/NOTIFY absence), open questions OQ-ST-01..08, phase 0 plan
This commit is contained in:
1 parent
5bcd1b7a2f
commit
f6531b5532
6 files changed
+504
-56
No files matched your search
@@ -111,13 +111,15 @@ cargo fmt --check # Format check
|
||||
For this project, also verify:
|
||||
|
||||
- No comments in code (per project convention)
|
||||
- Error handling uses `anyhow::Result` (application) / `thiserror` (library) — no
|
||||
- Error handling uses `thiserror` (library crate) — no
|
||||
panics in library code
|
||||
- Feature flags are used correctly (`tls`, `iroh`, `acme`) — base crate compiles
|
||||
lean
|
||||
- Feature flags are used correctly — base crate compiles lean
|
||||
- Public API is well-documented with `///` doc comments where appropriate
|
||||
- Module structure follows Rust conventions (`mod.rs`, `lib.rs`)
|
||||
- No unnecessary `unwrap()` or `expect()` in library code
|
||||
- For database code: SQL injection safety (parameterized queries only, no
|
||||
string-built SQL), connection/pool handling under cancellation (no
|
||||
panics poisoning a shared pool), and migration/DDL ordering
|
||||
|
||||
#### E. Security
|
||||
|
||||
|
||||
@@ -191,7 +191,7 @@ also include:
|
||||
Example prompt template:
|
||||
|
||||
```
|
||||
You are an implementation specialist for the @alkdev/alkblobs project.
|
||||
You are an implementation specialist for the @alkdev/alkstore project.
|
||||
|
||||
Your task: {{task}}
|
||||
|
||||
@@ -204,7 +204,7 @@ Your task: {{task}}
|
||||
7. Push: git push origin $(git branch --show-current)
|
||||
8. Notify: worktree({action: "notify", args: {message: "Task completed: {{task}}. <brief summary>", level: "info"}})
|
||||
|
||||
Key project constraints (@alkdev/alkblobs):
|
||||
Key project constraints (@alkdev/alkstore):
|
||||
- Rust: use cargo build, cargo clippy, cargo fmt, cargo test
|
||||
- No comments in code
|
||||
- thiserror for error types
|
||||
|
||||
@@ -207,7 +207,9 @@ This is especially important for complex tasks that span many file operations.
|
||||
|
||||
## Project Conventions
|
||||
|
||||
Read `AGENTS.md` at project root for full details. Key rules:
|
||||
Read `AGENTS.md` at project root for full details. Key rules (the
|
||||
cross-crate, stable ones — crate-specific conventions live in `AGENTS.md`
|
||||
and evolve with the project):
|
||||
|
||||
1. **No comments in code** — Per project convention. Doc comments (`///`, `//!`)
|
||||
are fine and expected on public API.
|
||||
@@ -217,55 +219,19 @@ Read `AGENTS.md` at project root for full details. Key rules:
|
||||
operation does not cascade to other operations.
|
||||
3. **`tokio` is the async runtime** — all I/O is async. Use `tokio::sync`
|
||||
primitives (`oneshot`, `mpsc`) for request correlation and subscription
|
||||
channels; `parking_lot` for short-held internal locks (`PendingRequestMap`).
|
||||
4. **No secret material on the wire** — `call.requested`/`call.responded`
|
||||
payloads and `OperationContext.metadata` carry no private keys, API keys, or
|
||||
decrypted credentials. Outbound credentials flow through `Capabilities`
|
||||
injected at the assembly layer → `HandlerRegistration.capabilities` →
|
||||
`OperationContext.capabilities` → handler.
|
||||
5. **No-env-vars invariant** — no handler reads outbound credentials from any
|
||||
source other than `OperationContext.capabilities`. This is a spec-level
|
||||
invariant, not a runtime convention.
|
||||
6. **`OperationEnv` must remain a trait** — the trait-based design enables
|
||||
registry layering (session overlays, connection overlays, peer-keyed
|
||||
composition). Do not make it concrete or hardcode the global registry.
|
||||
7. **Wire formats are stable** — `EventEnvelope` (`{ type, id, payload }` +
|
||||
length-prefixed JSON framing) and the channels 8-byte chunk header
|
||||
(`[channel_id:u32 BE][length:u32 BE][payload]`) are one-way doors. New event
|
||||
types may be added; existing shapes must not change.
|
||||
8. **Producer/consumer, not server/client** — both sides of a call or channels
|
||||
connection can initiate. Use "producer"/"consumer" or "accept side"/"connect
|
||||
side," not "server"/"client."
|
||||
9. **Vendored core types** — `Connection`, `ProtocolHandler`, `BiStream`,
|
||||
`BidiStreamSource`, `AuthContext`, `IdentityProvider`, `Identity`,
|
||||
`AuthToken`, `Capabilities`, `OwnershipProvider`, `HandlerError`,
|
||||
`StreamError` live in this crate. Do not add a separate `alkcore` dependency.
|
||||
Keep them lean (no TLS, no transport coupling, no endpoint/accept-loop).
|
||||
10. **BAST documents for wire formats** — every binary wire format carries a
|
||||
BAST (Binary Abstract Syntax Tree) document as its machine-readable spec
|
||||
(e.g. the channels chunk header's `docs/architecture/chunk-header.bast.json`,
|
||||
embedded as `CHUNK_HEADER_BAST`). BAST is plain JSON — no dependency
|
||||
required to author or consume it. The `alktype` crate compiles BAST into
|
||||
readers/writers/validators; future codegen derives language-specific
|
||||
implementations. Trivial or hot-path formats (chunk header, tty framing)
|
||||
stay hand-rolled with the BAST doc as the contract; complex formats (sftp)
|
||||
use the alktype engine or codegen. Do not roll your own offset map or
|
||||
validator for complex formats.
|
||||
11. **Feature flags** — transports may be feature-gated if the need arises. The
|
||||
base crate should compile lean (no `quinn`, no `iroh` unless the feature is
|
||||
on). Verify both `cargo test` (default) and `cargo test --all-features` pass
|
||||
if features are added.
|
||||
12. **Abort cascades to descendants** — `call.aborted` for a parent cascades to
|
||||
all non-terminal descendants. Default `abort-dependents`;
|
||||
`continue-running` opt-in. The composing handler decides the child's policy,
|
||||
not the wire caller.
|
||||
13. **Peer authorization via `AccessControl`** — a remote peer's call is
|
||||
authorized by `AccessControl::check(peer_identity)`. No `remote_safe` flag,
|
||||
no `trusted_peer` bypass. `Visibility::Internal` ops are never wire-callable.
|
||||
14. **Naming conventions** — Rust standard: `snake_case` for functions/variables/
|
||||
modules, `PascalCase` for types/traits, `SCREAMING_SNAKE_CASE` for constants.
|
||||
15. **Module structure** — one module per file under `src/`, re-exported from
|
||||
`src/lib.rs`. Public API surface is `lib.rs` re-exports.
|
||||
channels; `parking_lot` for short-held internal locks.
|
||||
4. **Naming conventions** — Rust standard: `snake_case` for functions/variables/
|
||||
modules, `PascalCase` for types/traits, `SCREAMING_SNAKE_CASE` for constants.
|
||||
5. **Module structure** — one module per file under `src/`, re-exported from
|
||||
`src/lib.rs`. Public API surface is `lib.rs` re-exports.
|
||||
6. **Feature flags** — optional dependencies (e.g. the postgres engine) may be
|
||||
feature-gated if the need arises. The base crate should compile lean.
|
||||
Verify both `cargo test` (default) and `cargo test --all-features` pass
|
||||
if features are added.
|
||||
7. **Decisions land in ADRs** — if you discover a decision the architecture
|
||||
didn't make (an undocumented trade-off, an interface shape you must
|
||||
choose), do not decide it silently: notify the coordinator (`level:
|
||||
"blocking"` if it changes the API surface) so an ADR gets written.
|
||||
|
||||
## Key Principles
|
||||
|
||||
|
||||
@@ -0,0 +1,121 @@
|
||||
# AGENTS.md
|
||||
|
||||
Operating instructions for opencode agents working in this repo. opencode
|
||||
auto-loads this file as instructions, overriding the built-in defaults for
|
||||
this project. Custom agents in `.opencode/agents/` inherit these rules
|
||||
unless their own prompts say otherwise.
|
||||
|
||||
## Repo posture
|
||||
|
||||
This repo is in **Phase 0 (Exploration)** per `docs/sdd_process.md`.
|
||||
There is no crate yet — no `Cargo.toml`, no `src/`, no README (the README
|
||||
gets written just before first publish, so it can be honest about what
|
||||
shipped). What exists is `docs/research/phase-0.md` (the Phase 0
|
||||
document: vision, prior art, open questions, POC register), the SDD
|
||||
process (`docs/sdd_process.md`), and the agent defs
|
||||
(`.opencode/agents/`). Do not scaffold implementation code
|
||||
from this posture unless a task or the user asks for a POC.
|
||||
|
||||
Do not treat `docs/research/phase-0.md` as settled architecture — it is
|
||||
a working research document whose open questions are genuinely open.
|
||||
The hedging rules of `docs/sdd_process.md` (no circular deferrals, no
|
||||
"resolved with escape hatches") still apply to it.
|
||||
|
||||
## Git Workflow
|
||||
|
||||
**Commit and push when reasonable.** When a change is complete and
|
||||
verified, commit and push to `origin/main` without asking. This
|
||||
overrides the built-in default of "only commit when explicitly asked."
|
||||
In Phase 0 "verified" means the doc checks below — there is no build
|
||||
gate yet.
|
||||
|
||||
Exceptions — **do not** commit or push without asking:
|
||||
|
||||
- The change is exploratory / speculative (you're not sure the user
|
||||
wants it kept)
|
||||
- The user is actively reviewing the diff and may ask for changes
|
||||
- You'd be force-pushing, amending a pushed commit, creating an empty
|
||||
commit, or skipping hooks
|
||||
|
||||
Never commit secrets, keys, or credentials. If a commit fails or hooks
|
||||
reject it, fix the issue and create a new commit — do not amend the
|
||||
failed one.
|
||||
|
||||
Git identity is preconfigured (`glm-5.3-flash <glm-5.3-flash@alk.dev>`). Do
|
||||
not change `git config`, skip hooks, or use `git commit -i`.
|
||||
|
||||
## Phase 0 conventions
|
||||
|
||||
1. **Findings land in `docs/research/`.** Research notes, POC
|
||||
specifications, and POC findings all go there, named consistently
|
||||
(`poc-<name>-spec.md` / `poc-<name>-findings.md` follows the
|
||||
alkblobs precedent). A POC that needs this repo's code runs in a
|
||||
worktree (`.worktrees/research/<task-id>/` per the SDD process); a
|
||||
self-contained POC (like the alkblobs ones) runs as a standalone
|
||||
crate in the global workspace. Findings land in this repo either
|
||||
way.
|
||||
2. **Open questions get OQ-ST-NN IDs** and live in the register in
|
||||
`docs/research/phase-0.md` until Phase 1 promotes them to
|
||||
`docs/architecture/open-questions.md`. Numbering is stable — never
|
||||
renumber; append.
|
||||
3. **Reference read-only checkouts by path and revision.** The external
|
||||
references for this crate (`/workspace/honker`,
|
||||
`/workspace/pgboss-rs`) are read-only reference checkouts of
|
||||
third-party projects. Note the checkout state when your findings
|
||||
depend on code specifics; licenses and provenance get recorded when
|
||||
we adopt code, not while only reading.
|
||||
4. **POC code stays out of this repo** until Phase 1 adopts it. A POC
|
||||
crate proves the hypothesis and reports findings; it does not
|
||||
pre-scaffold the real crate layout.
|
||||
5. **No mock/main-repo documentation.** No README, no CHANGELOG, no
|
||||
crate-level docs until the shape exists. Docs that describe a thing
|
||||
that doesn't exist are exactly the kind of dishonest artifact we
|
||||
write docs to avoid.
|
||||
6. **No comments in code** (when POC crates get written) unless the
|
||||
user explicitly asks. Doc comments (`///`, `//!`) are fine on public
|
||||
API. Inline `//` comments only when a non-obvious safety/correctness
|
||||
constraint would otherwise be missed, or the user asks.
|
||||
|
||||
## Conventions that will apply once the crate exists
|
||||
|
||||
These are family-standard and pre-committed; they'll be duplicated into
|
||||
task prompts and tightened by Phase 1 ADRs:
|
||||
|
||||
- Rust crate, `tokio` async runtime, `thiserror` error types, no panics
|
||||
in library code, no `unwrap()`/`expect()` outside tests.
|
||||
- No comments in code (see 6 above).
|
||||
- Module-per-file under `src/`, re-exported from `src/lib.rs`; public
|
||||
API surface is the lib re-exports.
|
||||
- Optional dependencies (the postgres engine likely being the first)
|
||||
are feature-gated; the base crate compiles lean; both
|
||||
`cargo test --features <set>` and default-feature builds pass when
|
||||
features exist.
|
||||
- Categorical estimates (`scope`, `risk`, `impact`, `level`) in every
|
||||
task file — per `docs/sdd_process.md` these are structurally
|
||||
required, not optional metadata.
|
||||
|
||||
## Verification Commands
|
||||
|
||||
Phase 0 — none (no code yet). The doc conventions above are checked by
|
||||
reading. When the crate appears, this section gets the standard Rust
|
||||
gates (`cargo test`, `cargo clippy --all-targets -- -D warnings`,
|
||||
`cargo fmt --check`, and the fuzz corpus replay if fuzzing is adopted —
|
||||
the alksocks/alktty/alktunnels pattern) and becomes the merge gate the
|
||||
coordinator runs.
|
||||
|
||||
## Architecture Context
|
||||
|
||||
- `docs/research/phase-0.md` — the Phase 0 document: vision, prior
|
||||
art, open questions (OQ-ST-01..NN), the POC register, and (eventually)
|
||||
the converged recommendation. Read it before any non-trivial work in
|
||||
this repo.
|
||||
- `docs/sdd_process.md` — the SDD process. Phase 0 in progress;
|
||||
`docs/architecture/` does not exist yet.
|
||||
- The SDD process and agent defs were copied from downstream projects
|
||||
(alkblobs) in the workspace; project-specific stale references were
|
||||
cleaned at Phase 0 setup (2026-10-03). If you find a section that
|
||||
assumes another crate's machinery, fix it rather than working around
|
||||
it — same discipline as stale TODOs.
|
||||
- If a TODO or doc reference cites a design direction that a later ADR
|
||||
or Phase 0 decision rejected, the reference is stale — remove it and
|
||||
align; do not implement the rejected design.
|
||||
@@ -0,0 +1,359 @@
|
||||
---
|
||||
status: draft
|
||||
last_updated: 2026-10-03 (initial draft from the setup discussion)
|
||||
---
|
||||
|
||||
# alkstore — Phase 0 (Exploration)
|
||||
|
||||
This document captures Phase 0 (Exploration) for the `alkstore` crate:
|
||||
vision, guiding principles, prior art, and the open-question register
|
||||
(OQ-ST-01..NN). Phase 0's objective per `docs/sdd_process.md`: *capture
|
||||
vision and guiding principles; research options; validate approaches;
|
||||
converge on a recommended approach.* This is the initial draft — none of
|
||||
the questions below are settled, and there is no POC register yet.
|
||||
|
||||
Context for why this crate starts now: **alkblobs**
|
||||
(`/workspace/@alkdev/alkblobs` — spec + POCs only, paused mid-planning)
|
||||
hit repeated circular hedging in its Phase 0, and a root cause was that
|
||||
the storage substrate it deploys onto was itself disjoint and fuzzy — a
|
||||
repo pattern with a default in-memory adapter across the alk* ecosystem,
|
||||
cache-invalidation patches in hot paths, non-invalidated caches where
|
||||
delay was tolerable, no single definition of "how does a change in the
|
||||
database become visible to other processes/connections?" alkblobs paused
|
||||
partly to let this crate answer that first. alkstore is the attempt to
|
||||
make that substrate real once, so downstream stores don't re-derive it.
|
||||
|
||||
## Vision and guiding principles
|
||||
|
||||
**One sentence (draft):** one reactive store interface over SQLite and
|
||||
Postgres — durable pub/sub notify, queues, streams, and the transactional
|
||||
integration (write + enqueue in one transaction) that honker delivers on
|
||||
SQLite — with the Postgres side building on the natively-available
|
||||
machinery (`pg_notify`/`LISTEN`, and the pgboss job-queue schema family)
|
||||
rather than emulating it.
|
||||
|
||||
**The honker relationship.** `/workspace/honker` (read-only reference;
|
||||
alpha-quality per its own README, MIT/Apache-2.0 dual) adds
|
||||
Postgres-style `NOTIFY`/`LISTEN` semantics to SQLite without a broker:
|
||||
durable at-least-once queues with retries/delay/priority/visibility
|
||||
timeouts/dead-letter, durable streams with per-consumer offsets,
|
||||
cron/`@every` scheduling, named locks, rate limits, transactional
|
||||
outbox — all as INSERTs inside the caller's transaction, with the
|
||||
cross-process wake delivered by a shared watcher that polls `PRAGMA
|
||||
data_version` (default 1 ms → single-digit-ms delivery) and re-reads
|
||||
indexed state after every wake. Its own Prior Art section names the
|
||||
Postgres side: `pg_notify` gives fast triggers with no retry or
|
||||
visibility semantics; pg-boss/Oban are the durable-layer gold standards
|
||||
— "If you already run Postgres, use the Postgres tools."
|
||||
|
||||
The goal is **not** "port honker to Postgres." The goal is the interface:
|
||||
one store API whose consumer code (queues, streams, notify) looks the
|
||||
same whether the backing engine is SQLite or Postgres, while each engine
|
||||
uses its own native wake/delivery story under the hood. The honker docs
|
||||
recommend pgboss + `pg_notify` for the Postgres equivalent — that
|
||||
recommendation is the design brief for this crate's Postgres engine.
|
||||
|
||||
**Consumer shape (from the paused alkblobs planning):** any crate that
|
||||
today uses the ecosystem's repo-pattern + in-memory adapter should be
|
||||
able to swap in an alkstore-backed engine and get durability + true
|
||||
cross-process/cross-instance reactivity. That means the reactive surface
|
||||
must compose with *client-side* caching: a subscriber that also holds a
|
||||
cache can invalidate on notification instead of re-querying or re-polling
|
||||
— the hot-path pattern the ecosystem already uses, given a real
|
||||
invalidation source.
|
||||
|
||||
Guiding principles:
|
||||
|
||||
1. **One interface, two engines, native underneath.** The abstraction
|
||||
layer unifies the consumer-visible features; the engines stay
|
||||
dialects, not two emulations of one dialect. SQLite follows honker's
|
||||
design (queue in the same file, same transaction, watcher-based
|
||||
wake); Postgres follows pg-boss' design (schema-based job tables +
|
||||
`pg_notify`-driven wake). A "lowest common denominator" unification
|
||||
(both sides polling, both sides emulating LISTEN) is explicitly the
|
||||
failure mode to avoid — it would re-create the fuzziness this crate
|
||||
exists to remove.
|
||||
2. **Transactional local-adjacency is the load-bearing property.**
|
||||
Honker's core claim: enqueue/publish/notify in the same transaction
|
||||
as the business write; rollback drops both. The unified surface must
|
||||
preserve this on any engine, because the ecosystem's repo pattern
|
||||
assumes it (a business write that loses its side-effect notification
|
||||
is the dual-write problem honker names).
|
||||
3. **Ownership of the whole stack.** honker is third-party; pgboss-rs
|
||||
is third-party. Whether any of them are adopted, forked, or used as
|
||||
schema/design reference only is a deliberate per-question decision —
|
||||
not inherited by adjacency. The workspace precedent is the targeted
|
||||
fork (alksocks' fast-socks5 extraction: adopt the design, own the
|
||||
code, port to our conventions).
|
||||
4. **Substrate-agnostic consumer API, engine-specific setup.** A
|
||||
consumer opens a `Store` from a connection string / file path and
|
||||
gets the same trait surface. Which engine is behind what can vary
|
||||
(per-deployment config), but consumer code must not branch on
|
||||
engine type.
|
||||
5. **No panics, `tokio`, `thiserror`, lean base crate, feature-gated
|
||||
optional engines** — family-standard, pre-committed (AGENTS.md).
|
||||
|
||||
## The driver conflict (Phase 0's central tension)
|
||||
|
||||
The immediate design fork, flagged by the user:
|
||||
|
||||
- **The alkblobs POCs used `tokio-postgres` + `deadpool-postgres`**
|
||||
(validated in `poc-postgres-kv-findings.md`, `poc-pglo-findings.md` —
|
||||
including Large Objects work).
|
||||
- **pgboss-rs (`/workspace/pgboss-rs`) uses `sqlx`** (`sqlx Postgres
|
||||
runtime-tokio`).
|
||||
- honker-core uses `rusqlite`.
|
||||
|
||||
A single crate with both engines means a driver decision, and the
|
||||
reactivity story is entangled with it:
|
||||
|
||||
- **pgboss-rs currently has no `LISTEN`/`NOTIFY` at all** (verified
|
||||
2026-10-03 against the checkout — `src/` contains no LISTEN/NOTIFY
|
||||
usage; consumption is `fetch_job` polling). The node original relies
|
||||
on `pg-boss`'s own maintenance/polling; the port did not pick up a
|
||||
push channel. So *even* "use pgboss for the queue" does not deliver
|
||||
reactivity — LISTEN/NOTIFY wiring would be new work either way, and
|
||||
the driver choice determines *whose* LISTEN plumbing (sqlx's
|
||||
`PgListener` is built-in; tokio-postgres uses its `Connection`
|
||||
notifications).
|
||||
- **Honker's reactivity on SQLite is a watcher polling `PRAGMA
|
||||
data_version`** — a fundamentally different mechanism from
|
||||
LISTEN/NOTIFY. The unified reactive trait must abstract over both
|
||||
without collapsing to the polling behavior of the weaker side.
|
||||
|
||||
This tension is OQ-ST-04 below. It is *not* resolved by "pgboss is well
|
||||
written so start there" — that is exactly the inherited-assumption
|
||||
shape the SDD process flags. What pgboss-rs genuinely offers (schema
|
||||
DDL, job states, retry semantics, the node-compatible API) is design
|
||||
reference regardless of driver.
|
||||
|
||||
## Prior art
|
||||
|
||||
Notes below are from reading the checkouts on 2026-10-03; both external
|
||||
projects are read-only references. Provenance/licensing gets recorded
|
||||
per AGENTS.md §3 when code is adopted, not while only reading.
|
||||
|
||||
### honker — the SQLite-side template
|
||||
|
||||
`/workspace/honker` (SQLite extension + bindings). What matters for
|
||||
this crate:
|
||||
|
||||
- **The full feature set to match on Postgres** (its §What It Does):
|
||||
notify/listen across processes, durable at-least-once queues
|
||||
(retries, delayed jobs, priority, visibility timeouts, dead-letter
|
||||
rows, result storage), durable streams with per-consumer offsets,
|
||||
cron/`@every` scheduling, named locks, rate limits, transactional
|
||||
outbox helpers. Deliberately excluded there: workflow DAGs, task
|
||||
chains/chords, multi-writer replication, cross-machine locking —
|
||||
scope line likely inherited, to be confirmed.
|
||||
- **The wake mechanism** — `PRAGMA data_version` polling watcher
|
||||
(default 1 ms; raise for idle CPU), re-read indexed state after
|
||||
wake, overtriggering on purpose ("one indexed SELECT is cheap; a
|
||||
missed wake is a correctness bug."). Optional kernel-events and WAL
|
||||
shared-memory backends exist in source builds.
|
||||
- **Single-machine honesty** — file-backed, one host; NFS-two-writers
|
||||
explicitly not supported. This posture needs an explicit Postgres
|
||||
counterpart (multi-host is Postgres' normal case, so the interface
|
||||
must not bake SQLite's single-host assumption into the shared
|
||||
surface).
|
||||
- **The transactional enqueue shape** — every feature is an INSERT
|
||||
inside the caller's transaction. This is the pattern the unified API
|
||||
must keep visible and cheap.
|
||||
|
||||
### pgboss-rs — the Postgres queue family reference
|
||||
|
||||
`/workspace/pgboss-rs` (v0.1.0-rc6, MIT/Apache-2.0 dual). Ported from
|
||||
node pg-boss: builder-based queue/job API, retry/delay/priority/
|
||||
singleton/dead-letter concepts, `sqlx` 0.8, schema-scoped DDL.
|
||||
|
||||
- Verified gap (2026-10-03): **no LISTEN/NOTIFY** anywhere in `src/`
|
||||
— consumption is polling `fetch_job`. Any push-reactivity is new
|
||||
work, not an adoption freebie.
|
||||
- Its value as reference: the pg-boss schema family (job states,
|
||||
maintenance/dead-letter behavior) is battle-tested against real
|
||||
Postgres semantics — worth borrowing *as design*, independent of the
|
||||
driver decision.
|
||||
- The node original (pg-boss) is the upstream of record for semantics
|
||||
the port may have dropped; compare against it when adopting queue
|
||||
semantics.
|
||||
|
||||
### Honker's Postgres-side recommendation
|
||||
|
||||
The honker README's own posture: if you run Postgres, use the Postgres
|
||||
tools. `pg_notify` + pgboss is the recommended assembly. The design
|
||||
brief: the queue machinery from the pg-boss family, the push semantics
|
||||
from LISTEN/NOTIFY, the unified API shape from honker's Rust binding.
|
||||
|
||||
### The alk* repo pattern — what this crate replaces
|
||||
|
||||
The ecosystem's current shape: a repository trait with a default
|
||||
in-memory adapter; cache-invalidation wiring in hot paths; uninvalidated
|
||||
(non-reactive) caches where delay was acceptable; each project
|
||||
composing these slightly differently. No persistence-backed reactive
|
||||
substrate exists in the family — alkblobs was the first project to try
|
||||
to plan against one, found it missing, and paused. This crate's reason
|
||||
to exist is precisely that that substrate should exist once, well,
|
||||
instead of per-project approximations.
|
||||
|
||||
### alkcall — the substrate (not a dependency of the store layer)
|
||||
|
||||
`/workspace/@alkdev/alkcall` (pure protocol crate, no transport). The
|
||||
alk* crates (alktty, alktunnels, alksocks) are its consumers; a future
|
||||
alkstore ops/protocol surface (if this crate ever exposes store access
|
||||
over alkcall channels) rides the same substrate. Like the alkblobs
|
||||
split (store layer stays substrate-free), the store layer here stays
|
||||
alkcall-free; any networked surface is an ops module/sibling concern
|
||||
and a separate decision.
|
||||
|
||||
## Open Questions
|
||||
|
||||
Register in `docs/research/phase-0.md`; IDs OQ-ST-NN (stable, append
|
||||
only). Promotion target: Phase 1 `docs/architecture/open-questions.md`.
|
||||
|
||||
### OQ-ST-01: Scope boundary — which honker features are in-scope?
|
||||
|
||||
Honker's feature list (queues, streams, notify, scheduling, locks, rate
|
||||
limits, outbox helpers) is large; per-feature scope decisions don't
|
||||
exist yet. Also undetermined: the exclusion lines (honker deliberately
|
||||
excludes DAGs, task chains/chords, multi-writer replication,
|
||||
distributed locking).
|
||||
|
||||
Not yet decidable without working through the concrete consumers (the
|
||||
paused alkblobs crates and alkfs planning) — per-feature need hasn't
|
||||
been articulated. Blocked on a consumer-driven inventory pass.
|
||||
|
||||
### OQ-ST-02: Crate scope — one store crate, or reactive-core + engines?
|
||||
|
||||
Options include: single crate with feature-gated engines (the alk*
|
||||
feature-gate pattern); a core trait crate + per-engine crates; engine
|
||||
crates consuming a thin core. The answer constrains the driver decision
|
||||
(OQ-ST-04) and the base-crate-lean invariant.
|
||||
|
||||
The use case isn't concrete yet (no engine code written, no consumer
|
||||
wired). Deferred(scope) — concrete use case: the first engine
|
||||
implementation would force this shape.
|
||||
|
||||
### OQ-ST-03: Driver story — sqlx, tokio-postgres, or per-engine drivers?
|
||||
|
||||
The named tension (§The driver conflict), restated as the decision:
|
||||
|
||||
- **One driver across engines**: sqlx (both sqlite + postgres native
|
||||
support, one API — but the alkblobs POC evidence is tokio-postgres)
|
||||
vs tokio-postgres per-engine (sqlite story unclear — tokio-postgres
|
||||
is pg-only; rusqlite is the sqlite native).
|
||||
- **Per-engine drivers under a unified trait**: tokio-postgres +
|
||||
deadpool-postgres (POC-validated in alkblobs findings) + rusqlite
|
||||
(honker's choice, so honker's SQLite machinery ports cleanly).
|
||||
- **Adopt/fork pgboss-rs**: brings sqlx along where the queue lives.
|
||||
|
||||
Honest unknowns worth surfacing: does sqlx support SQLite
|
||||
`data_version`/extension-style machinery equally well? Does a unified
|
||||
trait over `(tokio-postgres, rusqlite)` pay more trait-fitting cost
|
||||
than sqlx's single-API convenience costs elsewhere? What does
|
||||
SQLITE_ENABLE/extension loading look like under sqlx vs rusqlite?
|
||||
|
||||
Genuinely open; needs research rounds (library capabilities vs the
|
||||
unified-trait shape) and possibly a POC. Not deferred — this is the
|
||||
central Phase 0 research question.
|
||||
|
||||
### OQ-ST-04: The reactive abstraction — what does the unified notify surface look like?
|
||||
|
||||
The two engines' mechanisms are structurally different: SQLite =
|
||||
watcher polling `PRAGMA data_version` (deliver on commit; no
|
||||
server-side push exists), Postgres = LISTEN/NOTIFY (server push,
|
||||
connection-bound, no retry/visibility semantics). The reactive trait
|
||||
must have a shape both implement without one emulating the other's
|
||||
weaknesses:
|
||||
|
||||
- What is the subscription type (`channel`? stream of envelopes?)
|
||||
- What is the delivery guarantee contract on each engine (honker's
|
||||
wake-on-commit + re-read is *not* exactly-once — what does the trait
|
||||
promise?)
|
||||
- Does the trait absorb the enqueue+notify-in-one-transaction shape
|
||||
(honker's core) — and how does that compose with Postgres
|
||||
transaction-scoped LISTEN semantics?
|
||||
- How does a caching subscriber (the ecosystem's hot-path pattern)
|
||||
receive sufficient invalidation information (keys? table/channel
|
||||
names? opaque wake + re-read contract?)
|
||||
|
||||
Open; this is the second central research question, coupled to OQ-ST-03
|
||||
(the driver determines what LISTEN plumbing exists).
|
||||
|
||||
### OQ-ST-05: Queue semantics — adopt, fork, or re-derive?
|
||||
|
||||
If queues land in scope (OQ-ST-01), the pg-boss schema family is the
|
||||
Postgres-side incumbent and honker's queue design is the SQLite-side
|
||||
one. Options: adopt pgboss-rs as a dependency (new feature-gated
|
||||
option); targeted-fork the relevant subsystem (alksocks precedent,
|
||||
ported to our conventions); schema/design-reference only (re-derive on
|
||||
our driver). Fork-vs-derive depends on how much of pgboss-rs is
|
||||
queue-machinery vs driver-wiring (the sqlx coupling — OQ-ST-03) and on
|
||||
our tolerance for the alpha-state rc port.
|
||||
|
||||
Open; inputs: the OQ-ST-01 inventory + OQ-ST-03 resolution.
|
||||
|
||||
### OQ-ST-06: Honker relationship — reference, fork, or vendor?
|
||||
|
||||
Design-reference only (read, don't copy), targeted fork of
|
||||
honker-core's engine machinery, or vendor the extension? Honker is
|
||||
alpha-quality per its own README, MIT/Apache-2.0 dual-licensed, and
|
||||
covers only the SQLite side — but it embodies exactly the watcher/
|
||||
transactional design this crate wants on SQLite. Fork-postures in this
|
||||
workspace have precedent (alksocks' extraction) but have been for
|
||||
*owning* a needed subset, not for adopting an alpha wholesale.
|
||||
|
||||
Open; needs the license/provenance check (AGENTS.md §3) and a quality
|
||||
assessment honker's watcher/transactional core.
|
||||
|
||||
### OQ-ST-07: SQLite-side scope — loadable extension, embedded rusqlite, or both?
|
||||
|
||||
Honker ships as a loadable extension usable by *any* SQLite client,
|
||||
plus per-language bindings. This crate (a Rust library) may not need
|
||||
the loadable-extension surface at all — embedding the engine machinery
|
||||
in-process may be the whole story (the honker-core shape minus the
|
||||
extension/binding packaging). Determines how much of honker is even
|
||||
candidate material.
|
||||
|
||||
Open; rides the first consumer-driven scope pass (OQ-ST-01).
|
||||
|
||||
### OQ-ST-08: Multi-host / deployment posture
|
||||
|
||||
Honker is explicitly single-machine (file-backed SQLite). Postgres is
|
||||
natively multi-host. The unified surface must not pretend SQLite is
|
||||
multi-host, but where does the honest boundary live — per-engine
|
||||
capability flags? A documented deployment matrix? Does the trait need
|
||||
to expose engine capabilities at all?
|
||||
|
||||
Open; partially rides OQ-ST-04 (the trait's shape constrains where
|
||||
capability differences can surface).
|
||||
|
||||
## Phase 0 plan
|
||||
|
||||
Iteration expected; this register grows as the consumer inventory and
|
||||
research rounds land. Expected sequence (deliberately rough):
|
||||
|
||||
1. Consumer-driven scope inventory (OQ-ST-01) — what the paused
|
||||
alkblobs consumers and alkfs planning actually need from the reactive
|
||||
store; walks back the per-feature scope decisions.
|
||||
2. Research rounds on OQ-ST-03/04 (drivers + reactive shape) — library
|
||||
capability matrices, then a POC if the unified-trait shape needs
|
||||
validation (likely, given the structural mismatch noted in OQ-ST-04).
|
||||
3. Ownership decisions (OQ-ST-05/06) — adopt/fork/derive per subsystem,
|
||||
after the driver and shape questions narrow the option space.
|
||||
4. Converge; Phase 1 opens with the ADR backlog this register becomes.
|
||||
|
||||
## References
|
||||
|
||||
- honker — `/workspace/honker` (git checkout; README + `honker-core/src/`
|
||||
read 2026-10-03): the SQLite-side feature/wake template.
|
||||
- pgboss-rs — `/workspace/pgboss-rs` (git checkout of
|
||||
github.com/rustworthy/pgboss-rs, v0.1.0-rc6; read 2026-10-03,
|
||||
LISTEN/NOTIFY-absence verified): the Postgres queue-family reference.
|
||||
- honker's own prior-art section: pg_notify, pg-boss, Oban, Huey — the
|
||||
external lineage this crate inherits from both sides.
|
||||
- alkblobs — `/workspace/@alkdev/alkblobs` (spec+POCs, paused): the
|
||||
paused planning this crate unblocks; its POC findings
|
||||
(`poc-postgres-kv-findings.md`, `poc-pglo-findings.md`) are the
|
||||
tokio-postgres evidence base.
|
||||
- alkcall — `/workspace/@alkdev/alkcall`: the family substrate;
|
||||
referenced for the store-layer-isolation principle only.
|
||||
+1
-1
@@ -2,7 +2,7 @@
|
||||
|
||||
## Overview
|
||||
|
||||
This document defines the SDD process for the @alkdev/alkblobs package. It
|
||||
This document defines the SDD process for the @alkdev/alkstore package. It
|
||||
leverages:
|
||||
|
||||
- **OpenCode CLI** as the agent execution environment
|
||||
|
||||
Reference in new issue
Block a user