glm-5.1
6aa2fcc6ff
Reorient @alkdev/storage around a single SQLite database host with Honker
for pub/sub, event streams, and task queues. PostgreSQL is removed as a
target (ADR-038), eliminating dual schema maintenance and infrastructure
complexity. Honker provides DB + pubsub + queues in one .db file (ADR-039).
Add system/tenant DB model (ADR-040): identity tables in system.db, all
graph data in tenant-{orgId}.db files. Identity tables move from the hub
into storage (ADR-041). Scoping columns (ownerId, projectId) added to
graphs table (ADR-042). Graph types get scope (system/tenant/user) to
protect infrastructure schemas (ADR-043).
Define Drizzle-Honker session adapter (ADR-044): ~100-line adapter enabling
Drizzle typed queries and Honker pubsub/queue on a single connection with
transactional consistency.
Resolve OQ-03, OQ-04, OQ-19, OQ-21, OQ-22, OQ-23, OQ-24. Add new
open questions OQ-26 through OQ-29 for Honker integration specifics.
New docs: honker-integration.md (adapter, event patterns, migration).
Scrub all PG/jsonb/libsql references from existing spec docs.
57 lines
3.8 KiB
Markdown
57 lines
3.8 KiB
Markdown
# ADR-039: Honker as SQLite Extension and Transport
|
|
|
|
## Status
|
|
|
|
Accepted
|
|
|
|
## Context
|
|
|
|
The hub architecture was designed around three separate infrastructure components:
|
|
|
|
1. **PostgreSQL** — persistence (tables, queries, transactions)
|
|
2. **Redis** — pub/sub transport for event-driven communication
|
|
3. **Application-level task queues** — background job processing
|
|
|
|
This creates operational complexity (three services to deploy, monitor, and secure) and a dual-write problem: writing data to PostgreSQL and publishing events to Redis cannot happen in a single transaction. If the process crashes between the DB commit and the Redis publish, data and events become inconsistent.
|
|
|
|
Honker (`@russellthehippo/honker-node`) is a SQLite extension that adds Postgres-style `NOTIFY`/`LISTEN` semantics, durable event streams with per-consumer offsets, at-least-once work queues with retries and dead-letter handling, cron scheduling, advisory locks, and rate limiting — all within the same SQLite `.db` file.
|
|
|
|
## Decision
|
|
|
|
`@alkdev/storage` uses Honker as its SQLite extension and transport layer. Honker provides:
|
|
|
|
1. **Database operations** — SQLite with WAL mode, a bounded reader pool, and a single writer slot
|
|
2. **Ephemeral pub/sub** — `notify()`/`listen()` for fire-and-forget notifications within the DB transaction
|
|
3. **Durable event streams** — `stream()` with per-consumer offset tracking for replay-safe delivery
|
|
4. **Task queues** — `queue()` with at-least-once claims, retries, priority, delayed jobs, and dead-letter
|
|
5. **Advisory locks** — `tryLock()` for leader election and exclusive access
|
|
6. **Cron scheduling** — `scheduler()` for time-triggered operations
|
|
|
|
Drizzle ORM integrates with Honker via a thin session adapter (~100 lines) that wraps Honker's `query()`/`execute()` API inside Drizzle's `SQLiteSession<'sync'>` contract. No Drizzle fork required. The adapter exposes the Honker `Database` as `$client` on the Drizzle instance for direct access to pubsub/queue features.
|
|
|
|
## Consequences
|
|
|
|
**Positive:**
|
|
|
|
- **Transactional consistency**: `INSERT INTO nodes` and `queue.enqueue()` commit atomically. No dual-write problem.
|
|
- **No Redis dependency**: Honker's `stream()` replaces Redis as the durable pub/sub transport
|
|
- **No PostgreSQL dependency**: SQLite with Honker covers persistence + events + queues
|
|
- **Operational simplicity**: One `.db` file contains everything — data, events, queues, schedules
|
|
- **Drizzle integration**: Full Drizzle type safety for queries + Honker for pubsub/queue on the same connection
|
|
- **Ecosystem fit**: The @alkdev platform is event-driven. Honker's durable streams with per-consumer offsets map directly to `@alkdev/operations`' call protocol events and `@alkdev/flowgraph`'s event-sourced model
|
|
|
|
**Negative:**
|
|
|
|
- **Single-machine**: Honker is a single-process SQLite extension. No cross-server events. For multi-node deployment, a separate transport (Redis, NATS) would still be needed for internode communication.
|
|
- **`lastInsertRowid` overhead**: Honker's `execute()` returns only affected row count. Getting `lastInsertRowid` requires an extra `SELECT last_insert_rowid()` call via napi, or a small Rust addition to honker-node's Transaction class.
|
|
- **No prepared statement handles at JS level**: Every Drizzle query goes through `query(sql, params)`. Mitigated by honker-core's `prepare_cached` on the Rust side.
|
|
- **Honker is alpha software**: Not yet beta-quality. API may change. Risk mitigated by the thin adapter — if Honker's query API changes, only the adapter needs updating.
|
|
|
|
## References
|
|
|
|
- Honker source: `/workspace/honker/`
|
|
- Honker Node binding: `/workspace/honker/packages/honker-node/`
|
|
- ADR-038: SQLite-first, Postgres removed
|
|
- ADR-004: Injectable clients, no side effects (Honker client is injectable)
|
|
- `@alkdev/operations` call protocol architecture
|
|
- `@alkdev/pubsub` — Honker may replace or supplement the Redis transport |