Pivot: fold drizzlebox as utils, HonkerEventTarget, OperationSpecs as repo surface

- Update architecture docs to reflect pivot from @libsql/client to Honker
- Fold @alkdev/drizzlebox Phase 0 into src/sqlite/utils/ (ADR-046)
- Add HonkerEventTarget adapter for pubsub TypedEventTarget (ADR-047)
- Replace hand-written CRUD with OperationSpec generation (ADR-048)
- Resolved OQ-26: Honker replaces Redis for single-node pub/sub (POC validated)
- Updated OQ-17, OQ-18, OQ-19 for OperationSpec repository surface
- Added OQ-30 (composite event target), OQ-31 (consumer naming), OQ-32 (Drizzle Kit)
- POC results: adapter buildable, same-process pub/sub works, transactional
  outbox semantics confirmed, concurrent listeners/streams work
- Research doc at docs/research/pivot-honker-sqlite-adapter.md
This commit is contained in:
2026-06-01 16:31:40 +00:00
parent 6aa2fcc6ff
commit 412ad98f11
10 changed files with 1342 additions and 230 deletions

View File

@@ -1,12 +1,14 @@
---
status: draft
last_updated: 2026-05-31
last_updated: 2026-06-01
---
# Honker Integration
How @alkdev/storage integrates with Honker for SQLite database operations,
transactional pub/sub, durable event streams, and task queues.
transactional pub/sub, durable event streams, and task queues. Includes the
HonkerEventTarget adapter that bridges `@alkdev/pubsub`'s `TypedEventTarget`
to Honker primitives.
## Purpose
@@ -118,10 +120,134 @@ db.transaction((tx) => {
| Limitation | Impact | Mitigation |
|------------|--------|------------|
| No `lastInsertRowid` from `execute()` | `run()` needs extra `SELECT last_insert_rowid()` | Small Rust PR to honker-node |
| No `lastInsertRowid` from `execute()` | `run()` needs extra `SELECT last_insert_rowid()` | POC 1 confirmed `tx.query("SELECT last_insert_rowid() as id")` works |
| No prepared statement handle at JS level | Every query re-enters napi | Rust-side `prepare_cached` amortizes |
| Object rows only, no raw arrays | `values()` must convert `Object.values()` | Column order preserved from SQLite |
| Drizzle internal API dependency | Adapter imports from `drizzle-orm/sqlite-core/session` | Stable across minor versions |
| Drizzle internal API dependency | Adapter imports from `drizzle-orm/sqlite-core/session` | POC 1 confirmed all classes accessible and extendable |
| `:memory:` databases don't work | Reader pool gets separate in-memory DB | Always use file-based paths. Tests use temp files. |
### POC-Validated Adapter Architecture
POC 1 (2026-06-01) confirmed the adapter is buildable. Key findings:
- `SQLiteSession`, `SQLitePreparedQuery`, `SQLiteTransaction` are all
accessible from `drizzle-orm/sqlite-core/session` and extendable.
- `BaseSQLiteDatabase` is accessible from `drizzle-orm/sqlite-core/db`.
- `LibSQLSession` in `drizzle-orm/libsql/session` is the reference
implementation to follow.
- Honker `query()` returns `{ columnName: value }` object rows, compatible
with Drizzle's `mapResultRow()`.
- `last_insert_rowid()` accessible via `tx.query("SELECT last_insert_rowid() as id")`.
- `tx.execute()` returns a number (affected rows count).
- JSON-mode columns need manual `JSON.parse()` in the adapter.
## HonkerEventTarget
The `HonkerEventTarget` adapts `@alkdev/pubsub`'s `TypedEventTarget` interface
to Honker's `notify`/`listen` (ephemeral) and `stream`/`subscribe` (durable)
primitives. It lives in `src/sqlite/event-target.ts`.
### Interface
```ts
import type { TypedEventTarget } from "@alkdev/pubsub";
interface HonkerEventTargetOptions {
db: Database;
mode: "ephemeral" | "durable";
streamName?: string;
consumerName?: string;
prefix?: string;
}
function createHonkerEventTarget<TEvent>(
options: HonkerEventTargetOptions
): TypedEventTarget<TEvent> & { close(): void };
```
### Ephemeral Mode
Maps to Honker's `notify()`/`listen()`:
| pubsub operation | Honker operation |
|-------------------|------------------|
| `addEventListener("topic:id", callback)` | `db.listen("topic:id")` → start async consumer loop |
| `dispatchEvent(event)` | `db.notify(event.type, event.detail)` |
| `removeEventListener("topic:id", callback)` | Close the listener when no callbacks remain |
| `close()` | Close all active listeners |
POC 2 confirmed: same-process `notify``listen` works, multiple concurrent
listeners on different channels work, cross-channel isolation is correct.
POC 4 confirmed: `tx.notify()` within a Drizzle transaction only fires the
notification on `tx.commit()`. On `tx.rollback()`, the notification is
suppressed. This enables transactional outbox semantics for ephemeral events.
### Durable Mode
Maps to Honker's `Stream.publish()`/`Stream.subscribe()`:
| pubsub operation | Honker operation |
|-------------------|------------------|
| `addEventListener("topic:id", callback)` | `db.stream(name).subscribe(consumer)` → start async consumer loop |
| `dispatchEvent(event)` | `db.stream(name).publish(event.detail)` |
| `removeEventListener("topic:id", callback)` | Close the subscription when no callbacks remain |
| `close()` | Close all active subscriptions, save offsets |
POC 3 confirmed: `Stream.publish()`/`Stream.subscribe()` work, consumer
offsets are tracked and persisted, `publishTx(tx, payload)` works within
transactions, and concurrent stream + listener operation works.
Durable mode provides crash recovery — consumers resume from their last saved
offset after restart. Consumer names must be stable across restarts (not
PID-based).
### Topic Routing
pubsub uses `topic:id` composite topics. Honker channels and streams are flat
strings. The mapping:
- **Ephemeral**: Each unique `topic:id` gets its own `db.listen()` call.
For high-cardinality topics (many request IDs), consider topic prefix
matching with client-side filtering instead.
- **Durable**: Streams are name-keyed, not topic-keyed. A single stream
carries all events for a domain. Client-side filtering dispatches only
to matching listeners.
Suggested split:
| Event category | Mode | Reason |
|---------------|------|--------|
| Call protocol events (`call.requested`, `call.responded`, etc.) | Durable stream | Crash recovery, audit trail, flowgraph replay |
| Cache invalidation signals | Ephemeral | Fire-and-forget, loss acceptable |
| UI/dashboard push | Ephemeral | Low latency, loss acceptable |
| Schema migration jobs | Queue (not pubsub) | At-least-once processing |
### Latency Consideration
POC 2 measured ~17ms median latency for Honker `notify``listen` within a
single process. For hot-path call protocol request/response where sub-ms
latency matters, pair the Honker event target with an in-process `EventTarget`
(pubsub's default). A composite pattern (dispatch to both) provides both
in-process speed and Honker durability/cross-process coordination.
### Hub-Spoke Event Routing
```
Hub (system.db + tenant-{orgId}.db)
├── HonkerEventTarget (durable: call-protocol stream)
├── WebSocketServerEventTarget (spoke fan-out)
└── In-process EventTarget (local subscribers)
Spoke (tenant-{orgId}.db)
├── HonkerEventTarget (ephemeral: local channels)
├── WebSocketClientEventTarget (hub connection)
└── In-process EventTarget (local subscribers)
```
Both hub and spoke use the same `createPubSub({ eventTarget })` call.
Different event target instances determine the routing. No code changes
between hub and spoke — only configuration.
## Event-Driven Patterns
@@ -338,7 +464,7 @@ would require a coordinator — the hub fills this role.
- The metagraph Module system (CallGraph, SecretGraph, AclGraph Modules)
- Bridge functions (`moduleToDbSchema`, `validateNode`, `validateEdge`)
- Crypto utility (encrypt, decrypt, `EncryptedDataSchema`)
- TypeBox schemas from drizzlebox
- TypeBox schemas from local utils (folded from drizzlebox)
- The Drizzle query builder API (same `.select()`, `.insert()`, `.update()`, `.delete()` calls)
- The `@alkdev/operations` call protocol (events are now published via Honker streams instead of Redis)
@@ -359,13 +485,18 @@ would require a coordinator — the hub fills this role.
|-----|----------|--------|
| [039](decisions/039-honker-as-sqlite-extension.md) | Honker as SQLite extension and transport | Accepted |
| [044](decisions/044-drizzle-honker-adapter.md) | Drizzle-Honker session adapter | Accepted |
| [046](decisions/046-fold-drizzlebox-as-utils.md) | Fold @alkdev/drizzlebox as src/sqlite/utils/ | Accepted |
| [047](decisions/047-honker-event-target.md) | HonkerEventTarget adapter for pubsub | Accepted |
## Open Questions
- **OQ-26**: Can Honker fully replace `@alkdev/pubsub`'s Redis transport for single-node deployments?
- **OQ-26**: ~~Can Honker fully replace `@alkdev/pubsub`'s Redis transport for single-node deployments?~~ Resolved: Yes. HonkerEventTarget (ADR-047) provides the adapter. Redis still needed for multi-node.
- **OQ-27**: How are schema migrations applied across all tenant DBs?
- **OQ-28**: How does cross-tenant delegation work with separate DBs?
- **OQ-29**: Should the adapter be published as a standalone `drizzle-honker` npm package for community use?
- **OQ-30**: Composite event target pattern — how should an in-process EventTarget and HonkerEventTarget be combined for single-node hub deployments? POC 2 showed ~17ms Honker latency vs sub-ms in-process. Design needed.
- **OQ-31**: Consumer naming convention for durable subscriptions — must be stable across hub restarts (not PID-based).
- **OQ-32**: Drizzle Kit migration compatibility — does `drizzle-kit push`/`drizzle-kit generate` work with the custom Honker adapter, or do we need a custom migration runner?
## References