- Update architecture docs to reflect pivot from @libsql/client to Honker - Fold @alkdev/drizzlebox Phase 0 into src/sqlite/utils/ (ADR-046) - Add HonkerEventTarget adapter for pubsub TypedEventTarget (ADR-047) - Replace hand-written CRUD with OperationSpec generation (ADR-048) - Resolved OQ-26: Honker replaces Redis for single-node pub/sub (POC validated) - Updated OQ-17, OQ-18, OQ-19 for OperationSpec repository surface - Added OQ-30 (composite event target), OQ-31 (consumer naming), OQ-32 (Drizzle Kit) - POC results: adapter buildable, same-process pub/sub works, transactional outbox semantics confirmed, concurrent listeners/streams work - Research doc at docs/research/pivot-honker-sqlite-adapter.md
31 KiB
status, created, last_updated
| status | created | last_updated |
|---|---|---|
| draft | 2026-06-01 | 2026-06-01 |
Pivot: Honker SQLite Adapter, dbtype Fold, and Event Target Integration
Research document for the architecture pivot from @libsql/client to Honker,
folding @alkdev/dbtype Phase 0 into src/utils/, and adding a pubsub
HonkerEventTarget.
Motivation
- Honker provides SQLite + pub/sub + streams + queues + locks + scheduler in
a single
.dbfile via a Rust NAPI binding. This replaces both@libsql/client(SQLite only) and the conceptual need for Redis. @alkdev/dbtypePhase 0 is already consumed as@alkdev/drizzlebox. With SQLite as the sole target, the multi-dialect column mappings are dead weight. Folding the SQLite-relevant subset intosrc/utils/eliminates an external dependency and co-locates utility code with the tables it derives schemas from.@alkdev/pubsubdefines aTypedEventTargetinterface that Honker's primitives (notify/listen,stream/stream.subscribe) map to naturally. AHonkerEventTargetadapter lets storage's downstream consumers use the same pubsub API regardless of whether events route in-process, through Honker, through Redis, or over WebSocket.- The hub-spoke model needs events for call protocol routing, cache invalidation, flowgraph reactive updates, and cross-process coordination. Honker handles all of these within SQLite.
Architecture Changes
Current Structure
src/
├── graphs/ # Metagraph Module + bridge (no db deps)
│ ├── modules/ # CallGraph, SecretGraph, Metagraph
│ ├── bridge.ts # moduleToDbSchema, validateNode, validateEdge
│ ├── crypto.ts # encrypt, decrypt, generateEncryptionKey
│ └── mod.ts
├── pg/ # Placeholder (to be removed)
└── sqlite/ # SQLite host
├── client.ts # createSqliteDatabase(@libsql/client)
├── mod.ts
├── relations.ts
├── schema.ts
└── tables/ # Flat: graphTypes, nodeTypes, edgeTypes, graphs, nodes, edges, actors
Dependencies: @alkdev/drizzlebox (external npm), @libsql/client, drizzle-orm
Target Structure
src/
├── graphs/ # Unchanged
│ ├── modules/
│ ├── bridge.ts
│ ├── crypto.ts
│ └── mod.ts
└── sqlite/
├── adapter.ts # NEW: Drizzle-Honker session adapter
├── event-target.ts # NEW: HonkerEventTarget implements pubsub TypedEventTarget
├── client.ts # UPDATED: createSystemDatabase(client), createTenantDatabase(client)
├── mod.ts # UPDATED: re-exports
├── relations.ts # UPDATED: remove actors relations
├── schema.ts # UPDATED: remove actors, remove pg refs
├── utils/ # NEW: folded from @alkdev/dbtype Phase 0 (SQLite only)
│ ├── schema.ts # createSelectSchema, createInsertSchema, createUpdateSchema
│ ├── column.ts # Column→TypeBox mappings (SQLite-only dispatch)
│ ├── types.ts # Public TypeScript interfaces (merged from schema.types + schema.types.internal)
│ ├── constants.ts # Integer range constants
│ └── utils.ts # isColumnType, isWithEnum, type helpers
└── tables/ # Restructured into subdirectories
├── common.ts # shared column definitions, ACTOR_TYPE removed
├── metagraph/ # graph_types, node_types, edge_types, graphs, nodes, edges
│ └── index.ts
└── identity/ # NEW: accounts, organizations, org_members, api_keys, audit_logs
└── index.ts
Removed: src/pg/, actors table, @alkdev/drizzlebox external dep,
@libsql/client dep.
New deps: @alkdev/pubsub (peer dep for TypedEventTarget type only),
honker (replaces @libsql/client).
1. Fold dbtype Phase 0 → src/sqlite/utils/
What to fold
Only SQLite-relevant code from @alkdev/dbtype/src/:
| Source (dbtype) | Target (storage) | Changes |
|---|---|---|
schema.ts |
utils/schema.ts |
Remove PgEnum import/handling, remove isPgEnum calls. Remove handleEnum(). Keep createSelectSchema, createInsertSchema, createUpdateSchema. |
column.ts |
utils/column.ts |
Strip all non-SQLite dispatch branches. Remove PG, MySQL, SingleStore type imports and isColumnType branches. Keep: SQLiteInteger, SQLiteReal, SQLiteText handling. Keep generic dispatch (dataType-based) as fallback. Keep literalSchema, jsonSchema, bufferSchema, mapEnumValues. |
column.types.ts |
utils/types.ts (merged) |
Strip non-SQLite type branches. Keep SQLiteInteger → t.TInteger. Keep generic fallback types. |
schema.types.ts |
utils/types.ts (merged) |
Remove PgEnum overloads. |
schema.types.internal.ts |
utils/types.ts (merged) |
Keep Conditions, BuildRefine, BuildSchema, NoUnknownKeys. |
constants.ts |
utils/constants.ts |
Keep as-is (ranges used by SQLite integer/real/bigiint mappings). |
utils.ts |
utils/utils.ts |
Remove PgEnum type alias. Keep isColumnType, isWithEnum. Keep JsonSchema, BufferSchema types. Remove unused type helpers. |
What NOT to fold
- PG/MySQL/SingleStore column type handlers (dead code for SQLite-only)
isPgEnumandhandleEnum(PostgreSQL-specific)createSchemaFactory(no known consumer in storage; can add later if needed)- The
scripts/probe-e2e.ts(dbtype Phase 1 POC, not relevant)
Import changes in existing table files
Every table file currently imports from @alkdev/drizzlebox:
// Before
import { createInsertSchema, createSelectSchema } from "@alkdev/drizzlebox";
// After
import { createInsertSchema, createSelectSchema } from "../utils/schema.ts";
The createSelectSchema and createInsertSchema functions produce the same
TypeBox schemas — the API surface is identical, the implementation just
excludes non-SQLite branches.
Deno.json import map changes
// Before
"imports": {
"@alkdev/drizzlebox": "npm:@alkdev/drizzlebox",
"@libsql/client": "npm:@libsql/client",
// ...
}
// After (remove drizzlebox and libsql, add honker and pubsub peer)
"imports": {
"honker": "npm:honker",
"@alkdev/pubsub": "npm:@alkdev/pubsub",
// ...
}
Note: @alkdev/pubsub is a peer dependency for the TypedEventTarget type.
It is only needed by consumers who use the HonkerEventTarget. The
src/graphs/ module remains zero-dep.
2. Drizzle-Honker Session Adapter (src/sqlite/adapter.ts)
Problem
Drizzle ORM expects a session that implements SQLiteSession<'sync'>. Honker
provides Database.query(sql, params) for reads and Transaction.execute(sql, params) for writes. These don't implement Drizzle's session interface.
Adapter Architecture
Consumer code
├── db.select().from(graphs)... → Drizzle query builder
│ └── HonkerPreparedQuery.run/all/get/values
│ └── honkerDb.query(sql, params) / tx.execute(sql, params)
├── db.$client.notify('channel', p) → Honker ephemeral pub/sub
├── db.$client.stream('events') → Honker durable stream
├── db.$client.queue('jobs') → Honker work queue
└── db.$client.listen('channel') → Honker listener
Three adapter classes:
| Class | Extends | Purpose |
|---|---|---|
HonkerSQLiteSession |
SQLiteSession<'sync'> |
Routes Drizzle queries to honkerDb.query() / tx.execute() |
HonkerPreparedQuery |
SQLitePreparedQuery |
Implements run(), all(), get(), values() via Honker |
HonkerSQLiteTransaction |
SQLiteTransaction<'sync'> |
Drizzle transaction with $honkerTx for Honker primitives |
Accessing Honker from Drizzle
db.$client— the HonkerDatabaseinstance (outside transactions)tx.$honkerTx— the HonkerTransactioninstance (inside Drizzle tx callbacks)
db.transaction((tx) => {
tx.insert(schema.nodes).values({ graphId, key: "call-1", attributes: {} }).run();
tx.$honkerTx.notify("nodes:created", { graphId, key: "call-1" });
});
Both data write and notification commit atomically.
Known challenges (need POC validation)
-
No
lastInsertRowidfrom Honkerexecute()— Drizzle'srun()needslastInsertRowidfor auto-increment inserts. Honker's JS API doesn't expose it. POC 1 confirmed workaround:tx.query("SELECT last_insert_rowid() as id")returns the rowid. -
Object rows only — Honker
query()returnsArray<Record<string, any>>. Drizzle'svalues()mode expects raw arrays. The adapter must convert viaObject.values()with column order from the query metadata. -
Drizzle internal API dependency — The adapter imports from
drizzle-orm/sqlite-core/session, which is not a guaranteed-stable API. POC 1 confirmed all needed classes are accessible and extendable. -
Session constructor shape — POC 1 confirmed:
SQLiteSessiontakes(mode, executeMethod, query, cache?, queryMetadata?).BaseSQLiteDatabaseis accessible fromdrizzle-orm/sqlite-core/db. TheLibSQLSessionindrizzle-orm/libsql/sessionis a working reference implementation. -
:memory:databases don't work — Honker's reader pool creates separate in-memory databases. File-based paths must be used. POC 1 confirmed this. Not a real limitation — production uses file paths, tests use temp files.
3. HonkerEventTarget (src/sqlite/event-target.ts)
Interface
Implements @alkdev/pubsub's TypedEventTarget<TEvent>:
interface HonkerEventTargetOptions {
db: Database;
mode: "ephemeral" | "durable";
streamName?: string;
prefix?: string;
}
function createHonkerEventTarget<TEvent>(options: HonkerEventTargetOptions): HonkerEventTarget<TEvent>
Ephemeral mode mapping
Maps to Honker's notify() / listen():
addEventListener(type, callback):
if (!listeners.has(type)):
honkerListener = db.listen(type)
// start async loop: for await (notification of honkerListener)
// dispatch CustomEvent with notification as detail
listeners.set(type, callback)
dispatchEvent(event):
db.notify(event.type, event.detail) // or tx.notify() inside transaction
removeEventListener(type, callback):
listeners.delete(type, callback)
if (no more listeners for type):
honkerListener.close()
Durable mode mapping
Maps to Honker's Stream.publish() / Stream.subscribe():
addEventListener(type, callback):
if (!subscriptions.has(type)):
stream = db.stream(streamName ?? type)
subscription = stream.subscribe(consumerName)
// start async loop: for await (event of subscription)
// dispatch CustomEvent with event.payload as detail
subscriptions.set(type, { stream, subscription, callback })
dispatchEvent(event):
db.stream(streamName).publish(event.detail)
// or stream.publishTx(tx) inside transaction
removeEventListener(type, callback):
subscriptions.delete(type, callback)
if (no more subscriptions for type):
subscription.close()
Topic routing considerations
pubsub uses type:id composite topics (e.g., "call.responded:req-123").
Honker channels are flat strings. The mapping is 1:1 — no transformation
needed. But:
-
Ephemeral: Each unique
type:idgets its owndb.listen()call. For high-cardinality topics (many request IDs), this could mean many concurrent listeners. POC 3 confirmed multiple concurrent listeners work. -
Durable: Streams are name-keyed, not topic-keyed. A single stream (e.g.,
"call-protocol") carries all call events. The adapter would need to handle topic filtering client-side (dispatch only to matching listeners) rather than using separate streams per topic. -
Suggested split: Call protocol events → durable stream. Cache invalidation, UI push → ephemeral channels. The configuration for which topics go where is a consumer decision, not built into the adapter.
-
Latency consideration: POC 2 measured ~17ms median for
notify→listenround-trip. For hot-path call protocol request/response in a single process, pair the Honker event target with an in-processEventTarget(pubsub's default) for sub-ms latency. The Honker event target provides durability and cross-process coordination; the default EventTarget provides speed. A composite pattern (dispatch to both) could serve both needs.
Consumer name strategy
For durable mode, each consumer needs a unique name for offset tracking.
Suggested pattern: {processId}:{topic} or a configurable consumer group ID.
4. Client Factory Update (src/sqlite/client.ts)
Before
import { drizzle, type LibSQLDatabase } from "drizzle-orm/libsql";
import type { Client } from "@libsql/client";
export function createSqliteDatabase(client: Client): LibSQLDatabase<SqliteSchema> {
return drizzle(client, { schema });
}
After
The factory accepts a pre-created Honker Database instance (injectable
client pattern, per ADR-004). It constructs the Drizzle-Honker adapter
internally.
import type { Database as HonkerDatabase } from "honker";
export type SqliteDatabase = BaseSQLiteDatabase<'sync', SqliteSchema>;
export function createSystemDatabase(client: HonkerDatabase): SqliteDatabase {
return createDrizzleWithAdapter(client, { schema });
}
export function createTenantDatabase(client: HonkerDatabase): SqliteDatabase {
return createDrizzleWithAdapter(client, { schema });
}
createDrizzleWithAdapter is the internal factory that constructs the
HonkerSQLiteSession and wraps it in a BaseSQLiteDatabase. This may
require using drizzle-orm/sqlite-core internals rather than the
drizzle-orm/libsql high-level wrapper.
5. Table Restructure
Remove actors table
Per existing ADRs (035, 041), actors are being replaced by ACL nodes in
graph instances. Remove src/sqlite/tables/actors.ts and all exports.
Reorganize into subdirectories
src/sqlite/tables/
├── common.ts # commonCols (UPDATED: remove ACTOR_TYPE)
├── metagraph/
│ ├── index.ts # barrel
│ ├── graph-types.ts # renamed from graphTypes.ts
│ ├── node-types.ts # renamed from nodeTypes.ts
│ ├── edge-types.ts # renamed from edgeTypes.ts
│ ├── graphs.ts
│ ├── nodes.ts
│ └── edges.ts
└── identity/
├── index.ts # barrel
├── accounts.ts
├── organizations.ts
├── org-members.ts
├── api-keys.ts
└── audit-logs.ts
Import updates in table files
All table files currently importing @alkdev/drizzlebox update to
../utils/schema.ts (or ../../utils/schema.ts depending on depth).
The commonCols import path changes from ./common.ts to ../common.ts
for metagraph/identity subdirectories.
6. Domain-Specific Tables (Future, documented for scope)
The current metagraph tables store node/edge attributes as JSON. For known graph types (CallGraph, SecretGraph, AclGraph), native columns give better type safety and query performance.
This is not part of the initial pivot. But the pivot should not prevent
it. The folded utils/ code already provides createInsertSchema /
createSelectSchema which derives TypeBox schemas from Drizzle tables with
native columns. When domain tables are added later, they follow the same
pattern as the metagraph tables.
Where OperationSpecs fit
Domain tables would come with auto-generated OperationSpec[] — flat arrays
describing CRUD operations for each table. The hub/spoke imports these specs,
registers them in the operations registry, and the call protocol handles
execution.
This means storage's "repository layer" is not hand-written query functions. It's table definitions + operation contract definitions. The consumer (hub/spoke) provides the execution layer.
// In storage: table definition + operation contracts
export const callNodes = sqliteTable("call_nodes", { ... });
export const callNodeOperations: OperationSpec[] = [
{ name: "create", namespace: "call_nodes", type: "mutation", ... },
{ name: "find", namespace: "call_nodes", type: "query", ... },
// ...
];
// In hub: register operations
for (const spec of callNodeOperations) {
registry.registerSpec(spec);
registry.registerHandler(specId, (input) => db.insert(callNodes).values(input).returning());
}
The OperationSpec type is straightforward (@alkdev/operations peer dep for
type only). Storage defines the contract; the hub provides the handler.
7. Identity Tables (New, documented for scope)
Per sqlite-host.md and ADR-040/041, these live in the system database:
| Table | Key Columns | Purpose |
|---|---|---|
accounts |
id, email, name, accessLevel | User accounts |
organizations |
id, slug, name, ownerId | Tenant organizations |
org_members |
id, orgId, accountId, level | Membership junction |
api_keys |
id, keyHash, ownerId, name | API key auth |
audit_logs |
id, actorType, actorId, action, resource | Append-only audit |
These need actual implementation (currently only spec in architecture docs). They're part of the pivot scope but can be implemented after the adapter and utils fold.
8. Event Pipeline: Operations → Flowgraph → Storage
How the pieces connect after the pivot
[operations: OperationRegistry]
│
│ call.requested / call.responded / call.completed / call.aborted / call.error
▼
[pubsub: createPubSub({ eventTarget: honkerEventTarget })]
│
├──→ [flowgraph: WorkflowReactiveRoot.append(event)]
│ └── signal propagation → UI updates
│
├──→ [storage: db.stream("call-protocol").publish(event)]
│ └── durable replay for crash recovery
│
└──→ [spoke sync: via WebSocket or Honker fan-out]
└── cross-process coordination
The HonkerEventTarget is the single integration point that makes this work.
Operations doesn't need to know about Honker. Flowgraph doesn't need to know
about Honker. They both use @alkdev/pubsub's TypedEventTarget. The
HonkerEventTarget is just another adapter in pubsub's existing portfolio
(alongside Redis, WebSocket, Worker).
Hub-spoke event routing
Hub (system.db + tenant-{orgId}.db)
├── HonkerEventTarget (durable: call-protocol stream)
├── WebSocketServerEventTarget (spoke fan-out)
└── In-process EventTarget (local subscribers)
Spoke (tenant-{orgId}.db)
├── HonkerEventTarget (ephemeral: local channels)
├── WebSocketClientEventTarget (hub connection)
└── In-process EventTarget (local subscribers)
Both hub and spoke use the same code. Same createPubSub({ eventTarget }).
Different event target instance determines the routing.
POC Plan
Before committing to the full pivot, validate the critical path with targeted POCs. Each POC answers one specific hard question.
POC 1: Drizzle queries through Honker
Question: Can we construct a BaseSQLiteDatabase that routes queries
through Honker's Database.query() and Transaction.execute()?
Approach:
- Import
SQLiteSession,SQLitePreparedQuery,SQLiteTransactionfromdrizzle-orm/sqlite-core/session. - Implement
HonkerSQLiteSessionthat delegates toDatabase.query(). - Implement
HonkerPreparedQuerywithrun(),all(),get(). - Wire it into a
BaseSQLiteDatabaseinstance. - Run a simple
db.select().from(graphs)against a Honker-opened.dbfile.
Hard fail conditions:
- Drizzle's
SQLiteSessionrequires constructor arguments we can't provide from Honker's primitives. - The
BaseSQLiteDatabaseconstructor requires internals not exported fromdrizzle-orm/sqlite-core. lastInsertRowidis required and no workaround exists.
POC 2: Honker pub/sub from same process
Question: Does db.notify() → db.listen() work within the same Node.js
process? What's the latency?
Approach:
- Open a Honker database.
- Start
db.listen("test-channel"). - Call
db.notify("test-channel", { hello: "world" }). - Verify the listener receives the notification.
- Measure round-trip latency (10,000 iterations).
Hard fail conditions:
- Same-process
notify/listendoesn't work (would require two DB instances or fallback to in-process EventTarget). - Latency exceeds 10ms median (would be too slow for call protocol request/response).
POC 3: Concurrent listeners and stream subscriptions
Question: Can multiple Listener and StreamSubscription instances
coexist on the same Database? Do they interleave correctly?
Approach:
- Open a Honker database.
- Start two
db.listen()calls on different channels. - Start a
db.stream("test-stream").subscribe("consumer-1"). - Publish to both channels and the stream concurrently.
- Verify all three consumers receive their events without blocking each other.
Hard fail conditions:
- Only one
Listenercan be active at a time (would require serialization). Listener.next()blocks the JS event loop (would prevent concurrent subscriptions).
POC 4: Transactional notify + data write
Question: Does tx.notify() + tx.execute() in the same transaction
atomically commit both the data write and the notification?
Approach:
- Open a Honker database.
- Start
db.listen("test-channel"). - Begin a transaction:
const tx = db.transaction(). - Execute a data write:
tx.execute("INSERT INTO test ..."). - Call
tx.notify("test-channel", { inserted: true }). - Rollback the transaction.
- Verify the listener does NOT receive the notification.
- Repeat with commit instead of rollback.
- Verify the listener DOES receive the notification after commit.
Hard fail conditions:
- Notifications fire immediately on
tx.notify()regardless of commit/rollback (breaks transactional outbox semantics).
POC file structure
poc/
├── README.md # POC objectives, how to run
├── 01-drizzle-honker-adapter.ts # POC 1
├── 02-same-process-pubsub.ts # POC 2
├── 03-concurrent-listeners.ts # POC 3
├── 04-transactional-notify.ts # POC 4
└── shared/
└── honker-helpers.ts # Shared setup/teardown utilities
Each POC is a standalone script that can be run with deno run or node.
The POCs use the actual honker npm package from the workspace.
ADR Updates Required
| ADR | Action | Reason |
|---|---|---|
| ADR-018 (dbtype integration post-v1) | Revise | No longer deferred — Phase 0 folds in now as src/sqlite/utils/ |
| ADR-038 (SQLite-first, PG removed) | Confirm | Pivot confirms this. Remove src/pg/ and PG imports from deno.json. |
| ADR-039 (Honker as SQLite extension) | Confirm | Pivot is the implementation of this decision. |
| ADR-044 (Drizzle-Honker adapter) | Update | Add specifics about session/PreparedQuery/Transaction classes. |
| ADR-005 (drizzle + typebox via drizzlebox) | Revise | Drizzlebox is no longer an external dep. It's src/sqlite/utils/. |
| ADR-019 (JSON text for schema columns) | Confirm | Still relevant for metagraph attributes. Domain tables add native columns alongside JSON. |
| ADR-033 (JSON path queries for v1) | Revise | Repository layer is now "operation specs," not hand-written CRUD. The query approach (JSON path vs native) still applies, but the output format changes. |
| New ADR | Fold dbtype Phase 0 as utils | Records the decision to fold SQLite mappings into src/sqlite/utils/. |
| New ADR | HonkerEventTarget adapter | Records the decision to implement pubsub TypedEventTarget on Honker primitives. |
| New ADR | Operation specs as repository surface | Records that storage outputs OperationSpec[], not hand-written query functions. |
Architecture Doc Updates Required
| Document | Changes |
|---|---|
overview.md |
Subpath exports (./pg removed), dependency list, module structure, "what's not done" section, system/tenant DB model |
forward-look.md |
dbtype section (folded, not deferred), repository layer strategy (operations, not CRUD), pointer abstraction (still deferred) |
honker-integration.md |
Add HonkerEventTarget section, update adapter section with verified API, add event pipeline diagram, resolve OQ-26 (Honker replaces Redis for single-node) |
sqlite-host.md |
Table structure changes (subdirectories, actors removed, identity tables), client factory signature, Honker import instead of libsql |
metagraph-module.md |
No changes (Module format is independent of storage host) |
schema-evolution.md |
Minor: createSelectSchema/createInsertSchema now come from local utils, not external dep |
open-questions.md |
Resolve OQ-17 (operations, not CRUD), OQ-18 (auto-generated specs from tables), OQ-26 (Honker replaces Redis). Update OQ-19 (bridge package location → it's storage itself). |
encrypted-data.md |
No changes |
Implementation Order (post-POC)
If POCs pass, the implementation sequence should be:
- Fold dbtype →
src/sqlite/utils/— import path changes only, no behavioral changes. Run existing tests to verify. - Remove
src/pg/and actors table — clean up dead code. Update deno.json imports, removepostgresand PG-related entries. - Restructure tables into subdirectories — move files, update relative
imports, update
schema.tsandrelations.ts. - Implement
adapter.ts— Drizzle-Honker session adapter based on POC 1 findings. - Update
client.ts—createSystemDatabase()/createTenantDatabase()taking HonkerDatabaseinstances. - Implement
event-target.ts—HonkerEventTargetbased on POC 2-4 findings. - Implement identity tables —
src/sqlite/tables/identity/. - Update architecture docs — bring all documents in line with the new structure.
- Add domain tables (future, not in initial pivot) — native-column tables for CallGraph, SecretGraph, etc. with OperationSpec[] output.
Steps 1-3 are mechanical (no new behavior). Steps 4-5 depend on POC 1. Step 6 depends on POCs 2-4. Step 7 can proceed independently. Steps 8-9 are follow-ups.
POC Results (2026-06-01)
POC 1: Drizzle queries through Honker — PASS
SQLiteSession,SQLitePreparedQuery,SQLiteTransactionare all accessible fromdrizzle-orm/sqlite-core/session— can be extended.BaseSQLiteDatabaseis accessible fromdrizzle-orm/sqlite-core/db.- Honker
query()returns{ columnName: value }object rows, compatible with Drizzle'smapResultRow(). last_insert_rowid()accessible viatx.query("SELECT last_insert_rowid() as id").tx.execute()returns a number (affected rows count for UPDATE/DELETE).- JSON-mode columns need manual
JSON.parse()in the adapter. :memory:databases DON'T work — reader pool gets separate in-memory DBs. Always use file-based paths (or:memory:with a single reader, which defeats Honker's concurrent reader design).LibSQLSessionindrizzle-orm/libsql/sessionis a good reference implementation for the adapter pattern.
Session method signatures discovered:
SQLiteSession methods:
prepareOneTimeQuery — key method to implement
all, run, get, values — delegated to PreparedQuery
extractRaw*ValueFromBatchResult — batch result extraction
count — cursor count
SQLitePreparedQuery methods:
execute, getQuery, mapAllResult, mapGetResult, mapResult, mapRunResult
queryWithCache
SQLiteTransaction methods:
rollback — only explicit method (commit handled by Session.transaction)
POC 2: Same-process pub/sub — PASS (with latency caveat)
db.notify()→db.listen()works within the same process.- Multiple concurrent listeners on different channels work.
- Cross-channel isolation works — listeners only receive notifications on their subscribed channel.
- Latency: ~17ms median, ~29ms P99 (100 samples). This is higher than
ideal for call protocol request/response. The
Listeneruses a polling fallback (updateEvents().next()) internally. For hot-path events where sub-ms latency matters, the in-processEventTargetshould be preferred. Honker listeners are better suited for cross-process and durable scenarios.
POC 3: Concurrent listeners and streams — PASS
- Multiple
Listenerinstances coexist on the sameDatabase. Stream.publish()/Stream.subscribe()works.readSince(offset, limit)returns historical events.StreamSubscriptionimplementsAsyncIterableIterator— works withfor await...of.- Consumer offsets are tracked and persisted.
- Concurrent stream + listener operation works.
publishTx(tx, payload)works within a transaction.- Caveat:
db.close()doesn't cleanly exit when listeners are still polling. Need to close all listeners before closing the DB.
POC 4: Transactional notify + data write — PASS (atomicity confirmed)
tx.notify()+tx.execute()— notification fires ONLY aftertx.commit(). Before commit, no notification is received. Transactional semantics hold.- On
tx.rollback()— notification is NOT sent. Data is NOT persisted. Atomicity holds for both data and events. db.notifyTx(tx, channel, payload)— convenience wrapper, same semantics.queue.enqueueTx(tx, payload)— job is only visible after commit. On rollback, job does not appear in the queue. Transactional outbox works.- Read-your-writes: after receiving a notification, the data IS queryable (no stale read issue with single DB).
This is the most important result. Honker provides true transactional outbox semantics — data writes and event/queue publishes commit atomically. This eliminates the dual-write problem that necessitated Redis for PostgreSQL.
Open Research Questions
-
Drizzle session construction— RESOLVED by POC 1. All needed classes are accessible and extendable.LibSQLSessionis the reference. -
Honker listener concurrency model— RESOLVED by POC 3. Multiple listeners coexist. TheupdateEventsmechanism works for concurrent polling.db.close()requires all listeners to be closed first. -
Consumer name management for streams — When a hub restarts, durable stream consumers resume from their last saved offset. But the consumer name must be stable across restarts. Need a naming convention that's deterministic (e.g.,
{service}:{pid}is not restart-stable). -
Schema migrations — Honker opens a single
.dbfile. Drizzle Kit supports SQLite migrations but expectsbetter-sqlite3orlibsql. Need to verify thatdrizzle-kit pushordrizzle-kit generateworks with the Honker adapter, or whether we need a custom migration runner. -
OperationSpec generation from tables — What's the concrete mapping? For each table: one
createmutation, onefindquery, onelistquery, oneupdatemutation, onedeletemutation? Or something more domain-specific? This is a design question, not a POC question. -
In-process + Honker composite event target — POC 2 showed ~17ms latency for Honker notify→listen. For single-process hot paths, an in-process EventTarget is sub-ms. A composite that dispatches to both (in-process for speed, Honker for durability/cross-process) would be the ideal default for a hub. Needs design work.