Files
storage/docs/research/pivot-honker-sqlite-adapter.md
glm-5.1 412ad98f11 Pivot: fold drizzlebox as utils, HonkerEventTarget, OperationSpecs as repo surface
- Update architecture docs to reflect pivot from @libsql/client to Honker
- Fold @alkdev/drizzlebox Phase 0 into src/sqlite/utils/ (ADR-046)
- Add HonkerEventTarget adapter for pubsub TypedEventTarget (ADR-047)
- Replace hand-written CRUD with OperationSpec generation (ADR-048)
- Resolved OQ-26: Honker replaces Redis for single-node pub/sub (POC validated)
- Updated OQ-17, OQ-18, OQ-19 for OperationSpec repository surface
- Added OQ-30 (composite event target), OQ-31 (consumer naming), OQ-32 (Drizzle Kit)
- POC results: adapter buildable, same-process pub/sub works, transactional
  outbox semantics confirmed, concurrent listeners/streams work
- Research doc at docs/research/pivot-honker-sqlite-adapter.md
2026-06-01 16:31:40 +00:00

31 KiB

status, created, last_updated
status created last_updated
draft 2026-06-01 2026-06-01

Pivot: Honker SQLite Adapter, dbtype Fold, and Event Target Integration

Research document for the architecture pivot from @libsql/client to Honker, folding @alkdev/dbtype Phase 0 into src/utils/, and adding a pubsub HonkerEventTarget.

Motivation

  1. Honker provides SQLite + pub/sub + streams + queues + locks + scheduler in a single .db file via a Rust NAPI binding. This replaces both @libsql/client (SQLite only) and the conceptual need for Redis.
  2. @alkdev/dbtype Phase 0 is already consumed as @alkdev/drizzlebox. With SQLite as the sole target, the multi-dialect column mappings are dead weight. Folding the SQLite-relevant subset into src/utils/ eliminates an external dependency and co-locates utility code with the tables it derives schemas from.
  3. @alkdev/pubsub defines a TypedEventTarget interface that Honker's primitives (notify/listen, stream/stream.subscribe) map to naturally. A HonkerEventTarget adapter lets storage's downstream consumers use the same pubsub API regardless of whether events route in-process, through Honker, through Redis, or over WebSocket.
  4. The hub-spoke model needs events for call protocol routing, cache invalidation, flowgraph reactive updates, and cross-process coordination. Honker handles all of these within SQLite.

Architecture Changes

Current Structure

src/
├── graphs/           # Metagraph Module + bridge (no db deps)
│   ├── modules/      # CallGraph, SecretGraph, Metagraph
│   ├── bridge.ts     # moduleToDbSchema, validateNode, validateEdge
│   ├── crypto.ts     # encrypt, decrypt, generateEncryptionKey
│   └── mod.ts
├── pg/               # Placeholder (to be removed)
└── sqlite/           # SQLite host
    ├── client.ts     # createSqliteDatabase(@libsql/client)
    ├── mod.ts
    ├── relations.ts
    ├── schema.ts
    └── tables/       # Flat: graphTypes, nodeTypes, edgeTypes, graphs, nodes, edges, actors

Dependencies: @alkdev/drizzlebox (external npm), @libsql/client, drizzle-orm

Target Structure

src/
├── graphs/               # Unchanged
│   ├── modules/
│   ├── bridge.ts
│   ├── crypto.ts
│   └── mod.ts
└── sqlite/
    ├── adapter.ts        # NEW: Drizzle-Honker session adapter
    ├── event-target.ts   # NEW: HonkerEventTarget implements pubsub TypedEventTarget
    ├── client.ts         # UPDATED: createSystemDatabase(client), createTenantDatabase(client)
    ├── mod.ts            # UPDATED: re-exports
    ├── relations.ts      # UPDATED: remove actors relations
    ├── schema.ts         # UPDATED: remove actors, remove pg refs
    ├── utils/            # NEW: folded from @alkdev/dbtype Phase 0 (SQLite only)
    │   ├── schema.ts     # createSelectSchema, createInsertSchema, createUpdateSchema
    │   ├── column.ts     # Column→TypeBox mappings (SQLite-only dispatch)
    │   ├── types.ts      # Public TypeScript interfaces (merged from schema.types + schema.types.internal)
    │   ├── constants.ts  # Integer range constants
    │   └── utils.ts      # isColumnType, isWithEnum, type helpers
    └── tables/           # Restructured into subdirectories
        ├── common.ts     # shared column definitions, ACTOR_TYPE removed
        ├── metagraph/    # graph_types, node_types, edge_types, graphs, nodes, edges
        │   └── index.ts
        └── identity/     # NEW: accounts, organizations, org_members, api_keys, audit_logs
            └── index.ts

Removed: src/pg/, actors table, @alkdev/drizzlebox external dep, @libsql/client dep.

New deps: @alkdev/pubsub (peer dep for TypedEventTarget type only), honker (replaces @libsql/client).

1. Fold dbtype Phase 0 → src/sqlite/utils/

What to fold

Only SQLite-relevant code from @alkdev/dbtype/src/:

Source (dbtype) Target (storage) Changes
schema.ts utils/schema.ts Remove PgEnum import/handling, remove isPgEnum calls. Remove handleEnum(). Keep createSelectSchema, createInsertSchema, createUpdateSchema.
column.ts utils/column.ts Strip all non-SQLite dispatch branches. Remove PG, MySQL, SingleStore type imports and isColumnType branches. Keep: SQLiteInteger, SQLiteReal, SQLiteText handling. Keep generic dispatch (dataType-based) as fallback. Keep literalSchema, jsonSchema, bufferSchema, mapEnumValues.
column.types.ts utils/types.ts (merged) Strip non-SQLite type branches. Keep SQLiteIntegert.TInteger. Keep generic fallback types.
schema.types.ts utils/types.ts (merged) Remove PgEnum overloads.
schema.types.internal.ts utils/types.ts (merged) Keep Conditions, BuildRefine, BuildSchema, NoUnknownKeys.
constants.ts utils/constants.ts Keep as-is (ranges used by SQLite integer/real/bigiint mappings).
utils.ts utils/utils.ts Remove PgEnum type alias. Keep isColumnType, isWithEnum. Keep JsonSchema, BufferSchema types. Remove unused type helpers.

What NOT to fold

  • PG/MySQL/SingleStore column type handlers (dead code for SQLite-only)
  • isPgEnum and handleEnum (PostgreSQL-specific)
  • createSchemaFactory (no known consumer in storage; can add later if needed)
  • The scripts/probe-e2e.ts (dbtype Phase 1 POC, not relevant)

Import changes in existing table files

Every table file currently imports from @alkdev/drizzlebox:

// Before
import { createInsertSchema, createSelectSchema } from "@alkdev/drizzlebox";

// After
import { createInsertSchema, createSelectSchema } from "../utils/schema.ts";

The createSelectSchema and createInsertSchema functions produce the same TypeBox schemas — the API surface is identical, the implementation just excludes non-SQLite branches.

Deno.json import map changes

// Before
"imports": {
  "@alkdev/drizzlebox": "npm:@alkdev/drizzlebox",
  "@libsql/client": "npm:@libsql/client",
  // ...
}

// After (remove drizzlebox and libsql, add honker and pubsub peer)
"imports": {
  "honker": "npm:honker",
  "@alkdev/pubsub": "npm:@alkdev/pubsub",
  // ...
}

Note: @alkdev/pubsub is a peer dependency for the TypedEventTarget type. It is only needed by consumers who use the HonkerEventTarget. The src/graphs/ module remains zero-dep.

2. Drizzle-Honker Session Adapter (src/sqlite/adapter.ts)

Problem

Drizzle ORM expects a session that implements SQLiteSession<'sync'>. Honker provides Database.query(sql, params) for reads and Transaction.execute(sql, params) for writes. These don't implement Drizzle's session interface.

Adapter Architecture

Consumer code
  ├── db.select().from(graphs)...           → Drizzle query builder
  │     └── HonkerPreparedQuery.run/all/get/values
  │           └── honkerDb.query(sql, params) / tx.execute(sql, params)
  ├── db.$client.notify('channel', p)       → Honker ephemeral pub/sub
  ├── db.$client.stream('events')           → Honker durable stream
  ├── db.$client.queue('jobs')              → Honker work queue
  └── db.$client.listen('channel')          → Honker listener

Three adapter classes:

Class Extends Purpose
HonkerSQLiteSession SQLiteSession<'sync'> Routes Drizzle queries to honkerDb.query() / tx.execute()
HonkerPreparedQuery SQLitePreparedQuery Implements run(), all(), get(), values() via Honker
HonkerSQLiteTransaction SQLiteTransaction<'sync'> Drizzle transaction with $honkerTx for Honker primitives

Accessing Honker from Drizzle

  • db.$client — the Honker Database instance (outside transactions)
  • tx.$honkerTx — the Honker Transaction instance (inside Drizzle tx callbacks)
db.transaction((tx) => {
  tx.insert(schema.nodes).values({ graphId, key: "call-1", attributes: {} }).run();
  tx.$honkerTx.notify("nodes:created", { graphId, key: "call-1" });
});

Both data write and notification commit atomically.

Known challenges (need POC validation)

  1. No lastInsertRowid from Honker execute() — Drizzle's run() needs lastInsertRowid for auto-increment inserts. Honker's JS API doesn't expose it. POC 1 confirmed workaround: tx.query("SELECT last_insert_rowid() as id") returns the rowid.

  2. Object rows only — Honker query() returns Array<Record<string, any>>. Drizzle's values() mode expects raw arrays. The adapter must convert via Object.values() with column order from the query metadata.

  3. Drizzle internal API dependency — The adapter imports from drizzle-orm/sqlite-core/session, which is not a guaranteed-stable API. POC 1 confirmed all needed classes are accessible and extendable.

  4. Session constructor shape — POC 1 confirmed: SQLiteSession takes (mode, executeMethod, query, cache?, queryMetadata?). BaseSQLiteDatabase is accessible from drizzle-orm/sqlite-core/db. The LibSQLSession in drizzle-orm/libsql/session is a working reference implementation.

  5. :memory: databases don't work — Honker's reader pool creates separate in-memory databases. File-based paths must be used. POC 1 confirmed this. Not a real limitation — production uses file paths, tests use temp files.

3. HonkerEventTarget (src/sqlite/event-target.ts)

Interface

Implements @alkdev/pubsub's TypedEventTarget<TEvent>:

interface HonkerEventTargetOptions {
  db: Database;
  mode: "ephemeral" | "durable";
  streamName?: string;
  prefix?: string;
}

function createHonkerEventTarget<TEvent>(options: HonkerEventTargetOptions): HonkerEventTarget<TEvent>

Ephemeral mode mapping

Maps to Honker's notify() / listen():

addEventListener(type, callback):
  if (!listeners.has(type)):
    honkerListener = db.listen(type)
    // start async loop: for await (notification of honkerListener)
    //   dispatch CustomEvent with notification as detail
  listeners.set(type, callback)

dispatchEvent(event):
  db.notify(event.type, event.detail)  // or tx.notify() inside transaction

removeEventListener(type, callback):
  listeners.delete(type, callback)
  if (no more listeners for type):
    honkerListener.close()

Durable mode mapping

Maps to Honker's Stream.publish() / Stream.subscribe():

addEventListener(type, callback):
  if (!subscriptions.has(type)):
    stream = db.stream(streamName ?? type)
    subscription = stream.subscribe(consumerName)
    // start async loop: for await (event of subscription)
    //   dispatch CustomEvent with event.payload as detail
  subscriptions.set(type, { stream, subscription, callback })

dispatchEvent(event):
  db.stream(streamName).publish(event.detail)
  // or stream.publishTx(tx) inside transaction

removeEventListener(type, callback):
  subscriptions.delete(type, callback)
  if (no more subscriptions for type):
    subscription.close()

Topic routing considerations

pubsub uses type:id composite topics (e.g., "call.responded:req-123"). Honker channels are flat strings. The mapping is 1:1 — no transformation needed. But:

  • Ephemeral: Each unique type:id gets its own db.listen() call. For high-cardinality topics (many request IDs), this could mean many concurrent listeners. POC 3 confirmed multiple concurrent listeners work.

  • Durable: Streams are name-keyed, not topic-keyed. A single stream (e.g., "call-protocol") carries all call events. The adapter would need to handle topic filtering client-side (dispatch only to matching listeners) rather than using separate streams per topic.

  • Suggested split: Call protocol events → durable stream. Cache invalidation, UI push → ephemeral channels. The configuration for which topics go where is a consumer decision, not built into the adapter.

  • Latency consideration: POC 2 measured ~17ms median for notifylisten round-trip. For hot-path call protocol request/response in a single process, pair the Honker event target with an in-process EventTarget (pubsub's default) for sub-ms latency. The Honker event target provides durability and cross-process coordination; the default EventTarget provides speed. A composite pattern (dispatch to both) could serve both needs.

Consumer name strategy

For durable mode, each consumer needs a unique name for offset tracking. Suggested pattern: {processId}:{topic} or a configurable consumer group ID.

4. Client Factory Update (src/sqlite/client.ts)

Before

import { drizzle, type LibSQLDatabase } from "drizzle-orm/libsql";
import type { Client } from "@libsql/client";

export function createSqliteDatabase(client: Client): LibSQLDatabase<SqliteSchema> {
  return drizzle(client, { schema });
}

After

The factory accepts a pre-created Honker Database instance (injectable client pattern, per ADR-004). It constructs the Drizzle-Honker adapter internally.

import type { Database as HonkerDatabase } from "honker";

export type SqliteDatabase = BaseSQLiteDatabase<'sync', SqliteSchema>;

export function createSystemDatabase(client: HonkerDatabase): SqliteDatabase {
  return createDrizzleWithAdapter(client, { schema });
}

export function createTenantDatabase(client: HonkerDatabase): SqliteDatabase {
  return createDrizzleWithAdapter(client, { schema });
}

createDrizzleWithAdapter is the internal factory that constructs the HonkerSQLiteSession and wraps it in a BaseSQLiteDatabase. This may require using drizzle-orm/sqlite-core internals rather than the drizzle-orm/libsql high-level wrapper.

5. Table Restructure

Remove actors table

Per existing ADRs (035, 041), actors are being replaced by ACL nodes in graph instances. Remove src/sqlite/tables/actors.ts and all exports.

Reorganize into subdirectories

src/sqlite/tables/
├── common.ts              # commonCols (UPDATED: remove ACTOR_TYPE)
├── metagraph/
│   ├── index.ts            # barrel
│   ├── graph-types.ts      # renamed from graphTypes.ts
│   ├── node-types.ts       # renamed from nodeTypes.ts
│   ├── edge-types.ts       # renamed from edgeTypes.ts
│   ├── graphs.ts
│   ├── nodes.ts
│   └── edges.ts
└── identity/
    ├── index.ts            # barrel
    ├── accounts.ts
    ├── organizations.ts
    ├── org-members.ts
    ├── api-keys.ts
    └── audit-logs.ts

Import updates in table files

All table files currently importing @alkdev/drizzlebox update to ../utils/schema.ts (or ../../utils/schema.ts depending on depth).

The commonCols import path changes from ./common.ts to ../common.ts for metagraph/identity subdirectories.

6. Domain-Specific Tables (Future, documented for scope)

The current metagraph tables store node/edge attributes as JSON. For known graph types (CallGraph, SecretGraph, AclGraph), native columns give better type safety and query performance.

This is not part of the initial pivot. But the pivot should not prevent it. The folded utils/ code already provides createInsertSchema / createSelectSchema which derives TypeBox schemas from Drizzle tables with native columns. When domain tables are added later, they follow the same pattern as the metagraph tables.

Where OperationSpecs fit

Domain tables would come with auto-generated OperationSpec[] — flat arrays describing CRUD operations for each table. The hub/spoke imports these specs, registers them in the operations registry, and the call protocol handles execution.

This means storage's "repository layer" is not hand-written query functions. It's table definitions + operation contract definitions. The consumer (hub/spoke) provides the execution layer.

// In storage: table definition + operation contracts
export const callNodes = sqliteTable("call_nodes", { ... });
export const callNodeOperations: OperationSpec[] = [
  { name: "create", namespace: "call_nodes", type: "mutation", ... },
  { name: "find",   namespace: "call_nodes", type: "query",     ... },
  // ...
];

// In hub: register operations
for (const spec of callNodeOperations) {
  registry.registerSpec(spec);
  registry.registerHandler(specId, (input) => db.insert(callNodes).values(input).returning());
}

The OperationSpec type is straightforward (@alkdev/operations peer dep for type only). Storage defines the contract; the hub provides the handler.

7. Identity Tables (New, documented for scope)

Per sqlite-host.md and ADR-040/041, these live in the system database:

Table Key Columns Purpose
accounts id, email, name, accessLevel User accounts
organizations id, slug, name, ownerId Tenant organizations
org_members id, orgId, accountId, level Membership junction
api_keys id, keyHash, ownerId, name API key auth
audit_logs id, actorType, actorId, action, resource Append-only audit

These need actual implementation (currently only spec in architecture docs). They're part of the pivot scope but can be implemented after the adapter and utils fold.

8. Event Pipeline: Operations → Flowgraph → Storage

How the pieces connect after the pivot

[operations: OperationRegistry]
  │
  │ call.requested / call.responded / call.completed / call.aborted / call.error
  ▼
[pubsub: createPubSub({ eventTarget: honkerEventTarget })]
  │
  ├──→ [flowgraph: WorkflowReactiveRoot.append(event)]
  │        └── signal propagation → UI updates
  │
  ├──→ [storage: db.stream("call-protocol").publish(event)]
  │        └── durable replay for crash recovery
  │
  └──→ [spoke sync: via WebSocket or Honker fan-out]
           └── cross-process coordination

The HonkerEventTarget is the single integration point that makes this work. Operations doesn't need to know about Honker. Flowgraph doesn't need to know about Honker. They both use @alkdev/pubsub's TypedEventTarget. The HonkerEventTarget is just another adapter in pubsub's existing portfolio (alongside Redis, WebSocket, Worker).

Hub-spoke event routing

Hub (system.db + tenant-{orgId}.db)
  ├── HonkerEventTarget (durable: call-protocol stream)
  ├── WebSocketServerEventTarget (spoke fan-out)
  └── In-process EventTarget (local subscribers)

Spoke (tenant-{orgId}.db)
  ├── HonkerEventTarget (ephemeral: local channels)
  ├── WebSocketClientEventTarget (hub connection)
  └── In-process EventTarget (local subscribers)

Both hub and spoke use the same code. Same createPubSub({ eventTarget }). Different event target instance determines the routing.

POC Plan

Before committing to the full pivot, validate the critical path with targeted POCs. Each POC answers one specific hard question.

POC 1: Drizzle queries through Honker

Question: Can we construct a BaseSQLiteDatabase that routes queries through Honker's Database.query() and Transaction.execute()?

Approach:

  1. Import SQLiteSession, SQLitePreparedQuery, SQLiteTransaction from drizzle-orm/sqlite-core/session.
  2. Implement HonkerSQLiteSession that delegates to Database.query().
  3. Implement HonkerPreparedQuery with run(), all(), get().
  4. Wire it into a BaseSQLiteDatabase instance.
  5. Run a simple db.select().from(graphs) against a Honker-opened .db file.

Hard fail conditions:

  • Drizzle's SQLiteSession requires constructor arguments we can't provide from Honker's primitives.
  • The BaseSQLiteDatabase constructor requires internals not exported from drizzle-orm/sqlite-core.
  • lastInsertRowid is required and no workaround exists.

POC 2: Honker pub/sub from same process

Question: Does db.notify()db.listen() work within the same Node.js process? What's the latency?

Approach:

  1. Open a Honker database.
  2. Start db.listen("test-channel").
  3. Call db.notify("test-channel", { hello: "world" }).
  4. Verify the listener receives the notification.
  5. Measure round-trip latency (10,000 iterations).

Hard fail conditions:

  • Same-process notify/listen doesn't work (would require two DB instances or fallback to in-process EventTarget).
  • Latency exceeds 10ms median (would be too slow for call protocol request/response).

POC 3: Concurrent listeners and stream subscriptions

Question: Can multiple Listener and StreamSubscription instances coexist on the same Database? Do they interleave correctly?

Approach:

  1. Open a Honker database.
  2. Start two db.listen() calls on different channels.
  3. Start a db.stream("test-stream").subscribe("consumer-1").
  4. Publish to both channels and the stream concurrently.
  5. Verify all three consumers receive their events without blocking each other.

Hard fail conditions:

  • Only one Listener can be active at a time (would require serialization).
  • Listener.next() blocks the JS event loop (would prevent concurrent subscriptions).

POC 4: Transactional notify + data write

Question: Does tx.notify() + tx.execute() in the same transaction atomically commit both the data write and the notification?

Approach:

  1. Open a Honker database.
  2. Start db.listen("test-channel").
  3. Begin a transaction: const tx = db.transaction().
  4. Execute a data write: tx.execute("INSERT INTO test ...").
  5. Call tx.notify("test-channel", { inserted: true }).
  6. Rollback the transaction.
  7. Verify the listener does NOT receive the notification.
  8. Repeat with commit instead of rollback.
  9. Verify the listener DOES receive the notification after commit.

Hard fail conditions:

  • Notifications fire immediately on tx.notify() regardless of commit/rollback (breaks transactional outbox semantics).

POC file structure

poc/
├── README.md                      # POC objectives, how to run
├── 01-drizzle-honker-adapter.ts   # POC 1
├── 02-same-process-pubsub.ts      # POC 2
├── 03-concurrent-listeners.ts     # POC 3
├── 04-transactional-notify.ts     # POC 4
└── shared/
    └── honker-helpers.ts           # Shared setup/teardown utilities

Each POC is a standalone script that can be run with deno run or node. The POCs use the actual honker npm package from the workspace.

ADR Updates Required

ADR Action Reason
ADR-018 (dbtype integration post-v1) Revise No longer deferred — Phase 0 folds in now as src/sqlite/utils/
ADR-038 (SQLite-first, PG removed) Confirm Pivot confirms this. Remove src/pg/ and PG imports from deno.json.
ADR-039 (Honker as SQLite extension) Confirm Pivot is the implementation of this decision.
ADR-044 (Drizzle-Honker adapter) Update Add specifics about session/PreparedQuery/Transaction classes.
ADR-005 (drizzle + typebox via drizzlebox) Revise Drizzlebox is no longer an external dep. It's src/sqlite/utils/.
ADR-019 (JSON text for schema columns) Confirm Still relevant for metagraph attributes. Domain tables add native columns alongside JSON.
ADR-033 (JSON path queries for v1) Revise Repository layer is now "operation specs," not hand-written CRUD. The query approach (JSON path vs native) still applies, but the output format changes.
New ADR Fold dbtype Phase 0 as utils Records the decision to fold SQLite mappings into src/sqlite/utils/.
New ADR HonkerEventTarget adapter Records the decision to implement pubsub TypedEventTarget on Honker primitives.
New ADR Operation specs as repository surface Records that storage outputs OperationSpec[], not hand-written query functions.

Architecture Doc Updates Required

Document Changes
overview.md Subpath exports (./pg removed), dependency list, module structure, "what's not done" section, system/tenant DB model
forward-look.md dbtype section (folded, not deferred), repository layer strategy (operations, not CRUD), pointer abstraction (still deferred)
honker-integration.md Add HonkerEventTarget section, update adapter section with verified API, add event pipeline diagram, resolve OQ-26 (Honker replaces Redis for single-node)
sqlite-host.md Table structure changes (subdirectories, actors removed, identity tables), client factory signature, Honker import instead of libsql
metagraph-module.md No changes (Module format is independent of storage host)
schema-evolution.md Minor: createSelectSchema/createInsertSchema now come from local utils, not external dep
open-questions.md Resolve OQ-17 (operations, not CRUD), OQ-18 (auto-generated specs from tables), OQ-26 (Honker replaces Redis). Update OQ-19 (bridge package location → it's storage itself).
encrypted-data.md No changes

Implementation Order (post-POC)

If POCs pass, the implementation sequence should be:

  1. Fold dbtype → src/sqlite/utils/ — import path changes only, no behavioral changes. Run existing tests to verify.
  2. Remove src/pg/ and actors table — clean up dead code. Update deno.json imports, remove postgres and PG-related entries.
  3. Restructure tables into subdirectories — move files, update relative imports, update schema.ts and relations.ts.
  4. Implement adapter.ts — Drizzle-Honker session adapter based on POC 1 findings.
  5. Update client.tscreateSystemDatabase() / createTenantDatabase() taking Honker Database instances.
  6. Implement event-target.tsHonkerEventTarget based on POC 2-4 findings.
  7. Implement identity tablessrc/sqlite/tables/identity/.
  8. Update architecture docs — bring all documents in line with the new structure.
  9. Add domain tables (future, not in initial pivot) — native-column tables for CallGraph, SecretGraph, etc. with OperationSpec[] output.

Steps 1-3 are mechanical (no new behavior). Steps 4-5 depend on POC 1. Step 6 depends on POCs 2-4. Step 7 can proceed independently. Steps 8-9 are follow-ups.

POC Results (2026-06-01)

POC 1: Drizzle queries through Honker — PASS

  • SQLiteSession, SQLitePreparedQuery, SQLiteTransaction are all accessible from drizzle-orm/sqlite-core/session — can be extended.
  • BaseSQLiteDatabase is accessible from drizzle-orm/sqlite-core/db.
  • Honker query() returns { columnName: value } object rows, compatible with Drizzle's mapResultRow().
  • last_insert_rowid() accessible via tx.query("SELECT last_insert_rowid() as id").
  • tx.execute() returns a number (affected rows count for UPDATE/DELETE).
  • JSON-mode columns need manual JSON.parse() in the adapter.
  • :memory: databases DON'T work — reader pool gets separate in-memory DBs. Always use file-based paths (or :memory: with a single reader, which defeats Honker's concurrent reader design).
  • LibSQLSession in drizzle-orm/libsql/session is a good reference implementation for the adapter pattern.

Session method signatures discovered:

SQLiteSession methods:
  prepareOneTimeQuery  — key method to implement
  all, run, get, values — delegated to PreparedQuery
  extractRaw*ValueFromBatchResult — batch result extraction
  count — cursor count

SQLitePreparedQuery methods:
  execute, getQuery, mapAllResult, mapGetResult, mapResult, mapRunResult
  queryWithCache

SQLiteTransaction methods:
  rollback — only explicit method (commit handled by Session.transaction)

POC 2: Same-process pub/sub — PASS (with latency caveat)

  • db.notify()db.listen() works within the same process.
  • Multiple concurrent listeners on different channels work.
  • Cross-channel isolation works — listeners only receive notifications on their subscribed channel.
  • Latency: ~17ms median, ~29ms P99 (100 samples). This is higher than ideal for call protocol request/response. The Listener uses a polling fallback (updateEvents().next()) internally. For hot-path events where sub-ms latency matters, the in-process EventTarget should be preferred. Honker listeners are better suited for cross-process and durable scenarios.

POC 3: Concurrent listeners and streams — PASS

  • Multiple Listener instances coexist on the same Database.
  • Stream.publish() / Stream.subscribe() works.
  • readSince(offset, limit) returns historical events.
  • StreamSubscription implements AsyncIterableIterator — works with for await...of.
  • Consumer offsets are tracked and persisted.
  • Concurrent stream + listener operation works.
  • publishTx(tx, payload) works within a transaction.
  • Caveat: db.close() doesn't cleanly exit when listeners are still polling. Need to close all listeners before closing the DB.

POC 4: Transactional notify + data write — PASS (atomicity confirmed)

  • tx.notify() + tx.execute() — notification fires ONLY after tx.commit(). Before commit, no notification is received. Transactional semantics hold.
  • On tx.rollback() — notification is NOT sent. Data is NOT persisted. Atomicity holds for both data and events.
  • db.notifyTx(tx, channel, payload) — convenience wrapper, same semantics.
  • queue.enqueueTx(tx, payload) — job is only visible after commit. On rollback, job does not appear in the queue. Transactional outbox works.
  • Read-your-writes: after receiving a notification, the data IS queryable (no stale read issue with single DB).

This is the most important result. Honker provides true transactional outbox semantics — data writes and event/queue publishes commit atomically. This eliminates the dual-write problem that necessitated Redis for PostgreSQL.

Open Research Questions

  1. Drizzle session construction — RESOLVED by POC 1. All needed classes are accessible and extendable. LibSQLSession is the reference.

  2. Honker listener concurrency model — RESOLVED by POC 3. Multiple listeners coexist. The updateEvents mechanism works for concurrent polling. db.close() requires all listeners to be closed first.

  3. Consumer name management for streams — When a hub restarts, durable stream consumers resume from their last saved offset. But the consumer name must be stable across restarts. Need a naming convention that's deterministic (e.g., {service}:{pid} is not restart-stable).

  4. Schema migrations — Honker opens a single .db file. Drizzle Kit supports SQLite migrations but expects better-sqlite3 or libsql. Need to verify that drizzle-kit push or drizzle-kit generate works with the Honker adapter, or whether we need a custom migration runner.

  5. OperationSpec generation from tables — What's the concrete mapping? For each table: one create mutation, one find query, one list query, one update mutation, one delete mutation? Or something more domain-specific? This is a design question, not a POC question.

  6. In-process + Honker composite event target — POC 2 showed ~17ms latency for Honker notify→listen. For single-process hot paths, an in-process EventTarget is sub-ms. A composite that dispatches to both (in-process for speed, Honker for durability/cross-process) would be the ideal default for a hub. Needs design work.