storage/docs/research/pivot-honker-sqlite-adapter.md

---
status: draft
created: 2026-06-01
last_updated: 2026-06-01
---

# Pivot: Honker SQLite Adapter, dbtype Fold, and Event Target Integration

Research document for the architecture pivot from `@libsql/client` to Honker,
folding `@alkdev/dbtype` Phase 0 into `src/utils/`, and adding a pubsub
`HonkerEventTarget`.

## Motivation

1. Honker provides SQLite + pub/sub + streams + queues + locks + scheduler in
   a single `.db` file via a Rust NAPI binding. This replaces both
   `@libsql/client` (SQLite only) and the conceptual need for Redis.
2. `@alkdev/dbtype` Phase 0 is already consumed as `@alkdev/drizzlebox`. With
   SQLite as the sole target, the multi-dialect column mappings are dead
   weight. Folding the SQLite-relevant subset into `src/utils/` eliminates an
   external dependency and co-locates utility code with the tables it
   derives schemas from.
3. `@alkdev/pubsub` defines a `TypedEventTarget` interface that Honker's
   primitives (`notify`/`listen`, `stream`/`stream.subscribe`) map to
   naturally. A `HonkerEventTarget` adapter lets storage's downstream
   consumers use the same pubsub API regardless of whether events route
   in-process, through Honker, through Redis, or over WebSocket.
4. The hub-spoke model needs events for call protocol routing, cache
   invalidation, flowgraph reactive updates, and cross-process coordination.
   Honker handles all of these within SQLite.

## Architecture Changes

### Current Structure

```
src/
├── graphs/           # Metagraph Module + bridge (no db deps)
│   ├── modules/      # CallGraph, SecretGraph, Metagraph
│   ├── bridge.ts     # moduleToDbSchema, validateNode, validateEdge
│   ├── crypto.ts     # encrypt, decrypt, generateEncryptionKey
│   └── mod.ts
├── pg/               # Placeholder (to be removed)
└── sqlite/           # SQLite host
    ├── client.ts     # createSqliteDatabase(@libsql/client)
    ├── mod.ts
    ├── relations.ts
    ├── schema.ts
    └── tables/       # Flat: graphTypes, nodeTypes, edgeTypes, graphs, nodes, edges, actors
```

Dependencies: `@alkdev/drizzlebox` (external npm), `@libsql/client`, `drizzle-orm`

### Target Structure

```
src/
├── graphs/               # Unchanged
│   ├── modules/
│   ├── bridge.ts
│   ├── crypto.ts
│   └── mod.ts
└── sqlite/
    ├── adapter.ts        # NEW: Drizzle-Honker session adapter
    ├── event-target.ts   # NEW: HonkerEventTarget implements pubsub TypedEventTarget
    ├── client.ts         # UPDATED: createSystemDatabase(client), createTenantDatabase(client)
    ├── mod.ts            # UPDATED: re-exports
    ├── relations.ts      # UPDATED: remove actors relations
    ├── schema.ts         # UPDATED: remove actors, remove pg refs
    ├── utils/            # NEW: folded from @alkdev/dbtype Phase 0 (SQLite only)
    │   ├── schema.ts     # createSelectSchema, createInsertSchema, createUpdateSchema
    │   ├── column.ts     # Column→TypeBox mappings (SQLite-only dispatch)
    │   ├── types.ts      # Public TypeScript interfaces (merged from schema.types + schema.types.internal)
    │   ├── constants.ts  # Integer range constants
    │   └── utils.ts      # isColumnType, isWithEnum, type helpers
    └── tables/           # Restructured into subdirectories
        ├── common.ts     # shared column definitions, ACTOR_TYPE removed
        ├── metagraph/    # graph_types, node_types, edge_types, graphs, nodes, edges
        │   └── index.ts
        └── identity/     # NEW: accounts, organizations, org_members, api_keys, audit_logs
            └── index.ts
```

Removed: `src/pg/`, `actors` table, `@alkdev/drizzlebox` external dep,
`@libsql/client` dep.

New deps: `@alkdev/pubsub` (peer dep for `TypedEventTarget` type only),
`honker` (replaces `@libsql/client`).

## 1. Fold dbtype Phase 0 → `src/sqlite/utils/`

### What to fold

Only SQLite-relevant code from `@alkdev/dbtype/src/`:

| Source (dbtype) | Target (storage) | Changes |
|-----------------|-------------------|---------|
| `schema.ts` | `utils/schema.ts` | Remove `PgEnum` import/handling, remove `isPgEnum` calls. Remove `handleEnum()`. Keep `createSelectSchema`, `createInsertSchema`, `createUpdateSchema`. |
| `column.ts` | `utils/column.ts` | Strip all non-SQLite dispatch branches. Remove PG, MySQL, SingleStore type imports and `isColumnType` branches. Keep: `SQLiteInteger`, `SQLiteReal`, `SQLiteText` handling. Keep generic dispatch (`dataType`-based) as fallback. Keep `literalSchema`, `jsonSchema`, `bufferSchema`, `mapEnumValues`. |
| `column.types.ts` | `utils/types.ts` (merged) | Strip non-SQLite type branches. Keep `SQLiteInteger` → `t.TInteger`. Keep generic fallback types. |
| `schema.types.ts` | `utils/types.ts` (merged) | Remove `PgEnum` overloads. |
| `schema.types.internal.ts` | `utils/types.ts` (merged) | Keep `Conditions`, `BuildRefine`, `BuildSchema`, `NoUnknownKeys`. |
| `constants.ts` | `utils/constants.ts` | Keep as-is (ranges used by SQLite integer/real/bigiint mappings). |
| `utils.ts` | `utils/utils.ts` | Remove `PgEnum` type alias. Keep `isColumnType`, `isWithEnum`. Keep `JsonSchema`, `BufferSchema` types. Remove unused type helpers. |

### What NOT to fold

- PG/MySQL/SingleStore column type handlers (dead code for SQLite-only)
- `isPgEnum` and `handleEnum` (PostgreSQL-specific)
- `createSchemaFactory` (no known consumer in storage; can add later if needed)
- The `scripts/probe-e2e.ts` (dbtype Phase 1 POC, not relevant)

### Import changes in existing table files

Every table file currently imports from `@alkdev/drizzlebox`:

```ts
// Before
import { createInsertSchema, createSelectSchema } from "@alkdev/drizzlebox";

// After
import { createInsertSchema, createSelectSchema } from "../utils/schema.ts";
```

The `createSelectSchema` and `createInsertSchema` functions produce the same
TypeBox schemas — the API surface is identical, the implementation just
excludes non-SQLite branches.

### Deno.json import map changes

```jsonc
// Before
"imports": {
  "@alkdev/drizzlebox": "npm:@alkdev/drizzlebox",
  "@libsql/client": "npm:@libsql/client",
  // ...
}

// After (remove drizzlebox and libsql, add honker and pubsub peer)
"imports": {
  "honker": "npm:honker",
  "@alkdev/pubsub": "npm:@alkdev/pubsub",
  // ...
}
```

Note: `@alkdev/pubsub` is a peer dependency for the `TypedEventTarget` type.
It is only needed by consumers who use the `HonkerEventTarget`. The
`src/graphs/` module remains zero-dep.

## 2. Drizzle-Honker Session Adapter (`src/sqlite/adapter.ts`)

### Problem

Drizzle ORM expects a session that implements `SQLiteSession<'sync'>`. Honker
provides `Database.query(sql, params)` for reads and `Transaction.execute(sql,
params)` for writes. These don't implement Drizzle's session interface.

### Adapter Architecture

```
Consumer code
  ├── db.select().from(graphs)...           → Drizzle query builder
  │     └── HonkerPreparedQuery.run/all/get/values
  │           └── honkerDb.query(sql, params) / tx.execute(sql, params)
  ├── db.$client.notify('channel', p)       → Honker ephemeral pub/sub
  ├── db.$client.stream('events')           → Honker durable stream
  ├── db.$client.queue('jobs')              → Honker work queue
  └── db.$client.listen('channel')          → Honker listener
```

Three adapter classes:

| Class | Extends | Purpose |
|-------|---------|---------|
| `HonkerSQLiteSession` | `SQLiteSession<'sync'>` | Routes Drizzle queries to `honkerDb.query()` / `tx.execute()` |
| `HonkerPreparedQuery` | `SQLitePreparedQuery` | Implements `run()`, `all()`, `get()`, `values()` via Honker |
| `HonkerSQLiteTransaction` | `SQLiteTransaction<'sync'>` | Drizzle transaction with `$honkerTx` for Honker primitives |

### Accessing Honker from Drizzle

- `db.$client` — the Honker `Database` instance (outside transactions)
- `tx.$honkerTx` — the Honker `Transaction` instance (inside Drizzle tx callbacks)

```ts
db.transaction((tx) => {
  tx.insert(schema.nodes).values({ graphId, key: "call-1", attributes: {} }).run();
  tx.$honkerTx.notify("nodes:created", { graphId, key: "call-1" });
});
```

Both data write and notification commit atomically.

### Known challenges (need POC validation)

1. **No `lastInsertRowid` from Honker `execute()`** — Drizzle's `run()`
   needs `lastInsertRowid` for auto-increment inserts. Honker's JS API
   doesn't expose it. POC 1 confirmed workaround: `tx.query("SELECT last_insert_rowid() as id")` returns the rowid.

2. **Object rows only** — Honker `query()` returns
   `Array<Record<string, any>>`. Drizzle's `values()` mode expects raw
   arrays. The adapter must convert via `Object.values()` with column
   order from the query metadata.

3. **Drizzle internal API dependency** — The adapter imports from
   `drizzle-orm/sqlite-core/session`, which is not a guaranteed-stable API.
   POC 1 confirmed all needed classes are accessible and extendable.

4. **Session constructor shape** — POC 1 confirmed: `SQLiteSession` takes
   `(mode, executeMethod, query, cache?, queryMetadata?)`. `BaseSQLiteDatabase`
   is accessible from `drizzle-orm/sqlite-core/db`. The `LibSQLSession` in
   `drizzle-orm/libsql/session` is a working reference implementation.

5. **`:memory:` databases don't work** — Honker's reader pool creates
   separate in-memory databases. File-based paths must be used. POC 1
   confirmed this. Not a real limitation — production uses file paths,
   tests use temp files.

## 3. HonkerEventTarget (`src/sqlite/event-target.ts`)

### Interface

Implements `@alkdev/pubsub`'s `TypedEventTarget<TEvent>`:

```ts
interface HonkerEventTargetOptions {
  db: Database;
  mode: "ephemeral" | "durable";
  streamName?: string;
  prefix?: string;
}

function createHonkerEventTarget<TEvent>(options: HonkerEventTargetOptions): HonkerEventTarget<TEvent>
```

### Ephemeral mode mapping

Maps to Honker's `notify()` / `listen()`:

```ts
addEventListener(type, callback):
  if (!listeners.has(type)):
    honkerListener = db.listen(type)
    // start async loop: for await (notification of honkerListener)
    //   dispatch CustomEvent with notification as detail
  listeners.set(type, callback)

dispatchEvent(event):
  db.notify(event.type, event.detail)  // or tx.notify() inside transaction

removeEventListener(type, callback):
  listeners.delete(type, callback)
  if (no more listeners for type):
    honkerListener.close()
```

### Durable mode mapping

Maps to Honker's `Stream.publish()` / `Stream.subscribe()`:

```ts
addEventListener(type, callback):
  if (!subscriptions.has(type)):
    stream = db.stream(streamName ?? type)
    subscription = stream.subscribe(consumerName)
    // start async loop: for await (event of subscription)
    //   dispatch CustomEvent with event.payload as detail
  subscriptions.set(type, { stream, subscription, callback })

dispatchEvent(event):
  db.stream(streamName).publish(event.detail)
  // or stream.publishTx(tx) inside transaction

removeEventListener(type, callback):
  subscriptions.delete(type, callback)
  if (no more subscriptions for type):
    subscription.close()
```

### Topic routing considerations

pubsub uses `type:id` composite topics (e.g., `"call.responded:req-123"`).
Honker channels are flat strings. The mapping is 1:1 — no transformation
needed. But:

- **Ephemeral**: Each unique `type:id` gets its own `db.listen()` call. For
  high-cardinality topics (many request IDs), this could mean many concurrent
  listeners. POC 3 confirmed multiple concurrent listeners work.

- **Durable**: Streams are name-keyed, not topic-keyed. A single stream
  (e.g., `"call-protocol"`) carries all call events. The adapter would need
  to handle topic filtering client-side (dispatch only to matching listeners)
  rather than using separate streams per topic.

- **Suggested split**: Call protocol events → durable stream. Cache
  invalidation, UI push → ephemeral channels. The configuration for which
  topics go where is a consumer decision, not built into the adapter.

- **Latency consideration**: POC 2 measured ~17ms median for
  `notify`→`listen` round-trip. For hot-path call protocol request/response
  in a single process, pair the Honker event target with an in-process
  `EventTarget` (pubsub's default) for sub-ms latency. The Honker event
  target provides durability and cross-process coordination; the default
  EventTarget provides speed. A composite pattern (dispatch to both) could
  serve both needs.

### Consumer name strategy

For durable mode, each consumer needs a unique name for offset tracking.
Suggested pattern: `{processId}:{topic}` or a configurable consumer group ID.

## 4. Client Factory Update (`src/sqlite/client.ts`)

### Before

```ts
import { drizzle, type LibSQLDatabase } from "drizzle-orm/libsql";
import type { Client } from "@libsql/client";

export function createSqliteDatabase(client: Client): LibSQLDatabase<SqliteSchema> {
  return drizzle(client, { schema });
}
```

### After

The factory accepts a pre-created Honker `Database` instance (injectable
client pattern, per ADR-004). It constructs the Drizzle-Honker adapter
internally.

```ts
import type { Database as HonkerDatabase } from "honker";

export type SqliteDatabase = BaseSQLiteDatabase<'sync', SqliteSchema>;

export function createSystemDatabase(client: HonkerDatabase): SqliteDatabase {
  return createDrizzleWithAdapter(client, { schema });
}

export function createTenantDatabase(client: HonkerDatabase): SqliteDatabase {
  return createDrizzleWithAdapter(client, { schema });
}
```

`createDrizzleWithAdapter` is the internal factory that constructs the
`HonkerSQLiteSession` and wraps it in a `BaseSQLiteDatabase`. This may
require using `drizzle-orm/sqlite-core` internals rather than the
`drizzle-orm/libsql` high-level wrapper.

## 5. Table Restructure

### Remove actors table

Per existing ADRs (035, 041), actors are being replaced by ACL nodes in
graph instances. Remove `src/sqlite/tables/actors.ts` and all exports.

### Reorganize into subdirectories

```
src/sqlite/tables/
├── common.ts              # commonCols (UPDATED: remove ACTOR_TYPE)
├── metagraph/
│   ├── index.ts            # barrel
│   ├── graph-types.ts      # renamed from graphTypes.ts
│   ├── node-types.ts       # renamed from nodeTypes.ts
│   ├── edge-types.ts       # renamed from edgeTypes.ts
│   ├── graphs.ts
│   ├── nodes.ts
│   └── edges.ts
└── identity/
    ├── index.ts            # barrel
    ├── accounts.ts
    ├── organizations.ts
    ├── org-members.ts
    ├── api-keys.ts
    └── audit-logs.ts
```

### Import updates in table files

All table files currently importing `@alkdev/drizzlebox` update to
`../utils/schema.ts` (or `../../utils/schema.ts` depending on depth).

The `commonCols` import path changes from `./common.ts` to `../common.ts`
for metagraph/identity subdirectories.

## 6. Domain-Specific Tables (Future, documented for scope)

The current metagraph tables store node/edge attributes as JSON. For known
graph types (CallGraph, SecretGraph, AclGraph), native columns give better
type safety and query performance.

This is **not part of the initial pivot**. But the pivot should not prevent
it. The folded `utils/` code already provides `createInsertSchema` /
`createSelectSchema` which derives TypeBox schemas from Drizzle tables with
native columns. When domain tables are added later, they follow the same
pattern as the metagraph tables.

### Where OperationSpecs fit

Domain tables would come with auto-generated `OperationSpec[]` — flat arrays
describing CRUD operations for each table. The hub/spoke imports these specs,
registers them in the operations registry, and the call protocol handles
execution.

This means storage's "repository layer" is not hand-written query functions.
It's **table definitions + operation contract definitions**. The consumer
(hub/spoke) provides the execution layer.

```ts
// In storage: table definition + operation contracts
export const callNodes = sqliteTable("call_nodes", { ... });
export const callNodeOperations: OperationSpec[] = [
  { name: "create", namespace: "call_nodes", type: "mutation", ... },
  { name: "find",   namespace: "call_nodes", type: "query",     ... },
  // ...
];

// In hub: register operations
for (const spec of callNodeOperations) {
  registry.registerSpec(spec);
  registry.registerHandler(specId, (input) => db.insert(callNodes).values(input).returning());
}
```

The OperationSpec type is straightforward (`@alkdev/operations` peer dep for
type only). Storage defines the contract; the hub provides the handler.

## 7. Identity Tables (New, documented for scope)

Per `sqlite-host.md` and ADR-040/041, these live in the system database:

| Table | Key Columns | Purpose |
|-------|-------------|---------|
| `accounts` | id, email, name, accessLevel | User accounts |
| `organizations` | id, slug, name, ownerId | Tenant organizations |
| `org_members` | id, orgId, accountId, level | Membership junction |
| `api_keys` | id, keyHash, ownerId, name | API key auth |
| `audit_logs` | id, actorType, actorId, action, resource | Append-only audit |

These need actual implementation (currently only spec in architecture docs).
They're part of the pivot scope but can be implemented after the adapter and
utils fold.

## 8. Event Pipeline: Operations → Flowgraph → Storage

### How the pieces connect after the pivot

```
[operations: OperationRegistry]
  │
  │ call.requested / call.responded / call.completed / call.aborted / call.error
  ▼
[pubsub: createPubSub({ eventTarget: honkerEventTarget })]
  │
  ├──→ [flowgraph: WorkflowReactiveRoot.append(event)]
  │        └── signal propagation → UI updates
  │
  ├──→ [storage: db.stream("call-protocol").publish(event)]
  │        └── durable replay for crash recovery
  │
  └──→ [spoke sync: via WebSocket or Honker fan-out]
           └── cross-process coordination
```

The `HonkerEventTarget` is the single integration point that makes this work.
Operations doesn't need to know about Honker. Flowgraph doesn't need to know
about Honker. They both use `@alkdev/pubsub`'s `TypedEventTarget`. The
`HonkerEventTarget` is just another adapter in pubsub's existing portfolio
(alongside Redis, WebSocket, Worker).

### Hub-spoke event routing

```
Hub (system.db + tenant-{orgId}.db)
  ├── HonkerEventTarget (durable: call-protocol stream)
  ├── WebSocketServerEventTarget (spoke fan-out)
  └── In-process EventTarget (local subscribers)

Spoke (tenant-{orgId}.db)
  ├── HonkerEventTarget (ephemeral: local channels)
  ├── WebSocketClientEventTarget (hub connection)
  └── In-process EventTarget (local subscribers)
```

Both hub and spoke use the same code. Same `createPubSub({ eventTarget })`.
Different event target instance determines the routing.

## POC Plan

Before committing to the full pivot, validate the critical path with targeted
POCs. Each POC answers one specific hard question.

### POC 1: Drizzle queries through Honker

**Question**: Can we construct a `BaseSQLiteDatabase` that routes queries
through Honker's `Database.query()` and `Transaction.execute()`?

**Approach**:
1. Import `SQLiteSession`, `SQLitePreparedQuery`, `SQLiteTransaction` from
   `drizzle-orm/sqlite-core/session`.
2. Implement `HonkerSQLiteSession` that delegates to `Database.query()`.
3. Implement `HonkerPreparedQuery` with `run()`, `all()`, `get()`.
4. Wire it into a `BaseSQLiteDatabase` instance.
5. Run a simple `db.select().from(graphs)` against a Honker-opened `.db` file.

**Hard fail conditions**:
- Drizzle's `SQLiteSession` requires constructor arguments we can't provide
  from Honker's primitives.
- The `BaseSQLiteDatabase` constructor requires internals not exported from
  `drizzle-orm/sqlite-core`.
- `lastInsertRowid` is required and no workaround exists.

### POC 2: Honker pub/sub from same process

**Question**: Does `db.notify()` → `db.listen()` work within the same Node.js
process? What's the latency?

**Approach**:
1. Open a Honker database.
2. Start `db.listen("test-channel")`.
3. Call `db.notify("test-channel", { hello: "world" })`.
4. Verify the listener receives the notification.
5. Measure round-trip latency (10,000 iterations).

**Hard fail conditions**:
- Same-process `notify`/`listen` doesn't work (would require two DB instances
  or fallback to in-process EventTarget).
- Latency exceeds 10ms median (would be too slow for call protocol
  request/response).

### POC 3: Concurrent listeners and stream subscriptions

**Question**: Can multiple `Listener` and `StreamSubscription` instances
coexist on the same `Database`? Do they interleave correctly?

**Approach**:
1. Open a Honker database.
2. Start two `db.listen()` calls on different channels.
3. Start a `db.stream("test-stream").subscribe("consumer-1")`.
4. Publish to both channels and the stream concurrently.
5. Verify all three consumers receive their events without blocking each
   other.

**Hard fail conditions**:
- Only one `Listener` can be active at a time (would require serialization).
- `Listener.next()` blocks the JS event loop (would prevent concurrent
  subscriptions).

### POC 4: Transactional notify + data write

**Question**: Does `tx.notify()` + `tx.execute()` in the same transaction
atomically commit both the data write and the notification?

**Approach**:
1. Open a Honker database.
2. Start `db.listen("test-channel")`.
3. Begin a transaction: `const tx = db.transaction()`.
4. Execute a data write: `tx.execute("INSERT INTO test ...")`.
5. Call `tx.notify("test-channel", { inserted: true })`.
6. Rollback the transaction.
7. Verify the listener does NOT receive the notification.
8. Repeat with commit instead of rollback.
9. Verify the listener DOES receive the notification after commit.

**Hard fail conditions**:
- Notifications fire immediately on `tx.notify()` regardless of commit/rollback
  (breaks transactional outbox semantics).

### POC file structure

```
poc/
├── README.md                      # POC objectives, how to run
├── 01-drizzle-honker-adapter.ts   # POC 1
├── 02-same-process-pubsub.ts      # POC 2
├── 03-concurrent-listeners.ts     # POC 3
├── 04-transactional-notify.ts     # POC 4
└── shared/
    └── honker-helpers.ts           # Shared setup/teardown utilities
```

Each POC is a standalone script that can be run with `deno run` or `node`.
The POCs use the actual `honker` npm package from the workspace.

## ADR Updates Required

| ADR | Action | Reason |
|-----|--------|--------|
| ADR-018 (dbtype integration post-v1) | **Revise** | No longer deferred — Phase 0 folds in now as `src/sqlite/utils/` |
| ADR-038 (SQLite-first, PG removed) | **Confirm** | Pivot confirms this. Remove `src/pg/` and PG imports from deno.json. |
| ADR-039 (Honker as SQLite extension) | **Confirm** | Pivot is the implementation of this decision. |
| ADR-044 (Drizzle-Honker adapter) | **Update** | Add specifics about session/PreparedQuery/Transaction classes. |
| ADR-005 (drizzle + typebox via drizzlebox) | **Revise** | Drizzlebox is no longer an external dep. It's `src/sqlite/utils/`. |
| ADR-019 (JSON text for schema columns) | **Confirm** | Still relevant for metagraph attributes. Domain tables add native columns alongside JSON. |
| ADR-033 (JSON path queries for v1) | **Revise** | Repository layer is now "operation specs," not hand-written CRUD. The query approach (JSON path vs native) still applies, but the output format changes. |
| New ADR | Fold dbtype Phase 0 as utils | Records the decision to fold SQLite mappings into `src/sqlite/utils/`. |
| New ADR | HonkerEventTarget adapter | Records the decision to implement pubsub TypedEventTarget on Honker primitives. |
| New ADR | Operation specs as repository surface | Records that storage outputs OperationSpec[], not hand-written query functions. |

## Architecture Doc Updates Required

| Document | Changes |
|----------|---------|
| `overview.md` | Subpath exports (`./pg` removed), dependency list, module structure, "what's not done" section, system/tenant DB model |
| `forward-look.md` | dbtype section (folded, not deferred), repository layer strategy (operations, not CRUD), pointer abstraction (still deferred) |
| `honker-integration.md` | Add HonkerEventTarget section, update adapter section with verified API, add event pipeline diagram, resolve OQ-26 (Honker replaces Redis for single-node) |
| `sqlite-host.md` | Table structure changes (subdirectories, actors removed, identity tables), client factory signature, Honker import instead of libsql |
| `metagraph-module.md` | No changes (Module format is independent of storage host) |
| `schema-evolution.md` | Minor: `createSelectSchema`/`createInsertSchema` now come from local utils, not external dep |
| `open-questions.md` | Resolve OQ-17 (operations, not CRUD), OQ-18 (auto-generated specs from tables), OQ-26 (Honker replaces Redis). Update OQ-19 (bridge package location → it's storage itself). |
| `encrypted-data.md` | No changes |

## Implementation Order (post-POC)

If POCs pass, the implementation sequence should be:

1. **Fold dbtype → `src/sqlite/utils/`** — import path changes only, no
   behavioral changes. Run existing tests to verify.
2. **Remove `src/pg/` and actors table** — clean up dead code. Update deno.json
   imports, remove `postgres` and PG-related entries.
3. **Restructure tables into subdirectories** — move files, update relative
   imports, update `schema.ts` and `relations.ts`.
4. **Implement `adapter.ts`** — Drizzle-Honker session adapter based on POC
   1 findings.
5. **Update `client.ts`** — `createSystemDatabase()` / `createTenantDatabase()`
   taking Honker `Database` instances.
6. **Implement `event-target.ts`** — `HonkerEventTarget` based on POC 2-4
   findings.
7. **Implement identity tables** — `src/sqlite/tables/identity/`.
8. **Update architecture docs** — bring all documents in line with the new
   structure.
9. **Add domain tables** (future, not in initial pivot) — native-column
   tables for CallGraph, SecretGraph, etc. with OperationSpec[] output.

Steps 1-3 are mechanical (no new behavior). Steps 4-5 depend on POC 1.
Step 6 depends on POCs 2-4. Step 7 can proceed independently. Steps 8-9
are follow-ups.

## POC Results (2026-06-01)

### POC 1: Drizzle queries through Honker — PASS

- `SQLiteSession`, `SQLitePreparedQuery`, `SQLiteTransaction` are all
  accessible from `drizzle-orm/sqlite-core/session` — can be extended.
- `BaseSQLiteDatabase` is accessible from `drizzle-orm/sqlite-core/db`.
- Honker `query()` returns `{ columnName: value }` object rows, compatible
  with Drizzle's `mapResultRow()`.
- `last_insert_rowid()` accessible via `tx.query("SELECT last_insert_rowid() as id")`.
- `tx.execute()` returns a number (affected rows count for UPDATE/DELETE).
- JSON-mode columns need manual `JSON.parse()` in the adapter.
- `:memory:` databases DON'T work — reader pool gets separate in-memory DBs.
  Always use file-based paths (or `:memory:` with a single reader, which
  defeats Honker's concurrent reader design).
- `LibSQLSession` in `drizzle-orm/libsql/session` is a good reference
  implementation for the adapter pattern.

Session method signatures discovered:
```
SQLiteSession methods:
  prepareOneTimeQuery  — key method to implement
  all, run, get, values — delegated to PreparedQuery
  extractRaw*ValueFromBatchResult — batch result extraction
  count — cursor count

SQLitePreparedQuery methods:
  execute, getQuery, mapAllResult, mapGetResult, mapResult, mapRunResult
  queryWithCache

SQLiteTransaction methods:
  rollback — only explicit method (commit handled by Session.transaction)
```

### POC 2: Same-process pub/sub — PASS (with latency caveat)

- `db.notify()` → `db.listen()` works within the same process.
- Multiple concurrent listeners on different channels work.
- Cross-channel isolation works — listeners only receive notifications on
  their subscribed channel.
- **Latency**: ~17ms median, ~29ms P99 (100 samples). This is higher than
  ideal for call protocol request/response. The `Listener` uses a polling
  fallback (`updateEvents().next()`) internally. For hot-path events where
  sub-ms latency matters, the in-process `EventTarget` should be preferred.
  Honker listeners are better suited for cross-process and durable scenarios.

### POC 3: Concurrent listeners and streams — PASS

- Multiple `Listener` instances coexist on the same `Database`.
- `Stream.publish()` / `Stream.subscribe()` works.
- `readSince(offset, limit)` returns historical events.
- `StreamSubscription` implements `AsyncIterableIterator` — works with
  `for await...of`.
- Consumer offsets are tracked and persisted.
- Concurrent stream + listener operation works.
- `publishTx(tx, payload)` works within a transaction.
- **Caveat**: `db.close()` doesn't cleanly exit when listeners are still
  polling. Need to close all listeners before closing the DB.

### POC 4: Transactional notify + data write — PASS (atomicity confirmed)

- `tx.notify()` + `tx.execute()` — notification fires ONLY after `tx.commit()`.
  Before commit, no notification is received. **Transactional semantics hold.**
- On `tx.rollback()` — notification is NOT sent. Data is NOT persisted.
  **Atomicity holds for both data and events.**
- `db.notifyTx(tx, channel, payload)` — convenience wrapper, same semantics.
- `queue.enqueueTx(tx, payload)` — job is only visible after commit. On
  rollback, job does not appear in the queue. **Transactional outbox works.**
- Read-your-writes: after receiving a notification, the data IS queryable
  (no stale read issue with single DB).

**This is the most important result.** Honker provides true transactional
outbox semantics — data writes and event/queue publishes commit atomically.
This eliminates the dual-write problem that necessitated Redis for PostgreSQL.

## Open Research Questions

1. ~~**Drizzle session construction**~~ — RESOLVED by POC 1. All needed
   classes are accessible and extendable. `LibSQLSession` is the reference.

2. ~~**Honker listener concurrency model**~~ — RESOLVED by POC 3. Multiple
   listeners coexist. The `updateEvents` mechanism works for concurrent
   polling. `db.close()` requires all listeners to be closed first.

3. **Consumer name management for streams** — When a hub restarts, durable
   stream consumers resume from their last saved offset. But the consumer
   name must be stable across restarts. Need a naming convention that's
   deterministic (e.g., `{service}:{pid}` is not restart-stable).

4. **Schema migrations** — Honker opens a single `.db` file. Drizzle Kit
   supports SQLite migrations but expects `better-sqlite3` or `libsql`. Need
   to verify that `drizzle-kit push` or `drizzle-kit generate` works with the
   Honker adapter, or whether we need a custom migration runner.

5. **OperationSpec generation from tables** — What's the concrete mapping?
   For each table: one `create` mutation, one `find` query, one `list` query,
   one `update` mutation, one `delete` mutation? Or something more
   domain-specific? This is a design question, not a POC question.

6. **In-process + Honker composite event target** — POC 2 showed ~17ms
   latency for Honker notify→listen. For single-process hot paths, an
   in-process EventTarget is sub-ms. A composite that dispatches to both
   (in-process for speed, Honker for durability/cross-process) would be the
   ideal default for a hub. Needs design work.