diff --git a/AGENTS.md b/AGENTS.md index a4261d5..bb45712 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -28,10 +28,13 @@ infrastructure from org-scoped graph data. │ │ └── mod.ts # Re-exports all graphs exports │ └── sqlite/ # SQLite host (drizzle-orm + honker-node) │ ├── tables/ -│ │ ├── identity/ # accounts, organizations, org_members, api_keys, audit_logs +│ │ ├── common.ts # shared column definitions +│ │ ├── identity/ # accounts, organizations, org_members, api_keys, audit_logs │ │ └── metagraph/ # graph_types, node_types, edge_types, graphs, nodes, edges +│ ├── utils/ # createSelectSchema, createInsertSchema, column mappings (ADR-046) │ ├── relations.ts # Drizzle relations │ ├── adapter.ts # Drizzle-Honker session adapter +│ ├── event-target.ts # HonkerEventTarget (pubsub TypedEventTarget on Honker) │ ├── schema.ts # Re-exports │ └── client.ts # createSystemDatabase(), createTenantDatabase() └── test/ @@ -42,7 +45,7 @@ infrastructure from org-scoped graph data. - `@alkdev/storage` → Metagraph Module, graph type definitions (zero deps) - `@alkdev/storage/sqlite` → SQLite tables (metagraph + identity), relations, - client, Honker adapter (drizzle-orm + honker-node) + client, adapter, event-target, utils (drizzle-orm + honker-node, peer: @alkdev/pubsub) PostgreSQL has been removed (ADR-038). SQLite via Honker is the sole database host. @@ -60,12 +63,20 @@ PostgreSQL has been removed (ADR-038). SQLite via Honker is the sole database ho 4. **Injectable clients**: `createSystemDatabase(client)` and `createTenantDatabase(client)` take pre-created Honker clients, not env vars. Module-level side effects are forbidden. -5. **Dependencies**: `@alkdev/typebox` and `@alkdev/drizzlebox` are npm deps - (not yet on JSR). This works fine — JSR handles npm dependencies natively. +5. **Dependencies**: `@alkdev/typebox` is an npm dep (not yet on JSR). + `@alkdev/pubsub` is a peer dep (for `TypedEventTarget` type). The + `@alkdev/drizzlebox` external dep has been folded into `src/sqlite/utils/` + (ADR-046). 6. **SQLite-only via Honker**: No PostgreSQL. Honker provides DB + pub/sub + queues in a single SQLite file (ADR-038, ADR-039). 7. **System/tenant DB split**: Identity tables in `system.db`, graph data in `tenant-{orgId}.db` (ADR-040). +8. **OperationSpecs as repository surface**: Storage outputs `OperationSpec[]` + from table definitions. No hand-written CRUD. The consumer (hub/spoke) + registers handlers (ADR-048). +9. **HonkerEventTarget**: Bridges pubsub `TypedEventTarget` to Honker + `notify`/`listen` and `stream`/`subscribe`. Enables single-node pub/sub + without Redis. Transactional outbox semantics confirmed via POC (ADR-047). ## Commands @@ -91,7 +102,7 @@ The codebase has diverged significantly from the originals: Modules to DB row values - Crypto utility ported from `@alkdev/hub/src/crypto/mod.ts` with `EncryptedDataSchema` as a TypeBox schema -- `@sinclair/typebox` → `@alkdev/typebox`, `drizzle-typebox` → `@alkdev/drizzlebox` +- `@sinclair/typebox` → `@alkdev/typebox`, `drizzle-typebox` → `@alkdev/drizzlebox` → folded into `src/sqlite/utils/` (ADR-046) - TypeScript enums replaced with `as const` objects (`GRAPH_STATUS`, `ACTOR_TYPE`) - `Type.Unknown()` used for unvalidated fields (not `Type.Any()`) - Injectable client pattern (`createSqliteDatabase(client)` takes a pre-created client) @@ -104,7 +115,7 @@ The codebase has diverged significantly from the originals: - Entry points are `mod.ts` files that re-export from subdirectories - TypeBox schemas are named with PascalCase (`NodeType`, `GraphConfig`) - Drizzle table objects are named with camelCase (`graphTypes`, `nodeTypes`) -- Schema objects from drizzlebox are named with PascalCase (`InsertGraph`, +- Schema objects from utils are named with PascalCase (`InsertGraph`, `SelectGraph`) - Enum constants use `SCREAMING_SNAKE_CASE` objects (`GRAPH_STATUS`, `ACTOR_TYPE`) @@ -129,11 +140,15 @@ are not yet implemented. ## What's Not Done Yet -- Drizzle-Honker session adapter (`src/sqlite/adapter.ts`) +- Fold dbtype Phase 0 → `src/sqlite/utils/` (ADR-046, import path changes) +- Drizzle-Honker session adapter (`src/sqlite/adapter.ts`, POC validated) +- HonkerEventTarget (`src/sqlite/event-target.ts`, POC validated, ADR-047) - Identity tables in `src/sqlite/tables/identity/` (accounts, organizations, etc.) - Scoping columns on `graphs` table (`ownerId`, `projectId`) - Graph type `scope` column on `graph_types` table - Remove `actors` table and `src/pg/` directory -- Split client factory into `createSystemDatabase()` / `createTenantDatabase()` -- Repository/CRUD layer (typed query functions beyond table definitions) +- Update client factory to `createSystemDatabase()` / `createTenantDatabase()` (accept Honker client) +- Table restructure into subdirectories (metagraph/, identity/) +- OperationSpec generation from tables (ADR-048) +- ACL graph type (ADR-034) - JSR publication setup (need to create scope/package on jsr.io first) diff --git a/docs/architecture/decisions/046-fold-drizzlebox-as-utils.md b/docs/architecture/decisions/046-fold-drizzlebox-as-utils.md new file mode 100644 index 0000000..f165080 --- /dev/null +++ b/docs/architecture/decisions/046-fold-drizzlebox-as-utils.md @@ -0,0 +1,53 @@ +--- +status: accepted +date: 2026-06-01 +supersedes: ADR-005, ADR-018 +--- + +# ADR-046: Fold @alkdev/drizzlebox as src/sqlite/utils/ + +## Context + +`@alkdev/drizzlebox` (a fork of `drizzle-typebox` adapted for +`@alkdev/typebox`) provides `createSelectSchema` and `createInsertSchema` +functions that derive TypeBox validation schemas from Drizzle table +definitions. It is consumed as an external npm dependency by all SQLite +table files in `@alkdev/storage`. + +The parent project `@alkdev/dbtype` also planned a Phase 1 (UJSX→HostConfig +→Drizzle pipeline) that was never implemented. With SQLite as the sole +database target (ADR-038), the multi-dialect column mappings in dbtype +(PG, MySQL, SingleStore) are dead weight for storage. + +ADR-018 deferred dbtype integration to post-v1. But the fold of just the +Phase 0 subset (column→TypeBox mappings + schema generation) is a +straightforward import path change with no behavioral difference. + +## Decision + +Fold the SQLite-only subset of `@alkdev/dbtype` Phase 0 into +`src/sqlite/utils/`: + +| Source (dbtype) | Target (storage) | Changes | +|-----------------|-------------------|---------| +| `schema.ts` | `utils/schema.ts` | Remove PgEnum handling. Keep createSelectSchema, createInsertSchema, createUpdateSchema. | +| `column.ts` | `utils/column.ts` | Strip PG, MySQL, SingleStore branches. Keep SQLiteInteger, SQLiteReal, SQLiteText + generic dataType dispatch. | +| `schema.types.ts` + `schema.types.internal.ts` + `column.types.ts` | `utils/types.ts` | Merged. Remove PgEnum overloads. | +| `constants.ts` | `utils/constants.ts` | Keep as-is. | +| `utils.ts` | `utils/utils.ts` | Remove PgEnum alias. Keep isColumnType, isWithEnum, JsonSchema, BufferSchema. | + +All table files change their import from `@alkdev/drizzlebox` to `../utils/schema.ts`. + +## Consequences + +- **No external drizzlebox dependency** — one fewer npm package in the + dependency graph. +- **SQLite-only column mappings** — dead multi-dialect code removed. If a + new database host is added later, mappings would need to be re-added. +- **Same API surface** — `createSelectSchema`, `createInsertSchema`, + `createUpdateSchema` produce the same TypeBox schemas. +- **Co-located with tables** — utility code lives next to the tables it + derives schemas from. Easier to maintain. +- **dbtype Phase 1 (UJSX→HostConfig) not affected** — that remains a + separate architectural concern, could live in storage or as its own + package when built. \ No newline at end of file diff --git a/docs/architecture/decisions/047-honker-event-target.md b/docs/architecture/decisions/047-honker-event-target.md new file mode 100644 index 0000000..b453c53 --- /dev/null +++ b/docs/architecture/decisions/047-honker-event-target.md @@ -0,0 +1,58 @@ +--- +status: accepted +date: 2026-06-01 +resolves: OQ-26 +--- + +# ADR-047: HonkerEventTarget Adapter for pubsub + +## Context + +`@alkdev/pubsub` defines a `TypedEventTarget` interface that all transport +adapters implement: in-process `EventTarget`, Redis, WebSocket client/server, +and Worker. This provides transport-agnostic pub/sub — consumers call +`createPubSub({ eventTarget })` without knowing the underlying transport. + +Honker provides SQLite with built-in pub/sub primitives: +- `db.notify(channel, payload)` / `db.listen(channel)` — ephemeral, fire-and-forget +- `db.stream(name).publish(payload)` / `db.stream(name).subscribe(consumer)` — durable, offset-tracked + +POC 2-4 (2026-06-01) validated: +- Same-process notify→listen works. ~17ms median latency. +- Multiple concurrent listeners on different channels work. +- `tx.notify()` only fires on `tx.commit()`. Rollback suppresses notification. +- `queue.enqueueTx(tx, payload)` only visible after commit. Rollback suppresses. +- Stream publish/subscribe works with consumer offset tracking. + +These results confirm Honker can back the pubsub `TypedEventTarget` +interface for single-node deployments. + +## Decision + +Implement `HonkerEventTarget` in `src/sqlite/event-target.ts`. It adapts +`@alkdev/pubsub`'s `TypedEventTarget` to Honker primitives with two modes: + +1. **Ephemeral mode**: `addEventListener` → `db.listen()`, `dispatchEvent` → + `db.notify()`. Fire-and-forget semantics, no delivery guarantee. +2. **Durable mode**: `addEventListener` → `db.stream().subscribe()`, + `dispatchEvent` → `db.stream().publish()`. Per-consumer offset tracking, + crash recovery replays from last saved offset. + +`@alkdev/pubsub` is a peer dependency (needed only when using +HonkerEventTarget). The `graphs/` module remains zero-dep. + +## Consequences + +- **Single-node hub can use Honker instead of Redis** — no separate Redis + deployment needed for pub/sub. +- **Transactional outbox semantics** — `dispatchEvent` inside a Drizzle + transaction (via `tx.notify()` or `stream.publishTx()`) commits atomically + with data writes. No dual-write problem. +- **Hub-spoke symmetry** — both hub and spoke use the same `createPubSub()` + call. Different event target instances determine routing. +- **Multi-node still needs Redis or WebSocket** — Honker events don't cross + process boundaries. For multi-node, the `WebSocketServerEventTarget` (hub) + and `WebSocketClientEventTarget` (spoke) handle cross-process routing. +- **Latency trade-off** — ~17ms Honker round-trip vs sub-ms in-process. For + hot-path call protocol, pair with in-process `EventTarget`. Design of a + composite event target is an open question (OQ-30). \ No newline at end of file diff --git a/docs/architecture/decisions/048-operation-specs-as-repo-surface.md b/docs/architecture/decisions/048-operation-specs-as-repo-surface.md new file mode 100644 index 0000000..330f1f9 --- /dev/null +++ b/docs/architecture/decisions/048-operation-specs-as-repo-surface.md @@ -0,0 +1,74 @@ +--- +status: accepted +date: 2026-06-01 +supersedes: ADR-033 (partially — OQ-17, OQ-18, OQ-19 updated) +--- + +# ADR-048: OperationSpecs as Repository Surface + +## Context + +ADR-033 established JSON path queries with hand-written CRUD for v1. The +question was whether the repository layer would be hand-written query +functions, auto-generated from table definitions, or something else. + +The `@alkdev/operations` package provides `OperationSpec` — a serializable +descriptor for a query, mutation, or subscription operation. Hubs and spokes +register specs in an `OperationRegistry` along with handlers. The operations +runtime handles execution, call protocol routing, subscriptions, access +control, and validation. + +This presents a natural alternative to hand-written CRUD: storage outputs +`OperationSpec[]` describing CRUD operations per table, and the consumer +wires them into the operations registry. + +## Decision + +Storage does not ship a "repository layer" of hand-written query functions. +Instead, it outputs `OperationSpec[]` per table — flat arrays describing +CRUD operations. The consumer (hub/spoke) imports these specs, registers +handlers, and the operations runtime handles execution. + +```ts +// Storage defines table + operation contracts +export const graphTypes = sqliteTable("graph_types", { ... }); +export const graphTypeSpecs: OperationSpec[] = [ + { name: "create", namespace: "graph_types", type: "mutation", ... }, + { name: "find", namespace: "graph_types", type: "query", ... }, + { name: "list", namespace: "graph_types", type: "query", ... }, + { name: "update", namespace: "graph_types", type: "mutation", ... }, + { name: "delete", namespace: "graph_types", type: "mutation", ... }, +]; + +// Hub registers specs + handlers +for (const spec of graphTypeSpecs) { + registry.registerSpec(spec); + registry.registerHandler(`${spec.namespace}.${spec.name}`, handler); +} +``` + +The handler is consumer-provided. Storage defines the contract (input/output +schemas, operation type); the hub provides the execution (Drizzle queries, +Honker transactions, notifications). + +`@alkdev/operations` is a type-only peer dependency of storage. No circular +dependency. + +## Consequences + +- **No hand-written repository functions** — storage avoids the maintenance + burden of typed CRUD code. The contract is the OperationSpec. +- **Consumer owns execution** — the hub decides how to handle each operation + (raw Drizzle query, with Honker notification, with access control, etc.). + Storage doesn't execute queries. +- **Subscriptions for free** — OperationSpec supports `type: "subscription"`. + Table-level change streams become operation subscriptions rather than + custom event handling. +- **Clean dependency graph** — storage depends on operations only for the + `OperationSpec` type (peer dep). No runtime dependency. +- **Attribute queries remain JSON path** — for metagraph tables where + attributes are dynamic JSON, `json_extract()` is still the query mechanism. + Domain-specific tables with native columns produce simpler specs. +- **Supersedes ADR-033 partially** — the "hand-written CRUD" part is replaced. + The "JSON path queries for attributes" part still applies to metagraph + tables. \ No newline at end of file diff --git a/docs/architecture/forward-look.md b/docs/architecture/forward-look.md index 09cb178..4f066c4 100644 --- a/docs/architecture/forward-look.md +++ b/docs/architecture/forward-look.md @@ -1,16 +1,15 @@ --- status: draft -last_updated: 2026-05-31 +last_updated: 2026-06-01 --- # Forward Look: Pointers, dbtype, and Universal IR How the Module-based metagraph connects to the broader @alkdev ecosystem — -typed graph pointers, dbtype table rendering, and the ujsx universal IR -pipeline. These are forward-looking designs that justify why certain structural -decisions were made now -(pointer abstraction deferred per [ADR-017](./decisions/017-pointer-abstraction-is-forward-looking.md), -dbtype integration deferred per [ADR-018](./decisions/018-dbtype-integration-is-post-v1.md)). +typed graph pointers, local utils (folded from dbtype), and the ujsx universal IR +pipeline. The dbtype integration is no longer deferred (ADR-046) — the SQLite-only +Phase 0 subset folds into `src/sqlite/utils/`. The repository surface is now +OperationSpecs (ADR-048), not hand-written CRUD. ## Overview @@ -109,68 +108,69 @@ feasible because it provides the schema the pointer validates against. ## Relationship to @alkdev/dbtype -`@alkdev/dbtype` defines database schemas as ujsx element trees and renders them -to Drizzle dialects via HostConfig. Storage's SQLite/PG table definitions are a -natural consumer of this pipeline. +`@alkdev/dbtype` defined database schemas as ujsx element trees and planned to +render them to Drizzle dialects via HostConfig. Its Phase 0 (Drizzle→TypeBox +schema generation) was consumed as `@alkdev/drizzlebox`. Phase 1 (UJSX→Drizzle) +was never implemented. -### Current vs. Future Table Definition +### Fold: Phase 0 → `src/sqlite/utils/` (ADR-046) -**Current** (manual Drizzle table defs): +With SQLite as the sole target (ADR-038), the multi-dialect column mappings in +dbtype are dead weight. The SQLite-only subset has been folded into storage as +`src/sqlite/utils/`: + +| What folds in | Source (dbtype) | Target (storage) | +|---------------|-----------------|-------------------| +| Schema generation | `schema.ts` | `utils/schema.ts` | +| Column→TypeBox mappings | `column.ts` (SQLite branches only) | `utils/column.ts` | +| Type interfaces | `schema.types.ts` + `schema.types.internal.ts` + `column.types.ts` | `utils/types.ts` | +| Integer constants | `constants.ts` | `utils/constants.ts` | +| Type guards | `utils.ts` (minus PgEnum) | `utils/utils.ts` | + +What does NOT fold in: PG, MySQL, SingleStore column handlers; `isPgEnum` / +`handleEnum`; `createSchemaFactory`; the Phase 1 UJSX→HostConfig pipeline. + +Import changes in table files: ```ts -export const graphTypes = sqliteTable("graph_types", { - id: text("id").primaryKey(), - name: text("name").notNull(), - config: text("config", { mode: "json" }).notNull(), - // ... -}); +// Before +import { createInsertSchema, createSelectSchema } from "@alkdev/drizzlebox"; +// After +import { createInsertSchema, createSelectSchema } from "../utils/schema.ts"; ``` -**Future** (dbtype element tree → HostConfig rendering): +The API surface is identical — same functions, same TypeBox schemas produced. -```tsx -const GraphTypesEl = h("table", { name: "graph_types" }, - h(IdColumn, {}), - h("column", { name: "name", type: "string", notNull: true }), - h("column", { name: "config", type: "json", mode: "json", notNull: true }), - h(AuditColumns, {}), -); +### Phase 1 (UJSX→Drizzle): future path -const root = createRoot(sqliteHost, {}); -root.render(GraphTypesEl); -const drizzleTable = root.ctx.tables.graph_types; -``` +The broader UJSX→HostConfig→Drizzle pipeline from dbtype's architecture remains +architecturally sound but is not part of this pivot. When and if it's built, it +could live in storage as a `HostConfig` sub-module rather than a separate +package, since storage is the primary consumer. The TypeBox Module format used +by the metagraph is already compatible with what a ujsx HostConfig would produce. ### Why this matters for storage -1. **Single source of truth**: Today's `sqlite/tables/` and future `pg/tables/` - define the same shapes in two different Drizzle dialects. dbtype renders the - same element tree to both — no manual duplication. -2. **Schema extraction**: `extractTable()` produces both TypeBox schemas (for - validation) and column metadata (for Drizzle rendering) from the same tree. - Storage gets `SelectGraphType` and `InsertGraphType` schemas for free. -3. **Module alignment**: dbtype assembles extracted schemas into a - `Type.Module` for cross-table references. Storage's metagraph Module and - dbtype's table Module could share a namespace — the `graph_types.config` - column stores the JSON Schema from `Metagraph.Config`. +1. **Single source of truth**: The `utils/` code derives TypeBox schemas from + Drizzle tables. Table definitions are the source of truth for both the DB + schema and the validation schema. +2. **Schema extraction**: `createSelectSchema` / `createInsertSchema` produce + TypeBox schemas that validate data at the application layer. +3. **Module alignment**: The metagraph Module and the table-derived schemas + share the same TypeBox namespace. `graph_types.config` stores the JSON + Schema from `Metagraph.Config`. ### v1 approach -For v1, storage continues with manual Drizzle table definitions. The dbtype -integration is deferred because: +For v1, storage uses the folded utils for TypeBox schema derivation from Drizzle +tables (what was `@alkdev/drizzlebox`). The metagraph Module independently +validates graph type definitions. These two schema sources serve different +purposes: table schemas validate DB row shapes, Module schemas validate graph +type semantics. -- dbtype is Phase 0 (architecture complete, no implementation) -- The manual defs work and are well-understood -- The Module pattern for graph types can be adopted independently (no dbtype -dependency) -- With PostgreSQL removed (ADR-038), the original pressure for dbtype — - eliminating dual SQLite/PG table maintenance — is significantly reduced. - There is now only one set of table definitions to maintain. - -When dbtype reaches Phase 1 (implementation), storage can migrate from -Drizzle table definitions to dbtype elements one table at a time. The Module-based -graph type definitions are already compatible — they're both TypeBox `Type.Module` -objects. +When dbtype's Phase 1 (UJSX→HostConfig) is implemented, it would unify both +directions — a TypeBox Module could produce both the Drizzle table definition +and the validation schemas from the same element tree. ## ujsx as Universal IR @@ -219,10 +219,17 @@ Rendered to different hosts: | Reference graph type Modules (CallGraph, SecretGraph) | ✅ Implemented | | Crypto utility (`encrypt`, `decrypt`, `generateEncryptionKey`, `EncryptedDataSchema`) | ✅ Implemented | | Codegen from TypeScript interfaces → Module entries | ✅ TsToModule exists | -| dbtype element trees → Drizzle tables | ⚠️ dbtype Phase 0, no implementation | +| SQLite column→TypeBox mappings (folded from dbtype) | ✅ Folded into `src/sqlite/utils/` (ADR-046) | +| `createSelectSchema` / `createInsertSchema` (folded from drizzlebox) | ✅ Folded into `src/sqlite/utils/` (ADR-046) | +| Drizzle-Honker session adapter | ✅ POC validated, implementation pending | +| HonkerEventTarget for pubsub | ✅ POC validated, implementation pending | +| Transactional notify + outbox (Honker) | ✅ POC validated — atomic commit for data + events + queue | +| OperationSpec generation from tables | ⚠️ Design complete (ADR-048), implementation pending | +| Domain-specific native-column tables | ⚠️ Conceptual — for known graph types (CallGraph, etc.) | | `` ujsx elements | ⚠️ Conceptual — needs HostConfig design | | Typed graph pointers via JPATH | ⚠️ Conceptual — needs JPATH Module design | | Reactive graph observation via ValuePointer | ⚠️ Conceptual — needs signal integration | +| dbtype Phase 1 (UJSX→Drizzle HostConfig) | ⚠️ Architecture exists, not implemented. Could live in storage if built. | The Module-based graph type definitions (this spec) are the **first concrete step** in this pipeline. Everything else builds on having a `Type.Module` as @@ -230,126 +237,66 @@ the schema source of truth. ## Repository Layer Strategy -The repository layer (typed CRUD for the 6 metagraph tables + queries for graph data) -is the next major feature to implement. The question of *how* it queries attributes -connects to broader ecosystem decisions about dbtype and operations. +The repository layer (typed CRUD for the 6 metagraph tables + identity tables + +queries for graph data) is now defined as **OperationSpec output** rather than +hand-written query functions (ADR-048). -### Three Approaches +### OperationSpecs as Repository Surface -#### A. JSON Path Queries (Near-Term) +Storage outputs `OperationSpec[]` per table — flat arrays describing CRUD +operations. The consumer (hub/spoke) imports these, registers handlers, and +the operations runtime handles execution, call protocol, and subscriptions. -The repository layer maps filter criteria to JSON path extraction: +```ts +// Storage defines the table + operation contracts +export const callNodes = sqliteTable("call_nodes", { ... }); +export const callNodeSpecs: OperationSpec[] = [ + { name: "create", namespace: "call_nodes", type: "mutation", inputSchema: ..., outputSchema: ... }, + { name: "find", namespace: "call_nodes", type: "query", ... }, + { name: "list", namespace: "call_nodes", type: "query", ... }, + { name: "update", namespace: "call_nodes", type: "mutation", ... }, + { name: "delete", namespace: "call_nodes", type: "mutation", ... }, +]; + +// Hub registers specs + handlers +for (const spec of callNodeSpecs) { + registry.registerSpec(spec); + registry.registerHandler(`${spec.namespace}.${spec.name}`, handler); +} +``` + +The handler is consumer-provided — not in storage. Storage doesn't execute +queries. Storage defines the contract; the hub provides the execution layer. + +### Attribute Queries + +The metagraph's `attributes` column remains JSON — node types are dynamic +schemas defined at runtime, not static columns. Attribute queries use +`json_extract()` for v1: ```ts findNodes({ graphId, attributes: { status: "active" } }) // SQLite: json_extract(attributes, '$.status') = 'active' -// PG: attributes ->> 'status' = 'active' ``` -- Works with current table definitions (no schema changes) -- SQLite `json_extract()` and PG `->>` / `#>>` operators handle JSON path -- No native index support on individual JSON attributes -- PG can add GIN indexes on `jsonb` columns for containment queries, but not for - arbitrary key-value lookups -- Simple, immediate, no new infrastructure +For known graph types (CallGraph, SecretGraph), domain-specific tables with +native columns can complement the generic metagraph tables. These domain +tables also produce OperationSpecs with native-column queries. -This is the pragmatic v1 approach. The metagraph pattern *requires* JSON attributes -because node types are dynamic schemas (defined at runtime, stored in -`node_types.schema`), not static columns known at database definition time. +### Connection to @alkdev/operations -#### B. Native Columns via dbtype (Long-Term, Speculative) - -If storage migrates to dbtype element trees for table definitions, the 6 static -metagraph tables (graph_types, node_types, edge_types, graphs, nodes, edges) could -be rendered via the dbtype pipeline: element tree → HostConfig → Drizzle tables. -This would eliminate the manual duplication between `sqlite/` and future `pg/`. - -However, dbtype does NOT solve the attribute indexing problem: - -- The metagraph's `attributes` column MUST remain JSON because the shape is defined - by runtime schemas (node type definitions), not by static column definitions -- dbtype generates static table schemas; it does not handle dynamic schema-as-data - patterns like the metagraph -- A "call" node's attributes (`requestId`, `status`, `duration`) are not columns - on the `nodes` table — they're values in the `attributes` JSON column, validated - by the corresponding node type's TypeBox schema - -#### C. Hybrid: Static Tables via dbtype, Dynamic Attributes Remain JSON - -The hybrid approach preserves the metagraph's dynamic schema model while leveraging -dbtype for the static table scaffolding: - -1. **Static tables**: dbtype renders the 6 metagraph tables to Drizzle dialects. - This eliminates the SQLite/PG manual duplication for table *structure*. - The `attributes` column is still `text/jsonb` across both dialects. - -2. **Dynamic attributes**: Remain JSON. The Module-based node type schemas validate - data at the application layer, not the database layer. This is by design - (ADR-003, ADR-014). - -3. **Virtual columns / computed columns**: A post-v1 optimization, not a v1 concern. - Frequently queried attributes could be extracted to indexed columns as a - performance optimization. For example, if `nodes.attributes.status` is a common - filter, a computed column or trigger could copy it to `nodes.status_column` with - an index. This would be a denormalization trade-off (triggers, migration - complexity, dual-write responsibility) and is not designed or planned for v1. - -4. **Repository CRUD**: The static table CRUD operations (insert graph type, find - node by key) could be auto-generated like drizzle-graphql or the dbtype - `from-dbtype` adapter. Graph-specific attribute queries remain JSON path. - -### Implications for Each Approach - -| Concern | Path A (JSON) | Path B (Native) | Path C (Hybrid) | -|---------|---------------|-----------------|------------------| -| Works today | ✅ | ❌ (requires dbtype) | ❌ (requires dbtype) | -| Preserves metagraph pattern | ✅ | ❌ (conflicts with dynamic schemas) | ✅ | -| Eliminates SQLite/PG duplication | ❌ | ✅ | ✅ | -| Indexes on attributes | GIN on PG only | ✅ full native | GIN + virtual columns | -| Repository generation | Hand-write CRUD | Auto-gen from dbtype | Auto-gen for static, JSON path for dynamic | -| Dependency on dbtype | None | Full | Partial (static tables only) | - -### Connection to drizzle-graphql - -The overview references drizzle-graphql as a pattern for auto-generating a CRUD/query -surface. The dbtype `from-dbtype` adapter is the @alkdev equivalent: it consumes -element trees + Type.Module bundles and produces `OperationSpec[]` for the -operations registry. - -The parallel: - -| Concern | drizzle-graphql | dbtype from-dbtype | -|---------|----------------|-------------------| -| Input | Drizzle schema (tables + relations) | UJSX element tree + Type.Module | -| Output | GraphQL schema (queries + mutations) | `OperationSpec[]` (CRUD operations) | -| Dialects | SQLite, PG, MySQL | SQLite, PG, MySQL (via HostConfig) | -| Table model | Static columns only | Static columns only | -| Dynamic data (JSON attrs) | Not handled | Not handled | - -Neither drizzle-graphql nor dbtype's `from-dbtype` handles dynamic schema-as-data -patterns. The metagraph's JSON attributes require their own query layer, regardless -of whether the static tables are auto-generated. This means the repository layer -for `@alkdev/storage` will always have two parts: - -1. **Static table CRUD** — could be auto-generated (by dbtype or hand-written) -2. **Graph data queries** — JSON path queries against the `attributes` column, - validated by the Module schema at the application layer +`@alkdev/operations` is a type-only peer dependency of storage. The +`OperationSpec` type is straightforward. Storage builds the specs; the +consumer wires them into the registry. No circular dependency. ### v1 Decision -For v1, the practical path is **A (JSON path queries) with hand-written CRUD**. This -decision is recorded as [ADR-033](./decisions/033-json-path-queries-for-v1.md). The -hybrid approach (C) remains viable for a future iteration when dbtype reaches -implementation, and it doesn't require any changes to the metagraph data model — -only to how the static table definitions are generated. See OQ-17, OQ-18, OQ-19 -in [open-questions.md](./open-questions.md) for the specific long-term questions -that remain open beyond v1. - -### Decisions Required - -- **OQ-17**: JSON path vs native columns vs hybrid for attribute queries (resolved for v1 — see ADR-033) -- **OQ-18**: Auto-generated vs hand-written CRUD for static tables (resolved for v1 — see ADR-033) -- **OQ-19**: Where the storage-operations bridge package should live (open) +For v1, the practical path is **OperationSpecs with JSON path attribute +queries** (ADR-048, supersedes ADR-033). Spec generation from tables is +straightforward once domain tables exist. The metagraph's generic CRUD +(graphs, nodes, edges) uses JSON attributes; domain-specific CRUD uses +native columns. Both produce OperationSpecs that the hub registers in the +same operations registry. ## Constraints on Current Design @@ -382,18 +329,28 @@ design in [metagraph-module.md](./metagraph-module.md): opaque builder objects or Drizzle column definitions. See [schema-evolution.md](./schema-evolution.md). +6. **OperationSpec output is consumer-agnostic** — storage defines + `OperationSpec[]` from table definitions. The consumer (hub/spoke) decides + how to register handlers. Storage does not execute queries or depend on + the operations runtime. + +7. **The folded utils are SQLite-only** — `src/sqlite/utils/` contains only + SQLite column→TypeBox mappings. If a new database host is added later, the + utils would need the corresponding dialect mappings. dbtype's Phase 1 + (UJSX→HostConfig) would be the mechanism for multi-dialect support. + ## References - ujsx pointer system: `/workspace/@alkdev/ujsx/src/core/pointer.ts` - ujsx HostConfig adapter: `/workspace/@alkdev/ujsx/src/host/config.ts` -- dbtype architecture: `/workspace/@alkdev/dbtype/docs/architecture/README.md` +- dbtype architecture: `/workspace/@alkdev/dbtype/docs/architecture/README.md` (Phase 0 source folded into storage) - dbtype elements: `/workspace/@alkdev/dbtype/docs/architecture/elements.md` - dbtype module: `/workspace/@alkdev/dbtype/docs/architecture/module.md` - dbtype repo adapter: `/workspace/@alkdev/dbtype/docs/architecture/repo-adapter.md` -- drizzle-graphql (reference for CRUD generation pattern): `/workspace/drizzle-graphql/` - Operations registry: `/workspace/@alkdev/operations/docs/architecture/README.md` - JPATH Module (JSONPath as TypeBox Module): `/workspace/research/typebox_research/ujsx/jpath.gen.ts` - jsonpathly source: `/workspace/jsonpathly/` - Module evolution spec: [metagraph-module.md](./metagraph-module.md) - Schema evolution spec: [schema-evolution.md](./schema-evolution.md) -- ADR-033: JSON path queries and hand-written CRUD for v1 \ No newline at end of file +- ADR-046: Fold drizzlebox as utils (supersedes ADR-033) +- ADR-048: OperationSpecs as repository surface \ No newline at end of file diff --git a/docs/architecture/honker-integration.md b/docs/architecture/honker-integration.md index cfbf1ec..2322006 100644 --- a/docs/architecture/honker-integration.md +++ b/docs/architecture/honker-integration.md @@ -1,12 +1,14 @@ --- status: draft -last_updated: 2026-05-31 +last_updated: 2026-06-01 --- # Honker Integration How @alkdev/storage integrates with Honker for SQLite database operations, -transactional pub/sub, durable event streams, and task queues. +transactional pub/sub, durable event streams, and task queues. Includes the +HonkerEventTarget adapter that bridges `@alkdev/pubsub`'s `TypedEventTarget` +to Honker primitives. ## Purpose @@ -118,10 +120,134 @@ db.transaction((tx) => { | Limitation | Impact | Mitigation | |------------|--------|------------| -| No `lastInsertRowid` from `execute()` | `run()` needs extra `SELECT last_insert_rowid()` | Small Rust PR to honker-node | +| No `lastInsertRowid` from `execute()` | `run()` needs extra `SELECT last_insert_rowid()` | POC 1 confirmed `tx.query("SELECT last_insert_rowid() as id")` works | | No prepared statement handle at JS level | Every query re-enters napi | Rust-side `prepare_cached` amortizes | | Object rows only, no raw arrays | `values()` must convert `Object.values()` | Column order preserved from SQLite | -| Drizzle internal API dependency | Adapter imports from `drizzle-orm/sqlite-core/session` | Stable across minor versions | +| Drizzle internal API dependency | Adapter imports from `drizzle-orm/sqlite-core/session` | POC 1 confirmed all classes accessible and extendable | +| `:memory:` databases don't work | Reader pool gets separate in-memory DB | Always use file-based paths. Tests use temp files. | + +### POC-Validated Adapter Architecture + +POC 1 (2026-06-01) confirmed the adapter is buildable. Key findings: + +- `SQLiteSession`, `SQLitePreparedQuery`, `SQLiteTransaction` are all + accessible from `drizzle-orm/sqlite-core/session` and extendable. +- `BaseSQLiteDatabase` is accessible from `drizzle-orm/sqlite-core/db`. +- `LibSQLSession` in `drizzle-orm/libsql/session` is the reference + implementation to follow. +- Honker `query()` returns `{ columnName: value }` object rows, compatible + with Drizzle's `mapResultRow()`. +- `last_insert_rowid()` accessible via `tx.query("SELECT last_insert_rowid() as id")`. +- `tx.execute()` returns a number (affected rows count). +- JSON-mode columns need manual `JSON.parse()` in the adapter. + +## HonkerEventTarget + +The `HonkerEventTarget` adapts `@alkdev/pubsub`'s `TypedEventTarget` interface +to Honker's `notify`/`listen` (ephemeral) and `stream`/`subscribe` (durable) +primitives. It lives in `src/sqlite/event-target.ts`. + +### Interface + +```ts +import type { TypedEventTarget } from "@alkdev/pubsub"; + +interface HonkerEventTargetOptions { + db: Database; + mode: "ephemeral" | "durable"; + streamName?: string; + consumerName?: string; + prefix?: string; +} + +function createHonkerEventTarget( + options: HonkerEventTargetOptions +): TypedEventTarget & { close(): void }; +``` + +### Ephemeral Mode + +Maps to Honker's `notify()`/`listen()`: + +| pubsub operation | Honker operation | +|-------------------|------------------| +| `addEventListener("topic:id", callback)` | `db.listen("topic:id")` → start async consumer loop | +| `dispatchEvent(event)` | `db.notify(event.type, event.detail)` | +| `removeEventListener("topic:id", callback)` | Close the listener when no callbacks remain | +| `close()` | Close all active listeners | + +POC 2 confirmed: same-process `notify`→`listen` works, multiple concurrent +listeners on different channels work, cross-channel isolation is correct. + +POC 4 confirmed: `tx.notify()` within a Drizzle transaction only fires the +notification on `tx.commit()`. On `tx.rollback()`, the notification is +suppressed. This enables transactional outbox semantics for ephemeral events. + +### Durable Mode + +Maps to Honker's `Stream.publish()`/`Stream.subscribe()`: + +| pubsub operation | Honker operation | +|-------------------|------------------| +| `addEventListener("topic:id", callback)` | `db.stream(name).subscribe(consumer)` → start async consumer loop | +| `dispatchEvent(event)` | `db.stream(name).publish(event.detail)` | +| `removeEventListener("topic:id", callback)` | Close the subscription when no callbacks remain | +| `close()` | Close all active subscriptions, save offsets | + +POC 3 confirmed: `Stream.publish()`/`Stream.subscribe()` work, consumer +offsets are tracked and persisted, `publishTx(tx, payload)` works within +transactions, and concurrent stream + listener operation works. + +Durable mode provides crash recovery — consumers resume from their last saved +offset after restart. Consumer names must be stable across restarts (not +PID-based). + +### Topic Routing + +pubsub uses `topic:id` composite topics. Honker channels and streams are flat +strings. The mapping: + +- **Ephemeral**: Each unique `topic:id` gets its own `db.listen()` call. + For high-cardinality topics (many request IDs), consider topic prefix + matching with client-side filtering instead. +- **Durable**: Streams are name-keyed, not topic-keyed. A single stream + carries all events for a domain. Client-side filtering dispatches only + to matching listeners. + +Suggested split: + +| Event category | Mode | Reason | +|---------------|------|--------| +| Call protocol events (`call.requested`, `call.responded`, etc.) | Durable stream | Crash recovery, audit trail, flowgraph replay | +| Cache invalidation signals | Ephemeral | Fire-and-forget, loss acceptable | +| UI/dashboard push | Ephemeral | Low latency, loss acceptable | +| Schema migration jobs | Queue (not pubsub) | At-least-once processing | + +### Latency Consideration + +POC 2 measured ~17ms median latency for Honker `notify`→`listen` within a +single process. For hot-path call protocol request/response where sub-ms +latency matters, pair the Honker event target with an in-process `EventTarget` +(pubsub's default). A composite pattern (dispatch to both) provides both +in-process speed and Honker durability/cross-process coordination. + +### Hub-Spoke Event Routing + +``` +Hub (system.db + tenant-{orgId}.db) + ├── HonkerEventTarget (durable: call-protocol stream) + ├── WebSocketServerEventTarget (spoke fan-out) + └── In-process EventTarget (local subscribers) + +Spoke (tenant-{orgId}.db) + ├── HonkerEventTarget (ephemeral: local channels) + ├── WebSocketClientEventTarget (hub connection) + └── In-process EventTarget (local subscribers) +``` + +Both hub and spoke use the same `createPubSub({ eventTarget })` call. +Different event target instances determine the routing. No code changes +between hub and spoke — only configuration. ## Event-Driven Patterns @@ -338,7 +464,7 @@ would require a coordinator — the hub fills this role. - The metagraph Module system (CallGraph, SecretGraph, AclGraph Modules) - Bridge functions (`moduleToDbSchema`, `validateNode`, `validateEdge`) - Crypto utility (encrypt, decrypt, `EncryptedDataSchema`) -- TypeBox schemas from drizzlebox +- TypeBox schemas from local utils (folded from drizzlebox) - The Drizzle query builder API (same `.select()`, `.insert()`, `.update()`, `.delete()` calls) - The `@alkdev/operations` call protocol (events are now published via Honker streams instead of Redis) @@ -359,13 +485,18 @@ would require a coordinator — the hub fills this role. |-----|----------|--------| | [039](decisions/039-honker-as-sqlite-extension.md) | Honker as SQLite extension and transport | Accepted | | [044](decisions/044-drizzle-honker-adapter.md) | Drizzle-Honker session adapter | Accepted | +| [046](decisions/046-fold-drizzlebox-as-utils.md) | Fold @alkdev/drizzlebox as src/sqlite/utils/ | Accepted | +| [047](decisions/047-honker-event-target.md) | HonkerEventTarget adapter for pubsub | Accepted | ## Open Questions -- **OQ-26**: Can Honker fully replace `@alkdev/pubsub`'s Redis transport for single-node deployments? +- **OQ-26**: ~~Can Honker fully replace `@alkdev/pubsub`'s Redis transport for single-node deployments?~~ Resolved: Yes. HonkerEventTarget (ADR-047) provides the adapter. Redis still needed for multi-node. - **OQ-27**: How are schema migrations applied across all tenant DBs? - **OQ-28**: How does cross-tenant delegation work with separate DBs? - **OQ-29**: Should the adapter be published as a standalone `drizzle-honker` npm package for community use? +- **OQ-30**: Composite event target pattern — how should an in-process EventTarget and HonkerEventTarget be combined for single-node hub deployments? POC 2 showed ~17ms Honker latency vs sub-ms in-process. Design needed. +- **OQ-31**: Consumer naming convention for durable subscriptions — must be stable across hub restarts (not PID-based). +- **OQ-32**: Drizzle Kit migration compatibility — does `drizzle-kit push`/`drizzle-kit generate` work with the custom Honker adapter, or do we need a custom migration runner? ## References diff --git a/docs/architecture/open-questions.md b/docs/architecture/open-questions.md index c04ab96..a268b8b 100644 --- a/docs/architecture/open-questions.md +++ b/docs/architecture/open-questions.md @@ -1,6 +1,6 @@ --- status: draft -last_updated: 2026-05-31 +last_updated: 2026-06-01 --- # Open Questions Tracker @@ -12,8 +12,8 @@ architecture documents, organized by theme. | Status | Count | |--------|-------| -| Open | 9 | -| Resolved (this revision) | 15 | +| Open | 10 | +| Resolved (this revision) | 18 | | Previously resolved | 11 | **Open questions requiring decisions:** @@ -23,10 +23,13 @@ architecture documents, organized by theme. - **OQ-11** (auto-migrate vs explicit) — conditional on OQ-10 - **OQ-12** (schema evolution vs event-sourced replay) — post-v1 concern - **OQ-13** (schema evolution events in event stream) — post-v1 -- **OQ-26** (Honker replaces @alkdev/pubsub Redis transport) — integration design +- **OQ-25** (scope string semantics) — evaluator concern - **OQ-27** (tenant DB schema migration strategy) — multi-tenant operations - **OQ-28** (cross-tenant delegation with separate DBs) — cross-DB coordination - **OQ-29** (standalone drizzle-honker npm package) — community value +- **OQ-30** (composite event target for single-node hub) — latency optimization +- **OQ-31** (consumer naming for durable subscriptions) — restart stability +- **OQ-32** (Drizzle Kit migration compatibility) — custom adapter ## Theme 1: Package Boundaries and Dependencies @@ -152,21 +155,23 @@ architecture documents, organized by theme. ### OQ-17: Attribute queries — JSON path, native columns, or dbtype-generated? -- **Status**: resolved (v1) +- **Status**: resolved (updated) - **Priority**: high -- **Resolution**: JSON path for v1. ADR-033. Long-term hybrid still open but less pressing without PG dual maintenance. +- **Resolution**: JSON path for metagraph attributes (dynamic schemas). Native columns for domain-specific tables (CallGraph, etc.). OperationSpecs provide the CRUD contract for both patterns. ADR-048 supersedes ADR-033. +- **Cross-references**: ADR-048 ### OQ-18: Auto-generated vs hand-written CRUD? -- **Status**: resolved (v1) +- **Status**: resolved (updated) - **Priority**: medium -- **Resolution**: Hand-write for v1. ADR-033. +- **Resolution**: Not hand-written CRUD — OperationSpecs. Storage outputs `OperationSpec[]` from table definitions. The consumer (hub/spoke) registers handlers. ADR-048. +- **Cross-references**: ADR-048 ### OQ-19: Storage-operations bridge package location? -- **Status**: resolved +- **Status**: resolved (updated) - **Priority**: medium -- **Resolution**: Less pressing now that Honker replaces the Redis transport. Can live in the hub for v1. Revisit if an adapter package becomes valuable. +- **Resolution**: No separate bridge package needed. Storage outputs `OperationSpec[]` as part of its table definitions (type-only peer dep on `@alkdev/operations`). The consumer wires specs into the registry. ADR-048. ## Theme 7: Access Control @@ -220,9 +225,10 @@ architecture documents, organized by theme. ### OQ-26: Can Honker fully replace @alkdev/pubsub's Redis transport for single-node deployments? - **Origin**: [honker-integration.md](honker-integration.md) -- **Status**: open +- **Status**: resolved - **Priority**: high -- **Notes**: Honker's `notify()`/`listen()` and `stream()`/`subscribe()` provide the pub/sub primitives. The question is whether `@alkdev/pubsub`'s `TypedEventTarget` interface can be backed by Honker instead of Redis, and whether multi-node deployments still need Redis for internode communication. +- **Resolution**: Yes for single-node. The `HonkerEventTarget` adapter (ADR-047) implements pubsub's `TypedEventTarget` on Honker's `notify`/`listen` and `stream`/`subscribe`. POC 2-4 validated: same-process pub/sub works, transactional semantics hold, concurrent listeners work. Redis still needed for multi-node deployments. In-process EventTarget provides sub-ms latency for hot paths (vs ~17ms for Honker round-trip). +- **Cross-references**: ADR-047 ### OQ-27: How are schema migrations applied across all tenant DBs? @@ -245,6 +251,27 @@ architecture documents, organized by theme. - **Priority**: low - **Notes**: The adapter is ~100 lines and useful to anyone combining Drizzle with Honker. Publishing as `drizzle-honker` would benefit the community. Decision: start inside `@alkdev/storage`, extract later if there's demand. +### OQ-30: Composite event target for single-node hub deployments? + +- **Origin**: [honker-integration.md](honker-integration.md) +- **Status**: open +- **Priority**: medium +- **Notes**: POC 2 showed ~17ms median latency for Honker notify→listen vs sub-ms for in-process EventTarget. For single-node hubs, a composite that dispatches to both (in-process for speed, Honker for durability/cross-process) would be the ideal default. Design needed. + +### OQ-31: Consumer naming convention for durable stream subscriptions? + +- **Origin**: [honker-integration.md](honker-integration.md) +- **Status**: open +- **Priority**: medium +- **Notes**: Honker's `stream.subscribe(consumer)` requires a consumer name for offset tracking. The name must be stable across hub restarts (PID-based names don't survive restart). Need a convention: `{service}:{host}` or a configurable consumer group ID. + +### OQ-32: Drizzle Kit migration compatibility with Honker adapter? + +- **Origin**: [honker-integration.md](honker-integration.md) +- **Status**: open +- **Priority**: medium +- **Notes**: Drizzle Kit supports SQLite migrations but expects `better-sqlite3` or `libsql`. Need to verify `drizzle-kit push`/`drizzle-kit generate` works with the custom Honker adapter, or whether we need a custom migration runner. + ## ADR Impact | ADR | Resolves | Informs | @@ -255,7 +282,7 @@ architecture documents, organized by theme. | ADR-020 | OQ-24 | | | ADR-023 | OQ-14 | | | ADR-026 | OQ-15 | | -| ADR-033 | OQ-04, OQ-16, OQ-17, OQ-18 | | +| ADR-033 | OQ-04, OQ-16, OQ-17, OQ-18 | OQ-17 (superseded by ADR-048) | | ADR-034 | OQ-03, OQ-21 | OQ-25 | | ADR-035 | OQ-03 | | | ADR-038 | OQ-04 (moot) | OQ-17 (less pressure) | @@ -263,5 +290,8 @@ architecture documents, organized by theme. | ADR-041 | OQ-24 | | | ADR-042 | | OQ-24 | | ADR-043 | | | -| ADR-044 | OQ-19 (less pressure) | | -| ADR-045 | OQ-23 | OQ-20 | \ No newline at end of file +| ADR-044 | OQ-19 (less pressure) | OQ-29, OQ-32 | +| ADR-045 | OQ-23 | OQ-20 | +| ADR-046 | | OQ-17 | +| ADR-047 | OQ-26 | OQ-30 | +| ADR-048 | OQ-17 (updated), OQ-18 (updated), OQ-19 (updated) | | \ No newline at end of file diff --git a/docs/architecture/overview.md b/docs/architecture/overview.md index c0f2e75..e30baff 100644 --- a/docs/architecture/overview.md +++ b/docs/architecture/overview.md @@ -1,6 +1,6 @@ --- status: draft -last_updated: 2026-05-31 +last_updated: 2026-06-01 --- # @alkdev/storage — Overview @@ -24,6 +24,12 @@ organizations, api_keys, audit_logs) for multi-tenant authentication and authorization, and integrates with **Honker** for transactional pub/sub, event streams, and task queues within the same SQLite database. +Storage also provides a **HonkerEventTarget** adapter that bridges +`@alkdev/pubsub`'s `TypedEventTarget` interface to Honker's `notify`/`listen` +and `stream`/`subscribe` primitives, and **OperationSpec** generation from +table definitions so downstream hubs/spokes can register CRUD operations +directly into the `@alkdev/operations` registry. + ## Architecture ``` @@ -45,9 +51,11 @@ streams, and task queues within the same SQLite database. │ │ ├── identity/ → accounts, organizations, org_members, api_keys, audit_logs │ │ ├── metagraph/ → graph_types, node_types, edge_types, graphs, nodes, edges │ │ └── index.ts → barrel re-export +│ ├── utils/ → createSelectSchema, createInsertSchema, column mappings (folded from @alkdev/dbtype) │ ├── relations.ts → drizzle relational mappings │ ├── schema.ts → barrel re-export │ ├── adapter.ts → Drizzle-Honker session adapter +│ ├── event-target.ts → HonkerEventTarget (pubsub TypedEventTarget on Honker) │ └── client.ts → createSystemDatabase(), createTenantDatabase() └── test/ └── reference-modules.test.ts → Metagraph, bridge, crypto tests @@ -59,7 +67,7 @@ streams, and task queues within the same SQLite database. | ------------------------ | --------------------------------------- | --------------------------------------- | | `@alkdev/storage` | Graph schema types, Metagraph Module | `@alkdev/typebox` | | `@alkdev/storage/graphs` | Same as `.` — alias for the main export | Same as `.` | -| `@alkdev/storage/sqlite` | SQLite tables, relations, client, adapter | `@alkdev/drizzlebox`, `drizzle-orm`, `@russellthehipp/honker-node` | +| `@alkdev/storage/sqlite` | SQLite tables, relations, client, adapter, event-target, utils | `drizzle-orm`, `@russellthehippo/honker-node`, `@alkdev/pubsub` (peer) | The `pg/` subpath has been removed (ADR-038). SQLite via Honker is the sole database host. @@ -117,8 +125,10 @@ fixed — changes require a version bump and migration (ADR-029). | **Edge type** | A category of edge within a graph type. Defines the attribute schema and optionally restricts which node types can be source/target. Stored in the `edge_types` table. | | **Graph instance** | A concrete graph belonging to a graph type. Contains nodes and edges conforming to its type definitions. Stored in the `graphs` table. | | **Honker** | SQLite extension providing transactional pub/sub, durable event streams, task queues, advisory locks, and cron scheduling within the same `.db` file. | +| **HonkerEventTarget** | Adapter that implements `@alkdev/pubsub`'s `TypedEventTarget` interface on Honker's `notify`/`listen` and `stream`/`subscribe` primitives. Bridges pubsub to Honker for single-node and cross-process scenarios. | | **Consumer** | Code that imports `@alkdev/storage` (or a subpath) to define graph types and persist graph data. The hub, spokes, and other @alkdev packages are consumers. | -| **Repository layer** | ⚠️ Not yet implemented. The typed CRUD functions (insert, find, update, delete) that sit between consumer code and raw Drizzle queries. Performs schema validation before writes. | +| **Repository surface** | Not hand-written CRUD functions. Storage outputs `OperationSpec[]` — flat arrays describing CRUD and query operations for each table. The consumer (hub/spoke) imports these specs, registers handlers, and the operations runtime handles execution. | +| **Validation boundary** | The line where schema validation is enforced. In this package, validation happens in the Metagraph Module (at type definition time) and the bridge functions (at mutation time), NOT in the database. | | **Validation boundary** | The line where schema validation is enforced. In this package, validation happens in the Metagraph Module (at type definition time) and the repository layer (at mutation time), NOT in the database. | ## Design Decisions @@ -131,7 +141,10 @@ All design decisions are documented as ADRs in [decisions/](decisions/). | [002](decisions/002-metagraph-over-domain-tables.md) | Metagraph over domain-specific tables | 6 general-purpose tables serve all graph-shaped domains | | [003](decisions/003-typebox-module-as-api-surface.md) | TypeBox Module as API surface | `Type.Module()` replaces `SchemaBuilder`; `Metagraph.Import()` + `Type.Composite()` | | [004](decisions/004-injectable-clients-no-side-effects.md) | Injectable clients, no side effects | `createSystemDatabase(client)` / `createTenantDatabase(client)` take pre-created clients | -| [005](decisions/005-drizzle-plus-typebox-via-drizzlebox.md) | Drizzle + TypeBox via drizzlebox | Drizzle tables are single source of truth; drizzlebox generates TypeBox schemas | +| [005](decisions/005-drizzle-plus-typebox-via-drizzlebox.md) | Drizzle + TypeBox via local utils | Drizzle tables are single source of truth; `src/sqlite/utils/` generates TypeBox schemas (folded from @alkdev/drizzlebox) | +| [046](decisions/046-fold-drizzlebox-as-utils.md) | Fold @alkdev/drizzlebox as src/sqlite/utils | SQLite-only column mappings and schema generation co-located with tables | +| [047](decisions/047-honker-event-target.md) | HonkerEventTarget adapter | pubsub TypedEventTarget on Honker notify/listen and stream/subscribe | +| [048](decisions/048-operation-specs-as-repo-surface.md) | OperationSpecs as repository surface | Storage outputs OperationSpec[] not hand-written CRUD | | [006](decisions/006-enum-pattern-as-const-objects.md) | `as const` objects, not TypeScript enums | Avoids JSR slow-types; consistent pattern across codebase | | [007](decisions/007-no-comments-in-code.md) | No comments in code | Documentation lives in architecture docs and TypeBox descriptions | | [008](decisions/008-common-columns-pattern.md) | Common columns pattern | `id`, `metadata`, `createdAt`, `updatedAt` on every table | @@ -150,12 +163,14 @@ All design decisions are documented as ADRs in [decisions/](decisions/). | Package | Purpose | Layer | | -------------------- | ------------------------------------ | ------------------------ | | `@alkdev/typebox` | Runtime schema validation | graphs/ | -| `@alkdev/drizzlebox` | Generate TypeBox from Drizzle tables | sqlite/ | | `drizzle-orm` | ORM, table definitions, queries | sqlite/ | | `@russellthehippo/honker-node` | SQLite + pub/sub + queues + events | sqlite/ | +| `@alkdev/pubsub` | `TypedEventTarget` interface (peer) | sqlite/event-target.ts | -`@alkdev/typebox` and `@alkdev/drizzlebox` are npm packages (not yet on JSR). -JSR handles npm dependencies natively. +`@alkdev/typebox` is an npm package (not yet on JSR). JSR handles npm +dependencies natively. `@alkdev/pubsub` is a peer dependency — only needed +when using `HonkerEventTarget`. The `graphs/` module has zero dependencies +beyond `@alkdev/typebox`. **Ecosystem packages are not runtime dependencies of `@alkdev/storage`.** All ecosystem references describe consumer-side data shapes and integration @@ -169,22 +184,26 @@ patterns, not import dependencies. - Bridge functions (`moduleToDbSchema`, `validateNode`, `validateEdge`) - Reference graph type Modules (CallGraph, SecretGraph) - Crypto utility (AES-256-GCM + PBKDF2, `EncryptedDataSchema`) -- SQLite host: 6 metagraph tables + `actors` placeholder + Drizzle relations + client factory -- TypeBox select/insert schemas generated from Drizzle tables (drizzlebox) +- SQLite host: 6 metagraph tables (flat structure, pending reorganize) + Drizzle relations + client factory +- TypeBox select/insert schemas generated from Drizzle tables (current: `@alkdev/drizzlebox`; pending fold to `src/sqlite/utils/`, ADR-046) - Reference module tests (bridge functions, validation, Module composition) ### Not Yet Implemented | Gap | Priority | Notes | | ----------------------------------------- | ------------ | --------------------------------------------------------------------------------------------------- | -| Drizzle-Honker session adapter | High | ~100-line adapter wrapping `HonkerSQLiteSession` + `HonkerPreparedQuery`. ADR-044. | +| Drizzle-Honker session adapter | High | `HonkerSQLiteSession` + `HonkerPreparedQuery` + `HonkerSQLiteTransaction`. POC validated. ADR-044. | +| Fold dbtype Phase 0 → `src/sqlite/utils/` | High | SQLite-only column mappings + createSelectSchema/createInsertSchema. ADR-046. | | Identity tables (accounts, organizations, api_keys, audit_logs, organization_members) | High | Moving from hub into storage. ADR-041. | | Scoping columns on `graphs` table (`ownerId`, `projectId`) | High | ADR-042. | | Graph type `scope` column | High | ADR-043. | | Remove `actors` table and `pg/` directory | High | ADR-035 (actors), ADR-038 (pg). | -| `createSystemDatabase()` / `createTenantDatabase()` factories | High | Split from current `createSqliteDatabase()`. | -| Repository/CRUD layer | Medium | ⚠️ Typed insert, find, update, delete functions for graphs, nodes, edges. | +| `createSystemDatabase()` / `createTenantDatabase()` factories | High | Split from current `createSqliteDatabase()`. Accept Honker client. | +| Table restructure into subdirectories | High | `tables/metagraph/`, `tables/identity/`, update relative imports. | +| HonkerEventTarget adapter | High | pubsub `TypedEventTarget` on Honker primitives. POC validated. ADR-047. | +| OperationSpec generation from tables | Medium | `OperationSpec[]` per table for CRUD operations. ADR-048. | | ACL graph type | Medium | Access control as a metagraph. ADR-034. | +| Domain-specific native-column tables | Low | CallGraph, SecretGraph, etc. with native columns alongside JSON attributes. | | Task graph type | Low | Informed by `@alkdev/taskgraph`'s schemas. | | Graphology bridge | Low | `moduleToGraphology()` and `fromGraphologyExport()` — Phase 4. | @@ -204,10 +223,11 @@ other way. @alkdev/flowgraph ← call graph schema, operation graph, workflow templates ↑ (depends on: @alkdev/operations [peer], @alkdev/typebox) @alkdev/taskgraph ← task dependency graph schema, cost-benefit analysis - (depends on: @alkdev/typebox) + (depends on: @alkdev/typebox) @alkdev/storage ← YOU ARE HERE — typed graph persistence + identity - (depends on: @alkdev/typebox, @alkdev/drizzlebox, drizzle-orm, honker-node) + (depends on: @alkdev/typebox, drizzle-orm, honker-node) + (peer dep: @alkdev/pubsub for TypedEventTarget type) ↑ ↑ | | @@ -218,7 +238,9 @@ Hub / Spoke Any consumer that needs ### Event-Driven Architecture with Honker The @alkdev platform is event-driven. Honker provides the transport mechanism -within each SQLite database: +within each SQLite database. Storage provides the `HonkerEventTarget` adapter +so consumers can use `@alkdev/pubsub`'s `TypedEventTarget` interface regardless +of whether events route in-process, through Honker, or over WebSocket. | Concern | Honker Primitive | Example | |---------|------------------|---------| @@ -230,7 +252,17 @@ within each SQLite database: A single Drizzle transaction via the Honker adapter can insert graph data AND publish a notification AND enqueue a side-effect job — all committing atomically. -No dual-write problem between data and events. +No dual-write problem between data and events. POC 4 confirmed: `tx.notify()` +only fires on commit; on rollback, both the data write and the notification are +suppressed. + +### Latency Consideration + +POC 2 measured ~17ms median latency for Honker `notify`→`listen` within a +single process. For hot-path call protocol request/response where sub-ms +latency matters, pair the Honker event target with an in-process `EventTarget` +(pubsub's default). A composite pattern (dispatch to both) provides both +in-process speed and Honker durability/cross-process coordination. ### What Comes from Where @@ -241,26 +273,32 @@ No dual-write problem between data and events. | Access control | `@alkdev/operations` | Storage's AclGraph mirrors `AccessControl` schema as graph structure | | Call graph schema | `@alkdev/flowgraph` | Storage persists in-memory shapes to the database | | Task graph schema | `@alkdev/taskgraph` | Storage persists task dependency shapes | -| Event transport | Honker (within storage) | Replaces `@alkdev/pubsub`'s Redis transport for single-node deployments | +| Event transport | Honker (within storage) | `HonkerEventTarget` bridges pubsub `TypedEventTarget` to Honker primitives | +| Pubsub interface | `@alkdev/pubsub` | Storage provides `HonkerEventTarget` adapter; does not replace pubsub | -### Repository Layer Bridging Pattern +### Repository Surface as OperationSpecs -The repository layer in `@alkdev/storage` provides typed CRUD — no -`@alkdev/operations` dependency. A consumer-side bridging module wires CRUD -functions into the operations registry when needed. +Storage does not ship a "repository layer" of hand-written CRUD functions. +Instead, it outputs `OperationSpec[]` — flat arrays describing CRUD and query +operations for each table. The consumer (hub/spoke) imports these specs, +registers them in the `@alkdev/operations` registry along with handlers, and +the operations runtime handles execution, call protocol, and subscriptions. + +Storage depends on `@alkdev/operations` only as a type-level peer dependency +(for the `OperationSpec` type). No circular dependency. ``` -@alkdev/storage → defines types + tables + CRUD (no operations dependency) +@alkdev/storage → defines types + tables + OperationSpec[] (type-only operations dep) @alkdev/operations → defines call protocol + registry (no storage dependency) -Consumer (hub / adapter) → imports both, generates operations from schemas +Consumer (hub / spoke) → imports both, registers specs + handlers ``` ### Avoiding Circular Dependencies Neither `@alkdev/storage` nor `@alkdev/operations` should depend on each -other directly. Storage defines the schema types and database tables; operations -defines the call protocol and execution model. The consumer imports both and -bridges them. +other at runtime. Storage defines the schema types, database tables, and +operation contract definitions; operations defines the call protocol and +execution model. The consumer imports both and bridges them. ## Open Questions @@ -270,7 +308,7 @@ questions affecting this package: - **OQ-04**: Should the repository layer be host-specific or host-agnostic? (resolved: single host, question moot) - **OQ-22**: How are ACL graph instances created and managed? (open, tenant DB model simplifies: likely one per tenant DB) - **OQ-23**: BelongsToEdge derived or primary? (resolved: derived — ADR-045) -- **OQ-26**: Can Honker replace `@alkdev/pubsub`'s Redis transport for single-node deployments? (open) +- **OQ-26**: Can Honker replace `@alkdev/pubsub`'s Redis transport for single-node deployments? (resolved: yes — HonkerEventTarget adapter, POC validated. Redis still needed for multi-node. See ADR-047.) - **OQ-27**: How are schema migrations applied across all tenant DBs? (open) - **OQ-28**: How does cross-tenant delegation work with separate DBs? (open) diff --git a/docs/architecture/sqlite-host.md b/docs/architecture/sqlite-host.md index 0ff57ea..0c8cd12 100644 --- a/docs/architecture/sqlite-host.md +++ b/docs/architecture/sqlite-host.md @@ -1,13 +1,14 @@ --- status: draft -last_updated: 2026-05-31 +last_updated: 2026-06-01 --- # SQLite Host The SQLite database host for `@alkdev/storage`. Uses Drizzle ORM with Honker for database operations, pub/sub, event streams, and task queues. TypeBox -schemas are auto-generated from Drizzle table definitions via `@alkdev/drizzlebox`. +schemas are generated from Drizzle table definitions via `src/sqlite/utils/` +(folded from `@alkdev/dbtype`/`@alkdev/drizzlebox`, ADR-046). ## Overview @@ -18,11 +19,13 @@ The SQLite host provides: 2. **Identity tables** — accounts, organizations, organization_members, api_keys, audit_logs for multi-tenant authentication and authorization (ADR-041) 3. **Drizzle relations** for the relational query API -4. **TypeBox schemas** auto-generated from Drizzle tables (select/insert - validation) via `@alkdev/drizzlebox` +4. **TypeBox schemas** generated from Drizzle tables (select/insert + validation) via `src/sqlite/utils/` (folded from @alkdev/drizzlebox, ADR-046) 5. **Drizzle-Honker adapter** — thin session adapter for Honker integration - (ADR-044) -6. **Client factories** — `createSystemDatabase(client)` and + (ADR-044, POC validated) +6. **HonkerEventTarget** — pubsub `TypedEventTarget` on Honker primitives + (ADR-047, POC validated) +7. **Client factories** — `createSystemDatabase(client)` and `createTenantDatabase(client)` for the system/tenant DB model (ADR-040) ## Package Structure @@ -47,10 +50,17 @@ src/sqlite/ │ │ ├── edges.ts # edges table + select/insert schemas │ │ └── index.ts # barrel re-export │ └── index.ts # barrel re-export -├── relations.ts # Drizzle relational mappings +├── utils/ # folded from @alkdev/dbtype Phase 0 (ADR-046) +│ ├── schema.ts # createSelectSchema, createInsertSchema, createUpdateSchema +│ ├── column.ts # Column→TypeBox mappings (SQLite-only dispatch) +│ ├── types.ts # Public + internal TypeScript interfaces +│ ├── constants.ts # Integer range constants +│ └── utils.ts # isColumnType, isWithEnum, type helpers +├── relations.ts # Drizzle relational mappings ├── adapter.ts # Drizzle-Honker session adapter -├── schema.ts # re-exports all tables + relations -└── client.ts # createSystemDatabase(), createTenantDatabase() +├── event-target.ts # HonkerEventTarget (pubsub TypedEventTarget on Honker) +├── schema.ts # re-exports all tables + relations +└── client.ts # createSystemDatabase(), createTenantDatabase() ``` ## Common Columns @@ -339,8 +349,11 @@ db.transaction((tx) => { | [041](decisions/041-identity-tables-in-storage.md) | Identity tables in storage | accounts, organizations, api_keys, audit_logs | | [042](decisions/042-scoping-columns-on-graphs.md) | Scoping columns on graphs | `ownerId`, `projectId` on `graphs` table | | [043](decisions/043-graph-type-scope.md) | Graph type scope | `system` / `tenant` / `user` scope on `graph_types` | -| [044](decisions/044-drizzle-honker-adapter.md) | Drizzle-Honker adapter | ~100-line session adapter | +| [044](decisions/044-drizzle-honker-adapter.md) | Drizzle-Honker adapter | ~100-line session adapter, POC validated | | [045](decisions/045-org-members-authoritative-belongsto-derived.md) | org_members authoritative | SQL table is source of truth; BelongsToEdge is derived | +| [046](decisions/046-fold-drizzlebox-as-utils.md) | Fold drizzlebox as utils | SQLite-only column mappings in src/sqlite/utils/ | +| [047](decisions/047-honker-event-target.md) | HonkerEventTarget | pubsub TypedEventTarget on Honker | +| [048](decisions/048-operation-specs-as-repo-surface.md) | OperationSpecs as repo surface | Table-defined operation contracts | | [019](decisions/019-json-text-for-schema-columns.md) | JSON text for schema columns | SQLite uses `text` with JSON mode | | [020](decisions/020-no-nodetypeid-on-nodes.md) | No nodeTypeId on nodes | Node type enforced at application layer | | [022](decisions/022-composite-fks-for-node-references.md) | Composite FKs for node refs | Edges reference `(graphId, sourceNodeKey)` | diff --git a/docs/research/pivot-honker-sqlite-adapter.md b/docs/research/pivot-honker-sqlite-adapter.md new file mode 100644 index 0000000..890a796 --- /dev/null +++ b/docs/research/pivot-honker-sqlite-adapter.md @@ -0,0 +1,743 @@ +--- +status: draft +created: 2026-06-01 +last_updated: 2026-06-01 +--- + +# Pivot: Honker SQLite Adapter, dbtype Fold, and Event Target Integration + +Research document for the architecture pivot from `@libsql/client` to Honker, +folding `@alkdev/dbtype` Phase 0 into `src/utils/`, and adding a pubsub +`HonkerEventTarget`. + +## Motivation + +1. Honker provides SQLite + pub/sub + streams + queues + locks + scheduler in + a single `.db` file via a Rust NAPI binding. This replaces both + `@libsql/client` (SQLite only) and the conceptual need for Redis. +2. `@alkdev/dbtype` Phase 0 is already consumed as `@alkdev/drizzlebox`. With + SQLite as the sole target, the multi-dialect column mappings are dead + weight. Folding the SQLite-relevant subset into `src/utils/` eliminates an + external dependency and co-locates utility code with the tables it + derives schemas from. +3. `@alkdev/pubsub` defines a `TypedEventTarget` interface that Honker's + primitives (`notify`/`listen`, `stream`/`stream.subscribe`) map to + naturally. A `HonkerEventTarget` adapter lets storage's downstream + consumers use the same pubsub API regardless of whether events route + in-process, through Honker, through Redis, or over WebSocket. +4. The hub-spoke model needs events for call protocol routing, cache + invalidation, flowgraph reactive updates, and cross-process coordination. + Honker handles all of these within SQLite. + +## Architecture Changes + +### Current Structure + +``` +src/ +├── graphs/ # Metagraph Module + bridge (no db deps) +│ ├── modules/ # CallGraph, SecretGraph, Metagraph +│ ├── bridge.ts # moduleToDbSchema, validateNode, validateEdge +│ ├── crypto.ts # encrypt, decrypt, generateEncryptionKey +│ └── mod.ts +├── pg/ # Placeholder (to be removed) +└── sqlite/ # SQLite host + ├── client.ts # createSqliteDatabase(@libsql/client) + ├── mod.ts + ├── relations.ts + ├── schema.ts + └── tables/ # Flat: graphTypes, nodeTypes, edgeTypes, graphs, nodes, edges, actors +``` + +Dependencies: `@alkdev/drizzlebox` (external npm), `@libsql/client`, `drizzle-orm` + +### Target Structure + +``` +src/ +├── graphs/ # Unchanged +│ ├── modules/ +│ ├── bridge.ts +│ ├── crypto.ts +│ └── mod.ts +└── sqlite/ + ├── adapter.ts # NEW: Drizzle-Honker session adapter + ├── event-target.ts # NEW: HonkerEventTarget implements pubsub TypedEventTarget + ├── client.ts # UPDATED: createSystemDatabase(client), createTenantDatabase(client) + ├── mod.ts # UPDATED: re-exports + ├── relations.ts # UPDATED: remove actors relations + ├── schema.ts # UPDATED: remove actors, remove pg refs + ├── utils/ # NEW: folded from @alkdev/dbtype Phase 0 (SQLite only) + │ ├── schema.ts # createSelectSchema, createInsertSchema, createUpdateSchema + │ ├── column.ts # Column→TypeBox mappings (SQLite-only dispatch) + │ ├── types.ts # Public TypeScript interfaces (merged from schema.types + schema.types.internal) + │ ├── constants.ts # Integer range constants + │ └── utils.ts # isColumnType, isWithEnum, type helpers + └── tables/ # Restructured into subdirectories + ├── common.ts # shared column definitions, ACTOR_TYPE removed + ├── metagraph/ # graph_types, node_types, edge_types, graphs, nodes, edges + │ └── index.ts + └── identity/ # NEW: accounts, organizations, org_members, api_keys, audit_logs + └── index.ts +``` + +Removed: `src/pg/`, `actors` table, `@alkdev/drizzlebox` external dep, +`@libsql/client` dep. + +New deps: `@alkdev/pubsub` (peer dep for `TypedEventTarget` type only), +`honker` (replaces `@libsql/client`). + +## 1. Fold dbtype Phase 0 → `src/sqlite/utils/` + +### What to fold + +Only SQLite-relevant code from `@alkdev/dbtype/src/`: + +| Source (dbtype) | Target (storage) | Changes | +|-----------------|-------------------|---------| +| `schema.ts` | `utils/schema.ts` | Remove `PgEnum` import/handling, remove `isPgEnum` calls. Remove `handleEnum()`. Keep `createSelectSchema`, `createInsertSchema`, `createUpdateSchema`. | +| `column.ts` | `utils/column.ts` | Strip all non-SQLite dispatch branches. Remove PG, MySQL, SingleStore type imports and `isColumnType` branches. Keep: `SQLiteInteger`, `SQLiteReal`, `SQLiteText` handling. Keep generic dispatch (`dataType`-based) as fallback. Keep `literalSchema`, `jsonSchema`, `bufferSchema`, `mapEnumValues`. | +| `column.types.ts` | `utils/types.ts` (merged) | Strip non-SQLite type branches. Keep `SQLiteInteger` → `t.TInteger`. Keep generic fallback types. | +| `schema.types.ts` | `utils/types.ts` (merged) | Remove `PgEnum` overloads. | +| `schema.types.internal.ts` | `utils/types.ts` (merged) | Keep `Conditions`, `BuildRefine`, `BuildSchema`, `NoUnknownKeys`. | +| `constants.ts` | `utils/constants.ts` | Keep as-is (ranges used by SQLite integer/real/bigiint mappings). | +| `utils.ts` | `utils/utils.ts` | Remove `PgEnum` type alias. Keep `isColumnType`, `isWithEnum`. Keep `JsonSchema`, `BufferSchema` types. Remove unused type helpers. | + +### What NOT to fold + +- PG/MySQL/SingleStore column type handlers (dead code for SQLite-only) +- `isPgEnum` and `handleEnum` (PostgreSQL-specific) +- `createSchemaFactory` (no known consumer in storage; can add later if needed) +- The `scripts/probe-e2e.ts` (dbtype Phase 1 POC, not relevant) + +### Import changes in existing table files + +Every table file currently imports from `@alkdev/drizzlebox`: + +```ts +// Before +import { createInsertSchema, createSelectSchema } from "@alkdev/drizzlebox"; + +// After +import { createInsertSchema, createSelectSchema } from "../utils/schema.ts"; +``` + +The `createSelectSchema` and `createInsertSchema` functions produce the same +TypeBox schemas — the API surface is identical, the implementation just +excludes non-SQLite branches. + +### Deno.json import map changes + +```jsonc +// Before +"imports": { + "@alkdev/drizzlebox": "npm:@alkdev/drizzlebox", + "@libsql/client": "npm:@libsql/client", + // ... +} + +// After (remove drizzlebox and libsql, add honker and pubsub peer) +"imports": { + "honker": "npm:honker", + "@alkdev/pubsub": "npm:@alkdev/pubsub", + // ... +} +``` + +Note: `@alkdev/pubsub` is a peer dependency for the `TypedEventTarget` type. +It is only needed by consumers who use the `HonkerEventTarget`. The +`src/graphs/` module remains zero-dep. + +## 2. Drizzle-Honker Session Adapter (`src/sqlite/adapter.ts`) + +### Problem + +Drizzle ORM expects a session that implements `SQLiteSession<'sync'>`. Honker +provides `Database.query(sql, params)` for reads and `Transaction.execute(sql, +params)` for writes. These don't implement Drizzle's session interface. + +### Adapter Architecture + +``` +Consumer code + ├── db.select().from(graphs)... → Drizzle query builder + │ └── HonkerPreparedQuery.run/all/get/values + │ └── honkerDb.query(sql, params) / tx.execute(sql, params) + ├── db.$client.notify('channel', p) → Honker ephemeral pub/sub + ├── db.$client.stream('events') → Honker durable stream + ├── db.$client.queue('jobs') → Honker work queue + └── db.$client.listen('channel') → Honker listener +``` + +Three adapter classes: + +| Class | Extends | Purpose | +|-------|---------|---------| +| `HonkerSQLiteSession` | `SQLiteSession<'sync'>` | Routes Drizzle queries to `honkerDb.query()` / `tx.execute()` | +| `HonkerPreparedQuery` | `SQLitePreparedQuery` | Implements `run()`, `all()`, `get()`, `values()` via Honker | +| `HonkerSQLiteTransaction` | `SQLiteTransaction<'sync'>` | Drizzle transaction with `$honkerTx` for Honker primitives | + +### Accessing Honker from Drizzle + +- `db.$client` — the Honker `Database` instance (outside transactions) +- `tx.$honkerTx` — the Honker `Transaction` instance (inside Drizzle tx callbacks) + +```ts +db.transaction((tx) => { + tx.insert(schema.nodes).values({ graphId, key: "call-1", attributes: {} }).run(); + tx.$honkerTx.notify("nodes:created", { graphId, key: "call-1" }); +}); +``` + +Both data write and notification commit atomically. + +### Known challenges (need POC validation) + +1. **No `lastInsertRowid` from Honker `execute()`** — Drizzle's `run()` + needs `lastInsertRowid` for auto-increment inserts. Honker's JS API + doesn't expose it. POC 1 confirmed workaround: `tx.query("SELECT last_insert_rowid() as id")` returns the rowid. + +2. **Object rows only** — Honker `query()` returns + `Array>`. Drizzle's `values()` mode expects raw + arrays. The adapter must convert via `Object.values()` with column + order from the query metadata. + +3. **Drizzle internal API dependency** — The adapter imports from + `drizzle-orm/sqlite-core/session`, which is not a guaranteed-stable API. + POC 1 confirmed all needed classes are accessible and extendable. + +4. **Session constructor shape** — POC 1 confirmed: `SQLiteSession` takes + `(mode, executeMethod, query, cache?, queryMetadata?)`. `BaseSQLiteDatabase` + is accessible from `drizzle-orm/sqlite-core/db`. The `LibSQLSession` in + `drizzle-orm/libsql/session` is a working reference implementation. + +5. **`:memory:` databases don't work** — Honker's reader pool creates + separate in-memory databases. File-based paths must be used. POC 1 + confirmed this. Not a real limitation — production uses file paths, + tests use temp files. + +## 3. HonkerEventTarget (`src/sqlite/event-target.ts`) + +### Interface + +Implements `@alkdev/pubsub`'s `TypedEventTarget`: + +```ts +interface HonkerEventTargetOptions { + db: Database; + mode: "ephemeral" | "durable"; + streamName?: string; + prefix?: string; +} + +function createHonkerEventTarget(options: HonkerEventTargetOptions): HonkerEventTarget +``` + +### Ephemeral mode mapping + +Maps to Honker's `notify()` / `listen()`: + +```ts +addEventListener(type, callback): + if (!listeners.has(type)): + honkerListener = db.listen(type) + // start async loop: for await (notification of honkerListener) + // dispatch CustomEvent with notification as detail + listeners.set(type, callback) + +dispatchEvent(event): + db.notify(event.type, event.detail) // or tx.notify() inside transaction + +removeEventListener(type, callback): + listeners.delete(type, callback) + if (no more listeners for type): + honkerListener.close() +``` + +### Durable mode mapping + +Maps to Honker's `Stream.publish()` / `Stream.subscribe()`: + +```ts +addEventListener(type, callback): + if (!subscriptions.has(type)): + stream = db.stream(streamName ?? type) + subscription = stream.subscribe(consumerName) + // start async loop: for await (event of subscription) + // dispatch CustomEvent with event.payload as detail + subscriptions.set(type, { stream, subscription, callback }) + +dispatchEvent(event): + db.stream(streamName).publish(event.detail) + // or stream.publishTx(tx) inside transaction + +removeEventListener(type, callback): + subscriptions.delete(type, callback) + if (no more subscriptions for type): + subscription.close() +``` + +### Topic routing considerations + +pubsub uses `type:id` composite topics (e.g., `"call.responded:req-123"`). +Honker channels are flat strings. The mapping is 1:1 — no transformation +needed. But: + +- **Ephemeral**: Each unique `type:id` gets its own `db.listen()` call. For + high-cardinality topics (many request IDs), this could mean many concurrent + listeners. POC 3 confirmed multiple concurrent listeners work. + +- **Durable**: Streams are name-keyed, not topic-keyed. A single stream + (e.g., `"call-protocol"`) carries all call events. The adapter would need + to handle topic filtering client-side (dispatch only to matching listeners) + rather than using separate streams per topic. + +- **Suggested split**: Call protocol events → durable stream. Cache + invalidation, UI push → ephemeral channels. The configuration for which + topics go where is a consumer decision, not built into the adapter. + +- **Latency consideration**: POC 2 measured ~17ms median for + `notify`→`listen` round-trip. For hot-path call protocol request/response + in a single process, pair the Honker event target with an in-process + `EventTarget` (pubsub's default) for sub-ms latency. The Honker event + target provides durability and cross-process coordination; the default + EventTarget provides speed. A composite pattern (dispatch to both) could + serve both needs. + +### Consumer name strategy + +For durable mode, each consumer needs a unique name for offset tracking. +Suggested pattern: `{processId}:{topic}` or a configurable consumer group ID. + +## 4. Client Factory Update (`src/sqlite/client.ts`) + +### Before + +```ts +import { drizzle, type LibSQLDatabase } from "drizzle-orm/libsql"; +import type { Client } from "@libsql/client"; + +export function createSqliteDatabase(client: Client): LibSQLDatabase { + return drizzle(client, { schema }); +} +``` + +### After + +The factory accepts a pre-created Honker `Database` instance (injectable +client pattern, per ADR-004). It constructs the Drizzle-Honker adapter +internally. + +```ts +import type { Database as HonkerDatabase } from "honker"; + +export type SqliteDatabase = BaseSQLiteDatabase<'sync', SqliteSchema>; + +export function createSystemDatabase(client: HonkerDatabase): SqliteDatabase { + return createDrizzleWithAdapter(client, { schema }); +} + +export function createTenantDatabase(client: HonkerDatabase): SqliteDatabase { + return createDrizzleWithAdapter(client, { schema }); +} +``` + +`createDrizzleWithAdapter` is the internal factory that constructs the +`HonkerSQLiteSession` and wraps it in a `BaseSQLiteDatabase`. This may +require using `drizzle-orm/sqlite-core` internals rather than the +`drizzle-orm/libsql` high-level wrapper. + +## 5. Table Restructure + +### Remove actors table + +Per existing ADRs (035, 041), actors are being replaced by ACL nodes in +graph instances. Remove `src/sqlite/tables/actors.ts` and all exports. + +### Reorganize into subdirectories + +``` +src/sqlite/tables/ +├── common.ts # commonCols (UPDATED: remove ACTOR_TYPE) +├── metagraph/ +│ ├── index.ts # barrel +│ ├── graph-types.ts # renamed from graphTypes.ts +│ ├── node-types.ts # renamed from nodeTypes.ts +│ ├── edge-types.ts # renamed from edgeTypes.ts +│ ├── graphs.ts +│ ├── nodes.ts +│ └── edges.ts +└── identity/ + ├── index.ts # barrel + ├── accounts.ts + ├── organizations.ts + ├── org-members.ts + ├── api-keys.ts + └── audit-logs.ts +``` + +### Import updates in table files + +All table files currently importing `@alkdev/drizzlebox` update to +`../utils/schema.ts` (or `../../utils/schema.ts` depending on depth). + +The `commonCols` import path changes from `./common.ts` to `../common.ts` +for metagraph/identity subdirectories. + +## 6. Domain-Specific Tables (Future, documented for scope) + +The current metagraph tables store node/edge attributes as JSON. For known +graph types (CallGraph, SecretGraph, AclGraph), native columns give better +type safety and query performance. + +This is **not part of the initial pivot**. But the pivot should not prevent +it. The folded `utils/` code already provides `createInsertSchema` / +`createSelectSchema` which derives TypeBox schemas from Drizzle tables with +native columns. When domain tables are added later, they follow the same +pattern as the metagraph tables. + +### Where OperationSpecs fit + +Domain tables would come with auto-generated `OperationSpec[]` — flat arrays +describing CRUD operations for each table. The hub/spoke imports these specs, +registers them in the operations registry, and the call protocol handles +execution. + +This means storage's "repository layer" is not hand-written query functions. +It's **table definitions + operation contract definitions**. The consumer +(hub/spoke) provides the execution layer. + +```ts +// In storage: table definition + operation contracts +export const callNodes = sqliteTable("call_nodes", { ... }); +export const callNodeOperations: OperationSpec[] = [ + { name: "create", namespace: "call_nodes", type: "mutation", ... }, + { name: "find", namespace: "call_nodes", type: "query", ... }, + // ... +]; + +// In hub: register operations +for (const spec of callNodeOperations) { + registry.registerSpec(spec); + registry.registerHandler(specId, (input) => db.insert(callNodes).values(input).returning()); +} +``` + +The OperationSpec type is straightforward (`@alkdev/operations` peer dep for +type only). Storage defines the contract; the hub provides the handler. + +## 7. Identity Tables (New, documented for scope) + +Per `sqlite-host.md` and ADR-040/041, these live in the system database: + +| Table | Key Columns | Purpose | +|-------|-------------|---------| +| `accounts` | id, email, name, accessLevel | User accounts | +| `organizations` | id, slug, name, ownerId | Tenant organizations | +| `org_members` | id, orgId, accountId, level | Membership junction | +| `api_keys` | id, keyHash, ownerId, name | API key auth | +| `audit_logs` | id, actorType, actorId, action, resource | Append-only audit | + +These need actual implementation (currently only spec in architecture docs). +They're part of the pivot scope but can be implemented after the adapter and +utils fold. + +## 8. Event Pipeline: Operations → Flowgraph → Storage + +### How the pieces connect after the pivot + +``` +[operations: OperationRegistry] + │ + │ call.requested / call.responded / call.completed / call.aborted / call.error + ▼ +[pubsub: createPubSub({ eventTarget: honkerEventTarget })] + │ + ├──→ [flowgraph: WorkflowReactiveRoot.append(event)] + │ └── signal propagation → UI updates + │ + ├──→ [storage: db.stream("call-protocol").publish(event)] + │ └── durable replay for crash recovery + │ + └──→ [spoke sync: via WebSocket or Honker fan-out] + └── cross-process coordination +``` + +The `HonkerEventTarget` is the single integration point that makes this work. +Operations doesn't need to know about Honker. Flowgraph doesn't need to know +about Honker. They both use `@alkdev/pubsub`'s `TypedEventTarget`. The +`HonkerEventTarget` is just another adapter in pubsub's existing portfolio +(alongside Redis, WebSocket, Worker). + +### Hub-spoke event routing + +``` +Hub (system.db + tenant-{orgId}.db) + ├── HonkerEventTarget (durable: call-protocol stream) + ├── WebSocketServerEventTarget (spoke fan-out) + └── In-process EventTarget (local subscribers) + +Spoke (tenant-{orgId}.db) + ├── HonkerEventTarget (ephemeral: local channels) + ├── WebSocketClientEventTarget (hub connection) + └── In-process EventTarget (local subscribers) +``` + +Both hub and spoke use the same code. Same `createPubSub({ eventTarget })`. +Different event target instance determines the routing. + +## POC Plan + +Before committing to the full pivot, validate the critical path with targeted +POCs. Each POC answers one specific hard question. + +### POC 1: Drizzle queries through Honker + +**Question**: Can we construct a `BaseSQLiteDatabase` that routes queries +through Honker's `Database.query()` and `Transaction.execute()`? + +**Approach**: +1. Import `SQLiteSession`, `SQLitePreparedQuery`, `SQLiteTransaction` from + `drizzle-orm/sqlite-core/session`. +2. Implement `HonkerSQLiteSession` that delegates to `Database.query()`. +3. Implement `HonkerPreparedQuery` with `run()`, `all()`, `get()`. +4. Wire it into a `BaseSQLiteDatabase` instance. +5. Run a simple `db.select().from(graphs)` against a Honker-opened `.db` file. + +**Hard fail conditions**: +- Drizzle's `SQLiteSession` requires constructor arguments we can't provide + from Honker's primitives. +- The `BaseSQLiteDatabase` constructor requires internals not exported from + `drizzle-orm/sqlite-core`. +- `lastInsertRowid` is required and no workaround exists. + +### POC 2: Honker pub/sub from same process + +**Question**: Does `db.notify()` → `db.listen()` work within the same Node.js +process? What's the latency? + +**Approach**: +1. Open a Honker database. +2. Start `db.listen("test-channel")`. +3. Call `db.notify("test-channel", { hello: "world" })`. +4. Verify the listener receives the notification. +5. Measure round-trip latency (10,000 iterations). + +**Hard fail conditions**: +- Same-process `notify`/`listen` doesn't work (would require two DB instances + or fallback to in-process EventTarget). +- Latency exceeds 10ms median (would be too slow for call protocol + request/response). + +### POC 3: Concurrent listeners and stream subscriptions + +**Question**: Can multiple `Listener` and `StreamSubscription` instances +coexist on the same `Database`? Do they interleave correctly? + +**Approach**: +1. Open a Honker database. +2. Start two `db.listen()` calls on different channels. +3. Start a `db.stream("test-stream").subscribe("consumer-1")`. +4. Publish to both channels and the stream concurrently. +5. Verify all three consumers receive their events without blocking each + other. + +**Hard fail conditions**: +- Only one `Listener` can be active at a time (would require serialization). +- `Listener.next()` blocks the JS event loop (would prevent concurrent + subscriptions). + +### POC 4: Transactional notify + data write + +**Question**: Does `tx.notify()` + `tx.execute()` in the same transaction +atomically commit both the data write and the notification? + +**Approach**: +1. Open a Honker database. +2. Start `db.listen("test-channel")`. +3. Begin a transaction: `const tx = db.transaction()`. +4. Execute a data write: `tx.execute("INSERT INTO test ...")`. +5. Call `tx.notify("test-channel", { inserted: true })`. +6. Rollback the transaction. +7. Verify the listener does NOT receive the notification. +8. Repeat with commit instead of rollback. +9. Verify the listener DOES receive the notification after commit. + +**Hard fail conditions**: +- Notifications fire immediately on `tx.notify()` regardless of commit/rollback + (breaks transactional outbox semantics). + +### POC file structure + +``` +poc/ +├── README.md # POC objectives, how to run +├── 01-drizzle-honker-adapter.ts # POC 1 +├── 02-same-process-pubsub.ts # POC 2 +├── 03-concurrent-listeners.ts # POC 3 +├── 04-transactional-notify.ts # POC 4 +└── shared/ + └── honker-helpers.ts # Shared setup/teardown utilities +``` + +Each POC is a standalone script that can be run with `deno run` or `node`. +The POCs use the actual `honker` npm package from the workspace. + +## ADR Updates Required + +| ADR | Action | Reason | +|-----|--------|--------| +| ADR-018 (dbtype integration post-v1) | **Revise** | No longer deferred — Phase 0 folds in now as `src/sqlite/utils/` | +| ADR-038 (SQLite-first, PG removed) | **Confirm** | Pivot confirms this. Remove `src/pg/` and PG imports from deno.json. | +| ADR-039 (Honker as SQLite extension) | **Confirm** | Pivot is the implementation of this decision. | +| ADR-044 (Drizzle-Honker adapter) | **Update** | Add specifics about session/PreparedQuery/Transaction classes. | +| ADR-005 (drizzle + typebox via drizzlebox) | **Revise** | Drizzlebox is no longer an external dep. It's `src/sqlite/utils/`. | +| ADR-019 (JSON text for schema columns) | **Confirm** | Still relevant for metagraph attributes. Domain tables add native columns alongside JSON. | +| ADR-033 (JSON path queries for v1) | **Revise** | Repository layer is now "operation specs," not hand-written CRUD. The query approach (JSON path vs native) still applies, but the output format changes. | +| New ADR | Fold dbtype Phase 0 as utils | Records the decision to fold SQLite mappings into `src/sqlite/utils/`. | +| New ADR | HonkerEventTarget adapter | Records the decision to implement pubsub TypedEventTarget on Honker primitives. | +| New ADR | Operation specs as repository surface | Records that storage outputs OperationSpec[], not hand-written query functions. | + +## Architecture Doc Updates Required + +| Document | Changes | +|----------|---------| +| `overview.md` | Subpath exports (`./pg` removed), dependency list, module structure, "what's not done" section, system/tenant DB model | +| `forward-look.md` | dbtype section (folded, not deferred), repository layer strategy (operations, not CRUD), pointer abstraction (still deferred) | +| `honker-integration.md` | Add HonkerEventTarget section, update adapter section with verified API, add event pipeline diagram, resolve OQ-26 (Honker replaces Redis for single-node) | +| `sqlite-host.md` | Table structure changes (subdirectories, actors removed, identity tables), client factory signature, Honker import instead of libsql | +| `metagraph-module.md` | No changes (Module format is independent of storage host) | +| `schema-evolution.md` | Minor: `createSelectSchema`/`createInsertSchema` now come from local utils, not external dep | +| `open-questions.md` | Resolve OQ-17 (operations, not CRUD), OQ-18 (auto-generated specs from tables), OQ-26 (Honker replaces Redis). Update OQ-19 (bridge package location → it's storage itself). | +| `encrypted-data.md` | No changes | + +## Implementation Order (post-POC) + +If POCs pass, the implementation sequence should be: + +1. **Fold dbtype → `src/sqlite/utils/`** — import path changes only, no + behavioral changes. Run existing tests to verify. +2. **Remove `src/pg/` and actors table** — clean up dead code. Update deno.json + imports, remove `postgres` and PG-related entries. +3. **Restructure tables into subdirectories** — move files, update relative + imports, update `schema.ts` and `relations.ts`. +4. **Implement `adapter.ts`** — Drizzle-Honker session adapter based on POC + 1 findings. +5. **Update `client.ts`** — `createSystemDatabase()` / `createTenantDatabase()` + taking Honker `Database` instances. +6. **Implement `event-target.ts`** — `HonkerEventTarget` based on POC 2-4 + findings. +7. **Implement identity tables** — `src/sqlite/tables/identity/`. +8. **Update architecture docs** — bring all documents in line with the new + structure. +9. **Add domain tables** (future, not in initial pivot) — native-column + tables for CallGraph, SecretGraph, etc. with OperationSpec[] output. + +Steps 1-3 are mechanical (no new behavior). Steps 4-5 depend on POC 1. +Step 6 depends on POCs 2-4. Step 7 can proceed independently. Steps 8-9 +are follow-ups. + +## POC Results (2026-06-01) + +### POC 1: Drizzle queries through Honker — PASS + +- `SQLiteSession`, `SQLitePreparedQuery`, `SQLiteTransaction` are all + accessible from `drizzle-orm/sqlite-core/session` — can be extended. +- `BaseSQLiteDatabase` is accessible from `drizzle-orm/sqlite-core/db`. +- Honker `query()` returns `{ columnName: value }` object rows, compatible + with Drizzle's `mapResultRow()`. +- `last_insert_rowid()` accessible via `tx.query("SELECT last_insert_rowid() as id")`. +- `tx.execute()` returns a number (affected rows count for UPDATE/DELETE). +- JSON-mode columns need manual `JSON.parse()` in the adapter. +- `:memory:` databases DON'T work — reader pool gets separate in-memory DBs. + Always use file-based paths (or `:memory:` with a single reader, which + defeats Honker's concurrent reader design). +- `LibSQLSession` in `drizzle-orm/libsql/session` is a good reference + implementation for the adapter pattern. + +Session method signatures discovered: +``` +SQLiteSession methods: + prepareOneTimeQuery — key method to implement + all, run, get, values — delegated to PreparedQuery + extractRaw*ValueFromBatchResult — batch result extraction + count — cursor count + +SQLitePreparedQuery methods: + execute, getQuery, mapAllResult, mapGetResult, mapResult, mapRunResult + queryWithCache + +SQLiteTransaction methods: + rollback — only explicit method (commit handled by Session.transaction) +``` + +### POC 2: Same-process pub/sub — PASS (with latency caveat) + +- `db.notify()` → `db.listen()` works within the same process. +- Multiple concurrent listeners on different channels work. +- Cross-channel isolation works — listeners only receive notifications on + their subscribed channel. +- **Latency**: ~17ms median, ~29ms P99 (100 samples). This is higher than + ideal for call protocol request/response. The `Listener` uses a polling + fallback (`updateEvents().next()`) internally. For hot-path events where + sub-ms latency matters, the in-process `EventTarget` should be preferred. + Honker listeners are better suited for cross-process and durable scenarios. + +### POC 3: Concurrent listeners and streams — PASS + +- Multiple `Listener` instances coexist on the same `Database`. +- `Stream.publish()` / `Stream.subscribe()` works. +- `readSince(offset, limit)` returns historical events. +- `StreamSubscription` implements `AsyncIterableIterator` — works with + `for await...of`. +- Consumer offsets are tracked and persisted. +- Concurrent stream + listener operation works. +- `publishTx(tx, payload)` works within a transaction. +- **Caveat**: `db.close()` doesn't cleanly exit when listeners are still + polling. Need to close all listeners before closing the DB. + +### POC 4: Transactional notify + data write — PASS (atomicity confirmed) + +- `tx.notify()` + `tx.execute()` — notification fires ONLY after `tx.commit()`. + Before commit, no notification is received. **Transactional semantics hold.** +- On `tx.rollback()` — notification is NOT sent. Data is NOT persisted. + **Atomicity holds for both data and events.** +- `db.notifyTx(tx, channel, payload)` — convenience wrapper, same semantics. +- `queue.enqueueTx(tx, payload)` — job is only visible after commit. On + rollback, job does not appear in the queue. **Transactional outbox works.** +- Read-your-writes: after receiving a notification, the data IS queryable + (no stale read issue with single DB). + +**This is the most important result.** Honker provides true transactional +outbox semantics — data writes and event/queue publishes commit atomically. +This eliminates the dual-write problem that necessitated Redis for PostgreSQL. + +## Open Research Questions + +1. ~~**Drizzle session construction**~~ — RESOLVED by POC 1. All needed + classes are accessible and extendable. `LibSQLSession` is the reference. + +2. ~~**Honker listener concurrency model**~~ — RESOLVED by POC 3. Multiple + listeners coexist. The `updateEvents` mechanism works for concurrent + polling. `db.close()` requires all listeners to be closed first. + +3. **Consumer name management for streams** — When a hub restarts, durable + stream consumers resume from their last saved offset. But the consumer + name must be stable across restarts. Need a naming convention that's + deterministic (e.g., `{service}:{pid}` is not restart-stable). + +4. **Schema migrations** — Honker opens a single `.db` file. Drizzle Kit + supports SQLite migrations but expects `better-sqlite3` or `libsql`. Need + to verify that `drizzle-kit push` or `drizzle-kit generate` works with the + Honker adapter, or whether we need a custom migration runner. + +5. **OperationSpec generation from tables** — What's the concrete mapping? + For each table: one `create` mutation, one `find` query, one `list` query, + one `update` mutation, one `delete` mutation? Or something more + domain-specific? This is a design question, not a POC question. + +6. **In-process + Honker composite event target** — POC 2 showed ~17ms + latency for Honker notify→listen. For single-process hot paths, an + in-process EventTarget is sub-ms. A composite that dispatches to both + (in-process for speed, Honker for durability/cross-process) would be the + ideal default for a hub. Needs design work. \ No newline at end of file