Architect storage around SQLite+Honker: remove PG, add multi-tenant identity, scoping
Reorient @alkdev/storage around a single SQLite database host with Honker
for pub/sub, event streams, and task queues. PostgreSQL is removed as a
target (ADR-038), eliminating dual schema maintenance and infrastructure
complexity. Honker provides DB + pubsub + queues in one .db file (ADR-039).
Add system/tenant DB model (ADR-040): identity tables in system.db, all
graph data in tenant-{orgId}.db files. Identity tables move from the hub
into storage (ADR-041). Scoping columns (ownerId, projectId) added to
graphs table (ADR-042). Graph types get scope (system/tenant/user) to
protect infrastructure schemas (ADR-043).
Define Drizzle-Honker session adapter (ADR-044): ~100-line adapter enabling
Drizzle typed queries and Honker pubsub/queue on a single connection with
transactional consistency.
Resolve OQ-03, OQ-04, OQ-19, OQ-21, OQ-22, OQ-23, OQ-24. Add new
open questions OQ-26 through OQ-29 for Honker integration specifics.
New docs: honker-integration.md (adapter, event patterns, migration).
Scrub all PG/jsonb/libsql references from existing spec docs.
This commit is contained in:
@@ -1,11 +1,11 @@
|
||||
---
|
||||
status: reviewed
|
||||
last_updated: 2026-05-30
|
||||
status: draft
|
||||
last_updated: 2026-05-31
|
||||
---
|
||||
|
||||
# @alkdev/storage — Overview
|
||||
|
||||
Typed graph storage with dual database hosts. Deno-first, published via JSR.
|
||||
Typed graph storage with SQLite via Honker. Deno-first, published via JSR.
|
||||
|
||||
## Purpose
|
||||
|
||||
@@ -16,12 +16,13 @@ graphs populated with nodes and edges.
|
||||
|
||||
This pattern replaces domain-specific table proliferation with a small number of
|
||||
general-purpose tables that can model anything — call graphs, ACL rules, task
|
||||
dependencies, encrypted secrets — while enforcing schema integrity through
|
||||
TypeBox validation.
|
||||
dependencies, encrypted secrets, session trees, operation registries — while
|
||||
enforcing schema integrity through TypeBox validation.
|
||||
|
||||
The package evolved from `@ade/ade-v0/packages/core/graphs` and
|
||||
`@ade/ade-v0/packages/storage_sqlite`, simplified and refactored for the @alkdev
|
||||
ecosystem.
|
||||
The package also provides **identity infrastructure** tables (accounts,
|
||||
organizations, api_keys, audit_logs) for multi-tenant authentication and
|
||||
authorization, and integrates with **Honker** for transactional pub/sub, event
|
||||
streams, and task queues within the same SQLite database.
|
||||
|
||||
## Architecture
|
||||
|
||||
@@ -38,12 +39,16 @@ ecosystem.
|
||||
│ │ ├── bridge.ts → moduleToDbSchema, validateNode, validateEdge
|
||||
│ │ ├── crypto.ts → encrypt, decrypt, generateEncryptionKey, EncryptedDataSchema
|
||||
│ │ └── mod.ts → re-exports all graphs exports
|
||||
│ ├── sqlite/ → SQLite host (drizzle-orm/libsql)
|
||||
│ │ ├── tables/ → drizzle table definitions
|
||||
│ │ ├── relations.ts → drizzle relational mappings
|
||||
│ │ ├── schema.ts → barrel re-export
|
||||
│ │ └── client.ts → injectable createSqliteDatabase()
|
||||
│ └── pg/ → PostgreSQL host (NOT YET IMPLEMENTED)
|
||||
│ └── sqlite/ → SQLite host (Drizzle + Honker)
|
||||
│ ├── tables/ → drizzle table definitions
|
||||
│ │ ├── common.ts → commonCols
|
||||
│ │ ├── identity/ → accounts, organizations, org_members, api_keys, audit_logs
|
||||
│ │ ├── metagraph/ → graph_types, node_types, edge_types, graphs, nodes, edges
|
||||
│ │ └── index.ts → barrel re-export
|
||||
│ ├── relations.ts → drizzle relational mappings
|
||||
│ ├── schema.ts → barrel re-export
|
||||
│ ├── adapter.ts → Drizzle-Honker session adapter
|
||||
│ └── client.ts → createSystemDatabase(), createTenantDatabase()
|
||||
└── test/
|
||||
└── reference-modules.test.ts → Metagraph, bridge, crypto tests
|
||||
```
|
||||
@@ -54,26 +59,66 @@ ecosystem.
|
||||
| ------------------------ | --------------------------------------- | --------------------------------------- |
|
||||
| `@alkdev/storage` | Graph schema types, Metagraph Module | `@alkdev/typebox` |
|
||||
| `@alkdev/storage/graphs` | Same as `.` — alias for the main export | Same as `.` |
|
||||
| `@alkdev/storage/sqlite` | SQLite tables, relations, client | `@alkdev/drizzlebox`, `drizzle-orm`, `@libsql/client` |
|
||||
| `@alkdev/storage/pg` | PostgreSQL tables, relations, client | ⚠️ NOT YET IMPLEMENTED |
|
||||
| `@alkdev/storage/sqlite` | SQLite tables, relations, client, adapter | `@alkdev/drizzlebox`, `drizzle-orm`, `@russellthehipp/honker-node` |
|
||||
|
||||
The `./graphs` subpath exists because the source code lives in `src/graphs/` and
|
||||
the main `mod.ts` re-exports it. Importing from either `@alkdev/storage` or
|
||||
`@alkdev/storage/graphs` yields the same types and Metagraph Module.
|
||||
The `pg/` subpath has been removed (ADR-038). SQLite via Honker is the sole
|
||||
database host.
|
||||
|
||||
## Database Model
|
||||
|
||||
### System DB + Tenant DB
|
||||
|
||||
The package uses a two-database model for multi-tenant isolation (ADR-040):
|
||||
|
||||
| Database | Contents | Purpose |
|
||||
|----------|----------|---------|
|
||||
| `system.db` | accounts, organizations, organization_members, api_keys, audit_logs, system graph_types | Identity infrastructure, authentication, authorization anchors |
|
||||
| `tenant-{orgId}.db` | graphs, nodes, edges, graph_types, node_types, edge_types | All graph data for one org — call graphs, ACL instances, session trees, secrets, tasks |
|
||||
|
||||
The system DB is opened by `createSystemDatabase(client)`. Each tenant DB is
|
||||
opened by `createTenantDatabase(client)`. Both return typed Drizzle instances
|
||||
with their respective schemas attached.
|
||||
|
||||
**Why separate files**: File-level isolation means one tenant's data cannot leak
|
||||
to another, even via application bugs. Each tenant DB is independently
|
||||
backupable, migratable, and compactable. No `orgId` column is needed on tenant
|
||||
tables because the entire file IS the org scope.
|
||||
|
||||
**Cross-DB references**: The tenant DB's `graphs.ownerId` and `graphs.projectId`
|
||||
logically reference (not FK) the system DB's identity tables. The consumer
|
||||
enforces referential integrity at the application layer, consistent with
|
||||
ADR-020.
|
||||
|
||||
### Graph Type Scope
|
||||
|
||||
Graph types have a `scope` column (ADR-043) controlling who can create and
|
||||
modify them:
|
||||
|
||||
| Scope | Examples | Who can modify |
|
||||
|-------|----------|----------------|
|
||||
| `system` | acl, call-graph, secret, operation-registry, message-session | Setup/seeding only |
|
||||
| `tenant` | Custom org graph types (sprint-board, etc.) | Org admins |
|
||||
| `user` | Personal graph types (my-notes, etc.) | Creating user |
|
||||
|
||||
System-scoped graph types are seeded during initialization. Their schemas are
|
||||
fixed — changes require a version bump and migration (ADR-029).
|
||||
|
||||
## Terminology
|
||||
|
||||
| Term | Definition |
|
||||
| ----------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| **Metagraph** | A type system where graph types define schemas, node types define data shapes within those graphs, and edge types define typed relationships. Graph instances are concrete data conforming to these type definitions. |
|
||||
| **Hub** | The central service in the hub-spoke architecture. A consumer of `@alkdev/storage` — uses the PostgreSQL host for persistent graph storage. The hub also depends on `@alkdev/operations`, `@alkdev/pubsub`, `@alkdev/flowgraph`. |
|
||||
| **Spoke** | A local/embedded instance that runs per-project or per-session. A consumer of `@alkdev/storage` — uses the SQLite host for local graph storage. |
|
||||
| **Hub** | The central service in the hub-spoke architecture. A consumer of `@alkdev/storage` — opens both system and tenant databases. The hub also depends on `@alkdev/operations`, `@alkdev/pubsub`, `@alkdev/flowgraph`. |
|
||||
| **Spoke** | A local/embedded instance that runs per-project or per-session. A consumer of `@alkdev/storage` — opens a tenant database (or its own standalone DB for single-user mode). |
|
||||
| **System DB** | The SQLite database holding identity/auth tables and system graph type definitions. One per deployment. |
|
||||
| **Tenant DB** | The SQLite database holding all graph data for one organization. One per org. |
|
||||
| **Graph type** | A class of graphs (e.g., "call-graph", "acl"). Defines structural constraints (directed/undirected/mixed, multi-edges, self-loops) and the valid node/edge type vocabularies. Stored in the `graph_types` table. |
|
||||
| **Node type** | A category of node within a graph type. Defines the attribute schema for nodes of that type. Stored in the `node_types` table. |
|
||||
| **Edge type** | A category of edge within a graph type. Defines the attribute schema and optionally restricts which node types can be source/target. Stored in the `edge_types` table. |
|
||||
| **Graph instance** | A concrete graph belonging to a graph type. Contains nodes and edges conforming to its type definitions. Stored in the `graphs` table. |
|
||||
| **Honker** | SQLite extension providing transactional pub/sub, durable event streams, task queues, advisory locks, and cron scheduling within the same `.db` file. |
|
||||
| **Consumer** | Code that imports `@alkdev/storage` (or a subpath) to define graph types and persist graph data. The hub, spokes, and other @alkdev packages are consumers. |
|
||||
| **Repository layer** | ⚠️ Not yet implemented. The typed CRUD functions (insert, find, update, delete) that sit between consumer code and raw Drizzle queries. Performs schema validation before writes. No dependency on `@alkdev/operations` — the consumer wires CRUD into the registry. |
|
||||
| **Repository layer** | ⚠️ Not yet implemented. The typed CRUD functions (insert, find, update, delete) that sit between consumer code and raw Drizzle queries. Performs schema validation before writes. |
|
||||
| **Validation boundary** | The line where schema validation is enforced. In this package, validation happens in the Metagraph Module (at type definition time) and the repository layer (at mutation time), NOT in the database. |
|
||||
|
||||
## Design Decisions
|
||||
@@ -83,14 +128,22 @@ All design decisions are documented as ADRs in [decisions/](decisions/).
|
||||
| ADR | Decision | Summary |
|
||||
|-----|----------|---------|
|
||||
| [001](decisions/001-deno-first-jsr-publishes.md) | Deno-first, JSR publishes | Published to JSR; npm comes free via `@jsr/alkdev__storage` |
|
||||
| [002](decisions/002-metagraph-over-domain-tables.md) | Metagraph over domain-specific tables | 6 general-purpose tables serve all domains |
|
||||
| [002](decisions/002-metagraph-over-domain-tables.md) | Metagraph over domain-specific tables | 6 general-purpose tables serve all graph-shaped domains |
|
||||
| [003](decisions/003-typebox-module-as-api-surface.md) | TypeBox Module as API surface | `Type.Module()` replaces `SchemaBuilder`; `Metagraph.Import()` + `Type.Composite()` |
|
||||
| [004](decisions/004-injectable-clients-no-side-effects.md) | Injectable clients, no side effects | `createSqliteDatabase(client)` takes a pre-created client |
|
||||
| [004](decisions/004-injectable-clients-no-side-effects.md) | Injectable clients, no side effects | `createSystemDatabase(client)` / `createTenantDatabase(client)` take pre-created clients |
|
||||
| [005](decisions/005-drizzle-plus-typebox-via-drizzlebox.md) | Drizzle + TypeBox via drizzlebox | Drizzle tables are single source of truth; drizzlebox generates TypeBox schemas |
|
||||
| [006](decisions/006-enum-pattern-as-const-objects.md) | `as const` objects, not TypeScript enums | Avoids JSR slow-types; consistent pattern across codebase |
|
||||
| [007](decisions/007-no-comments-in-code.md) | No comments in code | Documentation lives in architecture docs and TypeBox descriptions |
|
||||
| [008](decisions/008-common-columns-pattern.md) | Common columns pattern | `id`, `metadata`, `createdAt`, `updatedAt` on every table |
|
||||
| [033](decisions/033-json-path-queries-for-v1.md) | JSON path queries and hand-written CRUD for v1 | Attribute queries use JSON path; CRUD is hand-written; dbtype and auto-generation are post-v1 |
|
||||
| [033](decisions/033-json-path-queries-for-v1.md) | JSON path queries and hand-written CRUD for v1 | Attribute queries use JSON path; CRUD is hand-written |
|
||||
| [038](decisions/038-sqlite-first-pg-removed.md) | SQLite-first, Postgres removed | Single database host; no dual maintenance |
|
||||
| [039](decisions/039-honker-as-sqlite-extension.md) | Honker as SQLite extension and transport | DB + pub/sub + queues + events in one SQLite file |
|
||||
| [040](decisions/040-system-db-tenant-db.md) | System DB + tenant DB separation | Identity in system.db, graph data in tenant-{orgId}.db |
|
||||
| [041](decisions/041-identity-tables-in-storage.md) | Identity tables in storage package | accounts, organizations, api_keys, audit_logs defined in storage |
|
||||
| [042](decisions/042-scoping-columns-on-graphs.md) | Scoping columns on graph instances | `ownerId`, `projectId` on `graphs` table |
|
||||
| [043](decisions/043-graph-type-scope.md) | Graph type scope | `system` / `tenant` / `user` scope controls who can modify graph types |
|
||||
| [044](decisions/044-drizzle-honker-adapter.md) | Drizzle-Honker session adapter | ~100-line adapter; no Drizzle fork; `$client` and `$honkerTx` for honker access |
|
||||
| [045](decisions/045-org-members-authoritative-belongsto-derived.md) | organization_members authoritative, BelongsToEdge derived | SQL table for fast lookups; ACL edge for traversal evaluation |
|
||||
|
||||
## Dependencies
|
||||
|
||||
@@ -98,18 +151,15 @@ All design decisions are documented as ADRs in [decisions/](decisions/).
|
||||
| -------------------- | ------------------------------------ | ------------------------ |
|
||||
| `@alkdev/typebox` | Runtime schema validation | graphs/ |
|
||||
| `@alkdev/drizzlebox` | Generate TypeBox from Drizzle tables | sqlite/ |
|
||||
| `drizzle-orm` | ORM, table definitions, queries | sqlite/ (and future pg/) |
|
||||
| `@libsql/client` | SQLite client (libsql/turso) | sqlite/ |
|
||||
| `postgres` | PostgreSQL client | pg/ (not yet used) |
|
||||
| `drizzle-orm` | ORM, table definitions, queries | sqlite/ |
|
||||
| `@russellthehippo/honker-node` | SQLite + pub/sub + queues + events | sqlite/ |
|
||||
|
||||
`@alkdev/typebox` and `@alkdev/drizzlebox` are npm packages (not yet on JSR).
|
||||
JSR handles npm dependencies natively.
|
||||
|
||||
**Ecosystem packages are not runtime dependencies of `@alkdev/storage`.** All
|
||||
ecosystem references in this document describe consumer-side data shapes and
|
||||
integration patterns, not import dependencies. The `@alkdev/operations`,
|
||||
`@alkdev/pubsub`, `@alkdev/flowgraph`, and `@alkdev/taskgraph` packages are
|
||||
consumed by the hub and spokes, not by storage itself.
|
||||
ecosystem references describe consumer-side data shapes and integration
|
||||
patterns, not import dependencies.
|
||||
|
||||
## What Exists vs. What's Needed
|
||||
|
||||
@@ -119,8 +169,7 @@ consumed by the hub and spokes, not by storage itself.
|
||||
- Bridge functions (`moduleToDbSchema`, `validateNode`, `validateEdge`)
|
||||
- Reference graph type Modules (CallGraph, SecretGraph)
|
||||
- Crypto utility (AES-256-GCM + PBKDF2, `EncryptedDataSchema`)
|
||||
- SQLite host: 6 metagraph tables + actors table + Drizzle relations + client
|
||||
factory
|
||||
- SQLite host: 6 metagraph tables + `actors` placeholder + Drizzle relations + client factory
|
||||
- TypeBox select/insert schemas generated from Drizzle tables (drizzlebox)
|
||||
- Reference module tests (bridge functions, validation, Module composition)
|
||||
|
||||
@@ -128,18 +177,22 @@ consumed by the hub and spokes, not by storage itself.
|
||||
|
||||
| Gap | Priority | Notes |
|
||||
| ----------------------------------------- | ------------ | --------------------------------------------------------------------------------------------------- |
|
||||
| Repository/CRUD layer | High | ⚠️ Not yet implemented. Typed insert, find, update, delete functions for graphs, nodes, edges. No dependency on `@alkdev/operations` — consumer wires CRUD into registry. |
|
||||
| PostgreSQL host | Medium | Same table shapes, `pgTable` + `jsonb` + `timestamp` + `pgEnum`. Stub only. |
|
||||
| ACL graph type | Medium | Access control as a metagraph. Principal/agent delegation, org scoping, resource access. See [acl.md](acl.md). |
|
||||
| Task graph type | Low | Informed by `@alkdev/taskgraph`'s `TaskGraphNodeAttributes` and `DependencyEdge` schemas. |
|
||||
| Graphology bridge | Low | `moduleToGraphology()` and `fromGraphologyExport()` — Phase 4 of the metagraph implementation path. |
|
||||
| Drizzle-Honker session adapter | High | ~100-line adapter wrapping `HonkerSQLiteSession` + `HonkerPreparedQuery`. ADR-044. |
|
||||
| Identity tables (accounts, organizations, api_keys, audit_logs, organization_members) | High | Moving from hub into storage. ADR-041. |
|
||||
| Scoping columns on `graphs` table (`ownerId`, `projectId`) | High | ADR-042. |
|
||||
| Graph type `scope` column | High | ADR-043. |
|
||||
| Remove `actors` table and `pg/` directory | High | ADR-035 (actors), ADR-038 (pg). |
|
||||
| `createSystemDatabase()` / `createTenantDatabase()` factories | High | Split from current `createSqliteDatabase()`. |
|
||||
| Repository/CRUD layer | Medium | ⚠️ Typed insert, find, update, delete functions for graphs, nodes, edges. |
|
||||
| ACL graph type | Medium | Access control as a metagraph. ADR-034. |
|
||||
| Task graph type | Low | Informed by `@alkdev/taskgraph`'s schemas. |
|
||||
| Graphology bridge | Low | `moduleToGraphology()` and `fromGraphologyExport()` — Phase 4. |
|
||||
|
||||
## Ecosystem Integration
|
||||
|
||||
`@alkdev/storage` is a **data layer package** consumed by other packages in the
|
||||
@alkdev ecosystem. It does not depend on the hub — the dependency flows the
|
||||
other way. The hub consumes storage (along with operations, pubsub, flowgraph,
|
||||
and taskgraph) as part of its architecture.
|
||||
other way.
|
||||
|
||||
### Dependency Direction
|
||||
|
||||
@@ -153,13 +206,8 @@ and taskgraph) as part of its architecture.
|
||||
@alkdev/taskgraph ← task dependency graph schema, cost-benefit analysis
|
||||
(depends on: @alkdev/typebox)
|
||||
|
||||
@alkdev/dbtype ← schema-first multi-dialect DB type system (Phase 0, not yet implemented)
|
||||
(depends on: @alkdev/typebox, @alkdev/ujsx)
|
||||
Renders UJSX element trees to Drizzle dialects; future: from-dbtype
|
||||
adapter generates CRUD OperationSpecs for @alkdev/operations
|
||||
|
||||
@alkdev/storage ← YOU ARE HERE — typed graph persistence
|
||||
(depends on: @alkdev/typebox, @alkdev/drizzlebox)
|
||||
@alkdev/storage ← YOU ARE HERE — typed graph persistence + identity
|
||||
(depends on: @alkdev/typebox, @alkdev/drizzlebox, drizzle-orm, honker-node)
|
||||
|
||||
↑ ↑
|
||||
| |
|
||||
@@ -167,119 +215,75 @@ Hub / Spoke Any consumer that needs
|
||||
(consumes all) persistent graph storage
|
||||
```
|
||||
|
||||
The key insight: `@alkdev/storage` provides the **persistence primitives**
|
||||
(schemas, tables, repository layer). The **domain semantics** (what a call graph
|
||||
means, what identity looks like, how access control works) are defined by the
|
||||
packages above. Storage stores the shapes those packages define; it does not
|
||||
define the semantics itself.
|
||||
### Event-Driven Architecture with Honker
|
||||
|
||||
The @alkdev platform is event-driven. Honker provides the transport mechanism
|
||||
within each SQLite database:
|
||||
|
||||
| Concern | Honker Primitive | Example |
|
||||
|---------|------------------|---------|
|
||||
| Fire-and-forget notifications | `notify(channel, payload)` | "graph:updated", "node:created" |
|
||||
| Durable per-consumer delivery | `stream(name).subscribe(consumer)` | Call protocol events, audit trail |
|
||||
| At-least-once background jobs | `queue(name).enqueue(payload)` | Key rotation, schema migration, retention cleanup |
|
||||
| Leader election | `tryLock(name, owner, ttl)` | Only one hub instance runs the scheduler |
|
||||
| Scheduled operations | `scheduler().add(cron, handler)` | Retention cleanup, key sweep, rate limit sweep |
|
||||
|
||||
A single Drizzle transaction via the Honker adapter can insert graph data AND
|
||||
publish a notification AND enqueue a side-effect job — all committing atomically.
|
||||
No dual-write problem between data and events.
|
||||
|
||||
### What Comes from Where
|
||||
|
||||
| Concept | Source package | Storage's role |
|
||||
|---------|---------------|----------------|
|
||||
| Call protocol events (`call.requested`, `call.responded`, etc.) | `@alkdev/operations` | Storage persists the outcomes — graphs with `CallNodeAttrs` nodes |
|
||||
| Identity (`id`, `scopes`, `resources`) | `@alkdev/operations` | Storage stores identity as node attributes; `Identity` is a data shape, not a storage concept |
|
||||
| Access control (`AccessControl`, `requiredScopes`) | `@alkdev/operations` | Storage's ACL graph type mirrors the operations `AccessControl` schema as graph structure |
|
||||
| Call graph schema (`CallNodeAttrs`, `CallEdgeAttrs`, `CallStatus`) | `@alkdev/flowgraph` | Storage persists these in-memory shapes to the database |
|
||||
| Task graph schema (`TaskGraphNodeAttributes`, `DependencyEdge`) | `@alkdev/taskgraph` | Storage persists task dependency shapes |
|
||||
| Event transport (`TypedEventTarget`, `EventEnvelope`) | `@alkdev/pubsub` | Storage is not involved in event routing; it stores the events' outcomes |
|
||||
| Database schema rendering (`<table>`, `<column>`, HostConfig) | `@alkdev/dbtype` | Storage's static metagraph tables could be dbtype-rendered in the future (OQ-17, OQ-18) |
|
||||
| Universal IR (`h()`, `createComponent`, `createRoot`) | `@alkdev/ujsx` | Storage's `Type.Module` format is structurally compatible with ujsx rendering; no runtime dependency |
|
||||
| Call protocol events | `@alkdev/operations` | Storage persists the outcomes as graph nodes + publishes via Honker stream |
|
||||
| Identity | `@alkdev/operations` | Storage stores identity in SQL accounts table + as PrincipalNode in ACL graphs |
|
||||
| Access control | `@alkdev/operations` | Storage's AclGraph mirrors `AccessControl` schema as graph structure |
|
||||
| Call graph schema | `@alkdev/flowgraph` | Storage persists in-memory shapes to the database |
|
||||
| Task graph schema | `@alkdev/taskgraph` | Storage persists task dependency shapes |
|
||||
| Event transport | Honker (within storage) | Replaces `@alkdev/pubsub`'s Redis transport for single-node deployments |
|
||||
|
||||
### Repository Layer Bridging Pattern (Consumer-Side Concern)
|
||||
### Repository Layer Bridging Pattern
|
||||
|
||||
The repository layer in `@alkdev/storage` provides typed CRUD — no `@alkdev/operations`
|
||||
dependency. A **consumer-side** bridging module can then wire these CRUD functions
|
||||
into the `@alkdev/operations` registry, analogous to how `drizzle-graphql`
|
||||
auto-generates a GraphQL schema from Drizzle tables — but using operations
|
||||
(queries, mutations, subscriptions) instead of GraphQL resolvers. This works
|
||||
because:
|
||||
|
||||
1. `@alkdev/operations` already maps closely to GraphQL's
|
||||
queries/mutations/subscriptions (it was modeled after that pattern)
|
||||
2. `@alkdev/pubsub` provides the subscription transport (forked from
|
||||
graphql-yoga's pubsub with additions like in-memory, Redis, WebSocket,
|
||||
WebWorker event targets)
|
||||
3. `@alkdev/storage`'s metagraph tables are the data source, analogous to
|
||||
Drizzle tables for drizzle-graphql
|
||||
|
||||
The bridging module would live in a consumer package (e.g., the hub or a
|
||||
dedicated `@alkdev/storage-operations` adapter), not in `@alkdev/storage` itself,
|
||||
to avoid circular dependencies:
|
||||
The repository layer in `@alkdev/storage` provides typed CRUD — no
|
||||
`@alkdev/operations` dependency. A consumer-side bridging module wires CRUD
|
||||
functions into the operations registry when needed.
|
||||
|
||||
```
|
||||
@alkdev/storage → defines types + tables (no operations dependency)
|
||||
@alkdev/storage → defines types + tables + CRUD (no operations dependency)
|
||||
@alkdev/operations → defines call protocol + registry (no storage dependency)
|
||||
Consumer (hub / adapter) → imports both, generates operations from schemas
|
||||
```
|
||||
|
||||
#### Ecosystem Context
|
||||
|
||||
The question of *where* this bridge lives and *how* it's generated connects to
|
||||
the broader ecosystem:
|
||||
|
||||
- **drizzle-graphql** (`/workspace/drizzle-graphql`): Auto-generates GraphQL
|
||||
CRUD from Drizzle tables. The reference pattern for "database schema → API
|
||||
surface." Produces `{ schema, entities }` from `buildSchema(db)`. No TypeBox,
|
||||
no metagraph.
|
||||
|
||||
- **@alkdev/dbtype**: Schema-first multi-dialect system using ujsx element trees.
|
||||
Defines `<table>`, `<column>` elements rendered to Drizzle via HostConfig. Has
|
||||
a designed `from-dbtype` adapter that generates `OperationSpec[]` from element
|
||||
trees + Type.Module bundles. Phase 0 (architecture only, no implementation).
|
||||
|
||||
- **@alkdev/operations**: Runtime-agnostic typed operations registry with
|
||||
adapters (`FromOpenAPI`, `from_mcp`, `from_typemap`) that generate
|
||||
`OperationSpec[]` from external specifications. The `from-dbtype` adapter would
|
||||
be another adapter in the same pattern.
|
||||
|
||||
The strategic question (OQ-17, OQ-18) is whether storage's repository CRUD
|
||||
operations should be hand-written, auto-generated from Drizzle schemas, or
|
||||
auto-generated from dbtype element trees once dbtype is implemented. For v1,
|
||||
hand-written CRUD is the simplest path and doesn't block any long-term option.
|
||||
See [forward-look.md](forward-look.md) for the full analysis.
|
||||
|
||||
### Avoiding Circular Dependencies
|
||||
|
||||
Neither `@alkdev/storage` nor `@alkdev/operations` should depend on each
|
||||
other directly. Storage defines the schema types and database tables; operations
|
||||
defines the call protocol and execution model. The consumer (hub, spoke, or
|
||||
adapter package) imports both and bridges them. This preserves the
|
||||
single-responsibility principle and allows each package to evolve independently.
|
||||
|
||||
If shared type definitions are needed (e.g., `Identity` referenced in both
|
||||
storage node attributes and operations call events), they should either:
|
||||
1. Be duplicated in each package with a documented correspondence (acceptable
|
||||
for small, stable types)
|
||||
2. Be extracted to a minimal shared types package if the duplication becomes
|
||||
burdensome
|
||||
defines the call protocol and execution model. The consumer imports both and
|
||||
bridges them.
|
||||
|
||||
## Open Questions
|
||||
|
||||
Open questions are tracked in [open-questions.md](open-questions.md). Key
|
||||
questions affecting this package:
|
||||
|
||||
- **OQ-03**: Should actors be a node type or a standalone table? (resolved: actors become PrincipalNode in ACL graph — ADR-035)
|
||||
- **OQ-04**: Should the repository layer be host-specific or host-agnostic? (open, start host-specific)
|
||||
- **OQ-14**: Should encryption be per-attribute, per-node, or per-graph? (resolved: per-attribute)
|
||||
- **OQ-15**: Should key management be in this package? (resolved: no, application provides key ring)
|
||||
- **OQ-16**: Should the repository layer live in storage or a consumer package? (resolved: CRUD in storage, operations bridging in consumer)
|
||||
- **OQ-17**: How should the repository layer handle attribute queries — JSON path, native columns, or dbtype-generated? (open, JSON path for v1)
|
||||
- **OQ-18**: Should CRUD operations be auto-generated or hand-written? (open, hand-write for v1)
|
||||
- **OQ-19**: Where does the storage-operations bridge package live? (open, depends on OQ-17/OQ-18)
|
||||
- **OQ-04**: Should the repository layer be host-specific or host-agnostic? (resolved: single host, question moot)
|
||||
- **OQ-22**: How are ACL graph instances created and managed? (open, tenant DB model simplifies: likely one per tenant DB)
|
||||
- **OQ-23**: BelongsToEdge derived or primary? (resolved: derived — ADR-045)
|
||||
- **OQ-26**: Can Honker replace `@alkdev/pubsub`'s Redis transport for single-node deployments? (open)
|
||||
- **OQ-27**: How are schema migrations applied across all tenant DBs? (open)
|
||||
- **OQ-28**: How does cross-tenant delegation work with separate DBs? (open)
|
||||
|
||||
## References
|
||||
|
||||
- Metagraph Module evolution: [metagraph-module.md](./metagraph-module.md)
|
||||
- Schema evolution via TypeBox value system: [schema-evolution.md](./schema-evolution.md)
|
||||
- Metagraph Module: [metagraph-module.md](./metagraph-module.md)
|
||||
- Honker integration: [honker-integration.md](./honker-integration.md)
|
||||
- SQLite host tables: [sqlite-host.md](./sqlite-host.md)
|
||||
- Schema evolution: [schema-evolution.md](./schema-evolution.md)
|
||||
- Encrypted data: [encrypted-data.md](./encrypted-data.md)
|
||||
- ACL graph: [acl.md](./acl.md)
|
||||
- Forward-looking connections: [forward-look.md](./forward-look.md)
|
||||
- Operations architecture: `/workspace/@alkdev/operations/docs/architecture/README.md`
|
||||
- Pubsub architecture: `/workspace/@alkdev/pubsub/docs/architecture/README.md`
|
||||
- Flowgraph architecture: `/workspace/@alkdev/flowgraph/docs/architecture/README.md`
|
||||
- Taskgraph architecture: `/workspace/@alkdev/taskgraph_ts/docs/architecture/README.md`
|
||||
- drizzle-graphql (reference for repo bridging pattern): `/workspace/drizzle-graphql/`
|
||||
- Source heritage: `@ade/ade-v0/packages/core/graphs` and
|
||||
`@ade/ade-v0/packages/storage_sqlite`
|
||||
- Drizzle ORM: https://orm.drizzle.team/
|
||||
- TypeBox: `/workspace/@alkdev/typebox/`
|
||||
- JSR: https://jsr.io/
|
||||
- Open questions: [open-questions.md](./open-questions.md)
|
||||
- Honker source: `/workspace/honker/`
|
||||
- Operations architecture: `/workspace/@alkdev/operations/docs/architecture/`
|
||||
- Flowgraph architecture: `/workspace/@alkdev/flowgraph/docs/architecture/`
|
||||
Reference in New Issue
Block a user