Architect storage around SQLite+Honker: remove PG, add multi-tenant identity, scoping
Reorient @alkdev/storage around a single SQLite database host with Honker
for pub/sub, event streams, and task queues. PostgreSQL is removed as a
target (ADR-038), eliminating dual schema maintenance and infrastructure
complexity. Honker provides DB + pubsub + queues in one .db file (ADR-039).
Add system/tenant DB model (ADR-040): identity tables in system.db, all
graph data in tenant-{orgId}.db files. Identity tables move from the hub
into storage (ADR-041). Scoping columns (ownerId, projectId) added to
graphs table (ADR-042). Graph types get scope (system/tenant/user) to
protect infrastructure schemas (ADR-043).
Define Drizzle-Honker session adapter (ADR-044): ~100-line adapter enabling
Drizzle typed queries and Honker pubsub/queue on a single connection with
transactional consistency.
Resolve OQ-03, OQ-04, OQ-19, OQ-21, OQ-22, OQ-23, OQ-24. Add new
open questions OQ-26 through OQ-29 for Honker integration specifics.
New docs: honker-integration.md (adapter, event patterns, migration).
Scrub all PG/jsonb/libsql references from existing spec docs.
This commit is contained in:
57
AGENTS.md
57
AGENTS.md
@@ -5,9 +5,10 @@ Project-specific guidance for agents working on this package.
|
||||
## Project Overview
|
||||
|
||||
`@alkdev/storage` is a deno-first TypeScript package providing typed graph
|
||||
storage with dual database hosts (SQLite for spokes, PostgreSQL for the hub). It
|
||||
uses the metagraph pattern (graphTypes → nodeTypes → edgeTypes → typed graph
|
||||
instances) from the earlier `@ade` prototype.
|
||||
storage with SQLite via Honker. It uses the metagraph pattern (graphTypes →
|
||||
nodeTypes → edgeTypes → typed graph instances) and includes identity tables
|
||||
for multi-tenant auth. The system/tenant DB model separates identity
|
||||
infrastructure from org-scoped graph data.
|
||||
|
||||
## Architecture Snapshot
|
||||
|
||||
@@ -25,12 +26,14 @@ instances) from the earlier `@ade` prototype.
|
||||
│ │ ├── bridge.ts # moduleToDbSchema, validateNode, validateEdge
|
||||
│ │ ├── crypto.ts # encrypt, decrypt, generateEncryptionKey, EncryptedDataSchema
|
||||
│ │ └── mod.ts # Re-exports all graphs exports
|
||||
│ ├── sqlite/ # SQLite host (drizzle-orm/libsql)
|
||||
│ │ ├── tables/ # Drizzle table definitions
|
||||
│ │ ├── relations.ts # Drizzle relations
|
||||
│ │ ├── schema.ts # Re-exports
|
||||
│ │ └── client.ts # Injectable createSqliteDatabase()
|
||||
│ └── pg/ # PostgreSQL host (NOT YET IMPLEMENTED)
|
||||
│ └── sqlite/ # SQLite host (drizzle-orm + honker-node)
|
||||
│ ├── tables/
|
||||
│ │ ├── identity/ # accounts, organizations, org_members, api_keys, audit_logs
|
||||
│ │ └── metagraph/ # graph_types, node_types, edge_types, graphs, nodes, edges
|
||||
│ ├── relations.ts # Drizzle relations
|
||||
│ ├── adapter.ts # Drizzle-Honker session adapter
|
||||
│ ├── schema.ts # Re-exports
|
||||
│ └── client.ts # createSystemDatabase(), createTenantDatabase()
|
||||
└── test/
|
||||
└── reference-modules.test.ts # Metagraph, bridge, crypto tests
|
||||
```
|
||||
@@ -38,12 +41,10 @@ instances) from the earlier `@ade` prototype.
|
||||
### Subpath Exports (JSR/npm)
|
||||
|
||||
- `@alkdev/storage` → Metagraph Module, graph type definitions (zero deps)
|
||||
- `@alkdev/storage/sqlite` → SQLite tables, relations, client (drizzle-orm +
|
||||
libsql)
|
||||
- `@alkdev/storage/pg` → PostgreSQL tables, relations, client (NOT YET
|
||||
IMPLEMENTED)
|
||||
- `@alkdev/storage/sqlite` → SQLite tables (metagraph + identity), relations,
|
||||
client, Honker adapter (drizzle-orm + honker-node)
|
||||
|
||||
This design ensures consumers don't bundle database drivers they don't use.
|
||||
PostgreSQL has been removed (ADR-038). SQLite via Honker is the sole database host.
|
||||
|
||||
## Key Decisions
|
||||
|
||||
@@ -56,10 +57,15 @@ This design ensures consumers don't bundle database drivers they don't use.
|
||||
annotations impractical. We use `--allow-slow-types` on publish and
|
||||
`"exclude": ["no-slow-types"]` in lint config. This is known technical debt —
|
||||
can be tightened iteratively.
|
||||
4. **Injectable clients**: `createSqliteDatabase(client)` takes a client, not
|
||||
4. **Injectable clients**: `createSystemDatabase(client)` and
|
||||
`createTenantDatabase(client)` take pre-created Honker clients, not
|
||||
env vars. Module-level side effects are forbidden.
|
||||
5. **Dependencies**: `@alkdev/typebox` and `@alkdev/drizzlebox` are npm deps
|
||||
(not yet on JSR). This works fine — JSR handles npm dependencies natively.
|
||||
6. **SQLite-only via Honker**: No PostgreSQL. Honker provides DB + pub/sub +
|
||||
queues in a single SQLite file (ADR-038, ADR-039).
|
||||
7. **System/tenant DB split**: Identity tables in `system.db`, graph data in
|
||||
`tenant-{orgId}.db` (ADR-040).
|
||||
|
||||
## Commands
|
||||
|
||||
@@ -107,24 +113,27 @@ The codebase has diverged significantly from the originals:
|
||||
|
||||
See `docs/architecture/` for detailed specifications:
|
||||
|
||||
- `overview.md` — Package purpose, exports, design decisions, open questions
|
||||
- `overview.md` — Package purpose, exports, database model, ecosystem integration
|
||||
- `metagraph-module.md` — Graph type definitions as TypeBox Modules, data model,
|
||||
naming conventions, implementation path
|
||||
- `forward-look.md` — Connections to dbtype, graph pointers, ujsx universal IR
|
||||
pipeline
|
||||
- `honker-integration.md` — Drizzle-Honker adapter, event patterns, DB coordination
|
||||
- `schema-evolution.md` — How graph type schemas evolve, TypeBox Value.Diff/Patch/Cast
|
||||
for schema change detection and data migration
|
||||
- `sqlite-host.md` — SQLite tables, relations, client factory, porting notes
|
||||
- `sqlite-host.md` — SQLite tables (metagraph + identity), client factories
|
||||
- `encrypted-data.md` — Encrypted data design, crypto utility, node type modeling
|
||||
- `forward-look.md` — Connections to dbtype, graph pointers, ujsx universal IR
|
||||
- `acl.md` — Access control graph, principal-agent framework, scoping
|
||||
|
||||
These docs describe what the package is AND what it's becoming. Items marked ⚠️
|
||||
are not yet implemented.
|
||||
|
||||
## What's Not Done Yet
|
||||
|
||||
- `src/pg/` — PostgreSQL host (same table shapes, `pgTable` + `jsonb` +
|
||||
`timestamp` + `pgEnum`)
|
||||
- Repository/CRUD layer (currently only table definitions, no typed query
|
||||
functions)
|
||||
- Hub-specific tables (sessions, messages, parts, call graphs, tasks, etc.)
|
||||
- Drizzle-Honker session adapter (`src/sqlite/adapter.ts`)
|
||||
- Identity tables in `src/sqlite/tables/identity/` (accounts, organizations, etc.)
|
||||
- Scoping columns on `graphs` table (`ownerId`, `projectId`)
|
||||
- Graph type `scope` column on `graph_types` table
|
||||
- Remove `actors` table and `src/pg/` directory
|
||||
- Split client factory into `createSystemDatabase()` / `createTenantDatabase()`
|
||||
- Repository/CRUD layer (typed query functions beyond table definitions)
|
||||
- JSR publication setup (need to create scope/package on jsr.io first)
|
||||
|
||||
Reference in New Issue
Block a user