glm-5.1
6aa2fcc6ff
Reorient @alkdev/storage around a single SQLite database host with Honker
for pub/sub, event streams, and task queues. PostgreSQL is removed as a
target (ADR-038), eliminating dual schema maintenance and infrastructure
complexity. Honker provides DB + pubsub + queues in one .db file (ADR-039).
Add system/tenant DB model (ADR-040): identity tables in system.db, all
graph data in tenant-{orgId}.db files. Identity tables move from the hub
into storage (ADR-041). Scoping columns (ownerId, projectId) added to
graphs table (ADR-042). Graph types get scope (system/tenant/user) to
protect infrastructure schemas (ADR-043).
Define Drizzle-Honker session adapter (ADR-044): ~100-line adapter enabling
Drizzle typed queries and Honker pubsub/queue on a single connection with
transactional consistency.
Resolve OQ-03, OQ-04, OQ-19, OQ-21, OQ-22, OQ-23, OQ-24. Add new
open questions OQ-26 through OQ-29 for Honker integration specifics.
New docs: honker-integration.md (adapter, event patterns, migration).
Scrub all PG/jsonb/libsql references from existing spec docs.
140 lines
6.9 KiB
Markdown
140 lines
6.9 KiB
Markdown
# AGENTS.md — @alkdev/storage
|
|
|
|
Project-specific guidance for agents working on this package.
|
|
|
|
## Project Overview
|
|
|
|
`@alkdev/storage` is a deno-first TypeScript package providing typed graph
|
|
storage with SQLite via Honker. It uses the metagraph pattern (graphTypes →
|
|
nodeTypes → edgeTypes → typed graph instances) and includes identity tables
|
|
for multi-tenant auth. The system/tenant DB model separates identity
|
|
infrastructure from org-scoped graph data.
|
|
|
|
## Architecture Snapshot
|
|
|
|
```
|
|
@alkdev/storage/
|
|
├── mod.ts # Re-exports graphs/ only (zero db deps)
|
|
├── deno.json # JSR config, imports, tasks, lint rules
|
|
├── src/
|
|
│ ├── graphs/ # Metagraph Module + bridge functions (no db deps)
|
|
│ │ ├── modules/ # TypeBox Module definitions
|
|
│ │ │ ├── metagraph.ts # Base Metagraph Module (Config, BaseNode, BaseEdge)
|
|
│ │ │ ├── call-graph.ts # CallGraph reference Module
|
|
│ │ │ ├── secret-graph.ts # SecretGraph reference Module
|
|
│ │ │ └── index.ts # Barrel re-export
|
|
│ │ ├── bridge.ts # moduleToDbSchema, validateNode, validateEdge
|
|
│ │ ├── crypto.ts # encrypt, decrypt, generateEncryptionKey, EncryptedDataSchema
|
|
│ │ └── mod.ts # Re-exports all graphs exports
|
|
│ └── sqlite/ # SQLite host (drizzle-orm + honker-node)
|
|
│ ├── tables/
|
|
│ │ ├── identity/ # accounts, organizations, org_members, api_keys, audit_logs
|
|
│ │ └── metagraph/ # graph_types, node_types, edge_types, graphs, nodes, edges
|
|
│ ├── relations.ts # Drizzle relations
|
|
│ ├── adapter.ts # Drizzle-Honker session adapter
|
|
│ ├── schema.ts # Re-exports
|
|
│ └── client.ts # createSystemDatabase(), createTenantDatabase()
|
|
└── test/
|
|
└── reference-modules.test.ts # Metagraph, bridge, crypto tests
|
|
```
|
|
|
|
### Subpath Exports (JSR/npm)
|
|
|
|
- `@alkdev/storage` → Metagraph Module, graph type definitions (zero deps)
|
|
- `@alkdev/storage/sqlite` → SQLite tables (metagraph + identity), relations,
|
|
client, Honker adapter (drizzle-orm + honker-node)
|
|
|
|
PostgreSQL has been removed (ADR-038). SQLite via Honker is the sole database host.
|
|
|
|
## Key Decisions
|
|
|
|
1. **Deno-first, npm-second via JSR**: Package is published to JSR
|
|
(`deno publish`). npm compatibility is automatic via JSR's npm layer
|
|
(`@jsr/alkdev__storage`). No separate dnt build step.
|
|
2. **No comments in code**: Per project convention across @alkdev packages.
|
|
3. **JSR slow types excluded from lint**: Drizzle's deeply inferred generics
|
|
(`sqliteTable`, `createInsertSchema`, `relations`) make explicit type
|
|
annotations impractical. We use `--allow-slow-types` on publish and
|
|
`"exclude": ["no-slow-types"]` in lint config. This is known technical debt —
|
|
can be tightened iteratively.
|
|
4. **Injectable clients**: `createSystemDatabase(client)` and
|
|
`createTenantDatabase(client)` take pre-created Honker clients, not
|
|
env vars. Module-level side effects are forbidden.
|
|
5. **Dependencies**: `@alkdev/typebox` and `@alkdev/drizzlebox` are npm deps
|
|
(not yet on JSR). This works fine — JSR handles npm dependencies natively.
|
|
6. **SQLite-only via Honker**: No PostgreSQL. Honker provides DB + pub/sub +
|
|
queues in a single SQLite file (ADR-038, ADR-039).
|
|
7. **System/tenant DB split**: Identity tables in `system.db`, graph data in
|
|
`tenant-{orgId}.db` (ADR-040).
|
|
|
|
## Commands
|
|
|
|
```bash
|
|
deno check mod.ts src/graphs/mod.ts src/sqlite/mod.ts # Type check
|
|
deno lint # Lint (slow-types, verbatim-module-syntax excluded)
|
|
deno task lint:analyze # Analyze lint issues by code/file grouping
|
|
deno fmt # Format
|
|
deno test --allow-all test/ # Run tests
|
|
deno publish --allow-slow-types --dry-run # Dry-run publish
|
|
```
|
|
|
|
## Source Heritage
|
|
|
|
The `graphs/` and `sqlite/` modules were adapted from
|
|
`@ade/ade-v0/packages/core/graphs` and `@ade/ade-v0/packages/storage_sqlite`.
|
|
The codebase has diverged significantly from the originals:
|
|
|
|
- All schemas use `Type.Module()` construction (not `SchemaBuilder`)
|
|
- `Metagraph`, `CallGraph`, `SecretGraph` are TypeBox Modules composing via
|
|
`Import()` and `Type.Composite()`
|
|
- Bridge functions (`moduleToDbSchema`, `validateNode`, `validateEdge`) project
|
|
Modules to DB row values
|
|
- Crypto utility ported from `@alkdev/hub/src/crypto/mod.ts` with `EncryptedDataSchema`
|
|
as a TypeBox schema
|
|
- `@sinclair/typebox` → `@alkdev/typebox`, `drizzle-typebox` → `@alkdev/drizzlebox`
|
|
- TypeScript enums replaced with `as const` objects (`GRAPH_STATUS`, `ACTOR_TYPE`)
|
|
- `Type.Unknown()` used for unvalidated fields (not `Type.Any()`)
|
|
- Injectable client pattern (`createSqliteDatabase(client)` takes a pre-created client)
|
|
- No module-level side effects or state
|
|
|
|
## File Conventions
|
|
|
|
- All source files use `.ts` extension with explicit extensions in imports (Deno
|
|
convention)
|
|
- Entry points are `mod.ts` files that re-export from subdirectories
|
|
- TypeBox schemas are named with PascalCase (`NodeType`, `GraphConfig`)
|
|
- Drizzle table objects are named with camelCase (`graphTypes`, `nodeTypes`)
|
|
- Schema objects from drizzlebox are named with PascalCase (`InsertGraph`,
|
|
`SelectGraph`)
|
|
- Enum constants use `SCREAMING_SNAKE_CASE` objects (`GRAPH_STATUS`,
|
|
`ACTOR_TYPE`)
|
|
|
|
## Architecture Docs
|
|
|
|
See `docs/architecture/` for detailed specifications:
|
|
|
|
- `overview.md` — Package purpose, exports, database model, ecosystem integration
|
|
- `metagraph-module.md` — Graph type definitions as TypeBox Modules, data model,
|
|
naming conventions, implementation path
|
|
- `honker-integration.md` — Drizzle-Honker adapter, event patterns, DB coordination
|
|
- `schema-evolution.md` — How graph type schemas evolve, TypeBox Value.Diff/Patch/Cast
|
|
for schema change detection and data migration
|
|
- `sqlite-host.md` — SQLite tables (metagraph + identity), client factories
|
|
- `encrypted-data.md` — Encrypted data design, crypto utility, node type modeling
|
|
- `forward-look.md` — Connections to dbtype, graph pointers, ujsx universal IR
|
|
- `acl.md` — Access control graph, principal-agent framework, scoping
|
|
|
|
These docs describe what the package is AND what it's becoming. Items marked ⚠️
|
|
are not yet implemented.
|
|
|
|
## What's Not Done Yet
|
|
|
|
- Drizzle-Honker session adapter (`src/sqlite/adapter.ts`)
|
|
- Identity tables in `src/sqlite/tables/identity/` (accounts, organizations, etc.)
|
|
- Scoping columns on `graphs` table (`ownerId`, `projectId`)
|
|
- Graph type `scope` column on `graph_types` table
|
|
- Remove `actors` table and `src/pg/` directory
|
|
- Split client factory into `createSystemDatabase()` / `createTenantDatabase()`
|
|
- Repository/CRUD layer (typed query functions beyond table definitions)
|
|
- JSR publication setup (need to create scope/package on jsr.io first)
|