storage/AGENTS.md

AGENTS.md — @alkdev/storage

Project-specific guidance for agents working on this package.

Project Overview

@alkdev/storage is a deno-first TypeScript package providing typed graph storage with SQLite via Honker. It uses the metagraph pattern (graphTypes → nodeTypes → edgeTypes → typed graph instances) and includes identity tables for multi-tenant auth. The system/tenant DB model separates identity infrastructure from org-scoped graph data.

Architecture Snapshot

@alkdev/storage/
├── mod.ts                   # Re-exports graphs/ only (zero db deps)
├── deno.json                # JSR config, imports, tasks, lint rules
├── src/
│   ├── graphs/              # Metagraph Module + bridge functions (no db deps)
│   │   ├── modules/         # TypeBox Module definitions
│   │   │   ├── metagraph.ts # Base Metagraph Module (Config, BaseNode, BaseEdge)
│   │   │   ├── call-graph.ts # CallGraph reference Module
│   │   │   ├── secret-graph.ts # SecretGraph reference Module
│   │   │   └── index.ts     # Barrel re-export
│   │   ├── bridge.ts       # moduleToDbSchema, validateNode, validateEdge
│   │   ├── crypto.ts       # encrypt, decrypt, generateEncryptionKey, EncryptedDataSchema
│   │   └── mod.ts           # Re-exports all graphs exports
│   └── sqlite/              # SQLite host (drizzle-orm + honker-node)
│       ├── tables/
│       │   ├── common.ts    # shared column definitions
│       │   ├── identity/    # accounts, organizations, org_members, api_keys, audit_logs
│       │   └── metagraph/   # graph_types, node_types, edge_types, graphs, nodes, edges
│       ├── utils/           # createSelectSchema, createInsertSchema, column mappings (ADR-046)
│       ├── relations.ts     # Drizzle relations
│       ├── adapter.ts       # Drizzle-Honker session adapter
│       ├── event-target.ts   # HonkerEventTarget (pubsub TypedEventTarget on Honker)
│       ├── schema.ts        # Re-exports
│       └── client.ts        # createSystemDatabase(), createTenantDatabase()
└── test/
    └── reference-modules.test.ts  # Metagraph, bridge, crypto tests

Subpath Exports (JSR/npm)

• @alkdev/storage → Metagraph Module, graph type definitions (zero deps)
• @alkdev/storage/sqlite → SQLite tables (metagraph + identity), relations, client, adapter, event-target, utils (drizzle-orm + honker-node, peer: @alkdev/pubsub)

PostgreSQL has been removed (ADR-038). SQLite via Honker is the sole database host.

Key Decisions

1. Deno-first, npm-second via JSR: Package is published to JSR (deno publish). npm compatibility is automatic via JSR's npm layer (@jsr/alkdev__storage). No separate dnt build step.
2. No comments in code: Per project convention across @alkdev packages.
3. JSR slow types excluded from lint: Drizzle's deeply inferred generics (sqliteTable, createInsertSchema, relations) make explicit type annotations impractical. We use --allow-slow-types on publish and "exclude": ["no-slow-types"] in lint config. This is known technical debt — can be tightened iteratively.
4. Injectable clients: createSystemDatabase(client) and createTenantDatabase(client) take pre-created Honker clients, not env vars. Module-level side effects are forbidden.
5. Dependencies: @alkdev/typebox is an npm dep (not yet on JSR). @alkdev/pubsub is a peer dep (for TypedEventTarget type). The @alkdev/drizzlebox external dep has been folded into src/sqlite/utils/ (ADR-046).
6. SQLite-only via Honker: No PostgreSQL. Honker provides DB + pub/sub + queues in a single SQLite file (ADR-038, ADR-039).
7. System/tenant DB split: Identity tables in system.db, graph data in tenant-{orgId}.db (ADR-040).
8. OperationSpecs as repository surface: Storage outputs OperationSpec[] from table definitions. No hand-written CRUD. The consumer (hub/spoke) registers handlers (ADR-048).
9. HonkerEventTarget: Bridges pubsub TypedEventTarget to Honker notify/listen and stream/subscribe. Enables single-node pub/sub without Redis. Transactional outbox semantics confirmed via POC (ADR-047).

Commands

deno check mod.ts src/graphs/mod.ts src/sqlite/mod.ts  # Type check
deno lint                                               # Lint (slow-types, verbatim-module-syntax excluded)
deno task lint:analyze                                  # Analyze lint issues by code/file grouping
deno fmt                                                # Format
deno test --allow-all test/                             # Run tests
deno publish --allow-slow-types --dry-run               # Dry-run publish

Source Heritage

The graphs/ and sqlite/ modules were adapted from @ade/ade-v0/packages/core/graphs and @ade/ade-v0/packages/storage_sqlite. The codebase has diverged significantly from the originals:

• All schemas use Type.Module() construction (not SchemaBuilder)
• Metagraph, CallGraph, SecretGraph are TypeBox Modules composing via Import() and Type.Composite()
• Bridge functions (moduleToDbSchema, validateNode, validateEdge) project Modules to DB row values
• Crypto utility ported from @alkdev/hub/src/crypto/mod.ts with EncryptedDataSchema as a TypeBox schema
• @sinclair/typebox → @alkdev/typebox, drizzle-typebox → @alkdev/drizzlebox → folded into src/sqlite/utils/ (ADR-046)
• TypeScript enums replaced with as const objects (GRAPH_STATUS, ACTOR_TYPE)
• Type.Unknown() used for unvalidated fields (not Type.Any())
• Injectable client pattern (createSqliteDatabase(client) takes a pre-created client)
• No module-level side effects or state

File Conventions

• All source files use .ts extension with explicit extensions in imports (Deno convention)
• Entry points are mod.ts files that re-export from subdirectories
• TypeBox schemas are named with PascalCase (NodeType, GraphConfig)
• Drizzle table objects are named with camelCase (graphTypes, nodeTypes)
• Schema objects from utils are named with PascalCase (InsertGraph, SelectGraph)
• Enum constants use SCREAMING_SNAKE_CASE objects (GRAPH_STATUS, ACTOR_TYPE)

Architecture Docs

See docs/architecture/ for detailed specifications:

• overview.md — Package purpose, exports, database model, ecosystem integration
• metagraph-module.md — Graph type definitions as TypeBox Modules, data model, naming conventions, implementation path
• honker-integration.md — Drizzle-Honker adapter, event patterns, DB coordination
• schema-evolution.md — How graph type schemas evolve, TypeBox Value.Diff/Patch/Cast for schema change detection and data migration
• sqlite-host.md — SQLite tables (metagraph + identity), client factories
• encrypted-data.md — Encrypted data design, crypto utility, node type modeling
• forward-look.md — Connections to dbtype, graph pointers, ujsx universal IR
• acl.md — Access control graph, principal-agent framework, scoping

These docs describe what the package is AND what it's becoming. Items marked ⚠️ are not yet implemented.

What's Not Done Yet

• Fold dbtype Phase 0 → src/sqlite/utils/ (ADR-046, import path changes)
• Drizzle-Honker session adapter (src/sqlite/adapter.ts, POC validated)
• HonkerEventTarget (src/sqlite/event-target.ts, POC validated, ADR-047)
• Identity tables in src/sqlite/tables/identity/ (accounts, organizations, etc.)
• Scoping columns on graphs table (ownerId, projectId)
• Graph type scope column on graph_types table
• Remove actors table and src/pg/ directory
• Update client factory to createSystemDatabase() / createTenantDatabase() (accept Honker client)
• Table restructure into subdirectories (metagraph/, identity/)
• OperationSpec generation from tables (ADR-048)
• ACL graph type (ADR-034)
• JSR publication setup (need to create scope/package on jsr.io first)