storage/AGENTS.md

# AGENTS.md — @alkdev/storage

Project-specific guidance for agents working on this package.

## Project Overview

`@alkdev/storage` is a deno-first TypeScript package providing typed graph
storage with SQLite via Honker. It uses the metagraph pattern (graphTypes →
nodeTypes → edgeTypes → typed graph instances) and includes identity tables
for multi-tenant auth. The system/tenant DB model separates identity
infrastructure from org-scoped graph data.

## Architecture Snapshot

```
@alkdev/storage/
├── mod.ts                   # Re-exports graphs/ only (zero db deps)
├── deno.json                # JSR config, imports, tasks, lint rules
├── src/
│   ├── graphs/              # Metagraph Module + bridge functions (no db deps)
│   │   ├── modules/         # TypeBox Module definitions
│   │   │   ├── metagraph.ts # Base Metagraph Module (Config, BaseNode, BaseEdge)
│   │   │   ├── call-graph.ts # CallGraph reference Module
│   │   │   ├── secret-graph.ts # SecretGraph reference Module
│   │   │   └── index.ts     # Barrel re-export
│   │   ├── bridge.ts       # moduleToDbSchema, validateNode, validateEdge
│   │   ├── crypto.ts       # encrypt, decrypt, generateEncryptionKey, EncryptedDataSchema
│   │   └── mod.ts           # Re-exports all graphs exports
│   └── sqlite/              # SQLite host (drizzle-orm + honker-node)
│       ├── tables/
│       │   ├── common.ts    # shared column definitions
│       │   ├── identity/    # accounts, organizations, org_members, api_keys, audit_logs
│       │   └── metagraph/   # graph_types, node_types, edge_types, graphs, nodes, edges
│       ├── utils/           # createSelectSchema, createInsertSchema, column mappings (ADR-046)
│       ├── relations.ts     # Drizzle relations
│       ├── adapter.ts       # Drizzle-Honker session adapter
│       ├── event-target.ts   # HonkerEventTarget (pubsub TypedEventTarget on Honker)
│       ├── schema.ts        # Re-exports
│       └── client.ts        # createSystemDatabase(), createTenantDatabase()
└── test/
    └── reference-modules.test.ts  # Metagraph, bridge, crypto tests
```

### Subpath Exports (JSR/npm)

- `@alkdev/storage` → Metagraph Module, graph type definitions (zero deps)
- `@alkdev/storage/sqlite` → SQLite tables (metagraph + identity), relations,
  client, adapter, event-target, utils (drizzle-orm + honker-node, peer: @alkdev/pubsub)

PostgreSQL has been removed (ADR-038). SQLite via Honker is the sole database host.

## Key Decisions

1. **Deno-first, npm-second via JSR**: Package is published to JSR
   (`deno publish`). npm compatibility is automatic via JSR's npm layer
   (`@jsr/alkdev__storage`). No separate dnt build step.
2. **No comments in code**: Per project convention across @alkdev packages.
3. **JSR slow types excluded from lint**: Drizzle's deeply inferred generics
   (`sqliteTable`, `createInsertSchema`, `relations`) make explicit type
   annotations impractical. We use `--allow-slow-types` on publish and
   `"exclude": ["no-slow-types"]` in lint config. This is known technical debt —
   can be tightened iteratively.
4. **Injectable clients**: `createSystemDatabase(client)` and
   `createTenantDatabase(client)` take pre-created Honker clients, not
   env vars. Module-level side effects are forbidden.
5. **Dependencies**: `@alkdev/typebox` is an npm dep (not yet on JSR).
   `@alkdev/pubsub` is a peer dep (for `TypedEventTarget` type). The
   `@alkdev/drizzlebox` external dep has been folded into `src/sqlite/utils/`
   (ADR-046).
6. **SQLite-only via Honker**: No PostgreSQL. Honker provides DB + pub/sub +
   queues in a single SQLite file (ADR-038, ADR-039).
7. **System/tenant DB split**: Identity tables in `system.db`, graph data in
   `tenant-{orgId}.db` (ADR-040).
8. **OperationSpecs as repository surface**: Storage outputs `OperationSpec[]`
   from table definitions. No hand-written CRUD. The consumer (hub/spoke)
   registers handlers (ADR-048).
9. **HonkerEventTarget**: Bridges pubsub `TypedEventTarget` to Honker
   `notify`/`listen` and `stream`/`subscribe`. Enables single-node pub/sub
   without Redis. Transactional outbox semantics confirmed via POC (ADR-047).

## Commands

```bash
deno check mod.ts src/graphs/mod.ts src/sqlite/mod.ts  # Type check
deno lint                                               # Lint (slow-types, verbatim-module-syntax excluded)
deno task lint:analyze                                  # Analyze lint issues by code/file grouping
deno fmt                                                # Format
deno test --allow-all test/                             # Run tests
deno publish --allow-slow-types --dry-run               # Dry-run publish
```

## Source Heritage

The `graphs/` and `sqlite/` modules were adapted from
`@ade/ade-v0/packages/core/graphs` and `@ade/ade-v0/packages/storage_sqlite`.
The codebase has diverged significantly from the originals:

- All schemas use `Type.Module()` construction (not `SchemaBuilder`)
- `Metagraph`, `CallGraph`, `SecretGraph` are TypeBox Modules composing via
  `Import()` and `Type.Composite()`
- Bridge functions (`moduleToDbSchema`, `validateNode`, `validateEdge`) project
  Modules to DB row values
- Crypto utility ported from `@alkdev/hub/src/crypto/mod.ts` with `EncryptedDataSchema`
  as a TypeBox schema
- `@sinclair/typebox` → `@alkdev/typebox`, `drizzle-typebox` → `@alkdev/drizzlebox` → folded into `src/sqlite/utils/` (ADR-046)
- TypeScript enums replaced with `as const` objects (`GRAPH_STATUS`, `ACTOR_TYPE`)
- `Type.Unknown()` used for unvalidated fields (not `Type.Any()`)
- Injectable client pattern (`createSqliteDatabase(client)` takes a pre-created client)
- No module-level side effects or state

## File Conventions

- All source files use `.ts` extension with explicit extensions in imports (Deno
  convention)
- Entry points are `mod.ts` files that re-export from subdirectories
- TypeBox schemas are named with PascalCase (`NodeType`, `GraphConfig`)
- Drizzle table objects are named with camelCase (`graphTypes`, `nodeTypes`)
- Schema objects from utils are named with PascalCase (`InsertGraph`,
  `SelectGraph`)
- Enum constants use `SCREAMING_SNAKE_CASE` objects (`GRAPH_STATUS`,
  `ACTOR_TYPE`)

## Architecture Docs

See `docs/architecture/` for detailed specifications:

- `overview.md` — Package purpose, exports, database model, ecosystem integration
- `metagraph-module.md` — Graph type definitions as TypeBox Modules, data model,
  naming conventions, implementation path
- `honker-integration.md` — Drizzle-Honker adapter, event patterns, DB coordination
- `schema-evolution.md` — How graph type schemas evolve, TypeBox Value.Diff/Patch/Cast
  for schema change detection and data migration
- `sqlite-host.md` — SQLite tables (metagraph + identity), client factories
- `encrypted-data.md` — Encrypted data design, crypto utility, node type modeling
- `forward-look.md` — Connections to dbtype, graph pointers, ujsx universal IR
- `acl.md` — Access control graph, principal-agent framework, scoping

These docs describe what the package is AND what it's becoming. Items marked ⚠️
are not yet implemented.

## What's Not Done Yet

- Fold dbtype Phase 0 → `src/sqlite/utils/` (ADR-046, import path changes)
- Drizzle-Honker session adapter (`src/sqlite/adapter.ts`, POC validated)
- HonkerEventTarget (`src/sqlite/event-target.ts`, POC validated, ADR-047)
- Identity tables in `src/sqlite/tables/identity/` (accounts, organizations, etc.)
- Scoping columns on `graphs` table (`ownerId`, `projectId`)
- Graph type `scope` column on `graph_types` table
- Remove `actors` table and `src/pg/` directory
- Update client factory to `createSystemDatabase()` / `createTenantDatabase()` (accept Honker client)
- Table restructure into subdirectories (metagraph/, identity/)
- OperationSpec generation from tables (ADR-048)
- ACL graph type (ADR-034)
- JSR publication setup (need to create scope/package on jsr.io first)