Architect storage around SQLite+Honker: remove PG, add multi-tenant identity, scoping
Reorient @alkdev/storage around a single SQLite database host with Honker
for pub/sub, event streams, and task queues. PostgreSQL is removed as a
target (ADR-038), eliminating dual schema maintenance and infrastructure
complexity. Honker provides DB + pubsub + queues in one .db file (ADR-039).
Add system/tenant DB model (ADR-040): identity tables in system.db, all
graph data in tenant-{orgId}.db files. Identity tables move from the hub
into storage (ADR-041). Scoping columns (ownerId, projectId) added to
graphs table (ADR-042). Graph types get scope (system/tenant/user) to
protect infrastructure schemas (ADR-043).
Define Drizzle-Honker session adapter (ADR-044): ~100-line adapter enabling
Drizzle typed queries and Honker pubsub/queue on a single connection with
transactional consistency.
Resolve OQ-03, OQ-04, OQ-19, OQ-21, OQ-22, OQ-23, OQ-24. Add new
open questions OQ-26 through OQ-29 for Honker integration specifics.
New docs: honker-integration.md (adapter, event patterns, migration).
Scrub all PG/jsonb/libsql references from existing spec docs.
This commit is contained in:
@@ -1,51 +1,59 @@
|
||||
---
|
||||
status: reviewed
|
||||
last_updated: 2026-05-30
|
||||
status: draft
|
||||
last_updated: 2026-05-31
|
||||
---
|
||||
|
||||
# SQLite Host
|
||||
|
||||
The SQLite database host for `@alkdev/storage`. Uses Drizzle ORM with
|
||||
libsql/Turso for the SQLite dialect and `@alkdev/drizzlebox` for TypeBox schema
|
||||
generation from Drizzle table definitions.
|
||||
The SQLite database host for `@alkdev/storage`. Uses Drizzle ORM with Honker
|
||||
for database operations, pub/sub, event streams, and task queues. TypeBox
|
||||
schemas are auto-generated from Drizzle table definitions via `@alkdev/drizzlebox`.
|
||||
|
||||
## Overview
|
||||
|
||||
The SQLite host provides:
|
||||
|
||||
1. **Drizzle table definitions** for the metagraph pattern (graph types, node
|
||||
types, edge types, graphs, nodes, edges) plus a standalone `actors` table
|
||||
2. **Drizzle relations** for the relational query API
|
||||
3. **TypeBox schemas** auto-generated from Drizzle tables (select/insert
|
||||
validation)
|
||||
4. **Injectable database factory** — `createSqliteDatabase(client)` accepts a
|
||||
pre-created client
|
||||
|
||||
The SQLite host is the first-class target. PostgreSQL will follow the same table
|
||||
shapes with appropriate dialect changes.
|
||||
1. **Metagraph tables** — 6 tables for graph types, node types, edge types,
|
||||
graphs, nodes, and edges (ADR-002)
|
||||
2. **Identity tables** — accounts, organizations, organization_members, api_keys,
|
||||
audit_logs for multi-tenant authentication and authorization (ADR-041)
|
||||
3. **Drizzle relations** for the relational query API
|
||||
4. **TypeBox schemas** auto-generated from Drizzle tables (select/insert
|
||||
validation) via `@alkdev/drizzlebox`
|
||||
5. **Drizzle-Honker adapter** — thin session adapter for Honker integration
|
||||
(ADR-044)
|
||||
6. **Client factories** — `createSystemDatabase(client)` and
|
||||
`createTenantDatabase(client)` for the system/tenant DB model (ADR-040)
|
||||
|
||||
## Package Structure
|
||||
|
||||
```
|
||||
src/sqlite/
|
||||
├── tables/
|
||||
│ ├── common.ts # commonCols, ACTOR_TYPE enum
|
||||
│ ├── graphTypes.ts # graph_types table + select/insert schemas
|
||||
│ ├── nodeTypes.ts # node_types table + select/insert schemas
|
||||
│ ├── edgeTypes.ts # edge_types table + select/insert schemas
|
||||
│ ├── graphs.ts # graphs table + select/insert schemas
|
||||
│ ├── nodes.ts # nodes table + select/insert schemas
|
||||
│ ├── edges.ts # edges table + select/insert schemas
|
||||
│ ├── actors.ts # actors table + select/insert schemas
|
||||
│ └── index.ts # barrel re-export
|
||||
├── relations.ts # Drizzle relational mappings
|
||||
├── schema.ts # re-exports tables + relations
|
||||
└── client.ts # createSqliteDatabase()
|
||||
│ ├── common.ts # commonCols
|
||||
│ ├── identity/
|
||||
│ │ ├── accounts.ts # accounts table + select/insert schemas
|
||||
│ │ ├── organizations.ts # organizations table + select/insert schemas
|
||||
│ │ ├── organization_members.ts # org membership + select/insert schemas
|
||||
│ │ ├── api_keys.ts # API keys (keypal) + select/insert schemas
|
||||
│ │ ├── audit_logs.ts # audit trail + select/insert schemas
|
||||
│ │ └── index.ts # barrel re-export
|
||||
│ ├── metagraph/
|
||||
│ │ ├── graphTypes.ts # graph_types table + select/insert schemas
|
||||
│ │ ├── nodeTypes.ts # node_types table + select/insert schemas
|
||||
│ │ ├── edgeTypes.ts # edge_types table + select/insert schemas
|
||||
│ │ ├── graphs.ts # graphs table + select/insert schemas
|
||||
│ │ ├── nodes.ts # nodes table + select/insert schemas
|
||||
│ │ ├── edges.ts # edges table + select/insert schemas
|
||||
│ │ └── index.ts # barrel re-export
|
||||
│ └── index.ts # barrel re-export
|
||||
├── relations.ts # Drizzle relational mappings
|
||||
├── adapter.ts # Drizzle-Honker session adapter
|
||||
├── schema.ts # re-exports all tables + relations
|
||||
└── client.ts # createSystemDatabase(), createTenantDatabase()
|
||||
```
|
||||
|
||||
## Tables
|
||||
|
||||
### Common Columns
|
||||
## Common Columns
|
||||
|
||||
All tables share these columns:
|
||||
|
||||
@@ -62,23 +70,15 @@ All tables share these columns:
|
||||
}
|
||||
```
|
||||
|
||||
**Notable differences from a typical PostgreSQL common columns pattern**:
|
||||
- `id` is a consumer-generated UUID text PK (no `$defaultFn`)
|
||||
- `metadata` is an extension namespace following `_subsystem.key` convention
|
||||
- `createdAt`/`updatedAt` are Unix epoch integers with timestamp mode
|
||||
- No `$onUpdate` — consumers must set `updatedAt` explicitly
|
||||
|
||||
| Column | SQLite | PostgreSQL (typical) |
|
||||
| ----------- | ------------------------------------- | ------------------------------------------------------------- |
|
||||
| `id` | text PK (consumer-generated) | text PK with `$defaultFn(() => crypto.randomUUID())` |
|
||||
| `metadata` | `text` with JSON mode | `jsonb` with `$type<Record<string, unknown>>()` |
|
||||
| `createdAt` | `integer` timestamp mode (Unix epoch) | `timestamp with timezone` defaulting `now()` |
|
||||
| `updatedAt` | `integer` timestamp mode (Unix epoch) | `timestamp with timezone` defaulting `now()` with `$onUpdate` |
|
||||
|
||||
The SQLite columns do NOT have `$defaultFn` for ID generation (the consumer
|
||||
provides IDs) and do NOT have `$onUpdate` for `updatedAt` (Drizzle's `$onUpdate`
|
||||
is application-level; consumers must set it explicitly).
|
||||
## Metagraph Tables
|
||||
|
||||
### `graph_types`
|
||||
|
||||
Stores graph type definitions (schemas for classes of graphs).
|
||||
|
||||
| Column | Type | Constraints | Notes |
|
||||
| ----------- | ------------------- | ----------------------- | ------------------------------------------------------------ |
|
||||
| id | text | PK | Consumer-generated UUID |
|
||||
@@ -88,13 +88,15 @@ Stores graph type definitions (schemas for classes of graphs).
|
||||
| name | text | not null, **unique** | Graph type name (e.g., "call-graph", "acl") |
|
||||
| description | text | default `""` | Human-readable description |
|
||||
| config | text (JSON) | not null | `GraphConfig` — directed/undirected/mixed, multi, self-loops |
|
||||
| version | integer | not null, default 1 | Breaking schema version |
|
||||
| version | integer | not null, default 1 | Breaking schema version (ADR-029) |
|
||||
| scope | text | not null, default `"system"` | `system` / `tenant` / `user` (ADR-043) |
|
||||
|
||||
The `scope` column (ADR-043) controls who can create and modify graph type
|
||||
definitions. System-scoped types (`acl`, `call-graph`) are seeded at setup time
|
||||
and cannot be modified through the repository API.
|
||||
|
||||
### `node_types`
|
||||
|
||||
Stores node type definitions within a graph type. Each node type has a TypeBox
|
||||
schema that validates node attributes.
|
||||
|
||||
| Column | Type | Constraints | Notes |
|
||||
| ----------- | ------------------- | -------------------------------------- | ---------------------------------------- |
|
||||
| id | text | PK | |
|
||||
@@ -111,8 +113,6 @@ a graph type.
|
||||
|
||||
### `edge_types`
|
||||
|
||||
Stores edge type definitions within a graph type.
|
||||
|
||||
| Column | Type | Constraints | Notes |
|
||||
| ------------------ | ------------------- | -------------------------------------- | ---------------------------------------------- |
|
||||
| id | text | PK | |
|
||||
@@ -126,22 +126,11 @@ Stores edge type definitions within a graph type.
|
||||
| allowedSourceTypes | text (JSON) | default `[]` | Node type names valid at source endpoint |
|
||||
| allowedTargetTypes | text (JSON) | default `[]` | Node type names valid at target endpoint |
|
||||
|
||||
**Unique constraint**: `(graphTypeId, name)` — edge type names are unique within
|
||||
a graph type.
|
||||
|
||||
**Empty array semantics**: `allowedSourceTypes` and `allowedTargetTypes` default
|
||||
to `[]` (empty JSON array) in the database. `[]` means "no restriction" — any
|
||||
node type is a valid endpoint — matching the behavior of `undefined` in the
|
||||
`EdgeType` schema layer. A non-empty array restricts endpoints to only the
|
||||
listed node types. There is no "no types allowed" state; if edge types need to
|
||||
be disabled, use a status or soft-delete pattern on the edge type definition.
|
||||
The repository layer must enforce this convention consistently. See
|
||||
[metagraph-module.md](./metagraph-module.md) for edge endpoint semantics.
|
||||
**Unique constraint**: `(graphTypeId, name)`.
|
||||
**Empty array semantics**: `[]` means "no restriction" — any node type is valid.
|
||||
|
||||
### `graphs`
|
||||
|
||||
Graph instances. Each graph belongs to a graph type.
|
||||
|
||||
| Column | Type | Constraints | Notes |
|
||||
| ----------- | ------------------- | --------------------------------------------- | ---------------------------------------------- |
|
||||
| id | text | PK | |
|
||||
@@ -152,20 +141,25 @@ Graph instances. Each graph belongs to a graph type.
|
||||
| name | text | not null | Graph instance name |
|
||||
| description | text | default `""` | |
|
||||
| status | text | not null, enum: `active`, `archived`, `draft` | Default: `draft` |
|
||||
| ownerId | text | nullable | Logical reference to accounts.id (ADR-042) |
|
||||
| projectId | text | nullable | Logical reference to project identity (ADR-042) |
|
||||
|
||||
**On `graphTypeId` set null**: When a graph type is deleted, its graphs become
|
||||
orphans with `graphTypeId = null`. The application should prevent graph type
|
||||
deletion if active graphs reference it, or set affected graphs' `status` to
|
||||
`archived` as part of a soft-delete workflow. Orphan graphs cannot validate
|
||||
their node/edge types against a missing type definition — queries against orphan
|
||||
graphs should check for `graphTypeId !== null` before performing type-aware
|
||||
operations.
|
||||
**Scoping columns** (ADR-042): `ownerId` and `projectId` are logical references
|
||||
to entities in the system DB (accounts, projects). No FK constraint because the
|
||||
referenced tables live in a different database file. The consumer enforces
|
||||
referential integrity at the application layer.
|
||||
|
||||
No `orgId` column — the tenant DB file itself IS the org scope (ADR-040).
|
||||
|
||||
**Indexes**: `idx_graphs_owner_id` on `(ownerId)`, `idx_graphs_project_id` on
|
||||
`(projectId)`, `idx_graphs_owner_id_project_id` on `(ownerId, projectId)`.
|
||||
|
||||
**On `graphTypeId` set null**: Orphan graphs cannot validate their node/edge
|
||||
types against a missing type definition. The application should prevent graph
|
||||
type deletion if active graphs reference it.
|
||||
|
||||
### `nodes`
|
||||
|
||||
Nodes within a graph instance. Keyed by `(graphId, key)` — unique within a
|
||||
graph.
|
||||
|
||||
| Column | Type | Constraints | Notes |
|
||||
| ---------- | ------------------- | ---------------------------------- | --------------------------------------------- |
|
||||
| id | text | PK | |
|
||||
@@ -177,18 +171,10 @@ graph.
|
||||
| attributes | text (JSON) | not null, default `{}` | Node attributes validated by node type schema |
|
||||
|
||||
**Unique constraint**: `(graphId, key)` — node keys are unique within a graph.
|
||||
|
||||
**No `nodeTypeId` column**: Nodes do not have a direct FK to `node_types`. The
|
||||
node type is determined at the application layer. This is a deliberate design
|
||||
decision — adding a `nodeTypeId` FK would couple the graph instance layer to the
|
||||
type definition layer. The repository layer can enforce node type constraints
|
||||
via validation against the graph type's schema.
|
||||
**No `nodeTypeId` column**: ADR-020.
|
||||
|
||||
### `edges`
|
||||
|
||||
Edges within a graph instance. Keyed by `(graphId, key)` — unique within a
|
||||
graph.
|
||||
|
||||
| Column | Type | Constraints | Notes |
|
||||
| ------------- | ------------------- | ---------------------------------- | ---------------------------------------------------- |
|
||||
| id | text | PK | |
|
||||
@@ -202,32 +188,95 @@ graph.
|
||||
| attributes | text (JSON) | not null, default `{}` | Edge attributes validated by edge type schema |
|
||||
| undirected | integer (boolean) | default false | Treat as undirected regardless of graph type |
|
||||
|
||||
**Unique constraint**: `(graphId, key)` — edge keys are unique within a graph.
|
||||
|
||||
**Unique constraint**: `(graphId, key)`.
|
||||
**Foreign keys**: `sourceNodeKey` and `targetNodeKey` reference
|
||||
`(nodes.graphId, nodes.key)` with cascade delete. Deleting a node removes all
|
||||
its edges.
|
||||
`(nodes.graphId, nodes.key)` with cascade delete (ADR-022).
|
||||
|
||||
### `actors`
|
||||
## Identity Tables
|
||||
|
||||
Standalone identity table. Currently not referenced by any relation — the
|
||||
`actors` table has no FK references to or from any metagraph table and is not
|
||||
included in `relations.ts`. This is a placeholder for identity data and may
|
||||
become a node type in an ACL graph (based on `@alkdev/operations`'s `Identity`
|
||||
interface) or remain a standalone table. See OQ-03 in [open-questions.md](./open-questions.md).
|
||||
Identity tables live in the **system DB** (ADR-040, ADR-041). They provide
|
||||
multi-tenant authentication and authorization infrastructure. These tables are
|
||||
derived from the hub's existing identity tables; the schemas are aligned but
|
||||
simplified for the storage package's scope.
|
||||
|
||||
| Column | Type | Constraints | Notes |
|
||||
| --------- | ------------------- | --------------------------------------- | ------------------ |
|
||||
| id | text | PK | |
|
||||
| metadata | text (JSON) | default `{}` | |
|
||||
| createdAt | integer (timestamp) | not null, default `now` | |
|
||||
| updatedAt | integer (timestamp) | not null, default `now` | |
|
||||
| name | text | not null | Actor display name |
|
||||
| type | text | not null, enum: `human`, `llm`, `agent` | Actor type |
|
||||
### `accounts`
|
||||
|
||||
| Column | Type | Notes |
|
||||
|---------------|---------------------|-------|
|
||||
| commonCols | — | id, metadata, createdAt, updatedAt |
|
||||
| email | text NOT NULL UNIQUE | Unique identifier |
|
||||
| displayName | text | Display name |
|
||||
| accessLevel | text NOT NULL DEFAULT `user` | `admin`, `user`, `service` |
|
||||
| status | text NOT NULL DEFAULT `active` | `active`, `suspended`, `deactivated` |
|
||||
|
||||
**Indexes**: `unq_accounts_email` UNIQUE on `(email)`.
|
||||
|
||||
### `organizations`
|
||||
|
||||
| Column | Type | Notes |
|
||||
|----------|---------------------|-------|
|
||||
| commonCols | — | id, metadata, createdAt, updatedAt |
|
||||
| name | text NOT NULL UNIQUE | Organization name |
|
||||
| slug | text NOT NULL UNIQUE | URL-friendly identifier |
|
||||
| ownerId | text NOT NULL | Logical reference to accounts.id |
|
||||
|
||||
**Indexes**: `unq_organizations_name` UNIQUE on `(name)`, `unq_organizations_slug` UNIQUE on `(slug)`.
|
||||
|
||||
### `organization_members`
|
||||
|
||||
| Column | Type | Notes |
|
||||
|-----------------|---------------------|-------|
|
||||
| commonCols | — | id, metadata, createdAt, updatedAt |
|
||||
| orgId | text NOT NULL | FK → organizations.id (cascade) |
|
||||
| accountId | text NOT NULL | FK → accounts.id (cascade) |
|
||||
| membershipLevel | text NOT NULL | `owner`, `admin`, `member` |
|
||||
|
||||
**Unique constraint**: `(orgId, accountId)`.
|
||||
**Indexes**: `idx_org_members_account_id` on `(accountId)`.
|
||||
|
||||
This table is the authoritative source for org membership (ADR-045). The ACL
|
||||
graph's `BelongsToEdge` is derived from it — when membership changes, the
|
||||
consumer writes the SQL row first, then creates or removes the ACL edge.
|
||||
|
||||
### `api_keys`
|
||||
|
||||
| Column | Type | Notes |
|
||||
|------------|---------------------|-------|
|
||||
| commonCols | — | id, metadata, createdAt, updatedAt |
|
||||
| ownerId | text NOT NULL | Logical reference to accounts.id |
|
||||
| keyHash | text NOT NULL UNIQUE | SHA-256 hash (never stores raw key) |
|
||||
| name | text | Human-readable key label |
|
||||
| enabled | integer NOT NULL DEFAULT 1 | Disable without revoking |
|
||||
| expiresAt | integer (timestamp) | When the key expires (null = never) |
|
||||
| revokedAt | integer (timestamp) | When revoked (null = active) |
|
||||
|
||||
**Indexes**: `unq_api_keys_key_hash` UNIQUE on `(keyHash)`, `idx_api_keys_owner_id` on `(ownerId)`.
|
||||
|
||||
Keypal scope data is stored in `metadata` (`metadata.scopes`, `metadata.resources`).
|
||||
The hub provides a `HubKeyStorage` adapter that reads/writes this table to
|
||||
implement keypal's `Storage` interface.
|
||||
|
||||
### `audit_logs`
|
||||
|
||||
| Column | Type | Notes |
|
||||
|----------|---------------------|-------|
|
||||
| commonCols | — | id, metadata, createdAt, updatedAt |
|
||||
| action | text NOT NULL | `created`, `revoked`, `rotated`, `login`, `access_denied` |
|
||||
| ownerId | text NOT NULL | Logical reference to accounts.id |
|
||||
| keyId | text | Logical reference to api_keys.id (nullable) |
|
||||
| orgId | text | Logical reference to organizations.id (nullable) |
|
||||
| details | text (JSON) | Action-specific context |
|
||||
|
||||
**Indexes**: `idx_audit_logs_owner_id` on `(ownerId)`, `idx_audit_logs_action` on `(action)`, `idx_audit_logs_created_at` on `(createdAt)`.
|
||||
|
||||
## Relations
|
||||
|
||||
Drizzle relational mappings define the following relationships:
|
||||
### System DB Relations
|
||||
|
||||
- **organizations → organization_members**: one-to-many
|
||||
- **accounts → organization_members**: one-to-many
|
||||
|
||||
### Tenant DB Relations
|
||||
|
||||
- **graphTypes → nodeTypes**: one-to-many
|
||||
- **graphTypes → edgeTypes**: one-to-many
|
||||
@@ -236,99 +285,101 @@ Drizzle relational mappings define the following relationships:
|
||||
- **graphs → edges**: one-to-many
|
||||
- **nodes → outgoing edges** (sourceNode): one-to-many
|
||||
- **nodes → incoming edges** (targetNode): one-to-many
|
||||
- **edges → source node**: one-to-one (via composite key)
|
||||
- **edges → target node**: one-to-one (via composite key)
|
||||
|
||||
## Client Factory
|
||||
## Client Factories
|
||||
|
||||
### `createSystemDatabase(client)`
|
||||
|
||||
Creates a Drizzle database instance with the identity schema (accounts,
|
||||
organizations, organization_members, api_keys, audit_logs) attached.
|
||||
|
||||
```ts
|
||||
import { createSqliteDatabase } from "@alkdev/storage/sqlite";
|
||||
import type { SqliteDatabase } from "@alkdev/storage/sqlite";
|
||||
import { createClient } from "@libsql/client";
|
||||
import { createSystemDatabase } from "@alkdev/storage/sqlite";
|
||||
import { open } from "@russellthehipp/honker-node";
|
||||
|
||||
const client = createClient({ url: "file:local.db" });
|
||||
const db: SqliteDatabase = createSqliteDatabase(client);
|
||||
const honkerClient = open("system.db");
|
||||
const db = createSystemDatabase(honkerClient);
|
||||
|
||||
// Drizzle typed queries
|
||||
const admins = db.select().from(accounts).where(eq(accounts.accessLevel, "admin"));
|
||||
|
||||
// Honker features on the same connection
|
||||
db.$client.notify("account:created", { accountId: "user-1" });
|
||||
```
|
||||
|
||||
The factory takes a pre-created `@libsql/client` client and returns a typed
|
||||
Drizzle database instance with the full schema attached. This enables:
|
||||
### `createTenantDatabase(client)`
|
||||
|
||||
- In-memory testing with `createClient({ url: ":memory:" })`
|
||||
- Turso remote connections
|
||||
- Custom client configuration (auth tokens, etc.)
|
||||
Creates a Drizzle database instance with the metagraph schema (graph_types,
|
||||
node_types, edge_types, graphs, nodes, edges) attached.
|
||||
|
||||
```ts
|
||||
import { createTenantDatabase } from "@alkdev/storage/sqlite";
|
||||
import { open } from "@russellthehipp/honker-node";
|
||||
|
||||
const honkerClient = open("tenant-acme.db");
|
||||
const db = createTenantDatabase(honkerClient);
|
||||
|
||||
// Drizzle typed queries
|
||||
const activeGraphs = db.select().from(graphs).where(eq(graphs.status, "active"));
|
||||
|
||||
// Transactional: insert node + notify in one commit
|
||||
db.transaction((tx) => {
|
||||
tx.insert(nodes).values({ graphId, key: "call-1", attributes: {} }).run();
|
||||
tx.$honkerTx.notify("nodes:created", { graphId, key: "call-1" });
|
||||
});
|
||||
```
|
||||
|
||||
## Design Decisions
|
||||
|
||||
All design decisions are documented as ADRs in [decisions/](decisions/).
|
||||
|
||||
| ADR | Decision | Summary |
|
||||
|-----|----------|---------|
|
||||
| [019](decisions/019-json-text-for-schema-columns.md) | JSON text for schema columns in SQLite | SQLite uses `text` with JSON mode; application-level validation |
|
||||
| [020](decisions/020-no-nodetypeid-on-nodes.md) | No nodeTypeId on nodes | Node type enforced at application layer, not via FK |
|
||||
| [021](decisions/021-edge-identity-uses-consumer-keys.md) | Edge identity uses consumer-defined keys | `(graphId, key)` as unique identity within a graph |
|
||||
| [022](decisions/022-composite-fks-for-node-references.md) | Composite foreign keys for node references | Edges reference `(graphId, sourceNodeKey) → (nodes.graphId, nodes.key)` |
|
||||
| [006](decisions/006-enum-pattern-as-const-objects.md) | `as const` objects, not TypeScript enums | `GRAPH_STATUS`, `ACTOR_TYPE` use const objects; TypeBox uses Literal unions |
|
||||
| [008](decisions/008-common-columns-pattern.md) | Common columns pattern | `id`, `metadata`, `createdAt`, `updatedAt` on every table |
|
||||
| [038](decisions/038-sqlite-first-pg-removed.md) | SQLite-first, PG removed | Single database host |
|
||||
| [039](decisions/039-honker-as-sqlite-extension.md) | Honker as SQLite extension | DB + pub/sub + queues in one file |
|
||||
| [040](decisions/040-system-db-tenant-db.md) | System DB + tenant DB | Identity in system.db, graphs in tenant-{orgId}.db |
|
||||
| [041](decisions/041-identity-tables-in-storage.md) | Identity tables in storage | accounts, organizations, api_keys, audit_logs |
|
||||
| [042](decisions/042-scoping-columns-on-graphs.md) | Scoping columns on graphs | `ownerId`, `projectId` on `graphs` table |
|
||||
| [043](decisions/043-graph-type-scope.md) | Graph type scope | `system` / `tenant` / `user` scope on `graph_types` |
|
||||
| [044](decisions/044-drizzle-honker-adapter.md) | Drizzle-Honker adapter | ~100-line session adapter |
|
||||
| [045](decisions/045-org-members-authoritative-belongsto-derived.md) | org_members authoritative | SQL table is source of truth; BelongsToEdge is derived |
|
||||
| [019](decisions/019-json-text-for-schema-columns.md) | JSON text for schema columns | SQLite uses `text` with JSON mode |
|
||||
| [020](decisions/020-no-nodetypeid-on-nodes.md) | No nodeTypeId on nodes | Node type enforced at application layer |
|
||||
| [022](decisions/022-composite-fks-for-node-references.md) | Composite FKs for node refs | Edges reference `(graphId, sourceNodeKey)` |
|
||||
| [008](decisions/008-common-columns-pattern.md) | Common columns pattern | `id`, `metadata`, `createdAt`, `updatedAt` |
|
||||
|
||||
## Metadata Convention
|
||||
## Removed: `actors` Table
|
||||
|
||||
Every table has a `metadata` JSON column defaulting to `{}`. This is an
|
||||
extension namespace for subsystem use, following a namespacing convention:
|
||||
`_subsystem.key` (e.g., `_keypal.scopes`, `_retention.expiresAt`).
|
||||
The `actors` table is removed per ADR-035. `ACTOR_TYPE` is replaced by the
|
||||
`IdentityType` enum in the AclGraph Module. Identity data lives in the
|
||||
`accounts` table (system DB) and `PrincipalNode` in ACL graph instances
|
||||
(tenant DB).
|
||||
|
||||
**What metadata is for**: Opaque key-value pairs that subsystems add without
|
||||
schema changes. It's never queried in WHERE clauses or JOINs.
|
||||
## Removed: PostgreSQL Porting Notes
|
||||
|
||||
**What metadata is NOT for**: A replacement for typed columns. If a field
|
||||
appears in WHERE clauses, JOIN conditions, or needs a constraint, it should be a
|
||||
proper column — not buried in metadata. When in doubt, add a column.
|
||||
|
||||
**Namespacing convention**: Subsystems should prefix their keys (e.g.,
|
||||
`_callgraph.payloadRef`, `_acl.inherited`). Unprefixed keys are reserved for the
|
||||
storage package itself.
|
||||
PostgreSQL is no longer a target (ADR-038). All porting notes from the previous
|
||||
version of this document are obsolete. The single database host is SQLite via
|
||||
Honker.
|
||||
|
||||
## Concurrency Model
|
||||
|
||||
The SQLite host targets spoke deployments where a single process accesses the
|
||||
database. For this model, SQLite's default journal mode is sufficient. However,
|
||||
for spoke deployments that may run concurrent writes (e.g., multiple worker
|
||||
threads), consumers should:
|
||||
Honker opens databases in WAL mode with a bounded reader pool and single writer
|
||||
slot. This handles the expected concurrency for the hub use case:
|
||||
|
||||
1. **Enable WAL mode**: `PRAGMA journal_mode=WAL;` — allows concurrent reads
|
||||
during writes
|
||||
2. **Set busy timeout**: `PRAGMA busy_timeout=5000;` — wait up to 5 seconds for
|
||||
lock acquisition
|
||||
3. **Use a single writer**: SQLite supports one writer at a time. If multiple
|
||||
threads write, route writes through a single queue or connection
|
||||
- **Reader pool**: Up to `maxReaders` (default 4) concurrent read connections.
|
||||
`db.$client.query()` uses the pool automatically.
|
||||
- **Writer slot**: Single exclusive writer, acquired by `transaction()`. If the
|
||||
slot is occupied, subsequent `transaction()` calls block until released.
|
||||
- **Write timeout**: Honker sets `busy_timeout=5000` by default. Configurable at
|
||||
`open()` time.
|
||||
- **WAL mode**: Enables concurrent reads during writes. Required by Honker's
|
||||
reader pool architecture.
|
||||
|
||||
The `createSqliteDatabase()` factory does not set these pragmas — it's the
|
||||
consumer's responsibility to configure the SQLite connection appropriately. The
|
||||
libsql client used to create the connection can be pre-configured before passing
|
||||
it to the factory.
|
||||
|
||||
## PostgreSQL Porting Notes
|
||||
|
||||
When implementing `src/pg/`, the table shapes remain the same but with these
|
||||
changes:
|
||||
|
||||
| SQLite | PostgreSQL |
|
||||
| -------------------------------- | ---------------------------------------- |
|
||||
| `sqliteTable` | `pgTable` |
|
||||
| `text` (JSON mode) | `jsonb` with `.$type<T>()` |
|
||||
| `integer` (timestamp mode) | `timestamp` with timezone |
|
||||
| `sql\`(strftime('%s', 'now'))\`` | `sql\`now()\`` |
|
||||
| `integer` (boolean mode) | `boolean` |
|
||||
| `text` (enum) | `pgEnum` or `text` with check constraint |
|
||||
|
||||
See a consumer's `commonCols` pattern (e.g., the hub's
|
||||
`/workspace/@alkdev/hub/docs/architecture/storage/table-reference.md`) for
|
||||
PostgreSQL reference patterns.
|
||||
For multi-process deployments, set WAL mode and ensure the busy timeout is
|
||||
sufficient for expected lock contention.
|
||||
|
||||
## References
|
||||
|
||||
- Drizzle ORM SQLite core: https://orm.drizzle.team/docs/sqlite-core
|
||||
- libsql client: https://github.com/tursodatabase/libsql
|
||||
- Hub common columns (reference consumer):
|
||||
`/workspace/@alkdev/hub/docs/architecture/storage/table-reference.md`
|
||||
- Operations AccessControl and Identity: `/workspace/@alkdev/operations/docs/architecture/api-surface.md`
|
||||
- Source: `src/sqlite/`
|
||||
- Honker source: `/workspace/honker/`
|
||||
- Honker Node binding: `/workspace/honker/packages/honker-node/`
|
||||
- Hub identity tables (provenance): `/workspace/@alkdev/hub/docs/architecture/storage/identity.md`
|
||||
- Operations AccessControl: `/workspace/@alkdev/operations/docs/architecture/api-surface.md`
|
||||
- Source: `src/sqlite/`
|
||||
Reference in New Issue
Block a user