--- status: draft last_updated: 2026-06-01 --- # Forward Look: Pointers, dbtype, and Universal IR How the Module-based metagraph connects to the broader @alkdev ecosystem — typed graph pointers, local utils (folded from dbtype), and the ujsx universal IR pipeline. The dbtype integration is no longer deferred (ADR-046) — the SQLite-only Phase 0 subset folds into `src/sqlite/utils/`. The repository surface is now OperationSpecs (ADR-048), not hand-written CRUD. ## Overview Three packages in the @alkdev ecosystem share the same pipeline shape: ``` Schema (TypeBox Module) → Element Tree (ujsx) → Host (HostConfig) ``` | Package | Schema | Element tree | Host | |---------|--------|-------------|------| | `@alkdev/ujsx` | `UJSX` Module | ``, `` | DOM, custom | | `@alkdev/dbtype` | Table/Column schemas | ``, `` | SQLite, PG, MySQL drizzle dialects | | `@alkdev/storage` | `Metagraph` Module | ⚠️ Future: ``, `` | ⚠️ Future: graph DB hosts | When storage's graph type definitions align with the Module pattern, they join this same pipeline. The immediate benefit is recursive/cross-referencing schemas (today). The forward benefit is that graph type definitions, table definitions, and pointer expressions can all be authored as ujsx element trees rendered to different hosts. ## Pointer Abstraction Addressing nodes and edges within a graph instance follows the same pattern as ujsx's `ValuePointer` and `selectNode`/`setNode` — and the same pattern as jsonpathly's JPATH Module for path expressions. ### ujsx's pointer system (proven) ujsx already implements a reactive pointer system: ```ts class ValuePointer { private _signal: Signal; private _path: string[]; get value(): T set value(v: T) get reactive(): ReadonlySignal get path(): string[] } function selectNode(root: UNode, path: string[]): UNode | undefined function setNode(root: UNode, path: string[], value: UNode): UNode ``` This addresses elements within a ujsx tree by path segments (child indices, prop names). A graph instance has analogous structure: nodes identified by key, edges identified by key, attributes addressed by JSON path. ### Graph pointer analogy ```ts // ujsx pointer: element tree → path → value selectNode(root, ["children", 0, "props", "name"]) // Graph pointer: graph instance → path → value selectNode(graph, ["nodes", "call-001", "attributes", "requestId"]) ``` The structural analogy: | ujsx concept | Graph concept | |-------------|---------------| | Element tree root | Graph instance | | `UNode` | Node or Edge | | `path: string[]` | Key path: `["nodes", key]` or `["edges", key]` | | `selectNode(root, path)` | `selectGraphNode(graph, path)` | | `setNode(root, path, value)` | `setGraphNode(graph, path, value)` (via repository) | ### JPATH Module (jsonpathly) The research shows that JSONPath expressions can themselves be a TypeBox Module (`JPATH = Type.Module({...})` with recursive `Type.Ref("Subscript")`). This means pointer paths are not just runtime strings — they're typed schemas that can be validated and composed. For graph storage, this opens the possibility of **typed graph queries** — a pointer expression like `nodes.call-001.attributes.requestId` has a schema that validates against the graph type's Module. If `CallNode` doesn't have a `requestId` field, the pointer expression is invalid at compile time. ### Scope for v1 The pointer abstraction is a forward-looking design. For v1: - **Repository functions** use direct key-based addressing: `findNode(graphId, nodeKey)`, `findEdge(graphId, edgeKey)` - **Attribute access** is untyped JSON retrieval: `node.attributes.requestId` - **The Module** validates attribute shapes, but query paths are strings The jump to typed pointers requires either the JPATH Module (for path validation) or ujsx-style `ValuePointer` with signals (for reactive graph observation). Both are post-v1 concerns, but the graph type Module makes them feasible because it provides the schema the pointer validates against. ## Relationship to @alkdev/dbtype `@alkdev/dbtype` defined database schemas as ujsx element trees and planned to render them to Drizzle dialects via HostConfig. Its Phase 0 (Drizzle→TypeBox schema generation) was consumed as `@alkdev/drizzlebox`. Phase 1 (UJSX→Drizzle) was never implemented. ### Fold: Phase 0 → `src/sqlite/utils/` (ADR-046) With SQLite as the sole target (ADR-038), the multi-dialect column mappings in dbtype are dead weight. The SQLite-only subset has been folded into storage as `src/sqlite/utils/`: | What folds in | Source (dbtype) | Target (storage) | |---------------|-----------------|-------------------| | Schema generation | `schema.ts` | `utils/schema.ts` | | Column→TypeBox mappings | `column.ts` (SQLite branches only) | `utils/column.ts` | | Type interfaces | `schema.types.ts` + `schema.types.internal.ts` + `column.types.ts` | `utils/types.ts` | | Integer constants | `constants.ts` | `utils/constants.ts` | | Type guards | `utils.ts` (minus PgEnum) | `utils/utils.ts` | What does NOT fold in: PG, MySQL, SingleStore column handlers; `isPgEnum` / `handleEnum`; `createSchemaFactory`; the Phase 1 UJSX→HostConfig pipeline. Import changes in table files: ```ts // Before import { createInsertSchema, createSelectSchema } from "@alkdev/drizzlebox"; // After import { createInsertSchema, createSelectSchema } from "../utils/schema.ts"; ``` The API surface is identical — same functions, same TypeBox schemas produced. ### Phase 1 (UJSX→Drizzle): future path The broader UJSX→HostConfig→Drizzle pipeline from dbtype's architecture remains architecturally sound but is not part of this pivot. When and if it's built, it could live in storage as a `HostConfig` sub-module rather than a separate package, since storage is the primary consumer. The TypeBox Module format used by the metagraph is already compatible with what a ujsx HostConfig would produce. ### Why this matters for storage 1. **Single source of truth**: The `utils/` code derives TypeBox schemas from Drizzle tables. Table definitions are the source of truth for both the DB schema and the validation schema. 2. **Schema extraction**: `createSelectSchema` / `createInsertSchema` produce TypeBox schemas that validate data at the application layer. 3. **Module alignment**: The metagraph Module and the table-derived schemas share the same TypeBox namespace. `graph_types.config` stores the JSON Schema from `Metagraph.Config`. ### v1 approach For v1, storage uses the folded utils for TypeBox schema derivation from Drizzle tables (what was `@alkdev/drizzlebox`). The metagraph Module independently validates graph type definitions. These two schema sources serve different purposes: table schemas validate DB row shapes, Module schemas validate graph type semantics. When dbtype's Phase 1 (UJSX→HostConfig) is implemented, it would unify both directions — a TypeBox Module could produce both the Drizzle table definition and the validation schemas from the same element tree. ## ujsx as Universal IR The three packages (ujsx, dbtype, storage) share the same pipeline shape: **Schema → Element Tree → Host**. This is not coincidental — ujsx is a universal declarative IR, and different "render targets" are just different HostConfigs. ### What this could look like ```tsx // Graph type definitions as ujsx elements (future) const CallGraphSchema = h("graphSchema", { name: "call-graph" }, h("config", { type: "directed", multi: false, allowSelfLoops: false }), h("nodeType", { name: "call" }, h(BaseNode, {}), h("attr", { name: "requestId", type: "string", required: true }), h("attr", { name: "status", ref: "CallStatus" }), ), h("edgeType", { name: "triggered" }, h(BaseEdge, {}), h("attr", { name: "type", literal: "triggered" }), ), h("edgeConstraints", { edgeType: "triggered", allowedSourceTypes: ["Call"], allowedTargetTypes: ["Call", "Subcall"] }), ); ``` Rendered to different hosts: | Host | Output | |------|--------| | TypeBox Host | `Type.Module({ CallNode: ..., TriggeredEdge: ... })` | | SQLite Host | `sqliteTable("node_types", { ... })` + `sqliteTable("edge_types", { ... })` | | PG Host | `pgTable("node_types", { ... })` + `pgTable("edge_types", { ... })` | | graphology Host | `SerializedGraph` format | | Documentation Host | Mermaid diagram, typed API docs | ### What's real today vs. aspirational | Capability | Status | |-----------|--------| | `Type.Module` for graph type definitions | ✅ Implemented — Metagraph, CallGraph, SecretGraph Modules | | Bridge functions (`moduleToDbSchema`, `validateNode`, `validateEdge`) | ✅ Implemented | | Reference graph type Modules (CallGraph, SecretGraph) | ✅ Implemented | | Crypto utility (`encrypt`, `decrypt`, `generateEncryptionKey`, `EncryptedDataSchema`) | ✅ Implemented | | Codegen from TypeScript interfaces → Module entries | ✅ TsToModule exists | | SQLite column→TypeBox mappings (folded from dbtype) | ✅ Folded into `src/sqlite/utils/` (ADR-046) | | `createSelectSchema` / `createInsertSchema` (folded from drizzlebox) | ✅ Folded into `src/sqlite/utils/` (ADR-046) | | Drizzle-Honker session adapter | ✅ POC validated, implementation pending | | HonkerEventTarget for pubsub | ✅ POC validated, implementation pending | | Transactional notify + outbox (Honker) | ✅ POC validated — atomic commit for data + events + queue | | OperationSpec generation from tables | ⚠️ Design complete (ADR-048), implementation pending | | Domain-specific native-column tables | ⚠️ Conceptual — for known graph types (CallGraph, etc.) | | `` ujsx elements | ⚠️ Conceptual — needs HostConfig design | | Typed graph pointers via JPATH | ⚠️ Conceptual — needs JPATH Module design | | Reactive graph observation via ValuePointer | ⚠️ Conceptual — needs signal integration | | dbtype Phase 1 (UJSX→Drizzle HostConfig) | ⚠️ Architecture exists, not implemented. Could live in storage if built. | The Module-based graph type definitions (this spec) are the **first concrete step** in this pipeline. Everything else builds on having a `Type.Module` as the schema source of truth. ## Repository Layer Strategy The repository layer (typed CRUD for the 6 metagraph tables + identity tables + queries for graph data) is now defined as **OperationSpec output** rather than hand-written query functions (ADR-048). ### OperationSpecs as Repository Surface Storage outputs `OperationSpec[]` per table — flat arrays describing CRUD operations. The consumer (hub/spoke) imports these, registers handlers, and the operations runtime handles execution, call protocol, and subscriptions. ```ts // Storage defines the table + operation contracts export const callNodes = sqliteTable("call_nodes", { ... }); export const callNodeSpecs: OperationSpec[] = [ { name: "create", namespace: "call_nodes", type: "mutation", inputSchema: ..., outputSchema: ... }, { name: "find", namespace: "call_nodes", type: "query", ... }, { name: "list", namespace: "call_nodes", type: "query", ... }, { name: "update", namespace: "call_nodes", type: "mutation", ... }, { name: "delete", namespace: "call_nodes", type: "mutation", ... }, ]; // Hub registers specs + handlers for (const spec of callNodeSpecs) { registry.registerSpec(spec); registry.registerHandler(`${spec.namespace}.${spec.name}`, handler); } ``` The handler is consumer-provided — not in storage. Storage doesn't execute queries. Storage defines the contract; the hub provides the execution layer. ### Attribute Queries The metagraph's `attributes` column remains JSON — node types are dynamic schemas defined at runtime, not static columns. Attribute queries use `json_extract()` for v1: ```ts findNodes({ graphId, attributes: { status: "active" } }) // SQLite: json_extract(attributes, '$.status') = 'active' ``` For known graph types (CallGraph, SecretGraph), domain-specific tables with native columns can complement the generic metagraph tables. These domain tables also produce OperationSpecs with native-column queries. ### Connection to @alkdev/operations `@alkdev/operations` is a type-only peer dependency of storage. The `OperationSpec` type is straightforward. Storage builds the specs; the consumer wires them into the registry. No circular dependency. ### v1 Decision For v1, the practical path is **OperationSpecs with JSON path attribute queries** (ADR-048, supersedes ADR-033). Spec generation from tables is straightforward once domain tables exist. The metagraph's generic CRUD (graphs, nodes, edges) uses JSON attributes; domain-specific CRUD uses native columns. Both produce OperationSpecs that the hub registers in the same operations registry. ## Constraints on Current Design The forward-looking patterns documented here constrain the Module evolution design in [metagraph-module.md](./metagraph-module.md): 1. **The Module format must be self-contained** — `Type.Module({...})` entries with `Type.Ref` and `Type.Composite` are the same structures that a ujsx TypeBox Host would produce. If the Module format were an ad-hoc builder output, it couldn't be rendered by a different host later. 2. **Edge constraints must be schema entries, not just DB columns** — the constraint data needs to survive serialization/deserialization and be validatable independently. DB-only columns can't do this. 3. **The base attribute schemas (`BaseNode`, `BaseEdge`) must be TypeBox schemas** — not Drizzle column definitions, not builder-internal objects. Only TypeBox schemas can be composed via `Type.Composite`, referenced via `Type.Ref`, and serialized to JSON Schema. 4. **No ujsx dependency** — storage's Module-based graph types join the pipeline conceptually, not as a runtime dependency. The `Type.Module` output is the same shape that a ujsx HostConfig would produce, but storage doesn't need ujsx to create it. The alignment is structural, not dependent. 5. **Schemas-as-JSON enables `Value.Diff`/`Value.Patch`/`Value.Cast`** — because TypeBox Modules serialize to JSON Schema, the TypeBox value system can operate on schemas themselves (diff to detect changes, patch to update stored schemas, cast to migrate data). This is not possible if schemas are opaque builder objects or Drizzle column definitions. See [schema-evolution.md](./schema-evolution.md). 6. **OperationSpec output is consumer-agnostic** — storage defines `OperationSpec[]` from table definitions. The consumer (hub/spoke) decides how to register handlers. Storage does not execute queries or depend on the operations runtime. 7. **The folded utils are SQLite-only** — `src/sqlite/utils/` contains only SQLite column→TypeBox mappings. If a new database host is added later, the utils would need the corresponding dialect mappings. dbtype's Phase 1 (UJSX→HostConfig) would be the mechanism for multi-dialect support. ## References - ujsx pointer system: `/workspace/@alkdev/ujsx/src/core/pointer.ts` - ujsx HostConfig adapter: `/workspace/@alkdev/ujsx/src/host/config.ts` - dbtype architecture: `/workspace/@alkdev/dbtype/docs/architecture/README.md` (Phase 0 source folded into storage) - dbtype elements: `/workspace/@alkdev/dbtype/docs/architecture/elements.md` - dbtype module: `/workspace/@alkdev/dbtype/docs/architecture/module.md` - dbtype repo adapter: `/workspace/@alkdev/dbtype/docs/architecture/repo-adapter.md` - Operations registry: `/workspace/@alkdev/operations/docs/architecture/README.md` - JPATH Module (JSONPath as TypeBox Module): `/workspace/research/typebox_research/ujsx/jpath.gen.ts` - jsonpathly source: `/workspace/jsonpathly/` - Module evolution spec: [metagraph-module.md](./metagraph-module.md) - Schema evolution spec: [schema-evolution.md](./schema-evolution.md) - ADR-046: Fold drizzlebox as utils (supersedes ADR-033) - ADR-048: OperationSpecs as repository surface