storage/docs/architecture/forward-look.md

status, last_updated

status	last_updated
draft	2026-06-01

Forward Look: Pointers, dbtype, and Universal IR

How the Module-based metagraph connects to the broader @alkdev ecosystem — typed graph pointers, local utils (folded from dbtype), and the ujsx universal IR pipeline. The dbtype integration is no longer deferred (ADR-046) — the SQLite-only Phase 0 subset folds into src/sqlite/utils/. The repository surface is now OperationSpecs (ADR-048), not hand-written CRUD.

Overview

Three packages in the @alkdev ecosystem share the same pipeline shape:

Schema (TypeBox Module)  →  Element Tree (ujsx)  →  Host (HostConfig)

Package	Schema	Element tree	Host
@alkdev/ujsx	UJSX Module	<element>, <root>	DOM, custom
@alkdev/dbtype	Table/Column schemas	<table>, <column>	SQLite, PG, MySQL drizzle dialects
@alkdev/storage	Metagraph Module	⚠️ Future: <graphSchema>, <nodeType>	⚠️ Future: graph DB hosts

When storage's graph type definitions align with the Module pattern, they join this same pipeline. The immediate benefit is recursive/cross-referencing schemas (today). The forward benefit is that graph type definitions, table definitions, and pointer expressions can all be authored as ujsx element trees rendered to different hosts.

Pointer Abstraction

Addressing nodes and edges within a graph instance follows the same pattern as ujsx's ValuePointer and selectNode/setNode — and the same pattern as jsonpathly's JPATH Module for path expressions.

ujsx's pointer system (proven)

ujsx already implements a reactive pointer system:

class ValuePointer<T> {
  private _signal: Signal<T>;
  private _path: string[];
  get value(): T
  set value(v: T)
  get reactive(): ReadonlySignal<T>
  get path(): string[]
}

function selectNode(root: UNode, path: string[]): UNode | undefined
function setNode(root: UNode, path: string[], value: UNode): UNode

This addresses elements within a ujsx tree by path segments (child indices, prop names). A graph instance has analogous structure: nodes identified by key, edges identified by key, attributes addressed by JSON path.

Graph pointer analogy

// ujsx pointer: element tree → path → value
selectNode(root, ["children", 0, "props", "name"])

// Graph pointer: graph instance → path → value
selectNode(graph, ["nodes", "call-001", "attributes", "requestId"])

The structural analogy:

ujsx concept	Graph concept
Element tree root	Graph instance
UNode	Node or Edge
path: string[]	Key path: ["nodes", key] or ["edges", key]
selectNode(root, path)	selectGraphNode(graph, path)
setNode(root, path, value)	setGraphNode(graph, path, value) (via repository)

JPATH Module (jsonpathly)

The research shows that JSONPath expressions can themselves be a TypeBox Module (JPATH = Type.Module({...}) with recursive Type.Ref("Subscript")). This means pointer paths are not just runtime strings — they're typed schemas that can be validated and composed.

For graph storage, this opens the possibility of typed graph queries — a pointer expression like nodes.call-001.attributes.requestId has a schema that validates against the graph type's Module. If CallNode doesn't have a requestId field, the pointer expression is invalid at compile time.

Scope for v1

The pointer abstraction is a forward-looking design. For v1:

• Repository functions use direct key-based addressing: findNode(graphId, nodeKey), findEdge(graphId, edgeKey)
• Attribute access is untyped JSON retrieval: node.attributes.requestId
• The Module validates attribute shapes, but query paths are strings

The jump to typed pointers requires either the JPATH Module (for path validation) or ujsx-style ValuePointer with signals (for reactive graph observation). Both are post-v1 concerns, but the graph type Module makes them feasible because it provides the schema the pointer validates against.

Relationship to @alkdev/dbtype

@alkdev/dbtype defined database schemas as ujsx element trees and planned to render them to Drizzle dialects via HostConfig. Its Phase 0 (Drizzle→TypeBox schema generation) was consumed as @alkdev/drizzlebox. Phase 1 (UJSX→Drizzle) was never implemented.

Fold: Phase 0 → src/sqlite/utils/ (ADR-046)

With SQLite as the sole target (ADR-038), the multi-dialect column mappings in dbtype are dead weight. The SQLite-only subset has been folded into storage as src/sqlite/utils/:

What folds in	Source (dbtype)	Target (storage)
Schema generation	schema.ts	utils/schema.ts
Column→TypeBox mappings	column.ts (SQLite branches only)	utils/column.ts
Type interfaces	schema.types.ts + schema.types.internal.ts + column.types.ts	utils/types.ts
Integer constants	constants.ts	utils/constants.ts
Type guards	utils.ts (minus PgEnum)	utils/utils.ts

What does NOT fold in: PG, MySQL, SingleStore column handlers; isPgEnum / handleEnum; createSchemaFactory; the Phase 1 UJSX→HostConfig pipeline.

Import changes in table files:

// Before
import { createInsertSchema, createSelectSchema } from "@alkdev/drizzlebox";
// After
import { createInsertSchema, createSelectSchema } from "../utils/schema.ts";

The API surface is identical — same functions, same TypeBox schemas produced.

Phase 1 (UJSX→Drizzle): future path

The broader UJSX→HostConfig→Drizzle pipeline from dbtype's architecture remains architecturally sound but is not part of this pivot. When and if it's built, it could live in storage as a HostConfig sub-module rather than a separate package, since storage is the primary consumer. The TypeBox Module format used by the metagraph is already compatible with what a ujsx HostConfig would produce.

Why this matters for storage

1. Single source of truth: The utils/ code derives TypeBox schemas from Drizzle tables. Table definitions are the source of truth for both the DB schema and the validation schema.
2. Schema extraction: createSelectSchema / createInsertSchema produce TypeBox schemas that validate data at the application layer.
3. Module alignment: The metagraph Module and the table-derived schemas share the same TypeBox namespace. graph_types.config stores the JSON Schema from Metagraph.Config.

v1 approach

For v1, storage uses the folded utils for TypeBox schema derivation from Drizzle tables (what was @alkdev/drizzlebox). The metagraph Module independently validates graph type definitions. These two schema sources serve different purposes: table schemas validate DB row shapes, Module schemas validate graph type semantics.

When dbtype's Phase 1 (UJSX→HostConfig) is implemented, it would unify both directions — a TypeBox Module could produce both the Drizzle table definition and the validation schemas from the same element tree.

ujsx as Universal IR

The three packages (ujsx, dbtype, storage) share the same pipeline shape: Schema → Element Tree → Host. This is not coincidental — ujsx is a universal declarative IR, and different "render targets" are just different HostConfigs.

What this could look like

// Graph type definitions as ujsx elements (future)
const CallGraphSchema = h("graphSchema", { name: "call-graph" },
  h("config", { type: "directed", multi: false, allowSelfLoops: false }),
  h("nodeType", { name: "call" },
    h(BaseNode, {}),
    h("attr", { name: "requestId", type: "string", required: true }),
    h("attr", { name: "status", ref: "CallStatus" }),
  ),
  h("edgeType", { name: "triggered" },
    h(BaseEdge, {}),
    h("attr", { name: "type", literal: "triggered" }),
  ),
  h("edgeConstraints", { edgeType: "triggered",
    allowedSourceTypes: ["Call"],
    allowedTargetTypes: ["Call", "Subcall"] }),
);

Rendered to different hosts:

Host	Output
TypeBox Host	Type.Module({ CallNode: ..., TriggeredEdge: ... })
SQLite Host	sqliteTable("node_types", { ... }) + sqliteTable("edge_types", { ... })
PG Host	pgTable("node_types", { ... }) + pgTable("edge_types", { ... })
graphology Host	SerializedGraph format
Documentation Host	Mermaid diagram, typed API docs

What's real today vs. aspirational

Capability	Status
Type.Module for graph type definitions	✅ Implemented — Metagraph, CallGraph, SecretGraph Modules
Bridge functions (moduleToDbSchema, validateNode, validateEdge)	✅ Implemented
Reference graph type Modules (CallGraph, SecretGraph)	✅ Implemented
Crypto utility (encrypt, decrypt, generateEncryptionKey, EncryptedDataSchema)	✅ Implemented
Codegen from TypeScript interfaces → Module entries	✅ TsToModule exists
SQLite column→TypeBox mappings (folded from dbtype)	✅ Folded into src/sqlite/utils/ (ADR-046)
createSelectSchema / createInsertSchema (folded from drizzlebox)	✅ Folded into src/sqlite/utils/ (ADR-046)
Drizzle-Honker session adapter	✅ POC validated, implementation pending
HonkerEventTarget for pubsub	✅ POC validated, implementation pending
Transactional notify + outbox (Honker)	✅ POC validated — atomic commit for data + events + queue
OperationSpec generation from tables	⚠️ Design complete (ADR-048), implementation pending
Domain-specific native-column tables	⚠️ Conceptual — for known graph types (CallGraph, etc.)
<graphSchema> ujsx elements	⚠️ Conceptual — needs HostConfig design
Typed graph pointers via JPATH	⚠️ Conceptual — needs JPATH Module design
Reactive graph observation via ValuePointer	⚠️ Conceptual — needs signal integration
dbtype Phase 1 (UJSX→Drizzle HostConfig)	⚠️ Architecture exists, not implemented. Could live in storage if built.

The Module-based graph type definitions (this spec) are the first concrete step in this pipeline. Everything else builds on having a Type.Module as the schema source of truth.

Repository Layer Strategy

The repository layer (typed CRUD for the 6 metagraph tables + identity tables + queries for graph data) is now defined as OperationSpec output rather than hand-written query functions (ADR-048).

OperationSpecs as Repository Surface

Storage outputs OperationSpec[] per table — flat arrays describing CRUD operations. The consumer (hub/spoke) imports these, registers handlers, and the operations runtime handles execution, call protocol, and subscriptions.

// Storage defines the table + operation contracts
export const callNodes = sqliteTable("call_nodes", { ... });
export const callNodeSpecs: OperationSpec[] = [
  { name: "create", namespace: "call_nodes", type: "mutation", inputSchema: ..., outputSchema: ... },
  { name: "find",   namespace: "call_nodes", type: "query", ... },
  { name: "list",   namespace: "call_nodes", type: "query", ... },
  { name: "update", namespace: "call_nodes", type: "mutation", ... },
  { name: "delete", namespace: "call_nodes", type: "mutation", ... },
];

// Hub registers specs + handlers
for (const spec of callNodeSpecs) {
  registry.registerSpec(spec);
  registry.registerHandler(`${spec.namespace}.${spec.name}`, handler);
}

The handler is consumer-provided — not in storage. Storage doesn't execute queries. Storage defines the contract; the hub provides the execution layer.

Attribute Queries

The metagraph's attributes column remains JSON — node types are dynamic schemas defined at runtime, not static columns. Attribute queries use json_extract() for v1:

findNodes({ graphId, attributes: { status: "active" } })
// SQLite: json_extract(attributes, '$.status') = 'active'

For known graph types (CallGraph, SecretGraph), domain-specific tables with native columns can complement the generic metagraph tables. These domain tables also produce OperationSpecs with native-column queries.

Connection to @alkdev/operations

@alkdev/operations is a type-only peer dependency of storage. The OperationSpec type is straightforward. Storage builds the specs; the consumer wires them into the registry. No circular dependency.

v1 Decision

For v1, the practical path is OperationSpecs with JSON path attribute queries (ADR-048, supersedes ADR-033). Spec generation from tables is straightforward once domain tables exist. The metagraph's generic CRUD (graphs, nodes, edges) uses JSON attributes; domain-specific CRUD uses native columns. Both produce OperationSpecs that the hub registers in the same operations registry.

Constraints on Current Design

The forward-looking patterns documented here constrain the Module evolution design in metagraph-module.md:

1. The Module format must be self-contained — Type.Module({...}) entries with Type.Ref and Type.Composite are the same structures that a ujsx TypeBox Host would produce. If the Module format were an ad-hoc builder output, it couldn't be rendered by a different host later.

2. Edge constraints must be schema entries, not just DB columns — the constraint data needs to survive serialization/deserialization and be validatable independently. DB-only columns can't do this.

3. The base attribute schemas (BaseNode, BaseEdge) must be TypeBox schemas — not Drizzle column definitions, not builder-internal objects. Only TypeBox schemas can be composed via Type.Composite, referenced via Type.Ref, and serialized to JSON Schema.

4. No ujsx dependency — storage's Module-based graph types join the pipeline conceptually, not as a runtime dependency. The Type.Module output is the same shape that a ujsx HostConfig would produce, but storage doesn't need ujsx to create it. The alignment is structural, not dependent.

5. Schemas-as-JSON enables Value.Diff/Value.Patch/Value.Cast — because TypeBox Modules serialize to JSON Schema, the TypeBox value system can operate on schemas themselves (diff to detect changes, patch to update stored schemas, cast to migrate data). This is not possible if schemas are opaque builder objects or Drizzle column definitions. See schema-evolution.md.

6. OperationSpec output is consumer-agnostic — storage defines OperationSpec[] from table definitions. The consumer (hub/spoke) decides how to register handlers. Storage does not execute queries or depend on the operations runtime.

7. The folded utils are SQLite-only — src/sqlite/utils/ contains only SQLite column→TypeBox mappings. If a new database host is added later, the utils would need the corresponding dialect mappings. dbtype's Phase 1 (UJSX→HostConfig) would be the mechanism for multi-dialect support.

References

• ujsx pointer system: /workspace/@alkdev/ujsx/src/core/pointer.ts
• ujsx HostConfig adapter: /workspace/@alkdev/ujsx/src/host/config.ts
• dbtype architecture: /workspace/@alkdev/dbtype/docs/architecture/README.md (Phase 0 source folded into storage)
• dbtype elements: /workspace/@alkdev/dbtype/docs/architecture/elements.md
• dbtype module: /workspace/@alkdev/dbtype/docs/architecture/module.md
• dbtype repo adapter: /workspace/@alkdev/dbtype/docs/architecture/repo-adapter.md
• Operations registry: /workspace/@alkdev/operations/docs/architecture/README.md
• JPATH Module (JSONPath as TypeBox Module): /workspace/research/typebox_research/ujsx/jpath.gen.ts
• jsonpathly source: /workspace/jsonpathly/
• Module evolution spec: metagraph-module.md
• Schema evolution spec: schema-evolution.md
• ADR-046: Fold drizzlebox as utils (supersedes ADR-033)
• ADR-048: OperationSpecs as repository surface