storage/docs/architecture/forward-look.md

---
status: draft
last_updated: 2026-06-01
---

# Forward Look: Pointers, dbtype, and Universal IR

How the Module-based metagraph connects to the broader @alkdev ecosystem —
typed graph pointers, local utils (folded from dbtype), and the ujsx universal IR
pipeline. The dbtype integration is no longer deferred (ADR-046) — the SQLite-only
Phase 0 subset folds into `src/sqlite/utils/`. The repository surface is now
OperationSpecs (ADR-048), not hand-written CRUD.

## Overview

Three packages in the @alkdev ecosystem share the same pipeline shape:

```
Schema (TypeBox Module)  →  Element Tree (ujsx)  →  Host (HostConfig)
```

| Package | Schema | Element tree | Host |
|---------|--------|-------------|------|
| `@alkdev/ujsx` | `UJSX` Module | `<element>`, `<root>` | DOM, custom |
| `@alkdev/dbtype` | Table/Column schemas | `<table>`, `<column>` | SQLite, PG, MySQL drizzle dialects |
| `@alkdev/storage` | `Metagraph` Module | ⚠️ Future: `<graphSchema>`, `<nodeType>` | ⚠️ Future: graph DB hosts |

When storage's graph type definitions align with the Module pattern, they
join this same pipeline. The immediate benefit is recursive/cross-referencing
schemas (today). The forward benefit is that graph type definitions, table
definitions, and pointer expressions can all be authored as ujsx element trees
rendered to different hosts.

## Pointer Abstraction

Addressing nodes and edges within a graph instance follows the same pattern as
ujsx's `ValuePointer` and `selectNode`/`setNode` — and the same pattern as
jsonpathly's JPATH Module for path expressions.

### ujsx's pointer system (proven)

ujsx already implements a reactive pointer system:

```ts
class ValuePointer<T> {
  private _signal: Signal<T>;
  private _path: string[];
  get value(): T
  set value(v: T)
  get reactive(): ReadonlySignal<T>
  get path(): string[]
}

function selectNode(root: UNode, path: string[]): UNode | undefined
function setNode(root: UNode, path: string[], value: UNode): UNode
```

This addresses elements within a ujsx tree by path segments (child indices,
prop names). A graph instance has analogous structure: nodes identified by
key, edges identified by key, attributes addressed by JSON path.

### Graph pointer analogy

```ts
// ujsx pointer: element tree → path → value
selectNode(root, ["children", 0, "props", "name"])

// Graph pointer: graph instance → path → value
selectNode(graph, ["nodes", "call-001", "attributes", "requestId"])
```

The structural analogy:

| ujsx concept | Graph concept |
|-------------|---------------|
| Element tree root | Graph instance |
| `UNode` | Node or Edge |
| `path: string[]` | Key path: `["nodes", key]` or `["edges", key]` |
| `selectNode(root, path)` | `selectGraphNode(graph, path)` |
| `setNode(root, path, value)` | `setGraphNode(graph, path, value)` (via repository) |

### JPATH Module (jsonpathly)

The research shows that JSONPath expressions can themselves be a TypeBox Module
(`JPATH = Type.Module({...})` with recursive `Type.Ref("Subscript")`). This means
pointer paths are not just runtime strings — they're typed schemas that can be
validated and composed.

For graph storage, this opens the possibility of **typed graph queries** — a
pointer expression like `nodes.call-001.attributes.requestId` has a schema that
validates against the graph type's Module. If `CallNode` doesn't have a
`requestId` field, the pointer expression is invalid at compile time.

### Scope for v1

The pointer abstraction is a forward-looking design. For v1:

- **Repository functions** use direct key-based addressing:
  `findNode(graphId, nodeKey)`, `findEdge(graphId, edgeKey)`
- **Attribute access** is untyped JSON retrieval:
  `node.attributes.requestId`
- **The Module** validates attribute shapes, but query paths are strings

The jump to typed pointers requires either the JPATH Module (for path
validation) or ujsx-style `ValuePointer` with signals (for reactive graph
observation). Both are post-v1 concerns, but the graph type Module makes them
feasible because it provides the schema the pointer validates against.

## Relationship to @alkdev/dbtype

`@alkdev/dbtype` defined database schemas as ujsx element trees and planned to
render them to Drizzle dialects via HostConfig. Its Phase 0 (Drizzle→TypeBox
schema generation) was consumed as `@alkdev/drizzlebox`. Phase 1 (UJSX→Drizzle)
was never implemented.

### Fold: Phase 0 → `src/sqlite/utils/` (ADR-046)

With SQLite as the sole target (ADR-038), the multi-dialect column mappings in
dbtype are dead weight. The SQLite-only subset has been folded into storage as
`src/sqlite/utils/`:

| What folds in | Source (dbtype) | Target (storage) |
|---------------|-----------------|-------------------|
| Schema generation | `schema.ts` | `utils/schema.ts` |
| Column→TypeBox mappings | `column.ts` (SQLite branches only) | `utils/column.ts` |
| Type interfaces | `schema.types.ts` + `schema.types.internal.ts` + `column.types.ts` | `utils/types.ts` |
| Integer constants | `constants.ts` | `utils/constants.ts` |
| Type guards | `utils.ts` (minus PgEnum) | `utils/utils.ts` |

What does NOT fold in: PG, MySQL, SingleStore column handlers; `isPgEnum` /
`handleEnum`; `createSchemaFactory`; the Phase 1 UJSX→HostConfig pipeline.

Import changes in table files:

```ts
// Before
import { createInsertSchema, createSelectSchema } from "@alkdev/drizzlebox";
// After
import { createInsertSchema, createSelectSchema } from "../utils/schema.ts";
```

The API surface is identical — same functions, same TypeBox schemas produced.

### Phase 1 (UJSX→Drizzle): future path

The broader UJSX→HostConfig→Drizzle pipeline from dbtype's architecture remains
architecturally sound but is not part of this pivot. When and if it's built, it
could live in storage as a `HostConfig` sub-module rather than a separate
package, since storage is the primary consumer. The TypeBox Module format used
by the metagraph is already compatible with what a ujsx HostConfig would produce.

### Why this matters for storage

1. **Single source of truth**: The `utils/` code derives TypeBox schemas from
   Drizzle tables. Table definitions are the source of truth for both the DB
   schema and the validation schema.
2. **Schema extraction**: `createSelectSchema` / `createInsertSchema` produce
   TypeBox schemas that validate data at the application layer.
3. **Module alignment**: The metagraph Module and the table-derived schemas
   share the same TypeBox namespace. `graph_types.config` stores the JSON
   Schema from `Metagraph.Config`.

### v1 approach

For v1, storage uses the folded utils for TypeBox schema derivation from Drizzle
tables (what was `@alkdev/drizzlebox`). The metagraph Module independently
validates graph type definitions. These two schema sources serve different
purposes: table schemas validate DB row shapes, Module schemas validate graph
type semantics.

When dbtype's Phase 1 (UJSX→HostConfig) is implemented, it would unify both
directions — a TypeBox Module could produce both the Drizzle table definition
and the validation schemas from the same element tree.

## ujsx as Universal IR

The three packages (ujsx, dbtype, storage) share the same pipeline shape:
**Schema → Element Tree → Host**. This is not coincidental — ujsx is a
universal declarative IR, and different "render targets" are just different
HostConfigs.

### What this could look like

```tsx
// Graph type definitions as ujsx elements (future)
const CallGraphSchema = h("graphSchema", { name: "call-graph" },
  h("config", { type: "directed", multi: false, allowSelfLoops: false }),
  h("nodeType", { name: "call" },
    h(BaseNode, {}),
    h("attr", { name: "requestId", type: "string", required: true }),
    h("attr", { name: "status", ref: "CallStatus" }),
  ),
  h("edgeType", { name: "triggered" },
    h(BaseEdge, {}),
    h("attr", { name: "type", literal: "triggered" }),
  ),
  h("edgeConstraints", { edgeType: "triggered",
    allowedSourceTypes: ["Call"],
    allowedTargetTypes: ["Call", "Subcall"] }),
);
```

Rendered to different hosts:

| Host | Output |
|------|--------|
| TypeBox Host | `Type.Module({ CallNode: ..., TriggeredEdge: ... })` |
| SQLite Host | `sqliteTable("node_types", { ... })` + `sqliteTable("edge_types", { ... })` |
| PG Host | `pgTable("node_types", { ... })` + `pgTable("edge_types", { ... })` |
| graphology Host | `SerializedGraph` format |
| Documentation Host | Mermaid diagram, typed API docs |

### What's real today vs. aspirational

| Capability | Status |
|-----------|--------|
| `Type.Module` for graph type definitions | ✅ Implemented — Metagraph, CallGraph, SecretGraph Modules |
| Bridge functions (`moduleToDbSchema`, `validateNode`, `validateEdge`) | ✅ Implemented |
| Reference graph type Modules (CallGraph, SecretGraph) | ✅ Implemented |
| Crypto utility (`encrypt`, `decrypt`, `generateEncryptionKey`, `EncryptedDataSchema`) | ✅ Implemented |
| Codegen from TypeScript interfaces → Module entries | ✅ TsToModule exists |
| SQLite column→TypeBox mappings (folded from dbtype) | ✅ Folded into `src/sqlite/utils/` (ADR-046) |
| `createSelectSchema` / `createInsertSchema` (folded from drizzlebox) | ✅ Folded into `src/sqlite/utils/` (ADR-046) |
| Drizzle-Honker session adapter | ✅ POC validated, implementation pending |
| HonkerEventTarget for pubsub | ✅ POC validated, implementation pending |
| Transactional notify + outbox (Honker) | ✅ POC validated — atomic commit for data + events + queue |
| OperationSpec generation from tables | ⚠️ Design complete (ADR-048), implementation pending |
| Domain-specific native-column tables | ⚠️ Conceptual — for known graph types (CallGraph, etc.) |
| `<graphSchema>` ujsx elements | ⚠️ Conceptual — needs HostConfig design |
| Typed graph pointers via JPATH | ⚠️ Conceptual — needs JPATH Module design |
| Reactive graph observation via ValuePointer | ⚠️ Conceptual — needs signal integration |
| dbtype Phase 1 (UJSX→Drizzle HostConfig) | ⚠️ Architecture exists, not implemented. Could live in storage if built. |

The Module-based graph type definitions (this spec) are the **first concrete
step** in this pipeline. Everything else builds on having a `Type.Module` as
the schema source of truth.

## Repository Layer Strategy

The repository layer (typed CRUD for the 6 metagraph tables + identity tables +
queries for graph data) is now defined as **OperationSpec output** rather than
hand-written query functions (ADR-048).

### OperationSpecs as Repository Surface

Storage outputs `OperationSpec[]` per table — flat arrays describing CRUD
operations. The consumer (hub/spoke) imports these, registers handlers, and
the operations runtime handles execution, call protocol, and subscriptions.

```ts
// Storage defines the table + operation contracts
export const callNodes = sqliteTable("call_nodes", { ... });
export const callNodeSpecs: OperationSpec[] = [
  { name: "create", namespace: "call_nodes", type: "mutation", inputSchema: ..., outputSchema: ... },
  { name: "find",   namespace: "call_nodes", type: "query", ... },
  { name: "list",   namespace: "call_nodes", type: "query", ... },
  { name: "update", namespace: "call_nodes", type: "mutation", ... },
  { name: "delete", namespace: "call_nodes", type: "mutation", ... },
];

// Hub registers specs + handlers
for (const spec of callNodeSpecs) {
  registry.registerSpec(spec);
  registry.registerHandler(`${spec.namespace}.${spec.name}`, handler);
}
```

The handler is consumer-provided — not in storage. Storage doesn't execute
queries. Storage defines the contract; the hub provides the execution layer.

### Attribute Queries

The metagraph's `attributes` column remains JSON — node types are dynamic
schemas defined at runtime, not static columns. Attribute queries use
`json_extract()` for v1:

```ts
findNodes({ graphId, attributes: { status: "active" } })
// SQLite: json_extract(attributes, '$.status') = 'active'
```

For known graph types (CallGraph, SecretGraph), domain-specific tables with
native columns can complement the generic metagraph tables. These domain
tables also produce OperationSpecs with native-column queries.

### Connection to @alkdev/operations

`@alkdev/operations` is a type-only peer dependency of storage. The
`OperationSpec` type is straightforward. Storage builds the specs; the
consumer wires them into the registry. No circular dependency.

### v1 Decision

For v1, the practical path is **OperationSpecs with JSON path attribute
queries** (ADR-048, supersedes ADR-033). Spec generation from tables is
straightforward once domain tables exist. The metagraph's generic CRUD
(graphs, nodes, edges) uses JSON attributes; domain-specific CRUD uses
native columns. Both produce OperationSpecs that the hub registers in the
same operations registry.

## Constraints on Current Design

The forward-looking patterns documented here constrain the Module evolution
design in [metagraph-module.md](./metagraph-module.md):

1. **The Module format must be self-contained** — `Type.Module({...})` entries
   with `Type.Ref` and `Type.Composite` are the same structures that a ujsx
   TypeBox Host would produce. If the Module format were an ad-hoc builder
   output, it couldn't be rendered by a different host later.

2. **Edge constraints must be schema entries, not just DB columns** — the
   constraint data needs to survive serialization/deserialization and be
   validatable independently. DB-only columns can't do this.

3. **The base attribute schemas (`BaseNode`, `BaseEdge`) must be TypeBox
   schemas** — not Drizzle column definitions, not builder-internal objects.
   Only TypeBox schemas can be composed via `Type.Composite`, referenced via
   `Type.Ref`, and serialized to JSON Schema.

4. **No ujsx dependency** — storage's Module-based graph types join the
   pipeline conceptually, not as a runtime dependency. The `Type.Module`
   output is the same shape that a ujsx HostConfig would produce, but storage
   doesn't need ujsx to create it. The alignment is structural, not dependent.

5. **Schemas-as-JSON enables `Value.Diff`/`Value.Patch`/`Value.Cast`** —
   because TypeBox Modules serialize to JSON Schema, the TypeBox value system
   can operate on schemas themselves (diff to detect changes, patch to update
   stored schemas, cast to migrate data). This is not possible if schemas are
   opaque builder objects or Drizzle column definitions. See
   [schema-evolution.md](./schema-evolution.md).

6. **OperationSpec output is consumer-agnostic** — storage defines
   `OperationSpec[]` from table definitions. The consumer (hub/spoke) decides
   how to register handlers. Storage does not execute queries or depend on
   the operations runtime.

7. **The folded utils are SQLite-only** — `src/sqlite/utils/` contains only
   SQLite column→TypeBox mappings. If a new database host is added later, the
   utils would need the corresponding dialect mappings. dbtype's Phase 1
   (UJSX→HostConfig) would be the mechanism for multi-dialect support.

## References

- ujsx pointer system: `/workspace/@alkdev/ujsx/src/core/pointer.ts`
- ujsx HostConfig adapter: `/workspace/@alkdev/ujsx/src/host/config.ts`
- dbtype architecture: `/workspace/@alkdev/dbtype/docs/architecture/README.md` (Phase 0 source folded into storage)
- dbtype elements: `/workspace/@alkdev/dbtype/docs/architecture/elements.md`
- dbtype module: `/workspace/@alkdev/dbtype/docs/architecture/module.md`
- dbtype repo adapter: `/workspace/@alkdev/dbtype/docs/architecture/repo-adapter.md`
- Operations registry: `/workspace/@alkdev/operations/docs/architecture/README.md`
- JPATH Module (JSONPath as TypeBox Module): `/workspace/research/typebox_research/ujsx/jpath.gen.ts`
- jsonpathly source: `/workspace/jsonpathly/`
- Module evolution spec: [metagraph-module.md](./metagraph-module.md)
- Schema evolution spec: [schema-evolution.md](./schema-evolution.md)
- ADR-046: Fold drizzlebox as utils (supersedes ADR-033)
- ADR-048: OperationSpecs as repository surface