Files
storage/docs/architecture/open-questions.md
glm-5.1 a2ee452a63 Add repository layer strategy: JSON path queries, CRUD decisions, ecosystem integration
Add three open questions (OQ-17, OQ-18, OQ-19) covering attribute query
strategy, CRUD generation approach, and storage-operations bridge placement.
Create ADR-033 recording the v1 decision: JSON path queries for attributes
with hand-written CRUD for static tables.

Expand forward-look.md with Repository Layer Strategy section analyzing
three approaches (JSON path, native columns via dbtype, hybrid) and their
implications for the metagraph pattern. Add drizzle-graphql and dbtype
from-dbtype comparison showing neither handles dynamic schema-as-data.

Update overview.md with dbtype/ujsx in the dependency diagram, expanded
ecosystem context in the bridging pattern section, and new open questions.

Align open-questions.md: resolve OQ-17 and OQ-18 for v1 (ADR-033), add
OQ-19 as open, update summary counts and ADR impact table.
2026-05-30 11:02:49 +00:00

11 KiB

status, last_updated
status last_updated
reviewed 2026-05-30

Open Questions Tracker

Cross-cutting compilation of all unresolved questions across the storage architecture documents, organized by theme. Questions that appear in multiple documents are unified here with cross-references.

When a question is resolved, update its status to resolved and add a resolution note. Once all questions in a theme are resolved, the theme section can be removed and the resolution noted in the relevant ADR.

Summary

Status Count
Open 8
Partially resolved 1
Resolved 10

Open questions requiring decisions:

  • OQ-03 (actors table design) — deferred to ACL design
  • OQ-04 (repository layer host-specific vs host-agnostic) — start host-specific
  • OQ-07 (encryptRaw performance) — low priority, add if needed
  • OQ-10 (Edit[] classification) — needs POC
  • OQ-11 (auto-migrate vs explicit consumer action) — conditional on OQ-10
  • OQ-12 (schema evolution vs event-sourced replay) — post-v1 concern
  • OQ-13 (schema evolution events in event stream) — post-v1
  • OQ-19 (storage-operations bridge package location) — depends on long-term CRUD strategy

Partially resolved:

  • OQ-01 (flowgraph Module export) — storage can start without it

Resolved (v1 direction decided, long-term question remains open):

  • OQ-17 (attribute query strategy) — JSON path for v1 (ADR-033), hybrid viable with dbtype later
  • OQ-18 (auto-generated vs hand-written CRUD) — hand-write for v1 (ADR-033), auto-gen remains an option

How to Use This Document

  • Each question has an ID (e.g., OQ-01), status, origin (which doc(s)), and priority
  • Cross-references link related questions and ADRs
  • Resolved questions have a resolution note

ADR Impact

ADR Resolves
ADR-003 OQ-01 (partial — storage can start without flowgraph Module)
ADR-015 OQ-05 (constraint semantics)
ADR-018 OQ-17 (v1 decision: dbtype integration deferred, JSON path for v1)
ADR-020 OQ-02 (no nodeTypeId for now, can add later)
ADR-033 OQ-17 (JSON path queries for v1), OQ-18 (hand-written CRUD for v1)

Theme 1: Package Boundaries and Dependencies

OQ-01: Should @alkdev/flowgraph export a Type.Module, or should storage define its own entries with documented correspondence?

  • Origin: metagraph-module.md
  • Status: partially resolved
  • Priority: high
  • Notes: Storage can start with standalone schemas and Type.Composite([BaseNode, CallNodeAttrs]) — no dependency on flowgraph. Adopt Import() when flowgraph provides a Module. This avoids a circular dependency: @alkdev/storage does NOT depend on @alkdev/flowgraph.
  • Cross-references: ADR-003, ADR-010

OQ-02: Should concrete graph type Modules live in storage or in their respective packages?

  • Origin: metagraph-module.md
  • Status: resolved
  • Priority: medium
  • Resolution: Both. Storage provides reference Modules in modules/ that consumers can use directly or replace. Flowgraph may also export a Module — the two are compatible via Module $defs.
  • Cross-references: ADR-003

Theme 2: Data Model

OQ-03: Should actors be a node type or a standalone table?

  • Origin: overview.md
  • Status: open
  • Priority: medium
  • Notes: Currently actors is a standalone table with no relations. If identity/authentication is a graph (ACL nodes based on @alkdev/operations' Identity interface), actors become node types. If identity needs special query patterns (auth lookups, session joins), standalone tables may be better. Decision deferred until ACL design.
  • Cross-references: ADR-024, encrypted-data.md

OQ-04: Should the repository layer be host-specific or host-agnostic?

  • Origin: overview.md
  • Status: open
  • Priority: medium
  • Notes: A host-agnostic repository requires an abstraction over Drizzle's query builder. A host-specific repository is simpler but means duplicating query logic for PG. Decision: start host-specific in SQLite, extract common patterns later.
  • Cross-references: sqlite-host.md

OQ-05: Should *EdgeConstraints entries use Type.Ref or Type.String for allowed source/target types?

  • Origin: metagraph-module.md
  • Status: resolved
  • Priority: low
  • Resolution: Type.String() — the constraint arrays contain node type names, not node type schemas.
  • Cross-references: ADR-015

OQ-06: How does the graph pointer abstraction interact with the repository layer?

  • Origin: metagraph-module.md
  • Status: resolved
  • Priority: low
  • Resolution: For v1, repository functions use direct key-based addressing. Validate on read — if data doesn't match the Module entry, throw. Typed pointers are post-v1 (ADR-017).
  • Cross-references: ADR-017, forward-look.md

Theme 3: Encryption and Security

OQ-07: Should we add encryptRaw() for performance?

  • Origin: encrypted-data.md
  • Status: open
  • Priority: low
  • Notes: PBKDF2 derivation adds ~100ms per operation. For batch operations (e.g., rotating 1000 keys), this adds up. An encryptRaw() that skips PBKDF2 would be much faster. Decision: add in a future iteration if performance demands it.

OQ-08: Should the key attribute on secret nodes be encrypted?

  • Origin: encrypted-data.md
  • Status: resolved
  • Priority: low
  • Resolution: Plaintext key names are acceptable for now. If secret names are sensitive, add a keyHash attribute for blind lookups.

OQ-09: Should secret nodes have lastUsedAt and expiresAt as first-class columns?

  • Origin: encrypted-data.md
  • Status: resolved
  • Priority: low
  • Resolution: For spoke use (occasional lookups), JSON attributes are fine. For hub use (high-throughput key validation), a standalone api_keys table with proper indexes is still needed.

Theme 4: Schema Evolution

OQ-10: Can Value.Diff Edit[] be reliably classified as breaking vs non-breaking?

  • Origin: schema-evolution.md
  • Status: open
  • Priority: high
  • Notes: The classification table in schema-evolution.md is theoretical. A POC should validate whether Edit[] output contains enough information to distinguish String → Literal("x") (narrowing, non-breaking) from String → Number (incompatible, breaking). Alternative: skip classification and just use Value.Check(newSchema, storedData) for verification.

OQ-11: Should the repository layer auto-migrate data on schema change, or require explicit consumer action?

  • Origin: schema-evolution.md
  • Status: open
  • Priority: high
  • Notes: Conditional on OQ-10 POC outcome. If classification is feasible, the repository layer auto-applies Value.Cast for non-breaking changes and requires explicit consumer action for breaking changes. If classification is not feasible, the repository layer auto-applies Value.Cast only when Value.Check(newSchema, storedData) passes for all stored data.

OQ-12: How does schema evolution interact with the hub's event-sourced call graph?

  • Origin: schema-evolution.md
  • Status: open
  • Priority: medium
  • Notes: If the hub migrates to event-sourced replay (projector evolution), storage's call graph tables become disposable projections. But other graph types (ACL, tasks, secrets) may not have an event stream to replay from. The schema evolution design should work for both projections and direct-persisted data.

OQ-13: Should schema evolution events be part of the event stream?

  • Origin: schema-evolution.md
  • Status: open
  • Priority: low
  • Notes: Post-v1. For v1, schema changes are applied directly via the repository layer with version tracking.

Theme 5: Encrypted Data Scope

OQ-14: Should encryption be per-attribute, per-node, or per-graph?

  • Origin: overview.md
  • Status: resolved
  • Priority: high
  • Resolution: Per-attribute. The EncryptedData schema is a single attribute within a node type, not the entire node. This preserves queryability on non-sensitive fields (ADR-023).

OQ-15: Should key management be in this package?

  • Origin: overview.md
  • Status: resolved
  • Priority: high
  • Resolution: No. @alkdev/storage provides encryption/decryption primitives but NOT key management. The consuming application provides the key ring (ADR-026).

Theme 6: Repository Layer

OQ-16: Should the repository layer live in @alkdev/storage or in a consumer package?

  • Origin: overview.md
  • Status: resolved
  • Priority: high
  • Resolution: The repository CRUD layer (host-specific typed queries, schema validation before writes) belongs in @alkdev/storage. The operations bridging layer (generating OperationSpecs from metagraph schemas) belongs in a consumer or adapter package. These are separate concerns — CRUD is a storage concern; call protocol integration is an application concern.

Theme 7: Repository Layer Strategy

OQ-17: How should the repository layer handle attribute queries — JSON path, native columns, or dbtype-generated?

  • Origin: forward-look.md
  • Status: resolved (v1)
  • Priority: high
  • Resolution: For v1, attribute queries use JSON path extraction (json_extract on SQLite, ->>/#>> on PG). Hand-written CRUD for static tables. dbtype integration and hybrid approach are post-v1. See ADR-033. The long-term question of whether to adopt the hybrid approach (static tables via dbtype, dynamic attributes remain JSON) remains open for future iterations.
  • Cross-references: ADR-033, ADR-018, forward-look.md

OQ-18: Should the repository layer's CRUD operations be auto-generated (drizzle-graphql pattern) or hand-written?

  • Origin: forward-look.md
  • Status: resolved (v1)
  • Priority: medium
  • Resolution: For v1, hand-write CRUD functions with explicit signatures. The three long-term options (hand-written, auto-generated from Drizzle, auto-generated from dbtype) remain open for future iterations. See ADR-033.
  • Cross-references: ADR-033, OQ-17

OQ-19: Where does the storage-operations bridge package live in the @alkdev workspace?

  • Origin: forward-look.md
  • Status: open
  • Priority: medium
  • Notes: Four options: (1) hub-internal code, (2) dedicated @alkdev/storage-operations adapter, (3) from-storage adapter inside @alkdev/operations, (4) part of @alkdev/dbtype's from-dbtype adapter. Option 1 is the most immediate (no new package). Option 2 is the cleanest separation. Option 3 creates an undesirable dependency direction (operations → storage). Option 4 is the long-term goal if dbtype is adopted. The choice depends on OQ-17/OQ-18 resolution: if hand-written CRUD, the bridge is trivial and can live in the hub; if auto-generated from dbtype, the bridge naturally lives with dbtype.
  • Cross-references: OQ-16, OQ-17, ADR-033