alkdev/storage
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix: use import type for GraphConfig, remove verbatim-module-syntax exclusion

Browse Source 
The verbatim-module-syntax lint rule was correctly flagging that
GraphConfig is only used in a type position (typeof GraphConfig). Since
typeof resolves purely at the type level, import type works fine here
and is the correct form. No lint exclusion needed.

Also: deno fmt across all files (markdown line wrapping).
This commit is contained in:
glm-5.1
2026-05-28 13:38:42 +00:00
parent b0298663dc
commit bb544469fd
34 changed files with 1279 additions and 617 deletions
Download Patch File Download Diff File Expand all files Collapse all files

164
docs/architecture/overview.md
View File

@@ -9,11 +9,19 @@ Typed graph storage with dual database hosts. Deno-first, published via JSR.
## Purpose
`@alkdev/storage` provides a **metagraph** storage model: graph types define schemas, node types define data shapes within those graphs, and edge types define typed relationships. Instances of these type definitions become actual graphs populated with nodes and edges.
`@alkdev/storage` provides a **metagraph** storage model: graph types define
schemas, node types define data shapes within those graphs, and edge types
define typed relationships. Instances of these type definitions become actual
graphs populated with nodes and edges.
This pattern replaces domain-specific table proliferation with a small number of general-purpose tables that can model anything — call graphs, ACL rules, task dependencies, encrypted secrets — while enforcing schema integrity through TypeBox validation.
This pattern replaces domain-specific table proliferation with a small number of
general-purpose tables that can model anything — call graphs, ACL rules, task
dependencies, encrypted secrets — while enforcing schema integrity through
TypeBox validation.
The package evolved from `@ade/ade-v0/packages/core/graphs` and `@ade/ade-v0/packages/storage_sqlite`, simplified and refactored for the @alkdev ecosystem.
The package evolved from `@ade/ade-v0/packages/core/graphs` and
`@ade/ade-v0/packages/storage_sqlite`, simplified and refactored for the @alkdev
ecosystem.
## Architecture
@@ -33,114 +41,160 @@ The package evolved from `@ade/ade-v0/packages/core/graphs` and `@ade/ade-v0/pac
### Subpath Exports (JSR/npm)
| Export | Contents | Dependencies |
|--------|----------|-------------|
| `@alkdev/storage` | Graph schema types, SchemaBuilder | `@alkdev/typebox`, `@alkdev/drizzlebox` |
| `@alkdev/storage/graphs` | Same as `.` — alias for the main export | Same as `.` |
| `@alkdev/storage/sqlite` | SQLite tables, relations, client | + `drizzle-orm`, `@libsql/client` |
| `@alkdev/storage/pg` | PostgreSQL tables, relations, client | ⚠️ NOT YET IMPLEMENTED |
| Export | Contents | Dependencies |
| ------------------------ | --------------------------------------- | --------------------------------------- |
| `@alkdev/storage` | Graph schema types, SchemaBuilder | `@alkdev/typebox`, `@alkdev/drizzlebox` |
| `@alkdev/storage/graphs` | Same as `.` — alias for the main export | Same as `.` |
| `@alkdev/storage/sqlite` | SQLite tables, relations, client | + `drizzle-orm`, `@libsql/client` |
| `@alkdev/storage/pg` | PostgreSQL tables, relations, client | ⚠️ NOT YET IMPLEMENTED |
The `./graphs` subpath exists because the source code lives in `src/graphs/` and the main `mod.ts` re-exports it. Importing from either `@alkdev/storage` or `@alkdev/storage/graphs` yields the same types and SchemaBuilder.
The `./graphs` subpath exists because the source code lives in `src/graphs/` and
the main `mod.ts` re-exports it. Importing from either `@alkdev/storage` or
`@alkdev/storage/graphs` yields the same types and SchemaBuilder.
## Terminology
| Term | Definition |
|------|-----------|
| **Metagraph** | A type system where graph types define schemas, node types define data shapes within those graphs, and edge types define typed relationships. Graph instances are concrete data conforming to these type definitions. |
| **Hub** | The central service in the hub-spoke architecture. Runs PostgreSQL, hosts API endpoints, coordinates spokes, and is the authoritative data store. `@alkdev/storage`'s PostgreSQL host (not yet implemented) targets the hub. |
| **Spoke** | A local/embedded instance that runs per-project or per-session. Uses SQLite for local storage. `@alkdev/storage`'s SQLite host targets spokes. |
| **Graph type** | A class of graphs (e.g., "call-graph", "acl"). Defines structural constraints (directed/undirected/mixed, multi-edges, self-loops) and the valid node/edge type vocabularies. Stored in the `graph_types` table. |
| **Node type** | A category of node within a graph type. Defines the attribute schema for nodes of that type. Stored in the `node_types` table. |
| **Edge type** | A category of edge within a graph type. Defines the attribute schema and optionally restricts which node types can be source/target. Stored in the `edge_types` table. |
| **Graph instance** | A concrete graph belonging to a graph type. Contains nodes and edges conforming to its type definitions. Stored in the `graphs` table. |
| **Consumer** | Code that imports `@alkdev/storage` (or a subpath) to define graph types and persist graph data. The hub and spokes are consumers. |
| **Repository layer** | ⚠️ Not yet implemented. The typed CRUD functions (insert, find, update, delete) that sit between consumer code and raw Drizzle queries. Performs schema validation before writes. |
| **Validation boundary** | The line where schema validation is enforced. In this package, validation happens in the SchemaBuilder (at type definition time) and the repository layer (at mutation time), NOT in the database. |
| Term | Definition |
| ----------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Metagraph** | A type system where graph types define schemas, node types define data shapes within those graphs, and edge types define typed relationships. Graph instances are concrete data conforming to these type definitions. |
| **Hub** | The central service in the hub-spoke architecture. Runs PostgreSQL, hosts API endpoints, coordinates spokes, and is the authoritative data store. `@alkdev/storage`'s PostgreSQL host (not yet implemented) targets the hub. |
| **Spoke** | A local/embedded instance that runs per-project or per-session. Uses SQLite for local storage. `@alkdev/storage`'s SQLite host targets spokes. |
| **Graph type** | A class of graphs (e.g., "call-graph", "acl"). Defines structural constraints (directed/undirected/mixed, multi-edges, self-loops) and the valid node/edge type vocabularies. Stored in the `graph_types` table. |
| **Node type** | A category of node within a graph type. Defines the attribute schema for nodes of that type. Stored in the `node_types` table. |
| **Edge type** | A category of edge within a graph type. Defines the attribute schema and optionally restricts which node types can be source/target. Stored in the `edge_types` table. |
| **Graph instance** | A concrete graph belonging to a graph type. Contains nodes and edges conforming to its type definitions. Stored in the `graphs` table. |
| **Consumer** | Code that imports `@alkdev/storage` (or a subpath) to define graph types and persist graph data. The hub and spokes are consumers. |
| **Repository layer** | ⚠️ Not yet implemented. The typed CRUD functions (insert, find, update, delete) that sit between consumer code and raw Drizzle queries. Performs schema validation before writes. |
| **Validation boundary** | The line where schema validation is enforced. In this package, validation happens in the SchemaBuilder (at type definition time) and the repository layer (at mutation time), NOT in the database. |
## Design Decisions
### D1: Deno-first, JSR publishes, npm comes free
The package is published to JSR (`deno publish`). npm compatibility is automatic via JSR's npm layer (`@jsr/alkdev__storage`). No separate dnt build step.
The package is published to JSR (`deno publish`). npm compatibility is automatic
via JSR's npm layer (`@jsr/alkdev__storage`). No separate dnt build step.
### D2: Metagraph over domain-specific tables
Instead of a table per domain concept (call graphs, ACL rules, task trees), we define graph types with typed node and edge schemas. A "call graph" is a graph type with specific node types (operation call, subcall) and edge types (triggered, depends_on). An "ACL graph" is a graph type with node types (account, resource) and edge types (can_read, can_write).
Instead of a table per domain concept (call graphs, ACL rules, task trees), we
define graph types with typed node and edge schemas. A "call graph" is a graph
type with specific node types (operation call, subcall) and edge types
(triggered, depends_on). An "ACL graph" is a graph type with node types
(account, resource) and edge types (can_read, can_write).
This trades some query convenience for generality. Domain-specific queries are built on top of the graph query layer, not baked into table schemas.
This trades some query convenience for generality. Domain-specific queries are
built on top of the graph query layer, not baked into table schemas.
### D3: SchemaBuilder as the primary API surface
The `SchemaBuilder` fluent API is the intended way to construct graph type definitions. It validates against TypeBox schemas at build time, ensuring that graph/node/edge type definitions are structurally sound before they're persisted to the database.
The `SchemaBuilder` fluent API is the intended way to construct graph type
definitions. It validates against TypeBox schemas at build time, ensuring that
graph/node/edge type definitions are structurally sound before they're persisted
to the database.
### D4: Injectable clients, no module-level side effects
`createSqliteDatabase(client)` receives a pre-created client. Module-level side effects (auto-connections, env-based configuration) are forbidden. This enables testing with in-memory databases and containerized deployment patterns.
`createSqliteDatabase(client)` receives a pre-created client. Module-level side
effects (auto-connections, env-based configuration) are forbidden. This enables
testing with in-memory databases and containerized deployment patterns.
### D5: Drizzle + TypeBox (via drizzlebox) as the table definition pattern
Drizzle table definitions are the single source of truth for database schema. `@alkdev/drizzlebox` generates TypeBox `Select*` and `Insert*` schemas from Drizzle tables, enabling runtime validation without manual schema duplication.
Drizzle table definitions are the single source of truth for database schema.
`@alkdev/drizzlebox` generates TypeBox `Select*` and `Insert*` schemas from
Drizzle tables, enabling runtime validation without manual schema duplication.
### D6: Enumeration pattern — `as const` objects, not TypeScript enums
All enumerations use the `as const` object pattern (e.g., `GRAPH_STATUS = { Active: "active", ... } as const`) rather than TypeScript `enum`. This avoids JSR slow-type issues (the existing lint exclusion for `no-slow-types` was needed partly because of TS enums) and provides a consistent pattern across the codebase. The TypeBox schemas use `Type.Union` of `Type.Literal` values derived from the const object.
All enumerations use the `as const` object pattern (e.g.,
`GRAPH_STATUS = { Active: "active", ... } as const`) rather than TypeScript
`enum`. This avoids JSR slow-type issues and provides a consistent pattern
across the codebase. The TypeBox schemas use `Type.Union` of `Type.Literal`
values derived from the const object.
### D7: No comments in code
Per project convention across @alkdev packages, source files contain no inline comments. Documentation lives in architecture docs and TypeBox schema descriptions.
Per project convention across @alkdev packages, source files contain no inline
comments. Documentation lives in architecture docs and TypeBox schema
descriptions.
### D8: Common columns pattern
All tables share `id` (text PK), `metadata` (JSON text defaulting to `{}`), `createdAt`, and `updatedAt` (integer timestamps in SQLite, will be timestamptz in PG). This ensures every row has auditability and extensibility.
All tables share `id` (text PK), `metadata` (JSON text defaulting to `{}`),
`createdAt`, and `updatedAt` (integer timestamps in SQLite, will be timestamptz
in PG). This ensures every row has auditability and extensibility.
## Dependencies
| Package | Purpose | Layer |
|---------|---------|-------|
| `@alkdev/typebox` | Runtime schema validation | graphs/ |
| `@alkdev/drizzlebox` | Generate TypeBox from Drizzle tables | sqlite/ |
| `drizzle-orm` | ORM, table definitions, queries | sqlite/ (and future pg/) |
| `@libsql/client` | SQLite client (libsql/turso) | sqlite/ |
| `postgres` | PostgreSQL client | pg/ (not yet used) |
| Package | Purpose | Layer |
| -------------------- | ------------------------------------ | ------------------------ |
| `@alkdev/typebox` | Runtime schema validation | graphs/ |
| `@alkdev/drizzlebox` | Generate TypeBox from Drizzle tables | sqlite/ |
| `drizzle-orm` | ORM, table definitions, queries | sqlite/ (and future pg/) |
| `@libsql/client` | SQLite client (libsql/turso) | sqlite/ |
| `postgres` | PostgreSQL client | pg/ (not yet used) |
`@alkdev/typebox` and `@alkdev/drizzlebox` are npm packages (not yet on JSR). JSR handles npm dependencies natively.
`@alkdev/typebox` and `@alkdev/drizzlebox` are npm packages (not yet on JSR).
JSR handles npm dependencies natively.
## What Exists vs. What's Needed
### Implemented
- Graph schema types and SchemaBuilder
- SQLite host: 6 metagraph tables + actors table + Drizzle relations + client factory
- SQLite host: 6 metagraph tables + actors table + Drizzle relations + client
factory
- TypeBox select/insert schemas generated from Drizzle tables (drizzlebox)
### Not Yet Implemented
| Gap | Priority | Notes |
|-----|----------|-------|
| Gap | Priority | Notes |
| ----------------------------------------- | ------------ | --------------------------------------------------------------------------------------------------- |
| Encrypted data node type + crypto utility | **Critical** | ⚠️ Not yet implemented. API keys and secrets at rest. See [encrypted-data.md](./encrypted-data.md). |
| Repository/CRUD layer | High | ⚠️ Not yet implemented. Typed insert, find, update, delete functions for graphs, nodes, edges |
| Tests | High | Zero tests exist. Needed before any real use. |
| PostgreSQL host | Medium | Same table shapes, `pgTable` + `jsonb` + `timestamp` + `pgEnum`. Stub only. |
| ACL graph type | Medium | Access control as a graph. Depends on encrypted data and CRUD layer. |
| Call graph type | Low | Hub-specific, uses metagraph. Deferred until hub consumes this package. |
| Session/message models | Low | Hub-specific, may remain domain tables. |
| Repository/CRUD layer | High | ⚠️ Not yet implemented. Typed insert, find, update, delete functions for graphs, nodes, edges |
| Tests | High | Zero tests exist. Needed before any real use. |
| PostgreSQL host | Medium | Same table shapes, `pgTable` + `jsonb` + `timestamp` + `pgEnum`. Stub only. |
| ACL graph type | Medium | Access control as a graph. Depends on encrypted data and CRUD layer. |
| Call graph type | Low | Hub-specific, uses metagraph. Deferred until hub consumes this package. |
| Session/message models | Low | Hub-specific, may remain domain tables. |
## Open Questions
1. **Should `actors` be a node type or a standalone table?** Currently `actors` is a standalone table in the SQLite host that isn't referenced by any relation. If identity/authentication is a graph (ACL nodes), actors become node types. If identity is a domain concept that needs special query patterns (auth lookups, session joins), standalone tables may be better. Decision: defer until ACL design.
1. **Should `actors` be a node type or a standalone table?** Currently `actors`
is a standalone table in the SQLite host that isn't referenced by any
relation. If identity/authentication is a graph (ACL nodes), actors become
node types. If identity is a domain concept that needs special query patterns
(auth lookups, session joins), standalone tables may be better. Decision:
defer until ACL design.
2. **Should the repository layer be host-specific or host-agnostic?** A host-agnostic repository (insert graph, find nodes by type) requires an abstraction over Drizzle's query builder. A host-specific repository is simpler but means duplicating query logic for PG. Decision: start host-specific in SQLite, extract common patterns later.
2. **Should the repository layer be host-specific or host-agnostic?** A
host-agnostic repository (insert graph, find nodes by type) requires an
abstraction over Drizzle's query builder. A host-specific repository is
simpler but means duplicating query logic for PG. Decision: start
host-specific in SQLite, extract common patterns later.
3. **Encrypted data scope**: Should encryption be per-attribute, per-node, or per-graph? Per-attribute (like hub's `client_secrets.value`) allows selective encryption. Per-node encrypts the entire `attributes` blob. Per-graph is overkill. Decision: per-attribute, modeled as an encrypted node type with a dedicated attribute for the ciphertext.
3. **Encrypted data scope**: Should encryption be per-attribute, per-node, or
per-graph? Per-attribute (like hub's `client_secrets.value`) allows selective
encryption. Per-node encrypts the entire `attributes` blob. Per-graph is
overkill. Decision: per-attribute, modeled as an encrypted node type with a
dedicated attribute for the ciphertext.
4. **Key management scope**: `@alkdev/storage` should provide the encryption/decryption primitives but NOT key management. The consuming application provides the key ring. This keeps the storage package agnostic to deployment-specific secret management.
4. **Key management scope**: `@alkdev/storage` should provide the
encryption/decryption primitives but NOT key management. The consuming
application provides the key ring. This keeps the storage package agnostic to
deployment-specific secret management.
5. **Migration strategy**: When graph type schemas evolve (new node types, changed attribute schemas), who handles migration? The repository layer should support schema version checking, but actual migration scripts are application-level. See [metagraph.md](./metagraph.md) for the versioning approach.
5. **Migration strategy**: When graph type schemas evolve (new node types,
changed attribute schemas), who handles migration? The repository layer
should support schema version checking, but actual migration scripts are
application-level. See [metagraph.md](./metagraph.md) for the versioning
approach.
## References
- Hub storage spec: `/workspace/@alkdev/hub/docs/architecture/storage/`
- Source heritage: `@ade/ade-v0/packages/core/graphs` and `@ade/ade-v0/packages/storage_sqlite`
- Source heritage: `@ade/ade-v0/packages/core/graphs` and
`@ade/ade-v0/packages/storage_sqlite`
- Drizzle ORM: https://orm.drizzle.team/
- TypeBox: https://github.com/sinclairzx/typebox
- JSR: https://jsr.io/
- JSR: https://jsr.io/