diff --git a/README.md b/README.md index dc36ff6..391fe01 100644 --- a/README.md +++ b/README.md @@ -1,73 +1,70 @@ # @alkdev/dbtype -Schema-first multi-dialect TypeBox/Drizzle bridge — define once, validate and deploy anywhere. +Schema-first multi-dialect database type system. Define your schema once as a UJSX element tree, validate it with TypeBox, and render it to any Drizzle dialect. -Based on [drizzle-typebox](https://github.com/drizzle-team/drizzle-orm/tree/main/drizzle-typebox) by the Drizzle Team, adapted for use with `@alkdev/typebox` (a maintained fork of `@sinclair/typebox`). +Based on [drizzle-typebox](https://github.com/drizzle-team/drizzle-orm/tree/main/drizzle-typebox) by the Drizzle Team, adapted for use with `@alkdev/typebox` and `@alkdev/ujsx`. ## Install ```bash -npm install @alkdev/dbtype -npm install @alkdev/typebox -npm install drizzle-orm +npm install @alkdev/dbtype @alkdev/typebox @alkdev/ujsx +npm install drizzle-orm # peer dependency, only the dialects you use ``` -## Features +## Architecture -- Create select schemas for tables, views, and enums -- Create insert and update schemas for tables -- Supports all dialects: PostgreSQL, MySQL, and SQLite -- Custom TypeBox instance support via `createSchemaFactory` +See [docs/architecture/README.md](docs/architecture/README.md) for the full architecture specification. -## Usage +### Core Principle -```ts -import { pgEnum, pgTable, serial, text, timestamp } from 'drizzle-orm/pg-core'; -import { createInsertSchema, createSelectSchema, createUpdateSchema } from '@alkdev/dbtype'; -import { Type } from '@alkdev/typebox'; -import { Value } from '@alkdev/typebox/value'; +**The element tree is the schema. The module is the bundle. The host is the dialect.** -const users = pgTable('users', { - id: serial('id').primaryKey(), - name: text('name').notNull(), - email: text('email').notNull(), - role: text('role', { enum: ['admin', 'user'] }).notNull(), - createdAt: timestamp('created_at').notNull().defaultNow(), -}); +- UJSX elements (``, ``) define schemas with composable function components +- `Type.Module` holds all tables, relations, and derived schemas with automatic `Type.Ref` resolution +- `HostConfig` renders the same tree to `sqliteTable`, `pgTable`, or `mysqlTable` -// Schema for inserting a user -const insertUserSchema = createInsertSchema(users); +### Quick Example -// Schema for updating a user -const updateUserSchema = createUpdateSchema(users); +```tsx +import { Type, FormatRegistry } from '@alkdev/typebox' +import { Value } from '@alkdev/typebox/value' +import { h, createComponent } from '@alkdev/ujsx' -// Schema for selecting a user -const selectUserSchema = createSelectSchema(users); +// Register custom format validators +FormatRegistry.Set('uuid', (v) => /^[0-9a-f]{8}-[0-9a-f]{4}-...$/i.test(v)) -// Overriding fields -const insertUserSchema = createInsertSchema(users, { - role: Type.String(), -}); +// Composable column components +const IdColumn = createComponent('IdColumn', () => + h('column', { name: 'id', type: 'uuid', primaryKey: true, default: 'uuid' }) +) -// Refining fields -const insertUserSchema = createInsertSchema(users, { - id: (schema) => Type.Number({ ...schema, minimum: 0 }), - role: Type.String(), -}); +const AuditColumns = createComponent('AuditColumns', () => [ + h('column', { name: 'createdAt', type: 'timestamp', notNull: true, default: 'now' }), + h('column', { name: 'updatedAt', type: 'timestamp', notNull: true, default: 'now' }), +]) -// Validation -const isUserValid: boolean = Value.Check(insertUserSchema, { - name: 'John Doe', - email: 'johndoe@test.com', - role: 'admin', -}); +// Define a table +const UsersEl = h('table', { name: 'users' }, + h(IdColumn, {}), + h('column', { name: 'name', type: 'string', notNull: true }), + h('column', { name: 'email', type: 'string', notNull: true }), + h(AuditColumns, {}), +) + +// Extract to Type.Module for validation +const { name, schema } = extractTable(UsersEl) +const M = Type.Module({ Users: schema, UsersRelations: Type.Object({ tasks: Type.Array(Type.Ref('Tasks')) }) }) +const Users = M.Import('Users') + +Value.Check(Users, { id: '...', name: 'alice', email: 'a@b.com', createdAt: 1, updatedAt: 1 }) +// → true ``` -## Differences from drizzle-typebox +## Current State -- Uses `@alkdev/typebox` instead of `@sinclair/typebox` -- Standalone package (no monorepo dependency) -- Published as `@alkdev/dbtype` on npm +**Phase 0: Exploration** — Architecture probing complete, implementation not started. + +The current `src/` contains the forked drizzle-typebox code. The new architecture (UJSX elements, Type.Module, HostConfig) is designed and validated but not yet implemented. ## Attribution diff --git a/docs/architecture/README.md b/docs/architecture/README.md new file mode 100644 index 0000000..4655fc7 --- /dev/null +++ b/docs/architecture/README.md @@ -0,0 +1,147 @@ +--- +status: draft +last_updated: 2026-05-22 +--- + +# @alkdev/dbtype Architecture + +Schema-first multi-dialect database type system. Define your database schema once as a UJSX element tree, validate it with TypeBox, and render it to any Drizzle dialect (SQLite, PostgreSQL, MySQL) — all from a single source of truth. + +## Why This Exists + +Every project that uses Drizzle ORM across multiple backends defines schemas repeatedly: once per dialect (SQLite for local dev, PostgreSQL for production), once again for input validation (TypeBox, Zod, etc.), and once more for API contracts (GraphQL types, OpenAPI schemas). The existing `drizzle-typebox` only goes one direction (Drizzle → TypeBox) and requires defining schemas in Drizzle's dialect-specific API first. + +The gap: there is no schema-first, dialect-agnostic way to define a database schema that simultaneously produces validation schemas, Drizzle table definitions, and (via the operations layer) CRUD operation specs — all from one source of truth. + +## Core Principle + +**The element tree is the schema. The module is the bundle. The host is the dialect.** + +- The UJSX element tree (`
`, ``) is the authoring surface — composable, reusable, and TypeBox-validatable +- The `Type.Module` is the schema bundle — all tables, relations, and derived schemas in one namespace with `Type.Ref` resolving cross-table references +- The `HostConfig` is the dialect adapter — render the same tree to `sqliteTable`, `pgTable`, `mysqlTable`, or any future target + +## Relationship to Sibling Packages + +| Package | Relationship | +|---------|-------------| +| `@alkdev/typebox` | Type system foundation. `Type.Module`, `Type.Ref`, `Value.Check/Diff/Edit`, `FormatRegistry` | +| `@alkdev/ujsx` | Element authoring. `h()`, `createComponent`, `HostConfig`, `createRoot`, reconciler | +| `@alkdev/operations` | CRUD generation. `OperationSpec`, `OperationRegistry`, `from-dbtype` adapter (future) | +| `@alkdev/pubsub` | Event transport. Used by operations call protocol | +| `drizzle-orm` | Peer dependency. Dialect-specific column builders consumed by hosts | + +## Current State + +**Phase 0: Exploration** — Architecture probing complete. Probe scripts in `scripts/probe-e2e.ts` validate the core architecture. No implementation yet. + +### What Exists +- Fork of `drizzle-typebox` with `@alkdev/typebox` support (current `src/`) +- Research docs in `docs/research/` (architecture, type-map, column diffs, typedef kind pattern) +- E2E probe script validating: UJSX → Type.Module → Drizzle rendering pipeline + +### What Doesn't Exist Yet +- Core `dbtype` package (the implementation) +- Host configs for SQLite, PostgreSQL, MySQL +- Schema extraction functions (`createSelectSchema`, `createInsertSchema`, `createUpdateSchema` from element trees) +- Repo/CRUD generation adapter (`from-dbtype` for `@alkdev/operations`) +- `@alkdev/ujsx` as a declared dependency (currently dev-only) + +## Architecture Documents + +| Document | Content | +|----------|---------| +| [schema.md](schema.md) | Type.Module structure, element tree types, column type vocabulary, schema derivation | +| [hosts.md](hosts.md) | HostConfig implementations per dialect, symbolic defaults, Drizzle rendering | +| [elements.md](elements.md) | UJSX element definitions, function components, props schemas, common components | +| [module.md](module.md) | Type.Module as the bundle, incremental construction, serialization, migration diffing | +| [repo-adapter.md](repo-adapter.md) | `from-dbtype` adapter for `@alkdev/operations`, filter/schema generation, access control | +| [build-distribution.md](build-distribution.md) | Package structure, sub-path exports, dependencies, tree-shaking | +| [open-questions.md](open-questions.md) | Cross-cutting unresolved questions | + +### Design Decisions + +| ADR | Decision | +|-----|----------| +| [001](decisions/001-ujsx-as-ir.md) | UJSX element tree as the IR, not a separate DbType builder API | +| [002](decisions/002-type-module-as-bundle.md) | Type.Module as the schema bundle, not a custom registry | +| [003](decisions/003-hostconfig-for-dialects.md) | HostConfig for dialect rendering, not a transform registry | +| [004](decisions/004-format-annotation-only.md) | Column formats are annotations, not validators — register explicitly | +| [005](decisions/005-repo-as-adapter.md) | CRUD generation as an operations adapter, not a core feature | + +### Open Questions + +All unresolved questions tracked in [open-questions.md](open-questions.md). + +## Source Structure (Planned) + +``` +src/ + core/ + elements.ts # h() wrappers, createComponent for table/column/index/fk + schema.ts # extractTable, createSelectSchema, createInsertSchema, createUpdateSchema + module.ts # buildModule, module construction helpers + column-types.ts # DbColumnType union, colToTypeBox mapping + defaults.ts # Symbolic default resolution ('now', 'uuid', 'autoincrement') + guards.ts # Type guards for element types + hosts/ + sqlite.ts # HostConfig for drizzle-orm/sqlite-core + pg.ts # HostConfig for drizzle-orm/pg-core + mysql.ts # HostConfig for drizzle-orm/mysql-core + repo/ + from-dbtype.ts # FromDbType adapter for @alkdev/operations + filters.ts # Filter schema generation per column type + handlers.ts # CRUD handler generation (findMany, insertOne, etc.) + index.ts +``` + +## Key Design Decisions + +### 1. UJSX Element Tree as the IR, Not a Separate Builder API + +The architecture docs in `docs/research/` proposed a `DbTypeBuilder` class with methods like `DbType.String()`, `DbType.Table()`. The probe results show that UJSX elements (`
`, ``) with props serve the same purpose — they carry all the metadata (type, notNull, primaryKey, default, references) and compose via function components (`IdCol`, `AuditCols`). + +A separate builder API would be redundant: both the element tree and the builder produce the same TypeBox + metadata. The element tree is strictly more capable (function components for composition, reconciler for migration diffing in the future) and already exists as `@alkdev/ujsx`. + +**Alternative considered**: `DbTypeBuilder` (the research docs pattern). Rejected because it duplicates what UJSX already provides and cannot compose as naturally. + +### 2. Type.Module as the Schema Bundle + +All table schemas and their relations live in a single `Type.Module` call. `Type.Ref` resolves cross-table references including mutual/circular ones. This eliminates the circular-import problem that the storage_sqlite pattern solves with a separate `relations.ts` file. + +The module is also serializable as JSON Schema with `$defs`, enabling `Value.Diff` for schema migration and `FromSchema` for round-tripping. + +**Alternative considered**: A custom registry with separate schema objects. Rejected because `Type.Module` already does everything needed (ref resolution, validation, serialization) and doesn't require a new abstraction. + +### 3. HostConfig for Dialect Rendering, Not a Transform Registry + +The research docs proposed a `TransformRegistry` with priority-sorted rules. UJSX's `HostConfig` serves the same purpose — each dialect is a host, `createInstance` maps element types to Drizzle column builders, `appendChild` assembles columns into tables. + +This leverages existing infrastructure (`createRoot`, `createComponent`, reconciler) rather than building a parallel dispatch system. It also positions the project for future migration support via `prepareUpdate`/`commitUpdate`. + +**Alternative considered**: TransformRegistry from `docs/research/architecture.md`. Rejected because HostConfig is the same pattern with more capabilities (reconciler, function components, context) and already exists in `@alkdev/ujsx`. + +### 4. Column Formats as Annotations + +`Type.String({ format: 'uuid' })` and `Type.String({ format: 'email' })` are annotations by default in TypeBox. `Value.Check` does not enforce format unless validators are explicitly registered via `FormatRegistry.Set`. This is correct JSON Schema behavior. + +For dbtype, formats serve as metadata that hosts can use (e.g., the PG host maps `format: 'uuid'` to `uuid()` column type, the SQLite host maps it to `text()` with `$defaultFn`). Validation is opt-in via format registration. + +### 5. CRUD Generation as an Operations Adapter + +The repo pattern (auto-generated CRUD for each table) is not a core feature of dbtype. It's an adapter for `@alkdev/operations` that consumes the same element tree and module bundle. This keeps dbtype focused on schema definition and rendering, while the operations integration is a separate concern. + +## Document Lifecycle + +| From | To | Condition | +|------|----|-----------| +| `draft` | `reviewed` | All open questions resolved | +| `reviewed` | `stable` | Implementation complete and verified by tests | +| `stable` | `deprecated` | Superseded by new architecture | + +## References + +- Research: `docs/research/architecture.md`, `docs/research/typemap-architecture.md`, `docs/research/dizzle-column-diffs.md`, `docs/research/typedef-kind-pattern.md` +- Probe results: `scripts/probe-e2e.ts` +- Sibling projects: `@alkdev/typebox`, `@alkdev/ujsx`, `@alkdev/operations`, `@alkdev/pubsub` +- Reference implementation: `@alkdev/drizzle-typebox` (current `src/`), `drizzle-graphql` (workspace) \ No newline at end of file diff --git a/docs/architecture/build-distribution.md b/docs/architecture/build-distribution.md new file mode 100644 index 0000000..1786419 --- /dev/null +++ b/docs/architecture/build-distribution.md @@ -0,0 +1,91 @@ +--- +status: draft +last_updated: 2026-05-22 +--- + +# Build & Distribution + +Package structure, exports, dependencies, and tree-shaking strategy. + +## Package + +- **Name**: `@alkdev/dbtype` +- **Type**: ESM with CJS fallback +- **Peer dependencies**: `@alkdev/typebox >=0.34.49`, `@alkdev/ujsx >=0.1.0`, `drizzle-orm >=0.36.0` +- **Dev dependencies**: All three peer deps for testing, plus `tsx`, `vitest`, `typescript` + +## Sub-path Exports + +```json +{ + "exports": { + ".": { + "import": { "types": "./index.d.mts", "default": "./index.mjs" }, + "require": { "types": "./index.d.cjs", "default": "./index.cjs" } + }, + "/core": { + "import": { "types": "./core/index.d.mts", "default": "./core/index.mjs" }, + "require": { "types": "./core/index.d.cjs", "default": "./core/index.cjs" } + }, + "/sqlite": { + "import": { "types": "./sqlite/index.d.mts", "default": "./sqlite/index.mjs" }, + "require": { "types": "./sqlite/index.d.cjs", "default": "./sqlite/index.cjs" } + }, + "/pg": { + "import": { "types": "./pg/index.d.mts", "default": "./pg/index.mjs" }, + "require": { "types": "./pg/index.d.cjs", "default": "./pg/index.cjs" } + }, + "/mysql": { + "import": { "types": "./mysql/index.d.mts", "default": "./mysql/index.mjs" }, + "require": { "types": "./mysql/index.d.cjs", "default": "./mysql/index.cjs" } + } + } +} +``` + +Tree-shaking: Users who only use SQLite import `/sqlite` and never pull in PG or MySQL column builders. + +## Source Structure + +``` +src/ + core/ + elements.ts # h() wrappers, createComponent for table/column/index/fk + schema.ts # extractTable, createSelectSchema, createInsertSchema, createUpdateSchema + module.ts # buildModule, module construction helpers + column-types.ts # DbColumnType union, colToTypeBox mapping + defaults.ts # Symbolic default resolution ('now', 'uuid', 'autoincrement') + guards.ts # Type guards for element types + index.ts + hosts/ + sqlite.ts # HostConfig for drizzle-orm/sqlite-core + pg.ts # HostConfig for drizzle-orm/pg-core + mysql.ts # HostConfig for drizzle-orm/mysql-core + repo/ + from-dbtype.ts # FromDbType adapter for @alkdev/operations (phase 2) + filters.ts # Filter schema generation per column type + handlers.ts # CRUD handler generation + index.ts # Re-exports core +``` + +## Dependencies + +``` +@alkdev/dbtype +├── @alkdev/typebox (peer) # Type.Module, Type.Ref, Value.Check, FormatRegistry +├── @alkdev/ujsx (peer) # h, createComponent, HostConfig, createRoot +└── drizzle-orm (peer) # Dialect-specific column builders (per sub-path) +``` + +The core package (`@alkdev/dbtype/core`) depends only on `@alkdev/typebox` and `@alkdev/ujsx`. The dialect modules (`/sqlite`, `/pg`, `/mysql`) add `drizzle-orm` as a peer dependency (scoped to the specific dialect sub-module). + +## Constraints + +- **Core must not import from drizzle-orm** — the core (elements, schema extraction, module) is dialect-agnostic +- **Dialect modules must not import from each other** — SQLite host doesn't pull in PG types +- **Build is ESM-first** — CJS is generated as a fallback for compatibility + +## References + +- Typemap sub-path exports pattern: `docs/research/typemap-architecture.md` +- Current package: `package.json` \ No newline at end of file diff --git a/docs/architecture/decisions/001-ujsx-as-ir.md b/docs/architecture/decisions/001-ujsx-as-ir.md new file mode 100644 index 0000000..004f240 --- /dev/null +++ b/docs/architecture/decisions/001-ujsx-as-ir.md @@ -0,0 +1,51 @@ +# ADR-001: UJSX Element Tree as the IR + +## Status + +Proposed + +## Context + +The research docs (`docs/research/architecture.md`) proposed a `DbTypeBuilder` class with methods like `DbType.String()`, `DbType.Table()` — a programmatic API that produces `TDbColumn` objects wrapping TypeBox schemas with database metadata. This is the same pattern used by drizzle-orm itself (builder methods returning column objects). + +Meanwhile, `@alkdev/ujsx` provides an element-based system where `
`, `` trees carry the same metadata in props, compose via function components, and render to dialect-specific outputs via `HostConfig`. + +These two approaches produce the same output (TypeBox schemas + Drizzle column builders) but differ in authoring surface and composability. + +## Decision + +Use UJSX element trees as the IR, not the `DbTypeBuilder` API. + +## Rationale + +1. **Composition**: Function components (`IdColumn`, `AuditColumns`) compose naturally in the element tree. A `DbTypeBuilder` API would need spread patterns or explicit column merging to achieve the same result. + +2. **No duplication**: The `DbTypeBuilder` pattern would create a parallel API to what UJSX elements already provide. Both would carry type, notNull, primaryKey, default metadata. The element tree IS the IR. + +3. **Future extensibility**: The UJSX reconciler (`prepareUpdate`/`commitUpdate`) can diff old and new element trees for schema migration. A builder API has no diffing capability — you'd need to compare two separate builder call sites. + +4. **Already proven**: The e2e probe (`scripts/probe-e2e.ts`) validates the full pipeline: UJSX elements → extractTable → Type.Module → Value.Check → Drizzle rendering. All 28 assertions pass. + +5. **Consistency with sibling projects**: Flowgraph uses UJSX for workflow templates. Using the same pattern for dbtype creates a unified mental model across the alkdev stack. + +## Consequences + +### Positive + +- Single authoring surface (elements) instead of two (elements + builder) +- Function components for reusable column groups (the storage_sqlite `commonCols` pattern, but as composable components) +- Reconciler-ready for future migration support +- Leverages existing `@alkdev/ujsx` infrastructure + +### Negative + +- Requires `@alkdev/ujsx` as a peer dependency (adds a dependency) +- JSX syntax requires build configuration (`jsxImportSource`), though `h()` calls work without it +- Less familiar to developers used to drizzle's builder API +- Function component resolution must happen before host rendering — the tree must be "resolved" (function components called) before the host processes it. The probe validates this works. + +## References + +- Probe validation: `scripts/probe-e2e.ts` +- Research comparison: `docs/research/architecture.md` (DbTypeBuilder section) +- UJSX architecture: `@alkdev/ujsx/docs/architecture/` \ No newline at end of file diff --git a/docs/architecture/decisions/002-type-module-as-bundle.md b/docs/architecture/decisions/002-type-module-as-bundle.md new file mode 100644 index 0000000..9004add --- /dev/null +++ b/docs/architecture/decisions/002-type-module-as-bundle.md @@ -0,0 +1,50 @@ +# ADR-002: Type.Module as the Schema Bundle + +## Status + +Proposed + +## Context + +Schemas need to be stored, validated, related, and serialized. The research docs proposed a custom `TransformRegistry` and `DbType` kind system for this. `@alkdev/typebox` already provides `Type.Module` with `Type.Ref` resolution, `Value.Check` validation, and JSON Schema serialization — exactly what we need. + +## Decision + +Use `Type.Module` as the schema bundle. No custom registry or kind system. + +## Rationale + +1. **Ref resolution works**: `Type.Ref('TableName')` inside a module resolves correctly for forward references, circular references, and mutual references. No import-order management, no separate `relations.ts`. + +2. **Validation works**: `Value.Check(M.Import('Users'), userData)` validates after format registration. The module carries all `$defs` and resolves `$ref` pointers. + +3. **Serialization works**: `JSON.stringify(M.Import('Users'))` produces valid JSON Schema with `$defs`. This enables migration diffing via `Value.Diff`. + +4. **Incremental construction**: Build the defs map as a plain `Record`, mutate it, then compile with `Type.Module`. No special API needed. + +5. **Derived schemas**: `Type.Partial(Type.Ref('Users'))` produces an update schema. Insert schemas can be added as explicit module entries. No custom derivation logic needed. + +6. **No new abstraction**: Using what TypeBox already provides means less code to maintain and fewer custom kind registrations. + +## Consequences + +### Positive + +- No custom kind system or registry to maintain +- Circular/mutual references handled natively +- Free JSON Schema serialization for migration diffing +- Free validation via Value.Check +- Free derived schemas via Type.Partial + Type.Ref + +### Negative + +- Module keys are a flat namespace — no nested paths like `"tables/Users"` (acceptable, table names are already unique within a database) +- `Type.Ref` outside a module has `static: unknown` — always use `M.Import(key)` for type inference +- Every `Import()` call embeds the full `$defs` — increases JSON Schema payload size for individual schema exports (acceptable for our use case) +- Custom column metadata (`notNull`, `primaryKey`, `default`) is not in the TypeBox schema — it lives in the element props and column metadata extracted during tree walking. The module carries validation types, not database constraints. + +## References + +- Probe validation: `scripts/probe-e2e.ts` +- TypeBox Module API: `@alkdev/typebox/src/type/module/module.ts` +- Research comparison: `docs/research/architecture.md` (DbType IR section), `docs/research/typedef-kind-pattern.md` \ No newline at end of file diff --git a/docs/architecture/decisions/003-hostconfig-for-dialects.md b/docs/architecture/decisions/003-hostconfig-for-dialects.md new file mode 100644 index 0000000..b00039d --- /dev/null +++ b/docs/architecture/decisions/003-hostconfig-for-dialects.md @@ -0,0 +1,49 @@ +# ADR-003: HostConfig for Dialect Rendering + +## Status + +Proposed + +## Context + +The research docs proposed a `TransformRegistry` with priority-sorted rules matching `DbType:*` kinds to dialect-specific Drizzle column builders. UJSX provides `HostConfig` — a similar pattern where `createInstance(tag, props, ctx)` maps element types to output instances. + +These solve the same problem: dispatch a schema element to the appropriate dialect-specific builder. + +## Decision + +Use `HostConfig` for dialect rendering, not a custom `TransformRegistry`. + +## Rationale + +1. **Same pattern, more capabilities**: Both `HostConfig.createInstance` and `TransformRule.transform` map an input type to an output instance. HostConfig adds `appendChild` (for composing columns into tables), `finalizeInstance` (for calling `sqliteTable()` after all columns are ready), and `prepareUpdate`/`commitUpdate` (for future migration diffing). + +2. **Function components are free composition**: ``, `` compose via the UJSX reconciliation. A TransformRegistry would need its own composition mechanism. + +3. **Reconciler for free**: The UJSX reconciler handles element diffing, key-based matching, and child reordering. For future schema migration, `prepareUpdate`/`commitUpdate` can diff old and new element props to compute ALTER TABLE statements. + +4. **Context for dialect config**: `createRootContext` carries dialect-specific configuration (naming conventions, type mappings, enum pre-declarations). + +5. **Already proven**: The probe scripts use HostConfig to render element trees to Drizzle SQLite tables. The pattern works end-to-end. + +## Consequences + +### Positive + +- Leverages existing UJSX infrastructure +- Future migration support via reconciler +- Function component composition for common column patterns +- Context propagation for dialect-specific config + +### Negative + +- `createTextInstance` is required by HostConfig but semantically wrong for DB schemas — must throw or noop +- More infrastructure than a simple TransformRegistry (fiber tree, reconciler) +- HostConfig is defined in `@alkdev/ujsx`, creating a tighter coupling between the packages +- The transform registry from the research docs could handle cross-dialect conversion (PG → SQLite), which HostConfig doesn't address — but we don't need cross-dialect conversion because the element tree IS dialect-agnostic + +## References + +- Probe host rendering: `scripts/probe-e2e.ts` +- UJSX HostConfig: `@alkdev/ujsx/docs/architecture/host-config.md` +- Research comparison: `docs/research/architecture.md` (Transform Registry section), `docs/research/typemap-architecture.md` (module structure) \ No newline at end of file diff --git a/docs/architecture/decisions/004-format-annotation-only.md b/docs/architecture/decisions/004-format-annotation-only.md new file mode 100644 index 0000000..0dab744 --- /dev/null +++ b/docs/architecture/decisions/004-format-annotation-only.md @@ -0,0 +1,46 @@ +# ADR-004: Column Formats Are Annotations + +## Status + +Proposed + +## Context + +When a column has `type: 'uuid'` or `type: 'string'` with `format: 'email'`, the `format` field in the TypeBox schema (`Type.String({ format: 'uuid' })`) serves two purposes: +1. **Validation metadata** — should `Value.Check` enforce the format? +2. **Database metadata** — should the PG host render it as `uuid()` instead of `text()`? + +TypeBox follows JSON Schema semantics: `format` is annotation-only by default. `Value.Check` does not enforce format strings unless validators are registered via `FormatRegistry.Set`. + +## Decision + +Treat `format` as annotation-only in the TypeBox schema. Register specific format validators when validation is needed. Use the column `type` prop (not `format`) for dialect-specific mapping. + +## Rationale + +1. **Probing confirmed this behavior**: `Type.String({ format: 'uuid' })` fails `Value.Check` until `FormatRegistry.Set('uuid', validator)` is called. This is correct JSON Schema behavior. + +2. **Dialect mapping uses `type`, not `format`**: The PG host maps `` to `uuid()`. The SQLite host maps the same to `text()`. The `format` field is an additional hint for validation, not the driver for database rendering. + +3. **Explicit is better than implicit**: Requiring `FormatRegistry.Set` for custom formats makes it clear when validation is active. No silent enforcement of formats the application doesn't care about. + +4. **`format` is still useful**: It provides machine-readable metadata in the JSON Schema serialization. API generators (OpenAPI, GraphQL) can use `format: 'uuid'` or `format: 'email'` for documentation and client-side validation. + +## Consequences + +### Positive + +- Correct JSON Schema semantics +- No surprise validation failures +- `format` still carries metadata for non-validation purposes +- Dialect mapping is explicit (based on `type` prop) + +### Negative + +- Applications must explicitly register format validators to get format enforcement +- Two fields (`type` and `format`) encode overlapping information (e.g., `type: 'uuid'` implies `format: 'uuid'`) + +## References + +- Format registration: `scripts/probe-e2e.ts` +- TypeBox FormatRegistry: `@alkdev/typebox/src/type/registry/format.ts` \ No newline at end of file diff --git a/docs/architecture/decisions/005-repo-as-adapter.md b/docs/architecture/decisions/005-repo-as-adapter.md new file mode 100644 index 0000000..83ef200 --- /dev/null +++ b/docs/architecture/decisions/005-repo-as-adapter.md @@ -0,0 +1,47 @@ +# ADR-005: CRUD Generation as an Operations Adapter + +## Status + +Proposed + +## Context + +drizzle-graphql auto-generates CRUD operations (findMany, insert, update, delete) from Drizzle schemas. The dbtype architecture has enough information (element tree → module → schemas + rendered Drizzle tables) to do the same, but outputting `OperationSpec`s for `@alkdev/operations` instead of GraphQL resolvers. + +This could be a core feature of dbtype, or a separate adapter package. + +## Decision + +CRUD generation is a separate adapter (`@alkdev/dbtype/repo` or a future `@alkdev/dbtype-operations` package), not a core feature. + +## Rationale + +1. **Separation of concerns**: dbtype's core job is "define once, validate and deploy anywhere" — schema definition, validation, and dialect rendering. CRUD generation is a different concern (API generation from schemas). + +2. **The adapter consumes the same inputs**: The `FromDbType` adapter takes the element tree, the Type.Module, and the rendered Drizzle tables — all produced by dbtype core. It's a consumer, not a producer. + +3. **Optional dependency**: Not every dbtype user needs CRUD operations. Keeping it in core would force `@alkdev/operations` as a dependency. + +4. **The adapter pattern is proven**: `@alkdev/operations` already has `from-openapi`, `from-mcp`, and `from-typemap` adapters. `from-dbtype` follows the same pattern. + +5. **Phase ordering**: Core (phase 1) must be stable before building the adapter (phase 2). The adapter depends on stable schema extraction APIs, which don't exist yet. + +## Consequences + +### Positive + +- Core package stays focused on schema + rendering +- Users who only need validation and Drizzle rendering don't pull in operations +- The adapter can evolve independently +- Follows the established adapter pattern from @alkdev/operations + +### Negative + +- Two packages to install for the full experience +- The adapter needs access to both Type.Module entries (for schemas) and rendered Drizzle tables (for query execution), which means the rendering pipeline must expose both outputs + +## References + +- Operations adapters: `@alkdev/operations/from-openapi`, `@alkdev/operations/from-mcp` +- drizzle-GraphQL reference: `/workspace/drizzle-graphql` +- Repo adapter design: [repo-adapter.md](repo-adapter.md) \ No newline at end of file diff --git a/docs/architecture/elements.md b/docs/architecture/elements.md new file mode 100644 index 0000000..5d5fa29 --- /dev/null +++ b/docs/architecture/elements.md @@ -0,0 +1,175 @@ +--- +status: draft +last_updated: 2026-05-22 +--- + +# Elements: UJSX Element Definitions + +The UJSX element types, their props, function components, and how they compose. + +## Overview + +dbtype elements are UJSX elements (`h()` calls or JSX) with a constrained set of tags: `table`, `column`, `index`, `fk`. Each element carries typed props that encode both validation metadata (for TypeBox) and database metadata (for Drizzle rendering). + +## Element Types + +### `
` + +The top-level schema element. Contains ``, ``, and `` children. + +| Prop | Type | Required | Description | +|------|------|----------|-------------| +| `name` | `string` | yes | Table name in the database | + +Children are column, index, and foreign key elements. Function component children that return column elements are transparent (their output is used, not the component itself). + +### `` + +A single column definition within a table. + +| Prop | Type | Required | Description | +|------|------|----------|-------------| +| `name` | `string` | yes | Column name | +| `type` | `DbColumnType` | yes | Column type vocabulary | +| `notNull` | `boolean` | no | Column is NOT NULL | +| `primaryKey` | `boolean` | no | Column is primary key | +| `unique` | `boolean` | no | Column has UNIQUE constraint | +| `default` | `DbDefault \| unknown` | no | Symbolic default or literal value | +| `references` | `string` | no | FK target table name | +| `format` | `string` | no | TypeBox format annotation (uuid, email, etc.) | +| `mode` | `'json' \| 'text'` | no | Storage mode for compound types | +| `values` | `string[]` | no | Enum values (for `type: 'enum'`) | +| `length` | `number` | no | Max length (for varchar) | +| `precision` | `number` | no | Numeric precision | +| `scale` | `number` | no | Numeric scale | +| `postgres` | `PgColumnOpts` | no | PG-specific overrides | +| `sqlite` | `SqliteColumnOpts` | no | SQLite-specific overrides | +| `mysql` | `MySqlColumnOpts` | no | MySQL-specific overrides | + +### `` + +An index definition on a table. + +| Prop | Type | Required | Description | +|------|------|----------|-------------| +| `name` | `string` | yes | Index name | +| `columns` | `string[]` | yes | Column names in the index | +| `unique` | `boolean` | no | Whether the index is unique | + +### `` + +A foreign key constraint. + +| Prop | Type | Required | Description | +|------|------|----------|-------------| +| `columns` | `string[]` | yes | Local column names | +| `references` | `string` | yes | Target table name | +| `foreignColumns` | `string[]` | yes | Target column names | +| `onDelete` | `'cascade' \| 'set null' \| 'restrict' \| 'no action'` | no | ON DELETE action | +| `onUpdate` | `'cascade' \| 'set null' \| 'restrict' \| 'no action'` | no | ON UPDATE action | + +## Column Type Vocabulary + +`DbColumnType` is the cross-dialect type vocabulary. Each value maps to a specific Drizzle column builder per dialect: + +```typescript +type DbColumnType = + | 'uuid' + | 'string' + | 'text' + | 'varchar' + | 'integer' + | 'bigint' + | 'boolean' + | 'timestamp' + | 'real' + | 'numeric' + | 'enum' + | 'json' + | 'array' + | 'object' +``` + +The mapping to TypeBox and Drizzle types is defined in the host, but the element tree uses `DbColumnType` as the universal vocabulary. + +## Symbolic Defaults + +```typescript +type DbDefault = + | 'now' // Current timestamp (dialect-specific SQL) + | 'uuid' // UUID generation (dialect-specific mechanism) + | 'autoincrement' // Auto-incrementing integer + | 'current_timestamp' // CURRENT_TIMESTAMP + | unknown // Literal value or SQL expression +``` + +## Function Components + +Reusable column groups defined as UJSX components: + +```typescript +// Common ID column +const IdColumn = createComponent('IdColumn', () => + h('column', { name: 'id', type: 'uuid', primaryKey: true, default: 'uuid' }) +) + +// Audit timestamp columns +const AuditColumns = createComponent('AuditColumns', () => [ + h('column', { name: 'createdAt', type: 'timestamp', notNull: true, default: 'now' }), + h('column', { name: 'updatedAt', type: 'timestamp', notNull: true, default: 'now' }), +]) + +// Usage +const UsersEl = h('table', { name: 'users' }, + h(IdColumn, {}), + h('column', { name: 'name', type: 'string', notNull: true }), + h(AuditColumns, {}), +) +``` + +Function components are transparent — the host never sees them. The `createInstance` method only receives resolved intrinsic elements (`column`, `table`). + +## Tree Walking + +The `extractTable()` function walks an element tree, resolving function components, and produces: + +1. **TypeBox schema**: `Type.Object({ ... })` for the module entry +2. **Column metadata**: `Record` for Drizzle rendering +3. **Table metadata**: `{ name, columns, indexes, foreignKeys }` for the host + +```typescript +function extractTable(el: UElement): { + name: string + schema: TObject // TypeBox schema for Type.Module + columns: Record // Props + computed metadata + indexes: IndexMeta[] + foreignKeys: FkMeta[] +} +``` + +For each column, `extractTable`: +- Calls `colToTypeBox(type, props)` to produce the inner TypeBox schema +- Stores the full props object as metadata for the host +- Tracks `primaryKey`, `default`, `notNull`, `references` for schema derivation (insert, update) + +## Constraints + +- **Column elements must have `name` and `type` props** — these are required for both TypeBox schema construction and Drizzle rendering +- **Function components must return column elements** (or arrays of column elements) — returning other element types inside a table is undefined +- **The `references` prop is metadata-only** — it's not part of the TypeBox schema. It informs the host about FK constraints and the repo adapter about relation structure +- **`default` values can be symbolic strings or literals** — symbolic defaults (`now`, `uuid`, `autoincrement`) are resolved by the host; literal values pass through directly + +## Open Questions + +1. **Should column elements support `inner` TypeBox schemas?** The research docs proposed `DbType.String({ notNull: true, inner: Type.String({ format: 'email', maxLength: 255 }) })`. With UJSX elements, this would be ``. Is the flat props model sufficient, or do we need nested TypeBox schemas? + +2. **Should `
` accept `extraConfig` props?** Drizzle tables accept a third callback argument for indexes and unique constraints defined in terms of column references. How does this map to element props? + +3. **Should we support JSX file extensions?** The current probe uses `h()` calls. JSX syntax (`.tsx` files with `jsxImportSource: '@alkdev/ujsx'`) would be more ergonomic for authoring but requires build configuration. + +## References + +- UJSX element factory: `@alkdev/ujsx/src/core/h.ts` +- UJSX HostConfig: `@alkdev/ujsx/src/host/config.ts` +- Probe element construction: `scripts/probe-e2e.ts` +- Research: `docs/research/architecture.md` (DbTypeBuilder section) \ No newline at end of file diff --git a/docs/architecture/hosts.md b/docs/architecture/hosts.md new file mode 100644 index 0000000..39a7933 --- /dev/null +++ b/docs/architecture/hosts.md @@ -0,0 +1,151 @@ +--- +status: draft +last_updated: 2026-05-22 +--- + +# Hosts: Dialect Rendering via HostConfig + +How dbtype renders UJSX element trees to Drizzle table definitions for each supported database dialect. + +## Overview + +Each database dialect (SQLite, PostgreSQL, MySQL) is a `HostConfig` implementation. The same UJSX element tree renders to `sqliteTable`, `pgTable`, or `mysqlTable` depending on which host is chosen. The host translates column type props, symbolic defaults, and constraints to the appropriate Drizzle column builder methods. + +## HostConfig Interface + +```typescript +interface DbtypeRootCtx { + dialect: Dialect + tables: Record +} + +interface DbtypeTableInst { + name: string + columns: Record // Drizzle column builders + constraints: any[] + result?: any // The final drizzle table +} + +interface DbtypeColumnInst { + name: string + columnType: string + builder: any // Drizzle column builder + dbMeta: Record +} +``` + +The `HostConfig` generic parameters: +- `TTag`: `"table" | "column" | "index" | "fk"` — the allowed element types +- `TInstance`: The union of table and column instance types for this dialect +- `TRootCtx`: The dialect-specific root context + +## Column Type Mapping + +### SQLite + +| Column Type Prop | Drizzle Builder | Notes | +|-----------------|----------------|-------| +| `uuid` | `text(name).primaryKey().$defaultFn(crypto.randomUUID)` | No native UUID type | +| `string` | `text(name)` | | +| `integer` | `integer(name)` | | +| `boolean` | `integer(name, { mode: 'boolean' })` | | +| `timestamp` | `integer(name, { mode: 'timestamp' })` | | +| `enum` | `text(name, { enum: values })` | SQLite has no native enum | +| `json` | `text(name, { mode: 'json' })` | Stored as JSON text | + +### PostgreSQL + +| Column Type Prop | Drizzle Builder | Notes | +|-----------------|----------------|-------| +| `uuid` | `uuid(name).defaultRandom()` or `.primaryKey()` | Native UUID | +| `string` | `text(name)` or `varchar(name, n)` | | +| `integer` | `integer(name)` | | +| `boolean` | `boolean(name)` | | +| `timestamp` | `timestamptz(name, { withTimezone: true })` | | +| `enum` | `pgEnum(name, values)` | Requires pre-declaration | +| `json` | `jsonb(name)` | | + +### MySQL + +| Column Type Prop | Drizzle Builder | Notes | +|-----------------|----------------|-------| +| `uuid` | `varchar(name, { length: 36 })` | No native UUID | +| `string` | `text(name)` or `varchar(name, n)` | | +| `integer` | `int(name)` | | +| `boolean` | `boolean(name)` or `tinyint(name)` | | +| `timestamp` | `timestamp(name)` | | +| `enum` | `mysqlEnum(name, values)` | | +| `json` | `json(name)` | | + +## Symbolic Defaults + +Column props like `default="now"` and `default="uuid"` are translated to dialect-specific SQL or JS functions by the host: + +| Symbol | SQLite | PostgreSQL | MySQL | +|--------|--------|------------|-------| +| `now` | `sql\`(strftime('%s', 'now'))\`` | `sql\`now()\`` | `sql\`NOW()\`` | +| `uuid` | `.$defaultFn(() => crypto.randomUUID())` | `.defaultRandom()` | `.$defaultFn(() => crypto.randomUUID())` | +| `autoincrement` | Implicit on `INTEGER PRIMARY KEY` | `serial()` type | `.autoincrement()` | + +## Common Column Components + +Function components that compose into any table: + +```typescript +const IdColumn = createComponent('IdColumn', () => + h('column', { name: 'id', type: 'uuid', primaryKey: true, default: 'uuid' }) +) + +const AuditColumns = createComponent('AuditColumns', () => [ + h('column', { name: 'createdAt', type: 'timestamp', notNull: true, default: 'now' }), + h('column', { name: 'updatedAt', type: 'timestamp', notNull: true, default: 'now' }), +]) +``` + +Used by spreading into a table: + +```typescript +const UsersEl = h('table', { name: 'users' }, + h(IdColumn, {}), + h('column', { name: 'name', type: 'string', notNull: true }), + h(AuditColumns, {}), +) +``` + +This replaces the storage_sqlite pattern of `const commonCols = { id: text('id').primaryKey(), ... }` with reusable, dialect-agnostic components. + +## Rendering Pipeline + +1. **Walk elements**: The host's `createInstance` method receives each `
`, ``, ``, `` element +2. **Build columns**: `createInstance('column', props, ctx)` maps `props.type` to the dialect-specific Drizzle builder, applying `notNull`, `primaryKey`, `unique`, `default` constraints +3. **Assemble tables**: `appendChild(tableInst, columnInst, ctx)` attaches column builders to the table's columns record +4. **Finalize**: `finalizeInstance(tableInst, ctx)` calls `sqliteTable(name, columns)` (or `pgTable`, `mysqlTable`) and stores the result + +```typescript +const root = createRoot(sqliteHost, {}) +root.render(UsersEl) +// root's context now contains: { tables: { users: drizzleSqliteTable } } +``` + +## Constraints + +- **`createTextInstance` is not supported** — DB schemas are purely structural, no text nodes +- **Column type props must be from the `DbColumnType` vocabulary** — hosts map these to dialect-specific builders; unknown types fall back to `text()` +- **Symbolic defaults are resolved by the host** — `default="now"` becomes `sql\`(strftime('%s', 'now'))\`` on SQLite and `sql\`now()\`` on PG +- **PG enums require pre-declaration** — the host must track enum types and emit `pgEnum()` calls before tables that reference them +- **A host renders one dialect at a time** — to generate schemas for multiple dialects, render multiple times with different hosts + +## Open Questions + +1. **How to handle PG enum pre-declaration?** PG requires `pgEnum()` at module scope before tables. Options: (A) return both enums and tables from render, (B) start with text for all enums, (C) per-column opt-in. Leaning toward A. + +2. **Should hosts return the rendered table or store it in context?** The probe scripts use context (`ctx.tables`), but returning from render would be more functional. Need to resolve this. + +3. **`prepareUpdate`/`commitUpdate` for migrations?** The UJSX reconciler could diff old and new element trees to produce ALTER TABLE statements. This is a future feature, not phase 1. + +## References + +- UJSX HostConfig: `@alkdev/ujsx/src/host/config.ts` +- Drizzle column diffs: `docs/research/dizzle-column-diffs.md` +- Storage pattern: `@alkdev/storage_sqlite/` +- Probe: `scripts/probe-e2e.ts` \ No newline at end of file diff --git a/docs/architecture/module.md b/docs/architecture/module.md new file mode 100644 index 0000000..ffedcc1 --- /dev/null +++ b/docs/architecture/module.md @@ -0,0 +1,170 @@ +--- +status: draft +last_updated: 2026-05-22 +--- + +# Module: Type.Module as the Schema Bundle + +Technical details on how dbtype uses `Type.Module` for schema construction, validation, serialization, and migration. + +## Overview + +dbtype uses `@alkdev/typebox`'s `Type.Module` as the schema storage and resolution mechanism. A module holds all table schemas, their relations, and derived schemas (insert, update, partial) in a single flat namespace. `Type.Ref` resolves cross-table references — including circular ones — without import ordering issues. + +This document covers the mechanics, constraints, and patterns discovered during architecture probing. + +## Construction Patterns + +### Basic Pattern + +```typescript +const defs: Record = { + Users: Type.Object({ id: Type.String({ format: 'uuid' }), name: Type.String() }), + Tasks: Type.Object({ id: Type.String({ format: 'uuid' }), title: Type.String() }), +} + +const M = Type.Module(defs) +const Users = M.Import('Users') +``` + +### With Relations + +```typescript +defs.UsersRelations = Type.Object({ tasks: Type.Array(Type.Ref('Tasks')) }) +defs.TasksRelations = Type.Object({ user: Type.Ref('Users') }) +``` + +`Type.Ref('Users')` within the module resolves to the Users schema. No circular import issues. + +### With Derived Schemas + +```typescript +defs.InsertUsers = Type.Object({ name: Type.String(), email: Type.String() }) // manual +defs.UpdateUsers = Type.Partial(Type.Ref('Users')) // computed +``` + +### Incremental Construction + +```typescript +// Build defs incrementally +const defs: Record = {} +defs.Users = extractTableSchema(UsersElement) +defs.Tasks = extractTableSchema(TasksElement) + +// Add a column later +defs.Users = Type.Object({ ...defs.Users.properties, role: Type.String() }) + +// Add relations +defs.UsersRelations = Type.Object({ tasks: Type.Array(Type.Ref('Tasks')) }) + +// Compile when ready +const M = Type.Module(defs) +``` + +## Validation + +### Format Registration Required + +TypeBox treats `format` as an annotation by default. To enforce format validation, register custom formats: + +```typescript +import { FormatRegistry } from '@alkdev/typebox' + +FormatRegistry.Set('uuid', (value) => /^[0-9a-f]{8}-...$/i.test(value)) +FormatRegistry.Set('email', (value) => /^[^@]+@[^@]+\.[^@]+$/.test(value)) +``` + +After registration, `Value.Check` enforces these formats. + +### Validation Pattern + +```typescript +const M = Type.Module(defs) +const Users = M.Import('Users') + +// Valid +Value.Check(Users, { id: '550e8400-e29b-41d4-a716-446655440000', name: 'alice', email: 'a@b.com', ... }) + +// Invalid — Value.Check returns false, Value.Errors provides details +Value.Check(Users, { id: 'bad-uuid', ... }) // false +for (const err of Value.Errors(Users, badData)) { ... } +``` + +## Serialization + +`JSON.stringify(M.Import(key))` produces JSON Schema with `$defs`: + +```json +{ + "$defs": { + "Users": { "$id": "Users", "type": "object", "properties": { ... }, "required": [...] }, + "Tasks": { "$id": "Tasks", "type": "object", "properties": { ... }, "required": [...] }, + "UsersRelations": { "$id": "UsersRelations", "type": "object", "properties": { "tasks": { "type": "array", "items": { "$ref": "Tasks" } } } } + }, + "$ref": "Users" +} +``` + +Key properties: +- Each `$defs` entry has an `$id` matching its key +- `Type.Ref` remains as `{ "$ref": "Key" }` — not inlined +- The entire structure is valid JSON Schema +- All entries in the module are present in `$defs` (even if only one was imported) + +### Roundtrip + +The serialized form can be parsed back into a schema-like structure. `Value.Diff` works on these serialized objects to produce structural edit lists. + +## Migration Diffing + +```typescript +const v1 = JSON.parse(JSON.stringify(M.Import('Users'))) + +// Modify schema +defs.Users = Type.Object({ ...defs.Users.properties, role: Type.String() }) + +const M2 = Type.Module(defs) +const v2 = JSON.parse(JSON.stringify(M2.Import('Users'))) + +const edits = Value.Diff(v1, v2) +// [ +// { type: 'insert', path: '/$defs/Users/properties/role', value: { type: 'string' } }, +// { type: 'update', path: '/$defs/Users/required/3', value: 'role' }, +// ] +``` + +Edits use JSON Pointer paths. A migration generator can translate these to: +- `INSERT` for new properties → `ALTER TABLE ADD COLUMN` +- `DELETE` for removed properties → `ALTER TABLE DROP COLUMN` +- `UPDATE` for type changes → `ALTER TABLE ALTER COLUMN TYPE` + +This is structural diffing, not semantic — it doesn't understand that changing `Type.String()` to `Type.String({ maxLength: 255 })` is a constraint addition, not a type change. Semantic diffing is a future concern. + +## Cross-Module References + +`Module.Import()` embeds the source module's `$defs` in the resulting `TImport` schema. This enables referencing types from another module: + +```typescript +const CommonM = Type.Module({ Uuid: Type.String({ format: 'uuid' }) }) +const CommonUuid = CommonM.Import('Uuid') + +const AppM = Type.Module({ + User: Type.Object({ id: CommonUuid, name: Type.String() }), +}) +``` + +However, this nests `$defs` within `$defs` (the User's `$defs` contains CommonUuid's `$defs`), which increases payload size. For dbtype's use case, keeping everything in a single module is simpler and avoids nesting. + +## Constraints + +- **Module entries are computed at construction time** — `Type.Partial(Type.Ref('Users'))` is resolved when the module is built, producing a concrete optional-property object +- **`Type.Ref` outside a module has `static: unknown`** — always use `M.Import(key)` for proper type inference +- **Module keys are a flat namespace** — no nested paths like `"tables/Users"`. Table names must be unique within the module. +- **`Module.Import` embeds all `$defs`** — every import carries the full module. This is correct for validation but increases JSON Schema size. +- **Symbol properties are lost in `JSON.stringify`** — `[Kind]`, `[Hint]`, etc. are stripped. The serialized form is JSON Schema, not TypeBox schema. Roundtripping requires `FromSchema` or reconstructed TypeBox objects. + +## References + +- TypeBox Module: `@alkdev/typebox/src/type/module/module.ts` +- TypeBox Format: `@alkdev/typebox/src/type/registry/format.ts` +- Probe: `scripts/probe-e2e.ts` \ No newline at end of file diff --git a/docs/architecture/open-questions.md b/docs/architecture/open-questions.md new file mode 100644 index 0000000..e787ce7 --- /dev/null +++ b/docs/architecture/open-questions.md @@ -0,0 +1,104 @@ +--- +status: draft +last_updated: 2026-05-22 +--- + +# Open Questions Tracker + +All unresolved architectural questions for dbtype, organized by theme. + +## Schema & Module + +### OQ-01: Should relation entries use a naming convention? + +- **Origin**: [schema.md](schema.md) +- **Status**: Open +- **Priority**: Medium +- **Context**: Currently `UsersRelations` / `TasksRelations`. Is this naming convention sufficient, or should relations be structured differently? A `relations` field on the table entry would couple relations to table definitions. +- **Cross-references**: OQ-03 + +### OQ-02: Should derived schemas live in the module or be extracted separately? + +- **Origin**: [schema.md](schema.md) +- **Status**: Open +- **Priority**: Low +- **Context**: Insert/update schemas can be added as module entries (`InsertUsers`, `UpdateUsers`) or extracted by walking the module. Adding them to the module increases size but makes everything accessible via `Import`. Extracting separately is more flexible but requires separate code paths. +- **Cross-references**: OQ-03 + +### OQ-03: Should the module support multiple database namespaces? + +- **Origin**: [schema.md](schema.md) +- **Status**: Open +- **Priority**: Low +- **Context**: One module per database, or one module with all tables across all databases? Probably one per database namespace, but this needs confirmation from real use cases. + +## Element & Host + +### OQ-04: Should column elements support nested TypeBox schemas? + +- **Origin**: [elements.md](elements.md) +- **Status**: Open +- **Priority**: High +- **Context**: The research docs proposed `DbType.String({ notNull: true, inner: Type.String({ format: 'email', maxLength: 255 }) })`. With UJSX elements, this would be ``. The flat props model works for common cases, but custom validation constraints (patterns, ranges, custom checks) may need a nested `inner` prop. Should we support ``? +- **Cross-references**: ADR-001 + +### OQ-05: How to handle PG enum pre-declaration? + +- **Origin**: [hosts.md](hosts.md) +- **Status**: Open +- **Priority**: High +- **Context**: PG requires `pgEnum()` at module scope before tables that reference it. Options: (A) return both enums and tables from render, (B) start with text for all enums, (C) per-column opt-in with `postgres: { nativeEnum: true }`. Option A is cleanest but changes the render API. Option B is simplest but loses PG enum validation. + +### OQ-06: Should hosts return rendered tables or store them in context? + +- **Origin**: [hosts.md](hosts.md) +- **Status**: Open +- **Priority**: Medium +- **Context**: The probe scripts use `ctx.tables` to collect rendered tables. A more functional approach would have `render()` return the rendered table directly. Need to resolve this before implementation. + +### OQ-07: Should we support JSX file extensions? + +- **Origin**: [elements.md](elements.md) +- **Status**: Open +- **Priority**: Low +- **Context**: JSX syntax (`.tsx` with `jsxImportSource: '@alkdev/ujsx'`) would be more ergonomic than `h()` calls. This requires build configuration (TSConfig `jsx`, bundler support). The `h()` API works universally; JSX is a convenience layer. + +## Repo Adapter + +### OQ-08: Per-dialect handler differences? + +- **Origin**: [repo-adapter.md](repo-adapter.md) +- **Status**: Open +- **Priority**: Medium +- **Context**: PG has `.returning()` on all mutations, MySQL often doesn't. Should the adapter handle this transparently (always try `.returning()`, fall back gracefully), or should it be explicit in the config? + +### OQ-09: Relation rendering responsibility? + +- **Origin**: [repo-adapter.md](repo-adapter.md) +- **Status**: Open +- **Priority**: Medium +- **Context**: Should the host render relations (new element type ``), or should the adapter generate them from the module's relation entries? The adapter knows the rendered table objects; the module knows the relation structure. Leaning toward the adapter generating them. + +## Migration & Diffing + +### OQ-10: How far should migration diffing go in phase 1? + +- **Origin**: [module.md](module.md) +- **Status**: Open +- **Priority**: Low +- **Context**: `Value.Diff` on serialized schemas produces structural edit lists (insert/delete/update property). Translating these to `ALTER TABLE` statements is straightforward for additive changes (new column) but complex for destructive ones (drop column, change type). Phase 1 likely skips migration generation entirely (rely on `drizzle-kit`), but the module structure should not prevent it later. + +## Summary Table + +| ID | Question | Origin | Priority | Status | +|----|----------|--------|----------|--------| +| OQ-01 | Relation naming convention | schema.md | Medium | Open | +| OQ-02 | Derived schemas in module or separate | schema.md | Low | Open | +| OQ-03 | Multiple database namespaces | schema.md | Low | Open | +| OQ-04 | Nested TypeBox schemas in column props | elements.md | High | Open | +| OQ-05 | PG enum pre-declaration | hosts.md | High | Open | +| OQ-06 | Host render return value vs context | hosts.md | Medium | Open | +| OQ-07 | JSX file extensions | elements.md | Low | Open | +| OQ-08 | Per-dialect handler differences | repo-adapter.md | Medium | Open | +| OQ-09 | Relation rendering responsibility | repo-adapter.md | Medium | Open | +| OQ-10 | Migration diffing scope | module.md | Low | Open | \ No newline at end of file diff --git a/docs/architecture/repo-adapter.md b/docs/architecture/repo-adapter.md new file mode 100644 index 0000000..a774ec3 --- /dev/null +++ b/docs/architecture/repo-adapter.md @@ -0,0 +1,155 @@ +--- +status: draft +last_updated: 2026-05-22 +--- + +# Repo Adapter: from-dbtype for @alkdev/operations + +How dbtype schemas produce CRUD `OperationSpec`s for the operations registry. + +## Overview + +The `from-dbtype` adapter consumes the same element tree and Type.Module bundle that produces Drizzle tables and validation schemas, and generates a complete set of CRUD `OperationSpec`s: `findMany`, `findFirst`, `insertOne`, `insert`, `update`, `delete`. + +This is phase 2 of the project. Phase 1 delivers the core schema, module, and hosts. This document defines the interface and design so phase 1 doesn't paint us into a corner. + +## Interface + +```typescript +interface FromDbTypeConfig { + namespace: string // Operation namespace (e.g., 'db') + db: AnyDrizzleDB // Drizzle database instance + tables: Record // UJSX table elements + host: HostConfig<...> // Rendered dialect host + schema?: Record // Optional: Type.Module entries for select/insert/update + operations?: Record // Per-table operation selection + accessControl?: Record // Per-table access control +} + +type OperationSelection = true | ('findMany' | 'findFirst' | 'insertOne' | 'insert' | 'update' | 'delete')[] + +type AccessControlMap = Record // operation name -> access control +``` + +Returns `OperationSpec[]` ready to register with `OperationRegistry`. + +## Schema Derivation + +### Select Schema + +The module entry as-is, with nullable columns wrapped in `Type.Union([inner, Type.Null()])`. + +### Insert Schema + +Derived from column metadata: +- Remove auto-generated primary keys (`primaryKey: true` with `default`) +- Make columns with defaults `Type.Optional` +- Make nullable columns `Type.Optional(Type.Union([inner, Type.Null()]))` +- Keep required not-null columns mandatory + +### Update Schema + +`Type.Partial(Type.Ref('TableName'))` — all columns optional. + +### Filter Schema + +Per-column comparison operators derived from `DbColumnType`: + +| Column Type | Available Operators | +|-------------|-------------------| +| `uuid`, `string` | `eq`, `ne`, `like`, `notLike`, `ilike`, `inArray`, `notInArray`, `isNull`, `isNotNull` | +| `integer`, `timestamp` | `eq`, `ne`, `lt`, `lte`, `gt`, `gte`, `inArray`, `notInArray`, `isNull`, `isNotNull` | +| `boolean` | `eq`, `ne`, `isNull`, `isNotNull` | +| `json`, `array`, `object` | `eq`, `isNull`, `isNotNull` | + +```typescript +function generateFilterSchema(tableMeta: TableMeta): TObject { + const filterProps: Record = {} + for (const [name, col] of Object.entries(tableMeta.columns)) { + filterProps[name] = Type.Optional(generateColumnFilter(col)) + } + return Type.Object(filterProps) +} +``` + +## Handler Generation + +Each operation's handler uses the rendered Drizzle table and the Drizzle query builder: + +| Operation | Handler Pattern | +|-----------|----------------| +| `findMany` | `db.query[tableName].findMany({ where, orderBy, limit, offset, with })` | +| `findFirst` | `db.query[tableName].findFirst({ where, orderBy, offset, with })` | +| `insertOne` | `db.insert(table).values(input).returning()` | +| `insert` | `db.insert(table).values(input.values).returning()` | +| `update` | `db.update(table).set(input.set).where(filterToWhere(input.where)).returning()` | +| `delete` | `db.delete(table).where(filterToWhere(input.where)).returning()` | + +`filterToWhere()` translates the TypeBox-validated filter object into Drizzle SQL operators (`eq()`, `and()`, `or()`, `like()`, etc.). + +## Access Control + +The adapter accepts per-table, per-operation `AccessControl` specs that map directly to `OperationSpec.accessControl`: + +```typescript +accessControl: { + users: { + findMany: { requiredScopes: ['users:read'] }, + insertOne: { requiredScopes: ['users:write'] }, + update: { requiredScopes: ['users:write'] }, + delete: { requiredScopes: ['users:admin'] }, + } +} +``` + +Operations called through `buildEnv()` are `trusted: true` — internal composition skips access control. + +## Relations + +The module's relation entries (`UsersRelations`, `TasksRelations`) drive the `with` parameter in `findMany` and `findFirst`. The adapter: + +1. Reads relation entries from the module +2. Generates the Drizzle `relations()` calls from the rendered table objects +3. Passes the full `{ tables, relations }` schema to the Drizzle relational query builder +4. Maps `with` parameter types from the module to the Drizzle `with` API + +## Overrides + +Individual operations can be overridden: + +```typescript +overrides: { + users: { + delete: async (input, ctx) => { + // Soft delete instead of hard delete + return ctx.env.db.update(usersTable) + .set({ deletedAt: new Date() }) + .where(eq(usersTable.id, input.where.id)) + } + } +} +``` + +Overrides replace the auto-generated handler but keep the auto-generated `OperationSpec` (input/output schemas, access control). + +## Constraints + +- **Phase 2 concern** — this adapter is not part of phase 1. Phase 1 delivers the schema, module, and hosts. +- **The adapter depends on both the element tree and the rendered Drizzle objects** — it needs the tree for schema derivation and the rendered tables for query execution +- **Filter operators are column-type-dependent** — a `uuid` column gets different operators than a `boolean` column +- **The `with` parameter for relations requires the Drizzle `relations` object** — the host must also render relations, not just tables +- **Returning clauses are dialect-specific** — PG supports `.returning()` on all mutations, MySQL does not. The adapter must handle this per-dialect. + +## Open Questions + +1. **Per-dialect handler differences?** PG has `.returning()` on all mutations, MySQL often doesn't. Should the adapter handle this transparently, or expose it in the config? + +2. **Schema bundle or separate schemas?** Should the adapter accept the `Type.Module` compiled bundle, or individual schemas? The module is convenient (everything in one place) but couples the adapter to TypeBox's module format. + +3. **Relation rendering responsibility?** Should the host render relations (new `` element), or should the adapter generate them from the module's relation entries? + +## References + +- Operations registry: `@alkdev/operations` — `OperationSpec`, `OperationRegistry`, `OperationHandler` +- Drizzle-GraphQL reference: `/workspace/drizzle-graphql` — CRUD generator pattern +- Research: `docs/research/architecture.md` (validation schemas section) \ No newline at end of file diff --git a/docs/architecture/schema.md b/docs/architecture/schema.md new file mode 100644 index 0000000..9a70969 --- /dev/null +++ b/docs/architecture/schema.md @@ -0,0 +1,166 @@ +--- +status: draft +last_updated: 2026-05-22 +--- + +# Schema: Type.Module as the Schema Bundle + +How dbtype uses `Type.Module` to store all table schemas, relations, and derived schemas in a single namespace with automatic `Type.Ref` resolution. + +## Overview + +The `Type.Module` is the central data structure in dbtype. It holds every table's TypeBox schema, all cross-table relations, and derived schemas (insert, update, select variants) in one flat namespace. `Type.Ref` resolves forward and circular references naturally, eliminating the need for separate relation files or import-order management. + +The module is also the serialization boundary: `JSON.stringify(module.Import('Users'))` produces valid JSON Schema with `$defs`, enabling migration diffing via `Value.Diff`. + +## Construction + +### From Element Tree to Module + +The element tree (`
`, ``) is walked to extract a `Record` map, then compiled into a module: + +``` +UJSX elements → extractTable() → { name, schema, columns } → defs map → Type.Module(defs) +``` + +Each `` element produces a TypeBox type based on its `type` prop: + +| Column Type Prop | TypeBox Schema | +|-----------------|---------------| +| `uuid` | `Type.String({ format: 'uuid' })` | +| `string` | `Type.String()` | +| `integer` | `Type.Integer()` | +| `boolean` | `Type.Boolean()` | +| `timestamp` | `Type.Number()` | +| `enum` | `Type.Union(values.map(v => Type.Literal(v)))` | + +### Incremental Construction + +The defs map is a plain `Record` — it can be built incrementally, mutated, and extended before compilation: + +```typescript +const defs: Record = {} + +// Add tables one at a time +defs.Users = Type.Object({ id: Type.String({ format: 'uuid' }), name: Type.String() }) +defs.Tasks = Type.Object({ id: Type.String({ format: 'uuid' }), userId: Type.String({ format: 'uuid' }), title: Type.String() }) + +// Add columns to an existing table +defs.Users = Type.Object({ ...defs.Users.properties, role: Type.String() }) + +// Add relations +defs.UsersRelations = Type.Object({ tasks: Type.Array(Type.Ref('Tasks')) }) +defs.TasksRelations = Type.Object({ user: Type.Ref('Users') }) + +// Compile +const M = Type.Module(defs) +``` + +Once compiled, `M.Import(key)` returns a `TImport` schema with the full `$defs` namespace embedded. + +## Schema Derivation + +### Select Schema + +The module entry as-is is the select schema. Every column is present, nullable columns become `Type.Union([innerType, Type.Null()])`. + +### Insert Schema + +Derive from the table entry by: +- Removing auto-generated primary keys (columns with `primaryKey: true` and `default` set) +- Making nullable columns and columns with defaults `Type.Optional` +- Keeping required (`notNull` without default) columns mandatory + +Implemented by adding a computed entry to the module: + +```typescript +defs.InsertUsers = Type.Object({ + name: Type.String(), + email: Type.String(), + // id, createdAt, updatedAt omitted (auto-generated) +}) +``` + +### Update Schema + +All columns optional. Use `Type.Partial(Type.Ref('TableName'))`: + +```typescript +defs.UpdateUsers = Type.Partial(Type.Ref('Users')) +``` + +### Filter Schema + +Per-column comparison operators derived from the column type. Generated by the repo adapter, not the core module. + +## Relations + +Relations are stored as separate entries in the module, using `Type.Ref` to reference other tables: + +```typescript +defs.UsersRelations = Type.Object({ tasks: Type.Array(Type.Ref('Tasks')) }) +defs.TasksRelations = Type.Object({ user: Type.Ref('Users') }) +``` + +This gives: +- **Type-safe validation**: `Value.Check(M.Import('UsersRelations'), { tasks: [...] })` validates the full nested structure +- **No circular import issues**: `Type.Ref` resolves within the module namespace regardless of definition order +- **Queryable structure**: The `$defs` map is enumerable — you can find all relations for a table by naming convention +- **Drizzle integration**: The repo adapter reads relation entries to generate `relations()` calls for drizzle's relational query builder + +Foreign key metadata lives on the column element's `references` prop (``), not in the relation entry. Relations describe the "from this side, I see many of those" semantics. + +## Serialization + +`JSON.stringify(M.Import('TableName'))` produces JSON Schema with `$defs`: + +```json +{ + "$defs": { + "Users": { "$id": "Users", "type": "object", "properties": { ... } }, + "Tasks": { "$id": "Tasks", "type": "object", "properties": { ... } }, + "UsersRelations": { "$id": "UsersRelations", "type": "object", "properties": { "tasks": { "items": { "$ref": "Tasks" }, "type": "array" } } } + }, + "$ref": "Users" +} +``` + +Key properties: +- Each `$defs` entry gets an `$id` matching its key name +- `Type.Ref` leaves `$ref` pointers (not inlined) — consumers must resolve them +- The serialized form is valid JSON Schema +- `Value.Diff` produces structural edits between two serialized schemas (useful for migration diffing) + +## Migration Diffing + +```typescript +const v1 = JSON.parse(JSON.stringify(M.Import('Users'))) +// ... add a column to defs.Users ... +const v2 = JSON.parse(JSON.stringify(M2.Import('Users'))) +const edits = Value.Diff(v1, v2) +// edits: [{ type: 'insert', path: '/$defs/Users/properties/role', value: { type: 'string' } }, ...] +``` + +The edits use JSON Pointer paths, which can be translated to `ALTER TABLE ADD COLUMN` statements. + +## Constraints + +- **Module keys must be unique** — two tables cannot have the same name in the same module +- **`Type.Ref` resolves within the module only** — no cross-module references without `Module.Import` +- **`Type.Ref` outside a module has `static: unknown`** — always access via `M.Import(key)` for proper type inference +- **Defs map is mutable until compiled** — once passed to `Type.Module`, mutations to the original map don't affect the compiled module +- **Format validation requires `FormatRegistry.Set`** — `uuid`, `email`, and other custom formats must be registered before `Value.Check` will enforce them + +## Open Questions + +1. **Should relation entries use a naming convention?** Currently `UsersRelations` / `TasksRelations`. Is this sufficient, or should relations be structured differently (e.g., a `relations` field on the table entry)? + +2. **Derived schemas in the module or separate?** Insert/update schemas can be added as module entries (`InsertUsers`, `UpdateUsers`) or extracted by walking the module schema. Which is cleaner for the repo adapter? + +3. **Should the module support multiple databases?** One module per database, or one module with all tables across all databases? Probably one per database namespace. + +## References + +- TypeBox Module API: `@alkdev/typebox` source — `type/module/module.ts`, `type/ref/ref.ts` +- Probe validation: `scripts/probe-e2e.ts` +- Research: `docs/research/architecture.md` \ No newline at end of file