From d4fd67f4d2688bc57a8a8650ea34a1586526dbb8 Mon Sep 17 00:00:00 2001 From: "glm-5.1" Date: Sat, 23 May 2026 12:06:51 +0000 Subject: [PATCH] docs: resolve architecture open questions, add type definitions, consolidate docs Architecture review session resolving all high-priority open questions and filling documentation gaps identified during review: Decisions resolved: - OQ-04: Flat props with inner escape hatch for column validation (ADR-007) - OQ-05: PG enum pre-declaration returns enums and tables (ADR-008) - OQ-06: Render results accumulate in root.ctx (resolved in hosts.md) - Column references vs fk: references is shorthand, explicit fk takes precedence (ADR-006) - ADR-001, 002, 003 promoted from Proposed to Accepted (probe-validated) Documentation improvements: - Complete DbColumnType mapping tables for all 14 types across 3 dialects - Define ColumnMeta, TableMeta, IndexMeta, FkMeta types in elements.md - Document inner prop, mode prop, and default prop semantics - Add PgRootCtx, SqliteRootCtx, MySqlRootCtx context types - Consolidate schema.md and module.md (remove duplication) - Add end-to-end pipeline walkthrough to README - Add glossary with 13 terms - Add error handling strategy - Remove duplicate content from hosts.md (cross-ref elements.md) --- docs/architecture/README.md | 103 +++++++++++++++- docs/architecture/decisions/001-ujsx-as-ir.md | 2 +- .../decisions/002-type-module-as-bundle.md | 2 +- .../decisions/003-hostconfig-for-dialects.md | 2 +- .../decisions/006-references-vs-fk.md | 53 ++++++++ .../decisions/007-inner-escape-hatch.md | 65 ++++++++++ .../decisions/008-pg-enum-predeclaration.md | 73 +++++++++++ docs/architecture/elements.md | 94 +++++++++++++-- docs/architecture/hosts.md | 113 +++++++----------- docs/architecture/module.md | 60 +++++----- docs/architecture/open-questions.md | 26 ++-- docs/architecture/schema.md | 104 +++------------- 12 files changed, 476 insertions(+), 221 deletions(-) create mode 100644 docs/architecture/decisions/006-references-vs-fk.md create mode 100644 docs/architecture/decisions/007-inner-escape-hatch.md create mode 100644 docs/architecture/decisions/008-pg-enum-predeclaration.md diff --git a/docs/architecture/README.md b/docs/architecture/README.md index 4655fc7..f89c701 100644 --- a/docs/architecture/README.md +++ b/docs/architecture/README.md @@ -1,6 +1,6 @@ --- status: draft -last_updated: 2026-05-22 +last_updated: 2026-05-23 --- # @alkdev/dbtype Architecture @@ -51,10 +51,10 @@ The gap: there is no schema-first, dialect-agnostic way to define a database sch | Document | Content | |----------|---------| -| [schema.md](schema.md) | Type.Module structure, element tree types, column type vocabulary, schema derivation | -| [hosts.md](hosts.md) | HostConfig implementations per dialect, symbolic defaults, Drizzle rendering | -| [elements.md](elements.md) | UJSX element definitions, function components, props schemas, common components | -| [module.md](module.md) | Type.Module as the bundle, incremental construction, serialization, migration diffing | +| [schema.md](schema.md) | Table/relation schema structure, schema derivation (insert/update/filter) | +| [module.md](module.md) | Type.Module mechanics: construction, validation, serialization, migration diffing | +| [elements.md](elements.md) | UJSX element definitions, column types, type definitions, defaults, function components | +| [hosts.md](hosts.md) | HostConfig implementations, column type mapping tables, rendering pipeline | | [repo-adapter.md](repo-adapter.md) | `from-dbtype` adapter for `@alkdev/operations`, filter/schema generation, access control | | [build-distribution.md](build-distribution.md) | Package structure, sub-path exports, dependencies, tree-shaking | | [open-questions.md](open-questions.md) | Cross-cutting unresolved questions | @@ -68,6 +68,9 @@ The gap: there is no schema-first, dialect-agnostic way to define a database sch | [003](decisions/003-hostconfig-for-dialects.md) | HostConfig for dialect rendering, not a transform registry | | [004](decisions/004-format-annotation-only.md) | Column formats are annotations, not validators — register explicitly | | [005](decisions/005-repo-as-adapter.md) | CRUD generation as an operations adapter, not a core feature | +| [006](decisions/006-references-vs-fk.md) | Column `references` as FK shorthand, explicit `` for complex FKs | +| [007](decisions/007-inner-escape-hatch.md) | Flat props with `inner` escape hatch for column validation | +| [008](decisions/008-pg-enum-predeclaration.md) | PG enum pre-declaration — return enums and tables from render context | ### Open Questions @@ -92,9 +95,65 @@ src/ from-dbtype.ts # FromDbType adapter for @alkdev/operations filters.ts # Filter schema generation per column type handlers.ts # CRUD handler generation (findMany, insertOne, etc.) - index.ts + index.ts ``` +## End-to-End Pipeline + +The dbtype pipeline has two walks over the same element tree: + +1. **Schema extraction** (`extractTable`) — walks the tree to produce TypeBox schemas and column metadata +2. **Host rendering** (`createRoot(host).render()`) — walks the tree to produce Drizzle table definitions + +These are separate walks because they serve different purposes: schema extraction feeds the Type.Module (for validation and serialization), while host rendering feeds Drizzle (for database operations). They consume the same element tree but produce different outputs. + +### Complete Flow + +```typescript +import { Type } from '@alkdev/typebox' +import { h, createComponent, createRoot } from '@alkdev/ujsx' +import { Value } from '@alkdev/typebox/value' + +// 1. Define elements +const UsersEl = h('table', { name: 'users' }, + h('column', { name: 'id', type: 'uuid', primaryKey: true, default: 'uuid' }), + h('column', { name: 'name', type: 'string', notNull: true }), +) + +// 2. Schema extraction — produces TypeBox schemas and column metadata +const { name, schema, columns, indexes, foreignKeys } = extractTable(UsersEl) + +// 3. Module construction — assemble into Type.Module +const defs = { Users: schema } +const M = Type.Module(defs) +const Users = M.Import('Users') + +// 4. Validation — check data against module schema +Value.Check(Users, { id: '...', name: 'alice' }) // true + +// 5. Host rendering — produce Drizzle tables +const root = createRoot(sqliteHost, {}) +root.render(UsersEl) +const drizzleUsers = root.ctx.tables.users // sqliteTable result + +// 6. Serialization — JSON Schema with $defs (for migration diffing) +const serialized = JSON.parse(JSON.stringify(Users)) + +// 7. Schema derivation — insert, update, filter schemas (for API validation) +defs.InsertUsers = Type.Object({ name: Type.String() }) // omit auto-generated columns +defs.UpdateUsers = Type.Partial(Type.Ref('Users')) // all optional +``` + +### Key Relationships + +| Step | Input | Output | Consumer | +|------|-------|--------|----------| +| Extract | UJSX elements | `TableMeta` (schema + metadata) | Module construction, Host rendering | +| Module | `Record` | `Type.Module` | Validation, Serialization | +| Import | Module key | `TImport` (resolved schema) | Value.Check, Value.Diff | +| Render | UJSX elements + HostConfig | Drizzle tables (in ctx) | Database queries | +| Derive | Module entries | Insert/Update/Partial schemas | API validation | + ## Key Design Decisions ### 1. UJSX Element Tree as the IR, Not a Separate Builder API @@ -131,6 +190,38 @@ For dbtype, formats serve as metadata that hosts can use (e.g., the PG host maps The repo pattern (auto-generated CRUD for each table) is not a core feature of dbtype. It's an adapter for `@alkdev/operations` that consumes the same element tree and module bundle. This keeps dbtype focused on schema definition and rendering, while the operations integration is a separate concern. +## Error Handling Strategy + +dbtype uses a layered validation approach: + +1. **TypeScript compile-time enforcement**: `DbColumnType`, column props, and element types are enforced by TypeScript types. Invalid `type` values, missing required props, and incorrect prop types are caught at compile time. + +2. **Runtime validation at extraction**: `extractTable()` validates the element tree — missing `name` or `type` on columns, duplicate column names, invalid `DbColumnType` values. These throw descriptive errors. + +3. **TypeBox validation at module compile**: `Type.Module(defs)` validates the schema map. Invalid `Type.Ref` targets, duplicate keys, and malformed schemas throw TypeBox errors. + +4. **Host rendering validation**: `createInstance()` in the host validates dialect-specific constraints — unknown column types fall back to `text()`, invalid symbolic defaults throw errors. PG enum names must be unique within a module. + +The general principle: **catch errors as early as possible**. Type errors at compile time, structural errors at extraction time, schema errors at module compile time, dialect errors at render time. + +## Glossary + +| Term | Definition | +|------|------------| +| **UJSX** | Universal JSX — `@alkdev/ujsx`'s element system. Uses `h()` and `createComponent` to create element trees that render to different hosts via `HostConfig`. Not an acronym, a project name. | +| **Element tree** | A tree of UJSX elements (``, ``, ``, ``) representing a database schema. The tree is the IR (intermediate representation). | +| **HostConfig** | UJSX's host configuration interface. Defines how elements map to output objects (`createInstance`, `appendChild`, `finalizeInstance`). Each dialect (SQLite, PG, MySQL) is a host. | +| **Type.Module** | `@alkdev/typebox`'s module system. Holds all schemas in a flat namespace with `Type.Ref` for cross-references. The module is dbtype's schema bundle. | +| **Type.Ref** | A TypeBox reference type that resolves within a `Type.Module`. Enables forward and circular references without import ordering issues. | +| **TImport** | The type returned by `M.Import(key)`. Embeds the full `$defs` namespace and resolves `$ref` pointers for validation. | +| **$defs** | JSON Schema keyword used by serialized Type.Modules. Contains all referenced schemas. `$ref` pointers reference entries in `$defs`. | +| **extractTable()** | Core function that walks an element tree, resolves function components, and produces `TableMeta` (TypeBox schema + column metadata + indexes + FKs). | +| **Function component** | A UJSX component created with `createComponent` that returns element(s). Transparent to the host — only resolved intrinsic elements (`column`, `table`) reach `createInstance`. | +| **Intrinsic element** | A built-in element type (`table`, `column`, `index`, `fk`). Not a function component — directly dispatched to the host's `createInstance`. | +| **Reconciler** | UJSX's diffing engine. Can compare old and new element trees to produce update instructions. Not used in phase 1, but positions dbtype for future schema migration support. | +| **Host rendering** | The process of walking an element tree through a `HostConfig` to produce output (Drizzle tables). Done via `createRoot(host).render(element)`. | +| **Schema derivation** | Producing insert, update, and filter schemas from the module. E.g., `Type.Partial(Type.Ref('Users'))` for update schemas. | + ## Document Lifecycle | From | To | Condition | diff --git a/docs/architecture/decisions/001-ujsx-as-ir.md b/docs/architecture/decisions/001-ujsx-as-ir.md index 004f240..f152a30 100644 --- a/docs/architecture/decisions/001-ujsx-as-ir.md +++ b/docs/architecture/decisions/001-ujsx-as-ir.md @@ -2,7 +2,7 @@ ## Status -Proposed +Accepted ## Context diff --git a/docs/architecture/decisions/002-type-module-as-bundle.md b/docs/architecture/decisions/002-type-module-as-bundle.md index 9004add..ea31f2d 100644 --- a/docs/architecture/decisions/002-type-module-as-bundle.md +++ b/docs/architecture/decisions/002-type-module-as-bundle.md @@ -2,7 +2,7 @@ ## Status -Proposed +Accepted ## Context diff --git a/docs/architecture/decisions/003-hostconfig-for-dialects.md b/docs/architecture/decisions/003-hostconfig-for-dialects.md index b00039d..4e342aa 100644 --- a/docs/architecture/decisions/003-hostconfig-for-dialects.md +++ b/docs/architecture/decisions/003-hostconfig-for-dialects.md @@ -2,7 +2,7 @@ ## Status -Proposed +Accepted ## Context diff --git a/docs/architecture/decisions/006-references-vs-fk.md b/docs/architecture/decisions/006-references-vs-fk.md new file mode 100644 index 0000000..9e47e20 --- /dev/null +++ b/docs/architecture/decisions/006-references-vs-fk.md @@ -0,0 +1,53 @@ +# ADR-006: Column `references` as FK Shorthand + +## Status + +Accepted + +## Context + +The `` element has a `references` prop (`string`) that specifies a foreign key target table name. There is also a separate `` element with full FK specification (`columns`, `references`, `foreignColumns`, `onDelete`, `onUpdate`). + +Two mechanisms exist for expressing foreign keys with different capabilities: +- `` — only specifies target table, no composite FK support, no ON DELETE/UPDATE +- `` — full FK specification with composite support and actions + +This creates ambiguity: which takes precedence? Can both be used simultaneously? What if they conflict? + +## Decision + +`references` on `` is a **shorthand for simple single-column foreign keys** that targets the referenced table's primary key by convention. `` is the **explicit form** for composite foreign keys and FKs with ON DELETE/ON UPDATE actions. + +Internally, `extractTable()` normalizes all `references` props into `FkMeta` entries alongside `` elements. When both `references` on a column and an explicit `` referencing the same column exist, the explicit `` takes precedence. + +## Rationale + +1. **Simple FKs are the common case**: Most foreign keys are single-column references to another table's primary key. `references="users"` covers this case ergonomically without requiring a separate `` element. + +2. **Composition is additive**: `references` props and `` elements both produce `FkMeta` entries. There's no separate representation for "simple" vs "complex" FKs in the output — they're unified. + +3. **Explicit overrides implicit**: When both forms specify the same relationship, the explicit `` form is more complete and should take precedence. This allows shorthand to be overridden without conflict. + +4. **Convention over configuration**: `references` assumes the target is the primary key. This convention covers 95% of FK relationships. For the remaining cases (non-PK targets, composite FKs), `` provides full control. + +5. **`extractTable()` normalizes**: Both forms produce the same `FkMeta` output, so downstream consumers (hosts, module, repo adapter) don't need to handle two different representations. + +## Consequences + +### Positive + +- Ergonomic shorthand for the most common FK pattern +- No ambiguity about precedence — explicit `` always wins +- Unified internal representation (`FkMeta`) regardless of authoring style +- `` maps to `{ columns: ['userId'], references: 'users', foreignColumns: ['id'] }` by convention + +### Negative + +- `references` on `` cannot specify target columns or ON DELETE/UPDATE — must use `` for those +- The convention that `references` targets the primary key must be documented and understood +- Two ways to express the same thing (simple FKs) — could confuse new users until they learn the convention + +## References + +- [elements.md](../elements.md) — column and fk element definitions +- [repo-adapter.md](../repo-adapter.md) — FK metadata consumption \ No newline at end of file diff --git a/docs/architecture/decisions/007-inner-escape-hatch.md b/docs/architecture/decisions/007-inner-escape-hatch.md new file mode 100644 index 0000000..2799d5f --- /dev/null +++ b/docs/architecture/decisions/007-inner-escape-hatch.md @@ -0,0 +1,65 @@ +# ADR-007: Flat Props with `inner` Escape Hatch for Column Validation + +## Status + +Accepted + +## Context + +Column elements carry both database metadata (notNull, primaryKey, default) and validation semantics (type, format, maxLength). The question is how to handle TypeBox validation constraints that don't have dedicated column props. + +Option A (flat props only) limits validation to what has explicit props: `format`, `maxLength`, `minLength`, `pattern`, etc. Each validation constraint requires a new column prop. Custom validation beyond these requires manipulating the module entry directly. + +Option B (flat props + `inner` escape hatch) uses flat props for the common 90% of cases and provides an `inner` prop that accepts a full TypeBox schema to override the auto-generated one. The host ignores `inner` — it's purely for the TypeBox schema. + +Option C (inner-first) makes the TypeBox schema primary and DB metadata secondary. This is close to the `DbTypeBuilder` pattern that ADR-001 specifically rejected. + +The research docs (`docs/research/architecture.md`) proposed `DbType.String({ notNull: true, inner: Type.String({ format: 'email', maxLength: 255 }) })`. With UJSX elements, this becomes ``. + +## Decision + +Use flat props for common cases with an `inner` escape hatch for custom TypeBox schemas (Option B). + +- Common validation constraints (`format`, `maxLength`, `minLength`, `pattern`, `values`) are top-level column props +- The `inner` prop accepts a TypeBox schema that overrides the auto-generated `colToTypeBox()` result +- When `inner` is provided, `extractTable()` uses it directly instead of calling `colToTypeBox(type, props)` +- The host ignores `inner` — database rendering uses only `type` and the other DB metadata props +- When `inner` is absent, `colToTypeBox(type, props)` generates the TypeBox schema from the column type and props + +## Rationale + +1. **Ergonomics for the common case**: Most columns only need `type`, `notNull`, `primaryKey`, `default`, and maybe `format`. Flat props (``) are concise and readable. + +2. **Escape hatch for complex validation**: Some columns need validation constraints that don't map to column props: `pattern`, `minimum`/`maximum` for numeric ranges, `contentMediaType` for embeddings, custom format validators. The `inner` prop lets you express any TypeBox validation without polluting the column props namespace. + +3. **Host independence**: The host renders columns to Drizzle based on `type` and DB metadata. `inner` doesn't affect rendering at all. This means adding `inner` doesn't require host changes — it's a pure TypeConcern. + +4. **Future-proofing**: Embedding vectors, custom types, and complex validation patterns will need `inner`. For instance, a vector column might be `` — the host still stores it as JSON, but the TypeBox schema validates the array structure. + +5. **Probe validated**: The probe scripts use flat props exclusively. `inner` adds a single conditional in `extractTable()` — if `inner` is provided, use it; otherwise, call `colToTypeBox()`. No structural change to the pipeline. + +6. **No alternative duplication**: Without `inner`, consumers who need `Type.String({ format: 'email', maxLength: 255, pattern: '^[a-z]' })` would have to manipulate the module entry after extraction, which defeats the purpose of a schema-first element tree. + +## Consequences + +### Positive + +- Flat props stay ergonomic for the 90% case +- Any TypeBox validation is expressible via `inner` without new column props +- Hosts don't need changes — `inner` is a TypeConcern only +- Future column types (vectors, custom point types) can embed complex validation +- Implementation cost is minimal: one conditional in `extractTable()` + +### Negative + +- Two ways to express validation (flat props vs `inner`) — could confuse new users +- `inner` TypeBox schemas are not reflected in column props — introspecting a column's validation requires looking at both `props` and `props.inner` +- The `colToTypeBox()` function must still exist for the no-`inner` case, and its output must be compatible with what `inner` would provide +- When `inner` is provided alongside flat validation props (`format`, `maxLength`), `inner` takes precedence. This must be clearly documented. + +## References + +- [elements.md](../elements.md) — Column element definition and `inner` prop +- [ADR-001](001-ujsx-as-ir.md) — UJSX as the IR (rejected separate builder API) +- [ADR-004](004-format-annotation-only.md) — Format as annotation (relevant: `format` is a flat prop, `inner` is for when it's not enough) +- Research: `docs/research/architecture.md` (DbTypeBuilder inner pattern) \ No newline at end of file diff --git a/docs/architecture/decisions/008-pg-enum-predeclaration.md b/docs/architecture/decisions/008-pg-enum-predeclaration.md new file mode 100644 index 0000000..16430ae --- /dev/null +++ b/docs/architecture/decisions/008-pg-enum-predeclaration.md @@ -0,0 +1,73 @@ +# ADR-008: PG Enum Pre-declaration — Return Both Enums and Tables + +## Status + +Accepted + +## Context + +PostgreSQL requires `pgEnum()` to be called at module scope before any table that references it. SQLite uses `text({ enum: [...] })` inline. MySQL uses `mysqlEnum()` inline. This creates a structural difference in the render output: PG enums must be declared separately. + +Three options were considered: + +**Option A**: Return both enums and tables from render. The PG host context accumulates enum declarations during the render walk and exposes them alongside tables. + +**Option B**: Use `text()` for all enums initially, add native PG enum support later as an opt-in. + +**Option C**: Per-column opt-in with `postgres: { nativeEnum: true }` — only generates `pgEnum` when explicitly requested. + +## Decision + +Use Option A: the PG host returns both enums and tables from the render context. + +The PG host context shape after rendering: + +```typescript +interface PgRootCtx { + dialect: 'pg' + tables: Record + enums: Record // Accumulated during render +} +``` + +When a `` is encountered, the PG host: +1. Registers the enum in `ctx.enums` (using the table and column name to derive a unique enum name) +2. Uses the registered enum in the column builder +3. The consumer includes both `ctx.enums` and `ctx.tables` in their Drizzle schema + +SQLite and MySQL hosts use `ctx.tables` only — no `enums` key. + +## Rationale + +1. **Correct PG behavior**: PG enums are a separate type declaration (`CREATE TYPE ... AS ENUM`). The render output must include them for the schema to be valid. + +2. **No information loss**: Option B (use `text()` for all) loses PG's native enum validation at the database level. The whole point of defining schema types is to get correct database behavior per dialect. + +3. **Clean API**: The host context already accumulates `tables`. Adding `enums` to the same context is natural and doesn't change the render pattern — it just adds a second accumulation target. + +4. **SQLite/MySQL unaffected**: These dialects don't have pre-declared enums. Their host contexts don't include `enums`, so the API is dialect-appropriate. + +5. **Consistent with resolved OQ-006**: The rendering pipeline accumulates results in `root.ctx`. PG enums are auxiliary state, just like tables. The context shape varies by dialect — this is expected. + +6. **Deriving enum names**: For a column `status` on table `users`, the PG enum name is `users_status` (table_column convention). This is the same convention Drizzle uses for inline enums. + +## Consequences + +### Positive + +- Correct PG enum support from day one +- No opt-in flag needed — enums just work when `type="enum"` is used with the PG host +- Dialect-appropriate context: PG has `enums + tables`, SQLite/MySQL have `tables` only +- Consistent with the context accumulation pattern + +### Negative + +- PG consumers must include `ctx.enums` in their schema file (alongside `ctx.tables`) +- The derived enum naming convention (`table_column`) could conflict with explicit enum names in the future — a naming override prop may be needed later +- Slightly more complex PG host implementation (tracking enum registrations during the walk) + +## References + +- [hosts.md](../hosts.md) — Host rendering pipeline and PG column type mapping +- [elements.md](../elements.md) — Column element `type="enum"` and `values` props +- [open-questions.md](../open-questions.md) — Original OQ-05 \ No newline at end of file diff --git a/docs/architecture/elements.md b/docs/architecture/elements.md index 5d5fa29..079c9a8 100644 --- a/docs/architecture/elements.md +++ b/docs/architecture/elements.md @@ -34,10 +34,10 @@ A single column definition within a table. | `notNull` | `boolean` | no | Column is NOT NULL | | `primaryKey` | `boolean` | no | Column is primary key | | `unique` | `boolean` | no | Column has UNIQUE constraint | -| `default` | `DbDefault \| unknown` | no | Symbolic default or literal value | -| `references` | `string` | no | FK target table name | +| `default` | `DbDefault \| unknown` | no | Symbolic default (`'now'`, `'uuid'`, `'autoincrement'`, `'current_timestamp'`), a literal value (string, number, boolean), or a Drizzle `sql` template expression. Symbolic defaults are resolved by the host to dialect-specific SQL. Literal values use `.default(value)`. SQL expressions use `.default(sql\`...\`)`. | +| `references` | `string` | no | FK target table name (shorthand — references the target table's primary key). Normalized to an `` entry internally by extractTable(). For composite FKs or FKs with ON DELETE/ON UPDATE, use the `` element instead. | | `format` | `string` | no | TypeBox format annotation (uuid, email, etc.) | -| `mode` | `'json' \| 'text'` | no | Storage mode for compound types | +| `mode` | `'json' \| 'text'` | no | Storage mode for compound types. `'json'` stores the value as JSON text (SQLite/MySQL) or JSONB (PG). `'text'` stores as plain text. Only meaningful for `type: 'json'`, `type: 'array'`, and `type: 'object'`. Defaults to `'json'` for those types. | | `values` | `string[]` | no | Enum values (for `type: 'enum'`) | | `length` | `number` | no | Max length (for varchar) | | `precision` | `number` | no | Numeric precision | @@ -45,6 +45,32 @@ A single column definition within a table. | `postgres` | `PgColumnOpts` | no | PG-specific overrides | | `sqlite` | `SqliteColumnOpts` | no | SQLite-specific overrides | | `mysql` | `MySqlColumnOpts` | no | MySQL-specific overrides | +| `inner` | `TSchema` | no | Override the auto-generated TypeBox schema. The host ignores this — it's purely for validation. When provided, `extractTable()` uses this schema directly instead of calling `colToTypeBox()`. | + +### Storage Mode + +The `mode` prop controls how compound types (json, array, object) are stored in the database: + +- `mode: 'json'` (default for json/array/object) — stored as JSON text in SQLite/MySQL, JSONB in PG +- `mode: 'text'` — stored as plain text, useful for columns that need JSON validation but plain-text storage + +For scalar types, `mode` has no effect. The host maps `mode: 'json'` to the appropriate dialect storage (e.g., `text({ mode: 'json' })` in SQLite, `jsonb()` in PG). + +### Default Values + +Column defaults support three categories: + +| Category | Example | Drizzle Output | +|----------|---------|----------------| +| Symbolic | `default: 'now'` | `.default(sql\`now()\`)` (PG) / `.default(sql\`(strftime('%s', 'now'))\`)` (SQLite) | +| Symbolic | `default: 'uuid'` | `.defaultRandom()` (PG) / `.$defaultFn(() => crypto.randomUUID())` (SQLite) | +| Symbolic | `default: 'autoincrement'` | Implicit on `INTEGER PRIMARY KEY` (SQLite) / `serial()` type (PG) | +| Literal | `default: true` | `.default(true)` | +| Literal | `default: 0` | `.default(0)` | +| SQL expression | `default: sql\`CURRENT_TIMESTAMP\`` | `.default(sql\`CURRENT_TIMESTAMP\`)` | +| JS function | `default: () => crypto.randomUUID()` | `.$defaultFn(() => crypto.randomUUID())` | + +Symbolic defaults are translated by the host. Literal and SQL expression defaults pass through directly. JS function defaults use Drizzle's `.$defaultFn()`. ### `` @@ -138,13 +164,7 @@ The `extractTable()` function walks an element tree, resolving function componen 3. **Table metadata**: `{ name, columns, indexes, foreignKeys }` for the host ```typescript -function extractTable(el: UElement): { - name: string - schema: TObject // TypeBox schema for Type.Module - columns: Record // Props + computed metadata - indexes: IndexMeta[] - foreignKeys: FkMeta[] -} +function extractTable(el: UElement): TableMeta ``` For each column, `extractTable`: @@ -157,11 +177,63 @@ For each column, `extractTable`: - **Column elements must have `name` and `type` props** — these are required for both TypeBox schema construction and Drizzle rendering - **Function components must return column elements** (or arrays of column elements) — returning other element types inside a table is undefined - **The `references` prop is metadata-only** — it's not part of the TypeBox schema. It informs the host about FK constraints and the repo adapter about relation structure +- **`references` is a shorthand** — a `` prop is syntactic sugar for `{columns: [column_name], references: 'users', foreignColumns: ['id']}` (targeting the referenced table's primary key by convention). Internally, `extractTable()` normalizes all `references` props into `FkMeta` entries alongside `` elements. When both `references` on a column and an explicit `` referencing the same column exist, the explicit `` takes precedence. - **`default` values can be symbolic strings or literals** — symbolic defaults (`now`, `uuid`, `autoincrement`) are resolved by the host; literal values pass through directly +- **The `inner` prop overrides the auto-generated TypeBox schema** — when `inner` is provided on a column element, `extractTable()` uses it directly without calling `colToTypeBox()`. The host ignores `inner` entirely — database rendering uses only `type` and DB metadata props. This is the escape hatch for validation constraints that don't have column prop shortcuts (e.g., `minItems`, `maximum`, `pattern`). See [ADR-007](decisions/007-inner-escape-hatch.md). + +## Type Definitions + +```typescript +/** Column metadata extracted from a element by extractTable() */ +interface ColumnMeta { + name: string + type: DbColumnType + notNull?: boolean + primaryKey?: boolean + unique?: boolean + default?: DbDefault | unknown + references?: string // Shorthand: FK target table name + format?: string // TypeBox format annotation + mode?: 'json' | 'text' // Storage mode for compound types + values?: string[] // Enum values (for type: 'enum') + length?: number // Max length (for varchar) + precision?: number // Numeric precision + scale?: number // Numeric scale + postgres?: PgColumnOpts // PG-specific overrides + sqlite?: SqliteColumnOpts // SQLite-specific overrides + mysql?: MySqlColumnOpts // MySQL-specific overrides + inner?: TSchema // Override TypeBox schema (escape hatch) +} + +/** Index metadata extracted from an element */ +interface IndexMeta { + name: string + columns: string[] + unique?: boolean +} + +/** Foreign key metadata extracted from elements and references props */ +interface FkMeta { + columns: string[] // Local column names + references: string // Target table name + foreignColumns: string[] // Target column names + onDelete?: 'cascade' | 'set null' | 'restrict' | 'no action' + onUpdate?: 'cascade' | 'set null' | 'restrict' | 'no action' +} + +/** Table metadata result from extractTable() */ +interface TableMeta { + name: string + schema: TObject // TypeBox schema for Type.Module entry + columns: Record + indexes: IndexMeta[] + foreignKeys: FkMeta[] +} +``` ## Open Questions -1. **Should column elements support `inner` TypeBox schemas?** The research docs proposed `DbType.String({ notNull: true, inner: Type.String({ format: 'email', maxLength: 255 }) })`. With UJSX elements, this would be ``. Is the flat props model sufficient, or do we need nested TypeBox schemas? +1. ~~Should column elements support `inner` TypeBox schemas?~~ **Resolved — Yes. ADR-007: flat props for common cases, `inner` as escape hatch for custom validation.** The `inner` prop is now documented in the column props table. 2. **Should `
` accept `extraConfig` props?** Drizzle tables accept a third callback argument for indexes and unique constraints defined in terms of column references. How does this map to element props? diff --git a/docs/architecture/hosts.md b/docs/architecture/hosts.md index 39a7933..fa1ef0b 100644 --- a/docs/architecture/hosts.md +++ b/docs/architecture/hosts.md @@ -32,6 +32,22 @@ interface DbtypeColumnInst { builder: any // Drizzle column builder dbMeta: Record } + +interface PgRootCtx { + dialect: 'pg' + tables: Record + enums: Record // Accumulated during render +} + +interface SqliteRootCtx { + dialect: 'sqlite' + tables: Record +} + +interface MySqlRootCtx { + dialect: 'mysql' + tables: Record +} ``` The `HostConfig` generic parameters: @@ -41,90 +57,43 @@ The `HostConfig` generic parameters: ## Column Type Mapping -### SQLite - -| Column Type Prop | Drizzle Builder | Notes | -|-----------------|----------------|-------| -| `uuid` | `text(name).primaryKey().$defaultFn(crypto.randomUUID)` | No native UUID type | -| `string` | `text(name)` | | -| `integer` | `integer(name)` | | -| `boolean` | `integer(name, { mode: 'boolean' })` | | -| `timestamp` | `integer(name, { mode: 'timestamp' })` | | -| `enum` | `text(name, { enum: values })` | SQLite has no native enum | -| `json` | `text(name, { mode: 'json' })` | Stored as JSON text | - -### PostgreSQL - -| Column Type Prop | Drizzle Builder | Notes | -|-----------------|----------------|-------| -| `uuid` | `uuid(name).defaultRandom()` or `.primaryKey()` | Native UUID | -| `string` | `text(name)` or `varchar(name, n)` | | -| `integer` | `integer(name)` | | -| `boolean` | `boolean(name)` | | -| `timestamp` | `timestamptz(name, { withTimezone: true })` | | -| `enum` | `pgEnum(name, values)` | Requires pre-declaration | -| `json` | `jsonb(name)` | | - -### MySQL - -| Column Type Prop | Drizzle Builder | Notes | -|-----------------|----------------|-------| -| `uuid` | `varchar(name, { length: 36 })` | No native UUID | -| `string` | `text(name)` or `varchar(name, n)` | | -| `integer` | `int(name)` | | -| `boolean` | `boolean(name)` or `tinyint(name)` | | -| `timestamp` | `timestamp(name)` | | -| `enum` | `mysqlEnum(name, values)` | | -| `json` | `json(name)` | | +| DbColumnType | SQLite | PostgreSQL | MySQL | TypeBox | Notes | +|---|---|---|---|---|---| +| `uuid` | `text(name).primaryKey().$defaultFn(crypto.randomUUID)` | `uuid(name).defaultRandom()` / `.primaryKey()` | `varchar(name, { length: 36 })` | `Type.String({ format: 'uuid' })` | No native UUID in SQLite/MySQL | +| `string` | `text(name)` | `text(name)` | `text(name)` | `Type.String()` | General unbounded text; for bounded strings use `varchar` | +| `text` | `text(name)` | `text(name)` | `text(name)` | `Type.String()` | Explicitly unbounded text content | +| `varchar` | `text(name)` | `varchar(name, { length })` | `varchar(name, { length })` | `Type.String({ maxLength })` | Maps to `text()` in SQLite (no varchar); PG/MySQL have native varchar with length | +| `integer` | `integer(name)` | `integer(name)` | `int(name)` | `Type.Integer()` | | +| `bigint` | `blob(name, { mode: 'bigint' })` | `bigint(name)` | `bigint(name)` | custom `TypeBigInt()` | SQLite uses blob with bigint mode; TypeBox has no standard BigInt type, needs annotation | +| `boolean` | `integer(name, { mode: 'boolean' })` | `boolean(name)` | `boolean(name)` / `tinyint(name)` | `Type.Boolean()` | SQLite has no native boolean type | +| `timestamp` | `integer(name, { mode: 'timestamp' })` | `timestamptz(name, { withTimezone: true })` | `timestamp(name)` | `Type.Number()` | | +| `real` | `real(name)` | `real(name)` | `float(name)` | `Type.Number()` | SQLite/PG use `real()`; MySQL uses `float()`; all map to `Type.Number()` | +| `numeric` | `numeric(name)` | `numeric(name, { precision, scale })` | `decimal(name, { precision, scale })` | `Type.String()` | Stored as string; precision/scale from column props | +| `enum` | `text(name, { enum: values })` | `pgEnum(name, values)` | `mysqlEnum(name, values)` | `Type.Union([...])` | SQLite has no native enum; PG requires pre-declaration | +| `json` | `text(name, { mode: 'json' })` | `jsonb(name)` | `json(name)` | `Type.Any()` | Stored as JSON text; PG uses `jsonb` by default | +| `array` | N/A (not supported) | `column(name).array()` | N/A (not supported) | `Type.Array(inner)` | Only PG supports native arrays; other dialects should use json mode; requires special handling | +| `object` | `text(name, { mode: 'json' })` | `jsonb(name)` | `json(name)` | `Type.Object(properties)` | Always stored as JSON; same as `json` type but with structured inner schema | ## Symbolic Defaults -Column props like `default="now"` and `default="uuid"` are translated to dialect-specific SQL or JS functions by the host: - -| Symbol | SQLite | PostgreSQL | MySQL | -|--------|--------|------------|-------| -| `now` | `sql\`(strftime('%s', 'now'))\`` | `sql\`now()\`` | `sql\`NOW()\`` | -| `uuid` | `.$defaultFn(() => crypto.randomUUID())` | `.defaultRandom()` | `.$defaultFn(() => crypto.randomUUID())` | -| `autoincrement` | Implicit on `INTEGER PRIMARY KEY` | `serial()` type | `.autoincrement()` | +Symbolic defaults (`'now'`, `'uuid'`, `'autoincrement'`, `'current_timestamp'`) are resolved to dialect-specific SQL or JS functions by the host. See [elements.md](elements.md#default-values) for the full default value specification. ## Common Column Components -Function components that compose into any table: - -```typescript -const IdColumn = createComponent('IdColumn', () => - h('column', { name: 'id', type: 'uuid', primaryKey: true, default: 'uuid' }) -) - -const AuditColumns = createComponent('AuditColumns', () => [ - h('column', { name: 'createdAt', type: 'timestamp', notNull: true, default: 'now' }), - h('column', { name: 'updatedAt', type: 'timestamp', notNull: true, default: 'now' }), -]) -``` - -Used by spreading into a table: - -```typescript -const UsersEl = h('table', { name: 'users' }, - h(IdColumn, {}), - h('column', { name: 'name', type: 'string', notNull: true }), - h(AuditColumns, {}), -) -``` - -This replaces the storage_sqlite pattern of `const commonCols = { id: text('id').primaryKey(), ... }` with reusable, dialect-agnostic components. +Function components like `IdColumn` and `AuditColumns` compose into any table. See [elements.md](elements.md#function-components) for component definitions. ## Rendering Pipeline 1. **Walk elements**: The host's `createInstance` method receives each `
`, ``, ``, `` element 2. **Build columns**: `createInstance('column', props, ctx)` maps `props.type` to the dialect-specific Drizzle builder, applying `notNull`, `primaryKey`, `unique`, `default` constraints 3. **Assemble tables**: `appendChild(tableInst, columnInst, ctx)` attaches column builders to the table's columns record -4. **Finalize**: `finalizeInstance(tableInst, ctx)` calls `sqliteTable(name, columns)` (or `pgTable`, `mysqlTable`) and stores the result +4. **Finalize**: `finalizeInstance(tableInst, ctx)` calls `sqliteTable(name, columns)` (or `pgTable`, `mysqlTable`) and stores the result on the table instance ```typescript const root = createRoot(sqliteHost, {}) root.render(UsersEl) -// root's context now contains: { tables: { users: drizzleSqliteTable } } +// root.ctx.tables: { users: drizzleSqliteTable } — accessible after render +// For PG hosts, root.ctx also accumulates auxiliary state like pgEnum declarations ``` ## Constraints @@ -132,16 +101,16 @@ root.render(UsersEl) - **`createTextInstance` is not supported** — DB schemas are purely structural, no text nodes - **Column type props must be from the `DbColumnType` vocabulary** — hosts map these to dialect-specific builders; unknown types fall back to `text()` - **Symbolic defaults are resolved by the host** — `default="now"` becomes `sql\`(strftime('%s', 'now'))\`` on SQLite and `sql\`now()\`` on PG -- **PG enums require pre-declaration** — the host must track enum types and emit `pgEnum()` calls before tables that reference them +- **PG enums are accumulated in context** — when a `` is encountered, the PG host registers the enum in `ctx.enums` (using `table_column` naming convention) and references it in the column builder. The consumer includes both `ctx.enums` and `ctx.tables` in their Drizzle schema. See [ADR-008](decisions/008-pg-enum-predeclaration.md). - **A host renders one dialect at a time** — to generate schemas for multiple dialects, render multiple times with different hosts ## Open Questions -1. **How to handle PG enum pre-declaration?** PG requires `pgEnum()` at module scope before tables. Options: (A) return both enums and tables from render, (B) start with text for all enums, (C) per-column opt-in. Leaning toward A. +1. **[RESOLVED — ADR-008](decisions/008-pg-enum-predeclaration.md) PG enum pre-declaration**: PG requires `pgEnum()` at module scope before tables. The host accumulates enum declarations in `root.ctx.enums` during the render walk, and `finalizeRoot` emits them. This is option A — enum state lives in context alongside tables. -2. **Should hosts return the rendered table or store it in context?** The probe scripts use context (`ctx.tables`), but returning from render would be more functional. Need to resolve this. +2. **[Resolved] render() return value vs context**: `render()` on the UJSX `Root` returns `void` — the rendered Drizzle tables are collected in `root.ctx.tables`. Context is the right home for this because PG hosts need auxiliary state (enum declarations) alongside the primary output. The context shape varies by dialect: SQLite uses `{ dialect: 'sqlite', tables: Record }`, PG uses `{ dialect: 'pg', enums: Record, tables: Record }`. -3. **`prepareUpdate`/`commitUpdate` for migrations?** The UJSX reconciler could diff old and new element trees to produce ALTER TABLE statements. This is a future feature, not phase 1. +3. **`prepareUpdate`/`commitUpdate` for migrations**: The UJSX reconciler's `prepareUpdate`/`commitUpdate` hooks could diff old and new element trees to produce ALTER TABLE statements. This is out of scope for phase 1, but the reconciler architecture positions us for it in a future phase. ## References diff --git a/docs/architecture/module.md b/docs/architecture/module.md index ffedcc1..f38dfc4 100644 --- a/docs/architecture/module.md +++ b/docs/architecture/module.md @@ -1,17 +1,11 @@ --- -status: draft -last_updated: 2026-05-22 +status: stable +last_updated: 2026-05-23 --- -# Module: Type.Module as the Schema Bundle +# Module: Type.Module Mechanical Reference -Technical details on how dbtype uses `Type.Module` for schema construction, validation, serialization, and migration. - -## Overview - -dbtype uses `@alkdev/typebox`'s `Type.Module` as the schema storage and resolution mechanism. A module holds all table schemas, their relations, and derived schemas (insert, update, partial) in a single flat namespace. `Type.Ref` resolves cross-table references — including circular ones — without import ordering issues. - -This document covers the mechanics, constraints, and patterns discovered during architecture probing. +How `Type.Module` works mechanically — construction, validation, serialization, migration diffing, cross-module references, and constraints. For the domain-specific content (what goes into a dbtype module, schema derivation semantics, relations), see [schema.md](schema.md). ## Construction Patterns @@ -27,6 +21,27 @@ const M = Type.Module(defs) const Users = M.Import('Users') ``` +### Incremental Construction + +The defs map is a plain `Record` — it can be built incrementally, mutated, and extended before compilation: + +```typescript +const defs: Record = {} +defs.Users = extractTableSchema(UsersElement) +defs.Tasks = extractTableSchema(TasksElement) + +// Add a column later +defs.Users = Type.Object({ ...defs.Users.properties, role: Type.String() }) + +// Add relations +defs.UsersRelations = Type.Object({ tasks: Type.Array(Type.Ref('Tasks')) }) + +// Compile when ready +const M = Type.Module(defs) +``` + +Once compiled, mutations to the original defs map don't affect the compiled module. + ### With Relations ```typescript @@ -43,24 +58,6 @@ defs.InsertUsers = Type.Object({ name: Type.String(), email: Type.String() }) / defs.UpdateUsers = Type.Partial(Type.Ref('Users')) // computed ``` -### Incremental Construction - -```typescript -// Build defs incrementally -const defs: Record = {} -defs.Users = extractTableSchema(UsersElement) -defs.Tasks = extractTableSchema(TasksElement) - -// Add a column later -defs.Users = Type.Object({ ...defs.Users.properties, role: Type.String() }) - -// Add relations -defs.UsersRelations = Type.Object({ tasks: Type.Array(Type.Ref('Tasks')) }) - -// Compile when ready -const M = Type.Module(defs) -``` - ## Validation ### Format Registration Required @@ -107,13 +104,13 @@ for (const err of Value.Errors(Users, badData)) { ... } Key properties: - Each `$defs` entry has an `$id` matching its key -- `Type.Ref` remains as `{ "$ref": "Key" }` — not inlined +- `Type.Ref` remains as `{ "$ref": "Key" }` — not inlined; consumers must resolve them - The entire structure is valid JSON Schema - All entries in the module are present in `$defs` (even if only one was imported) ### Roundtrip -The serialized form can be parsed back into a schema-like structure. `Value.Diff` works on these serialized objects to produce structural edit lists. +The serialized form can be parsed back into a schema-like structure. `Value.Diff` works on these serialized objects to produce structural edit lists. Note that Symbol properties (`[Kind]`, `[Hint]`, etc.) are stripped by `JSON.stringify` — the serialized form is JSON Schema, not TypeBox schema. Roundtripping requires `FromSchema` or reconstructed TypeBox objects. ## Migration Diffing @@ -160,7 +157,10 @@ However, this nests `$defs` within `$defs` (the User's `$defs` contains CommonUu - **Module entries are computed at construction time** — `Type.Partial(Type.Ref('Users'))` is resolved when the module is built, producing a concrete optional-property object - **`Type.Ref` outside a module has `static: unknown`** — always use `M.Import(key)` for proper type inference - **Module keys are a flat namespace** — no nested paths like `"tables/Users"`. Table names must be unique within the module. +- **`Type.Ref` resolves within the module only** — no cross-module references without `Module.Import` - **`Module.Import` embeds all `$defs`** — every import carries the full module. This is correct for validation but increases JSON Schema size. +- **Defs map is mutable until compiled** — once passed to `Type.Module`, mutations to the original map don't affect the compiled module +- **Format validation requires `FormatRegistry.Set`** — `uuid`, `email`, and other custom formats must be registered before `Value.Check` will enforce them - **Symbol properties are lost in `JSON.stringify`** — `[Kind]`, `[Hint]`, etc. are stripped. The serialized form is JSON Schema, not TypeBox schema. Roundtripping requires `FromSchema` or reconstructed TypeBox objects. ## References diff --git a/docs/architecture/open-questions.md b/docs/architecture/open-questions.md index e787ce7..1197eab 100644 --- a/docs/architecture/open-questions.md +++ b/docs/architecture/open-questions.md @@ -1,6 +1,6 @@ --- status: draft -last_updated: 2026-05-22 +last_updated: 2026-05-23 --- # Open Questions Tracker @@ -37,24 +37,26 @@ All unresolved architectural questions for dbtype, organized by theme. ### OQ-04: Should column elements support nested TypeBox schemas? - **Origin**: [elements.md](elements.md) -- **Status**: Open -- **Priority**: High -- **Context**: The research docs proposed `DbType.String({ notNull: true, inner: Type.String({ format: 'email', maxLength: 255 }) })`. With UJSX elements, this would be ``. The flat props model works for common cases, but custom validation constraints (patterns, ranges, custom checks) may need a nested `inner` prop. Should we support ``? -- **Cross-references**: ADR-001 +- **Status**: Resolved +- **Priority**: ~~High~~ Resolved +- **Context**: The research docs proposed `DbType.String({ notNull: true, inner: Type.String({ format: 'email', maxLength: 255 }) })`. With UJSX elements, this would be ``. The flat props model works for common cases, but custom validation constraints (patterns, ranges, custom checks) may need a nested `inner` prop. +- **Resolution**: Yes — flat props for common cases, `inner` as escape hatch for custom validation. When `inner` is provided, `extractTable()` uses it directly instead of calling `colToTypeBox()`. The host ignores `inner` — it's purely for the TypeBox schema. See [ADR-007](decisions/007-inner-escape-hatch.md). ### OQ-05: How to handle PG enum pre-declaration? - **Origin**: [hosts.md](hosts.md) -- **Status**: Open -- **Priority**: High -- **Context**: PG requires `pgEnum()` at module scope before tables that reference it. Options: (A) return both enums and tables from render, (B) start with text for all enums, (C) per-column opt-in with `postgres: { nativeEnum: true }`. Option A is cleanest but changes the render API. Option B is simplest but loses PG enum validation. +- **Status**: Resolved +- **Priority**: ~~High~~ Resolved +- **Context**: PG requires `pgEnum()` at module scope before tables that reference it. Options: (A) return both enums and tables from render, (B) start with text for all enums, (C) per-column opt-in with `postgres: { nativeEnum: true }`. +- **Resolution**: Option A — the PG host accumulates enums in `ctx.enums` during the render walk. The consumer includes both `ctx.enums` and `ctx.tables` in their Drizzle schema. Enum names follow the `table_column` convention. See [ADR-008](decisions/008-pg-enum-predeclaration.md). ### OQ-06: Should hosts return rendered tables or store them in context? - **Origin**: [hosts.md](hosts.md) -- **Status**: Open +- **Status**: Resolved - **Priority**: Medium - **Context**: The probe scripts use `ctx.tables` to collect rendered tables. A more functional approach would have `render()` return the rendered table directly. Need to resolve this before implementation. +- **Resolution**: render() accumulates results in root.ctx (tables, enums). The context shape varies by dialect — SQLite has ctx.tables, PG has ctx.tables and ctx.enums. See hosts.md Rendering Pipeline. ### OQ-07: Should we support JSX file extensions? @@ -95,9 +97,9 @@ All unresolved architectural questions for dbtype, organized by theme. | OQ-01 | Relation naming convention | schema.md | Medium | Open | | OQ-02 | Derived schemas in module or separate | schema.md | Low | Open | | OQ-03 | Multiple database namespaces | schema.md | Low | Open | -| OQ-04 | Nested TypeBox schemas in column props | elements.md | High | Open | -| OQ-05 | PG enum pre-declaration | hosts.md | High | Open | -| OQ-06 | Host render return value vs context | hosts.md | Medium | Open | +| OQ-04 | Nested TypeBox schemas in column props | elements.md | ~~High~~ Resolved | **Resolved** — ADR-007 | +| OQ-05 | PG enum pre-declaration | hosts.md | ~~High~~ Resolved | **Resolved** — ADR-008 | +| OQ-06 | Host render return value vs context | hosts.md | Medium | Resolved | | OQ-07 | JSX file extensions | elements.md | Low | Open | | OQ-08 | Per-dialect handler differences | repo-adapter.md | Medium | Open | | OQ-09 | Relation rendering responsibility | repo-adapter.md | Medium | Open | diff --git a/docs/architecture/schema.md b/docs/architecture/schema.md index 9a70969..dfc5666 100644 --- a/docs/architecture/schema.md +++ b/docs/architecture/schema.md @@ -1,21 +1,13 @@ --- -status: draft -last_updated: 2026-05-22 +status: stable +last_updated: 2026-05-23 --- -# Schema: Type.Module as the Schema Bundle +# Schema: Domain Data Inside the Module -How dbtype uses `Type.Module` to store all table schemas, relations, and derived schemas in a single namespace with automatic `Type.Ref` resolution. +What goes into a dbtype `Type.Module` — table schemas, relation entries, derived schemas (insert, update, filter), and the schema derivation semantics. For the mechanical details of how `Type.Module` works (construction, validation, serialization, migration diffing, constraints), see [module.md](module.md). -## Overview - -The `Type.Module` is the central data structure in dbtype. It holds every table's TypeBox schema, all cross-table relations, and derived schemas (insert, update, select variants) in one flat namespace. `Type.Ref` resolves forward and circular references naturally, eliminating the need for separate relation files or import-order management. - -The module is also the serialization boundary: `JSON.stringify(module.Import('Users'))` produces valid JSON Schema with `$defs`, enabling migration diffing via `Value.Diff`. - -## Construction - -### From Element Tree to Module +## From Element Tree to Module The element tree (`
`, ``) is walked to extract a `Record` map, then compiled into a module: @@ -34,29 +26,24 @@ Each `` element produces a TypeBox type based on its `type` prop: | `timestamp` | `Type.Number()` | | `enum` | `Type.Union(values.map(v => Type.Literal(v)))` | -### Incremental Construction +For incremental construction patterns and compilation mechanics, see [module.md](module.md). -The defs map is a plain `Record` — it can be built incrementally, mutated, and extended before compilation: +## Relations + +Relations are stored as separate entries in the module, using `Type.Ref` to reference other tables: ```typescript -const defs: Record = {} - -// Add tables one at a time -defs.Users = Type.Object({ id: Type.String({ format: 'uuid' }), name: Type.String() }) -defs.Tasks = Type.Object({ id: Type.String({ format: 'uuid' }), userId: Type.String({ format: 'uuid' }), title: Type.String() }) - -// Add columns to an existing table -defs.Users = Type.Object({ ...defs.Users.properties, role: Type.String() }) - -// Add relations defs.UsersRelations = Type.Object({ tasks: Type.Array(Type.Ref('Tasks')) }) defs.TasksRelations = Type.Object({ user: Type.Ref('Users') }) - -// Compile -const M = Type.Module(defs) ``` -Once compiled, `M.Import(key)` returns a `TImport` schema with the full `$defs` namespace embedded. +This gives: +- **Type-safe validation**: `Value.Check(M.Import('UsersRelations'), { tasks: [...] })` validates the full nested structure +- **No circular import issues**: `Type.Ref` resolves within the module namespace regardless of definition order +- **Queryable structure**: The `$defs` map is enumerable — you can find all relations for a table by naming convention +- **Drizzle integration**: The repo adapter reads relation entries to generate `relations()` calls for drizzle's relational query builder + +Foreign key metadata lives on the column element's `references` prop (``), not in the relation entry. Relations describe the "from this side, I see many of those" semantics. ## Schema Derivation @@ -93,64 +80,6 @@ defs.UpdateUsers = Type.Partial(Type.Ref('Users')) Per-column comparison operators derived from the column type. Generated by the repo adapter, not the core module. -## Relations - -Relations are stored as separate entries in the module, using `Type.Ref` to reference other tables: - -```typescript -defs.UsersRelations = Type.Object({ tasks: Type.Array(Type.Ref('Tasks')) }) -defs.TasksRelations = Type.Object({ user: Type.Ref('Users') }) -``` - -This gives: -- **Type-safe validation**: `Value.Check(M.Import('UsersRelations'), { tasks: [...] })` validates the full nested structure -- **No circular import issues**: `Type.Ref` resolves within the module namespace regardless of definition order -- **Queryable structure**: The `$defs` map is enumerable — you can find all relations for a table by naming convention -- **Drizzle integration**: The repo adapter reads relation entries to generate `relations()` calls for drizzle's relational query builder - -Foreign key metadata lives on the column element's `references` prop (``), not in the relation entry. Relations describe the "from this side, I see many of those" semantics. - -## Serialization - -`JSON.stringify(M.Import('TableName'))` produces JSON Schema with `$defs`: - -```json -{ - "$defs": { - "Users": { "$id": "Users", "type": "object", "properties": { ... } }, - "Tasks": { "$id": "Tasks", "type": "object", "properties": { ... } }, - "UsersRelations": { "$id": "UsersRelations", "type": "object", "properties": { "tasks": { "items": { "$ref": "Tasks" }, "type": "array" } } } - }, - "$ref": "Users" -} -``` - -Key properties: -- Each `$defs` entry gets an `$id` matching its key name -- `Type.Ref` leaves `$ref` pointers (not inlined) — consumers must resolve them -- The serialized form is valid JSON Schema -- `Value.Diff` produces structural edits between two serialized schemas (useful for migration diffing) - -## Migration Diffing - -```typescript -const v1 = JSON.parse(JSON.stringify(M.Import('Users'))) -// ... add a column to defs.Users ... -const v2 = JSON.parse(JSON.stringify(M2.Import('Users'))) -const edits = Value.Diff(v1, v2) -// edits: [{ type: 'insert', path: '/$defs/Users/properties/role', value: { type: 'string' } }, ...] -``` - -The edits use JSON Pointer paths, which can be translated to `ALTER TABLE ADD COLUMN` statements. - -## Constraints - -- **Module keys must be unique** — two tables cannot have the same name in the same module -- **`Type.Ref` resolves within the module only** — no cross-module references without `Module.Import` -- **`Type.Ref` outside a module has `static: unknown`** — always access via `M.Import(key)` for proper type inference -- **Defs map is mutable until compiled** — once passed to `Type.Module`, mutations to the original map don't affect the compiled module -- **Format validation requires `FormatRegistry.Set`** — `uuid`, `email`, and other custom formats must be registered before `Value.Check` will enforce them - ## Open Questions 1. **Should relation entries use a naming convention?** Currently `UsersRelations` / `TasksRelations`. Is this sufficient, or should relations be structured differently (e.g., a `relations` field on the table entry)? @@ -162,5 +91,6 @@ The edits use JSON Pointer paths, which can be translated to `ALTER TABLE ADD CO ## References - TypeBox Module API: `@alkdev/typebox` source — `type/module/module.ts`, `type/ref/ref.ts` +- Mechanical details: [module.md](module.md) - Probe validation: `scripts/probe-e2e.ts` - Research: `docs/research/architecture.md` \ No newline at end of file