docs: resolve architecture open questions, add type definitions, consolidate docs
Architecture review session resolving all high-priority open questions and filling documentation gaps identified during review: Decisions resolved: - OQ-04: Flat props with inner escape hatch for column validation (ADR-007) - OQ-05: PG enum pre-declaration returns enums and tables (ADR-008) - OQ-06: Render results accumulate in root.ctx (resolved in hosts.md) - Column references vs fk: references is shorthand, explicit fk takes precedence (ADR-006) - ADR-001, 002, 003 promoted from Proposed to Accepted (probe-validated) Documentation improvements: - Complete DbColumnType mapping tables for all 14 types across 3 dialects - Define ColumnMeta, TableMeta, IndexMeta, FkMeta types in elements.md - Document inner prop, mode prop, and default prop semantics - Add PgRootCtx, SqliteRootCtx, MySqlRootCtx context types - Consolidate schema.md and module.md (remove duplication) - Add end-to-end pipeline walkthrough to README - Add glossary with 13 terms - Add error handling strategy - Remove duplicate content from hosts.md (cross-ref elements.md)
This commit is contained in:
@@ -1,6 +1,6 @@
|
||||
---
|
||||
status: draft
|
||||
last_updated: 2026-05-22
|
||||
last_updated: 2026-05-23
|
||||
---
|
||||
|
||||
# @alkdev/dbtype Architecture
|
||||
@@ -51,10 +51,10 @@ The gap: there is no schema-first, dialect-agnostic way to define a database sch
|
||||
|
||||
| Document | Content |
|
||||
|----------|---------|
|
||||
| [schema.md](schema.md) | Type.Module structure, element tree types, column type vocabulary, schema derivation |
|
||||
| [hosts.md](hosts.md) | HostConfig implementations per dialect, symbolic defaults, Drizzle rendering |
|
||||
| [elements.md](elements.md) | UJSX element definitions, function components, props schemas, common components |
|
||||
| [module.md](module.md) | Type.Module as the bundle, incremental construction, serialization, migration diffing |
|
||||
| [schema.md](schema.md) | Table/relation schema structure, schema derivation (insert/update/filter) |
|
||||
| [module.md](module.md) | Type.Module mechanics: construction, validation, serialization, migration diffing |
|
||||
| [elements.md](elements.md) | UJSX element definitions, column types, type definitions, defaults, function components |
|
||||
| [hosts.md](hosts.md) | HostConfig implementations, column type mapping tables, rendering pipeline |
|
||||
| [repo-adapter.md](repo-adapter.md) | `from-dbtype` adapter for `@alkdev/operations`, filter/schema generation, access control |
|
||||
| [build-distribution.md](build-distribution.md) | Package structure, sub-path exports, dependencies, tree-shaking |
|
||||
| [open-questions.md](open-questions.md) | Cross-cutting unresolved questions |
|
||||
@@ -68,6 +68,9 @@ The gap: there is no schema-first, dialect-agnostic way to define a database sch
|
||||
| [003](decisions/003-hostconfig-for-dialects.md) | HostConfig for dialect rendering, not a transform registry |
|
||||
| [004](decisions/004-format-annotation-only.md) | Column formats are annotations, not validators — register explicitly |
|
||||
| [005](decisions/005-repo-as-adapter.md) | CRUD generation as an operations adapter, not a core feature |
|
||||
| [006](decisions/006-references-vs-fk.md) | Column `references` as FK shorthand, explicit `<fk>` for complex FKs |
|
||||
| [007](decisions/007-inner-escape-hatch.md) | Flat props with `inner` escape hatch for column validation |
|
||||
| [008](decisions/008-pg-enum-predeclaration.md) | PG enum pre-declaration — return enums and tables from render context |
|
||||
|
||||
### Open Questions
|
||||
|
||||
@@ -92,9 +95,65 @@ src/
|
||||
from-dbtype.ts # FromDbType adapter for @alkdev/operations
|
||||
filters.ts # Filter schema generation per column type
|
||||
handlers.ts # CRUD handler generation (findMany, insertOne, etc.)
|
||||
index.ts
|
||||
index.ts
|
||||
```
|
||||
|
||||
## End-to-End Pipeline
|
||||
|
||||
The dbtype pipeline has two walks over the same element tree:
|
||||
|
||||
1. **Schema extraction** (`extractTable`) — walks the tree to produce TypeBox schemas and column metadata
|
||||
2. **Host rendering** (`createRoot(host).render()`) — walks the tree to produce Drizzle table definitions
|
||||
|
||||
These are separate walks because they serve different purposes: schema extraction feeds the Type.Module (for validation and serialization), while host rendering feeds Drizzle (for database operations). They consume the same element tree but produce different outputs.
|
||||
|
||||
### Complete Flow
|
||||
|
||||
```typescript
|
||||
import { Type } from '@alkdev/typebox'
|
||||
import { h, createComponent, createRoot } from '@alkdev/ujsx'
|
||||
import { Value } from '@alkdev/typebox/value'
|
||||
|
||||
// 1. Define elements
|
||||
const UsersEl = h('table', { name: 'users' },
|
||||
h('column', { name: 'id', type: 'uuid', primaryKey: true, default: 'uuid' }),
|
||||
h('column', { name: 'name', type: 'string', notNull: true }),
|
||||
)
|
||||
|
||||
// 2. Schema extraction — produces TypeBox schemas and column metadata
|
||||
const { name, schema, columns, indexes, foreignKeys } = extractTable(UsersEl)
|
||||
|
||||
// 3. Module construction — assemble into Type.Module
|
||||
const defs = { Users: schema }
|
||||
const M = Type.Module(defs)
|
||||
const Users = M.Import('Users')
|
||||
|
||||
// 4. Validation — check data against module schema
|
||||
Value.Check(Users, { id: '...', name: 'alice' }) // true
|
||||
|
||||
// 5. Host rendering — produce Drizzle tables
|
||||
const root = createRoot(sqliteHost, {})
|
||||
root.render(UsersEl)
|
||||
const drizzleUsers = root.ctx.tables.users // sqliteTable result
|
||||
|
||||
// 6. Serialization — JSON Schema with $defs (for migration diffing)
|
||||
const serialized = JSON.parse(JSON.stringify(Users))
|
||||
|
||||
// 7. Schema derivation — insert, update, filter schemas (for API validation)
|
||||
defs.InsertUsers = Type.Object({ name: Type.String() }) // omit auto-generated columns
|
||||
defs.UpdateUsers = Type.Partial(Type.Ref('Users')) // all optional
|
||||
```
|
||||
|
||||
### Key Relationships
|
||||
|
||||
| Step | Input | Output | Consumer |
|
||||
|------|-------|--------|----------|
|
||||
| Extract | UJSX elements | `TableMeta` (schema + metadata) | Module construction, Host rendering |
|
||||
| Module | `Record<string, TSchema>` | `Type.Module` | Validation, Serialization |
|
||||
| Import | Module key | `TImport` (resolved schema) | Value.Check, Value.Diff |
|
||||
| Render | UJSX elements + HostConfig | Drizzle tables (in ctx) | Database queries |
|
||||
| Derive | Module entries | Insert/Update/Partial schemas | API validation |
|
||||
|
||||
## Key Design Decisions
|
||||
|
||||
### 1. UJSX Element Tree as the IR, Not a Separate Builder API
|
||||
@@ -131,6 +190,38 @@ For dbtype, formats serve as metadata that hosts can use (e.g., the PG host maps
|
||||
|
||||
The repo pattern (auto-generated CRUD for each table) is not a core feature of dbtype. It's an adapter for `@alkdev/operations` that consumes the same element tree and module bundle. This keeps dbtype focused on schema definition and rendering, while the operations integration is a separate concern.
|
||||
|
||||
## Error Handling Strategy
|
||||
|
||||
dbtype uses a layered validation approach:
|
||||
|
||||
1. **TypeScript compile-time enforcement**: `DbColumnType`, column props, and element types are enforced by TypeScript types. Invalid `type` values, missing required props, and incorrect prop types are caught at compile time.
|
||||
|
||||
2. **Runtime validation at extraction**: `extractTable()` validates the element tree — missing `name` or `type` on columns, duplicate column names, invalid `DbColumnType` values. These throw descriptive errors.
|
||||
|
||||
3. **TypeBox validation at module compile**: `Type.Module(defs)` validates the schema map. Invalid `Type.Ref` targets, duplicate keys, and malformed schemas throw TypeBox errors.
|
||||
|
||||
4. **Host rendering validation**: `createInstance()` in the host validates dialect-specific constraints — unknown column types fall back to `text()`, invalid symbolic defaults throw errors. PG enum names must be unique within a module.
|
||||
|
||||
The general principle: **catch errors as early as possible**. Type errors at compile time, structural errors at extraction time, schema errors at module compile time, dialect errors at render time.
|
||||
|
||||
## Glossary
|
||||
|
||||
| Term | Definition |
|
||||
|------|------------|
|
||||
| **UJSX** | Universal JSX — `@alkdev/ujsx`'s element system. Uses `h()` and `createComponent` to create element trees that render to different hosts via `HostConfig`. Not an acronym, a project name. |
|
||||
| **Element tree** | A tree of UJSX elements (`<table>`, `<column>`, `<index>`, `<fk>`) representing a database schema. The tree is the IR (intermediate representation). |
|
||||
| **HostConfig** | UJSX's host configuration interface. Defines how elements map to output objects (`createInstance`, `appendChild`, `finalizeInstance`). Each dialect (SQLite, PG, MySQL) is a host. |
|
||||
| **Type.Module** | `@alkdev/typebox`'s module system. Holds all schemas in a flat namespace with `Type.Ref` for cross-references. The module is dbtype's schema bundle. |
|
||||
| **Type.Ref** | A TypeBox reference type that resolves within a `Type.Module`. Enables forward and circular references without import ordering issues. |
|
||||
| **TImport** | The type returned by `M.Import(key)`. Embeds the full `$defs` namespace and resolves `$ref` pointers for validation. |
|
||||
| **$defs** | JSON Schema keyword used by serialized Type.Modules. Contains all referenced schemas. `$ref` pointers reference entries in `$defs`. |
|
||||
| **extractTable()** | Core function that walks an element tree, resolves function components, and produces `TableMeta` (TypeBox schema + column metadata + indexes + FKs). |
|
||||
| **Function component** | A UJSX component created with `createComponent` that returns element(s). Transparent to the host — only resolved intrinsic elements (`column`, `table`) reach `createInstance`. |
|
||||
| **Intrinsic element** | A built-in element type (`table`, `column`, `index`, `fk`). Not a function component — directly dispatched to the host's `createInstance`. |
|
||||
| **Reconciler** | UJSX's diffing engine. Can compare old and new element trees to produce update instructions. Not used in phase 1, but positions dbtype for future schema migration support. |
|
||||
| **Host rendering** | The process of walking an element tree through a `HostConfig` to produce output (Drizzle tables). Done via `createRoot(host).render(element)`. |
|
||||
| **Schema derivation** | Producing insert, update, and filter schemas from the module. E.g., `Type.Partial(Type.Ref('Users'))` for update schemas. |
|
||||
|
||||
## Document Lifecycle
|
||||
|
||||
| From | To | Condition |
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
|
||||
## Status
|
||||
|
||||
Proposed
|
||||
Accepted
|
||||
|
||||
## Context
|
||||
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
|
||||
## Status
|
||||
|
||||
Proposed
|
||||
Accepted
|
||||
|
||||
## Context
|
||||
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
|
||||
## Status
|
||||
|
||||
Proposed
|
||||
Accepted
|
||||
|
||||
## Context
|
||||
|
||||
|
||||
53
docs/architecture/decisions/006-references-vs-fk.md
Normal file
53
docs/architecture/decisions/006-references-vs-fk.md
Normal file
@@ -0,0 +1,53 @@
|
||||
# ADR-006: Column `references` as FK Shorthand
|
||||
|
||||
## Status
|
||||
|
||||
Accepted
|
||||
|
||||
## Context
|
||||
|
||||
The `<column>` element has a `references` prop (`string`) that specifies a foreign key target table name. There is also a separate `<fk>` element with full FK specification (`columns`, `references`, `foreignColumns`, `onDelete`, `onUpdate`).
|
||||
|
||||
Two mechanisms exist for expressing foreign keys with different capabilities:
|
||||
- `<column references="users">` — only specifies target table, no composite FK support, no ON DELETE/UPDATE
|
||||
- `<fk>` — full FK specification with composite support and actions
|
||||
|
||||
This creates ambiguity: which takes precedence? Can both be used simultaneously? What if they conflict?
|
||||
|
||||
## Decision
|
||||
|
||||
`references` on `<column>` is a **shorthand for simple single-column foreign keys** that targets the referenced table's primary key by convention. `<fk>` is the **explicit form** for composite foreign keys and FKs with ON DELETE/ON UPDATE actions.
|
||||
|
||||
Internally, `extractTable()` normalizes all `references` props into `FkMeta` entries alongside `<fk>` elements. When both `references` on a column and an explicit `<fk>` referencing the same column exist, the explicit `<fk>` takes precedence.
|
||||
|
||||
## Rationale
|
||||
|
||||
1. **Simple FKs are the common case**: Most foreign keys are single-column references to another table's primary key. `references="users"` covers this case ergonomically without requiring a separate `<fk>` element.
|
||||
|
||||
2. **Composition is additive**: `references` props and `<fk>` elements both produce `FkMeta` entries. There's no separate representation for "simple" vs "complex" FKs in the output — they're unified.
|
||||
|
||||
3. **Explicit overrides implicit**: When both forms specify the same relationship, the explicit `<fk>` form is more complete and should take precedence. This allows shorthand to be overridden without conflict.
|
||||
|
||||
4. **Convention over configuration**: `references` assumes the target is the primary key. This convention covers 95% of FK relationships. For the remaining cases (non-PK targets, composite FKs), `<fk>` provides full control.
|
||||
|
||||
5. **`extractTable()` normalizes**: Both forms produce the same `FkMeta` output, so downstream consumers (hosts, module, repo adapter) don't need to handle two different representations.
|
||||
|
||||
## Consequences
|
||||
|
||||
### Positive
|
||||
|
||||
- Ergonomic shorthand for the most common FK pattern
|
||||
- No ambiguity about precedence — explicit `<fk>` always wins
|
||||
- Unified internal representation (`FkMeta`) regardless of authoring style
|
||||
- `<column references="users">` maps to `{ columns: ['userId'], references: 'users', foreignColumns: ['id'] }` by convention
|
||||
|
||||
### Negative
|
||||
|
||||
- `references` on `<column>` cannot specify target columns or ON DELETE/UPDATE — must use `<fk>` for those
|
||||
- The convention that `references` targets the primary key must be documented and understood
|
||||
- Two ways to express the same thing (simple FKs) — could confuse new users until they learn the convention
|
||||
|
||||
## References
|
||||
|
||||
- [elements.md](../elements.md) — column and fk element definitions
|
||||
- [repo-adapter.md](../repo-adapter.md) — FK metadata consumption
|
||||
65
docs/architecture/decisions/007-inner-escape-hatch.md
Normal file
65
docs/architecture/decisions/007-inner-escape-hatch.md
Normal file
@@ -0,0 +1,65 @@
|
||||
# ADR-007: Flat Props with `inner` Escape Hatch for Column Validation
|
||||
|
||||
## Status
|
||||
|
||||
Accepted
|
||||
|
||||
## Context
|
||||
|
||||
Column elements carry both database metadata (notNull, primaryKey, default) and validation semantics (type, format, maxLength). The question is how to handle TypeBox validation constraints that don't have dedicated column props.
|
||||
|
||||
Option A (flat props only) limits validation to what has explicit props: `format`, `maxLength`, `minLength`, `pattern`, etc. Each validation constraint requires a new column prop. Custom validation beyond these requires manipulating the module entry directly.
|
||||
|
||||
Option B (flat props + `inner` escape hatch) uses flat props for the common 90% of cases and provides an `inner` prop that accepts a full TypeBox schema to override the auto-generated one. The host ignores `inner` — it's purely for the TypeBox schema.
|
||||
|
||||
Option C (inner-first) makes the TypeBox schema primary and DB metadata secondary. This is close to the `DbTypeBuilder` pattern that ADR-001 specifically rejected.
|
||||
|
||||
The research docs (`docs/research/architecture.md`) proposed `DbType.String({ notNull: true, inner: Type.String({ format: 'email', maxLength: 255 }) })`. With UJSX elements, this becomes `<column name="email" type="string" notNull inner={Type.String({ format: 'email', maxLength: 255 })} />`.
|
||||
|
||||
## Decision
|
||||
|
||||
Use flat props for common cases with an `inner` escape hatch for custom TypeBox schemas (Option B).
|
||||
|
||||
- Common validation constraints (`format`, `maxLength`, `minLength`, `pattern`, `values`) are top-level column props
|
||||
- The `inner` prop accepts a TypeBox schema that overrides the auto-generated `colToTypeBox()` result
|
||||
- When `inner` is provided, `extractTable()` uses it directly instead of calling `colToTypeBox(type, props)`
|
||||
- The host ignores `inner` — database rendering uses only `type` and the other DB metadata props
|
||||
- When `inner` is absent, `colToTypeBox(type, props)` generates the TypeBox schema from the column type and props
|
||||
|
||||
## Rationale
|
||||
|
||||
1. **Ergonomics for the common case**: Most columns only need `type`, `notNull`, `primaryKey`, `default`, and maybe `format`. Flat props (`<column name="email" type="string" notNull format="email" />`) are concise and readable.
|
||||
|
||||
2. **Escape hatch for complex validation**: Some columns need validation constraints that don't map to column props: `pattern`, `minimum`/`maximum` for numeric ranges, `contentMediaType` for embeddings, custom format validators. The `inner` prop lets you express any TypeBox validation without polluting the column props namespace.
|
||||
|
||||
3. **Host independence**: The host renders columns to Drizzle based on `type` and DB metadata. `inner` doesn't affect rendering at all. This means adding `inner` doesn't require host changes — it's a pure TypeConcern.
|
||||
|
||||
4. **Future-proofing**: Embedding vectors, custom types, and complex validation patterns will need `inner`. For instance, a vector column might be `<column name="embedding" type="string" mode="json" inner={Type.Array(Type.Number(), { minItems: 1536, maxItems: 1536 })} />` — the host still stores it as JSON, but the TypeBox schema validates the array structure.
|
||||
|
||||
5. **Probe validated**: The probe scripts use flat props exclusively. `inner` adds a single conditional in `extractTable()` — if `inner` is provided, use it; otherwise, call `colToTypeBox()`. No structural change to the pipeline.
|
||||
|
||||
6. **No alternative duplication**: Without `inner`, consumers who need `Type.String({ format: 'email', maxLength: 255, pattern: '^[a-z]' })` would have to manipulate the module entry after extraction, which defeats the purpose of a schema-first element tree.
|
||||
|
||||
## Consequences
|
||||
|
||||
### Positive
|
||||
|
||||
- Flat props stay ergonomic for the 90% case
|
||||
- Any TypeBox validation is expressible via `inner` without new column props
|
||||
- Hosts don't need changes — `inner` is a TypeConcern only
|
||||
- Future column types (vectors, custom point types) can embed complex validation
|
||||
- Implementation cost is minimal: one conditional in `extractTable()`
|
||||
|
||||
### Negative
|
||||
|
||||
- Two ways to express validation (flat props vs `inner`) — could confuse new users
|
||||
- `inner` TypeBox schemas are not reflected in column props — introspecting a column's validation requires looking at both `props` and `props.inner`
|
||||
- The `colToTypeBox()` function must still exist for the no-`inner` case, and its output must be compatible with what `inner` would provide
|
||||
- When `inner` is provided alongside flat validation props (`format`, `maxLength`), `inner` takes precedence. This must be clearly documented.
|
||||
|
||||
## References
|
||||
|
||||
- [elements.md](../elements.md) — Column element definition and `inner` prop
|
||||
- [ADR-001](001-ujsx-as-ir.md) — UJSX as the IR (rejected separate builder API)
|
||||
- [ADR-004](004-format-annotation-only.md) — Format as annotation (relevant: `format` is a flat prop, `inner` is for when it's not enough)
|
||||
- Research: `docs/research/architecture.md` (DbTypeBuilder inner pattern)
|
||||
73
docs/architecture/decisions/008-pg-enum-predeclaration.md
Normal file
73
docs/architecture/decisions/008-pg-enum-predeclaration.md
Normal file
@@ -0,0 +1,73 @@
|
||||
# ADR-008: PG Enum Pre-declaration — Return Both Enums and Tables
|
||||
|
||||
## Status
|
||||
|
||||
Accepted
|
||||
|
||||
## Context
|
||||
|
||||
PostgreSQL requires `pgEnum()` to be called at module scope before any table that references it. SQLite uses `text({ enum: [...] })` inline. MySQL uses `mysqlEnum()` inline. This creates a structural difference in the render output: PG enums must be declared separately.
|
||||
|
||||
Three options were considered:
|
||||
|
||||
**Option A**: Return both enums and tables from render. The PG host context accumulates enum declarations during the render walk and exposes them alongside tables.
|
||||
|
||||
**Option B**: Use `text()` for all enums initially, add native PG enum support later as an opt-in.
|
||||
|
||||
**Option C**: Per-column opt-in with `postgres: { nativeEnum: true }` — only generates `pgEnum` when explicitly requested.
|
||||
|
||||
## Decision
|
||||
|
||||
Use Option A: the PG host returns both enums and tables from the render context.
|
||||
|
||||
The PG host context shape after rendering:
|
||||
|
||||
```typescript
|
||||
interface PgRootCtx {
|
||||
dialect: 'pg'
|
||||
tables: Record<string, PgTable>
|
||||
enums: Record<string, PgEnum> // Accumulated during render
|
||||
}
|
||||
```
|
||||
|
||||
When a `<column type="enum" values={['happy', 'sad', 'neutral']}>` is encountered, the PG host:
|
||||
1. Registers the enum in `ctx.enums` (using the table and column name to derive a unique enum name)
|
||||
2. Uses the registered enum in the column builder
|
||||
3. The consumer includes both `ctx.enums` and `ctx.tables` in their Drizzle schema
|
||||
|
||||
SQLite and MySQL hosts use `ctx.tables` only — no `enums` key.
|
||||
|
||||
## Rationale
|
||||
|
||||
1. **Correct PG behavior**: PG enums are a separate type declaration (`CREATE TYPE ... AS ENUM`). The render output must include them for the schema to be valid.
|
||||
|
||||
2. **No information loss**: Option B (use `text()` for all) loses PG's native enum validation at the database level. The whole point of defining schema types is to get correct database behavior per dialect.
|
||||
|
||||
3. **Clean API**: The host context already accumulates `tables`. Adding `enums` to the same context is natural and doesn't change the render pattern — it just adds a second accumulation target.
|
||||
|
||||
4. **SQLite/MySQL unaffected**: These dialects don't have pre-declared enums. Their host contexts don't include `enums`, so the API is dialect-appropriate.
|
||||
|
||||
5. **Consistent with resolved OQ-006**: The rendering pipeline accumulates results in `root.ctx`. PG enums are auxiliary state, just like tables. The context shape varies by dialect — this is expected.
|
||||
|
||||
6. **Deriving enum names**: For a column `status` on table `users`, the PG enum name is `users_status` (table_column convention). This is the same convention Drizzle uses for inline enums.
|
||||
|
||||
## Consequences
|
||||
|
||||
### Positive
|
||||
|
||||
- Correct PG enum support from day one
|
||||
- No opt-in flag needed — enums just work when `type="enum"` is used with the PG host
|
||||
- Dialect-appropriate context: PG has `enums + tables`, SQLite/MySQL have `tables` only
|
||||
- Consistent with the context accumulation pattern
|
||||
|
||||
### Negative
|
||||
|
||||
- PG consumers must include `ctx.enums` in their schema file (alongside `ctx.tables`)
|
||||
- The derived enum naming convention (`table_column`) could conflict with explicit enum names in the future — a naming override prop may be needed later
|
||||
- Slightly more complex PG host implementation (tracking enum registrations during the walk)
|
||||
|
||||
## References
|
||||
|
||||
- [hosts.md](../hosts.md) — Host rendering pipeline and PG column type mapping
|
||||
- [elements.md](../elements.md) — Column element `type="enum"` and `values` props
|
||||
- [open-questions.md](../open-questions.md) — Original OQ-05
|
||||
@@ -34,10 +34,10 @@ A single column definition within a table.
|
||||
| `notNull` | `boolean` | no | Column is NOT NULL |
|
||||
| `primaryKey` | `boolean` | no | Column is primary key |
|
||||
| `unique` | `boolean` | no | Column has UNIQUE constraint |
|
||||
| `default` | `DbDefault \| unknown` | no | Symbolic default or literal value |
|
||||
| `references` | `string` | no | FK target table name |
|
||||
| `default` | `DbDefault \| unknown` | no | Symbolic default (`'now'`, `'uuid'`, `'autoincrement'`, `'current_timestamp'`), a literal value (string, number, boolean), or a Drizzle `sql` template expression. Symbolic defaults are resolved by the host to dialect-specific SQL. Literal values use `.default(value)`. SQL expressions use `.default(sql\`...\`)`. |
|
||||
| `references` | `string` | no | FK target table name (shorthand — references the target table's primary key). Normalized to an `<fk>` entry internally by extractTable(). For composite FKs or FKs with ON DELETE/ON UPDATE, use the `<fk>` element instead. |
|
||||
| `format` | `string` | no | TypeBox format annotation (uuid, email, etc.) |
|
||||
| `mode` | `'json' \| 'text'` | no | Storage mode for compound types |
|
||||
| `mode` | `'json' \| 'text'` | no | Storage mode for compound types. `'json'` stores the value as JSON text (SQLite/MySQL) or JSONB (PG). `'text'` stores as plain text. Only meaningful for `type: 'json'`, `type: 'array'`, and `type: 'object'`. Defaults to `'json'` for those types. |
|
||||
| `values` | `string[]` | no | Enum values (for `type: 'enum'`) |
|
||||
| `length` | `number` | no | Max length (for varchar) |
|
||||
| `precision` | `number` | no | Numeric precision |
|
||||
@@ -45,6 +45,32 @@ A single column definition within a table.
|
||||
| `postgres` | `PgColumnOpts` | no | PG-specific overrides |
|
||||
| `sqlite` | `SqliteColumnOpts` | no | SQLite-specific overrides |
|
||||
| `mysql` | `MySqlColumnOpts` | no | MySQL-specific overrides |
|
||||
| `inner` | `TSchema` | no | Override the auto-generated TypeBox schema. The host ignores this — it's purely for validation. When provided, `extractTable()` uses this schema directly instead of calling `colToTypeBox()`. |
|
||||
|
||||
### Storage Mode
|
||||
|
||||
The `mode` prop controls how compound types (json, array, object) are stored in the database:
|
||||
|
||||
- `mode: 'json'` (default for json/array/object) — stored as JSON text in SQLite/MySQL, JSONB in PG
|
||||
- `mode: 'text'` — stored as plain text, useful for columns that need JSON validation but plain-text storage
|
||||
|
||||
For scalar types, `mode` has no effect. The host maps `mode: 'json'` to the appropriate dialect storage (e.g., `text({ mode: 'json' })` in SQLite, `jsonb()` in PG).
|
||||
|
||||
### Default Values
|
||||
|
||||
Column defaults support three categories:
|
||||
|
||||
| Category | Example | Drizzle Output |
|
||||
|----------|---------|----------------|
|
||||
| Symbolic | `default: 'now'` | `.default(sql\`now()\`)` (PG) / `.default(sql\`(strftime('%s', 'now'))\`)` (SQLite) |
|
||||
| Symbolic | `default: 'uuid'` | `.defaultRandom()` (PG) / `.$defaultFn(() => crypto.randomUUID())` (SQLite) |
|
||||
| Symbolic | `default: 'autoincrement'` | Implicit on `INTEGER PRIMARY KEY` (SQLite) / `serial()` type (PG) |
|
||||
| Literal | `default: true` | `.default(true)` |
|
||||
| Literal | `default: 0` | `.default(0)` |
|
||||
| SQL expression | `default: sql\`CURRENT_TIMESTAMP\`` | `.default(sql\`CURRENT_TIMESTAMP\`)` |
|
||||
| JS function | `default: () => crypto.randomUUID()` | `.$defaultFn(() => crypto.randomUUID())` |
|
||||
|
||||
Symbolic defaults are translated by the host. Literal and SQL expression defaults pass through directly. JS function defaults use Drizzle's `.$defaultFn()`.
|
||||
|
||||
### `<index>`
|
||||
|
||||
@@ -138,13 +164,7 @@ The `extractTable()` function walks an element tree, resolving function componen
|
||||
3. **Table metadata**: `{ name, columns, indexes, foreignKeys }` for the host
|
||||
|
||||
```typescript
|
||||
function extractTable(el: UElement): {
|
||||
name: string
|
||||
schema: TObject // TypeBox schema for Type.Module
|
||||
columns: Record<string, ColumnMeta> // Props + computed metadata
|
||||
indexes: IndexMeta[]
|
||||
foreignKeys: FkMeta[]
|
||||
}
|
||||
function extractTable(el: UElement): TableMeta
|
||||
```
|
||||
|
||||
For each column, `extractTable`:
|
||||
@@ -157,11 +177,63 @@ For each column, `extractTable`:
|
||||
- **Column elements must have `name` and `type` props** — these are required for both TypeBox schema construction and Drizzle rendering
|
||||
- **Function components must return column elements** (or arrays of column elements) — returning other element types inside a table is undefined
|
||||
- **The `references` prop is metadata-only** — it's not part of the TypeBox schema. It informs the host about FK constraints and the repo adapter about relation structure
|
||||
- **`references` is a shorthand** — a `<column references="users">` prop is syntactic sugar for `{columns: [column_name], references: 'users', foreignColumns: ['id']}` (targeting the referenced table's primary key by convention). Internally, `extractTable()` normalizes all `references` props into `FkMeta` entries alongside `<fk>` elements. When both `references` on a column and an explicit `<fk>` referencing the same column exist, the explicit `<fk>` takes precedence.
|
||||
- **`default` values can be symbolic strings or literals** — symbolic defaults (`now`, `uuid`, `autoincrement`) are resolved by the host; literal values pass through directly
|
||||
- **The `inner` prop overrides the auto-generated TypeBox schema** — when `inner` is provided on a column element, `extractTable()` uses it directly without calling `colToTypeBox()`. The host ignores `inner` entirely — database rendering uses only `type` and DB metadata props. This is the escape hatch for validation constraints that don't have column prop shortcuts (e.g., `minItems`, `maximum`, `pattern`). See [ADR-007](decisions/007-inner-escape-hatch.md).
|
||||
|
||||
## Type Definitions
|
||||
|
||||
```typescript
|
||||
/** Column metadata extracted from a <column> element by extractTable() */
|
||||
interface ColumnMeta {
|
||||
name: string
|
||||
type: DbColumnType
|
||||
notNull?: boolean
|
||||
primaryKey?: boolean
|
||||
unique?: boolean
|
||||
default?: DbDefault | unknown
|
||||
references?: string // Shorthand: FK target table name
|
||||
format?: string // TypeBox format annotation
|
||||
mode?: 'json' | 'text' // Storage mode for compound types
|
||||
values?: string[] // Enum values (for type: 'enum')
|
||||
length?: number // Max length (for varchar)
|
||||
precision?: number // Numeric precision
|
||||
scale?: number // Numeric scale
|
||||
postgres?: PgColumnOpts // PG-specific overrides
|
||||
sqlite?: SqliteColumnOpts // SQLite-specific overrides
|
||||
mysql?: MySqlColumnOpts // MySQL-specific overrides
|
||||
inner?: TSchema // Override TypeBox schema (escape hatch)
|
||||
}
|
||||
|
||||
/** Index metadata extracted from an <index> element */
|
||||
interface IndexMeta {
|
||||
name: string
|
||||
columns: string[]
|
||||
unique?: boolean
|
||||
}
|
||||
|
||||
/** Foreign key metadata extracted from <fk> elements and references props */
|
||||
interface FkMeta {
|
||||
columns: string[] // Local column names
|
||||
references: string // Target table name
|
||||
foreignColumns: string[] // Target column names
|
||||
onDelete?: 'cascade' | 'set null' | 'restrict' | 'no action'
|
||||
onUpdate?: 'cascade' | 'set null' | 'restrict' | 'no action'
|
||||
}
|
||||
|
||||
/** Table metadata result from extractTable() */
|
||||
interface TableMeta {
|
||||
name: string
|
||||
schema: TObject // TypeBox schema for Type.Module entry
|
||||
columns: Record<string, ColumnMeta>
|
||||
indexes: IndexMeta[]
|
||||
foreignKeys: FkMeta[]
|
||||
}
|
||||
```
|
||||
|
||||
## Open Questions
|
||||
|
||||
1. **Should column elements support `inner` TypeBox schemas?** The research docs proposed `DbType.String({ notNull: true, inner: Type.String({ format: 'email', maxLength: 255 }) })`. With UJSX elements, this would be `<column name="email" type="string" notNull format="email" maxLength={255} />`. Is the flat props model sufficient, or do we need nested TypeBox schemas?
|
||||
1. ~~Should column elements support `inner` TypeBox schemas?~~ **Resolved — Yes. ADR-007: flat props for common cases, `inner` as escape hatch for custom validation.** The `inner` prop is now documented in the column props table.
|
||||
|
||||
2. **Should `<table>` accept `extraConfig` props?** Drizzle tables accept a third callback argument for indexes and unique constraints defined in terms of column references. How does this map to element props?
|
||||
|
||||
|
||||
@@ -32,6 +32,22 @@ interface DbtypeColumnInst {
|
||||
builder: any // Drizzle column builder
|
||||
dbMeta: Record<string, any>
|
||||
}
|
||||
|
||||
interface PgRootCtx {
|
||||
dialect: 'pg'
|
||||
tables: Record<string, PgTable>
|
||||
enums: Record<string, PgEnum> // Accumulated during render
|
||||
}
|
||||
|
||||
interface SqliteRootCtx {
|
||||
dialect: 'sqlite'
|
||||
tables: Record<string, SqliteTable>
|
||||
}
|
||||
|
||||
interface MySqlRootCtx {
|
||||
dialect: 'mysql'
|
||||
tables: Record<string, MySqlTable>
|
||||
}
|
||||
```
|
||||
|
||||
The `HostConfig` generic parameters:
|
||||
@@ -41,90 +57,43 @@ The `HostConfig` generic parameters:
|
||||
|
||||
## Column Type Mapping
|
||||
|
||||
### SQLite
|
||||
|
||||
| Column Type Prop | Drizzle Builder | Notes |
|
||||
|-----------------|----------------|-------|
|
||||
| `uuid` | `text(name).primaryKey().$defaultFn(crypto.randomUUID)` | No native UUID type |
|
||||
| `string` | `text(name)` | |
|
||||
| `integer` | `integer(name)` | |
|
||||
| `boolean` | `integer(name, { mode: 'boolean' })` | |
|
||||
| `timestamp` | `integer(name, { mode: 'timestamp' })` | |
|
||||
| `enum` | `text(name, { enum: values })` | SQLite has no native enum |
|
||||
| `json` | `text(name, { mode: 'json' })` | Stored as JSON text |
|
||||
|
||||
### PostgreSQL
|
||||
|
||||
| Column Type Prop | Drizzle Builder | Notes |
|
||||
|-----------------|----------------|-------|
|
||||
| `uuid` | `uuid(name).defaultRandom()` or `.primaryKey()` | Native UUID |
|
||||
| `string` | `text(name)` or `varchar(name, n)` | |
|
||||
| `integer` | `integer(name)` | |
|
||||
| `boolean` | `boolean(name)` | |
|
||||
| `timestamp` | `timestamptz(name, { withTimezone: true })` | |
|
||||
| `enum` | `pgEnum(name, values)` | Requires pre-declaration |
|
||||
| `json` | `jsonb(name)` | |
|
||||
|
||||
### MySQL
|
||||
|
||||
| Column Type Prop | Drizzle Builder | Notes |
|
||||
|-----------------|----------------|-------|
|
||||
| `uuid` | `varchar(name, { length: 36 })` | No native UUID |
|
||||
| `string` | `text(name)` or `varchar(name, n)` | |
|
||||
| `integer` | `int(name)` | |
|
||||
| `boolean` | `boolean(name)` or `tinyint(name)` | |
|
||||
| `timestamp` | `timestamp(name)` | |
|
||||
| `enum` | `mysqlEnum(name, values)` | |
|
||||
| `json` | `json(name)` | |
|
||||
| DbColumnType | SQLite | PostgreSQL | MySQL | TypeBox | Notes |
|
||||
|---|---|---|---|---|---|
|
||||
| `uuid` | `text(name).primaryKey().$defaultFn(crypto.randomUUID)` | `uuid(name).defaultRandom()` / `.primaryKey()` | `varchar(name, { length: 36 })` | `Type.String({ format: 'uuid' })` | No native UUID in SQLite/MySQL |
|
||||
| `string` | `text(name)` | `text(name)` | `text(name)` | `Type.String()` | General unbounded text; for bounded strings use `varchar` |
|
||||
| `text` | `text(name)` | `text(name)` | `text(name)` | `Type.String()` | Explicitly unbounded text content |
|
||||
| `varchar` | `text(name)` | `varchar(name, { length })` | `varchar(name, { length })` | `Type.String({ maxLength })` | Maps to `text()` in SQLite (no varchar); PG/MySQL have native varchar with length |
|
||||
| `integer` | `integer(name)` | `integer(name)` | `int(name)` | `Type.Integer()` | |
|
||||
| `bigint` | `blob(name, { mode: 'bigint' })` | `bigint(name)` | `bigint(name)` | custom `TypeBigInt()` | SQLite uses blob with bigint mode; TypeBox has no standard BigInt type, needs annotation |
|
||||
| `boolean` | `integer(name, { mode: 'boolean' })` | `boolean(name)` | `boolean(name)` / `tinyint(name)` | `Type.Boolean()` | SQLite has no native boolean type |
|
||||
| `timestamp` | `integer(name, { mode: 'timestamp' })` | `timestamptz(name, { withTimezone: true })` | `timestamp(name)` | `Type.Number()` | |
|
||||
| `real` | `real(name)` | `real(name)` | `float(name)` | `Type.Number()` | SQLite/PG use `real()`; MySQL uses `float()`; all map to `Type.Number()` |
|
||||
| `numeric` | `numeric(name)` | `numeric(name, { precision, scale })` | `decimal(name, { precision, scale })` | `Type.String()` | Stored as string; precision/scale from column props |
|
||||
| `enum` | `text(name, { enum: values })` | `pgEnum(name, values)` | `mysqlEnum(name, values)` | `Type.Union([...])` | SQLite has no native enum; PG requires pre-declaration |
|
||||
| `json` | `text(name, { mode: 'json' })` | `jsonb(name)` | `json(name)` | `Type.Any()` | Stored as JSON text; PG uses `jsonb` by default |
|
||||
| `array` | N/A (not supported) | `column(name).array()` | N/A (not supported) | `Type.Array(inner)` | Only PG supports native arrays; other dialects should use json mode; requires special handling |
|
||||
| `object` | `text(name, { mode: 'json' })` | `jsonb(name)` | `json(name)` | `Type.Object(properties)` | Always stored as JSON; same as `json` type but with structured inner schema |
|
||||
|
||||
## Symbolic Defaults
|
||||
|
||||
Column props like `default="now"` and `default="uuid"` are translated to dialect-specific SQL or JS functions by the host:
|
||||
|
||||
| Symbol | SQLite | PostgreSQL | MySQL |
|
||||
|--------|--------|------------|-------|
|
||||
| `now` | `sql\`(strftime('%s', 'now'))\`` | `sql\`now()\`` | `sql\`NOW()\`` |
|
||||
| `uuid` | `.$defaultFn(() => crypto.randomUUID())` | `.defaultRandom()` | `.$defaultFn(() => crypto.randomUUID())` |
|
||||
| `autoincrement` | Implicit on `INTEGER PRIMARY KEY` | `serial()` type | `.autoincrement()` |
|
||||
Symbolic defaults (`'now'`, `'uuid'`, `'autoincrement'`, `'current_timestamp'`) are resolved to dialect-specific SQL or JS functions by the host. See [elements.md](elements.md#default-values) for the full default value specification.
|
||||
|
||||
## Common Column Components
|
||||
|
||||
Function components that compose into any table:
|
||||
|
||||
```typescript
|
||||
const IdColumn = createComponent('IdColumn', () =>
|
||||
h('column', { name: 'id', type: 'uuid', primaryKey: true, default: 'uuid' })
|
||||
)
|
||||
|
||||
const AuditColumns = createComponent('AuditColumns', () => [
|
||||
h('column', { name: 'createdAt', type: 'timestamp', notNull: true, default: 'now' }),
|
||||
h('column', { name: 'updatedAt', type: 'timestamp', notNull: true, default: 'now' }),
|
||||
])
|
||||
```
|
||||
|
||||
Used by spreading into a table:
|
||||
|
||||
```typescript
|
||||
const UsersEl = h('table', { name: 'users' },
|
||||
h(IdColumn, {}),
|
||||
h('column', { name: 'name', type: 'string', notNull: true }),
|
||||
h(AuditColumns, {}),
|
||||
)
|
||||
```
|
||||
|
||||
This replaces the storage_sqlite pattern of `const commonCols = { id: text('id').primaryKey(), ... }` with reusable, dialect-agnostic components.
|
||||
Function components like `IdColumn` and `AuditColumns` compose into any table. See [elements.md](elements.md#function-components) for component definitions.
|
||||
|
||||
## Rendering Pipeline
|
||||
|
||||
1. **Walk elements**: The host's `createInstance` method receives each `<table>`, `<column>`, `<index>`, `<fk>` element
|
||||
2. **Build columns**: `createInstance('column', props, ctx)` maps `props.type` to the dialect-specific Drizzle builder, applying `notNull`, `primaryKey`, `unique`, `default` constraints
|
||||
3. **Assemble tables**: `appendChild(tableInst, columnInst, ctx)` attaches column builders to the table's columns record
|
||||
4. **Finalize**: `finalizeInstance(tableInst, ctx)` calls `sqliteTable(name, columns)` (or `pgTable`, `mysqlTable`) and stores the result
|
||||
4. **Finalize**: `finalizeInstance(tableInst, ctx)` calls `sqliteTable(name, columns)` (or `pgTable`, `mysqlTable`) and stores the result on the table instance
|
||||
|
||||
```typescript
|
||||
const root = createRoot(sqliteHost, {})
|
||||
root.render(UsersEl)
|
||||
// root's context now contains: { tables: { users: drizzleSqliteTable } }
|
||||
// root.ctx.tables: { users: drizzleSqliteTable } — accessible after render
|
||||
// For PG hosts, root.ctx also accumulates auxiliary state like pgEnum declarations
|
||||
```
|
||||
|
||||
## Constraints
|
||||
@@ -132,16 +101,16 @@ root.render(UsersEl)
|
||||
- **`createTextInstance` is not supported** — DB schemas are purely structural, no text nodes
|
||||
- **Column type props must be from the `DbColumnType` vocabulary** — hosts map these to dialect-specific builders; unknown types fall back to `text()`
|
||||
- **Symbolic defaults are resolved by the host** — `default="now"` becomes `sql\`(strftime('%s', 'now'))\`` on SQLite and `sql\`now()\`` on PG
|
||||
- **PG enums require pre-declaration** — the host must track enum types and emit `pgEnum()` calls before tables that reference them
|
||||
- **PG enums are accumulated in context** — when a `<column type="enum" values={[...]}>` is encountered, the PG host registers the enum in `ctx.enums` (using `table_column` naming convention) and references it in the column builder. The consumer includes both `ctx.enums` and `ctx.tables` in their Drizzle schema. See [ADR-008](decisions/008-pg-enum-predeclaration.md).
|
||||
- **A host renders one dialect at a time** — to generate schemas for multiple dialects, render multiple times with different hosts
|
||||
|
||||
## Open Questions
|
||||
|
||||
1. **How to handle PG enum pre-declaration?** PG requires `pgEnum()` at module scope before tables. Options: (A) return both enums and tables from render, (B) start with text for all enums, (C) per-column opt-in. Leaning toward A.
|
||||
1. **[RESOLVED — ADR-008](decisions/008-pg-enum-predeclaration.md) PG enum pre-declaration**: PG requires `pgEnum()` at module scope before tables. The host accumulates enum declarations in `root.ctx.enums` during the render walk, and `finalizeRoot` emits them. This is option A — enum state lives in context alongside tables.
|
||||
|
||||
2. **Should hosts return the rendered table or store it in context?** The probe scripts use context (`ctx.tables`), but returning from render would be more functional. Need to resolve this.
|
||||
2. **[Resolved] render() return value vs context**: `render()` on the UJSX `Root` returns `void` — the rendered Drizzle tables are collected in `root.ctx.tables`. Context is the right home for this because PG hosts need auxiliary state (enum declarations) alongside the primary output. The context shape varies by dialect: SQLite uses `{ dialect: 'sqlite', tables: Record<string, SqliteTable> }`, PG uses `{ dialect: 'pg', enums: Record<string, PgEnum>, tables: Record<string, PgTable> }`.
|
||||
|
||||
3. **`prepareUpdate`/`commitUpdate` for migrations?** The UJSX reconciler could diff old and new element trees to produce ALTER TABLE statements. This is a future feature, not phase 1.
|
||||
3. **`prepareUpdate`/`commitUpdate` for migrations**: The UJSX reconciler's `prepareUpdate`/`commitUpdate` hooks could diff old and new element trees to produce ALTER TABLE statements. This is out of scope for phase 1, but the reconciler architecture positions us for it in a future phase.
|
||||
|
||||
## References
|
||||
|
||||
|
||||
@@ -1,17 +1,11 @@
|
||||
---
|
||||
status: draft
|
||||
last_updated: 2026-05-22
|
||||
status: stable
|
||||
last_updated: 2026-05-23
|
||||
---
|
||||
|
||||
# Module: Type.Module as the Schema Bundle
|
||||
# Module: Type.Module Mechanical Reference
|
||||
|
||||
Technical details on how dbtype uses `Type.Module` for schema construction, validation, serialization, and migration.
|
||||
|
||||
## Overview
|
||||
|
||||
dbtype uses `@alkdev/typebox`'s `Type.Module` as the schema storage and resolution mechanism. A module holds all table schemas, their relations, and derived schemas (insert, update, partial) in a single flat namespace. `Type.Ref` resolves cross-table references — including circular ones — without import ordering issues.
|
||||
|
||||
This document covers the mechanics, constraints, and patterns discovered during architecture probing.
|
||||
How `Type.Module` works mechanically — construction, validation, serialization, migration diffing, cross-module references, and constraints. For the domain-specific content (what goes into a dbtype module, schema derivation semantics, relations), see [schema.md](schema.md).
|
||||
|
||||
## Construction Patterns
|
||||
|
||||
@@ -27,6 +21,27 @@ const M = Type.Module(defs)
|
||||
const Users = M.Import('Users')
|
||||
```
|
||||
|
||||
### Incremental Construction
|
||||
|
||||
The defs map is a plain `Record<string, TSchema>` — it can be built incrementally, mutated, and extended before compilation:
|
||||
|
||||
```typescript
|
||||
const defs: Record<string, any> = {}
|
||||
defs.Users = extractTableSchema(UsersElement)
|
||||
defs.Tasks = extractTableSchema(TasksElement)
|
||||
|
||||
// Add a column later
|
||||
defs.Users = Type.Object({ ...defs.Users.properties, role: Type.String() })
|
||||
|
||||
// Add relations
|
||||
defs.UsersRelations = Type.Object({ tasks: Type.Array(Type.Ref('Tasks')) })
|
||||
|
||||
// Compile when ready
|
||||
const M = Type.Module(defs)
|
||||
```
|
||||
|
||||
Once compiled, mutations to the original defs map don't affect the compiled module.
|
||||
|
||||
### With Relations
|
||||
|
||||
```typescript
|
||||
@@ -43,24 +58,6 @@ defs.InsertUsers = Type.Object({ name: Type.String(), email: Type.String() }) /
|
||||
defs.UpdateUsers = Type.Partial(Type.Ref('Users')) // computed
|
||||
```
|
||||
|
||||
### Incremental Construction
|
||||
|
||||
```typescript
|
||||
// Build defs incrementally
|
||||
const defs: Record<string, any> = {}
|
||||
defs.Users = extractTableSchema(UsersElement)
|
||||
defs.Tasks = extractTableSchema(TasksElement)
|
||||
|
||||
// Add a column later
|
||||
defs.Users = Type.Object({ ...defs.Users.properties, role: Type.String() })
|
||||
|
||||
// Add relations
|
||||
defs.UsersRelations = Type.Object({ tasks: Type.Array(Type.Ref('Tasks')) })
|
||||
|
||||
// Compile when ready
|
||||
const M = Type.Module(defs)
|
||||
```
|
||||
|
||||
## Validation
|
||||
|
||||
### Format Registration Required
|
||||
@@ -107,13 +104,13 @@ for (const err of Value.Errors(Users, badData)) { ... }
|
||||
|
||||
Key properties:
|
||||
- Each `$defs` entry has an `$id` matching its key
|
||||
- `Type.Ref` remains as `{ "$ref": "Key" }` — not inlined
|
||||
- `Type.Ref` remains as `{ "$ref": "Key" }` — not inlined; consumers must resolve them
|
||||
- The entire structure is valid JSON Schema
|
||||
- All entries in the module are present in `$defs` (even if only one was imported)
|
||||
|
||||
### Roundtrip
|
||||
|
||||
The serialized form can be parsed back into a schema-like structure. `Value.Diff` works on these serialized objects to produce structural edit lists.
|
||||
The serialized form can be parsed back into a schema-like structure. `Value.Diff` works on these serialized objects to produce structural edit lists. Note that Symbol properties (`[Kind]`, `[Hint]`, etc.) are stripped by `JSON.stringify` — the serialized form is JSON Schema, not TypeBox schema. Roundtripping requires `FromSchema` or reconstructed TypeBox objects.
|
||||
|
||||
## Migration Diffing
|
||||
|
||||
@@ -160,7 +157,10 @@ However, this nests `$defs` within `$defs` (the User's `$defs` contains CommonUu
|
||||
- **Module entries are computed at construction time** — `Type.Partial(Type.Ref('Users'))` is resolved when the module is built, producing a concrete optional-property object
|
||||
- **`Type.Ref` outside a module has `static: unknown`** — always use `M.Import(key)` for proper type inference
|
||||
- **Module keys are a flat namespace** — no nested paths like `"tables/Users"`. Table names must be unique within the module.
|
||||
- **`Type.Ref` resolves within the module only** — no cross-module references without `Module.Import`
|
||||
- **`Module.Import` embeds all `$defs`** — every import carries the full module. This is correct for validation but increases JSON Schema size.
|
||||
- **Defs map is mutable until compiled** — once passed to `Type.Module`, mutations to the original map don't affect the compiled module
|
||||
- **Format validation requires `FormatRegistry.Set`** — `uuid`, `email`, and other custom formats must be registered before `Value.Check` will enforce them
|
||||
- **Symbol properties are lost in `JSON.stringify`** — `[Kind]`, `[Hint]`, etc. are stripped. The serialized form is JSON Schema, not TypeBox schema. Roundtripping requires `FromSchema` or reconstructed TypeBox objects.
|
||||
|
||||
## References
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
---
|
||||
status: draft
|
||||
last_updated: 2026-05-22
|
||||
last_updated: 2026-05-23
|
||||
---
|
||||
|
||||
# Open Questions Tracker
|
||||
@@ -37,24 +37,26 @@ All unresolved architectural questions for dbtype, organized by theme.
|
||||
### OQ-04: Should column elements support nested TypeBox schemas?
|
||||
|
||||
- **Origin**: [elements.md](elements.md)
|
||||
- **Status**: Open
|
||||
- **Priority**: High
|
||||
- **Context**: The research docs proposed `DbType.String({ notNull: true, inner: Type.String({ format: 'email', maxLength: 255 }) })`. With UJSX elements, this would be `<column name="email" type="string" notNull format="email" maxLength={255} />`. The flat props model works for common cases, but custom validation constraints (patterns, ranges, custom checks) may need a nested `inner` prop. Should we support `<column name="email" type="string" notNull inner={Type.String({ format: 'email', maxLength: 255 })} />`?
|
||||
- **Cross-references**: ADR-001
|
||||
- **Status**: Resolved
|
||||
- **Priority**: ~~High~~ Resolved
|
||||
- **Context**: The research docs proposed `DbType.String({ notNull: true, inner: Type.String({ format: 'email', maxLength: 255 }) })`. With UJSX elements, this would be `<column name="email" type="string" notNull format="email" maxLength={255} />`. The flat props model works for common cases, but custom validation constraints (patterns, ranges, custom checks) may need a nested `inner` prop.
|
||||
- **Resolution**: Yes — flat props for common cases, `inner` as escape hatch for custom validation. When `inner` is provided, `extractTable()` uses it directly instead of calling `colToTypeBox()`. The host ignores `inner` — it's purely for the TypeBox schema. See [ADR-007](decisions/007-inner-escape-hatch.md).
|
||||
|
||||
### OQ-05: How to handle PG enum pre-declaration?
|
||||
|
||||
- **Origin**: [hosts.md](hosts.md)
|
||||
- **Status**: Open
|
||||
- **Priority**: High
|
||||
- **Context**: PG requires `pgEnum()` at module scope before tables that reference it. Options: (A) return both enums and tables from render, (B) start with text for all enums, (C) per-column opt-in with `postgres: { nativeEnum: true }`. Option A is cleanest but changes the render API. Option B is simplest but loses PG enum validation.
|
||||
- **Status**: Resolved
|
||||
- **Priority**: ~~High~~ Resolved
|
||||
- **Context**: PG requires `pgEnum()` at module scope before tables that reference it. Options: (A) return both enums and tables from render, (B) start with text for all enums, (C) per-column opt-in with `postgres: { nativeEnum: true }`.
|
||||
- **Resolution**: Option A — the PG host accumulates enums in `ctx.enums` during the render walk. The consumer includes both `ctx.enums` and `ctx.tables` in their Drizzle schema. Enum names follow the `table_column` convention. See [ADR-008](decisions/008-pg-enum-predeclaration.md).
|
||||
|
||||
### OQ-06: Should hosts return rendered tables or store them in context?
|
||||
|
||||
- **Origin**: [hosts.md](hosts.md)
|
||||
- **Status**: Open
|
||||
- **Status**: Resolved
|
||||
- **Priority**: Medium
|
||||
- **Context**: The probe scripts use `ctx.tables` to collect rendered tables. A more functional approach would have `render()` return the rendered table directly. Need to resolve this before implementation.
|
||||
- **Resolution**: render() accumulates results in root.ctx (tables, enums). The context shape varies by dialect — SQLite has ctx.tables, PG has ctx.tables and ctx.enums. See hosts.md Rendering Pipeline.
|
||||
|
||||
### OQ-07: Should we support JSX file extensions?
|
||||
|
||||
@@ -95,9 +97,9 @@ All unresolved architectural questions for dbtype, organized by theme.
|
||||
| OQ-01 | Relation naming convention | schema.md | Medium | Open |
|
||||
| OQ-02 | Derived schemas in module or separate | schema.md | Low | Open |
|
||||
| OQ-03 | Multiple database namespaces | schema.md | Low | Open |
|
||||
| OQ-04 | Nested TypeBox schemas in column props | elements.md | High | Open |
|
||||
| OQ-05 | PG enum pre-declaration | hosts.md | High | Open |
|
||||
| OQ-06 | Host render return value vs context | hosts.md | Medium | Open |
|
||||
| OQ-04 | Nested TypeBox schemas in column props | elements.md | ~~High~~ Resolved | **Resolved** — ADR-007 |
|
||||
| OQ-05 | PG enum pre-declaration | hosts.md | ~~High~~ Resolved | **Resolved** — ADR-008 |
|
||||
| OQ-06 | Host render return value vs context | hosts.md | Medium | Resolved |
|
||||
| OQ-07 | JSX file extensions | elements.md | Low | Open |
|
||||
| OQ-08 | Per-dialect handler differences | repo-adapter.md | Medium | Open |
|
||||
| OQ-09 | Relation rendering responsibility | repo-adapter.md | Medium | Open |
|
||||
|
||||
@@ -1,21 +1,13 @@
|
||||
---
|
||||
status: draft
|
||||
last_updated: 2026-05-22
|
||||
status: stable
|
||||
last_updated: 2026-05-23
|
||||
---
|
||||
|
||||
# Schema: Type.Module as the Schema Bundle
|
||||
# Schema: Domain Data Inside the Module
|
||||
|
||||
How dbtype uses `Type.Module` to store all table schemas, relations, and derived schemas in a single namespace with automatic `Type.Ref` resolution.
|
||||
What goes into a dbtype `Type.Module` — table schemas, relation entries, derived schemas (insert, update, filter), and the schema derivation semantics. For the mechanical details of how `Type.Module` works (construction, validation, serialization, migration diffing, constraints), see [module.md](module.md).
|
||||
|
||||
## Overview
|
||||
|
||||
The `Type.Module` is the central data structure in dbtype. It holds every table's TypeBox schema, all cross-table relations, and derived schemas (insert, update, select variants) in one flat namespace. `Type.Ref` resolves forward and circular references naturally, eliminating the need for separate relation files or import-order management.
|
||||
|
||||
The module is also the serialization boundary: `JSON.stringify(module.Import('Users'))` produces valid JSON Schema with `$defs`, enabling migration diffing via `Value.Diff`.
|
||||
|
||||
## Construction
|
||||
|
||||
### From Element Tree to Module
|
||||
## From Element Tree to Module
|
||||
|
||||
The element tree (`<table>`, `<column>`) is walked to extract a `Record<string, TSchema>` map, then compiled into a module:
|
||||
|
||||
@@ -34,29 +26,24 @@ Each `<column>` element produces a TypeBox type based on its `type` prop:
|
||||
| `timestamp` | `Type.Number()` |
|
||||
| `enum` | `Type.Union(values.map(v => Type.Literal(v)))` |
|
||||
|
||||
### Incremental Construction
|
||||
For incremental construction patterns and compilation mechanics, see [module.md](module.md).
|
||||
|
||||
The defs map is a plain `Record<string, TSchema>` — it can be built incrementally, mutated, and extended before compilation:
|
||||
## Relations
|
||||
|
||||
Relations are stored as separate entries in the module, using `Type.Ref` to reference other tables:
|
||||
|
||||
```typescript
|
||||
const defs: Record<string, any> = {}
|
||||
|
||||
// Add tables one at a time
|
||||
defs.Users = Type.Object({ id: Type.String({ format: 'uuid' }), name: Type.String() })
|
||||
defs.Tasks = Type.Object({ id: Type.String({ format: 'uuid' }), userId: Type.String({ format: 'uuid' }), title: Type.String() })
|
||||
|
||||
// Add columns to an existing table
|
||||
defs.Users = Type.Object({ ...defs.Users.properties, role: Type.String() })
|
||||
|
||||
// Add relations
|
||||
defs.UsersRelations = Type.Object({ tasks: Type.Array(Type.Ref('Tasks')) })
|
||||
defs.TasksRelations = Type.Object({ user: Type.Ref('Users') })
|
||||
|
||||
// Compile
|
||||
const M = Type.Module(defs)
|
||||
```
|
||||
|
||||
Once compiled, `M.Import(key)` returns a `TImport` schema with the full `$defs` namespace embedded.
|
||||
This gives:
|
||||
- **Type-safe validation**: `Value.Check(M.Import('UsersRelations'), { tasks: [...] })` validates the full nested structure
|
||||
- **No circular import issues**: `Type.Ref` resolves within the module namespace regardless of definition order
|
||||
- **Queryable structure**: The `$defs` map is enumerable — you can find all relations for a table by naming convention
|
||||
- **Drizzle integration**: The repo adapter reads relation entries to generate `relations()` calls for drizzle's relational query builder
|
||||
|
||||
Foreign key metadata lives on the column element's `references` prop (`<column name="userId" type="uuid" references="users" />`), not in the relation entry. Relations describe the "from this side, I see many of those" semantics.
|
||||
|
||||
## Schema Derivation
|
||||
|
||||
@@ -93,64 +80,6 @@ defs.UpdateUsers = Type.Partial(Type.Ref('Users'))
|
||||
|
||||
Per-column comparison operators derived from the column type. Generated by the repo adapter, not the core module.
|
||||
|
||||
## Relations
|
||||
|
||||
Relations are stored as separate entries in the module, using `Type.Ref` to reference other tables:
|
||||
|
||||
```typescript
|
||||
defs.UsersRelations = Type.Object({ tasks: Type.Array(Type.Ref('Tasks')) })
|
||||
defs.TasksRelations = Type.Object({ user: Type.Ref('Users') })
|
||||
```
|
||||
|
||||
This gives:
|
||||
- **Type-safe validation**: `Value.Check(M.Import('UsersRelations'), { tasks: [...] })` validates the full nested structure
|
||||
- **No circular import issues**: `Type.Ref` resolves within the module namespace regardless of definition order
|
||||
- **Queryable structure**: The `$defs` map is enumerable — you can find all relations for a table by naming convention
|
||||
- **Drizzle integration**: The repo adapter reads relation entries to generate `relations()` calls for drizzle's relational query builder
|
||||
|
||||
Foreign key metadata lives on the column element's `references` prop (`<column name="userId" type="uuid" references="users" />`), not in the relation entry. Relations describe the "from this side, I see many of those" semantics.
|
||||
|
||||
## Serialization
|
||||
|
||||
`JSON.stringify(M.Import('TableName'))` produces JSON Schema with `$defs`:
|
||||
|
||||
```json
|
||||
{
|
||||
"$defs": {
|
||||
"Users": { "$id": "Users", "type": "object", "properties": { ... } },
|
||||
"Tasks": { "$id": "Tasks", "type": "object", "properties": { ... } },
|
||||
"UsersRelations": { "$id": "UsersRelations", "type": "object", "properties": { "tasks": { "items": { "$ref": "Tasks" }, "type": "array" } } }
|
||||
},
|
||||
"$ref": "Users"
|
||||
}
|
||||
```
|
||||
|
||||
Key properties:
|
||||
- Each `$defs` entry gets an `$id` matching its key name
|
||||
- `Type.Ref` leaves `$ref` pointers (not inlined) — consumers must resolve them
|
||||
- The serialized form is valid JSON Schema
|
||||
- `Value.Diff` produces structural edits between two serialized schemas (useful for migration diffing)
|
||||
|
||||
## Migration Diffing
|
||||
|
||||
```typescript
|
||||
const v1 = JSON.parse(JSON.stringify(M.Import('Users')))
|
||||
// ... add a column to defs.Users ...
|
||||
const v2 = JSON.parse(JSON.stringify(M2.Import('Users')))
|
||||
const edits = Value.Diff(v1, v2)
|
||||
// edits: [{ type: 'insert', path: '/$defs/Users/properties/role', value: { type: 'string' } }, ...]
|
||||
```
|
||||
|
||||
The edits use JSON Pointer paths, which can be translated to `ALTER TABLE ADD COLUMN` statements.
|
||||
|
||||
## Constraints
|
||||
|
||||
- **Module keys must be unique** — two tables cannot have the same name in the same module
|
||||
- **`Type.Ref` resolves within the module only** — no cross-module references without `Module.Import`
|
||||
- **`Type.Ref` outside a module has `static: unknown`** — always access via `M.Import(key)` for proper type inference
|
||||
- **Defs map is mutable until compiled** — once passed to `Type.Module`, mutations to the original map don't affect the compiled module
|
||||
- **Format validation requires `FormatRegistry.Set`** — `uuid`, `email`, and other custom formats must be registered before `Value.Check` will enforce them
|
||||
|
||||
## Open Questions
|
||||
|
||||
1. **Should relation entries use a naming convention?** Currently `UsersRelations` / `TasksRelations`. Is this sufficient, or should relations be structured differently (e.g., a `relations` field on the table entry)?
|
||||
@@ -162,5 +91,6 @@ The edits use JSON Pointer paths, which can be translated to `ALTER TABLE ADD CO
|
||||
## References
|
||||
|
||||
- TypeBox Module API: `@alkdev/typebox` source — `type/module/module.ts`, `type/ref/ref.ts`
|
||||
- Mechanical details: [module.md](module.md)
|
||||
- Probe validation: `scripts/probe-e2e.ts`
|
||||
- Research: `docs/research/architecture.md`
|
||||
Reference in New Issue
Block a user