docs: restructure architecture docs to flowgraph pattern
- Create decisions/ directory with 32 numbered ADRs (ADR-001 through ADR-032) extracted from inline DD/SD/ED/SE decision sections - Create open-questions.md with 16 OQs organized by theme, cross-referenced to ADRs, with status tracking (resolved/open) - Create README.md as architecture index with doc table, ADR table, and lifecycle status definitions (draft/reviewed/stable/deprecated) - Replace inline decision sections in all spec docs with ADR reference tables - Replace inline open questions with OQ references to centralized tracker - Update frontmatter: metagraph-module.md, overview.md, sqlite-host.md → reviewed; schema-evolution.md and encrypted-data.md remain draft - DD1-DD10 → ADR-009 through ADR-018 - D1-D8 → ADR-001 through ADR-008 - SD1-SD5 → ADR-019 through ADR-023 (SD5 folded into ADR-006/008) - ED1-ED5 → ADR-023 through ADR-027 - SE1-SE5 → ADR-028 through ADR-032
This commit is contained in:
@@ -475,77 +475,25 @@ DB) avoids this problem — the stored schema is already dereferenced.
|
||||
|
||||
## Design Decisions
|
||||
|
||||
### SE1: Additive-only for v1, Cast migration when needed
|
||||
All design decisions are documented as ADRs in [decisions/](decisions/).
|
||||
|
||||
For v1, schema changes should be additive (new optional fields, new types,
|
||||
new enum values). This avoids data migration entirely. When additive-only is
|
||||
insufficient, `Value.Cast` handles the common migration cases. Custom
|
||||
migration functions are the consumer's responsibility.
|
||||
|
||||
### SE2: Version as a coarse-grained breaking-change signal
|
||||
|
||||
The `version` integer on `graph_types` tracks **breaking** schema changes.
|
||||
Non-breaking changes (additive) do not require a version bump. This is a
|
||||
coarse signal — the repository layer checks version before processing and
|
||||
knows to run migration logic when it doesn't match.
|
||||
|
||||
### SE3: Schema change detection via Value.Diff, not manual tracking
|
||||
|
||||
Rather than maintaining a separate "schema version log" or changelog, the
|
||||
repository layer uses `Value.Diff(storedSchema, moduleEntry)` to detect when
|
||||
a stored schema has diverged from the current Module entry. This is
|
||||
schema-agnostic and works for any change.
|
||||
|
||||
### SE4: moduleToDbSchema() for schema updates, not Value.Patch
|
||||
|
||||
When updating stored schemas, re-run `moduleToDbSchema()` on the full Module
|
||||
rather than using `Value.Patch` to apply edits. This is more reliable because
|
||||
it doesn't depend on Diff correctly capturing `Type.Ref`/`$defs` changes.
|
||||
Patch-based schema update is an optimization for later.
|
||||
|
||||
### SE5: Single-author model, not CRDT
|
||||
|
||||
Schema evolution assumes single-author per graph type. There is no concurrent
|
||||
multi-author editing of graph types. If this changes (multiple consumers
|
||||
defining the same graph type with different schemas), a merge/CRDT layer would
|
||||
be needed. That's a post-v1 concern.
|
||||
| ADR | Decision | Summary |
|
||||
|-----|----------|---------|
|
||||
| [028](decisions/028-additive-only-with-cast-migration.md) | Additive-only for v1, Cast migration when needed | Additive changes avoid migration; `Value.Cast` for common cases |
|
||||
| [029](decisions/029-version-as-breaking-change-signal.md) | Version as a coarse-grained breaking-change signal | Only breaking changes bump the version; even/odd for migration state |
|
||||
| [030](decisions/030-schema-change-detection-via-diff.md) | Schema change detection via Value.Diff | No manual changelog; diff stored vs current schemas |
|
||||
| [031](decisions/031-moduletodbschema-for-updates.md) | moduleToDbSchema() for schema updates | Re-run full Module projection, not Value.Patch |
|
||||
| [032](decisions/032-single-author-not-crdt.md) | Single-author model, not CRDT | No concurrent multi-author graph types |
|
||||
|
||||
## Open Questions
|
||||
|
||||
1. **Can `Edit[]` from `Value.Diff` be reliably classified as breaking vs
|
||||
non-breaking?** The classification table above is theoretical. A POC should
|
||||
validate whether the `Edit[]` output contains enough information to
|
||||
distinguish, for example, `String → Literal("x")` (narrowing, non-breaking)
|
||||
from `String → Number` (incompatible, breaking). Alternative: skip
|
||||
classification and just use `Value.Check(newSchema, storedData)` for
|
||||
verification.
|
||||
Open questions are tracked in [open-questions.md](open-questions.md). Key
|
||||
questions affecting schema evolution:
|
||||
|
||||
2. **Should the repository layer auto-migrate data on schema change, or
|
||||
require explicit consumer action?** Auto-migration is simpler for consumers
|
||||
but risky (data transformation without consumer awareness). Explicit
|
||||
migration is safer but more boilerplate. **Decision (conditional on OQ1
|
||||
POC outcome):** if classification is feasible (OQ1 POC succeeds), the
|
||||
repository layer auto-applies `Value.Cast` for changes it classifies as
|
||||
non-breaking, and requires explicit consumer action for breaking changes.
|
||||
If classification is not feasible, the fallback is: the repository layer
|
||||
auto-applies `Value.Cast` only when `Value.Check(newSchema, storedData)`
|
||||
passes for all stored data (verification, not classification), and requires
|
||||
explicit consumer action otherwise. This ensures auto-migration never
|
||||
corrupts data — if in doubt, the consumer decides.
|
||||
|
||||
3. **How does this interact with the hub's event-sourced call graph
|
||||
persistence?** If the hub migrates to event-sourced replay (projector
|
||||
evolution), storage's call graph tables become disposable projections and
|
||||
`Value.Cast` migration is unnecessary. But other graph types (ACL, tasks,
|
||||
secrets) may not have an event stream to replay from. The schema evolution
|
||||
design should work for both projections and direct-persisted data.
|
||||
|
||||
4. **Should schema evolution events be part of the event stream?** If the
|
||||
system is event-sourced, schema changes themselves could be events
|
||||
(`schema.updated`, `schema.version_bumped`). This would give a full audit
|
||||
trail of schema evolution, but adds complexity. **Decision: post-v1.** For
|
||||
v1, schema changes are applied directly via the repository layer with version
|
||||
tracking.
|
||||
- **OQ-10**: Can `Edit[]` from `Value.Diff` be reliably classified as breaking vs non-breaking?
|
||||
- **OQ-11**: Should the repository layer auto-migrate data on schema change, or require explicit consumer action?
|
||||
- **OQ-12**: How does schema evolution interact with the hub's event-sourced call graph persistence?
|
||||
- **OQ-13**: Should schema evolution events be part of the event stream?
|
||||
|
||||
## References
|
||||
|
||||
|
||||
Reference in New Issue
Block a user