- C-05: Add flowgraph-api.md with complete public API surface - C-06: Document <Map> component in workflow-templates.md - C-07: Specify Conditional else-branch behavior - C-08: Add lifecycle/ownership section to reactive-execution.md - C-09: Add consumer-integration.md end-to-end walkthrough - W-02: Add reactive error boundary semantics (3 levels) - W-03: Complete ReactiveContext interface definition - W-04: Add template composition rules (8 rules) - W-05: Document removeChild for both HostConfigs - W-06: Document signal/effect disposal lifecycle - W-07: Add ADR-004 (no schema version field) - W-08: Add type compatibility depth/contract to analysis.md - W-11: Add performance characteristics section - S-01: Getting Started merged into consumer-integration.md - S-02: Add flow diagrams for template rendering pipeline - S-03: Add node status state machine diagram - S-04: Add testing strategy section - S-06: Validate source structure cross-references Review round 2 fixes: - Define TemplateNodeAttrs as alias for OperationNodeAttrs - Document CallEventMapValue and CallResult types in schema.md - Standardize CycleError naming (replace CircularDependencyError) - Add function form to Map.over type definition - Define Map aggregate completion/failure semantics - Fix immutability claim for fromCallEvents - Clarify edgeType storage alongside OperationEdgeAttrs - Clarify WorkflowNode.status === statusMap (same Signal) - Add component-to-tag mapping for WorkflowTag
197 lines
12 KiB
Markdown
197 lines
12 KiB
Markdown
---
|
|
status: draft
|
|
last_updated: 2026-05-20
|
|
---
|
|
|
|
# @alkdev/flowgraph Architecture
|
|
|
|
Workflow graph library — DAG-based operation orchestration over graphology, with ujsx template composition and reactive execution.
|
|
|
|
## Why This Exists
|
|
|
|
Flowgraph fills the gap between the operation registry (`@alkdev/operations`) and the call graph observability layer (`@alkdev/alkhub`). Operations define *what can be called*. The call graph records *what was called*. Flowgraph defines *how calls are orchestrated* — the structure, validation, and execution of workflows.
|
|
|
|
Without flowgraph:
|
|
|
|
- **Workflows are ad-hoc** — the hub coordinator manually chains `registry.execute()` calls with no structural validation, no type checking between steps, and no way to reuse workflow patterns.
|
|
- **Call templates are hardcoded** — the SDD pipeline (architect → reviewer → decomposer → coordinator → specialist) is a recurring pattern with no reusable definition.
|
|
- **Abort cascading is manual** — when the 3rd of 5 operations fails, the coordinator must explicitly cancel the remaining operations. Flowgraph's DAG enables structural abort propagation.
|
|
- **No precondition checking** — there's no way to validate that operation A's output schema is compatible with operation B's input schema before attempting the call.
|
|
|
|
Flowgraph provides three conceptual graphs, each built for a different purpose:
|
|
|
|
1. **Operation Graph (Static)** — built from `OperationSpec`s at startup; nodes are operations, edges are type-compatibility relationships. Enables cycle detection, topological ordering, and call template validation.
|
|
|
|
2. **Call Graph (Dynamic)** — built at runtime from call events; nodes are call invocations with status and timestamps, edges are parent-child relationships. Enables abort cascading, observability, and DAG queries.
|
|
|
|
3. **Workflow Template (Declarative)** — a ujsx tree that defines a reusable workflow structure. A template is a validated path through the operation graph, instantiated as a call graph at runtime.
|
|
|
|
## Core Principle
|
|
|
|
**The graph is the specification. The template is the authoring surface. The call graph is the execution record.**
|
|
|
|
The operation graph provides static type checking and structural validation. The ujsx template provides human-readable, composable workflow definitions. The call graph captures what actually happened. Flowgraph is the bridge between all three.
|
|
|
|
## Relationship to Sibling Packages
|
|
|
|
| Package | Relationship |
|
|
|---------|-------------|
|
|
| `@alkdev/operations` | **Peer dependency**. Provides `OperationSpec`, `OperationRegistry`, `CallEvent`, `PendingRequestMap`. Flowgraph consumes operation types but does not depend on a specific runtime. |
|
|
| `@alkdev/ujsx` | **Direct dependency**. Workflow templates are `UNode` trees rendered through `HostConfig`s. Flowgraph provides the workflow-specific host configurations (graphology DAG, reactive execution). |
|
|
| `@alkdev/taskgraph` | **Pattern reference**. Flowgraph follows the same graphology-wrapping pattern (`FlowGraph` class like `TaskGraph` class) but enforces DAG invariants instead of allowing cycles. |
|
|
| `@alkdev/typebox` | **Direct dependency**. All schemas are TypeBox Modules. Runtime validation, JSON Schema export, and `Value.Check`/`Value.Errors`. |
|
|
| `@alkdev/pubsub` | **Optional peer dependency**. For event-driven call graph population. Flowgraph works in-memory; pubsub connects it to the call protocol. |
|
|
| `@alkdev/cograph` | **Future consumer**. The cognitive graph depends on flowgraph for workflow templates and execution tracking. |
|
|
|
|
## Current State
|
|
|
|
Flowgraph is in Phase 0/1 (exploration → architecture). No code exists yet. This architecture document set defines the WHAT and WHY before any implementation.
|
|
|
|
## Architecture Documents
|
|
|
|
| Document | Content |
|
|
|----------|---------|
|
|
| [schema.md](schema.md) | TypeBox Module, TypeScript types, enums (CallStatus, EdgeType, NodeStatus), node/edge attribute schemas, SerializedGraph factory |
|
|
| [operation-graph.md](operation-graph.md) | Static graph from OperationSpecs, type-compatibility edges, construction paths, validation |
|
|
| [call-graph.md](call-graph.md) | Dynamic graph from call events, node lifecycle, abort cascading, fromCallEvents construction |
|
|
| [workflow-templates.md](workflow-templates.md) | ujsx components (`<Operation>`, `<Sequential>`, `<Parallel>`, `<Conditional>`, `<Map>`), composition rules, template→DAG hydration, serialization |
|
|
| [host-configs.md](host-configs.md) | Graphology HostConfig (template→DAG analysis), Reactive HostConfig (template→execution engine), Instance types, removeChild |
|
|
| [reactive-execution.md](reactive-execution.md) | Signal-driven status propagation, computed preconditions, abort cascade via signals, ReactiveRoot integration, lifecycle and ownership, error boundaries |
|
|
| [analysis.md](analysis.md) | Type-compatibility checking (input/output schema matching), compatibility depth, precondition validation, execution ordering, performance characteristics |
|
|
| [error-handling.md](error-handling.md) | FlowgraphError hierarchy, CycleError, TypeIncompatError, ValidationError, error collection strategy |
|
|
| [build-distribution.md](build-distribution.md) | Package structure, exports map, dependencies, platform targets |
|
|
| [flowgraph-api.md](flowgraph-api.md) | FlowGraph class public API: constructor, type parameters, methods, delegation model, immutability guarantees |
|
|
| [consumer-integration.md](consumer-integration.md) | End-to-end walkthrough from operation specs to running workflow, common patterns, module dependency map |
|
|
|
|
### Design Decisions
|
|
|
|
| ADR | Decision |
|
|
|-----|----------|
|
|
| [001](decisions/001-ujsx-as-template-ir.md) | ujsx tree as workflow template intermediate representation |
|
|
| [002](decisions/002-dag-only-graph.md) | Enforce DAG invariants — no cycles in flowgraph |
|
|
| [003](decisions/003-storage-decoupled.md) | Storage is not flowgraph's concern — in-memory graph with export/import boundary |
|
|
| [004](decisions/004-no-schema-version.md) | No schema version field in serialized format — consumers wrap in their own versioned envelope |
|
|
|
|
## Consumer Context
|
|
|
|
### alkhub (hub-spoke coordinator)
|
|
|
|
The hub instantiates flowgraph to:
|
|
- Build the operation graph at startup from the registry
|
|
- Validate call templates before execution
|
|
- Populate call graphs at runtime from call protocol events
|
|
- Query call graphs for observability (what's running, what failed, what's blocked)
|
|
- Persist call graph state via `export()` → Postgres
|
|
|
|
### OpenCode Plugin (future)
|
|
|
|
An OpenCode plugin that provides workflow tools:
|
|
- `workflow.validate` — validate a template against the operation graph
|
|
- `workflow.run` — instantiate a template as a call graph and execute it
|
|
- `workflow.status` — query a running call graph
|
|
|
|
### Cograph (future)
|
|
|
|
The cognitive graph uses flowgraph's templates and operation graph to define procedural knowledge: which operations can be composed, what the valid execution paths are, and what preconditions each step requires.
|
|
|
|
## Source Structure
|
|
|
|
```
|
|
src/
|
|
component/ # ujsx components for workflow definition
|
|
operation.ts # <Operation name="classify" />
|
|
sequential.ts # <Sequential>...</Sequential>
|
|
parallel.ts # <Parallel>...</Parallel>
|
|
conditional.ts # <Conditional test={fn}>...</Conditional>
|
|
map.ts # <Map over={array} as="item">...</Map>
|
|
index.ts
|
|
host/
|
|
graphology.ts # HostConfig: ujsx tree → graphology DAG
|
|
reactive.ts # HostConfig: ujsx tree → reactive execution engine
|
|
schema/
|
|
enums.ts # CallStatus, NodeStatus, EdgeType, OperationType
|
|
node.ts # OperationNodeAttributes, CallNodeAttributes
|
|
edge.ts # OperationEdgeAttrs, CallEdgeAttrs, TemplateEdgeAttrs
|
|
graph.ts # FlowGraphSerialized (graphology export format)
|
|
index.ts
|
|
graph/
|
|
construction.ts # FlowGraph class (like TaskGraph)
|
|
validation.ts # Cycle detection, type-compat validation, precondition checks
|
|
queries.ts # topologicalOrder, ancestors, descendants, hasCycles
|
|
mutation.ts # updateNode, updateEdge, removeNode, removeEdge
|
|
index.ts
|
|
reactive/
|
|
workflow.ts # ReactiveRoot for workflow state
|
|
node-status.ts # Per-node status signals + computed preconditions + blockedByFailure
|
|
index.ts
|
|
analysis/
|
|
type-compat.ts # Schema compatibility checking between operation input/output
|
|
workflow.ts # Execution ordering, precondition resolution, path validation
|
|
defaults.ts # Default status, edge type, etc.
|
|
index.ts
|
|
error/
|
|
index.ts # FlowgraphError, CycleError, TypeIncompatError, ValidationError
|
|
index.ts # Barrel export
|
|
```
|
|
|
|
## Key Design Decisions
|
|
|
|
### 1. ujsx as Template IR
|
|
|
|
Workflow templates are `UNode` trees. This gives us:
|
|
- **Composability** — `<Sequential>` and `<Parallel>` compose naturally as parent-child structure
|
|
- **Serialization** — `UNode` trees are JSON, trivially stored and transmitted
|
|
- **Host targets** — the same template renders to a graphology DAG (analysis) or a reactive execution engine (runtime) via different `HostConfig` implementations
|
|
- **Reconciler support** — incremental template updates via ujsx's reconciler (add/remove/reorder steps without full rebuild)
|
|
|
|
This is a design decision worth documenting because it's a non-obvious choice. The alternative is to define templates as plain data structures (arrays of step objects), which is simpler but loses composability, host switching, and reconciler benefits.
|
|
|
|
### 2. DAG-Only, No Cycles
|
|
|
|
Flowgraph enforces acyclicity. The `FlowGraph` class rejects cycle-creating edges at mutation time (unlike `TaskGraph`, which allows cycles and detects them via `hasCycles()`). This is because:
|
|
|
|
- **Operation graphs** represent type flow — a cycle means an operation's output feeds back into its own input, which is almost certainly a design error.
|
|
- **Call graphs** represent execution order — cycles are physically impossible (you can't have a call that is its own ancestor).
|
|
- **Workflow templates** represent validated paths through the operation graph — they must be DAGs by construction.
|
|
|
|
### 3. Storage is Decoupled
|
|
|
|
Flowgraph handles in-memory graph construction, validation, and analysis. Persistence is the caller's concern. The `export()`/`fromJSON()` boundary provides a clean serialization format (graphology native JSON) that the hub can store in Postgres. This follows the same pattern as taskgraph.
|
|
|
|
### 4. Template → DAG → Execution is a Pipeline, Not a Monolith
|
|
|
|
The three representations serve different phases:
|
|
- **Template** (ujsx tree) → authoring, composition, serialization
|
|
- **DAG** (graphology) → validation, type checking, topological ordering
|
|
- **Execution** (reactive signals) → runtime status tracking, preconditions, abort propagation
|
|
|
|
Each can exist independently. You can validate a template without executing it. You can build a call graph from events without a template. You can run a reactive workflow directly from a DAG.
|
|
|
|
## Document Lifecycle
|
|
|
|
Architecture documents use YAML frontmatter with `status` and `last_updated` fields:
|
|
|
|
```yaml
|
|
---
|
|
status: draft | stable | deprecated
|
|
last_updated: YYYY-MM-DD
|
|
---
|
|
```
|
|
|
|
| Status | Meaning | Transitions |
|
|
|--------|---------|-------------|
|
|
| `draft` | Under active development. Content may change significantly. Implementation should not start until the document reaches `stable`. | → `stable` when implementation is complete and API contract is verified by tests. |
|
|
| `stable` | API contracts are locked. Changes require a review cycle and may warrant an ADR if they affect documented decisions. | → `deprecated` when superseded. → `draft` if a fundamental redesign is needed (rare). |
|
|
| `deprecated` | Superseded by another document. Kept for reference. Links should point to the replacement. | Removed when no longer referenced. |
|
|
|
|
ADR documents use a separate `Status` field in their body: `Proposed`, `Accepted`, `Deprecated`, or `Superseded`. ADRs never revert from `Accepted`.
|
|
|
|
## References
|
|
|
|
- Call protocol architecture: `@alkdev/alkhub_ts/docs/architecture/call-graph.md`
|
|
- Call graph storage schema: `@alkdev/alkhub_ts/docs/architecture/storage/call-graph.md`
|
|
- Operation types and registry: `@alkdev/operations/src/types.ts`, `@alkdev/operations/src/registry.ts`
|
|
- ujsx architecture: `@alkdev/ujsx/docs/architecture/`
|
|
- ujsx research on flowgraph HostConfigs: `@alkdev/ujsx/docs/research/reconciler/05-flowgraph-host-configs.md`
|
|
- Taskgraph architecture: `@alkdev/taskgraph_ts/docs/architecture/`
|
|
- SDD process: `docs/sdd_process.md` |