Files
flowgraph/docs/architecture/workflow-templates.md
glm-5.1 f3e084d02f resolve all remaining open questions (OQ-03–OQ-29), add ADR-006
Resolve all 19 remaining open questions across the architecture. Every
question now has a documented resolution with rationale:

- OQ-004/OQ-029: edgeType is a universal required attribute on all edges,
  single graph per FlowGraph instance (ADR-006)
- OQ-011: No OR preconditions for v1; preconditionMode as v2 extension
- OQ-012: maxConcurrency enforced via reactive counting semaphore
- OQ-014: Unknown operationId creates node with pending status
- OQ-017: Expose common graphology traversal methods on FlowGraph (80/20)
- OQ-020: condition as Type.Unknown() with string/function documentation
- OQ-022: Identity imported from @alkdev/operations peer dep
- All other questions resolved with documented rationale

Fix three critical issues found by architecture review:
1. edgeType serialization/validation gap: document two-step validation
2. CallEdgeAttrs runtime discrimination: edgeType as runtime discriminant,
   depends_on edges clarified as observability-only (not execution)
3. ADR-005 signal mutation inconsistency: explicitly distinguish call-level
   statuses (event-log-driven) from workflow-derived statuses (signal-mutation)

Additional clarifications:
- dataFlow inference uses conservative strategy (defaults false)
- Conditional.test string resolution: operationName → status === completed
- Add negated field to TemplateEdgeAttrs for else-branch conditions
- Document edge key priority convention for composite keys
- Add maxConcurrency semaphore design to reactive-execution.md
2026-05-21 09:25:55 +00:00

407 lines
24 KiB
Markdown

---
status: draft
last_updated: 2026-05-22
---
# Workflow Templates
ujsx-based workflow definition — compose operations as declarative template trees, render them to DAGs or reactive execution engines.
## Overview
Workflow templates are ujsx trees that define reusable call patterns. Instead of hardcoding operation sequences in the hub coordinator, templates provide a declarative, composable way to define "what should happen in what order":
```typescript
import { h, createRoot } from "@alkdev/ujsx";
import { Operation, Sequential, Parallel, Conditional, Map } from "@alkdev/flowgraph/component";
import { GraphologyHostConfig } from "@alkdev/flowgraph/host/graphology";
const sddPipeline = h(Sequential, {},
h(Operation, { name: "architect" }),
h(Operation, { name: "architecture-reviewer" }),
h(Conditional, {
test: (results) => results["architecture-reviewer"].approved
},
h(Sequential, {},
h(Operation, { name: "decomposer" }),
h(Operation, { name: "coordinator" }),
h(Operation, { name: "specialist" }),
),
),
h(Operation, { name: "code-reviewer" }),
);
```
The template is a `UNode` tree — a plain data structure that can be:
- **Serialized** to JSON for storage and transmission
- **Validated** against the operation graph (are all referenced operations registered? are there type mismatches?)
- **Rendered to a graphology DAG** via the `GraphologyHostConfig` for structural analysis
- **Rendered to a reactive execution engine** via the `ReactiveHostConfig` for runtime status tracking
This is the same `UNode` tree that ujsx defines, with flowgraph-specific component functions (`Operation`, `Sequential`, `Parallel`, `Conditional`, `Map`) that produce `UElement` nodes with workflow-specific props and meaning.
## Why ujsx as Template IR
The alternative to ujsx would be a custom template format — an array of step objects with type discriminators:
```typescript
// Alternative: custom template format
const template = [
{ type: "operation", name: "architect" },
{ type: "sequential", steps: [
{ type: "operation", name: "decomposer" },
{ type: "operation", name: "coordinator" },
]},
];
```
ujsx is better for several reasons:
1. **Composability** — Nested elements are the natural representation of hierarchical workflows. `Sequential({ children: [...] })` is cleaner than a recursive type discriminator.
2. **No new format** — ujsx already defines the tree structure, type guards, reactive layer, and reconciler. We don't need to design, implement, and maintain a template parser/serializer.
3. **Host target switching** — The `HostConfig` pattern means the same template renders to different targets without template-specific logic. Graphology for analysis, reactive engine for runtime. No template→IR→DAG compilation step.
4. **Incremental updates** — The ujsx reconciler enables incremental template updates. Add a step, remove a step, reorder steps — the reconciler computes the diff and applies minimal mutations to the DAG, rather than rebuilding the entire graph.
5. **Reactive props**`@preact/signals-core` enables signal-driven prop updates. An `Operation` node's `name` could be a signal, enabling dynamic workflow modification at runtime.
See [ADR-001](decisions/001-ujsx-as-template-ir.md) for the full decision record.
## Component Definitions
### `<Operation>`
Represents a single operation invocation in the workflow:
```typescript
const Operation: UComponent<{
name: string; // Operation name (namespace.name or just name if namespace is inherited)
input?: unknown; // Static input for the call
retries?: number; // Number of retries on failure (default: 0)
timeout?: number; // Deadline in ms from call start
}>;
```
`Operation` produces a `UElement` with `type: "operation"` and the given props. When rendered to a graphology DAG, it creates a node with the operation's attributes. When rendered to the reactive engine, it creates a `signal<NodeStatus>` that tracks the call's lifecycle.
### `<Sequential>`
Represents sequential execution — children run in order, each child waits for the previous to complete:
```typescript
const Sequential: UComponent<{
id?: string; // Optional identifier for the sequence
}>;
```
`Sequential` children are rendered in order. In the graphology DAG, each child has a `sequential` edge to the next child. In the reactive engine, each child's precondition is "previous child is `completed`".
### `<Parallel>`
Represents parallel execution — all children start simultaneously:
```typescript
const Parallel: UComponent<{
id?: string; // Optional identifier for the parallel group
maxConcurrency?: number; // Maximum concurrent children (default: unlimited)
}>;
```
`Parallel` children have no ordering edges between them. In the reactive engine, all children's preconditions are "parent's prerequisites are met", so they all become `ready` at the same time.
`maxConcurrency` is a runtime hint, not a structural constraint. The DAG doesn't encode it — it's a scheduling hint for the execution engine.
### `<Conditional>`
Represents conditional branching — children only execute if the test passes:
```typescript
const Conditional: UComponent<{
test: ((results: Record<string, CallResult>) => boolean) | string;
// If string: operation name whose result to check
// If function: receives results of previous steps, returns boolean
else?: UNode; // Alternative branch if test fails
}>;
```
When rendered to a graphology DAG, `Conditional` creates an edge with `edgeType: "conditional"` and `dataFlow: true` (conditional edges always carry data — the test reads a predecessor's result). When rendered to the reactive engine, the condition is evaluated as a `computed` that depends on the result projection (from the event log per ADR-005).
If the test evaluates to `false` and no `else` branch is provided, the branch nodes transition to `skipped` in `NodeStatus`.
#### String condition resolution
When `Conditional.test` is a string (rather than a function), the HostConfig resolves it at render time using the operation registry. The resolution algorithm is:
- `test: "operationName"` → resolves to `(results) => results["operationName"]?.status === "completed"`, meaning "the then-branch is taken if the referenced operation completed successfully."
- If the referenced operation failed or was aborted, the condition evaluates to `false` and the else-branch is taken (or the then-branch is `skipped` if no else-branch).
- String conditions can only reference predecessor operations by name. For more complex conditions (checking output fields, combining multiple results, etc.), use the function form.
This resolution algorithm is deterministic and produces the same behavior regardless of which HostConfig performs the resolution.
#### Else-branch behavior
When the `else` prop is provided, the `Conditional` renders two subgraphs:
**DAG rendering (GraphologyHostConfig)**:
- The `then` branch (child) renders with an edge from the conditional's predecessor to the first child, with `edgeType: "conditional"` and `condition: <test>`.
- The `else` branch renders as a separate subgraph with `edgeType: "conditional"`, `condition: <test>`, and `negated: true`. The `negated` flag on `TemplateEdgeAttrs` indicates that the condition is logically negated for the else-branch. At render time, the HostConfig resolves the negation differently depending on the condition form:
- **String condition**: `condition: "fetch-data"` with `negated: true` resolves to `(results) => results["fetch-data"]?.status !== "completed"`.
- **Function condition**: The HostConfig wraps the original function: `condition: (results) => !originalTest(results)`.
- This ensures the else-branch is taken when the original condition evaluates to `false`, regardless of condition form.
- Both branches share the same predecessor — the `Conditional` node's structural position in the template determines the common starting point.
**Reactive rendering (ReactiveHostConfig)**:
- When `test` evaluates to `true`: `then`-branch nodes become `ready` (preconditions met). `else`-branch nodes transition to `skipped`. Their `preconditions` are satisfied by the `skipped` state — downstream nodes see the `Conditional` as completed regardless of which branch was taken.
- When `test` evaluates to `false`: `else`-branch nodes become `ready`. `then`-branch nodes transition to `skipped`. Downstream nodes after the `Conditional` see all branches as resolved.
- When no `else` prop is provided: the `false` branch simply doesn't exist. Nodes after the `Conditional` that depend on it still see it as `completed` because the `Conditional` itself resolves regardless of which path is taken.
- The `test` function receives its data from the **result projection** (ADR-005). `results["nodeName"]` reads from `getResult("nodeName")`, which derives from the event log. This ensures retries are reflected — if a node is retried, its result updates when the retry's `call.responded` event arrives.
This means a `Conditional` with an `else` branch acts as a **complete error boundary** — downstream nodes are insulated from the branch choice. The `Conditional` is `completed` whether the `then` or `else` branch executed.
### `<Map>`
Represents mapping over an array — creates one child instance per array item:
```typescript
const Map: UComponent<{
over: Signal<unknown[]> | unknown[] | ((results: Record<string, CallResult>) => unknown[]);
// Static array, signal, or function that resolves against predecessor results
as: string; // Variable name for each item
children: UNode; // Template rendered per item
}>;
```
The `<Map>` component dynamically replicates its child template for each element in the `over` array. Each replica gets the current element bound to the variable named by `as`.
**DAG rendering (GraphologyHostConfig)**:
- For each item in `over`, renders a copy of the child template as a node.
- Each mapped node has a `sequential` edge from the `Map`'s predecessor (all mapped nodes start at the same point, like `Parallel`).
- Mapped nodes are named with a composite key: `${parentKey}.${as}[${index}]`. For example, `<Map over={items} as="item">` with 3 items creates nodes `item[0]`, `item[1]`, `item[2]`.
- The `Map` container itself is transparent in the graph (no node for the container).
**Reactive rendering (ReactiveHostConfig)**:
- For each item in `over`, creates a `WorkflowNode` with its own `signal<NodeStatus>` and `computed` preconditions.
- All mapped nodes' preconditions are identical: the `Map`'s predecessor must be `completed` (same as `Parallel`).
- Each mapped node's result is available from the **result projection** (ADR-005). `getResult(nodeKey)` derives from the event log.
- The `Map` result is available as an aggregated computed containing all mapped nodes' results from the result projection.
**Example**:
```typescript
h(Sequential, {},
h(Operation, { name: "fetch-items" }),
h(Map, {
over: (results) => results["fetch-items"].output.items,
as: "item"
},
h(Operation, { name: "process-item", input: (results, { item }) => item }),
),
)
```
This creates a `Sequential` where `process-item` is called once per item returned by `fetch-items`. Each call gets its corresponding item as input.
**Edge type**: Mapped children use the same `TemplateEdgeAttrs` as `Parallel` children (no `sequential` edges between siblings). The `Map` component is structurally equivalent to a `Parallel` group where the children are dynamically generated from an array.
**Aggregate completion semantics**: A `Map` node's status follows "worst-case" semantics:
- If **all** mapped nodes reach a satisfying terminal state (`completed` or `skipped`), the `Map` is considered `completed`.
- If **any** mapped node reaches `failed`, the `Map` is considered `failed` (unless caught by a `Conditional`).
- If **any** mapped node reaches `aborted`, the `Map` is considered `aborted`.
- Downstream nodes whose preconditions include the `Map` will see `blockedByFailure = true` if the `Map` has any `failed` or `aborted` children.
This means a `Map` with partial failure (some nodes succeeded, one failed) propagates as a failure to downstream dependents. If partial success is needed, the template author should use a `Conditional` to handle the failure case, or process the `Map` results individually rather than treating the `Map` as a single dependency.
**Reactive behavior for mapped nodes**:
- All mapped nodes become `ready` simultaneously when the `Map`'s predecessor completes (parallel start).
- If any mapped node fails, only that node transitions to `failed`. Other mapped nodes continue independently (failure follows dependency edges, not structural scope, consistent with `Parallel` behavior).
- Mapped nodes participate in failure propagation like any other node: downstream dependents see `blockedByFailure` if a mapped node fails and they depend on it.
## Template → DAG Conversion
The `GraphologyHostConfig` renders a template to a graphology DAG:
```typescript
import { createRoot } from "@alkdev/ujsx";
import { GraphologyHostConfig } from "@alkdev/flowgraph/host/graphology";
const host = new GraphologyHostConfig();
const root = createRoot(host, new DirectedGraph());
const template = h(Sequential, {},
h(Operation, { name: "architect" }),
h(Operation, { name: "reviewer" }),
h(Operation, { name: "decomposer" }),
);
root.render(template);
// Now root.ctx is a DirectedGraph with:
// - nodes: "architect", "reviewer", "decomposer"
// - edges: "architect" → "reviewer" → "decomposer" (sequential)
```
The HostConfig maps ujsx component types to graphology operations:
| UElement type | Graphology operation |
|---------------|---------------------|
| `"operation"` | Add node with `OperationNodeAttrs` |
| `"sequential"` | Add `sequential` edges between consecutive children |
| `"parallel"` | No edges between children (they run concurrently) |
| `"conditional"` | Add `conditional` edge with test attribute |
### Edge creation rules
- **Sequential**: For children C1, C2, ..., Cn, edges C1→C2, C2→C3, ..., C(n-1)→Cn are added. Each edge carries `edgeType: "sequential"`. If the downstream node references the upstream node's result (via `Conditional.test`, `Map.over`, or `Operation.input`), the edge also carries `dataFlow: true`. Otherwise, `dataFlow: false` (temporal ordering only).
- **Parallel**: No edges between children. All children have the same preconditions as the parallel group itself.
- **Conditional**: Edge from the conditional node's prerequisite to the first child of the branch, with `edgeType: "conditional"` and `dataFlow: true` (conditional edges always carry data — the condition reads a predecessor's result).
- **Nested**: A `Sequential` inside a `Parallel` has its own internal edges. A `Parallel` inside a `Sequential` creates a subgraph where all parallel children share the same predecessor.
### Root node handling
The template's root `URoot` is transparent — its children are mounted directly into the graph. `Sequential` and `Parallel` component functions are also transparent in terms of graph structure — they produce edges between their children, but do not create nodes for themselves.
This means a template like:
```typescript
h(Sequential, {},
h(Operation, { name: "A" }),
h(Parallel, {},
h(Operation, { name: "B" }),
h(Operation, { name: "C" }),
),
h(Operation, { name: "D" }),
);
```
Produces a DAG with nodes A, B, C, D and edges A→B, A→C, B→D, C→D. No "parallel" or "sequential" nodes.
## Template → Reactive Execution
The `ReactiveHostConfig` renders a template to a reactive execution engine:
```typescript
import { createRoot } from "@alkdev/ujsx";
import { ReactiveHostConfig } from "@alkdev/flowgraph/host/reactive";
const host = new ReactiveHostConfig(operationRegistry, workflowRoot);
const root = createRoot(host, {});
const template = h(Sequential, {},
h(Operation, { name: "architect" }),
h(Operation, { name: "reviewer" }),
);
root.render(template);
// Now each operation node has a signal<NodeStatus>:
// - "architect": signal("idle")
// - "reviewer": signal("idle")
// The reviewer's precondition is: architect.status === "completed"
```
See [reactive-execution.md](reactive-execution.md) for the full reactive execution architecture.
## Serialization
Since workflow templates are ujsx `UNode` trees, they are JSON-serializable by design:
```typescript
import { Value } from "@alkdev/typebox/value";
import { UJSX } from "@alkdev/ujsx";
const template = h(Sequential, {},
h(Operation, { name: "architect" }),
h(Operation, { name: "reviewer" }),
);
// Serialize
const json = JSON.stringify(template);
// → {"type":"sequential","props":{"name":"sequential"},"children":[...]}
// Deserialize
const parsed = JSON.parse(json);
if (Value.Check(UJSX.Import("UElement"), parsed)) {
// Valid UElement — can render to any HostConfig
}
```
Note: function-valued props (like `Conditional.test` with a function) are not serializable. For storage, conditional tests must be expressed as strings (operation references) rather than functions. The HostConfig resolves string references to functions at render time.
## Validation
A workflow template can be validated against an operation graph before execution:
```typescript
function validateTemplate(
template: UNode,
operationGraph: FlowGraph<OperationNodeAttrs, OperationEdgeAttrs>,
): ValidationError[]
```
Validation checks:
1. **All operation names exist in the registry** — every `<Operation name="X">` must have a matching node in the operation graph
2. **Type compatibility** — sequential operations have type-compatible edges in the operation graph
3. **No cycles** — the rendered DAG has no cycles (inherited from FlowGraph's DAG enforcement)
4. **Reachability from start** — all operations in the template are reachable from the first operation
Validation returns an array of `ValidationError` objects (never throws). See [analysis.md](analysis.md) for the full validation algorithm.
## Composition Rules
Not all component combinations are valid. The following rules govern which components can appear as children of which:
| Parent | Valid children | Notes |
|--------|---------------|-------|
| `Sequential` | `Operation`, `Sequential`, `Parallel`, `Conditional`, `Map` | Children execute in order |
| `Parallel` | `Operation`, `Sequential`, `Parallel`, `Conditional`, `Map` | Children execute concurrently |
| `Conditional` (then) | `Operation`, `Sequential`, `Parallel`, `Map` | Single child or wrapped in structural container |
| `Conditional` (else) | `Operation`, `Sequential`, `Parallel`, `Map` | Single child or wrapped in structural container |
| `Map` | `Operation`, `Sequential`, `Parallel`, `Conditional` | Template rendered per item |
### Rules
1. **`Operation` has no children** — an `Operation` is a leaf node. Nesting inside `Operation` is a template validation error.
2. **`Conditional` takes a single then-child via children, and optional else via `else` prop** — the `children` of `Conditional` are the then-branch. The `else` prop is the alternative branch. Both branches can be single `Operation` nodes or structural containers (`Sequential`, `Parallel`, `Map`).
3. **`Conditional.test` cannot reference an `Operation` inside the Conditional** — the test evaluates results from predecessor operations, not from the conditional branch itself. This would create a circular dependency.
4. **`Map.over` must be a serializable expression or signal** — the array can be a static value, a signal, or a function that receives results from predecessor operations. Function-valued `over` props don't survive JSON round-trips (same limitation as `Conditional.test`).
5. **`Sequential` with one child is valid but degenerate** — it produces no edges (no sequential ordering needed). A single-child `Sequential` is equivalent to the child alone.
6. **`Parallel` with one child is valid but degenerate** — it produces no edges (no concurrency needed). A single-child `Parallel` is equivalent to the child alone.
7. **Nesting is allowed to any depth**`Sequential` inside `Parallel` inside `Sequential` is valid. The DAG flattens nesting into edges between leaf `Operation` nodes.
8. **Template root must be a structural container** — the root element must be `Sequential`, `Parallel`, or `Map`. A bare `Operation` as root is technically valid but produces a single-node DAG with no edges.
## Constraints
- **Templates are ujsx trees** — no custom format, no parser, no compiler. Components are `UComponent` functions that produce `UElement` nodes.
- **`Operation` props are workflow metadata** — `name`, `input`, `retries`, `timeout` are NOT passed to the HostConfig's `createInstance`. They're workflow-level configuration that the reactive execution engine uses to configure the call.
- **Function props are not serializable** — `Conditional.test` with a function cannot be round-tripped through JSON. Use string references for stored templates.
- **Sequential ordering is structural, not temporal** — a `Sequential` group means "these operations should complete in order", not "start the next only after the previous completes" (though the reactive engine implements this via preconditions).
- **Parallel has no structural edges** — a `Parallel` group produces no DAG edges between its children. The execution engine starts them concurrently when the group's preconditions are met.
- **Conditional branches are either/or** — a `Conditional` node renders to one branch or the `else` branch. There's no "both" evaluation.
## Open Questions
1. ~~**Should `Sequential` and `Parallel` be transparent in the graph?**~~ **Resolved (OQ-05)**: Containers stay transparent. No nodes for `Sequential`, `Parallel`, or `Conditional` in the DAG. Aggregate status for containers is computed as a projection from children's statuses. The `parentMap` and `siblingMap` in `ReactiveContext` provide the structural context for precondition computation.
2. ~~**Should templates support loops?**~~ **Resolved**: The `<Map>` component provides array iteration — one child per array element. It does NOT support general loops (while, do-while). For repeated execution with conditional exit, use `Conditional` inside a `Sequential` group. General-purpose loops are not supported because they would require cycle-supporting templates, which conflicts with the DAG-only invariant.
3. ~~**Should templates support `depends_on` edges explicitly?**~~ **Resolved (OQ-021)**: No for v1. ADR-005's `dataFlow` inference and the result projection make explicit `depends_on` unnecessary for current use cases. Data dependencies are expressed through the result projection — if B needs A's output, B reads `getResult("A")`. The `dataFlow: true` attribute on edges captures which edges carry data. An explicit `<DependsOn>` component would add template syntax complexity and potentially conflict with structural ordering. If a future use case requires non-adjacent data dependencies that can't be expressed by restructuring the template, `<DependsOn>` can be added as a v2 extension. But v1 intentionally restricts dependencies to follow the structural flow.
4. ~~**How does template instantiation interact with the call protocol?**~~ **Resolved (ADR-005)**: The template bridges to the call protocol through the event log. The hub coordinator appends call protocol events; the reactive layer projects them. Each `<Operation>` node's `requestId` maps to call protocol events via the `nodeKeyToRequestId` map. No callback, no boomerang — the event log is the bridge.
## References
- ujsx architecture: `@alkdev/ujsx/docs/architecture/`
- ujsx HostConfig: `@alkdev/ujsx/docs/architecture/host-config.md`
- ujsx reactive layer: `@alkdev/ujsx/docs/architecture/reactive-layer.md`
- Host configs: [host-configs.md](host-configs.md)
- Reactive execution: [reactive-execution.md](reactive-execution.md)
- Analysis and validation: [analysis.md](analysis.md)