glm-5.1
de7fc88f99
- Remove src/call.ts (PendingRequestMap, CallEventSchema, CallError) — call protocol belongs in @alkdev/operations
- Add EventEnvelope type ({ type, id, payload }) as the cross-platform serialization contract
- Simplify createPubSub: replace PubSubPublishArgsByKey tuple model with PubSubEventMap; publish(type, id, payload) and subscribe(type, id) use explicit id for topic scoping
- Update Redis adapter to serialize/deserialize full EventEnvelope
- Expand operators: add take, reduce, toArray, batch, dedupe, window, flat, groupBy, chain, join
- Remove @alkdev/typebox runtime dependency (was only used by call.ts)
- Remove ./call sub-path export from package.json and tsup config
- Update all architecture docs to reflect transport-only scope, add Worker adapter, remove call protocol references
- Remove docs/architecture/call-protocol.md
- Update AGENTS.md with new source layout and transport-only principle
108 lines
6.7 KiB
Markdown
108 lines
6.7 KiB
Markdown
---
|
|
status: draft
|
|
last_updated: 2026-05-01
|
|
---
|
|
|
|
# @alkdev/pubsub Architecture
|
|
|
|
Type-safe publish/subscribe with pluggable event target adapters. The core (`createPubSub` + `TypedEventTarget` + `EventEnvelope` + operators) has no transport dependency. Each adapter (Redis, WebSocket, Worker, Iroh) is an isolated module that only imports its own peer dependency.
|
|
|
|
This package is a **transport layer only**. It carries events between processes and does not prescribe what those events mean or how downstream systems coordinate. Higher-level protocols (call/response, operation invocation, workflow coordination) belong in downstream packages like `@alkdev/operations`.
|
|
|
|
## Core Principle
|
|
|
|
**The TypedEventTarget interface is the contract.** All transports implement the same `addEventListener` / `dispatchEvent` / `removeEventListener` surface. `createPubSub` doesn't know or care which transport is in use — it just dispatches events to whatever `TypedEventTarget` it was given.
|
|
|
|
**The EventEnvelope is the cross-platform format.** Every event dispatched through pubsub is `{ type, id, payload }`. This is a minimal, JSON-serializable envelope that any transport adapter can route and any downstream consumer can interpret. Domain-specific data lives in `payload`. Correlation lives in `id`. The event type lives in `type`. No `parent` field — causal relationships are managed by downstream coordination layers, not the transport.
|
|
|
|
Swapping transports is a one-line config change:
|
|
|
|
```ts
|
|
const pubsub = createPubSub<MyEventMap>({
|
|
eventTarget: createRedisEventTarget({ publishClient, subscribeClient }),
|
|
});
|
|
```
|
|
|
|
## Why This Exists
|
|
|
|
Extracted from `@alkdev/alkhub_ts/packages/core/pubsub/`, which itself was adapted from `@graphql-yoga/subscription` and `@graphql-yoga/typed-event-target`. The pubsub module was already self-contained within alkhub — zero cross-module imports from operations, config, logger, or MCP. Extracting it into a standalone package:
|
|
|
|
1. **Reduces coupling** — alkhub depends on pubsub, not the other way around
|
|
2. **Enables reuse** — multiple alkhub packages can share the same pubsub instance
|
|
3. **Isolates peer deps** — Redis and Iroh are heavy native dependencies; consumers that don't need them shouldn't carry them
|
|
4. **Matches established pattern** — `@alkdev/taskgraph` and `@alkdev/typemap` already use the standalone-package pattern
|
|
|
|
## What This Package Provides
|
|
|
|
- **Core** — `createPubSub`, `TypedEventTarget`, `TypedEvent`, `EventEnvelope`, stream operators (`filter`, `map`, `pipe`, `take`, `reduce`, `toArray`, `batch`, `dedupe`, `window`, `flat`, `groupBy`, `chain`, `join`), `Repeater` (inlined from @repeaterjs/repeater)
|
|
- **Adapters** (each is a peer-dep island, importable via sub-path export):
|
|
- In-process (default `EventTarget`, no adapter needed)
|
|
- Redis (`@alkdev/pubsub/event-target-redis`, peer dep: `ioredis`)
|
|
- WebSocket (future: `@alkdev/pubsub/event-target-websocket`)
|
|
- Worker (future: `@alkdev/pubsub/event-target-worker`)
|
|
- Iroh (future: `@alkdev/pubsub/event-target-iroh`, peer dep: `@rayhanadev/iroh`)
|
|
|
|
## What This Package Does NOT Provide
|
|
|
|
- **Call protocol** — request/response coordination, `PendingRequestMap`, `CallEventSchema`, and `CallError` have been moved to `@alkdev/operations`. The pubsub transport is substrate-agnostic.
|
|
- **Workflow coordination** — causal chains, parent/child relationships, and abort cascading are domain-level concerns managed by downstream packages.
|
|
- **Abort/cancellation primitives** — these belong in the coordination layer, not the transport. The `EventEnvelope` intentionally omits a `parent` field to avoid conflating transport with coordination semantics.
|
|
|
|
## Consumer Context
|
|
|
|
### alkhub (hub-spoke coordinator)
|
|
|
|
The hub uses pubsub for event routing between operations, runners, and the SSE interface. Transport choice depends on deployment:
|
|
|
|
| Deployment | Transport |
|
|
|------------|-----------|
|
|
| Single-process hub | In-process (default) |
|
|
| Hub + worker processes | Redis |
|
|
| Hub + remote spokes | WebSocket or Iroh |
|
|
|
|
### Downstream packages
|
|
|
|
- `@alkdev/operations` uses `createPubSub` with its own event maps for call/response coordination. It defines its own event schemas and `PendingRequestMap` on top of the pubsub transport.
|
|
- `@alkdev/taskgraph` will use pubsub events for task lifecycle notifications and workflow coordination.
|
|
|
|
## Threat Model
|
|
|
|
- **Fork provenance** — core pubsub and typed event target are adapted from graphql-yoga (MIT). All original copyright notices are preserved in file headers. See [ADR-001](decisions/001-graphql-yoga-fork.md).
|
|
- **Peer dep isolation** — Redis and Iroh are optional peer dependencies. A consumer that only needs in-process transport installs zero extra packages. A consumer using Redis but not Iroh installs `ioredis` only.
|
|
- **Type-only imports** — `event-target-redis.ts` imports `ioredis` types only at compile time. At runtime, the consumer must provide the actual `Redis`/`Cluster` instances.
|
|
- **Minimal envelope** — the `EventEnvelope` format (`{ type, id, payload }`) is intentionally minimal and JSON-serializable. Any platform that supports JSON can produce or consume these events (Rust, Python, etc.).
|
|
|
|
## Architecture Documents
|
|
|
|
| Document | Content |
|
|
|----------|---------|
|
|
| [api-surface.md](api-surface.md) | createPubSub factory, EventEnvelope, PubSub types, operators, TypedEventTarget types |
|
|
| [event-targets.md](event-targets.md) | In-process, Redis, WebSocket, Worker adapters — interface, configuration, limitations |
|
|
| [iroh-transport.md](iroh-transport.md) | Iroh P2P QUIC transport — protocol, framing, identity, hub/spoke sides, reconnection |
|
|
| [build-distribution.md](build-distribution.md) | Dependencies, project structure, tree-shaking, sub-path exports, targets |
|
|
|
|
## Document Lifecycle
|
|
|
|
Architecture documents use YAML frontmatter with `status` and `last_updated` fields:
|
|
|
|
```yaml
|
|
---
|
|
status: draft | stable | deprecated
|
|
last_updated: YYYY-MM-DD
|
|
---
|
|
```
|
|
|
|
| Status | Meaning | Transitions |
|
|
|--------|---------|-------------|
|
|
| `draft` | Under active development. Content may change. | → `stable` when implementation is complete and tests verify API contract. |
|
|
| `stable` | API contracts are locked. Changes require review cycle. | → `deprecated` when superseded. |
|
|
| `deprecated` | Superseded. Kept for reference. | Removed when no longer referenced. |
|
|
|
|
## References
|
|
|
|
- Source: `@alkdev/alkhub_ts/packages/core/pubsub/`
|
|
- Upstream: `@graphql-yoga/subscription` and `@graphql-yoga/typed-event-target` (MIT)
|
|
- alkhub pubsub-redis doc: `@alkdev/alkhub_ts/docs/architecture/pubsub-redis.md`
|
|
- alkhub spoke-runner doc: `@alkdev/alkhub_ts/docs/architecture/spoke-runner.md`
|
|
- Migration research: `docs/research/migration.md`
|
|
- Research: Event sourcing types — `docs/research/event_sourcing/` (not in this repo, in global workspace) |