alkdev/pubsub
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Simplify to transport-only: remove call protocol, add EventEnvelope, expand stream operators

Browse Source 
- Remove src/call.ts (PendingRequestMap, CallEventSchema, CallError) — call protocol belongs in @alkdev/operations
- Add EventEnvelope type ({ type, id, payload }) as the cross-platform serialization contract
- Simplify createPubSub: replace PubSubPublishArgsByKey tuple model with PubSubEventMap; publish(type, id, payload) and subscribe(type, id) use explicit id for topic scoping
- Update Redis adapter to serialize/deserialize full EventEnvelope
- Expand operators: add take, reduce, toArray, batch, dedupe, window, flat, groupBy, chain, join
- Remove @alkdev/typebox runtime dependency (was only used by call.ts)
- Remove ./call sub-path export from package.json and tsup config
- Update all architecture docs to reflect transport-only scope, add Worker adapter, remove call protocol references
- Remove docs/architecture/call-protocol.md
- Update AGENTS.md with new source layout and transport-only principle
This commit is contained in:
glm-5.1
2026-05-01 19:40:25 +00:00
parent 04b3464c36
commit de7fc88f99
17 changed files with 446 additions and 764 deletions
Download Patch File Download Diff File Expand all files Collapse all files

58
docs/architecture/README.md
View File

@@ -1,11 +1,27 @@
---
status: draft
last_updated: 2026-04-30
last_updated: 2026-05-01
---
# @alkdev/pubsub Architecture
Type-safe publish/subscribe with pluggable event target adapters. The core (`createPubSub` + `TypedEventTarget` + operators) has no transport dependency. Each adapter (Redis, WebSocket, Iroh) is an isolated module that only imports its own peer dependency.
Type-safe publish/subscribe with pluggable event target adapters. The core (`createPubSub` + `TypedEventTarget` + `EventEnvelope` + operators) has no transport dependency. Each adapter (Redis, WebSocket, Worker, Iroh) is an isolated module that only imports its own peer dependency.
This package is a **transport layer only**. It carries events between processes and does not prescribe what those events mean or how downstream systems coordinate. Higher-level protocols (call/response, operation invocation, workflow coordination) belong in downstream packages like `@alkdev/operations`.
## Core Principle
**The TypedEventTarget interface is the contract.** All transports implement the same `addEventListener` / `dispatchEvent` / `removeEventListener` surface. `createPubSub` doesn't know or care which transport is in use — it just dispatches events to whatever `TypedEventTarget` it was given.
**The EventEnvelope is the cross-platform format.** Every event dispatched through pubsub is `{ type, id, payload }`. This is a minimal, JSON-serializable envelope that any transport adapter can route and any downstream consumer can interpret. Domain-specific data lives in `payload`. Correlation lives in `id`. The event type lives in `type`. No `parent` field — causal relationships are managed by downstream coordination layers, not the transport.
Swapping transports is a one-line config change:
```ts
const pubsub = createPubSub<MyEventMap>({
eventTarget: createRedisEventTarget({ publishClient, subscribeClient }),
});
```
## Why This Exists
@@ -16,33 +32,27 @@ Extracted from `@alkdev/alkhub_ts/packages/core/pubsub/`, which itself was adapt
3. **Isolates peer deps** — Redis and Iroh are heavy native dependencies; consumers that don't need them shouldn't carry them
4. **Matches established pattern** — `@alkdev/taskgraph` and `@alkdev/typemap` already use the standalone-package pattern
## Core Principle
**The TypedEventTarget interface is the contract.** All transports implement the same `addEventListener` / `dispatchEvent` / `removeEventListener` surface. `createPubSub` doesn't know or care which transport is in use — it just dispatches events to whatever `TypedEventTarget` it was given.
This means swapping from in-process to Redis to WebSocket to Iroh is a one-line config change:
```ts
const pubsub = createPubSub<MyEventMap>({
eventTarget: createRedisEventTarget({ publishClient, subscribeClient }),
});
```
## What This Package Provides
- **Core** — `createPubSub`, `TypedEventTarget`, `TypedEvent`, topic scoping, `filter`/`map`/`pipe` operators, `Repeater` (inlined from @repeaterjs/repeater)
- **Call protocol** (`@alkdev/pubsub/call`) — `PendingRequestMap`, `CallEventSchema`, `CallError`, event types for request/response and streaming operations
- **Core** — `createPubSub`, `TypedEventTarget`, `TypedEvent`, `EventEnvelope`, stream operators (`filter`, `map`, `pipe`, `take`, `reduce`, `toArray`, `batch`, `dedupe`, `window`, `flat`, `groupBy`, `chain`, `join`), `Repeater` (inlined from @repeaterjs/repeater)
- **Adapters** (each is a peer-dep island, importable via sub-path export):
- In-process (default `EventTarget`, no adapter needed)
- Redis (`@alkdev/pubsub/event-target-redis`, peer dep: `ioredis`)
- WebSocket (future: `@alkdev/pubsub/event-target-websocket`)
- Worker (future: `@alkdev/pubsub/event-target-worker`)
- Iroh (future: `@alkdev/pubsub/event-target-iroh`, peer dep: `@rayhanadev/iroh`)
## What This Package Does NOT Provide
- **Call protocol** — request/response coordination, `PendingRequestMap`, `CallEventSchema`, and `CallError` have been moved to `@alkdev/operations`. The pubsub transport is substrate-agnostic.
- **Workflow coordination** — causal chains, parent/child relationships, and abort cascading are domain-level concerns managed by downstream packages.
- **Abort/cancellation primitives** — these belong in the coordination layer, not the transport. The `EventEnvelope` intentionally omits a `parent` field to avoid conflating transport with coordination semantics.
## Consumer Context
### alkhub (hub-spoke coordinator)
The hub uses pubsub for event routing between operations, runners, and the SSE interface. The event map is the call protocol — typed JSON events (`call.requested`, `call.responded`, `session.status`, etc.). Transport choice depends on deployment:
The hub uses pubsub for event routing between operations, runners, and the SSE interface. Transport choice depends on deployment:
| Deployment | Transport |
|------------|-----------|
@@ -50,23 +60,24 @@ The hub uses pubsub for event routing between operations, runners, and the SSE i
| Hub + worker processes | Redis |
| Hub + remote spokes | WebSocket or Iroh |
### Future: standalone spoke SDK
### Downstream packages
Spokes will import `@alkdev/pubsub` directly to create their event target (WebSocket or Iroh) and wire it into `createPubSub`. Call protocol types and `PendingRequestMap` are available from `@alkdev/pubsub/call`.
- `@alkdev/operations` uses `createPubSub` with its own event maps for call/response coordination. It defines its own event schemas and `PendingRequestMap` on top of the pubsub transport.
- `@alkdev/taskgraph` will use pubsub events for task lifecycle notifications and workflow coordination.
## Threat Model
- **Fork provenance** — core pubsub and typed event target are adapted from graphql-yoga (MIT). All original copyright notices are preserved in file headers. See [ADR-001](decisions/001-graphql-yoga-fork.md).
- **Peer dep isolation** — Redis and Iroh are optional peer dependencies. A consumer that only needs in-process transport installs zero extra packages. A consumer using Redis but not Iroh installs `ioredis` only.
- **Type-only imports** — `event-target-redis.ts` imports `ioredis` types only at compile time. At runtime, the consumer must provide the actual `Redis`/`Cluster` instances.
- **Minimal envelope** — the `EventEnvelope` format (`{ type, id, payload }`) is intentionally minimal and JSON-serializable. Any platform that supports JSON can produce or consume these events (Rust, Python, etc.).
## Architecture Documents
| Document | Content |
|----------|---------|
| [api-surface.md](api-surface.md) | createPubSub factory, PubSub types, operators, TypedEventTarget types |
| [call-protocol.md](call-protocol.md) | Call/subscribe protocol — event types, PendingRequestMap, streaming, error model, transport mapping |
| [event-targets.md](event-targets.md) | In-process, Redis, WebSocket adapters — interface, configuration, limitations |
| [api-surface.md](api-surface.md) | createPubSub factory, EventEnvelope, PubSub types, operators, TypedEventTarget types |
| [event-targets.md](event-targets.md) | In-process, Redis, WebSocket, Worker adapters — interface, configuration, limitations |
| [iroh-transport.md](iroh-transport.md) | Iroh P2P QUIC transport — protocol, framing, identity, hub/spoke sides, reconnection |
| [build-distribution.md](build-distribution.md) | Dependencies, project structure, tree-shaking, sub-path exports, targets |
@@ -93,4 +104,5 @@ last_updated: YYYY-MM-DD
- Upstream: `@graphql-yoga/subscription` and `@graphql-yoga/typed-event-target` (MIT)
- alkhub pubsub-redis doc: `@alkdev/alkhub_ts/docs/architecture/pubsub-redis.md`
- alkhub spoke-runner doc: `@alkdev/alkhub_ts/docs/architecture/spoke-runner.md`
- Migration research: `docs/research/migration.md`
- Migration research: `docs/research/migration.md`
- Research: Event sourcing types — `docs/research/event_sourcing/` (not in this repo, in global workspace)