Set up project structure, source files, and architecture docs
- Copy core source from alkhub_ts/packages/core/pubsub/ with import path fixups (typed_event_target.ts → types.ts, .ts → .js extensions) - Make PubSubPublishArgsByKey exported (was private type, needed by barrel) - Add package.json with sub-path exports and optional peer deps (ioredis) - Add tsup.config.ts with multi-entry + splitting for tree-shaking - Add tsconfig.json, vitest.config.ts, .gitignore - Add AGENTS.md with project conventions and adapter checklist - Add architecture docs following taskgraph/alkhub pattern: docs/architecture/README.md, api-surface.md, event-targets.md, iroh-transport.md, build-distribution.md - Add ADRs: 001-graphql-yoga-fork, 002-tree-shake-pattern - Copy migration research doc to docs/research/migration.md - Dual-license MIT OR Apache-2.0 (matching taskgraph)
This commit is contained in:
94
docs/architecture/README.md
Normal file
94
docs/architecture/README.md
Normal file
@@ -0,0 +1,94 @@
|
||||
---
|
||||
status: draft
|
||||
last_updated: 2026-04-30
|
||||
---
|
||||
|
||||
# @alkdev/pubsub Architecture
|
||||
|
||||
Type-safe publish/subscribe with pluggable event target adapters. The core (`createPubSub` + `TypedEventTarget` + operators) has no transport dependency. Each adapter (Redis, WebSocket, Iroh) is an isolated module that only imports its own peer dependency.
|
||||
|
||||
## Why This Exists
|
||||
|
||||
Extracted from `@alkdev/alkhub_ts/packages/core/pubsub/`, which itself was adapted from `@graphql-yoga/subscription` and `@graphql-yoga/typed-event-target`. The pubsub module was already self-contained within alkhub — zero cross-module imports from operations, config, logger, or MCP. Extracting it into a standalone package:
|
||||
|
||||
1. **Reduces coupling** — alkhub depends on pubsub, not the other way around
|
||||
2. **Enables reuse** — multiple alkhub packages can share the same pubsub instance
|
||||
3. **Isolates peer deps** — Redis and Iroh are heavy native dependencies; consumers that don't need them shouldn't carry them
|
||||
4. **Matches established pattern** — `@alkdev/taskgraph` and `@alkdev/typemap` already use the standalone-package pattern
|
||||
|
||||
## Core Principle
|
||||
|
||||
**The TypedEventTarget interface is the contract.** All transports implement the same `addEventListener` / `dispatchEvent` / `removeEventListener` surface. `createPubSub` doesn't know or care which transport is in use — it just dispatches events to whatever `TypedEventTarget` it was given.
|
||||
|
||||
This means swapping from in-process to Redis to WebSocket to Iroh is a one-line config change:
|
||||
|
||||
```ts
|
||||
const pubsub = createPubSub<MyEventMap>({
|
||||
eventTarget: createRedisEventTarget({ publishClient, subscribeClient }),
|
||||
});
|
||||
```
|
||||
|
||||
## What This Package Provides
|
||||
|
||||
- **Core** — `createPubSub`, `TypedEventTarget`, `TypedEvent`, topic scoping, `filter`/`map`/`pipe` operators
|
||||
- **Adapters** (each is a peer-dep island, importable via sub-path export):
|
||||
- In-process (default `EventTarget`, no adapter needed)
|
||||
- Redis (`@alkdev/pubsub/event-target-redis`, peer dep: `ioredis`)
|
||||
- WebSocket (future: `@alkdev/pubsub/event-target-websocket`)
|
||||
- Iroh (future: `@alkdev/pubsub/event-target-iroh`, peer dep: `@rayhanadev/iroh`)
|
||||
|
||||
## Consumer Context
|
||||
|
||||
### alkhub (hub-spoke coordinator)
|
||||
|
||||
The hub uses pubsub for event routing between operations, runners, and the SSE interface. The event map is the call protocol — typed JSON events (`call.requested`, `call.responded`, `session.status`, etc.). Transport choice depends on deployment:
|
||||
|
||||
| Deployment | Transport |
|
||||
|------------|-----------|
|
||||
| Single-process hub | In-process (default) |
|
||||
| Hub + worker processes | Redis |
|
||||
| Hub + remote spokes | WebSocket or Iroh |
|
||||
|
||||
### Future: standalone spoke SDK
|
||||
|
||||
Spokes will import `@alkdev/pubsub` directly to create their event target (WebSocket or Iroh) and wire it into `createPubSub`. The call protocol types live in a separate `@alkdev/call-protocol` package (not yet extracted).
|
||||
|
||||
## Threat Model
|
||||
|
||||
- **Fork provenance** — core pubsub and typed event target are adapted from graphql-yoga (MIT). All original copyright notices are preserved in file headers. See [ADR-001](decisions/001-graphql-yoga-fork.md).
|
||||
- **Peer dep isolation** — Redis and Iroh are optional peer dependencies. A consumer that only needs in-process transport installs zero extra packages. A consumer using Redis but not Iroh installs `ioredis` only.
|
||||
- **Type-only imports** — `event-target-redis.ts` imports `ioredis` types only at compile time. At runtime, the consumer must provide the actual `Redis`/`Cluster` instances.
|
||||
|
||||
## Architecture Documents
|
||||
|
||||
| Document | Content |
|
||||
|----------|---------|
|
||||
| [api-surface.md](api-surface.md) | createPubSub factory, PubSub types, operators, TypedEventTarget types |
|
||||
| [event-targets.md](event-targets.md) | In-process, Redis, WebSocket adapters — interface, configuration, limitations |
|
||||
| [iroh-transport.md](iroh-transport.md) | Iroh P2P QUIC transport — protocol, framing, identity, hub/spoke sides, reconnection |
|
||||
| [build-distribution.md](build-distribution.md) | Dependencies, project structure, tree-shaking, sub-path exports, targets |
|
||||
|
||||
## Document Lifecycle
|
||||
|
||||
Architecture documents use YAML frontmatter with `status` and `last_updated` fields:
|
||||
|
||||
```yaml
|
||||
---
|
||||
status: draft | stable | deprecated
|
||||
last_updated: YYYY-MM-DD
|
||||
---
|
||||
```
|
||||
|
||||
| Status | Meaning | Transitions |
|
||||
|--------|---------|-------------|
|
||||
| `draft` | Under active development. Content may change. | → `stable` when implementation is complete and tests verify API contract. |
|
||||
| `stable` | API contracts are locked. Changes require review cycle. | → `deprecated` when superseded. |
|
||||
| `deprecated` | Superseded. Kept for reference. | Removed when no longer referenced. |
|
||||
|
||||
## References
|
||||
|
||||
- Source: `@alkdev/alkhub_ts/packages/core/pubsub/`
|
||||
- Upstream: `@graphql-yoga/subscription` and `@graphql-yoga/typed-event-target` (MIT)
|
||||
- alkhub pubsub-redis doc: `@alkdev/alkhub_ts/docs/architecture/pubsub-redis.md`
|
||||
- alkhub spoke-runner doc: `@alkdev/alkhub_ts/docs/architecture/spoke-runner.md`
|
||||
- Migration research: `docs/research/migration.md`
|
||||
101
docs/architecture/api-surface.md
Normal file
101
docs/architecture/api-surface.md
Normal file
@@ -0,0 +1,101 @@
|
||||
---
|
||||
status: draft
|
||||
last_updated: 2026-04-30
|
||||
---
|
||||
|
||||
# API Surface
|
||||
|
||||
Core pubsub creation, types, and operators. No transport dependencies.
|
||||
|
||||
## `createPubSub`
|
||||
|
||||
```ts
|
||||
function createPubSub<TPubSubPublishArgsByKey extends PubSubPublishArgsByKey>(
|
||||
config?: PubSubConfig<TPubSubPublishArgsByKey>,
|
||||
): PubSub<TPubSubPublishArgsByKey>;
|
||||
```
|
||||
|
||||
Factory function. Accepts an optional `eventTarget` config. If none is provided, uses `new EventTarget()` (in-process).
|
||||
|
||||
### Topic Scoping
|
||||
|
||||
Topics can be scoped with an id:
|
||||
|
||||
- `pubsub.publish("session.status", projectId, payload)` → dispatches to topic `session.status:{projectId}`
|
||||
- `pubsub.subscribe("session.status", projectId)` → subscribes to topic `session.status:{projectId}` only
|
||||
- `pubsub.publish("session.status", payload)` → dispatches to topic `session.status` (unscoped)
|
||||
- `pubsub.subscribe("session.status")` → subscribes to topic `session.status` (unscoped)
|
||||
|
||||
The topic string is either the routing key directly (unscoped) or `{routingKey}:{id}` (scoped). This maps naturally to Redis channel naming and WebSocket message routing.
|
||||
|
||||
### `PubSubPublishArgsByKey`
|
||||
|
||||
The type parameter that defines the event map:
|
||||
|
||||
```ts
|
||||
type PubSubPublishArgsByKey = {
|
||||
[key: string]: [] | [unknown] | [number | string, unknown];
|
||||
};
|
||||
```
|
||||
|
||||
- `[]` — event with no payload (trigger only)
|
||||
- `[payload]` — unscoped event with payload
|
||||
- `[id, payload]` — scoped event with id and payload
|
||||
|
||||
### `PubSub.subscribe()`
|
||||
|
||||
Returns a `Repeater<unknown>` (async iterable). Consumers iterate with `for await`:
|
||||
|
||||
```ts
|
||||
for await (const payload of pubsub.subscribe("session.status")) {
|
||||
// handle payload
|
||||
}
|
||||
```
|
||||
|
||||
The `Repeater` automatically cleans up its `addEventListener` when the consumer breaks out of the loop (the `stop` promise resolves).
|
||||
|
||||
## Types
|
||||
|
||||
| Export | Source | Description |
|
||||
|--------|--------|-------------|
|
||||
| `TypedEvent<TType, TDetail>` | `types.ts` | Event with typed `type` and `detail`. Omits `CustomEvent`'s untyped fields. |
|
||||
| `TypedEventTarget<TEvent>` | `types.ts` | Extends `EventTarget` with typed `addEventListener`, `dispatchEvent`, `removeEventListener`. |
|
||||
| `TypedEventListener<TEvent>` | `types.ts` | `(evt: TEvent) => void` |
|
||||
| `TypedEventListenerObject<TEvent>` | `types.ts` | `{ handleEvent(object: TEvent): void }` |
|
||||
| `TypedEventListenerOrEventListenerObject<TEvent>` | `types.ts` | Union of the above |
|
||||
| `PubSub<TPubSubPublishArgsByKey>` | `create_pubsub.ts` | `{ publish, subscribe }` |
|
||||
| `PubSubConfig<TPubSubPublishArgsByKey>` | `create_pubsub.ts` | `{ eventTarget?: PubSubEventTarget }` |
|
||||
| `PubSubEvent<TPubSubPublishArgsByKey, TKey>` | `create_pubsub.ts` | Derived `TypedEvent` for a specific event key |
|
||||
| `PubSubEventTarget<TPubSubPublishArgsByKey>` | `create_pubsub.ts` | `TypedEventTarget<PubSubEvent<...>>` |
|
||||
|
||||
## Operators
|
||||
|
||||
All operators return `Repeater` instances and work with any async iterable.
|
||||
|
||||
### `filter`
|
||||
|
||||
```ts
|
||||
function filter<T>(filterFn: (value: T) => Promise<boolean> | boolean): (source: AsyncIterable<T>) => Repeater<T>;
|
||||
```
|
||||
|
||||
Type-narrowing overload available: `filter<T, U extends T>(fn: (input: T) => input is U)`.
|
||||
|
||||
### `map`
|
||||
|
||||
```ts
|
||||
function map<T, O>(mapper: (input: T) => Promise<O> | O): (source: AsyncIterable<T>) => Repeater<O>;
|
||||
```
|
||||
|
||||
### `pipe`
|
||||
|
||||
```ts
|
||||
function pipe<A, B>(a: A, ab: (a: A) => B): B;
|
||||
function pipe<A, B, C>(a: A, ab: (a: A) => B, bc: (b: B) => C): C;
|
||||
// up to 5 arguments
|
||||
```
|
||||
|
||||
Compose operators: `pipe(pubsub.subscribe("myEvent"), filter(isRelevant), map(transform))`
|
||||
|
||||
## Attribution
|
||||
|
||||
`createPubSub` and operators are adapted from `@graphql-yoga/subscription` (MIT). `TypedEventTarget` types are adapted from `@graphql-yoga/typed-event-target` (MIT). See file headers for full license text.
|
||||
113
docs/architecture/build-distribution.md
Normal file
113
docs/architecture/build-distribution.md
Normal file
@@ -0,0 +1,113 @@
|
||||
---
|
||||
status: draft
|
||||
last_updated: 2026-04-30
|
||||
---
|
||||
|
||||
# Build & Distribution
|
||||
|
||||
Dependencies, project structure, tree-shaking, sub-path exports, and build targets.
|
||||
|
||||
## Dependencies
|
||||
|
||||
| Package | Type | Purpose |
|
||||
|---------|------|---------|
|
||||
| `@repeaterjs/repeater` | direct | Small (~3KB). Core async iterable primitive for `subscribe()`. |
|
||||
| `ioredis` | peer (optional) | Redis client. Only imported by `event-target-redis.ts`. Type-only import at compile time. |
|
||||
| `@rayhanadev/iroh` | peer (optional, future) | Iroh NAPI-RS binding. Only imported by `event-target-iroh.ts`. |
|
||||
|
||||
No other external dependencies. No logger dependency.
|
||||
|
||||
## Project Structure
|
||||
|
||||
```
|
||||
@alkdev/pubsub/
|
||||
src/
|
||||
index.ts # Barrel: re-exports core API
|
||||
types.ts # TypedEvent, TypedEventTarget, etc.
|
||||
create_pubsub.ts # createPubSub factory
|
||||
operators.ts # filter, map, pipe
|
||||
event-target-redis.ts # createRedisEventTarget (peer dep: ioredis)
|
||||
# Future adapters (each is its own entry point + peer dep island):
|
||||
# event-target-websocket.ts # peer dep: none (web standard)
|
||||
# event-target-iroh.ts # peer dep: @rayhanadev/iroh
|
||||
test/
|
||||
create_pubsub.test.ts
|
||||
operators.test.ts
|
||||
event-target-redis.test.ts
|
||||
# event-target-websocket.test.ts
|
||||
# event-target-iroh.test.ts
|
||||
docs/
|
||||
architecture.md
|
||||
architecture/
|
||||
research/
|
||||
package.json
|
||||
tsconfig.json
|
||||
tsup.config.ts
|
||||
vitest.config.ts
|
||||
```
|
||||
|
||||
## Sub-Path Exports
|
||||
|
||||
We use explicit sub-path exports rather than barrel-only + tree-shaking. Each adapter is importable by its own path:
|
||||
|
||||
```json
|
||||
{
|
||||
"exports": {
|
||||
".": { ... },
|
||||
"./event-target-redis": { ... },
|
||||
"./event-target-websocket": { ... },
|
||||
"./event-target-iroh": { ... }
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Why Sub-Path Exports
|
||||
|
||||
- **Explicit** — doesn't rely on bundler tree-shaking behavior
|
||||
- **Peer dep isolation** — `import from '@alkdev/pubsub/event-target-redis'` makes the dependency on ioredis explicit at the import site
|
||||
- **Consistent with typemap pattern** — typemap's peer deps (zod, valibot, typebox) are each their own module; sub-path exports make this explicit at the package boundary
|
||||
|
||||
Sub-path entries are added as adapters are implemented. The barrel `index.ts` also re-exports everything for convenience — consumers who want tree-shaking can import from the barrel and rely on their bundler.
|
||||
|
||||
## Peer Dependencies
|
||||
|
||||
| Peer Dep | Required By | Optional |
|
||||
|----------|-------------|----------|
|
||||
| `ioredis@^5.0.0` | `event-target-redis` | Yes |
|
||||
| `@rayhanadev/iroh` | `event-target-iroh` (future) | Yes |
|
||||
|
||||
Optional peer deps means `npm install @alkdev/pubsub` does NOT install ioredis or iroh. Consumers opt in by installing the peer dep and importing from the sub-path.
|
||||
|
||||
## Build
|
||||
|
||||
- **Tool**: `tsup` — produces dual ESM + CJS with declarations automatically
|
||||
- **Entry points**: `src/index.ts`, `src/event-target-redis.ts`, plus future adapters
|
||||
- **Format**: ESM + CJS
|
||||
- **Target**: `es2022`
|
||||
- **Splitting**: enabled (tsup code splitting for shared chunks)
|
||||
|
||||
```ts
|
||||
// tsup.config.ts
|
||||
import { defineConfig } from 'tsup';
|
||||
|
||||
export default defineConfig({
|
||||
entry: ['src/index.ts', 'src/event-target-redis.ts'],
|
||||
format: ['esm', 'cjs'],
|
||||
dts: true,
|
||||
sourcemap: true,
|
||||
clean: true,
|
||||
splitting: true,
|
||||
target: 'es2022',
|
||||
});
|
||||
```
|
||||
|
||||
## Testing
|
||||
|
||||
- **Runner**: `vitest` — matches taskgraph, natural fit with tsup/Node build pipeline
|
||||
- **Config**: `vitest.config.ts` with `globals: true`
|
||||
|
||||
## Targets
|
||||
|
||||
- **Publish**: npm (`@alkdev/pubsub`)
|
||||
- **Runtime**: Node 18+, Deno, Bun — pure JS (except iroh adapter which requires NAPI-RS)
|
||||
- **Deno compatibility**: Source is standard TypeScript with no Deno-specific APIs. Deno can import from npm or JSR.
|
||||
28
docs/architecture/decisions/001-graphql-yoga-fork.md
Normal file
28
docs/architecture/decisions/001-graphql-yoga-fork.md
Normal file
@@ -0,0 +1,28 @@
|
||||
# ADR-001: Fork graphql-yoga pubsub
|
||||
|
||||
**Status**: Accepted
|
||||
**Date**: 2026-04-30
|
||||
|
||||
## Context
|
||||
|
||||
`createPubSub`, `TypedEventTarget`, and operators are adapted from `@graphql-yoga/subscription` and `@graphql-yoga/typed-event-target` (MIT). We carried these into alkhub with modifications (native CustomEvent, our TypedEventTarget types, removed tslib). Now we're extracting to a standalone package.
|
||||
|
||||
## Decision
|
||||
|
||||
Fork (continue carrying adapted code) rather than depend on graphql-yoga packages directly.
|
||||
|
||||
## Rationale
|
||||
|
||||
1. **Different evolution path** — graphql-yoga's pubsub is tailored for GraphQL subscriptions. Our use case is general-purpose event routing with multiple transports. The APIs will diverge further as we add WebSocket and Iroh adapters.
|
||||
|
||||
2. **Dependency reduction** — graphql-yoga's subscription package pulls in `@whatwg-node/events` and `tslib`. We don't need either — we use native `CustomEvent` and no tslib runtime.
|
||||
|
||||
3. **Control over types** — graphql-yoga's `TypedEventTarget` uses their own event type hierarchy. We use a simpler one that maps directly to `CustomEvent`. Maintaining our own types avoids version coupling.
|
||||
|
||||
4. **Already forked** — the code in alkhub already diverged from the original. The license headers are in place. A clean extraction doesn't change the provenance story.
|
||||
|
||||
## Consequences
|
||||
|
||||
- Must preserve MIT license headers in all forked files
|
||||
- Must update attribution if we make significant changes beyond the original fork scope
|
||||
- No automatic updates from graphql-yoga — we carry our own maintenance burden
|
||||
36
docs/architecture/decisions/002-tree-shake-pattern.md
Normal file
36
docs/architecture/decisions/002-tree-shake-pattern.md
Normal file
@@ -0,0 +1,36 @@
|
||||
# ADR-002: Sub-Path Exports + Peer Deps for Adapter Isolation
|
||||
|
||||
**Status**: Accepted
|
||||
**Date**: 2026-04-30
|
||||
|
||||
## Context
|
||||
|
||||
Each event target adapter has different peer dependencies (ioredis for Redis, @rayhanadev/iroh for Iroh). Consumers that only use in-process or one adapter should not be forced to install peer deps for adapters they don't use. Two approaches:
|
||||
|
||||
1. **Barrel + tree-shaking** — single entry point, rely on bundler to drop unused adapters
|
||||
2. **Sub-path exports** — explicit per-adapter entry points in `package.json` exports map
|
||||
|
||||
Typemap uses barrel-only. Taskgraph uses plain barrel with no sub-path exports.
|
||||
|
||||
## Decision
|
||||
|
||||
Use sub-path exports with optional peer dependencies.
|
||||
|
||||
## Rationale
|
||||
|
||||
1. **Explicit dependency declaration** — `import { createRedisEventTarget } from '@alkdev/pubsub/event-target-redis'` makes it clear at the import site that this module needs ioredis. A barrel import doesn't.
|
||||
|
||||
2. **No bundler reliance** — sub-path exports don't depend on the consumer's bundler correctly tree-shaking. Not all consumers use bundlers (Deno, Node with `--experimental-strip-types`).
|
||||
|
||||
3. **peerDependenciesMeta** — npm treats optional peer deps as install warnings, not errors. `npm install @alkdev/pubsub` installs only the core. `npm install @alkdev/pubsub ioredis` gets Redis support.
|
||||
|
||||
4. **Consistent with typemap's philosophy** — typemap's peer deps (zod, valibot, typebox) are each their own module island. Sub-path exports make this explicit at the package boundary. We're adding the package.json entry points that typemap doesn't have.
|
||||
|
||||
5. **Incremental** — adapters can be added one at a time. Each new adapter adds one entry to the exports map and one entry point to tsup.
|
||||
|
||||
## Consequences
|
||||
|
||||
- More entries in `package.json` exports — maintenance burden scales with adapter count
|
||||
- Both barrel and sub-path work — barrel re-exports everything for convenience, sub-path for explicitness
|
||||
- tsup must list each adapter as a separate entry point
|
||||
- Consumer docs should recommend sub-path imports for adapter-specific code
|
||||
111
docs/architecture/event-targets.md
Normal file
111
docs/architecture/event-targets.md
Normal file
@@ -0,0 +1,111 @@
|
||||
---
|
||||
status: draft
|
||||
last_updated: 2026-04-30
|
||||
---
|
||||
|
||||
# Event Target Adapters
|
||||
|
||||
In-process, Redis, and WebSocket event targets. All implement `TypedEventTarget<TEvent>`.
|
||||
|
||||
## Interface Contract
|
||||
|
||||
Every adapter must implement:
|
||||
|
||||
| Method | Behavior |
|
||||
|--------|----------|
|
||||
| `addEventListener(type, callback)` | Register listener for event type. Callback receives `CustomEvent` with typed `detail`. |
|
||||
| `dispatchEvent(event)` | Send/dispatch event. Returns `boolean` (always `true` for non-cancelable events). |
|
||||
| `removeEventListener(type, callback)` | Unregister listener. Clean up underlying subscription when no listeners remain for a topic. |
|
||||
|
||||
## In-Process (Default)
|
||||
|
||||
No adapter needed. `createPubSub` uses `new EventTarget()` by default. This works for single-process deployments where all pubsub participants share the same memory.
|
||||
|
||||
No explicit `InProcessEventTarget` class — the web standard `EventTarget` already implements the interface. Could be formalized later if a name makes the API clearer, but `new EventTarget()` is already the standard.
|
||||
|
||||
## Redis
|
||||
|
||||
**Import**: `@alkdev/pubsub/event-target-redis`
|
||||
**Peer dep**: `ioredis@^5.0.0` (optional)
|
||||
|
||||
### `createRedisEventTarget`
|
||||
|
||||
```ts
|
||||
function createRedisEventTarget<TEvent extends TypedEvent>(
|
||||
args: CreateRedisEventTargetArgs,
|
||||
): TypedEventTarget<TEvent>;
|
||||
```
|
||||
|
||||
### `CreateRedisEventTargetArgs`
|
||||
|
||||
| Field | Type | Required | Description |
|
||||
|-------|------|----------|-------------|
|
||||
| `publishClient` | `Redis \| Cluster` | Yes | ioredis client for publishing. Can share a connection. |
|
||||
| `subscribeClient` | `Redis \| Cluster` | Yes | ioredis client for subscribing. Must be dedicated — Redis requires subscriber connections to only receive messages. |
|
||||
| `serializer` | `{ stringify, parse }` | No | Custom serializer. Defaults to `JSON`. |
|
||||
|
||||
### How It Works
|
||||
|
||||
- `dispatchEvent` → `publishClient.publish(event.type, serializer.stringify(event.detail))`
|
||||
- `addEventListener` → `subscribeClient.subscribe(topic)`, track callbacks per topic
|
||||
- `removeEventListener` → remove callback; if no callbacks remain for topic, `subscribeClient.unsubscribe(topic)`
|
||||
|
||||
### Channel Naming
|
||||
|
||||
Currently uses raw event type as Redis channel name (e.g., `session.status:proj_123`). Architecture recommends `alk:events:{eventType}` prefix but this is not yet implemented. Should be configurable: `createRedisEventTarget({ ..., prefix: "alk:events:" })`.
|
||||
|
||||
### Limitations (Current)
|
||||
|
||||
- **No error handling** — connection failures, reconnection, and message parse errors are not handled
|
||||
- **No channel prefix** — raw event types as channel names risk collision in shared Redis instances
|
||||
- **No unsubscribe cleanup on client disconnect** — if the subscribe client disconnects, registered callbacks remain in the map but will never fire
|
||||
|
||||
### Test Coverage
|
||||
|
||||
5 tests in alkhub (publish path only, mocked ioredis). No tests for subscription-receive path, unsubscribe cleanup, or error handling.
|
||||
|
||||
## WebSocket
|
||||
|
||||
**Import**: `@alkdev/pubsub/event-target-websocket` (not yet implemented)
|
||||
**Peer dep**: none (WebSocket is a web standard)
|
||||
|
||||
### Design (Spec from `spoke-runner.md`)
|
||||
|
||||
```ts
|
||||
class WebSocketEventTarget implements TypedEventTarget<any> {
|
||||
private listeners = new Map<string, Set<(event: CustomEvent) => void>>()
|
||||
|
||||
constructor(private ws: WebSocket) {
|
||||
ws.onmessage = (msg) => {
|
||||
const { type, payload } = JSON.parse(msg.data as string)
|
||||
const event = new CustomEvent(type, { detail: payload })
|
||||
for (const listener of this.listeners.get(type) ?? []) {
|
||||
listener(event)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
dispatchEvent(event: CustomEvent): boolean {
|
||||
this.ws.send(JSON.stringify({ type: event.type, payload: event.detail }))
|
||||
return true
|
||||
}
|
||||
|
||||
addEventListener(type: string, listener: (event: CustomEvent) => void): void { ... }
|
||||
removeEventListener(type: string, listener: (event: CustomEvent) => void): void { ... }
|
||||
}
|
||||
```
|
||||
|
||||
### Key Properties
|
||||
|
||||
- **Bidirectional** — `dispatchEvent` sends over WS, `addEventListener` receives from WS
|
||||
- **Per-connection** — hub creates one per spoke connection
|
||||
- **JSON framing** — WebSocket provides native message boundaries (no length-prefix needed)
|
||||
- **No native deps** — works in browsers and Node
|
||||
|
||||
### Gap: Reconnection
|
||||
|
||||
WebSocket connections drop. On reconnect, the spoke must re-register with the hub (same `hub.register` flow). The `WebSocketEventTarget` itself is per-connection — a new connection means a new event target instance. Reconnection logic belongs to the spoke lifecycle, not the event target.
|
||||
|
||||
### Gap: Hub-Side Architecture
|
||||
|
||||
The hub needs per-connection event target + `PendingRequestMap` creation on accept, cleanup on disconnect. This is a hub architectural concern, not a pubsub concern. See `@alkdev/alkhub_ts/docs/architecture/spoke-runner.md`.
|
||||
140
docs/architecture/iroh-transport.md
Normal file
140
docs/architecture/iroh-transport.md
Normal file
@@ -0,0 +1,140 @@
|
||||
---
|
||||
status: draft
|
||||
last_updated: 2026-04-30
|
||||
---
|
||||
|
||||
# Iroh Transport
|
||||
|
||||
P2P QUIC event target using iroh. More complex than the other transports due to NAT traversal, crypto identity, and byte-stream framing.
|
||||
|
||||
**Import**: `@alkdev/pubsub/event-target-iroh` (not yet implemented)
|
||||
**Peer dep**: `@rayhanadev/iroh` (optional, NAPI-RS native addon)
|
||||
|
||||
## Why Iroh
|
||||
|
||||
WebSocket requires the hub to have a publicly reachable address. Iroh solves:
|
||||
|
||||
1. **Hub behind NAT** — spokes dial by `NodeId` through relay servers, no public IP needed
|
||||
2. **Spoke push** — hub can initiate connections to spokes by `NodeId` (impossible with WS without polling)
|
||||
3. **P2P spoke-to-spoke** — direct spoke-to-spoke communication without routing through hub
|
||||
4. **Cryptographic identity** — Ed25519 `NodeId` doubles as spoke authentication
|
||||
|
||||
## Iroh Binding
|
||||
|
||||
Using `@rayhanadev/iroh` (v0.1.1) as the NAPI-RS binding. Community binding, one author, no tests. It has everything needed for hub-spoke 1:1 bidirectional streams:
|
||||
|
||||
| Method | Purpose |
|
||||
|--------|---------|
|
||||
| `Endpoint.create()` / `createWithOptions({ alpns })` | Create QUIC endpoint |
|
||||
| `Endpoint.connect(nodeId, alpn)` | Connect to a peer by public key |
|
||||
| `Endpoint.accept()` | Accept incoming connection |
|
||||
| `Endpoint.nodeId()` | Get our public key identity |
|
||||
| `Connection.openBi()` | Open bidirectional stream (spoke side) |
|
||||
| `Connection.acceptBi()` | Accept bidirectional stream (hub side) |
|
||||
| `SendStream.writeAll(data)` | Send data on stream |
|
||||
| `RecvStream.readExact(len)` | Read exact bytes from stream |
|
||||
| `Connection.remoteNodeId()` | Get peer's public key |
|
||||
| `Connection.sendDatagram()` / `readDatagram()` | Unreliable datagrams |
|
||||
|
||||
Not exposed (not critical): `Endpoint.watch_addr()`, `Connection.close_reason()`, `Connection.stats()`.
|
||||
|
||||
## Protocol
|
||||
|
||||
Single bidirectional QUIC stream per connection. Length-prefixed JSON messages.
|
||||
|
||||
### Framing
|
||||
|
||||
QUIC streams are byte streams (no message boundaries). We use 4-byte big-endian length prefix:
|
||||
|
||||
```
|
||||
[4 bytes: length N][N bytes: JSON payload]
|
||||
```
|
||||
|
||||
`RecvStream.readExact(4)` reads the length, then `readExact(N)` reads the payload. This is trivial with iroh's `readExact()` API.
|
||||
|
||||
### Message Format
|
||||
|
||||
Same `type` + `detail` shape as all other transports:
|
||||
|
||||
```json
|
||||
{ "type": "call.requested", "detail": { ... } }
|
||||
```
|
||||
|
||||
Maps directly to `new CustomEvent(type, { detail })`.
|
||||
|
||||
## Two-Sided Design
|
||||
|
||||
Unlike Redis and WebSocket, Iroh has distinct hub and spoke connection patterns:
|
||||
|
||||
### Spoke Side
|
||||
|
||||
```ts
|
||||
const conn = await endpoint.connect(hubNodeId, "alkhub/1");
|
||||
const eventTarget = await createSpokeIrohEventTarget(conn);
|
||||
```
|
||||
|
||||
Spoke opens the bidirectional stream with `openBi()`. The event target wraps the `SendStream` and `RecvStream`.
|
||||
|
||||
### Hub Side
|
||||
|
||||
```ts
|
||||
const conn = await endpoint.accept();
|
||||
const eventTarget = await createHubIrohEventTarget(conn);
|
||||
```
|
||||
|
||||
Hub accepts the connection, then accepts the stream with `acceptBi()`. Same `TypedEventTarget` interface on both sides.
|
||||
|
||||
### Why Two Factories?
|
||||
|
||||
The connection initiator (spoke) calls `openBi()`. The listener (hub) calls `acceptBi()`. Both get `SendStream` + `RecvStream` — the framing and event handling are identical. The split is about connection establishment, not event handling. Could be unified as `createIrohEventTarget(sendStream, recvStream)` with separate helpers for connection, but the two-factory pattern makes the hub/spoke asymmetry explicit.
|
||||
|
||||
## Identity
|
||||
|
||||
`Connection.remoteNodeId()` returns the peer's Ed25519 public key. This is cryptographic identity — no separate API key exchange needed for authentication. The hub can verify that a connection comes from an expected spoke by checking its `NodeId`.
|
||||
|
||||
This is strictly better than WebSocket's token-in-URL or first-message approach. It's also harder to revoke — disabling a spoke requires a denylist of `NodeId`s rather than rotating a token.
|
||||
|
||||
## Connection Startup
|
||||
|
||||
On connection, both sides exchange the operations they expose (same `hub.register` pattern as WebSocket). The `NodeId` serves as identity — no separate API key exchange.
|
||||
|
||||
## Reconnection
|
||||
|
||||
Same pattern as WebSocket — detect connection failure, reconnect, re-register. QUIC handles multipath better than TCP but the application still needs reconnection logic.
|
||||
|
||||
Detection: `RecvStream.readExact()` throws on connection close. The event target should propagate this as an error event or let the caller handle it.
|
||||
|
||||
## Browser Limitations
|
||||
|
||||
Iroh in browsers is relay-only (no UDP hole punching from browser sandbox). This means:
|
||||
- Browser spokes always route through relay servers
|
||||
- WebSocketEventTarget is the right browser transport today (native, no extra deps)
|
||||
- IrohEventTarget for browsers would use the WASM build over relay — future option
|
||||
|
||||
## Multi-Node (Future)
|
||||
|
||||
For 1:N fan-out, `iroh-gossip` is the right tool. No TS binding exists yet. Options:
|
||||
1. Write a minimal Rust NAPI crate wrapping `iroh-gossip::Gossip.subscribe() + broadcast()`
|
||||
2. Contribute gossip to `@rayhanadev/iroh`
|
||||
3. Use hub as a relay point (hub receives once, fans out to each spoke's `IrohEventTarget` individually)
|
||||
|
||||
For now, 1:1 connections are sufficient. The hub fans out to multiple spokes by dispatching to each spoke's `IrohEventTarget` individually — same pattern as WebSocketEventTarget on the hub side.
|
||||
|
||||
## Comparison with WebSocketEventTarget
|
||||
|
||||
| Aspect | WebSocket | Iroh |
|
||||
|--------|-----------|------|
|
||||
| Connection | `new WebSocket(url)` | `endpoint.connect(nodeId, alpn)` |
|
||||
| Accept | Hono WS upgrade | `endpoint.accept()` |
|
||||
| Identity | API key/token | Ed25519 NodeId (cryptographic, mutual) |
|
||||
| NAT traversal | Requires reverse proxy / tunnel | Built-in (relay + hole punching) |
|
||||
| Framing | WS frames (built-in) | QUIC stream (length-prefix needed) |
|
||||
| Hub behind NAT | Not possible without tunneling | Yes |
|
||||
| Browser | Yes (native) | Limited (WASM build, relay-only) |
|
||||
| Native addon | No | Yes (NAPI-RS) |
|
||||
|
||||
## Open Questions
|
||||
|
||||
1. **Binding stability** — `@rayhanadev/iroh` has one author and no tests. If it breaks, we may need to fork or write our own NAPI wrapper. Mitigation: the API surface we use is small (10 methods) and the binding is thin.
|
||||
2. **NAPI under Deno** — NAPI-RS `.node` binaries need testing under Deno 2.x. Since we're building with tsup for npm, the runtime is Node.js.
|
||||
3. **Datagram support** — `sendDatagram`/`readDatagram` could be used for fire-and-forget events (no response expected). Not needed for hub-spoke but could be useful for broadcast. Deferred.
|
||||
Reference in New Issue
Block a user