pubsub/docs/research/migration.md

Research: @alkdev/pubsub Package Extraction

Note (2026-05-01): This document reflects the original migration from alkhub_ts. Since extraction, the architecture has been simplified — call.ts (PendingRequestMap, CallEventSchema, CallError) has been removed from this package and moved to @alkdev/operations. The EventEnvelope type ({ type, id, payload }) is now the cross-platform serialization contract. The PubSubPublishArgsByKey tuple-based model has been replaced with a simpler PubSubEventMap. Stream operators have been expanded beyond filter/map/pipe. See docs/architecture/ for current spec.

Goal

Extract packages/core/pubsub/ into a standalone @alkdev/pubsub package, following the same peer-dependency tree-shaking pattern as @alkdev/typemap. Each event target adapter (Redis, WebSocket, Iroh) is an isolated module that only imports its own peer dependency. The core createPubSub + TypedEventTarget + operators has no peer deps beyond @repeaterjs/repeater.

Current State

Source: packages/core/pubsub/

File	Lines	Key Exports	Dependencies
typed_event_target.ts	59	TypedEvent, TypedEventTarget, TypedEventListener etc.	None (pure types)
create_pubsub.ts	108	createPubSub, PubSub, PubSubConfig, PubSubPublishArgsByKey	@repeaterjs/repeater
redis_event_target.ts	117	createRedisEventTarget, CreateRedisEventTargetArgs	ioredis (types only), typed_event_target.ts
operators.ts	67	filter, map, pipe	@repeaterjs/repeater
mod.ts	5	Re-exports all + Repeater	All above

Zero cross-module dependencies. The pubsub module imports nothing from operations/, mcp/, config/, or logger/. It is already self-contained.

Test Coverage

Test File	Tests	Coverage
tests/pubsub/redis_event_target.test.ts	5 tests	Redis publish path only (mocked ioredis). No subscription-receive path, no real Redis.
create_pubsub.ts	0 tests	No tests. Core pubsub creation, topic scoping, event delivery, Repeater iteration all untested.
operators.ts	0 tests	No tests. filter, map, pipe all untested.
typed_event_target.ts	N/A	Pure type definitions — no runtime to test.

What's Missing (Not Yet Implemented)

1. WebSocketEventTarget — Spec in spoke-runner.md (lines 158-204). Implements TypedEventTarget over a WebSocket connection. Bidirectional: dispatchEvent sends over WS, addEventListener receives from WS. Per-connection instance on hub side.
2. IrohEventTarget — P2P QUIC transport using iroh. Same role as WebSocketEventTarget but with crypto identity (Ed25519 NodeId) and automatic NAT traversal. The @rayhanadev/iroh NAPI-RS binding has everything needed — Endpoint.connect()/accept(), Connection.openBi()/acceptBi(), SendStream/RecvStream. No gossip required for hub↔spoke (1:1 bidirectional). See "Iroh Research" below.
3. In-process EventTarget — Currently createPubSub defaults to new EventTarget(), which works single-process. No explicit adapter class for this (it's just the default). Could be formalized as InProcessEventTarget for clarity, or left as-is since EventTarget is a web standard.
4. Redis channel prefixing — Architecture doc recommends alk:events:{eventType} namespacing. Not implemented.
5. Redis reconnection/error handling — No error handling for connection failures, reconnection, or message parse errors.

Proposed Package Structure

@alkdev/pubsub/
  src/
    index.ts                    # Barrel: re-exports all public API
    types.ts                    # TypedEvent, TypedEventTarget, etc. (from typed_event_target.ts)
    create_pubsub.ts            # createPubSub factory (no changes)
    operators.ts                # filter, map, pipe (no changes)
    
    # Adapter modules (tree-shakeable, each is its own peer dep island)
    event-target-in-process.ts  # Explicit InProcessEventTarget (or just re-export web EventTarget)
    event-target-redis.ts       # createRedisEventTarget (peer dep: ioredis)
    event-target-websocket.ts   # createWebSocketEventTarget (peer dep: none — WS is a web standard)
    event-target-iroh.ts        # createIrohEventTarget (peer dep: @rayhanadev/iroh)
  tests/
    create_pubsub.test.ts       # Core pubsub: publish, subscribe, topic scoping, Repeater
    operators.test.ts           # filter, map, pipe
    event-target-in-process.test.ts
    event-target-redis.test.ts  # Mocked + integration
    event-target-websocket.test.ts
    event-target-iroh.test.ts    # Mocked or integration
  package.json
  tsconfig.json

The barrel index.ts re-exports everything (like typemap). Tree-shaking works because ESM re-exports are statically analyzable. Users who want minimal bundles import specific adapter files directly (e.g., import { createRedisEventTarget } from '@alkdev/pubsub/event-target-redis').

Alternatively, if we want sub-path exports (which typemap doesn't use but many packages do), we could add them to package.json exports:

{
  "exports": {
    ".": { "import": "./dist/index.mjs", "types": "./dist/index.d.mts" },
    "./event-target-redis": { "import": "./dist/event-target-redis.mjs", "types": "./dist/event-target-redis.d.mts" },
    "./event-target-websocket": { "import": "./dist/event-target-websocket.mjs", "types": "./dist/event-target-websocket.d.mts" },
    "./event-target-iroh": { "import": "./dist/event-target-iroh.mjs", "types": "./dist/event-target-iroh.d.mts" }
  }
}

Sub-path exports are more explicit and don't rely on bundler tree-shaking, but add maintenance burden. We should pick one approach and use it consistently across @alkdev packages.

Dependencies

Dependency	Type	Notes
@repeaterjs/repeater	direct	Small (~3KB), stable. Core async iterable primitive for subscribe().
ioredis	peer	Only imported by event-target-redis.ts. Type-only import at compile time. Consumers who don't need Redis skip it.
@rayhanadev/iroh	peer	Only imported by event-target-iroh.ts. NAPI-RS native addon (~15-20MB). Consumers who don't need P2P QUIC skip it.

No other external dependencies. No logger dependency.

Build & Publish

Following @alkdev/taskgraph precedent:

• Build tool: tsup — produces dual ESM + CJS with types automatically
• Target: es2022
• Publish target: npm (@alkdev/pubsub)
• Deno compatibility: Source is standard TypeScript with no Deno-specific APIs (all web standard). Deno can import from npm or JSR.
• Testing: vitest (matching taskgraph) or deno test (matching current alkhub_ts). Decision needed.

Build Config Sketch

// tsup.config.ts
import { defineConfig } from 'tsup';

export default defineConfig({
  entry: [
    'src/index.ts',
    'src/event-target-redis.ts',
    'src/event-target-websocket.ts',
    'src/event-target-iroh.ts',
  ],
  format: ['esm', 'cjs'],
  dts: true,
  splitting: true,
  clean: true,
  target: 'es2022',
});

Iroh Research Summary

What Is Iroh?

Iroh is a Rust P2P QUIC library by n0.computer. Peers connect by public key (Ed25519), not IP address. Key capabilities:

• NAT traversal: Automatic UDP hole punching (~90% success rate), QUIC Address Discovery
• Relay fallback: If direct connection fails, routes through stateless relay servers (end-to-end encrypted)
• Public key addressing: Peers identified by NodeId, no DNS or IP needed
• QUIC transport: Multiplexed streams, built-in encryption, 0-RTT
• Gossip protocol (iroh-gossip): Epidemic broadcast trees for topic-based pub/sub (not needed for hub↔spoke — that's 1:1, not 1:N)

Why It Matters for alkhub

WebSocket transport requires the hub to have a publicly reachable address. Spokes behind NAT can't be reached by the hub for push operations. Iroh solves:

1. Hub behind NAT — No public IP needed. Spokes dial the hub by its NodeId through relay servers.
2. Spoke push — Hub can initiate connections to spokes by NodeId (impossible with WS without polling).
3. P2P spoke↔spoke — Direct spoke-to-spoke communication without routing through hub.
4. Cryptographic identity — Ed25519 NodeId doubles as spoke authentication — strictly better than API keys for identification.

Current TS Binding — @rayhanadev/iroh

NAPI-RS binding (v0.1.1) at /workspace/iroh-ts. The binding has everything needed to build IrohEventTarget. No gossip required — hub↔spoke is 1:1 bidirectional JSON event channels over QUIC streams.

Core API that we use:

Method	Purpose
Endpoint.create() / createWithOptions({ alpns })	Create QUIC endpoint
Endpoint.connect(nodeId, alpn)	Connect to a peer by public key
Endpoint.accept()	Accept incoming connection
Endpoint.nodeId()	Get our public key identity
Connection.openBi()	Open bidirectional stream (spoke side)
Connection.acceptBi()	Accept bidirectional stream (hub side)
SendStream.writeAll(data)	Send data on stream
RecvStream.readExact(len)	Read exact bytes from stream
Connection.remoteNodeId()	Get peer's public key
Connection.sendDatagram(data) / readDatagram()	Unreliable datagrams (fire-and-forget events)

Not exposed (but not critical):

• Endpoint.watch_addr() — detect network changes (workaround: detect connection failure)
• Connection.close_reason() — synchronous close check (workaround: await closed())
• Connection.stats() — observability (nice to have, not required)

IrohEventTarget Design

Same TypedEventTarget interface as WebSocketEventTarget and RedisEventTarget. Hub and spoke each create one per connection.

Protocol: Single bidirectional QUIC stream per connection, length-prefixed JSON messages. Spoke opens the stream with openBi(), hub accepts with acceptBi(). Same type + detail event shape as all other transports.

// Spoke side
const conn = await endpoint.connect(hubNodeId, "alkhub/1");
const eventTarget = await createSpokeIrohEventTarget(conn);

// Hub side
const conn = await endpoint.accept();
const eventTarget = await createHubIrohEventTarget(conn);

// Both sides — same TypedEventTarget interface
eventTarget.addEventListener("call.responded", (event) => { ... });
eventTarget.dispatchEvent(new CustomEvent("call.requested", { detail: { ... } }));

Framing: 4-byte big-endian length prefix + JSON payload. Necessary because QUIC streams are byte streams, not message streams. readExact() makes this trivial.

Connection startup: On connection, both sides exchange the operations they expose (same hub.register pattern as WebSocket). The NodeId serves as cryptographic identity — no separate API key exchange needed for authentication.

Reconnection: Same pattern as WebSocket — detect connection failure, reconnect, re-register. QUIC handles multipath better than TCP but the application still needs reconnection logic.

Comparison with WebSocketEventTarget:

Aspect	WebSocketEventTarget	IrohEventTarget
Connection	new WebSocket(url)	endpoint.connect(nodeId, alpn)
Accept	Hono WS upgrade	endpoint.accept()
Identity	API key/token in URL or first message	Ed25519 NodeId (cryptographic, mutual)
NAT traversal	Requires reverse proxy / CDN / tunnel	Built-in (relay + hole punching)
Framing	WS frames (built-in message boundary)	QUIC stream (needs length-prefix framing)
Hub behind NAT	Not possible without tunneling	Yes — spoke dials by NodeId
Browser	Yes (native WS)	Limited (WASM build, relay-only — use WS for browsers)

Multi-Node Scenarios (Future)

For 1:N fan-out (e.g., one event to 50 spokes), iroh-gossip is the right tool. No TS binding exposes it yet. Options when we need it:

1. Write a minimal Rust NAPI crate wrapping iroh-gossip::Gossip.subscribe() + broadcast() (~500 lines Rust)
2. Contribute gossip to @rayhanadev/iroh or @salvatoret/iroh
3. Use hub as a relay point (hub receives once, fans out to each spoke's IrohEventTarget individually)

For now, 1:1 connections are sufficient. The hub can fan out to multiple spokes by dispatching to each spoke's IrohEventTarget individually — same pattern as WebSocketEventTarget on the hub side.

Browser Considerations

Iroh in browsers is relay-only (no UDP hole punching from browser sandbox). This means:

• Browser spokes always route through relay servers
• WebSocketEventTarget is the right browser transport today (native, no extra deps)
• IrohEventTarget for browsers would use the WASM build over relay — future option

Migration Steps

Phase 1: Extract to standalone package

1. Create @alkdev/pubsub repo (or directory in a monorepo)
2. Copy source files from packages/core/pubsub/ with no modifications to core logic:
   • typed_event_target.ts → types.ts
   • create_pubsub.ts → create_pubsub.ts
   • redis_event_target.ts → event-target-redis.ts
   • operators.ts → operators.ts
3. Set up build pipeline (tsup, package.json, tsconfig)
4. Move Redis to peer dependency in package.json
5. Write missing tests: create_pubsub.test.ts, operators.test.ts
6. Add Redis subscription-receive and unsubscribe cleanup tests
7. Publish v0.1.0 to npm

Phase 2: Add adapters and improve coverage

8. Implement WebSocketEventTarget per spoke-runner.md spec
9. Implement IrohEventTarget — createHubIrohEventTarget / createSpokeIrohEventTarget with length-prefixed JSON framing over QUIC streams
10. Add Redis channel prefixing (alk:events:* or configurable prefix)
11. Add Redis error handling (connection errors, reconnection, parse errors)
12. Formalize InProcessEventTarget (explicit or just document that EventTarget is the default)
13. Write adapter tests (mock WS bidirectional flow, mock iroh connect/accept/stream)

Phase 3: Production hardening

14. Redis integration tests with real Redis instance
15. WebSocket integration tests with real WS server/client
16. Iroh integration tests — requires relay server or direct P2P between two endpoints
17. Reconnection logic for both WebSocket and Iroh adapters
18. Error propagation — connection failures should propagate to listeners gracefully

Phase 4: Integration back into alkhub_ts

19. Replace packages/core/pubsub/ with @alkdev/pubsub npm/JSR dependency
20. Update packages/core/deno.json and packages/core/mod.ts to import from @alkdev/pubsub
21. Remove ioredis from packages/core/deno.json (it moves to @alkdev/pubsub's peer deps, and hub uses it directly)
22. Update call protocol, hub, and spoke to use @alkdev/pubsub directly

Open Questions

1. Sub-path exports vs. barrel + tree-shaking? Typemap uses barrel-only + tree-shaking. Taskgraph uses barrel-only. Do we want sub-path exports for explicit adapter imports, or rely on tree-shaking?
2. Test runner: vitest (matches taskgraph) or deno test (matches current alkhub_ts)? If the package publishes to npm via tsup, vitest is the natural choice. If we also want to test in Deno, we could support both.
3. Deno-first or Node-first development? Current code has no Deno-specific APIs (it's all web standard). We could develop in either. Deno can import from npm. Node can't import from JSR without the JSR npm mirror. If we're using tsup for build, we're effectively Node-first for publishing, Deno-compatible for source.
4. When to implement WebSocketEventTarget and IrohEventTarget? Before or after extracting the package? The specs and interfaces are clear. Could implement both as part of the initial adapter set, since both follow the same TypedEventTarget pattern.
5. Iroh binding: Should we use @rayhanadev/iroh directly (v0.1.1, community binding, 9 commits, no tests) or write/publish our own @alkdev/iroh NAPI wrapper? The current binding works but has no tests and one author. Forking/forking-and-maintaining gives us control of the build pipeline.
6. Iroh + Deno: NAPI-RS .node binaries may need testing under Deno 2.x. If we're building with tsup for npm publish, the runtime is Node.js. For Deno-first development, we need to verify NAPI addons work.
7. Redis channel prefixing: Should the prefix be configurable per createRedisEventTarget({ prefix })? or hardcoded to alk:events:? Configurable is more flexible for multi-tenant scenarios.

Architecture Decision: WebSocket vs Iroh as Primary Transport

WebSocket is the right default for most deployments — it's native in browsers and Deno, well-supported, and requires no native addons. Iroh is the right choice when:

• The hub is behind NAT (dev laptops, home servers, no CDN)
• Spokes need to be reachable by the hub (push notifications to client spokes)
• Cryptographic identity is preferred over token-based auth
• P2P spoke-to-spoke communication is needed

A deployment can use both: WebSocketEventTarget for browser clients, IrohEventTarget for native spokes. Same TypedEventTarget interface, same call protocol, same PendingRequestMap.