Set up project structure, source files, and architecture docs
- Copy core source from alkhub_ts/packages/core/pubsub/ with import path fixups (typed_event_target.ts → types.ts, .ts → .js extensions) - Make PubSubPublishArgsByKey exported (was private type, needed by barrel) - Add package.json with sub-path exports and optional peer deps (ioredis) - Add tsup.config.ts with multi-entry + splitting for tree-shaking - Add tsconfig.json, vitest.config.ts, .gitignore - Add AGENTS.md with project conventions and adapter checklist - Add architecture docs following taskgraph/alkhub pattern: docs/architecture/README.md, api-surface.md, event-targets.md, iroh-transport.md, build-distribution.md - Add ADRs: 001-graphql-yoga-fork, 002-tree-shake-pattern - Copy migration research doc to docs/research/migration.md - Dual-license MIT OR Apache-2.0 (matching taskgraph)
This commit is contained in:
4
.gitignore
vendored
Normal file
4
.gitignore
vendored
Normal file
@@ -0,0 +1,4 @@
|
|||||||
|
dist/
|
||||||
|
node_modules/
|
||||||
|
coverage/
|
||||||
|
*.tsbuildinfo
|
||||||
110
AGENTS.md
Normal file
110
AGENTS.md
Normal file
@@ -0,0 +1,110 @@
|
|||||||
|
## Memory Tools (via @alkdev/open-memory plugin)
|
||||||
|
|
||||||
|
You have access to two tools for managing your context and accessing session history:
|
||||||
|
|
||||||
|
### memory({tool: "...", args: {...}})
|
||||||
|
|
||||||
|
Read-only tool for introspecting your session history and context state. Available operations:
|
||||||
|
- `memory({tool: "help"})` — full reference with examples
|
||||||
|
- `memory({tool: "summary"})` — quick counts of projects, sessions, messages, todos
|
||||||
|
- `memory({tool: "sessions"})` — list recent sessions (useful for finding past work)
|
||||||
|
- `memory({tool: "children", args: {sessionId: "ses_..."}})` — list sub-agent sessions spawned from a parent
|
||||||
|
- `memory({tool: "messages", args: {sessionId: "..."}})` — read a session's conversation
|
||||||
|
- `memory({tool: "messages", args: {sessionId: "...", role: "assistant"}})` — read only assistant messages
|
||||||
|
- `memory({tool: "messages", args: {sessionId: "...", showTools: true}})` — include tool-call output
|
||||||
|
- `memory({tool: "message", args: {messageId: "msg_..."}})` — read a single message by ID
|
||||||
|
- `memory({tool: "search", args: {query: "..."}})` — search across all conversations
|
||||||
|
- `memory({tool: "compactions", args: {sessionId: "..."}})` — view compaction checkpoints
|
||||||
|
- `memory({tool: "context"})` — check your current context window usage
|
||||||
|
|
||||||
|
### memory_compact()
|
||||||
|
|
||||||
|
Trigger compaction on the current session. This summarizes the conversation so far to free context space.
|
||||||
|
|
||||||
|
**When to use memory_compact:**
|
||||||
|
- When context is above 80% (check with `memory({tool: "context"})`)
|
||||||
|
- When you notice you're losing track of earlier conversation details
|
||||||
|
- At natural breakpoints in multi-step tasks (after completing a subtask, before starting a new one)
|
||||||
|
- When the system prompt shows a yellow/red/critical context warning
|
||||||
|
- Proactively, rather than waiting for automatic compaction at 92%
|
||||||
|
|
||||||
|
**When NOT to use memory_compact:**
|
||||||
|
- When context is below 50% (it wastes a compaction cycle)
|
||||||
|
- In the middle of a complex edit that you need immediate context for
|
||||||
|
- When the task is nearly complete (just finish the task instead)
|
||||||
|
|
||||||
|
Compaction preserves your most important context in a structured summary — you will continue the session with the summary as your starting point.
|
||||||
|
|
||||||
|
## Worktree Tool (via @alkimiadev/open-coordinator plugin)
|
||||||
|
|
||||||
|
You have access to the `worktree` tool for git worktree management and session coordination. Call with `{action, args}`:
|
||||||
|
|
||||||
|
### Coordinator Operations (available when session is not spawned by another session)
|
||||||
|
|
||||||
|
- `worktree({action: "list"})` — List git worktrees
|
||||||
|
- `worktree({action: "dashboard"})` — Worktree dashboard with session info
|
||||||
|
- `worktree({action: "spawn", args: {tasks: [...], prompt: "..."}})` — Spawn parallel worktrees + sessions
|
||||||
|
- `worktree({action: "sessions"})` — Query spawned session status
|
||||||
|
- `worktree({action: "message", args: {sessionID: "...", message: "..."}})` — Message a session
|
||||||
|
- `worktree({action: "abort", args: {sessionID: "..."}})` — Abort a session
|
||||||
|
- `worktree({action: "cleanup", args: {action: "remove", pathOrBranch: "..."}})` — Remove worktree
|
||||||
|
- `worktree({action: "help"})` — Show all available operations
|
||||||
|
|
||||||
|
### Implementation Operations (available when session is spawned by a coordinator)
|
||||||
|
|
||||||
|
- `worktree({action: "current"})` — Show your worktree mapping
|
||||||
|
- `worktree({action: "notify", args: {message: "...", level: "info|blocking"}})` — Report to coordinator
|
||||||
|
- `worktree({action: "status"})` — Show worktree git status
|
||||||
|
|
||||||
|
The plugin auto-injects `workdir` for bash commands when the session is mapped to a worktree.
|
||||||
|
|
||||||
|
## Project: @alkdev/pubsub
|
||||||
|
|
||||||
|
Type-safe publish/subscribe with pluggable event target adapters (in-process, Redis, WebSocket, Iroh). Core is adapted from graphql-yoga (MIT). Dual-licensed MIT / Apache-2.0.
|
||||||
|
|
||||||
|
### Commands
|
||||||
|
|
||||||
|
- `npm run build` — Build with tsup (ESM + CJS + declarations)
|
||||||
|
- `npm run lint` — Type-check with `tsc --noEmit`
|
||||||
|
- `npm test` — Run tests with vitest
|
||||||
|
- `npm run test:watch` — Watch mode
|
||||||
|
- `npm run test:coverage` — Coverage report (v8)
|
||||||
|
|
||||||
|
### Architecture
|
||||||
|
|
||||||
|
See `docs/architecture/` for full spec. Key points:
|
||||||
|
|
||||||
|
- **Barrel + sub-path exports**: `src/index.ts` re-exports core + operators. Each adapter has its own sub-path entry (`@alkdev/pubsub/event-target-redis`, etc.).
|
||||||
|
- **Peer dep isolation**: Redis and Iroh adapters are optional peer deps. Consumers only install the ones they need.
|
||||||
|
- **TypedEventTarget contract**: All adapters implement the same `addEventListener`/`dispatchEvent`/`removeEventListener` interface. `createPubSub` is transport-agnostic.
|
||||||
|
- **No comments in source**: Do not add comments to code unless explicitly asked.
|
||||||
|
- **License headers**: Files adapted from graphql-yoga must preserve their MIT attribution headers.
|
||||||
|
|
||||||
|
### Source Layout
|
||||||
|
|
||||||
|
```
|
||||||
|
src/
|
||||||
|
index.ts — Barrel: re-exports core API + operators
|
||||||
|
types.ts — TypedEvent, TypedEventTarget, etc. (adapted from graphql-yoga)
|
||||||
|
create_pubsub.ts — createPubSub factory (adapted from graphql-yoga)
|
||||||
|
operators.ts — filter, map, pipe (adapted from graphql-yoga)
|
||||||
|
event-target-redis.ts — createRedisEventTarget (peer dep: ioredis)
|
||||||
|
# Future adapters:
|
||||||
|
# event-target-websocket.ts — (peer dep: none, web standard)
|
||||||
|
# event-target-iroh.ts — (peer dep: @rayhanadev/iroh)
|
||||||
|
```
|
||||||
|
|
||||||
|
### Dependencies
|
||||||
|
|
||||||
|
Runtime: `@repeaterjs/repeater` (direct, ~3KB).
|
||||||
|
Peer (optional): `ioredis@^5.0.0` (Redis adapter), `@rayhanadev/iroh` (Iroh adapter, future).
|
||||||
|
Dev: `tsup`, `typescript`, `vitest`, `@vitest/coverage-v8`, `ioredis` (for type resolution).
|
||||||
|
|
||||||
|
### Adding an Adapter Checklist
|
||||||
|
|
||||||
|
1. Create `src/event-target-{name}.ts` implementing `TypedEventTarget`
|
||||||
|
2. Add entry to `tsup.config.ts` entry array
|
||||||
|
3. Add sub-path export to `package.json` exports map
|
||||||
|
4. Add peer dep to `package.json` peerDependencies (with peerDependenciesMeta optional: true)
|
||||||
|
5. Add to `src/index.ts` barrel re-export
|
||||||
|
6. Write tests in `test/event-target-{name}.test.ts`
|
||||||
190
LICENSE-APACHE
Normal file
190
LICENSE-APACHE
Normal file
@@ -0,0 +1,190 @@
|
|||||||
|
Apache License
|
||||||
|
Version 2.0, January 2004
|
||||||
|
http://www.apache.org/licenses/
|
||||||
|
|
||||||
|
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
|
||||||
|
|
||||||
|
1. Definitions.
|
||||||
|
|
||||||
|
"License" shall mean the terms and conditions for use, reproduction,
|
||||||
|
and distribution as defined by Sections 1 through 9 of this document.
|
||||||
|
|
||||||
|
"Licensor" shall mean the copyright owner or entity authorized by
|
||||||
|
the copyright owner that is granting the License.
|
||||||
|
|
||||||
|
"Legal Entity" shall mean the union of the acting entity and all
|
||||||
|
other entities that control, are controlled by, or are under common
|
||||||
|
control with that entity. For the purposes of this definition,
|
||||||
|
"control" means (i) the power, direct or indirect, to cause the
|
||||||
|
direction or management of such entity, whether by contract or
|
||||||
|
otherwise, or (ii) ownership of fifty percent (50%) or more of the
|
||||||
|
outstanding shares, or (iii) beneficial ownership of such entity.
|
||||||
|
|
||||||
|
"You" (or "Your") shall mean an individual or Legal Entity
|
||||||
|
exercising permissions granted by this License.
|
||||||
|
|
||||||
|
"Source" form shall mean the preferred form for making modifications,
|
||||||
|
including but not limited to software source code, documentation
|
||||||
|
source, and configuration files.
|
||||||
|
|
||||||
|
"Object" form shall mean any form resulting from mechanical
|
||||||
|
transformation or translation of a Source form, including but
|
||||||
|
not limited to compiled object code, generated documentation,
|
||||||
|
and conversions to other media types.
|
||||||
|
|
||||||
|
"Work" shall mean the work of authorship, whether in Source or
|
||||||
|
Object form, made available under the License, as indicated by a
|
||||||
|
copyright notice that is included in or attached to the work
|
||||||
|
(an example is provided in the Appendix below).
|
||||||
|
|
||||||
|
"Derivative Works" shall mean any work, whether in Source or Object
|
||||||
|
form, that is based on (or derived from) the Work and for which the
|
||||||
|
editorial revisions, annotations, elaborations, or other modifications
|
||||||
|
represent, as a whole, an original work of authorship. For the purposes
|
||||||
|
of this License, Derivative Works shall not include works that remain
|
||||||
|
separable from, or merely link (or bind by name) to the interfaces of,
|
||||||
|
the Work and Derivative Works thereof.
|
||||||
|
|
||||||
|
"Contribution" shall mean any work of authorship, including
|
||||||
|
the original version of the Work and any modifications or additions
|
||||||
|
to that Work or Derivative Works thereof, that is intentionally
|
||||||
|
submitted to Licensor for inclusion in the Work by the copyright owner
|
||||||
|
or by an individual or Legal Entity authorized to submit on behalf of
|
||||||
|
the copyright owner. For the purposes of this definition, "submitted"
|
||||||
|
means any form of electronic, verbal, or written communication sent
|
||||||
|
to the Licensor or its representatives, including but not limited to
|
||||||
|
communication on electronic mailing lists, source code control systems,
|
||||||
|
and issue tracking systems that are managed by, or on behalf of, the
|
||||||
|
Licensor for the purpose of discussing and improving the Work, but
|
||||||
|
excluding communication that is conspicuously marked or otherwise
|
||||||
|
designated in writing by the copyright owner as "Not a Contribution."
|
||||||
|
|
||||||
|
"Contributor" shall mean Licensor and any individual or Legal Entity
|
||||||
|
on behalf of whom a Contribution has been received by Licensor and
|
||||||
|
subsequently incorporated within the Work.
|
||||||
|
|
||||||
|
2. Grant of Copyright License. Subject to the terms and conditions of
|
||||||
|
this License, each Contributor hereby grants to You a perpetual,
|
||||||
|
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
||||||
|
copyright license to reproduce, prepare Derivative Works of,
|
||||||
|
publicly display, publicly perform, sublicense, and distribute the
|
||||||
|
Work and such Derivative Works in Source or Object form.
|
||||||
|
|
||||||
|
3. Grant of Patent License. Subject to the terms and conditions of
|
||||||
|
this License, each Contributor hereby grants to You a perpetual,
|
||||||
|
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
||||||
|
(except as stated in this section) patent license to make, have made,
|
||||||
|
use, offer to sell, sell, import, and otherwise transfer the Work,
|
||||||
|
where such license applies only to those patent claims licensable
|
||||||
|
by such Contributor that are necessarily infringed by their
|
||||||
|
Contribution(s) alone or by combination of their Contribution(s)
|
||||||
|
with the Work to which such Contribution(s) was submitted. If You
|
||||||
|
institute patent litigation against any entity (including a
|
||||||
|
cross-claim or counterclaim in a lawsuit) alleging that the Work
|
||||||
|
or a Contribution incorporated within the Work constitutes direct
|
||||||
|
or contributory patent infringement, then any patent licenses
|
||||||
|
granted to You under this License for that Work shall terminate
|
||||||
|
as of the date such litigation is filed.
|
||||||
|
|
||||||
|
4. Redistribution. You may reproduce and distribute copies of the
|
||||||
|
Work or Derivative Works thereof in any medium, with or without
|
||||||
|
modifications, and in Source or Object form, provided that You
|
||||||
|
meet the following conditions:
|
||||||
|
|
||||||
|
(a) You must give any other recipients of the Work or
|
||||||
|
Derivative Works a copy of this License; and
|
||||||
|
|
||||||
|
(b) You must cause any modified files to carry prominent notices
|
||||||
|
stating that You changed the files; and
|
||||||
|
|
||||||
|
(c) You must retain, in the Source form of any Derivative Works
|
||||||
|
that You distribute, all copyright, patent, trademark, and
|
||||||
|
attribution notices from the Source form of the Work,
|
||||||
|
excluding those notices that do not pertain to any part of
|
||||||
|
the Derivative Works; and
|
||||||
|
|
||||||
|
(d) If the Work includes a "NOTICE" text file as part of its
|
||||||
|
distribution, then any Derivative Works that You distribute must
|
||||||
|
include a readable copy of the attribution notices contained
|
||||||
|
within such NOTICE file, excluding those notices that do not
|
||||||
|
pertain to any part of the Derivative Works, in at least one
|
||||||
|
of the following places: within a NOTICE text file distributed
|
||||||
|
as part of the Derivative Works; within the Source form or
|
||||||
|
documentation, if provided along with the Derivative Works; or,
|
||||||
|
within a display generated by the Derivative Works, if and
|
||||||
|
wherever such third-party notices normally appear. The contents
|
||||||
|
of the NOTICE file are for informational purposes only and
|
||||||
|
do not modify the License. You may add Your own attribution
|
||||||
|
notices within Derivative Works that You distribute, alongside
|
||||||
|
or as an addendum to the NOTICE text from the Work, provided
|
||||||
|
that such additional attribution notices cannot be construed
|
||||||
|
as modifying the License.
|
||||||
|
|
||||||
|
You may add Your own copyright statement to Your modifications and
|
||||||
|
may provide additional or different license terms and conditions
|
||||||
|
for use, reproduction, or distribution of Your modifications, or
|
||||||
|
for any such Derivative Works as a whole, provided Your use,
|
||||||
|
reproduction, and distribution of the Work otherwise complies with
|
||||||
|
the conditions stated in this License.
|
||||||
|
|
||||||
|
5. Submission of Contributions. Unless You explicitly state otherwise,
|
||||||
|
any Contribution intentionally submitted for inclusion in the Work
|
||||||
|
by You to the Licensor shall be under the terms and conditions of
|
||||||
|
this License, without any additional terms or conditions.
|
||||||
|
Notwithstanding the above, nothing herein shall supersede or modify
|
||||||
|
the terms of any separate license agreement you may have executed
|
||||||
|
with Licensor regarding such Contributions.
|
||||||
|
|
||||||
|
6. Trademarks. This License does not grant permission to use the trade
|
||||||
|
names, trademarks, service marks, or product names of the Licensor,
|
||||||
|
except as required for reasonable and customary use in describing the
|
||||||
|
origin of the Work and reproducing the content of the NOTICE file.
|
||||||
|
|
||||||
|
7. Disclaimer of Warranty. Unless required by applicable law or
|
||||||
|
agreed to in writing, Licensor provides the Work (and each
|
||||||
|
Contributor provides its Contributions) on an "AS IS" BASIS,
|
||||||
|
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
|
||||||
|
implied, including, without limitation, any warranties or conditions
|
||||||
|
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
|
||||||
|
PARTICULAR PURPOSE. You are solely responsible for determining the
|
||||||
|
appropriateness of using or redistributing the Work and assume any
|
||||||
|
risks associated with Your exercise of permissions under this License.
|
||||||
|
|
||||||
|
8. Limitation of Liability. In no event and under no legal theory,
|
||||||
|
whether in tort (including negligence), contract, or otherwise,
|
||||||
|
unless required by applicable law (such as deliberate and grossly
|
||||||
|
negligent acts) or agreed to in writing, shall any Contributor be
|
||||||
|
liable to You for damages, including any direct, indirect, special,
|
||||||
|
incidental, or consequential damages of any character arising as a
|
||||||
|
result of this License or out of the use or inability to use the
|
||||||
|
Work (including but not limited to damages for loss of goodwill,
|
||||||
|
work stoppage, computer failure or malfunction, or any and all
|
||||||
|
other commercial damages or losses), even if such Contributor
|
||||||
|
has been advised of the possibility of such damages.
|
||||||
|
|
||||||
|
9. Accepting Warranty or Additional Liability. While redistributing
|
||||||
|
the Work or Derivative Works thereof, You may choose to offer,
|
||||||
|
and charge a fee for, acceptance of support, warranty, indemnity,
|
||||||
|
or other liability obligations and/or rights consistent with this
|
||||||
|
License. However, in accepting such obligations, You may act only
|
||||||
|
on Your own behalf and on Your sole responsibility, not on behalf
|
||||||
|
of any other Contributor, and only if You agree to indemnify,
|
||||||
|
defend, and hold each Contributor harmless for any liability
|
||||||
|
incurred by, or claims asserted against, such Contributor by reason
|
||||||
|
of your accepting any such warranty or additional liability.
|
||||||
|
|
||||||
|
END OF TERMS AND CONDITIONS
|
||||||
|
|
||||||
|
Copyright 2026 Alkimiadev
|
||||||
|
|
||||||
|
Licensed under the Apache License, Version 2.0 (the "License");
|
||||||
|
you may not use this file except in compliance with the License.
|
||||||
|
You may obtain a copy of the License at
|
||||||
|
|
||||||
|
http://www.apache.org/licenses/LICENSE-2.0
|
||||||
|
|
||||||
|
Unless required by applicable law or agreed to in writing, software
|
||||||
|
distributed under the License is distributed on an "AS IS" BASIS,
|
||||||
|
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||||
|
See the License for the specific language governing permissions and
|
||||||
|
limitations under the License.
|
||||||
21
LICENSE-MIT
Normal file
21
LICENSE-MIT
Normal file
@@ -0,0 +1,21 @@
|
|||||||
|
MIT License
|
||||||
|
|
||||||
|
Copyright (c) 2026 Alkimiadev
|
||||||
|
|
||||||
|
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||||
|
of this software and associated documentation files (the "Software"), to deal
|
||||||
|
in the Software without restriction, including without limitation the rights
|
||||||
|
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||||
|
copies of the Software, and to permit persons to whom the Software is
|
||||||
|
furnished to do so, subject to the following conditions:
|
||||||
|
|
||||||
|
The above copyright notice and this permission notice shall be included in all
|
||||||
|
copies or substantial portions of the Software.
|
||||||
|
|
||||||
|
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||||
|
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||||
|
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||||
|
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||||
|
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||||
|
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||||
|
SOFTWARE.
|
||||||
18
docs/architecture.md
Normal file
18
docs/architecture.md
Normal file
@@ -0,0 +1,18 @@
|
|||||||
|
# Architecture
|
||||||
|
|
||||||
|
> **This document has been decomposed into modular documents.** See [docs/architecture/](architecture/) for the current architecture specification.
|
||||||
|
|
||||||
|
| Document | Content |
|
||||||
|
|----------|---------|
|
||||||
|
| [architecture/README.md](architecture/README.md) | Overview, why this exists, interface contract, consumer context |
|
||||||
|
| [architecture/api-surface.md](architecture/api-surface.md) | createPubSub, PubSub types, operators |
|
||||||
|
| [architecture/event-targets.md](architecture/event-targets.md) | In-process, Redis, WebSocket adapters |
|
||||||
|
| [architecture/iroh-transport.md](architecture/iroh-transport.md) | Iroh P2P QUIC transport, framing, identity, hub/spoke |
|
||||||
|
| [architecture/build-distribution.md](architecture/build-distribution.md) | Dependencies, project structure, tree-shaking, sub-path exports |
|
||||||
|
|
||||||
|
### Design Decisions
|
||||||
|
|
||||||
|
| ADR | Decision |
|
||||||
|
|-----|----------|
|
||||||
|
| [001](architecture/decisions/001-graphql-yoga-fork.md) | Fork graphql-yoga pubsub rather than depend on it |
|
||||||
|
| [002](architecture/decisions/002-tree-shake-pattern.md) | Sub-path exports + peer deps for adapter isolation |
|
||||||
94
docs/architecture/README.md
Normal file
94
docs/architecture/README.md
Normal file
@@ -0,0 +1,94 @@
|
|||||||
|
---
|
||||||
|
status: draft
|
||||||
|
last_updated: 2026-04-30
|
||||||
|
---
|
||||||
|
|
||||||
|
# @alkdev/pubsub Architecture
|
||||||
|
|
||||||
|
Type-safe publish/subscribe with pluggable event target adapters. The core (`createPubSub` + `TypedEventTarget` + operators) has no transport dependency. Each adapter (Redis, WebSocket, Iroh) is an isolated module that only imports its own peer dependency.
|
||||||
|
|
||||||
|
## Why This Exists
|
||||||
|
|
||||||
|
Extracted from `@alkdev/alkhub_ts/packages/core/pubsub/`, which itself was adapted from `@graphql-yoga/subscription` and `@graphql-yoga/typed-event-target`. The pubsub module was already self-contained within alkhub — zero cross-module imports from operations, config, logger, or MCP. Extracting it into a standalone package:
|
||||||
|
|
||||||
|
1. **Reduces coupling** — alkhub depends on pubsub, not the other way around
|
||||||
|
2. **Enables reuse** — multiple alkhub packages can share the same pubsub instance
|
||||||
|
3. **Isolates peer deps** — Redis and Iroh are heavy native dependencies; consumers that don't need them shouldn't carry them
|
||||||
|
4. **Matches established pattern** — `@alkdev/taskgraph` and `@alkdev/typemap` already use the standalone-package pattern
|
||||||
|
|
||||||
|
## Core Principle
|
||||||
|
|
||||||
|
**The TypedEventTarget interface is the contract.** All transports implement the same `addEventListener` / `dispatchEvent` / `removeEventListener` surface. `createPubSub` doesn't know or care which transport is in use — it just dispatches events to whatever `TypedEventTarget` it was given.
|
||||||
|
|
||||||
|
This means swapping from in-process to Redis to WebSocket to Iroh is a one-line config change:
|
||||||
|
|
||||||
|
```ts
|
||||||
|
const pubsub = createPubSub<MyEventMap>({
|
||||||
|
eventTarget: createRedisEventTarget({ publishClient, subscribeClient }),
|
||||||
|
});
|
||||||
|
```
|
||||||
|
|
||||||
|
## What This Package Provides
|
||||||
|
|
||||||
|
- **Core** — `createPubSub`, `TypedEventTarget`, `TypedEvent`, topic scoping, `filter`/`map`/`pipe` operators
|
||||||
|
- **Adapters** (each is a peer-dep island, importable via sub-path export):
|
||||||
|
- In-process (default `EventTarget`, no adapter needed)
|
||||||
|
- Redis (`@alkdev/pubsub/event-target-redis`, peer dep: `ioredis`)
|
||||||
|
- WebSocket (future: `@alkdev/pubsub/event-target-websocket`)
|
||||||
|
- Iroh (future: `@alkdev/pubsub/event-target-iroh`, peer dep: `@rayhanadev/iroh`)
|
||||||
|
|
||||||
|
## Consumer Context
|
||||||
|
|
||||||
|
### alkhub (hub-spoke coordinator)
|
||||||
|
|
||||||
|
The hub uses pubsub for event routing between operations, runners, and the SSE interface. The event map is the call protocol — typed JSON events (`call.requested`, `call.responded`, `session.status`, etc.). Transport choice depends on deployment:
|
||||||
|
|
||||||
|
| Deployment | Transport |
|
||||||
|
|------------|-----------|
|
||||||
|
| Single-process hub | In-process (default) |
|
||||||
|
| Hub + worker processes | Redis |
|
||||||
|
| Hub + remote spokes | WebSocket or Iroh |
|
||||||
|
|
||||||
|
### Future: standalone spoke SDK
|
||||||
|
|
||||||
|
Spokes will import `@alkdev/pubsub` directly to create their event target (WebSocket or Iroh) and wire it into `createPubSub`. The call protocol types live in a separate `@alkdev/call-protocol` package (not yet extracted).
|
||||||
|
|
||||||
|
## Threat Model
|
||||||
|
|
||||||
|
- **Fork provenance** — core pubsub and typed event target are adapted from graphql-yoga (MIT). All original copyright notices are preserved in file headers. See [ADR-001](decisions/001-graphql-yoga-fork.md).
|
||||||
|
- **Peer dep isolation** — Redis and Iroh are optional peer dependencies. A consumer that only needs in-process transport installs zero extra packages. A consumer using Redis but not Iroh installs `ioredis` only.
|
||||||
|
- **Type-only imports** — `event-target-redis.ts` imports `ioredis` types only at compile time. At runtime, the consumer must provide the actual `Redis`/`Cluster` instances.
|
||||||
|
|
||||||
|
## Architecture Documents
|
||||||
|
|
||||||
|
| Document | Content |
|
||||||
|
|----------|---------|
|
||||||
|
| [api-surface.md](api-surface.md) | createPubSub factory, PubSub types, operators, TypedEventTarget types |
|
||||||
|
| [event-targets.md](event-targets.md) | In-process, Redis, WebSocket adapters — interface, configuration, limitations |
|
||||||
|
| [iroh-transport.md](iroh-transport.md) | Iroh P2P QUIC transport — protocol, framing, identity, hub/spoke sides, reconnection |
|
||||||
|
| [build-distribution.md](build-distribution.md) | Dependencies, project structure, tree-shaking, sub-path exports, targets |
|
||||||
|
|
||||||
|
## Document Lifecycle
|
||||||
|
|
||||||
|
Architecture documents use YAML frontmatter with `status` and `last_updated` fields:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
---
|
||||||
|
status: draft | stable | deprecated
|
||||||
|
last_updated: YYYY-MM-DD
|
||||||
|
---
|
||||||
|
```
|
||||||
|
|
||||||
|
| Status | Meaning | Transitions |
|
||||||
|
|--------|---------|-------------|
|
||||||
|
| `draft` | Under active development. Content may change. | → `stable` when implementation is complete and tests verify API contract. |
|
||||||
|
| `stable` | API contracts are locked. Changes require review cycle. | → `deprecated` when superseded. |
|
||||||
|
| `deprecated` | Superseded. Kept for reference. | Removed when no longer referenced. |
|
||||||
|
|
||||||
|
## References
|
||||||
|
|
||||||
|
- Source: `@alkdev/alkhub_ts/packages/core/pubsub/`
|
||||||
|
- Upstream: `@graphql-yoga/subscription` and `@graphql-yoga/typed-event-target` (MIT)
|
||||||
|
- alkhub pubsub-redis doc: `@alkdev/alkhub_ts/docs/architecture/pubsub-redis.md`
|
||||||
|
- alkhub spoke-runner doc: `@alkdev/alkhub_ts/docs/architecture/spoke-runner.md`
|
||||||
|
- Migration research: `docs/research/migration.md`
|
||||||
101
docs/architecture/api-surface.md
Normal file
101
docs/architecture/api-surface.md
Normal file
@@ -0,0 +1,101 @@
|
|||||||
|
---
|
||||||
|
status: draft
|
||||||
|
last_updated: 2026-04-30
|
||||||
|
---
|
||||||
|
|
||||||
|
# API Surface
|
||||||
|
|
||||||
|
Core pubsub creation, types, and operators. No transport dependencies.
|
||||||
|
|
||||||
|
## `createPubSub`
|
||||||
|
|
||||||
|
```ts
|
||||||
|
function createPubSub<TPubSubPublishArgsByKey extends PubSubPublishArgsByKey>(
|
||||||
|
config?: PubSubConfig<TPubSubPublishArgsByKey>,
|
||||||
|
): PubSub<TPubSubPublishArgsByKey>;
|
||||||
|
```
|
||||||
|
|
||||||
|
Factory function. Accepts an optional `eventTarget` config. If none is provided, uses `new EventTarget()` (in-process).
|
||||||
|
|
||||||
|
### Topic Scoping
|
||||||
|
|
||||||
|
Topics can be scoped with an id:
|
||||||
|
|
||||||
|
- `pubsub.publish("session.status", projectId, payload)` → dispatches to topic `session.status:{projectId}`
|
||||||
|
- `pubsub.subscribe("session.status", projectId)` → subscribes to topic `session.status:{projectId}` only
|
||||||
|
- `pubsub.publish("session.status", payload)` → dispatches to topic `session.status` (unscoped)
|
||||||
|
- `pubsub.subscribe("session.status")` → subscribes to topic `session.status` (unscoped)
|
||||||
|
|
||||||
|
The topic string is either the routing key directly (unscoped) or `{routingKey}:{id}` (scoped). This maps naturally to Redis channel naming and WebSocket message routing.
|
||||||
|
|
||||||
|
### `PubSubPublishArgsByKey`
|
||||||
|
|
||||||
|
The type parameter that defines the event map:
|
||||||
|
|
||||||
|
```ts
|
||||||
|
type PubSubPublishArgsByKey = {
|
||||||
|
[key: string]: [] | [unknown] | [number | string, unknown];
|
||||||
|
};
|
||||||
|
```
|
||||||
|
|
||||||
|
- `[]` — event with no payload (trigger only)
|
||||||
|
- `[payload]` — unscoped event with payload
|
||||||
|
- `[id, payload]` — scoped event with id and payload
|
||||||
|
|
||||||
|
### `PubSub.subscribe()`
|
||||||
|
|
||||||
|
Returns a `Repeater<unknown>` (async iterable). Consumers iterate with `for await`:
|
||||||
|
|
||||||
|
```ts
|
||||||
|
for await (const payload of pubsub.subscribe("session.status")) {
|
||||||
|
// handle payload
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
The `Repeater` automatically cleans up its `addEventListener` when the consumer breaks out of the loop (the `stop` promise resolves).
|
||||||
|
|
||||||
|
## Types
|
||||||
|
|
||||||
|
| Export | Source | Description |
|
||||||
|
|--------|--------|-------------|
|
||||||
|
| `TypedEvent<TType, TDetail>` | `types.ts` | Event with typed `type` and `detail`. Omits `CustomEvent`'s untyped fields. |
|
||||||
|
| `TypedEventTarget<TEvent>` | `types.ts` | Extends `EventTarget` with typed `addEventListener`, `dispatchEvent`, `removeEventListener`. |
|
||||||
|
| `TypedEventListener<TEvent>` | `types.ts` | `(evt: TEvent) => void` |
|
||||||
|
| `TypedEventListenerObject<TEvent>` | `types.ts` | `{ handleEvent(object: TEvent): void }` |
|
||||||
|
| `TypedEventListenerOrEventListenerObject<TEvent>` | `types.ts` | Union of the above |
|
||||||
|
| `PubSub<TPubSubPublishArgsByKey>` | `create_pubsub.ts` | `{ publish, subscribe }` |
|
||||||
|
| `PubSubConfig<TPubSubPublishArgsByKey>` | `create_pubsub.ts` | `{ eventTarget?: PubSubEventTarget }` |
|
||||||
|
| `PubSubEvent<TPubSubPublishArgsByKey, TKey>` | `create_pubsub.ts` | Derived `TypedEvent` for a specific event key |
|
||||||
|
| `PubSubEventTarget<TPubSubPublishArgsByKey>` | `create_pubsub.ts` | `TypedEventTarget<PubSubEvent<...>>` |
|
||||||
|
|
||||||
|
## Operators
|
||||||
|
|
||||||
|
All operators return `Repeater` instances and work with any async iterable.
|
||||||
|
|
||||||
|
### `filter`
|
||||||
|
|
||||||
|
```ts
|
||||||
|
function filter<T>(filterFn: (value: T) => Promise<boolean> | boolean): (source: AsyncIterable<T>) => Repeater<T>;
|
||||||
|
```
|
||||||
|
|
||||||
|
Type-narrowing overload available: `filter<T, U extends T>(fn: (input: T) => input is U)`.
|
||||||
|
|
||||||
|
### `map`
|
||||||
|
|
||||||
|
```ts
|
||||||
|
function map<T, O>(mapper: (input: T) => Promise<O> | O): (source: AsyncIterable<T>) => Repeater<O>;
|
||||||
|
```
|
||||||
|
|
||||||
|
### `pipe`
|
||||||
|
|
||||||
|
```ts
|
||||||
|
function pipe<A, B>(a: A, ab: (a: A) => B): B;
|
||||||
|
function pipe<A, B, C>(a: A, ab: (a: A) => B, bc: (b: B) => C): C;
|
||||||
|
// up to 5 arguments
|
||||||
|
```
|
||||||
|
|
||||||
|
Compose operators: `pipe(pubsub.subscribe("myEvent"), filter(isRelevant), map(transform))`
|
||||||
|
|
||||||
|
## Attribution
|
||||||
|
|
||||||
|
`createPubSub` and operators are adapted from `@graphql-yoga/subscription` (MIT). `TypedEventTarget` types are adapted from `@graphql-yoga/typed-event-target` (MIT). See file headers for full license text.
|
||||||
113
docs/architecture/build-distribution.md
Normal file
113
docs/architecture/build-distribution.md
Normal file
@@ -0,0 +1,113 @@
|
|||||||
|
---
|
||||||
|
status: draft
|
||||||
|
last_updated: 2026-04-30
|
||||||
|
---
|
||||||
|
|
||||||
|
# Build & Distribution
|
||||||
|
|
||||||
|
Dependencies, project structure, tree-shaking, sub-path exports, and build targets.
|
||||||
|
|
||||||
|
## Dependencies
|
||||||
|
|
||||||
|
| Package | Type | Purpose |
|
||||||
|
|---------|------|---------|
|
||||||
|
| `@repeaterjs/repeater` | direct | Small (~3KB). Core async iterable primitive for `subscribe()`. |
|
||||||
|
| `ioredis` | peer (optional) | Redis client. Only imported by `event-target-redis.ts`. Type-only import at compile time. |
|
||||||
|
| `@rayhanadev/iroh` | peer (optional, future) | Iroh NAPI-RS binding. Only imported by `event-target-iroh.ts`. |
|
||||||
|
|
||||||
|
No other external dependencies. No logger dependency.
|
||||||
|
|
||||||
|
## Project Structure
|
||||||
|
|
||||||
|
```
|
||||||
|
@alkdev/pubsub/
|
||||||
|
src/
|
||||||
|
index.ts # Barrel: re-exports core API
|
||||||
|
types.ts # TypedEvent, TypedEventTarget, etc.
|
||||||
|
create_pubsub.ts # createPubSub factory
|
||||||
|
operators.ts # filter, map, pipe
|
||||||
|
event-target-redis.ts # createRedisEventTarget (peer dep: ioredis)
|
||||||
|
# Future adapters (each is its own entry point + peer dep island):
|
||||||
|
# event-target-websocket.ts # peer dep: none (web standard)
|
||||||
|
# event-target-iroh.ts # peer dep: @rayhanadev/iroh
|
||||||
|
test/
|
||||||
|
create_pubsub.test.ts
|
||||||
|
operators.test.ts
|
||||||
|
event-target-redis.test.ts
|
||||||
|
# event-target-websocket.test.ts
|
||||||
|
# event-target-iroh.test.ts
|
||||||
|
docs/
|
||||||
|
architecture.md
|
||||||
|
architecture/
|
||||||
|
research/
|
||||||
|
package.json
|
||||||
|
tsconfig.json
|
||||||
|
tsup.config.ts
|
||||||
|
vitest.config.ts
|
||||||
|
```
|
||||||
|
|
||||||
|
## Sub-Path Exports
|
||||||
|
|
||||||
|
We use explicit sub-path exports rather than barrel-only + tree-shaking. Each adapter is importable by its own path:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"exports": {
|
||||||
|
".": { ... },
|
||||||
|
"./event-target-redis": { ... },
|
||||||
|
"./event-target-websocket": { ... },
|
||||||
|
"./event-target-iroh": { ... }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### Why Sub-Path Exports
|
||||||
|
|
||||||
|
- **Explicit** — doesn't rely on bundler tree-shaking behavior
|
||||||
|
- **Peer dep isolation** — `import from '@alkdev/pubsub/event-target-redis'` makes the dependency on ioredis explicit at the import site
|
||||||
|
- **Consistent with typemap pattern** — typemap's peer deps (zod, valibot, typebox) are each their own module; sub-path exports make this explicit at the package boundary
|
||||||
|
|
||||||
|
Sub-path entries are added as adapters are implemented. The barrel `index.ts` also re-exports everything for convenience — consumers who want tree-shaking can import from the barrel and rely on their bundler.
|
||||||
|
|
||||||
|
## Peer Dependencies
|
||||||
|
|
||||||
|
| Peer Dep | Required By | Optional |
|
||||||
|
|----------|-------------|----------|
|
||||||
|
| `ioredis@^5.0.0` | `event-target-redis` | Yes |
|
||||||
|
| `@rayhanadev/iroh` | `event-target-iroh` (future) | Yes |
|
||||||
|
|
||||||
|
Optional peer deps means `npm install @alkdev/pubsub` does NOT install ioredis or iroh. Consumers opt in by installing the peer dep and importing from the sub-path.
|
||||||
|
|
||||||
|
## Build
|
||||||
|
|
||||||
|
- **Tool**: `tsup` — produces dual ESM + CJS with declarations automatically
|
||||||
|
- **Entry points**: `src/index.ts`, `src/event-target-redis.ts`, plus future adapters
|
||||||
|
- **Format**: ESM + CJS
|
||||||
|
- **Target**: `es2022`
|
||||||
|
- **Splitting**: enabled (tsup code splitting for shared chunks)
|
||||||
|
|
||||||
|
```ts
|
||||||
|
// tsup.config.ts
|
||||||
|
import { defineConfig } from 'tsup';
|
||||||
|
|
||||||
|
export default defineConfig({
|
||||||
|
entry: ['src/index.ts', 'src/event-target-redis.ts'],
|
||||||
|
format: ['esm', 'cjs'],
|
||||||
|
dts: true,
|
||||||
|
sourcemap: true,
|
||||||
|
clean: true,
|
||||||
|
splitting: true,
|
||||||
|
target: 'es2022',
|
||||||
|
});
|
||||||
|
```
|
||||||
|
|
||||||
|
## Testing
|
||||||
|
|
||||||
|
- **Runner**: `vitest` — matches taskgraph, natural fit with tsup/Node build pipeline
|
||||||
|
- **Config**: `vitest.config.ts` with `globals: true`
|
||||||
|
|
||||||
|
## Targets
|
||||||
|
|
||||||
|
- **Publish**: npm (`@alkdev/pubsub`)
|
||||||
|
- **Runtime**: Node 18+, Deno, Bun — pure JS (except iroh adapter which requires NAPI-RS)
|
||||||
|
- **Deno compatibility**: Source is standard TypeScript with no Deno-specific APIs. Deno can import from npm or JSR.
|
||||||
28
docs/architecture/decisions/001-graphql-yoga-fork.md
Normal file
28
docs/architecture/decisions/001-graphql-yoga-fork.md
Normal file
@@ -0,0 +1,28 @@
|
|||||||
|
# ADR-001: Fork graphql-yoga pubsub
|
||||||
|
|
||||||
|
**Status**: Accepted
|
||||||
|
**Date**: 2026-04-30
|
||||||
|
|
||||||
|
## Context
|
||||||
|
|
||||||
|
`createPubSub`, `TypedEventTarget`, and operators are adapted from `@graphql-yoga/subscription` and `@graphql-yoga/typed-event-target` (MIT). We carried these into alkhub with modifications (native CustomEvent, our TypedEventTarget types, removed tslib). Now we're extracting to a standalone package.
|
||||||
|
|
||||||
|
## Decision
|
||||||
|
|
||||||
|
Fork (continue carrying adapted code) rather than depend on graphql-yoga packages directly.
|
||||||
|
|
||||||
|
## Rationale
|
||||||
|
|
||||||
|
1. **Different evolution path** — graphql-yoga's pubsub is tailored for GraphQL subscriptions. Our use case is general-purpose event routing with multiple transports. The APIs will diverge further as we add WebSocket and Iroh adapters.
|
||||||
|
|
||||||
|
2. **Dependency reduction** — graphql-yoga's subscription package pulls in `@whatwg-node/events` and `tslib`. We don't need either — we use native `CustomEvent` and no tslib runtime.
|
||||||
|
|
||||||
|
3. **Control over types** — graphql-yoga's `TypedEventTarget` uses their own event type hierarchy. We use a simpler one that maps directly to `CustomEvent`. Maintaining our own types avoids version coupling.
|
||||||
|
|
||||||
|
4. **Already forked** — the code in alkhub already diverged from the original. The license headers are in place. A clean extraction doesn't change the provenance story.
|
||||||
|
|
||||||
|
## Consequences
|
||||||
|
|
||||||
|
- Must preserve MIT license headers in all forked files
|
||||||
|
- Must update attribution if we make significant changes beyond the original fork scope
|
||||||
|
- No automatic updates from graphql-yoga — we carry our own maintenance burden
|
||||||
36
docs/architecture/decisions/002-tree-shake-pattern.md
Normal file
36
docs/architecture/decisions/002-tree-shake-pattern.md
Normal file
@@ -0,0 +1,36 @@
|
|||||||
|
# ADR-002: Sub-Path Exports + Peer Deps for Adapter Isolation
|
||||||
|
|
||||||
|
**Status**: Accepted
|
||||||
|
**Date**: 2026-04-30
|
||||||
|
|
||||||
|
## Context
|
||||||
|
|
||||||
|
Each event target adapter has different peer dependencies (ioredis for Redis, @rayhanadev/iroh for Iroh). Consumers that only use in-process or one adapter should not be forced to install peer deps for adapters they don't use. Two approaches:
|
||||||
|
|
||||||
|
1. **Barrel + tree-shaking** — single entry point, rely on bundler to drop unused adapters
|
||||||
|
2. **Sub-path exports** — explicit per-adapter entry points in `package.json` exports map
|
||||||
|
|
||||||
|
Typemap uses barrel-only. Taskgraph uses plain barrel with no sub-path exports.
|
||||||
|
|
||||||
|
## Decision
|
||||||
|
|
||||||
|
Use sub-path exports with optional peer dependencies.
|
||||||
|
|
||||||
|
## Rationale
|
||||||
|
|
||||||
|
1. **Explicit dependency declaration** — `import { createRedisEventTarget } from '@alkdev/pubsub/event-target-redis'` makes it clear at the import site that this module needs ioredis. A barrel import doesn't.
|
||||||
|
|
||||||
|
2. **No bundler reliance** — sub-path exports don't depend on the consumer's bundler correctly tree-shaking. Not all consumers use bundlers (Deno, Node with `--experimental-strip-types`).
|
||||||
|
|
||||||
|
3. **peerDependenciesMeta** — npm treats optional peer deps as install warnings, not errors. `npm install @alkdev/pubsub` installs only the core. `npm install @alkdev/pubsub ioredis` gets Redis support.
|
||||||
|
|
||||||
|
4. **Consistent with typemap's philosophy** — typemap's peer deps (zod, valibot, typebox) are each their own module island. Sub-path exports make this explicit at the package boundary. We're adding the package.json entry points that typemap doesn't have.
|
||||||
|
|
||||||
|
5. **Incremental** — adapters can be added one at a time. Each new adapter adds one entry to the exports map and one entry point to tsup.
|
||||||
|
|
||||||
|
## Consequences
|
||||||
|
|
||||||
|
- More entries in `package.json` exports — maintenance burden scales with adapter count
|
||||||
|
- Both barrel and sub-path work — barrel re-exports everything for convenience, sub-path for explicitness
|
||||||
|
- tsup must list each adapter as a separate entry point
|
||||||
|
- Consumer docs should recommend sub-path imports for adapter-specific code
|
||||||
111
docs/architecture/event-targets.md
Normal file
111
docs/architecture/event-targets.md
Normal file
@@ -0,0 +1,111 @@
|
|||||||
|
---
|
||||||
|
status: draft
|
||||||
|
last_updated: 2026-04-30
|
||||||
|
---
|
||||||
|
|
||||||
|
# Event Target Adapters
|
||||||
|
|
||||||
|
In-process, Redis, and WebSocket event targets. All implement `TypedEventTarget<TEvent>`.
|
||||||
|
|
||||||
|
## Interface Contract
|
||||||
|
|
||||||
|
Every adapter must implement:
|
||||||
|
|
||||||
|
| Method | Behavior |
|
||||||
|
|--------|----------|
|
||||||
|
| `addEventListener(type, callback)` | Register listener for event type. Callback receives `CustomEvent` with typed `detail`. |
|
||||||
|
| `dispatchEvent(event)` | Send/dispatch event. Returns `boolean` (always `true` for non-cancelable events). |
|
||||||
|
| `removeEventListener(type, callback)` | Unregister listener. Clean up underlying subscription when no listeners remain for a topic. |
|
||||||
|
|
||||||
|
## In-Process (Default)
|
||||||
|
|
||||||
|
No adapter needed. `createPubSub` uses `new EventTarget()` by default. This works for single-process deployments where all pubsub participants share the same memory.
|
||||||
|
|
||||||
|
No explicit `InProcessEventTarget` class — the web standard `EventTarget` already implements the interface. Could be formalized later if a name makes the API clearer, but `new EventTarget()` is already the standard.
|
||||||
|
|
||||||
|
## Redis
|
||||||
|
|
||||||
|
**Import**: `@alkdev/pubsub/event-target-redis`
|
||||||
|
**Peer dep**: `ioredis@^5.0.0` (optional)
|
||||||
|
|
||||||
|
### `createRedisEventTarget`
|
||||||
|
|
||||||
|
```ts
|
||||||
|
function createRedisEventTarget<TEvent extends TypedEvent>(
|
||||||
|
args: CreateRedisEventTargetArgs,
|
||||||
|
): TypedEventTarget<TEvent>;
|
||||||
|
```
|
||||||
|
|
||||||
|
### `CreateRedisEventTargetArgs`
|
||||||
|
|
||||||
|
| Field | Type | Required | Description |
|
||||||
|
|-------|------|----------|-------------|
|
||||||
|
| `publishClient` | `Redis \| Cluster` | Yes | ioredis client for publishing. Can share a connection. |
|
||||||
|
| `subscribeClient` | `Redis \| Cluster` | Yes | ioredis client for subscribing. Must be dedicated — Redis requires subscriber connections to only receive messages. |
|
||||||
|
| `serializer` | `{ stringify, parse }` | No | Custom serializer. Defaults to `JSON`. |
|
||||||
|
|
||||||
|
### How It Works
|
||||||
|
|
||||||
|
- `dispatchEvent` → `publishClient.publish(event.type, serializer.stringify(event.detail))`
|
||||||
|
- `addEventListener` → `subscribeClient.subscribe(topic)`, track callbacks per topic
|
||||||
|
- `removeEventListener` → remove callback; if no callbacks remain for topic, `subscribeClient.unsubscribe(topic)`
|
||||||
|
|
||||||
|
### Channel Naming
|
||||||
|
|
||||||
|
Currently uses raw event type as Redis channel name (e.g., `session.status:proj_123`). Architecture recommends `alk:events:{eventType}` prefix but this is not yet implemented. Should be configurable: `createRedisEventTarget({ ..., prefix: "alk:events:" })`.
|
||||||
|
|
||||||
|
### Limitations (Current)
|
||||||
|
|
||||||
|
- **No error handling** — connection failures, reconnection, and message parse errors are not handled
|
||||||
|
- **No channel prefix** — raw event types as channel names risk collision in shared Redis instances
|
||||||
|
- **No unsubscribe cleanup on client disconnect** — if the subscribe client disconnects, registered callbacks remain in the map but will never fire
|
||||||
|
|
||||||
|
### Test Coverage
|
||||||
|
|
||||||
|
5 tests in alkhub (publish path only, mocked ioredis). No tests for subscription-receive path, unsubscribe cleanup, or error handling.
|
||||||
|
|
||||||
|
## WebSocket
|
||||||
|
|
||||||
|
**Import**: `@alkdev/pubsub/event-target-websocket` (not yet implemented)
|
||||||
|
**Peer dep**: none (WebSocket is a web standard)
|
||||||
|
|
||||||
|
### Design (Spec from `spoke-runner.md`)
|
||||||
|
|
||||||
|
```ts
|
||||||
|
class WebSocketEventTarget implements TypedEventTarget<any> {
|
||||||
|
private listeners = new Map<string, Set<(event: CustomEvent) => void>>()
|
||||||
|
|
||||||
|
constructor(private ws: WebSocket) {
|
||||||
|
ws.onmessage = (msg) => {
|
||||||
|
const { type, payload } = JSON.parse(msg.data as string)
|
||||||
|
const event = new CustomEvent(type, { detail: payload })
|
||||||
|
for (const listener of this.listeners.get(type) ?? []) {
|
||||||
|
listener(event)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
dispatchEvent(event: CustomEvent): boolean {
|
||||||
|
this.ws.send(JSON.stringify({ type: event.type, payload: event.detail }))
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
|
||||||
|
addEventListener(type: string, listener: (event: CustomEvent) => void): void { ... }
|
||||||
|
removeEventListener(type: string, listener: (event: CustomEvent) => void): void { ... }
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### Key Properties
|
||||||
|
|
||||||
|
- **Bidirectional** — `dispatchEvent` sends over WS, `addEventListener` receives from WS
|
||||||
|
- **Per-connection** — hub creates one per spoke connection
|
||||||
|
- **JSON framing** — WebSocket provides native message boundaries (no length-prefix needed)
|
||||||
|
- **No native deps** — works in browsers and Node
|
||||||
|
|
||||||
|
### Gap: Reconnection
|
||||||
|
|
||||||
|
WebSocket connections drop. On reconnect, the spoke must re-register with the hub (same `hub.register` flow). The `WebSocketEventTarget` itself is per-connection — a new connection means a new event target instance. Reconnection logic belongs to the spoke lifecycle, not the event target.
|
||||||
|
|
||||||
|
### Gap: Hub-Side Architecture
|
||||||
|
|
||||||
|
The hub needs per-connection event target + `PendingRequestMap` creation on accept, cleanup on disconnect. This is a hub architectural concern, not a pubsub concern. See `@alkdev/alkhub_ts/docs/architecture/spoke-runner.md`.
|
||||||
140
docs/architecture/iroh-transport.md
Normal file
140
docs/architecture/iroh-transport.md
Normal file
@@ -0,0 +1,140 @@
|
|||||||
|
---
|
||||||
|
status: draft
|
||||||
|
last_updated: 2026-04-30
|
||||||
|
---
|
||||||
|
|
||||||
|
# Iroh Transport
|
||||||
|
|
||||||
|
P2P QUIC event target using iroh. More complex than the other transports due to NAT traversal, crypto identity, and byte-stream framing.
|
||||||
|
|
||||||
|
**Import**: `@alkdev/pubsub/event-target-iroh` (not yet implemented)
|
||||||
|
**Peer dep**: `@rayhanadev/iroh` (optional, NAPI-RS native addon)
|
||||||
|
|
||||||
|
## Why Iroh
|
||||||
|
|
||||||
|
WebSocket requires the hub to have a publicly reachable address. Iroh solves:
|
||||||
|
|
||||||
|
1. **Hub behind NAT** — spokes dial by `NodeId` through relay servers, no public IP needed
|
||||||
|
2. **Spoke push** — hub can initiate connections to spokes by `NodeId` (impossible with WS without polling)
|
||||||
|
3. **P2P spoke-to-spoke** — direct spoke-to-spoke communication without routing through hub
|
||||||
|
4. **Cryptographic identity** — Ed25519 `NodeId` doubles as spoke authentication
|
||||||
|
|
||||||
|
## Iroh Binding
|
||||||
|
|
||||||
|
Using `@rayhanadev/iroh` (v0.1.1) as the NAPI-RS binding. Community binding, one author, no tests. It has everything needed for hub-spoke 1:1 bidirectional streams:
|
||||||
|
|
||||||
|
| Method | Purpose |
|
||||||
|
|--------|---------|
|
||||||
|
| `Endpoint.create()` / `createWithOptions({ alpns })` | Create QUIC endpoint |
|
||||||
|
| `Endpoint.connect(nodeId, alpn)` | Connect to a peer by public key |
|
||||||
|
| `Endpoint.accept()` | Accept incoming connection |
|
||||||
|
| `Endpoint.nodeId()` | Get our public key identity |
|
||||||
|
| `Connection.openBi()` | Open bidirectional stream (spoke side) |
|
||||||
|
| `Connection.acceptBi()` | Accept bidirectional stream (hub side) |
|
||||||
|
| `SendStream.writeAll(data)` | Send data on stream |
|
||||||
|
| `RecvStream.readExact(len)` | Read exact bytes from stream |
|
||||||
|
| `Connection.remoteNodeId()` | Get peer's public key |
|
||||||
|
| `Connection.sendDatagram()` / `readDatagram()` | Unreliable datagrams |
|
||||||
|
|
||||||
|
Not exposed (not critical): `Endpoint.watch_addr()`, `Connection.close_reason()`, `Connection.stats()`.
|
||||||
|
|
||||||
|
## Protocol
|
||||||
|
|
||||||
|
Single bidirectional QUIC stream per connection. Length-prefixed JSON messages.
|
||||||
|
|
||||||
|
### Framing
|
||||||
|
|
||||||
|
QUIC streams are byte streams (no message boundaries). We use 4-byte big-endian length prefix:
|
||||||
|
|
||||||
|
```
|
||||||
|
[4 bytes: length N][N bytes: JSON payload]
|
||||||
|
```
|
||||||
|
|
||||||
|
`RecvStream.readExact(4)` reads the length, then `readExact(N)` reads the payload. This is trivial with iroh's `readExact()` API.
|
||||||
|
|
||||||
|
### Message Format
|
||||||
|
|
||||||
|
Same `type` + `detail` shape as all other transports:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{ "type": "call.requested", "detail": { ... } }
|
||||||
|
```
|
||||||
|
|
||||||
|
Maps directly to `new CustomEvent(type, { detail })`.
|
||||||
|
|
||||||
|
## Two-Sided Design
|
||||||
|
|
||||||
|
Unlike Redis and WebSocket, Iroh has distinct hub and spoke connection patterns:
|
||||||
|
|
||||||
|
### Spoke Side
|
||||||
|
|
||||||
|
```ts
|
||||||
|
const conn = await endpoint.connect(hubNodeId, "alkhub/1");
|
||||||
|
const eventTarget = await createSpokeIrohEventTarget(conn);
|
||||||
|
```
|
||||||
|
|
||||||
|
Spoke opens the bidirectional stream with `openBi()`. The event target wraps the `SendStream` and `RecvStream`.
|
||||||
|
|
||||||
|
### Hub Side
|
||||||
|
|
||||||
|
```ts
|
||||||
|
const conn = await endpoint.accept();
|
||||||
|
const eventTarget = await createHubIrohEventTarget(conn);
|
||||||
|
```
|
||||||
|
|
||||||
|
Hub accepts the connection, then accepts the stream with `acceptBi()`. Same `TypedEventTarget` interface on both sides.
|
||||||
|
|
||||||
|
### Why Two Factories?
|
||||||
|
|
||||||
|
The connection initiator (spoke) calls `openBi()`. The listener (hub) calls `acceptBi()`. Both get `SendStream` + `RecvStream` — the framing and event handling are identical. The split is about connection establishment, not event handling. Could be unified as `createIrohEventTarget(sendStream, recvStream)` with separate helpers for connection, but the two-factory pattern makes the hub/spoke asymmetry explicit.
|
||||||
|
|
||||||
|
## Identity
|
||||||
|
|
||||||
|
`Connection.remoteNodeId()` returns the peer's Ed25519 public key. This is cryptographic identity — no separate API key exchange needed for authentication. The hub can verify that a connection comes from an expected spoke by checking its `NodeId`.
|
||||||
|
|
||||||
|
This is strictly better than WebSocket's token-in-URL or first-message approach. It's also harder to revoke — disabling a spoke requires a denylist of `NodeId`s rather than rotating a token.
|
||||||
|
|
||||||
|
## Connection Startup
|
||||||
|
|
||||||
|
On connection, both sides exchange the operations they expose (same `hub.register` pattern as WebSocket). The `NodeId` serves as identity — no separate API key exchange.
|
||||||
|
|
||||||
|
## Reconnection
|
||||||
|
|
||||||
|
Same pattern as WebSocket — detect connection failure, reconnect, re-register. QUIC handles multipath better than TCP but the application still needs reconnection logic.
|
||||||
|
|
||||||
|
Detection: `RecvStream.readExact()` throws on connection close. The event target should propagate this as an error event or let the caller handle it.
|
||||||
|
|
||||||
|
## Browser Limitations
|
||||||
|
|
||||||
|
Iroh in browsers is relay-only (no UDP hole punching from browser sandbox). This means:
|
||||||
|
- Browser spokes always route through relay servers
|
||||||
|
- WebSocketEventTarget is the right browser transport today (native, no extra deps)
|
||||||
|
- IrohEventTarget for browsers would use the WASM build over relay — future option
|
||||||
|
|
||||||
|
## Multi-Node (Future)
|
||||||
|
|
||||||
|
For 1:N fan-out, `iroh-gossip` is the right tool. No TS binding exists yet. Options:
|
||||||
|
1. Write a minimal Rust NAPI crate wrapping `iroh-gossip::Gossip.subscribe() + broadcast()`
|
||||||
|
2. Contribute gossip to `@rayhanadev/iroh`
|
||||||
|
3. Use hub as a relay point (hub receives once, fans out to each spoke's `IrohEventTarget` individually)
|
||||||
|
|
||||||
|
For now, 1:1 connections are sufficient. The hub fans out to multiple spokes by dispatching to each spoke's `IrohEventTarget` individually — same pattern as WebSocketEventTarget on the hub side.
|
||||||
|
|
||||||
|
## Comparison with WebSocketEventTarget
|
||||||
|
|
||||||
|
| Aspect | WebSocket | Iroh |
|
||||||
|
|--------|-----------|------|
|
||||||
|
| Connection | `new WebSocket(url)` | `endpoint.connect(nodeId, alpn)` |
|
||||||
|
| Accept | Hono WS upgrade | `endpoint.accept()` |
|
||||||
|
| Identity | API key/token | Ed25519 NodeId (cryptographic, mutual) |
|
||||||
|
| NAT traversal | Requires reverse proxy / tunnel | Built-in (relay + hole punching) |
|
||||||
|
| Framing | WS frames (built-in) | QUIC stream (length-prefix needed) |
|
||||||
|
| Hub behind NAT | Not possible without tunneling | Yes |
|
||||||
|
| Browser | Yes (native) | Limited (WASM build, relay-only) |
|
||||||
|
| Native addon | No | Yes (NAPI-RS) |
|
||||||
|
|
||||||
|
## Open Questions
|
||||||
|
|
||||||
|
1. **Binding stability** — `@rayhanadev/iroh` has one author and no tests. If it breaks, we may need to fork or write our own NAPI wrapper. Mitigation: the API surface we use is small (10 methods) and the binding is thin.
|
||||||
|
2. **NAPI under Deno** — NAPI-RS `.node` binaries need testing under Deno 2.x. Since we're building with tsup for npm, the runtime is Node.js.
|
||||||
|
3. **Datagram support** — `sendDatagram`/`readDatagram` could be used for fire-and-forget events (no response expected). Not needed for hub-spoke but could be useful for broadcast. Deferred.
|
||||||
280
docs/research/migration.md
Normal file
280
docs/research/migration.md
Normal file
@@ -0,0 +1,280 @@
|
|||||||
|
# Research: `@alkdev/pubsub` Package Extraction
|
||||||
|
|
||||||
|
## Goal
|
||||||
|
|
||||||
|
Extract `packages/core/pubsub/` into a standalone `@alkdev/pubsub` package, following the same peer-dependency tree-shaking pattern as `@alkdev/typemap`. Each event target adapter (Redis, WebSocket, Iroh) is an isolated module that only imports its own peer dependency. The core `createPubSub + TypedEventTarget + operators` has no peer deps beyond `@repeaterjs/repeater`.
|
||||||
|
|
||||||
|
## Current State
|
||||||
|
|
||||||
|
### Source: `packages/core/pubsub/`
|
||||||
|
|
||||||
|
| File | Lines | Key Exports | Dependencies |
|
||||||
|
|------|-------|-------------|--------------|
|
||||||
|
| `typed_event_target.ts` | 59 | `TypedEvent`, `TypedEventTarget`, `TypedEventListener` etc. | None (pure types) |
|
||||||
|
| `create_pubsub.ts` | 108 | `createPubSub`, `PubSub`, `PubSubConfig`, `PubSubPublishArgsByKey` | `@repeaterjs/repeater` |
|
||||||
|
| `redis_event_target.ts` | 117 | `createRedisEventTarget`, `CreateRedisEventTargetArgs` | `ioredis` (types only), `typed_event_target.ts` |
|
||||||
|
| `operators.ts` | 67 | `filter`, `map`, `pipe` | `@repeaterjs/repeater` |
|
||||||
|
| `mod.ts` | 5 | Re-exports all + `Repeater` | All above |
|
||||||
|
|
||||||
|
**Zero cross-module dependencies.** The pubsub module imports nothing from `operations/`, `mcp/`, `config/`, or `logger/`. It is already self-contained.
|
||||||
|
|
||||||
|
### Test Coverage
|
||||||
|
|
||||||
|
| Test File | Tests | Coverage |
|
||||||
|
|-----------|-------|----------|
|
||||||
|
| `tests/pubsub/redis_event_target.test.ts` | 5 tests | Redis publish path only (mocked ioredis). No subscription-receive path, no real Redis. |
|
||||||
|
| `create_pubsub.ts` | 0 tests | **No tests.** Core pubsub creation, topic scoping, event delivery, Repeater iteration all untested. |
|
||||||
|
| `operators.ts` | 0 tests | **No tests.** `filter`, `map`, `pipe` all untested. |
|
||||||
|
| `typed_event_target.ts` | N/A | Pure type definitions — no runtime to test. |
|
||||||
|
|
||||||
|
### What's Missing (Not Yet Implemented)
|
||||||
|
|
||||||
|
1. **WebSocketEventTarget** — Spec in `spoke-runner.md` (lines 158-204). Implements `TypedEventTarget` over a WebSocket connection. Bidirectional: `dispatchEvent` sends over WS, `addEventListener` receives from WS. Per-connection instance on hub side.
|
||||||
|
2. **IrohEventTarget** — P2P QUIC transport using iroh. Same role as WebSocketEventTarget but with crypto identity (Ed25519 NodeId) and automatic NAT traversal. The `@rayhanadev/iroh` NAPI-RS binding has everything needed — `Endpoint.connect()`/`accept()`, `Connection.openBi()`/`acceptBi()`, `SendStream`/`RecvStream`. No gossip required for hub↔spoke (1:1 bidirectional). See "Iroh Research" below.
|
||||||
|
3. **In-process EventTarget** — Currently `createPubSub` defaults to `new EventTarget()`, which works single-process. No explicit adapter class for this (it's just the default). Could be formalized as `InProcessEventTarget` for clarity, or left as-is since `EventTarget` is a web standard.
|
||||||
|
4. **Redis channel prefixing** — Architecture doc recommends `alk:events:{eventType}` namespacing. Not implemented.
|
||||||
|
5. **Redis reconnection/error handling** — No error handling for connection failures, reconnection, or message parse errors.
|
||||||
|
|
||||||
|
## Proposed Package Structure
|
||||||
|
|
||||||
|
```
|
||||||
|
@alkdev/pubsub/
|
||||||
|
src/
|
||||||
|
index.ts # Barrel: re-exports all public API
|
||||||
|
types.ts # TypedEvent, TypedEventTarget, etc. (from typed_event_target.ts)
|
||||||
|
create_pubsub.ts # createPubSub factory (no changes)
|
||||||
|
operators.ts # filter, map, pipe (no changes)
|
||||||
|
|
||||||
|
# Adapter modules (tree-shakeable, each is its own peer dep island)
|
||||||
|
event-target-in-process.ts # Explicit InProcessEventTarget (or just re-export web EventTarget)
|
||||||
|
event-target-redis.ts # createRedisEventTarget (peer dep: ioredis)
|
||||||
|
event-target-websocket.ts # createWebSocketEventTarget (peer dep: none — WS is a web standard)
|
||||||
|
event-target-iroh.ts # createIrohEventTarget (peer dep: @rayhanadev/iroh)
|
||||||
|
tests/
|
||||||
|
create_pubsub.test.ts # Core pubsub: publish, subscribe, topic scoping, Repeater
|
||||||
|
operators.test.ts # filter, map, pipe
|
||||||
|
event-target-in-process.test.ts
|
||||||
|
event-target-redis.test.ts # Mocked + integration
|
||||||
|
event-target-websocket.test.ts
|
||||||
|
event-target-iroh.test.ts # Mocked or integration
|
||||||
|
package.json
|
||||||
|
tsconfig.json
|
||||||
|
```
|
||||||
|
|
||||||
|
The barrel `index.ts` re-exports everything (like typemap). Tree-shaking works because ESM re-exports are statically analyzable. Users who want minimal bundles import specific adapter files directly (e.g., `import { createRedisEventTarget } from '@alkdev/pubsub/event-target-redis'`).
|
||||||
|
|
||||||
|
Alternatively, if we want sub-path exports (which typemap doesn't use but many packages do), we could add them to `package.json` exports:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"exports": {
|
||||||
|
".": { "import": "./dist/index.mjs", "types": "./dist/index.d.mts" },
|
||||||
|
"./event-target-redis": { "import": "./dist/event-target-redis.mjs", "types": "./dist/event-target-redis.d.mts" },
|
||||||
|
"./event-target-websocket": { "import": "./dist/event-target-websocket.mjs", "types": "./dist/event-target-websocket.d.mts" },
|
||||||
|
"./event-target-iroh": { "import": "./dist/event-target-iroh.mjs", "types": "./dist/event-target-iroh.d.mts" }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Sub-path exports are more explicit and don't rely on bundler tree-shaking, but add maintenance burden. We should pick one approach and use it consistently across `@alkdev` packages.
|
||||||
|
|
||||||
|
## Dependencies
|
||||||
|
|
||||||
|
| Dependency | Type | Notes |
|
||||||
|
|-----------|------|-------|
|
||||||
|
| `@repeaterjs/repeater` | direct | Small (~3KB), stable. Core async iterable primitive for `subscribe()`. |
|
||||||
|
| `ioredis` | peer | Only imported by `event-target-redis.ts`. Type-only import at compile time. Consumers who don't need Redis skip it. |
|
||||||
|
| `@rayhanadev/iroh` | peer | Only imported by `event-target-iroh.ts`. NAPI-RS native addon (~15-20MB). Consumers who don't need P2P QUIC skip it. |
|
||||||
|
|
||||||
|
No other external dependencies. No logger dependency.
|
||||||
|
|
||||||
|
## Build & Publish
|
||||||
|
|
||||||
|
Following `@alkdev/taskgraph` precedent:
|
||||||
|
|
||||||
|
- **Build tool**: `tsup` — produces dual ESM + CJS with types automatically
|
||||||
|
- **Target**: `es2022`
|
||||||
|
- **Publish target**: npm (`@alkdev/pubsub`)
|
||||||
|
- **Deno compatibility**: Source is standard TypeScript with no Deno-specific APIs (all web standard). Deno can import from npm or JSR.
|
||||||
|
- **Testing**: `vitest` (matching taskgraph) or `deno test` (matching current alkhub_ts). Decision needed.
|
||||||
|
|
||||||
|
### Build Config Sketch
|
||||||
|
|
||||||
|
```ts
|
||||||
|
// tsup.config.ts
|
||||||
|
import { defineConfig } from 'tsup';
|
||||||
|
|
||||||
|
export default defineConfig({
|
||||||
|
entry: [
|
||||||
|
'src/index.ts',
|
||||||
|
'src/event-target-redis.ts',
|
||||||
|
'src/event-target-websocket.ts',
|
||||||
|
'src/event-target-iroh.ts',
|
||||||
|
],
|
||||||
|
format: ['esm', 'cjs'],
|
||||||
|
dts: true,
|
||||||
|
splitting: true,
|
||||||
|
clean: true,
|
||||||
|
target: 'es2022',
|
||||||
|
});
|
||||||
|
```
|
||||||
|
|
||||||
|
## Iroh Research Summary
|
||||||
|
|
||||||
|
### What Is Iroh?
|
||||||
|
|
||||||
|
Iroh is a Rust P2P QUIC library by n0.computer. Peers connect by **public key** (Ed25519), not IP address. Key capabilities:
|
||||||
|
|
||||||
|
- **NAT traversal**: Automatic UDP hole punching (~90% success rate), QUIC Address Discovery
|
||||||
|
- **Relay fallback**: If direct connection fails, routes through stateless relay servers (end-to-end encrypted)
|
||||||
|
- **Public key addressing**: Peers identified by `NodeId`, no DNS or IP needed
|
||||||
|
- **QUIC transport**: Multiplexed streams, built-in encryption, 0-RTT
|
||||||
|
- **Gossip protocol** (`iroh-gossip`): Epidemic broadcast trees for topic-based pub/sub (not needed for hub↔spoke — that's 1:1, not 1:N)
|
||||||
|
|
||||||
|
### Why It Matters for alkhub
|
||||||
|
|
||||||
|
WebSocket transport requires the hub to have a publicly reachable address. Spokes behind NAT can't be reached by the hub for push operations. Iroh solves:
|
||||||
|
|
||||||
|
1. **Hub behind NAT** — No public IP needed. Spokes dial the hub by its `NodeId` through relay servers.
|
||||||
|
2. **Spoke push** — Hub can initiate connections to spokes by `NodeId` (impossible with WS without polling).
|
||||||
|
3. **P2P spoke↔spoke** — Direct spoke-to-spoke communication without routing through hub.
|
||||||
|
4. **Cryptographic identity** — Ed25519 `NodeId` doubles as spoke authentication — strictly better than API keys for identification.
|
||||||
|
|
||||||
|
### Current TS Binding — `@rayhanadev/iroh`
|
||||||
|
|
||||||
|
NAPI-RS binding (v0.1.1) at `/workspace/iroh-ts`. **The binding has everything needed to build IrohEventTarget.** No gossip required — hub↔spoke is 1:1 bidirectional JSON event channels over QUIC streams.
|
||||||
|
|
||||||
|
**Core API that we use:**
|
||||||
|
|
||||||
|
| Method | Purpose |
|
||||||
|
|--------|---------|
|
||||||
|
| `Endpoint.create()` / `createWithOptions({ alpns })` | Create QUIC endpoint |
|
||||||
|
| `Endpoint.connect(nodeId, alpn)` | Connect to a peer by public key |
|
||||||
|
| `Endpoint.accept()` | Accept incoming connection |
|
||||||
|
| `Endpoint.nodeId()` | Get our public key identity |
|
||||||
|
| `Connection.openBi()` | Open bidirectional stream (spoke side) |
|
||||||
|
| `Connection.acceptBi()` | Accept bidirectional stream (hub side) |
|
||||||
|
| `SendStream.writeAll(data)` | Send data on stream |
|
||||||
|
| `RecvStream.readExact(len)` | Read exact bytes from stream |
|
||||||
|
| `Connection.remoteNodeId()` | Get peer's public key |
|
||||||
|
| `Connection.sendDatagram(data)` / `readDatagram()` | Unreliable datagrams (fire-and-forget events) |
|
||||||
|
|
||||||
|
**Not exposed (but not critical):**
|
||||||
|
- `Endpoint.watch_addr()` — detect network changes (workaround: detect connection failure)
|
||||||
|
- `Connection.close_reason()` — synchronous close check (workaround: await `closed()`)
|
||||||
|
- `Connection.stats()` — observability (nice to have, not required)
|
||||||
|
|
||||||
|
### IrohEventTarget Design
|
||||||
|
|
||||||
|
Same `TypedEventTarget` interface as `WebSocketEventTarget` and `RedisEventTarget`. Hub and spoke each create one per connection.
|
||||||
|
|
||||||
|
**Protocol**: Single bidirectional QUIC stream per connection, length-prefixed JSON messages. Spoke opens the stream with `openBi()`, hub accepts with `acceptBi()`. Same `type` + `detail` event shape as all other transports.
|
||||||
|
|
||||||
|
```ts
|
||||||
|
// Spoke side
|
||||||
|
const conn = await endpoint.connect(hubNodeId, "alkhub/1");
|
||||||
|
const eventTarget = await createSpokeIrohEventTarget(conn);
|
||||||
|
|
||||||
|
// Hub side
|
||||||
|
const conn = await endpoint.accept();
|
||||||
|
const eventTarget = await createHubIrohEventTarget(conn);
|
||||||
|
|
||||||
|
// Both sides — same TypedEventTarget interface
|
||||||
|
eventTarget.addEventListener("call.responded", (event) => { ... });
|
||||||
|
eventTarget.dispatchEvent(new CustomEvent("call.requested", { detail: { ... } }));
|
||||||
|
```
|
||||||
|
|
||||||
|
**Framing**: 4-byte big-endian length prefix + JSON payload. Necessary because QUIC streams are byte streams, not message streams. `readExact()` makes this trivial.
|
||||||
|
|
||||||
|
**Connection startup**: On connection, both sides exchange the operations they expose (same hub.register pattern as WebSocket). The `NodeId` serves as cryptographic identity — no separate API key exchange needed for authentication.
|
||||||
|
|
||||||
|
**Reconnection**: Same pattern as WebSocket — detect connection failure, reconnect, re-register. QUIC handles multipath better than TCP but the application still needs reconnection logic.
|
||||||
|
|
||||||
|
**Comparison with WebSocketEventTarget:**
|
||||||
|
|
||||||
|
| Aspect | WebSocketEventTarget | IrohEventTarget |
|
||||||
|
|--------|---------------------|-----------------|
|
||||||
|
| Connection | `new WebSocket(url)` | `endpoint.connect(nodeId, alpn)` |
|
||||||
|
| Accept | Hono WS upgrade | `endpoint.accept()` |
|
||||||
|
| Identity | API key/token in URL or first message | Ed25519 NodeId (cryptographic, mutual) |
|
||||||
|
| NAT traversal | Requires reverse proxy / CDN / tunnel | Built-in (relay + hole punching) |
|
||||||
|
| Framing | WS frames (built-in message boundary) | QUIC stream (needs length-prefix framing) |
|
||||||
|
| Hub behind NAT | Not possible without tunneling | Yes — spoke dials by NodeId |
|
||||||
|
| Browser | Yes (native WS) | Limited (WASM build, relay-only — use WS for browsers) |
|
||||||
|
|
||||||
|
### Multi-Node Scenarios (Future)
|
||||||
|
|
||||||
|
For 1:N fan-out (e.g., one event to 50 spokes), `iroh-gossip` is the right tool. No TS binding exposes it yet. Options when we need it:
|
||||||
|
1. Write a minimal Rust NAPI crate wrapping `iroh-gossip::Gossip.subscribe() + broadcast()` (~500 lines Rust)
|
||||||
|
2. Contribute gossip to `@rayhanadev/iroh` or `@salvatoret/iroh`
|
||||||
|
3. Use hub as a relay point (hub receives once, fans out to each spoke's `IrohEventTarget` individually)
|
||||||
|
|
||||||
|
For now, 1:1 connections are sufficient. The hub can fan out to multiple spokes by dispatching to each spoke's `IrohEventTarget` individually — same pattern as WebSocketEventTarget on the hub side.
|
||||||
|
|
||||||
|
### Browser Considerations
|
||||||
|
|
||||||
|
Iroh in browsers is relay-only (no UDP hole punching from browser sandbox). This means:
|
||||||
|
- Browser spokes always route through relay servers
|
||||||
|
- WebSocketEventTarget is the right browser transport today (native, no extra deps)
|
||||||
|
- IrohEventTarget for browsers would use the WASM build over relay — future option
|
||||||
|
|
||||||
|
## Migration Steps
|
||||||
|
|
||||||
|
### Phase 1: Extract to standalone package
|
||||||
|
|
||||||
|
1. **Create `@alkdev/pubsub` repo** (or directory in a monorepo)
|
||||||
|
2. **Copy source files** from `packages/core/pubsub/` with no modifications to core logic:
|
||||||
|
- `typed_event_target.ts` → `types.ts`
|
||||||
|
- `create_pubsub.ts` → `create_pubsub.ts`
|
||||||
|
- `redis_event_target.ts` → `event-target-redis.ts`
|
||||||
|
- `operators.ts` → `operators.ts`
|
||||||
|
3. **Set up build pipeline** (tsup, package.json, tsconfig)
|
||||||
|
4. **Move Redis to peer dependency** in `package.json`
|
||||||
|
5. **Write missing tests**: `create_pubsub.test.ts`, `operators.test.ts`
|
||||||
|
6. **Add Redis subscription-receive and unsubscribe cleanup tests**
|
||||||
|
7. **Publish v0.1.0 to npm**
|
||||||
|
|
||||||
|
### Phase 2: Add adapters and improve coverage
|
||||||
|
|
||||||
|
8. **Implement `WebSocketEventTarget`** per `spoke-runner.md` spec
|
||||||
|
9. **Implement `IrohEventTarget`** — `createHubIrohEventTarget` / `createSpokeIrohEventTarget` with length-prefixed JSON framing over QUIC streams
|
||||||
|
10. **Add Redis channel prefixing** (`alk:events:*` or configurable prefix)
|
||||||
|
11. **Add Redis error handling** (connection errors, reconnection, parse errors)
|
||||||
|
12. **Formalize `InProcessEventTarget`** (explicit or just document that `EventTarget` is the default)
|
||||||
|
13. **Write adapter tests** (mock WS bidirectional flow, mock iroh connect/accept/stream)
|
||||||
|
|
||||||
|
### Phase 3: Production hardening
|
||||||
|
|
||||||
|
14. **Redis integration tests** with real Redis instance
|
||||||
|
15. **WebSocket integration tests** with real WS server/client
|
||||||
|
16. **Iroh integration tests** — requires relay server or direct P2P between two endpoints
|
||||||
|
17. **Reconnection logic** for both WebSocket and Iroh adapters
|
||||||
|
18. **Error propagation** — connection failures should propagate to listeners gracefully
|
||||||
|
|
||||||
|
### Phase 4: Integration back into alkhub_ts
|
||||||
|
|
||||||
|
19. **Replace** `packages/core/pubsub/` with `@alkdev/pubsub` npm/JSR dependency
|
||||||
|
20. **Update** `packages/core/deno.json` and `packages/core/mod.ts` to import from `@alkdev/pubsub`
|
||||||
|
21. **Remove** `ioredis` from `packages/core/deno.json` (it moves to `@alkdev/pubsub`'s peer deps, and hub uses it directly)
|
||||||
|
22. **Update call protocol, hub, and spoke** to use `@alkdev/pubsub` directly
|
||||||
|
|
||||||
|
## Open Questions
|
||||||
|
|
||||||
|
1. **Sub-path exports vs. barrel + tree-shaking?** Typemap uses barrel-only + tree-shaking. Taskgraph uses barrel-only. Do we want sub-path exports for explicit adapter imports, or rely on tree-shaking?
|
||||||
|
2. **Test runner**: `vitest` (matches taskgraph) or `deno test` (matches current alkhub_ts)? If the package publishes to npm via tsup, `vitest` is the natural choice. If we also want to test in Deno, we could support both.
|
||||||
|
3. **Deno-first or Node-first development?** Current code has no Deno-specific APIs (it's all web standard). We could develop in either. Deno can import from npm. Node can't import from JSR without the JSR npm mirror. If we're using tsup for build, we're effectively Node-first for publishing, Deno-compatible for source.
|
||||||
|
4. **When to implement `WebSocketEventTarget` and `IrohEventTarget`?** Before or after extracting the package? The specs and interfaces are clear. Could implement both as part of the initial adapter set, since both follow the same `TypedEventTarget` pattern.
|
||||||
|
5. **Iroh binding**: Should we use `@rayhanadev/iroh` directly (v0.1.1, community binding, 9 commits, no tests) or write/publish our own `@alkdev/iroh` NAPI wrapper? The current binding works but has no tests and one author. Forking/forking-and-maintaining gives us control of the build pipeline.
|
||||||
|
6. **Iroh + Deno**: NAPI-RS `.node` binaries may need testing under Deno 2.x. If we're building with tsup for npm publish, the runtime is Node.js. For Deno-first development, we need to verify NAPI addons work.
|
||||||
|
7. **Redis channel prefixing**: Should the prefix be configurable per `createRedisEventTarget({ prefix })?` or hardcoded to `alk:events:`? Configurable is more flexible for multi-tenant scenarios.
|
||||||
|
|
||||||
|
### Architecture Decision: WebSocket vs Iroh as Primary Transport
|
||||||
|
|
||||||
|
WebSocket is the right default for most deployments — it's native in browsers and Deno, well-supported, and requires no native addons. Iroh is the right choice when:
|
||||||
|
|
||||||
|
- The hub is behind NAT (dev laptops, home servers, no CDN)
|
||||||
|
- Spokes need to be reachable by the hub (push notifications to client spokes)
|
||||||
|
- Cryptographic identity is preferred over token-based auth
|
||||||
|
- P2P spoke-to-spoke communication is needed
|
||||||
|
|
||||||
|
A deployment can use both: `WebSocketEventTarget` for browser clients, `IrohEventTarget` for native spokes. Same `TypedEventTarget` interface, same call protocol, same `PendingRequestMap`.
|
||||||
3027
package-lock.json
generated
Normal file
3027
package-lock.json
generated
Normal file
File diff suppressed because it is too large
Load Diff
77
package.json
Normal file
77
package.json
Normal file
@@ -0,0 +1,77 @@
|
|||||||
|
{
|
||||||
|
"name": "@alkdev/pubsub",
|
||||||
|
"version": "0.1.0",
|
||||||
|
"description": "Type-safe publish/subscribe with pluggable event target adapters (in-process, Redis, WebSocket, Iroh)",
|
||||||
|
"type": "module",
|
||||||
|
"main": "./dist/index.cjs",
|
||||||
|
"module": "./dist/index.js",
|
||||||
|
"types": "./dist/index.d.ts",
|
||||||
|
"exports": {
|
||||||
|
".": {
|
||||||
|
"import": {
|
||||||
|
"types": "./dist/index.d.ts",
|
||||||
|
"default": "./dist/index.js"
|
||||||
|
},
|
||||||
|
"require": {
|
||||||
|
"types": "./dist/index.d.cts",
|
||||||
|
"default": "./dist/index.cjs"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"./event-target-redis": {
|
||||||
|
"import": {
|
||||||
|
"types": "./dist/event-target-redis.d.ts",
|
||||||
|
"default": "./dist/event-target-redis.js"
|
||||||
|
},
|
||||||
|
"require": {
|
||||||
|
"types": "./dist/event-target-redis.d.cts",
|
||||||
|
"default": "./dist/event-target-redis.cjs"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"publishConfig": {
|
||||||
|
"access": "public"
|
||||||
|
},
|
||||||
|
"files": [
|
||||||
|
"dist"
|
||||||
|
],
|
||||||
|
"scripts": {
|
||||||
|
"build": "tsup",
|
||||||
|
"build:tsc": "tsc --noEmit",
|
||||||
|
"test": "vitest run",
|
||||||
|
"test:watch": "vitest",
|
||||||
|
"test:coverage": "vitest run --coverage",
|
||||||
|
"lint": "tsc --noEmit",
|
||||||
|
"prepublishOnly": "npm run build"
|
||||||
|
},
|
||||||
|
"keywords": [
|
||||||
|
"pubsub",
|
||||||
|
"typed-event-target",
|
||||||
|
"redis",
|
||||||
|
"websocket",
|
||||||
|
"iroh",
|
||||||
|
"quic"
|
||||||
|
],
|
||||||
|
"license": "MIT OR Apache-2.0",
|
||||||
|
"dependencies": {
|
||||||
|
"@repeaterjs/repeater": "^3.0.0"
|
||||||
|
},
|
||||||
|
"peerDependencies": {
|
||||||
|
"ioredis": "^5.0.0"
|
||||||
|
},
|
||||||
|
"peerDependenciesMeta": {
|
||||||
|
"ioredis": {
|
||||||
|
"optional": true
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"devDependencies": {
|
||||||
|
"@types/node": "^22.0.0",
|
||||||
|
"@vitest/coverage-v8": "^3.2.4",
|
||||||
|
"ioredis": "^5.10.1",
|
||||||
|
"tsup": "^8.5.1",
|
||||||
|
"typescript": "^5.7.0",
|
||||||
|
"vitest": "^3.1.0"
|
||||||
|
},
|
||||||
|
"engines": {
|
||||||
|
"node": ">=18.0.0"
|
||||||
|
}
|
||||||
|
}
|
||||||
108
src/create_pubsub.ts
Normal file
108
src/create_pubsub.ts
Normal file
@@ -0,0 +1,108 @@
|
|||||||
|
/*
|
||||||
|
* Adapted from @graphql-yoga/subscription
|
||||||
|
* Original source: https://github.com/graphql-hive/graphql-yoga
|
||||||
|
* License: MIT
|
||||||
|
*
|
||||||
|
* Copyright (c) 2024 The Guild, GraphQL Yoga Contributors
|
||||||
|
*
|
||||||
|
* Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||||
|
* of this software and associated documentation files (the "Software"), to deal
|
||||||
|
* in the Software without restriction, including without limitation the rights
|
||||||
|
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||||
|
* copies of the Software, and to permit persons to whom the Software is
|
||||||
|
* furnished to do so, subject to the following conditions:
|
||||||
|
*
|
||||||
|
* The above copyright notice and this permission notice shall be included in all
|
||||||
|
* copies or substantial portions of the Software.
|
||||||
|
*
|
||||||
|
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||||
|
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||||
|
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||||
|
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||||
|
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||||
|
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||||
|
* SOFTWARE.
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { Repeater } from "@repeaterjs/repeater";
|
||||||
|
import type { TypedEventTarget, TypedEvent } from "./types.js";
|
||||||
|
|
||||||
|
export type PubSubPublishArgsByKey = {
|
||||||
|
[key: string]: [] | [unknown] | [number | string, unknown];
|
||||||
|
};
|
||||||
|
|
||||||
|
export type PubSubEvent<
|
||||||
|
TPubSubPublishArgsByKey extends PubSubPublishArgsByKey,
|
||||||
|
TKey extends Extract<keyof TPubSubPublishArgsByKey, string>,
|
||||||
|
> = TypedEvent<
|
||||||
|
TKey,
|
||||||
|
TPubSubPublishArgsByKey[TKey][1] extends undefined
|
||||||
|
? TPubSubPublishArgsByKey[TKey][0]
|
||||||
|
: TPubSubPublishArgsByKey[TKey][1]
|
||||||
|
>;
|
||||||
|
|
||||||
|
export type PubSubEventTarget<TPubSubPublishArgsByKey extends PubSubPublishArgsByKey> =
|
||||||
|
TypedEventTarget<
|
||||||
|
PubSubEvent<TPubSubPublishArgsByKey, Extract<keyof TPubSubPublishArgsByKey, string>>
|
||||||
|
>;
|
||||||
|
|
||||||
|
export type PubSubConfig<TPubSubPublishArgsByKey extends PubSubPublishArgsByKey> = {
|
||||||
|
eventTarget?: PubSubEventTarget<TPubSubPublishArgsByKey>;
|
||||||
|
};
|
||||||
|
|
||||||
|
export type PubSub<TPubSubPublishArgsByKey extends PubSubPublishArgsByKey> = {
|
||||||
|
publish<TKey extends Extract<keyof TPubSubPublishArgsByKey, string>>(
|
||||||
|
routingKey: TKey,
|
||||||
|
...args: TPubSubPublishArgsByKey[TKey]
|
||||||
|
): void;
|
||||||
|
subscribe<TKey extends Extract<keyof TPubSubPublishArgsByKey, string>>(
|
||||||
|
...[routingKey, id]: TPubSubPublishArgsByKey[TKey][1] extends undefined
|
||||||
|
? [TKey]
|
||||||
|
: [TKey, TPubSubPublishArgsByKey[TKey][0]]
|
||||||
|
): Repeater<unknown>;
|
||||||
|
};
|
||||||
|
|
||||||
|
export function createPubSub<TPubSubPublishArgsByKey extends PubSubPublishArgsByKey>(
|
||||||
|
config?: PubSubConfig<TPubSubPublishArgsByKey>,
|
||||||
|
): PubSub<TPubSubPublishArgsByKey> {
|
||||||
|
const target =
|
||||||
|
config?.eventTarget ?? (new EventTarget() as PubSubEventTarget<TPubSubPublishArgsByKey>);
|
||||||
|
|
||||||
|
return {
|
||||||
|
publish<TKey extends Extract<keyof TPubSubPublishArgsByKey, string>>(
|
||||||
|
routingKey: TKey,
|
||||||
|
...args: TPubSubPublishArgsByKey[TKey]
|
||||||
|
) {
|
||||||
|
const payload = args[1] ?? args[0] ?? null;
|
||||||
|
const topic = args[1] === undefined ? routingKey : `${routingKey}:${args[0] as number}`;
|
||||||
|
|
||||||
|
const event = new CustomEvent(topic, { detail: payload }) as PubSubEvent<
|
||||||
|
TPubSubPublishArgsByKey,
|
||||||
|
TKey
|
||||||
|
>;
|
||||||
|
target.dispatchEvent(event);
|
||||||
|
},
|
||||||
|
subscribe<TKey extends Extract<keyof TPubSubPublishArgsByKey, string>>(
|
||||||
|
...[routingKey, id]: TPubSubPublishArgsByKey[TKey][1] extends undefined
|
||||||
|
? [TKey]
|
||||||
|
: [TKey, TPubSubPublishArgsByKey[TKey][0]]
|
||||||
|
): Repeater<unknown> {
|
||||||
|
const topic: TKey = (id === undefined ? routingKey : `${routingKey}:${id as number}`) as TKey;
|
||||||
|
|
||||||
|
return new Repeater(function subscriptionRepeater(
|
||||||
|
next: (value: unknown) => Promise<void>,
|
||||||
|
stop: Promise<void>,
|
||||||
|
) {
|
||||||
|
function pubsubEventListener(event: CustomEvent) {
|
||||||
|
next(event.detail);
|
||||||
|
}
|
||||||
|
|
||||||
|
stop.then(function subscriptionRepeaterStopHandler() {
|
||||||
|
target.removeEventListener(topic, pubsubEventListener as EventListener);
|
||||||
|
});
|
||||||
|
|
||||||
|
target.addEventListener(topic, pubsubEventListener as EventListener, undefined);
|
||||||
|
});
|
||||||
|
},
|
||||||
|
};
|
||||||
|
}
|
||||||
117
src/event-target-redis.ts
Normal file
117
src/event-target-redis.ts
Normal file
@@ -0,0 +1,117 @@
|
|||||||
|
/*
|
||||||
|
* Adapted from @graphql-yoga/redis-event-target
|
||||||
|
* Original source: https://github.com/graphql-hive/graphql-yoga
|
||||||
|
* License: MIT
|
||||||
|
*
|
||||||
|
* Copyright (c) 2024 The Guild, GraphQL Yoga Contributors
|
||||||
|
*
|
||||||
|
* Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||||
|
* of this software and associated documentation files (the "Software"), to deal
|
||||||
|
* in the Software without restriction, including without limitation the rights
|
||||||
|
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||||
|
* copies of the Software, and to permit persons to whom the Software is
|
||||||
|
* furnished to do so, subject to the following conditions:
|
||||||
|
*
|
||||||
|
* The above copyright notice and this permission notice shall be included in all
|
||||||
|
* copies or substantial portions of the Software.
|
||||||
|
*
|
||||||
|
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||||
|
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||||
|
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||||
|
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||||
|
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||||
|
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||||
|
* SOFTWARE.
|
||||||
|
*
|
||||||
|
* Changes from original:
|
||||||
|
* - Uses native CustomEvent instead of @whatwg-node/events ponyfill (Deno has CustomEvent)
|
||||||
|
* - Uses our TypedEventTarget/TypedEvent types from types.ts
|
||||||
|
* - Removed tslib dependency
|
||||||
|
* - Uses ioredis types directly (already a dependency)
|
||||||
|
*/
|
||||||
|
|
||||||
|
import type { Cluster, Redis } from "ioredis";
|
||||||
|
import type { TypedEventTarget, TypedEvent } from "./types.js";
|
||||||
|
|
||||||
|
export type CreateRedisEventTargetArgs = {
|
||||||
|
publishClient: Redis | Cluster;
|
||||||
|
subscribeClient: Redis | Cluster;
|
||||||
|
serializer?: {
|
||||||
|
stringify: (message: unknown) => string;
|
||||||
|
parse: (message: string) => unknown;
|
||||||
|
};
|
||||||
|
};
|
||||||
|
|
||||||
|
export function createRedisEventTarget<TEvent extends TypedEvent>(
|
||||||
|
args: CreateRedisEventTargetArgs,
|
||||||
|
): TypedEventTarget<TEvent> {
|
||||||
|
const { publishClient, subscribeClient } = args;
|
||||||
|
|
||||||
|
const serializer = args.serializer ?? JSON;
|
||||||
|
|
||||||
|
const callbacksForTopic = new Map<string, Set<EventListener>>();
|
||||||
|
|
||||||
|
function onMessage(channel: string, message: string) {
|
||||||
|
const callbacks = callbacksForTopic.get(channel);
|
||||||
|
if (callbacks === undefined) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
const event = new CustomEvent(channel, {
|
||||||
|
detail: message === "" ? null : serializer.parse(message),
|
||||||
|
}) as TEvent;
|
||||||
|
for (const callback of callbacks) {
|
||||||
|
callback(event);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
(subscribeClient as Redis).on("message", onMessage);
|
||||||
|
|
||||||
|
function addCallback(topic: string, callback: EventListener) {
|
||||||
|
let callbacks = callbacksForTopic.get(topic);
|
||||||
|
if (callbacks === undefined) {
|
||||||
|
callbacks = new Set();
|
||||||
|
callbacksForTopic.set(topic, callbacks);
|
||||||
|
|
||||||
|
subscribeClient.subscribe(topic);
|
||||||
|
}
|
||||||
|
callbacks.add(callback);
|
||||||
|
}
|
||||||
|
|
||||||
|
function removeCallback(topic: string, callback: EventListener) {
|
||||||
|
const callbacks = callbacksForTopic.get(topic);
|
||||||
|
if (callbacks === undefined) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
callbacks.delete(callback);
|
||||||
|
if (callbacks.size > 0) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
callbacksForTopic.delete(topic);
|
||||||
|
subscribeClient.unsubscribe(topic);
|
||||||
|
}
|
||||||
|
|
||||||
|
return {
|
||||||
|
addEventListener(topic, callbackOrOptions: EventListenerOrEventListenerObject) {
|
||||||
|
if (callbackOrOptions != null) {
|
||||||
|
const callback =
|
||||||
|
"handleEvent" in callbackOrOptions ? callbackOrOptions.handleEvent : callbackOrOptions;
|
||||||
|
addCallback(topic, callback);
|
||||||
|
}
|
||||||
|
},
|
||||||
|
dispatchEvent(event: TEvent) {
|
||||||
|
publishClient.publish(
|
||||||
|
event.type,
|
||||||
|
event.detail === undefined ? "" : serializer.stringify(event.detail),
|
||||||
|
);
|
||||||
|
return true;
|
||||||
|
},
|
||||||
|
removeEventListener(topic, callbackOrOptions: EventListenerOrEventListenerObject) {
|
||||||
|
if (callbackOrOptions != null) {
|
||||||
|
const callback =
|
||||||
|
"handleEvent" in callbackOrOptions ? callbackOrOptions.handleEvent : callbackOrOptions;
|
||||||
|
removeCallback(topic, callback);
|
||||||
|
}
|
||||||
|
},
|
||||||
|
};
|
||||||
|
}
|
||||||
4
src/index.ts
Normal file
4
src/index.ts
Normal file
@@ -0,0 +1,4 @@
|
|||||||
|
export { createPubSub, type PubSub, type PubSubConfig, type PubSubEvent, type PubSubEventTarget, type PubSubPublishArgsByKey } from "./create_pubsub.js";
|
||||||
|
export { type TypedEvent, type TypedEventTarget, type TypedEventListener, type TypedEventListenerObject, type TypedEventListenerOrEventListenerObject } from "./types.js";
|
||||||
|
export { filter, map, pipe } from "./operators.js";
|
||||||
|
export { Repeater } from "@repeaterjs/repeater";
|
||||||
67
src/operators.ts
Normal file
67
src/operators.ts
Normal file
@@ -0,0 +1,67 @@
|
|||||||
|
/*
|
||||||
|
* Adapted from @graphql-yoga/subscription
|
||||||
|
* Original source: https://github.com/graphql-hive/graphql-yoga
|
||||||
|
* License: MIT
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { Repeater, type Stop, type Push } from "@repeaterjs/repeater";
|
||||||
|
|
||||||
|
export function filter<T, U extends T>(
|
||||||
|
filterFn: (input: T) => input is U,
|
||||||
|
): (source: AsyncIterable<T>) => Repeater<U, void, unknown>;
|
||||||
|
export function filter<T>(
|
||||||
|
filterFn: (input: T) => Promise<boolean> | boolean,
|
||||||
|
): (source: AsyncIterable<T>) => Repeater<T, void, unknown>;
|
||||||
|
export function filter(filterFn: (value: unknown) => Promise<boolean> | boolean) {
|
||||||
|
return (source: AsyncIterable<unknown>) =>
|
||||||
|
new Repeater(async (push: Push<unknown>, stop: Stop) => {
|
||||||
|
const iterable = source[Symbol.asyncIterator]();
|
||||||
|
stop.then(() => {
|
||||||
|
iterable.return?.();
|
||||||
|
});
|
||||||
|
|
||||||
|
let latest: IteratorResult<unknown>;
|
||||||
|
while ((latest = await iterable.next()).done === false) {
|
||||||
|
if (await filterFn(latest.value)) {
|
||||||
|
await push(latest.value);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
stop();
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
export function map<T, O>(
|
||||||
|
mapper: (input: T) => Promise<O> | O,
|
||||||
|
): (source: AsyncIterable<T>) => Repeater<O, void, unknown> {
|
||||||
|
return (source: AsyncIterable<T>) =>
|
||||||
|
new Repeater(async (push: Push<O>, stop: Stop) => {
|
||||||
|
const iterable = source[Symbol.asyncIterator]();
|
||||||
|
stop.then(() => {
|
||||||
|
iterable.return?.();
|
||||||
|
});
|
||||||
|
|
||||||
|
let latest: IteratorResult<T>;
|
||||||
|
while ((latest = await iterable.next()).done === false) {
|
||||||
|
await push(await mapper(latest.value));
|
||||||
|
}
|
||||||
|
stop();
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
export function pipe<A>(a: A): A;
|
||||||
|
export function pipe<A, B>(a: A, ab: (a: A) => B): B;
|
||||||
|
export function pipe<A, B, C>(a: A, ab: (a: A) => B, bc: (b: B) => C): C;
|
||||||
|
export function pipe<A, B, C, D>(a: A, ab: (a: A) => B, bc: (b: B) => C, cd: (c: C) => D): D;
|
||||||
|
export function pipe<A, B, C, D, E>(
|
||||||
|
a: A,
|
||||||
|
ab: (a: A) => B,
|
||||||
|
bc: (b: B) => C,
|
||||||
|
cd: (c: C) => D,
|
||||||
|
de: (d: D) => E,
|
||||||
|
): E;
|
||||||
|
export function pipe(
|
||||||
|
a: unknown,
|
||||||
|
...fns: ((arg: unknown) => unknown)[]
|
||||||
|
): unknown {
|
||||||
|
return fns.reduce((acc, fn) => fn(acc), a);
|
||||||
|
}
|
||||||
59
src/types.ts
Normal file
59
src/types.ts
Normal file
@@ -0,0 +1,59 @@
|
|||||||
|
/*
|
||||||
|
* Adapted from @graphql-yoga/typed-event-target
|
||||||
|
* Original source: https://github.com/graphql-hive/graphql-yoga
|
||||||
|
* License: MIT
|
||||||
|
*
|
||||||
|
* Copyright (c) 2024 The Guild, GraphQL Yoga Contributors
|
||||||
|
*
|
||||||
|
* Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||||
|
* of this software and associated documentation files (the "Software"), to deal
|
||||||
|
* in the Software without restriction, including without limitation the rights
|
||||||
|
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||||
|
* copies of the Software, and to permit persons to whom the Software is
|
||||||
|
* furnished to do so, subject to the following conditions:
|
||||||
|
*
|
||||||
|
* The above copyright notice and this permission notice shall be included in all
|
||||||
|
* copies or substantial portions of the Software.
|
||||||
|
*
|
||||||
|
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||||
|
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||||
|
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||||
|
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||||
|
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||||
|
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||||
|
* SOFTWARE.
|
||||||
|
*/
|
||||||
|
|
||||||
|
export type TypedEvent<TType extends string = string, TDetail = unknown> = Omit<
|
||||||
|
CustomEvent<TDetail>,
|
||||||
|
"detail" | "type"
|
||||||
|
> & {
|
||||||
|
type: TType;
|
||||||
|
detail: TDetail;
|
||||||
|
};
|
||||||
|
|
||||||
|
export interface TypedEventListener<TEvent extends TypedEvent> {
|
||||||
|
(evt: TEvent): void;
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface TypedEventListenerObject<TEvent extends TypedEvent> {
|
||||||
|
handleEvent(object: TEvent): void;
|
||||||
|
}
|
||||||
|
|
||||||
|
export type TypedEventListenerOrEventListenerObject<TEvent extends TypedEvent> =
|
||||||
|
| TypedEventListener<TEvent>
|
||||||
|
| TypedEventListenerObject<TEvent>;
|
||||||
|
|
||||||
|
export interface TypedEventTarget<TEvent extends TypedEvent> extends EventTarget {
|
||||||
|
addEventListener<TCurrEvent extends TEvent>(
|
||||||
|
type: TCurrEvent["type"],
|
||||||
|
callback: TypedEventListenerOrEventListenerObject<TCurrEvent> | null,
|
||||||
|
options?: AddEventListenerOptions | boolean,
|
||||||
|
): void;
|
||||||
|
dispatchEvent(event: TEvent): boolean;
|
||||||
|
removeEventListener<TCurrEvent extends TEvent>(
|
||||||
|
type: TCurrEvent["type"],
|
||||||
|
callback: TypedEventListenerOrEventListenerObject<TCurrEvent> | null,
|
||||||
|
options?: EventListenerOptions | boolean,
|
||||||
|
): void;
|
||||||
|
}
|
||||||
20
tsconfig.json
Normal file
20
tsconfig.json
Normal file
@@ -0,0 +1,20 @@
|
|||||||
|
{
|
||||||
|
"compilerOptions": {
|
||||||
|
"target": "es2022",
|
||||||
|
"module": "nodenext",
|
||||||
|
"moduleResolution": "nodenext",
|
||||||
|
"strict": true,
|
||||||
|
"esModuleInterop": true,
|
||||||
|
"skipLibCheck": true,
|
||||||
|
"forceConsistentCasingInFileNames": true,
|
||||||
|
"declaration": true,
|
||||||
|
"declarationMap": true,
|
||||||
|
"sourceMap": true,
|
||||||
|
"outDir": "./dist",
|
||||||
|
"rootDir": "./src",
|
||||||
|
"resolveJsonModule": true,
|
||||||
|
"isolatedModules": true
|
||||||
|
},
|
||||||
|
"include": ["src/**/*.ts"],
|
||||||
|
"exclude": ["node_modules", "dist"]
|
||||||
|
}
|
||||||
14
tsup.config.ts
Normal file
14
tsup.config.ts
Normal file
@@ -0,0 +1,14 @@
|
|||||||
|
import { defineConfig } from 'tsup';
|
||||||
|
|
||||||
|
export default defineConfig({
|
||||||
|
entry: [
|
||||||
|
'src/index.ts',
|
||||||
|
'src/event-target-redis.ts',
|
||||||
|
],
|
||||||
|
format: ['esm', 'cjs'],
|
||||||
|
dts: true,
|
||||||
|
sourcemap: true,
|
||||||
|
clean: true,
|
||||||
|
splitting: true,
|
||||||
|
target: 'es2022',
|
||||||
|
});
|
||||||
8
vitest.config.ts
Normal file
8
vitest.config.ts
Normal file
@@ -0,0 +1,8 @@
|
|||||||
|
import { defineConfig } from 'vitest/config';
|
||||||
|
|
||||||
|
export default defineConfig({
|
||||||
|
test: {
|
||||||
|
globals: true,
|
||||||
|
include: ['test/**/*.test.ts'],
|
||||||
|
},
|
||||||
|
});
|
||||||
Reference in New Issue
Block a user