719 lines
25 KiB
Markdown
719 lines
25 KiB
Markdown
# iroh-ts Reference Implementation Research Report
|
|
|
|
**Project:** [iroh-ts](https://github.com/rayhanadev/iroh-ts) — TypeScript/JavaScript bindings for iroh (peer-to-peer QUIC networking)
|
|
**Date:** 2026-04-23
|
|
**Purpose:** Reference for building taskgraph-ts napi-rs wrapper
|
|
|
|
---
|
|
|
|
## 1. Project Structure
|
|
|
|
iroh-ts is a minimal, single-crate napi-rs project with a flat layout:
|
|
|
|
```
|
|
iroh-ts/
|
|
├── .cargo/
|
|
│ └── config.toml # Cross-compilation rustflags (Windows static CRT)
|
|
├── .github/
|
|
│ └── workflows/
|
|
│ └── CI.yml # GitHub Actions CI/CD
|
|
├── .gitattributes # EOL normalization + mark generated files
|
|
├── .gitignore # Node + Rust ignores
|
|
├── build.rs # napi-build setup (1 line)
|
|
├── Cargo.toml # Rust crate config
|
|
├── bun.lock # Bun lockfile
|
|
├── index.d.ts # Auto-generated TypeScript declarations
|
|
├── index.js # Auto-generated JS loader (platform resolution)
|
|
├── package.json # npm package config
|
|
├── src/
|
|
│ ├── lib.rs # Crate root: module declarations + init_logging
|
|
│ ├── error.rs # IrohError enum + conversion helpers
|
|
│ ├── endpoint.rs # Endpoint class (napi-wrapped)
|
|
│ ├── connection.rs # Connection class + BiStreamResult struct
|
|
│ └── stream.rs # SendStream + RecvStream classes
|
|
├── tsconfig.json # TypeScript config (bundler mode, noEmit)
|
|
└── README.md
|
|
```
|
|
|
|
**Key observations:**
|
|
- No separate `npm/` or `crates/` directory — it is a single Rust crate mapped to a single npm package
|
|
- Generated `index.js` and `index.d.ts` are committed to the repo (auto-generated by `napi build`)
|
|
- Platform-specific `.node` binaries are not committed; they are built in CI and published as separate npm packages
|
|
- No test directory exists in the repo (tests would use the `ava` framework listed in bun.lock)
|
|
|
|
---
|
|
|
|
## 2. Cargo.toml Configuration
|
|
|
|
```toml
|
|
[package]
|
|
name = "iroh"
|
|
version = "0.1.0"
|
|
edition = "2021"
|
|
|
|
[lib]
|
|
crate-type = ["cdylib"] # Critical: produces a C dynamic library for Node
|
|
|
|
[dependencies]
|
|
napi-derive = "3.0.0" # Procedural macro crate (#[napi] attribute)
|
|
iroh = "0.95.1" # The wrapped Rust library
|
|
iroh-base = "0.95.1"
|
|
bytes = "1.11.0"
|
|
thiserror = "2.0.17"
|
|
tracing = "0.1.41"
|
|
|
|
[dependencies.napi]
|
|
version = "3.0.0"
|
|
features = ["async", "tokio_rt"] # Required for async Rust functions exposed to JS
|
|
|
|
[dependencies.tokio]
|
|
version = "1.48.0"
|
|
features = ["sync"] # Only sync feature (tokio_rt comes from napi)
|
|
|
|
[dependencies.tracing-subscriber]
|
|
version = "0.3.20"
|
|
features = ["env-filter"]
|
|
|
|
[build-dependencies]
|
|
napi-build = "2" # Build script helper
|
|
|
|
[profile.release]
|
|
lto = true # Link-Time Optimization for smaller binaries
|
|
strip = "symbols" # Strip debug symbols for smaller binaries
|
|
```
|
|
|
|
**Key takeaways:**
|
|
- `crate-type = ["cdylib"]` — **mandatory** for napi-rs; produces `.node` shared library
|
|
- `napi` and `napi-derive` are separate crates with matched major versions (both 3.0.0)
|
|
- The `async` + `tokio_rt` features on napi are essential if you expose async Rust functions
|
|
- `napi-build = "2"` is a minimal build dependency that just calls `napi_build::setup()`
|
|
- Release profile uses LTO + symbol stripping — standard practice for napi-rs to minimize binary size
|
|
|
|
---
|
|
|
|
## 3. package.json Configuration
|
|
|
|
```json
|
|
{
|
|
"name": "@rayhanadev/iroh",
|
|
"version": "0.1.1",
|
|
"main": "index.js",
|
|
"types": "index.d.ts",
|
|
|
|
"napi": {
|
|
"binaryName": "iroh",
|
|
"targets": [
|
|
"x86_64-pc-windows-msvc",
|
|
"x86_64-apple-darwin",
|
|
"x86_64-unknown-linux-gnu",
|
|
"aarch64-apple-darwin"
|
|
]
|
|
},
|
|
|
|
"scripts": {
|
|
"artifacts": "napi artifacts",
|
|
"build": "napi build --platform --release",
|
|
"build:debug": "napi build --platform",
|
|
"prepublishOnly": "napi prepublish -t npm",
|
|
"preversion": "napi build --platform && git add .",
|
|
"version": "napi version"
|
|
},
|
|
|
|
"devDependencies": {
|
|
"@emnapi/core": "^1.5.0",
|
|
"@emnapi/runtime": "^1.5.0",
|
|
"@napi-rs/cli": "^3.2.0",
|
|
"@tybys/wasm-util": "^0.10.0"
|
|
},
|
|
|
|
"peerDependencies": {
|
|
"typescript": "^5"
|
|
},
|
|
|
|
"optionalDependencies": {
|
|
"@rayhanadev/iroh-win32-x64-msvc": "0.1.0",
|
|
"@rayhanadev/iroh-darwin-x64": "0.1.0",
|
|
"@rayhanadev/iroh-linux-x64-gnu": "0.1.0",
|
|
"@rayhanadev/iroh-darwin-arm64": "0.1.0"
|
|
}
|
|
}
|
|
```
|
|
|
|
**Key takeaways:**
|
|
- The `napi` section in package.json is the core napi-rs config — it defines:
|
|
- `binaryName`: the base name for the `.node` file (must match the crate name)
|
|
- `targets`: which platform triples to build for (used by CI and `napi` CLI)
|
|
- `--platform` flag on `napi build` makes it generate the correct platform-specific filename
|
|
- `optionalDependencies` lists the per-platform npm packages — npm auto-selects the right one
|
|
- These are separate npm packages published from the `npm/` directory created by `napi create-npm-dirs`
|
|
- Names follow the pattern: `@scope/{binaryName}-{platform}-{arch}-{abi}`
|
|
- `@emnapi/core` and `@emnapi/runtime` are for WASI/WASM support (optional, for browser targets)
|
|
- No `@napi-rs/client` dependency — the CLI (`@napi-rs/cli`) is the only napi-rs JS dependency needed
|
|
- The `prepublishOnly` script runs `napi prepublish` which handles version alignment across platform packages
|
|
|
|
---
|
|
|
|
## 4. Rust Function/Class Export Patterns via napi-rs Macros
|
|
|
|
### 4.1 Crate Root (`lib.rs`)
|
|
|
|
```rust
|
|
#![deny(clippy::all)]
|
|
|
|
use napi_derive::napi;
|
|
|
|
mod connection;
|
|
mod endpoint;
|
|
mod error;
|
|
mod stream;
|
|
|
|
pub use connection::*;
|
|
pub use endpoint::*;
|
|
pub use error::*;
|
|
pub use stream::*;
|
|
|
|
#[napi]
|
|
pub fn init_logging(level: Option<String>) {
|
|
let filter = level.unwrap_or_else(|| "info".to_string());
|
|
let _ = tracing_subscriber::fmt().with_env_filter(filter).try_init();
|
|
}
|
|
```
|
|
|
|
**Patterns:**
|
|
- `#![deny(clippy::all)]` — strict linting at crate level
|
|
- Module-based organization with `pub use` re-exports from the crate root
|
|
- Free functions annotated with `#[napi]` become top-level JS exports
|
|
- `Option<String>` maps to `string | undefined | null` in TypeScript
|
|
|
|
### 4.2 Class Pattern (Endpoint)
|
|
|
|
```rust
|
|
#[napi]
|
|
pub struct Endpoint {
|
|
inner: Arc<iroh::Endpoint>, // Wrap the Rust type in Arc for shared ownership
|
|
}
|
|
|
|
#[napi]
|
|
impl Endpoint {
|
|
#[napi(factory)] // Static factory method → Endpoint.create()
|
|
pub async fn create() -> Result<Endpoint> { ... }
|
|
|
|
#[napi(factory)] // With options → Endpoint.createWithOptions(opts)
|
|
pub async fn create_with_options(options: EndpointOptions) -> Result<Endpoint> { ... }
|
|
|
|
#[napi] // Instance method → endpoint.nodeId()
|
|
pub fn node_id(&self) -> String { ... }
|
|
|
|
#[napi] // Instance method → endpoint.connect(addr, alpn)
|
|
pub async fn connect(&self, addr: String, alpn: String) -> Result<Connection> { ... }
|
|
}
|
|
```
|
|
|
|
**Key patterns:**
|
|
1. **Struct + impl separation:** The struct definition gets `#[napi]`, AND a separate `#[napi] impl` block exposes methods
|
|
2. **`Arc<InnerType>`** pattern: Rust objects shared across JS/Rust boundary are wrapped in `Arc`. This is critical because:
|
|
- Node.js is single-threaded but napi-rs creates a reference that may outlive the original scope
|
|
- The Arc allows the JS wrapper to own a reference-counted Rust object
|
|
3. **`#[napi(factory)]`**: Creates a static factory method instead of a constructor. This is the idiomatic way to handle async construction (since WASM/native constructors can't be async)
|
|
4. **`snake_case` Rust → `camelCase` JS**: napi-rs auto-converts `node_id` to `nodeId`, `create_with_options` to `createWithOptions`
|
|
5. **Return types:** `Result<T>` maps to JS `T | Error` (throws on Err)
|
|
|
|
### 4.3 Struct with Mutex for Interior Mutability (SendStream/RecvStream)
|
|
|
|
```rust
|
|
#[derive(Clone)]
|
|
#[napi]
|
|
pub struct SendStream {
|
|
inner: Arc<Mutex<iroh::endpoint::SendStream>>, // Arc<Mutex<T>> for mutable access
|
|
}
|
|
|
|
#[napi]
|
|
impl SendStream {
|
|
pub async fn write(&self, data: Buffer) -> Result<u32> {
|
|
let mut stream = self.inner.lock().await; // tokio Mutex for async-compatible locking
|
|
let written = stream.write(&data).await.map_err(to_napi_error)?;
|
|
Ok(written as u32)
|
|
}
|
|
}
|
|
```
|
|
|
|
**Key patterns:**
|
|
- `Arc<Mutex<T>>` with tokio's async Mutex — required when async methods need mutable access
|
|
- `#[derive(Clone)]` because the `Arc<Mutex<>>` is cheaply cloneable
|
|
- `Buffer` type from `napi::bindgen_prelude::*` maps to Node.js `Buffer`
|
|
- Manual `as u32` cast because napi-rs doesn't support `usize` (platform-dependent size)
|
|
|
|
### 4.4 Object Struct (Options)
|
|
|
|
```rust
|
|
#[napi(object)]
|
|
#[derive(Default)]
|
|
pub struct EndpointOptions {
|
|
pub alpns: Option<Vec<String>>,
|
|
pub secret_key: Option<String>,
|
|
}
|
|
```
|
|
|
|
**Pattern:**
|
|
- `#[napi(object)]` makes this a plain JS object (not a class) — fields become properties
|
|
- `#[derive(Default)]` allows easy construction on the Rust side when JS omits fields
|
|
- `Option<T>` makes fields optional in the TypeScript definition
|
|
|
|
### 4.5 Getter Properties (BiStreamResult)
|
|
|
|
```rust
|
|
#[napi]
|
|
pub struct BiStreamResult {
|
|
send: SendStream,
|
|
recv: RecvStream,
|
|
}
|
|
|
|
#[napi]
|
|
impl BiStreamResult {
|
|
#[napi(getter)]
|
|
pub fn get_send(&self) -> SendStream {
|
|
self.send.clone()
|
|
}
|
|
|
|
#[napi(getter)]
|
|
pub fn get_recv(&self) -> RecvStream {
|
|
self.recv.clone()
|
|
}
|
|
}
|
|
```
|
|
|
|
**Pattern:**
|
|
- Use `#[napi(getter)]` to expose Rust struct fields as JS getter properties
|
|
- The `get_` prefix is stripped: `get_send` → `.send` property in JS
|
|
- Must return cloned values since the Rust struct owns the data
|
|
|
|
### 4.6 Type Mappings (Rust → TypeScript)
|
|
|
|
| Rust Type | TypeScript Type |
|
|
|-----------|---------------|
|
|
| `String` | `string` |
|
|
| `u32`, `f64` | `number` |
|
|
| `bool` | `boolean` |
|
|
| `Option<T>` | `T \| null \| undefined` |
|
|
| `Option<u32>` | `number \| null` |
|
|
| `Vec<String>` | `Array<string>` |
|
|
| `Buffer` (from napi) | `Buffer` |
|
|
| `Result<T>` | `T` (throws on Err) |
|
|
| `Result<Option<T>>` | `T \| null` |
|
|
| `#[napi] struct` | `class` |
|
|
| `#[napi(object)] struct` | `interface` |
|
|
|
|
---
|
|
|
|
## 5. TypeScript Type Definitions
|
|
|
|
### Auto-Generated `index.d.ts`
|
|
|
|
The `index.d.ts` file is **entirely auto-generated** by `napi build --platform`. It is not hand-written.
|
|
|
|
Generated output structure:
|
|
```typescript
|
|
/* auto-generated by NAPI-RS */
|
|
/* eslint-disable */
|
|
|
|
export declare class BiStreamResult {
|
|
get send(): SendStream
|
|
get recv(): RecvStream
|
|
}
|
|
|
|
export declare class Connection {
|
|
remoteNodeId(): string
|
|
alpn(): Buffer
|
|
openBi(): Promise<BiStreamResult>
|
|
openUni(): Promise<SendStream>
|
|
// ... etc
|
|
}
|
|
|
|
export declare class Endpoint {
|
|
static create(): Promise<Endpoint>
|
|
static createWithOptions(options: EndpointOptions): Promise<Endpoint>
|
|
nodeId(): string
|
|
// ... etc
|
|
}
|
|
|
|
export interface EndpointOptions {
|
|
alpns?: Array<string>
|
|
secretKey?: string
|
|
}
|
|
|
|
export declare function initLogging(level?: string | undefined | null): void
|
|
```
|
|
|
|
**Key observations:**
|
|
- `#[napi]` structs become `declare class` with methods
|
|
- `#[napi(object)]` structs become `interface`
|
|
- `#[napi(factory)]` methods become `static` methods
|
|
- `#[napi(getter)]` methods become `get` accessor properties
|
|
- Doc comments from Rust (`/// ...`) are preserved as JSDoc in the `.d.ts`
|
|
- Async Rust functions correctly become `Promise<T>` return types
|
|
- The `@ts-nocheck` and `/* eslint-disable */` in `index.js` suppress linting of generated code
|
|
|
|
### tsconfig.json
|
|
|
|
```json
|
|
{
|
|
"compilerOptions": {
|
|
"lib": ["ESNext"],
|
|
"target": "ESNext",
|
|
"module": "Preserve",
|
|
"moduleResolution": "bundler",
|
|
"allowImportingTsExtensions": true,
|
|
"verbatimModuleSyntax": true,
|
|
"noEmit": true,
|
|
"strict": true,
|
|
"skipLibCheck": true
|
|
}
|
|
}
|
|
```
|
|
|
|
This is a development-only config — the package doesn't ship compiled TS, only `.d.ts` declarations.
|
|
|
|
---
|
|
|
|
## 6. Build Pipeline
|
|
|
|
### 6.1 Local Development Build
|
|
|
|
```bash
|
|
bun install # Install @napi-rs/cli + deps
|
|
bun run build # napi build --platform --release
|
|
```
|
|
|
|
What `napi build --platform --release` does:
|
|
1. Invokes `cargo build --release` to compile the Rust crate as `cdylib`
|
|
2. Copies the resulting `.so`/`.dylib`/`.dll` to the project root as `{binaryName}.{platform}.node`
|
|
- e.g., `iroh.linux-x64-gnu.node`
|
|
3. Generates `index.d.ts` (TypeScript declarations) from the `#[napi]` annotated code
|
|
4. Generates `index.js` (CommonJS loader) that resolves the correct `.node` file per platform
|
|
|
|
### 6.2 CI Build (Cross-Compilation)
|
|
|
|
The CI matrix builds for 4 targets:
|
|
|
|
| Host Runner | Target | Build Command |
|
|
|-------------|--------|---------------|
|
|
| `macos-latest` | `x86_64-apple-darwin` | `bun run build --target x86_64-apple-darwin` |
|
|
| `windows-latest` | `x86_64-pc-windows-msvc` | `bun run build --target x86_64-pc-windows-msvc` |
|
|
| `ubuntu-latest` | `x86_64-unknown-linux-gnu` | `bun run build --target x86_64-unknown-linux-gnu --use-napi-cross` |
|
|
| `macos-latest` | `aarch64-apple-darwin` | `bun run build --target aarch64-apple-darwin` |
|
|
|
|
Cross-compilation notes:
|
|
- **macOS → aarch64**: Uses `dtolnay/rust-toolchain` with `targets` parameter; macOS cross-compilation "just works" with Rust
|
|
- **Linux x64**: Uses `--use-napi-cross` flag which downloads cross-compilation toolchains via `@napi-rs/cross-toolchain`
|
|
- **Musl targets**: Would use `cargo-zigbuild` via Zig (configured in CI but not in the current target matrix)
|
|
- **Windows**: Uses MSVC target; `.cargo/config.toml` sets `+crt-static` for static CRT linking
|
|
|
|
### 6.3 Artifacts and Publishing
|
|
|
|
1. Each matrix job uploads its `.node` file as a GitHub artifact
|
|
2. The `publish` job:
|
|
- Downloads all artifacts
|
|
- Runs `napi create-npm-dirs` to create `npm/` directory structure
|
|
- Runs `napi artifacts` to move `.node` files into per-platform npm packages under `npm/`
|
|
- Publishes the main package + all platform packages to npm
|
|
|
|
### 6.4 The `index.js` Loader
|
|
|
|
The auto-generated `index.js` is ~580 lines and handles:
|
|
- Platform detection (`process.platform` + `process.arch`)
|
|
- musl vs glibc detection on Linux (tries filesystem, process report, and child process)
|
|
- Resolution ordering: try local `.node` file first, then fall back to `@scope/pkg-platform` npm package
|
|
- `NAPI_RS_NATIVE_LIBRARY_PATH` environment variable override for custom paths
|
|
- WASI/WASM fallback for browser environments
|
|
- Version mismatch checking between main package and platform packages
|
|
|
|
---
|
|
|
|
## 7. Error Handling Across the Rust/TS Boundary
|
|
|
|
### 7.1 Error Definition Pattern
|
|
|
|
```rust
|
|
#[derive(Debug, Error)]
|
|
pub enum IrohError {
|
|
#[error("Bind error: {0}")]
|
|
Bind(String),
|
|
#[error("Connect error: {0}")]
|
|
Connect(String),
|
|
#[error("Stream error: {0}")]
|
|
Stream(String),
|
|
#[error("Parse error: {0}")]
|
|
Parse(String),
|
|
#[error("Connection closed: {0}")]
|
|
ConnectionClosed(String),
|
|
#[error("Internal error: {0}")]
|
|
Internal(String),
|
|
}
|
|
```
|
|
|
|
### 7.2 Error Conversion
|
|
|
|
Two approaches are used:
|
|
|
|
**Approach 1: `From` trait implementation**
|
|
```rust
|
|
impl From<IrohError> for napi::Error {
|
|
fn from(err: IrohError) -> Self {
|
|
napi::Error::from_reason(err.to_string())
|
|
}
|
|
}
|
|
```
|
|
|
|
**Approach 2: Generic helper function (preferred for inline use)**
|
|
```rust
|
|
pub fn to_napi_error<E: std::fmt::Display>(err: E) -> napi::Error {
|
|
napi::Error::from_reason(err.to_string())
|
|
}
|
|
```
|
|
|
|
### 7.3 Usage in Methods
|
|
|
|
```rust
|
|
pub async fn connect(&self, addr: String, alpn: String) -> Result<Connection> {
|
|
let endpoint_id: iroh::EndpointId = addr
|
|
.parse()
|
|
.map_err(|e| to_napi_error(format!("Invalid node ID: {e}")))?; // Add context
|
|
|
|
let conn = self.inner.connect(endpoint_id, alpn.as_bytes())
|
|
.await
|
|
.map_err(to_napi_error)?; // Simple conversion
|
|
|
|
Ok(Connection::new(conn))
|
|
}
|
|
```
|
|
|
|
**Key patterns:**
|
|
- All Rust errors are converted to `napi::Error::from_reason(string)` — they become JS `Error` objects with a `.message` property
|
|
- `thiserror` is used on the Rust side for structured error categories, but these are **flattened to strings** at the boundary
|
|
- The `to_napi_error` helper is more flexible than the `From` impl because it works with any `Display` type
|
|
- Context is added manually with `format!("Context: {e}")` wrapping, since the error types lose structure at the boundary
|
|
- `Result<T>` in Rust maps to throwing in JS (no explicit `Result` type in TypeScript)
|
|
|
|
### 7.4 TypeScript Side
|
|
|
|
On the TS side, errors appear as standard JavaScript `Error` objects thrown from native code. There is no typed error handling — everything is a string message:
|
|
|
|
```typescript
|
|
try {
|
|
const conn = await endpoint.connect(addr, alpn);
|
|
} catch (err) {
|
|
// err is a standard Error with a message like "Connect error: connection refused"
|
|
}
|
|
```
|
|
|
|
**Critique:** This is a weak point of iroh-ts. Structured error information (error kind, codes) is lost at the boundary. For taskgraph-ts, we should consider:
|
|
- Using `napi::Error::from_status_with_message()` with custom status codes
|
|
- Or defining a custom error class on the JS side that parses the error string
|
|
- Or using napi-rs's built-in `#[napi(ts_type)]` for more detailed type info
|
|
|
|
---
|
|
|
|
## 8. CI/CD and Cross-Compilation Setup
|
|
|
|
### 8.1 GitHub Actions Workflow
|
|
|
|
**Jobs:**
|
|
1. **`lint`** (ubuntu-latest): `cargo fmt -- --check` + `cargo clippy`
|
|
2. **`build`** (matrix): Cross-compile for 4 targets, upload `.node` artifacts
|
|
3. **`publish`** (ubuntu-latest): Download artifacts, create npm dirs, publish to npm
|
|
|
|
**Key CI details:**
|
|
- Bun is used instead of npm/yarn (`oven-sh/setup-bun@v1`)
|
|
- Rust toolchain via `dtolnay/rust-toolchain@stable` with `components: clippy, rustfmt`
|
|
- Cargo cache keyed by `target-host` pair
|
|
- Zig + cargo-zigbuild installed conditionally for musl targets (though no musl targets in current matrix)
|
|
- Version-based publishing: only publishes if the latest commit message matches a semver pattern
|
|
- `npm config set provenance true` for supply chain security
|
|
- Sequential `needs` dependency: `build` requires `lint`, `publish` requires both
|
|
|
|
### 8.2 Cross-Compilation Config
|
|
|
|
**`.cargo/config.toml`:**
|
|
```toml
|
|
[target.x86_64-pc-windows-msvc]
|
|
rustflags = ["-C", "target-feature=+crt-static"]
|
|
```
|
|
|
|
This ensures the Windows MSVC build statically links the C runtime, avoiding DLL dependency issues.
|
|
|
|
### 8.3 Platform Package Structure
|
|
|
|
When published, napi-rs creates a directory structure like:
|
|
```
|
|
npm/
|
|
├── darwin-arm64/ → @rayhanadev/iroh-darwin-arm64
|
|
├── darwin-x64/ → @rayhanadev/iroh-darwin-x64
|
|
├── linux-x64-gnu/ → @rayhanadev/iroh-linux-x64-gnu
|
|
└── win32-x64-msvc/ → @rayhanadev/iroh-win32-x64-msvc
|
|
```
|
|
|
|
Each contains a minimal `package.json` with `os` + `cpu` constraints and the `.node` file.
|
|
|
|
---
|
|
|
|
## 9. napi-rs Version and Dependency Pinning
|
|
|
|
| Package | Version | Notes |
|
|
|---------|---------|-------|
|
|
| `napi` (Rust) | `3.0.0` | Core runtime crate |
|
|
| `napi-derive` (Rust) | `3.0.0` | Procedural macros (must match napi major) |
|
|
| `napi-build` (Rust) | `2` | Build script helper |
|
|
| `@napi-rs/cli` (JS) | `^3.2.0` | CLI tool (resolved to 3.4.1 in lock) |
|
|
|
|
**Important:** `napi` and `napi-derive` MUST share the same major version. Version 3.x is the latest stable line.
|
|
|
|
The `features = ["async", "tokio_rt"]` on the `napi` crate are critical:
|
|
- `async`: Enables `#[napi]` on async functions
|
|
- `tokio_rt`: Uses the tokio runtime for async napi-rs tasks (there is also `napi4` feature for custom threads)
|
|
|
|
---
|
|
|
|
## 10. Patterns Worth Replicating and Avoiding
|
|
|
|
### Patterns to Replicate
|
|
|
|
1. **Arc wrapping for shared Rust types** — `Arc<InnerType>` for structs that are shared across the JS/Rust boundary. This is essential for correct lifetime management.
|
|
|
|
2. **`#[napi(factory)]` for async constructors** — Since JS constructors can't be async, use factory methods that return `Promise<T>`. The `Endpoint::create()` pattern is clean and idiomatic.
|
|
|
|
3. **Module-per-domain** — Separate files for each domain concept (endpoint, connection, stream, error). Clean separation of concerns.
|
|
|
|
4. **`#[napi(object)]` for options structs** — Clean mapping of Rust option structs to JS plain objects without class overhead.
|
|
|
|
5. **`to_napi_error` helper** — Generic error conversion function that works with any `Display` type. Much more ergonomic than writing `From` impls for every error type.
|
|
|
|
6. **Release profile optimization** — `lto = true` + `strip = "symbols"` significantly reduces binary size.
|
|
|
|
7. **Doc comments preserved** — Rust doc comments (`///`) flow through to TypeScript declarations, providing IDE autocomplete.
|
|
|
|
8. **`.gitattributes` marking generated files** — Using `linguist-detectable=false` for `index.js`, `index.d.ts`, and WASM files keeps GitHub language stats clean.
|
|
|
|
9. **Separate `impl` blocks** — Having a non-`#[napi]` `impl` block for the `new()` constructor and a `#[napi] impl` block for JS-exposed methods cleanly separates internal Rust API from external JS API.
|
|
|
|
10. **Platform-specific cargo config** — The `.cargo/config.toml` with `+crt-static` for Windows avoids runtime DLL issues.
|
|
|
|
### Patterns to Improve Upon
|
|
|
|
1. **Error handling is lossy** — All structured Rust errors are flattened to strings. Consider:
|
|
- Custom napi error classes that preserve error categories
|
|
- Error status codes using `napi::Error` with custom `Status` values
|
|
- A richer error object on the JS side
|
|
|
|
2. **No test infrastructure** — The project has no unit tests. For taskgraph-ts, we should add:
|
|
- Rust unit tests for internal logic
|
|
- JS/TS integration tests that verify the napi boundary
|
|
- CI test step that runs after build
|
|
|
|
3. **No TypeScript wrapper layer** — `index.d.ts` is purely auto-generated. Consider adding a hand-written `src/index.ts` that:
|
|
- Re-exports from the native module
|
|
- Adds convenience methods
|
|
- Provides typed error classes
|
|
- Adds input validation before calling native code
|
|
|
|
4. **Missing musl/Linux ARM64 targets** — Only 4 targets in the `napi` config. For broader compatibility, consider adding:
|
|
- `aarch64-unknown-linux-gnu` (ARM Linux, AWS Graviton)
|
|
- `x86_64-unknown-linux-musl` (Alpine Docker images)
|
|
|
|
5. **No `Cargo.lock` committed** — The `.gitignore` excludes `Cargo.lock`. For a binary crate (as opposed to a library), it's generally recommended to commit the lock file for reproducible builds. However, napi-rs projects often skip this since the CI builds from scratch.
|
|
|
|
6. **No `napi.config.js` or `napi.config.ts`** — Some larger napi-rs projects use a dedicated config file. For a small project, `package.json` `napi` section is fine, but it doesn't scale well.
|
|
|
|
7. **`Arc<Mutex<T>>` everywhere** — The SendStream/RecvStream use `tokio::sync::Mutex` even for read-only operations (like `id()`). Consider whether some operations could use `Arc<tokio::sync::RwLock>` or even `std::sync::RwLock` for non-async read paths.
|
|
|
|
8. **`init_logging` as a public API** — Exposing `tracing_subscriber::fmt().try_init()` is fragile: it can only succeed once, and multiple calls silently fail. Consider returning a boolean/result indicating whether initialization succeeded.
|
|
|
|
9. **No version consistency checks** — The Rust crate version (`0.1.0`) and npm package version (`0.1.1`) differ. This could cause confusion. Consider aligning them or using a script to keep them in sync.
|
|
|
|
10. **No `#[napi(custom_finalize)]`** — Missing custom finalize/destructor logic. If Rust objects need cleanup (e.g., flushing streams before drop), the default finalization may not be sufficient.
|
|
|
|
---
|
|
|
|
## Summary: Recommended Architecture for taskgraph-ts
|
|
|
|
Based on this research, here is a recommended approach:
|
|
|
|
### Cargo.toml
|
|
```toml
|
|
[package]
|
|
name = "taskgraph"
|
|
version = "0.1.0"
|
|
edition = "2021"
|
|
|
|
[lib]
|
|
crate-type = ["cdylib"]
|
|
|
|
[dependencies]
|
|
napi-derive = "3.0.0"
|
|
# taskgraph = "..." (the Rust library being wrapped)
|
|
thiserror = "2"
|
|
tracing = "0.1"
|
|
|
|
[dependencies.napi]
|
|
version = "3.0.0"
|
|
features = ["async", "tokio_rt"]
|
|
|
|
[build-dependencies]
|
|
napi-build = "2"
|
|
|
|
[profile.release]
|
|
lto = true
|
|
strip = "symbols"
|
|
```
|
|
|
|
### package.json
|
|
```json
|
|
{
|
|
"name": "@alkdev/taskgraph",
|
|
"main": "index.js",
|
|
"types": "index.d.ts",
|
|
"napi": {
|
|
"binaryName": "taskgraph",
|
|
"targets": [
|
|
"x86_64-apple-darwin",
|
|
"aarch64-apple-darwin",
|
|
"x86_64-unknown-linux-gnu",
|
|
"aarch64-unknown-linux-gnu",
|
|
"x86_64-pc-windows-msvc"
|
|
]
|
|
},
|
|
"scripts": {
|
|
"build": "napi build --platform --release",
|
|
"build:debug": "napi build --platform",
|
|
"artifacts": "napi artifacts",
|
|
"prepublishOnly": "napi prepublish -t npm",
|
|
"version": "napi version"
|
|
},
|
|
"devDependencies": {
|
|
"@napi-rs/cli": "^3.2.0"
|
|
}
|
|
}
|
|
```
|
|
|
|
### Rust Source Layout
|
|
```
|
|
src/
|
|
├── lib.rs # Module declarations + any free functions
|
|
├── error.rs # Error types + to_napi_error helper
|
|
├── graph.rs # Graph class (#[napi] struct + impl)
|
|
├── task.rs # Task class
|
|
├── executor.rs # Executor class (async factory methods)
|
|
└── types.rs # Shared types, #[napi(object)] structs
|
|
```
|
|
|
|
### Key Conventions to Adopt
|
|
1. `Arc<T>` for all napi struct inner types
|
|
2. `#[napi(factory)]` for async constructors
|
|
3. `#[napi(object)]` for option/config structs
|
|
4. `to_napi_error` generic helper for error conversion
|
|
5. Preserve doc comments for auto-generated `.d.ts`
|
|
6. Always match `napi` and `napi-derive` major versions
|
|
7. Use `tokio::sync::Mutex` for async interior mutability
|
|
8. Add `.cargo/config.toml` with Windows CRT static linking
|
|
9. Use `bun` as the JS runtime (fast, napi-rs compatible)
|
|
10. Commit generated `index.js` and `index.d.ts` to the repo
|