Add architecture doc and research reports for taskgraph_ts napi wrapper
This commit is contained in:
718
docs/research/iroh_ts_reference.md
Normal file
718
docs/research/iroh_ts_reference.md
Normal file
@@ -0,0 +1,718 @@
|
||||
# iroh-ts Reference Implementation Research Report
|
||||
|
||||
**Project:** [iroh-ts](https://github.com/rayhanadev/iroh-ts) — TypeScript/JavaScript bindings for iroh (peer-to-peer QUIC networking)
|
||||
**Date:** 2026-04-23
|
||||
**Purpose:** Reference for building taskgraph-ts napi-rs wrapper
|
||||
|
||||
---
|
||||
|
||||
## 1. Project Structure
|
||||
|
||||
iroh-ts is a minimal, single-crate napi-rs project with a flat layout:
|
||||
|
||||
```
|
||||
iroh-ts/
|
||||
├── .cargo/
|
||||
│ └── config.toml # Cross-compilation rustflags (Windows static CRT)
|
||||
├── .github/
|
||||
│ └── workflows/
|
||||
│ └── CI.yml # GitHub Actions CI/CD
|
||||
├── .gitattributes # EOL normalization + mark generated files
|
||||
├── .gitignore # Node + Rust ignores
|
||||
├── build.rs # napi-build setup (1 line)
|
||||
├── Cargo.toml # Rust crate config
|
||||
├── bun.lock # Bun lockfile
|
||||
├── index.d.ts # Auto-generated TypeScript declarations
|
||||
├── index.js # Auto-generated JS loader (platform resolution)
|
||||
├── package.json # npm package config
|
||||
├── src/
|
||||
│ ├── lib.rs # Crate root: module declarations + init_logging
|
||||
│ ├── error.rs # IrohError enum + conversion helpers
|
||||
│ ├── endpoint.rs # Endpoint class (napi-wrapped)
|
||||
│ ├── connection.rs # Connection class + BiStreamResult struct
|
||||
│ └── stream.rs # SendStream + RecvStream classes
|
||||
├── tsconfig.json # TypeScript config (bundler mode, noEmit)
|
||||
└── README.md
|
||||
```
|
||||
|
||||
**Key observations:**
|
||||
- No separate `npm/` or `crates/` directory — it is a single Rust crate mapped to a single npm package
|
||||
- Generated `index.js` and `index.d.ts` are committed to the repo (auto-generated by `napi build`)
|
||||
- Platform-specific `.node` binaries are not committed; they are built in CI and published as separate npm packages
|
||||
- No test directory exists in the repo (tests would use the `ava` framework listed in bun.lock)
|
||||
|
||||
---
|
||||
|
||||
## 2. Cargo.toml Configuration
|
||||
|
||||
```toml
|
||||
[package]
|
||||
name = "iroh"
|
||||
version = "0.1.0"
|
||||
edition = "2021"
|
||||
|
||||
[lib]
|
||||
crate-type = ["cdylib"] # Critical: produces a C dynamic library for Node
|
||||
|
||||
[dependencies]
|
||||
napi-derive = "3.0.0" # Procedural macro crate (#[napi] attribute)
|
||||
iroh = "0.95.1" # The wrapped Rust library
|
||||
iroh-base = "0.95.1"
|
||||
bytes = "1.11.0"
|
||||
thiserror = "2.0.17"
|
||||
tracing = "0.1.41"
|
||||
|
||||
[dependencies.napi]
|
||||
version = "3.0.0"
|
||||
features = ["async", "tokio_rt"] # Required for async Rust functions exposed to JS
|
||||
|
||||
[dependencies.tokio]
|
||||
version = "1.48.0"
|
||||
features = ["sync"] # Only sync feature (tokio_rt comes from napi)
|
||||
|
||||
[dependencies.tracing-subscriber]
|
||||
version = "0.3.20"
|
||||
features = ["env-filter"]
|
||||
|
||||
[build-dependencies]
|
||||
napi-build = "2" # Build script helper
|
||||
|
||||
[profile.release]
|
||||
lto = true # Link-Time Optimization for smaller binaries
|
||||
strip = "symbols" # Strip debug symbols for smaller binaries
|
||||
```
|
||||
|
||||
**Key takeaways:**
|
||||
- `crate-type = ["cdylib"]` — **mandatory** for napi-rs; produces `.node` shared library
|
||||
- `napi` and `napi-derive` are separate crates with matched major versions (both 3.0.0)
|
||||
- The `async` + `tokio_rt` features on napi are essential if you expose async Rust functions
|
||||
- `napi-build = "2"` is a minimal build dependency that just calls `napi_build::setup()`
|
||||
- Release profile uses LTO + symbol stripping — standard practice for napi-rs to minimize binary size
|
||||
|
||||
---
|
||||
|
||||
## 3. package.json Configuration
|
||||
|
||||
```json
|
||||
{
|
||||
"name": "@rayhanadev/iroh",
|
||||
"version": "0.1.1",
|
||||
"main": "index.js",
|
||||
"types": "index.d.ts",
|
||||
|
||||
"napi": {
|
||||
"binaryName": "iroh",
|
||||
"targets": [
|
||||
"x86_64-pc-windows-msvc",
|
||||
"x86_64-apple-darwin",
|
||||
"x86_64-unknown-linux-gnu",
|
||||
"aarch64-apple-darwin"
|
||||
]
|
||||
},
|
||||
|
||||
"scripts": {
|
||||
"artifacts": "napi artifacts",
|
||||
"build": "napi build --platform --release",
|
||||
"build:debug": "napi build --platform",
|
||||
"prepublishOnly": "napi prepublish -t npm",
|
||||
"preversion": "napi build --platform && git add .",
|
||||
"version": "napi version"
|
||||
},
|
||||
|
||||
"devDependencies": {
|
||||
"@emnapi/core": "^1.5.0",
|
||||
"@emnapi/runtime": "^1.5.0",
|
||||
"@napi-rs/cli": "^3.2.0",
|
||||
"@tybys/wasm-util": "^0.10.0"
|
||||
},
|
||||
|
||||
"peerDependencies": {
|
||||
"typescript": "^5"
|
||||
},
|
||||
|
||||
"optionalDependencies": {
|
||||
"@rayhanadev/iroh-win32-x64-msvc": "0.1.0",
|
||||
"@rayhanadev/iroh-darwin-x64": "0.1.0",
|
||||
"@rayhanadev/iroh-linux-x64-gnu": "0.1.0",
|
||||
"@rayhanadev/iroh-darwin-arm64": "0.1.0"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Key takeaways:**
|
||||
- The `napi` section in package.json is the core napi-rs config — it defines:
|
||||
- `binaryName`: the base name for the `.node` file (must match the crate name)
|
||||
- `targets`: which platform triples to build for (used by CI and `napi` CLI)
|
||||
- `--platform` flag on `napi build` makes it generate the correct platform-specific filename
|
||||
- `optionalDependencies` lists the per-platform npm packages — npm auto-selects the right one
|
||||
- These are separate npm packages published from the `npm/` directory created by `napi create-npm-dirs`
|
||||
- Names follow the pattern: `@scope/{binaryName}-{platform}-{arch}-{abi}`
|
||||
- `@emnapi/core` and `@emnapi/runtime` are for WASI/WASM support (optional, for browser targets)
|
||||
- No `@napi-rs/client` dependency — the CLI (`@napi-rs/cli`) is the only napi-rs JS dependency needed
|
||||
- The `prepublishOnly` script runs `napi prepublish` which handles version alignment across platform packages
|
||||
|
||||
---
|
||||
|
||||
## 4. Rust Function/Class Export Patterns via napi-rs Macros
|
||||
|
||||
### 4.1 Crate Root (`lib.rs`)
|
||||
|
||||
```rust
|
||||
#![deny(clippy::all)]
|
||||
|
||||
use napi_derive::napi;
|
||||
|
||||
mod connection;
|
||||
mod endpoint;
|
||||
mod error;
|
||||
mod stream;
|
||||
|
||||
pub use connection::*;
|
||||
pub use endpoint::*;
|
||||
pub use error::*;
|
||||
pub use stream::*;
|
||||
|
||||
#[napi]
|
||||
pub fn init_logging(level: Option<String>) {
|
||||
let filter = level.unwrap_or_else(|| "info".to_string());
|
||||
let _ = tracing_subscriber::fmt().with_env_filter(filter).try_init();
|
||||
}
|
||||
```
|
||||
|
||||
**Patterns:**
|
||||
- `#![deny(clippy::all)]` — strict linting at crate level
|
||||
- Module-based organization with `pub use` re-exports from the crate root
|
||||
- Free functions annotated with `#[napi]` become top-level JS exports
|
||||
- `Option<String>` maps to `string | undefined | null` in TypeScript
|
||||
|
||||
### 4.2 Class Pattern (Endpoint)
|
||||
|
||||
```rust
|
||||
#[napi]
|
||||
pub struct Endpoint {
|
||||
inner: Arc<iroh::Endpoint>, // Wrap the Rust type in Arc for shared ownership
|
||||
}
|
||||
|
||||
#[napi]
|
||||
impl Endpoint {
|
||||
#[napi(factory)] // Static factory method → Endpoint.create()
|
||||
pub async fn create() -> Result<Endpoint> { ... }
|
||||
|
||||
#[napi(factory)] // With options → Endpoint.createWithOptions(opts)
|
||||
pub async fn create_with_options(options: EndpointOptions) -> Result<Endpoint> { ... }
|
||||
|
||||
#[napi] // Instance method → endpoint.nodeId()
|
||||
pub fn node_id(&self) -> String { ... }
|
||||
|
||||
#[napi] // Instance method → endpoint.connect(addr, alpn)
|
||||
pub async fn connect(&self, addr: String, alpn: String) -> Result<Connection> { ... }
|
||||
}
|
||||
```
|
||||
|
||||
**Key patterns:**
|
||||
1. **Struct + impl separation:** The struct definition gets `#[napi]`, AND a separate `#[napi] impl` block exposes methods
|
||||
2. **`Arc<InnerType>`** pattern: Rust objects shared across JS/Rust boundary are wrapped in `Arc`. This is critical because:
|
||||
- Node.js is single-threaded but napi-rs creates a reference that may outlive the original scope
|
||||
- The Arc allows the JS wrapper to own a reference-counted Rust object
|
||||
3. **`#[napi(factory)]`**: Creates a static factory method instead of a constructor. This is the idiomatic way to handle async construction (since WASM/native constructors can't be async)
|
||||
4. **`snake_case` Rust → `camelCase` JS**: napi-rs auto-converts `node_id` to `nodeId`, `create_with_options` to `createWithOptions`
|
||||
5. **Return types:** `Result<T>` maps to JS `T | Error` (throws on Err)
|
||||
|
||||
### 4.3 Struct with Mutex for Interior Mutability (SendStream/RecvStream)
|
||||
|
||||
```rust
|
||||
#[derive(Clone)]
|
||||
#[napi]
|
||||
pub struct SendStream {
|
||||
inner: Arc<Mutex<iroh::endpoint::SendStream>>, // Arc<Mutex<T>> for mutable access
|
||||
}
|
||||
|
||||
#[napi]
|
||||
impl SendStream {
|
||||
pub async fn write(&self, data: Buffer) -> Result<u32> {
|
||||
let mut stream = self.inner.lock().await; // tokio Mutex for async-compatible locking
|
||||
let written = stream.write(&data).await.map_err(to_napi_error)?;
|
||||
Ok(written as u32)
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Key patterns:**
|
||||
- `Arc<Mutex<T>>` with tokio's async Mutex — required when async methods need mutable access
|
||||
- `#[derive(Clone)]` because the `Arc<Mutex<>>` is cheaply cloneable
|
||||
- `Buffer` type from `napi::bindgen_prelude::*` maps to Node.js `Buffer`
|
||||
- Manual `as u32` cast because napi-rs doesn't support `usize` (platform-dependent size)
|
||||
|
||||
### 4.4 Object Struct (Options)
|
||||
|
||||
```rust
|
||||
#[napi(object)]
|
||||
#[derive(Default)]
|
||||
pub struct EndpointOptions {
|
||||
pub alpns: Option<Vec<String>>,
|
||||
pub secret_key: Option<String>,
|
||||
}
|
||||
```
|
||||
|
||||
**Pattern:**
|
||||
- `#[napi(object)]` makes this a plain JS object (not a class) — fields become properties
|
||||
- `#[derive(Default)]` allows easy construction on the Rust side when JS omits fields
|
||||
- `Option<T>` makes fields optional in the TypeScript definition
|
||||
|
||||
### 4.5 Getter Properties (BiStreamResult)
|
||||
|
||||
```rust
|
||||
#[napi]
|
||||
pub struct BiStreamResult {
|
||||
send: SendStream,
|
||||
recv: RecvStream,
|
||||
}
|
||||
|
||||
#[napi]
|
||||
impl BiStreamResult {
|
||||
#[napi(getter)]
|
||||
pub fn get_send(&self) -> SendStream {
|
||||
self.send.clone()
|
||||
}
|
||||
|
||||
#[napi(getter)]
|
||||
pub fn get_recv(&self) -> RecvStream {
|
||||
self.recv.clone()
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Pattern:**
|
||||
- Use `#[napi(getter)]` to expose Rust struct fields as JS getter properties
|
||||
- The `get_` prefix is stripped: `get_send` → `.send` property in JS
|
||||
- Must return cloned values since the Rust struct owns the data
|
||||
|
||||
### 4.6 Type Mappings (Rust → TypeScript)
|
||||
|
||||
| Rust Type | TypeScript Type |
|
||||
|-----------|---------------|
|
||||
| `String` | `string` |
|
||||
| `u32`, `f64` | `number` |
|
||||
| `bool` | `boolean` |
|
||||
| `Option<T>` | `T \| null \| undefined` |
|
||||
| `Option<u32>` | `number \| null` |
|
||||
| `Vec<String>` | `Array<string>` |
|
||||
| `Buffer` (from napi) | `Buffer` |
|
||||
| `Result<T>` | `T` (throws on Err) |
|
||||
| `Result<Option<T>>` | `T \| null` |
|
||||
| `#[napi] struct` | `class` |
|
||||
| `#[napi(object)] struct` | `interface` |
|
||||
|
||||
---
|
||||
|
||||
## 5. TypeScript Type Definitions
|
||||
|
||||
### Auto-Generated `index.d.ts`
|
||||
|
||||
The `index.d.ts` file is **entirely auto-generated** by `napi build --platform`. It is not hand-written.
|
||||
|
||||
Generated output structure:
|
||||
```typescript
|
||||
/* auto-generated by NAPI-RS */
|
||||
/* eslint-disable */
|
||||
|
||||
export declare class BiStreamResult {
|
||||
get send(): SendStream
|
||||
get recv(): RecvStream
|
||||
}
|
||||
|
||||
export declare class Connection {
|
||||
remoteNodeId(): string
|
||||
alpn(): Buffer
|
||||
openBi(): Promise<BiStreamResult>
|
||||
openUni(): Promise<SendStream>
|
||||
// ... etc
|
||||
}
|
||||
|
||||
export declare class Endpoint {
|
||||
static create(): Promise<Endpoint>
|
||||
static createWithOptions(options: EndpointOptions): Promise<Endpoint>
|
||||
nodeId(): string
|
||||
// ... etc
|
||||
}
|
||||
|
||||
export interface EndpointOptions {
|
||||
alpns?: Array<string>
|
||||
secretKey?: string
|
||||
}
|
||||
|
||||
export declare function initLogging(level?: string | undefined | null): void
|
||||
```
|
||||
|
||||
**Key observations:**
|
||||
- `#[napi]` structs become `declare class` with methods
|
||||
- `#[napi(object)]` structs become `interface`
|
||||
- `#[napi(factory)]` methods become `static` methods
|
||||
- `#[napi(getter)]` methods become `get` accessor properties
|
||||
- Doc comments from Rust (`/// ...`) are preserved as JSDoc in the `.d.ts`
|
||||
- Async Rust functions correctly become `Promise<T>` return types
|
||||
- The `@ts-nocheck` and `/* eslint-disable */` in `index.js` suppress linting of generated code
|
||||
|
||||
### tsconfig.json
|
||||
|
||||
```json
|
||||
{
|
||||
"compilerOptions": {
|
||||
"lib": ["ESNext"],
|
||||
"target": "ESNext",
|
||||
"module": "Preserve",
|
||||
"moduleResolution": "bundler",
|
||||
"allowImportingTsExtensions": true,
|
||||
"verbatimModuleSyntax": true,
|
||||
"noEmit": true,
|
||||
"strict": true,
|
||||
"skipLibCheck": true
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
This is a development-only config — the package doesn't ship compiled TS, only `.d.ts` declarations.
|
||||
|
||||
---
|
||||
|
||||
## 6. Build Pipeline
|
||||
|
||||
### 6.1 Local Development Build
|
||||
|
||||
```bash
|
||||
bun install # Install @napi-rs/cli + deps
|
||||
bun run build # napi build --platform --release
|
||||
```
|
||||
|
||||
What `napi build --platform --release` does:
|
||||
1. Invokes `cargo build --release` to compile the Rust crate as `cdylib`
|
||||
2. Copies the resulting `.so`/`.dylib`/`.dll` to the project root as `{binaryName}.{platform}.node`
|
||||
- e.g., `iroh.linux-x64-gnu.node`
|
||||
3. Generates `index.d.ts` (TypeScript declarations) from the `#[napi]` annotated code
|
||||
4. Generates `index.js` (CommonJS loader) that resolves the correct `.node` file per platform
|
||||
|
||||
### 6.2 CI Build (Cross-Compilation)
|
||||
|
||||
The CI matrix builds for 4 targets:
|
||||
|
||||
| Host Runner | Target | Build Command |
|
||||
|-------------|--------|---------------|
|
||||
| `macos-latest` | `x86_64-apple-darwin` | `bun run build --target x86_64-apple-darwin` |
|
||||
| `windows-latest` | `x86_64-pc-windows-msvc` | `bun run build --target x86_64-pc-windows-msvc` |
|
||||
| `ubuntu-latest` | `x86_64-unknown-linux-gnu` | `bun run build --target x86_64-unknown-linux-gnu --use-napi-cross` |
|
||||
| `macos-latest` | `aarch64-apple-darwin` | `bun run build --target aarch64-apple-darwin` |
|
||||
|
||||
Cross-compilation notes:
|
||||
- **macOS → aarch64**: Uses `dtolnay/rust-toolchain` with `targets` parameter; macOS cross-compilation "just works" with Rust
|
||||
- **Linux x64**: Uses `--use-napi-cross` flag which downloads cross-compilation toolchains via `@napi-rs/cross-toolchain`
|
||||
- **Musl targets**: Would use `cargo-zigbuild` via Zig (configured in CI but not in the current target matrix)
|
||||
- **Windows**: Uses MSVC target; `.cargo/config.toml` sets `+crt-static` for static CRT linking
|
||||
|
||||
### 6.3 Artifacts and Publishing
|
||||
|
||||
1. Each matrix job uploads its `.node` file as a GitHub artifact
|
||||
2. The `publish` job:
|
||||
- Downloads all artifacts
|
||||
- Runs `napi create-npm-dirs` to create `npm/` directory structure
|
||||
- Runs `napi artifacts` to move `.node` files into per-platform npm packages under `npm/`
|
||||
- Publishes the main package + all platform packages to npm
|
||||
|
||||
### 6.4 The `index.js` Loader
|
||||
|
||||
The auto-generated `index.js` is ~580 lines and handles:
|
||||
- Platform detection (`process.platform` + `process.arch`)
|
||||
- musl vs glibc detection on Linux (tries filesystem, process report, and child process)
|
||||
- Resolution ordering: try local `.node` file first, then fall back to `@scope/pkg-platform` npm package
|
||||
- `NAPI_RS_NATIVE_LIBRARY_PATH` environment variable override for custom paths
|
||||
- WASI/WASM fallback for browser environments
|
||||
- Version mismatch checking between main package and platform packages
|
||||
|
||||
---
|
||||
|
||||
## 7. Error Handling Across the Rust/TS Boundary
|
||||
|
||||
### 7.1 Error Definition Pattern
|
||||
|
||||
```rust
|
||||
#[derive(Debug, Error)]
|
||||
pub enum IrohError {
|
||||
#[error("Bind error: {0}")]
|
||||
Bind(String),
|
||||
#[error("Connect error: {0}")]
|
||||
Connect(String),
|
||||
#[error("Stream error: {0}")]
|
||||
Stream(String),
|
||||
#[error("Parse error: {0}")]
|
||||
Parse(String),
|
||||
#[error("Connection closed: {0}")]
|
||||
ConnectionClosed(String),
|
||||
#[error("Internal error: {0}")]
|
||||
Internal(String),
|
||||
}
|
||||
```
|
||||
|
||||
### 7.2 Error Conversion
|
||||
|
||||
Two approaches are used:
|
||||
|
||||
**Approach 1: `From` trait implementation**
|
||||
```rust
|
||||
impl From<IrohError> for napi::Error {
|
||||
fn from(err: IrohError) -> Self {
|
||||
napi::Error::from_reason(err.to_string())
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Approach 2: Generic helper function (preferred for inline use)**
|
||||
```rust
|
||||
pub fn to_napi_error<E: std::fmt::Display>(err: E) -> napi::Error {
|
||||
napi::Error::from_reason(err.to_string())
|
||||
}
|
||||
```
|
||||
|
||||
### 7.3 Usage in Methods
|
||||
|
||||
```rust
|
||||
pub async fn connect(&self, addr: String, alpn: String) -> Result<Connection> {
|
||||
let endpoint_id: iroh::EndpointId = addr
|
||||
.parse()
|
||||
.map_err(|e| to_napi_error(format!("Invalid node ID: {e}")))?; // Add context
|
||||
|
||||
let conn = self.inner.connect(endpoint_id, alpn.as_bytes())
|
||||
.await
|
||||
.map_err(to_napi_error)?; // Simple conversion
|
||||
|
||||
Ok(Connection::new(conn))
|
||||
}
|
||||
```
|
||||
|
||||
**Key patterns:**
|
||||
- All Rust errors are converted to `napi::Error::from_reason(string)` — they become JS `Error` objects with a `.message` property
|
||||
- `thiserror` is used on the Rust side for structured error categories, but these are **flattened to strings** at the boundary
|
||||
- The `to_napi_error` helper is more flexible than the `From` impl because it works with any `Display` type
|
||||
- Context is added manually with `format!("Context: {e}")` wrapping, since the error types lose structure at the boundary
|
||||
- `Result<T>` in Rust maps to throwing in JS (no explicit `Result` type in TypeScript)
|
||||
|
||||
### 7.4 TypeScript Side
|
||||
|
||||
On the TS side, errors appear as standard JavaScript `Error` objects thrown from native code. There is no typed error handling — everything is a string message:
|
||||
|
||||
```typescript
|
||||
try {
|
||||
const conn = await endpoint.connect(addr, alpn);
|
||||
} catch (err) {
|
||||
// err is a standard Error with a message like "Connect error: connection refused"
|
||||
}
|
||||
```
|
||||
|
||||
**Critique:** This is a weak point of iroh-ts. Structured error information (error kind, codes) is lost at the boundary. For taskgraph-ts, we should consider:
|
||||
- Using `napi::Error::from_status_with_message()` with custom status codes
|
||||
- Or defining a custom error class on the JS side that parses the error string
|
||||
- Or using napi-rs's built-in `#[napi(ts_type)]` for more detailed type info
|
||||
|
||||
---
|
||||
|
||||
## 8. CI/CD and Cross-Compilation Setup
|
||||
|
||||
### 8.1 GitHub Actions Workflow
|
||||
|
||||
**Jobs:**
|
||||
1. **`lint`** (ubuntu-latest): `cargo fmt -- --check` + `cargo clippy`
|
||||
2. **`build`** (matrix): Cross-compile for 4 targets, upload `.node` artifacts
|
||||
3. **`publish`** (ubuntu-latest): Download artifacts, create npm dirs, publish to npm
|
||||
|
||||
**Key CI details:**
|
||||
- Bun is used instead of npm/yarn (`oven-sh/setup-bun@v1`)
|
||||
- Rust toolchain via `dtolnay/rust-toolchain@stable` with `components: clippy, rustfmt`
|
||||
- Cargo cache keyed by `target-host` pair
|
||||
- Zig + cargo-zigbuild installed conditionally for musl targets (though no musl targets in current matrix)
|
||||
- Version-based publishing: only publishes if the latest commit message matches a semver pattern
|
||||
- `npm config set provenance true` for supply chain security
|
||||
- Sequential `needs` dependency: `build` requires `lint`, `publish` requires both
|
||||
|
||||
### 8.2 Cross-Compilation Config
|
||||
|
||||
**`.cargo/config.toml`:**
|
||||
```toml
|
||||
[target.x86_64-pc-windows-msvc]
|
||||
rustflags = ["-C", "target-feature=+crt-static"]
|
||||
```
|
||||
|
||||
This ensures the Windows MSVC build statically links the C runtime, avoiding DLL dependency issues.
|
||||
|
||||
### 8.3 Platform Package Structure
|
||||
|
||||
When published, napi-rs creates a directory structure like:
|
||||
```
|
||||
npm/
|
||||
├── darwin-arm64/ → @rayhanadev/iroh-darwin-arm64
|
||||
├── darwin-x64/ → @rayhanadev/iroh-darwin-x64
|
||||
├── linux-x64-gnu/ → @rayhanadev/iroh-linux-x64-gnu
|
||||
└── win32-x64-msvc/ → @rayhanadev/iroh-win32-x64-msvc
|
||||
```
|
||||
|
||||
Each contains a minimal `package.json` with `os` + `cpu` constraints and the `.node` file.
|
||||
|
||||
---
|
||||
|
||||
## 9. napi-rs Version and Dependency Pinning
|
||||
|
||||
| Package | Version | Notes |
|
||||
|---------|---------|-------|
|
||||
| `napi` (Rust) | `3.0.0` | Core runtime crate |
|
||||
| `napi-derive` (Rust) | `3.0.0` | Procedural macros (must match napi major) |
|
||||
| `napi-build` (Rust) | `2` | Build script helper |
|
||||
| `@napi-rs/cli` (JS) | `^3.2.0` | CLI tool (resolved to 3.4.1 in lock) |
|
||||
|
||||
**Important:** `napi` and `napi-derive` MUST share the same major version. Version 3.x is the latest stable line.
|
||||
|
||||
The `features = ["async", "tokio_rt"]` on the `napi` crate are critical:
|
||||
- `async`: Enables `#[napi]` on async functions
|
||||
- `tokio_rt`: Uses the tokio runtime for async napi-rs tasks (there is also `napi4` feature for custom threads)
|
||||
|
||||
---
|
||||
|
||||
## 10. Patterns Worth Replicating and Avoiding
|
||||
|
||||
### Patterns to Replicate
|
||||
|
||||
1. **Arc wrapping for shared Rust types** — `Arc<InnerType>` for structs that are shared across the JS/Rust boundary. This is essential for correct lifetime management.
|
||||
|
||||
2. **`#[napi(factory)]` for async constructors** — Since JS constructors can't be async, use factory methods that return `Promise<T>`. The `Endpoint::create()` pattern is clean and idiomatic.
|
||||
|
||||
3. **Module-per-domain** — Separate files for each domain concept (endpoint, connection, stream, error). Clean separation of concerns.
|
||||
|
||||
4. **`#[napi(object)]` for options structs** — Clean mapping of Rust option structs to JS plain objects without class overhead.
|
||||
|
||||
5. **`to_napi_error` helper** — Generic error conversion function that works with any `Display` type. Much more ergonomic than writing `From` impls for every error type.
|
||||
|
||||
6. **Release profile optimization** — `lto = true` + `strip = "symbols"` significantly reduces binary size.
|
||||
|
||||
7. **Doc comments preserved** — Rust doc comments (`///`) flow through to TypeScript declarations, providing IDE autocomplete.
|
||||
|
||||
8. **`.gitattributes` marking generated files** — Using `linguist-detectable=false` for `index.js`, `index.d.ts`, and WASM files keeps GitHub language stats clean.
|
||||
|
||||
9. **Separate `impl` blocks** — Having a non-`#[napi]` `impl` block for the `new()` constructor and a `#[napi] impl` block for JS-exposed methods cleanly separates internal Rust API from external JS API.
|
||||
|
||||
10. **Platform-specific cargo config** — The `.cargo/config.toml` with `+crt-static` for Windows avoids runtime DLL issues.
|
||||
|
||||
### Patterns to Improve Upon
|
||||
|
||||
1. **Error handling is lossy** — All structured Rust errors are flattened to strings. Consider:
|
||||
- Custom napi error classes that preserve error categories
|
||||
- Error status codes using `napi::Error` with custom `Status` values
|
||||
- A richer error object on the JS side
|
||||
|
||||
2. **No test infrastructure** — The project has no unit tests. For taskgraph-ts, we should add:
|
||||
- Rust unit tests for internal logic
|
||||
- JS/TS integration tests that verify the napi boundary
|
||||
- CI test step that runs after build
|
||||
|
||||
3. **No TypeScript wrapper layer** — `index.d.ts` is purely auto-generated. Consider adding a hand-written `src/index.ts` that:
|
||||
- Re-exports from the native module
|
||||
- Adds convenience methods
|
||||
- Provides typed error classes
|
||||
- Adds input validation before calling native code
|
||||
|
||||
4. **Missing musl/Linux ARM64 targets** — Only 4 targets in the `napi` config. For broader compatibility, consider adding:
|
||||
- `aarch64-unknown-linux-gnu` (ARM Linux, AWS Graviton)
|
||||
- `x86_64-unknown-linux-musl` (Alpine Docker images)
|
||||
|
||||
5. **No `Cargo.lock` committed** — The `.gitignore` excludes `Cargo.lock`. For a binary crate (as opposed to a library), it's generally recommended to commit the lock file for reproducible builds. However, napi-rs projects often skip this since the CI builds from scratch.
|
||||
|
||||
6. **No `napi.config.js` or `napi.config.ts`** — Some larger napi-rs projects use a dedicated config file. For a small project, `package.json` `napi` section is fine, but it doesn't scale well.
|
||||
|
||||
7. **`Arc<Mutex<T>>` everywhere** — The SendStream/RecvStream use `tokio::sync::Mutex` even for read-only operations (like `id()`). Consider whether some operations could use `Arc<tokio::sync::RwLock>` or even `std::sync::RwLock` for non-async read paths.
|
||||
|
||||
8. **`init_logging` as a public API** — Exposing `tracing_subscriber::fmt().try_init()` is fragile: it can only succeed once, and multiple calls silently fail. Consider returning a boolean/result indicating whether initialization succeeded.
|
||||
|
||||
9. **No version consistency checks** — The Rust crate version (`0.1.0`) and npm package version (`0.1.1`) differ. This could cause confusion. Consider aligning them or using a script to keep them in sync.
|
||||
|
||||
10. **No `#[napi(custom_finalize)]`** — Missing custom finalize/destructor logic. If Rust objects need cleanup (e.g., flushing streams before drop), the default finalization may not be sufficient.
|
||||
|
||||
---
|
||||
|
||||
## Summary: Recommended Architecture for taskgraph-ts
|
||||
|
||||
Based on this research, here is a recommended approach:
|
||||
|
||||
### Cargo.toml
|
||||
```toml
|
||||
[package]
|
||||
name = "taskgraph"
|
||||
version = "0.1.0"
|
||||
edition = "2021"
|
||||
|
||||
[lib]
|
||||
crate-type = ["cdylib"]
|
||||
|
||||
[dependencies]
|
||||
napi-derive = "3.0.0"
|
||||
# taskgraph = "..." (the Rust library being wrapped)
|
||||
thiserror = "2"
|
||||
tracing = "0.1"
|
||||
|
||||
[dependencies.napi]
|
||||
version = "3.0.0"
|
||||
features = ["async", "tokio_rt"]
|
||||
|
||||
[build-dependencies]
|
||||
napi-build = "2"
|
||||
|
||||
[profile.release]
|
||||
lto = true
|
||||
strip = "symbols"
|
||||
```
|
||||
|
||||
### package.json
|
||||
```json
|
||||
{
|
||||
"name": "@alkdev/taskgraph",
|
||||
"main": "index.js",
|
||||
"types": "index.d.ts",
|
||||
"napi": {
|
||||
"binaryName": "taskgraph",
|
||||
"targets": [
|
||||
"x86_64-apple-darwin",
|
||||
"aarch64-apple-darwin",
|
||||
"x86_64-unknown-linux-gnu",
|
||||
"aarch64-unknown-linux-gnu",
|
||||
"x86_64-pc-windows-msvc"
|
||||
]
|
||||
},
|
||||
"scripts": {
|
||||
"build": "napi build --platform --release",
|
||||
"build:debug": "napi build --platform",
|
||||
"artifacts": "napi artifacts",
|
||||
"prepublishOnly": "napi prepublish -t npm",
|
||||
"version": "napi version"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@napi-rs/cli": "^3.2.0"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Rust Source Layout
|
||||
```
|
||||
src/
|
||||
├── lib.rs # Module declarations + any free functions
|
||||
├── error.rs # Error types + to_napi_error helper
|
||||
├── graph.rs # Graph class (#[napi] struct + impl)
|
||||
├── task.rs # Task class
|
||||
├── executor.rs # Executor class (async factory methods)
|
||||
└── types.rs # Shared types, #[napi(object)] structs
|
||||
```
|
||||
|
||||
### Key Conventions to Adopt
|
||||
1. `Arc<T>` for all napi struct inner types
|
||||
2. `#[napi(factory)]` for async constructors
|
||||
3. `#[napi(object)]` for option/config structs
|
||||
4. `to_napi_error` generic helper for error conversion
|
||||
5. Preserve doc comments for auto-generated `.d.ts`
|
||||
6. Always match `napi` and `napi-derive` major versions
|
||||
7. Use `tokio::sync::Mutex` for async interior mutability
|
||||
8. Add `.cargo/config.toml` with Windows CRT static linking
|
||||
9. Use `bun` as the JS runtime (fast, napi-rs compatible)
|
||||
10. Commit generated `index.js` and `index.d.ts` to the repo
|
||||
Reference in New Issue
Block a user