From ba8c382d531f14bd440d99f232f1a4dd90e10ef0 Mon Sep 17 00:00:00 2001 From: "glm-5.1" Date: Thu, 23 Apr 2026 10:30:40 +0000 Subject: [PATCH] Add architecture doc and research reports for taskgraph_ts napi wrapper --- docs/architecture.md | 266 ++++ docs/research/iroh_ts_reference.md | 718 +++++++++++ docs/research/napi_rs_framework.md | 1270 +++++++++++++++++++ docs/research/taskgraph_rust_source.md | 1140 +++++++++++++++++ docs/research/taskgraph_ts_current_state.md | 172 +++ 5 files changed, 3566 insertions(+) create mode 100644 docs/architecture.md create mode 100644 docs/research/iroh_ts_reference.md create mode 100644 docs/research/napi_rs_framework.md create mode 100644 docs/research/taskgraph_rust_source.md create mode 100644 docs/research/taskgraph_ts_current_state.md diff --git a/docs/architecture.md b/docs/architecture.md new file mode 100644 index 0000000..486d1b4 --- /dev/null +++ b/docs/architecture.md @@ -0,0 +1,266 @@ +# taskgraph_ts Architecture + +> Status: draft — this is a starting point for iteration, not a final spec. + +## Why This Exists + +The taskgraph CLI is useful but requires bash access. In agent systems, bash + untrusted data sources (web content, academic papers, etc.) is a security risk akin to SQL injection — adversarial content can instruct agents to exfiltrate data or take harmful actions through the shell. We've seen this in practice: researchers hiding prompt injections in academic papers using Unicode steganography that bypassed review systems. + +Rather than restricting which agents get bash access and hoping nothing goes wrong, we expose the graph and cost-benefit operations as a library callable as a native tool — no shell involved. + +The same math and graph code also serves a different purpose: implementation agents that *do* have bash access can call these operations directly as tools rather than shelling out to the CLI, which is faster and avoids argument parsing issues. + +## Core Principle + +**The graph algorithms and cost-benefit math are the value.** Markdown parsing, file discovery, and CLI output formatting are input/output concerns that belong to the caller, not to this library. + +This crate is a standalone implementation. It copies the essential logic from `/workspace/@alkimiadev/taskgraph` but does not depend on it. The upstream CLI continues to exist for human use and offline analysis. + +## What We Copy (Rewritten) + +From the taskgraph Rust crate, adapted for library use: + +- **DependencyGraph** — all algorithms: + - Cycle detection (`hasCycles`, `findCycles`) + - Topological sort (`topologicalOrder`) + - Dependency queries (`dependencies`, `dependents`) + - Parallel groups (`parallelGroups`) + - Critical path (`criticalPath`) + - Weighted critical path (`weightedCriticalPath`) + - Bottleneck detection (`bottlenecks`) + - DOT export (`toDot`) + +- **Categorical enums** with their numeric methods: + - `TaskScope` — `costEstimate()` (1.0–5.0) + - `TaskRisk` — `successProbability()` (0.50–0.98) + - `TaskImpact` — `weight()` (1.0–3.0) + - `TaskLevel` — labeling, no numeric method currently + +- **Cost-benefit math**: + - `calculateTaskEv(p, scopeCost, impactWeight)` — expected value calculation + - Risk-path finding (highest cumulative risk path) + - Decomposition flagging (tasks that should be broken down) + +- **Error types** — cleaned up, mapped to typed JS error classes + +## What We Don't Copy + +- `Task` / `TaskFrontmatter` — markdown-specific structs +- `TaskCollection` / directory scanning — filesystem discovery +- `Config` / `.taskgraph.toml` — CLI configuration +- `clap` command definitions — CLI dispatch +- `gray_matter` / `serde_yaml` — markdown frontmatter parsing +- `chrono` — timestamp handling (the graph doesn't need it) + +These may be added later as a opt-in feature but are not part of the core. + +## Input Model + +The graph must be constructable from multiple sources. The DB consumer sends query results. The markdown consumer sends parsed files. The programmatic consumer builds incrementally. + +### TaskInput + +The universal input shape for a task: + +```typescript +interface TaskInput { + id: string + name?: string + dependsOn: string[] + scope?: "single" | "narrow" | "moderate" | "broad" | "system" + risk?: "trivial" | "low" | "medium" | "high" | "critical" + impact?: "isolated" | "component" | "phase" | "project" + level?: "planning" | "decomposition" | "implementation" | "review" | "research" + priority?: "low" | "medium" | "high" | "critical" +} +``` + +All categorical fields are optional. The graph algorithms only need `id` and `dependsOn`. The cost-benefit and weighted-path functions need the categorical fields. + +### DependencyEdge + +For constructing from DB rows where tasks and edges are separate: + +```typescript +interface DependencyEdge { + from: string // prerequisite task id + to: string // dependent task id +} +``` + +### Construction Paths + +```typescript +// 1. From DB query results (the primary use case) +const graph = DependencyGraph.fromRecords(tasks, edges) + +// 2. Incremental construction (programmatic) +const graph = new DependencyGraph() +graph.addTask("a") +graph.addTask("b") +graph.addDependency("a", "b") + +// 3. From TaskInput array (convenience, extracts id + dependsOn) +const graph = DependencyGraph.fromTasks(tasks) +``` + +Path 1 is what alkhub's coordinator agent will use most. Path 2 is for programmatic/testing use. Path 3 is where the categorical data attaches for weighted analysis. + +### Markdown Support + +Not in scope for v1. If needed later, it would be a separate module or package that parses markdown files into `TaskInput[]` and then feeds them into the graph. The parsing is straightforward (gray_matter/YAML) and is better kept outside the core library — callers with bash access already have the CLI. + +## API Surface (Draft) + +### DependencyGraph + +The primary class. Wraps petgraph internally. + +```typescript +class DependencyGraph { + // Construction + static fromRecords(tasks: TaskInput[], edges: DependencyEdge[]): DependencyGraph + static fromTasks(tasks: TaskInput[]): DependencyGraph + addTask(id: string): void + addDependency(from: string, to: string): void + + // Queries + hasCycles(): boolean + findCycles(): string[][] + topologicalOrder(): string[] | null + dependencies(taskId: string): string[] + dependents(taskId: string): string[] + taskCount(): number + + // Analysis + parallelGroups(): string[][] + criticalPath(): string[] + weightedCriticalPath(weights: Record): string[] + bottlenecks(): Array<[string, number]> + + // Export + toDot(): string +} +``` + +### Categorical Enums + +Exposed as JS string enums with numeric accessor methods: + +```typescript +// The enum values are strings (matching the DB and frontmatter conventions) +// The numeric methods are exposed as standalone functions + +function scopeCostEstimate(scope: TaskScope): number // 1.0–5.0 +function riskSuccessProbability(risk: TaskRisk): number // 0.50–0.98 +function impactWeight(impact: TaskImpact): number // 1.0–3.0 + +// Or as static methods on enum-like objects — TBD +``` + +The exact JS API shape (string union types vs enum objects vs namespace + functions) is open for iteration. The Rust side is unambiguous — these are enums with `match`-based methods. + +### Cost-Benefit Analysis + +```typescript +function calculateTaskEv( + probability: number, + scopeCost: number, + impactWeight: number +): number + +function riskPath( + graph: DependencyGraph, + tasks: TaskInput[] +): string[] + +function shouldDecompose(task: TaskInput): boolean +``` + +### Error Types + +Typed JS error classes instead of flat strings: + +```typescript +class TaskgraphError extends Error {} +class TaskNotFoundError extends TaskgraphError { taskId: string } +class CircularDependencyError extends TaskgraphError { cycles: string[][] } +class InvalidInputError extends TaskgraphError { field: string; message: string } +``` + +This lets callers distinguish error types programmatically — important when an agent needs to decide how to recover. + +## Project Structure + +``` +taskgraph_ts/ +├── Cargo.toml # Rust crate config (cdylib, napi deps) +├── build.rs # napi-build setup +├── package.json # npm package config + napi targets +├── src/ +│ ├── lib.rs # Crate root, module declarations +│ ├── graph.rs # DependencyGraph + all algorithms +│ ├── enums.rs # TaskScope, TaskRisk, TaskImpact, TaskLevel +│ ├── cost_benefit.rs # EV calculation, risk-path, decompose +│ ├── input.rs # TaskInput, DependencyEdge napi structs +│ ├── error.rs # Error types + napi conversion +│ └── napi_types.rs # napi attribute structs (or inline) +├── index.js # Auto-generated by napi build +├── index.d.ts # Auto-generated TypeScript declarations +└── ts/ + └── index.ts # Hand-written wrapper layer + # - Re-exports from native module + # - Typed error classes + # - Input validation + # - Convenience helpers +``` + +The `ts/` wrapper layer is where we add ergonomic value on top of the raw napi bindings. This is also where we'd handle things like making `weightedCriticalPath` accept either a weight map or a `TaskInput[]` that carries categorical data. + +## Build & Distribution + +- **Rust crate**: `taskgraph_ts` (or `taskgraph_napi` — TBD), compiled as `cdylib` +- **napi-rs**: v3, with `async` + `tokio_rt` features (for future async I/O operations) +- **Targets**: macOS x64/ARM64, Linux x64/ARM64, Windows x64 +- **Package**: `@alkdev/taskgraph` on npm, with per-platform optional dependencies +- **TypeScript**: Auto-generated `.d.ts` from napi macros, plus hand-written wrapper + +## Dependencies (Rust) + +| Crate | Purpose | +|-------|---------| +| `napi` / `napi-derive` | Node-API bindings | +| `napi-build` | Build script configuration | +| `petgraph` | Directed graph data structure and algorithms | +| `thiserror` | Error derive macros | +| `serde` + `serde_json` | Serialization for napi object conversion | + +Notably absent: `gray_matter`, `serde_yaml`, `chrono`, `clap`. The core doesn't need them. + +## Open Questions + +These will be resolved through iteration, not upfront design: + +1. **Package name** — `@alkdev/taskgraph` vs something else. The npm namespace and whether to match the CLI crate name. + +2. **Async boundary** — `DependencyGraph` operations are CPU-bound and synchronous. Should we offer `Promise`-wrapped versions anyway for non-blocking use in event-loop-sensitive environments? Or is that the caller's job? + +3. **Task metadata on the graph** — Currently `DependencyGraph` only stores task IDs as node weights. For `weightedCriticalPath` and `riskPath`, the weight data comes from `TaskInput[]` passed alongside the graph. Should the graph store metadata (scope, risk, etc.) on nodes so callers don't need to pass it separately? + +4. **Risk-path return type** — Should `riskPath` return just `string[]` (the path) or include the cumulative risk score? The CLI command outputs both. + +5. **Enum representation in napi** — `#[napi(string_enum)]` gives JS string values, which aligns with the DB enum values. Or we could use `#[napi(enum)]` for numeric values with a TS mapping. String enums match the existing ecosystem better. + +6. **Relationship to alkhub's graphology** — The alkhub spec currently uses graphology for runtime graph ops in the hub. This napi module could replace graphology for the algorithms it supports (and petgraph is faster). Or they could coexist — graphology for the coordinator's runtime queries, taskgraph_napi for deep analysis. Needs iteration to see what feels right. + +7. **How `shouldDecompose` works** — The CLI `decompose` command checks if `scope > moderate` or `risk > medium`. Should this be a simple function, a method on a `Task` object, or configurable thresholds? + +8. **Markdown feature flag** — Whether to ever add an optional markdown-parsing feature, and if so whether it lives in this crate or a companion package. + +## Threat Model Context + +For background on the security motivation: + +- **Attack vector**: Agents with bash access processing untrusted content (web pages, academic papers, API responses) can be manipulated via prompt injection. This includes subtle attacks like Unicode steganography hiding instructions in otherwise legitimate content. +- **Defense in depth**: The instruction firewall project (using Ternary Bonsai 1.7b classifier to detect instruction-bearing content) addresses detection. This project addresses the other side — reducing the blast radius by removing bash as a requirement for analysis operations. +- **Tool-based access**: Instead of `taskgraph --json list | jq`, agents call `taskgraph.listTasks()` as a tool. No shell, no injection surface, no data exfiltration path through bash. \ No newline at end of file diff --git a/docs/research/iroh_ts_reference.md b/docs/research/iroh_ts_reference.md new file mode 100644 index 0000000..d17ca88 --- /dev/null +++ b/docs/research/iroh_ts_reference.md @@ -0,0 +1,718 @@ +# iroh-ts Reference Implementation Research Report + +**Project:** [iroh-ts](https://github.com/rayhanadev/iroh-ts) — TypeScript/JavaScript bindings for iroh (peer-to-peer QUIC networking) +**Date:** 2026-04-23 +**Purpose:** Reference for building taskgraph-ts napi-rs wrapper + +--- + +## 1. Project Structure + +iroh-ts is a minimal, single-crate napi-rs project with a flat layout: + +``` +iroh-ts/ +├── .cargo/ +│ └── config.toml # Cross-compilation rustflags (Windows static CRT) +├── .github/ +│ └── workflows/ +│ └── CI.yml # GitHub Actions CI/CD +├── .gitattributes # EOL normalization + mark generated files +├── .gitignore # Node + Rust ignores +├── build.rs # napi-build setup (1 line) +├── Cargo.toml # Rust crate config +├── bun.lock # Bun lockfile +├── index.d.ts # Auto-generated TypeScript declarations +├── index.js # Auto-generated JS loader (platform resolution) +├── package.json # npm package config +├── src/ +│ ├── lib.rs # Crate root: module declarations + init_logging +│ ├── error.rs # IrohError enum + conversion helpers +│ ├── endpoint.rs # Endpoint class (napi-wrapped) +│ ├── connection.rs # Connection class + BiStreamResult struct +│ └── stream.rs # SendStream + RecvStream classes +├── tsconfig.json # TypeScript config (bundler mode, noEmit) +└── README.md +``` + +**Key observations:** +- No separate `npm/` or `crates/` directory — it is a single Rust crate mapped to a single npm package +- Generated `index.js` and `index.d.ts` are committed to the repo (auto-generated by `napi build`) +- Platform-specific `.node` binaries are not committed; they are built in CI and published as separate npm packages +- No test directory exists in the repo (tests would use the `ava` framework listed in bun.lock) + +--- + +## 2. Cargo.toml Configuration + +```toml +[package] +name = "iroh" +version = "0.1.0" +edition = "2021" + +[lib] +crate-type = ["cdylib"] # Critical: produces a C dynamic library for Node + +[dependencies] +napi-derive = "3.0.0" # Procedural macro crate (#[napi] attribute) +iroh = "0.95.1" # The wrapped Rust library +iroh-base = "0.95.1" +bytes = "1.11.0" +thiserror = "2.0.17" +tracing = "0.1.41" + +[dependencies.napi] +version = "3.0.0" +features = ["async", "tokio_rt"] # Required for async Rust functions exposed to JS + +[dependencies.tokio] +version = "1.48.0" +features = ["sync"] # Only sync feature (tokio_rt comes from napi) + +[dependencies.tracing-subscriber] +version = "0.3.20" +features = ["env-filter"] + +[build-dependencies] +napi-build = "2" # Build script helper + +[profile.release] +lto = true # Link-Time Optimization for smaller binaries +strip = "symbols" # Strip debug symbols for smaller binaries +``` + +**Key takeaways:** +- `crate-type = ["cdylib"]` — **mandatory** for napi-rs; produces `.node` shared library +- `napi` and `napi-derive` are separate crates with matched major versions (both 3.0.0) +- The `async` + `tokio_rt` features on napi are essential if you expose async Rust functions +- `napi-build = "2"` is a minimal build dependency that just calls `napi_build::setup()` +- Release profile uses LTO + symbol stripping — standard practice for napi-rs to minimize binary size + +--- + +## 3. package.json Configuration + +```json +{ + "name": "@rayhanadev/iroh", + "version": "0.1.1", + "main": "index.js", + "types": "index.d.ts", + + "napi": { + "binaryName": "iroh", + "targets": [ + "x86_64-pc-windows-msvc", + "x86_64-apple-darwin", + "x86_64-unknown-linux-gnu", + "aarch64-apple-darwin" + ] + }, + + "scripts": { + "artifacts": "napi artifacts", + "build": "napi build --platform --release", + "build:debug": "napi build --platform", + "prepublishOnly": "napi prepublish -t npm", + "preversion": "napi build --platform && git add .", + "version": "napi version" + }, + + "devDependencies": { + "@emnapi/core": "^1.5.0", + "@emnapi/runtime": "^1.5.0", + "@napi-rs/cli": "^3.2.0", + "@tybys/wasm-util": "^0.10.0" + }, + + "peerDependencies": { + "typescript": "^5" + }, + + "optionalDependencies": { + "@rayhanadev/iroh-win32-x64-msvc": "0.1.0", + "@rayhanadev/iroh-darwin-x64": "0.1.0", + "@rayhanadev/iroh-linux-x64-gnu": "0.1.0", + "@rayhanadev/iroh-darwin-arm64": "0.1.0" + } +} +``` + +**Key takeaways:** +- The `napi` section in package.json is the core napi-rs config — it defines: + - `binaryName`: the base name for the `.node` file (must match the crate name) + - `targets`: which platform triples to build for (used by CI and `napi` CLI) +- `--platform` flag on `napi build` makes it generate the correct platform-specific filename +- `optionalDependencies` lists the per-platform npm packages — npm auto-selects the right one + - These are separate npm packages published from the `npm/` directory created by `napi create-npm-dirs` + - Names follow the pattern: `@scope/{binaryName}-{platform}-{arch}-{abi}` +- `@emnapi/core` and `@emnapi/runtime` are for WASI/WASM support (optional, for browser targets) +- No `@napi-rs/client` dependency — the CLI (`@napi-rs/cli`) is the only napi-rs JS dependency needed +- The `prepublishOnly` script runs `napi prepublish` which handles version alignment across platform packages + +--- + +## 4. Rust Function/Class Export Patterns via napi-rs Macros + +### 4.1 Crate Root (`lib.rs`) + +```rust +#![deny(clippy::all)] + +use napi_derive::napi; + +mod connection; +mod endpoint; +mod error; +mod stream; + +pub use connection::*; +pub use endpoint::*; +pub use error::*; +pub use stream::*; + +#[napi] +pub fn init_logging(level: Option) { + let filter = level.unwrap_or_else(|| "info".to_string()); + let _ = tracing_subscriber::fmt().with_env_filter(filter).try_init(); +} +``` + +**Patterns:** +- `#![deny(clippy::all)]` — strict linting at crate level +- Module-based organization with `pub use` re-exports from the crate root +- Free functions annotated with `#[napi]` become top-level JS exports +- `Option` maps to `string | undefined | null` in TypeScript + +### 4.2 Class Pattern (Endpoint) + +```rust +#[napi] +pub struct Endpoint { + inner: Arc, // Wrap the Rust type in Arc for shared ownership +} + +#[napi] +impl Endpoint { + #[napi(factory)] // Static factory method → Endpoint.create() + pub async fn create() -> Result { ... } + + #[napi(factory)] // With options → Endpoint.createWithOptions(opts) + pub async fn create_with_options(options: EndpointOptions) -> Result { ... } + + #[napi] // Instance method → endpoint.nodeId() + pub fn node_id(&self) -> String { ... } + + #[napi] // Instance method → endpoint.connect(addr, alpn) + pub async fn connect(&self, addr: String, alpn: String) -> Result { ... } +} +``` + +**Key patterns:** +1. **Struct + impl separation:** The struct definition gets `#[napi]`, AND a separate `#[napi] impl` block exposes methods +2. **`Arc`** pattern: Rust objects shared across JS/Rust boundary are wrapped in `Arc`. This is critical because: + - Node.js is single-threaded but napi-rs creates a reference that may outlive the original scope + - The Arc allows the JS wrapper to own a reference-counted Rust object +3. **`#[napi(factory)]`**: Creates a static factory method instead of a constructor. This is the idiomatic way to handle async construction (since WASM/native constructors can't be async) +4. **`snake_case` Rust → `camelCase` JS**: napi-rs auto-converts `node_id` to `nodeId`, `create_with_options` to `createWithOptions` +5. **Return types:** `Result` maps to JS `T | Error` (throws on Err) + +### 4.3 Struct with Mutex for Interior Mutability (SendStream/RecvStream) + +```rust +#[derive(Clone)] +#[napi] +pub struct SendStream { + inner: Arc>, // Arc> for mutable access +} + +#[napi] +impl SendStream { + pub async fn write(&self, data: Buffer) -> Result { + let mut stream = self.inner.lock().await; // tokio Mutex for async-compatible locking + let written = stream.write(&data).await.map_err(to_napi_error)?; + Ok(written as u32) + } +} +``` + +**Key patterns:** +- `Arc>` with tokio's async Mutex — required when async methods need mutable access +- `#[derive(Clone)]` because the `Arc>` is cheaply cloneable +- `Buffer` type from `napi::bindgen_prelude::*` maps to Node.js `Buffer` +- Manual `as u32` cast because napi-rs doesn't support `usize` (platform-dependent size) + +### 4.4 Object Struct (Options) + +```rust +#[napi(object)] +#[derive(Default)] +pub struct EndpointOptions { + pub alpns: Option>, + pub secret_key: Option, +} +``` + +**Pattern:** +- `#[napi(object)]` makes this a plain JS object (not a class) — fields become properties +- `#[derive(Default)]` allows easy construction on the Rust side when JS omits fields +- `Option` makes fields optional in the TypeScript definition + +### 4.5 Getter Properties (BiStreamResult) + +```rust +#[napi] +pub struct BiStreamResult { + send: SendStream, + recv: RecvStream, +} + +#[napi] +impl BiStreamResult { + #[napi(getter)] + pub fn get_send(&self) -> SendStream { + self.send.clone() + } + + #[napi(getter)] + pub fn get_recv(&self) -> RecvStream { + self.recv.clone() + } +} +``` + +**Pattern:** +- Use `#[napi(getter)]` to expose Rust struct fields as JS getter properties +- The `get_` prefix is stripped: `get_send` → `.send` property in JS +- Must return cloned values since the Rust struct owns the data + +### 4.6 Type Mappings (Rust → TypeScript) + +| Rust Type | TypeScript Type | +|-----------|---------------| +| `String` | `string` | +| `u32`, `f64` | `number` | +| `bool` | `boolean` | +| `Option` | `T \| null \| undefined` | +| `Option` | `number \| null` | +| `Vec` | `Array` | +| `Buffer` (from napi) | `Buffer` | +| `Result` | `T` (throws on Err) | +| `Result>` | `T \| null` | +| `#[napi] struct` | `class` | +| `#[napi(object)] struct` | `interface` | + +--- + +## 5. TypeScript Type Definitions + +### Auto-Generated `index.d.ts` + +The `index.d.ts` file is **entirely auto-generated** by `napi build --platform`. It is not hand-written. + +Generated output structure: +```typescript +/* auto-generated by NAPI-RS */ +/* eslint-disable */ + +export declare class BiStreamResult { + get send(): SendStream + get recv(): RecvStream +} + +export declare class Connection { + remoteNodeId(): string + alpn(): Buffer + openBi(): Promise + openUni(): Promise + // ... etc +} + +export declare class Endpoint { + static create(): Promise + static createWithOptions(options: EndpointOptions): Promise + nodeId(): string + // ... etc +} + +export interface EndpointOptions { + alpns?: Array + secretKey?: string +} + +export declare function initLogging(level?: string | undefined | null): void +``` + +**Key observations:** +- `#[napi]` structs become `declare class` with methods +- `#[napi(object)]` structs become `interface` +- `#[napi(factory)]` methods become `static` methods +- `#[napi(getter)]` methods become `get` accessor properties +- Doc comments from Rust (`/// ...`) are preserved as JSDoc in the `.d.ts` +- Async Rust functions correctly become `Promise` return types +- The `@ts-nocheck` and `/* eslint-disable */` in `index.js` suppress linting of generated code + +### tsconfig.json + +```json +{ + "compilerOptions": { + "lib": ["ESNext"], + "target": "ESNext", + "module": "Preserve", + "moduleResolution": "bundler", + "allowImportingTsExtensions": true, + "verbatimModuleSyntax": true, + "noEmit": true, + "strict": true, + "skipLibCheck": true + } +} +``` + +This is a development-only config — the package doesn't ship compiled TS, only `.d.ts` declarations. + +--- + +## 6. Build Pipeline + +### 6.1 Local Development Build + +```bash +bun install # Install @napi-rs/cli + deps +bun run build # napi build --platform --release +``` + +What `napi build --platform --release` does: +1. Invokes `cargo build --release` to compile the Rust crate as `cdylib` +2. Copies the resulting `.so`/`.dylib`/`.dll` to the project root as `{binaryName}.{platform}.node` + - e.g., `iroh.linux-x64-gnu.node` +3. Generates `index.d.ts` (TypeScript declarations) from the `#[napi]` annotated code +4. Generates `index.js` (CommonJS loader) that resolves the correct `.node` file per platform + +### 6.2 CI Build (Cross-Compilation) + +The CI matrix builds for 4 targets: + +| Host Runner | Target | Build Command | +|-------------|--------|---------------| +| `macos-latest` | `x86_64-apple-darwin` | `bun run build --target x86_64-apple-darwin` | +| `windows-latest` | `x86_64-pc-windows-msvc` | `bun run build --target x86_64-pc-windows-msvc` | +| `ubuntu-latest` | `x86_64-unknown-linux-gnu` | `bun run build --target x86_64-unknown-linux-gnu --use-napi-cross` | +| `macos-latest` | `aarch64-apple-darwin` | `bun run build --target aarch64-apple-darwin` | + +Cross-compilation notes: +- **macOS → aarch64**: Uses `dtolnay/rust-toolchain` with `targets` parameter; macOS cross-compilation "just works" with Rust +- **Linux x64**: Uses `--use-napi-cross` flag which downloads cross-compilation toolchains via `@napi-rs/cross-toolchain` +- **Musl targets**: Would use `cargo-zigbuild` via Zig (configured in CI but not in the current target matrix) +- **Windows**: Uses MSVC target; `.cargo/config.toml` sets `+crt-static` for static CRT linking + +### 6.3 Artifacts and Publishing + +1. Each matrix job uploads its `.node` file as a GitHub artifact +2. The `publish` job: + - Downloads all artifacts + - Runs `napi create-npm-dirs` to create `npm/` directory structure + - Runs `napi artifacts` to move `.node` files into per-platform npm packages under `npm/` + - Publishes the main package + all platform packages to npm + +### 6.4 The `index.js` Loader + +The auto-generated `index.js` is ~580 lines and handles: +- Platform detection (`process.platform` + `process.arch`) +- musl vs glibc detection on Linux (tries filesystem, process report, and child process) +- Resolution ordering: try local `.node` file first, then fall back to `@scope/pkg-platform` npm package +- `NAPI_RS_NATIVE_LIBRARY_PATH` environment variable override for custom paths +- WASI/WASM fallback for browser environments +- Version mismatch checking between main package and platform packages + +--- + +## 7. Error Handling Across the Rust/TS Boundary + +### 7.1 Error Definition Pattern + +```rust +#[derive(Debug, Error)] +pub enum IrohError { + #[error("Bind error: {0}")] + Bind(String), + #[error("Connect error: {0}")] + Connect(String), + #[error("Stream error: {0}")] + Stream(String), + #[error("Parse error: {0}")] + Parse(String), + #[error("Connection closed: {0}")] + ConnectionClosed(String), + #[error("Internal error: {0}")] + Internal(String), +} +``` + +### 7.2 Error Conversion + +Two approaches are used: + +**Approach 1: `From` trait implementation** +```rust +impl From for napi::Error { + fn from(err: IrohError) -> Self { + napi::Error::from_reason(err.to_string()) + } +} +``` + +**Approach 2: Generic helper function (preferred for inline use)** +```rust +pub fn to_napi_error(err: E) -> napi::Error { + napi::Error::from_reason(err.to_string()) +} +``` + +### 7.3 Usage in Methods + +```rust +pub async fn connect(&self, addr: String, alpn: String) -> Result { + let endpoint_id: iroh::EndpointId = addr + .parse() + .map_err(|e| to_napi_error(format!("Invalid node ID: {e}")))?; // Add context + + let conn = self.inner.connect(endpoint_id, alpn.as_bytes()) + .await + .map_err(to_napi_error)?; // Simple conversion + + Ok(Connection::new(conn)) +} +``` + +**Key patterns:** +- All Rust errors are converted to `napi::Error::from_reason(string)` — they become JS `Error` objects with a `.message` property +- `thiserror` is used on the Rust side for structured error categories, but these are **flattened to strings** at the boundary +- The `to_napi_error` helper is more flexible than the `From` impl because it works with any `Display` type +- Context is added manually with `format!("Context: {e}")` wrapping, since the error types lose structure at the boundary +- `Result` in Rust maps to throwing in JS (no explicit `Result` type in TypeScript) + +### 7.4 TypeScript Side + +On the TS side, errors appear as standard JavaScript `Error` objects thrown from native code. There is no typed error handling — everything is a string message: + +```typescript +try { + const conn = await endpoint.connect(addr, alpn); +} catch (err) { + // err is a standard Error with a message like "Connect error: connection refused" +} +``` + +**Critique:** This is a weak point of iroh-ts. Structured error information (error kind, codes) is lost at the boundary. For taskgraph-ts, we should consider: +- Using `napi::Error::from_status_with_message()` with custom status codes +- Or defining a custom error class on the JS side that parses the error string +- Or using napi-rs's built-in `#[napi(ts_type)]` for more detailed type info + +--- + +## 8. CI/CD and Cross-Compilation Setup + +### 8.1 GitHub Actions Workflow + +**Jobs:** +1. **`lint`** (ubuntu-latest): `cargo fmt -- --check` + `cargo clippy` +2. **`build`** (matrix): Cross-compile for 4 targets, upload `.node` artifacts +3. **`publish`** (ubuntu-latest): Download artifacts, create npm dirs, publish to npm + +**Key CI details:** +- Bun is used instead of npm/yarn (`oven-sh/setup-bun@v1`) +- Rust toolchain via `dtolnay/rust-toolchain@stable` with `components: clippy, rustfmt` +- Cargo cache keyed by `target-host` pair +- Zig + cargo-zigbuild installed conditionally for musl targets (though no musl targets in current matrix) +- Version-based publishing: only publishes if the latest commit message matches a semver pattern +- `npm config set provenance true` for supply chain security +- Sequential `needs` dependency: `build` requires `lint`, `publish` requires both + +### 8.2 Cross-Compilation Config + +**`.cargo/config.toml`:** +```toml +[target.x86_64-pc-windows-msvc] +rustflags = ["-C", "target-feature=+crt-static"] +``` + +This ensures the Windows MSVC build statically links the C runtime, avoiding DLL dependency issues. + +### 8.3 Platform Package Structure + +When published, napi-rs creates a directory structure like: +``` +npm/ +├── darwin-arm64/ → @rayhanadev/iroh-darwin-arm64 +├── darwin-x64/ → @rayhanadev/iroh-darwin-x64 +├── linux-x64-gnu/ → @rayhanadev/iroh-linux-x64-gnu +└── win32-x64-msvc/ → @rayhanadev/iroh-win32-x64-msvc +``` + +Each contains a minimal `package.json` with `os` + `cpu` constraints and the `.node` file. + +--- + +## 9. napi-rs Version and Dependency Pinning + +| Package | Version | Notes | +|---------|---------|-------| +| `napi` (Rust) | `3.0.0` | Core runtime crate | +| `napi-derive` (Rust) | `3.0.0` | Procedural macros (must match napi major) | +| `napi-build` (Rust) | `2` | Build script helper | +| `@napi-rs/cli` (JS) | `^3.2.0` | CLI tool (resolved to 3.4.1 in lock) | + +**Important:** `napi` and `napi-derive` MUST share the same major version. Version 3.x is the latest stable line. + +The `features = ["async", "tokio_rt"]` on the `napi` crate are critical: +- `async`: Enables `#[napi]` on async functions +- `tokio_rt`: Uses the tokio runtime for async napi-rs tasks (there is also `napi4` feature for custom threads) + +--- + +## 10. Patterns Worth Replicating and Avoiding + +### Patterns to Replicate + +1. **Arc wrapping for shared Rust types** — `Arc` for structs that are shared across the JS/Rust boundary. This is essential for correct lifetime management. + +2. **`#[napi(factory)]` for async constructors** — Since JS constructors can't be async, use factory methods that return `Promise`. The `Endpoint::create()` pattern is clean and idiomatic. + +3. **Module-per-domain** — Separate files for each domain concept (endpoint, connection, stream, error). Clean separation of concerns. + +4. **`#[napi(object)]` for options structs** — Clean mapping of Rust option structs to JS plain objects without class overhead. + +5. **`to_napi_error` helper** — Generic error conversion function that works with any `Display` type. Much more ergonomic than writing `From` impls for every error type. + +6. **Release profile optimization** — `lto = true` + `strip = "symbols"` significantly reduces binary size. + +7. **Doc comments preserved** — Rust doc comments (`///`) flow through to TypeScript declarations, providing IDE autocomplete. + +8. **`.gitattributes` marking generated files** — Using `linguist-detectable=false` for `index.js`, `index.d.ts`, and WASM files keeps GitHub language stats clean. + +9. **Separate `impl` blocks** — Having a non-`#[napi]` `impl` block for the `new()` constructor and a `#[napi] impl` block for JS-exposed methods cleanly separates internal Rust API from external JS API. + +10. **Platform-specific cargo config** — The `.cargo/config.toml` with `+crt-static` for Windows avoids runtime DLL issues. + +### Patterns to Improve Upon + +1. **Error handling is lossy** — All structured Rust errors are flattened to strings. Consider: + - Custom napi error classes that preserve error categories + - Error status codes using `napi::Error` with custom `Status` values + - A richer error object on the JS side + +2. **No test infrastructure** — The project has no unit tests. For taskgraph-ts, we should add: + - Rust unit tests for internal logic + - JS/TS integration tests that verify the napi boundary + - CI test step that runs after build + +3. **No TypeScript wrapper layer** — `index.d.ts` is purely auto-generated. Consider adding a hand-written `src/index.ts` that: + - Re-exports from the native module + - Adds convenience methods + - Provides typed error classes + - Adds input validation before calling native code + +4. **Missing musl/Linux ARM64 targets** — Only 4 targets in the `napi` config. For broader compatibility, consider adding: + - `aarch64-unknown-linux-gnu` (ARM Linux, AWS Graviton) + - `x86_64-unknown-linux-musl` (Alpine Docker images) + +5. **No `Cargo.lock` committed** — The `.gitignore` excludes `Cargo.lock`. For a binary crate (as opposed to a library), it's generally recommended to commit the lock file for reproducible builds. However, napi-rs projects often skip this since the CI builds from scratch. + +6. **No `napi.config.js` or `napi.config.ts`** — Some larger napi-rs projects use a dedicated config file. For a small project, `package.json` `napi` section is fine, but it doesn't scale well. + +7. **`Arc>` everywhere** — The SendStream/RecvStream use `tokio::sync::Mutex` even for read-only operations (like `id()`). Consider whether some operations could use `Arc` or even `std::sync::RwLock` for non-async read paths. + +8. **`init_logging` as a public API** — Exposing `tracing_subscriber::fmt().try_init()` is fragile: it can only succeed once, and multiple calls silently fail. Consider returning a boolean/result indicating whether initialization succeeded. + +9. **No version consistency checks** — The Rust crate version (`0.1.0`) and npm package version (`0.1.1`) differ. This could cause confusion. Consider aligning them or using a script to keep them in sync. + +10. **No `#[napi(custom_finalize)]`** — Missing custom finalize/destructor logic. If Rust objects need cleanup (e.g., flushing streams before drop), the default finalization may not be sufficient. + +--- + +## Summary: Recommended Architecture for taskgraph-ts + +Based on this research, here is a recommended approach: + +### Cargo.toml +```toml +[package] +name = "taskgraph" +version = "0.1.0" +edition = "2021" + +[lib] +crate-type = ["cdylib"] + +[dependencies] +napi-derive = "3.0.0" +# taskgraph = "..." (the Rust library being wrapped) +thiserror = "2" +tracing = "0.1" + +[dependencies.napi] +version = "3.0.0" +features = ["async", "tokio_rt"] + +[build-dependencies] +napi-build = "2" + +[profile.release] +lto = true +strip = "symbols" +``` + +### package.json +```json +{ + "name": "@alkdev/taskgraph", + "main": "index.js", + "types": "index.d.ts", + "napi": { + "binaryName": "taskgraph", + "targets": [ + "x86_64-apple-darwin", + "aarch64-apple-darwin", + "x86_64-unknown-linux-gnu", + "aarch64-unknown-linux-gnu", + "x86_64-pc-windows-msvc" + ] + }, + "scripts": { + "build": "napi build --platform --release", + "build:debug": "napi build --platform", + "artifacts": "napi artifacts", + "prepublishOnly": "napi prepublish -t npm", + "version": "napi version" + }, + "devDependencies": { + "@napi-rs/cli": "^3.2.0" + } +} +``` + +### Rust Source Layout +``` +src/ +├── lib.rs # Module declarations + any free functions +├── error.rs # Error types + to_napi_error helper +├── graph.rs # Graph class (#[napi] struct + impl) +├── task.rs # Task class +├── executor.rs # Executor class (async factory methods) +└── types.rs # Shared types, #[napi(object)] structs +``` + +### Key Conventions to Adopt +1. `Arc` for all napi struct inner types +2. `#[napi(factory)]` for async constructors +3. `#[napi(object)]` for option/config structs +4. `to_napi_error` generic helper for error conversion +5. Preserve doc comments for auto-generated `.d.ts` +6. Always match `napi` and `napi-derive` major versions +7. Use `tokio::sync::Mutex` for async interior mutability +8. Add `.cargo/config.toml` with Windows CRT static linking +9. Use `bun` as the JS runtime (fast, napi-rs compatible) +10. Commit generated `index.js` and `index.d.ts` to the repo diff --git a/docs/research/napi_rs_framework.md b/docs/research/napi_rs_framework.md new file mode 100644 index 0000000..d4e322f --- /dev/null +++ b/docs/research/napi_rs_framework.md @@ -0,0 +1,1270 @@ +# NAPI-RS Framework Research Report + +> Comprehensive research on the napi-rs project based on the source at `/workspace/napi-rs` +> and supplementary documentation from https://napi.rs + +--- + +## Table of Contents + +1. [Project Structure and Key Packages/Crates](#1-project-structure-and-key-packagescrates) +2. [Setting Up a New napi-rs Project](#2-setting-up-a-new-napi-rs-project) +3. [Core Patterns for Exposing Rust to JavaScript](#3-core-patterns-for-exposing-rust-to-javascript) +4. [Result/Option Types and Error Propagation](#4-resultoption-types-and-error-propagation) +5. [Async Support (Promises, async functions)](#5-async-support-promises-async-functions) +6. [Build System Configuration](#6-build-system-configuration) +7. [@napi-rs/cli Tool and Multi-Platform Builds](#7-napi-rscli-tool-and-multi-platform-builds) +8. [Thread-Safe Function Patterns (tsfn)](#8-thread-safe-function-patterns-tsfn) +9. [Serde/Serialization with napi-rs Types](#9-serdeserialization-with-napi-rs-types) +10. [Version Compatibility Notes](#10-version-compatibility-notes) + +--- + +## 1. Project Structure and Key Packages/Crates + +The napi-rs repository at `/workspace/napi-rs` is a monorepo using **Cargo workspaces** (Rust) and **Yarn workspaces** (JavaScript). The root `Cargo.toml` defines the workspace members. + +### Rust Crates (`/crates/`) + +| Crate | Path | Version | Purpose | +|-------|------|---------|---------| +| **`napi`** | `crates/napi/` | 3.8.5 | Main runtime library. Provides the high-level Node-API bindings, type conversions, error types, async runtime, thread-safe functions, and all `bindgen_prelude` types. | +| **`napi-sys`** | `crates/sys/` | 3.2.1 | Low-level FFI bindings. Raw `napi_*` C function declarations and type definitions. Uses `libloading` for dynamic symbol resolution. | +| **`napi-derive`** | `crates/macro/` | 3.5.4 | Procedural macro crate. Provides the `#[napi]` attribute macro that is the primary way to expose Rust code to JavaScript. | +| **`napi-derive-backend`** | `crates/backend/` | 5.0.3 | Code generation backend for `napi-derive`. Handles AST parsing, Rust-to-JS codegen, and TypeScript type definition generation. | +| **`napi-build`** | `crates/build/` | 2.3.1 | Build script utilities. Called from `build.rs` to configure linker flags for each platform (macOS dynamic_lookup, Android, WASI, Windows GNU). | + +### JavaScript Packages + +| Package | Path | Version | Purpose | +|---------|------|---------|---------| +| **`@napi-rs/cli`** | `cli/` | 3.6.2 | CLI tool for scaffolding, building, packaging, and publishing napi-rs projects. | +| **`@examples/napi`** | `examples/napi/` | (private) | Comprehensive test suite showcasing all napi-rs features. | + +### Key Source Files + +- `/crates/napi/src/lib.rs` -- Main library entry; re-exports modules, defines `bindgen_prelude` +- `/crates/napi/src/error.rs` -- `Error` struct, `Result`, `JsError`/`JsTypeError`/`JsRangeError` +- `/crates/napi/src/threadsafe_function.rs` -- `ThreadsafeFunction` implementation (869 lines) +- `/crates/napi/src/tokio_runtime.rs` -- Tokio runtime management and `execute_tokio_future` +- `/crates/napi/src/task.rs` -- `Task` and `ScopedTask` traits for async work on libuv threads +- `/crates/napi/src/async_work.rs` -- `AsyncWorkPromise` for libuv-based async tasks +- `/crates/napi/src/bindgen_runtime/` -- Core trait implementations: `ToNapiValue`, `FromNapiValue`, `TypeName`, `ValidateNapiValue`, class registration, module registration, iterator support +- `/crates/backend/src/typegen.rs` -- TypeScript `.d.ts` generation logic (981 lines) +- `/crates/macro/src/lib.rs` -- `#[napi]`, `#[module_init]`, `#[module_exports]` proc macros +- `/crates/build/src/lib.rs` -- Platform-specific linker configuration + +--- + +## 2. Setting Up a New napi-rs Project + +### Recommended: `napi new` (Scaffolding) + +The `@napi-rs/cli` provides a `new` command that generates a fully configured project: + +```sh +napi new [options] +``` + +**Available options:** + +| Option | CLI Flag | Default | Description | +|--------|----------|---------|-------------| +| `path` | `` | -- | Directory where project is created | +| `name` | `--name,-n` | directory name | Project name | +| `minNodeApiVersion` | `--min-node-api,-v` | 4 | Minimum N-API version | +| `packageManager` | `--package-manager` | yarn | Package manager (yarn 4.x only for now) | +| `license` | `--license,-l` | MIT | License | +| `targets` | `--targets,-t` | [] | Compilation targets | +| `enableDefaultTargets` | `--enable-default-targets` | true | Enable default platform targets | +| `enableAllTargets` | `--enable-all-targets` | false | Enable all platform targets | +| `enableTypeDef` | `--enable-type-def` | true | Auto-generate TypeScript definitions | +| `enableGithubActions` | `--enable-github-actions` | true | Generate GitHub Actions CI workflow | +| `testFramework` | `--test-framework` | ava | JS test framework (ava only for now) | + +**Example:** + +```sh +napi new ./my-addon --name my-addon --min-node-api 4 +``` + +### Alternative: package-template + +The GitHub repository [napi-rs/package-template](https://github.com/napi-rs/package-template) is the canonical template referenced in the README. + +### Minimal Manual Setup + +If not using scaffolding, a minimal napi-rs project requires: + +1. **`Cargo.toml`** with `crate-type = ["cdylib"]` +2. **`build.rs`** calling `napi_build::setup()` +3. **`package.json`** with `@napi-rs/cli` devDependency and napi config +4. Rust source with `#[napi]` annotated functions + +(See Section 6 for full configuration details.) + +--- + +## 3. Core Patterns for Exposing Rust to JavaScript + +### 3.1 Functions + +The `#[napi]` attribute on functions exposes them as JavaScript functions: + +```rust +use napi::bindgen_prelude::*; +use napi_derive::napi; + +#[napi] +pub fn fibonacci(n: u32) -> u32 { + match n { + 1 | 2 => 1, + _ => fibonacci(n - 1) + fibonacci(n - 2), + } +} +``` + +**Key behaviors:** +- Function name is converted to camelCase in JavaScript (e.g., `fibonacci` stays `fibonacci`, `my_func` becomes `myFunc`) +- JSDoc comments (`///`) on Rust functions become TypeScript documentation +- The `#[napi(js_name = "...")]` attribute allows custom JavaScript naming + +### 3.2 Structs as JavaScript Classes + +There are two patterns for structs: + +#### a) Struct with `#[napi]` -- JavaScript Class + +```rust +#[napi] +pub struct Animal { + #[napi(readonly)] + pub kind: Kind, + name: String, + optional_value: Option, +} + +#[napi] +impl Animal { + #[napi(constructor)] + pub fn new(kind: Kind, name: String) -> Self { + Animal { kind, name, optional_value: None } + } + + #[napi(factory)] + pub fn with_kind(kind: Kind) -> Self { + Animal { kind, name: "Default".to_owned(), optional_value: None } + } + + #[napi(getter)] + pub fn get_name(&self) -> &str { self.name.as_str() } + + #[napi(setter)] + pub fn set_name(&mut self, name: String) { self.name = name; } + + #[napi(getter, js_name = "type")] + pub fn kind(&self) -> Kind { self.kind } + + #[napi] + pub fn whoami(&self) -> String { /* ... */ } + + #[napi] + pub fn get_dog_kind() -> Kind { Kind::Dog } // static method +} +``` + +**Key attributes on struct:** +- `#[napi(constructor)]` -- All public fields become constructor parameters; a default constructor is generated +- `#[napi(js_name = "Assets")]` -- Rename in JavaScript +- `#[napi(custom_finalize)]` -- Enable `ObjectFinalize` trait for custom cleanup + +**Key attributes on impl methods:** +- `#[napi(constructor)]` -- Mark as constructor +- `#[napi(factory)]` -- Static factory method +- `#[napi(getter)]` / `#[napi(setter)]` -- Define getters/setters +- `#[napi(js_name = "type")]` -- Custom JS name +- `#[napi(ts_arg_type = "...")]` -- Override TypeScript arg type +- `#[napi(skip_typescript)]` -- Exclude from TypeScript definitions +- `#[napi(writable = false)]` -- Make property read-only in JS +- `#[napi(catch_unwind)]` -- Catch panics and convert to JS errors + +#### b) Struct with `#[napi(object)]` -- JavaScript Plain Object (Interface) + +```rust +#[napi(object)] +struct AllOptionalObject { + pub name: Option, + pub age: Option, +} +``` + +**Key attributes:** +- `#[napi(object)]` -- Generate as a TypeScript interface, not a class +- `#[napi(object, object_to_js = false)]` -- Only deserialize from JS (input-only) +- `#[napi(object, object_from_js = false)]` -- Only serialize to JS (output-only) +- `#[napi(object, use_nullable = true)]` -- Generate `nullable` TypeScript types instead of `optional` +- `#[napi(ts_type = "object")]` -- Override TypeScript field type +- `#[napi(js_name = "customField")]` -- Rename field in JS + +### 3.3 Enums + +#### a) Numeric Enums (default) + +```rust +#[napi] +#[derive(Debug, Clone, Copy)] +pub enum Kind { + Dog, // 0 + Cat, // 1 + Duck, // 2 +} +``` + +Custom discriminant values with step resolution: + +```rust +#[napi] +pub enum CustomNumEnum { + One = 1, // 1 + Two, // 2 + Three = 3, // 3 + Four, // 4 + Six = 6, + Eight = 8, + Nine, // 9 + Ten, // 10 +} +``` + +#### b) String Enums + +```rust +#[napi(string_enum)] +pub enum Status { + Pristine, // "Pristine" + Loading, // "Loading" + Ready, // "Ready" +} + +#[napi(string_enum = "lowercase")] +pub enum StringEnum { + VariantOne, // "variantone" + VariantTwo, // "varianttwo" +} + +#[napi(string_enum)] +pub enum CustomStringEnum { + #[napi(value = "my-custom-value")] + Foo, // "my-custom-value" + Bar, // "Bar" + Baz, // "Baz" +} +``` + +#### c) Structured Enums (Tagged Unions) + +```rust +#[napi(discriminant = "type2")] +pub enum StructuredKind { + Hello, + Greeting { name: String }, + Birthday { name: String, age: u8 }, + Tuple(u32, u32), +} + +#[napi(discriminant_case = "lowercase")] +pub enum StructuredKindLowercase { + Hello, // { type2: "hello" } + Greeting { name: String }, // { type2: "greeting", name: "..." } +} +``` + +### 3.4 Transparent Types + +Newtype pattern that wraps an existing JS-compatible type: + +```rust +#[napi(transparent)] +struct MyVec(Vec>); + +#[napi] +fn get_my_vec() -> MyVec { + MyVec(vec![Either::A(42), Either::B("a string".to_owned())]) +} +``` + +### 3.5 Constants + +```rust +#[napi] +/// This is a const +pub const DEFAULT_COST: u32 = 12; + +#[napi(skip_typescript)] +pub const TYPE_SKIPPED_CONST: u32 = 12; +``` + +### 3.6 Callbacks (Fn/FnMut/FnOnce traits) + +```rust +#[napi] +pub fn get_cwd Result<()>>(callback: T) { + callback(std::env::current_dir().unwrap().to_string_lossy().to_string()).unwrap(); +} + +#[napi] +pub fn test_callback(callback: T) -> Result<()> +where + T: Fn(String) -> Result<()>, +{ + callback(std::env::current_dir()?.to_string_lossy().to_string()) +} +``` + +### 3.7 TypeScript Customization + +The `#[napi]` macro supports several attributes for TypeScript generation: + +- `ts_args_type` -- Override all function argument types +- `ts_return_type` -- Override function return type +- `ts_generic_types` -- Add generic type parameters +- `ts_type` -- Override field type in objects +- `skip_typescript` -- Skip generation in `.d.ts` +- `js_name` -- Rename in JS/TS + +```rust +#[napi( + ts_generic_types = "T", + ts_args_type = "functionInput: () => T | Promise, callback: (err: Error | null, result: T) => void", + ts_return_type = "T | Promise" +)] +fn callback_return_promise<'env>(/* ... */) -> Result> { /* ... */ } +``` + +### 3.8 Either Types + +napi-rs provides `Either`, `Either3`, and `Either4` for union types: + +```rust +#[napi] +fn either_string_or_number(input: Either) -> u32 { + match input { + Either::A(s) => s.len() as u32, + Either::B(n) => n, + } +} + +#[napi] +fn receive_class_or_number(either: Either) -> u32 { + match either { + Either::A(n) => n + 1, + Either::B(_) => 100, + } +} + +#[napi] +pub async fn promise_in_either(input: Either>) -> Result { + match input { + Either::A(a) => Ok(a > 10), + Either::B(b) => { + let r = b.await?; + Ok(r > 10) + } + } +} +``` + +### 3.9 Module Exports / Module Init + +Custom module initialization can be done with `#[napi(module_exports)]` or `#[napi_derive::module_init]`: + +```rust +#[napi(module_exports)] +pub fn exports(mut export: Object) -> Result<()> { + let symbol = Symbol::for_desc("NAPI_RS_SYMBOL"); + export.set_named_property("NAPI_RS_SYMBOL", symbol)?; + Ok(()) +} + +// For custom tokio runtime: +#[napi_derive::module_init] +fn init() { + let rt = tokio::runtime::Builder::new_multi_thread() + .enable_all() + .build() + .unwrap(); + create_custom_tokio_runtime(rt); +} +``` + +--- + +## 4. Result/Option Types and Error Propagation + +### 4.1 The `Result` Type + +napi-rs defines its own `Result` type alias: + +```rust +pub type Result = std::result::Result>; +``` + +Returning `Result` from a `#[napi]` function causes errors to be thrown as JavaScript errors. Returning `Ok(value)` resolves normally. + +### 4.2 The `Error` Struct + +```rust +pub struct Error = Status> { + pub status: S, + pub reason: String, + pub cause: Option>, + maybe_raw: sys::napi_ref, + maybe_env: sys::napi_env, +} +``` + +**Creating errors:** + +```rust +// Standard error with status code +Err(Error::new(Status::InvalidArg, "Manual Error".to_owned())) + +// Error with cause (chained errors) +let mut err = Error::new(Status::GenericFailure, "Manual Error".to_owned()); +err.set_cause(Error::new(Status::InvalidArg, "Inner Error".to_owned())); +Err(err) + +// From a reason string (uses GenericFailure status) +Error::from_reason("something went wrong") +``` + +### 4.3 Error Type Variants + +napi-rs provides specialized error types for different JavaScript error classes: + +| Rust Type | JavaScript Equivalent | +|-----------|----------------------| +| `JsError` | `Error` | +| `JsTypeError` | `TypeError` | +| `JsRangeError` | `RangeError` | +| `JsSyntaxError` (napi9) | `SyntaxError` | + +### 4.4 Custom Error Status + +You can define custom error status types by implementing `AsRef` and `From`: + +```rust +pub enum CustomError { + NapiError(Error), + Panic, +} + +impl AsRef for CustomError { + fn as_ref(&self) -> &str { + match self { + CustomError::Panic => "Panic", + CustomError::NapiError(e) => e.status.as_ref(), + } + } +} + +#[napi] +pub fn custom_status_code() -> Result<(), CustomError> { + Err(Error::new(CustomError::Panic, "don't panic")) +} +``` + +### 4.5 Automatic Conversions from std::io::Error and Others + +The `Error` struct implements `From` for common Rust error types: + +- `From` +- `From` +- `From` (with `error_anyhow` feature) +- `From` (with `serde-json` feature) + +This allows using `?` operator to propagate standard Rust errors. + +### 4.6 Catch Unwind + +The `#[napi(catch_unwind)]` attribute catches Rust panics and converts them to JavaScript errors: + +```rust +#[napi(catch_unwind)] +pub fn panic() { + panic!("Don't panic"); +} +``` + +### 4.7 Option Type Mapping + +| Rust `Option` | JavaScript | +|------------------|-----------| +| `Option` | `number \| undefined` | +| `Option` | `string \| undefined` | +| `None` | `undefined` (or `null` depending on context) | +| `Option` where Struct is a class | `Struct \| null` | + +Null and Undefined are explicit types: + +```rust +#[napi] +fn return_null() -> Null { Null } + +#[napi] +fn return_undefined() -> Undefined {} +``` + +For objects, `#[napi(object, use_nullable = true)]` generates `nullable` TS types (e.g., `string | null`) instead of `optional` (e.g., `string | undefined`). + +--- + +## 5. Async Support (Promises, async functions) + +### 5.1 Async Functions (tokio-based) + +With the `async` feature enabled, any `async fn` annotated with `#[napi]` returns a JavaScript `Promise`: + +```rust +// Cargo.toml: napi = { version = "3", features = ["async"] } + +#[napi] +async fn read_file_async(path: String) -> Result { + Ok(tokio::fs::read(path).await?.into()) +} + +#[napi] +async fn async_multi_two(arg: u32) -> Result { + tokio::task::spawn(async move { Ok(arg * 2) }) + .await + .unwrap() +} +``` + +**Requirements:** +- Enable the `async` feature (which includes `tokio_rt`) +- napi-rs manages a Tokio runtime internally (multi-threaded by default) +- Async errors are properly propagated as rejected promises + +### 5.2 Custom Tokio Runtime + +You can provide a custom Tokio runtime configuration: + +```rust +#[napi_derive::module_init] +fn init() { + let rt = tokio::runtime::Builder::new_multi_thread() + .enable_all() + .thread_stack_size(32 * 1024 * 1024) + .build() + .unwrap(); + napi::bindgen_prelude::create_custom_tokio_runtime(rt); +} +``` + +### 5.3 `async_runtime` Attribute + +Run a synchronous function inside the async runtime context: + +```rust +#[napi(async_runtime)] +pub fn within_async_runtime_if_available() { + tokio::spawn(async { + println!("within_runtime_if_available"); + }); +} +``` + +### 5.4 AsyncTask (libuv thread pool) + +For CPU-intensive work that should run on the libuv thread pool rather than the Tokio runtime, use the `Task` trait: + +```rust +pub struct DelaySum(u32, u32); + +#[napi] +impl napi::Task for DelaySum { + type Output = u32; + type JsValue = u32; + + fn compute(&mut self) -> Result { + // Runs on libuv thread pool + std::thread::sleep(std::time::Duration::from_millis(100)); + Ok(self.0 + self.1) + } + + fn resolve(&mut self, _env: napi::Env, output: Self::Output) -> Result { + Ok(output) + } + + fn finally(self, _env: napi::Env) -> Result<()> { + Ok(()) + } +} + +#[napi] +pub fn with_abort_controller(a: u32, b: u32, signal: AbortSignal) -> AsyncTask { + AsyncTask::with_signal(DelaySum(a, b), signal) +} +``` + +**Task vs ScopedTask:** +- `Task` -- `resolve` takes `Env` by value; output and JsValue must be `'static` +- `ScopedTask<'task>` -- `resolve` takes `&'task Env`; JsValue can borrow from the env (e.g., `BufferSlice<'task>`, `Array<'task>`) + +### 5.5 Promise and PromiseRaw + +Work directly with JavaScript Promise objects: + +```rust +// Await a Promise from JavaScript +#[napi] +pub async fn async_plus_100(p: Promise) -> Result { + let v = p.await?; + Ok(v + 100) +} + +// Create resolved/rejected promises +#[napi] +pub fn create_resolved_promise<'env>(env: &'env Env, value: u32) -> Result> { + PromiseRaw::resolve(env, value) +} + +#[napi] +pub fn create_rejected_promise<'env>(env: &'env Env, message: String) -> Result> { + PromiseRaw::reject(env, Error::from_reason(message)) +} + +// Chain .then/.catch/.finally +#[napi] +pub fn call_then_on_promise(input: PromiseRaw) -> Result> { + input.then(|v| Ok(format!("{}", v.value))) +} + +#[napi] +pub fn call_catch_on_promise(input: PromiseRaw<'_, u32>) -> Result> { + input.catch(|e: CallbackContext| Ok(e.value)) +} +``` + +### 5.6 Spawning Futures Manually + +```rust +env.spawn_future(async move { Ok(some_value) }) +env.spawn_future_with_callback(async move { Ok(some_value) }, |env, val| { + env.create_string(format!("{}", val)) +}) +``` + +--- + +## 6. Build System Configuration + +### 6.1 Cargo.toml + +```toml +[package] +name = "my-addon" +edition = "2021" + +[lib] +crate-type = ["cdylib"] + +[dependencies] +napi = { version = "3", features = ["napi4"] } # or more features +napi-derive = "3" + +[build-dependencies] +napi-build = "2" + +[profile.release] +lto = true +``` + +**Critical:** `crate-type = ["cdylib"]` is required so cargo builds a C-style shared library that Node can dynamically load. + +### 6.2 Feature Flags on `napi` Crate + +| Feature | Requires | Description | +|---------|----------|-------------| +| `napi1` through `napi10` | incremental | Progressive N-API version support | +| `async` / `tokio_rt` | napi4 | Tokio runtime + async fn support | +| `serde-json` | -- | serde Serialize/Deserialize for JS <-> Rust | +| `serde-json-ordered` | serde-json | Preserves key order with `serde_json/preserve_order` | +| `latin1` | -- | Latin1 string decoding via `encoding_rs` | +| `chrono_date` | napi5 | `chrono::DateTime` support | +| `error_anyhow` | -- | `From` for napi::Error | +| `web_stream` | napi4, tokio_rt | Web Streams API support | +| `deferred_trace` | napi4 | Deferred stack trace | +| `object_indexmap` | -- | `indexmap::IndexMap` support | +| `tracing` | -- | `tracing` crate integration | +| `dyn-symbols` | -- | Dynamic symbol resolution (default) | +| `compat-mode` | -- | Deprecated types/traits for v2 compatibility | +| `noop` | -- | Generate no-op code (for testing) | + +**Common feature combinations:** +- Minimal: `napi = { version = "3", default-features = false, features = ["napi4"] }` +- With async: `napi = { version = "3", features = ["napi4", "async"] }` +- Full: `napi = { version = "3", features = ["full"] }` (includes napi10, async, serde-json, experimental, chrono_date, latin1) + +### 6.3 napi-derive Features + +| Feature | Description | +|---------|-------------| +| `type-def` (default) | Auto-generate TypeScript `.d.ts` definitions | +| `strict` (default) | Strict type checking in macro expansion | +| `compat-mode` | Deprecated attribute compatibility | +| `tracing` | Tracing in macro expansion | +| `noop` | Generate no-op code | + +### 6.4 build.rs + +Every napi-rs project must have a `build.rs` that calls `napi_build::setup()`: + +```rust +fn main() { + napi_build::setup(); +} +``` + +**What `napi_build::setup()` does:** +1. Sets `cargo:rerun-if-env-changed` for various NAPI environment variables +2. On **macOS**: adds linker flags `-Wl,-undefined,dynamic_lookup` (needed because Node.js symbols are resolved at runtime, not link time) +3. On **Windows (GNU)**: configures GNU toolchain linker settings +4. On **Android/WASI**: platform-specific setup +5. On **GNU libc / FreeBSD**: adds `-Wl,-z,nodelete` to prevent DSO unloading issues with pthread_key_create destructors + +### 6.5 package.json (napi config) + +```json +{ + "name": "my-addon", + "devDependencies": { + "@napi-rs/cli": "^3.0.0" + }, + "napi": { + "name": "jarvis", + "binaryName": "example", + "wasm": { + "initialMemory": 16384, + "browser": { "fs": true, "buffer": true } + }, + "dtsHeader": "type MaybePromise = T | Promise", + "dtsHeaderFile": "./dts-header.d.ts", + "targets": ["wasm32-wasip1-threads"] + }, + "scripts": { + "build": "napi build --release", + "build:debug": "napi build", + "build:platform": "napi build --platform" + } +} +``` + +The `napi.name` / `napi.binaryName` field determines the output `.node` file name. The naming convention converts hyphens to underscores: `my-addon` -> `my_addon.node`. + +--- + +## 7. @napi-rs/cli Tool and Multi-Platform Builds + +### 7.1 CLI Version: 3.6.2 + +The `@napi-rs/cli` package (at `/workspace/napi-rs/cli/`) is the primary tooling for building, packaging, and releasing napi-rs projects. + +### 7.2 Commands Overview + +| Command | Description | +|---------|-------------| +| `napi new` | Create a new project with pre-configured boilerplate | +| `napi build` | Build the napi-rs project | +| `napi create-npm-dirs` | Create per-platform npm package directories | +| `napi artifacts` | Copy build artifacts from GitHub Actions | +| `napi rename` | Rename the project | +| `napi universalize` | Combine binaries into a universal binary (e.g., macOS arm64 + x64) | +| `napi version` | Update version across per-platform npm packages | +| `napi pre-publish` | Prepare packages for npm publish | + +### 7.3 Build Command Details + +```sh +napi build [--release] [--platform] [--target ] [options] +``` + +**Key options:** + +| Option | Description | +|--------|-------------| +| `--target,-t` | Target triple (passed to `cargo build --target`) | +| `--platform` | Add platform triple suffix (e.g., `.linux-x64-gnu.node`) | +| `--release,-r` | Build in release mode | +| `--js` | Path/filename for generated JS binding | +| `--no-js` | Disable JS binding generation | +| `--dts` | Path/filename for generated TypeScript definitions | +| `--strip,-s` | Strip debug symbols for minimum file size | +| `--cross-compile,-x` | Cross-compile using `cargo-xwin` / `cargo-zigbuild` | +| `--use-cross` | Use [cross](https://github.com/cross-rs/cross) instead of `cargo` | +| `--use-napi-cross` | Use `@napi-rs/cross-toolchain` for Linux ARM/ARM64/x64 GNU | +| `--watch,-w` | Watch and rebuild continuously | +| `--features,-F` | Space-separatedCargo features to activate | +| `--output-dir,-o` | Output directory for built files | +| `--esm` | Generate ESM JS binding instead of CJS | +| `--const-enum` | Generate const enums in TypeScript | + +### 7.4 Multi-Platform Build Workflow + +The typical cross-compilation workflow: + +1. **Build** for each target: + ```sh + napi build --platform --target x86_64-unknown-linux-gnu + napi build --platform --target aarch64-apple-darwin + napi build --platform --target x86_64-pc-windows-msvc + ``` + +2. **Universalize** (macOS only -- combine arm64 + x64 into a single universal binary): + ```sh + napi universalize + ``` + +3. **Create npm directories** for per-platform packages: + ```sh + napi create-npm-dirs + ``` + +4. **Artifacts** -- Collect `.node` files from GitHub Actions CI: + ```sh + napi artifacts --output-dir ./artifacts + ``` + +5. **Pre-publish** -- Copy platform-specific `.node` files into per-platform npm packages: + ```sh + napi pre-publish --npm-dir npm + ``` + +### 7.5 Supported Platforms + +| Platform | Architectures | Variants | +|----------|---------------|----------| +| Windows | x64, x86, arm64 | MSVC, GNU | +| macOS | x64, aarch64 | - | +| Linux | x64, aarch64, arm, riscv64, s390x, ppc64le, loong64 | gnu, musl, gnueabihf, musleabihf | +| FreeBSD | x64 | - | +| Android | aarch64, armv7 | - | + +--- + +## 8. Thread-Safe Function Patterns (tsfn) + +Thread-safe functions (TSFNs) are the mechanism for calling JavaScript from background threads. They are the cornerstone of async and concurrent interop in napi-rs. + +### 8.1 Basic Usage + +```rust +use napi::threadsafe_function::{ThreadsafeFunction, ThreadsafeFunctionCallMode}; + +#[napi] +pub fn call_threadsafe_function( + tsfn: Arc>, +) -> Result<()> { + for n in 0..100 { + let tsfn = tsfn.clone(); + thread::spawn(move || { + tsfn.call(Ok(n), ThreadsafeFunctionCallMode::NonBlocking); + }); + } + Ok(()) +} +``` + +### 8.2 Type Parameters + +```rust +ThreadsafeFunction< + T: 'static, // Input type (what you send from Rust) + Return: FromNapiValue, // Return type from JS callback + CallJsBackArgs: JsValuesTupleIntoVec, // Arguments passed to JS callback + ErrorStatus: AsRef + From, // Custom error status + const CalleeHandled: bool, // Whether callback follows (err, result) pattern + const Weak: bool, // Weak reference (won't prevent event loop exit) + const MaxQueueSize: usize, // Max queued calls (0 = unlimited) +> +``` + +### 8.3 Caller-Handled vs Callee-Handled + +**Callee-handled** (`CalleeHandled = true`, default): Follows Node.js callback convention, prepending `null` as the first arg on success: + +```rust +// Callee-handled: JS receives (null, value) on success, (error, undefined) on failure +let tsfn: ThreadsafeFunction; +tsfn.call(Ok(42), ThreadsafeFunctionCallMode::NonBlocking); +``` + +**Caller-handled** (`CalleeHandled = false`): No error-first argument. On error, calls `napi_fatal_exception`: + +```rust +// Fatal mode: JS receives just (value), no error-first convention +let tsfn: ThreadsafeFunction; +tsfn.call(42, ThreadsafeFunctionCallMode::NonBlocking); +``` + +### 8.4 Async Call with Return Value + +```rust +#[napi] +pub async fn tsfn_return_promise(func: ThreadsafeFunction>) -> Result { + let val = func.call_async(Ok(1)).await?.await?; + Ok(val + 2) +} +``` + +### 8.5 Call with Return Value Callback + +```rust +#[napi] +pub fn tsfn_call_with_callback(tsfn: ThreadsafeFunction<(), String>) -> napi::Result<()> { + tsfn.call_with_return_value( + Ok(()), + ThreadsafeFunctionCallMode::NonBlocking, + |value: Result, _| { + let value = value.expect("Failed to retrieve value from JS"); + println!("{}", value); + Ok(()) + }, + ); + Ok(()) +} +``` + +### 8.6 Building from a Function + +The builder pattern allows customizing TSFN behavior: + +```rust +#[napi] +pub fn build_threadsafe_function_from_function( + callback: Function, u32>, +) -> Result<()> { + let tsfn = callback.build_threadsafe_function().build()?; + let tsfn_fatal = callback + .build_threadsafe_function() + .callee_handled::() + .build()?; + let tsfn_max_queue = callback + .build_threadsafe_function() + .max_queue_size::<1>() + .build()?; + let tsfn_weak = callback + .build_threadsafe_function() + .weak::() + .build()?; + Ok(()) +} +``` + +### 8.7 Custom Error Status + +```rust +pub struct ErrorStatus(String); +impl AsRef for ErrorStatus { + fn as_ref(&self) -> &str { &self.0 } +} +impl From for ErrorStatus { + fn from(value: Status) -> Self { ErrorStatus(value.to_string()) } +} + +#[napi] +pub fn threadsafe_function_throw_error_with_status( + cb: ThreadsafeFunction, +) -> Result<()> { + thread::spawn(move || { + cb.call( + Err(Error::new(ErrorStatus("CustomErrorStatus".to_string()), "ThrowFromNative".to_owned())), + ThreadsafeFunctionCallMode::Blocking, + ); + }); + Ok(()) +} +``` + +### 8.8 Weak Threadsafe Functions + +Weak TSFNs do not prevent the Node.js event loop from exiting: + +```rust +#[napi] +pub async fn tsfn_weak( + tsfn: ThreadsafeFunction<(), (), (), Status, false, true>, +) -> napi::Result<()> { + tsfn.call_async(()).await +} +``` + +### 8.9 Tuple Arguments + +```rust +#[napi] +pub fn accept_threadsafe_function_tuple_args( + func: ThreadsafeFunction>, +) { + thread::spawn(move || { + func.call( + Ok((1, false, "NAPI-RS".into()).into()), + ThreadsafeFunctionCallMode::NonBlocking, + ); + }); +} +``` + +--- + +## 9. Serde/Serialization with napi-rs Types + +### 9.1 Feature Flag + +Enable `serde-json` feature: +```toml +[dependencies] +napi = { version = "3", features = ["serde-json"] } +``` + +This adds: +- `impl ser::Error for Error` and `impl de::Error for Error` +- `impl From for Error` +- `env.from_js_value()` and `env.to_js_value()` for serde types +- Automatic (de)serialization between `serde_json::Value` and JS values +- Automatic (de)serialization between `serde_json::Map` and JS objects + +### 9.2 Using `#[napi(object)]` with Serde + +```rust +#[napi(object)] +#[derive(Serialize, Deserialize, Debug)] +struct PackageJson { + pub name: String, + pub version: String, + pub dependencies: Option>, + #[serde(rename = "devDependencies")] + pub dev_dependencies: Option>, +} + +#[napi] +fn read_package_json() -> Result { + let raw = fs::read_to_string("package.json")?; + let p: PackageJson = serde_json::from_str(&raw)?; + Ok(p) +} +``` + +When a struct has both `#[napi(object)]` and `#[derive(Serialize, Deserialize)]`, napi-rs uses serde for bidirectional conversion. The struct acts as a TypeScript interface with automatic serialization from JS objects and deserialization to JS objects. + +### 9.3 Direct serde_json::Value + +```rust +#[napi] +fn test_serde_roundtrip(data: Value) -> Value { + data // serde_json::Value <-> JavaScript any +} + +#[napi] +fn test_serde_big_number_precision(number: String) -> Value { + let data = format!("{{\"number\":{}}}", number); + serde_json::from_str(&data).unwrap() +} +``` + +### 9.4 Manual Serde with env.from_js_value / env.to_js_value + +```rust +#[derive(Serialize, Debug, Deserialize)] +struct BytesObject { + #[serde(with = "serde_bytes")] + code: Vec, +} + +#[napi] +fn test_serde_buffer_bytes(obj: Object, env: Env) -> napi::Result { + let obj: BytesObject = env.from_js_value(obj)?; + Ok(obj.code.len()) +} +``` + +### 9.5 Class with Serde + +```rust +#[napi] +struct PackageJsonReader { + i: Value, +} + +#[napi] +impl PackageJsonReader { + #[napi(constructor)] + pub fn new() -> Result { + let raw = fs::read_to_string("package.json")?; + Ok(Self { i: serde_json::from_str(&raw)? }) + } + + #[napi] + pub fn read(&self) -> &Value { + &self.i + } +} +``` + +### 9.6 serde_bytes Support + +For `Vec` fields that should be serialized as binary data (Buffer in JS), use `serde_bytes`: + +```rust +#[derive(Serialize, Deserialize)] +struct BytesObject { + #[serde(with = "serde_bytes")] + code: Vec, +} +``` + +### 9.7 Ordered JSON + +Use `serde-json-ordered` feature to preserve JSON key insertion order: + +```toml +napi = { version = "3", features = ["serde-json-ordered"] } +``` + +--- + +## 10. Version Compatibility Notes + +### 10.1 Current Checkout Version + +Based on git log and Cargo.toml files at `/workspace/napi-rs`: + +| Component | Version | +|-----------|---------| +| **napi crate** | 3.8.5 | +| **napi-sys crate** | 3.2.1 | +| **napi-derive crate** | 3.5.4 | +| **napi-derive-backend crate** | 5.0.3 | +| **napi-build crate** | 2.3.1 | +| **@napi-rs/cli** | 3.6.2 | +| **Git tag** | `napi-v3.8.5` | +| **Rust MSRV** | 1.88.0 | + +### 10.2 Versioning Scheme + +napi-rs uses a monorepo with independent crate versioning. The major version of the `napi` crate (v3) and `napi-derive` (v3) should match. `napi-build` is v2.x. The npm CLI package follows its own SemVer (v3.6.x). + +### 10.3 N-API Version Matrix + +napi-rs supports N-API versions 1 through 10 via feature flags: + +| Feature | N-API Version | Min Node.js | Key Capabilities | +|---------|---------------|-------------|------------------| +| `napi1` | 1 | v8.0.0 | Basic types, functions | +| `napi2` | 2 | v8.10.0 | Thread-safe functions (experimental) | +| `napi3` | 3 | v9.11.0 | Cleanup hooks | +| `napi4` | 4 | v10.6.0 | Thread-safe functions (stable), tokio_rt | +| `napi5` | 5 | v10.17.0 / v12.0.0 | Date | +| `napi6` | 6 | v10.7.0 / v12.0.0 | BigInt | +| `napi7` | 7 | v10.12.0 | Detached array buffers | +| `napi8` | 8 | v10.23+ / v12.23+ | Async cleanup hooks | +| `napi9` | 9 | v14.21+ / v16.17+ | SyntaxError, object property management | +| `napi10` | 10 | v18.17.0 | create_object_with_properties | + +### 10.4 Type Conversion Table + +From the README features table: + +| Rust Type | JavaScript Type | N-API Version | Feature Flag | +|-----------|----------------|---------------|--------------| +| `u32` | Number | 1 | -- | +| `i32` / `i64` | Number | 1 | -- | +| `f64` | Number | 1 | -- | +| `bool` | Boolean | 1 | -- | +| `String` / `&str` | String | 1 | -- | +| `Latin1String` | String | 1 | `latin1` | +| `UTF16String` | String | 1 | -- | +| `Object` | Object | 1 | -- | +| `serde_json::Map` | Object | 1 | `serde-json` | +| `serde_json::Value` | any | 1 | `serde-json` | +| `Array` | Array | 1 | -- | +| `Vec` | Array | 1 | -- | +| `Buffer` | Buffer | 1 | -- | +| `External` | External | 1 | -- | +| `Null` | null | 1 | -- | +| `Undefined` / `()` | undefined | 1 | -- | +| `T: Fn(...) -> Result` | Function | 1 | -- | +| `Async/Future` | Promise | 4 | `async` | +| `AsyncTask` | Promise | 1 | -- | +| `JsGlobal` | global | 1 | -- | +| `JsSymbol` | Symbol | 1 | -- | +| `Int8Array`/`Uint8Array`... | TypedArray | 1 | -- | +| `JsFunction` | threadsafe function | 4 | `napi4` | +| `BigInt` | BigInt | 6 | `napi6` | + +### 10.5 Electron Support + +napi-rs works in Electron. The tokio runtime is designed to handle Electron renderer process environment recycling (Node env exits and recreates on window reload). The `start_async_runtime()` and `shutdown_async_runtime()` functions manage the runtime lifecycle. + +### 10.6 WebAssembly Support + +napi-rs has experimental WASM support via the `wasm32-wasip1-threads` target. Key notes: +- The `@napi-rs/wasm-runtime` and `@emnapi/runtime` packages provide the WASM runtime +- Build with: `napi build --platform --target wasm32-wasip1-threads` +- Some features (like `tokio/net`, file I/O) are conditionally compiled out for WASM +- `tokio_unstable` cfg is used for WASM-specific tokio configuration + +### 10.7 Breaking Changes Notes + +- Version 3.x uses declarative `#[ctor]` for module registration (as of commit `ba6597b3`) +- `compat-mode` feature enables deprecated v2 types/traits +- The `ThreadSafeCallContext` type was renamed to `ThreadsafeCallContext` (v2.17.0+) +- Manual `refer()`/`unref()`/`abort()` methods on ThreadsafeFunction are deprecated; use `Clone`/`Drop` instead + +--- + +## Appendix: Type Conversion Quick Reference + +### Primitive Types + +| Rust | TypeScript | +|------|-----------| +| `u32`, `i32`, `f64` | `number` | +| `i64`, `u64` | `number` (or `BigInt` with `napi6`) | +| `bool` | `boolean` | +| `String`, `&str` | `string` | +| `()` | `void` | +| `Null` | `null` | +| `Undefined` | `undefined` | + +### Compound Types + +| Rust | TypeScript | +|------|-----------| +| `Vec` | `Array` | +| `Buffer` / `Uint8Array` | `Buffer` / `Uint8Array` | +| `Object` | `object` | +| `Option` | `T \| undefined` (or `T \| null`) | +| `Either` | `A \| B` | +| `Either3` | `A \| B \| C` | +| `Either4` | `A \| B \| C \| D` | +| `Result` | throw/rethrow | +| `Promise` | `Promise` | +| `Function` | `Function` | +| `serde_json::Value` | `any` | + +### Class Patterns + +| Rust Attribute | TypeScript | +|----------------|-----------| +| `#[napi] struct` | `class` | +| `#[napi(object)] struct` | `interface` | +| `#[napi(transparent)] struct` | same as inner type | +| `#[napi] enum` | numeric `enum` | +| `#[napi(string_enum)] enum` | string union type | +| `#[napi(discriminant = "type")] enum` | tagged union / discriminated union | + +--- + +*Report generated from source at `/workspace/napi-rs` (commit `ba6597b3`, tag `napi-v3.8.5`) and https://napi.rs* diff --git a/docs/research/taskgraph_rust_source.md b/docs/research/taskgraph_rust_source.md new file mode 100644 index 0000000..5ad72db --- /dev/null +++ b/docs/research/taskgraph_rust_source.md @@ -0,0 +1,1140 @@ +# TaskGraph Rust Source - Comprehensive Research Report + +> Source: `/workspace/@alkimiadev/taskgraph` (Rust CLI project) +> Report date: 2026-04-23 +> Version: 0.1.3 + +--- + +## Table of Contents + +1. [Project Structure](#1-project-structure) +2. [Cargo.toml Details](#2-cargotoml-details) +3. [Core Data Types and Public APIs](#3-core-data-types-and-public-apis) +4. [Functions/Methods to Expose via NAPI](#4-functionsmethods-to-expose-via-napi) +5. [Serialization (Serde) Support](#5-serialization-serde-support) +6. [Error Types and Error Handling](#6-error-types-and-error-handling) +7. [Input/Output Patterns](#7-inputoutput-patterns) +8. [Existing Tests and Benchmarks](#8-existing-tests-and-benchmarks) + +--- + +## 1. Project Structure + +### Directory Layout + +``` +taskgraph/ +├── Cargo.toml # Package manifest (single crate, not a workspace) +├── Cargo.lock # Locked dependencies +├── LICENSE-APACHE # Apache-2.0 license +├── LICENSE-MIT # MIT license +├── README.md # User-facing documentation +├── AGENTS.md # AI agent context file +├── opencode.json # OpenCode configuration +├── .github/ +│ └── workflows/ +│ └── ci.yml # CI: fmt, clippy, test, coverage +├── docs/ +│ ├── ARCHITECTURE.md # Full architecture spec +│ ├── framework.md # Cost-benefit framework rationale +│ ├── workflow.md # Practical workflow guide +│ ├── implementation.md # Tools/models/guidelines +│ ├── phase-1.md through phase-4.md # Phase plans +│ ├── issues/ # Blocking issues tracking +│ ├── reviews/ # Code review docs +│ └── research/ +│ └── cost_benefit_analysis_framework.py +├── scripts/ +│ └── benchmark.sh # Manual benchmark script +├── benches/ +│ └── graph_benchmarks.rs # Criterion benchmarks +├── src/ +│ ├── main.rs # Binary entry point (thin: parse CLI, execute) +│ ├── lib.rs # Library root - re-exports public API +│ ├── cli.rs # CLI argument definitions (clap derive) +│ ├── task.rs # Task, TaskFrontmatter, enums (serde types) +│ ├── graph.rs # DependencyGraph (petgraph wrapper) +│ ├── error.rs # Error enum (thiserror) +│ ├── config.rs # Config loading (.taskgraph.toml) +│ ├── discovery.rs # TaskCollection (directory scanning) +│ └── commands/ +│ ├── mod.rs # Command module re-exports +│ ├── init.rs # `init` command +│ ├── validate.rs # `validate` command +│ ├── list.rs # `list` command +│ ├── show.rs # `show` command +│ ├── deps.rs # `deps` command +│ ├── topo.rs # `topo` command +│ ├── cycles.rs # `cycles` command +│ ├── parallel.rs # `parallel` command +│ ├── critical.rs # `critical` command +│ ├── bottleneck.rs # `bottleneck` command +│ ├── risk.rs # `risk` command +│ ├── decompose.rs # `decompose` command +│ ├── workflow_cost.rs # `workflow-cost` command +│ ├── risk_path.rs # `risk-path` command +│ └── graph_cmd.rs # `graph` command (DOT output) +└── tests/ + ├── integration/ + │ └── commands.rs # 25 integration tests (assert_cmd) + └── fixtures/ + ├── tasks/ # 3 valid tasks (one depends on another) + ├── cycles/ # 3 tasks forming a cycle + ├── invalid/ # 1 task with missing dependency + ├── risk/ # 5 tasks with various risk levels + └── decompose/ # 4 tasks for decomposition testing +``` + +### Module Dependency Graph + +``` +lib.rs + ├── cli → commands::*, config, discovery, graph + ├── commands/* → cli, discovery, graph, task + ├── config → error + ├── discovery → task, error + ├── error → (thiserror, std, serde_yaml, serde_json) + ├── graph → discovery, task, petgraph + └── task → (serde, chrono, gray_matter, error) +``` + +### Crates + +This is a **single crate** project (not a Cargo workspace). It produces: +- **Library**: `libtaskgraph` (from `src/lib.rs`) +- **Binary**: `taskgraph` (from `src/main.rs`) + +--- + +## 2. Cargo.toml Details + +### Package Metadata + +| Field | Value | +|-------|-------| +| name | `taskgraph` | +| version | `0.1.3` | +| edition | `2021` | +| license | `MIT OR Apache-2.0` | +| description | CLI tool for managing task dependencies using markdown files | +| repository | `https://github.com/alkimiadev/taskgraph` | +| keywords | `task`, `dependency`, `graph`, `cli`, `markdown` | +| categories | `command-line-utilities`, `development-tools` | + +### Dependencies (Production) + +| Crate | Version | Features | Purpose | +|-------|---------|----------|---------| +| `petgraph` | `0.7` | - | Directed graph data structure & algorithms (toposort, cycle detection, etc.) | +| `gray_matter` | `0.2` | - | Markdown frontmatter extraction (YAML engine) | +| `serde` | `1.0` | `derive` | Serialization/deserialization framework | +| `serde_json` | `1.0` | - | JSON serialization (for `--format json` output) | +| `serde_yaml` | `0.9` | - | YAML serialization (for frontmatter parsing & roundtrip) | +| `clap` | `4.5` | `derive` | CLI argument parsing | +| `clap_complete` | `4.5` | - | Shell completion generation | +| `chrono` | `0.4` | `serde` | Date/time with serde support | +| `anyhow` | `1.0` | - | Ergonomic error handling (used in CLI/binary) | +| `thiserror` | `2.0` | - | Derived error types (used in library) | +| `dirs` | `6.0` | - | Platform directories (future: global config) | +| `walkdir` | `2.5` | - | Recursive directory walking | +| `tracing` | `0.1` | - | Structured logging | +| `tracing-subscriber` | `0.3` | `env-filter` | Log output formatting | +| `toml` | `0.8` | - | Config file parsing | + +### Dev Dependencies + +| Crate | Version | Purpose | +|-------|---------|---------| +| `tempfile` | `3.0` | Temporary directories for tests | +| `assert_cmd` | `2.0` | CLI integration testing | +| `predicates` | `3.0` | Assertion predicates for integration tests | +| `criterion` | `0.5` | Benchmarking framework | + +### Features + +```toml +[features] +default = [] +``` + +No feature flags exist yet. This is a good candidate for adding `napi` feature. + +### Release Profile + +```toml +[profile.release] +opt-level = 3 +lto = true +strip = true +``` + +--- + +## 3. Core Data Types and Public APIs + +### 3.1 Task (`src/task.rs`) + +The central data type. Represents a single task file. + +```rust +/// A task with its content. +#[derive(Debug, Clone)] +pub struct Task { + pub frontmatter: TaskFrontmatter, + pub body: String, // Markdown body content + pub source: Option, // Source file path (if loaded from file) +} +``` + +**Methods:** + +| Method | Signature | Returns | Description | +|--------|-----------|---------|-------------| +| `id()` | `&self -> &str` | Task ID | Accessor for frontmatter.id | +| `name()` | `&self -> &str` | Task name | Accessor for frontmatter.name | +| `status()` | `&self -> TaskStatus` | Status enum | Accessor for frontmatter.status | +| `depends_on()` | `&self -> &[String]` | Dependency list | Accessor for frontmatter.depends_on | +| `from_file()` | `&Path -> Result` | Parsed Task | Parse from a .md file on disk | +| `from_markdown()` | `&str, Option -> Result` | Parsed Task | Parse from markdown string + optional source name | +| `to_markdown()` | `&self -> Result` | Markdown string | Serialize back to markdown with YAML frontmatter | + +**Key observation:** `Task` itself does **NOT** derive `Serialize` or `Deserialize`. Only `TaskFrontmatter` does. The `body` and `source` fields are not serialized through serde - they're managed separately during parse/render. + +### 3.2 TaskFrontmatter (`src/task.rs`) + +The structured metadata extracted from YAML frontmatter: + +```rust +#[derive(Debug, Clone, Serialize, Deserialize)] +pub struct TaskFrontmatter { + pub id: String, + pub name: String, + #[serde(default)] + pub status: TaskStatus, + #[serde(default, rename = "depends_on")] + pub depends_on: Vec, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub priority: Option, + #[serde(default, skip_serializing_if = "Vec::is_empty")] + pub tags: Vec, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub created: Option>, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub modified: Option>, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub assignee: Option, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub due: Option, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub scope: Option, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub risk: Option, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub impact: Option, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub level: Option, +} +``` + +**Serde details:** +- All enums use `#[serde(rename_all = "kebab-case")]` for YAML keys +- Optional fields use `skip_serializing_if` to keep output clean +- Tags use `skip_serializing_if = "Vec::is_empty"` +- `depends_on` renamed from Rust `depends_on` (same, but explicitly) +- `status` has a default of `TaskStatus::Pending` + +### 3.3 Enum Types (`src/task.rs`) + +All enums derive `Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize, Default`. + +#### TaskStatus + +```rust +#[serde(rename_all = "kebab-case")] +pub enum TaskStatus { + Pending, // default + InProgress, // "in-progress" in YAML/JSON + Completed, + Failed, + Blocked, +} +``` + +Also implements `Display` (kebab-case strings). + +#### TaskScope + +```rust +#[serde(rename_all = "kebab-case")] +pub enum TaskScope { + Single, // ~500 tokens, cost 1.0 + Narrow, // default, ~1500 tokens, cost 2.0 + Moderate, // ~3000 tokens, cost 3.0 + Broad, // ~6000 tokens, cost 4.0 + System, // ~10000 tokens, cost 5.0 +} +``` + +Methods: `token_estimate() -> u32`, `cost_estimate() -> f64`, `Display` + +#### TaskRisk + +```rust +#[serde(rename_all = "kebab-case")] +pub enum TaskRisk { + Trivial, // p=0.98 + Low, // default, p=0.90 + Medium, // p=0.80 + High, // p=0.65 + Critical, // p=0.50 +} +``` + +Methods: `success_probability() -> f64`, `Display` + +#### TaskImpact + +```rust +#[serde(rename_all = "kebab-case")] +pub enum TaskImpact { + Isolated, // default, weight 1.0 + Component, // weight 1.5 + Phase, // weight 2.0 + Project, // weight 3.0 +} +``` + +Methods: `weight() -> f64`, `Display` + +#### TaskLevel + +```rust +#[serde(rename_all = "kebab-case")] +pub enum TaskLevel { + Planning, + Decomposition, + Implementation, // default + Review, + Research, +} +``` + +Methods: `Display` only + +### 3.4 DependencyGraph (`src/graph.rs`) + +A directed graph of task dependencies built from a `TaskCollection`. + +```rust +pub struct DependencyGraph { + graph: DiGraph, // petgraph directed graph + index_map: HashMap, // task ID -> node index +} +``` + +**Edge direction:** `from -> to` means "from must complete before to" (dependency must complete first). + +**Public API:** + +| Method | Signature | Returns | Description | +|--------|-----------|---------|-------------| +| `new()` | `-> Self` | Empty graph | Create empty graph | +| `from_collection()` | `&TaskCollection -> Self` | Built graph | Build from discovered tasks | +| `from_tasks()` | `Vec<&Task> -> Self` | Built graph | Build from explicit task list | +| `add_task()` | `&mut self, TaskId` | () | Add node | +| `add_dependency()` | `&mut self, &str, &str` | () | Add edge (from->to); silently ignores unknown IDs | +| `has_cycles()` | `&self -> bool` | Boolean | Uses `petgraph::algo::is_cyclic_directed` | +| `find_cycles()` | `&self -> Vec>` | Cycles | Custom DFS cycle finder | +| `topological_order()` | `&self -> Option>` | Order or None | Uses `petgraph::algo::toposort` | +| `dependencies()` | `&self, &str -> Vec` | Incoming neighbors | What this task depends on (direct) | +| `dependents()` | `&self, &str -> Vec` | Outgoing neighbors | What depends on this (direct) | +| `parallel_groups()` | `&self -> Vec>` | Generations | Tasks grouped by level (can run concurrently) | +| `critical_path()` | `&self -> Vec` | Path | Longest path through the graph | +| `weighted_critical_path()` | `&self, F: Fn(&str)->f64 -> Vec` | Weighted path | Path with highest cumulative weight | +| `bottlenecks()` | `&self -> Vec<(TaskId, usize)>` | Ranked list | Betweenness centrality via path counting | +| `to_dot()` | `&self -> String` | DOT string | GraphViz DOT format export | + +Also implements `Default` (returns `new()`). + +**Important:** `DependencyGraph` does **NOT** implement `Serialize`/`Deserialize`. It's a compute-only structure built fresh each time from tasks. + +### 3.5 TaskCollection (`src/discovery.rs`) + +Collection of tasks discovered from a directory: + +```rust +#[derive(Debug, Default)] +pub struct TaskCollection { + tasks: HashMap, // Tasks indexed by ID + paths: HashMap, // File paths indexed by ID + errors: Vec, // Parse errors encountered +} +``` + +**Public API:** + +| Method | Signature | Returns | Description | +|--------|-----------|---------|-------------| +| `new()` | `-> Self` | Empty collection | Constructor | +| `from_directory()` | `&Path -> Self` | Populated collection | Scan directory recursively for .md files | +| `get()` | `&self, &str -> Option<&Task>` | Task or None | Lookup by ID | +| `path()` | `&self, &str -> Option<&PathBuf>` | Path or None | File path for task ID | +| `tasks()` | `&self -> impl Iterator` | Iterator | All tasks | +| `ids()` | `&self -> impl Iterator` | Iterator | All task IDs | +| `len()` | `&self -> usize` | Count | Number of tasks | +| `is_empty()` | `&self -> bool` | Boolean | Empty check | +| `errors()` | `&self -> &[DiscoveryError]` | Errors | Parse errors from discovery | +| `missing_dependencies()` | `&self -> HashMap>` | Map | Task ID -> missing dep IDs | +| `validate()` | `&self -> ValidationResult` | Result | Full validation | + +**Important:** `TaskCollection` does **NOT** implement `Serialize`/`Deserialize` either. It's built procedurally. + +### 3.6 DiscoveryError (`src/discovery.rs`) + +```rust +#[derive(Debug, Clone)] +pub struct DiscoveryError { + pub path: PathBuf, + pub message: String, +} +``` + +No serde derives. Simple struct for error reporting. + +### 3.7 ValidationResult (`src/discovery.rs`) + +```rust +#[derive(Debug)] +pub struct ValidationResult { + pub task_count: usize, + pub errors: Vec, + pub missing_dependencies: HashMap>, +} +``` + +Methods: `is_valid() -> bool`, `issue_count() -> usize` + +No serde derives on the Rust type itself, but it's converted to `ValidationOutput` (which does derive `Serialize`) in the validate command. + +### 3.8 Config (`src/config.rs`) + +```rust +#[derive(Debug, Default, Serialize, Deserialize)] +pub struct Config { + #[serde(default)] + pub project: ProjectConfig, +} + +#[derive(Debug, Serialize, Deserialize)] +pub struct ProjectConfig { + #[serde(default = "default_tasks_dir")] + pub tasks_dir: String, // default: "tasks" +} +``` + +**API:** + +| Method | Signature | Returns | Description | +|--------|-----------|---------|-------------| +| `from_file()` | `&Path -> Result` | Config | Load from .taskgraph.toml | +| `find_and_load()` | `-> Option` | Config or None | Search up directory tree | +| `tasks_path()` | `&self -> PathBuf` | Path | Get tasks directory | + +### 3.9 CLI Types (`src/cli.rs`) + +```rust +#[derive(Clone, Copy, Debug, Default, ValueEnum)] +pub enum OutputFormat { + Plain, // default + Json, +} + +#[derive(Parser, Debug)] +pub struct Cli { + pub path: Option, + pub format: OutputFormat, + pub command: Commands, +} + +#[derive(Subcommand, Debug)] +pub enum Commands { + Init { id, name, scope, risk }, + Validate { strict }, + List { status, tag }, + Show { id }, + Deps { id }, + Dependents { id }, + Topo { status }, + Cycles, + Parallel, + Critical, + Bottleneck, + Risk, + Decompose, + WorkflowCost { include_completed, limit }, + RiskPath, + Graph { output }, + Completions { shell }, +} +``` + +The `Cli::execute()` method dispatches all commands. It creates `TaskCollection` from directory for each command. + +### 3.10 Lib.rs Public Re-exports + +```rust +pub mod cli; +pub mod commands; +pub mod config; +pub mod discovery; +pub mod error; +pub mod graph; +pub mod task; + +pub use config::Config; +pub use discovery::{DiscoveryError, TaskCollection, ValidationResult}; +pub use error::{Error, Result}; +pub use graph::DependencyGraph; +pub use task::{Task, TaskFrontmatter, TaskImpact, TaskLevel, TaskRisk, TaskScope, TaskStatus}; +``` + +--- + +## 4. Functions/Methods to Expose via NAPI + +### Priority 1: Core Data Types (Must Have) + +These are the foundational types that everything else depends on: + +| Rust Type | NAPI Class | Why | +|-----------|------------|-----| +| `Task` | `Task` | Central unit of work; must be creatable, readable, serializable from JS | +| `TaskFrontmatter` | Embedded in `Task` or separate class | All metadata is here; JS needs to read/write fields | +| `TaskStatus` | String enum mapping | Simple 5-variant enum; map to JS string union | +| `TaskScope` | String enum mapping | 5 variants with numeric mappings; map to JS string union | +| `TaskRisk` | String enum mapping | 5 variants with probability; map to JS string union | +| `TaskImpact` | String enum mapping | 4 variants with weight; map to JS string union | +| `TaskLevel` | String enum mapping | 5 variants; map to JS string union | + +### Priority 2: Core Functions (Must Have) + +| Rust Function | NAPI Method | Input | Output | Why | +|---------------|-------------|-------|--------|-----| +| `Task::from_markdown()` | `Task.fromMarkdown(content, source?)` | `string, string?` | `Task` | Parse task from markdown string | +| `Task::from_file()` | `Task.fromFile(path)` | `string` | `Task` | Parse task from file path | +| `Task::to_markdown()` | `task.toMarkdown()` | - | `string` | Serialize task back to markdown | +| `Task::id()` | `task.id` (getter) | - | `string` | Accessor | +| `Task::name()` | `task.name` (getter) | - | `string` | Accessor | +| `Task::status()` | `task.status` (getter) | - | `string` | Accessor | +| `Task::depends_on()` | `task.dependsOn` (getter) | - | `string[]` | Accessor | +| `TaskScope::token_estimate()` | `scope.tokenEstimate()` | - | `number` | Numeric mapping | +| `TaskScope::cost_estimate()` | `scope.costEstimate()` | - | `number` | Numeric mapping | +| `TaskRisk::success_probability()` | `risk.successProbability()` | - | `number` | Numeric mapping | +| `TaskImpact::weight()` | `impact.weight()` | - | `number` | Numeric mapping | + +### Priority 3: Collection & Discovery (Must Have) + +| Rust Function | NAPI Method | Input | Output | Why | +|---------------|-------------|-------|--------|-----| +| `TaskCollection::from_directory()` | `TaskCollection.fromDirectory(path)` | `string` | `TaskCollection` | Primary entry point: discover all tasks | +| `TaskCollection::new()` | `new TaskCollection()` | - | `TaskCollection` | Empty constructor for building manually | +| `TaskCollection::get()` | `collection.get(id)` | `string` | `Task\|null` | Lookup by ID | +| `TaskCollection::len()` | `collection.length` (getter) | - | `number` | Task count | +| `TaskCollection::ids()` | `collection.ids()` | - | `string[]` | All task IDs | +| `TaskCollection::tasks()` | `collection.tasks()` | - | `Task[]` | All tasks | +| `TaskCollection::errors()` | `collection.errors` (getter) | - | `DiscoveryError[]` | Parse errors | +| `TaskCollection::missing_dependencies()` | `collection.missingDependencies()` | - | `Record` | Find broken deps | +| `TaskCollection::validate()` | `collection.validate()` | - | `ValidationResult` | Full validation | + +### Priority 4: Graph Operations (Must Have) + +| Rust Function | NAPI Method | Input | Output | Why | +|---------------|-------------|-------|--------|-----| +| `DependencyGraph::from_collection()` | `DependencyGraph.fromCollection(collection)` | `TaskCollection` | `DependencyGraph` | Build graph | +| `DependencyGraph::new()` | `new DependencyGraph()` | - | `DependencyGraph` | Empty graph constructor | +| `DependencyGraph::from_tasks()` | `DependencyGraph.fromTasks(tasks[])` | `Task[]` | `DependencyGraph` | Build from JS array | +| `add_task()` | `graph.addTask(id)` | `string` | `void` | Add node | +| `add_dependency()` | `graph.addDependency(from, to)` | `string, string` | `void` | Add edge | +| `has_cycles()` | `graph.hasCycles()` | - | `boolean` | Cycle detection | +| `find_cycles()` | `graph.findCycles()` | - | `string[][]` | Get actual cycles | +| `topological_order()` | `graph.topologicalOrder()` | - | `string[]\|null` | Execution order | +| `dependencies()` | `graph.dependencies(id)` | `string` | `string[]` | Direct deps | +| `dependents()` | `graph.dependents(id)` | `string` | `string[]` | What depends on this | +| `parallel_groups()` | `graph.parallelGroups()` | - | `string[][]` | Parallel work groups | +| `critical_path()` | `graph.criticalPath()` | - | `string[]` | Longest path | +| `weighted_critical_path()` | `graph.weightedCriticalPath(weightFn)` | `(id: string) => number` | `string[]` | Weighted longest path | +| `bottlenecks()` | `graph.bottlenecks()` | - | `[string, number][]` | Betweenness centrality | +| `to_dot()` | `graph.toDot()` | - | `string` | GraphViz DOT format | + +### Priority 5: Config (Nice to Have) + +| Rust Function | NAPI Method | Input | Output | Why | +|---------------|-------------|-------|--------|-----| +| `Config::from_file()` | `Config.fromFile(path)` | `string` | `Config` | Load config | +| `Config::find_and_load()` | `Config.findAndLoad()` | - | `Config\|null` | Auto-discover config | +| `Config::tasks_path()` | `config.tasksPath` (getter) | - | `string` | Get tasks dir | + +### Priority 6: Workflow Cost Calculation (Nice to Have) + +The `workflow_cost` command uses `calculate_task_ev()` which is a private function. Consider exposing: + +| Function | NAPI Method | Input | Output | Why | +|----------|-------------|-------|--------|-----| +| `calculate_task_ev()` (currently private) | `calculateTaskEv(p, scopeCost, impactWeight)` | `number, number, number` | `number` | Expected value calculation | + +This would need to be made `pub` or reimplemented in the NAPI layer. + +### Notes on `weighted_critical_path` for NAPI + +The `weighted_critical_path` takes a Rust closure `F: Fn(&str) -> f64`. For NAPI, this would need to: +1. Accept a JavaScript function callback, OR +2. Accept a `Record` map of task ID -> weight + +Option 2 is simpler and avoids cross-language callback overhead. For example: + +```typescript +// NAPI signature option A (callback approach - complex) +graph.weightedCriticalPath((taskId: string) => number): string[] + +// NAPI signature option B (map approach - simpler) +graph.weightedCriticalPath(weights: Record): string[] +``` + +--- + +## 5. Serialization (Serde) Support + +### Full Serde Support (Serialize + Deserialize) + +| Type | Serialize | Deserialize | Notes | +|------|-----------|-------------|-------| +| `TaskStatus` | Yes | Yes | `rename_all = "kebab-case"` | +| `TaskScope` | Yes | Yes | `rename_all = "kebab-case"` | +| `TaskRisk` | Yes | Yes | `rename_all = "kebab-case"` | +| `TaskImpact` | Yes | Yes | `rename_all = "kebab-case"` | +| `TaskLevel` | Yes | Yes | `rename_all = "kebab-case"` | +| `TaskFrontmatter` | Yes | Yes | Rich serde attributes (skip_serializing_if, rename, default) | +| `Config` | Yes | Yes | Via TOML | +| `ProjectConfig` | Yes | Yes | Via TOML | + +### No Serde Support + +| Type | Serialize | Deserialize | Reason | +|------|-----------|-------------|--------| +| `Task` | No | No | `body` and `source` are separate from frontmatter; `to_markdown()` handles serialization manually | +| `DependencyGraph` | No | No | Computed structure; rebuilt from tasks each time | +| `TaskCollection` | No | No | Procedurally built from directory scanning | +| `DiscoveryError` | No | No | Error reporting struct | +| `ValidationResult` | No | No | Internal result type | +| `Error` | No | No | Error enum | +| `OutputFormat` | No | No | CLI-only (ValueEnum, not serde) | +| `Cli` | No | No | CLI-only (clap derive) | +| `Commands` | No | No | CLI-only enum | + +### JSON Serialization in Commands (Ad-hoc) + +Several command modules define private structs that derive `Serialize` for JSON output: + +| File | Struct | Fields | +|------|--------|--------| +| `validate.rs` | `ValidationOutput` | valid, task_count, error_count, errors[], missing_deps | +| `validate.rs` | `ValidationError` | path, message | +| `list.rs` | `TaskSummary` | id, name, status, scope | +| `show.rs` | `TaskDetails` | id, name, status, depends_on, scope, risk, impact, level, tags, body | +| `deps.rs` | `DependencyInfo` | id, status, exists | +| `deps.rs` | `DependenciesOutput` | task_id, dependencies[] | +| `topo.rs` | `TopoTask` | position, id, name, status | +| `topo.rs` | `TopoOutput` | order[], has_cycles | +| `cycles.rs` | `CyclesOutput` | has_cycles, cycle_count, cycles[] | +| `workflow_cost.rs` | `TaskCost` | id, name, cost | + +These are **private** to each command module and not part of the public API. For NAPI, we would define equivalent TypeScript interfaces or create new public serializable structs. + +### Serialization Format Details + +**YAML (frontmatter):** `TaskFrontmatter` uses `serde_yaml` with: +- `rename_all = "kebab-case"` on enums → `in-progress`, `narrow`, `high`, etc. +- `rename = "depends_on"` on the `depends_on` field (explicit) +- `default` on required-ish fields +- `skip_serializing_if = "Option::is_none"` for optional fields +- `skip_serializing_if = "Vec::is_empty"` for tags + +**JSON (output):** Uses `serde_json::to_string_pretty()` in commands. + +**TOML (config):** `Config` uses `toml::from_str()`. + +**Roundtrip:** `Task::from_markdown()` + `Task::to_markdown()` should produce equivalent output (tested implicitly). + +--- + +## 6. Error Types and Error Handling + +### Library Error Type (`src/error.rs`) + +```rust +#[derive(Error, Debug)] +pub enum Error { + #[error("Task not found: {0}")] + TaskNotFound(String), + + #[error("Task already exists: {0}")] + TaskAlreadyExists(String), + + #[error("Circular dependency detected: {0}")] + CircularDependency(String), + + #[error("Invalid frontmatter in {file}: {message}")] + InvalidFrontmatter { file: String, message: String }, + + #[error("Missing required field '{field}' in {file}")] + MissingField { file: String, field: String }, + + #[error("IO error: {0}")] + Io(#[from] std::io::Error), + + #[error("YAML parsing error: {0}")] + Yaml(#[from] serde_yaml::Error), + + #[error("JSON serialization error: {0}")] + Json(#[from] serde_json::Error), + + #[error("Graph error: {0}")] + Graph(String), +} + +pub type Result = std::result::Result; +``` + +**Error conversion:** `From` impls via `#[from]` for `std::io::Error`, `serde_yaml::Error`, `serde_json::Error`. + +**Usage patterns:** +- Library code returns `crate::Result` (= `Result`) +- `anyhow::Result` is used only in `main.rs` for the binary entry point +- `thiserror` provides `Display` impls automatically + +### CLI Error Handling + +The `Cli::execute()` method returns `anyhow::Result<()>`. Each command function returns `crate::Result<()>`. The `?` operator converts between them naturally. + +**Error handling at boundaries:** +- `Task::from_file()`: IO errors → `Error::Io`, parse errors → `Error::InvalidFrontmatter` +- `TaskCollection::from_directory()`: Silently skips files without frontmatter, stores errors in `DiscoveryError` list (non-fatal) +- `Config::from_file()`: TOML parse errors → `Error::Graph(format!(...))` (note: reuses Graph variant) +- Command functions: `Error::TaskNotFound` when task ID missing, `Error::TaskAlreadyExists` on duplicate init + +### NAPI Error Mapping Strategy + +For the Node.js wrapper, we should map: + +| Rust Error | Node.js Error | Notes | +|------------|---------------|-------| +| `TaskNotFound(id)` | Generic `Error` with message | JS: `throw new Error("Task not found: ")` | +| `TaskAlreadyExists(id)` | Generic `Error` with message | JS: `throw new Error("Task already exists: ")` | +| `CircularDependency(msg)` | Generic `Error` with message | JS: `throw new Error("Circular dependency: ")` | +| `InvalidFrontmatter { file, message }` | Generic `Error` with message | JS: `throw new Error("Invalid frontmatter in : ")` | +| `MissingField { file, field }` | Generic `Error` with message | JS: `throw new Error("Missing field in ")` | +| `Io(err)` | Generic `Error` with message | JS: `throw new Error("IO error: ")` | +| `Yaml(err)` | Generic `Error` with message | JS: `throw new Error("YAML parsing error: ")` | +| `Json(err)` | Generic `Error` with message | JS: `throw new Error("JSON error: ")` | +| `Graph(msg)` | Generic `Error` with message | JS: `throw new Error("Graph error: ")` | + +Alternatively, we could create custom JS error classes for better programmatic handling: + +```typescript +class TaskNotFoundError extends Error { taskId: string } +class CircularDependencyError extends Error { } +class InvalidFrontmatterError extends Error { file: string; message: string } +``` + +--- + +## 7. Input/Output Patterns + +### Data Flow Overview + +``` + DISCOVERY +tasks/*.md files ──────────────> TaskCollection + (disk) (HashMap) + │ + │ from_collection() / from_tasks() + ▼ + DependencyGraph + (DiGraph) + │ + ┌────────────────┼────────────────────┐ + │ │ │ + ▼ ▼ ▼ + topological parallel_groups critical_path + order() () () + │ │ │ + └────────────────┴─────────────────────┘ + │ + ▼ + Output (plain/JSON) +``` + +### Input Patterns + +1. **File-based input (primary):** `TaskCollection::from_directory(path)` scans a directory recursively for `.md` files, parses each, and builds the collection. This is the main entry point. + +2. **String-based input:** `Task::from_markdown(content, source)` parses a single markdown string. Useful for programmatic construction. + +3. **Path-based input:** `Task::from_file(path)` reads a single file and parses it. + +4. **Programmatic construction:** `DependencyGraph::new()` + `add_task()` + `add_dependency()` for building graphs manually. + +### Output Patterns + +1. **Plain text (default):** Human-readable terminal output with tables, arrows, and formatting. + +2. **JSON output (`--format json`):** Structured JSON using ad-hoc `Serialize` structs in each command. This is the primary programmatic output format. + +3. **DOT format:** `DependencyGraph::to_dot()` returns GraphViz DOT format string. + +4. **Markdown roundtrip:** `Task::to_markdown()` produces valid markdown with YAML frontmatter. + +### Typical Usage Flow + +```rust +// 1. Discover tasks +let collection = TaskCollection::from_directory(Path::new("./tasks")); + +// 2. Validate +let result = collection.validate(); +if !result.is_valid() { /* handle errors */ } + +// 3. Build graph +let graph = DependencyGraph::from_collection(&collection); + +// 4. Analyze +let has_cycles = graph.has_cycles(); +let order = graph.topological_order(); +let parallel = graph.parallel_groups(); +let critical = graph.critical_path(); +let bottlenecks = graph.bottlenecks(); +``` + +### NAPI Data Flow Design + +For the Node.js wrapper, the recommended data flow is: + +```typescript +// Option A: File-based (mirrors Rust CLI) +const collection = TaskCollection.fromDirectory('./tasks'); +const graph = DependencyGraph.fromCollection(collection); + +// Option B: Programmatic (unique to NAPI) +const tasks = [ + Task.fromMarkdown('---\nid: t1\nname: Task 1\n---\nBody'), + Task.fromMarkdown('---\nid: t2\nname: Task 2\ndepends_on: [t1]\n---\nBody'), +]; +const graph = DependencyGraph.fromTasks(tasks); + +// Option C: Manual graph construction +const graph = new DependencyGraph(); +graph.addTask('t1'); +graph.addTask('t2'); +graph.addDependency('t1', 't2'); +``` + +### Memory/Ownership Considerations for NAPI + +- `Task` is `Clone` (cheap to clone; contains String, TaskFrontmatter, Option) +- `TaskCollection` owns all `Task` objects (HashMap) +- `DependencyGraph` owns the graph structure (not the tasks themselves; only stores task IDs as node weights) +- `DependencyGraph::from_collection()` borrows `&TaskCollection` (doesn't take ownership) +- `Task::from_file()` and `from_markdown()` return owned `Task` values + +For NAPI, we need to decide: +1. **Should `TaskCollection` hold JS-managed task objects or Rust-owned?** Probably Rust-owned (tasks are parsed from files/strings, not constructed in JS). +2. **Should graph operations return strings or Task references?** Currently returns `Vec` (strings). The JS side can look up tasks from the collection. This is efficient. +3. **Should `DependencyGraph` keep a reference to `TaskCollection`?** Currently no. This means JS must pass the collection alongside the graph for enriched output. We could create a combined `TaskGraph` class in the NAPI layer. + +--- + +## 8. Existing Tests and Benchmarks + +### Unit Tests (in-source) + +| File | Test Count | Key Tests | +|------|-----------|-----------| +| `src/graph.rs` | 12 | Empty graph, add task/dep, missing deps, cycle detection, topo sort, parallel groups, critical path, bottleneck, DOT output, unknown task queries | +| `src/discovery.rs` | 5 | Single task discovery, skip files without frontmatter, duplicate ID detection, missing dependencies, validation result | +| `src/config.rs` | 2 | Default config, load from file | + +### Integration Tests (`tests/integration/commands.rs`) + +25 tests total using `assert_cmd`: + +| Test | Command | What It Verifies | +|------|---------|-----------------| +| `test_list_command` | `list` | Lists all 3 fixture tasks | +| `test_list_with_status_filter` | `list --status completed` | Filters correctly | +| `test_show_command` | `show task-one` | Shows task details | +| `test_show_missing_task` | `show missing-task` | Fails on missing | +| `test_validate_command` | `validate` | Succeeds on valid fixtures | +| `test_validate_with_missing_dependency` | `validate` (invalid) | Reports missing deps | +| `test_topo_command` | `topo` | Outputs topological order | +| `test_deps_command` | `deps task-two` | Shows task-one as dependency | +| `test_dependents_command` | `dependents task-one` | Shows tasks two and three | +| `test_cycles_command_no_cycles` | `cycles` | No cycles in valid fixtures | +| `test_cycles_command_with_cycles` | `cycles` (cycles fixtures) | Detects cycle | +| `test_parallel_command` | `parallel` | Shows generation groups | +| `test_critical_command` | `critical` | Shows critical path | +| `test_graph_command` | `graph` | Outputs DOT format | +| `test_bottleneck_command` | `bottleneck` | Shows bottleneck tasks | +| `test_init_command` | `init new-task` | Creates file | +| `test_init_duplicate_task` | `init task-one` | Fails on duplicate | +| `test_init_with_options` | `init --scope narrow --risk low` | Writes scope/risk to file | +| `test_risk_command` | `risk` | Distribution with counts | +| `test_risk_command_empty` | `risk` (empty dir) | "No tasks found" | +| `test_decompose_command` | `decompose` | Flags high-risk/broad-scope tasks | +| `test_decompose_command_none_needed` | `decompose` (low-risk tasks) | "No tasks need decomposition" | +| `test_workflow_cost_command` | `workflow-cost` | Shows cost analysis | +| `test_workflow_cost_command_empty` | `workflow-cost` (empty) | "No tasks found" | +| `test_risk_path_command` | `risk-path` | Shows risk path | +| `test_risk_path_command_empty` | `risk-path` (empty) | "No tasks found" | +| `test_help_flag` | `--help` | Shows help text | +| `test_version_flag` | `--version` | Succeeds | +| `test_completions_bash` | `completions bash` | Bash completion output | +| `test_completions_zsh` | `completions zsh` | Zsh completion output | +| `test_completions_fish` | `completions fish` | Fish completion output | + +### Benchmark Suite (`benches/graph_benchmarks.rs`) + +Uses Criterion. Two benchmark groups: + +1. **`load_tasks`**: Measures `TaskCollection::from_directory()` + `DependencyGraph::from_collection()` for 50, 100, 500, 1000 tasks. + +2. **`graph_ops`**: On 1000-task graph, measures: + - `topological_sort_1000` + - `cycle_detection_1000` + - `critical_path_1000` + - `bottlenecks_1000` + +Test data: linear chain of tasks (task-i depends on task-(i-1)). + +### Performance Numbers (from README) + +| Tasks | Load Time | Topo Sort | Cycles | Critical Path | +|-------|-----------|-----------|--------|---------------| +| 50 | 3ms | 3ms | 2ms | 8ms | +| 500 | 19ms | 21ms | 14ms | 52ms | +| 1,000 | 34ms | 42ms | 26ms | 82ms | + +(Benchmarked on AMD EPYC 9004 series) + +### CI Pipeline (`.github/workflows/ci.yml`) + +Two jobs: +1. **Test**: checkout -> install Rust (with rustfmt, clippy) -> cache -> fmt check -> clippy -> test -> build release +2. **Coverage**: checkout -> install Rust -> cache -> install cargo-llvm-cov -> generate lcov -> upload to Codecov + +### Test Coverage + +Reported at 89% (meeting the 80% target from AGENTS.md). + +--- + +## Appendix A: Complete Type Reference for NAPI Mapping + +### Enums to JS String Unions + +```typescript +// task.ts +type TaskStatus = "pending" | "in-progress" | "completed" | "failed" | "blocked"; +type TaskScope = "single" | "narrow" | "moderate" | "broad" | "system"; +type TaskRisk = "trivial" | "low" | "medium" | "high" | "critical"; +type TaskImpact = "isolated" | "component" | "phase" | "project"; +type TaskLevel = "planning" | "decomposition" | "implementation" | "review" | "research"; +``` + +### Proposed NAPI Class Structure + +```typescript +// task.ts +class Task { + // Static constructors + static fromMarkdown(content: string, source?: string): Task; + static fromFile(path: string): Task; + + // Getters + get id(): string; + get name(): string; + get status(): TaskStatus; + get dependsOn(): string[]; + get body(): string; + get source(): string | null; + + // Frontmatter access (via JS object) + get frontmatter(): TaskFrontmatter; + + // Serialization + toMarkdown(): string; +} + +interface TaskFrontmatter { + id: string; + name: string; + status: TaskStatus; + dependsOn: string[]; + priority?: string; + tags: string[]; + created?: string; // ISO 8601 + modified?: string; // ISO 8601 + assignee?: string; + due?: string; + scope?: TaskScope; + risk?: TaskRisk; + impact?: TaskImpact; + level?: TaskLevel; +} + +// collection.ts +class TaskCollection { + static fromDirectory(path: string): TaskCollection; + get(id: string): Task | null; + get length(): number; + ids(): string[]; + tasks(): Task[]; + get errors(): DiscoveryError[]; + missingDependencies(): Record; + validate(): ValidationResult; +} + +interface DiscoveryError { + path: string; + message: string; +} + +interface ValidationResult { + taskCount: number; + errors: DiscoveryError[]; + missingDependencies: Record; + isValid(): boolean; + issueCount(): number; +} + +// graph.ts +class DependencyGraph { + static fromCollection(collection: TaskCollection): DependencyGraph; + static fromTasks(tasks: Task[]): DependencyGraph; + + addTask(id: string): void; + addDependency(from: string, to: string): void; + hasCycles(): boolean; + findCycles(): string[][]; + topologicalOrder(): string[] | null; + dependencies(taskId: string): string[]; + dependents(taskId: string): string[]; + parallelGroups(): string[][]; + criticalPath(): string[]; + weightedCriticalPath(weights: Record): string[]; + bottlenecks(): [string, number][]; + toDot(): string; +} + +// config.ts +class Config { + static fromFile(path: string): Config; + static findAndLoad(): Config | null; + get tasksPath(): string; +} + +// workflow.ts +function calculateTaskEv(p: number, scopeCost: number, impactWeight: number): number; +``` + +### Key Decisions for NAPI Implementation + +1. **Task mutability:** The Rust `Task` struct is `Clone` but has no setters. For NAPI, we should either: + - Make the JS `Task` immutable (read-only after creation) - simpler, matches Rust + - Add a `TaskBuilder` pattern for constructing tasks programmatically + +2. **Enum representation:** Use JS string literals (not numeric enums) to match the `kebab-case` serde serialization. + +3. **Error handling:** Throw JS `Error` objects from NAPI. Consider custom error classes for `TaskNotFound` and `InvalidFrontmatter`. + +4. **DateTime handling:** `chrono::DateTime` maps to ISO 8601 strings in JS. No need for JS `Date` objects in the NAPI layer. + +5. **Graph lifetime:** The Rust `DependencyGraph` borrows nothing (stores owned `String` node weights). It can be freely moved/owned in NAPI. + +6. **Collection lifetime:** `TaskCollection` owns its tasks. The NAPI class should hold the Rust struct. Returning `Task` references from `collection.get()` requires careful lifetime management - consider returning clones. + +7. **`weighted_critical_path` callback:** Replace the Rust closure with a JS `Record` dict lookup to avoid FFI callback overhead and complexity. + +--- + +## Appendix B: Notable Implementation Details + +### Bottleneck Algorithm + +The current `bottlenecks()` implementation uses an O(n^2 * P) algorithm where P is the number of paths between nodes. It enumerates all paths between all pairs, then counts how many paths each task appears on. This is **not** true betweenness centrality (which uses Brandes' O(VE) algorithm) but a simpler path-counting approach. For large graphs, this could be slow. The benchmark only tests up to 1000 nodes with linear topology. + +### Critical Path Algorithm + +Uses recursive memoized longest-path computation. Works well for DAGs but will return empty/incorrect results if cycles exist (the `parallel_groups` method also silently breaks if cycles exist). + +### Missing: Task Serialization + +`Task` does not implement `Serialize`/`Deserialize`. The `to_markdown()` method manually concatenates YAML frontmatter + markdown body. If we need JSON serialization of the full `Task` (including body), we should add a new serializable struct like: + +```rust +#[derive(Serialize)] +pub struct SerializableTask { + pub frontmatter: TaskFrontmatter, + pub body: String, + pub source: Option, +} +``` + +Or implement `Serialize` for `Task` directly. + +### Missing: Task Mutability + +There are no methods to update a task's status, dependencies, etc. in place. The current design assumes files are the source of truth and are edited directly. For an NAPI wrapper, we may want to add: +- `task.set_status(status: TaskStatus)` +- `task.set_depends_on(deps: Vec)` +- etc. + +Or use a builder pattern for creating new tasks. + +### Missing: Partial Graph Building + +`DependencyGraph::from_collection()` adds edges only for dependencies that exist as nodes in the graph. Missing dependencies are silently ignored (no error, no warning). This matches the `add_dependency()` behavior which checks `index_map` before adding edges. + +### walkdir::FollowLinks(false) + +`TaskCollection::from_directory()` does not follow symlinks. This is intentional for safety. + +--- + +## Appendix C: Dependency Version Compatibility Notes + +| Crate | Version | Notes for NAPI | +|-------|---------|---------------| +| `petgraph` | `0.7` | Stable API; `DiGraph` and algorithms are well-defined | +| `gray_matter` | `0.2` | Minor version; API may change in `0.3` | +| `serde` | `1.0` | Very stable; `derive` feature needed | +| `serde_json` | `1.0` | Very stable | +| `serde_yaml` | `0.9` | Note: `serde_yaml` 0.9 is the last version before potential breaking changes | +| `chrono` | `0.4` | Stable; `serde` feature for serialization | +| `clap` | `4.5` | CLI-only; not needed in NAPI lib | +| `thiserror` | `2.0` | Error derive; v2 is newer than commonly seen | +| `toml` | `0.8` | For config loading | +| `walkdir` | `2.5` | For directory scanning | + +For NAPI, we can exclude from the build: +- `clap` / `clap_complete` (CLI-only, not needed for library) +- `tracing` / `tracing-subscriber` (logging, optional) +- `dirs` (platform directories, only for CLI default paths) + +This could be done with feature flags: +```toml +[features] +default = ["cli"] +cli = ["clap", "clap_complete", "tracing", "tracing-subscriber", "dirs"] +napi = [] # Minimal dependencies for Node.js binding +``` diff --git a/docs/research/taskgraph_ts_current_state.md b/docs/research/taskgraph_ts_current_state.md new file mode 100644 index 0000000..eb8e13a --- /dev/null +++ b/docs/research/taskgraph_ts_current_state.md @@ -0,0 +1,172 @@ +# TaskGraph_TS — Current State Research Report + +**Date:** 2026-04-23 +**Repository:** git@git.alk.dev:alkdev/taskgraph_ts.git +**Branch:** main +**Commit:** 1517b54 ("inital setup") + +--- + +## 1. File Structure + +The repository is in an extremely early, freshly-initialized state. The entire non-git content consists of only two files: + +``` +/workspace/@alkdev/taskgraph_ts/ +├── .git/ # Git repository data +├── AGENTS.md # Agent/instructions file (32 lines) +└── docs/ + └── research/ + └── .gitkeep # Empty placeholder to keep directory in git +``` + +That is the **complete** file tree. There are no source files, no configuration files, no tests, no documentation beyond what is listed above. + +--- + +## 2. Configuration Files + +### What exists + +| File | Status | +|---|---| +| `package.json` | **Does not exist** | +| `tsconfig.json` | **Does not exist** | +| `Cargo.toml` | **Does not exist** | +| `tslint.json` / `.eslintrc*` | **Does not exist** | +| `.prettierrc*` | **Does not exist** | +| `.gitignore` | **Does not exist** | +| `.editorconfig` | **Does not exist** | +| `README.md` | **Does not exist** | +| `LICENSE` | **Does not exist** | +| Any Makefile / build script | **Does not exist** | + +### Summary + +There are **zero configuration files** in the repository. No package manager is initialized. No TypeScript compiler configuration exists. No linters, formatters, or CI/CD pipelines have been set up. The project name `taskgraph_ts` implies a TypeScript project, but no `package.json` or `tsconfig.json` has been created yet. + +--- + +## 3. Existing Code / Boilerplate + +There is **no source code** in the repository. No `src/` directory exists. No `lib/`, `bin/`, `test/`, or `examples/` directories exist. No `.ts`, `.js`, `.rs`, or any other source files are present. + +The only non-git file with actual content is `AGENTS.md`, which is a meta-tool instruction file (not project code). The only other file is `docs/research/.gitkeep`, which is an empty placeholder. + +--- + +## 4. AGENTS.md File Contents + +The `AGENTS.md` file is 32 lines long and contains instructions for AI agent memory/tool usage. Full content: + +```markdown +## Memory Tools + +You have access to two tools for managing your context and accessing session history: + +### memory({tool: "...", args: {...}}) + +Read-only tool for introspecting your session history and context state. Available operations: +- `memory({tool: "help"})` — full reference with examples +- `memory({tool: "summary"})` — quick counts of projects, sessions, messages, todos +- `memory({tool: "sessions"})` — list recent sessions (useful for finding past work) +- `memory({tool: "messages", args: {sessionId: "..."}})` — read a session's conversation +- `memory({tool: "search", args: {query: "..."}})` — search across all conversations +- `memory({tool: "compactions", args: {sessionId: "..."}})` — view compaction checkpoints +- `memory({tool: "context"})` — check your current context usage + +### memory_compact() + +Trigger compaction on the current session. This summarizes the conversation so far to free context space. + +**When to use memory_compact:** +- When context is above 80% (check with `memory({tool: "context"})`) +- When you notice you're losing track of earlier conversation details +- At natural breakpoints in multi-step tasks (after completing a subtask, before starting a new one) +- When the system prompt shows a yellow/red/critical context warning +- Proactively, rather than waiting for automatic compaction at 92% + +**When NOT to use memory_compact:** +- When context is below 50% (it wastes a compaction cycle) +- In the middle of a complex edit that you need immediate context for +- When the task is nearly complete (just finish the task instead) + +Compaction preserves your most important context in a structured summary — you will continue the session with the summary as your starting point. +``` + +This file is **not** project-specific documentation — it is a shared instruction set for AI coding agents describing how to use memory introspection and compaction tools. It contains no information about the `taskgraph_ts` project's design, architecture, or goals. + +--- + +## 5. Documentation and Design Notes + +There is **no project documentation** in the repository. Key absences: + +- No `README.md` describing the project +- No design documents or architecture notes +- No ADRs (Architecture Decision Records) +- No API specifications +- No diagrams or schematics +- The `docs/research/` directory exists but contains only an empty `.gitkeep` file — it was created as a placeholder for future research documents (like this one) + +--- + +## 6. Git History + +### Full Git Log + +``` +1517b54 inital setup +``` + +There is exactly **one commit** in the repository, authored on 2026-04-23: + +| Field | Value | +|---|---| +| **Commit** | `1517b5459e1d79388a96057d17a72ac53064a068` | +| **Author** | glm-5.1 | +| **Date** | Thu Apr 23 08:13:39 2026 +0000 | +| **Message** | "inital setup" (note: typo — "inital" instead of "initial") | + +### Files in that commit + +| File | Change | +|---|---| +| `AGENTS.md` | Added (32 lines) | +| `docs/research/.gitkeep` | Added (empty file) | + +### Branches + +| Branch | Status | +|---|---| +| `main` (local) | Current branch, up to date with remote | +| `remotes/origin/main` | Remote tracking branch | + +### Remote + +``` +origin git@git.alk.dev:alkdev/taskgraph_ts.git (fetch) +origin git@git.alk.dev:alkdev/taskgraph_ts.git (push) +``` + +### Working tree status + +Clean — no uncommitted changes, no untracked files. + +--- + +## Summary and Key Observations + +1. **This is a blank-slate project.** The repository was just initialized with a single commit containing only an agent instructions file and an empty directory placeholder. No code, configuration, or documentation has been written yet. + +2. **The project name implies TypeScript.** `taskgraph_ts` strongly suggests a TypeScript implementation of a "task graph" — likely a DAG (Directed Acyclic Graph) based task scheduling/execution system. However, no TypeScript tooling (`package.json`, `tsconfig.json`) has been set up. + +3. **No `.gitignore` exists.** This should be created before adding any source files to avoid accidentally committing `node_modules/`, build artifacts, etc. + +4. **No README or design docs.** There are no records of the project's purpose, scope, or intended architecture. Any future work will need to define these from scratch. + +5. **The `docs/research/` directory** was explicitly created as a placeholder, suggesting the project founders intended for research documentation to be generated — which is what this report fulfills. + +6. **Single author so far.** The only commit was by `glm-5.1@alk.dev`, an AI agent, indicating the project was initialized programmatically. + +7. **Everything needs to be built from scratch:** package initialization, TypeScript configuration, source code structure, testing framework, linting/formatting, CI/CD, documentation, and the actual task graph implementation itself.