docs(research): add iroh suite deep-dive references for iroh, irpc, iroh-blobs, iroh-gossip, iroh-live, and iroh-docs
This commit is contained in:
160
docs/research/references/iroh/iroh/01-overview-architecture.md
Normal file
160
docs/research/references/iroh/iroh/01-overview-architecture.md
Normal file
@@ -0,0 +1,160 @@
|
||||
# Iroh: Overview & Architecture
|
||||
|
||||
**Version**: 0.98.1
|
||||
**Repository**: https://github.com/n0-computer/iroh
|
||||
**License**: MIT OR Apache-2.0
|
||||
**Rust Edition**: 2024
|
||||
**MSRV**: 1.89
|
||||
|
||||
## What is Iroh?
|
||||
|
||||
Iroh is a Rust library for establishing **peer-to-peer QUIC connections dialed by public key**. You provide an `EndpointAddr` (which identifies a peer), and iroh finds and maintains the fastest connection route — whether direct (hole-punched) or relayed through a server.
|
||||
|
||||
Core value propositions:
|
||||
- **Dial by public key** — no IP addresses or hostnames needed at the application layer
|
||||
- **Hole-punching** — automatically attempts direct P2P connectivity
|
||||
- **Relay fallback** — encrypted relay servers ensure connectivity even behind NATs
|
||||
- **Built on QUIC** — uses the `noq` QUIC implementation for multiplexed, encrypted streams
|
||||
- **Address Lookup** — pluggable discovery system to resolve `EndpointId → addressing info`
|
||||
|
||||
## Workspace Structure
|
||||
|
||||
```
|
||||
iroh/ # Core library (p2p QUIC connections)
|
||||
├── iroh-base/ # Fundamental types: SecretKey, PublicKey, EndpointId, RelayUrl, EndpointAddr
|
||||
├── iroh-dns/ # DNS resolver + endpoint info serialization (pkarr)
|
||||
├── iroh-dns-server/ # DNS server implementation (powers dns.iroh.link)
|
||||
├── iroh-relay/ # Relay server + client implementation
|
||||
└── iroh/bench/ # Benchmarks
|
||||
```
|
||||
|
||||
### Dependency Graph
|
||||
|
||||
```
|
||||
iroh depends on:
|
||||
├── iroh-base (key types, EndpointAddr, RelayUrl)
|
||||
├── iroh-dns (DNS resolution, EndpointInfo serialization)
|
||||
├── iroh-relay (RelayMap, RelayConfig, relay client/server, QUIC client)
|
||||
├── noq (QUIC implementation)
|
||||
├── noq-proto (QUIC protocol types)
|
||||
├── noq-udp (UDP socket abstraction)
|
||||
├── netwatch (network interface monitoring)
|
||||
├── portmapper (UPnP/PCP/NAT-PMP port mapping, optional)
|
||||
├── n0-future (async utilities)
|
||||
├── n0-watcher (watch/subscribe primitives)
|
||||
└── iroh-metrics (metrics collection)
|
||||
```
|
||||
|
||||
## Key Concepts
|
||||
|
||||
### EndpointId / PublicKey
|
||||
Every iroh endpoint has a unique Ed25519 cryptographic key pair. The public key doubles as the endpoint identifier (`EndpointId`). It's used for both:
|
||||
- **Identity** — unique addressing in the network
|
||||
- **Encryption** — TLS authentication (via RFC 7250 Raw Public Keys, no X.509 certificates)
|
||||
|
||||
### EndpointAddr
|
||||
The addressing structure that combines identity with network paths:
|
||||
```rust
|
||||
pub struct EndpointAddr {
|
||||
pub id: EndpointId, // Who to connect to
|
||||
pub addrs: BTreeSet<TransportAddr>, // How to reach them
|
||||
}
|
||||
|
||||
pub enum TransportAddr {
|
||||
Relay(RelayUrl), // Via relay server
|
||||
Ip(SocketAddr), // Direct IP address
|
||||
Custom(CustomAddr), // Via custom transport
|
||||
}
|
||||
```
|
||||
|
||||
### Relay Servers
|
||||
Relay servers provide:
|
||||
1. **Reliable connectivity** — always reachable, forward encrypted traffic to the correct endpoint by `EndpointId`
|
||||
2. **Hole-punching assistance** — QUIC Address Discovery (QAD), STUN-like services
|
||||
3. **Traffic relay** — fallback when direct connections are impossible
|
||||
|
||||
Connections to relays use HTTP/1.1 with TLS, then upgrade to a custom protocol. The relay only sees encrypted traffic.
|
||||
|
||||
### Connection Flow
|
||||
1. Endpoint binds, connects to a "home relay"
|
||||
2. To connect to peer: resolve `EndpointId` → `EndpointAddr` via Address Lookup
|
||||
3. Establish initial connection via relay
|
||||
4. Attempt direct connection (hole-punching if needed)
|
||||
5. Migrate to direct connection when available (relay becomes backup)
|
||||
|
||||
## Crate: `iroh` (Core Library)
|
||||
|
||||
### Main Types
|
||||
| Type | Module | Purpose |
|
||||
|------|--------|---------|
|
||||
| `Endpoint` | `endpoint` | Central API — connect, accept, manage connections |
|
||||
| `Builder` | `endpoint` | Configure and construct an `Endpoint` |
|
||||
| `Router` | `protocol` | Accept loop that dispatches to `ProtocolHandler`s |
|
||||
| `ProtocolHandler` | `protocol` | Trait for handling incoming connections by ALPN |
|
||||
| `Connection` | `endpoint::connection` | QUIC connection wrapper |
|
||||
| `Incoming` | `endpoint::connection` | Pre-handshake incoming connection |
|
||||
| `Accepting` | `endpoint::connection` | Post-accept, pre-handshake state |
|
||||
|
||||
### Feature Flags
|
||||
- `default` = `["metrics", "fast-apple-datapath", "portmapper", "tls-ring"]`
|
||||
- `metrics` — Prometheus-style metrics collection
|
||||
- `portmapper` — UPnP/PCP/NAT-PMP support
|
||||
- `test-utils` — Testing utilities
|
||||
- `platform-verifier` — Use OS TLS trust anchors
|
||||
- `qlog` — QUIC event logging
|
||||
- `fast-apple-datapath` — Private Apple APIs for batched sends
|
||||
- `tls-ring` / `tls-aws-lc-rs` — Choose TLS crypto backend
|
||||
- `unstable-custom-transports` — Custom transport API (unstable)
|
||||
|
||||
### WASM Support
|
||||
The crate compiles to `wasm32-unknown-unknown` for browser targets. Browser builds:
|
||||
- Use `PkarrResolver` instead of `DnsAddressLookup` (DNS-over-HTTPS)
|
||||
- Cannot bind IP sockets (no direct connectivity)
|
||||
- Use `wasm-bindgen-futures` for async runtime
|
||||
|
||||
## Presets
|
||||
|
||||
The `presets` module provides common configurations:
|
||||
|
||||
| Preset | Description |
|
||||
|--------|-------------|
|
||||
| `Empty` | No defaults — you must set all required options yourself |
|
||||
| `Minimal` | Sets only the crypto provider (ring or aws-lc-rs) |
|
||||
| `N0` | Full n0 defaults: crypto provider, Pkarr publisher, DNS resolver, n0 relay servers |
|
||||
| `N0DisableRelay` | N0 defaults but with `RelayMode::Disabled` |
|
||||
|
||||
```rust
|
||||
// Quick start with full n0 infrastructure
|
||||
let endpoint = Endpoint::bind(presets::N0).await?;
|
||||
|
||||
// Minimal — just crypto, no relay or address lookup
|
||||
let endpoint = Endpoint::bind(presets::Minimal).await?;
|
||||
```
|
||||
|
||||
## Encryption & Authentication
|
||||
|
||||
Iroh uses **RFC 7250 Raw Public Keys** for TLS — no X.509 certificates. Each endpoint has:
|
||||
- `SecretKey` (Ed25519) — used for TLS authentication and signing
|
||||
- `PublicKey`/`EndpointId` — derived from `SecretKey`, used as identity
|
||||
|
||||
The TLS server name is encoded as `<base32-dnssec-encoded-public-key>.iroh.invalid` to ensure 0-RTT session ticket separation per endpoint.
|
||||
|
||||
## 0-RTT Support
|
||||
|
||||
Iroh supports QUIC 0-RTT connections:
|
||||
- `Connecting::into_0rtt()` on the client side
|
||||
- `Accepting::into_0rtt()` on the server side
|
||||
- TLS session tickets cached per remote endpoint (default 256 tickets = ~150 KiB)
|
||||
- `max_tls_tickets()` builder option to tune cache size
|
||||
|
||||
## Default Infrastructure (n0)
|
||||
|
||||
Production relay servers (4 regions):
|
||||
| Region | Hostname |
|
||||
|--------|----------|
|
||||
| NA East | `use1-1.relay.n0.iroh-canary.iroh.link` |
|
||||
| NA West | `usw1-1.relay.n0.iroh-canary.iroh.link` |
|
||||
| EU | `euc1-1.relay.n0.iroh-canary.iroh.link` |
|
||||
| AP | `aps1-1.relay.n0.iroh-canary.iroh.link` |
|
||||
|
||||
DNS Address Lookup origin: `dns.iroh.link`
|
||||
392
docs/research/references/iroh/iroh/02-key-types-traits.md
Normal file
392
docs/research/references/iroh/iroh/02-key-types-traits.md
Normal file
@@ -0,0 +1,392 @@
|
||||
# Iroh: Key Types and Traits
|
||||
|
||||
## Core Identity Types (`iroh-base`)
|
||||
|
||||
### `SecretKey`
|
||||
Ed25519 signing key (32 bytes). Used for:
|
||||
- TLS authentication (RFC 7250 Raw Public Key)
|
||||
- Signing pkarr packets for address discovery
|
||||
- Generating the corresponding `PublicKey`/`EndpointId`
|
||||
|
||||
```rust
|
||||
// Generation
|
||||
let secret_key = SecretKey::generate();
|
||||
|
||||
// From bytes
|
||||
let secret_key = SecretKey::from_bytes(&[0u8; 32]);
|
||||
|
||||
// Access public key
|
||||
let public_key: PublicKey = secret_key.public();
|
||||
```
|
||||
|
||||
### `PublicKey` / `EndpointId`
|
||||
`EndpointId` is a type alias for `PublicKey`. Both are 32-byte Ed25519 compressed points.
|
||||
|
||||
```rust
|
||||
pub type EndpointId = PublicKey;
|
||||
|
||||
impl PublicKey {
|
||||
pub const LENGTH: usize = 32;
|
||||
pub fn from_bytes(bytes: &[u8; 32]) -> Result<Self, KeyParsingError>;
|
||||
pub fn as_bytes(&self) -> &[u8; 32];
|
||||
pub fn verify(&self, message: &[u8], signature: &Signature) -> Result<(), SignatureError>;
|
||||
pub fn fmt_short(&self) -> impl Display; // First 5 bytes hex
|
||||
}
|
||||
```
|
||||
|
||||
Serialization: Human-readable → base32 z-base-32 encoding; Binary → 32 raw bytes.
|
||||
|
||||
### `Signature`
|
||||
Ed25519 signature (64 bytes). Used in pkarr for signing endpoint discovery records.
|
||||
|
||||
### `KeyParsingError`
|
||||
Error type for key parsing failures.
|
||||
|
||||
## Addressing Types (`iroh-base`)
|
||||
|
||||
### `EndpointAddr`
|
||||
The primary addressing type — combines identity with network paths:
|
||||
|
||||
```rust
|
||||
pub struct EndpointAddr {
|
||||
pub id: EndpointId,
|
||||
pub addrs: BTreeSet<TransportAddr>,
|
||||
}
|
||||
|
||||
impl EndpointAddr {
|
||||
pub fn new(id: PublicKey) -> Self;
|
||||
pub fn from_parts(id: PublicKey, addrs: impl IntoIterator<Item = TransportAddr>) -> Self;
|
||||
pub fn with_relay_url(self, relay_url: RelayUrl) -> Self;
|
||||
pub fn with_ip_addr(self, addr: SocketAddr) -> Self;
|
||||
pub fn is_empty(&self) -> bool;
|
||||
pub fn ip_addrs(&self) -> impl Iterator<Item = &SocketAddr>;
|
||||
pub fn relay_urls(&self) -> impl Iterator<Item = &RelayUrl>;
|
||||
}
|
||||
```
|
||||
|
||||
Can be constructed from just an `EndpointId` (relies on Address Lookup), or with explicit paths:
|
||||
```rust
|
||||
// From just EndpointId — needs Address Lookup
|
||||
let addr = EndpointAddr::new(endpoint_id);
|
||||
|
||||
// With relay URL
|
||||
let addr = EndpointAddr::new(endpoint_id).with_relay_url(relay_url);
|
||||
|
||||
// With both
|
||||
let addr = EndpointAddr::from_parts(endpoint_id, [
|
||||
TransportAddr::Relay(relay_url),
|
||||
TransportAddr::Ip(socket_addr),
|
||||
]);
|
||||
```
|
||||
|
||||
### `TransportAddr`
|
||||
```rust
|
||||
pub enum TransportAddr {
|
||||
Relay(RelayUrl),
|
||||
Ip(SocketAddr),
|
||||
Custom(CustomAddr),
|
||||
}
|
||||
```
|
||||
|
||||
### `CustomAddr`
|
||||
Opaque custom transport address (for `unstable-custom-transports` feature):
|
||||
```rust
|
||||
pub struct CustomAddr {
|
||||
id: u32,
|
||||
addr: Vec<u8>,
|
||||
}
|
||||
```
|
||||
|
||||
### `RelayUrl`
|
||||
Arc-wrapped `Url` identifying a relay server. Cheaply clonable. Encourages fully-qualified DNS names (trailing dot).
|
||||
|
||||
```rust
|
||||
let url: RelayUrl = "https://use1-1.relay.n0.iroh-canary.iroh.link.".parse()?;
|
||||
```
|
||||
|
||||
## Endpoint Trait (`iroh`)
|
||||
|
||||
### `Endpoint`
|
||||
The central type — created via `Builder`, used for all connection operations:
|
||||
|
||||
```rust
|
||||
impl Endpoint {
|
||||
// Construction
|
||||
pub fn builder(preset: impl Preset) -> Builder;
|
||||
pub async fn bind(preset: impl Preset) -> Result<Self, BindError>;
|
||||
|
||||
// Connection
|
||||
pub async fn connect(&self, addr: impl Into<EndpointAddr>, alpn: &[u8]) -> Result<Connection, ConnectError>;
|
||||
pub async fn connect_with_opts(&self, addr: impl Into<EndpointAddr>, alpn: &[u8], opts: ConnectOptions) -> Result<Connecting, ConnectWithOptsError>;
|
||||
pub fn accept(&self) -> Accept<'_>;
|
||||
|
||||
// Identity
|
||||
pub fn id(&self) -> EndpointId;
|
||||
pub fn secret_key(&self) -> &SecretKey;
|
||||
pub fn addr(&self) -> EndpointAddr;
|
||||
pub fn watch_addr(&self) -> impl Watcher<Value = EndpointAddr>;
|
||||
|
||||
// Lifecycle
|
||||
pub async fn close(&self);
|
||||
pub fn is_closed(&self) -> bool;
|
||||
pub fn closed(&self) -> EndpointClosed;
|
||||
pub async fn online(&self); // Wait for relay connection
|
||||
|
||||
// Configuration changes
|
||||
pub fn set_alpns(&self, alpns: Vec<Vec<u8>>);
|
||||
pub async fn insert_relay(&self, relay: RelayUrl, config: Arc<RelayConfig>) -> Option<Arc<RelayConfig>>;
|
||||
pub async fn remove_relay(&self, relay: &RelayUrl) -> Option<Arc<RelayConfig>>;
|
||||
pub async fn add_external_addr(&self, addr: SocketAddr);
|
||||
pub async fn remove_external_addr(&self, addr: &SocketAddr) -> bool;
|
||||
pub fn set_user_data_for_address_lookup(&self, user_data: Option<UserData>);
|
||||
pub async fn network_change(&self);
|
||||
|
||||
// Observers
|
||||
pub fn home_relay_status(&self) -> impl Watcher<Value = Vec<RelayStatus>>;
|
||||
pub fn net_report(&self) -> impl Watcher<Value = Option<NetReport>>;
|
||||
pub fn remote_info(&self, id: EndpointId) -> Option<RemoteInfo>;
|
||||
pub fn metrics(&self) -> &EndpointMetrics;
|
||||
pub fn bound_sockets(&self) -> Vec<SocketAddr>;
|
||||
pub fn dns_resolver(&self) -> Result<&DnsResolver, EndpointError>;
|
||||
pub fn tls_config(&self) -> &rustls::ClientConfig;
|
||||
pub fn address_lookup(&self) -> Result<&AddressLookupServices, EndpointError>;
|
||||
}
|
||||
```
|
||||
|
||||
### `Builder`
|
||||
Fluent builder for `Endpoint`:
|
||||
|
||||
```rust
|
||||
let ep = Endpoint::builder(presets::N0)
|
||||
.secret_key(secret_key) // Identity
|
||||
.alpns(vec![b"my-alpn".to_vec()]) // Accepted protocols
|
||||
.relay_mode(RelayMode::Default) // Relay configuration
|
||||
.address_lookup(PkarrPublisher::n0_dns()) // Address discovery
|
||||
.address_lookup(DnsAddressLookup::n0_dns()) // DNS resolution
|
||||
.addr_filter(AddrFilter::relay_only()) // Filter published addresses
|
||||
.user_data_for_address_lookup(user_data) // Custom discovery data
|
||||
.transport_config(QuicTransportConfig::default()) // QUIC tuning
|
||||
.dns_resolver(dns_resolver) // Custom DNS resolver
|
||||
.proxy_url(proxy_url) // HTTP proxy
|
||||
.ca_roots_config(CaRootsConfig::default()) // TLS CA roots
|
||||
.keylog(true) // SSLKEYLOGFILE debug
|
||||
.max_tls_tickets(256) // 0-RTT ticket cache
|
||||
.hooks(my_hook) // Connection hooks
|
||||
.portmapper_config(PortmapperConfig::Enabled) // UPnP/NAT-PMP
|
||||
.external_addr(addr) // Advertised external addr
|
||||
.bind_addr("0.0.0.0:0")? // Bind specific socket
|
||||
.bind() // Build & bind
|
||||
.await?;
|
||||
```
|
||||
|
||||
### `RelayMode`
|
||||
```rust
|
||||
pub enum RelayMode {
|
||||
Disabled, // No relay
|
||||
Default, // n0 production relays
|
||||
Staging, // n0 staging relays
|
||||
Custom(RelayMap), // Custom relay configuration
|
||||
}
|
||||
```
|
||||
|
||||
## Protocol Handler (`iroh::protocol`)
|
||||
|
||||
### `ProtocolHandler`
|
||||
Trait for handling incoming connections by ALPN:
|
||||
|
||||
```rust
|
||||
pub trait ProtocolHandler: Send + Sync + Debug + 'static {
|
||||
// Optional: intercept at Accepting stage (supports 0-RTT)
|
||||
fn on_accepting(&self, accepting: Accepting) -> impl Future<Output = Result<Connection, AcceptError>> + Send;
|
||||
|
||||
// Required: handle the established connection
|
||||
fn accept(&self, connection: Connection) -> impl Future<Output = Result<(), AcceptError>> + Send;
|
||||
|
||||
// Optional: called on graceful shutdown
|
||||
fn shutdown(&self) -> impl Future<Output = ()> + Send;
|
||||
}
|
||||
```
|
||||
|
||||
### `Router`
|
||||
Spawns an accept loop that dispatches incoming connections to registered handlers:
|
||||
|
||||
```rust
|
||||
let router = Router::builder(endpoint)
|
||||
.accept(b"/my-alpn", Arc::new(MyHandler))
|
||||
.incoming_filter(|incoming| {
|
||||
if !incoming.remote_addr_validated() {
|
||||
IncomingFilterOutcome::Retry
|
||||
} else {
|
||||
IncomingFilterOutcome::Accept
|
||||
}
|
||||
})
|
||||
.spawn();
|
||||
|
||||
// Later...
|
||||
router.shutdown().await?;
|
||||
```
|
||||
|
||||
### `IncomingFilterOutcome`
|
||||
```rust
|
||||
pub enum IncomingFilterOutcome {
|
||||
Accept, // Allow the connection
|
||||
Retry, // Send QUIC retry (address validation)
|
||||
Reject, // Refuse with CONNECTION_REFUSED
|
||||
Ignore, // Drop silently (remote times out)
|
||||
}
|
||||
```
|
||||
|
||||
### `AccessLimit`
|
||||
Wrapper that limits connections to allowed `EndpointId`s:
|
||||
|
||||
```rust
|
||||
let handler = AccessLimit::new(MyHandler, |endpoint_id| allowed_set.contains(&endpoint_id));
|
||||
```
|
||||
|
||||
### `EndpointHooks`
|
||||
Intercept connection establishment at two points:
|
||||
|
||||
```rust
|
||||
pub trait EndpointHooks: Debug + Send + Sync {
|
||||
// Before outgoing connection starts
|
||||
fn before_connect<'a>(&'a self, remote_addr: &'a EndpointAddr, alpn: &'a [u8])
|
||||
-> BoxFuture<'a, BeforeConnectOutcome>;
|
||||
|
||||
// After TLS handshake completes (on both sides)
|
||||
fn after_handshake<'a>(&'a self, info: &'a ConnectionInfo)
|
||||
-> BoxFuture<'a, AfterHandshakeOutcome>;
|
||||
}
|
||||
```
|
||||
|
||||
## Connection Types (`iroh::endpoint::connection`)
|
||||
|
||||
### `Connecting`
|
||||
The state between initiating a connection and completing the handshake:
|
||||
|
||||
```rust
|
||||
impl Connecting {
|
||||
pub async fn await?(self) -> Result<Connection, ConnectingError>;
|
||||
pub fn into_0rtt(self) -> Result<(OutgoingZeroRttConnection, Connection), Connecting>;
|
||||
pub fn alpn(&self) -> Result<Vec<u8>, ConnectingError>;
|
||||
pub fn remote_id(&self) -> Result<EndpointId, RemoteEndpointIdError>;
|
||||
}
|
||||
```
|
||||
|
||||
### `Connection`
|
||||
Wraps a `noq::Connection` with iroh-specific metadata:
|
||||
|
||||
```rust
|
||||
impl Connection {
|
||||
// Stream operations
|
||||
pub async fn open_bi(&self) -> Result<(SendStream, RecvStream), OpenBi>;
|
||||
pub async fn accept_bi(&self) -> Result<(SendStream, RecvStream), AcceptBi>;
|
||||
pub async fn open_uni(&self) -> Result<SendStream, OpenUni>;
|
||||
pub async fn accept_uni(&self) -> Result<RecvStream, AcceptUni>;
|
||||
|
||||
// Datagrams
|
||||
pub fn send_datagram(&self, data: SendDatagram) -> Result<(), SendDatagramError>;
|
||||
pub async fn read_datagram(&self) -> Result<Bytes, ReadDatagram>;
|
||||
|
||||
// Connection lifecycle
|
||||
pub fn close(&self, error_code: VarInt, reason: &[u8]);
|
||||
pub async fn closed(&self) -> ConnectionError;
|
||||
|
||||
// Identity
|
||||
pub fn remote_id(&self) -> EndpointId;
|
||||
pub fn alpn(&self) -> Vec<u8>;
|
||||
|
||||
// Path observation
|
||||
pub fn paths(&self) -> PathWatcher;
|
||||
|
||||
// Keying material export
|
||||
pub fn export_keying_material(&self, output: &mut [u8], label: &[u8], context: Option<&[u8]>) -> Result<(), ExportKeyingMaterialError>;
|
||||
}
|
||||
```
|
||||
|
||||
### `Incoming`
|
||||
Pre-accept incoming connection:
|
||||
|
||||
```rust
|
||||
impl Incoming {
|
||||
pub fn accept(self) -> Result<Accepting, ConnectionError>;
|
||||
pub fn accept_with(self, server_config: Arc<ServerConfig>) -> Result<Accepting, ConnectionError>;
|
||||
pub fn refuse(self);
|
||||
pub fn retry(self) -> Result<(), RetryError>;
|
||||
pub fn ignore(self);
|
||||
pub fn remote_addr(&self) -> IncomingAddr;
|
||||
pub fn local_ip(&self) -> Option<IpAddr>;
|
||||
pub fn remote_addr_validated(&self) -> bool;
|
||||
pub fn decrypt(&self) -> Option<DecryptedInitial>;
|
||||
}
|
||||
```
|
||||
|
||||
### `IncomingAddr`
|
||||
```rust
|
||||
pub enum IncomingAddr {
|
||||
Ip(SocketAddr),
|
||||
Relay { url: RelayUrl, endpoint_id: EndpointId },
|
||||
Custom(CustomAddr),
|
||||
}
|
||||
```
|
||||
|
||||
## `RelayMap` and `RelayConfig` (`iroh-relay`)
|
||||
|
||||
### `RelayMap`
|
||||
Thread-safe map of relay servers:
|
||||
|
||||
```rust
|
||||
let map = RelayMap::from_iter([
|
||||
"https://relay1.example.org".parse()?,
|
||||
"https://relay2.example.org".parse()?,
|
||||
]);
|
||||
```
|
||||
|
||||
### `RelayConfig`
|
||||
```rust
|
||||
pub struct RelayConfig {
|
||||
pub url: RelayUrl,
|
||||
pub quic: Option<RelayQuicConfig>, // QAD support
|
||||
}
|
||||
|
||||
pub struct RelayQuicConfig {
|
||||
pub port: u16, // Default: 3478
|
||||
}
|
||||
```
|
||||
|
||||
## `EndpointData` and `EndpointInfo` (`iroh-dns`)
|
||||
|
||||
### `EndpointData`
|
||||
The data published about an endpoint:
|
||||
|
||||
```rust
|
||||
pub struct EndpointData {
|
||||
addrs: Vec<TransportAddr>,
|
||||
user_data: Option<UserData>,
|
||||
}
|
||||
```
|
||||
|
||||
### `EndpointInfo`
|
||||
Combines `EndpointId` with `EndpointData`:
|
||||
|
||||
```rust
|
||||
pub struct EndpointInfo {
|
||||
pub endpoint_id: EndpointId,
|
||||
pub data: EndpointData,
|
||||
}
|
||||
```
|
||||
|
||||
### `UserData`
|
||||
Application-defined string data published alongside addressing info:
|
||||
|
||||
```rust
|
||||
pub struct UserData(String); // Max 256 bytes
|
||||
```
|
||||
|
||||
### `AddrFilter`
|
||||
Controls which addresses are published to address lookup services:
|
||||
|
||||
```rust
|
||||
let filter = AddrFilter::relay_only(); // Only relay URLs
|
||||
let filter = AddrFilter::unfiltered(); // All addresses
|
||||
let filter = AddrFilter::custom(|addrs| { /* custom logic */ });
|
||||
```
|
||||
401
docs/research/references/iroh/iroh/03-networking-protocols.md
Normal file
401
docs/research/references/iroh/iroh/03-networking-protocols.md
Normal file
@@ -0,0 +1,401 @@
|
||||
# Iroh: Networking & Protocol Details
|
||||
|
||||
## Connection Establishment
|
||||
|
||||
### Overview
|
||||
The connection process follows this sequence:
|
||||
|
||||
```
|
||||
Caller Callee
|
||||
| |
|
||||
|--- connect(EndpointAddr, alpn) -------->| (via relay first)
|
||||
| |
|
||||
|<------ TLS Handshake (Raw Public Key) ->|
|
||||
| |
|
||||
|<====== QUIC Connection Established ====|
|
||||
| |
|
||||
| (iroh attempts direct path migration) |
|
||||
| |
|
||||
|--- open_bi() / open_uni() ------------->|
|
||||
|<--- accept_bi() / accept_uni() ----------|
|
||||
```
|
||||
|
||||
### Step-by-Step
|
||||
|
||||
1. **Resolve addressing** — `resolve_remote(EndpointAddr)` starts a `RemoteStateActor` for the peer. If no direct addresses or relay URL are provided, Address Lookup services are queried.
|
||||
|
||||
2. **Map addresses** — `EndpointId` is mapped to a synthetic IPv6 address for the QUIC layer (`EndpointIdMappedAddr`). Relay and custom transport addresses are similarly mapped.
|
||||
|
||||
3. **TLS connection** — Uses RFC 7250 Raw Public Keys. The server name is encoded as `<z32-encoded-pubkey>.iroh.invalid`. Both sides authenticate by `EndpointId`.
|
||||
|
||||
4. **ALPN negotiation** — The Application-Layer Protocol Negotiation determines which protocol handler receives the connection.
|
||||
|
||||
5. **Path migration** — Once a QUIC connection is established (initially via relay), iroh continuously searches for better paths. Direct IP paths are preferred when available.
|
||||
|
||||
## Transport Layer Architecture
|
||||
|
||||
### The `Socket` — Core Connectivity Engine
|
||||
|
||||
The `Socket` struct is the heart of iroh's networking. It manages:
|
||||
- Multiple transport paths (IPv4, IPv6, relay, custom)
|
||||
- Address discovery and NAT traversal
|
||||
- Path migration between relay and direct connections
|
||||
|
||||
```
|
||||
┌──────────────┐
|
||||
│ Endpoint │ (Public API)
|
||||
│ (Arc<EndpointInner>) │
|
||||
└──────┬───────┘
|
||||
│
|
||||
┌──────▼───────┐
|
||||
│ Socket │ (Connectivity engine)
|
||||
│ (Arc<Socket>) │
|
||||
└──────┬───────┘
|
||||
│
|
||||
┌────────────┼────────────┐
|
||||
│ │ │
|
||||
┌─────▼─────┐ ┌───▼────┐ ┌──────▼──────┐
|
||||
│IpTransport│ │Relay │ │CustomTransport│
|
||||
│(IPv4/v6) │ │Transport│ │(unstable) │
|
||||
└─────┬─────┘ └───┬────┘ └──────┬──────┘
|
||||
│ │ │
|
||||
┌─────▼─────┐ ┌───▼────┐ │
|
||||
│ UdpSocket │ │WebSocket│ │
|
||||
│ (netwatch)│ │ Actor │ │
|
||||
└────────────┘ └────────┘ │
|
||||
```
|
||||
|
||||
### Transport Configuration
|
||||
|
||||
```rust
|
||||
pub enum TransportConfig {
|
||||
Ip {
|
||||
config: IpConfig, // IPv4 or IPv6 socket config
|
||||
is_user_defined: bool,
|
||||
},
|
||||
Relay {
|
||||
relay_map: RelayMap, // Which relay servers to use
|
||||
is_user_defined: bool,
|
||||
},
|
||||
#[cfg(feature = "unstable-custom-transports")]
|
||||
Custom(Arc<dyn CustomTransport>),
|
||||
}
|
||||
|
||||
pub enum IpConfig {
|
||||
V4 { ip_net: Ipv4Net, port: u16, is_required: bool, is_default: bool },
|
||||
V6 { ip_net: Ipv6Net, scope_id: u32, port: u16, is_required: bool, is_default: bool },
|
||||
}
|
||||
```
|
||||
|
||||
### Address Mapping
|
||||
|
||||
Iroh maps all transport addresses to IPv6 for the QUIC layer:
|
||||
|
||||
- **IPv4/IPv6 addresses** → used directly as QUIC path addresses
|
||||
- **Relay addresses** → mapped to synthetic IPv6 addresses in a dedicated range
|
||||
- **Custom addresses** → mapped to synthetic IPv6 addresses in another range
|
||||
|
||||
The `MappedAddrs` struct maintains these mappings:
|
||||
```rust
|
||||
pub(crate) struct MappedAddrs {
|
||||
pub(super) endpoint_addrs: AddrMap<EndpointId, EndpointIdMappedAddr>,
|
||||
pub(super) relay_addrs: AddrMap<(RelayUrl, EndpointId), RelayMappedAddr>,
|
||||
pub(super) custom_addrs: AddrMap<CustomAddr, CustomMappedAddr>,
|
||||
}
|
||||
```
|
||||
|
||||
### Transport Bias
|
||||
|
||||
Path selection uses a configurable bias system:
|
||||
|
||||
```rust
|
||||
let endpoint = Endpoint::builder(presets::N0)
|
||||
.transport_bias(AddrKind::Custom(42), TransportBias::primary())
|
||||
.bind()
|
||||
.await?;
|
||||
```
|
||||
|
||||
Default biases:
|
||||
- IPv4 and IPv6 are **primary** (IPv6 gets small RTT advantage)
|
||||
- Relay is **backup** (only used when no primary transport available)
|
||||
|
||||
## Relay Protocol
|
||||
|
||||
### Architecture
|
||||
|
||||
The relay system is based on a revised version of Tailscale's DERP (Designated Encrypted Relay for Packets) protocol.
|
||||
|
||||
```
|
||||
Client A Relay Server Client B
|
||||
│ │ │
|
||||
│─── HTTP CONNECT ──>| │
|
||||
│<── 200 OK ─────────│ │
|
||||
│ │<─── HTTP CONNECT ────│
|
||||
│ │──── 200 OK ────────>│
|
||||
│ │ │
|
||||
│─── Encrypted QUIC ─>│─── Encrypted QUIC ─>│
|
||||
│<── Encrypted QUIC ──│<── Encrypted QUIC ──│
|
||||
```
|
||||
|
||||
### Relay Actor
|
||||
|
||||
The `RelayActor` manages the WebSocket connection to the relay:
|
||||
- Connects to relay via HTTPS, upgrades to custom protocol
|
||||
- Sends/receives encrypted datagrams on behalf of the local endpoint
|
||||
- Manages reconnection on network changes or relay restarts
|
||||
- Reports connection status via `HomeRelayWatch`
|
||||
|
||||
### Relay Data Flow
|
||||
1. Outgoing packet → `RelayTransport::send()` → `RelayActor` → WebSocket → Relay server → WebSocket → remote `RelayActor` → remote `RelayTransport::recv()` → QUIC
|
||||
2. The relay only sees encrypted QUIC packets — it cannot decode application data
|
||||
|
||||
### Home Relay Selection
|
||||
|
||||
The `net_report` module continuously probes relay servers and maintains latency statistics. The "home relay" is selected based on:
|
||||
- Lowest recent latency (with hysteresis to avoid flapping)
|
||||
- At most a 2/3 improvement threshold to switch from current relay
|
||||
|
||||
## Hole-Punching & NAT Traversal
|
||||
|
||||
### QUIC Address Discovery (QAD)
|
||||
|
||||
Iroh uses QUIC Address Discovery (based on [draft-ietf-quic-address-discovery](https://datatracker.ietf.org/doc/draft-ietf-quic-address-discovery/)) to discover external IP addresses. The relay servers expose QAD endpoints.
|
||||
|
||||
The `net_report` module:
|
||||
1. Establishes QUIC connections to relay servers
|
||||
2. Uses `observed_external_addr()` to learn external addresses
|
||||
3. Reports NAT type, mapping behavior, and preferred relay
|
||||
|
||||
### NAT Traversal Strategy
|
||||
|
||||
```
|
||||
┌──────────────────────────────┐
|
||||
│ NAT Traversal │
|
||||
│ │
|
||||
│ 1. Direct connection attempt │
|
||||
│ (simultaneous open) │
|
||||
│ │
|
||||
│ 2. QAD-discovered addresses │
|
||||
│ (relay reports observed IP)│
|
||||
│ │
|
||||
│ 3. Port mapping (UPnP/PCP/NAT-PMP)│
|
||||
│ (if supported by gateway) │
|
||||
│ │
|
||||
│ 4. Relay fallback │
|
||||
│ (always available) │
|
||||
└──────────────────────────────┘
|
||||
```
|
||||
|
||||
### Port Mapper
|
||||
|
||||
```rust
|
||||
pub enum PortmapperConfig {
|
||||
Enabled {}, // Default: tries UPnP, PCP, NAT-PMP
|
||||
Disabled, // No port mapping
|
||||
}
|
||||
```
|
||||
|
||||
When enabled, the port mapper:
|
||||
- Discovers gateway devices
|
||||
- Requests port mappings
|
||||
- Provides external addresses to the endpoint
|
||||
- Updates when mappings change
|
||||
|
||||
### Net Report
|
||||
|
||||
`NetReport` discovers network conditions:
|
||||
- IPv4/IPv6 connectivity
|
||||
- NAT mapping behavior (varies by destination or not)
|
||||
- Captive portal detection
|
||||
- Preferred relay selection
|
||||
- External IP addresses (via QAD)
|
||||
|
||||
Key timeouts:
|
||||
- `NET_REPORT_TIMEOUT` = 10 seconds
|
||||
- `FULL_REPORT_INTERVAL` = 5 minutes
|
||||
- `HEARTBEAT_INTERVAL` = 5 seconds (keepalive)
|
||||
- `PATH_MAX_IDLE_TIMEOUT` = 15 seconds (direct)
|
||||
- `RELAY_PATH_MAX_IDLE_TIMEOUT` = 30 seconds (relay)
|
||||
|
||||
## Address Lookup System
|
||||
|
||||
### Trait Definition
|
||||
|
||||
```rust
|
||||
pub trait AddressLookup: Debug + Send + Sync + 'static {
|
||||
fn publish(&self, data: &EndpointData);
|
||||
fn resolve(&self, endpoint_id: EndpointId) -> Option<BoxStream<Result<Item, Error>>>;
|
||||
}
|
||||
```
|
||||
|
||||
### `AddressLookupServices`
|
||||
A composite that runs multiple lookup services concurrently:
|
||||
|
||||
```rust
|
||||
let services = AddressLookupServices::default();
|
||||
services.set_addr_filter(AddrFilter::relay_only());
|
||||
services.add(publisher);
|
||||
services.add(resolver);
|
||||
```
|
||||
|
||||
Resolution merges results from all services. Individual service errors don't block other services.
|
||||
|
||||
### Built-in Implementations
|
||||
|
||||
#### `PkarrPublisher`
|
||||
Publishes endpoint info to a pkarr relay via HTTP PUT:
|
||||
```rust
|
||||
let publisher = PkarrPublisher::builder(pkarr_url)
|
||||
.addr_filter(AddrFilter::relay_only()) // Default: relay-only
|
||||
.build(secret_key, tls_config);
|
||||
```
|
||||
|
||||
#### `PkarrResolver` (browser/WASM)
|
||||
Resolves endpoint info from a pkarr relay via HTTP GET.
|
||||
|
||||
#### `DnsAddressLookup` (non-browser)
|
||||
Resolves endpoint info via DNS TXT records:
|
||||
```rust
|
||||
// Default n0 DNS
|
||||
let lookup = DnsAddressLookup::n0_dns();
|
||||
|
||||
// Custom DNS origin
|
||||
let lookup = DnsAddressLookup::new(dns_resolver, origin);
|
||||
```
|
||||
|
||||
#### `MemoryLookup`
|
||||
In-memory address lookup for testing:
|
||||
```rust
|
||||
let lookup = MemoryLookup::new();
|
||||
lookup.add_endpoint(endpoint_id, endpoint_data);
|
||||
```
|
||||
|
||||
### DNS Record Format
|
||||
```
|
||||
_iroh.<z32-encoded-endpoint-id>.<origin-domain> TXT
|
||||
```
|
||||
Attributes:
|
||||
- `relay=<url>` — Home relay URL
|
||||
- `addr=<addr> <addr>` — Space-separated socket addresses
|
||||
- `user_data=<base64-encoded-data>` — Application-specific data
|
||||
|
||||
## TLS Configuration
|
||||
|
||||
### `TlsConfig`
|
||||
Manages TLS state shared across sessions:
|
||||
```rust
|
||||
struct TlsConfig {
|
||||
secret_key: SecretKey,
|
||||
cert_resolver: Arc<ResolveRawPublicKeyCert>,
|
||||
server_verifier: Arc<ServerCertificateVerifier>,
|
||||
client_verifier: Arc<ClientCertificateVerifier>,
|
||||
session_store: Arc<dyn ClientSessionStore>,
|
||||
crypto_provider: Arc<CryptoProvider>,
|
||||
}
|
||||
```
|
||||
|
||||
### Raw Public Key Certificate
|
||||
Uses RFC 7250 — no X.509 certificates. The `ResolveRawPublicKeyCert` resolver creates TLS certificates on-the-fly from the Ed25519 public key.
|
||||
|
||||
### Verification Flow
|
||||
- **Client verifies server**: The `ServerCertificateVerifier` checks that the server's `EndpointId` matches the expected `EndpointId` encoded in the TLS server name.
|
||||
- **Server verifies client**: The `ClientCertificateVerifier` ensures the client presents a valid raw public key.
|
||||
|
||||
### Crypto Providers
|
||||
Two built-in options via feature flags:
|
||||
- `tls-ring` — uses `ring` crypto (default)
|
||||
- `tls-aws-lc-rs` — uses AWS LC-RS crypto
|
||||
|
||||
Custom providers can be set via `Builder::crypto_provider()`.
|
||||
|
||||
## Multipath & Path Migration
|
||||
|
||||
Iroh supports QUIC multipath connections. Multiple paths can be active simultaneously:
|
||||
|
||||
```rust
|
||||
// Watch path changes
|
||||
let paths = connection.paths();
|
||||
while let Some(infos) = paths.stream().next().await {
|
||||
for info in infos.iter() {
|
||||
if info.is_ip() { /* direct path */ }
|
||||
if info.is_relay() { /* relay path */ }
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Maximum multipath paths per connection: 12 (`MAX_MULTIPATH_PATHS`).
|
||||
|
||||
### Path Types
|
||||
```rust
|
||||
pub struct PathInfo {
|
||||
pub addr: TransportAddr,
|
||||
pub usage: TransportAddrUsage,
|
||||
}
|
||||
|
||||
pub enum TransportAddrUsage {
|
||||
DefaultRoute,
|
||||
SubnetRoute,
|
||||
Backup,
|
||||
}
|
||||
```
|
||||
|
||||
## Connection Hooks
|
||||
|
||||
```rust
|
||||
#[derive(Debug, Clone)]
|
||||
struct MyHook;
|
||||
|
||||
impl EndpointHooks for MyHook {
|
||||
fn before_connect<'a>(
|
||||
&'a self,
|
||||
remote_addr: &'a EndpointAddr,
|
||||
alpn: &'a [u8],
|
||||
) -> BoxFuture<'a, BeforeConnectOutcome> {
|
||||
Box::pin(async move {
|
||||
if is_allowed(remote_addr.id()) {
|
||||
BeforeConnectOutcome::Accept
|
||||
} else {
|
||||
BeforeConnectOutcome::Reject
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
fn after_handshake<'a>(
|
||||
&'a self,
|
||||
info: &'a ConnectionInfo,
|
||||
) -> BoxFuture<'a, AfterHandshakeOutcome> {
|
||||
Box::pin(async move {
|
||||
AfterHandshakeOutcome::Accept
|
||||
})
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Custom Transports (Unstable)
|
||||
|
||||
```rust
|
||||
pub trait CustomTransport: Send + Sync + Debug + 'static {
|
||||
// Create an endpoint for this transport
|
||||
fn create_endpoint(&self, config: CustomEndpointConfig) -> Result<Arc<dyn CustomEndpoint>, CustomTransportError>;
|
||||
}
|
||||
|
||||
pub trait CustomEndpoint: Send + Sync + Debug + 'static {
|
||||
fn send(&self, item: CustomSendItem) -> Result<(), CustomTransportError>;
|
||||
fn recv(&self) -> Result<CustomRecvItem, CustomTransportError>;
|
||||
}
|
||||
|
||||
// Register:
|
||||
let ep = Endpoint::builder(presets::N0)
|
||||
.add_custom_transport(Arc::new(MyTransport))
|
||||
.bind()
|
||||
.await?;
|
||||
```
|
||||
|
||||
Transport IDs (from `TRANSPORTS.md`):
|
||||
|
||||
| ID | Transport | Address format |
|
||||
|----|-----------|---------------|
|
||||
| `0x00-0x1F` | Reserved | - |
|
||||
| `0x20` | Test | Ed25519 public key (32 bytes) |
|
||||
| `0x544F52` | Tor | Ed25519 public key (32 bytes) |
|
||||
| `0x424C45` | BLE | Bluetooth MAC address (6 bytes) |
|
||||
294
docs/research/references/iroh/iroh/04-sub-crates.md
Normal file
294
docs/research/references/iroh/iroh/04-sub-crates.md
Normal file
@@ -0,0 +1,294 @@
|
||||
# Iroh: Sub-Crates
|
||||
|
||||
## `iroh-base`
|
||||
|
||||
**Purpose**: Fundamental types shared across all iroh crates.
|
||||
**Features**: `key` (default), `relay` (default)
|
||||
|
||||
### Key Types
|
||||
|
||||
| Type | Description |
|
||||
|------|-------------|
|
||||
| `SecretKey` | Ed25519 signing key (32 bytes). Generated randomly or from bytes. |
|
||||
| `PublicKey` | Ed25519 public key (32 bytes). Verifies signatures. |
|
||||
| `EndpointId` | Type alias for `PublicKey` — used as network identity. |
|
||||
| `Signature` | Ed25519 signature (64 bytes). |
|
||||
| `RelayUrl` | Arc-wrapped `Url` identifying a relay server. |
|
||||
| `EndpointAddr` | Combines `EndpointId` + `BTreeSet<TransportAddr>`. Primary addressing type. |
|
||||
| `TransportAddr` | Enum: `Relay(RelayUrl)`, `Ip(SocketAddr)`, `Custom(CustomAddr)`. |
|
||||
| `CustomAddr` | Opaque address for custom transports (id + bytes). |
|
||||
| `KeyParsingError` | Error type for key parsing. |
|
||||
| `RelayUrlParseError` | Error type for URL parsing. |
|
||||
|
||||
### `EndpointAddr` Methods
|
||||
|
||||
```rust
|
||||
impl EndpointAddr {
|
||||
pub fn new(id: PublicKey) -> Self;
|
||||
pub fn from_parts(id: PublicKey, addrs: impl IntoIterator<Item = TransportAddr>) -> Self;
|
||||
pub fn with_relay_url(self, relay_url: RelayUrl) -> Self;
|
||||
pub fn with_ip_addr(self, addr: SocketAddr) -> Self;
|
||||
pub fn with_addrs(self, addrs: impl IntoIterator<Item = TransportAddr>) -> Self;
|
||||
pub fn is_empty(&self) -> bool;
|
||||
pub fn ip_addrs(&self) -> impl Iterator<Item = &SocketAddr>;
|
||||
pub fn relay_urls(&self) -> impl Iterator<Item = &RelayUrl>;
|
||||
}
|
||||
```
|
||||
|
||||
### Serialization
|
||||
- `PublicKey`/`EndpointId`: Human-readable → base32 z-base-32; Binary → 32 raw bytes
|
||||
- `EndpointAddr`: Serialized as `{id, addrs}` with `TransportAddr` as tagged enum
|
||||
- `RelayUrl`: Serialized as URL string
|
||||
|
||||
---
|
||||
|
||||
## `iroh-dns`
|
||||
|
||||
**Purpose**: DNS resolver and endpoint info serialization for address discovery.
|
||||
**Key Features**: pkarr signed packet creation/verification, DNS TXT record parsing, configurable DNS resolver.
|
||||
|
||||
### Modules
|
||||
|
||||
| Module | Description |
|
||||
|--------|-------------|
|
||||
| `dns` | `DnsResolver` — configurable async DNS resolver with IPv4/IPv6 staggered lookup |
|
||||
| `endpoint_info` | `EndpointInfo`, `EndpointData`, `AddrFilter`, `UserData` — serialization/deserialization |
|
||||
| `pkarr` | Pkarr signed packet creation and verification |
|
||||
| `attrs` | Low-level TXT record attribute parsing |
|
||||
|
||||
### `DnsResolver`
|
||||
|
||||
```rust
|
||||
impl DnsResolver {
|
||||
pub fn new() -> Self;
|
||||
pub fn with_nameserver(addr: SocketAddr) -> Self;
|
||||
pub fn with_nameservers(addrs: Vec<SocketAddr>) -> Self;
|
||||
|
||||
// Lookup methods
|
||||
pub async fn lookup_ipv4(&self, host: String) -> Result<...>;
|
||||
pub async fn lookup_ipv6(&self, host: String) -> Result<...>;
|
||||
pub async fn lookup_ipv4_ipv6_staggered(&self, host: &str, timeout: Duration, delays: &[u64]) -> Result<...>;
|
||||
pub async fn lookup_txt(&self, host: String) -> Result<...>;
|
||||
pub async fn lookup_endpoint_by_id(&self, id: &EndpointId, origin: &str) -> Result<EndpointInfo>;
|
||||
|
||||
// Cache management
|
||||
pub fn clear_cache(&self);
|
||||
pub fn reset_resolver(&self);
|
||||
}
|
||||
```
|
||||
|
||||
### `EndpointInfo` & `EndpointData`
|
||||
|
||||
```rust
|
||||
pub struct EndpointInfo {
|
||||
pub endpoint_id: EndpointId,
|
||||
pub data: EndpointData,
|
||||
}
|
||||
|
||||
pub struct EndpointData {
|
||||
addrs: Vec<TransportAddr>,
|
||||
user_data: Option<UserData>,
|
||||
}
|
||||
|
||||
impl EndpointData {
|
||||
pub fn new(addrs: Vec<TransportAddr>) -> Self;
|
||||
pub fn from_iter(addrs: impl IntoIterator<Item = TransportAddr>) -> Self;
|
||||
pub fn with_user_data(mut self, user_data: UserData) -> Self;
|
||||
pub fn addrs(&self) -> impl Iterator<Item = &TransportAddr>;
|
||||
pub fn user_data(&self) -> Option<&UserData>;
|
||||
pub fn apply_filter(&self, filter: &AddrFilter) -> Cow<'_, EndpointData>;
|
||||
}
|
||||
```
|
||||
|
||||
### `AddrFilter`
|
||||
|
||||
Controls which addresses are published in address lookup:
|
||||
|
||||
```rust
|
||||
pub enum AddrFilter {
|
||||
RelayOnly, // Only relay URLs
|
||||
Unfiltered, // All addresses
|
||||
Custom(fn(&[TransportAddr]) -> Vec<TransportAddr>),
|
||||
}
|
||||
```
|
||||
|
||||
### Pkarr Integration
|
||||
|
||||
```rust
|
||||
// Creating signed packets
|
||||
let info = EndpointInfo::new(secret_key.public())
|
||||
.with_relay_url(relay_url);
|
||||
let packet = info.to_pkarr_signed_packet(&secret_key, 30)?; // 30 second TTL
|
||||
|
||||
// Verifying and extracting
|
||||
let info = EndpointInfo::from_pkarr_signed_packet(&packet)?;
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## `iroh-relay`
|
||||
|
||||
**Purpose**: Relay server and client implementation. Provides DERP-like relay protocol, QAD support, and relay server binary.
|
||||
|
||||
### Key Exports
|
||||
|
||||
| Type | Description |
|
||||
|------|-------------|
|
||||
| `RelayMap` | Thread-safe map of `RelayUrl → RelayConfig` |
|
||||
| `RelayConfig` | Configuration for a single relay server |
|
||||
| `RelayQuicConfig` | QUIC address discovery configuration |
|
||||
| `KeyCache` | Cache for relay server public keys |
|
||||
| `PingTracker` | Ping/pong tracking for relay connections |
|
||||
| `MAX_PACKET_SIZE` | Maximum relay packet size (64KB - overhead) |
|
||||
|
||||
### Modules
|
||||
|
||||
| Module | Description |
|
||||
|--------|-------------|
|
||||
| `client` | HTTP client for relay server connections |
|
||||
| `http` | HTTP-related relay functionality |
|
||||
| `protos` | Protocol definitions (handshake, relay, streams) |
|
||||
| `quic` | QUIC client for QAD probing |
|
||||
| `server` | Full relay server implementation (`feature = "server"`) |
|
||||
| `tls` | TLS configuration utilities |
|
||||
|
||||
### `RelayConfig`
|
||||
|
||||
```rust
|
||||
pub struct RelayConfig {
|
||||
pub url: RelayUrl,
|
||||
pub quic: Option<RelayQuicConfig>,
|
||||
}
|
||||
|
||||
impl RelayConfig {
|
||||
pub fn new(url: RelayUrl, quic: Option<RelayQuicConfig>) -> Self;
|
||||
pub fn from(url: RelayUrl) -> Self; // No QAD
|
||||
}
|
||||
```
|
||||
|
||||
### `RelayMap`
|
||||
|
||||
```rust
|
||||
impl RelayMap {
|
||||
pub fn empty() -> Self;
|
||||
pub fn from(relay: RelayConfig) -> Self;
|
||||
pub fn from_iter(iter: impl IntoIterator<Item = impl Into<RelayConfig>>) -> Self;
|
||||
pub fn try_from_iter(iter: impl IntoIterator<Item = &str>) -> Result<Self, RelayUrlParseError>;
|
||||
pub fn insert(&self, url: RelayUrl, config: Arc<RelayConfig>) -> Option<Arc<RelayConfig>>;
|
||||
pub fn remove(&self, url: &RelayUrl) -> Option<Arc<RelayConfig>>;
|
||||
pub fn len(&self) -> usize;
|
||||
pub fn is_empty(&self) -> bool;
|
||||
pub fn urls<T: FromIterator<RelayUrl>>(&self) -> T;
|
||||
pub fn relays<T: FromIterator<Arc<RelayConfig>>>(&self) -> T;
|
||||
}
|
||||
```
|
||||
|
||||
### Relay Protocol (DERP-like)
|
||||
|
||||
The relay protocol is based on Tailscale's DERP protocol, adapted for iroh:
|
||||
|
||||
1. Client connects via HTTPS, upgrades to custom protocol
|
||||
2. Authentication via raw public key (Ed25519)
|
||||
3. Encrypted datagram forwarding by `EndpointId`
|
||||
4. QAD probes via QUIC for address discovery
|
||||
5. Ping/pong keepalive mechanism
|
||||
|
||||
### TLS Utilities
|
||||
|
||||
```rust
|
||||
pub use iroh_relay::tls::{CaRootsConfig, default_provider};
|
||||
|
||||
// Skip certificate verification (testing only)
|
||||
let config = CaRootsConfig::insecure_skip_verify();
|
||||
|
||||
// Use system trust roots
|
||||
let config = CaRootsConfig::platform_verifier();
|
||||
|
||||
// Use specific roots
|
||||
let config = CaRootsConfig::from_pem(pem_bytes);
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## `iroh-dns-server`
|
||||
|
||||
**Purpose**: DNS server that resolves iroh `EndpointId`s to addressing information. Powers `dns.iroh.link`.
|
||||
|
||||
### Key Features
|
||||
- Serves DNS TXT records for `_iroh.<z32-endpoint-id>.<origin>` queries
|
||||
- Integrates with pkarr for signed record verification
|
||||
- Supports production (`dns.iroh.link`) and staging (`staging-dns.iroh.link`) origins
|
||||
- Includes benchmarking support
|
||||
|
||||
### Configuration Files
|
||||
- `config.dev.toml` — Development configuration
|
||||
- `config.prod.toml` — Production configuration
|
||||
|
||||
---
|
||||
|
||||
## Internal Modules in `iroh` Crate
|
||||
|
||||
### `socket` Module
|
||||
The connectivity layer — manages the `Socket` struct that orchestrates:
|
||||
- Multiple transport paths
|
||||
- Network change detection
|
||||
- Address discovery and publication
|
||||
- Remote state actors (per-peer state machines)
|
||||
|
||||
**Key sub-modules**:
|
||||
|
||||
| Sub-module | Description |
|
||||
|-----------|-------------|
|
||||
| `transports/` | Transport implementations (IP, relay, custom) |
|
||||
| `transports/ip.rs` | IPv4/IPv6 UDP transport |
|
||||
| `transports/relay.rs` | Relay WebSocket transport |
|
||||
| `transports/relay/actor.rs` | Relay connection management actor |
|
||||
| `transports/custom.rs` | Unstable custom transport API |
|
||||
| `remote_map.rs` | Per-peer `RemoteStateActor` management |
|
||||
| `remote_map/remote_state.rs` | State machine for connecting to a peer |
|
||||
| `mapped_addrs.rs` | Address mapping for QUIC layer |
|
||||
| `concurrent_read_map.rs` | Lock-free concurrent map for remote actors |
|
||||
| `metrics.rs` | Socket-level metrics |
|
||||
|
||||
### `net_report` Module
|
||||
Network condition reporter:
|
||||
- Discovers external IP addresses (QAD)
|
||||
- Measures relay latencies
|
||||
- Detects NAT types
|
||||
- Detects captive portals
|
||||
- Selects preferred relay
|
||||
|
||||
### `portmapper` Module
|
||||
UPnP/PCP/NAT-PMP port mapping:
|
||||
- Gateway discovery
|
||||
- Port mapping procurement
|
||||
- External address monitoring
|
||||
|
||||
### `address_lookup` Module
|
||||
Pluggable address discovery:
|
||||
|
||||
| Sub-module | Description |
|
||||
|-----------|-------------|
|
||||
| `dns.rs` | `DnsAddressLookup` — resolves via DNS TXT records |
|
||||
| `pkarr.rs` | `PkarrPublisher` — publishes via HTTP PUT to pkarr relay; `PkarrResolver` — resolves from pkarr relay |
|
||||
| `memory.rs` | `MemoryLookup` — in-memory lookup for testing |
|
||||
|
||||
### `runtime` Module
|
||||
Tokio-based async runtime wrapper for `noq`:
|
||||
- Task spawning with cancellation support
|
||||
- Timer management
|
||||
- Graceful and abrupt shutdown
|
||||
- WASM browser support (delegates to `wasm-bindgen-futures`)
|
||||
|
||||
### `defaults` Module
|
||||
Default configuration values:
|
||||
- Production relay servers (4 regions)
|
||||
- Staging relay servers (2 regions)
|
||||
- Timeout constants
|
||||
- Environment variable for forcing staging (`IROH_FORCE_STAGING_RELAYS`)
|
||||
|
||||
### `metrics` Module
|
||||
`EndpointMetrics` collection:
|
||||
- Socket metrics (datagrams sent/received, data by transport type)
|
||||
- Net report metrics (reports generated, full vs incremental)
|
||||
- Port mapper metrics
|
||||
261
docs/research/references/iroh/iroh/05-data-flow-internals.md
Normal file
261
docs/research/references/iroh/iroh/05-data-flow-internals.md
Normal file
@@ -0,0 +1,261 @@
|
||||
# Iroh: Data Flow & Internal Architecture
|
||||
|
||||
## Data Flow: Connecting to a Remote Endpoint
|
||||
|
||||
```
|
||||
Endpoint::connect(endpoint_addr, alpn)
|
||||
│
|
||||
▼
|
||||
resolve_remote(endpoint_addr)
|
||||
│
|
||||
├─ If addr has direct IPs or relay URL → use those
|
||||
│
|
||||
└─ If addr is just EndpointId → query AddressLookupServices
|
||||
│
|
||||
├─ PkarrPublisher/PkarrResolver (HTTP)
|
||||
├─ DnsAddressLookup (DNS TXT)
|
||||
├─ MemoryLookup (in-memory)
|
||||
└─ ...custom implementations
|
||||
│
|
||||
▼
|
||||
Map EndpointId → MappedAddr for QUIC layer
|
||||
│
|
||||
▼
|
||||
noq::Endpoint::connect(client_config, dest_addr, server_name)
|
||||
│
|
||||
├─ TLS handshake with Raw Public Key authentication
|
||||
│ server_name = "<z32-encoded-endpoint-id>.iroh.invalid"
|
||||
│
|
||||
└─ QUIC connection established
|
||||
│
|
||||
▼
|
||||
Connecting → Connection
|
||||
│
|
||||
├─ Connection stays on relay path initially
|
||||
│
|
||||
└─ RemoteStateActor discovers direct paths
|
||||
│
|
||||
├─ QAD-discovered addresses
|
||||
├─ Addresses from Address Lookup
|
||||
├─ Port mapper external addresses
|
||||
│
|
||||
└─ Path migration: relay → direct (if possible)
|
||||
```
|
||||
|
||||
## Data Flow: Accepting Connections
|
||||
|
||||
```
|
||||
Endpoint::accept() → Accept<'_>
|
||||
│
|
||||
▼ (incoming QUIC packet arrives on any transport)
|
||||
│
|
||||
noq::Endpoint::accept()
|
||||
│
|
||||
▼
|
||||
Incoming
|
||||
│
|
||||
├─ incoming.remote_addr() → IncomingAddr (Ip/Relay/Custom)
|
||||
├─ incoming.remote_addr_validated() → bool
|
||||
├─ incoming.accept() → Accepting
|
||||
├─ incoming.refuse() → reject
|
||||
├─ incoming.retry() → QUIC retry (address validation)
|
||||
└─ incoming.ignore() → drop silently
|
||||
│
|
||||
Accepting
|
||||
│
|
||||
├─ accepting.alpn().await → alpn bytes
|
||||
├─ accepting.into_0rtt() → (OutgoingZeroRtt, Connection) [optional]
|
||||
└─ accepting.await → Connection
|
||||
```
|
||||
|
||||
## Data Flow: Router Accept Loop
|
||||
|
||||
```
|
||||
Router::spawn()
|
||||
│
|
||||
├─ endpoint.set_alpns(registered_alpns)
|
||||
│
|
||||
└─ Loop:
|
||||
│
|
||||
├─ endpoint.accept().await → Incoming
|
||||
│ │
|
||||
│ ├─ Apply incoming_filter (optional)
|
||||
│ │ ├─ Accept → continue
|
||||
│ │ ├─ Retry → incoming.retry()
|
||||
│ │ ├─ Reject → incoming.refuse()
|
||||
│ │ └─ Ignore → incoming.ignore()
|
||||
│ │
|
||||
│ ├─ incoming.accept() → Accepting
|
||||
│ ├─ accepting.alpn().await → determine ALPN
|
||||
│ │
|
||||
│ └─ protocols.get(alpn) → handler
|
||||
│ │
|
||||
│ ├─ handler.on_accepting(accepting).await
|
||||
│ └─ handler.accept(connection).await
|
||||
│
|
||||
└─ On shutdown:
|
||||
├─ protocols.shutdown().await
|
||||
├─ handler_cancel_token.cancel()
|
||||
└─ endpoint.close().await
|
||||
```
|
||||
|
||||
## Actor Model: Per-Remote State
|
||||
|
||||
Each remote peer gets a `RemoteStateActor` that manages the connection state:
|
||||
|
||||
```
|
||||
┌───────────────────────────────────────────────┐
|
||||
│ RemoteStateActor │
|
||||
│ │
|
||||
│ ┌─────────────┐ ┌─────────────────┐ │
|
||||
│ │ Address │ │ Connection │ │
|
||||
│ │ Lookup │ │ Tracker │ │
|
||||
│ │ Resolution │ │ │ │
|
||||
│ └──────┬──────┘ └────────┬────────┘ │
|
||||
│ │ │ │
|
||||
│ ▼ ▼ │
|
||||
│ ┌──────────────────────────────────┐ │
|
||||
│ │ Path Selection │ │
|
||||
│ │ ┌────────┐ ┌────────┐ │ │
|
||||
│ │ │ IPv4 │ │ IPv6 │ │ │
|
||||
│ │ │primary │ │primary │ │ │
|
||||
│ │ └────────┘ └────────┘ │ │
|
||||
│ │ ┌────────┐ ┌────────┐ │ │
|
||||
│ │ │ Relay │ │Custom │ │ │
|
||||
│ │ │backup │ │primary │ │ │
|
||||
│ │ └────────┘ └────────┘ │ │
|
||||
│ └──────────────────────────────────┘ │
|
||||
│ │
|
||||
│ ┌──────────────────────────────────┐ │
|
||||
│ │ Mapped Addresses │ │
|
||||
│ │ EndpointId → MappedIPv6Addr │ │
|
||||
│ │ (RelayUrl, EndpointId) → Addr │ │
|
||||
│ │ CustomAddr → MappedIPv6Addr │ │
|
||||
│ └──────────────────────────────────┘ │
|
||||
│ │
|
||||
│ Messages: │
|
||||
│ ├─ ResolveRemote(EndpointAddr, reply) │
|
||||
│ ├─ AddConnection(EndpointId, WeakConn, reply)│
|
||||
│ └─ RemoteInfo(reply) │
|
||||
└───────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
## Data Flow: Socket Actor
|
||||
|
||||
The `Actor` in `Socket` runs as a background task handling network changes:
|
||||
|
||||
```
|
||||
┌────────────────────────────────────────────────────────────┐
|
||||
│ Socket Actor │
|
||||
│ │
|
||||
│ ┌──────────────────┐ ┌─────────────────┐ │
|
||||
│ │ Network Monitor │ │ Direct Addr │ │
|
||||
│ │ (netwatch) │ │ Update State │ │
|
||||
│ │ │ │ │ │
|
||||
│ │ Detects: │ │ Manages: │ │
|
||||
│ │ - Interface up/down│ │ - NetReport runs │ │
|
||||
│ │ - Address changes │ │ - Port mapper │ │
|
||||
│ │ - Route changes │ │ - Direct addrs │ │
|
||||
│ └────────┬─────────┘ └────────┬──────────┘ │
|
||||
│ │ │ │
|
||||
│ ▼ ▼ │
|
||||
│ ┌──────────────────────────────────────────────┐ │
|
||||
│ │ Triggers │ │
|
||||
│ │ - NetworkChange (major/minor) │ │
|
||||
│ │ - PeriodicReStun (every 30s-5min) │ │
|
||||
│ │ - PortmapUpdated │ │
|
||||
│ │ - RelayMapChange │ │
|
||||
│ │ - DirectAddrRefresh │ │
|
||||
│ │ - ResolveRemote (from connect) │ │
|
||||
│ │ - AddConnection (from new QUIC conn) │ │
|
||||
│ └──────────────────────────────────────────────┘ │
|
||||
│ │
|
||||
│ On address change: │
|
||||
│ ┌──────────────────────────────────────────────┐ │
|
||||
│ │ 1. Run net_report to discover external addrs │ │
|
||||
│ │ 2. Update direct_addrs watchable │ │
|
||||
│ │ 3. Publish new addresses to AddressLookup │ │
|
||||
│ │ 4. Notify noq of network changes │ │
|
||||
│ └──────────────────────────────────────────────┘ │
|
||||
└────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
## Shutdown Sequence
|
||||
|
||||
```
|
||||
Endpoint::close()
|
||||
│
|
||||
├─ Cancel at_close_start token
|
||||
│ (stops net_reports, address lookups)
|
||||
│
|
||||
├─ Clear address_lookup services
|
||||
│
|
||||
├─ noq_endpoint.close(0, b"")
|
||||
│ (refuses new connections, starts close for existing)
|
||||
│
|
||||
├─ noq_endpoint.wait_idle().await
|
||||
│ (waits for close frames to be acknowledged)
|
||||
│
|
||||
├─ Cancel at_endpoint_closed token
|
||||
│
|
||||
├─ Wait for actor task (100ms timeout, then abort)
|
||||
│
|
||||
└─ runtime.shutdown().await
|
||||
(waits for all spawned tasks)
|
||||
```
|
||||
|
||||
## WASM/Browser Differences
|
||||
|
||||
When compiled to `wasm32-unknown-unknown`:
|
||||
|
||||
| Feature | Native | WASM/Browser |
|
||||
|---------|--------|-------------|
|
||||
| IP transports | Yes (IPv4 + IPv6) | No (no socket access) |
|
||||
| DNS resolution | `DnsAddressLookup` (system DNS) | `PkarrResolver` (HTTP) |
|
||||
| Network monitoring | `netwatch` (interface changes) | Not available |
|
||||
| Port mapping | UPnP/PCP/NAT-PMP | Not available |
|
||||
| Net report | Full (QAD, HTTPS probes) | Limited |
|
||||
| Runtime | Tokio | `wasm-bindgen-futures` |
|
||||
| Timer | Tokio timer | `web::Timer` wrapping `sleep_until` |
|
||||
|
||||
## Thread Safety & Concurrency
|
||||
|
||||
- `Endpoint` is `Clone` (wraps `Arc<EndpointInner>`)
|
||||
- `Socket` is `Arc<Socket>` — shared across all connections
|
||||
- `RemoteMap` uses `ConcurrentReadMap` — lock-free reads for hot path
|
||||
- `AddressLookupServices` uses `RwLock` — infrequent writes, frequent reads
|
||||
- `DirectAddrs` uses `Watchable` — publishes changes to watchers
|
||||
- `HomeRelayWatch` uses `n0_watcher::Direct` — efficient change notification
|
||||
|
||||
## Error Handling Patterns
|
||||
|
||||
Iroh uses the `n0_error::stack_error` macro for rich error chains:
|
||||
|
||||
```rust
|
||||
#[stack_error(derive, add_meta, from_sources)]
|
||||
pub enum ConnectError {
|
||||
#[error(transparent)]
|
||||
Connect { source: ConnectWithOptsError },
|
||||
#[error(transparent)]
|
||||
Connecting { source: ConnectingError },
|
||||
#[error(transparent)]
|
||||
Connection { source: ConnectionError },
|
||||
}
|
||||
|
||||
// Usage:
|
||||
// ConnectError::Connect { source: ConnectWithOptsError::SelfConnect }
|
||||
// ConnectError::Connecting { source: ConnectingError::AuthenticationError { .. } }
|
||||
```
|
||||
|
||||
## Key Constants & Timeouts
|
||||
|
||||
| Constant | Value | Purpose |
|
||||
|----------|-------|---------|
|
||||
| `HEARTBEAT_INTERVAL` | 5s | Keepalive PING interval |
|
||||
| `PATH_MAX_IDLE_TIMEOUT` | 15s | Max idle before closing direct path |
|
||||
| `RELAY_PATH_MAX_IDLE_TIMEOUT` | 30s | Max idle before closing relay path |
|
||||
| `MAX_MULTIPATH_PATHS` | 12 | Max concurrent paths per connection |
|
||||
| `DEFAULT_MAX_TLS_TICKETS` | 256 (8×32) | TLS session ticket cache size |
|
||||
| `NET_REPORT_TIMEOUT` | 10s | Max time for net report |
|
||||
| `FULL_REPORT_INTERVAL` | 5min | Time between full net reports |
|
||||
| `DEFAULT_RELAY_QUIC_PORT` | 3478 | QAD port on relay servers |
|
||||
Reference in New Issue
Block a user