docs(research): add nats-async and nats-server deep-dive references
This commit is contained in:
@@ -0,0 +1,200 @@
|
||||
# nats.rs: Overview and Architecture
|
||||
|
||||
**Version**: async-nats 0.49.1, nats-server 0.1.0
|
||||
**Repository**: https://github.com/nats-io/nats.rs
|
||||
**License**: Apache-2.0
|
||||
**Rust Edition**: 2021
|
||||
**MSRV**: 1.88.0
|
||||
**Protocol**: NATS Client Protocol (INFO/CONNECT/PUB/SUB/UNSUB/PING/PONG)
|
||||
|
||||
## What It Is
|
||||
|
||||
The `nats.rs` repository contains the **official Rust client for NATS.io**, a high-performance messaging system. The active crate is **`async-nats`** — a fully async, Tokio-based NATS client. The deprecated `nats` crate (synchronous) receives security fixes only.
|
||||
|
||||
The `nats-server` crate is **not** an implementation of the NATS server. It is a **test harness** that spawns the Go-based `nats-server` binary for integration tests. The actual NATS server is a separate Go project at `github.com/nats-io/nats-server`.
|
||||
|
||||
Core design decisions:
|
||||
- **Fully async** — all I/O is Tokio-based with async/await throughout
|
||||
- **Cloneable Client handle** — `Client` is cheap to clone (Arc internals), all protocol work happens in a single `ConnectionHandler` task
|
||||
- **Channel-based internal communication** — `Client` sends `Command` variants via `mpsc` channel to `ConnectionHandler`
|
||||
- **Multiplexed request-reply** — one internal subscription handles all request-response patterns via inbox token routing
|
||||
- **Automatic reconnection** — exponential backoff with configurable server pool rotation
|
||||
- **Feature-gated subsystems** — JetStream, KV, Object Store, Service API, NKeys, WebSockets, and crypto backends are all optional
|
||||
|
||||
## Workspace Structure
|
||||
|
||||
```
|
||||
nats.rs/
|
||||
├── async-nats/ # Primary crate — async NATS client
|
||||
│ ├── src/
|
||||
│ │ ├── lib.rs # Entry point: connect(), ServerOp, ClientOp, Command, ConnectionHandler, Subscriber
|
||||
│ │ ├── client.rs # Client handle: publish, subscribe, request, flush, drain
|
||||
│ │ ├── connection.rs # Low-level I/O: protocol parsing, read/write buffers
|
||||
│ │ ├── connector.rs # Connection establishment, reconnection, server pool
|
||||
│ │ ├── options.rs # ConnectOptions builder
|
||||
│ │ ├── auth.rs # Auth struct (credentials container)
|
||||
│ │ ├── auth_utils.rs # Credential file parsing (.creds files)
|
||||
│ │ ├── error.rs # Generic Error<Kind> type
|
||||
│ │ ├── header.rs # HeaderMap — NATS message headers
|
||||
│ │ ├── subject.rs # Subject type, ToSubject trait
|
||||
│ │ ├── status.rs # StatusCode (100-999 NATS protocol codes)
|
||||
│ │ ├── message.rs # Message and OutboundMessage types
|
||||
│ │ ├── tls.rs # TLS configuration helpers
|
||||
│ │ ├── crypto.rs # Crypto feature support
|
||||
│ │ ├── id_generator.rs # NUID/rand-based unique ID generation
|
||||
│ │ ├── datetime.rs # DateTime helpers for JetStream/Service
|
||||
│ │ ├── jetstream/ # JetStream API (feature-gated)
|
||||
│ │ │ ├── mod.rs # Module root, jetstream::new(), with_domain()
|
||||
│ │ │ ├── context.rs # JetStream Context — streams, publishing, consumers
|
||||
│ │ │ ├── stream.rs # Stream management, Config, Info, Consumer creation
|
||||
│ │ │ ├── consumer/ # Pull, Push, Ordered consumers
|
||||
│ │ │ ├── message.rs # JetStream Message with ack methods
|
||||
│ │ │ ├── publish.rs # PublishAck
|
||||
│ │ │ ├── response.rs # Response wrapper
|
||||
│ │ │ ├── errors.rs # JetStream error codes
|
||||
│ │ │ ├── account.rs # Account info
|
||||
│ │ │ ├── kv/ # Key-Value store (feature: "kv")
|
||||
│ │ │ └── object_store/ # Object store (feature: "object-store")
|
||||
│ │ └── service/ # Service API (feature-gated)
|
||||
│ │ ├── mod.rs # Service, ServiceBuilder
|
||||
│ │ ├── endpoint.rs # Endpoint handling
|
||||
│ │ └── error.rs # Service errors
|
||||
│ ├── tests/ # Integration tests (require nats-server binary)
|
||||
│ ├── examples/ # Runnable examples
|
||||
│ └── benches/ # Criterion benchmarks
|
||||
├── nats-server/ # Test harness — spawns Go nats-server for tests
|
||||
│ ├── src/lib.rs # Server struct, run_server(), run_cluster()
|
||||
│ └── configs/ # Server config files for tests
|
||||
│ └── jetstream.conf
|
||||
└── nats/ # DEPRECATED sync client — do not modify
|
||||
```
|
||||
|
||||
## Architecture Diagram
|
||||
|
||||
```
|
||||
┌──────────────────────────────────────────────────────────┐
|
||||
│ Application Layer │
|
||||
│ │
|
||||
│ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐ │
|
||||
│ │ JetStream│ │ KV │ │ Object │ │ Service │ │
|
||||
│ │ Context │ │ Store │ │ Store │ │ API │ │
|
||||
│ └────┬─────┘ └────┬─────┘ └────┬─────┘ └────┬─────┘ │
|
||||
│ └──────────────┴─────────────┴─────────────┘ │
|
||||
│ │ │
|
||||
│ ┌──────┴──────┐ │
|
||||
│ │ Client │ Cloneable handle │
|
||||
│ │ (mpsc::Sender) │
|
||||
│ └──────┬──────┘ │
|
||||
│ │ Command channel │
|
||||
└──────────────────────────┼────────────────────────────────┘
|
||||
│
|
||||
┌──────────────────────────┼────────────────────────────────┐
|
||||
│ ConnectionHandler │
|
||||
│ (single Tokio task) │
|
||||
│ │ │
|
||||
│ ┌───────────┐ ┌───────┴───────┐ ┌──────────────────┐ │
|
||||
│ │Subscriptions│ │ Multiplexer │ │ Flush Observers │ │
|
||||
│ │ HashMap │ │ (request-reply)│ │ │ │
|
||||
│ └──────┬──────┘ └───────┬───────┘ └──────────────────┘ │
|
||||
│ └────────────────┼ │
|
||||
│ ┌──────┴──────┐ │
|
||||
│ │ Connector │ Server pool, reconnect │
|
||||
│ └──────┬──────┘ │
|
||||
│ │ │
|
||||
│ ┌──────┴──────┐ │
|
||||
│ │ Connection │ Protocol I/O │
|
||||
│ │ (read/write)│ ServerOp / ClientOp │
|
||||
│ └──────┬──────┘ │
|
||||
└──────────────────────────┼────────────────────────────────┘
|
||||
│
|
||||
┌──────┴──────┐
|
||||
│ NATS Server │ (Go binary, TCP/TLS/WS)
|
||||
└─────────────┘
|
||||
```
|
||||
|
||||
## Key Concepts
|
||||
|
||||
### Subject
|
||||
NATS uses subject strings for message addressing. A `Subject` is a validated, immutable, UTF-8 string backed by `Bytes`. Subjects use dot-delimited tokens (e.g., `events.data.sensor1`). Wildcards `*` (single token) and `>` (multi-token suffix) are supported for subscriptions.
|
||||
|
||||
### ClientOp / ServerOp
|
||||
The NATS client-server protocol is text-based with binary payloads. The client sends `ClientOp` variants (CONNECT, PUB/HPUB, SUB, UNSUB, PING, PONG) and receives `ServerOp` variants (INFO, MSG/HMSG, +OK, -ERR, PING, PONG).
|
||||
|
||||
### Command
|
||||
Internal command type sent from `Client` to `ConnectionHandler` via `mpsc` channel. Includes Publish, Request, Subscribe, Unsubscribe, Flush, Drain, Reconnect, SetServerPool, ServerPool.
|
||||
|
||||
### Multiplexer
|
||||
A single internal subscription (SID 0) that routes all request-reply responses. When a `Request` is made, a unique inbox token is registered in the multiplexer's sender map, and the response is dispatched to the corresponding `oneshot::Sender`.
|
||||
|
||||
### ConnectionHandler
|
||||
A single Tokio task that drives all protocol I/O. It processes server operations from `Connection`, handles client commands from the `mpsc` channel, manages subscriptions, maintains ping/pong health, and orchestrates reconnection.
|
||||
|
||||
## nats-server Test Harness
|
||||
|
||||
The `nats-server` crate provides utilities for launching real NATS server instances in tests:
|
||||
|
||||
- `run_server(cfg)` — starts a single server with optional config
|
||||
- `run_cluster(cfg)` — starts a 3-node cluster
|
||||
- `Server` struct — holds the child process, cleans up on drop
|
||||
- `Server::restart()` — kills and restarts the server process
|
||||
- `Server::client_url()` — reads the INFO from the server to get the client URL
|
||||
- `set_lame_duck_mode(server)` — sends LDM signal to the server process
|
||||
|
||||
The test harness spawns the Go `nats-server` binary via `std::process::Command`, using dynamic ports for parallel test execution. It auto-discovers the client URL by connecting to the server's TCP port and parsing the `INFO` JSON. On `Drop`, it kills the child process and cleans up JetStream storage directories.
|
||||
|
||||
## Feature Flags
|
||||
|
||||
```toml
|
||||
# Default: everything enabled
|
||||
default = ["server_2_10", "server_2_11", "server_2_12", "server_2_14",
|
||||
"service", "ring", "jetstream", "nkeys", "crypto",
|
||||
"object-store", "kv", "websockets", "nuid"]
|
||||
|
||||
# Subsystems
|
||||
jetstream # JetStream API
|
||||
kv # Key-Value store (requires jetstream)
|
||||
object-store # Object store (requires jetstream + crypto)
|
||||
service # Service API
|
||||
|
||||
# Crypto backends (pick one)
|
||||
ring # Default crypto backend
|
||||
aws-lc-rs # Alternative backend
|
||||
fips # FIPS mode (requires aws-lc-rs)
|
||||
|
||||
# Auth
|
||||
nkeys # NKey authentication
|
||||
|
||||
# Other
|
||||
nuid # NUID-based ID generation (falls back to rand)
|
||||
crypto # Encryption support
|
||||
websockets # WebSocket transport
|
||||
experimental # Experimental features
|
||||
|
||||
# Server version markers (enable version-specific API fields)
|
||||
server_2_10
|
||||
server_2_11
|
||||
server_2_12
|
||||
server_2_14
|
||||
```
|
||||
|
||||
## Dependencies (Key)
|
||||
|
||||
| Dependency | Purpose |
|
||||
|-----------|---------|
|
||||
| `tokio` | Async runtime (macros, rt, net, sync, time, io-util) |
|
||||
| `bytes` | Zero-copy byte buffers for payloads |
|
||||
| `tokio-rustls` | TLS via rustls |
|
||||
| `rustls-native-certs` | Load native TLS root certificates |
|
||||
| `serde` / `serde_json` | JSON serialization for protocol messages and JetStream API |
|
||||
| `memchr` | Fast CRLF search for protocol parsing |
|
||||
| `futures-util` | Stream trait, Sink trait, StreamExt |
|
||||
| `tracing` | Structured logging |
|
||||
| `thiserror` | Error type derivation |
|
||||
| `url` | URL parsing for server addresses |
|
||||
| `portable-atomic` | Portable atomic operations |
|
||||
|
||||
## References
|
||||
|
||||
- [NATS Protocol Specification](https://docs.nats.io/reference/reference-protocols/nats-protocol)
|
||||
- [NATS JetStream Documentation](https://docs.nats.io/nats-concepts/jetstream)
|
||||
- [async-nats on docs.rs](https://docs.rs/async-nats)
|
||||
@@ -0,0 +1,281 @@
|
||||
# NATS Client Protocol and Wire Format
|
||||
|
||||
**Protocol**: NATS Client Protocol v1 (with dynamic reconfiguration)
|
||||
**Transport**: TCP (port 4222), TLS, WebSocket (ws/wss)
|
||||
|
||||
## Protocol Overview
|
||||
|
||||
The NATS client-server protocol is a simple, text-based protocol with binary payload support. All operations are terminated with `\r\n`. Messages carry their payload length, allowing efficient binary data transfer.
|
||||
|
||||
### Connection Lifecycle
|
||||
|
||||
```
|
||||
Client Server
|
||||
│ │
|
||||
│◄──────────── INFO {json} ────────────────────│ Server sends INFO first
|
||||
│ │
|
||||
│────────────── CONNECT {json} ────────────────►│ Client sends CONNECT
|
||||
│────────────── PING ──────────────────────────►│ Client sends PING
|
||||
│◄──────────── PONG ────────────────────────── │ Server confirms connection
|
||||
│ │
|
||||
│──── SUB/UNSUB/PUB/HPUB ──────────────────────►│ Normal operation
|
||||
│◄─── MSG/HMSG/+OK/-ERR/PING ─────────────────│
|
||||
│ │
|
||||
```
|
||||
|
||||
## Server Operations (ServerOp)
|
||||
|
||||
These are operations received from the server. The `Connection` module parses these from the read buffer.
|
||||
|
||||
### INFO
|
||||
|
||||
Sent by the server upon connection and asynchronously when cluster topology changes.
|
||||
|
||||
```
|
||||
INFO {json}\r\n
|
||||
```
|
||||
|
||||
JSON fields (see `ServerInfo` struct):
|
||||
|
||||
| Field | Type | Description |
|
||||
|-------|------|-------------|
|
||||
| `server_id` | String | Unique server identifier |
|
||||
| `server_name` | String | Generated server name |
|
||||
| `host` | String | Cluster host |
|
||||
| `port` | u16 | Cluster port |
|
||||
| `version` | String | Server version |
|
||||
| `auth_required` | bool | Authentication required |
|
||||
| `tls_required` | bool | TLS required |
|
||||
| `max_payload` | usize | Maximum payload size |
|
||||
| `proto` | i8 | Protocol version (0 or 1) |
|
||||
| `client_id` | u64 | Server-assigned client ID |
|
||||
| `go` | String | Go build version |
|
||||
| `nonce` | String | Nonce for nkey auth |
|
||||
| `connect_urls` | Vec<String> | Cluster server URLs |
|
||||
| `client_ip` | String | Client IP as seen by server |
|
||||
| `headers` | bool | Server supports headers |
|
||||
| `ldm` | bool | Lame duck mode |
|
||||
| `cluster` | Option<String> | Cluster name |
|
||||
| `domain` | Option<String> | NATS domain |
|
||||
| `jetstream` | bool | JetStream enabled |
|
||||
|
||||
### MSG
|
||||
|
||||
Delivers a message to a subscription (no headers):
|
||||
|
||||
```
|
||||
MSG <subject> <sid> [reply-to] <#bytes>\r\n
|
||||
<payload>\r\n
|
||||
```
|
||||
|
||||
### HMSG
|
||||
|
||||
Delivers a message with headers:
|
||||
|
||||
```
|
||||
HMSG <subject> <sid> [reply-to] <#header-bytes> <#total-bytes>\r\n
|
||||
<NATS/1.0 [status] [description]>\r\n
|
||||
<header-name>: <header-value>\r\n
|
||||
\r\n
|
||||
<payload>\r\n
|
||||
```
|
||||
|
||||
Header format follows the NATS/1.0 header spec:
|
||||
- First line: `NATS/1.0` optionally followed by status code and description
|
||||
- Subsequent lines: `name: value` headers
|
||||
- Empty line separates headers from payload
|
||||
- Header values may span multiple lines (continuation lines start with whitespace)
|
||||
|
||||
### PING / PONG
|
||||
|
||||
```
|
||||
PING\r\n → Client responds with PONG
|
||||
PONG\r\n → Acknowledges client's PING
|
||||
```
|
||||
|
||||
### +OK / -ERR
|
||||
|
||||
```
|
||||
+OK\r\n → Success acknowledgment (verbose mode)
|
||||
-ERR <description>\r\n → Error from server
|
||||
```
|
||||
|
||||
Common server errors:
|
||||
- `authorization violation` → parsed as `ServerError::AuthorizationViolation`
|
||||
- Other strings → `ServerError::Other(String)`
|
||||
|
||||
## Client Operations (ClientOp)
|
||||
|
||||
These are operations sent from the client to the server. The `Connection` module serializes these to the write buffer.
|
||||
|
||||
### CONNECT
|
||||
|
||||
Sent as the first client operation after receiving INFO. Contains authentication and capability information.
|
||||
|
||||
```
|
||||
CONNECT {json}\r\n
|
||||
```
|
||||
|
||||
JSON fields (see `ConnectInfo` struct):
|
||||
|
||||
| Field | Type | Description |
|
||||
|-------|------|-------------|
|
||||
| `verbose` | bool | Enable +OK acknowledgments (always false in this client) |
|
||||
| `pedantic` | bool | Strict format checking (always false) |
|
||||
| `jwt` | Option<String> | User JWT for auth |
|
||||
| `nkey` | Option<String> | Public nkey for auth |
|
||||
| `sig` | Option<String> | Signed nonce (Base64URL encoded) |
|
||||
| `name` | Option<String> | Client name |
|
||||
| `echo` | bool | Whether server should echo messages back |
|
||||
| `lang` | String | Implementation language ("rust") |
|
||||
| `version` | String | Client version |
|
||||
| `protocol` | u8 | Protocol version (1 = dynamic) |
|
||||
| `tls_required` | bool | TLS required |
|
||||
| `user` | Option<String> | Username |
|
||||
| `pass` | Option<String> | Password |
|
||||
| `auth_token` | Option<String> | Auth token |
|
||||
| `headers` | bool | Client supports headers (always true) |
|
||||
| `no_responders` | bool | Client supports no-responders (always true) |
|
||||
|
||||
### PUB / HPUB
|
||||
|
||||
Publish a message:
|
||||
|
||||
```
|
||||
PUB <subject> [reply-to] <#payload-bytes>\r\n
|
||||
<payload>\r\n
|
||||
```
|
||||
|
||||
Publish with headers:
|
||||
|
||||
```
|
||||
HPUB <subject> [reply-to] <#header-bytes> <#total-bytes>\r\n
|
||||
<NATS/1.0\r\n
|
||||
<header-name>: <header-value>\r\n
|
||||
\r\n
|
||||
<payload>\r\n
|
||||
```
|
||||
|
||||
### SUB
|
||||
|
||||
Subscribe to a subject:
|
||||
|
||||
```
|
||||
SUB <subject> [queue-group] <sid>\r\n
|
||||
```
|
||||
|
||||
### UNSUB
|
||||
|
||||
Unsubscribe from a subscription:
|
||||
|
||||
```
|
||||
UNSUB <sid> [max]\r\n
|
||||
```
|
||||
|
||||
The optional `max` parameter tells the server to auto-unsubscribe after receiving the specified number of messages.
|
||||
|
||||
### PING / PONG
|
||||
|
||||
```
|
||||
PING\r\n → Health check / keepalive
|
||||
PONG\r\n → Response to server PING
|
||||
```
|
||||
|
||||
## Protocol Version
|
||||
|
||||
The `Protocol` enum has two variants:
|
||||
|
||||
| Value | Name | Description |
|
||||
|-------|------|-------------|
|
||||
| 0 | Original | Basic protocol |
|
||||
| 1 | Dynamic | Supports async INFO for cluster topology changes, lame duck mode |
|
||||
|
||||
This client always sends `protocol: 1` (Dynamic), enabling:
|
||||
- Asynchronous INFO messages with updated server lists
|
||||
- Lame duck mode notifications
|
||||
- Dynamic reconfiguration of cluster topology
|
||||
|
||||
## Wire Format Details
|
||||
|
||||
### Message Length Calculation
|
||||
|
||||
For plain `MSG`:
|
||||
```
|
||||
length = subject.len() + reply.map_or(0, |r| r.len()) + payload.len()
|
||||
```
|
||||
|
||||
For `HMSG`:
|
||||
```
|
||||
length = subject.len() + reply.map_or(0, |r| r.len()) + header_len + payload.len()
|
||||
```
|
||||
|
||||
Where `header_len` = serialized header bytes and `total_len` = `header_len + payload.len()`.
|
||||
|
||||
### Write Buffer Architecture
|
||||
|
||||
The `Connection` uses a two-tier write buffer:
|
||||
|
||||
1. **`flattened_writes`** (`BytesMut`) — for small writes (< 4096 bytes). Protocol headers, short commands, and small messages are flattened into this buffer for efficient sequential writing.
|
||||
|
||||
2. **`write_buf`** (`VecDeque<Bytes>`) — for large writes (>= 4096 bytes). Large payloads are appended as separate `Bytes` chunks. Supports vectored writes (`write_vectored`) when the underlying stream supports it, writing up to 64 chunks at once.
|
||||
|
||||
The soft limit for the total write buffer is 65,535 bytes (`SOFT_WRITE_BUF_LIMIT`). When exceeded, the `ConnectionHandler` stops processing new commands until the buffer drains.
|
||||
|
||||
### Read Buffer Architecture
|
||||
|
||||
The `Connection` uses a single `BytesMut` read buffer with configurable initial capacity (default 65,535 bytes). Protocol parsing uses `memchr::memmem::find` to locate CRLF delimiters efficiently. If a partial message is in the buffer, the parser returns `None` and waits for more data.
|
||||
|
||||
### Header Serialization
|
||||
|
||||
Headers are serialized in NATS/1.0 format:
|
||||
|
||||
```
|
||||
NATS/1.0\r\n
|
||||
Header-Name: Header-Value\r\n
|
||||
Multi-Line-Header: value part 1\r\n
|
||||
continuation of value\r\n
|
||||
Another-Header: another value\r\n
|
||||
\r\n
|
||||
```
|
||||
|
||||
The `HeaderMap::to_bytes()` method handles this serialization, using `httparse`-compatible line folding for multi-line values.
|
||||
|
||||
### Status Codes in Headers
|
||||
|
||||
NATS status codes are embedded in the `HMSG` header version line:
|
||||
|
||||
```
|
||||
NATS/1.0 404 No Messages\r\n
|
||||
NATS/1.0 408 Request Timeout\r\n
|
||||
NATS/1.0 503 No Responders\r\n
|
||||
```
|
||||
|
||||
Common codes used by the client:
|
||||
|
||||
| Code | Constant | Meaning |
|
||||
|------|----------|---------|
|
||||
| 100 | `IDLE_HEARTBEAT` | JetStream idle heartbeat |
|
||||
| 200 | `OK` | Success |
|
||||
| 404 | `NOT_FOUND` | Message/stream not found |
|
||||
| 408 | `TIMEOUT` | Request timeout |
|
||||
| 409 | `REQUEST_TERMINATED` | Request terminated |
|
||||
| 503 | `NO_RESPONDERS` | No responders available |
|
||||
|
||||
## Protocol Parsing Implementation
|
||||
|
||||
The `Connection::try_read_op()` method handles all protocol parsing:
|
||||
|
||||
1. Search for `\r\n` delimiter using `memchr::memmem::find`
|
||||
2. Match the operation prefix:
|
||||
- `+OK` → `ServerOp::Ok`
|
||||
- `PING` → `ServerOp::Ping`
|
||||
- `PONG` → `ServerOp::Pong`
|
||||
- `-ERR` → parse error description → `ServerOp::Error`
|
||||
- `INFO ` → parse JSON → `ServerOp::Info`
|
||||
- `MSG ` → parse subject/sid/reply/length, read payload → `ServerOp::Message`
|
||||
- `HMSG ` → parse headers + payload → `ServerOp::Message`
|
||||
3. Unknown prefix → return `io::Error` with `InvalidInput`
|
||||
|
||||
For `MSG` and `HMSG`, if the complete payload isn't yet in the read buffer (checked via `len + payload_len + 4 > remaining`), the method returns `Ok(None)` and the buffer accumulates more data before retrying.
|
||||
|
||||
Non-UTF8 subjects in server messages are handled gracefully — the parser returns an `io::Error` rather than panicking, which is critical because the Go server does not enforce UTF-8 in subjects (regression fix for issue #1572).
|
||||
@@ -0,0 +1,443 @@
|
||||
# Key Types and Traits
|
||||
|
||||
This document covers the core data types in the `async-nats` crate that form the public API and internal plumbing.
|
||||
|
||||
## Public Types
|
||||
|
||||
### Client
|
||||
|
||||
**Location**: `client.rs`
|
||||
|
||||
`Client` is the primary user-facing type. It is a lightweight, cloneable handle to a NATS connection.
|
||||
|
||||
```rust
|
||||
#[derive(Clone, Debug)]
|
||||
pub struct Client {
|
||||
info: tokio::sync::watch::Receiver<Option<ServerInfo>>,
|
||||
state: tokio::sync::watch::Receiver<State>,
|
||||
sender: mpsc::Sender<Command>,
|
||||
poll_sender: PollSender<Command>,
|
||||
next_subscription_id: Arc<AtomicU64>,
|
||||
subscription_capacity: usize,
|
||||
inbox_prefix: Arc<str>,
|
||||
request_timeout: Option<Duration>,
|
||||
max_payload: Arc<AtomicUsize>,
|
||||
connection_stats: Arc<Statistics>,
|
||||
skip_subject_validation: bool,
|
||||
}
|
||||
```
|
||||
|
||||
Key methods:
|
||||
- `publish(subject, payload)` — fire-and-forget publish
|
||||
- `publish_with_headers(subject, headers, payload)` — publish with NATS headers
|
||||
- `publish_with_reply(subject, reply, payload)` — publish with reply subject
|
||||
- `request(subject, payload)` — request-response (returns `Message`)
|
||||
- `send_request(subject, request)` — request with `Request` builder
|
||||
- `subscribe(subject)` — subscribe to a subject, returns `Subscriber`
|
||||
- `queue_subscribe(subject, queue_group)` — subscribe as part of a queue group
|
||||
- `flush()` — ensure all pending messages are written to the wire
|
||||
- `drain()` — gracefully drain all subscriptions and close
|
||||
- `force_reconnect()` — trigger immediate reconnection
|
||||
- `new_inbox()` — generate a unique inbox subject for request-reply
|
||||
- `server_info()` — get last received `ServerInfo`
|
||||
- `max_payload()` — get server's maximum payload size
|
||||
- `connection_state()` — get current connection `State`
|
||||
- `statistics()` — get `Arc<Statistics>` for connection metrics
|
||||
- `is_server_compatible(major, minor, patch)` — check server version compatibility
|
||||
- `set_server_pool(addrs)` / `server_pool()` — manage server pool
|
||||
|
||||
`Client` also implements `Sink<OutboundMessage>` for backpressure-aware publishing.
|
||||
|
||||
### Subscriber
|
||||
|
||||
**Location**: `lib.rs`
|
||||
|
||||
A `Subscriber` receives messages from a single subscription. It implements `futures::Stream`.
|
||||
|
||||
```rust
|
||||
#[derive(Debug)]
|
||||
pub struct Subscriber {
|
||||
sid: u64,
|
||||
receiver: mpsc::Receiver<Message>,
|
||||
sender: mpsc::Sender<Command>,
|
||||
}
|
||||
```
|
||||
|
||||
Key methods:
|
||||
- `unsubscribe()` — unsubscribe and close the stream
|
||||
- `unsubscribe_after(max)` — auto-unsubscribe after N messages
|
||||
- `drain()` — gracefully drain remaining messages then close
|
||||
|
||||
On `Drop`, `Subscriber` automatically sends an `Unsubscribe` command and closes the receiver channel.
|
||||
|
||||
### Message
|
||||
|
||||
**Location**: `message.rs`
|
||||
|
||||
Represents an inbound NATS message.
|
||||
|
||||
```rust
|
||||
#[derive(Clone, Debug, Serialize, Deserialize)]
|
||||
pub struct Message {
|
||||
pub subject: Subject,
|
||||
pub reply: Option<Subject>,
|
||||
pub payload: Bytes,
|
||||
pub headers: Option<HeaderMap>,
|
||||
pub status: Option<StatusCode>,
|
||||
pub description: Option<String>,
|
||||
pub length: usize,
|
||||
}
|
||||
```
|
||||
|
||||
### OutboundMessage
|
||||
|
||||
**Location**: `message.rs`
|
||||
|
||||
Represents a message to be published. No status/description fields (those are inbound-only).
|
||||
|
||||
```rust
|
||||
#[derive(Clone, Debug)]
|
||||
pub struct OutboundMessage {
|
||||
pub subject: Subject,
|
||||
pub reply: Option<Subject>,
|
||||
pub payload: Bytes,
|
||||
pub headers: Option<HeaderMap>,
|
||||
}
|
||||
```
|
||||
|
||||
### Subject
|
||||
|
||||
**Location**: `subject.rs`
|
||||
|
||||
An immutable, validated UTF-8 string backed by `Bytes`. Used throughout the crate instead of raw `String` for subjects.
|
||||
|
||||
```rust
|
||||
#[derive(Clone, Debug, PartialEq, Eq, PartialOrd, Ord)]
|
||||
pub struct Subject {
|
||||
bytes: Bytes,
|
||||
}
|
||||
```
|
||||
|
||||
Implements `Deref<Target = str>`, `From<&str>`, `From<String>`, `TryFrom<Bytes>`, `Serialize`, `Deserialize`.
|
||||
|
||||
Validation methods:
|
||||
- `is_valid()` — checks NATS subject rules (no leading/trailing dots, no consecutive dots, no whitespace)
|
||||
- `validated(s)` — construct with validation, returns `Result<Subject, SubjectError>`
|
||||
- `from_static_validated(s)` — const-time validation for static strings (compile-time panic on invalid)
|
||||
|
||||
### ToSubject Trait
|
||||
|
||||
**Location**: `subject.rs`
|
||||
|
||||
```rust
|
||||
pub trait ToSubject {
|
||||
fn to_subject(&self) -> Subject;
|
||||
}
|
||||
```
|
||||
|
||||
Implemented for `Subject`, `&'static str`, `String`. All methods accepting subjects are generic over `impl ToSubject`.
|
||||
|
||||
### HeaderMap
|
||||
|
||||
**Location**: `header.rs`
|
||||
|
||||
NATS message headers, modeled after the `http::header` crate.
|
||||
|
||||
```rust
|
||||
#[derive(Clone, PartialEq, Eq, Debug, Default)]
|
||||
pub struct HeaderMap {
|
||||
inner: HashMap<HeaderName, Vec<HeaderValue>>,
|
||||
}
|
||||
```
|
||||
|
||||
Supports multiple values per header name (like HTTP). Key methods:
|
||||
- `insert(name, value)` — replace all values for a name
|
||||
- `append(name, value)` — add a value to a name
|
||||
- `get(name)` — get the first value
|
||||
- `get_all(name)` — get all values as an iterator
|
||||
- `len()` / `is_empty()` — number of header entries
|
||||
- `to_bytes()` — serialize to NATS/1.0 wire format
|
||||
- `wire_len()` — size in wire format (for payload size checks)
|
||||
|
||||
### StatusCode
|
||||
|
||||
**Location**: `status.rs`
|
||||
|
||||
NATS status codes (100-999), structurally similar to HTTP status codes.
|
||||
|
||||
```rust
|
||||
#[derive(Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash, Serialize, Deserialize)]
|
||||
pub struct StatusCode(NonZeroU16);
|
||||
```
|
||||
|
||||
Constants:
|
||||
| Constant | Code | Meaning |
|
||||
|----------|------|---------|
|
||||
| `IDLE_HEARTBEAT` | 100 | JetStream idle heartbeat |
|
||||
| `OK` | 200 | Success |
|
||||
| `NOT_FOUND` | 404 | Not found |
|
||||
| `TIMEOUT` | 408 | Timeout |
|
||||
| `REQUEST_TERMINATED` | 409 | Request terminated |
|
||||
| `NO_RESPONDERS` | 503 | No responders |
|
||||
|
||||
### ServerInfo
|
||||
|
||||
**Location**: `lib.rs`
|
||||
|
||||
Deserialized from the server's `INFO` JSON message. Contains server capabilities, connection details, and cluster information.
|
||||
|
||||
### ConnectInfo
|
||||
|
||||
**Location**: `lib.rs`
|
||||
|
||||
Serialized into the client's `CONNECT` JSON message. Contains authentication credentials, client capabilities, and protocol preferences.
|
||||
|
||||
### ServerAddr
|
||||
|
||||
**Location**: `lib.rs`
|
||||
|
||||
A validated NATS server URL, supporting schemes `nats://`, `tls://`, `ws://`, `wss://`.
|
||||
|
||||
```rust
|
||||
#[derive(Debug, Clone, PartialEq, Eq, Hash)]
|
||||
pub struct ServerAddr(Url);
|
||||
```
|
||||
|
||||
Methods:
|
||||
- `from_url(url)` — validate and create
|
||||
- `tls_required()` — true for `tls://` scheme
|
||||
- `is_websocket()` — true for `ws://` or `wss://`
|
||||
- `host()` / `port()` / `scheme()` — URL component accessors
|
||||
- `socket_addrs()` — async DNS resolution
|
||||
- `username()` / `password()` — embedded credentials
|
||||
|
||||
### Auth
|
||||
|
||||
**Location**: `auth.rs`
|
||||
|
||||
Container for authentication credentials.
|
||||
|
||||
```rust
|
||||
#[derive(Clone, Default)]
|
||||
pub struct Auth {
|
||||
pub jwt: Option<String>,
|
||||
pub nkey: Option<String>,
|
||||
pub signature_callback: Option<CallbackArg1<String, Result<String, AuthError>>>,
|
||||
pub signature: Option<Vec<u8>>,
|
||||
pub username: Option<String>,
|
||||
pub password: Option<String>,
|
||||
pub token: Option<String>,
|
||||
}
|
||||
```
|
||||
|
||||
### Request
|
||||
|
||||
**Location**: `client.rs`
|
||||
|
||||
Builder for customized request-response operations.
|
||||
|
||||
```rust
|
||||
#[derive(Default)]
|
||||
pub struct Request {
|
||||
pub payload: Option<Bytes>,
|
||||
pub headers: Option<HeaderMap>,
|
||||
pub timeout: Option<Option<Duration>>,
|
||||
pub inbox: Option<String>,
|
||||
}
|
||||
```
|
||||
|
||||
### Statistics
|
||||
|
||||
**Location**: `client.rs`
|
||||
|
||||
Atomic connection statistics shared between Client and ConnectionHandler.
|
||||
|
||||
```rust
|
||||
#[derive(Default, Debug)]
|
||||
pub struct Statistics {
|
||||
pub in_bytes: AtomicU64,
|
||||
pub out_bytes: AtomicU64,
|
||||
pub in_messages: AtomicU64,
|
||||
pub out_messages: AtomicU64,
|
||||
pub connects: AtomicU64,
|
||||
}
|
||||
```
|
||||
|
||||
### Event
|
||||
|
||||
**Location**: `lib.rs`
|
||||
|
||||
Events emitted by the client for connection lifecycle monitoring.
|
||||
|
||||
```rust
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
pub enum Event {
|
||||
Connected,
|
||||
Disconnected,
|
||||
LameDuckMode,
|
||||
Draining,
|
||||
Closed,
|
||||
SlowConsumer(u64),
|
||||
ServerError(ServerError),
|
||||
ClientError(ClientError),
|
||||
}
|
||||
```
|
||||
|
||||
## Internal Types
|
||||
|
||||
### Command
|
||||
|
||||
**Location**: `lib.rs`
|
||||
|
||||
Internal commands sent from `Client` to `ConnectionHandler` via `mpsc` channel.
|
||||
|
||||
```rust
|
||||
pub(crate) enum Command {
|
||||
Publish(OutboundMessage),
|
||||
Request { subject, payload, respond, headers, sender: oneshot::Sender<Message> },
|
||||
Subscribe { sid, subject, queue_group, sender: mpsc::Sender<Message> },
|
||||
Unsubscribe { sid, max: Option<u64> },
|
||||
Flush { observer: oneshot::Sender<()> },
|
||||
Drain { sid: Option<u64> },
|
||||
Reconnect,
|
||||
SetServerPool { servers: Vec<ServerAddr>, result: oneshot::Sender<Result<(), String>> },
|
||||
ServerPool { result: oneshot::Sender<Vec<connector::Server>> },
|
||||
}
|
||||
```
|
||||
|
||||
### ClientOp / ServerOp
|
||||
|
||||
**Location**: `lib.rs`
|
||||
|
||||
Protocol-level operation types used by `Connection` for wire format parsing and serialization.
|
||||
|
||||
### Subscription (Internal)
|
||||
|
||||
**Location**: `lib.rs`
|
||||
|
||||
```rust
|
||||
struct Subscription {
|
||||
subject: Subject,
|
||||
sender: mpsc::Sender<Message>,
|
||||
queue_group: Option<String>,
|
||||
delivered: u64,
|
||||
max: Option<u64>,
|
||||
}
|
||||
```
|
||||
|
||||
### Multiplexer (Internal)
|
||||
|
||||
**Location**: `lib.rs`
|
||||
|
||||
```rust
|
||||
struct Multiplexer {
|
||||
subject: Subject, // Wildcard subscription subject (e.g., "_INBOX.xxx.*")
|
||||
prefix: Subject, // Prefix for routing (e.g., "_INBOX.xxx.")
|
||||
senders: HashMap<String, oneshot::Sender<Message>>, // token → sender
|
||||
}
|
||||
```
|
||||
|
||||
### Connection State
|
||||
|
||||
**Location**: `connection.rs`
|
||||
|
||||
```rust
|
||||
#[derive(Debug, Eq, PartialEq, Clone)]
|
||||
pub enum State {
|
||||
Pending,
|
||||
Connected,
|
||||
Disconnected,
|
||||
}
|
||||
```
|
||||
|
||||
### Protocol
|
||||
|
||||
**Location**: `lib.rs`
|
||||
|
||||
```rust
|
||||
#[derive(Serialize_repr, Deserialize_repr, PartialEq, Eq, Debug, Clone, Copy)]
|
||||
#[repr(u8)]
|
||||
pub enum Protocol {
|
||||
Original = 0,
|
||||
Dynamic = 1,
|
||||
}
|
||||
```
|
||||
|
||||
## Error Type Pattern
|
||||
|
||||
The crate uses a generic `Error<Kind>` type throughout. Every subsystem defines its own `ErrorKind` enum and a type alias:
|
||||
|
||||
```rust
|
||||
// Define the kind enum
|
||||
#[derive(Clone, Debug, PartialEq)]
|
||||
pub enum PublishErrorKind {
|
||||
MaxPayloadExceeded,
|
||||
InvalidSubject,
|
||||
Send,
|
||||
}
|
||||
|
||||
// Define the error type alias
|
||||
pub type PublishError = Error<PublishErrorKind>;
|
||||
|
||||
// Construct errors
|
||||
PublishError::new(PublishErrorKind::MaxPayloadExceeded)
|
||||
PublishError::with_source(PublishErrorKind::Send, io_error)
|
||||
|
||||
// Match on errors
|
||||
if err.kind() == PublishErrorKind::MaxPayloadExceeded { ... }
|
||||
```
|
||||
|
||||
Error kinds in the crate:
|
||||
|
||||
| Error Type | Kind Enum | Context |
|
||||
|-----------|-----------|---------|
|
||||
| `ConnectError` | `ConnectErrorKind` | Initial connection failures |
|
||||
| `PublishError` | `PublishErrorKind` | Publish validation failures |
|
||||
| `RequestError` | `RequestErrorKind` | Request-response failures |
|
||||
| `SubscribeError` | `SubscribeErrorKind` | Subscription failures |
|
||||
| `FlushError` | `FlushErrorKind` | Flush failures |
|
||||
| `ServerPoolError` | `ServerPoolErrorKind` | Server pool query failures |
|
||||
| `SetServerPoolError` | `SetServerPoolErrorKind` | Server pool modification failures |
|
||||
|
||||
## Trait Implementations
|
||||
|
||||
### Client Trait Interfaces
|
||||
|
||||
The `Client` implements several traits defined in `client::traits`:
|
||||
|
||||
```rust
|
||||
// Publisher trait — publish with optional reply subject
|
||||
trait Publisher {
|
||||
fn publish_with_reply<S, R>(&self, subject: S, reply: R, payload: Bytes) -> impl Future<Output = Result<(), PublishError>>;
|
||||
fn publish_message(&self, msg: OutboundMessage) -> impl Future<Output = Result<(), PublishError>>;
|
||||
}
|
||||
|
||||
// Subscriber trait — subscribe to a subject
|
||||
trait Subscriber {
|
||||
fn subscribe<S>(&self, subject: S) -> impl Future<Output = Result<crate::Subscriber, SubscribeError>>;
|
||||
}
|
||||
|
||||
// Requester trait — send request-response
|
||||
trait Requester {
|
||||
fn send_request<S>(&self, subject: S, request: Request) -> impl Future<Output = Result<Message, RequestError>>;
|
||||
}
|
||||
|
||||
// TimeoutProvider trait — access request timeout
|
||||
trait TimeoutProvider {
|
||||
fn timeout(&self) -> Option<Duration>;
|
||||
}
|
||||
```
|
||||
|
||||
### ToServerAddrs Trait
|
||||
|
||||
**Location**: `lib.rs`
|
||||
|
||||
Converts various address types into server address iterators. Implemented for `ServerAddr`, `str`, `String`, `&[T]`, `Vec<T>`, `&[ServerAddr]`, and references.
|
||||
|
||||
### Sink<OutboundMessage>
|
||||
|
||||
`Client` implements `futures::Sink<OutboundMessage>` for backpressure-aware publishing through the `PollSender` adapter.
|
||||
|
||||
### Stream for Subscriber
|
||||
|
||||
`Subscriber` implements `futures::Stream` with `Item = Message`, delegating to the internal `mpsc::Receiver`.
|
||||
@@ -0,0 +1,338 @@
|
||||
# Connection Handler and Data Flow
|
||||
|
||||
This document covers the internal `ConnectionHandler` that drives all protocol I/O, and the data flow through the system.
|
||||
|
||||
## ConnectionHandler
|
||||
|
||||
**Location**: `lib.rs`
|
||||
|
||||
The `ConnectionHandler` is the heart of the client. It runs as a single Tokio task and manages all communication with the NATS server.
|
||||
|
||||
```rust
|
||||
pub(crate) struct ConnectionHandler {
|
||||
connection: Connection, // Low-level I/O
|
||||
connector: Connector, // Server pool, reconnection
|
||||
subscriptions: HashMap<u64, Subscription>, // Active subscriptions
|
||||
multiplexer: Option<Multiplexer>, // Request-reply multiplexer
|
||||
pending_pings: usize, // Unanswered PINGs
|
||||
info_sender: tokio::sync::watch::Sender<Option<ServerInfo>>,
|
||||
ping_interval: Interval, // Periodic PING timer
|
||||
should_reconnect: bool, // Flag for forced reconnect
|
||||
flush_observers: Vec<oneshot::Sender<()>>, // Pending flush callbacks
|
||||
is_draining: bool, // Connection is draining
|
||||
drain_pings: VecDeque<u64>, // SIDs being drained
|
||||
}
|
||||
```
|
||||
|
||||
## Data Flow: Publish
|
||||
|
||||
```
|
||||
Application
|
||||
│
|
||||
│ client.publish("events.data", payload)
|
||||
│
|
||||
▼
|
||||
Client
|
||||
│ validates subject & payload size
|
||||
│ sends Command::Publish(OutboundMessage) via mpsc channel
|
||||
│
|
||||
▼
|
||||
ConnectionHandler::handle_command(Command::Publish)
|
||||
│ increments out_messages, out_bytes statistics
|
||||
│ calls connection.enqueue_write_op(&ClientOp::Publish { ... })
|
||||
│
|
||||
▼
|
||||
Connection::enqueue_write_op
|
||||
│ serializes to wire format:
|
||||
│ "PUB events.data 11\r\n" or "HPUB events.data 23 34\r\n"
|
||||
│ appends to flattened_writes or write_buf
|
||||
│
|
||||
▼
|
||||
Connection::poll_write
|
||||
│ uses vectored writes (64 chunks) if supported
|
||||
│ or sequential writes otherwise
|
||||
│
|
||||
▼
|
||||
Connection::poll_flush
|
||||
│ flushes the TCP/TLS/WS stream
|
||||
│ notifies flush_observers
|
||||
│
|
||||
▼
|
||||
NATS Server (TCP/TLS/WebSocket)
|
||||
```
|
||||
|
||||
## Data Flow: Subscribe
|
||||
|
||||
```
|
||||
Application
|
||||
│
|
||||
│ client.subscribe("events.>")
|
||||
│
|
||||
▼
|
||||
Client::subscribe
|
||||
│ validates subject (always, regardless of skip_subject_validation)
|
||||
│ allocates next sid via AtomicU64
|
||||
│ creates mpsc channel for messages
|
||||
│ sends Command::Subscribe { sid, subject, sender }
|
||||
│ returns Subscriber { sid, receiver }
|
||||
│
|
||||
▼
|
||||
ConnectionHandler::handle_command(Command::Subscribe)
|
||||
│ creates Subscription { subject, sender, delivered: 0, max: None }
|
||||
│ inserts into subscriptions HashMap
|
||||
│ calls connection.enqueue_write_op(&ClientOp::Subscribe { sid, subject, queue_group })
|
||||
│
|
||||
▼
|
||||
Connection::enqueue_write_op
|
||||
│ serializes: "SUB events.> 42\r\n"
|
||||
│
|
||||
▼
|
||||
Server sends MSG for matching subjects:
|
||||
│
|
||||
▼
|
||||
ConnectionHandler::handle_server_op(ServerOp::Message { sid, subject, ... })
|
||||
│ looks up sid in subscriptions HashMap
|
||||
│ constructs Message { subject, reply, payload, headers, status, description }
|
||||
│ tries subscription.sender.try_send(message)
|
||||
│
|
||||
├── Ok → increments subscription.delivered, checks max
|
||||
├── Full → emits Event::SlowConsumer(sid)
|
||||
└── Closed → removes subscription, sends ClientOp::Unsubscribe
|
||||
│
|
||||
▼
|
||||
Subscriber::poll_next (Stream impl)
|
||||
│ receives from mpsc::Receiver
|
||||
│
|
||||
▼
|
||||
Application processes Message
|
||||
```
|
||||
|
||||
## Data Flow: Request-Response
|
||||
|
||||
The request-response pattern uses the **multiplexer** — a single wildcard subscription that routes responses to their waiting requesters.
|
||||
|
||||
```
|
||||
Application
|
||||
│
|
||||
│ client.request("service", payload)
|
||||
│
|
||||
▼
|
||||
Client::send_request
|
||||
│ validates subject & payload size
|
||||
│ creates oneshot channel for response
|
||||
│ generates unique inbox: "_INBOX.<nuid>.<token>"
|
||||
│ sends Command::Request { subject, payload, respond, sender }
|
||||
│
|
||||
▼
|
||||
ConnectionHandler::handle_command(Command::Request)
|
||||
│ extracts token from respond subject (after last '.')
|
||||
│ if no multiplexer exists:
|
||||
│ creates Multiplexer with wildcard sub "_INBOX.<id>.*" (SID 0)
|
||||
│ sends ClientOp::Subscribe { sid: 0, subject: "_INBOX.<id>.*" }
|
||||
│ inserts token → oneshot::Sender in multiplexer.senders
|
||||
│ sends ClientOp::Publish { subject, payload, respond: "<prefix><token>" }
|
||||
│
|
||||
▼
|
||||
Server routes request to service:
|
||||
│
|
||||
▼
|
||||
Service responds by publishing to the reply subject:
|
||||
│
|
||||
▼
|
||||
ConnectionHandler::handle_server_op(ServerOp::Message { sid: 0, ... })
|
||||
│ sid == MULTIPLEXER_SID (0), so enters multiplexer path
|
||||
│ extracts token by stripping prefix from subject
|
||||
│ looks up token in multiplexer.senders
|
||||
│ sends Message via oneshot::Sender
|
||||
│
|
||||
▼
|
||||
Client::send_request receives via oneshot::Receiver
|
||||
│ applies timeout (default 10s)
|
||||
│ checks for NO_RESPONDERS status (503)
|
||||
│
|
||||
▼
|
||||
Application receives Message
|
||||
```
|
||||
|
||||
### Custom Inbox Request
|
||||
|
||||
If the `Request` builder specifies a custom `inbox`, the flow is different:
|
||||
- The client subscribes to the inbox directly (not via multiplexer)
|
||||
- Publishes with the inbox as the reply subject
|
||||
- Waits for the message on that subscription
|
||||
- No multiplexer involvement
|
||||
|
||||
## Data Flow: Flush
|
||||
|
||||
```
|
||||
Application
|
||||
│
|
||||
│ client.flush()
|
||||
│
|
||||
▼
|
||||
Client::flush
|
||||
│ creates oneshot channel
|
||||
│ sends Command::Flush { observer }
|
||||
│
|
||||
▼
|
||||
ConnectionHandler::handle_command(Command::Flush)
|
||||
│ pushes observer into flush_observers Vec
|
||||
│
|
||||
▼
|
||||
ProcessFut::poll (main loop)
|
||||
│ after writing all pending data...
|
||||
│ checks should_flush():
|
||||
│ Yes (write buffers empty, not yet flushed) → poll_flush
|
||||
│ May (write buffers not empty) → poll_flush
|
||||
│ No (already flushed) → skip
|
||||
│ on successful flush:
|
||||
│ drains flush_observers, sending () to each
|
||||
│
|
||||
▼
|
||||
Client::flush receives via oneshot::Receiver
|
||||
```
|
||||
|
||||
## Data Flow: Drain
|
||||
|
||||
```
|
||||
Application
|
||||
│
|
||||
│ client.drain() or subscriber.drain()
|
||||
│
|
||||
▼
|
||||
Client::drain / Subscriber::drain
|
||||
│ sends Command::Drain { sid: None } (whole client)
|
||||
│ or Command::Drain { sid: Some(n) } (single subscription)
|
||||
│
|
||||
▼
|
||||
ConnectionHandler::handle_command(Command::Drain)
|
||||
│ if sid is Some:
|
||||
│ pushes sid to drain_pings
|
||||
│ sends ClientOp::Unsubscribe { sid, max: None }
|
||||
│ if sid is None (whole client):
|
||||
│ sets is_draining = true
|
||||
│ emits Event::Draining
|
||||
│ for each subscription: drain_pings.push(sid), Unsubscribe
|
||||
│ sends ClientOp::Ping (to flush the UNSUB messages)
|
||||
│
|
||||
▼
|
||||
ProcessFut::poll (main loop)
|
||||
│ processes any remaining server messages
|
||||
│ removes drained subscriptions from HashMap
|
||||
│ if is_draining: returns ExitReason::Closed
|
||||
│
|
||||
▼
|
||||
ConnectionHandler exits, emits Event::Closed
|
||||
```
|
||||
|
||||
## Main Processing Loop
|
||||
|
||||
The `ConnectionHandler::process` method implements the core event loop via a custom `Future` (`ProcessFut`):
|
||||
|
||||
```rust
|
||||
impl Future for ProcessFut<'_> {
|
||||
type Output = ExitReason;
|
||||
|
||||
fn poll(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Self::Output> {
|
||||
// 1. Check ping interval — send PING if due, disconnect if too many pending
|
||||
while self.handler.ping_interval.poll_tick(cx).is_ready() {
|
||||
if let Poll::Ready(exit) = self.ping() { return Poll::Ready(exit); }
|
||||
}
|
||||
|
||||
// 2. Read all available server operations
|
||||
loop {
|
||||
match self.handler.connection.poll_read_op(cx) {
|
||||
Poll::Pending => break,
|
||||
Poll::Ready(Ok(Some(server_op))) => self.handler.handle_server_op(server_op),
|
||||
Poll::Ready(Ok(None)) => return Poll::Ready(ExitReason::Disconnected(None)),
|
||||
Poll::Ready(Err(err)) => return Poll::Ready(ExitReason::Disconnected(Some(err))),
|
||||
}
|
||||
}
|
||||
|
||||
// 3. Clean up drained subscriptions
|
||||
while let Some(sid) = self.handler.drain_pings.pop_front() {
|
||||
self.handler.subscriptions.remove(&sid);
|
||||
}
|
||||
|
||||
// 4. If draining, exit
|
||||
if self.handler.is_draining { return Poll::Ready(ExitReason::Closed); }
|
||||
|
||||
// 5. Process client commands (batch of up to 16)
|
||||
// while write buffer not full
|
||||
loop {
|
||||
while !self.handler.connection.is_write_buf_full() {
|
||||
match receiver.poll_recv_many(cx, recv_buf, 16) {
|
||||
Poll::Pending => break,
|
||||
Poll::Ready(1..) => { for cmd in recv_buf.drain(..) { handler.handle_command(cmd); } }
|
||||
Poll::Ready(0) => return Poll::Ready(ExitReason::Closed),
|
||||
}
|
||||
}
|
||||
|
||||
// 6. Write pending data to stream
|
||||
match self.handler.connection.poll_write(cx) {
|
||||
Poll::Pending => break,
|
||||
Poll::Ready(Ok(())) => continue, // write buffer empty, try more commands
|
||||
Poll::Ready(Err(err)) => return Poll::Ready(ExitReason::Disconnected(Some(err))),
|
||||
}
|
||||
}
|
||||
|
||||
// 7. Flush stream and notify observers
|
||||
match self.handler.connection.poll_flush(cx) { ... }
|
||||
|
||||
// 8. Check for forced reconnect
|
||||
if mem::take(&mut self.handler.should_reconnect) {
|
||||
return Poll::Ready(ExitReason::ReconnectRequested);
|
||||
}
|
||||
|
||||
Poll::Pending
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Exit Reasons
|
||||
|
||||
The main loop exits for three reasons:
|
||||
|
||||
| Reason | Action |
|
||||
|--------|--------|
|
||||
| `Disconnected(Option<io::Error>)` | Attempt reconnection via `handle_disconnect()` |
|
||||
| `ReconnectRequested` | Force reconnect (user-triggered) |
|
||||
| `Closed` | Connection handler terminates, emit `Event::Closed` |
|
||||
|
||||
On disconnection, `handle_disconnect()` is called which:
|
||||
1. Resets `pending_pings` to 0
|
||||
2. Emits `Event::Disconnected`
|
||||
3. Updates connection state to `Disconnected`
|
||||
4. Calls `handle_reconnect()` which uses `Connector::connect()`
|
||||
5. On successful reconnect, re-subscribes all active subscriptions
|
||||
6. Re-subscribes the multiplexer wildcard if present
|
||||
|
||||
## Slow Consumer Handling
|
||||
|
||||
When a subscription's `mpsc::Sender` channel is full (the application isn't consuming messages fast enough):
|
||||
|
||||
1. `try_send` returns `TrySendError::Full`
|
||||
2. The `ConnectionHandler` emits `Event::SlowConsumer(sid)`
|
||||
3. The message is **dropped** (not queued)
|
||||
4. The subscription remains active
|
||||
|
||||
When a subscription's receiver is dropped (application closed the stream):
|
||||
|
||||
1. `try_send` returns `TrySendError::Closed`
|
||||
2. The subscription is removed from the HashMap
|
||||
3. An `UNSUB` command is sent to the server
|
||||
|
||||
## Ping/Pong Health Check
|
||||
|
||||
The `ConnectionHandler` maintains a periodic PING interval (default 60 seconds):
|
||||
|
||||
1. `ping_interval` fires every N seconds
|
||||
2. A `ClientOp::Ping` is enqueued
|
||||
3. `pending_pings` counter increments
|
||||
4. If `pending_pings > MAX_PENDING_PINGS (2)`, the connection is considered dead
|
||||
5. When `ServerOp::Pong` is received, `pending_pings` decrements
|
||||
6. Any server operation resets the ping interval timer
|
||||
|
||||
## Batched Command Processing
|
||||
|
||||
Commands from the `Client` are received in batches of up to 16 (`RECV_CHUNK_SIZE`) using `poll_recv_many`. This amortizes the cost of waking the task and enables pipelining multiple operations (e.g., publishing many messages) in a single poll cycle.
|
||||
@@ -0,0 +1,277 @@
|
||||
# Connection and Reconnection
|
||||
|
||||
This document covers how connections are established, TLS handling, the server pool, and the reconnection mechanism.
|
||||
|
||||
## Connector
|
||||
|
||||
**Location**: `connector.rs`
|
||||
|
||||
The `Connector` manages the server pool and handles connection establishment and reconnection.
|
||||
|
||||
```rust
|
||||
pub(crate) struct Connector {
|
||||
servers: Vec<Server>, // Server pool with per-server metadata
|
||||
options: ConnectorOptions, // Connection configuration
|
||||
connect_stats: Arc<Statistics>, // Shared statistics
|
||||
attempts: usize, // Global reconnection attempt counter
|
||||
events_tx: mpsc::Sender<Event>, // Event channel
|
||||
state_tx: watch::Sender<State>, // Connection state watcher
|
||||
max_payload: Arc<AtomicUsize>, // Server's max payload
|
||||
last_info: ServerInfo, // Last known server info
|
||||
}
|
||||
```
|
||||
|
||||
### Server Pool
|
||||
|
||||
Each server in the pool carries metadata:
|
||||
|
||||
```rust
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct Server {
|
||||
pub addr: ServerAddr,
|
||||
pub failed_attempts: usize, // Consecutive failed attempts
|
||||
pub did_connect: bool, // Ever successfully connected?
|
||||
pub is_discovered: bool, // Discovered via INFO, not user-configured
|
||||
pub last_error: Option<String>, // Last connection error
|
||||
}
|
||||
```
|
||||
|
||||
### ConnectorOptions
|
||||
|
||||
```rust
|
||||
pub(crate) struct ConnectorOptions {
|
||||
pub tls_required: bool,
|
||||
pub certificates: Vec<PathBuf>,
|
||||
pub client_cert: Option<PathBuf>,
|
||||
pub client_key: Option<PathBuf>,
|
||||
pub tls_client_config: Option<rustls::ClientConfig>,
|
||||
pub tls_first: bool,
|
||||
pub auth: Auth,
|
||||
pub no_echo: bool,
|
||||
pub connection_timeout: Duration, // Default: 5 seconds
|
||||
pub name: Option<String>,
|
||||
pub ignore_discovered_servers: bool,
|
||||
pub retain_servers_order: bool,
|
||||
pub read_buffer_capacity: u16, // Default: 65535
|
||||
pub reconnect_delay_callback: Arc<dyn Fn(usize) -> Duration>,
|
||||
pub auth_callback: Option<CallbackArg1<Vec<u8>, Result<Auth, AuthError>>>,
|
||||
pub max_reconnects: Option<usize>,
|
||||
pub local_address: Option<SocketAddr>,
|
||||
pub reconnect_to_server_callback: Option<ReconnectToServerCallback>,
|
||||
}
|
||||
```
|
||||
|
||||
## Connection Establishment Flow
|
||||
|
||||
```
|
||||
Connector::try_connect_to_server(addr)
|
||||
│
|
||||
├── 1. DNS resolution
|
||||
│ server_addr.socket_addrs()
|
||||
│
|
||||
├── 2. For each resolved address:
|
||||
│ │
|
||||
│ ├── 2a. Connect with timeout
|
||||
│ │ tokio::time::timeout(connection_timeout, try_connect_to(socket_addr, ...))
|
||||
│ │
|
||||
│ └── 2b. try_connect_to():
|
||||
│ │
|
||||
│ ├── Select transport:
|
||||
│ │ ├── "ws" → WebSocket (tokio_websockets)
|
||||
│ │ ├── "wss" → WebSocket over TLS
|
||||
│ │ └── default → TCP (TcpStream)
|
||||
│ │
|
||||
│ ├── Optional: bind to local_address
|
||||
│ ├── Set TCP_NODELAY
|
||||
│ ├── Create Connection with read_buffer_capacity
|
||||
│ │
|
||||
│ ├── If tls_first: upgrade to TLS before INFO
|
||||
│ │
|
||||
│ ├── Read INFO from server
|
||||
│ │
|
||||
│ ├── If TLS required (by option, server, or URL scheme):
|
||||
│ │ upgrade to TLS (rustls)
|
||||
│ │
|
||||
│ ├── Discover servers from INFO.connect_urls
|
||||
│ │ (unless ignore_discovered_servers)
|
||||
│ │
|
||||
│ ├── Build ConnectInfo with auth:
|
||||
│ │ ├── username/password (from Auth or URL)
|
||||
│ │ ├── token (from Auth)
|
||||
│ │ ├── nkey + signed nonce (feature: nkeys)
|
||||
│ │ ├── JWT + signature callback (feature: nkeys)
|
||||
│ │ └── auth_callback (custom async callback)
|
||||
│ │
|
||||
│ ├── Send CONNECT + PING
|
||||
│ │
|
||||
│ └── Wait for response:
|
||||
│ ├── -ERR (authorization violation) → error
|
||||
│ ├── PONG or +OK → success
|
||||
│ └── EOF → error
|
||||
│
|
||||
└── 3. On success:
|
||||
├── Reset attempt counter
|
||||
├── Increment connects statistic
|
||||
├── Emit Event::Connected
|
||||
├── Update State::Connected
|
||||
├── Store max_payload
|
||||
├── Update per-server metadata (did_connect, failed_attempts)
|
||||
└── Return (ServerInfo, Connection)
|
||||
```
|
||||
|
||||
## TLS Handling
|
||||
|
||||
The client supports three TLS modes:
|
||||
|
||||
### 1. Standard TLS (INFO → TLS)
|
||||
Default behavior. The client receives the `INFO` message in plaintext, then upgrades to TLS if:
|
||||
- `tls_required` option is set
|
||||
- Server's `INFO.tls_required` is true
|
||||
- URL scheme is `tls://`
|
||||
|
||||
### 2. TLS First (TLS → INFO)
|
||||
When `ConnectOptions::tls_first()` is enabled, the client establishes TLS before reading INFO. This requires the server to have `handshake_first` enabled. Useful for environments where plaintext INFO is not acceptable.
|
||||
|
||||
### 3. WebSocket TLS
|
||||
For `wss://` URLs, TLS is handled by the WebSocket library (`tokio-websockets`) directly, not by the client's TLS layer.
|
||||
|
||||
### TLS Configuration
|
||||
The client uses `rustls` via `tokio-rustls`. Configuration steps:
|
||||
1. Load root certificates from system store (`rustls-native-certs`)
|
||||
2. Optionally add custom root certificates from PEM files
|
||||
3. Optionally configure client certificate and key for mTLS
|
||||
4. Optionally pass a custom `rustls::ClientConfig`
|
||||
|
||||
Crypto backend is selectable via feature flags:
|
||||
- `ring` (default)
|
||||
- `aws-lc-rs`
|
||||
- `fips` (requires aws-lc-rs)
|
||||
|
||||
## Reconnection
|
||||
|
||||
### Reconnection Trigger
|
||||
|
||||
Reconnection is triggered when:
|
||||
1. I/O error during read or write (`ExitReason::Disconnected`)
|
||||
2. Too many pending PINGs (no PONG received)
|
||||
3. User calls `Client::force_reconnect()` (`ExitReason::ReconnectRequested`)
|
||||
|
||||
### Reconnection Flow
|
||||
|
||||
```
|
||||
ConnectionHandler::handle_disconnect()
|
||||
│
|
||||
├── Reset pending_pings to 0
|
||||
├── Emit Event::Disconnected
|
||||
├── Update State::Disconnected
|
||||
│
|
||||
└── handle_reconnect()
|
||||
│
|
||||
└── Connector::connect()
|
||||
│
|
||||
└── Loop: try_connect()
|
||||
│
|
||||
├── If reconnect_to_server_callback is set:
|
||||
│ │ Call callback with (server_pool, server_info)
|
||||
│ │ If returns Some(ReconnectToServer):
|
||||
│ │ Validate server is in pool
|
||||
│ │ Use callback's delay or default backoff
|
||||
│ │ Try connecting to selected server
|
||||
│ └── If None or invalid: fall through to default
|
||||
│
|
||||
├── Default selection:
|
||||
│ ├── Shuffle servers (unless retain_servers_order)
|
||||
│ ├── Sort by failed_attempts (ascending)
|
||||
│ └── Try each server in order
|
||||
│
|
||||
├── For each server:
|
||||
│ ├── Increment attempts counter
|
||||
│ ├── Check max_reconnects limit
|
||||
│ ├── Apply reconnect delay (exponential backoff)
|
||||
│ └── try_connect_to_server(addr)
|
||||
│
|
||||
├── On success:
|
||||
│ ├── Reset attempts to 0
|
||||
│ ├── Re-subscribe all active subscriptions
|
||||
│ │ (filter out closed subscription channels)
|
||||
│ ├── Re-subscribe multiplexer wildcard
|
||||
│ └── Return (ServerInfo, Connection)
|
||||
│
|
||||
└── On failure:
|
||||
├── Update per-server metadata (failed_attempts, last_error)
|
||||
├── Auth errors → propagate immediately
|
||||
└── Other errors → continue to next server
|
||||
```
|
||||
|
||||
### Exponential Backoff
|
||||
|
||||
Default reconnect delay function:
|
||||
|
||||
```rust
|
||||
fn reconnect_delay_callback_default(attempts: usize) -> Duration {
|
||||
if attempts <= 1 {
|
||||
Duration::from_millis(0)
|
||||
} else {
|
||||
let exp: u32 = (attempts - 1).try_into().unwrap_or(u32::MAX);
|
||||
let max = Duration::from_secs(4);
|
||||
cmp::min(Duration::from_millis(2_u64.saturating_pow(exp)), max)
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
| Attempt | Delay |
|
||||
|---------|-------|
|
||||
| 1 | 0ms |
|
||||
| 2 | 0ms |
|
||||
| 3 | 2ms |
|
||||
| 4 | 4ms |
|
||||
| 5 | 8ms |
|
||||
| ... | ... |
|
||||
| 13 | 4096ms |
|
||||
| 14+ | 4000ms (capped) |
|
||||
|
||||
Custom delay functions can be provided via `ConnectOptions::reconnect_delay_callback()`.
|
||||
|
||||
### Server Pool Updates
|
||||
|
||||
The server pool is dynamic:
|
||||
|
||||
1. **Initial pool**: from `connect()` / `ConnectOptions::connect()` URL(s)
|
||||
2. **Discovered servers**: added from `INFO.connect_urls` on each connection (unless `ignore_discovered_servers` is set)
|
||||
3. **Runtime updates**: via `Client::set_server_pool()` — replaces the entire pool while preserving per-server state for servers that appear in both old and new pools
|
||||
4. **Order**: servers are shuffled by default (random selection), unless `retain_servers_order` is set
|
||||
|
||||
### Max Reconnects
|
||||
|
||||
The `max_reconnects` option limits total reconnection attempts:
|
||||
- `None` or `0` → unlimited (default)
|
||||
- `Some(n)` → give up after `n` total attempts
|
||||
- Counter is reset on successful connection and when `set_server_pool()` is called
|
||||
|
||||
## ConnectOptions Defaults
|
||||
|
||||
| Option | Default |
|
||||
|--------|---------|
|
||||
| `connection_timeout` | 5 seconds |
|
||||
| `ping_interval` | 60 seconds |
|
||||
| `sender_capacity` | 2048 |
|
||||
| `subscription_capacity` | 65536 |
|
||||
| `inbox_prefix` | `"_INBOX"` |
|
||||
| `request_timeout` | 10 seconds |
|
||||
| `retry_on_initial_connect` | false |
|
||||
| `ignore_discovered_servers` | false |
|
||||
| `retain_servers_order` | false |
|
||||
| `read_buffer_capacity` | 65535 |
|
||||
| `skip_subject_validation` | false |
|
||||
| `no_echo` | false |
|
||||
| `tls_required` | false |
|
||||
| `tls_first` | false |
|
||||
| `max_reconnects` | None (unlimited) |
|
||||
|
||||
## Background Connection
|
||||
|
||||
When `ConnectOptions::retry_on_initial_connect()` is enabled, the `connect()` function returns a `Client` immediately, before the connection is established. The connection is established in a background Tokio task. This means:
|
||||
- `client.server_info()` returns `ServerInfo::default()` until connected
|
||||
- `client.connection_state()` returns `State::Pending`
|
||||
- Operations like `publish()` will queue in the command channel
|
||||
- The `Client` becomes usable once the background task connects
|
||||
@@ -0,0 +1,472 @@
|
||||
# JetStream Internals
|
||||
|
||||
This document covers the JetStream subsystem — how it provides stream-based messaging with persistence, consumer management, and higher-level APIs like KV and Object Store.
|
||||
|
||||
## JetStream Context
|
||||
|
||||
**Location**: `jetstream/context.rs`
|
||||
|
||||
The `Context` is the entry point to the JetStream API. It wraps a `Client` and provides stream management, publishing, and consumer operations.
|
||||
|
||||
```rust
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct Context {
|
||||
pub(crate) client: Client,
|
||||
pub(crate) prefix: String, // API subject prefix (default: "$JS.API")
|
||||
pub(crate) timeout: Duration, // Default request timeout
|
||||
pub(crate) max_ack_semaphore: Arc<Semaphore>, // Limits in-flight ack waits
|
||||
pub(crate) ack_sender: mpsc::Sender<(oneshot::Receiver<Message>, OwnedSemaphorePermit)>,
|
||||
pub(crate) backpressure_on_inflight: bool,
|
||||
}
|
||||
```
|
||||
|
||||
### Context Creation
|
||||
|
||||
```rust
|
||||
// Default context (prefix = "$JS.API")
|
||||
let jetstream = async_nats::jetstream::new(client);
|
||||
|
||||
// With domain (prefix = "$JS.hub.API")
|
||||
let jetstream = async_nats::jetstream::with_domain(client, "hub");
|
||||
|
||||
// With custom prefix
|
||||
let jetstream = async_nats::jetstream::with_prefix(client, "JS.acc@hub.API");
|
||||
|
||||
// Builder pattern for more options
|
||||
let jetstream = async_nats::jetstream::Context::builder(client)
|
||||
.domain("hub")
|
||||
.prefix("$JS.API")
|
||||
.timeout(Duration::from_secs(30))
|
||||
.max_ack_pending(256)
|
||||
.backpressure_on_inflight(true)
|
||||
.build();
|
||||
```
|
||||
|
||||
### JetStream API Subject Convention
|
||||
|
||||
All JetStream API calls are request-response messages sent to subjects following the pattern:
|
||||
|
||||
```
|
||||
$JS.API.<operation>.<stream-name>[.<consumer-name>]
|
||||
```
|
||||
|
||||
Examples:
|
||||
- `$JS.API.STREAM.CREATE.events` — create stream "events"
|
||||
- `$JS.API.STREAM.INFO.events` — get stream info
|
||||
- `$JS.API.CONSUMER.DURABLE.CREATE.events.myconsumer` — create durable consumer
|
||||
- `$JS.API.CONSUMER.MSG.NEXT.events.myconsumer` — pull next message
|
||||
|
||||
With a domain, the prefix changes to `$JS.<domain>.API`.
|
||||
|
||||
## Stream Management
|
||||
|
||||
**Location**: `jetstream/stream.rs`
|
||||
|
||||
### Stream Config
|
||||
|
||||
```rust
|
||||
#[derive(Serialize, Deserialize, Debug, Clone, PartialEq, Eq)]
|
||||
pub struct Config {
|
||||
pub name: String,
|
||||
pub subjects: Vec<String>, // Subject filter
|
||||
pub retention: RetentionPolicy, // Limits, Interest, WorkQueue
|
||||
pub max_consumers: i32,
|
||||
pub max_messages: i64, // Per-stream message limit
|
||||
pub max_messages_per_subject: i64,
|
||||
pub max_bytes: i64, // Per-stream byte limit
|
||||
pub max_age: Duration, // Message TTL
|
||||
pub max_message_size: Option<i32>, // Max individual message size
|
||||
pub storage: StorageType, // File or Memory
|
||||
pub num_replicas: usize,
|
||||
pub no_ack: bool, // Don't require ack
|
||||
pub discard: DiscardPolicy, // Old or New
|
||||
pub duplicate_window: Duration,
|
||||
pub allow_rollup_hdrs: bool,
|
||||
pub allow_direct: bool,
|
||||
pub mirror: Option<External>,
|
||||
pub sources: Vec<External>,
|
||||
pub sealed: bool,
|
||||
pub compression: Option<Compression>, // server_2_10+
|
||||
pub first_sequence: Option<u64>, // server_2_11+
|
||||
pub subject_transform: Option<SubjectTransform>, // server_2_12+
|
||||
pub metadata: Option<HashMap<String, String>>, // server_2_10+
|
||||
pub placement: Option<Placement>,
|
||||
pub republish: Option<RePublish>,
|
||||
}
|
||||
```
|
||||
|
||||
### Stream Operations
|
||||
|
||||
Via `Context`:
|
||||
|
||||
| Method | API Subject | Description |
|
||||
|--------|------------|-------------|
|
||||
| `create_stream(config)` | `STREAM.CREATE.<name>` | Create a new stream |
|
||||
| `get_stream(name)` | `STREAM.INFO.<name>` | Get existing stream |
|
||||
| `get_or_create_stream(config)` | `STREAM.INFO` → `STREAM.CREATE` | Get or create |
|
||||
| `delete_stream(name)` | `STREAM.DELETE.<name>` | Delete a stream |
|
||||
| `update_stream(name, config)` | `STREAM.UPDATE.<name>` | Update stream config |
|
||||
| `purge_stream(name)` | `STREAM.PURGE.<name>` | Purge all messages |
|
||||
| `streams()` | `STREAM.LIST` | List all streams (paged iterator) |
|
||||
| `stream_names()` | `STREAM.NAMES` | List stream names (paged iterator) |
|
||||
| `account_info()` | `ACCOUNT.INFO` | Get account info |
|
||||
|
||||
Via `Stream`:
|
||||
|
||||
| Method | API Subject | Description |
|
||||
|--------|------------|-------------|
|
||||
| `info()` | `STREAM.INFO.<name>` | Refresh stream info |
|
||||
| `purge()` | `STREAM.PURGE.<name>` | Purge messages |
|
||||
| `delete()` | `STREAM.DELETE.<name>` | Delete this stream |
|
||||
| `update(config)` | `STREAM.UPDATE.<name>` | Update config |
|
||||
| `get_raw_message(seq)` | `STREAM.MSG.GET.<name>` | Get message by sequence (stored mode) |
|
||||
| `get_last_message(subject)` | `STREAM.MSG.GET.<name>` | Get last message for subject (stored mode) |
|
||||
| `direct_get_last(subject)` | `DIRECT.GET.<name>` | Direct get last (bypasses RAA) |
|
||||
| `direct_get(seq)` | `DIRECT.GET.<name>` | Direct get by sequence |
|
||||
| `delete_message(seq)` | `STREAM.MSG.DELETE.<name>` | Delete a specific message |
|
||||
| `create_consumer(config)` | `CONSUMER.CREATE.<stream>` | Create consumer |
|
||||
| `get_or_create_consumer(name, config)` | `CONSUMER.DURABLE.CREATE.<stream>.<name>` | Get or create durable |
|
||||
| `get_consumer(name)` | `CONSUMER.INFO.<stream>.<name>` | Get existing consumer |
|
||||
|
||||
### Stream Info
|
||||
|
||||
```rust
|
||||
#[derive(Serialize, Deserialize, Debug, Clone)]
|
||||
pub struct Info {
|
||||
pub config: Config,
|
||||
pub created: DateTime,
|
||||
pub state: State, // Messages, bytes, first/last sequence, consumer count
|
||||
pub cluster: Option<ClusterInfo>,
|
||||
pub timestamp: DateTime,
|
||||
pub leader: Option<String>,
|
||||
pub subjects: Option<HashMap<String, u64>>, // Subject → message count
|
||||
}
|
||||
```
|
||||
|
||||
### Paged List Operations
|
||||
|
||||
Stream and consumer listing uses a paged iterator pattern:
|
||||
|
||||
```rust
|
||||
// streams() returns an iterator that automatically pages
|
||||
let mut streams = jetstream.streams();
|
||||
while let Some(stream) = streams.next().await {
|
||||
let stream = stream?;
|
||||
// process stream
|
||||
}
|
||||
|
||||
// stream_names() similarly pages
|
||||
let mut names = jetstream.stream_names();
|
||||
while let Some(name) = names.next().await {
|
||||
println!("{}", name?);
|
||||
}
|
||||
```
|
||||
|
||||
The paged iterator sends an initial request with `offset: 0` and continues fetching pages until no more results are returned.
|
||||
|
||||
## Publishing
|
||||
|
||||
**Location**: `jetstream/context.rs`, `jetstream/publish.rs`
|
||||
|
||||
### Publish
|
||||
|
||||
```rust
|
||||
// Basic publish (fire-and-forget)
|
||||
jetstream.publish("events.data", "payload".into()).await?;
|
||||
|
||||
// Publish with custom message builder
|
||||
jetstream.publish_message(
|
||||
jetstream::message::PublishMessage::build()
|
||||
.payload("data".into())
|
||||
.message_id("unique-id") // Nats-Msg-Id header for dedup
|
||||
.expected_last_message_id("prev") // Nats-Expected-Last-Msg-Id
|
||||
.expected_last_sequence(42) // Nats-Expected-Last-Sequence
|
||||
.expected_last_subject_sequence("events", 10) // Per-subject sequence
|
||||
.header("Custom", "Value")
|
||||
).await?;
|
||||
```
|
||||
|
||||
### PublishAck
|
||||
|
||||
When a message is published to a JetStream stream, the server responds with a `PublishAck`:
|
||||
|
||||
```rust
|
||||
#[derive(Serialize, Deserialize, Debug, Clone, PartialEq, Eq)]
|
||||
pub struct PublishAck {
|
||||
pub stream: String,
|
||||
pub sequence: u64,
|
||||
pub domain: Option<String>,
|
||||
pub duplicate: bool,
|
||||
}
|
||||
```
|
||||
|
||||
### PublishAckFuture
|
||||
|
||||
Publishing returns a `PublishAckFuture` that resolves to `PublishAck`. The future uses a semaphore (`max_ack_semaphore`) to limit in-flight ack waits and prevent backpressure issues.
|
||||
|
||||
When `backpressure_on_inflight` is enabled, the publish operation blocks if there are too many pending acks, preventing the command channel from filling up with unbounded publish operations.
|
||||
|
||||
### Idempotent Publishing
|
||||
|
||||
Headers for exactly-once semantics:
|
||||
|
||||
| Header | Purpose |
|
||||
|--------|---------|
|
||||
| `Nats-Msg-Id` | Message ID for deduplication within the stream's duplicate window |
|
||||
| `Nats-Expected-Last-Msg-Id` | Expected last message ID (conditional publish) |
|
||||
| `Nats-Expected-Last-Sequence` | Expected last sequence number |
|
||||
| `Nats-Expected-Last-Subject-Sequence` | Expected last sequence for a specific subject |
|
||||
|
||||
## Consumers
|
||||
|
||||
**Location**: `jetstream/consumer/`
|
||||
|
||||
### Consumer Types
|
||||
|
||||
| Type | Description |
|
||||
|------|-------------|
|
||||
| `PullConsumer` | Client pulls messages on demand |
|
||||
| `PushConsumer` | Server pushes messages to a delivery subject |
|
||||
| `OrderedConsumer` | Push consumer with automatic re-creation on failure |
|
||||
|
||||
### Consumer Config
|
||||
|
||||
```rust
|
||||
#[derive(Serialize, Deserialize, Debug, Clone, PartialEq, Eq)]
|
||||
pub struct Config {
|
||||
pub name: Option<String>,
|
||||
pub durable_name: Option<String>,
|
||||
pub description: Option<String>,
|
||||
pub deliver_subject: Option<String>, // Push consumers only
|
||||
pub ack_policy: AckPolicy,
|
||||
pub ack_wait: Duration,
|
||||
pub max_deliver: i64,
|
||||
pub max_ack_pending: i32,
|
||||
pub max_waiting: i32, // Pull consumers only
|
||||
pub filter_subject: Option<String>,
|
||||
pub replay_policy: ReplayPolicy,
|
||||
pub sample_frequency: Option<i8>,
|
||||
pub max_batch: i32, // Pull consumers
|
||||
pub max_expires: Duration, // Pull consumers
|
||||
pub inactive_threshold: Duration,
|
||||
pub flow_control: bool, // Push consumers
|
||||
pub heartbeat: Option<Duration>, // Push consumers
|
||||
pub backoff: Vec<Duration>,
|
||||
pub deliver_group: Option<String>,
|
||||
pub num_replicas: usize,
|
||||
pub mem_storage: bool,
|
||||
pub metadata: Option<HashMap<String, String>>,
|
||||
pub ack_markers: Option<Vec<String>>, // server_2_12+
|
||||
}
|
||||
```
|
||||
|
||||
### Pull Consumer
|
||||
|
||||
**Location**: `jetstream/consumer/pull.rs`
|
||||
|
||||
Pull consumers require explicit requests for messages:
|
||||
|
||||
```rust
|
||||
// Batch request
|
||||
let mut messages = consumer.messages().await?.take(100);
|
||||
while let Some(message) = messages.next().await {
|
||||
let message = message?;
|
||||
message.ack().await?;
|
||||
}
|
||||
|
||||
// Sequence-based batch
|
||||
let mut batches = consumer.sequence(50)?.take(10);
|
||||
while let Some(mut batch) = batches.try_next().await? {
|
||||
while let Some(Ok(message)) = batch.next().await {
|
||||
message.ack().await?;
|
||||
}
|
||||
}
|
||||
|
||||
// Single message fetch
|
||||
let message = consumer.fetch().await?;
|
||||
```
|
||||
|
||||
Pull requests are sent to: `$JS.API.CONSUMER.MSG.NEXT.<stream>.<consumer>`
|
||||
|
||||
The request payload is JSON:
|
||||
```json
|
||||
{"batch": 10, "expires": 5000, "no_wait": false}
|
||||
```
|
||||
|
||||
### Push Consumer
|
||||
|
||||
**Location**: `jetstream/consumer/push.rs`
|
||||
|
||||
Push consumers receive messages automatically on a delivery subject. The client subscribes to the delivery subject and processes messages as they arrive.
|
||||
|
||||
Features:
|
||||
- **Flow control** — server sends flow control messages, client responds to maintain delivery rate
|
||||
- **Heartbeats** — idle heartbeats (status code 100) when no messages are available
|
||||
- **Ordered consumers** — automatically recreated on delivery failures with correct sequence positioning
|
||||
|
||||
### Acknowledgment
|
||||
|
||||
**Location**: `jetstream/message.rs`
|
||||
|
||||
JetStream messages support multiple acknowledgment types:
|
||||
|
||||
```rust
|
||||
pub enum AckKind {
|
||||
Ack, // Ack (message processed)
|
||||
Nack, // Nak (re-deliver)
|
||||
Progress, // Progress (still working)
|
||||
Next, // Next (ack + pull next)
|
||||
Term, // Term (don't redeliver, remove from stream)
|
||||
All, // Ack all messages up to this sequence
|
||||
}
|
||||
```
|
||||
|
||||
Methods on JetStream `Message`:
|
||||
- `ack()` — simple acknowledgment
|
||||
- `ack_with(kind)` — acknowledgment with specific type
|
||||
- `double_ack()` — exactly-once ack (ACK + separate ack message)
|
||||
- `nack()` — negative acknowledgment (request redelivery)
|
||||
- `in_progress()` — progress indicator
|
||||
- `term()` — terminate message (no redelivery)
|
||||
|
||||
## JetStream Message
|
||||
|
||||
**Location**: `jetstream/message.rs`
|
||||
|
||||
JetStream messages wrap core `Message` with metadata extracted from headers:
|
||||
|
||||
```rust
|
||||
#[derive(Debug)]
|
||||
pub struct Message {
|
||||
pub message: crate::Message, // The underlying NATS message
|
||||
pub context: Context, // JetStream context for acking
|
||||
pub ack_pending: Arc<AtomicU64>, // Pending ack counter
|
||||
}
|
||||
|
||||
impl Message {
|
||||
pub fn info(&self) -> Result<Info, MessageInfoError> // Parse message info from headers
|
||||
pub async fn ack(&self) -> Result<(), AckError>
|
||||
pub async fn ack_with(&self, kind: AckKind) -> Result<(), AckError>
|
||||
pub async fn double_ack(&self) -> Result<(), AckError>
|
||||
pub async fn nack(&self) -> Result<(), AckError>
|
||||
pub async fn in_progress(&self) -> Result<(), AckError>
|
||||
pub async fn term(&self) -> Result<(), AckError>
|
||||
}
|
||||
```
|
||||
|
||||
Message info is extracted from the `HMSG` headers:
|
||||
- `Nats-Stream` — stream name
|
||||
- `Nats-Consumer` — consumer name
|
||||
- `Nats-Delivered` — delivery count
|
||||
- `Nats-Sequence` — stream sequence
|
||||
- `Nats-Time-Stamp` — timestamp
|
||||
- `Nats-Subject` — original subject
|
||||
- `Nats-Pending-Messages` / `Nats-Pending-Bytes` — pending counts
|
||||
|
||||
## Key-Value Store
|
||||
|
||||
**Location**: `jetstream/kv/`
|
||||
|
||||
The KV store is a JetStream-based key-value API. Each bucket maps to a JetStream stream with specific configuration:
|
||||
|
||||
```rust
|
||||
// Create a KV store
|
||||
let kv = jetstream
|
||||
.create_key_value(async_nats::jetstream::kv::Config {
|
||||
bucket: "my_bucket".to_string(),
|
||||
history: 5, // Max history per key (1-64)
|
||||
ttl: Duration::from_secs(3600), // Key TTL
|
||||
max_bytes: 1024 * 1024, // Max bucket size
|
||||
storage: StorageType::File,
|
||||
replicas: 1,
|
||||
..Default::default()
|
||||
})
|
||||
.await?;
|
||||
```
|
||||
|
||||
Under the hood:
|
||||
- Each key is stored as a message with subject `$KV.<bucket>.<key>`
|
||||
- Keys support wildcard patterns (`$KV.bucket.prefix.*`)
|
||||
- History is managed via stream `max_messages_per_subject`
|
||||
- TTL is managed via stream `max_age`
|
||||
- `put(key, value)` publishes to the key subject
|
||||
- `get(key)` reads the last message for the key subject
|
||||
- `delete(key)` publishes an internal delete marker
|
||||
- `purge(key)` uses stream purge API
|
||||
- `watch()` subscribes to key changes and returns a `Watch` stream
|
||||
- `keys()` / `history(key)` list keys and history
|
||||
|
||||
## Object Store
|
||||
|
||||
**Location**: `jetstream/object_store/`
|
||||
|
||||
The Object Store provides large object storage built on JetStream. Objects are chunked and stored across multiple messages in a stream.
|
||||
|
||||
```rust
|
||||
// Create an object store
|
||||
let store = jetstream
|
||||
.create_object_store(async_nats::jetstream::object_store::Config {
|
||||
bucket: "my_objects".to_string(),
|
||||
..Default::default()
|
||||
})
|
||||
.await?;
|
||||
|
||||
// Put an object
|
||||
let info = store.put("file.txt", stream).await?;
|
||||
|
||||
// Get an object
|
||||
let mut object_stream = store.get("file.txt").await?;
|
||||
```
|
||||
|
||||
Under the hood:
|
||||
- Objects are chunked into ~128KB messages
|
||||
- Metadata (object info) is stored as the first "chunk 0" message
|
||||
- Each chunk is a message with subject `$OBJ.<bucket>.<object-nuid>.C<chunk-number>`
|
||||
- Metadata includes: name, description, headers, size, chunks, digest (SHA-256)
|
||||
- `get()` returns a stream of chunks
|
||||
- Links allow referencing one object from another (like symlinks)
|
||||
|
||||
## JetStream Error Codes
|
||||
|
||||
**Location**: `jetstream/errors.rs`
|
||||
|
||||
Standard JetStream error codes returned by the server:
|
||||
|
||||
| Code | Constant | Description |
|
||||
|------|----------|-------------|
|
||||
| 10001 | `NOT_FOUND` | Resource not found |
|
||||
| 10002 | `STREAM_NOT_FOUND` | Stream not found |
|
||||
| 10003 | `CONSUMER_NOT_FOUND` | Consumer not found |
|
||||
| 10004 | `REQUEST_NOT_FOUND` | Request not found |
|
||||
| 10005 | `STREAM_WRONG_LAST_SEQ` | Wrong last sequence |
|
||||
| 10006 | `STREAM_NAME_EXISTS` | Stream already exists |
|
||||
| 10007 | `CONSUMER_NAME_EXISTS` | Consumer already exists |
|
||||
| 10008 | `INSUFFICIENT_RESOURCES` | Insufficient resources |
|
||||
| 10009 | `NO_MESSAGE_FOUND` | No message found |
|
||||
| 10013 | `CONSUMER_EXISTS` | Consumer already exists (duplicate) |
|
||||
| 10014 | `STREAM_NOT_CONFIGURED` | Stream not configured |
|
||||
| 10015 | `CLUSTER_NOT_ACTIVE` | Cluster not active |
|
||||
| 10016 | `CLUSTER_NOT_LEADER` | Not the cluster leader |
|
||||
| 10017 | `CLUSTER_NOT_ENOUGH_PEERS` | Not enough peers |
|
||||
| 10018 | `CLUSTER_INCOMPLETE` | Cluster incomplete |
|
||||
| 10019 | `CONSUMER_DELETED` | Consumer was deleted |
|
||||
| 10020 | `CONSUMER_BAD_ACK` | Bad acknowledgment |
|
||||
| 10021 | `CONSUMER_BAD_SUBJECT` | Bad consumer subject |
|
||||
| 10022 | `CONSUMER_DELETED_DRIFT` | Consumer deleted due to drift |
|
||||
| ... | ... | Additional codes |
|
||||
|
||||
## Account
|
||||
|
||||
**Location**: `jetstream/account.rs`
|
||||
|
||||
The `Account` struct provides information about the JetStream account:
|
||||
|
||||
```rust
|
||||
pub struct Account {
|
||||
pub memory: i64,
|
||||
pub storage: i64,
|
||||
pub streams: i64,
|
||||
pub consumers: i64,
|
||||
pub limits: AccountLimits,
|
||||
}
|
||||
```
|
||||
@@ -0,0 +1,292 @@
|
||||
# Authentication and Security
|
||||
|
||||
This document covers the authentication mechanisms, TLS configuration, and security-related features of the async-nats client.
|
||||
|
||||
## Authentication Methods
|
||||
|
||||
The NATS server supports multiple authentication methods. The client implements all of them.
|
||||
|
||||
### 1. Username/Password
|
||||
|
||||
The simplest authentication method.
|
||||
|
||||
```rust
|
||||
// Via ConnectOptions
|
||||
let client = ConnectOptions::with_user_and_password("user".into(), "pass".into())
|
||||
.connect("nats://localhost")
|
||||
.await?;
|
||||
|
||||
// Via URL
|
||||
let client = connect("nats://user:pass@localhost:4222").await?;
|
||||
```
|
||||
|
||||
These credentials are sent in the `CONNECT` message as `user` and `pass` fields.
|
||||
|
||||
### 2. Token Authentication
|
||||
|
||||
A single token used for authentication.
|
||||
|
||||
```rust
|
||||
let client = ConnectOptions::with_token("my-token".into())
|
||||
.connect("nats://localhost")
|
||||
.await?;
|
||||
```
|
||||
|
||||
Token is sent in the `CONNECT` message as `auth_token` field.
|
||||
|
||||
### 3. NKey Authentication
|
||||
|
||||
NKey-based authentication using Ed25519 key pairs. Requires the `nkeys` feature.
|
||||
|
||||
```rust
|
||||
let seed = "SUANQDPB2RUOE4ETUA26CNX7FUKE5ZZKFCQIIW63OX225F2CO7UEXTM7ZY";
|
||||
let client = ConnectOptions::with_nkey(seed.into())
|
||||
.connect("nats://localhost")
|
||||
.await?;
|
||||
```
|
||||
|
||||
Flow:
|
||||
1. Server sends `INFO` with a `nonce` field
|
||||
2. Client creates a `KeyPair` from the seed
|
||||
3. Client signs the nonce: `key_pair.sign(nonce.as_bytes())`
|
||||
4. Client sends `CONNECT` with `nkey` (public key) and `sig` (Base64URL-encoded signature)
|
||||
5. Server verifies the signature against the public key and nonce
|
||||
|
||||
### 4. JWT Authentication
|
||||
|
||||
User JWT with a signing callback. Requires the `nkeys` feature.
|
||||
|
||||
```rust
|
||||
let key_pair = Arc::new(nkeys::KeyPair::from_seed(seed)?);
|
||||
let jwt = load_jwt().await?;
|
||||
|
||||
let client = ConnectOptions::with_jwt(jwt, move |nonce| {
|
||||
let key_pair = key_pair.clone();
|
||||
async move { key_pair.sign(&nonce).map_err(AuthError::new) }
|
||||
})
|
||||
.connect("nats://localhost")
|
||||
.await?;
|
||||
```
|
||||
|
||||
Flow:
|
||||
1. Server sends `INFO` with a `nonce` field
|
||||
2. Client sends `CONNECT` with `jwt` (user JWT) and `sig` (Base64URL-encoded nonce signature)
|
||||
3. The signing callback is async, allowing integration with external signing services (e.g., HSM)
|
||||
|
||||
### 5. Credentials File
|
||||
|
||||
Combines JWT and NKey from a `.creds` file. Requires the `nkeys` feature.
|
||||
|
||||
```rust
|
||||
// From file
|
||||
let client = ConnectOptions::with_credentials_file("path/to/my.creds")
|
||||
.await?
|
||||
.connect("nats://localhost")
|
||||
.await?;
|
||||
|
||||
// From string
|
||||
let client = ConnectOptions::with_credentials(creds_string)
|
||||
.connect("nats://localhost")
|
||||
.await?;
|
||||
```
|
||||
|
||||
Credentials file format:
|
||||
```
|
||||
-----BEGIN NATS USER JWT-----
|
||||
eyJ0eXAiOiJqd3QiLCJhbGciOiJlZDI1NTE5...
|
||||
------END NATS USER JWT------
|
||||
|
||||
************************* IMPORTANT *************************
|
||||
NKEY Seed printed below can be used sign and prove identity.
|
||||
|
||||
-----BEGIN USER NKEY SEED-----
|
||||
SUAIO3FHUX5PNV2LQIIP7TZ3N4L7TX3W53MQGEIVYFIGA635OZCKEYHFLM
|
||||
------END USER NKEY SEED------
|
||||
```
|
||||
|
||||
**Location**: `auth_utils.rs` handles parsing:
|
||||
- `load_creds(path)` — async file read + parse
|
||||
- `parse_jwt_and_key_from_creds(creds)` — extracts JWT and KeyPair from the string
|
||||
|
||||
### 6. Auth Callback
|
||||
|
||||
A custom async callback that receives the server nonce and returns an `Auth` struct. This is the most flexible mechanism.
|
||||
|
||||
```rust
|
||||
let client = ConnectOptions::with_auth_callback(move |nonce| {
|
||||
async move {
|
||||
let mut auth = Auth::new();
|
||||
auth.username = Some("user".to_string());
|
||||
auth.password = Some("pass".to_string());
|
||||
// Can also set jwt, nkey, signature, token
|
||||
Ok(auth)
|
||||
}
|
||||
})
|
||||
.connect("nats://localhost")
|
||||
.await?;
|
||||
```
|
||||
|
||||
The callback is invoked on each connection/reconnection, allowing dynamic credential refresh (e.g., refreshing JWTs from an auth server).
|
||||
|
||||
### 7. URL-Embedded Credentials
|
||||
|
||||
```rust
|
||||
// Username and password in URL
|
||||
let client = connect("nats://user:pass@localhost:4222").await?;
|
||||
|
||||
// Token in URL (username field)
|
||||
let client = connect("nats://token@localhost:4222").await?;
|
||||
```
|
||||
|
||||
## Auth Struct
|
||||
|
||||
**Location**: `auth.rs`
|
||||
|
||||
The `Auth` struct is a container for all authentication methods. Multiple fields can be set simultaneously:
|
||||
|
||||
```rust
|
||||
#[derive(Clone, Default)]
|
||||
pub struct Auth {
|
||||
pub jwt: Option<String>,
|
||||
pub nkey: Option<String>,
|
||||
pub signature_callback: Option<CallbackArg1<String, Result<String, AuthError>>>,
|
||||
pub signature: Option<Vec<u8>>,
|
||||
pub username: Option<String>,
|
||||
pub password: Option<String>,
|
||||
pub token: Option<String>,
|
||||
}
|
||||
```
|
||||
|
||||
Priority in `Connector::try_connect_to()`:
|
||||
1. Auth callback overrides all other methods
|
||||
2. NKey authentication (if `auth.nkey` is set)
|
||||
3. JWT authentication (if `auth.jwt` is set)
|
||||
4. Username/password/token from `Auth` struct
|
||||
5. Username/password from URL
|
||||
|
||||
## TLS Configuration
|
||||
|
||||
### TLS Modes
|
||||
|
||||
| Mode | When | Description |
|
||||
|------|------|-------------|
|
||||
| None | Default | Plaintext connection |
|
||||
| Standard | `tls_required` or server requires | TLS after INFO |
|
||||
| TLS First | `tls_first` option | TLS before INFO |
|
||||
| WebSocket | `wss://` URL | TLS handled by WebSocket library |
|
||||
|
||||
### TLS Setup
|
||||
|
||||
**Location**: `tls.rs`
|
||||
|
||||
The `config_tls()` function builds a `rustls::ClientConfig`:
|
||||
|
||||
1. Create `RootCertStore` and load native system certificates
|
||||
2. Add custom root certificates from configured PEM files
|
||||
3. Build `ClientConfig` with the chosen crypto provider:
|
||||
- `ring` (default)
|
||||
- `aws-lc-rs`
|
||||
- `fips` (aws-lc-rs in FIPS mode)
|
||||
4. If client certificate + key are configured, add them for mTLS
|
||||
5. If a custom `rustls::ClientConfig` was provided, use it directly
|
||||
|
||||
### TLS First
|
||||
|
||||
```rust
|
||||
let client = ConnectOptions::new()
|
||||
.tls_first()
|
||||
.connect("nats://localhost")
|
||||
.await?;
|
||||
```
|
||||
|
||||
This sets both `tls_first = true` and `tls_required = true`. The client performs TLS handshake before reading the `INFO` message. The server must have `handshake_first: true` in its configuration.
|
||||
|
||||
### Custom TLS Configuration
|
||||
|
||||
```rust
|
||||
let tls_client = rustls::ClientConfig::builder()
|
||||
.with_root_certificates(root_store)
|
||||
.with_no_client_auth();
|
||||
|
||||
let client = ConnectOptions::new()
|
||||
.require_tls(true)
|
||||
.tls_client_config(tls_client)
|
||||
.connect("nats://localhost")
|
||||
.await?;
|
||||
```
|
||||
|
||||
### mTLS (Mutual TLS)
|
||||
|
||||
```rust
|
||||
let client = ConnectOptions::new()
|
||||
.add_root_certificates("ca.pem".into())
|
||||
.add_client_certificate("cert.pem".into(), "key.pem".into())
|
||||
.connect("tls://localhost")
|
||||
.await?;
|
||||
```
|
||||
|
||||
## WebSocket Transport
|
||||
|
||||
Requires the `websockets` feature. Supports `ws://` and `wss://` schemes.
|
||||
|
||||
```rust
|
||||
let client = connect("ws://localhost:8080").await?;
|
||||
let client = connect("wss://localhost:443").await?;
|
||||
```
|
||||
|
||||
Implementation uses `tokio-websockets` with a `WebSocketAdapter` that wraps the WebSocket stream to implement `AsyncRead + AsyncWrite`:
|
||||
|
||||
```rust
|
||||
// WebSocketAdapter bridges WebSocket messages to byte streams
|
||||
pub(crate) struct WebSocketAdapter<T> {
|
||||
pub(crate) inner: WebSocketStream<T>,
|
||||
pub(crate) read_buf: BytesMut, // Buffered incoming WebSocket messages
|
||||
}
|
||||
```
|
||||
|
||||
For `wss://`, TLS is configured within the WebSocket connector, not via the client's TLS layer.
|
||||
|
||||
## Security Considerations
|
||||
|
||||
### Nonce Signing
|
||||
|
||||
The server's `nonce` in the `INFO` message prevents replay attacks:
|
||||
- Each connection gets a unique nonce
|
||||
- The nonce must be signed with the client's private key
|
||||
- The signature is verified server-side against the public key
|
||||
|
||||
### Authorization Violations
|
||||
|
||||
When the server sends `-ERR 'authorization violation'`:
|
||||
- The client parses this as `ServerError::AuthorizationViolation`
|
||||
- The `Connector` immediately propagates this error (does not retry)
|
||||
- The error is converted to `ConnectErrorKind::AuthorizationViolation`
|
||||
|
||||
### Subject Validation
|
||||
|
||||
By default, the client validates subjects for protocol safety:
|
||||
- **Publish subjects**: checked for emptiness and whitespace (can be disabled with `skip_subject_validation`)
|
||||
- **Subscribe subjects**: always checked for emptiness, whitespace, leading/trailing dots, consecutive dots
|
||||
- **Queue group names**: checked for emptiness and whitespace
|
||||
|
||||
The server enforces its own validation, but client-side checks prevent protocol-framing errors.
|
||||
|
||||
### Max Payload Size
|
||||
|
||||
The client checks payload size against the server's `max_payload` before publishing:
|
||||
- For plain messages: `payload.len() > max_payload`
|
||||
- For messages with headers: `headers.wire_len() + payload.len() > max_payload`
|
||||
- Returns `PublishErrorKind::MaxPayloadExceeded` if exceeded
|
||||
|
||||
### No Echo
|
||||
|
||||
When `no_echo` is set, the `CONNECT` message includes `echo: false`. The server will not deliver messages published by this connection back to its own subscriptions. This prevents feedback loops.
|
||||
|
||||
### Lame Duck Mode
|
||||
|
||||
When the server enters lame duck mode (draining for shutdown):
|
||||
1. Server sends `INFO` with `ldm: true`
|
||||
2. Client emits `Event::LameDuckMode`
|
||||
3. Application should gracefully close or reconnect to another server
|
||||
|
||||
The `nats-server` test harness provides `set_lame_duck_mode(server)` for testing this behavior.
|
||||
347
docs/research/references/nats.rs/nats-server/08-test-harness.md
Normal file
347
docs/research/references/nats.rs/nats-server/08-test-harness.md
Normal file
@@ -0,0 +1,347 @@
|
||||
# nats-server Test Harness
|
||||
|
||||
This document covers the `nats-server` crate — a test harness for spawning real NATS server instances in integration tests.
|
||||
|
||||
**Location**: `nats-server/src/lib.rs`
|
||||
**Version**: 0.1.0
|
||||
**License**: Apache-2.0
|
||||
**Dependencies**: `lazy_static`, `regex`, `serde_json`, `nuid`, `rand`, `tokio-retry`
|
||||
|
||||
## What It Is
|
||||
|
||||
The `nats-server` crate is **not** a NATS server implementation. It is a thin test harness that:
|
||||
- Spawns the Go-based `nats-server` binary as a child process
|
||||
- Configures it for test use (dynamic ports, temp storage, log files)
|
||||
- Discovers the client URL from the server's `INFO` protocol message
|
||||
- Cleans up resources (JetStream storage, logs, PID files) on `Drop`
|
||||
- Supports single servers and 3-node clusters
|
||||
|
||||
The actual NATS server must be installed separately (Go binary from `github.com/nats-io/nats-server`).
|
||||
|
||||
## Server Struct
|
||||
|
||||
```rust
|
||||
pub struct Server {
|
||||
inner: Inner,
|
||||
}
|
||||
|
||||
struct Inner {
|
||||
cfg: String, // Config file path
|
||||
id: String, // Unique server ID (NUID)
|
||||
port: Option<String>, // Explicit port (None = dynamic)
|
||||
child: Child, // Child process handle
|
||||
logfile: PathBuf, // Log file path in temp dir
|
||||
pidfile: PathBuf, // PID file path in temp dir
|
||||
}
|
||||
```
|
||||
|
||||
## Public API
|
||||
|
||||
### run_server
|
||||
|
||||
```rust
|
||||
pub fn run_server(cfg: &str) -> Server
|
||||
```
|
||||
|
||||
Starts a single NATS server with optional config file.
|
||||
|
||||
- Uses dynamic port (`-1` flag) for parallel test execution
|
||||
- Stores JetStream data in temp directory
|
||||
- Writes logs to temp file: `nats-server-<id>.log`
|
||||
- Writes PID to temp file: `nats-server-<id>.pid`
|
||||
- If `cfg` is non-empty, passes `-c <cfg>` to the server
|
||||
|
||||
Example:
|
||||
```rust
|
||||
let server = nats_server::run_server("tests/configs/jetstream.conf");
|
||||
let client = async_nats::connect(server.client_url()).await.unwrap();
|
||||
```
|
||||
|
||||
### run_basic_server
|
||||
|
||||
```rust
|
||||
pub fn run_basic_server() -> Server
|
||||
```
|
||||
|
||||
Starts a server with no config (bare minimum). Equivalent to `run_server("")`.
|
||||
|
||||
### run_server_with_port
|
||||
|
||||
```rust
|
||||
pub fn run_server_with_port(cfg: &str, port: Option<&str>) -> Server
|
||||
```
|
||||
|
||||
Starts a server with an explicit port. If `None`, uses dynamic port.
|
||||
|
||||
### run_cluster
|
||||
|
||||
```rust
|
||||
pub fn run_cluster<'a, C: IntoConfig<'a>>(cfg: C) -> Cluster
|
||||
```
|
||||
|
||||
Starts a 3-node cluster with the given config.
|
||||
|
||||
- Allocates 3 random port ranges (base, base+100, base+200)
|
||||
- Configures cluster routes between nodes
|
||||
- Each node gets: `--cluster nats://127.0.0.1:<cluster_port>`, `--routes <other_routes>`, `--cluster_name cluster`, `-n nodeN`
|
||||
- Waits 2 seconds for cluster formation and leader election
|
||||
|
||||
The `IntoConfig` trait allows passing either a single config string (applied to all 3 nodes) or an array of 3 configs (one per node):
|
||||
|
||||
```rust
|
||||
// Same config for all nodes
|
||||
let cluster = run_cluster("configs/jetstream.conf");
|
||||
|
||||
// Different configs per node
|
||||
let cluster = run_cluster(["node1.conf", "node2.conf", "node3.conf"]);
|
||||
```
|
||||
|
||||
### Cluster Struct
|
||||
|
||||
```rust
|
||||
pub struct Cluster {
|
||||
pub servers: Vec<Server>,
|
||||
}
|
||||
|
||||
impl Cluster {
|
||||
pub fn client_url(&self) -> String {
|
||||
self.servers[0].client_url()
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Server Methods
|
||||
|
||||
```rust
|
||||
impl Server {
|
||||
pub fn restart(&mut self)
|
||||
pub fn client_url(&self) -> String
|
||||
pub fn client_port(&self) -> u16
|
||||
pub fn client_url_with(&self, user: &str, pass: &str) -> String
|
||||
pub fn client_url_with_token(&self, token: &str) -> String
|
||||
pub fn client_pid(&self) -> usize
|
||||
}
|
||||
```
|
||||
|
||||
#### restart()
|
||||
|
||||
Kills the current server process, waits for it to exit, then restarts with the same config, port, and ID. Used for testing reconnection behavior.
|
||||
|
||||
#### client_url()
|
||||
|
||||
Connects to the server's TCP port, reads the `INFO` line, parses the JSON, and constructs a URL:
|
||||
- `nats://localhost:<port>` for non-TLS
|
||||
- `tls://localhost:<port>` for TLS-required servers
|
||||
|
||||
Polls the log file (up to 10 seconds) to discover the client address, since the port may be dynamically allocated.
|
||||
|
||||
#### client_pid()
|
||||
|
||||
Reads the PID file and returns the server process ID. Used for sending signals.
|
||||
|
||||
### set_lame_duck_mode
|
||||
|
||||
```rust
|
||||
pub fn set_lame_duck_mode(s: &Server)
|
||||
```
|
||||
|
||||
Sends the lame duck mode signal to the server:
|
||||
|
||||
```bash
|
||||
nats-server --signal ldm=<pid>
|
||||
```
|
||||
|
||||
### is_port_available
|
||||
|
||||
```rust
|
||||
pub fn is_port_available(port: usize) -> bool
|
||||
```
|
||||
|
||||
Tests if a TCP port is available by attempting to bind to it.
|
||||
|
||||
## Server Lifecycle
|
||||
|
||||
### Spawning
|
||||
|
||||
The `do_run` function constructs and spawns the server process:
|
||||
|
||||
```rust
|
||||
fn do_run(cfg: &str, port: Option<&str>, id: Option<String>) -> Inner {
|
||||
let id = id.unwrap_or_else(|| nuid::next().to_string());
|
||||
let logfile = env::temp_dir().join(format!("nats-server-{id}.log"));
|
||||
let pidfile = env::temp_dir().join(format!("nats-server-{id}.pid"));
|
||||
let store_dir = env::temp_dir().join(format!("store-dir-{id}"));
|
||||
|
||||
let mut cmd = Command::new("nats-server");
|
||||
cmd.arg("--store_dir").arg(store_dir.as_path())
|
||||
.arg("-p");
|
||||
|
||||
match port {
|
||||
Some(port) => cmd.arg(port),
|
||||
None => cmd.arg("-1"), // Dynamic port
|
||||
};
|
||||
|
||||
cmd.arg("-l").arg(logfile.as_os_str())
|
||||
.arg("-P").arg(pidfile.as_os_str());
|
||||
|
||||
if !cfg.is_empty() {
|
||||
cmd.arg("-c").arg(cfg);
|
||||
}
|
||||
|
||||
let child = cmd.spawn().unwrap();
|
||||
// ...
|
||||
}
|
||||
```
|
||||
|
||||
Key flags:
|
||||
- `--store_dir` — JetStream storage directory in temp
|
||||
- `-p -1` — Dynamic port allocation (or explicit port)
|
||||
- `-l` — Log file path
|
||||
- `-P` — PID file path
|
||||
- `-c` — Config file path
|
||||
|
||||
### Cleanup (Drop)
|
||||
|
||||
```rust
|
||||
impl Drop for Server {
|
||||
fn drop(&mut self) {
|
||||
self.inner.child.kill().unwrap();
|
||||
self.inner.child.wait().unwrap();
|
||||
|
||||
if let Ok(log) = fs::read_to_string(self.inner.logfile.as_os_str()) {
|
||||
// Clean up JetStream storage directory if found in log
|
||||
if let Some(caps) = SD_RE.captures(&log) {
|
||||
let sd = caps.get(1).map_or("", |m| m.as_str());
|
||||
fs::remove_dir_all(sd).ok();
|
||||
}
|
||||
// Remove log file
|
||||
fs::remove_file(self.inner.logfile.as_os_str()).ok();
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
The regex `SD_RE` matches the "Store Directory" line in the server log:
|
||||
```
|
||||
.+\sStore Directory:\s+"([^"]+)"
|
||||
```
|
||||
|
||||
### Client URL Discovery
|
||||
|
||||
The `client_addr` method polls the log file to find the server's listen address:
|
||||
|
||||
```rust
|
||||
fn client_addr(&self) -> String {
|
||||
for _ in 0..100 { // 100 iterations × 500ms = 50s max
|
||||
match fs::read_to_string(self.inner.logfile.as_os_str()) {
|
||||
Ok(l) => {
|
||||
if let Some(cre) = CLIENT_RE.captures(&l) {
|
||||
return cre.get(1).unwrap().as_str()
|
||||
.replace("0.0.0.0", "localhost");
|
||||
} else {
|
||||
thread::sleep(Duration::from_millis(500));
|
||||
}
|
||||
}
|
||||
_ => thread::sleep(Duration::from_millis(500)),
|
||||
}
|
||||
}
|
||||
panic!("no client addr info");
|
||||
}
|
||||
```
|
||||
|
||||
The regex `CLIENT_RE` matches:
|
||||
```
|
||||
.+\sclient connections on\s+(\S+)
|
||||
```
|
||||
|
||||
After finding the address, `client_url()` connects to it and parses the `INFO` JSON to get the port and TLS requirements.
|
||||
|
||||
## Cluster Setup
|
||||
|
||||
The `run_cluster_node_with_port` function spawns a single cluster node:
|
||||
|
||||
```rust
|
||||
fn run_cluster_node_with_port(
|
||||
cfg: &str,
|
||||
port: Option<&str>,
|
||||
routes: Vec<usize>,
|
||||
name: String,
|
||||
cluster_name: String,
|
||||
cluster: usize,
|
||||
) -> Server
|
||||
```
|
||||
|
||||
Additional flags for cluster nodes:
|
||||
- `--routes nats://127.0.0.1:<port1>,nats://127.0.0.1:<port2>` — routes to other cluster members
|
||||
- `--cluster nats://127.0.0.1:<cluster_port>` — cluster listen address
|
||||
- `--cluster_name <name>` — cluster name for grouping
|
||||
- `-n <name>` — server name
|
||||
|
||||
Port allocation for a cluster:
|
||||
```
|
||||
Base port: random in 3000..50000
|
||||
Node 1: client_port=base, cluster_port=base+1
|
||||
Node 2: client_port=base+100, cluster_port=base+101
|
||||
Node 3: client_port=base+200, cluster_port=base+201
|
||||
```
|
||||
|
||||
Each port is checked for availability with `is_port_available()`, including the +1 cluster port.
|
||||
|
||||
## JetStream Config
|
||||
|
||||
**Location**: `configs/jetstream.conf`
|
||||
|
||||
```conf
|
||||
jetstream: {
|
||||
strict: true,
|
||||
max_mem_store: 8MiB,
|
||||
max_file_store: 10GiB
|
||||
}
|
||||
```
|
||||
|
||||
This is the default test config for JetStream-enabled servers. It enables strict mode and sets memory/file storage limits suitable for testing.
|
||||
|
||||
## Test Usage Patterns
|
||||
|
||||
```rust
|
||||
#[tokio::test]
|
||||
async fn basic_test() {
|
||||
let server = nats_server::run_server("configs/jetstream.conf");
|
||||
let client = async_nats::connect(server.client_url()).await.unwrap();
|
||||
// ... test logic ...
|
||||
// Server cleaned up on drop
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn cluster_test() {
|
||||
let cluster = nats_server::run_cluster("configs/jetstream.conf");
|
||||
let client = async_nats::connect(cluster.client_url()).await.unwrap();
|
||||
// ... test logic ...
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn reconnect_test() {
|
||||
let mut server = nats_server::run_server("");
|
||||
let client = async_nats::connect(server.client_url()).await.unwrap();
|
||||
|
||||
// Restart the server to test reconnection
|
||||
server.restart();
|
||||
|
||||
// Client should reconnect automatically
|
||||
client.publish("test", "data".into()).await.unwrap();
|
||||
}
|
||||
```
|
||||
|
||||
## Dependencies
|
||||
|
||||
| Dependency | Version | Purpose |
|
||||
|-----------|---------|---------|
|
||||
| `lazy_static` | 1.4.0 | Static regex initialization |
|
||||
| `regex` | 1.7.1 | Log parsing (store directory, client address) |
|
||||
| `url` | 2 | URL manipulation for client_url_with |
|
||||
| `serde_json` | 1.0.104 | INFO JSON parsing |
|
||||
| `nuid` | 0.5 | Unique server ID generation |
|
||||
| `rand` | 0.10.1 | Random port selection |
|
||||
| `tokio-retry` | 0.3.0 | Exponential backoff for cluster operations |
|
||||
|
||||
Note: `async-nats` is only a dev-dependency, used in the crate's own integration tests.
|
||||
@@ -0,0 +1,307 @@
|
||||
# Service API and Higher-Level Abstractions
|
||||
|
||||
This document covers the Service API and other higher-level abstractions built on top of the core NATS client.
|
||||
|
||||
## Service API
|
||||
|
||||
**Location**: `service/` (feature: `service`)
|
||||
|
||||
The Service API provides a framework for building NATS-based microservices with built-in monitoring, health checks, and statistics.
|
||||
|
||||
### Service
|
||||
|
||||
```rust
|
||||
#[derive(Debug)]
|
||||
pub struct Service {
|
||||
client: Client,
|
||||
info: Info,
|
||||
endpoints: HashMap<String, Endpoint>,
|
||||
started: DateTime,
|
||||
stats_handler: Arc<dyn Fn(&str, &Stats) -> serde_json::Value + Send + Sync>,
|
||||
stop_sender: mpsc::Sender<()>,
|
||||
stop_receiver: Option<mpsc::Receiver<()>>,
|
||||
}
|
||||
```
|
||||
|
||||
### Creating a Service
|
||||
|
||||
```rust
|
||||
use async_nats::service::ServiceExt;
|
||||
|
||||
let mut service = client
|
||||
.service_builder()
|
||||
.description("Product service")
|
||||
.stats_handler(|endpoint, stats| {
|
||||
serde_json::json!({
|
||||
"endpoint": endpoint,
|
||||
"requests": stats.num_requests,
|
||||
"errors": stats.num_errors,
|
||||
})
|
||||
})
|
||||
.start("products", "1.0.0")
|
||||
.await?;
|
||||
```
|
||||
|
||||
### ServiceBuilder
|
||||
|
||||
```rust
|
||||
impl ServiceBuilder {
|
||||
pub fn description(mut self, description: impl Into<String>) -> Self
|
||||
pub fn stats_handler<F>(mut self, handler: F) -> Self
|
||||
pub async fn start(self, name: impl Into<String>, version: impl Into<String>) -> Result<Service, ServiceError>
|
||||
}
|
||||
```
|
||||
|
||||
### Endpoints
|
||||
|
||||
A service exposes one or more endpoints, each handling requests on a specific subject:
|
||||
|
||||
```rust
|
||||
// Add an endpoint
|
||||
let mut endpoint = service
|
||||
.endpoint("get_product")
|
||||
.await?;
|
||||
|
||||
// Process requests
|
||||
while let Some(request) = endpoint.next().await {
|
||||
let request = request?;
|
||||
// Handle the request
|
||||
request.respond(serde_json::json!({ "id": 1, "name": "Widget" })).await?;
|
||||
}
|
||||
```
|
||||
|
||||
### Endpoint
|
||||
|
||||
**Location**: `service/endpoint.rs`
|
||||
|
||||
```rust
|
||||
pub struct Endpoint {
|
||||
subject: Subject,
|
||||
queue_group: Option<String>,
|
||||
info: EndpointInfo,
|
||||
stats: Stats,
|
||||
subscriber: Subscriber,
|
||||
}
|
||||
```
|
||||
|
||||
Implements `futures::Stream` yielding `ServiceRequest` objects.
|
||||
|
||||
### ServiceRequest
|
||||
|
||||
```rust
|
||||
pub struct ServiceRequest {
|
||||
pub subject: Subject,
|
||||
pub payload: Bytes,
|
||||
pub headers: Option<HeaderMap>,
|
||||
pub reply: Option<Subject>,
|
||||
pub client: Client,
|
||||
}
|
||||
```
|
||||
|
||||
Methods:
|
||||
- `respond(payload)` — send a response to the requester
|
||||
- `respond_with_headers(payload, headers)` — send a response with headers
|
||||
|
||||
### Monitoring Subjects
|
||||
|
||||
The Service API automatically creates monitoring endpoints:
|
||||
|
||||
| Subject | Description |
|
||||
|---------|-------------|
|
||||
| `$SRV.PING` | Ping all services (returns service info) |
|
||||
| `$SRV.PING.<name>` | Ping specific service by name |
|
||||
| `$SRV.PING.<name>.<id>` | Ping specific service instance |
|
||||
| `$SRV.INFO` | Get service info |
|
||||
| `$SRV.STATS` | Get service statistics |
|
||||
|
||||
### Service Info
|
||||
|
||||
```rust
|
||||
pub struct Info {
|
||||
pub name: String,
|
||||
pub id: String,
|
||||
pub version: String,
|
||||
pub description: String,
|
||||
pub endpoints: Vec<EndpointInfo>,
|
||||
}
|
||||
```
|
||||
|
||||
### Stats
|
||||
|
||||
```rust
|
||||
pub struct Stats {
|
||||
pub num_requests: u64,
|
||||
pub num_errors: u64,
|
||||
pub last_error: Option<String>,
|
||||
pub processing_time: Duration,
|
||||
pub average_processing_time: Duration,
|
||||
}
|
||||
```
|
||||
|
||||
## ID Generation
|
||||
|
||||
**Location**: `id_generator.rs`
|
||||
|
||||
The client needs unique IDs for inbox subjects and other purposes.
|
||||
|
||||
### With `nuid` Feature (Default)
|
||||
|
||||
Uses the NUID library for high-performance, cryptographically strong, collision-resistant IDs:
|
||||
|
||||
```rust
|
||||
pub(crate) fn next() -> String {
|
||||
nuid::next().to_string()
|
||||
}
|
||||
```
|
||||
|
||||
NUID generates 22-character alphanumeric strings using a combination of a random prefix and a sequential counter.
|
||||
|
||||
### Without `nuid` Feature
|
||||
|
||||
Falls back to `rand`-based generation:
|
||||
|
||||
```rust
|
||||
pub(crate) fn next() -> String {
|
||||
rng()
|
||||
.sample_iter(Alphanumeric)
|
||||
.take(22)
|
||||
.map(char::from)
|
||||
.collect()
|
||||
}
|
||||
```
|
||||
|
||||
Both approaches produce 22-character alphanumeric strings, but NUID is more performant and has better collision resistance.
|
||||
|
||||
## Inbox Generation
|
||||
|
||||
The `Client::new_inbox()` method generates globally unique inbox subjects for request-reply:
|
||||
|
||||
```rust
|
||||
pub fn new_inbox(&self) -> String {
|
||||
format!("{}.{}", self.inbox_prefix, crate::id_generator::next())
|
||||
}
|
||||
```
|
||||
|
||||
Default prefix is `_INBOX`, producing subjects like `_INBOX.UaBG3f3q5NxX3KdNcRmF2f`.
|
||||
|
||||
Custom prefix via `ConnectOptions::custom_inbox_prefix()`:
|
||||
```rust
|
||||
let client = ConnectOptions::new()
|
||||
.custom_inbox_prefix("MYAPP")
|
||||
.connect("demo.nats.io")
|
||||
.await?;
|
||||
// Inbox subjects: MYAPP.UaBG3f3q5KdNcRmF2f
|
||||
```
|
||||
|
||||
## DateTime Helpers
|
||||
|
||||
**Location**: `datetime.rs` (feature: `jetstream` or `service` or `chrono`)
|
||||
|
||||
Provides date/time types for JetStream and Service API timestamps:
|
||||
|
||||
- Uses the `time` crate by default
|
||||
- Optionally uses `chrono` via the `chrono` feature flag
|
||||
- Supports RFC 3339 formatting and parsing
|
||||
- `DateTime` type wraps either `time::OffsetDateTime` or `chrono::DateTime<Utc>`
|
||||
|
||||
## Crypto Module
|
||||
|
||||
**Location**: `crypto.rs` (feature: `crypto`)
|
||||
|
||||
Provides encryption/decryption support used by the Object Store for server-side encryption.
|
||||
|
||||
## Subject Validation
|
||||
|
||||
**Location**: `lib.rs`
|
||||
|
||||
The client provides two levels of subject validation:
|
||||
|
||||
### is_valid_publish_subject
|
||||
|
||||
```rust
|
||||
pub(crate) fn is_valid_publish_subject<T: AsRef<str>>(subject: T) -> bool
|
||||
```
|
||||
|
||||
Checks for protocol safety only:
|
||||
- Not empty
|
||||
- No whitespace (space, tab, CR, LF) which would break protocol framing
|
||||
|
||||
Used for publish operations. Can be disabled with `skip_subject_validation`.
|
||||
|
||||
### is_valid_subject
|
||||
|
||||
```rust
|
||||
pub(crate) fn is_valid_subject<T: AsRef<str>>(subject: T) -> bool
|
||||
```
|
||||
|
||||
Checks structural validity:
|
||||
- Not empty
|
||||
- No leading/trailing dots
|
||||
- No consecutive dots (`..`)
|
||||
- No whitespace
|
||||
|
||||
Used for subscribe operations (always runs, matching Go/Java behavior).
|
||||
|
||||
### is_valid_queue_group
|
||||
|
||||
```rust
|
||||
pub(crate) fn is_valid_queue_group(queue_group: &str) -> bool
|
||||
```
|
||||
|
||||
Checks:
|
||||
- Not empty
|
||||
- No whitespace
|
||||
|
||||
## JetStream Name Validation
|
||||
|
||||
**Location**: `jetstream/mod.rs`
|
||||
|
||||
```rust
|
||||
pub(crate) fn is_valid_name(name: &str) -> bool {
|
||||
!name.is_empty()
|
||||
&& name.bytes().all(|c| !c.is_ascii_whitespace() && c != b'.' && c != b'*' && c != b'>')
|
||||
}
|
||||
```
|
||||
|
||||
JetStream names (stream names, consumer names) must not contain:
|
||||
- Whitespace
|
||||
- Dots (`.`) — would conflict with subject delimiters
|
||||
- Wildcards (`*`, `>`) — would conflict with subject wildcards
|
||||
|
||||
## CallbackArg1
|
||||
|
||||
**Location**: `options.rs`
|
||||
|
||||
A type-erased async callback wrapper used throughout the crate:
|
||||
|
||||
```rust
|
||||
pub(crate) type AsyncCallbackArg1<A, T> =
|
||||
Arc<dyn Fn(A) -> Pin<Box<dyn Future<Output = T> + Send + Sync + 'static>> + Send + Sync>;
|
||||
|
||||
#[derive(Clone)]
|
||||
pub(crate) struct CallbackArg1<A, T>(AsyncCallbackArg1<A, T>);
|
||||
|
||||
impl<A, T> CallbackArg1<A, T> {
|
||||
pub(crate) async fn call(&self, arg: A) -> T {
|
||||
(self.0.as_ref())(arg).await
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Used for:
|
||||
- `event_callback` — `CallbackArg1<Event, ()>`
|
||||
- `auth_callback` — `CallbackArg1<Vec<u8>, Result<Auth, AuthError>>`
|
||||
- `reconnect_to_server_callback` — `CallbackArg1<(Vec<Server>, ServerInfo), Option<ReconnectToServer>>`
|
||||
- `signature_callback` — `CallbackArg1<String, Result<String, AuthError>>`
|
||||
|
||||
## Version Compatibility Checking
|
||||
|
||||
The `Client::is_server_compatible` method checks if the server version meets a minimum requirement:
|
||||
|
||||
```rust
|
||||
pub fn is_server_compatible(&self, major: i64, minor: i64, patch: i64) -> bool
|
||||
```
|
||||
|
||||
This parses the server version string from `ServerInfo::version` using a regex and compares major/minor/patch components. Note: this checks the directly-connected server, not necessarily the JetStream leader.
|
||||
|
||||
The `server_2_10`, `server_2_11`, `server_2_12`, and `server_2_14` feature flags enable version-specific API fields and methods without runtime checks.
|
||||
@@ -0,0 +1,215 @@
|
||||
# Quick Reference
|
||||
|
||||
## Crate Summary
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| **Crate** | `async-nats` |
|
||||
| **Version** | 0.49.1 |
|
||||
| **Edition** | 2021 |
|
||||
| **MSRV** | 1.88.0 |
|
||||
| **License** | Apache-2.0 |
|
||||
| **Runtime** | Tokio |
|
||||
| **Protocol** | NATS Client Protocol v1 (Dynamic) |
|
||||
| **TLS** | rustls (ring / aws-lc-rs / fips) |
|
||||
| **WebSocket** | tokio-websockets (feature-gated) |
|
||||
|
||||
## Quick Start
|
||||
|
||||
```rust
|
||||
use async_nats::connect;
|
||||
use futures_util::StreamExt;
|
||||
|
||||
#[tokio::main]
|
||||
async fn main() -> Result<(), async_nats::Error> {
|
||||
let client = connect("demo.nats.io").await?;
|
||||
|
||||
// Publish
|
||||
client.publish("events.data", "hello".into()).await?;
|
||||
|
||||
// Subscribe
|
||||
let mut sub = client.subscribe("events.>").await?;
|
||||
while let Some(msg) = sub.next().await {
|
||||
println!("{:?}", msg);
|
||||
}
|
||||
|
||||
// Request-Response
|
||||
let response = client.request("service", "input".into()).await?;
|
||||
|
||||
Ok(())
|
||||
}
|
||||
```
|
||||
|
||||
## Architecture at a Glance
|
||||
|
||||
```
|
||||
Client (cloneable handle, mpsc::Sender<Command>)
|
||||
│
|
||||
▼
|
||||
ConnectionHandler (single Tokio task)
|
||||
├── Subscriptions HashMap<u64, Subscription>
|
||||
├── Multiplexer (request-reply, SID 0)
|
||||
├── Flush Observers
|
||||
└── Ping/Pong health check
|
||||
│
|
||||
▼
|
||||
Connection (protocol I/O, read/write buffers)
|
||||
│
|
||||
▼
|
||||
Connector (server pool, reconnection)
|
||||
│
|
||||
▼
|
||||
NATS Server (Go binary, TCP/TLS/WebSocket)
|
||||
```
|
||||
|
||||
## Key Types
|
||||
|
||||
| Type | Location | Purpose |
|
||||
|------|----------|---------|
|
||||
| `Client` | `client.rs` | Cloneable connection handle |
|
||||
| `Subscriber` | `lib.rs` | Message stream (impl `futures::Stream`) |
|
||||
| `Message` | `message.rs` | Inbound NATS message |
|
||||
| `OutboundMessage` | `message.rs` | Outbound publish message |
|
||||
| `Subject` | `subject.rs` | Validated subject string (backed by `Bytes`) |
|
||||
| `HeaderMap` | `header.rs` | NATS message headers |
|
||||
| `StatusCode` | `status.rs` | NATS protocol status codes |
|
||||
| `ServerInfo` | `lib.rs` | Server INFO data |
|
||||
| `ConnectInfo` | `lib.rs` | Client CONNECT data |
|
||||
| `ServerAddr` | `lib.rs` | Validated server URL |
|
||||
| `Auth` | `auth.rs` | Authentication credentials |
|
||||
| `ConnectOptions` | `options.rs` | Connection configuration builder |
|
||||
| `Event` | `lib.rs` | Connection lifecycle events |
|
||||
| `State` | `connection.rs` | Connection state (Pending/Connected/Disconnected) |
|
||||
| `Statistics` | `client.rs` | Atomic connection metrics |
|
||||
| `Request` | `client.rs` | Request-response builder |
|
||||
|
||||
## JetStream Types
|
||||
|
||||
| Type | Location | Purpose |
|
||||
|------|----------|---------|
|
||||
| `jetstream::Context` | `jetstream/context.rs` | JetStream API entry point |
|
||||
| `jetstream::stream::Stream` | `jetstream/stream.rs` | Stream management |
|
||||
| `jetstream::stream::Config` | `jetstream/stream.rs` | Stream configuration |
|
||||
| `jetstream::stream::Info` | `jetstream/stream.rs` | Stream info/state |
|
||||
| `jetstream::consumer::PullConsumer` | `jetstream/consumer/pull.rs` | Pull-based consumer |
|
||||
| `jetstream::consumer::PushConsumer` | `jetstream/consumer/push.rs` | Push-based consumer |
|
||||
| `jetstream::consumer::Config` | `jetstream/consumer/mod.rs` | Consumer configuration |
|
||||
| `jetstream::Message` | `jetstream/message.rs` | Message with ack methods |
|
||||
| `jetstream::PublishAck` | `jetstream/publish.rs` | Publish acknowledgment |
|
||||
| `jetstream::kv::Store` | `jetstream/kv/bucket.rs` | Key-Value store |
|
||||
| `jetstream::object_store::ObjectStore` | `jetstream/object_store/mod.rs` | Object store |
|
||||
| `jetstream::ErrorCode` | `jetstream/errors.rs` | JetStream error codes |
|
||||
|
||||
## Protocol Operations
|
||||
|
||||
### Client → Server (ClientOp)
|
||||
|
||||
| Op | Wire Format | Purpose |
|
||||
|----|-----------|---------|
|
||||
| `CONNECT` | `CONNECT {json}\r\n` | Authentication and capabilities |
|
||||
| `PUB` | `PUB <subject> [reply] <len>\r\n<payload>\r\n` | Publish message |
|
||||
| `HPUB` | `HPUB <subject> [reply] <hlen> <tlen>\r\n<hdrs><payload>\r\n` | Publish with headers |
|
||||
| `SUB` | `SUB <subject> [queue] <sid>\r\n` | Subscribe |
|
||||
| `UNSUB` | `UNSUB <sid> [max]\r\n` | Unsubscribe |
|
||||
| `PING` | `PING\r\n` | Keepalive / health check |
|
||||
| `PONG` | `PONG\r\n` | Response to server PING |
|
||||
|
||||
### Server → Client (ServerOp)
|
||||
|
||||
| Op | Wire Format | Purpose |
|
||||
|----|-----------|---------|
|
||||
| `INFO` | `INFO {json}\r\n` | Server capabilities, cluster info |
|
||||
| `MSG` | `MSG <subj> <sid> [reply] <len>\r\n<payload>\r\n` | Deliver message |
|
||||
| `HMSG` | `HMSG <subj> <sid> [reply] <hlen> <tlen>\r\n<hdrs><payload>\r\n` | Message with headers |
|
||||
| `+OK` | `+OK\r\n` | Success (verbose mode) |
|
||||
| `-ERR` | `-ERR <desc>\r\n` | Server error |
|
||||
| `PING` | `PING\r\n` | Server health check |
|
||||
| `PONG` | `PONG\r\n` | Ack client PING |
|
||||
|
||||
## Internal Commands (Command → ConnectionHandler)
|
||||
|
||||
| Command | Purpose |
|
||||
|---------|---------|
|
||||
| `Publish(OutboundMessage)` | Queue message for sending |
|
||||
| `Request { subject, payload, respond, headers, sender }` | Request-response via multiplexer |
|
||||
| `Subscribe { sid, subject, queue_group, sender }` | Create subscription |
|
||||
| `Unsubscribe { sid, max }` | Remove subscription |
|
||||
| `Flush { observer }` | Wait for write buffer flush |
|
||||
| `Drain { sid }` | Gracefully drain (sub or whole client) |
|
||||
| `Reconnect` | Force reconnection |
|
||||
| `SetServerPool { servers, result }` | Replace server pool |
|
||||
| `ServerPool { result }` | Query server pool |
|
||||
|
||||
## Feature Flags
|
||||
|
||||
| Feature | Default | Enables |
|
||||
|---------|---------|---------|
|
||||
| `jetstream` | ✓ | JetStream API (streams, consumers, publish) |
|
||||
| `kv` | ✓ | Key-Value store (requires jetstream) |
|
||||
| `object-store` | ✓ | Object store (requires jetstream + crypto) |
|
||||
| `service` | ✓ | Service API |
|
||||
| `nkeys` | ✓ | NKey/JWT authentication |
|
||||
| `crypto` | ✓ | Encryption support |
|
||||
| `websockets` | ✓ | WebSocket transport |
|
||||
| `nuid` | ✓ | NUID ID generation |
|
||||
| `ring` | ✓ | Ring crypto backend |
|
||||
| `aws-lc-rs` | ✗ | Alternative crypto backend |
|
||||
| `fips` | ✗ | FIPS mode (requires aws-lc-rs) |
|
||||
| `chrono` | ✗ | Use chrono instead of time |
|
||||
| `experimental` | ✗ | Experimental features |
|
||||
| `server_2_10` | ✓ | Server 2.10+ API fields |
|
||||
| `server_2_11` | ✓ | Server 2.11+ API fields |
|
||||
| `server_2_12` | ✓ | Server 2.12+ API fields |
|
||||
| `server_2_14` | ✓ | Server 2.14+ API fields |
|
||||
|
||||
## Connection Defaults
|
||||
|
||||
| Parameter | Default |
|
||||
|-----------|---------|
|
||||
| Connection timeout | 5 seconds |
|
||||
| Ping interval | 60 seconds |
|
||||
| Max pending pings | 2 |
|
||||
| Request timeout | 10 seconds |
|
||||
| Command channel capacity | 2048 |
|
||||
| Subscription capacity | 65536 |
|
||||
| Read buffer capacity | 65535 |
|
||||
| Inbox prefix | `_INBOX` |
|
||||
| Reconnect delay | Exponential (0ms → 4s cap) |
|
||||
| Max reconnects | Unlimited |
|
||||
| TLS required | Auto (server-dependent) |
|
||||
|
||||
## Error Hierarchy
|
||||
|
||||
```
|
||||
ConnectError (ConnectErrorKind::ServerParse | Dns | Authentication | AuthorizationViolation | TimedOut | Tls | Io | MaxReconnects)
|
||||
PublishError (PublishErrorKind::MaxPayloadExceeded | InvalidSubject | Send)
|
||||
RequestError (RequestErrorKind::TimedOut | NoResponders | InvalidSubject | MaxPayloadExceeded | Other)
|
||||
SubscribeError (SubscribeErrorKind::InvalidSubject | InvalidQueueName | Other)
|
||||
FlushError (FlushErrorKind::SendError | FlushError)
|
||||
```
|
||||
|
||||
## nats-server Test Harness
|
||||
|
||||
| Function | Description |
|
||||
|----------|-------------|
|
||||
| `run_server(cfg)` | Start single server with config |
|
||||
| `run_basic_server()` | Start bare server |
|
||||
| `run_cluster(cfg)` | Start 3-node cluster |
|
||||
| `set_lame_duck_mode(s)` | Send LDM signal |
|
||||
|
||||
## JetStream API Subjects
|
||||
|
||||
| Operation | Subject Pattern |
|
||||
|-----------|---------------|
|
||||
| Create stream | `$JS.API.STREAM.CREATE.<name>` |
|
||||
| Stream info | `$JS.API.STREAM.INFO.<name>` |
|
||||
| Update stream | `$JS.API.STREAM.UPDATE.<name>` |
|
||||
| Delete stream | `$JS.API.STREAM.DELETE.<name>` |
|
||||
| Purge stream | `$JS.API.STREAM.PURGE.<name>` |
|
||||
| List streams | `$JS.API.STREAM.LIST` |
|
||||
| Create consumer | `$JS.API.CONSUMER.CREATE.<stream>` |
|
||||
| Create durable | `$JS.API.CONSUMER.DURABLE.CREATE.<stream>.<name>` |
|
||||
| Consumer info | `$JS.API.CONSUMER.INFO.<stream>.<name>` |
|
||||
| Pull next | `$JS.API.CONSUMER.MSG.NEXT.<stream>.<name>` |
|
||||
| Account info | `$JS.API.ACCOUNT.INFO` |
|
||||
| Direct get | `$JS.API.DIRECT.GET.<name>` |
|
||||
Reference in New Issue
Block a user