docs(research): add nats-async and nats-server deep-dive references
This commit is contained in:
@@ -0,0 +1,170 @@
|
||||
# async-nats: Overview & Architecture
|
||||
|
||||
**Crate**: `async-nats`
|
||||
**Version**: 0.49.1
|
||||
**Repository**: https://github.com/nats-io/nats.rs
|
||||
**License**: Apache-2.0
|
||||
**Rust Edition**: 2021
|
||||
**MSRV**: 1.88.0
|
||||
**Async Runtime**: Tokio
|
||||
|
||||
## What is async-nats?
|
||||
|
||||
`async-nats` is the official async Rust client for the [NATS messaging system](https://nats.io). It provides a Tokio-based asynchronous interface to NATS server features including:
|
||||
|
||||
- **Core NATS** — publish/subscribe, request/reply, queue groups
|
||||
- **JetStream** — persistent stream-based messaging with at-least-once and exactly-once semantics
|
||||
- **Key-Value Store** — KV abstraction built on JetStream streams
|
||||
- **Object Store** — large-object storage built on JetStream streams
|
||||
- **Service API** — microservice request/reply pattern with built-in PING/INFO/STATS verbs
|
||||
|
||||
The crate is positioned as the **core client** in the NATS Rust ecosystem. A separate project, [Orbit](https://github.com/synadia-io/orbit.rs), provides higher-level opinionated abstractions on top.
|
||||
|
||||
```
|
||||
┌──────────────────────────────────────────────────────┐
|
||||
│ Application code │
|
||||
└──────────────┬───────────────────────────┬───────────┘
|
||||
│ │
|
||||
▼ ▼
|
||||
┌───────────────────┐ ┌───────────────────┐
|
||||
│ Orbit crates │ uses │ async-nats (core) │
|
||||
│ (opinionated, │──────▶│ (parity, stable, │
|
||||
│ per-crate semver) │ │ protocol-level) │
|
||||
└───────────────────┘ └─────────┬─────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────┐
|
||||
│ nats-server │
|
||||
└─────────────┘
|
||||
```
|
||||
|
||||
## Feature Flags
|
||||
|
||||
Features are extensive and control which subsystems are compiled:
|
||||
|
||||
| Feature | Default | Description |
|
||||
|---------|---------|-------------|
|
||||
| `jetstream` | ✅ | JetStream API (streams, consumers, publish) |
|
||||
| `kv` | ✅ | Key-Value store (depends on `jetstream`) |
|
||||
| `object-store` | ✅ | Object store (depends on `jetstream` + `crypto`) |
|
||||
| `service` | ✅ | Service API (microservice pattern) |
|
||||
| `nkeys` | ✅ | NKey/JWT authentication |
|
||||
| `nuid` | ✅ | NUID-based unique ID generation |
|
||||
| `crypto` | ✅ | Cryptographic primitives (SHA-256 for object store) |
|
||||
| `websockets` | ✅ | WebSocket transport (`ws://`/`wss://`) |
|
||||
| `ring` | ✅ | Use `ring` as TLS crypto backend |
|
||||
| `aws-lc-rs` | ❌ | Use `aws-lc-rs` as TLS crypto backend |
|
||||
| `fips` | ❌ | FIPS 140-2 compliant via `aws-lc-rs` |
|
||||
| `chrono` | ❌ | Use `chrono` instead of `time` for datetime types |
|
||||
| `server_2_10` | ✅ | Server 2.10+ features |
|
||||
| `server_2_11` | ✅ | Server 2.11+ features |
|
||||
| `server_2_12` | ✅ | Server 2.12+ features |
|
||||
| `server_2_14` | ✅ | Server 2.14+ features |
|
||||
| `experimental` | ❌ | Experimental features |
|
||||
|
||||
## Source Structure
|
||||
|
||||
```
|
||||
async-nats/src/
|
||||
├── lib.rs # Entry point: connect(), ServerInfo, Command, ClientOp, ServerOp,
|
||||
│ ConnectionHandler, Subscriber, Event, ServerAddr, ConnectInfo
|
||||
├── client.rs # Client struct, publish/subscribe/request/drain/flush APIs,
|
||||
│ Request builder, Statistics, trait definitions
|
||||
├── connection.rs # Framed connection: NATS protocol parser/serializer,
|
||||
│ read/write buffer management, WebSocket adapter
|
||||
├── connector.rs # Server pool, reconnection logic, TLS setup, DNS resolution,
|
||||
│ authentication handshake
|
||||
├── options.rs # ConnectOptions builder, auth methods, TLS config, callbacks
|
||||
├── auth.rs # Auth struct (username, password, token, JWT, nkey, signature)
|
||||
├── auth_utils.rs # Credentials file parsing (JWT + NKey seed)
|
||||
├── message.rs # Message (inbound), OutboundMessage (outbound)
|
||||
├── header.rs # HeaderMap, HeaderName, HeaderValue (NATS headers)
|
||||
├── subject.rs # Subject type, ToSubject trait, SubjectError
|
||||
├── status.rs # StatusCode enum (NATS status codes)
|
||||
├── error.rs # Generic Error<K> type used throughout
|
||||
├── datetime.rs # DateTime type (time or chrono backend)
|
||||
├── id_generator.rs # Unique ID generation (NUID or rand fallback)
|
||||
├── tls.rs # TLS configuration helper
|
||||
├── crypto.rs # SHA-256 for object store integrity
|
||||
├── jetstream/
|
||||
│ ├── mod.rs # Module entry: new(), with_domain(), with_prefix()
|
||||
│ ├── context.rs # Context: JetStream API (streams, consumers, KV, OS, publish)
|
||||
│ ├── stream.rs # Stream handle, Config, Info, purge/delete/message ops
|
||||
│ ├── consumer/
|
||||
│ │ ├── mod.rs # Consumer trait, Info, Config base
|
||||
│ │ ├── pull.rs # PullConsumer: batch fetch, sequence, messages stream
|
||||
│ │ └── push.rs # PushConsumer: Ordered push consumer with auto-recreate
|
||||
│ ├── publish.rs # PublishAck, PublishAckFuture, PublishMessage builder
|
||||
│ ├── message.rs # JetStream Message (with ack methods), AckKind
|
||||
│ ├── response.rs # Response<T> (Ok/Err) for JetStream API calls
|
||||
│ ├── errors.rs # ErrorCode, Error for JetStream
|
||||
│ ├── account.rs # Account info
|
||||
│ ├── kv/
|
||||
│ │ ├── mod.rs # Store: put/get/delete/purge/watch/history/keys
|
||||
│ │ └── bucket.rs # Bucket Status
|
||||
│ └── object_store/
|
||||
│ └── mod.rs # ObjectStore: put/get/delete/watch/list/seal, Object (AsyncRead)
|
||||
└── service/
|
||||
├── mod.rs # Service, ServiceBuilder, Group, EndpointBuilder, Request
|
||||
└── endpoint.rs # Endpoint stream, Stats, Info
|
||||
```
|
||||
|
||||
## Architecture: Core Connection Model
|
||||
|
||||
The client uses a **single-connection, actor-model** design:
|
||||
|
||||
```
|
||||
┌──────────────────────────────────────┐
|
||||
Client (clone) ──▶│ mpsc::Sender<Command> │
|
||||
(many handles) │ (bounded channel) │
|
||||
└────────────┬────────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌────────────────────────────────────────┐
|
||||
│ ConnectionHandler (tokio::task) │
|
||||
│ - Receives Command from channel │
|
||||
│ - Converts to ClientOp │
|
||||
│ - Manages subscriptions map │
|
||||
│ - Manages multiplexer (request/reply)│
|
||||
│ - Pings server on interval │
|
||||
│ - Handles reconnection │
|
||||
└────────────┬──────────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌────────────────────────────────────────┐
|
||||
│ Connection (framed TCP/TLS/WS) │
|
||||
│ - Protocol parser (try_read_op) │
|
||||
│ - Write buffer (VecDeque<Bytes>) │
|
||||
│ - Vectored I/O support │
|
||||
│ - Read buffer (BytesMut) │
|
||||
└────────────┬──────────────────────────┘
|
||||
│
|
||||
▼
|
||||
nats-server
|
||||
```
|
||||
|
||||
### Key Design Decisions
|
||||
|
||||
1. **Cloneable Client**: `Client` is `Clone` (via `mpsc::Sender` clone), enabling shared use across tasks
|
||||
2. **Single TCP connection**: All traffic (Core NATS, JetStream API, etc.) multiplexes over one connection
|
||||
3. **Background task**: `ConnectionHandler` runs as a spawned Tokio task, bridging the mpsc channel to the TCP stream
|
||||
4. **Automatic reconnection**: On disconnect, `Connector` retries servers from the pool with exponential backoff
|
||||
5. **Subscription rehydration**: On reconnect, all active subscriptions are re-subscribed with adjusted `max` counts
|
||||
6. **Multiplexer for request/reply**: A single wildcard subscription (`_INBOX.<id>.*`) multiplexes all pending request/reply correlations
|
||||
|
||||
## Dependencies (Key)
|
||||
|
||||
| Crate | Purpose |
|
||||
|-------|---------|
|
||||
| `tokio` | Async runtime, TCP, time, sync, io-util |
|
||||
| `bytes` | Efficient byte buffer (`Bytes`, `BytesMut`) |
|
||||
| `tokio-rustls` | TLS via rustls |
|
||||
| `rustls-native-certs` | Load system root certificates |
|
||||
| `serde` / `serde_json` | JSON serialization for JetStream API |
|
||||
| `futures-util` | Stream trait, Sink trait, StreamExt |
|
||||
| `tracing` | Structured logging |
|
||||
| `thiserror` | Error derive macros |
|
||||
| `memchr` | Fast substring search for protocol parsing |
|
||||
| `portable-atomic` | Atomic types with portable-atomic fallback |
|
||||
| `tokio-util` | `PollSender` for Sink implementation |
|
||||
| `tokio-stream` | `ReceiverStream` adapter |
|
||||
@@ -0,0 +1,404 @@
|
||||
# async-nats: Key Types & Traits
|
||||
|
||||
## Core Types
|
||||
|
||||
### `Client`
|
||||
|
||||
The primary handle to a NATS connection. Cheaply cloneable (wraps `mpsc::Sender<Command>`).
|
||||
|
||||
```rust
|
||||
#[derive(Clone, Debug)]
|
||||
pub struct Client {
|
||||
info: tokio::sync::watch::Receiver<Option<ServerInfo>>,
|
||||
state: tokio::sync::watch::Receiver<State>,
|
||||
sender: mpsc::Sender<Command>,
|
||||
poll_sender: PollSender<Command>,
|
||||
next_subscription_id: Arc<AtomicU64>,
|
||||
subscription_capacity: usize,
|
||||
inbox_prefix: Arc<str>,
|
||||
request_timeout: Option<Duration>,
|
||||
max_payload: Arc<AtomicUsize>,
|
||||
connection_stats: Arc<Statistics>,
|
||||
skip_subject_validation: bool,
|
||||
}
|
||||
```
|
||||
|
||||
**Key methods**:
|
||||
- `publish(subject, payload)` — fire-and-forget publish
|
||||
- `publish_with_headers(subject, headers, payload)` — publish with NATS headers
|
||||
- `publish_with_reply(subject, reply, payload)` — publish with reply-to subject
|
||||
- `subscribe(subject)` → `Subscriber` — subscribe to a subject
|
||||
- `queue_subscribe(subject, queue_group)` → `Subscriber` — queue group subscription
|
||||
- `request(subject, payload)` → `Message` — request/reply with default timeout
|
||||
- `send_request(subject, request)` → `Message` — request with custom `Request` builder
|
||||
- `flush()` — wait until all buffered writes are flushed to the server
|
||||
- `drain()` — drain all subscriptions, flush, then close
|
||||
- `force_reconnect()` — force a reconnection (e.g., to re-trigger auth)
|
||||
- `new_inbox()` — generate a unique inbox subject (`_INBOX.<id>`)
|
||||
- `server_info()` → `ServerInfo` — last known server info
|
||||
- `connection_state()` → `State` — `Pending`/`Connected`/`Disconnected`
|
||||
- `statistics()` → `Arc<Statistics>` — connection statistics (bytes, messages, connects)
|
||||
- `max_payload()` → `usize` — server's max payload size
|
||||
- `set_server_pool(addrs)` — replace the server pool for reconnection
|
||||
- `server_pool()` — snapshot of current server pool
|
||||
|
||||
### `Subscriber`
|
||||
|
||||
A `Stream` yielding `Message` values from a subscription.
|
||||
|
||||
```rust
|
||||
#[derive(Debug)]
|
||||
pub struct Subscriber {
|
||||
sid: u64,
|
||||
receiver: mpsc::Receiver<Message>,
|
||||
sender: mpsc::Sender<Command>,
|
||||
}
|
||||
```
|
||||
|
||||
Implements `futures_util::Stream<Item = Message>`. Methods:
|
||||
- `unsubscribe()` — immediately unsubscribe
|
||||
- `unsubscribe_after(n)` — unsubscribe after `n` total delivered messages
|
||||
- `drain()` — unsubscribe after in-flight messages are delivered
|
||||
|
||||
**Drop behavior**: When a `Subscriber` is dropped, it spawns a task to send `Command::Unsubscribe` to the connection handler, ensuring the server is always notified.
|
||||
|
||||
### `Message`
|
||||
|
||||
An inbound NATS message:
|
||||
|
||||
```rust
|
||||
#[derive(Clone, Debug, Serialize, Deserialize)]
|
||||
pub struct Message {
|
||||
pub subject: Subject,
|
||||
pub reply: Option<Subject>,
|
||||
pub payload: Bytes,
|
||||
pub headers: Option<HeaderMap>,
|
||||
pub status: Option<StatusCode>,
|
||||
pub description: Option<String>,
|
||||
pub length: usize,
|
||||
}
|
||||
```
|
||||
|
||||
### `OutboundMessage`
|
||||
|
||||
An outbound message for publishing (no status/description):
|
||||
|
||||
```rust
|
||||
#[derive(Clone, Debug)]
|
||||
pub struct OutboundMessage {
|
||||
pub subject: Subject,
|
||||
pub reply: Option<Subject>,
|
||||
pub payload: Bytes,
|
||||
pub headers: Option<HeaderMap>,
|
||||
}
|
||||
```
|
||||
|
||||
### `Request`
|
||||
|
||||
Builder for request/reply calls:
|
||||
|
||||
```rust
|
||||
#[derive(Default)]
|
||||
pub struct Request {
|
||||
pub payload: Option<Bytes>,
|
||||
pub headers: Option<HeaderMap>,
|
||||
pub timeout: Option<Option<Duration>>,
|
||||
pub inbox: Option<String>,
|
||||
}
|
||||
```
|
||||
|
||||
Builder methods: `payload()`, `headers()`, `timeout()`, `inbox()`. The `inbox` field, when set, bypasses the multiplexer and uses a dedicated subscription instead.
|
||||
|
||||
### `ServerInfo`
|
||||
|
||||
Server metadata received during connection handshake:
|
||||
|
||||
```rust
|
||||
#[derive(Debug, Deserialize, Default, Clone, Eq, PartialEq)]
|
||||
pub struct ServerInfo {
|
||||
pub server_id: String,
|
||||
pub server_name: String,
|
||||
pub host: String,
|
||||
pub port: u16,
|
||||
pub version: String,
|
||||
pub auth_required: bool,
|
||||
pub tls_required: bool,
|
||||
pub max_payload: usize,
|
||||
pub proto: i8,
|
||||
pub client_id: u64,
|
||||
pub go: String,
|
||||
pub nonce: String,
|
||||
pub connect_urls: Vec<String>,
|
||||
pub client_ip: String,
|
||||
pub headers: bool,
|
||||
pub lame_duck_mode: bool,
|
||||
pub cluster: Option<String>,
|
||||
pub domain: Option<String>,
|
||||
pub jetstream: bool,
|
||||
}
|
||||
```
|
||||
|
||||
### `ConnectInfo`
|
||||
|
||||
Client → server `CONNECT` message payload:
|
||||
|
||||
```rust
|
||||
#[derive(Clone, Debug, Serialize)]
|
||||
pub struct ConnectInfo {
|
||||
pub verbose: bool,
|
||||
pub pedantic: bool,
|
||||
pub user_jwt: Option<String>,
|
||||
pub nkey: Option<String>,
|
||||
pub signature: Option<String>,
|
||||
pub name: Option<String>,
|
||||
pub echo: bool,
|
||||
pub lang: String,
|
||||
pub version: String,
|
||||
pub protocol: Protocol, // Original(0) or Dynamic(1)
|
||||
pub tls_required: bool,
|
||||
pub user: Option<String>,
|
||||
pub pass: Option<String>,
|
||||
pub auth_token: Option<String>,
|
||||
pub headers: bool,
|
||||
pub no_responders: bool,
|
||||
}
|
||||
```
|
||||
|
||||
The client always sets: `verbose=false`, `pedantic=false`, `lang="rust"`, `protocol=Dynamic`, `headers=true`, `no_responders=true`.
|
||||
|
||||
### `Statistics`
|
||||
|
||||
Atomic connection statistics (shared via `Arc`):
|
||||
|
||||
```rust
|
||||
#[derive(Default, Debug)]
|
||||
pub struct Statistics {
|
||||
pub in_bytes: AtomicU64,
|
||||
pub out_bytes: AtomicU64,
|
||||
pub in_messages: AtomicU64,
|
||||
pub out_messages: AtomicU64,
|
||||
pub connects: AtomicU64,
|
||||
}
|
||||
```
|
||||
|
||||
## Subject Types
|
||||
|
||||
### `Subject`
|
||||
|
||||
A validated NATS subject string (newtype over `String`):
|
||||
|
||||
```rust
|
||||
// Usage:
|
||||
let subject: Subject = "foo.bar.baz".into();
|
||||
```
|
||||
|
||||
### `ToSubject` trait
|
||||
|
||||
Conversion trait for subjects:
|
||||
|
||||
```rust
|
||||
pub trait ToSubject {
|
||||
fn to_subject(self) -> Result<Subject, SubjectError>;
|
||||
}
|
||||
```
|
||||
|
||||
Implemented for `&str`, `String`, `Subject` directly.
|
||||
|
||||
### `SubjectError`
|
||||
|
||||
```rust
|
||||
pub enum SubjectError {
|
||||
InvalidFormat,
|
||||
}
|
||||
```
|
||||
|
||||
## Header Types
|
||||
|
||||
### `HeaderMap`
|
||||
|
||||
A multimap of header name → values:
|
||||
|
||||
```rust
|
||||
pub struct HeaderMap {
|
||||
inner: VecMap<HeaderName, Vec<HeaderValue>>,
|
||||
}
|
||||
```
|
||||
|
||||
Methods: `insert()`, `append()`, `get()`, `len()`, `is_empty()`, `iter()`, `to_bytes()`.
|
||||
|
||||
### `HeaderName`
|
||||
|
||||
Case-insensitive header name. Created via `FromStr`:
|
||||
|
||||
```rust
|
||||
let name: HeaderName = "Nats-Expected-Last-Subject-Sequence".parse()?;
|
||||
```
|
||||
|
||||
### `HeaderValue`
|
||||
|
||||
Header value string. Created via `FromStr` or `From<u64>`:
|
||||
|
||||
```rust
|
||||
let val: HeaderValue = "some value".parse()?;
|
||||
let val: HeaderValue = HeaderValue::from(42u64);
|
||||
```
|
||||
|
||||
## Server Address Types
|
||||
|
||||
### `ServerAddr`
|
||||
|
||||
Wraps a `url::Url` with NATS-specific validation. Supports schemes: `nats://`, `tls://`, `ws://`, `wss://`. Default port is `4222`.
|
||||
|
||||
```rust
|
||||
let addr: ServerAddr = "demo.nats.io".parse()?;
|
||||
let addr: ServerAddr = "nats://demo.nats.io:4222".parse()?;
|
||||
let addr: ServerAddr = "tls://demo.nats.io".parse()?;
|
||||
```
|
||||
|
||||
### `ToServerAddrs` trait
|
||||
|
||||
Flexible server address input (single URL, `Vec`, slice, etc.):
|
||||
|
||||
```rust
|
||||
pub trait ToServerAddrs {
|
||||
type Iter: Iterator<Item = ServerAddr>;
|
||||
fn to_server_addrs(&self) -> io::Result<Self::Iter>;
|
||||
}
|
||||
```
|
||||
|
||||
### `Server`
|
||||
|
||||
Metadata about a server in the pool:
|
||||
|
||||
```rust
|
||||
pub struct Server {
|
||||
pub addr: ServerAddr,
|
||||
pub failed_attempts: usize,
|
||||
pub did_connect: bool,
|
||||
pub is_discovered: bool,
|
||||
pub last_error: Option<String>,
|
||||
}
|
||||
```
|
||||
|
||||
## Event & State Types
|
||||
|
||||
### `Event`
|
||||
|
||||
Asynchronous notifications from the connection:
|
||||
|
||||
```rust
|
||||
pub enum Event {
|
||||
Connected,
|
||||
Disconnected,
|
||||
LameDuckMode,
|
||||
Draining,
|
||||
Closed,
|
||||
SlowConsumer(u64), // subscription sid
|
||||
ServerError(ServerError),
|
||||
ClientError(ClientError),
|
||||
}
|
||||
```
|
||||
|
||||
Received via `ConnectOptions::event_callback()`.
|
||||
|
||||
### `State`
|
||||
|
||||
Connection state observable via `watch::Receiver`:
|
||||
|
||||
```rust
|
||||
pub enum State {
|
||||
Pending,
|
||||
Connected,
|
||||
Disconnected,
|
||||
}
|
||||
```
|
||||
|
||||
### `StatusCode`
|
||||
|
||||
NATS protocol status codes (e.g., `NO_RESPONDERS = 404`, `TIMEOUT = 408`).
|
||||
|
||||
## Error Types
|
||||
|
||||
All error types follow the pattern `Error<Kind>` from `crate::error`:
|
||||
|
||||
| Error Type | Kind | Used By |
|
||||
|------------|------|---------|
|
||||
| `ConnectError` | `ConnectErrorKind` | Connection establishment |
|
||||
| `PublishError` | `PublishErrorKind` | Publish operations |
|
||||
| `RequestError` | `RequestErrorKind` | Request/reply |
|
||||
| `SubscribeError` | `SubscribeErrorKind` | Subscribe |
|
||||
| `FlushError` | `FlushErrorKind` | Flush |
|
||||
| `DrainError` | — | Drain |
|
||||
|
||||
### `ConnectErrorKind`
|
||||
|
||||
```rust
|
||||
pub enum ConnectErrorKind {
|
||||
ServerParse, // URL parsing failed
|
||||
Dns, // DNS resolution failed
|
||||
Authentication, // Auth signing failed
|
||||
AuthorizationViolation, // Server rejected auth
|
||||
TimedOut, // Connection handshake timeout
|
||||
Tls, // TLS error
|
||||
Io, // Other I/O error
|
||||
MaxReconnects, // Exceeded max reconnect attempts
|
||||
}
|
||||
```
|
||||
|
||||
## Trait Definitions
|
||||
|
||||
The `client::traits` module defines abstract interfaces:
|
||||
|
||||
```rust
|
||||
pub trait Publisher {
|
||||
fn publish_with_reply(&self, subject, reply, payload) -> Future<Output = Result<(), PublishError>>;
|
||||
fn publish_message(&self, msg: OutboundMessage) -> Future<Output = Result<(), PublishError>>;
|
||||
}
|
||||
|
||||
pub trait Subscriber {
|
||||
fn subscribe(&self, subject) -> Future<Output = Result<crate::Subscriber, SubscribeError>>;
|
||||
}
|
||||
|
||||
pub trait Requester {
|
||||
fn send_request(&self, subject, request: Request) -> Future<Output = Result<Message, RequestError>>;
|
||||
}
|
||||
|
||||
pub trait TimeoutProvider {
|
||||
fn timeout(&self) -> Option<Duration>;
|
||||
}
|
||||
```
|
||||
|
||||
`Client` implements all of these. The JetStream `Context` also implements them via delegation.
|
||||
|
||||
## Authentication Types
|
||||
|
||||
### `Auth`
|
||||
|
||||
Container for all authentication methods:
|
||||
|
||||
```rust
|
||||
pub struct Auth {
|
||||
pub jwt: Option<String>,
|
||||
pub nkey: Option<String>,
|
||||
pub signature_callback: Option<CallbackArg1<String, Result<String, AuthError>>>,
|
||||
pub signature: Option<Vec<u8>>,
|
||||
pub username: Option<String>,
|
||||
pub password: Option<String>,
|
||||
pub token: Option<String>,
|
||||
}
|
||||
```
|
||||
|
||||
### `AuthError`
|
||||
|
||||
Simple string error for auth callback failures.
|
||||
|
||||
### `ReconnectToServer`
|
||||
|
||||
Returned by `reconnect_to_server_callback` to select a server and delay:
|
||||
|
||||
```rust
|
||||
pub struct ReconnectToServer {
|
||||
pub addr: ServerAddr,
|
||||
pub delay: Option<Duration>,
|
||||
}
|
||||
```
|
||||
@@ -0,0 +1,278 @@
|
||||
# async-nats: NATS Protocol & Wire Format
|
||||
|
||||
## Protocol Overview
|
||||
|
||||
NATS uses a simple, text-based protocol over TCP. Messages are terminated with `\r\n`. The protocol is symmetric for client and server operations.
|
||||
|
||||
### Client → Server Operations (`ClientOp`)
|
||||
|
||||
```rust
|
||||
pub(crate) enum ClientOp {
|
||||
Publish { subject, payload, respond, headers },
|
||||
Subscribe { sid, subject, queue_group },
|
||||
Unsubscribe { sid, max },
|
||||
Ping,
|
||||
Pong,
|
||||
Connect(ConnectInfo),
|
||||
}
|
||||
```
|
||||
|
||||
### Server → Client Operations (`ServerOp`)
|
||||
|
||||
```rust
|
||||
pub(crate) enum ServerOp {
|
||||
Ok,
|
||||
Info(Box<ServerInfo>),
|
||||
Ping,
|
||||
Pong,
|
||||
Error(ServerError),
|
||||
Message { sid, subject, reply, payload, headers, status, description, length },
|
||||
}
|
||||
```
|
||||
|
||||
## Wire Format: Client Operations
|
||||
|
||||
### CONNECT
|
||||
|
||||
Sent immediately after receiving the first `INFO` from the server:
|
||||
|
||||
```
|
||||
CONNECT {"verbose":false,"pedantic":false,...}\r\n
|
||||
```
|
||||
|
||||
The JSON payload is `ConnectInfo` serialized inline on the same line.
|
||||
|
||||
### PUB (Publish without headers)
|
||||
|
||||
```
|
||||
PUB <subject> [reply-to] <payload-size>\r\n
|
||||
<payload>\r\n
|
||||
```
|
||||
|
||||
Example:
|
||||
```
|
||||
PUB events.data INBOX.67 11\r\n
|
||||
Hello World\r\n
|
||||
```
|
||||
|
||||
### HPUB (Publish with headers)
|
||||
|
||||
When headers are present and non-empty:
|
||||
|
||||
```
|
||||
HPUB <subject> [reply-to] <header-size> <total-size>\r\n
|
||||
<headers>\r\n
|
||||
<payload>\r\n
|
||||
```
|
||||
|
||||
The `<total-size>` = `<header-size>` + `<payload-size>`.
|
||||
|
||||
Header block format:
|
||||
```
|
||||
NATS/1.0\r\n
|
||||
Header-Name: Header-Value\r\n
|
||||
Another-Header: Another-Value\r\n
|
||||
\r\n
|
||||
```
|
||||
|
||||
The version line (`NATS/1.0`) may include a status code and description:
|
||||
```
|
||||
NATS/1.0 404 No Messages\r\n
|
||||
\r\n
|
||||
```
|
||||
|
||||
### SUB (Subscribe)
|
||||
|
||||
```
|
||||
SUB <subject> [queue-group] <sid>\r\n
|
||||
```
|
||||
|
||||
The `sid` (subscription ID) is a client-assigned u64, unique per connection.
|
||||
|
||||
### UNSUB (Unsubscribe)
|
||||
|
||||
```
|
||||
UNSUB <sid> [max]\r\n
|
||||
```
|
||||
|
||||
The optional `max` tells the server to auto-unsubscribe after `max` messages are delivered.
|
||||
|
||||
### PING / PONG
|
||||
|
||||
```
|
||||
PING\r\n
|
||||
PONG\r\n
|
||||
```
|
||||
|
||||
Client sends PING periodically (default every 60s). If 2+ pings are pending without PONG, the connection is considered dead.
|
||||
|
||||
## Wire Format: Server Operations
|
||||
|
||||
### INFO
|
||||
|
||||
First message sent by the server on connection:
|
||||
|
||||
```
|
||||
INFO {"server_id":"NATSxxx","version":"2.10"...}\r\n
|
||||
```
|
||||
|
||||
Also sent asynchronously when cluster topology changes.
|
||||
|
||||
### MSG (Message without headers)
|
||||
|
||||
```
|
||||
MSG <subject> <sid> [reply-to] <payload-size>\r\n
|
||||
<payload>\r\n
|
||||
```
|
||||
|
||||
### HMSG (Message with headers)
|
||||
|
||||
```
|
||||
HMSG <subject> <sid> [reply-to] <header-size> <total-size>\r\n
|
||||
<headers + payload>\r\n
|
||||
```
|
||||
|
||||
### +OK / -ERR
|
||||
|
||||
```
|
||||
+OK\r\n
|
||||
-ERR <description>\r\n
|
||||
```
|
||||
|
||||
Sent only when `verbose=true` in `CONNECT`. The client always sets `verbose=false`, so `+OK` is not expected.
|
||||
|
||||
## Protocol Parser
|
||||
|
||||
The `Connection` struct handles all protocol parsing and serialization:
|
||||
|
||||
### Read Path (`try_read_op`)
|
||||
|
||||
1. Search for `\r\n` in `read_buf` using `memchr::memmem::find`
|
||||
2. Inspect the first bytes to determine the operation type:
|
||||
- `+OK` → `ServerOp::Ok`
|
||||
- `PING` → `ServerOp::Ping`
|
||||
- `PONG` → `ServerOp::Pong`
|
||||
- `-ERR` → `ServerOp::Error(...)` (description is `trim_matches('\'')`)
|
||||
- `INFO ` → `ServerOp::Info(...)` (serde_json deserialization)
|
||||
- `MSG ` → Parse subject/sid/reply/size, then read payload
|
||||
- `HMSG ` → Parse subject/sid/reply/header_len/total_len, then read headers + payload
|
||||
3. For `MSG`/`HMSG`: if the full message body hasn't been read yet, return `None` (wait for more data)
|
||||
4. For `HMSG`: parse the header block — extract version line (`NATS/1.0[ <status>[ <description>]]`), then key-value pairs (supports folded/multi-line header values)
|
||||
|
||||
### Write Path (`enqueue_write_op`)
|
||||
|
||||
Writes into a buffer strategy:
|
||||
- **Small writes** (< 4096 bytes): flattened into `flattened_writes: BytesMut`
|
||||
- **Large writes** (≥ 4096 bytes): appended as separate `Bytes` chunks in `write_buf: VecDeque<Bytes>`
|
||||
|
||||
This enables efficient vectored I/O when the underlying stream supports it.
|
||||
|
||||
### Write Flush Strategy
|
||||
|
||||
The `should_flush()` method returns:
|
||||
- `Yes` — buffers empty but haven't flushed yet
|
||||
- `May` — buffers not empty and haven't flushed
|
||||
- `No` — already flushed or nothing to flush
|
||||
|
||||
The `ConnectionHandler` calls `poll_flush()` after processing commands, ensuring data is actually sent to the server.
|
||||
|
||||
## Vectored I/O
|
||||
|
||||
When `stream.is_write_vectored()` returns true, the connection uses `poll_write_vectored()` to write up to 64 `IoSlice`s at once. This is significantly more efficient for bursty publish patterns.
|
||||
|
||||
```rust
|
||||
const WRITE_VECTORED_CHUNKS: usize = 64;
|
||||
```
|
||||
|
||||
## WebSocket Transport
|
||||
|
||||
When the `websockets` feature is enabled, `WebSocketAdapter<T>` wraps `tokio_websockets::WebSocketStream<T>` to implement `AsyncRead + AsyncWrite`, making WebSocket connections transparent to the protocol layer.
|
||||
|
||||
```rust
|
||||
#[cfg(feature = "websockets")]
|
||||
pub(crate) struct WebSocketAdapter<T> {
|
||||
pub(crate) inner: WebSocketStream<T>,
|
||||
pub(crate) read_buf: BytesMut,
|
||||
}
|
||||
```
|
||||
|
||||
WebSocket connections use `ws://` or `wss://` scheme in the server URL. TLS for `wss://` is handled by the WebSocket library's built-in TLS support.
|
||||
|
||||
## Connection Lifecycle
|
||||
|
||||
### Initial Connection Flow
|
||||
|
||||
```
|
||||
Client Server
|
||||
│ │
|
||||
│──── TCP connect ────────────────────▶ │
|
||||
│◀──── INFO {server_id, nonce, ...} ─── │
|
||||
│──── CONNECT {auth, ...} ──────────▶ │
|
||||
│──── PING ─────────────────────────▶ │
|
||||
│◀──── PONG (or -ERR) ─────────────── │
|
||||
│ │
|
||||
│ [connected, ConnectionHandler runs] │
|
||||
```
|
||||
|
||||
If `tls_first` is enabled, TLS is established before reading INFO:
|
||||
|
||||
```
|
||||
Client Server
|
||||
│ │
|
||||
│──── TCP connect ────────────────────▶ │
|
||||
│──── TLS handshake ─────────────────▶ │
|
||||
│◀──── TLS handshake ──────────────── │
|
||||
│◀──── INFO {...} ──────────────────── │
|
||||
│──── CONNECT + PING ────────────────▶ │
|
||||
│◀──── PONG ────────────────────────── │
|
||||
```
|
||||
|
||||
### Ping/Pong Keepalive
|
||||
|
||||
- Client sends PING every `ping_interval` (default 60s)
|
||||
- Server responds with PONG
|
||||
- If `pending_pings > MAX_PENDING_PINGS (2)`, connection is considered dead
|
||||
- Any server operation resets the ping interval timer
|
||||
|
||||
### Reconnection Flow
|
||||
|
||||
On disconnect:
|
||||
1. `handle_disconnect()` sends `Event::Disconnected` and sets state to `Disconnected`
|
||||
2. `handle_reconnect()` calls `connector.connect()` which:
|
||||
- Shuffles servers (unless `retain_servers_order`)
|
||||
- Sorts by `failed_attempts` (ascending)
|
||||
- Iterates through servers with exponential backoff delay
|
||||
- On each server: DNS resolve → TCP connect → INFO → TLS (if needed) → CONNECT+PING → PONG
|
||||
3. On success:
|
||||
- Sends `Event::Connected`, sets state to `Connected`
|
||||
- Removes closed subscriptions
|
||||
- Re-subscribes all active subscriptions (with adjusted `max = max - delivered`)
|
||||
- Re-subscribes the multiplexer (if active)
|
||||
4. On failure with `MaxReconnects` reached, the handler loop exits
|
||||
|
||||
### Default Reconnect Delay
|
||||
|
||||
Exponential backoff capped at 4 seconds:
|
||||
|
||||
```rust
|
||||
fn reconnect_delay_callback_default(attempts: usize) -> Duration {
|
||||
if attempts <= 1 {
|
||||
Duration::from_millis(0)
|
||||
} else {
|
||||
let exp: u32 = (attempts - 1).try_into().unwrap_or(u32::MAX);
|
||||
cmp::min(Duration::from_millis(2_u64.saturating_pow(exp)), Duration::from_secs(4))
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
| Attempt | Delay |
|
||||
|---------|-------|
|
||||
| 1 | 0ms |
|
||||
| 2 | 0ms |
|
||||
| 3 | 2ms |
|
||||
| 4 | 8ms |
|
||||
| 5 | 32ms |
|
||||
| 6 | 128ms |
|
||||
| 7 | 512ms |
|
||||
| 8 | 2048ms |
|
||||
| 9+ | 4000ms (cap) |
|
||||
@@ -0,0 +1,221 @@
|
||||
# async-nats: Connection Management & Configuration
|
||||
|
||||
## ConnectOptions Builder
|
||||
|
||||
`ConnectOptions` provides a builder for all connection configuration:
|
||||
|
||||
```rust
|
||||
let client = ConnectOptions::new()
|
||||
.require_tls(true)
|
||||
.ping_interval(Duration::from_secs(10))
|
||||
.name("my-service")
|
||||
.connect("demo.nats.io")
|
||||
.await?;
|
||||
```
|
||||
|
||||
### Authentication Methods
|
||||
|
||||
| Method | Description |
|
||||
|--------|-------------|
|
||||
| `with_token(token)` | Token-based auth |
|
||||
| `with_user_and_password(user, pass)` | Username/password auth |
|
||||
| `with_nkey(seed)` | NKey auth (requires `nkeys` feature) |
|
||||
| `with_jwt(jwt, sign_cb)` | JWT + signing callback (requires `nkeys`) |
|
||||
| `with_credentials_file(path)` | Load from `.creds` file (requires `nkeys`) |
|
||||
| `with_credentials(creds_str)` | Parse credentials string (requires `nkeys`) |
|
||||
| `with_auth_callback(cb)` | Dynamic auth callback receiving nonce, returning `Auth` |
|
||||
|
||||
The auth callback is the most flexible — it receives the server nonce and can return any combination of auth fields:
|
||||
|
||||
```rust
|
||||
ConnectOptions::with_auth_callback(move |nonce| async move {
|
||||
let mut auth = Auth::new();
|
||||
auth.username = Some("user".to_string());
|
||||
auth.password = Some("pass".to_string());
|
||||
Ok(auth)
|
||||
})
|
||||
```
|
||||
|
||||
### TLS Configuration
|
||||
|
||||
| Option | Description |
|
||||
|-------|-------------|
|
||||
| `require_tls(bool)` | Require TLS for the connection |
|
||||
| `tls_first()` | Establish TLS before INFO (requires server `handshake_first`) |
|
||||
| `add_root_certificates(path)` | Load root CA certificates from PEM file |
|
||||
| `add_client_certificate(cert, key)` | Load client certificate for mTLS |
|
||||
| `tls_client_config(config)` | Pass a custom `rustls::ClientConfig` |
|
||||
|
||||
Two TLS crypto backends: `ring` (default) or `aws-lc-rs` (via feature flags). FIPS mode available via `aws-lc-rs` + `fips` features.
|
||||
|
||||
### Connection Behavior
|
||||
|
||||
| Option | Default | Description |
|
||||
|--------|---------|-------------|
|
||||
| `connection_timeout` | 5s | Timeout for full connection establishment |
|
||||
| `request_timeout` | 10s | Default timeout for `Client::request` |
|
||||
| `ping_interval` | 60s | How often client sends PING |
|
||||
| `retry_on_initial_connect` | false | Return client immediately, connect in background |
|
||||
| `max_reconnects` | None (unlimited) | Max consecutive reconnect attempts |
|
||||
| `ignore_discovered_servers` | false | Ignore servers advertised in INFO |
|
||||
| `retain_servers_order` | false | Don't shuffle server list on reconnect |
|
||||
| `skip_subject_validation` | false | Skip whitespace validation on publish subjects |
|
||||
| `subscription_capacity` | 65536 | mpsc channel capacity per subscription |
|
||||
| `client_capacity` | 2048 | mpsc channel capacity for command sender |
|
||||
| `custom_inbox_prefix` | `_INBOX` | Custom prefix for inbox subjects |
|
||||
| `read_buffer_capacity` | 65535 | Initial size of the protocol read buffer |
|
||||
| `local_address` | None | Local socket address to bind to |
|
||||
| `no_echo` | false | Don't deliver messages published by this connection |
|
||||
|
||||
### Reconnection Callbacks
|
||||
|
||||
**`reconnect_delay_callback`**: Custom backoff strategy:
|
||||
|
||||
```rust
|
||||
.reconnect_delay_callback(|attempts| {
|
||||
Duration::from_millis(std::cmp::min((attempts * 100) as u64, 8000))
|
||||
})
|
||||
```
|
||||
|
||||
**`reconnect_to_server_callback`**: Select which server to connect to on each reconnect attempt:
|
||||
|
||||
```rust
|
||||
.reconnect_to_server_callback(|servers, _info| async move {
|
||||
servers.first().map(|s| ReconnectToServer {
|
||||
addr: s.addr.clone(),
|
||||
delay: Some(Duration::ZERO),
|
||||
})
|
||||
})
|
||||
```
|
||||
|
||||
Receives `(Vec<Server>, ServerInfo)`, returns `Option<ReconnectToServer>`. If the returned server isn't in the pool, falls back to default selection.
|
||||
|
||||
**`event_callback`**: Receive async notifications:
|
||||
|
||||
```rust
|
||||
.event_callback(|event| async move {
|
||||
match event {
|
||||
Event::Disconnected => println!("disconnected"),
|
||||
Event::Connected => println!("connected"),
|
||||
Event::SlowConsumer(sid) => eprintln!("slow consumer: {sid}"),
|
||||
_ => {}
|
||||
}
|
||||
})
|
||||
```
|
||||
|
||||
## Connection Handler Internals
|
||||
|
||||
### ProcessFut — The Core Event Loop
|
||||
|
||||
The `ConnectionHandler::process()` method creates a custom `Future` (`ProcessFut`) that drives the connection forward. Each `poll()` call:
|
||||
|
||||
1. **Check ping interval** — if timer ticked, send PING; if too many pending pings, disconnect
|
||||
2. **Read server operations** — drain all available `ServerOp`s from `Connection::poll_read_op()`
|
||||
3. **Process drain completions** — remove subscriptions that finished draining
|
||||
4. **Handle commands** — receive up to 16 `Command`s from the mpsc channel and process them
|
||||
5. **Write to socket** — flush the write buffer via `Connection::poll_write()`
|
||||
6. **Flush** — call `poll_flush()` on the underlying stream when needed
|
||||
7. **Check reconnect flag** — if `should_reconnect` is set, shut down and reconnect
|
||||
|
||||
```rust
|
||||
const RECV_CHUNK_SIZE: usize = 16;
|
||||
```
|
||||
|
||||
### Exit Reasons
|
||||
|
||||
The event loop exits with one of:
|
||||
|
||||
| Reason | Action |
|
||||
|--------|--------|
|
||||
| `Disconnected(Option<io::Error>)` | Attempt reconnection |
|
||||
| `ReconnectRequested` | Shut down stream, attempt reconnection |
|
||||
| `Closed` | Send `Event::Closed`, exit loop |
|
||||
|
||||
### Handle Disconnect & Reconnect
|
||||
|
||||
```rust
|
||||
async fn handle_disconnect(&mut self) -> Result<(), ConnectError> {
|
||||
self.pending_pings = 0;
|
||||
self.connector.events_tx.try_send(Event::Disconnected).ok();
|
||||
self.connector.state_tx.send(State::Disconnected).ok();
|
||||
self.handle_reconnect().await
|
||||
}
|
||||
|
||||
async fn handle_reconnect(&mut self) -> Result<(), ConnectError> {
|
||||
let (info, connection) = self.connector.connect().await?;
|
||||
self.connection = connection;
|
||||
let _ = self.info_sender.send(Some(info));
|
||||
|
||||
// Remove closed subscriptions
|
||||
self.subscriptions.retain(|_, sub| !sub.sender.is_closed());
|
||||
|
||||
// Re-subscribe all active subscriptions
|
||||
for (sid, subscription) in &self.subscriptions {
|
||||
self.connection.enqueue_write_op(&ClientOp::Subscribe {
|
||||
sid: *sid,
|
||||
subject: subscription.subject.to_owned(),
|
||||
queue_group: subscription.queue_group.to_owned(),
|
||||
});
|
||||
if let Some(max) = subscription.max {
|
||||
self.connection.enqueue_write_op(&ClientOp::Unsubscribe {
|
||||
sid: *sid,
|
||||
max: Some(max.saturating_sub(subscription.delivered)),
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
// Re-subscribe multiplexer if active
|
||||
if let Some(multiplexer) = &self.multiplexer {
|
||||
self.connection.enqueue_write_op(&ClientOp::Subscribe {
|
||||
sid: MULTIPLEXER_SID,
|
||||
subject: multiplexer.subject.to_owned(),
|
||||
queue_group: None,
|
||||
});
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
```
|
||||
|
||||
## Request/Reply Multiplexer
|
||||
|
||||
The client uses a **multiplexer** pattern for request/reply to avoid creating a separate subscription per request:
|
||||
|
||||
1. A single wildcard subscription is created on first request: `_INBOX.<random_id>.*`
|
||||
2. Each request gets a unique token appended to the inbox: `_INBOX.<random_id>.<token>`
|
||||
3. When a response arrives, the token is extracted from the subject and used to look up the `oneshot::Sender` in `multiplexer.senders`
|
||||
4. The response is forwarded through the oneshot channel to the waiting `send_request()` future
|
||||
|
||||
```rust
|
||||
struct Multiplexer {
|
||||
subject: Subject, // _INBOX.<id>.*
|
||||
prefix: Subject, // _INBOX.<id>.
|
||||
senders: HashMap<String, oneshot::Sender<Message>>, // token → sender
|
||||
}
|
||||
```
|
||||
|
||||
The multiplexer subscription uses `sid = 0` (`MULTIPLEXER_SID`), which is separate from regular subscription IDs (which start at 1).
|
||||
|
||||
### Custom Inbox Bypass
|
||||
|
||||
If a `Request` has a custom `inbox` set, the multiplexer is bypassed — a dedicated subscription is created for that specific request, and the timeout/response logic is handled locally within `send_request()`.
|
||||
|
||||
## Server Pool Management
|
||||
|
||||
The `Connector` maintains a `Vec<Server>` pool. Servers can come from:
|
||||
1. **Explicit URLs** — provided by the user at connect time
|
||||
2. **Discovered servers** — advertised in `INFO.connect_urls` (unless `ignore_discovered_servers` is set)
|
||||
|
||||
On reconnection:
|
||||
- Servers are shuffled (unless `retain_servers_order`)
|
||||
- Sorted by `failed_attempts` (ascending) — prefer servers that haven't failed recently
|
||||
- Each server is tried with exponential backoff delay
|
||||
- On success: `failed_attempts` reset to 0, `did_connect` set to true
|
||||
- On failure: `failed_attempts` incremented, `last_error` updated
|
||||
|
||||
### Dynamic Server Pool Updates
|
||||
|
||||
`Client::set_server_pool()` replaces the pool at runtime:
|
||||
- Per-server state is preserved for servers that appear in both old and new pools
|
||||
- The global reconnection attempt counter is reset
|
||||
- Cannot mix WebSocket and non-WebSocket URLs
|
||||
- Pool cannot be empty
|
||||
373
docs/research/references/nats.rs/nats-async/05-jetstream.md
Normal file
373
docs/research/references/nats.rs/nats-async/05-jetstream.md
Normal file
@@ -0,0 +1,373 @@
|
||||
# async-nats: JetStream
|
||||
|
||||
## Overview
|
||||
|
||||
JetStream is NATS' built-in persistence layer, providing stream-based messaging with at-least-once and exactly-once delivery semantics. The `async-nats` JetStream API is accessed through a `Context` object.
|
||||
|
||||
### Creating a Context
|
||||
|
||||
```rust
|
||||
// Default context (prefix: $JS.API)
|
||||
let jetstream = async_nats::jetstream::new(client);
|
||||
|
||||
// With domain (prefix: $JS.<domain>.API)
|
||||
let jetstream = async_nats::jetstream::with_domain(client, "hub");
|
||||
|
||||
// With custom prefix
|
||||
let jetstream = async_nats::jetstream::with_prefix(client, "JS.acc@hub.API");
|
||||
|
||||
// Builder with fine-grained control
|
||||
let context = ContextBuilder::new()
|
||||
.timeout(Duration::from_secs(5))
|
||||
.api_prefix("MY.JS.API")
|
||||
.max_ack_inflight(1000)
|
||||
.backpressure_on_inflight(true)
|
||||
.ack_timeout(Duration::from_secs(30))
|
||||
.build(client);
|
||||
```
|
||||
|
||||
## Context
|
||||
|
||||
```rust
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct Context {
|
||||
pub(crate) client: Client,
|
||||
pub(crate) prefix: String,
|
||||
pub(crate) timeout: Duration,
|
||||
pub(crate) max_ack_semaphore: Arc<tokio::sync::Semaphore>,
|
||||
pub(crate) ack_sender: mpsc::Sender<(oneshot::Receiver<Message>, OwnedSemaphorePermit)>,
|
||||
pub(crate) backpressure_on_inflight: bool,
|
||||
pub(crate) semaphore_capacity: usize,
|
||||
}
|
||||
```
|
||||
|
||||
### Publish Backpressure
|
||||
|
||||
The context uses a semaphore to limit the number of pending publish acknowledgments:
|
||||
|
||||
- `max_ack_inflight(n)` — sets semaphore capacity (default 5000)
|
||||
- `backpressure_on_inflight(true)` — `publish()` waits for a permit when limit is reached
|
||||
- `backpressure_on_inflight(false)` — `publish()` returns `MaxAckPending` error immediately when limit is reached
|
||||
|
||||
A background **acker task** monitors pending acks with a timeout (`ack_timeout`, default 30s), releasing permits when acks arrive or time out.
|
||||
|
||||
### JetStream API Request Pattern
|
||||
|
||||
All JetStream API calls follow the same pattern:
|
||||
|
||||
1. Build a subject from the prefix: `format!("{}.STREAM.CREATE.<name>", self.prefix)`
|
||||
2. Serialize the request payload as JSON
|
||||
3. Send a request via `client.send_request()` with the API subject
|
||||
4. Deserialize the response as `Response<T>` (which is `Ok(T)` or `Err(ErrorCode)`)
|
||||
|
||||
## Streams
|
||||
|
||||
### Stream Handle
|
||||
|
||||
```rust
|
||||
pub struct Stream<I = Info> {
|
||||
context: Context,
|
||||
info: I,
|
||||
name: String,
|
||||
}
|
||||
```
|
||||
|
||||
`Stream<Info>` carries server-side info. `Stream<()>` is a lightweight handle that skips the INFO fetch. `Stream` (no generic) defaults to `Stream<Info>`.
|
||||
|
||||
### Stream Config
|
||||
|
||||
```rust
|
||||
pub struct Config {
|
||||
pub name: String,
|
||||
pub description: Option<String>,
|
||||
pub subjects: Vec<String>,
|
||||
pub retention: RetentionPolicy,
|
||||
pub max_consumers: i64,
|
||||
pub max_messages: i64,
|
||||
pub max_messages_per_subject: i64,
|
||||
pub max_bytes: i64,
|
||||
pub max_age: Duration,
|
||||
pub max_messages_per_stream: i64,
|
||||
pub max_msg_size: i32,
|
||||
pub discard: DiscardPolicy,
|
||||
pub discard_new_per_subject: bool,
|
||||
pub storage: StorageType,
|
||||
pub num_replicas: usize,
|
||||
pub no_ack: bool,
|
||||
pub duplicate_window: Duration,
|
||||
pub placement: Option<Placement>,
|
||||
pub mirror: Option<Source>,
|
||||
pub sources: Option<Vec<Source>>,
|
||||
pub sealed: bool,
|
||||
pub allow_direct: bool,
|
||||
pub allow_rollup_hdrs: bool,
|
||||
// server_2_10 features:
|
||||
pub compression: Option<Compression>,
|
||||
pub first_sequence: Option<u64>,
|
||||
pub subject_transform: Option<SubjectTransform>,
|
||||
pub republish: Option<Republish>,
|
||||
pub metadata: Option<HashMap<String, String>>,
|
||||
}
|
||||
```
|
||||
|
||||
### Stream Operations
|
||||
|
||||
| Method | Description |
|
||||
|--------|-------------|
|
||||
| `create_stream(config)` | Create a new stream |
|
||||
| `get_stream(name)` | Get stream handle (with INFO) |
|
||||
| `get_stream_no_info(name)` | Get lightweight handle (no server round-trip) |
|
||||
| `get_or_create_stream(config)` | Get existing or create new |
|
||||
| `delete_stream(name)` | Delete a stream |
|
||||
| `update_stream(config)` | Update stream configuration |
|
||||
| `create_or_update_stream(config)` | Update or create if not found |
|
||||
| `stream_names()` | `Stream` of stream names (paginated) |
|
||||
| `streams()` | `Stream` of stream info (paginated) |
|
||||
| `stream_by_subject(subject)` | Find stream name containing subject |
|
||||
|
||||
### Stream Handle Methods
|
||||
|
||||
```rust
|
||||
let stream: Stream = jetstream.get_stream("events").await?;
|
||||
|
||||
// Info
|
||||
let info: Info = stream.info().await?; // Fresh info from server
|
||||
let info: &Info = stream.cached_info(); // Cached info from last fetch
|
||||
|
||||
// Message operations
|
||||
stream.get_raw_message(seq).await?; // Get raw message by sequence
|
||||
stream.get_last_raw_message_by_subject(subj).await?; // Get last message for subject
|
||||
stream.direct_get(seq).await?; // Direct get (if allow_direct)
|
||||
stream.direct_get_last_for_subject(subj).await?; // Direct last by subject
|
||||
stream.delete_message(seq).await?; // Delete a specific message
|
||||
stream.purge().await?; // Purge all messages
|
||||
stream.purge().filter(subj).await?; // Purge messages for subject
|
||||
|
||||
// Consumers
|
||||
stream.create_consumer(config).await?; // Create consumer bound to stream
|
||||
stream.get_consumer(name).await?; // Get existing consumer
|
||||
stream.delete_consumer(name).await?; // Delete consumer
|
||||
```
|
||||
|
||||
## Consumers
|
||||
|
||||
### Consumer Types
|
||||
|
||||
Two consumer types, each with distinct delivery models:
|
||||
|
||||
1. **Pull Consumer** (`pull::Config` / `PullConsumer`) — Client explicitly requests batches of messages
|
||||
2. **Push Consumer** (`push::Config` / `PushConsumer`) — Server pushes messages to a deliver subject
|
||||
|
||||
### Pull Consumer
|
||||
|
||||
```rust
|
||||
let consumer: PullConsumer = stream
|
||||
.get_or_create_consumer("my-consumer", pull::Config {
|
||||
durable_name: Some("my-consumer".to_string()),
|
||||
..Default::default()
|
||||
})
|
||||
.await?;
|
||||
```
|
||||
|
||||
**Key methods**:
|
||||
- `consumer.batch(n).await?` — Fetch up to `n` messages (one-shot batch)
|
||||
- `consumer.messages().await?` — Continuous `Stream` of messages
|
||||
- `consumer.sequence(n).await?` — Continuous `Stream` of batches of `n` messages
|
||||
- `consumer.fetch().max(n).expires(dur).await?` — Configurable fetch
|
||||
|
||||
Each message from a pull consumer is a `jetstream::Message` which has `ack()` methods.
|
||||
|
||||
### Push Consumer
|
||||
|
||||
Two push consumer variants:
|
||||
|
||||
1. **Standard** (`push::Config`) — messages delivered to a specific subject
|
||||
2. **Ordered** (`push::OrderedConfig`) — auto-recreated on failure, with flow control
|
||||
|
||||
```rust
|
||||
// Standard push
|
||||
let consumer = stream.create_consumer(push::Config {
|
||||
deliver_subject: "deliver.subject".to_string(),
|
||||
durable_name: Some("push-consumer".to_string()),
|
||||
..Default::default()
|
||||
}).await?;
|
||||
|
||||
// Ordered push (no durable name, auto-recreates on failure)
|
||||
let consumer = stream.create_consumer(push::OrderedConfig {
|
||||
deliver_subject: client.new_inbox(),
|
||||
filter_subject: "events.>".to_string(),
|
||||
..Default::default()
|
||||
}).await?;
|
||||
```
|
||||
|
||||
### Consumer Config (Shared Fields)
|
||||
|
||||
```rust
|
||||
pub struct Config {
|
||||
// Pull fields
|
||||
pub durable_name: Option<String>,
|
||||
pub name: Option<String>,
|
||||
|
||||
// Push fields
|
||||
pub deliver_subject: Option<String>,
|
||||
pub deliver_group: Option<String>,
|
||||
pub deliver_policy: DeliverPolicy,
|
||||
pub opt_start_time: Option<DateTime>,
|
||||
pub opt_start_sequence: Option<u64>,
|
||||
pub ack_policy: AckPolicy,
|
||||
pub ack_wait: Duration,
|
||||
pub max_deliver: i64,
|
||||
pub backoff: Vec<Duration>,
|
||||
pub filter_subject: String,
|
||||
pub filter_subjects: Vec<String>, // server_2_10+
|
||||
pub replay_policy: ReplayPolicy,
|
||||
pub rate_limit_bps: Option<u64>,
|
||||
pub max_waiting: i64, // pull: max outstanding pull requests
|
||||
pub max_ack_pending: i64,
|
||||
pub flow_control: bool,
|
||||
pub idle_heartbeat: Duration,
|
||||
pub headers_only: bool,
|
||||
pub num_replicas: usize,
|
||||
pub mem_storage: bool,
|
||||
pub description: Option<String>,
|
||||
pub metadata: Option<HashMap<String, String>>,
|
||||
pub inactive_threshold: Option<Duration>, // for ephemeral consumers
|
||||
}
|
||||
```
|
||||
|
||||
### Deliver Policy
|
||||
|
||||
```rust
|
||||
pub enum DeliverPolicy {
|
||||
All, // Deliver all messages
|
||||
Last, // Deliver last message only
|
||||
New, // Deliver only new messages
|
||||
ByStartSequence { start_sequence: u64 },
|
||||
ByStartTime { start_time: DateTime },
|
||||
LastPerSubject, // Deliver last message per subject
|
||||
}
|
||||
```
|
||||
|
||||
### Ack Policy
|
||||
|
||||
```rust
|
||||
pub enum AckPolicy {
|
||||
None, // No acknowledgment needed
|
||||
All, // Ack all messages up to this one
|
||||
Explicit, // Ack each message individually
|
||||
}
|
||||
```
|
||||
|
||||
## JetStream Messages
|
||||
|
||||
### `jetstream::Message`
|
||||
|
||||
Wraps a core `Message` with JetStream-specific metadata:
|
||||
|
||||
```rust
|
||||
pub struct Message {
|
||||
pub message: crate::Message, // The underlying NATS message
|
||||
pub ack_subject: Subject, // Subject for sending acks
|
||||
pub stream: String, // Stream name
|
||||
pub consumer: String, // Consumer name
|
||||
pub stream_sequence: u64, // Sequence in stream
|
||||
pub consumer_sequence: u64, // Sequence for this consumer
|
||||
pub delivered: u64, // Delivery count
|
||||
pub pending: u64, // Pending message count
|
||||
pub published: DateTime, // Original publish time
|
||||
}
|
||||
```
|
||||
|
||||
### Ack Methods
|
||||
|
||||
```rust
|
||||
// In-memory ack (non-persistent, fast)
|
||||
message.ack().await?;
|
||||
|
||||
// Ack with specific type
|
||||
message.ack_with(AckKind::Nak).await?;
|
||||
message.ack_with(AckKind::Progress).await?;
|
||||
message.ack_with(AckKind::Term).await?;
|
||||
message.ack_with(AckKind::NakWithDelay(duration)).await?;
|
||||
message.ack_with(AckKind::TermWithReason("reason")).await?;
|
||||
```
|
||||
|
||||
### `AckKind`
|
||||
|
||||
```rust
|
||||
pub enum AckKind {
|
||||
Ack, // +ACK — message processed
|
||||
Nak, // -NAK — re-deliver
|
||||
Progress, // PRI — still working
|
||||
Term, // +TERM — don't redeliver
|
||||
NakWithDelay(Duration), // -NAK with re-delivery delay
|
||||
TermWithReason(String), // +TERM with reason
|
||||
}
|
||||
```
|
||||
|
||||
## JetStream Publish
|
||||
|
||||
### `Context::publish()`
|
||||
|
||||
JetStream publish returns a `PublishAckFuture` — a future that resolves to a `PublishAck`:
|
||||
|
||||
```rust
|
||||
let ack_future = jetstream.publish("events", "data".into()).await?;
|
||||
let ack: PublishAck = ack_future.await?; // Wait for server acknowledgment
|
||||
```
|
||||
|
||||
### `PublishAck`
|
||||
|
||||
```rust
|
||||
pub struct PublishAck {
|
||||
pub stream: String,
|
||||
pub sequence: u64,
|
||||
pub domain: String,
|
||||
pub duplicate: bool,
|
||||
}
|
||||
```
|
||||
|
||||
### `PublishMessage` Builder
|
||||
|
||||
```rust
|
||||
let ack = jetstream.send_publish(
|
||||
"events",
|
||||
PublishMessage::build()
|
||||
.payload("data".into())
|
||||
.message_id("uuid-123") // Deduplication ID
|
||||
.expected_stream("events") // Fail if wrong stream
|
||||
.expected_last_msg_id("prev-id")
|
||||
.expected_last_sequence(42)
|
||||
.headers(headers),
|
||||
).await?;
|
||||
```
|
||||
|
||||
## Pagination
|
||||
|
||||
Stream and consumer listing uses pagination internally:
|
||||
|
||||
```rust
|
||||
pub struct StreamNames {
|
||||
context: Context,
|
||||
offset: usize,
|
||||
page_request: Option<Request>,
|
||||
streams: Vec<String>,
|
||||
subject: Option<String>,
|
||||
done: bool,
|
||||
}
|
||||
```
|
||||
|
||||
Implements `futures_util::Stream<Item = Result<String, Error>>`, lazily fetching pages as needed.
|
||||
|
||||
## Error Handling
|
||||
|
||||
JetStream errors follow the `Response<T>` pattern:
|
||||
|
||||
```rust
|
||||
pub enum Response<T> {
|
||||
Ok(T),
|
||||
Err { error: ErrorCode },
|
||||
}
|
||||
```
|
||||
|
||||
`ErrorCode` carries the server's error code and description. Most JetStream-specific errors map to typed error enums (e.g., `CreateStreamError`, `ConsumerError`, etc.).
|
||||
@@ -0,0 +1,237 @@
|
||||
# async-nats: Key-Value Store
|
||||
|
||||
## Overview
|
||||
|
||||
The Key-Value (KV) store is an abstraction built on top of JetStream streams. Each KV bucket is backed by a JetStream stream with the naming convention `KV_<bucket_name>`. Keys are mapped to subjects under the `$KV.<bucket>.<key>` prefix.
|
||||
|
||||
The KV feature requires `kv` (which implies `jetstream`).
|
||||
|
||||
## Store Handle
|
||||
|
||||
```rust
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct Store {
|
||||
pub name: String,
|
||||
pub stream_name: String,
|
||||
pub prefix: String, // $KV.<bucket>.
|
||||
pub put_prefix: Option<String>, // For mirrored buckets
|
||||
pub use_jetstream_prefix: bool, // Whether to prepend JS API prefix
|
||||
pub stream: Stream,
|
||||
}
|
||||
```
|
||||
|
||||
## Bucket Config
|
||||
|
||||
```rust
|
||||
#[derive(Debug, Clone, Default)]
|
||||
pub struct Config {
|
||||
pub bucket: String,
|
||||
pub description: String,
|
||||
pub max_value_size: i32,
|
||||
pub history: i64, // Max historical entries per key (1-64)
|
||||
pub max_age: Duration, // Max age of any entry
|
||||
pub max_bytes: i64, // Total bucket size limit
|
||||
pub storage: StorageType, // File or Memory
|
||||
pub num_replicas: usize,
|
||||
pub republish: Option<Republish>,
|
||||
pub mirror: Option<Source>, // Mirror another bucket
|
||||
pub sources: Option<Vec<Source>>,
|
||||
pub mirror_direct: bool,
|
||||
pub compression: bool, // server_2_10+
|
||||
pub placement: Option<Placement>,
|
||||
pub limit_markers: Option<Duration>, // server_2_11+
|
||||
}
|
||||
```
|
||||
|
||||
## Creating/Accessing Buckets
|
||||
|
||||
```rust
|
||||
// Create a new bucket
|
||||
let kv = jetstream.create_key_value(kv::Config {
|
||||
bucket: "my-bucket".to_string(),
|
||||
history: 10,
|
||||
max_age: Duration::from_secs(3600),
|
||||
..Default::default()
|
||||
}).await?;
|
||||
|
||||
// Get an existing bucket
|
||||
let kv = jetstream.get_key_value("my-bucket").await?;
|
||||
|
||||
// Create or update
|
||||
let kv = jetstream.create_or_update_key_value(kv::Config { ... }).await?;
|
||||
|
||||
// Delete a bucket
|
||||
jetstream.delete_key_value("my-bucket").await?;
|
||||
```
|
||||
|
||||
## KV Operations
|
||||
|
||||
### Put
|
||||
|
||||
```rust
|
||||
let revision: u64 = kv.put("key", "value".into()).await?;
|
||||
```
|
||||
|
||||
Publishes to `$KV.<bucket>.<key>` (or with JS prefix). The JetStream stream stores it, and the returned sequence number serves as the revision.
|
||||
|
||||
### Get
|
||||
|
||||
```rust
|
||||
let value: Option<Bytes> = kv.get("key").await?;
|
||||
```
|
||||
|
||||
Returns `None` if the key doesn't exist or was deleted/purged. Uses either direct get (if `allow_direct`) or the standard message API.
|
||||
|
||||
### Entry
|
||||
|
||||
```rust
|
||||
let entry: Option<Entry> = kv.entry("key").await?;
|
||||
let entry: Option<Entry> = kv.entry_for_revision("key", 2).await?;
|
||||
```
|
||||
|
||||
Returns full entry metadata:
|
||||
|
||||
```rust
|
||||
pub struct Entry {
|
||||
pub bucket: String,
|
||||
pub key: String,
|
||||
pub value: Bytes,
|
||||
pub revision: u64,
|
||||
pub created: DateTime,
|
||||
pub delta: u64,
|
||||
pub operation: Operation,
|
||||
pub seen_current: bool,
|
||||
}
|
||||
```
|
||||
|
||||
### Create (Put if not exists)
|
||||
|
||||
```rust
|
||||
let revision: u64 = kv.create("key", "value".into()).await?;
|
||||
```
|
||||
|
||||
Uses `update` with `expected_last_subject_sequence = 0` (create-only). If the key exists and is deleted/purged, it's re-created.
|
||||
|
||||
### Update (Conditional Put)
|
||||
|
||||
```rust
|
||||
let revision: u64 = kv.update("key", "value".into(), last_revision).await?;
|
||||
```
|
||||
|
||||
Uses the `Nats-Expected-Last-Subject-Sequence` header for optimistic concurrency control. Only succeeds if the key's current revision matches.
|
||||
|
||||
### Delete
|
||||
|
||||
```rust
|
||||
kv.delete("key").await?;
|
||||
kv.delete_expect_revision("key", Some(revision)).await?;
|
||||
```
|
||||
|
||||
Non-destructive — publishes a `DEL` marker message. The key appears deleted to `get()`, but history is preserved (up to `history` limit).
|
||||
|
||||
### Purge
|
||||
|
||||
```rust
|
||||
kv.purge("key").await?;
|
||||
kv.purge_with_ttl("key", Duration::from_secs(10)).await?;
|
||||
kv.purge_expect_revision("key", Some(revision)).await?;
|
||||
```
|
||||
|
||||
Destructive — publishes a `PURGE` marker with rollup header, removing all previous revisions of the key. Leaves a single purge entry.
|
||||
|
||||
### Watch
|
||||
|
||||
```rust
|
||||
// Watch for new changes
|
||||
let mut watch = kv.watch("key").await?;
|
||||
// Watch with initial value
|
||||
let mut watch = kv.watch_with_history("key").await?;
|
||||
// Watch from specific revision
|
||||
let mut watch = kv.watch_from_revision("key", 5).await?;
|
||||
// Watch all keys
|
||||
let mut watch = kv.watch_all().await?;
|
||||
// Watch multiple keys (server_2_10+)
|
||||
let mut watch = kv.watch_many(["foo", "bar"]).await?;
|
||||
```
|
||||
|
||||
`Watch` implements `futures_util::Stream<Item = Result<Entry, WatcherError>>`.
|
||||
|
||||
Under the hood, each watch creates an **ordered push consumer** on the KV stream with:
|
||||
- `filter_subject` matching `$KV.<bucket>.<key>`
|
||||
- `replay_policy: Instant`
|
||||
- Appropriate `deliver_policy`
|
||||
|
||||
### History
|
||||
|
||||
```rust
|
||||
let mut history = kv.history("key").await?;
|
||||
```
|
||||
|
||||
Returns a `Stream` of all past `Entry` values for a key (including deletes/purges).
|
||||
|
||||
### Keys
|
||||
|
||||
```rust
|
||||
let mut keys = kv.keys().await?;
|
||||
```
|
||||
|
||||
Returns a `Stream<String>` of all current keys. Uses a headers-only consumer with `LastPerSubject` deliver policy to efficiently scan the bucket.
|
||||
|
||||
## Entry Operations
|
||||
|
||||
```rust
|
||||
#[derive(Debug, Clone, Copy, Eq, PartialEq)]
|
||||
pub enum Operation {
|
||||
Put, // Value was put
|
||||
Delete, // Value was deleted (DEL marker)
|
||||
Purge, // Value was purged (PURGE marker with rollup)
|
||||
}
|
||||
```
|
||||
|
||||
The operation type is determined from the `KV-Operation` header (`PUT`, `DEL`, `PURGE`) or the `Nats-Marker-Reason` header (fallback for server-generated markers like `MaxAge`, `Purge`, `Remove`).
|
||||
|
||||
## Key and Bucket Name Validation
|
||||
|
||||
```rust
|
||||
// Bucket: alphanumeric, dash, underscore only
|
||||
VALID_BUCKET_RE: \A[a-zA-Z0-9_-]+\z
|
||||
|
||||
// Key: alphanumeric, dash, slash, underscore, equals, dot; no leading/trailing dots
|
||||
VALID_KEY_RE: \A[-/_=\.a-zA-Z0-9]+\z
|
||||
```
|
||||
|
||||
## Bucket Status
|
||||
|
||||
```rust
|
||||
let status: Status = kv.status().await?;
|
||||
```
|
||||
|
||||
Wraps stream info to provide bucket-level statistics (bucket name, message count, byte count, etc.).
|
||||
|
||||
## Mirrored Buckets
|
||||
|
||||
When a bucket is configured as a mirror of another (potentially in a different account/domain):
|
||||
|
||||
- `prefix` is set to `$KV.<mirror_bucket>.`
|
||||
- `put_prefix` may be set to the source bucket's API prefix for cross-domain writes
|
||||
- `use_jetstream_prefix` is adjusted based on whether the mirror is in the same domain
|
||||
|
||||
## KV → Stream Config Mapping
|
||||
|
||||
When creating a KV bucket, the `Config` is converted to a JetStream `stream::Config`:
|
||||
|
||||
| KV Config | Stream Config |
|
||||
|-----------|---------------|
|
||||
| `bucket` | `name = "KV_<bucket>"` |
|
||||
| `subjects` | `["$KV.<bucket>.>"]` |
|
||||
| `max_messages_per_subject` | `history` (max 64) |
|
||||
| `max_age` | `max_age` |
|
||||
| `max_bytes` | `max_bytes` |
|
||||
| `storage` | `storage` |
|
||||
| `num_replicas` | `num_replicas` |
|
||||
| `republish` | `republish` |
|
||||
| `mirror` | `mirror` |
|
||||
| `discard` | `DiscardPolicy::New` |
|
||||
| `allow_direct` | `true` |
|
||||
| `allow_rollup_hdrs` | `true |
|
||||
| `max_msg_size` | `max_value_size` |
|
||||
245
docs/research/references/nats.rs/nats-async/07-object-store.md
Normal file
245
docs/research/references/nats.rs/nats-async/07-object-store.md
Normal file
@@ -0,0 +1,245 @@
|
||||
# async-nats: Object Store
|
||||
|
||||
## Overview
|
||||
|
||||
The Object Store provides large-object storage built on JetStream. Objects are chunked and stored as messages in a JetStream stream, with metadata stored separately. The stream is named `OBJ_<bucket_name>`.
|
||||
|
||||
The object-store feature requires `object-store` (which implies `jetstream` + `crypto`).
|
||||
|
||||
## ObjectStore Handle
|
||||
|
||||
```rust
|
||||
#[derive(Clone)]
|
||||
pub struct ObjectStore {
|
||||
pub(crate) name: String,
|
||||
pub(crate) stream: Stream,
|
||||
}
|
||||
```
|
||||
|
||||
## Object Store Config
|
||||
|
||||
```rust
|
||||
#[derive(Debug, Default, Clone, Serialize, Deserialize)]
|
||||
pub struct Config {
|
||||
pub bucket: String,
|
||||
pub description: Option<String>,
|
||||
pub max_age: Duration,
|
||||
pub max_bytes: i64,
|
||||
pub storage: StorageType,
|
||||
pub num_replicas: usize,
|
||||
pub compression: bool,
|
||||
pub placement: Option<Placement>,
|
||||
}
|
||||
```
|
||||
|
||||
## Creating/Accessing Object Stores
|
||||
|
||||
```rust
|
||||
// Create
|
||||
let bucket = jetstream.create_object_store(object_store::Config {
|
||||
bucket: "my-bucket".to_string(),
|
||||
..Default::default()
|
||||
}).await?;
|
||||
|
||||
// Get existing
|
||||
let bucket = jetstream.get_object_store("my-bucket").await?;
|
||||
|
||||
// Delete
|
||||
jetstream.delete_object_store("my-bucket").await?;
|
||||
```
|
||||
|
||||
## Object Store Operations
|
||||
|
||||
### Put
|
||||
|
||||
```rust
|
||||
let info: ObjectInfo = bucket.put("file.txt", &mut async_read).await?;
|
||||
```
|
||||
|
||||
The put operation:
|
||||
1. Reads data from any `AsyncRead + Unpin` source in chunks (default 128KB)
|
||||
2. Each chunk is published to `$O.<bucket>.C.<nuid>` (chunk subject)
|
||||
3. SHA-256 digest is computed incrementally
|
||||
4. After all chunks, metadata is published to `$O.<bucket>.M.<encoded_name>` with a rollup header
|
||||
5. If the object previously existed, old chunks are purged
|
||||
|
||||
### Get
|
||||
|
||||
```rust
|
||||
let mut object: Object = bucket.get("file.txt").await?;
|
||||
```
|
||||
|
||||
Returns an `Object` that implements `tokio::io::AsyncRead`:
|
||||
|
||||
```rust
|
||||
let mut bytes = Vec::new();
|
||||
object.read_to_end(&mut bytes).await?;
|
||||
```
|
||||
|
||||
On read, the Object:
|
||||
1. Creates an ordered push consumer on `$O.<bucket>.C.<nuid>`
|
||||
2. Streams chunk messages, feeding bytes to the reader
|
||||
3. Verifies SHA-256 digest after the last chunk
|
||||
4. If digest doesn't match, returns `io::ErrorKind::InvalidData`
|
||||
|
||||
### Delete
|
||||
|
||||
```rust
|
||||
bucket.delete("file.txt").await?;
|
||||
```
|
||||
|
||||
Marks the object as deleted in metadata (sets `deleted = true`, `chunks = 0`, `size = 0`) with a rollup, then purges all chunk messages.
|
||||
|
||||
### Info
|
||||
|
||||
```rust
|
||||
let info: ObjectInfo = bucket.info("file.txt").await?;
|
||||
```
|
||||
|
||||
Fetches the last metadata message for the object (from `$O.<bucket>.M.<encoded_name>`).
|
||||
|
||||
### Watch
|
||||
|
||||
```rust
|
||||
let mut watcher = bucket.watch().await?;
|
||||
let mut watcher = bucket.watch_with_history().await?;
|
||||
```
|
||||
|
||||
Returns a `Stream<Item = Result<ObjectInfo, WatcherError>>`. Uses an ordered push consumer on `$O.<bucket>.M.>`.
|
||||
|
||||
### List
|
||||
|
||||
```rust
|
||||
let mut list = bucket.list().await?;
|
||||
```
|
||||
|
||||
Returns a `Stream<Item = Result<ObjectInfo, ListerError>>`. Lists all non-deleted objects. Uses `DeliverPolicy::All` to replay all metadata.
|
||||
|
||||
### Seal
|
||||
|
||||
```rust
|
||||
bucket.seal().await?;
|
||||
```
|
||||
|
||||
Sets the underlying stream's `sealed = true`, preventing any further modifications.
|
||||
|
||||
### Links
|
||||
|
||||
```rust
|
||||
// Link to another object (same or different bucket)
|
||||
let info = bucket.add_link("link_name", &object).await?;
|
||||
|
||||
// Link to another bucket
|
||||
let info = bucket.add_bucket_link("link_name", "other_bucket").await?;
|
||||
```
|
||||
|
||||
Links are followed automatically when `get()` is called (one level deep). Cannot link to a deleted object or create a link to a link.
|
||||
|
||||
### Update Metadata
|
||||
|
||||
```rust
|
||||
bucket.update_metadata("object", object_store::UpdateMetadata {
|
||||
name: "new_name".to_string(),
|
||||
description: Some("updated description".to_string()),
|
||||
..Default::default()
|
||||
}).await?;
|
||||
```
|
||||
|
||||
If the name changes, old metadata is purged and new metadata is published.
|
||||
|
||||
## Object Types
|
||||
|
||||
### ObjectInfo
|
||||
|
||||
```rust
|
||||
#[derive(Debug, Clone, Serialize, Deserialize, Eq, PartialEq)]
|
||||
pub struct ObjectInfo {
|
||||
pub name: String,
|
||||
pub description: Option<String>,
|
||||
pub metadata: HashMap<String, String>,
|
||||
pub headers: Option<HeaderMap>,
|
||||
pub options: Option<ObjectOptions>,
|
||||
pub bucket: String,
|
||||
pub nuid: String,
|
||||
pub size: usize,
|
||||
pub chunks: usize,
|
||||
pub modified: Option<DateTime>,
|
||||
pub digest: Option<String>, // Format: "SHA-256=<base64url-digest>"
|
||||
pub deleted: bool,
|
||||
}
|
||||
```
|
||||
|
||||
### ObjectMetadata
|
||||
|
||||
```rust
|
||||
#[derive(Debug, Default, Clone, Serialize, Deserialize, Eq, PartialEq)]
|
||||
pub struct ObjectMetadata {
|
||||
pub name: String,
|
||||
pub description: Option<String>,
|
||||
pub chunk_size: Option<usize>,
|
||||
pub metadata: HashMap<String, String>,
|
||||
pub headers: Option<HeaderMap>,
|
||||
}
|
||||
```
|
||||
|
||||
### ObjectLink
|
||||
|
||||
```rust
|
||||
#[derive(Debug, Default, Clone, Serialize, Deserialize, Eq, PartialEq)]
|
||||
pub struct ObjectLink {
|
||||
pub name: Option<String>, // None = bucket link, Some = object link
|
||||
pub bucket: String,
|
||||
}
|
||||
```
|
||||
|
||||
### Object
|
||||
|
||||
```rust
|
||||
pub struct Object {
|
||||
pub info: ObjectInfo,
|
||||
remaining_bytes: VecDeque<u8>,
|
||||
has_pending_messages: bool,
|
||||
digest: Option<Sha256>,
|
||||
subscription: Option<crate::jetstream::consumer::push::Ordered>,
|
||||
subscription_future: Option<BoxFuture<'static, Result<Ordered, StreamError>>>,
|
||||
stream: Stream,
|
||||
}
|
||||
```
|
||||
|
||||
Implements `tokio::io::AsyncRead`. Lazy-creates the consumer on first read.
|
||||
|
||||
## Subject Naming Convention
|
||||
|
||||
| Purpose | Subject Pattern |
|
||||
|---------|----------------|
|
||||
| Chunks | `$O.<bucket>.C.<nuid>` |
|
||||
| Metadata | `$O.<bucket>.M.<base64url-encoded-name>` |
|
||||
|
||||
Object names are base64url-encoded in metadata subjects to allow arbitrary characters (the raw name might contain characters invalid in NATS subjects).
|
||||
|
||||
## Validation
|
||||
|
||||
```rust
|
||||
// Bucket: alphanumeric, dash, underscore only
|
||||
BUCKET_NAME_RE: \A[a-zA-Z0-9_-]+\z
|
||||
|
||||
// Object name: alphanumeric, dash, slash, underscore, equals, dot; no leading/trailing dots
|
||||
OBJECT_NAME_RE: \A[-/_=\.a-zA-Z0-9]+\z
|
||||
```
|
||||
|
||||
## Data Integrity
|
||||
|
||||
The object store uses SHA-256 hashing (from the `crypto` module) to verify data integrity:
|
||||
|
||||
1. On `put()`: SHA-256 is computed incrementally as chunks are read. The digest is stored in `ObjectInfo.digest` as `"SHA-256=<base64url>"`.
|
||||
2. On `get()` (via `AsyncRead`): SHA-256 is verified after the last chunk is read. If the computed digest doesn't match the stored digest, `io::ErrorKind::InvalidData` is returned.
|
||||
|
||||
```rust
|
||||
// crypto module
|
||||
pub(crate) struct Sha256 { ... }
|
||||
impl Sha256 {
|
||||
pub fn new() -> Self;
|
||||
pub fn update(&mut self, data: &[u8]);
|
||||
pub fn finish(self) -> [u8; 32];
|
||||
}
|
||||
```
|
||||
272
docs/research/references/nats.rs/nats-async/08-service-api.md
Normal file
272
docs/research/references/nats.rs/nats-async/08-service-api.md
Normal file
@@ -0,0 +1,272 @@
|
||||
# async-nats: Service API
|
||||
|
||||
## Overview
|
||||
|
||||
The Service API provides a microservice request/reply pattern with built-in service discovery, health checking, and statistics. It follows the [NATS Micro v1 specification](https://github.com/nats-io/nats-architecture-design/blob/main/adr/ADR-33.md).
|
||||
|
||||
The `service` feature is required.
|
||||
|
||||
## Service
|
||||
|
||||
```rust
|
||||
#[derive(Debug)]
|
||||
pub struct Service {
|
||||
endpoints_state: Arc<Mutex<Endpoints>>,
|
||||
info: Info,
|
||||
client: Client,
|
||||
handle: JoinHandle<Result<(), Error>>,
|
||||
shutdown_tx: Sender<()>,
|
||||
subjects: Arc<Mutex<Vec<String>>>,
|
||||
queue_group: String,
|
||||
}
|
||||
```
|
||||
|
||||
## Creating a Service
|
||||
|
||||
Via the `ServiceExt` trait on `Client`:
|
||||
|
||||
```rust
|
||||
use async_nats::service::ServiceExt;
|
||||
|
||||
// Builder pattern
|
||||
let mut service = client
|
||||
.service_builder()
|
||||
.description("product service")
|
||||
.stats_handler(|endpoint, stats| serde_json::json!({ "endpoint": endpoint }))
|
||||
.metadata(HashMap::from([("version".into(), "v2".into())]))
|
||||
.queue_group("products-group")
|
||||
.start("products", "1.0.0")
|
||||
.await?;
|
||||
|
||||
// Direct config
|
||||
let mut service = client
|
||||
.add_service(service::Config {
|
||||
name: "products".to_string(),
|
||||
version: "1.0.0".to_string(),
|
||||
description: Some("product service".to_string()),
|
||||
stats_handler: None,
|
||||
metadata: None,
|
||||
queue_group: None,
|
||||
})
|
||||
.await?;
|
||||
```
|
||||
|
||||
Service name must match `^[A-Za-z0-9\-_]+$`. Version must be valid SemVer.
|
||||
|
||||
## Service Verbs
|
||||
|
||||
Every service automatically subscribes to three verb subjects for discovery and monitoring:
|
||||
|
||||
| Verb | Subject Pattern | Purpose |
|
||||
|------|----------------|---------|
|
||||
| PING | `$SRV.PING`, `$SRV.PING.<name>`, `$SRV.PING.<name>.<id>` | Lightweight health check |
|
||||
| INFO | `$SRV.INFO.<name>`, `$SRV.INFO.<name>.<id>` | Service metadata |
|
||||
| STATS | `$SRV.STATS.<name>`, `$SRV.STATS.<name>.<id>` | Service + endpoint statistics |
|
||||
|
||||
A background task handles these verb requests and responds with JSON payloads.
|
||||
|
||||
## Service Config
|
||||
|
||||
```rust
|
||||
#[derive(Serialize, Deserialize, Debug)]
|
||||
pub struct Config {
|
||||
pub name: String,
|
||||
pub description: Option<String>,
|
||||
pub version: String,
|
||||
pub stats_handler: Option<StatsHandler>,
|
||||
pub metadata: Option<HashMap<String, String>>,
|
||||
pub queue_group: Option<String>,
|
||||
}
|
||||
```
|
||||
|
||||
## Adding Endpoints
|
||||
|
||||
```rust
|
||||
// Simple endpoint
|
||||
let mut endpoint = service.endpoint("get-products").await?;
|
||||
|
||||
// Endpoint with custom name and metadata
|
||||
let endpoint = service
|
||||
.endpoint_builder()
|
||||
.name("api")
|
||||
.metadata(HashMap::from([("auth".into(), "required".into())]))
|
||||
.queue_group("custom-group")
|
||||
.add("products")
|
||||
.await?;
|
||||
|
||||
// Grouped endpoints
|
||||
let v1 = service.group("v1");
|
||||
let products = v1.endpoint("products").await?;
|
||||
let orders = v1.endpoint("orders").await?;
|
||||
|
||||
// Nested groups
|
||||
let v1_api = service.group("api").group("v1");
|
||||
```
|
||||
|
||||
## Endpoint
|
||||
|
||||
```rust
|
||||
pub struct Endpoint {
|
||||
requests: Subscriber,
|
||||
stats: Arc<Mutex<Endpoints>>,
|
||||
client: Client,
|
||||
endpoint: String,
|
||||
shutdown: Option<ShutdownRx>,
|
||||
shutdown_future: Option<ShutdownReceiverFuture>,
|
||||
}
|
||||
```
|
||||
|
||||
Implements `futures_util::Stream<Item = Request>`.
|
||||
|
||||
```rust
|
||||
while let Some(request) = endpoint.next().await {
|
||||
request.respond(Ok("response data".into())).await?;
|
||||
}
|
||||
```
|
||||
|
||||
## Service Request
|
||||
|
||||
```rust
|
||||
#[derive(Debug)]
|
||||
pub struct Request {
|
||||
issued: Instant,
|
||||
client: Client,
|
||||
pub message: Message,
|
||||
endpoint: String,
|
||||
stats: Arc<Mutex<Endpoints>>,
|
||||
}
|
||||
```
|
||||
|
||||
### Responding
|
||||
|
||||
```rust
|
||||
// Success
|
||||
request.respond(Ok("result".into())).await?;
|
||||
|
||||
// Success with headers
|
||||
request.respond_with_headers(Ok("result".into()), headers).await?;
|
||||
|
||||
// Error
|
||||
request.respond(Err(service::error::Error {
|
||||
code: 500,
|
||||
status: "internal error".to_string(),
|
||||
})).await?;
|
||||
```
|
||||
|
||||
Error responses always include `Nats-Service-Error` and `Nats-Service-Error-Code` headers. If user-supplied headers contain these headers, they are overridden by the error values.
|
||||
|
||||
### Stats Tracking
|
||||
|
||||
Each response updates endpoint statistics:
|
||||
- `requests` — total requests
|
||||
- `processing_time` — cumulative processing time
|
||||
- `average_processing_time` — average per request
|
||||
- `errors` — error count
|
||||
- `last_error` — last error details
|
||||
|
||||
## Service Info Types
|
||||
|
||||
### PingResponse
|
||||
|
||||
```rust
|
||||
pub struct PingResponse {
|
||||
pub kind: String, // "io.nats.micro.v1.ping_response"
|
||||
pub name: String,
|
||||
pub id: String,
|
||||
pub version: String,
|
||||
pub metadata: HashMap<String, String>,
|
||||
}
|
||||
```
|
||||
|
||||
### Info
|
||||
|
||||
```rust
|
||||
pub struct Info {
|
||||
pub kind: String, // "io.nats.micro.v1.info_response"
|
||||
pub name: String,
|
||||
pub id: String,
|
||||
pub description: String,
|
||||
pub version: String,
|
||||
pub metadata: HashMap<String, String>,
|
||||
pub endpoints: Vec<endpoint::Info>,
|
||||
}
|
||||
```
|
||||
|
||||
### Stats
|
||||
|
||||
```rust
|
||||
pub struct Stats {
|
||||
pub kind: String, // "io.nats.micro.v1.stats_response"
|
||||
pub name: String,
|
||||
pub id: String,
|
||||
pub version: String,
|
||||
pub started: DateTime,
|
||||
pub endpoints: Vec<endpoint::Stats>,
|
||||
}
|
||||
```
|
||||
|
||||
### Endpoint Stats
|
||||
|
||||
```rust
|
||||
pub struct endpoint::Stats {
|
||||
pub name: String,
|
||||
pub subject: String,
|
||||
pub queue_group: String,
|
||||
pub data: Option<serde_json::Value>, // Custom data from stats_handler
|
||||
pub errors: u64,
|
||||
pub processing_time: Duration,
|
||||
pub average_processing_time: Duration,
|
||||
pub requests: u64,
|
||||
pub last_error: Option<error::Error>,
|
||||
}
|
||||
```
|
||||
|
||||
## Service Groups
|
||||
|
||||
Groups provide subject prefixing for endpoint organization:
|
||||
|
||||
```rust
|
||||
let service = client.service_builder().start("api", "1.0.0").await?;
|
||||
|
||||
// Endpoints subscribe to "products" and "orders"
|
||||
let products = service.endpoint("products").await?;
|
||||
let orders = service.endpoint("orders").await?;
|
||||
|
||||
// Grouped: subscribe to "v1.products" and "v1.orders"
|
||||
let v1 = service.group("v1");
|
||||
let products = v1.endpoint("products").await?;
|
||||
let orders = v1.endpoint("orders").await?;
|
||||
|
||||
// Nested: subscribe to "api.v1.products"
|
||||
let api_v1 = service.group("api").group("v1");
|
||||
let products = api_v1.endpoint("products").await?;
|
||||
```
|
||||
|
||||
Each group can have its own queue group:
|
||||
|
||||
```rust
|
||||
let v1 = service.group_with_queue_group("v1", "v1-workers");
|
||||
```
|
||||
|
||||
## Stopping a Service
|
||||
|
||||
```rust
|
||||
service.stop().await?;
|
||||
```
|
||||
|
||||
Sends a shutdown signal and aborts the verb-handling task. Other service instances with the same name continue running.
|
||||
|
||||
## Resetting Stats
|
||||
|
||||
```rust
|
||||
service.reset().await?;
|
||||
```
|
||||
|
||||
Resets all endpoint statistics (errors, processing time, requests, average processing time) to zero.
|
||||
|
||||
## Querying Service State
|
||||
|
||||
```rust
|
||||
let stats: HashMap<String, endpoint::Stats> = service.stats().await?;
|
||||
let info: Info = service.info().await?;
|
||||
```
|
||||
@@ -0,0 +1,312 @@
|
||||
# async-nats: Quick Reference
|
||||
|
||||
## Connection
|
||||
|
||||
```rust
|
||||
// Basic connect
|
||||
let client = async_nats::connect("demo.nats.io").await?;
|
||||
|
||||
// With options
|
||||
let client = async_nats::ConnectOptions::new()
|
||||
.require_tls(true)
|
||||
.name("my-service")
|
||||
.ping_interval(Duration::from_secs(10))
|
||||
.request_timeout(Some(Duration::from_secs(5)))
|
||||
.connect("demo.nats.io")
|
||||
.await?;
|
||||
|
||||
// Multiple servers
|
||||
let client = async_nats::connect(vec![
|
||||
"nats://server1:4222".parse()?,
|
||||
"nats://server2:4222".parse()?,
|
||||
]).await?;
|
||||
|
||||
// Background connect
|
||||
let client = async_nats::ConnectOptions::new()
|
||||
.retry_on_initial_connect()
|
||||
.connect("demo.nats.io")
|
||||
.await?;
|
||||
```
|
||||
|
||||
## Core NATS: Publish
|
||||
|
||||
```rust
|
||||
// Simple publish
|
||||
client.publish("subject", "payload".into()).await?;
|
||||
|
||||
// With reply-to
|
||||
client.publish_with_reply("subject", "reply-to", "payload".into()).await?;
|
||||
|
||||
// With headers
|
||||
let mut headers = HeaderMap::new();
|
||||
headers.insert("X-Custom", "value");
|
||||
client.publish_with_headers("subject", headers, "payload".into()).await?;
|
||||
|
||||
// Full control
|
||||
client.publish_with_reply_and_headers("subject", "reply-to", headers, "payload".into()).await?;
|
||||
|
||||
// Flush (ensure all published messages are sent)
|
||||
client.flush().await?;
|
||||
```
|
||||
|
||||
## Core NATS: Subscribe
|
||||
|
||||
```rust
|
||||
use futures_util::StreamExt;
|
||||
|
||||
// Basic subscribe
|
||||
let mut subscriber = client.subscribe("subject").await?;
|
||||
|
||||
// Queue group
|
||||
let mut subscriber = client.queue_subscribe("subject", "group".into()).await?;
|
||||
|
||||
// Receive messages (Subscriber implements Stream)
|
||||
while let Some(message) = subscriber.next().await {
|
||||
println!("subject: {}, payload: {:?}", message.subject, message.payload);
|
||||
}
|
||||
|
||||
// Unsubscribe
|
||||
subscriber.unsubscribe().await?;
|
||||
|
||||
// Unsubscribe after N messages
|
||||
subscriber.unsubscribe_after(10).await?;
|
||||
|
||||
// Drain (wait for in-flight, then unsubscribe)
|
||||
subscriber.drain().await?;
|
||||
```
|
||||
|
||||
## Core NATS: Request/Reply
|
||||
|
||||
```rust
|
||||
// Simple request (uses default timeout)
|
||||
let response = client.request("subject", "data".into()).await?;
|
||||
|
||||
// With custom timeout and headers
|
||||
let request = async_nats::Request::new()
|
||||
.payload("data".into())
|
||||
.timeout(Some(Duration::from_secs(5)))
|
||||
.headers(headers);
|
||||
let response = client.send_request("subject", request).await?;
|
||||
|
||||
// Custom inbox (bypasses multiplexer)
|
||||
let request = async_nats::Request::new()
|
||||
.payload("data".into())
|
||||
.inbox("custom-inbox".into());
|
||||
let response = client.send_request("subject", request).await?;
|
||||
```
|
||||
|
||||
## Message Structure
|
||||
|
||||
```rust
|
||||
pub struct Message {
|
||||
pub subject: Subject,
|
||||
pub reply: Option<Subject>,
|
||||
pub payload: Bytes,
|
||||
pub headers: Option<HeaderMap>,
|
||||
pub status: Option<StatusCode>,
|
||||
pub description: Option<String>,
|
||||
pub length: usize,
|
||||
}
|
||||
```
|
||||
|
||||
## JetStream
|
||||
|
||||
```rust
|
||||
let jetstream = async_nats::jetstream::new(client);
|
||||
|
||||
// Publish (returns ack future)
|
||||
let ack = jetstream.publish("events", "data".into()).await?;
|
||||
let publish_ack = ack.await?;
|
||||
|
||||
// Stream management
|
||||
let stream = jetstream.create_stream(stream::Config {
|
||||
name: "events".to_string(),
|
||||
subjects: vec!["events.>".to_string()],
|
||||
max_messages: 10_000,
|
||||
..Default::default()
|
||||
}).await?;
|
||||
|
||||
let stream = jetstream.get_stream("events").await?;
|
||||
let stream = jetstream.get_or_create_stream(config).await?;
|
||||
jetstream.delete_stream("events").await?;
|
||||
jetstream.update_stream(config).await?;
|
||||
|
||||
// Consumer management
|
||||
let consumer: PullConsumer = stream.create_consumer(pull::Config {
|
||||
durable_name: Some("my-consumer".to_string()),
|
||||
..Default::default()
|
||||
}).await?;
|
||||
|
||||
// Pull consumer: fetch messages
|
||||
let mut messages = consumer.messages().await?;
|
||||
while let Some(message) = messages.next().await {
|
||||
let message = message?;
|
||||
message.ack().await?;
|
||||
}
|
||||
|
||||
// Push consumer (ordered)
|
||||
let consumer = stream.create_consumer(push::OrderedConfig {
|
||||
deliver_subject: client.new_inbox(),
|
||||
filter_subject: "events.>".to_string(),
|
||||
..Default::default()
|
||||
}).await?;
|
||||
let mut messages = consumer.messages().await?;
|
||||
```
|
||||
|
||||
## Key-Value Store
|
||||
|
||||
```rust
|
||||
let kv = jetstream.create_key_value(kv::Config {
|
||||
bucket: "my-bucket".to_string(),
|
||||
history: 10,
|
||||
..Default::default()
|
||||
}).await?;
|
||||
|
||||
// CRUD
|
||||
let revision = kv.put("key", "value".into()).await?;
|
||||
let revision = kv.create("key", "value".into()).await?;
|
||||
let value: Option<Bytes> = kv.get("key").await?;
|
||||
let entry: Option<Entry> = kv.entry("key").await?;
|
||||
let revision = kv.update("key", "new-value".into(), revision).await?;
|
||||
kv.delete("key").await?;
|
||||
kv.purge("key").await?;
|
||||
|
||||
// Watch
|
||||
let mut watch = kv.watch("key").await?;
|
||||
let mut watch_all = kv.watch_all().await?;
|
||||
|
||||
// History & Keys
|
||||
let mut history = kv.history("key").await?;
|
||||
let mut keys = kv.keys().await?;
|
||||
```
|
||||
|
||||
## Object Store
|
||||
|
||||
```rust
|
||||
let bucket = jetstream.create_object_store(object_store::Config {
|
||||
bucket: "files".to_string(),
|
||||
..Default::default()
|
||||
}).await?;
|
||||
|
||||
// Put (from any AsyncRead)
|
||||
let info = bucket.put("file.txt", &mut file).await?;
|
||||
|
||||
// Get (returns AsyncRead)
|
||||
let mut object = bucket.get("file.txt").await?;
|
||||
let mut bytes = Vec::new();
|
||||
object.read_to_end(&mut bytes).await?;
|
||||
|
||||
// Info, delete, list, watch
|
||||
let info = bucket.info("file.txt").await?;
|
||||
bucket.delete("file.txt").await?;
|
||||
let mut list = bucket.list().await?;
|
||||
let mut watch = bucket.watch().await?;
|
||||
```
|
||||
|
||||
## Service API
|
||||
|
||||
```rust
|
||||
use async_nats::service::ServiceExt;
|
||||
use futures_util::StreamExt;
|
||||
|
||||
let mut service = client
|
||||
.service_builder()
|
||||
.description("product service")
|
||||
.start("products", "1.0.0")
|
||||
.await?;
|
||||
|
||||
let mut endpoint = service.endpoint("get").await?;
|
||||
|
||||
while let Some(request) = endpoint.next().await {
|
||||
request.respond(Ok("result".into())).await?;
|
||||
}
|
||||
```
|
||||
|
||||
## Client State & Events
|
||||
|
||||
```rust
|
||||
// Check connection state
|
||||
match client.connection_state() {
|
||||
State::Connected => {},
|
||||
State::Disconnected => {},
|
||||
State::Pending => {},
|
||||
}
|
||||
|
||||
// Get server info
|
||||
let info: ServerInfo = client.server_info();
|
||||
println!("max_payload: {}", info.max_payload);
|
||||
println!("jetstream: {}", info.jetstream);
|
||||
|
||||
// Get statistics
|
||||
let stats = client.statistics();
|
||||
println!("in_messages: {}", stats.in_messages.load(Ordering::Relaxed));
|
||||
|
||||
// Force reconnect
|
||||
client.force_reconnect().await?;
|
||||
|
||||
// Server pool management
|
||||
client.set_server_pool(["nats://s1:4222".parse()?, "nats://s2:4222".parse()?].as_slice()).await?;
|
||||
let pool = client.server_pool().await?;
|
||||
|
||||
// Drain
|
||||
client.drain().await?;
|
||||
```
|
||||
|
||||
## Error Handling Patterns
|
||||
|
||||
```rust
|
||||
// Connect errors
|
||||
match async_nats::connect("server").await {
|
||||
Err(e) => match e.kind() {
|
||||
ConnectErrorKind::TimedOut => {},
|
||||
ConnectErrorKind::Authentication => {},
|
||||
ConnectErrorKind::AuthorizationViolation => {},
|
||||
_ => {},
|
||||
},
|
||||
Ok(client) => {},
|
||||
}
|
||||
|
||||
// Publish errors
|
||||
match client.publish("subject", "data".into()).await {
|
||||
Err(e) => match e.kind() {
|
||||
PublishErrorKind::MaxPayloadExceeded => {},
|
||||
PublishErrorKind::InvalidSubject => {},
|
||||
PublishErrorKind::Send => {},
|
||||
_ => {},
|
||||
},
|
||||
_ => {},
|
||||
}
|
||||
|
||||
// Request errors
|
||||
match client.request("subject", "data".into()).await {
|
||||
Err(e) => match e.kind() {
|
||||
RequestErrorKind::TimedOut => {},
|
||||
RequestErrorKind::NoResponders => {},
|
||||
RequestErrorKind::InvalidSubject => {},
|
||||
RequestErrorKind::MaxPayloadExceeded => {},
|
||||
_ => {},
|
||||
},
|
||||
Ok(message) => {},
|
||||
}
|
||||
```
|
||||
|
||||
## Feature Flag Quick Reference
|
||||
|
||||
| Feature | Enables | Default |
|
||||
|---------|---------|---------|
|
||||
| `jetstream` | JetStream streams, consumers, publish | ✅ |
|
||||
| `kv` | Key-Value store (implies `jetstream`) | ✅ |
|
||||
| `object-store` | Object store (implies `jetstream` + `crypto`) | ✅ |
|
||||
| `service` | Service API | ✅ |
|
||||
| `nkeys` | NKey/JWT authentication | ✅ |
|
||||
| `nuid` | NUID-based ID generation | ✅ |
|
||||
| `crypto` | SHA-256 (for object store) | ✅ |
|
||||
| `websockets` | WebSocket transport | ✅ |
|
||||
| `ring` | `ring` TLS crypto backend | ✅ |
|
||||
| `aws-lc-rs` | `aws-lc-rs` TLS crypto backend | ❌ |
|
||||
| `fips` | FIPS mode via `aws-lc-rs` | ❌ |
|
||||
| `chrono` | `chrono` datetime instead of `time` | ❌ |
|
||||
| `server_2_10` | Server 2.10+ features | ✅ |
|
||||
| `server_2_11` | Server 2.11+ features | ✅ |
|
||||
| `server_2_12` | Server 2.12+ features | ✅ |
|
||||
| `server_2_14` | Server 2.14+ features | ✅ |
|
||||
23
docs/research/references/nats.rs/nats-async/README.md
Normal file
23
docs/research/references/nats.rs/nats-async/README.md
Normal file
@@ -0,0 +1,23 @@
|
||||
# async-nats Reference Documentation
|
||||
|
||||
**Crate**: `async-nats` v0.49.1
|
||||
**Source**: https://github.com/nats-io/nats.rs (`async-nats/` directory)
|
||||
**License**: Apache-2.0
|
||||
|
||||
## Contents
|
||||
|
||||
| # | File | Topic |
|
||||
|---|------|-------|
|
||||
| 01 | [Overview & Architecture](01-overview-and-architecture.md) | Crate overview, feature flags, source structure, core connection model, dependency graph |
|
||||
| 02 | [Key Types & Traits](02-key-types-and-traits.md) | `Client`, `Subscriber`, `Message`, `Request`, `ServerInfo`, `ConnectInfo`, `Statistics`, subject/header types, event/state types, error types, trait definitions |
|
||||
| 03 | [Protocol & Wire Format](03-protocol-and-wire-format.md) | NATS wire protocol (PUB/HPUB/SUB/UNSUB/PING/PONG, MSG/HMSG/INFO/ERR), parser/serializer internals, vectored I/O, WebSocket transport, connection lifecycle, reconnection |
|
||||
| 04 | [Connection Management](04-connection-management.md) | `ConnectOptions` builder, authentication methods, TLS configuration, reconnection callbacks, event callbacks, `ConnectionHandler` internals, multiplexer, server pool management |
|
||||
| 05 | [JetStream](05-jetstream.md) | `Context` and `ContextBuilder`, streams, consumers (pull/push/ordered), JetStream messages and acks, publish with ack futures, pagination |
|
||||
| 06 | [Key-Value Store](06-key-value-store.md) | KV `Store` handle, bucket CRUD, put/get/create/update/delete/purge, watch/history/keys, entry operations, mirrored buckets, KV-to-stream mapping |
|
||||
| 07 | [Object Store](07-object-store.md) | `ObjectStore` handle, put/get/delete/watch/list/seal, links, `Object` (AsyncRead), chunking, SHA-256 integrity, subject naming |
|
||||
| 08 | [Service API](08-service-api.md) | `Service` and `ServiceBuilder`, endpoints, groups, verb subscriptions (PING/INFO/STATS), request/respond with stats tracking |
|
||||
| 09 | [Quick Reference](09-quick-reference.md) | Code examples for all major operations, feature flag reference |
|
||||
|
||||
## How This Documentation Was Produced
|
||||
|
||||
All information was derived by reading the source code of the `async-nats` crate at version 0.49.1 from the `nats.rs` repository. No external documentation was consulted — this is a ground-up reference based purely on the source.
|
||||
@@ -0,0 +1,200 @@
|
||||
# nats.rs: Overview and Architecture
|
||||
|
||||
**Version**: async-nats 0.49.1, nats-server 0.1.0
|
||||
**Repository**: https://github.com/nats-io/nats.rs
|
||||
**License**: Apache-2.0
|
||||
**Rust Edition**: 2021
|
||||
**MSRV**: 1.88.0
|
||||
**Protocol**: NATS Client Protocol (INFO/CONNECT/PUB/SUB/UNSUB/PING/PONG)
|
||||
|
||||
## What It Is
|
||||
|
||||
The `nats.rs` repository contains the **official Rust client for NATS.io**, a high-performance messaging system. The active crate is **`async-nats`** — a fully async, Tokio-based NATS client. The deprecated `nats` crate (synchronous) receives security fixes only.
|
||||
|
||||
The `nats-server` crate is **not** an implementation of the NATS server. It is a **test harness** that spawns the Go-based `nats-server` binary for integration tests. The actual NATS server is a separate Go project at `github.com/nats-io/nats-server`.
|
||||
|
||||
Core design decisions:
|
||||
- **Fully async** — all I/O is Tokio-based with async/await throughout
|
||||
- **Cloneable Client handle** — `Client` is cheap to clone (Arc internals), all protocol work happens in a single `ConnectionHandler` task
|
||||
- **Channel-based internal communication** — `Client` sends `Command` variants via `mpsc` channel to `ConnectionHandler`
|
||||
- **Multiplexed request-reply** — one internal subscription handles all request-response patterns via inbox token routing
|
||||
- **Automatic reconnection** — exponential backoff with configurable server pool rotation
|
||||
- **Feature-gated subsystems** — JetStream, KV, Object Store, Service API, NKeys, WebSockets, and crypto backends are all optional
|
||||
|
||||
## Workspace Structure
|
||||
|
||||
```
|
||||
nats.rs/
|
||||
├── async-nats/ # Primary crate — async NATS client
|
||||
│ ├── src/
|
||||
│ │ ├── lib.rs # Entry point: connect(), ServerOp, ClientOp, Command, ConnectionHandler, Subscriber
|
||||
│ │ ├── client.rs # Client handle: publish, subscribe, request, flush, drain
|
||||
│ │ ├── connection.rs # Low-level I/O: protocol parsing, read/write buffers
|
||||
│ │ ├── connector.rs # Connection establishment, reconnection, server pool
|
||||
│ │ ├── options.rs # ConnectOptions builder
|
||||
│ │ ├── auth.rs # Auth struct (credentials container)
|
||||
│ │ ├── auth_utils.rs # Credential file parsing (.creds files)
|
||||
│ │ ├── error.rs # Generic Error<Kind> type
|
||||
│ │ ├── header.rs # HeaderMap — NATS message headers
|
||||
│ │ ├── subject.rs # Subject type, ToSubject trait
|
||||
│ │ ├── status.rs # StatusCode (100-999 NATS protocol codes)
|
||||
│ │ ├── message.rs # Message and OutboundMessage types
|
||||
│ │ ├── tls.rs # TLS configuration helpers
|
||||
│ │ ├── crypto.rs # Crypto feature support
|
||||
│ │ ├── id_generator.rs # NUID/rand-based unique ID generation
|
||||
│ │ ├── datetime.rs # DateTime helpers for JetStream/Service
|
||||
│ │ ├── jetstream/ # JetStream API (feature-gated)
|
||||
│ │ │ ├── mod.rs # Module root, jetstream::new(), with_domain()
|
||||
│ │ │ ├── context.rs # JetStream Context — streams, publishing, consumers
|
||||
│ │ │ ├── stream.rs # Stream management, Config, Info, Consumer creation
|
||||
│ │ │ ├── consumer/ # Pull, Push, Ordered consumers
|
||||
│ │ │ ├── message.rs # JetStream Message with ack methods
|
||||
│ │ │ ├── publish.rs # PublishAck
|
||||
│ │ │ ├── response.rs # Response wrapper
|
||||
│ │ │ ├── errors.rs # JetStream error codes
|
||||
│ │ │ ├── account.rs # Account info
|
||||
│ │ │ ├── kv/ # Key-Value store (feature: "kv")
|
||||
│ │ │ └── object_store/ # Object store (feature: "object-store")
|
||||
│ │ └── service/ # Service API (feature-gated)
|
||||
│ │ ├── mod.rs # Service, ServiceBuilder
|
||||
│ │ ├── endpoint.rs # Endpoint handling
|
||||
│ │ └── error.rs # Service errors
|
||||
│ ├── tests/ # Integration tests (require nats-server binary)
|
||||
│ ├── examples/ # Runnable examples
|
||||
│ └── benches/ # Criterion benchmarks
|
||||
├── nats-server/ # Test harness — spawns Go nats-server for tests
|
||||
│ ├── src/lib.rs # Server struct, run_server(), run_cluster()
|
||||
│ └── configs/ # Server config files for tests
|
||||
│ └── jetstream.conf
|
||||
└── nats/ # DEPRECATED sync client — do not modify
|
||||
```
|
||||
|
||||
## Architecture Diagram
|
||||
|
||||
```
|
||||
┌──────────────────────────────────────────────────────────┐
|
||||
│ Application Layer │
|
||||
│ │
|
||||
│ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐ │
|
||||
│ │ JetStream│ │ KV │ │ Object │ │ Service │ │
|
||||
│ │ Context │ │ Store │ │ Store │ │ API │ │
|
||||
│ └────┬─────┘ └────┬─────┘ └────┬─────┘ └────┬─────┘ │
|
||||
│ └──────────────┴─────────────┴─────────────┘ │
|
||||
│ │ │
|
||||
│ ┌──────┴──────┐ │
|
||||
│ │ Client │ Cloneable handle │
|
||||
│ │ (mpsc::Sender) │
|
||||
│ └──────┬──────┘ │
|
||||
│ │ Command channel │
|
||||
└──────────────────────────┼────────────────────────────────┘
|
||||
│
|
||||
┌──────────────────────────┼────────────────────────────────┐
|
||||
│ ConnectionHandler │
|
||||
│ (single Tokio task) │
|
||||
│ │ │
|
||||
│ ┌───────────┐ ┌───────┴───────┐ ┌──────────────────┐ │
|
||||
│ │Subscriptions│ │ Multiplexer │ │ Flush Observers │ │
|
||||
│ │ HashMap │ │ (request-reply)│ │ │ │
|
||||
│ └──────┬──────┘ └───────┬───────┘ └──────────────────┘ │
|
||||
│ └────────────────┼ │
|
||||
│ ┌──────┴──────┐ │
|
||||
│ │ Connector │ Server pool, reconnect │
|
||||
│ └──────┬──────┘ │
|
||||
│ │ │
|
||||
│ ┌──────┴──────┐ │
|
||||
│ │ Connection │ Protocol I/O │
|
||||
│ │ (read/write)│ ServerOp / ClientOp │
|
||||
│ └──────┬──────┘ │
|
||||
└──────────────────────────┼────────────────────────────────┘
|
||||
│
|
||||
┌──────┴──────┐
|
||||
│ NATS Server │ (Go binary, TCP/TLS/WS)
|
||||
└─────────────┘
|
||||
```
|
||||
|
||||
## Key Concepts
|
||||
|
||||
### Subject
|
||||
NATS uses subject strings for message addressing. A `Subject` is a validated, immutable, UTF-8 string backed by `Bytes`. Subjects use dot-delimited tokens (e.g., `events.data.sensor1`). Wildcards `*` (single token) and `>` (multi-token suffix) are supported for subscriptions.
|
||||
|
||||
### ClientOp / ServerOp
|
||||
The NATS client-server protocol is text-based with binary payloads. The client sends `ClientOp` variants (CONNECT, PUB/HPUB, SUB, UNSUB, PING, PONG) and receives `ServerOp` variants (INFO, MSG/HMSG, +OK, -ERR, PING, PONG).
|
||||
|
||||
### Command
|
||||
Internal command type sent from `Client` to `ConnectionHandler` via `mpsc` channel. Includes Publish, Request, Subscribe, Unsubscribe, Flush, Drain, Reconnect, SetServerPool, ServerPool.
|
||||
|
||||
### Multiplexer
|
||||
A single internal subscription (SID 0) that routes all request-reply responses. When a `Request` is made, a unique inbox token is registered in the multiplexer's sender map, and the response is dispatched to the corresponding `oneshot::Sender`.
|
||||
|
||||
### ConnectionHandler
|
||||
A single Tokio task that drives all protocol I/O. It processes server operations from `Connection`, handles client commands from the `mpsc` channel, manages subscriptions, maintains ping/pong health, and orchestrates reconnection.
|
||||
|
||||
## nats-server Test Harness
|
||||
|
||||
The `nats-server` crate provides utilities for launching real NATS server instances in tests:
|
||||
|
||||
- `run_server(cfg)` — starts a single server with optional config
|
||||
- `run_cluster(cfg)` — starts a 3-node cluster
|
||||
- `Server` struct — holds the child process, cleans up on drop
|
||||
- `Server::restart()` — kills and restarts the server process
|
||||
- `Server::client_url()` — reads the INFO from the server to get the client URL
|
||||
- `set_lame_duck_mode(server)` — sends LDM signal to the server process
|
||||
|
||||
The test harness spawns the Go `nats-server` binary via `std::process::Command`, using dynamic ports for parallel test execution. It auto-discovers the client URL by connecting to the server's TCP port and parsing the `INFO` JSON. On `Drop`, it kills the child process and cleans up JetStream storage directories.
|
||||
|
||||
## Feature Flags
|
||||
|
||||
```toml
|
||||
# Default: everything enabled
|
||||
default = ["server_2_10", "server_2_11", "server_2_12", "server_2_14",
|
||||
"service", "ring", "jetstream", "nkeys", "crypto",
|
||||
"object-store", "kv", "websockets", "nuid"]
|
||||
|
||||
# Subsystems
|
||||
jetstream # JetStream API
|
||||
kv # Key-Value store (requires jetstream)
|
||||
object-store # Object store (requires jetstream + crypto)
|
||||
service # Service API
|
||||
|
||||
# Crypto backends (pick one)
|
||||
ring # Default crypto backend
|
||||
aws-lc-rs # Alternative backend
|
||||
fips # FIPS mode (requires aws-lc-rs)
|
||||
|
||||
# Auth
|
||||
nkeys # NKey authentication
|
||||
|
||||
# Other
|
||||
nuid # NUID-based ID generation (falls back to rand)
|
||||
crypto # Encryption support
|
||||
websockets # WebSocket transport
|
||||
experimental # Experimental features
|
||||
|
||||
# Server version markers (enable version-specific API fields)
|
||||
server_2_10
|
||||
server_2_11
|
||||
server_2_12
|
||||
server_2_14
|
||||
```
|
||||
|
||||
## Dependencies (Key)
|
||||
|
||||
| Dependency | Purpose |
|
||||
|-----------|---------|
|
||||
| `tokio` | Async runtime (macros, rt, net, sync, time, io-util) |
|
||||
| `bytes` | Zero-copy byte buffers for payloads |
|
||||
| `tokio-rustls` | TLS via rustls |
|
||||
| `rustls-native-certs` | Load native TLS root certificates |
|
||||
| `serde` / `serde_json` | JSON serialization for protocol messages and JetStream API |
|
||||
| `memchr` | Fast CRLF search for protocol parsing |
|
||||
| `futures-util` | Stream trait, Sink trait, StreamExt |
|
||||
| `tracing` | Structured logging |
|
||||
| `thiserror` | Error type derivation |
|
||||
| `url` | URL parsing for server addresses |
|
||||
| `portable-atomic` | Portable atomic operations |
|
||||
|
||||
## References
|
||||
|
||||
- [NATS Protocol Specification](https://docs.nats.io/reference/reference-protocols/nats-protocol)
|
||||
- [NATS JetStream Documentation](https://docs.nats.io/nats-concepts/jetstream)
|
||||
- [async-nats on docs.rs](https://docs.rs/async-nats)
|
||||
@@ -0,0 +1,281 @@
|
||||
# NATS Client Protocol and Wire Format
|
||||
|
||||
**Protocol**: NATS Client Protocol v1 (with dynamic reconfiguration)
|
||||
**Transport**: TCP (port 4222), TLS, WebSocket (ws/wss)
|
||||
|
||||
## Protocol Overview
|
||||
|
||||
The NATS client-server protocol is a simple, text-based protocol with binary payload support. All operations are terminated with `\r\n`. Messages carry their payload length, allowing efficient binary data transfer.
|
||||
|
||||
### Connection Lifecycle
|
||||
|
||||
```
|
||||
Client Server
|
||||
│ │
|
||||
│◄──────────── INFO {json} ────────────────────│ Server sends INFO first
|
||||
│ │
|
||||
│────────────── CONNECT {json} ────────────────►│ Client sends CONNECT
|
||||
│────────────── PING ──────────────────────────►│ Client sends PING
|
||||
│◄──────────── PONG ────────────────────────── │ Server confirms connection
|
||||
│ │
|
||||
│──── SUB/UNSUB/PUB/HPUB ──────────────────────►│ Normal operation
|
||||
│◄─── MSG/HMSG/+OK/-ERR/PING ─────────────────│
|
||||
│ │
|
||||
```
|
||||
|
||||
## Server Operations (ServerOp)
|
||||
|
||||
These are operations received from the server. The `Connection` module parses these from the read buffer.
|
||||
|
||||
### INFO
|
||||
|
||||
Sent by the server upon connection and asynchronously when cluster topology changes.
|
||||
|
||||
```
|
||||
INFO {json}\r\n
|
||||
```
|
||||
|
||||
JSON fields (see `ServerInfo` struct):
|
||||
|
||||
| Field | Type | Description |
|
||||
|-------|------|-------------|
|
||||
| `server_id` | String | Unique server identifier |
|
||||
| `server_name` | String | Generated server name |
|
||||
| `host` | String | Cluster host |
|
||||
| `port` | u16 | Cluster port |
|
||||
| `version` | String | Server version |
|
||||
| `auth_required` | bool | Authentication required |
|
||||
| `tls_required` | bool | TLS required |
|
||||
| `max_payload` | usize | Maximum payload size |
|
||||
| `proto` | i8 | Protocol version (0 or 1) |
|
||||
| `client_id` | u64 | Server-assigned client ID |
|
||||
| `go` | String | Go build version |
|
||||
| `nonce` | String | Nonce for nkey auth |
|
||||
| `connect_urls` | Vec<String> | Cluster server URLs |
|
||||
| `client_ip` | String | Client IP as seen by server |
|
||||
| `headers` | bool | Server supports headers |
|
||||
| `ldm` | bool | Lame duck mode |
|
||||
| `cluster` | Option<String> | Cluster name |
|
||||
| `domain` | Option<String> | NATS domain |
|
||||
| `jetstream` | bool | JetStream enabled |
|
||||
|
||||
### MSG
|
||||
|
||||
Delivers a message to a subscription (no headers):
|
||||
|
||||
```
|
||||
MSG <subject> <sid> [reply-to] <#bytes>\r\n
|
||||
<payload>\r\n
|
||||
```
|
||||
|
||||
### HMSG
|
||||
|
||||
Delivers a message with headers:
|
||||
|
||||
```
|
||||
HMSG <subject> <sid> [reply-to] <#header-bytes> <#total-bytes>\r\n
|
||||
<NATS/1.0 [status] [description]>\r\n
|
||||
<header-name>: <header-value>\r\n
|
||||
\r\n
|
||||
<payload>\r\n
|
||||
```
|
||||
|
||||
Header format follows the NATS/1.0 header spec:
|
||||
- First line: `NATS/1.0` optionally followed by status code and description
|
||||
- Subsequent lines: `name: value` headers
|
||||
- Empty line separates headers from payload
|
||||
- Header values may span multiple lines (continuation lines start with whitespace)
|
||||
|
||||
### PING / PONG
|
||||
|
||||
```
|
||||
PING\r\n → Client responds with PONG
|
||||
PONG\r\n → Acknowledges client's PING
|
||||
```
|
||||
|
||||
### +OK / -ERR
|
||||
|
||||
```
|
||||
+OK\r\n → Success acknowledgment (verbose mode)
|
||||
-ERR <description>\r\n → Error from server
|
||||
```
|
||||
|
||||
Common server errors:
|
||||
- `authorization violation` → parsed as `ServerError::AuthorizationViolation`
|
||||
- Other strings → `ServerError::Other(String)`
|
||||
|
||||
## Client Operations (ClientOp)
|
||||
|
||||
These are operations sent from the client to the server. The `Connection` module serializes these to the write buffer.
|
||||
|
||||
### CONNECT
|
||||
|
||||
Sent as the first client operation after receiving INFO. Contains authentication and capability information.
|
||||
|
||||
```
|
||||
CONNECT {json}\r\n
|
||||
```
|
||||
|
||||
JSON fields (see `ConnectInfo` struct):
|
||||
|
||||
| Field | Type | Description |
|
||||
|-------|------|-------------|
|
||||
| `verbose` | bool | Enable +OK acknowledgments (always false in this client) |
|
||||
| `pedantic` | bool | Strict format checking (always false) |
|
||||
| `jwt` | Option<String> | User JWT for auth |
|
||||
| `nkey` | Option<String> | Public nkey for auth |
|
||||
| `sig` | Option<String> | Signed nonce (Base64URL encoded) |
|
||||
| `name` | Option<String> | Client name |
|
||||
| `echo` | bool | Whether server should echo messages back |
|
||||
| `lang` | String | Implementation language ("rust") |
|
||||
| `version` | String | Client version |
|
||||
| `protocol` | u8 | Protocol version (1 = dynamic) |
|
||||
| `tls_required` | bool | TLS required |
|
||||
| `user` | Option<String> | Username |
|
||||
| `pass` | Option<String> | Password |
|
||||
| `auth_token` | Option<String> | Auth token |
|
||||
| `headers` | bool | Client supports headers (always true) |
|
||||
| `no_responders` | bool | Client supports no-responders (always true) |
|
||||
|
||||
### PUB / HPUB
|
||||
|
||||
Publish a message:
|
||||
|
||||
```
|
||||
PUB <subject> [reply-to] <#payload-bytes>\r\n
|
||||
<payload>\r\n
|
||||
```
|
||||
|
||||
Publish with headers:
|
||||
|
||||
```
|
||||
HPUB <subject> [reply-to] <#header-bytes> <#total-bytes>\r\n
|
||||
<NATS/1.0\r\n
|
||||
<header-name>: <header-value>\r\n
|
||||
\r\n
|
||||
<payload>\r\n
|
||||
```
|
||||
|
||||
### SUB
|
||||
|
||||
Subscribe to a subject:
|
||||
|
||||
```
|
||||
SUB <subject> [queue-group] <sid>\r\n
|
||||
```
|
||||
|
||||
### UNSUB
|
||||
|
||||
Unsubscribe from a subscription:
|
||||
|
||||
```
|
||||
UNSUB <sid> [max]\r\n
|
||||
```
|
||||
|
||||
The optional `max` parameter tells the server to auto-unsubscribe after receiving the specified number of messages.
|
||||
|
||||
### PING / PONG
|
||||
|
||||
```
|
||||
PING\r\n → Health check / keepalive
|
||||
PONG\r\n → Response to server PING
|
||||
```
|
||||
|
||||
## Protocol Version
|
||||
|
||||
The `Protocol` enum has two variants:
|
||||
|
||||
| Value | Name | Description |
|
||||
|-------|------|-------------|
|
||||
| 0 | Original | Basic protocol |
|
||||
| 1 | Dynamic | Supports async INFO for cluster topology changes, lame duck mode |
|
||||
|
||||
This client always sends `protocol: 1` (Dynamic), enabling:
|
||||
- Asynchronous INFO messages with updated server lists
|
||||
- Lame duck mode notifications
|
||||
- Dynamic reconfiguration of cluster topology
|
||||
|
||||
## Wire Format Details
|
||||
|
||||
### Message Length Calculation
|
||||
|
||||
For plain `MSG`:
|
||||
```
|
||||
length = subject.len() + reply.map_or(0, |r| r.len()) + payload.len()
|
||||
```
|
||||
|
||||
For `HMSG`:
|
||||
```
|
||||
length = subject.len() + reply.map_or(0, |r| r.len()) + header_len + payload.len()
|
||||
```
|
||||
|
||||
Where `header_len` = serialized header bytes and `total_len` = `header_len + payload.len()`.
|
||||
|
||||
### Write Buffer Architecture
|
||||
|
||||
The `Connection` uses a two-tier write buffer:
|
||||
|
||||
1. **`flattened_writes`** (`BytesMut`) — for small writes (< 4096 bytes). Protocol headers, short commands, and small messages are flattened into this buffer for efficient sequential writing.
|
||||
|
||||
2. **`write_buf`** (`VecDeque<Bytes>`) — for large writes (>= 4096 bytes). Large payloads are appended as separate `Bytes` chunks. Supports vectored writes (`write_vectored`) when the underlying stream supports it, writing up to 64 chunks at once.
|
||||
|
||||
The soft limit for the total write buffer is 65,535 bytes (`SOFT_WRITE_BUF_LIMIT`). When exceeded, the `ConnectionHandler` stops processing new commands until the buffer drains.
|
||||
|
||||
### Read Buffer Architecture
|
||||
|
||||
The `Connection` uses a single `BytesMut` read buffer with configurable initial capacity (default 65,535 bytes). Protocol parsing uses `memchr::memmem::find` to locate CRLF delimiters efficiently. If a partial message is in the buffer, the parser returns `None` and waits for more data.
|
||||
|
||||
### Header Serialization
|
||||
|
||||
Headers are serialized in NATS/1.0 format:
|
||||
|
||||
```
|
||||
NATS/1.0\r\n
|
||||
Header-Name: Header-Value\r\n
|
||||
Multi-Line-Header: value part 1\r\n
|
||||
continuation of value\r\n
|
||||
Another-Header: another value\r\n
|
||||
\r\n
|
||||
```
|
||||
|
||||
The `HeaderMap::to_bytes()` method handles this serialization, using `httparse`-compatible line folding for multi-line values.
|
||||
|
||||
### Status Codes in Headers
|
||||
|
||||
NATS status codes are embedded in the `HMSG` header version line:
|
||||
|
||||
```
|
||||
NATS/1.0 404 No Messages\r\n
|
||||
NATS/1.0 408 Request Timeout\r\n
|
||||
NATS/1.0 503 No Responders\r\n
|
||||
```
|
||||
|
||||
Common codes used by the client:
|
||||
|
||||
| Code | Constant | Meaning |
|
||||
|------|----------|---------|
|
||||
| 100 | `IDLE_HEARTBEAT` | JetStream idle heartbeat |
|
||||
| 200 | `OK` | Success |
|
||||
| 404 | `NOT_FOUND` | Message/stream not found |
|
||||
| 408 | `TIMEOUT` | Request timeout |
|
||||
| 409 | `REQUEST_TERMINATED` | Request terminated |
|
||||
| 503 | `NO_RESPONDERS` | No responders available |
|
||||
|
||||
## Protocol Parsing Implementation
|
||||
|
||||
The `Connection::try_read_op()` method handles all protocol parsing:
|
||||
|
||||
1. Search for `\r\n` delimiter using `memchr::memmem::find`
|
||||
2. Match the operation prefix:
|
||||
- `+OK` → `ServerOp::Ok`
|
||||
- `PING` → `ServerOp::Ping`
|
||||
- `PONG` → `ServerOp::Pong`
|
||||
- `-ERR` → parse error description → `ServerOp::Error`
|
||||
- `INFO ` → parse JSON → `ServerOp::Info`
|
||||
- `MSG ` → parse subject/sid/reply/length, read payload → `ServerOp::Message`
|
||||
- `HMSG ` → parse headers + payload → `ServerOp::Message`
|
||||
3. Unknown prefix → return `io::Error` with `InvalidInput`
|
||||
|
||||
For `MSG` and `HMSG`, if the complete payload isn't yet in the read buffer (checked via `len + payload_len + 4 > remaining`), the method returns `Ok(None)` and the buffer accumulates more data before retrying.
|
||||
|
||||
Non-UTF8 subjects in server messages are handled gracefully — the parser returns an `io::Error` rather than panicking, which is critical because the Go server does not enforce UTF-8 in subjects (regression fix for issue #1572).
|
||||
@@ -0,0 +1,443 @@
|
||||
# Key Types and Traits
|
||||
|
||||
This document covers the core data types in the `async-nats` crate that form the public API and internal plumbing.
|
||||
|
||||
## Public Types
|
||||
|
||||
### Client
|
||||
|
||||
**Location**: `client.rs`
|
||||
|
||||
`Client` is the primary user-facing type. It is a lightweight, cloneable handle to a NATS connection.
|
||||
|
||||
```rust
|
||||
#[derive(Clone, Debug)]
|
||||
pub struct Client {
|
||||
info: tokio::sync::watch::Receiver<Option<ServerInfo>>,
|
||||
state: tokio::sync::watch::Receiver<State>,
|
||||
sender: mpsc::Sender<Command>,
|
||||
poll_sender: PollSender<Command>,
|
||||
next_subscription_id: Arc<AtomicU64>,
|
||||
subscription_capacity: usize,
|
||||
inbox_prefix: Arc<str>,
|
||||
request_timeout: Option<Duration>,
|
||||
max_payload: Arc<AtomicUsize>,
|
||||
connection_stats: Arc<Statistics>,
|
||||
skip_subject_validation: bool,
|
||||
}
|
||||
```
|
||||
|
||||
Key methods:
|
||||
- `publish(subject, payload)` — fire-and-forget publish
|
||||
- `publish_with_headers(subject, headers, payload)` — publish with NATS headers
|
||||
- `publish_with_reply(subject, reply, payload)` — publish with reply subject
|
||||
- `request(subject, payload)` — request-response (returns `Message`)
|
||||
- `send_request(subject, request)` — request with `Request` builder
|
||||
- `subscribe(subject)` — subscribe to a subject, returns `Subscriber`
|
||||
- `queue_subscribe(subject, queue_group)` — subscribe as part of a queue group
|
||||
- `flush()` — ensure all pending messages are written to the wire
|
||||
- `drain()` — gracefully drain all subscriptions and close
|
||||
- `force_reconnect()` — trigger immediate reconnection
|
||||
- `new_inbox()` — generate a unique inbox subject for request-reply
|
||||
- `server_info()` — get last received `ServerInfo`
|
||||
- `max_payload()` — get server's maximum payload size
|
||||
- `connection_state()` — get current connection `State`
|
||||
- `statistics()` — get `Arc<Statistics>` for connection metrics
|
||||
- `is_server_compatible(major, minor, patch)` — check server version compatibility
|
||||
- `set_server_pool(addrs)` / `server_pool()` — manage server pool
|
||||
|
||||
`Client` also implements `Sink<OutboundMessage>` for backpressure-aware publishing.
|
||||
|
||||
### Subscriber
|
||||
|
||||
**Location**: `lib.rs`
|
||||
|
||||
A `Subscriber` receives messages from a single subscription. It implements `futures::Stream`.
|
||||
|
||||
```rust
|
||||
#[derive(Debug)]
|
||||
pub struct Subscriber {
|
||||
sid: u64,
|
||||
receiver: mpsc::Receiver<Message>,
|
||||
sender: mpsc::Sender<Command>,
|
||||
}
|
||||
```
|
||||
|
||||
Key methods:
|
||||
- `unsubscribe()` — unsubscribe and close the stream
|
||||
- `unsubscribe_after(max)` — auto-unsubscribe after N messages
|
||||
- `drain()` — gracefully drain remaining messages then close
|
||||
|
||||
On `Drop`, `Subscriber` automatically sends an `Unsubscribe` command and closes the receiver channel.
|
||||
|
||||
### Message
|
||||
|
||||
**Location**: `message.rs`
|
||||
|
||||
Represents an inbound NATS message.
|
||||
|
||||
```rust
|
||||
#[derive(Clone, Debug, Serialize, Deserialize)]
|
||||
pub struct Message {
|
||||
pub subject: Subject,
|
||||
pub reply: Option<Subject>,
|
||||
pub payload: Bytes,
|
||||
pub headers: Option<HeaderMap>,
|
||||
pub status: Option<StatusCode>,
|
||||
pub description: Option<String>,
|
||||
pub length: usize,
|
||||
}
|
||||
```
|
||||
|
||||
### OutboundMessage
|
||||
|
||||
**Location**: `message.rs`
|
||||
|
||||
Represents a message to be published. No status/description fields (those are inbound-only).
|
||||
|
||||
```rust
|
||||
#[derive(Clone, Debug)]
|
||||
pub struct OutboundMessage {
|
||||
pub subject: Subject,
|
||||
pub reply: Option<Subject>,
|
||||
pub payload: Bytes,
|
||||
pub headers: Option<HeaderMap>,
|
||||
}
|
||||
```
|
||||
|
||||
### Subject
|
||||
|
||||
**Location**: `subject.rs`
|
||||
|
||||
An immutable, validated UTF-8 string backed by `Bytes`. Used throughout the crate instead of raw `String` for subjects.
|
||||
|
||||
```rust
|
||||
#[derive(Clone, Debug, PartialEq, Eq, PartialOrd, Ord)]
|
||||
pub struct Subject {
|
||||
bytes: Bytes,
|
||||
}
|
||||
```
|
||||
|
||||
Implements `Deref<Target = str>`, `From<&str>`, `From<String>`, `TryFrom<Bytes>`, `Serialize`, `Deserialize`.
|
||||
|
||||
Validation methods:
|
||||
- `is_valid()` — checks NATS subject rules (no leading/trailing dots, no consecutive dots, no whitespace)
|
||||
- `validated(s)` — construct with validation, returns `Result<Subject, SubjectError>`
|
||||
- `from_static_validated(s)` — const-time validation for static strings (compile-time panic on invalid)
|
||||
|
||||
### ToSubject Trait
|
||||
|
||||
**Location**: `subject.rs`
|
||||
|
||||
```rust
|
||||
pub trait ToSubject {
|
||||
fn to_subject(&self) -> Subject;
|
||||
}
|
||||
```
|
||||
|
||||
Implemented for `Subject`, `&'static str`, `String`. All methods accepting subjects are generic over `impl ToSubject`.
|
||||
|
||||
### HeaderMap
|
||||
|
||||
**Location**: `header.rs`
|
||||
|
||||
NATS message headers, modeled after the `http::header` crate.
|
||||
|
||||
```rust
|
||||
#[derive(Clone, PartialEq, Eq, Debug, Default)]
|
||||
pub struct HeaderMap {
|
||||
inner: HashMap<HeaderName, Vec<HeaderValue>>,
|
||||
}
|
||||
```
|
||||
|
||||
Supports multiple values per header name (like HTTP). Key methods:
|
||||
- `insert(name, value)` — replace all values for a name
|
||||
- `append(name, value)` — add a value to a name
|
||||
- `get(name)` — get the first value
|
||||
- `get_all(name)` — get all values as an iterator
|
||||
- `len()` / `is_empty()` — number of header entries
|
||||
- `to_bytes()` — serialize to NATS/1.0 wire format
|
||||
- `wire_len()` — size in wire format (for payload size checks)
|
||||
|
||||
### StatusCode
|
||||
|
||||
**Location**: `status.rs`
|
||||
|
||||
NATS status codes (100-999), structurally similar to HTTP status codes.
|
||||
|
||||
```rust
|
||||
#[derive(Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash, Serialize, Deserialize)]
|
||||
pub struct StatusCode(NonZeroU16);
|
||||
```
|
||||
|
||||
Constants:
|
||||
| Constant | Code | Meaning |
|
||||
|----------|------|---------|
|
||||
| `IDLE_HEARTBEAT` | 100 | JetStream idle heartbeat |
|
||||
| `OK` | 200 | Success |
|
||||
| `NOT_FOUND` | 404 | Not found |
|
||||
| `TIMEOUT` | 408 | Timeout |
|
||||
| `REQUEST_TERMINATED` | 409 | Request terminated |
|
||||
| `NO_RESPONDERS` | 503 | No responders |
|
||||
|
||||
### ServerInfo
|
||||
|
||||
**Location**: `lib.rs`
|
||||
|
||||
Deserialized from the server's `INFO` JSON message. Contains server capabilities, connection details, and cluster information.
|
||||
|
||||
### ConnectInfo
|
||||
|
||||
**Location**: `lib.rs`
|
||||
|
||||
Serialized into the client's `CONNECT` JSON message. Contains authentication credentials, client capabilities, and protocol preferences.
|
||||
|
||||
### ServerAddr
|
||||
|
||||
**Location**: `lib.rs`
|
||||
|
||||
A validated NATS server URL, supporting schemes `nats://`, `tls://`, `ws://`, `wss://`.
|
||||
|
||||
```rust
|
||||
#[derive(Debug, Clone, PartialEq, Eq, Hash)]
|
||||
pub struct ServerAddr(Url);
|
||||
```
|
||||
|
||||
Methods:
|
||||
- `from_url(url)` — validate and create
|
||||
- `tls_required()` — true for `tls://` scheme
|
||||
- `is_websocket()` — true for `ws://` or `wss://`
|
||||
- `host()` / `port()` / `scheme()` — URL component accessors
|
||||
- `socket_addrs()` — async DNS resolution
|
||||
- `username()` / `password()` — embedded credentials
|
||||
|
||||
### Auth
|
||||
|
||||
**Location**: `auth.rs`
|
||||
|
||||
Container for authentication credentials.
|
||||
|
||||
```rust
|
||||
#[derive(Clone, Default)]
|
||||
pub struct Auth {
|
||||
pub jwt: Option<String>,
|
||||
pub nkey: Option<String>,
|
||||
pub signature_callback: Option<CallbackArg1<String, Result<String, AuthError>>>,
|
||||
pub signature: Option<Vec<u8>>,
|
||||
pub username: Option<String>,
|
||||
pub password: Option<String>,
|
||||
pub token: Option<String>,
|
||||
}
|
||||
```
|
||||
|
||||
### Request
|
||||
|
||||
**Location**: `client.rs`
|
||||
|
||||
Builder for customized request-response operations.
|
||||
|
||||
```rust
|
||||
#[derive(Default)]
|
||||
pub struct Request {
|
||||
pub payload: Option<Bytes>,
|
||||
pub headers: Option<HeaderMap>,
|
||||
pub timeout: Option<Option<Duration>>,
|
||||
pub inbox: Option<String>,
|
||||
}
|
||||
```
|
||||
|
||||
### Statistics
|
||||
|
||||
**Location**: `client.rs`
|
||||
|
||||
Atomic connection statistics shared between Client and ConnectionHandler.
|
||||
|
||||
```rust
|
||||
#[derive(Default, Debug)]
|
||||
pub struct Statistics {
|
||||
pub in_bytes: AtomicU64,
|
||||
pub out_bytes: AtomicU64,
|
||||
pub in_messages: AtomicU64,
|
||||
pub out_messages: AtomicU64,
|
||||
pub connects: AtomicU64,
|
||||
}
|
||||
```
|
||||
|
||||
### Event
|
||||
|
||||
**Location**: `lib.rs`
|
||||
|
||||
Events emitted by the client for connection lifecycle monitoring.
|
||||
|
||||
```rust
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
pub enum Event {
|
||||
Connected,
|
||||
Disconnected,
|
||||
LameDuckMode,
|
||||
Draining,
|
||||
Closed,
|
||||
SlowConsumer(u64),
|
||||
ServerError(ServerError),
|
||||
ClientError(ClientError),
|
||||
}
|
||||
```
|
||||
|
||||
## Internal Types
|
||||
|
||||
### Command
|
||||
|
||||
**Location**: `lib.rs`
|
||||
|
||||
Internal commands sent from `Client` to `ConnectionHandler` via `mpsc` channel.
|
||||
|
||||
```rust
|
||||
pub(crate) enum Command {
|
||||
Publish(OutboundMessage),
|
||||
Request { subject, payload, respond, headers, sender: oneshot::Sender<Message> },
|
||||
Subscribe { sid, subject, queue_group, sender: mpsc::Sender<Message> },
|
||||
Unsubscribe { sid, max: Option<u64> },
|
||||
Flush { observer: oneshot::Sender<()> },
|
||||
Drain { sid: Option<u64> },
|
||||
Reconnect,
|
||||
SetServerPool { servers: Vec<ServerAddr>, result: oneshot::Sender<Result<(), String>> },
|
||||
ServerPool { result: oneshot::Sender<Vec<connector::Server>> },
|
||||
}
|
||||
```
|
||||
|
||||
### ClientOp / ServerOp
|
||||
|
||||
**Location**: `lib.rs`
|
||||
|
||||
Protocol-level operation types used by `Connection` for wire format parsing and serialization.
|
||||
|
||||
### Subscription (Internal)
|
||||
|
||||
**Location**: `lib.rs`
|
||||
|
||||
```rust
|
||||
struct Subscription {
|
||||
subject: Subject,
|
||||
sender: mpsc::Sender<Message>,
|
||||
queue_group: Option<String>,
|
||||
delivered: u64,
|
||||
max: Option<u64>,
|
||||
}
|
||||
```
|
||||
|
||||
### Multiplexer (Internal)
|
||||
|
||||
**Location**: `lib.rs`
|
||||
|
||||
```rust
|
||||
struct Multiplexer {
|
||||
subject: Subject, // Wildcard subscription subject (e.g., "_INBOX.xxx.*")
|
||||
prefix: Subject, // Prefix for routing (e.g., "_INBOX.xxx.")
|
||||
senders: HashMap<String, oneshot::Sender<Message>>, // token → sender
|
||||
}
|
||||
```
|
||||
|
||||
### Connection State
|
||||
|
||||
**Location**: `connection.rs`
|
||||
|
||||
```rust
|
||||
#[derive(Debug, Eq, PartialEq, Clone)]
|
||||
pub enum State {
|
||||
Pending,
|
||||
Connected,
|
||||
Disconnected,
|
||||
}
|
||||
```
|
||||
|
||||
### Protocol
|
||||
|
||||
**Location**: `lib.rs`
|
||||
|
||||
```rust
|
||||
#[derive(Serialize_repr, Deserialize_repr, PartialEq, Eq, Debug, Clone, Copy)]
|
||||
#[repr(u8)]
|
||||
pub enum Protocol {
|
||||
Original = 0,
|
||||
Dynamic = 1,
|
||||
}
|
||||
```
|
||||
|
||||
## Error Type Pattern
|
||||
|
||||
The crate uses a generic `Error<Kind>` type throughout. Every subsystem defines its own `ErrorKind` enum and a type alias:
|
||||
|
||||
```rust
|
||||
// Define the kind enum
|
||||
#[derive(Clone, Debug, PartialEq)]
|
||||
pub enum PublishErrorKind {
|
||||
MaxPayloadExceeded,
|
||||
InvalidSubject,
|
||||
Send,
|
||||
}
|
||||
|
||||
// Define the error type alias
|
||||
pub type PublishError = Error<PublishErrorKind>;
|
||||
|
||||
// Construct errors
|
||||
PublishError::new(PublishErrorKind::MaxPayloadExceeded)
|
||||
PublishError::with_source(PublishErrorKind::Send, io_error)
|
||||
|
||||
// Match on errors
|
||||
if err.kind() == PublishErrorKind::MaxPayloadExceeded { ... }
|
||||
```
|
||||
|
||||
Error kinds in the crate:
|
||||
|
||||
| Error Type | Kind Enum | Context |
|
||||
|-----------|-----------|---------|
|
||||
| `ConnectError` | `ConnectErrorKind` | Initial connection failures |
|
||||
| `PublishError` | `PublishErrorKind` | Publish validation failures |
|
||||
| `RequestError` | `RequestErrorKind` | Request-response failures |
|
||||
| `SubscribeError` | `SubscribeErrorKind` | Subscription failures |
|
||||
| `FlushError` | `FlushErrorKind` | Flush failures |
|
||||
| `ServerPoolError` | `ServerPoolErrorKind` | Server pool query failures |
|
||||
| `SetServerPoolError` | `SetServerPoolErrorKind` | Server pool modification failures |
|
||||
|
||||
## Trait Implementations
|
||||
|
||||
### Client Trait Interfaces
|
||||
|
||||
The `Client` implements several traits defined in `client::traits`:
|
||||
|
||||
```rust
|
||||
// Publisher trait — publish with optional reply subject
|
||||
trait Publisher {
|
||||
fn publish_with_reply<S, R>(&self, subject: S, reply: R, payload: Bytes) -> impl Future<Output = Result<(), PublishError>>;
|
||||
fn publish_message(&self, msg: OutboundMessage) -> impl Future<Output = Result<(), PublishError>>;
|
||||
}
|
||||
|
||||
// Subscriber trait — subscribe to a subject
|
||||
trait Subscriber {
|
||||
fn subscribe<S>(&self, subject: S) -> impl Future<Output = Result<crate::Subscriber, SubscribeError>>;
|
||||
}
|
||||
|
||||
// Requester trait — send request-response
|
||||
trait Requester {
|
||||
fn send_request<S>(&self, subject: S, request: Request) -> impl Future<Output = Result<Message, RequestError>>;
|
||||
}
|
||||
|
||||
// TimeoutProvider trait — access request timeout
|
||||
trait TimeoutProvider {
|
||||
fn timeout(&self) -> Option<Duration>;
|
||||
}
|
||||
```
|
||||
|
||||
### ToServerAddrs Trait
|
||||
|
||||
**Location**: `lib.rs`
|
||||
|
||||
Converts various address types into server address iterators. Implemented for `ServerAddr`, `str`, `String`, `&[T]`, `Vec<T>`, `&[ServerAddr]`, and references.
|
||||
|
||||
### Sink<OutboundMessage>
|
||||
|
||||
`Client` implements `futures::Sink<OutboundMessage>` for backpressure-aware publishing through the `PollSender` adapter.
|
||||
|
||||
### Stream for Subscriber
|
||||
|
||||
`Subscriber` implements `futures::Stream` with `Item = Message`, delegating to the internal `mpsc::Receiver`.
|
||||
@@ -0,0 +1,338 @@
|
||||
# Connection Handler and Data Flow
|
||||
|
||||
This document covers the internal `ConnectionHandler` that drives all protocol I/O, and the data flow through the system.
|
||||
|
||||
## ConnectionHandler
|
||||
|
||||
**Location**: `lib.rs`
|
||||
|
||||
The `ConnectionHandler` is the heart of the client. It runs as a single Tokio task and manages all communication with the NATS server.
|
||||
|
||||
```rust
|
||||
pub(crate) struct ConnectionHandler {
|
||||
connection: Connection, // Low-level I/O
|
||||
connector: Connector, // Server pool, reconnection
|
||||
subscriptions: HashMap<u64, Subscription>, // Active subscriptions
|
||||
multiplexer: Option<Multiplexer>, // Request-reply multiplexer
|
||||
pending_pings: usize, // Unanswered PINGs
|
||||
info_sender: tokio::sync::watch::Sender<Option<ServerInfo>>,
|
||||
ping_interval: Interval, // Periodic PING timer
|
||||
should_reconnect: bool, // Flag for forced reconnect
|
||||
flush_observers: Vec<oneshot::Sender<()>>, // Pending flush callbacks
|
||||
is_draining: bool, // Connection is draining
|
||||
drain_pings: VecDeque<u64>, // SIDs being drained
|
||||
}
|
||||
```
|
||||
|
||||
## Data Flow: Publish
|
||||
|
||||
```
|
||||
Application
|
||||
│
|
||||
│ client.publish("events.data", payload)
|
||||
│
|
||||
▼
|
||||
Client
|
||||
│ validates subject & payload size
|
||||
│ sends Command::Publish(OutboundMessage) via mpsc channel
|
||||
│
|
||||
▼
|
||||
ConnectionHandler::handle_command(Command::Publish)
|
||||
│ increments out_messages, out_bytes statistics
|
||||
│ calls connection.enqueue_write_op(&ClientOp::Publish { ... })
|
||||
│
|
||||
▼
|
||||
Connection::enqueue_write_op
|
||||
│ serializes to wire format:
|
||||
│ "PUB events.data 11\r\n" or "HPUB events.data 23 34\r\n"
|
||||
│ appends to flattened_writes or write_buf
|
||||
│
|
||||
▼
|
||||
Connection::poll_write
|
||||
│ uses vectored writes (64 chunks) if supported
|
||||
│ or sequential writes otherwise
|
||||
│
|
||||
▼
|
||||
Connection::poll_flush
|
||||
│ flushes the TCP/TLS/WS stream
|
||||
│ notifies flush_observers
|
||||
│
|
||||
▼
|
||||
NATS Server (TCP/TLS/WebSocket)
|
||||
```
|
||||
|
||||
## Data Flow: Subscribe
|
||||
|
||||
```
|
||||
Application
|
||||
│
|
||||
│ client.subscribe("events.>")
|
||||
│
|
||||
▼
|
||||
Client::subscribe
|
||||
│ validates subject (always, regardless of skip_subject_validation)
|
||||
│ allocates next sid via AtomicU64
|
||||
│ creates mpsc channel for messages
|
||||
│ sends Command::Subscribe { sid, subject, sender }
|
||||
│ returns Subscriber { sid, receiver }
|
||||
│
|
||||
▼
|
||||
ConnectionHandler::handle_command(Command::Subscribe)
|
||||
│ creates Subscription { subject, sender, delivered: 0, max: None }
|
||||
│ inserts into subscriptions HashMap
|
||||
│ calls connection.enqueue_write_op(&ClientOp::Subscribe { sid, subject, queue_group })
|
||||
│
|
||||
▼
|
||||
Connection::enqueue_write_op
|
||||
│ serializes: "SUB events.> 42\r\n"
|
||||
│
|
||||
▼
|
||||
Server sends MSG for matching subjects:
|
||||
│
|
||||
▼
|
||||
ConnectionHandler::handle_server_op(ServerOp::Message { sid, subject, ... })
|
||||
│ looks up sid in subscriptions HashMap
|
||||
│ constructs Message { subject, reply, payload, headers, status, description }
|
||||
│ tries subscription.sender.try_send(message)
|
||||
│
|
||||
├── Ok → increments subscription.delivered, checks max
|
||||
├── Full → emits Event::SlowConsumer(sid)
|
||||
└── Closed → removes subscription, sends ClientOp::Unsubscribe
|
||||
│
|
||||
▼
|
||||
Subscriber::poll_next (Stream impl)
|
||||
│ receives from mpsc::Receiver
|
||||
│
|
||||
▼
|
||||
Application processes Message
|
||||
```
|
||||
|
||||
## Data Flow: Request-Response
|
||||
|
||||
The request-response pattern uses the **multiplexer** — a single wildcard subscription that routes responses to their waiting requesters.
|
||||
|
||||
```
|
||||
Application
|
||||
│
|
||||
│ client.request("service", payload)
|
||||
│
|
||||
▼
|
||||
Client::send_request
|
||||
│ validates subject & payload size
|
||||
│ creates oneshot channel for response
|
||||
│ generates unique inbox: "_INBOX.<nuid>.<token>"
|
||||
│ sends Command::Request { subject, payload, respond, sender }
|
||||
│
|
||||
▼
|
||||
ConnectionHandler::handle_command(Command::Request)
|
||||
│ extracts token from respond subject (after last '.')
|
||||
│ if no multiplexer exists:
|
||||
│ creates Multiplexer with wildcard sub "_INBOX.<id>.*" (SID 0)
|
||||
│ sends ClientOp::Subscribe { sid: 0, subject: "_INBOX.<id>.*" }
|
||||
│ inserts token → oneshot::Sender in multiplexer.senders
|
||||
│ sends ClientOp::Publish { subject, payload, respond: "<prefix><token>" }
|
||||
│
|
||||
▼
|
||||
Server routes request to service:
|
||||
│
|
||||
▼
|
||||
Service responds by publishing to the reply subject:
|
||||
│
|
||||
▼
|
||||
ConnectionHandler::handle_server_op(ServerOp::Message { sid: 0, ... })
|
||||
│ sid == MULTIPLEXER_SID (0), so enters multiplexer path
|
||||
│ extracts token by stripping prefix from subject
|
||||
│ looks up token in multiplexer.senders
|
||||
│ sends Message via oneshot::Sender
|
||||
│
|
||||
▼
|
||||
Client::send_request receives via oneshot::Receiver
|
||||
│ applies timeout (default 10s)
|
||||
│ checks for NO_RESPONDERS status (503)
|
||||
│
|
||||
▼
|
||||
Application receives Message
|
||||
```
|
||||
|
||||
### Custom Inbox Request
|
||||
|
||||
If the `Request` builder specifies a custom `inbox`, the flow is different:
|
||||
- The client subscribes to the inbox directly (not via multiplexer)
|
||||
- Publishes with the inbox as the reply subject
|
||||
- Waits for the message on that subscription
|
||||
- No multiplexer involvement
|
||||
|
||||
## Data Flow: Flush
|
||||
|
||||
```
|
||||
Application
|
||||
│
|
||||
│ client.flush()
|
||||
│
|
||||
▼
|
||||
Client::flush
|
||||
│ creates oneshot channel
|
||||
│ sends Command::Flush { observer }
|
||||
│
|
||||
▼
|
||||
ConnectionHandler::handle_command(Command::Flush)
|
||||
│ pushes observer into flush_observers Vec
|
||||
│
|
||||
▼
|
||||
ProcessFut::poll (main loop)
|
||||
│ after writing all pending data...
|
||||
│ checks should_flush():
|
||||
│ Yes (write buffers empty, not yet flushed) → poll_flush
|
||||
│ May (write buffers not empty) → poll_flush
|
||||
│ No (already flushed) → skip
|
||||
│ on successful flush:
|
||||
│ drains flush_observers, sending () to each
|
||||
│
|
||||
▼
|
||||
Client::flush receives via oneshot::Receiver
|
||||
```
|
||||
|
||||
## Data Flow: Drain
|
||||
|
||||
```
|
||||
Application
|
||||
│
|
||||
│ client.drain() or subscriber.drain()
|
||||
│
|
||||
▼
|
||||
Client::drain / Subscriber::drain
|
||||
│ sends Command::Drain { sid: None } (whole client)
|
||||
│ or Command::Drain { sid: Some(n) } (single subscription)
|
||||
│
|
||||
▼
|
||||
ConnectionHandler::handle_command(Command::Drain)
|
||||
│ if sid is Some:
|
||||
│ pushes sid to drain_pings
|
||||
│ sends ClientOp::Unsubscribe { sid, max: None }
|
||||
│ if sid is None (whole client):
|
||||
│ sets is_draining = true
|
||||
│ emits Event::Draining
|
||||
│ for each subscription: drain_pings.push(sid), Unsubscribe
|
||||
│ sends ClientOp::Ping (to flush the UNSUB messages)
|
||||
│
|
||||
▼
|
||||
ProcessFut::poll (main loop)
|
||||
│ processes any remaining server messages
|
||||
│ removes drained subscriptions from HashMap
|
||||
│ if is_draining: returns ExitReason::Closed
|
||||
│
|
||||
▼
|
||||
ConnectionHandler exits, emits Event::Closed
|
||||
```
|
||||
|
||||
## Main Processing Loop
|
||||
|
||||
The `ConnectionHandler::process` method implements the core event loop via a custom `Future` (`ProcessFut`):
|
||||
|
||||
```rust
|
||||
impl Future for ProcessFut<'_> {
|
||||
type Output = ExitReason;
|
||||
|
||||
fn poll(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Self::Output> {
|
||||
// 1. Check ping interval — send PING if due, disconnect if too many pending
|
||||
while self.handler.ping_interval.poll_tick(cx).is_ready() {
|
||||
if let Poll::Ready(exit) = self.ping() { return Poll::Ready(exit); }
|
||||
}
|
||||
|
||||
// 2. Read all available server operations
|
||||
loop {
|
||||
match self.handler.connection.poll_read_op(cx) {
|
||||
Poll::Pending => break,
|
||||
Poll::Ready(Ok(Some(server_op))) => self.handler.handle_server_op(server_op),
|
||||
Poll::Ready(Ok(None)) => return Poll::Ready(ExitReason::Disconnected(None)),
|
||||
Poll::Ready(Err(err)) => return Poll::Ready(ExitReason::Disconnected(Some(err))),
|
||||
}
|
||||
}
|
||||
|
||||
// 3. Clean up drained subscriptions
|
||||
while let Some(sid) = self.handler.drain_pings.pop_front() {
|
||||
self.handler.subscriptions.remove(&sid);
|
||||
}
|
||||
|
||||
// 4. If draining, exit
|
||||
if self.handler.is_draining { return Poll::Ready(ExitReason::Closed); }
|
||||
|
||||
// 5. Process client commands (batch of up to 16)
|
||||
// while write buffer not full
|
||||
loop {
|
||||
while !self.handler.connection.is_write_buf_full() {
|
||||
match receiver.poll_recv_many(cx, recv_buf, 16) {
|
||||
Poll::Pending => break,
|
||||
Poll::Ready(1..) => { for cmd in recv_buf.drain(..) { handler.handle_command(cmd); } }
|
||||
Poll::Ready(0) => return Poll::Ready(ExitReason::Closed),
|
||||
}
|
||||
}
|
||||
|
||||
// 6. Write pending data to stream
|
||||
match self.handler.connection.poll_write(cx) {
|
||||
Poll::Pending => break,
|
||||
Poll::Ready(Ok(())) => continue, // write buffer empty, try more commands
|
||||
Poll::Ready(Err(err)) => return Poll::Ready(ExitReason::Disconnected(Some(err))),
|
||||
}
|
||||
}
|
||||
|
||||
// 7. Flush stream and notify observers
|
||||
match self.handler.connection.poll_flush(cx) { ... }
|
||||
|
||||
// 8. Check for forced reconnect
|
||||
if mem::take(&mut self.handler.should_reconnect) {
|
||||
return Poll::Ready(ExitReason::ReconnectRequested);
|
||||
}
|
||||
|
||||
Poll::Pending
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Exit Reasons
|
||||
|
||||
The main loop exits for three reasons:
|
||||
|
||||
| Reason | Action |
|
||||
|--------|--------|
|
||||
| `Disconnected(Option<io::Error>)` | Attempt reconnection via `handle_disconnect()` |
|
||||
| `ReconnectRequested` | Force reconnect (user-triggered) |
|
||||
| `Closed` | Connection handler terminates, emit `Event::Closed` |
|
||||
|
||||
On disconnection, `handle_disconnect()` is called which:
|
||||
1. Resets `pending_pings` to 0
|
||||
2. Emits `Event::Disconnected`
|
||||
3. Updates connection state to `Disconnected`
|
||||
4. Calls `handle_reconnect()` which uses `Connector::connect()`
|
||||
5. On successful reconnect, re-subscribes all active subscriptions
|
||||
6. Re-subscribes the multiplexer wildcard if present
|
||||
|
||||
## Slow Consumer Handling
|
||||
|
||||
When a subscription's `mpsc::Sender` channel is full (the application isn't consuming messages fast enough):
|
||||
|
||||
1. `try_send` returns `TrySendError::Full`
|
||||
2. The `ConnectionHandler` emits `Event::SlowConsumer(sid)`
|
||||
3. The message is **dropped** (not queued)
|
||||
4. The subscription remains active
|
||||
|
||||
When a subscription's receiver is dropped (application closed the stream):
|
||||
|
||||
1. `try_send` returns `TrySendError::Closed`
|
||||
2. The subscription is removed from the HashMap
|
||||
3. An `UNSUB` command is sent to the server
|
||||
|
||||
## Ping/Pong Health Check
|
||||
|
||||
The `ConnectionHandler` maintains a periodic PING interval (default 60 seconds):
|
||||
|
||||
1. `ping_interval` fires every N seconds
|
||||
2. A `ClientOp::Ping` is enqueued
|
||||
3. `pending_pings` counter increments
|
||||
4. If `pending_pings > MAX_PENDING_PINGS (2)`, the connection is considered dead
|
||||
5. When `ServerOp::Pong` is received, `pending_pings` decrements
|
||||
6. Any server operation resets the ping interval timer
|
||||
|
||||
## Batched Command Processing
|
||||
|
||||
Commands from the `Client` are received in batches of up to 16 (`RECV_CHUNK_SIZE`) using `poll_recv_many`. This amortizes the cost of waking the task and enables pipelining multiple operations (e.g., publishing many messages) in a single poll cycle.
|
||||
@@ -0,0 +1,277 @@
|
||||
# Connection and Reconnection
|
||||
|
||||
This document covers how connections are established, TLS handling, the server pool, and the reconnection mechanism.
|
||||
|
||||
## Connector
|
||||
|
||||
**Location**: `connector.rs`
|
||||
|
||||
The `Connector` manages the server pool and handles connection establishment and reconnection.
|
||||
|
||||
```rust
|
||||
pub(crate) struct Connector {
|
||||
servers: Vec<Server>, // Server pool with per-server metadata
|
||||
options: ConnectorOptions, // Connection configuration
|
||||
connect_stats: Arc<Statistics>, // Shared statistics
|
||||
attempts: usize, // Global reconnection attempt counter
|
||||
events_tx: mpsc::Sender<Event>, // Event channel
|
||||
state_tx: watch::Sender<State>, // Connection state watcher
|
||||
max_payload: Arc<AtomicUsize>, // Server's max payload
|
||||
last_info: ServerInfo, // Last known server info
|
||||
}
|
||||
```
|
||||
|
||||
### Server Pool
|
||||
|
||||
Each server in the pool carries metadata:
|
||||
|
||||
```rust
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct Server {
|
||||
pub addr: ServerAddr,
|
||||
pub failed_attempts: usize, // Consecutive failed attempts
|
||||
pub did_connect: bool, // Ever successfully connected?
|
||||
pub is_discovered: bool, // Discovered via INFO, not user-configured
|
||||
pub last_error: Option<String>, // Last connection error
|
||||
}
|
||||
```
|
||||
|
||||
### ConnectorOptions
|
||||
|
||||
```rust
|
||||
pub(crate) struct ConnectorOptions {
|
||||
pub tls_required: bool,
|
||||
pub certificates: Vec<PathBuf>,
|
||||
pub client_cert: Option<PathBuf>,
|
||||
pub client_key: Option<PathBuf>,
|
||||
pub tls_client_config: Option<rustls::ClientConfig>,
|
||||
pub tls_first: bool,
|
||||
pub auth: Auth,
|
||||
pub no_echo: bool,
|
||||
pub connection_timeout: Duration, // Default: 5 seconds
|
||||
pub name: Option<String>,
|
||||
pub ignore_discovered_servers: bool,
|
||||
pub retain_servers_order: bool,
|
||||
pub read_buffer_capacity: u16, // Default: 65535
|
||||
pub reconnect_delay_callback: Arc<dyn Fn(usize) -> Duration>,
|
||||
pub auth_callback: Option<CallbackArg1<Vec<u8>, Result<Auth, AuthError>>>,
|
||||
pub max_reconnects: Option<usize>,
|
||||
pub local_address: Option<SocketAddr>,
|
||||
pub reconnect_to_server_callback: Option<ReconnectToServerCallback>,
|
||||
}
|
||||
```
|
||||
|
||||
## Connection Establishment Flow
|
||||
|
||||
```
|
||||
Connector::try_connect_to_server(addr)
|
||||
│
|
||||
├── 1. DNS resolution
|
||||
│ server_addr.socket_addrs()
|
||||
│
|
||||
├── 2. For each resolved address:
|
||||
│ │
|
||||
│ ├── 2a. Connect with timeout
|
||||
│ │ tokio::time::timeout(connection_timeout, try_connect_to(socket_addr, ...))
|
||||
│ │
|
||||
│ └── 2b. try_connect_to():
|
||||
│ │
|
||||
│ ├── Select transport:
|
||||
│ │ ├── "ws" → WebSocket (tokio_websockets)
|
||||
│ │ ├── "wss" → WebSocket over TLS
|
||||
│ │ └── default → TCP (TcpStream)
|
||||
│ │
|
||||
│ ├── Optional: bind to local_address
|
||||
│ ├── Set TCP_NODELAY
|
||||
│ ├── Create Connection with read_buffer_capacity
|
||||
│ │
|
||||
│ ├── If tls_first: upgrade to TLS before INFO
|
||||
│ │
|
||||
│ ├── Read INFO from server
|
||||
│ │
|
||||
│ ├── If TLS required (by option, server, or URL scheme):
|
||||
│ │ upgrade to TLS (rustls)
|
||||
│ │
|
||||
│ ├── Discover servers from INFO.connect_urls
|
||||
│ │ (unless ignore_discovered_servers)
|
||||
│ │
|
||||
│ ├── Build ConnectInfo with auth:
|
||||
│ │ ├── username/password (from Auth or URL)
|
||||
│ │ ├── token (from Auth)
|
||||
│ │ ├── nkey + signed nonce (feature: nkeys)
|
||||
│ │ ├── JWT + signature callback (feature: nkeys)
|
||||
│ │ └── auth_callback (custom async callback)
|
||||
│ │
|
||||
│ ├── Send CONNECT + PING
|
||||
│ │
|
||||
│ └── Wait for response:
|
||||
│ ├── -ERR (authorization violation) → error
|
||||
│ ├── PONG or +OK → success
|
||||
│ └── EOF → error
|
||||
│
|
||||
└── 3. On success:
|
||||
├── Reset attempt counter
|
||||
├── Increment connects statistic
|
||||
├── Emit Event::Connected
|
||||
├── Update State::Connected
|
||||
├── Store max_payload
|
||||
├── Update per-server metadata (did_connect, failed_attempts)
|
||||
└── Return (ServerInfo, Connection)
|
||||
```
|
||||
|
||||
## TLS Handling
|
||||
|
||||
The client supports three TLS modes:
|
||||
|
||||
### 1. Standard TLS (INFO → TLS)
|
||||
Default behavior. The client receives the `INFO` message in plaintext, then upgrades to TLS if:
|
||||
- `tls_required` option is set
|
||||
- Server's `INFO.tls_required` is true
|
||||
- URL scheme is `tls://`
|
||||
|
||||
### 2. TLS First (TLS → INFO)
|
||||
When `ConnectOptions::tls_first()` is enabled, the client establishes TLS before reading INFO. This requires the server to have `handshake_first` enabled. Useful for environments where plaintext INFO is not acceptable.
|
||||
|
||||
### 3. WebSocket TLS
|
||||
For `wss://` URLs, TLS is handled by the WebSocket library (`tokio-websockets`) directly, not by the client's TLS layer.
|
||||
|
||||
### TLS Configuration
|
||||
The client uses `rustls` via `tokio-rustls`. Configuration steps:
|
||||
1. Load root certificates from system store (`rustls-native-certs`)
|
||||
2. Optionally add custom root certificates from PEM files
|
||||
3. Optionally configure client certificate and key for mTLS
|
||||
4. Optionally pass a custom `rustls::ClientConfig`
|
||||
|
||||
Crypto backend is selectable via feature flags:
|
||||
- `ring` (default)
|
||||
- `aws-lc-rs`
|
||||
- `fips` (requires aws-lc-rs)
|
||||
|
||||
## Reconnection
|
||||
|
||||
### Reconnection Trigger
|
||||
|
||||
Reconnection is triggered when:
|
||||
1. I/O error during read or write (`ExitReason::Disconnected`)
|
||||
2. Too many pending PINGs (no PONG received)
|
||||
3. User calls `Client::force_reconnect()` (`ExitReason::ReconnectRequested`)
|
||||
|
||||
### Reconnection Flow
|
||||
|
||||
```
|
||||
ConnectionHandler::handle_disconnect()
|
||||
│
|
||||
├── Reset pending_pings to 0
|
||||
├── Emit Event::Disconnected
|
||||
├── Update State::Disconnected
|
||||
│
|
||||
└── handle_reconnect()
|
||||
│
|
||||
└── Connector::connect()
|
||||
│
|
||||
└── Loop: try_connect()
|
||||
│
|
||||
├── If reconnect_to_server_callback is set:
|
||||
│ │ Call callback with (server_pool, server_info)
|
||||
│ │ If returns Some(ReconnectToServer):
|
||||
│ │ Validate server is in pool
|
||||
│ │ Use callback's delay or default backoff
|
||||
│ │ Try connecting to selected server
|
||||
│ └── If None or invalid: fall through to default
|
||||
│
|
||||
├── Default selection:
|
||||
│ ├── Shuffle servers (unless retain_servers_order)
|
||||
│ ├── Sort by failed_attempts (ascending)
|
||||
│ └── Try each server in order
|
||||
│
|
||||
├── For each server:
|
||||
│ ├── Increment attempts counter
|
||||
│ ├── Check max_reconnects limit
|
||||
│ ├── Apply reconnect delay (exponential backoff)
|
||||
│ └── try_connect_to_server(addr)
|
||||
│
|
||||
├── On success:
|
||||
│ ├── Reset attempts to 0
|
||||
│ ├── Re-subscribe all active subscriptions
|
||||
│ │ (filter out closed subscription channels)
|
||||
│ ├── Re-subscribe multiplexer wildcard
|
||||
│ └── Return (ServerInfo, Connection)
|
||||
│
|
||||
└── On failure:
|
||||
├── Update per-server metadata (failed_attempts, last_error)
|
||||
├── Auth errors → propagate immediately
|
||||
└── Other errors → continue to next server
|
||||
```
|
||||
|
||||
### Exponential Backoff
|
||||
|
||||
Default reconnect delay function:
|
||||
|
||||
```rust
|
||||
fn reconnect_delay_callback_default(attempts: usize) -> Duration {
|
||||
if attempts <= 1 {
|
||||
Duration::from_millis(0)
|
||||
} else {
|
||||
let exp: u32 = (attempts - 1).try_into().unwrap_or(u32::MAX);
|
||||
let max = Duration::from_secs(4);
|
||||
cmp::min(Duration::from_millis(2_u64.saturating_pow(exp)), max)
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
| Attempt | Delay |
|
||||
|---------|-------|
|
||||
| 1 | 0ms |
|
||||
| 2 | 0ms |
|
||||
| 3 | 2ms |
|
||||
| 4 | 4ms |
|
||||
| 5 | 8ms |
|
||||
| ... | ... |
|
||||
| 13 | 4096ms |
|
||||
| 14+ | 4000ms (capped) |
|
||||
|
||||
Custom delay functions can be provided via `ConnectOptions::reconnect_delay_callback()`.
|
||||
|
||||
### Server Pool Updates
|
||||
|
||||
The server pool is dynamic:
|
||||
|
||||
1. **Initial pool**: from `connect()` / `ConnectOptions::connect()` URL(s)
|
||||
2. **Discovered servers**: added from `INFO.connect_urls` on each connection (unless `ignore_discovered_servers` is set)
|
||||
3. **Runtime updates**: via `Client::set_server_pool()` — replaces the entire pool while preserving per-server state for servers that appear in both old and new pools
|
||||
4. **Order**: servers are shuffled by default (random selection), unless `retain_servers_order` is set
|
||||
|
||||
### Max Reconnects
|
||||
|
||||
The `max_reconnects` option limits total reconnection attempts:
|
||||
- `None` or `0` → unlimited (default)
|
||||
- `Some(n)` → give up after `n` total attempts
|
||||
- Counter is reset on successful connection and when `set_server_pool()` is called
|
||||
|
||||
## ConnectOptions Defaults
|
||||
|
||||
| Option | Default |
|
||||
|--------|---------|
|
||||
| `connection_timeout` | 5 seconds |
|
||||
| `ping_interval` | 60 seconds |
|
||||
| `sender_capacity` | 2048 |
|
||||
| `subscription_capacity` | 65536 |
|
||||
| `inbox_prefix` | `"_INBOX"` |
|
||||
| `request_timeout` | 10 seconds |
|
||||
| `retry_on_initial_connect` | false |
|
||||
| `ignore_discovered_servers` | false |
|
||||
| `retain_servers_order` | false |
|
||||
| `read_buffer_capacity` | 65535 |
|
||||
| `skip_subject_validation` | false |
|
||||
| `no_echo` | false |
|
||||
| `tls_required` | false |
|
||||
| `tls_first` | false |
|
||||
| `max_reconnects` | None (unlimited) |
|
||||
|
||||
## Background Connection
|
||||
|
||||
When `ConnectOptions::retry_on_initial_connect()` is enabled, the `connect()` function returns a `Client` immediately, before the connection is established. The connection is established in a background Tokio task. This means:
|
||||
- `client.server_info()` returns `ServerInfo::default()` until connected
|
||||
- `client.connection_state()` returns `State::Pending`
|
||||
- Operations like `publish()` will queue in the command channel
|
||||
- The `Client` becomes usable once the background task connects
|
||||
@@ -0,0 +1,472 @@
|
||||
# JetStream Internals
|
||||
|
||||
This document covers the JetStream subsystem — how it provides stream-based messaging with persistence, consumer management, and higher-level APIs like KV and Object Store.
|
||||
|
||||
## JetStream Context
|
||||
|
||||
**Location**: `jetstream/context.rs`
|
||||
|
||||
The `Context` is the entry point to the JetStream API. It wraps a `Client` and provides stream management, publishing, and consumer operations.
|
||||
|
||||
```rust
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct Context {
|
||||
pub(crate) client: Client,
|
||||
pub(crate) prefix: String, // API subject prefix (default: "$JS.API")
|
||||
pub(crate) timeout: Duration, // Default request timeout
|
||||
pub(crate) max_ack_semaphore: Arc<Semaphore>, // Limits in-flight ack waits
|
||||
pub(crate) ack_sender: mpsc::Sender<(oneshot::Receiver<Message>, OwnedSemaphorePermit)>,
|
||||
pub(crate) backpressure_on_inflight: bool,
|
||||
}
|
||||
```
|
||||
|
||||
### Context Creation
|
||||
|
||||
```rust
|
||||
// Default context (prefix = "$JS.API")
|
||||
let jetstream = async_nats::jetstream::new(client);
|
||||
|
||||
// With domain (prefix = "$JS.hub.API")
|
||||
let jetstream = async_nats::jetstream::with_domain(client, "hub");
|
||||
|
||||
// With custom prefix
|
||||
let jetstream = async_nats::jetstream::with_prefix(client, "JS.acc@hub.API");
|
||||
|
||||
// Builder pattern for more options
|
||||
let jetstream = async_nats::jetstream::Context::builder(client)
|
||||
.domain("hub")
|
||||
.prefix("$JS.API")
|
||||
.timeout(Duration::from_secs(30))
|
||||
.max_ack_pending(256)
|
||||
.backpressure_on_inflight(true)
|
||||
.build();
|
||||
```
|
||||
|
||||
### JetStream API Subject Convention
|
||||
|
||||
All JetStream API calls are request-response messages sent to subjects following the pattern:
|
||||
|
||||
```
|
||||
$JS.API.<operation>.<stream-name>[.<consumer-name>]
|
||||
```
|
||||
|
||||
Examples:
|
||||
- `$JS.API.STREAM.CREATE.events` — create stream "events"
|
||||
- `$JS.API.STREAM.INFO.events` — get stream info
|
||||
- `$JS.API.CONSUMER.DURABLE.CREATE.events.myconsumer` — create durable consumer
|
||||
- `$JS.API.CONSUMER.MSG.NEXT.events.myconsumer` — pull next message
|
||||
|
||||
With a domain, the prefix changes to `$JS.<domain>.API`.
|
||||
|
||||
## Stream Management
|
||||
|
||||
**Location**: `jetstream/stream.rs`
|
||||
|
||||
### Stream Config
|
||||
|
||||
```rust
|
||||
#[derive(Serialize, Deserialize, Debug, Clone, PartialEq, Eq)]
|
||||
pub struct Config {
|
||||
pub name: String,
|
||||
pub subjects: Vec<String>, // Subject filter
|
||||
pub retention: RetentionPolicy, // Limits, Interest, WorkQueue
|
||||
pub max_consumers: i32,
|
||||
pub max_messages: i64, // Per-stream message limit
|
||||
pub max_messages_per_subject: i64,
|
||||
pub max_bytes: i64, // Per-stream byte limit
|
||||
pub max_age: Duration, // Message TTL
|
||||
pub max_message_size: Option<i32>, // Max individual message size
|
||||
pub storage: StorageType, // File or Memory
|
||||
pub num_replicas: usize,
|
||||
pub no_ack: bool, // Don't require ack
|
||||
pub discard: DiscardPolicy, // Old or New
|
||||
pub duplicate_window: Duration,
|
||||
pub allow_rollup_hdrs: bool,
|
||||
pub allow_direct: bool,
|
||||
pub mirror: Option<External>,
|
||||
pub sources: Vec<External>,
|
||||
pub sealed: bool,
|
||||
pub compression: Option<Compression>, // server_2_10+
|
||||
pub first_sequence: Option<u64>, // server_2_11+
|
||||
pub subject_transform: Option<SubjectTransform>, // server_2_12+
|
||||
pub metadata: Option<HashMap<String, String>>, // server_2_10+
|
||||
pub placement: Option<Placement>,
|
||||
pub republish: Option<RePublish>,
|
||||
}
|
||||
```
|
||||
|
||||
### Stream Operations
|
||||
|
||||
Via `Context`:
|
||||
|
||||
| Method | API Subject | Description |
|
||||
|--------|------------|-------------|
|
||||
| `create_stream(config)` | `STREAM.CREATE.<name>` | Create a new stream |
|
||||
| `get_stream(name)` | `STREAM.INFO.<name>` | Get existing stream |
|
||||
| `get_or_create_stream(config)` | `STREAM.INFO` → `STREAM.CREATE` | Get or create |
|
||||
| `delete_stream(name)` | `STREAM.DELETE.<name>` | Delete a stream |
|
||||
| `update_stream(name, config)` | `STREAM.UPDATE.<name>` | Update stream config |
|
||||
| `purge_stream(name)` | `STREAM.PURGE.<name>` | Purge all messages |
|
||||
| `streams()` | `STREAM.LIST` | List all streams (paged iterator) |
|
||||
| `stream_names()` | `STREAM.NAMES` | List stream names (paged iterator) |
|
||||
| `account_info()` | `ACCOUNT.INFO` | Get account info |
|
||||
|
||||
Via `Stream`:
|
||||
|
||||
| Method | API Subject | Description |
|
||||
|--------|------------|-------------|
|
||||
| `info()` | `STREAM.INFO.<name>` | Refresh stream info |
|
||||
| `purge()` | `STREAM.PURGE.<name>` | Purge messages |
|
||||
| `delete()` | `STREAM.DELETE.<name>` | Delete this stream |
|
||||
| `update(config)` | `STREAM.UPDATE.<name>` | Update config |
|
||||
| `get_raw_message(seq)` | `STREAM.MSG.GET.<name>` | Get message by sequence (stored mode) |
|
||||
| `get_last_message(subject)` | `STREAM.MSG.GET.<name>` | Get last message for subject (stored mode) |
|
||||
| `direct_get_last(subject)` | `DIRECT.GET.<name>` | Direct get last (bypasses RAA) |
|
||||
| `direct_get(seq)` | `DIRECT.GET.<name>` | Direct get by sequence |
|
||||
| `delete_message(seq)` | `STREAM.MSG.DELETE.<name>` | Delete a specific message |
|
||||
| `create_consumer(config)` | `CONSUMER.CREATE.<stream>` | Create consumer |
|
||||
| `get_or_create_consumer(name, config)` | `CONSUMER.DURABLE.CREATE.<stream>.<name>` | Get or create durable |
|
||||
| `get_consumer(name)` | `CONSUMER.INFO.<stream>.<name>` | Get existing consumer |
|
||||
|
||||
### Stream Info
|
||||
|
||||
```rust
|
||||
#[derive(Serialize, Deserialize, Debug, Clone)]
|
||||
pub struct Info {
|
||||
pub config: Config,
|
||||
pub created: DateTime,
|
||||
pub state: State, // Messages, bytes, first/last sequence, consumer count
|
||||
pub cluster: Option<ClusterInfo>,
|
||||
pub timestamp: DateTime,
|
||||
pub leader: Option<String>,
|
||||
pub subjects: Option<HashMap<String, u64>>, // Subject → message count
|
||||
}
|
||||
```
|
||||
|
||||
### Paged List Operations
|
||||
|
||||
Stream and consumer listing uses a paged iterator pattern:
|
||||
|
||||
```rust
|
||||
// streams() returns an iterator that automatically pages
|
||||
let mut streams = jetstream.streams();
|
||||
while let Some(stream) = streams.next().await {
|
||||
let stream = stream?;
|
||||
// process stream
|
||||
}
|
||||
|
||||
// stream_names() similarly pages
|
||||
let mut names = jetstream.stream_names();
|
||||
while let Some(name) = names.next().await {
|
||||
println!("{}", name?);
|
||||
}
|
||||
```
|
||||
|
||||
The paged iterator sends an initial request with `offset: 0` and continues fetching pages until no more results are returned.
|
||||
|
||||
## Publishing
|
||||
|
||||
**Location**: `jetstream/context.rs`, `jetstream/publish.rs`
|
||||
|
||||
### Publish
|
||||
|
||||
```rust
|
||||
// Basic publish (fire-and-forget)
|
||||
jetstream.publish("events.data", "payload".into()).await?;
|
||||
|
||||
// Publish with custom message builder
|
||||
jetstream.publish_message(
|
||||
jetstream::message::PublishMessage::build()
|
||||
.payload("data".into())
|
||||
.message_id("unique-id") // Nats-Msg-Id header for dedup
|
||||
.expected_last_message_id("prev") // Nats-Expected-Last-Msg-Id
|
||||
.expected_last_sequence(42) // Nats-Expected-Last-Sequence
|
||||
.expected_last_subject_sequence("events", 10) // Per-subject sequence
|
||||
.header("Custom", "Value")
|
||||
).await?;
|
||||
```
|
||||
|
||||
### PublishAck
|
||||
|
||||
When a message is published to a JetStream stream, the server responds with a `PublishAck`:
|
||||
|
||||
```rust
|
||||
#[derive(Serialize, Deserialize, Debug, Clone, PartialEq, Eq)]
|
||||
pub struct PublishAck {
|
||||
pub stream: String,
|
||||
pub sequence: u64,
|
||||
pub domain: Option<String>,
|
||||
pub duplicate: bool,
|
||||
}
|
||||
```
|
||||
|
||||
### PublishAckFuture
|
||||
|
||||
Publishing returns a `PublishAckFuture` that resolves to `PublishAck`. The future uses a semaphore (`max_ack_semaphore`) to limit in-flight ack waits and prevent backpressure issues.
|
||||
|
||||
When `backpressure_on_inflight` is enabled, the publish operation blocks if there are too many pending acks, preventing the command channel from filling up with unbounded publish operations.
|
||||
|
||||
### Idempotent Publishing
|
||||
|
||||
Headers for exactly-once semantics:
|
||||
|
||||
| Header | Purpose |
|
||||
|--------|---------|
|
||||
| `Nats-Msg-Id` | Message ID for deduplication within the stream's duplicate window |
|
||||
| `Nats-Expected-Last-Msg-Id` | Expected last message ID (conditional publish) |
|
||||
| `Nats-Expected-Last-Sequence` | Expected last sequence number |
|
||||
| `Nats-Expected-Last-Subject-Sequence` | Expected last sequence for a specific subject |
|
||||
|
||||
## Consumers
|
||||
|
||||
**Location**: `jetstream/consumer/`
|
||||
|
||||
### Consumer Types
|
||||
|
||||
| Type | Description |
|
||||
|------|-------------|
|
||||
| `PullConsumer` | Client pulls messages on demand |
|
||||
| `PushConsumer` | Server pushes messages to a delivery subject |
|
||||
| `OrderedConsumer` | Push consumer with automatic re-creation on failure |
|
||||
|
||||
### Consumer Config
|
||||
|
||||
```rust
|
||||
#[derive(Serialize, Deserialize, Debug, Clone, PartialEq, Eq)]
|
||||
pub struct Config {
|
||||
pub name: Option<String>,
|
||||
pub durable_name: Option<String>,
|
||||
pub description: Option<String>,
|
||||
pub deliver_subject: Option<String>, // Push consumers only
|
||||
pub ack_policy: AckPolicy,
|
||||
pub ack_wait: Duration,
|
||||
pub max_deliver: i64,
|
||||
pub max_ack_pending: i32,
|
||||
pub max_waiting: i32, // Pull consumers only
|
||||
pub filter_subject: Option<String>,
|
||||
pub replay_policy: ReplayPolicy,
|
||||
pub sample_frequency: Option<i8>,
|
||||
pub max_batch: i32, // Pull consumers
|
||||
pub max_expires: Duration, // Pull consumers
|
||||
pub inactive_threshold: Duration,
|
||||
pub flow_control: bool, // Push consumers
|
||||
pub heartbeat: Option<Duration>, // Push consumers
|
||||
pub backoff: Vec<Duration>,
|
||||
pub deliver_group: Option<String>,
|
||||
pub num_replicas: usize,
|
||||
pub mem_storage: bool,
|
||||
pub metadata: Option<HashMap<String, String>>,
|
||||
pub ack_markers: Option<Vec<String>>, // server_2_12+
|
||||
}
|
||||
```
|
||||
|
||||
### Pull Consumer
|
||||
|
||||
**Location**: `jetstream/consumer/pull.rs`
|
||||
|
||||
Pull consumers require explicit requests for messages:
|
||||
|
||||
```rust
|
||||
// Batch request
|
||||
let mut messages = consumer.messages().await?.take(100);
|
||||
while let Some(message) = messages.next().await {
|
||||
let message = message?;
|
||||
message.ack().await?;
|
||||
}
|
||||
|
||||
// Sequence-based batch
|
||||
let mut batches = consumer.sequence(50)?.take(10);
|
||||
while let Some(mut batch) = batches.try_next().await? {
|
||||
while let Some(Ok(message)) = batch.next().await {
|
||||
message.ack().await?;
|
||||
}
|
||||
}
|
||||
|
||||
// Single message fetch
|
||||
let message = consumer.fetch().await?;
|
||||
```
|
||||
|
||||
Pull requests are sent to: `$JS.API.CONSUMER.MSG.NEXT.<stream>.<consumer>`
|
||||
|
||||
The request payload is JSON:
|
||||
```json
|
||||
{"batch": 10, "expires": 5000, "no_wait": false}
|
||||
```
|
||||
|
||||
### Push Consumer
|
||||
|
||||
**Location**: `jetstream/consumer/push.rs`
|
||||
|
||||
Push consumers receive messages automatically on a delivery subject. The client subscribes to the delivery subject and processes messages as they arrive.
|
||||
|
||||
Features:
|
||||
- **Flow control** — server sends flow control messages, client responds to maintain delivery rate
|
||||
- **Heartbeats** — idle heartbeats (status code 100) when no messages are available
|
||||
- **Ordered consumers** — automatically recreated on delivery failures with correct sequence positioning
|
||||
|
||||
### Acknowledgment
|
||||
|
||||
**Location**: `jetstream/message.rs`
|
||||
|
||||
JetStream messages support multiple acknowledgment types:
|
||||
|
||||
```rust
|
||||
pub enum AckKind {
|
||||
Ack, // Ack (message processed)
|
||||
Nack, // Nak (re-deliver)
|
||||
Progress, // Progress (still working)
|
||||
Next, // Next (ack + pull next)
|
||||
Term, // Term (don't redeliver, remove from stream)
|
||||
All, // Ack all messages up to this sequence
|
||||
}
|
||||
```
|
||||
|
||||
Methods on JetStream `Message`:
|
||||
- `ack()` — simple acknowledgment
|
||||
- `ack_with(kind)` — acknowledgment with specific type
|
||||
- `double_ack()` — exactly-once ack (ACK + separate ack message)
|
||||
- `nack()` — negative acknowledgment (request redelivery)
|
||||
- `in_progress()` — progress indicator
|
||||
- `term()` — terminate message (no redelivery)
|
||||
|
||||
## JetStream Message
|
||||
|
||||
**Location**: `jetstream/message.rs`
|
||||
|
||||
JetStream messages wrap core `Message` with metadata extracted from headers:
|
||||
|
||||
```rust
|
||||
#[derive(Debug)]
|
||||
pub struct Message {
|
||||
pub message: crate::Message, // The underlying NATS message
|
||||
pub context: Context, // JetStream context for acking
|
||||
pub ack_pending: Arc<AtomicU64>, // Pending ack counter
|
||||
}
|
||||
|
||||
impl Message {
|
||||
pub fn info(&self) -> Result<Info, MessageInfoError> // Parse message info from headers
|
||||
pub async fn ack(&self) -> Result<(), AckError>
|
||||
pub async fn ack_with(&self, kind: AckKind) -> Result<(), AckError>
|
||||
pub async fn double_ack(&self) -> Result<(), AckError>
|
||||
pub async fn nack(&self) -> Result<(), AckError>
|
||||
pub async fn in_progress(&self) -> Result<(), AckError>
|
||||
pub async fn term(&self) -> Result<(), AckError>
|
||||
}
|
||||
```
|
||||
|
||||
Message info is extracted from the `HMSG` headers:
|
||||
- `Nats-Stream` — stream name
|
||||
- `Nats-Consumer` — consumer name
|
||||
- `Nats-Delivered` — delivery count
|
||||
- `Nats-Sequence` — stream sequence
|
||||
- `Nats-Time-Stamp` — timestamp
|
||||
- `Nats-Subject` — original subject
|
||||
- `Nats-Pending-Messages` / `Nats-Pending-Bytes` — pending counts
|
||||
|
||||
## Key-Value Store
|
||||
|
||||
**Location**: `jetstream/kv/`
|
||||
|
||||
The KV store is a JetStream-based key-value API. Each bucket maps to a JetStream stream with specific configuration:
|
||||
|
||||
```rust
|
||||
// Create a KV store
|
||||
let kv = jetstream
|
||||
.create_key_value(async_nats::jetstream::kv::Config {
|
||||
bucket: "my_bucket".to_string(),
|
||||
history: 5, // Max history per key (1-64)
|
||||
ttl: Duration::from_secs(3600), // Key TTL
|
||||
max_bytes: 1024 * 1024, // Max bucket size
|
||||
storage: StorageType::File,
|
||||
replicas: 1,
|
||||
..Default::default()
|
||||
})
|
||||
.await?;
|
||||
```
|
||||
|
||||
Under the hood:
|
||||
- Each key is stored as a message with subject `$KV.<bucket>.<key>`
|
||||
- Keys support wildcard patterns (`$KV.bucket.prefix.*`)
|
||||
- History is managed via stream `max_messages_per_subject`
|
||||
- TTL is managed via stream `max_age`
|
||||
- `put(key, value)` publishes to the key subject
|
||||
- `get(key)` reads the last message for the key subject
|
||||
- `delete(key)` publishes an internal delete marker
|
||||
- `purge(key)` uses stream purge API
|
||||
- `watch()` subscribes to key changes and returns a `Watch` stream
|
||||
- `keys()` / `history(key)` list keys and history
|
||||
|
||||
## Object Store
|
||||
|
||||
**Location**: `jetstream/object_store/`
|
||||
|
||||
The Object Store provides large object storage built on JetStream. Objects are chunked and stored across multiple messages in a stream.
|
||||
|
||||
```rust
|
||||
// Create an object store
|
||||
let store = jetstream
|
||||
.create_object_store(async_nats::jetstream::object_store::Config {
|
||||
bucket: "my_objects".to_string(),
|
||||
..Default::default()
|
||||
})
|
||||
.await?;
|
||||
|
||||
// Put an object
|
||||
let info = store.put("file.txt", stream).await?;
|
||||
|
||||
// Get an object
|
||||
let mut object_stream = store.get("file.txt").await?;
|
||||
```
|
||||
|
||||
Under the hood:
|
||||
- Objects are chunked into ~128KB messages
|
||||
- Metadata (object info) is stored as the first "chunk 0" message
|
||||
- Each chunk is a message with subject `$OBJ.<bucket>.<object-nuid>.C<chunk-number>`
|
||||
- Metadata includes: name, description, headers, size, chunks, digest (SHA-256)
|
||||
- `get()` returns a stream of chunks
|
||||
- Links allow referencing one object from another (like symlinks)
|
||||
|
||||
## JetStream Error Codes
|
||||
|
||||
**Location**: `jetstream/errors.rs`
|
||||
|
||||
Standard JetStream error codes returned by the server:
|
||||
|
||||
| Code | Constant | Description |
|
||||
|------|----------|-------------|
|
||||
| 10001 | `NOT_FOUND` | Resource not found |
|
||||
| 10002 | `STREAM_NOT_FOUND` | Stream not found |
|
||||
| 10003 | `CONSUMER_NOT_FOUND` | Consumer not found |
|
||||
| 10004 | `REQUEST_NOT_FOUND` | Request not found |
|
||||
| 10005 | `STREAM_WRONG_LAST_SEQ` | Wrong last sequence |
|
||||
| 10006 | `STREAM_NAME_EXISTS` | Stream already exists |
|
||||
| 10007 | `CONSUMER_NAME_EXISTS` | Consumer already exists |
|
||||
| 10008 | `INSUFFICIENT_RESOURCES` | Insufficient resources |
|
||||
| 10009 | `NO_MESSAGE_FOUND` | No message found |
|
||||
| 10013 | `CONSUMER_EXISTS` | Consumer already exists (duplicate) |
|
||||
| 10014 | `STREAM_NOT_CONFIGURED` | Stream not configured |
|
||||
| 10015 | `CLUSTER_NOT_ACTIVE` | Cluster not active |
|
||||
| 10016 | `CLUSTER_NOT_LEADER` | Not the cluster leader |
|
||||
| 10017 | `CLUSTER_NOT_ENOUGH_PEERS` | Not enough peers |
|
||||
| 10018 | `CLUSTER_INCOMPLETE` | Cluster incomplete |
|
||||
| 10019 | `CONSUMER_DELETED` | Consumer was deleted |
|
||||
| 10020 | `CONSUMER_BAD_ACK` | Bad acknowledgment |
|
||||
| 10021 | `CONSUMER_BAD_SUBJECT` | Bad consumer subject |
|
||||
| 10022 | `CONSUMER_DELETED_DRIFT` | Consumer deleted due to drift |
|
||||
| ... | ... | Additional codes |
|
||||
|
||||
## Account
|
||||
|
||||
**Location**: `jetstream/account.rs`
|
||||
|
||||
The `Account` struct provides information about the JetStream account:
|
||||
|
||||
```rust
|
||||
pub struct Account {
|
||||
pub memory: i64,
|
||||
pub storage: i64,
|
||||
pub streams: i64,
|
||||
pub consumers: i64,
|
||||
pub limits: AccountLimits,
|
||||
}
|
||||
```
|
||||
@@ -0,0 +1,292 @@
|
||||
# Authentication and Security
|
||||
|
||||
This document covers the authentication mechanisms, TLS configuration, and security-related features of the async-nats client.
|
||||
|
||||
## Authentication Methods
|
||||
|
||||
The NATS server supports multiple authentication methods. The client implements all of them.
|
||||
|
||||
### 1. Username/Password
|
||||
|
||||
The simplest authentication method.
|
||||
|
||||
```rust
|
||||
// Via ConnectOptions
|
||||
let client = ConnectOptions::with_user_and_password("user".into(), "pass".into())
|
||||
.connect("nats://localhost")
|
||||
.await?;
|
||||
|
||||
// Via URL
|
||||
let client = connect("nats://user:pass@localhost:4222").await?;
|
||||
```
|
||||
|
||||
These credentials are sent in the `CONNECT` message as `user` and `pass` fields.
|
||||
|
||||
### 2. Token Authentication
|
||||
|
||||
A single token used for authentication.
|
||||
|
||||
```rust
|
||||
let client = ConnectOptions::with_token("my-token".into())
|
||||
.connect("nats://localhost")
|
||||
.await?;
|
||||
```
|
||||
|
||||
Token is sent in the `CONNECT` message as `auth_token` field.
|
||||
|
||||
### 3. NKey Authentication
|
||||
|
||||
NKey-based authentication using Ed25519 key pairs. Requires the `nkeys` feature.
|
||||
|
||||
```rust
|
||||
let seed = "SUANQDPB2RUOE4ETUA26CNX7FUKE5ZZKFCQIIW63OX225F2CO7UEXTM7ZY";
|
||||
let client = ConnectOptions::with_nkey(seed.into())
|
||||
.connect("nats://localhost")
|
||||
.await?;
|
||||
```
|
||||
|
||||
Flow:
|
||||
1. Server sends `INFO` with a `nonce` field
|
||||
2. Client creates a `KeyPair` from the seed
|
||||
3. Client signs the nonce: `key_pair.sign(nonce.as_bytes())`
|
||||
4. Client sends `CONNECT` with `nkey` (public key) and `sig` (Base64URL-encoded signature)
|
||||
5. Server verifies the signature against the public key and nonce
|
||||
|
||||
### 4. JWT Authentication
|
||||
|
||||
User JWT with a signing callback. Requires the `nkeys` feature.
|
||||
|
||||
```rust
|
||||
let key_pair = Arc::new(nkeys::KeyPair::from_seed(seed)?);
|
||||
let jwt = load_jwt().await?;
|
||||
|
||||
let client = ConnectOptions::with_jwt(jwt, move |nonce| {
|
||||
let key_pair = key_pair.clone();
|
||||
async move { key_pair.sign(&nonce).map_err(AuthError::new) }
|
||||
})
|
||||
.connect("nats://localhost")
|
||||
.await?;
|
||||
```
|
||||
|
||||
Flow:
|
||||
1. Server sends `INFO` with a `nonce` field
|
||||
2. Client sends `CONNECT` with `jwt` (user JWT) and `sig` (Base64URL-encoded nonce signature)
|
||||
3. The signing callback is async, allowing integration with external signing services (e.g., HSM)
|
||||
|
||||
### 5. Credentials File
|
||||
|
||||
Combines JWT and NKey from a `.creds` file. Requires the `nkeys` feature.
|
||||
|
||||
```rust
|
||||
// From file
|
||||
let client = ConnectOptions::with_credentials_file("path/to/my.creds")
|
||||
.await?
|
||||
.connect("nats://localhost")
|
||||
.await?;
|
||||
|
||||
// From string
|
||||
let client = ConnectOptions::with_credentials(creds_string)
|
||||
.connect("nats://localhost")
|
||||
.await?;
|
||||
```
|
||||
|
||||
Credentials file format:
|
||||
```
|
||||
-----BEGIN NATS USER JWT-----
|
||||
eyJ0eXAiOiJqd3QiLCJhbGciOiJlZDI1NTE5...
|
||||
------END NATS USER JWT------
|
||||
|
||||
************************* IMPORTANT *************************
|
||||
NKEY Seed printed below can be used sign and prove identity.
|
||||
|
||||
-----BEGIN USER NKEY SEED-----
|
||||
SUAIO3FHUX5PNV2LQIIP7TZ3N4L7TX3W53MQGEIVYFIGA635OZCKEYHFLM
|
||||
------END USER NKEY SEED------
|
||||
```
|
||||
|
||||
**Location**: `auth_utils.rs` handles parsing:
|
||||
- `load_creds(path)` — async file read + parse
|
||||
- `parse_jwt_and_key_from_creds(creds)` — extracts JWT and KeyPair from the string
|
||||
|
||||
### 6. Auth Callback
|
||||
|
||||
A custom async callback that receives the server nonce and returns an `Auth` struct. This is the most flexible mechanism.
|
||||
|
||||
```rust
|
||||
let client = ConnectOptions::with_auth_callback(move |nonce| {
|
||||
async move {
|
||||
let mut auth = Auth::new();
|
||||
auth.username = Some("user".to_string());
|
||||
auth.password = Some("pass".to_string());
|
||||
// Can also set jwt, nkey, signature, token
|
||||
Ok(auth)
|
||||
}
|
||||
})
|
||||
.connect("nats://localhost")
|
||||
.await?;
|
||||
```
|
||||
|
||||
The callback is invoked on each connection/reconnection, allowing dynamic credential refresh (e.g., refreshing JWTs from an auth server).
|
||||
|
||||
### 7. URL-Embedded Credentials
|
||||
|
||||
```rust
|
||||
// Username and password in URL
|
||||
let client = connect("nats://user:pass@localhost:4222").await?;
|
||||
|
||||
// Token in URL (username field)
|
||||
let client = connect("nats://token@localhost:4222").await?;
|
||||
```
|
||||
|
||||
## Auth Struct
|
||||
|
||||
**Location**: `auth.rs`
|
||||
|
||||
The `Auth` struct is a container for all authentication methods. Multiple fields can be set simultaneously:
|
||||
|
||||
```rust
|
||||
#[derive(Clone, Default)]
|
||||
pub struct Auth {
|
||||
pub jwt: Option<String>,
|
||||
pub nkey: Option<String>,
|
||||
pub signature_callback: Option<CallbackArg1<String, Result<String, AuthError>>>,
|
||||
pub signature: Option<Vec<u8>>,
|
||||
pub username: Option<String>,
|
||||
pub password: Option<String>,
|
||||
pub token: Option<String>,
|
||||
}
|
||||
```
|
||||
|
||||
Priority in `Connector::try_connect_to()`:
|
||||
1. Auth callback overrides all other methods
|
||||
2. NKey authentication (if `auth.nkey` is set)
|
||||
3. JWT authentication (if `auth.jwt` is set)
|
||||
4. Username/password/token from `Auth` struct
|
||||
5. Username/password from URL
|
||||
|
||||
## TLS Configuration
|
||||
|
||||
### TLS Modes
|
||||
|
||||
| Mode | When | Description |
|
||||
|------|------|-------------|
|
||||
| None | Default | Plaintext connection |
|
||||
| Standard | `tls_required` or server requires | TLS after INFO |
|
||||
| TLS First | `tls_first` option | TLS before INFO |
|
||||
| WebSocket | `wss://` URL | TLS handled by WebSocket library |
|
||||
|
||||
### TLS Setup
|
||||
|
||||
**Location**: `tls.rs`
|
||||
|
||||
The `config_tls()` function builds a `rustls::ClientConfig`:
|
||||
|
||||
1. Create `RootCertStore` and load native system certificates
|
||||
2. Add custom root certificates from configured PEM files
|
||||
3. Build `ClientConfig` with the chosen crypto provider:
|
||||
- `ring` (default)
|
||||
- `aws-lc-rs`
|
||||
- `fips` (aws-lc-rs in FIPS mode)
|
||||
4. If client certificate + key are configured, add them for mTLS
|
||||
5. If a custom `rustls::ClientConfig` was provided, use it directly
|
||||
|
||||
### TLS First
|
||||
|
||||
```rust
|
||||
let client = ConnectOptions::new()
|
||||
.tls_first()
|
||||
.connect("nats://localhost")
|
||||
.await?;
|
||||
```
|
||||
|
||||
This sets both `tls_first = true` and `tls_required = true`. The client performs TLS handshake before reading the `INFO` message. The server must have `handshake_first: true` in its configuration.
|
||||
|
||||
### Custom TLS Configuration
|
||||
|
||||
```rust
|
||||
let tls_client = rustls::ClientConfig::builder()
|
||||
.with_root_certificates(root_store)
|
||||
.with_no_client_auth();
|
||||
|
||||
let client = ConnectOptions::new()
|
||||
.require_tls(true)
|
||||
.tls_client_config(tls_client)
|
||||
.connect("nats://localhost")
|
||||
.await?;
|
||||
```
|
||||
|
||||
### mTLS (Mutual TLS)
|
||||
|
||||
```rust
|
||||
let client = ConnectOptions::new()
|
||||
.add_root_certificates("ca.pem".into())
|
||||
.add_client_certificate("cert.pem".into(), "key.pem".into())
|
||||
.connect("tls://localhost")
|
||||
.await?;
|
||||
```
|
||||
|
||||
## WebSocket Transport
|
||||
|
||||
Requires the `websockets` feature. Supports `ws://` and `wss://` schemes.
|
||||
|
||||
```rust
|
||||
let client = connect("ws://localhost:8080").await?;
|
||||
let client = connect("wss://localhost:443").await?;
|
||||
```
|
||||
|
||||
Implementation uses `tokio-websockets` with a `WebSocketAdapter` that wraps the WebSocket stream to implement `AsyncRead + AsyncWrite`:
|
||||
|
||||
```rust
|
||||
// WebSocketAdapter bridges WebSocket messages to byte streams
|
||||
pub(crate) struct WebSocketAdapter<T> {
|
||||
pub(crate) inner: WebSocketStream<T>,
|
||||
pub(crate) read_buf: BytesMut, // Buffered incoming WebSocket messages
|
||||
}
|
||||
```
|
||||
|
||||
For `wss://`, TLS is configured within the WebSocket connector, not via the client's TLS layer.
|
||||
|
||||
## Security Considerations
|
||||
|
||||
### Nonce Signing
|
||||
|
||||
The server's `nonce` in the `INFO` message prevents replay attacks:
|
||||
- Each connection gets a unique nonce
|
||||
- The nonce must be signed with the client's private key
|
||||
- The signature is verified server-side against the public key
|
||||
|
||||
### Authorization Violations
|
||||
|
||||
When the server sends `-ERR 'authorization violation'`:
|
||||
- The client parses this as `ServerError::AuthorizationViolation`
|
||||
- The `Connector` immediately propagates this error (does not retry)
|
||||
- The error is converted to `ConnectErrorKind::AuthorizationViolation`
|
||||
|
||||
### Subject Validation
|
||||
|
||||
By default, the client validates subjects for protocol safety:
|
||||
- **Publish subjects**: checked for emptiness and whitespace (can be disabled with `skip_subject_validation`)
|
||||
- **Subscribe subjects**: always checked for emptiness, whitespace, leading/trailing dots, consecutive dots
|
||||
- **Queue group names**: checked for emptiness and whitespace
|
||||
|
||||
The server enforces its own validation, but client-side checks prevent protocol-framing errors.
|
||||
|
||||
### Max Payload Size
|
||||
|
||||
The client checks payload size against the server's `max_payload` before publishing:
|
||||
- For plain messages: `payload.len() > max_payload`
|
||||
- For messages with headers: `headers.wire_len() + payload.len() > max_payload`
|
||||
- Returns `PublishErrorKind::MaxPayloadExceeded` if exceeded
|
||||
|
||||
### No Echo
|
||||
|
||||
When `no_echo` is set, the `CONNECT` message includes `echo: false`. The server will not deliver messages published by this connection back to its own subscriptions. This prevents feedback loops.
|
||||
|
||||
### Lame Duck Mode
|
||||
|
||||
When the server enters lame duck mode (draining for shutdown):
|
||||
1. Server sends `INFO` with `ldm: true`
|
||||
2. Client emits `Event::LameDuckMode`
|
||||
3. Application should gracefully close or reconnect to another server
|
||||
|
||||
The `nats-server` test harness provides `set_lame_duck_mode(server)` for testing this behavior.
|
||||
347
docs/research/references/nats.rs/nats-server/08-test-harness.md
Normal file
347
docs/research/references/nats.rs/nats-server/08-test-harness.md
Normal file
@@ -0,0 +1,347 @@
|
||||
# nats-server Test Harness
|
||||
|
||||
This document covers the `nats-server` crate — a test harness for spawning real NATS server instances in integration tests.
|
||||
|
||||
**Location**: `nats-server/src/lib.rs`
|
||||
**Version**: 0.1.0
|
||||
**License**: Apache-2.0
|
||||
**Dependencies**: `lazy_static`, `regex`, `serde_json`, `nuid`, `rand`, `tokio-retry`
|
||||
|
||||
## What It Is
|
||||
|
||||
The `nats-server` crate is **not** a NATS server implementation. It is a thin test harness that:
|
||||
- Spawns the Go-based `nats-server` binary as a child process
|
||||
- Configures it for test use (dynamic ports, temp storage, log files)
|
||||
- Discovers the client URL from the server's `INFO` protocol message
|
||||
- Cleans up resources (JetStream storage, logs, PID files) on `Drop`
|
||||
- Supports single servers and 3-node clusters
|
||||
|
||||
The actual NATS server must be installed separately (Go binary from `github.com/nats-io/nats-server`).
|
||||
|
||||
## Server Struct
|
||||
|
||||
```rust
|
||||
pub struct Server {
|
||||
inner: Inner,
|
||||
}
|
||||
|
||||
struct Inner {
|
||||
cfg: String, // Config file path
|
||||
id: String, // Unique server ID (NUID)
|
||||
port: Option<String>, // Explicit port (None = dynamic)
|
||||
child: Child, // Child process handle
|
||||
logfile: PathBuf, // Log file path in temp dir
|
||||
pidfile: PathBuf, // PID file path in temp dir
|
||||
}
|
||||
```
|
||||
|
||||
## Public API
|
||||
|
||||
### run_server
|
||||
|
||||
```rust
|
||||
pub fn run_server(cfg: &str) -> Server
|
||||
```
|
||||
|
||||
Starts a single NATS server with optional config file.
|
||||
|
||||
- Uses dynamic port (`-1` flag) for parallel test execution
|
||||
- Stores JetStream data in temp directory
|
||||
- Writes logs to temp file: `nats-server-<id>.log`
|
||||
- Writes PID to temp file: `nats-server-<id>.pid`
|
||||
- If `cfg` is non-empty, passes `-c <cfg>` to the server
|
||||
|
||||
Example:
|
||||
```rust
|
||||
let server = nats_server::run_server("tests/configs/jetstream.conf");
|
||||
let client = async_nats::connect(server.client_url()).await.unwrap();
|
||||
```
|
||||
|
||||
### run_basic_server
|
||||
|
||||
```rust
|
||||
pub fn run_basic_server() -> Server
|
||||
```
|
||||
|
||||
Starts a server with no config (bare minimum). Equivalent to `run_server("")`.
|
||||
|
||||
### run_server_with_port
|
||||
|
||||
```rust
|
||||
pub fn run_server_with_port(cfg: &str, port: Option<&str>) -> Server
|
||||
```
|
||||
|
||||
Starts a server with an explicit port. If `None`, uses dynamic port.
|
||||
|
||||
### run_cluster
|
||||
|
||||
```rust
|
||||
pub fn run_cluster<'a, C: IntoConfig<'a>>(cfg: C) -> Cluster
|
||||
```
|
||||
|
||||
Starts a 3-node cluster with the given config.
|
||||
|
||||
- Allocates 3 random port ranges (base, base+100, base+200)
|
||||
- Configures cluster routes between nodes
|
||||
- Each node gets: `--cluster nats://127.0.0.1:<cluster_port>`, `--routes <other_routes>`, `--cluster_name cluster`, `-n nodeN`
|
||||
- Waits 2 seconds for cluster formation and leader election
|
||||
|
||||
The `IntoConfig` trait allows passing either a single config string (applied to all 3 nodes) or an array of 3 configs (one per node):
|
||||
|
||||
```rust
|
||||
// Same config for all nodes
|
||||
let cluster = run_cluster("configs/jetstream.conf");
|
||||
|
||||
// Different configs per node
|
||||
let cluster = run_cluster(["node1.conf", "node2.conf", "node3.conf"]);
|
||||
```
|
||||
|
||||
### Cluster Struct
|
||||
|
||||
```rust
|
||||
pub struct Cluster {
|
||||
pub servers: Vec<Server>,
|
||||
}
|
||||
|
||||
impl Cluster {
|
||||
pub fn client_url(&self) -> String {
|
||||
self.servers[0].client_url()
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Server Methods
|
||||
|
||||
```rust
|
||||
impl Server {
|
||||
pub fn restart(&mut self)
|
||||
pub fn client_url(&self) -> String
|
||||
pub fn client_port(&self) -> u16
|
||||
pub fn client_url_with(&self, user: &str, pass: &str) -> String
|
||||
pub fn client_url_with_token(&self, token: &str) -> String
|
||||
pub fn client_pid(&self) -> usize
|
||||
}
|
||||
```
|
||||
|
||||
#### restart()
|
||||
|
||||
Kills the current server process, waits for it to exit, then restarts with the same config, port, and ID. Used for testing reconnection behavior.
|
||||
|
||||
#### client_url()
|
||||
|
||||
Connects to the server's TCP port, reads the `INFO` line, parses the JSON, and constructs a URL:
|
||||
- `nats://localhost:<port>` for non-TLS
|
||||
- `tls://localhost:<port>` for TLS-required servers
|
||||
|
||||
Polls the log file (up to 10 seconds) to discover the client address, since the port may be dynamically allocated.
|
||||
|
||||
#### client_pid()
|
||||
|
||||
Reads the PID file and returns the server process ID. Used for sending signals.
|
||||
|
||||
### set_lame_duck_mode
|
||||
|
||||
```rust
|
||||
pub fn set_lame_duck_mode(s: &Server)
|
||||
```
|
||||
|
||||
Sends the lame duck mode signal to the server:
|
||||
|
||||
```bash
|
||||
nats-server --signal ldm=<pid>
|
||||
```
|
||||
|
||||
### is_port_available
|
||||
|
||||
```rust
|
||||
pub fn is_port_available(port: usize) -> bool
|
||||
```
|
||||
|
||||
Tests if a TCP port is available by attempting to bind to it.
|
||||
|
||||
## Server Lifecycle
|
||||
|
||||
### Spawning
|
||||
|
||||
The `do_run` function constructs and spawns the server process:
|
||||
|
||||
```rust
|
||||
fn do_run(cfg: &str, port: Option<&str>, id: Option<String>) -> Inner {
|
||||
let id = id.unwrap_or_else(|| nuid::next().to_string());
|
||||
let logfile = env::temp_dir().join(format!("nats-server-{id}.log"));
|
||||
let pidfile = env::temp_dir().join(format!("nats-server-{id}.pid"));
|
||||
let store_dir = env::temp_dir().join(format!("store-dir-{id}"));
|
||||
|
||||
let mut cmd = Command::new("nats-server");
|
||||
cmd.arg("--store_dir").arg(store_dir.as_path())
|
||||
.arg("-p");
|
||||
|
||||
match port {
|
||||
Some(port) => cmd.arg(port),
|
||||
None => cmd.arg("-1"), // Dynamic port
|
||||
};
|
||||
|
||||
cmd.arg("-l").arg(logfile.as_os_str())
|
||||
.arg("-P").arg(pidfile.as_os_str());
|
||||
|
||||
if !cfg.is_empty() {
|
||||
cmd.arg("-c").arg(cfg);
|
||||
}
|
||||
|
||||
let child = cmd.spawn().unwrap();
|
||||
// ...
|
||||
}
|
||||
```
|
||||
|
||||
Key flags:
|
||||
- `--store_dir` — JetStream storage directory in temp
|
||||
- `-p -1` — Dynamic port allocation (or explicit port)
|
||||
- `-l` — Log file path
|
||||
- `-P` — PID file path
|
||||
- `-c` — Config file path
|
||||
|
||||
### Cleanup (Drop)
|
||||
|
||||
```rust
|
||||
impl Drop for Server {
|
||||
fn drop(&mut self) {
|
||||
self.inner.child.kill().unwrap();
|
||||
self.inner.child.wait().unwrap();
|
||||
|
||||
if let Ok(log) = fs::read_to_string(self.inner.logfile.as_os_str()) {
|
||||
// Clean up JetStream storage directory if found in log
|
||||
if let Some(caps) = SD_RE.captures(&log) {
|
||||
let sd = caps.get(1).map_or("", |m| m.as_str());
|
||||
fs::remove_dir_all(sd).ok();
|
||||
}
|
||||
// Remove log file
|
||||
fs::remove_file(self.inner.logfile.as_os_str()).ok();
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
The regex `SD_RE` matches the "Store Directory" line in the server log:
|
||||
```
|
||||
.+\sStore Directory:\s+"([^"]+)"
|
||||
```
|
||||
|
||||
### Client URL Discovery
|
||||
|
||||
The `client_addr` method polls the log file to find the server's listen address:
|
||||
|
||||
```rust
|
||||
fn client_addr(&self) -> String {
|
||||
for _ in 0..100 { // 100 iterations × 500ms = 50s max
|
||||
match fs::read_to_string(self.inner.logfile.as_os_str()) {
|
||||
Ok(l) => {
|
||||
if let Some(cre) = CLIENT_RE.captures(&l) {
|
||||
return cre.get(1).unwrap().as_str()
|
||||
.replace("0.0.0.0", "localhost");
|
||||
} else {
|
||||
thread::sleep(Duration::from_millis(500));
|
||||
}
|
||||
}
|
||||
_ => thread::sleep(Duration::from_millis(500)),
|
||||
}
|
||||
}
|
||||
panic!("no client addr info");
|
||||
}
|
||||
```
|
||||
|
||||
The regex `CLIENT_RE` matches:
|
||||
```
|
||||
.+\sclient connections on\s+(\S+)
|
||||
```
|
||||
|
||||
After finding the address, `client_url()` connects to it and parses the `INFO` JSON to get the port and TLS requirements.
|
||||
|
||||
## Cluster Setup
|
||||
|
||||
The `run_cluster_node_with_port` function spawns a single cluster node:
|
||||
|
||||
```rust
|
||||
fn run_cluster_node_with_port(
|
||||
cfg: &str,
|
||||
port: Option<&str>,
|
||||
routes: Vec<usize>,
|
||||
name: String,
|
||||
cluster_name: String,
|
||||
cluster: usize,
|
||||
) -> Server
|
||||
```
|
||||
|
||||
Additional flags for cluster nodes:
|
||||
- `--routes nats://127.0.0.1:<port1>,nats://127.0.0.1:<port2>` — routes to other cluster members
|
||||
- `--cluster nats://127.0.0.1:<cluster_port>` — cluster listen address
|
||||
- `--cluster_name <name>` — cluster name for grouping
|
||||
- `-n <name>` — server name
|
||||
|
||||
Port allocation for a cluster:
|
||||
```
|
||||
Base port: random in 3000..50000
|
||||
Node 1: client_port=base, cluster_port=base+1
|
||||
Node 2: client_port=base+100, cluster_port=base+101
|
||||
Node 3: client_port=base+200, cluster_port=base+201
|
||||
```
|
||||
|
||||
Each port is checked for availability with `is_port_available()`, including the +1 cluster port.
|
||||
|
||||
## JetStream Config
|
||||
|
||||
**Location**: `configs/jetstream.conf`
|
||||
|
||||
```conf
|
||||
jetstream: {
|
||||
strict: true,
|
||||
max_mem_store: 8MiB,
|
||||
max_file_store: 10GiB
|
||||
}
|
||||
```
|
||||
|
||||
This is the default test config for JetStream-enabled servers. It enables strict mode and sets memory/file storage limits suitable for testing.
|
||||
|
||||
## Test Usage Patterns
|
||||
|
||||
```rust
|
||||
#[tokio::test]
|
||||
async fn basic_test() {
|
||||
let server = nats_server::run_server("configs/jetstream.conf");
|
||||
let client = async_nats::connect(server.client_url()).await.unwrap();
|
||||
// ... test logic ...
|
||||
// Server cleaned up on drop
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn cluster_test() {
|
||||
let cluster = nats_server::run_cluster("configs/jetstream.conf");
|
||||
let client = async_nats::connect(cluster.client_url()).await.unwrap();
|
||||
// ... test logic ...
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn reconnect_test() {
|
||||
let mut server = nats_server::run_server("");
|
||||
let client = async_nats::connect(server.client_url()).await.unwrap();
|
||||
|
||||
// Restart the server to test reconnection
|
||||
server.restart();
|
||||
|
||||
// Client should reconnect automatically
|
||||
client.publish("test", "data".into()).await.unwrap();
|
||||
}
|
||||
```
|
||||
|
||||
## Dependencies
|
||||
|
||||
| Dependency | Version | Purpose |
|
||||
|-----------|---------|---------|
|
||||
| `lazy_static` | 1.4.0 | Static regex initialization |
|
||||
| `regex` | 1.7.1 | Log parsing (store directory, client address) |
|
||||
| `url` | 2 | URL manipulation for client_url_with |
|
||||
| `serde_json` | 1.0.104 | INFO JSON parsing |
|
||||
| `nuid` | 0.5 | Unique server ID generation |
|
||||
| `rand` | 0.10.1 | Random port selection |
|
||||
| `tokio-retry` | 0.3.0 | Exponential backoff for cluster operations |
|
||||
|
||||
Note: `async-nats` is only a dev-dependency, used in the crate's own integration tests.
|
||||
@@ -0,0 +1,307 @@
|
||||
# Service API and Higher-Level Abstractions
|
||||
|
||||
This document covers the Service API and other higher-level abstractions built on top of the core NATS client.
|
||||
|
||||
## Service API
|
||||
|
||||
**Location**: `service/` (feature: `service`)
|
||||
|
||||
The Service API provides a framework for building NATS-based microservices with built-in monitoring, health checks, and statistics.
|
||||
|
||||
### Service
|
||||
|
||||
```rust
|
||||
#[derive(Debug)]
|
||||
pub struct Service {
|
||||
client: Client,
|
||||
info: Info,
|
||||
endpoints: HashMap<String, Endpoint>,
|
||||
started: DateTime,
|
||||
stats_handler: Arc<dyn Fn(&str, &Stats) -> serde_json::Value + Send + Sync>,
|
||||
stop_sender: mpsc::Sender<()>,
|
||||
stop_receiver: Option<mpsc::Receiver<()>>,
|
||||
}
|
||||
```
|
||||
|
||||
### Creating a Service
|
||||
|
||||
```rust
|
||||
use async_nats::service::ServiceExt;
|
||||
|
||||
let mut service = client
|
||||
.service_builder()
|
||||
.description("Product service")
|
||||
.stats_handler(|endpoint, stats| {
|
||||
serde_json::json!({
|
||||
"endpoint": endpoint,
|
||||
"requests": stats.num_requests,
|
||||
"errors": stats.num_errors,
|
||||
})
|
||||
})
|
||||
.start("products", "1.0.0")
|
||||
.await?;
|
||||
```
|
||||
|
||||
### ServiceBuilder
|
||||
|
||||
```rust
|
||||
impl ServiceBuilder {
|
||||
pub fn description(mut self, description: impl Into<String>) -> Self
|
||||
pub fn stats_handler<F>(mut self, handler: F) -> Self
|
||||
pub async fn start(self, name: impl Into<String>, version: impl Into<String>) -> Result<Service, ServiceError>
|
||||
}
|
||||
```
|
||||
|
||||
### Endpoints
|
||||
|
||||
A service exposes one or more endpoints, each handling requests on a specific subject:
|
||||
|
||||
```rust
|
||||
// Add an endpoint
|
||||
let mut endpoint = service
|
||||
.endpoint("get_product")
|
||||
.await?;
|
||||
|
||||
// Process requests
|
||||
while let Some(request) = endpoint.next().await {
|
||||
let request = request?;
|
||||
// Handle the request
|
||||
request.respond(serde_json::json!({ "id": 1, "name": "Widget" })).await?;
|
||||
}
|
||||
```
|
||||
|
||||
### Endpoint
|
||||
|
||||
**Location**: `service/endpoint.rs`
|
||||
|
||||
```rust
|
||||
pub struct Endpoint {
|
||||
subject: Subject,
|
||||
queue_group: Option<String>,
|
||||
info: EndpointInfo,
|
||||
stats: Stats,
|
||||
subscriber: Subscriber,
|
||||
}
|
||||
```
|
||||
|
||||
Implements `futures::Stream` yielding `ServiceRequest` objects.
|
||||
|
||||
### ServiceRequest
|
||||
|
||||
```rust
|
||||
pub struct ServiceRequest {
|
||||
pub subject: Subject,
|
||||
pub payload: Bytes,
|
||||
pub headers: Option<HeaderMap>,
|
||||
pub reply: Option<Subject>,
|
||||
pub client: Client,
|
||||
}
|
||||
```
|
||||
|
||||
Methods:
|
||||
- `respond(payload)` — send a response to the requester
|
||||
- `respond_with_headers(payload, headers)` — send a response with headers
|
||||
|
||||
### Monitoring Subjects
|
||||
|
||||
The Service API automatically creates monitoring endpoints:
|
||||
|
||||
| Subject | Description |
|
||||
|---------|-------------|
|
||||
| `$SRV.PING` | Ping all services (returns service info) |
|
||||
| `$SRV.PING.<name>` | Ping specific service by name |
|
||||
| `$SRV.PING.<name>.<id>` | Ping specific service instance |
|
||||
| `$SRV.INFO` | Get service info |
|
||||
| `$SRV.STATS` | Get service statistics |
|
||||
|
||||
### Service Info
|
||||
|
||||
```rust
|
||||
pub struct Info {
|
||||
pub name: String,
|
||||
pub id: String,
|
||||
pub version: String,
|
||||
pub description: String,
|
||||
pub endpoints: Vec<EndpointInfo>,
|
||||
}
|
||||
```
|
||||
|
||||
### Stats
|
||||
|
||||
```rust
|
||||
pub struct Stats {
|
||||
pub num_requests: u64,
|
||||
pub num_errors: u64,
|
||||
pub last_error: Option<String>,
|
||||
pub processing_time: Duration,
|
||||
pub average_processing_time: Duration,
|
||||
}
|
||||
```
|
||||
|
||||
## ID Generation
|
||||
|
||||
**Location**: `id_generator.rs`
|
||||
|
||||
The client needs unique IDs for inbox subjects and other purposes.
|
||||
|
||||
### With `nuid` Feature (Default)
|
||||
|
||||
Uses the NUID library for high-performance, cryptographically strong, collision-resistant IDs:
|
||||
|
||||
```rust
|
||||
pub(crate) fn next() -> String {
|
||||
nuid::next().to_string()
|
||||
}
|
||||
```
|
||||
|
||||
NUID generates 22-character alphanumeric strings using a combination of a random prefix and a sequential counter.
|
||||
|
||||
### Without `nuid` Feature
|
||||
|
||||
Falls back to `rand`-based generation:
|
||||
|
||||
```rust
|
||||
pub(crate) fn next() -> String {
|
||||
rng()
|
||||
.sample_iter(Alphanumeric)
|
||||
.take(22)
|
||||
.map(char::from)
|
||||
.collect()
|
||||
}
|
||||
```
|
||||
|
||||
Both approaches produce 22-character alphanumeric strings, but NUID is more performant and has better collision resistance.
|
||||
|
||||
## Inbox Generation
|
||||
|
||||
The `Client::new_inbox()` method generates globally unique inbox subjects for request-reply:
|
||||
|
||||
```rust
|
||||
pub fn new_inbox(&self) -> String {
|
||||
format!("{}.{}", self.inbox_prefix, crate::id_generator::next())
|
||||
}
|
||||
```
|
||||
|
||||
Default prefix is `_INBOX`, producing subjects like `_INBOX.UaBG3f3q5NxX3KdNcRmF2f`.
|
||||
|
||||
Custom prefix via `ConnectOptions::custom_inbox_prefix()`:
|
||||
```rust
|
||||
let client = ConnectOptions::new()
|
||||
.custom_inbox_prefix("MYAPP")
|
||||
.connect("demo.nats.io")
|
||||
.await?;
|
||||
// Inbox subjects: MYAPP.UaBG3f3q5KdNcRmF2f
|
||||
```
|
||||
|
||||
## DateTime Helpers
|
||||
|
||||
**Location**: `datetime.rs` (feature: `jetstream` or `service` or `chrono`)
|
||||
|
||||
Provides date/time types for JetStream and Service API timestamps:
|
||||
|
||||
- Uses the `time` crate by default
|
||||
- Optionally uses `chrono` via the `chrono` feature flag
|
||||
- Supports RFC 3339 formatting and parsing
|
||||
- `DateTime` type wraps either `time::OffsetDateTime` or `chrono::DateTime<Utc>`
|
||||
|
||||
## Crypto Module
|
||||
|
||||
**Location**: `crypto.rs` (feature: `crypto`)
|
||||
|
||||
Provides encryption/decryption support used by the Object Store for server-side encryption.
|
||||
|
||||
## Subject Validation
|
||||
|
||||
**Location**: `lib.rs`
|
||||
|
||||
The client provides two levels of subject validation:
|
||||
|
||||
### is_valid_publish_subject
|
||||
|
||||
```rust
|
||||
pub(crate) fn is_valid_publish_subject<T: AsRef<str>>(subject: T) -> bool
|
||||
```
|
||||
|
||||
Checks for protocol safety only:
|
||||
- Not empty
|
||||
- No whitespace (space, tab, CR, LF) which would break protocol framing
|
||||
|
||||
Used for publish operations. Can be disabled with `skip_subject_validation`.
|
||||
|
||||
### is_valid_subject
|
||||
|
||||
```rust
|
||||
pub(crate) fn is_valid_subject<T: AsRef<str>>(subject: T) -> bool
|
||||
```
|
||||
|
||||
Checks structural validity:
|
||||
- Not empty
|
||||
- No leading/trailing dots
|
||||
- No consecutive dots (`..`)
|
||||
- No whitespace
|
||||
|
||||
Used for subscribe operations (always runs, matching Go/Java behavior).
|
||||
|
||||
### is_valid_queue_group
|
||||
|
||||
```rust
|
||||
pub(crate) fn is_valid_queue_group(queue_group: &str) -> bool
|
||||
```
|
||||
|
||||
Checks:
|
||||
- Not empty
|
||||
- No whitespace
|
||||
|
||||
## JetStream Name Validation
|
||||
|
||||
**Location**: `jetstream/mod.rs`
|
||||
|
||||
```rust
|
||||
pub(crate) fn is_valid_name(name: &str) -> bool {
|
||||
!name.is_empty()
|
||||
&& name.bytes().all(|c| !c.is_ascii_whitespace() && c != b'.' && c != b'*' && c != b'>')
|
||||
}
|
||||
```
|
||||
|
||||
JetStream names (stream names, consumer names) must not contain:
|
||||
- Whitespace
|
||||
- Dots (`.`) — would conflict with subject delimiters
|
||||
- Wildcards (`*`, `>`) — would conflict with subject wildcards
|
||||
|
||||
## CallbackArg1
|
||||
|
||||
**Location**: `options.rs`
|
||||
|
||||
A type-erased async callback wrapper used throughout the crate:
|
||||
|
||||
```rust
|
||||
pub(crate) type AsyncCallbackArg1<A, T> =
|
||||
Arc<dyn Fn(A) -> Pin<Box<dyn Future<Output = T> + Send + Sync + 'static>> + Send + Sync>;
|
||||
|
||||
#[derive(Clone)]
|
||||
pub(crate) struct CallbackArg1<A, T>(AsyncCallbackArg1<A, T>);
|
||||
|
||||
impl<A, T> CallbackArg1<A, T> {
|
||||
pub(crate) async fn call(&self, arg: A) -> T {
|
||||
(self.0.as_ref())(arg).await
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Used for:
|
||||
- `event_callback` — `CallbackArg1<Event, ()>`
|
||||
- `auth_callback` — `CallbackArg1<Vec<u8>, Result<Auth, AuthError>>`
|
||||
- `reconnect_to_server_callback` — `CallbackArg1<(Vec<Server>, ServerInfo), Option<ReconnectToServer>>`
|
||||
- `signature_callback` — `CallbackArg1<String, Result<String, AuthError>>`
|
||||
|
||||
## Version Compatibility Checking
|
||||
|
||||
The `Client::is_server_compatible` method checks if the server version meets a minimum requirement:
|
||||
|
||||
```rust
|
||||
pub fn is_server_compatible(&self, major: i64, minor: i64, patch: i64) -> bool
|
||||
```
|
||||
|
||||
This parses the server version string from `ServerInfo::version` using a regex and compares major/minor/patch components. Note: this checks the directly-connected server, not necessarily the JetStream leader.
|
||||
|
||||
The `server_2_10`, `server_2_11`, `server_2_12`, and `server_2_14` feature flags enable version-specific API fields and methods without runtime checks.
|
||||
@@ -0,0 +1,215 @@
|
||||
# Quick Reference
|
||||
|
||||
## Crate Summary
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| **Crate** | `async-nats` |
|
||||
| **Version** | 0.49.1 |
|
||||
| **Edition** | 2021 |
|
||||
| **MSRV** | 1.88.0 |
|
||||
| **License** | Apache-2.0 |
|
||||
| **Runtime** | Tokio |
|
||||
| **Protocol** | NATS Client Protocol v1 (Dynamic) |
|
||||
| **TLS** | rustls (ring / aws-lc-rs / fips) |
|
||||
| **WebSocket** | tokio-websockets (feature-gated) |
|
||||
|
||||
## Quick Start
|
||||
|
||||
```rust
|
||||
use async_nats::connect;
|
||||
use futures_util::StreamExt;
|
||||
|
||||
#[tokio::main]
|
||||
async fn main() -> Result<(), async_nats::Error> {
|
||||
let client = connect("demo.nats.io").await?;
|
||||
|
||||
// Publish
|
||||
client.publish("events.data", "hello".into()).await?;
|
||||
|
||||
// Subscribe
|
||||
let mut sub = client.subscribe("events.>").await?;
|
||||
while let Some(msg) = sub.next().await {
|
||||
println!("{:?}", msg);
|
||||
}
|
||||
|
||||
// Request-Response
|
||||
let response = client.request("service", "input".into()).await?;
|
||||
|
||||
Ok(())
|
||||
}
|
||||
```
|
||||
|
||||
## Architecture at a Glance
|
||||
|
||||
```
|
||||
Client (cloneable handle, mpsc::Sender<Command>)
|
||||
│
|
||||
▼
|
||||
ConnectionHandler (single Tokio task)
|
||||
├── Subscriptions HashMap<u64, Subscription>
|
||||
├── Multiplexer (request-reply, SID 0)
|
||||
├── Flush Observers
|
||||
└── Ping/Pong health check
|
||||
│
|
||||
▼
|
||||
Connection (protocol I/O, read/write buffers)
|
||||
│
|
||||
▼
|
||||
Connector (server pool, reconnection)
|
||||
│
|
||||
▼
|
||||
NATS Server (Go binary, TCP/TLS/WebSocket)
|
||||
```
|
||||
|
||||
## Key Types
|
||||
|
||||
| Type | Location | Purpose |
|
||||
|------|----------|---------|
|
||||
| `Client` | `client.rs` | Cloneable connection handle |
|
||||
| `Subscriber` | `lib.rs` | Message stream (impl `futures::Stream`) |
|
||||
| `Message` | `message.rs` | Inbound NATS message |
|
||||
| `OutboundMessage` | `message.rs` | Outbound publish message |
|
||||
| `Subject` | `subject.rs` | Validated subject string (backed by `Bytes`) |
|
||||
| `HeaderMap` | `header.rs` | NATS message headers |
|
||||
| `StatusCode` | `status.rs` | NATS protocol status codes |
|
||||
| `ServerInfo` | `lib.rs` | Server INFO data |
|
||||
| `ConnectInfo` | `lib.rs` | Client CONNECT data |
|
||||
| `ServerAddr` | `lib.rs` | Validated server URL |
|
||||
| `Auth` | `auth.rs` | Authentication credentials |
|
||||
| `ConnectOptions` | `options.rs` | Connection configuration builder |
|
||||
| `Event` | `lib.rs` | Connection lifecycle events |
|
||||
| `State` | `connection.rs` | Connection state (Pending/Connected/Disconnected) |
|
||||
| `Statistics` | `client.rs` | Atomic connection metrics |
|
||||
| `Request` | `client.rs` | Request-response builder |
|
||||
|
||||
## JetStream Types
|
||||
|
||||
| Type | Location | Purpose |
|
||||
|------|----------|---------|
|
||||
| `jetstream::Context` | `jetstream/context.rs` | JetStream API entry point |
|
||||
| `jetstream::stream::Stream` | `jetstream/stream.rs` | Stream management |
|
||||
| `jetstream::stream::Config` | `jetstream/stream.rs` | Stream configuration |
|
||||
| `jetstream::stream::Info` | `jetstream/stream.rs` | Stream info/state |
|
||||
| `jetstream::consumer::PullConsumer` | `jetstream/consumer/pull.rs` | Pull-based consumer |
|
||||
| `jetstream::consumer::PushConsumer` | `jetstream/consumer/push.rs` | Push-based consumer |
|
||||
| `jetstream::consumer::Config` | `jetstream/consumer/mod.rs` | Consumer configuration |
|
||||
| `jetstream::Message` | `jetstream/message.rs` | Message with ack methods |
|
||||
| `jetstream::PublishAck` | `jetstream/publish.rs` | Publish acknowledgment |
|
||||
| `jetstream::kv::Store` | `jetstream/kv/bucket.rs` | Key-Value store |
|
||||
| `jetstream::object_store::ObjectStore` | `jetstream/object_store/mod.rs` | Object store |
|
||||
| `jetstream::ErrorCode` | `jetstream/errors.rs` | JetStream error codes |
|
||||
|
||||
## Protocol Operations
|
||||
|
||||
### Client → Server (ClientOp)
|
||||
|
||||
| Op | Wire Format | Purpose |
|
||||
|----|-----------|---------|
|
||||
| `CONNECT` | `CONNECT {json}\r\n` | Authentication and capabilities |
|
||||
| `PUB` | `PUB <subject> [reply] <len>\r\n<payload>\r\n` | Publish message |
|
||||
| `HPUB` | `HPUB <subject> [reply] <hlen> <tlen>\r\n<hdrs><payload>\r\n` | Publish with headers |
|
||||
| `SUB` | `SUB <subject> [queue] <sid>\r\n` | Subscribe |
|
||||
| `UNSUB` | `UNSUB <sid> [max]\r\n` | Unsubscribe |
|
||||
| `PING` | `PING\r\n` | Keepalive / health check |
|
||||
| `PONG` | `PONG\r\n` | Response to server PING |
|
||||
|
||||
### Server → Client (ServerOp)
|
||||
|
||||
| Op | Wire Format | Purpose |
|
||||
|----|-----------|---------|
|
||||
| `INFO` | `INFO {json}\r\n` | Server capabilities, cluster info |
|
||||
| `MSG` | `MSG <subj> <sid> [reply] <len>\r\n<payload>\r\n` | Deliver message |
|
||||
| `HMSG` | `HMSG <subj> <sid> [reply] <hlen> <tlen>\r\n<hdrs><payload>\r\n` | Message with headers |
|
||||
| `+OK` | `+OK\r\n` | Success (verbose mode) |
|
||||
| `-ERR` | `-ERR <desc>\r\n` | Server error |
|
||||
| `PING` | `PING\r\n` | Server health check |
|
||||
| `PONG` | `PONG\r\n` | Ack client PING |
|
||||
|
||||
## Internal Commands (Command → ConnectionHandler)
|
||||
|
||||
| Command | Purpose |
|
||||
|---------|---------|
|
||||
| `Publish(OutboundMessage)` | Queue message for sending |
|
||||
| `Request { subject, payload, respond, headers, sender }` | Request-response via multiplexer |
|
||||
| `Subscribe { sid, subject, queue_group, sender }` | Create subscription |
|
||||
| `Unsubscribe { sid, max }` | Remove subscription |
|
||||
| `Flush { observer }` | Wait for write buffer flush |
|
||||
| `Drain { sid }` | Gracefully drain (sub or whole client) |
|
||||
| `Reconnect` | Force reconnection |
|
||||
| `SetServerPool { servers, result }` | Replace server pool |
|
||||
| `ServerPool { result }` | Query server pool |
|
||||
|
||||
## Feature Flags
|
||||
|
||||
| Feature | Default | Enables |
|
||||
|---------|---------|---------|
|
||||
| `jetstream` | ✓ | JetStream API (streams, consumers, publish) |
|
||||
| `kv` | ✓ | Key-Value store (requires jetstream) |
|
||||
| `object-store` | ✓ | Object store (requires jetstream + crypto) |
|
||||
| `service` | ✓ | Service API |
|
||||
| `nkeys` | ✓ | NKey/JWT authentication |
|
||||
| `crypto` | ✓ | Encryption support |
|
||||
| `websockets` | ✓ | WebSocket transport |
|
||||
| `nuid` | ✓ | NUID ID generation |
|
||||
| `ring` | ✓ | Ring crypto backend |
|
||||
| `aws-lc-rs` | ✗ | Alternative crypto backend |
|
||||
| `fips` | ✗ | FIPS mode (requires aws-lc-rs) |
|
||||
| `chrono` | ✗ | Use chrono instead of time |
|
||||
| `experimental` | ✗ | Experimental features |
|
||||
| `server_2_10` | ✓ | Server 2.10+ API fields |
|
||||
| `server_2_11` | ✓ | Server 2.11+ API fields |
|
||||
| `server_2_12` | ✓ | Server 2.12+ API fields |
|
||||
| `server_2_14` | ✓ | Server 2.14+ API fields |
|
||||
|
||||
## Connection Defaults
|
||||
|
||||
| Parameter | Default |
|
||||
|-----------|---------|
|
||||
| Connection timeout | 5 seconds |
|
||||
| Ping interval | 60 seconds |
|
||||
| Max pending pings | 2 |
|
||||
| Request timeout | 10 seconds |
|
||||
| Command channel capacity | 2048 |
|
||||
| Subscription capacity | 65536 |
|
||||
| Read buffer capacity | 65535 |
|
||||
| Inbox prefix | `_INBOX` |
|
||||
| Reconnect delay | Exponential (0ms → 4s cap) |
|
||||
| Max reconnects | Unlimited |
|
||||
| TLS required | Auto (server-dependent) |
|
||||
|
||||
## Error Hierarchy
|
||||
|
||||
```
|
||||
ConnectError (ConnectErrorKind::ServerParse | Dns | Authentication | AuthorizationViolation | TimedOut | Tls | Io | MaxReconnects)
|
||||
PublishError (PublishErrorKind::MaxPayloadExceeded | InvalidSubject | Send)
|
||||
RequestError (RequestErrorKind::TimedOut | NoResponders | InvalidSubject | MaxPayloadExceeded | Other)
|
||||
SubscribeError (SubscribeErrorKind::InvalidSubject | InvalidQueueName | Other)
|
||||
FlushError (FlushErrorKind::SendError | FlushError)
|
||||
```
|
||||
|
||||
## nats-server Test Harness
|
||||
|
||||
| Function | Description |
|
||||
|----------|-------------|
|
||||
| `run_server(cfg)` | Start single server with config |
|
||||
| `run_basic_server()` | Start bare server |
|
||||
| `run_cluster(cfg)` | Start 3-node cluster |
|
||||
| `set_lame_duck_mode(s)` | Send LDM signal |
|
||||
|
||||
## JetStream API Subjects
|
||||
|
||||
| Operation | Subject Pattern |
|
||||
|-----------|---------------|
|
||||
| Create stream | `$JS.API.STREAM.CREATE.<name>` |
|
||||
| Stream info | `$JS.API.STREAM.INFO.<name>` |
|
||||
| Update stream | `$JS.API.STREAM.UPDATE.<name>` |
|
||||
| Delete stream | `$JS.API.STREAM.DELETE.<name>` |
|
||||
| Purge stream | `$JS.API.STREAM.PURGE.<name>` |
|
||||
| List streams | `$JS.API.STREAM.LIST` |
|
||||
| Create consumer | `$JS.API.CONSUMER.CREATE.<stream>` |
|
||||
| Create durable | `$JS.API.CONSUMER.DURABLE.CREATE.<stream>.<name>` |
|
||||
| Consumer info | `$JS.API.CONSUMER.INFO.<stream>.<name>` |
|
||||
| Pull next | `$JS.API.CONSUMER.MSG.NEXT.<stream>.<name>` |
|
||||
| Account info | `$JS.API.ACCOUNT.INFO` |
|
||||
| Direct get | `$JS.API.DIRECT.GET.<name>` |
|
||||
Reference in New Issue
Block a user