docs(research): add nats-async and nats-server deep-dive references
This commit is contained in:
@@ -0,0 +1,170 @@
|
||||
# async-nats: Overview & Architecture
|
||||
|
||||
**Crate**: `async-nats`
|
||||
**Version**: 0.49.1
|
||||
**Repository**: https://github.com/nats-io/nats.rs
|
||||
**License**: Apache-2.0
|
||||
**Rust Edition**: 2021
|
||||
**MSRV**: 1.88.0
|
||||
**Async Runtime**: Tokio
|
||||
|
||||
## What is async-nats?
|
||||
|
||||
`async-nats` is the official async Rust client for the [NATS messaging system](https://nats.io). It provides a Tokio-based asynchronous interface to NATS server features including:
|
||||
|
||||
- **Core NATS** — publish/subscribe, request/reply, queue groups
|
||||
- **JetStream** — persistent stream-based messaging with at-least-once and exactly-once semantics
|
||||
- **Key-Value Store** — KV abstraction built on JetStream streams
|
||||
- **Object Store** — large-object storage built on JetStream streams
|
||||
- **Service API** — microservice request/reply pattern with built-in PING/INFO/STATS verbs
|
||||
|
||||
The crate is positioned as the **core client** in the NATS Rust ecosystem. A separate project, [Orbit](https://github.com/synadia-io/orbit.rs), provides higher-level opinionated abstractions on top.
|
||||
|
||||
```
|
||||
┌──────────────────────────────────────────────────────┐
|
||||
│ Application code │
|
||||
└──────────────┬───────────────────────────┬───────────┘
|
||||
│ │
|
||||
▼ ▼
|
||||
┌───────────────────┐ ┌───────────────────┐
|
||||
│ Orbit crates │ uses │ async-nats (core) │
|
||||
│ (opinionated, │──────▶│ (parity, stable, │
|
||||
│ per-crate semver) │ │ protocol-level) │
|
||||
└───────────────────┘ └─────────┬─────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────┐
|
||||
│ nats-server │
|
||||
└─────────────┘
|
||||
```
|
||||
|
||||
## Feature Flags
|
||||
|
||||
Features are extensive and control which subsystems are compiled:
|
||||
|
||||
| Feature | Default | Description |
|
||||
|---------|---------|-------------|
|
||||
| `jetstream` | ✅ | JetStream API (streams, consumers, publish) |
|
||||
| `kv` | ✅ | Key-Value store (depends on `jetstream`) |
|
||||
| `object-store` | ✅ | Object store (depends on `jetstream` + `crypto`) |
|
||||
| `service` | ✅ | Service API (microservice pattern) |
|
||||
| `nkeys` | ✅ | NKey/JWT authentication |
|
||||
| `nuid` | ✅ | NUID-based unique ID generation |
|
||||
| `crypto` | ✅ | Cryptographic primitives (SHA-256 for object store) |
|
||||
| `websockets` | ✅ | WebSocket transport (`ws://`/`wss://`) |
|
||||
| `ring` | ✅ | Use `ring` as TLS crypto backend |
|
||||
| `aws-lc-rs` | ❌ | Use `aws-lc-rs` as TLS crypto backend |
|
||||
| `fips` | ❌ | FIPS 140-2 compliant via `aws-lc-rs` |
|
||||
| `chrono` | ❌ | Use `chrono` instead of `time` for datetime types |
|
||||
| `server_2_10` | ✅ | Server 2.10+ features |
|
||||
| `server_2_11` | ✅ | Server 2.11+ features |
|
||||
| `server_2_12` | ✅ | Server 2.12+ features |
|
||||
| `server_2_14` | ✅ | Server 2.14+ features |
|
||||
| `experimental` | ❌ | Experimental features |
|
||||
|
||||
## Source Structure
|
||||
|
||||
```
|
||||
async-nats/src/
|
||||
├── lib.rs # Entry point: connect(), ServerInfo, Command, ClientOp, ServerOp,
|
||||
│ ConnectionHandler, Subscriber, Event, ServerAddr, ConnectInfo
|
||||
├── client.rs # Client struct, publish/subscribe/request/drain/flush APIs,
|
||||
│ Request builder, Statistics, trait definitions
|
||||
├── connection.rs # Framed connection: NATS protocol parser/serializer,
|
||||
│ read/write buffer management, WebSocket adapter
|
||||
├── connector.rs # Server pool, reconnection logic, TLS setup, DNS resolution,
|
||||
│ authentication handshake
|
||||
├── options.rs # ConnectOptions builder, auth methods, TLS config, callbacks
|
||||
├── auth.rs # Auth struct (username, password, token, JWT, nkey, signature)
|
||||
├── auth_utils.rs # Credentials file parsing (JWT + NKey seed)
|
||||
├── message.rs # Message (inbound), OutboundMessage (outbound)
|
||||
├── header.rs # HeaderMap, HeaderName, HeaderValue (NATS headers)
|
||||
├── subject.rs # Subject type, ToSubject trait, SubjectError
|
||||
├── status.rs # StatusCode enum (NATS status codes)
|
||||
├── error.rs # Generic Error<K> type used throughout
|
||||
├── datetime.rs # DateTime type (time or chrono backend)
|
||||
├── id_generator.rs # Unique ID generation (NUID or rand fallback)
|
||||
├── tls.rs # TLS configuration helper
|
||||
├── crypto.rs # SHA-256 for object store integrity
|
||||
├── jetstream/
|
||||
│ ├── mod.rs # Module entry: new(), with_domain(), with_prefix()
|
||||
│ ├── context.rs # Context: JetStream API (streams, consumers, KV, OS, publish)
|
||||
│ ├── stream.rs # Stream handle, Config, Info, purge/delete/message ops
|
||||
│ ├── consumer/
|
||||
│ │ ├── mod.rs # Consumer trait, Info, Config base
|
||||
│ │ ├── pull.rs # PullConsumer: batch fetch, sequence, messages stream
|
||||
│ │ └── push.rs # PushConsumer: Ordered push consumer with auto-recreate
|
||||
│ ├── publish.rs # PublishAck, PublishAckFuture, PublishMessage builder
|
||||
│ ├── message.rs # JetStream Message (with ack methods), AckKind
|
||||
│ ├── response.rs # Response<T> (Ok/Err) for JetStream API calls
|
||||
│ ├── errors.rs # ErrorCode, Error for JetStream
|
||||
│ ├── account.rs # Account info
|
||||
│ ├── kv/
|
||||
│ │ ├── mod.rs # Store: put/get/delete/purge/watch/history/keys
|
||||
│ │ └── bucket.rs # Bucket Status
|
||||
│ └── object_store/
|
||||
│ └── mod.rs # ObjectStore: put/get/delete/watch/list/seal, Object (AsyncRead)
|
||||
└── service/
|
||||
├── mod.rs # Service, ServiceBuilder, Group, EndpointBuilder, Request
|
||||
└── endpoint.rs # Endpoint stream, Stats, Info
|
||||
```
|
||||
|
||||
## Architecture: Core Connection Model
|
||||
|
||||
The client uses a **single-connection, actor-model** design:
|
||||
|
||||
```
|
||||
┌──────────────────────────────────────┐
|
||||
Client (clone) ──▶│ mpsc::Sender<Command> │
|
||||
(many handles) │ (bounded channel) │
|
||||
└────────────┬────────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌────────────────────────────────────────┐
|
||||
│ ConnectionHandler (tokio::task) │
|
||||
│ - Receives Command from channel │
|
||||
│ - Converts to ClientOp │
|
||||
│ - Manages subscriptions map │
|
||||
│ - Manages multiplexer (request/reply)│
|
||||
│ - Pings server on interval │
|
||||
│ - Handles reconnection │
|
||||
└────────────┬──────────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌────────────────────────────────────────┐
|
||||
│ Connection (framed TCP/TLS/WS) │
|
||||
│ - Protocol parser (try_read_op) │
|
||||
│ - Write buffer (VecDeque<Bytes>) │
|
||||
│ - Vectored I/O support │
|
||||
│ - Read buffer (BytesMut) │
|
||||
└────────────┬──────────────────────────┘
|
||||
│
|
||||
▼
|
||||
nats-server
|
||||
```
|
||||
|
||||
### Key Design Decisions
|
||||
|
||||
1. **Cloneable Client**: `Client` is `Clone` (via `mpsc::Sender` clone), enabling shared use across tasks
|
||||
2. **Single TCP connection**: All traffic (Core NATS, JetStream API, etc.) multiplexes over one connection
|
||||
3. **Background task**: `ConnectionHandler` runs as a spawned Tokio task, bridging the mpsc channel to the TCP stream
|
||||
4. **Automatic reconnection**: On disconnect, `Connector` retries servers from the pool with exponential backoff
|
||||
5. **Subscription rehydration**: On reconnect, all active subscriptions are re-subscribed with adjusted `max` counts
|
||||
6. **Multiplexer for request/reply**: A single wildcard subscription (`_INBOX.<id>.*`) multiplexes all pending request/reply correlations
|
||||
|
||||
## Dependencies (Key)
|
||||
|
||||
| Crate | Purpose |
|
||||
|-------|---------|
|
||||
| `tokio` | Async runtime, TCP, time, sync, io-util |
|
||||
| `bytes` | Efficient byte buffer (`Bytes`, `BytesMut`) |
|
||||
| `tokio-rustls` | TLS via rustls |
|
||||
| `rustls-native-certs` | Load system root certificates |
|
||||
| `serde` / `serde_json` | JSON serialization for JetStream API |
|
||||
| `futures-util` | Stream trait, Sink trait, StreamExt |
|
||||
| `tracing` | Structured logging |
|
||||
| `thiserror` | Error derive macros |
|
||||
| `memchr` | Fast substring search for protocol parsing |
|
||||
| `portable-atomic` | Atomic types with portable-atomic fallback |
|
||||
| `tokio-util` | `PollSender` for Sink implementation |
|
||||
| `tokio-stream` | `ReceiverStream` adapter |
|
||||
@@ -0,0 +1,404 @@
|
||||
# async-nats: Key Types & Traits
|
||||
|
||||
## Core Types
|
||||
|
||||
### `Client`
|
||||
|
||||
The primary handle to a NATS connection. Cheaply cloneable (wraps `mpsc::Sender<Command>`).
|
||||
|
||||
```rust
|
||||
#[derive(Clone, Debug)]
|
||||
pub struct Client {
|
||||
info: tokio::sync::watch::Receiver<Option<ServerInfo>>,
|
||||
state: tokio::sync::watch::Receiver<State>,
|
||||
sender: mpsc::Sender<Command>,
|
||||
poll_sender: PollSender<Command>,
|
||||
next_subscription_id: Arc<AtomicU64>,
|
||||
subscription_capacity: usize,
|
||||
inbox_prefix: Arc<str>,
|
||||
request_timeout: Option<Duration>,
|
||||
max_payload: Arc<AtomicUsize>,
|
||||
connection_stats: Arc<Statistics>,
|
||||
skip_subject_validation: bool,
|
||||
}
|
||||
```
|
||||
|
||||
**Key methods**:
|
||||
- `publish(subject, payload)` — fire-and-forget publish
|
||||
- `publish_with_headers(subject, headers, payload)` — publish with NATS headers
|
||||
- `publish_with_reply(subject, reply, payload)` — publish with reply-to subject
|
||||
- `subscribe(subject)` → `Subscriber` — subscribe to a subject
|
||||
- `queue_subscribe(subject, queue_group)` → `Subscriber` — queue group subscription
|
||||
- `request(subject, payload)` → `Message` — request/reply with default timeout
|
||||
- `send_request(subject, request)` → `Message` — request with custom `Request` builder
|
||||
- `flush()` — wait until all buffered writes are flushed to the server
|
||||
- `drain()` — drain all subscriptions, flush, then close
|
||||
- `force_reconnect()` — force a reconnection (e.g., to re-trigger auth)
|
||||
- `new_inbox()` — generate a unique inbox subject (`_INBOX.<id>`)
|
||||
- `server_info()` → `ServerInfo` — last known server info
|
||||
- `connection_state()` → `State` — `Pending`/`Connected`/`Disconnected`
|
||||
- `statistics()` → `Arc<Statistics>` — connection statistics (bytes, messages, connects)
|
||||
- `max_payload()` → `usize` — server's max payload size
|
||||
- `set_server_pool(addrs)` — replace the server pool for reconnection
|
||||
- `server_pool()` — snapshot of current server pool
|
||||
|
||||
### `Subscriber`
|
||||
|
||||
A `Stream` yielding `Message` values from a subscription.
|
||||
|
||||
```rust
|
||||
#[derive(Debug)]
|
||||
pub struct Subscriber {
|
||||
sid: u64,
|
||||
receiver: mpsc::Receiver<Message>,
|
||||
sender: mpsc::Sender<Command>,
|
||||
}
|
||||
```
|
||||
|
||||
Implements `futures_util::Stream<Item = Message>`. Methods:
|
||||
- `unsubscribe()` — immediately unsubscribe
|
||||
- `unsubscribe_after(n)` — unsubscribe after `n` total delivered messages
|
||||
- `drain()` — unsubscribe after in-flight messages are delivered
|
||||
|
||||
**Drop behavior**: When a `Subscriber` is dropped, it spawns a task to send `Command::Unsubscribe` to the connection handler, ensuring the server is always notified.
|
||||
|
||||
### `Message`
|
||||
|
||||
An inbound NATS message:
|
||||
|
||||
```rust
|
||||
#[derive(Clone, Debug, Serialize, Deserialize)]
|
||||
pub struct Message {
|
||||
pub subject: Subject,
|
||||
pub reply: Option<Subject>,
|
||||
pub payload: Bytes,
|
||||
pub headers: Option<HeaderMap>,
|
||||
pub status: Option<StatusCode>,
|
||||
pub description: Option<String>,
|
||||
pub length: usize,
|
||||
}
|
||||
```
|
||||
|
||||
### `OutboundMessage`
|
||||
|
||||
An outbound message for publishing (no status/description):
|
||||
|
||||
```rust
|
||||
#[derive(Clone, Debug)]
|
||||
pub struct OutboundMessage {
|
||||
pub subject: Subject,
|
||||
pub reply: Option<Subject>,
|
||||
pub payload: Bytes,
|
||||
pub headers: Option<HeaderMap>,
|
||||
}
|
||||
```
|
||||
|
||||
### `Request`
|
||||
|
||||
Builder for request/reply calls:
|
||||
|
||||
```rust
|
||||
#[derive(Default)]
|
||||
pub struct Request {
|
||||
pub payload: Option<Bytes>,
|
||||
pub headers: Option<HeaderMap>,
|
||||
pub timeout: Option<Option<Duration>>,
|
||||
pub inbox: Option<String>,
|
||||
}
|
||||
```
|
||||
|
||||
Builder methods: `payload()`, `headers()`, `timeout()`, `inbox()`. The `inbox` field, when set, bypasses the multiplexer and uses a dedicated subscription instead.
|
||||
|
||||
### `ServerInfo`
|
||||
|
||||
Server metadata received during connection handshake:
|
||||
|
||||
```rust
|
||||
#[derive(Debug, Deserialize, Default, Clone, Eq, PartialEq)]
|
||||
pub struct ServerInfo {
|
||||
pub server_id: String,
|
||||
pub server_name: String,
|
||||
pub host: String,
|
||||
pub port: u16,
|
||||
pub version: String,
|
||||
pub auth_required: bool,
|
||||
pub tls_required: bool,
|
||||
pub max_payload: usize,
|
||||
pub proto: i8,
|
||||
pub client_id: u64,
|
||||
pub go: String,
|
||||
pub nonce: String,
|
||||
pub connect_urls: Vec<String>,
|
||||
pub client_ip: String,
|
||||
pub headers: bool,
|
||||
pub lame_duck_mode: bool,
|
||||
pub cluster: Option<String>,
|
||||
pub domain: Option<String>,
|
||||
pub jetstream: bool,
|
||||
}
|
||||
```
|
||||
|
||||
### `ConnectInfo`
|
||||
|
||||
Client → server `CONNECT` message payload:
|
||||
|
||||
```rust
|
||||
#[derive(Clone, Debug, Serialize)]
|
||||
pub struct ConnectInfo {
|
||||
pub verbose: bool,
|
||||
pub pedantic: bool,
|
||||
pub user_jwt: Option<String>,
|
||||
pub nkey: Option<String>,
|
||||
pub signature: Option<String>,
|
||||
pub name: Option<String>,
|
||||
pub echo: bool,
|
||||
pub lang: String,
|
||||
pub version: String,
|
||||
pub protocol: Protocol, // Original(0) or Dynamic(1)
|
||||
pub tls_required: bool,
|
||||
pub user: Option<String>,
|
||||
pub pass: Option<String>,
|
||||
pub auth_token: Option<String>,
|
||||
pub headers: bool,
|
||||
pub no_responders: bool,
|
||||
}
|
||||
```
|
||||
|
||||
The client always sets: `verbose=false`, `pedantic=false`, `lang="rust"`, `protocol=Dynamic`, `headers=true`, `no_responders=true`.
|
||||
|
||||
### `Statistics`
|
||||
|
||||
Atomic connection statistics (shared via `Arc`):
|
||||
|
||||
```rust
|
||||
#[derive(Default, Debug)]
|
||||
pub struct Statistics {
|
||||
pub in_bytes: AtomicU64,
|
||||
pub out_bytes: AtomicU64,
|
||||
pub in_messages: AtomicU64,
|
||||
pub out_messages: AtomicU64,
|
||||
pub connects: AtomicU64,
|
||||
}
|
||||
```
|
||||
|
||||
## Subject Types
|
||||
|
||||
### `Subject`
|
||||
|
||||
A validated NATS subject string (newtype over `String`):
|
||||
|
||||
```rust
|
||||
// Usage:
|
||||
let subject: Subject = "foo.bar.baz".into();
|
||||
```
|
||||
|
||||
### `ToSubject` trait
|
||||
|
||||
Conversion trait for subjects:
|
||||
|
||||
```rust
|
||||
pub trait ToSubject {
|
||||
fn to_subject(self) -> Result<Subject, SubjectError>;
|
||||
}
|
||||
```
|
||||
|
||||
Implemented for `&str`, `String`, `Subject` directly.
|
||||
|
||||
### `SubjectError`
|
||||
|
||||
```rust
|
||||
pub enum SubjectError {
|
||||
InvalidFormat,
|
||||
}
|
||||
```
|
||||
|
||||
## Header Types
|
||||
|
||||
### `HeaderMap`
|
||||
|
||||
A multimap of header name → values:
|
||||
|
||||
```rust
|
||||
pub struct HeaderMap {
|
||||
inner: VecMap<HeaderName, Vec<HeaderValue>>,
|
||||
}
|
||||
```
|
||||
|
||||
Methods: `insert()`, `append()`, `get()`, `len()`, `is_empty()`, `iter()`, `to_bytes()`.
|
||||
|
||||
### `HeaderName`
|
||||
|
||||
Case-insensitive header name. Created via `FromStr`:
|
||||
|
||||
```rust
|
||||
let name: HeaderName = "Nats-Expected-Last-Subject-Sequence".parse()?;
|
||||
```
|
||||
|
||||
### `HeaderValue`
|
||||
|
||||
Header value string. Created via `FromStr` or `From<u64>`:
|
||||
|
||||
```rust
|
||||
let val: HeaderValue = "some value".parse()?;
|
||||
let val: HeaderValue = HeaderValue::from(42u64);
|
||||
```
|
||||
|
||||
## Server Address Types
|
||||
|
||||
### `ServerAddr`
|
||||
|
||||
Wraps a `url::Url` with NATS-specific validation. Supports schemes: `nats://`, `tls://`, `ws://`, `wss://`. Default port is `4222`.
|
||||
|
||||
```rust
|
||||
let addr: ServerAddr = "demo.nats.io".parse()?;
|
||||
let addr: ServerAddr = "nats://demo.nats.io:4222".parse()?;
|
||||
let addr: ServerAddr = "tls://demo.nats.io".parse()?;
|
||||
```
|
||||
|
||||
### `ToServerAddrs` trait
|
||||
|
||||
Flexible server address input (single URL, `Vec`, slice, etc.):
|
||||
|
||||
```rust
|
||||
pub trait ToServerAddrs {
|
||||
type Iter: Iterator<Item = ServerAddr>;
|
||||
fn to_server_addrs(&self) -> io::Result<Self::Iter>;
|
||||
}
|
||||
```
|
||||
|
||||
### `Server`
|
||||
|
||||
Metadata about a server in the pool:
|
||||
|
||||
```rust
|
||||
pub struct Server {
|
||||
pub addr: ServerAddr,
|
||||
pub failed_attempts: usize,
|
||||
pub did_connect: bool,
|
||||
pub is_discovered: bool,
|
||||
pub last_error: Option<String>,
|
||||
}
|
||||
```
|
||||
|
||||
## Event & State Types
|
||||
|
||||
### `Event`
|
||||
|
||||
Asynchronous notifications from the connection:
|
||||
|
||||
```rust
|
||||
pub enum Event {
|
||||
Connected,
|
||||
Disconnected,
|
||||
LameDuckMode,
|
||||
Draining,
|
||||
Closed,
|
||||
SlowConsumer(u64), // subscription sid
|
||||
ServerError(ServerError),
|
||||
ClientError(ClientError),
|
||||
}
|
||||
```
|
||||
|
||||
Received via `ConnectOptions::event_callback()`.
|
||||
|
||||
### `State`
|
||||
|
||||
Connection state observable via `watch::Receiver`:
|
||||
|
||||
```rust
|
||||
pub enum State {
|
||||
Pending,
|
||||
Connected,
|
||||
Disconnected,
|
||||
}
|
||||
```
|
||||
|
||||
### `StatusCode`
|
||||
|
||||
NATS protocol status codes (e.g., `NO_RESPONDERS = 404`, `TIMEOUT = 408`).
|
||||
|
||||
## Error Types
|
||||
|
||||
All error types follow the pattern `Error<Kind>` from `crate::error`:
|
||||
|
||||
| Error Type | Kind | Used By |
|
||||
|------------|------|---------|
|
||||
| `ConnectError` | `ConnectErrorKind` | Connection establishment |
|
||||
| `PublishError` | `PublishErrorKind` | Publish operations |
|
||||
| `RequestError` | `RequestErrorKind` | Request/reply |
|
||||
| `SubscribeError` | `SubscribeErrorKind` | Subscribe |
|
||||
| `FlushError` | `FlushErrorKind` | Flush |
|
||||
| `DrainError` | — | Drain |
|
||||
|
||||
### `ConnectErrorKind`
|
||||
|
||||
```rust
|
||||
pub enum ConnectErrorKind {
|
||||
ServerParse, // URL parsing failed
|
||||
Dns, // DNS resolution failed
|
||||
Authentication, // Auth signing failed
|
||||
AuthorizationViolation, // Server rejected auth
|
||||
TimedOut, // Connection handshake timeout
|
||||
Tls, // TLS error
|
||||
Io, // Other I/O error
|
||||
MaxReconnects, // Exceeded max reconnect attempts
|
||||
}
|
||||
```
|
||||
|
||||
## Trait Definitions
|
||||
|
||||
The `client::traits` module defines abstract interfaces:
|
||||
|
||||
```rust
|
||||
pub trait Publisher {
|
||||
fn publish_with_reply(&self, subject, reply, payload) -> Future<Output = Result<(), PublishError>>;
|
||||
fn publish_message(&self, msg: OutboundMessage) -> Future<Output = Result<(), PublishError>>;
|
||||
}
|
||||
|
||||
pub trait Subscriber {
|
||||
fn subscribe(&self, subject) -> Future<Output = Result<crate::Subscriber, SubscribeError>>;
|
||||
}
|
||||
|
||||
pub trait Requester {
|
||||
fn send_request(&self, subject, request: Request) -> Future<Output = Result<Message, RequestError>>;
|
||||
}
|
||||
|
||||
pub trait TimeoutProvider {
|
||||
fn timeout(&self) -> Option<Duration>;
|
||||
}
|
||||
```
|
||||
|
||||
`Client` implements all of these. The JetStream `Context` also implements them via delegation.
|
||||
|
||||
## Authentication Types
|
||||
|
||||
### `Auth`
|
||||
|
||||
Container for all authentication methods:
|
||||
|
||||
```rust
|
||||
pub struct Auth {
|
||||
pub jwt: Option<String>,
|
||||
pub nkey: Option<String>,
|
||||
pub signature_callback: Option<CallbackArg1<String, Result<String, AuthError>>>,
|
||||
pub signature: Option<Vec<u8>>,
|
||||
pub username: Option<String>,
|
||||
pub password: Option<String>,
|
||||
pub token: Option<String>,
|
||||
}
|
||||
```
|
||||
|
||||
### `AuthError`
|
||||
|
||||
Simple string error for auth callback failures.
|
||||
|
||||
### `ReconnectToServer`
|
||||
|
||||
Returned by `reconnect_to_server_callback` to select a server and delay:
|
||||
|
||||
```rust
|
||||
pub struct ReconnectToServer {
|
||||
pub addr: ServerAddr,
|
||||
pub delay: Option<Duration>,
|
||||
}
|
||||
```
|
||||
@@ -0,0 +1,278 @@
|
||||
# async-nats: NATS Protocol & Wire Format
|
||||
|
||||
## Protocol Overview
|
||||
|
||||
NATS uses a simple, text-based protocol over TCP. Messages are terminated with `\r\n`. The protocol is symmetric for client and server operations.
|
||||
|
||||
### Client → Server Operations (`ClientOp`)
|
||||
|
||||
```rust
|
||||
pub(crate) enum ClientOp {
|
||||
Publish { subject, payload, respond, headers },
|
||||
Subscribe { sid, subject, queue_group },
|
||||
Unsubscribe { sid, max },
|
||||
Ping,
|
||||
Pong,
|
||||
Connect(ConnectInfo),
|
||||
}
|
||||
```
|
||||
|
||||
### Server → Client Operations (`ServerOp`)
|
||||
|
||||
```rust
|
||||
pub(crate) enum ServerOp {
|
||||
Ok,
|
||||
Info(Box<ServerInfo>),
|
||||
Ping,
|
||||
Pong,
|
||||
Error(ServerError),
|
||||
Message { sid, subject, reply, payload, headers, status, description, length },
|
||||
}
|
||||
```
|
||||
|
||||
## Wire Format: Client Operations
|
||||
|
||||
### CONNECT
|
||||
|
||||
Sent immediately after receiving the first `INFO` from the server:
|
||||
|
||||
```
|
||||
CONNECT {"verbose":false,"pedantic":false,...}\r\n
|
||||
```
|
||||
|
||||
The JSON payload is `ConnectInfo` serialized inline on the same line.
|
||||
|
||||
### PUB (Publish without headers)
|
||||
|
||||
```
|
||||
PUB <subject> [reply-to] <payload-size>\r\n
|
||||
<payload>\r\n
|
||||
```
|
||||
|
||||
Example:
|
||||
```
|
||||
PUB events.data INBOX.67 11\r\n
|
||||
Hello World\r\n
|
||||
```
|
||||
|
||||
### HPUB (Publish with headers)
|
||||
|
||||
When headers are present and non-empty:
|
||||
|
||||
```
|
||||
HPUB <subject> [reply-to] <header-size> <total-size>\r\n
|
||||
<headers>\r\n
|
||||
<payload>\r\n
|
||||
```
|
||||
|
||||
The `<total-size>` = `<header-size>` + `<payload-size>`.
|
||||
|
||||
Header block format:
|
||||
```
|
||||
NATS/1.0\r\n
|
||||
Header-Name: Header-Value\r\n
|
||||
Another-Header: Another-Value\r\n
|
||||
\r\n
|
||||
```
|
||||
|
||||
The version line (`NATS/1.0`) may include a status code and description:
|
||||
```
|
||||
NATS/1.0 404 No Messages\r\n
|
||||
\r\n
|
||||
```
|
||||
|
||||
### SUB (Subscribe)
|
||||
|
||||
```
|
||||
SUB <subject> [queue-group] <sid>\r\n
|
||||
```
|
||||
|
||||
The `sid` (subscription ID) is a client-assigned u64, unique per connection.
|
||||
|
||||
### UNSUB (Unsubscribe)
|
||||
|
||||
```
|
||||
UNSUB <sid> [max]\r\n
|
||||
```
|
||||
|
||||
The optional `max` tells the server to auto-unsubscribe after `max` messages are delivered.
|
||||
|
||||
### PING / PONG
|
||||
|
||||
```
|
||||
PING\r\n
|
||||
PONG\r\n
|
||||
```
|
||||
|
||||
Client sends PING periodically (default every 60s). If 2+ pings are pending without PONG, the connection is considered dead.
|
||||
|
||||
## Wire Format: Server Operations
|
||||
|
||||
### INFO
|
||||
|
||||
First message sent by the server on connection:
|
||||
|
||||
```
|
||||
INFO {"server_id":"NATSxxx","version":"2.10"...}\r\n
|
||||
```
|
||||
|
||||
Also sent asynchronously when cluster topology changes.
|
||||
|
||||
### MSG (Message without headers)
|
||||
|
||||
```
|
||||
MSG <subject> <sid> [reply-to] <payload-size>\r\n
|
||||
<payload>\r\n
|
||||
```
|
||||
|
||||
### HMSG (Message with headers)
|
||||
|
||||
```
|
||||
HMSG <subject> <sid> [reply-to] <header-size> <total-size>\r\n
|
||||
<headers + payload>\r\n
|
||||
```
|
||||
|
||||
### +OK / -ERR
|
||||
|
||||
```
|
||||
+OK\r\n
|
||||
-ERR <description>\r\n
|
||||
```
|
||||
|
||||
Sent only when `verbose=true` in `CONNECT`. The client always sets `verbose=false`, so `+OK` is not expected.
|
||||
|
||||
## Protocol Parser
|
||||
|
||||
The `Connection` struct handles all protocol parsing and serialization:
|
||||
|
||||
### Read Path (`try_read_op`)
|
||||
|
||||
1. Search for `\r\n` in `read_buf` using `memchr::memmem::find`
|
||||
2. Inspect the first bytes to determine the operation type:
|
||||
- `+OK` → `ServerOp::Ok`
|
||||
- `PING` → `ServerOp::Ping`
|
||||
- `PONG` → `ServerOp::Pong`
|
||||
- `-ERR` → `ServerOp::Error(...)` (description is `trim_matches('\'')`)
|
||||
- `INFO ` → `ServerOp::Info(...)` (serde_json deserialization)
|
||||
- `MSG ` → Parse subject/sid/reply/size, then read payload
|
||||
- `HMSG ` → Parse subject/sid/reply/header_len/total_len, then read headers + payload
|
||||
3. For `MSG`/`HMSG`: if the full message body hasn't been read yet, return `None` (wait for more data)
|
||||
4. For `HMSG`: parse the header block — extract version line (`NATS/1.0[ <status>[ <description>]]`), then key-value pairs (supports folded/multi-line header values)
|
||||
|
||||
### Write Path (`enqueue_write_op`)
|
||||
|
||||
Writes into a buffer strategy:
|
||||
- **Small writes** (< 4096 bytes): flattened into `flattened_writes: BytesMut`
|
||||
- **Large writes** (≥ 4096 bytes): appended as separate `Bytes` chunks in `write_buf: VecDeque<Bytes>`
|
||||
|
||||
This enables efficient vectored I/O when the underlying stream supports it.
|
||||
|
||||
### Write Flush Strategy
|
||||
|
||||
The `should_flush()` method returns:
|
||||
- `Yes` — buffers empty but haven't flushed yet
|
||||
- `May` — buffers not empty and haven't flushed
|
||||
- `No` — already flushed or nothing to flush
|
||||
|
||||
The `ConnectionHandler` calls `poll_flush()` after processing commands, ensuring data is actually sent to the server.
|
||||
|
||||
## Vectored I/O
|
||||
|
||||
When `stream.is_write_vectored()` returns true, the connection uses `poll_write_vectored()` to write up to 64 `IoSlice`s at once. This is significantly more efficient for bursty publish patterns.
|
||||
|
||||
```rust
|
||||
const WRITE_VECTORED_CHUNKS: usize = 64;
|
||||
```
|
||||
|
||||
## WebSocket Transport
|
||||
|
||||
When the `websockets` feature is enabled, `WebSocketAdapter<T>` wraps `tokio_websockets::WebSocketStream<T>` to implement `AsyncRead + AsyncWrite`, making WebSocket connections transparent to the protocol layer.
|
||||
|
||||
```rust
|
||||
#[cfg(feature = "websockets")]
|
||||
pub(crate) struct WebSocketAdapter<T> {
|
||||
pub(crate) inner: WebSocketStream<T>,
|
||||
pub(crate) read_buf: BytesMut,
|
||||
}
|
||||
```
|
||||
|
||||
WebSocket connections use `ws://` or `wss://` scheme in the server URL. TLS for `wss://` is handled by the WebSocket library's built-in TLS support.
|
||||
|
||||
## Connection Lifecycle
|
||||
|
||||
### Initial Connection Flow
|
||||
|
||||
```
|
||||
Client Server
|
||||
│ │
|
||||
│──── TCP connect ────────────────────▶ │
|
||||
│◀──── INFO {server_id, nonce, ...} ─── │
|
||||
│──── CONNECT {auth, ...} ──────────▶ │
|
||||
│──── PING ─────────────────────────▶ │
|
||||
│◀──── PONG (or -ERR) ─────────────── │
|
||||
│ │
|
||||
│ [connected, ConnectionHandler runs] │
|
||||
```
|
||||
|
||||
If `tls_first` is enabled, TLS is established before reading INFO:
|
||||
|
||||
```
|
||||
Client Server
|
||||
│ │
|
||||
│──── TCP connect ────────────────────▶ │
|
||||
│──── TLS handshake ─────────────────▶ │
|
||||
│◀──── TLS handshake ──────────────── │
|
||||
│◀──── INFO {...} ──────────────────── │
|
||||
│──── CONNECT + PING ────────────────▶ │
|
||||
│◀──── PONG ────────────────────────── │
|
||||
```
|
||||
|
||||
### Ping/Pong Keepalive
|
||||
|
||||
- Client sends PING every `ping_interval` (default 60s)
|
||||
- Server responds with PONG
|
||||
- If `pending_pings > MAX_PENDING_PINGS (2)`, connection is considered dead
|
||||
- Any server operation resets the ping interval timer
|
||||
|
||||
### Reconnection Flow
|
||||
|
||||
On disconnect:
|
||||
1. `handle_disconnect()` sends `Event::Disconnected` and sets state to `Disconnected`
|
||||
2. `handle_reconnect()` calls `connector.connect()` which:
|
||||
- Shuffles servers (unless `retain_servers_order`)
|
||||
- Sorts by `failed_attempts` (ascending)
|
||||
- Iterates through servers with exponential backoff delay
|
||||
- On each server: DNS resolve → TCP connect → INFO → TLS (if needed) → CONNECT+PING → PONG
|
||||
3. On success:
|
||||
- Sends `Event::Connected`, sets state to `Connected`
|
||||
- Removes closed subscriptions
|
||||
- Re-subscribes all active subscriptions (with adjusted `max = max - delivered`)
|
||||
- Re-subscribes the multiplexer (if active)
|
||||
4. On failure with `MaxReconnects` reached, the handler loop exits
|
||||
|
||||
### Default Reconnect Delay
|
||||
|
||||
Exponential backoff capped at 4 seconds:
|
||||
|
||||
```rust
|
||||
fn reconnect_delay_callback_default(attempts: usize) -> Duration {
|
||||
if attempts <= 1 {
|
||||
Duration::from_millis(0)
|
||||
} else {
|
||||
let exp: u32 = (attempts - 1).try_into().unwrap_or(u32::MAX);
|
||||
cmp::min(Duration::from_millis(2_u64.saturating_pow(exp)), Duration::from_secs(4))
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
| Attempt | Delay |
|
||||
|---------|-------|
|
||||
| 1 | 0ms |
|
||||
| 2 | 0ms |
|
||||
| 3 | 2ms |
|
||||
| 4 | 8ms |
|
||||
| 5 | 32ms |
|
||||
| 6 | 128ms |
|
||||
| 7 | 512ms |
|
||||
| 8 | 2048ms |
|
||||
| 9+ | 4000ms (cap) |
|
||||
@@ -0,0 +1,221 @@
|
||||
# async-nats: Connection Management & Configuration
|
||||
|
||||
## ConnectOptions Builder
|
||||
|
||||
`ConnectOptions` provides a builder for all connection configuration:
|
||||
|
||||
```rust
|
||||
let client = ConnectOptions::new()
|
||||
.require_tls(true)
|
||||
.ping_interval(Duration::from_secs(10))
|
||||
.name("my-service")
|
||||
.connect("demo.nats.io")
|
||||
.await?;
|
||||
```
|
||||
|
||||
### Authentication Methods
|
||||
|
||||
| Method | Description |
|
||||
|--------|-------------|
|
||||
| `with_token(token)` | Token-based auth |
|
||||
| `with_user_and_password(user, pass)` | Username/password auth |
|
||||
| `with_nkey(seed)` | NKey auth (requires `nkeys` feature) |
|
||||
| `with_jwt(jwt, sign_cb)` | JWT + signing callback (requires `nkeys`) |
|
||||
| `with_credentials_file(path)` | Load from `.creds` file (requires `nkeys`) |
|
||||
| `with_credentials(creds_str)` | Parse credentials string (requires `nkeys`) |
|
||||
| `with_auth_callback(cb)` | Dynamic auth callback receiving nonce, returning `Auth` |
|
||||
|
||||
The auth callback is the most flexible — it receives the server nonce and can return any combination of auth fields:
|
||||
|
||||
```rust
|
||||
ConnectOptions::with_auth_callback(move |nonce| async move {
|
||||
let mut auth = Auth::new();
|
||||
auth.username = Some("user".to_string());
|
||||
auth.password = Some("pass".to_string());
|
||||
Ok(auth)
|
||||
})
|
||||
```
|
||||
|
||||
### TLS Configuration
|
||||
|
||||
| Option | Description |
|
||||
|-------|-------------|
|
||||
| `require_tls(bool)` | Require TLS for the connection |
|
||||
| `tls_first()` | Establish TLS before INFO (requires server `handshake_first`) |
|
||||
| `add_root_certificates(path)` | Load root CA certificates from PEM file |
|
||||
| `add_client_certificate(cert, key)` | Load client certificate for mTLS |
|
||||
| `tls_client_config(config)` | Pass a custom `rustls::ClientConfig` |
|
||||
|
||||
Two TLS crypto backends: `ring` (default) or `aws-lc-rs` (via feature flags). FIPS mode available via `aws-lc-rs` + `fips` features.
|
||||
|
||||
### Connection Behavior
|
||||
|
||||
| Option | Default | Description |
|
||||
|--------|---------|-------------|
|
||||
| `connection_timeout` | 5s | Timeout for full connection establishment |
|
||||
| `request_timeout` | 10s | Default timeout for `Client::request` |
|
||||
| `ping_interval` | 60s | How often client sends PING |
|
||||
| `retry_on_initial_connect` | false | Return client immediately, connect in background |
|
||||
| `max_reconnects` | None (unlimited) | Max consecutive reconnect attempts |
|
||||
| `ignore_discovered_servers` | false | Ignore servers advertised in INFO |
|
||||
| `retain_servers_order` | false | Don't shuffle server list on reconnect |
|
||||
| `skip_subject_validation` | false | Skip whitespace validation on publish subjects |
|
||||
| `subscription_capacity` | 65536 | mpsc channel capacity per subscription |
|
||||
| `client_capacity` | 2048 | mpsc channel capacity for command sender |
|
||||
| `custom_inbox_prefix` | `_INBOX` | Custom prefix for inbox subjects |
|
||||
| `read_buffer_capacity` | 65535 | Initial size of the protocol read buffer |
|
||||
| `local_address` | None | Local socket address to bind to |
|
||||
| `no_echo` | false | Don't deliver messages published by this connection |
|
||||
|
||||
### Reconnection Callbacks
|
||||
|
||||
**`reconnect_delay_callback`**: Custom backoff strategy:
|
||||
|
||||
```rust
|
||||
.reconnect_delay_callback(|attempts| {
|
||||
Duration::from_millis(std::cmp::min((attempts * 100) as u64, 8000))
|
||||
})
|
||||
```
|
||||
|
||||
**`reconnect_to_server_callback`**: Select which server to connect to on each reconnect attempt:
|
||||
|
||||
```rust
|
||||
.reconnect_to_server_callback(|servers, _info| async move {
|
||||
servers.first().map(|s| ReconnectToServer {
|
||||
addr: s.addr.clone(),
|
||||
delay: Some(Duration::ZERO),
|
||||
})
|
||||
})
|
||||
```
|
||||
|
||||
Receives `(Vec<Server>, ServerInfo)`, returns `Option<ReconnectToServer>`. If the returned server isn't in the pool, falls back to default selection.
|
||||
|
||||
**`event_callback`**: Receive async notifications:
|
||||
|
||||
```rust
|
||||
.event_callback(|event| async move {
|
||||
match event {
|
||||
Event::Disconnected => println!("disconnected"),
|
||||
Event::Connected => println!("connected"),
|
||||
Event::SlowConsumer(sid) => eprintln!("slow consumer: {sid}"),
|
||||
_ => {}
|
||||
}
|
||||
})
|
||||
```
|
||||
|
||||
## Connection Handler Internals
|
||||
|
||||
### ProcessFut — The Core Event Loop
|
||||
|
||||
The `ConnectionHandler::process()` method creates a custom `Future` (`ProcessFut`) that drives the connection forward. Each `poll()` call:
|
||||
|
||||
1. **Check ping interval** — if timer ticked, send PING; if too many pending pings, disconnect
|
||||
2. **Read server operations** — drain all available `ServerOp`s from `Connection::poll_read_op()`
|
||||
3. **Process drain completions** — remove subscriptions that finished draining
|
||||
4. **Handle commands** — receive up to 16 `Command`s from the mpsc channel and process them
|
||||
5. **Write to socket** — flush the write buffer via `Connection::poll_write()`
|
||||
6. **Flush** — call `poll_flush()` on the underlying stream when needed
|
||||
7. **Check reconnect flag** — if `should_reconnect` is set, shut down and reconnect
|
||||
|
||||
```rust
|
||||
const RECV_CHUNK_SIZE: usize = 16;
|
||||
```
|
||||
|
||||
### Exit Reasons
|
||||
|
||||
The event loop exits with one of:
|
||||
|
||||
| Reason | Action |
|
||||
|--------|--------|
|
||||
| `Disconnected(Option<io::Error>)` | Attempt reconnection |
|
||||
| `ReconnectRequested` | Shut down stream, attempt reconnection |
|
||||
| `Closed` | Send `Event::Closed`, exit loop |
|
||||
|
||||
### Handle Disconnect & Reconnect
|
||||
|
||||
```rust
|
||||
async fn handle_disconnect(&mut self) -> Result<(), ConnectError> {
|
||||
self.pending_pings = 0;
|
||||
self.connector.events_tx.try_send(Event::Disconnected).ok();
|
||||
self.connector.state_tx.send(State::Disconnected).ok();
|
||||
self.handle_reconnect().await
|
||||
}
|
||||
|
||||
async fn handle_reconnect(&mut self) -> Result<(), ConnectError> {
|
||||
let (info, connection) = self.connector.connect().await?;
|
||||
self.connection = connection;
|
||||
let _ = self.info_sender.send(Some(info));
|
||||
|
||||
// Remove closed subscriptions
|
||||
self.subscriptions.retain(|_, sub| !sub.sender.is_closed());
|
||||
|
||||
// Re-subscribe all active subscriptions
|
||||
for (sid, subscription) in &self.subscriptions {
|
||||
self.connection.enqueue_write_op(&ClientOp::Subscribe {
|
||||
sid: *sid,
|
||||
subject: subscription.subject.to_owned(),
|
||||
queue_group: subscription.queue_group.to_owned(),
|
||||
});
|
||||
if let Some(max) = subscription.max {
|
||||
self.connection.enqueue_write_op(&ClientOp::Unsubscribe {
|
||||
sid: *sid,
|
||||
max: Some(max.saturating_sub(subscription.delivered)),
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
// Re-subscribe multiplexer if active
|
||||
if let Some(multiplexer) = &self.multiplexer {
|
||||
self.connection.enqueue_write_op(&ClientOp::Subscribe {
|
||||
sid: MULTIPLEXER_SID,
|
||||
subject: multiplexer.subject.to_owned(),
|
||||
queue_group: None,
|
||||
});
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
```
|
||||
|
||||
## Request/Reply Multiplexer
|
||||
|
||||
The client uses a **multiplexer** pattern for request/reply to avoid creating a separate subscription per request:
|
||||
|
||||
1. A single wildcard subscription is created on first request: `_INBOX.<random_id>.*`
|
||||
2. Each request gets a unique token appended to the inbox: `_INBOX.<random_id>.<token>`
|
||||
3. When a response arrives, the token is extracted from the subject and used to look up the `oneshot::Sender` in `multiplexer.senders`
|
||||
4. The response is forwarded through the oneshot channel to the waiting `send_request()` future
|
||||
|
||||
```rust
|
||||
struct Multiplexer {
|
||||
subject: Subject, // _INBOX.<id>.*
|
||||
prefix: Subject, // _INBOX.<id>.
|
||||
senders: HashMap<String, oneshot::Sender<Message>>, // token → sender
|
||||
}
|
||||
```
|
||||
|
||||
The multiplexer subscription uses `sid = 0` (`MULTIPLEXER_SID`), which is separate from regular subscription IDs (which start at 1).
|
||||
|
||||
### Custom Inbox Bypass
|
||||
|
||||
If a `Request` has a custom `inbox` set, the multiplexer is bypassed — a dedicated subscription is created for that specific request, and the timeout/response logic is handled locally within `send_request()`.
|
||||
|
||||
## Server Pool Management
|
||||
|
||||
The `Connector` maintains a `Vec<Server>` pool. Servers can come from:
|
||||
1. **Explicit URLs** — provided by the user at connect time
|
||||
2. **Discovered servers** — advertised in `INFO.connect_urls` (unless `ignore_discovered_servers` is set)
|
||||
|
||||
On reconnection:
|
||||
- Servers are shuffled (unless `retain_servers_order`)
|
||||
- Sorted by `failed_attempts` (ascending) — prefer servers that haven't failed recently
|
||||
- Each server is tried with exponential backoff delay
|
||||
- On success: `failed_attempts` reset to 0, `did_connect` set to true
|
||||
- On failure: `failed_attempts` incremented, `last_error` updated
|
||||
|
||||
### Dynamic Server Pool Updates
|
||||
|
||||
`Client::set_server_pool()` replaces the pool at runtime:
|
||||
- Per-server state is preserved for servers that appear in both old and new pools
|
||||
- The global reconnection attempt counter is reset
|
||||
- Cannot mix WebSocket and non-WebSocket URLs
|
||||
- Pool cannot be empty
|
||||
373
docs/research/references/nats.rs/nats-async/05-jetstream.md
Normal file
373
docs/research/references/nats.rs/nats-async/05-jetstream.md
Normal file
@@ -0,0 +1,373 @@
|
||||
# async-nats: JetStream
|
||||
|
||||
## Overview
|
||||
|
||||
JetStream is NATS' built-in persistence layer, providing stream-based messaging with at-least-once and exactly-once delivery semantics. The `async-nats` JetStream API is accessed through a `Context` object.
|
||||
|
||||
### Creating a Context
|
||||
|
||||
```rust
|
||||
// Default context (prefix: $JS.API)
|
||||
let jetstream = async_nats::jetstream::new(client);
|
||||
|
||||
// With domain (prefix: $JS.<domain>.API)
|
||||
let jetstream = async_nats::jetstream::with_domain(client, "hub");
|
||||
|
||||
// With custom prefix
|
||||
let jetstream = async_nats::jetstream::with_prefix(client, "JS.acc@hub.API");
|
||||
|
||||
// Builder with fine-grained control
|
||||
let context = ContextBuilder::new()
|
||||
.timeout(Duration::from_secs(5))
|
||||
.api_prefix("MY.JS.API")
|
||||
.max_ack_inflight(1000)
|
||||
.backpressure_on_inflight(true)
|
||||
.ack_timeout(Duration::from_secs(30))
|
||||
.build(client);
|
||||
```
|
||||
|
||||
## Context
|
||||
|
||||
```rust
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct Context {
|
||||
pub(crate) client: Client,
|
||||
pub(crate) prefix: String,
|
||||
pub(crate) timeout: Duration,
|
||||
pub(crate) max_ack_semaphore: Arc<tokio::sync::Semaphore>,
|
||||
pub(crate) ack_sender: mpsc::Sender<(oneshot::Receiver<Message>, OwnedSemaphorePermit)>,
|
||||
pub(crate) backpressure_on_inflight: bool,
|
||||
pub(crate) semaphore_capacity: usize,
|
||||
}
|
||||
```
|
||||
|
||||
### Publish Backpressure
|
||||
|
||||
The context uses a semaphore to limit the number of pending publish acknowledgments:
|
||||
|
||||
- `max_ack_inflight(n)` — sets semaphore capacity (default 5000)
|
||||
- `backpressure_on_inflight(true)` — `publish()` waits for a permit when limit is reached
|
||||
- `backpressure_on_inflight(false)` — `publish()` returns `MaxAckPending` error immediately when limit is reached
|
||||
|
||||
A background **acker task** monitors pending acks with a timeout (`ack_timeout`, default 30s), releasing permits when acks arrive or time out.
|
||||
|
||||
### JetStream API Request Pattern
|
||||
|
||||
All JetStream API calls follow the same pattern:
|
||||
|
||||
1. Build a subject from the prefix: `format!("{}.STREAM.CREATE.<name>", self.prefix)`
|
||||
2. Serialize the request payload as JSON
|
||||
3. Send a request via `client.send_request()` with the API subject
|
||||
4. Deserialize the response as `Response<T>` (which is `Ok(T)` or `Err(ErrorCode)`)
|
||||
|
||||
## Streams
|
||||
|
||||
### Stream Handle
|
||||
|
||||
```rust
|
||||
pub struct Stream<I = Info> {
|
||||
context: Context,
|
||||
info: I,
|
||||
name: String,
|
||||
}
|
||||
```
|
||||
|
||||
`Stream<Info>` carries server-side info. `Stream<()>` is a lightweight handle that skips the INFO fetch. `Stream` (no generic) defaults to `Stream<Info>`.
|
||||
|
||||
### Stream Config
|
||||
|
||||
```rust
|
||||
pub struct Config {
|
||||
pub name: String,
|
||||
pub description: Option<String>,
|
||||
pub subjects: Vec<String>,
|
||||
pub retention: RetentionPolicy,
|
||||
pub max_consumers: i64,
|
||||
pub max_messages: i64,
|
||||
pub max_messages_per_subject: i64,
|
||||
pub max_bytes: i64,
|
||||
pub max_age: Duration,
|
||||
pub max_messages_per_stream: i64,
|
||||
pub max_msg_size: i32,
|
||||
pub discard: DiscardPolicy,
|
||||
pub discard_new_per_subject: bool,
|
||||
pub storage: StorageType,
|
||||
pub num_replicas: usize,
|
||||
pub no_ack: bool,
|
||||
pub duplicate_window: Duration,
|
||||
pub placement: Option<Placement>,
|
||||
pub mirror: Option<Source>,
|
||||
pub sources: Option<Vec<Source>>,
|
||||
pub sealed: bool,
|
||||
pub allow_direct: bool,
|
||||
pub allow_rollup_hdrs: bool,
|
||||
// server_2_10 features:
|
||||
pub compression: Option<Compression>,
|
||||
pub first_sequence: Option<u64>,
|
||||
pub subject_transform: Option<SubjectTransform>,
|
||||
pub republish: Option<Republish>,
|
||||
pub metadata: Option<HashMap<String, String>>,
|
||||
}
|
||||
```
|
||||
|
||||
### Stream Operations
|
||||
|
||||
| Method | Description |
|
||||
|--------|-------------|
|
||||
| `create_stream(config)` | Create a new stream |
|
||||
| `get_stream(name)` | Get stream handle (with INFO) |
|
||||
| `get_stream_no_info(name)` | Get lightweight handle (no server round-trip) |
|
||||
| `get_or_create_stream(config)` | Get existing or create new |
|
||||
| `delete_stream(name)` | Delete a stream |
|
||||
| `update_stream(config)` | Update stream configuration |
|
||||
| `create_or_update_stream(config)` | Update or create if not found |
|
||||
| `stream_names()` | `Stream` of stream names (paginated) |
|
||||
| `streams()` | `Stream` of stream info (paginated) |
|
||||
| `stream_by_subject(subject)` | Find stream name containing subject |
|
||||
|
||||
### Stream Handle Methods
|
||||
|
||||
```rust
|
||||
let stream: Stream = jetstream.get_stream("events").await?;
|
||||
|
||||
// Info
|
||||
let info: Info = stream.info().await?; // Fresh info from server
|
||||
let info: &Info = stream.cached_info(); // Cached info from last fetch
|
||||
|
||||
// Message operations
|
||||
stream.get_raw_message(seq).await?; // Get raw message by sequence
|
||||
stream.get_last_raw_message_by_subject(subj).await?; // Get last message for subject
|
||||
stream.direct_get(seq).await?; // Direct get (if allow_direct)
|
||||
stream.direct_get_last_for_subject(subj).await?; // Direct last by subject
|
||||
stream.delete_message(seq).await?; // Delete a specific message
|
||||
stream.purge().await?; // Purge all messages
|
||||
stream.purge().filter(subj).await?; // Purge messages for subject
|
||||
|
||||
// Consumers
|
||||
stream.create_consumer(config).await?; // Create consumer bound to stream
|
||||
stream.get_consumer(name).await?; // Get existing consumer
|
||||
stream.delete_consumer(name).await?; // Delete consumer
|
||||
```
|
||||
|
||||
## Consumers
|
||||
|
||||
### Consumer Types
|
||||
|
||||
Two consumer types, each with distinct delivery models:
|
||||
|
||||
1. **Pull Consumer** (`pull::Config` / `PullConsumer`) — Client explicitly requests batches of messages
|
||||
2. **Push Consumer** (`push::Config` / `PushConsumer`) — Server pushes messages to a deliver subject
|
||||
|
||||
### Pull Consumer
|
||||
|
||||
```rust
|
||||
let consumer: PullConsumer = stream
|
||||
.get_or_create_consumer("my-consumer", pull::Config {
|
||||
durable_name: Some("my-consumer".to_string()),
|
||||
..Default::default()
|
||||
})
|
||||
.await?;
|
||||
```
|
||||
|
||||
**Key methods**:
|
||||
- `consumer.batch(n).await?` — Fetch up to `n` messages (one-shot batch)
|
||||
- `consumer.messages().await?` — Continuous `Stream` of messages
|
||||
- `consumer.sequence(n).await?` — Continuous `Stream` of batches of `n` messages
|
||||
- `consumer.fetch().max(n).expires(dur).await?` — Configurable fetch
|
||||
|
||||
Each message from a pull consumer is a `jetstream::Message` which has `ack()` methods.
|
||||
|
||||
### Push Consumer
|
||||
|
||||
Two push consumer variants:
|
||||
|
||||
1. **Standard** (`push::Config`) — messages delivered to a specific subject
|
||||
2. **Ordered** (`push::OrderedConfig`) — auto-recreated on failure, with flow control
|
||||
|
||||
```rust
|
||||
// Standard push
|
||||
let consumer = stream.create_consumer(push::Config {
|
||||
deliver_subject: "deliver.subject".to_string(),
|
||||
durable_name: Some("push-consumer".to_string()),
|
||||
..Default::default()
|
||||
}).await?;
|
||||
|
||||
// Ordered push (no durable name, auto-recreates on failure)
|
||||
let consumer = stream.create_consumer(push::OrderedConfig {
|
||||
deliver_subject: client.new_inbox(),
|
||||
filter_subject: "events.>".to_string(),
|
||||
..Default::default()
|
||||
}).await?;
|
||||
```
|
||||
|
||||
### Consumer Config (Shared Fields)
|
||||
|
||||
```rust
|
||||
pub struct Config {
|
||||
// Pull fields
|
||||
pub durable_name: Option<String>,
|
||||
pub name: Option<String>,
|
||||
|
||||
// Push fields
|
||||
pub deliver_subject: Option<String>,
|
||||
pub deliver_group: Option<String>,
|
||||
pub deliver_policy: DeliverPolicy,
|
||||
pub opt_start_time: Option<DateTime>,
|
||||
pub opt_start_sequence: Option<u64>,
|
||||
pub ack_policy: AckPolicy,
|
||||
pub ack_wait: Duration,
|
||||
pub max_deliver: i64,
|
||||
pub backoff: Vec<Duration>,
|
||||
pub filter_subject: String,
|
||||
pub filter_subjects: Vec<String>, // server_2_10+
|
||||
pub replay_policy: ReplayPolicy,
|
||||
pub rate_limit_bps: Option<u64>,
|
||||
pub max_waiting: i64, // pull: max outstanding pull requests
|
||||
pub max_ack_pending: i64,
|
||||
pub flow_control: bool,
|
||||
pub idle_heartbeat: Duration,
|
||||
pub headers_only: bool,
|
||||
pub num_replicas: usize,
|
||||
pub mem_storage: bool,
|
||||
pub description: Option<String>,
|
||||
pub metadata: Option<HashMap<String, String>>,
|
||||
pub inactive_threshold: Option<Duration>, // for ephemeral consumers
|
||||
}
|
||||
```
|
||||
|
||||
### Deliver Policy
|
||||
|
||||
```rust
|
||||
pub enum DeliverPolicy {
|
||||
All, // Deliver all messages
|
||||
Last, // Deliver last message only
|
||||
New, // Deliver only new messages
|
||||
ByStartSequence { start_sequence: u64 },
|
||||
ByStartTime { start_time: DateTime },
|
||||
LastPerSubject, // Deliver last message per subject
|
||||
}
|
||||
```
|
||||
|
||||
### Ack Policy
|
||||
|
||||
```rust
|
||||
pub enum AckPolicy {
|
||||
None, // No acknowledgment needed
|
||||
All, // Ack all messages up to this one
|
||||
Explicit, // Ack each message individually
|
||||
}
|
||||
```
|
||||
|
||||
## JetStream Messages
|
||||
|
||||
### `jetstream::Message`
|
||||
|
||||
Wraps a core `Message` with JetStream-specific metadata:
|
||||
|
||||
```rust
|
||||
pub struct Message {
|
||||
pub message: crate::Message, // The underlying NATS message
|
||||
pub ack_subject: Subject, // Subject for sending acks
|
||||
pub stream: String, // Stream name
|
||||
pub consumer: String, // Consumer name
|
||||
pub stream_sequence: u64, // Sequence in stream
|
||||
pub consumer_sequence: u64, // Sequence for this consumer
|
||||
pub delivered: u64, // Delivery count
|
||||
pub pending: u64, // Pending message count
|
||||
pub published: DateTime, // Original publish time
|
||||
}
|
||||
```
|
||||
|
||||
### Ack Methods
|
||||
|
||||
```rust
|
||||
// In-memory ack (non-persistent, fast)
|
||||
message.ack().await?;
|
||||
|
||||
// Ack with specific type
|
||||
message.ack_with(AckKind::Nak).await?;
|
||||
message.ack_with(AckKind::Progress).await?;
|
||||
message.ack_with(AckKind::Term).await?;
|
||||
message.ack_with(AckKind::NakWithDelay(duration)).await?;
|
||||
message.ack_with(AckKind::TermWithReason("reason")).await?;
|
||||
```
|
||||
|
||||
### `AckKind`
|
||||
|
||||
```rust
|
||||
pub enum AckKind {
|
||||
Ack, // +ACK — message processed
|
||||
Nak, // -NAK — re-deliver
|
||||
Progress, // PRI — still working
|
||||
Term, // +TERM — don't redeliver
|
||||
NakWithDelay(Duration), // -NAK with re-delivery delay
|
||||
TermWithReason(String), // +TERM with reason
|
||||
}
|
||||
```
|
||||
|
||||
## JetStream Publish
|
||||
|
||||
### `Context::publish()`
|
||||
|
||||
JetStream publish returns a `PublishAckFuture` — a future that resolves to a `PublishAck`:
|
||||
|
||||
```rust
|
||||
let ack_future = jetstream.publish("events", "data".into()).await?;
|
||||
let ack: PublishAck = ack_future.await?; // Wait for server acknowledgment
|
||||
```
|
||||
|
||||
### `PublishAck`
|
||||
|
||||
```rust
|
||||
pub struct PublishAck {
|
||||
pub stream: String,
|
||||
pub sequence: u64,
|
||||
pub domain: String,
|
||||
pub duplicate: bool,
|
||||
}
|
||||
```
|
||||
|
||||
### `PublishMessage` Builder
|
||||
|
||||
```rust
|
||||
let ack = jetstream.send_publish(
|
||||
"events",
|
||||
PublishMessage::build()
|
||||
.payload("data".into())
|
||||
.message_id("uuid-123") // Deduplication ID
|
||||
.expected_stream("events") // Fail if wrong stream
|
||||
.expected_last_msg_id("prev-id")
|
||||
.expected_last_sequence(42)
|
||||
.headers(headers),
|
||||
).await?;
|
||||
```
|
||||
|
||||
## Pagination
|
||||
|
||||
Stream and consumer listing uses pagination internally:
|
||||
|
||||
```rust
|
||||
pub struct StreamNames {
|
||||
context: Context,
|
||||
offset: usize,
|
||||
page_request: Option<Request>,
|
||||
streams: Vec<String>,
|
||||
subject: Option<String>,
|
||||
done: bool,
|
||||
}
|
||||
```
|
||||
|
||||
Implements `futures_util::Stream<Item = Result<String, Error>>`, lazily fetching pages as needed.
|
||||
|
||||
## Error Handling
|
||||
|
||||
JetStream errors follow the `Response<T>` pattern:
|
||||
|
||||
```rust
|
||||
pub enum Response<T> {
|
||||
Ok(T),
|
||||
Err { error: ErrorCode },
|
||||
}
|
||||
```
|
||||
|
||||
`ErrorCode` carries the server's error code and description. Most JetStream-specific errors map to typed error enums (e.g., `CreateStreamError`, `ConsumerError`, etc.).
|
||||
@@ -0,0 +1,237 @@
|
||||
# async-nats: Key-Value Store
|
||||
|
||||
## Overview
|
||||
|
||||
The Key-Value (KV) store is an abstraction built on top of JetStream streams. Each KV bucket is backed by a JetStream stream with the naming convention `KV_<bucket_name>`. Keys are mapped to subjects under the `$KV.<bucket>.<key>` prefix.
|
||||
|
||||
The KV feature requires `kv` (which implies `jetstream`).
|
||||
|
||||
## Store Handle
|
||||
|
||||
```rust
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct Store {
|
||||
pub name: String,
|
||||
pub stream_name: String,
|
||||
pub prefix: String, // $KV.<bucket>.
|
||||
pub put_prefix: Option<String>, // For mirrored buckets
|
||||
pub use_jetstream_prefix: bool, // Whether to prepend JS API prefix
|
||||
pub stream: Stream,
|
||||
}
|
||||
```
|
||||
|
||||
## Bucket Config
|
||||
|
||||
```rust
|
||||
#[derive(Debug, Clone, Default)]
|
||||
pub struct Config {
|
||||
pub bucket: String,
|
||||
pub description: String,
|
||||
pub max_value_size: i32,
|
||||
pub history: i64, // Max historical entries per key (1-64)
|
||||
pub max_age: Duration, // Max age of any entry
|
||||
pub max_bytes: i64, // Total bucket size limit
|
||||
pub storage: StorageType, // File or Memory
|
||||
pub num_replicas: usize,
|
||||
pub republish: Option<Republish>,
|
||||
pub mirror: Option<Source>, // Mirror another bucket
|
||||
pub sources: Option<Vec<Source>>,
|
||||
pub mirror_direct: bool,
|
||||
pub compression: bool, // server_2_10+
|
||||
pub placement: Option<Placement>,
|
||||
pub limit_markers: Option<Duration>, // server_2_11+
|
||||
}
|
||||
```
|
||||
|
||||
## Creating/Accessing Buckets
|
||||
|
||||
```rust
|
||||
// Create a new bucket
|
||||
let kv = jetstream.create_key_value(kv::Config {
|
||||
bucket: "my-bucket".to_string(),
|
||||
history: 10,
|
||||
max_age: Duration::from_secs(3600),
|
||||
..Default::default()
|
||||
}).await?;
|
||||
|
||||
// Get an existing bucket
|
||||
let kv = jetstream.get_key_value("my-bucket").await?;
|
||||
|
||||
// Create or update
|
||||
let kv = jetstream.create_or_update_key_value(kv::Config { ... }).await?;
|
||||
|
||||
// Delete a bucket
|
||||
jetstream.delete_key_value("my-bucket").await?;
|
||||
```
|
||||
|
||||
## KV Operations
|
||||
|
||||
### Put
|
||||
|
||||
```rust
|
||||
let revision: u64 = kv.put("key", "value".into()).await?;
|
||||
```
|
||||
|
||||
Publishes to `$KV.<bucket>.<key>` (or with JS prefix). The JetStream stream stores it, and the returned sequence number serves as the revision.
|
||||
|
||||
### Get
|
||||
|
||||
```rust
|
||||
let value: Option<Bytes> = kv.get("key").await?;
|
||||
```
|
||||
|
||||
Returns `None` if the key doesn't exist or was deleted/purged. Uses either direct get (if `allow_direct`) or the standard message API.
|
||||
|
||||
### Entry
|
||||
|
||||
```rust
|
||||
let entry: Option<Entry> = kv.entry("key").await?;
|
||||
let entry: Option<Entry> = kv.entry_for_revision("key", 2).await?;
|
||||
```
|
||||
|
||||
Returns full entry metadata:
|
||||
|
||||
```rust
|
||||
pub struct Entry {
|
||||
pub bucket: String,
|
||||
pub key: String,
|
||||
pub value: Bytes,
|
||||
pub revision: u64,
|
||||
pub created: DateTime,
|
||||
pub delta: u64,
|
||||
pub operation: Operation,
|
||||
pub seen_current: bool,
|
||||
}
|
||||
```
|
||||
|
||||
### Create (Put if not exists)
|
||||
|
||||
```rust
|
||||
let revision: u64 = kv.create("key", "value".into()).await?;
|
||||
```
|
||||
|
||||
Uses `update` with `expected_last_subject_sequence = 0` (create-only). If the key exists and is deleted/purged, it's re-created.
|
||||
|
||||
### Update (Conditional Put)
|
||||
|
||||
```rust
|
||||
let revision: u64 = kv.update("key", "value".into(), last_revision).await?;
|
||||
```
|
||||
|
||||
Uses the `Nats-Expected-Last-Subject-Sequence` header for optimistic concurrency control. Only succeeds if the key's current revision matches.
|
||||
|
||||
### Delete
|
||||
|
||||
```rust
|
||||
kv.delete("key").await?;
|
||||
kv.delete_expect_revision("key", Some(revision)).await?;
|
||||
```
|
||||
|
||||
Non-destructive — publishes a `DEL` marker message. The key appears deleted to `get()`, but history is preserved (up to `history` limit).
|
||||
|
||||
### Purge
|
||||
|
||||
```rust
|
||||
kv.purge("key").await?;
|
||||
kv.purge_with_ttl("key", Duration::from_secs(10)).await?;
|
||||
kv.purge_expect_revision("key", Some(revision)).await?;
|
||||
```
|
||||
|
||||
Destructive — publishes a `PURGE` marker with rollup header, removing all previous revisions of the key. Leaves a single purge entry.
|
||||
|
||||
### Watch
|
||||
|
||||
```rust
|
||||
// Watch for new changes
|
||||
let mut watch = kv.watch("key").await?;
|
||||
// Watch with initial value
|
||||
let mut watch = kv.watch_with_history("key").await?;
|
||||
// Watch from specific revision
|
||||
let mut watch = kv.watch_from_revision("key", 5).await?;
|
||||
// Watch all keys
|
||||
let mut watch = kv.watch_all().await?;
|
||||
// Watch multiple keys (server_2_10+)
|
||||
let mut watch = kv.watch_many(["foo", "bar"]).await?;
|
||||
```
|
||||
|
||||
`Watch` implements `futures_util::Stream<Item = Result<Entry, WatcherError>>`.
|
||||
|
||||
Under the hood, each watch creates an **ordered push consumer** on the KV stream with:
|
||||
- `filter_subject` matching `$KV.<bucket>.<key>`
|
||||
- `replay_policy: Instant`
|
||||
- Appropriate `deliver_policy`
|
||||
|
||||
### History
|
||||
|
||||
```rust
|
||||
let mut history = kv.history("key").await?;
|
||||
```
|
||||
|
||||
Returns a `Stream` of all past `Entry` values for a key (including deletes/purges).
|
||||
|
||||
### Keys
|
||||
|
||||
```rust
|
||||
let mut keys = kv.keys().await?;
|
||||
```
|
||||
|
||||
Returns a `Stream<String>` of all current keys. Uses a headers-only consumer with `LastPerSubject` deliver policy to efficiently scan the bucket.
|
||||
|
||||
## Entry Operations
|
||||
|
||||
```rust
|
||||
#[derive(Debug, Clone, Copy, Eq, PartialEq)]
|
||||
pub enum Operation {
|
||||
Put, // Value was put
|
||||
Delete, // Value was deleted (DEL marker)
|
||||
Purge, // Value was purged (PURGE marker with rollup)
|
||||
}
|
||||
```
|
||||
|
||||
The operation type is determined from the `KV-Operation` header (`PUT`, `DEL`, `PURGE`) or the `Nats-Marker-Reason` header (fallback for server-generated markers like `MaxAge`, `Purge`, `Remove`).
|
||||
|
||||
## Key and Bucket Name Validation
|
||||
|
||||
```rust
|
||||
// Bucket: alphanumeric, dash, underscore only
|
||||
VALID_BUCKET_RE: \A[a-zA-Z0-9_-]+\z
|
||||
|
||||
// Key: alphanumeric, dash, slash, underscore, equals, dot; no leading/trailing dots
|
||||
VALID_KEY_RE: \A[-/_=\.a-zA-Z0-9]+\z
|
||||
```
|
||||
|
||||
## Bucket Status
|
||||
|
||||
```rust
|
||||
let status: Status = kv.status().await?;
|
||||
```
|
||||
|
||||
Wraps stream info to provide bucket-level statistics (bucket name, message count, byte count, etc.).
|
||||
|
||||
## Mirrored Buckets
|
||||
|
||||
When a bucket is configured as a mirror of another (potentially in a different account/domain):
|
||||
|
||||
- `prefix` is set to `$KV.<mirror_bucket>.`
|
||||
- `put_prefix` may be set to the source bucket's API prefix for cross-domain writes
|
||||
- `use_jetstream_prefix` is adjusted based on whether the mirror is in the same domain
|
||||
|
||||
## KV → Stream Config Mapping
|
||||
|
||||
When creating a KV bucket, the `Config` is converted to a JetStream `stream::Config`:
|
||||
|
||||
| KV Config | Stream Config |
|
||||
|-----------|---------------|
|
||||
| `bucket` | `name = "KV_<bucket>"` |
|
||||
| `subjects` | `["$KV.<bucket>.>"]` |
|
||||
| `max_messages_per_subject` | `history` (max 64) |
|
||||
| `max_age` | `max_age` |
|
||||
| `max_bytes` | `max_bytes` |
|
||||
| `storage` | `storage` |
|
||||
| `num_replicas` | `num_replicas` |
|
||||
| `republish` | `republish` |
|
||||
| `mirror` | `mirror` |
|
||||
| `discard` | `DiscardPolicy::New` |
|
||||
| `allow_direct` | `true` |
|
||||
| `allow_rollup_hdrs` | `true |
|
||||
| `max_msg_size` | `max_value_size` |
|
||||
245
docs/research/references/nats.rs/nats-async/07-object-store.md
Normal file
245
docs/research/references/nats.rs/nats-async/07-object-store.md
Normal file
@@ -0,0 +1,245 @@
|
||||
# async-nats: Object Store
|
||||
|
||||
## Overview
|
||||
|
||||
The Object Store provides large-object storage built on JetStream. Objects are chunked and stored as messages in a JetStream stream, with metadata stored separately. The stream is named `OBJ_<bucket_name>`.
|
||||
|
||||
The object-store feature requires `object-store` (which implies `jetstream` + `crypto`).
|
||||
|
||||
## ObjectStore Handle
|
||||
|
||||
```rust
|
||||
#[derive(Clone)]
|
||||
pub struct ObjectStore {
|
||||
pub(crate) name: String,
|
||||
pub(crate) stream: Stream,
|
||||
}
|
||||
```
|
||||
|
||||
## Object Store Config
|
||||
|
||||
```rust
|
||||
#[derive(Debug, Default, Clone, Serialize, Deserialize)]
|
||||
pub struct Config {
|
||||
pub bucket: String,
|
||||
pub description: Option<String>,
|
||||
pub max_age: Duration,
|
||||
pub max_bytes: i64,
|
||||
pub storage: StorageType,
|
||||
pub num_replicas: usize,
|
||||
pub compression: bool,
|
||||
pub placement: Option<Placement>,
|
||||
}
|
||||
```
|
||||
|
||||
## Creating/Accessing Object Stores
|
||||
|
||||
```rust
|
||||
// Create
|
||||
let bucket = jetstream.create_object_store(object_store::Config {
|
||||
bucket: "my-bucket".to_string(),
|
||||
..Default::default()
|
||||
}).await?;
|
||||
|
||||
// Get existing
|
||||
let bucket = jetstream.get_object_store("my-bucket").await?;
|
||||
|
||||
// Delete
|
||||
jetstream.delete_object_store("my-bucket").await?;
|
||||
```
|
||||
|
||||
## Object Store Operations
|
||||
|
||||
### Put
|
||||
|
||||
```rust
|
||||
let info: ObjectInfo = bucket.put("file.txt", &mut async_read).await?;
|
||||
```
|
||||
|
||||
The put operation:
|
||||
1. Reads data from any `AsyncRead + Unpin` source in chunks (default 128KB)
|
||||
2. Each chunk is published to `$O.<bucket>.C.<nuid>` (chunk subject)
|
||||
3. SHA-256 digest is computed incrementally
|
||||
4. After all chunks, metadata is published to `$O.<bucket>.M.<encoded_name>` with a rollup header
|
||||
5. If the object previously existed, old chunks are purged
|
||||
|
||||
### Get
|
||||
|
||||
```rust
|
||||
let mut object: Object = bucket.get("file.txt").await?;
|
||||
```
|
||||
|
||||
Returns an `Object` that implements `tokio::io::AsyncRead`:
|
||||
|
||||
```rust
|
||||
let mut bytes = Vec::new();
|
||||
object.read_to_end(&mut bytes).await?;
|
||||
```
|
||||
|
||||
On read, the Object:
|
||||
1. Creates an ordered push consumer on `$O.<bucket>.C.<nuid>`
|
||||
2. Streams chunk messages, feeding bytes to the reader
|
||||
3. Verifies SHA-256 digest after the last chunk
|
||||
4. If digest doesn't match, returns `io::ErrorKind::InvalidData`
|
||||
|
||||
### Delete
|
||||
|
||||
```rust
|
||||
bucket.delete("file.txt").await?;
|
||||
```
|
||||
|
||||
Marks the object as deleted in metadata (sets `deleted = true`, `chunks = 0`, `size = 0`) with a rollup, then purges all chunk messages.
|
||||
|
||||
### Info
|
||||
|
||||
```rust
|
||||
let info: ObjectInfo = bucket.info("file.txt").await?;
|
||||
```
|
||||
|
||||
Fetches the last metadata message for the object (from `$O.<bucket>.M.<encoded_name>`).
|
||||
|
||||
### Watch
|
||||
|
||||
```rust
|
||||
let mut watcher = bucket.watch().await?;
|
||||
let mut watcher = bucket.watch_with_history().await?;
|
||||
```
|
||||
|
||||
Returns a `Stream<Item = Result<ObjectInfo, WatcherError>>`. Uses an ordered push consumer on `$O.<bucket>.M.>`.
|
||||
|
||||
### List
|
||||
|
||||
```rust
|
||||
let mut list = bucket.list().await?;
|
||||
```
|
||||
|
||||
Returns a `Stream<Item = Result<ObjectInfo, ListerError>>`. Lists all non-deleted objects. Uses `DeliverPolicy::All` to replay all metadata.
|
||||
|
||||
### Seal
|
||||
|
||||
```rust
|
||||
bucket.seal().await?;
|
||||
```
|
||||
|
||||
Sets the underlying stream's `sealed = true`, preventing any further modifications.
|
||||
|
||||
### Links
|
||||
|
||||
```rust
|
||||
// Link to another object (same or different bucket)
|
||||
let info = bucket.add_link("link_name", &object).await?;
|
||||
|
||||
// Link to another bucket
|
||||
let info = bucket.add_bucket_link("link_name", "other_bucket").await?;
|
||||
```
|
||||
|
||||
Links are followed automatically when `get()` is called (one level deep). Cannot link to a deleted object or create a link to a link.
|
||||
|
||||
### Update Metadata
|
||||
|
||||
```rust
|
||||
bucket.update_metadata("object", object_store::UpdateMetadata {
|
||||
name: "new_name".to_string(),
|
||||
description: Some("updated description".to_string()),
|
||||
..Default::default()
|
||||
}).await?;
|
||||
```
|
||||
|
||||
If the name changes, old metadata is purged and new metadata is published.
|
||||
|
||||
## Object Types
|
||||
|
||||
### ObjectInfo
|
||||
|
||||
```rust
|
||||
#[derive(Debug, Clone, Serialize, Deserialize, Eq, PartialEq)]
|
||||
pub struct ObjectInfo {
|
||||
pub name: String,
|
||||
pub description: Option<String>,
|
||||
pub metadata: HashMap<String, String>,
|
||||
pub headers: Option<HeaderMap>,
|
||||
pub options: Option<ObjectOptions>,
|
||||
pub bucket: String,
|
||||
pub nuid: String,
|
||||
pub size: usize,
|
||||
pub chunks: usize,
|
||||
pub modified: Option<DateTime>,
|
||||
pub digest: Option<String>, // Format: "SHA-256=<base64url-digest>"
|
||||
pub deleted: bool,
|
||||
}
|
||||
```
|
||||
|
||||
### ObjectMetadata
|
||||
|
||||
```rust
|
||||
#[derive(Debug, Default, Clone, Serialize, Deserialize, Eq, PartialEq)]
|
||||
pub struct ObjectMetadata {
|
||||
pub name: String,
|
||||
pub description: Option<String>,
|
||||
pub chunk_size: Option<usize>,
|
||||
pub metadata: HashMap<String, String>,
|
||||
pub headers: Option<HeaderMap>,
|
||||
}
|
||||
```
|
||||
|
||||
### ObjectLink
|
||||
|
||||
```rust
|
||||
#[derive(Debug, Default, Clone, Serialize, Deserialize, Eq, PartialEq)]
|
||||
pub struct ObjectLink {
|
||||
pub name: Option<String>, // None = bucket link, Some = object link
|
||||
pub bucket: String,
|
||||
}
|
||||
```
|
||||
|
||||
### Object
|
||||
|
||||
```rust
|
||||
pub struct Object {
|
||||
pub info: ObjectInfo,
|
||||
remaining_bytes: VecDeque<u8>,
|
||||
has_pending_messages: bool,
|
||||
digest: Option<Sha256>,
|
||||
subscription: Option<crate::jetstream::consumer::push::Ordered>,
|
||||
subscription_future: Option<BoxFuture<'static, Result<Ordered, StreamError>>>,
|
||||
stream: Stream,
|
||||
}
|
||||
```
|
||||
|
||||
Implements `tokio::io::AsyncRead`. Lazy-creates the consumer on first read.
|
||||
|
||||
## Subject Naming Convention
|
||||
|
||||
| Purpose | Subject Pattern |
|
||||
|---------|----------------|
|
||||
| Chunks | `$O.<bucket>.C.<nuid>` |
|
||||
| Metadata | `$O.<bucket>.M.<base64url-encoded-name>` |
|
||||
|
||||
Object names are base64url-encoded in metadata subjects to allow arbitrary characters (the raw name might contain characters invalid in NATS subjects).
|
||||
|
||||
## Validation
|
||||
|
||||
```rust
|
||||
// Bucket: alphanumeric, dash, underscore only
|
||||
BUCKET_NAME_RE: \A[a-zA-Z0-9_-]+\z
|
||||
|
||||
// Object name: alphanumeric, dash, slash, underscore, equals, dot; no leading/trailing dots
|
||||
OBJECT_NAME_RE: \A[-/_=\.a-zA-Z0-9]+\z
|
||||
```
|
||||
|
||||
## Data Integrity
|
||||
|
||||
The object store uses SHA-256 hashing (from the `crypto` module) to verify data integrity:
|
||||
|
||||
1. On `put()`: SHA-256 is computed incrementally as chunks are read. The digest is stored in `ObjectInfo.digest` as `"SHA-256=<base64url>"`.
|
||||
2. On `get()` (via `AsyncRead`): SHA-256 is verified after the last chunk is read. If the computed digest doesn't match the stored digest, `io::ErrorKind::InvalidData` is returned.
|
||||
|
||||
```rust
|
||||
// crypto module
|
||||
pub(crate) struct Sha256 { ... }
|
||||
impl Sha256 {
|
||||
pub fn new() -> Self;
|
||||
pub fn update(&mut self, data: &[u8]);
|
||||
pub fn finish(self) -> [u8; 32];
|
||||
}
|
||||
```
|
||||
272
docs/research/references/nats.rs/nats-async/08-service-api.md
Normal file
272
docs/research/references/nats.rs/nats-async/08-service-api.md
Normal file
@@ -0,0 +1,272 @@
|
||||
# async-nats: Service API
|
||||
|
||||
## Overview
|
||||
|
||||
The Service API provides a microservice request/reply pattern with built-in service discovery, health checking, and statistics. It follows the [NATS Micro v1 specification](https://github.com/nats-io/nats-architecture-design/blob/main/adr/ADR-33.md).
|
||||
|
||||
The `service` feature is required.
|
||||
|
||||
## Service
|
||||
|
||||
```rust
|
||||
#[derive(Debug)]
|
||||
pub struct Service {
|
||||
endpoints_state: Arc<Mutex<Endpoints>>,
|
||||
info: Info,
|
||||
client: Client,
|
||||
handle: JoinHandle<Result<(), Error>>,
|
||||
shutdown_tx: Sender<()>,
|
||||
subjects: Arc<Mutex<Vec<String>>>,
|
||||
queue_group: String,
|
||||
}
|
||||
```
|
||||
|
||||
## Creating a Service
|
||||
|
||||
Via the `ServiceExt` trait on `Client`:
|
||||
|
||||
```rust
|
||||
use async_nats::service::ServiceExt;
|
||||
|
||||
// Builder pattern
|
||||
let mut service = client
|
||||
.service_builder()
|
||||
.description("product service")
|
||||
.stats_handler(|endpoint, stats| serde_json::json!({ "endpoint": endpoint }))
|
||||
.metadata(HashMap::from([("version".into(), "v2".into())]))
|
||||
.queue_group("products-group")
|
||||
.start("products", "1.0.0")
|
||||
.await?;
|
||||
|
||||
// Direct config
|
||||
let mut service = client
|
||||
.add_service(service::Config {
|
||||
name: "products".to_string(),
|
||||
version: "1.0.0".to_string(),
|
||||
description: Some("product service".to_string()),
|
||||
stats_handler: None,
|
||||
metadata: None,
|
||||
queue_group: None,
|
||||
})
|
||||
.await?;
|
||||
```
|
||||
|
||||
Service name must match `^[A-Za-z0-9\-_]+$`. Version must be valid SemVer.
|
||||
|
||||
## Service Verbs
|
||||
|
||||
Every service automatically subscribes to three verb subjects for discovery and monitoring:
|
||||
|
||||
| Verb | Subject Pattern | Purpose |
|
||||
|------|----------------|---------|
|
||||
| PING | `$SRV.PING`, `$SRV.PING.<name>`, `$SRV.PING.<name>.<id>` | Lightweight health check |
|
||||
| INFO | `$SRV.INFO.<name>`, `$SRV.INFO.<name>.<id>` | Service metadata |
|
||||
| STATS | `$SRV.STATS.<name>`, `$SRV.STATS.<name>.<id>` | Service + endpoint statistics |
|
||||
|
||||
A background task handles these verb requests and responds with JSON payloads.
|
||||
|
||||
## Service Config
|
||||
|
||||
```rust
|
||||
#[derive(Serialize, Deserialize, Debug)]
|
||||
pub struct Config {
|
||||
pub name: String,
|
||||
pub description: Option<String>,
|
||||
pub version: String,
|
||||
pub stats_handler: Option<StatsHandler>,
|
||||
pub metadata: Option<HashMap<String, String>>,
|
||||
pub queue_group: Option<String>,
|
||||
}
|
||||
```
|
||||
|
||||
## Adding Endpoints
|
||||
|
||||
```rust
|
||||
// Simple endpoint
|
||||
let mut endpoint = service.endpoint("get-products").await?;
|
||||
|
||||
// Endpoint with custom name and metadata
|
||||
let endpoint = service
|
||||
.endpoint_builder()
|
||||
.name("api")
|
||||
.metadata(HashMap::from([("auth".into(), "required".into())]))
|
||||
.queue_group("custom-group")
|
||||
.add("products")
|
||||
.await?;
|
||||
|
||||
// Grouped endpoints
|
||||
let v1 = service.group("v1");
|
||||
let products = v1.endpoint("products").await?;
|
||||
let orders = v1.endpoint("orders").await?;
|
||||
|
||||
// Nested groups
|
||||
let v1_api = service.group("api").group("v1");
|
||||
```
|
||||
|
||||
## Endpoint
|
||||
|
||||
```rust
|
||||
pub struct Endpoint {
|
||||
requests: Subscriber,
|
||||
stats: Arc<Mutex<Endpoints>>,
|
||||
client: Client,
|
||||
endpoint: String,
|
||||
shutdown: Option<ShutdownRx>,
|
||||
shutdown_future: Option<ShutdownReceiverFuture>,
|
||||
}
|
||||
```
|
||||
|
||||
Implements `futures_util::Stream<Item = Request>`.
|
||||
|
||||
```rust
|
||||
while let Some(request) = endpoint.next().await {
|
||||
request.respond(Ok("response data".into())).await?;
|
||||
}
|
||||
```
|
||||
|
||||
## Service Request
|
||||
|
||||
```rust
|
||||
#[derive(Debug)]
|
||||
pub struct Request {
|
||||
issued: Instant,
|
||||
client: Client,
|
||||
pub message: Message,
|
||||
endpoint: String,
|
||||
stats: Arc<Mutex<Endpoints>>,
|
||||
}
|
||||
```
|
||||
|
||||
### Responding
|
||||
|
||||
```rust
|
||||
// Success
|
||||
request.respond(Ok("result".into())).await?;
|
||||
|
||||
// Success with headers
|
||||
request.respond_with_headers(Ok("result".into()), headers).await?;
|
||||
|
||||
// Error
|
||||
request.respond(Err(service::error::Error {
|
||||
code: 500,
|
||||
status: "internal error".to_string(),
|
||||
})).await?;
|
||||
```
|
||||
|
||||
Error responses always include `Nats-Service-Error` and `Nats-Service-Error-Code` headers. If user-supplied headers contain these headers, they are overridden by the error values.
|
||||
|
||||
### Stats Tracking
|
||||
|
||||
Each response updates endpoint statistics:
|
||||
- `requests` — total requests
|
||||
- `processing_time` — cumulative processing time
|
||||
- `average_processing_time` — average per request
|
||||
- `errors` — error count
|
||||
- `last_error` — last error details
|
||||
|
||||
## Service Info Types
|
||||
|
||||
### PingResponse
|
||||
|
||||
```rust
|
||||
pub struct PingResponse {
|
||||
pub kind: String, // "io.nats.micro.v1.ping_response"
|
||||
pub name: String,
|
||||
pub id: String,
|
||||
pub version: String,
|
||||
pub metadata: HashMap<String, String>,
|
||||
}
|
||||
```
|
||||
|
||||
### Info
|
||||
|
||||
```rust
|
||||
pub struct Info {
|
||||
pub kind: String, // "io.nats.micro.v1.info_response"
|
||||
pub name: String,
|
||||
pub id: String,
|
||||
pub description: String,
|
||||
pub version: String,
|
||||
pub metadata: HashMap<String, String>,
|
||||
pub endpoints: Vec<endpoint::Info>,
|
||||
}
|
||||
```
|
||||
|
||||
### Stats
|
||||
|
||||
```rust
|
||||
pub struct Stats {
|
||||
pub kind: String, // "io.nats.micro.v1.stats_response"
|
||||
pub name: String,
|
||||
pub id: String,
|
||||
pub version: String,
|
||||
pub started: DateTime,
|
||||
pub endpoints: Vec<endpoint::Stats>,
|
||||
}
|
||||
```
|
||||
|
||||
### Endpoint Stats
|
||||
|
||||
```rust
|
||||
pub struct endpoint::Stats {
|
||||
pub name: String,
|
||||
pub subject: String,
|
||||
pub queue_group: String,
|
||||
pub data: Option<serde_json::Value>, // Custom data from stats_handler
|
||||
pub errors: u64,
|
||||
pub processing_time: Duration,
|
||||
pub average_processing_time: Duration,
|
||||
pub requests: u64,
|
||||
pub last_error: Option<error::Error>,
|
||||
}
|
||||
```
|
||||
|
||||
## Service Groups
|
||||
|
||||
Groups provide subject prefixing for endpoint organization:
|
||||
|
||||
```rust
|
||||
let service = client.service_builder().start("api", "1.0.0").await?;
|
||||
|
||||
// Endpoints subscribe to "products" and "orders"
|
||||
let products = service.endpoint("products").await?;
|
||||
let orders = service.endpoint("orders").await?;
|
||||
|
||||
// Grouped: subscribe to "v1.products" and "v1.orders"
|
||||
let v1 = service.group("v1");
|
||||
let products = v1.endpoint("products").await?;
|
||||
let orders = v1.endpoint("orders").await?;
|
||||
|
||||
// Nested: subscribe to "api.v1.products"
|
||||
let api_v1 = service.group("api").group("v1");
|
||||
let products = api_v1.endpoint("products").await?;
|
||||
```
|
||||
|
||||
Each group can have its own queue group:
|
||||
|
||||
```rust
|
||||
let v1 = service.group_with_queue_group("v1", "v1-workers");
|
||||
```
|
||||
|
||||
## Stopping a Service
|
||||
|
||||
```rust
|
||||
service.stop().await?;
|
||||
```
|
||||
|
||||
Sends a shutdown signal and aborts the verb-handling task. Other service instances with the same name continue running.
|
||||
|
||||
## Resetting Stats
|
||||
|
||||
```rust
|
||||
service.reset().await?;
|
||||
```
|
||||
|
||||
Resets all endpoint statistics (errors, processing time, requests, average processing time) to zero.
|
||||
|
||||
## Querying Service State
|
||||
|
||||
```rust
|
||||
let stats: HashMap<String, endpoint::Stats> = service.stats().await?;
|
||||
let info: Info = service.info().await?;
|
||||
```
|
||||
@@ -0,0 +1,312 @@
|
||||
# async-nats: Quick Reference
|
||||
|
||||
## Connection
|
||||
|
||||
```rust
|
||||
// Basic connect
|
||||
let client = async_nats::connect("demo.nats.io").await?;
|
||||
|
||||
// With options
|
||||
let client = async_nats::ConnectOptions::new()
|
||||
.require_tls(true)
|
||||
.name("my-service")
|
||||
.ping_interval(Duration::from_secs(10))
|
||||
.request_timeout(Some(Duration::from_secs(5)))
|
||||
.connect("demo.nats.io")
|
||||
.await?;
|
||||
|
||||
// Multiple servers
|
||||
let client = async_nats::connect(vec![
|
||||
"nats://server1:4222".parse()?,
|
||||
"nats://server2:4222".parse()?,
|
||||
]).await?;
|
||||
|
||||
// Background connect
|
||||
let client = async_nats::ConnectOptions::new()
|
||||
.retry_on_initial_connect()
|
||||
.connect("demo.nats.io")
|
||||
.await?;
|
||||
```
|
||||
|
||||
## Core NATS: Publish
|
||||
|
||||
```rust
|
||||
// Simple publish
|
||||
client.publish("subject", "payload".into()).await?;
|
||||
|
||||
// With reply-to
|
||||
client.publish_with_reply("subject", "reply-to", "payload".into()).await?;
|
||||
|
||||
// With headers
|
||||
let mut headers = HeaderMap::new();
|
||||
headers.insert("X-Custom", "value");
|
||||
client.publish_with_headers("subject", headers, "payload".into()).await?;
|
||||
|
||||
// Full control
|
||||
client.publish_with_reply_and_headers("subject", "reply-to", headers, "payload".into()).await?;
|
||||
|
||||
// Flush (ensure all published messages are sent)
|
||||
client.flush().await?;
|
||||
```
|
||||
|
||||
## Core NATS: Subscribe
|
||||
|
||||
```rust
|
||||
use futures_util::StreamExt;
|
||||
|
||||
// Basic subscribe
|
||||
let mut subscriber = client.subscribe("subject").await?;
|
||||
|
||||
// Queue group
|
||||
let mut subscriber = client.queue_subscribe("subject", "group".into()).await?;
|
||||
|
||||
// Receive messages (Subscriber implements Stream)
|
||||
while let Some(message) = subscriber.next().await {
|
||||
println!("subject: {}, payload: {:?}", message.subject, message.payload);
|
||||
}
|
||||
|
||||
// Unsubscribe
|
||||
subscriber.unsubscribe().await?;
|
||||
|
||||
// Unsubscribe after N messages
|
||||
subscriber.unsubscribe_after(10).await?;
|
||||
|
||||
// Drain (wait for in-flight, then unsubscribe)
|
||||
subscriber.drain().await?;
|
||||
```
|
||||
|
||||
## Core NATS: Request/Reply
|
||||
|
||||
```rust
|
||||
// Simple request (uses default timeout)
|
||||
let response = client.request("subject", "data".into()).await?;
|
||||
|
||||
// With custom timeout and headers
|
||||
let request = async_nats::Request::new()
|
||||
.payload("data".into())
|
||||
.timeout(Some(Duration::from_secs(5)))
|
||||
.headers(headers);
|
||||
let response = client.send_request("subject", request).await?;
|
||||
|
||||
// Custom inbox (bypasses multiplexer)
|
||||
let request = async_nats::Request::new()
|
||||
.payload("data".into())
|
||||
.inbox("custom-inbox".into());
|
||||
let response = client.send_request("subject", request).await?;
|
||||
```
|
||||
|
||||
## Message Structure
|
||||
|
||||
```rust
|
||||
pub struct Message {
|
||||
pub subject: Subject,
|
||||
pub reply: Option<Subject>,
|
||||
pub payload: Bytes,
|
||||
pub headers: Option<HeaderMap>,
|
||||
pub status: Option<StatusCode>,
|
||||
pub description: Option<String>,
|
||||
pub length: usize,
|
||||
}
|
||||
```
|
||||
|
||||
## JetStream
|
||||
|
||||
```rust
|
||||
let jetstream = async_nats::jetstream::new(client);
|
||||
|
||||
// Publish (returns ack future)
|
||||
let ack = jetstream.publish("events", "data".into()).await?;
|
||||
let publish_ack = ack.await?;
|
||||
|
||||
// Stream management
|
||||
let stream = jetstream.create_stream(stream::Config {
|
||||
name: "events".to_string(),
|
||||
subjects: vec!["events.>".to_string()],
|
||||
max_messages: 10_000,
|
||||
..Default::default()
|
||||
}).await?;
|
||||
|
||||
let stream = jetstream.get_stream("events").await?;
|
||||
let stream = jetstream.get_or_create_stream(config).await?;
|
||||
jetstream.delete_stream("events").await?;
|
||||
jetstream.update_stream(config).await?;
|
||||
|
||||
// Consumer management
|
||||
let consumer: PullConsumer = stream.create_consumer(pull::Config {
|
||||
durable_name: Some("my-consumer".to_string()),
|
||||
..Default::default()
|
||||
}).await?;
|
||||
|
||||
// Pull consumer: fetch messages
|
||||
let mut messages = consumer.messages().await?;
|
||||
while let Some(message) = messages.next().await {
|
||||
let message = message?;
|
||||
message.ack().await?;
|
||||
}
|
||||
|
||||
// Push consumer (ordered)
|
||||
let consumer = stream.create_consumer(push::OrderedConfig {
|
||||
deliver_subject: client.new_inbox(),
|
||||
filter_subject: "events.>".to_string(),
|
||||
..Default::default()
|
||||
}).await?;
|
||||
let mut messages = consumer.messages().await?;
|
||||
```
|
||||
|
||||
## Key-Value Store
|
||||
|
||||
```rust
|
||||
let kv = jetstream.create_key_value(kv::Config {
|
||||
bucket: "my-bucket".to_string(),
|
||||
history: 10,
|
||||
..Default::default()
|
||||
}).await?;
|
||||
|
||||
// CRUD
|
||||
let revision = kv.put("key", "value".into()).await?;
|
||||
let revision = kv.create("key", "value".into()).await?;
|
||||
let value: Option<Bytes> = kv.get("key").await?;
|
||||
let entry: Option<Entry> = kv.entry("key").await?;
|
||||
let revision = kv.update("key", "new-value".into(), revision).await?;
|
||||
kv.delete("key").await?;
|
||||
kv.purge("key").await?;
|
||||
|
||||
// Watch
|
||||
let mut watch = kv.watch("key").await?;
|
||||
let mut watch_all = kv.watch_all().await?;
|
||||
|
||||
// History & Keys
|
||||
let mut history = kv.history("key").await?;
|
||||
let mut keys = kv.keys().await?;
|
||||
```
|
||||
|
||||
## Object Store
|
||||
|
||||
```rust
|
||||
let bucket = jetstream.create_object_store(object_store::Config {
|
||||
bucket: "files".to_string(),
|
||||
..Default::default()
|
||||
}).await?;
|
||||
|
||||
// Put (from any AsyncRead)
|
||||
let info = bucket.put("file.txt", &mut file).await?;
|
||||
|
||||
// Get (returns AsyncRead)
|
||||
let mut object = bucket.get("file.txt").await?;
|
||||
let mut bytes = Vec::new();
|
||||
object.read_to_end(&mut bytes).await?;
|
||||
|
||||
// Info, delete, list, watch
|
||||
let info = bucket.info("file.txt").await?;
|
||||
bucket.delete("file.txt").await?;
|
||||
let mut list = bucket.list().await?;
|
||||
let mut watch = bucket.watch().await?;
|
||||
```
|
||||
|
||||
## Service API
|
||||
|
||||
```rust
|
||||
use async_nats::service::ServiceExt;
|
||||
use futures_util::StreamExt;
|
||||
|
||||
let mut service = client
|
||||
.service_builder()
|
||||
.description("product service")
|
||||
.start("products", "1.0.0")
|
||||
.await?;
|
||||
|
||||
let mut endpoint = service.endpoint("get").await?;
|
||||
|
||||
while let Some(request) = endpoint.next().await {
|
||||
request.respond(Ok("result".into())).await?;
|
||||
}
|
||||
```
|
||||
|
||||
## Client State & Events
|
||||
|
||||
```rust
|
||||
// Check connection state
|
||||
match client.connection_state() {
|
||||
State::Connected => {},
|
||||
State::Disconnected => {},
|
||||
State::Pending => {},
|
||||
}
|
||||
|
||||
// Get server info
|
||||
let info: ServerInfo = client.server_info();
|
||||
println!("max_payload: {}", info.max_payload);
|
||||
println!("jetstream: {}", info.jetstream);
|
||||
|
||||
// Get statistics
|
||||
let stats = client.statistics();
|
||||
println!("in_messages: {}", stats.in_messages.load(Ordering::Relaxed));
|
||||
|
||||
// Force reconnect
|
||||
client.force_reconnect().await?;
|
||||
|
||||
// Server pool management
|
||||
client.set_server_pool(["nats://s1:4222".parse()?, "nats://s2:4222".parse()?].as_slice()).await?;
|
||||
let pool = client.server_pool().await?;
|
||||
|
||||
// Drain
|
||||
client.drain().await?;
|
||||
```
|
||||
|
||||
## Error Handling Patterns
|
||||
|
||||
```rust
|
||||
// Connect errors
|
||||
match async_nats::connect("server").await {
|
||||
Err(e) => match e.kind() {
|
||||
ConnectErrorKind::TimedOut => {},
|
||||
ConnectErrorKind::Authentication => {},
|
||||
ConnectErrorKind::AuthorizationViolation => {},
|
||||
_ => {},
|
||||
},
|
||||
Ok(client) => {},
|
||||
}
|
||||
|
||||
// Publish errors
|
||||
match client.publish("subject", "data".into()).await {
|
||||
Err(e) => match e.kind() {
|
||||
PublishErrorKind::MaxPayloadExceeded => {},
|
||||
PublishErrorKind::InvalidSubject => {},
|
||||
PublishErrorKind::Send => {},
|
||||
_ => {},
|
||||
},
|
||||
_ => {},
|
||||
}
|
||||
|
||||
// Request errors
|
||||
match client.request("subject", "data".into()).await {
|
||||
Err(e) => match e.kind() {
|
||||
RequestErrorKind::TimedOut => {},
|
||||
RequestErrorKind::NoResponders => {},
|
||||
RequestErrorKind::InvalidSubject => {},
|
||||
RequestErrorKind::MaxPayloadExceeded => {},
|
||||
_ => {},
|
||||
},
|
||||
Ok(message) => {},
|
||||
}
|
||||
```
|
||||
|
||||
## Feature Flag Quick Reference
|
||||
|
||||
| Feature | Enables | Default |
|
||||
|---------|---------|---------|
|
||||
| `jetstream` | JetStream streams, consumers, publish | ✅ |
|
||||
| `kv` | Key-Value store (implies `jetstream`) | ✅ |
|
||||
| `object-store` | Object store (implies `jetstream` + `crypto`) | ✅ |
|
||||
| `service` | Service API | ✅ |
|
||||
| `nkeys` | NKey/JWT authentication | ✅ |
|
||||
| `nuid` | NUID-based ID generation | ✅ |
|
||||
| `crypto` | SHA-256 (for object store) | ✅ |
|
||||
| `websockets` | WebSocket transport | ✅ |
|
||||
| `ring` | `ring` TLS crypto backend | ✅ |
|
||||
| `aws-lc-rs` | `aws-lc-rs` TLS crypto backend | ❌ |
|
||||
| `fips` | FIPS mode via `aws-lc-rs` | ❌ |
|
||||
| `chrono` | `chrono` datetime instead of `time` | ❌ |
|
||||
| `server_2_10` | Server 2.10+ features | ✅ |
|
||||
| `server_2_11` | Server 2.11+ features | ✅ |
|
||||
| `server_2_12` | Server 2.12+ features | ✅ |
|
||||
| `server_2_14` | Server 2.14+ features | ✅ |
|
||||
23
docs/research/references/nats.rs/nats-async/README.md
Normal file
23
docs/research/references/nats.rs/nats-async/README.md
Normal file
@@ -0,0 +1,23 @@
|
||||
# async-nats Reference Documentation
|
||||
|
||||
**Crate**: `async-nats` v0.49.1
|
||||
**Source**: https://github.com/nats-io/nats.rs (`async-nats/` directory)
|
||||
**License**: Apache-2.0
|
||||
|
||||
## Contents
|
||||
|
||||
| # | File | Topic |
|
||||
|---|------|-------|
|
||||
| 01 | [Overview & Architecture](01-overview-and-architecture.md) | Crate overview, feature flags, source structure, core connection model, dependency graph |
|
||||
| 02 | [Key Types & Traits](02-key-types-and-traits.md) | `Client`, `Subscriber`, `Message`, `Request`, `ServerInfo`, `ConnectInfo`, `Statistics`, subject/header types, event/state types, error types, trait definitions |
|
||||
| 03 | [Protocol & Wire Format](03-protocol-and-wire-format.md) | NATS wire protocol (PUB/HPUB/SUB/UNSUB/PING/PONG, MSG/HMSG/INFO/ERR), parser/serializer internals, vectored I/O, WebSocket transport, connection lifecycle, reconnection |
|
||||
| 04 | [Connection Management](04-connection-management.md) | `ConnectOptions` builder, authentication methods, TLS configuration, reconnection callbacks, event callbacks, `ConnectionHandler` internals, multiplexer, server pool management |
|
||||
| 05 | [JetStream](05-jetstream.md) | `Context` and `ContextBuilder`, streams, consumers (pull/push/ordered), JetStream messages and acks, publish with ack futures, pagination |
|
||||
| 06 | [Key-Value Store](06-key-value-store.md) | KV `Store` handle, bucket CRUD, put/get/create/update/delete/purge, watch/history/keys, entry operations, mirrored buckets, KV-to-stream mapping |
|
||||
| 07 | [Object Store](07-object-store.md) | `ObjectStore` handle, put/get/delete/watch/list/seal, links, `Object` (AsyncRead), chunking, SHA-256 integrity, subject naming |
|
||||
| 08 | [Service API](08-service-api.md) | `Service` and `ServiceBuilder`, endpoints, groups, verb subscriptions (PING/INFO/STATS), request/respond with stats tracking |
|
||||
| 09 | [Quick Reference](09-quick-reference.md) | Code examples for all major operations, feature flag reference |
|
||||
|
||||
## How This Documentation Was Produced
|
||||
|
||||
All information was derived by reading the source code of the `async-nats` crate at version 0.49.1 from the `nats.rs` repository. No external documentation was consulted — this is a ground-up reference based purely on the source.
|
||||
Reference in New Issue
Block a user