alknet/docs/research/references/nats.rs/nats-server/02-protocol-and-wire-format.md

NATS Client Protocol and Wire Format

Protocol: NATS Client Protocol v1 (with dynamic reconfiguration)
Transport: TCP (port 4222), TLS, WebSocket (ws/wss)

Protocol Overview

The NATS client-server protocol is a simple, text-based protocol with binary payload support. All operations are terminated with \r\n. Messages carry their payload length, allowing efficient binary data transfer.

Connection Lifecycle

Client                                         Server
  │                                               │
  │◄──────────── INFO {json} ────────────────────│  Server sends INFO first
  │                                               │
  │────────────── CONNECT {json} ────────────────►│  Client sends CONNECT
  │────────────── PING ──────────────────────────►│  Client sends PING
  │◄──────────── PONG ──────────────────────────  │  Server confirms connection
  │                                               │
  │──── SUB/UNSUB/PUB/HPUB ──────────────────────►│  Normal operation
  │◄─── MSG/HMSG/+OK/-ERR/PING ─────────────────│
  │                                               │

Server Operations (ServerOp)

These are operations received from the server. The Connection module parses these from the read buffer.

INFO

Sent by the server upon connection and asynchronously when cluster topology changes.

INFO {json}\r\n

JSON fields (see ServerInfo struct):

Field	Type	Description
server_id	String	Unique server identifier
server_name	String	Generated server name
host	String	Cluster host
port	u16	Cluster port
version	String	Server version
auth_required	bool	Authentication required
tls_required	bool	TLS required
max_payload	usize	Maximum payload size
proto	i8	Protocol version (0 or 1)
client_id	u64	Server-assigned client ID
go	String	Go build version
nonce	String	Nonce for nkey auth
connect_urls	Vec	Cluster server URLs
client_ip	String	Client IP as seen by server
headers	bool	Server supports headers
ldm	bool	Lame duck mode
cluster	Option	Cluster name
domain	Option	NATS domain
jetstream	bool	JetStream enabled

MSG

Delivers a message to a subscription (no headers):

MSG <subject> <sid> [reply-to] <#bytes>\r\n
<payload>\r\n

HMSG

Delivers a message with headers:

HMSG <subject> <sid> [reply-to] <#header-bytes> <#total-bytes>\r\n
<NATS/1.0 [status] [description]>\r\n
<header-name>: <header-value>\r\n
\r\n
<payload>\r\n

Header format follows the NATS/1.0 header spec:

• First line: NATS/1.0 optionally followed by status code and description
• Subsequent lines: name: value headers
• Empty line separates headers from payload
• Header values may span multiple lines (continuation lines start with whitespace)

PING / PONG

PING\r\n    →  Client responds with PONG
PONG\r\n    →  Acknowledges client's PING

+OK / -ERR

+OK\r\n                 →  Success acknowledgment (verbose mode)
-ERR <description>\r\n  →  Error from server

Common server errors:

• authorization violation → parsed as ServerError::AuthorizationViolation
• Other strings → ServerError::Other(String)

Client Operations (ClientOp)

These are operations sent from the client to the server. The Connection module serializes these to the write buffer.

CONNECT

Sent as the first client operation after receiving INFO. Contains authentication and capability information.

CONNECT {json}\r\n

JSON fields (see ConnectInfo struct):

Field	Type	Description
verbose	bool	Enable +OK acknowledgments (always false in this client)
pedantic	bool	Strict format checking (always false)
jwt	Option	User JWT for auth
nkey	Option	Public nkey for auth
sig	Option	Signed nonce (Base64URL encoded)
name	Option	Client name
echo	bool	Whether server should echo messages back
lang	String	Implementation language ("rust")
version	String	Client version
protocol	u8	Protocol version (1 = dynamic)
tls_required	bool	TLS required
user	Option	Username
pass	Option	Password
auth_token	Option	Auth token
headers	bool	Client supports headers (always true)
no_responders	bool	Client supports no-responders (always true)

PUB / HPUB

Publish a message:

PUB <subject> [reply-to] <#payload-bytes>\r\n
<payload>\r\n

Publish with headers:

HPUB <subject> [reply-to] <#header-bytes> <#total-bytes>\r\n
<NATS/1.0\r\n
<header-name>: <header-value>\r\n
\r\n
<payload>\r\n

SUB

Subscribe to a subject:

SUB <subject> [queue-group] <sid>\r\n

UNSUB

Unsubscribe from a subscription:

UNSUB <sid> [max]\r\n

The optional max parameter tells the server to auto-unsubscribe after receiving the specified number of messages.

PING / PONG

PING\r\n    →  Health check / keepalive
PONG\r\n    →  Response to server PING

Protocol Version

The Protocol enum has two variants:

Value	Name	Description
0	Original	Basic protocol
1	Dynamic	Supports async INFO for cluster topology changes, lame duck mode

This client always sends protocol: 1 (Dynamic), enabling:

• Asynchronous INFO messages with updated server lists
• Lame duck mode notifications
• Dynamic reconfiguration of cluster topology

Wire Format Details

Message Length Calculation

For plain MSG:

length = subject.len() + reply.map_or(0, |r| r.len()) + payload.len()

For HMSG:

length = subject.len() + reply.map_or(0, |r| r.len()) + header_len + payload.len()

Where header_len = serialized header bytes and total_len = header_len + payload.len().

Write Buffer Architecture

The Connection uses a two-tier write buffer:

1. flattened_writes (BytesMut) — for small writes (< 4096 bytes). Protocol headers, short commands, and small messages are flattened into this buffer for efficient sequential writing.

2. write_buf (VecDeque<Bytes>) — for large writes (>= 4096 bytes). Large payloads are appended as separate Bytes chunks. Supports vectored writes (write_vectored) when the underlying stream supports it, writing up to 64 chunks at once.

The soft limit for the total write buffer is 65,535 bytes (SOFT_WRITE_BUF_LIMIT). When exceeded, the ConnectionHandler stops processing new commands until the buffer drains.

Read Buffer Architecture

The Connection uses a single BytesMut read buffer with configurable initial capacity (default 65,535 bytes). Protocol parsing uses memchr::memmem::find to locate CRLF delimiters efficiently. If a partial message is in the buffer, the parser returns None and waits for more data.

Header Serialization

Headers are serialized in NATS/1.0 format:

NATS/1.0\r\n
Header-Name: Header-Value\r\n
Multi-Line-Header: value part 1\r\n
 continuation of value\r\n
Another-Header: another value\r\n
\r\n

The HeaderMap::to_bytes() method handles this serialization, using httparse-compatible line folding for multi-line values.

Status Codes in Headers

NATS status codes are embedded in the HMSG header version line:

NATS/1.0 404 No Messages\r\n
NATS/1.0 408 Request Timeout\r\n
NATS/1.0 503 No Responders\r\n

Common codes used by the client:

Code	Constant	Meaning
100	IDLE_HEARTBEAT	JetStream idle heartbeat
200	OK	Success
404	NOT_FOUND	Message/stream not found
408	TIMEOUT	Request timeout
409	REQUEST_TERMINATED	Request terminated
503	NO_RESPONDERS	No responders available

Protocol Parsing Implementation

The Connection::try_read_op() method handles all protocol parsing:

1. Search for \r\n delimiter using memchr::memmem::find
2. Match the operation prefix:
   • +OK → ServerOp::Ok
   • PING → ServerOp::Ping
   • PONG → ServerOp::Pong
   • -ERR → parse error description → ServerOp::Error
   • INFO → parse JSON → ServerOp::Info
   • MSG → parse subject/sid/reply/length, read payload → ServerOp::Message
   • HMSG → parse headers + payload → ServerOp::Message
3. Unknown prefix → return io::Error with InvalidInput

For MSG and HMSG, if the complete payload isn't yet in the read buffer (checked via len + payload_len + 4 > remaining), the method returns Ok(None) and the buffer accumulates more data before retrying.

Non-UTF8 subjects in server messages are handled gracefully — the parser returns an io::Error rather than panicking, which is critical because the Go server does not enforce UTF-8 in subjects (regression fix for issue #1572).