diff --git a/docs/research/references/ssh/russh/01-overview-architecture.md b/docs/research/references/ssh/russh/01-overview-architecture.md new file mode 100644 index 0000000..c4c6cbe --- /dev/null +++ b/docs/research/references/ssh/russh/01-overview-architecture.md @@ -0,0 +1,153 @@ +# Russh: Overview & Architecture + +**Version**: 0.60.2 +**Repository**: https://github.com/warp-tech/russh +**License**: Apache-2.0 +**Rust Edition**: 2024 +**MSRV**: 1.85 +**Origin**: Fork of [Thrussh](https://nest.pijul.com/pijul/thrussh) by Pierre-Étienne Meunier + +## What is Russh? + +Russh is a **low-level, asynchronous SSH2 client and server implementation** for Rust, built on Tokio and Futures. It provides a complete SSH-2 protocol stack — key exchange, authentication, channel multiplexing, port forwarding, and subsystems — exposed through handler traits that users implement for their specific needs. + +Core characteristics: +- **Async-native** — built entirely on Tokio and Futures; no blocking I/O +- **Handler-driven** — both client and server are used by implementing trait handlers (`client::Handler`, `server::Handler`) +- **Both client and server** — a single crate supports both sides of the SSH connection +- **Streaming I/O** — channels implement `AsyncRead`/`AsyncWrite` for ergonomic integration +- **Safety-conscious** — `deny(clippy::unwrap_used)`, `deny(clippy::expect_used)`, `deny(clippy::panic)`, `deny(clippy::indexing_slicing)` by default; sensitive data uses `CryptoVec` with `mlock` +- **Two crypto backends** — `aws-lc-rs` (default) or `ring`, at least one required + +## Workspace Structure + +``` +russh/ # Main SSH library crate +├── russh-util/ # Runtime abstraction utilities (WASM support, time, spawn) +├── russh-config/ # SSH config file parser + ProxyCommand support +├── cryptovec/ # Zeroing-on-drop vector with mlock (CryptoVec) +└── pageant/ # Windows Pageant SSH agent transport client +``` + +### Dependency Graph + +``` +russh depends on: + ├── russh-cryptovec (CryptoVec: zeroing vector with mlock, ssh-encoding support) + ├── russh-util (runtime abstraction: tokio spawn, time, WASM compat) + ├── ssh-key (internal-russh-forked-ssh-key: key types, parsing, certificates) + ├── ssh-encoding (SSH wire format encode/decode) + ├── tokio (async runtime, IO, sync, time) + ├── futures (future combinators) + ├── curve25519-dalek (Curve25519 DH) + ├── ed25519-dalek (Ed25519 signing) + ├── p256/p384/p521 (NIST ECDSA curves + ECDH) + ├── aes (AES cipher implementations) + ├── flate2 (zlib compression, optional) + ├── rsa (RSA signing, optional feature) + └── aws-lc-rs / ring (crypto backend for GCM etc.) + +russh-config depends on: + ├── tokio (async IO, process for ProxyCommand) + ├── futures (future utilities) + └── globset / whoami (config matching, username detection) + +cryptovec depends on: + ├── nix (Unix) (mlock/munlock via mmap) + ├── windows-sys (Win) (VirtualLock/VirtualUnlock) + └── ssh-encoding (optional, for Encode support) +``` + +## Architecture Overview + +Russh implements the SSH-2 protocol as a **state machine** driven by an event loop. The core design separates protocol handling from application logic via handler traits. + +``` +┌─────────────────────────────────────────────────────────┐ +│ Application Layer │ +│ ┌──────────────────┐ ┌──────────────────┐ │ +│ │ client::Handler │ │ server::Handler │ │ +│ │ (user implements) │ │ (user implements) │ │ +│ └────────┬─────────┘ └────────┬─────────┘ │ +├───────────┼──────────────────────────┼───────────────────┤ +│ │ Russh Library │ │ +│ ┌────────▼─────────────────────────▼─────────┐ │ +│ │ Event Loop (tokio::select!) │ │ +│ │ ┌──────────┐ ┌─────────┐ ┌──────────────┐ │ │ +│ │ │ Reading │ │ Writing │ │ Handler │ │ │ +│ │ │ packets │ │ packets │ │ dispatch │ │ │ +│ │ └────┬─────┘ └────┬────┘ └──────┬───────┘ │ │ +│ │ │ │ │ │ │ +│ │ ┌────▼────────────▼─────────────▼───────┐ │ │ +│ │ │ Session State Machine │ │ │ +│ │ │ KEX → Auth → Channels → Forwarding │ │ │ +│ │ └──────────────────────────────────────┘ │ │ +│ └─────────────────────────────────────────────┘ │ +├──────────────────────────────────────────────────────────┤ +│ Transport Layer │ +│ ┌────────────────┐ ┌────────────┐ ┌────────────┐ │ +│ │ Cipher (enc) │ │ MAC (auth) │ │ Compression │ │ +│ └────────────────┘ └────────────┘ └────────────┘ │ +│ ┌────────────────┐ ┌────────────────────────────┐ │ +│ │ PacketWriter │ │ SSHBuffer / SshRead │ │ +│ └────────────────┘ └────────────────────────────┘ │ +├──────────────────────────────────────────────────────────┤ +│ TCP Stream (tokio::net) │ +└──────────────────────────────────────────────────────────┘ +``` + +### Key Design Principle: Buffered Writes + +From the library documentation: + +> It might seem a little odd that the read/write methods for server or client sessions often return neither `Result` nor `Future`. This is because the data sent to the remote side is **buffered**, because it needs to be encrypted first, and encryption works on buffers, and for many algorithms, not in place. + +The event loop works as follows: +1. Wait for incoming packets +2. React by calling the provided `Handler`, which fills some buffers +3. If buffers are non-empty, send them to the socket, flush, empty the buffers +4. For servers, unsolicited messages sent through a `server::Handle` are processed when there is no incoming packet + +## Module Map + +| Module | Purpose | +|--------|---------| +| `client/` | Client-side SSH: `Handler` trait, `Session`, `Handle`, `Config`, kex, encrypted state | +| `server/` | Server-side SSH: `Handler` trait, `Server` trait, `Session`, `Handle`, `Config` | +| `kex/` | Key exchange algorithms: Curve25519, DH groups, ECDH-NIST, ML-KEM hybrid | +| `cipher/` | Encryption: Chacha20-Poly1305, AES-GCM, AES-CTR, AES-CBC, 3DES-CBC | +| `mac/` | Message authentication: HMAC-SHA2, ETM variants | +| `keys/` | Key loading, parsing, SSH agent protocol, known hosts | +| `channels/` | Channel abstraction, `Channel`/`ChannelMsg`, `AsyncRead`/`AsyncWrite` streams | +| `negotiation` | Algorithm negotiation: `Preferred`, `Names`, `write_kex`, `read_kex` | +| `compression` | zlib and zlib@openssh.com compression | +| `session` | `CommonSession`, `Encrypted` state, `NewKeys`, `Exchange` | +| `sshbuffer` | `SSHBuffer`, `PacketWriter`, `SshId` | +| `auth` | Auth methods: `Method`, `MethodSet`, `Signer`, `AuthResult` | +| `cert` | OpenSSH certificate handling | +| `pty` | PTY terminal modes | +| `msg` | SSH message type constants (RFC 4250/4253/4254) | +| `parsing` | Wire format parsing helpers | + +## Crypto Backend Selection + +At least one crypto backend feature must be enabled, or compilation fails: + +```rust +#[cfg(not(any(feature = "ring", feature = "aws-lc-rs")))] +compile_error!( + "`russh` requires enabling either the `ring` or `aws-lc-rs` feature as a crypto backend." +); +``` + +- **`aws-lc-rs`** (default): Used for AES-GCM via the `aws-lc-rs` crate's AEAD interface +- **`ring`**: Alternative backend for AES-GCM via the `ring` crate's AEAD interface + +Other features: +- **`rsa`** (default): Enables RSA key support +- **`des`**: Enables insecure 3DES-CBC cipher +- **`dsa`**: Enables insecure DSA key support +- **`flate2`** (default): Enables zlib compression +- **`async-trait`**: Enables `#[async_trait]` attribute on handler traits +- **`legacy-ed25519-pkcs8-parser`**: Enables ASN1-based Ed25519 PKCS#8 parsing +- **`serde`**: Enables serde support for ssh-key types \ No newline at end of file diff --git a/docs/research/references/ssh/russh/02-key-types-traits.md b/docs/research/references/ssh/russh/02-key-types-traits.md new file mode 100644 index 0000000..9e21079 --- /dev/null +++ b/docs/research/references/ssh/russh/02-key-types-traits.md @@ -0,0 +1,410 @@ +# Russh: Key Types and Traits + +## Client API + +### `client::Handler` Trait + +The primary interface for implementing SSH clients. All methods have default implementations (mostly no-ops returning `Ok(())`), except `check_server_key` which defaults to rejecting all keys. + +```rust +pub trait Handler: Sized + Send { + type Error: From + Send + core::fmt::Debug; + + // --- Must implement (security critical) --- + fn check_server_key( + &mut self, + server_public_key: &ssh_key::PublicKey, + ) -> impl Future> + Send; + // Default: async { Ok(false) } — REJECTS ALL KEYS + + // --- Optional callbacks --- + fn auth_banner(&mut self, banner: &str, session: &mut Session) -> ...; + fn kex_done(&mut self, shared_secret: Option<&[u8]>, names: &Names, session: &mut Session) -> ...; + fn channel_open_confirmation(&mut self, id: ChannelId, max_packet_size: u32, window_size: u32, session: &mut Session) -> ...; + fn channel_success(&mut self, channel: ChannelId, session: &mut Session) -> ...; + fn channel_failure(&mut self, channel: ChannelId, session: &mut Session) -> ...; + fn channel_close(&mut self, channel: ChannelId, session: &mut Session) -> ...; + fn channel_eof(&mut self, channel: ChannelId, session: &mut Session) -> ...; + fn channel_open_failure(&mut self, channel: ChannelId, reason: ChannelOpenFailure, description: &str, language: &str, session: &mut Session) -> ...; + fn data(&mut self, channel: ChannelId, data: &[u8], session: &mut Session) -> ...; + fn extended_data(&mut self, channel: ChannelId, ext: u32, data: &[u8], session: &mut Session) -> ...; + fn exit_status(&mut self, channel: ChannelId, exit_status: u32, session: &mut Session) -> ...; + fn exit_signal(&mut self, channel: ChannelId, signal_name: Sig, core_dumped: bool, error_message: &str, lang_tag: &str, session: &mut Session) -> ...; + fn window_adjusted(&mut self, channel: ChannelId, new_size: u32, session: &mut Session) -> ...; + fn adjust_window(&mut self, channel: ChannelId, window: u32) -> u32; + + // Server-initiated channels (port forwarding, agent, X11, etc.) + fn server_channel_open_forwarded_tcpip(&mut self, channel: Channel, ...) -> ...; + fn server_channel_open_forwarded_streamlocal(&mut self, channel: Channel, ...) -> ...; + fn server_channel_open_agent_forward(&mut self, channel: Channel, ...) -> ...; + fn server_channel_open_session(&mut self, channel: Channel, ...) -> ...; + fn server_channel_open_x11(&mut self, channel: Channel, ...) -> ...; + fn server_channel_open_direct_tcpip(&mut self, channel: Channel, ...) -> ...; + fn server_channel_open_direct_streamlocal(&mut self, channel: Channel, ...) -> ...; + + // OpenSSH extensions + fn openssh_ext_host_keys_announced(&mut self, keys: Vec, session: &mut Session) -> ...; + fn disconnected(&mut self, reason: DisconnectReason) -> ...; +} +``` + +### `client::Config` + +```rust +pub struct Config { + pub client_id: SshId, // SSH version string, default: "SSH-2.0-russh_0.60.2" + pub limits: Limits, // Rekey limits (1GB read/write, 1hr time) + pub window_size: u32, // Initial channel window (default: 2097152 = 2MB) + pub maximum_packet_size: u32, // Max single packet (default: 32768) + pub channel_buffer_size: usize, // Channel message buffer (default: 100) + pub preferred: Preferred, // Algorithm preferences + pub inactivity_timeout: Option, // Connection timeout (default: None) + pub keepalive_interval: Option, // Keepalive frequency (default: None) + pub keepalive_max: usize, // Max missed keepalives (default: 3) + pub anonymous: bool, // Skip authentication (default: false) + pub gex: GexParams, // DH-GEX parameters (default: 3072-8192 bits) + pub nodelay: bool, // TCP_NODELAY (default: false) +} +``` + +### `client::Handle` + +The handle returned after connecting. Used to send commands, open channels, authenticate, etc. + +```rust +impl Handle { + // Authentication + pub async fn authenticate_none(&mut self, user: U) -> Result; + pub async fn authenticate_password(&mut self, user: U, password: P) -> Result; + pub async fn authenticate_publickey(&mut self, user: U, key: PrivateKeyWithHashAlg) -> Result; + pub async fn authenticate_openssh_cert(&mut self, user: U, key: Arc, cert: Certificate) -> Result; + pub async fn authenticate_publickey_with(&mut self, user: U, key: PublicKey, hash_alg: Option, signer: &mut S) -> Result; + pub async fn authenticate_keyboard_interactive_start(&mut self, user: U, submethods: S) -> Result; + pub async fn authenticate_keyboard_interactive_respond(&mut self, responses: Vec) -> Result; + + // Channels + pub async fn channel_open_session(&self) -> Result, Error>; + pub async fn channel_open_x11(&self, originator_address: A, originator_port: u32) -> Result, Error>; + pub async fn channel_open_direct_tcpip(&self, host_to_connect: A, port_to_connect: u32, originator_address: B, originator_port: u32) -> Result, Error>; + pub async fn channel_open_direct_streamlocal(&self, socket_path: S) -> Result, Error>; + + // Port forwarding + pub async fn tcpip_forward(&self, address: A, port: u32) -> Result; + pub async fn cancel_tcpip_forward(&self, address: A, port: u32) -> Result<(), Error>; + pub async fn streamlocal_forward(&self, socket_path: A) -> Result<(), Error>; + pub async fn cancel_streamlocal_forward(&self, socket_path: A) -> Result<(), Error>; + + // Connection management + pub async fn disconnect(&self, reason: Disconnect, description: &str, language_tag: &str) -> Result<(), Error>; + pub async fn rekey_soon(&self) -> Result<(), Error>; + pub async fn send_keepalive(&self, want_reply: bool) -> Result<(), Error>; + pub async fn send_ping(&self) -> Result<(), Error>; + pub async fn no_more_sessions(&self, want_reply: bool) -> Result<(), Error>; + pub async fn best_supported_rsa_hash(&self) -> Result>, Error>; + pub fn is_closed(&self) -> bool; +} +``` + +`Handle` also implements `Future>`, so you can `.await` it to wait for the session to end. + +### `client::connect` and `client::connect_stream` + +```rust +// Connect via TCP +pub async fn connect( + config: Arc, addrs: A, handler: H, +) -> Result, H::Error>; + +// Connect via any AsyncRead+AsyncWrite stream +pub async fn connect_stream( + config: Arc, stream: R, handler: H, +) -> Result, H::Error> +where + H: Handler + Send + 'static, + R: AsyncRead + AsyncWrite + Unpin + Send + 'static; +``` + +--- + +## Server API + +### `server::Server` Trait + +Factory trait that creates a new `Handler` for each client connection: + +```rust +pub trait Server { + type Handler: Handler + Send + 'static; + fn new_client(&mut self, peer_addr: Option) -> Self::Handler; + fn handle_session_error(&mut self, _error: ::Error) {} + + // Run on a pre-bound TcpListener + fn run_on_socket(&mut self, config: Arc, socket: &TcpListener) -> RunningServer<...>; + + // Bind and run on an address + fn run_on_address(&mut self, config: Arc, addrs: A) -> impl Future<...>; +} +``` + +### `server::Handler` Trait + +Per-client handler, similar to `client::Handler` but with different callback signatures (receives `&mut Session` for sending responses): + +```rust +pub trait Handler: Sized { + type Error: From + Send; + + // Authentication callbacks + fn auth_none(&mut self, user: &str) -> impl Future> + Send; + fn auth_password(&mut self, user: &str, password: &str) -> impl Future> + Send; + fn auth_publickey_offered(&mut self, user: &str, public_key: &PublicKey) -> impl Future<...> + Send; + fn auth_publickey(&mut self, user: &str, public_key: &PublicKey) -> impl Future<...> + Send; + fn auth_openssh_certificate(&mut self, user: &str, certificate: &Certificate) -> impl Future<...> + Send; + fn auth_keyboard_interactive<'a>(&'a mut self, user: &str, submethods: &str, response: Option>) -> impl Future<...> + Send; + fn auth_succeeded(&mut self, session: &mut Session) -> impl Future<...> + Send; + fn authentication_banner(&mut self) -> impl Future, Self::Error>> + Send; + + // Channel callbacks (return bool = whether to grant the channel) + fn channel_open_session(&mut self, channel: Channel, session: &mut Session) -> impl Future> + Send; + fn channel_open_x11(&mut self, channel: Channel, originator_address: &str, originator_port: u32, session: &mut Session) -> impl Future<...> + Send; + fn channel_open_direct_tcpip(&mut self, channel: Channel, host_to_connect: &str, port_to_connect: u32, originator_address: &str, originator_port: u32, session: &mut Session) -> impl Future<...> + Send; + fn channel_open_forwarded_tcpip(&mut self, channel: Channel, ...) -> impl Future<...> + Send; + fn channel_open_direct_streamlocal(&mut self, channel: Channel, socket_path: &str, session: &mut Session) -> impl Future<...> + Send; + + // Channel events + fn data(&mut self, channel: ChannelId, data: &[u8], session: &mut Session) -> impl Future<...> + Send; + fn extended_data(&mut self, channel: ChannelId, code: u32, data: &[u8], session: &mut Session) -> impl Future<...> + Send; + fn channel_close(&mut self, channel: ChannelId, session: &mut Session) -> impl Future<...> + Send; + fn channel_eof(&mut self, channel: ChannelId, session: &mut Session) -> impl Future<...> + Send; + fn window_adjusted(&mut self, channel: ChannelId, new_size: u32, session: &mut Session) -> impl Future<...> + Send; + fn adjust_window(&mut self, channel: ChannelId, current: u32) -> u32; + + // Channel requests (use session.channel_success/failure to respond) + fn pty_request(&mut self, channel: ChannelId, term: &str, col_width: u32, row_height: u32, pix_width: u32, pix_height: u32, modes: &[(Pty, u32)], session: &mut Session) -> impl Future<...> + Send; + fn x11_request(&mut self, channel: ChannelId, ...) -> impl Future<...> + Send; + fn env_request(&mut self, channel: ChannelId, variable_name: &str, variable_value: &str, session: &mut Session) -> impl Future<...> + Send; + fn shell_request(&mut self, channel: ChannelId, session: &mut Session) -> impl Future<...> + Send; + fn exec_request(&mut self, channel: ChannelId, data: &[u8], session: &mut Session) -> impl Future<...> + Send; + fn subsystem_request(&mut self, channel: ChannelId, name: &str, session: &mut Session) -> impl Future<...> + Send; + fn window_change_request(&mut self, channel: ChannelId, ...) -> impl Future<...> + Send; + fn agent_request(&mut self, channel: ChannelId, session: &mut Session) -> impl Future<...> + Send; + fn signal(&mut self, channel: ChannelId, signal: Sig, session: &mut Session) -> impl Future<...> + Send; + + // Port forwarding + fn tcpip_forward(&mut self, address: &str, port: &mut u32, session: &mut Session) -> impl Future<...> + Send; + fn cancel_tcpip_forward(&mut self, address: &str, port: u32, session: &mut Session) -> impl Future<...> + Send; + fn streamlocal_forward(&mut self, socket_path: &str, session: &mut Session) -> impl Future<...> + Send; + fn cancel_streamlocal_forward(&mut self, socket_path: &str, session: &mut Session) -> impl Future<...> + Send; + + // DH-GEX group lookup + fn lookup_dh_gex_group(&mut self, gex_params: &GexParams) -> impl Future, Self::Error>> + Send; +} +``` + +### `server::Auth` Enum + +```rust +pub enum Auth { + Reject { proceed_with_methods: Option, partial_success: bool }, + Accept, + UnsupportedMethod, + Partial { name: Cow<'static, str>, instructions: Cow<'static, str>, prompts: Cow<'static, [(Cow<'static, str>, bool)]> }, +} +``` + +### `server::Config` + +```rust +pub struct Config { + pub server_id: SshId, // "SSH-2.0-russh_0.60.2" + pub methods: auth::MethodSet, // All methods by default + pub auth_rejection_time: Duration, // Constant-time rejection (default: 1s) + pub auth_rejection_time_initial: Option, // For "none" probe (default: None) + pub keys: Vec, // Server host keys + pub limits: Limits, + pub window_size: u32, // Default: 2097152 + pub maximum_packet_size: u32, // Default: 32768 + pub channel_buffer_size: usize, // Default: 100 + pub event_buffer_size: usize, // Default: 10 + pub preferred: Preferred, + pub max_auth_attempts: usize, // Default: 10 + pub inactivity_timeout: Option, // Default: 600s + pub keepalive_interval: Option, // Default: None + pub keepalive_max: usize, // Default: 3 + pub nodelay: bool, // Default: false +} +``` + +### `server::Handle` + +Server-side handle for sending unsolicited messages to a client: + +```rust +impl Handle { + pub async fn data(&self, id: ChannelId, data: impl Into) -> Result<(), Bytes>; + pub async fn extended_data(&self, id: ChannelId, ext: u32, data: impl Into) -> Result<(), Bytes>; + pub async fn eof(&self, id: ChannelId) -> Result<(), ()>; + pub async fn channel_success(&self, id: ChannelId) -> Result<(), ()>; + pub async fn channel_failure(&self, id: ChannelId) -> Result<(), ()>; + pub async fn close(&self, id: ChannelId) -> Result<(), ()>; + pub async fn xon_xoff_request(&self, id: ChannelId, client_can_do: bool) -> Result<(), ()>; + pub async fn exit_status_request(&self, id: ChannelId, exit_status: u32) -> Result<(), ()>; + pub async fn forward_tcpip(&self, address: String, port: u32) -> Result; + pub async fn cancel_tcpip_forward(&self, address: String, port: u32) -> Result<(), ()>; + // ... etc. +} +``` + +--- + +## Channel Types + +### `Channel` + +A bidirectional handle to an SSH channel. `Send` is the message type (`client::Msg` or `server::Msg`). + +```rust +pub struct Channel> { + pub read_half: ChannelReadHalf, + pub write_half: ChannelWriteHalf, +} + +impl + Send + Sync + 'static> Channel { + pub fn id(&self) -> ChannelId; + pub async fn writable_packet_size(&self) -> usize; + pub fn split(self) -> (ChannelReadHalf, ChannelWriteHalf); + pub async fn wait(&mut self) -> Option; + + // Client-side operations + pub async fn request_pty(&self, ...) -> Result<(), Error>; + pub async fn request_shell(&self, want_reply: bool) -> Result<(), Error>; + pub async fn exec(&self, want_reply: bool, command: A) -> Result<(), Error>; + pub async fn signal(&self, signal: Sig) -> Result<(), Error>; + pub async fn request_subsystem(&self, want_reply: bool, name: A) -> Result<(), Error>; + pub async fn request_x11(&self, ...) -> Result<(), Error>; + pub async fn set_env(&self, ...) -> Result<(), Error>; + pub async fn window_change(&self, ...) -> Result<(), Error>; + pub async fn agent_forward(&self, want_reply: bool) -> Result<(), Error>; + pub async fn data(&self, data: R) -> Result<(), Error>; + pub async fn extended_data(&self, ext: u32, data: R) -> Result<(), Error>; + pub async fn eof(&self) -> Result<(), Error>; + pub async fn exit_status(&self, exit_status: u32) -> Result<(), Error>; + pub async fn close(&self) -> Result<(), Error>; + + // Streaming + pub fn into_stream(self) -> ChannelStream; + pub fn make_reader(&mut self) -> impl AsyncRead + '_; + pub fn make_reader_ext(&mut self, ext: Option) -> impl AsyncRead + '_; + pub fn make_writer(&self) -> impl AsyncWrite + 'static; + pub fn make_writer_ext(&self, ext: Option) -> impl AsyncWrite + 'static; +} +``` + +### `ChannelMsg` Enum + +All possible messages receivable on a channel: + +```rust +pub enum ChannelMsg { + Open { id: ChannelId, max_packet_size: u32, window_size: u32 }, + Data { data: Bytes }, + ExtendedData { data: Bytes, ext: u32 }, + Eof, + Close, + RequestPty { want_reply: bool, term: String, col_width: u32, row_height: u32, pix_width: u32, pix_height: u32, terminal_modes: Vec<(Pty, u32)> }, + RequestShell { want_reply: bool }, + Exec { want_reply: bool, command: Vec }, + Signal { signal: Sig }, + RequestSubsystem { want_reply: bool, name: String }, + RequestX11 { want_reply: bool, single_connection: bool, x11_authentication_protocol: String, x11_authentication_cookie: String, x11_screen_number: u32 }, + SetEnv { want_reply: bool, variable_name: String, variable_value: String }, + WindowChange { col_width: u32, row_height: u32, pix_width: u32, pix_height: u32 }, + AgentForward { want_reply: bool }, + XonXoff { client_can_do: bool }, + ExitStatus { exit_status: u32 }, + ExitSignal { signal_name: Sig, core_dumped: bool, error_message: String, lang_tag: String }, + WindowAdjusted { new_size: u32 }, + Success, + Failure, + OpenFailure(ChannelOpenFailure), +} +``` + +### `ChannelId` + +A `u32` wrapper identifying a channel within a session: + +```rust +#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, Ord, PartialOrd)] +pub struct ChannelId(u32); +``` + +--- + +## Shared Types + +### `Limits` + +Rekey thresholds (following RFC 4253 Section 9): + +```rust +pub struct Limits { + pub rekey_write_limit: usize, // Default: 1 << 30 (1 GB) + pub rekey_read_limit: usize, // Default: 1 << 30 (1 GB) + pub rekey_time_limit: Duration, // Default: 3600s (1 hour) +} +``` + +### `Preferred` + +Algorithm preference lists for negotiation: + +```rust +pub struct Preferred { + pub kex: Cow<'static, [kex::Name]>, + pub key: Cow<'static, [Algorithm]>, + pub cipher: Cow<'static, [cipher::Name]>, + pub mac: Cow<'static, [mac::Name]>, + pub compression: Cow<'static, [compression::Name]>, +} +``` + +Default order prioritizes modern algorithms: +- **KEX**: ML-KEM-768-X25519 → Curve25519 → DH-GEX-SHA256 → DH-G18/17/16/15/14 +- **Key**: Ed25519 → ECDSA-P256/P384/P521 → RSA-SHA512/256 +- **Cipher**: Chacha20-Poly1305 → AES-256-GCM → AES-256/192/128-CTR +- **MAC**: HMAC-SHA512-ETM → HMAC-SHA256-ETM → HMAC-SHA512/256 + +### `Disconnect` Enum + +RFC 4253 Section 11.1 disconnect reason codes: + +```rust +pub enum Disconnect { + HostNotAllowedToConnect = 1, + ProtocolError = 2, + KeyExchangeFailed = 3, + MACError = 5, + CompressionError = 6, + ServiceNotAvailable = 7, + ProtocolVersionNotSupported = 8, + HostKeyNotVerifiable = 9, + ConnectionLost = 10, + ByApplication = 11, + TooManyConnections = 12, + AuthCancelledByUser = 13, + NoMoreAuthMethodsAvailable = 14, + IllegalUserName = 15, +} +``` + +### `CryptoVec` + +A vector that zeroes its memory on clears and reallocations, using `mlock` on Unix and `VirtualLock` on Windows. Used for all sensitive key material. + +```rust +// From russh-cryptovec +pub struct CryptoVec { /* ... */ } +// Implements mlock/munlock for sensitive data +// Zeroes memory on drop/resize +``` \ No newline at end of file diff --git a/docs/research/references/ssh/russh/03-ssh-protocol-implementation.md b/docs/research/references/ssh/russh/03-ssh-protocol-implementation.md new file mode 100644 index 0000000..ba18a68 --- /dev/null +++ b/docs/research/references/ssh/russh/03-ssh-protocol-implementation.md @@ -0,0 +1,298 @@ +# Russh: SSH Protocol Implementation + +This document covers how russh implements the SSH-2 protocol — key exchange, authentication, channel multiplexing, port forwarding, and extensions. + +## Protocol Message Constants (`msg` module) + +Russh defines SSH protocol message type codes as constants: + +| Constant | Value | RFC Reference | +|----------|-------|---------------| +| `DISCONNECT` | 1 | RFC 4253 §11.1 | +| `IGNORE` | 2 | RFC 4253 §11.2 | +| `UNIMPLEMENTED` | 3 | RFC 4253 §11.3 | +| `DEBUG` | 4 | RFC 4253 §11.4 | +| `SERVICE_REQUEST` | 5 | RFC 4253 §7 | +| `SERVICE_ACCEPT` | 6 | RFC 4253 §7 | +| `EXT_INFO` | 7 | RFC 8308 | +| `KEXINIT` | 20 | RFC 4253 §7.1 | +| `NEWKEYS` | 21 | RFC 4253 §7.3 | +| `KEX_ECDH_INIT` | 30 | RFC 5656 §4 | +| `KEX_ECDH_REPLY` | 31 | RFC 5656 §4 | +| `KEX_DH_GEX_REQUEST` | 34 | RFC 4419 §3 | +| `KEX_DH_GEX_GROUP` | 31 | RFC 4419 §3 | +| `KEX_DH_GEX_INIT` | 32 | RFC 4419 §3 | +| `KEX_DH_GEX_REPLY` | 33 | RFC 4419 §3 | +| `USERAUTH_REQUEST` | 50 | RFC 4252 §5 | +| `USERAUTH_FAILURE` | 51 | RFC 4252 §5 | +| `USERAUTH_SUCCESS` | 52 | RFC 4252 §5 | +| `USERAUTH_BANNER` | 53 | RFC 4252 §5.4 | +| `USERAUTH_INFO_REQUEST` | 60 | RFC 4256 §5 | +| `USERAUTH_INFO_RESPONSE` | 61 | RFC 4256 §5 | +| `GLOBAL_REQUEST` | 80 | RFC 4254 §4 | +| `REQUEST_SUCCESS` | 81 | RFC 4254 §4 | +| `REQUEST_FAILURE` | 82 | RFC 4254 §4 | +| `CHANNEL_OPEN` | 90 | RFC 4254 §5.1 | +| `CHANNEL_OPEN_CONFIRMATION` | 91 | RFC 4254 §5.1 | +| `CHANNEL_OPEN_FAILURE` | 92 | RFC 4254 §5.1 | +| `CHANNEL_WINDOW_ADJUST` | 93 | RFC 4254 §5.2 | +| `CHANNEL_DATA` | 94 | RFC 4254 §5.2 | +| `CHANNEL_EXTENDED_DATA` | 95 | RFC 4254 §5.2 | +| `CHANNEL_EOF` | 96 | RFC 4254 §5.3 | +| `CHANNEL_CLOSE` | 97 | RFC 4254 §5.3 | +| `CHANNEL_REQUEST` | 98 | RFC 4254 §5.4 | +| `CHANNEL_SUCCESS` | 99 | RFC 4254 §5.4 | +| `CHANNEL_FAILURE` | 100 | RFC 4254 §5.4 | + +--- + +## Connection Lifecycle + +### 1. Version Exchange + +Both sides send their SSH identification string: + +``` +SSH-2.0-russh_0.60.2\r\n +``` + +The `SshId` type handles this: + +```rust +pub enum SshId { + Standard(Cow<'static, str>), // Appends \r\n + Raw(Cow<'static, str>), // Sent as-is +} +``` + +Client sends first, then reads server's ID. Server reads client's ID first, then sends its own. + +### 2. Key Exchange (KEXINIT) + +The `negotiation` module handles algorithm negotiation. Each side sends a `KEXINIT` packet containing lists of supported algorithms. The first algorithm from the client's list that also appears in the server's list is chosen (for client), or vice versa (for server). + +**Algorithm selection order** (per `Preferred::DEFAULT`): +- **KEX**: `mlkem768x25519-sha256` → `curve25519-sha256` → `curve25519-sha256@libssh.org` → `diffie-hellman-group-exchange-sha256` → group18/17/16/15/14 +- **Host Key**: `ssh-ed25519` → `ecdsa-sha2-nistp256/384/521` → `rsa-sha2-512/256` → `ssh-rsa` +- **Cipher**: `chacha20-poly1305@openssh.com` → `aes256-gcm@openssh.com` → `aes256-ctr` → `aes192-ctr` → `aes128-ctr` +- **MAC**: `hmac-sha2-512-etm@openssh.com` → `hmac-sha2-256-etm@openssh.com` → `hmac-sha2-512` → `hmac-sha2-256` + +**Negotiation flow** (`write_kex` / `read_kex`): +1. Both sides send KEXINIT with their algorithm lists +2. Each side parses the other's KEXINIT using `Client::read_kex()` or `Server::read_kex()` +3. The `Select` trait implements the first-match algorithm +4. `Names` struct is produced with the negotiated algorithms +5. Extension kex names (like `ext-info-c`, `kex-strict-c-v00@openssh.com`) are handled specially — filtered from negotiation but used for capability detection + +### 3. Diffie-Hellman Key Exchange + +After KEXINIT, the actual key exchange runs. This is managed by `ClientKex` and `ServerKex` state machines. + +**Client KEX state machine** (`ClientKex`): +``` +Created → (receive KEXINIT) → WaitingForDhReply or WaitingForGexReply → WaitingForNewKeys → Done +``` + +Steps for a typical Curve25519 exchange: +1. **Client sends KEX_ECDH_INIT**: Contains client's ephemeral public key +2. **Server sends KEX_ECDH_REPLY**: Contains server host key, server's ephemeral public key, and a signature +3. **Client verifies**: + - Parses the server host key + - Computes the shared secret from the ephemeral keys + - Computes the exchange hash (H) from: `V_C || V_S || I_C || I_S || K_S || e || f || K` + - Verifies the server's signature on H using the server host key + - Calls `handler.check_server_key()` for application-level trust verification +4. **Both sides send NEWKEYS**: Activate the newly computed encryption keys + +**Server KEX** mirrors this, responding to client's init and sending the reply. + +**DH-GEX (Group Exchange) flow** has an extra step: +1. Client sends `KEX_DH_GEX_REQUEST` with min/preferred/max group sizes +2. Server sends `KEX_DH_GEX_GROUP` with the prime and generator +3. Client validates group size and sends `KEX_DH_GEX_INIT` +4. Server sends `KEX_DH_GEX_REPLY` +5. Both send `NEWKEYS` + +### 4. Key Derivation (`compute_keys`) + +After the shared secret and exchange hash are computed, keys are derived per RFC 4253 §7.2: + +``` +Key = HASH(K || H || X || session_id) +``` + +Where `X` is a single character: +- `A` = client-to-server IV (nonce) +- `B` = server-to-client IV (nonce) +- `C` = client-to-server encryption key +- `D` = server-to-client encryption key +- `E` = client-to-server MAC key +- `F` = server-to-client MAC key + +If the derived key is shorter than needed, it's extended by hashing `K || H || Key[0..n]` repeatedly. + +The `session_id` is set to the exchange hash from the first key exchange and remains constant across rekeys. + +### 5. Strict Key Exchange + +Russh supports the OpenSSH strict key exchange extension (`kex-strict-c-v00@openssh.com` / `kex-strict-s-v00@openssh.com`), which: +- Resets sequence numbers after NEWKEYS +- Enforces that only KEXINIT, DH init, and NEWKEYS messages appear during initial kex +- Validated by `validate_client_msg_strict_kex` / `validate_server_msg_strict_kex` +- Prevents the Terrapin attack (CVE-2023-48799) + +### 6. Authentication + +After NEWKEYS, the encrypted state machine transitions through: + +```rust +pub enum EncryptedState { + WaitingAuthServiceRequest { sent: bool, accepted: bool }, + WaitingAuthRequest(auth::AuthRequest), + InitCompression, + Authenticated, +} +``` + +Flow: +1. Client sends `SERVICE_REQUEST` for "ssh-connection" +2. Server responds with `SERVICE_ACCEPT` +3. Client sends authentication requests (password, publickey, keyboard-interactive, none) +4. Server responds with `USERAUTH_SUCCESS`, `USERAUTH_FAILURE`, or `USERAUTH_INFO_REQUEST` +5. Upon success, state transitions to `InitCompression` → `Authenticated` + +**Authentication methods** (`auth::Method`): +```rust +pub enum Method { + None, + Password { password: String }, + PublicKey { key: PrivateKeyWithHashAlg }, + OpenSshCertificate { key: Arc, cert: Certificate }, + FuturePublicKey { key: PublicKey, hash_alg: Option }, + FutureCertificate { cert: Certificate, hash_alg: Option }, + KeyboardInteractive { submethods: String }, +} +``` + +The `FuturePublicKey` / `FutureCertificate` variants are used with the `Signer` trait (e.g., SSH agent), where signing is delegated to an external component. + +### 7. Rekeying + +Rekeying is triggered automatically when: +- Bytes written exceed `Limits::rekey_write_limit` (default 1 GB) +- Bytes read exceed `Limits::rekey_read_limit` (default 1 GB) +- Time since last rekey exceeds `Limits::rekey_time_limit` (default 1 hour) +- Explicitly via `Handle::rekey_soon()` + +During rekey: +- New `KEXINIT` packets are exchanged +- Pending channel data is buffered until rekey completes +- Sequence numbers are reset if strict kex is negotiated + +--- + +## Channel Multiplexing + +SSH channels are multiplexed over a single connection. Each channel has: +- A `ChannelId` (local identifier) +- A `recipient_channel` (remote identifier) +- Window size for flow control +- Maximum packet size + +### Channel Types + +| Type | Open Method | Description | +|------|-------------|-------------| +| `session` | `channel_open_session()` | Standard shell/exec channel | +| `x11` | `channel_open_x11()` | X11 forwarding | +| `direct-tcpip` | `channel_open_direct_tcpip()` | Local TCP port forwarding | +| `forwarded-tcpip` | Server-initiated | Remote TCP port forwarding | +| `direct-streamlocal` | `channel_open_direct_streamlocal()` | Local UNIX socket forwarding | +| `forwarded-streamlocal` | Server-initiated | Remote UNIX socket forwarding | +| `auth-agent@openssh.com` | Server-initiated | Agent forwarding | + +### Flow Control + +SSH uses a window-based flow control mechanism: +- Each side maintains a `sender_window_size` and `recipient_window_size` +- Data cannot be sent if the recipient's window is exhausted +- `CHANNEL_WINDOW_ADJUST` messages increase the window +- `adjust_window()` is called when data is received to determine the next target window +- Data is queued in `pending_data` when the window is full, and flushed when the window is adjusted + +### Channel Request/Response Pattern + +Many channel operations follow a request-response pattern: +1. Client sends `CHANNEL_REQUEST` with `want_reply = true` +2. Server processes the request via `Handler` callback +3. Server calls `session.channel_success(channel)` or `session.channel_failure(channel)` +4. Client receives `CHANNEL_SUCCESS` or `CHANNEL_FAILURE` + +--- + +## Port Forwarding + +### Local TCP Forwarding (direct-tcpip) + +Client opens a channel to forward a TCP connection through the server: + +```rust +// Client side +let channel = handle.channel_open_direct_tcpip( + "target-host", 8080, // host:port to connect to + "origin-host", 12345, // originator info +).await?; +``` + +### Remote TCP Forwarding (forward-tcpip) + +**Step 1**: Client requests the server to listen on a port: +```rust +let allocated_port = handle.tcpip_forward("0.0.0.0", 0).await?; +``` + +**Step 2**: When a connection arrives, the server calls `Handler::channel_open_forwarded_tcpip()`. + +**Step 3**: Client receives the channel via `Handler::server_channel_open_forwarded_tcpip()`. + +**Step 4**: To stop: +```rust +handle.cancel_tcpip_forward("0.0.0.0", allocated_port).await?; +``` + +### UNIX Socket Forwarding (streamlocal) + +Similar to TCP forwarding but for UNIX domain sockets: + +```rust +// Local +handle.channel_open_direct_streamlocal("/tmp/socket").await?; + +// Remote +handle.streamlocal_forward("/tmp/socket").await?; +handle.cancel_streamlocal_forward("/tmp/socket").await?; +``` + +--- + +## Extensions + +### `ext-info-c` / `ext-info-s` (RFC 8308) + +Extension info is sent after NEWKEYS. Russh supports: +- **`server-sig-algs`**: Server tells client which signature algorithms it accepts for public key authentication. Used by `Handle::best_supported_rsa_hash()`. + +### `kex-strict-c-v00@openssh.com` / `kex-strict-s-v00@openssh.com` + +OpenSSH strict key exchange (Terrapin attack mitigation). Negotiated during KEXINIT and enforced during the initial key exchange. + +### OpenSSH Agent Forwarding + +The server can open an `auth-agent@openssh.com` channel to forward the client's SSH agent. The client handles this via `Handler::server_channel_open_agent_forward()`. + +### OpenSSH `no-more-sessions@openssh.com` + +Requests that the other side not open any more sessions: +```rust +handle.no_more_sessions(true).await?; +``` \ No newline at end of file diff --git a/docs/research/references/ssh/russh/04-crypto.md b/docs/research/references/ssh/russh/04-crypto.md new file mode 100644 index 0000000..08e2fd4 --- /dev/null +++ b/docs/research/references/ssh/russh/04-crypto.md @@ -0,0 +1,299 @@ +# Russh: Cryptographic Primitives + +This document covers the cryptographic implementations in russh — key exchange algorithms, ciphers, MACs, and key handling. + +## Key Exchange Algorithms (`kex` module) + +All KEX algorithms implement the `KexAlgorithmImplementor` trait: + +```rust +pub(crate) trait KexAlgorithmImplementor { + fn skip_exchange(&self) -> bool; + fn is_dh_gex(&self) -> bool; + fn client_dh_gex_init(&mut self, gex: &GexParams, writer: &mut impl Writer) -> Result<(), Error>; + fn dh_gex_set_group(&mut self, group: DhGroup) -> Result<(), Error>; + fn server_dh(&mut self, exchange: &mut Exchange, payload: &[u8]) -> Result<(), Error>; + fn client_dh(&mut self, client_ephemeral: &mut Vec, writer: &mut impl Writer) -> Result<(), Error>; + fn compute_shared_secret(&mut self, remote_pubkey: &[u8]) -> Result<(), Error>; + fn shared_secret_bytes(&self) -> Option<&[u8]>; + fn compute_exchange_hash(&self, key: &[u8], exchange: &Exchange, buffer: &mut CryptoVec) -> Result, Error>; + fn compute_keys(&self, session_id: &[u8], exchange_hash: &[u8], cipher: cipher::Name, remote_to_local_mac: mac::Name, local_to_remote_mac: mac::Name, is_server: bool) -> Result; +} +``` + +The `KexAlgorithm` enum dispatches via `enum_dispatch`: + +```rust +pub(crate) enum KexAlgorithm { + DhGroupKexSha1(DhGroupKex), + DhGroupKexSha256(DhGroupKex), + DhGroupKexSha512(DhGroupKex), + Curve25519Kex(Curve25519Kex), + EcdhNistP256Kex(EcdhNistPKex), + EcdhNistP384Kex(EcdhNistPKex), + EcdhNistP521Kex(EcdhNistPKex), + MlKem768X25519Kex(MlKem768X25519Kex), + NoneKexAlgorithm(NoneKexAlgorithm), +} +``` + +### Supported KEX Algorithms + +| Algorithm Name | Constant | Type | Hash | Notes | +|----------------|----------|------|------|-------| +| `mlkem768x25519-sha256` | `MLKEM768X25519_SHA256` | PQ/T Hybrid (ML-KEM + X25519) | SHA-256 | **Default first choice**, post-quantum | +| `curve25519-sha256` | `CURVE25519` | Curve25519 ECDH | SHA-256 | Recommended | +| `curve25519-sha256@libssh.org` | `CURVE25519_PRE_RFC_8731` | Curve25519 ECDH | SHA-256 | Pre-RFC name, same impl | +| `ecdh-sha2-nistp256` | `ECDH_SHA2_NISTP256` | NIST P-256 ECDH | SHA-256 | | +| `ecdh-sha2-nistp384` | `ECDH_SHA2_NISTP384` | NIST P-384 ECDH | SHA-384 | | +| `ecdh-sha2-nistp521` | `ECDH_SHA2_NISTP521` | NIST P-521 ECDH | SHA-512 | | +| `diffie-hellman-group-exchange-sha256` | `DH_GEX_SHA256` | DH-GEX | SHA-256 | Dynamic group | +| `diffie-hellman-group-exchange-sha1` | `DH_GEX_SHA1` | DH-GEX | SHA-1 | Legacy | +| `diffie-hellman-group14-sha256` | `DH_G14_SHA256` | DH Group 14 | SHA-256 | 2048-bit | +| `diffie-hellman-group14-sha1` | `DH_G14_SHA1` | DH Group 14 | SHA-1 | Legacy | +| `diffie-hellman-group15-sha512` | `DH_G15_SHA512` | DH Group 15 | SHA-512 | 3072-bit | +| `diffie-hellman-group16-sha512` | `DH_G16_SHA512` | DH Group 16 | SHA-512 | 4096-bit | +| `diffie-hellman-group17-sha512` | `DH_G17_SHA512` | DH Group 17 | SHA-512 | 6144-bit | +| `diffie-hellman-group18-sha512` | `DH_G18_SHA512` | DH Group 18 | SHA-512 | 8192-bit | +| `diffie-hellman-group1-sha1` | `DH_G1_SHA1` | DH Group 1 | SHA-1 | **Insecure**, 1024-bit | +| `none` | `NONE` | No exchange | N/A | Testing only | + +### Curve25519 Implementation + +Located in `kex/curve25519.rs`. Uses `curve25519-dalek` for the Diffie-Hellman computation: +- Generates an ephemeral keypair +- Sends the public key as the DH init +- Computes the shared secret from the remote public key +- The shared secret is encoded as a raw 32-byte string (not mpint) in the exchange hash + +### NIST ECDH Implementation + +Located in `kex/ecdh_nistp.rs`. Generic over the curve (`NistP256`, `NistP384`, `NistP521`): +- Uses `p256`/`p384`/`p521` crates for ECDH +- Points are encoded in the SSH EC point format (32/48/66 bytes for uncompressed) + +### DH Group Exchange Implementation + +Located in `kex/dh/`. Two variants: `DhGroupKex` (GEX) and fixed-group variants. +- GEX uses `num-bigint` for modular exponentiation with safe primes +- Fixed groups use pre-defined safe primes from RFC 3526 +- `DhGroup` struct: `{ prime: CryptoVec, generator: CryptoVec }` +- Server provides groups via `Handler::lookup_dh_gex_group()` — default uses `BUILTIN_SAFE_DH_GROUPS` +- `GexParams` controls min/preferred/max group sizes (default: 3072/8192/8192 bits) + +### ML-KEM Hybrid Implementation + +Located in `kex/hybrid_mlkem.rs`. Implements the `mlkem768x25519-sha256` hybrid key exchange: +- Combines ML-KEM-768 (post-quantum) with X25519 (classical) +- Both shared secrets are concatenated before hashing +- Provides quantum-resistant key exchange while maintaining classical security + +### Exchange Hash Computation + +The exchange hash `H` is computed from the following data (per RFC 4253 §8): + +``` +H = HASH(V_C || V_S || I_C || I_S || K_S || e || f || K) +``` + +Where: +- `V_C` = client version string +- `V_S` = server version string +- `I_C` = client's KEXINIT payload +- `I_S` = server's KEXINIT payload +- `K_S` = server host key +- `e` = client ephemeral public key +- `f` = server ephemeral public key +- `K` = shared secret + +For DH-GEX, additional fields are included (p, g, and the GEX parameters). + +--- + +## Ciphers (`cipher` module) + +The `Cipher` trait defines how to construct opening and sealing keys: + +```rust +pub(crate) trait Cipher { + fn needs_mac(&self) -> bool; // Whether a separate MAC is needed + fn key_len(&self) -> usize; + fn nonce_len(&self) -> usize; + fn make_opening_key(&self, key, nonce, mac_key, mac) -> Box; + fn make_sealing_key(&self, key, nonce, mac_key, mac) -> Box; +} +``` + +### Supported Ciphers + +| Algorithm Name | Constant | Type | Key Size | MAC Required | AEAD | +|----------------|----------|------|----------|-------------|------| +| `chacha20-poly1305@openssh.com` | `CHACHA20_POLY1305` | Stream cipher + Poly1305 | 256-bit + 256-bit | No | Yes | +| `aes256-gcm@openssh.com` | `AES_256_GCM` | AES-GCM | 256-bit | No | Yes | +| `aes128-gcm@openssh.com` | `AES_128_GCM` | AES-GCM | 128-bit | No | Yes | +| `aes256-ctr` | `AES_256_CTR` | AES-CTR | 256-bit | Yes | No | +| `aes192-ctr` | `AES_192_CTR` | AES-CTR | 192-bit | Yes | No | +| `aes128-ctr` | `AES_128_CTR` | AES-CTR | 128-bit | Yes | No | +| `aes256-cbc` | `AES_256_CBC` | AES-CBC | 256-bit | Yes | No | +| `aes192-cbc` | `AES_192_CBC` | AES-CBC | 192-bit | Yes | No | +| `aes128-cbc` | `AES_128_CBC` | AES-CBC | 128-bit | Yes | No | +| `3des-cbc` | `TRIPLE_DES_CBC` | 3DES-CBC | 168-bit | Yes | No | (feature `des`) | + +### AEAD Ciphers (Chacha20-Poly1305, AES-GCM) + +AEAD ciphers do not need a separate MAC. They handle both encryption and authentication: +- **Chacha20-Poly1305**: Uses the OpenSSH construction where the sequence number is used as the Chacha20 counter. The Poly1305 key is derived from a separate Chacha20 stream. +- **AES-GCM**: Uses `aws-lc-rs` or `ring` as the backend depending on feature flags. The nonce is constructed from the sequence number. + +### Block Ciphers (AES-CTR, AES-CBC, 3DES-CBC) + +Block ciphers require a separate MAC. They implement `SshBlockCipher`: +- **CTR mode**: Uses `ctr::Ctr128BE` from the `ctr` crate +- **CBC mode**: Uses `cbc::Encryptor`/`cbc::Decryptor` from the `cbc` crate with PKCS#7 padding +- The `needs_mac()` method returns `true` + +### `OpeningKey` and `SealingKey` Traits + +```rust +pub(crate) trait OpeningKey { + fn packet_length_to_read_for_block_length(&self) -> usize { 4 } + fn decrypt_packet_length(&self, seqn: u32, encrypted_packet_length: &[u8]) -> [u8; 4]; + fn tag_len(&self) -> usize; + fn open<'a>(&mut self, seqn: u32, ciphertext_and_tag: &'a mut [u8]) -> Result<&'a [u8], Error>; +} + +pub(crate) trait SealingKey { + fn padding_length(&self, plaintext: &[u8]) -> usize; + fn fill_padding(&self, padding_out: &mut [u8]); + fn tag_len(&self) -> usize; + fn seal(&mut self, seqn: u32, plaintext_in_ciphertext_out: &mut [u8], tag_out: &mut [u8]); + fn write(&mut self, payload: &[u8], buffer: &mut SSHBuffer); +} +``` + +The `write()` method on `SealingKey` handles the full packet construction: +1. Compute padding length (minimum 4 bytes, block-aligned) +2. Write packet length (4 bytes) + padding length (1 byte) + payload + padding +3. Encrypt the packet +4. Append the authentication tag +5. Increment the sequence number + +--- + +## MACs (`mac` module) + +MAC algorithms are split into two categories: regular and Encrypt-Then-MAC (ETM). + +### Supported MACs + +| Algorithm Name | Constant | Hash | Key Length | ETM | +|----------------|----------|------|-----------|-----| +| `hmac-sha2-512-etm@openssh.com` | `HMAC_SHA512_ETM` | SHA-512 | 64 bytes | Yes | +| `hmac-sha2-256-etm@openssh.com` | `HMAC_SHA256_ETM` | SHA-256 | 32 bytes | Yes | +| `hmac-sha2-512` | `HMAC_SHA512` | SHA-512 | 64 bytes | No | +| `hmac-sha2-256` | `HMAC_SHA256` | SHA-256 | 32 bytes | No | +| `hmac-sha1-etm@openssh.com` | `HMAC_SHA1_ETM` | SHA-1 | 20 bytes | Yes | +| `hmac-sha1` | `HMAC_SHA1` | SHA-1 | 20 bytes | No | +| `none` | `NONE` | — | 0 | — | + +### ETM (Encrypt-Then-MAC) Mode + +With ETM MACs: +- The packet length field is sent unencrypted (read first) +- The rest of the packet is encrypted +- The MAC is computed over the unencrypted length + encrypted payload +- This prevents padding oracle attacks + +### Regular MAC Mode + +With regular MACs: +- The entire packet (including length) is encrypted +- The MAC is computed over the sequence number + unencrypted packet +- This is the traditional SSH MAC construction + +--- + +## Key Handling (`keys` module) + +### Key Formats Supported + +- **OpenSSH format** (private keys): `-----BEGIN OPENSSH PRIVATE KEY-----` with bcrypt-pbkdf encrypted keys +- **PKCS#8** (unencrypted and encrypted): `-----BEGIN PRIVATE KEY-----` / `-----BEGIN ENCRYPTED PRIVATE KEY-----` +- **PKCS#1** (RSA): `-----BEGIN RSA PRIVATE KEY-----` +- **SEC1** (EC): `-----BEGIN EC PRIVATE KEY-----` +- **OpenSSH public keys**: `ssh-ed25519 AAAAC3N...` +- **PPK format**: PuTTY private key files +- **OpenSSH certificates**: `-----BEGIN OPENSSH SSH2 CERTIFICATE-----` + +### Key Algorithms + +| Algorithm | Key Type | Signing | +|-----------|----------|---------| +| `ssh-ed25519` | Ed25519 | Ed25519 | +| `ecdsa-sha2-nistp256` | ECDSA P-256 | SHA-256 | +| `ecdsa-sha2-nistp384` | ECDSA P-384 | SHA-384 | +| `ecdsa-sha2-nistp521` | ECDSA P-521 | SHA-512 | +| `rsa-sha2-512` | RSA | SHA-512 | +| `rsa-sha2-256` | RSA | SHA-256 | +| `ssh-rsa` | RSA | SHA-1 (legacy) | + +### `PrivateKeyWithHashAlg` + +Wrapper that pairs a private key with a specific hash algorithm (needed for RSA key disambiguation): + +```rust +pub struct PrivateKeyWithHashAlg { + key: Arc, + hash: Option, +} +``` + +This is required because RSA keys can sign with different hash algorithms (`rsa-sha2-256`, `rsa-sha2-512`, `ssh-rsa`), and the server needs to know which one the client intends. + +### SSH Agent Protocol (`keys::agent`) + +Russh implements both the client and server sides of the SSH agent protocol: + +**Client** (`agent::client::AgentClient`): +```rust +impl AgentClient { + pub async fn request_identities(&mut self) -> Result, Error>; + pub async fn sign_request(&mut self, key: &AgentIdentity, hash_alg: Option, data: Vec) -> Result, Error>; + pub async fn add_identity(&mut self, key: &PrivateKey, constraints: &[Constraint]) -> Result<(), Error>; + pub async fn remove_identity(&mut self, key: &PublicKey) -> Result<(), Error>; + pub async fn remove_all_identities(&mut self) -> Result<(), Error>; + pub async fn lock(&mut self, passphrase: &str) -> Result<(), Error>; + pub async fn unlock(&mut self, passphrase: &str) -> Result<(), Error>; + // ... ping, add smartcard, etc. +} +``` + +**Server** (`agent::server::Agent`): +```rust +pub trait Agent: Clone + Send + 'static { + fn confirm(self, key: Arc) -> Box + Send + Unpin>; +} +``` + +**AgentIdentity** distinguishes between plain public keys and certificates: +```rust +pub enum AgentIdentity { + PublicKey { pubkey: PublicKey, comment: String }, + Certificate { certificate: Certificate, comment: String }, +} +``` + +### Known Hosts (`keys::known_hosts`) + +```rust +pub fn check_known_hosts(host: &str, port: u16, key: &PublicKey) -> Result<(), Error>; +pub fn check_known_hosts_path>(path: P, host: &str, port: u16, key: &PublicKey) -> Result<(), Error>; +``` + +These functions read `~/.ssh/known_hosts` and verify the server's public key matches. Returns `Error::KeyChanged` on mismatch. + +### Windows Pageant Support + +On Windows, russh uses the `pageant` crate to communicate with the Pageant SSH agent via either: +- WM_MESSAGE-based protocol (legacy) +- Named pipes protocol (since PuTTY 0.75) \ No newline at end of file diff --git a/docs/research/references/ssh/russh/05-internal-architecture.md b/docs/research/references/ssh/russh/05-internal-architecture.md new file mode 100644 index 0000000..eff522c --- /dev/null +++ b/docs/research/references/ssh/russh/05-internal-architecture.md @@ -0,0 +1,430 @@ +# Russh: Internal Architecture & Data Flow + +This document covers the internal mechanics — session state machines, the event loop, packet handling, buffering, and the interaction between components. + +## Session State + +### `CommonSession` + +The shared session state used by both client and server: + +```rust +pub(crate) struct CommonSession { + pub auth_user: String, + pub remote_sshid: Vec, + pub config: C, // Arc or Arc + pub encrypted: Option, + pub auth_method: Option, // Client only + pub auth_attempts: usize, + pub packet_writer: PacketWriter, + pub remote_to_local: Box, + pub wants_reply: bool, + pub disconnected: bool, + pub buffer: Vec, // Incoming packet scratch buffer + pub strict_kex: bool, + pub alive_timeouts: usize, + pub received_data: bool, +} +``` + +### `Encrypted` + +The state after encryption keys are established: + +```rust +pub(crate) struct Encrypted { + pub state: EncryptedState, + pub exchange: Option, + pub kex: KexAlgorithm, + pub key: usize, + pub client_mac: mac::Name, + pub server_mac: mac::Name, + pub session_id: CryptoVec, // Constant across rekeys + pub channels: HashMap, + pub last_channel_id: Wrapping, + pub write: Vec, // Outgoing packet assembly buffer + pub write_cursor: usize, // Current position in write buffer + pub last_rekey: Instant, + pub server_compression: Compression, + pub client_compression: Compression, + pub decompress: Decompress, + pub rekey_wanted: bool, + pub received_extensions: Vec, + pub extension_info_awaiters: HashMap>>, +} +``` + +### `EncryptedState` + +```rust +pub enum EncryptedState { + WaitingAuthServiceRequest { sent: bool, accepted: bool }, + WaitingAuthRequest(auth::AuthRequest), + InitCompression, + Authenticated, +} +``` + +### `Exchange` + +Protocol values collected during key exchange (all non-secret, visible on wire): + +```rust +pub struct Exchange { + pub client_id: Vec, + pub server_id: Vec, + pub client_kex_init: Vec, + pub server_kex_init: Vec, + pub client_ephemeral: Vec, + pub server_ephemeral: Vec, + pub gex: Option<(GexParams, DhGroup)>, +} +``` + +### `NewKeys` + +Produced when key exchange completes, contains everything needed to activate encryption: + +```rust +pub(crate) struct NewKeys { + pub exchange: Exchange, + pub names: negotiation::Names, + pub kex: KexAlgorithm, + pub key: usize, + pub cipher: CipherPair, // { local_to_remote, remote_to_local } + pub session_id: CryptoVec, +} +``` + +### `ChannelParams` + +Internal channel state (not exposed to users): + +```rust +pub(crate) struct ChannelParams { + pub recipient_channel: u32, // Remote channel ID + pub sender_channel: ChannelId, // Local channel ID + pub recipient_window_size: u32, + pub sender_window_size: u32, + pub recipient_maximum_packet_size: u32, + pub sender_maximum_packet_size: u32, + pub confirmed: bool, // Whether server confirmed the channel + pub wants_reply: bool, + pub pending_data: VecDeque<(Bytes, Option, usize)>, // (data, ext_code, offset) + pub pending_eof: bool, + pub pending_close: bool, +} +``` + +--- + +## Client Event Loop + +The client event loop is the core of `client::Session::run_inner()`: + +```rust +async fn run_inner(&mut self, stream_read, stream_write, handler, kex_done_signal) + -> Result +{ + // Initial setup + self.flush()?; + self.common.packet_writer.flush_into(stream_write).await?; + + // Set up timers + let keepalive_timer = ...; + let inactivity_timer = ...; + let reading = start_reading(stream_read, buffer, opening_cipher); + + while !self.common.disconnected { + tokio::select! { + // 1. Incoming SSH packet + r = &mut reading => { + // Decrypt and decompress packet + // Process DISCONNECT or pass to reply() + // Restart reading + } + + // 2. Keepalive timer + () = &mut keepalive_timer => { + // Send keepalive if authenticated + // Track timeout count + } + + // 3. Inactivity timer + () = &mut inactivity_timer => { + // Return InactivityTimeout error + } + + // 4. Outgoing messages from Handle + msg = self.receiver.recv(), if !self.kex.active() => { + // Process message (auth, channel open, data, etc.) + // Batch all pending outgoing messages + } + + // 5. Inbound channel messages + msg = self.inbound_channel_receiver.recv(), if !self.kex.active() => { + // Process channel data/eof/close + // Batch all pending messages + } + }; + + // Flush all pending writes + self.flush()?; + self.common.packet_writer.flush_into(stream_write).await?; + + // Handle deferred compression after authentication + if EncryptedState::InitCompression { ... } { + // Init client compression if deferred (zlib@openssh.com) + // Transition to Authenticated + } + + // Reset timers if data received or keepalive sent + } +} +``` + +### Key Event Loop Behaviors + +1. **Kex blocking**: When `self.kex.active()` is true, outgoing messages from `Handle` are NOT processed. This prevents sending data during key exchange. + +2. **Batching**: After receiving one message from `receiver`, all pending messages are drained with `try_recv()` to batch writes. + +3. **Keepalive management**: Keepalive timer resets when data is received. `alive_timeouts` tracks consecutive unanswered keepalives. + +4. **Compression activation**: `zlib@openssh.com` compression is deferred until after authentication succeeds (handled by `InitCompression` state). + +--- + +## Server Event Loop + +Similar to the client, but the server accepts connections via `run_on_socket()` or `run_stream()`: + +```rust +// Server::run_on_socket +loop { + tokio::select! { + _ = shutdown_rx.recv() => { /* Graceful shutdown */ }, + accept_result = socket.accept() => { + // For each connection: + // 1. Create a new Handler via Server::new_client() + // 2. Call run_stream() in a spawned task + // 3. Wait for session or shutdown + }, + error = error_rx.recv() => { + // Report session errors + } + } +} +``` + +### `run_stream` + +```rust +pub async fn run_stream(config, stream, handler) -> Result, H::Error> +{ + // 1. Write server SSH ID + // 2. Read client SSH ID + // 3. Create Session with CommonSession state + // 4. Begin initial rekey (sends KEXINIT) + // 5. Spawn session.run() in a task + // 6. Return RunningSession (implements Future) +} +``` + +### `RunningServer` and `RunningServerHandle` + +The server returns a `RunningServer` that implements `Future` and a `RunningServerHandle` for graceful shutdown: + +```rust +pub struct RunningServer> + Unpin + Send> { + inner: F, + shutdown_tx: broadcast::Sender, +} + +impl RunningServerHandle { + pub fn shutdown(&self, reason: String) { + let _ = self.shutdown_tx.send(reason); + } +} +``` + +--- + +## Packet Handling Pipeline + +### Reading Packets (`cipher::read`) + +``` +1. Read 4 bytes (or more for block ciphers) → encrypted packet length +2. Decrypt packet length +3. Parse length, check against MAXIMUM_PACKET_LEN (262159 bytes) +4. Read remaining bytes (length + tag_len - already_read) +5. Decrypt the ciphertext (including tag verification for AEAD) +6. Remove padding +7. Increment sequence number +``` + +### Writing Packets (`SealingKey::write`) + +``` +1. Compute padding length (block-aligned, min 4 bytes) +2. Write: [packet_length (4B)] [padding_length (1B)] [payload] [padding] [tag] +3. Encrypt the packet +4. Compute and append MAC/tag +5. Increment sequence number +6. Add payload bytes to rekey counter +``` + +### Packet Assembly (`PacketWriter`) + +```rust +pub(crate) struct PacketWriter { + cipher: Box, + compress: Compress, + compress_buffer: Vec, + write_buffer: SSHBuffer, +} +``` + +Methods: +- `packet_raw(buf)`: Compress and encrypt a raw packet +- `packet(f)`: Build a packet via closure, compress, encrypt, return the plaintext +- `flush_into(w)`: Write all buffered data to the async writer +- `set_cipher(c)`: Swap the cipher (for rekeying) +- `reset_seqn()`: Reset sequence number (for strict kex) + +### `push_packet!` Macro + +Used throughout for building packets: + +```rust +macro_rules! push_packet { + ( $buffer:expr, $x:expr ) => {{ + let i0 = $buffer.len(); + $buffer.extend(b"\0\0\0\0"); // Placeholder for length + let x = $x; // Build the packet body + let i1 = $buffer.len(); + BigEndian::write_u32(&mut buf[i0..], (i1 - i0 - 4) as u32); + x + }}; +} +``` + +--- + +## Channel Data Flow + +### Client-side Channel Creation + +``` +1. Handle::channel_open_session() + → Creates ChannelRef with mpsc::channel + → Sends Msg::ChannelOpenSession to session + +2. Session::handle_msg(Msg::ChannelOpenSession) + → Calls self.channel_open_session() + → Sends CHANNEL_OPEN packet with channel type "session" + → Stores ChannelRef in self.channels + +3. Server responds with CHANNEL_OPEN_CONFIRMATION + → Session updates ChannelParams (recipient_channel, window sizes) + → Sends ChannelMsg::Open through ChannelRef's sender + +4. wait_channel_confirmation() receives the Open message + → Creates Channel { read_half, write_half } + → Returns the Channel to the caller +``` + +### Data Transmission (Client → Server) + +``` +1. channel.data(reader) + → Uses ChannelTx (AsyncWrite) that reads from the reader + → Sends ChannelMsg::Data through the channel sender + → Waits for window availability via WindowSizeRef + +2. Session receives ChannelMsg::Data + → Calls Encrypted::data(channel, bytes, is_rekeying) + → If pending data exists or rekeying: queues in pending_data + → Otherwise: writes CHANNEL_DATA packet immediately + → If window exhausted: queues remaining data + +3. PacketWriter encrypts and buffers the packet +4. Event loop flushes the buffer to the TCP stream +``` + +### Data Reception (Server → Client) + +``` +1. Encrypted packet arrives, decrypted, decompressed +2. Packet type = CHANNEL_DATA +3. Encrypted::adjust_window_size() checks if window needs adjustment +4. ChannelMsg::Data sent through ChannelRef's sender +5. Channel::wait() returns the ChannelMsg::Data +6. ChannelRx (AsyncRead) reads from the channel receiver +``` + +--- + +## Rekeying Flow + +``` +1. Trigger: + - Bytes written/read exceed limits + - Time since last rekey exceeds limit + - Explicit: Handle::rekey_soon() + - Server-initiated: receiving KEXINIT when idle + +2. Session::begin_rekey(): + - Creates new ClientKex/ServerKex + - Sends new KEXINIT + - Sets kex state to InProgress + +3. During rekey: + - Outgoing messages from Handle are blocked (!kex.active()) + - Incoming non-kex packets are buffered in pending_reads + - Kex state machine processes kex packets (KEXINIT, DH init/reply, NEWKEYS) + +4. On completion: + - Flush all pending channel data + - Process buffered pending_reads + - Call CommonSession::newkeys() to swap ciphers + - Reset byte counters + - Set kex state to Idle + - Resume processing outgoing messages +``` + +--- + +## Sub-Crates + +### `russh-cryptovec` (cryptovec/) + +A `Vec` alternative that: +- Zeroes memory on drop and reallocation (via `memset` / `ExplicitZero`) +- Locks memory pages with `mlock` (Unix) / `VirtualLock` (Windows) to prevent swapping +- Uses `unsafe` for performance-critical operations (copying, initialization) +- Integrates with `ssh-encoding` for Encode support + +Used for all sensitive data: session keys, shared secrets, MAC keys, exchange hashes. + +### `russh-util` (russh-util/) + +Runtime abstraction layer: +- `russh_util::runtime::spawn()` — spawns a task (tokio or wasm) +- `russh_util::runtime::JoinHandle` — task join handle +- `russh_util::time::Instant` — time source (tokio or chrono for WASM) +- WASM compatibility: uses `wasm-bindgen-futures` and `chrono` when target is `wasm32` + +### `russh-config` (russh-config/) + +SSH config file parser: +- Parses `~/.ssh/config` format +- Supports `Host` matching with `globset` +- Provides `Stream::tcp_connect()` and `Stream::proxy_command()` for establishing connections based on config + +### `pageant` (pageant/) + +Windows Pageant SSH agent transport: +- `wmmessage` feature: Classic Pageant protocol via Windows messages +- `namedpipes` feature: Modern Pageant protocol via named pipes (PuTTY ≥ 0.75) \ No newline at end of file diff --git a/docs/research/references/ssh/russh/06-usage-patterns.md b/docs/research/references/ssh/russh/06-usage-patterns.md new file mode 100644 index 0000000..b455278 --- /dev/null +++ b/docs/research/references/ssh/russh/06-usage-patterns.md @@ -0,0 +1,413 @@ +# Russh: Usage Patterns & Examples + +This document provides practical usage patterns for both client and server sides of russh. + +## Minimal Client + +The simplest client connects, authenticates, opens a session channel, and runs a command: + +```rust +use std::sync::Arc; +use russh::*; +use russh::keys::*; + +struct Client; + +impl client::Handler for Client { + type Error = russh::Error; + + async fn check_server_key(&mut self, key: &ssh_key::PublicKey) -> Result { + // In production: verify against known_hosts + Ok(true) + } +} + +async fn run_command(host: &str, command: &str) -> Result> { + let config = Arc::new(client::Config::default()); + let mut session = client::connect(config, (host, 22), Client).await?; + + let auth = session.authenticate_publickey( + "user", + PrivateKeyWithHashAlg::new( + Arc::new(keys::load_secret_key("/path/to/key", None)?), + session.best_supported_rsa_hash().await?.flatten(), + ), + ).await?; + + if !auth.success() { + return Err("Auth failed".into()); + } + + let mut channel = session.channel_open_session().await?; + channel.exec(true, command).await?; + + let mut exit_code = 0; + let mut stdout = tokio::io::stdout(); + + loop { + let Some(msg) = channel.wait().await else { break }; + match msg { + ChannelMsg::Data { data } => { stdout.write_all(&data).await?; } + ChannelMsg::ExtendedData { data, ext } => { + // ext == 1 is stderr + eprint!("{}", String::from_utf8_lossy(&data)); + } + ChannelMsg::ExitStatus { exit_status } => { exit_code = exit_status; } + _ => {} + } + } + + session.disconnect(Disconnect::ByApplication, "", "en").await?; + Ok(exit_code) +} +``` + +## Interactive PTY Client + +For interactive sessions (shell access), request a PTY and handle window changes: + +```rust +async fn interactive_session(session: &client::Handle) -> Result, russh::Error> { + let mut channel = session.channel_open_session().await?; + + // Request PTY + channel.request_pty( + false, // want_reply + "xterm-256bit", // term type + 80, 24, // cols, rows + 0, 0, // pixel width/height (0 = not specified) + &[], // terminal modes + ).await?; + + // Request shell + channel.request_shell(true).await?; + + Ok(channel) +} +``` + +## Server Implementation + +A basic server that accepts all public keys and echoes input: + +```rust +use std::sync::Arc; +use russh::server::{self, Server as _, Session}; +use russh::*; + +#[derive(Clone)] +struct App; + +impl server::Server for App { + type Handler = ClientHandler; + fn new_client(&mut self, _: Option) -> ClientHandler { + ClientHandler + } +} + +struct ClientHandler; + +impl server::Handler for ClientHandler { + type Error = russh::Error; + + async fn auth_publickey(&mut self, _: &str, _: &ssh_key::PublicKey) -> Result { + Ok(server::Auth::Accept) + } + + async fn channel_open_session( + &mut self, + channel: Channel, + session: &mut Session, + ) -> Result { + Ok(true) + } + + async fn data( + &mut self, + channel: ChannelId, + data: &[u8], + session: &mut Session, + ) -> Result<(), Self::Error> { + // Echo back + session.data(channel, data.to_vec().into())?; + Ok(()) + } + + async fn exec_request( + &mut self, + channel: ChannelId, + data: &[u8], + session: &mut Session, + ) -> Result<(), Self::Error> { + // Handle exec request + session.channel_success(channel); + // Process the command... + session.channel_failure(channel); + Ok(()) + } +} + +async fn run_server() { + let config = Arc::new(server::Config { + keys: vec![russh::keys::PrivateKey::random(&mut rand::rng(), russh::keys::Algorithm::Ed25519).unwrap()], + ..Default::default() + }); + + let mut app = App; + let socket = tokio::net::TcpListener::bind(("0.0.0.0", 22)).await.unwrap(); + app.run_on_socket(config, &socket).await.unwrap(); +} +``` + +## Authentication with SSH Agent + +```rust +use russh::keys::agent::client::AgentClient; + +async fn auth_with_agent( + session: &mut client::Handle, + user: &str, +) -> Result> { + // Connect to SSH agent (Unix socket or Windows Pageant) + let stream = tokio::net::UnixStream::connect(std::env::var("SSH_AUTH_SOCK")?).await?; + let mut agent = AgentClient::connect(stream); + + // List available identities + let identities = agent.request_identities().await?; + + // Try each identity + for identity in &identities { + let result = session.authenticate_publickey_with( + user, + identity.public_key(), + None, // hash_alg (None for Ed25519) + &mut agent, + ).await?; + + if result.success() { + return Ok(result); + } + } + + Err("No matching key found in agent".into()) +} +``` + +## Port Forwarding + +### Local TCP Forwarding + +```rust +async fn local_forward( + session: &client::Handle, + target_host: &str, + target_port: u16, +) -> Result, russh::Error> { + let channel = session.channel_open_direct_tcpip( + target_host, + target_port as u32, + "127.0.0.1", // originator + 0, // originator port + ).await?; + Ok(channel) +} +``` + +### Remote TCP Forwarding + +```rust +async fn remote_forward( + session: &client::Handle, + listen_addr: &str, + listen_port: u32, +) -> Result { + // Request server to listen + let assigned_port = session.tcpip_forward(listen_addr, listen_port).await?; + Ok(assigned_port) +} +``` + +## Channel as AsyncRead/AsyncWrite + +Channels can be converted to `AsyncRead` + `AsyncWrite` streams: + +```rust +async fn use_channel_as_stream(channel: Channel) { + // Convert to bidirectional stream + let stream = channel.into_stream(); + + // Or split into read/write halves + let (read_half, write_half) = channel.split(); + let mut reader = read_half.make_reader(); + let writer = write_half.make_writer(); + + // Use with any tokio IO utility + let mut buf = vec![0u8; 1024]; + use tokio::io::AsyncReadExt; + let n = reader.read(&mut buf).await.unwrap(); + + // Read stderr separately + let stderr_reader = read_half.make_reader_ext(Some(1)); // ext code 1 = stderr +} +``` + +## Custom Algorithm Preferences + +```rust +use std::borrow::Cow; + +let config = client::Config { + preferred: Preferred { + kex: Cow::Owned(vec![ + russh::kex::CURVE25519, + russh::kex::EXTENSION_SUPPORT_AS_CLIENT, + ]), + cipher: Cow::Owned(vec![ + russh::cipher::CHACHA20_POLY1305, + russh::cipher::AES_256_GCM, + ]), + mac: Cow::Owned(vec![ + russh::mac::HMAC_SHA256_ETM, + ]), + ..Default::default() + }, + ..Default::default() +}; +``` + +## Rekeying + +```rust +// Automatic rekeying happens based on Limits (default 1GB / 1 hour) +// Explicit rekeying: +session.rekey_soon().await?; + +// Access shared secret after kex (for custom key derivation): +struct MyHandler; +impl client::Handler for MyHandler { + type Error = russh::Error; + + async fn kex_done( + &mut self, + shared_secret: Option<&[u8]>, + names: &Names, + session: &mut client::Session, + ) -> Result<(), Self::Error> { + // shared_secret is the raw DH shared secret + // names contains all negotiated algorithms + Ok(()) + } + + async fn check_server_key(&mut self, key: &ssh_key::PublicKey) -> Result { + Ok(true) + } +} +``` + +## Keyboard-Interactive Authentication + +```rust +async fn keyboard_interactive_auth( + session: &mut client::Handle, + user: &str, +) -> Result { + let response = session.authenticate_keyboard_interactive_start(user, None).await?; + + match response { + KeyboardInteractiveAuthResponse::Success => Ok(AuthResult::Success), + KeyboardInteractiveAuthResponse::Failure { .. } => Ok(AuthResult::Failure { remaining_methods: MethodSet::empty(), partial_success: false }), + KeyboardInteractiveAuthResponse::InfoRequest { prompts, .. } => { + // Respond to each prompt + let responses: Vec = prompts.iter() + .map(|p| /* get user input for p.prompt */ String::new()) + .collect(); + + let response = session.authenticate_keyboard_interactive_respond(responses).await?; + // May need to loop if server sends more challenges + // ... + todo!() + } + } +} +``` + +## Server: Handling Channel Requests + +Server-side channel requests (shell, exec, pty, etc.) must be explicitly accepted or rejected: + +```rust +impl server::Handler for MyHandler { + type Error = russh::Error; + + async fn shell_request( + &mut self, + channel: ChannelId, + session: &mut Session, + ) -> Result<(), Self::Error> { + // Must call success or failure! + session.channel_success(channel); + Ok(()) + } + + async fn exec_request( + &mut self, + channel: ChannelId, + data: &[u8], + session: &mut Session, + ) -> Result<(), Self::Error> { + let command = String::from_utf8_lossy(data); + if is_allowed(&command) { + session.channel_success(channel); + // Process command, send data back... + session.data(channel, output.into())?; + session.eof(channel)?; + } else { + session.channel_failure(channel); + } + Ok(()) + } + + async fn pty_request( + &mut self, + channel: ChannelId, + term: &str, + col_width: u32, + row_height: u32, + pix_width: u32, + pix_height: u32, + modes: &[(Pty, u32)], + session: &mut Session, + ) -> Result<(), Self::Error> { + session.channel_success(channel); + Ok(()) + } +} +``` + +## Known Hosts Verification + +```rust +use russh::keys::check_known_hosts; + +impl client::Handler for SecureClient { + type Error = russh::Error; + + async fn check_server_key(&mut self, key: &ssh_key::PublicKey) -> Result { + match check_known_hosts("example.com", 22, key) { + Ok(()) => Ok(true), + Err(e) => { + if matches!(e, russh::keys::Error::KeyChanged { .. }) { + // Possible MITM attack! + eprintln!("WARNING: Host key changed! Possible MITM attack."); + Ok(false) + } else { + // Unknown host - prompt user to verify fingerprint + eprintln!("Unknown host key. Fingerprint: {}", key.fingerprint(ssh_key::HashAlg::Sha256)); + // In a real app, ask the user + Ok(false) + } + } + } + } +} +``` \ No newline at end of file diff --git a/docs/research/references/ssh/sftp-rs/01-overview-and-architecture.md b/docs/research/references/ssh/sftp-rs/01-overview-and-architecture.md new file mode 100644 index 0000000..16bbab5 --- /dev/null +++ b/docs/research/references/ssh/sftp-rs/01-overview-and-architecture.md @@ -0,0 +1,125 @@ +# sftp-rs: Overview and Architecture + +**Version**: 0.3.0 +**Repository**: https://github.com/jelmer/sftp-rs +**License**: Apache-2.0 +**Rust Edition**: 2021 + +## What It Is + +`sftp-rs` is a Rust crate implementing an **SFTP client** — the SSH File Transfer Protocol as defined in [draft-ietf-secsh-filexfer-02](https://datatracker.ietf.org/doc/html/draft-ietf-secsh-filexfer-02). It provides a pure wire-protocol codec plus both synchronous and asynchronous client implementations that layer on top of any transport providing `Read + Write` (sync) or `AsyncRead + AsyncWrite` (async) byte streams. + +Core design decisions: +- **Transport-agnostic** — does not include an SSH implementation itself; operates on top of an already-established SSH channel or any byte stream +- **Protocol v3** — targets SFTP protocol version 3, following the published draft but deviating where other servers and clients ignore the RFC +- **Pure codec** — the `protocol` module contains zero I/O; it builds and parses raw bytes, shared by both sync and async clients +- **Two concurrency models** — a synchronous `SftpClient` and an async `AsyncSftpClient` with a background reader task for concurrent pipelining + +## Source Layout + +``` +sftp-rs/src/ +├── lib.rs # Crate root, re-exports, feature gating +├── protocol.rs # Pure wire-protocol codec: types, builders, parsers +├── sync.rs # Synchronous SftpClient +├── async.rs # AsyncSftpClient with background reader +├── russh.rs # russh transport glue (optional, feature-gated) +└── bin/ + └── sftp.rs # CLI interactive sftp client binary +``` + +## Feature Flags + +| Feature | Default | Dependencies | Description | +|---------|---------|-------------|-------------| +| `default` | ✅ | `bin` | Includes the CLI binary | +| `bin` | ✅ (via default) | `rustyline`, `shell-words` | Interactive CLI binary | +| `ssh2` | ❌ | `ssh2` | Integration with the `ssh2` crate (libssh2 bindings) | +| `async` | ❌ | `tokio` | Async client (`AsyncSftpClient`) | +| `russh` | ❌ | `russh`, `async`, `tokio` | russh transport integration | + +The `russh` feature implies `async` (it requires `tokio` and the async client). + +## Key Dependencies + +| Dependency | Version | Purpose | +|------------|---------|---------| +| `byteorder` | 1 | Big-endian binary read/write for wire protocol | +| `russh` | 0.61 (optional) | Pure-Rust SSH implementation, provides channel + stream | +| `ssh2` | 0.9 (optional) | libssh2 bindings, provides `Channel` | +| `tokio` | 1 (optional) | Async runtime, `AsyncRead`/`AsyncWrite`, `io::split` | +| `rustyline` | 18 (optional) | Readline library for CLI binary | +| `shell-words` | 1 (optional) | Shell-style token parsing for CLI | + +## Architecture Diagram + +``` +┌───────────────────────────────────────────────────────────────┐ +│ Application Layer │ +│ │ +│ ┌─────────────────┐ ┌──────────────────┐ ┌──────────────┐ │ +│ │ CLI Binary │ │ SftpClient │ │ AsyncSftp- │ │ +│ │ (bin/sftp) │ │ (sync.rs) │ │ Client │ │ +│ │ │ │ │ │ (async.rs) │ │ +│ │ SshChannel → │ │ C: Read+Write │ │ W: AsyncW │ │ +│ │ SftpClient │ │ │ │ │ │ +│ └────────┬─────────┘ └────────┬─────────┘ └──────┬──────┘ │ +│ │ │ │ │ +│ └──────────────────────┴────────────────────┘ │ +│ │ │ +│ ┌─────────────┴─────────────┐ │ +│ │ protocol.rs (codec) │ │ +│ │ │ │ +│ │ • Error, Result │ │ +│ │ • Attributes (serde) │ │ +│ │ • OpenOptions │ │ +│ │ • build_*() builders │ │ +│ │ • parse_*() / expect_*() │ │ +│ │ • read/write_raw_packet │ │ +│ │ • with/split_request_id │ │ +│ └────────────────────────────┘ │ +│ │ │ +└──────────────────────────────────┼──────────────────────────────┘ + │ + ┌────────────────────┼────────────────────┐ + │ │ │ + ┌─────────┴──────┐ ┌─────────┴───────┐ ┌─────────┴──────┐ + │ ssh subprocess│ │ russh Channel │ │ ssh2 Channel │ + │ (stdin/stdout)│ │ (via Channel- │ │ (libssh2) │ + │ │ │ Stream) │ │ │ + └────────────────┘ └────────────────┘ └────────────────┘ +``` + +## Protocol Version Negotiation + +Both clients perform the same handshake on construction: + +1. **Client sends `SSH_FXP_INIT`** with version `3` +2. **Server responds `SSH_FXP_VERSION`** with its version and optional extensions +3. If server version ≠ 3, the constructor returns an error + +```rust +// From protocol.rs +pub fn build_init() -> Vec { + let mut buf = Vec::with_capacity(4); + buf.write_u32::(3).unwrap(); // version = 3 + buf +} +``` + +The handshake is the only unnumbered (no request-id) exchange. All subsequent requests include a 4-byte request-id used for demultiplexing responses. + +## Re-exports from `lib.rs` + +The crate root re-exports the most commonly used types: + +```rust +pub use protocol::{ + Attributes, Directory, Error, File, Kind, OpenOptions, Result, TextHint, + SSH_FILEXFER_ATTR_ACCESSTIME, SSH_FILEXFER_ATTR_ACL, /* ... all flag constants */ +}; +pub use sync::SftpClient; + +#[cfg(feature = "async")] +pub use r#async::AsyncSftpClient; +``` \ No newline at end of file diff --git a/docs/research/references/ssh/sftp-rs/02-wire-protocol-codec.md b/docs/research/references/ssh/sftp-rs/02-wire-protocol-codec.md new file mode 100644 index 0000000..07a5645 --- /dev/null +++ b/docs/research/references/ssh/sftp-rs/02-wire-protocol-codec.md @@ -0,0 +1,276 @@ +# sftp-rs: Wire Protocol Codec (`protocol.rs`) + +The `protocol` module is the heart of the crate — a pure, I/O-free codec that encodes and decodes SFTP wire messages. Both the synchronous and asynchronous clients delegate all serialization and parsing to these functions. + +## Packet Framing + +Every SFTP message on the wire has this layout: + +``` +┌────────────┬──────────┬──────────────────┐ +│ length │ type │ data │ +│ (4 bytes │ (1 byte │ (length-1 bytes) │ +│ BE u32) │ │ │ +└────────────┴──────────┴──────────────────┘ +``` + +For all numbered requests/responses (everything after INIT/VERSION), the `data` field begins with a 4-byte big-endian `request-id`: + +``` +data = [request_id (4 bytes BE u32)] [payload] +``` + +### Raw Packet I/O + +```rust +// Sync I/O (used by SftpClient) +pub fn read_raw_packet(channel: &mut C) -> std::io::Result<(u8, Vec)> +pub fn write_raw_packet(channel: &mut C, kind: u8, buf: &[u8]) -> std::io::Result<()> + +// Async I/O (used by AsyncSftpClient) +async fn read_packet_async(r: &mut R) -> std::io::Result<(u8, Vec)> +async fn write_packet_async(w: &mut W, kind: u8, body: &[u8]) -> std::io::Result<()> +``` + +Both return/accept `(kind: u8, body: Vec)` where `kind` is the SFTP message type byte and `body` is everything after it (including the request-id for numbered messages). + +### Request-ID Helpers + +```rust +// Prepend a 4-byte request-id to a request body +pub fn with_request_id(request_id: u32, body: &[u8]) -> Vec + +// Strip the 4-byte request-id prefix from a response body +pub fn split_request_id(buf: &[u8]) -> std::io::Result<(u32, &[u8])> +``` + +## Message Type Constants + +### Client → Server (Requests) + +| Constant | Value | Description | +|----------|-------|-------------| +| `SSH_FXP_INIT` | 1 | Protocol version negotiation (unnumbered) | +| `SSH_FXP_OPEN` | 3 | Open a file | +| `SSH_FXP_CLOSE` | 4 | Close a handle | +| `SSH_FXP_READ` | 5 | Read from a file | +| `SSH_FXP_WRITE` | 6 | Write to a file | +| `SSH_FXP_LSTAT` | 7 | Get file attributes (don't follow symlinks) | +| `SSH_FXP_FSTAT` | 8 | Get file attributes by handle | +| `SSH_FXP_SETSTAT` | 9 | Set file attributes by path | +| `SSH_FXP_FSETSTAT` | 10 | Set file attributes by handle | +| `SSH_FXP_OPENDIR` | 11 | Open a directory for listing | +| `SSH_FXP_READDIR` | 12 | Read directory entries | +| `SSH_FXP_REMOVE` | 13 | Remove a file | +| `SSH_FXP_MKDIR` | 14 | Create a directory | +| `SSH_FXP_RMDIR` | 15 | Remove a directory | +| `SSH_FXP_REALPATH` | 16 | Canonicalize a path | +| `SSH_FXP_STAT` | 17 | Get file attributes (follow symlinks) | +| `SSH_FXP_RENAME` | 18 | Rename a file/directory | +| `SSH_FXP_READLINK` | 19 | Read the target of a symlink | +| `SSH_FXP_SYMLINK` | 20 | Create a symbolic link | +| `SSH_FXP_LINK` | 21 | Create a hard link | +| `SSH_FXP_BLOCK` | 22 | Byte-range lock | +| `SSH_FXP_UNBLOCK` | 23 | Byte-range unlock | +| `SSH_FXP_EXTENDED` | 200 | Vendor-specific extension request | + +### Server → Client (Responses) + +| Constant | Value | Description | +|----------|-------|-------------| +| `SSH_FXP_VERSION` | 2 | Version reply (unnumbered) | +| `SSH_FXP_STATUS` | 101 | Status response (success or error) | +| `SSH_FXP_HANDLE` | 102 | Returns a file/directory handle | +| `SSH_FXP_DATA` | 103 | Returns file data | +| `SSH_FXP_NAME` | 104 | Returns filename entries | +| `SSH_FXP_ATTRS` | 105 | Returns file attributes | +| `SSH_FXP_EXTENDED_REPLY` | 201 | Extension response data | + +## Status Codes + +| Constant | Value | Error Variant | +|----------|-------|---------------| +| `SSH_FX_OK` | 0 | `Ok(())` | +| `SSH_FX_EOF` | 1 | `Eof` | +| `SSH_FX_NO_SUCH_FILE` | 2 | `NoSuchFile` | +| `SSH_FX_PERMISSION_DENIED` | 3 | `PermissionDenied` | +| `SSH_FX_FAILURE` | 4 | `Failure` | +| `SSH_FX_BAD_MESSAGE` | 5 | `BadMessage` | +| `SSH_FX_NO_CONNECTION` | 6 | `NoConnection` | +| `SSH_FX_CONNECTION_LOST` | 7 | `ConnectionLost` | +| `SSH_FX_OP_UNSUPPORTED` | 8 | `OpUnsupported` | +| `SSH_FX_INVALID_HANDLE` | 9 | `InvalidHandle` | +| `SSH_FX_NO_SUCH_PATH` | 10 | `NoSuchPath` | +| `SSH_FX_FILE_ALREADY_EXISTS` | 11 | `FileAlreadyExists` | +| `SSH_FX_WRITE_PROTECT` | 12 | `WriteProtect` | +| `SSH_FX_NO_MEDIA` | 13 | `NoMedia` | +| `SSH_FX_NO_SPACE_ON_FILESYSTEM` | 14 | `NoSpaceOnFilesystem` | +| `SSH_FX_QUOTA_EXCEEDED` | 15 | `QuotaExceeded` | +| `SSH_FX_UNKNOWN_PRINCIPAL` | 16 | `UnknownPrincipal` | +| `SSH_FX_LOCK_CONFLICT` | 17 | `LockConflict` | +| `SSH_FX_DIR_NOT_EMPTY` | 18 | `DirNotEmpty` | +| `SSH_FX_NOT_A_DIRECTORY` | 19 | `NotADirectory` | +| `SSH_FX_INVALID_FILENAME` | 20 | `InvalidFilename` | +| `SSH_FX_LINK_LOOP` | 21 | `LinkLoop` | +| `SSH_FX_CANNOT_DELETE` | 22 | `CannotDelete` | +| `SSH_FX_INVALID_PARAMETER` | 23 | `InvalidParameter` | +| `SSH_FX_FILE_IS_A_DIRECTORY` | 24 | `FileIsADirectory` | +| `SSH_FX_BYTE_RANGE_LOCK_CONFLICT` | 25 | `ByteRangeLockConflict` | +| `SSH_FX_BYTE_RANGE_LOCK_REFUSED` | 26 | `ByteRangeLockRefused` | +| `SSH_FX_DELETE_PENDING` | 27 | `DeletePending` | +| `SSH_FX_FILE_CORRUPT` | 28 | `FileCorrupt` | +| `SSH_FX_OWNER_INVALID` | 29 | `OwnerInvalid` | +| `SSH_FX_GROUP_INVALID` | 30 | `GroupInvalid` | +| `SSH_FX_NO_MATCHING_BYTE_RANGE_LOCK` | 31 | `NoMatchingByteRangeLock` | + +Any unrecognized status code maps to `Error::Other(status, message, lang_tag)`. + +## Request Body Builders + +Each builder produces the `data` portion (without type byte or request-id) for a specific SFTP request: + +### Handshake + +```rust +// INIT: version 3 (no request-id) +pub fn build_init() -> Vec + +// VERSION: parse server response +pub fn parse_version(body: &[u8]) -> std::io::Result<(u32, Vec<(String, String)>)> +``` + +### Single-Field Bodies + +```rust +// Path-only requests: LSTAT, STAT, OPENDIR, REMOVE, MKDIR, RMDIR, READLINK +pub fn build_path_only(path: &str) -> Vec + +// Handle-only requests: CLOSE, READDIR +pub fn build_handle_only(handle: &[u8]) -> Vec +``` + +### Composite Bodies + +```rust +// OPEN: path + flags + attributes +pub fn build_open(path: &str, options: u32, attr: &Attributes) -> std::io::Result> + +// READ: handle + offset + length +pub fn build_pread(handle: &[u8], offset: u64, length: u32) -> Vec + +// WRITE: handle + offset + data +pub fn build_pwrite(handle: &[u8], offset: u64, data: &[u8]) -> Vec + +// RENAME: oldpath + newpath + flags +pub fn build_rename(oldpath: &str, newpath: &str, flags: Option) -> Vec +// Default flags: OVERWRITE | ATOMIC | NATIVE = 0x07 + +// SYMLINK: path + target +pub fn build_two_paths(a: &str, b: &str) -> Vec + +// LINK: path + target + symlink_flag +pub fn build_link(path: &str, target: &str, symlink: bool) -> Vec + +// Path + attributes: SETSTAT, MKDIR +pub fn build_path_and_attrs(path: &str, attr: &Attributes) -> std::io::Result> + +// Handle + attributes: FSETSTAT +pub fn build_handle_and_attrs(handle: &[u8], attr: &Attributes) -> std::io::Result> + +// Path + flags: STAT, LSTAT +pub fn build_path_and_flags(path: &str, flags: u32) -> Vec + +// Handle + flags: FSTAT +pub fn build_handle_and_flags(handle: &[u8], flags: u32) -> Vec + +// REALPATH: path + optional control byte + optional compose path +pub fn build_realpath(path: &str, control_byte: Option, compose: Option<&str>) -> Vec + +// BLOCK: handle + offset + length + lockmask +pub fn build_block(handle: &[u8], offset: u64, length: u64, lockmask: u32) -> Vec + +// UNBLOCK: handle + offset + length +pub fn build_unblock(handle: &[u8], offset: u64, length: u64) -> Vec + +// EXTENDED: request name + data +pub fn build_extended(request: &str, data: &[u8]) -> Vec +``` + +### Wire Encoding Helpers + +```rust +// Write a length-prefixed UTF-8 string (4-byte BE length + bytes) +fn put_str(buf: &mut Vec, s: &str) + +// Write a length-prefixed byte string (4-byte BE length + bytes) +fn put_bytes(buf: &mut Vec, b: &[u8]) + +// Read a length-prefixed UTF-8 string from a cursor +fn read_string(reader: &mut Cursor<&[u8]>, what: &str) -> std::io::Result +``` + +## Response Parsers + +Each parser takes the raw `data` portion (after stripping type byte and request-id) and returns a `Result`: + +```rust +// Parse SSH_FXP_STATUS body → Ok(()) or typed Error +pub fn parse_status(respdata: &[u8]) -> Result<()> + +// Parse SSH_FXP_HANDLE body → raw handle bytes +pub fn parse_handle(respdata: &[u8]) -> Result> + +// Parse SSH_FXP_DATA body → raw data bytes +pub fn parse_data(respdata: &[u8]) -> Result> + +// Parse SSH_FXP_ATTRS body → Attributes +pub fn parse_attrs(respdata: &[u8]) -> Result + +// Parse SSH_FXP_NAME body (for READLINK, REALPATH) → (name, attrs) pairs +pub fn parse_name(respdata: &[u8]) -> Result> + +// Parse SSH_FXP_NAME body (for READDIR) → (name, longname, attrs) triples +pub fn parse_readdir(respdata: &[u8]) -> Result> +``` + +## Response Expectation Functions + +These are the primary entry points used by the client implementations. They take `(cmd, data)` — the type byte and payload from the server — and dispatch to the correct parser, or convert an unexpected `SSH_FXP_STATUS` into the appropriate typed error: + +```rust +// Expect SSH_FXP_STATUS (for operations that return nothing on success) +pub fn expect_status(cmd: u8, data: &[u8]) -> Result<()> + +// Expect SSH_FXP_HANDLE (for OPEN, OPENDIR) +pub fn expect_handle(cmd: u8, data: &[u8]) -> Result> + +// Expect SSH_FXP_ATTRS (for STAT, LSTAT, FSTAT) +pub fn expect_attrs(cmd: u8, data: &[u8]) -> Result + +// Expect SSH_FXP_DATA (for READ) +pub fn expect_data(cmd: u8, data: &[u8]) -> Result> + +// Expect SSH_FXP_NAME with name+attrs (for READLINK, REALPATH) +pub fn expect_name(cmd: u8, data: &[u8]) -> Result> + +// Expect SSH_FXP_NAME with name+longname+attrs (for READDIR) +pub fn expect_readdir(cmd: u8, data: &[u8]) -> Result> + +// Expect SSH_FXP_EXTENDED_REPLY or SSH_FXP_STATUS +pub fn expect_extended(cmd: u8, data: Vec) -> Result>> +``` + +If the server returns a different message type than expected, these functions produce `Error::Io("Unexpected response: ...")`. If the server returns `SSH_FXP_STATUS` where a data-bearing response was expected (even `SSH_FX_OK`), it is treated as a protocol violation and converted to the appropriate typed error. + +## String Encoding + +All strings in SFTP are length-prefixed with a 4-byte big-endian length followed by raw UTF-8 bytes: + +``` +┌───────────────┬──────────────────┐ +│ length (4B) │ UTF-8 bytes │ +│ BE u32 │ │ +└───────────────┴──────────────────┘ +``` + +Byte arrays (handles, ACLs) use the same length-prefix format but are not required to be valid UTF-8. \ No newline at end of file diff --git a/docs/research/references/ssh/sftp-rs/03-key-types.md b/docs/research/references/ssh/sftp-rs/03-key-types.md new file mode 100644 index 0000000..9d51a63 --- /dev/null +++ b/docs/research/references/ssh/sftp-rs/03-key-types.md @@ -0,0 +1,308 @@ +# sftp-rs: Key Types + +## `Error` Enum + +The universal error type for all SFTP operations, covering both I/O failures and SFTP status codes: + +```rust +#[derive(Debug)] +pub enum Error { + Io(std::io::Error), + Utf8(std::str::Utf8Error), + Other(u32, String, String), // (status_code, message, lang_tag) + Eof(String, String), // End of file/directory + NoSuchFile(String, String), + PermissionDenied(String, String), + Failure(String, String), + BadMessage(String, String), + NoConnection(String, String), + ConnectionLost(String, String), + OpUnsupported(String, String), + InvalidHandle(String, String), + NoSuchPath(String, String), + FileAlreadyExists(String, String), + WriteProtect(String, String), + NoMedia(String, String), + NoSpaceOnFilesystem(String, String), + QuotaExceeded(String, String), + UnknownPrincipal(String, String), + LockConflict(String, String), + DirNotEmpty(String, String), + NotADirectory(String, String), + InvalidFilename(String, String), + LinkLoop(String, String), + CannotDelete(String, String), + InvalidParameter(String, String), + FileIsADirectory(String, String), + ByteRangeLockConflict(String, String), + ByteRangeLockRefused(String, String), + DeletePending(String, String), + FileCorrupt(String, String), + OwnerInvalid(String, String), + GroupInvalid(String, String), + NoMatchingByteRangeLock(String, String), +} +``` + +The `String` pairs in each variant are `(error_message, language_tag)` as returned by the server. The `Error` type implements `From` and `From`, and there is a `From for std::io::Error` conversion that maps SFTP error codes to appropriate `std::io::ErrorKind` values: + +| Error Variant | io::ErrorKind | +|---------------|---------------| +| `Eof` | `UnexpectedEof` | +| `NoSuchFile` | `NotFound` | +| `NoSuchPath` | `NotFound` | +| `PermissionDenied` | `PermissionDenied` | +| `WriteProtect` | `PermissionDenied` | +| `QuotaExceeded` | `PermissionDenied` | +| `LockConflict` | `PermissionDenied` | +| `NoConnection` | `NotConnected` | +| `ConnectionLost` | `ConnectionReset` | +| `InvalidHandle` | `InvalidInput` | +| `FileAlreadyExists` | `AlreadyExists` | +| `InvalidFilename` | `InvalidInput` | +| All others | formatted via `Error::other()` | + +```rust +pub type Result = std::result::Result; +``` + +## `Attributes` + +Represents SFTP file attributes — a flag-driven, extensible structure where only fields present in the `valid_attribute_flags` mask are serialized: + +```rust +#[derive(Debug, PartialEq, Eq, Clone, Default)] +pub struct Attributes { + pub size: Option, + pub uid: Option, + pub gid: Option, + pub allocation_size: Option, + pub owner: Option, + pub group: Option, + pub permissions: Option, + pub access_time: Option<(u64, Option)>, // (seconds, nanoseconds) + pub create_time: Option<(u64, Option)>, + pub modify_time: Option<(u64, Option)>, + pub ctime: Option<(u64, Option)>, + pub acl: Option>, + pub attrib_bits: Option, + pub attrib_bits_valid: Option, + pub text_hint: Option, + pub mime_type: Option, + pub link_count: Option, + pub untranslated_name: Option>, + pub extended: Option>, +} +``` + +### Serialization + +The `serialize()` method builds the wire format by writing a 4-byte `valid_attribute_flags` placeholder, then appending fields conditionally, then back-patching the flags: + +``` +┌──────────────────────────┐ +│ valid_attribute_flags │ 4 bytes, BE u32 +│ (back-patched at end) │ +├──────────────────────────┤ +│ size (if flag set) │ 8 bytes, BE u64 +│ uid (if flag set) │ 4 bytes, BE u32 +│ gid (if flag set) │ 4 bytes, BE u32 +│ allocation_size │ 8 bytes, BE u64 +│ owner (length-prefixed) │ +│ group (length-prefixed) │ +│ permissions │ 4 bytes, BE u32 +│ access_time │ 8 bytes + opt 4 bytes ns +│ create_time │ 8 bytes + opt 4 bytes ns +│ modify_time │ 8 bytes + opt 4 bytes ns +│ ctime │ 8 bytes + opt 4 bytes ns +│ acl (length-prefixed) │ +│ attrib_bits │ 4 bytes +│ attrib_bits_valid │ 4 bytes +│ text_hint │ 1 byte +│ mime_type (len-prefixed)│ +│ link_count │ 4 bytes +│ untranslated_name │ length-prefixed bytes +│ extended │ count + key/value pairs +└──────────────────────────┘ +``` + +Constraints enforced by serialization: +- `uid` and `gid` must both be present or both absent (same `SSH_FILEXFER_ATTR_UIDGID` flag) +- `owner` and `group` share the `SSH_FILEXFER_ATTR_OWNERGROUP` flag +- `attrib_bits` and `attrib_bits_valid` share the `SSH_FILEXFER_ATTR_BITS` flag +- `SSH_FILEXFER_ATTR_SUBSECOND_TIMES` is a shared flag — if set, all time fields include a 4-byte nanosecond component; if not, none do + +### Deserialization + +`deserialize(reader: &mut Cursor<&[u8]>)` reads the flags first, then conditionally reads each field based on flag bits. Subsecond nanoseconds are read for all time fields when `SSH_FILEXFER_ATTR_SUBSECOND_TIMES` is set. + +## Attribute Flag Constants + +| Constant | Value | Field(s) | +|----------|-------|----------| +| `SSH_FILEXFER_ATTR_SIZE` | 0x00000001 | `size` | +| `SSH_FILEXFER_ATTR_UIDGID` | 0x00000002 | `uid`, `gid` | +| `SSH_FILEXFER_ATTR_PERMISSIONS` | 0x00000004 | `permissions` | +| `SSH_FILEXFER_ATTR_ACCESSTIME` | 0x00000008 | `access_time` | +| `SSH_FILEXFER_ATTR_CREATETIME` | 0x00000010 | `create_time` | +| `SSH_FILEXFER_ATTR_MODIFYTIME` | 0x00000020 | `modify_time` | +| `SSH_FILEXFER_ATTR_ACL` | 0x00000040 | `acl` | +| `SSH_FILEXFER_ATTR_OWNERGROUP` | 0x00000080 | `owner`, `group` | +| `SSH_FILEXFER_ATTR_SUBSECOND_TIMES` | 0x00000100 | nanoseconds for all times | +| `SSH_FILEXFER_ATTR_BITS` | 0x00000200 | `attrib_bits`, `attrib_bits_valid` | +| `SSH_FILEXFER_ATTR_ALLOCATION_SIZE` | 0x00000400 | `allocation_size` | +| `SSH_FILEXFER_ATTR_TEXT_HINT` | 0x00000800 | `text_hint` | +| `SSH_FILEXFER_ATTR_MIME_TYPE` | 0x00001000 | `mime_type` | +| `SSH_FILEXFER_ATTR_LINK_COUNT` | 0x00002000 | `link_count` | +| `SSH_FILEXFER_ATTR_UNTRANSLATED_NAME` | 0x00004000 | `untranslated_name` | +| `SSH_FILEXFER_ATTR_CTIME` | 0x00008000 | `ctime` | +| `SSH_FILEXFER_ATTR_EXTENDED` | 0x80000000 | `extended` | + +## `Kind` — File Type + +Represents the type of a filesystem entry, encoded as a `u8` in the attributes: + +```rust +#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)] +pub enum Kind { + Regular, // 1 + Directory, // 2 + Symlink, // 3 + Special, // 4 + #[default] + Unknown, // 5 + Socket, // 6 + CharDevice, // 7 + BlockDevice, // 8 + Fifo, // 9 +} +``` + +Implements `From for u8` and `From for Kind` (panics on unknown values). + +## `TextHint` + +Indicates whether a file is text or binary, and whether that classification is known or guessed: + +```rust +#[derive(Clone, Copy, PartialEq, Eq, Debug)] +pub enum TextHint { + KnownText, // 0x00 + GuessedText, // 0x01 + KnownBinary, // 0x02 + GuessedBinary, // 0x03 +} +``` + +## `OpenOptions` + +Builder-style type for controlling file open behavior: + +```rust +#[derive(Debug, PartialEq, Eq, Clone, Copy, Default)] +pub struct OpenOptions(u32); + +impl OpenOptions { + pub fn new() -> OpenOptions; + pub fn read(mut self, read: bool) -> OpenOptions; + pub fn write(mut self, write: bool) -> OpenOptions; + pub fn append(mut self, append: bool) -> OpenOptions; + pub fn create(mut self, create: bool) -> OpenOptions; + pub fn truncate(mut self, truncate: bool) -> OpenOptions; + pub fn excl(mut self, excl: bool) -> OpenOptions; + pub fn mode(&mut self, mode: u32) -> &mut OpenOptions; + pub fn get(&self) -> u32; +} +``` + +### Open Flag Constants + +| Constant | Value | Description | +|----------|-------|-------------| +| `SFTP_FLAG_READ` | 0x01 | Read access | +| `SFTP_FLAG_WRITE` | 0x02 | Write access | +| `SFTP_FLAG_APPEND` | 0x04 | Append data | +| `SFTP_FLAG_CREAT` | 0x08 | Create if not exists | +| `SFTP_FLAG_TRUNC` | 0x10 | Truncate to zero length | +| `SFTP_FLAG_EXCL` | 0x20 | Fail if file exists (exclusive create) | + +Usage example: +```rust +let opts = OpenOptions::new() + .read(true) + .write(true) + .create(true) + .truncate(true); +// opts.get() == 0x1B (READ | WRITE | CREAT | TRUNC) +``` + +## `File` and `Directory` — Handle Wrappers + +Opaque wrappers around the raw handle bytes returned by the server: + +```rust +#[derive(Debug, Clone)] +pub struct File(pub Vec); + +#[derive(Debug, Clone)] +pub struct Directory(pub Vec); +``` + +These are newtype wrappers that distinguish file handles from directory handles at the type level. The inner `Vec` is the server-assigned handle value (obtained from `SSH_FXP_HANDLE` responses), used in subsequent operations like `pread`, `pwrite`, `fclose`, `readdir`, `closedir`, etc. + +## Rename Flags + +Used with `SSH_FXP_RENAME`: + +| Constant | Value | Description | +|----------|-------|-------------| +| `SSH_FXF_RENAME_OVERWRITE` | 0x01 | Overwrite existing target | +| `SSH_FXF_RENAME_ATOMIC` | 0x02 | Atomic rename | +| `SSH_FXF_RENAME_NATIVE` | 0x04 | Use native OS rename semantics | + +Default for `build_rename()` when `flags` is `None`: `OVERWRITE | ATOMIC | NATIVE` = 0x07. + +## Attribute Bits Flags + +Used in `Attributes::attrib_bits`: + +| Constant | Value | +|----------|-------| +| `SSH_FILEXFER_ATTR_FLAGS_READONLY` | 0x00000001 | +| `SSH_FILEXFER_ATTR_FLAGS_SYSTEM` | 0x00000002 | +| `SSH_FILEXFER_ATTR_FLAGS_HIDDEN` | 0x00000004 | +| `SSH_FILEXFER_ATTR_FLAGS_CASE_INSENSITIVE` | 0x00000008 | +| `SSH_FILEXFER_ATTR_FLAGS_ARCHIVE` | 0x00000010 | +| `SSH_FILEXFER_ATTR_FLAGS_ENCRYPTED` | 0x00000020 | +| `SSH_FILEXFER_ATTR_FLAGS_COMPRESSED` | 0x00000040 | +| `SSH_FILEXFER_ATTR_FLAGS_SPARSE` | 0x00000080 | +| `SSH_FILEXFER_ATTR_FLAGS_APPEND_ONLY` | 0x00000100 | +| `SSH_FILEXFER_ATTR_FLAGS_IMMUTABLE` | 0x00000200 | +| `SSH_FILEXFER_ATTR_FLAGS_SYNC` | 0x00000400 | +| `SSH_FILEXFER_ATTR_FLAGS_TRANSLATION_ERR` | 0x00000800 | + +## ACE/MISC Open Flags (v5+ extensions) + +These are defined for completeness but the crate targets v3: + +| Constant | Value | +|----------|-------| +| `SSH_FXF_ACCESS_DISPOSITION` | 0x00000007 | +| `SSH_FXF_CREATE_NEW` | 0x00000000 | +| `SSH_FXF_CREATE_TRUNCATE` | 0x00000001 | +| `SSH_FXF_OPEN_EXISTING` | 0x00000002 | +| `SSH_FXF_OPEN_OR_CREATE` | 0x00000003 | +| `SSH_FXF_TRUNCATE_EXISTING` | 0x00000004 | +| `SSH_FXF_APPEND_DATA` | 0x00000008 | +| `SSH_FXF_APPEND_DATA_ATOMIC` | 0x00000010 | +| `SSH_FXF_TEXT_MODE` | 0x00000020 | +| `SSH_FXF_BLOCK_READ` | 0x00000040 | +| `SSH_FXF_BLOCK_WRITE` | 0x00000080 | +| `SSH_FXF_BLOCK_DELETE` | 0x00000100 | +| `SSH_FXF_BLOCK_ADVISORY` | 0x00000200 | +| `SSH_FXF_NOFOLLOW` | 0x00000400 | +| `SSH_FXF_DELETE_ON_CLOSE` | 0x00000800 | +| `SSH_FXF_ACCESS_AUDIT_ALARM_INFO` | 0x00001000 | +| `SSH_FXF_ACCESS_BACKUP` | 0x00002000 | +| `SSH_FXF_BACKUP_STREAM` | 0x00004000 | +| `SSH_FXF_OVERRIDE_OWNER` | 0x00008000 | \ No newline at end of file diff --git a/docs/research/references/ssh/sftp-rs/04-sync-client.md b/docs/research/references/ssh/sftp-rs/04-sync-client.md new file mode 100644 index 0000000..a87942c --- /dev/null +++ b/docs/research/references/ssh/sftp-rs/04-sync-client.md @@ -0,0 +1,185 @@ +# sftp-rs: Synchronous Client (`sync.rs`) + +## `SftpClient` + +A synchronous SFTP client parameterized over any `Read + Write` channel: + +```rust +pub struct SftpClient { + channel: Mutex, + last_request_id: std::sync::atomic::AtomicU32, + version: u32, + extensions: Vec<(String, String)>, +} +``` + +The `Mutex` ensures the channel is accessed by one thread at a time — each `process()` call writes a request, then reads the response, atomically. The `AtomicU32` for request IDs allows the client to be shared across threads via `&self` (all methods take `&self`, not `&mut self`). + +## Construction + +```rust +impl SftpClient { + pub fn new(mut channel: C) -> std::io::Result +} +``` + +The constructor performs the SFTP handshake inline: +1. Writes `SSH_FXP_INIT` with version 3 +2. Flushes the channel +3. Reads the response, expecting `SSH_FXP_VERSION` +4. Parses version and extensions +5. Returns the client if version == 3 + +### Platform-Specific Construction + +```rust +impl SftpClient { + #[cfg(unix)] + pub fn from_fd(fd: i32) -> std::io::Result + + #[cfg(windows)] + pub fn from_handle(handle: RawHandle) -> std::io::Result +} +``` + +These wrap an OS file descriptor or handle into a `std::fs::File`, then delegate to `new()`. This is the typical way to connect to an SSH subprocess that has the SFTP subsystem active on its stdin/stdout. + +### ssh2 Integration + +```rust +#[cfg(feature = "ssh2")] +impl TryFrom for SftpClient { + type Error = std::io::Error; + fn try_from(mut channel: ssh2::Channel) -> std::result::Result +} +``` + +Requests the `sftp` subsystem on the libssh2 channel, then delegates to `SftpClient::new()`. + +## Request-Response Cycle: `process()` + +The core internal method that drives all operations: + +```rust +fn process(&self, cmd: u8, body: &[u8]) -> std::io::Result<(u8, Vec)> +``` + +1. Allocate a new `request_id` via `AtomicU32::fetch_add(1, SeqCst)` +2. Prepend the request-id to the body: `with_request_id(request_id, body)` +3. Lock the channel mutex +4. Write the packet: `write_raw_packet(channel, cmd, &body_with_id)` +5. Read the response: `read_raw_packet(channel)` +6. Strip the request-id from the response: `split_request_id(&buf)` +7. Assert the response request-id matches the sent one +8. Return `(response_cmd, payload)` + +**Important limitation**: Because `process()` sends then immediately reads, the sync client cannot pipeline requests. Each request blocks until its response arrives. Multiple concurrent operations require separate connections. + +## Public API + +All methods take `&self` and return `Result`: + +### Directory Operations + +```rust +pub fn mkdir(&self, path: &str, attr: &Attributes) -> Result<()> +pub fn rmdir(&self, path: &str) -> Result<()> +pub fn opendir(&self, path: &str) -> Result +pub fn readdir(&self, dir: &Directory) -> Result> +pub fn closedir(&self, dir: &Directory) -> Result<()> +``` + +`readdir()` returns `(filename, longname, attributes)` triples. The `longname` is a human-readable string (like `ls -l` output) provided by the server in v3. Callers should loop on `readdir()` until it returns `Error::Eof` to exhaust all directory entries. + +### File Operations + +```rust +pub fn open(&self, path: &str, options: OpenOptions, attr: &Attributes) -> Result +pub fn pread(&self, file: &File, offset: u64, length: u32) -> Result> +pub fn pwrite(&self, file: &File, offset: u64, data: &[u8]) -> Result<()> +pub fn fclose(&self, file: &File) -> Result<()> +``` + +File I/O is positional (`pread`/`pwrite`) — there is no implicit cursor. The caller tracks the offset. + +### Attribute Operations + +```rust +pub fn stat(&self, path: &str, flags: Option) -> Result // follows symlinks +pub fn lstat(&self, path: &str, flags: Option) -> Result // doesn't follow +pub fn fstat(&self, file: &File, flags: Option) -> Result +pub fn setstat(&self, path: &str, attr: &Attributes) -> Result<()> +pub fn fsetstat(&self, file: &File, attr: &Attributes) -> Result<()> +``` + +### Path Operations + +```rust +pub fn realpath(&self, path: &str, control_byte: Option, compose_path: Option<&str>) -> Result +pub fn readlink(&self, path: &str) -> Result +pub fn remove(&self, path: &str) -> Result<()> +pub fn rename(&self, oldpath: &str, newpath: &str, flags: Option) -> Result<()> +``` + +### Link Operations + +```rust +pub fn symlink(&self, path: &str, target: &str) -> Result<()> +pub fn hardlink(&self, path: &str, target: &str) -> Result<()> +pub fn link(&self, path: &str, target: &str, symlink: bool) -> Result<()> +``` + +`symlink()` uses `SSH_FXP_SYMLINK` (v3). `link()` and `hardlink()` use `SSH_FXP_LINK` (v5+ extension). + +### Lock Operations + +```rust +pub fn block(&self, file: &File, offset: u64, length: u64, lockmask: u32) -> Result<()> +pub fn unblock(&self, file: &File, offset: u64, length: u64) -> Result<()> +``` + +### Extended Operations + +```rust +pub fn extended(&self, request: &str, data: &[u8]) -> Result>> +pub fn flineseek(&self, file: &File, lineno: u64) -> Result<()> +``` + +`flineseek()` sends the `text-seek` extended request with the handle and line number. `extended()` returns `Some(data)` if the server sends `SSH_FXP_EXTENDED_REPLY`, or `None` if it sends `SSH_FXP_STATUS` with `SSH_FX_OK`. + +## Introspection + +```rust +pub fn version(&self) -> u32 +pub fn extensions(&self) -> &[(String, String)] +``` + +Returns the negotiated protocol version and any server-advertised extensions from the handshake. + +## Request-Response Mapping + +| Method | Request Type | Expected Response | +|--------|-------------|-------------------| +| `open` | `SSH_FXP_OPEN` | `SSH_FXP_HANDLE` | +| `opendir` | `SSH_FXP_OPENDIR` | `SSH_FXP_HANDLE` | +| `pread` | `SSH_FXP_READ` | `SSH_FXP_DATA` | +| `pwrite` | `SSH_FXP_WRITE` | `SSH_FXP_STATUS` | +| `fclose` | `SSH_FXP_CLOSE` | `SSH_FXP_STATUS` | +| `stat` | `SSH_FXP_STAT` | `SSH_FXP_ATTRS` | +| `lstat` | `SSH_FXP_LSTAT` | `SSH_FXP_ATTRS` | +| `fstat` | `SSH_FXP_FSTAT` | `SSH_FXP_ATTRS` | +| `setstat` | `SSH_FXP_SETSTAT` | `SSH_FXP_STATUS` | +| `fsetstat` | `SSH_FXP_FSETSTAT` | `SSH_FXP_STATUS` | +| `mkdir` | `SSH_FXP_MKDIR` | `SSH_FXP_STATUS` | +| `rmdir` | `SSH_FXP_RMDIR` | `SSH_FXP_STATUS` | +| `remove` | `SSH_FXP_REMOVE` | `SSH_FXP_STATUS` | +| `rename` | `SSH_FXP_RENAME` | `SSH_FXP_STATUS` | +| `readdir` | `SSH_FXP_READDIR` | `SSH_FXP_NAME` | +| `closedir` | `SSH_FXP_CLOSE` | `SSH_FXP_STATUS` | +| `realpath` | `SSH_FXP_REALPATH` | `SSH_FXP_NAME` | +| `readlink` | `SSH_FXP_READLINK` | `SSH_FXP_NAME` | +| `symlink` | `SSH_FXP_SYMLINK` | `SSH_FXP_STATUS` | +| `link` | `SSH_FXP_LINK` | `SSH_FXP_STATUS` | +| `block` | `SSH_FXP_BLOCK` | `SSH_FXP_STATUS` | +| `unblock` | `SSH_FXP_UNBLOCK` | `SSH_FXP_STATUS` | +| `extended` | `SSH_FXP_EXTENDED` | `SSH_FXP_EXTENDED_REPLY` or `SSH_FXP_STATUS` | \ No newline at end of file diff --git a/docs/research/references/ssh/sftp-rs/05-async-client.md b/docs/research/references/ssh/sftp-rs/05-async-client.md new file mode 100644 index 0000000..0ce519e --- /dev/null +++ b/docs/research/references/ssh/sftp-rs/05-async-client.md @@ -0,0 +1,211 @@ +# sftp-rs: Asynchronous Client (`async.rs`) + +## `AsyncSftpClient` + +An async SFTP client that supports **concurrent pipelined requests** over a single connection via a background reader task: + +```rust +pub struct AsyncSftpClient { + writer: TokioMutex, + pending: Pending, + last_request_id: AtomicU32, + version: u32, + extensions: Vec<(String, String)>, + reader_task: TokioMutex>>, +} +``` + +Where: +```rust +type Pending = Arc)>>>>; +``` + +## Architecture: Background Reader + Oneshot Channels + +Unlike the sync client (which does send-then-receive per request), the async client decouples writing from reading: + +1. **Writer side**: Each call to `process()` writes a request packet (with a unique request-id) to the `writer`, protected by a `TokioMutex` +2. **Reader side**: A spawned tokio task (`run_reader`) continuously reads packets from the reader half, strips the request-id from each response, and routes it to the matching `oneshot::Sender` in the `pending` map +3. **Caller**: Awaits on the `oneshot::Receiver`, which resolves when the reader task delivers the matching response + +This allows multiple requests to be in flight simultaneously — the client can send requests 1, 2, and 3, and the reader will route each response to the correct waiter regardless of arrival order. + +``` +┌─────────────────┐ write ┌──────────────┐ +│ calling task │──────────────→│ writer (W) │ +│ (await rx) │ └──────────────┘ +└────────┬────────┘ + │ oneshot channel + │ (tx inserted into pending map) + │ +┌────────┴────────┐ read ┌──────────────┐ +│ reader task │←──────────────│ reader (R) │ +│ (run_reader) │ └──────────────┘ +│ │ +│ 1. read packet │ +│ 2. split req_id│ +│ 3. lookup pending[req_id] +│ 4. send via tx │ +└─────────────────┘ +``` + +## Construction + +```rust +impl AsyncSftpClient { + pub async fn new(mut reader: R, mut writer: W) -> std::io::Result + where + R: AsyncRead + Unpin + Send + 'static, +} +``` + +The constructor: +1. Sends `SSH_FXP_INIT` with version 3 +2. Reads the response, expects `SSH_FXP_VERSION` +3. Parses version and extensions +4. Spawns the background reader task (`run_reader`) +5. Returns the client + +The reader and writer are provided as separate halves — typically obtained via `tokio::io::split()` on a duplex stream. + +## Drop Implementation + +```rust +impl Drop for AsyncSftpClient { + fn drop(&mut self) { + if let Ok(mut guard) = self.reader_task.try_lock() { + if let Some(handle) = guard.take() { + handle.abort(); + } + } + } +} +``` + +When the client is dropped, the background reader task is aborted. This prevents the task from running after the client's channels are gone. The `try_lock()` avoids blocking in the drop handler. + +## Request-Response Cycle: `process()` + +```rust +async fn process(&self, cmd: u8, body: &[u8]) -> std::io::Result<(u8, Vec)> +``` + +1. Allocate `request_id` via `AtomicU32::fetch_add(1, SeqCst)` +2. Create a `oneshot::channel()` +3. Insert `tx` into `pending[request_id]` +4. Prepend request-id: `with_request_id(request_id, body)` +5. Lock `writer` and send the packet via `write_packet_async` +6. If the write fails, remove the pending entry and return the error +7. Await on `rx` — resolves with `(cmd, payload)` when the reader task delivers the response + +## Background Reader: `run_reader()` + +```rust +async fn run_reader(mut reader: R, pending: Pending) +``` + +Runs in a loop: +1. Read a packet via `read_packet_async` +2. Split the request-id from the body +3. Look up `pending[request_id]` and remove it +4. Send `(cmd, payload)` via the oneshot channel +5. If the read fails (EOF, connection error), clear the entire pending map so all waiting tasks get a `RecvError` and return errors + +## Async Packet I/O + +```rust +async fn read_packet_async(r: &mut R) -> std::io::Result<(u8, Vec)> +async fn write_packet_async(w: &mut W, kind: u8, body: &[u8]) -> std::io::Result<()> +``` + +These mirror the sync `read_raw_packet` / `write_raw_packet` but use `AsyncReadExt` / `AsyncWriteExt`. The write function builds the header inline: + +```rust +let mut hdr = Vec::with_capacity(5); +hdr.extend_from_slice(&(body.len() as u32 + 1).to_be_bytes()); // length (includes type byte) +hdr.push(kind); // type +w.write_all(&hdr).await?; +w.write_all(body).await?; +w.flush().await?; +``` + +## Public API + +The async client exposes the same operations as the sync client, but all methods are `async`: + +```rust +// Directory operations +pub async fn mkdir(&self, path: &str, attr: &Attributes) -> Result<()> +pub async fn rmdir(&self, path: &str) -> Result<()> +pub async fn opendir(&self, path: &str) -> Result +pub async fn readdir(&self, dir: &Directory) -> Result> +pub async fn closedir(&self, dir: &Directory) -> Result<()> + +// File operations +pub async fn open(&self, path: &str, options: OpenOptions, attr: &Attributes) -> Result +pub async fn pread(&self, file: &File, offset: u64, length: u32) -> Result> +pub async fn pwrite(&self, file: &File, offset: u64, data: &[u8]) -> Result<()> +pub async fn fclose(&self, file: &File) -> Result<()> + +// Attribute operations +pub async fn stat(&self, path: &str, flags: Option) -> Result +pub async fn lstat(&self, path: &str, flags: Option) -> Result +pub async fn fstat(&self, file: &File, flags: Option) -> Result +pub async fn setstat(&self, path: &str, attr: &Attributes) -> Result<()> +pub async fn fsetstat(&self, file: &File, attr: &Attributes) -> Result<()> + +// Path operations +pub async fn realpath(&self, path: &str, control_byte: Option, compose_path: Option<&str>) -> Result +pub async fn readlink(&self, path: &str) -> Result +pub async fn remove(&self, path: &str) -> Result<()> +pub async fn rename(&self, oldpath: &str, newpath: &str, flags: Option) -> Result<()> + +// Link operations +pub async fn symlink(&self, path: &str, target: &str) -> Result<()> +pub async fn hardlink(&self, path: &str, target: &str) -> Result<()> +pub async fn link(&self, path: &str, target: &str, symlink: bool) -> Result<()> + +// Lock operations +pub async fn block(&self, file: &File, offset: u64, length: u64, lockmask: u32) -> Result<()> +pub async fn unblock(&self, file: &File, offset: u64, length: u64) -> Result<()> + +// Extended operations +pub async fn extended(&self, request: &str, data: &[u8]) -> Result>> +pub async fn flineseek(&self, file: &File, lineno: u64) -> Result<()> + +// Introspection +pub fn extensions(&self) -> &[(String, String)] +pub fn version(&self) -> u32 +``` + +## Concurrency Benefits + +Because the reader task decouples receiving from sending, multiple async operations can run concurrently: + +```rust +// Three concurrent mkdir requests — all three are sent before any +// response arrives, and the reader task routes each response correctly +let (r1, r2, r3) = tokio::join!( + client.mkdir("/a", &attrs), + client.mkdir("/b", &attrs), + client.rmdir("/c"), +); +``` + +The sync client cannot do this — each `process()` call blocks on its response before the next request can be sent. + +## Error Propagation on Disconnect + +When the reader task encounters a read error (connection closed), it: +1. Clears the entire `pending` map +2. All `oneshot::Receiver`s in waiting tasks receive `Err(RecvError)` +3. The `process()` method converts this to `std::io::Error("reader task closed before response arrived")` + +This ensures that pending operations fail promptly rather than hanging indefinitely when the connection drops. + +## Pending Map: `StdMutex` vs `TokioMutex` + +The `pending` map uses `std::sync::Mutex` rather than `tokio::sync::Mutex` because: +- The critical section is tiny (insert/remove from a HashMap) +- The reader task and writer are on different async tasks but need shared access +- `StdMutex` avoids holding a lock across `.await` points (the oneshot `rx.await` is outside the lock) \ No newline at end of file diff --git a/docs/research/references/ssh/sftp-rs/06-russh-integration.md b/docs/research/references/ssh/sftp-rs/06-russh-integration.md new file mode 100644 index 0000000..07b0ebe --- /dev/null +++ b/docs/research/references/ssh/sftp-rs/06-russh-integration.md @@ -0,0 +1,132 @@ +# sftp-rs: russh Integration (`russh.rs`) + +The `russh` module provides transport glue for connecting an `AsyncSftpClient` to a russh SSH session. It handles the SSH-level work of requesting the SFTP subsystem and converting the russh channel into a split read/write stream. + +**Feature gate**: `russh` (implies `async` + `tokio`) + +## Core Type Alias + +```rust +pub type RusshSftpClient = AsyncSftpClient>>; +``` + +The concrete client type when operating over a russh channel. The write half of a `ChannelStream` serves as the `W` type parameter for `AsyncSftpClient`. + +## `from_channel()` + +```rust +pub async fn from_channel(channel: Channel) -> std::io::Result +``` + +The primary entry point. Takes an already-open russh session channel and: + +1. **Requests the `sftp` subsystem** via `channel.request_subsystem(true, "sftp")` + - The `true` parameter means "want reply" — the server must acknowledge the subsystem request + - If the subsystem request fails, returns an IO error +2. **Delegates to `from_subsystem_channel()`** to wrap the channel into a client + +The caller is responsible for establishing the SSH session (host-key verification, authentication, proxy jumps, etc.) and opening the channel, e.g.: + +```rust +let session = /* established russh client session */; +let channel = session.channel_open_session().await?; +let sftp = sftp::russh::from_channel(channel).await?; +``` + +## `from_subsystem_channel()` + +```rust +pub async fn from_subsystem_channel(channel: Channel) -> std::io::Result +``` + +For use when the subsystem request has already been made (or the caller wants to manage it differently): + +1. **Converts the channel to a stream**: `channel.into_stream()` → `ChannelStream` +2. **Splits the stream**: `tokio::io::split(stream)` → `(read_half, write_half)` +3. **Constructs the client**: `AsyncSftpClient::new(read_half, write_half).await` + - This performs the SFTP handshake (INIT/VERSION) + +Use cases: +- Custom subsystem names (not "sftp") +- Passing environment variables before the subsystem request +- Managing the subsystem request lifecycle externally + +## Data Flow: russh → AsyncSftpClient + +``` +┌──────────────────────────────────────────────────────────────┐ +│ Application Code │ +│ │ +│ let sftp = from_channel(channel).await?; │ +│ sftp.open("/path", opts, &attrs).await? │ +│ │ +└───────────────┬──────────────────────────────────────────────┘ + │ + ▼ +┌──────────────────────────────────────────────────────────────┐ +│ AsyncSftpClient>> │ +│ │ +│ ┌──────────────┐ ┌────────────────────────────────┐ │ +│ │ writer │ │ reader task │ │ +│ │ (WriteHalf) │ │ (ReadHalf) │ │ +│ └──────┬───────┘ └────────────┬───────────────────┘ │ +│ │ │ │ +│ │ SFTP packets │ SFTP packets │ +│ ▼ ▼ │ +└─────────┼───────────────────────────┼─────────────────────────┘ + │ │ + ▼ ▼ +┌──────────────────────────────────────────────────────────────┐ +│ tokio::io::split(ChannelStream) │ +│ │ +│ ┌───────────────────────────────────────────────────────┐ │ +│ │ ChannelStream │ │ +│ │ (implements AsyncRead + AsyncWrite) │ │ +│ └───────────────────────┬───────────────────────────────┘ │ +│ │ │ +│ ┌───────────────────────┴───────────────────────────────┐ │ +│ │ Channel (russh) │ │ +│ │ • request_subsystem(true, "sftp") │ │ +│ │ • into_stream() → ChannelStream │ │ +│ └───────────────────────┬───────────────────────────────┘ │ +│ │ │ +│ ┌───────────────────────┴───────────────────────────────┐ │ +│ │ SSH Session (russh client) │ │ +│ │ • Authentication, host key verification │ │ +│ │ • channel_open_session() │ │ +│ └───────────────────────────────────────────────────────┘ │ +└──────────────────────────────────────────────────────────────┘ +``` + +## Key russh Types Used + +| Type | Source | Purpose | +|------|--------|---------| +| `Channel` | `russh` | An open SSH channel, used to request subsystems | +| `ChannelStream` | `russh` | Adapter from `Channel` to `AsyncRead + AsyncWrite` | +| `Msg` | `russh::client` | Message type parameter for russh channels | +| `WriteHalf>` | `tokio::io` | Write half after splitting the stream | + +## Error Handling + +The `from_channel()` function converts russh errors to `std::io::Error`: + +```rust +channel + .request_subsystem(true, "sftp") + .await + .map_err(|e| std::io::Error::other(format!("sftp subsystem request failed: {:?}", e)))?; +``` + +This means the caller only needs to handle `std::io::Error`, not russh-specific error types. + +## Responsibility Split + +| Layer | Responsibility | +|-------|---------------| +| **Caller** | SSH session creation, host-key verification, user authentication, channel opening | +| **`from_channel()`** | Subsystem request, stream creation, SFTP handshake | +| **`from_subsystem_channel()`** | Stream creation, SFTP handshake (no subsystem request) | +| **`AsyncSftpClient`** | All SFTP protocol operations (open, read, write, etc.) | + +The russh module is intentionally thin — it does the minimal work to bridge from a russh channel to an `AsyncSftpClient`, keeping all SFTP logic in the shared `async.rs` and `protocol.rs` modules. \ No newline at end of file diff --git a/docs/research/references/ssh/sftp-rs/07-cli-binary.md b/docs/research/references/ssh/sftp-rs/07-cli-binary.md new file mode 100644 index 0000000..cdbd887 --- /dev/null +++ b/docs/research/references/ssh/sftp-rs/07-cli-binary.md @@ -0,0 +1,243 @@ +# sftp-rs: CLI Binary (`bin/sftp.rs`) + +The crate includes an interactive command-line SFTP client binary, similar in spirit to OpenSSH's `sftp(1)`. It connects to a remote host via an `ssh -s sftp` subprocess and provides a readline-based shell for file operations. + +**Feature gate**: `bin` (enabled by default) + +## Dependencies + +- `rustyline` — readline library with history support +- `shell-words` — shell-style token parsing (handles quoting) + +## Connection: `SshChannel` + +The binary doesn't use russh or ssh2 — it spawns an `ssh` subprocess and talks to its stdin/stdout: + +```rust +struct SshChannel { + child: Child, + stdin: Option, + stdout: Option, +} +``` + +### Spawning + +```rust +fn spawn(destination: &str) -> std::io::Result +``` + +Runs: +``` +ssh -s sftp +``` + +The `-s` flag tells ssh to request a subsystem (rather than execute a command). The `sftp` argument is the subsystem name. + +### `Read` / `Write` Implementation + +```rust +impl Read for SshChannel { + fn read(&mut self, buf: &mut [u8]) -> std::io::Result + // Reads from child stdout +} + +impl Write for SshChannel { + fn write(&mut self, buf: &[u8]) -> std::io::Result + fn flush(&mut self) -> std::io::Result<()> + // Writes to child stdin +} +``` + +### Drop Behavior + +```rust +impl Drop for SshChannel { + fn drop(&mut self) { + drop(self.stdin.take()); // Close stdin → ssh sees EOF + drop(self.stdout.take()); + let _ = self.child.wait(); // Reap the subprocess + } +} +``` + +Closing stdin before `wait()` is critical — without it, the ssh subprocess blocks waiting for input and `wait()` hangs forever. + +## Shell: `Shell` + +```rust +struct Shell { + client: SftpClient, + remote_cwd: String, +} +``` + +The shell wraps a `SftpClient` and tracks the remote working directory. + +### Construction + +```rust +fn new(client: SftpClient) -> Result +``` + +Initializes `remote_cwd` by calling `realpath(".", None, None)` to get the server's current directory. + +### Path Resolution + +```rust +fn resolve_remote(&self, path: &str) -> String +``` + +Relative paths are joined with `remote_cwd`: +- If path starts with `/`, used as-is +- If `remote_cwd` ends with `/`, concatenated directly +- Otherwise, joined with `/` + +## Commands + +### Remote Commands + +| Command | Method | SFTP Operations | +|---------|--------|-----------------| +| `pwd` | `cmd_pwd()` | None (prints cached `remote_cwd`) | +| `cd [path]` | `cmd_cd()` | `realpath` + `stat` (validates it's a directory) | +| `ls [-l] [path]` | `cmd_ls()` | `stat` → `opendir` → loop `readdir` → `closedir` | +| `mkdir path` | `cmd_mkdir()` | `mkdir` | +| `rmdir path` | `cmd_rmdir()` | `rmdir` | +| `rm path` | `cmd_rm()` | `remove` | +| `rename old new` | `cmd_rename()` | `rename` | +| `symlink target link` | `cmd_symlink()` | `symlink` | +| `ln [-s] target link` | `cmd_ln()` | `symlink` or `hardlink` | +| `chmod mode path` | `cmd_chmod()` | `setstat` with `permissions` set | +| `stat path` | `cmd_stat()` | `stat` | +| `get remote [local]` | `cmd_get()` | `open` → loop `pread` → `fclose` | +| `put local [remote]` | `cmd_put()` | `open` → loop `pwrite` → `fclose` | + +### Local Commands + +| Command | Implementation | +|---------|---------------| +| `lpwd` | `std::env::current_dir()` | +| `lcd [path]` | `std::env::set_current_dir()` (defaults to `$HOME`) | +| `lls [path]` | `std::fs::read_dir()` | + +### Meta Commands + +| Command | Action | +|---------|--------| +| `help`, `?` | Print command listing | +| `quit`, `exit`, `bye` | Exit the loop | + +## File Transfer: `get` and `put` + +### `get remote [local]` + +Downloads a remote file to the local filesystem: + +```rust +fn cmd_get(&self, remote: &str, local: Option<&str>) -> Result<(), SftpError> +``` + +1. Opens the remote file with `OpenOptions::new().read(true)` +2. Creates a local file with `std::fs::File::create()` +3. Loops calling `pread(file, offset, 32*1024)` until `Eof` or empty data +4. Writes each chunk to the local file +5. Closes the remote file handle with `fclose` +6. Prints the total bytes transferred + +If no local path is given, derives the filename from the remote path's basename. + +### `put local [remote]` + +Uploads a local file to the remote server: + +```rust +fn cmd_put(&self, local: &str, remote: Option<&str>) -> Result<(), SftpError> +``` + +1. Opens the local file with `std::fs::File::open()` +2. Opens the remote file with `OpenOptions::new().write(true).create(true).truncate(true)` +3. Loops reading 32 KiB chunks from the local file +4. Writes each chunk with `pwrite(file, offset, data)` +5. Closes the remote file handle with `fclose` +6. Prints the total bytes transferred + +If no remote path is given, derives the remote filename from the local path's basename. + +Both operations use a **32 KiB chunk size** as a balance between latency and throughput. + +## Directory Listing: `ls` + +The `ls` command handles both files and directories: + +1. Calls `stat()` on the target path +2. If the target is a directory: + - Opens with `opendir()` + - Loops on `readdir()` until `Eof` + - Sorts entries by name + - Closes with `closedir()` +3. If the target is a regular file: + - Shows just that file's entry + +With `-l` flag, prints the `longname` field from the server (human-readable `ls -l` style output). Without `-l`, prints just filenames. + +### Directory Detection + +```rust +fn is_dir(attrs: &Attributes) -> bool { + attrs.permissions.is_some_and(|p| (p & 0o170000) == 0o040000) +} +``` + +Checks the POSIX file type bits in the permissions mask. `0o040000` is `S_IFDIR`. + +## Attribute Display + +### `format_long()` + +```rust +fn format_long(path: &str, attrs: &Attributes) -> String +``` + +Produces a line with octal permissions, size, and path: +``` +100644 1234 myfile.txt +``` + +### `print_attrs()` + +```rust +fn print_attrs(path: &str, attrs: &Attributes) +``` + +Detailed attribute display for the `stat` command: +``` +/path: + size: 1234 + permissions: 100644 + uid/gid: 1000/1000 + owner/group: alice/staff + mtime: 1700000000 +``` + +## Main Loop + +```rust +fn main() -> std::io::Result<()> +``` + +1. Parses the destination from `args[1]` (e.g., `user@host`) +2. Spawns the `SshChannel` +3. Creates a `SftpClient` +4. Initializes the `Shell` +5. Creates a `rustyline::DefaultEditor` with history at `~/.sftp_history` +6. Loops on `readline("sftp> ")`, dispatches commands via `dispatch()` +7. On `Interrupted`, continues; on `Eof`, exits + +## Dispatch + +```rust +fn dispatch(shell: &mut Shell, line: &str) -> bool +``` + +Parses the input line with `shell_words::split()`, matches the first token against known commands, and calls the appropriate method. Returns `false` for quit/exit/bye (breaking the loop), `true` otherwise. Errors are printed to stderr with `{:?}` formatting. \ No newline at end of file diff --git a/docs/research/references/ssh/sftp-rs/08-data-flow-and-examples.md b/docs/research/references/ssh/sftp-rs/08-data-flow-and-examples.md new file mode 100644 index 0000000..8b6a2fd --- /dev/null +++ b/docs/research/references/ssh/sftp-rs/08-data-flow-and-examples.md @@ -0,0 +1,266 @@ +# sftp-rs: Data Flow and Examples + +## File Download Flow (Sync Client) + +The most common SFTP operation — reading a file from the server: + +``` +Client Server + │ │ + │ SSH_FXP_OPEN [req_id=1] [path] [flags] [attrs] │ + │───────────────────────────────────────────────→│ + │ │ + │ SSH_FXP_HANDLE [req_id=1] [handle_bytes] │ + │←───────────────────────────────────────────────│ + │ │ + │ SSH_FXP_READ [req_id=2] [handle] [offset=0] [len=32K] + │───────────────────────────────────────────────→│ + │ │ + │ SSH_FXP_DATA [req_id=2] [data_bytes] │ + │←───────────────────────────────────────────────│ + │ │ + │ SSH_FXP_READ [req_id=3] [handle] [offset=N] [len=32K] + │───────────────────────────────────────────────→│ + │ │ + │ SSH_FXP_STATUS [req_id=3] [SSH_FX_EOF] │ + │←───────────────────────────────────────────────│ + │ │ + │ SSH_FXP_CLOSE [req_id=4] [handle] │ + │───────────────────────────────────────────────→│ + │ │ + │ SSH_FXP_STATUS [req_id=4] [SSH_FX_OK] │ + │←───────────────────────────────────────────────│ +``` + +### Sync Code Path + +```rust +let file = client.open("/remote/file", OpenOptions::new().read(true), &Attributes::new())?; +let mut offset: u64 = 0; +const CHUNK: u32 = 32 * 1024; +loop { + match client.pread(&file, offset, CHUNK) { + Ok(data) if data.is_empty() => break, + Ok(data) => { /* write data locally */ offset += data.len() as u64; } + Err(Error::Eof(_, _)) => break, + Err(e) => return Err(e), + } +} +client.fclose(&file)?; +``` + +### Async Code Path + +```rust +let file = client.open("/remote/file", OpenOptions::new().read(true), &Attributes::new()).await?; +let mut offset: u64 = 0; +const CHUNK: u32 = 32 * 1024; +loop { + match client.pread(&file, offset, CHUNK).await { + Ok(data) if data.is_empty() => break, + Ok(data) => { /* write data locally */ offset += data.len() as u64; } + Err(Error::Eof(_, _)) => break, + Err(e) => return Err(e), + } +} +client.fclose(&file).await?; +``` + +## File Upload Flow + +``` +Client Server + │ │ + │ SSH_FXP_OPEN [path] [WRITE|CREAT|TRUNC] [attrs]│ + │───────────────────────────────────────────────→│ + │ │ + │ SSH_FXP_HANDLE [handle] │ + │←───────────────────────────────────────────────│ + │ │ + │ SSH_FXP_WRITE [handle] [offset=0] [data] │ + │───────────────────────────────────────────────→│ + │ │ + │ SSH_FXP_STATUS [SSH_FX_OK] │ + │←───────────────────────────────────────────────│ + │ │ + │ ... (repeat for each chunk) ... │ + │ │ + │ SSH_FXP_CLOSE [handle] │ + │───────────────────────────────────────────────→│ + │ │ + │ SSH_FXP_STATUS [SSH_FX_OK] │ + │←───────────────────────────────────────────────│ +``` + +## Directory Listing Flow + +``` +Client Server + │ │ + │ SSH_FXP_OPENDIR [path] │ + │───────────────────────────────────────────────→│ + │ │ + │ SSH_FXP_HANDLE [dir_handle] │ + │←───────────────────────────────────────────────│ + │ │ + │ SSH_FXP_READDIR [dir_handle] │ + │───────────────────────────────────────────────→│ + │ │ + │ SSH_FXP_NAME [count] [name+longname+attrs] │ + │←───────────────────────────────────────────────│ + │ │ + │ SSH_FXP_READDIR [dir_handle] │ + │───────────────────────────────────────────────→│ + │ │ + │ SSH_FXP_STATUS [SSH_FX_EOF] │ + │←───────────────────────────────────────────────│ + │ │ + │ SSH_FXP_CLOSE [dir_handle] │ + │───────────────────────────────────────────────→│ + │ │ + │ SSH_FXP_STATUS [SSH_FX_OK] │ + │←───────────────────────────────────────────────│ +``` + +## Async Pipelining Flow + +The async client can send multiple requests before any responses arrive: + +``` +Task 1 Task 2 Task 3 Reader Task + │ │ │ │ + │ process() │ process() │ process() │ + │ req_id=1 │ req_id=2 │ req_id=3 │ + │ insert tx1 │ insert tx2 │ insert tx3 │ + │ write pkt │ write pkt │ write pkt │ + │ await rx1 │ await rx2 │ await rx3 │ + │ │ │ │ read pkt + │ │ │ │ req_id=2 → tx2 + │ │ │ │ + │ │ rx2 resolved │ │ + │ │ returns ───────│ │ + │ │ │ │ read pkt + │ │ │ │ req_id=1 → tx1 + │ rx1 resolved │ │ │ + │ returns ─────│ │ │ + │ │ │ │ read pkt + │ │ │ │ req_id=3 → tx3 + │ │ │ rx3 resolved │ + │ │ │ returns ───────│ +``` + +## russh Integration Flow + +``` +┌──────────────────────────────────────────────────────────┐ +│ Application Code │ +│ │ +│ // 1. Establish SSH session │ +│ let session = /* russh client session setup */; │ +│ │ +│ // 2. Open a channel │ +│ let channel = session.channel_open_session().await?; │ +│ │ +│ // 3. Request SFTP subsystem + SFTP handshake │ +│ let sftp = sftp::russh::from_channel(channel).await?; │ +│ │ +│ // 4. Use SFTP │ +│ let file = sftp.open("/path", opts, &attrs).await?; │ +│ let data = sftp.pread(&file, 0, 4096).await?; │ +│ sftp.fclose(&file).await?; │ +└──────────────────────────────────────────────────────────┘ + +Expanded step 3: + from_channel(channel) + │ + ├─ channel.request_subsystem(true, "sftp") + │ └─ SSH_MSG_CHANNEL_REQUEST subsystem=sftp + │ └─ SSH_MSG_CHANNEL_SUCCESS + │ + ├─ channel.into_stream() → ChannelStream + │ + ├─ tokio::io::split(stream) → (read_half, write_half) + │ + └─ AsyncSftpClient::new(read_half, write_half) + ├─ write SSH_FXP_INIT version=3 + ├─ read SSH_FXP_VERSION + ├─ parse version + extensions + └─ spawn reader task +``` + +## Wire Format: `build_open` Example + +Opening a file `/hello.txt` for read: + +```rust +let opts = OpenOptions::new().read(true); +let body = build_open("/hello.txt", opts.get(), &Attributes::new())?; +// body layout: +// [0x00 0x00 0x00 0x0A] path length = 10 +// [0x2F 0x68 0x65 0x6C 0x6C 0x6F 0x2E 0x74 0x78 0x74] "/hello.txt" +// [0x00 0x00 0x00 0x01] flags = SFTP_FLAG_READ +// [0x00 0x00 0x00 0x00] attrs (empty: valid_attribute_flags = 0) +``` + +Full packet on wire (with request-id = 42): +``` +[0x00 0x00 0x00 0x19] length = 25 (1 type + 4 req_id + 10 path + 4 flags + 4 flags + 4 attrs_len) +[0x03] type = SSH_FXP_OPEN +[0x00 0x00 0x00 0x2A] request_id = 42 +[0x00 0x00 0x00 0x0A] path length = 10 +"/hello.txt" path +[0x00 0x00 0x00 0x01] open flags = READ +[0x00 0x00 0x00 0x00] attrs valid_flags = 0 (no attributes) +``` + +## Wire Format: `build_pread` Example + +Reading 1024 bytes from offset 4096: + +```rust +let body = build_pread(&handle, 4096, 1024); +// body layout: +// [0x00 0x00 0x00 0x04] handle length = 4 +// [0x48 0x41 0x4E 0x44] handle = "HAND" +// [0x00 0x00 0x00 0x00 0x00 0x00 0x10 0x00] offset = 4096 +// [0x00 0x00 0x04 0x00] length = 1024 +``` + +## Wire Format: `SSH_FXP_STATUS` Response + +Server returns `SSH_FX_NO_SUCH_FILE`: + +``` +[0x00 0x00 0x00 0x2A] request_id = 42 (matching the request) +[0x00 0x00 0x00 0x02] status = SSH_FX_NO_SUCH_FILE +[0x00 0x00 0x00 0x0F] error message length = 15 +"No such file" error message +[0x00 0x00 0x00 0x02] language tag length = 2 +"en" language tag +``` + +The `parse_status()` function maps this to `Error::NoSuchFile("No such file".into(), "en".into())`. + +## Error Handling Pattern + +Both sync and async clients follow the same pattern for response handling: + +```rust +// For operations that expect a handle (open, opendir): +let (cmd, data) = self.process(SSH_FXP_OPEN, &body).await?; +Ok(File(expect_handle(cmd, &data)?)) +// If server sends SSH_FXP_STATUS instead of SSH_FXP_HANDLE, +// expect_handle() converts the status error to the appropriate Error variant. + +// For operations that expect only status (mkdir, remove, close): +let (cmd, data) = self.process(SSH_FXP_MKDIR, &body).await?; +expect_status(cmd, &data) +// If server sends anything other than SSH_FXP_STATUS, returns unexpected response error. + +// For operations that expect data (read): +let (cmd, data) = self.process(SSH_FXP_READ, &body).await?; +expect_data(cmd, &data) +// SSH_FXP_DATA → returns the bytes +// SSH_FXP_STATUS with SSH_FX_EOF → Error::Eof (used to signal end-of-file) +// SSH_FXP_STATUS with other codes → appropriate Error variant +``` \ No newline at end of file diff --git a/docs/research/references/ssh/sftp-rs/09-quick-reference.md b/docs/research/references/ssh/sftp-rs/09-quick-reference.md new file mode 100644 index 0000000..35dd89a --- /dev/null +++ b/docs/research/references/ssh/sftp-rs/09-quick-reference.md @@ -0,0 +1,159 @@ +# sftp-rs: Quick Reference + +## Crate Info + +| Field | Value | +|-------|-------| +| Name | `sftp` | +| Version | 0.3.0 | +| License | Apache-2.0 | +| Edition | 2021 | +| Repository | https://github.com/jelmer/sftp-rs | +| Author | Jelmer Vernooij | + +## Feature Flags + +| Feature | Default | Requires | Provides | +|---------|---------|----------|----------| +| `default` | ✅ | `bin` | CLI binary | +| `bin` | ✅ (via default) | `rustyline`, `shell-words` | `sftp` binary | +| `ssh2` | ❌ | `ssh2` crate | `TryFrom` | +| `async` | ❌ | `tokio` | `AsyncSftpClient` | +| `russh` | ❌ | `russh`, `async`, `tokio` | russh transport glue | + +## Module Map + +| Module | Feature Gate | Contents | +|--------|-------------|----------| +| `protocol` | always | Wire codec: types, builders, parsers | +| `sync` | always | `SftpClient` | +| `r#async` | `async` | `AsyncSftpClient` | +| `russh` | `russh` | `from_channel()`, `from_subsystem_channel()` | + +## Re-exports (`lib.rs`) + +```rust +// Always available +pub use protocol::{Attributes, Directory, Error, File, Kind, OpenOptions, Result, TextHint, + /* all SSH_FILEXFER_ATTR_* constants */}; +pub use sync::SftpClient; + +// With "async" feature +pub use r#async::AsyncSftpClient; +``` + +## Client Construction + +```rust +// Sync: from any Read+Write +let client = SftpClient::new(channel)?; // channel: impl Read + Write +let client = SftpClient::from_fd(fd)?; // Unix: from raw fd +let client = SftpClient::from_handle(handle)?; // Windows: from raw handle +let client = SftpClient::try_from(ssh2_channel)?; // ssh2 feature + +// Async: from split halves +let client = AsyncSftpClient::new(reader, writer).await?; // impl AsyncRead + AsyncWrite + +// russh: from channel +let client = sftp::russh::from_channel(channel).await?; // requests sftp subsystem +let client = sftp::russh::from_subsystem_channel(channel).await?; // subsystem already requested +``` + +## Operations Cheat Sheet + +### File Operations + +| Operation | Sync | Async | Request | Response | +|-----------|------|-------|---------|----------| +| Open | `open(path, opts, attrs)` | `open(path, opts, attrs).await` | OPEN | HANDLE → File | +| Read | `pread(&file, offset, len)` | `pread(&file, offset, len).await` | READ | DATA → Vec\ | +| Write | `pwrite(&file, offset, data)` | `pwrite(&file, offset, data).await` | WRITE | STATUS | +| Close | `fclose(&file)` | `fclose(&file).await` | CLOSE | STATUS | +| Line seek | `flineseek(&file, lineno)` | `flineseek(&file, lineno).await` | EXTENDED "text-seek" | STATUS/REPLY | + +### Directory Operations + +| Operation | Sync | Async | Request | Response | +|-----------|------|-------|---------|----------| +| Open dir | `opendir(path)` | `opendir(path).await` | OPENDIR | HANDLE → Directory | +| Read dir | `readdir(&dir)` | `readdir(&dir).await` | READDIR | NAME → Vec\<(name, long, attrs)\> | +| Close dir | `closedir(&dir)` | `closedir(&dir).await` | CLOSE | STATUS | +| Make dir | `mkdir(path, attrs)` | `mkdir(path, attrs).await` | MKDIR | STATUS | +| Remove dir | `rmdir(path)` | `rmdir(path).await` | RMDIR | STATUS | + +### Attribute Operations + +| Operation | Sync | Async | Request | Response | +|-----------|------|-------|---------|----------| +| Stat (follow) | `stat(path, flags)` | `stat(path, flags).await` | STAT | ATTRS | +| Lstat (no follow) | `lstat(path, flags)` | `lstat(path, flags).await` | LSTAT | ATTRS | +| Fstat (by handle) | `fstat(&file, flags)` | `fstat(&file, flags).await` | FSTAT | ATTRS | +| Set stat (path) | `setstat(path, attrs)` | `setstat(path, attrs).await` | SETSTAT | STATUS | +| Set stat (handle) | `fsetstat(&file, attrs)` | `fsetstat(&file, attrs).await` | FSETSTAT | STATUS | + +### Path Operations + +| Operation | Sync | Async | Request | Response | +|-----------|------|-------|---------|----------| +| Canonicalize | `realpath(path, ctrl, compose)` | `realpath(path, ctrl, compose).await` | REALPATH | NAME → String | +| Read symlink | `readlink(path)` | `readlink(path).await` | READLINK | NAME → String | +| Remove file | `remove(path)` | `remove(path).await` | REMOVE | STATUS | +| Rename | `rename(old, new, flags)` | `rename(old, new, flags).await` | RENAME | STATUS | +| Symlink | `symlink(path, target)` | `symlink(path, target).await` | SYMLINK | STATUS | +| Hard link | `hardlink(path, target)` | `hardlink(path, target).await` | LINK | STATUS | +| Link (generic) | `link(path, target, sym)` | `link(path, target, sym).await` | LINK | STATUS | + +### Lock Operations + +| Operation | Sync | Async | Request | Response | +|-----------|------|-------|---------|----------| +| Block | `block(&file, off, len, mask)` | `block(&file, off, len, mask).await` | BLOCK | STATUS | +| Unblock | `unblock(&file, off, len)` | `unblock(&file, off, len).await` | UNBLOCK | STATUS | + +### Extension Operations + +| Operation | Sync | Async | Request | Response | +|-----------|------|-------|---------|----------| +| Extended | `extended(req, data)` | `extended(req, data).await` | EXTENDED | REPLY → Option\\> | + +## OpenOptions Builder + +```rust +OpenOptions::new() + .read(true) // SFTP_FLAG_READ = 0x01 + .write(true) // SFTP_FLAG_WRITE = 0x02 + .append(true) // SFTP_FLAG_APPEND = 0x04 + .create(true) // SFTP_FLAG_CREAT = 0x08 + .truncate(true) // SFTP_FLAG_TRUNC = 0x10 + .excl(true) // SFTP_FLAG_EXCL = 0x20 +``` + +## Error Variants (Most Common) + +| Error | When | +|-------|------| +| `Eof(msg, lang)` | End of file read, or end of directory listing | +| `NoSuchFile(msg, lang)` | File/path does not exist | +| `PermissionDenied(msg, lang)` | Insufficient permissions | +| `Failure(msg, lang)` | Generic failure | +| `DirNotEmpty(msg, lang)` | rmdir on non-empty directory | +| `NotADirectory(msg, lang)` | Expected directory, got file | +| `FileAlreadyExists(msg, lang)` | Exclusive create on existing file | +| `Io(err)` | Local I/O error or unexpected protocol message | +| `Other(code, msg, lang)` | Unrecognized SFTP status code | + +## Testing Approach + +Both sync and async clients have extensive test suites using **stub servers**: + +- **Sync**: `spawn_stub()` on TCP sockets — a background thread that handles INIT/VERSION then routes requests through a handler closure +- **Async**: `with_stub()` using `tokio::io::duplex()` — a spawned async task that runs a router against a handler closure + +Both approaches allow per-test programmable server behavior: the handler receives `(cmd, body_without_req_id)` and returns `(response_cmd, response_body_without_req_id)`. + +The protocol module has unit tests for: +- `Attributes` round-trip serialization (empty, all fields, individual field groups) +- Builder byte layout verification (exact byte-level assertions) +- Request-ID wrap/unwrap +- Status parsing (OK, EOF, typed errors) +- Expect functions (correct type, status-as-error, unexpected-type rejection) \ No newline at end of file