diff --git a/docs/research/references/ssh/russh-sftp/01-overview-and-architecture.md b/docs/research/references/ssh/russh-sftp/01-overview-and-architecture.md new file mode 100644 index 0000000..ccc2491 --- /dev/null +++ b/docs/research/references/ssh/russh-sftp/01-overview-and-architecture.md @@ -0,0 +1,212 @@ +# russh-sftp: Overview and Architecture + +**Version**: 2.3.0 +**Repository**: https://github.com/AspectUnk/russh-sftp +**License**: Apache-2.0 +**Rust Edition**: 2021 +**Protocol**: SFTP v3 ([draft-ietf-secsh-filexfer-02](https://datatracker.ietf.org/doc/html/draft-ietf-secsh-filexfer-02)) + +## What It Is + +`russh-sftp` is a Rust crate providing **both SFTP client and server** implementations. It operates on any transport that provides `AsyncRead + AsyncWrite` byte streams — not just russh. The crate targets SFTP protocol version 3, the most widely deployed version. + +Core design decisions: +- **Transport-agnostic** — works with any `AsyncRead + AsyncWrite` stream, not tied to a specific SSH library +- **Both client and server** — provides handler traits and run functions for both sides +- **Two client tiers** — a low-level `RawSftpSession` (request–response) and a high-level `SftpSession` (std::fs-like API with `AsyncRead`/`AsyncWrite` file I/O) +- **Custom serde wire format** — implements its own `Serializer`/`Deserializer` over `bytes::BytesMut` for SFTP binary encoding, not using serde's typical self-describing formats +- **Concurrent writes** — the high-level `File` type supports pipelined writes with configurable concurrency +- **WASM support** — the server module is gated behind `not(target_arch = "wasm32")`; client runtime abstracts over tokio and wasm-bindgen-futures + +## Source Layout + +``` +russh-sftp/src/ +├── lib.rs # Crate root: module declarations, macro imports +├── buf.rs # TryBuf trait: try_get_bytes/try_get_string on Buf +├── de.rs # Custom serde Deserializer (binary SFTP wire format) +├── ser.rs # Custom serde Serializer (binary SFTP wire format) +├── error.rs # Top-level Error enum +├── extensions.rs # OpenSSH extension types: limits, hardlink, fsync, statvfs +├── utils.rs # unix() time helper, read_packet() async wire reader +├── protocol/ # SFTP v3 message types and Packet enum +│ ├── mod.rs # Packet enum, type constants, TryFrom/Into +│ ├── init.rs # SSH_FXP_INIT +│ ├── version.rs # SSH_FXP_VERSION +│ ├── open.rs # SSH_FXP_OPEN + OpenFlags bitflags +│ ├── close.rs # SSH_FXP_CLOSE +│ ├── read.rs # SSH_FXP_READ +│ ├── write.rs # SSH_FXP_WRITE +│ ├── lstat.rs # SSH_FXP_LSTAT +│ ├── stat.rs # SSH_FXP_STAT +│ ├── fstat.rs # SSH_FXP_FSTAT +│ ├── setstat.rs # SSH_FXP_SETSTAT +│ ├── fsetstat.rs # SSH_FXP_FSETSTAT +│ ├── opendir.rs # SSH_FXP_OPENDIR +│ ├── readdir.rs # SSH_FXP_READDIR +│ ├── remove.rs # SSH_FXP_REMOVE +│ ├── mkdir.rs # SSH_FXP_MKDIR +│ ├── rmdir.rs # SSH_FXP_RMDIR +│ ├── realpath.rs # SSH_FXP_REALPATH +│ ├── rename.rs # SSH_FXP_RENAME +│ ├── readlink.rs # SSH_FXP_READLINK +│ ├── symlink.rs # SSH_FXP_SYMLINK +│ ├── status.rs # SSH_FXP_STATUS + StatusCode enum +│ ├── handle.rs # SSH_FXP_HANDLE +│ ├── data.rs # SSH_FXP_DATA +│ ├── name.rs # SSH_FXP_NAME +│ ├── attrs.rs # SSH_FXP_ATTRS +│ ├── extended.rs # SSH_FXP_EXTENDED / SSH_FXP_EXTENDED_REPLY +│ ├── file.rs # File struct (filename + longname + attrs) +│ └── file_attrs.rs # FileAttributes, FileAttr flags, FileMode, FileType, FilePermissions +├── client/ # Client-side implementation +│ ├── mod.rs # Config, run(), execute_handler() +│ ├── handler.rs # Client Handler trait +│ ├── rawsession.rs # RawSftpSession: request-response SFTP client +│ ├── session.rs # SftpSession: high-level std::fs-like client +│ ├── error.rs # Client-specific Error enum +│ ├── runtime.rs # Runtime abstraction (tokio native vs WASM) +│ └── fs/ +│ ├── mod.rs # Re-exports: File, DirEntry, ReadDir, Metadata +│ ├── file.rs # File: AsyncRead + AsyncWrite + AsyncSeek +│ └── dir.rs # DirEntry, ReadDir iterator +└── server/ # Server-side implementation + ├── mod.rs # Config, run(), run_with_config(), process_request() + ├── handler.rs # Server Handler trait + └── reply.rs # StatusReply type for error responses +``` + +## Key Dependencies + +| Dependency | Version | Purpose | +|------------|---------|---------| +| `tokio` | 1 | Async runtime: io-util, rt, sync, time, macros | +| `tokio-util` | 0.7.18 | Runtime utilities | +| `serde` | 1.0 | Derive macros for protocol types | +| `serde_bytes` | 0.11 | Efficient byte array serialization | +| `bitflags` | 2.11 | Bitflag types: OpenFlags, FileAttr, FileMode, FilePermissionFlags | +| `bytes` | 1.11 | BytesMut/Bytes for zero-copy wire I/O | +| `dashmap` | 6.1 | Concurrent HashMap for request/response tracking | +| `chrono` | 0.4 | DateTime for File::longname formatting | +| `thiserror` | 2.0 | Error derive macros | +| `log` | 0.4 | Logging facade | + +Dev dependencies: `russh` 0.61.0 (for examples), `criterion` 0.8.2 (benchmarks). + +## Feature Flags + +| Feature | Default | Description | +|---------|---------|-------------| +| `async-trait` | ❌ | Enables `#[async_trait]` attribute on Handler traits | + +## Architecture Diagram + +``` +┌──────────────────────────────────────────────────────────────────────┐ +│ Application Layer │ +│ │ +│ ┌───────────────────┐ ┌──────────────────────┐ ┌──────────────┐ │ +│ │ SftpSession │ │ RawSftpSession │ │ server:: │ │ +│ │ (high-level) │ │ (low-level) │ │ Handler │ │ +│ │ │ │ │ │ (user impl) │ │ +│ │ • open/create │ │ • init/open/close │ │ │ │ +│ │ • read/write │ │ • read/write │ │ init, open, │ │ +│ │ • metadata │ │ • stat/lstat/fstat │ │ read, write │ │ +│ │ • read_dir │ │ • opendir/readdir │ │ close, ... │ │ +│ │ • canonicalize │ │ • mkdir/rmdir/remove │ └──────┬───────┘ │ +│ │ • hardlink/fsync │ │ • symlink/readlink │ │ │ +│ │ • fs_info (statvfs)│ │ • extended │ │ │ +│ └────────┬──────────┘ └──────────┬────────────┘ │ │ +│ │ │ │ │ +│ │ File (AsyncIO) │ │ │ +│ │ ┌─────────────────┐ │ │ │ +│ │ │ AsyncRead/Write │ │ │ │ +│ │ │ AsyncSeek │ │ │ │ +│ │ │ • pipelined writes│ │ │ │ +│ │ │ • handle tracking │ │ │ │ +│ │ └────────┬─────────┘ │ │ │ +│ └───────────┼─────────────┘ │ │ +│ │ │ │ +├───────────────────────┼─────────────────────────────────────┼─────────┤ +│ │ Protocol Layer │ │ +│ ┌────────────────────┼─────────────────────────────────────┼───┐ │ +│ │ ▼ ▼ │ │ +│ │ ┌─────────────────────────────────────────────────────┐ │ │ +│ │ │ Packet enum │ │ │ +│ │ │ Init, Version, Open, Close, Read, Write, │ │ │ +│ │ │ Lstat, Fstat, SetStat, FSetStat, OpenDir, │ │ │ +│ │ │ ReadDir, Remove, MkDir, RmDir, RealPath, │ │ │ +│ │ │ Stat, Rename, ReadLink, Symlink, Status, │ │ │ +│ │ │ Handle, Data, Name, Attrs, Extended, ExtendedReply │ │ │ +│ │ └─────────────────────────────────────────────────────┘ │ │ +│ │ │ │ │ +│ │ ┌────────────┴────────────┐ │ │ +│ │ │ ser.rs / de.rs │ │ │ +│ │ │ Custom serde for binary │ │ │ +│ │ │ SFTP wire format │ │ │ +│ │ └─────────────────────────┘ │ │ +│ └────────────────────────────────────────────────────────────┘ │ +│ │ │ +├──────────────────────────────────┼──────────────────────────────────┤ +│ ▼ │ +│ ┌──────────────────────────┐ │ +│ │ utils::read_packet() │ │ +│ │ buf::TryBuf │ │ +│ │ Wire I/O (length-prefixed) │ +│ └────────────┬─────────────┘ │ +│ │ │ +├─────────────────────────────────┼───────────────────────────────────┤ +│ ▼ │ +│ ┌──────────────────────────────────────┐ │ +│ │ Transport (AsyncRead + AsyncWrite) │ │ +│ │ e.g., russh Channel::into_stream() │ │ +│ └──────────────────────────────────────┘ │ +└──────────────────────────────────────────────────────────────────────┘ +``` + +## How russh Integration Works + +The crate does **not** depend on `russh` at runtime — it only appears as a dev-dependency for examples. Integration is by the caller providing a stream: + +```rust +// From examples/client.rs — typical russh integration +let channel = session.channel_open_session().await.unwrap(); +channel.request_subsystem(true, "sftp").await.unwrap(); +let sftp = SftpSession::new(channel.into_stream()).await.unwrap(); +``` + +```rust +// From examples/server.rs — typical russh server integration +async fn subsystem_request(&mut self, channel_id: ChannelId, name: &str, session: &mut Session) + -> Result<(), Self::Error> +{ + if name == "sftp" { + let channel = self.get_channel(channel_id).await; + let sftp = SftpSession::default(); + session.channel_success(channel_id)?; + russh_sftp::server::run(channel.into_stream(), sftp).await; + } + Ok(()) +} +``` + +The `into_stream()` method on russh's `Channel` produces a type implementing `AsyncRead + AsyncWrite + Unpin + Send + 'static`, which is exactly what `russh-sftp`'s `run()` and `SftpSession::new()` accept. + +## Re-exports from `lib.rs` + +```rust +pub mod client; +pub mod de; +pub mod extensions; +pub mod protocol; +pub mod ser; +#[cfg(not(target_arch = "wasm32"))] +pub mod server; + +// Key re-exports: +pub use client::Handler; // client::Handler +pub use client::RawSftpSession; // low-level client +pub use client::SftpSession; // high-level client +pub use server::Handler; // server::Handler +pub use server::StatusReply; // server error reply type +``` \ No newline at end of file diff --git a/docs/research/references/ssh/russh-sftp/02-wire-protocol-and-codec.md b/docs/research/references/ssh/russh-sftp/02-wire-protocol-and-codec.md new file mode 100644 index 0000000..08c4953 --- /dev/null +++ b/docs/research/references/ssh/russh-sftp/02-wire-protocol-and-codec.md @@ -0,0 +1,231 @@ +# russh-sftp: Wire Protocol and Codec + +## SFTP v3 Wire Format + +The SFTP protocol (draft-ietf-secsh-filexfer-02) transmits packets over the SSH channel as: + +``` +┌────────────┬──────────┬─────────────────┐ +│ length │ type │ payload │ +│ (u32 BE) │ (u8) │ (variable) │ +│ 4 bytes │ 1 byte │ length-1 bytes│ +└────────────┴──────────┴─────────────────┘ +``` + +- `length` includes the type byte but not itself +- All multi-byte integers are **big-endian** (network byte order) +- Strings are encoded as `u32 length + UTF-8 bytes` +- Byte arrays are encoded as `u32 length + raw bytes` + +### Packet Type Constants + +Defined in `protocol/mod.rs`: + +| Constant | Value | Direction | Description | +|----------|-------|-----------|-------------| +| `SSH_FXP_INIT` | 1 | C→S | Client initialization | +| `SSH_FXP_VERSION` | 2 | S→C | Server version response | +| `SSH_FXP_OPEN` | 3 | C→S | Open a file | +| `SSH_FXP_CLOSE` | 4 | C→S | Close a handle | +| `SSH_FXP_READ` | 5 | C→S | Read from a handle | +| `SSH_FXP_WRITE` | 6 | C→S | Write to a handle | +| `SSH_FXP_LSTAT` | 7 | C→S | Stat a path (no follow) | +| `SSH_FXP_FSTAT` | 8 | C→S | Stat an open handle | +| `SSH_FXP_SETSTAT` | 9 | C→S | Set file attributes by path | +| `SSH_FXP_FSETSTAT` | 10 | C→S | Set file attributes by handle | +| `SSH_FXP_OPENDIR` | 11 | C→S | Open a directory | +| `SSH_FXP_READDIR` | 12 | C→S | Read directory entries | +| `SSH_FXP_REMOVE` | 13 | C→S | Remove a file | +| `SSH_FXP_MKDIR` | 14 | C→S | Create a directory | +| `SSH_FXP_RMDIR` | 15 | C→S | Remove a directory | +| `SSH_FXP_REALPATH` | 16 | C→S | Canonicalize a path | +| `SSH_FXP_STAT` | 17 | C→S | Stat a path (follow symlinks) | +| `SSH_FXP_RENAME` | 18 | C→S | Rename a file | +| `SSH_FXP_READLINK` | 19 | C→S | Read a symbolic link | +| `SSH_FXP_SYMLINK` | 20 | C→S | Create a symbolic link | +| `SSH_FXP_STATUS` | 101 | S→C / C→S | Status response | +| `SSH_FXP_HANDLE` | 102 | S→C | Handle response | +| `SSH_FXP_DATA` | 103 | S→C | Data response | +| `SSH_FXP_NAME` | 104 | S→C | Name list response | +| `SSH_FXP_ATTRS` | 105 | S→C | File attributes response | +| `SSH_FXP_EXTENDED` | 200 | C→S | Extended request | +| `SSH_FXP_EXTENDED_REPLY` | 201 | S→C | Extended reply | + +## Packet Reading + +Wire I/O is handled by `utils::read_packet()`: + +```rust +pub(crate) async fn read_packet( + stream: &mut S, + max_length: u32, +) -> Result { + let length = stream.read_u32().await?; + if length > max_length { + return Err(Error::BadMessage("packet length limit exceeded".to_owned())); + } + let mut buf = vec![0; length as usize]; + stream.read_exact(&mut buf).await?; + Ok(Bytes::from(buf)) +} +``` + +The read packet buffer **includes the type byte** as the first byte, followed by the payload. This design means the caller can distinguish packet types before full deserialization. + +## Packet Enum and Dispatch + +All packets are unified into a single `Packet` enum: + +```rust +pub enum Packet { + Init(Init), Version(Version), Open(Open), + Close(Close), Read(Read), Write(Write), + Lstat(Lstat), Fstat(Fstat), SetStat(SetStat), + FSetStat(FSetStat), OpenDir(OpenDir), ReadDir(ReadDir), + Remove(Remove), MkDir(MkDir), RmDir(RmDir), + RealPath(RealPath), Stat(Stat), Rename(Rename), + ReadLink(ReadLink), Symlink(Symlink), Status(Status), + Handle(Handle), Data(Data), Name(Name), + Attrs(Attrs), Extended(Extended), ExtendedReply(ExtendedReply), +} +``` + +### Deserialization (`TryFrom<&mut Bytes> for Packet`) + +Reads the type byte first, then delegates to the custom serde deserializer: + +```rust +fn try_from(bytes: &mut Bytes) -> Result { + let r#type = bytes.try_get_u8()?; + match r#type { + SSH_FXP_INIT => Self::Init(de::from_bytes(bytes)?), + SSH_FXP_OPEN => Self::Open(de::from_bytes(bytes)?), + // ... all 26 variants + _ => Err(Error::BadMessage("unknown type".to_owned())), + } +} +``` + +### Serialization (`TryFrom for Bytes`) + +Converts each variant to bytes via `ser::to_bytes()`, prepends type byte, and wraps with the 4-byte length: + +```rust +fn try_from(packet: Packet) -> Result { + let (r#type, payload): (u8, Bytes) = match packet { + Packet::Init(init) => (SSH_FXP_INIT, ser::to_bytes(&init)?), + Packet::Open(open) => (SSH_FXP_OPEN, ser::to_bytes(&open)?), + // ... all variants + }; + let length = payload.len() as u32 + 1; + let mut bytes = BytesMut::new(); + bytes.put_u32(length); + bytes.put_u8(r#type); + bytes.put_slice(&payload); + Ok(bytes.freeze()) +} +``` + +## Custom Serde Wire Codec + +The crate implements a **custom serde `Serializer` and `Deserializer`** that directly maps Rust types to the SFTP binary format. This is NOT JSON, Bincode, or any standard serde format — it is a bespoke binary encoding matching the SFTP v3 wire specification. + +### Serializer (`ser.rs`) + +The `Serializer` writes directly into a `BytesMut` buffer: + +| Rust Type | Wire Encoding | +|-----------|---------------| +| `u8` | 1 byte raw | +| `u32` | 4 bytes big-endian | +| `u64` | 8 bytes big-endian | +| `str` / `String` | `u32 length` + UTF-8 bytes | +| `bytes` | `u32 length` + raw bytes | +| `struct` | Fields concatenated in order (no field names) | +| `seq` | `u32 count` + elements | +| `map` | Key-value pairs (no length prefix) | +| `enum` | Variant index as `u32` + variant content | +| `None` | Nothing (zero bytes) | +| `Some(T)` | Serialized as `T` | +| `bool`, `i8`–`i64`, `u16`, `f32`/`f64`, `char` | **Not supported** — returns `BadMessage` error | + +Key detail: `struct` serialization uses `serialize_struct` which delegates to `serialize_tuple` — fields are written in declaration order with **no field names or tags**. This matches SFTP's positional binary layout. + +The `data_serialize` helper serializes `Vec` as a raw byte sequence **without** a length prefix (used for `Extended.data` and `ExtendedReply.data`). + +### Deserializer (`de.rs`) + +The `Deserializer` reads from a `&mut Bytes` buffer, consuming bytes as it goes: + +| Wire Pattern | Rust Deserialize Target | +|--------------|------------------------| +| 1 byte | `u8` | +| 4 bytes BE | `u32` | +| 8 bytes BE | `u64` | +| `u32 len` + bytes | `String` / `str` | +| `u32 len` + bytes | `Vec` / byte buf | +| `u32 count` + elements | `Vec` / seq | +| Positional fields | struct (tuple-like) | +| `u32 variant` + content | enum | +| Key-value pairs | `HashMap` | + +The `data_deserialize` helper reads all remaining bytes into a `Vec` (no length prefix) — used for `Extended.data` and `ExtendedReply.data`. + +### TryBuf Helper (`buf.rs`) + +A small extension trait on `bytes::Buf`: + +```rust +pub trait TryBuf: Buf { + fn try_get_bytes(&mut self) -> Result, Error>; // u32-length-prefixed + fn try_get_string(&mut self) -> Result; // u32-length-prefixed UTF-8 +} +``` + +These are used internally by the deserializer for reading SFTP's length-prefixed byte and string fields. + +## FileAttributes Serialization + +`FileAttributes` has a custom `Serialize`/`Deserialize` implementation because the SFTP wire format uses a **flags bitmask** to indicate which optional fields are present. This is fundamentally different from serde's typical self-describing formats. + +### Serialization Flow + +1. Compute `FileAttr` flags bitmask based on which `Option` fields are `Some`: + - `SIZE` (0x1) — `size` is present + - `UIDGID` (0x2) — `uid`/`gid` are present + - `PERMISSIONS` (0x4) — `permissions` is present + - `ACMODTIME` (0x8) — `atime`/`mtime` are present + - `EXTENDED` (0x80000000) — extended fields (not yet implemented) +2. Write flags as `u32` +3. Write fields conditionally based on flags + +### Deserialization Flow + +1. Read `u32` flags bitmask +2. Conditionally read fields based on which bits are set: + - If `SIZE`: read `u64` for `size` + - If `UIDGID`: read `u32` for `uid`, `u32` for `gid` + - If `PERMISSIONS`: read `u32` for `permissions` + - If `ACMODTIME`: read `u32` for `atime`, `u32` for `mtime` + +This ensures that fields not flagged are left as `None` in the `FileAttributes` struct. + +## Request ID Tracking + +All request packets (except `Init`) carry a `u32 id` field used as a request identifier. The `RequestId` trait and macro provide uniform access: + +```rust +pub(crate) trait RequestId: Sized { + fn get_request_id(&self) -> u32; +} + +macro_rules! impl_request_id { + ($packet:ty) => { + impl RequestId for $packet { + fn get_request_id(&self) -> u32 { self.id } + } + }; +} +``` + +This is used by the server to extract the request ID for constructing status responses on error, and by the client for demultiplexing responses. \ No newline at end of file diff --git a/docs/research/references/ssh/russh-sftp/03-key-types.md b/docs/research/references/ssh/russh-sftp/03-key-types.md new file mode 100644 index 0000000..d266787 --- /dev/null +++ b/docs/research/references/ssh/russh-sftp/03-key-types.md @@ -0,0 +1,356 @@ +# russh-sftp: Key Types + +## Protocol Types (`protocol/`) + +### Init and Version (Handshake) + +```rust +// SSH_FXP_INIT — sent by client to begin +pub struct Init { + pub version: u32, // Always 3 + pub extensions: HashMap, +} + +// SSH_FXP_VERSION — server response +pub struct Version { + pub version: u32, + pub extensions: HashMap, +} +``` + +Both implement `Default` with `version: 3` and empty extensions. Extensions are negotiated during the handshake (e.g., `"limits@openssh.com" → "1"`). + +### Open and OpenFlags + +```rust +// SSH_FXP_OPEN — open or create a file +pub struct Open { + pub id: u32, + pub filename: String, + pub pflags: OpenFlags, // Bitflags for access mode + pub attrs: FileAttributes, // Initial attributes for new files +} + +bitflags! { + pub struct OpenFlags: u32 { + const READ = 0x00000001; + const WRITE = 0x00000002; + const APPEND = 0x00000004; + const CREATE = 0x00000008; + const TRUNCATE = 0x00000010; + const EXCLUDE = 0x00000020; + } +} +``` + +`OpenFlags` implements `From for std::fs::OpenOptions`, converting SFTP flags into Rust's `OpenOptions` for server implementations. Notable behavior: if both `CREATE` and `EXCLUDE` are set, it maps to `create_new(true)`; otherwise `CREATE` maps to `create(true)`. + +### Handle + +```rust +// SSH_FXP_HANDLE — server returns a handle string for open files/dirs +pub struct Handle { + pub id: u32, + pub handle: String, +} +``` + +Handles are opaque strings identifying open file or directory references. They are returned by `SSH_FXP_OPEN` and `SSH_FXP_OPENDIR` responses, and used in subsequent `READ`, `WRITE`, `FSTAT`, `FSETSTAT`, `READDIR`, `FSYNC`, and `CLOSE` operations. + +### Data and Write + +```rust +// SSH_FXP_DATA — file data response +pub struct Data { + pub id: u32, + pub data: Vec, // serde_bytes — no length prefix in inner serialization +} + +// SSH_FXP_WRITE — write data to file +pub struct Write { + pub id: u32, + pub handle: String, + pub offset: u64, + pub data: Vec, // serde_bytes +} +``` + +Both use `serde_bytes` for the `data` field, which serializes as a length-prefixed byte array in the outer packet encoding. + +### Name and File + +```rust +// SSH_FXP_NAME — directory listing / path resolution response +pub struct Name { + pub id: u32, + pub files: Vec, +} + +// Represents a single file entry +pub struct File { + pub filename: String, + pub longname: String, // `ls -l` style long name + pub attrs: FileAttributes, +} +``` + +`File` provides constructors: +- `File::dummy(filename)` — Creates a file with empty longname and default attributes (for `realpath` responses per spec) +- `File::new(filename, attrs)` — Creates a file with auto-generated longname from attributes + +The `longname()` method generates an `ls -l`-style string: `"{type}{permissions} 0 {user} {group} {size} {mtime} {filename}"` + +### Attrs + +```rust +// SSH_FXP_ATTRS — file attributes response +pub struct Attrs { + pub id: u32, + pub attrs: FileAttributes, +} +``` + +### Status and StatusCode + +```rust +pub enum StatusCode { + Ok = 0, // Successful completion + Eof = 1, // End of file / no more directory entries + NoSuchFile = 2, // File does not exist + PermissionDenied = 3, // Permission denied + Failure = 4, // Generic failure + BadMessage = 5, // Badly formatted packet + NoConnection = 6, // Client-side only: no connection + ConnectionLost = 7, // Client-side only: connection lost + OpUnsupported = 8, // Operation not supported +} + +pub struct Status { + pub id: u32, + pub status_code: StatusCode, + pub error_message: String, + pub language_tag: String, // e.g., "en-US" +} +``` + +`StatusCode` derives `Error` (thiserror), providing human-readable `Display` output for each variant. + +### FileAttributes + +The core metadata type. See the wire codec doc for serialization details. + +```rust +pub struct FileAttributes { + pub size: Option, + pub uid: Option, + pub user: Option, // User name for longname display + pub gid: Option, + pub group: Option, // Group name for longname display + pub permissions: Option, // Unix permission + file type bits + pub atime: Option, // Access time (unix epoch) + pub mtime: Option, // Modification time (unix epoch) +} +``` + +Key methods: +- `is_dir()`, `is_regular()`, `is_symlink()`, `is_character()`, `is_block()`, `is_fifo()` — check `FileMode` bits +- `set_dir()`, `set_regular()`, etc. — set `FileMode` bits +- `file_type()` → `FileType` — simplified type classification +- `len()` → `u64` — file size (defaults to 0) +- `permissions()` → `FilePermissions` — simplified permission struct +- `accessed()` → `io::Result` — convert atime +- `modified()` → `io::Result` — convert mtime +- `empty()` — all fields `None` +- `From<&std::fs::Metadata>` — convert OS metadata (unix-specific for uid/gid/mode) + +#### Supporting Bitflag Types + +```rust +bitflags! { + pub struct FileAttr: u32 { + const SIZE = 0x00000001; + const UIDGID = 0x00000002; + const PERMISSIONS = 0x00000004; + const ACMODTIME = 0x00000008; + const EXTENDED = 0x80000000; + } + + pub struct FileMode: u32 { + const FIFO = 0x1000; // Named pipe + const CHR = 0x2000; // Character device + const DIR = 0x4000; // Directory + const NAM = 0x5000; // Named file (rare) + const BLK = 0x6000; // Block device + const REG = 0x8000; // Regular file + const LNK = 0xA000; // Symbolic link + const SOCK = 0xC000; // Socket + } + + pub struct FilePermissionFlags: u32 { + const OTHER_READ = 0o4; + const OTHER_WRITE = 0o2; + const OTHER_EXEC = 0o1; + const GROUP_READ = 0o40; + const GROUP_WRITE = 0o20; + const GROUP_EXEC = 0o10; + const OWNER_READ = 0o400; + const OWNER_WRITE = 0o200; + const OWNER_EXEC = 0o100; + } +} + +pub enum FileType { Dir, File, Symlink, Other } + +pub struct FilePermissions { + pub other_exec: bool, pub other_read: bool, pub other_write: bool, + pub group_exec: bool, pub group_read: bool, pub group_write: bool, + pub owner_exec: bool, pub owner_read: bool, pub owner_write: bool, +} +``` + +### Extended and ExtendedReply + +```rust +pub struct Extended { + pub id: u32, + pub request: String, // Extension name, e.g., "limits@openssh.com" + pub data: Vec, // serde_bytes, no inner length prefix +} + +pub struct ExtendedReply { + pub id: u32, + pub data: Vec, // serde_bytes +} +``` + +## Other Protocol Packets + +All follow the same pattern of `id: u32` plus operation-specific fields: + +| Packet | Fields | +|--------|--------| +| `Close` | `id`, `handle` | +| `Read` | `id`, `handle`, `offset: u64`, `len: u32` | +| `Lstat` | `id`, `path` | +| `Fstat` | `id`, `handle` | +| `SetStat` | `id`, `path`, `attrs` | +| `FSetStat` | `id`, `handle`, `attrs` | +| `OpenDir` | `id`, `path` | +| `ReadDir` | `id`, `handle` | +| `Remove` | `id`, `filename` | +| `MkDir` | `id`, `path`, `attrs` | +| `RmDir` | `id`, `path` | +| `RealPath` | `id`, `path` | +| `Stat` | `id`, `path` | +| `Rename` | `id`, `oldpath`, `newpath` | +| `ReadLink` | `id`, `path` | +| `Symlink` | `id`, `linkpath`, `targetpath` | + +## Extension Types (`extensions.rs`) + +OpenSSH extension constants and structures: + +```rust +pub const LIMITS: &str = "limits@openssh.com"; +pub const HARDLINK: &str = "hardlink@openssh.com"; +pub const FSYNC: &str = "fsync@openssh.com"; +pub const STATVFS: &str = "statvfs@openssh.com"; + +// Server limits advertisement +pub struct LimitsExtension { + pub max_packet_len: u64, + pub max_read_len: u64, + pub max_write_len: u64, + pub max_open_handles: u64, +} + +// Hardlink request data +pub struct HardlinkExtension { + pub oldpath: String, + pub newpath: String, +} + +// Fsync request data +pub struct FsyncExtension { + pub handle: String, +} + +// Statvfs request data +pub struct StatvfsExtension { + pub path: String, +} + +// Statvfs response +pub struct Statvfs { + pub block_size: u64, + pub fragment_size: u64, + pub blocks: u64, + pub blocks_free: u64, + pub blocks_avail: u64, + pub inodes: u64, + pub inodes_free: u64, + pub inodes_avail: u64, + pub fs_id: u64, + pub flags: u64, + pub name_max: u64, +} +``` + +## Error Types + +### Top-level Error (`error.rs`) + +```rust +pub enum Error { + IO(String), // I/O errors + UnexpectedEof, // Stream EOF + BadMessage(String), // Malformed packet + Client(String), // Wraps client::error::Error + UnexpectedBehavior(String), // Protocol violations +} +``` + +### Client Error (`client/error.rs`) + +```rust +pub enum Error { + Status(Status), // Server returned error status + IO(String), // I/O errors + Timeout, // Request timed out + Limited(String), // Limits exceeded + UnexpectedPacket, // Wrong packet type received + UnexpectedBehavior(String), // Protocol violations +} +``` + +### StatusReply (Server-side error conversion, `server/reply.rs`) + +```rust +pub struct StatusReply { + pub status_code: StatusCode, + pub error_message: Option, + pub language_tag: Option, +} +``` + +Server `Handler` errors must implement `Into`, which allows the framework to convert any handler error into a `SSH_FXP_STATUS` response. `StatusCode` has a `.with_message()` helper. + +## Config Types + +### Client Config (`client/mod.rs`) + +```rust +pub struct Config { + pub max_packet_len: u32, // Default: 262144 (256 KiB) + pub max_concurrent_writes: usize, // Default: 8 + pub request_timeout_secs: u64, // Default: 10 +} +``` + +### Server Config (`server/mod.rs`) + +```rust +pub struct Config { + pub max_client_packet_len: u32, // Default: 262144 (256 KiB) +} +``` \ No newline at end of file diff --git a/docs/research/references/ssh/russh-sftp/04-client-api.md b/docs/research/references/ssh/russh-sftp/04-client-api.md new file mode 100644 index 0000000..aeeead5 --- /dev/null +++ b/docs/research/references/ssh/russh-sftp/04-client-api.md @@ -0,0 +1,362 @@ +# russh-sftp: Client Implementation + +## Client Architecture Overview + +The client side provides two tiers of API: + +1. **`RawSftpSession`** — Low-level request-response client that sends individual SFTP packets and awaits their responses. Suitable for custom or non-standard SFTP interactions. + +2. **`SftpSession`** — High-level client modeled after `std::fs`. Provides ergonomic methods for file operations and creates `File` objects implementing `AsyncRead`/`AsyncWrite`/`AsyncSeek`. + +Both operate on a stream type `S: AsyncRead + AsyncWrite + Unpin + Send + 'static`. + +## Client Handler Trait + +The raw client infrastructure uses an internal `Handler` trait to dispatch incoming response packets: + +```rust +pub trait Handler: Sized { + type Error: Into; + + fn version(&mut self, version: Version) -> impl Future> + Send; + fn status(&mut self, status: Status) -> impl Future> + Send; + fn handle(&mut self, handle: Handle) -> impl Future> + Send; + fn data(&mut self, data: Data) -> impl Future> + Send; + fn name(&mut self, name: Name) -> impl Future> + Send; + fn attrs(&mut self, attrs: Attrs) -> impl Future> + Send; + fn extended_reply(&mut self, reply: ExtendedReply) -> impl Future> + Send; +} +``` + +With the `async-trait` feature enabled, this becomes `#[async_trait]`. + +The `run()` function spawns two tasks: +- **Reader task**: reads packets from the stream, dispatches to the handler +- **Writer task**: sends serialized packets from an `mpsc::UnboundedSender` + +```rust +pub fn run(stream: S, handler: H) -> mpsc::UnboundedSender +where + S: AsyncRead + AsyncWrite + Unpin + Send + 'static, + H: Handler + Send + 'static, +``` + +The returned sender is the write handle — callers push `Bytes` (serialized packets) into it. + +## RawSftpSession + +`RawSftpSession` is the core request-response client. It implements `Handler` internally via `SessionInner` to route response packets back to awaiting request futures. + +### Construction + +```rust +impl RawSftpSession { + pub fn new(stream: S) -> Self + where S: AsyncRead + AsyncWrite + Unpin + Send + 'static; + + pub fn new_with_config(stream: S, cfg: Config) -> Self + where S: AsyncRead + AsyncWrite + Unpin + Send + 'static; +} +``` + +### Internal Architecture + +``` +┌─────────────────────────────────────────────────────┐ +│ RawSftpSession │ +│ │ +│ tx: UnboundedSender ←── write to stream │ +│ requests: Arc, oneshot::Sender>>│ +│ next_req_id: AtomicU32 │ +│ handles: AtomicU64 │ +│ timeout: AtomicU64 │ +│ limits: Limits │ +│ │ +│ ┌──────────────────────────────────────────────┐ │ +│ │ SessionInner (Handler impl) │ │ +│ │ │ │ +│ │ version() → stores version, replies to init │ │ +│ │ status() → replies by request ID │ │ +│ │ handle() → replies by request ID │ │ +│ │ data() → replies by request ID │ │ +│ │ name() → replies by request ID │ │ +│ │ attrs() → replies by request ID │ │ +│ │ extended_reply() → replies by request ID │ │ +│ └──────────────────────────────────────────────┘ │ +└─────────────────────────────────────────────────────┘ +``` + +### Request-Response Mechanism + +1. Caller invokes a method like `raw.open(filename, flags, attrs).await` +2. `RawSftpSession` generates a unique `id` via `AtomicU32::fetch_add` +3. Creates a `oneshot::channel` and inserts `(Some(id), sender)` into the `DashMap` +4. Serializes the packet to `Bytes` and sends it via the `tx` channel +5. Awaits the `oneshot::Receiver` with a timeout +6. When the reader task receives a response packet, `SessionInner::reply()` removes the entry from the `DashMap` and sends the `Packet` through the `oneshot` sender +7. The original request future receives the `Packet` and pattern-matches it + +The `init()` method is special: it uses `id: None` (no request ID) and the version handler stores the negotiated version. + +### Limits Tracking + +```rust +pub struct Limits { + pub packet_len: Option, + pub read_len: Option, + pub write_len: Option, + pub open_handles: Option, +} +``` + +- Fetched from the server via the `limits@openssh.com` extension during `SftpSession::new_with_config()` +- Enforced client-side: `open()` and `opendir()` check `open_handles`, `read()` checks `read_len`, `write()` and `write_nowait()` check `write_len`, `send()` checks `packet_len` + +### API Methods + +| Method | SFTP Packet | Response | Notes | +|--------|------------|----------|-------| +| `init()` | `Init` | `Version` | Handshake; version negotiation | +| `open(filename, flags, attrs)` | `Open` | `Handle` | Opens a file; increments handle count | +| `close(handle)` | `Close` | `Status` | Closes a handle; decrements handle count | +| `close_nowait(handle)` | `Close` | — | Fire-and-forget close (returns oneshot receiver) | +| `read(handle, offset, len)` | `Read` | `Data` | Reads data; respects `Limits.read_len` | +| `write(handle, offset, data)` | `Write` | `Status` | Writes data; respects `Limits.write_len` | +| `write_nowait(handle, offset, data)` | `Write` | — | Fire-and-forget write | +| `lstat(path)` | `Lstat` | `Attrs` | Stat without following symlinks | +| `fstat(handle)` | `Fstat` | `Attrs` | Stat an open handle | +| `setstat(path, attrs)` | `SetStat` | `Status` | Set file attributes by path | +| `fsetstat(handle, attrs)` | `FSetStat` | `Status` | Set file attributes by handle | +| `opendir(path)` | `OpenDir` | `Handle` | Opens a directory; increments handle count | +| `readdir(handle)` | `ReadDir` | `Name` | Reads directory entries | +| `remove(filename)` | `Remove` | `Status` | Deletes a file | +| `mkdir(path, attrs)` | `MkDir` | `Status` | Creates a directory | +| `rmdir(path)` | `RmDir` | `Status` | Removes a directory | +| `realpath(path)` | `RealPath` | `Name` | Canonicalizes a path | +| `stat(path)` | `Stat` | `Status` | Stat following symlinks | +| `rename(oldpath, newpath)` | `Rename` | `Status` | Renames a file | +| `readlink(path)` | `ReadLink` | `Name` | Reads symlink target | +| `symlink(path, target)` | `Symlink` | `Status` | Creates a symbolic link | +| `extended(request, data)` | `Extended` | `Packet` | Generic extension mechanism | +| `limits()` | `Extended` | `LimitsExtension` | `limits@openssh.com` | +| `hardlink(oldpath, newpath)` | `Extended` | `Status` | `hardlink@openssh.com` | +| `fsync(handle)` | `Extended` | `Status` | `fsync@openssh.com` | +| `statvfs(path)` | `Extended` | `Statvfs` | `statvfs@openssh.com` v2 | + +### Response Classification + +Two macros handle the common response patterns: + +```rust +// For responses that return a specific packet type or error status +macro_rules! into_with_status { + ($result:ident, $packet:ident) => { + match $result { + Packet::$packet(p) => Ok(p), + Packet::Status(p) => Err(p.into()), + _ => Err(Error::UnexpectedPacket), + } + }; +} + +// For responses that only return success/failure status +macro_rules! into_status { + ($result:ident) => { + match $result { + Packet::Status(status) if status.status_code == StatusCode::Ok => Ok(status), + Packet::Status(status) => Err(status.into()), + _ => Err(Error::UnexpectedPacket), + } + }; +} +``` + +### Drop Behavior + +`RawSftpSession` implements `Drop` to call `close_session()`, which sends an empty `Bytes` to signal the writer task to shut down the stream. + +## SftpSession (High-Level Client) + +`SftpSession` wraps `RawSftpSession` in an `Arc` and provides `std::fs`-like methods: + +```rust +pub struct SftpSession { + session: Arc, + features: Features, +} +``` + +### Construction and Version Negotiation + +```rust +impl SftpSession { + pub async fn new(stream: S) -> SftpResult; + pub async fn new_with_config(stream: S, cfg: Config) -> SftpResult; +} +``` + +Construction performs: +1. Creates `RawSftpSession` with config +2. Sends `SSH_FXP_INIT` via `session.init().await` +3. Checks for supported extensions in the version response +4. If `limits@openssh.com` extension is available, fetches and applies limits +5. Stores detected feature flags in `Features` + +### Features Detection + +```rust +pub(crate) struct Features { + pub hardlink: bool, // hardlink@openssh.com v1 + pub fsync: bool, // fsync@openssh.com v1 + pub statvfs: bool, // statvfs@openssh.com v2 + pub limits: Option, // from limits@openssh.com + pub max_concurrent_writes: usize, + pub max_packet_len: u32, +} +``` + +### High-Level API + +| Method | Description | +|--------|-------------| +| `open(filename)` | Opens file read-only | +| `create(filename)` | Creates/truncates file write-only | +| `open_with_flags(filename, flags)` | Opens with specified `OpenFlags` | +| `open_with_flags_and_attributes(filename, flags, attrs)` | Opens with flags and initial attrs | +| `canonicalize(path)` | Resolves to absolute path via `realpath` | +| `create_dir(path)` | Creates directory | +| `read(path)` | Reads entire file to `Vec` | +| `write(path, data)` | Writes data to file | +| `try_exists(path)` | Checks existence (returns `Ok(false)` on `NoSuchFile`) | +| `read_dir(path)` | Returns `ReadDir` iterator over directory entries | +| `read_link(path)` | Reads symlink target | +| `remove_dir(path)` | Removes directory | +| `remove_file(filename)` | Removes file | +| `rename(oldpath, newpath)` | Renames file/directory | +| `symlink(path, target)` | Creates symbolic link | +| `metadata(path)` | Gets `FileAttributes` via `stat` | +| `set_metadata(path, metadata)` | Sets attributes via `setstat` | +| `symlink_metadata(path)` | Gets `FileAttributes` via `lstat` | +| `hardlink(oldpath, newpath)` | Creates hard link (returns `Ok(false)` if unsupported) | +| `fs_info(path)` | Gets filesystem stats via `statvfs` (returns `Ok(None)` if unsupported) | +| `set_timeout(secs)` | Sets request timeout | +| `close()` | Closes the SFTP session stream | + +## File (Async I/O) + +`client::fs::File` implements `AsyncRead`, `AsyncWrite`, and `AsyncSeek` for remote file I/O: + +```rust +pub struct File { + session: Arc, + handle: String, + state: FileState, + pos: u64, + closed: bool, + features: Features, +} + +struct FileState { + f_read: Option>>>>>>, + f_seek: Option>>>>, + f_flush: Option>>>>, + f_shutdown: Option>>>>, + write_acks: VecDeque>>, +} +``` + +### AsyncRead Implementation + +- Uses a state-machine pattern: `f_read` stores the in-progress read future +- Calculates read length as `min(remaining_buffer, max_read_len)` where `max_read_len` comes from `Limits.read_len` or `max_packet_len - 9` (read overhead) +- On `StatusCode::Eof`, returns `Ok(None)` → signals clean EOF to the caller +- Advances `pos` by the data length on successful read + +### AsyncWrite Implementation + +- Implements **pipelined writes** with configurable concurrency (`max_concurrent_writes`) +- Uses `write_nowait()` to fire off write requests without awaiting each ACK +- Stores `oneshot::Receiver`s in `write_acks: VecDeque` +- When `write_acks.len() >= max_concurrent_writes`, polls the oldest pending ACK before accepting new writes +- Write chunk size: `min(data.len(), max_write_len)` where `max_write_len` comes from `Limits.write_len` or `max_packet_len - 21 - handle_length` + +Overhead constants: +```rust +const READ_OVERHEAD_LENGTH: u32 = 9; // type(1) + id(4) + data_len(4) +const WRITE_OVERHEAD_LENGTH: u32 = 21; // type(1) + id(4) + handle_len(4) + offset(8) + data_len(4) +``` + +### AsyncSeek Implementation + +- `SeekFrom::Start(pos)` — direct position set +- `SeekFrom::Current(delta)` — arithmetic on current `pos` +- `SeekFrom::End(delta)` — requires an `fstat()` round-trip to get file size, then computes position + +### Flush and Shutdown + +- `poll_flush()`: Drains all pending write ACKs, then optionally calls `fsync` if the server supports it +- `poll_shutdown()`: Drains pending ACKs, then sends `close()` on the handle and marks `closed = true` + +### Drop Behavior + +If `closed` is not yet true, `File::drop()` sends a `close_nowait()` (fire-and-forget) to avoid blocking in a destructor. + +### File Metadata Methods + +```rust +impl File { + pub async fn metadata(&self) -> SftpResult; // fstat + pub async fn set_metadata(&self, metadata: Metadata) -> SftpResult<()>; // fsetstat + pub async fn sync_all(&self) -> SftpResult<()>; // fsync (no-op if unsupported) +} +``` + +## ReadDir and DirEntry + +```rust +pub struct ReadDir { + parent: Arc, + entries: VecDeque<(String, Metadata)>, +} + +pub struct DirEntry { + parent: Arc, + file: String, + metadata: Metadata, +} +``` + +- `ReadDir` implements `Iterator` +- Automatically filters out `.` and `..` entries +- `DirEntry::path()` constructs full paths using POSIX-style `/` separator +- `DirEntry::file_name()`, `file_type()`, `metadata()` provide accessors + +## Runtime Abstraction (`client/runtime.rs`) + +The client runtime abstracts over native tokio and WASM environments: + +- **Native** (`not(target_arch = "wasm32")`): Uses `tokio::spawn` and `tokio::time::timeout` +- **WASM** (`target_arch = "wasm32"`): Uses `wasm_bindgen_futures::spawn_local` and `gloo_timers::future::TimeoutFuture` + +The `spawn()` function returns a `JoinHandle` that wraps a `oneshot::Receiver`, providing a unified API for both platforms. + +## Timeout Behavior + +Each request in `RawSftpSession` has a configurable timeout (default 10 seconds, set via `Config::request_timeout_secs`): + +```rust +async fn request(&self, id: Option, packet: Packet) -> SftpResult { + let rx = self.send(id, packet)?; + let timeout = self.timeout.load(Ordering::Relaxed); + match runtime::timeout(Duration::from_secs(timeout), rx).await { + Ok(Ok(result)) => result, + Ok(Err(_)) => Err(Error::UnexpectedBehavior("sender dropped".into())), + Err(error) => { + self.requests.remove(&id); + Err(error) // Error::Timeout + } + } +} +``` + +On timeout, the pending request entry is cleaned up from the `DashMap`. \ No newline at end of file diff --git a/docs/research/references/ssh/russh-sftp/05-server-api.md b/docs/research/references/ssh/russh-sftp/05-server-api.md new file mode 100644 index 0000000..06a96bf --- /dev/null +++ b/docs/research/references/ssh/russh-sftp/05-server-api.md @@ -0,0 +1,272 @@ +# russh-sftp: Server Implementation + +## Server Architecture + +The server side of russh-sftp follows a **handler trait pattern**: users implement the `server::Handler` trait, and the framework calls their methods in response to incoming SFTP requests. Each request is processed and a response packet is sent back on the same stream. + +The server processes requests **sequentially** — one at a time, in order. There is no concurrent request handling within a single SFTP session. + +## Server Handler Trait + +```rust +pub trait Handler: Sized { + type Error: Into + Send; + + fn unimplemented(&self) -> Self::Error; + + // --- Lifecycle --- + fn init(&mut self, version: u32, extensions: HashMap) + -> impl Future> + Send; + + // --- File operations --- + fn open(&mut self, id: u32, filename: String, pflags: OpenFlags, attrs: FileAttributes) + -> impl Future> + Send; + fn close(&mut self, id: u32, handle: String) + -> impl Future> + Send; + fn read(&mut self, id: u32, handle: String, offset: u64, len: u32) + -> impl Future> + Send; + fn write(&mut self, id: u32, handle: String, offset: u64, data: Vec) + -> impl Future> + Send; + + // --- Metadata --- + fn lstat(&mut self, id: u32, path: String) + -> impl Future> + Send; + fn fstat(&mut self, id: u32, handle: String) + -> impl Future> + Send; + fn setstat(&mut self, id: u32, path: String, attrs: FileAttributes) + -> impl Future> + Send; + fn fsetstat(&mut self, id: u32, handle: String, attrs: FileAttributes) + -> impl Future> + Send; + + // --- Directory operations --- + fn opendir(&mut self, id: u32, path: String) + -> impl Future> + Send; + fn readdir(&mut self, id: u32, handle: String) + -> impl Future> + Send; + + // --- Filesystem operations --- + fn remove(&mut self, id: u32, filename: String) + -> impl Future> + Send; + fn mkdir(&mut self, id: u32, path: String, attrs: FileAttributes) + -> impl Future> + Send; + fn rmdir(&mut self, id: u32, path: String) + -> impl Future> + Send; + fn realpath(&mut self, id: u32, path: String) + -> impl Future> + Send; + fn stat(&mut self, id: u32, path: String) + -> impl Future> + Send; + fn rename(&mut self, id: u32, oldpath: String, newpath: String) + -> impl Future> + Send; + fn readlink(&mut self, id: u32, path: String) + -> impl Future> + Send; + fn symlink(&mut self, id: u32, linkpath: String, targetpath: String) + -> impl Future> + Send; + + // --- Extensions --- + fn extended(&mut self, id: u32, request: String, data: Vec) + -> impl Future> + Send; +} +``` + +Every method has a default implementation that calls `self.unimplemented()` and returns it as `Err`, making it safe to only implement the methods you need. + +### Handler Error Type + +The associated `Error` type must implement `Into`. When a handler method returns `Err(e)`, the framework converts the error to a `StatusReply` and sends an `SSH_FXP_STATUS` packet with the appropriate `StatusCode`, error message, and language tag. + +### StatusReply + +```rust +pub struct StatusReply { + pub status_code: StatusCode, + pub error_message: Option, + pub language_tag: Option, +} +``` + +Convenience constructors: +```rust +impl StatusReply { + pub fn new(status_code: StatusCode) -> Self; + pub fn with_message(self, message: impl Into) -> Self; + pub fn with_language_tag(self, tag: impl Into) -> Self; +} + +impl StatusCode { + pub fn with_message(self, message: impl Into) -> StatusReply; +} +``` + +Example: +```rust +// Using StatusCode directly — minimal allocation +Err(StatusCode::NoSuchFile) + +// With custom message +Err(StatusCode::PermissionDenied.with_message("access denied")) + +// Full control +Err(StatusReply::new(StatusCode::Failure) + .with_message("disk full") + .with_language_tag("en-US")) +``` + +All of these produce `StatusReply`, which then maps to: +``` +SSH_FXP_STATUS { id, status_code, error_message, language_tag } +``` + +## Request Processing Pipeline + +### The `into_wrap!` Macro + +Each request is processed via a macro that extracts fields from the packet and calls the handler: + +```rust +macro_rules! into_wrap { + ($id:expr, $handler:expr, $var:ident; $($arg:ident),*) => { + match $handler.$var($($var.$arg),*).await { + Err(err) => { + let StatusReply { status_code, error_message, language_tag } = err.into(); + Packet::Status(Status { + id: $id, + status_code, + error_message: error_message.unwrap_or_else(|| status_code.to_string()), + language_tag: language_tag.unwrap_or_else(|| "en-US".to_string()), + }) + }, + Ok(packet) => packet.into(), + } + }; +} +``` + +This macro: +1. Calls the handler method with the packet fields +2. On `Ok(packet)` — converts the result to a `Packet` via `.into()` +3. On `Err(err)` — converts error to `StatusReply`, then constructs a `Packet::Status` with defaults for missing fields + +### The `process_request` Function + +```rust +async fn process_request(packet: Packet, handler: &mut H) -> Packet +where H: Handler + Send +{ + let id = packet.get_request_id(); + match packet { + Packet::Init(init) => into_wrap!(id, handler, init; version, extensions), + Packet::Open(open) => into_wrap!(id, handler, open; id, filename, pflags, attrs), + Packet::Close(close) => into_wrap!(id, handler, close; id, handle), + Packet::Read(read) => into_wrap!(id, handler, read; id, handle, offset, len), + // ... all other variants + Packet::Extended(extended) => into_wrap!(id, handler, extended; id, request, data), + _ => Packet::error(0, StatusCode::BadMessage), + } +} +``` + +### The `run` Functions + +```rust +pub async fn run(stream: S, handler: H) +where + S: AsyncRead + AsyncWrite + Unpin + Send + 'static, + H: Handler + Send + 'static; + +pub async fn run_with_config(mut stream: S, mut handler: H, cfg: Config) +where + S: AsyncRead + AsyncWrite + Unpin + Send + 'static, + H: Handler + Send + 'static; +``` + +The server loop: +1. Reads a packet from the stream (`read_packet`) +2. Deserializes to `Packet` +3. On deserialization error, sends `Packet::error(0, StatusCode::BadMessage)` +4. Calls `process_request()` to dispatch to the handler +5. Serializes the response and writes it back to the stream +6. Flushes the stream +7. Repeats until `UnexpectedEof` or cancellation + +The server is spawned on `tokio::spawn` (or `wasm_bindgen_futures::spawn_local` on WASM) and runs independently. + +### Response Types + +Each handler method return type maps to a specific response packet: + +| Handler Method | Success Return | Packet Type | +|---------------|----------------|-------------| +| `init` | `Version` | `SSH_FXP_VERSION` | +| `open` | `Handle` | `SSH_FXP_HANDLE` | +| `close` | `Status` | `SSH_FXP_STATUS` | +| `read` | `Data` | `SSH_FXP_DATA` | +| `write` | `Status` | `SSH_FXP_STATUS` | +| `lstat` / `stat` / `fstat` | `Attrs` | `SSH_FXP_ATTRS` | +| `setstat` / `fsetstat` | `Status` | `SSH_FXP_STATUS` | +| `opendir` | `Handle` | `SSH_FXP_HANDLE` | +| `readdir` | `Name` | `SSH_FXP_NAME` | +| `remove` / `mkdir` / `rmdir` / `rename` / `symlink` | `Status` | `SSH_FXP_STATUS` | +| `realpath` / `readlink` | `Name` | `SSH_FXP_NAME` | +| `extended` | `Packet` | Any response type | + +## Server Example + +A minimal SFTP server with russh integration: + +```rust +#[derive(Default)] +struct SftpSession { + version: Option, +} + +impl russh_sftp::server::Handler for SftpSession { + type Error = StatusCode; + + fn unimplemented(&self) -> Self::Error { + StatusCode::OpUnsupported + } + + async fn init(&mut self, version: u32, _ext: HashMap) + -> Result + { + self.version = Some(version); + Ok(Version::new()) + } + + async fn close(&mut self, id: u32, _handle: String) -> Result { + Ok(Status { id, status_code: StatusCode::Ok, error_message: "Ok".into(), language_tag: "en-US".into() }) + } + + async fn opendir(&mut self, id: u32, path: String) -> Result { + Ok(Handle { id, handle: path }) + } + + async fn readdir(&mut self, id: u32, handle: String) -> Result { + // Return EOF when done + Err(StatusCode::Eof) + } + + async fn realpath(&mut self, id: u32, path: String) -> Result { + Ok(Name { id, files: vec![File::dummy("/")] }) + } +} + +// In the russh Handler: +async fn subsystem_request(&mut self, channel_id: ChannelId, name: &str, session: &mut Session) + -> Result<(), Self::Error> +{ + if name == "sftp" { + let channel = self.get_channel(channel_id).await; + let sftp = SftpSession::default(); + session.channel_success(channel_id)?; + russh_sftp::server::run(channel.into_stream(), sftp).await; + } else { + session.channel_failure(channel_id)?; + } + Ok(()) +} +``` + +## WASM Considerations + +The `server` module is gated behind `#[cfg(not(target_arch = "wasm32"))]` and is not available on WASM targets. The `client` module is available on both native and WASM via the `runtime.rs` abstraction. \ No newline at end of file diff --git a/docs/research/references/ssh/russh-sftp/06-data-flow-and-integration.md b/docs/research/references/ssh/russh-sftp/06-data-flow-and-integration.md new file mode 100644 index 0000000..ef315a3 --- /dev/null +++ b/docs/research/references/ssh/russh-sftp/06-data-flow-and-integration.md @@ -0,0 +1,278 @@ +# russh-sftp: Data Flow and Integration + +## Client Data Flow: File Read + +``` +Application code + │ + ▼ +SftpSession::open("file.txt") + │ + ▼ +RawSftpSession::open("file.txt", OpenFlags::READ, FileAttributes::empty()) + │ + ├── 1. Generate request ID: AtomicU32::fetch_add(1) + ├── 2. Check Limits.open_handles + ├── 3. Create oneshot channel, insert (Some(id), tx) into DashMap + ├── 4. Serialize Open packet → Bytes + ├── 5. Send Bytes via mpsc::UnboundedSender + └── 6. Await oneshot::Receiver with timeout + │ + ▼ + Writer Task (tokio::spawn) Reader Task (tokio::spawn) + ┌──────────────────────┐ ┌──────────────────────┐ + │ rx.recv() → Bytes │ │ read_packet(stream) │ + │ → write_all(stream) │ │ → Packet::try_from() │ + │ → flush() │ │ → SessionInner::reply │ + └──────────────────────┘ │ → DashMap.remove │ + │ → oneshot::tx.send │ + └──────────┬───────────┘ + │ + ┌───────────┘ + ▼ + oneshot::rx.recv() + │ + ▼ + Packet::Handle(Handle { id, handle: "..." }) + │ + ▼ + Return handle string + │ + ▼ +File::new(session, handle, features) + │ + ▼ (user calls AsyncRead::poll_read) +File::poll_read() + │ + ├── Compute max_read_len (from Limits or max_packet_len - 9) + ├── Compute offset (self.pos) and len (min of remaining buffer, max_read_len) + ├── Create async future: session.read(handle, offset, len) + │ └── RawSftpSession::read() → request/response cycle as above + ├── Poll future; on Ready: + │ ├── Err(Status::Eof) → Ok(None) → return Poll::Ready(Ok(())) + │ ├── Ok(data) → buf.put_slice(&data), advance pos + │ └── Other errors → return Poll::Ready(Err(...)) + └── On Pending: return Poll::Pending +``` + +## Client Data Flow: File Write (Pipelined) + +``` +Application code + │ + ▼ +file.write_all(&data).await (AsyncWrite::poll_write) + │ + ├── 1. If write_acks.len() >= max_concurrent_writes: + │ Poll oldest pending ACK; if not ready, return Pending + │ + ├── 2. Compute write chunk: min(data.len(), max_write_len) + │ max_write_len = Limits.write_len or (max_packet_len - 21 - handle_len) + │ + ├── 3. Call session.write_nowait(handle, offset, data) + │ └── RawSftpSession::write_nowait() + │ ├── Generate request ID + │ ├── Check Limits.write_len + │ ├── Create oneshot channel, insert into DashMap + │ ├── Send packet via mpsc channel + │ └── Return oneshot::Receiver (does NOT await) + │ + ├── 4. Store oneshot::Receiver in write_acks VecDeque + ├── 5. Advance pos by chunk length + └── 6. Return Poll::Ready(Ok(chunk_len)) + + Later: file.flush().await (AsyncWrite::poll_flush) + │ + ├── 1. Drain all pending write_acks (poll each one) + └── 2. If fsync supported: call session.fsync(handle) + └── RawSftpSession::fsync() → full request/response cycle + + Later: file.shutdown().await (AsyncWrite::poll_shutdown) + │ + ├── 1. Drain all pending write_acks + └── 2. Call session.close(handle) → full request/response cycle + └── Marks self.closed = true +``` + +## Server Data Flow + +``` +Client SSH connection + │ + ▼ (subsystem request for "sftp") +russh server::Handler::subsystem_request() + │ + ├── session.channel_success(channel_id) + └── russh_sftp::server::run(channel.into_stream(), sftp_handler).await + │ + ▼ + ┌──────────────────────────────────────────────┐ + │ tokio::spawn loop: │ + │ │ + │ loop { │ + │ 1. read_packet(stream, max_len) │ + │ 2. Packet::try_from(bytes) │ + │ └── On error: Packet::error(0, BadMsg)│ + │ 3. process_request(packet, handler) │ + │ └── Match packet variant │ + │ └── Call handler method with fields │ + │ └── Ok → convert to Packet::from() │ + │ └── Err → StatusReply → Packet::Status │ + │ 4. Bytes::try_from(response) │ + │ 5. stream.write_all(&packet) │ + │ 6. stream.flush() │ + │ } │ + │ // Break on UnexpectedEof │ + └──────────────────────────────────────────────┘ +``` + +## SFTP Handshake Protocol + +### Client-side handshake (in SftpSession::new) + +``` +Client Server + │ │ + │──── SSH_FXP_INIT {version:3} ─────────►│ + │ │ + │◄─── SSH_FXP_VERSION {version:3, │ + │ extensions: {...}} ────────────────│ + │ │ + │ (now SftpSession checks extensions │ + │ and optionally fetches limits) │ + │ │ + │──── SSH_FXP_EXTENDED { │ + │ request: "limits@openssh.com", │ + │ data: [] } ───────────────────────►│ + │ │ + │◄─── SSH_FXP_EXTENDED_REPLY { │ + │ data: LimitsExtension } ──────────│ + │ │ + │ (SftpSession stores limits in │ + │ RawSftpSession and Features) │ +``` + +### Server-side handshake (in Handler::init) + +``` +Server receives SSH_FXP_INIT + │ + ├── process_request() calls handler.init(version, extensions) + ├── Handler returns Ok(Version { version: 3, extensions }) + └── Version packet is serialized and sent back +``` + +## Error Handling Flows + +### Client Error Chain + +``` +RawSftpSession method call + │ + ├── Timeout → Error::Timeout + │ (DashMap entry cleaned up) + │ + ├── Unexpected packet type → Error::UnexpectedPacket + │ (e.g., expected Handle, got Data) + │ + ├── Server status error → Error::Status(Status) + │ (StatusCode != Ok) + │ + ├── Limit exceeded → Error::Limited(String) + │ (packet/read/write/handle limits from server) + │ + └── Channel closed → Error::UnexpectedBehavior("session closed") +``` + +### Server Error Chain + +``` +Handler method returns Err(e) + │ + ├── e.into() → StatusReply { status_code, error_message, language_tag } + │ + ├── Default fill: + │ error_message: None → status_code.to_string() + │ language_tag: None → "en-US" + │ + └── Packet::Status { id, status_code, error_message, language_tag } + sent back to client +``` + +## russh Integration Patterns + +### Client Pattern + +```rust +use russh::client; +use russh_sftp::client::SftpSession; + +struct Client; + +impl client::Handler for Client { + type Error = anyhow::Error; + + async fn check_server_key(&mut self, key: &russh::keys::PublicKey) + -> Result { Ok(true) } +} + +async fn connect_sftp() -> anyhow::Result { + let config = russh::client::Config::default(); + let mut session = client::connect(Arc::new(config), ("host", 22), Client).await?; + let auth_ok = session.authenticate_password("user", "pass").await?; + if !auth_ok.success() { return Err(anyhow!("auth failed")); } + + let channel = session.channel_open_session().await?; + channel.request_subsystem(true, "sftp").await?; + + // channel.into_stream() provides AsyncRead + AsyncWrite + Unpin + Send + 'static + let sftp = SftpSession::new(channel.into_stream()).await?; + Ok(sftp) +} +``` + +### Server Pattern + +```rust +use russh::server; +use russh_sftp::server; + +struct SshSession { /* ... */ } + +impl server::Handler for SshSession { + type Error = anyhow::Error; + // ... auth methods ... + + async fn subsystem_request(&mut self, channel_id: ChannelId, name: &str, session: &mut server::Session) + -> Result<(), Self::Error> + { + if name == "sftp" { + let channel = self.get_channel(channel_id).await; + let sftp_handler = MySftpHandler::new(); + session.channel_success(channel_id)?; + server::run(channel.into_stream(), sftp_handler).await; + } else { + session.channel_failure(channel_id)?; + } + Ok(()) + } +} +``` + +### Non-russh Transport + +Since russh-sftp only requires `AsyncRead + AsyncWrite + Unpin + Send + 'static`, any transport providing these traits works: + +```rust +// Using a TCP stream directly (e.g., for testing without SSH) +let stream = TcpStream::connect("127.0.0.1:2222").await?; +let sftp = SftpSession::new(stream).await?; + +// Using a Unix socket +let stream = UnixStream::connect("/tmp/sftp-sock").await?; +let sftp = SftpSession::new(stream).await?; + +// Using tokio::io::duplex for in-process testing +let (client, server) = tokio::io::duplex(8192); +// Feed server side to server::run(), client side to SftpSession::new() +``` \ No newline at end of file diff --git a/docs/research/references/ssh/russh-sftp/07-quick-reference.md b/docs/research/references/ssh/russh-sftp/07-quick-reference.md new file mode 100644 index 0000000..6844433 --- /dev/null +++ b/docs/research/references/ssh/russh-sftp/07-quick-reference.md @@ -0,0 +1,216 @@ +# russh-sftp: Quick Reference + +## Crate Overview + +| Property | Value | +|----------|-------| +| Version | 2.3.0 | +| License | Apache-2.0 | +| Protocol | SFTP v3 (draft-ietf-secsh-filexfer-02) | +| Min Rust | 2021 edition | +| WASM | Client works on wasm32; server not supported | + +## Feature Flags + +| Feature | Default | Description | +|---------|---------|-------------| +| `async-trait` | ❌ | Enables `#[async_trait]` on Handler traits | + +## Key Dependencies + +`tokio` (io-util, rt, sync, time, macros), `tokio-util`, `serde`, `serde_bytes`, `bitflags`, `bytes`, `dashmap`, `chrono`, `thiserror`, `log` + +## Public Modules + +| Module | Description | +|--------|-------------| +| `client` | Client-side: `RawSftpSession`, `SftpSession`, `Handler`, `Config`, `fs::File`, `fs::ReadDir`, `fs::DirEntry` | +| `server` | Server-side: `Handler`, `StatusReply`, `Config`, `run()`, `run_with_config()` | +| `protocol` | All SFTP packet types, `Packet` enum, `StatusCode`, `OpenFlags`, `FileAttributes`, `File`, etc. | +| `extensions` | OpenSSH extensions: `LimitsExtension`, `HardlinkExtension`, `FsyncExtension`, `StatvfsExtension`, `Statvfs` | +| `de` | `from_bytes()` — public deserialization function for extension data | +| `ser` | `to_bytes()` — public serialization function | + +## Client Quick Start + +```rust +use russh_sftp::client::SftpSession; +use russh_sftp::protocol::OpenFlags; + +// Connect (using any AsyncRead+AsyncWrite stream) +let sftp = SftpSession::new(stream).await?; + +// Or with config: +let sftp = SftpSession::new_with_config(stream, Config { + max_packet_len: 262144, + max_concurrent_writes: 8, + request_timeout_secs: 30, +}).await?; + +// File operations +let mut file = sftp.open("remote.txt").await?; // read-only +let mut file = sftp.create("new.txt").await?; // create+truncate+write +let mut file = sftp.open_with_flags("f", OpenFlags::READ | OpenFlags::WRITE).await?; + +file.write_all(b"hello").await?; +file.flush().await?; // drains write pipeline + optional fsync +file.rewind().await?; +let mut buf = Vec::new(); +file.read_to_end(&mut buf).await?; +file.shutdown().await?; // properly closes handle + +// Directory operations +for entry in sftp.read_dir(".").await? { + println!("{}: {:?}", entry.file_name(), entry.file_type()); +} + +// Other operations +sftp.canonicalize(".").await?; +sftp.metadata("file").await?; +sftp.symlink_metadata("link").await?; +sftp.create_dir("dir").await?; +sftp.remove_dir("dir").await?; +sftp.remove_file("file").await?; +sftp.rename("old", "new").await?; +sftp.symlink("target", "link").await?; +sftp.read_link("link").await?; +sftp.hardlink("src", "dst").await?; // returns false if unsupported +sftp.fs_info("/").await?; // returns Ok(None) if unsupported +sftp.close().await?; +``` + +## Server Quick Start + +```rust +use russh_sftp::protocol::{File, Handle, Name, Status, StatusCode, Version}; +use russh_sftp::server::{Handler, StatusReply}; + +struct MyHandler; + +impl Handler for MyHandler { + type Error = StatusCode; + + fn unimplemented(&self) -> Self::Error { + StatusCode::OpUnsupported + } + + async fn init(&mut self, version: u32, _ext: HashMap) + -> Result + { + Ok(Version::new()) + } + + // ... implement methods as needed +} + +// In your SSH server handler: +async fn subsystem_request(&mut self, channel_id: ChannelId, name: &str, session: &mut Session) + -> Result<(), Error> +{ + if name == "sftp" { + let channel = self.get_channel(channel_id).await; + session.channel_success(channel_id)?; + russh_sftp::server::run(channel.into_stream(), MyHandler).await; + } + Ok(()) +} +``` + +## RawSftpSession Quick Reference + +```rust +use russh_sftp::client::RawSftpSession; + +let session = RawSftpSession::new(stream); +// or: RawSftpSession::new_with_config(stream, config); + +// Must call init first +let version = session.init().await?; + +// Request-response methods +let handle = session.open("file", OpenFlags::READ, FileAttributes::empty()).await?; +let data = session.read(handle.handle, 0, 32768).await?; +session.write(handle.handle, 0, vec![1,2,3]).await?; +session.close(handle.handle).await?; +let attrs = session.stat("/path").await?; +let name = session.realpath(".").await?; +let dir_handle = session.opendir("/").await?; +let entries = session.readdir(dir_handle.handle).await?; +session.close(dir_handle.handle).await?; + +// Extensions +let limits = session.limits().await?; +session.hardlink("/old", "/new").await?; +session.fsync(handle).await?; +let fs_info = session.statvfs("/").await?; +``` + +## StatusCode Reference + +| Code | Constant | Meaning | +|------|----------|---------| +| 0 | `Ok` | Successful completion | +| 1 | `Eof` | End of file / no more directory entries | +| 2 | `NoSuchFile` | File does not exist | +| 3 | `PermissionDenied` | Insufficient permissions | +| 4 | `Failure` | Generic failure | +| 5 | `BadMessage` | Badly formatted packet | +| 6 | `NoConnection` | Client-side: no connection (never from server) | +| 7 | `ConnectionLost` | Client-side: connection lost (never from server) | +| 8 | `OpUnsupported` | Operation not supported | + +## OpenFlags Reference + +| Flag | Value | Description | +|------|-------|-------------| +| `READ` | 0x01 | Open for reading | +| `WRITE` | 0x02 | Open for writing | +| `APPEND` | 0x04 | Append to existing data | +| `CREATE` | 0x08 | Create if doesn't exist | +| `TRUNCATE` | 0x10 | Truncate to zero length | +| `EXCLUDE` | 0x20 | Fail if file exists (must be with CREATE) | + +## FileAttr (Attribute Flags) Reference + +| Flag | Value | Fields Present | +|------|-------|---------------| +| `SIZE` | 0x00000001 | `size: u64` | +| `UIDGID` | 0x00000002 | `uid: u32`, `gid: u32` | +| `PERMISSIONS` | 0x00000004 | `permissions: u32` | +| `ACMODTIME` | 0x00000008 | `atime: u32`, `mtime: u32` | +| `EXTENDED` | 0x80000000 | (not yet implemented) | + +## FileMode (File Type) Reference + +| Constant | Value | Type | +|----------|-------|------| +| `FIFO` | 0x1000 | Named pipe | +| `CHR` | 0x2000 | Character device | +| `DIR` | 0x4000 | Directory | +| `NAM` | 0x5000 | Named file | +| `BLK` | 0x6000 | Block device | +| `REG` | 0x8000 | Regular file | +| `LNK` | 0xA000 | Symbolic link | +| `SOCK` | 0xC000 | Socket | + +## Packet Wire Format + +All SFTP packets: + +``` +[u32 length] [u8 type] [payload...] +``` + +- `length` = size of type byte + payload (does not include the length field itself) +- Strings: `[u32 len] [utf8 bytes]` +- Byte arrays: `[u32 len] [raw bytes]` +- `FileAttributes`: `[u32 flags] [conditional fields based on flags]` + +## Extension Names + +| Extension | Name | Version | +|-----------|------|---------| +| Limits | `limits@openssh.com` | 1 | +| Hardlink | `hardlink@openssh.com` | 1 | +| Fsync | `fsync@openssh.com` | 1 | +| Statvfs | `statvfs@openssh.com` | 2 | \ No newline at end of file