docs(research): add russh-sftp deep-dive reference
This commit is contained in:
@@ -0,0 +1,212 @@
|
||||
# russh-sftp: Overview and Architecture
|
||||
|
||||
**Version**: 2.3.0
|
||||
**Repository**: https://github.com/AspectUnk/russh-sftp
|
||||
**License**: Apache-2.0
|
||||
**Rust Edition**: 2021
|
||||
**Protocol**: SFTP v3 ([draft-ietf-secsh-filexfer-02](https://datatracker.ietf.org/doc/html/draft-ietf-secsh-filexfer-02))
|
||||
|
||||
## What It Is
|
||||
|
||||
`russh-sftp` is a Rust crate providing **both SFTP client and server** implementations. It operates on any transport that provides `AsyncRead + AsyncWrite` byte streams — not just russh. The crate targets SFTP protocol version 3, the most widely deployed version.
|
||||
|
||||
Core design decisions:
|
||||
- **Transport-agnostic** — works with any `AsyncRead + AsyncWrite` stream, not tied to a specific SSH library
|
||||
- **Both client and server** — provides handler traits and run functions for both sides
|
||||
- **Two client tiers** — a low-level `RawSftpSession` (request–response) and a high-level `SftpSession` (std::fs-like API with `AsyncRead`/`AsyncWrite` file I/O)
|
||||
- **Custom serde wire format** — implements its own `Serializer`/`Deserializer` over `bytes::BytesMut` for SFTP binary encoding, not using serde's typical self-describing formats
|
||||
- **Concurrent writes** — the high-level `File` type supports pipelined writes with configurable concurrency
|
||||
- **WASM support** — the server module is gated behind `not(target_arch = "wasm32")`; client runtime abstracts over tokio and wasm-bindgen-futures
|
||||
|
||||
## Source Layout
|
||||
|
||||
```
|
||||
russh-sftp/src/
|
||||
├── lib.rs # Crate root: module declarations, macro imports
|
||||
├── buf.rs # TryBuf trait: try_get_bytes/try_get_string on Buf
|
||||
├── de.rs # Custom serde Deserializer (binary SFTP wire format)
|
||||
├── ser.rs # Custom serde Serializer (binary SFTP wire format)
|
||||
├── error.rs # Top-level Error enum
|
||||
├── extensions.rs # OpenSSH extension types: limits, hardlink, fsync, statvfs
|
||||
├── utils.rs # unix() time helper, read_packet() async wire reader
|
||||
├── protocol/ # SFTP v3 message types and Packet enum
|
||||
│ ├── mod.rs # Packet enum, type constants, TryFrom<Bytes>/Into<Bytes>
|
||||
│ ├── init.rs # SSH_FXP_INIT
|
||||
│ ├── version.rs # SSH_FXP_VERSION
|
||||
│ ├── open.rs # SSH_FXP_OPEN + OpenFlags bitflags
|
||||
│ ├── close.rs # SSH_FXP_CLOSE
|
||||
│ ├── read.rs # SSH_FXP_READ
|
||||
│ ├── write.rs # SSH_FXP_WRITE
|
||||
│ ├── lstat.rs # SSH_FXP_LSTAT
|
||||
│ ├── stat.rs # SSH_FXP_STAT
|
||||
│ ├── fstat.rs # SSH_FXP_FSTAT
|
||||
│ ├── setstat.rs # SSH_FXP_SETSTAT
|
||||
│ ├── fsetstat.rs # SSH_FXP_FSETSTAT
|
||||
│ ├── opendir.rs # SSH_FXP_OPENDIR
|
||||
│ ├── readdir.rs # SSH_FXP_READDIR
|
||||
│ ├── remove.rs # SSH_FXP_REMOVE
|
||||
│ ├── mkdir.rs # SSH_FXP_MKDIR
|
||||
│ ├── rmdir.rs # SSH_FXP_RMDIR
|
||||
│ ├── realpath.rs # SSH_FXP_REALPATH
|
||||
│ ├── rename.rs # SSH_FXP_RENAME
|
||||
│ ├── readlink.rs # SSH_FXP_READLINK
|
||||
│ ├── symlink.rs # SSH_FXP_SYMLINK
|
||||
│ ├── status.rs # SSH_FXP_STATUS + StatusCode enum
|
||||
│ ├── handle.rs # SSH_FXP_HANDLE
|
||||
│ ├── data.rs # SSH_FXP_DATA
|
||||
│ ├── name.rs # SSH_FXP_NAME
|
||||
│ ├── attrs.rs # SSH_FXP_ATTRS
|
||||
│ ├── extended.rs # SSH_FXP_EXTENDED / SSH_FXP_EXTENDED_REPLY
|
||||
│ ├── file.rs # File struct (filename + longname + attrs)
|
||||
│ └── file_attrs.rs # FileAttributes, FileAttr flags, FileMode, FileType, FilePermissions
|
||||
├── client/ # Client-side implementation
|
||||
│ ├── mod.rs # Config, run(), execute_handler()
|
||||
│ ├── handler.rs # Client Handler trait
|
||||
│ ├── rawsession.rs # RawSftpSession: request-response SFTP client
|
||||
│ ├── session.rs # SftpSession: high-level std::fs-like client
|
||||
│ ├── error.rs # Client-specific Error enum
|
||||
│ ├── runtime.rs # Runtime abstraction (tokio native vs WASM)
|
||||
│ └── fs/
|
||||
│ ├── mod.rs # Re-exports: File, DirEntry, ReadDir, Metadata
|
||||
│ ├── file.rs # File: AsyncRead + AsyncWrite + AsyncSeek
|
||||
│ └── dir.rs # DirEntry, ReadDir iterator
|
||||
└── server/ # Server-side implementation
|
||||
├── mod.rs # Config, run(), run_with_config(), process_request()
|
||||
├── handler.rs # Server Handler trait
|
||||
└── reply.rs # StatusReply type for error responses
|
||||
```
|
||||
|
||||
## Key Dependencies
|
||||
|
||||
| Dependency | Version | Purpose |
|
||||
|------------|---------|---------|
|
||||
| `tokio` | 1 | Async runtime: io-util, rt, sync, time, macros |
|
||||
| `tokio-util` | 0.7.18 | Runtime utilities |
|
||||
| `serde` | 1.0 | Derive macros for protocol types |
|
||||
| `serde_bytes` | 0.11 | Efficient byte array serialization |
|
||||
| `bitflags` | 2.11 | Bitflag types: OpenFlags, FileAttr, FileMode, FilePermissionFlags |
|
||||
| `bytes` | 1.11 | BytesMut/Bytes for zero-copy wire I/O |
|
||||
| `dashmap` | 6.1 | Concurrent HashMap for request/response tracking |
|
||||
| `chrono` | 0.4 | DateTime for File::longname formatting |
|
||||
| `thiserror` | 2.0 | Error derive macros |
|
||||
| `log` | 0.4 | Logging facade |
|
||||
|
||||
Dev dependencies: `russh` 0.61.0 (for examples), `criterion` 0.8.2 (benchmarks).
|
||||
|
||||
## Feature Flags
|
||||
|
||||
| Feature | Default | Description |
|
||||
|---------|---------|-------------|
|
||||
| `async-trait` | ❌ | Enables `#[async_trait]` attribute on Handler traits |
|
||||
|
||||
## Architecture Diagram
|
||||
|
||||
```
|
||||
┌──────────────────────────────────────────────────────────────────────┐
|
||||
│ Application Layer │
|
||||
│ │
|
||||
│ ┌───────────────────┐ ┌──────────────────────┐ ┌──────────────┐ │
|
||||
│ │ SftpSession │ │ RawSftpSession │ │ server:: │ │
|
||||
│ │ (high-level) │ │ (low-level) │ │ Handler │ │
|
||||
│ │ │ │ │ │ (user impl) │ │
|
||||
│ │ • open/create │ │ • init/open/close │ │ │ │
|
||||
│ │ • read/write │ │ • read/write │ │ init, open, │ │
|
||||
│ │ • metadata │ │ • stat/lstat/fstat │ │ read, write │ │
|
||||
│ │ • read_dir │ │ • opendir/readdir │ │ close, ... │ │
|
||||
│ │ • canonicalize │ │ • mkdir/rmdir/remove │ └──────┬───────┘ │
|
||||
│ │ • hardlink/fsync │ │ • symlink/readlink │ │ │
|
||||
│ │ • fs_info (statvfs)│ │ • extended │ │ │
|
||||
│ └────────┬──────────┘ └──────────┬────────────┘ │ │
|
||||
│ │ │ │ │
|
||||
│ │ File (AsyncIO) │ │ │
|
||||
│ │ ┌─────────────────┐ │ │ │
|
||||
│ │ │ AsyncRead/Write │ │ │ │
|
||||
│ │ │ AsyncSeek │ │ │ │
|
||||
│ │ │ • pipelined writes│ │ │ │
|
||||
│ │ │ • handle tracking │ │ │ │
|
||||
│ │ └────────┬─────────┘ │ │ │
|
||||
│ └───────────┼─────────────┘ │ │
|
||||
│ │ │ │
|
||||
├───────────────────────┼─────────────────────────────────────┼─────────┤
|
||||
│ │ Protocol Layer │ │
|
||||
│ ┌────────────────────┼─────────────────────────────────────┼───┐ │
|
||||
│ │ ▼ ▼ │ │
|
||||
│ │ ┌─────────────────────────────────────────────────────┐ │ │
|
||||
│ │ │ Packet enum │ │ │
|
||||
│ │ │ Init, Version, Open, Close, Read, Write, │ │ │
|
||||
│ │ │ Lstat, Fstat, SetStat, FSetStat, OpenDir, │ │ │
|
||||
│ │ │ ReadDir, Remove, MkDir, RmDir, RealPath, │ │ │
|
||||
│ │ │ Stat, Rename, ReadLink, Symlink, Status, │ │ │
|
||||
│ │ │ Handle, Data, Name, Attrs, Extended, ExtendedReply │ │ │
|
||||
│ │ └─────────────────────────────────────────────────────┘ │ │
|
||||
│ │ │ │ │
|
||||
│ │ ┌────────────┴────────────┐ │ │
|
||||
│ │ │ ser.rs / de.rs │ │ │
|
||||
│ │ │ Custom serde for binary │ │ │
|
||||
│ │ │ SFTP wire format │ │ │
|
||||
│ │ └─────────────────────────┘ │ │
|
||||
│ └────────────────────────────────────────────────────────────┘ │
|
||||
│ │ │
|
||||
├──────────────────────────────────┼──────────────────────────────────┤
|
||||
│ ▼ │
|
||||
│ ┌──────────────────────────┐ │
|
||||
│ │ utils::read_packet() │ │
|
||||
│ │ buf::TryBuf │ │
|
||||
│ │ Wire I/O (length-prefixed) │
|
||||
│ └────────────┬─────────────┘ │
|
||||
│ │ │
|
||||
├─────────────────────────────────┼───────────────────────────────────┤
|
||||
│ ▼ │
|
||||
│ ┌──────────────────────────────────────┐ │
|
||||
│ │ Transport (AsyncRead + AsyncWrite) │ │
|
||||
│ │ e.g., russh Channel::into_stream() │ │
|
||||
│ └──────────────────────────────────────┘ │
|
||||
└──────────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
## How russh Integration Works
|
||||
|
||||
The crate does **not** depend on `russh` at runtime — it only appears as a dev-dependency for examples. Integration is by the caller providing a stream:
|
||||
|
||||
```rust
|
||||
// From examples/client.rs — typical russh integration
|
||||
let channel = session.channel_open_session().await.unwrap();
|
||||
channel.request_subsystem(true, "sftp").await.unwrap();
|
||||
let sftp = SftpSession::new(channel.into_stream()).await.unwrap();
|
||||
```
|
||||
|
||||
```rust
|
||||
// From examples/server.rs — typical russh server integration
|
||||
async fn subsystem_request(&mut self, channel_id: ChannelId, name: &str, session: &mut Session)
|
||||
-> Result<(), Self::Error>
|
||||
{
|
||||
if name == "sftp" {
|
||||
let channel = self.get_channel(channel_id).await;
|
||||
let sftp = SftpSession::default();
|
||||
session.channel_success(channel_id)?;
|
||||
russh_sftp::server::run(channel.into_stream(), sftp).await;
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
```
|
||||
|
||||
The `into_stream()` method on russh's `Channel` produces a type implementing `AsyncRead + AsyncWrite + Unpin + Send + 'static`, which is exactly what `russh-sftp`'s `run()` and `SftpSession::new()` accept.
|
||||
|
||||
## Re-exports from `lib.rs`
|
||||
|
||||
```rust
|
||||
pub mod client;
|
||||
pub mod de;
|
||||
pub mod extensions;
|
||||
pub mod protocol;
|
||||
pub mod ser;
|
||||
#[cfg(not(target_arch = "wasm32"))]
|
||||
pub mod server;
|
||||
|
||||
// Key re-exports:
|
||||
pub use client::Handler; // client::Handler
|
||||
pub use client::RawSftpSession; // low-level client
|
||||
pub use client::SftpSession; // high-level client
|
||||
pub use server::Handler; // server::Handler
|
||||
pub use server::StatusReply; // server error reply type
|
||||
```
|
||||
@@ -0,0 +1,231 @@
|
||||
# russh-sftp: Wire Protocol and Codec
|
||||
|
||||
## SFTP v3 Wire Format
|
||||
|
||||
The SFTP protocol (draft-ietf-secsh-filexfer-02) transmits packets over the SSH channel as:
|
||||
|
||||
```
|
||||
┌────────────┬──────────┬─────────────────┐
|
||||
│ length │ type │ payload │
|
||||
│ (u32 BE) │ (u8) │ (variable) │
|
||||
│ 4 bytes │ 1 byte │ length-1 bytes│
|
||||
└────────────┴──────────┴─────────────────┘
|
||||
```
|
||||
|
||||
- `length` includes the type byte but not itself
|
||||
- All multi-byte integers are **big-endian** (network byte order)
|
||||
- Strings are encoded as `u32 length + UTF-8 bytes`
|
||||
- Byte arrays are encoded as `u32 length + raw bytes`
|
||||
|
||||
### Packet Type Constants
|
||||
|
||||
Defined in `protocol/mod.rs`:
|
||||
|
||||
| Constant | Value | Direction | Description |
|
||||
|----------|-------|-----------|-------------|
|
||||
| `SSH_FXP_INIT` | 1 | C→S | Client initialization |
|
||||
| `SSH_FXP_VERSION` | 2 | S→C | Server version response |
|
||||
| `SSH_FXP_OPEN` | 3 | C→S | Open a file |
|
||||
| `SSH_FXP_CLOSE` | 4 | C→S | Close a handle |
|
||||
| `SSH_FXP_READ` | 5 | C→S | Read from a handle |
|
||||
| `SSH_FXP_WRITE` | 6 | C→S | Write to a handle |
|
||||
| `SSH_FXP_LSTAT` | 7 | C→S | Stat a path (no follow) |
|
||||
| `SSH_FXP_FSTAT` | 8 | C→S | Stat an open handle |
|
||||
| `SSH_FXP_SETSTAT` | 9 | C→S | Set file attributes by path |
|
||||
| `SSH_FXP_FSETSTAT` | 10 | C→S | Set file attributes by handle |
|
||||
| `SSH_FXP_OPENDIR` | 11 | C→S | Open a directory |
|
||||
| `SSH_FXP_READDIR` | 12 | C→S | Read directory entries |
|
||||
| `SSH_FXP_REMOVE` | 13 | C→S | Remove a file |
|
||||
| `SSH_FXP_MKDIR` | 14 | C→S | Create a directory |
|
||||
| `SSH_FXP_RMDIR` | 15 | C→S | Remove a directory |
|
||||
| `SSH_FXP_REALPATH` | 16 | C→S | Canonicalize a path |
|
||||
| `SSH_FXP_STAT` | 17 | C→S | Stat a path (follow symlinks) |
|
||||
| `SSH_FXP_RENAME` | 18 | C→S | Rename a file |
|
||||
| `SSH_FXP_READLINK` | 19 | C→S | Read a symbolic link |
|
||||
| `SSH_FXP_SYMLINK` | 20 | C→S | Create a symbolic link |
|
||||
| `SSH_FXP_STATUS` | 101 | S→C / C→S | Status response |
|
||||
| `SSH_FXP_HANDLE` | 102 | S→C | Handle response |
|
||||
| `SSH_FXP_DATA` | 103 | S→C | Data response |
|
||||
| `SSH_FXP_NAME` | 104 | S→C | Name list response |
|
||||
| `SSH_FXP_ATTRS` | 105 | S→C | File attributes response |
|
||||
| `SSH_FXP_EXTENDED` | 200 | C→S | Extended request |
|
||||
| `SSH_FXP_EXTENDED_REPLY` | 201 | S→C | Extended reply |
|
||||
|
||||
## Packet Reading
|
||||
|
||||
Wire I/O is handled by `utils::read_packet()`:
|
||||
|
||||
```rust
|
||||
pub(crate) async fn read_packet<S: AsyncRead + Unpin>(
|
||||
stream: &mut S,
|
||||
max_length: u32,
|
||||
) -> Result<Bytes, Error> {
|
||||
let length = stream.read_u32().await?;
|
||||
if length > max_length {
|
||||
return Err(Error::BadMessage("packet length limit exceeded".to_owned()));
|
||||
}
|
||||
let mut buf = vec![0; length as usize];
|
||||
stream.read_exact(&mut buf).await?;
|
||||
Ok(Bytes::from(buf))
|
||||
}
|
||||
```
|
||||
|
||||
The read packet buffer **includes the type byte** as the first byte, followed by the payload. This design means the caller can distinguish packet types before full deserialization.
|
||||
|
||||
## Packet Enum and Dispatch
|
||||
|
||||
All packets are unified into a single `Packet` enum:
|
||||
|
||||
```rust
|
||||
pub enum Packet {
|
||||
Init(Init), Version(Version), Open(Open),
|
||||
Close(Close), Read(Read), Write(Write),
|
||||
Lstat(Lstat), Fstat(Fstat), SetStat(SetStat),
|
||||
FSetStat(FSetStat), OpenDir(OpenDir), ReadDir(ReadDir),
|
||||
Remove(Remove), MkDir(MkDir), RmDir(RmDir),
|
||||
RealPath(RealPath), Stat(Stat), Rename(Rename),
|
||||
ReadLink(ReadLink), Symlink(Symlink), Status(Status),
|
||||
Handle(Handle), Data(Data), Name(Name),
|
||||
Attrs(Attrs), Extended(Extended), ExtendedReply(ExtendedReply),
|
||||
}
|
||||
```
|
||||
|
||||
### Deserialization (`TryFrom<&mut Bytes> for Packet`)
|
||||
|
||||
Reads the type byte first, then delegates to the custom serde deserializer:
|
||||
|
||||
```rust
|
||||
fn try_from(bytes: &mut Bytes) -> Result<Self, Self::Error> {
|
||||
let r#type = bytes.try_get_u8()?;
|
||||
match r#type {
|
||||
SSH_FXP_INIT => Self::Init(de::from_bytes(bytes)?),
|
||||
SSH_FXP_OPEN => Self::Open(de::from_bytes(bytes)?),
|
||||
// ... all 26 variants
|
||||
_ => Err(Error::BadMessage("unknown type".to_owned())),
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Serialization (`TryFrom<Packet> for Bytes`)
|
||||
|
||||
Converts each variant to bytes via `ser::to_bytes()`, prepends type byte, and wraps with the 4-byte length:
|
||||
|
||||
```rust
|
||||
fn try_from(packet: Packet) -> Result<Self, Self::Error> {
|
||||
let (r#type, payload): (u8, Bytes) = match packet {
|
||||
Packet::Init(init) => (SSH_FXP_INIT, ser::to_bytes(&init)?),
|
||||
Packet::Open(open) => (SSH_FXP_OPEN, ser::to_bytes(&open)?),
|
||||
// ... all variants
|
||||
};
|
||||
let length = payload.len() as u32 + 1;
|
||||
let mut bytes = BytesMut::new();
|
||||
bytes.put_u32(length);
|
||||
bytes.put_u8(r#type);
|
||||
bytes.put_slice(&payload);
|
||||
Ok(bytes.freeze())
|
||||
}
|
||||
```
|
||||
|
||||
## Custom Serde Wire Codec
|
||||
|
||||
The crate implements a **custom serde `Serializer` and `Deserializer`** that directly maps Rust types to the SFTP binary format. This is NOT JSON, Bincode, or any standard serde format — it is a bespoke binary encoding matching the SFTP v3 wire specification.
|
||||
|
||||
### Serializer (`ser.rs`)
|
||||
|
||||
The `Serializer` writes directly into a `BytesMut` buffer:
|
||||
|
||||
| Rust Type | Wire Encoding |
|
||||
|-----------|---------------|
|
||||
| `u8` | 1 byte raw |
|
||||
| `u32` | 4 bytes big-endian |
|
||||
| `u64` | 8 bytes big-endian |
|
||||
| `str` / `String` | `u32 length` + UTF-8 bytes |
|
||||
| `bytes` | `u32 length` + raw bytes |
|
||||
| `struct` | Fields concatenated in order (no field names) |
|
||||
| `seq` | `u32 count` + elements |
|
||||
| `map` | Key-value pairs (no length prefix) |
|
||||
| `enum` | Variant index as `u32` + variant content |
|
||||
| `None` | Nothing (zero bytes) |
|
||||
| `Some(T)` | Serialized as `T` |
|
||||
| `bool`, `i8`–`i64`, `u16`, `f32`/`f64`, `char` | **Not supported** — returns `BadMessage` error |
|
||||
|
||||
Key detail: `struct` serialization uses `serialize_struct` which delegates to `serialize_tuple` — fields are written in declaration order with **no field names or tags**. This matches SFTP's positional binary layout.
|
||||
|
||||
The `data_serialize` helper serializes `Vec<u8>` as a raw byte sequence **without** a length prefix (used for `Extended.data` and `ExtendedReply.data`).
|
||||
|
||||
### Deserializer (`de.rs`)
|
||||
|
||||
The `Deserializer` reads from a `&mut Bytes` buffer, consuming bytes as it goes:
|
||||
|
||||
| Wire Pattern | Rust Deserialize Target |
|
||||
|--------------|------------------------|
|
||||
| 1 byte | `u8` |
|
||||
| 4 bytes BE | `u32` |
|
||||
| 8 bytes BE | `u64` |
|
||||
| `u32 len` + bytes | `String` / `str` |
|
||||
| `u32 len` + bytes | `Vec<u8>` / byte buf |
|
||||
| `u32 count` + elements | `Vec<T>` / seq |
|
||||
| Positional fields | struct (tuple-like) |
|
||||
| `u32 variant` + content | enum |
|
||||
| Key-value pairs | `HashMap` |
|
||||
|
||||
The `data_deserialize` helper reads all remaining bytes into a `Vec<u8>` (no length prefix) — used for `Extended.data` and `ExtendedReply.data`.
|
||||
|
||||
### TryBuf Helper (`buf.rs`)
|
||||
|
||||
A small extension trait on `bytes::Buf`:
|
||||
|
||||
```rust
|
||||
pub trait TryBuf: Buf {
|
||||
fn try_get_bytes(&mut self) -> Result<Vec<u8>, Error>; // u32-length-prefixed
|
||||
fn try_get_string(&mut self) -> Result<String, Error>; // u32-length-prefixed UTF-8
|
||||
}
|
||||
```
|
||||
|
||||
These are used internally by the deserializer for reading SFTP's length-prefixed byte and string fields.
|
||||
|
||||
## FileAttributes Serialization
|
||||
|
||||
`FileAttributes` has a custom `Serialize`/`Deserialize` implementation because the SFTP wire format uses a **flags bitmask** to indicate which optional fields are present. This is fundamentally different from serde's typical self-describing formats.
|
||||
|
||||
### Serialization Flow
|
||||
|
||||
1. Compute `FileAttr` flags bitmask based on which `Option` fields are `Some`:
|
||||
- `SIZE` (0x1) — `size` is present
|
||||
- `UIDGID` (0x2) — `uid`/`gid` are present
|
||||
- `PERMISSIONS` (0x4) — `permissions` is present
|
||||
- `ACMODTIME` (0x8) — `atime`/`mtime` are present
|
||||
- `EXTENDED` (0x80000000) — extended fields (not yet implemented)
|
||||
2. Write flags as `u32`
|
||||
3. Write fields conditionally based on flags
|
||||
|
||||
### Deserialization Flow
|
||||
|
||||
1. Read `u32` flags bitmask
|
||||
2. Conditionally read fields based on which bits are set:
|
||||
- If `SIZE`: read `u64` for `size`
|
||||
- If `UIDGID`: read `u32` for `uid`, `u32` for `gid`
|
||||
- If `PERMISSIONS`: read `u32` for `permissions`
|
||||
- If `ACMODTIME`: read `u32` for `atime`, `u32` for `mtime`
|
||||
|
||||
This ensures that fields not flagged are left as `None` in the `FileAttributes` struct.
|
||||
|
||||
## Request ID Tracking
|
||||
|
||||
All request packets (except `Init`) carry a `u32 id` field used as a request identifier. The `RequestId` trait and macro provide uniform access:
|
||||
|
||||
```rust
|
||||
pub(crate) trait RequestId: Sized {
|
||||
fn get_request_id(&self) -> u32;
|
||||
}
|
||||
|
||||
macro_rules! impl_request_id {
|
||||
($packet:ty) => {
|
||||
impl RequestId for $packet {
|
||||
fn get_request_id(&self) -> u32 { self.id }
|
||||
}
|
||||
};
|
||||
}
|
||||
```
|
||||
|
||||
This is used by the server to extract the request ID for constructing status responses on error, and by the client for demultiplexing responses.
|
||||
356
docs/research/references/ssh/russh-sftp/03-key-types.md
Normal file
356
docs/research/references/ssh/russh-sftp/03-key-types.md
Normal file
@@ -0,0 +1,356 @@
|
||||
# russh-sftp: Key Types
|
||||
|
||||
## Protocol Types (`protocol/`)
|
||||
|
||||
### Init and Version (Handshake)
|
||||
|
||||
```rust
|
||||
// SSH_FXP_INIT — sent by client to begin
|
||||
pub struct Init {
|
||||
pub version: u32, // Always 3
|
||||
pub extensions: HashMap<String, String>,
|
||||
}
|
||||
|
||||
// SSH_FXP_VERSION — server response
|
||||
pub struct Version {
|
||||
pub version: u32,
|
||||
pub extensions: HashMap<String, String>,
|
||||
}
|
||||
```
|
||||
|
||||
Both implement `Default` with `version: 3` and empty extensions. Extensions are negotiated during the handshake (e.g., `"limits@openssh.com" → "1"`).
|
||||
|
||||
### Open and OpenFlags
|
||||
|
||||
```rust
|
||||
// SSH_FXP_OPEN — open or create a file
|
||||
pub struct Open {
|
||||
pub id: u32,
|
||||
pub filename: String,
|
||||
pub pflags: OpenFlags, // Bitflags for access mode
|
||||
pub attrs: FileAttributes, // Initial attributes for new files
|
||||
}
|
||||
|
||||
bitflags! {
|
||||
pub struct OpenFlags: u32 {
|
||||
const READ = 0x00000001;
|
||||
const WRITE = 0x00000002;
|
||||
const APPEND = 0x00000004;
|
||||
const CREATE = 0x00000008;
|
||||
const TRUNCATE = 0x00000010;
|
||||
const EXCLUDE = 0x00000020;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
`OpenFlags` implements `From<OpenFlags> for std::fs::OpenOptions`, converting SFTP flags into Rust's `OpenOptions` for server implementations. Notable behavior: if both `CREATE` and `EXCLUDE` are set, it maps to `create_new(true)`; otherwise `CREATE` maps to `create(true)`.
|
||||
|
||||
### Handle
|
||||
|
||||
```rust
|
||||
// SSH_FXP_HANDLE — server returns a handle string for open files/dirs
|
||||
pub struct Handle {
|
||||
pub id: u32,
|
||||
pub handle: String,
|
||||
}
|
||||
```
|
||||
|
||||
Handles are opaque strings identifying open file or directory references. They are returned by `SSH_FXP_OPEN` and `SSH_FXP_OPENDIR` responses, and used in subsequent `READ`, `WRITE`, `FSTAT`, `FSETSTAT`, `READDIR`, `FSYNC`, and `CLOSE` operations.
|
||||
|
||||
### Data and Write
|
||||
|
||||
```rust
|
||||
// SSH_FXP_DATA — file data response
|
||||
pub struct Data {
|
||||
pub id: u32,
|
||||
pub data: Vec<u8>, // serde_bytes — no length prefix in inner serialization
|
||||
}
|
||||
|
||||
// SSH_FXP_WRITE — write data to file
|
||||
pub struct Write {
|
||||
pub id: u32,
|
||||
pub handle: String,
|
||||
pub offset: u64,
|
||||
pub data: Vec<u8>, // serde_bytes
|
||||
}
|
||||
```
|
||||
|
||||
Both use `serde_bytes` for the `data` field, which serializes as a length-prefixed byte array in the outer packet encoding.
|
||||
|
||||
### Name and File
|
||||
|
||||
```rust
|
||||
// SSH_FXP_NAME — directory listing / path resolution response
|
||||
pub struct Name {
|
||||
pub id: u32,
|
||||
pub files: Vec<File>,
|
||||
}
|
||||
|
||||
// Represents a single file entry
|
||||
pub struct File {
|
||||
pub filename: String,
|
||||
pub longname: String, // `ls -l` style long name
|
||||
pub attrs: FileAttributes,
|
||||
}
|
||||
```
|
||||
|
||||
`File` provides constructors:
|
||||
- `File::dummy(filename)` — Creates a file with empty longname and default attributes (for `realpath` responses per spec)
|
||||
- `File::new(filename, attrs)` — Creates a file with auto-generated longname from attributes
|
||||
|
||||
The `longname()` method generates an `ls -l`-style string: `"{type}{permissions} 0 {user} {group} {size} {mtime} {filename}"`
|
||||
|
||||
### Attrs
|
||||
|
||||
```rust
|
||||
// SSH_FXP_ATTRS — file attributes response
|
||||
pub struct Attrs {
|
||||
pub id: u32,
|
||||
pub attrs: FileAttributes,
|
||||
}
|
||||
```
|
||||
|
||||
### Status and StatusCode
|
||||
|
||||
```rust
|
||||
pub enum StatusCode {
|
||||
Ok = 0, // Successful completion
|
||||
Eof = 1, // End of file / no more directory entries
|
||||
NoSuchFile = 2, // File does not exist
|
||||
PermissionDenied = 3, // Permission denied
|
||||
Failure = 4, // Generic failure
|
||||
BadMessage = 5, // Badly formatted packet
|
||||
NoConnection = 6, // Client-side only: no connection
|
||||
ConnectionLost = 7, // Client-side only: connection lost
|
||||
OpUnsupported = 8, // Operation not supported
|
||||
}
|
||||
|
||||
pub struct Status {
|
||||
pub id: u32,
|
||||
pub status_code: StatusCode,
|
||||
pub error_message: String,
|
||||
pub language_tag: String, // e.g., "en-US"
|
||||
}
|
||||
```
|
||||
|
||||
`StatusCode` derives `Error` (thiserror), providing human-readable `Display` output for each variant.
|
||||
|
||||
### FileAttributes
|
||||
|
||||
The core metadata type. See the wire codec doc for serialization details.
|
||||
|
||||
```rust
|
||||
pub struct FileAttributes {
|
||||
pub size: Option<u64>,
|
||||
pub uid: Option<u32>,
|
||||
pub user: Option<String>, // User name for longname display
|
||||
pub gid: Option<u32>,
|
||||
pub group: Option<String>, // Group name for longname display
|
||||
pub permissions: Option<u32>, // Unix permission + file type bits
|
||||
pub atime: Option<u32>, // Access time (unix epoch)
|
||||
pub mtime: Option<u32>, // Modification time (unix epoch)
|
||||
}
|
||||
```
|
||||
|
||||
Key methods:
|
||||
- `is_dir()`, `is_regular()`, `is_symlink()`, `is_character()`, `is_block()`, `is_fifo()` — check `FileMode` bits
|
||||
- `set_dir()`, `set_regular()`, etc. — set `FileMode` bits
|
||||
- `file_type()` → `FileType` — simplified type classification
|
||||
- `len()` → `u64` — file size (defaults to 0)
|
||||
- `permissions()` → `FilePermissions` — simplified permission struct
|
||||
- `accessed()` → `io::Result<SystemTime>` — convert atime
|
||||
- `modified()` → `io::Result<SystemTime>` — convert mtime
|
||||
- `empty()` — all fields `None`
|
||||
- `From<&std::fs::Metadata>` — convert OS metadata (unix-specific for uid/gid/mode)
|
||||
|
||||
#### Supporting Bitflag Types
|
||||
|
||||
```rust
|
||||
bitflags! {
|
||||
pub struct FileAttr: u32 {
|
||||
const SIZE = 0x00000001;
|
||||
const UIDGID = 0x00000002;
|
||||
const PERMISSIONS = 0x00000004;
|
||||
const ACMODTIME = 0x00000008;
|
||||
const EXTENDED = 0x80000000;
|
||||
}
|
||||
|
||||
pub struct FileMode: u32 {
|
||||
const FIFO = 0x1000; // Named pipe
|
||||
const CHR = 0x2000; // Character device
|
||||
const DIR = 0x4000; // Directory
|
||||
const NAM = 0x5000; // Named file (rare)
|
||||
const BLK = 0x6000; // Block device
|
||||
const REG = 0x8000; // Regular file
|
||||
const LNK = 0xA000; // Symbolic link
|
||||
const SOCK = 0xC000; // Socket
|
||||
}
|
||||
|
||||
pub struct FilePermissionFlags: u32 {
|
||||
const OTHER_READ = 0o4;
|
||||
const OTHER_WRITE = 0o2;
|
||||
const OTHER_EXEC = 0o1;
|
||||
const GROUP_READ = 0o40;
|
||||
const GROUP_WRITE = 0o20;
|
||||
const GROUP_EXEC = 0o10;
|
||||
const OWNER_READ = 0o400;
|
||||
const OWNER_WRITE = 0o200;
|
||||
const OWNER_EXEC = 0o100;
|
||||
}
|
||||
}
|
||||
|
||||
pub enum FileType { Dir, File, Symlink, Other }
|
||||
|
||||
pub struct FilePermissions {
|
||||
pub other_exec: bool, pub other_read: bool, pub other_write: bool,
|
||||
pub group_exec: bool, pub group_read: bool, pub group_write: bool,
|
||||
pub owner_exec: bool, pub owner_read: bool, pub owner_write: bool,
|
||||
}
|
||||
```
|
||||
|
||||
### Extended and ExtendedReply
|
||||
|
||||
```rust
|
||||
pub struct Extended {
|
||||
pub id: u32,
|
||||
pub request: String, // Extension name, e.g., "limits@openssh.com"
|
||||
pub data: Vec<u8>, // serde_bytes, no inner length prefix
|
||||
}
|
||||
|
||||
pub struct ExtendedReply {
|
||||
pub id: u32,
|
||||
pub data: Vec<u8>, // serde_bytes
|
||||
}
|
||||
```
|
||||
|
||||
## Other Protocol Packets
|
||||
|
||||
All follow the same pattern of `id: u32` plus operation-specific fields:
|
||||
|
||||
| Packet | Fields |
|
||||
|--------|--------|
|
||||
| `Close` | `id`, `handle` |
|
||||
| `Read` | `id`, `handle`, `offset: u64`, `len: u32` |
|
||||
| `Lstat` | `id`, `path` |
|
||||
| `Fstat` | `id`, `handle` |
|
||||
| `SetStat` | `id`, `path`, `attrs` |
|
||||
| `FSetStat` | `id`, `handle`, `attrs` |
|
||||
| `OpenDir` | `id`, `path` |
|
||||
| `ReadDir` | `id`, `handle` |
|
||||
| `Remove` | `id`, `filename` |
|
||||
| `MkDir` | `id`, `path`, `attrs` |
|
||||
| `RmDir` | `id`, `path` |
|
||||
| `RealPath` | `id`, `path` |
|
||||
| `Stat` | `id`, `path` |
|
||||
| `Rename` | `id`, `oldpath`, `newpath` |
|
||||
| `ReadLink` | `id`, `path` |
|
||||
| `Symlink` | `id`, `linkpath`, `targetpath` |
|
||||
|
||||
## Extension Types (`extensions.rs`)
|
||||
|
||||
OpenSSH extension constants and structures:
|
||||
|
||||
```rust
|
||||
pub const LIMITS: &str = "limits@openssh.com";
|
||||
pub const HARDLINK: &str = "hardlink@openssh.com";
|
||||
pub const FSYNC: &str = "fsync@openssh.com";
|
||||
pub const STATVFS: &str = "statvfs@openssh.com";
|
||||
|
||||
// Server limits advertisement
|
||||
pub struct LimitsExtension {
|
||||
pub max_packet_len: u64,
|
||||
pub max_read_len: u64,
|
||||
pub max_write_len: u64,
|
||||
pub max_open_handles: u64,
|
||||
}
|
||||
|
||||
// Hardlink request data
|
||||
pub struct HardlinkExtension {
|
||||
pub oldpath: String,
|
||||
pub newpath: String,
|
||||
}
|
||||
|
||||
// Fsync request data
|
||||
pub struct FsyncExtension {
|
||||
pub handle: String,
|
||||
}
|
||||
|
||||
// Statvfs request data
|
||||
pub struct StatvfsExtension {
|
||||
pub path: String,
|
||||
}
|
||||
|
||||
// Statvfs response
|
||||
pub struct Statvfs {
|
||||
pub block_size: u64,
|
||||
pub fragment_size: u64,
|
||||
pub blocks: u64,
|
||||
pub blocks_free: u64,
|
||||
pub blocks_avail: u64,
|
||||
pub inodes: u64,
|
||||
pub inodes_free: u64,
|
||||
pub inodes_avail: u64,
|
||||
pub fs_id: u64,
|
||||
pub flags: u64,
|
||||
pub name_max: u64,
|
||||
}
|
||||
```
|
||||
|
||||
## Error Types
|
||||
|
||||
### Top-level Error (`error.rs`)
|
||||
|
||||
```rust
|
||||
pub enum Error {
|
||||
IO(String), // I/O errors
|
||||
UnexpectedEof, // Stream EOF
|
||||
BadMessage(String), // Malformed packet
|
||||
Client(String), // Wraps client::error::Error
|
||||
UnexpectedBehavior(String), // Protocol violations
|
||||
}
|
||||
```
|
||||
|
||||
### Client Error (`client/error.rs`)
|
||||
|
||||
```rust
|
||||
pub enum Error {
|
||||
Status(Status), // Server returned error status
|
||||
IO(String), // I/O errors
|
||||
Timeout, // Request timed out
|
||||
Limited(String), // Limits exceeded
|
||||
UnexpectedPacket, // Wrong packet type received
|
||||
UnexpectedBehavior(String), // Protocol violations
|
||||
}
|
||||
```
|
||||
|
||||
### StatusReply (Server-side error conversion, `server/reply.rs`)
|
||||
|
||||
```rust
|
||||
pub struct StatusReply {
|
||||
pub status_code: StatusCode,
|
||||
pub error_message: Option<String>,
|
||||
pub language_tag: Option<String>,
|
||||
}
|
||||
```
|
||||
|
||||
Server `Handler` errors must implement `Into<StatusReply>`, which allows the framework to convert any handler error into a `SSH_FXP_STATUS` response. `StatusCode` has a `.with_message()` helper.
|
||||
|
||||
## Config Types
|
||||
|
||||
### Client Config (`client/mod.rs`)
|
||||
|
||||
```rust
|
||||
pub struct Config {
|
||||
pub max_packet_len: u32, // Default: 262144 (256 KiB)
|
||||
pub max_concurrent_writes: usize, // Default: 8
|
||||
pub request_timeout_secs: u64, // Default: 10
|
||||
}
|
||||
```
|
||||
|
||||
### Server Config (`server/mod.rs`)
|
||||
|
||||
```rust
|
||||
pub struct Config {
|
||||
pub max_client_packet_len: u32, // Default: 262144 (256 KiB)
|
||||
}
|
||||
```
|
||||
362
docs/research/references/ssh/russh-sftp/04-client-api.md
Normal file
362
docs/research/references/ssh/russh-sftp/04-client-api.md
Normal file
@@ -0,0 +1,362 @@
|
||||
# russh-sftp: Client Implementation
|
||||
|
||||
## Client Architecture Overview
|
||||
|
||||
The client side provides two tiers of API:
|
||||
|
||||
1. **`RawSftpSession`** — Low-level request-response client that sends individual SFTP packets and awaits their responses. Suitable for custom or non-standard SFTP interactions.
|
||||
|
||||
2. **`SftpSession`** — High-level client modeled after `std::fs`. Provides ergonomic methods for file operations and creates `File` objects implementing `AsyncRead`/`AsyncWrite`/`AsyncSeek`.
|
||||
|
||||
Both operate on a stream type `S: AsyncRead + AsyncWrite + Unpin + Send + 'static`.
|
||||
|
||||
## Client Handler Trait
|
||||
|
||||
The raw client infrastructure uses an internal `Handler` trait to dispatch incoming response packets:
|
||||
|
||||
```rust
|
||||
pub trait Handler: Sized {
|
||||
type Error: Into<Error>;
|
||||
|
||||
fn version(&mut self, version: Version) -> impl Future<Output = Result<(), Self::Error>> + Send;
|
||||
fn status(&mut self, status: Status) -> impl Future<Output = Result<(), Self::Error>> + Send;
|
||||
fn handle(&mut self, handle: Handle) -> impl Future<Output = Result<(), Self::Error>> + Send;
|
||||
fn data(&mut self, data: Data) -> impl Future<Output = Result<(), Self::Error>> + Send;
|
||||
fn name(&mut self, name: Name) -> impl Future<Output = Result<(), Self::Error>> + Send;
|
||||
fn attrs(&mut self, attrs: Attrs) -> impl Future<Output = Result<(), Self::Error>> + Send;
|
||||
fn extended_reply(&mut self, reply: ExtendedReply) -> impl Future<Output = Result<(), Self::Error>> + Send;
|
||||
}
|
||||
```
|
||||
|
||||
With the `async-trait` feature enabled, this becomes `#[async_trait]`.
|
||||
|
||||
The `run()` function spawns two tasks:
|
||||
- **Reader task**: reads packets from the stream, dispatches to the handler
|
||||
- **Writer task**: sends serialized packets from an `mpsc::UnboundedSender<Bytes>`
|
||||
|
||||
```rust
|
||||
pub fn run<S, H>(stream: S, handler: H) -> mpsc::UnboundedSender<Bytes>
|
||||
where
|
||||
S: AsyncRead + AsyncWrite + Unpin + Send + 'static,
|
||||
H: Handler + Send + 'static,
|
||||
```
|
||||
|
||||
The returned sender is the write handle — callers push `Bytes` (serialized packets) into it.
|
||||
|
||||
## RawSftpSession
|
||||
|
||||
`RawSftpSession` is the core request-response client. It implements `Handler` internally via `SessionInner` to route response packets back to awaiting request futures.
|
||||
|
||||
### Construction
|
||||
|
||||
```rust
|
||||
impl RawSftpSession {
|
||||
pub fn new<S>(stream: S) -> Self
|
||||
where S: AsyncRead + AsyncWrite + Unpin + Send + 'static;
|
||||
|
||||
pub fn new_with_config<S>(stream: S, cfg: Config) -> Self
|
||||
where S: AsyncRead + AsyncWrite + Unpin + Send + 'static;
|
||||
}
|
||||
```
|
||||
|
||||
### Internal Architecture
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────┐
|
||||
│ RawSftpSession │
|
||||
│ │
|
||||
│ tx: UnboundedSender<Bytes> ←── write to stream │
|
||||
│ requests: Arc<DashMap<Option<u32>, oneshot::Sender>>│
|
||||
│ next_req_id: AtomicU32 │
|
||||
│ handles: AtomicU64 │
|
||||
│ timeout: AtomicU64 │
|
||||
│ limits: Limits │
|
||||
│ │
|
||||
│ ┌──────────────────────────────────────────────┐ │
|
||||
│ │ SessionInner (Handler impl) │ │
|
||||
│ │ │ │
|
||||
│ │ version() → stores version, replies to init │ │
|
||||
│ │ status() → replies by request ID │ │
|
||||
│ │ handle() → replies by request ID │ │
|
||||
│ │ data() → replies by request ID │ │
|
||||
│ │ name() → replies by request ID │ │
|
||||
│ │ attrs() → replies by request ID │ │
|
||||
│ │ extended_reply() → replies by request ID │ │
|
||||
│ └──────────────────────────────────────────────┘ │
|
||||
└─────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
### Request-Response Mechanism
|
||||
|
||||
1. Caller invokes a method like `raw.open(filename, flags, attrs).await`
|
||||
2. `RawSftpSession` generates a unique `id` via `AtomicU32::fetch_add`
|
||||
3. Creates a `oneshot::channel` and inserts `(Some(id), sender)` into the `DashMap`
|
||||
4. Serializes the packet to `Bytes` and sends it via the `tx` channel
|
||||
5. Awaits the `oneshot::Receiver` with a timeout
|
||||
6. When the reader task receives a response packet, `SessionInner::reply()` removes the entry from the `DashMap` and sends the `Packet` through the `oneshot` sender
|
||||
7. The original request future receives the `Packet` and pattern-matches it
|
||||
|
||||
The `init()` method is special: it uses `id: None` (no request ID) and the version handler stores the negotiated version.
|
||||
|
||||
### Limits Tracking
|
||||
|
||||
```rust
|
||||
pub struct Limits {
|
||||
pub packet_len: Option<u64>,
|
||||
pub read_len: Option<u64>,
|
||||
pub write_len: Option<u64>,
|
||||
pub open_handles: Option<u64>,
|
||||
}
|
||||
```
|
||||
|
||||
- Fetched from the server via the `limits@openssh.com` extension during `SftpSession::new_with_config()`
|
||||
- Enforced client-side: `open()` and `opendir()` check `open_handles`, `read()` checks `read_len`, `write()` and `write_nowait()` check `write_len`, `send()` checks `packet_len`
|
||||
|
||||
### API Methods
|
||||
|
||||
| Method | SFTP Packet | Response | Notes |
|
||||
|--------|------------|----------|-------|
|
||||
| `init()` | `Init` | `Version` | Handshake; version negotiation |
|
||||
| `open(filename, flags, attrs)` | `Open` | `Handle` | Opens a file; increments handle count |
|
||||
| `close(handle)` | `Close` | `Status` | Closes a handle; decrements handle count |
|
||||
| `close_nowait(handle)` | `Close` | — | Fire-and-forget close (returns oneshot receiver) |
|
||||
| `read(handle, offset, len)` | `Read` | `Data` | Reads data; respects `Limits.read_len` |
|
||||
| `write(handle, offset, data)` | `Write` | `Status` | Writes data; respects `Limits.write_len` |
|
||||
| `write_nowait(handle, offset, data)` | `Write` | — | Fire-and-forget write |
|
||||
| `lstat(path)` | `Lstat` | `Attrs` | Stat without following symlinks |
|
||||
| `fstat(handle)` | `Fstat` | `Attrs` | Stat an open handle |
|
||||
| `setstat(path, attrs)` | `SetStat` | `Status` | Set file attributes by path |
|
||||
| `fsetstat(handle, attrs)` | `FSetStat` | `Status` | Set file attributes by handle |
|
||||
| `opendir(path)` | `OpenDir` | `Handle` | Opens a directory; increments handle count |
|
||||
| `readdir(handle)` | `ReadDir` | `Name` | Reads directory entries |
|
||||
| `remove(filename)` | `Remove` | `Status` | Deletes a file |
|
||||
| `mkdir(path, attrs)` | `MkDir` | `Status` | Creates a directory |
|
||||
| `rmdir(path)` | `RmDir` | `Status` | Removes a directory |
|
||||
| `realpath(path)` | `RealPath` | `Name` | Canonicalizes a path |
|
||||
| `stat(path)` | `Stat` | `Status` | Stat following symlinks |
|
||||
| `rename(oldpath, newpath)` | `Rename` | `Status` | Renames a file |
|
||||
| `readlink(path)` | `ReadLink` | `Name` | Reads symlink target |
|
||||
| `symlink(path, target)` | `Symlink` | `Status` | Creates a symbolic link |
|
||||
| `extended(request, data)` | `Extended` | `Packet` | Generic extension mechanism |
|
||||
| `limits()` | `Extended` | `LimitsExtension` | `limits@openssh.com` |
|
||||
| `hardlink(oldpath, newpath)` | `Extended` | `Status` | `hardlink@openssh.com` |
|
||||
| `fsync(handle)` | `Extended` | `Status` | `fsync@openssh.com` |
|
||||
| `statvfs(path)` | `Extended` | `Statvfs` | `statvfs@openssh.com` v2 |
|
||||
|
||||
### Response Classification
|
||||
|
||||
Two macros handle the common response patterns:
|
||||
|
||||
```rust
|
||||
// For responses that return a specific packet type or error status
|
||||
macro_rules! into_with_status {
|
||||
($result:ident, $packet:ident) => {
|
||||
match $result {
|
||||
Packet::$packet(p) => Ok(p),
|
||||
Packet::Status(p) => Err(p.into()),
|
||||
_ => Err(Error::UnexpectedPacket),
|
||||
}
|
||||
};
|
||||
}
|
||||
|
||||
// For responses that only return success/failure status
|
||||
macro_rules! into_status {
|
||||
($result:ident) => {
|
||||
match $result {
|
||||
Packet::Status(status) if status.status_code == StatusCode::Ok => Ok(status),
|
||||
Packet::Status(status) => Err(status.into()),
|
||||
_ => Err(Error::UnexpectedPacket),
|
||||
}
|
||||
};
|
||||
}
|
||||
```
|
||||
|
||||
### Drop Behavior
|
||||
|
||||
`RawSftpSession` implements `Drop` to call `close_session()`, which sends an empty `Bytes` to signal the writer task to shut down the stream.
|
||||
|
||||
## SftpSession (High-Level Client)
|
||||
|
||||
`SftpSession` wraps `RawSftpSession` in an `Arc` and provides `std::fs`-like methods:
|
||||
|
||||
```rust
|
||||
pub struct SftpSession {
|
||||
session: Arc<RawSftpSession>,
|
||||
features: Features,
|
||||
}
|
||||
```
|
||||
|
||||
### Construction and Version Negotiation
|
||||
|
||||
```rust
|
||||
impl SftpSession {
|
||||
pub async fn new<S>(stream: S) -> SftpResult<Self>;
|
||||
pub async fn new_with_config<S>(stream: S, cfg: Config) -> SftpResult<Self>;
|
||||
}
|
||||
```
|
||||
|
||||
Construction performs:
|
||||
1. Creates `RawSftpSession` with config
|
||||
2. Sends `SSH_FXP_INIT` via `session.init().await`
|
||||
3. Checks for supported extensions in the version response
|
||||
4. If `limits@openssh.com` extension is available, fetches and applies limits
|
||||
5. Stores detected feature flags in `Features`
|
||||
|
||||
### Features Detection
|
||||
|
||||
```rust
|
||||
pub(crate) struct Features {
|
||||
pub hardlink: bool, // hardlink@openssh.com v1
|
||||
pub fsync: bool, // fsync@openssh.com v1
|
||||
pub statvfs: bool, // statvfs@openssh.com v2
|
||||
pub limits: Option<Limits>, // from limits@openssh.com
|
||||
pub max_concurrent_writes: usize,
|
||||
pub max_packet_len: u32,
|
||||
}
|
||||
```
|
||||
|
||||
### High-Level API
|
||||
|
||||
| Method | Description |
|
||||
|--------|-------------|
|
||||
| `open(filename)` | Opens file read-only |
|
||||
| `create(filename)` | Creates/truncates file write-only |
|
||||
| `open_with_flags(filename, flags)` | Opens with specified `OpenFlags` |
|
||||
| `open_with_flags_and_attributes(filename, flags, attrs)` | Opens with flags and initial attrs |
|
||||
| `canonicalize(path)` | Resolves to absolute path via `realpath` |
|
||||
| `create_dir(path)` | Creates directory |
|
||||
| `read(path)` | Reads entire file to `Vec<u8>` |
|
||||
| `write(path, data)` | Writes data to file |
|
||||
| `try_exists(path)` | Checks existence (returns `Ok(false)` on `NoSuchFile`) |
|
||||
| `read_dir(path)` | Returns `ReadDir` iterator over directory entries |
|
||||
| `read_link(path)` | Reads symlink target |
|
||||
| `remove_dir(path)` | Removes directory |
|
||||
| `remove_file(filename)` | Removes file |
|
||||
| `rename(oldpath, newpath)` | Renames file/directory |
|
||||
| `symlink(path, target)` | Creates symbolic link |
|
||||
| `metadata(path)` | Gets `FileAttributes` via `stat` |
|
||||
| `set_metadata(path, metadata)` | Sets attributes via `setstat` |
|
||||
| `symlink_metadata(path)` | Gets `FileAttributes` via `lstat` |
|
||||
| `hardlink(oldpath, newpath)` | Creates hard link (returns `Ok(false)` if unsupported) |
|
||||
| `fs_info(path)` | Gets filesystem stats via `statvfs` (returns `Ok(None)` if unsupported) |
|
||||
| `set_timeout(secs)` | Sets request timeout |
|
||||
| `close()` | Closes the SFTP session stream |
|
||||
|
||||
## File (Async I/O)
|
||||
|
||||
`client::fs::File` implements `AsyncRead`, `AsyncWrite`, and `AsyncSeek` for remote file I/O:
|
||||
|
||||
```rust
|
||||
pub struct File {
|
||||
session: Arc<RawSftpSession>,
|
||||
handle: String,
|
||||
state: FileState,
|
||||
pos: u64,
|
||||
closed: bool,
|
||||
features: Features,
|
||||
}
|
||||
|
||||
struct FileState {
|
||||
f_read: Option<Pin<Box<dyn Future<Output = io::Result<Option<Vec<u8>>>>>>>,
|
||||
f_seek: Option<Pin<Box<dyn Future<Output = io::Result<u64>>>>>,
|
||||
f_flush: Option<Pin<Box<dyn Future<Output = io::Result<()>>>>>,
|
||||
f_shutdown: Option<Pin<Box<dyn Future<Output = io::Result<()>>>>>,
|
||||
write_acks: VecDeque<oneshot::Receiver<SftpResult<Packet>>>,
|
||||
}
|
||||
```
|
||||
|
||||
### AsyncRead Implementation
|
||||
|
||||
- Uses a state-machine pattern: `f_read` stores the in-progress read future
|
||||
- Calculates read length as `min(remaining_buffer, max_read_len)` where `max_read_len` comes from `Limits.read_len` or `max_packet_len - 9` (read overhead)
|
||||
- On `StatusCode::Eof`, returns `Ok(None)` → signals clean EOF to the caller
|
||||
- Advances `pos` by the data length on successful read
|
||||
|
||||
### AsyncWrite Implementation
|
||||
|
||||
- Implements **pipelined writes** with configurable concurrency (`max_concurrent_writes`)
|
||||
- Uses `write_nowait()` to fire off write requests without awaiting each ACK
|
||||
- Stores `oneshot::Receiver`s in `write_acks: VecDeque`
|
||||
- When `write_acks.len() >= max_concurrent_writes`, polls the oldest pending ACK before accepting new writes
|
||||
- Write chunk size: `min(data.len(), max_write_len)` where `max_write_len` comes from `Limits.write_len` or `max_packet_len - 21 - handle_length`
|
||||
|
||||
Overhead constants:
|
||||
```rust
|
||||
const READ_OVERHEAD_LENGTH: u32 = 9; // type(1) + id(4) + data_len(4)
|
||||
const WRITE_OVERHEAD_LENGTH: u32 = 21; // type(1) + id(4) + handle_len(4) + offset(8) + data_len(4)
|
||||
```
|
||||
|
||||
### AsyncSeek Implementation
|
||||
|
||||
- `SeekFrom::Start(pos)` — direct position set
|
||||
- `SeekFrom::Current(delta)` — arithmetic on current `pos`
|
||||
- `SeekFrom::End(delta)` — requires an `fstat()` round-trip to get file size, then computes position
|
||||
|
||||
### Flush and Shutdown
|
||||
|
||||
- `poll_flush()`: Drains all pending write ACKs, then optionally calls `fsync` if the server supports it
|
||||
- `poll_shutdown()`: Drains pending ACKs, then sends `close()` on the handle and marks `closed = true`
|
||||
|
||||
### Drop Behavior
|
||||
|
||||
If `closed` is not yet true, `File::drop()` sends a `close_nowait()` (fire-and-forget) to avoid blocking in a destructor.
|
||||
|
||||
### File Metadata Methods
|
||||
|
||||
```rust
|
||||
impl File {
|
||||
pub async fn metadata(&self) -> SftpResult<Metadata>; // fstat
|
||||
pub async fn set_metadata(&self, metadata: Metadata) -> SftpResult<()>; // fsetstat
|
||||
pub async fn sync_all(&self) -> SftpResult<()>; // fsync (no-op if unsupported)
|
||||
}
|
||||
```
|
||||
|
||||
## ReadDir and DirEntry
|
||||
|
||||
```rust
|
||||
pub struct ReadDir {
|
||||
parent: Arc<str>,
|
||||
entries: VecDeque<(String, Metadata)>,
|
||||
}
|
||||
|
||||
pub struct DirEntry {
|
||||
parent: Arc<str>,
|
||||
file: String,
|
||||
metadata: Metadata,
|
||||
}
|
||||
```
|
||||
|
||||
- `ReadDir` implements `Iterator<Item = DirEntry>`
|
||||
- Automatically filters out `.` and `..` entries
|
||||
- `DirEntry::path()` constructs full paths using POSIX-style `/` separator
|
||||
- `DirEntry::file_name()`, `file_type()`, `metadata()` provide accessors
|
||||
|
||||
## Runtime Abstraction (`client/runtime.rs`)
|
||||
|
||||
The client runtime abstracts over native tokio and WASM environments:
|
||||
|
||||
- **Native** (`not(target_arch = "wasm32")`): Uses `tokio::spawn` and `tokio::time::timeout`
|
||||
- **WASM** (`target_arch = "wasm32"`): Uses `wasm_bindgen_futures::spawn_local` and `gloo_timers::future::TimeoutFuture`
|
||||
|
||||
The `spawn()` function returns a `JoinHandle<T>` that wraps a `oneshot::Receiver`, providing a unified API for both platforms.
|
||||
|
||||
## Timeout Behavior
|
||||
|
||||
Each request in `RawSftpSession` has a configurable timeout (default 10 seconds, set via `Config::request_timeout_secs`):
|
||||
|
||||
```rust
|
||||
async fn request(&self, id: Option<u32>, packet: Packet) -> SftpResult<Packet> {
|
||||
let rx = self.send(id, packet)?;
|
||||
let timeout = self.timeout.load(Ordering::Relaxed);
|
||||
match runtime::timeout(Duration::from_secs(timeout), rx).await {
|
||||
Ok(Ok(result)) => result,
|
||||
Ok(Err(_)) => Err(Error::UnexpectedBehavior("sender dropped".into())),
|
||||
Err(error) => {
|
||||
self.requests.remove(&id);
|
||||
Err(error) // Error::Timeout
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
On timeout, the pending request entry is cleaned up from the `DashMap`.
|
||||
272
docs/research/references/ssh/russh-sftp/05-server-api.md
Normal file
272
docs/research/references/ssh/russh-sftp/05-server-api.md
Normal file
@@ -0,0 +1,272 @@
|
||||
# russh-sftp: Server Implementation
|
||||
|
||||
## Server Architecture
|
||||
|
||||
The server side of russh-sftp follows a **handler trait pattern**: users implement the `server::Handler` trait, and the framework calls their methods in response to incoming SFTP requests. Each request is processed and a response packet is sent back on the same stream.
|
||||
|
||||
The server processes requests **sequentially** — one at a time, in order. There is no concurrent request handling within a single SFTP session.
|
||||
|
||||
## Server Handler Trait
|
||||
|
||||
```rust
|
||||
pub trait Handler: Sized {
|
||||
type Error: Into<StatusReply> + Send;
|
||||
|
||||
fn unimplemented(&self) -> Self::Error;
|
||||
|
||||
// --- Lifecycle ---
|
||||
fn init(&mut self, version: u32, extensions: HashMap<String, String>)
|
||||
-> impl Future<Output = Result<Version, Self::Error>> + Send;
|
||||
|
||||
// --- File operations ---
|
||||
fn open(&mut self, id: u32, filename: String, pflags: OpenFlags, attrs: FileAttributes)
|
||||
-> impl Future<Output = Result<Handle, Self::Error>> + Send;
|
||||
fn close(&mut self, id: u32, handle: String)
|
||||
-> impl Future<Output = Result<Status, Self::Error>> + Send;
|
||||
fn read(&mut self, id: u32, handle: String, offset: u64, len: u32)
|
||||
-> impl Future<Output = Result<Data, Self::Error>> + Send;
|
||||
fn write(&mut self, id: u32, handle: String, offset: u64, data: Vec<u8>)
|
||||
-> impl Future<Output = Result<Status, Self::Error>> + Send;
|
||||
|
||||
// --- Metadata ---
|
||||
fn lstat(&mut self, id: u32, path: String)
|
||||
-> impl Future<Output = Result<Attrs, Self::Error>> + Send;
|
||||
fn fstat(&mut self, id: u32, handle: String)
|
||||
-> impl Future<Output = Result<Attrs, Self::Error>> + Send;
|
||||
fn setstat(&mut self, id: u32, path: String, attrs: FileAttributes)
|
||||
-> impl Future<Output = Result<Status, Self::Error>> + Send;
|
||||
fn fsetstat(&mut self, id: u32, handle: String, attrs: FileAttributes)
|
||||
-> impl Future<Output = Result<Status, Self::Error>> + Send;
|
||||
|
||||
// --- Directory operations ---
|
||||
fn opendir(&mut self, id: u32, path: String)
|
||||
-> impl Future<Output = Result<Handle, Self::Error>> + Send;
|
||||
fn readdir(&mut self, id: u32, handle: String)
|
||||
-> impl Future<Output = Result<Name, Self::Error>> + Send;
|
||||
|
||||
// --- Filesystem operations ---
|
||||
fn remove(&mut self, id: u32, filename: String)
|
||||
-> impl Future<Output = Result<Status, Self::Error>> + Send;
|
||||
fn mkdir(&mut self, id: u32, path: String, attrs: FileAttributes)
|
||||
-> impl Future<Output = Result<Status, Self::Error>> + Send;
|
||||
fn rmdir(&mut self, id: u32, path: String)
|
||||
-> impl Future<Output = Result<Status, Self::Error>> + Send;
|
||||
fn realpath(&mut self, id: u32, path: String)
|
||||
-> impl Future<Output = Result<Name, Self::Error>> + Send;
|
||||
fn stat(&mut self, id: u32, path: String)
|
||||
-> impl Future<Output = Result<Attrs, Self::Error>> + Send;
|
||||
fn rename(&mut self, id: u32, oldpath: String, newpath: String)
|
||||
-> impl Future<Output = Result<Status, Self::Error>> + Send;
|
||||
fn readlink(&mut self, id: u32, path: String)
|
||||
-> impl Future<Output = Result<Name, Self::Error>> + Send;
|
||||
fn symlink(&mut self, id: u32, linkpath: String, targetpath: String)
|
||||
-> impl Future<Output = Result<Status, Self::Error>> + Send;
|
||||
|
||||
// --- Extensions ---
|
||||
fn extended(&mut self, id: u32, request: String, data: Vec<u8>)
|
||||
-> impl Future<Output = Result<Packet, Self::Error>> + Send;
|
||||
}
|
||||
```
|
||||
|
||||
Every method has a default implementation that calls `self.unimplemented()` and returns it as `Err`, making it safe to only implement the methods you need.
|
||||
|
||||
### Handler Error Type
|
||||
|
||||
The associated `Error` type must implement `Into<StatusReply>`. When a handler method returns `Err(e)`, the framework converts the error to a `StatusReply` and sends an `SSH_FXP_STATUS` packet with the appropriate `StatusCode`, error message, and language tag.
|
||||
|
||||
### StatusReply
|
||||
|
||||
```rust
|
||||
pub struct StatusReply {
|
||||
pub status_code: StatusCode,
|
||||
pub error_message: Option<String>,
|
||||
pub language_tag: Option<String>,
|
||||
}
|
||||
```
|
||||
|
||||
Convenience constructors:
|
||||
```rust
|
||||
impl StatusReply {
|
||||
pub fn new(status_code: StatusCode) -> Self;
|
||||
pub fn with_message(self, message: impl Into<String>) -> Self;
|
||||
pub fn with_language_tag(self, tag: impl Into<String>) -> Self;
|
||||
}
|
||||
|
||||
impl StatusCode {
|
||||
pub fn with_message(self, message: impl Into<String>) -> StatusReply;
|
||||
}
|
||||
```
|
||||
|
||||
Example:
|
||||
```rust
|
||||
// Using StatusCode directly — minimal allocation
|
||||
Err(StatusCode::NoSuchFile)
|
||||
|
||||
// With custom message
|
||||
Err(StatusCode::PermissionDenied.with_message("access denied"))
|
||||
|
||||
// Full control
|
||||
Err(StatusReply::new(StatusCode::Failure)
|
||||
.with_message("disk full")
|
||||
.with_language_tag("en-US"))
|
||||
```
|
||||
|
||||
All of these produce `StatusReply`, which then maps to:
|
||||
```
|
||||
SSH_FXP_STATUS { id, status_code, error_message, language_tag }
|
||||
```
|
||||
|
||||
## Request Processing Pipeline
|
||||
|
||||
### The `into_wrap!` Macro
|
||||
|
||||
Each request is processed via a macro that extracts fields from the packet and calls the handler:
|
||||
|
||||
```rust
|
||||
macro_rules! into_wrap {
|
||||
($id:expr, $handler:expr, $var:ident; $($arg:ident),*) => {
|
||||
match $handler.$var($($var.$arg),*).await {
|
||||
Err(err) => {
|
||||
let StatusReply { status_code, error_message, language_tag } = err.into();
|
||||
Packet::Status(Status {
|
||||
id: $id,
|
||||
status_code,
|
||||
error_message: error_message.unwrap_or_else(|| status_code.to_string()),
|
||||
language_tag: language_tag.unwrap_or_else(|| "en-US".to_string()),
|
||||
})
|
||||
},
|
||||
Ok(packet) => packet.into(),
|
||||
}
|
||||
};
|
||||
}
|
||||
```
|
||||
|
||||
This macro:
|
||||
1. Calls the handler method with the packet fields
|
||||
2. On `Ok(packet)` — converts the result to a `Packet` via `.into()`
|
||||
3. On `Err(err)` — converts error to `StatusReply`, then constructs a `Packet::Status` with defaults for missing fields
|
||||
|
||||
### The `process_request` Function
|
||||
|
||||
```rust
|
||||
async fn process_request<H>(packet: Packet, handler: &mut H) -> Packet
|
||||
where H: Handler + Send
|
||||
{
|
||||
let id = packet.get_request_id();
|
||||
match packet {
|
||||
Packet::Init(init) => into_wrap!(id, handler, init; version, extensions),
|
||||
Packet::Open(open) => into_wrap!(id, handler, open; id, filename, pflags, attrs),
|
||||
Packet::Close(close) => into_wrap!(id, handler, close; id, handle),
|
||||
Packet::Read(read) => into_wrap!(id, handler, read; id, handle, offset, len),
|
||||
// ... all other variants
|
||||
Packet::Extended(extended) => into_wrap!(id, handler, extended; id, request, data),
|
||||
_ => Packet::error(0, StatusCode::BadMessage),
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### The `run` Functions
|
||||
|
||||
```rust
|
||||
pub async fn run<S, H>(stream: S, handler: H)
|
||||
where
|
||||
S: AsyncRead + AsyncWrite + Unpin + Send + 'static,
|
||||
H: Handler + Send + 'static;
|
||||
|
||||
pub async fn run_with_config<S, H>(mut stream: S, mut handler: H, cfg: Config)
|
||||
where
|
||||
S: AsyncRead + AsyncWrite + Unpin + Send + 'static,
|
||||
H: Handler + Send + 'static;
|
||||
```
|
||||
|
||||
The server loop:
|
||||
1. Reads a packet from the stream (`read_packet`)
|
||||
2. Deserializes to `Packet`
|
||||
3. On deserialization error, sends `Packet::error(0, StatusCode::BadMessage)`
|
||||
4. Calls `process_request()` to dispatch to the handler
|
||||
5. Serializes the response and writes it back to the stream
|
||||
6. Flushes the stream
|
||||
7. Repeats until `UnexpectedEof` or cancellation
|
||||
|
||||
The server is spawned on `tokio::spawn` (or `wasm_bindgen_futures::spawn_local` on WASM) and runs independently.
|
||||
|
||||
### Response Types
|
||||
|
||||
Each handler method return type maps to a specific response packet:
|
||||
|
||||
| Handler Method | Success Return | Packet Type |
|
||||
|---------------|----------------|-------------|
|
||||
| `init` | `Version` | `SSH_FXP_VERSION` |
|
||||
| `open` | `Handle` | `SSH_FXP_HANDLE` |
|
||||
| `close` | `Status` | `SSH_FXP_STATUS` |
|
||||
| `read` | `Data` | `SSH_FXP_DATA` |
|
||||
| `write` | `Status` | `SSH_FXP_STATUS` |
|
||||
| `lstat` / `stat` / `fstat` | `Attrs` | `SSH_FXP_ATTRS` |
|
||||
| `setstat` / `fsetstat` | `Status` | `SSH_FXP_STATUS` |
|
||||
| `opendir` | `Handle` | `SSH_FXP_HANDLE` |
|
||||
| `readdir` | `Name` | `SSH_FXP_NAME` |
|
||||
| `remove` / `mkdir` / `rmdir` / `rename` / `symlink` | `Status` | `SSH_FXP_STATUS` |
|
||||
| `realpath` / `readlink` | `Name` | `SSH_FXP_NAME` |
|
||||
| `extended` | `Packet` | Any response type |
|
||||
|
||||
## Server Example
|
||||
|
||||
A minimal SFTP server with russh integration:
|
||||
|
||||
```rust
|
||||
#[derive(Default)]
|
||||
struct SftpSession {
|
||||
version: Option<u32>,
|
||||
}
|
||||
|
||||
impl russh_sftp::server::Handler for SftpSession {
|
||||
type Error = StatusCode;
|
||||
|
||||
fn unimplemented(&self) -> Self::Error {
|
||||
StatusCode::OpUnsupported
|
||||
}
|
||||
|
||||
async fn init(&mut self, version: u32, _ext: HashMap<String, String>)
|
||||
-> Result<Version, Self::Error>
|
||||
{
|
||||
self.version = Some(version);
|
||||
Ok(Version::new())
|
||||
}
|
||||
|
||||
async fn close(&mut self, id: u32, _handle: String) -> Result<Status, Self::Error> {
|
||||
Ok(Status { id, status_code: StatusCode::Ok, error_message: "Ok".into(), language_tag: "en-US".into() })
|
||||
}
|
||||
|
||||
async fn opendir(&mut self, id: u32, path: String) -> Result<Handle, Self::Error> {
|
||||
Ok(Handle { id, handle: path })
|
||||
}
|
||||
|
||||
async fn readdir(&mut self, id: u32, handle: String) -> Result<Name, Self::Error> {
|
||||
// Return EOF when done
|
||||
Err(StatusCode::Eof)
|
||||
}
|
||||
|
||||
async fn realpath(&mut self, id: u32, path: String) -> Result<Name, Self::Error> {
|
||||
Ok(Name { id, files: vec![File::dummy("/")] })
|
||||
}
|
||||
}
|
||||
|
||||
// In the russh Handler:
|
||||
async fn subsystem_request(&mut self, channel_id: ChannelId, name: &str, session: &mut Session)
|
||||
-> Result<(), Self::Error>
|
||||
{
|
||||
if name == "sftp" {
|
||||
let channel = self.get_channel(channel_id).await;
|
||||
let sftp = SftpSession::default();
|
||||
session.channel_success(channel_id)?;
|
||||
russh_sftp::server::run(channel.into_stream(), sftp).await;
|
||||
} else {
|
||||
session.channel_failure(channel_id)?;
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
```
|
||||
|
||||
## WASM Considerations
|
||||
|
||||
The `server` module is gated behind `#[cfg(not(target_arch = "wasm32"))]` and is not available on WASM targets. The `client` module is available on both native and WASM via the `runtime.rs` abstraction.
|
||||
@@ -0,0 +1,278 @@
|
||||
# russh-sftp: Data Flow and Integration
|
||||
|
||||
## Client Data Flow: File Read
|
||||
|
||||
```
|
||||
Application code
|
||||
│
|
||||
▼
|
||||
SftpSession::open("file.txt")
|
||||
│
|
||||
▼
|
||||
RawSftpSession::open("file.txt", OpenFlags::READ, FileAttributes::empty())
|
||||
│
|
||||
├── 1. Generate request ID: AtomicU32::fetch_add(1)
|
||||
├── 2. Check Limits.open_handles
|
||||
├── 3. Create oneshot channel, insert (Some(id), tx) into DashMap
|
||||
├── 4. Serialize Open packet → Bytes
|
||||
├── 5. Send Bytes via mpsc::UnboundedSender
|
||||
└── 6. Await oneshot::Receiver with timeout
|
||||
│
|
||||
▼
|
||||
Writer Task (tokio::spawn) Reader Task (tokio::spawn)
|
||||
┌──────────────────────┐ ┌──────────────────────┐
|
||||
│ rx.recv() → Bytes │ │ read_packet(stream) │
|
||||
│ → write_all(stream) │ │ → Packet::try_from() │
|
||||
│ → flush() │ │ → SessionInner::reply │
|
||||
└──────────────────────┘ │ → DashMap.remove │
|
||||
│ → oneshot::tx.send │
|
||||
└──────────┬───────────┘
|
||||
│
|
||||
┌───────────┘
|
||||
▼
|
||||
oneshot::rx.recv()
|
||||
│
|
||||
▼
|
||||
Packet::Handle(Handle { id, handle: "..." })
|
||||
│
|
||||
▼
|
||||
Return handle string
|
||||
│
|
||||
▼
|
||||
File::new(session, handle, features)
|
||||
│
|
||||
▼ (user calls AsyncRead::poll_read)
|
||||
File::poll_read()
|
||||
│
|
||||
├── Compute max_read_len (from Limits or max_packet_len - 9)
|
||||
├── Compute offset (self.pos) and len (min of remaining buffer, max_read_len)
|
||||
├── Create async future: session.read(handle, offset, len)
|
||||
│ └── RawSftpSession::read() → request/response cycle as above
|
||||
├── Poll future; on Ready:
|
||||
│ ├── Err(Status::Eof) → Ok(None) → return Poll::Ready(Ok(()))
|
||||
│ ├── Ok(data) → buf.put_slice(&data), advance pos
|
||||
│ └── Other errors → return Poll::Ready(Err(...))
|
||||
└── On Pending: return Poll::Pending
|
||||
```
|
||||
|
||||
## Client Data Flow: File Write (Pipelined)
|
||||
|
||||
```
|
||||
Application code
|
||||
│
|
||||
▼
|
||||
file.write_all(&data).await (AsyncWrite::poll_write)
|
||||
│
|
||||
├── 1. If write_acks.len() >= max_concurrent_writes:
|
||||
│ Poll oldest pending ACK; if not ready, return Pending
|
||||
│
|
||||
├── 2. Compute write chunk: min(data.len(), max_write_len)
|
||||
│ max_write_len = Limits.write_len or (max_packet_len - 21 - handle_len)
|
||||
│
|
||||
├── 3. Call session.write_nowait(handle, offset, data)
|
||||
│ └── RawSftpSession::write_nowait()
|
||||
│ ├── Generate request ID
|
||||
│ ├── Check Limits.write_len
|
||||
│ ├── Create oneshot channel, insert into DashMap
|
||||
│ ├── Send packet via mpsc channel
|
||||
│ └── Return oneshot::Receiver (does NOT await)
|
||||
│
|
||||
├── 4. Store oneshot::Receiver in write_acks VecDeque
|
||||
├── 5. Advance pos by chunk length
|
||||
└── 6. Return Poll::Ready(Ok(chunk_len))
|
||||
|
||||
Later: file.flush().await (AsyncWrite::poll_flush)
|
||||
│
|
||||
├── 1. Drain all pending write_acks (poll each one)
|
||||
└── 2. If fsync supported: call session.fsync(handle)
|
||||
└── RawSftpSession::fsync() → full request/response cycle
|
||||
|
||||
Later: file.shutdown().await (AsyncWrite::poll_shutdown)
|
||||
│
|
||||
├── 1. Drain all pending write_acks
|
||||
└── 2. Call session.close(handle) → full request/response cycle
|
||||
└── Marks self.closed = true
|
||||
```
|
||||
|
||||
## Server Data Flow
|
||||
|
||||
```
|
||||
Client SSH connection
|
||||
│
|
||||
▼ (subsystem request for "sftp")
|
||||
russh server::Handler::subsystem_request()
|
||||
│
|
||||
├── session.channel_success(channel_id)
|
||||
└── russh_sftp::server::run(channel.into_stream(), sftp_handler).await
|
||||
│
|
||||
▼
|
||||
┌──────────────────────────────────────────────┐
|
||||
│ tokio::spawn loop: │
|
||||
│ │
|
||||
│ loop { │
|
||||
│ 1. read_packet(stream, max_len) │
|
||||
│ 2. Packet::try_from(bytes) │
|
||||
│ └── On error: Packet::error(0, BadMsg)│
|
||||
│ 3. process_request(packet, handler) │
|
||||
│ └── Match packet variant │
|
||||
│ └── Call handler method with fields │
|
||||
│ └── Ok → convert to Packet::from() │
|
||||
│ └── Err → StatusReply → Packet::Status │
|
||||
│ 4. Bytes::try_from(response) │
|
||||
│ 5. stream.write_all(&packet) │
|
||||
│ 6. stream.flush() │
|
||||
│ } │
|
||||
│ // Break on UnexpectedEof │
|
||||
└──────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
## SFTP Handshake Protocol
|
||||
|
||||
### Client-side handshake (in SftpSession::new)
|
||||
|
||||
```
|
||||
Client Server
|
||||
│ │
|
||||
│──── SSH_FXP_INIT {version:3} ─────────►│
|
||||
│ │
|
||||
│◄─── SSH_FXP_VERSION {version:3, │
|
||||
│ extensions: {...}} ────────────────│
|
||||
│ │
|
||||
│ (now SftpSession checks extensions │
|
||||
│ and optionally fetches limits) │
|
||||
│ │
|
||||
│──── SSH_FXP_EXTENDED { │
|
||||
│ request: "limits@openssh.com", │
|
||||
│ data: [] } ───────────────────────►│
|
||||
│ │
|
||||
│◄─── SSH_FXP_EXTENDED_REPLY { │
|
||||
│ data: LimitsExtension } ──────────│
|
||||
│ │
|
||||
│ (SftpSession stores limits in │
|
||||
│ RawSftpSession and Features) │
|
||||
```
|
||||
|
||||
### Server-side handshake (in Handler::init)
|
||||
|
||||
```
|
||||
Server receives SSH_FXP_INIT
|
||||
│
|
||||
├── process_request() calls handler.init(version, extensions)
|
||||
├── Handler returns Ok(Version { version: 3, extensions })
|
||||
└── Version packet is serialized and sent back
|
||||
```
|
||||
|
||||
## Error Handling Flows
|
||||
|
||||
### Client Error Chain
|
||||
|
||||
```
|
||||
RawSftpSession method call
|
||||
│
|
||||
├── Timeout → Error::Timeout
|
||||
│ (DashMap entry cleaned up)
|
||||
│
|
||||
├── Unexpected packet type → Error::UnexpectedPacket
|
||||
│ (e.g., expected Handle, got Data)
|
||||
│
|
||||
├── Server status error → Error::Status(Status)
|
||||
│ (StatusCode != Ok)
|
||||
│
|
||||
├── Limit exceeded → Error::Limited(String)
|
||||
│ (packet/read/write/handle limits from server)
|
||||
│
|
||||
└── Channel closed → Error::UnexpectedBehavior("session closed")
|
||||
```
|
||||
|
||||
### Server Error Chain
|
||||
|
||||
```
|
||||
Handler method returns Err(e)
|
||||
│
|
||||
├── e.into() → StatusReply { status_code, error_message, language_tag }
|
||||
│
|
||||
├── Default fill:
|
||||
│ error_message: None → status_code.to_string()
|
||||
│ language_tag: None → "en-US"
|
||||
│
|
||||
└── Packet::Status { id, status_code, error_message, language_tag }
|
||||
sent back to client
|
||||
```
|
||||
|
||||
## russh Integration Patterns
|
||||
|
||||
### Client Pattern
|
||||
|
||||
```rust
|
||||
use russh::client;
|
||||
use russh_sftp::client::SftpSession;
|
||||
|
||||
struct Client;
|
||||
|
||||
impl client::Handler for Client {
|
||||
type Error = anyhow::Error;
|
||||
|
||||
async fn check_server_key(&mut self, key: &russh::keys::PublicKey)
|
||||
-> Result<bool, Self::Error> { Ok(true) }
|
||||
}
|
||||
|
||||
async fn connect_sftp() -> anyhow::Result<SftpSession> {
|
||||
let config = russh::client::Config::default();
|
||||
let mut session = client::connect(Arc::new(config), ("host", 22), Client).await?;
|
||||
let auth_ok = session.authenticate_password("user", "pass").await?;
|
||||
if !auth_ok.success() { return Err(anyhow!("auth failed")); }
|
||||
|
||||
let channel = session.channel_open_session().await?;
|
||||
channel.request_subsystem(true, "sftp").await?;
|
||||
|
||||
// channel.into_stream() provides AsyncRead + AsyncWrite + Unpin + Send + 'static
|
||||
let sftp = SftpSession::new(channel.into_stream()).await?;
|
||||
Ok(sftp)
|
||||
}
|
||||
```
|
||||
|
||||
### Server Pattern
|
||||
|
||||
```rust
|
||||
use russh::server;
|
||||
use russh_sftp::server;
|
||||
|
||||
struct SshSession { /* ... */ }
|
||||
|
||||
impl server::Handler for SshSession {
|
||||
type Error = anyhow::Error;
|
||||
// ... auth methods ...
|
||||
|
||||
async fn subsystem_request(&mut self, channel_id: ChannelId, name: &str, session: &mut server::Session)
|
||||
-> Result<(), Self::Error>
|
||||
{
|
||||
if name == "sftp" {
|
||||
let channel = self.get_channel(channel_id).await;
|
||||
let sftp_handler = MySftpHandler::new();
|
||||
session.channel_success(channel_id)?;
|
||||
server::run(channel.into_stream(), sftp_handler).await;
|
||||
} else {
|
||||
session.channel_failure(channel_id)?;
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Non-russh Transport
|
||||
|
||||
Since russh-sftp only requires `AsyncRead + AsyncWrite + Unpin + Send + 'static`, any transport providing these traits works:
|
||||
|
||||
```rust
|
||||
// Using a TCP stream directly (e.g., for testing without SSH)
|
||||
let stream = TcpStream::connect("127.0.0.1:2222").await?;
|
||||
let sftp = SftpSession::new(stream).await?;
|
||||
|
||||
// Using a Unix socket
|
||||
let stream = UnixStream::connect("/tmp/sftp-sock").await?;
|
||||
let sftp = SftpSession::new(stream).await?;
|
||||
|
||||
// Using tokio::io::duplex for in-process testing
|
||||
let (client, server) = tokio::io::duplex(8192);
|
||||
// Feed server side to server::run(), client side to SftpSession::new()
|
||||
```
|
||||
216
docs/research/references/ssh/russh-sftp/07-quick-reference.md
Normal file
216
docs/research/references/ssh/russh-sftp/07-quick-reference.md
Normal file
@@ -0,0 +1,216 @@
|
||||
# russh-sftp: Quick Reference
|
||||
|
||||
## Crate Overview
|
||||
|
||||
| Property | Value |
|
||||
|----------|-------|
|
||||
| Version | 2.3.0 |
|
||||
| License | Apache-2.0 |
|
||||
| Protocol | SFTP v3 (draft-ietf-secsh-filexfer-02) |
|
||||
| Min Rust | 2021 edition |
|
||||
| WASM | Client works on wasm32; server not supported |
|
||||
|
||||
## Feature Flags
|
||||
|
||||
| Feature | Default | Description |
|
||||
|---------|---------|-------------|
|
||||
| `async-trait` | ❌ | Enables `#[async_trait]` on Handler traits |
|
||||
|
||||
## Key Dependencies
|
||||
|
||||
`tokio` (io-util, rt, sync, time, macros), `tokio-util`, `serde`, `serde_bytes`, `bitflags`, `bytes`, `dashmap`, `chrono`, `thiserror`, `log`
|
||||
|
||||
## Public Modules
|
||||
|
||||
| Module | Description |
|
||||
|--------|-------------|
|
||||
| `client` | Client-side: `RawSftpSession`, `SftpSession`, `Handler`, `Config`, `fs::File`, `fs::ReadDir`, `fs::DirEntry` |
|
||||
| `server` | Server-side: `Handler`, `StatusReply`, `Config`, `run()`, `run_with_config()` |
|
||||
| `protocol` | All SFTP packet types, `Packet` enum, `StatusCode`, `OpenFlags`, `FileAttributes`, `File`, etc. |
|
||||
| `extensions` | OpenSSH extensions: `LimitsExtension`, `HardlinkExtension`, `FsyncExtension`, `StatvfsExtension`, `Statvfs` |
|
||||
| `de` | `from_bytes()` — public deserialization function for extension data |
|
||||
| `ser` | `to_bytes()` — public serialization function |
|
||||
|
||||
## Client Quick Start
|
||||
|
||||
```rust
|
||||
use russh_sftp::client::SftpSession;
|
||||
use russh_sftp::protocol::OpenFlags;
|
||||
|
||||
// Connect (using any AsyncRead+AsyncWrite stream)
|
||||
let sftp = SftpSession::new(stream).await?;
|
||||
|
||||
// Or with config:
|
||||
let sftp = SftpSession::new_with_config(stream, Config {
|
||||
max_packet_len: 262144,
|
||||
max_concurrent_writes: 8,
|
||||
request_timeout_secs: 30,
|
||||
}).await?;
|
||||
|
||||
// File operations
|
||||
let mut file = sftp.open("remote.txt").await?; // read-only
|
||||
let mut file = sftp.create("new.txt").await?; // create+truncate+write
|
||||
let mut file = sftp.open_with_flags("f", OpenFlags::READ | OpenFlags::WRITE).await?;
|
||||
|
||||
file.write_all(b"hello").await?;
|
||||
file.flush().await?; // drains write pipeline + optional fsync
|
||||
file.rewind().await?;
|
||||
let mut buf = Vec::new();
|
||||
file.read_to_end(&mut buf).await?;
|
||||
file.shutdown().await?; // properly closes handle
|
||||
|
||||
// Directory operations
|
||||
for entry in sftp.read_dir(".").await? {
|
||||
println!("{}: {:?}", entry.file_name(), entry.file_type());
|
||||
}
|
||||
|
||||
// Other operations
|
||||
sftp.canonicalize(".").await?;
|
||||
sftp.metadata("file").await?;
|
||||
sftp.symlink_metadata("link").await?;
|
||||
sftp.create_dir("dir").await?;
|
||||
sftp.remove_dir("dir").await?;
|
||||
sftp.remove_file("file").await?;
|
||||
sftp.rename("old", "new").await?;
|
||||
sftp.symlink("target", "link").await?;
|
||||
sftp.read_link("link").await?;
|
||||
sftp.hardlink("src", "dst").await?; // returns false if unsupported
|
||||
sftp.fs_info("/").await?; // returns Ok(None) if unsupported
|
||||
sftp.close().await?;
|
||||
```
|
||||
|
||||
## Server Quick Start
|
||||
|
||||
```rust
|
||||
use russh_sftp::protocol::{File, Handle, Name, Status, StatusCode, Version};
|
||||
use russh_sftp::server::{Handler, StatusReply};
|
||||
|
||||
struct MyHandler;
|
||||
|
||||
impl Handler for MyHandler {
|
||||
type Error = StatusCode;
|
||||
|
||||
fn unimplemented(&self) -> Self::Error {
|
||||
StatusCode::OpUnsupported
|
||||
}
|
||||
|
||||
async fn init(&mut self, version: u32, _ext: HashMap<String, String>)
|
||||
-> Result<Version, Self::Error>
|
||||
{
|
||||
Ok(Version::new())
|
||||
}
|
||||
|
||||
// ... implement methods as needed
|
||||
}
|
||||
|
||||
// In your SSH server handler:
|
||||
async fn subsystem_request(&mut self, channel_id: ChannelId, name: &str, session: &mut Session)
|
||||
-> Result<(), Error>
|
||||
{
|
||||
if name == "sftp" {
|
||||
let channel = self.get_channel(channel_id).await;
|
||||
session.channel_success(channel_id)?;
|
||||
russh_sftp::server::run(channel.into_stream(), MyHandler).await;
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
```
|
||||
|
||||
## RawSftpSession Quick Reference
|
||||
|
||||
```rust
|
||||
use russh_sftp::client::RawSftpSession;
|
||||
|
||||
let session = RawSftpSession::new(stream);
|
||||
// or: RawSftpSession::new_with_config(stream, config);
|
||||
|
||||
// Must call init first
|
||||
let version = session.init().await?;
|
||||
|
||||
// Request-response methods
|
||||
let handle = session.open("file", OpenFlags::READ, FileAttributes::empty()).await?;
|
||||
let data = session.read(handle.handle, 0, 32768).await?;
|
||||
session.write(handle.handle, 0, vec![1,2,3]).await?;
|
||||
session.close(handle.handle).await?;
|
||||
let attrs = session.stat("/path").await?;
|
||||
let name = session.realpath(".").await?;
|
||||
let dir_handle = session.opendir("/").await?;
|
||||
let entries = session.readdir(dir_handle.handle).await?;
|
||||
session.close(dir_handle.handle).await?;
|
||||
|
||||
// Extensions
|
||||
let limits = session.limits().await?;
|
||||
session.hardlink("/old", "/new").await?;
|
||||
session.fsync(handle).await?;
|
||||
let fs_info = session.statvfs("/").await?;
|
||||
```
|
||||
|
||||
## StatusCode Reference
|
||||
|
||||
| Code | Constant | Meaning |
|
||||
|------|----------|---------|
|
||||
| 0 | `Ok` | Successful completion |
|
||||
| 1 | `Eof` | End of file / no more directory entries |
|
||||
| 2 | `NoSuchFile` | File does not exist |
|
||||
| 3 | `PermissionDenied` | Insufficient permissions |
|
||||
| 4 | `Failure` | Generic failure |
|
||||
| 5 | `BadMessage` | Badly formatted packet |
|
||||
| 6 | `NoConnection` | Client-side: no connection (never from server) |
|
||||
| 7 | `ConnectionLost` | Client-side: connection lost (never from server) |
|
||||
| 8 | `OpUnsupported` | Operation not supported |
|
||||
|
||||
## OpenFlags Reference
|
||||
|
||||
| Flag | Value | Description |
|
||||
|------|-------|-------------|
|
||||
| `READ` | 0x01 | Open for reading |
|
||||
| `WRITE` | 0x02 | Open for writing |
|
||||
| `APPEND` | 0x04 | Append to existing data |
|
||||
| `CREATE` | 0x08 | Create if doesn't exist |
|
||||
| `TRUNCATE` | 0x10 | Truncate to zero length |
|
||||
| `EXCLUDE` | 0x20 | Fail if file exists (must be with CREATE) |
|
||||
|
||||
## FileAttr (Attribute Flags) Reference
|
||||
|
||||
| Flag | Value | Fields Present |
|
||||
|------|-------|---------------|
|
||||
| `SIZE` | 0x00000001 | `size: u64` |
|
||||
| `UIDGID` | 0x00000002 | `uid: u32`, `gid: u32` |
|
||||
| `PERMISSIONS` | 0x00000004 | `permissions: u32` |
|
||||
| `ACMODTIME` | 0x00000008 | `atime: u32`, `mtime: u32` |
|
||||
| `EXTENDED` | 0x80000000 | (not yet implemented) |
|
||||
|
||||
## FileMode (File Type) Reference
|
||||
|
||||
| Constant | Value | Type |
|
||||
|----------|-------|------|
|
||||
| `FIFO` | 0x1000 | Named pipe |
|
||||
| `CHR` | 0x2000 | Character device |
|
||||
| `DIR` | 0x4000 | Directory |
|
||||
| `NAM` | 0x5000 | Named file |
|
||||
| `BLK` | 0x6000 | Block device |
|
||||
| `REG` | 0x8000 | Regular file |
|
||||
| `LNK` | 0xA000 | Symbolic link |
|
||||
| `SOCK` | 0xC000 | Socket |
|
||||
|
||||
## Packet Wire Format
|
||||
|
||||
All SFTP packets:
|
||||
|
||||
```
|
||||
[u32 length] [u8 type] [payload...]
|
||||
```
|
||||
|
||||
- `length` = size of type byte + payload (does not include the length field itself)
|
||||
- Strings: `[u32 len] [utf8 bytes]`
|
||||
- Byte arrays: `[u32 len] [raw bytes]`
|
||||
- `FileAttributes`: `[u32 flags] [conditional fields based on flags]`
|
||||
|
||||
## Extension Names
|
||||
|
||||
| Extension | Name | Version |
|
||||
|-----------|------|---------|
|
||||
| Limits | `limits@openssh.com` | 1 |
|
||||
| Hardlink | `hardlink@openssh.com` | 1 |
|
||||
| Fsync | `fsync@openssh.com` | 1 |
|
||||
| Statvfs | `statvfs@openssh.com` | 2 |
|
||||
Reference in New Issue
Block a user