alkfs/docs/research/phase-0.md

---

status: draft last_updated: 2026-09-23 (second pass, same day — storage identity reframed to git-compatible sha1/sha256: the ancestor POCs' BLAKE3 was an iroh-blobs artifact, not a choice; chunk-level dedup adopted as the working goal; alkblobs/alkfs two-crate split added as the working direction; the remote-mount SFTP use case recorded as the anchor workload. Still unconverged — every OQ is open, the POC register holds proposals not results)

alkfs — Phase 0 (Exploration)

This document captures Phase 0 (Exploration) for the alkfs crate: vision, guiding principles, prior art, open questions (OQ-FS-01..NN), and the candidate POC register. Phase 0's objective per docs/sdd_process.md: capture vision and guiding principles; research options; validate approaches; converge on a recommended approach. It is the input to Phase 1 (Architecture), where the Architect will produce docs/architecture/ specs, ADRs, and the open-questions tracker.

Drafted 2026-09-23 from the initial setup discussion, revised the same day as the storage-identity reframe landed (§Working direction). The crate is the storage-engine member of the alk* family: alkcall (call + channels — the substrate), alktty/alktunnels/alksocks (the protocol-crate siblings), alkgit (the first in-family storage consumer, currently paused on exactly the problem this crate exists to solve).

Honest framing: this Phase 0 is earlier in its lifecycle than alksocks' was at the equivalent commit. There are more unknowns here than in any sibling so far — a filesystem is both a storage engine (hashing, chunking, durability, GC) and a serving protocol (verbs, sessions, wire framing), and almost none of those decisions have empirical ground yet. The POC register below is proposals, not results. Expect this document to be rewritten several times before convergence.

Vision and guiding principles

One sentence: a content-addressed, branch-aware virtual filesystem for the alk family — a local storage engine (path-tree metadata + content-addressed blobs) with a producer/consumer serving protocol on alkcall channels, so that alkgit's object storage, the coming alksftp door, and the alknet filesystem vision all compose on the same substrate — and it never touches the host filesystem unless explicitly configured to.

Working direction (2026-09-23, from the setup discussion — leanings to converge, not decisions):

• Git-compatible identity, not BLAKE3. The ancestor POCs used BLAKE3 because iroh-blobs does; flipping the causality, the blob layer should carry the appfile concept with the hashes git uses (sha1/sha256). This dissolves the "BLAKE3 must hash the whole file before the address exists" problem into a chosen design (OQ-FS-04), and makes git object identity native rather than mapped.
• Chunk-level dedup is the working goal for large content (workspace-scale files, datasets, logs). Whole-file hashing dedups only identical files; git itself dedups at object granularity (packs delta-compress within a pack, but the address space is per-object); chunking is the storage-layer generalization (OQ-FS-04 — measure whether the workloads pay before committing).
• The two-crate split: alkblobs + alkfs. A bucket-style content-addressed blob store (the S3 shape) beneath the path-tree mapping (the alknet-filesystem POC's layer 2). alkgit can consume the blob store without a filesystem; the FS layer keeps its own crate (OQ-FS-01).
• The anchor workload is remote mounting. The concrete use case: the dev-server workspace mounted on a local Linux machine under remote/ — today over vanilla SFTP (sshfs), which is slow in known ways (a round trip per syscall, stat storms, full inode work per stat). An appfile-indexed VFS fixes the server side; caching and pipelining close the rest (OQ-FS-16).

Why this crate exists. The alknet mono-repo's filesystem research (three POC iterations, 24 passing tests) proved a viable three-layer architecture but lived inside a project with excessive scope. The alk* family has since been decomposed and published (alkcall, alktty, alktunnels, alkvault; alksocks and alkgit in flight). alkgit paused because its storage problem — content-addressed blob storage with branch/CAS semantics — is not a git-specific problem, and solving it inside alkgit would leave alksftp and the alknet filesystem rewrite to solve it again. The same basic "shape" recurs across the family: git + git-lfs is small-object CAS metadata over big-blob content storage; iroh-blobs' FsStore is inline-small/outboard-large; the alknet-filesystem POC is SQLite path edges over a blob store. alkfs is that shape, made a crate.

Scope sketch (unconverged; restructured 2026-09-23 around the two-crate working direction — Phase 0 must confirm or cut, OQ-FS-01):

1. alkblobs (candidate companion crate) — bucket-style content-addressed blob storage (the S3/rudolfs shape): put/get/has/ delete by hash, tags/pins for liveness, GC, the appfile layout (one transactional store for blob metadata + inline small content; file-backed spill-over for large content), git-compatible sha1/ sha256 identity, chunk-level dedup for large content. Fully local, embeddable in-process, no channels required; engine-only in v1 (no serving ALPN of its own yet — OQ-FS-09).
2. alkfs (this crate) — the path-tree mapping on top (the POC's layer 2): paths, branches, snapshots, tombstones, cached sizes, write sessions (branch-on-write/merge-on-close); plus the producer/consumer serving protocol on alkcall channels — a producer registers openable filesystem channels (ChannelCore::register_openable); a consumer opens them and speaks the file protocol inside. The alk/fs-shaped ALPN resource, the "ALPN as a service" shape of alktty/alktunnels/alksocks. Depends on alkblobs, or a narrow blob seam if the split puts a trait there (OQ-FS-01).
3. Doors stay out — alksftp (SFTP door), FUSE/mount adapters, sync tools are separate crates/features that consume alkfs, per the family rule (alkgit vision: doors are family infrastructure, not crate contents).

Guiding principles, inherited from the alk* family plus the filesystem research:

1. No forced host-FS access — the defining requirement. The analogue of alksocks' "never binds a port": the engine manages its own content store and never reads, writes, or mounts the host filesystem unless explicitly configured to. Every bridge to the real world (a blob-file store, an SFTP door, a mount, a sync tool) is an explicit, optional, feature-gated assembly shape. The engine itself is storage-agnostic about what backs its blob layer (filesystem files, SQLite, remote).
2. Metadata/content split is load-bearing. Small structured state (path edges, branch pointers, tombstones, sizes) lives in a transactional store; content bytes live under their hash in the blob layer. Path operations are O(path edges), never O(bytes) — rename is O(1) regardless of file size; identical content dedups across branches and paths by construction (and, with chunking, dedups at chunk granularity). This is the alknet-filesystem POC's core conclusion and the same shape as git + git-lfs and iroh-blobs' inline/outboard split — with git-compatible identity instead of BLAKE3.
3. Content addressing with a durability contract. Content is identified by its hash, not its path — with the hashes git uses (sha1/sha256), so git object identity is native to the store, not mapped. The invariant that must survive every design choice: content is durable before any metadata row naming it commits; a crash mid-write leaves the old version intact and visible; no torn version is ever readable; no metadata commit orphans its content. With the two-crate direction the contract crosses the put/get seam (alkfs must be able to trust that an alkblobs put returning means durable, or get an explicit durability level back) — the seam contract is an OQ. Exact hashing/chunking is an OQ; the invariant is not.
4. Branch-aware from day one. Fossil-style branches (a branch is a parent chain + per-branch overrides + tombstones), branch-on-write/ merge-on-close for the write path. This is what gives free multi-agent/multi-fork content sharing and the POSIX concurrent-reader-sees-old-version property — and it is the property git's object model needs (content shared across refs by hash).
5. "ALPN as a service," not "a server." The filesystem is a produced, ACL-scoped resource on a channels connection. The protocol never opens host files or binds anything; door-shaped conveniences (local loopback front door, in-process consumer handle) are optional assembly-layer shapes on either side. Producer/consumer vocabulary, never server/client.
6. Substrate-agnostic protocol, native engine. The file-protocol state machine is generic over T: AsyncRead + AsyncWrite + Unpin (fast-socks5/alksocks genericity precedent); the storage engine is native-only behind features (real FS I/O, SQLite). Whether the protocol layer itself stays wasm-clean is an OQ, not an assumption.
7. Multi-tenancy is a where-clause, not an afterthought. The POC's bucket column is the isolation unit; identity → bucket mapping is an assembly-layer auth question; per-path/per-bucket policy is a Phase 1 design question. A filesystem is arbitrary read/write by nature — the open gate and path-scope policy are the security boundary.
8. Bounded resources on every serving path. alkgit's ADR-009 posture (wall-clock, size, round limits) is the consumer shape this crate must serve; the serving protocol inherits alkcall's backpressure and channel-cap invariants rather than re-deriving them.

What is already settled

The foundation is POC-validated and (upstream) ADR-pinned; this crate does not start from zero. It inherits:

• The channels substrate — demux→Connection→handler→mux, the establishment story (register_openable_with_establisher, alkcall ADR-049; refused sessions are typed channel:open_failed call errors, never phantom channels), the identity seam (CF-005/CF-006: per-call opener identity on the open-op hooks), ACL-for-free via register_openable, the two-pump data plane (channels::pump_bidi, ADR-050 — use it, do not hand-roll), hub relay (ChannelRelay/HubLegTemplate, ADR-051). All production-validated by alktty/alktunnels/alksocks.
• The metadata/content split, POC-proven. alknet-filesystem POC iteration 1: SQLite path tree over a blob store, 8 tests — branch inheritance, tombstones, O(1) rename, bucket isolation, honker notify-atomic-with-mutation.
• The write path, POC-proven. Iteration 2: branch-on-write/ merge-on-close reconciles "BLAKE3 must hash the complete file" with "writes arrive in chunks, possibly out of order" — concurrent readers see the old version until close() commits atomically; crash/abort leaves the old version intact. 7 tests.
• The distributed-sync shape, POC-proven but scope-uncommitted. Iteration 3: path tree as an automerge CRDT synced over QUIC; 9 tests; metadata and content sync separately; concurrent-root-map initialization is a real constraint (root structures must be created by one node and synced before others write). Whether any of this is alkfs v1 scope is OQ-FS-14.
• The iroh-blobs store-seam findings (the external-store probe). The alknet-blobs probe traced iroh-blobs 0.103's full Command actor surface: there is no Store trait — Store is a concrete struct over an irpc::Client; MemStore/FsStore are actors; a third store is a third actor speaking the pub Command enum. Blockers are a ~4-line visibility PR (Store::from_sender, ref_from_sender, ApiClient, the Scope field); GC (run_gc) is pub and reusable; ~120-160 lines of bao-tree glue must be re-implemented per store. The probe's cautionary half: TempTags/TempTagScope are sealed pub(crate) — sealed surface is exactly what turns "external store" into "fork" (AGENTS.md convention 13's cautionary example).
• The consumer shapes this crate must serve. alkgit's backend trait family (docs/architecture/backend.md there: GitRefs, GitPackGen, GitPackIngest — streaming pack generation/ingestion, CAS ref transactions, budgeted resources) and the russh-sftp Handler trait (the POC's near-1:1 SFTP↔PathTree mapping table). "Optimize for git and sftp-like" is the working posture.
• The SQLite-as-application-file-format insight. BLOBs < ~100KB are faster inline in SQLite than as filesystem files; atomic transactions over path-tree metadata; the schema is the documentation. iroh-blobs' FsStore discovered the same shape independently (redb inline tables + filesystem files). The inline-small/outboard-large hybrid is the metadata/content split at storage granularity.
• The family conventions — AGENTS.md in this repo (error handling, tokio subset, feature-gating, upstream posture, one-way doors) and the alkcall ADR set listed there.

Prior art

alknet-filesystem POCs — the direct ancestor

/workspace/@alkdev/alknet/docs/research/alknet-filesystem/ (poc-summary.md, alknet-blobs-external-store-probe.md), POC crates at /workspace/alknet-filesystem-poc and /workspace/alknet-fs-sync-poc. The three-layer conclusion (SQLite path tree + iroh-blobs content + honker coordination) and the write-path resolution (branch-on-write/merge-on-close) are this crate's inheritance, not prior art to re-litigate — but every dependency choice in those POCs is open again:

• SQLite path tree (rusqlite 0.39, bundled): buckets/branches/ paths/tombstones + write_sessions/write_chunks; recursive-CTE chain walk resolves (bucket, branch, path) → content link in sub-millisecond time; only deltas are stored per branch.
• iroh-blobs MemStore as the content layer (deliberately not FsStore — no redb, no fsync rabbit hole in the POC).
• honker-core for notify-in-transaction (the transactional-outbox pattern built in): SELECT notify(...) inside the same transaction as a path-tree mutation; watchers wake on commit, not poll.
• What did NOT transfer and why — the POC's remaining unknowns (FsStore/redb coexistence, SFTP wiring, GC/tags, chain-depth perf, snapshot semantics) are this Phase 0's OQ list; the probe resolved the two-database question by reframing it (external actor, own SQLite file), and the redb question dissolves entirely if alkfs owns its blob metadata in its own store.

iroh-blobs — the nearest CAS store (evaluated; NOT adopted, and the BLAKE3 causality)

/workspace/iroh-blobs (v0.100 checkout) + published 0.103. DESIGN.md is the best available treatment of the blob-store trade space (files are hard: the bitfield/data/outboard write-ordering problem). What it gives: content addressing, bao-tree verified streaming (per-chunk integrity with an outboard), dedup by construction, partial-range reads, tags + mark-sweep GC, a hybrid store (small inline in redb, large as filesystem files) — the same metadata/content split as principle 2.

The causality correction (2026-09-23): the ancestor POCs' BLAKE3 choice was an iroh-blobs artifact — BLAKE3 is load-bearing across the whole iroh-blobs surface (Hash type, bao outboards, the Command actor's payload types), so consuming iroh-blobs means inheriting BLAKE3 and its whole-file addressing. Flipping the causality — we choose the identity — dissolves the "must have the full file for blake3" problem into a design question (OQ-FS-04) and dissolves the "consume iroh-blobs as a store" option (OQ-FS-03's option (a)) entirely: a sha1/sha256 identity layer over iroh-blobs' Command surface would be a mapping shim on top of a foreign hash, not a store. What remains worth taking from iroh-blobs is findings, not code: the inline/outboard hybrid layout, the write-ordering analysis, the mark-sweep GC shape, and the probe's actor-seam map (what a store API needs to expose to stay consumable).

The other frictions (record, for the research they hold):

• The transfer protocol is welded to iroh/QUIC. iroh-blobs' network story is an iroh ProtocolHandler speaking its own ALPN over QUIC. For alkfs this is irrelevant-by-design — content transfer rides alkcall channels — but it means iroh-blobs is at most a local store dependency, never the transfer layer. (The store side itself is transport-independent — that is what the Command-actor probe proved.)
• Whole-file naming vs incremental writes. BLAKE3/bao names a blob by the root hash of its complete byte sequence; you cannot compute the address until all bytes exist. Partial data can live under temp tags (BlobStatus::Partial) but cannot be pinned by its final hash. The POC's branch-on-write/merge-on-close is one reconciliation; chunk lists / Merkle-DAG naming is the other (OQ-FS-04). git escapes the problem structurally: objects are small and independently hashed; packs are the exception, and pack generation streams to a consumer that hashes on arrival.
• Two-database coexistence (redb for blob metadata + SQLite for the path tree): two WALs, two fsync paths, two crash-recovery stories — the POC summary's unknown #1, reframed by the probe ("neither fork nor coexist: a third store actor with its own single SQLite file"), and dissolvable entirely if alkfs owns blob metadata in its own store (OQ-FS-02/03).
• Sealed internals (TempTags, the BaoTreeSender) — the probe's fork-vs-PR calculus. Third-party upstream, not ours, never will be (AGENTS.md convention 17).

git + git-lfs — the shape to optimize for

The user-level framing this crate must honor: "the same basic shape exists with git + gitlfs as with how iroh's blobs handled the mix of small and large blobs." git itself is the small-object CAS: every object individually hashed (sha1 today, sha256 in the object-format transition), refs are names over hashes, content shared across refs/branches for free, rename is metadata-only. git-lfs is the large-blob sidecar: content- addressed pointer rows in the tree, bytes in a separate content store (rudolfs is the family's reference server for that half — namespace/ bucket isolation, LRU cache decorator, fanout streaming). The alkfs analogue: the metadata layer is git's object/ref shape; the blob layer is the LFS store; small content can inline into the metadata store exactly as iroh-blobs inlines small blobs into redb. With sha1/sha256 identity (the 2026-09-23 reframe), git objects are natively addressable in the blob layer — no oids→hash mapping layer, no dedup seam between git's CAS and the store's CAS.

One honest calibration: git dedups at object granularity, not chunk granularity. Packs delta-compress similar objects within a pack, but the address space is per-object. Chunk-level dedup (the working goal, §Working direction) is the storage-layer generalization that pays on workspace-scale files (datasets, logs, agent workspaces with large append-only artifacts) — not on git objects, which are effectively single-chunk. If alkblobs chunks large content, git's pack/idx regime still rides above it as object-granular rows; the two granularities compose (OQ-FS-04, OQ-FS-12). Whether alkgit keeps gix-odb native or serves its object storage from alkfs/alkblobs is OQ-FS-12 — the highest-stakes scope question in this document.

alkgit — the first consumer

/workspace/@alkdev/alkgit — paused on the storage problem. Its backend doc (docs/architecture/backend.md) pins the seam: five traits (GitRegistry, GitRegistryStore, GitRefs, GitPackGen, GitPackIngest), kept small/orthogonal/substrate-blind after the alknet-blobs sealed-surface lesson. The properties alkfs must serve efficiently: streaming pack generation (O(counts) memory, blocking-thread friendly), streaming pack ingestion (budgeted, fsck/connectivity report), CAS ref transactions, registry records with grants. Its research (docs/research/gitoxide.md, poc2-findings.md) carries the gix-odb API contract notes — including the durability shape (pack bytes durable before the ref transaction that names them commits) that mirrors principle 3 exactly.

russh-sftp / alksftp — the door shape

/workspace/russh-sftp (read-only reference): the Handler trait is a near-1:1 mirror of POSIX syscalls as SFTP packets, and the POC mapped every op onto the PathTree + WriteSession API (open→resolve or WriteSession::open; write→write_chunk with offset as key; close→hash+ merge+notify; readdir/list_dir; stat→indexed row — cheaper than a real FS fstat; rename→O(1) edge move). The client's File already implements AsyncRead + AsyncSeek + AsyncWrite with pipelined writes. alksftp (the coming family crate) would speak this verb set; whether alkfs's serving protocol adopts SFTP-shaped verbs natively or defines its own framing that alksftp translates is OQ-FS-08. The alknet-tty async-IO adapter patterns (AsyncWrite-over-mpsc, kill-on-drop) are reusable for the write bridge.

SQLite (application file format) + honker

https://sqlite.org/appfileformat.html — the insight that anchored the POC. honker (/workspace/honker, honker-core 0.2.4) supplies notify/locks/queues as SQL functions on your connection — the transactional-outbox property. Caveats: honker is third-party (not ours, convention 17), single-machine by explicit design ("two servers writing the same .db over NFS is not a Honker deployment strategy"), and its scope in alkfs is an OQ (notify-on-commit may be the only load-bearing piece — locks/queues/scheduler may be excess).

rudolfs — the git-lfs server reference

/workspace/@alkdev/alknet/docs/research/references/gitlfs/ rudolfs-reference.md: StorageKey = (Namespace, Oid) tenant isolation; the decorator composition Verify ↔ Encrypted ↔ Cached ↔ Retrying(Disk → S3); LRU cache in front of permanent storage; fanout() streaming to client and cache simultaneously. The cache/permanent split and the fanout pattern are the production-shape references for alkfs's blob layer and its serving read path.

Anti-prior-art (what NOT to carry over)

• The alknet mono-repo's scope. The filesystem research is inherited as findings, not as a code path; the crate stays small per the decomposition discipline (alksocks' "socks + channels, nothing else" resolution is the template — alkfs's version is OQ-FS-01).
• iroh-blobs' transfer protocol as the content path. Content transfer rides alkcall channels, full stop; an iroh/QUIC side-protocol would fork the family's transport story.
• iroh-blobs' FsStore partial-file lifecycle as a requirement. Its 749-line bao_file.rs partial-blob-on-filesystem machinery exists because redb/filesystem is its substrate; a SQLite-inline store (POC write_chunks) or an alkfs-owned blob store has no need for it.
• gix-odb's layout as alkfs's layout. Loose-objects + packfile directory layout is git's on-disk contract; alkfs is a general engine. If alkgit keeps gix-odb native, alkfs serves the general shape and git storage stays in alkgit (OQ-FS-12).
• NFS-style shared-file multi-node assumptions. honker's own warning applies to the whole engine: single-machine local state first; distribution is an explicit layer (OQ-FS-14), not an ambient property.

Open Questions

These are the design questions Phase 0 must resolve (or explicitly defer) before the architecture spec. Numbered OQ-FS-01.. so they can be referenced, tracked, and promoted into docs/architecture/ open-questions.md in Phase 1. Status is open unless stated; the point of this document is to hold the reasoning without forcing premature decisions. Grouped by theme; numbering is stable across reorganizations (do not renumber — findings files and commits reference these ids).

Core architecture

OQ-FS-01: Crate scope — one crate or the alkblobs/alkfs split?

Working direction (2026-09-23): the two-crate split — alkblobs (the bucket-style CAS blob store) + alkfs (the path-tree mapping + serving protocol). Rationale: a git-like blob store is a complete, useful thing without a filesystem (alkgit could consume it directly; S3-bucket- shaped usage exists on its own), the FS layer is a separate concern with its own consumers, and the POC's layer separation (blob bridge as an explicit module) already drew the seam. Open sub-questions:

• One repo or two? The family has both shapes (alkcall single-crate; alknet's crates/ workspace). Two crates in one repo (a workspace) keeps the seam cheap to iterate until first publish; two repos match the family's one-crate-per-repo posture but tax early co-development. Working lean: one repo, two crates, until the seam stabilizes.
• The seam shape: does alkfs consume a concrete alkblobs (the POC's blob_bridge pattern), or does a small blob trait live in alkfs (or a tiny alkblobs-types leaf) so an embedder can substitute its own store? The sealed-surface lesson (alknet-blobs-external-store-probe.md) says decide before consumers exist; the alkgit trait-family precedent (docs/architecture/backend.md there) is the template for keeping it small. The put/get durability contract (principle 3) rides this seam regardless.
• What is explicitly out: doors (alksftp, FUSE, mount), sync tooling, UI/registry applications, the alknet rewrite's client stack.

OQ-FS-02: Metadata store substrate — and is blob metadata in the same file?

The POC used rusqlite (bundled) for the path tree and it worked well (schema-as-documentation, recursive CTEs, BLOB columns at chunk granularity). Open sub-questions:

• SQLite/rusqlite vs redb vs something else for the path tree. The POC evidence favors SQLite (honker integration, transactional outbox, sub-ms resolves); redb buys a lighter footprint but loses SQL and the honker seam.
• The one-file-vs-two question: if alkfs owns the blob layer too, blob metadata (hash index, sizes, refcounts/liveness) can live in the same SQLite file as the path tree — one WAL, one commit boundary, and the durability ordering of principle 3 becomes a single transaction instead of a cross-store fsync dance. The probe's external-actor shape (iroh-blobs with its own store) vs an alkfs-owned store decides whether this is even on the table. Small content can then inline as BLOBs (the appfileformat sweet spot) — possibly making the "blob store" a layer inside one database with file-backed spill-over for large content (the iroh-blobs FsStore hybrid shape).
• Concurrency model: single writer connection + reader pool? WAL settings (synchronous=NORMAL vs FULL) as a durability-vs-throughput knob? What is the honest fsync contract per operation class?

OQ-FS-03: Blob substrate — own the store (the direction) — what does alkblobs take from where?

Reframed 2026-09-23: option (a) "consume iroh-blobs as a store" is dissolved — BLAKE3 is load-bearing across its entire surface (Hash type, bao outboards, Command payload types), and the identity decision has flipped to git-compatible sha1/sha256 (§Working direction). A sha1/sha256 layer over iroh-blobs' Command surface would be a mapping shim on a foreign hash, paying its dependency weight and its sealed- surface risks to inherit a hash we don't want. The working direction is option (b): own the store (alkblobs). What that leaves open:

• What to take as findings vs build fresh: iroh-blobs' inline/ outboard hybrid layout, write-ordering analysis, and mark-sweep GC shape transfer as findings (§Prior art); the bao_tree crate (BLAKE3-keyed) does not transfer for verified streaming — if per-chunk integrity is wanted under sha1/sha256 identity, the chunk index itself becomes the verification structure (each chunk row carries its own hash; reads re-verify per chunk), which is a simpler design than outboard files and falls out of the chunk manifest (OQ-FS-04) for free.
• The appfile layout (OQ-FS-02's one-file question, now scoped to alkblobs): one transactional store for blob metadata + inline small content, file-backed spill-over for large content, single commit boundary for the durability contract.
• Bucket semantics: the S3/rudolfs shape — namespace/bucket as the isolation + naming prefix. How buckets relate to the alkfs layer (alkfs buckets = alkblobs buckets? a mapping table? one alkblobs store per alkfs bucket?) is an OQ-FS-11 sub-question; multi-tenancy at the blob layer is where-clause isolation per the POC.
• sha1 vs sha256 vs both: git's dual-object-format reality (sha1 repos today, sha256 transition) says the store should carry algorithm-tagged identities (a small enum per hash) rather than hard-coding either — git object ids are natively addressable then (OQ-FS-12's coexistence sub-question mostly dissolves into this).
• Versioned-hash hazard worth recording: sha1/sha256 are not length-extended-safe for streaming concatenation the way tree hashes are — any chunk-manifest scheme must be its own construction (hash list / Merkle over chunk hashes, e.g. the git tree-object shape itself), not a naive linear hash chain. This is a design constraint, not a blocker.

Write path, branching, GC

OQ-FS-04: Hashing and chunking — object-granular identity over a chunk manifest

Reframed 2026-09-23. The identity half is decided in direction: sha1/sha256, algorithm-tagged (OQ-FS-03) — git-compatible, native to alkgit. What remains open is the dedup granularity and the manifest shape — the load-bearing storage-format one-way door once content exists in the wild:

• Whole-file sha256 (the git shape). Objects are addressed whole; identical files dedup, similar-but-not-identical do not. Simplest, matches git objects exactly, and matches the POC's tested write path verbatim. For large workspace content the dedup miss is real (two 1GB files sharing a 500MB prefix dedup zero bytes).
• Chunk manifest (the working goal for large content). A file is a root manifest over chunk hashes (a hash list — the git tree-object shape generalized, per OQ-FS-03's versioned-hash constraint); chunks dedup across files, partial availability and resumable writes come along, and per-chunk re-verification on read falls out of the manifest (each row carries its own hash — the bao-outboard role, without bao). Chunker choice (CDC vs fixed-size) and boundary-shift sensitivity are the design cost.
• Hybrid (the probable landing): whole-file identity under a size threshold (git objects, source files — effectively single-chunk anyway); chunk manifests for large content (datasets, logs, pack- scale blobs). The threshold is a knob; the manifest format for the chunked side is the one-way door.

Sub-questions: CDC (rollsum/buzhash) vs fixed-size vs git-style delta-encoding as adjacent dedup (packs delta-compress similar objects — a chunk-manifest store could optionally delta-compress chunk rows, but that is a compression layer, not identity); where chunk indices live (alkblobs manifest tables); whether cross-bucket chunk dedup is ever allowed (working answer: never — tenants are isolated, OQ-FS-11); interaction with OQ-FS-05 (a chunk-manifest write can commit incrementally, weakening the branch-on-write staging argument for chunked files — the session model may collapse to "extend the manifest" for large content).

OQ-FS-05: Write path semantics beyond the POC

Branch-on-write/merge-on-close is POC-proven for the happy path. What Phase 0 must still answer:

• Durability details — per-chunk transaction commit (POC shape) vs batched WAL; synchronous level; whether fsync of spilled large content must precede the path-row commit (principle 3) and how that is ordered when content lives outside the DB file; crash-mid-close behavior.
• Concurrency — the POC left honker named locks unwired; concurrent writers to the same path need a real policy (last-close-wins is session-safe but not POSIX; O_EXCL-ish create semantics; advisory byte-range locks for the sftp door?).
• Streaming large writes — pack generation and 1GB+ uploads: the POC's per-chunk SQLite rows are proven at ~32KB×1MB; the pack-size regime needs a spill story (OQ-FS-02's file-backed overflow) and a backpressure story inherited from alkcall channels.
• Sparse/out-of-order/ftruncate — SFTP pipelines out-of-order (proven); sparse files and explicit truncation/extend are unaddressed.

OQ-FS-06: Branch/snapshot/commit model

The POC has Fossil-style branches (name → parent chain + deltas) but no explicit commit/snapshot op — writes to a branch are immediately visible on that branch. Open questions:

• Is a snapshot a new branch (Fossil's model — maps naturally to the chain walk) or a recorded point-in-time within a branch (git's model)? What do alkgit (refs + CAS) and the sftp door each need?
• Refs as a first-class concept: alkfs stores refs rows naming commits (branch heads), enabling CAS ref transactions (alkgit's GitRefs::apply shape) over the same metadata store. Is the ref namespace git-shaped from day one, or a later addition?
• Merge semantics across branches: the POC has none (branch-on-write collapses trivially at close). Real merges (three-way? automerge-CRDT for the metadata layer per POC 3?) are presumably out of v1 — say so explicitly rather than leaving it implied.
• Chain-depth performance: the recursive CTE degrades on deep chains (POC unknown #6); resolve-cache/materialized-view shape if needed — perf probe before spec, or accept and document.

OQ-FS-07: GC and liveness

Content is shared across branches/paths by hash, so naive per-path tags (Poc unknown #5's iroh-tags shape) break sharing — deleting a path must not collect content another path/branch still names. The candidate shape: liveness = reachability walk (mark from all live branch heads

• refs + write sessions; sweep unreachable) — a metadata-store query over path rows, no separate tag table needed if blob liveness rows are refcounted or mark-computed at GC time. Sub-questions: incremental vs stop-the-world GC; tombstone retention (content behind tombstones is unreachable-but-recent — grace windows); orphaned write-session chunk reaping (crash cleanup, the POC's known orphans); cross-bucket sharing (never — buckets are tenants).

Serving protocol

OQ-FS-08: The file protocol over channels — verbs, sessions, framing

The biggest unspecced surface, and the crate's first real wire format (one-way door — ADR before the first consumer):

• Verb set. SFTP-shaped (the russh-sftp Handler mapping is proven 1:1 against the engine) vs a leaner custom set (open/read/write/ close/stat/readdir/rename/remove + extended ops). Working posture: SFTP-shaped verbs, SFTP-versioned semantics where it matters (handles, pipelined writes, SeekFrom::End cheapness).
• Session model. One channel per file session (alksocks' one-channel- per-CONNECT shape) vs one control channel multiplexing many file handles (the alktty demux shape). A directory traversal or a git clone opens many files; per-file channels spend channel IDs and opens (the 256-cap arithmetic, OQ-SK-09's twin); a multiplexed control channel reintroduces the demux trade alksocks resolved for SOCKS5 (whose RFC structure genuinely has one command per connection — a file protocol has no such constraint). This trade has different physics here than it had there; no inherited answer.
• Framing on the data path. If sessions are multiplexed, requests need IDs and payloads need framing (length-prefix + request-id — the one-way door). If per-session, the payload may be near-pass-through. Chunk sizes, backpressure (bounded buffers are inherited), and flow-control for pipelined writes all live here.
• Read-path streaming. Range reads (offset+len), EOF semantics, verified-streaming vs trust-the-store; zero-copy ambitions explicitly out (channels are the substrate, not splice(2)).
• Establishment. register_openable_with_establisher with a scope gate (OQ-FS-10); refused opens are typed errors with per-verb error mapping (SFTP status codes at the door, faithful engine errors in protocol — convention 2).

OQ-FS-09: ALPN, params, and the resource model

Provisional alk/fs (final naming per alkcall ADR-004/006 convention). Open sub-questions (mostly Phase 1 ADRs with Phase 0 ground):

• The params-object shape (ADR-039 precedent: ALPN-specific, interpreted by the open handler) — what identifies a produced filesystem resource? (bucket? branch? scope string? limits?)
• In-process and loopback consumers: the engine half must be directly embeddable (no channels) — the protocol layer is one substrate among several, generic over T (principle 6). The in-process shape is also the test seam, the alksocks POC pattern.
• Hub relay story: a filesystem resource traverses relays terminate-and-re-produce like any produced resource; nothing alkfs-specific expected — verify during POC rather than assume.

OQ-FS-10: Doors — alksftp, mounts, sync (out of crate, but shapes the API)

Family rule: doors are separate crates/features. Phase 0's job is to keep the protocol layer door-ready: the verb set (OQ-FS-08) must map onto russh-sftp's Handler without translation loss; a FUSE/mount door needs POSIX-ish error mapping and inode-ish handle semantics — do not design for FUSE in v1, but record what it would demand so the protocol doesn't preclude it. Directory-sync tooling (alkfs ↔ host dir) is the explicit, feature-gated host-FS bridge if/when wanted — never ambient.

OQ-FS-11: Multi-tenancy, identity, and path-scope policy

Buckets are the isolation unit (POC-proven free). Open questions: the ALKFS_OPEN_SCOPE-shaped scope gate (alktty/alksocks precedent) — does scope name buckets, path prefixes, or both; identity → bucket mapping at the assembly layer (vault references, never plaintext secrets — family rule); per-path/per-bucket policy as a producer-side policy callback vs declarative ACL (alksocks OQ-SK-05's dialer-refuses resolution is the evidence-backed template); write-vs-read grant separation (a read-only bucket mount is the obvious first split).

Consumer fit

OQ-FS-12: alkgit fit — the question that un-pauses alkgit

The first consumer and the reason this crate moved ahead of alkgit. The decision tree:

• (a) alkgit keeps gix-odb native; alkfs serves the general FS. Lowest integration risk; but then alkfs doesn't solve the problem that paused alkgit, and the family carries two storage stories.
• (b) alkgit's backend traits get an alkfs-backed implementation. GitPackGen streams a pack out of alkfs content (pack assembly is alkgit-side; the blob bytes come from alkfs reads); GitPackIngest streams a pack into alkfs (each unpacked object lands as content; refs land as metadata rows); GitRefs becomes CAS transactions over alkfs refs. gix-odb's on-disk layout disappears; git semantics ride alkfs's content addressing. This is the "optimize for git" path and the probable answer — but it makes alkgit's workload (pack-scale blobs, O(counts) memory, streaming) the benchmark OQ-FS-13 must meet.
• (c) gix-odb on top of alkfs (an odb backend where loose objects/packs are alkfs files) — keeps gix-odb's pack machinery but treats alkfs as the disk. Probably the worst of both: gix-odb assumes filesystem semantics alkfs won't natively provide (mmap, file locks).

Sub-questions: does git's object-id namespace (sha1/sha256) coexist with alkfs's content addresses — mostly dissolved by the identity reframe (2026-09-23): git oids are the blob layer's addresses now; what remains is whether git objects ride as whole-file identities (the natural fit — git objects are small) while alkfs path rows may point at chunk-manifest roots for large content (OQ-FS-04's hybrid), and how pack files relate (a pack is just a large blob to alkblobs; unpacked object rows are the useful representation — the ingestion mapping is the question). CAS ref transaction shape; whether pack ingestion budget enforcement (alkgit ADR-009) maps onto alkfs write sessions.

OQ-FS-13: Performance shape and budgets

No numbers exist yet. What Phase 0 should collect (empirically, via the POC register) so Phase 1 can pin budgets: path-resolve cost at realistic branch depths (POC: sub-ms in-memory; chains of 10+?); small-file throughput (the inline-BLOB sweet spot claim vs reality at 4KB metadata ops); large-file write throughput through chunked sessions; pack gen/ingest streaming rates over a channel vs native gix-odb; memory bounds under the alkcall channel caps. The alkgit ADR-009 limits shape is the consumer contract to serve.

Posture and deferred

OQ-FS-14: Distributed sync / multi-node — in scope at all?

POC 3 proved automerge-CRDT sync of the path tree (9 tests, LWW conflicts, the concurrent-root-init constraint) with content fetched separately. But sync is a product-layer concern that may not belong in the engine crate: the family pattern is composition (alkcall relay for transport, automerge as a third-party sync layer, honker queues as the local outbox). Working posture: deferred — out of v1 scope, recorded so the metadata schema doesn't preclude it (branch/paths rows are already CRDT-encodable per POC 3's document structure). Revisit when a consumer needs multi-node.

OQ-FS-15: WASM posture

The expected shape (convention 4): wasm-clean protocol/ops layer; native storage engine behind features (real FS/SQLite never compiles to wasm32-unknown-unknown). Open: does any consumer actually want the protocol layer in wasm (a sandboxed alkfs consumer, mirroring alksocks' sandboxed-SOCKS-consumer story)? If none is in sight, the honest answer may be alkhttp-style native-only (the "wasm-clean is preferred, not mandatory" precedent) — but the protocol layer being wasm-clean by construction costs little if the engine split exists anyway. Decide deliberately once the split (OQ-FS-01) exists; verify with cargo check --target wasm32-unknown-unknown.

OQ-FS-16: The remote-mount workload — the anchor use case

Added 2026-09-23. The concrete deployment this crate exists for (the user's own daily workflow): a dev-server workspace mounted on a local Linux machine under remote/ — today via sshfs/vanilla SFTP, with known slowness:

• Why vanilla SFTP is slow (the honest decomposition): a network round trip per syscall (stat storms during directory walks; sshfs even translates SeekFrom::End into a real fstat round trip); attribute-invalidation chatter; no server-side index (the SFTP door shells to real FS syscalls, each paying host-FS costs); attr-cache and cache-timeout knobs that trade staleness for speed. The POC's mapping table already noted stat → single indexed lookup is cheaper than a real FS fstat — an appfile-indexed VFS fixes the server-side half structurally.
• What the alkfs shape adds: server-side indexed metadata (stat/ readdir from index rows, no syscall per attribute); pipelined writes and batched reads native to the protocol (the russh-sftp client's write_nowait + ack window rides the chunked write session); optional client-side caching/prefetch as door-local policy (a mount door can cache content chunks locally — content-addressed means cache coherency is a branch-head-invalidation problem, not a byte-consistency problem); directory-listing snapshots.
• What this anchors: the verb set (OQ-FS-08) must make the readdir/stat/read/write path fast, not merely correct; the session model (OQ-FS-08's per-file-channel vs multiplexed question) matters most for directory-walk fan-out; OQ-FS-13's budget table should measure against sshfs-over-localhost baselines.
• alksftp vs a mount protocol: vanilla SFTP clients (sshfs) work against any SFTP door with zero client install — that compatibility is worth keeping as the floor. A leaner alkfs-native mount protocol (OQ-FS-08's framing question) is the speed tier; whether it warrants its own FUSE-facing door early or later is a sequencing question deliberately parked until the protocol shape exists.

This OQ is the consumer pull behind the whole crate — if the mount use case stops mattering, OQ-FS-08's urgency drops with it.

Candidate POC register (proposals — none run yet)

POC placement conventions (inherited from alktunnels/alksocks): a POC needing this repo's code runs in a worktree (.worktrees/research/ <task-id>/ per docs/sdd_process.md); a self-contained POC runs as a standalone crate in the global workspace with findings written into docs/research/ here. Findings always land in docs/research/.

#	Candidate	Hypothesis to validate	Feeds
1	Engine skeleton: path tree + inline-blob store in one SQLite file	One-file metadata+blob-metadata (OQ-FS-02) holds up under the POC's 15-test suite ported over; single-transaction durability ordering (principle 3) is expressible	OQ-FS-02, OQ-FS-05
2	Write path at pack scale	Chunked write sessions handle 100MB+ streams (spill-to-file regime) with correct durability ordering and crash-abort semantics	OQ-FS-05, OQ-FS-13
3	alkblobs store skeleton (sha1/sha256 identity)	Own-store direction (OQ-FS-03): algorithm-tagged hash ids, bucket isolation, inline-vs-spill layout, manifest-of-chunks for large content — measure against the probe's store-seam map as the API checklist	OQ-FS-03, OQ-FS-04
4	Chunking probe (paper + micro-bench)	Whole-file vs chunk manifest: dedup gains on realistic workloads (agent workspaces, datasets, logs — git objects are single-chunk and ride whole-file), write-path impact, metadata size — enough to decide the format one-way door or at least stage it	OQ-FS-04
5	Serving protocol spike	SFTP-verb protocol over channels: per-file-channel vs multiplexed-handles session model, open cost at directory-walk and git-clone fan-out, framing sketch (not a wire ADR)	OQ-FS-08, OQ-FS-09
6	alkgit seam spike	GitPackGen/GitPackIngest shaped against an alkblobs-backed engine — does the trait family survive contact; git oids as native addresses	OQ-FS-12
7	GC walk prototype	Reachability-mark GC over path rows + refs: correctness on shared content across branches, cost model, orphaned-session reaping	OQ-FS-07
8	Mount-workload baseline	Measure the current pain (sshfs against this dev server): op latencies for stat/readdir/read/write at realistic workspace scale — the baseline OQ-FS-16 improvements must beat	OQ-FS-16, OQ-FS-13

Order is indicative, not committed: #1/#3 de-risk the engine cores (alkfs metadata, alkblobs store); #2 stress-tests the write path; #4 resolves the storage-format one-way door; #5/#6 are only worth running once #1/#3 exist (the protocol needs an engine to serve); #8 can run anytime (it measures today's sshfs, no alkfs code needed) and its baseline ages well if taken early. The register exists so POC work has home ids; expect entries to be split, merged, or dropped as OQs sharpen.

Survey / prior-art list

Read/verified (cited above):

• alknet-filesystem POCs — /workspace/@alkdev/alknet/docs/research/alknet-filesystem/ poc-summary.md, alknet-blobs-external-store-probe.md; POC crates /workspace/alknet-filesystem-poc, /workspace/alknet-fs-sync-poc.
• iroh-blobs — /workspace/iroh-blobs (v0.100 checkout + DESIGN.md), published 0.103 in cargo cache (probe source); the family's reference notes at /workspace/@alkdev/alknet/docs/research/references/iroh/ iroh-blobs/ (overview, key types, transfer protocol, storage).
• alkgit — docs/architecture/backend.md (the trait seam), docs/research/{vision,gitoxide,git-protocol,poc2-findings}.md.
• russh-sftp — /workspace/russh-sftp (server/handler.rs, client File pipelining).
• SQLite appfileformat — https://sqlite.org/appfileformat.html; honker — /workspace/honker (honker-core 0.2.4).
• rudolfs (git-lfs server) — /workspace/@alkdev/alknet/docs/research/references/gitlfs/ rudolfs-reference.md.
• alkcall — ADRs 034/035/036/037/039/042/049/050/051, ledger CF-005/CF-006 (docs/architecture/decisions/); alktty/alktunnels/ alksocks Phase 0 docs as the format/template precedents.

To evaluate (research-specialist queue, feeding the OQs named):

• Chunking/CDC prior art — FastCDC, rollsum (bup), rsync-style boundaries; git tree-object hashing as the manifest shape (the versioned-hash constraint's precedent, OQ-FS-03/04); bao_tree's chunker as a chunk-boundary algorithm candidate (decoupled from its BLAKE3 identity).
• CAS filesystems — Plan 9 fossil+venti (the branch/CAS ancestry the POC name-checks), IPFS unixfs/iros DAG shapes, OST/lessfs-style inline-vs-outboard hybrids (OQ-FS-04, OQ-FS-06).
• git storage internals — gix-odb pack/idx layout and gix-pack::data::input streaming (alkgit research has the survey; re-read for OQ-FS-12's object/pack mapping questions).
• SQLite durability posture — WAL + synchronous=NORMAL vs FULL under the commit-boundary contract; SQLite-as-blob-store limits (OQ-FS-02, OQ-FS-05).
• FUSE/mount doors — rust bindings shape, POSIX semantics alkfs would need to fake; sshfs latency characteristics (the OQ-FS-16 baseline's "why it's slow" decomposition, verified against real measurements from POC #8 rather than folklore) (OQ-FS-10, OQ-FS-16; defer design, record constraints).

Convergence checklist (what Phase 0 must produce)

• Vision + guiding principles captured (this doc, §Vision — the scope sketch needs confirmation or cutting, OQ-FS-01)
• Prior-art pass complete: alknet-filesystem POCs re-read and mapped (done); iroh-blobs surface re-verified against the probe's 0.103 findings; git/git-lfs shape pinned against alkgit's backend contract; CDC/chunking survey (OQ-FS-04 input); CAS-filesystem survey
• OQ-FS-01 (crate scope) — converged: the alkblobs/alkfs split confirmed or cut; repo shape (workspace vs two repos); trait seam at the blob boundary or concrete dependency; explicit non-goals
• OQ-FS-02 (metadata store) — converged: substrate, one-file-vs-two (blob metadata placement, now scoped to alkblobs), durability knob
• OQ-FS-03 (blob substrate) — converged: the own-store direction pinned down (what transfers as findings, what's built fresh) (POC #3)
• OQ-FS-04 (hashing/chunking) — converged: identity settled in direction (sha1/sha256); dedup granularity + manifest format are the one-way door (POC #4)
• OQ-FS-05 (write path) — durability/concurrency/spill semantics pinned (POC #2)
• OQ-FS-06 (branch/snapshot/refs) — model chosen; ref namespace decided; merge explicitly in-or-out of v1
• OQ-FS-07 (GC) — liveness shape chosen; retention/reaping policy pinned (POC #7)
• OQ-FS-08 (file protocol) — verbs, session model, framing sketch; the wire ADR itself is Phase 1's (one-way door, first consumer in sight) (POC #5)
• OQ-FS-09 (ALPN/params) — naming + params ground collected; ADR in Phase 1
• OQ-FS-10 (doors) — protocol kept door-ready; nothing designed in
• OQ-FS-11 (tenancy/policy) — scope-string + policy shape settled; bucket semantics across the alkblobs/alkfs seam
• OQ-FS-12 (alkgit fit) — the (a)/(b)/(c) decision made; alkgit un-pause plan written (POC #6)
• OQ-FS-13 (budgets) — numbers collected (incl. the sshfs baseline, POC #8); budget table drafted
• OQ-FS-14 (sync) — explicitly deferred or scoped with evidence
• OQ-FS-15 (wasm) — decided once the engine/protocol split exists
• OQ-FS-16 (remote mount) — the anchor workload's requirements written down (verb-set implications, caching shape); sequencing (SFTP floor first vs native protocol tier) decided
• Targeted POCs run + findings in docs/research/
• Converge: recommended approach written up, ready to hand to the Architect for Phase 1