# iroh-docs: Engine and Live Sync ## Overview The `Engine` is the top-level coordinator for live document synchronization. It brings together: 1. **SyncHandle/Actor** — Single-threaded actor for all store and replica operations 2. **LiveActor** — Async event loop coordinating sync, gossip, and content downloads 3. **GossipState** — Integration with `iroh-gossip` for broadcasting updates 4. **Blobs/Downloader** — Integration with `iroh-blobs` for content transfer ## Engine ```rust pub struct Engine { pub endpoint: Endpoint, pub sync: SyncHandle, pub default_author: DefaultAuthor, to_live_actor: mpsc::Sender, actor_handle: AbortOnDropHandle<()>, content_status_cb: ContentStatusCallback, blob_store: iroh_blobs::api::Store, _gc_protect_task: AbortOnDropHandle<()>, } ``` ### Initialization ```rust Engine::spawn( endpoint, // iroh Endpoint for QUIC connections gossip, // iroh-gossip instance replica_store, // Store for document data bao_store, // iroh-blobs Store for content blobs downloader, // Downloader for fetching blobs default_author_storage, // Where to persist the default author protect_cb, // Optional GC protection callback ) -> Result ``` During spawn: 1. A `ContentStatusCallback` is created that checks blob availability in `iroh-blobs` 2. A `SyncHandle` actor is spawned on a dedicated thread 3. A `LiveActor` is spawned as a tokio task 4. The default author is loaded or created 5. A GC protection task is started (if callback provided) ### Key Engine Methods ```rust // Start syncing a document with given peers async fn start_sync(&self, namespace: NamespaceId, peers: Vec) -> Result<()> // Stop syncing and leave gossip swarm async fn leave(&self, namespace: NamespaceId, kill_subscribers: bool) -> Result<()> // Subscribe to document events async fn subscribe(&self, namespace: NamespaceId) -> Result>> // Handle incoming QUIC connections async fn handle_connection(&self, conn: Connection) -> Result<()> // Shutdown the engine async fn shutdown(&self) -> Result<()> ``` ### GC Protection The `ProtectCallbackHandler` bridges iroh-docs with iroh-blobs' garbage collection: ```rust let (handler, protect_cb) = ProtectCallbackHandler::new(); // protect_cb goes into iroh-blobs GC config // handler goes into Engine::spawn ``` When iroh-blobs runs GC, it calls `protect_cb` which queries the docs store for all content hashes, ensuring blobs referenced by document entries are not garbage-collected. ## SyncHandle / Actor The `SyncHandle` is a handle to a single-threaded actor that processes all store and replica operations sequentially: ```rust pub struct SyncHandle { tx: async_channel::Sender, join_handle: Arc>>, metrics: Arc, } ``` ### Actor Architecture ``` External Code ──async──▶ SyncHandle ──channel──▶ Actor Thread │ Store (redb) Replica operations Flush on timeout (500ms) ``` The actor runs on a **dedicated OS thread** (not a tokio task), using `tokio::runtime::Builder::new_current_thread()` internally. This ensures store operations are never concurrent. ### Action Types ```rust enum Action { ImportAuthor { author, reply }, ExportAuthor { author, reply }, DeleteAuthor { author, reply }, ImportNamespace { capability, reply }, ListAuthors { reply }, ListReplicas { reply }, ContentHashes { reply }, FlushStore { reply }, Replica(NamespaceId, ReplicaAction), Shutdown { reply }, } enum ReplicaAction { Open { reply, opts }, Close { reply }, GetState { reply }, SetSync { sync, reply }, Subscribe { sender, reply }, Unsubscribe { sender, reply }, InsertLocal { author, key, hash, len, reply }, DeletePrefix { author, key, reply }, InsertRemote { entry, from, content_status, reply }, SyncInitialMessage { reply }, SyncProcessMessage { message, from, state, reply }, GetSyncPeers { reply }, RegisterUsefulPeer { peer, reply }, GetExact { author, key, include_empty, reply }, GetMany { query, reply }, DropReplica { reply }, ExportSecretKey { reply }, HasNewsForUs { heads, reply }, SetDownloadPolicy { policy, reply }, GetDownloadPolicy { reply }, } ``` ### Replica Opening When a replica is opened via the actor, an `OpenReplica` struct is created: ```rust struct OpenReplica { info: ReplicaInfo, // Capability, subscribers, content status callback sync: bool, // Whether to accept sync requests handles: usize, // Reference count for open handles } ``` Multiple handles to the same replica are supported via reference counting. ## LiveActor The `LiveActor` is the central async coordinator: ```rust pub struct LiveActor { inbox: mpsc::Receiver, sync: SyncHandle, endpoint: Endpoint, bao_store: Store, downloader: Downloader, memory_lookup: MemoryLookup, replica_events_tx: async_channel::Sender, replica_events_rx: async_channel::Receiver, sync_actor_tx: mpsc::Sender, gossip: GossipState, running_sync_connect: JoinSet, running_sync_accept: JoinSet, download_tasks: JoinSet, missing_hashes: HashSet, queued_hashes: QueuedHashes, hash_providers: ProviderNodes, subscribers: SubscribersMap, state: NamespaceStates, metrics: Arc, } ``` ### Event Loop The `LiveActor::run_inner()` loop uses `tokio::select!` with biased polling: ```rust tokio::select! { biased; msg = self.inbox.recv() => { /* handle actor messages */ } event = self.replica_events_rx.recv() => { /* handle replica insert events */ } res = self.running_sync_connect.join_next() => { /* sync connect finished */ } res = self.running_sync_accept.join_next() => { /* sync accept finished */ } res = self.download_tasks.join_next() => { /* download completed */ } res = self.gossip.progress() => { /* gossip task progress */ } } ``` ### ToLiveActor Messages ```rust pub enum ToLiveActor { StartSync { namespace, peers, reply }, Leave { namespace, kill_subscribers, reply }, Shutdown { reply }, Subscribe { namespace, sender, reply }, HandleConnection { conn }, AcceptSyncRequest { namespace, peer, reply }, IncomingSyncReport { from, report }, NeighborContentReady { namespace, node, hash }, NeighborUp { namespace, peer }, NeighborDown { namespace, peer }, } ``` ### Gossip Operations (Op) ```rust pub enum Op { Put(SignedEntry), // New entry inserted ContentReady(Hash), // Content blob now available SyncReport(SyncReport), // Heads summary after sync } ``` Gossip broadcasts `Op` messages to all swarm participants. When a `Put` is received, the entry is inserted into the local replica. When a `ContentReady` is received, peers know they can download the blob. When a `SyncReport` is received, peers check `has_news_for_us()` to decide if they should sync. ### Content Download Flow 1. When a `RemoteInsert` event occurs with `should_download: true`, the entry's content hash is queued for download 2. The `LiveActor` uses `iroh_blobs::downloader::Downloader` to fetch the blob 3. Known providers (peers who had `ContentStatus::Complete`) are used as download sources 4. On download completion, a `LiveEvent::ContentReady` event is emitted ### LiveEvent (Public API) ```rust pub enum LiveEvent { InsertLocal { entry: Entry }, InsertRemote { from: PublicKey, entry: Entry, content_status: ContentStatus }, ContentReady { hash: Hash }, PendingContentReady, NeighborUp(PublicKey), NeighborDown(PublicKey), SyncFinished(SyncEvent), } ``` `SyncEvent` wraps `SyncFinished`: ```rust pub struct SyncFinished { pub namespace: NamespaceId, pub peer: PublicKey, pub outcome: SyncOutcome, pub timings: Timings, } ``` ## NamespaceStates ```rust pub struct NamespaceStates(BTreeMap); struct NamespaceState { nodes: BTreeMap, may_emit_ready: bool, } ``` Each peer has a `PeerState` tracking sync progress: ```rust struct PeerState { state: SyncState, // Idle or Running resync_requested: bool, // Whether a resync was requested during active sync last_sync: Option<(Instant, Result)>, } ``` This state machine prevents concurrent syncs with the same peer for the same namespace and queues resync requests when needed. ## DefaultAuthor ```rust pub struct DefaultAuthor { value: RwLock, storage: DefaultAuthorStorage, } ``` - `DefaultAuthorStorage::Mem` — Ephemeral, creates a new author each time - `DefaultAuthorStorage::Persistent(path)` — Stores the author ID as hex in a file, loads it on startup The default author provides a convenient "current user" identity for applications. ## Docs Protocol Handler ```rust pub struct Docs { engine: Arc, api: DocsApi, } ``` `Docs` implements `ProtocolHandler` for integration with iroh's `Router`: ```rust impl ProtocolHandler for Docs { async fn accept(&self, connection: Connection) -> Result<(), AcceptError> { ... } async fn shutdown(&self) { ... } } ``` The `Builder` pattern configures storage: ```rust let docs = Docs::memory() .spawn(endpoint, blobs, gossip) .await?; // or let docs = Docs::persistent(path) .protect_handler(handler) .spawn(endpoint, blobs, gossip) .await?; ``` ## DocTicket ```rust pub struct DocTicket { pub capability: Capability, pub nodes: Vec, } ``` A `DocTicket` encapsulates everything needed to join a document: - A `Capability` (Read or Write) — provides the namespace key - A list of `EndpointAddr` — bootstrap peers to connect to Tickets are serialized as base32-encoded postcard data with a `"doc"` prefix, using the `iroh_tickets::Ticket` trait.