The call protocol spec describes streaming (call.responded*N +
call.completed, PendingRequestMap::Subscribe, CallConnection::subscribe),
but the server-side Handler type returned a single ResponseEnvelope —
a Subscription op had no way to produce a stream. The TS predecessor
(@alkdev/operations) had separate OperationHandler / SubscriptionHandler
types; the Rust port collapsed them, losing the streaming path. This
restores it end-to-end: StreamingHandler type, HandlerKind on
HandlerRegistration validated against op_type, invoke_streaming() on
OperationRegistry, server-side dispatch branches on op_type, new
INVALID_OPERATION_TYPE protocol code for wrong-dispatch-path misuse,
GatewayDispatch::invoke_streaming() for /subscribe SSE, from_call stream
forwarding via CallConnection::subscribe(), from_openapi SSE forwarding.
OperationEnv::invoke() stays request/response-only (stream composition is
handler-level, not protocol-level). Amends ADR-023's protocol-code list
(five → six). Tracks the stream-operators library as OQ-41 (feature
extension, not an unmade decision).
Commits the concrete adapter shape deferred by ADR-033: read-sync /
write-async split with honker NOTIFY/LISTEN for no-restart cache
invalidation, against SQLite, in a separate alknet-store-sqlite crate.
Two constraints drive the design: (1) the hot-path read trait
(IdentityProvider::resolve_from_fingerprint, CredentialStore::get) is
sync — called in the accept loop, no .await — so a SQLite-backed
adapter must cache in memory and serve sync reads from the cache; (2)
auth changes must take effect without a restart (an early issue the
project already fixed for ConfigIdentityProvider via ArcSwap config
reload). honker's SQLite NOTIFY/LISTEN (single-digit-ms wake, no
polling) is the cache-invalidation mechanism that makes both hold:
write commits to SQLite + emits NOTIFY, the running process's LISTEN
wakes, the in-memory index reloads and atomically swaps, the next
read sees the new state. Same ArcSwap-reload pattern as config,
generalized from 'config file is source of truth' to 'SQLite is
source of truth, honker signals when it changed.'
New async IdentityStore write trait (put_peer / update_peer /
remove_peer) extends the sync IdentityProvider read trait for peer
mutations. ConfigIdentityProvider does NOT implement it (config
reload is its write path — a posture enforced by the absence of a
backend, not a type-system constraint); SqliteIdentityProvider
implements both. CredentialStore::put/delete refined to async (within
ADR-031's one-way door — the contract was get/put/delete keyed by
provider persisting EncryptedData never decrypting; sync-vs-async was
unspecified). CredentialStoreError renamed to shared StoreError
covering both traits.
alknet-store-sqlite is one crate implementing both IdentityStore and
CredentialStore with shared SQLite connection + honker LISTEN infra
(splitting later is a two-way door). Schema shape committed (one row
per PeerEntry with JSON columns for fingerprints/scopes/resources;
one row per EncryptedData blob keyed by provider); exact DDL is an
implementation-detail two-way door in the adapter crate. The keypal
adapter-factory pattern is intentionally not ported to Rust (runtime
column-mapping is a TS affordance; in Rust each adapter is a concrete
type, cross-cutting concerns are a shared helper module).
Amends ADR-031 (put/delete async refinement, StoreError rename),
ADR-033 (concrete adapter shape now specified, two-crate framing
collapsed to one), ADR-034 (OQ-36 now resolved), auth.md (IdentityStore
section, cache-invalidation summary, OQ-36 reference), config.md (two
write paths note), and the OQ-36/OQ-34 entries in open-questions.md.
Review fixed 4 criticals (error-type name divergence, duplicate
IdentityProvider sketch, upsert/Duplicate ambiguity, 'shape unchanged'
contradiction), 7 warnings, 5 suggestions.
Untangles the conflation of three distinct remote roles under 'X.509
endpoint': (1) public X.509 endpoint — a remote HTTPS/call-over-TLS
server the local node is a client of (no PeerEntry, no PeerId, not in
the peer graph; CA verification + bearer token); (2) transport relay —
iroh's DERP-equivalent, infrastructure, not an alknet peer; (3) hub /
hosting node — an alknet peer that also exposes a public domain + X.509
for browsers (mixed-fingerprint PeerEntry, already supported by
ADR-030).
The load-bearing one-way door is the client-side verifier selection
rule: known peer (PeerEntry present) → fingerprint pin; unknown X.509
remote → CA verification (WebPkiServerVerifier); unknown Ed25519
remote → fails closed. This closes the AcceptAnyServerCertVerifier
security hole OQ-29 flagged, with the peer-model criterion (PeerEntry
presence) made explicit. The 'make PeerEntry symmetric' instinct is
rejected — pure-client connections to public APIs have no stable
logical identity to pin.
Documents that CallCredentials.remote_identity: None is load-bearing
(None = public X.509 endpoint → CA path, not a missing field; Some =
known peer → fingerprint pin), closing a subtle gap where an
implementer could have defaulted to a placeholder or treated None as
skip-verify.
Records WebTransport relay-as-proxy (deferred with h3/WebTransport,
new OQ-HTTP-07) and on-chain/smart-contract peer discovery (fits the
OQ-36 repo/adapter pattern, no auth-model change) so they aren't lost.
Amends auth.md and client-and-adapters.md with the three-role naming,
the verifier selection rule, and the Option semantics; updates OQ-37
to resolved in open-questions.md, README.md, and both crate READMEs.
Amend ADR-030 with three changes from the auth-type analysis:
1. PeerEntry is now multi-credential: fingerprints: Vec<String> (Ed25519
and/or X.509) + auth_token_hash: Option<String> (bearer token). All
resolve to the same peer_id. A peer that authenticates via Ed25519
today and via auth_token tomorrow gets the same PeerId. The 'peer
bearer vs auth bearer' distinction was wrong — the correct framing is
the three credential types (Ed25519, X.509, bearer token) and whether
the token needs a stable logical id across rotation (PeerEntry) or not
(ApiKeyEntry).
2. Fingerprint normalization (§6): quinn extracts the raw Ed25519 public
key from the SPKI cert and formats as ed25519:<hex>, matching iroh.
The same key has the same fingerprint regardless of transport. X.509
fingerprints stay as SHA256:<hex of DER>. This also simplifies the
coming WebTransport relay work.
3. The 'API keys' section is replaced with 'Bearer tokens' — correctly
framing the three auth types and the two bearer-token paths
(PeerEntry.auth_token_hash vs ApiKeyEntry).
Resolve OQ-29 (CallClient TLS client-auth): wire quinn client-auth (present
Ed25519 key as raw public key client cert — the server-side extraction
already works); key-type-aware server cert verification (raw key =
fingerprint match, X.509 = CA verification via WebPkiServerVerifier —
AcceptAnyServerCertVerifier is only safe for raw keys); fingerprint
normalization. The iroh path already works (RFC 7250 raw keys, both sides
exchange automatically); the gap was quinn-only.
Dissolve OQ-35: the 'API key asymmetry' framing was wrong. PeerEntry
supports multiple credential paths; ApiKeyEntry is for tokens that ARE the
identity.
Add OQ-37: X.509 outgoing-only case — the three auth types and how X.509
server identity fits the peer model. Not blocking the ADR-029 migration;
downstream (HTTP crate phase).
Update auth.md, config.md, client-and-adapters.md, call/README.md,
core/README.md, open-questions.md, README.md, and call_client.rs source
comment.
Workspace green: 326 tests pass, build clean.
Resolve the call-crate open questions where the decision is made —
OQ-27 (auto-re-import), OQ-28 (same-peer collision = error), OQ-30
(PeerRef::Any insertion-order first-match), OQ-31 (services/list-peers
opt-in). These were previously marked 'open' with 'v1' hedging language
despite having a decided default. What remains (refresh(), richer routing,
services/list-peers the op) is genuine feature addition, not unmade
architecture.
Reframe OQ-32 (multi-hop) as a feature extension rather than a 'v1'
deferral — the one-hop model is the architectural commitment; extending
to multi-hop doesn't break downstream.
Promote OQ-29 (CallClient TLS client-auth) from medium to high priority
and surface its real interaction with ADR-030. Previously framed as
'additive — two-way-door remainder,' but ADR-030's PeerEntry fingerprint
→ peer_id resolution requires the client to present a TLS client cert.
With with_no_client_auth(), no fingerprint is extracted, the PeerEntry
path is dormant, and PeerCompositeEnv keys on None or the API-key prefix
instead of the stable peer_id. This is the activation path for ADR-030's
primary use case, not an additive feature. Three options laid out: (a)
wire client-auth with the ADR-029 migration, (b) ship token-only and
switch later (the 'compounds into a mess' path), (c) extend PeerEntry
to cover auth_token-based identity. Requires a decision before the
migration lands.
Clarify OQ-36 (concrete adapter shapes): the trait shapes and in-memory
adapters ship with core — the deferral is only for the persistence
adapters (SQLite, etc.). The in-memory adapters are real implementations
of a full repo pattern, not stubs.
Update call_client.rs source comment to reference OQ-29 instead of the
'v1' / 'two-way-door remainder' framing.
Workspace green: 326 tests pass, build clean.
Land the storage and auth strategy research (findings.md) as four
accepted ADRs and amend the core and call specs to match:
- ADR-030: PeerEntry and Identity.id decoupling. Replaces
authorized_fingerprints with peers: Vec<PeerEntry>; Identity.id becomes
the stable peer_id, decoupled from the rotating fingerprint. Supersedes
ADR-029 Assumption 1's UUID source (one-way door preserved, source
changes). Resolves OQ-33 and the storage-boundary half of OQ-34. Records
the API-key asymmetry as deliberate (OQ-35).
- ADR-031: CredentialStore repo trait + InMemoryCredentialStore default
adapter in core. Second repo trait alongside IdentityProvider. Vault
encrypts; the store persists the EncryptedData blob; assembly layer
loads into Capabilities. EncryptedData core mirror includes salt for
wire-format compat.
- ADR-032: Forwarded-for identity. forwarded_for field on call.requested
and OperationContext — metadata only, never read by AccessControl::check
(enforced structurally via the check signature). The from_call handler
populates it. Wire-format one-way door, folded into the ADR-029
migration window.
- ADR-033: Storage boundary and repo/adapter pattern. Core defines repo
traits + in-memory defaults; persistence adapters are separate crates;
assembly layer wires. Resolves OQ-34. Concrete adapter shapes deferred
for exploration (OQ-36).
Amends auth.md, config.md, operation-registry.md, client-and-adapters.md,
open-questions.md, README.md, crates/core/README.md. Marks ADR-029
Accepted (Assumption 1 carries the ADR-030 superseded note). Marks the
research findings doc reviewed.
OQ-26 (resolved): AdapterError variants decided — DiscoveryFailed,
SchemaParse, Transport, Unauthorized, SamePeerCollision (replaces flat
Conflict per ADR-029 §5). #[non_exhaustive] for downstream extension.
Two-way door; the initial set is the code's return type.
OQ-33 (resolved): PeerId is a logical identifier, NOT Identity.id. The
research's v1 default (PeerId = fingerprint) is overridden: coupling PeerId
to crypto material breaks every in-flight PeerRef::Specific and every ACL
entry on key rotation. v1 source is a connection-assigned UUID — a
no-storage workaround that works for the immediate use case (head→workers,
reconnect produces fresh PeerRef, in-flight gets NOT_FOUND which is correct).
The one-way door: PeerId is logical, not crypto — this determines
PeerCompositeEnv key type and PeerRef::Specific payload. The id source
(UUID vs configured name vs peer registry) is the two-way-door remainder.
OQ-34 (new): the storage dimension OQ-33 surfaced. The core crates are
deliberately DB-free (smaller, fewer deps, simpler testing) — this served
local-only state (vault, registry) well, but peer identity is the first
cross-node state that wants persistence. The real solution (a persistent
peer registry mapping stable logical name → current crypto material,
surviving key rotation) is not a v1 blocker (UUID works), but tracked so the
no-DB posture's limit is deliberate, not accidental. The storage boundary
(core gets a PeerRegistry trait vs stays storage-free) is the one-way door;
the backend choice is two-way. Key-rotation/ACL note: decoupling PeerId from
crypto keeps the door open for ACL entries that persist across key rotation
— when the peer registry is built, ACLs key on the logical name and key
rotation becomes vault-only with no remote-side ACL update.
ADR-028's remote_safe/trusted_peer was a parallel, weaker authorization system
that duplicated the existing AccessControl/Identity machinery and couldn't
express the head→N-workers pattern (the primary use case). The flat-namespace
single-peer overlay model (one connection layer in CompositeOperationEnv)
structurally breaks the moment a head has two workers both exposing
/container/exec.
ADR-029 replaces it with:
- Peer-keyed overlays: PeerCompositeEnv { connections: HashMap<PeerId, ...> }
replaces CompositeOperationEnv's singular connection layer. A head node
routes invoke_peer() to the right peer via PeerRef::Specific / PeerRef::Any.
- AccessControl-based peer authorization: the existing AccessControl::check
(peer_identity) gates peer calls — the same mechanism that gates every other
call. remote_safe/trusted_peer/RemoteFilter/list_operations_peer_scoped/
services_list_handler_peer_scoped are retired. The op's AccessControl IS the
peer-authorization policy; no parallel system.
- ScopedPeerEnv: peer-qualified reachability (peer-pinned allowlist) replaces
from_call's namespace_prefix as the disambiguation mechanism. Cross-peer
collision dissolves (separate sub-overlays); same-peer collision stays error.
- services/list-peers opt-in for peer-attributed re-export listing.
POC-validated against real types (scratch module written, type-checked,
removed; build clean, 207 tests pass). Petgraph not needed for v1 (one-hop,
shallow); nested HashMap suffices; extends to multi-hop without redesign (OQ-32).
OQ impact: OQ-25 dissolved (no marking); OQ-28 cross-peer dissolved / same-peer
stays; OQ-26/27/29 stay; new OQ-30 (Any routing policy), OQ-31 (list-peers
semantics), OQ-32 (multi-hop federation).
Research: docs/research/alknet-call-peer-routing/findings.md (POC shapes,
prior art — Ray.io actors, Dapr service invocation, full ADR draft).
ADR-028 marked Superseded; ADR-017 DC-1 amendment updated to point at ADR-029.
Post-implementation spec sync after the call-completion batch landed
(commits e4a2594..a3825f5). The sub-agent review flagged no spec drift, but
comparing the implemented types against the spec sketches surfaced five
details the specs didn't name — filled in here so the spec matches what was
built:
- client-and-adapters.md: name the shared Dispatcher (protocol/dispatch.rs)
+ RemoteFilter mechanism that enforces ADR-028's default-deny at dispatch
time (the load-bearing security gate — checks remote_safe before building
context, before any capability material reaches the handler). Add
ClientError/RemoteIdentity types, the spawn_dispatch lower-level API, and
the services_list_handler_peer_scoped wiring (the assembly layer must
register the peer-scoped services/list handler for a CallClient's registry,
not the plain one). Record the v1 TLS client-auth gap (AcceptAnyServerCertVerifier,
with_no_client_auth) as OQ-29.
- call-protocol.md: point the adapter dispatch-loop description at the shared
Dispatcher (dispatch.rs) so readers find the mechanism ADR-017 §1 commits to.
- open-questions.md: OQ-29 — CallClient TLS client-auth + remote-identity
verification is a two-way-door remainder; the no-env-vars invariant is
unaffected (auth_token flows via call-protocol payload, not TLS).
- READMEs: current-state now reflects completion done + reviewed (207 lib +
2 integration tests); OQ-29 added to both OQ summaries.
Resolves the four gap-analysis decisions (DC-1..4) blocking the alknet-call
client/adapter surface specced in ADR-017:
- ADR-028 (new): locks the one-way door for DC-1 — CallClient registry is
default-deny (remote_safe: bool on HandlerRegistration, default false across
all provenance); share-global is an explicit trusted-peer opt-in; filtering
is a dispatch-time read over the single Layer-0 registry, not a copy.
- client-and-adapters.md (new spec): operationally fills the gap ADR-017 left
to implementation — CallClient, from_call, from_jsonschema, OperationAdapter
trait, adapter location map, no-env-vars invariant, exchange-of-operations
pattern. Keeps call-protocol.md and operation-registry.md under the
700-line split threshold.
- ADR-017 amended: records DC-2/3/4 v1 defaults (auto-on-reconnect,
error-on-collision, Result error type) and points DC-1 at ADR-028.
- OQ-25..28 (new): two-way-door remainders (remote_safe shape, AdapterError
variants, re-import trigger, namespace collision) with v1 defaults recorded.
- Index/cross-ref updates across READMEs and the two existing call specs.
Tasks: 6 task files under tasks/call/ decomposing the completion work along
the gap-analysis priority order — remote-safe-marking (one-way door, first)
→ call-client (phase-risk) → from-call → operation-adapter-trait →
from-jsonschema (parallel with call-client) → review-completion. Graph
validated with taskgraph; parallelism designed in (from-jsonschema runs
concurrent with call-client/from-call once the trait lands).
