Land the storage and auth strategy research (findings.md) as four accepted ADRs and amend the core and call specs to match: - ADR-030: PeerEntry and Identity.id decoupling. Replaces authorized_fingerprints with peers: Vec<PeerEntry>; Identity.id becomes the stable peer_id, decoupled from the rotating fingerprint. Supersedes ADR-029 Assumption 1's UUID source (one-way door preserved, source changes). Resolves OQ-33 and the storage-boundary half of OQ-34. Records the API-key asymmetry as deliberate (OQ-35). - ADR-031: CredentialStore repo trait + InMemoryCredentialStore default adapter in core. Second repo trait alongside IdentityProvider. Vault encrypts; the store persists the EncryptedData blob; assembly layer loads into Capabilities. EncryptedData core mirror includes salt for wire-format compat. - ADR-032: Forwarded-for identity. forwarded_for field on call.requested and OperationContext — metadata only, never read by AccessControl::check (enforced structurally via the check signature). The from_call handler populates it. Wire-format one-way door, folded into the ADR-029 migration window. - ADR-033: Storage boundary and repo/adapter pattern. Core defines repo traits + in-memory defaults; persistence adapters are separate crates; assembly layer wires. Resolves OQ-34. Concrete adapter shapes deferred for exploration (OQ-36). Amends auth.md, config.md, operation-registry.md, client-and-adapters.md, open-questions.md, README.md, crates/core/README.md. Marks ADR-029 Accepted (Assumption 1 carries the ADR-030 superseded note). Marks the research findings doc reviewed.
16 KiB
ADR-030: PeerEntry and Identity.id Decoupling
Status
Accepted (supersedes the "v1 UUID" source in ADR-029 Assumption 1; resolves the "real solution" half of OQ-33 and the storage-boundary half of OQ-34)
Context
Identity.id is the string that keys authorization decisions across the
alknet crate graph. Today it is coupled to the cryptographic material:
// crates/alknet-core/src/config.rs — current implementation
pub struct AuthPolicy {
pub authorized_fingerprints: HashSet<String>, // just strings, no stable id
pub api_keys: Vec<ApiKeyEntry>,
}
impl AuthPolicy {
pub fn resolve_identity_from_fingerprint(&self, fingerprint: &str) -> Option<Identity> {
if self.authorized_fingerprints.contains(fingerprint) {
Some(Identity {
id: fingerprint.to_string(), // ← identity IS the crypto material
scopes: vec!["relay:connect".to_string()],
...
})
}
}
}
This coupling is a latent bug for any cross-node authorization decision:
- A TLS fingerprint or raw-key identity changes when the node rotates its key.
- When it changes, every ACL entry that references the old fingerprint stops matching — the peer "disappears" from the authorization system even though it is the same logical node.
PeerRef::Specific(PeerId)(ADR-029) routes byIdentity.id; a key rotation would break in-flight routing references the same way.- The hub's
authorized_fingerprintsset has to be manually updated on every rotation on the remote side, which is exactly the operational pain the vault's local key rotation (ADR-021) was meant to remove.
ADR-029 §1 set PeerId = Identity.id and made PeerId a logical identifier
"NOT Identity.id (the fingerprint)" — but left the source of that logical
identifier as a connection-assigned UUID (OQ-33's v1 workaround). The UUID
is ephemeral: it survives only for the connection's lifetime, changes on
reconnect, and cannot persist across restarts or key rotations. It is a
no-storage workaround, not a real identity.
The research at docs/research/alknet-storage-strategy/findings.md §4
established the real fix: introduce a PeerEntry config model that maps a
stable logical peer id to its current cryptographic material and
authorization scopes, and have ConfigIdentityProvider resolve
fingerprint → PeerEntry → Identity { id: peer_entry.peer_id, scopes: peer_entry.scopes, ... }. The Identity.id becomes the stable peer_id,
decoupled from the fingerprint. Key rotation is a single field update in the
peer entry; the peer_id and every ACL / routing reference to it stay
stable.
This is the storage-boundary question OQ-34 tracks. With ADR-033 (the
repo/adapter pattern) establishing that core defines repo traits and the
default in-memory adapter lives alongside the trait, the answer is: core
gets the PeerEntry config model and the
ConfigIdentityProvider::resolve_from_fingerprint → Identity { id: peer_id } resolution path now, with no SQLite dependency in core. A future
alknet-peer-store-sqlite adapter that persists PeerEntry records is
additive — it implements the same IdentityProvider trait against a peers
table instead of config. The trait is the one-way door; the adapter is the
two-way door.
Decision
1. Add PeerEntry to AuthPolicy, replacing authorized_fingerprints
pub struct PeerEntry {
/// Stable logical peer id ("worker-a", "alice"). Does NOT change on
/// key rotation. This becomes Identity.id on resolution.
pub peer_id: String,
/// Current cryptographic material — the fingerprint the endpoint
/// extracts from the TLS handshake (SHA256:... for X.509, ed25519:...
/// for RFC 7250 raw keys). Changes on key rotation.
pub fingerprint: String,
/// Authorization scopes granted to this peer. Resolved into
/// Identity.scopes.
pub scopes: Vec<String>,
/// Named resource lists granted to this peer. Resolved into
/// Identity.resources. Populated from config (not just composition, as
/// the pre-ADR-030 limitation in auth.md §"Resource-scoped ACLs and
/// external identities" required).
pub resources: HashMap<String, Vec<String>>,
/// Human-readable display name for logs / UIs. Optional.
pub display_name: Option<String>,
/// Whether this peer is authorized at all. false = the fingerprint
/// is recognized but the peer is disabled (token-revoked-equivalent
/// for fingerprints). Resolution returns None.
pub enabled: bool,
}
pub struct AuthPolicy {
/// Replaces authorized_fingerprints: HashSet<String>. Each entry maps
/// a stable logical peer_id to its current fingerprint + scopes +
/// resources. The list is keyed by peer_id; resolution looks up by
/// fingerprint.
pub peers: Vec<PeerEntry>,
/// API keys — unchanged by this ADR (see "API keys" below).
pub api_keys: Vec<ApiKeyEntry>,
}
2. Identity.id becomes PeerEntry.peer_id on fingerprint resolution
ConfigIdentityProvider::resolve_from_fingerprint resolves fingerprint →
matching PeerEntry → Identity { id: peer_entry.peer_id, scopes: peer_entry.scopes, resources: peer_entry.resources }. The Identity.id is
the stable peer_id, not the fingerprint.
impl AuthPolicy {
pub fn resolve_identity_from_fingerprint(&self, fingerprint: &str) -> Option<Identity> {
self.peers.iter()
.find(|p| p.enabled && p.fingerprint == fingerprint)
.map(|p| Identity {
id: p.peer_id.clone(),
scopes: p.scopes.clone(),
resources: p.resources.clone(),
})
}
}
This removes the pre-ADR-030 limitation in auth.md
§"Resource-scoped ACLs and external identities" — fingerprint-resolved
identities now carry resources from the PeerEntry, not just from the
composition path. The composition path (CompositionAuthority::as_identity,
ADR-015/022) still produces its own Identity for internal calls; the
external-auth path now also carries resources when configured.
3. Key rotation is a single PeerEntry.fingerprint update
Rotating a peer's TLS key:
- The vault derives the new key locally (ADR-020/021).
- The remote side's config updates the
PeerEntry.fingerprintfield for thatpeer_id. Thepeer_id,scopes,resources, ACL entries, and anyPeerRef::Specific(peer_id)references stay stable. - A config reload (
ConfigReloadHandle::reload) makes the change live.
No ACL update, no routing reference invalidation, no peer "disappears." The vault's local rotation + a remote-side config edit is the full key rotation story across nodes.
4. PeerId source changes from UUID to Identity.id from PeerEntry
ADR-029 Assumption 1 said PeerId is a connection-assigned UUID (v4). With
Identity.id now stable (peer_id), the UUID workaround is no longer
needed: PeerId = Identity.id from IdentityProvider resolution. This is
the one-way-door tightening — PeerId was always specified as logical-not-
crypto (ADR-029), the UUID was the source; the source now becomes the
auth system.
// ADR-029 §1, updated by this ADR:
pub type PeerId = String; // = Identity.id from IdentityProvider resolution
// = PeerEntry.peer_id (stable, not crypto material)
ADR-029 §2's invoke_peer / PeerRef::Specific(PeerId) signatures are
unchanged. The PeerId payload is now stable across reconnects and key
rotations, instead of ephemeral. An in-flight PeerRef::Specific that
survives a reconnect now keeps resolving (the peer_id is unchanged), which
is the property the UUID workaround could not provide.
5. The PeerId for a connection comes from IdentityProvider resolution
The dispatch path that builds a CallConnection and assigns a PeerId to
the peer-keyed overlay (PeerCompositeEnv::attach_peer) reads
connection.identity().id — the resolved Identity.id from the
IdentityProvider. If identity resolution returns None (no client cert,
unrecognized fingerprint), the peer has no PeerId and the connection
cannot be added to the peer-keyed overlay. The handler either rejects the
connection or falls back to a connection-without-peer-identity path (the
caller-id-is-the-connection case, e.g., anonymous dial-in).
The UUID fallback is removed. A connection with no resolved identity has no
PeerId, not a random one.
API keys
API keys (ApiKeyEntry) are not given the PeerEntry treatment. The
two identity sources have different semantics:
| Axis | Fingerprint (PeerEntry) | API key (ApiKeyEntry) |
|---|---|---|
| Identity source | TLS handshake / SSH key | Bearer token in protocol frame |
| Key rotation | Same logical node, new material | New identity (revocation = new key) |
Identity.id |
peer_id (stable across rotation) |
prefix (changes with the key) |
| Resource binding | PeerEntry.resources (per-peer) |
Empty (Option B, auth.md) — resources are composition-only |
An API key's prefix IS the identity — rotating the key means a new prefix and a new identity, by design (revocation is the rotation mechanism for API keys). Decoupling the API key identity from the prefix would be solving a different problem (persistent logical identity across key rotation) that API keys don't have: they're bearer tokens, not node identities.
ApiKeyEntry stays as-is. The asymmetry is documented here and in
auth.md so the difference between the two auth paths is explicit, not an
oversight.
What this does NOT change
Identitystruct shape —id: String,scopes: Vec<String>,resources: HashMap<String, Vec<String>>are unchanged. Only the meaning ofidon the fingerprint path changes (fingerprint → peer_id).IdentityProvidertrait — unchanged. The adapter's resolution semantics change, not the trait.AccessControl::check— unchanged. Still a flat scope/resource match againstIdentity. TheIdentityit checks now has a stableidon the fingerprint path, butcheckdoesn't key onid(it checks scopes and resources).AuthToken,AuthContext— unchanged.PeerRef::Specific(PeerId)signature — unchanged. The payload is now stable.CompositeOperationEnv→PeerCompositeEnvmigration — unchanged. This ADR provides the stablePeerIdsource; ADR-029 still owns the overlay-keying model.
Consequences
Positive:
- Key rotation no longer breaks ACL entries or routing references on the
remote side. The vault's local rotation story (ADR-021) is now the
complete story —
rotatelocally, edit the peer entry's fingerprint remotely, reload. PeerRef::Specificsurvives reconnects. An in-flight routing reference to "worker-a" keeps resolving after worker-a's TLS key rotates and after worker-a reconnects.- OQ-33's UUID workaround is removed — the stable logical id is the real thing, not an ephemeral stand-in.
- OQ-34's storage-boundary question is resolved: core has the config model
(
PeerEntry) + the in-memory adapter (ConfigIdentityProvider); a futurealknet-peer-store-sqliteadapter that persistsPeerEntryrecords is additive, implementing the sameIdentityProvidertrait against apeerstable. See ADR-033. - Fingerprint-resolved identities now carry
resources(the pre-ADR-030 limitation is lifted) —AccessControl::checkagainstresource_type/resource_actionworks for external fingerprint-authenticated callers when configured.
Negative:
AuthPolicy.authorized_fingerprints: HashSet<String>is replaced withAuthPolicy.peers: Vec<PeerEntry>. This is a breaking config change — existing config files withauthorized_fingerprintsmigrate topeersentries. The migration is mechanical (each fingerprint becomes aPeerEntry { peer_id: <chosen name>, fingerprint: <old value>, scopes: ["relay:connect"], ... }), and operators must choose apeer_idper peer, but it is a config break.Identity.idfor fingerprint-resolved identities changes from the fingerprint to thepeer_id. Code that logs or comparesIdentity.idon the fingerprint path and assumed it was the fingerprint string will see thepeer_idinstead. This is the correct behavior (logs should show the logical name, not the rotating crypto material), but it's a behavior change in log output.- The pre-ADR-030
auth.md"Resource-scoped ACLs and external identities" limitation note is removed — fingerprint-resolved identities now populateresources. Code that relied on fingerprint identities always having emptyresources(an unintended invariant) will see populated resources when configured. - ADR-029 Assumption 1 is superseded on the
PeerIdsource dimension: the one-way door (PeerIdis logical, not crypto) is preserved, but the v1 UUID source is replaced byIdentity.idfromPeerEntry. The Assumption's framing of "no-storage workaround" is no longer accurate — the storage boundary is now explicitlyconfig + in-memory adapter(this ADR + ADR-033), with the SQLite adapter additive.
Assumptions
-
The dispatch path can require identity resolution for peer-keyed overlay membership. A connection that fails
IdentityProviderresolution has noPeerIdand is not added toPeerCompositeEnv. The caller either authenticates with a recognized fingerprint (and gets apeer_id) or is rejected / falls back to a no-peer-identity path. The v1 UUID fallback is removed deliberately — anonymous dial-in to a peer-keyed composition env is a contradiction. -
PeerEntry.peer_idis operator-chosen and unique within a config. Config validation enforces uniqueness; duplicatepeer_idvalues inAuthPolicy.peersare a config error. -
API keys stay as-is. The
ApiKeyEntrymodel is correct for bearer- token identity where rotation = new identity. This ADR does not add aPeerEntry-equivalent for API keys. See "API keys" above. -
The
peerslist resolution is O(peers) per fingerprint lookup. The expected peer count per node is small (10s–100s); a linear scan with a side index is fine. AHashMap<fingerprint, &PeerEntry>index is an implementation-detail two-way door. -
Adapter crates that persist
PeerEntryrecords are additive and not specified here. ADR-033 establishes the pattern (core trait + in-memory default; persistence adapters are separate crates); the concrete adapter shapes are deferred for exploration per the user's note. This ADR's commitment is to thePeerEntryconfig model + the resolution semantics + thePeerIdsource, not to any specific backend.
References
- ADR-004: Auth as Shared Core (
IdentityProviderin core) - ADR-015: Privilege Model and Authority Context (
AccessControl::checkagainstIdentity) - ADR-021: Key Rotation via Version-Indexed Paths (the local rotation half this ADR completes across nodes)
- ADR-022: Handler Registration, Provenance, and Composition Authority
(the registration bundle's
composition_authoritypath produces its ownIdentity; this ADR'sPeerEntry.resourcespopulates the external-auth path'sIdentity.resources) - ADR-029: Peer-Graph Routing Model (the
PeerId = Identity.idmodel; Assumption 1's UUID source is superseded by this ADR'sPeerEntry.peer_idsource — the one-way door is preserved) - ADR-033: Storage Boundary and Repo/Adapter Pattern (the overarching pattern
this ADR's
PeerEntry+ConfigIdentityProviderfollows) - OQ-33: PeerId — Cryptographic Identity vs Stable Logical Identifier (resolved by this ADR — the "real solution" half, replacing the UUID workaround)
- OQ-34: Persistent Peer Registry (resolved by this ADR + ADR-033 — the
storage boundary is
config + in-memory adapternow, SQLite adapter additive) - OQ-35: API Key Identity vs Peer Identity (recorded by this ADR — the asymmetry is deliberate, see "API keys" above)
docs/research/alknet-storage-strategy/findings.md§4 (thePeerEntrymodel and resolution path)docs/architecture/crates/core/auth.md(the spec amended by this ADR)docs/architecture/crates/core/config.md(theAuthPolicychange)