alkdev/alknet
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: refactor hub/spoke to head/worker, add service layer and HD key derivation

Browse Source 
- Replace hub/spoke terminology with head/worker throughout all research docs
- Add irpc service layer architecture (AuthProtocol, SecretProtocol,
  ConfigProtocol, StorageProtocol)
- Add BIP39/SLIP-0010 HD key derivation for secrets management
- Add event boundary discipline (domain events vs integration events)
- Add application services layer (Docker, Node, Wallet, Proxy, Compute)
- New docs/research/services.md defining irpc service protocols
- Update core.md with service layer section and head/worker model
- Update configuration.md to delegate auth to AuthService (irpc)
- Update storage.md with secrets/key derivation and event boundaries
- Update flow.md with event boundary decision and cross-references
This commit is contained in:
glm-5.1
2026-06-06 15:33:35 +00:00
parent 2315a211ff
commit d291a485f0
5 changed files with 1007 additions and 49 deletions
Download Patch File Download Diff File Expand all files Collapse all files

124
docs/research/storage.md
View File

@@ -1,11 +1,18 @@
# Alknet Storage: Metagraph, Identity, ACL, and Honker Integration
# Alknet Storage: Metagraph, Identity, ACL, Secrets, and Honker Integration
> Status: Research / Draft
> Last updated: 2026-06-05
> Last updated: 2026-06-06
## Overview
`alknet-storage` is a Rust crate providing SQLite-backed graph storage, identity management, access control, and reactivity via honker. It mirrors the TypeScript `@alkdev/storage` package's design (`sqlite-host.md`, `metagraph-module.md`, `acl.md`) while leveraging Rust's type system and petgraph's performance.
`alknet-storage` is a Rust crate providing SQLite-backed graph storage, identity management, access control, secrets management, and reactivity via honker. It mirrors the TypeScript `@alkdev/storage` package's design (`sqlite-host.md`, `metagraph-module.md`, `acl.md`) while leveraging Rust's type system and petgraph's performance.
## Terminology
This document uses **head/worker** terminology instead of hub/spoke:
- **Head node**: Coordinating node that can also be a worker
- **Worker node**: Node that connects to a head and registers services
- **Node**: Any participant in the network
## Crate Decomposition
@@ -14,6 +21,7 @@ alknet-storage
├── metagraph/ — GraphType, NodeType, EdgeType definitions and persistence
├── identity/ — accounts, organizations, peer_credentials, api_keys, audit_logs
├── acl/ — PrincipalNode, DelegatesEdge, access control graph
├── secrets/ — HD key derivation (BIP39/SLIP-0010), encrypted data, secret service bridge
├── honker/ — honker integration: notify, stream, queue, event bridge
├── graph/ — GraphInstance, Node, Edge CRUD with schema validation
└── schema/ — JSON Schema definitions (serde + jsonschema for runtime validation)
@@ -199,7 +207,7 @@ pub struct DelegatesEdgeAttrs {
- **Account** nodes represent individual users
- **Org** nodes represent organizations
- **Service** nodes represent automated agents (LLM workers, spoke credentials)
- **Service** nodes represent automated agents (LLM workers, node credentials)
- **Role** nodes represent named permission sets
Delegation edges (`delegates`) carry `narrowed_scopes` — the delegate can only exercise scopes that are a subset of the delegator's. Liability flows upward; permissions flow downward with narrowing.
@@ -318,6 +326,103 @@ For the distributed use case (later):
Replication mindset from the start: **every write is atomic with a notification**. The honker stream event is the replication unit. A future replicator reads `_honker_stream_*` tables and propagates changes to subscribed relays.
### Event Boundary Discipline
Following [event_source_types.md](/workspace/research/event_sourcing/event_source_types.md), honker streams serve different roles in different contexts. Preventing conflation is critical:
| Event Type | Source | Consumer | Boundary |
|-----------|--------|----------|----------|
| **Domain events** (Event Sourcing) | Service that owns the data | Same service, for state reconstruction | Internal — never published directly to other services |
| **Integration events** (State Transfer) | Projected from domain events | Other services/nodes, for cache updates | Cross-service — simple, versioned, stripped of internals |
| **Notifications** (Thin Events) | Service that owns the data | Any subscriber, for triggering workflows | Cross-node — just entity ID + action, consumer fetches details |
Conflation anti-patterns to avoid:
- **Leaky event store**: Don't let other services read honker stream events directly to drive business logic. Project domain events into integration events first.
- **Boomerang coupling**: If a consumer of an integration event must call back to the source service synchronously, the event payload is too thin. Upgrade to a fat event.
- **Fat notification trap**: If a notification event carries the full entity state, use state transfer instead.
The call protocol's `EventEnvelope` is the **integration boundary** between nodes. Domain events in honker streams stay within the service that owns them.
## Secrets and HD Key Derivation
### Key Categories
Different categories of secrets require different storage and derivation strategies:
| Category | Example | Derived from seed? | Storage |
|-----------|---------|-------------------|---------|
| **Identity keys** | Ed25519 keypair for alknet auth | Yes — SLIP-0010 `m/74'/0'/0'/0'` | Only derivation path in DB |
| **Encryption keys** | AES-256-GCM key for encrypted nodes | Yes — SLIP-0010 `m/74'/2'/0'/0'` | Only derivation path in DB |
| **External credentials** | OpenAI API key, OAuth token | No — third-party issued | Encrypted in DB with derived key |
| **On-chain identity** | Ethereum key for contract signing | Yes — SLIP-0010 `m/44'/60'/0'/0/0` | Only derivation path in DB |
| **Service registration** | NFT token ID, replicator endpoint | No — on-chain data | Plain in DB or on-chain |
### BIP39 Seed Phrase as Root of Trust
The master seed phrase (BIP39 mnemonic) is the single recovery mechanism for the entire system. From one seed phrase, all self-generated secrets can be derived on demand:
```rust
// Seed phrase → master seed (BIP39)
let mnemonic = Mnemonic::from_phrase(&phrase, Language::English)?;
let seed = mnemonic.to_seed(Some(&passphrase));
// Master seed → SLIP-0010 Ed25519 master key
let master_key = ExtendedPrivKey::new_master(Network::Alknet, &seed)?;
// Derive identity keypair
let identity_key = master_key.derive_path("m/74'/0'/0'/0'")?;
// Derive encryption key material (use first 32 bytes of derived key as AES-256 key)
let encryption_key = master_key.derive_path("m/74'/2'/0'/0'")?;
// Derive Ethereum signing key (for smart contract interactions)
let eth_key = master_key.derive_path("m/44'/60'/0'/0/0")?;
```
### External Credentials: Encryption with Derived Keys
For external credentials (API keys, OAuth tokens) that can't be derived, the existing `EncryptedDataSchema` pattern from `@alkdev/storage` applies — but the encryption key is itself derived from the seed:
1. The secret service derives an AES-256-GCM key via SLIP-0010 path `m/74'/2'/0'/0'`
2. External credentials are encrypted with this derived key using the existing encrypt/decrypt functions
3. The encrypted data is stored as a `SecretNode` in the metagraph
4. Only the derivation path and key version are stored in plain attributes
5. The seed phrase (or derived encryption key) is held only by the secret service — never in the database
### Secret Service
The secret service is an irpc service (see [services.md](services.md)) that:
- Holds the master seed phrase in memory (never persisted to disk in plain text)
- Derives keys on demand via SLIP-0010/BIP39
- Encrypts/decrypts external credentials using derived keys
- Is the **only** component that ever sees the master seed
Workers request derived keys through the secret service's irpc protocol. They never see the seed or the encryption key.
### Derivation Path Conventions
| Path | Purpose |
|------|---------|
| `m/74'/0'/0'/0'` | Primary Ed25519 identity keypair (alknet auth) |
| `m/74'/0'/0'/1'` | Secondary identity keypair (device key) |
| `m/74'/0'/1'/0'` | SSH host key (for server identity) |
| `m/74'/1'/0'/{site_hash}'` | Site-specific password derivation |
| `m/74'/2'/0'/0'` | AES-256-GCM encryption key (for external credentials) |
| `m/44'/60'/0'/0/0` | Ethereum signing key (for smart contract interactions) |
The `74'` coin type is unallocated per SLIP-0044 and can be registered for alknet. The `0'`/`1'`/`2'` account levels divide identity, password, and encryption purposes.
### Rust Crates Required
| Crate | Purpose |
|-------|---------|
| `bip39` | Mnemonic generation and seed derivation |
| `ed25519-bip32` (IOHK) or `rust-bip32-ed25519` (BitBoxSwiss) | SLIP-0010 Ed25519 HD key derivation |
| `aes-gcm` | AES-256-GCM encryption for external credentials |
| `sha2` | SHA-256 for key hashing |
| `irpc` | Service protocol definitions |
## Design Decisions (mapped from TypeScript ADRs)
| Original ADR | Decision | Rust adaptation |
@@ -335,14 +440,21 @@ Replication mindset from the start: **every write is atomic with a notification*
| 047 | Honker event target | honker stream/notify as pub/sub mechanism |
| 049 | Identity schema restructuring | Separate credential tables, no Gitea columns |
| 050 | SHA-256 for API key hashing | Fast hash for high-entropy machine keys |
| 051 | BIP39/SLIP-0010 for HD key derivation | Seed phrase as root of trust for identity, encryption, and signing keys |
| 052 | Secrets as irpc service | Secret service holds seed, derives keys, encrypts/decrypts external creds |
| 053 | Event boundary discipline | Honker streams are domain events; call protocol is integration boundary |
## References
- `@alkdev/storage` — TypeScript metagraph, identity, ACL implementation
- `@alkdev/storage` — TypeScript metagraph, identity, ACL, encrypted data implementation
- `@alkdev/flowgraph` — TypeScript call-graph and operation-graph (maps to petgraph in Rust)
- `@alkdev/operations` — TypeScript OperationSpec, CallHandler, registry
- `/workspace/honker` — SQLite extension with pub/sub, streams, queues
- `/workspace/polyglot` — SQL transpiler (future: schema migration validation)
- `/workspace/petgraph` — Graph data structure library (used in alknet-flowgraph)
- `/workspace/jsonschema` — JSON Schema validation (Rust, replaces TypeBox at runtime)
- `/workspace/iroh/iroh-dns` — DNS resolver and endpoint info
- `/workspace/iroh/iroh-dns` — DNS resolver and endpoint info
- `/workspace/@alkdev/storage/docs/architecture/encrypted-data.md` — Original encrypted data design (TypeScript)
- `/workspace/research/event_sourcing/event_source_types.md` — Event-driven architecture patterns
- [services.md](services.md) — Service layer architecture (irpc protocols)
- [core.md](core.md) — Core overview, head/worker terminology