docs(architecture): Phase 1 spec tree — 7 specs, ADRs 001-006, OQ tracker
- overview/hashing/store-api/backends/gc/ops specs + README index, all draft status with YAML frontmatter and cross-referenced ADR/OQ - ADR-001: substrate posture (store layer alkcall-free = family conformance, not deviation) + ops placement decided (feature-gated module, no consumer-waited deferral) - ADR-002: canonical git-blob-sha-256 + key encoding — the declared wire-format ADR (one-way door) - ADR-003: backend contract, sqlite/fs/mem tiers, pure-function dispatch, no migration, unknown-length buffering semantics pinned - ADR-004: exactly two production backends; manifest/path-tree layers are consumer-side (the corrected two-vs-three-backends framing) - ADR-005: pooled CAS, three liveness sources, delete windows with pin-before-publish + delete-time arbitration (race closed), no ambient scheduling - ADR-006: whole-blob verification; chunk-tree encodings excluded by scoping, not deferred on a paused consumer - open-questions.md: OQ-BL-01..06 resolved with ADR cross-refs; OQ-07 /08 externally-owned, OQ-09 deferred(scope) with SDD tracker task - AGENTS.md: status updated to Phase 1, gix-odb path corrected Verification: cargo test / clippy -D warnings / fmt --check clean
This commit is contained in:
glm-5.3-flash committed
1 parent
7d9ac8a6ec
commit
4b5009d85a
16 files changed
+1953
-8
No files matched your search
@@ -0,0 +1,167 @@
|
||||
---
|
||||
status: draft
|
||||
last_updated: 2026-10-01
|
||||
---
|
||||
|
||||
# Ops surface (alkcall-backed network operations)
|
||||
|
||||
## What this is
|
||||
|
||||
The network-facing operation family over the store: have/need
|
||||
announcements, verified fetch, and streaming put — riding the alkcall
|
||||
substrate behind a **feature-gated `ops` module in this crate**
|
||||
(feature `ops`, default-off; ADR-001). The base crate stays lean and
|
||||
alkcall-free; enabling `ops` adds the alkcall dependency and registers
|
||||
the op family.
|
||||
|
||||
## Why alkcall (ADR-001, the short form)
|
||||
|
||||
The op family is exactly alkcall's mixed-shape sweet spot (validated
|
||||
machinery, not parallel invention):
|
||||
|
||||
- **JSON control plane** — have/need announcements, `stat`-shaped
|
||||
probes, offer/carry lengths: alkcall `OperationSpec` + JSON Schema
|
||||
validation (alkcall ADR-016) + typed error schemas + `from_call`
|
||||
discovery + External/Internal visibility.
|
||||
- **Binary data plane** — bulk bytes one way (fetch: `Sub`,
|
||||
alkcall ADR-021) or the other (put: `Pub`/`Sink`, alkcall ADR-046),
|
||||
allocated via the `channel_open` marker (alkcall ADR-047);
|
||||
established pump patterns (`pump_bidi`, alkcall ADR-050).
|
||||
- **ACL is solved there** — `AccessControl` + ownership checks
|
||||
(alkcall ADR-011) under the ADR-017 privilege model map onto
|
||||
namespace gating directly; an in-band invented scheme would be a
|
||||
second, unreviewed authorization story in the family (AGENTS
|
||||
convention 6).
|
||||
|
||||
Producer/consumer vocabulary throughout; no transport enters either
|
||||
layer — the caller (an embedder, alkgit's replicator, a future alkfs
|
||||
sync) dials and hands an established `Connection` in (alkcall is a
|
||||
pure protocol crate; ADR-012 there).
|
||||
|
||||
## Op family (WHAT is exposed)
|
||||
|
||||
**Namespaces on the wire.** Every ops payload carries an explicit
|
||||
`namespace` field — the consumer-scoped identifier the embedder's
|
||||
registry recognizes (syntax: non-empty UTF-8 string, embedder-validated;
|
||||
the *minting* of namespaces is the embedder's act — e.g. alkgit's
|
||||
registry maps repo ids to namespaces, alkfs maps workspace roots). The
|
||||
store core never sees this string (backends are namespace-blind,
|
||||
ADR-005); it selects the liveness source and the ACL resource. The ops
|
||||
module's job is to bind `namespace → the embedder-registered liveness
|
||||
source` for pin conversion, and to present the namespace as the
|
||||
alkcall ACL resource.
|
||||
|
||||
JSON control ops (`Visibility` per ADR-001 §Decision):
|
||||
|
||||
- **`blobs/stat`** `{namespace, digest}` → `{len}` — probe before
|
||||
offering; ACL-gated like fetch (read action on the namespace)
|
||||
- **`blobs/have`** `{namespace, digests[]}` → `{present[]}` — the
|
||||
have half of have/need set diffing (hashes only — content never
|
||||
traverses this op); read-gated: an existence probe over arbitrary
|
||||
digests is a discovery surface, so it is gated exactly like fetch,
|
||||
not public
|
||||
- **`blobs/delete`** `{namespace, digests[]}` — operator machinery;
|
||||
`Visibility::Internal`, evaluated under the internal authority
|
||||
context per alkcall ADR-017 (internal calls switch authority context,
|
||||
never skip ACL): requires the embedder's operator/admin authority
|
||||
context, not a namespace grant. The `namespace` field exists for
|
||||
audit/triage scoping (which slice of the pool the deletion targets),
|
||||
not for gating.
|
||||
|
||||
Binary channel ops (registered via the `channel_open` marker):
|
||||
|
||||
- **`blobs/fetch`** (`Sub` — the server→client streaming op shape)
|
||||
— `{namespace, digest, ranges?}` in;
|
||||
verified bytes out: the consumer hash-checks each received chunk/whole against
|
||||
the carried digest (ADR-006). Broadcast fanout above the store:
|
||||
one reader, store arm + subscriber arms (POC #3 finding A3); late
|
||||
joiners degrade to a normal post-commit `get` — identical bytes under
|
||||
CAS. Slow-subscriber policy (drop-and-late-join) is ops-layer
|
||||
policy, not a store concern.
|
||||
- **`blobs/put`** (`Sink` — the client→server streaming op shape)
|
||||
— `{namespace, digest?, len?}` offer +
|
||||
byte stream in; server verifies against the canonical derivation
|
||||
before commit. A known-length offer is the encouraged path (one-
|
||||
pass); unknown-length rides the store's pre-threshold buffering path
|
||||
(ADR-003). **Need half of have/need**: a fetch miss *is* the need
|
||||
announcement — the consumer computes its need set by diffing
|
||||
`blobs/have` results and fetches the absent digests; no separate
|
||||
need op exists (the diff is caller-side; the wire carries only
|
||||
concrete fetch requests).
|
||||
|
||||
**Pin hand-over for remote puts (the cross-wire contract).** The
|
||||
server-side put handler owns the `Pin` guard; the offering side's
|
||||
reference registration happens in the *remote* process, invisible to
|
||||
the server. The hand-over contract:
|
||||
|
||||
1. the put lands pinned (ADR-005) — the entry cannot be swept while
|
||||
the handler holds the pin;
|
||||
2. the `blobs/put` response returns a **pin token** (opaque handle);
|
||||
the putter can later confirm its registration landed by observing
|
||||
the digest present via `blobs/have`;
|
||||
3. the embedder's registered liveness source is the *actual* root of
|
||||
record: the contract on the embedder (documented on the ops module)
|
||||
is that it registers the namespace's roots such that a digest
|
||||
referenced by a namespace's manifest is either (a) already
|
||||
registered before the caller's next sweep could run on the serving
|
||||
node, or (b) protected by holding the pin token until its
|
||||
registration is confirmed (`blobs/have` returns present after
|
||||
registration). In short: **the remote putter may not rely on an
|
||||
unconfirmed put surviving the next sweep — it holds its pin token
|
||||
until its registration shows up in `blobs/have`.** A `Pin`
|
||||
conversion on the server side (token → registered liveness source)
|
||||
is the mechanism the embedder plugs its registry into.
|
||||
|
||||
Placement of the op *registration* (which embedders wire where they
|
||||
want them exposed) matches the alkgit ops pattern (its
|
||||
`git/repo/*` call ops): the module exports spec/handler pairs; the
|
||||
embedder assembles.
|
||||
|
||||
## ACL mapping
|
||||
|
||||
Resources and actions, evaluated by alkcall's `AccessControl` machinery
|
||||
at op entry — this module never re-implements an authorization check
|
||||
(convention 6; ADR-001):
|
||||
|
||||
- **Namespace = resource.** `resource_type: "blob-namespace"`,
|
||||
`resource_id_path` selecting the payload's `namespace` field;
|
||||
actions map onto read/put/manage:
|
||||
- `blobs/fetch`, `blobs/stat`, `blobs/have` — the **read** action;
|
||||
unlisted namespaces (no ACL entries) deny by default (closed
|
||||
posture until OQ-09 decides a different default)
|
||||
- `blobs/put` — the **write** action
|
||||
- `blobs/delete` — not namespace-gated at all; internal authority
|
||||
context only (alkcall ADR-017: internal switches context, never
|
||||
skips ACL)
|
||||
|
||||
## Store-facing relationship
|
||||
|
||||
The ops module is a *consumer* of the store core (store-api.md), not a
|
||||
side channel: fetch handlers call `get`/`read_range`/`stat`; put
|
||||
handlers call `put`; both honor pins and delete windows. There is no
|
||||
ops-internal path around the store facade (ADR-001 §Consequences —
|
||||
this is what makes "re-homing the ops surface later loses nothing"
|
||||
true).
|
||||
|
||||
## Design Decisions
|
||||
|
||||
| ADR | Decision | Summary |
|
||||
|---|---|---|
|
||||
| [001](decisions/001-substrate-posture-and-ops-placement.md) | Substrate posture & placement | alkcall-backed, feature-gated ops module; conformance-not-deviation rationale |
|
||||
| [006](decisions/006-verification-posture-and-transfer-encoding.md) | Verification | verified-fetch via carried digests; no chunk trees |
|
||||
|
||||
## Open Questions
|
||||
|
||||
- **OQ-09**: default visibility of namespaces in network ops
|
||||
(open-by-default vs closed-by-default) — deferred(scope), decided at
|
||||
the first embedded ops deployment; ships closed-by-default meanwhile.
|
||||
|
||||
## References
|
||||
|
||||
- `docs/research/phase-0.md` OQ-BL-01 (the substrate settlement + its
|
||||
justification, promoted verbatim into ADR-001 §Context)
|
||||
- alkcall docs/architecture — ADR-011/012/016/017/021/046/047/050 (the
|
||||
machinery this module rides)
|
||||
- `docs/research/poc-largeblob-findings.md` finding A3 (fanout seam)
|
||||
- ADR-001; [store-api.md](store-api.md); [gc-and-namespaces.md](gc-and-namespaces.md)
|
||||
(namespace-as-resource)
|
||||
Reference in new issue
Block a user