alkblobs/AGENTS.md

AGENTS.md

Operating instructions for opencode agents working in this repo. opencode auto-loads this file as instructions, overriding the built-in defaults for this project. Custom agents in .opencode/agents/ inherit these rules unless their own prompts say otherwise.

Git Workflow

Commit and push when reasonable. When a change is complete and verified (build + lint + tests pass), commit and push to origin/main without asking. This overrides the built-in default of "only commit when explicitly asked."

The workflow:

1. Make the change
2. Verify: cargo test, cargo clippy --all-targets -- -D warnings, cargo fmt --check, cargo doc --no-deps if docs changed
3. Inspect git status and git diff before staging — stage only the intended files, never secrets
4. Write a concise commit message matching the repo style (see git log --oneline -10). For multi-point changes, use a summary line plus a body with bullet points and a verification block.
5. git push origin main
6. Report the commit hash and the verification summary

Exceptions — do not commit or push without asking:

• The change is exploratory / speculative (you're not sure the user wants it kept)
• The user is actively reviewing the diff and may ask for changes
• The change touches the wire format, a trait shape backends implement, or the hashing/verification story (one-way doors once consumers exist — see convention 7; nothing is wire-stable yet, but the first wire-format ADR must be written before the first consumer)
• You'd be force-pushing, amending a published commit, creating an empty commit, or skipping hooks

Never commit secrets, keys, or credentials. If a commit fails or hooks reject it, fix the issue and create a new commit — do not amend the failed one.

Git identity is preconfigured (glm-5.3-flash <glm-5.3-flash@alk.dev>). Do not change git config, skip hooks, or use git commit -i.

Project Conventions (Rust / blob storage crate)

This is the blob storage crate — content-addressed blob storage in the alk* family, inspired by iroh-blobs' store work but deliberately not a fork of it. It sits downstream of alkcall (the call + channels substrate its network operations ride, when they exist) and its first planned consumer is alkgit (git object storage, where the iroh-blobs hashing story collides with git's own object hashes). Status: Phase 1 complete — publish-ready (the publish hold lifted at the wave-13 gate; publication runs manually, coordinator-side) — Phase 0 converged (see docs/research/phase-0.md); docs/architecture/ records the design (ADRs 001-020 accepted, open-questions tracker); the tracker's register is resolved — OQ-07/08 remain externally-owned; implementation is complete through wave 13 — the core, all five engines, fleet GC, ops, review-001's three remediation waves, fuzzing, release readiness, and review-002's round (the git-object derivation family, the token admission posture, the sqlite contention posture, the borrowed-key overloads), each landed at its review gate (docs/plans/implementation.md). Open as of 2026-10-18: review-003 (docs/reviews/003-family-wire-gap-review.md) recorded the wave-13 ops-surface scoping (ADR-018 §Decision 2, the blob-only wire posture) as a coherence defect, and ADR-020 (docs/architecture/decisions/020-family-explicit-wire-payloads.md) is accepted as the correction — family-explicit payloads under the closed op set (the wire speaks tag-prefixed key hex; bare hex dead; the put offer's key + len mandatory; the put response echoes the key) and one derivation-explicit put seam (put_as; put, put_git_object, from_git_oid_hex, and the Digest::derive/Key::derive silent forms are all deleted — do not extend or rely on any superseded shape). Wave 14 (the ADR-020 implementation round) completed 2026-10-19 at its review gate — the landed surface verified against ADR-020 §Decision 1's invariant; the code no longer carries the wave-13 shapes, and no doc advertises them as current surface. The conventions below apply to all work in src/ and tests/. They mirror .opencode/agents/implementation-specialist.md §Project Conventions and are repeated here so they apply to every session.

1. No comments in code unless the user explicitly asks. This is a project-wide convention. Doc comments (///, //!) are fine and expected on public API. Inline // comments only when the user asks or when a non-obvious safety/correctness constraint would otherwise be missed.

2. Error handling — thiserror for library error types. No panics in library code. No unwrap() or expect() outside tests. If you reach for unwrap, the error path wasn't specified — stop and decide what should actually happen. For poisoned RwLock/Mutex, use unwrap_or_else(|e| e.into_inner()) so a panic in one operation does not cascade to other operations.

3. tokio is the async runtime — all I/O is async. Use tokio::sync primitives (oneshot, mpsc) for lifecycle correlation; parking_lot for short-held internal locks. Do not introduce blocking I/O on the async path.

4. Substrate-agnostic by construction — the store layer must not know whether bytes arrive over the network, from a local writer, or get reassembled anywhere particular. iroh-blobs welds its store to its provider/ticket/postcard stack; this crate's defining posture is the opposite split: the store (put/get/verify against a hash) is its own layer, and transport/protocol concerns live above it or in feature-gated modules. This is the same inversion-point pattern as the alk* family (alktty TtyBackend, alktunnels pump halves).

5. Canonical hash is the git-family derivation — the store keys entries under git's oid derivation ("blob <len>\0" + content), SHA-256 canonically, SHA-1 tolerated (existing repos; git's own hardened-SHA-1 threat model inherited). The hash-conflict framing dissolved 2026-10-01: BLAKE3's presence was an iroh-blobs inheritance, not a requirement — it is demoted to a conditional large-blob encoding consideration (docs/research/phase-0.md OQ-BL-03). Do not hardcode beyond the small enum abstraction: the key encoding is a one-way door.

6. Auth-gated operations ride the alkcall authorization seam — when network-facing operations exist, authorization happens via alkcall's AccessControl/identity mechanisms (the producer/consumer model), not via an in-band invented auth scheme. Producer/consumer vocabulary, not "server"/"client" (alkcall ADR-022/037). Not yet pinned in detail — the ops surface is a Phase 0 question.

7. Do not adopt iroh-blobs' wire surface — tickets, the postcard serialization, and the provider protocol are iroh-blobs' design-welded choices we have decided not to inherit. Reading /workspace/iroh-blobs (a fresh upstream checkout) is encouraged for store-shape lessons (src/store, the kv + flat file backends); borrowing its conclusions is fine, wiring in its types, framing, or serialization as load-bearing dependencies is not (yet — the dependency posture is a Phase 0 question if a small pure piece, like bao outboard encoding, earns its place).

8. Feature flags — substrate backends (kv, sqlite, flat fs) may be feature-gated if the need arises. The base crate should compile lean. Verify both cargo test (default) and cargo test --all-features pass if features are added.

9. Naming — Rust standard: snake_case for functions/variables/ modules, PascalCase for types/traits, SCREAMING_SNAKE_CASE for constants.

10. Module structure — one module per file under src/, re-exported from src/lib.rs. Public API surface is lib.rs re-exports. The expected shape (pending Phase 0 convergence) separates the backend/store layer from any protocol/ops layer.

11. Prose typography — the three gitea-flagged confusable characters (U+2013 en-dash, U+00D7 multiply sign, U+2228 logical or) stay out of repo prose: ASCII hyphen, x, and "or" instead. The rest of the house typography survives — em-dash, §, arrows, ellipsis, ≤/≥/≈/≠, box-drawing, ±, µ, ·.

12. Parameterize over the registry, never enumerate it — the Algorithm registry is the abstraction at every seam: facade methods, wire payloads, ops ingress. A method or op whose name contains a registry member (put_git_object, fetch_git_object-shaped) is a defect, not a pattern; the derivation-parameterized shapes (put_known_as, algorithm-naming payload fields under unchanged op names) are the form. Length-based family inference is the same violation and stays rejected (convention 5).

Verification Commands

Run these before committing. All must pass. The block is feature-aware (the default feature set includes mem, per ADR-013 §4's amendment — what each shape proves is recorded here, not left to session discipline).

cargo test                                    # the default build: `mem` + `sqlite` — runs the conformance suite's contract-reference column (mem) and the local column; the reference rows anchor the plain run
cargo test --all-features                     # adds the pg columns; server-less, they SKIP cleanly — a green run here is not their acceptance
cargo clippy --all-targets -- -D warnings
cargo clippy --all-features --all-targets -- -D warnings
cargo test --manifest-path fuzz/shared/Cargo.toml   # the fuzz corpus-replay gate (stable) — campaigns run only via fuzz/run-detached.sh
cargo fmt --check
cargo doc --no-deps                           # if docs changed
cargo publish --dry-run --allow-dirty         # before a release

The pg columns' acceptance shape: the postgres/pg-lo/fleet conformance and unit tests are env-driven (ALKBLOBS_PG_HOST gates; ALKBLOBS_PG_PORT/USER/PASSWORD/DB default to the harness server's values) and skip cleanly without one. Acceptance for those shapes is the same cargo test --all-features run against the live dockerized harness (pglo-poc: postgres:16-alpine on :15432, postgres/poc, db blobs), where the columns run for real.

The harness capacity posture (recorded 2026-10-15 with the test-isolation-harness task): the container runs postgres -c max_connections=200 (the stock image's 100 could saturate under a default-parallelism full-suite run — the saturation math: ~8 parallel pg-heavy tests x up to 2 pools each x pool max_size 8, worst case ~128 sessions + per-test raw connections, over the stock ceiling). Every pooled pg session carries the engine's application_name tag (PgOpts::application_name, default alkblobs); tests pass a unique tag per instance (their fresh schema/database name), so any server-global observability probe (pg_stat_activity-shaped) filters by tag and one test's zero-asserts cannot count another test's transient sessions.

Run-discipline: the full suite's acceptance posture is the default test parallelism against the live harness — no --test-threads reduction needed for a green run. The coverage-instrumented run may still take --test-threads=2 when it flakes an unrelated wall-clock cadence test (recorded on the f2-cancellation-guards task); the test bodies themselves are load-tolerant (cadence arms poll within bounded windows rather than sampling once at a fixed wall time), so a loaded-but-unsaturated server is not a failure cause. Keep --test-threads reduction as the coverage-run workaround, not the plain-run acceptance.

Coverage runs in two configurations (cargo llvm-cov): the server-less --all-features number misleads — the pg modules read single-digit coverage because their tests skip, dragging the total (~57%). The live-harness configuration (ALKBLOBS_PG_* set) is the honest number (~96%); record that one.

Fuzzing (adopted 2026-10-16 — wave 11; fuzz/ is a sidecar workspace excluded from the root workspace, pinning nightly inside fuzz/ only so the root's stable toolchain is untouched). Two targets over the parse surfaces: key_encoding (Key::from_bytes / Key::as_bytes / the Algorithm registry incl. the explicit Algorithm::derive — the one-way door's reject-never-guess discipline; ADR-020 deleted the length-routed/silent constructor forms) and ops_ingress (fetch_ranges_ingress, PinToken::from_hex). The fuzz binaries are nightly-only; the stable fuzz/shared crate replays the committed corpora as a plain cargo test gate (the block above — quinn's CI pattern). Campaigns run only via the detached runner (fuzz/run-detached.sh — setsid+nohup, never a foreground child of an agent session; an OOM in a target must not take down the agent host). Corpus policy: committed hand-made seeds only (fuzz/corpus/<target>/seed-*, regenerate with python3 fuzz/gen_fuzz_seeds.py — a no-op diff on the committed set) plus promoted campaign crash inputs (seed-900+ numbering); grown corpora, artifacts, and coverage output are gitignored and are not merged back into the committed seeds. See fuzz/README.md.

Architecture Context

• docs/research/ — the Phase 0 (Exploration) state of this repo: research findings, POC records, and phase-0.md (vision, prior art, open questions, converged recommendation). Read it before non-trivial work. The SDD process lives in docs/sdd_process.md.
• docs/architecture/ — exists (release-facing specs stable, ADRs 001-019 accepted, open-questions tracker active). The SDD process applies: specs describe WHAT, decisions/ ADRs explain WHY, open-questions.md tracks what's unresolved.
• Key prior art (read-only, in the global workspace):
  • iroh-blobs — /workspace/iroh-blobs (fresh upstream checkout): the shape inspiration, specifically its store work (kv + flat file backends, bao verification, chunking). Its wire-surface choices (tickets, postcard, BLAKE3-only) are the ones we are deliberately diverging from.
  • gix-odb — /workspace/gitoxide/gix-odb: git's own object database; the backend alkgit currently plans against and the hashing-algorithm baseline git actually uses.
  • alknet's blobs research — /workspace/@alkdev/alknet/docs/research/alknet-filesystem/ (alknet-blobs-external-store-probe.md, poc-summary.md): the appfile external-store probe — small blobs in a kv/sqlite-ish store, large blobs on the filesystem fallback, filename↔hash mapping. Written against an older iroh-blobs than the current checkout; re-verify conclusions before relying on them.
  • alkcall — /workspace/@alkdev/alkcall (the substrate): call + channels; its AccessControl/identity seam is the authorization story for network-facing blob ops.
• If a TODO references a design direction that a later ADR has decided against, the TODO is stale — remove it and align with the ADR. Do not implement the rejected design.