docs(research): POC #7 — postgres Large Objects as the fs-tier pg-lo engine, passed
The ADR-008 pg-lo admission POC ran in a standalone crate (/workspace/alkblobs-pglo-poc): PgLoBackend over the ADR-003/008 trait contract (including size), 10/10 exact-count sweep-outcome contract tests, clippy/fmt clean; dockerized postgres:16-alpine on :15432, POC #5 driver stack (tokio-postgres + deadpool) via SQL lo_* functions, no new dependency. Gate verdict: passed, with named deltas. - Performance: durable put 60-65 MB/s at >=1 MiB, within 1.5x of — and below 1 MiB beating — durable local fs on this fsync-slow disk; cached gets 70-180 MB/s single-stream, ~0.7 GB/s aggregate over 16 readers (20-50x behind page-cache fs — the honest named delta) - Contract: companion table is the list()/size()/CAS authority (never the catalogs); stage-then-commit; GC-participating lo_unlink delete - Handles: the tx-scoped descriptor is real but pool-compatible via descriptorless lo_get(oid, off, len) windows — window gets keep handle-acquire p99 at 1-6 ms under readers <= pool; held descriptor is the fallback posture - Vacuum: pg_largeobject pages churn-reused, never returned; tracked by autovacuum; rel-size monitoring named as an ops requirement - Crash/orphan: LO creation is transactional — kill/terminate mid-write-tx leaves zero orphan pages; the only orphan class is a committed LO bypassing the companion table (planted, reaped by the ~7 ms/oid sweep; committed content survives byte-exact) - Harness lessons: lo_lseek is int4 — the 64 variants are the >2 GiB discipline; shared-table parallel tests are unsound (per-test CREATE DATABASE isolation) Docs: new poc-pglo-findings.md; poc-pglo-spec.md status passed; register OQ-BL-06 #7 marked passed; ADR-008 pg-lo bullet updated (duplicate bullet removed) + backends-and-dispatch/open-questions cross-references. Verification: cargo test --release (10 passed), clippy -D warnings, fmt --check in /workspace/alkblobs-pglo-poc.
This commit is contained in:
1 parent
281c37876e
commit
cee97defba
6 files changed
+329
-19
No files matched your search
@@ -171,7 +171,14 @@ valid:
|
||||
pooling, `pg_largeobject`/vacuum posture under churn, crash-orphan
|
||||
behavior) and its own engine ADR. **Not shipped speculatively;**
|
||||
REQ-2's immediate fleet answers are shared media or re-routing;
|
||||
pg-lo is the consolidation option if those are unacceptable.
|
||||
pg-lo is the consolidation option if those are unacceptable. *Update
|
||||
2026-10-03: POC #7 ran and passed its gate —
|
||||
`docs/research/poc-pglo-findings.md` (companion-table authority,
|
||||
`lo_get` window gets, durable put ≈ fs's durable put, cached gets
|
||||
behind page-cache fs by a measured 20-50x aggregate-multiplied to
|
||||
~0.7 GB/s, clean crash-orphan behavior, bounded vacuum posture).
|
||||
The engine ADR's remaining evidence path is open; admission remains
|
||||
gated on it, not on speculation.*
|
||||
- **The no-mixed-fs-tiers rule is a deployment invariant, enforced at
|
||||
the enforceable seam.** Cross-node configuration cannot be validated
|
||||
by any one constructor (it sees only its own node). What the
|
||||
@@ -184,15 +191,6 @@ valid:
|
||||
verifies, not something a constructor can prove; a partitioned
|
||||
tier's observable signature (get fall-through misses for entries
|
||||
another node wrote) is the documented detection symptom.
|
||||
- **`pg-lo` (candidate engine, not shipped):** postgres Large Objects
|
||||
as the fs tier's storage — the same "engine behind one contract"
|
||||
move as ADR-007, one tier over. Named here so the door is explicit,
|
||||
with its admission gated on measured evidence (POC #7: LO write/read
|
||||
curves at the packfile regime, tx-scoped handle cost under
|
||||
pooling, `pg_largeobject`/vacuum posture under churn, crash-orphan
|
||||
behavior) and its own engine ADR. **Not shipped speculatively;**
|
||||
REQ-2's immediate fleet answers are shared media or re-routing;
|
||||
pg-lo is the consolidation option if those are unacceptable.
|
||||
|
||||
In sum: a fleet's fs tier over pool content is either shared media (all
|
||||
nodes' `local` roots on the same fleet-shared media) or routed (one
|
||||
@@ -234,7 +232,8 @@ partitioning failure this decision exists to prevent.
|
||||
- `pg-lo`, if admitted, rides the same driver stack ADR-007 already
|
||||
ships (tokio-postgres + deadpool) via SQL `lo_*` functions — no
|
||||
new driver dependency (the `postgres_large_object` crate is a dead
|
||||
0.15-era io-trait glue; unnecessary).
|
||||
0.15-era io-trait glue; unnecessary; POC #7 confirms: ~400 lines of
|
||||
SQL-statement shapes over the existing stack).
|
||||
|
||||
## References
|
||||
|
||||
|
||||
Reference in new issue
Block a user