75 lines
3.4 KiB
Markdown
75 lines
3.4 KiB
Markdown
---
|
|
id: review-002-oai11-ref-memoization
|
|
name: Memoize $ref expansion to bound resolver work (OAI-11)
|
|
status: completed
|
|
depends_on: []
|
|
scope: moderate
|
|
risk: medium
|
|
impact: component
|
|
level: implementation
|
|
tags: [adapters, review-002, from-openapi, openapi-spec]
|
|
---
|
|
|
|
## Description
|
|
|
|
Review 002 OAI-11 [major, borderline critical]. The OAI-01 fix
|
|
(cycle guard + depth budget, `openapi_spec.rs:382-437`) bounds *stack*,
|
|
not *work*: every `$ref` hop re-expands its target from a fresh clone
|
|
with no memoization. A chain of shared but acyclic refs —
|
|
|
|
```
|
|
S_i = { a: { $ref: "S_{i+1}" }, b: { $ref: "S_{i+1}" } }
|
|
```
|
|
|
|
(the doubly-linked-list `prev`/`next` shape — valid, common) expands as
|
|
a full binary tree. Empirically (verbatim harness): 20 levels → 7.3M
|
|
node visits / 3.6 s; growth ≈ 2^levels → 30 levels ≈ ~1 hour, 40 ≈ days.
|
|
Neither guard fires (`err=false`); the import just wedges — no stack
|
|
overflow, no error. The same "valid document kills the process" class
|
|
as OAI-01, one mitigation shy.
|
|
|
|
## Acceptance Criteria
|
|
|
|
- [ ] `resolve_refs_bounded` memoizes resolved, cycle-free `$ref`
|
|
targets per document (cache keyed on the ref string; clone the
|
|
resolved `Value` on hit instead of re-resolving) — or an
|
|
equivalent total-node-count budget with a clean
|
|
`SchemaParse`-family error (implementer's choice; memoization
|
|
strongly preferred — it preserves full acyclic support)
|
|
- [ ] The cycle guard and both depth checks remain (memoization must
|
|
not alter cycle detection semantics — a cycle still errors)
|
|
- [ ] Acceptance gate: a 30+ level shared-chain spec imports in
|
|
bounded time (add the test with a wall-clock or node-count
|
|
assert; a 30-level chain must import in < 1 s)
|
|
- [ ] Tests: diamond (non-cyclic shared refs) unchanged-correct;
|
|
cycle-through-shared-node still errors; memoized result equals
|
|
the old expansion for a shared-ref schema (golden compare on one
|
|
case)
|
|
- [ ] The four missing cycle-shape tests from OAI-16 (array-items,
|
|
allOf, additionalProperties, `$ref`-sibling) land here too —
|
|
same function, same risk envelope
|
|
- [ ] `cargo test`, `cargo clippy --all-targets -- -D warnings`,
|
|
`cargo fmt --check` pass
|
|
|
|
## References
|
|
|
|
- docs/reviews/002-post-remediation-review.md (Part E', OAI-11 + OAI-16)
|
|
- src/adapters/openapi_spec.rs:382-437 (the resolver), :713-805 (existing cycle tests), from_openapi.rs:736 (the requestBody-chain test)
|
|
- tasks/adapters/review-001-ref-cycle-guard.md (the OAI-01 fix this completes)
|
|
- docs/architecture/decisions/066-from-jsonschema-as-http-adapter.md
|
|
|
|
## Notes
|
|
|
|
Clone cost per hit is fine (schemas are small; the win is avoiding
|
|
exponential re-resolution, not allocation). If memoizing, note the
|
|
`resolving`-set interaction: a memo entry must only be written for
|
|
*completed* (cycle-free) expansions — a partial expansion under an
|
|
active cycle attempt must never be cached (write-on-success at the
|
|
top of the recursion return path, or post-order). This is the one
|
|
subtle part; the existing cycle tests + the four new shape tests are
|
|
the safety net.
|
|
|
|
## Summary
|
|
|
|
Memoized $ref expansion (write-on-success cache) + node-budget accounting on memo hits + separate MAX_REF_HOP_DEPTH (64) from structural depth (128). Exponential diamond chains now fail with a clean budget error (<1s) instead of OOM. Note: the original WIP's unbounded memo-hit clone OOM-killed the dev server twice during implementation; verified under a 2G systemd scope.
|