docs: resolve 4 open questions, add research, spec codebook package structure
Research-driven resolution of OQ-01, OQ-02, OQ-05, OQ-06: - OQ-01: Remove ONNX Runtime from scope entirely — doesn't support activation extraction natively (optimum #972 closed as not planned), bloated model exports; burn/cublas via safetensors is a better future path - OQ-02: Codebook compresses ~65% (1,245 → 500-600 lines); add Package Structure and Extraction from PoC sections to codebook.md based on PoC analysis of metaspline firewall_codebook.py - OQ-05: Standalone API + thin adapter pattern (ADR-011); Phase 1 ships Firewall.screen() only, Phase 2 adds <100-line adapter packages for LlamaFirewall, OpenAI Agents SDK, NeMo Guardrails - OQ-06: TOML for file-based config — standard modern Python, two-way door Also: research OQ-03 rolling windows from taskgraph-semantic reference code, remove onnxruntime/optimum from dependencies, move streaming screening to Phase 2, add burn/cublas as Phase 3 alternative backend.
This commit is contained in:
@@ -151,6 +151,71 @@ model. The bundled codebook is specific to the default detector model
|
||||
(SmolLM2-135M at the pinned revision). Users who switch to a different
|
||||
detector model must provide a matching codebook via `codebook_path`.
|
||||
|
||||
## Package Structure
|
||||
|
||||
Based on analysis of the PoC codebook
|
||||
([poc-architecture.md](../research/codebook-analysis/poc-architecture.md)),
|
||||
the production codebook decomposes into:
|
||||
|
||||
```
|
||||
src/alknet_firewall/
|
||||
├── codebook/
|
||||
│ ├── __init__.py # Public exports
|
||||
│ ├── codebook.py # Codebook class (init, load, project, score)
|
||||
│ ├── transforms.py # simplex, reverse_bary3d, bary_to_simplex
|
||||
│ ├── splines.py # MonotonicCubicSpline, SplineDistribution
|
||||
│ ├── profiles.py # DirectionProfile, population stats
|
||||
│ ├── classifiers.py # DirectionClassifier (logistic weights)
|
||||
│ ├── results.py # DetectionResult, DimensionSignal, AlarmLevel
|
||||
│ ├── projection.py # project(), decompose()
|
||||
│ └── detection.py # detect(), threshold comparison
|
||||
├── training/
|
||||
│ ├── __init__.py
|
||||
│ ├── compiler.py # build() — SVD, spline fitting, profile comp
|
||||
│ ├── stats.py # pooled_std, cohen_d, silhouette
|
||||
│ └── data_loader.py # Condition catalog, prompt sets, data loading
|
||||
└── data/
|
||||
└── codebook/
|
||||
├── basis.safetensors
|
||||
├── regions.safetensors
|
||||
├── splines.json
|
||||
└── config.json
|
||||
```
|
||||
|
||||
### Extraction from PoC
|
||||
|
||||
The PoC `firewall_codebook.py` is 1,245 lines with significant duplication
|
||||
(the decomposition pipeline z → CDF → simplex → barycentric → (sum, u, v) is
|
||||
repeated 5 times). Analysis identifies:
|
||||
|
||||
- **~480 lines of essential runtime code** in the PoC
|
||||
- **~178 lines needed from metaspline core** (SplineDistribution,
|
||||
MonotonicCubicSpline, ensure_strictly_increasing, simplex)
|
||||
- **~130 lines of histogram classifier** — exploratory alternative, not MVP
|
||||
(the continuous logistic classifier is superior)
|
||||
- **~95 lines of AUC evaluation** — testing tool, not runtime
|
||||
- **~429 lines in `build()`** — must be decomposed: training moves to
|
||||
`training/compiler.py`, runtime state becomes immutable serialized data
|
||||
|
||||
Target: **~400–500 lines runtime + ~150–200 lines training = ~65% compression**
|
||||
from the PoC's 1,245 lines.
|
||||
|
||||
### Key Extraction Decisions
|
||||
|
||||
1. **`build()` moves entirely to `training/compiler.py`** — Runtime codebook
|
||||
is read-only. The codebook class should not have a `build()` method.
|
||||
2. **`decompose()` becomes a pure function** — `decompose(z, splines)` is a
|
||||
pure mathematical transform. No state dependencies beyond splines.
|
||||
3. **Detection is separate from the codebook class** — `detect()` is a
|
||||
stateless function given codebook data. Enables swapping detection
|
||||
strategies without touching the codebook.
|
||||
4. **Only 4 of 502 metaspline core lines are needed at runtime** —
|
||||
`SplineDistribution`, `MonotonicCubicSpline`, `ensure_strictly_increasing`,
|
||||
and `simplex()`. Everything else (DensitySpline, unfold/fold, dcs_norm) is
|
||||
dropped entirely.
|
||||
5. **Saved `.pt` files from the PoC provide golden test data** — manifold
|
||||
projection results for Qwen3-0.6B/1.7B can be reused for integration tests.
|
||||
|
||||
## Data Format
|
||||
|
||||
The codebook is stored as:
|
||||
@@ -243,6 +308,5 @@ class Codebook:
|
||||
Open questions are tracked in [open-questions.md](open-questions.md). Key
|
||||
questions affecting this document:
|
||||
|
||||
- **OQ-02**: What is the minimum viable codebook — can the 1,245-line PoC
|
||||
codebook be compressed? (open)
|
||||
- **OQ-04**: Should detection thresholds be per-model or globally configurable? (open)
|
||||
- **OQ-02**: ~~What is the minimum viable codebook — can the 1,245-line PoC codebook be compressed?~~ (resolved — ~65% compression to 500–600 lines; see Package Structure section)
|
||||
- ~~**OQ-04**~~: ~~Should detection thresholds be per-model or globally configurable?~~ (resolved — both: model-specific defaults, user-overridable)
|
||||
Reference in New Issue
Block a user