glm-5.1
cf464c2296
Phase 0→1 setup for alknet-firewall — a behavioral signal detection library that screens untrusted LLM inputs using small model activations. Architecture docs (5 specs, 10 ADRs, 7 open questions): - overview: vision, scope, dependencies, package structure - firewall: core API, alarm protocol, score composition, error handling - codebook: SVD basis, spline distributions, calibration, tensor format - model: activation extraction, model-agnostic interface, lazy loading - configuration: thresholds, model selection, detection tuning Research reports: - modern-python-project-setup: uv, pyproject.toml, src layout, ruff, CI - python-ml-packaging: optional PyTorch, HF Hub download, safetensors - llm-input-safety-landscape: threat taxonomy, defenses, academic evidence Agent role adaptations for Python project (replaced Rust conventions).
64 lines
2.1 KiB
Markdown
64 lines
2.1 KiB
Markdown
# ADR-006: PyTorch as Optional Dependency
|
|
|
|
## Status
|
|
|
|
Accepted
|
|
|
|
## Context
|
|
|
|
PyTorch is the primary inference backend for the detector model. However,
|
|
PyTorch is large:
|
|
|
|
- `torch` (CPU): ~200MB download, ~700MB installed
|
|
- `torch` (CUDA): ~2.5GB download, ~5GB+ installed
|
|
- `onnxruntime`: ~30-50MB download, ~300MB installed
|
|
|
|
Making PyTorch a required dependency would force a 200MB-2.5GB download on
|
|
every user, even those who already have PyTorch installed or prefer ONNX
|
|
Runtime. This is the standard problem for ML libraries, and the HuggingFace
|
|
ecosystem has converged on a solution.
|
|
|
|
## Decision
|
|
|
|
Make PyTorch an optional dependency via extras (`pip install
|
|
alknet-firewall[torch]`). The base install includes all non-ML dependencies
|
|
(sklearn, huggingface-hub, safetensors, tokenizers, numpy). ML inference
|
|
backends are installed separately.
|
|
|
|
Use lazy imports with clear error messages when PyTorch is not installed:
|
|
|
|
```python
|
|
try:
|
|
import torch
|
|
except ImportError:
|
|
raise ImportError(
|
|
"PyTorch is required for alknet-firewall inference. "
|
|
"Install with: pip install 'alknet-firewall[torch]' "
|
|
"or pip install torch --index-url https://download.pytorch.org/whl/cpu"
|
|
)
|
|
```
|
|
|
|
## Consequences
|
|
|
|
**Positive**:
|
|
- Base install is ~30MB download, ~100MB installed — very lightweight
|
|
- Users with existing PyTorch installations don't re-download
|
|
- ONNX Runtime alternative available for minimal footprint (~100MB total)
|
|
- Follows HuggingFace ecosystem conventions (transformers, safetensors, HF
|
|
hub all use this pattern)
|
|
- uv supports CPU/GPU torch variant selection via `[tool.uv.sources]` and
|
|
`[[tool.uv.index]]`
|
|
|
|
**Negative**:
|
|
- More complex dependency specification in pyproject.toml
|
|
- Users must read installation docs to choose the right extra
|
|
- Runtime import errors if users forget to install a backend
|
|
- CPU-only torch requires two-step install or uv configuration (can't be
|
|
expressed in pip extras alone)
|
|
|
|
## References
|
|
|
|
- [modern-python-project-setup.md](../research/modern-python-project-setup.md) —
|
|
Section 2: PyTorch handling
|
|
- [python-ml-packaging.md](../research/python-ml-packaging.md) — Section 1:
|
|
PyTorch as dependency |