Files

glm-5.1 cf464c2296 feat: initial architecture specification and research

Phase 0→1 setup for alknet-firewall — a behavioral signal detection
library that screens untrusted LLM inputs using small model activations.

Architecture docs (5 specs, 10 ADRs, 7 open questions):
- overview: vision, scope, dependencies, package structure
- firewall: core API, alarm protocol, score composition, error handling
- codebook: SVD basis, spline distributions, calibration, tensor format
- model: activation extraction, model-agnostic interface, lazy loading
- configuration: thresholds, model selection, detection tuning

Research reports:
- modern-python-project-setup: uv, pyproject.toml, src layout, ruff, CI
- python-ml-packaging: optional PyTorch, HF Hub download, safetensors
- llm-input-safety-landscape: threat taxonomy, defenses, academic evidence

Agent role adaptations for Python project (replaced Rust conventions).

2026-06-13 05:17:40 +00:00

2.1 KiB

Raw Blame History

ADR-006: PyTorch as Optional Dependency

Status

Accepted

Context

PyTorch is the primary inference backend for the detector model. However, PyTorch is large:

torch (CPU): ~200MB download, ~700MB installed
torch (CUDA): ~2.5GB download, ~5GB+ installed
onnxruntime: ~30-50MB download, ~300MB installed

Making PyTorch a required dependency would force a 200MB-2.5GB download on every user, even those who already have PyTorch installed or prefer ONNX Runtime. This is the standard problem for ML libraries, and the HuggingFace ecosystem has converged on a solution.

Decision

Make PyTorch an optional dependency via extras (pip install alknet-firewall[torch]). The base install includes all non-ML dependencies (sklearn, huggingface-hub, safetensors, tokenizers, numpy). ML inference backends are installed separately.

Use lazy imports with clear error messages when PyTorch is not installed:

try:
    import torch
except ImportError:
    raise ImportError(
        "PyTorch is required for alknet-firewall inference. "
        "Install with: pip install 'alknet-firewall[torch]' "
        "or pip install torch --index-url https://download.pytorch.org/whl/cpu"
    )

Consequences

Positive:

Base install is ~30MB download, ~100MB installed — very lightweight
Users with existing PyTorch installations don't re-download
ONNX Runtime alternative available for minimal footprint (~100MB total)
Follows HuggingFace ecosystem conventions (transformers, safetensors, HF hub all use this pattern)
uv supports CPU/GPU torch variant selection via [tool.uv.sources] and [[tool.uv.index]]

Negative:

More complex dependency specification in pyproject.toml
Users must read installation docs to choose the right extra
Runtime import errors if users forget to install a backend
CPU-only torch requires two-step install or uv configuration (can't be expressed in pip extras alone)

References

modern-python-project-setup.md — Section 2: PyTorch handling
python-ml-packaging.md — Section 1: PyTorch as dependency

2.1 KiB Raw Blame History