Update architecture docs to address security review #003 findings
Add three ADRs (025-027) and update five spec documents to close gaps identified in the security and bug review: - ADR-025: Rate limiter IP source must be ConnectInfo only (C1 fix) - ADR-026: Connector timeout ceiling of 30s for per-site timeouts (C3 fix) - ADR-027: Admin socket resource limits — 5s timeout, 4096 byte line limit (W4 fix) Spec changes: - proxy.md: add rate limiter IP source section, URI error handling constraint, connector ceiling description, renumber sections - operations.md: add ConnectInfo-only IP source, in-flight counter architectural requirement (C2), JSON format guarantee (C4), admin socket resource limits, 100ms drain polling interval - config.md: fix http_port type u32→u16 (W12), tighten upstream host validation (W1), tighten ACME contact validation (W2), add X-Forwarded-Proto cross-reference, clarify alknet ADR-030 reference - overview.md: fix ambiguous C1 reference, add ADR/OQ cross-references - open-questions.md: update OQ-09 resolution, add OQ-13 (acme_contact Vec) and OQ-14 (eviction configurability) - README.md: add ADR-025/026/027 and OQ-13/14, update doc statuses to draft Also fix reviewer findings: alknet ADR-030 scope clarification, RFC 2616 reference updated to RFC 7230.
This commit is contained in:
@@ -1,5 +1,5 @@
|
||||
---
|
||||
status: reviewed
|
||||
status: draft
|
||||
last_updated: 2026-06-12
|
||||
---
|
||||
|
||||
@@ -25,11 +25,11 @@ connections remain HTTP/1.1.
|
||||
|
||||
| Document | Status | Description |
|
||||
|----------|--------|-------------|
|
||||
| [overview.md](overview.md) | Reviewed | Vision, scope, crate dependencies, exports |
|
||||
| [proxy.md](proxy.md) | Reviewed | Reverse proxy handler, request flow, header injection |
|
||||
| [overview.md](overview.md) | Draft | Vision, scope, crate dependencies, exports |
|
||||
| [proxy.md](proxy.md) | Draft | Reverse proxy handler, request flow, header injection |
|
||||
| [tls.md](tls.md) | Reviewed | TLS termination, ACME, manual certs, SNI, ALPN |
|
||||
| [config.md](config.md) | Reviewed | TOML config format, static/dynamic split, ArcSwap reload |
|
||||
| [operations.md](operations.md) | Reviewed | Rate limiting, logging, health check, systemd, shutdown |
|
||||
| [config.md](config.md) | Draft | TOML config format, static/dynamic split, ArcSwap reload |
|
||||
| [operations.md](operations.md) | Draft | Rate limiting, logging, health check, systemd, shutdown |
|
||||
|
||||
## ADR Table
|
||||
|
||||
@@ -59,6 +59,9 @@ connections remain HTTP/1.1.
|
||||
| [022](decisions/022-health-check-scope.md) | Health Check Scope — Local Port and Admin Socket Only | Accepted |
|
||||
| [023](decisions/023-http2-client-facing.md) | HTTP/2 Client-Facing Support | Accepted |
|
||||
| [024](decisions/024-ansi-disabled-logging.md) | ANSI-Disabled Logging for Container Deployments | Accepted |
|
||||
| [025](decisions/025-rate-limiter-ip-source.md) | Rate Limiter IP Source — ConnectInfo Only | Accepted |
|
||||
| [026](decisions/026-connector-timeout-ceiling.md) | Connector Timeout Ceiling for Per-Site Timeouts | Accepted |
|
||||
| [027](decisions/027-admin-socket-resource-limits.md) | Admin Socket Resource Limits | Accepted |
|
||||
|
||||
## Open Questions
|
||||
|
||||
@@ -74,10 +77,12 @@ See [open-questions.md](open-questions.md) for the full tracker.
|
||||
| ~~OQ-06~~ | ~~Should upstream timeouts be configurable per-site?~~ | ~~low~~ | **resolved** (ADR-015) |
|
||||
| ~~OQ-07~~ | ~~Should per-site TLS overrides be supported for mixed ACME/manual domains?~~ | ~~low~~ | **resolved** (ADR-019) |
|
||||
| ~~OQ-08~~ | ~~Should `/health` use a less common path to avoid upstream collision?~~ | ~~medium~~ | **resolved** (ADR-022: no `/health` route on main listener) |
|
||||
| ~~OQ-09~~ | ~~How should `upstream_connect_timeout_secs` be enforced?~~ | ~~medium~~ | **resolved** (implementation gap — ADR-015 already decides this) |
|
||||
| ~~OQ-09~~ | ~~How should `upstream_connect_timeout_secs` be enforced?~~ | ~~medium~~ | **resolved** (ADR-026: 30s connector ceiling) |
|
||||
| ~~OQ-10~~ | ~~Should ACME contact email be a required config field?~~ | ~~high~~ | **resolved** (already specified in config.md; implementation bug C2) |
|
||||
| ~~OQ-11~~ | ~~How should `X-Forwarded-Proto` be derived per-listener?~~ | ~~medium~~ | **resolved** (hardcoded `https` is correct for TLS-terminating proxy) |
|
||||
| ~~OQ-12~~ | ~~Should request access logging be mandatory or optional?~~ | ~~high~~ | **resolved** (mandatory, always-on per operations.md) |
|
||||
| OQ-13 | Should `acme_contact` support multiple email addresses? | low | open |
|
||||
| OQ-14 | Should rate limiter eviction interval and max age be configurable? | low | open |
|
||||
|
||||
## Document Lifecycle
|
||||
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
---
|
||||
status: reviewed
|
||||
status: draft
|
||||
last_updated: 2026-06-12
|
||||
---
|
||||
|
||||
@@ -75,9 +75,9 @@ config.toml
|
||||
|
||||
## Static vs Dynamic Configuration
|
||||
|
||||
This split follows the pattern established in alknet (ADR-030) and adapted
|
||||
for our simpler use case. See ADR-019 for the rationale behind the
|
||||
`[[listeners]]` configuration format.
|
||||
This split follows the pattern established in alknet (alknet ADR-030, not
|
||||
this project) and adapted for our simpler use case. See ADR-019 for the
|
||||
rationale behind the `[[listeners]]` configuration format.
|
||||
|
||||
### StaticConfig
|
||||
|
||||
@@ -114,16 +114,23 @@ will be handled via signal-based or built-in rotation.
|
||||
| Field | Type | Description |
|
||||
|-------|------|-------------|
|
||||
| `bind_addr` | `String` | IP address to bind to (must be explicit, no `0.0.0.0`; see ADR-016) |
|
||||
| `http_port` | `u32` | Port for HTTP→HTTPS redirect (default: `80`; set to `0` to disable; valid values: 0 or 1–65535) |
|
||||
| `http_port` | `u16` | Port for HTTP→HTTPS redirect (default: `80`; set to `0` to disable; valid values: 0 or 1–65535). Note: the implementation currently uses `u32`; this must be changed to `u16` to match the architecture spec (see Security Review W12). |
|
||||
| `https_port` | `u16` | Port for TLS listener (default: `443`) |
|
||||
| `tls.mode` | `"acme"` or `"manual"` | Certificate provisioning mode |
|
||||
| `tls.acme_domains` | `Vec<String>` | Domains for ACME SAN certificate (ACME mode only) |
|
||||
| `tls.acme_cache_dir` | `String` | ACME state cache directory |
|
||||
| `tls.acme_directory` | `"production"` or `"staging"` | Let's Encrypt directory |
|
||||
| `tls.acme_contact` | `String` | Contact email for ACME registration (e.g., `"mailto:admin@example.com"`). Required for production; Let's Encrypt rejects registrations without a contact email. See OQ-10. |
|
||||
| `tls.acme_contact` | `String` | Contact email for ACME registration (e.g., `"mailto:admin@example.com"`). Required for production; Let's Encrypt rejects registrations without a contact email. Must contain a non-empty email after `mailto:` with an `@` sign. See OQ-10, OQ-13. |
|
||||
| `tls.cert_path` | `String` | Certificate file path (manual mode only) |
|
||||
| `tls.key_path` | `String` | Private key file path (manual mode only) |
|
||||
|
||||
**Note on `X-Forwarded-Proto`**: The `X-Forwarded-Proto` header is derived
|
||||
from which listener port received the request: `https` for requests on the
|
||||
listener's `https_port`, `http` for requests on the `http_port`. In practice,
|
||||
since the HTTP listener sends a 301 redirect rather than proxying,
|
||||
`X-Forwarded-Proto` is always `"https"` for proxied requests. See proxy.md and
|
||||
OQ-11.
|
||||
|
||||
**Why listeners are static:** Each listener requires binding a TCP socket and
|
||||
constructing a TLS acceptor — operations that fundamentally require a restart.
|
||||
Changing a listener's bind address, TLS mode, or certificate configuration
|
||||
@@ -401,14 +408,23 @@ On startup, the config is validated:
|
||||
16. Site `host` values must be valid hostnames (not IP addresses, not
|
||||
including ports). Hostnames are normalized to lowercase during validation.
|
||||
17. `upstream` must be in `host:port` format where `port` is a required integer
|
||||
1–65535. Examples: `gitea:3000`, `127.0.0.1:3000`, `[::1]:3000`. Invalid
|
||||
examples: `gitea` (missing port), `http://gitea:3000` (includes scheme),
|
||||
`10.0.0.5` (missing port). The `upstream_scheme` field handles the protocol.
|
||||
1–65535 and the host part must be a valid DNS hostname or IP address.
|
||||
IPv6 addresses must use bracket notation (e.g., `[::1]:3000`). Values
|
||||
like `!!!bad!!!:3000` or `@#$%:8080` are rejected. The host part is
|
||||
validated as follows: bracket-enclosed values are parsed as IPv6
|
||||
addresses; otherwise the host part must parse as a valid `IpAddr` or
|
||||
pass `is_valid_hostname` validation (same rules as site `host` values).
|
||||
Examples: `gitea:3000`, `127.0.0.1:3000`, `[::1]:3000`. Invalid examples:
|
||||
`gitea` (missing port), `http://gitea:3000` (includes scheme), `10.0.0.5`
|
||||
(missing port), `!!!bad!!!:3000` (invalid host part). The
|
||||
`upstream_scheme` field handles the protocol.
|
||||
18. `upstream_scheme` values are case-sensitive: only `"http"` or `"https"`
|
||||
(lowercase). Default is `"http"`.
|
||||
19. In ACME mode, `tls.acme_contact` must be a valid `mailto:` URI
|
||||
(e.g., `"mailto:admin@example.com"`). Let's Encrypt requires a contact
|
||||
email for production certificate requests.
|
||||
19. In ACME mode, `tls.acme_contact` must be a valid `mailto:` URI with a
|
||||
non-empty email address containing an `@` sign
|
||||
(e.g., `"mailto:admin@example.com"`). Values like `"mailto:"` (empty
|
||||
email) or `"mailto:user"` (no `@`) are rejected. Let's Encrypt requires
|
||||
a contact email for production certificate requests.
|
||||
|
||||
On SIGHUP reload, the same validation applies. If the new config fails
|
||||
validation, the reload is rejected and the old config remains active. An error
|
||||
@@ -434,6 +450,8 @@ All design decisions are documented as ADRs in [decisions/](decisions/).
|
||||
| [016](decisions/016-explicit-bind-address.md) | Explicit bind address required | Rejects `0.0.0.0` to prevent accidental exposure |
|
||||
| [019](decisions/019-multi-config-listeners.md) | Multi-config listeners | `[[listeners]]` supporting both dedicated-IP and shared-IP deployment models |
|
||||
| [020](decisions/020-container-deployment.md) | Container deployment model | Flexible upstream addressing; `allow_wildcard_bind` override for containers |
|
||||
| [026](decisions/026-connector-timeout-ceiling.md) | Connector timeout ceiling | 30s ceiling on connector, per-site timeout via tokio::time::timeout |
|
||||
| [027](decisions/027-admin-socket-resource-limits.md) | Admin socket resource limits | 5s read timeout, 4096 byte line length limit |
|
||||
|
||||
## Open Questions
|
||||
|
||||
@@ -443,4 +461,8 @@ questions affecting this document:
|
||||
- ~~**OQ-04**: Should config reload support a Unix domain socket API in addition
|
||||
to SIGHUP?~~ (resolved — ADR-014: Unix domain socket admin API added)
|
||||
- ~~**OQ-07**: Should per-site TLS overrides be supported for mixed ACME/manual
|
||||
domains?~~ (resolved — ADR-019: `[[listeners]]` with per-listener TLS config)
|
||||
domains?~~ (resolved — ADR-019: `[[listeners]]` with per-listener TLS config)
|
||||
- **OQ-13**: Should `acme_contact` support multiple email addresses? (see
|
||||
[open-questions.md](open-questions.md))
|
||||
- **OQ-14**: Should rate limiter eviction interval and max age be configurable?
|
||||
(see [open-questions.md](open-questions.md))
|
||||
93
docs/architecture/decisions/025-rate-limiter-ip-source.md
Normal file
93
docs/architecture/decisions/025-rate-limiter-ip-source.md
Normal file
@@ -0,0 +1,93 @@
|
||||
# ADR-025: Rate Limiter IP Source Must Be ConnectInfo Only
|
||||
|
||||
## Status
|
||||
|
||||
Accepted
|
||||
|
||||
## Context
|
||||
|
||||
The rate limiter identifies clients by IP address to enforce per-IP token bucket
|
||||
limits. The question is: what is the authoritative source of the client IP for
|
||||
rate limiting?
|
||||
|
||||
Two potential sources exist:
|
||||
|
||||
1. **`ConnectInfo<SocketAddr>`**: The TCP peer address, extracted from
|
||||
`TcpStream::peer_addr()` before TLS handshake and propagated to axum via
|
||||
`ConnectInfoService`. This is the real client IP at the TCP level.
|
||||
|
||||
2. **`X-Forwarded-For` header**: A client-supplied HTTP header that may contain
|
||||
an IP address. This header is set by the proxy handler *after* rate limiting
|
||||
(the rate limiter runs as middleware before the handler), so the value
|
||||
present during rate limit checks is whatever the client sent — completely
|
||||
untrusted.
|
||||
|
||||
ADR-021 already establishes that the proxy is an edge proxy and client-supplied
|
||||
`X-Forwarded-For` headers are untrusted. The proxy handler replaces
|
||||
`X-Forwarded-For` with the `ConnectInfo` IP before forwarding upstream
|
||||
specifically to prevent spoofing. However, ADR-021 only addresses the proxy
|
||||
handler's header injection — it does not specify the rate limiter's IP source.
|
||||
|
||||
The current implementation checks `X-Forwarded-For` *first* and falls back to
|
||||
`ConnectInfo`, which creates two attack vectors:
|
||||
|
||||
- **Rate limit bypass**: A client sends each request with a different random
|
||||
`X-Forwarded-For` value. Every request appears to come from a different IP,
|
||||
evading the per-IP token bucket entirely.
|
||||
- **Denial-of-service via IP spoofing**: A client sends requests with
|
||||
`X-Forwarded-For: <victim IP>`. The victim's bucket is depleted and their
|
||||
legitimate requests receive 429 responses.
|
||||
|
||||
## Decision
|
||||
|
||||
The rate limiter must use `ConnectInfo<SocketAddr>` as the **sole** source of
|
||||
client IP addresses. `X-Forwarded-For` must not be consulted for rate limiting
|
||||
purposes.
|
||||
|
||||
The rate limiter runs as middleware before the proxy handler. At that point in
|
||||
the request lifecycle, no proxy headers have been injected — any `X-Forwarded-For`
|
||||
header present is from the client and is untrusted (ADR-021).
|
||||
|
||||
`ConnectInfo<SocketAddr>` is always present because each listener populates it
|
||||
via `into_make_service_with_connect_info::<SocketAddr>()`. If `ConnectInfo` is
|
||||
absent (which should not happen in normal operation), the request should be
|
||||
rejected rather than falling back to an untrusted header.
|
||||
|
||||
## Rationale
|
||||
|
||||
- The proxy is the edge proxy (ADR-021). Client-supplied headers are
|
||||
untrusted at the edge.
|
||||
- Rate limiting is a security mechanism — it must use the most trustworthy
|
||||
IP source available. `ConnectInfo` is set by the kernel's TCP stack, not by
|
||||
the client.
|
||||
- The rate limiter's position in the middleware stack (before the handler)
|
||||
means it sees raw client headers, not the replaced `X-Forwarded-For` that the
|
||||
proxy handler injects.
|
||||
- Falling back to `X-Forwarded-For` when `ConnectInfo` is absent creates a
|
||||
downgrade attack — an attacker could potentially strip `ConnectInfo` from
|
||||
extensions to force the fallback path. Better to reject than to accept an
|
||||
untrusted IP.
|
||||
|
||||
## Consequences
|
||||
|
||||
**Positive:**
|
||||
- Rate limiting cannot be bypassed via header spoofing
|
||||
- Rate limiting cannot be weaponized to DoS legitimate clients
|
||||
- Consistent with ADR-021's edge proxy trust model
|
||||
- No ambiguity about the IP source — one source, one code path
|
||||
|
||||
**Negative:**
|
||||
- If the proxy is ever placed behind a trusted CDN or load balancer,
|
||||
`ConnectInfo` will reflect the CDN's IP, not the end client's. This would
|
||||
require a "trusted proxies" configuration (already noted as a Phase 2
|
||||
consideration in ADR-021).
|
||||
- Requests without `ConnectInfo` are rejected. This should not happen in
|
||||
normal operation but adds a hard failure mode.
|
||||
|
||||
## References
|
||||
|
||||
- [proxy.md](../proxy.md) — Proxy header injection, request flow
|
||||
- [operations.md](../operations.md) — Rate limiting design
|
||||
- ADR-006 — Token bucket rate limiting
|
||||
- ADR-021 — X-Forwarded-For edge proxy model
|
||||
- Security Review C1 — Rate limiter X-Forwarded-For spoofing vulnerability
|
||||
90
docs/architecture/decisions/026-connector-timeout-ceiling.md
Normal file
90
docs/architecture/decisions/026-connector-timeout-ceiling.md
Normal file
@@ -0,0 +1,90 @@
|
||||
# ADR-026: Connector Timeout Ceiling for Per-Site Timeouts
|
||||
|
||||
## Status
|
||||
|
||||
Accepted
|
||||
|
||||
## Context
|
||||
|
||||
ADR-015 specifies per-site upstream connect timeout configuration with a default
|
||||
of 5 seconds. The proxy enforces connect timeouts using two mechanisms:
|
||||
|
||||
1. **`tokio::time::timeout`**: Wraps the entire `client.request()` call with the
|
||||
per-site `upstream_connect_timeout_secs` value.
|
||||
2. **`HttpConnector::set_connect_timeout()`**: Sets the TCP-level connect timeout
|
||||
on the hyper `HttpConnector` inside the shared client.
|
||||
|
||||
The problem: the HTTP connector's `set_connect_timeout()` is set once at client
|
||||
creation time and applies to all requests through that client. The current
|
||||
implementation hardcodes this to 5 seconds. Since the connector's internal
|
||||
timeout fires before `tokio::time::timeout`, any per-site connect timeout
|
||||
greater than 5 seconds is silently capped — the connector times out at 5s
|
||||
regardless of the configured value.
|
||||
|
||||
Three approaches exist:
|
||||
|
||||
1. **Raise the connector timeout to a high ceiling**: Set the connector's
|
||||
`set_connect_timeout` to a value higher than any reasonable per-site timeout
|
||||
(e.g., 30s). Let `tokio::time::timeout` enforce the actual per-site limit.
|
||||
The connector timeout becomes a safety ceiling, not the primary enforcement
|
||||
mechanism.
|
||||
|
||||
2. **Remove the connector timeout entirely**: Set `set_connect_timeout` to
|
||||
`None` and rely solely on `tokio::time::timeout`. This removes one layer
|
||||
of timeout enforcement but simplifies the model.
|
||||
|
||||
3. **Create per-site client instances**: Each site gets its own hyper Client
|
||||
with its own connector configured with the site's connect timeout. This is
|
||||
the most precise approach but creates many client instances and connection
|
||||
pools, increasing resource usage.
|
||||
|
||||
## Decision
|
||||
|
||||
Use approach 1: set the connector timeout to a high ceiling value (30 seconds)
|
||||
and let `tokio::time::timeout` enforce the actual per-site connect timeout.
|
||||
|
||||
The connector timeout serves as a safety ceiling — it ensures that even if the
|
||||
`tokio::time::timeout` wrapper fails or is misconfigured, TCP connections
|
||||
cannot hang indefinitely. The ceiling of 30s is well above the default 5s and
|
||||
any reasonable per-site override.
|
||||
|
||||
The `tokio::time::timeout` wrapper with the per-site value is the primary
|
||||
enforcement mechanism. It fires at the correct per-site threshold.
|
||||
|
||||
## Rationale
|
||||
|
||||
- The shared client architecture (ADR-017) means one connector timeout for all
|
||||
sites. Creating per-site clients would undermine connection pooling and
|
||||
increase resource usage.
|
||||
- A ceiling approach preserves the defense-in-depth benefit of two timeout
|
||||
layers while allowing per-site values to actually work.
|
||||
- 30s is a reasonable ceiling — no legitimate upstream connect should take
|
||||
longer than 30s. Sites that need a higher connect timeout can set the ceiling
|
||||
even higher if needed, but 30s covers all practical cases.
|
||||
- Removing the connector timeout (approach 2) removes the safety ceiling
|
||||
entirely. If `tokio::time::timeout` has a bug or is misapplied, TCP connects
|
||||
could hang indefinitely. The ceiling provides a backstop.
|
||||
- The HTTPS client uses `HttpsConnector<HttpConnector>`, which wraps the
|
||||
`HttpConnector`. The same ceiling applies to both HTTP and HTTPS clients.
|
||||
|
||||
## Consequences
|
||||
|
||||
**Positive:**
|
||||
- Per-site connect timeouts work as documented (ADR-015)
|
||||
- Maintains defense-in-depth with two timeout layers
|
||||
- No change to the shared client / connection pooling architecture
|
||||
- Simple implementation: change one constant
|
||||
|
||||
**Negative:**
|
||||
- The connector timeout no longer matches the default connect timeout (5s
|
||||
default vs. 30s ceiling). Operators who read the connector timeout might
|
||||
be confused — documentation must make the ceiling role clear.
|
||||
- If a site needs a connect timeout > 30s, the ceiling must be raised. This
|
||||
is unlikely in practice but creates a hidden upper bound.
|
||||
|
||||
## References
|
||||
|
||||
- [proxy.md](../proxy.md) — Upstream connection, per-site timeouts
|
||||
- ADR-015 — Per-site upstream timeouts with defaults
|
||||
- ADR-017 — Upstream connection defaults
|
||||
- Security Review C3 — Connect timeout silently capped at 5s
|
||||
@@ -0,0 +1,77 @@
|
||||
# ADR-027: Admin Socket Resource Limits
|
||||
|
||||
## Status
|
||||
|
||||
Accepted
|
||||
|
||||
## Context
|
||||
|
||||
The admin Unix domain socket (ADR-014) accepts connections from local
|
||||
processes and reads one newline-terminated command per connection. The current
|
||||
implementation has no read timeout and no line length limit.
|
||||
|
||||
Two attack vectors exist for processes with access to the admin socket
|
||||
(controlled by Unix file permissions):
|
||||
|
||||
1. **Connection hold**: A client connects and sends no data, keeping a
|
||||
connection and tokio task open indefinitely. An attacker with socket access
|
||||
can open many such connections to exhaust file descriptors or task slots.
|
||||
|
||||
2. **Unbounded memory allocation**: A client sends data without a newline,
|
||||
causing `read_line` to buffer indefinitely. This allows unbounded memory
|
||||
consumption from a single connection.
|
||||
|
||||
While the admin socket is protected by Unix file permissions (only processes
|
||||
with access to the socket file can connect), defense-in-depth warrants basic
|
||||
resource limits. The socket is a local diagnostic interface, not a
|
||||
high-performance endpoint — strict limits are appropriate.
|
||||
|
||||
## Decision
|
||||
|
||||
Apply two resource limits to admin socket connections:
|
||||
|
||||
1. **Read timeout**: 5 seconds per connection. If no complete command is
|
||||
received within 5 seconds, the connection is closed and a timeout error
|
||||
is logged at `debug` level.
|
||||
|
||||
2. **Line length limit**: 4096 bytes per command. If a client sends more
|
||||
than 4096 bytes without a newline, the connection is closed and the event
|
||||
is logged at `warn` level. The longest valid command (`reload`) is 6 bytes —
|
||||
4096 bytes provides ample room for future commands while preventing abuse.
|
||||
|
||||
Both limits apply per connection. The admin socket creates one tokio task per
|
||||
connection, so the limits also bound per-connection resource usage.
|
||||
|
||||
## Rationale
|
||||
|
||||
- Defense-in-depth: even with Unix permission protection, basic resource
|
||||
limits prevent accidental or deliberate resource exhaustion.
|
||||
- 5 seconds is generous for a local socket — a local process should be able
|
||||
to send a command within milliseconds. The timeout handles stuck clients and
|
||||
network issues on tunnel-mounted sockets.
|
||||
- 4096 bytes is 600x the longest current command. It accommodates future
|
||||
multi-field commands without allowing abuse.
|
||||
- These limits have no impact on legitimate admin socket usage — the current
|
||||
commands (`reload`, `status`) are tiny and complete instantly.
|
||||
|
||||
## Consequences
|
||||
|
||||
**Positive:**
|
||||
- No unbounded memory allocation from admin socket connections
|
||||
- No indefinite connection holding
|
||||
- Simple implementation: `tokio::time::timeout` + `tokio::io::take` (or
|
||||
`read_until` with a byte limit)
|
||||
- No impact on legitimate usage
|
||||
|
||||
**Negative:**
|
||||
- If a future admin command requires more than 4096 bytes (unlikely), the
|
||||
limit must be raised.
|
||||
- A 5-second timeout could cause issues if the admin socket is accessed via
|
||||
a slow network tunnel. In practice, this is unlikely — admin socket access
|
||||
is typically local.
|
||||
|
||||
## References
|
||||
|
||||
- ADR-014 — Unix domain socket config reload API
|
||||
- [operations.md](../operations.md) — Admin socket protocol
|
||||
- Security Review W4 — Admin socket has no read timeout or line length limit
|
||||
@@ -1,5 +1,5 @@
|
||||
---
|
||||
status: reviewed
|
||||
status: draft
|
||||
last_updated: 2026-06-12
|
||||
---
|
||||
|
||||
@@ -111,18 +111,21 @@ last_updated: 2026-06-12
|
||||
|
||||
### ~~OQ-09: How should `upstream_connect_timeout_secs` be enforced?~~
|
||||
|
||||
- **Origin**: Implementation review finding W4, ADR-015, ADR-017
|
||||
- **Origin**: Implementation review finding W4, ADR-015, ADR-017, Security
|
||||
Review C3
|
||||
- **Status**: resolved
|
||||
- **Priority**: medium
|
||||
- **Resolution**: Implemented using a two-phase `tokio::time::timeout` approach.
|
||||
The inner timeout uses the per-site `upstream_connect_timeout_secs` (default
|
||||
5s) for the connect + first-byte phase, and the outer timeout uses
|
||||
`upstream_request_timeout_secs` (default 60s) for the full request/response
|
||||
cycle. Additionally, `HttpConnector::set_connect_timeout()` enforces the
|
||||
TCP-level connect timeout on both HTTP and HTTPS clients. The implementation
|
||||
is in `handler.rs` and `create_http_client()`/`create_https_client()`.
|
||||
No new ADR needed; the decision was already made in ADR-015.
|
||||
- **Cross-references**: ADR-015, ADR-017
|
||||
cycle. The shared `HttpConnector` uses a 30-second connect timeout ceiling
|
||||
via `set_connect_timeout()` — this is a safety backstop, not the primary
|
||||
enforcement mechanism. The per-site `tokio::time::timeout` enforces the
|
||||
actual connect timeout. This ensures per-site values >5s work correctly
|
||||
(previously the hardcoded 5s connector timeout silently capped them). See
|
||||
ADR-026.
|
||||
- **Cross-references**: ADR-015, ADR-017, ADR-026
|
||||
|
||||
### ~~OQ-10: Should ACME contact email be a required config field?~~
|
||||
|
||||
@@ -168,4 +171,33 @@ last_updated: 2026-06-12
|
||||
disable access logging." The `log_request!` macro exists in the codebase
|
||||
but is not called — this is an implementation gap (W13), not an
|
||||
architectural question. No ADR needed; ADR-007 already covers the log format.
|
||||
- **Cross-references**: ADR-007
|
||||
- **Cross-references**: ADR-007
|
||||
|
||||
## Configuration
|
||||
|
||||
### OQ-13: Should `acme_contact` support multiple email addresses?
|
||||
|
||||
- **Origin**: Security Review S9, [config.md](config.md), [tls.md](tls.md)
|
||||
- **Status**: open
|
||||
- **Priority**: low
|
||||
- **Details**: `acme_contact` is currently a single `String`, but ACME supports
|
||||
multiple contact emails. The `AcmeTlsConfig.contact` field in the
|
||||
implementation is already `Vec<String>`, and the single config value is
|
||||
wrapped in `vec![...]`. Changing `acme_contact` to `Vec<String>` in the
|
||||
config schema would provide consistency with the ACME protocol. However,
|
||||
this is a config format change that requires migration documentation and
|
||||
backward compatibility considerations. For Phase 1, a single email is
|
||||
sufficient.
|
||||
- **Cross-references**: ADR-004
|
||||
|
||||
### OQ-14: Should rate limiter eviction interval and max age be configurable?
|
||||
|
||||
- **Origin**: Security Review S2, [operations.md](operations.md)
|
||||
- **Status**: open
|
||||
- **Priority**: low
|
||||
- **Details**: The eviction task interval (60s) and max age (300s) are
|
||||
currently hardcoded. In high-traffic deployments, a shorter interval or
|
||||
longer max age might be desirable. These would be dynamic config fields
|
||||
(hot-reloadable via ArcSwap) if added. For Phase 1, the hardcoded values
|
||||
are reasonable defaults.
|
||||
- **Cross-references**: ADR-006
|
||||
@@ -1,5 +1,5 @@
|
||||
---
|
||||
status: reviewed
|
||||
status: draft
|
||||
last_updated: 2026-06-12
|
||||
---
|
||||
|
||||
@@ -32,6 +32,12 @@ The rate limiter runs as axum middleware before the proxy handler. It uses a
|
||||
token bucket algorithm per client IP, matching nginx's `limit_req burst`
|
||||
semantics.
|
||||
|
||||
The client IP for rate limiting is determined **exclusively** from
|
||||
`ConnectInfo<SocketAddr>` — the TCP peer address set before TLS handshake.
|
||||
Client-supplied `X-Forwarded-For` headers must not be consulted because the
|
||||
rate limiter runs before the proxy handler injects trusted headers. See
|
||||
ADR-025.
|
||||
|
||||
Rate limits are global per-IP in Phase 1 (not per-site). A request from IP
|
||||
address X counts against the same bucket regardless of which site it targets.
|
||||
Per-site rate limits may be added in Phase 2.
|
||||
@@ -142,6 +148,11 @@ ADR-024.
|
||||
The `tracing-subscriber` layer configuration supports both simultaneously via
|
||||
`Layer` composition.
|
||||
|
||||
Both output destinations must respect the `format` config value: when
|
||||
`format = "json"`, both file and stdout output must use JSON formatting.
|
||||
When `format = "text"`, both use text formatting. The format must not be
|
||||
silently ignored in any output path (see Security Review C4).
|
||||
|
||||
### File Logging and fail2ban
|
||||
|
||||
File logging is the primary integration point for fail2ban. A log file on a
|
||||
@@ -286,6 +297,11 @@ rationale.
|
||||
one newline-terminated command, receives one newline-terminated JSON
|
||||
response, then the server closes the connection.
|
||||
- **Message framing**: Newline-delimited (`\n`). Responses end with `\n`.
|
||||
- **Resource limits** (see ADR-027):
|
||||
- Read timeout: 5 seconds. Connections that send no complete command within
|
||||
5 seconds are closed. The timeout is logged at `debug` level.
|
||||
- Line length limit: 4096 bytes. Connections that send more than 4096 bytes
|
||||
without a newline are closed. The event is logged at `warn` level.
|
||||
- **Commands**:
|
||||
- `reload` — Re-read config file, validate, and swap DynamicConfig. Returns
|
||||
`{"status": "ok"}` or `{"status": "error", "message": "..."}`.
|
||||
@@ -309,9 +325,17 @@ On SIGTERM or SIGINT, the proxy performs a graceful shutdown:
|
||||
2. **Close idle keep-alive connections** — Send `Connection: close` on any idle
|
||||
connections in the keep-alive pool.
|
||||
3. **Wait for in-flight requests** — Up to `shutdown_timeout_secs` (default: 30)
|
||||
for active requests to complete. Server tasks are joined (not aborted) so
|
||||
that in-flight requests can drain normally. Only after the timeout expires
|
||||
are remaining tasks aborted.
|
||||
for active requests to complete. The proxy tracks in-flight requests using
|
||||
an atomic counter: each request **must** increment the counter when it
|
||||
begins and decrement when it completes (via guard drop). The increment
|
||||
must happen before the request task is spawned — if the counter is not
|
||||
incremented, the drain logic is broken (see Security Review C2). During
|
||||
drain, the proxy polls the counter every 100ms and exits early
|
||||
when it reaches zero. If the timeout expires before all requests complete,
|
||||
the proxy logs how many in-flight requests remain and proceeds to
|
||||
force-close. Server tasks are joined (not aborted) so that in-flight
|
||||
requests can drain normally. Only after the timeout expires are remaining
|
||||
tasks aborted.
|
||||
4. **Force-close remaining connections** — After the timeout, any remaining
|
||||
connections are forcefully closed via TCP RST.
|
||||
5. **Cancel background tasks** — ACME renewal tasks, rate limiter eviction task,
|
||||
@@ -592,11 +616,13 @@ All design decisions are documented as ADRs in [decisions/](decisions/).
|
||||
| [014](decisions/014-unix-socket-reload.md) | Unix domain socket config reload API | Programmatic reload with success/failure feedback |
|
||||
| [020](decisions/020-container-deployment.md) | Container deployment model | Defense-in-depth via container isolation; file-primary logging |
|
||||
| [024](decisions/024-ansi-disabled-logging.md) | ANSI-disabled logging | All log output uses `with_ansi(false)` for fail2ban and Docker compatibility |
|
||||
| [025](decisions/025-rate-limiter-ip-source.md) | Rate limiter IP source | ConnectInfo only, never client-supplied X-Forwarded-For |
|
||||
| [027](decisions/027-admin-socket-resource-limits.md) | Admin socket resource limits | 5s read timeout, 4096 byte line length limit |
|
||||
|
||||
## Open Questions
|
||||
|
||||
Open questions are tracked in [open-questions.md](open-questions.md). All
|
||||
questions affecting this document have been resolved:
|
||||
Open questions are tracked in [open-questions.md](open-questions.md). Key
|
||||
questions affecting this document:
|
||||
|
||||
- ~~**OQ-03**: Should the health check endpoint be on a separate port?~~ (resolved
|
||||
— ADR-013: separate local port, default 9900, localhost only)
|
||||
@@ -605,4 +631,6 @@ questions affecting this document have been resolved:
|
||||
9900 and admin socket only)
|
||||
- ~~**OQ-12**: Should request access logging be mandatory or optional?~~ (resolved
|
||||
— access logging is mandatory and always-on at `info` level; no configuration
|
||||
option to disable it)
|
||||
option to disable it)
|
||||
- **OQ-14**: Should rate limiter eviction interval and max age be configurable?
|
||||
(see [open-questions.md](open-questions.md))
|
||||
@@ -1,5 +1,5 @@
|
||||
---
|
||||
status: reviewed
|
||||
status: draft
|
||||
last_updated: 2026-06-12
|
||||
---
|
||||
|
||||
@@ -133,7 +133,8 @@ but all routers share `Arc<ArcSwap<DynamicConfig>>` and
|
||||
`Arc<Mutex<HashMap<IpAddr, TokenBucket>>>` via axum State. Site routing is
|
||||
global: the `Host` header is matched against a single routing table collected
|
||||
from all listeners' site definitions. Hostnames must be unique across all
|
||||
listeners — see C1 resolution in the architecture review.
|
||||
listeners. Hostnames must be unique across all listeners — see Security & Bug
|
||||
Review #003, finding C1, resolved by ADR-025.
|
||||
|
||||
In container deployments (ADR-020), the proxy runs in a minimal container with
|
||||
`0.0.0.0` bind address and Docker port publishing. Upstream addresses use Docker
|
||||
@@ -217,6 +218,9 @@ All design decisions are documented as ADRs in [decisions/](decisions/).
|
||||
| [022](decisions/022-health-check-scope.md) | Health check scope — local port and admin socket only | No `/health` route on main listener; health check is port 9900/admin socket only |
|
||||
| [023](decisions/023-http2-client-facing.md) | HTTP/2 client-facing support | ALPN-based protocol detection; HTTP/2 to clients, HTTP/1.1 to upstreams |
|
||||
| [024](decisions/024-ansi-disabled-logging.md) | ANSI-disabled logging | All log output uses `with_ansi(false)` for fail2ban and Docker compatibility |
|
||||
| [025](decisions/025-rate-limiter-ip-source.md) | Rate limiter IP source | ConnectInfo only, never client-supplied X-Forwarded-For |
|
||||
| [026](decisions/026-connector-timeout-ceiling.md) | Connector timeout ceiling | 30s ceiling on connector, per-site timeout via tokio::time::timeout |
|
||||
| [027](decisions/027-admin-socket-resource-limits.md) | Admin socket resource limits | 5s read timeout, 4096 byte line length limit |
|
||||
|
||||
## Open Questions
|
||||
|
||||
@@ -227,4 +231,6 @@ questions affecting this document have been resolved:
|
||||
- ~~**OQ-03**: Should the health check endpoint be on a separate port?~~ (resolved — ADR-013)
|
||||
- ~~**OQ-05**: Should the proxy bind to multiple addresses?~~ (resolved — single `bind_addr` per listener)
|
||||
- ~~**OQ-07**: Should per-site TLS overrides be supported for mixed ACME/manual domains?~~ (resolved — ADR-019: `[[listeners]]` with per-listener TLS config)
|
||||
- ~~**OQ-08**: Should `/health` use a less common path?~~ (resolved — ADR-022: no `/health` route on main listener; health check is port 9900/admin socket only)
|
||||
- ~~**OQ-08**: Should `/health` use a less common path?~~ (resolved — ADR-022: no `/health` route on main listener; health check is port 9900/admin socket only)
|
||||
- **OQ-13**: Should `acme_contact` support multiple email addresses? (see [open-questions.md](open-questions.md))
|
||||
- **OQ-14**: Should rate limiter eviction interval and max age be configurable? (see [open-questions.md](open-questions.md))
|
||||
@@ -1,5 +1,5 @@
|
||||
---
|
||||
status: reviewed
|
||||
status: draft
|
||||
last_updated: 2026-06-12
|
||||
---
|
||||
|
||||
@@ -46,7 +46,7 @@ Incoming HTTPS request (HTTP/1.1 or HTTP/2)
|
||||
▼
|
||||
┌─────────────────┐
|
||||
│ Rate Limiting │ ← tower middleware layer
|
||||
│ Middleware │
|
||||
│ Middleware │ ← IP from ConnectInfo only (ADR-025)
|
||||
└───────┬─────────┘
|
||||
│
|
||||
▼
|
||||
@@ -116,7 +116,21 @@ port (default: 9900, bound to `127.0.0.1` only) and the admin socket's `status`
|
||||
command — not by intercepting traffic on the public-facing proxy. See ADR-013
|
||||
and ADR-022.
|
||||
|
||||
### 2. Proxy Header Injection
|
||||
### 2. Rate Limiter IP Source
|
||||
|
||||
The rate limiting middleware runs **before** the proxy handler. At that point,
|
||||
no proxy headers have been injected — any `X-Forwarded-For` header present is
|
||||
from the client and is untrusted. The rate limiter must use
|
||||
`ConnectInfo<SocketAddr>` as the **sole** source of client IP addresses.
|
||||
Client-supplied `X-Forwarded-For` headers must not be consulted for rate
|
||||
limiting. See ADR-025.
|
||||
|
||||
`ConnectInfo<SocketAddr>` is always present because each listener populates it
|
||||
via `into_make_service_with_connect_info::<SocketAddr>()`. If `ConnectInfo`
|
||||
is absent, the request must be rejected rather than falling back to an
|
||||
untrusted header.
|
||||
|
||||
### 3. Proxy Header Injection
|
||||
|
||||
Headers are injected before forwarding. The proxy is an **edge proxy** — it
|
||||
sits directly in front of the internet with no trusted proxies upstream. This
|
||||
@@ -135,12 +149,16 @@ extracting `TcpStream::peer_addr()` before wrapping the connection in
|
||||
`TlsStream`. Each listener provides this information to its axum Router via
|
||||
`axum::ServiceExt::into_make_service_with_connect_info::<SocketAddr>()`.
|
||||
|
||||
### 3. Request Forwarding
|
||||
### 4. Request Forwarding
|
||||
|
||||
The proxy handler constructs a new request to the upstream:
|
||||
|
||||
1. Build the upstream URI using the site's `upstream_scheme` and `upstream`
|
||||
address, preserving the original path and query string
|
||||
address, preserving the original path and query string. **If URI
|
||||
construction fails** (e.g., the resulting URI is malformed), the proxy must
|
||||
return 502 Bad Gateway and log the error at `warn` level. The proxy must
|
||||
never silently drop parts of the URI (such as the query string) — a
|
||||
malformed upstream URI is an error, not a recoverable condition.
|
||||
2. Copy the request method, headers, and body from the original
|
||||
3. Inject proxy headers (X-Real-IP, X-Forwarded-For, X-Forwarded-Proto)
|
||||
4. Remove hop-by-hop headers (Connection, Keep-Alive, Transfer-Encoding, etc.)
|
||||
@@ -173,12 +191,12 @@ specified, defaults of 5s connect and 60s request are used. Both timeouts are
|
||||
enforced using `tokio::time::timeout`, with the connect timeout nested inside
|
||||
the request timeout to ensure the overall deadline is respected.
|
||||
|
||||
### 4. Header Handling
|
||||
### 5. Header Handling
|
||||
|
||||
The proxy must handle request and response headers correctly to avoid security
|
||||
issues and protocol violations.
|
||||
|
||||
**Headers removed before forwarding (hop-by-hop headers per RFC 2616 §13.5.1):**
|
||||
**Headers removed before forwarding (hop-by-hop headers per RFC 7230 §6.1):**
|
||||
|
||||
- `Connection`
|
||||
- `Keep-Alive`
|
||||
@@ -217,7 +235,7 @@ exceptions:
|
||||
- The `Server` header is removed (defense-in-depth: hiding upstream identity)
|
||||
- The proxy does not add a `Server` header to responses
|
||||
|
||||
### 5. Error Handling
|
||||
### 6. Error Handling
|
||||
|
||||
All error responses use plain text bodies with no proxy version or identity
|
||||
information. No upstream error details are included. Response format:
|
||||
@@ -237,7 +255,7 @@ information. No upstream error details are included. Response format:
|
||||
| Unknown Host header | 404 Not Found | `Not Found` | No matching site definition |
|
||||
| Missing Host header (and no URI host) | 400 Bad Request | `Bad Request` | Required for routing; HTTP/2 clients use `:authority` |
|
||||
|
||||
### 6. HTTP → HTTPS Redirect
|
||||
### 7. HTTP → HTTPS Redirect
|
||||
|
||||
A separate HTTP listener on port 80 (per listener) handles redirect. It reads
|
||||
the `Host` header from the incoming request and returns a 301 Permanent Redirect
|
||||
@@ -280,9 +298,13 @@ Two shared hyper Client instances handle upstream connections:
|
||||
- **HTTPS client** (`Client<HttpsConnector<HttpConnector>, Body>`): For
|
||||
`https://` upstreams, using `hyper-rustls` with system native certificates
|
||||
|
||||
Both clients enforce the per-site connect timeout (default 5s) at the TCP level
|
||||
via `HttpConnector::set_connect_timeout()` and the overall request timeout
|
||||
(default 60s) via `tokio::time::timeout`.
|
||||
Both clients use a shared `HttpConnector` with a connect timeout ceiling
|
||||
(30 seconds) set via `HttpConnector::set_connect_timeout()`. This ceiling
|
||||
ensures TCP connections cannot hang indefinitely even if the per-site
|
||||
`tokio::time::timeout` wrapper fails. The per-site connect timeout (default
|
||||
5s) is enforced by `tokio::time::timeout`, which fires at the correct
|
||||
per-site threshold. The connector ceiling is a safety backstop, not the
|
||||
primary enforcement mechanism. See ADR-026.
|
||||
|
||||
## Body Size Limit
|
||||
|
||||
@@ -306,11 +328,13 @@ All design decisions are documented as ADRs in [decisions/](decisions/).
|
||||
| [018](decisions/018-body-size-limit.md) | Request body size limit | 100 MB default matching nginx, Gitea push compatibility |
|
||||
| [021](decisions/021-x-forwarded-for-edge-proxy.md) | X-Forwarded-For edge proxy model | Replace, don't append — proxy is the edge, no trusted upstream proxies |
|
||||
| [023](decisions/023-http2-client-facing.md) | HTTP/2 client-facing support | ALPN-based protocol detection; HTTP/2 to clients, HTTP/1.1 to upstreams |
|
||||
| [025](decisions/025-rate-limiter-ip-source.md) | Rate limiter IP source | ConnectInfo only, never client-supplied X-Forwarded-For |
|
||||
| [026](decisions/026-connector-timeout-ceiling.md) | Connector timeout ceiling | 30s ceiling on connector, per-site timeout via tokio::time::timeout |
|
||||
|
||||
## Open Questions
|
||||
|
||||
Open questions are tracked in [open-questions.md](open-questions.md). All
|
||||
questions affecting this document have been resolved:
|
||||
Open questions are tracked in [open-questions.md](open-questions.md). Key
|
||||
questions affecting this document:
|
||||
|
||||
- ~~**OQ-06**: Should upstream timeouts be configurable per-site?~~ (resolved —
|
||||
ADR-015: per-site timeout overrides with defaults)
|
||||
@@ -318,5 +342,7 @@ questions affecting this document have been resolved:
|
||||
upstream collision?~~ (resolved — ADR-022: no `/health` route on the main
|
||||
listener; health checking is via port 9900 and admin socket only)
|
||||
- ~~**OQ-09**: How should `upstream_connect_timeout_secs` be enforced?~~
|
||||
(resolved — two-phase timeout with `tokio::time::timeout`; connect timeout
|
||||
nested inside request timeout; TCP-level `set_connect_timeout` on connector)
|
||||
(resolved — ADR-026: 30s connector ceiling, per-site timeout via
|
||||
`tokio::time::timeout`)
|
||||
- **OQ-13**: Should `acme_contact` support multiple email addresses? (see
|
||||
[open-questions.md](open-questions.md))
|
||||
Reference in New Issue
Block a user