Fluxheim can distribute HTTP requests across several application servers. Each server is a backend; the group of servers is a pool. Health checks remove unavailable members from selection, and an optional persistence policy keeps a client on the same available member.
Start with two identical application instances and round-robin selection. Add affinity only if the application actually needs it. A shared session store is usually easier to operate than depending on a particular application instance.
This guide covers native HTTP proxy pools. TCP stream routing has its own configuration; do not assume that HTTP cookies, response-status rules or HTTP retry behavior apply to it. For every field and default, see the load-balancer option reference.
Use a binary with proxy and load-balancer enabled. The packaged load-balancer
profile includes them. For a small local HTTP-only build:
cargo build --locked --no-default-features --features proxy,load-balancer --bin fluxheimUse a TLS-enabled profile for an Internet-facing deployment. The examples below intentionally bind only to loopback and do not configure certificates, public DNS, an admin listener or a metrics listener.
Latency scope: native HTTP supplies final-response-header wait samples to
least-time and passive max_latency_ms. These are not full response times;
use the measurement boundary below when choosing a threshold. See
retry and latency boundaries.
Run these commands from the repository root, each in a separate terminal:
python3 -m http.server 9001 --bind 127.0.0.1 --directory examples/load-balancer/origin-apython3 -m http.server 9002 --bind 127.0.0.1 --directory examples/load-balancer/origin-bBoth serve /healthz; their home pages identify the origin. The complete
local configuration contains this pool:
[proxy]
upstreams = ["127.0.0.1:9001", "127.0.0.1:9002"]
upstream_aliases = ["app-a", "app-b"]
upstream_weights = [1, 1]
upstream_tls = false
connect_timeout_secs = 1
read_timeout_secs = 3
[proxy.load_balance]
selection = "round-robin"
all_down_status = 503
[proxy.load_balance.health_check]
enabled = true
protocol = "http"
path = "/healthz"
interval_secs = 1
consecutive_success = 1
consecutive_failure = 1
expected_statuses = [200]
expected_body_contains = ["ready"]
connect_timeout_secs = 1
read_timeout_secs = 1Validate and start Fluxheim in a third terminal:
target/debug/fluxheim --config examples/load-balancer/local.toml --validate-config
target/debug/fluxheim --config examples/load-balancer/local.tomlSend several requests in another terminal:
for attempt in 1 2 3 4 5 6; do
curl --fail --max-time 5 http://127.0.0.1:8088/
doneYou should see both origin-a and origin-b. Stop origin A with Ctrl-C, wait a
few health-check intervals, and repeat: only B should respond. Restart A and it
should rejoin after a successful probe. Stop both origins and Fluxheim returns
503 once neither is selectable. During failure detection a request can instead
receive a transport-error response; health checks are not instantaneous.
The Python servers are test fixtures, not production application servers. Stop
all three processes with Ctrl-C when finished. On Windows, use the .exe binary
and repeat curl.exe requests instead of the shell loop.
local.toml is a complete configuration. The other files in
examples/load-balancer are annotated fragments.
Edit matching fields into the existing tables; do not concatenate duplicate
[proxy] or [proxy.load_balance] tables. The discovery recipes replace the
whole proxy block, including its nested tables, because a pool cannot use static
members and a discovery feed simultaneously.
The same pool settings can live under [vhosts.proxy] or
[vhosts.routes.proxy]. Use the corresponding nested prefix for its
load_balance tables. Keep the members and policy together; consult
vhost configuration for inheritance and route matching rather
than assuming that separate route pools share counters or persistence entries.
| Need | Selection | What to expect |
|---|---|---|
| Similar, stateless application instances | round-robin |
Weighted distribution over eligible members; not a promise of exact alternation |
| Requests with different processing times | least-connections |
Lowest weighted number of in-flight request permits, not the number of idle TCP sockets |
| Spread new sticky sessions | least-sessions |
Lowest weighted number of local persistence entries; requires persistence enabled |
| Low-cost approximate balancing | power-of-two |
Compare sampled candidates using current load |
| Stable mapping by source, URI, header or cookie | *-hash families |
Use the matching request key; missing keys fall back rather than authenticating a client |
| Reduce remapping during membership changes | consistent-*-hash |
Consistent hashing; not shared cross-node session state |
| Consistent hashing with load escape | bounded-load-consistent-*-hash |
Allows selection away from the preferred hash member when its load is excessive |
| Static compatible hash layout | nginx-consistent-*-hash (Ketama), maglev-*-hash |
Static-ring restrictions apply; no discovery or runtime backend-set edits |
| Latency-aware balancing | least-time |
Prefer the lowest weighted response-header latency; unknown/stale eligible members are sampled first |
For a header key, change these fields in the pool:
[proxy.load_balance]
selection = "consistent-header-hash"
hash_header = "x-session"For a cookie key, use selection = "consistent-cookie-hash" and
hash_cookie = "session", removing hash_header. A bounded-load selector can
add bounded_load_factor_per_mille = 1250; larger values tolerate more imbalance.
The reference lists all names,
aliases and required companion fields.
Source-based selection uses the native request's effective/peer address. Do not
assume an arbitrary X-Forwarded-For value changes that key. Configure trusted
ingress and PROXY protocol deliberately. Client-supplied headers and cookies are
not credentials, even when they are useful routing keys. Hashing is not an
authorization or tenant-isolation boundary.
The lists alongside upstreams are positional: the first weight, locality and
limit belong to the first address. Keep them aligned when editing members.
Aliases give operators readable names; tags are status metadata, not selectors.
The pool-policy recipe illustrates weights, priority groups, locality preference and in-flight limits together. A weight of 2 versus 1 requests roughly twice the share with weighted round-robin; least-connections instead compares load relative to weight. A cap limits active request permits for that backend, not all connections to that machine.
Higher priority groups are preferred. upstream_priority_group_min_active
controls when lower groups are needed. Preferred localities are a preference
with fallback, not a strict regional routing or data-residency guarantee.
For a dedicated spare, use a third backend:
[proxy]
upstreams = ["10.0.0.10:8080", "10.0.0.11:8080", "10.0.1.10:8080"]
upstream_weights = [2, 2, 1]
backup_upstreams = ["10.0.1.10:8080"]drain_upstreams and disabled_upstreams also contain configured addresses,
not aliases. They prevent new selection; they do not kill requests already in
flight. Backup, drain and disabled lists cannot overlap. Keep at least one
normal primary member in the static configuration. Use
admin member controls for temporary maintenance instead of
repeatedly editing the configuration.
Active checks run independently of customer requests. They are enabled by default, using TCP checks every second. A successful TCP connection proves reachability, not that an application or its database is ready.
| Protocol | What the monitor proves | Configuration example |
|---|---|---|
tcp |
Connection establishment, with the configured upstream TLS policy where applicable | Set protocol = "tcp"; remove HTTP matchers |
http |
HTTP status and optional headers, body strings and JSON scalars | HTTP recipe |
grpc |
Standard gRPC health service reports serving | Set protocol = "grpc", optionally grpc_service = "my.application.Service"; remove HTTP response matchers |
exec |
An explicitly allowlisted local helper exits successfully within its timeout | Exec monitor |
redis |
Redis responds to a PING probe | Redis monitor |
mysql |
MySQL/MariaDB handshake is recognizable | MySQL monitor |
postgres |
PostgreSQL responds to the pre-authentication SSLRequest probe | PostgreSQL monitor |
Database monitors do not log in, run application queries or prove schema health.
The Redis/MySQL/PostgreSQL monitors currently reject upstream_tls = true;
their probes are not TLS-enabled database clients.
They monitor a pool; they do not turn an HTTP proxy into a database proxy. Use
the appropriate TCP stream route for database traffic.
The advanced HTTP recipe expects /healthz to return status 200, header
x-health-state: ready, and a body such as:
{"status":"ready","checks":{"database":true}}Status values/ranges are alternatives; the other configured matchers must all pass. With both status lists empty, exactly 200 is accepted. The simple Python origins above do not send the extra header, so use the base recipe for them or implement the advanced contract in your application.
Increase consecutive_failure to avoid removing an origin for one transient
failure, and consecutive_success to require stable recovery. parallel = true
probes members concurrently for supported protocols. Set timeouts below the
operational detection delay you can tolerate; a shorter interval alone cannot
make a slow probe finish sooner.
X-Health-Weight on an HTTP health response can reduce effective member weight,
bounded by health_weight_min_percent. It does not bypass failed readiness
checks. Only let trusted application instances supply this signal.
TCP, HTTP and gRPC monitors accept port_override. It changes only the probe
port: each member retains its own IP address and application port for customer
traffic, discovery, health state and administrative controls. For example,
members at 10.0.0.10:9001 and 10.0.0.11:9001 can expose readiness at port 9100:
[proxy.load_balance.health_check]
enabled = true
protocol = "tcp"
port_override = 9100
interval_secs = 1
consecutive_success = 1
consecutive_failure = 1
connect_timeout_secs = 1The probes connect to 10.0.0.10:9100 and 10.0.0.11:9100; application requests
still go to port 9001. See the copyable TCP recipe.
Omit port_override to probe each member's application port. The override is
one common port per pool, not a separate destination address. Readiness failure
removes the original member from selection and recovery returns that member.
With upstream_tls = true, TCP probes still perform a verified TLS handshake
using the configured upstream SNI and the health client's trusted CA roots.
The readiness port must therefore speak TLS with an appropriately trusted
certificate for that name; changing the port does not disable verification.
A successful TCP/TLS probe does not prove HTTP application readiness. Choose
an HTTP or gRPC monitor when that is required. Exec and database monitors do
not accept port_override.
Exec helpers run as the Fluxheim service identity, without a shell and with a
restricted environment. They receive backend address/host/port through
FLUXHEIM_HEALTH_BACKEND_ADDR, FLUXHEIM_HEALTH_BACKEND_HOST and
FLUXHEIM_HEALTH_BACKEND_PORT. They are trusted local programs, not sandboxed
plugins. Keep secrets out of arguments; helpers should read protected files.
Do not depend on timeout handling to contain arbitrary helper descendants.
By default, every health probe opens a fresh connection. To reuse connections, add this to an HTTP or gRPC health-check table:
[proxy.load_balance.health_check]
enabled = true
protocol = "http" # Or "grpc" for the standard gRPC Health service.
path = "/healthz"
interval_secs = 5
consecutive_success = 2
consecutive_failure = 3
reuse_connection = true
connect_timeout_secs = 1
read_timeout_secs = 2HTTP uses HTTP/1.1 keep-alive; gRPC uses successive streams on one HTTP/2
connection. TLS certificate and hostname verification remain enabled when the
upstream uses TLS. Reuse is not supported for TCP, exec or database probes;
those protocols reject reuse_connection = true.
Each pool keeps at most one connection or in-flight probe per backend, with a
maximum of 256 slots. Different backends, pools and TLS configurations do not
share connections, even when a port_override points them at the same readiness
listener. Concurrent probes for the same backend fail admission instead of
opening extra sockets or accumulating waiters. parallel = true can still probe
different members concurrently.
Idle connections expire after 30 seconds; connections older than five minutes
are retired before reuse. Idle cleanup runs once per second. A long probe
interval may therefore prevent reuse. The connection is returned only after the
complete bounded response, including trailers and configured health matchers,
passes validation. HTTP HEAD responses do not require a body. Response bodies
are limited to 64 KiB and response header/trailer parsing is bounded.
Do not set Content-Length in request_headers with reuse enabled; the client
owns request framing and rejects that override.
A timeout, malformed/incomplete response, failed health result, cancellation or
server-requested closure discards the connection. Removed/retargeted members
lose their cached connections; replacing the monitor configuration creates a
separate pool, and shutdown closes the pool. Reusable exchanges have one
read_timeout_secs deadline covering request write, response and trailers,
rather than restarting that deadline for every body fragment.
There is no automatic resend after an exchange fails. An idle peer close or
HTTP/2 GOAWAY can therefore cause one failed scheduled probe before a subsequent
probe reconnects. Use suitable consecutive-failure thresholds rather than
assuming every transport interruption should immediately remove a member.
Setting reuse_connection = false preserves fresh-per-probe behavior.
Passive checks observe actual transport failures and configured error responses.
After enough consecutive failures, the member is temporarily ejected. This is
separate from the active probe: a passing /healthz does not imply every request
works. Use queue-passive.toml
as a starting point.
With no explicit passive status list/ranges, all 5xx responses count as failures.
Specify failure_statuses or failure_status_ranges to narrow this. The minimum
healthy-member setting affects selection during ejection; it is not a promise
that the application always has that many healthy servers. In status, an open
circuit means passive ejection, not a separate configurable circuit-breaker.
Slow start gradually introduces newly seen and passively recovered members. It is a selection ramp, not a hard requests-per-second limit; fallback can bypass the ramp when otherwise no member is available.
Queueing is disabled by default. Set both max_waiting and timeout_ms to
positive values to wait briefly when no member can be selected. The queue is
bounded per pool and retries selection at retry_interval_ms. It does not
reserve a backend permit while waiting, guarantee FIFO fairness, or replace
application capacity planning. If admission/waiting fails, the first selection
returns all_down_status (502 by default; the example uses 503).
The unreleased 1.9.0 native HTTP pool dispatcher applies
[proxy.load_balance.retry] to ordinary requests, foreground revalidation,
background stale-while-revalidate and origin slice fills. This is a behavior
change: disabled (the default) now means one application attempt per origin
fetch, with no pool
failover or hidden HTTP/1 connection-pool/HTTP/2 request replay. Enable retries
explicitly when your service needs them:
[proxy.load_balance.retry]
enabled = true
max_retries = 1
methods = ["GET", "HEAD", "OPTIONS"]
statuses = [503]
budget_per_window = 100
budget_window_secs = 10The initial attempt is not a retry. max_retries = 1 allows at most two
application sends per origin fetch, and 0 allows only the initial send. Only configured safe
methods with an empty body, no trailers and no upgrade request can be replayed.
WebSocket/CONNECT tunnels are not retried. A transport failure does not prove
that the origin never received the request; safe-method application handlers
must genuinely tolerate replay.
Transport errors are eligible when retries are enabled. A received HTTP response
is eligible only when its status matches statuses or an inclusive
status_ranges entry. Each backend address is attempted at most once per
dispatch. Selection still respects health, capacity, priority and persistence,
but persistence cannot send a retry back to an already attempted member. If no
alternative is selectable, Fluxheim stops and handles the last response/error
normally, including stale-if-error and configured error pages. Only the initial
selection may enter the admission queue; retries do not wait for another slot.
budget_per_window limits retry admissions shared by the same pool's requests
and cache workers. When enabled = true and max_retries > 0, you must explicitly
set a value from 1 to 1000000; an omitted or zero budget is rejected. Runtime
admission also denies all retries for a zero budget if a programmatic caller
bypasses configuration validation. Disabled retries and max_retries = 0 may
retain the default zero budget. Budget
windows use a monotonic clock and reset when the next reservation observes an expired
window. A new pool/configuration rebuild starts a new budget; this is not a
process-wide, cluster-wide or restart-persistent limit. Separately constructed
pools have separate budgets, so account for all pools when sizing aggregate
origin capacity. There is no global retry semaphore or token bucket. Prefer a
small budget during origin failures and combine it with admission/in-flight
limits; a large explicitly configured budget can still amplify traffic.
Unused reservations are returned.
An admitted send remains charged if it subsequently fails or is cancelled.
Status exposes the cumulative retry.admitted count for that pool instance;
ordinary request telemetry records each application attempt separately.
Responses remain buffered and bounded by the configured upstream limits before
downstream delivery. A slice fetch keeps its tighter slice-body cap on every
attempt. Multiple slices and a full-response fallback after an unsuccessful
slice fetch are separate origin fetches. Their initial sends do not consume
retry budget, but their retries share the same pool budget: fallback does not
reset it. Consequently, max_retries is not a bound on all origin requests
needed to assemble one downstream ranged response; also configure slice limits.
Use connection/read/write timeouts and in-flight caps as well: the retry count
does not impose a new end-to-end wall-clock deadline. Protocol negotiation
(including an h2c probe) is distinct from replaying the application request.
These settings apply to a native HTTP load-balancer pool (multiple configured members or discovery), not to a single static upstream, static-proxy-only builds, PHP/FastCGI or raw TCP/UDP. See the complete retry fragment.
Native HTTP/1.1 and HTTP/2 measure response-header wait with a monotonic clock: from handing the completed outbound request to the upstream transport until the final response headers arrive. Informational responses do not end the measurement. Connection/TLS setup, pool/queue waits, request upload, response-body transfer, downstream backpressure and WebSocket tunnel lifetime are excluded. This is neither end-to-end latency nor proof that the origin application processed the request within that time.
A sample is published only after the bounded upstream response completes successfully. A complete 4xx/5xx response still supplies a sample; its status may also count as a passive failure. Transport/protocol/body failures count through the existing failure policy without a latency sample. Cancellation supplies neither a sample nor an invented failure and releases the connection permit. Each retry reports only its own attempt. Cache hits supply no samples; revalidation and slice origin fetches do. Active probes and upgraded tunnels do not train this estimator. A successful response is not retried solely because its latency crossed the passive threshold.
least-time stores a per-member microsecond EWMA (75% previous, 25% newest,
rounded to nearest) and compares latency divided by effective weight, using
weighted in-flight requests to break ties. Samples are clamped to at least one
microsecond. An eligible member without a sample is tried before sampled
members. After 30 seconds without a completed sample, its selection score becomes
unknown so ordinary traffic can try it again; the next sample replaces its stale
EWMA. Resampling needs traffic, remains subject to health, priority, affinity,
backup and capacity rules, and is not a dedicated background probe. Concurrent
requests can select the same unsampled member; use per-member in-flight caps.
For example, merge this into an existing pool:
[proxy.load_balance]
selection = "least-time"
[proxy.load_balance.passive_health]
enabled = true
max_latency_ms = 500
consecutive_failure = 3
ejection_secs = 30
min_healthy_backends = 1A completed sample at or above 500 ms counts as a failure. Setting
max_latency_ms = 0 disables latency classification, not status/transport
failure handling. The same latency threshold works with other selectors too.
Existing minimum-healthy and strict-health/fail-open rules still govern whether
an ejected member can be selected. Do not tune this threshold using full-body
download time. Parsing a config is not live proof or release acceptance evidence.
Persistence maintains a bounded local table of keys and member assignments. It still respects backend eligibility; it cannot force requests onto a dead or drained member. Choose one mode:
[proxy.load_balance.persistence]
enabled = true
mode = "source-ip"
ttl_secs = 300
table_max_entries = 4096source-ip: convenient, but many clients behind NAT can share one key.header: change mode to"header"and addheader = "x-session".cookie: change mode to"cookie"and addcookie = "session"; the application must already provide the request cookie.managed-cookie: Fluxheim issues its own affinity cookie. Use managed-cookie.toml and set its domain to your actual site, or omit the domain for a host-only cookie.
Remove the previous mode's header/cookie fields when switching. Missing or
unusable keys fall back to normal selection. Table entries are not authenticated
user sessions; least-sessions measures these routing entries only.
Managed cookies default to Secure, HttpOnly and SameSite=Lax. Use HTTPS in a browser; the loopback HTTP demo is not a browser cookie deployment recipe. SameSite=None requires Secure. The signing keys are process-local and rotate; they are not shared across nodes or restored by the runtime state file. A restart or another node can therefore invalidate an affinity cookie. Persistence is unavailable in privacy-mode builds.
Select exactly one source. Use host:port authorities, not upstream URLs.
File: replace the proxy block with discovery-file.toml. Its backends.txt uses one address per line and supports comments. Keep the file and its parents trusted and readable by the service user. Relative file paths are resolved from the configuration.
DNS: discovery-dns.toml
periodically resolves configured service names. Set
upstream_dns_allow_private_backends = true only when private application
networks are intentional. Without a refresh interval, hostname resolution does
not provide periodic DNS discovery.
HTTP control plane: discovery-http.toml polls a numeric-loopback test service. A real remote service must use HTTPS and return HTTP 200 with an address array:
{"upstreams":["10.0.0.10:8080","10.0.0.11:8080"]}A bare array is also accepted. Send Content-Type: application/json; if a
content type is supplied, it must be JSON-compatible. Responses are bounded to
64 KiB, redirects are not followed, and fetches have a five-second timeout.
Private backend addresses require the explicit allow-private flag. Protect any
bearer-token file and use HTTPS for secrets. Do not publish this discovery API
without authorization and independent control-plane protection.
Discovered sets must have 2-64 distinct addresses. Invalid later refreshes keep the previous set; an unusable initial source cannot establish a working pool. Static parallel attributes (weights, aliases, tags, priorities, localities, caps and backup/drain/disabled lists) are incompatible with discovery. Maglev and nginx/Ketama static-ring modes also reject discovery. When file/HTTP pools use upstream TLS, configure the explicit shared server name and trust roots.
Enable a protected admin listener before using these routes. A control token is required; the observer token is not a load-balancer management credential. Do not expose an unprotected admin port to the Internet.
| Operation | API |
|---|---|
| Inspect pools and members | GET /_fluxheim/load-balancer/status |
| Drain, disable, force down, resume or restore a member | POST /_fluxheim/load-balancer/member-state |
| Override/reset weight | POST /_fluxheim/load-balancer/member-weight |
| Add, remove or retarget a runtime member | POST /_fluxheim/load-balancer/member-add, member-remove, member-update |
| Clear affinity entries | POST /_fluxheim/load-balancer/persistence/clear |
The admin reference gives query
parameters and examples, including vhost/route targeting. For a named vhost
app, member=app-a&state=drain stops new work on that member. Watch in-flight
requests reach zero before maintenance. Removal checks are best-effort rather
than a transaction that stops all concurrent requests.
Runtime weight overrides apply to round-robin, least-connections, least-sessions and least-time, not the hash/static-ring or power-of-two selectors. Runtime backend-set edits are for static mutable pools, not discovery or static-ring pools. They are not written back to the TOML file.
Optional runtime_state_file = "/var/lib/fluxheim/load-balancer/app.json" under
the pool's load_balance table saves supported member overrides and persistence
entries across restarts. It does not persist runtime backend-set edits or
managed-cookie signing keys and does not replicate state between servers. Header
and cookie persistence can write identifying values to disk: protect the file,
its directory and backups accordingly. Provision the private directory before
startup; do not use a shared writable temporary directory.
Pools with background health/discovery services impose process-upgrade reload constraints. Do not assume every pool edit is a live snapshot change. Use the configuration validation/reload classification and upgrade workflow for the installed platform.
Use metrics and tracing with admin status. Inspect member
health, disabled/drain/backup flags, effective weights, in-flight permits,
passive ejection, discovery refresh errors and persistence counts together.
latency_micros = null means no sample has been recorded by a least-time
estimator (or that this selector has no estimator); it never means zero latency.
The numeric value is the last EWMA, not a percentile or a freshness promise:
status retains an expired sample even while selection treats it as unknown.
| Symptom | Check |
|---|---|
| Startup says a module is not compiled | Use a profile containing proxy and load-balancer |
| All requests return 503/502 | Reachability, probe Host/path/status/body, upstream TLS/SNI, drain/disable flags, caps and queue saturation |
| Only one backend receives traffic | Affinity, priority/locality preferences, backup status, weights and health |
| Slow backend still selected | Check header wait versus body time, sample warm-up/30-second resampling, affinity, weights, priorities and minimum-healthy/fail-open policy |
| Configured 503 retries do not occur | Check pool/profile selection, enabled, method/body restrictions, remaining count/budget and eligible unattempted members |
| Cookie affinity disappears | Cookie domain/path/Secure rules, TTL/table capacity, process restart or another Fluxheim node |
| Discovery changes do not appear | Source validity, refresh errors, private-address policy and whether DNS refresh was enabled |
| Reload requires an upgrade | Background health/discovery service lifecycle, not necessarily invalid configuration |
For migration comparisons, see nginx, HAProxy and F5 examples. For test coverage and unqualified boundaries, see the documentation coverage ledger.