Skip to content

Latest commit

 

History

31 Commits

Folders and files

Repository files navigation

Zentinel

Audit logger agent

Writes one structured audit record per request, in JSON, CEF or LEEF, with PII redacted, from the request and response headers Zentinel sends it.

CI License: Apache-2.0 Agent protocol v2

Registry · Docs · All repositories

For teams that need an access trail a SIEM can ingest: the agent builds an audit event when the response headers arrive, redacts sensitive values (named headers and regex patterns such as emails, card numbers, SSNs, JWTs), and writes it to stdout, stderr, a rotating file, syslog or an HTTP webhook. It never blocks or changes a request; every decision it returns is "allow". Because of that, run it with failure-mode "open", so an unavailable logger does not take traffic down.

Quick start

zentinel bundle install audit-logger
zentinel-audit-logger-agent --socket /tmp/zentinel-audit-logger.sock

Without --config, the agent looks for audit-logger.yaml in the working directory and falls back to the defaults (JSON to stdout, redaction on).

system {
    worker-threads 0
}

listeners {
    listener "http" {
        address "0.0.0.0:8080"
        protocol "http"
    }
}

upstreams {
    upstream "backend" {
        target "127.0.0.1:3000"
    }
}

agents {
    agent "audit-logger" type="custom" {
        unix-socket "/tmp/zentinel-audit-logger.sock"
        events "request_headers" "response_headers"
        timeout-ms 100
    }
}

filters {
    filter "audit-logger" {
        type "agent"
        agent "audit-logger"
        failure-mode "open"
    }
}

routes {
    route "default" {
        matches { path-prefix "/" }
        upstream "backend"
        filters "audit-logger"
    }
}

The agent needs both request_headers and response_headers: the record is written on the response event, from the request cached on the first.

Install

Bundle (recommended)

zentinel bundle install audit-logger

Downloads the release binary pinned in Zentinel's bundle lock and installs it as zentinel-audit-logger-agent.

Warning

The published 0.4.0 release, which Zentinel 26.10_7's bundle lock pins, was not built from its tag (it reports no version at all): it speaks the removed v1 framing on the Unix socket, so the proxy never completes a handshake and, under failure-mode "closed", refuses every request (503), and nothing is audited. Use 0.5.0 or later, the first release built and smoke-tested by the shared release pipeline (below); Zentinel 26.10_8's bundle lock pins 0.5.0.

Release binaries

Tarballs for linux-x86_64, linux-aarch64 and darwin-aarch64 are attached to each release.

From source

cargo install --git does not work for this crate: it depends on a sibling checkout of zentinel-agent-rust-sdk by path. Clone both side by side:

git clone https://github.com/zentinelproxy/zentinel-agent-rust-sdk
git clone https://github.com/zentinelproxy/zentinel-agent-audit-logger
cd zentinel-agent-audit-logger
cargo build --release   # target/release/zentinel-audit-logger-agent

Building needs protoc (CI installs protobuf-compiler).

Configuration

Zentinel 26.10_8 and later send the agent block's config { } on every connection (releases up to 0.6.50 never did), but the agent has no handler for it: it acknowledges the block and applies nothing. Configure the agent with the flags and the YAML file below.

Option Default Meaning
-c, --config <PATH> audit-logger.yaml YAML configuration file. If it does not exist, the defaults are used.
-s, --socket <PATH> /tmp/zentinel-audit-logger.sock Unix socket the agent listens on.
--grpc-address <ADDR> unset Also serve gRPC on this address (for example 0.0.0.0:50051), in addition to the socket.
-L, --log-level <LEVEL> info trace, debug, info, warn or error.
--print-config Print the default configuration as YAML and exit.
--validate Parse the configuration file, print a summary and exit (fails if the file is missing).
-V, --version Print zentinel-audit-logger-agent <version> and exit.

A minimal configuration file:

format:
  format_type: json        # json, cef or leef
outputs:
  - type: stdout
  - type: file
    path: /var/log/zentinel/audit.log
    max_size: 104857600    # rotate at 100 MiB
    max_files: 10
redaction:
  patterns: [email, credit_card, ssn, phone, jwt]
  redact_fields: [password, user.ssn]
filters:
  - name: skip-health
    condition: { type: path_prefix, prefix: /health }
    action: exclude

Every key is optional; run zentinel-audit-logger-agent --print-config for the full default file.

Option Default Meaning
format.format_type json json, cef (ArcSight) or leef (QRadar).
format.pretty false Pretty-print JSON.
format.timestamp_format %Y-%m-%dT%H:%M:%S%.3fZ strftime format for the timestamp.
format.device_vendor, device_product, device_version Zentinel, AuditLogger, crate version CEF/LEEF header fields.
trusted_proxies empty IPs or CIDRs of proxies whose X-Forwarded-For / X-Real-IP are believed for client_ip. Only needed with Zentinel older than 0.6.48; newer versions resolve the client behind the listener's trusted-proxies themselves.
fields.* most on Booleans per field: correlation_id, timestamp, client_ip, method, path, query_string, protocol, host, status_code, duration_ms, upstream_duration_ms, route_id, upstream, user_id, session_id, agent_decisions, request_body_size, response_body_size (on); request_body, response_body (off).
fields.request_headers a list of ten common headers none, all, !include [..] or !exclude [..].
fields.response_headers none Same forms as request_headers.
fields.user_id_header, session_id_header x-user-id, x-session-id Headers the user and session IDs are read from.
fields.custom_fields {} Map of record field name to request header name.
outputs [{type: stdout}] List of destinations, see below.
redaction.enabled true Turn redaction on or off.
redaction.patterns email, credit_card, ssn, phone Built-in patterns: also ip_address, jwt, aws_key, api_key.
redaction.custom_patterns [] Entries of name, pattern (regex), optional replacement.
redaction.replacement [REDACTED] Text that replaces a match.
redaction.hash_original false Replace a match with its SHA-256 hex instead, so values stay correlatable.
redaction.redact_headers authorization, cookie, set-cookie, x-api-key, x-auth-token Headers whose value is always replaced.
redaction.redact_fields [] JSON body fields to redact, dot notation (user.email).
body.* bodies off, max_body_size 4096 log_request_body, log_response_body, max_body_size, content_types, truncation_indicator. See the limitation below.
compliance_template unset soc2, hipaa, pci or gdpr: presets applied over the file at startup.
sample_rate 1.0 Fraction of requests to log, 0.0 to 1.0.
filters [] Ordered rules of name, condition and action; the first match decides.

Outputs (type selects the variant):

Option Default Meaning
type: stdout / type: stderr One line per record.
type: file with path, max_size, max_files no rotation; max_files 5 when rotating Appends to path; with max_size (bytes) set, rotates to path.1 ... path.N.
type: syslog with address, protocol, facility all required host:port, udp or tcp, and a facility such as local0 or auth. Severity is informational.
type: http with url, method, headers, batch_size, flush_interval_secs, timeout_secs, retries all required Webhook (POST or PUT) with retries and exponential backoff. Records are sent when batch_size is reached; flush_interval_secs is not used yet.

Filter conditions: method (values), path_prefix (prefix), path_regex (pattern), status_code (min, max), header (name, optional value) and all. Actions: include, exclude, verbose (currently the same as include).

Full reference: https://registry.zentinelproxy.io/audit-logger/

How it works

  • On request_headers the agent notes the start time, keyed by the request's x-correlation-id or x-request-id header; on response_headers it applies sampling and filters, builds the record and writes it to every output. An output that fails is logged and skipped; the others still receive the record.
  • duration_ms is only filled when the request carries one of those two headers. Client IP is the connection address Zentinel reports (unknown if it has none); X-Forwarded-For / X-Real-IP are only read when that address is in trusted_proxies, taking the right-most hop that is not a trusted proxy. Route, upstream, upstream duration and agent decisions come from the x-zentinel-route-id, x-zentinel-upstream, x-zentinel-upstream-duration-ms and x-zentinel-agent-decisions headers when they are present.
  • Pattern redaction applies to header values and logged bodies; the path and query string are logged as received.
  • Body logging (partial): the body settings and the request_body / response_body fields exist, but the record is built from header events only, so no body content reaches the log today.
  • Kafka output (not implemented): the kafka-output cargo feature adds a type: kafka variant, which falls back to stdout with a warning.

Development

git clone https://github.com/zentinelproxy/zentinel-agent-rust-sdk ../zentinel-agent-rust-sdk
cargo build
cargo test --workspace
cargo clippy --workspace --all-targets -- -D warnings
cargo fmt --all --check

Stable Rust (CI uses dtolnay/rust-toolchain@stable) and protoc. The HTTP output is the default http-output feature; build with --no-default-features to drop it.

Releases and smoke test

Releases go through the shared agent pipeline (zentinelproxy/.github): pushing a tag vX.Y.Z that matches Cargo.toml builds zentinel-audit-logger-agent with --locked for linux-x86_64, linux-aarch64 and darwin-aarch64, checks that zentinel-audit-logger-agent --version prints zentinel-audit-logger-agent X.Y.Z, runs scripts/smoke-test.sh behind the latest Zentinel release on every platform, and only then publishes the tarballs zentinel bundle install downloads. CI runs the same pipeline on linux-x86_64 for every push and pull request.

The smoke test runs the release binary behind a real Zentinel with the agent, filter and route all failure-mode "closed", over the Unix socket (the transport zentinel bundle install uses) and gRPC, subscribed to request and response headers. It checks that the agent starts without a config file, that requests (GET with a query string, POST) are forwarded, that each one is written to the configured audit file with its method, path, query string and status, that the client's Authorization header does not appear in the file unredacted, and that the request is refused once the agent is stopped (so the agent really is on the request path).

Run it locally:

gh release download 26.10_8 --repo zentinelproxy/zentinel --pattern 'zentinel-26.10_8-darwin-arm64.tar.gz*'
shasum -a 256 -c zentinel-26.10_8-darwin-arm64.tar.gz.sha256 && tar xzf zentinel-26.10_8-darwin-arm64.tar.gz
cargo build --release --locked
AGENT_SMOKE=../.github/agent-smoke/smoke.py scripts/smoke-test.sh target/release/zentinel-audit-logger-agent ./zentinel

Related

License

Apache-2.0, see LICENSE.


Part of Zentinel, the security-first reverse proxy. The shisa guards the gate.

Releases

Packages

Contributors

Languages