diff --git a/src/content/docs/explanations/packages.mdx b/src/content/docs/explanations/packages.mdx index 3aef25352..0d816765d 100644 --- a/src/content/docs/explanations/packages.mdx +++ b/src/content/docs/explanations/packages.mdx @@ -36,6 +36,7 @@ A package is a directory with the following structure: - pipelines/ Fully deployable pipelines - tests/ Integration tests - inputs/ Sample data for tests + - constants.tql Package-level constants - package.yaml Package manifest @@ -77,6 +78,14 @@ remains predictable. The node creates these contexts when you install the package, making them available for lookup and enrichment operations. +### Constants + +**Constants** in the `constants.tql` file are package-level `let` bindings that +Tenzir evaluates once when it loads the package. Reference them as `pkg::$name` +from the package's own operators and pipelines or from any external pipeline. +Use them to share lookup tables, enumerations, and fixed thresholds that would +otherwise be duplicated across files. + ### Examples **Examples** in the `examples` directory demonstrate how to use the package. @@ -134,7 +143,7 @@ straightforward, and good test coverage enables confident iteration. ## Package lifecycle 1. **Create**: Set up the package structure and manifest -2. **Develop**: Add operators, pipelines, contexts, and examples +2. **Develop**: Add operators, pipelines, contexts, constants, and examples 3. **Test**: Validate behavior with the Test Framework 4. **Install**: Deploy locally or through the Tenzir Library 5. **Maintain**: Update the changelog and publish releases @@ -146,5 +155,6 @@ straightforward, and good test coverage enables confident iteration. - packages/add-operators - packages/add-pipelines - packages/add-contexts +- packages/add-constants - packages/maintain-a-changelog - write-a-package diff --git a/src/content/docs/guides/packages/add-constants.mdx b/src/content/docs/guides/packages/add-constants.mdx new file mode 100644 index 000000000..c761c762e --- /dev/null +++ b/src/content/docs/guides/packages/add-constants.mdx @@ -0,0 +1,105 @@ +--- +title: Add constants +--- + +This guide shows you how to define package-wide constants in a `constants.tql` +file and reference them as `pkg::$name` from the package's own operators and +pipelines, as well as from any pipeline that uses the package. You'll learn the +`let` syntax, how bindings build on one another, the rules each binding must +satisfy, and when to use a constant instead of an input. + +## Define constants + +Place a `constants.tql` file at the root of your package. Each `let` binding +defines a named constant that Tenzir evaluates once when it loads the package: + +```tql title="constants.tql" +let $threshold = 8 +let $severities = {low: 2, medium: 3, high: 4, critical: 5} +``` + +A constant can be any value—a number, string, list, or record—which makes +`constants.tql` a natural home for the lookup tables and magic numbers that +would otherwise be copy-pasted across your operators. + +## Reference constants + +Reference a constant from anywhere with `::$name`, using your package's +ID as the prefix. In a package with ID `acme`, the bindings above become +`acme::$threshold` and `acme::$severities`. + +### From the package's own operators + +Inside the package, operators and pipelines reference constants so the same +value lives in exactly one place: + +```tql title="operators/ocsf/map.tql" +ocsf.severity_id = acme::$severities[severity] +``` + +### From external pipelines + +Once the `acme` package is available, any pipeline can reference its constants: + +```tql +from {severity: 9}, {severity: 3} +where severity >= acme::$threshold +``` + +```tql +{severity: 9} +``` + +This lets a package publish named thresholds and enumerations that consumers +reuse instead of hardcoding their own copies. + +## Build on earlier constants + +A binding may reference any constant declared before it, so you can derive one +constant from another: + +```tql title="constants.tql" +let $high_severity = 8 +let $threshold = $high_severity + 1 +``` + +References resolve in order. A binding that refers to a constant declared later +in the file fails to load the package. + +## Binding rules + +Each binding must evaluate to a deterministic constant value, because Tenzir +computes it once when the package loads and folds the result into every +reference. Tenzir rejects a binding that is: + +- **Non-deterministic**, such as `now()` or `random()`. +- A **pipeline** rather than a value. +- A **function** (lambda). +- A **duplicate name**, since all constants share one flat namespace. + +When a binding violates these rules, the package fails to load with a diagnostic +that points at the offending `let`. + +## Constants versus inputs + +Constants and [inputs](/guides/packages/configure-inputs) both parameterize a +package, but they sit on opposite sides of the install boundary: + +| | Constants | Inputs | +| ------------- | ---------------------------------------------------- | ----------------------------------------------------------------------- | +| Defined in | `constants.tql` | `package.yaml` | +| Set by | the package author (fixed) | the person installing the package | +| Referenced as | `pkg::$name` | `{{ inputs.name }}` | +| Resolved | const-evaluated when the package loads | substituted at install time | +| Use for | shared lookup tables, enumerations, fixed thresholds | endpoints, credentials, intervals, and other deployment-specific values | + +Reach for a constant when the package author owns the value and wants to share +it. Reach for an input when the person installing the package must supply it. + +## See also + +- packages/add-operators +- packages/configure-inputs +- packages/create-a-package +- write-a-package +- packages diff --git a/src/content/docs/guides/packages/add-operators.mdx b/src/content/docs/guides/packages/add-operators.mdx index 2f5a23995..b1820626c 100644 --- a/src/content/docs/guides/packages/add-operators.mdx +++ b/src/content/docs/guides/packages/add-operators.mdx @@ -582,5 +582,6 @@ When building operator hierarchies, follow these guidelines: - packages/create-a-package - packages/add-pipelines +- packages/add-constants - packages/test-packages - write-a-package diff --git a/src/content/docs/guides/packages/configure-inputs.mdx b/src/content/docs/guides/packages/configure-inputs.mdx index b795642ab..544a08be4 100644 --- a/src/content/docs/guides/packages/configure-inputs.mdx +++ b/src/content/docs/guides/packages/configure-inputs.mdx @@ -241,4 +241,5 @@ inputs: - packages/create-a-package - packages/install-a-package +- packages/add-constants - write-a-package diff --git a/src/content/docs/guides/packages/create-a-package.mdx b/src/content/docs/guides/packages/create-a-package.mdx index 88a223733..57c0d650e 100644 --- a/src/content/docs/guides/packages/create-a-package.mdx +++ b/src/content/docs/guides/packages/create-a-package.mdx @@ -160,6 +160,7 @@ usage details. - packages/configure-inputs - packages/add-operators - packages/add-pipelines +- packages/add-constants - packages/maintain-a-changelog - write-a-package - packages diff --git a/src/sidebar.ts b/src/sidebar.ts index 4ca95cf3c..98455cb59 100644 --- a/src/sidebar.ts +++ b/src/sidebar.ts @@ -198,6 +198,7 @@ export const guides = [ "guides/packages/add-operators", "guides/packages/add-pipelines", "guides/packages/add-contexts", + "guides/packages/add-constants", "guides/packages/configure-inputs", "guides/packages/maintain-a-changelog", "guides/packages/publish-a-package",