Skip to content

ci_pipelines.md

Latest commit

 

History

History
269 lines (210 loc) · 22 KB

ci_pipelines.md

File metadata and controls

269 lines (210 loc) · 22 KB
Raw

CI integrations pipelines

This section describes the CI pipelines available in this repository.

The following pipelines are available in this repository:

Pull Requests and pushes to specific branches

In every push to a Pull Request as well as commits to main, backport-*, or feature/* branches, this pipeline is triggered automatically: https://buildkite.com/elastic/integrations

This pipeline is in charge of testing the packages with a local Elastic stack.

Special comments that can be added in the Pull Request (by Elastic employees):

  • /test or buildkite test this: a new build is triggered.
  • /test benchmark fullreport: a new build is triggered creating a full benchmark report (it will be posted as a GitHub comment in the PR).
  • /test stack <version>: a new build is triggered in integration-test-stack pipeline where the packages updated in the PR will be tested using the Elastic stack version <version> set in the comment.
    • The new build will appear as another check status (not required).
    • Examples:
      • /test stack 8.17.0
      • /test stack 8.18.0-SNAPSHOT
      • /test stack 9.0.0-SNAPSHOT

Changelog link check (check-changelog-pr-links): on pull requests that modify a changelog.yml file, a step validates that new or modified changelog entries link to the current PR rather than a different one (soft_fail: true on link mismatches — a failure posts an annotation but does not block merge, unless the entry still contains a REPLACE_ME placeholder). To skip this check on a PR, add the label changelog-link-check:skip. This label is added automatically by the sync-backport-changelog workflow on changelog sync PRs, because their entries intentionally link to the original backport PR rather than the sync PR itself.

There are some environment variables that can be added into this pipeline to enable customizations:

  • FORCE_CHECK_ALL: If true, this forces the CI to check all packages even if those packages have no file updated/added/deleted. Default: false.
  • STACK_VERSION: Force the CI steps to spin up a specific Elastic stack version to run the tests. Default: unset.
    • If this variable is set, the packages tested are the ones that support that version according to the constraint set in their manifest (conditions.kibana.version).
    • Examples:
      STACK_VERSION Kibana constraint Stack version used for testing
      7.17.28 ^7.17.0 || ^8.15.0 || ^9.0.0 ^7.17.28
      7.17.28 ^^8.15.0 || ^9.0.0 Skip
      8.17.0-SNAPSHOT ^8.18.0 || ^9.0.0 Skip
      8.19.0-SNAPSHOT ^8.15.0 || ^9.0.0 ^8.19.0-SNAPSHOT
      9.1.0-SNAPSHOT ^8.15.0 || ^9.0.0 ^9.1.0-SNAPSHOT
  • STACK_LOGSDB_ENABLED: Enable LogsDB setting in Elasticsearch service. Default: false.
    • If this variable is set and STACK_VERSION is unset, the CI will ensure that the stack version running will be either the version used in PR builds or 8.17.0 (the GA version for LogsDB index mode), whichever is higher.
    • Examples:
      Kibana constraint Stack version used for testing
      ^8.15.0 || ^9.0.0 ^8.17.0
      ^8.18.0 || ^9.0.0 ^8.18.0
      ^7.18.0 Skip
  • ELASTIC_SUBSCRIPTION: Set the subscription to be used in the Elastic stack (basic or trial). Default: trial.
  • PUBLISH_COVERAGE_REPORTS: If true, it enables reporting coverage reports. Currently, it is just set for the build triggered with the current major Elastic stack from the daily job. Default: false.

Apart from these environment variables, these other environment variables related to elastic-package can be set too for further customizations.

These environment variables can be defined in different locations:

More details about this CI pipeline:

  • Builds running in a Pull Request are canceled if a new commit is pushed to the PR branch.

  • Builds running from main, backport-*, or feature/* branches are finished even if new commits are merged.

  • As part of the Pull Requests, there are some benchmark files created and this data is pushed to the PR as a comment.

  • This CI pipeline tries to test the minimum packages possible::

    • In the Pull Request context, it is checked the files modified/added/deleted in the given PR:
      • Those packages with changes (packages/*) will be added to the list of packages to be tested.
      • If files outside packages folder are updated (for example go.mod , .buildkite/*), all packages are going to be tested. There are some exceptions to this, for instance the .github/CODEOWNERS file or .docs/ folder (files excluded).
    • In branches context (main, backport-*, or feature/*):
      • The latest Buildkite build that finished successfully in that branch is retrieved, and all the file changes in the working copy between the changeset of that build and the merged commit are obtained.
      • Given all those changes, the packages selected to be tested follow the same rules as in the PR.
      • First push (no previous successful build):
        • For backport-* and feature/* branches: the merge-base between the pushed commit and origin/main is computed to find commits exclusive to the branch. If those commits contain changes under packages/, the affected packages are tested. If only CI infrastructure files changed (for example .buildkite/, dev/), no package tests are run.
        • For main: diffs against origin/main^ (the previous commit on main).

  • Container logs, as they could contain sensitive information, are uploaded to a private Google Bucket.

  • Packages are tested running the Elastic stack with the minimum Kibana version supported according to their manifest (.conditions.kibana.version). If a package defines a Kibana version that is not released yet, elastic-package will be using the SNAPSHOT version. This can be overridden if the STACK_VERSION variable is defined in the environment. In the following table, there are some examples:

    STACK_VERSION env Kibana Condition Package Released Elastic stack run
    "" ^7.16.0 || ^8.0.0 Yes 7.16.0
    "" ^8.12.0 Yes 8.12.0
    "" ^8.14.1 Yes 8.14.1
    "" ^8.15.0 No 8.15.0-SNAPSHOT
    8.14.0 ^8.13.0 Yes 8.14.0

    If the STACK_VERSION environment variable is defined, just the packages supporting that stack version are tested. For instance:

    STACK_VERSION env Kibana Condition Package Tested
    7.17.0 ^7.16.0 || ^8.0.0 Yes
    7.17.0 ^8.12.0 No
    8.12.0 ^8.13.0 No
    8.14.0 ^8.13.0 Yes

Publish packages

Note: Available only to Elastic employees.

For every Pull Request merged onto the main branch or any backport-* branches, this pipeline is triggered automatically https://buildkite.com/elastic/integrations-publish to publish new versions of the packages if any. These new versions are published and made available in https://epr.elastic.co.

Environment variables that can be defined in this pipeline:

  • DRY_RUN: If true, packages will not be published. Default: false.
  • SKIP_PUBLISHING: If true, not run at all publish procedures. Default: false.

These environment variables can be defined:

Serverless Pipeline

Note: Available only to Elastic employees.

Pipeline in charge of testing all packages with an Elastic Serverless project (https://buildkite.com/elastic/integrations-serverless).

This pipeline follows these steps:

  1. Create an Elastic Serverless project of a given type. Currently supported: Observability or Security
  2. For each package in the repository:
    1. Check if the package is supported in the given Elastic Serverless project.
      1. Packages can define which capabilities they need. Depending on those, the package could be available or not in the Serverless project, and therefore they will be tested or not.
      2. There are some packages that are excluded in the Kibana configuration explicitly:
    2. Runs test for all the selected packages:
      • Currently, system tests are not run in Serverless.
      • As in the pipeline for Pull Requests, container logs as they could contain sensitive information, they are uploaded to a private Google Bucket.
  3. Deletes the Elastic Serverless project.

Environment variables that can be defined in this pipeline:

  • SERVERLESS_PROJECT: Serverless project to be created to test packages. Default: observability.

This environment variable can be defined at:

Daily job

Note: Available only to Elastic employees.

On weekdays (Monday to Friday) it is configured to run a daily job that will be in charge of testing all packages with different scenarios: https://buildkite.com/elastic/integrations-schedule-daily

The schedules of this job are defined in the schedules section of the integrations-schedule-daily pipeline in catalog-info.yaml.

In these daily jobs, the environment variable FORCE_CHECK_ALL is set to true ensuring that all packages that fulfill all the requirements set in the pipeline are tested. Those other requirements are related to setting STACK_VERSION, STACK_LOGSDB_ENABLED, etc... or other environment variables. More details about these environments variables here.

The scenarios that are tested in this daily job testing all the affected packages are:

Those tests that have failed in these scenarios will be reported as GitHub issues notifying the owner teams as defined in .github/CODEOWNERS file.

As part of this pipeline, it is also ensured that the latest versions of the packages merged into main branch have been published by triggering the pipeline https://buildkite.com/elastic/integrations-publish.

Each step triggering a new pipeline can be customized through environment variables. Environment variables that can be used in each pipeline are detailed in the corresponding sections of each pipeline.

Weekly job

Note:

  • Available only to Elastic employees.
  • This pipeline is currently disabled.

Every week it is configured to run a job that will be in charge of testing all packages with non-Wolfi Elastic Agent docker images: https://buildkite.com/elastic/integrations-schedule-weekly

The schedule of this job can be checked here.

The scenarios that are tested in this weekly job are:

  • Test packages with a local Elastic stack running the latest 8.x version of the stack with Elastic Agent images based on Ubuntu images:
  • Test packages with a local Elastic stack running the latest major version of the stack with Elastic Agent images based on non-Wolfi images:

Each step triggering a new pipeline can be customized through environment variables. Environment variables that can be used in each pipeline are detailed in the corresponding sections of each pipeline.

Backport branch creation pipelines

Note: Available only to Elastic employees.

Releasing hotfixes from earlier versions of packages requires creating backport-* branches from specific commits in the main branch. The pipeline https://buildkite.com/elastic/integrations-backport handles this creation and can be triggered in two ways:

  • Automatically (recommended): when a PR adding a new entry to .backports.yml is merged into main, the integrations-backport-dispatch pipeline detects the change and triggers integrations-backport to create the branch. A comment is posted on the merged PR reporting success or failure of the branch creation.
  • Manually from the UI: restricted to members of the ecosystem Buildkite team.

The integrations-backport-dispatch pipeline is responsible for triggering branch creation on pushes to main. This split from the main integrations pipeline exists because Buildkite restricts pipeline triggers in certain scenarios (e.g. a public pipeline triggering a private one) — a dedicated dispatch pipeline eliminates that failure mode. As a safeguard, trigger_backport.sh exits with an error if branch creation is attempted from any pipeline other than integrations-backport-dispatch.

As part of the PR that modifies .backports.yml, CI automatically:

  • Validates the new inventory schema (check-backports-inventory step). In the public integrations pipeline this step runs on PRs targeting main only — it is skipped on PRs targeting backport-* branches, which carry only a subset of packages and would fail the validation unnecessarily. On pushes to main, the integrations-backport-dispatch pipeline runs its own validation before triggering branch creation.
  • Runs a dry run of the branch creation (trigger-backport-dryrun step), verifying the commit exists and the branch does not already exist, without pushing anything.

When remove_other_packages: true is set in .backports.yml, the created branch contains the target package along with its requires.* dependencies and .link file source packages, transitively expanded — all unrelated packages in packages/ are removed to keep the branch lean.

On pull requests targeting a backport-* branch that modify a changelog.yml file, the check-changelog-versions-in-main step verifies that no changelog version introduced by the PR already exists on main, catching sync collisions before merge.

The following parameters can be configured when triggering manually from the UI:

Parameter Required Default Description
DRY_RUN true Commit the backport branch locally but skip push to the remote; prints a diff of the changes instead.
BASE_COMMIT ✅ Commit SHA to branch from (the output of step 1 in the backport guide).
PACKAGE_NAME ✅ Package name as defined in manifest.yml.
PACKAGE_VERSION ✅ Package version to branch from (e.g. 1.5.7, 1.0.0-beta1).
REMOVE_OTHER_PACKAGES true If true, the target package and its requires.* dependencies and .link file source packages, transitively expanded, are kept; all unrelated packages are removed from packages/.
BACKPORT_BRANCH_NAME auto Override the generated branch name (default: backport-<package>-<major>.<minor>).
PR_NUMBER PR number to notify on completion (posts a comment with success or failure).

More information about this pipeline and how to create these hotfixes in: https://www.elastic.co/guide/en/integrations-developer/current/developer-workflow-support-old-package.html

Feature branches

feature/* branches are long-lived branches used for developing larger features before merging into main. They follow the same CI process as Pull Requests, ensuring all checks pass before code is integrated.

  • Every push to a feature/* branch triggers the main CI pipeline (https://buildkite.com/elastic/integrations), running the same package tests as a regular Pull Request.
  • Pull Requests targeting a feature/* branch also trigger CI checks, so all changes must pass CI before being merged into the feature branch.
  • Intermediate builds are cancelled or skipped when new commits arrive, consistent with backport-* branches.
  • Changeset detection works the same as for backport-* branches: only packages modified since the last successful build are tested. On the first push to a new feature/* branch, the merge-base with origin/main is used to find commits exclusive to the branch — packages are tested only if those commits contain changes under packages/; if only CI infrastructure files changed, no package tests are run.
  • Publishing is not triggered for feature/* branches. Packages are published only when PRs are merged into main or backport-* branches.