diff --git a/charts/langsmith/scripts/mirror_langsmith_images.README.md b/charts/langsmith/scripts/mirror_langsmith_images.README.md new file mode 100644 index 00000000..2327381a --- /dev/null +++ b/charts/langsmith/scripts/mirror_langsmith_images.README.md @@ -0,0 +1,169 @@ +# Mirroring LangSmith self-hosted images + +`mirror_langsmith_images.sh` pulls every image the LangSmith self-hosted Helm chart deploys, +retags it, and pushes it to a private registry — for air-gapped clusters or environments that +must pull from an internal registry. + +It mirrors the **complete** set for a deployment that includes the add-ons (Fleet, Insights, +Polly/Chat) with in-cluster databases. + +## Quick start + +Always dry-run first to see exactly what would be pushed: + +```bash +./mirror_langsmith_images.sh --registry your-registry.example.com --dry-run +``` + +Then mirror for real (defaults to the latest GA app version): + +```bash +./mirror_langsmith_images.sh --registry your-registry.example.com +``` + +## Flags + +| Flag | Purpose | +|------|---------| +| `--registry` | **Required.** Destination registry prefix (e.g. `your-registry.example.com` or `123456789012.dkr.ecr.us-east-1.amazonaws.com`). | +| `--version` | App-image tag for all `langchain/*` LangSmith images. **This is the chart's `appVersion`, not the Helm chart version** (see below). Defaults to the latest GA app version. | +| `--operator-version` | Tag for `langgraph-operator`. Defaults to the value pinned by the current chart. | +| `--platform` | Architecture to pull (`linux/amd64` default, or `linux/arm64`). | +| `--dest-repo` | "Marketplace" mode: push every image into a single repo, tagged `-` (useful for registries like ECR that prefer one repo). | +| `--with-presidio` | Also mirror `mcr.microsoft.com/presidio-analyzer` (only needed if you enable PII redaction). | +| `--with-smithdb` | Also mirror `langchain/smithdb` (0.16+ only). | +| `--dry-run` | Print the `docker` commands without executing them. | + +## Examples + +```bash +# 1. Preview the GA image set without pushing +./mirror_langsmith_images.sh --registry your-registry.example.com --dry-run + +# 2. Mirror the latest GA release (app version 0.15.18) +./mirror_langsmith_images.sh --registry your-registry.example.com --version 0.15.18 + +# 3. Mirror a release candidate instead of GA +./mirror_langsmith_images.sh --registry your-registry.example.com \ + --version 0.16.13rc1 --operator-version 0.1.47 + +# 4. arm64 nodes +./mirror_langsmith_images.sh --registry your-registry.example.com \ + --version 0.15.18 --platform linux/arm64 + +# 5. Single-repo ("marketplace") mode, e.g. AWS ECR +./mirror_langsmith_images.sh --registry 123456789012.dkr.ecr.us-east-1.amazonaws.com \ + --dest-repo langchain/langsmith --version 0.15.18 + +# 6. Include the optional components (PII redaction + smithdb) +./mirror_langsmith_images.sh --registry your-registry.example.com \ + --version 0.16.13rc1 --operator-version 0.1.47 --with-presidio --with-smithdb +``` + +## What gets mirrored + +| Group | Images | +|-------|--------| +| Core | `langsmith-backend`, `langsmith-frontend`, `langsmith-go-backend`, `langsmith-ace-backend`, `langsmith-playground`, `hosted-langserve-backend`, `langgraph-operator` | +| Add-ons | `langsmith-clio` (Insights), `agent-builder-deep-agent` + `agent-builder-tool-server` + `agent-builder-trigger-server` (Fleet), `langsmith-polly` (Polly/Chat) | +| In-cluster dependencies | `postgres`, `redis`, `clickhouse-server`, `pgvector/pgvector` | +| Optional (flags) | `presidio-analyzer` (`--with-presidio`), `smithdb` (`--with-smithdb`) | + +## IMPORTANT: chart version vs. image tag + +There are **two different version numbers**, and mixing them up is the most common mistake: + +- **Helm chart version** — what the self-hosted changelog and GitHub releases show (e.g. `0.15.13`), + and what you pass to `helm upgrade --install … --version`. +- **App image tag (`appVersion`)** — the tag on the container images (e.g. `0.15.18`), and what this + script's `--version` expects. + +They increment independently. For example, chart **`0.15.13`** ships app images tagged **`0.15.18`**. +Passing the chart version (`0.15.13`) to `--version` would try to pull image tags that don't exist. + +| | Helm chart `--version` (helm) | App images (`--version` here) | Operator (`--operator-version`) | postgres | redis | clickhouse | pgvector | +|---|---|---|---|---|---|---|---| +| Latest GA | `0.15.13` | `0.15.18` | `0.1.47` | `14.7` | `7` | `25.12` | `pg15` | +| Latest RC | `0.16.0-rc.12` | `0.16.13rc1` | `0.1.47` | `14.7` | `7` | `25.12` | `pg15` | + +_(As of 2026-07-14. Always re-verify against the chart you install — see next section.)_ + +## Picking the right versions for a release (do this on every upgrade) + +The app-image tag follows `--version`, but the **dependency tags** (`postgres`, `redis`, `clickhouse`, +`pgvector`) are pinned by the chart release and are **independent of `--version`**. They're set as +constants at the top of the script. On the current GA and RC they're identical (`14.7` / `7` / `25.12` / +`pg15`), so moving between those only changes `--version`. A future release could bump one of them — so +when you adopt a new chart version, verify the tags against that chart and update the constants if needed. + +Read the exact tags a given chart version deploys straight from the chart source (replace `CHART_VER` +with the Helm chart version you're installing): + +```bash +CHART_VER=0.15.13 +BASE="https://raw.githubusercontent.com/langchain-ai/helm/langsmith-${CHART_VER}/charts/langsmith" + +# App-image tag to pass as --version (this is the chart's appVersion): +curl -s "$BASE/Chart.yaml" | grep '^appVersion:' + +# Operator + dependency tags (compare against the script's constants): +curl -s "$BASE/values.yaml" | grep -E '^ (operatorImage|postgresImage|redisImage|clickhouseImage):' -A3 \ + | grep -E 'Image:|tag:' + +# Operator-created database images (pgvector + inline redis): +curl -s "$BASE/values.yaml" | grep -E 'pgvector/pgvector:pg|image: docker.io/redis:' +``` + +(If you've added the LangSmith Helm repo, `helm show values --version ` gives the +same information.) + +Then set: +- `--version` = the `appVersion` value +- `--operator-version` = `operatorImage.tag` +- the script's `PG_TAG` / `REDIS_TAG` / `CLICKHOUSE_TAG` / `PGVECTOR_TAG` constants = the dependency tags, + **only if they changed** from what the script already has. + +## After mirroring: point the chart at your registry + +In your `values.yaml`: + +```yaml +images: + registry: "your-registry.example.com" + imagePullSecrets: + - name: langsmith-pull-secret # a docker-registry Secret in the install namespace + # For each image, strip the leading "docker.io/" from the repository so it isn't doubled, and pin the tag. + backendImage: { repository: "langchain/langsmith-backend", tag: "0.15.18" } + # …one entry per image… +``` + +**One thing `images.registry` does NOT cover:** the langgraph-operator creates a Redis and a Postgres +(`pgvector`) for each agent deployment from raw image strings in `operator.templates.redis` and +`operator.templates.db`. Those are not rewritten by `images.registry`, so override the image line in each: + +```yaml +operator: + templates: + redis: | # …change the image line to your-registry.example.com/redis:7 + db: | # …change the image line to your-registry.example.com/pgvector/pgvector:pg15 +``` + +The full template blocks (with a placeholder registry) are in the docs — see the reference below. + +## Optional: verify image signatures + +v15+ `docker.io/langchain/*` images are keyless Cosign-signed. You can verify before and after mirroring: + +```bash +cosign verify \ + --certificate-oidc-issuer https://token.actions.githubusercontent.com \ + --certificate-identity-regexp 'https://github\.com/langchain-ai/langchainplus/\.github/workflows/release_self_hosted_on_version_bump\.yaml@refs/heads/v[0-9]+-stable' \ + docker.io/langchain/langsmith-backend: +``` + +## References + +- Mirror images guide (incl. the full operator-template overrides and signature verification): + https://docs.langchain.com/langsmith/self-host-mirroring-images +- Self-hosted changelog (chart versions): https://docs.langchain.com/langsmith/self-hosted-changelog +- Kubernetes install guide: https://docs.langchain.com/langsmith/kubernetes diff --git a/charts/langsmith/scripts/mirror_langsmith_images.sh b/charts/langsmith/scripts/mirror_langsmith_images.sh index 30addf53..af40705b 100755 --- a/charts/langsmith/scripts/mirror_langsmith_images.sh +++ b/charts/langsmith/scripts/mirror_langsmith_images.sh @@ -8,15 +8,32 @@ # Marketplace mode (--dest-repo): retag as /:- # # Examples: -# ./mirror_langsmith_images.sh --registry myregistry --version 0.10.66 --platform linux/arm64 +# ./mirror_langsmith_images.sh --registry myregistry --version 0.15.18 --platform linux/amd64 +# Target a release candidate instead of the GA default: +# ./mirror_langsmith_images.sh --registry myregistry --version 0.16.13rc1 --operator-version 0.1.47 --platform linux/amd64 # ./mirror_langsmith_images.sh --registry 709825985650.dkr.ecr.us-east-1.amazonaws.com \ -# --dest-repo langchain/langchain-repository --version 0.10.67 --platform linux/amd64 --dry-run +# --dest-repo langchain/langchain-repository --version 0.15.18 --platform linux/amd64 --dry-run set -euo pipefail -# Default version -DEFAULT_VERSION="0.13.9" -DEFAULT_OPERATOR_VERSION="0.1.37" +############################################################################### +# Versions +############################################################################### +# LangSmith application images + operator. Bump these when targeting a new release. +DEFAULT_VERSION="0.15.18" # latest GA appVersion -> all langchain/* application images (override with --version, e.g. an RC) +DEFAULT_OPERATOR_VERSION="0.1.47" # langgraph-operator image tag (same in current GA and RC) + +# In-cluster dependency image tags. These are pinned by the chart (charts/langsmith/values.yaml) +# and are NOT tied to --version. Re-verify them against values.yaml whenever you bump the chart: +# postgresImage.tag, redisImage.tag, clickhouseImage.tag, and the operator.templates pgvector tag. +PG_TAG="14.7" +REDIS_TAG="7" +CLICKHOUSE_TAG="25.12" +PGVECTOR_TAG="pg15" + +# Optional components (off by default; enable with --with-presidio / --with-smithdb). +PRESIDIO_TAG="2.2.354" # mcr.microsoft.com/presidio-analyzer (PII redaction) +SMITHDB_TAG="latest" # langchain/smithdb (0.16+ only, absent in GA; floating tag -> pin a digest for air-gap) ############################################################################### # CLI parsing @@ -27,16 +44,22 @@ OPERATOR_VERSION="" PLATFORM="linux/amd64" DRY_RUN=false DEST_REPO="" +WITH_PRESIDIO=false +WITH_SMITHDB=false usage() { cat < [--dest-repo ] [--version ] [--platform linux/arm64] [--dry-run] +Usage: $0 --registry [--dest-repo ] [--version ] + [--operator-version ] [--platform linux/arm64] + [--with-presidio] [--with-smithdb] [--dry-run] - --registry Mandatory. Destination registry (e.g. myregistry or 12345678.dkr.ecr.us-east-1.amazonaws.com) - --dest-repo Single destination repo (e.g. langchain/langchain_repository). Tags become -. - --version Version to use for LangSmith images (default: $DEFAULT_VERSION) + --registry Mandatory. Destination registry (e.g. myregistry or 12345678.dkr.ecr.us-east-1.amazonaws.com) + --dest-repo Single destination repo (e.g. langchain/langchain_repository). Tags become -. + --version Version for LangSmith application images (default: $DEFAULT_VERSION) --operator-version Version for langgraph-operator (default: $DEFAULT_OPERATOR_VERSION) --platform Architecture to pull (default: linux/amd64) + --with-presidio Also mirror mcr.microsoft.com/presidio-analyzer:$PRESIDIO_TAG (PII redaction add-on) + --with-smithdb Also mirror docker.io/langchain/smithdb:$SMITHDB_TAG (0.16+ only) --dry-run Only print the docker commands EOF exit 1 @@ -49,6 +72,8 @@ while [[ $# -gt 0 ]]; do --version) VERSION="$2"; shift 2 ;; --operator-version) OPERATOR_VERSION="$2"; shift 2 ;; --platform) PLATFORM="$2"; shift 2 ;; + --with-presidio) WITH_PRESIDIO=true; shift ;; + --with-smithdb) WITH_SMITHDB=true; shift ;; --dry-run) DRY_RUN=true; shift ;; *) usage ;; esac @@ -56,33 +81,58 @@ done [[ -z $REGISTRY ]] && { echo "ERROR: --registry is required"; usage; } -# Use provided version or default +# Warn if falling back to the built-in default versions instead of an explicit --version. +[[ -z $VERSION ]] && echo "WARNING: --version not provided; using default ${DEFAULT_VERSION}. Pass --version to target a specific release." >&2 +[[ -z $OPERATOR_VERSION ]] && echo "WARNING: --operator-version not provided; using default ${DEFAULT_OPERATOR_VERSION}." >&2 VERSION="${VERSION:-$DEFAULT_VERSION}" OPERATOR_VERSION="${OPERATOR_VERSION:-$DEFAULT_OPERATOR_VERSION}" -# Build images array with the specified version +############################################################################### +# Image list +############################################################################### +# Application images are tagged with --version; the operator with --operator-version; +# in-cluster dependencies with the chart-pinned constants above. This is the complete +# set the chart deploys for core + Fleet + Insights + Polly with in-cluster databases. IMAGES=( - "docker.io/langchain/langsmith-ace-backend:${VERSION}" + # Core LangSmith services "docker.io/langchain/langsmith-backend:${VERSION}" - "docker.io/langchain/langsmith-clio:${VERSION}" "docker.io/langchain/langsmith-frontend:${VERSION}" - "docker.io/langchain/hosted-langserve-backend:${VERSION}" - "docker.io/langchain/langgraph-operator:${OPERATOR_VERSION}" "docker.io/langchain/langsmith-go-backend:${VERSION}" + "docker.io/langchain/langsmith-ace-backend:${VERSION}" "docker.io/langchain/langsmith-playground:${VERSION}" - "docker.io/langchain/agent-builder-tool-server:${VERSION}" - "docker.io/langchain/agent-builder-trigger-server:${VERSION}" - "docker.io/langchain/agent-builder-deep-agent:${VERSION}" - "docker.io/postgres:15.15" - "docker.io/redis:8" - "docker.io/clickhouse/clickhouse-server:25.12" + "docker.io/langchain/hosted-langserve-backend:${VERSION}" + "docker.io/langchain/langgraph-operator:${OPERATOR_VERSION}" + # Add-ons: Insights, Fleet, Polly/Chat + "docker.io/langchain/langsmith-clio:${VERSION}" # Insights + "docker.io/langchain/agent-builder-deep-agent:${VERSION}" # Fleet + "docker.io/langchain/agent-builder-tool-server:${VERSION}" # Fleet + "docker.io/langchain/agent-builder-trigger-server:${VERSION}" # Fleet + "docker.io/langchain/langsmith-polly:${VERSION}" # Polly / Chat + # In-cluster dependencies (tags pinned by the chart, not by --version) + "docker.io/postgres:${PG_TAG}" # platform + feature-app Postgres + "docker.io/redis:${REDIS_TAG}" # platform + feature-app + operator Redis + "docker.io/clickhouse/clickhouse-server:${CLICKHOUSE_TAG}" + "docker.io/pgvector/pgvector:${PGVECTOR_TAG}" # operator-created per-deployment Postgres ) -echo "Using version: ${VERSION}" -echo "Registry: ${REGISTRY}" -[[ -n $DEST_REPO ]] && echo "Dest repo: ${DEST_REPO}" -echo "Platform: ${PLATFORM}" -echo "Dry-run: ${DRY_RUN}" +# Optional components +if $WITH_PRESIDIO; then + IMAGES+=( "mcr.microsoft.com/presidio-analyzer:${PRESIDIO_TAG}" ) +fi +if $WITH_SMITHDB; then + echo "WARNING: mirroring langchain/smithdb:${SMITHDB_TAG} (floating tag). Pin to a digest for reproducible/air-gapped installs." >&2 + IMAGES+=( "docker.io/langchain/smithdb:${SMITHDB_TAG}" ) +fi + +echo "Using version: ${VERSION}" +echo "Operator version: ${OPERATOR_VERSION}" +echo "Registry: ${REGISTRY}" +[[ -n $DEST_REPO ]] && echo "Dest repo: ${DEST_REPO}" +echo "Platform: ${PLATFORM}" +echo "With presidio: ${WITH_PRESIDIO}" +echo "With smithdb: ${WITH_SMITHDB}" +echo "Dry-run: ${DRY_RUN}" +echo "Images to mirror: ${#IMAGES[@]}" echo ############################################################################### @@ -103,7 +153,7 @@ run_cmd() { # Main loop ############################################################################### for SRC in "${IMAGES[@]}"; do - repo_tag=${SRC#*/} # strip first path element (docker.io/…) + repo_tag=${SRC#*/} # strip registry host (docker.io/…, mcr.microsoft.com/…) tag=${repo_tag##*:} # version tag if [[ -n $DEST_REPO ]]; then @@ -124,4 +174,4 @@ for SRC in "${IMAGES[@]}"; do echo done -echo "✓ All images processed." +echo "✓ All ${#IMAGES[@]} images processed."