Skip to content

Fix/mirror script complete image list #805

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Haitham Shahin (haithamshahin333) wants to merge 3 commits into
base: main
Choose a base branch
from
Open

Fix/mirror script complete image list #805

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
b85a8d5
Fix mirror script: complete image set + correct dependency tags
haithamshahin333 Jul 14, 2026
e0b51e8
adding readme
haithamshahin333 Jul 14, 2026
e03188b
Merge branch 'main' into fix/mirror-script-complete-image-list
haithamshahin333 Jul 21, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
169 changes: 169 additions & 0 deletions charts/langsmith/scripts/mirror_langsmith_images.README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,169 @@
# Mirroring LangSmith self-hosted images

`mirror_langsmith_images.sh` pulls every image the LangSmith self-hosted Helm chart deploys,
retags it, and pushes it to a private registry — for air-gapped clusters or environments that
must pull from an internal registry.

It mirrors the **complete** set for a deployment that includes the add-ons (Fleet, Insights,
Polly/Chat) with in-cluster databases.

## Quick start

Always dry-run first to see exactly what would be pushed:

```bash
./mirror_langsmith_images.sh --registry your-registry.example.com --dry-run
```

Then mirror for real (defaults to the latest GA app version):

```bash
./mirror_langsmith_images.sh --registry your-registry.example.com
```

## Flags

| Flag | Purpose |
|------|---------|
| `--registry` | **Required.** Destination registry prefix (e.g. `your-registry.example.com` or `123456789012.dkr.ecr.us-east-1.amazonaws.com`). |
| `--version` | App-image tag for all `langchain/*` LangSmith images. **This is the chart's `appVersion`, not the Helm chart version** (see below). Defaults to the latest GA app version. |
| `--operator-version` | Tag for `langgraph-operator`. Defaults to the value pinned by the current chart. |
| `--platform` | Architecture to pull (`linux/amd64` default, or `linux/arm64`). |
| `--dest-repo` | "Marketplace" mode: push every image into a single repo, tagged `<image-name>-<version>` (useful for registries like ECR that prefer one repo). |
| `--with-presidio` | Also mirror `mcr.microsoft.com/presidio-analyzer` (only needed if you enable PII redaction). |
| `--with-smithdb` | Also mirror `langchain/smithdb` (0.16+ only). |
| `--dry-run` | Print the `docker` commands without executing them. |

## Examples

```bash
# 1. Preview the GA image set without pushing
./mirror_langsmith_images.sh --registry your-registry.example.com --dry-run

# 2. Mirror the latest GA release (app version 0.15.18)
./mirror_langsmith_images.sh --registry your-registry.example.com --version 0.15.18

# 3. Mirror a release candidate instead of GA
./mirror_langsmith_images.sh --registry your-registry.example.com \
--version 0.16.13rc1 --operator-version 0.1.47

# 4. arm64 nodes
./mirror_langsmith_images.sh --registry your-registry.example.com \
--version 0.15.18 --platform linux/arm64

# 5. Single-repo ("marketplace") mode, e.g. AWS ECR
./mirror_langsmith_images.sh --registry 123456789012.dkr.ecr.us-east-1.amazonaws.com \
--dest-repo langchain/langsmith --version 0.15.18

# 6. Include the optional components (PII redaction + smithdb)
./mirror_langsmith_images.sh --registry your-registry.example.com \
--version 0.16.13rc1 --operator-version 0.1.47 --with-presidio --with-smithdb
```

## What gets mirrored

| Group | Images |
|-------|--------|
| Core | `langsmith-backend`, `langsmith-frontend`, `langsmith-go-backend`, `langsmith-ace-backend`, `langsmith-playground`, `hosted-langserve-backend`, `langgraph-operator` |
| Add-ons | `langsmith-clio` (Insights), `agent-builder-deep-agent` + `agent-builder-tool-server` + `agent-builder-trigger-server` (Fleet), `langsmith-polly` (Polly/Chat) |
| In-cluster dependencies | `postgres`, `redis`, `clickhouse-server`, `pgvector/pgvector` |
| Optional (flags) | `presidio-analyzer` (`--with-presidio`), `smithdb` (`--with-smithdb`) |

## IMPORTANT: chart version vs. image tag

There are **two different version numbers**, and mixing them up is the most common mistake:

- **Helm chart version** — what the self-hosted changelog and GitHub releases show (e.g. `0.15.13`),
and what you pass to `helm upgrade --install … --version`.
- **App image tag (`appVersion`)** — the tag on the container images (e.g. `0.15.18`), and what this
script's `--version` expects.

They increment independently. For example, chart **`0.15.13`** ships app images tagged **`0.15.18`**.
Passing the chart version (`0.15.13`) to `--version` would try to pull image tags that don't exist.

| | Helm chart `--version` (helm) | App images (`--version` here) | Operator (`--operator-version`) | postgres | redis | clickhouse | pgvector |
|---|---|---|---|---|---|---|---|
| Latest GA | `0.15.13` | `0.15.18` | `0.1.47` | `14.7` | `7` | `25.12` | `pg15` |
| Latest RC | `0.16.0-rc.12` | `0.16.13rc1` | `0.1.47` | `14.7` | `7` | `25.12` | `pg15` |

_(As of 2026-07-14. Always re-verify against the chart you install — see next section.)_

## Picking the right versions for a release (do this on every upgrade)

The app-image tag follows `--version`, but the **dependency tags** (`postgres`, `redis`, `clickhouse`,
`pgvector`) are pinned by the chart release and are **independent of `--version`**. They're set as
constants at the top of the script. On the current GA and RC they're identical (`14.7` / `7` / `25.12` /
`pg15`), so moving between those only changes `--version`. A future release could bump one of them — so
when you adopt a new chart version, verify the tags against that chart and update the constants if needed.

Read the exact tags a given chart version deploys straight from the chart source (replace `CHART_VER`
with the Helm chart version you're installing):

```bash
CHART_VER=0.15.13
BASE="https://raw.githubusercontent.com/langchain-ai/helm/langsmith-${CHART_VER}/charts/langsmith"

# App-image tag to pass as --version (this is the chart's appVersion):
curl -s "$BASE/Chart.yaml" | grep '^appVersion:'

# Operator + dependency tags (compare against the script's constants):
curl -s "$BASE/values.yaml" | grep -E '^ (operatorImage|postgresImage|redisImage|clickhouseImage):' -A3 \
| grep -E 'Image:|tag:'

# Operator-created database images (pgvector + inline redis):
curl -s "$BASE/values.yaml" | grep -E 'pgvector/pgvector:pg|image: docker.io/redis:'
```

(If you've added the LangSmith Helm repo, `helm show values <chart> --version <CHART_VER>` gives the
same information.)

Then set:
- `--version` = the `appVersion` value
- `--operator-version` = `operatorImage.tag`
- the script's `PG_TAG` / `REDIS_TAG` / `CLICKHOUSE_TAG` / `PGVECTOR_TAG` constants = the dependency tags,
**only if they changed** from what the script already has.

## After mirroring: point the chart at your registry

In your `values.yaml`:

```yaml
images:
registry: "your-registry.example.com"
imagePullSecrets:
- name: langsmith-pull-secret # a docker-registry Secret in the install namespace
# For each image, strip the leading "docker.io/" from the repository so it isn't doubled, and pin the tag.
backendImage: { repository: "langchain/langsmith-backend", tag: "0.15.18" }
# …one entry per image…
```

**One thing `images.registry` does NOT cover:** the langgraph-operator creates a Redis and a Postgres
(`pgvector`) for each agent deployment from raw image strings in `operator.templates.redis` and
`operator.templates.db`. Those are not rewritten by `images.registry`, so override the image line in each:

```yaml
operator:
templates:
redis: | # …change the image line to your-registry.example.com/redis:7
db: | # …change the image line to your-registry.example.com/pgvector/pgvector:pg15
```

The full template blocks (with a placeholder registry) are in the docs — see the reference below.

## Optional: verify image signatures

v15+ `docker.io/langchain/*` images are keyless Cosign-signed. You can verify before and after mirroring:

```bash
cosign verify \
--certificate-oidc-issuer https://token.actions.githubusercontent.com \
--certificate-identity-regexp 'https://github\.com/langchain-ai/langchainplus/\.github/workflows/release_self_hosted_on_version_bump\.yaml@refs/heads/v[0-9]+-stable' \
docker.io/langchain/langsmith-backend:<tag>
```

## References

- Mirror images guide (incl. the full operator-template overrides and signature verification):
https://docs.langchain.com/langsmith/self-host-mirroring-images
- Self-hosted changelog (chart versions): https://docs.langchain.com/langsmith/self-hosted-changelog
- Kubernetes install guide: https://docs.langchain.com/langsmith/kubernetes
106 changes: 78 additions & 28 deletions charts/langsmith/scripts/mirror_langsmith_images.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,15 +8,32 @@
# Marketplace mode (--dest-repo): retag as <REGISTRY>/<dest-repo>:<image-name>-<tag>
#
# Examples:
# ./mirror_langsmith_images.sh --registry myregistry --version 0.10.66 --platform linux/arm64
# ./mirror_langsmith_images.sh --registry myregistry --version 0.15.18 --platform linux/amd64
# Target a release candidate instead of the GA default:
# ./mirror_langsmith_images.sh --registry myregistry --version 0.16.13rc1 --operator-version 0.1.47 --platform linux/amd64
# ./mirror_langsmith_images.sh --registry 709825985650.dkr.ecr.us-east-1.amazonaws.com \
# --dest-repo langchain/langchain-repository --version 0.10.67 --platform linux/amd64 --dry-run
# --dest-repo langchain/langchain-repository --version 0.15.18 --platform linux/amd64 --dry-run

set -euo pipefail

# Default version
DEFAULT_VERSION="0.13.9"
DEFAULT_OPERATOR_VERSION="0.1.37"
###############################################################################
# Versions
###############################################################################
# LangSmith application images + operator. Bump these when targeting a new release.
DEFAULT_VERSION="0.15.18" # latest GA appVersion -> all langchain/* application images (override with --version, e.g. an RC)
DEFAULT_OPERATOR_VERSION="0.1.47" # langgraph-operator image tag (same in current GA and RC)

# In-cluster dependency image tags. These are pinned by the chart (charts/langsmith/values.yaml)
# and are NOT tied to --version. Re-verify them against values.yaml whenever you bump the chart:
# postgresImage.tag, redisImage.tag, clickhouseImage.tag, and the operator.templates pgvector tag.
PG_TAG="14.7"
REDIS_TAG="7"
CLICKHOUSE_TAG="25.12"
PGVECTOR_TAG="pg15"

# Optional components (off by default; enable with --with-presidio / --with-smithdb).
PRESIDIO_TAG="2.2.354" # mcr.microsoft.com/presidio-analyzer (PII redaction)
SMITHDB_TAG="latest" # langchain/smithdb (0.16+ only, absent in GA; floating tag -> pin a digest for air-gap)

###############################################################################
# CLI parsing
Expand All @@ -27,16 +44,22 @@ OPERATOR_VERSION=""
PLATFORM="linux/amd64"
DRY_RUN=false
DEST_REPO=""
WITH_PRESIDIO=false
WITH_SMITHDB=false

usage() {
cat <<EOF
Usage: $0 --registry <registry-prefix> [--dest-repo <repo>] [--version <version>] [--platform linux/arm64] [--dry-run]
Usage: $0 --registry <registry-prefix> [--dest-repo <repo>] [--version <version>]
[--operator-version <version>] [--platform linux/arm64]
[--with-presidio] [--with-smithdb] [--dry-run]

--registry Mandatory. Destination registry (e.g. myregistry or 12345678.dkr.ecr.us-east-1.amazonaws.com)
--dest-repo Single destination repo (e.g. langchain/langchain_repository). Tags become <image-name>-<version>.
--version Version to use for LangSmith images (default: $DEFAULT_VERSION)
--registry Mandatory. Destination registry (e.g. myregistry or 12345678.dkr.ecr.us-east-1.amazonaws.com)
--dest-repo Single destination repo (e.g. langchain/langchain_repository). Tags become <image-name>-<version>.
--version Version for LangSmith application images (default: $DEFAULT_VERSION)
--operator-version Version for langgraph-operator (default: $DEFAULT_OPERATOR_VERSION)
--platform Architecture to pull (default: linux/amd64)
--with-presidio Also mirror mcr.microsoft.com/presidio-analyzer:$PRESIDIO_TAG (PII redaction add-on)
--with-smithdb Also mirror docker.io/langchain/smithdb:$SMITHDB_TAG (0.16+ only)
--dry-run Only print the docker commands
EOF
exit 1
Expand All @@ -49,40 +72,67 @@ while [[ $# -gt 0 ]]; do
--version) VERSION="$2"; shift 2 ;;
--operator-version) OPERATOR_VERSION="$2"; shift 2 ;;
--platform) PLATFORM="$2"; shift 2 ;;
--with-presidio) WITH_PRESIDIO=true; shift ;;
--with-smithdb) WITH_SMITHDB=true; shift ;;
--dry-run) DRY_RUN=true; shift ;;
*) usage ;;
esac
done

[[ -z $REGISTRY ]] && { echo "ERROR: --registry is required"; usage; }

# Use provided version or default
# Warn if falling back to the built-in default versions instead of an explicit --version.
[[ -z $VERSION ]] && echo "WARNING: --version not provided; using default ${DEFAULT_VERSION}. Pass --version to target a specific release." >&2
[[ -z $OPERATOR_VERSION ]] && echo "WARNING: --operator-version not provided; using default ${DEFAULT_OPERATOR_VERSION}." >&2
VERSION="${VERSION:-$DEFAULT_VERSION}"
OPERATOR_VERSION="${OPERATOR_VERSION:-$DEFAULT_OPERATOR_VERSION}"

# Build images array with the specified version
###############################################################################
# Image list
###############################################################################
# Application images are tagged with --version; the operator with --operator-version;
# in-cluster dependencies with the chart-pinned constants above. This is the complete
# set the chart deploys for core + Fleet + Insights + Polly with in-cluster databases.
IMAGES=(
"docker.io/langchain/langsmith-ace-backend:${VERSION}"
# Core LangSmith services
"docker.io/langchain/langsmith-backend:${VERSION}"
"docker.io/langchain/langsmith-clio:${VERSION}"
"docker.io/langchain/langsmith-frontend:${VERSION}"
"docker.io/langchain/hosted-langserve-backend:${VERSION}"
"docker.io/langchain/langgraph-operator:${OPERATOR_VERSION}"
"docker.io/langchain/langsmith-go-backend:${VERSION}"
"docker.io/langchain/langsmith-ace-backend:${VERSION}"
"docker.io/langchain/langsmith-playground:${VERSION}"
"docker.io/langchain/agent-builder-tool-server:${VERSION}"
"docker.io/langchain/agent-builder-trigger-server:${VERSION}"
"docker.io/langchain/agent-builder-deep-agent:${VERSION}"
"docker.io/postgres:15.15"
"docker.io/redis:8"
"docker.io/clickhouse/clickhouse-server:25.12"
"docker.io/langchain/hosted-langserve-backend:${VERSION}"
"docker.io/langchain/langgraph-operator:${OPERATOR_VERSION}"
# Add-ons: Insights, Fleet, Polly/Chat
"docker.io/langchain/langsmith-clio:${VERSION}" # Insights
"docker.io/langchain/agent-builder-deep-agent:${VERSION}" # Fleet
"docker.io/langchain/agent-builder-tool-server:${VERSION}" # Fleet
"docker.io/langchain/agent-builder-trigger-server:${VERSION}" # Fleet
"docker.io/langchain/langsmith-polly:${VERSION}" # Polly / Chat
# In-cluster dependencies (tags pinned by the chart, not by --version)
"docker.io/postgres:${PG_TAG}" # platform + feature-app Postgres
"docker.io/redis:${REDIS_TAG}" # platform + feature-app + operator Redis
"docker.io/clickhouse/clickhouse-server:${CLICKHOUSE_TAG}"
"docker.io/pgvector/pgvector:${PGVECTOR_TAG}" # operator-created per-deployment Postgres
)

echo "Using version: ${VERSION}"
echo "Registry: ${REGISTRY}"
[[ -n $DEST_REPO ]] && echo "Dest repo: ${DEST_REPO}"
echo "Platform: ${PLATFORM}"
echo "Dry-run: ${DRY_RUN}"
# Optional components
if $WITH_PRESIDIO; then
IMAGES+=( "mcr.microsoft.com/presidio-analyzer:${PRESIDIO_TAG}" )
fi
if $WITH_SMITHDB; then
echo "WARNING: mirroring langchain/smithdb:${SMITHDB_TAG} (floating tag). Pin to a digest for reproducible/air-gapped installs." >&2
IMAGES+=( "docker.io/langchain/smithdb:${SMITHDB_TAG}" )
fi

echo "Using version: ${VERSION}"
echo "Operator version: ${OPERATOR_VERSION}"
echo "Registry: ${REGISTRY}"
[[ -n $DEST_REPO ]] && echo "Dest repo: ${DEST_REPO}"
echo "Platform: ${PLATFORM}"
echo "With presidio: ${WITH_PRESIDIO}"
echo "With smithdb: ${WITH_SMITHDB}"
echo "Dry-run: ${DRY_RUN}"
echo "Images to mirror: ${#IMAGES[@]}"
echo

###############################################################################
Expand All @@ -103,7 +153,7 @@ run_cmd() {
# Main loop
###############################################################################
for SRC in "${IMAGES[@]}"; do
repo_tag=${SRC#*/} # strip first path element (docker.io/…)
repo_tag=${SRC#*/} # strip registry host (docker.io/…, mcr.microsoft.com/…)
tag=${repo_tag##*:} # version tag

if [[ -n $DEST_REPO ]]; then
Expand All @@ -124,4 +174,4 @@ for SRC in "${IMAGES[@]}"; do
echo
done

echo "✓ All images processed."
echo "✓ All ${#IMAGES[@]} images processed."
Loading