upstream-watch

upstream-watch is a git-ops tool to monitor an upstream repository and execute update steps if necessary. It only supports git.

I wrote this tool to support my personal container infrastructure, which is completely managed via a single git repository. There are two different modes:

• single directory
• Subdirectories per service

Single directory

The target repository could look like this:

    .
    ├── .upstream-watch.yaml
    ├── README.md
    ├── .update-hooks.yaml
    ├── docker-compose.yml
    └── upstream-watch

All configuration files are in a single directory, which must be the root of the git directory. Use this mode by setting single_directory_mode: true. The rest of the needed configuration is identical to the subdirectory documentation.

Use subdirectories for different services

The target repository could look like this:

    .
    ├── .upstream-watch.yaml
    ├── README.md
    ├── service-1
    │   ├── .update-hooks.yaml
    │   ├── docker-compose.yml
    │   └── README.md
    ├── service-2
    │   ├── .update-hooks.yaml
    │   ├── docker-compose.yml
    │   └── README.md
    └── upstream-watch

The .upstream-watch.yaml is the main configuration file for this instance of upstream-watch. You can set the retry interval (in seconds) and folders that should be ignored.

    single_directory_mode: false
    retry_interval: 10
    ignore_folders: [".git", ".test"]

There are two services, each in its own subfolder. Each of these services holds a README.md (which is not interesting), a docker-compose.yml that defines the containers and a .update-hooks.yaml, which is the configuration file of upstream-watch for this specific service.

In case of an update to any of these files in a subfolder, upstream-watch will execute the pre- and post-hooks defined in the corresponding .update-hooks.yaml.

An example for a .update-hooks.yaml:

	pre_update_commands: ["docker compose down"]
	update_commands: ["docker compose pull"]
	post_update_commands: ["docker compose up -d"]

upstream-watch will stop all containers, pull updates from the registry and start them afterwards. Of course, you can do almost anything in these hooks, depending on the needs of your service.

Requirements on the server

• Pull access to upstream repository
• git installed

Running upstream-watch itself as a container

The image (ghcr.io/andresterba/upstream-watch) bundles git and a docker/docker compose CLI, and starts as root just long enough to grant its non-root user access to the bind-mounted /var/run/docker.sock, then drops privileges before running upstream-watch.

The following things matter for a working setup:

• Mount /var/run/docker.sock so the container can talk to the host's Docker daemon.
• Mount your repository at the same absolute path inside the container as it has on the host, and pass that path as the command argument. upstream-watch runs docker compose (and any other hook commands) with that path as their working directory, but those commands are executed by the host's Docker daemon — so any relative paths inside a service's docker-compose.yml (bind mounts, build contexts, env files) only resolve correctly if the path the container sees matches the path the host daemon sees. Using the default /workdir for both host and container path also works, but only because it happens to match on both sides — the important thing is that container path == host path, not the specific name.
• Mount your SSH keys and known_hosts read-only at /home/user/.ssh, since upstream-watch (and its bundled git) runs as the non-root user account, whose home is /home/user. git pull needs both a private key that's allowed to read the upstream repository and a known_hosts entry for the upstream host — without the latter you'll get Host key verification failed. and the pull (and the container, since a failed pull is currently fatal) will keep failing. Mount your existing ~/.ssh directory, or a purpose-built one containing just a deploy key and known_hosts. Keep the private key file itself chmod 600 on the host; a read-only bind mount doesn't relax the permission check ssh does on it.
• If any update_commands/pre_update_commands/post_update_commands pull images from a private registry (e.g. docker compose pull), also mount your Docker client config read-only at /home/user/.docker. Registry credentials from docker login live client-side in ~/.docker/config.json, so even though the actual pull is performed by the host's Docker daemon, the container's docker CLI still needs that config to authenticate the request.

services:
    upstream-watch:
        image: ghcr.io/andresterba/upstream-watch:latest
        restart: unless-stopped
        volumes:
            - /var/run/docker.sock:/var/run/docker.sock
            - /home/deploy/services:/home/deploy/services
            - /home/deploy/.ssh:/home/user/.ssh:ro
            - /home/deploy/.docker:/home/user/.docker:ro
        command: ["/home/deploy/services"]