Skip to content
Merged
Show file tree
Hide file tree
Changes from 6 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .bun-version
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
1.4.0
247 changes: 247 additions & 0 deletions .github/workflows/build-binary.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,247 @@
name: Build Binaries

# Builds the Bun single-file executables (`npm run build:binary`, six targets)
# and smoke-tests every one of them on real hardware:
#
# build (Linux x64) -> smoke linux-arm64 (Blacksmith Ubuntu ARM)
# -> smoke windows-x64 (Blacksmith Windows Server 2025)
# -> smoke windows-arm64 (GitHub-hosted windows-11-arm)
# -> smoke darwin-arm64 (Blacksmith macOS 15, Apple Silicon)
# -> smoke darwin-x64 (GitHub-hosted macos-15-intel)
#
# All six targets are cross-compiled on ONE Linux runner: Bun downloads a
# ~80 MB runtime per target and compiling is a matter of seconds, whereas
# cross-compiling ON a Windows runner is broken upstream (oven-sh/bun#11198,
# the workspace and the Bun cache sit on different drives). The Linux job also
# smoke-tests its own linux-x64 output twice, once against SQLite and once
# against the PostgreSQL service container (service containers are Linux-only,
# which is one reason the smoke matrix is a separate job).
#
# The smoke matrix mixes runner vendors because Blacksmith offers no Windows
# ARM64 and no macOS Intel runners; those two targets run on GitHub-hosted
# runners, which are free for public repositories. Every smoke job only
# downloads its artifact and runs `node scripts/smoke-binary.ts`, so it needs
# neither `npm ci` nor Bun.
#
# The build job's `npm run build:binary` runs `npm run build:web`, which fetches
# https://oomol.com/en/apps/catalog.json (the provider icon map) at build time,
# so an outage there fails the build job. This is the first PR-gated path
# through the web build.
#
# darwin binaries built on Linux carry an invalid ad-hoc signature (Bun 1.4.0
# writes a wrong page hash, which macOS 27 rejects), so the macOS smoke jobs
# re-sign the binary with `codesign` and verify the signature before running it.
#
# A `runs-on` label that no runner serves queues forever instead of failing, so
# `workflow_dispatch` is here to verify the labels before relying on them.
on:
push:
branches: [main]
pull_request:
workflow_dispatch:

# Cancel superseded runs for the same ref (e.g. new push to an open PR).
concurrency:
group: build-binary-${{ github.ref }}
cancel-in-progress: true

# Least privilege: these jobs only read the repository; the binaries are
# workflow artifacts, nothing is published.
permissions:
contents: read

env:
# This project runs TypeScript directly with `node` (native type stripping) and
# forbids `tsx` / `--experimental-strip-types` (see AGENTS.md). Type stripping is
# on by default only on Node >= 22.18 / >= 23.6, so pin an LTS that supports it.
# Do NOT downgrade below 22.18 — `npm ci` (postinstall codegen) and typecheck fail.
NODE_VERSION: "24"

jobs:
build:
name: Build all targets
runs-on: blacksmith-4vcpu-ubuntu-2404
timeout-minutes: 30
services:
postgres:
image: postgres:15-alpine
env:
POSTGRES_DB: open_connector_test
POSTGRES_PASSWORD: postgres
POSTGRES_USER: postgres
ports:
- 5432:5432
options: >-
--health-cmd "pg_isready -U postgres"
--health-interval 5s
--health-timeout 5s
--health-retries 10
steps:
- name: Checkout
uses: actions/checkout@v7
with:
# No step needs the persisted GITHUB_TOKEN in git config; dropping it
# keeps `npm ci` postinstall scripts from reading it (zizmor hardening).
persist-credentials: false

- name: Setup Node.js ${{ env.NODE_VERSION }}
uses: actions/setup-node@v7
with:
node-version: ${{ env.NODE_VERSION }}
cache: npm

# Pinned to an immutable SHA rather than the mutable v2 tag: this
# third-party action installs the toolchain that produces the shipped
# binaries, so its supply chain must not be silently updatable.
# `.bun-version` is the single owner of the pinned Bun version; the build
# script refuses to run under any other version.
- name: Setup Bun
uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6 # v2.2.0
with:
bun-version-file: .bun-version

# Bun downloads one runtime per cross-compile target (~80 MB each) from
# registry.npmjs.org into ~/.bun/install/cache; setup-bun caches only the
# host executable. The key changes with the pinned Bun version.
- name: Cache Bun cross-compile runtimes
uses: actions/cache@v6
with:
path: ~/.bun/install/cache/bun-*-v*
key: bun-runtimes-${{ runner.os }}-${{ hashFiles('.bun-version') }}

# Runs postinstall codegen (provider registry + catalog).
- name: Install dependencies
run: npm ci

# Regenerates the catalog, builds the web console, then compiles all six targets.
- name: Build binaries
run: npm run build:binary

# SQLite mode: the embedded catalog, migrations and console all come from the binary.
- name: Smoke test linux-x64 (SQLite)
run: node scripts/smoke-binary.ts dist/open-connector-linux-x64

# PostgreSQL mode: the binary's own `migrate` subcommand applies the embedded
# postgresql/ migrations, then the server must pass its schema check.
- name: Migrate PostgreSQL with linux-x64
env:
OOMOL_CONNECT_DATABASE_URL: postgresql://postgres:postgres@127.0.0.1:5432/open_connector_test
run: dist/open-connector-linux-x64 migrate

- name: Smoke test linux-x64 (PostgreSQL)
env:
OOMOL_CONNECT_DATABASE_URL: postgresql://postgres:postgres@127.0.0.1:5432/open_connector_test
run: node scripts/smoke-binary.ts dist/open-connector-linux-x64

# One artifact per target so each smoke job downloads only its own ~35 MB
# zip. upload-artifact drops the exec bit; the smoke jobs chmod after download.
- name: Upload linux-x64
uses: actions/upload-artifact@v7
with:
name: binary-linux-x64
path: dist/open-connector-linux-x64
if-no-files-found: error
retention-days: 1

- name: Upload linux-arm64
uses: actions/upload-artifact@v7
with:
name: binary-linux-arm64
path: dist/open-connector-linux-arm64
if-no-files-found: error
retention-days: 1

- name: Upload darwin-x64
uses: actions/upload-artifact@v7
with:
name: binary-darwin-x64
path: dist/open-connector-darwin-x64
if-no-files-found: error
retention-days: 1

- name: Upload darwin-arm64
uses: actions/upload-artifact@v7
with:
name: binary-darwin-arm64
path: dist/open-connector-darwin-arm64
if-no-files-found: error
retention-days: 1

- name: Upload windows-x64
uses: actions/upload-artifact@v7
with:
name: binary-windows-x64
path: dist/open-connector-windows-x64.exe
if-no-files-found: error
retention-days: 1

- name: Upload windows-arm64
uses: actions/upload-artifact@v7
with:
name: binary-windows-arm64
path: dist/open-connector-windows-arm64.exe
if-no-files-found: error
retention-days: 1

# Run each remaining target on its native OS and architecture. linux-x64 was
# already exercised by the build job.
smoke:
name: Smoke ${{ matrix.target }}
needs: build
timeout-minutes: 15
strategy:
fail-fast: false
matrix:
include:
- target: linux-arm64
runner: blacksmith-2vcpu-ubuntu-2404-arm
file: open-connector-linux-arm64
- target: windows-x64
runner: blacksmith-2vcpu-windows-2025
file: open-connector-windows-x64.exe
# Blacksmith has no Windows ARM64 runners.
- target: windows-arm64
runner: windows-11-arm
file: open-connector-windows-arm64.exe
- target: darwin-arm64
runner: blacksmith-6vcpu-macos-15
file: open-connector-darwin-arm64
# Blacksmith has no macOS Intel runners.
- target: darwin-x64
runner: macos-15-intel
file: open-connector-darwin-x64
runs-on: ${{ matrix.runner }}
steps:
- name: Checkout
uses: actions/checkout@v7
with:
persist-credentials: false

# The smoke script uses only node: builtins, so no `npm ci` is needed.
- name: Setup Node.js ${{ env.NODE_VERSION }}
uses: actions/setup-node@v7
with:
node-version: ${{ env.NODE_VERSION }}

- name: Download ${{ matrix.target }} binary
uses: actions/download-artifact@v8
with:
name: binary-${{ matrix.target }}
path: dist

# Artifact zips do not preserve file modes.
- name: Restore executable bit
if: runner.os != 'Windows'
run: chmod +x dist/${{ matrix.file }}

# Linux-built darwin binaries carry an invalid ad-hoc signature (Bun 1.4.0
# writes a wrong page hash); re-sign them and assert the signature is valid,
# since macOS 15 would tolerate an invalid one silently.
- name: Ad-hoc sign and verify
if: runner.os == 'macOS'
run: |
codesign --force --sign - dist/${{ matrix.file }}
codesign --verify --verbose=2 dist/${{ matrix.file }}

- name: Smoke test ${{ matrix.target }}
run: node scripts/smoke-binary.ts dist/${{ matrix.file }}
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -275,6 +275,7 @@ Issues and pull requests are welcome.
- [Fly.io deployment](docs/fly-io.md)
- [Cloudflare deployment](docs/cloudflare.md)
- [Docker image (GHCR)](docs/docker-ghcr.md)
- [Single binary](docs/single-binary.md)
- [Configuration](docs/configuration.md)
- [Credentials and OAuth](docs/credentials.md)
- [Catalog format](docs/catalog-format.md)
Expand Down
109 changes: 109 additions & 0 deletions docs/single-binary.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,109 @@
# Single Binary

OpenConnector can be compiled into one self-contained executable per platform with
[Bun](https://bun.com/docs/bundler/executables). The binary embeds the runtime, the generated
provider catalog, the database migrations, and the built web console, so it runs from any directory
without a Node.js installation, a checkout, or `node_modules`. Nothing is extracted to disk at
runtime.

## Build

Building requires Node.js for the npm scripts and the Bun version pinned in `.bun-version`; the
build script exits with an error under any other Bun version. Compile on Linux or macOS.
Cross-compiling on a Windows host fails upstream
([oven-sh/bun#11198](https://github.com/oven-sh/bun/issues/11198)).

```bash
npm install
npm run build:binary
```

`npm run build:binary` regenerates the catalog, builds the web console, and then writes six files
under `dist/`:

```text
dist/open-connector-linux-x64
dist/open-connector-linux-arm64
dist/open-connector-darwin-x64
dist/open-connector-darwin-arm64
dist/open-connector-windows-x64.exe
dist/open-connector-windows-arm64.exe
```

Each file is roughly 145 to 170 MiB. To build a subset, pass one or more target names after `--`:

```bash
npm run build:binary -- linux-x64 darwin-arm64
```

Bun downloads a runtime for every target that differs from the host (about 80 MB each, from
`registry.npmjs.org`) into `~/.bun/install/cache` on first use. These downloads are not integrity
checked by Bun; TLS is the only protection.
Comment thread
coderabbitai[bot] marked this conversation as resolved.

`npm run build:web` also fetches the provider icon map from https://oomol.com/en/apps/catalog.json
and fails when it is unreachable.

`.bun-version` and the `@types/bun` devDependency pin in `package.json` are bumped together.

## Run

The binary takes the same environment variables as `npm start`; see
[configuration.md](configuration.md) for the full reference. The ones you will usually set:

| Variable | Default | Meaning |
| ---------------------------- | ----------- | ----------------------------------------------------------------- |
| `OOMOL_CONNECT_DATA_DIR` | `./data` | SQLite database, transit files, and upload staging. |
| `PORT` | `3000` | HTTP port. |
| `HOST` | `127.0.0.1` | Bind address. |
| `OOMOL_CONNECT_DATABASE_URL` | unset | PostgreSQL connection URL. When unset, SQLite under the data dir. |

```bash
OOMOL_CONNECT_DATA_DIR="$HOME/open-connector-data" \
PORT=3000 \
./dist/open-connector-linux-x64
```

With SQLite, migrations are applied automatically when the database opens, exactly as with
`npm start`.

### PostgreSQL Migrations

PostgreSQL migrations are explicit. The binary has a `migrate` subcommand that applies the embedded
migrations and exits without starting the server. Run it before the first start and before starting
a newer binary that contains pending migrations:

```bash
OOMOL_CONNECT_DATABASE_URL="postgresql://open_connector:password@db.example.com:5432/open_connector?sslmode=verify-full" \
./dist/open-connector-linux-x64 migrate

OOMOL_CONNECT_DATABASE_URL="postgresql://open_connector:password@db.example.com:5432/open_connector?sslmode=verify-full" \
./dist/open-connector-linux-x64
```

The server only checks schema readiness at startup and refuses to start when migrations are
missing; it never applies PostgreSQL DDL itself. Without `OOMOL_CONNECT_DATABASE_URL`, `migrate`
prints a notice that SQLite migrations are applied automatically and exits.

## Differences From `npm start`

- `NODE_ENV` is fixed to `production` inside the binary, so logs are always JSON (no pretty
printing). `OOMOL_CONNECT_LOG_LEVEL` and every other environment variable are read at runtime as
usual.
- macOS: binaries built on macOS are ad-hoc signed by the build script. Binaries built on another
operating system carry an invalid ad-hoc signature, and macOS 27 and newer refuses to run them
until you re-sign them:

```bash
codesign --force --sign - dist/open-connector-darwin-arm64
```

`codesign --verify --verbose=2 <file>` prints "valid on disk" for a usable binary and "invalid
signature" for one that still needs re-signing.

## Notes

- Like `npm start`, the binary does not load `.env` files (Bun's automatic loading is disabled at
build time).
- On Windows, stopping the process from a process manager or `taskkill` terminates it immediately;
the graceful shutdown hook that closes the HTTP server and the database on Linux and macOS does
not run. This matches `node src/server/index.ts` on Windows.
Loading