Skip to content

Restore static linking and fix --version for release binaries - #397

Merged
matsen merged 4 commits into
masterfrom
396-version-and-static-gsl
Sep 2, 2026
Merged

matsen merged 4 commits into
masterfrom
396-version-and-static-gsl

Conversation

@matsen

@matsen matsen commented Sep 1, 2026 •

Copy link
Copy Markdown
Owner

Summary

#396 reports two regressions from the OCaml 5 / dune migration (#386):

  • Release binaries are dynamically linked against whatever GSL happens to
    be on the CI build image (libgsl.so.27 on Ubuntu 22.04), where they used
    to ship fully static (the old ocamlbuild config passed -ccopt -static;
    that flag was dropped in the dune migration and never replaced).
  • --version reports dev instead of the actual tag, because version.ml
    ran git describe at runtime — which always fails for a distributed
    binary with no .git directory.

Both are fixed for x86_64 and macOS, confirmed via the actual
build-release.yml CI workflow (triggered manually via workflow_dispatch
on this branch) as well as locally on Ubuntu 24.04:

$ ldd _build/default/pplacer.exe
	not a dynamic executable
$ ./_build/default/pplacer.exe --version
v1.1.alpha22-dirty

arm64 keeps its current dynamic linking (see the dedicated bullet below for
why full static isn't achievable there) — no regression, just not the
improvement the other platforms get.

  • dune: add link_flags restoring -ccopt -static, plus an explicit
    repeat of -lgsl -lgslcblas. Dune links our own GSL-calling C stubs
    (pam.c) after the library-derived link flags in the final link
    command, so the first -lgsl occurrence (from the gsl library
    dependency) has already resolved dynamically by the time our stubs'
    references appear; repeating it statically afterward is what actually
    removes the runtime GSL dependency.
  • common_src/dune: new rule generates version.ml at build time from
    $PPLACER_VERSION if set, else git describe, else "dev". Replaces
    the runtime-git-subprocess version.ml.
  • Dockerfile: the release build had its own copy of the dune stanza,
    hand-duplicated via a RUN echo ... >> dune heredoc, which was
    functionally identical to the root dune file it was overwriting (the
    swap did nothing). Removed it — the checked-in dune file already
    builds statically — and wired through PPLACER_VERSION.
  • scripts/build-docker.sh / build-release.yml: compute/pass
    PPLACER_VERSION (via git describe on the host, or the release tag
    for a release event) since the Docker build context has no .git dir.
  • scripts/build-common.sh: fixed create_static_dune_config, which put
    the static flag in the wrong dune stanza field (flags instead of
    link_flags) — it would never have produced a static binary. This path
    is exercised by scripts/build-linux.sh, which isn't used by CI but is
    a user-facing native-Linux build option.
  • Removed the dead pre-dune ocamlbuild build system (myocamlbuild.ml,
    root Makefile): confirmed no CI workflow, build script, Dockerfile, or
    documented instruction (BUILD.md points at dune build) invokes it —
    the project builds exclusively through dune. Its docs target was
    already broken (referenced a nonexistent gen_docs.native), and its own
    setup_git_version targeted the same common_src/version.ml path the
    new dune rule now generates, which would have collided with it. Also
    dropped the now-meaningless ocamlbuild-artifact entries from
    .gitignore (*.native, *.byte, *.mltop, TAGS, bin, etc.);
    docs/Makefile (Sphinx) and mcl's own generated build files are
    unrelated and untouched.
  • arm64 can't be fully static with the current toolchain. The first CI
    run failed on arm64 with ld: relocation truncated to fit: R_AARCH64_LD64_GOTPAGE_LO15 against symbol '__stack_chk_guard' ... too many GOT entries for -fpic. Tried -no-pie (a plausible first guess,
    since gcc on Ubuntu 22.04's aarch64 target defaults to PIE even under
    -static) — it made no difference, byte-for-byte identical error.
    Researched the actual cause: Ubuntu's prebuilt libc.a crt startup code
    (libc-start.o) accesses __stack_chk_guard through a short-range
    GOT-relative relocation (~1MB range) that overflows once a fully static
    binary this large (the whole OCaml runtime plus every module) is linked.
    That's baked into glibc's own prebuilt object — not something a link flag
    on our side can fix — and matches reports of the same error in other
    large aarch64 static builds (bpftrace, zeek). Real fixes (musl-based
    static libc, a different linker) are out of scope for version number / dynamic libs #396, whose
    libgsl.so.28 mention is an x86_64-style package name anyway. dune now
    detects the architecture at build time (uname -m, which correctly
    reports the emulated arch under the CI's QEMU cross-build) via a
    (:include ...) rule and skips -static/-no-pie only on
    aarch64/arm64; applied the same guard to build-linux.sh's native build
    path for consistency.

Closes #396.

Test plan

  • dune build produces pplacer.exe/guppy.exe/rppr.exe with
    ldd reporting "not a dynamic executable" (x86_64)
  • --version reports the git tag (v1.1.alpha22-dirty) instead of dev
  • PPLACER_VERSION=x dune build overrides the baked-in version (the
    path the Docker build actually uses)
  • Full test suite passes: ./_build/default/tests.exe → 224/224 OK
    (re-verified after every change in this PR)
  • build-release.yml on this branch (workflow_dispatch): linux/amd64
    and macOS succeeded across all runs; linux/arm64 failed twice on
    static linking (-static alone, then -static -no-pie) before
    landing on "skip static linking on arm64" — re-run in progress to
    confirm arm64 now builds (dynamically, as before) without error

🤖 Generated with Claude Code

matsen and others added 4 commits September 1, 2026 15:35
The OCaml 5 / dune migration (#386) dropped the old ocamlbuild `-static`
flag, so release binaries have been dynamically linked against whatever
GSL/zlib/sqlite3 happen to be on the build image (e.g. libgsl.so.27 on
Ubuntu 22.04) instead of shipping the fully static binary pplacer used
to ship. Separately, `--version` read `git describe` at runtime, which
always falls back to "dev" for a distributed binary with no .git dir.

- dune: add link_flags restoring `-ccopt -static`, plus an explicit
  repeat of `-lgsl -lgslcblas` (dune links our own GSL-calling C stubs
  after the library-derived link flags, so the first occurrence must
  be repeated afterward for the linker to resolve them).
- common_src/dune: new rule generates version.ml at build time from
  $PPLACER_VERSION (if set) or `git describe`, falling back to "dev".
  Replaces the old runtime-git-subprocess version.ml.
- Dockerfile: drop the now-redundant duplicate dune heredoc (root dune
  already builds statically); wire through PPLACER_VERSION.
- scripts/build-docker.sh, build-release.yml: compute/pass
  PPLACER_VERSION (git describe on the host, or the release tag) since
  the Docker build context has no .git dir.
- scripts/build-common.sh: fix create_static_dune_config, which put
  the static flag in the wrong dune stanza field (flags instead of
  link_flags) and so never actually produced a static binary.

Closes #396.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Verified no CI workflow, build script, Dockerfile, or documented build
instruction (BUILD.md points at `dune build`) invokes any Makefile
target or ocamlbuild directly; the project builds exclusively through
dune. The Makefile's own `docs` target was already broken (referenced
a nonexistent gen_docs.native). Its `setup_git_version` in
myocamlbuild.ml also targeted common_src/version.ml, the same path the
new dune rule generates, which would have collided with it. Also drop
the now-meaningless ocamlbuild-artifact entries from .gitignore
(*.native, *.byte, *.mltop, TAGS, bin, etc.); docs/Makefile (Sphinx)
and mcl's own generated build files are unrelated and untouched.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
gcc on Ubuntu 22.04's aarch64 target still defaults to PIE even with
-static, and static+PIE isn't properly supported there without
special glibc scrt1/rcrt1 support. That produced a link failure
specific to the arm64 CI leg:

  relocation truncated to fit: R_AARCH64_LD64_GOTPAGE_LO15 against
  symbol `__stack_chk_guard' ... too many GOT entries for -fpic

-no-pie is the standard fix. Verified it doesn't affect the x86_64
build (still static, --version and all 224 tests still pass).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
CI confirmed -no-pie doesn't fix the arm64 link failure either (same
byte-for-byte error). The actual cause: Ubuntu's prebuilt libc.a crt
startup code (libc-start.o) accesses __stack_chk_guard through a
short-range GOT-relative relocation (R_AARCH64_LD64_GOTPAGE_LO15, a
~1MB range) that overflows once a fully static binary this large is
linked. That's baked into glibc's own prebuilt object; no link flag on
our side can fix it (confirmed by searching prior reports of the same
error in other large aarch64 static builds: bpftrace, zeek, etc.).

Going further (musl-based static libc, alternate linker) is out of
scope for #396, whose "libgsl.so.28" mention is an x86_64-style
package name anyway. Detect the architecture at build time (uname -m,
which correctly reports the emulated arch under the CI's QEMU
cross-build) and skip -static/-no-pie only there, via a dune
`(:include ...)` rule so the flag list itself is generated rather than
hand-duplicated per platform. arm64 keeps its pre-existing dynamic
linking; x86_64 and macOS are unaffected.

Also applied the same aarch64 guard to build-linux.sh's native build
path for consistency, since the same glibc limitation would apply
there too.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@matsen
matsen merged commit 29f6885 into master Sep 2, 2026
3 checks passed
@matsen
matsen deleted the 396-version-and-static-gsl branch September 2, 2026 11:39
@colinbrislawn

colinbrislawn commented Oct 7, 2026 •

Copy link
Copy Markdown

Pplacer still requires GSL 2.8 in the current Bioconda macOS ARM64 package.

The recipe (https://github.com/bioconda/bioconda-recipes/blob/master/recipes/pplacer/meta.yaml) downloads the older v1.1.alpha22 binary and pins gsl=2.8. Its build script
(https://github.com/bioconda/bioconda-recipes/blob/master/recipes/pplacer/build.sh) rewrites:

/opt/homebrew/opt/gsl/lib/libgsl.28.dylib
→ @rpath/../lib/libgsl.28.dylib

That conflicts with r-base=4.5.2, which requires GSL 2.7.

Thanks for helping me (and my robots) troubleshoot the conda build. I've been using these tools in more places!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

version number / dynamic libs

2 participants