Skip to content

Split libcuopt wheel into separate routing/LP packages for PyPI publication #1635

Description

@ramakrishnap-nv

Background

Publishing cuOpt to pypi.org (instead of only pypi.nvidia.com) would simplify installation, remove the need for --extra-index-url, and unblock security-conscious users who restrict installs to the main PyPI index.

The blocker is package size. The current libcuopt-cu12 wheel is ~542 MB compressed. PyPI enforces a 1 GB per-package limit and a soft 100 GiB per-project limit; headroom exists today but is not comfortable enough to commit to indefinitely, especially with features like multi-GPU PDLP added recently.

What this issue tracks

Split the single libcuopt wheel into separate wheels along solver boundaries:

  • libcuopt-base-cu12libcuopt_base.so; dependency of every other package
  • libcuopt-routing-cu12 — routing/VRP engine (libcuopt_routing.so)
  • libcuopt-lp-cu12 — LP/MILP/QP solver (libcuopt_lp.so)
  • libcuopt-grpc-cu12 — gRPC client (libcuopt_grpc.so); depends on both engines, see below
  • libcuopt-cu12 — metapackage depending on all of the above, so existing installs are unaffected

libcuopt_base.so must be its own package rather than duplicated into the routing and LP wheels — two wheels owning the same installed path conflict.

The corresponding Python meta-packages (cuopt-vrp, cuopt-lp, etc.) would declare only the lib package they need, so users who only use routing don't pull in the LP solver and vice versa.

Component boundaries (measured on #1622, stripped)

Component Size External deps beyond rmm / rapids_logger
cuopt_base 1.3 MB
cuopt_routing 37 MB cublas
cuopt_lp 59 MB cublas, cusparse, nccl, cudss, TBB
cuopt_grpc 1.0 MB grpc, protobuf, abseil

readelf -d confirms libcuopt_routing.so has no DT_NEEDED on libcuopt_lp.so and vice versa — the two engines are independent, so no C++ untangling is required.

libcuopt_grpc.so is the exception: it links both engines (proto mappers for LP and routing). As specified above, libcuopt-grpc-cu12 therefore pulls in everything. Either accept that, or split its LP and routing mappers into separate translation units so a routing-only gRPC client is possible. This is the only part of the split with real C++ work behind it.

Secondary benefit, distinct from the PyPI size limit: cusparse, nccl, and cudss are LP-only and ship as separate nvidia-* wheels — ~1.2 GB of uncompressed on-disk libraries in a local build environment. A routing-only install currently pulls all of them and never calls into them. This does not affect the per-package limit that motivates this issue, but it is a large win for end-user install footprint.

Prerequisites

PR #1622 (feat/split-routing-lp-libs) splits the monolithic libcuopt.so into component .so files (libcuopt_base, libcuopt_routing, libcuopt_lp, libcuopt_grpc). That clean boundary is required before this package split is possible without rearchitecting the C++ build. This issue should not be started until that PR merges.

Prior art

A POC splitting cuOpt into cuopt-common + per-solver packages was explored previously but deferred for other priorities. The .so boundary work in #1622 makes this cleaner than the prior attempt.

Work involved

  • Relink the Cython extension modules against their actual components. All four of python/cuopt/cuopt/{routing,linear_programming/solver,grpc/linear_programming,distance_engine}/CMakeLists.txt currently set linked_libraries cuopt::cuopt. Until they point at cuopt::routing / cuopt::lp / cuopt::grpc, every extension pulls the full graph and the optional-dependency groups achieve nothing. Worth doing first as a standalone PR: it is small, it de-risks the packaging work, and a mislinked module fails to link rather than silently over-depending.
  • Give each component its own install component; install(TARGETS ${CUOPT_COMPONENT_TARGETS} ... COMPONENT runtime) currently stages them together.
  • Split header installation — install(DIRECTORY include/cuopt/) is monolithic today, so dev headers need to follow their component.
  • New ci/build_wheel_libcuopt_{base,routing,lp,grpc}.sh scripts (modeled on existing ci/build_wheel_libcuopt.sh)
  • New python/libcuopt_{base,routing,lp,grpc}/ package directories with pyproject.toml
  • Per-package dependency sets in dependencies.yaml — this is where cudss/nccl/cusparse get attached to LP only
  • Update python/cuopt/pyproject.toml optional-dependency groups to reference the split packages
  • Update conda recipes accordingly
  • Verify compressed sizes stay well under 750 MB per package (per RAPIDS guidance)
  • Coordinate with RAPIDS build-infra on PyPI org registration (see rapidsai/build-infra#356)

Target

26.10 (too late for 26.08)

Metadata

Metadata

Assignees

Labels

awaiting responseThis expects a response from maintainer or contributor depending on who requested in last comment.

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions