Skip to content

Containerize standalone tarball build #2595

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
jameslamb wants to merge 5 commits into
base: release/26.10
Choose a base branch
from
Open

Containerize standalone tarball build #2595

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
016e315
apply changes from https://github.com/NVIDIA/cuvs/pull/2443 to releas…
jameslamb Sep 11, 2026
6e6d6b6
first round of simplification and hardening
jameslamb Sep 11, 2026
fbe71f8
simplify GitHub Actions diff, no TTY for docker run
jameslamb Sep 11, 2026
8a9be2b
pass PARALLEL_LEVEL through
jameslamb Sep 11, 2026
1b46d1d
use the build cluster, it makes the builds go vroom vroom
jameslamb Sep 11, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
58 changes: 39 additions & 19 deletions .github/workflows/build.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -61,34 +61,54 @@ jobs:
contents: read
uses: rapidsai/shared-workflows/.github/workflows/compute-matrix.yaml@release/26.10
with:
build_type: pull-request
build_type: ${{ inputs.build_type || 'branch' }}

@jameslamb jameslamb Sep 11, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks like a copy-past mistake. Not a big deal since this is based on conda-cpp-build and those are the same for PRs and branch/nightly builds, but should fix this anyway for consistency.

matrix_name: conda-cpp-build
rocky8-clib-standalone-build:
needs: [build-details, rocky8-clib-standalone-build-matrix]
needs: [rocky8-clib-standalone-build-matrix]
runs-on: linux-${{ matrix.ARCH }}-cpu16
env:
RAPIDS_BUILD_TYPE: ${{ inputs.build_type || 'branch' }}
permissions:
actions: read
contents: read
id-token: write
packages: read
pull-requests: read
secrets: inherit # zizmor: ignore[secrets-inherit]
uses: rapidsai/shared-workflows/.github/workflows/custom-job.yaml@release/26.10
strategy:
fail-fast: false
matrix: ${{ fromJSON(needs.rocky8-clib-standalone-build-matrix.outputs.matrix) }}
with:
build_type: ${{ inputs.build_type || 'branch' }}
build-datetime: ${{ needs.build-details.outputs.build-datetime }}
branch: ${{ inputs.branch }}
arch: "${{matrix.ARCH}}"
date: ${{ inputs.date }}
container_image: "rapidsai/ci-wheel:26.10-cuda${{ matrix.CUDA_VER }}-${{ matrix.LINUX_VER }}-py${{ matrix.PY_VER }}"
node_type: "cpu16"
requires_license_builder: true

@jameslamb jameslamb Sep 11, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Notice that this PR doesn't have a git clone of spdx-license-builder like custom-job.yaml does when this input is provided.

Instead, build_standalone_c.sh now does this:

pip install 'git+https://github.com/rapidsai/spdx-license-builder.git'

I like that, it keeps things self-contained in the container (ha!). It'd be annoying to do an outside clone and thread it through to the container in a way that's also easy to replicate locally.

script: "ci/build_standalone_c.sh"
artifact-name: "libcuvs_c_${{ matrix.CUDA_VER }}_${{ matrix.ARCH }}.tar.gz"
file_to_upload: "libcuvs_c.tar.gz"
sha: ${{ inputs.sha }}
steps:

@jameslamb jameslamb Sep 11, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think @cjnolet originally pulled all these steps from custom-job.yaml out here individually because this script ends up running a docker run and custom-job.yaml requires running in a container.

Maybe this could be made to work with a docker-in-docker approach or making the container image optional for custom-job.yaml but IMO that doesn't need to become part of the scope here. I think this inlining is ok for right now.

- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
ref: ${{ inputs.sha || github.sha }}
fetch-depth: 0
persist-credentials: false
- name: Standardize repository information
uses: rapidsai/shared-actions/rapids-github-info@main
with:
branch: ${{ inputs.branch }}
date: ${{ inputs.date }}
sha: ${{ inputs.sha }}
- uses: aws-actions/configure-aws-credentials@cbe3b392738ccf3f987d68400dafcf4b0624a56c # v6.2.4
with:
role-to-assume: ${{ vars.AWS_ROLE_ARN }}
aws-region: ${{ vars.AWS_REGION }}
role-duration-seconds: 10800
- name: Setup sccache-dist
uses: rapidsai/shared-actions/setup-sccache-dist@main
with:
request-timeout: 7140
- name: Build standalone C tarball
env:
CUVS_TARBALL_CUDA_VERSION: ${{ matrix.CUDA_VER }}
CUVS_TARBALL_PYTHON_VERSION: ${{ matrix.PY_VER }}
run: |
./build.sh tarball --tarball-build-tests
- name: Upload artifacts
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: libcuvs_c_${{ matrix.CUDA_VER }}_${{ matrix.ARCH }}.tar.gz
path: libcuvs_c.tar.gz
rust-build-matrix:
needs: cpp-build
permissions:
Expand Down Expand Up @@ -128,7 +148,7 @@ jobs:
contents: read
uses: rapidsai/shared-workflows/.github/workflows/compute-matrix.yaml@release/26.10
with:
build_type: pull-request
build_type: ${{ inputs.build_type || 'branch' }}
matrix_name: conda-cpp-build
matrix_filter: map(select(.ARCH == "amd64"))
go-build:
Expand Down Expand Up @@ -161,7 +181,7 @@ jobs:
contents: read
uses: rapidsai/shared-workflows/.github/workflows/compute-matrix.yaml@release/26.10
with:
build_type: pull-request
build_type: ${{ inputs.build_type || 'branch' }}
matrix_name: conda-cpp-build
matrix_filter: map(select(.ARCH == "amd64"))
java-build:
Expand Down
45 changes: 30 additions & 15 deletions .github/workflows/pr.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -563,30 +563,45 @@ jobs:
build_type: pull-request
matrix_name: conda-cpp-build
rocky8-clib-standalone-build:
needs: [build-details, rocky8-clib-standalone-build-matrix]
needs: [rocky8-clib-standalone-build-matrix]
runs-on: linux-${{ matrix.ARCH }}-cpu16
env:
RAPIDS_BUILD_TYPE: pull-request
permissions:
actions: read
contents: read
id-token: write
packages: read
pull-requests: read
secrets: inherit # zizmor: ignore[secrets-inherit]
uses: rapidsai/shared-workflows/.github/workflows/custom-job.yaml@release/26.10
strategy:
fail-fast: false
matrix: ${{ fromJSON(needs.rocky8-clib-standalone-build-matrix.outputs.matrix) }}
with:
build_type: pull-request
build-datetime: ${{ needs.build-details.outputs.build-datetime }}
arch: "${{matrix.arch}}"
date: ${{ inputs.date }}_c
container_image: "rapidsai/ci-wheel:26.10-cuda${{ matrix.CUDA_VER }}-${{ matrix.LINUX_VER }}-py${{ matrix.PY_VER }}"
node_type: "cpu16"
requires_license_builder: true
script: "ci/build_standalone_c.sh --build-tests"
artifact-name: "libcuvs_c_${{ matrix.CUDA_VER }}_${{ matrix.ARCH }}.tar.gz"
file_to_upload: "libcuvs_c.tar.gz"
sha: ${{ inputs.sha }}
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
ref: ${{ inputs.sha || github.sha }}
fetch-depth: 0
persist-credentials: false
- uses: aws-actions/configure-aws-credentials@cbe3b392738ccf3f987d68400dafcf4b0624a56c # v6.2.4
with:
role-to-assume: ${{ vars.AWS_ROLE_ARN }}
aws-region: ${{ vars.AWS_REGION }}
role-duration-seconds: 10800
- name: Setup sccache-dist
uses: rapidsai/shared-actions/setup-sccache-dist@main
with:
request-timeout: 7140
- name: Build standalone C tarball
env:
CUVS_TARBALL_CUDA_VERSION: ${{ matrix.CUDA_VER }}
CUVS_TARBALL_PYTHON_VERSION: ${{ matrix.PY_VER }}
run: |
./build.sh tarball --tarball-build-tests
- name: Upload artifacts
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: libcuvs_c_${{ matrix.CUDA_VER }}_${{ matrix.ARCH }}.tar.gz
path: libcuvs_c.tar.gz
rocky8-clib-tests-matrix:
needs: [rocky8-clib-standalone-build, changed-files]
permissions:
Expand Down
2 changes: 2 additions & 0 deletions .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ dist/
python/**/**/*.cpp
python/cuvs/record.txt
log
infrequent_licenses/
.ipynb_checkpoints
.DS_Store
dask-worker-space/
Expand All @@ -29,6 +30,7 @@ temporary_*.json
rust/target/
rust/Cargo.lock
rmm_log.txt
*.tar.gz

## example notebooks
notebooks/simplewiki-2020-11-01-nq-distilbert-base-v1.pt
Expand Down
23 changes: 23 additions & 0 deletions Dockerfile.standalone
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
# SPDX-FileCopyrightText: Copyright (c) 2025-2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0
#
# Environment and runner for the standalone C build.
#
# End-user instructions: https://docs.nvidia.com/cuvs/installation#build-the-standalone-c-library-with-docker

ARG CUDA_VERSION=notset
ARG PYTHON_VERSION=notset
ARG RAPIDS_VERSION=notset
FROM rapidsai/ci-wheel:${RAPIDS_VERSION}-cuda${CUDA_VERSION}-rockylinux8-py${PYTHON_VERSION}

# Output directory for the standalone archive. Bind-mount a host folder here
# (e.g. -v $(pwd)/build:/build) so libcuvs_c.tar.gz is written to the host.
ENV CUVS_TARBALL_BUILD_OUTPUT_DIR=/build
ENV CUVS_TARBALL_IN_CONTAINER=1

# Run from repo root; the repo is expected to be bind-mounted at /workspace.
WORKDIR /workspace

# Run the build script, which writes the standalone tarball to the mounted output dir.
# Example: docker run -v $(pwd):/workspace -v $(pwd)/build:/build cuvs-standalone-c [--tarball-build-tests]
ENTRYPOINT ["/bin/bash", "-c", "ci/standalone_c/build_standalone_c.sh \"$@\"", "--"]
4 changes: 4 additions & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,10 @@ cuVS comes with pre-built packages that can be installed through [conda](https:/

Please see the [Build and Install Guide](https://docs.rapids.ai/api/cuvs/nightly/build/) for more information on installing the available cuVS packages and building from source.

### Standalone C library (Docker)

To build the standalone C library tarball (`libcuvs_c.tar.gz`) for use in your own C/C++ projects, see **[Building the standalone C library with Docker](https://docs.nvidia.com/cuvs/installation#build-the-standalone-c-library-with-docker)**.

## Getting Started

The following code snippets train an approximate nearest neighbors index for the CAGRA algorithm in the various different languages supported by cuVS.
Expand Down
57 changes: 56 additions & 1 deletion build.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ ARGS=$*
# scripts, and that this script resides in the repo dir!
REPODIR=$(cd "$(dirname "$0")"; pwd)

VALIDARGS="clean libcuvs python rust go java lucene docs tests bench-ann examples --uninstall -v -g -n --allgpuarch --no-mg --mnmg-tests --no-cpu --cpu-only --no-shared-libs --no-nvtx --show_depr_warn --incl-cache-stats --time -h --run-java-tests --build-java-examples"
VALIDARGS="clean libcuvs python rust go java lucene docs tests bench-ann examples tarball --tarball-build-tests --uninstall -v -g -n --allgpuarch --no-mg --mnmg-tests --no-cpu --cpu-only --no-shared-libs --no-nvtx --show_depr_warn --incl-cache-stats --time -h --run-java-tests --build-java-examples"
HELP="$0 [<target> ...] [<flag> ...] [--cmake-args=\"<args>\"] [--cache-tool=<tool>] [--limit-tests=<targets>] [--limit-bench-ann=<targets>] [--build-metrics=<filename>]
where <target> is:
clean - remove all existing build artifacts and configuration (start over)
Expand All @@ -34,6 +34,7 @@ HELP="$0 [<target> ...] [<flag> ...] [--cmake-args=\"<args>\"] [--cache-tool=<to
tests - build the tests
bench-ann - build end-to-end ann benchmarks
examples - build the examples
tarball - build the standalone C library tarball with Docker

and <flag> is:
-v - verbose build mode
Expand All @@ -59,6 +60,7 @@ HELP="$0 [<target> ...] [<flag> ...] [--cmake-args=\"<args>\"] [--cache-tool=<to
--run-java-tests - run Java tests after building
--build-java-examples - also build the examples/java/cuvs-java and examples/java/cuvs-lucene
projects against the jars just built (with the 'java'/'lucene' targets)
--tarball-build-tests - include the C library tests in the standalone tarball
--build-metrics - filename for generating build metrics report for libcuvs
--incl-cache-stats - include cache statistics in build metrics report
--cmake-args=\\\"<args>\\\" - pass arbitrary list of CMake configuration options (escape all quotes in argument)
Expand Down Expand Up @@ -607,3 +609,56 @@ if hasArg examples; then
./build.sh
popd
fi

################################################################################
# Build the standalone C library tarball (if requested)

if hasArg tarball; then
if [[ "${CUVS_TARBALL_IN_CONTAINER:-0}" == "1" ]]; then
CUVS_TARBALL_BUILD_OUTPUT_DIR="${CUVS_TARBALL_BUILD_OUTPUT_DIR:-${REPODIR}}"
tar czf "${CUVS_TARBALL_BUILD_OUTPUT_DIR}/libcuvs_c.tar.gz" -C "${REPODIR}/c/build/install" .
else
CUVS_TARBALL_CUDA_VERSION="${CUVS_TARBALL_CUDA_VERSION:-13.3.0}"
CUVS_TARBALL_PYTHON_VERSION="${CUVS_TARBALL_PYTHON_VERSION:-3.14}"
CUVS_TARBALL_BUILD_OUTPUT_DIR="${CUVS_TARBALL_BUILD_OUTPUT_DIR:-${REPODIR}/build}"
CUVS_TARBALL_IMAGE_NAME="nvidia/cuvs-standalone-c:local-cuda${CUVS_TARBALL_CUDA_VERSION}-py${CUVS_TARBALL_PYTHON_VERSION}"

mkdir -p "${CUVS_TARBALL_BUILD_OUTPUT_DIR}"
BUILD_OUTPUT_DIR_ABS=$(realpath "${CUVS_TARBALL_BUILD_OUTPUT_DIR}")

echo "Building Docker image ${CUVS_TARBALL_IMAGE_NAME} (CUDA ${CUVS_TARBALL_CUDA_VERSION}, Python ${CUVS_TARBALL_PYTHON_VERSION})..."
docker build \
-f "${REPODIR}/Dockerfile.standalone" \
--build-arg CUDA_VERSION="${CUVS_TARBALL_CUDA_VERSION}" \
--build-arg PYTHON_VERSION="${CUVS_TARBALL_PYTHON_VERSION}" \
--build-arg RAPIDS_VERSION="${RAPIDS_VERSION_MAJOR_MINOR}" \
-t "${CUVS_TARBALL_IMAGE_NAME}" \
"${REPODIR}"

# optionally pass additional arguments through to the container's entrypoint
DOCKER_ENTRYPOINT_ARGS=()
if hasArg --tarball-build-tests; then
DOCKER_ENTRYPOINT_ARGS+=(--tarball-build-tests)
fi

echo "Running standalone C build in container..."
# NOTE: the '--env-file' trick for AWS credentials keeps them out of 'ps aux' / 'docker ps' output
docker run \
--rm \
-v "${REPODIR}:/workspace:rw" \
-v "${BUILD_OUTPUT_DIR_ABS}:/build:rw" \
--env CI="${CI:-false}" \
--env PARALLEL_LEVEL="${PARALLEL_LEVEL}" \
--env RAPIDS_BUILD_TYPE="${RAPIDS_BUILD_TYPE:-}" \
--env-file <(env | grep -E '^AWS_(ACCESS_KEY_ID|SECRET_ACCESS_KEY|SESSION_TOKEN)=') \
--env-file <(env | grep -E '^SCCACHE_.*=') \
Comment on lines +653 to +654

@jameslamb jameslamb Sep 11, 2026 •
edited
Loading

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As the comment I wrote a few lines up here say... this --env-file trick keeps these sensitive values out of command history and ps types of output.

These lines are primarily here for CI.

  • AWS_* for sccache
  • SCCACHE_* for sccache-dist (the build cluster)

I confirmed in the logs that sccache is working well, haven't confirmed that the build cluster's being used. Might not be because I'd already done some runs and we might get 100% cache hit rates until something changes and requires a recompile.

right now I see:

...
Cache hits rate                           100.00 %
Cache hits rate (c [gcc])                 100.00 %
Cache hits rate (c++ [gcc])               100.00 %
Cache hits rate (cuda [nvcc])             100.00 %
...

(build link)

"${CUVS_TARBALL_IMAGE_NAME}" \
"${DOCKER_ENTRYPOINT_ARGS[@]}"

cp -v "${BUILD_OUTPUT_DIR_ABS}/libcuvs_c.tar.gz" "${REPODIR}/libcuvs_c.tar.gz"
echo "Copied libcuvs_c.tar.gz to ${REPODIR}/libcuvs_c.tar.gz"
fi

# print contents of the tarball
ls -lh "${CUVS_TARBALL_BUILD_OUTPUT_DIR}/libcuvs_c.tar.gz"
fi
61 changes: 19 additions & 42 deletions ci/build_standalone_c.sh → ci/standalone_c/build_standalone_c.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,67 +1,47 @@
#!/bin/bash
# SPDX-FileCopyrightText: Copyright (c) 2025-2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0
#
# Build script for the standalone C library.
#
# Use 'Dockerfile.standalone' to build an image with all the prerequisites
# and run this in a container.

set -euo pipefail

SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
REPO_ROOT="$(cd "${SCRIPT_DIR}/../.." && pwd)"

TOOLSET_VERSION=14
NINJA_VERSION=v1.13.1

BUILD_C_LIB_TESTS="OFF"
if [[ "${1:-}" == "--build-tests" ]]; then
if [[ "${1:-}" == "--tarball-build-tests" ]]; then
BUILD_C_LIB_TESTS="ON"
fi

dnf install -y \
patch \
tar \
unzip \
wget

if ! command -V ninja >/dev/null 2>&1; then
case "$(uname -m)" in
x86_64)
wget --no-hsts -q -O /tmp/ninja-linux.zip "https://github.com/ninja-build/ninja/releases/download/${NINJA_VERSION}/ninja-linux.zip";
;;
aarch64)
wget --no-hsts -q -O /tmp/ninja-linux.zip "https://github.com/ninja-build/ninja/releases/download/${NINJA_VERSION}/ninja-linux-aarch64.zip";
;;
*)
echo "Unrecognized platform '$(uname -m)'" >&2
exit 1
;;
esac
unzip -d /usr/bin /tmp/ninja-linux.zip
chmod +x /usr/bin/ninja
rm /tmp/ninja-linux.zip
fi

source rapids-install-sccache
source rapids-configure-sccache
source rapids-datetime-string

rapids-pip-retry install cmake
PIP_PACKAGES=(
'cmake>=4.0'
'git+https://github.com/rapidsai/spdx-license-builder.git'
'ninja>=1.13'
)

RAPIDS_CUDA_MAJOR="${RAPIDS_CUDA_VERSION%%.*}"
if [[ "${RAPIDS_CUDA_MAJOR}" == "13" ]]; then
rapids-pip-retry install cuda-tile "cuda-toolkit[tileiras]==${RAPIDS_CUDA_VERSION%.*}.*"
PIP_PACKAGES+=(
cuda-tile
)
fi

rapids-pip-retry install "${PIP_PACKAGES[@]}"
pyenv rehash

rapids-print-env

rapids-logger "Begin cpp build"

sccache --stop-server 2>/dev/null || true

RAPIDS_PACKAGE_VERSION=$(rapids-generate-version)
export RAPIDS_PACKAGE_VERSION

RAPIDS_ARTIFACTS_DIR=${RAPIDS_ARTIFACTS_DIR:-"${PWD}/artifacts"}
mkdir -p "${RAPIDS_ARTIFACTS_DIR}"
export RAPIDS_ARTIFACTS_DIR

Comment on lines -58 to -64

@jameslamb jameslamb Sep 11, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks copied over from conda build scripts. None of this stuff is needed for building the tarball.

scl enable gcc-toolset-${TOOLSET_VERSION} -- \
cmake -S cpp -B cpp/build/ -GNinja \
-DCMAKE_CUDA_HOST_COMPILER=/opt/rh/gcc-toolset-${TOOLSET_VERSION}/root/usr/bin/gcc \
Expand Down Expand Up @@ -107,11 +87,8 @@ if [ "${BUILD_C_LIB_TESTS}" != "OFF" ]; then
cmake --install c/build --prefix c/build/install --component testing
fi


rapids-logger "Begin gathering licenses"
rapids-pip-retry install git+https://github.com/rapidsai/spdx-license-builder.git
license-builder . --output-json c/build/install/licenses.json --output-txt c/build/install/LICENSE

rapids-logger "Begin c tarball creation"
tar czf libcuvs_c.tar.gz -C c/build/install/ .
ls -lh libcuvs_c.tar.gz
"${REPO_ROOT}/build.sh" tarball
Loading
Loading