Skip to content

CI: pin and verify sccache via a shared composite action - #3248

Merged
juenglin merged 3 commits into
NVIDIA:mainfrom
juenglin:emudhra-sccache-pin
Oct 27, 2026
Merged

juenglin merged 3 commits into
NVIDIA:mainfrom
juenglin:emudhra-sccache-pin

Conversation

@juenglin

@juenglin juenglin commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

Description

Refs #3195

The eMudhra/emSign CA advisory is about TLS clients that still trust the affected roots. Refreshing the OS trust store and certifi (#3195) does not cover clients that ship their own root list. sccache is one of those clients in our CI: in the wheel builds it reads and writes compiled objects through the GitHub Actions cache over HTTPS. A release built with an embedded webpki-roots / webpki-root-certs list from before the fix would keep trusting the affected roots regardless of the runner's trust store.

Across mozilla/sccache and the downstream rapidsai/sccache fork (which rebases onto each upstream 0.x.0 release), only the 0.18.0 release series defers purely to the OS trust store:

Release series TLS root configuration Affected by eMudhra (emSign ECC Root CA - G3)?
<= v0.17.0 (including rapidsai/sccache <= v0.17.0-rapids.4) reqwest 0.12 with rustls-tls, embedding webpki-roots 1.0.2 Yes — trusts compiled-in webpki-roots 1.0.2 regardless of OS ca-certificates
v0.18.0 (mozilla/sccache v0.18.0, rapidsai/sccache v0.18.0-rapids.0..7) reqwest 0.13 with rustls-platform-verifier only (no embedded roots) No — uses the OS trust store
v0.19.0 (mozilla/sccache v0.19.0, commit 2eeea913) reqwest 0.13 calling .tls_certs_merge(...) with webpki-root-certs 1.0.9 Yes — merges compiled-in webpki-root-certs 1.0.9 (pre-1.0.10 fix) into the platform verifier, so updating OS ca-certificates does not distrust the root

Before this PR, neither workflow was protected against moving off 0.18.0:

  • build-wheel.yml piped rapidsai/sccache releases/latest directly into sudo tar -C /usr/local/bin with no version pin and no SHA-256 check. As soon as rapidsai/sccache rebases onto upstream v0.19.0 (unless webpki-root-certs is bumped to >= 1.0.10), releases/latest will silently serve a binary that trusts the compromised root again.
  • test-sdist-linux.yml used mozilla-actions/sccache-action with mozilla/sccache v0.18.0 (verified only against the .sha256 file fetched from the same release URL), where a routine bump to v0.19.0 would reintroduce the affected root.

This PR pins a known-good binary in one place and makes any change to it deliberate:

  • Add .github/actions/install-sccache, which downloads rapidsai/sccache v0.18.0-rapids.7 for x86_64 or aarch64, verifies its SHA-256 digest before extracting, installs it to /usr/local/bin, exports SCCACHE_PATH, and logs sccache --version.
  • Use this action in build-wheel.yml, replacing the unpinned releases/latest download.
  • Use this action in test-sdist-linux.yml, replacing mozilla-actions/sccache-action. This job only uses sccache's local disk cache, so it does not need the Actions cache token that action exported.
  • Document in the action how to regenerate the hashes, and require repeating the embedded-roots check on every version bump, with a grep recipe and pass criteria for the extracted binary. Also record why not to switch to mozilla/sccache v0.19.0.

The pinned hashes match both the downloaded tarballs and the .sha256 files published on the release.

Checklist

  • New or existing tests cover these changes. (Covered by the Linux wheel builds on linux-64 and linux-aarch64 and by the sdist job.)
  • The documentation is up to date with these changes.

Signed-off-by: Ralf Juengling <rjuengling@nvidia.com>
@juenglin juenglin added this to the cuda.bindings next milestone Oct 27, 2026
@juenglin juenglin added the CI/CD CI/CD infrastructure label Oct 27, 2026
@copy-pr-bot

copy-pr-bot Bot commented Oct 27, 2026

Copy link
Copy Markdown
Contributor

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@juenglin juenglin self-assigned this Oct 27, 2026
Signed-off-by: Ralf Juengling <rjuengling@nvidia.com>
@juenglin
juenglin marked this pull request as ready for review October 27, 2026 20:52
@coderabbitai

coderabbitai Bot commented Oct 27, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

No actionable comments were generated in the recent review.

Recent review info
Run configuration
Commits

Reviewing files that changed from the base of the PR and between 4b09d11 and 5762894.

Files selected for processing: 1
  • .github/actions/install-sccache/action.yml
Files skipped from review as they are similar to previous changes: 1
  • .github/actions/install-sccache/action.yml

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 9 remain after this review.

📝 Summary

Summary by CodeRabbit

  • Chores
    • Standardized compiler-cache setup across Linux build and source distribution workflows. The setup uses a fixed, checksum-verified version and supports Linux on x86_64 and aarch64; unsupported platforms or architectures are not supported by this setup.
    • Existing workflow build conditions remain unchanged, and the source distribution job continues to use local disk caching. No end-user-facing changes are included in this update.

A composite GitHub Action installs sccache v0.18.0-rapids.7 on supported Linux architectures. The Linux wheel and source distribution workflows now use this action instead of their previous sccache setup steps.

Changes: sccache setup

Layer / File(s) Summary
Pinned sccache installation
.github/actions/install-sccache/action.yml
The composite action supports Linux x86_64 and aarch64, verifies the pinned release archive with architecture-specific checksums, installs sccache, exports SCCACHE_PATH, and reports the installed version.
Linux workflow integration
.github/workflows/build-wheel.yml, .github/workflows/test-sdist-linux.yml
Both workflows call the local installation action. Their existing build conditions remain unchanged.

Suggested reviewers: rwgk, andy-jost, rparolin

Priority: Low

Change: Other

Merge Risk: Minimal · up to 57628

The pinned installer is wired into both Linux workflows, with no identified issue requiring a fix before merge.

- [ ] Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

webpki-roots 1.0.10 removes only emSign ECC Root CA - G3 and keeps the other emSign roots, so grepping for 'emSign' would reject a fixed root list.

Signed-off-by: Ralf Juengling <rjuengling@nvidia.com>
@juenglin

Copy link
Copy Markdown
Contributor Author

/ok to test 5762894

@jpascucci-nv jpascucci-nv left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@github-actions

github-actions Bot commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor
Doc Preview CI
Preview removed because the pull request was closed or merged.

@juenglin
juenglin merged commit 2828358 into NVIDIA:main Oct 27, 2026
119 checks passed
github-actions Bot pushed a commit that referenced this pull request Oct 28, 2026
Removed preview folders for the following PRs:
- PR #2713
- PR #3128
- PR #3149
- PR #3229
- PR #3244
- PR #3248
- PR #3249
- PR #3255
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CI/CD CI/CD infrastructure

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants