Skip to content

CI: test and document both bindings majors on main - #3176

Open
rwgk wants to merge 18 commits into
rwgk/cb12-on-main-04-ci-buildsfrom
rwgk/cb12-on-main-05-ci-tests-docs
Open

rwgk wants to merge 18 commits into
rwgk/cb12-on-main-04-ci-buildsfrom
rwgk/cb12-on-main-05-ci-tests-docs

Conversation

@rwgk

@rwgk rwgk commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Exercise the CUDA 12 and CUDA 13 wheels built from main, with explicit component and dependency-source coverage in the PR checks view.

  • Name matrix checks for the enabled suites and CUDA major, distinguishing this run's bindings artifacts from published bindings and local from wheel-installed toolkits. Metapackage suites display meta12 and meta13.
  • Share matrix selection in one Python function across Linux and Windows. Preserve caller-provided jq filters and reject filters that remove required local-wheel coverage. Intentional nightly subsets can continue using published floor bindings.
  • Keep matrix data at the source revision selected for the build/test run, while loading the helper from a separate workflow-control checkout. The control revision supplies the workflow and orchestration tools, and can be newer for nightly/manual runs. Unaffected components may still reuse baseline wheels; the workplan identifies rebuild/reuse choices and the exact baseline.
  • Preserve mandatory Python 3.15/3.15t coverage, current main's JUnit reports, log folding, retries, and Windows ARM64 behavior. Validate metapackage installability without requiring an unrelated Core build for a bindings release.
  • Route Pixi source checks, coverage, and documentation to the intended root. Both CPU build-smoke and GPU full-test prepare maintenance SCM versions before source builds, with their required tooling installed. Build CUDA 13 documentation against the checkout's bindings API, retaining the existing Lychee and rendered-link checks.
  • Remove the orphaned lookup-run-id --artifact option and reduce CI-tool tests to silent-failure outcomes. Scoped test guidance records this policy; package/runtime tests retain their own rules.
  • Refresh source-build, documentation-build, and CI instructions, including transferring handwritten fixes with format-patch / git am, using a compatible first nightly artifact source, and rerunning historical releases/docs from their historical workflow revision. Correct the documented CUDA 12 driver baseline; this adds no runtime driver-version gate.

Shared GPU jobs keep runner usage bounded; their names and coverage summaries expose which suites actually run.

CUDA 12 NVML test maintenance accompanies CI activation: device visibility, PCI-information and Orin/Thor naming accommodations (#1590, #2065, #2437, #2788); serialized initialization and the driver 13.4 refcount skip (#2229, #2512); removal of the unstable energy-timing assertion (#2644); and the field-enum spelling correction (#2560). Driver probing occurs during test execution. These changes affect tests and their helper, not the installed runtime.

Process-global coredump tests are serialized for free-threaded runs, carrying Mike's #3226 fix and the matching CUDA 12 marker. Tests that only manipulate local helper storage remain parallel. This addresses the coredump setting race seen in the previous full runs without serializing unrelated tests.

Reading the Test Coverage

bindings PR means the selected root's artifact supplied by this CI run; the workplan identifies whether it was rebuilt or reused from the exact baseline. bindings floor means the oldest published bindings supported by the Core wheel for that CUDA major. Floor rows test Core or the selected nightly integration, without running the bindings or metapackage suites. wheels versus local CTK describes how toolkit libraries are supplied, independently of the bindings source.

Development documentation builds the current CUDA 13 API from the checkout. The local CUDA 12 Pixi docs environment defaults to a published 12.9.* bindings package; install a matching source build when documenting API changes in the checkout. It needs a separate environment because both roots install the same package. Tagged releases publish their selected version without replacing shared latest pages or version selectors; the full publication procedure follows in #3178.

Stack / Review Boundary

Previous: #3175. Next: #3177. The stack overview lists all six open PRs and the intended workflow.

#3174 through #3176 are intended to land together, with separate review scopes and cumulative CI at #3176; the first two are incomplete intermediate states. Release routing (#3177), maintenance policy (#3178), and the runtime fixes (#3182) follow that group. The complete proposed source tree is at #3182.

Baseline: main b4a5e5513a1f.

Release Boundary

Future CUDA 13 releases tagged from this state retain the existing release flow. The transitional backport_branch field keeps legacy release-note and backport consumers working; toolkit selection uses the package-root registry. Releasing CUDA 12 from main is enabled separately by #3177.

Starting at this layer, release-documentation reruns and dry-runs against pre-registry tags require compatible historical tooling: the new docs workflow reads package configuration from the release tag. The historical rerun recipe explicitly dispatches release.yml from 13.4.x for its releases, including a v13.4.4 documentation dry run.

Nightly artifact auto-detection needs the first successful, complete main CI run containing the new registry and both bindings majors. Until then it reports an incompatible older source instead of guessing its layout. The rollout instructions also describe using a successful compatible run explicitly; the override selects both artifacts and their source revision.

Suggested Review Order

  1. ci/tools/select_test_matrix.py - shared source selection, suite selection, and local-wheel coverage protection.
  2. .github/workflows/test-wheel-linux.yml and test-wheel-windows.yml - thin matrix wrappers and execution of selected suites.
  3. ci/tools/run-tests - matching-root tests and Core's published bindings floor.
  4. .github/workflows/ci-pixi-source-test.yml - source imports, Cython placement, and documentation smoke checks.
  5. .github/workflows/build-docs.yml - selected package roots and separation of release sources from workflow tooling.
  6. ci/README.md - how to read coverage, baseline reuse, and source/control revisions.

Counterpart Review

Inspected both bindings roots' test setup, README, install/support documentation, coredump tests, and documentation environments, plus Core floor testing and the shared Linux/Windows matrix rules. Both READMEs now use the same --group test recipe. CUDA 12 preserves its legacy sample/NVML layout and requires its own environment. All three marked pytest-bootstrap blocks and all four shared build-helper copies match byte-for-byte.

Validation

Full hosted CI passed for this exact head, feb25b1b3d3c1744b680bf54ff10986ed4e59953: run 40426114619. This validates the cumulative #3174-#3176 integration, including native builds, GPU tests, source/sdist checks, and documentation.

328 CI-tool tests pass in a full Git checkout, with full local pre-commit run --all-files, including actionlint, Lychee, generated-file seals, registry checks, mypy, and security checks. The suite includes Git-based Pixi workspace discovery; it is not an archive-only test suite.

Local semantic checks cover plan selection, release routing, SCM milestones, Core ABI variants, bindings-source selection, and mandatory local-wheel coverage. A one-time comparison matched the former jq selector across platforms and run modes; the workflow-control helper was also exercised with an older artifact source that lacks it. Six extracted shell cases cover Pixi environment selection and nightly-source diagnostics, alongside 15 existing SCM/registry cases. These shell cases do not constitute a separate hosted nightly run.

Hosted artifact-reuse rehearsals. Two temporary draft PRs use the production resolver unchanged against successful baseline run 40426114619, at feb25b1b3d3c1744b680bf54ff10986ed4e59953. Both actual workplans exactly match the expected selective plans; neither fell back to a full rebuild.

  • CUDA 12 source-change case, #3237: run 40456909677 rebuilt CUDA 12 bindings/metapackage wheels while reusing Core, Pathfinder, and CUDA 13 wheels. All 28 build jobs passed, including bindings and Core Cython test-extension builds on Linux x86-64, Linux ARM64, and Windows. A re-uploaded Linux CPython 3.12 Core wheel is byte-identical to its baseline wheel. All 20 selected GPU tests (13 Linux and 7 Windows) and the final aggregate passed on the first attempt. The workflow is fully green; no fixes or retries were needed.
  • Linux test-workflow case, #3238: run 40460578171 skipped every package-wheel build, downloaded and re-uploaded the baseline packages, and successfully built all selected Cython test extensions and Core test binaries. All 45 expected reused package artifacts exist in the new run. All 42 selected GPU tests (24 x86-64 plus 18 ARM64 Linux rows, covering both CUDA majors) and the final aggregate passed on the first attempt. The seven planned Windows, sdist, and Core API skips were accepted.

Both complete hosted workflows passed on their first attempts, including documentation/link checks and their final aggregates. No fixes or retries were needed. These probes cover the two scenarios above; no separate Core-only or metapackage-only hosted probe is claimed.

Full hosted CI also passed for the complete stack at the unchanged #3182 head 775392e1a6b372b8d046a95466513bd114b05f6a: run 40456627846, attempt 2. The only retry was a self-hosted H100 runner disconnect during driver setup, before package tests began. Its replacement passed, together with the final gate; exact-head auxiliary checks are also green or intentionally skipped. This tests the cumulative stack, rather than hosted release publication or backport automation.

Checklist

  • Focused tests protect selection, packaging, and release outcomes that could fail silently.
  • This description identifies the layer's scope and validation boundary.

@rwgk rwgk added this to the cuda.bindings next milestone Oct 21, 2026
@rwgk rwgk added enhancement Any code-related improvements CI/CD CI/CD infrastructure cuda.bindings Everything related to the cuda.bindings module labels Oct 21, 2026
@rwgk rwgk self-assigned this Oct 21, 2026
@copy-pr-bot

copy-pr-bot Bot commented Oct 21, 2026

Copy link
Copy Markdown
Contributor

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

@rwgk

rwgk commented Oct 22, 2026

Copy link
Copy Markdown
Contributor Author

/ok to test 9b8ac76

@rwgk

rwgk commented Oct 22, 2026

Copy link
Copy Markdown
Contributor Author

/ok to test 19865b0

@rwgk
rwgk force-pushed the rwgk/cb12-on-main-05-ci-tests-docs branch from d20b7f2 to 45ffc98 Compare October 26, 2026 21:35
@github-actions github-actions Bot added the cuda.core Everything related to the cuda.core module label Oct 26, 2026
@rwgk

rwgk commented Oct 26, 2026

Copy link
Copy Markdown
Contributor Author

codex GPT-6.1-Sol ultra on behalf of Ralf:

Thanks for the detailed review. The updated #3174–#3176 review group is now published, with these changes:

  • Smaller, outcome-focused tests. ci/tools/tests/AGENTS.md now records the proposed policy specifically for CI tooling. The revised suites retain plan selection, release routing, SCM progression, complete Core ABI variants, and local-wheel coverage. Workflow-text extraction, cosmetic-label assertions, exact pip/gh command checks, redundant schema/error cases, and brittle fixtures tied to live version pins were removed.
  • Shared Python matrix selection. Linux and Windows call one directly testable selector. Caller-provided jq filters remain supported. Matrix data comes from the artifact source; the helper comes from the workflow-control checkout, so older artifact sources need not contain newly introduced tooling. An independent comparison matched the former jq behavior in 63 actual-matrix scenarios.
  • Less shell and dead code. Python now writes the two build environments. The unused lookup-run-id --artifact option, validation, and example are removed.
  • Core reuse. A bindings-only change reuses the unchanged merged Core wheel from a usable exact baseline, while retaining Core tests against the affected bindings major. Core/shared changes and unavailable baselines still rebuild both variants.
  • Updated contributor and agent guidance. Instructions cover source builds, separate documentation environments, release publication, and transferring a scoped patch between roots with format-patch / git am. Implementation and review must inspect the counterpart root in either direction.

Two suggested simplifications remain follow-ups. The synthetic version bootstrap is temporarily retained with its actual callers in #3175: introducing a retained maintenance milestone should follow landing the validated per-line SCM configuration, after which the helper and its callers can be removed. No retained tag was pushed. Extracting the duplicated build blocks into a composite action would also change the sequencing around toolkit switching, Core wheel merging, artifact reuse, and extension cleanup; that refactor is deferred.

The verbose Markdown plan table is gone. Concise component/major job names remain to meet the requested visibility in the PR checks view; the workplan summary exposes expandable JSON, and tests do not depend on the cosmetic names.

The stack is rebased onto main 2e6aa4e9ef898b849b0724679d85bbda2a05a758. Local validation passes 328 CI-tool tests at #3176 and 341 at #3182, with full pre-commit passing at both heads, including actionlint and Lychee. The runtime backport diff at #3182 is unchanged. Fresh full CI is being requested for the cumulative #3176 and #3182 heads; these local results do not claim that the new hosted runs have passed.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to GitHub limitations.

⚠️ Outside diff range comments (1)

Major · important: Install or resolve the published pathfinder before… · test-wheel-linux.yml:384-387

.github/workflows/test-wheel-linux.yml:384-387
🩺 Stability & Availability | 🟠 Major | ⚡ Quick win

important: Install or resolve the published pathfinder before reading its installed version.

When PATHFINDER_SOURCE is not artifact, the metapackage install step calls version("cuda-pathfinder") without a prior pathfinder install. A metapackage-only workplan can select a published pathfinder source while it skips the pathfinder, bindings, and Core suites. No earlier step installs cuda-pathfinder for that source. The lookup then raises PackageNotFoundError, and the install step fails. The Windows workflow has the same call at line 370.

Use the published version from the workplan or configuration, or install the published pathfinder before this step.

🧹 Nitpick comments: 1
ci/tools/tests/test_test_sources.py: 1

96-96: Trivial · Maintainability & Code Quality · Quick win

suggestion: Assert the selected bindings version, not the pip command.

The pip stub only records text, so this assertion checks an exact command rather than the effective bindings version. Keep coverage for an incorrect floor if that result could be silent, but make the fixture record the selected installation outcome and assert it. As per coding guidelines, “Assert those outcomes rather than ... intermediate command order”; the guidance also identifies “exact pip command lines” as unsuitable test targets.

Source: Coding guidelines


Review info
Run configuration
Commits

Reviewing files that changed from the base of the PR and between d20b7f2 and 45ffc98.

Files ignored due to path filters: 1
  • cuda_bindings_13/pixi.lock is excluded by !**/*.lock
Files selected for processing: 18
  • .github/workflows/test-wheel-linux.yml
  • .github/workflows/test-wheel-windows.yml
  • CONTRIBUTING.md
  • ci/README.md
  • ci/ci-pipeline.mmd
  • ci/tools/lookup-run-id
  • ci/tools/select_test_matrix.py
  • ci/tools/tests/test_merge_cuda_core_wheels.py
  • ci/tools/tests/test_select_test_matrix.py
  • ci/tools/tests/test_test_sources.py
  • cuda_bindings_12/docs/README.md
  • cuda_bindings_12/docs/source/install.md
  • cuda_bindings_13/README.md
  • cuda_bindings_13/docs/README.md
  • cuda_bindings_13/docs/source/install.rst
  • cuda_bindings_13/docs/source/support.rst
  • cuda_core/docs/source/install.rst
  • cuda_python/docs/README.md

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 11 remain after this review.

@rwgk

rwgk commented Oct 26, 2026 •

Copy link
Copy Markdown
Contributor Author

Fable 5.1 "from scratch" review (claude session ID: f7f67118-ae33-4305-b4c7-8128424b4064):

Review complete. Everything below is based only on main at 2e6aa4e and the PR stack itself. All of Matt's earlier review threads on 3174 to 3176 are resolved, and I did not re-raise them.

Defects and risks in 3174 to 3176

Ordered by severity. Nothing I found is a hard blocker; the first three deserve a decision before the group merges.

  1. Undocumented wheel-payload change for CUDA 12 (3174). cuda_bindings_12/pyproject.toml:84 copies include-package-data = false plus the package-data table from the 13 root. The commit that introduced that setting on main, Reduce wheel size by excluding source files #1588, is not an ancestor of 12.9.x, so 12.9.10 wheels ship every .pyx, .pyx.in, .pxd.in and .pxi.in under cuda/ via MANIFEST.in. A 12.9.11 wheel built from this stack drops 34 .pyx files and 10 .in templates. Probably desirable, but neither the PR text nor the 12.X.Y release notes mention it, and the description frames 3174 as not changing the maintenance line's behavior.

  2. The artifact-reuse path has never run in hosted CI (3175, 3176). Every hosted run for the stack was a force-all run. Stacked PRs look for a push run on their base branch at the merge base, see .github/workflows/ci.yml:199, and rwgk/* bases never get push runs. Main's pre-stack runs lack cuda-python-wheel-cuda* and CUDA 12 bindings artifacts, so 3174 also falls back to force-all. The reuse-specific logic in build-wheel.yml is covered only by planner unit tests: downloading baseline wheels, re-uploading them under the new SHA, building Core Cython tests from a downloaded merged wheel, building test binaries with whichever CTK happens to be active, and the checks aggregator seeing one sdist matrix leg whose inner job is skipped. I traced the bindings-12-only, core-only, metapackage-only and test-infra-only scenarios by hand and found no condition errors, but the first real exercise will be the first PR after the merge push completes. A deliberate rehearsal with two trivial PRs would be cheap insurance, and the Validation sections should state this limitation.

  3. The cited hosted runs are unfinished and contain failures. Run 40405590714 for the 3176 head has one failed job on win-64 h100 x2 where no step executed, which looks runner-level. Run 40406327331 for the 3182 head has three failures, two of them in the step that runs cuda.bindings tests on CUDA 13 rows, win-64 py3.15t and linux-aarch64 py3.14t. Logs are not retrievable until the runs leave the queue, so I could not triage them. The descriptions present these runs as validation.

  4. Docs re-runs for pre-registry tags fail from main's workflows (3176). .github/workflows/build-docs.yml:83 now resolves the package from the tagged source's registry, and release.yml is unchanged in this group. That includes the live 13.4.x branch, which carries the old schema and shipped v13.4.4 on October 13. The description acknowledges "compatible historical tooling" but no document names 13.4.x or says that its releases and docs re-runs must be dispatched from that branch. Likewise .github/workflows/ci-nightly.yml:123 reads the registry from the newest successful main run, so nightly fails until the first post-merge push run succeeds.

  5. Cross-root documentation miss (3176). cuda_bindings_13/README.md moved to --group test, but cuda_bindings_12/README.md:18 still documents the [test] extra, whose dependency list lacks the pytest-randomly and pytest-repeat plugins that ci/tools/run-tests flags assume. This is precisely the counterpart check the new root AGENTS.md rule mandates, and none of the three PR descriptions contains the "counterpart files inspected" statement that rule requires.

  6. Nits. .pre-commit-config.yaml:46 still triggers the Core floor hook on ci/versions.yml although the hook no longer reads it. .pre-commit-config.yaml:166 broadens the large-file exclusion to the 13 root's driver.pyx and runtime.pyx.in, which did not need it. .github/workflows/ci.yml:258 keeps a dead CPython 3.10 exclusion. ci/tools/tests/test_cuda_python_setup.py:51 relies on pytest's temp directory having no git repository above it, because setup.py resolves the SCM root one level up. The pixi full-test job never runs the SCM bootstrap, so a cu12 dispatch builds bindings versioned 12.9.1.devN; tests still pass because Core's cu12 pixi environment uses published bindings, but build-smoke and full-test disagree on versioning.

Things I checked that hold up: the planner's release-tag routing and both tag flows, Windows path handling in the Python build-environment writer, latest and selector-file protection on release docs, identical landing templates for both roots, the three SYNCED conftest blocks and the four _build_shared.py copies, pytest 9.1's built-in subtests covering the dropped pytest-subtests dependency, and the format-patch/am transfer recipe, which I ran in a scratch repo.

Documentation and agent-instruction gaps

  • No cuda_bindings_12/AGENTS.md until 3178. The root rule tells agents to follow each package's generation instructions, but the CUDA 12 root's model differs: header parsing with pyclibrary at build time, .pyx.in and .pxd.in templates, no synced generator. In the 3174 to 3176 state an agent fixing a 12 bug has no package guidance. Pulling that file forward into 3174 would close the gap.
  • scm-bootstrap-release has no lifecycle. cuda_bindings_12/pyproject.toml:166 introduces it and ci/tools/bindings_scm_version.py:88 consumes it, but nothing says when to bump it. If an emergency v12.9.11 ships from 12.9.x, main keeps producing 12.9.11a0.dev1 previews that sort below the published release, and v12.9.11 can never be tagged on main. The root AGENTS.md version-lifecycle section does not mention the field. Matt's suggestion to push a retained v12.9.11a0 on main's history is permitted by that policy and would delete the helper, the field and four workflow steps; it deserves an explicit decision in 3174 or 3175.
  • Minor-bump procedure for the hard-coded selectors. ci/README.md:314 says to update the --match pattern when a root moves to a new toolkit minor, but not that every build then yields a bogus 0.1.devN version until a v13.5.0a0-style tag exists, since the bootstrap only covers the maintenance root. 3178 adds a "Changing Supported Release Lines" section; the group that lands first does not have it.
  • Stale maintenance release notes. cuda_bindings_12/docs/source/release/12.X.Y-notes.rst:1 is a 2025 template. The 12.9.11 notes need the Python 3.10 drop, 3.14 and 3.15 support, the wheel-payload change above, and the new driver baseline introduced at cuda_bindings_12/docs/source/install.md:9, which is a content change the 3176 summary does not call out.
  • Validation claims need scope. "Full CI passes" without saying the reuse path was never hit, and "328 CI-tool tests pass" without saying test_list_pixi_workspaces requires a .git directory, both lead a reader to over-trust the evidence.

rwgk and others added 18 commits October 26, 2026 17:21
Signed-off-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
Signed-off-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
Relocate the established hardware, driver, visibility and timing accommodations from the source-integration layer. Preserve their contents exactly; the FieldId correction moves with the test maintenance.

Signed-off-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
Signed-off-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
Signed-off-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
Signed-off-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
Signed-off-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
Signed-off-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
Signed-off-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
Signed-off-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
Signed-off-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
Signed-off-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
Signed-off-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
Signed-off-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
Signed-off-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
(cherry picked from commit 5f93b1a)
Signed-off-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
Signed-off-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
Signed-off-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
@rwgk
rwgk force-pushed the rwgk/cb12-on-main-05-ci-tests-docs branch from 45ffc98 to feb25b1 Compare October 27, 2026 00:23

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to GitHub limitations.

⚠️ Outside diff range comments (1)

Minor · 🗄️ Data Integrity & Integration · ci-pixi-source-test.yml:242-248

.github/workflows/ci-pixi-source-test.yml:242-248
🗄️ Data Integrity & Integration | 🟡 Minor | ⚡ Quick win

important: Build CUDA 12 documentation from the selected checkout.

The CUDA 12 docs environment installs published cuda-bindings = "12.9.*", while Sphinx autodoc imports cuda.bindings from that environment. Source or API changes in the selected checkout can therefore remain untested. Use the same path dependency as CUDA 13.

- cuda-bindings = "12.9.*"
+ cuda-bindings = { path = "." }

Review info
Run configuration
Commits

Reviewing files that changed from the base of the PR and between 45ffc98 and feb25b1.

Files ignored due to path filters: 1
  • cuda_bindings_13/pixi.lock is excluded by !**/*.lock
Files selected for processing: 7
  • .github/workflows/ci-nightly.yml
  • .github/workflows/ci-pixi-source-test.yml
  • CONTRIBUTING.md
  • ci/README.md
  • cuda_bindings_12/docs/source/release/12.X.Y-notes.rst
  • cuda_bindings_12/tests/test_cuda.py
  • cuda_bindings_13/tests/test_cuda.py
Files skipped from review as they are similar to previous changes: 1
  • ci/README.md

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 11 remain after this review.

@rwgk

rwgk commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor Author

codex GPT-6.1-Sol ultra on behalf of Ralf:

Following up on the Fable 5.1 review: thanks for the thorough review. The actionable fixes are now in their owning layers, and the descriptions distinguish the review boundaries, compatibility changes, and evidence.

Packaging and support policy (#3174). Keeping the smaller wheel payload is intentional. The source comment and upcoming maintenance notes now say that wheels omit implementation sources/templates while retaining Cython declarations/includes; sdists retain the build inputs. An audit of the actual Linux cp312 candidate wheel found all 38 historical declaration/include filenames, with all 34 package-owned dependency edges resolving. A separate sdist-only build retained all 76 tracked Cython/template inputs and the packaging helpers. This was an archive audit, not another native build. The notes also record Python >=3.11. Python 3.14/3.15 support was already released, so it is not presented as new. The CUDA 12 driver guidance is a documentation correction, not a new runtime gate. Updated notes.

Hosted coverage. Full CI passed at the refreshed #3176 head feb25b1b3d3c1744b680bf54ff10986ed4e59953: run 40426114619. The completed previous failure logs exposed a process-global coredump setting race in free-threaded tests. We carried Mike's #3226 serialization fix and marked the corresponding CUDA 12 global test; helper-storage tests remain parallel. We have removed the stale run references from the validation sections.

Hosted artifact-reuse rehearsals. Two temporary draft PRs use the production resolver unchanged against successful baseline run 40426114619, at feb25b1b3d3c1744b680bf54ff10986ed4e59953. Both actual workplans exactly match the expected selective plans; neither fell back to a full rebuild.

  • CUDA 12 source-change case, #3237: run 40456909677 rebuilt CUDA 12 bindings/metapackage wheels while reusing Core, Pathfinder, and CUDA 13 wheels. All 28 build jobs passed, including bindings and Core Cython test-extension builds on Linux x86-64, Linux ARM64, and Windows. A re-uploaded Linux CPython 3.12 Core wheel is byte-identical to its baseline wheel. All 20 selected GPU tests (13 Linux and 7 Windows) and the final aggregate passed on the first attempt. The workflow is fully green; no fixes or retries were needed.
  • Linux test-workflow case, #3238: run 40460578171 skipped every package-wheel build, downloaded and re-uploaded the baseline packages, and successfully built all selected Cython test extensions and Core test binaries. All 45 expected reused package artifacts exist in the new run. All 42 selected GPU tests (24 x86-64 plus 18 ARM64 Linux rows, covering both CUDA majors) and the final aggregate passed on the first attempt. The seven planned Windows, sdist, and Core API skips were accepted.

Both complete hosted workflows passed on their first attempts, including documentation/link checks and their final aggregates. No fixes or retries were needed. These probes cover the two scenarios above; no separate Core-only or metapackage-only hosted probe is claimed.

Full hosted CI also passed for the complete stack at the unchanged #3182 head 775392e1a6b372b8d046a95466513bd114b05f6a: run 40456627846, attempt 2. The only retry was a self-hosted H100 runner disconnect during driver setup, before package tests began. Its replacement passed, together with the final gate; exact-head auxiliary checks are also green or intentionally skipped. This tests the cumulative stack, rather than hosted release publication or backport automation.

Historical docs and nightly rollout (#3176). The CI guide now gives an actual release.yml --ref 13.4.x dry-run recipe for v13.4.4; both the release workflow and its reusable docs workflow come from that historical branch. New main tooling still requires the registry recorded in the tagged source. Nightly explicitly diagnoses an incompatible artifact-source registry. Its instructions require the first successful, complete new-layout main run, and describe supplying a successful compatible run ID while that baseline is unavailable. The override selects the artifacts and source together; it does not reinterpret old artifacts. Rollout guidance, historical rerun recipe.

Instructions and counterpart checks (#3174). CUDA 12 now gets its standalone AGENTS guide at the first layer, and both bindings READMEs use --group test. The descriptions name the counterpart files inspected. One clarification: CUDA 12 does contain sealed cybind-generated inputs as well as build-time header parsing/template expansion; the guide distinguishes those from handwritten helpers and forbids manual edits to sealed output. CUDA 12 guide.

SCM decision. Retain the temporary local-only bridge through the first integration group, then make the explicit retained-milestone decision on validated main history and remove the bridge in a follow-up. Before the group lands, main still has broad tag selectors and the old CUDA 12 prerelease parser; introducing the shared milestone there would be premature. The lifecycle instructions now reserve preview targets across all release branches, explain what to do if an unreachable branch release consumes a target, and require matching milestones when moving minor selectors. No retained tag was created or pushed. Lifecycle policy.

Remaining small findings. Removed the obsolete Core-floor trigger, narrowed the large-file exceptions, and deleted the dead Python 3.10/Windows ARM64 branch. Pixi GPU full-test now prepares maintenance SCM before source builds and explicitly installs the required dependency, matching CPU build-smoke. The sdist fallback test is unchanged: the installed SCM backend does not search arbitrary ancestor repositories, and that proposed contamination did not reproduce. That test still checks real PKG-INFO fallback without a pretend version. The descriptions also state that the CI-tool suite runs in a full Git checkout, including Git-based Pixi workspace discovery; it is not an archive-only suite.

#3174-#3176 remain one integration group with separate review scopes. The later release, policy, and optional runtime layers remain separate. Hosted release publication/backport execution and instrumented proof of the optional coredump ownership fix are not claimed by these checks.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CI/CD CI/CD infrastructure cuda.bindings Everything related to the cuda.bindings module cuda.core Everything related to the cuda.core module cuda.pathfinder Everything related to the cuda.pathfinder module enhancement Any code-related improvements

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants