[Andy-Jost]

Summary

cuda.core had host callbacks for graphs but no way to enqueue one on a plain stream, so mixed host and device ordering needed cuda.bindings. This PR adds cuda.core.host_launch(stream, fn, *, user_data=None), the stream-side counterpart of GraphBuilder.callback. It enqueues a host function on a stream in stream order through cuLaunchHostFunc: a Python callable, which runs with the GIL, or a ctypes function pointer with the CUhostFn prototype, which cuda.core passes to the driver as is, so a pointer to compiled code runs without the GIL (a ctypes callback that wraps a Python function still acquires it through ctypes). The fn and user_data forms and their validation are the ones the graph callback methods already use, and a GraphBuilder argument forwards to its callback, so the same call works under stream capture.

cuda.core owns the callable and a copied user_data until the function has run. The trampoline hands the launch record to the existing deferred-cleanup queue after the user function returns, so the owners are released on the main thread, never on the driver's callback thread, where releasing a Python object would take the GIL and a finalizer could call CUDA. A plain Stream under active capture raises RuntimeError and points to GraphBuilder.callback, because the record's owners could not be attached to the captured node and the record must not be reused by graph replays. A Python exception inside the callback is reported through sys.unraisablehook and the stream continues, as on the graph path. The docstring states the driver's rules: the function must not call CUDA, and a thread that holds the GIL while blocked in a CUDA wait on the stream deadlocks with a Python callback.

Changes

• cuda_core/cuda/core/_launcher.pyx: host_launch, exported from cuda.core.
• cuda_core/cuda/core/_host_callback.pyx, .pxd: the shared callback helper, moved from cuda/core/graph/ unchanged; a core module must not import the graph package at module level. The three graph cimports follow.
• cuda_core/cuda/core/_cpp/rt/host_launch.cpp: the launch record and trampoline behind one entry point, launch_host_function, declared in api.hpp, _rt.pxd, and _rt.pyx; cuLaunchHostFunc joins the driver table in driver_api.hpp.
• cuda_core/docs/source/api.rst: host_launch next to launch; _cpp/rt/DESIGN.md and GRAPH_ATTACHMENTS.md name the new file and the queue's second producer.
• Release-note entry for 1.3.0.
• cuda_core/examples/host_launch.py: a host function ordered between two kernel launches, linked from docs/source/examples.rst.
• Tests in cuda_core/tests/test_host_launch.py: ordering between prior and later stream work, release of the callable after it ran, copied bytes user_data, pass-through of an int user_data, the TypeError and ValueError cases, forwarding of a GraphBuilder with two graph launches, rejection of a capturing stream without invalidating the capture, unraisable reporting with the stream still usable, two streams keeping their own data, rejection of None, and the deprecated stream-protocol path.

Related Work

Closes #2058. The host copy function requested in #3256 builds on this.

🤖 Generated with Claude Code

[coderabbitai]

📝 Summary

Summary by CodeRabbit

• New Features
  • Added host_launch to enqueue Python callbacks or compatible function pointers in stream order, with optional user data.
  • Callbacks can also be recorded and replayed through graph builders.
  • Callback and user-data lifetimes are maintained until execution; callback errors are reported as unraisable Python exceptions.
  • Added an example demonstrating a host callback between stream-ordered kernel launches.
• Documentation
  • Added API reference and release-note coverage for host_launch.

The change adds host_launch for stream-ordered Python and ctypes callbacks. It adds a runtime launch path with callback and user-data lifetime handling, forwards GraphBuilder inputs to its callback API, and includes tests, an example, and API documentation.

Changes: Host launch API

Layer / File(s)	Summary
Runtime host-function launch cuda_core/cuda/core/_cpp/rt/api.hpp, cuda_core/cuda/core/_cpp/rt/driver_api.hpp, cuda_core/cuda/core/_cpp/rt/host_launch.cpp, cuda_core/cuda/core/_rt.pxd, cuda_core/cuda/core/_rt.pyx, cuda_core/cuda/core/_cpp/rt/DESIGN.md, cuda_core/cuda/core/_cpp/rt/GRAPH_ATTACHMENTS.md	The runtime adds cuLaunchHostFunc support and retains callback and user-data owners until invocation. The trampoline queues the record for cleanup after the callback returns.
Python API and graph integration cuda_core/cuda/core/_launcher.pyx, cuda_core/cuda/core/_launcher.pyi, cuda_core/cuda/core/_host_callback.pyi, cuda_core/cuda/core/graph/_graph_builder.pyx, cuda_core/cuda/core/graph/_graph_node.pyx, cuda_core/cuda/core/graph/_subclasses.pyx	host_launch accepts streams and forwards GraphBuilder inputs to GraphBuilder.callback. It resolves callback inputs and rejects active stream capture. Graph modules now import the callback resolver from cuda.core._host_callback.
Validation, example, and API documentation cuda_core/tests/test_host_launch.py, cuda_core/examples/host_launch.py, cuda_core/docs/source/api.rst, cuda_core/docs/source/examples.rst, cuda_core/docs/source/release/1.3.0-notes.rst	Tests cover ordering, callback lifetime, input validation, graph behavior, exceptions, and streams. The example demonstrates a callback between two kernels. The documentation lists and describes host_launch.

Assessment against linked issues

Objective	Addressed	Explanation
Add a top-level API for scheduling Python callables or C function pointers in stream order [#2058]	✅
Add a corresponding HostNode type in cuda.core.graph._subclasses [#2058]	❌	The change forwards GraphBuilder inputs to GraphBuilder.callback, but does not add a HostNode type.
Document callback threading restrictions, add an example, and add API reference entries [#2058]	✅

Possibly related issues

• cuda.core: a host copy function with the CUhostFn signature for host_launch #3256: The change provides the host_launch and user_data mechanism, but does not add the compiled host-copy function.
• cuda.core: release synchronous allocations in stream order through a host callback instead of blocking #2989: The change provides the host-callback primitive, but does not change resource deallocation behavior.

Suggested labels: enhancement

Suggested reviewers: mdboom, juenglin

Priority: Normal

Change: Feature

Merge Risk: Low · up to 2c23d

The API already prohibits CUDA calls in callbacks, but its warning should also mention that they can deadlock. The remaining risk is limited to misuse.

- [ ] Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

_{Comment @coderabbitai help to get the list of available commands.}

[github-actions]

Doc Preview CI
🚀 View preview at https://nvidia.github.io/cuda-python/pr-preview/pr-3258/
https://nvidia.github.io/cuda-python/pr-preview/pr-3258/cuda-core/
https://nvidia.github.io/cuda-python/pr-preview/pr-3258/cuda-bindings/
https://nvidia.github.io/cuda-python/pr-preview/pr-3258/cuda-pathfinder/
Preview will be ready when the GitHub Pages deployment is complete.

[coderabbitai]

Actionable comments posted: 1

---

Review info

Run configuration

• Source mode: Auto
• Repository: NVIDIA/cuda-python
• Branch: main
• File: .coderabbit.yaml
• Commit: 94625c1
• Layers: Repository YAML
• Review profile: CHILL
• Plan: Enterprise
• Run ID: 7a78139b-bfca-4094-acd9-991e49bd39f7

Commits

Reviewing files that changed from the base of the PR and between 426ec5a and 2c23d6f.

Files selected for processing: 8

• cuda_core/cuda/core/_cpp/rt/DESIGN.md
• cuda_core/cuda/core/_cpp/rt/GRAPH_ATTACHMENTS.md
• cuda_core/cuda/core/_cpp/rt/host_launch.cpp
• cuda_core/cuda/core/_launcher.pyi
• cuda_core/cuda/core/_launcher.pyx
• cuda_core/docs/source/examples.rst
• cuda_core/docs/source/release/1.3.0-notes.rst
• cuda_core/tests/test_host_launch.py

Files skipped from review as they are similar to previous changes: 3

• cuda_core/cuda/core/_cpp/rt/DESIGN.md
• cuda_core/cuda/core/_cpp/rt/GRAPH_ATTACHMENTS.md
• cuda_core/docs/source/release/1.3.0-notes.rst

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 10 remain after this review.

[coderabbitai]

Minor · Stability & Availability · Quick win

suggestion: State that CUDA calls from the callback can deadlock.

The driver does not guarantee an error for a CUDA call from a host callback. Such a call can also deadlock. State that consequence here and regenerate the matching stub documentation. (docs.nvidia.com)