Skip to content

Broadcast Tangents like their primal - #724

Open
devmotion wants to merge 4 commits into
mainfrom
dmw/tangent-broadcastable
Open

devmotion wants to merge 4 commits into
mainfrom
dmw/tangent-broadcastable

Conversation

@devmotion

@devmotion devmotion commented Oct 29, 2026 •

Copy link
Copy Markdown
Member

Tangent defines no Broadcast.broadcastable, so broadcasting over one falls back to collect, which iterates over the backing and produces a Vector. That is wrong in different ways depending on the primal:

julia> Tangent{Any}(1.0, [2.0]) .* 2          # tuple: element types are lost
2-element Vector{Any}:
 2.0
  [4.0]

julia> Tangent{Foo}(; y=2.0, x=1.0) .* 2      # struct: broadcasts over the fields, in keyword order
2-element Vector{Float64}:
 4.0
 2.0

julia> Tangent{Dict}(Dict(:a => 1.0)) .* 2     # Dict: broadcasts over the `Pair`s
ERROR: MethodError: no method matching *(::Pair{Symbol, Float64}, ::Int64)

This PR makes a Tangent broadcast like its primal:

Base.Broadcast.broadcastable(tangent::Tangent{<:Any,<:Tuple}) = backing(tangent)
Base.Broadcast.broadcastable(tangent::Tangent) = Ref(tangent)
  • Tuples are containers in broadcasting ((1, 2) .+ 1 == (2, 3)), so the tangent of a tuple broadcasts like its backing tuple, as indexed_iterate and map already treat it. Tangent{Any}(1.0, [2.0]) .* 2 now returns (2.0, [4.0]), which is inferrable.
  • Other primals, e.g. structs, are scalars in broadcasting, so their tangents are too, just like AbstractZero. The NamedTuple (or Dict) backing only stores the fields; it is not a collection that the primal broadcasts over. Tangent{Foo}(; y=2.0, x=1.0) .* 2 now returns Tangent{Foo}(; x=2.0, y=4.0), and Tangent{Dict}(Dict(:a => 1.0)) .* 2 returns Tangent{Dict}(Dict(:a => 2.0)).

Neither choice works for both. Forwarding all tangents to their backing would make struct tangents throw (broadcasting over dictionaries and NamedTuples is reserved), and treating tuple tangents as scalars would stop them from lining up with the elements of the tuple.

Why this matters for AD

The pullbacks of ChainRules' broadcast rules broadcast the output cotangent together with the per-element pullbacks, so the cotangent must broadcast with the same shape as the output. With ChainRules + Zygote:

Broadcasting over a Tangent therefore no longer returns a Vector: tuple tangents give a Tuple, all others behave as scalars.

🤖 Generated with Claude Code

@codecov

codecov Bot commented Oct 29, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.65%. Comparing base (4e9c8aa) to head (80dc992).

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #724      +/-   ##
==========================================
+ Coverage   93.54%   93.65%   +0.11%     
==========================================
  Files          15       15              
  Lines        1022     1024       +2     
==========================================
+ Hits          956      959       +3     
+ Misses         66       65       -1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@devmotion devmotion changed the title Broadcast Tangent of tuples like their backing tuple Broadcast Tangents like their primal Oct 29, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
devmotion and others added 3 commits October 30, 2026 01:12
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant