Repository navigation
Conversation
…hash_tdigest Signed-off-by: Christopher Krah <48338417+chrikrah@users.noreply.github.com>
chrikrah
requested review from
AlenkaF,
raulcd,
rok and
zanmato1984
as code owners
November 5, 2026 07:21
|
|
Collaborator
|
This is a new PR. You should be able to see checkboxes in the PR description template regarding AI-generated code https://arrow.apache.org/docs/dev/developers/overview.html#ai-generated-code. Why were they removed? |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Rationale for this change
tdigestandhash_tdigestaccept aqoutside [0, 1] and returnNaN, whilequantileraisesArrowInvalidon the same value.TDigestOptionsandQuantileOptionsdocument the range as a requirement ("All values must be in [0, 1]",python/pyarrow/_compute.pyx;api_aggregate.hsays the same).q=NaNalso gets past the check inquantile, and it aborts all three on a Debug build. Closes #55372.What changes are included in this PR?
CheckQuantileOptions's range loop moves toValidateQuantileLevelsinaggregate_internal.h, negated so thatNaNfails it.TDigestInitandGroupedTDigestImpl::Initnow call it, with the messagequantilealready uses.Are these changes tested?
Yes.
TestTDigestKernel.InvalidQuantileLevel,TestQuantileKernel.InvalidQuantileLevelandGroupBy.TDigestInvalidQuantileLevelare new, each withNaNand empty input.test_quantileandtest_tdigestgain cases.Are there any user-facing changes?
Yes. A
qoutside [0, 1],NaNincluded, now raisesArrowInvalidfromtdigestandhash_tdigest. It used to returnNaN, ornullon empty or all-null input.quantilenow raises onq=NaNtoo. On the 26.0.0 wheel it returnedNaN, or withinterpolation='lower'or'nearest'a real-looking1.0on[1.0, 2.0, 3.0].The check runs at init here and at exec in
quantile, so a bad option fails before any data is read. Tell me if you'd rather have it at exec in both.