Repository navigation
fix: Allow NaN in bounded float columns when allow_nan is set - #423
Merged
Oliver Borchert (borchero) merged 1 commit intoNov 11, 2026
Merged
Conversation
Bound comparisons reject NaN, so allow_nan was ignored once min or max was set. Accept NaN in those rules and leave the decision to allow_nan.
Aarab Malik (aarab-malik)
requested review from
Andreas Albert (AndreasAlbertQC),
Oliver Borchert (borchero) and
Daniel Elsner (delsner)
as code owners
November 11, 2026 11:42
Contributor
There was a problem hiding this comment.
🟢 Approval recommended
There are no active review issues for the change allowing NaN in bounded float columns when explicitly enabled.
No open issues 🎉
What changed in this PR
Fixes #416 by allowing NaN values in bounded float columns when allow_nan=True, keeping validation and checked sampling consistent.
Changes:
- Makes all four float bound rules accept NaN when explicitly allowed.
- Adds parameterized regression tests for bounds with NaN allowed and disallowed.
- Covers NaN preservation in checked sampling and validation of nested float lists.
| File | Description |
|---|---|
tests/column_types/test_float.py |
Adds regression coverage for bounded NaN validation, sampling, and lists. |
dataframely/columns/float.py |
Allows NaN through bound rules when allow_nan=True. |
Review effort: Balanced
Oliver Borchert (borchero)
approved these changes
Nov 11, 2026
Oliver Borchert (borchero)
left a comment
Member
There was a problem hiding this comment.
Nice, thank you!
Oliver Borchert (borchero)
enabled auto-merge (squash)
November 11, 2026 15:37
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #423 +/- ##
=========================================
Coverage 100.00% 100.00%
=========================================
Files 46 46
Lines 2584 2587 +3
=========================================
+ Hits 2584 2587 +3 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
Float(allow_nan=True)still rejects NaN once a bound is set. Polars ranks NaN above every number, soNaN <= maxis false and themax/max_exclusiverules fail even though NaN is allowed.NaN >= minhappens to pass today, but that depends on the same ordering, sominandmin_exclusivehave the same bug waiting.allow_nanshould be the only switch for NaN.I also see it in sampling.
test_sample_unchecked_nandraws NaNs forFloat(min=0, max=10, allow_nan=True), butSchema.sample()filters those rows back out through themaxrule, so the checked sampler returns no NaN at all.Closes #416.
Changes
_BaseFloat.validation_rules, whenallow_nanis set, I ORexpr.is_nan()into each bound rule (min,min_exclusive,max,max_exclusive). Finite values outside the bound are still rejected. NaN is still rejected whenallow_nanis false, by thenanrule.allow_nan=Trueandallow_nan=False.allow_nanis set, and a list of such floats accepts an inner NaN. Unchecked sampling was already consistent and still is.