Repository navigation
BUG: Reject invalid DBSCAN Epsilon and match Minimum Points checks - #1958
Merged
imikejackson merged 2 commits intoNov 14, 2026
Merged
imikejackson merged 2 commits into
imikejackson merged 2 commits into
Conversation
joeykleingers
force-pushed
the
fix/dbscan-epsilon-minpoints-checks
branch
from
November 13, 2026 15:14
3eb8802 to
4299234
Compare
joeykleingers
approved these changes
Nov 13, 2026
joeykleingers
left a comment
Contributor
There was a problem hiding this comment.
Approved. The review findings are fixed in de04381: the Minimum Points check in the algorithm and its direct-call test are removed, and execute now rejects a DBSCAN search grid too large to index (-54091, -54092) and an unmasked non-finite coordinate (-54093). Tested in the in-core and out-of-core Release builds.
imikejackson
force-pushed
the
fix/dbscan-epsilon-minpoints-checks
branch
from
November 13, 2026 20:22
de04381 to
ce3de68
Compare
Contributor
Author
|
/azp run |
|
Azure Pipelines: Successfully started running 1 pipeline(s). |
* DBSCANFilter preflight compared Epsilon with `<= 0.0f`. That comparison is false for NaN, so a NaN Epsilon got to the grid sizing and its float-to-integer conversion. Preflight now rejects each Epsilon that is not finite or not greater than 0 (-7584). * The out-of-core path rejected only a negative Minimum Points value, and the in-core paths did not check it. DBSCAN now rejects a Minimum Points value of 0 or less before it selects the in-core or out-of-core path (-54063), so both paths agree with the filter. * Add tests for the preflight errors and for the Minimum Points error on each algorithm path. Closes BlueQuartzSoftware#1887 Signed-off-by: Michael Jackson <mike.jackson@bluequartz.net>
* Remove the Minimum Points check (-54063) from DBSCAN::operator() and its direct-call test. Preflight already rejects a Minimum Points value of 0 or less (-7585), and the knowledge-base rule "Run a filter, not its algorithm" (BlueQuartzSoftware#1959) keeps input checks in preflight only. -54063 is also a Compute K Means error code. * A very small Epsilon or a very large coordinate range made the grid sizing overflow: the cast of an axis cell count to usize was undefined, and the dense-grid cell count could wrap, so binning wrote past the end of the grid. The new DBSCANCommon.hpp computes the grid dimensions with checks before the grid is allocated, on the in-core path and both out-of-core paths: - -54091: one axis spans too many grid cells, or its extent padded by one cell leaves the float32 range. - -54092: the dense-grid cell count, or its byte count at one usize per cell, does not fit in usize. Every grid that passes keeps its old dimensions. * An unmasked NaN or infinite coordinate was skipped by the bounds scan and then converted to a grid index. Execute now stops at the first one with -54093, which names the array, tuple and axis. Masked tuples are not checked. * Remove UnitTest::LoadPlugins() from the preflight test. * Add tests through the filter on both algorithm paths, in 2D and 3D, and a hidden HDF5 out-of-core store test for the external path. * Update the filter documentation and the DBSCAN V&V report. Signed-off-by: Joey Kleingers <joey.kleingers@bluequartz.net>
imikejackson
force-pushed
the
fix/dbscan-epsilon-minpoints-checks
branch
from
November 14, 2026 18:07
ce3de68 to
56e3e5f
Compare
Contributor
Author
|
/azp run |
imikejackson
enabled auto-merge (squash)
November 14, 2026 18:07
|
Azure Pipelines: Successfully started running 1 pipeline(s). |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR fixes the parameter checks in DBSCAN, stops a very small Epsilon from overflowing the DBSCAN search grid, and rejects coordinates that are NaN or infinite.
DBSCANFilterpreflight compared Epsilon with<= 0.0f. That comparison is false for NaN, so a NaN Epsilon passed preflight and got to the grid sizing, where a float-to-integer conversion of a NaN is undefined behavior. Preflight now rejects each Epsilon that is not finite or not greater than 0 (-7584). An infinite Epsilon is also rejected, because it causes the same conversion problem.-54063, which is also a Compute K Means error code. That check is removed. Filter preflight stays the only Minimum Points check (-7585), as the knowledge-base rule "Run a filter, not its algorithm" (ENH: Check filter inputs once, in preflight #1959) requires.usizewas undefined, and the cell count of the dense grid could wrap, so the grid was too small and binning wrote past its end. Execute now computes the grid dimensions with checks in the newDBSCANCommon.hpp, before it allocates the grid, in the in-core path and in both out-of-core paths:-54091: one axis would span more grid cells than a grid index can hold, or its extent padded by one grid cell leaves the 32-bit floating-point range. The two cases have different messages: only the first one is fixed by a larger Epsilon. The external out-of-core path, which keeps a sparse grid, makes only this check.-54092: the cell count of the dense grid, or its byte count at oneusizeper cell, would be larger than memory can address.-54093, which names the array, the tuple and the axis. Masked tuples are not checked, so a mask can exclude tuples that have no valid position.Test Plan
SimplnxCore::DBSCAN: Reject invalid parameters in preflight: Epsilon 0, -1, NaN and infinity give-7584, and Minimum Points 0 and -1 give-7585.SimplnxCore::DBSCAN: Reject a search grid too large to index: through the filter, on both algorithm paths, in 2D and 3D. Epsilons1e-30and1e-36give-54091with "Increase Epsilon", and the largest 32-bit Epsilon gives-54091with the advice to rescale. One Epsilon whose cell count does not fit inusizeand one whose cell count fits but whose byte count does not both give-54092.SimplnxCore::DBSCAN: Reject a non-finite unmasked coordinate: through the filter, on both algorithm paths, in 2D and 3D, for NaN and infinity. One unmasked value and every value give-54093with the first bad tuple and its axis; a masked value is ignored and the run succeeds.SimplnxCore::DBSCAN: genuine HDF5 search grid and coordinate checks: the external out-of-core path on an HDF5-OOC store gives-54091and-54093.ctest -R DBSCAN, 19 of 19 passed.ctest -R DBSCAN, 27 of 27 passed. The fullooclabel suite (without benchmarks) passed 1050 of 1052, includingSimplnxCoreOocStoreContracts, which runs the hidden HDF5 test. The two failures are in code this PR does not change:ComputeArrayHistogramByFeature: HDF5 OOC scanline storage witness(the local SimplnxOoc checkout did not yet have today's SimplnxOoc change to that expected value, which matches the value the test got) and the DREAM3D-NXNXVtkLibImportedHistogramTests.Related Issue
Closes #1887
Related to #1908 (fixes item L86, the NaN Epsilon)