Repository navigation
BUG: Clamp the quaternion scalar part before acos() in conversions - #80
Closed
imikejackson wants to merge 1 commit into
Closed
imikejackson wants to merge 1 commit into
imikejackson wants to merge 1 commit into
Conversation
* Quaternion::toAxisAngle(), toRodrigues() and toHomochoric() passed the scalar part straight to acos(). Float rounding can leave a unit quaternion with |w| one ULP above 1.0 (for example the misorientation a^-1 * (a * b) for a small rotation b), and acos() then returns NaN, so the angle, Rodrigues vector or homochoric vector was NaN. * The scalar part is now clamped to [-1, 1] first. Values inside that range are unchanged, so results for other input do not change. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
imikejackson
force-pushed
the
fix/orientation-acos-clamp
branch
from
November 13, 2026 02:47
95cbdcc to
b257d8f
Compare
This was referenced Nov 13, 2026
Collaborator
Author
|
Closed in favor of #86, which merges this branch (with its history) together with the other open EbsdLib fixes for the next release, so simplnx can build against one EbsdLib branch. The branch is kept. 🤖 Generated with Claude Code |
59 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Quaternion::toAxisAngle(),toRodrigues()andtoHomochoric()passed the scalar partwstraight toacos(). Float rounding can leave a unit quaternion with|w|one ULP above 1.0.acos()then returns NaN, and the angle, the Rodrigues vector or the homochoric vector is NaN.(-0.000147104263, 0.00024650991, 0.000268757343, 1.00000012)is the misorientationa^-1 * (a * b)of a small rotationb, computed in float.isValid()accepts it, but all three conversions returned NaN.wis now clamped to [-1, 1] beforeacos(). A value already inside that range does not change, so results for all other input stay the same.Other inverse-trigonometric calls I checked:
OrientationMatrix::toEuler()callsacos(om[8])only when|om[8]|is more than 1e-6 away from 1, so it is already safe.Quaternion::getMisorientationVector()already clamps.asin()calls exist in the orientation classes.Homochoric::toAxisAngle()has a related NaN: the constant term of its fit (tfit[0] = 1.0000000000018852) is above 1.0, so in double precision every rotation smaller than about 4e-6 rad gives a NaN angle. Commit e101edf on BUG: Correct ODF sampling and MDF normalization #61 (topic/synthetic_changes, also on ENH: Standard ODF section images and in-ball ODF sampling #60) already fixes it, so this PR does not changeHomochoric.hpp. The inputHomochoricDType(1.0e-7, 2.0e-7, 0.0)gives a NaN angle fromtoAxisAngle()and a NaN quaternion fromtoQuaternion()ondevelop, so it could be added as a regression test there.Core/OrientationTransformation.hpphas the same unclamped pattern inqu2ax,qu2ro,qu2hoandho2ax. I did not change it: it includesEbsdLib/Math/EbsdMatrixMath.h, which now exists only underMath/deprecated/, so no translation unit can compile it.Test Plan
EbsdLib::ebsdlib::OrientationTransformationTest::Quaternion_ScalarPartRoundingErrorfails ondevelop(NaN axis-angle, Rodrigues and homochoric results) and passes with this change.ctestsuite passes, 430/430.Related Issue
Related to BlueQuartzSoftware/simplnx#1875 (the unclamped
acosclass, itemComputeTwinBoundariesFilter#4). This PR fixes the same pattern in the EbsdLib quaternion conversions that simplnx filters call. Theacoscall in simplnxComputeTwinBoundaries.cppstill needs its own clamp.🤖 Generated with Claude Code