Skip to content

BUG: xpx.testing: fix assertions in release builds - #1069

Open
steppi wants to merge 4 commits into
data-apis:mainfrom
steppi:assert
Open

steppi wants to merge 4 commits into
data-apis:mainfrom
steppi:assert

Conversation

@steppi

@steppi steppi commented Oct 28, 2026 •

Copy link
Copy Markdown
Contributor

Reference issue

Closes #1068

What does this implement/fix?

This replaces the use of assert statements in array_api_extra.testing with a helper that branches on the condition and directly raises an AssertionError if it holds. This is done so that these checks will still work even if the CPython interpreter is run with -O, -OO or with the environment variable PYTHONOPTIMIZE set to a value greater than or equal to 1. asserts are stripped out when the interpreter is run in such a way.

I found that NumPy also takes care to avoid direct uses of assert in numpy.testing and have adapted the helper from https://numpy.org/doc/stable/reference/generated/numpy.testing.assert_.html, but had to change it a little to work with array_api_extra's more strict linting and type checking.

Additional information

I had to flip the order of some casts and checks at one point because the linter complained. This seems to have been a pre-existing issue that was grandfathered in prior to the strict checks being set up. There are assertions a value is not None where the value couldn't possibly be None if the casts are correct.

I wonder if a CI job should be added that runs the Python interpreter with -O. This will trigger a warning from pytest that currently gets turned into an error. SciPy currently filters this warning in its pytest.ini (https://github.com/scipy/scipy/blob/f46f55fc2ba8f81239494b40f2f3a3d844476650/pytest.ini#L25).

AI Generation Disclosure

I used GPT 6.1 Sol in a chat to help debug the mysterious failure in scipy/scipy#26512 and to review my work. The LLM identified that the problem was due to PYTHONOPTIMIZE being set in the failing CI job. It was my idea to replace assert statements with a helper function and I found the prior art in numpy.testing.assert_ while investigating why things from numpy.testing don't cause the failures we saw on scipy/scipy#26512. The LLM was the one that suggested the need to flip the order of some assertions and casts, so that transposition can be considered AI generated 😎

@lucascolley lucascolley left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Happy with the cast stuff

I wonder if a CI job should be added that runs the Python interpreter with -O. This will trigger a warning from pytest that currently gets turned into an error. SciPy currently filters this warning in its pytest.ini (scipy/scipy@f46f55f/pytest.ini#L25).

happy to add one, our CI is pretty fast

@lucascolley lucascolley added bug Something isn't working xpx.testing labels Oct 28, 2026
@lucascolley lucascolley added this to the 0.12.0 milestone Oct 28, 2026
@lucascolley lucascolley changed the title BUG: Use _assert helper instead of assert statements BUG: xpx.testing: fix assertions in release builds Oct 28, 2026
@lucascolley

Copy link
Copy Markdown
Member

happy to add one, our CI is pretty fast

(if you're wanting to do that here, or a follow-up is fine)

@steppi

steppi commented Oct 28, 2026

Copy link
Copy Markdown
Contributor Author

happy to add one, our CI is pretty fast

(if you're wanting to do that here, or a follow-up is fine)

Thanks. I'll do it here.

Comment thread .github/workflows/ci.yml
Comment on lines 73 to 80
environment:
- tests-py312
- tests-py314
- tests-numpy1
- tests-nogil
- tests-backends
- tests-backends-py312
- tests-run-deps

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it would be better to avoid adding a whole new job and instead add to the matrix here. Try adding a new Pixi env with activation.env.PYTHONOPTIMIZE = 1 and listing it here

@lucascolley

Copy link
Copy Markdown
Member

thanks, as a nit I would rename the specific 'asserts-removed' to the more general 'python-optimized'. Otherwise LGTM

@steppi

steppi commented Oct 29, 2026

Copy link
Copy Markdown
Contributor Author

thanks, as a nit I would rename the specific 'asserts-removed' to the more general 'python-optimized'. Otherwise LGTM

Thanks! Done

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working xpx.testing

Projects

None yet

Development

Successfully merging this pull request may close these issues.

BUG: array_api_extra.testing relies on assert statements that get stripped out with PYTHONOPTIMIZE>=1

2 participants