Repository navigation
WIP, TST: yield array API infra from sklearn #1064
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
tylerjereddy
wants to merge
1
commit into
data-apis:main
Choose a base branch
from
tylerjereddy:treddy_sklearn_yield_utils
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+93
−0
Open
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -6,6 +6,7 @@ | |
| import enum | ||
| import functools | ||
| import inspect | ||
| import itertools | ||
| import math | ||
| import typing | ||
| import warnings | ||
|
|
@@ -35,6 +36,8 @@ | |
| "assert_less", | ||
| "lazy_xp_function", | ||
| "patch_lazy_xp_functions", | ||
| "yield_namespace_device_dtype_combinations", | ||
| "yield_namespaces", | ||
| ] | ||
|
|
||
|
|
||
|
|
@@ -1008,3 +1011,89 @@ def assert_close_nulp( | |
| actual_np = _as_numpy_array(actual, xp=xp) | ||
| desired_np = _as_numpy_array(desired, xp=xp) | ||
| np.testing.assert_array_almost_equal_nulp(actual_np, desired_np, nulp=nulp) | ||
|
|
||
|
|
||
| def yield_namespaces(include_numpy_namespaces: bool = True) -> Generator[str]: | ||
| """Yield supported namespace. | ||
|
|
||
| This is meant to be used for testing purposes only. | ||
|
|
||
| Parameters | ||
| ---------- | ||
| include_numpy_namespaces : bool, default=True | ||
| If True, also yield numpy namespaces. | ||
|
|
||
| Returns | ||
| ------- | ||
| str | ||
| The name of the Array API namespace. | ||
| """ | ||
| _NUMPY_NAMESPACE_NAMES = {"numpy", "sklearn.externals.array_api_compat.numpy"} | ||
| for array_namespace in [ | ||
| # The following is used to test the array_api_compat wrapper when | ||
| # array_api_dispatch is enabled: in particular, the arrays used in the | ||
| # tests are regular numpy arrays without any "device" attribute. | ||
| "numpy", | ||
| # Stricter NumPy-based Array API implementation. The | ||
| # array_api_strict.Array instances always have a dummy "device" attribute. | ||
| "array_api_strict", | ||
| "cupy", | ||
| "torch", | ||
| "dpnp", | ||
| ]: | ||
| if not include_numpy_namespaces and array_namespace in _NUMPY_NAMESPACE_NAMES: | ||
| continue | ||
| yield array_namespace | ||
|
|
||
|
|
||
| def yield_namespace_device_dtype_combinations( | ||
| include_numpy_namespaces: bool = True, | ||
| ) -> Generator[Any]: | ||
| """Yield supported namespace, device_name, dtype_name tuples for testing. | ||
|
|
||
| Use this to test that an estimator works with all combinations. | ||
| Pass the yielded values to `_array_api_for_tests` which returns (xp, device) | ||
|
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This is |
||
| for array allocation. | ||
|
|
||
| Parameters | ||
| ---------- | ||
| include_numpy_namespaces : bool, default=True | ||
| If True, also yield numpy namespaces. | ||
|
|
||
| Returns | ||
| ------- | ||
| array_namespace : str | ||
| The name of the Array API namespace. | ||
|
|
||
| device_name : str or None | ||
| The name of the device on which to allocate the arrays. Can be None to | ||
| indicate that the default value should be used. | ||
|
|
||
| dtype_name : str | ||
| The name of the data type to use for arrays. Can be None to indicate | ||
| that the default value should be used. | ||
| """ | ||
| for array_namespace in yield_namespaces( | ||
| include_numpy_namespaces=include_numpy_namespaces | ||
| ): | ||
| if array_namespace == "torch": | ||
| for device_name, dtype in itertools.product( | ||
| ("cpu", "cuda", "xpu"), ("float64", "float32") | ||
| ): | ||
| yield array_namespace, device_name, dtype | ||
| yield array_namespace, "mps", "float32" | ||
|
|
||
| elif array_namespace == "dpnp": # pragma: nocover | ||
| # XXX: add "accelerator" device type? | ||
| for device_name, dtype in itertools.product( | ||
| ("cpu", "gpu"), ("float64", "float32") | ||
| ): | ||
| yield array_namespace, device_name, dtype | ||
|
|
||
| elif array_namespace == "array_api_strict": | ||
| # Always yield strings for consistent parametrization; _array_api_for_tests | ||
| # creates Device objects when needed. | ||
| yield array_namespace, "CPU_DEVICE", "float64" | ||
| yield array_namespace, "device1", "float32" | ||
| else: | ||
| yield array_namespace, None, None | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
can we safely do this outside of
sklearn?