diff --git a/AGENTS.md b/AGENTS.md index d307ec739..011505c00 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -93,7 +93,7 @@ Every npm package except EQL lives under `languages/typescript/`: packages in `l - `packages/stack-auth`, `packages/stack-profile`, `packages/stack-kms`, `packages/stack-encrypt`, `packages/stack-encrypt-derive`, `packages/stack-guest-abi`: The Rust crates imported from `cipherstash/cipherstash-suite` with their history — `stack-auth` and `stack-profile` (published to crates.io, one version group), `stack-kms` (published to crates.io from 0.1.0, its own version group, re-exported by `stack-encrypt` as `stack_encrypt::kms`), `stack-encrypt` and `stack-encrypt-derive` (published to crates.io from 0.1.0, one version group; `eql-bindings`' `stack-encrypt` feature depends on them from the registry), and `stack-guest-abi` (`publish = false`). They are the members of the **root Cargo workspace**, with the three node binding crates below. See "Working on the Rust crates". - `languages/typescript/packages/auth`, `languages/typescript/packages/profile`, `languages/typescript/packages/stack-auth-wasm`: The node bindings of those crates. `@cipherstash/auth` (napi-rs v2) and its six `platforms/*` packages are published to npm from this repository by `release.yml` (`auth-artifacts`, `publish-auth`); a change to what it ships, the `stack-auth` crate included, needs an `@cipherstash/auth` changeset (`require-auth-npm-changeset.yml`). `@cipherstash/profile` and its platforms are private and never published; `@cipherstash/stack-auth-wasm` is private and builds the wasm that `@cipherstash/auth` ships. Their `build` and `test` scripts never invoke cargo; `build:native`, `build:debug` and `test:cargo` do. - `languages/golang`: The Go module — the SDK `encrypt` with its generated-code support `encrypt/gensupport` and the policy packages `encrypt/policy` and `encrypt/policy/protosource`; the credential package `auth`; the generator `stashgen` and its command `cmd/stashgen`; and `internal` (the shared guest plumbing and the `record` wire model). A wazero host with no cgo. Its two WASI guests (`encrypt/guest`, `auth/guest`) are detached Cargo workspaces built by `mise run wasm:guest:build` and `mise run wasm:auth-guest:build`; `mise run wasm:guest:build:deterministic` builds the seeded TEST build the hermetic Go tests use; the `.wasm` files they embed are gitignored. The stack-encrypt guest has a second build with the EQL types (`mise run wasm:guest:build:eql`, cargo feature `eql`, linking `eql-bindings` by path; `wasm:guest:build:eql:deterministic` is its test build), embedded by `encrypt/eql` and registered on import, so a program whose generated code names an EQL type runs it (ADR-0007, amended). `encrypt/eql`'s types and `Types` table are generated from the EQL catalog by `eql-codegen` (`eql_gen.go`, written by `mise run types:generate` in `packages/eql` and drift-gated by `types:check` and `cargo test -p eql-codegen`). Generated `*_stash.go` files are committed and CI fails when `go generate ./...` changes one. Releases are cut by hand with `.github/workflows/release-golang.yml` (`gh workflow run release-golang.yml -f version=X.Y.Z[-pre] --ref `): because the guests are gitignored on main and Go fetches a version as the files committed at its tag, the workflow builds them, commits them on a release-only commit no branch points at, tags it `languages/golang/vX.Y.Z` and pushes the tag alone, then attaches the EQL v4 SQL bundle to the GitHub release. The ref must carry the EQL v4 bundle, or the build stops at "Build the EQL SQL bundles". `scripts/go-release-version.mjs` refuses, before anything is built, every version Go would refuse (non-canonical semver, `+build` metadata, a v2+ major on a module path with no `/vN` suffix, a version shaped like a pseudo-version), a final version dispatched from anywhere but `main`, and a final version while `languages/golang/go.mod` pins a `github.com/cipherstash/vitaminc/…` module at a pseudo-version (vitaminc's Go bindings have no tags, and a TEMPORARY pin on an unreleased commit must be gone before the module is tagged); a pre-release may come from a branch and may carry such a pin. The build job also refuses a version the Go proxy already lists in `@v/list`, which catches a tag deleted after the proxy fetched it; it never asks the proxy for the version itself before the tag exists, because the proxy would cache "not found" for up to 30 minutes and `verify` would fail after the push. The build checks the `Cargo.lock` of the EQL workspace and both guest workspaces with `--locked` first, since none of those builds passes it. It runs in a job with no write token and hands the guests to a `publish` job that holds `contents: write` and runs only git and gh. `publish` fetches the commit with plain unauthenticated git rather than `actions/checkout`, whose `token` input defaults to the job's writable token, and refuses a release commit whose tree is not the one the build tested. `publish` takes the paths it commits from the workflow's fixed `GUESTS` list, never from the artifact, and refuses a `guests.sha256` that is not exactly one SHA-256 line for each of those three guests: the build job runs third-party code after writing that manifest, so a path read from it could put any file on the tag (`scripts/__tests__/go-release-artifact.test.mjs`). A version is immutable once the Go proxy has fetched it, so a tag is never moved or deleted: if `publish` or `verify` fails after the tag is pushed, use "Re-run failed jobs" on the same run, which keeps a tag that run pushed and creates or completes the release; a new dispatch of the same version is refused. Never tag a main commit by hand: `go get` would fetch a module with no guests. -- `languages/python`: The Python SDK, `stackencrypt`, a pyo3 extension over `stack-encrypt`, built by maturin. **The fourth Cargo workspace**, excluded from the root one. It has two crates. `crates/core` has no pyo3: the engine-side logic, linked and tested without a Python interpreter. `crates/engine` is the `_engine` cdylib, `abi3-py39`, one wheel per platform for Python 3.9 to 3.14. The `extension-module` feature is set only by maturin in `pyproject.toml`, never in a crate manifest, so `cargo test` links libpython. Its tasks live in its own `mise.toml` and run from that folder. `mise run build` builds the wheel into `.venv`. `mise run test` runs pytest. `mise run test:cargo` runs plain `cargo test --locked` for both crates, with libpython on the loader path. It does not use nextest, and its CI job sets no `MISE_ENV`. The nextest rule under "Working on the Rust crates" is for the root workspace, whose `stack-kms` tests set environment variables. No test in this workspace does, and nextest comes from `mise.test.toml`, which this workspace's CI does not load. The wheel's version is `crates/engine/Cargo.toml`'s. `stackencrypt.__engine__` is the commit of this checkout, recorded by `crates/core/build.rs`. Its `Cargo.lock` is seeded from the root one, so every crate it shares with the root workspace resolves to the version the root tests. After a root dependency moves, re-seed it the same way: copy the root `Cargo.lock` over it and run `cargo metadata --offline` there. That command prunes the copy to what the workspace uses. Until then, `tests-python.yml` fails on `--locked`. Nothing is published. CI is `tests-python.yml`. It runs the Rust checks and pytest on Python 3.9 and 3.14, where `MISE_PYTHON_VERSION` moves a leg off the local version. It then builds one abi3 wheel per FFI platform, uploads them as artifacts, and writes each build time to the job summary. `scripts/__tests__/python-ci.test.mjs` compares its path filter with the workspace's `Cargo.lock` and its wheel matrix with the protect-ffi platform packages. There is no Python lockfile yet: the package has no runtime dependency, so only pytest could be locked. `python/stackencrypt/_eql.py`, the EQL type markers, is generated from the EQL catalog by `eql-codegen py-eql`. To regenerate it, run `mise run types:generate` in `packages/eql`. Never edit it by hand: `tests-python.yml` runs its parity tests and fails on a hand edit. Design: Python ADR-0001 under `languages/python/docs/adr/`, the spec in #1341. +- `languages/python`: The Python SDK, `stackencrypt`, a pyo3 extension over `stack-encrypt`, built by maturin. **The fourth Cargo workspace**, excluded from the root one. It has two crates. `crates/core` has no pyo3: the engine-side logic, linked and tested without a Python interpreter. `crates/engine` is the `_engine` cdylib, `abi3-py39`, one wheel per platform for Python 3.9 to 3.14. The `extension-module` feature is set only by maturin in `pyproject.toml`, never in a crate manifest, so `cargo test` links libpython. Its tasks live in its own `mise.toml` and run from that folder. `mise run build` builds the wheel into `.venv`. `mise run test` runs pytest. `mise run test:cargo` runs plain `cargo test --locked` for both crates, with libpython on the loader path. It does not use nextest, and its CI job sets no `MISE_ENV`. The nextest rule under "Working on the Rust crates" is for the root workspace, whose `stack-kms` tests set environment variables. No test in this workspace does, and nextest comes from `mise.test.toml`, which this workspace's CI does not load. The wheel's version is `crates/engine/Cargo.toml`'s. `stackencrypt.__engine__` is the commit of this checkout, recorded by `crates/core/build.rs`. Its `Cargo.lock` is seeded from the root one, so every crate it shares with the root workspace resolves to the version the root tests. After a root dependency moves, re-seed it the same way: copy the root `Cargo.lock` over it and run `cargo metadata --offline` there. That command prunes the copy to what the workspace uses. Until then, `tests-python.yml` fails on `--locked`. Nothing is published. CI is `tests-python.yml`. It runs the Rust checks and pytest on Python 3.9 and 3.14, where `MISE_PYTHON_VERSION` moves a leg off the local version. It then builds one abi3 wheel per FFI platform, uploads them as artifacts, and writes each build time to the job summary. `scripts/__tests__/python-ci.test.mjs` compares its path filter with the workspace's `Cargo.lock` and its wheel matrix with the protect-ffi platform packages. There is no Python lockfile yet. The package has no runtime dependency, so only the `test` extra could be locked: pytest, plus attrs, Pydantic and SQLAlchemy for the declaration tests. Those three have no upper bounds, so the tests meet each new release when users do. `python/stackencrypt/_eql.py`, the EQL type markers, is generated from the EQL catalog by `eql-codegen py-eql`. To regenerate it, run `mise run types:generate` in `packages/eql`. Never edit it by hand: `tests-python.yml` runs its parity tests and fails on a hand edit. Design: Python ADR-0001 under `languages/python/docs/adr/`, the spec in #1341. - `e2e/*`: Cross-package end-to-end tests (package managers, supply chain, Prisma example README) - `languages/typescript/examples/*`: Working apps (basic, prisma, supabase-worker) - `docs/plans/*`: Internal design plans. User-facing documentation lives at https://cipherstash.com/docs (not in this repo). diff --git a/languages/python/mise.toml b/languages/python/mise.toml index 6b94d4e57..8b2948f7b 100644 --- a/languages/python/mise.toml +++ b/languages/python/mise.toml @@ -25,10 +25,12 @@ set -euo pipefail # the Rust tests link. A `.venv` made for another interpreter version is # recreated, so a change to `[tools]` cannot leave the tests on the old one. # No lockfile is written: `uv pip install` resolves against the index each -# time. The package has no runtime dependency, and the only thing resolved -# here is the `test` extra (pytest) plus maturin as the build backend, so a -# lockfile would fix test tooling versions alone. One comes, with its Dependabot -# entry, when the package gains a runtime dependency. +# time. The package has no runtime dependency. The only things resolved here +# are maturin, as the build backend, and the `test` extra: pytest, plus attrs, +# Pydantic and SQLAlchemy for the declaration tests. So a lockfile would fix +# test tooling versions alone. One comes, with its Dependabot entry, when the +# package gains a runtime dependency. See pyproject.toml for why the extra has +# no upper bounds. want="$(python3 -c 'import sys; print("%d.%d" % sys.version_info[:2])')" have="$([ -x .venv/bin/python ] && .venv/bin/python -c 'import sys; print("%d.%d" % sys.version_info[:2])' || true)" if [ "$have" != "$want" ]; then diff --git a/languages/python/pyproject.toml b/languages/python/pyproject.toml index 86de5154d..948311cab 100644 --- a/languages/python/pyproject.toml +++ b/languages/python/pyproject.toml @@ -11,7 +11,12 @@ dynamic = ["version"] classifiers = ["Private :: Do Not Upload"] [project.optional-dependencies] -test = ["pytest>=8"] +# attrs, Pydantic and SQLAlchemy only for the tests that declare a model of +# each kind; the package itself imports none of them. They have lower bounds +# and no upper bounds on purpose. They are the libraries users build their +# classes with, so the tests meet each new release when users do. A new +# release that fails a test is a finding about users' classes. +test = ["pytest>=8", "attrs>=23", "pydantic>=2", "sqlalchemy>=2"] [tool.maturin] # The pure-Python package is the distribution; the Rust extension is its diff --git a/languages/python/python/stackencrypt/__init__.py b/languages/python/python/stackencrypt/__init__.py index aa7d47136..1acd092e4 100644 --- a/languages/python/python/stackencrypt/__init__.py +++ b/languages/python/python/stackencrypt/__init__.py @@ -1,14 +1,16 @@ """Searchable encryption for Python, over the CipherStash Stack engine. Walking skeleton. The package loads its native extension, ``_engine``, exposes -two facts about the build, and exports the markers a model's fields are -declared with. Nothing encrypts yet. +two facts about the build, exports the markers a model's fields are declared +with, and reads a model's declaration with ``model``. Nothing encrypts yet. """ from stackencrypt import _engine, _eql +from stackencrypt._declaration import model # The markers for the EQL types the engine produces today. `_eql.__all__` # lists them; a marker the engine cannot produce yet stays in `_eql` only. from stackencrypt._eql import * # noqa: F403 +from stackencrypt._errors import DeclarationError, StashError from stackencrypt._markers import Ignore, Passthrough # The distribution's version, and the commit of the engine it was built from. @@ -16,5 +18,13 @@ __engine__: str __version__, __engine__ = _engine.build_info() -__all__ = ["Ignore", "Passthrough", "__engine__", "__version__"] +__all__ = [ + "DeclarationError", + "Ignore", + "Passthrough", + "StashError", + "__engine__", + "__version__", + "model", +] __all__ += _eql.__all__ diff --git a/languages/python/python/stackencrypt/_declaration.py b/languages/python/python/stackencrypt/_declaration.py new file mode 100644 index 000000000..c826dd71a --- /dev/null +++ b/languages/python/python/stackencrypt/_declaration.py @@ -0,0 +1,311 @@ +"""The ``model`` decorator: reads a class's fields and checks rule A. + +Rule A: every field has exactly one stash marker, in the ``Annotated`` +directly around its value type. A plain field has ``Passthrough``, and an +attribute that is not a column has ``Ignore``. The decorator raises +``DeclarationError`` at the class statement when a field breaks the rule. + +The fields are the library's own list where one exists (dataclasses, attrs, +Pydantic, ``NamedTuple``), and otherwise the class's type hints minus +``ClassVar`` and ``InitVar``, as on a SQLAlchemy declarative class. One generic +class around ``Annotated`` is read by structure, so ``Mapped[Annotated[str, +TextEq]]`` reads as ``TextEq``, with no SQLAlchemy import. A list, ``Final`` +and the other generics of the standard library are not read through. + +The checks of each field's meaning come later: the value type against the +marker, the constructor, and the engine's own check of the plan. +""" + +import dataclasses +import sys +from dataclasses import dataclass +from typing import ( + Annotated, + Any, + Callable, + ClassVar, + Dict, + List, + Optional, + Tuple, + Type, + TypeVar, + get_args, + get_origin, + get_type_hints, +) + +from stackencrypt import _eql +from stackencrypt._errors import DeclarationError +from stackencrypt._markers import Ignore, Marker, Passthrough + +T = TypeVar("T", bound=type) + +# The one attribute the decorator stores, in the class's own __dict__. +DECLARATION_ATTRIBUTE = "__stash_declaration__" + +# Every stash marker. A marker is found by identity, so a subclass of one, or +# another package's class of the same name, is not a marker. +_MARKERS = frozenset((Passthrough, Ignore) + _eql.TYPES) + +# The modules whose generics are value types, such as a list or a dict, or +# typing forms, such as Final. A one-argument generic class from any other +# module, such as SQLAlchemy's Mapped, is read as a wrapper. +_VALUE_TYPE_MODULES = frozenset( + {"builtins", "collections", "collections.abc", "types", "typing"} +) + + +@dataclass(frozen=True) +class DeclaredField: + """One field in the plan: its name, which is the column, and its marker.""" + + name: str + marker: Type[Marker] + + +@dataclass(frozen=True) +class Declaration: + """What the decorator read from a class: the table, and every field that + is a column, in field order. ``Ignore`` fields are not in it.""" + + table: str + fields: Tuple[DeclaredField, ...] + + +def model(table: str) -> Callable[[T], T]: + """Declare a class as a model stored in ``table``. + + Apply it outermost, above ``@dataclass`` or ``@attrs.define``:: + + @stackencrypt.model("users") + @dataclass + class User: + email: Annotated[str, TextEq] + age: Annotated[int, Passthrough] + """ + if not isinstance(table, str) or not table: + raise TypeError('model takes the table name: @stackencrypt.model("users")') + + def decorate(cls: T) -> T: + existing = declaration_of(cls) + if existing is not None: + raise DeclarationError( + "declaration_already_declared", + f"class {cls.__name__!r} is already declared, with table " + f"{existing.table!r}. Apply stackencrypt.model once.", + ) + declaration = Declaration(table=table, fields=tuple(_read_fields(cls))) + setattr(cls, DECLARATION_ATTRIBUTE, declaration) + return cls + + return decorate + + +def declaration_of(cls: type) -> Optional[Declaration]: + """The declaration ``model`` stored on ``cls`` itself, or ``None``. + + Read from the class's own ``__dict__``, so a subclass of a declared class + is not declared until it is decorated itself. + """ + found = cls.__dict__.get(DECLARATION_ATTRIBUTE) + return found if isinstance(found, Declaration) else None + + +def _read_fields(cls: type) -> List[DeclaredField]: + try: + hints = get_type_hints(cls, include_extras=True) + except (NameError, TypeError) as error: + raise _hints_unresolved(cls, error) from error + names = _library_field_names(cls) + from_hints = names is None + if names is None: + names = [name for name, hint in hints.items() if not _is_class_level(hint)] + declared = [] + for name in names: + marker = _marker_of(name, hints.get(name), from_hints=from_hints) + if marker is not Ignore: + declared.append(DeclaredField(name=name, marker=marker)) + return declared + + +def _library_field_names(cls: type) -> Optional[List[str]]: + """The field list of the class's own library, or ``None`` if it has none.""" + if dataclasses.is_dataclass(cls): + return [field.name for field in dataclasses.fields(cls)] + attrs_fields = getattr(cls, "__attrs_attrs__", None) + if attrs_fields is not None: + return [field.name for field in attrs_fields] + pydantic_fields = getattr(cls, "__pydantic_fields__", None) + if isinstance(pydantic_fields, dict): + return list(pydantic_fields) + named_tuple_fields = getattr(cls, "_fields", None) + if issubclass(cls, tuple) and isinstance(named_tuple_fields, tuple): + return list(named_tuple_fields) + return None + + +def _is_class_level(hint: Any) -> bool: + """A ``ClassVar`` or ``InitVar`` hint, which is not a field.""" + if isinstance(hint, dataclasses.InitVar) or hint is dataclasses.InitVar: + return True + if get_origin(hint) is Annotated: + hint = get_args(hint)[0] + return hint is ClassVar or get_origin(hint) is ClassVar + + +def _hints_unresolved(cls: type, error: Exception) -> DeclarationError: + field = _unresolved_field(cls) + where = f"field {field!r}" if field else f"a type hint of class {cls.__name__!r}" + return DeclarationError( + "declaration_hints_unresolved", + f"{where} cannot be resolved ({error}). Define or import every name in " + "it before the decorator runs, or declare the class after those names " + 'exist: stackencrypt.model("users")(User).', + field=field, + ) + + +def _unresolved_field(cls: type) -> Optional[str]: + """The first field whose hint does not resolve, or ``None``. + + The hints are resolved one at a time, in the order ``get_type_hints`` + resolves them, with the same namespaces. On 3.14, reading a class's + annotations can itself raise, and then no one field can be named. + """ + for base in reversed(cls.__mro__): + try: + own = _own_annotations(base) + except (NameError, TypeError): + return None + module = sys.modules.get(base.__module__) + global_names = dict(vars(module)) if module is not None else {} + for name, hint in own.items(): + + def probe() -> None: ... + + probe.__annotations__ = {name: hint} + try: + get_type_hints(probe, global_names, dict(vars(base)), include_extras=True) + except (NameError, TypeError): + return name + return None + + +def _own_annotations(base: type) -> Dict[str, Any]: + """The annotations written on ``base`` itself, not on its bases.""" + if sys.version_info < (3, 10): + # Before 3.10, base.__annotations__ can be a base class's. + found = vars(base).get("__annotations__") + else: + found = getattr(base, "__annotations__", None) + return found if isinstance(found, dict) else {} + + +def _is_marker(value: Any) -> bool: + return isinstance(value, type) and value in _MARKERS + + +def _markers_in(hint: Any) -> List[Type[Marker]]: + """The markers in the ``Annotated`` metadata of ``hint`` itself.""" + if get_origin(hint) is not Annotated: + return [] + return [value for value in hint.__metadata__ if _is_marker(value)] + + +def _nested_markers(hint: Any) -> bool: + """Whether any ``Annotated`` below ``hint``'s own level holds a marker.""" + inner = get_args(hint)[:1] if get_origin(hint) is Annotated else get_args(hint) + return any(_markers_in(arg) or _nested_markers(arg) for arg in inner) + + +def _is_wrapper(hint: Any) -> bool: + """A generic class with exactly one type argument, such as ``Mapped[str]``. + + A list, a tuple, ``Final`` and the other generics of the standard library + are not wrappers: ``List[str]`` is a value type, not a wrapper around one. + """ + origin = get_origin(hint) + return ( + isinstance(origin, type) + and origin.__module__ not in _VALUE_TYPE_MODULES + and len(get_args(hint)) == 1 + ) + + +def _marker_of(name: str, hint: Any, *, from_hints: bool) -> Type[Marker]: + """The one marker of a field, or a ``DeclarationError``. + + ``from_hints`` says the field list came from the type hints, as on a + SQLAlchemy class. Only there is a wrapper inside the marker's + ``Annotated`` refused as misplaced: ``Annotated[Mapped[str], TextEq]``. + Elsewhere a generic value type is a type mismatch, checked later. + """ + if hint is None: + raise _no_marker(name) + # One wrapper around Annotated is read by structure: Mapped[Annotated[...]]. + unwrapped = False + if get_origin(hint) is not Annotated and _is_wrapper(hint): + (inner,) = get_args(hint) + if get_origin(inner) is Annotated: + hint = inner + unwrapped = True + markers = _markers_in(hint) + value_type = get_args(hint)[0] if get_origin(hint) is Annotated else hint + # Annotated outermost, with the wrapper inside it, is the order reversed. + # Inside a wrapper already read, a generic value type is the value's own, + # as in a relationship: Mapped[Annotated[list[Post], Ignore]]. + if from_hints and not unwrapped and markers and _is_wrapper(value_type): + raise _marker_not_outermost(name, markers[0], value_type) + if _nested_markers(hint): + raise _marker_not_outermost(name, None, None) + if not markers: + raise _no_marker(name) + if len(markers) > 1: + names = ", ".join(marker.__name__ for marker in markers) + raise DeclarationError( + "declaration_two_markers", + f"field {name!r} has more than one stash marker ({names}). " + "Give each field exactly one.", + field=name, + reason="rule_a", + ) + return markers[0] + + +def _no_marker(name: str) -> DeclarationError: + return DeclarationError( + "declaration_no_marker", + f"field {name!r} has no stash marker. Every field needs one: an EQL " + "type such as TextEq to encrypt it, Passthrough to store it as it is, " + "or Ignore if it is not a column.", + field=name, + reason="rule_a", + ) + + +def _marker_not_outermost( + name: str, marker: Optional[Type[Marker]], value_type: Any +) -> DeclarationError: + if marker is not None: + wrapper = getattr(get_origin(value_type), "__name__", "the wrapper") + (inner,) = get_args(value_type) + fix = ( + f"Put the wrapper outside the Annotated: " + f"{wrapper}[Annotated[{_type_name(inner)}, {marker.__name__}]]." + ) + else: + fix = "Put the marker in the Annotated around the field's own type." + return DeclarationError( + "declaration_marker_not_outermost", + f"field {name!r} has its stash marker in the wrong place. {fix}", + field=name, + reason="rule_a", + ) + + +def _type_name(hint: Any) -> str: + """A type as the user writes it: ``str``, or ``Optional[str]``.""" + if isinstance(hint, type) and not get_args(hint): + return hint.__name__ + return repr(hint).replace("typing.", "") diff --git a/languages/python/python/stackencrypt/_errors.py b/languages/python/python/stackencrypt/_errors.py new file mode 100644 index 000000000..f308926da --- /dev/null +++ b/languages/python/python/stackencrypt/_errors.py @@ -0,0 +1,38 @@ +"""The exceptions the SDK raises. + +Every error carries the same four fields, whoever raised it: ``code`` (a fixed +lower_snake_case token), ``message`` (prose for a person), ``reason`` (a fixed +lower_snake_case token or ``None``) and ``field`` (the field name, or +``None``). The code's prefix tells where the error came from. An engine +error's code is the engine's diagnostic code with ``::`` replaced by ``_``. +""" + +from typing import Optional + + +class StashError(Exception): + """The base of every SDK error, and the one to catch for all of them.""" + + def __init__( + self, + code: str, + message: str, + *, + field: Optional[str] = None, + reason: Optional[str] = None, + ) -> None: + # The code and message are the exception's args, so that pickle, which + # rebuilds an exception as cls(*args), can rebuild this one. The other + # attributes come back from __dict__. + super().__init__(code, message) + self.code = code + self.message = message + self.field = field + self.reason = reason + + def __str__(self) -> str: + return f"{self.code}: {self.message}" + + +class DeclarationError(StashError): + """A model class that cannot be declared, raised at its class statement.""" diff --git a/languages/python/tests/_declaration_cases.py b/languages/python/tests/_declaration_cases.py new file mode 100644 index 000000000..ff17e5604 --- /dev/null +++ b/languages/python/tests/_declaration_cases.py @@ -0,0 +1,350 @@ +"""The declaration tests, compiled twice: as written, and with +``from __future__ import annotations``. See ``test_declaration.py`` and +``test_declaration_future_annotations.py``. + +The classes the tests declare refer only to module-level names, because +under the future import a hint is resolved from the module's globals. +""" + +import dataclasses +import pickle +from typing import Annotated, ClassVar, Final, List, NamedTuple, Optional + +import attrs +import pydantic +import pytest +from sqlalchemy.orm import DeclarativeBase, Mapped, mapped_column, relationship +from sqlalchemy import ForeignKey + +import stackencrypt +from stackencrypt import DeclarationError, Ignore, Passthrough, Text, TextEq +from stackencrypt._declaration import declaration_of + +EXPECTED_FIELDS = [("email", TextEq), ("age", Passthrough)] + + +def planned(cls): + declaration = declaration_of(cls) + assert declaration is not None, "the class was not declared" + return [(field.name, field.marker) for field in declaration.fields] + + +def refusal(build): + with pytest.raises(DeclarationError) as raised: + build() + return raised.value + + +# The five class kinds, each with one TextEq, one Passthrough and one Ignore. + + +def test_a_dataclass_declares(): + @stackencrypt.model("users") + @dataclasses.dataclass + class User: + email: Annotated[str, TextEq] + age: Annotated[int, Passthrough] + note: Annotated[str, Ignore] = "" + + assert planned(User) == EXPECTED_FIELDS + assert declaration_of(User).table == "users" + + +def test_an_attrs_class_declares(): + @stackencrypt.model("users") + @attrs.define + class User: + email: Annotated[str, TextEq] + age: Annotated[int, Passthrough] + note: Annotated[str, Ignore] = "" + + assert planned(User) == EXPECTED_FIELDS + + +def test_a_pydantic_model_declares(): + @stackencrypt.model("users") + class User(pydantic.BaseModel): + email: Annotated[str, TextEq] + age: Annotated[int, Passthrough] + note: Annotated[str, Ignore] = "" + + assert planned(User) == EXPECTED_FIELDS + + +class UserTuple(NamedTuple): + email: Annotated[str, TextEq] + age: Annotated[int, Passthrough] + note: Annotated[str, Ignore] = "" + + +def test_a_named_tuple_declares(): + declared = stackencrypt.model("users")(UserTuple) + + assert declared is UserTuple + assert planned(UserTuple) == EXPECTED_FIELDS + + +class Base(DeclarativeBase): + pass + + +class Post(Base): + __tablename__ = "posts" + id: Mapped[Annotated[int, Passthrough]] = mapped_column(primary_key=True) + user_id: Mapped[Annotated[int, Passthrough]] = mapped_column(ForeignKey("users.id")) + + +@stackencrypt.model("users") +class User(Base): + __tablename__ = "users" + id: Mapped[Annotated[int, Passthrough]] = mapped_column(primary_key=True) + email: Mapped[Annotated[str, TextEq]] + age: Mapped[Annotated[int, Passthrough]] + posts: Mapped[Annotated[List["Post"], Ignore]] = relationship() + + +def test_a_sqlalchemy_class_declares_through_mapped(): + assert planned(User) == [("id", Passthrough)] + EXPECTED_FIELDS + + +def test_a_relationship_marked_ignore_is_not_in_the_plan(): + assert "posts" not in [name for name, _ in planned(User)] + + +def test_sqlalchemy_still_maps_every_column_it_was_given(): + assert set(User.__table__.columns.keys()) == {"id", "email", "age"} + + +# Rule A and its codes. + + +def test_a_field_without_a_marker_is_refused(): + def build(): + @stackencrypt.model("users") + @dataclasses.dataclass + class User: + email: Annotated[str, TextEq] + age: int + + error = refusal(build) + assert error.code == "declaration_no_marker" + assert error.field == "age" + assert "declaration_no_marker" in str(error) + assert "'age'" in str(error) + + +def test_annotated_metadata_from_another_library_is_not_a_marker(): + def build(): + @stackencrypt.model("users") + @dataclasses.dataclass + class User: + email: Annotated[str, "not a marker"] + + assert refusal(build).code == "declaration_no_marker" + + +def test_a_field_with_two_markers_is_refused(): + def build(): + @stackencrypt.model("users") + @dataclasses.dataclass + class User: + email: Annotated[str, TextEq, Text] + + error = refusal(build) + assert error.code == "declaration_two_markers" + assert error.field == "email" + assert "declaration_two_markers" in str(error) + assert "'email'" in str(error) + + +def test_ignore_with_another_marker_is_two_markers(): + def build(): + @stackencrypt.model("users") + @dataclasses.dataclass + class User: + email: Annotated[str, TextEq, Ignore] + + assert refusal(build).code == "declaration_two_markers" + + +def test_a_wrapper_inside_the_marker_annotated_is_refused(): + # A plain class, whose fields come from its hints as a SQLAlchemy class's + # do. On a mapped class SQLAlchemy refuses this order itself, with + # MappedAnnotationError, before the decorator runs. + def build(): + @stackencrypt.model("users") + class User: + email: Annotated[Mapped[str], TextEq] + + error = refusal(build) + assert error.code == "declaration_marker_not_outermost" + assert error.field == "email" + assert error.reason == "rule_a" + assert "Mapped[Annotated[str, TextEq]]" in str(error) + + +def test_the_remedy_names_the_inner_type_as_written(): + def build(): + @stackencrypt.model("users") + class User: + email: Annotated[Mapped[Optional[str]], TextEq] + + message = str(refusal(build)) + assert "Union" not in message + assert ( + "Mapped[Annotated[Optional[str], TextEq]]" in message + or "Mapped[Annotated[str | None, TextEq]]" in message + ) + + +def test_a_collection_value_type_is_not_a_misplaced_wrapper(): + @stackencrypt.model("users") + class User: + tags: Annotated[List[str], Passthrough] + + assert planned(User) == [("tags", Passthrough)] + + +@pytest.mark.parametrize("kind", ["dataclass", "plain"]) +def test_a_marker_inside_a_list_or_final_is_refused(kind): + def build_list(): + class User: + tags: List[Annotated[str, TextEq]] + + if kind == "dataclass": + User = dataclasses.dataclass(User) + stackencrypt.model("users")(User) + + def build_final(): + class User: + email: Final[Annotated[str, TextEq]] + + if kind == "dataclass": + User = dataclasses.dataclass(User) + stackencrypt.model("users")(User) + + for build in (build_list, build_final): + error = refusal(build) + assert error.code == "declaration_marker_not_outermost" + assert error.reason == "rule_a" + + +def test_a_marker_inside_another_type_is_refused(): + def build(): + @stackencrypt.model("users") + @dataclasses.dataclass + class User: + email: Optional[Annotated[str, TextEq]] + + error = refusal(build) + assert error.code == "declaration_marker_not_outermost" + assert error.field == "email" + + +class MyText(TextEq): + """A user's subclass of a marker, which is not a marker.""" + + +def test_a_subclass_of_a_marker_is_not_a_marker(): + def build(): + @stackencrypt.model("users") + @dataclasses.dataclass + class User: + email: Annotated[str, MyText] + + assert refusal(build).code == "declaration_no_marker" + + +def test_an_unresolved_hint_is_refused_with_its_field(): + # "Undefined" is a forward reference that never resolves, as a + # relationship to a class written later does. + def build(): + @stackencrypt.model("users") + @dataclasses.dataclass + class User: + email: Annotated[str, TextEq] + posts: Annotated[List["Undefined"], Ignore] = dataclasses.field( # noqa: F821 + default_factory=list + ) + + error = refusal(build) + assert error.code == "declaration_hints_unresolved" + assert error.field == "posts" + assert "'posts'" in str(error) + assert isinstance(error.__cause__, NameError) + + +def test_a_class_is_declared_once(): + def build(): + @stackencrypt.model("users") + @stackencrypt.model("accounts") + @dataclasses.dataclass + class User: + email: Annotated[str, TextEq] + + error = refusal(build) + assert error.code == "declaration_already_declared" + assert "'accounts'" in str(error) + + +def test_a_declaration_error_survives_pickle(): + error = DeclarationError("declaration_no_marker", "no marker", field="age", reason="rule_a") + + copy = pickle.loads(pickle.dumps(error)) + + assert type(copy) is DeclarationError + assert (copy.code, copy.message, copy.field, copy.reason) == ( + "declaration_no_marker", + "no marker", + "age", + "rule_a", + ) + assert str(copy) == str(error) == "declaration_no_marker: no marker" + + +def test_class_level_names_are_not_fields(): + @stackencrypt.model("users") + @dataclasses.dataclass + class User: + email: Annotated[str, TextEq] + registry: ClassVar[dict] = {} + + assert planned(User) == [("email", TextEq)] + + +def test_a_subclass_is_not_declared_until_it_is_decorated(): + @stackencrypt.model("users") + @dataclasses.dataclass + class User: + email: Annotated[str, TextEq] + + @dataclasses.dataclass + class Admin(User): + level: Annotated[int, Passthrough] = 0 + + assert declaration_of(Admin) is None + + +def test_model_needs_the_table_name(): + with pytest.raises(TypeError, match="table name"): + + @stackencrypt.model + class User: + email: Annotated[str, TextEq] + + +def test_model_refuses_an_empty_table_name(): + with pytest.raises(TypeError, match="table name"): + stackencrypt.model("") + + +def test_declaring_a_class_makes_no_engine_call_and_needs_no_credentials(monkeypatch): + for name in ("CS_WORKSPACE_CRN", "CS_CLIENT_ID", "CS_CLIENT_KEY", "CS_CLIENT_ACCESS_KEY"): + monkeypatch.delenv(name, raising=False) + + @stackencrypt.model("users") + @dataclasses.dataclass + class User: + email: Annotated[str, TextEq] + + assert planned(User) == [("email", TextEq)] diff --git a/languages/python/tests/test_declaration.py b/languages/python/tests/test_declaration.py new file mode 100644 index 000000000..4ef9304f1 --- /dev/null +++ b/languages/python/tests/test_declaration.py @@ -0,0 +1,10 @@ +"""The declaration tests, with annotations evaluated as written. + +The tests themselves are in ``_declaration_cases.py``, compiled here without +``from __future__ import annotations``. +""" + +import pathlib + +_CASES = pathlib.Path(__file__).with_name("_declaration_cases.py") +exec(compile(_CASES.read_text(), str(_CASES), "exec", dont_inherit=True), globals()) diff --git a/languages/python/tests/test_declaration_future_annotations.py b/languages/python/tests/test_declaration_future_annotations.py new file mode 100644 index 000000000..3d9d4f5a7 --- /dev/null +++ b/languages/python/tests/test_declaration_future_annotations.py @@ -0,0 +1,20 @@ +"""The declaration tests, compiled with ``from __future__ import annotations``. + +Under that import every annotation is a string until the decorator resolves +it, which is the case a real model module with the import is in. +""" + +import __future__ +import pathlib + +_CASES = pathlib.Path(__file__).with_name("_declaration_cases.py") +exec( + compile( + _CASES.read_text(), + str(_CASES), + "exec", + flags=__future__.annotations.compiler_flag, + dont_inherit=True, + ), + globals(), +)