diff --git a/changelog.d/ordered-scenario-plans.added.md b/changelog.d/ordered-scenario-plans.added.md new file mode 100644 index 00000000..c299780a --- /dev/null +++ b/changelog.d/ordered-scenario-plans.added.md @@ -0,0 +1 @@ +Added ordered UK Scenario plans that preserve modifier order and data-load phases, bind trusted callable sources, and support offline identity verification and worker rebinding. diff --git a/docs/ordered-scenarios.md b/docs/ordered-scenarios.md new file mode 100644 index 00000000..c228de8f --- /dev/null +++ b/docs/ordered-scenarios.md @@ -0,0 +1,40 @@ +# Ordered UK scenarios + +Use `build_ordered_scenario` to preserve several module-level modifiers before and after data loading. Pass actual callables from trusted job code, with a full repository and commit claim for each occurrence. The builder returns `BoundOrderedScenario`, which exposes a native UK `Scenario`, an immutable execution plan and a live receipt identity. + +```python +from policyengine.provenance import build_ordered_scenario + +bound = build_ordered_scenario( + complete_parameter_changes, + before_modifiers=[ + (first_modifier, first_revision), + (second_modifier, second_revision), + ], + after_modifiers=[(third_modifier, third_revision)], +) +simulation = NativeUKSimulation(situation=situation, scenario=bound.scenario) +reform_identity = bound.identity +``` + +Each revision uses `{"repository": "owner/repository", "git_sha": ""}`. The caller supplies the functions, revision claims and `NativeUKSimulation` from its trusted application. See [the runnable example](../examples/ordered_scenario.py) for a synthetic phase demonstration. + +## Execution and identity + +The adapter applies the complete parameter map once before loading data. It then calls every before-load modifier in order, lets the native simulation build its data, and calls every after-load modifier in order. Repeated occurrences execute repeatedly. Resolve a shared policy reference and each member's raw changes into the complete map before calling the builder; keep the application's existing member-conflict checks. + +`BoundOrderedScenario.plan` stores the application order, adapter source digest, complete before-load parameters and ordered leaf proofs for both phases. Each proof records the callable's module, qualified name, source digest, module-source digest, typed module defaults and code revision. The after-load phase accepts no parameter map. The adapter and receipt identity check the same plan, and `recheck()` refuses changes to original parameters, leaf lists, revisions, defaults, loaded code or native phase fields. + +Plans with modifiers use `policyengine-uk.phased_scenario_composition/v1`. Their digests cover `{"representation": ..., "payload": plan}` with RFC 8785 canonicalization and nulls retained. Parameter-only plans keep the existing before-load `scenario/v1` identity; an empty plan keeps `current_law`. Existing Scenario, phased Scenario and legacy composition digests retain their meaning. Legacy composition cannot flatten an ordered plan's phases. + +## Offline verification and workers + +`ordered_scenario_reform_identity(plan)` validates the closed plan and recomputes its official identity from data. It never imports, fetches or executes a module or qualified name from the payload. A source proof remains a producer's claim; applications must check approvals and provenance separately. + +Workers call `bind_ordered_scenario_plan(plan, before_modifiers=..., after_modifiers=...)` with their actual trusted job callables. The binder checks each current source/default/revision proof against the complete carried plan before exposing the native scenario. Trusted `BoundOrderedScenario` objects also support pickle transport and rebind on load. Never unpickle untrusted input. JSON round-trips carry plan data; callers must rebind trusted functions before execution. + +Receipt builders recheck live identities after execution. Keep the bound object that supplied `scenario`, and pass its live `identity` to the receipt builder. Loading an identity from JSON carries no live execution handle. + +The [plan schema](../src/policyengine/data/schemas/ordered_scenario_plan.v1.schema.json) and [API-generated vectors](../src/policyengine/data/schemas/ordered_scenario_plan.v1.vectors.json) support offline implementations. Apply the schema's semantic rules as well as structural validation: preserve ordered typed dictionaries and repetitions, require canonical float hex values, and enforce the existing strict JSON numeric domain. Only the exact reserved `{"$nonfinite": "Infinity"}` and `{"$nonfinite": "-Infinity"}` objects encode infinite parameter values. + +The API rejects arbitrary closures, dynamic modifier chains, methods, partials and dated reform objects. It does not inspect arbitrary closure cells, prove Python write sets, approve jobs or certify data. Callers retain source-based structural consistency checks and data limits. diff --git a/docs/receipts.md b/docs/receipts.md index c6b0317f..42930b5d 100644 --- a/docs/receipts.md +++ b/docs/receipts.md @@ -83,6 +83,8 @@ None of these depends on where the code is checked out or which Python version r The old `scenario_reform_identity` refuses mixed phases and effective-phase/aggregate-flag mismatches instead of assigning the aggregate flag to every part. `composed_reform_identity` v1 refuses phased members because its contract overrides member timing. Bind the actual combined Scenario through the phased API instead. Dated reform objects are outside both Scenario representations: bind an original reform dictionary as its dictionary representation rather than silently omitting it. Closures and computed modifier chains remain unsupported. +For several modifiers in either phase, use the [ordered Scenario API](ordered-scenarios.md). It binds actual trusted callable leaves, preserves their order and repetitions, and provides an offline plan recomposer. + Build the receipt after the run. An identity returned by `scenario_reform_identity` keeps a private handle to the same scenario and code revision. Both receipt builders recompute `reform` and `baseline` identities with handles, and refuse if recomputing is refused or an identity member differs. The handle is excluded from equality, serialization, schemas and digests. A serialized identity loaded back has no handle and is not re-checked; its producer must ensure it describes the run. These checks take snapshots at identity time and receipt build time, so they cannot describe later changes. **The modifier's module.** A digest can only name what the module's source fixes, so the module holding a bound modifier must have a shape whose state that source determines. At module level it may hold only: @@ -190,7 +192,7 @@ The pre-promotion replay interface with policyengine-scorecard follows a differe | `reform.reform_sha256` | `{"representation": ..., "payload": ...}`, with the payload exactly as given, nulls kept (strict JSON conversion with infinities as `{"$nonfinite": "Infinity"}` or `"-Infinity"`, then RFC 8785, with no null removal). For a composition the payload is `{"applied_before_data_load": ..., "application": "parameter_changes_then_modifiers", "members": [{"representation", "reform_sha256", "binding"}, ...]}`, with members in application order. | | `outputs_sha256` | The output payload. | -Reform digests are never normalized across representations. The same keys can legitimately mean different things in different representations: for example, a bare year in a policyengine-uk country dictionary changes that one year only. So each digest is tied to the representation that was submitted. The v1 representations are `policyengine.parameter_values/v1`, `policyengine.reform_dict/v1`, `policyengine-uk.reform_dict/v1`, `policyengine-uk.scenario/v1`, `policyengine-uk.scenario_composition/v1` and `policyengine-sim-api.policy/v1`. +Reform digests are never normalized across representations. The same keys can legitimately mean different things in different representations: for example, a bare year in a policyengine-uk country dictionary changes that one year only. So each digest is tied to the representation that was submitted. Receipts support `policyengine.parameter_values/v1`, `policyengine.reform_dict/v1`, `policyengine-uk.reform_dict/v1`, `policyengine-uk.scenario/v1`, `policyengine-uk.scenario_composition/v1`, `policyengine-uk.phased_scenario/v1`, `policyengine-uk.phased_scenario_composition/v1` and `policyengine-sim-api.policy/v1`. ## Rules JSON Schema cannot express diff --git a/examples/ordered_scenario.py b/examples/ordered_scenario.py new file mode 100644 index 00000000..1f654dd4 --- /dev/null +++ b/examples/ordered_scenario.py @@ -0,0 +1,36 @@ +"""Demonstrate ordered phases with synthetic state and no household data. + +Run from a checkout containing these example functions. Supply that actual +checkout's repository and full commit SHA; the API records the caller's claim. +""" + +import argparse +from types import SimpleNamespace + +from ordered_scenario_modifiers import add_one, double + +from policyengine.provenance import build_ordered_scenario + + +def main(): + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("--repository", required=True) + parser.add_argument("--git-sha", required=True) + args = parser.parse_args() + revision = {"repository": args.repository, "git_sha": args.git_sha} + bound = build_ordered_scenario( + before_modifiers=[(add_one, revision), (double, revision)], + after_modifiers=[(double, revision), (add_one, revision)], + ) + before, after = bound.scenario.parts_by_data_load() + state = SimpleNamespace(value=1) + before.simulation_modifier(state) + assert state.value == 4 + # A native simulation constructs its data between these two calls. + after.simulation_modifier(state) + assert state.value == 9 + print(bound.identity.model_dump_json(indent=2)) + + +if __name__ == "__main__": + main() diff --git a/examples/ordered_scenario_modifiers.py b/examples/ordered_scenario_modifiers.py new file mode 100644 index 00000000..4ad82ed2 --- /dev/null +++ b/examples/ordered_scenario_modifiers.py @@ -0,0 +1,9 @@ +"""Synthetic arithmetic modifiers for the ordered-phase API example.""" + + +def add_one(simulation): + simulation.value += 1 + + +def double(simulation): + simulation.value *= 2 diff --git a/scripts/generate_ordered_scenario_vectors.py b/scripts/generate_ordered_scenario_vectors.py new file mode 100644 index 00000000..bb7c0e23 --- /dev/null +++ b/scripts/generate_ordered_scenario_vectors.py @@ -0,0 +1,118 @@ +"""Generate data-only verifier vectors from the actual ordered-plan producer. + +The modifiers and revision below are synthetic test fixtures. These vectors +assert canonicalization behavior, not approval or certification of a job. +""" + +from __future__ import annotations + +import argparse +import copy +import json +import os +import sys +from pathlib import Path + +ROOT = Path(__file__).resolve().parents[1] +os.environ.setdefault("POLICYENGINE_SKIP_COUNTRY_IMPORTS", "1") +sys.path.insert(0, str(ROOT / "src")) +sys.path.insert(0, str(ROOT)) + +from policyengine.provenance.ordered_scenario import ( # noqa: E402 + build_ordered_scenario, +) +from tests import ordered_receipt_modifiers as modifiers # noqa: E402 + +PATH = ROOT / "src/policyengine/data/schemas/ordered_scenario_plan.v1.vectors.json" +REVISION = {"repository": "PolicyEngine/test", "git_sha": "a" * 40} + + +def build_vectors(): + def leaves(*functions): + return [(function, dict(REVISION)) for function in functions] + + cases = [ + ("empty", build_ordered_scenario()), + ("parameter_only", build_ordered_scenario({"gov.example.rate": {"2027": 0.2}})), + ( + "parameter_infinity", + build_ordered_scenario({"gov.example.cap": {"2027": float("inf")}}), + ), + ( + "two_each_phase", + build_ordered_scenario( + {"gov.example.rate": {"2027": 0.2}}, + before_modifiers=leaves(modifiers.add_one, modifiers.double), + after_modifiers=leaves(modifiers.double, modifiers.add_one), + ), + ), + ( + "reverse_before", + build_ordered_scenario( + before_modifiers=leaves(modifiers.double, modifiers.add_one) + ), + ), + ( + "repeated_leaf", + build_ordered_scenario( + after_modifiers=leaves(modifiers.add_one, modifiers.add_one) + ), + ), + ( + "typed_defaults", + build_ordered_scenario(before_modifiers=leaves(modifiers.add_default)), + ), + ] + valid = [ + { + "id": name, + "plan": bound.plan.model_dump(mode="json"), + "identity": bound.identity.model_dump(mode="json"), + } + for name, bound in cases + ] + base = valid[3]["plan"] + invalid = [] + for name in ( + "unknown_field", + "unknown_application", + "after_parameters", + "unknown_leaf_field", + "unknown_default_type", + ): + plan = copy.deepcopy(base) + if name == "unknown_field": + plan["invented"] = True + elif name == "unknown_application": + plan["application"] = "after_then_before" + elif name == "after_parameters": + plan["after"]["parameter_changes"] = {"gov.example.rate": 1} + elif name == "unknown_leaf_field": + plan["before"]["modifier_leaves"][0]["claimed_leaf_count"] = 1 + else: + plan["before"]["modifier_leaves"][0]["module_defaults"] = { + "x": {"type": "object"} + } + invalid.append({"id": name, "plan": plan}) + return { + "schema_version": 1, + "purpose": "Synthetic canonicalization vectors; no job approval or certification.", + "valid": valid, + "invalid": invalid, + } + + +def main(): + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("--check", action="store_true") + args = parser.parse_args() + content = json.dumps(build_vectors(), indent=2, sort_keys=True) + "\n" + if args.check: + if not PATH.exists() or PATH.read_text() != content: + raise SystemExit("Ordered Scenario vectors are stale.") + else: + PATH.write_text(content) + + +if __name__ == "__main__": + main() diff --git a/scripts/generate_receipt_schemas.py b/scripts/generate_receipt_schemas.py index 8340948e..35adc490 100644 --- a/scripts/generate_receipt_schemas.py +++ b/scripts/generate_receipt_schemas.py @@ -24,10 +24,12 @@ os.environ.setdefault("POLICYENGINE_SKIP_COUNTRY_IMPORTS", "1") sys.path.insert(0, str(REPO_ROOT / "src")) +from policyengine.provenance.ordered_scenario import OrderedScenarioPlan # noqa: E402 from policyengine.provenance.receipt import ExecutionReceipt # noqa: E402 SCHEMAS = { "execution_receipt.v1.schema.json": ExecutionReceipt.json_schema, + "ordered_scenario_plan.v1.schema.json": OrderedScenarioPlan.json_schema, } diff --git a/src/policyengine/data/schemas/execution_receipt.v1.schema.json b/src/policyengine/data/schemas/execution_receipt.v1.schema.json index 05079f6b..ac1a688c 100644 --- a/src/policyengine/data/schemas/execution_receipt.v1.schema.json +++ b/src/policyengine/data/schemas/execution_receipt.v1.schema.json @@ -382,6 +382,30 @@ }, "ReformIdentity": { "additionalProperties": false, + "allOf": [ + { + "if": { + "properties": { + "representation": { + "const": "policyengine-uk.phased_scenario_composition/v1" + } + }, + "required": [ + "representation" + ] + }, + "then": { + "properties": { + "binding": { + "const": "code_revision" + } + }, + "required": [ + "binding" + ] + } + } + ], "description": "A reform or baseline, bound by the digest of its canonical form.", "properties": { "binding": { @@ -440,7 +464,8 @@ "policyengine-sim-api.policy/v1", "policyengine-uk.scenario/v1", "policyengine-uk.scenario_composition/v1", - "policyengine-uk.phased_scenario/v1" + "policyengine-uk.phased_scenario/v1", + "policyengine-uk.phased_scenario_composition/v1" ], "type": "string" }, @@ -832,6 +857,7 @@ "execution_identity_sha256": "{schema_version, country_id, wrapper, model, core, data, certification_status, certificate?, regional_certification?} where certificate keeps only bundle_version, country_id, certificate_sha256, compatibility_basis, certified_for_model_version, certified_for_core_version, certified_dataset_sha256 and certified_by", "outputs_sha256": "the canonical output payload the receipt binds", "receipt_sha256": "the whole receipt without runtime and without reform.name and baseline.name", + "reform.ordered_scenario_sha256": "For policyengine-uk.phased_scenario_composition/v1, RFC 8785 over {representation, payload: OrderedScenarioPlan}; retain nulls and every ordered modifier occurrence, including repetitions. Parameters occur only before data load; leaf proofs carry actual source/module/default/revision identities and the plan binds the adapter source. Binding is code_revision. Use ordered_scenario_reform_identity for data-only recomposition. Parameter-only ordered plans retain the existing before-load scenario/v1 identity.", "reform.phased_scenario_sha256": "For policyengine-uk.phased_scenario/v1, RFC 8785 over {representation, payload}, with payload {application: before_data_load_then_after_data_load, before: {representation, reform_sha256, binding}, after: {representation, reform_sha256, binding}}; each phase binds the native Scenario.parts_by_data_load() result through scenario/v1. Common parameter changes occur once, in the before phase. A scenario_composition/v1 cannot contain this representation.", "reform.reform_sha256": "RFC 8785 over {representation, payload} where payload is the reform exactly as given in that representation, nulls kept (a null in a reform can mean something, unlike an absent receipt field); for policyengine-uk.scenario_composition/v1 the payload is {applied_before_data_load, application: parameter_changes_then_modifiers, members: [{representation, reform_sha256, binding}, ...]} with members in application order; an infinite number in a payload is the object {\"$nonfinite\": \"Infinity\"} or {\"$nonfinite\": \"-Infinity\"}, and a payload may not use the key $nonfinite itself", "regional_certification.record_sha256": "the bundle manifest's data_releases..regional_certification record, its evidence included" diff --git a/src/policyengine/data/schemas/ordered_scenario_plan.v1.schema.json b/src/policyengine/data/schemas/ordered_scenario_plan.v1.schema.json new file mode 100644 index 00000000..6fa0ade2 --- /dev/null +++ b/src/policyengine/data/schemas/ordered_scenario_plan.v1.schema.json @@ -0,0 +1,369 @@ +{ + "$defs": { + "ModifierCodeRevision": { + "additionalProperties": false, + "description": "The same full repository/commit claim accepted by the Scenario binder.", + "properties": { + "git_sha": { + "pattern": "^(?:[0-9a-f]{40}|[0-9a-f]{64})$", + "title": "Git Sha", + "type": "string" + }, + "repository": { + "minLength": 1, + "title": "Repository", + "type": "string" + } + }, + "required": [ + "repository", + "git_sha" + ], + "title": "ModifierCodeRevision", + "type": "object" + }, + "ModifierLeafIdentity": { + "additionalProperties": false, + "description": "Inert provenance for one occurrence of an actual module-level modifier.", + "properties": { + "code_revision": { + "$ref": "#/$defs/ModifierCodeRevision" + }, + "module": { + "pattern": "^[A-Za-z_]\\w*(?:\\.[A-Za-z_]\\w*)*$", + "title": "Module", + "type": "string" + }, + "module_defaults": { + "anyOf": [ + { + "type": "null" + }, + { + "additionalProperties": { + "$ref": "#/$defs/TypedDefault" + }, + "minProperties": 1, + "propertyNames": { + "minLength": 1 + }, + "type": "object" + } + ] + }, + "module_source_sha256": { + "pattern": "^[0-9a-f]{64}$", + "title": "Module Source Sha256", + "type": "string" + }, + "qualname": { + "pattern": "^[A-Za-z_]\\w*$", + "title": "Qualname", + "type": "string" + }, + "source_sha256": { + "pattern": "^[0-9a-f]{64}$", + "title": "Source Sha256", + "type": "string" + } + }, + "required": [ + "module", + "qualname", + "source_sha256", + "module_source_sha256", + "module_defaults", + "code_revision" + ], + "title": "ModifierLeafIdentity", + "type": "object" + }, + "OrderedScenarioPhase": { + "additionalProperties": false, + "description": "The complete parameter map and ordered modifier occurrences of one phase.", + "properties": { + "modifier_leaves": { + "items": { + "$ref": "#/$defs/ModifierLeafIdentity" + }, + "title": "Modifier Leaves", + "type": "array" + }, + "parameter_changes": { + "additionalProperties": { + "anyOf": [ + { + "anyOf": [ + { + "type": "boolean" + }, + { + "type": "number" + }, + { + "additionalProperties": false, + "properties": { + "$nonfinite": { + "enum": [ + "Infinity", + "-Infinity" + ] + } + }, + "required": [ + "$nonfinite" + ], + "type": "object" + } + ] + }, + { + "additionalProperties": { + "anyOf": [ + { + "type": "boolean" + }, + { + "type": "number" + }, + { + "additionalProperties": false, + "properties": { + "$nonfinite": { + "enum": [ + "Infinity", + "-Infinity" + ] + } + }, + "required": [ + "$nonfinite" + ], + "type": "object" + } + ] + }, + "propertyNames": { + "minLength": 1, + "not": { + "const": "$nonfinite" + } + }, + "type": "object" + } + ] + }, + "propertyNames": { + "minLength": 1, + "not": { + "const": "$nonfinite" + } + }, + "type": "object" + } + }, + "required": [ + "parameter_changes", + "modifier_leaves" + ], + "title": "OrderedScenarioPhase", + "type": "object" + }, + "TypedDefault": { + "oneOf": [ + { + "additionalProperties": false, + "properties": { + "type": { + "const": "none" + } + }, + "required": [ + "type" + ], + "type": "object" + }, + { + "additionalProperties": false, + "properties": { + "type": { + "const": "bool" + }, + "value": { + "type": "boolean" + } + }, + "required": [ + "type", + "value" + ], + "type": "object" + }, + { + "additionalProperties": false, + "properties": { + "type": { + "const": "int" + }, + "value": { + "maximum": 9007199254740991, + "minimum": -9007199254740991, + "type": "integer" + } + }, + "required": [ + "type", + "value" + ], + "type": "object" + }, + { + "additionalProperties": false, + "properties": { + "type": { + "const": "str" + }, + "value": { + "type": "string" + } + }, + "required": [ + "type", + "value" + ], + "type": "object" + }, + { + "additionalProperties": false, + "properties": { + "hex": { + "pattern": "^(?:-?inf|-?0x[0-9a-f]+\\.[0-9a-f]+p[+-][0-9]+)$", + "type": "string" + }, + "type": { + "const": "float" + } + }, + "required": [ + "type", + "hex" + ], + "type": "object" + }, + { + "additionalProperties": false, + "properties": { + "items": { + "items": { + "$ref": "#/$defs/TypedDefault" + }, + "type": "array" + }, + "type": { + "const": "list" + } + }, + "required": [ + "type", + "items" + ], + "type": "object" + }, + { + "additionalProperties": false, + "properties": { + "items": { + "items": { + "$ref": "#/$defs/TypedDefault" + }, + "type": "array" + }, + "type": { + "const": "tuple" + } + }, + "required": [ + "type", + "items" + ], + "type": "object" + }, + { + "additionalProperties": false, + "properties": { + "items": { + "items": { + "maxItems": 2, + "minItems": 2, + "prefixItems": [ + { + "type": "string" + }, + { + "$ref": "#/$defs/TypedDefault" + } + ], + "type": "array" + }, + "type": "array" + }, + "type": { + "const": "dict" + } + }, + "required": [ + "type", + "items" + ], + "type": "object" + } + ] + } + }, + "$id": "https://policyengine.org/schemas/ordered_scenario_plan.v1.schema.json", + "additionalProperties": false, + "description": "An immutable JSON execution recipe; reading one executes no locators.", + "properties": { + "adapter_source_sha256": { + "pattern": "^[0-9a-f]{64}$", + "title": "Adapter Source Sha256", + "type": "string" + }, + "after": { + "allOf": [ + { + "$ref": "#/$defs/OrderedScenarioPhase" + }, + { + "properties": { + "parameter_changes": { + "const": {} + } + } + } + ] + }, + "application": { + "const": "before_data_load_then_after_data_load", + "title": "Application", + "type": "string" + }, + "before": { + "$ref": "#/$defs/OrderedScenarioPhase" + } + }, + "required": [ + "application", + "adapter_source_sha256", + "before", + "after" + ], + "title": "OrderedScenarioPlan", + "type": "object", + "x-policyengine-rules": [ + "Canonicalization is RFC8785 with nulls retained; no locator is imported or executed.", + "Default float hex strings must equal float.fromhex(value).hex(); typed dictionary keys are unique and ordered.", + "Apply existing strict JSON/RFC8785 numeric-domain checks; JSON NaN and numeric Infinity are invalid. Only exact reserved objects encode parameter infinities.", + "Offline provenance is an assertion, not permission to execute. Rebind trusted actual leaf callables and compare the complete plan before runtime." + ] +} diff --git a/src/policyengine/data/schemas/ordered_scenario_plan.v1.vectors.json b/src/policyengine/data/schemas/ordered_scenario_plan.v1.vectors.json new file mode 100644 index 00000000..374c7802 --- /dev/null +++ b/src/policyengine/data/schemas/ordered_scenario_plan.v1.vectors.json @@ -0,0 +1,1313 @@ +{ + "invalid": [ + { + "id": "unknown_field", + "plan": { + "adapter_source_sha256": "692a6848be3e366a1243fe50668b152692374c88fd4cc94bd67008b78157519c", + "after": { + "modifier_leaves": [ + { + "code_revision": { + "git_sha": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", + "repository": "PolicyEngine/test" + }, + "module": "tests.ordered_receipt_modifiers", + "module_defaults": { + "add_default": { + "items": [ + [ + "defaults", + { + "items": [ + { + "type": "int", + "value": 3 + } + ], + "type": "list" + } + ], + [ + "kwdefaults", + { + "items": [], + "type": "dict" + } + ] + ], + "type": "dict" + } + }, + "module_source_sha256": "d1649b4c79b32c9d0cccbe42f5f5aca301b81ed853ad8bbb972c396d8eae9823", + "qualname": "double", + "source_sha256": "9c2b30ed7c115520b94c29a615568ea8e3611b8d81bf9722b67070e3f7af958c" + }, + { + "code_revision": { + "git_sha": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", + "repository": "PolicyEngine/test" + }, + "module": "tests.ordered_receipt_modifiers", + "module_defaults": { + "add_default": { + "items": [ + [ + "defaults", + { + "items": [ + { + "type": "int", + "value": 3 + } + ], + "type": "list" + } + ], + [ + "kwdefaults", + { + "items": [], + "type": "dict" + } + ] + ], + "type": "dict" + } + }, + "module_source_sha256": "d1649b4c79b32c9d0cccbe42f5f5aca301b81ed853ad8bbb972c396d8eae9823", + "qualname": "add_one", + "source_sha256": "a87a9c457524c35b0d3623ca94f2cbe965325463f8195d30df6ea3fae8061fcb" + } + ], + "parameter_changes": {} + }, + "application": "before_data_load_then_after_data_load", + "before": { + "modifier_leaves": [ + { + "code_revision": { + "git_sha": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", + "repository": "PolicyEngine/test" + }, + "module": "tests.ordered_receipt_modifiers", + "module_defaults": { + "add_default": { + "items": [ + [ + "defaults", + { + "items": [ + { + "type": "int", + "value": 3 + } + ], + "type": "list" + } + ], + [ + "kwdefaults", + { + "items": [], + "type": "dict" + } + ] + ], + "type": "dict" + } + }, + "module_source_sha256": "d1649b4c79b32c9d0cccbe42f5f5aca301b81ed853ad8bbb972c396d8eae9823", + "qualname": "add_one", + "source_sha256": "a87a9c457524c35b0d3623ca94f2cbe965325463f8195d30df6ea3fae8061fcb" + }, + { + "code_revision": { + "git_sha": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", + "repository": "PolicyEngine/test" + }, + "module": "tests.ordered_receipt_modifiers", + "module_defaults": { + "add_default": { + "items": [ + [ + "defaults", + { + "items": [ + { + "type": "int", + "value": 3 + } + ], + "type": "list" + } + ], + [ + "kwdefaults", + { + "items": [], + "type": "dict" + } + ] + ], + "type": "dict" + } + }, + "module_source_sha256": "d1649b4c79b32c9d0cccbe42f5f5aca301b81ed853ad8bbb972c396d8eae9823", + "qualname": "double", + "source_sha256": "9c2b30ed7c115520b94c29a615568ea8e3611b8d81bf9722b67070e3f7af958c" + } + ], + "parameter_changes": { + "gov.example.rate": { + "2027": 0.2 + } + } + }, + "invented": true + } + }, + { + "id": "unknown_application", + "plan": { + "adapter_source_sha256": "692a6848be3e366a1243fe50668b152692374c88fd4cc94bd67008b78157519c", + "after": { + "modifier_leaves": [ + { + "code_revision": { + "git_sha": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", + "repository": "PolicyEngine/test" + }, + "module": "tests.ordered_receipt_modifiers", + "module_defaults": { + "add_default": { + "items": [ + [ + "defaults", + { + "items": [ + { + "type": "int", + "value": 3 + } + ], + "type": "list" + } + ], + [ + "kwdefaults", + { + "items": [], + "type": "dict" + } + ] + ], + "type": "dict" + } + }, + "module_source_sha256": "d1649b4c79b32c9d0cccbe42f5f5aca301b81ed853ad8bbb972c396d8eae9823", + "qualname": "double", + "source_sha256": "9c2b30ed7c115520b94c29a615568ea8e3611b8d81bf9722b67070e3f7af958c" + }, + { + "code_revision": { + "git_sha": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", + "repository": "PolicyEngine/test" + }, + "module": "tests.ordered_receipt_modifiers", + "module_defaults": { + "add_default": { + "items": [ + [ + "defaults", + { + "items": [ + { + "type": "int", + "value": 3 + } + ], + "type": "list" + } + ], + [ + "kwdefaults", + { + "items": [], + "type": "dict" + } + ] + ], + "type": "dict" + } + }, + "module_source_sha256": "d1649b4c79b32c9d0cccbe42f5f5aca301b81ed853ad8bbb972c396d8eae9823", + "qualname": "add_one", + "source_sha256": "a87a9c457524c35b0d3623ca94f2cbe965325463f8195d30df6ea3fae8061fcb" + } + ], + "parameter_changes": {} + }, + "application": "after_then_before", + "before": { + "modifier_leaves": [ + { + "code_revision": { + "git_sha": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", + "repository": "PolicyEngine/test" + }, + "module": "tests.ordered_receipt_modifiers", + "module_defaults": { + "add_default": { + "items": [ + [ + "defaults", + { + "items": [ + { + "type": "int", + "value": 3 + } + ], + "type": "list" + } + ], + [ + "kwdefaults", + { + "items": [], + "type": "dict" + } + ] + ], + "type": "dict" + } + }, + "module_source_sha256": "d1649b4c79b32c9d0cccbe42f5f5aca301b81ed853ad8bbb972c396d8eae9823", + "qualname": "add_one", + "source_sha256": "a87a9c457524c35b0d3623ca94f2cbe965325463f8195d30df6ea3fae8061fcb" + }, + { + "code_revision": { + "git_sha": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", + "repository": "PolicyEngine/test" + }, + "module": "tests.ordered_receipt_modifiers", + "module_defaults": { + "add_default": { + "items": [ + [ + "defaults", + { + "items": [ + { + "type": "int", + "value": 3 + } + ], + "type": "list" + } + ], + [ + "kwdefaults", + { + "items": [], + "type": "dict" + } + ] + ], + "type": "dict" + } + }, + "module_source_sha256": "d1649b4c79b32c9d0cccbe42f5f5aca301b81ed853ad8bbb972c396d8eae9823", + "qualname": "double", + "source_sha256": "9c2b30ed7c115520b94c29a615568ea8e3611b8d81bf9722b67070e3f7af958c" + } + ], + "parameter_changes": { + "gov.example.rate": { + "2027": 0.2 + } + } + } + } + }, + { + "id": "after_parameters", + "plan": { + "adapter_source_sha256": "692a6848be3e366a1243fe50668b152692374c88fd4cc94bd67008b78157519c", + "after": { + "modifier_leaves": [ + { + "code_revision": { + "git_sha": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", + "repository": "PolicyEngine/test" + }, + "module": "tests.ordered_receipt_modifiers", + "module_defaults": { + "add_default": { + "items": [ + [ + "defaults", + { + "items": [ + { + "type": "int", + "value": 3 + } + ], + "type": "list" + } + ], + [ + "kwdefaults", + { + "items": [], + "type": "dict" + } + ] + ], + "type": "dict" + } + }, + "module_source_sha256": "d1649b4c79b32c9d0cccbe42f5f5aca301b81ed853ad8bbb972c396d8eae9823", + "qualname": "double", + "source_sha256": "9c2b30ed7c115520b94c29a615568ea8e3611b8d81bf9722b67070e3f7af958c" + }, + { + "code_revision": { + "git_sha": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", + "repository": "PolicyEngine/test" + }, + "module": "tests.ordered_receipt_modifiers", + "module_defaults": { + "add_default": { + "items": [ + [ + "defaults", + { + "items": [ + { + "type": "int", + "value": 3 + } + ], + "type": "list" + } + ], + [ + "kwdefaults", + { + "items": [], + "type": "dict" + } + ] + ], + "type": "dict" + } + }, + "module_source_sha256": "d1649b4c79b32c9d0cccbe42f5f5aca301b81ed853ad8bbb972c396d8eae9823", + "qualname": "add_one", + "source_sha256": "a87a9c457524c35b0d3623ca94f2cbe965325463f8195d30df6ea3fae8061fcb" + } + ], + "parameter_changes": { + "gov.example.rate": 1 + } + }, + "application": "before_data_load_then_after_data_load", + "before": { + "modifier_leaves": [ + { + "code_revision": { + "git_sha": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", + "repository": "PolicyEngine/test" + }, + "module": "tests.ordered_receipt_modifiers", + "module_defaults": { + "add_default": { + "items": [ + [ + "defaults", + { + "items": [ + { + "type": "int", + "value": 3 + } + ], + "type": "list" + } + ], + [ + "kwdefaults", + { + "items": [], + "type": "dict" + } + ] + ], + "type": "dict" + } + }, + "module_source_sha256": "d1649b4c79b32c9d0cccbe42f5f5aca301b81ed853ad8bbb972c396d8eae9823", + "qualname": "add_one", + "source_sha256": "a87a9c457524c35b0d3623ca94f2cbe965325463f8195d30df6ea3fae8061fcb" + }, + { + "code_revision": { + "git_sha": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", + "repository": "PolicyEngine/test" + }, + "module": "tests.ordered_receipt_modifiers", + "module_defaults": { + "add_default": { + "items": [ + [ + "defaults", + { + "items": [ + { + "type": "int", + "value": 3 + } + ], + "type": "list" + } + ], + [ + "kwdefaults", + { + "items": [], + "type": "dict" + } + ] + ], + "type": "dict" + } + }, + "module_source_sha256": "d1649b4c79b32c9d0cccbe42f5f5aca301b81ed853ad8bbb972c396d8eae9823", + "qualname": "double", + "source_sha256": "9c2b30ed7c115520b94c29a615568ea8e3611b8d81bf9722b67070e3f7af958c" + } + ], + "parameter_changes": { + "gov.example.rate": { + "2027": 0.2 + } + } + } + } + }, + { + "id": "unknown_leaf_field", + "plan": { + "adapter_source_sha256": "692a6848be3e366a1243fe50668b152692374c88fd4cc94bd67008b78157519c", + "after": { + "modifier_leaves": [ + { + "code_revision": { + "git_sha": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", + "repository": "PolicyEngine/test" + }, + "module": "tests.ordered_receipt_modifiers", + "module_defaults": { + "add_default": { + "items": [ + [ + "defaults", + { + "items": [ + { + "type": "int", + "value": 3 + } + ], + "type": "list" + } + ], + [ + "kwdefaults", + { + "items": [], + "type": "dict" + } + ] + ], + "type": "dict" + } + }, + "module_source_sha256": "d1649b4c79b32c9d0cccbe42f5f5aca301b81ed853ad8bbb972c396d8eae9823", + "qualname": "double", + "source_sha256": "9c2b30ed7c115520b94c29a615568ea8e3611b8d81bf9722b67070e3f7af958c" + }, + { + "code_revision": { + "git_sha": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", + "repository": "PolicyEngine/test" + }, + "module": "tests.ordered_receipt_modifiers", + "module_defaults": { + "add_default": { + "items": [ + [ + "defaults", + { + "items": [ + { + "type": "int", + "value": 3 + } + ], + "type": "list" + } + ], + [ + "kwdefaults", + { + "items": [], + "type": "dict" + } + ] + ], + "type": "dict" + } + }, + "module_source_sha256": "d1649b4c79b32c9d0cccbe42f5f5aca301b81ed853ad8bbb972c396d8eae9823", + "qualname": "add_one", + "source_sha256": "a87a9c457524c35b0d3623ca94f2cbe965325463f8195d30df6ea3fae8061fcb" + } + ], + "parameter_changes": {} + }, + "application": "before_data_load_then_after_data_load", + "before": { + "modifier_leaves": [ + { + "claimed_leaf_count": 1, + "code_revision": { + "git_sha": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", + "repository": "PolicyEngine/test" + }, + "module": "tests.ordered_receipt_modifiers", + "module_defaults": { + "add_default": { + "items": [ + [ + "defaults", + { + "items": [ + { + "type": "int", + "value": 3 + } + ], + "type": "list" + } + ], + [ + "kwdefaults", + { + "items": [], + "type": "dict" + } + ] + ], + "type": "dict" + } + }, + "module_source_sha256": "d1649b4c79b32c9d0cccbe42f5f5aca301b81ed853ad8bbb972c396d8eae9823", + "qualname": "add_one", + "source_sha256": "a87a9c457524c35b0d3623ca94f2cbe965325463f8195d30df6ea3fae8061fcb" + }, + { + "code_revision": { + "git_sha": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", + "repository": "PolicyEngine/test" + }, + "module": "tests.ordered_receipt_modifiers", + "module_defaults": { + "add_default": { + "items": [ + [ + "defaults", + { + "items": [ + { + "type": "int", + "value": 3 + } + ], + "type": "list" + } + ], + [ + "kwdefaults", + { + "items": [], + "type": "dict" + } + ] + ], + "type": "dict" + } + }, + "module_source_sha256": "d1649b4c79b32c9d0cccbe42f5f5aca301b81ed853ad8bbb972c396d8eae9823", + "qualname": "double", + "source_sha256": "9c2b30ed7c115520b94c29a615568ea8e3611b8d81bf9722b67070e3f7af958c" + } + ], + "parameter_changes": { + "gov.example.rate": { + "2027": 0.2 + } + } + } + } + }, + { + "id": "unknown_default_type", + "plan": { + "adapter_source_sha256": "692a6848be3e366a1243fe50668b152692374c88fd4cc94bd67008b78157519c", + "after": { + "modifier_leaves": [ + { + "code_revision": { + "git_sha": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", + "repository": "PolicyEngine/test" + }, + "module": "tests.ordered_receipt_modifiers", + "module_defaults": { + "add_default": { + "items": [ + [ + "defaults", + { + "items": [ + { + "type": "int", + "value": 3 + } + ], + "type": "list" + } + ], + [ + "kwdefaults", + { + "items": [], + "type": "dict" + } + ] + ], + "type": "dict" + } + }, + "module_source_sha256": "d1649b4c79b32c9d0cccbe42f5f5aca301b81ed853ad8bbb972c396d8eae9823", + "qualname": "double", + "source_sha256": "9c2b30ed7c115520b94c29a615568ea8e3611b8d81bf9722b67070e3f7af958c" + }, + { + "code_revision": { + "git_sha": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", + "repository": "PolicyEngine/test" + }, + "module": "tests.ordered_receipt_modifiers", + "module_defaults": { + "add_default": { + "items": [ + [ + "defaults", + { + "items": [ + { + "type": "int", + "value": 3 + } + ], + "type": "list" + } + ], + [ + "kwdefaults", + { + "items": [], + "type": "dict" + } + ] + ], + "type": "dict" + } + }, + "module_source_sha256": "d1649b4c79b32c9d0cccbe42f5f5aca301b81ed853ad8bbb972c396d8eae9823", + "qualname": "add_one", + "source_sha256": "a87a9c457524c35b0d3623ca94f2cbe965325463f8195d30df6ea3fae8061fcb" + } + ], + "parameter_changes": {} + }, + "application": "before_data_load_then_after_data_load", + "before": { + "modifier_leaves": [ + { + "code_revision": { + "git_sha": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", + "repository": "PolicyEngine/test" + }, + "module": "tests.ordered_receipt_modifiers", + "module_defaults": { + "x": { + "type": "object" + } + }, + "module_source_sha256": "d1649b4c79b32c9d0cccbe42f5f5aca301b81ed853ad8bbb972c396d8eae9823", + "qualname": "add_one", + "source_sha256": "a87a9c457524c35b0d3623ca94f2cbe965325463f8195d30df6ea3fae8061fcb" + }, + { + "code_revision": { + "git_sha": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", + "repository": "PolicyEngine/test" + }, + "module": "tests.ordered_receipt_modifiers", + "module_defaults": { + "add_default": { + "items": [ + [ + "defaults", + { + "items": [ + { + "type": "int", + "value": 3 + } + ], + "type": "list" + } + ], + [ + "kwdefaults", + { + "items": [], + "type": "dict" + } + ] + ], + "type": "dict" + } + }, + "module_source_sha256": "d1649b4c79b32c9d0cccbe42f5f5aca301b81ed853ad8bbb972c396d8eae9823", + "qualname": "double", + "source_sha256": "9c2b30ed7c115520b94c29a615568ea8e3611b8d81bf9722b67070e3f7af958c" + } + ], + "parameter_changes": { + "gov.example.rate": { + "2027": 0.2 + } + } + } + } + } + ], + "purpose": "Synthetic canonicalization vectors; no job approval or certification.", + "schema_version": 1, + "valid": [ + { + "id": "empty", + "identity": { + "binding": "content", + "kind": "current_law", + "name": null, + "reform_sha256": null, + "representation": null + }, + "plan": { + "adapter_source_sha256": "692a6848be3e366a1243fe50668b152692374c88fd4cc94bd67008b78157519c", + "after": { + "modifier_leaves": [], + "parameter_changes": {} + }, + "application": "before_data_load_then_after_data_load", + "before": { + "modifier_leaves": [], + "parameter_changes": {} + } + } + }, + { + "id": "parameter_only", + "identity": { + "binding": "content", + "kind": "reform", + "name": null, + "reform_sha256": "41da903bbae85350d60fda120cccb32ebfbed0ace0102b9b4e1bdaca0c556b06", + "representation": "policyengine-uk.scenario/v1" + }, + "plan": { + "adapter_source_sha256": "692a6848be3e366a1243fe50668b152692374c88fd4cc94bd67008b78157519c", + "after": { + "modifier_leaves": [], + "parameter_changes": {} + }, + "application": "before_data_load_then_after_data_load", + "before": { + "modifier_leaves": [], + "parameter_changes": { + "gov.example.rate": { + "2027": 0.2 + } + } + } + } + }, + { + "id": "parameter_infinity", + "identity": { + "binding": "content", + "kind": "reform", + "name": null, + "reform_sha256": "2e7342806436230de7e82b6364d217829175b4b74806b33e46e87d8b9a77107a", + "representation": "policyengine-uk.scenario/v1" + }, + "plan": { + "adapter_source_sha256": "692a6848be3e366a1243fe50668b152692374c88fd4cc94bd67008b78157519c", + "after": { + "modifier_leaves": [], + "parameter_changes": {} + }, + "application": "before_data_load_then_after_data_load", + "before": { + "modifier_leaves": [], + "parameter_changes": { + "gov.example.cap": { + "2027": { + "$nonfinite": "Infinity" + } + } + } + } + } + }, + { + "id": "two_each_phase", + "identity": { + "binding": "code_revision", + "kind": "reform", + "name": null, + "reform_sha256": "1c007f55edc6f8054a28035c0784cdb4e420b87927e534ee77d0bf391b51148c", + "representation": "policyengine-uk.phased_scenario_composition/v1" + }, + "plan": { + "adapter_source_sha256": "692a6848be3e366a1243fe50668b152692374c88fd4cc94bd67008b78157519c", + "after": { + "modifier_leaves": [ + { + "code_revision": { + "git_sha": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", + "repository": "PolicyEngine/test" + }, + "module": "tests.ordered_receipt_modifiers", + "module_defaults": { + "add_default": { + "items": [ + [ + "defaults", + { + "items": [ + { + "type": "int", + "value": 3 + } + ], + "type": "list" + } + ], + [ + "kwdefaults", + { + "items": [], + "type": "dict" + } + ] + ], + "type": "dict" + } + }, + "module_source_sha256": "d1649b4c79b32c9d0cccbe42f5f5aca301b81ed853ad8bbb972c396d8eae9823", + "qualname": "double", + "source_sha256": "9c2b30ed7c115520b94c29a615568ea8e3611b8d81bf9722b67070e3f7af958c" + }, + { + "code_revision": { + "git_sha": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", + "repository": "PolicyEngine/test" + }, + "module": "tests.ordered_receipt_modifiers", + "module_defaults": { + "add_default": { + "items": [ + [ + "defaults", + { + "items": [ + { + "type": "int", + "value": 3 + } + ], + "type": "list" + } + ], + [ + "kwdefaults", + { + "items": [], + "type": "dict" + } + ] + ], + "type": "dict" + } + }, + "module_source_sha256": "d1649b4c79b32c9d0cccbe42f5f5aca301b81ed853ad8bbb972c396d8eae9823", + "qualname": "add_one", + "source_sha256": "a87a9c457524c35b0d3623ca94f2cbe965325463f8195d30df6ea3fae8061fcb" + } + ], + "parameter_changes": {} + }, + "application": "before_data_load_then_after_data_load", + "before": { + "modifier_leaves": [ + { + "code_revision": { + "git_sha": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", + "repository": "PolicyEngine/test" + }, + "module": "tests.ordered_receipt_modifiers", + "module_defaults": { + "add_default": { + "items": [ + [ + "defaults", + { + "items": [ + { + "type": "int", + "value": 3 + } + ], + "type": "list" + } + ], + [ + "kwdefaults", + { + "items": [], + "type": "dict" + } + ] + ], + "type": "dict" + } + }, + "module_source_sha256": "d1649b4c79b32c9d0cccbe42f5f5aca301b81ed853ad8bbb972c396d8eae9823", + "qualname": "add_one", + "source_sha256": "a87a9c457524c35b0d3623ca94f2cbe965325463f8195d30df6ea3fae8061fcb" + }, + { + "code_revision": { + "git_sha": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", + "repository": "PolicyEngine/test" + }, + "module": "tests.ordered_receipt_modifiers", + "module_defaults": { + "add_default": { + "items": [ + [ + "defaults", + { + "items": [ + { + "type": "int", + "value": 3 + } + ], + "type": "list" + } + ], + [ + "kwdefaults", + { + "items": [], + "type": "dict" + } + ] + ], + "type": "dict" + } + }, + "module_source_sha256": "d1649b4c79b32c9d0cccbe42f5f5aca301b81ed853ad8bbb972c396d8eae9823", + "qualname": "double", + "source_sha256": "9c2b30ed7c115520b94c29a615568ea8e3611b8d81bf9722b67070e3f7af958c" + } + ], + "parameter_changes": { + "gov.example.rate": { + "2027": 0.2 + } + } + } + } + }, + { + "id": "reverse_before", + "identity": { + "binding": "code_revision", + "kind": "reform", + "name": null, + "reform_sha256": "5efde0b2a8b07f024b70f383d7da18a00181482249519f0b300f1488b86c2886", + "representation": "policyengine-uk.phased_scenario_composition/v1" + }, + "plan": { + "adapter_source_sha256": "692a6848be3e366a1243fe50668b152692374c88fd4cc94bd67008b78157519c", + "after": { + "modifier_leaves": [], + "parameter_changes": {} + }, + "application": "before_data_load_then_after_data_load", + "before": { + "modifier_leaves": [ + { + "code_revision": { + "git_sha": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", + "repository": "PolicyEngine/test" + }, + "module": "tests.ordered_receipt_modifiers", + "module_defaults": { + "add_default": { + "items": [ + [ + "defaults", + { + "items": [ + { + "type": "int", + "value": 3 + } + ], + "type": "list" + } + ], + [ + "kwdefaults", + { + "items": [], + "type": "dict" + } + ] + ], + "type": "dict" + } + }, + "module_source_sha256": "d1649b4c79b32c9d0cccbe42f5f5aca301b81ed853ad8bbb972c396d8eae9823", + "qualname": "double", + "source_sha256": "9c2b30ed7c115520b94c29a615568ea8e3611b8d81bf9722b67070e3f7af958c" + }, + { + "code_revision": { + "git_sha": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", + "repository": "PolicyEngine/test" + }, + "module": "tests.ordered_receipt_modifiers", + "module_defaults": { + "add_default": { + "items": [ + [ + "defaults", + { + "items": [ + { + "type": "int", + "value": 3 + } + ], + "type": "list" + } + ], + [ + "kwdefaults", + { + "items": [], + "type": "dict" + } + ] + ], + "type": "dict" + } + }, + "module_source_sha256": "d1649b4c79b32c9d0cccbe42f5f5aca301b81ed853ad8bbb972c396d8eae9823", + "qualname": "add_one", + "source_sha256": "a87a9c457524c35b0d3623ca94f2cbe965325463f8195d30df6ea3fae8061fcb" + } + ], + "parameter_changes": {} + } + } + }, + { + "id": "repeated_leaf", + "identity": { + "binding": "code_revision", + "kind": "reform", + "name": null, + "reform_sha256": "007409e0f2eefd9b4ae5c9db81d129cce7d3956602052bc853dd354fc25c5ed2", + "representation": "policyengine-uk.phased_scenario_composition/v1" + }, + "plan": { + "adapter_source_sha256": "692a6848be3e366a1243fe50668b152692374c88fd4cc94bd67008b78157519c", + "after": { + "modifier_leaves": [ + { + "code_revision": { + "git_sha": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", + "repository": "PolicyEngine/test" + }, + "module": "tests.ordered_receipt_modifiers", + "module_defaults": { + "add_default": { + "items": [ + [ + "defaults", + { + "items": [ + { + "type": "int", + "value": 3 + } + ], + "type": "list" + } + ], + [ + "kwdefaults", + { + "items": [], + "type": "dict" + } + ] + ], + "type": "dict" + } + }, + "module_source_sha256": "d1649b4c79b32c9d0cccbe42f5f5aca301b81ed853ad8bbb972c396d8eae9823", + "qualname": "add_one", + "source_sha256": "a87a9c457524c35b0d3623ca94f2cbe965325463f8195d30df6ea3fae8061fcb" + }, + { + "code_revision": { + "git_sha": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", + "repository": "PolicyEngine/test" + }, + "module": "tests.ordered_receipt_modifiers", + "module_defaults": { + "add_default": { + "items": [ + [ + "defaults", + { + "items": [ + { + "type": "int", + "value": 3 + } + ], + "type": "list" + } + ], + [ + "kwdefaults", + { + "items": [], + "type": "dict" + } + ] + ], + "type": "dict" + } + }, + "module_source_sha256": "d1649b4c79b32c9d0cccbe42f5f5aca301b81ed853ad8bbb972c396d8eae9823", + "qualname": "add_one", + "source_sha256": "a87a9c457524c35b0d3623ca94f2cbe965325463f8195d30df6ea3fae8061fcb" + } + ], + "parameter_changes": {} + }, + "application": "before_data_load_then_after_data_load", + "before": { + "modifier_leaves": [], + "parameter_changes": {} + } + } + }, + { + "id": "typed_defaults", + "identity": { + "binding": "code_revision", + "kind": "reform", + "name": null, + "reform_sha256": "2f091d519022595340e76c34c32efbedae37d9c70ab72073eab7ed0a5791a305", + "representation": "policyengine-uk.phased_scenario_composition/v1" + }, + "plan": { + "adapter_source_sha256": "692a6848be3e366a1243fe50668b152692374c88fd4cc94bd67008b78157519c", + "after": { + "modifier_leaves": [], + "parameter_changes": {} + }, + "application": "before_data_load_then_after_data_load", + "before": { + "modifier_leaves": [ + { + "code_revision": { + "git_sha": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", + "repository": "PolicyEngine/test" + }, + "module": "tests.ordered_receipt_modifiers", + "module_defaults": { + "add_default": { + "items": [ + [ + "defaults", + { + "items": [ + { + "type": "int", + "value": 3 + } + ], + "type": "list" + } + ], + [ + "kwdefaults", + { + "items": [], + "type": "dict" + } + ] + ], + "type": "dict" + } + }, + "module_source_sha256": "d1649b4c79b32c9d0cccbe42f5f5aca301b81ed853ad8bbb972c396d8eae9823", + "qualname": "add_default", + "source_sha256": "010d9132edaa988604384d6b95f16b2ecd87e4d141fb6822a31eddc3033978b9" + } + ], + "parameter_changes": {} + } + } + } + ] +} diff --git a/src/policyengine/provenance/__init__.py b/src/policyengine/provenance/__init__.py index 595d1372..3fc5f688 100644 --- a/src/policyengine/provenance/__init__.py +++ b/src/policyengine/provenance/__init__.py @@ -93,6 +93,30 @@ from .manifest import ( resolve_managed_dataset_reference as resolve_managed_dataset_reference, ) +from .ordered_scenario import ( + BoundOrderedScenario as BoundOrderedScenario, +) +from .ordered_scenario import ( + ModifierCodeRevision as ModifierCodeRevision, +) +from .ordered_scenario import ( + ModifierLeafIdentity as ModifierLeafIdentity, +) +from .ordered_scenario import ( + OrderedScenarioPhase as OrderedScenarioPhase, +) +from .ordered_scenario import ( + OrderedScenarioPlan as OrderedScenarioPlan, +) +from .ordered_scenario import ( + bind_ordered_scenario_plan as bind_ordered_scenario_plan, +) +from .ordered_scenario import ( + build_ordered_scenario as build_ordered_scenario, +) +from .ordered_scenario import ( + ordered_scenario_reform_identity as ordered_scenario_reform_identity, +) from .receipt import ( ExecutionReceipt as ExecutionReceipt, ) diff --git a/src/policyengine/provenance/ordered_scenario.py b/src/policyengine/provenance/ordered_scenario.py new file mode 100644 index 00000000..a3c2cbf4 --- /dev/null +++ b/src/policyengine/provenance/ordered_scenario.py @@ -0,0 +1,702 @@ +"""Ordered native UK execution plans and their inert, portable receipt payloads. + +Only trusted job definitions supply callables. A JSON plan never resolves a +module or qualified name, and cannot grant permission to execute Python. +""" + +from __future__ import annotations + +import ast +import inspect +import math +import sys +from collections.abc import Callable, Mapping, Sequence +from types import CodeType +from typing import Any, Literal + +from pydantic import BaseModel, ConfigDict, Field, PrivateAttr, field_validator + +from policyengine.provenance.bytecode import code_shape +from policyengine.provenance.canonical import canonical_sha256 +from policyengine.provenance.receipt import ( + SHA256_PATTERN, + ReceiptError, + ReformIdentity, + _modifier_identity, + _payload_sha256, + _sha256_text, + _strict_json_value, + _typed_value, + strict_json_value, +) + +ORDERED_SCENARIO_REPRESENTATION = "policyengine-uk.phased_scenario_composition/v1" +ORDERED_SCENARIO_APPLICATION = "before_data_load_then_after_data_load" + + +def _immutable(*args, **kwargs): + raise TypeError("An ordered plan is immutable; build a new plan.") + + +class _FrozenDict(dict): + """JSON-shaped read-only values; low-level Python mutation is not a sandbox.""" + + def __init__(self, value=()): + dict.__init__(self, ((key, _freeze(item)) for key, item in dict(value).items())) + + __setitem__ = __delitem__ = clear = pop = popitem = setdefault = update = _immutable + __ior__ = _immutable + + def __reduce__(self): + return _FrozenDict, (dict(self),) + + +def _freeze(value): + if isinstance(value, Mapping): + return _FrozenDict(value) + if isinstance(value, list | tuple): + return tuple(_freeze(item) for item in value) + return value + + +def _plain(value): + if isinstance(value, Mapping): + return {key: _plain(item) for key, item in value.items()} + if isinstance(value, list | tuple): + return [_plain(item) for item in value] + return value + + +def _same_content(first, second): + """Compare JSON content: distinguish true/1 and equate numeric1/1.0.""" + return canonical_sha256(_plain(first)) == canonical_sha256(_plain(second)) + + +class _PlanModel(BaseModel): + model_config = ConfigDict(extra="forbid", frozen=True, strict=True) + + +class ModifierCodeRevision(_PlanModel): + """The same full repository/commit claim accepted by the Scenario binder.""" + + repository: str = Field(min_length=1) + git_sha: str = Field(pattern=r"^(?:[0-9a-f]{40}|[0-9a-f]{64})$") + + +def _literal(encoded): + """Validate the existing lossless default-argument encoding, without code.""" + if not isinstance(encoded, Mapping): + raise ReceiptError("A typed default must be an object.") + kind = encoded.get("type") + if kind == "none" and set(encoded) == {"type"}: + return None + if kind in ("bool", "int", "str") and set(encoded) == {"type", "value"}: + expected = {"bool": bool, "int": int, "str": str}[kind] + if type(encoded["value"]) is expected: + return encoded["value"] + if kind == "float" and set(encoded) == {"type", "hex"}: + if type(encoded["hex"]) is str: + try: + value = float.fromhex(encoded["hex"]) + except ValueError: + pass + else: + if not math.isnan(value) and value.hex() == encoded["hex"]: + return value + if kind in ("list", "tuple", "dict") and set(encoded) == {"type", "items"}: + items = encoded["items"] + if isinstance(items, tuple | list): + if kind != "dict": + values = [_literal(item) for item in items] + return tuple(values) if kind == "tuple" else values + result = {} + for pair in items: + if not isinstance(pair, tuple | list) or len(pair) != 2: + raise ReceiptError("A typed dictionary needs key/value pairs.") + key, item = pair + if type(key) is not str or key in result: + raise ReceiptError("Typed dictionary keys must be unique strings.") + result[key] = _literal(item) + return result + raise ReceiptError("Invalid typed default encoding.") + + +class ModifierLeafIdentity(_PlanModel): + """Inert provenance for one occurrence of an actual module-level modifier.""" + + module: str = Field(pattern=r"^[A-Za-z_]\w*(?:\.[A-Za-z_]\w*)*$") + qualname: str = Field(pattern=r"^[A-Za-z_]\w*$") + source_sha256: str = Field(pattern=SHA256_PATTERN) + module_source_sha256: str = Field(pattern=SHA256_PATTERN) + module_defaults: dict[str, Any] | None + code_revision: ModifierCodeRevision + + @field_validator("module_defaults") + @classmethod + def _defaults(cls, value): + if value is not None: + if not value: + raise ReceiptError("Empty module defaults must be null.") + for name, encoded in value.items(): + if not name or _typed_value(_literal(encoded)) != encoded: + raise ReceiptError("Invalid module-default proof.") + return _FrozenDict(value) + return value + + +def _parameter_values(changes, *, encoded): + """Closed native scalar/period-map input; infinities have a reserved encoding.""" + if not isinstance(changes, Mapping): + raise ReceiptError("parameter_changes must be an object.") + result = ( + strict_json_value(changes) + if encoded + else _strict_json_value(changes, "$.parameter_changes", nonfinite=True) + ) + + def scalar(value): + if type(value) in (bool, int, float): + return True + return ( + isinstance(value, Mapping) + and set(value) == {"$nonfinite"} + and value["$nonfinite"] in ("Infinity", "-Infinity") + ) + + for path, value in result.items(): + if not path or path == "$nonfinite": + raise ReceiptError("A parameter path must be nonempty and not reserved.") + if scalar(value): + continue + if not isinstance(value, Mapping) or any( + not key or key == "$nonfinite" or not scalar(item) + for key, item in value.items() + ): + raise ReceiptError( + "Native parameter values must be numeric/bool scalars or period maps." + ) + # Enforce RFC8785's numeric domain now, not only when a digest is requested. + canonical_sha256(result) + return result + + +def _native_parameters(changes): + def decode(value): + if isinstance(value, Mapping): + if set(value) == {"$nonfinite"}: + return math.inf if value["$nonfinite"] == "Infinity" else -math.inf + return {key: decode(item) for key, item in value.items()} + return value + + return decode(changes) + + +class OrderedScenarioPhase(_PlanModel): + """The complete parameter map and ordered modifier occurrences of one phase.""" + + parameter_changes: dict[str, Any] + modifier_leaves: tuple[ModifierLeafIdentity, ...] + + @field_validator("parameter_changes") + @classmethod + def _parameters(cls, value): + return _FrozenDict(_parameter_values(value, encoded=True)) + + @field_validator("modifier_leaves", mode="before") + @classmethod + def _leaves(cls, value): + if not isinstance(value, list | tuple): + raise ReceiptError("Modifier leaves must be an ordered list.") + return tuple(value) + + +class OrderedScenarioPlan(_PlanModel): + """An immutable JSON execution recipe; reading one executes no locators.""" + + application: Literal["before_data_load_then_after_data_load"] + adapter_source_sha256: str = Field(pattern=SHA256_PATTERN) + before: OrderedScenarioPhase + after: OrderedScenarioPhase + + @field_validator("after") + @classmethod + def _no_after_parameters(cls, value): + if value.parameter_changes: + raise ReceiptError( + "Ordered plans apply all parameters before data loading, once." + ) + return value + + @classmethod + def json_schema(cls) -> dict[str, Any]: + """Publish the closed wire schema, including native parameter shapes. + + Canonical hexadecimal floats/defaults and RFC8785 numeric-domain + checks remain the explicit semantic rules beside the schema. + """ + schema = cls.model_json_schema() + scalar = { + "anyOf": [ + {"type": "boolean"}, + {"type": "number"}, + { + "type": "object", + "additionalProperties": False, + "required": ["$nonfinite"], + "properties": {"$nonfinite": {"enum": ["Infinity", "-Infinity"]}}, + }, + ] + } + parameters = { + "type": "object", + "propertyNames": {"minLength": 1, "not": {"const": "$nonfinite"}}, + "additionalProperties": { + "anyOf": [ + scalar, + { + "type": "object", + "propertyNames": { + "minLength": 1, + "not": {"const": "$nonfinite"}, + }, + "additionalProperties": scalar, + }, + ] + }, + } + schema["$defs"]["OrderedScenarioPhase"]["properties"]["parameter_changes"] = ( + parameters + ) + schema["properties"]["after"] = { + "allOf": [ + {"$ref": "#/$defs/OrderedScenarioPhase"}, + {"properties": {"parameter_changes": {"const": {}}}}, + ] + } + typed = [] + for kind, field, shape in ( + ("none", None, None), + ("bool", "value", {"type": "boolean"}), + ( + "int", + "value", + {"type": "integer", "minimum": -(2**53 - 1), "maximum": 2**53 - 1}, + ), + ("str", "value", {"type": "string"}), + ( + "float", + "hex", + { + "type": "string", + "pattern": r"^(?:-?inf|-?0x[0-9a-f]+\.[0-9a-f]+p[+-][0-9]+)$", + }, + ), + ( + "list", + "items", + {"type": "array", "items": {"$ref": "#/$defs/TypedDefault"}}, + ), + ( + "tuple", + "items", + {"type": "array", "items": {"$ref": "#/$defs/TypedDefault"}}, + ), + ( + "dict", + "items", + { + "type": "array", + "items": { + "type": "array", + "prefixItems": [ + {"type": "string"}, + {"$ref": "#/$defs/TypedDefault"}, + ], + "minItems": 2, + "maxItems": 2, + }, + }, + ), + ): + properties = {"type": {"const": kind}} + if field: + properties[field] = shape + typed.append( + { + "type": "object", + "additionalProperties": False, + "properties": properties, + "required": list(properties), + } + ) + schema["$defs"]["TypedDefault"] = {"oneOf": typed} + schema["$defs"]["ModifierLeafIdentity"]["properties"]["module_defaults"] = { + "anyOf": [ + {"type": "null"}, + { + "type": "object", + "minProperties": 1, + "propertyNames": {"minLength": 1}, + "additionalProperties": {"$ref": "#/$defs/TypedDefault"}, + }, + ] + } + schema["$id"] = ( + "https://policyengine.org/schemas/ordered_scenario_plan.v1.schema.json" + ) + schema["x-policyengine-rules"] = [ + "Canonicalization is RFC8785 with nulls retained; no locator is imported or executed.", + "Default float hex strings must equal float.fromhex(value).hex(); typed dictionary keys are unique and ordered.", + "Apply existing strict JSON/RFC8785 numeric-domain checks; JSON NaN and numeric Infinity are invalid. Only exact reserved objects encode parameter infinities.", + "Offline provenance is an assertion, not permission to execute. Rebind trusted actual leaf callables and compare the complete plan before runtime.", + ] + return schema + + +def _validated_plan(plan): + if isinstance(plan, OrderedScenarioPlan): + # Revalidate even an instance constructed/copied past Pydantic's rules. + plan = plan.model_dump(mode="json") + try: + return OrderedScenarioPlan.model_validate(plan) + except ValueError as exc: + raise ReceiptError(f"Invalid ordered Scenario plan: {exc}") from exc + + +def ordered_scenario_reform_identity( + plan: OrderedScenarioPlan | Mapping[str, Any], *, name: str | None = None +) -> ReformIdentity: + """Recompose the official identity from closed data, without importing leaves. + + This validates provenance's form, not its truth or approval. A producer or + worker must additionally bind its actual trusted callables with + :func:`bind_ordered_scenario_plan` before execution. + """ + plan = _validated_plan(plan) + if not plan.before.modifier_leaves and not plan.after.modifier_leaves: + changes = _native_parameters(plan.before.parameter_changes) + if not changes: + return ReformIdentity(kind="current_law", name=name) + return ReformIdentity( + kind="reform", + representation="policyengine-uk.scenario/v1", + reform_sha256=_payload_sha256( + { + "representation": "policyengine-uk.scenario/v1", + "payload": { + "applied_before_data_load": True, + "parameter_changes": changes, + "simulation_modifier": None, + }, + } + ), + name=name, + ) + return ReformIdentity( + kind="reform", + representation=ORDERED_SCENARIO_REPRESENTATION, + reform_sha256=canonical_sha256( + { + "representation": ORDERED_SCENARIO_REPRESENTATION, + "payload": plan.model_dump(mode="json"), + } + ), + binding="code_revision", + name=name, + ) + + +def _codes(code): + result = {code.co_qualname: code} + for child in code.co_consts: + if isinstance(child, CodeType): + # Python 3.14 emits annotation helpers beside the callable, with + # no corresponding module/class attribute at this qualified name. + # The existing leaf binder checks modifier annotations separately. + if child.co_name == "__annotate__": + continue + result.update(_codes(child)) + return result + + +def _adapter_digest(): + """Bind the reviewed adapter source and reject changed loaded implementations.""" + module = sys.modules[__name__] + source = inspect.getsource(module) + compiled = _codes(compile(source, __file__, "exec", dont_inherit=True)) + class_bodies = { + node.name for node in ast.parse(source).body if isinstance(node, ast.ClassDef) + } + for name, expected in _ADAPTER_BINDINGS.items(): + if vars(module).get(name) is not expected: + raise ReceiptError(f"Ordered adapter binding {name} changed after import.") + for qualname, expected in compiled.items(): + if qualname == "" or "<" in qualname: + continue + loaded = module + for part in qualname.split("."): + loaded = getattr(loaded, part, None) + if qualname in class_bodies: + if not inspect.isclass(loaded): + raise ReceiptError(f"Ordered adapter class {qualname} changed.") + continue + if isinstance(loaded, property): + loaded = loaded.fget + if inspect.ismethod(loaded): + loaded = loaded.__func__ + if not inspect.isfunction(loaded): + raise ReceiptError(f"Ordered adapter implementation {qualname} changed.") + if code_shape(loaded.__code__) != code_shape(expected): + raise ReceiptError( + f"Ordered adapter code {qualname} no longer matches source." + ) + return _sha256_text(source) + + +def _specs(value): + if isinstance(value, str | bytes | Mapping) or not isinstance(value, Sequence): + raise ReceiptError( + "Modifiers must be an ordered sequence of (callable, revision) pairs." + ) + result = [] + for pair in value: + if not isinstance(pair, tuple | list) or len(pair) != 2: + raise ReceiptError( + "Each modifier needs its actual callable and code revision." + ) + function, revision = pair + try: + checked = ModifierCodeRevision.model_validate(revision) + except ValueError as exc: + raise ReceiptError(f"Invalid modifier code revision: {exc}") from exc + proof = _modifier_identity(function, checked.model_dump()) + result.append((function, revision, ModifierLeafIdentity.model_validate(proof))) + return tuple(result) + + +def _make_plan(parameters, before, after): + early = _specs(before) + late = _specs(after) + plan = OrderedScenarioPlan( + application=ORDERED_SCENARIO_APPLICATION, + adapter_source_sha256=_adapter_digest(), + before=OrderedScenarioPhase( + parameter_changes=_parameter_values(parameters, encoded=False), + modifier_leaves=tuple(item[2] for item in early), + ), + after=OrderedScenarioPhase( + parameter_changes={}, modifier_leaves=tuple(item[2] for item in late) + ), + ) + return plan, early, late + + +class _OrderedModifier: + def __init__(self, owner, phase): + self.owner = owner + self.phase = phase + + def __call__(self, simulation): + self.owner.recheck() + leaves = self.owner._before if self.phase == "before" else self.owner._after + for function, revision, proof in leaves: + function(simulation) + self.owner.recheck() + + +class BoundOrderedScenario(_PlanModel): + """A JSON plan with process-local trusted execution and live rechecks. + + JSON round-trips the plan only. Rebind trusted job leaves after loading + JSON. Pickle transport is supported only for trusted job objects; unpickle + revalidates actual callable sources against the carried plan. + """ + + plan: OrderedScenarioPlan + name: str | None = None + _parameter_source: Any = PrivateAttr(default=None) + _before_source: Any = PrivateAttr(default=None) + _after_source: Any = PrivateAttr(default=None) + _before: Any = PrivateAttr(default=None) + _after: Any = PrivateAttr(default=None) + _scenario: Any = PrivateAttr(default=None) + _adapters: Any = PrivateAttr(default=None) + _native_parts: Any = PrivateAttr(default=None) + + def _validate_live(self): + from policyengine_uk.utils.scenario import Scenario + + if self._scenario is None: + raise ReceiptError( + "A data-only plan cannot execute; rebind trusted modifiers first." + ) + live, before, after = _make_plan( + self._parameter_source, self._before_source, self._after_source + ) + if not _same_content( + live.model_dump(mode="json"), + _validated_plan(self.plan).model_dump(mode="json"), + ): + raise ReceiptError("Ordered Scenario inputs changed after binding.") + for held, rebuilt in ((self._before, before), (self._after, after)): + if len(held) != len(rebuilt) or any( + original[0] is not current[0] + or not _same_content( + original[2].model_dump(mode="json"), + current[2].model_dump(mode="json"), + ) + for original, current in zip(held, rebuilt) + ): + raise ReceiptError("Ordered modifier leaves changed after binding.") + scenario = self._scenario + if ( + type(scenario) is not Scenario + or type(scenario).parts_by_data_load is not self._native_parts + or ("parts_by_data_load" in vars(scenario)) + ): + raise ReceiptError( + "The native Scenario phase adapter changed after binding." + ) + parts = scenario.parts_by_data_load() + for phase, part, proof, adapter in zip( + (True, False), parts, (live.before, live.after), self._adapters + ): + expected = bool(proof.parameter_changes or proof.modifier_leaves) + if not expected: + if part is not None and ( + part.parameter_changes + or part.simulation_modifier + or part.dated_reforms + ): + raise ReceiptError( + "An empty ordered phase acquired executable content." + ) + continue + if ( + part is None + or part.applied_before_data_load is not phase + or part.dated_reforms + ): + raise ReceiptError("The native Scenario timing changed after binding.") + actual = _parameter_values(part.parameter_changes or {}, encoded=False) + if ( + not _same_content(actual, proof.parameter_changes) + or part.simulation_modifier is not adapter + ): + raise ReceiptError( + "The native Scenario parameters/modifier changed after binding." + ) + if adapter is not None and ( + type(adapter) is not _OrderedModifier + or adapter.owner is not self + or adapter.phase != ("before" if phase else "after") + ): + raise ReceiptError( + "The ordered execution adapter changed after binding." + ) + + def recheck(self) -> ReformIdentity: + """Refuse changed plans, original inputs, source/defaults or native phases.""" + self._validate_live() + return ordered_scenario_reform_identity(self.plan, name=self.name) + + @property + def identity(self) -> ReformIdentity: + """An official identity whose receipt-time handle rechecks this execution.""" + identity = self.recheck() + object.__setattr__(identity, "_recompute_identity", self.recheck) + return identity + + @property + def scenario(self) -> Any: + """The actual native UK Scenario, with checked package-owned adapters.""" + self.recheck() + return self._scenario + + def __reduce__(self): + self.recheck() + before = tuple( + (function, revision) for function, revision, proof in self._before + ) + after = tuple((function, revision) for function, revision, proof in self._after) + return _restore_bound, ( + self.plan.model_dump(mode="json"), + before, + after, + self.name, + ) + + +def _restore_bound(plan, before, after, name): + return bind_ordered_scenario_plan( + plan, before_modifiers=before, after_modifiers=after, name=name + ) + + +def build_ordered_scenario( + parameter_changes: Mapping[str, Any] | None = None, + *, + before_modifiers: Sequence[tuple[Callable, Mapping[str, str]]] = (), + after_modifiers: Sequence[tuple[Callable, Mapping[str, str]]] = (), + name: str | None = None, +) -> BoundOrderedScenario: + """Build and bind one ordered native Scenario from trusted actual leaves. + + Pass the already-resolved complete common/reference/member parameter map; + this API applies it once before loading. It never guesses merge/conflict + policy or extracts leaves from arbitrary native modifier-chain closures. + """ + from policyengine_uk.utils.scenario import Scenario + + parameters = {} if parameter_changes is None else parameter_changes + plan, early, late = _make_plan(parameters, before_modifiers, after_modifiers) + bound = BoundOrderedScenario(plan=plan, name=name) + bound._parameter_source = parameters + bound._before_source = before_modifiers + bound._after_source = after_modifiers + bound._before = early + bound._after = late + before = _OrderedModifier(bound, "before") if early else None + after = _OrderedModifier(bound, "after") if late else None + first = Scenario( + parameter_changes=_native_parameters(plan.before.parameter_changes) or None, + simulation_modifier=before, + applied_before_data_load=True, + ) + second = Scenario(simulation_modifier=after) + # Native addition supplies its public, reviewed data-load split. The + # aggregate closure is never inspected or treated as a modifier leaf. + bound._scenario = first + second + bound._adapters = (before, after) + bound._native_parts = Scenario.parts_by_data_load + bound.recheck() + return bound + + +def bind_ordered_scenario_plan( + plan: OrderedScenarioPlan | Mapping[str, Any], + *, + before_modifiers: Sequence[tuple[Callable, Mapping[str, str]]] = (), + after_modifiers: Sequence[tuple[Callable, Mapping[str, str]]] = (), + name: str | None = None, +) -> BoundOrderedScenario: + """Rebind portable plan data to actual trusted job callables; never locators.""" + expected = _validated_plan(plan) + bound = build_ordered_scenario( + _native_parameters(expected.before.parameter_changes), + before_modifiers=before_modifiers, + after_modifiers=after_modifiers, + name=name, + ) + if not _same_content( + bound.plan.model_dump(mode="json"), expected.model_dump(mode="json") + ): + raise ReceiptError( + "Trusted execution leaves or adapter do not match the ordered plan." + ) + return bound + + +_ADAPTER_BINDINGS = { + name: value for name, value in globals().copy().items() if not name.startswith("__") +} diff --git a/src/policyengine/provenance/receipt.py b/src/policyengine/provenance/receipt.py index 559b525d..f8277894 100644 --- a/src/policyengine/provenance/receipt.py +++ b/src/policyengine/provenance/receipt.py @@ -208,6 +208,7 @@ "policyengine-uk.scenario_composition/v1", # A native Scenario whose before/after-data-load parts keep their phases. "policyengine-uk.phased_scenario/v1", + "policyengine-uk.phased_scenario_composition/v1", ] #: Representation of a composition of member reforms. @@ -248,6 +249,7 @@ "reform.reform_sha256": 'RFC 8785 over {representation, payload} where payload is the reform exactly as given in that representation, nulls kept (a null in a reform can mean something, unlike an absent receipt field); for policyengine-uk.scenario_composition/v1 the payload is {applied_before_data_load, application: parameter_changes_then_modifiers, members: [{representation, reform_sha256, binding}, ...]} with members in application order; an infinite number in a payload is the object {"$nonfinite": "Infinity"} or {"$nonfinite": "-Infinity"}, and a payload may not use the key $nonfinite itself', "outputs_sha256": "the canonical output payload the receipt binds", "reform.phased_scenario_sha256": "For policyengine-uk.phased_scenario/v1, RFC 8785 over {representation, payload}, with payload {application: before_data_load_then_after_data_load, before: {representation, reform_sha256, binding}, after: {representation, reform_sha256, binding}}; each phase binds the native Scenario.parts_by_data_load() result through scenario/v1. Common parameter changes occur once, in the before phase. A scenario_composition/v1 cannot contain this representation.", + "reform.ordered_scenario_sha256": "For policyengine-uk.phased_scenario_composition/v1, RFC 8785 over {representation, payload: OrderedScenarioPlan}; retain nulls and every ordered modifier occurrence, including repetitions. Parameters occur only before data load; leaf proofs carry actual source/module/default/revision identities and the plan binds the adapter source. Binding is code_revision. Use ordered_scenario_reform_identity for data-only recomposition. Parameter-only ordered plans retain the existing before-load scenario/v1 identity.", } HashEvidence = Literal["installed_record_verified", "installer_archive_hash"] @@ -523,6 +525,11 @@ def _recheck(self) -> ReformIdentity: @model_validator(mode="after") def _current_law_has_no_digest(self) -> ReformIdentity: + if ( + self.representation == "policyengine-uk.phased_scenario_composition/v1" + and self.binding != "code_revision" + ): + raise ValueError("An ordered modifier plan requires code_revision binding.") if self.kind == "current_law": if self.reform_sha256 is not None or self.representation is not None: raise ValueError("current_law carries no representation or digest.") @@ -749,6 +756,22 @@ def json_schema(cls) -> dict[str, Any]: """JSON Schema (2020-12), with the rules it cannot express attached.""" schema = cls.model_json_schema() + schema["$defs"]["ReformIdentity"]["allOf"] = [ + { + "if": { + "properties": { + "representation": { + "const": "policyengine-uk.phased_scenario_composition/v1" + } + }, + "required": ["representation"], + }, + "then": { + "properties": {"binding": {"const": "code_revision"}}, + "required": ["binding"], + }, + } + ] return { "$schema": "https://json-schema.org/draft/2020-12/schema", "$id": cls.SCHEMA_ID, @@ -976,6 +999,10 @@ def reform_identity( raise ReceiptError( f"A reform payload is a JSON object or array, not {type(payload).__name__}." ) + if representation == "policyengine-uk.phased_scenario_composition/v1": + raise ReceiptError( + "Bind a validated ordered plan with ordered_scenario_reform_identity()." + ) if not payload: return ReformIdentity(kind="current_law", name=name) if representation is None: @@ -3524,7 +3551,10 @@ def checked(identity: ReformIdentity) -> ReformIdentity: ) from exc if member.kind == "current_law": continue - if member.representation == PHASED_SCENARIO_REPRESENTATION: + if member.representation in ( + PHASED_SCENARIO_REPRESENTATION, + "policyengine-uk.phased_scenario_composition/v1", + ): raise ReceiptError( f"Member {index} is a phased Scenario; scenario_composition/v1 " "cannot flatten or override its data-load timing. Bind the " diff --git a/tests/ordered_receipt_modifiers.py b/tests/ordered_receipt_modifiers.py new file mode 100644 index 00000000..d3d3e02b --- /dev/null +++ b/tests/ordered_receipt_modifiers.py @@ -0,0 +1,28 @@ +"""Source-bindable, noncommutative modifiers for ordered execution tests.""" + + +def add_one(simulation): + simulation.ordered_value += 1 + simulation.ordered_trace.append("add") + + +def double(simulation): + simulation.ordered_value *= 2 + simulation.ordered_trace.append("double") + + +def add_default(simulation, amount=3): + simulation.ordered_value += amount + simulation.ordered_trace.append("default") + + +def remember_before(simulation): + simulation.before_value = simulation.ordered_value + + +def bump_limit(simulation): + simulation.max_spiral_loops += 1 + + +def scale_limit(simulation): + simulation.max_spiral_loops *= 2 diff --git a/tests/test_ordered_scenario_receipt.py b/tests/test_ordered_scenario_receipt.py new file mode 100644 index 00000000..a227e6ec --- /dev/null +++ b/tests/test_ordered_scenario_receipt.py @@ -0,0 +1,548 @@ +"""The public ordered plan binds exactly the native phases it executes.""" + +import importlib +import multiprocessing +import pickle +from concurrent.futures import ProcessPoolExecutor +from pathlib import Path +from types import SimpleNamespace + +import jsonschema +import pytest +from hypothesis import given, settings +from hypothesis import strategies as st +from policyengine_uk.utils.scenario import Scenario + +from policyengine.provenance.ordered_scenario import ( + BoundOrderedScenario, + OrderedScenarioPlan, + bind_ordered_scenario_plan, + build_ordered_scenario, + ordered_scenario_reform_identity, +) +from policyengine.provenance.receipt import ( + ExecutionReceipt, + ReceiptError, + ReformIdentity, + composed_reform_identity, + reform_identity, + scenario_reform_identity, +) +from tests import ordered_receipt_modifiers as modifiers + +REVISION = {"repository": "PolicyEngine/test", "git_sha": "a" * 40} + + +def leaf(function): + return function, dict(REVISION) + + +def test_two_leaves_in_each_phase_preserve_noncommutative_order_and_repetition(): + bound = build_ordered_scenario( + before_modifiers=[leaf(modifiers.add_one), leaf(modifiers.double)], + after_modifiers=[leaf(modifiers.double), leaf(modifiers.add_one)], + ) + before, after = bound.scenario.parts_by_data_load() + simulation = SimpleNamespace(ordered_value=1, ordered_trace=[]) + before.simulation_modifier(simulation) + assert simulation.ordered_value == 4 + simulation.ordered_trace.append("load") + after.simulation_modifier(simulation) + assert simulation.ordered_value == 9 + assert simulation.ordered_trace == ["add", "double", "load", "double", "add"] + assert ordered_scenario_reform_identity(bound.plan) == bound.identity + + +def test_parameters_occur_once_before_data_load(): + bound = build_ordered_scenario( + {"gov.example.rate": {"2027": 0.2}}, + after_modifiers=[leaf(modifiers.add_one)], + ) + before, after = bound.scenario.parts_by_data_load() + assert before.parameter_changes == {"gov.example.rate": {"2027": 0.2}} + assert not after.parameter_changes + + +def test_original_parameter_mutation_is_refused(): + parameters = {"gov.example.rate": {"2027": 0.2}} + bound = build_ordered_scenario(parameters) + identity = bound.identity + parameters["gov.example.rate"]["2027"] = 0.3 + with pytest.raises(ValueError, match="changed after"): + identity._recheck() + + +def execute_bound(bound): + """A trusted worker definition; callable locators never come from plan data.""" + before, after = bound.scenario.parts_by_data_load() + simulation = SimpleNamespace(ordered_value=1, ordered_trace=[]) + if before and before.simulation_modifier: + before.simulation_modifier(simulation) + simulation.ordered_trace.append("load") + if after and after.simulation_modifier: + after.simulation_modifier(simulation) + return ( + simulation.ordered_value, + simulation.ordered_trace, + bound.identity.model_dump(), + ) + + +@settings(max_examples=8, deadline=None) +@given( + before=st.lists(st.booleans(), max_size=4), + after=st.lists(st.booleans(), max_size=4), +) +def test_arbitrary_order_and_repetition_match_the_executed_plan(before, after): + functions = {True: modifiers.add_one, False: modifiers.double} + bound = build_ordered_scenario( + before_modifiers=[leaf(functions[x]) for x in before], + after_modifiers=[leaf(functions[x]) for x in after], + ) + value = 1 + for operation in before + after: + value = value + 1 if operation else value * 2 + assert execute_bound(bound)[0] == value + assert len(bound.plan.before.modifier_leaves) == len(before) + assert len(bound.plan.after.modifier_leaves) == len(after) + + +def test_order_and_phase_change_both_identity_and_actual_effect(): + first = build_ordered_scenario( + before_modifiers=[leaf(modifiers.add_one), leaf(modifiers.double)] + ) + reverse = build_ordered_scenario( + before_modifiers=[leaf(modifiers.double), leaf(modifiers.add_one)] + ) + shifted = build_ordered_scenario( + after_modifiers=[leaf(modifiers.add_one), leaf(modifiers.double)] + ) + assert execute_bound(first)[0] == 4 + assert execute_bound(reverse)[0] == 3 + assert execute_bound(first)[1] != execute_bound(shifted)[1] + assert len({x.identity.reform_sha256 for x in (first, reverse, shifted)}) == 3 + + +def test_native_simulation_runs_the_same_two_before_and_two_after_leaves(monkeypatch): + from policyengine_uk import Simulation + + observed = [] + original = Simulation.build_from_situation + + def record_load(simulation, situation): + observed.append(simulation.max_spiral_loops) + return original(simulation, situation) + + monkeypatch.setattr(Simulation, "build_from_situation", record_load) + bound = build_ordered_scenario( + before_modifiers=[leaf(modifiers.bump_limit), leaf(modifiers.scale_limit)], + after_modifiers=[leaf(modifiers.scale_limit), leaf(modifiers.bump_limit)], + ) + simulation = Simulation( + scenario=bound.scenario, + situation={ + "people": {"p": {"age": {2027: 40}}}, + "benunits": {"b": {"members": ["p"]}}, + "households": {"h": {"members": ["p"]}}, + }, + ) + assert observed == [22, 10] # Reform loads after early leaves; baseline is native. + assert simulation.max_spiral_loops == 45 + assert bound.identity._recheck() == bound.identity + + +@pytest.mark.parametrize( + "changes", + [ + {}, + {"gov.example.rate": {"2027": 0.2}}, + {"gov.example.cap": {"2027": float("inf")}}, + ], +) +def test_parameter_only_plans_keep_the_exact_existing_scenario_digest(changes): + bound = build_ordered_scenario(changes) + expected = scenario_reform_identity( + Scenario(parameter_changes=changes, applied_before_data_load=True) + ) + assert bound.identity == expected + loaded = OrderedScenarioPlan.model_validate_json(bound.plan.model_dump_json()) + assert ordered_scenario_reform_identity(loaded) == expected + + +def test_plan_is_deeply_immutable_and_json_round_trips(): + bound = build_ordered_scenario( + {"gov.example.rate": {"2027": 0.2}}, + before_modifiers=[leaf(modifiers.add_default)], + ) + with pytest.raises(TypeError): + bound.plan.before.parameter_changes["gov.example.rate"]["2027"] = 0.3 + with pytest.raises(ValueError): + bound.plan.before.modifier_leaves = () + encoded = bound.model_dump_json() + loaded = BoundOrderedScenario.model_validate_json(encoded) + assert loaded.model_dump_json() == encoded + with pytest.raises(ReceiptError, match="data-only"): + loaded.scenario + rebound = bind_ordered_scenario_plan( + loaded.plan, before_modifiers=[leaf(modifiers.add_default)] + ) + assert rebound.identity == bound.identity + + +@pytest.mark.parametrize("target", ["before", "after"]) +def test_original_leaf_list_and_revision_mutations_fail_live_recheck(target): + leaves = [leaf(modifiers.add_one)] + bound = build_ordered_scenario(**{target + "_modifiers": leaves}) + identity = bound.identity + leaves.append(leaf(modifiers.double)) + with pytest.raises(ReceiptError, match="changed after"): + identity._recheck() + leaves.pop() + leaves[0][1]["git_sha"] = "b" * 40 + with pytest.raises(ReceiptError, match="changed after"): + identity._recheck() + + +@pytest.mark.parametrize("target", ["phase", "parameters", "modifier", "adapter"]) +def test_mutated_actual_native_execution_is_refused(target): + bound = build_ordered_scenario( + {"gov.example.rate": {"2027": 0.2}}, + before_modifiers=[leaf(modifiers.add_one)], + after_modifiers=[leaf(modifiers.double)], + ) + identity = bound.identity + native = bound.scenario + if target == "phase": + native.applied_before_data_load = True + elif target == "parameters": + native.parameter_changes["gov.example.rate"]["2027"] = 0.3 + elif target == "modifier": + native.simulation_modifier = modifiers.add_one + else: + bound._adapters[0].phase = "after" + with pytest.raises(ReceiptError): + identity._recheck() + + +def test_modified_leaf_defaults_code_and_module_state_are_refused(monkeypatch): + bound = build_ordered_scenario(before_modifiers=[leaf(modifiers.add_default)]) + identity = bound.identity + with monkeypatch.context() as patch: + patch.setattr(modifiers.add_default, "__defaults__", (4,)) + with pytest.raises(ReceiptError): + identity._recheck() + with monkeypatch.context() as patch: + patch.setattr(modifiers.add_one, "__code__", modifiers.double.__code__) + with pytest.raises(ReceiptError): + identity._recheck() + with monkeypatch.context() as patch: + patch.setattr(modifiers, "unexpected_state", 3, raising=False) + with pytest.raises(ReceiptError): + identity._recheck() + + +def test_adapter_implementation_replacement_is_refused(monkeypatch): + from policyengine.provenance import ordered_scenario as api + + bound = build_ordered_scenario(before_modifiers=[leaf(modifiers.add_one)]) + identity = bound.identity + monkeypatch.setattr(api._OrderedModifier, "__call__", lambda self, simulation: None) + with pytest.raises(ReceiptError, match="adapter"): + identity._recheck() + + +def test_pickle_and_real_spawn_worker_rebind_the_actual_trusted_leaves(): + bound = build_ordered_scenario( + before_modifiers=[leaf(modifiers.add_one), leaf(modifiers.double)], + after_modifiers=[leaf(modifiers.double), leaf(modifiers.add_one)], + ) + restored = pickle.loads(pickle.dumps(bound)) + assert execute_bound(restored) == execute_bound(bound) + with ProcessPoolExecutor( + max_workers=1, mp_context=multiprocessing.get_context("spawn") + ) as pool: + assert pool.submit(execute_bound, bound).result(timeout=60) == execute_bound( + bound + ) + + +def test_tampered_portable_proofs_cannot_rebind_to_trusted_jobs(): + bound = build_ordered_scenario(before_modifiers=[leaf(modifiers.add_one)]) + portable = bound.plan.model_dump(mode="json") + portable["before"]["modifier_leaves"][0]["source_sha256"] = "b" * 64 + assert ordered_scenario_reform_identity(portable) != bound.identity + with pytest.raises(ReceiptError, match="do not match"): + bind_ordered_scenario_plan(portable, before_modifiers=[leaf(modifiers.add_one)]) + + +def test_data_only_recomposer_never_imports_receipt_locators(monkeypatch): + bound = build_ordered_scenario(before_modifiers=[leaf(modifiers.add_one)]) + portable = bound.plan.model_dump(mode="json") + portable["before"]["modifier_leaves"][0]["module"] = ( + "nonexistent_receipt_locator_never_import" + ) + + def refuse_import(*args, **kwargs): + raise AssertionError("Receipt locators must remain data") + + monkeypatch.setattr(importlib, "import_module", refuse_import) + assert ordered_scenario_reform_identity(portable).binding == "code_revision" + + +@pytest.mark.parametrize( + "mutation", ["extra", "phase", "after_parameters", "leaf_extra", "defaults"] +) +def test_unknown_or_unsupported_plan_fields_fail_closed(mutation): + portable = build_ordered_scenario( + before_modifiers=[leaf(modifiers.add_one)] + ).plan.model_dump(mode="json") + if mutation == "extra": + portable["invented"] = True + elif mutation == "phase": + portable["application"] = "after_then_before" + elif mutation == "after_parameters": + portable["after"]["parameter_changes"] = {"gov.example.rate": 1} + elif mutation == "leaf_extra": + portable["before"]["modifier_leaves"][0]["leaf_count"] = 1 + else: + portable["before"]["modifier_leaves"][0]["module_defaults"] = { + "x": {"type": "object"} + } + with pytest.raises(ValueError): + ordered_scenario_reform_identity(portable) + + +def test_arbitrary_closures_dated_reforms_and_old_composition_remain_refused(): + with pytest.raises(ReceiptError): + build_ordered_scenario(before_modifiers=[leaf(lambda simulation: None)]) + dated = Scenario.from_reform({"gov.example.rate": {"2027": 0.2}}) + with pytest.raises(ReceiptError): + build_ordered_scenario(before_modifiers=[leaf(dated.simulation_modifier)]) + bound = build_ordered_scenario(before_modifiers=[leaf(modifiers.add_one)]) + with pytest.raises(ReceiptError, match="cannot flatten or override"): + composed_reform_identity([bound.identity], applied_before_data_load=True) + with pytest.raises(ReceiptError, match="ordered_scenario_reform_identity"): + reform_identity( + bound.plan.model_dump(mode="json"), bound.identity.representation + ) + + +def test_execution_receipt_schema_accepts_new_identity(): + identity = build_ordered_scenario( + before_modifiers=[leaf(modifiers.add_one)] + ).identity + schema = ExecutionReceipt.json_schema() + jsonschema.validate( + identity.model_dump(mode="json"), + {"$defs": schema["$defs"], "$ref": "#/$defs/ReformIdentity"}, + ) + + +def test_published_schema_and_vectors_reproduce_the_official_api(): + import json + + from scripts.generate_ordered_scenario_vectors import build_vectors + + directory = Path(__file__).resolve().parents[1] / "src/policyengine/data/schemas" + schema = json.loads( + (directory / "ordered_scenario_plan.v1.schema.json").read_text() + ) + assert schema == OrderedScenarioPlan.json_schema() + vectors = json.loads( + (directory / "ordered_scenario_plan.v1.vectors.json").read_text() + ) + assert vectors == build_vectors() + for vector in vectors["valid"]: + jsonschema.validate(vector["plan"], schema) + identity = ordered_scenario_reform_identity(vector["plan"]) + assert identity.model_dump(mode="json") == vector["identity"] + for vector in vectors["invalid"]: + with pytest.raises(jsonschema.ValidationError): + jsonschema.validate(vector["plan"], schema) + with pytest.raises(ValueError): + ordered_scenario_reform_identity(vector["plan"]) + + +def test_worker_rebinding_refuses_source_defaults_changed_after_planning(monkeypatch): + bound = build_ordered_scenario(before_modifiers=[leaf(modifiers.add_default)]) + packet = pickle.dumps(bound) + monkeypatch.setattr(modifiers.add_default, "__defaults__", (9,)) + with pytest.raises(ReceiptError): + pickle.loads(packet) + + +def test_leaf_source_file_edits_after_planning_fail_recheck(tmp_path): + import importlib.util + import sys + + path = tmp_path / "ordered_temporary_leaf.py" + path.write_text("def change(simulation):\n simulation.ordered_value += 1\n") + spec = importlib.util.spec_from_file_location("ordered_temporary_leaf", path) + module = importlib.util.module_from_spec(spec) + sys.modules[module.__name__] = module + try: + spec.loader.exec_module(module) + bound = build_ordered_scenario(before_modifiers=[leaf(module.change)]) + identity = bound.identity + path.write_text(path.read_text() + "\n# Changed source bytes.\n") + with pytest.raises(ReceiptError): + identity._recheck() + finally: + sys.modules.pop(module.__name__, None) + + +def test_bypassing_frozen_plan_fields_cannot_rewrite_a_bound_execution(): + bound = build_ordered_scenario(before_modifiers=[leaf(modifiers.add_one)]) + identity = bound.identity + object.__setattr__(bound.plan.before, "modifier_leaves", ()) + with pytest.raises(ReceiptError): + identity._recheck() + + +def test_ordered_modifier_identity_cannot_downgrade_to_content_binding(): + identity = build_ordered_scenario(before_modifiers=[leaf(modifiers.add_one)]) + payload = identity.identity.model_dump(mode="json") + payload["binding"] = "content" + with pytest.raises(ValueError, match="code_revision"): + ReformIdentity.model_validate(payload) + schema = ExecutionReceipt.json_schema() + with pytest.raises(jsonschema.ValidationError): + jsonschema.validate( + payload, {"$defs": schema["$defs"], "$ref": "#/$defs/ReformIdentity"} + ) + + +def test_a_lookalike_callable_cannot_replace_the_package_phase_adapter(): + bound = build_ordered_scenario( + before_modifiers=[leaf(modifiers.add_one)], + after_modifiers=[leaf(modifiers.double)], + ) + native = bound.scenario + identity = bound.identity + + class Lookalike: + owner = bound + phase = "before" + + def __call__(self, simulation): + raise AssertionError("A substituted adapter must never execute") + + fake = Lookalike() + bound._adapters = (fake, bound._adapters[1]) + native._timed_modifiers = (fake, native._timed_modifiers[1]) + with pytest.raises(ReceiptError, match="adapter changed"): + identity._recheck() + + +def test_compiler_annotation_helpers_do_not_claim_native_runtime_callables(): + from policyengine.provenance.ordered_scenario import _codes + + code = compile( + "from __future__ import annotations\n" + "class Example:\n" + " value: int\n" + " def method(self, other: int) -> str: return str(other)\n", + "synthetic_annotations.py", + "exec", + dont_inherit=True, + ) + found = _codes(code) + assert "Example.method" in found + assert not any(name.rsplit(".", 1)[-1] == "__annotate__" for name in found) + + +def test_callable_class_cannot_replace_a_source_declared_adapter_method(monkeypatch): + """C1: callable classes must not bypass the method's source-code check.""" + from policyengine.provenance import ordered_scenario as api + + bound = build_ordered_scenario( + before_modifiers=[leaf(modifiers.add_one), leaf(modifiers.double)], + ) + identity = bound.identity + assert execute_bound(bound)[0] == 4 + + class SkipAdapterExecution: + def __init__(self, simulation): + self.skipped = True + + monkeypatch.setattr(api._OrderedModifier, "__call__", SkipAdapterExecution) + with pytest.raises(ReceiptError, match="adapter implementation"): + identity._recheck() + with pytest.raises(ReceiptError, match="adapter implementation"): + bound.scenario + + +def test_callable_class_replacement_refuses_the_real_native_run(monkeypatch): + """C1: refuse the replacement that changed native execution from 45 to 10.""" + from policyengine_uk import Simulation + + from policyengine.provenance import ordered_scenario as api + + bound = build_ordered_scenario( + before_modifiers=[leaf(modifiers.bump_limit), leaf(modifiers.scale_limit)], + after_modifiers=[leaf(modifiers.scale_limit), leaf(modifiers.bump_limit)], + ) + identity = bound.identity + situation = { + "people": {"p": {"age": {2027: 40}}}, + "benunits": {"b": {"members": ["p"]}}, + "households": {"h": {"members": ["p"]}}, + } + assert ( + Simulation(situation=situation, scenario=bound.scenario).max_spiral_loops == 45 + ) + + class SkipAdapterExecution: + def __init__(self, simulation): + self.skipped = True + + monkeypatch.setattr(api._OrderedModifier, "__call__", SkipAdapterExecution) + with pytest.raises(ReceiptError, match="adapter implementation"): + Simulation(situation=situation, scenario=bound.scenario) + with pytest.raises(ReceiptError, match="adapter implementation"): + identity._recheck() + + +@pytest.mark.parametrize( + "initial, replacement", [(True, 1), (1, True), (False, 0), (0, False)] +) +@pytest.mark.parametrize("ordered", [False, True]) +@pytest.mark.parametrize("target", ["original", "native"]) +def test_boolean_numeric_parameter_mutations_refuse_stale_identity( + initial, replacement, ordered, target +): + """C2: canonical JSON distinguishes true/1 and false/0 in either direction.""" + parameters = {"gov.example.flag": initial} + leaves = [leaf(modifiers.add_one)] if ordered else [] + bound = build_ordered_scenario(parameters, before_modifiers=leaves) + identity = bound.identity + native = bound.scenario + fresh = build_ordered_scenario( + {"gov.example.flag": replacement}, before_modifiers=leaves + ) + assert fresh.identity.reform_sha256 != identity.reform_sha256 + if target == "original": + parameters["gov.example.flag"] = replacement + else: + native.parameter_changes["gov.example.flag"] = replacement + with pytest.raises(ReceiptError, match="changed after"): + identity._recheck() + + +@pytest.mark.parametrize("ordered", [False, True]) +@pytest.mark.parametrize("target", ["original", "native"]) +def test_equal_integer_float_parameters_keep_official_canonical_identity( + ordered, target +): + """C2 control: official RFC 8785 numeric 1 and 1.0 have the same content.""" + parameters = {"gov.example.number": 1} + bound = build_ordered_scenario( + parameters, before_modifiers=[leaf(modifiers.add_one)] if ordered else [] + ) + identity = bound.identity + native = bound.scenario + if target == "original": + parameters["gov.example.number"] = 1.0 + else: + native.parameter_changes["gov.example.number"] = 1.0 + assert identity._recheck() == identity