Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions changelog.d/ordered-scenario-plans.added.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Added ordered UK Scenario plans that preserve modifier order and data-load phases, bind trusted callable sources, and support offline identity verification and worker rebinding.
40 changes: 40 additions & 0 deletions docs/ordered-scenarios.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
# Ordered UK scenarios

Use `build_ordered_scenario` to preserve several module-level modifiers before and after data loading. Pass actual callables from trusted job code, with a full repository and commit claim for each occurrence. The builder returns `BoundOrderedScenario`, which exposes a native UK `Scenario`, an immutable execution plan and a live receipt identity.

```python
from policyengine.provenance import build_ordered_scenario

bound = build_ordered_scenario(
complete_parameter_changes,
before_modifiers=[
(first_modifier, first_revision),
(second_modifier, second_revision),
],
after_modifiers=[(third_modifier, third_revision)],
)
simulation = NativeUKSimulation(situation=situation, scenario=bound.scenario)
reform_identity = bound.identity
```

Each revision uses `{"repository": "owner/repository", "git_sha": "<full commit SHA>"}`. The caller supplies the functions, revision claims and `NativeUKSimulation` from its trusted application. See [the runnable example](../examples/ordered_scenario.py) for a synthetic phase demonstration.

## Execution and identity

The adapter applies the complete parameter map once before loading data. It then calls every before-load modifier in order, lets the native simulation build its data, and calls every after-load modifier in order. Repeated occurrences execute repeatedly. Resolve a shared policy reference and each member's raw changes into the complete map before calling the builder; keep the application's existing member-conflict checks.

`BoundOrderedScenario.plan` stores the application order, adapter source digest, complete before-load parameters and ordered leaf proofs for both phases. Each proof records the callable's module, qualified name, source digest, module-source digest, typed module defaults and code revision. The after-load phase accepts no parameter map. The adapter and receipt identity check the same plan, and `recheck()` refuses changes to original parameters, leaf lists, revisions, defaults, loaded code or native phase fields.

Plans with modifiers use `policyengine-uk.phased_scenario_composition/v1`. Their digests cover `{"representation": ..., "payload": plan}` with RFC 8785 canonicalization and nulls retained. Parameter-only plans keep the existing before-load `scenario/v1` identity; an empty plan keeps `current_law`. Existing Scenario, phased Scenario and legacy composition digests retain their meaning. Legacy composition cannot flatten an ordered plan's phases.

## Offline verification and workers

`ordered_scenario_reform_identity(plan)` validates the closed plan and recomputes its official identity from data. It never imports, fetches or executes a module or qualified name from the payload. A source proof remains a producer's claim; applications must check approvals and provenance separately.

Workers call `bind_ordered_scenario_plan(plan, before_modifiers=..., after_modifiers=...)` with their actual trusted job callables. The binder checks each current source/default/revision proof against the complete carried plan before exposing the native scenario. Trusted `BoundOrderedScenario` objects also support pickle transport and rebind on load. Never unpickle untrusted input. JSON round-trips carry plan data; callers must rebind trusted functions before execution.

Receipt builders recheck live identities after execution. Keep the bound object that supplied `scenario`, and pass its live `identity` to the receipt builder. Loading an identity from JSON carries no live execution handle.

The [plan schema](../src/policyengine/data/schemas/ordered_scenario_plan.v1.schema.json) and [API-generated vectors](../src/policyengine/data/schemas/ordered_scenario_plan.v1.vectors.json) support offline implementations. Apply the schema's semantic rules as well as structural validation: preserve ordered typed dictionaries and repetitions, require canonical float hex values, and enforce the existing strict JSON numeric domain. Only the exact reserved `{"$nonfinite": "Infinity"}` and `{"$nonfinite": "-Infinity"}` objects encode infinite parameter values.

The API rejects arbitrary closures, dynamic modifier chains, methods, partials and dated reform objects. It does not inspect arbitrary closure cells, prove Python write sets, approve jobs or certify data. Callers retain source-based structural consistency checks and data limits.
4 changes: 3 additions & 1 deletion docs/receipts.md
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,8 @@ None of these depends on where the code is checked out or which Python version r

The old `scenario_reform_identity` refuses mixed phases and effective-phase/aggregate-flag mismatches instead of assigning the aggregate flag to every part. `composed_reform_identity` v1 refuses phased members because its contract overrides member timing. Bind the actual combined Scenario through the phased API instead. Dated reform objects are outside both Scenario representations: bind an original reform dictionary as its dictionary representation rather than silently omitting it. Closures and computed modifier chains remain unsupported.

For several modifiers in either phase, use the [ordered Scenario API](ordered-scenarios.md). It binds actual trusted callable leaves, preserves their order and repetitions, and provides an offline plan recomposer.

Build the receipt after the run. An identity returned by `scenario_reform_identity` keeps a private handle to the same scenario and code revision. Both receipt builders recompute `reform` and `baseline` identities with handles, and refuse if recomputing is refused or an identity member differs. The handle is excluded from equality, serialization, schemas and digests. A serialized identity loaded back has no handle and is not re-checked; its producer must ensure it describes the run. These checks take snapshots at identity time and receipt build time, so they cannot describe later changes.

**The modifier's module.** A digest can only name what the module's source fixes, so the module holding a bound modifier must have a shape whose state that source determines. At module level it may hold only:
Expand Down Expand Up @@ -190,7 +192,7 @@ The pre-promotion replay interface with policyengine-scorecard follows a differe
| `reform.reform_sha256` | `{"representation": ..., "payload": ...}`, with the payload exactly as given, nulls kept (strict JSON conversion with infinities as `{"$nonfinite": "Infinity"}` or `"-Infinity"`, then RFC 8785, with no null removal). For a composition the payload is `{"applied_before_data_load": ..., "application": "parameter_changes_then_modifiers", "members": [{"representation", "reform_sha256", "binding"}, ...]}`, with members in application order. |
| `outputs_sha256` | The output payload. |

Reform digests are never normalized across representations. The same keys can legitimately mean different things in different representations: for example, a bare year in a policyengine-uk country dictionary changes that one year only. So each digest is tied to the representation that was submitted. The v1 representations are `policyengine.parameter_values/v1`, `policyengine.reform_dict/v1`, `policyengine-uk.reform_dict/v1`, `policyengine-uk.scenario/v1`, `policyengine-uk.scenario_composition/v1` and `policyengine-sim-api.policy/v1`.
Reform digests are never normalized across representations. The same keys can legitimately mean different things in different representations: for example, a bare year in a policyengine-uk country dictionary changes that one year only. So each digest is tied to the representation that was submitted. Receipts support `policyengine.parameter_values/v1`, `policyengine.reform_dict/v1`, `policyengine-uk.reform_dict/v1`, `policyengine-uk.scenario/v1`, `policyengine-uk.scenario_composition/v1`, `policyengine-uk.phased_scenario/v1`, `policyengine-uk.phased_scenario_composition/v1` and `policyengine-sim-api.policy/v1`.

## Rules JSON Schema cannot express

Expand Down
36 changes: 36 additions & 0 deletions examples/ordered_scenario.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
"""Demonstrate ordered phases with synthetic state and no household data.

Run from a checkout containing these example functions. Supply that actual
checkout's repository and full commit SHA; the API records the caller's claim.
"""

import argparse
from types import SimpleNamespace

from ordered_scenario_modifiers import add_one, double

from policyengine.provenance import build_ordered_scenario


def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--repository", required=True)
parser.add_argument("--git-sha", required=True)
args = parser.parse_args()
revision = {"repository": args.repository, "git_sha": args.git_sha}
bound = build_ordered_scenario(
before_modifiers=[(add_one, revision), (double, revision)],
after_modifiers=[(double, revision), (add_one, revision)],
)
before, after = bound.scenario.parts_by_data_load()
state = SimpleNamespace(value=1)
before.simulation_modifier(state)
assert state.value == 4
# A native simulation constructs its data between these two calls.
after.simulation_modifier(state)
assert state.value == 9
print(bound.identity.model_dump_json(indent=2))


if __name__ == "__main__":
main()
9 changes: 9 additions & 0 deletions examples/ordered_scenario_modifiers.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
"""Synthetic arithmetic modifiers for the ordered-phase API example."""


def add_one(simulation):
simulation.value += 1


def double(simulation):
simulation.value *= 2
118 changes: 118 additions & 0 deletions scripts/generate_ordered_scenario_vectors.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,118 @@
"""Generate data-only verifier vectors from the actual ordered-plan producer.

The modifiers and revision below are synthetic test fixtures. These vectors
assert canonicalization behavior, not approval or certification of a job.
"""

from __future__ import annotations

import argparse
import copy
import json
import os
import sys
from pathlib import Path

ROOT = Path(__file__).resolve().parents[1]
os.environ.setdefault("POLICYENGINE_SKIP_COUNTRY_IMPORTS", "1")
sys.path.insert(0, str(ROOT / "src"))
sys.path.insert(0, str(ROOT))

from policyengine.provenance.ordered_scenario import ( # noqa: E402
build_ordered_scenario,
)
from tests import ordered_receipt_modifiers as modifiers # noqa: E402

PATH = ROOT / "src/policyengine/data/schemas/ordered_scenario_plan.v1.vectors.json"
REVISION = {"repository": "PolicyEngine/test", "git_sha": "a" * 40}


def build_vectors():
def leaves(*functions):
return [(function, dict(REVISION)) for function in functions]

cases = [
("empty", build_ordered_scenario()),
("parameter_only", build_ordered_scenario({"gov.example.rate": {"2027": 0.2}})),
(
"parameter_infinity",
build_ordered_scenario({"gov.example.cap": {"2027": float("inf")}}),
),
(
"two_each_phase",
build_ordered_scenario(
{"gov.example.rate": {"2027": 0.2}},
before_modifiers=leaves(modifiers.add_one, modifiers.double),
after_modifiers=leaves(modifiers.double, modifiers.add_one),
),
),
(
"reverse_before",
build_ordered_scenario(
before_modifiers=leaves(modifiers.double, modifiers.add_one)
),
),
(
"repeated_leaf",
build_ordered_scenario(
after_modifiers=leaves(modifiers.add_one, modifiers.add_one)
),
),
(
"typed_defaults",
build_ordered_scenario(before_modifiers=leaves(modifiers.add_default)),
),
]
valid = [
{
"id": name,
"plan": bound.plan.model_dump(mode="json"),
"identity": bound.identity.model_dump(mode="json"),
}
for name, bound in cases
]
base = valid[3]["plan"]
invalid = []
for name in (
"unknown_field",
"unknown_application",
"after_parameters",
"unknown_leaf_field",
"unknown_default_type",
):
plan = copy.deepcopy(base)
if name == "unknown_field":
plan["invented"] = True
elif name == "unknown_application":
plan["application"] = "after_then_before"
elif name == "after_parameters":
plan["after"]["parameter_changes"] = {"gov.example.rate": 1}
elif name == "unknown_leaf_field":
plan["before"]["modifier_leaves"][0]["claimed_leaf_count"] = 1
else:
plan["before"]["modifier_leaves"][0]["module_defaults"] = {
"x": {"type": "object"}
}
invalid.append({"id": name, "plan": plan})
return {
"schema_version": 1,
"purpose": "Synthetic canonicalization vectors; no job approval or certification.",
"valid": valid,
"invalid": invalid,
}


def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--check", action="store_true")
args = parser.parse_args()
content = json.dumps(build_vectors(), indent=2, sort_keys=True) + "\n"
if args.check:
if not PATH.exists() or PATH.read_text() != content:
raise SystemExit("Ordered Scenario vectors are stale.")
else:
PATH.write_text(content)


if __name__ == "__main__":
main()
2 changes: 2 additions & 0 deletions scripts/generate_receipt_schemas.py
Original file line number Diff line number Diff line change
Expand Up @@ -24,10 +24,12 @@
os.environ.setdefault("POLICYENGINE_SKIP_COUNTRY_IMPORTS", "1")
sys.path.insert(0, str(REPO_ROOT / "src"))

from policyengine.provenance.ordered_scenario import OrderedScenarioPlan # noqa: E402
from policyengine.provenance.receipt import ExecutionReceipt # noqa: E402

SCHEMAS = {
"execution_receipt.v1.schema.json": ExecutionReceipt.json_schema,
"ordered_scenario_plan.v1.schema.json": OrderedScenarioPlan.json_schema,
}


Expand Down
28 changes: 27 additions & 1 deletion src/policyengine/data/schemas/execution_receipt.v1.schema.json
Original file line number Diff line number Diff line change
Expand Up @@ -382,6 +382,30 @@
},
"ReformIdentity": {
"additionalProperties": false,
"allOf": [
{
"if": {
"properties": {
"representation": {
"const": "policyengine-uk.phased_scenario_composition/v1"
}
},
"required": [
"representation"
]
},
"then": {
"properties": {
"binding": {
"const": "code_revision"
}
},
"required": [
"binding"
]
}
}
],
"description": "A reform or baseline, bound by the digest of its canonical form.",
"properties": {
"binding": {
Expand Down Expand Up @@ -440,7 +464,8 @@
"policyengine-sim-api.policy/v1",
"policyengine-uk.scenario/v1",
"policyengine-uk.scenario_composition/v1",
"policyengine-uk.phased_scenario/v1"
"policyengine-uk.phased_scenario/v1",
"policyengine-uk.phased_scenario_composition/v1"
],
"type": "string"
},
Expand Down Expand Up @@ -832,6 +857,7 @@
"execution_identity_sha256": "{schema_version, country_id, wrapper, model, core, data, certification_status, certificate?, regional_certification?} where certificate keeps only bundle_version, country_id, certificate_sha256, compatibility_basis, certified_for_model_version, certified_for_core_version, certified_dataset_sha256 and certified_by",
"outputs_sha256": "the canonical output payload the receipt binds",
"receipt_sha256": "the whole receipt without runtime and without reform.name and baseline.name",
"reform.ordered_scenario_sha256": "For policyengine-uk.phased_scenario_composition/v1, RFC 8785 over {representation, payload: OrderedScenarioPlan}; retain nulls and every ordered modifier occurrence, including repetitions. Parameters occur only before data load; leaf proofs carry actual source/module/default/revision identities and the plan binds the adapter source. Binding is code_revision. Use ordered_scenario_reform_identity for data-only recomposition. Parameter-only ordered plans retain the existing before-load scenario/v1 identity.",
"reform.phased_scenario_sha256": "For policyengine-uk.phased_scenario/v1, RFC 8785 over {representation, payload}, with payload {application: before_data_load_then_after_data_load, before: {representation, reform_sha256, binding}, after: {representation, reform_sha256, binding}}; each phase binds the native Scenario.parts_by_data_load() result through scenario/v1. Common parameter changes occur once, in the before phase. A scenario_composition/v1 cannot contain this representation.",
"reform.reform_sha256": "RFC 8785 over {representation, payload} where payload is the reform exactly as given in that representation, nulls kept (a null in a reform can mean something, unlike an absent receipt field); for policyengine-uk.scenario_composition/v1 the payload is {applied_before_data_load, application: parameter_changes_then_modifiers, members: [{representation, reform_sha256, binding}, ...]} with members in application order; an infinite number in a payload is the object {\"$nonfinite\": \"Infinity\"} or {\"$nonfinite\": \"-Infinity\"}, and a payload may not use the key $nonfinite itself",
"regional_certification.record_sha256": "the bundle manifest's data_releases.<country_id>.regional_certification record, its evidence included"
Expand Down
Loading
Loading