diff --git a/changelog.d/c1-17-budget-window-reruns.added.md b/changelog.d/c1-17-budget-window-reruns.added.md new file mode 100644 index 000000000..481580859 --- /dev/null +++ b/changelog.d/c1-17-budget-window-reruns.added.md @@ -0,0 +1 @@ +`POST /reports/{report_id}/runs` also re-runs a stored budget-window report, as a new budget-window run with a report run per year on each year's own simulations, its first years started as a new window's are. diff --git a/changelog.d/c1-17-household-reruns.added.md b/changelog.d/c1-17-household-reruns.added.md new file mode 100644 index 000000000..941c77014 --- /dev/null +++ b/changelog.d/c1-17-household-reruns.added.md @@ -0,0 +1 @@ +`POST /reports/{report_id}/runs` also re-runs a stored household report, in the report's own year, as a new report run with a new run of each side's own household simulation, started by one household function call; household runs are never reused, so `force` changes nothing. diff --git a/changelog.d/c1-17-reruns.added.md b/changelog.d/c1-17-reruns.added.md new file mode 100644 index 000000000..9519707f1 --- /dev/null +++ b/changelog.d/c1-17-reruns.added.md @@ -0,0 +1 @@ +The Simulation Entrypoint re-runs a stored economy report or simulation on the current .py version through `POST /reports/{report_id}/runs` and `POST /simulations/{simulation_id}/runs`, as a new run started by run ID, keeping every earlier run. A report's re-run reuses a run of each side's own simulation that succeeded on the same versions unless `force` is set; a record with a run pending or running is refused with 409 `run_in_progress`. diff --git a/docs/migration/stage-12-modal-worker-foundation.md b/docs/migration/stage-12-modal-worker-foundation.md index ae572dd91..5b5900672 100644 --- a/docs/migration/stage-12-modal-worker-foundation.md +++ b/docs/migration/stage-12-modal-worker-foundation.md @@ -334,6 +334,18 @@ as for a report comparing it with itself, so a report whose plan needs more columns, for example a reform that turns on labour supply responses, can't reuse it and fails with `reused_simulation_output_incompatible`. +**A re-run.** The service re-runs a stored report or simulation with the same +two calls, by run ID. It creates the record's new run (a report run with +trigger `manual`) and, for a report, a new `simulation_runs` row of each side's +own simulation, or names that simulation's run that already succeeded on the +same versions, unless the re-run is forced. The new runs record the catalog +version, dataset and region of the current .py version's catalog; the +`simulations` and `reports` rows are unchanged. A budget-window report's +re-run is a new `budget_window_runs` row (trigger `manual`) with a report run +per year, each on its year's own simulations, started as a new window's +first years are. The coordinator and workers claim and calculate them as +they would a new record's: nothing on the Modal side tells a re-run apart. + **Each worker** claims its simulation run under its own Modal call ID: a pending or running run that records no call or its own. It records its output file, or its failure, only while it holds the run running. It returns the output's @@ -789,6 +801,12 @@ pre-emption records nothing. An error in the report run's claim, or in the read of its simulation runs, fails the report run if the claim applied before the error, and leaves its simulation runs to the stuck-run sweep. +A household report's re-run (`POST /reports/{report_id}/runs`) is a new +report run, trigger `manual`, naming a new run of each side's own household +simulation on the current catalog version, in the report's year. The +service starts one call for it, as for a new household report, and the +function claims and calculates it as it would a new report's run. + The stuck-run sweep (`src/modal/run_sweep_app.py`) fails household jobs nothing will finish by the rule it applies to report runs and simulation runs (a household report run and its simulation runs are judged as any diff --git a/libs/policyengine-simulation-persistence/README.md b/libs/policyengine-simulation-persistence/README.md index 15b5b51d7..7a7bab9ac 100644 --- a/libs/policyengine-simulation-persistence/README.md +++ b/libs/policyengine-simulation-persistence/README.md @@ -148,6 +148,59 @@ the catalog version lacks; `create_report` raises inserted; and `record_report_call` and `fail_unstarted_report` use the economy report run writes. +## Re-running a report or simulation + +`economy_rerun_records.py` holds a re-run's reads and writes. A re-run +calculates a stored record again on the current versions, as a new run, and +changes no `reports` or `simulations` row, which the service's role may not +update: a report keeps its simulations, and each new run records the catalog +version, dataset and region it calculates on. + +- `read_report_definition` and `read_simulation_definition` read what the + record calculates: its kind (`economy`, `budget_window`, `household` or + `unsupported`), country, year, its region's code and dataset's name, and + each simulation's policy and dynamic. +- `find_reusable_run_of` searches one simulation's runs, by its ID, under + `find_reusable_run`'s conditions on a run (versions, the run's own catalog + version, dataset, region and year, the output file, a succeeded report run + with the same options, no refused reuse); it doesn't compare the + simulation row, which keeps the catalog rows it was created on. +- `insert_report_rerun` creates a report's new run, trigger `manual`, and a + new run of each side's simulation that reuses none; + `insert_simulation_rerun` creates a simulation's new run. Each refuses with + `RunInProgressError`, naming the run, while the record has a run `pending` + or `running` (for a report, a report run; for a simulation, any of its + runs), and first takes `lock_reruns_of`, a transaction-level + advisory lock on the record's ID, so concurrent re-runs of one record + wait for each other. When the database refuses the new run's key to the + record itself, the record was deleted after it was read: + `RerunRecordDeletedError`. + +A budget window's definition also names its newest budget-window run's +years (`WindowYearDefinition`): each year's sides are the simulations of the +runs that year's report run names. `insert_window_rerun` creates the +window's new budget-window run (trigger `manual`, `recalculate` set), a +report run per year and each year's new runs of its own simulations. Every +report re-run is refused while a budget-window run or a report run of the +report is `pending` or `running`. + +A household report's definition also names its household (`household_id`). +`insert_household_report_rerun` creates its new report run (trigger +`manual`) and a new run of each side's own household simulation (no dataset +or region), recording the current catalog version and the report's year: +household runs are never reused. It takes the same lock and makes the same +check for a run under way, and shares `insert_household_simulation_run` with +`insert_household_report`. `HouseholdReportStore.create_report_rerun` runs it +in a transaction of its own; the store's `read_inputs` reads the re-run's +household and policies as it does a new report's. + +`EconomyRunStore.read_report_rerun_inputs` reads a re-run's catalog rows by +the stored region code and dataset name, and each side's values and +reusable run of its own simulation, never giving both sides the same run. +`read_window_rerun_inputs` does the same for each year of a window. +`create_report_rerun`, `create_window_rerun` and `create_simulation_rerun` +run the inserts in a transaction of their own. + ## Reading a policy `read_policy_reform` (`policy_reform.py`) reads a stored policy as the reform diff --git a/libs/policyengine-simulation-persistence/src/policyengine_simulation_persistence/__init__.py b/libs/policyengine-simulation-persistence/src/policyengine_simulation_persistence/__init__.py index 461bda49c..5af229675 100644 --- a/libs/policyengine-simulation-persistence/src/policyengine_simulation_persistence/__init__.py +++ b/libs/policyengine-simulation-persistence/src/policyengine_simulation_persistence/__init__.py @@ -9,8 +9,33 @@ from policyengine_simulation_persistence.budget_window_records import ( BudgetWindowYear, NewBudgetWindowReport, + WindowSide, + WindowYearRuns, insert_budget_window_report, ) +from policyengine_simulation_persistence.economy_rerun_records import ( + HouseholdRerunSide, + NewHouseholdReportRerun, + NewReportRerun, + NewSimulationRerun, + NewWindowRerun, + RecordKind, + ReportDefinition, + RerunRecordDeletedError, + RerunSide, + RunInProgressError, + SideDefinition, + SimulationDefinition, + WindowRerunYear, + WindowYearDefinition, + insert_household_report_rerun, + insert_report_rerun, + insert_simulation_rerun, + insert_window_rerun, + lock_reruns_of, + read_report_definition, + read_simulation_definition, +) from policyengine_simulation_persistence.economy_run_reads import ( EconomyReportState, EconomySimulationState, @@ -33,6 +58,7 @@ fail_unstarted_report_run, fail_unstarted_simulation_run, find_reusable_run, + find_reusable_run_of, insert_economy_report, insert_economy_simulation, read_dataset, @@ -148,25 +174,37 @@ "HouseholdReportInputs", "HouseholdReportSimulation", "HouseholdReportStore", + "HouseholdRerunSide", "MissingPolicyParametersError", "NewBudgetWindowReport", "NewEconomyReport", "NewEconomySimulation", "NewHouseholdJob", "NewHouseholdReport", + "NewHouseholdReportRerun", + "NewReportRerun", + "NewSimulationRerun", + "NewWindowRerun", "OverlappingReformError", "PolicyReform", + "RecordKind", + "ReportDefinition", "ReportRunState", "ReportRunStatus", "ReportRunTrigger", "ReportSide", "ReportSideError", "ReportSimulation", + "RerunRecordDeletedError", + "RerunSide", "ReusableRun", + "RunInProgressError", "RunReuse", "RunVersions", + "SideDefinition", "SideRefusal", "SimulationChoice", + "SimulationDefinition", "SimulationInputs", "SimulationRunState", "SimulationStatus", @@ -178,6 +216,10 @@ "UnknownHouseholdError", "UnknownPolicyError", "UnknownRegionError", + "WindowRerunYear", + "WindowSide", + "WindowYearDefinition", + "WindowYearRuns", "bounded_transaction", "budget_window_runs", "combine_reforms", @@ -186,6 +228,7 @@ "fail_unstarted_report_run", "fail_unstarted_simulation_run", "find_reusable_run", + "find_reusable_run_of", "household_jobs", "input_metadata", "insert_budget_window_report", @@ -193,6 +236,11 @@ "insert_economy_simulation", "insert_household_job", "insert_household_report", + "insert_household_report_rerun", + "insert_report_rerun", + "insert_simulation_rerun", + "insert_window_rerun", + "lock_reruns_of", "missing_table_privileges", "normalized_database_url", "read_catalog_version_id", @@ -206,6 +254,8 @@ "read_model_id", "read_policy_reform", "read_region", + "read_report_definition", + "read_simulation_definition", "read_stored_household", "record_household_call", "record_report_run_call", diff --git a/libs/policyengine-simulation-persistence/src/policyengine_simulation_persistence/budget_window_records.py b/libs/policyengine-simulation-persistence/src/policyengine_simulation_persistence/budget_window_records.py index 111a025ff..9f40b6c40 100644 --- a/libs/policyengine-simulation-persistence/src/policyengine_simulation_persistence/budget_window_records.py +++ b/libs/policyengine-simulation-persistence/src/policyengine_simulation_persistence/budget_window_records.py @@ -17,7 +17,7 @@ from __future__ import annotations from dataclasses import dataclass, replace -from typing import TYPE_CHECKING, Any +from typing import TYPE_CHECKING, Any, Protocol from sqlalchemy import insert @@ -40,11 +40,59 @@ ) if TYPE_CHECKING: + from collections.abc import Sequence from uuid import UUID from sqlalchemy import Connection +class WindowSide(Protocol): + """A window year's side: its simulation and the run the year names.""" + + @property + def simulation_id(self) -> UUID: + """The side's simulation.""" + ... + + @property + def simulation_run_id(self) -> UUID: + """The run of it the year names.""" + ... + + @property + def reused(self) -> bool: + """Whether the run already succeeded, rather than being created.""" + ... + + +class WindowYearRuns(Protocol): + """A year of a window run: its year, its report run and each side. + + A new window's ``BudgetWindowYear`` and a re-run's ``WindowRerunYear`` + are both one. + """ + + @property + def year(self) -> int: + """The year.""" + ... + + @property + def report_run_id(self) -> UUID: + """The year's report run.""" + ... + + @property + def baseline(self) -> WindowSide: + """The baseline side.""" + ... + + @property + def reform(self) -> WindowSide: + """The reform side.""" + ... + + @dataclass(frozen=True) class BudgetWindowYear: """One year of a window: its report run and its two sides' runs.""" @@ -78,20 +126,7 @@ class NewBudgetWindowReport: def __post_init__(self) -> None: """Refuse years that aren't 1 to 75 consecutive ones from the first.""" - first_year = self.population.year - expected = tuple(range(first_year, first_year + len(self.years))) - if ( - not self.years - or len(self.years) > MAX_BUDGET_WINDOW_YEARS - or tuple(year.year for year in self.years) != expected - or expected[-1] > LATEST_YEAR - ): - msg = ( - "a budget window's years must be 1 to " - f"{MAX_BUDGET_WINDOW_YEARS} consecutive years from its first, " - f"ending by {LATEST_YEAR}" - ) - raise ValueError(msg) + check_window_years(self.population.year, [year.year for year in self.years]) @property def first_year(self) -> BudgetWindowYear: @@ -103,6 +138,26 @@ def population_in(self, year: int) -> EconomyPopulation: return replace(self.population, year=year) +def check_window_years(first_year: int, years: Sequence[int]) -> None: + """Raise ValueError unless ``years`` are 1 to 75 in a row from ``first_year``. + + The last of them must also be no later than the tables' latest year. + """ + expected = tuple(range(first_year, first_year + len(years))) + if ( + not years + or len(years) > MAX_BUDGET_WINDOW_YEARS + or tuple(years) != expected + or expected[-1] > LATEST_YEAR + ): + msg = ( + "a budget window's years must be 1 to " + f"{MAX_BUDGET_WINDOW_YEARS} consecutive years from its first, " + f"ending by {LATEST_YEAR}" + ) + raise ValueError(msg) + + def insert_budget_window_report( connection: Connection, report: NewBudgetWindowReport ) -> None: @@ -159,24 +214,42 @@ def insert_budget_window_report( ) connection.execute( insert(report_runs), - [_year_run_row(report, window_year) for window_year in report.years], + [ + year_run_row( + window_year, + report_id=report.id, + window_run_id=report.window_run_id, + versions=report.versions, + trigger=ReportRunTrigger.INITIAL, + ) + for window_year in report.years + ], ) -def _year_run_row( - report: NewBudgetWindowReport, window_year: BudgetWindowYear +def year_run_row( + window_year: WindowYearRuns, + *, + report_id: UUID, + window_run_id: UUID, + versions: RunVersions, + trigger: ReportRunTrigger, ) -> dict[str, Any]: - """Return a year's report run, pending with no call, as inserted.""" + """Return a year's report run in a window run, pending with no call, as inserted. + + ``window_year`` is a new window's year or a re-run's; ``trigger`` says + which. + """ return { "id": window_year.report_run_id, - "report_id": report.id, + "report_id": report_id, "baseline_simulation_run_id": window_year.baseline.simulation_run_id, "reform_simulation_run_id": window_year.reform.simulation_run_id, - "budget_window_run_id": report.window_run_id, + "budget_window_run_id": window_run_id, "year": window_year.year, - "country_package_version": report.versions.country_package_version, - "policyengine_version": report.versions.policyengine_version, + "country_package_version": versions.country_package_version, + "policyengine_version": versions.policyengine_version, "status": ReportRunStatus.PENDING, - "trigger": ReportRunTrigger.INITIAL, + "trigger": trigger, "modal_call_id": None, } diff --git a/libs/policyengine-simulation-persistence/src/policyengine_simulation_persistence/economy_rerun_records.py b/libs/policyengine-simulation-persistence/src/policyengine_simulation_persistence/economy_rerun_records.py new file mode 100644 index 000000000..b2be0d6a5 --- /dev/null +++ b/libs/policyengine-simulation-persistence/src/policyengine_simulation_persistence/economy_rerun_records.py @@ -0,0 +1,692 @@ +"""The simulation service's reads and writes for re-running a report or simulation. + +A re-run calculates a stored report or simulation again on the versions the +service runs now, as a new run, and keeps every earlier run. The service: + +- reads what the record calculates (``read_report_definition``, + ``read_simulation_definition``): its country, year, region code, dataset + name and each simulation's policy and dynamic, so the current catalog + version's rows with the same code and name can be found; +- creates, in one transaction, the new run and, for each report side not + reusing a run of its own simulation, a new ``simulation_runs`` row of that + simulation, all ``pending`` with no Modal call ID. A report's new run has + trigger ``manual``. A budget window's new run is a ``budget_window_runs`` + row with a report run per year, each year on the simulations its report + run in the report's newest window run names. A household report's new run + gives each side a new run of its own household simulation: household runs + are never reused. + +The record itself is never changed: the service may not update ``reports`` +or ``simulations``, so a report keeps naming the simulations it was created +with, and each new run records the catalog version, dataset and region it +calculates on. Nothing is created while the record has a run that is +``pending`` or ``running`` (for a report, a budget-window run or a report +run of it); a transaction-level advisory lock on the +record's ID makes concurrent re-runs of one record wait for each other, as +a row lock on ``reports`` would need ``UPDATE`` on it. + +The service then starts and records the new run's call as it does a new +record's (``economy_run_records``). +""" + +from __future__ import annotations + +from contextlib import contextmanager +from dataclasses import dataclass, replace +from typing import TYPE_CHECKING, Any, Final, Literal + +from sqlalchemy import func, insert, literal, select + +from policyengine_simulation_persistence.budget_window_records import ( + check_window_years, + year_run_row, +) +from policyengine_simulation_persistence.economy_run_records import ( + ECONOMY_REPORT_TYPE, + EconomyRecordReferenceError, + SimulationChoice, + insert_simulation_run, + references_checked, +) +from policyengine_simulation_persistence.household_report_records import ( + HOUSEHOLD_REPORT_TYPE, + insert_household_simulation_run, +) +from policyengine_simulation_persistence.input_tables import ( + MODEL_NAME_PREFIX, + datasets, + regions, + tax_benefit_model_versions, + tax_benefit_models, +) +from policyengine_simulation_persistence.tables import ( + ReportRunStatus, + ReportRunTrigger, + SimulationStatus, + SimulationType, + budget_window_runs, + report_runs, + reports, + simulation_runs, + simulations, +) + +if TYPE_CHECKING: + from collections.abc import Iterator + from uuid import UUID + + from sqlalchemy import Column, ColumnElement, Connection, RowMapping, Table + + from policyengine_simulation_persistence.economy_run_records import ( + EconomyPopulation, + RunVersions, + ) + +# What a stored report or simulation calculates, as a re-run tells them apart: +# a single-year economy report or simulation, a budget window, a household +# report or simulation, or anything else, such as a simulation of a filtered +# population. +type RecordKind = Literal["economy", "budget_window", "household", "unsupported"] +# Every run table's statuses (simulation, report and budget-window runs) use +# these values for a run under way. +_UNDER_WAY: Final = (SimulationStatus.PENDING.value, SimulationStatus.RUNNING.value) + + +@dataclass(frozen=True) +class SideDefinition: + """A report side's stored simulation, and its policy and dynamic.""" + + simulation_id: UUID + choice: SimulationChoice + + +@dataclass(frozen=True) +class WindowYearDefinition: + """One year of a budget window's newest calculation, and its sides' simulations. + + They are the simulations of the runs that year's report run names: a + year that reused a run names that run's simulation, which may be another + report's. + """ + + year: int + baseline_simulation_id: UUID + reform_simulation_id: UUID + + +@dataclass(frozen=True) +class ReportDefinition: + """What a stored report calculates, as a re-run reads it. + + ``region_code`` and ``dataset_name`` are the code and name of the + report's region and dataset, or None when the report names none (or a + row it named was deleted). ``baseline`` and ``reform`` are None when the + report names no such simulation; a budget window's are its first + year's. A budget window's ``window_years`` are the years of its newest + budget-window run whose report run names both sides, earliest first, + empty when it has none. ``household_id`` is a household report's stored + household, None for any other report. + """ + + id: UUID + country: str + kind: RecordKind + year: int | None + region_code: str | None + dataset_name: str | None + inputs: dict[str, Any] + baseline: SideDefinition | None + reform: SideDefinition | None + budget_window_size: int | None = None + window_years: tuple[WindowYearDefinition, ...] = () + household_id: UUID | None = None + + +@dataclass(frozen=True) +class SimulationDefinition: + """What a stored simulation calculates, as a re-run reads it. + + ``country`` comes from the simulation's catalog version's model; it is + None for a model the service does not name ``policyengine-``. + """ + + id: UUID + country: str | None + kind: RecordKind + year: int | None + region_code: str | None + dataset_name: str | None + choice: SimulationChoice + + +@dataclass(frozen=True) +class RerunSide: + """The simulation run one side of a report's re-run names. + + The simulation is always the report's own. With ``reused``, the run is + one of its runs that succeeded on the current versions, and nothing is + created for the side; otherwise the run is created, pending. + """ + + simulation_id: UUID + simulation_run_id: UUID + reused: bool + + +@dataclass(frozen=True) +class NewReportRerun: + """A stored report's new run, on the current versions and catalog rows.""" + + report_id: UUID + run_id: UUID + population: EconomyPopulation + versions: RunVersions + baseline: RerunSide + reform: RerunSide + + @property + def new_simulation_run_ids(self) -> tuple[UUID, ...]: + """Return the IDs of the simulation runs created for the re-run.""" + return tuple( + side.simulation_run_id + for side in (self.baseline, self.reform) + if not side.reused + ) + + +@dataclass(frozen=True) +class WindowRerunYear: + """One year of a budget window's re-run: its report run and each side's run.""" + + year: int + report_run_id: UUID + baseline: RerunSide + reform: RerunSide + + +@dataclass(frozen=True) +class NewWindowRerun: + """A stored budget window's new calculation, a report run per year. + + ``population`` is the first year's, on the current catalog version; + each year calculates on the same catalog version, region and dataset in + its own year. ``recalculate`` records whether the re-run was forced. + """ + + report_id: UUID + window_run_id: UUID + population: EconomyPopulation + versions: RunVersions + recalculate: bool + years: tuple[WindowRerunYear, ...] + + def __post_init__(self) -> None: + """Refuse years that aren't 1 to 75 consecutive ones from the first.""" + check_window_years(self.population.year, [year.year for year in self.years]) + + def population_in(self, year: int) -> EconomyPopulation: + """Return what the window's simulations of ``year`` calculate on.""" + return replace(self.population, year=year) + + +@dataclass(frozen=True) +class HouseholdRerunSide: + """One side of a household report's re-run: a new run of its own simulation.""" + + simulation_id: UUID + simulation_run_id: UUID + + +@dataclass(frozen=True) +class NewHouseholdReportRerun: + """A stored household report's new run, on the current catalog version. + + Each side gets a new run of its own household simulation, in the + report's ``year``; ``reform`` is None for a report of the baseline alone. + """ + + report_id: UUID + run_id: UUID + tax_benefit_model_version_id: UUID + year: int + versions: RunVersions + baseline: HouseholdRerunSide + reform: HouseholdRerunSide | None + + @property + def sides(self) -> tuple[HouseholdRerunSide, ...]: + """Return each side the re-run calculates, the baseline's first.""" + return (self.baseline,) if self.reform is None else (self.baseline, self.reform) + + @property + def simulation_run_ids(self) -> tuple[UUID, ...]: + """Return the IDs of the simulation runs created for the re-run.""" + return tuple(side.simulation_run_id for side in self.sides) + + +@dataclass(frozen=True) +class NewSimulationRerun: + """A stored simulation's new run, on the current versions and catalog rows.""" + + simulation_id: UUID + run_id: UUID + population: EconomyPopulation + versions: RunVersions + + +class RerunRecordDeletedError(LookupError): + """The re-run's report or simulation was deleted after the re-run read it. + + The database refused the new run's key to the record itself; nothing was + created. + """ + + +class RunInProgressError(Exception): + """The record has a run that is pending or running, so no re-run is created. + + ``run_id`` names that run: the newest such one. + """ + + def __init__(self, run_id: UUID) -> None: + """Keep the run under way.""" + super().__init__(f"run {run_id} is pending or running") + self.run_id = run_id + + +def read_report_definition( + connection: Connection, report_id: UUID +) -> ReportDefinition | None: + """Return what the stored report calculates, or None when there is none.""" + region = regions.alias("report_region") + dataset = datasets.alias("report_dataset") + baseline = simulations.alias("baseline_simulation") + reform = simulations.alias("reform_simulation") + row = ( + connection.execute( + select( + reports.c.id, + reports.c.country, + reports.c.type, + reports.c.household_id, + reports.c.budget_window_size, + reports.c.year, + reports.c.inputs, + region.c.code.label("region_code"), + dataset.c.name.label("dataset_name"), + baseline.c.id.label("baseline_id"), + baseline.c.policy_id.label("baseline_policy_id"), + baseline.c.dynamic_id.label("baseline_dynamic_id"), + reform.c.id.label("reform_id"), + reform.c.policy_id.label("reform_policy_id"), + reform.c.dynamic_id.label("reform_dynamic_id"), + ) + .select_from(reports) + .outerjoin(region, region.c.id == reports.c.region_id) + .outerjoin(dataset, dataset.c.id == reports.c.dataset_id) + .outerjoin(baseline, baseline.c.id == reports.c.baseline_simulation_id) + .outerjoin(reform, reform.c.id == reports.c.reform_simulation_id) + .where(reports.c.id == report_id) + ) + .mappings() + .one_or_none() + ) + if row is None: + return None + kind = _report_kind(row["type"], row["household_id"], row["budget_window_size"]) + return ReportDefinition( + id=row["id"], + country=row["country"], + kind=kind, + year=row["year"], + region_code=row["region_code"], + dataset_name=row["dataset_name"], + inputs=row["inputs"], + baseline=_side(row, "baseline"), + reform=_side(row, "reform"), + budget_window_size=row["budget_window_size"], + window_years=( + _window_years(connection, report_id) if kind == "budget_window" else () + ), + household_id=row["household_id"], + ) + + +def _window_years( + connection: Connection, report_id: UUID +) -> tuple[WindowYearDefinition, ...]: + """Return the years of the report's newest budget-window run, earliest first. + + A year whose report run names no reform is left out. + """ + newest_window_run = ( + select(budget_window_runs.c.id) + .where(budget_window_runs.c.report_id == report_id) + .order_by( + budget_window_runs.c.created_at.desc(), budget_window_runs.c.id.desc() + ) + .limit(1) + .scalar_subquery() + ) + baseline_run = simulation_runs.alias("baseline_run") + reform_run = simulation_runs.alias("reform_run") + rows = connection.execute( + select( + report_runs.c.year, + baseline_run.c.simulation_id.label("baseline_simulation_id"), + reform_run.c.simulation_id.label("reform_simulation_id"), + ) + .join( + baseline_run, baseline_run.c.id == report_runs.c.baseline_simulation_run_id + ) + .join(reform_run, reform_run.c.id == report_runs.c.reform_simulation_run_id) + .where(report_runs.c.budget_window_run_id == newest_window_run) + .order_by(report_runs.c.year) + ) + return tuple( + WindowYearDefinition( + year=row.year, + baseline_simulation_id=row.baseline_simulation_id, + reform_simulation_id=row.reform_simulation_id, + ) + for row in rows + ) + + +def _report_kind( + report_type: str | None, household_id: UUID | None, budget_window_size: int | None +) -> RecordKind: + if household_id is not None or report_type == HOUSEHOLD_REPORT_TYPE: + return "household" + if report_type != ECONOMY_REPORT_TYPE: + return "unsupported" + return "economy" if budget_window_size is None else "budget_window" + + +def _side( + row: RowMapping, side: Literal["baseline", "reform"] +) -> SideDefinition | None: + simulation_id = row[f"{side}_id"] + if simulation_id is None: + return None + return SideDefinition( + simulation_id=simulation_id, + choice=SimulationChoice( + policy_id=row[f"{side}_policy_id"], dynamic_id=row[f"{side}_dynamic_id"] + ), + ) + + +def read_simulation_definition( + connection: Connection, simulation_id: UUID +) -> SimulationDefinition | None: + """Return what the stored simulation calculates, or None when there is none.""" + row = ( + connection.execute( + select( + simulations.c.id, + simulations.c.simulation_type, + simulations.c.filter_field, + simulations.c.year, + simulations.c.policy_id, + simulations.c.dynamic_id, + tax_benefit_models.c.name.label("model_name"), + regions.c.code.label("region_code"), + datasets.c.name.label("dataset_name"), + ) + .select_from(simulations) + .join( + tax_benefit_model_versions, + tax_benefit_model_versions.c.id + == simulations.c.tax_benefit_model_version_id, + ) + .join( + tax_benefit_models, + tax_benefit_models.c.id == tax_benefit_model_versions.c.model_id, + ) + .outerjoin(regions, regions.c.id == simulations.c.region_id) + .outerjoin(datasets, datasets.c.id == simulations.c.dataset_id) + .where(simulations.c.id == simulation_id) + ) + .mappings() + .one_or_none() + ) + if row is None: + return None + model_name: str = row["model_name"] + simulation_type = SimulationType(row["simulation_type"]) + kind: RecordKind = ( + "household" + if simulation_type is SimulationType.HOUSEHOLD + else "economy" + if row["filter_field"] is None + else "unsupported" + ) + return SimulationDefinition( + id=row["id"], + country=model_name.removeprefix(MODEL_NAME_PREFIX) + if model_name.startswith(MODEL_NAME_PREFIX) + else None, + kind=kind, + year=row["year"], + region_code=row["region_code"], + dataset_name=row["dataset_name"], + choice=SimulationChoice( + policy_id=row["policy_id"], dynamic_id=row["dynamic_id"] + ), + ) + + +def lock_reruns_of(connection: Connection, record_id: UUID) -> None: + """Wait for, then hold until the transaction ends, the record's re-run lock. + + It is a transaction-level advisory lock keyed on the record's ID, which + any role may take: a second re-run of the same report or simulation + waits here until the first one's transaction ends, then sees its run. + """ + connection.execute( + select( + func.pg_advisory_xact_lock( + func.hashtextextended(literal(f"rerun:{record_id}"), 0) + ) + ) + ) + + +def _run_under_way( + connection: Connection, table: Table, condition: ColumnElement[bool] +) -> UUID | None: + """Return the newest pending or running run meeting ``condition``, or None.""" + return connection.execute( + select(table.c.id) + .where(condition, table.c.status.in_(_UNDER_WAY)) + .order_by(table.c.created_at.desc(), table.c.id.desc()) + .limit(1) + ).scalar_one_or_none() + + +def insert_report_rerun(connection: Connection, rerun: NewReportRerun) -> None: + """Create the report's new run and its sides' new simulation runs, pending. + + Take the report's re-run lock first. Raise RunInProgressError, creating + nothing, when the report has a run pending or running. Raise + RerunRecordDeletedError when the database refuses the new run's key to + the report, which no longer exists, and EconomyRecordReferenceError when + it refuses another row the new runs name, such as a side's simulation; + either refusal ends the caller's transaction, or its savepoint. + """ + lock_reruns_of(connection, rerun.report_id) + _refuse_while_report_runs(connection, rerun.report_id) + with _record_checked(report_runs.c.report_id), references_checked(): + for side in (rerun.baseline, rerun.reform): + if not side.reused: + insert_simulation_run( + connection, + run_id=side.simulation_run_id, + simulation_id=side.simulation_id, + population=rerun.population, + versions=rerun.versions, + ) + connection.execute( + insert(report_runs).values( + id=rerun.run_id, + report_id=rerun.report_id, + baseline_simulation_run_id=rerun.baseline.simulation_run_id, + reform_simulation_run_id=rerun.reform.simulation_run_id, + budget_window_run_id=None, + year=None, + country_package_version=rerun.versions.country_package_version, + policyengine_version=rerun.versions.policyengine_version, + status=ReportRunStatus.PENDING, + trigger=ReportRunTrigger.MANUAL, + modal_call_id=None, + ) + ) + + +def insert_window_rerun(connection: Connection, rerun: NewWindowRerun) -> None: + """Create the budget window's new run, each year's report run and new runs. + + The budget-window run has trigger ``manual`` and records ``recalculate``; + each year's report run names it and its year, and each side reusing no + run gets a new run of its year's simulation. All are pending with no + Modal call; the service then starts the first years. Raise as + ``insert_report_rerun`` does. + """ + lock_reruns_of(connection, rerun.report_id) + _refuse_while_report_runs(connection, rerun.report_id) + population = rerun.population + with ( + _record_checked(budget_window_runs.c.report_id, report_runs.c.report_id), + references_checked(), + ): + # First, so a report deleted after its reads is told apart from a + # year's simulation deleted with it. + connection.execute( + insert(budget_window_runs).values( + id=rerun.window_run_id, + report_id=rerun.report_id, + tax_benefit_model_version_id=population.tax_benefit_model_version_id, + trigger=ReportRunTrigger.MANUAL, + recalculate=rerun.recalculate, + status=ReportRunStatus.PENDING, + ) + ) + for window_year in rerun.years: + for side in (window_year.baseline, window_year.reform): + if not side.reused: + insert_simulation_run( + connection, + run_id=side.simulation_run_id, + simulation_id=side.simulation_id, + population=rerun.population_in(window_year.year), + versions=rerun.versions, + ) + connection.execute( + insert(report_runs), + [ + year_run_row( + window_year, + report_id=rerun.report_id, + window_run_id=rerun.window_run_id, + versions=rerun.versions, + trigger=ReportRunTrigger.MANUAL, + ) + for window_year in rerun.years + ], + ) + + +def insert_household_report_rerun( + connection: Connection, rerun: NewHouseholdReportRerun +) -> None: + """Create the household report's new run and a new run of each side, pending. + + The report run has trigger ``manual`` and names no budget-window run; + each side's run is of the report's own household simulation, with no + Modal call: the service then starts one household function call for the + report run. Raise as ``insert_report_rerun`` does. + """ + lock_reruns_of(connection, rerun.report_id) + _refuse_while_report_runs(connection, rerun.report_id) + with _record_checked(report_runs.c.report_id), references_checked(): + for side in rerun.sides: + insert_household_simulation_run( + connection, + run_id=side.simulation_run_id, + simulation_id=side.simulation_id, + year=rerun.year, + tax_benefit_model_version_id=rerun.tax_benefit_model_version_id, + versions=rerun.versions, + ) + connection.execute( + insert(report_runs).values( + id=rerun.run_id, + report_id=rerun.report_id, + baseline_simulation_run_id=rerun.baseline.simulation_run_id, + reform_simulation_run_id=( + None if rerun.reform is None else rerun.reform.simulation_run_id + ), + budget_window_run_id=None, + year=None, + country_package_version=rerun.versions.country_package_version, + policyengine_version=rerun.versions.policyengine_version, + status=ReportRunStatus.PENDING, + trigger=ReportRunTrigger.MANUAL, + modal_call_id=None, + ) + ) + + +def _refuse_while_report_runs(connection: Connection, report_id: UUID) -> None: + """Raise RunInProgressError while the report has a run pending or running. + + A budget-window run under way is named first, then a report run. + """ + under_way = _run_under_way( + connection, budget_window_runs, budget_window_runs.c.report_id == report_id + ) or _run_under_way(connection, report_runs, report_runs.c.report_id == report_id) + if under_way is not None: + raise RunInProgressError(under_way) + + +def insert_simulation_rerun(connection: Connection, rerun: NewSimulationRerun) -> None: + """Create the simulation's new run, pending with no call. + + Take the simulation's re-run lock first. Raise RunInProgressError, + creating nothing, when the simulation has a run pending or running, and + RerunRecordDeletedError or EconomyRecordReferenceError as + ``insert_report_rerun`` does. + """ + lock_reruns_of(connection, rerun.simulation_id) + under_way = _run_under_way( + connection, + simulation_runs, + simulation_runs.c.simulation_id == rerun.simulation_id, + ) + if under_way is not None: + raise RunInProgressError(under_way) + with _record_checked(simulation_runs.c.simulation_id), references_checked(): + insert_simulation_run( + connection, + run_id=rerun.run_id, + simulation_id=rerun.simulation_id, + population=rerun.population, + versions=rerun.versions, + ) + + +@contextmanager +def _record_checked(*own_keys: Column[Any]) -> Iterator[None]: + """Turn a refusal of a new run's key to the record into RerunRecordDeletedError. + + ``own_keys`` are those keys' columns; a refusal of any other row + propagates unchanged. + """ + try: + yield + except EconomyRecordReferenceError as error: + refused = (error.table_name, error.column_name) + if refused in {(key.table.name, key.key) for key in own_keys}: + raise RerunRecordDeletedError from None + raise diff --git a/libs/policyengine-simulation-persistence/src/policyengine_simulation_persistence/economy_run_records.py b/libs/policyengine-simulation-persistence/src/policyengine_simulation_persistence/economy_run_records.py index 10c7ea28a..b842d2b00 100644 --- a/libs/policyengine-simulation-persistence/src/policyengine_simulation_persistence/economy_run_records.py +++ b/libs/policyengine-simulation-persistence/src/policyengine_simulation_persistence/economy_run_records.py @@ -293,7 +293,7 @@ def find_reusable_run( # noqa: PLR0913 # each keyword is one part of the match - of an economy simulation with the same policy, dynamic, catalog version, dataset, region and year; - that succeeded on the same country package and .py versions, for that - year, dataset and region; + catalog version, year, dataset and region; - whose output file is under ``output_uri_prefix``, the output bucket, which never deletes a file; - that a succeeded report run used, in a report with the same ``inputs`` @@ -309,6 +309,77 @@ def find_reusable_run( # noqa: PLR0913 # each keyword is one part of the match responses: the refusal fails that report run, and the run is not offered again. """ + return _newest_reusable_run( + connection, + ( + simulations.c.simulation_type == SimulationType.ECONOMY, + simulations.c.policy_id.is_not_distinct_from(choice.policy_id), + simulations.c.dynamic_id.is_not_distinct_from(choice.dynamic_id), + simulations.c.tax_benefit_model_version_id + == population.tax_benefit_model_version_id, + simulations.c.dataset_id == population.dataset_id, + simulations.c.region_id == population.region_id, + simulations.c.year == population.year, + simulations.c.filter_field.is_(None), + ), + _WantedRun( + population=population, + versions=versions, + inputs=inputs, + output_uri_prefix=output_uri_prefix, + excluded_run_id=excluded_run_id, + ), + ) + + +def find_reusable_run_of( # noqa: PLR0913 # the simulation, then each part of the match + connection: Connection, + *, + simulation_id: UUID, + population: EconomyPopulation, + versions: RunVersions, + inputs: dict[str, Any], + output_uri_prefix: str, + excluded_run_id: UUID | None = None, +) -> ReusableRun | None: + """Return the succeeded run of one simulation a re-run of its report may name. + + The run is the newest, by completion, of the simulation's runs that meets + every condition ``find_reusable_run`` sets on a run, or None. Its catalog + version, dataset and region are compared on the run itself: a re-run on + a later .py version calculates the simulation on that version's catalog + rows, while the simulation keeps the ones it was created on. + """ + return _newest_reusable_run( + connection, + (simulation_runs.c.simulation_id == simulation_id,), + _WantedRun( + population=population, + versions=versions, + inputs=inputs, + output_uri_prefix=output_uri_prefix, + excluded_run_id=excluded_run_id, + ), + ) + + +@dataclass(frozen=True) +class _WantedRun: + """What a reusable run must have calculated, and the run it must not be.""" + + population: EconomyPopulation + versions: RunVersions + inputs: dict[str, Any] + output_uri_prefix: str + excluded_run_id: UUID | None + + +def _newest_reusable_run( + connection: Connection, + simulation_conditions: tuple[ColumnElement[bool], ...], + wanted: _WantedRun, +) -> ReusableRun | None: + """Return the newest run meeting the conditions and every run condition.""" used_by = report_runs.alias("used_by_report_run") refused_by = report_runs.alias("refused_by_report_run") used_with_same_options = ( @@ -317,7 +388,7 @@ def find_reusable_run( # noqa: PLR0913 # each keyword is one part of the match .where( used_by.c.status == ReportRunStatus.SUCCEEDED, _names_run(used_by), - cast(reports.c.inputs, JSONB) == literal(inputs, JSONB), + cast(reports.c.inputs, JSONB) == literal(wanted.inputs, JSONB), ) .exists() ) @@ -332,29 +403,27 @@ def find_reusable_run( # noqa: PLR0913 # each keyword is one part of the match ) .exists() ) + population = wanted.population + versions = wanted.versions conditions = [ - simulations.c.simulation_type == SimulationType.ECONOMY, - simulations.c.policy_id.is_not_distinct_from(choice.policy_id), - simulations.c.dynamic_id.is_not_distinct_from(choice.dynamic_id), - simulations.c.tax_benefit_model_version_id - == population.tax_benefit_model_version_id, - simulations.c.dataset_id == population.dataset_id, - simulations.c.region_id == population.region_id, - simulations.c.year == population.year, - simulations.c.filter_field.is_(None), + *simulation_conditions, simulation_runs.c.status == SimulationStatus.SUCCEEDED, simulation_runs.c.year == population.year, + simulation_runs.c.tax_benefit_model_version_id + == population.tax_benefit_model_version_id, simulation_runs.c.country_package_version == versions.country_package_version, simulation_runs.c.policyengine_version == versions.policyengine_version, simulation_runs.c.dataset_id == population.dataset_id, simulation_runs.c.region_id == population.region_id, - simulation_runs.c.output_uri.startswith(output_uri_prefix, autoescape=True), + simulation_runs.c.output_uri.startswith( + wanted.output_uri_prefix, autoescape=True + ), simulation_runs.c.output_sha256.is_not(None), used_with_same_options, ~refused, ] - if excluded_run_id is not None: - conditions.append(simulation_runs.c.id != excluded_run_id) + if wanted.excluded_run_id is not None: + conditions.append(simulation_runs.c.id != wanted.excluded_run_id) row = connection.execute( select(simulation_runs.c.id, simulation_runs.c.simulation_id) .join(simulations, simulations.c.id == simulation_runs.c.simulation_id) @@ -472,6 +541,28 @@ def insert_simulation_with_run( # noqa: PLR0913 # the simulation's and its run year=population.year, ) ) + insert_simulation_run( + connection, + run_id=run_id, + simulation_id=simulation_id, + population=population, + versions=versions, + ) + + +def insert_simulation_run( + connection: Connection, + *, + run_id: UUID, + simulation_id: UUID, + population: EconomyPopulation, + versions: RunVersions, +) -> None: + """Create a run of a stored simulation on ``population``, pending with no call. + + The run records the catalog version, dataset and region it calculates + on, which a re-run's may differ from its simulation's. + """ connection.execute( insert(simulation_runs).values( id=run_id, diff --git a/libs/policyengine-simulation-persistence/src/policyengine_simulation_persistence/economy_run_store.py b/libs/policyengine-simulation-persistence/src/policyengine_simulation_persistence/economy_run_store.py index 2ba26499c..4666ed427 100644 --- a/libs/policyengine-simulation-persistence/src/policyengine_simulation_persistence/economy_run_store.py +++ b/libs/policyengine-simulation-persistence/src/policyengine_simulation_persistence/economy_run_store.py @@ -9,7 +9,7 @@ from __future__ import annotations -from dataclasses import dataclass +from dataclasses import dataclass, replace from typing import TYPE_CHECKING, Any, Literal, get_args from sqlalchemy.exc import SQLAlchemyError @@ -17,6 +17,13 @@ from policyengine_simulation_persistence.budget_window_records import ( insert_budget_window_report, ) +from policyengine_simulation_persistence.economy_rerun_records import ( + insert_report_rerun, + insert_simulation_rerun, + insert_window_rerun, + read_report_definition, + read_simulation_definition, +) from policyengine_simulation_persistence.economy_run_reads import ( read_economy_report, read_economy_simulation, @@ -31,6 +38,7 @@ fail_unstarted_report_run, fail_unstarted_simulation_run, find_reusable_run, + find_reusable_run_of, insert_economy_report, insert_economy_simulation, read_dataset, @@ -58,7 +66,7 @@ ) if TYPE_CHECKING: - from collections.abc import Callable, Sequence + from collections.abc import Callable, Iterable, Sequence from contextlib import AbstractContextManager from uuid import UUID @@ -67,6 +75,15 @@ from policyengine_simulation_persistence.budget_window_records import ( NewBudgetWindowReport, ) + from policyengine_simulation_persistence.economy_rerun_records import ( + NewReportRerun, + NewSimulationRerun, + NewWindowRerun, + ReportDefinition, + SideDefinition, + SimulationDefinition, + WindowYearDefinition, + ) from policyengine_simulation_persistence.economy_run_reads import ( EconomyReportState, EconomySimulationState, @@ -84,6 +101,11 @@ type TransactionFactory = Callable[[], AbstractContextManager[Connection]] # Which of a report's simulations a value belongs to. type ReportSide = Literal["baseline", "reform"] +# Finds a report side's reusable run, given the side, its policy and dynamic, +# and the run the other side reuses, which it must not. +type ReusableRunSearch = Callable[ + [ReportSide, SimulationChoice, UUID | None], ReusableRun | None +] # The errors a simulation's own policy or dynamic can raise. type SideRefusal = ( UnknownPolicyError @@ -274,39 +296,138 @@ def read_report_inputs( """ with self._transaction() as connection: catalog = _catalog_rows(connection, request) - population = catalog.population(request.year) - sides = [] - excluded_run_id = None - sides_requested: tuple[tuple[ReportSide, SimulationChoice], ...] = ( - ("baseline", baseline), - ("reform", reform), + search = ( + None + if reuse is None + else _search_by_definition( + connection, catalog.population(request.year), reuse + ) ) - for side, choice in sides_requested: - try: - inputs = _simulation_inputs( - connection, request.country, catalog.catalog_version_id, choice - ) - except _SIDE_ERRORS as error: - raise ReportSideError(side, error) from error - if reuse is not None: - reusable_run = find_reusable_run( - connection, - choice=choice, - population=population, - versions=reuse.versions, - inputs=reuse.inputs, - output_uri_prefix=reuse.output_uri_prefix, - excluded_run_id=excluded_run_id, + return _report_inputs( + connection, + request, + catalog, + baseline=baseline, + reform=reform, + search=search, + ) + + def read_report_definition(self, report_id: UUID) -> ReportDefinition | None: + """Return what the stored report calculates, or None when there is none.""" + with self._transaction() as connection: + return read_report_definition(connection, report_id) + + def read_simulation_definition( + self, simulation_id: UUID + ) -> SimulationDefinition | None: + """Return what the stored simulation calculates, or None when there is none.""" + with self._transaction() as connection: + return read_simulation_definition(connection, simulation_id) + + def read_report_rerun_inputs( + self, + request: EconomyRequest, + *, + baseline: SideDefinition, + reform: SideDefinition, + reuse: RunReuse | None, + ) -> EconomyReportInputs: + """Read a report re-run's catalog rows, both sides' values and reusable runs. + + ``request`` names the stored report's region code and dataset name, + found in the catalog version of the current .py version. With + ``reuse``, each side gets the run ``find_reusable_run_of`` finds among + its own simulation's runs, and the reform never gets the baseline's. + Raise the errors ``read_report_inputs`` raises. + """ + with self._transaction() as connection: + catalog = _catalog_rows(connection, request) + search = ( + None + if reuse is None + else _search_own_runs( + connection, + catalog.population(request.year), + reuse, + { + "baseline": baseline.simulation_id, + "reform": reform.simulation_id, + }, + ) + ) + return _report_inputs( + connection, + request, + catalog, + baseline=baseline.choice, + reform=reform.choice, + search=search, + ) + + def read_window_rerun_inputs( + self, + request: EconomyRequest, + *, + baseline: SimulationChoice, + reform: SimulationChoice, + years: Sequence[WindowYearDefinition], + reuse: RunReuse | None, + ) -> BudgetWindowInputs: + """Read a window re-run's catalog rows, both sides' values and each year's runs. + + ``request.year`` is the window's first year. With ``reuse``, each + year's sides get the runs ``find_reusable_run_of`` finds among that + year's own simulations' runs, and the reform never gets the + baseline's. Raise the errors ``read_report_inputs`` raises. + """ + with self._transaction() as connection: + catalog = _catalog_rows(connection, request) + return _window_inputs( + connection, + request, + catalog, + baseline=baseline, + reform=reform, + searches=( + ( + window_year.year, + None + if reuse is None + else _search_own_runs( + connection, + catalog.population(window_year.year), + reuse, + { + "baseline": window_year.baseline_simulation_id, + "reform": window_year.reform_simulation_id, + }, + ), ) - if reusable_run is not None: - excluded_run_id = reusable_run.simulation_run_id - inputs = SimulationInputs( - choice=inputs.choice, - reform=inputs.reform, - reusable_run=reusable_run, - ) - sides.append(inputs) - return EconomyReportInputs(catalog=catalog, baseline=sides[0], reform=sides[1]) + for window_year in years + ), + ) + + def create_window_rerun(self, rerun: NewWindowRerun) -> None: + """Create the window's new run and its years' runs; see ``insert_window_rerun``. + + Raise EconomyRecordOutcomeUnknownError as ``create_report`` does. + """ + self._create(lambda connection: insert_window_rerun(connection, rerun)) + + def create_report_rerun(self, rerun: NewReportRerun) -> None: + """Create the report's new run and its sides' new simulation runs. + + Raise ``insert_report_rerun``'s errors, and + EconomyRecordOutcomeUnknownError as ``create_report`` does. + """ + self._create(lambda connection: insert_report_rerun(connection, rerun)) + + def create_simulation_rerun(self, rerun: NewSimulationRerun) -> None: + """Create the simulation's new run; see ``insert_simulation_rerun``. + + Raise EconomyRecordOutcomeUnknownError as ``create_report`` does. + """ + self._create(lambda connection: insert_simulation_rerun(connection, rerun)) def read_budget_window_inputs( self, @@ -326,43 +447,24 @@ def read_budget_window_inputs( """ with self._transaction() as connection: catalog = _catalog_rows(connection, request) - baseline_inputs = _side_inputs( - connection, request, catalog, "baseline", baseline - ) - reform_inputs = _side_inputs(connection, request, catalog, "reform", reform) - reusable_runs: dict[int, tuple[ReusableRun | None, ReusableRun | None]] = {} - for year in years: - baseline_run = reform_run = None - if reuse is not None: - population = catalog.population(year) - baseline_run = find_reusable_run( - connection, - choice=baseline, - population=population, - versions=reuse.versions, - inputs=reuse.inputs, - output_uri_prefix=reuse.output_uri_prefix, - ) - reform_run = find_reusable_run( - connection, - choice=reform, - population=population, - versions=reuse.versions, - inputs=reuse.inputs, - output_uri_prefix=reuse.output_uri_prefix, - excluded_run_id=( - None - if baseline_run is None - else baseline_run.simulation_run_id + return _window_inputs( + connection, + request, + catalog, + baseline=baseline, + reform=reform, + searches=( + ( + year, + None + if reuse is None + else _search_by_definition( + connection, catalog.population(year), reuse ), ) - reusable_runs[year] = (baseline_run, reform_run) - return BudgetWindowInputs( - catalog=catalog, - baseline=baseline_inputs, - reform=reform_inputs, - reusable_runs=reusable_runs, - ) + for year in years + ), + ) def create_budget_window_report(self, report: NewBudgetWindowReport) -> None: """Create the window's report, its budget-window run and each year's runs. @@ -567,3 +669,112 @@ def _side_inputs( ) except _SIDE_ERRORS as error: raise ReportSideError(side, error) from error + + +def _report_inputs( # noqa: PLR0913 # the request's rows, both sides and the search + connection: Connection, + request: EconomyRequest, + catalog: EconomyCatalogRows, + *, + baseline: SimulationChoice, + reform: SimulationChoice, + search: ReusableRunSearch | None, +) -> EconomyReportInputs: + """Read both sides' values and, with ``search``, each side's reusable run. + + The reform never gets the run the baseline reuses. + """ + sides: list[SimulationInputs] = [] + excluded_run_id: UUID | None = None + requested: tuple[tuple[ReportSide, SimulationChoice], ...] = ( + ("baseline", baseline), + ("reform", reform), + ) + for side, choice in requested: + inputs = _side_inputs(connection, request, catalog, side, choice) + reusable_run = None if search is None else search(side, choice, excluded_run_id) + if reusable_run is not None: + excluded_run_id = reusable_run.simulation_run_id + inputs = replace(inputs, reusable_run=reusable_run) + sides.append(inputs) + return EconomyReportInputs(catalog=catalog, baseline=sides[0], reform=sides[1]) + + +def _window_inputs( # noqa: PLR0913 # the request's rows, both sides and each year's search + connection: Connection, + request: EconomyRequest, + catalog: EconomyCatalogRows, + *, + baseline: SimulationChoice, + reform: SimulationChoice, + searches: Iterable[tuple[int, ReusableRunSearch | None]], +) -> BudgetWindowInputs: + """Read both sides' values, and each year's reusable runs with its search. + + A year without a search reuses nothing; in each year the reform never + gets the run the baseline reuses. + """ + baseline_inputs = _side_inputs(connection, request, catalog, "baseline", baseline) + reform_inputs = _side_inputs(connection, request, catalog, "reform", reform) + reusable_runs: dict[int, tuple[ReusableRun | None, ReusableRun | None]] = {} + for year, search in searches: + baseline_run = reform_run = None + if search is not None: + baseline_run = search("baseline", baseline, None) + reform_run = search( + "reform", + reform, + None if baseline_run is None else baseline_run.simulation_run_id, + ) + reusable_runs[year] = (baseline_run, reform_run) + return BudgetWindowInputs( + catalog=catalog, + baseline=baseline_inputs, + reform=reform_inputs, + reusable_runs=reusable_runs, + ) + + +def _search_by_definition( + connection: Connection, population: EconomyPopulation, reuse: RunReuse +) -> ReusableRunSearch: + """Search any simulation with the side's definition (a new report's sides).""" + + def search( + _side: ReportSide, choice: SimulationChoice, excluded_run_id: UUID | None + ) -> ReusableRun | None: + return find_reusable_run( + connection, + choice=choice, + population=population, + versions=reuse.versions, + inputs=reuse.inputs, + output_uri_prefix=reuse.output_uri_prefix, + excluded_run_id=excluded_run_id, + ) + + return search + + +def _search_own_runs( + connection: Connection, + population: EconomyPopulation, + reuse: RunReuse, + simulation_ids: dict[ReportSide, UUID], +) -> ReusableRunSearch: + """Search only each side's own simulation's runs (a stored report's re-run).""" + + def search( + side: ReportSide, _choice: SimulationChoice, excluded_run_id: UUID | None + ) -> ReusableRun | None: + return find_reusable_run_of( + connection, + simulation_id=simulation_ids[side], + population=population, + versions=reuse.versions, + inputs=reuse.inputs, + output_uri_prefix=reuse.output_uri_prefix, + excluded_run_id=excluded_run_id, + ) + + return search diff --git a/libs/policyengine-simulation-persistence/src/policyengine_simulation_persistence/household_report_records.py b/libs/policyengine-simulation-persistence/src/policyengine_simulation_persistence/household_report_records.py index 0ed192df0..3650d9c8d 100644 --- a/libs/policyengine-simulation-persistence/src/policyengine_simulation_persistence/household_report_records.py +++ b/libs/policyengine-simulation-persistence/src/policyengine_simulation_persistence/household_report_records.py @@ -19,7 +19,8 @@ records the call's ID on it, or fails it and its simulation runs at once, with ``economy_run_records``' report run writes. The household function writes the rest: each run's ``household_result``, and every status after -``pending``. +``pending``. A re-run (``economy_rerun_records``) adds a run to each of a +stored report's household simulations with ``insert_household_simulation_run``. """ from __future__ import annotations @@ -165,14 +166,38 @@ def _insert_household_simulation( year=report.year, ) ) + insert_household_simulation_run( + connection, + run_id=side.simulation_run_id, + simulation_id=side.simulation_id, + year=report.year, + tax_benefit_model_version_id=report.tax_benefit_model_version_id, + versions=report.versions, + ) + + +def insert_household_simulation_run( # noqa: PLR0913 # the run and what it calculates on + connection: Connection, + *, + run_id: UUID, + simulation_id: UUID, + year: int, + tax_benefit_model_version_id: UUID, + versions: RunVersions, +) -> None: + """Create a run of a household simulation, pending with no Modal call ID. + + It records the year, catalog version and versions it calculates on; a + household run names no dataset or region. + """ connection.execute( insert(simulation_runs).values( - id=side.simulation_run_id, - simulation_id=side.simulation_id, - year=report.year, - tax_benefit_model_version_id=report.tax_benefit_model_version_id, - country_package_version=report.versions.country_package_version, - policyengine_version=report.versions.policyengine_version, + id=run_id, + simulation_id=simulation_id, + year=year, + tax_benefit_model_version_id=tax_benefit_model_version_id, + country_package_version=versions.country_package_version, + policyengine_version=versions.policyengine_version, dataset_id=None, region_id=None, status=SimulationStatus.PENDING, diff --git a/libs/policyengine-simulation-persistence/src/policyengine_simulation_persistence/household_report_store.py b/libs/policyengine-simulation-persistence/src/policyengine_simulation_persistence/household_report_store.py index 9fa51b68a..2d5490c25 100644 --- a/libs/policyengine-simulation-persistence/src/policyengine_simulation_persistence/household_report_store.py +++ b/libs/policyengine-simulation-persistence/src/policyengine_simulation_persistence/household_report_store.py @@ -3,9 +3,12 @@ The Simulation Entrypoint creates a household report in three steps, each its own transaction, with the household function's call started between them: it reads the report's inputs, creates the rows, then records the call -on the report run or fails the run. ``HouseholdReportStore`` runs the reads -and writes of ``household_report_records``, ``household_job_records``, -``policy_reform`` and ``economy_run_records``' report run writes that way. +on the report run or fails the run. A stored report's re-run takes the same +steps, creating the report's new runs instead. ``HouseholdReportStore`` runs +the reads and writes of ``household_report_records``, +``household_job_records``, ``policy_reform``, ``economy_rerun_records``' +household report re-run and ``economy_run_records``' report run writes that +way. """ from __future__ import annotations @@ -15,6 +18,9 @@ from sqlalchemy.exc import SQLAlchemyError +from policyengine_simulation_persistence.economy_rerun_records import ( + insert_household_report_rerun, +) from policyengine_simulation_persistence.economy_run_records import ( fail_unstarted_report_run, read_model_id, @@ -49,6 +55,9 @@ from sqlalchemy import Connection, Engine + from policyengine_simulation_persistence.economy_rerun_records import ( + NewHouseholdReportRerun, + ) from policyengine_simulation_persistence.economy_run_store import ReportSide from policyengine_simulation_persistence.household_job_records import ( StoredHousehold, @@ -174,10 +183,24 @@ def create_report(self, report: NewHouseholdReport) -> None: EconomyRecordOutcomeUnknownError when the commit fails after the rows were inserted: they may exist. """ + self._create(lambda connection: insert_household_report(connection, report)) + + def create_report_rerun(self, rerun: NewHouseholdReportRerun) -> None: + """Create a stored report's new run and each side's new simulation run. + + Raise ``insert_household_report_rerun``'s errors, and + EconomyRecordOutcomeUnknownError as ``create_report`` does. + """ + self._create( + lambda connection: insert_household_report_rerun(connection, rerun) + ) + + def _create(self, insert_rows: Callable[[Connection], None]) -> None: + """Insert the rows in a transaction; tell a lost commit from a refusal.""" inserted = False try: with self._transaction() as connection: - insert_household_report(connection, report) + insert_rows(connection) inserted = True except SQLAlchemyError as error: if not inserted: diff --git a/libs/policyengine-simulation-persistence/tests/economy_rows.py b/libs/policyengine-simulation-persistence/tests/economy_rows.py index 4cb6a9014..b203b6aea 100644 --- a/libs/policyengine-simulation-persistence/tests/economy_rows.py +++ b/libs/policyengine-simulation-persistence/tests/economy_rows.py @@ -44,6 +44,12 @@ country_package_version=COUNTRY_PACKAGE_VERSION, policyengine_version=POLICYENGINE_VERSION, ) +# The versions of a release after ``VERSIONS``, with its own catalog version. +LATER_POLICYENGINE_VERSION = "6.18.0" +LATER_VERSIONS = RunVersions( + country_package_version="1.770.0", + policyengine_version=LATER_POLICYENGINE_VERSION, +) OUTPUT_BUCKET_PREFIX = "gs://policyengine-v2-simulation-outputs-staging/" CURRENT_LAW = SimulationChoice(policy_id=None, dynamic_id=None) OPTIONS: dict[str, Any] = {"include_cliffs": False} @@ -75,13 +81,52 @@ def insert_economy_catalog( parameter_names: tuple[str, ...] = (), ) -> EconomyCatalog: """Insert the country's model, a catalog version, a dataset and a region.""" - catalog = insert_catalog_version( + return _insert_catalog_rows( connection, country=country, model_id=insert_model(connection, country), policyengine_version=POLICYENGINE_VERSION, parameter_names=parameter_names, ) + + +def insert_later_catalog( + connection: Connection, + economy: EconomyCatalog, + *, + policyengine_version: str = LATER_POLICYENGINE_VERSION, + parameter_names: tuple[str, ...] = (), +) -> EconomyCatalog: + """Insert a later catalog version of ``economy``'s model. + + It has a dataset and a region with the same name and code as + ``economy``'s, as a release publishing the next .py version's catalog + would. + """ + return _insert_catalog_rows( + connection, + country=economy.catalog.country, + model_id=economy.catalog.model_id, + policyengine_version=policyengine_version, + parameter_names=parameter_names, + ) + + +def _insert_catalog_rows( + connection: Connection, + *, + country: str, + model_id: UUID, + policyengine_version: str, + parameter_names: tuple[str, ...], +) -> EconomyCatalog: + catalog = insert_catalog_version( + connection, + country=country, + model_id=model_id, + policyengine_version=policyengine_version, + parameter_names=parameter_names, + ) dataset_id = insert_dataset(connection, catalog, f"populace_{country}_2024") region_id = insert_region( connection, catalog, country, default_dataset_id=dataset_id @@ -120,9 +165,6 @@ def insert_simulation_run( """Insert an economy simulation and one run of it, as the superuser.""" shape = shape or RunShape() simulation_id = uuid4() - run_id = uuid4() - finished = shape.status in {SimulationStatus.SUCCEEDED, SimulationStatus.FAILED} - succeeded = shape.status is SimulationStatus.SUCCEEDED connection.execute( insert(simulations).values( id=simulation_id, @@ -135,6 +177,23 @@ def insert_simulation_run( year=population.year, ) ) + return insert_run_of(connection, simulation_id, population, shape) + + +def insert_run_of( + connection: Connection, + simulation_id: UUID, + population: EconomyPopulation, + shape: RunShape | None = None, +) -> StoredRun: + """Insert a run of a stored simulation on ``population``, as the superuser. + + ``shape.choice`` is ignored: the run keeps its simulation's policy. + """ + shape = shape or RunShape() + run_id = uuid4() + finished = shape.status in {SimulationStatus.SUCCEEDED, SimulationStatus.FAILED} + succeeded = shape.status is SimulationStatus.SUCCEEDED connection.execute( insert(simulation_runs).values( id=run_id, @@ -179,9 +238,26 @@ def insert_report_run_naming( report run's ID. """ outcome = outcome or ReportRunOutcome() + report_id = insert_report_of(connection, economy, compared, inputs=outcome.inputs) + return insert_report_run_of(connection, report_id, compared, outcome) + + +def insert_report_of( # noqa: PLR0913 # the report's sides and what else it may set + connection: Connection, + economy: EconomyCatalog, + compared: tuple[StoredRun, StoredRun], + *, + inputs: dict[str, Any] | None = None, + budget_window_size: int | None = None, + household_id: UUID | None = None, +) -> UUID: + """Insert an economy report naming the two runs' simulations, as the superuser. + + ``compared`` holds the baseline's run, then the reform's. The report is + on ``economy``'s region and dataset in 2026. Return its ID. + """ baseline, reform = compared report_id = uuid4() - run_id = uuid4() connection.execute( insert(reports).values( id=report_id, @@ -191,12 +267,30 @@ def insert_report_run_naming( tax_benefit_model_id=economy.catalog.model_id, baseline_simulation_id=baseline.simulation_id, reform_simulation_id=reform.simulation_id, + household_id=household_id, dataset_id=economy.dataset_id, region_id=economy.region_id, year=2026, - inputs=OPTIONS if outcome.inputs is None else outcome.inputs, + inputs=OPTIONS if inputs is None else inputs, + budget_window_size=budget_window_size, ) ) + return report_id + + +def insert_report_run_of( + connection: Connection, + report_id: UUID, + compared: tuple[StoredRun, StoredRun], + outcome: ReportRunOutcome | None = None, +) -> UUID: + """Insert a run of a stored report comparing the two runs, as the superuser. + + Return the report run's ID; ``outcome.inputs`` is ignored. + """ + outcome = outcome or ReportRunOutcome() + baseline, reform = compared + run_id = uuid4() finished = outcome.status in {ReportRunStatus.SUCCEEDED, ReportRunStatus.FAILED} connection.execute( insert(report_runs).values( diff --git a/libs/policyengine-simulation-persistence/tests/test_economy_reruns.py b/libs/policyengine-simulation-persistence/tests/test_economy_reruns.py new file mode 100644 index 000000000..53babd5aa --- /dev/null +++ b/libs/policyengine-simulation-persistence/tests/test_economy_reruns.py @@ -0,0 +1,902 @@ +"""Re-running a stored economy report or simulation on a later catalog version. + +A re-run reads what the stored report or simulation calculates, creates a new +run of it on the current versions, and keeps every earlier run. A report's +side may reuse a run of its own simulation that succeeded on those versions; +nothing is created while the record has a run pending or running. + +These run against a database the Public API's own v2 migration chain built +(``PUBLIC_API_SCHEMA_DATABASE_URL``), as the simulation service's role, each +in a transaction rolled back afterwards. +""" + +from __future__ import annotations + +from contextlib import contextmanager +from dataclasses import dataclass, replace +from datetime import UTC, datetime +from typing import TYPE_CHECKING +from uuid import UUID, uuid4 + +import pytest +from conftest import act_as_the_service, rolled_back_transaction +from economy_rows import ( + CURRENT_LAW, + LATER_POLICYENGINE_VERSION, + LATER_VERSIONS, + OPTIONS, + OUTPUT_BUCKET_PREFIX, + VERSIONS, + EconomyCatalog, + ReportRunOutcome, + RunShape, + StoredRun, + insert_economy_catalog, + insert_later_catalog, + insert_report_of, + insert_report_run_of, + insert_run_of, + insert_simulation_run, +) +from policyengine_simulation_persistence import ( + EconomyRecordReferenceError, + EconomyRequest, + EconomyRunStore, + MissingPolicyParametersError, + NewReportRerun, + NewSimulationRerun, + ReportDefinition, + ReportRunStatus, + ReportRunTrigger, + ReportSideError, + RerunRecordDeletedError, + RerunSide, + ReusableRun, + RunInProgressError, + RunReuse, + SideDefinition, + SimulationChoice, + SimulationDefinition, + SimulationStatus, + SimulationType, + UnknownDatasetError, + UnknownRegionError, + find_reusable_run, + find_reusable_run_of, + insert_report_rerun, + insert_simulation_rerun, + lock_reruns_of, + read_report_definition, + read_simulation_definition, + simulations, +) +from public_api_rows import ( + StoredValue, + insert_catalog_version, + insert_dataset, + insert_household, + insert_policy, + insert_region, +) +from sqlalchemy import insert, text +from sqlalchemy.exc import OperationalError + +if TYPE_CHECKING: + from collections.abc import Iterator + + from sqlalchemy import Connection + +pytestmark = pytest.mark.public_api_schema + +START = datetime(2026, 1, 1, tzinfo=UTC) +CTC_AMOUNT = "gov.irs.credits.ctc.amount.base[0].amount" +# What a re-run asks the current catalog version for: the stored report's +# region and dataset, on the later .py version. +LATER_REQUEST = EconomyRequest( + country="us", + policyengine_version=LATER_POLICYENGINE_VERSION, + region_code="us", + dataset_name="populace_us_2024", + year=2026, +) +LATER_REUSE = RunReuse( + versions=LATER_VERSIONS, inputs=OPTIONS, output_uri_prefix=OUTPUT_BUCKET_PREFIX +) + + +@dataclass(frozen=True) +class StoredReport: + """A stored report, the simulation runs its first run compared, and that run.""" + + report_id: UUID + baseline: StoredRun + reform: StoredRun + report_run_id: UUID + + +def _stored_report( + connection: Connection, + economy: EconomyCatalog, + *, + reform_choice: SimulationChoice = CURRENT_LAW, + run_status: ReportRunStatus = ReportRunStatus.SUCCEEDED, +) -> StoredReport: + population = economy.population() + baseline = insert_simulation_run(connection, population) + reform = insert_simulation_run( + connection, population, RunShape(choice=reform_choice) + ) + report_id = insert_report_of(connection, economy, (baseline, reform)) + report_run_id = insert_report_run_of( + connection, report_id, (baseline, reform), ReportRunOutcome(status=run_status) + ) + return StoredReport( + report_id=report_id, + baseline=baseline, + reform=reform, + report_run_id=report_run_id, + ) + + +def _row(connection: Connection, table_name: str, row_id: UUID) -> dict: + row = connection.execute( + text(f"SELECT * FROM public.{table_name} WHERE id = :id"), # noqa: S608 # fixed table names + {"id": row_id}, + ).mappings() + return dict(row.one()) + + +def _count_runs_of(connection: Connection, simulation_id: UUID) -> int: + return connection.execute( + text("SELECT count(*) FROM public.simulation_runs WHERE simulation_id = :id"), + {"id": simulation_id}, + ).scalar_one() + + +def _store_on(connection: Connection) -> EconomyRunStore: + """Return a store whose transactions are savepoints of ``connection``.""" + + @contextmanager + def savepoint() -> Iterator[Connection]: + with connection.begin_nested(): + yield connection + + return EconomyRunStore(savepoint) + + +def test_a_stored_economy_reports_definition_is_read(public_api_database_url) -> None: + with rolled_back_transaction(public_api_database_url) as connection: + economy = insert_economy_catalog(connection) + policy_id = insert_policy(connection, economy.catalog) + reform_choice = SimulationChoice(policy_id=policy_id, dynamic_id=None) + stored = _stored_report(connection, economy, reform_choice=reform_choice) + act_as_the_service(connection) + + definition = read_report_definition(connection, stored.report_id) + + assert definition == ReportDefinition( + id=stored.report_id, + country="us", + kind="economy", + year=2026, + region_code="us", + dataset_name="populace_us_2024", + inputs=OPTIONS, + baseline=SideDefinition( + simulation_id=stored.baseline.simulation_id, choice=CURRENT_LAW + ), + reform=SideDefinition( + simulation_id=stored.reform.simulation_id, choice=reform_choice + ), + ) + + +def test_a_budget_window_or_household_report_is_read_as_its_kind( + public_api_database_url, +) -> None: + with rolled_back_transaction(public_api_database_url) as connection: + economy = insert_economy_catalog(connection) + population = economy.population() + sides = ( + insert_simulation_run(connection, population), + insert_simulation_run(connection, population), + ) + window_id = insert_report_of(connection, economy, sides, budget_window_size=3) + household_id = insert_household( + connection, country="us", default_year=2026, document={"people": {}} + ) + household_report_id = insert_report_of( + connection, economy, sides, household_id=household_id + ) + act_as_the_service(connection) + + window = read_report_definition(connection, window_id) + household = read_report_definition(connection, household_report_id) + + assert window is not None + assert household is not None + assert (window.kind, household.kind) == ("budget_window", "household") + + +def test_an_unknown_report_or_simulation_has_no_definition( + public_api_database_url, +) -> None: + with rolled_back_transaction(public_api_database_url) as connection: + act_as_the_service(connection) + + assert read_report_definition(connection, uuid4()) is None + assert read_simulation_definition(connection, uuid4()) is None + + +def test_a_stored_economy_simulations_definition_is_read( + public_api_database_url, +) -> None: + with rolled_back_transaction(public_api_database_url) as connection: + economy = insert_economy_catalog(connection, country="uk") + policy_id = insert_policy(connection, economy.catalog) + choice = SimulationChoice(policy_id=policy_id, dynamic_id=None) + stored = insert_simulation_run( + connection, economy.population(year=2027), RunShape(choice=choice) + ) + act_as_the_service(connection) + + definition = read_simulation_definition(connection, stored.simulation_id) + + assert definition == SimulationDefinition( + id=stored.simulation_id, + country="uk", + kind="economy", + year=2027, + region_code="uk", + dataset_name="populace_uk_2024", + choice=choice, + ) + + +def _find_later_run_of( + connection: Connection, + simulation_id: UUID, + later: EconomyCatalog, + *, + excluded_run_id: UUID | None = None, +) -> ReusableRun | None: + return find_reusable_run_of( + connection, + simulation_id=simulation_id, + population=later.population(), + versions=LATER_VERSIONS, + inputs=OPTIONS, + output_uri_prefix=OUTPUT_BUCKET_PREFIX, + excluded_run_id=excluded_run_id, + ) + + +def _used_on_later_catalog( + connection: Connection, stored: StoredReport, later: EconomyCatalog +) -> StoredRun: + """Insert a succeeded later run of the baseline that a succeeded re-run used.""" + later_shape = RunShape(versions=LATER_VERSIONS) + later_baseline = insert_run_of( + connection, stored.baseline.simulation_id, later.population(), later_shape + ) + later_reform = insert_run_of( + connection, + stored.reform.simulation_id, + later.population(), + replace(later_shape, status=SimulationStatus.FAILED), + ) + insert_report_run_of(connection, stored.report_id, (later_baseline, later_reform)) + return later_baseline + + +def test_a_simulations_own_run_on_a_later_catalog_version_is_reused( + public_api_database_url, +) -> None: + with rolled_back_transaction(public_api_database_url) as connection: + economy = insert_economy_catalog(connection) + later = insert_later_catalog(connection, economy) + stored = _stored_report(connection, economy) + later_baseline = _used_on_later_catalog(connection, stored, later) + act_as_the_service(connection) + + found = _find_later_run_of(connection, stored.baseline.simulation_id, later) + excluded = _find_later_run_of( + connection, + stored.baseline.simulation_id, + later, + excluded_run_id=later_baseline.simulation_run_id, + ) + # The run on the earlier catalog version is not on the current versions. + earlier = find_reusable_run_of( + connection, + simulation_id=stored.baseline.simulation_id, + population=economy.population(), + versions=LATER_VERSIONS, + inputs=OPTIONS, + output_uri_prefix=OUTPUT_BUCKET_PREFIX, + ) + + assert found == ReusableRun( + simulation_id=stored.baseline.simulation_id, + simulation_run_id=later_baseline.simulation_run_id, + ) + assert excluded is None + assert earlier is None + + +def test_another_simulations_run_is_not_reused_for_a_simulations_rerun( + public_api_database_url, +) -> None: + with rolled_back_transaction(public_api_database_url) as connection: + economy = insert_economy_catalog(connection) + later = insert_later_catalog(connection, economy) + stored = _stored_report(connection, economy) + _used_on_later_catalog(connection, stored, later) + other = _stored_report(connection, economy) + act_as_the_service(connection) + + found = _find_later_run_of(connection, other.baseline.simulation_id, later) + + assert found is None + + +def _rerun_of( + stored: StoredReport, + later: EconomyCatalog, + *, + baseline_reused: StoredRun | None = None, +) -> NewReportRerun: + baseline = ( + RerunSide( + simulation_id=stored.baseline.simulation_id, + simulation_run_id=uuid4(), + reused=False, + ) + if baseline_reused is None + else RerunSide( + simulation_id=baseline_reused.simulation_id, + simulation_run_id=baseline_reused.simulation_run_id, + reused=True, + ) + ) + return NewReportRerun( + report_id=stored.report_id, + run_id=uuid4(), + population=later.population(), + versions=LATER_VERSIONS, + baseline=baseline, + reform=RerunSide( + simulation_id=stored.reform.simulation_id, + simulation_run_id=uuid4(), + reused=False, + ), + ) + + +def test_a_report_rerun_creates_runs_of_the_reports_own_simulations( + public_api_database_url, +) -> None: + with rolled_back_transaction(public_api_database_url) as connection: + economy = insert_economy_catalog(connection) + later = insert_later_catalog(connection, economy) + stored = _stored_report(connection, economy) + reused = _used_on_later_catalog(connection, stored, later) + rerun = _rerun_of(stored, later, baseline_reused=reused) + act_as_the_service(connection) + + insert_report_rerun(connection, rerun) + report_run = _row(connection, "report_runs", rerun.run_id) + reform_run = _row(connection, "simulation_runs", rerun.reform.simulation_run_id) + earlier_run = _row(connection, "report_runs", stored.report_run_id) + baseline_runs = _count_runs_of(connection, stored.baseline.simulation_id) + + assert rerun.new_simulation_run_ids == (rerun.reform.simulation_run_id,) + assert { + key: report_run[key] + for key in ( + "report_id", + "baseline_simulation_run_id", + "reform_simulation_run_id", + "country_package_version", + "policyengine_version", + "status", + "trigger", + "modal_call_id", + "budget_window_run_id", + "year", + ) + } == { + "report_id": stored.report_id, + "baseline_simulation_run_id": reused.simulation_run_id, + "reform_simulation_run_id": rerun.reform.simulation_run_id, + "country_package_version": LATER_VERSIONS.country_package_version, + "policyengine_version": LATER_VERSIONS.policyengine_version, + "status": ReportRunStatus.PENDING, + "trigger": ReportRunTrigger.MANUAL, + "modal_call_id": None, + "budget_window_run_id": None, + "year": None, + } + assert { + key: reform_run[key] + for key in ( + "simulation_id", + "year", + "tax_benefit_model_version_id", + "dataset_id", + "region_id", + "country_package_version", + "policyengine_version", + "status", + "modal_call_id", + ) + } == { + "simulation_id": stored.reform.simulation_id, + "year": 2026, + "tax_benefit_model_version_id": later.catalog.version_id, + "dataset_id": later.dataset_id, + "region_id": later.region_id, + "country_package_version": LATER_VERSIONS.country_package_version, + "policyengine_version": LATER_VERSIONS.policyengine_version, + "status": SimulationStatus.PENDING, + "modal_call_id": None, + } + # The earlier run is kept, and the reused side gets no new run. + assert earlier_run["status"] == ReportRunStatus.SUCCEEDED + assert baseline_runs == 2 + + +@pytest.mark.parametrize("status", [ReportRunStatus.PENDING, ReportRunStatus.RUNNING]) +def test_a_report_rerun_is_refused_while_a_run_is_pending_or_running( + public_api_database_url, status: ReportRunStatus +) -> None: + with rolled_back_transaction(public_api_database_url) as connection: + economy = insert_economy_catalog(connection) + later = insert_later_catalog(connection, economy) + stored = _stored_report(connection, economy, run_status=status) + rerun = _rerun_of(stored, later) + act_as_the_service(connection) + + with pytest.raises(RunInProgressError) as refused: + insert_report_rerun(connection, rerun) + created = connection.execute( + text("SELECT count(*) FROM public.report_runs WHERE id = :id"), + {"id": rerun.run_id}, + ).scalar_one() + + assert refused.value.run_id == stored.report_run_id + assert created == 0 + + +def test_a_report_whose_runs_have_ended_is_rerun_again(public_api_database_url) -> None: + with rolled_back_transaction(public_api_database_url) as connection: + economy = insert_economy_catalog(connection) + later = insert_later_catalog(connection, economy) + stored = _stored_report(connection, economy, run_status=ReportRunStatus.FAILED) + first = _rerun_of(stored, later) + act_as_the_service(connection) + insert_report_rerun(connection, first) + # The first re-run's run fails before a second re-run. + connection.execute(text("RESET ROLE")) + connection.execute( + text( + "UPDATE public.report_runs SET status = 'failed', " + "completed_at = now() WHERE id = :id" + ), + {"id": first.run_id}, + ) + second = _rerun_of(stored, later) + act_as_the_service(connection) + + insert_report_rerun(connection, second) + + assert _row(connection, "report_runs", second.run_id)["status"] == ( + ReportRunStatus.PENDING + ) + + +def test_a_simulation_rerun_creates_a_run_and_is_refused_while_one_is_under_way( + public_api_database_url, +) -> None: + with rolled_back_transaction(public_api_database_url) as connection: + economy = insert_economy_catalog(connection) + later = insert_later_catalog(connection, economy) + stored = insert_simulation_run(connection, economy.population()) + rerun = NewSimulationRerun( + simulation_id=stored.simulation_id, + run_id=uuid4(), + population=later.population(), + versions=LATER_VERSIONS, + ) + act_as_the_service(connection) + + insert_simulation_rerun(connection, rerun) + created = _row(connection, "simulation_runs", rerun.run_id) + with pytest.raises(RunInProgressError) as refused: + insert_simulation_rerun(connection, replace(rerun, run_id=uuid4())) + earlier = _row(connection, "simulation_runs", stored.simulation_run_id) + + assert ( + created["simulation_id"], + created["tax_benefit_model_version_id"], + created["policyengine_version"], + created["status"], + ) == ( + stored.simulation_id, + later.catalog.version_id, + LATER_POLICYENGINE_VERSION, + SimulationStatus.PENDING, + ) + assert refused.value.run_id == rerun.run_id + assert earlier["status"] == SimulationStatus.SUCCEEDED + + +def test_a_rerun_holds_its_records_lock_until_its_transaction_ends( + public_api_database_url, +) -> None: + """A second re-run of the record waits; one of another record does not.""" + with rolled_back_transaction(public_api_database_url) as holder: + economy = insert_economy_catalog(holder) + later = insert_later_catalog(holder, economy) + stored = _stored_report(holder, economy, run_status=ReportRunStatus.FAILED) + act_as_the_service(holder) + insert_report_rerun(holder, _rerun_of(stored, later)) + + with rolled_back_transaction(public_api_database_url) as waiter: + act_as_the_service(waiter) + waiter.execute(text("SET LOCAL lock_timeout = '200ms'")) + lock_reruns_of(waiter, uuid4()) + with pytest.raises(OperationalError, match="lock timeout"): + lock_reruns_of(waiter, stored.report_id) + + +def test_a_report_reruns_inputs_are_read_on_the_current_catalog_version( + public_api_database_url, +) -> None: + with rolled_back_transaction(public_api_database_url) as connection: + economy = insert_economy_catalog(connection, parameter_names=(CTC_AMOUNT,)) + later = insert_later_catalog(connection, economy, parameter_names=(CTC_AMOUNT,)) + policy_id = insert_policy( + connection, economy.catalog, [StoredValue(CTC_AMOUNT, 3_000, START)] + ) + reform_choice = SimulationChoice(policy_id=policy_id, dynamic_id=None) + stored = _stored_report(connection, economy, reform_choice=reform_choice) + reused = _used_on_later_catalog(connection, stored, later) + act_as_the_service(connection) + store = _store_on(connection) + sides = { + "baseline": SideDefinition(stored.baseline.simulation_id, CURRENT_LAW), + "reform": SideDefinition(stored.reform.simulation_id, reform_choice), + } + + inputs = store.read_report_rerun_inputs( + LATER_REQUEST, **sides, reuse=LATER_REUSE + ) + forced = store.read_report_rerun_inputs(LATER_REQUEST, **sides, reuse=None) + + assert (inputs.catalog.catalog_version_id, inputs.catalog.region.id) == ( + later.catalog.version_id, + later.region_id, + ) + assert inputs.catalog.dataset.id == later.dataset_id + assert inputs.baseline.reusable_run == ReusableRun( + simulation_id=stored.baseline.simulation_id, + simulation_run_id=reused.simulation_run_id, + ) + assert inputs.reform.reusable_run is None + assert inputs.reform.reform == {CTC_AMOUNT: {"2026-01-01.2200-12-31": 3_000}} + assert forced.baseline.reusable_run is None + + +def test_both_sides_of_one_simulation_never_reuse_the_same_run( + public_api_database_url, +) -> None: + with rolled_back_transaction(public_api_database_url) as connection: + economy = insert_economy_catalog(connection) + later = insert_later_catalog(connection, economy) + stored = _stored_report(connection, economy) + reused = _used_on_later_catalog(connection, stored, later) + act_as_the_service(connection) + side = SideDefinition(stored.baseline.simulation_id, CURRENT_LAW) + + inputs = _store_on(connection).read_report_rerun_inputs( + LATER_REQUEST, baseline=side, reform=side, reuse=LATER_REUSE + ) + + assert inputs.baseline.reusable_run == ReusableRun( + simulation_id=stored.baseline.simulation_id, + simulation_run_id=reused.simulation_run_id, + ) + assert inputs.reform.reusable_run is None + + +def test_a_rerun_names_the_side_whose_policy_the_current_catalog_cannot_set( + public_api_database_url, +) -> None: + with rolled_back_transaction(public_api_database_url) as connection: + economy = insert_economy_catalog(connection, parameter_names=(CTC_AMOUNT,)) + insert_later_catalog(connection, economy) + policy_id = insert_policy( + connection, economy.catalog, [StoredValue(CTC_AMOUNT, 3_000, START)] + ) + reform_choice = SimulationChoice(policy_id=policy_id, dynamic_id=None) + stored = _stored_report(connection, economy, reform_choice=reform_choice) + act_as_the_service(connection) + + with pytest.raises(ReportSideError) as refused: + _store_on(connection).read_report_rerun_inputs( + LATER_REQUEST, + baseline=SideDefinition(stored.baseline.simulation_id, CURRENT_LAW), + reform=SideDefinition(stored.reform.simulation_id, reform_choice), + reuse=None, + ) + + assert refused.value.side == "reform" + assert isinstance(refused.value.error, MissingPolicyParametersError) + assert refused.value.error.parameter_names == (CTC_AMOUNT,) + + +@pytest.mark.parametrize( + ("missing", "expected"), + [("region", UnknownRegionError), ("dataset", UnknownDatasetError)], +) +def test_a_rerun_is_refused_when_the_current_catalog_lacks_its_region_or_dataset( + public_api_database_url, missing: str, expected: type[Exception] +) -> None: + with rolled_back_transaction(public_api_database_url) as connection: + economy = insert_economy_catalog(connection) + later = insert_later_catalog(connection, economy) + other_dataset_id = insert_dataset(connection, later.catalog, "acs_2024") + insert_region( + connection, + later.catalog, + "state/ca", + default_dataset_id=other_dataset_id, + region_type="state", + ) + stored = _stored_report(connection, economy) + act_as_the_service(connection) + request = ( + replace(LATER_REQUEST, region_code="state/tx") + if missing == "region" + else replace(LATER_REQUEST, dataset_name="populace_us_2023") + ) + + with pytest.raises(expected): + _store_on(connection).read_report_rerun_inputs( + request, + baseline=SideDefinition(stored.baseline.simulation_id, CURRENT_LAW), + reform=SideDefinition(stored.reform.simulation_id, CURRENT_LAW), + reuse=None, + ) + + +def test_a_simulation_reruns_inputs_are_read_on_the_current_catalog_version( + public_api_database_url, +) -> None: + with rolled_back_transaction(public_api_database_url) as connection: + economy = insert_economy_catalog(connection, parameter_names=(CTC_AMOUNT,)) + later = insert_later_catalog(connection, economy, parameter_names=(CTC_AMOUNT,)) + policy_id = insert_policy( + connection, economy.catalog, [StoredValue(CTC_AMOUNT, 3_000, START)] + ) + choice = SimulationChoice(policy_id=policy_id, dynamic_id=None) + act_as_the_service(connection) + + # A simulation's re-run reads the current catalog version as a new one's. + inputs = _store_on(connection).read_simulation_inputs(LATER_REQUEST, choice) + + assert inputs.catalog.catalog_version_id == later.catalog.version_id + assert inputs.simulation.choice == choice + assert inputs.simulation.reform == {CTC_AMOUNT: {"2026-01-01.2200-12-31": 3_000}} + + +def test_a_rerun_and_its_runs_are_created_through_the_store( + public_api_database_url, +) -> None: + with rolled_back_transaction(public_api_database_url) as connection: + economy = insert_economy_catalog(connection) + later = insert_later_catalog(connection, economy) + stored = _stored_report(connection, economy) + rerun = _rerun_of(stored, later) + simulation_rerun = NewSimulationRerun( + simulation_id=insert_simulation_run( + connection, economy.population() + ).simulation_id, + run_id=uuid4(), + population=later.population(), + versions=LATER_VERSIONS, + ) + act_as_the_service(connection) + store = _store_on(connection) + + store.create_report_rerun(rerun) + store.create_simulation_rerun(simulation_rerun) + definition = store.read_report_definition(stored.report_id) + simulation = store.read_simulation_definition(simulation_rerun.simulation_id) + with pytest.raises(RunInProgressError): + store.create_report_rerun(_rerun_of(stored, later)) + + assert _row(connection, "report_runs", rerun.run_id)["trigger"] == ( + ReportRunTrigger.MANUAL + ) + assert _row(connection, "simulation_runs", simulation_rerun.run_id)[ + "status" + ] == (SimulationStatus.PENDING) + assert definition is not None + assert definition.id == stored.report_id + assert simulation is not None + assert simulation.id == simulation_rerun.simulation_id + + +def test_a_run_on_another_catalog_version_is_not_reused_by_a_new_report( + public_api_database_url, +) -> None: + """Only the run's own catalog version differs from the request's. + + It is also the newest run, so it would be found without that condition. + """ + with rolled_back_transaction(public_api_database_url) as connection: + economy = insert_economy_catalog(connection) + later = insert_later_catalog(connection, economy) + stored = _stored_report(connection, economy) + connection.execute( + text( + "UPDATE public.simulation_runs " + "SET tax_benefit_model_version_id = :later, " + "completed_at = completed_at + interval '1 day' WHERE id = :id" + ), + { + "later": later.catalog.version_id, + "id": stored.baseline.simulation_run_id, + }, + ) + act_as_the_service(connection) + + found = find_reusable_run( + connection, + choice=CURRENT_LAW, + population=economy.population(), + versions=VERSIONS, + inputs=OPTIONS, + output_uri_prefix=OUTPUT_BUCKET_PREFIX, + ) + + # The reform's run, the same simulation definition on the right catalog + # version, is the only one left to reuse. + assert found == ReusableRun( + simulation_id=stored.reform.simulation_id, + simulation_run_id=stored.reform.simulation_run_id, + ) + + +def test_a_household_or_filtered_simulation_and_another_model_are_read_as_such( + public_api_database_url, +) -> None: + with rolled_back_transaction(public_api_database_url) as connection: + economy = insert_economy_catalog(connection) + household_id = insert_household( + connection, country="us", default_year=2026, document={"people": {}} + ) + household_simulation_id = uuid4() + filtered_simulation_id = uuid4() + connection.execute( + insert(simulations).values( + id=household_simulation_id, + simulation_type=SimulationType.HOUSEHOLD, + household_id=household_id, + tax_benefit_model_version_id=economy.catalog.version_id, + year=2026, + ) + ) + connection.execute( + insert(simulations).values( + id=filtered_simulation_id, + simulation_type=SimulationType.ECONOMY, + dataset_id=economy.dataset_id, + region_id=economy.region_id, + tax_benefit_model_version_id=economy.catalog.version_id, + year=2026, + filter_field="state_code", + filter_value="CA", + ) + ) + other_model_id = uuid4() + connection.execute( + text( + "INSERT INTO public.tax_benefit_models (id, name) VALUES (:id, :name)" + ), + {"id": other_model_id, "name": "another-model"}, + ) + other = _insert_catalog_of(connection, other_model_id) + other_simulation = insert_simulation_run(connection, other.population()) + act_as_the_service(connection) + + household = read_simulation_definition(connection, household_simulation_id) + filtered = read_simulation_definition(connection, filtered_simulation_id) + of_another_model = read_simulation_definition( + connection, other_simulation.simulation_id + ) + + assert household is not None + assert (household.kind, household.country) == ("household", "us") + assert (household.region_code, household.dataset_name) == (None, None) + assert filtered is not None + assert filtered.kind == "unsupported" + assert of_another_model is not None + assert (of_another_model.kind, of_another_model.country) == ("economy", None) + + +def _insert_catalog_of(connection: Connection, model_id: UUID) -> EconomyCatalog: + catalog = insert_catalog_version( + connection, + country="xx", + model_id=model_id, + policyengine_version=LATER_POLICYENGINE_VERSION, + ) + dataset_id = insert_dataset(connection, catalog, "another_dataset") + region_id = insert_region( + connection, catalog, "everywhere", default_dataset_id=dataset_id + ) + return EconomyCatalog(catalog=catalog, dataset_id=dataset_id, region_id=region_id) + + +def test_a_report_of_another_type_is_read_as_unsupported( + public_api_database_url, +) -> None: + with rolled_back_transaction(public_api_database_url) as connection: + economy = insert_economy_catalog(connection) + stored = _stored_report(connection, economy) + connection.execute( + text("UPDATE public.reports SET type = 'custom_total' WHERE id = :id"), + {"id": stored.report_id}, + ) + act_as_the_service(connection) + + definition = read_report_definition(connection, stored.report_id) + + assert definition is not None + assert definition.kind == "unsupported" + + +def test_a_rerun_of_a_deleted_report_or_simulation_says_the_record_is_gone( + public_api_database_url, +) -> None: + """The record's own key refuses the new run: the record was deleted.""" + with rolled_back_transaction(public_api_database_url) as connection: + economy = insert_economy_catalog(connection) + later = insert_later_catalog(connection, economy) + stored = _stored_report(connection, economy, run_status=ReportRunStatus.FAILED) + act_as_the_service(connection) + + with pytest.raises(RerunRecordDeletedError), connection.begin_nested(): + insert_report_rerun( + connection, replace(_rerun_of(stored, later), report_id=uuid4()) + ) + with pytest.raises(RerunRecordDeletedError), connection.begin_nested(): + insert_simulation_rerun( + connection, + NewSimulationRerun( + simulation_id=uuid4(), + run_id=uuid4(), + population=later.population(), + versions=LATER_VERSIONS, + ), + ) + # A report whose side simulation is gone names that simulation instead. + gone_side = replace( + _rerun_of(stored, later), + reform=RerunSide( + simulation_id=uuid4(), simulation_run_id=uuid4(), reused=False + ), + ) + with ( + pytest.raises(EconomyRecordReferenceError) as refused, + connection.begin_nested(), + ): + insert_report_rerun(connection, gone_side) + + assert (refused.value.table_name, refused.value.column_name) == ( + "simulation_runs", + "simulation_id", + ) diff --git a/libs/policyengine-simulation-persistence/tests/test_economy_window_reruns.py b/libs/policyengine-simulation-persistence/tests/test_economy_window_reruns.py new file mode 100644 index 000000000..6d7619f6b --- /dev/null +++ b/libs/policyengine-simulation-persistence/tests/test_economy_window_reruns.py @@ -0,0 +1,608 @@ +"""Re-running a stored budget-window report on a later catalog version. + +A budget window's re-run is a new budget-window run of the report, with a +report run per year. Each year's sides are the simulations that year's +report run of the report's newest window run names, which may belong to +another report when that year reused a run. A side may reuse a run of its +own simulation that succeeded on the current versions. These run against a +database the Public API's own v2 migration chain built +(``PUBLIC_API_SCHEMA_DATABASE_URL``), as the simulation service's role, in +a transaction rolled back afterwards. +""" + +from __future__ import annotations + +from contextlib import contextmanager +from dataclasses import dataclass, replace +from typing import TYPE_CHECKING +from uuid import UUID, uuid4 + +import pytest +from conftest import act_as_the_service, rolled_back_transaction +from economy_rows import ( + CURRENT_LAW, + LATER_POLICYENGINE_VERSION, + LATER_VERSIONS, + OPTIONS, + OUTPUT_BUCKET_PREFIX, + VERSIONS, + EconomyCatalog, + RunShape, + insert_economy_catalog, + insert_later_catalog, + insert_report_run_of, + insert_run_of, +) +from policyengine_simulation_persistence import ( + BudgetWindowYear, + EconomyPopulation, + EconomyRecordReferenceError, + EconomyRequest, + EconomyRunStore, + NewBudgetWindowReport, + NewReportRerun, + NewWindowRerun, + ReportRunStatus, + ReportRunTrigger, + ReportSimulation, + RerunRecordDeletedError, + RerunSide, + ReusableRun, + RunInProgressError, + RunReuse, + SimulationStatus, + WindowRerunYear, + WindowYearDefinition, + insert_budget_window_report, + insert_report_rerun, + insert_window_rerun, + read_report_definition, +) +from sqlalchemy import text + +if TYPE_CHECKING: + from collections.abc import Iterator + + from sqlalchemy import Connection + +pytestmark = pytest.mark.public_api_schema + +WINDOW_YEARS = (2026, 2027, 2028) +LATER_REQUEST = EconomyRequest( + country="us", + policyengine_version=LATER_POLICYENGINE_VERSION, + region_code="us", + dataset_name="populace_us_2024", + year=2026, +) +LATER_REUSE = RunReuse( + versions=LATER_VERSIONS, inputs=OPTIONS, output_uri_prefix=OUTPUT_BUCKET_PREFIX +) + + +@dataclass(frozen=True) +class StoredWindow: + """A stored budget-window report, its window run and each year's rows.""" + + report_id: UUID + window_run_id: UUID + years: tuple[BudgetWindowYear, ...] + + +def _new_side() -> ReportSimulation: + return ReportSimulation( + simulation_id=uuid4(), + simulation_run_id=uuid4(), + choice=CURRENT_LAW, + reused=False, + ) + + +def _stored_window( + connection: Connection, + economy: EconomyCatalog, + *, + window_status: ReportRunStatus = ReportRunStatus.SUCCEEDED, +) -> StoredWindow: + """Insert a budget-window report as the service would, then end its runs.""" + years = tuple( + BudgetWindowYear( + year=year, + report_run_id=uuid4(), + baseline=_new_side(), + reform=_new_side(), + ) + for year in WINDOW_YEARS + ) + report = NewBudgetWindowReport( + id=uuid4(), + window_run_id=uuid4(), + country="us", + label="Ten-year cost", + description=None, + tax_benefit_model_id=economy.catalog.model_id, + population=economy.population(year=WINDOW_YEARS[0]), + versions=VERSIONS, + inputs=OPTIONS, + years=years, + ) + insert_budget_window_report(connection, report) + # Earlier than anything else the test's transaction creates, whose rows + # all get the transaction's time. + for table, condition in ( + ("budget_window_runs", "id = :window_run_id"), + ("report_runs", "budget_window_run_id = :window_run_id"), + ): + connection.execute( + text( + f"UPDATE public.{table} " # noqa: S608 # fixed table names + f"SET created_at = created_at - interval '1 hour' WHERE {condition}" + ), + {"window_run_id": report.window_run_id}, + ) + if window_status is not ReportRunStatus.PENDING: + ended = window_status in {ReportRunStatus.SUCCEEDED, ReportRunStatus.FAILED} + for table, condition in ( + ("budget_window_runs", "id = :window_run_id"), + ("report_runs", "budget_window_run_id = :window_run_id"), + ( + "simulation_runs", + "id IN (SELECT baseline_simulation_run_id FROM public.report_runs " + "WHERE budget_window_run_id = :window_run_id UNION " + "SELECT reform_simulation_run_id FROM public.report_runs " + "WHERE budget_window_run_id = :window_run_id)", + ), + ): + connection.execute( + text( + f"UPDATE public.{table} SET status = :status, " # noqa: S608 # fixed table names + "completed_at = CASE WHEN :ended THEN now() END " + f"WHERE {condition}" + ), + { + "status": window_status.value, + "ended": ended, + "window_run_id": report.window_run_id, + }, + ) + return StoredWindow( + report_id=report.id, window_run_id=report.window_run_id, years=years + ) + + +def _store_on(connection: Connection) -> EconomyRunStore: + """Return a store whose transactions are savepoints of ``connection``.""" + + @contextmanager + def savepoint() -> Iterator[Connection]: + with connection.begin_nested(): + yield connection + + return EconomyRunStore(savepoint) + + +def _row(connection: Connection, table_name: str, row_id: UUID) -> dict: + row = connection.execute( + text(f"SELECT * FROM public.{table_name} WHERE id = :id"), # noqa: S608 # fixed table names + {"id": row_id}, + ).mappings() + return dict(row.one()) + + +def _window_rerun_of( + stored: StoredWindow, later: EconomyCatalog, *, recalculate: bool = False +) -> NewWindowRerun: + return NewWindowRerun( + report_id=stored.report_id, + window_run_id=uuid4(), + population=later.population(year=WINDOW_YEARS[0]), + versions=LATER_VERSIONS, + recalculate=recalculate, + years=tuple( + WindowRerunYear( + year=window_year.year, + report_run_id=uuid4(), + baseline=RerunSide( + simulation_id=window_year.baseline.simulation_id, + simulation_run_id=uuid4(), + reused=False, + ), + reform=RerunSide( + simulation_id=window_year.reform.simulation_id, + simulation_run_id=uuid4(), + reused=False, + ), + ) + for window_year in stored.years + ), + ) + + +def test_a_budget_windows_definition_names_each_years_simulations( + public_api_database_url, +) -> None: + with rolled_back_transaction(public_api_database_url) as connection: + economy = insert_economy_catalog(connection) + stored = _stored_window(connection, economy) + act_as_the_service(connection) + + definition = read_report_definition(connection, stored.report_id) + + assert definition is not None + assert (definition.kind, definition.year, definition.budget_window_size) == ( + "budget_window", + 2026, + 3, + ) + assert definition.window_years == tuple( + WindowYearDefinition( + year=window_year.year, + baseline_simulation_id=window_year.baseline.simulation_id, + reform_simulation_id=window_year.reform.simulation_id, + ) + for window_year in stored.years + ) + + +def test_a_definitions_years_come_from_the_newest_window_run( + public_api_database_url, +) -> None: + """A year that reused another report's run names that report's simulation.""" + with rolled_back_transaction(public_api_database_url) as connection: + economy = insert_economy_catalog(connection) + later = insert_later_catalog(connection, economy) + stored = _stored_window(connection, economy) + other_2027 = _stored_window(connection, economy).years[1] + rerun = _window_rerun_of(stored, later) + years = list(rerun.years) + years[1] = replace( + years[1], + baseline=RerunSide( + simulation_id=other_2027.baseline.simulation_id, + simulation_run_id=other_2027.baseline.simulation_run_id, + reused=True, + ), + ) + act_as_the_service(connection) + insert_window_rerun(connection, replace(rerun, years=tuple(years))) + + definition = read_report_definition(connection, stored.report_id) + + assert definition is not None + assert [year.baseline_simulation_id for year in definition.window_years] == [ + stored.years[0].baseline.simulation_id, + other_2027.baseline.simulation_id, + stored.years[2].baseline.simulation_id, + ] + + +def test_a_window_rerun_creates_a_window_run_and_each_years_runs( + public_api_database_url, +) -> None: + with rolled_back_transaction(public_api_database_url) as connection: + economy = insert_economy_catalog(connection) + later = insert_later_catalog(connection, economy) + stored = _stored_window(connection, economy) + reused = insert_run_of( + connection, + stored.years[1].baseline.simulation_id, + later.population(year=2027), + RunShape(versions=LATER_VERSIONS), + ) + rerun = _window_rerun_of(stored, later, recalculate=True) + years = list(rerun.years) + years[1] = replace( + years[1], + baseline=RerunSide( + simulation_id=reused.simulation_id, + simulation_run_id=reused.simulation_run_id, + reused=True, + ), + ) + rerun = replace(rerun, years=tuple(years)) + act_as_the_service(connection) + + insert_window_rerun(connection, rerun) + window_run = _row(connection, "budget_window_runs", rerun.window_run_id) + year_runs = [ + _row(connection, "report_runs", year.report_run_id) for year in rerun.years + ] + reform_run_2028 = _row( + connection, "simulation_runs", rerun.years[2].reform.simulation_run_id + ) + earlier_window = _row(connection, "budget_window_runs", stored.window_run_id) + + assert { + key: window_run[key] + for key in ( + "report_id", + "tax_benefit_model_version_id", + "trigger", + "recalculate", + "status", + ) + } == { + "report_id": stored.report_id, + "tax_benefit_model_version_id": later.catalog.version_id, + "trigger": ReportRunTrigger.MANUAL, + "recalculate": True, + "status": ReportRunStatus.PENDING, + } + assert [ + ( + run["year"], + run["budget_window_run_id"], + run["trigger"], + run["status"], + run["policyengine_version"], + ) + for run in year_runs + ] == [ + ( + year, + rerun.window_run_id, + ReportRunTrigger.MANUAL, + ReportRunStatus.PENDING, + LATER_POLICYENGINE_VERSION, + ) + for year in WINDOW_YEARS + ] + assert year_runs[1]["baseline_simulation_run_id"] == reused.simulation_run_id + assert ( + reform_run_2028["simulation_id"], + reform_run_2028["year"], + reform_run_2028["tax_benefit_model_version_id"], + reform_run_2028["status"], + ) == ( + stored.years[2].reform.simulation_id, + 2028, + later.catalog.version_id, + SimulationStatus.PENDING, + ) + assert earlier_window["status"] == ReportRunStatus.SUCCEEDED + + +@pytest.mark.parametrize("status", [ReportRunStatus.PENDING, ReportRunStatus.RUNNING]) +def test_a_window_rerun_is_refused_while_a_window_run_is_under_way( + public_api_database_url, status: ReportRunStatus +) -> None: + with rolled_back_transaction(public_api_database_url) as connection: + economy = insert_economy_catalog(connection) + later = insert_later_catalog(connection, economy) + stored = _stored_window(connection, economy, window_status=status) + act_as_the_service(connection) + + with pytest.raises(RunInProgressError) as window_refused: + insert_window_rerun(connection, _window_rerun_of(stored, later)) + # A single-year re-run of the report checks the window runs too. + with pytest.raises(RunInProgressError) as report_refused: + insert_report_rerun( + connection, + NewReportRerun( + report_id=stored.report_id, + run_id=uuid4(), + population=later.population(), + versions=LATER_VERSIONS, + baseline=RerunSide( + simulation_id=stored.years[0].baseline.simulation_id, + simulation_run_id=uuid4(), + reused=False, + ), + reform=RerunSide( + simulation_id=stored.years[0].reform.simulation_id, + simulation_run_id=uuid4(), + reused=False, + ), + ), + ) + + assert window_refused.value.run_id == stored.window_run_id + assert report_refused.value.run_id == stored.window_run_id + + +def test_a_window_reruns_inputs_name_each_years_own_reusable_runs( + public_api_database_url, +) -> None: + with rolled_back_transaction(public_api_database_url) as connection: + economy = insert_economy_catalog(connection) + later = insert_later_catalog(connection, economy) + stored = _stored_window(connection, economy) + later_shape = RunShape(versions=LATER_VERSIONS) + reused_2027 = insert_run_of( + connection, + stored.years[1].baseline.simulation_id, + later.population(year=2027), + later_shape, + ) + failed_reform_2027 = insert_run_of( + connection, + stored.years[1].reform.simulation_id, + later.population(year=2027), + replace(later_shape, status=SimulationStatus.FAILED), + ) + insert_report_run_of( + connection, stored.report_id, (reused_2027, failed_reform_2027) + ) + act_as_the_service(connection) + years = _definition_years(stored) + + inputs = _store_on(connection).read_window_rerun_inputs( + LATER_REQUEST, + baseline=CURRENT_LAW, + reform=CURRENT_LAW, + years=years, + reuse=LATER_REUSE, + ) + forced = _store_on(connection).read_window_rerun_inputs( + LATER_REQUEST, + baseline=CURRENT_LAW, + reform=CURRENT_LAW, + years=years, + reuse=None, + ) + + assert inputs.catalog.catalog_version_id == later.catalog.version_id + assert inputs.reusable_runs == { + 2026: (None, None), + 2027: ( + ReusableRun( + simulation_id=stored.years[1].baseline.simulation_id, + simulation_run_id=reused_2027.simulation_run_id, + ), + None, + ), + 2028: (None, None), + } + assert forced.reusable_runs == dict.fromkeys(WINDOW_YEARS, (None, None)) + + +def _definition_years(stored: StoredWindow) -> tuple[WindowYearDefinition, ...]: + """Return the stored window's years as its definition names them.""" + return tuple( + WindowYearDefinition( + year=window_year.year, + baseline_simulation_id=window_year.baseline.simulation_id, + reform_simulation_id=window_year.reform.simulation_id, + ) + for window_year in stored.years + ) + + +def test_a_window_rerun_is_created_through_the_store(public_api_database_url) -> None: + with rolled_back_transaction(public_api_database_url) as connection: + economy = insert_economy_catalog(connection) + later = insert_later_catalog(connection, economy) + stored = _stored_window(connection, economy) + rerun = _window_rerun_of(stored, later) + act_as_the_service(connection) + store = _store_on(connection) + + store.create_window_rerun(rerun) + with pytest.raises(RunInProgressError) as refused: + store.create_window_rerun(_window_rerun_of(stored, later)) + + assert _row(connection, "budget_window_runs", rerun.window_run_id)[ + "trigger" + ] == (ReportRunTrigger.MANUAL) + assert refused.value.run_id == rerun.window_run_id + + +def test_a_window_rerun_refuses_years_that_are_not_its_window() -> None: + """Its years run one at a time from its first year, at most 75 of them.""" + year = WindowRerunYear( + year=2026, + report_run_id=uuid4(), + baseline=RerunSide( + simulation_id=uuid4(), simulation_run_id=uuid4(), reused=False + ), + reform=RerunSide( + simulation_id=uuid4(), simulation_run_id=uuid4(), reused=False + ), + ) + rerun = NewWindowRerun( + report_id=uuid4(), + window_run_id=uuid4(), + population=EconomyPopulation( + tax_benefit_model_version_id=uuid4(), + region_id=uuid4(), + dataset_id=uuid4(), + year=2026, + ), + versions=LATER_VERSIONS, + recalculate=False, + years=(year,), + ) + + for years in ( + (), + (replace(year, year=2027),), + (year, replace(year, year=2028)), + tuple(replace(year, year=2026 + offset) for offset in range(76)), + ): + with pytest.raises(ValueError, match="consecutive years"): + replace(rerun, years=years) + + +def test_a_year_naming_no_reform_is_left_out_of_a_windows_definition( + public_api_database_url, +) -> None: + with rolled_back_transaction(public_api_database_url) as connection: + economy = insert_economy_catalog(connection) + stored = _stored_window(connection, economy) + connection.execute( + text( + "UPDATE public.report_runs SET reform_simulation_run_id = NULL " + "WHERE id = :id" + ), + {"id": stored.years[1].report_run_id}, + ) + act_as_the_service(connection) + + definition = read_report_definition(connection, stored.report_id) + + assert definition is not None + assert [year.year for year in definition.window_years] == [2026, 2028] + + +def test_a_window_rerun_of_a_deleted_report_or_year_simulation_is_refused( + public_api_database_url, +) -> None: + with rolled_back_transaction(public_api_database_url) as connection: + economy = insert_economy_catalog(connection) + later = insert_later_catalog(connection, economy) + stored = _stored_window(connection, economy) + act_as_the_service(connection) + rerun = _window_rerun_of(stored, later) + gone_year = replace( + rerun.years[2], + reform=RerunSide( + simulation_id=uuid4(), simulation_run_id=uuid4(), reused=False + ), + ) + + with pytest.raises(RerunRecordDeletedError), connection.begin_nested(): + insert_window_rerun(connection, replace(rerun, report_id=uuid4())) + # A report deleted with a year's simulation: the window run is + # inserted first, so the report's own key refuses it. + with pytest.raises(RerunRecordDeletedError), connection.begin_nested(): + insert_window_rerun( + connection, + replace(rerun, report_id=uuid4(), years=(*rerun.years[:2], gone_year)), + ) + with ( + pytest.raises(EconomyRecordReferenceError) as refused, + connection.begin_nested(), + ): + insert_window_rerun( + connection, replace(rerun, years=(*rerun.years[:2], gone_year)) + ) + + assert (refused.value.table_name, refused.value.column_name) == ( + "simulation_runs", + "simulation_id", + ) + + +def test_a_window_rerun_is_refused_while_a_year_still_runs_after_its_window_ended( + public_api_database_url, +) -> None: + with rolled_back_transaction(public_api_database_url) as connection: + economy = insert_economy_catalog(connection) + later = insert_later_catalog(connection, economy) + stored = _stored_window( + connection, economy, window_status=ReportRunStatus.FAILED + ) + still_running = stored.years[2].report_run_id + connection.execute( + text( + "UPDATE public.report_runs SET status = 'running', " + "started_at = now(), completed_at = NULL WHERE id = :id" + ), + {"id": still_running}, + ) + act_as_the_service(connection) + + with pytest.raises(RunInProgressError) as refused: + insert_window_rerun(connection, _window_rerun_of(stored, later)) + + assert refused.value.run_id == still_running diff --git a/libs/policyengine-simulation-persistence/tests/test_household_report_reruns.py b/libs/policyengine-simulation-persistence/tests/test_household_report_reruns.py new file mode 100644 index 000000000..1a7c6c370 --- /dev/null +++ b/libs/policyengine-simulation-persistence/tests/test_household_report_reruns.py @@ -0,0 +1,421 @@ +"""Re-running a stored household report: what the re-run reads, and what it creates. + +A household report's re-run gives each side a new run of its own household +simulation, never reusing one, under a new report run with trigger +``manual``. These run against a database the Public API's own v2 migration +chain built (``PUBLIC_API_SCHEMA_DATABASE_URL``). The stored report and its +first runs are written as the superuser; the re-run's reads and writes run +as the simulation service's role, in a transaction the test rolls back. +""" + +from __future__ import annotations + +from contextlib import contextmanager +from dataclasses import replace +from typing import TYPE_CHECKING +from uuid import uuid4 + +import pytest +from conftest import act_as_the_service, rolled_back_transaction +from policyengine_simulation_persistence import ( + EconomyRecordOutcomeUnknownError, + EconomyRecordReferenceError, + HouseholdReportSimulation, + HouseholdReportStore, + HouseholdRerunSide, + NewHouseholdReport, + NewHouseholdReportRerun, + ReportRunStatus, + ReportRunTrigger, + RerunRecordDeletedError, + RunInProgressError, + RunVersions, + SideDefinition, + SimulationChoice, + SimulationStatus, + fail_unstarted_report_run, + insert_household_report, + read_report_definition, +) +from public_api_rows import ( + insert_catalog_version, + insert_household, + insert_model, + insert_policy, +) +from sqlalchemy import text +from sqlalchemy.exc import OperationalError + +if TYPE_CHECKING: + from collections.abc import Iterator + from uuid import UUID + + from public_api_rows import CatalogVersion + from sqlalchemy import Connection + +pytestmark = pytest.mark.public_api_schema + +VERSIONS = RunVersions(country_package_version="1.764.6", policyengine_version="6.17.1") +# A later release's versions, with a catalog version of its own. +LATER_VERSIONS = RunVersions( + country_package_version="1.770.0", policyengine_version="6.18.0" +) +CTC_AMOUNT = "gov.irs.credits.ctc.amount.base[0].amount" +DOCUMENT = { + "people": [ + { + "id": "you", + "values": {"age": 40}, + "memberships": {"household": "your household"}, + } + ], + "household": [{"id": "your household", "values": {}}], +} +# The year the stored report calculates, which its household's default year +# no longer is. +REPORT_YEAR = 2025 +CURRENT_LAW = SimulationChoice(policy_id=None, dynamic_id=None) + + +def _store_on(connection: Connection) -> HouseholdReportStore: + """Return a store whose transactions are savepoints of ``connection``.""" + + @contextmanager + def savepoint() -> Iterator[Connection]: + with connection.begin_nested(): + yield connection + + return HouseholdReportStore(savepoint) + + +def _catalogs(connection: Connection) -> tuple[CatalogVersion, CatalogVersion]: + """Insert the US model's catalog version for the report, and a later one.""" + model_id = insert_model(connection, "us") + + def catalog_for(versions: RunVersions) -> CatalogVersion: + return insert_catalog_version( + connection, + country="us", + model_id=model_id, + policyengine_version=versions.policyengine_version, + parameter_names=(CTC_AMOUNT,), + ) + + return catalog_for(VERSIONS), catalog_for(LATER_VERSIONS) + + +def _stored_report( + connection: Connection, + catalog: CatalogVersion, + *, + with_reform: bool = True, + first_run_ended: bool = True, +) -> NewHouseholdReport: + """Insert a household report and its first runs, as the superuser. + + With ``first_run_ended``, its first run failed before its call started, + so the report has no run under way. + """ + household_id = insert_household( + connection, country="us", default_year=2026, document=DOCUMENT + ) + report = NewHouseholdReport( + id=uuid4(), + run_id=uuid4(), + country="us", + label="A larger child tax credit", + description="For one family", + tax_benefit_model_id=catalog.model_id, + tax_benefit_model_version_id=catalog.version_id, + household_id=household_id, + year=REPORT_YEAR, + versions=VERSIONS, + inputs={}, + baseline=HouseholdReportSimulation( + simulation_id=uuid4(), simulation_run_id=uuid4(), policy_id=None + ), + reform=( + HouseholdReportSimulation( + simulation_id=uuid4(), + simulation_run_id=uuid4(), + policy_id=insert_policy(connection, catalog), + ) + if with_reform + else None + ), + ) + insert_household_report(connection, report) + if first_run_ended: + fail_unstarted_report_run( + connection, + report_run_id=report.run_id, + simulation_run_ids=report.simulation_run_ids, + error_message="run_not_started: no call was recorded", + ) + return report + + +def _rerun_of( + report: NewHouseholdReport, catalog: CatalogVersion +) -> NewHouseholdReportRerun: + """Return the report's re-run on ``catalog``: a new run of each side's own.""" + return NewHouseholdReportRerun( + report_id=report.id, + run_id=uuid4(), + tax_benefit_model_version_id=catalog.version_id, + year=report.year, + versions=LATER_VERSIONS, + baseline=HouseholdRerunSide( + simulation_id=report.baseline.simulation_id, simulation_run_id=uuid4() + ), + reform=( + None + if report.reform is None + else HouseholdRerunSide( + simulation_id=report.reform.simulation_id, simulation_run_id=uuid4() + ) + ), + ) + + +def _row(connection: Connection, table_name: str, row_id: UUID) -> dict: + return dict( + connection.execute( + text(f"SELECT * FROM public.{table_name} WHERE id = :id"), # noqa: S608 # fixed table names + {"id": row_id}, + ) + .mappings() + .one() + ) + + +def _count(connection: Connection, table_name: str, column: str, value: UUID) -> int: + return connection.execute( + text(f"SELECT count(*) FROM public.{table_name} WHERE {column} = :value"), # noqa: S608 # fixed table and column names + {"value": value}, + ).scalar_one() + + +def test_a_household_report_is_read_with_its_household_and_each_sides_policy( + public_api_database_url, +) -> None: + with rolled_back_transaction(public_api_database_url) as connection: + catalog, _ = _catalogs(connection) + report = _stored_report(connection, catalog) + baseline_alone = _stored_report(connection, catalog, with_reform=False) + act_as_the_service(connection) + + definition = read_report_definition(connection, report.id) + alone_definition = read_report_definition(connection, baseline_alone.id) + + assert definition is not None + assert alone_definition is not None + assert report.reform is not None + assert (definition.kind, definition.country, definition.year) == ( + "household", + "us", + REPORT_YEAR, + ) + assert definition.household_id == report.household_id + assert (definition.region_code, definition.dataset_name) == (None, None) + assert definition.inputs == {} + assert definition.baseline == SideDefinition( + simulation_id=report.baseline.simulation_id, choice=CURRENT_LAW + ) + assert definition.reform == SideDefinition( + simulation_id=report.reform.simulation_id, + choice=SimulationChoice(policy_id=report.reform.policy_id, dynamic_id=None), + ) + assert alone_definition.kind == "household" + assert alone_definition.household_id == baseline_alone.household_id + assert alone_definition.reform is None + + +def test_a_household_reports_rerun_is_a_manual_run_of_its_own_simulations( + public_api_database_url, +) -> None: + with rolled_back_transaction(public_api_database_url) as connection: + catalog, later_catalog = _catalogs(connection) + report = _stored_report(connection, catalog) + act_as_the_service(connection) + rerun = _rerun_of(report, later_catalog) + + _store_on(connection).create_report_rerun(rerun) + report_run = _row(connection, "report_runs", rerun.run_id) + side_runs = [ + _row(connection, "simulation_runs", side.simulation_run_id) + for side in rerun.sides + ] + stored_simulations = [ + _row(connection, "simulations", side.simulation_id) for side in rerun.sides + ] + first_run = _row(connection, "report_runs", report.run_id) + + assert rerun.simulation_run_ids == tuple( + side.simulation_run_id for side in rerun.sides + ) + assert report_run["report_id"] == report.id + assert report_run["trigger"] == ReportRunTrigger.MANUAL + assert report_run["status"] == ReportRunStatus.PENDING + assert report_run["modal_call_id"] is None + assert ( + report_run["baseline_simulation_run_id"], + report_run["reform_simulation_run_id"], + ) == rerun.simulation_run_ids + assert (report_run["budget_window_run_id"], report_run["year"]) == (None, None) + assert ( + report_run["country_package_version"], + report_run["policyengine_version"], + ) == (LATER_VERSIONS.country_package_version, LATER_VERSIONS.policyengine_version) + # The insert left both times to the server's defaults, so they are equal. + assert report_run["updated_at"] == report_run["created_at"] + for side, side_run in zip(rerun.sides, side_runs, strict=True): + assert side_run["simulation_id"] == side.simulation_id + assert side_run["status"] == SimulationStatus.PENDING + assert side_run["modal_call_id"] is None + assert side_run["year"] == REPORT_YEAR + assert side_run["tax_benefit_model_version_id"] == later_catalog.version_id + assert (side_run["dataset_id"], side_run["region_id"]) == (None, None) + assert ( + side_run["country_package_version"], + side_run["policyengine_version"], + ) == ( + LATER_VERSIONS.country_package_version, + LATER_VERSIONS.policyengine_version, + ) + assert side_run["updated_at"] == side_run["created_at"] + # The simulations keep the catalog version they were created on, and the + # report's first run is kept. + assert [ + simulation["tax_benefit_model_version_id"] for simulation in stored_simulations + ] == [catalog.version_id, catalog.version_id] + assert first_run["status"] == ReportRunStatus.FAILED + + +def test_a_rerun_of_a_report_of_the_baseline_alone_names_no_reform_run( + public_api_database_url, +) -> None: + with rolled_back_transaction(public_api_database_url) as connection: + catalog, later_catalog = _catalogs(connection) + report = _stored_report(connection, catalog, with_reform=False) + act_as_the_service(connection) + rerun = _rerun_of(report, later_catalog) + + _store_on(connection).create_report_rerun(rerun) + report_run = _row(connection, "report_runs", rerun.run_id) + baseline_runs = _count( + connection, + "simulation_runs", + "simulation_id", + report.baseline.simulation_id, + ) + + assert rerun.simulation_run_ids == (rerun.baseline.simulation_run_id,) + assert report_run["baseline_simulation_run_id"] == rerun.baseline.simulation_run_id + assert report_run["reform_simulation_run_id"] is None + assert baseline_runs == 2 + + +def test_a_household_report_with_a_run_under_way_is_not_rerun( + public_api_database_url, +) -> None: + with rolled_back_transaction(public_api_database_url) as connection: + catalog, later_catalog = _catalogs(connection) + report = _stored_report(connection, catalog, first_run_ended=False) + act_as_the_service(connection) + + with pytest.raises(RunInProgressError) as under_way: + _store_on(connection).create_report_rerun(_rerun_of(report, later_catalog)) + report_runs = _count(connection, "report_runs", "report_id", report.id) + baseline_runs = _count( + connection, + "simulation_runs", + "simulation_id", + report.baseline.simulation_id, + ) + + assert under_way.value.run_id == report.run_id + assert (report_runs, baseline_runs) == (1, 1) + + +def test_a_rerun_naming_a_deleted_report_or_simulation_is_refused_by_its_key( + public_api_database_url, +) -> None: + with rolled_back_transaction(public_api_database_url) as connection: + catalog, later_catalog = _catalogs(connection) + report = _stored_report(connection, catalog) + act_as_the_service(connection) + store = _store_on(connection) + rerun = _rerun_of(report, later_catalog) + + # A report deleted after the re-run read it, its simulations kept. + with pytest.raises(RerunRecordDeletedError): + store.create_report_rerun(replace(rerun, report_id=uuid4())) + # A side's simulation deleted after the re-run read it. + with pytest.raises(EconomyRecordReferenceError) as refused: + store.create_report_rerun( + replace( + rerun, + reform=HouseholdRerunSide( + simulation_id=uuid4(), simulation_run_id=uuid4() + ), + ) + ) + report_runs = _count(connection, "report_runs", "report_id", report.id) + + assert refused.value.column_name == "simulation_id" + assert report_runs == 1 + + +class _CommitFails: + """A transaction whose block runs, and whose commit then fails.""" + + def __init__(self, connection) -> None: + self.connection = connection + + def __enter__(self): + return self.connection + + def __exit__(self, error_type, error, traceback) -> None: + if error is None: + statement = "COMMIT" + raise OperationalError(statement, {}, Exception("connection lost")) + + +class _RecordingConnection: + """Records each statement it is given, and answers each with no rows.""" + + def __init__(self) -> None: + self.statements: list[object] = [] + + def execute(self, statement, *arguments): + self.statements.append(statement) + return _NoRows() + + +class _NoRows: + def scalar_one_or_none(self) -> None: + return None + + +def test_a_rerun_whose_commit_fails_may_have_been_written() -> None: + connection = _RecordingConnection() + store = HouseholdReportStore(lambda: _CommitFails(connection)) + + with pytest.raises(EconomyRecordOutcomeUnknownError) as raised: + store.create_report_rerun( + NewHouseholdReportRerun( + report_id=uuid4(), + run_id=uuid4(), + tax_benefit_model_version_id=uuid4(), + year=REPORT_YEAR, + versions=LATER_VERSIONS, + baseline=HouseholdRerunSide( + simulation_id=uuid4(), simulation_run_id=uuid4() + ), + reform=None, + ) + ) + + assert isinstance(raised.value.__cause__, OperationalError) + assert connection.statements diff --git a/libs/policyengine-stage12-persistence/tests/budget_window_database.py b/libs/policyengine-stage12-persistence/tests/budget_window_database.py index b8548feb6..0313febe7 100644 --- a/libs/policyengine-stage12-persistence/tests/budget_window_database.py +++ b/libs/policyengine-stage12-persistence/tests/budget_window_database.py @@ -10,6 +10,7 @@ from __future__ import annotations from dataclasses import dataclass +from datetime import timedelta from typing import TYPE_CHECKING, Any from uuid import UUID, uuid4 @@ -147,6 +148,86 @@ def seed_window(self, first_year: int = 2026, size: int = 7) -> SeededWindow: model_id=model_id, ) + def seed_window_rerun(self, window: SeededWindow) -> SeededWindow: + """Insert a later pending window run of ``window``'s report, as a re-run. + + Each year gets a new report run and two new runs of the report's + simulations; the earlier window run and its years are left as they are. + """ + table = self.seeder.table + with self.engine.connect() as connection: + first_run = connection.execute( + select(table("simulation_runs")).where( + table("simulation_runs").c.id + == window.years[0].baseline_simulation_run_id + ) + ).one() + window_run_id = uuid4() + years = tuple( + SeededYear( + year=seeded.year, + report_run_id=uuid4(), + baseline_simulation_run_id=uuid4(), + reform_simulation_run_id=uuid4(), + ) + for seeded in window.years + ) + later = {"created_at": NOW + timedelta(minutes=1), "updated_at": NOW} + with self.engine.begin() as connection: + connection.execute( + insert(table("budget_window_runs")).values( + id=window_run_id, + report_id=window.report_id, + tax_benefit_model_version_id=first_run.tax_benefit_model_version_id, + trigger="manual", + recalculate=True, + status="pending", + **later, + ) + ) + for seeded in years: + for run_id, simulation_id in zip( + ( + seeded.baseline_simulation_run_id, + seeded.reform_simulation_run_id, + ), + window.simulation_ids, + strict=True, + ): + connection.execute( + insert(table("simulation_runs")).values( + id=run_id, + simulation_id=simulation_id, + year=seeded.year, + tax_benefit_model_version_id=first_run.tax_benefit_model_version_id, + dataset_id=first_run.dataset_id, + status="pending", + **later, + **VERSIONS, + ) + ) + connection.execute( + insert(table("report_runs")).values( + id=seeded.report_run_id, + report_id=window.report_id, + baseline_simulation_run_id=seeded.baseline_simulation_run_id, + reform_simulation_run_id=seeded.reform_simulation_run_id, + budget_window_run_id=window_run_id, + year=seeded.year, + status="pending", + trigger="manual", + **later, + **VERSIONS, + ) + ) + return SeededWindow( + window_run_id=window_run_id, + report_id=window.report_id, + years=years, + simulation_ids=window.simulation_ids, + model_id=window.model_id, + ) + def set_years( self, window: SeededWindow, years: Iterable[int], **values: Any ) -> None: # column values of any type diff --git a/libs/policyengine-stage12-persistence/tests/test_budget_window_store.py b/libs/policyengine-stage12-persistence/tests/test_budget_window_store.py index d7b6b8bb7..4c3f2cad4 100644 --- a/libs/policyengine-stage12-persistence/tests/test_budget_window_store.py +++ b/libs/policyengine-stage12-persistence/tests/test_budget_window_store.py @@ -641,3 +641,25 @@ def test_the_sweep_lists_the_waiting_years_of_a_finished_window( listed = _listed_report_runs(run_store) assert set(_run_ids(window, ALL_YEARS)) <= set(listed) + + +def test_a_reports_later_window_run_starts_only_its_own_years( + store, windows, window +) -> None: + """A re-run's window run of the same report and simulations advances alone.""" + _succeeded(windows, window, ALL_YEARS) + windows.set_window(window, status="succeeded", started_at=NOW, completed_at=NOW) + rerun = windows.seed_window_rerun(window) + starter = RecordingStarter() + + advance = store.advance_window(rerun.window_run_id, now=LATER, start_year=starter) + + assert starter.started == _run_ids(rerun, range(2026, 2031)) + assert advance.started_years == (2026, 2027, 2028, 2029, 2030) + assert windows.window_row(window)["status"] == "succeeded" + assert {row["status"] for row in windows.year_rows(window).values()} == { + "succeeded" + } + assert {row["modal_call_id"] for row in windows.year_rows(window).values()} == { + "fc-finished" + } diff --git a/projects/policyengine-apis-integ/tests/test_economy_generated_client.py b/projects/policyengine-apis-integ/tests/test_economy_generated_client.py index 5f32692db..0f58c439f 100644 --- a/projects/policyengine-apis-integ/tests/test_economy_generated_client.py +++ b/projects/policyengine-apis-integ/tests/test_economy_generated_client.py @@ -14,8 +14,11 @@ create_simulation, get_report, get_simulation, + rerun_report, + rerun_simulation, ) from policyengine_api_simulation_client.models import ( + BudgetWindowReportCreated, EconomyError, EconomyReportCreated, EconomyReportRequest, @@ -24,6 +27,7 @@ EconomySimulationCreated, EconomySimulationRequest, EconomySimulationState, + ReportRerunRequest, ReportRunStatus, ) @@ -194,3 +198,109 @@ def test_a_report_and_a_simulation_are_read_with_their_newest_runs(): assert run.status is ReportRunStatus.SUCCEEDED assert isinstance(simulation_response.parsed, EconomySimulationState) assert simulation_response.parsed.to_dict() == simulation + + +def test_a_report_rerun_is_sent_with_force_and_read_with_its_new_run(): + sent: list[httpx.Request] = [] + + def answer(request: httpx.Request) -> httpx.Response: + sent.append(request) + return httpx.Response(201, json=CREATED_REPORT) + + client = AuthenticatedClient(base_url="https://simulation.example", token="test") + client.set_httpx_client( + httpx.Client( + base_url="https://simulation.example", + transport=httpx.MockTransport(answer), + ) + ) + + response = rerun_report.sync_detailed( + UUID(REPORT_ID), client=client, body=ReportRerunRequest(force=True) + ) + + (request,) = sent + assert request.url.path == f"/reports/{REPORT_ID}/runs" + assert request.read() == b'{"force":true}' + assert isinstance(response.parsed, EconomyReportCreated) + assert response.parsed.to_dict() == CREATED_REPORT + + +def test_a_simulation_rerun_is_read_with_its_new_run(): + created = { + "simulation_id": REPORT_ID, + "simulation_run_id": RUN_ID, + "status": "pending", + } + + response = rerun_simulation.sync_detailed( + UUID(REPORT_ID), client=_client_answering(201, created) + ) + + assert isinstance(response.parsed, EconomySimulationCreated) + assert response.parsed.to_dict() == created + + +@pytest.mark.parametrize( + ("status_code", "body"), + [ + (404, {"code": "unknown_report", "detail": f"There is no report {REPORT_ID}."}), + ( + 409, + { + "code": "run_in_progress", + "detail": "The report has a run pending or running.", + "run_id": RUN_ID, + }, + ), + (409, {"code": "rerun_not_supported", "detail": "Not yet."}), + ( + 503, + { + "code": "modal_call_refused", + "detail": "Modal did not accept the call of the report's new run.", + "report_id": REPORT_ID, + "run_id": RUN_ID, + }, + ), + ], +) +def test_a_refused_rerun_is_read_with_its_code_and_any_run(status_code, body): + response = rerun_report.sync_detailed( + UUID(REPORT_ID), + client=_client_answering(status_code, body), + body=ReportRerunRequest(), + ) + + assert isinstance(response.parsed, EconomyError) + assert response.parsed.to_dict() == body + + +def test_a_budget_window_rerun_is_read_with_its_new_window_run(): + window = { + "report_id": REPORT_ID, + "report_run_id": RUN_ID, + "budget_window_run_id": SIDE_IDS[0], + "status": "pending", + "years": [ + { + "year": 2026, + "report_run_id": RUN_ID, + "baseline_simulation_id": SIDE_IDS[1], + "baseline_simulation_run_id": SIDE_IDS[2], + "baseline_reused": False, + "reform_simulation_id": SIDE_IDS[3], + "reform_simulation_run_id": SIDE_IDS[0], + "reform_reused": True, + } + ], + } + + response = rerun_report.sync_detailed( + UUID(REPORT_ID), + client=_client_answering(201, window), + body=ReportRerunRequest(), + ) + + assert isinstance(response.parsed, BudgetWindowReportCreated) + assert response.parsed.to_dict() == window diff --git a/projects/policyengine-apis-integ/tests/test_household_report_generated_client.py b/projects/policyengine-apis-integ/tests/test_household_report_generated_client.py index f341d7c8b..73fa1684e 100644 --- a/projects/policyengine-apis-integ/tests/test_household_report_generated_client.py +++ b/projects/policyengine-apis-integ/tests/test_household_report_generated_client.py @@ -3,18 +3,22 @@ The Public API sends a household report to ``POST /reports`` with ``report_type`` ``household``, and reads the 201 naming the report, its run, its year and each side's simulation and simulation run, or the error -answers economy reports share. +answers economy reports share. A household report's re-run +(``POST /reports/{report_id}/runs``) answers with the same 201. """ +from uuid import UUID + import httpx from policyengine_api_simulation_client import AuthenticatedClient -from policyengine_api_simulation_client.api.default import create_report +from policyengine_api_simulation_client.api.default import create_report, rerun_report from policyengine_api_simulation_client.models import ( EconomyError, EconomyReportCreated, EconomyRequestUnprocessable, HouseholdReportCreated, HouseholdReportRequest, + ReportRerunRequest, ReportRunStatus, ) @@ -114,6 +118,25 @@ def test_a_created_report_of_the_baseline_alone_has_no_reform(): assert response.parsed.reform_simulation_run_id is None +def test_a_household_report_rerun_is_read_with_its_new_run(): + client = AuthenticatedClient(base_url="https://simulation.example", token="test") + client.set_httpx_client( + httpx.Client( + base_url="https://simulation.example", + transport=httpx.MockTransport( + lambda _: httpx.Response(201, json=CREATED_HOUSEHOLD_REPORT) + ), + ) + ) + + response = rerun_report.sync_detailed( + UUID(REPORT_ID), client=client, body=ReportRerunRequest() + ) + + assert isinstance(response.parsed, HouseholdReportCreated) + assert response.parsed.to_dict() == CREATED_HOUSEHOLD_REPORT + + def test_a_created_economy_report_is_still_read_as_one(): response, _ = _create(201, CREATED_ECONOMY_REPORT) diff --git a/projects/policyengine-simulation-entry/README.md b/projects/policyengine-simulation-entry/README.md index 0d1408c33..1b8fce00b 100644 --- a/projects/policyengine-simulation-entry/README.md +++ b/projects/policyengine-simulation-entry/README.md @@ -44,8 +44,10 @@ Auth0 permission: | `GET /household-calculations/{household_job_id}` | `simulation:read` | | `POST /reports` | `simulation:submit` | | `GET /reports/{report_id}` | `simulation:read` | +| `POST /reports/{report_id}/runs` | `simulation:submit` | | `POST /simulations` | `simulation:submit` | | `GET /simulations/{simulation_id}` | `simulation:read` | +| `POST /simulations/{simulation_id}/runs` | `simulation:submit` | `GET /health`, `GET /ready`, `GET /versions`, `GET /versions/{kind}` and `POST /ping` stay public. The entrypoint answers `GET /health` and `POST /ping` @@ -225,3 +227,68 @@ refuses what the report would calculate, located at the field it traces to: A policy setting parameters the catalog version lacks is 422 `missing_parameters`, located at that side's `policy_id`. +## Re-running a report or simulation + +`POST /reports/{report_id}/runs` calculates a stored economy report again, +as a new run on the default worker's versions, and keeps every earlier run. +Its body is required: `{}` or `{"force": false}` re-runs the report, and +`{"force": true}` forces a recalculation. The report keeps its simulations: +each side gets a new run of its own simulation, or, without `force`, reuses +that simulation's newest run on the same versions, catalog version, dataset, +region and year that a new report's side could reuse; `force` calculates +both again. The new report run's trigger is `manual`. The service reads the +report's region code and dataset name, finds the rows with that code and +name in the catalog version of the default worker's .py version, and reads +each simulation's stored policy and dynamic on it. +A budget-window report's re-run is a new budget-window run, trigger +`manual`, with `recalculate` set by `force`, and a report run per year. Each +year's sides are the simulations that year's report run of the report's +newest window run names (a year that reused another report's run names that +report's simulation), and each reuses a run of them as a single-year side +does. Up to five years start at once, as a new window's do (a later year +Modal doesn't confirm in time waits for an earlier year's coordinator); it +answers 201 with the budget-window create's fields, naming the window run +and every year. +A household report's re-run calculates its stored household again in the +report's own year (not the household's current `default_year`), under each +side's stored policy read on the current catalog version. Each side gets a +new run of its own household simulation, recording that catalog version: +household runs are never reused, so `force` changes nothing. The new report +run (trigger `manual`) is started by one household function call, as a new +household report's is, and the re-run answers 201 with the household report +create's fields. +`POST /simulations/{simulation_id}/runs`, with no body, creates and starts +a stored economy simulation's new run, which is always calculated. Each +answers 201 with the same fields as the create, naming the new run. + +A re-run is refused before anything is written: 404 (`unknown_report`, +`unknown_simulation`) for a record that doesn't exist; 409 +`run_in_progress`, naming the run in `run_id`, while the record has a run +pending or running (for a report, its budget-window run, or else its report +run); 409 `rerun_not_supported` for a household simulation (re-run only +through its report), a household report whose simulation names a dynamic (a +household calculation applies none), a budget window whose newest window run +lacks a year, or a record that names no region, dataset, year, simulation, +household or country the service calculates (a household report names no +region or dataset); and the create's 400 and 422 codes when the stored rows +can't be calculated on the current catalog version, worded for the stored +record (a 422 reason's `loc` starts with `report` or `simulation`), with a +household report's own codes: 400 `unknown_household` and +`no_household_function`, and 422 `household_not_calculable`, located at the +report's `household_id` or a side's `policy_id`. Without household reports +configured, a household report the service could re-run answers 503 +`not_configured`. The check for a run under way and the inserts run under a +transaction-level advisory lock on the record's ID, so two re-runs of one +record never both create a run. A report's re-run checks only the report's +own runs: a side whose simulation has a run under way for another report, or +for the simulation's own re-run, still gets a new run, calculated +separately. A simulation's re-run checks every run of the simulation, +including those reports created. A report or simulation deleted after the +re-run read it answers 404. A report deleted with its simulations answers +400 `missing_reference`, naming a simulation, when a side gets a new run, as +a household report's sides always do, and 404 when both sides reuse a run; a +deleted budget-window report always answers 404, as its window run is +created first. When Modal refuses the new run's call, the run is failed, +with a household report's new simulation runs, and the 503 +`modal_call_refused` names the record and the run (`run_id`). + diff --git a/projects/policyengine-simulation-entry/src/policyengine_simulation_entry/app.py b/projects/policyengine-simulation-entry/src/policyengine_simulation_entry/app.py index c14a96dc5..614c48111 100644 --- a/projects/policyengine-simulation-entry/src/policyengine_simulation_entry/app.py +++ b/projects/policyengine-simulation-entry/src/policyengine_simulation_entry/app.py @@ -52,18 +52,17 @@ CallerAuthorizer, ) from policyengine_simulation_entry.backend import OldGatewayBackend, SimulationBackend -from policyengine_simulation_entry.budget_window_submissions import ( - ReportSubmissionService, -) from policyengine_simulation_entry.config import Settings +from policyengine_simulation_entry.economy_rerun_routes import ( + EconomyService, + add_economy_rerun_routes, +) +from policyengine_simulation_entry.economy_reruns import EconomyRerunService from policyengine_simulation_entry.economy_routes import ( - EconomySubmissions, HouseholdReportSubmissions, add_economy_routes, ) -from policyengine_simulation_entry.economy_submissions import ( - ServiceSettings, -) +from policyengine_simulation_entry.economy_submissions import ServiceSettings from policyengine_simulation_entry.economy_workers import ( ManifestEconomyWorkers, ModalEconomyRunCaller, @@ -84,7 +83,8 @@ from policyengine_simulation_entry.household_calculations import ( HouseholdCalculationService, ) -from policyengine_simulation_entry.household_report_submissions import ( +from policyengine_simulation_entry.household_report_reruns import ( + HouseholdReportReruns, household_reports_for, ) from policyengine_simulation_entry.household_routes import ( @@ -547,13 +547,13 @@ def household_calculations_for( def _register_economy_routes( # noqa: PLR0913 # the app, its services and permissions app: FastAPI, entry: EntryRuntime, - submissions: EconomySubmissions | None, + submissions: EconomyService | None, household_reports: HouseholdReportSubmissions | None, *, submitter: AuthorizationDependency, reader: AuthorizationDependency, ) -> None: - """Add the routes that create reports and economy simulations and read them.""" + """Add the routes that create, read and re-run reports and economy simulations.""" add_economy_routes( app, entry, @@ -562,22 +562,29 @@ def _register_economy_routes( # noqa: PLR0913 # the app, its services and perm reader=reader, household_reports=household_reports, ) + add_economy_rerun_routes(app, entry, submissions, submitter=submitter) def economy_submissions_for( - settings: Settings, database: SimulationDatabaseConnection | None -) -> EconomySubmissions | None: + settings: Settings, + database: SimulationDatabaseConnection | None, + *, + household_reports: HouseholdReportReruns | None = None, +) -> EconomyService | None: """Build the economy submission service when the service is configured. It needs the simulation database and the v2 Modal environment; without them the economy routes answer 503. Nothing is read from Modal until the - first submission. Without the output bucket, no report reuses a run. + first submission. Without the output bucket, no report reuses a run. It + re-runs a household report through ``household_reports``; without it, a + household report it could re-run answers 503. """ if not settings.economy_runs_configured or not isinstance( database, SimulationDatabase ): return None - return ReportSubmissionService( + return EconomyRerunService( + household_reports=household_reports, # The service starts a budget window's first years as its own role. windows=BudgetWindowStore(engine=database.engine), store=EconomyRunStore.from_engine( @@ -752,11 +759,15 @@ async def lifespan(_: FastAPI) -> AsyncIterator[None]: submitter=simulation_submitter, reader=simulation_reader, ) + # One household report service creates household reports and re-runs them. + household_reports = household_reports_for(runtime_settings, runtime_database) _register_economy_routes( app, entry, - economy_submissions_for(runtime_settings, runtime_database), - household_reports_for(runtime_settings, runtime_database), + economy_submissions_for( + runtime_settings, runtime_database, household_reports=household_reports + ), + household_reports, submitter=simulation_submitter, reader=simulation_reader, ) diff --git a/projects/policyengine-simulation-entry/src/policyengine_simulation_entry/budget_window_submissions.py b/projects/policyengine-simulation-entry/src/policyengine_simulation_entry/budget_window_submissions.py index 748905f3c..ca120591f 100644 --- a/projects/policyengine-simulation-entry/src/policyengine_simulation_entry/budget_window_submissions.py +++ b/projects/policyengine-simulation-entry/src/policyengine_simulation_entry/budget_window_submissions.py @@ -49,13 +49,13 @@ from policyengine_simulation_entry.economy_calculation_inputs import ( report_run_calculation, ) +from policyengine_simulation_entry.economy_submission_errors import call_refused from policyengine_simulation_entry.economy_submissions import ( CALL_START_TIMEOUT_SECONDS, EconomyRuns, EconomySubmissionService, ServiceSettings, calculation_scope, - call_refused, economy_request_for, simulation_choice, ) @@ -68,6 +68,7 @@ EconomyRequest, ReusableRun, SimulationInputs, + WindowYearRuns, ) from policyengine_stage12_persistence import ( WindowAdvance, @@ -124,14 +125,19 @@ def advance_window( ... -class ReportSubmissionService(EconomySubmissionService): - """Create economy and budget-window reports and simulations, and start them.""" +class ReportSubmissionService[Store: BudgetWindowRuns = BudgetWindowRuns]( + EconomySubmissionService[Store] +): + """Create economy and budget-window reports and simulations, and start them. + + ``Store`` is the store's type, as for ``EconomySubmissionService``. + """ def __init__( # noqa: PLR0913 # the window seams, then the economy service's self, *, windows: BudgetWindowAdvancer, - store: BudgetWindowRuns, + store: Store, workers: EconomyWorkerSource, caller: EconomyRunCaller, settings: ServiceSettings, @@ -150,7 +156,6 @@ def __init__( # noqa: PLR0913 # the window seams, then the economy service's clock=clock, ) self._windows = windows - self._window_store = store async def submit_budget_window( self, @@ -178,7 +183,7 @@ async def submit_budget_window( ) economy_request = economy_request_for(request, worker) inputs = await self._read_inputs( - lambda: self._window_store.read_budget_window_inputs( + lambda: self._store.read_budget_window_inputs( economy_request, baseline=simulation_choice(request.baseline), reform=simulation_choice(request.reform), @@ -204,17 +209,19 @@ async def submit_budget_window( worker, calculation_scope(request, worker, inputs.catalog, spm=options.get("spm")), self._origin(request_id), - run_ids=_run_ids(first_year), + run_ids=year_run_ids(first_year), baseline_policy=inputs.baseline.reform, reform_policy=inputs.reform.reform, ) await self._create( - "report", lambda: self._window_store.create_budget_window_report(report) + "report", lambda: self._store.create_budget_window_report(report) ) status = await self._start_first_years( worker, - report, first_calculation, + report_id=report.id, + window_run_id=report.window_run_id, + first_year=first_year.year, observability_context=observability_context, ) return BudgetWindowReportCreated( @@ -222,7 +229,7 @@ async def submit_budget_window( report_run_id=first_year.report_run_id, budget_window_run_id=report.window_run_id, status=status, - years=[_year_created(window_year) for window_year in report.years], + years=[year_created(window_year) for window_year in report.years], ) def _window_year(self, inputs: BudgetWindowInputs, year: int) -> BudgetWindowYear: @@ -235,15 +242,18 @@ def _window_year(self, inputs: BudgetWindowInputs, year: int) -> BudgetWindowYea reform=self._side(_with_reusable(inputs.reform, reform_run)), ) - async def _start_first_years( + async def _start_first_years( # noqa: PLR0913 # the call, then the window it starts self, worker: EconomyWorker, - report: NewBudgetWindowReport, first_calculation: RunCalculation, *, + report_id: UUID, + window_run_id: UUID, + first_year: int, observability_context: dict[str, Any] | None, + rerun: bool = False, ) -> ReportRunStatus: - """Advance the new window, starting its first years; return its status. + """Advance the new window run, starting its first years; return its status. Raise EconomySubmissionUnavailableError naming the report when the window failed because Modal refused a year, or didn't confirm the @@ -251,12 +261,13 @@ async def _start_first_years( left waiting. An advance that can't be written leaves the window pending: the report exists, so it is answered as created, and a year Modal started claims itself, while the stuck-run sweep fails a window no - year advances. + year advances. ``rerun`` says the report existed before the window + run, which the 503 then names. """ start_year = _YearStarts( worker=worker, caller=self._caller, - first_year=report.first_year.year, + first_year=first_year, first_calculation=first_calculation, observability_context=observability_context, loop=asyncio.get_running_loop(), @@ -266,7 +277,7 @@ async def _start_first_years( try: advance = await asyncio.to_thread( self._windows.advance_window, - report.window_run_id, + window_run_id, now=self._clock(), start_year=start_year, ) @@ -275,7 +286,7 @@ async def _start_first_years( "budget_window_not_started", extra={ "record": "report", - "record_id": str(report.id), + "record_id": str(report_id), "error_type": type(error).__name__, }, ) @@ -286,11 +297,13 @@ async def _start_first_years( "budget_window_call_unconfirmed", extra={ "record": "report", - "record_id": str(report.id), + "record_id": str(report_id), "started_years": list(advance.started_years), }, ) - refused = call_refused("report", report.id) + refused = call_refused( + "report", report_id, rerun_id=window_run_id if rerun else None + ) raise refused from None return ReportRunStatus(advance.status.value) @@ -398,7 +411,8 @@ def _deferral(year_to_start: YearToStart) -> dict[str, Any]: } -def _run_ids(window_year: BudgetWindowYear) -> ReportRunIds: +def year_run_ids(window_year: WindowYearRuns) -> ReportRunIds: + """Return a window year's report run and its sides' runs, as its call names them.""" return ReportRunIds( report_run_id=window_year.report_run_id, baseline_simulation_run_id=window_year.baseline.simulation_run_id, @@ -406,7 +420,8 @@ def _run_ids(window_year: BudgetWindowYear) -> ReportRunIds: ) -def _year_created(window_year: BudgetWindowYear) -> BudgetWindowYearCreated: +def year_created(window_year: WindowYearRuns) -> BudgetWindowYearCreated: + """Return a new window's year, or a re-run's, as the answer lists it.""" return BudgetWindowYearCreated( year=window_year.year, report_run_id=window_year.report_run_id, diff --git a/projects/policyengine-simulation-entry/src/policyengine_simulation_entry/economy_api_models.py b/projects/policyengine-simulation-entry/src/policyengine_simulation_entry/economy_api_models.py index cd0e71b0a..8ab9bf797 100644 --- a/projects/policyengine-simulation-entry/src/policyengine_simulation_entry/economy_api_models.py +++ b/projects/policyengine-simulation-entry/src/policyengine_simulation_entry/economy_api_models.py @@ -9,16 +9,25 @@ Answers the Public API reads: -- 201 with the created record's ID and its first run's ID; +- 201 with the created record's ID and its first run's ID, or, for a + re-run, the record's ID and its new run's ID; - 400 with ``code`` and ``detail``: one of ``CallerRowRefusalCode`` when - the region, dataset or a dynamic the request names does not exist, or - another code for the service's own state, such as a catalog version - missing while a release deploys. Nothing is written; + the region, dataset or a dynamic the request names (or, for a re-run, the + stored record names) does not exist, or another code for the service's own + state, such as a catalog version missing while a release deploys. Nothing + is written; +- 404 ``unknown_report`` or ``unknown_simulation`` when a re-run names no + stored record; +- 409 with ``code`` and ``detail`` when a stored record is not re-run now: + ``run_in_progress``, naming the run pending or running (``run_id``), or + ``rerun_not_supported`` for a record this service does not re-run. + Nothing is written; - 422 when the request's fields, or the stored values it names, cannot be calculated. Nothing is written; - 503 with ``code`` and ``detail`` when the service cannot answer now. Only ``modal_call_refused`` follows a write, and it names the record created - (``report_id`` or ``simulation_id``), whose run was failed; + or re-run (``report_id`` or ``simulation_id``), whose new run was failed, + and for a re-run that run (``run_id``); - 500 with ``code`` ``internal_error``, possibly after a write: an unexpected failure, or a commit that failed after the rows were inserted. @@ -45,10 +54,13 @@ ReportDescription = Annotated[str, Field(max_length=10_000)] # Why a request is refused (400), naming a row the caller chose that doesn't -# exist: the Public API answers these as its caller's error. +# exist: the Public API answers these as its caller's error. On a re-run they +# name the stored record's own region, dataset or dynamic, which the current +# catalog version lacks or which was deleted. CallerRowRefusalCode = Literal["unknown_region", "unknown_dataset", "unknown_dynamic"] # Why a request is refused (400) for the service's own state, or a row the -# Public API checks before asking. +# Public API checks before asking. On a report's re-run, ``unknown_policy`` and +# ``missing_reference`` name the stored report's own policy or simulation. ServiceStateRefusalCode = Literal[ # The country has no catalog version for the default worker's .py version. "unknown_catalog_version", @@ -74,6 +86,16 @@ # would calculate: located at the household or a side's policy. "household_not_calculable", ] +# Why a stored record is not re-run now (409). +EconomyConflictCode = Literal[ + # The record has a run pending or running; ``run_id`` names it: for a + # report, its budget-window run, or else its report run; for a + # simulation, its simulation run. + "run_in_progress", + # The service does not re-run this kind of record, or the record lacks a + # row it calculates on. + "rerun_not_supported", +] # Why a route cannot answer now (503); retry after ``Retry-After`` seconds. EconomyUnavailableCode = Literal[ # The service has no database or v2 Modal environment configured. @@ -88,6 +110,7 @@ EconomyErrorCode = Literal[ EconomyRefusalCode, EconomyUnprocessableCode, + EconomyConflictCode, EconomyUnavailableCode, "internal_error", "unknown_report", @@ -172,12 +195,23 @@ def options(self) -> dict[str, Any]: return options +class ReportRerunRequest(_Request): + """A stored report's re-run on the default worker's versions. + + Without ``force``, a side reuses a run of its own simulation that + succeeded on those versions, as a new report's side does; with it, both + simulations are calculated again. + """ + + force: StrictBool = False + + class _Answer(BaseModel): model_config = ConfigDict(extra="forbid", frozen=True) class EconomySimulationCreated(_Answer): - """The simulation a request created, its first run and that run's status. + """The simulation a request created or re-ran, its new run and its status. The status is the run's when its call's ID was recorded: ``pending``, or whatever the simulation function had made it by then if it claimed the @@ -191,7 +225,7 @@ class EconomySimulationCreated(_Answer): class EconomyReportCreated(_Answer): - """The report a request created, its first run and that run's status. + """The report a request created or re-ran, its new run and its status. The status is the run's when its call's ID was recorded: ``pending``, or whatever the coordinator had made it by then if it claimed the run @@ -267,14 +301,19 @@ class EconomyError(_Answer): ``report_id`` (on ``/reports``) or ``simulation_id`` (on ``/simulations``) is set only with ``modal_call_refused``: it names the - record the service created, whose run it failed because Modal refused - the call or didn't accept it in time. + record the service created or re-ran, whose new run it failed because + Modal refused the call or didn't accept it in time. ``run_id`` names + that new run on a re-run (a budget window's: its budget-window run), + and with ``run_in_progress`` the record's run that is pending or + running: a report's budget-window run, or else its report run, or a + simulation's run. """ code: EconomyErrorCode detail: str report_id: UUID | None = None simulation_id: UUID | None = None + run_id: UUID | None = None class EconomyRequestReason(BaseModel): diff --git a/projects/policyengine-simulation-entry/src/policyengine_simulation_entry/economy_rerun_routes.py b/projects/policyengine-simulation-entry/src/policyengine_simulation_entry/economy_rerun_routes.py new file mode 100644 index 000000000..ae4675171 --- /dev/null +++ b/projects/policyengine-simulation-entry/src/policyengine_simulation_entry/economy_rerun_routes.py @@ -0,0 +1,329 @@ +"""The re-run routes: calculate a stored report or simulation again as a new run. + +- ``POST /reports/{report_id}/runs`` (permission ``simulation:submit``) + creates the stored report's new run on the v2 version manifest's default + worker and starts its coordinator, answering 201 with the report's ID, the + new run's ID and status, and each side's simulation and simulation run. + ``{"force": true}`` calculates both simulations again; otherwise a side + reuses a run of its own simulation that succeeded on the same versions. + A budget-window report's new run is a budget-window run with a report run + per year, up to five of them started at once as a new window's are; it + answers as the budget-window create does, naming the window run and every + year's runs. A household report's new run gives each side a new run of its + own household simulation, never reusing one, started by one household + function call; it answers as the household report create does. +- ``POST /simulations/{simulation_id}/runs`` (``simulation:submit``) creates + the stored simulation's new run and starts its simulation function, + answering 201 with the simulation's ID, the new run's ID and its status. + +Every earlier run is kept. A re-run answers 404 for a record that does not +exist, and 409 when the record has a run pending or running +(``run_in_progress``, naming it in ``run_id``) or is one this service does +not re-run (``rerun_not_supported``); then nothing is written. The other +answers are the creates': 400 or 422 when the stored record cannot be +calculated on the current catalog version, 503 with ``Retry-After`` before +any write or when Modal refused the new run's call (then the run was failed, +and the answer names the record and the run), and the JSON 500. +""" + +from __future__ import annotations + +from typing import TYPE_CHECKING, Any, Protocol +from uuid import UUID + +from fastapi import APIRouter, Depends, Request, status +from fastapi.responses import JSONResponse + +from policyengine_simulation_entry.budget_window_api_models import ( + BudgetWindowReportCreated, +) +from policyengine_simulation_entry.economy_api_models import ( + EconomyError, + EconomyReportCreated, + EconomyRequestUnprocessable, + EconomySimulationCreated, + ReportRerunRequest, +) +from policyengine_simulation_entry.economy_routes import ( + ERROR_ANSWER, + FORBIDDEN_ANSWER, + RETRY_AFTER_HEADER, + SUBMISSION_FAILURES, + EconomySubmissions, + call_request_id_for, + error_response, + not_configured_response, + submission_failure_response, + unexpected_failure_response, + unknown_record_response, +) +from policyengine_simulation_entry.economy_submission_errors import ( + EconomyRecordNotFoundError, + EconomyRerunConflictError, +) +from policyengine_simulation_entry.household_report_api_models import ( + HouseholdReportCreated, +) +from policyengine_simulation_entry.route_support import observability_context + +if TYPE_CHECKING: + from collections.abc import Awaitable, Callable + + from fastapi import FastAPI + from pydantic import BaseModel + + from policyengine_simulation_entry.auth import AuthorizationDependency + from policyengine_simulation_entry.economy_submission_errors import RecordName + from policyengine_simulation_entry.entry_runtime import EntryRuntime + +REPORT_RUNS_PATH = "/reports/{report_id}/runs" +SIMULATION_RUNS_PATH = "/simulations/{simulation_id}/runs" + + +class EconomyReruns(Protocol): + """Re-run stored reports and economy simulations (``EconomyRerunService``).""" + + async def rerun_report( + self, + report_id: UUID, + request: ReportRerunRequest, + *, + request_id: str, + observability_context: dict[str, Any] | None, + ) -> EconomyReportCreated | BudgetWindowReportCreated | HouseholdReportCreated: + """Create the report's new run and start it, or a window's first years.""" + ... + + async def rerun_simulation( + self, + simulation_id: UUID, + *, + request_id: str, + observability_context: dict[str, Any] | None, + ) -> EconomySimulationCreated: + """Create the simulation's new run and start its function.""" + ... + + +class EconomyService(EconomySubmissions, EconomyReruns, Protocol): + """Create, read and re-run economy reports and simulations.""" + + +# The run a 409 run_in_progress names, by the record re-run. +_RUN_UNDER_WAY: dict[RecordName, str] = { + "report": "the report's budget-window run, or else its report run", + "simulation": "the simulation's run", +} +# The 400 codes that name a stored record's own rows on its re-run: a +# report's re-run also names its stored simulations (missing_reference) and +# a household report's its household. +_STORED_ROW_CODES: dict[RecordName, str] = { + "report": ( + "unknown_region, unknown_dataset, unknown_dynamic, unknown_policy, " + "unknown_household and missing_reference" + ), + "simulation": "unknown_region, unknown_dataset, unknown_dynamic and unknown_policy", +} +# What only a household report's re-run answers, by status, after the +# economy codes. +_HOUSEHOLD_ANSWERS: dict[RecordName, dict[int, str]] = { + "report": { + 400: "; or a household report's household no longer exists", + 422: ( + "; or the stored household cannot be calculated " + "(household_not_calculable, located under report)" + ), + 503: "Economy runs or household reports are not configured", + }, + "simulation": {503: "Economy runs are not configured"}, +} + + +def _responses(record: RecordName) -> dict[int | str, dict[str, Any]]: + """Return every answer a re-run of ``record`` declares.""" + return { + 201: {"description": f"The {record}'s new run was created and started"}, + 400: { + "description": ( + f"The stored {record}'s region, dataset, policy or dynamic is " + "missing from the catalog version of the default worker's .py " + f"version, or a function the {record} needs is missing" + f"{_HOUSEHOLD_ANSWERS[record].get(400, '')}; nothing was written. " + f"{_STORED_ROW_CODES[record]} name the stored {record}'s own " + "rows; any other code is the service's state" + ), + **ERROR_ANSWER, + }, + 403: FORBIDDEN_ANSWER, + 404: { + "description": f"There is no such {record} (unknown_{record})", + **ERROR_ANSWER, + }, + 409: { + "description": ( + f"The {record} is not re-run now, and nothing was written: it " + "has a run pending or running (run_in_progress; run_id names " + f"{_RUN_UNDER_WAY[record]}), or the service does not re-run it " + "(rerun_not_supported)" + ), + **ERROR_ANSWER, + }, + 422: { + "description": ( + "The ID or the body is malformed, or the stored policies and " + "dynamics cannot be calculated on the current catalog version " + "(missing_parameters or overlapping_parameters, located under " + f"{record}){_HOUSEHOLD_ANSWERS[record].get(422, '')}; nothing was " + "written" + ), + "model": EconomyRequestUnprocessable, + }, + 500: { + "description": ( + "The service failed unexpectedly, or could not confirm the new " + "run was written (internal_error), possibly after creating it" + ), + **ERROR_ANSWER, + }, + 503: { + "description": ( + f"{_HOUSEHOLD_ANSWERS[record][503]}, or the v2 version manifest or " + "the database cannot be read, and nothing was written; or Modal " + "refused the new run's call, or didn't accept it in time " + f"(modal_call_refused): then the {record} named by {record}_id " + "has a new run, named by run_id, which was failed. Retry after " + "Retry-After seconds" + ), + "headers": RETRY_AFTER_HEADER, + **ERROR_ANSWER, + }, + } + + +def _rerun_failure_response( + error: Exception, record: RecordName, record_id: UUID +) -> JSONResponse: + """Answer a re-run the service refused or could not serve.""" + if isinstance(error, EconomyRecordNotFoundError): + return unknown_record_response(record, record_id) + if isinstance(error, EconomyRerunConflictError): + return error_response( + status.HTTP_409_CONFLICT, + EconomyError(code=error.code, detail=error.detail, run_id=error.run_id), + ) + return submission_failure_response(error, record) + + +_RERUN_FAILURES = ( + *SUBMISSION_FAILURES, + EconomyRecordNotFoundError, + EconomyRerunConflictError, +) + + +def add_economy_rerun_routes( + app: FastAPI, + entry: EntryRuntime, + reruns: EconomyReruns | None, + *, + submitter: AuthorizationDependency, +) -> None: + """Add the re-run routes; without a service they answer 503.""" + log_string_limit = entry.runtime.config.limits.max_string_length + router = APIRouter() + + async def answer( # the route, its record and how to re-run it + request: Request, + rerun: Callable[ + [EconomyReruns, str, dict[str, Any] | None], Awaitable[BaseModel] + ], + *, + route: str, + record: RecordName, + record_id: UUID, + ) -> JSONResponse: + if reruns is None: + return not_configured_response() + observability_id = entry.start_workflow_observability(request) + try: + created = await rerun( + reruns, + call_request_id_for(request), + observability_context(entry, observability_id), + ) + except _RERUN_FAILURES as error: + return _rerun_failure_response(error, record, record_id) + # Every other failure is still answered as the JSON 500 declared. + except Exception as error: # noqa: BLE001 # answered as internal_error + return unexpected_failure_response( + route, error, max_characters=log_string_limit + ) + return JSONResponse( + created.model_dump(mode="json"), status_code=status.HTTP_201_CREATED + ) + + @router.post( + REPORT_RUNS_PATH, + summary="Re-run an economy, budget-window or household report", + description=( + "Create the stored report's new run on the v2 version manifest's " + "default worker, and start it; every earlier run is kept. Without " + "force, a side reuses a run of its own simulation that succeeded " + "on the same versions; with force, both simulations are " + "calculated again. A budget-window report's new run is a " + "budget-window run with a report run per year, up to five of them " + "started at once, each year on its own year's simulations. A " + "household report's new run calculates its stored household in " + "the report's year with a new run of each side's own household " + "simulation, never reused, all in one household function call." + ), + operation_id="rerun_report", + status_code=status.HTTP_201_CREATED, + response_model=( + EconomyReportCreated | BudgetWindowReportCreated | HouseholdReportCreated + ), + responses=_responses("report"), + dependencies=[Depends(submitter)], + ) + async def rerun_report( + report_id: UUID, body: ReportRerunRequest, request: Request + ) -> JSONResponse: + return await answer( + request, + lambda service, request_id, context: service.rerun_report( + report_id, + body, + request_id=request_id, + observability_context=context, + ), + route=REPORT_RUNS_PATH, + record="report", + record_id=report_id, + ) + + @router.post( + SIMULATION_RUNS_PATH, + summary="Re-run an economy simulation", + description=( + "Create the stored simulation's new run on the v2 version " + "manifest's default worker, and start it; every earlier run is " + "kept, and the new run is always calculated." + ), + operation_id="rerun_simulation", + status_code=status.HTTP_201_CREATED, + response_model=EconomySimulationCreated, + responses=_responses("simulation"), + dependencies=[Depends(submitter)], + ) + async def rerun_simulation(simulation_id: UUID, request: Request) -> JSONResponse: + return await answer( + request, + lambda service, request_id, context: service.rerun_simulation( + simulation_id, request_id=request_id, observability_context=context + ), + route=SIMULATION_RUNS_PATH, + record="simulation", + record_id=simulation_id, + ) + + app.include_router(router) diff --git a/projects/policyengine-simulation-entry/src/policyengine_simulation_entry/economy_reruns.py b/projects/policyengine-simulation-entry/src/policyengine_simulation_entry/economy_reruns.py new file mode 100644 index 000000000..6573aea78 --- /dev/null +++ b/projects/policyengine-simulation-entry/src/policyengine_simulation_entry/economy_reruns.py @@ -0,0 +1,792 @@ +"""Re-run a stored economy report or simulation on the current .py version. + +A re-run calculates a stored record again on the v2 version manifest's +default worker, as a new run, and keeps every earlier run. The service: + +1. reads what the record calculates: its country, year, region code, + dataset name and each simulation's policy and dynamic. A record this + service does not re-run, or one missing a row it calculates on, is + refused (409 ``rerun_not_supported``), as is one that does not exist + (404); +2. finds the default worker's economy functions for the country, and reads, + in one transaction, the country's catalog version for that worker's .py + version, its region and dataset with the record's code and name, and each + simulation's policy and dynamic values on that catalog version. For a + report, unless the re-run is forced, each side's reusable run is a run of + that side's own simulation that succeeded on the current versions, as a + new report's would be (``find_reusable_run_of``); +3. creates, in one transaction, the new run (a report's with trigger + ``manual``) and each report side's new simulation run of its own + simulation, all pending with no Modal call ID, unless the record has a + run pending or running (409 ``run_in_progress``); +4. starts and records the new run's call as a new record's: the report run's + coordinator, or the simulation function, by run ID (C1 · 8's + coordination), failing the new runs when Modal refuses the call. + +A budget-window report's re-run is a new budget-window run with a report +run per year, each year on the simulations that year's report run of the +report's newest window run names; its first years start as a new window's +do. A household report's re-run is the household report service's +(``household_report_reruns``): a new run of each side's own household +simulation, started by one household function call. A household simulation +is re-run only through its report. Whatever is missing or cannot be +calculated is refused in steps 1 and 2, before anything is written. +""" + +from __future__ import annotations + +from dataclasses import dataclass +from datetime import UTC, datetime +from typing import TYPE_CHECKING, Any, Final, Protocol, get_args +from uuid import uuid4 + +from policyengine_simulation_contract.stage12_bundle import CountryId +from policyengine_simulation_contract.stage12_execution import ReportRunIds +from policyengine_simulation_persistence import ( + EconomyRequest, + NewReportRerun, + NewSimulationRerun, + NewWindowRerun, + RerunSide, + RunReuse, + WindowRerunYear, +) + +from policyengine_simulation_entry.budget_window_api_models import ( + BudgetWindowReportCreated, +) +from policyengine_simulation_entry.budget_window_submissions import ( + BudgetWindowRuns, + ReportSubmissionService, + year_created, + year_run_ids, +) +from policyengine_simulation_entry.economy_api_models import ( + EconomyReportCreated, + EconomySimulationCreated, +) +from policyengine_simulation_entry.economy_calculation_inputs import ( + CalculationScope, + DatasetNotInBundleError, + bundle_dataset, + report_run_calculation, + simulation_options, + simulation_run_calculation, +) +from policyengine_simulation_entry.economy_submission_errors import ( + STORED_REPORT, + STORED_SIMULATION, + EconomyRecordNotFoundError, + EconomyRerunConflictError, + EconomySubmissionRefusedError, + EconomySubmissionUnavailableError, + rerun_refusals, +) +from policyengine_simulation_entry.economy_submissions import ( + CALL_START_TIMEOUT_SECONDS, +) +from policyengine_simulation_entry.household_report_reruns import ( + StoredHouseholdReport, +) + +if TYPE_CHECKING: + from collections.abc import Callable, Sequence + from uuid import UUID + + from policyengine_simulation_persistence import ( + BudgetWindowInputs, + EconomyCatalogRows, + EconomyReportInputs, + RecordKind, + ReportDefinition, + ReusableRun, + SideDefinition, + SimulationChoice, + SimulationDefinition, + WindowYearDefinition, + ) + + from policyengine_simulation_entry.budget_window_submissions import ( + BudgetWindowAdvancer, + ) + from policyengine_simulation_entry.economy_api_models import ( + ReportRerunRequest, + ) + from policyengine_simulation_entry.economy_submission_errors import ( + RecordName, + RefusalSource, + ) + from policyengine_simulation_entry.economy_submissions import ServiceSettings + from policyengine_simulation_entry.economy_workers import ( + EconomyRunCaller, + EconomyWorker, + EconomyWorkerSource, + ) + from policyengine_simulation_entry.household_report_api_models import ( + HouseholdReportCreated, + ) + from policyengine_simulation_entry.household_report_reruns import ( + HouseholdReportReruns, + ) + +# The kinds of stored economy report this service re-runs itself. +_RERUN_REPORT_KINDS: Final[frozenset[RecordKind]] = frozenset( + {"economy", "budget_window"} +) +# The countries this service calculates, as the contract names them. +_COUNTRIES: Final[tuple[CountryId, ...]] = get_args(CountryId) +_REPORT: Final[RecordName] = "report" +_SIMULATION: Final[RecordName] = "simulation" + + +class EconomyRerunRuns(BudgetWindowRuns, Protocol): + """The economy store's reads and writes, with budget windows' and re-runs'. + + ``EconomyRunStore`` is one. + """ + + def read_report_definition(self, report_id: UUID) -> ReportDefinition | None: + """Return what the stored report calculates, or None.""" + ... + + def read_simulation_definition( + self, simulation_id: UUID + ) -> SimulationDefinition | None: + """Return what the stored simulation calculates, or None.""" + ... + + def read_report_rerun_inputs( + self, + request: EconomyRequest, + *, + baseline: SideDefinition, + reform: SideDefinition, + reuse: RunReuse | None, + ) -> EconomyReportInputs: + """Read a re-run's catalog rows, both sides' values and reusable runs.""" + ... + + def read_window_rerun_inputs( + self, + request: EconomyRequest, + *, + baseline: SimulationChoice, + reform: SimulationChoice, + years: Sequence[WindowYearDefinition], + reuse: RunReuse | None, + ) -> BudgetWindowInputs: + """Read a window re-run's catalog rows, both sides' values and years' runs.""" + ... + + def create_window_rerun(self, rerun: NewWindowRerun) -> None: + """Create the window's new run, each year's report run and new runs.""" + ... + + def create_report_rerun(self, rerun: NewReportRerun) -> None: + """Create the report's new run and its sides' new simulation runs.""" + ... + + def create_simulation_rerun(self, rerun: NewSimulationRerun) -> None: + """Create the simulation's new run.""" + ... + + +class EconomyRerunService(ReportSubmissionService[EconomyRerunRuns]): + """Create and re-run economy and budget-window reports and simulations. + + A household report's re-run goes to ``household_reports``; without it, + one the service could re-run answers 503, after the 409s. + """ + + def __init__( # noqa: PLR0913 # the household service, then the report service's seams + self, + *, + household_reports: HouseholdReportReruns | None = None, + windows: BudgetWindowAdvancer, + store: EconomyRerunRuns, + workers: EconomyWorkerSource, + caller: EconomyRunCaller, + settings: ServiceSettings, + call_start_timeout_seconds: float = CALL_START_TIMEOUT_SECONDS, + new_id: Callable[[], UUID] = uuid4, + clock: Callable[[], datetime] = lambda: datetime.now(UTC), + ) -> None: + """Use these seams; tests replace each of them.""" + super().__init__( + windows=windows, + store=store, + workers=workers, + caller=caller, + settings=settings, + call_start_timeout_seconds=call_start_timeout_seconds, + new_id=new_id, + clock=clock, + ) + self._household_reports = household_reports + + async def rerun_report( + self, + report_id: UUID, + request: ReportRerunRequest, + *, + request_id: str, + observability_context: dict[str, Any] | None, + ) -> EconomyReportCreated | BudgetWindowReportCreated | HouseholdReportCreated: + """Create the stored report's new run on the current versions, and start it. + + A budget window's new run is a budget-window run with a report run + per year, whose first years are started (``_rerun_window``); a + household report's is the household report service's, which + ``force`` changes nothing for. Raise EconomyRecordNotFoundError (404) + or EconomyRerunConflictError (409) as the module describes, and + otherwise the errors ``submit_report`` raises, a 503 for Modal's + refusal naming the new run as well as the report. + """ + definition = await self._read_state( + lambda: self._store.read_report_definition(report_id) + ) + if definition is None: + raise EconomyRecordNotFoundError(_REPORT) + if definition.kind == "household": + return await self._rerun_household_report( + definition, observability_context=observability_context + ) + stored = _rerunnable_report(definition) + window_years = ( + _complete_window_years(definition) + if definition.kind == "budget_window" + else () + ) + worker = await self._default_worker(stored.country) + economy_request = EconomyRequest( + country=stored.country, + policyengine_version=worker.policyengine_version, + region_code=stored.region_code, + dataset_name=stored.dataset_name, + year=stored.year, + ) + reuse = ( + None + if request.force or self._settings.reuse_prefix is None + else RunReuse( + versions=worker.versions, + inputs=definition.inputs, + output_uri_prefix=self._settings.reuse_prefix, + ) + ) + if window_years: + return await self._rerun_window( + _WindowRerunRequest( + stored=stored, + options=definition.inputs, + years=window_years, + economy_request=economy_request, + reuse=reuse, + recalculate=request.force, + ), + worker, + request_id=request_id, + observability_context=observability_context, + ) + inputs = await self._read_inputs( + lambda: self._store.read_report_rerun_inputs( + economy_request, + baseline=stored.baseline, + reform=stored.reform, + reuse=reuse, + ), + request=economy_request, + source=STORED_REPORT, + ) + rerun = NewReportRerun( + report_id=report_id, + run_id=self._new_id(), + population=inputs.catalog.population(stored.year), + versions=worker.versions, + baseline=self._rerun_side( + stored.baseline.simulation_id, inputs.baseline.reusable_run + ), + reform=self._rerun_side( + stored.reform.simulation_id, inputs.reform.reusable_run + ), + ) + calculation = report_run_calculation( + worker, + _rerun_scope( + stored.country, + economy_request, + worker, + inputs.catalog, + options=definition.inputs, + ), + self._origin(request_id), + run_ids=ReportRunIds( + report_run_id=rerun.run_id, + baseline_simulation_run_id=rerun.baseline.simulation_run_id, + reform_simulation_run_id=rerun.reform.simulation_run_id, + ), + baseline_policy=inputs.baseline.reform, + reform_policy=inputs.reform.reform, + ) + await self._create_rerun( + _REPORT, + lambda: self._store.create_report_rerun(rerun), + source=STORED_REPORT, + ) + status = await self._start_report_run( + worker, + calculation, + report_id=report_id, + run_id=rerun.run_id, + new_simulation_run_ids=rerun.new_simulation_run_ids, + observability_context=observability_context, + rerun=True, + ) + return EconomyReportCreated( + report_id=report_id, + report_run_id=rerun.run_id, + status=status, + baseline_simulation_id=rerun.baseline.simulation_id, + baseline_simulation_run_id=rerun.baseline.simulation_run_id, + baseline_reused=rerun.baseline.reused, + reform_simulation_id=rerun.reform.simulation_id, + reform_simulation_run_id=rerun.reform.simulation_run_id, + reform_reused=rerun.reform.reused, + ) + + async def rerun_simulation( + self, + simulation_id: UUID, + *, + request_id: str, + observability_context: dict[str, Any] | None, + ) -> EconomySimulationCreated: + """Create the stored simulation's new run on the current versions, and start it. + + The run is always calculated: a simulation run is the calculation, + with nothing of it to reuse. Raise the errors ``rerun_report`` raises, + naming the simulation. + """ + definition = await self._read_state( + lambda: self._store.read_simulation_definition(simulation_id) + ) + if definition is None: + raise EconomyRecordNotFoundError(_SIMULATION) + stored = _rerunnable_simulation(definition) + worker = await self._default_worker(stored.country) + economy_request = EconomyRequest( + country=stored.country, + policyengine_version=worker.policyengine_version, + region_code=stored.region_code, + dataset_name=stored.dataset_name, + year=stored.year, + ) + inputs = await self._read_inputs( + lambda: self._store.read_simulation_inputs( + economy_request, definition.choice + ), + request=economy_request, + source=STORED_SIMULATION, + ) + rerun = NewSimulationRerun( + simulation_id=simulation_id, + run_id=self._new_id(), + population=inputs.catalog.population(stored.year), + versions=worker.versions, + ) + calculation = simulation_run_calculation( + worker, + _rerun_scope( + stored.country, economy_request, worker, inputs.catalog, options={} + ), + self._origin(request_id), + simulation_run_id=rerun.run_id, + policy=inputs.simulation.reform, + ) + await self._create_rerun( + _SIMULATION, + lambda: self._store.create_simulation_rerun(rerun), + source=STORED_SIMULATION, + ) + status = await self._start_simulation_run( + worker, + calculation, + simulation_id=simulation_id, + run_id=rerun.run_id, + observability_context=observability_context, + rerun=True, + ) + return EconomySimulationCreated( + simulation_id=simulation_id, + simulation_run_id=rerun.run_id, + status=status, + ) + + async def _rerun_household_report( + self, + definition: ReportDefinition, + *, + observability_context: dict[str, Any] | None, + ) -> HouseholdReportCreated: + """Hand a stored household report the service can re-run to its service.""" + stored = _rerunnable_household_report(definition) + if self._household_reports is None: + raise EconomySubmissionUnavailableError( + code="not_configured", detail="Household reports are not configured." + ) + return await self._household_reports.rerun_report( + stored, observability_context=observability_context + ) + + async def _rerun_window( + self, + window: _WindowRerunRequest, + worker: EconomyWorker, + *, + request_id: str, + observability_context: dict[str, Any] | None, + ) -> BudgetWindowReportCreated: + """Create the window's new run and each year's runs; start its first years. + + Each year's sides are that year's own simulations, as its report run + in the report's newest window run names them, and each year reuses a + run of its own simulations unless the re-run is forced. The years + start as a new window's do (``_start_first_years``). + """ + stored = window.stored + inputs = await self._read_inputs( + lambda: self._store.read_window_rerun_inputs( + window.economy_request, + baseline=stored.baseline.choice, + reform=stored.reform.choice, + years=window.years, + reuse=window.reuse, + ), + request=window.economy_request, + source=STORED_REPORT, + ) + window_run_id = self._new_id() + rerun = NewWindowRerun( + report_id=stored.report_id, + window_run_id=window_run_id, + population=inputs.catalog.population(stored.year), + versions=worker.versions, + recalculate=window.recalculate, + years=tuple(self._window_rerun_year(year, inputs) for year in window.years), + ) + first_year = rerun.years[0] + first_calculation = report_run_calculation( + worker, + _rerun_scope( + stored.country, + window.economy_request, + worker, + inputs.catalog, + options=window.options, + ), + self._origin(request_id), + run_ids=year_run_ids(first_year), + baseline_policy=inputs.baseline.reform, + reform_policy=inputs.reform.reform, + ) + await self._create_rerun( + _REPORT, + lambda: self._store.create_window_rerun(rerun), + source=STORED_REPORT, + ) + status = await self._start_first_years( + worker, + first_calculation, + report_id=stored.report_id, + window_run_id=window_run_id, + first_year=first_year.year, + observability_context=observability_context, + rerun=True, + ) + return BudgetWindowReportCreated( + report_id=stored.report_id, + report_run_id=first_year.report_run_id, + budget_window_run_id=window_run_id, + status=status, + years=[year_created(window_year) for window_year in rerun.years], + ) + + def _window_rerun_year( + self, year: WindowYearDefinition, inputs: BudgetWindowInputs + ) -> WindowRerunYear: + """Return a year naming its own simulations' reusable runs, or new runs.""" + baseline_run, reform_run = inputs.reusable_runs[year.year] + return WindowRerunYear( + year=year.year, + report_run_id=self._new_id(), + baseline=self._rerun_side(year.baseline_simulation_id, baseline_run), + reform=self._rerun_side(year.reform_simulation_id, reform_run), + ) + + def _rerun_side( + self, simulation_id: UUID, reusable_run: ReusableRun | None + ) -> RerunSide: + """Return a side naming its own simulation's reusable run, or a new run.""" + if reusable_run is not None: + return RerunSide( + simulation_id=simulation_id, + simulation_run_id=reusable_run.simulation_run_id, + reused=True, + ) + return RerunSide( + simulation_id=simulation_id, + simulation_run_id=self._new_id(), + reused=False, + ) + + async def _create_rerun( + self, + record: RecordName, + create: Callable[[], None], + *, + source: RefusalSource, + ) -> None: + """Create the re-run's rows; refuse it while the record has a run under way. + + A record deleted after its reads is answered as not found. Any other + row the database refuses is named as ``source``'s; for a report, + that includes a side's simulation deleted with it. + """ + with rerun_refusals(record): + await self._create(record, create, rerun=True, source=source) + + +@dataclass(frozen=True) +class _StoredReportRows: + """The country and rows a stored report calculates on, each present.""" + + report_id: UUID + country: CountryId + year: int + region_code: str + dataset_name: str + baseline: SideDefinition + reform: SideDefinition + + +@dataclass(frozen=True) +class _StoredSimulationRows: + """The country and rows a stored simulation calculates on, each present.""" + + country: CountryId + year: int + region_code: str + dataset_name: str + + +def _calculated_country(country: str | None) -> CountryId | None: + """Return the country if this service calculates it, or None.""" + return next((known for known in _COUNTRIES if known == country), None) + + +def _refusal( + record: RecordName, kind: RecordKind, missing: tuple[tuple[str, object], ...] +) -> EconomyRerunConflictError: + """Return why a stored record is not re-run: its kind, or the first row it lacks. + + ``missing`` pairs each row's description with the value the record holds. + """ + if kind == "unsupported": + detail = f"The service does not re-run this type of {record}." + elif record == _SIMULATION and kind == "household": + detail = ( + "The service re-runs a household simulation only through its report " + "(POST /reports/{report_id}/runs)." + ) + else: + lacking = next(name for name, value in missing if value is None) + detail = f"The {record} names no {lacking}, so it cannot be re-run." + return EconomyRerunConflictError(code="rerun_not_supported", detail=detail) + + +def _rerunnable_report(definition: ReportDefinition) -> _StoredReportRows: + """Return the stored report's rows, or refuse one this service can't re-run.""" + baseline, reform = definition.baseline, definition.reform + region_code, dataset_name = definition.region_code, definition.dataset_name + country, year = _calculated_country(definition.country), definition.year + if ( + definition.kind not in _RERUN_REPORT_KINDS + or country is None + or baseline is None + or reform is None + or region_code is None + or dataset_name is None + or year is None + ): + raise _refusal( + _REPORT, + definition.kind, + ( + ("country this service calculates", country), + ("baseline simulation", baseline), + ("reform simulation", reform), + ("region", region_code), + ("dataset", dataset_name), + ("year", year), + ), + ) + return _StoredReportRows( + report_id=definition.id, + country=country, + year=year, + region_code=region_code, + dataset_name=dataset_name, + baseline=baseline, + reform=reform, + ) + + +def _rerunnable_household_report( + definition: ReportDefinition, +) -> StoredHouseholdReport: + """Return the stored household report's rows, or refuse one it can't re-run. + + A side naming a dynamic is refused: a household calculation applies none. + """ + country, year = _calculated_country(definition.country), definition.year + household_id, baseline = definition.household_id, definition.baseline + if country is None or household_id is None or baseline is None or year is None: + raise _refusal( + _REPORT, + definition.kind, + ( + ("country this service calculates", country), + ("household", household_id), + ("baseline simulation", baseline), + ("year", year), + ), + ) + if any( + side is not None and side.choice.dynamic_id is not None + for side in (baseline, definition.reform) + ): + raise EconomyRerunConflictError( + code="rerun_not_supported", + detail=( + "The report's household simulation names a dynamic, which a " + "household calculation does not apply, so it cannot be re-run." + ), + ) + return StoredHouseholdReport( + report_id=definition.id, + country=country, + household_id=household_id, + year=year, + baseline=baseline, + reform=definition.reform, + ) + + +@dataclass(frozen=True) +class _WindowRerunRequest: + """What a budget window's re-run calculates, as the service has read it.""" + + stored: _StoredReportRows + options: dict[str, Any] + years: tuple[WindowYearDefinition, ...] + economy_request: EconomyRequest + reuse: RunReuse | None + recalculate: bool + + +def _complete_window_years( + definition: ReportDefinition, +) -> tuple[WindowYearDefinition, ...]: + """Return the stored window's years, or refuse a window that lacks one. + + Its newest calculation must have a report run naming both sides in each + year of its window, from the report's first year; the definition leaves + out a year naming no reform. + """ + expected = ( + () + if definition.year is None or definition.budget_window_size is None + else tuple( + range(definition.year, definition.year + definition.budget_window_size) + ) + ) + years = definition.window_years + if not expected or tuple(year.year for year in years) != expected: + raise EconomyRerunConflictError( + code="rerun_not_supported", + detail=( + "The report names no complete budget-window calculation, a " + "report run naming both sides in each year of its window, so " + "it cannot be re-run." + ), + ) + return years + + +def _rerunnable_simulation(definition: SimulationDefinition) -> _StoredSimulationRows: + """Return the stored simulation's country and rows, or refuse it. + + Its country is None when its catalog version's model is of no country + this service calculates. + """ + country, year = _calculated_country(definition.country), definition.year + region_code, dataset_name = definition.region_code, definition.dataset_name + if ( + definition.kind != "economy" + or country is None + or region_code is None + or dataset_name is None + or year is None + ): + raise _refusal( + _SIMULATION, + definition.kind, + ( + ("country this service calculates", country), + ("region", region_code), + ("dataset", dataset_name), + ("year", year), + ), + ) + return _StoredSimulationRows( + country=country, + year=year, + region_code=region_code, + dataset_name=dataset_name, + ) + + +def _rerun_scope( + country: CountryId, + request: EconomyRequest, + worker: EconomyWorker, + catalog: EconomyCatalogRows, + *, + options: dict[str, Any], +) -> CalculationScope: + """Return what a re-run calculates on, with the bundle's dataset. + + ``options`` are the stored report's (``reports.inputs``). Raise + EconomySubmissionRefusedError (``dataset_not_in_bundle``) when the + default worker's bundle lacks the stored dataset. + """ + try: + dataset = bundle_dataset(worker, catalog.dataset.name) + except DatasetNotInBundleError: + raise EconomySubmissionRefusedError( + code="dataset_not_in_bundle", + detail=( + f"The default v2 worker, .py {worker.version}, has no dataset " + f"{catalog.dataset.name}, which the stored record names." + ), + ) from None + return CalculationScope( + country=country, + region_code=catalog.region.code, + dataset=dataset, + year=request.year, + options=simulation_options( + include_cliffs=bool(options.get("include_cliffs", False)), + spm=options.get("spm"), + ), + ) diff --git a/projects/policyengine-simulation-entry/src/policyengine_simulation_entry/economy_routes.py b/projects/policyengine-simulation-entry/src/policyengine_simulation_entry/economy_routes.py index 129ef37d1..80a46798f 100644 --- a/projects/policyengine-simulation-entry/src/policyengine_simulation_entry/economy_routes.py +++ b/projects/policyengine-simulation-entry/src/policyengine_simulation_entry/economy_routes.py @@ -107,14 +107,14 @@ SIMULATION_PATH = "/simulations/{simulation_id}" RETRY_AFTER_SECONDS = 10 -_ERROR = {"model": EconomyError} -_RETRY_AFTER = { +ERROR_ANSWER = {"model": EconomyError} +RETRY_AFTER_HEADER = { "Retry-After": { "description": "Seconds to wait before asking again", "schema": {"type": "integer"}, } } -_FORBIDDEN = { +FORBIDDEN_ANSWER = { "description": "The caller's token is missing, invalid or lacks the permission" } _REFUSED = { @@ -127,7 +127,7 @@ "service's state, such as a catalog version missing while a release " "deploys" ), - **_ERROR, + **ERROR_ANSWER, } _UNPROCESSABLE = { "description": ( @@ -144,7 +144,7 @@ "The service failed unexpectedly, or could not confirm its rows were " "written (internal_error), possibly after creating the record" ), - **_ERROR, + **ERROR_ANSWER, } @@ -157,12 +157,12 @@ "Economy runs are not configured, or the database cannot be read; " "retry after Retry-After seconds" ), - "headers": _RETRY_AFTER, - **_ERROR, + "headers": RETRY_AFTER_HEADER, + **ERROR_ANSWER, } _READ_INTERNAL_ERROR = { "description": "The service failed unexpectedly (internal_error)", - **_ERROR, + **ERROR_ANSWER, } @@ -175,8 +175,8 @@ def _unavailable_description(record: RecordName) -> dict[str, Any]: f"(modal_call_refused): then the {record} named by {record}_id was " "created and its run failed. Retry after Retry-After seconds" ), - "headers": _RETRY_AFTER, - **_ERROR, + "headers": RETRY_AFTER_HEADER, + **ERROR_ANSWER, } @@ -235,7 +235,8 @@ async def submit_report( ... -def _error_response(status_code: int, error: EconomyError) -> JSONResponse: +def error_response(status_code: int, error: EconomyError) -> JSONResponse: + """Answer ``error`` with its status, and ``Retry-After`` on a 503.""" headers = ( {"Retry-After": str(RETRY_AFTER_SECONDS)} if status_code == status.HTTP_503_SERVICE_UNAVAILABLE @@ -248,16 +249,18 @@ def _error_response(status_code: int, error: EconomyError) -> JSONResponse: ) -def _not_configured( +def not_configured_response( detail: str = "Economy runs are not configured.", ) -> JSONResponse: - return _error_response( + """Answer 503 ``not_configured`` with ``detail``, naming what is not set up.""" + return error_response( status.HTTP_503_SERVICE_UNAVAILABLE, EconomyError(code="not_configured", detail=detail), ) -_SUBMISSION_FAILURES = ( +# The submission service's own failures, each answered with its status. +SUBMISSION_FAILURES = ( EconomySubmissionRefusedError, EconomySubmissionUnprocessableError, EconomySubmissionUnavailableError, @@ -265,10 +268,10 @@ def _not_configured( ) -def _submission_failure_response(error: Exception, record: RecordName) -> JSONResponse: +def submission_failure_response(error: Exception, record: RecordName) -> JSONResponse: """Answer a submission the service refused or could not serve.""" if isinstance(error, EconomySubmissionRefusedError): - return _error_response( + return error_response( status.HTTP_400_BAD_REQUEST, EconomyError(code=error.code, detail=error.detail), ) @@ -285,11 +288,13 @@ def _submission_failure_response(error: Exception, record: RecordName) -> JSONRe ) if isinstance(error, EconomySubmissionOutcomeUnknownError): return _outcome_unknown_response(error) - assert isinstance(error, EconomySubmissionUnavailableError) # noqa: S101 # the last of _SUBMISSION_FAILURES + assert isinstance(error, EconomySubmissionUnavailableError) # noqa: S101 # the last of SUBMISSION_FAILURES record_ids = {} if error.record_id is None else {f"{record}_id": error.record_id} - return _error_response( + return error_response( status.HTTP_503_SERVICE_UNAVAILABLE, - EconomyError(code=error.code, detail=error.detail, **record_ids), + EconomyError( + code=error.code, detail=error.detail, run_id=error.run_id, **record_ids + ), ) @@ -316,24 +321,29 @@ def _outcome_unknown_response( it nothing was created. The log is the fixed event ``economy_record_outcome_unknown``. """ - fields = {"record": error.record, "error_type": error.error_type} + fields = { + "record": error.record, + "error_type": error.error_type, + "rerun": error.rerun, + } + written = f"the {error.record}'s new run" if error.rerun else f"the {error.record}" logger.error( "economy_record_outcome_unknown", extra={**fields, "policyengine_attributes": fields}, ) - return _error_response( + return error_response( status.HTTP_500_INTERNAL_SERVER_ERROR, EconomyError( code="internal_error", detail=( - f"The service may have created the {error.record}, but could not " - "confirm it was written." + f"The service may have created {written}, but could not confirm " + "it was written." ), ), ) -def _unexpected_failure_response( +def unexpected_failure_response( route: str, error: Exception, *, max_characters: int ) -> JSONResponse: """Answer an unexpected failure with the JSON 500 the routes declare. @@ -348,7 +358,7 @@ def _unexpected_failure_response( error=error, max_characters=max_characters, ) - return _error_response( + return error_response( status.HTTP_500_INTERNAL_SERVER_ERROR, EconomyError( code="internal_error", detail="The service could not answer the request." @@ -399,8 +409,9 @@ def _simulation_state(stored: StoredSimulation) -> EconomySimulationState: ) -def _unknown(record: RecordName, record_id: UUID) -> JSONResponse: - return _error_response( +def unknown_record_response(record: RecordName, record_id: UUID) -> JSONResponse: + """Answer 404: there is no such report or simulation.""" + return error_response( status.HTTP_404_NOT_FOUND, EconomyError( code="unknown_report" if record == "report" else "unknown_simulation", @@ -448,7 +459,7 @@ async def _economy_report_answer( ) -> JSONResponse: """Create an economy or budget-window report; answer as ``POST /reports`` does.""" if submissions is None: - return _not_configured() + return not_configured_response() observability_id = entry.start_workflow_observability(request) try: call_request_id = call_request_id_for(request) @@ -462,11 +473,11 @@ async def _economy_report_answer( created = await submissions.submit_report( body, request_id=call_request_id, observability_context=context ) - except _SUBMISSION_FAILURES as error: - return _submission_failure_response(error, "report") + except SUBMISSION_FAILURES as error: + return submission_failure_response(error, "report") # Every other failure is still answered as the JSON 500 declared. except Exception as error: # noqa: BLE001 # answered as internal_error - return _unexpected_failure_response( + return unexpected_failure_response( REPORTS_PATH, error, max_characters=log_string_limit ) return JSONResponse( @@ -484,18 +495,18 @@ async def _household_report_answer( ) -> JSONResponse: """Create a household report and answer as ``POST /reports`` does.""" if household_reports is None: - return _not_configured("Household reports are not configured.") + return not_configured_response("Household reports are not configured.") observability_id = entry.start_workflow_observability(request) try: created = await household_reports.submit_report( body, observability_context=observability_context(entry, observability_id) ) - except _SUBMISSION_FAILURES as error: - return _submission_failure_response(error, "report") + except SUBMISSION_FAILURES as error: + return submission_failure_response(error, "report") # Every other failure, a request the service built wrong included, is # still answered as the JSON 500 declared. except Exception as error: # noqa: BLE001 # answered as internal_error - return _unexpected_failure_response( + return unexpected_failure_response( REPORTS_PATH, error, max_characters=log_string_limit ) return JSONResponse( @@ -541,7 +552,7 @@ def _submission_routes( responses={ 201: {"description": "The report and its first runs were created"}, 400: _REFUSED, - 403: _FORBIDDEN, + 403: FORBIDDEN_ANSWER, 422: _UNPROCESSABLE, 500: _INTERNAL_ERROR, 503: _unavailable_description("report"), @@ -577,7 +588,7 @@ async def create_report(body: ReportRequest, request: Request) -> JSONResponse: responses={ 201: {"description": "The simulation and its first run were created"}, 400: _REFUSED, - 403: _FORBIDDEN, + 403: FORBIDDEN_ANSWER, 422: _UNPROCESSABLE, 500: _INTERNAL_ERROR, 503: _unavailable_description("simulation"), @@ -588,7 +599,7 @@ async def create_simulation( body: EconomySimulationRequest, request: Request ) -> JSONResponse: if submissions is None: - return _not_configured() + return not_configured_response() observability_id = entry.start_workflow_observability(request) try: created = await submissions.submit_simulation( @@ -596,11 +607,11 @@ async def create_simulation( request_id=call_request_id_for(request), observability_context=observability_context(entry, observability_id), ) - except _SUBMISSION_FAILURES as error: - return _submission_failure_response(error, "simulation") + except SUBMISSION_FAILURES as error: + return submission_failure_response(error, "simulation") # Every other failure is still answered as the JSON 500 declared. except Exception as error: # noqa: BLE001 # answered as internal_error - return _unexpected_failure_response( + return unexpected_failure_response( SIMULATIONS_PATH, error, max_characters=log_string_limit ) return JSONResponse( @@ -621,20 +632,20 @@ async def _read_answer[Stored]( # noqa: PLR0913 # the read, its record and how ) -> JSONResponse: """Answer a read: the record as it stands, 404, 503 or the JSON 500.""" if submissions is None: - return _not_configured() + return not_configured_response() route = REPORT_PATH if record == "report" else SIMULATION_PATH try: stored = await read(submissions) state = None if stored is None else answer(stored) except EconomySubmissionUnavailableError as error: - return _submission_failure_response(error, record) + return submission_failure_response(error, record) # Every other failure is still answered as the JSON 500 declared. except Exception as error: # noqa: BLE001 # answered as internal_error - return _unexpected_failure_response( + return unexpected_failure_response( route, error, max_characters=log_string_limit ) if state is None: - return _unknown(record, record_id) + return unknown_record_response(record, record_id) return JSONResponse(state.model_dump(mode="json")) @@ -658,8 +669,11 @@ def _read_routes( response_model=EconomyReportState, responses={ 200: {"description": "The report as it stands"}, - 403: _FORBIDDEN, - 404: {"description": "There is no such report (unknown_report)", **_ERROR}, + 403: FORBIDDEN_ANSWER, + 404: { + "description": "There is no such report (unknown_report)", + **ERROR_ANSWER, + }, 422: _READ_UNPROCESSABLE, 500: _READ_INTERNAL_ERROR, 503: _READ_UNAVAILABLE, @@ -687,10 +701,10 @@ async def get_report(report_id: UUID) -> JSONResponse: response_model=EconomySimulationState, responses={ 200: {"description": "The simulation as it stands"}, - 403: _FORBIDDEN, + 403: FORBIDDEN_ANSWER, 404: { "description": "There is no such simulation (unknown_simulation)", - **_ERROR, + **ERROR_ANSWER, }, 422: _READ_UNPROCESSABLE, 500: _READ_INTERNAL_ERROR, diff --git a/projects/policyengine-simulation-entry/src/policyengine_simulation_entry/economy_submission_errors.py b/projects/policyengine-simulation-entry/src/policyengine_simulation_entry/economy_submission_errors.py index a284af30e..ac50b0132 100644 --- a/projects/policyengine-simulation-entry/src/policyengine_simulation_entry/economy_submission_errors.py +++ b/projects/policyengine-simulation-entry/src/policyengine_simulation_entry/economy_submission_errors.py @@ -3,14 +3,17 @@ The submission service raises these, and the routes answer each with its status: EconomySubmissionRefusedError 400, EconomySubmissionUnprocessableError 422, EconomySubmissionUnavailableError 503 and -EconomySubmissionOutcomeUnknownError 500. ``read_refusal`` and -``reference_refusal`` turn the store's errors into them, writing each detail -from the request rather than from an error's text, since the Public API -passes a refusal's detail on to its callers. +EconomySubmissionOutcomeUnknownError 500; a re-run also raises +EconomyRecordNotFoundError 404 and EconomyRerunConflictError 409. ``read_refusal``, +``reference_refusal`` and ``rerun_refusals`` turn the store's errors into +them, and ``call_refused`` builds the 503 for a call Modal did not accept, +writing each detail from the request rather than from an error's text, +since the Public API passes a refusal's detail on to its callers. """ from __future__ import annotations +from contextlib import contextmanager from dataclasses import dataclass from typing import TYPE_CHECKING, Literal @@ -18,6 +21,8 @@ MissingPolicyParametersError, OverlappingReformError, ReportSideError, + RerunRecordDeletedError, + RunInProgressError, UnknownCatalogVersionError, UnknownDatasetError, UnknownDynamicError, @@ -26,6 +31,7 @@ ) if TYPE_CHECKING: + from collections.abc import Iterator from uuid import UUID from policyengine_simulation_persistence import ( @@ -35,6 +41,7 @@ ) from policyengine_simulation_entry.economy_api_models import ( + EconomyConflictCode, EconomyRefusalCode, EconomyUnavailableCode, EconomyUnprocessableCode, @@ -42,9 +49,12 @@ # Each column whose foreign key can refuse a submission's rows: the code the # refusal answers with and the row it names. A region, dataset or dynamic is -# the caller's own choice; any other row is one the Public API checked first, -# or one the service chose, so its refusal is the service's state. +# the caller's own choice, or on a re-run the stored record's; any other row +# is one the Public API checked first, or one the service chose, so its +# refusal is the service's state. A simulation is refused only on a re-run, +# whose new runs name the report's stored simulations. REFERENCE_REFUSALS: dict[str, tuple[EconomyRefusalCode, str]] = { + "simulation_id": ("missing_reference", "simulation"), "region_id": ("unknown_region", "region"), "dataset_id": ("unknown_dataset", "dataset"), "dynamic_id": ("unknown_dynamic", "dynamic"), @@ -53,10 +63,28 @@ "tax_benefit_model_version_id": ("unknown_catalog_version", "catalog version"), "tax_benefit_model_id": ("unknown_catalog_version", "tax-benefit model"), } -# What a submission creates. +# What a submission creates, or a re-run calculates again. RecordName = Literal["report", "simulation"] +@dataclass(frozen=True) +class RefusalSource: + """Where refused values come from, as a refusal names and locates them. + + ``named_by`` says what names the values in a refusal's detail, and + ``location`` starts each unprocessable reason's ``loc``. + """ + + named_by: str + location: str + + +# A new record's request body, or a stored record a re-run reads. +REQUEST_BODY = RefusalSource(named_by="the request", location="body") +STORED_REPORT = RefusalSource(named_by="the report", location="report") +STORED_SIMULATION = RefusalSource(named_by="the simulation", location="simulation") + + class EconomySubmissionRefusedError(Exception): """A row the submission names, or a function it needs, is missing (400).""" @@ -99,21 +127,52 @@ class EconomySubmissionOutcomeUnknownError(Exception): The commit failed after the rows were inserted. The route answers 500, which the Public API reads as a record that may exist, never a 503, - which it reads as nothing created. + which it reads as nothing created. ``rerun`` says the rows are a stored + record's new run, not a new record. """ - def __init__(self, record: RecordName, *, error_type: str) -> None: + def __init__( + self, record: RecordName, *, error_type: str, rerun: bool = False + ) -> None: """Keep what the submission creates, and the commit error's type.""" super().__init__(f"the {record} may have been created") self.record: RecordName = record self.error_type = error_type + self.rerun = rerun + + +class EconomyRecordNotFoundError(LookupError): + """A re-run names no stored report or simulation (404).""" + + def __init__(self, record: RecordName) -> None: + """Keep which kind of record is missing.""" + super().__init__(f"no such {record}") + self.record: RecordName = record + + +class EconomyRerunConflictError(Exception): + """A stored record is not re-run now (409); nothing was written. + + ``run_id`` names the record's run pending or running, with + ``run_in_progress``. + """ + + def __init__( + self, *, code: EconomyConflictCode, detail: str, run_id: UUID | None = None + ) -> None: + """Keep the reason's code, its description and any run under way.""" + super().__init__(detail) + self.code: EconomyConflictCode = code + self.detail = detail + self.run_id = run_id class EconomySubmissionUnavailableError(Exception): """A dependency could not be read or refused the call; ask again later. - ``record_id`` names the report or simulation that was created and whose - run was failed, when one was. + ``record_id`` names the report or simulation that was created, or + re-run, and whose new run was failed, when one was; ``run_id`` names a + re-run's failed run. """ def __init__( @@ -122,41 +181,52 @@ def __init__( code: EconomyUnavailableCode, detail: str, record_id: UUID | None = None, + run_id: UUID | None = None, ) -> None: """Keep the reason's code, its description and any created record.""" super().__init__(detail) self.code: EconomyUnavailableCode = code self.detail = detail self.record_id = record_id + self.run_id = run_id def read_refusal( - error: Exception, request: EconomyRequest + error: Exception, + request: EconomyRequest, + source: RefusalSource = REQUEST_BODY, ) -> EconomySubmissionRefusedError | EconomySubmissionUnprocessableError | None: """Return the refusal a store read's error means, or None for any other error. A missing catalog version, region, dataset, policy or dynamic is refused (400); a policy or dynamic setting parameters the catalog version lacks, or both setting one, cannot be calculated (422). A report side's refusal - (ReportSideError) is located on that side's fields. Every detail is + (ReportSideError) is located on that side's fields, under ``source``: + the request's body, or the stored record a re-run reads. Every detail is written from ``request``, never from the error's text. """ side: ReportSide | None = None if isinstance(error, ReportSideError): side, error = error.side, error.error - return _missing_row_refusal(error, request, side) or _unprocessable( - error, request, side + return _missing_row_refusal(error, request, side, source) or _unprocessable( + error, request, side, source ) def _missing_row_refusal( - error: Exception, request: EconomyRequest, side: ReportSide | None + error: Exception, + request: EconomyRequest, + side: ReportSide | None, + source: RefusalSource, ) -> EconomySubmissionRefusedError | None: """Return the 400 a missing row means, or None for another error.""" catalog = ( f"{request.country}'s catalog version for .py {request.policyengine_version}" ) - named_by = "the request" if side is None else f"the request's {side}" + named_by = source.named_by if side is None else f"{source.named_by}'s {side}" + # A stored record's region and dataset are named by the record, not asked + # for; a request's are the caller's own. + stored_by = "" if source is REQUEST_BODY else f", which {source.named_by} names" refusal: tuple[EconomyRefusalCode, str] | None = None if isinstance(error, UnknownCatalogVersionError): refusal = ( @@ -165,14 +235,17 @@ def _missing_row_refusal( f"{request.policyengine_version}, the default v2 worker's version.", ) elif isinstance(error, UnknownRegionError): - refusal = ("unknown_region", f"{catalog} has no region {request.region_code}.") + refusal = ( + "unknown_region", + f"{catalog} has no region {request.region_code}{stored_by}.", + ) elif isinstance(error, UnknownDatasetError): dataset = ( f"default dataset for region {request.region_code}" if request.dataset_name is None else f"dataset {request.dataset_name}" ) - refusal = ("unknown_dataset", f"{catalog} has no {dataset}.") + refusal = ("unknown_dataset", f"{catalog} has no {dataset}{stored_by}.") elif isinstance(error, UnknownPolicyError): refusal = ( "unknown_policy", @@ -190,14 +263,19 @@ def _missing_row_refusal( def _unprocessable( - error: Exception, request: EconomyRequest, side: ReportSide | None + error: Exception, + request: EconomyRequest, + side: ReportSide | None, + source: RefusalSource, ) -> EconomySubmissionUnprocessableError | None: """Return the 422 refused values mean, or None for another error. Each reason is located at the field that names the values: the request's - own, or the report side's. + own or the stored record's, or the report side's. """ - location: tuple[str, ...] = ("body",) if side is None else ("body", side) + location: tuple[str, ...] = ( + (source.location,) if side is None else (source.location, side) + ) if isinstance(error, MissingPolicyParametersError): field_name = "policy_id" if error.source == "policy" else "dynamic_id" return EconomySubmissionUnprocessableError( @@ -230,15 +308,66 @@ def _unprocessable( def reference_refusal( - error: EconomyRecordReferenceError, + error: EconomyRecordReferenceError, source: RefusalSource = REQUEST_BODY ) -> EconomySubmissionRefusedError: """Return the refusal of rows the database refused for naming a missing row. - It names the kind of row, never its ID, which the caller sent. + It names the kind of row, never its ID, which the caller sent, and what + named it: the request, or the stored record a re-run reads. """ code, row = REFERENCE_REFUSALS.get( error.column_name or "", ("missing_reference", "row") ) return EconomySubmissionRefusedError( - code=code, detail=f"a {row} the request names no longer exists" + code=code, detail=f"a {row} {source.named_by} names no longer exists" ) + + +def call_refused( + record: RecordName, record_id: UUID, *, rerun_id: UUID | None = None +) -> EconomySubmissionUnavailableError: + """Return the 503 naming a record whose new run's call Modal did not accept. + + ``rerun_id`` names a stored record's new run, which was failed. + """ + if rerun_id is not None: + return EconomySubmissionUnavailableError( + code="modal_call_refused", + detail=( + f"Modal did not accept the call of the {record}'s new run; " + "that run was failed." + ), + record_id=record_id, + run_id=rerun_id, + ) + return EconomySubmissionUnavailableError( + code="modal_call_refused", + detail=( + f"Modal did not accept the {record}'s call; the {record} was created " + "and its run failed." + ), + record_id=record_id, + ) + + +@contextmanager +def rerun_refusals(record: RecordName) -> Iterator[None]: + """Answer a re-run's create that the record's state refused. + + A record deleted after the re-run read it is not found (404); a record + with a run pending or running is not re-run now (409 ``run_in_progress``, + naming that run). Nothing was created either way. + """ + try: + yield + except RerunRecordDeletedError: + raise EconomyRecordNotFoundError(record) from None + except RunInProgressError as error: + raise EconomyRerunConflictError( + code="run_in_progress", + detail=( + f"The {record} has a run pending or running; re-run it once " + "that run has ended." + ), + run_id=error.run_id, + ) from None diff --git a/projects/policyengine-simulation-entry/src/policyengine_simulation_entry/economy_submissions.py b/projects/policyengine-simulation-entry/src/policyengine_simulation_entry/economy_submissions.py index a6011bae7..e9fef6726 100644 --- a/projects/policyengine-simulation-entry/src/policyengine_simulation_entry/economy_submissions.py +++ b/projects/policyengine-simulation-entry/src/policyengine_simulation_entry/economy_submissions.py @@ -69,10 +69,13 @@ simulation_run_calculation, ) from policyengine_simulation_entry.economy_submission_errors import ( + REQUEST_BODY, EconomySubmissionOutcomeUnknownError, EconomySubmissionRefusedError, EconomySubmissionUnavailableError, RecordName, + RefusalSource, + call_refused, read_refusal, reference_refusal, ) @@ -214,17 +217,18 @@ def simulation_choice(policy: PolicyChoice) -> SimulationChoice: return SimulationChoice(policy_id=policy.policy_id, dynamic_id=policy.dynamic_id) -class EconomySubmissionService: +class EconomySubmissionService[Store: EconomyRuns = EconomyRuns]: """Create economy reports and simulations, and start their calls. The database and the manifest are read on threads, so the event loop - never waits on them. + never waits on them. ``Store`` is the store's type: a subclass that + reads more of the database names a protocol with more methods. """ def __init__( # noqa: PLR0913 # each seam tests replace self, *, - store: EconomyRuns, + store: Store, workers: EconomyWorkerSource, caller: EconomyRunCaller, settings: ServiceSettings, @@ -310,7 +314,12 @@ async def submit_report( ) await self._create("report", lambda: self._store.create_report(report)) status = await self._start_report_run( - worker, report, calculation, observability_context=observability_context + worker, + calculation, + report_id=report.id, + run_id=report.run_id, + new_simulation_run_ids=report.new_simulation_run_ids, + observability_context=observability_context, ) return EconomyReportCreated( report_id=report.id, @@ -365,8 +374,9 @@ async def submit_simulation( ) status = await self._start_simulation_run( worker, - simulation, calculation, + simulation_id=simulation.id, + run_id=simulation.run_id, observability_context=observability_context, ) return EconomySimulationCreated( @@ -449,9 +459,13 @@ async def _default_worker(self, country: str) -> EconomyWorker: ) from None async def _read_inputs[Inputs]( - self, read: Callable[[], Inputs], *, request: EconomyRequest + self, + read: Callable[[], Inputs], + *, + request: EconomyRequest, + source: RefusalSource = REQUEST_BODY, ) -> Inputs: - """Run ``read`` on a thread; answer each refusal as the request's error.""" + """Run ``read`` on a thread; answer each refusal as ``source``'s error.""" try: return await asyncio.to_thread(read) except SQLAlchemyError as error: @@ -461,26 +475,34 @@ async def _read_inputs[Inputs]( ) raise _database_unavailable() from None except Exception as error: - refusal = read_refusal(error, request) + refusal = read_refusal(error, request, source) if refusal is None: raise raise refusal from None - async def _create(self, record: RecordName, create: Callable[[], None]) -> None: + async def _create( + self, + record: RecordName, + create: Callable[[], None], + *, + rerun: bool = False, + source: RefusalSource = REQUEST_BODY, + ) -> None: """Create the rows; answer a refusal, an unknown outcome or an outage. A commit that failed after the rows were inserted may have written them, so it is never answered as unavailable, which would tell the - caller nothing was created. + caller nothing was created. ``rerun`` says the rows are a stored + record's new run, and ``source`` what names a refused row. """ try: await asyncio.to_thread(create) except EconomyRecordReferenceError as error: - raise reference_refusal(error) from None + raise reference_refusal(error, source) from None except EconomyRecordOutcomeUnknownError as error: # The route logs it, with the commit error's type. raise EconomySubmissionOutcomeUnknownError( - record, error_type=type(error.__cause__).__name__ + record, error_type=type(error.__cause__).__name__, rerun=rerun ) from None except SQLAlchemyError as error: logger.warning( @@ -489,14 +511,23 @@ async def _create(self, record: RecordName, create: Callable[[], None]) -> None: ) raise _database_unavailable() from None - async def _start_report_run( + async def _start_report_run( # noqa: PLR0913 # the call, then the runs it calculates self, worker: EconomyWorker, - report: NewEconomyReport, calculation: RunCalculation, *, + report_id: UUID, + run_id: UUID, + new_simulation_run_ids: Sequence[UUID], observability_context: dict[str, Any] | None, + rerun: bool = False, ) -> ReportRunStatus: + """Start the report run's coordinator and record its call, or fail the run. + + ``new_simulation_run_ids`` are the simulation runs created with the + report run, which fail with it; ``rerun`` says the report existed + before its new run. + """ try: call_id = await asyncio.wait_for( self._caller.start_report_run( @@ -512,31 +543,39 @@ async def _start_report_run( except Exception as error: # noqa: BLE001 # every unconfirmed call is answered alike failed = await self._fail_unconfirmed( "report", - report.id, + report_id, error, lambda message: self._store.fail_unstarted_report( - report.run_id, report.new_simulation_run_ids, message + run_id, new_simulation_run_ids, message ), ) if failed: - refused = call_refused("report", report.id) + refused = call_refused( + "report", report_id, rerun_id=run_id if rerun else None + ) raise refused from None return ReportRunStatus.PENDING status = await self._record_call( "report", - report.id, - lambda: self._store.record_report_call(report.run_id, call_id), + report_id, + lambda: self._store.record_report_call(run_id, call_id), ) return ReportRunStatus.PENDING if status is None else ReportRunStatus(status) - async def _start_simulation_run( + async def _start_simulation_run( # noqa: PLR0913 # the call, then the run it calculates self, worker: EconomyWorker, - simulation: NewEconomySimulation, calculation: RunCalculation, *, + simulation_id: UUID, + run_id: UUID, observability_context: dict[str, Any] | None, + rerun: bool = False, ) -> SimulationStatus: + """Start the simulation run's function and record its call, or fail the run. + + ``rerun`` says the simulation existed before its new run. + """ try: call_id = await asyncio.wait_for( self._caller.start_simulation_run( @@ -551,20 +590,20 @@ async def _start_simulation_run( except Exception as error: # noqa: BLE001 # every unconfirmed call is answered alike failed = await self._fail_unconfirmed( "simulation", - simulation.id, + simulation_id, error, - lambda message: self._store.fail_unstarted_simulation( - simulation.run_id, message - ), + lambda message: self._store.fail_unstarted_simulation(run_id, message), ) if failed: - refused = call_refused("simulation", simulation.id) + refused = call_refused( + "simulation", simulation_id, rerun_id=run_id if rerun else None + ) raise refused from None return SimulationStatus.PENDING status = await self._record_call( "simulation", - simulation.id, - lambda: self._store.record_simulation_call(simulation.run_id, call_id), + simulation_id, + lambda: self._store.record_simulation_call(run_id, call_id), ) return SimulationStatus.PENDING if status is None else SimulationStatus(status) @@ -635,20 +674,6 @@ async def _record_call( return None if status is None else str(status) -def call_refused( - record: RecordName, record_id: UUID -) -> EconomySubmissionUnavailableError: - """Return the 503 naming a record whose run's call Modal did not accept.""" - return EconomySubmissionUnavailableError( - code="modal_call_refused", - detail=( - f"Modal did not accept the {record}'s call; the {record} was created " - "and its run failed." - ), - record_id=record_id, - ) - - def economy_request_for( request: EconomyReportRequest | EconomySimulationRequest diff --git a/projects/policyengine-simulation-entry/src/policyengine_simulation_entry/household_report_api_models.py b/projects/policyengine-simulation-entry/src/policyengine_simulation_entry/household_report_api_models.py index 11e613c9c..d3764b54b 100644 --- a/projects/policyengine-simulation-entry/src/policyengine_simulation_entry/household_report_api_models.py +++ b/projects/policyengine-simulation-entry/src/policyengine_simulation_entry/household_report_api_models.py @@ -65,15 +65,16 @@ def options(self) -> dict[str, Any]: class HouseholdReportCreated(BaseModel): - """The household report a request created, its first run and that run's status. + """The household report a request created or re-ran, its new run and its status. The status is the run's when its call's ID was recorded: ``pending``, or whatever the household function had made it by then if it claimed the run first. A run whose call Modal didn't confirm, or whose call's ID - couldn't be recorded, is answered ``pending``. ``year`` is the - household's default year, which the report calculates. Each side names - its new simulation and simulation run; the reform's are null for a - report of the baseline alone. + couldn't be recorded, is answered ``pending``. ``year`` is the year the + report calculates: a new report's is its household's default year, and a + re-run keeps the report's. Each side names its simulation, new for a new + report and the report's own on a re-run, and its new simulation run; the + reform's are null for a report of the baseline alone. """ model_config = ConfigDict(extra="forbid", frozen=True) diff --git a/projects/policyengine-simulation-entry/src/policyengine_simulation_entry/household_report_reruns.py b/projects/policyengine-simulation-entry/src/policyengine_simulation_entry/household_report_reruns.py new file mode 100644 index 000000000..95e0f9980 --- /dev/null +++ b/projects/policyengine-simulation-entry/src/policyengine_simulation_entry/household_report_reruns.py @@ -0,0 +1,285 @@ +"""Re-run a stored household report on the current .py version. + +A household report's re-run calculates its stored household again, in the +report's own year, under each side's stored policy, as a new run, and keeps +every earlier run. The re-run service (``economy_reruns``) reads the stored +report and hands a household report to ``HouseholdReportRerunService``, +which: + +1. finds the v2 version manifest's default worker's household function for + the country; +2. reads, in one transaction, the country's catalog version for that + worker's .py version, its model, the stored household and each side's + policy values, and builds each side's household contract request with + the side's new simulation run ID as its calculation ID; +3. creates, in one transaction, the report's new run (trigger ``manual``) + and a new run of each side's own household simulation, all pending with + no Modal call ID, unless the report has a run pending or running; +4. starts one household function call for the new report run and records + its ID, as for a new report, failing the new runs when Modal refuses the + call. + +A household run is never reused, so a forced re-run creates the same runs. +The re-run service hands over only a report naming each row this needs and +no dynamic, which a household calculation does not apply. Whatever is +missing or cannot be calculated is refused in steps 1 and 2, before +anything is written, located at the stored report. +""" + +from __future__ import annotations + +from dataclasses import dataclass, replace +from typing import TYPE_CHECKING, Any, Protocol + +from policyengine_simulation_persistence import ( + HouseholdReportStore, + HouseholdRerunSide, + NewHouseholdReportRerun, + RunVersions, +) + +from policyengine_simulation_entry.economy_submission_errors import ( + REQUEST_BODY, + STORED_REPORT, + EconomySubmissionUnprocessableError, + rerun_refusals, +) +from policyengine_simulation_entry.household_report_submissions import ( + HouseholdReports, + HouseholdReportSideInputs, + HouseholdReportSubmissionService, + household_report_calculation, + household_report_created, +) +from policyengine_simulation_entry.household_workers import ( + ManifestHouseholdWorkers, + ModalHouseholdFunctionCaller, +) +from policyengine_simulation_entry.simulation_database import SimulationDatabase + +if TYPE_CHECKING: + from uuid import UUID + + from policyengine_simulation_contract.household_report_execution import ( + HouseholdReportRunCalculation, + ) + from policyengine_simulation_contract.stage12_bundle import CountryId + from policyengine_simulation_persistence import ( + PolicyReform, + ReportSide, + SideDefinition, + StoredHousehold, + ) + + from policyengine_simulation_entry.config import Settings + from policyengine_simulation_entry.economy_submission_errors import ( + UnprocessableReason, + ) + from policyengine_simulation_entry.entry_runtime import ( + SimulationDatabaseConnection, + ) + from policyengine_simulation_entry.household_report_api_models import ( + HouseholdReportCreated, + ) + from policyengine_simulation_entry.household_workers import HouseholdWorker + +# How long one of the service's statements may run, as a household job's. +STATEMENT_TIMEOUT_MILLISECONDS = 5_000 + + +@dataclass(frozen=True) +class StoredHouseholdReport: + """A stored household report a re-run calculates again, each row it needs present. + + ``baseline`` and ``reform`` are the report's own household simulations, + with their stored policies; ``reform`` is None for a report of the + baseline alone. Neither names a dynamic, which a household calculation + does not apply: the re-run service refuses a report whose side does. + """ + + report_id: UUID + country: CountryId + household_id: UUID + year: int + baseline: SideDefinition + reform: SideDefinition | None + + @property + def sides(self) -> dict[ReportSide, SideDefinition]: + """Return each side the report calculates, the baseline's first.""" + sides: dict[ReportSide, SideDefinition] = {"baseline": self.baseline} + if self.reform is not None: + sides["reform"] = self.reform + return sides + + +class HouseholdReportRerunRows(HouseholdReports, Protocol): + """The household report reads and writes, with a stored report's re-run. + + ``HouseholdReportStore`` is one. + """ + + def create_report_rerun(self, rerun: NewHouseholdReportRerun) -> None: + """Create the report's new run and each side's new simulation run.""" + ... + + +class HouseholdReportReruns(Protocol): + """Re-run stored household reports (``HouseholdReportRerunService``).""" + + async def rerun_report( + self, + stored: StoredHouseholdReport, + *, + observability_context: dict[str, Any] | None, + ) -> HouseholdReportCreated: + """Create the stored report's new run, and start its household function call.""" + ... + + +class HouseholdReportRerunService( + HouseholdReportSubmissionService[HouseholdReportRerunRows] +): + """Create and re-run household reports; each run is one household function call.""" + + async def rerun_report( + self, + stored: StoredHouseholdReport, + *, + observability_context: dict[str, Any] | None, + ) -> HouseholdReportCreated: + """Create the stored report's new run on the current versions, and start it. + + Raise EconomyRecordNotFoundError (404) for a report deleted after it + was read, EconomyRerunConflictError (409 ``run_in_progress``) while + it has a run pending or running, and otherwise the errors + ``submit_report`` raises, located at the stored report; a 503 for + Modal's refusal names the new run as well as the report. + """ + worker = await self._default_worker(stored.country) + inputs = await self._read_inputs( + country=stored.country, + household_id=stored.household_id, + policy_ids={ + side: definition.choice.policy_id + for side, definition in stored.sides.items() + }, + worker=worker, + source=STORED_REPORT, + ) + run_id = self._new_id() + new_sides = { + side: HouseholdRerunSide( + simulation_id=definition.simulation_id, + simulation_run_id=self._new_id(), + ) + for side, definition in stored.sides.items() + } + rerun = NewHouseholdReportRerun( + report_id=stored.report_id, + run_id=run_id, + tax_benefit_model_version_id=inputs.catalog_version_id, + year=stored.year, + versions=RunVersions( + country_package_version=worker.bundle.country_package_version, + policyengine_version=worker.bundle.policyengine_version, + ), + baseline=new_sides["baseline"], + reform=new_sides.get("reform"), + ) + calculation = _stored_report_calculation( + inputs.household, rerun, reforms=inputs.reforms, worker=worker + ) + with rerun_refusals("report"): + await self._create( + lambda: self._store.create_report_rerun(rerun), + rerun=True, + source=STORED_REPORT, + ) + status = await self._start_report_run( + worker, + calculation, + report_id=rerun.report_id, + run_id=rerun.run_id, + simulation_run_ids=rerun.simulation_run_ids, + observability_context=observability_context, + rerun=True, + ) + return household_report_created( + report_id=rerun.report_id, + run_id=rerun.run_id, + status=status, + year=rerun.year, + baseline=rerun.baseline, + reform=rerun.reform, + ) + + +def _stored_report_calculation( + household: StoredHousehold, + rerun: NewHouseholdReportRerun, + *, + reforms: dict[ReportSide, PolicyReform], + worker: HouseholdWorker, +) -> HouseholdReportRunCalculation: + """Build the re-run's household function payload, in the stored report's year. + + A household or policy the contract refuses is located at the stored + report's fields rather than a request's. + """ + + def side_inputs( + side: HouseholdRerunSide, name: ReportSide + ) -> HouseholdReportSideInputs: + return HouseholdReportSideInputs( + simulation_run_id=side.simulation_run_id, policy_values=reforms[name] + ) + + try: + return household_report_calculation( + household=household, + year=rerun.year, + baseline=side_inputs(rerun.baseline, "baseline"), + reform=None + if rerun.reform is None + else side_inputs(rerun.reform, "reform"), + worker=worker, + ) + except EconomySubmissionUnprocessableError as error: + raise EconomySubmissionUnprocessableError( + code=error.code, + reasons=tuple(_at_stored_report(reason) for reason in error.reasons), + ) from None + + +def _at_stored_report(reason: UnprocessableReason) -> UnprocessableReason: + """Locate a reason at the stored report's field rather than a request's.""" + if reason.loc[:1] != (REQUEST_BODY.location,): + return reason + return replace(reason, loc=(STORED_REPORT.location, *reason.loc[1:])) + + +def household_reports_for( + settings: Settings, database: SimulationDatabaseConnection | None +) -> HouseholdReportRerunService | None: + """Build the household report service as the household calculation one is. + + It creates and re-runs household reports. It needs the simulation + database and the v2 Modal environment; without them household reports + answer 503. Nothing is read from Modal until the first submission. + """ + if not settings.household_calculations_configured or not isinstance( + database, SimulationDatabase + ): + return None + return HouseholdReportRerunService( + store=HouseholdReportStore.from_engine( + database.engine, + statement_timeout_milliseconds=STATEMENT_TIMEOUT_MILLISECONDS, + ), + workers=ManifestHouseholdWorkers.from_modal( + manifest_name=settings.v2_manifest_name, + environment=settings.v2_modal_environment, + ), + caller=ModalHouseholdFunctionCaller(settings.v2_modal_environment), + ) diff --git a/projects/policyengine-simulation-entry/src/policyengine_simulation_entry/household_report_submissions.py b/projects/policyengine-simulation-entry/src/policyengine_simulation_entry/household_report_submissions.py index 987be2b7d..768e83cfd 100644 --- a/projects/policyengine-simulation-entry/src/policyengine_simulation_entry/household_report_submissions.py +++ b/projects/policyengine-simulation-entry/src/policyengine_simulation_entry/household_report_submissions.py @@ -27,7 +27,8 @@ finishes the report run; the stuck-run sweep fails a pending run no call was recorded for. The errors are the economy routes' (``economy_submission_errors``), so ``POST /reports`` answers both kinds -alike. +alike. A stored report's re-run (``household_report_reruns``) takes the same +steps through this service's reads, writes and call. """ from __future__ import annotations @@ -48,7 +49,6 @@ EconomyRecordOutcomeUnknownError, EconomyRecordReferenceError, HouseholdReportSimulation, - HouseholdReportStore, MissingPolicyParametersError, NewHouseholdReport, ReportRunStatus, @@ -62,11 +62,13 @@ from sqlalchemy.exc import SQLAlchemyError from policyengine_simulation_entry.economy_submission_errors import ( + REQUEST_BODY, EconomySubmissionOutcomeUnknownError, EconomySubmissionRefusedError, EconomySubmissionUnavailableError, EconomySubmissionUnprocessableError, UnprocessableReason, + call_refused, reference_refusal, ) from policyengine_simulation_entry.household_contract_fields import ( @@ -83,11 +85,8 @@ ) from policyengine_simulation_entry.household_workers import ( HouseholdFunctionUnavailableError, - ManifestHouseholdWorkers, - ModalHouseholdFunctionCaller, VersionManifestUnavailableError, ) -from policyengine_simulation_entry.simulation_database import SimulationDatabase if TYPE_CHECKING: from collections.abc import Callable, Mapping, Sequence @@ -99,10 +98,9 @@ StoredHousehold, ) - from policyengine_simulation_entry.config import Settings - from policyengine_simulation_entry.economy_submission_errors import RecordName - from policyengine_simulation_entry.entry_runtime import ( - SimulationDatabaseConnection, + from policyengine_simulation_entry.economy_submission_errors import ( + RecordName, + RefusalSource, ) from policyengine_simulation_entry.household_report_api_models import ( HouseholdReportRequest, @@ -118,8 +116,6 @@ # How long a submission waits for Modal to accept the household function's # call before failing the report run. CALL_START_TIMEOUT_SECONDS = 10.0 -# How long one of the service's statements may run, as a household job's. -STATEMENT_TIMEOUT_MILLISECONDS = 5_000 class HouseholdReports(Protocol): @@ -156,6 +152,45 @@ def fail_unstarted_report( ... +class HouseholdReportSideRuns(Protocol): + """A report side's simulation and the simulation run its report run names.""" + + @property + def simulation_id(self) -> UUID: + """Return the side's simulation.""" + ... + + @property + def simulation_run_id(self) -> UUID: + """Return the side's simulation run.""" + ... + + +def household_report_created( # noqa: PLR0913 # the report, its run and each side's + *, + report_id: UUID, + run_id: UUID, + status: ReportRunStatus, + year: int, + baseline: HouseholdReportSideRuns, + reform: HouseholdReportSideRuns | None, +) -> HouseholdReportCreated: + """Return the 201 answer naming the report, its new run and each side's runs. + + ``reform`` is None for a report of the baseline alone. + """ + return HouseholdReportCreated( + report_id=report_id, + report_run_id=run_id, + status=status, + year=year, + baseline_simulation_id=baseline.simulation_id, + baseline_simulation_run_id=baseline.simulation_run_id, + reform_simulation_id=None if reform is None else reform.simulation_id, + reform_simulation_run_id=None if reform is None else reform.simulation_run_id, + ) + + class HouseholdReportBuildError(Exception): """The household function's contract refused a field the service built. @@ -187,24 +222,30 @@ def _policy_ids(request: HouseholdReportRequest) -> dict[ReportSide, UUID | None def _read_refusal( - error: Exception, request: HouseholdReportRequest, policyengine_version: str + error: Exception, + *, + country: str, + policyengine_version: str, + source: RefusalSource, ) -> EconomySubmissionRefusedError | EconomySubmissionUnprocessableError | None: """Return the refusal a store read's error means, or None for another error. - Every detail is written from the request, never from the error's text. + ``source`` is what names the household and policies: the request, or + the stored report a re-run reads. Every detail is written from it, + never from the error's text. """ if isinstance(error, UnknownCatalogVersionError): return EconomySubmissionRefusedError( code="unknown_catalog_version", detail=( - f"{request.country} has no catalog version for .py " + f"{country} has no catalog version for .py " f"{policyengine_version}, the default v2 worker's version." ), ) if isinstance(error, UnknownHouseholdError): return EconomySubmissionRefusedError( code="unknown_household", - detail=f"{request.country} has no stored household the request names.", + detail=f"{country} has no stored household {source.named_by} names.", ) if not isinstance(error, ReportSideError): return None @@ -213,7 +254,7 @@ def _read_refusal( return EconomySubmissionRefusedError( code="unknown_policy", detail=( - f"{request.country} has no stored policy the request's {side} names." + f"{country} has no stored policy {source.named_by}'s {side} names." ), ) if isinstance(refused, MissingPolicyParametersError): @@ -221,7 +262,7 @@ def _read_refusal( code="missing_parameters", reasons=tuple( UnprocessableReason( - loc=("body", side, "policy_id", name), + loc=(source.location, side, "policy_id", name), msg=( f"the catalog version for .py {policyengine_version} has " "no such parameter" @@ -398,17 +439,18 @@ def _report_year(household: StoredHousehold) -> int: return household.default_year -class HouseholdReportSubmissionService: +class HouseholdReportSubmissionService[Store: HouseholdReports = HouseholdReports]: """Create household reports and start the household function call for each. The database and the manifest are read on threads, so the event loop - never waits on them. + never waits on them. ``Store`` is the store's type: a subclass that + writes more of the database names a protocol with more methods. """ def __init__( self, *, - store: HouseholdReports, + store: Store, workers: HouseholdWorkerSource, caller: HouseholdReportRunCaller, call_start_timeout_seconds: float = CALL_START_TIMEOUT_SECONDS, @@ -439,25 +481,29 @@ async def submit_report( (500) when the service built a request the contract refuses. """ worker = await self._default_worker(request.country) - inputs = await self._read_inputs(request, worker) + inputs = await self._read_inputs( + country=request.country, + household_id=request.household_id, + policy_ids=_policy_ids(request), + worker=worker, + ) report, calculation = self._prepare(request, worker, inputs) - await self._create(report) + await self._create(lambda: self._store.create_report(report)) status = await self._start_report_run( - worker, report, calculation, observability_context=observability_context + worker, + calculation, + report_id=report.id, + run_id=report.run_id, + simulation_run_ids=report.simulation_run_ids, + observability_context=observability_context, ) - return HouseholdReportCreated( + return household_report_created( report_id=report.id, - report_run_id=report.run_id, + run_id=report.run_id, status=status, year=report.year, - baseline_simulation_id=report.baseline.simulation_id, - baseline_simulation_run_id=report.baseline.simulation_run_id, - reform_simulation_id=( - None if report.reform is None else report.reform.simulation_id - ), - reform_simulation_run_id=( - None if report.reform is None else report.reform.simulation_run_id - ), + baseline=report.baseline, + reform=report.reform, ) async def _default_worker(self, country: str) -> HouseholdWorker: @@ -481,15 +527,22 @@ async def _default_worker(self, country: str) -> HouseholdWorker: ) from None async def _read_inputs( - self, request: HouseholdReportRequest, worker: HouseholdWorker + self, + *, + country: str, + household_id: UUID, + policy_ids: Mapping[ReportSide, UUID | None], + worker: HouseholdWorker, + source: RefusalSource = REQUEST_BODY, ) -> HouseholdReportInputs: + """Read the report's inputs; answer each refusal as ``source``'s error.""" policyengine_version = worker.bundle.policyengine_version try: return await asyncio.to_thread( lambda: self._store.read_inputs( - country=request.country, - household_id=request.household_id, - policy_ids=_policy_ids(request), + country=country, + household_id=household_id, + policy_ids=policy_ids, policyengine_version=policyengine_version, ) ) @@ -500,7 +553,12 @@ async def _read_inputs( ) raise _database_unavailable() from None except Exception as error: - refusal = _read_refusal(error, request, policyengine_version) + refusal = _read_refusal( + error, + country=country, + policyengine_version=policyengine_version, + source=source, + ) if refusal is None: raise raise refusal from None @@ -557,22 +615,29 @@ def _prepare( ) return report, calculation - async def _create(self, report: NewHouseholdReport) -> None: + async def _create( + self, + create: Callable[[], None], + *, + rerun: bool = False, + source: RefusalSource = REQUEST_BODY, + ) -> None: """Create the rows; answer a refusal, an unknown outcome or an outage. A commit that failed after the rows were inserted may have written them, so it is never answered as unavailable, which would tell the - caller nothing was created. + caller nothing was created. ``rerun`` says the rows are a stored + report's new run, and ``source`` what names a refused row. """ try: - await asyncio.to_thread(self._store.create_report, report) + await asyncio.to_thread(create) except EconomyRecordReferenceError as error: - raise reference_refusal(error) from None + raise reference_refusal(error, source) from None except EconomyRecordOutcomeUnknownError as error: # The route logs it, with the commit error's type. record: RecordName = "report" raise EconomySubmissionOutcomeUnknownError( - record, error_type=type(error.__cause__).__name__ + record, error_type=type(error.__cause__).__name__, rerun=rerun ) from None except SQLAlchemyError as error: logger.warning( @@ -581,20 +646,29 @@ async def _create(self, report: NewHouseholdReport) -> None: ) raise _database_unavailable() from None - async def _start_report_run( + async def _start_report_run( # noqa: PLR0913 # the call, then the runs it calculates self, worker: HouseholdWorker, - report: NewHouseholdReport, calculation: HouseholdReportRunCalculation, *, + report_id: UUID, + run_id: UUID, + simulation_run_ids: Sequence[UUID], observability_context: dict[str, Any] | None, + rerun: bool = False, ) -> ReportRunStatus: + """Start the report run's household function call and record it, or fail it. + + ``simulation_run_ids`` are the simulation runs created with the + report run, which fail with it; ``rerun`` says the report existed + before its new run. + """ try: call_id = await asyncio.wait_for( self._caller.start_report_run( worker, calculation.model_dump(mode="json"), - report_run_id=report.run_id, + report_run_id=run_id, observability_context=observability_context, ), timeout=self._call_start_timeout_seconds, @@ -602,19 +676,21 @@ async def _start_report_run( # Modal raises its own errors; any of them, or the timeout, means the # call was not confirmed, and the run is failed unless it has started. except Exception as error: # noqa: BLE001 # every unconfirmed call is answered alike - if await self._fail_unconfirmed(report, error): - raise EconomySubmissionUnavailableError( - code="modal_call_refused", - detail=( - "Modal did not accept the report's call; the report was " - "created and its run failed." - ), - record_id=report.id, - ) from None + failed = await self._fail_unconfirmed( + error, + report_id=report_id, + run_id=run_id, + simulation_run_ids=simulation_run_ids, + ) + if failed: + refused = call_refused( + "report", report_id, rerun_id=run_id if rerun else None + ) + raise refused from None return ReportRunStatus.PENDING try: status = await asyncio.to_thread( - self._store.record_report_call, report.run_id, call_id + self._store.record_report_call, run_id, call_id ) except SQLAlchemyError as error: # The household function claims a pending run with no call ID @@ -622,7 +698,7 @@ async def _start_report_run( logger.warning( "household_report_call_not_recorded", extra={ - "report_id": str(report.id), + "report_id": str(report_id), "error_type": type(error).__name__, }, ) @@ -630,7 +706,12 @@ async def _start_report_run( return ReportRunStatus.PENDING if status is None else status async def _fail_unconfirmed( - self, report: NewHouseholdReport, error: BaseException + self, + error: BaseException, + *, + report_id: UUID, + run_id: UUID, + simulation_run_ids: Sequence[UUID], ) -> bool: """Fail the report run of a call Modal didn't confirm, unless it started. @@ -643,7 +724,7 @@ async def _fail_unconfirmed( # The log names the error type only; its text could repeat request data. logger.error( "household_report_call_unconfirmed", - extra={"report_id": str(report.id), "error_type": type(error).__name__}, + extra={"report_id": str(report_id), "error_type": type(error).__name__}, ) message = ( "modal_call_refused: Modal did not accept the report's call " @@ -652,42 +733,16 @@ async def _fail_unconfirmed( try: return await asyncio.to_thread( self._store.fail_unstarted_report, - report.run_id, - report.simulation_run_ids, + run_id, + simulation_run_ids, message, ) except SQLAlchemyError as write_error: logger.warning( "household_report_run_not_failed", extra={ - "report_id": str(report.id), + "report_id": str(report_id), "error_type": type(write_error).__name__, }, ) return False - - -def household_reports_for( - settings: Settings, database: SimulationDatabaseConnection | None -) -> HouseholdReportSubmissionService | None: - """Build the household report service as the household calculation one is. - - It needs the simulation database and the v2 Modal environment; without - them household reports answer 503. Nothing is read from Modal until the - first submission. - """ - if not settings.household_calculations_configured or not isinstance( - database, SimulationDatabase - ): - return None - return HouseholdReportSubmissionService( - store=HouseholdReportStore.from_engine( - database.engine, - statement_timeout_milliseconds=STATEMENT_TIMEOUT_MILLISECONDS, - ), - workers=ManifestHouseholdWorkers.from_modal( - manifest_name=settings.v2_manifest_name, - environment=settings.v2_modal_environment, - ), - caller=ModalHouseholdFunctionCaller(settings.v2_modal_environment), - ) diff --git a/projects/policyengine-simulation-entry/tests/economy_rerun_fakes.py b/projects/policyengine-simulation-entry/tests/economy_rerun_fakes.py new file mode 100644 index 000000000..3bbb02623 --- /dev/null +++ b/projects/policyengine-simulation-entry/tests/economy_rerun_fakes.py @@ -0,0 +1,255 @@ +"""A fake of the economy store with re-runs, and stored records to re-run. + +The store's own reads and writes are tested against PostgreSQL in the +persistence library; here a fake records what the re-run service asks of it. +""" + +from __future__ import annotations + +from dataclasses import dataclass, field +from typing import TYPE_CHECKING, Any, NoReturn +from uuid import UUID + +from economy_service_fakes import ( + CREATED_AT, + DYNAMIC_ID, + OUTPUT_BUCKET, + POLICY_ID, + REFORM, + FakeCaller, + FakeWorkers, + numbered_ids, +) +from policyengine_simulation_entry.economy_reruns import EconomyRerunService +from policyengine_simulation_entry.economy_submissions import ServiceSettings +from policyengine_simulation_persistence import ( + BudgetWindowInputs, + EconomyReportInputs, + EconomySimulationInputs, + ReportDefinition, + SideDefinition, + SimulationChoice, + SimulationDefinition, + SimulationInputs, + WindowYearDefinition, +) +from test_budget_window_submissions import FakeAdvancer, FakeWindowStore + +if TYPE_CHECKING: + from collections.abc import Sequence + + from policyengine_simulation_entry.household_report_reruns import ( + HouseholdReportReruns, + ) + from policyengine_simulation_persistence import ( + EconomyRequest, + NewReportRerun, + NewSimulationRerun, + NewWindowRerun, + RunReuse, + ) + +REPORT_ID = UUID("00000000-0000-4000-8000-0000000000a1") +BASELINE_SIMULATION_ID = UUID("00000000-0000-4000-8000-0000000000a2") +REFORM_SIMULATION_ID = UUID("00000000-0000-4000-8000-0000000000a3") +SIMULATION_ID = UUID("00000000-0000-4000-8000-0000000000a4") +RUN_UNDER_WAY_ID = UUID("00000000-0000-4000-8000-0000000000a5") +CURRENT_LAW = SimulationChoice(policy_id=None, dynamic_id=None) +REFORM_CHOICE = SimulationChoice(policy_id=POLICY_ID, dynamic_id=DYNAMIC_ID) +STORED_OPTIONS: dict[str, Any] = { + "include_cliffs": True, + "spm": {"geography_kind": "national"}, +} +STORED_REPORT = ReportDefinition( + id=REPORT_ID, + country="us", + kind="economy", + year=2026, + region_code="us", + dataset_name="populace_us_2024", + inputs=STORED_OPTIONS, + baseline=SideDefinition(simulation_id=BASELINE_SIMULATION_ID, choice=CURRENT_LAW), + reform=SideDefinition(simulation_id=REFORM_SIMULATION_ID, choice=REFORM_CHOICE), +) +# A stored budget window of three years, each with its own simulations. +WINDOW_SIMULATION_IDS = { + year: ( + UUID(f"00000000-0000-4000-8000-00000000{year}"), + UUID(f"00000000-0000-4000-8000-00000001{year}"), + ) + for year in (2026, 2027, 2028) +} +STORED_WINDOW = ReportDefinition( + id=REPORT_ID, + country="us", + kind="budget_window", + year=2026, + region_code="us", + dataset_name="populace_us_2024", + inputs=STORED_OPTIONS, + baseline=SideDefinition( + simulation_id=WINDOW_SIMULATION_IDS[2026][0], choice=CURRENT_LAW + ), + reform=SideDefinition( + simulation_id=WINDOW_SIMULATION_IDS[2026][1], choice=REFORM_CHOICE + ), + budget_window_size=3, + window_years=tuple( + WindowYearDefinition( + year=year, + baseline_simulation_id=baseline_id, + reform_simulation_id=reform_id, + ) + for year, (baseline_id, reform_id) in WINDOW_SIMULATION_IDS.items() + ), +) +STORED_SIMULATION = SimulationDefinition( + id=SIMULATION_ID, + country="us", + kind="economy", + year=2027, + region_code="us", + dataset_name="populace_us_2024", + choice=SimulationChoice(policy_id=POLICY_ID, dynamic_id=None), +) + + +@dataclass +class FakeRerunStore(FakeWindowStore): + """Records the re-run reads and writes, and answers as told. + + A budget window's re-run is read with ``reusable_by_year``, as a new + window is, and recorded in ``windows``, where the fake advancer finds it. + + ``inputs_failure`` is raised by a re-run's read of its inputs, and + ``rerun_failure`` by its create instead of recording it. + """ + + report_definition: ReportDefinition | None = STORED_REPORT + simulation_definition: SimulationDefinition | None = STORED_SIMULATION + inputs_failure: Exception | None = None + rerun_failure: Exception | None = None + rerun_reads: list[tuple[EconomyRequest, Any]] = field(default_factory=list) + report_reruns: list[NewReportRerun] = field(default_factory=list) + simulation_reruns: list[NewSimulationRerun] = field(default_factory=list) + + def read_report_definition(self, report_id: UUID) -> ReportDefinition | None: + if self.read_failure is not None: + raise self.read_failure + stored = self.report_definition + return stored if stored is not None and stored.id == report_id else None + + def read_simulation_definition( + self, simulation_id: UUID + ) -> SimulationDefinition | None: + if self.read_failure is not None: + raise self.read_failure + stored = self.simulation_definition + return stored if stored is not None and stored.id == simulation_id else None + + def read_report_rerun_inputs( + self, + request: EconomyRequest, + *, + baseline: SideDefinition, + reform: SideDefinition, + reuse: RunReuse | None, + ) -> EconomyReportInputs: + self.rerun_reads.append((request, (baseline, reform, reuse))) + if self.inputs_failure is not None: + raise self.inputs_failure + baseline_reusable, reform_reusable = self.reusable + return EconomyReportInputs( + catalog=self.catalog, + baseline=SimulationInputs( + choice=baseline.choice, + reform={}, + reusable_run=None if reuse is None else baseline_reusable, + ), + reform=SimulationInputs( + choice=reform.choice, + reform=REFORM, + reusable_run=None if reuse is None else reform_reusable, + ), + ) + + def read_simulation_inputs( + self, request: EconomyRequest, choice: SimulationChoice + ) -> EconomySimulationInputs: + if self.inputs_failure is not None: + self.simulation_reads.append((request, choice)) + raise self.inputs_failure + return super().read_simulation_inputs(request, choice) + + def read_window_rerun_inputs( + self, + request: EconomyRequest, + *, + baseline: SimulationChoice, + reform: SimulationChoice, + years: Sequence[WindowYearDefinition], + reuse: RunReuse | None, + ) -> BudgetWindowInputs: + self.rerun_reads.append((request, (baseline, reform, tuple(years), reuse))) + if self.inputs_failure is not None: + raise self.inputs_failure + return BudgetWindowInputs( + catalog=self.catalog, + baseline=SimulationInputs(choice=baseline, reform={}), + reform=SimulationInputs(choice=reform, reform=REFORM), + reusable_runs={ + year.year: (None, None) + if reuse is None + else self.reusable_by_year.get(year.year, (None, None)) + for year in years + }, + ) + + def create_window_rerun(self, rerun: NewWindowRerun) -> None: + if self.rerun_failure is not None: + raise self.rerun_failure + self.windows.append(rerun) + + def create_report_rerun(self, rerun: NewReportRerun) -> None: + if self.rerun_failure is not None: + raise self.rerun_failure + self.report_reruns.append(rerun) + + def create_simulation_rerun(self, rerun: NewSimulationRerun) -> None: + if self.rerun_failure is not None: + raise self.rerun_failure + self.simulation_reruns.append(rerun) + + +class UnusedAdvancer: + """A window advancer a test of a single-year re-run never expects to be used.""" + + def advance_window(self, window_run_id: UUID, **_: Any) -> NoReturn: + msg = f"a single-year re-run advanced window {window_run_id}" + raise AssertionError(msg) + + +def rerun_service( # noqa: PLR0913 # the store, then each seam a test may replace + store: FakeRerunStore, + *, + workers: FakeWorkers | None = None, + caller: FakeCaller | None = None, + output_bucket: str | None = OUTPUT_BUCKET, + advancer: FakeAdvancer | None = None, + household_reports: HouseholdReportReruns | None = None, +) -> EconomyRerunService: + return EconomyRerunService( + household_reports=household_reports, + windows=advancer or UnusedAdvancer(), + store=store, + workers=workers or FakeWorkers(), + caller=caller or FakeCaller(), + settings=ServiceSettings( + environment="staging", + modal_environment="staging", + output_bucket=output_bucket, + ), + call_start_timeout_seconds=0.05, + new_id=numbered_ids(), + clock=lambda: CREATED_AT, + ) diff --git a/projects/policyengine-simulation-entry/tests/household_report_fakes.py b/projects/policyengine-simulation-entry/tests/household_report_fakes.py index 851afa294..84bc86cc5 100644 --- a/projects/policyengine-simulation-entry/tests/household_report_fakes.py +++ b/projects/policyengine-simulation-entry/tests/household_report_fakes.py @@ -17,6 +17,9 @@ from policyengine_simulation_entry.household_report_api_models import ( HouseholdReportRequest, ) +from policyengine_simulation_entry.household_report_reruns import ( + HouseholdReportRerunService, +) from policyengine_simulation_entry.household_report_submissions import ( HouseholdReportSubmissionService, ) @@ -33,7 +36,10 @@ if TYPE_CHECKING: from collections.abc import Mapping, Sequence - from policyengine_simulation_persistence import NewHouseholdReport + from policyengine_simulation_persistence import ( + NewHouseholdReport, + NewHouseholdReportRerun, + ) HOUSEHOLD_ID = UUID("00000000-0000-4000-8000-0000000000bb") POLICY_ID = UUID("00000000-0000-4000-8000-0000000000cc") @@ -115,7 +121,8 @@ class FakeHouseholdReportStore: """Records what the service asks of the database, and answers as told. ``claimed`` marks the report run as claimed by its household function: a - failure write then changes nothing. + failure write then changes nothing. ``rerun_failure`` is raised by a + re-run's create instead of recording it. """ default_year: int | None = 2026 @@ -124,9 +131,11 @@ class FakeHouseholdReportStore: create_failure: Exception | None = None record_failure: Exception | None = None fail_failure: Exception | None = None + rerun_failure: Exception | None = None claimed: bool = False reads: list[dict[str, Any]] = field(default_factory=list) reports: list[NewHouseholdReport] = field(default_factory=list) + report_reruns: list[NewHouseholdReportRerun] = field(default_factory=list) recorded_calls: list[tuple[UUID, str]] = field(default_factory=list) failed_runs: list[tuple[UUID, tuple[UUID, ...], str]] = field(default_factory=list) @@ -168,6 +177,11 @@ def create_report(self, report: NewHouseholdReport) -> None: raise self.create_failure self.reports.append(report) + def create_report_rerun(self, rerun: NewHouseholdReportRerun) -> None: + if self.rerun_failure is not None: + raise self.rerun_failure + self.report_reruns.append(rerun) + def record_report_call( self, report_run_id: UUID, modal_call_id: str ) -> ReportRunStatus | None: @@ -238,3 +252,19 @@ def household_report_service( call_start_timeout_seconds=0.05, new_id=numbered_ids(), ) + + +def household_rerun_service( + store: FakeHouseholdReportStore, + *, + workers: FakeHouseholdWorkers | None = None, + caller: FakeHouseholdReportCaller | None = None, +) -> HouseholdReportRerunService: + """Return the service that creates and re-runs household reports, on fakes.""" + return HouseholdReportRerunService( + store=store, + workers=workers or FakeHouseholdWorkers(), + caller=caller or FakeHouseholdReportCaller(), + call_start_timeout_seconds=0.05, + new_id=numbered_ids(), + ) diff --git a/projects/policyengine-simulation-entry/tests/test_budget_window_routes.py b/projects/policyengine-simulation-entry/tests/test_budget_window_routes.py index 5a84f1567..7e60d752d 100644 --- a/projects/policyengine-simulation-entry/tests/test_budget_window_routes.py +++ b/projects/policyengine-simulation-entry/tests/test_budget_window_routes.py @@ -72,7 +72,7 @@ def build(submissions: FakeSubmissions) -> TestClient: monkeypatch.setattr( app_module, "economy_submissions_for", - lambda settings, database: submissions, + lambda settings, database, **_: submissions, ) return TestClient( create_app( diff --git a/projects/policyengine-simulation-entry/tests/test_economy_rerun_routes.py b/projects/policyengine-simulation-entry/tests/test_economy_rerun_routes.py new file mode 100644 index 000000000..99cd90072 --- /dev/null +++ b/projects/policyengine-simulation-entry/tests/test_economy_rerun_routes.py @@ -0,0 +1,418 @@ +"""The re-run routes' answers, with the re-run service faked.""" + +from __future__ import annotations + +from dataclasses import dataclass, field +from typing import Any, NoReturn +from uuid import UUID + +import pytest +from conftest import FakeBackend, make_settings +from fastapi.testclient import TestClient +from policyengine_simulation_entry import app as app_module +from policyengine_simulation_entry.app import create_app +from policyengine_simulation_entry.budget_window_api_models import ( + BudgetWindowReportCreated, + BudgetWindowYearCreated, +) +from policyengine_simulation_entry.economy_api_models import ( + EconomyReportCreated, + EconomySimulationCreated, + ReportRerunRequest, +) +from policyengine_simulation_entry.economy_submission_errors import ( + EconomyRecordNotFoundError, + EconomyRerunConflictError, + EconomySubmissionOutcomeUnknownError, + EconomySubmissionRefusedError, + EconomySubmissionUnavailableError, + EconomySubmissionUnprocessableError, + UnprocessableReason, +) +from policyengine_simulation_entry.household_report_api_models import ( + HouseholdReportCreated, +) +from policyengine_simulation_observability.identifiers import OBSERVABILITY_ID_HEADER +from policyengine_simulation_persistence import ReportRunStatus, SimulationStatus + +OBSERVABILITY_ID = "6e1f3d4c-0a5b-4c39-9d2e-7f8a9b0c1d2e" +RECORD_ID = UUID("00000000-0000-4000-8000-0000000000e1") +RUN_ID = UUID("00000000-0000-4000-8000-0000000000e2") +SIDE_IDS = [UUID(f"00000000-0000-4000-8000-0000000000f{digit}") for digit in "1234"] +REPORT_RUNS = f"/reports/{RECORD_ID}/runs" +SIMULATION_RUNS = f"/simulations/{RECORD_ID}/runs" +RERUN_REPORT = EconomyReportCreated( + report_id=RECORD_ID, + report_run_id=RUN_ID, + status=ReportRunStatus.PENDING, + baseline_simulation_id=SIDE_IDS[0], + baseline_simulation_run_id=SIDE_IDS[1], + baseline_reused=True, + reform_simulation_id=SIDE_IDS[2], + reform_simulation_run_id=SIDE_IDS[3], + reform_reused=False, +) +RERUN_SIMULATION = EconomySimulationCreated( + simulation_id=RECORD_ID, simulation_run_id=RUN_ID, status=SimulationStatus.PENDING +) + + +def _raise(failure: Exception) -> NoReturn: + raise failure + + +@dataclass +class FakeEconomyReruns: + """Records each re-run, and answers as told.""" + + failure: Exception | None = None + report_answer: ( + EconomyReportCreated | BudgetWindowReportCreated | HouseholdReportCreated + ) = RERUN_REPORT + report_reruns: list[tuple[UUID, ReportRerunRequest, str, Any]] = field( + default_factory=list + ) + simulation_reruns: list[tuple[UUID, str, Any]] = field(default_factory=list) + + async def rerun_report( + self, + report_id: UUID, + request: ReportRerunRequest, + *, + request_id: str, + observability_context: dict[str, Any] | None, + ) -> EconomyReportCreated | BudgetWindowReportCreated | HouseholdReportCreated: + self.report_reruns.append( + (report_id, request, request_id, observability_context) + ) + if self.failure is not None: + _raise(self.failure) + return self.report_answer + + async def rerun_simulation( + self, + simulation_id: UUID, + *, + request_id: str, + observability_context: dict[str, Any] | None, + ) -> EconomySimulationCreated: + self.simulation_reruns.append( + (simulation_id, request_id, observability_context) + ) + if self.failure is not None: + _raise(self.failure) + return RERUN_SIMULATION + + +@pytest.fixture +def client_for(monkeypatch): + """Return a function building a client whose app uses ``reruns``. + + None leaves the app as an unconfigured service builds it. + """ + + def build(reruns: FakeEconomyReruns | None) -> TestClient: + if reruns is not None: + monkeypatch.setattr( + app_module, + "economy_submissions_for", + lambda settings, database, **_: reruns, + ) + return TestClient( + create_app( + settings=make_settings(), + backend=FakeBackend(), + auth_dependency=lambda: None, + ) + ) + + return build + + +# Each re-run route, with a body it accepts. +ROUTES = [(REPORT_RUNS, {}), (SIMULATION_RUNS, None)] + + +def test_a_report_rerun_answers_201_with_its_new_run(client_for) -> None: + reruns = FakeEconomyReruns() + + with client_for(reruns) as client: + result = client.post( + REPORT_RUNS, + json={"force": True}, + headers={OBSERVABILITY_ID_HEADER: OBSERVABILITY_ID}, + ) + + assert result.status_code == 201 + assert result.json() == RERUN_REPORT.model_dump(mode="json") + ((report_id, request, request_id, context),) = reruns.report_reruns + assert (report_id, request) == (RECORD_ID, ReportRerunRequest(force=True)) + assert request_id != OBSERVABILITY_ID + assert context["observability_id"] == OBSERVABILITY_ID + + +def test_a_budget_window_rerun_answers_201_with_its_new_window_run(client_for) -> None: + window = BudgetWindowReportCreated( + report_id=RECORD_ID, + report_run_id=RUN_ID, + budget_window_run_id=SIDE_IDS[0], + status=ReportRunStatus.PENDING, + years=[ + BudgetWindowYearCreated( + year=2026, + report_run_id=RUN_ID, + baseline_simulation_id=SIDE_IDS[1], + baseline_simulation_run_id=SIDE_IDS[2], + baseline_reused=False, + reform_simulation_id=SIDE_IDS[3], + reform_simulation_run_id=SIDE_IDS[0], + reform_reused=False, + ) + ], + ) + reruns = FakeEconomyReruns(report_answer=window) + + with client_for(reruns) as client: + result = client.post(REPORT_RUNS, json={}) + + assert result.status_code == 201 + assert result.json() == window.model_dump(mode="json") + + +def test_a_household_report_rerun_answers_201_with_its_new_run(client_for) -> None: + household = HouseholdReportCreated( + report_id=RECORD_ID, + report_run_id=RUN_ID, + status=ReportRunStatus.PENDING, + year=2025, + baseline_simulation_id=SIDE_IDS[0], + baseline_simulation_run_id=SIDE_IDS[1], + reform_simulation_id=None, + reform_simulation_run_id=None, + ) + reruns = FakeEconomyReruns(report_answer=household) + + with client_for(reruns) as client: + result = client.post(REPORT_RUNS, json={}) + + assert result.status_code == 201 + assert result.json() == household.model_dump(mode="json") + + +def test_a_report_rerun_without_force_reuses_runs(client_for) -> None: + reruns = FakeEconomyReruns() + + with client_for(reruns) as client: + result = client.post(REPORT_RUNS, json={}) + + assert result.status_code == 201 + assert reruns.report_reruns[0][1] == ReportRerunRequest(force=False) + + +def test_a_simulation_rerun_answers_201_with_its_new_run(client_for) -> None: + reruns = FakeEconomyReruns() + + with client_for(reruns) as client: + result = client.post(SIMULATION_RUNS) + + assert result.status_code == 201 + assert result.json() == { + "simulation_id": str(RECORD_ID), + "simulation_run_id": str(RUN_ID), + "status": "pending", + } + ((simulation_id, _, context),) = reruns.simulation_reruns + assert simulation_id == RECORD_ID + assert context is not None + + +@pytest.mark.parametrize( + ("path", "body"), + [ + (REPORT_RUNS, {"force": "yes"}), + (REPORT_RUNS, {"force": None}), + (REPORT_RUNS, {"policyengine_version": "6.17.1"}), + ("/reports/not-a-uuid/runs", {}), + ("/simulations/not-a-uuid/runs", None), + ], +) +def test_a_malformed_rerun_is_refused_with_its_reasons(client_for, path, body): + reruns = FakeEconomyReruns() + + with client_for(reruns) as client: + result = client.post(path, json=body) + + assert result.status_code == 422 + assert all({"loc", "msg"} <= set(reason) for reason in result.json()["detail"]) + assert (reruns.report_reruns, reruns.simulation_reruns) == ([], []) + + +@pytest.mark.parametrize( + ("path", "body", "record"), + [(REPORT_RUNS, {}, "report"), (SIMULATION_RUNS, None, "simulation")], +) +def test_an_unknown_record_answers_404(client_for, path, body, record) -> None: + reruns = FakeEconomyReruns(failure=EconomyRecordNotFoundError(record)) + + with client_for(reruns) as client: + result = client.post(path, json=body) + + assert result.status_code == 404 + assert result.json() == { + "code": f"unknown_{record}", + "detail": f"There is no {record} {RECORD_ID}.", + } + + +@pytest.mark.parametrize(("path", "body"), ROUTES) +def test_a_run_under_way_answers_409_naming_it(client_for, path, body) -> None: + reruns = FakeEconomyReruns( + failure=EconomyRerunConflictError( + code="run_in_progress", detail="a run is under way", run_id=RUN_ID + ) + ) + + with client_for(reruns) as client: + result = client.post(path, json=body) + + assert result.status_code == 409 + assert result.json() == { + "code": "run_in_progress", + "detail": "a run is under way", + "run_id": str(RUN_ID), + } + + +@pytest.mark.parametrize(("path", "body"), ROUTES) +def test_a_record_the_service_cannot_rerun_answers_409(client_for, path, body) -> None: + reruns = FakeEconomyReruns( + failure=EconomyRerunConflictError( + code="rerun_not_supported", detail="not budget windows yet" + ) + ) + + with client_for(reruns) as client: + result = client.post(path, json=body) + + assert result.status_code == 409 + assert result.json() == { + "code": "rerun_not_supported", + "detail": "not budget windows yet", + } + + +@pytest.mark.parametrize(("path", "body"), ROUTES) +def test_a_refused_rerun_answers_400_or_422(client_for, path, body) -> None: + refused = FakeEconomyReruns( + failure=EconomySubmissionRefusedError( + code="unknown_region", detail="no region us, which the report names" + ) + ) + unprocessable = FakeEconomyReruns( + failure=EconomySubmissionUnprocessableError( + code="missing_parameters", + reasons=( + UnprocessableReason( + loc=("report", "reform", "policy_id", "gov.a"), + msg="no such parameter", + type="missing_parameter", + ), + ), + ) + ) + + with client_for(refused) as client: + refusal = client.post(path, json=body) + with client_for(unprocessable) as client: + unprocessable_answer = client.post(path, json=body) + + assert (refusal.status_code, refusal.json()["code"]) == (400, "unknown_region") + assert unprocessable_answer.status_code == 422 + assert unprocessable_answer.json() == { + "detail": [ + { + "loc": ["report", "reform", "policy_id", "gov.a"], + "msg": "no such parameter", + "type": "missing_parameter", + } + ], + "code": "missing_parameters", + } + + +@pytest.mark.parametrize( + ("path", "body", "record"), + [(REPORT_RUNS, {}, "report"), (SIMULATION_RUNS, None, "simulation")], +) +def test_a_rerun_modal_refused_answers_503_naming_the_record_and_its_run( + client_for, path, body, record +) -> None: + reruns = FakeEconomyReruns( + failure=EconomySubmissionUnavailableError( + code="modal_call_refused", + detail="Modal refused it", + record_id=RECORD_ID, + run_id=RUN_ID, + ) + ) + + with client_for(reruns) as client: + result = client.post(path, json=body) + + assert result.status_code == 503 + assert result.headers["Retry-After"] == "10" + assert result.json() == { + "code": "modal_call_refused", + "detail": "Modal refused it", + f"{record}_id": str(RECORD_ID), + "run_id": str(RUN_ID), + } + + +@pytest.mark.parametrize( + ("path", "body", "record"), + [(REPORT_RUNS, {}, "report"), (SIMULATION_RUNS, None, "simulation")], +) +def test_a_rerun_whose_commit_is_unconfirmed_answers_500( + client_for, path, body, record +) -> None: + reruns = FakeEconomyReruns( + failure=EconomySubmissionOutcomeUnknownError( + record, error_type="OperationalError", rerun=True + ) + ) + + with client_for(reruns) as client: + result = client.post(path, json=body) + + assert result.status_code == 500 + assert result.json() == { + "code": "internal_error", + "detail": ( + f"The service may have created the {record}'s new run, but could not " + "confirm it was written." + ), + } + + +@pytest.mark.parametrize(("path", "body"), ROUTES) +def test_an_unexpected_failure_answers_the_json_500(client_for, path, body) -> None: + reruns = FakeEconomyReruns(failure=RuntimeError("secret detail")) + + with client_for(reruns) as client: + result = client.post(path, json=body) + + assert result.status_code == 500 + assert result.json() == { + "code": "internal_error", + "detail": "The service could not answer the request.", + } + + +@pytest.mark.parametrize(("path", "body"), ROUTES) +def test_an_unconfigured_service_answers_503(client_for, path, body) -> None: + with client_for(None) as client: + result = client.post(path, json=body) + + assert result.status_code == 503 + assert result.json()["code"] == "not_configured" diff --git a/projects/policyengine-simulation-entry/tests/test_economy_reruns.py b/projects/policyengine-simulation-entry/tests/test_economy_reruns.py new file mode 100644 index 000000000..eb4efe3e7 --- /dev/null +++ b/projects/policyengine-simulation-entry/tests/test_economy_reruns.py @@ -0,0 +1,611 @@ +"""Re-running stored economy reports and simulations; the store and Modal are faked.""" + +from __future__ import annotations + +import asyncio +from dataclasses import replace +from uuid import UUID + +import pytest +from economy_rerun_fakes import ( + BASELINE_SIMULATION_ID, + REFORM_SIMULATION_ID, + REPORT_ID, + RUN_UNDER_WAY_ID, + SIMULATION_ID, + STORED_OPTIONS, + STORED_REPORT, + STORED_SIMULATION, + STORED_WINDOW, + WINDOW_SIMULATION_IDS, + FakeRerunStore, + rerun_service, +) +from economy_service_fakes import ( + CATALOG, + OUTPUT_BUCKET, + REFORM, + FakeCaller, +) +from household_fixtures import POLICYENGINE_VERSION +from policyengine_simulation_contract.stage12_execution import ( + ReportExecutionInput, + ReportRunIds, + Stage12InvocationContext, +) +from policyengine_simulation_entry.economy_api_models import ReportRerunRequest +from policyengine_simulation_entry.economy_submission_errors import ( + EconomyRecordNotFoundError, + EconomyRerunConflictError, + EconomySubmissionOutcomeUnknownError, + EconomySubmissionRefusedError, + EconomySubmissionUnavailableError, + EconomySubmissionUnprocessableError, +) +from policyengine_simulation_persistence import ( + CatalogDataset, + EconomyRecordOutcomeUnknownError, + EconomyRecordReferenceError, + MissingPolicyParametersError, + NewReportRerun, + NewSimulationRerun, + ReportRunStatus, + ReportSideError, + RerunRecordDeletedError, + RerunSide, + ReusableRun, + RunInProgressError, + RunReuse, + RunVersions, + SimulationStatus, + UnknownRegionError, +) +from sqlalchemy.exc import OperationalError +from test_budget_window_submissions import FakeAdvancer + +DATABASE_ERROR = OperationalError("SELECT 1", {}, Exception("server closed")) +VERSIONS = RunVersions( + country_package_version="2.2.1", policyengine_version=POLICYENGINE_VERSION +) +OBSERVED = {"observability_id": "observed"} +REUSED_2027 = ReusableRun( + simulation_id=UUID("00000000-0000-4000-8000-000000002027"), + simulation_run_id=UUID("00000000-0000-4000-8000-0000000000b2"), +) +REUSED_BASELINE = ReusableRun( + simulation_id=BASELINE_SIMULATION_ID, + simulation_run_id=UUID("00000000-0000-4000-8000-0000000000b1"), +) + + +def _rerun_report(service, request: ReportRerunRequest | None = None): + return asyncio.run( + service.rerun_report( + REPORT_ID, + request or ReportRerunRequest(), + request_id="observed", + observability_context=OBSERVED, + ) + ) + + +def _rerun_simulation(service, simulation_id: UUID = SIMULATION_ID): + return asyncio.run( + service.rerun_simulation( + simulation_id, request_id="observed", observability_context=OBSERVED + ) + ) + + +def test_a_report_is_rerun_on_the_default_workers_versions_as_a_new_run() -> None: + store = FakeRerunStore(reusable=(REUSED_BASELINE, None)) + caller = FakeCaller() + + rerun = _rerun_report(rerun_service(store, caller=caller)) + + ((request, (baseline, reform, reuse)),) = store.rerun_reads + assert (request.country, request.policyengine_version, request.year) == ( + "us", + POLICYENGINE_VERSION, + 2026, + ) + assert (request.region_code, request.dataset_name) == ("us", "populace_us_2024") + assert (baseline, reform) == (STORED_REPORT.baseline, STORED_REPORT.reform) + assert reuse == RunReuse( + versions=VERSIONS, + inputs=STORED_OPTIONS, + output_uri_prefix=f"gs://{OUTPUT_BUCKET}/", + ) + assert store.report_reruns == [ + NewReportRerun( + report_id=REPORT_ID, + run_id=UUID(int=1), + population=CATALOG.population(2026), + versions=VERSIONS, + baseline=RerunSide( + simulation_id=BASELINE_SIMULATION_ID, + simulation_run_id=REUSED_BASELINE.simulation_run_id, + reused=True, + ), + reform=RerunSide( + simulation_id=REFORM_SIMULATION_ID, + simulation_run_id=UUID(int=2), + reused=False, + ), + ) + ] + ((_, payload, context, observed),) = caller.report_runs + assert observed == OBSERVED + report = ReportExecutionInput.model_validate(payload) + assert report.evaluation_id == UUID(int=1) + assert report.baseline.simulation_execution_id == REUSED_BASELINE.simulation_run_id + assert (report.reform.simulation_execution_id, report.reform.policy) == ( + UUID(int=2), + REFORM, + ) + assert report.reform.options == STORED_OPTIONS + assert Stage12InvocationContext.model_validate(context).report_run == ReportRunIds( + report_run_id=UUID(int=1), + baseline_simulation_run_id=REUSED_BASELINE.simulation_run_id, + reform_simulation_run_id=UUID(int=2), + ) + assert store.recorded_calls == [(UUID(int=1), "fc-coordinator")] + assert rerun.model_dump() == { + "report_id": REPORT_ID, + "report_run_id": UUID(int=1), + "status": ReportRunStatus.PENDING, + "baseline_simulation_id": BASELINE_SIMULATION_ID, + "baseline_simulation_run_id": REUSED_BASELINE.simulation_run_id, + "baseline_reused": True, + "reform_simulation_id": REFORM_SIMULATION_ID, + "reform_simulation_run_id": UUID(int=2), + "reform_reused": False, + } + + +def test_a_forced_rerun_recalculates_both_simulations() -> None: + store = FakeRerunStore(reusable=(REUSED_BASELINE, None)) + + rerun = _rerun_report(rerun_service(store), ReportRerunRequest(force=True)) + + assert store.rerun_reads[0][1][2] is None + (created,) = store.report_reruns + assert (created.baseline.reused, created.reform.reused) == (False, False) + assert created.new_simulation_run_ids == (UUID(int=2), UUID(int=3)) + assert created.baseline.simulation_id == BASELINE_SIMULATION_ID + assert (rerun.baseline_reused, rerun.reform_reused) == (False, False) + + +def test_without_the_output_bucket_a_rerun_looks_for_no_run_to_reuse() -> None: + store = FakeRerunStore(reusable=(REUSED_BASELINE, None)) + + _rerun_report(rerun_service(store, output_bucket=None)) + + assert store.rerun_reads[0][1][2] is None + + +def test_an_unknown_report_or_simulation_is_not_found() -> None: + store = FakeRerunStore(report_definition=None, simulation_definition=None) + service = rerun_service(store) + + with pytest.raises(EconomyRecordNotFoundError) as report: + _rerun_report(service) + with pytest.raises(EconomyRecordNotFoundError) as simulation: + _rerun_simulation(service) + + assert (report.value.record, simulation.value.record) == ("report", "simulation") + + +def test_a_report_kind_the_service_does_not_rerun_is_refused() -> None: + store = FakeRerunStore(report_definition=replace(STORED_REPORT, kind="unsupported")) + caller = FakeCaller() + + with pytest.raises(EconomyRerunConflictError) as refused: + _rerun_report(rerun_service(store, caller=caller)) + + assert refused.value.code == "rerun_not_supported" + assert "this type of" in refused.value.detail + assert (store.rerun_reads, caller.report_runs) == ([], []) + + +@pytest.mark.parametrize( + ("missing", "named"), + [ + ({"country": "fr"}, "country"), + ({"baseline": None}, "baseline simulation"), + ({"reform": None}, "reform simulation"), + ({"region_code": None}, "region"), + ({"dataset_name": None}, "dataset"), + ({"year": None}, "year"), + ], +) +def test_a_report_missing_a_row_it_calculates_on_is_not_rerun(missing, named) -> None: + store = FakeRerunStore(report_definition=replace(STORED_REPORT, **missing)) + + with pytest.raises(EconomyRerunConflictError) as refused: + _rerun_report(rerun_service(store)) + + assert refused.value.code == "rerun_not_supported" + assert named in refused.value.detail + assert store.rerun_reads == [] + + +def test_a_rerun_is_refused_while_a_run_is_under_way() -> None: + store = FakeRerunStore(rerun_failure=RunInProgressError(RUN_UNDER_WAY_ID)) + caller = FakeCaller() + service = rerun_service(store, caller=caller) + + with pytest.raises(EconomyRerunConflictError) as report: + _rerun_report(service) + with pytest.raises(EconomyRerunConflictError) as simulation: + _rerun_simulation(service) + + for refused, record in ((report, "report"), (simulation, "simulation")): + assert refused.value.code == "run_in_progress" + assert refused.value.run_id == RUN_UNDER_WAY_ID + assert f"The {record} has a run" in refused.value.detail + assert (caller.report_runs, caller.simulation_runs) == ([], []) + + +def test_a_rerun_modal_refuses_fails_only_its_new_runs_and_names_them() -> None: + store = FakeRerunStore(reusable=(REUSED_BASELINE, None)) + + with pytest.raises(EconomySubmissionUnavailableError) as unavailable: + _rerun_report( + rerun_service(store, caller=FakeCaller(failure=ConnectionError("no"))) + ) + + error = unavailable.value + assert (error.code, error.record_id, error.run_id) == ( + "modal_call_refused", + REPORT_ID, + UUID(int=1), + ) + assert "the report's new run" in error.detail + ((run_id, simulation_run_ids, message),) = store.failed_runs + assert (run_id, simulation_run_ids) == (UUID(int=1), (UUID(int=2),)) + assert message.startswith("modal_call_refused: Modal did not accept the report") + + +def test_a_rerun_its_coordinator_claimed_meanwhile_is_answered_created() -> None: + store = FakeRerunStore(claimed=True) + + rerun = _rerun_report( + rerun_service(store, caller=FakeCaller(failure=TimeoutError("late"))) + ) + + assert store.failed_runs == [] + assert rerun.report_run_id == UUID(int=1) + + +def test_a_stored_policy_the_current_catalog_cannot_set_is_located_on_its_side() -> ( + None +): + store = FakeRerunStore( + inputs_failure=ReportSideError( + "reform", MissingPolicyParametersError(("gov.old.parameter",)) + ) + ) + + with pytest.raises(EconomySubmissionUnprocessableError) as refused: + _rerun_report(rerun_service(store)) + + assert refused.value.code == "missing_parameters" + ((reason,),) = (refused.value.reasons,) + assert reason.loc == ("report", "reform", "policy_id", "gov.old.parameter") + assert POLICYENGINE_VERSION in reason.msg + assert store.report_reruns == [] + + +def test_a_stored_region_the_current_catalog_lacks_is_refused_naming_the_report() -> ( + None +): + store = FakeRerunStore(inputs_failure=UnknownRegionError("the region is gone")) + + with pytest.raises(EconomySubmissionRefusedError) as refused: + _rerun_report(rerun_service(store)) + + assert refused.value.code == "unknown_region" + assert refused.value.detail == ( + f"us's catalog version for .py {POLICYENGINE_VERSION} has no region us, " + "which the report names." + ) + + +def test_a_stored_dataset_the_bundle_lacks_is_refused() -> None: + store = FakeRerunStore( + catalog=replace( + CATALOG, dataset=CatalogDataset(id=CATALOG.dataset.id, name="cps_2019") + ) + ) + + with pytest.raises(EconomySubmissionRefusedError) as refused: + _rerun_report(rerun_service(store)) + + assert refused.value.code == "dataset_not_in_bundle" + assert "cps_2019" in refused.value.detail + assert store.report_reruns == [] + + +def test_a_database_that_cannot_be_read_is_unavailable_before_any_call() -> None: + store = FakeRerunStore(read_failure=DATABASE_ERROR) + caller = FakeCaller() + + with pytest.raises(EconomySubmissionUnavailableError) as unavailable: + _rerun_report(rerun_service(store, caller=caller)) + + assert unavailable.value.code == "database_unavailable" + assert caller.report_runs == [] + + +def test_a_rerun_whose_commit_is_unconfirmed_may_have_written_its_run() -> None: + store = FakeRerunStore( + rerun_failure=EconomyRecordOutcomeUnknownError("commit failed") + ) + store.rerun_failure.__cause__ = DATABASE_ERROR + + with pytest.raises(EconomySubmissionOutcomeUnknownError) as unknown: + _rerun_report(rerun_service(store)) + + assert (unknown.value.record, unknown.value.rerun) == ("report", True) + assert unknown.value.error_type == "OperationalError" + + +def test_a_simulation_is_rerun_as_a_new_run_of_it() -> None: + store = FakeRerunStore() + caller = FakeCaller() + + rerun = _rerun_simulation(rerun_service(store, caller=caller)) + + ((request, choice),) = store.simulation_reads + assert (request.country, request.region_code, request.dataset_name) == ( + "us", + "us", + "populace_us_2024", + ) + assert (request.policyengine_version, request.year) == (POLICYENGINE_VERSION, 2027) + assert choice == STORED_SIMULATION.choice + assert store.simulation_reruns == [ + NewSimulationRerun( + simulation_id=SIMULATION_ID, + run_id=UUID(int=1), + population=CATALOG.population(2027), + versions=VERSIONS, + ) + ] + ((_, payload, context, observed),) = caller.simulation_runs + assert payload["simulation_execution_id"] == str(UUID(int=1)) + assert (payload["policy"], payload["year"]) == (REFORM, 2027) + assert context["simulation_run_id"] == str(UUID(int=1)) + assert observed == OBSERVED + assert store.recorded_calls == [(UUID(int=1), "fc-worker")] + assert rerun.model_dump() == { + "simulation_id": SIMULATION_ID, + "simulation_run_id": UUID(int=1), + "status": SimulationStatus.PENDING, + } + + +@pytest.mark.parametrize( + ("changes", "named"), + [ + ({"kind": "household"}, "only through its report"), + ({"kind": "unsupported"}, "this type of"), + ({"country": None}, "country"), + ({"country": "fr"}, "country"), + ({"region_code": None}, "region"), + ({"dataset_name": None}, "dataset"), + ({"year": None}, "year"), + ], +) +def test_a_simulation_the_service_cannot_rerun_is_refused(changes, named) -> None: + store = FakeRerunStore(simulation_definition=replace(STORED_SIMULATION, **changes)) + + with pytest.raises(EconomyRerunConflictError) as refused: + _rerun_simulation(rerun_service(store)) + + assert refused.value.code == "rerun_not_supported" + assert named in refused.value.detail + assert store.simulation_reads == [] + + +def test_a_simulation_rerun_modal_refuses_is_failed_and_named() -> None: + store = FakeRerunStore() + + with pytest.raises(EconomySubmissionUnavailableError) as unavailable: + _rerun_simulation( + rerun_service(store, caller=FakeCaller(failure=ConnectionError("no"))) + ) + + error = unavailable.value + assert (error.code, error.record_id, error.run_id) == ( + "modal_call_refused", + SIMULATION_ID, + UUID(int=1), + ) + assert store.failed_runs[0][0] == UUID(int=1) + + +def test_a_simulations_stored_policy_is_located_on_the_simulation() -> None: + store = FakeRerunStore( + inputs_failure=MissingPolicyParametersError( + ("gov.old.elasticity",), source="dynamic" + ) + ) + + with pytest.raises(EconomySubmissionUnprocessableError) as refused: + _rerun_simulation(rerun_service(store)) + + ((reason,),) = (refused.value.reasons,) + assert reason.loc == ("simulation", "dynamic_id", "gov.old.elasticity") + + +@pytest.mark.parametrize("record", ["report", "simulation"]) +def test_a_record_deleted_after_its_reads_is_not_found(record) -> None: + service = rerun_service(FakeRerunStore(rerun_failure=RerunRecordDeletedError())) + rerun = _rerun_report if record == "report" else _rerun_simulation + + with pytest.raises(EconomyRecordNotFoundError) as missing: + rerun(service) + + assert missing.value.record == record + + +@pytest.mark.parametrize( + ("deleted", "code", "detail"), + [ + ( + EconomyRecordReferenceError("simulation_runs", "dataset_id"), + "unknown_dataset", + "a dataset the report names no longer exists", + ), + ( + EconomyRecordReferenceError("simulation_runs", "simulation_id"), + "missing_reference", + "a simulation the report names no longer exists", + ), + ], +) +def test_a_row_deleted_after_a_reruns_reads_is_refused_naming_the_report( + deleted, code, detail +) -> None: + store = FakeRerunStore(rerun_failure=deleted) + + with pytest.raises(EconomySubmissionRefusedError) as refused: + _rerun_report(rerun_service(store)) + + assert (refused.value.code, refused.value.detail) == (code, detail) + + +def test_a_budget_window_is_rerun_as_a_new_window_run_and_its_first_years_started(): + store = FakeRerunStore( + report_definition=STORED_WINDOW, reusable_by_year={2027: (REUSED_2027, None)} + ) + caller = FakeCaller() + advancer = FakeAdvancer(store) + + rerun = _rerun_report(rerun_service(store, caller=caller, advancer=advancer)) + + ((request, (baseline, reform, years, reuse)),) = store.rerun_reads + assert (request.policyengine_version, request.year) == (POLICYENGINE_VERSION, 2026) + assert (request.region_code, request.dataset_name) == ("us", "populace_us_2024") + assert (baseline, reform, years) == ( + STORED_WINDOW.baseline.choice, + STORED_WINDOW.reform.choice, + STORED_WINDOW.window_years, + ) + assert reuse == RunReuse( + versions=VERSIONS, + inputs=STORED_OPTIONS, + output_uri_prefix=f"gs://{OUTPUT_BUCKET}/", + ) + (window,) = store.windows + assert (window.report_id, window.window_run_id, window.recalculate) == ( + REPORT_ID, + UUID(int=1), + False, + ) + assert (window.population, window.versions) == (CATALOG.population(2026), VERSIONS) + assert [window_year.year for window_year in window.years] == [2026, 2027, 2028] + for window_year in window.years: + baseline_id, reform_id = WINDOW_SIMULATION_IDS[window_year.year] + assert window_year.baseline.simulation_id == baseline_id + assert window_year.reform.simulation_id == reform_id + year_2027 = window.years[1] + assert (year_2027.baseline.simulation_run_id, year_2027.baseline.reused) == ( + REUSED_2027.simulation_run_id, + True, + ) + assert advancer.advances == [window.window_run_id] + assert len(caller.report_runs) == 3 + for (year_to_start, _), (_, payload, context, observed) in zip( + advancer.started, caller.report_runs, strict=True + ): + report = ReportExecutionInput.model_validate(payload) + assert report.baseline.year == report.reform.year == year_to_start.year + assert report.reform.policy == REFORM + assert report.reform.options == STORED_OPTIONS + assert Stage12InvocationContext.model_validate(context).report_run == ( + year_to_start.run_ids + ) + assert observed == OBSERVED + assert rerun.model_dump() == { + "report_id": REPORT_ID, + "report_run_id": window.years[0].report_run_id, + "budget_window_run_id": window.window_run_id, + "status": ReportRunStatus.PENDING, + "years": [ + { + "year": window_year.year, + "report_run_id": window_year.report_run_id, + "baseline_simulation_id": window_year.baseline.simulation_id, + "baseline_simulation_run_id": window_year.baseline.simulation_run_id, + "baseline_reused": window_year.baseline.reused, + "reform_simulation_id": window_year.reform.simulation_id, + "reform_simulation_run_id": window_year.reform.simulation_run_id, + "reform_reused": False, + } + for window_year in window.years + ], + } + + +def test_a_forced_window_rerun_recalculates_every_year() -> None: + store = FakeRerunStore( + report_definition=STORED_WINDOW, reusable_by_year={2027: (REUSED_2027, None)} + ) + + rerun = _rerun_report( + rerun_service(store, advancer=FakeAdvancer(store)), + ReportRerunRequest(force=True), + ) + + assert store.rerun_reads[0][1][3] is None + (window,) = store.windows + assert window.recalculate is True + assert not any(window_year.baseline.reused for window_year in window.years) + assert not any(year.baseline_reused for year in rerun.years) + + +def test_a_window_rerun_whose_first_year_modal_refuses_names_the_window_run() -> None: + store = FakeRerunStore(report_definition=STORED_WINDOW) + caller = FakeCaller(failure=RuntimeError("Modal is unavailable")) + + with pytest.raises(EconomySubmissionUnavailableError) as refused: + _rerun_report(rerun_service(store, caller=caller, advancer=FakeAdvancer(store))) + + error = refused.value + (window,) = store.windows + assert (error.code, error.record_id, error.run_id) == ( + "modal_call_refused", + REPORT_ID, + window.window_run_id, + ) + assert "the report's new run" in error.detail + + +@pytest.mark.parametrize( + "changes", + [ + {"window_years": ()}, + {"budget_window_size": 4}, + # A year missing from the newest window run, as one naming no reform is. + {"window_years": STORED_WINDOW.window_years[::2]}, + { + "window_years": ( + STORED_WINDOW.window_years[0], + STORED_WINDOW.window_years[2], + STORED_WINDOW.window_years[1], + ) + }, + ], +) +def test_a_window_whose_years_cannot_be_rerun_is_refused(changes) -> None: + store = FakeRerunStore(report_definition=replace(STORED_WINDOW, **changes)) + + with pytest.raises(EconomyRerunConflictError) as refused: + _rerun_report(rerun_service(store, advancer=FakeAdvancer(store))) + + assert refused.value.code == "rerun_not_supported" + assert "complete budget-window calculation" in refused.value.detail + assert (store.rerun_reads, store.windows) == ([], []) diff --git a/projects/policyengine-simulation-entry/tests/test_economy_routes.py b/projects/policyengine-simulation-entry/tests/test_economy_routes.py index 90ca6e108..30a1bcffc 100644 --- a/projects/policyengine-simulation-entry/tests/test_economy_routes.py +++ b/projects/policyengine-simulation-entry/tests/test_economy_routes.py @@ -134,7 +134,7 @@ def build(submissions: FakeEconomySubmissions | None) -> TestClient: monkeypatch.setattr( app_module, "economy_submissions_for", - lambda settings, database: submissions, + lambda settings, database, **_: submissions, ) return TestClient( create_app( diff --git a/projects/policyengine-simulation-entry/tests/test_household_report_reruns.py b/projects/policyengine-simulation-entry/tests/test_household_report_reruns.py new file mode 100644 index 000000000..5cab40adc --- /dev/null +++ b/projects/policyengine-simulation-entry/tests/test_household_report_reruns.py @@ -0,0 +1,513 @@ +"""Re-running a stored household report; the stores and Modal are faked. + +The re-run service reads the stored report through the economy store and +hands a household report to the household report service, which reads its +household and policies, creates its new runs and starts its household +function call. +""" + +from __future__ import annotations + +import asyncio +from dataclasses import replace +from uuid import UUID + +import pytest +from conftest import FakeBackend, make_settings +from economy_rerun_fakes import ( + REPORT_ID, + RUN_UNDER_WAY_ID, + FakeRerunStore, + rerun_service, +) +from economy_service_fakes import DYNAMIC_ID, FakeCaller +from household_report_fakes import ( + CATALOG_VERSION_ID, + CTC_AMOUNT, + HOUSEHOLD_ID, + POLICY_ID, + REFORM, + SITUATION, + FakeHouseholdReportCaller, + FakeHouseholdReportStore, + FakeHouseholdWorkers, + household_rerun_service, + manifest_household_worker, +) +from policyengine_simulation_contract.household_report_execution import ( + HouseholdReportRunCalculation, +) +from policyengine_simulation_entry import app as app_module +from policyengine_simulation_entry.app import create_app +from policyengine_simulation_entry.economy_api_models import ReportRerunRequest +from policyengine_simulation_entry.economy_submission_errors import ( + EconomyRecordNotFoundError, + EconomyRerunConflictError, + EconomySubmissionOutcomeUnknownError, + EconomySubmissionRefusedError, + EconomySubmissionUnavailableError, + EconomySubmissionUnprocessableError, +) +from policyengine_simulation_entry.household_report_api_models import ( + HouseholdReportCreated, +) +from policyengine_simulation_entry.household_report_reruns import ( + HouseholdReportRerunService, +) +from policyengine_simulation_entry.household_workers import ( + HouseholdFunctionUnavailableError, +) +from policyengine_simulation_entry.simulation_database import SimulationDatabase +from policyengine_simulation_persistence import ( + EconomyRecordOutcomeUnknownError, + EconomyRecordReferenceError, + HouseholdRerunSide, + MissingPolicyParametersError, + NewHouseholdReportRerun, + ReportDefinition, + ReportRunStatus, + ReportSideError, + RerunRecordDeletedError, + RunInProgressError, + RunVersions, + SideDefinition, + SimulationChoice, + UnknownCatalogVersionError, + UnknownHouseholdError, + UnknownPolicyError, +) +from sqlalchemy.exc import OperationalError + +DATABASE_ERROR = OperationalError("SELECT 1", {}, Exception("server closed")) +DATABASE_URL = "postgresql://simulation_service:secret@db.example:5432/postgres" +CONTEXT = {"observability_id": "observed"} +BASELINE_SIMULATION_ID = UUID("00000000-0000-4000-8000-0000000000e1") +REFORM_SIMULATION_ID = UUID("00000000-0000-4000-8000-0000000000e2") +# The household report service's IDs, in the order it takes them: the new +# report run, then each side's new simulation run. +RUN_ID, BASELINE_RUN_ID, REFORM_RUN_ID = (UUID(int=number) for number in range(1, 4)) +# The stored report calculates 2025, which its household's default year +# (2026, in the fake store) no longer is. +STORED_HOUSEHOLD_REPORT = ReportDefinition( + id=REPORT_ID, + country="us", + kind="household", + year=2025, + region_code=None, + dataset_name=None, + inputs={}, + baseline=SideDefinition( + simulation_id=BASELINE_SIMULATION_ID, + choice=SimulationChoice(policy_id=None, dynamic_id=None), + ), + reform=SideDefinition( + simulation_id=REFORM_SIMULATION_ID, + choice=SimulationChoice(policy_id=POLICY_ID, dynamic_id=None), + ), + household_id=HOUSEHOLD_ID, +) +BASELINE_ALONE = replace(STORED_HOUSEHOLD_REPORT, reform=None) + + +def _services( + definition: ReportDefinition = STORED_HOUSEHOLD_REPORT, + *, + households: FakeHouseholdReportStore | None = None, + workers: FakeHouseholdWorkers | None = None, + caller: FakeHouseholdReportCaller | None = None, +): + """Return the re-run service, its economy store and its economy caller.""" + economy_store = FakeRerunStore(report_definition=definition) + economy_caller = FakeCaller() + service = rerun_service( + economy_store, + caller=economy_caller, + household_reports=household_rerun_service( + households or FakeHouseholdReportStore(), + workers=workers, + caller=caller, + ), + ) + return service, economy_store, economy_caller + + +def _rerun(service, *, force: bool = False): + return asyncio.run( + service.rerun_report( + REPORT_ID, + ReportRerunRequest(force=force), + request_id="observed", + observability_context=CONTEXT, + ) + ) + + +def _side_request(calculation_id: UUID, parameters: dict) -> dict: + worker = manifest_household_worker("us") + return { + "schema_version": 1, + "calculation_id": str(calculation_id), + "country": "us", + "year": "2025", + "household": SITUATION, + "policy": {"parameters": parameters}, + "add_missing_variables": True, + "spm_selection_source": "request", + "bundle": worker.bundle.model_dump(mode="json"), + } + + +# A household run is never reused, so a forced re-run creates the same runs. +@pytest.mark.parametrize("force", [False, True]) +def test_a_household_report_is_rerun_as_new_runs_of_its_own_simulations( + force, +) -> None: + households = FakeHouseholdReportStore() + caller = FakeHouseholdReportCaller() + service, economy_store, economy_caller = _services( + households=households, caller=caller + ) + worker = manifest_household_worker("us") + + rerun = _rerun(service, force=force) + + assert households.reads == [ + { + "country": "us", + "household_id": HOUSEHOLD_ID, + "policy_ids": {"baseline": None, "reform": POLICY_ID}, + "policyengine_version": worker.bundle.policyengine_version, + } + ] + assert households.report_reruns == [ + NewHouseholdReportRerun( + report_id=REPORT_ID, + run_id=RUN_ID, + tax_benefit_model_version_id=CATALOG_VERSION_ID, + year=2025, + versions=RunVersions( + country_package_version=worker.bundle.country_package_version, + policyengine_version=worker.bundle.policyengine_version, + ), + baseline=HouseholdRerunSide( + simulation_id=BASELINE_SIMULATION_ID, simulation_run_id=BASELINE_RUN_ID + ), + reform=HouseholdRerunSide( + simulation_id=REFORM_SIMULATION_ID, simulation_run_id=REFORM_RUN_ID + ), + ) + ] + ((called_worker, payload, report_run_id, context),) = caller.calls + assert called_worker == worker + assert (report_run_id, context) == (RUN_ID, CONTEXT) + assert payload == { + "schema_version": 1, + "baseline": _side_request(BASELINE_RUN_ID, {}), + "reform": _side_request(REFORM_RUN_ID, REFORM), + } + HouseholdReportRunCalculation.model_validate(payload) + assert households.recorded_calls == [(RUN_ID, "fc-household-report")] + assert rerun == HouseholdReportCreated( + report_id=REPORT_ID, + report_run_id=RUN_ID, + status=ReportRunStatus.PENDING, + year=2025, + baseline_simulation_id=BASELINE_SIMULATION_ID, + baseline_simulation_run_id=BASELINE_RUN_ID, + reform_simulation_id=REFORM_SIMULATION_ID, + reform_simulation_run_id=REFORM_RUN_ID, + ) + # Nothing of it goes through the economy store or coordinator. + assert economy_store.rerun_reads == [] + assert economy_caller.report_runs == [] + + +def test_a_household_report_of_the_baseline_alone_is_rerun_without_a_reform() -> None: + households = FakeHouseholdReportStore() + caller = FakeHouseholdReportCaller() + service, _, _ = _services(BASELINE_ALONE, households=households, caller=caller) + + rerun = _rerun(service) + + (read,) = households.reads + assert read["policy_ids"] == {"baseline": None} + (created,) = households.report_reruns + assert created.reform is None + assert created.simulation_run_ids == (BASELINE_RUN_ID,) + ((_, payload, _, _),) = caller.calls + assert payload["reform"] is None + assert (rerun.reform_simulation_id, rerun.reform_simulation_run_id) == (None, None) + + +@pytest.mark.parametrize( + ("changes", "named"), + [ + ({"household_id": None}, "household"), + ({"baseline": None}, "baseline simulation"), + ({"year": None}, "year"), + ({"country": "fr"}, "country"), + # A household calculation applies no dynamic, so a side naming one + # would be calculated as it was not stored. + ( + { + "baseline": SideDefinition( + simulation_id=BASELINE_SIMULATION_ID, + choice=SimulationChoice(policy_id=None, dynamic_id=DYNAMIC_ID), + ) + }, + "dynamic", + ), + ( + { + "reform": SideDefinition( + simulation_id=REFORM_SIMULATION_ID, + choice=SimulationChoice(policy_id=POLICY_ID, dynamic_id=DYNAMIC_ID), + ) + }, + "dynamic", + ), + ], +) +def test_a_household_report_the_service_cannot_rerun_is_refused(changes, named) -> None: + households = FakeHouseholdReportStore() + service, _, _ = _services( + replace(STORED_HOUSEHOLD_REPORT, **changes), households=households + ) + + with pytest.raises(EconomyRerunConflictError) as refused: + _rerun(service) + + assert refused.value.code == "rerun_not_supported" + assert named in refused.value.detail + assert households.reads == [] + + +def test_a_household_rerun_answers_503_when_household_reports_are_not_configured() -> ( + None +): + store = FakeRerunStore(report_definition=STORED_HOUSEHOLD_REPORT) + + with pytest.raises(EconomySubmissionUnavailableError) as unavailable: + _rerun(rerun_service(store)) + + assert unavailable.value.code == "not_configured" + assert unavailable.value.detail == "Household reports are not configured." + + +def test_a_household_report_that_cannot_be_rerun_is_refused_even_unconfigured() -> None: + """A report that can never be re-run is not answered as one to retry.""" + dynamic_reform = replace( + STORED_HOUSEHOLD_REPORT, + reform=SideDefinition( + simulation_id=REFORM_SIMULATION_ID, + choice=SimulationChoice(policy_id=POLICY_ID, dynamic_id=DYNAMIC_ID), + ), + ) + store = FakeRerunStore(report_definition=dynamic_reform) + + with pytest.raises(EconomyRerunConflictError) as refused: + _rerun(rerun_service(store)) + + assert refused.value.code == "rerun_not_supported" + + +def test_no_household_function_for_the_country_is_refused() -> None: + households = FakeHouseholdReportStore() + service, _, _ = _services( + households=households, + workers=FakeHouseholdWorkers( + HouseholdFunctionUnavailableError("no household function") + ), + ) + + with pytest.raises(EconomySubmissionRefusedError) as refused: + _rerun(service) + + assert refused.value.code == "no_household_function" + assert households.reads == [] + + +@pytest.mark.parametrize( + ("failure", "code", "named"), + [ + (UnknownCatalogVersionError("none"), "unknown_catalog_version", "catalog"), + (UnknownHouseholdError("gone"), "unknown_household", "the report names"), + ( + ReportSideError("reform", UnknownPolicyError("gone")), + "unknown_policy", + "the report's reform names", + ), + ], +) +def test_a_stored_row_missing_from_the_catalog_is_refused_as_the_reports( + failure, code, named +) -> None: + households = FakeHouseholdReportStore(read_failure=failure) + service, _, _ = _services(households=households) + + with pytest.raises(EconomySubmissionRefusedError) as refused: + _rerun(service) + + assert refused.value.code == code + assert named in refused.value.detail + assert households.report_reruns == [] + + +def test_a_policy_setting_parameters_the_catalog_lacks_is_located_at_the_report() -> ( + None +): + households = FakeHouseholdReportStore( + read_failure=ReportSideError( + "reform", MissingPolicyParametersError([CTC_AMOUNT]) + ) + ) + service, _, _ = _services(households=households) + + with pytest.raises(EconomySubmissionUnprocessableError) as unprocessable: + _rerun(service) + + assert unprocessable.value.code == "missing_parameters" + assert [reason.loc for reason in unprocessable.value.reasons] == [ + ("report", "reform", "policy_id", CTC_AMOUNT) + ] + + +def test_a_stored_household_the_contract_cannot_read_is_located_at_the_report() -> None: + households = FakeHouseholdReportStore(document={"people": "not a list of people"}) + service, _, _ = _services(households=households) + + with pytest.raises(EconomySubmissionUnprocessableError) as unprocessable: + _rerun(service) + + assert unprocessable.value.code == "household_not_calculable" + assert [reason.loc for reason in unprocessable.value.reasons] == [ + ("report", "household_id") + ] + assert households.report_reruns == [] + + +def test_a_household_rerun_whose_inputs_cannot_be_read_answers_503() -> None: + service, _, _ = _services( + households=FakeHouseholdReportStore(read_failure=DATABASE_ERROR) + ) + + with pytest.raises(EconomySubmissionUnavailableError) as unavailable: + _rerun(service) + + assert unavailable.value.code == "database_unavailable" + + +def test_a_household_report_with_a_run_under_way_is_not_rerun() -> None: + households = FakeHouseholdReportStore( + rerun_failure=RunInProgressError(RUN_UNDER_WAY_ID) + ) + caller = FakeHouseholdReportCaller() + service, _, _ = _services(households=households, caller=caller) + + with pytest.raises(EconomyRerunConflictError) as refused: + _rerun(service) + + assert (refused.value.code, refused.value.run_id) == ( + "run_in_progress", + RUN_UNDER_WAY_ID, + ) + assert caller.calls == [] + + +def test_a_household_report_or_simulation_deleted_after_its_read_is_answered() -> None: + deleted_report, _, _ = _services( + households=FakeHouseholdReportStore(rerun_failure=RerunRecordDeletedError()) + ) + deleted_simulation, _, _ = _services( + households=FakeHouseholdReportStore( + rerun_failure=EconomyRecordReferenceError( + "simulation_runs", "simulation_id" + ) + ) + ) + + with pytest.raises(EconomyRecordNotFoundError) as not_found: + _rerun(deleted_report) + with pytest.raises(EconomySubmissionRefusedError) as refused: + _rerun(deleted_simulation) + + assert not_found.value.record == "report" + assert refused.value.code == "missing_reference" + assert refused.value.detail == "a simulation the report names no longer exists" + + +def test_a_household_rerun_whose_commit_is_unconfirmed_may_have_been_written() -> None: + commit_failed = EconomyRecordOutcomeUnknownError("the commit failed") + commit_failed.__cause__ = DATABASE_ERROR + service, _, _ = _services( + households=FakeHouseholdReportStore(rerun_failure=commit_failed) + ) + + with pytest.raises(EconomySubmissionOutcomeUnknownError) as unknown: + _rerun(service) + + assert (unknown.value.record, unknown.value.rerun) == ("report", True) + assert unknown.value.error_type == "OperationalError" + + +def test_a_household_rerun_modal_refuses_is_failed_and_named() -> None: + households = FakeHouseholdReportStore() + service, _, _ = _services( + households=households, + caller=FakeHouseholdReportCaller(failure=RuntimeError("Modal refused")), + ) + + with pytest.raises(EconomySubmissionUnavailableError) as unavailable: + _rerun(service) + + assert unavailable.value.code == "modal_call_refused" + assert (unavailable.value.record_id, unavailable.value.run_id) == ( + REPORT_ID, + RUN_ID, + ) + ((failed_run, failed_simulation_runs, message),) = households.failed_runs + assert (failed_run, failed_simulation_runs) == ( + RUN_ID, + (BASELINE_RUN_ID, REFORM_RUN_ID), + ) + assert message.startswith("modal_call_refused:") + assert households.recorded_calls == [] + + +def test_the_app_reruns_household_reports_through_its_household_report_service( + monkeypatch, +) -> None: + household_reports = household_rerun_service(FakeHouseholdReportStore()) + builders_given: list[dict] = [] + monkeypatch.setattr( + app_module, + "household_reports_for", + lambda settings, database: household_reports, + ) + monkeypatch.setattr( + app_module, + "economy_submissions_for", + lambda settings, database, **builders: builders_given.append(builders), + ) + + create_app( + settings=make_settings(), backend=FakeBackend(), auth_dependency=lambda: None + ) + + assert builders_given == [{"household_reports": household_reports}] + + +def test_a_configured_service_builds_a_household_report_service_that_reruns() -> None: + """Nothing reaches the database or Modal until the first submission.""" + database = SimulationDatabase.from_url(DATABASE_URL) + try: + household_reports = app_module.household_reports_for( + make_settings( + simulation_database_url=DATABASE_URL, v2_modal_environment="staging" + ), + database, + ) + finally: + database.dispose() + + assert isinstance(household_reports, HouseholdReportRerunService) diff --git a/projects/policyengine-simulation-entry/tests/test_household_report_routes.py b/projects/policyengine-simulation-entry/tests/test_household_report_routes.py index 3f250723d..f5ad1c753 100644 --- a/projects/policyengine-simulation-entry/tests/test_household_report_routes.py +++ b/projects/policyengine-simulation-entry/tests/test_household_report_routes.py @@ -79,7 +79,7 @@ def build( monkeypatch.setattr( app_module, "economy_submissions_for", - lambda settings, database: economy or FakeEconomySubmissions(), + lambda settings, database, **_: economy or FakeEconomySubmissions(), ) monkeypatch.setattr( app_module, diff --git a/projects/policyengine-simulation-entry/tests/test_openapi.py b/projects/policyengine-simulation-entry/tests/test_openapi.py index de8226b4d..248e2f478 100644 --- a/projects/policyengine-simulation-entry/tests/test_openapi.py +++ b/projects/policyengine-simulation-entry/tests/test_openapi.py @@ -40,9 +40,11 @@ class OpenAPIDocument(TypedDict): ("POST", "/household-calculations"), ("POST", "/ping"), ("POST", "/reports"), + ("POST", "/reports/{report_id}/runs"), ("POST", "/simulate/economy/budget-window"), ("POST", "/simulate/economy/comparison"), ("POST", "/simulations"), + ("POST", "/simulations/{simulation_id}/runs"), } @@ -86,8 +88,10 @@ def operation_ids(spec: OpenAPIDocument) -> dict[tuple[str, str], str]: ECONOMY_PATHS = ( "/reports", "/reports/{report_id}", + "/reports/{report_id}/runs", "/simulations", "/simulations/{simulation_id}", + "/simulations/{simulation_id}/runs", ) ECONOMY_SCHEMAS = ( "BudgetWindowReportCreated", @@ -106,6 +110,7 @@ def operation_ids(spec: OpenAPIDocument) -> dict[tuple[str, str], str]: "HouseholdReportCreated", "HouseholdReportRequest", "PolicyChoice", + "ReportRerunRequest", "ReportRunRead", "ReportRunStatus", "SimulationRunRead", @@ -390,7 +395,9 @@ def test_economy_routes_declare_every_answer(): assert f"{record}_id" in responses["503"]["description"] assert "nothing was written" in responses["400"]["description"] error_properties = spec["components"]["schemas"]["EconomyError"]["properties"] - assert {"code", "detail", "report_id", "simulation_id"} == set(error_properties) + assert {"code", "detail", "report_id", "simulation_id", "run_id"} == set( + error_properties + ) codes = set(error_properties["code"]["enum"]) assert { "unknown_region", @@ -417,6 +424,60 @@ def test_every_economy_503_declares_its_retry_after_header(): assert headers["Retry-After"]["schema"] == {"type": "integer"} +def test_rerun_routes_declare_every_answer(): + spec = create_app().openapi() + error = {"$ref": "#/components/schemas/EconomyError"} + unprocessable = {"$ref": "#/components/schemas/EconomyRequestUnprocessable"} + + for path, rerun in ( + ( + "/reports/{report_id}/runs", + ( + "EconomyReportCreated", + "BudgetWindowReportCreated", + "HouseholdReportCreated", + ), + ), + ("/simulations/{simulation_id}/runs", ("EconomySimulationCreated",)), + ): + responses = _responses(spec, "post", path) + assert set(responses) == { + "201", + "400", + "403", + "404", + "409", + "422", + "500", + "503", + } + rerun_schema = _response_schema(responses["201"]) + assert rerun_schema.get("anyOf", [rerun_schema]) == [ + {"$ref": f"#/components/schemas/{name}"} for name in rerun + ] + for status_code in ("400", "404", "409", "500", "503"): + assert _response_schema(responses[status_code]) == error + assert _response_schema(responses["422"]) == unprocessable + assert "run_id" in responses["409"]["description"] + assert "run_id" in responses["503"]["description"] + codes = set( + spec["components"]["schemas"]["EconomyError"]["properties"]["code"]["enum"] + ) + assert {"run_in_progress", "rerun_not_supported"} <= codes + # A household report's re-run names its household's own codes. + report_answers = _responses(spec, "post", "/reports/{report_id}/runs") + assert "unknown_household" in report_answers["400"]["description"] + assert "household_not_calculable" in report_answers["422"]["description"] + report_rerun = spec["components"]["schemas"]["ReportRerunRequest"] + assert set(report_rerun["properties"]) == {"force"} + assert ( + "requestBody" not in spec["paths"]["/simulations/{simulation_id}/runs"]["post"] + ) + names = operation_ids(cast("OpenAPIDocument", spec)) + assert names[("POST", "/reports/{report_id}/runs")] == "rerun_report" + assert names[("POST", "/simulations/{simulation_id}/runs")] == "rerun_simulation" + + def test_economy_routes_name_their_operations(): names = operation_ids(cast("OpenAPIDocument", create_app().openapi()))