Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions changelog.d/741.removed.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Removed `policyengine.provenance.supported_years`, `UnsupportedDatasetYearError` and the `allow_unsupported_years` argument of the UK and US `create_datasets` and `ensure_datasets`, which no longer check requested years; the Public API defines the supported years instead. The US functions again create year files for any requested year, including years before the dataset's data year such as 2022 and 2023 for `populace_us_2024`, which holds almost no data for them, and a UK year outside policyengine-uk's projection again fails with `No dataset found for year ...`.
36 changes: 0 additions & 36 deletions docs/microsim.md
Original file line number Diff line number Diff line change
Expand Up @@ -152,42 +152,6 @@ List datasets already known to the country:
pe.us.load_datasets() # or pe.uk.load_datasets()
```

### Supported years

A dataset is supported only for the years the installed country model
projects it to. `supported_years` builds that projection from a single-year
dataset file as the model's own `Simulation` does, with the model's
`extend_single_year_dataset` and without running any formula, and returns
the years from the dataset's data year onwards:

```python
from policyengine.provenance import supported_years

supported_years("us", "./data/populace_us_2024.h5") # 2024 to 2035
supported_years("uk", "./data/microcosm_uk_2024_25.h5") # 2024 to 2032
```

- **UK:** from the data year to policyengine-uk's default projection end,
`PROJECTION_END_YEAR` (2032 in policyengine-uk 2.151.0), or the end of a
capital gains projection that ends earlier.
- **US:** from the data year to the earlier of policyengine-us's default
projection end (the last year in its `gov.bls.cpi.cpi_u` parameter file,
2035 in policyengine-us 2.2.1) and the last year of spm-calculator's SPM
forecast (2035 in spm-calculator 1.0.0), after which the poverty figures
have no thresholds. Society-wide runs are not supported for years before
the data year, such as 2022 and 2023 for Populace US 2024, because the
certified dataset holds almost no data for them.

A calculation for another year still returns results, because
policyengine-core carries the latest projected inputs forward or uprates
them, so a successful run is not evidence of support. `create_datasets` and
`ensure_datasets` therefore raise `UnsupportedDatasetYearError` (from
`policyengine.provenance`) for any other year before they create a year
file. The error's `country_id`, `dataset`, `years` and `supported_years`
attributes name the request and the years that are supported. Pass
`allow_unsupported_years=True` to create the year files anyway. Year files
that already exist in the data folder are loaded without the check.

### US local-area dataset

Alongside the certified national default, the bundle registers a US dataset for
Expand Down
7 changes: 0 additions & 7 deletions src/policyengine/provenance/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,6 @@
.. code-block:: python

from policyengine.provenance import (
supported_years,
get_release_manifest,
get_data_release_manifest,
build_trace_tro_from_release_bundle,
Expand Down Expand Up @@ -43,12 +42,6 @@
from .dataset_materialization import (
materialize_dataset as materialize_dataset,
)
from .dataset_years import (
UnsupportedDatasetYearError as UnsupportedDatasetYearError,
)
from .dataset_years import (
supported_years as supported_years,
)
from .manifest import (
CertifiedDataArtifact as CertifiedDataArtifact,
)
Expand Down
192 changes: 0 additions & 192 deletions src/policyengine/provenance/dataset_years.py

This file was deleted.

22 changes: 16 additions & 6 deletions src/policyengine/provenance/receipt.py
Original file line number Diff line number Diff line change
Expand Up @@ -127,7 +127,6 @@
model_validator,
)

from policyengine.provenance import dataset_years
from policyengine.provenance.bytecode import code_shape as _code_shape
from policyengine.provenance.bytecode import compile_source
from policyengine.provenance.canonical import (
Expand Down Expand Up @@ -3819,14 +3818,25 @@ def _receipt_years(years: Any) -> list[int]:
def _projected_years(
dataset_path: str, dataset_sha256: str, model_version: str
) -> tuple[int, ...]:
"""Years the installed policyengine-uk supports a single-year dataset for.
"""Years the installed policyengine-uk projects a single-year dataset to.

:func:`policyengine.provenance.dataset_years.supported_years` for the UK,
the rule the qualification ``supported_years`` check also uses, cached by
file bytes and model version.
The same rule as the qualification ``supported_years`` gate: build the
multi-year dataset as ``policyengine_uk.Simulation`` does for a
single-year file, without running a formula, and keep the years from the
dataset's own year onwards. Cached by file bytes and model version.
"""

return tuple(dataset_years.supported_years("uk", dataset_path))
from policyengine_uk.data.dataset_schema import UKSingleYearDataset
from policyengine_uk.data.economic_assumptions import extend_single_year_dataset
from policyengine_uk.system import system

dataset = UKSingleYearDataset(dataset_path)
first = int(str(dataset.time_period)[:4])
multi_year = extend_single_year_dataset(dataset, system.parameters)
years = getattr(multi_year, "years", None)
if years is None:
years = list(multi_year.datasets)
return tuple(sorted(int(year) for year in years if int(year) >= first))


def _certificate_fields(
Expand Down
22 changes: 15 additions & 7 deletions src/policyengine/provenance/uk_candidate.py
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,6 @@
from packaging.specifiers import SpecifierSet
from packaging.version import Version

from policyengine.provenance import dataset_years
from policyengine.provenance.manifest import DataReleaseManifest, _specifier_matches
from policyengine.provenance.qualification import (
CandidatePair,
Expand Down Expand Up @@ -423,15 +422,24 @@ def data_year(dataset_path: Path) -> int:


def projected_years_from_model(dataset_path: Path) -> list[int]:
"""The years the installed policyengine-uk supports a dataset for.
"""The years policyengine-uk projects a single-year dataset to.

:func:`policyengine.provenance.dataset_years.supported_years` for the UK:
the years ``extend_single_year_dataset`` projects the single-year file
to, built as ``policyengine_uk.Simulation`` builds them, from the data
year onwards.
Builds the multi-year dataset exactly as ``policyengine_uk.Simulation``
does for a single-year file (``extend_single_year_dataset`` with the
system's parameters), without running any formula.
"""

return dataset_years.supported_years("uk", dataset_path)
from policyengine_uk.data.dataset_schema import UKSingleYearDataset
from policyengine_uk.data.economic_assumptions import extend_single_year_dataset
from policyengine_uk.system import system

multi_year = extend_single_year_dataset(
UKSingleYearDataset(str(dataset_path)), system.parameters
)
years = getattr(multi_year, "years", None)
if years is None:
years = list(multi_year.datasets)
return sorted(int(year) for year in years)


def supported_years(
Expand Down
Loading
Loading