Skip to content

BoE SVAR: re-pin to 17f5d82 and refresh every BoE figure (merging redeploys the hosted MCP server) - #282

Open
vahid-ahmadi wants to merge 1 commit into
mainfrom
vahid/boe-repin-17f5d82
Open

vahid-ahmadi wants to merge 1 commit into
mainfrom
vahid/boe-repin-17f5d82

Conversation

@vahid-ahmadi

@vahid-ahmadi vahid-ahmadi commented Nov 21, 2026 •

Copy link
Copy Markdown
Contributor

Re-pins the site's BoE SVAR dependency from c827ab8 to boe-var-model main 17f5d82 (after boe-var-model #34, world CPI seasonal adjustment, and #35, November 2026 MPR comparator and 2026Q3 scoring) and moves every BoE figure the site quotes.

Merging this redeploys the hosted MCP server (deploy-mcp.yml runs on changes under integration/). After the redeploy forecast_uk and latest_shocks run 17f5d82. Nothing was redeployed from this branch.

What changed

  • Pins: integration/pyproject.toml and integration/modal_app.py (BOE_REVISION) to 17f5d8247e72e93e1d4bd34731666fea9d655ba1.
  • Evaluation files: papers/boe-svar/figures/rolling_evaluation.json and coverage_evaluation.json are byte copies of results/ at 17f5d82. Every chart on /svar/validation is regenerated from them.
  • Comparison numbers: make_comparison.py production constants updated from results/summary.md at 17f5d82; the 600-draw check configuration re-run locally against that checkout.
  • Shock reading: latest_shocks.json re-run locally at 17f5d82 (6,000 draws). It reproduces results/forecast_summary.md at 17f5d82 (430 accepted, ESS 185.6, world supply 0.55 / 0.45).
  • Band factors: forecasts/band_calibration.json follows the pin (band_scale_factors_yoy.production_spec); the forecast record and the homepage fan are redrawn by their generators.
  • Prose: /svar (overview, methodology, validation, use it), the paper landing page, the models page, the README table, the capability registry and the two topic pages that quote it.
  • Generators: the FEVD chart description now states the direction of each gap from the data (it hard-coded "short"); the outlook run note says which run is newer from the two files' provenance.

Before and after

Source for every "after" value is a file at boe-var-model 17f5d82 or a local run at that pin.

Figure Before (c827ab8) After (17f5d82) Source
Production run, accepted of 10,000 / ESS 765 / 368.0 731 / 333.3 results/summary.md
Global share of UK GDP variance, 1 year (68% band) 38.7% (24.0 to 53.1) 37.6% (23.2 to 50.3) results/summary.md
Global share of UK CPI variance, 1 year (68% band) 41.5% (25.6 to 56.3), 8.5pp short 52.0% (34.3 to 67.9), 2.0pp above results/summary.md
Domestic share, UK GDP / UK CPI 38.6% / 36.8% 39.0% / 24.8% (CPI about 15pp short of ~40%) results/summary.md
Identified total, UK GDP / UK CPI 77.3% / 78.3% 76.6% / 76.8% sums of the two rows above
Monetary policy share 8.9% of CPI (median table), "third of three" 7.2% of CPI, 11.2% of GDP (posterior means); third of three for GDP, second for CPI results/summary.md
Adding-up identity, max error 3.64e-12 8.19e-12 results/forecast_summary.md
Smallest adjusted q against drift (64 tests) 0.39 0.36 results/rolling_evaluation.json, multiplicity.headline.drift
No-change wins after adjustment, published spec 3 of 64 1 of 64 (world CPI, h=1, q = 0.048) same file, multiplicity.production_spec
Bank Rate against drift, h=1 (p) 0.79 (0.019) 0.80 (0.021); world CPI 0.86 has the same p same file
World CPI against drift, h=1 / h=8 0.94 / 1.07 0.86 (significant, unadjusted) / 1.06 same file
UK GDP against drift h=1; ex-Covid h=1 vs no-change / vs drift 1.06; 0.74 / 1.20 1.07; 0.78 / 1.27 same file
UK GDP against drift, p range 0.33 to 0.43 0.32 to 0.42 same file
68% band, mean of 8 variables, h=1 / h=8 70% / 57% 72% / 57%; at or below nominal from h=2 (was h=3) results/coverage_evaluation.json, coverage
UK CPI 90% band, year on year, h=1 / h=8 78% / 54% 76% / 56% same file, production_spec.coverage_yoy
68% band with Covid dummies, year on year, h=8 59% 58% same file
Held-out rescaled 90% coverage (95% interval) 82.7% (68 to 95) 82.5% (67 to 95) same file; docs/band_decision.md
Conformal 90% coverage 83.2% 83.0% same file, conformal_calibration
Pandemic volatility scaling, alone / with rescaling 81.2% / 86.9% 80.7% / 87.4% results/covid_volatility_evaluation.json
Satellite RMSE vs no-change, ex-Covid; all targets h2 to h4 0.82 to 0.99; 1.12 to 1.35 0.82 to 0.98; 1.12 to 1.34 results/unemployment_satellite_validation.json
Pre-Covid-sample CPI plateau 2.0% near 2.1% results/forecast_vs_boe_obr.md (2.10 / 2.05 in 2028Q3 / 2029Q3)
Shock reading 2026Q2, 6,000 draws 455 accepted, ESS 181.4 430 accepted, ESS 185.6 local run; matches results/forecast_summary.md
World supply shock negative, 70% no call (55% positive) same
World demand / UK demand / UK supply / monetary policy 95 / 95 / 75 / 80% 95 / 90 / 70 / 70% (nearest 5%) same
Hosted default, 2,000 draws 159 accepted, ESS 48.7 147 accepted, ESS 55.3 local run at 17f5d82
3,500 draws 264 accepted, ESS 91.5 260 accepted, ESS 106.1 local run at 17f5d82
Check configuration (600 draws) 51 accepted; GDP 40.1%, CPI 40.1% 45 accepted; GDP 39.3%, CPI 53.9% local run of make_comparison.py

Band factors (forecasts/band_calibration.json): eight of 32 moved, seven by one step of measured coverage (one origin in 50) and CPI 68% h1 by two.

Factor Before After
GDP 68% h1 / h3 0.8464 / 1.1328 0.8829 / 1.1816
GDP 90% h7 1.3999 1.4603
CPI 68% h1 / h2 / h4 1.1816 / 1.3460 / 1.7062 1.2878 / 1.2878 / 1.6228
CPI 90% h1 / h8 1.3411 / 2.2262 1.3999 / 2.1301

Published fan cells these factors moved (12 Nov 2026 round on /forecasts; medians unchanged; the homepage fan is the same bands):

Cell Before After
GDP 2026Q3, 68% 1.09 to 2.06 1.07 to 2.08
GDP 2027Q1, 68% 0.26 to 2.67 0.21 to 2.72
GDP 2028Q1, 90% −1.64 to 4.19 −1.77 to 4.32
CPI 2026Q3, 68% / 90% 2.43 to 3.56 / 1.94 to 4.02 2.38 to 3.61 / 1.89 to 4.07
CPI 2026Q4, 68% 2.40 to 4.50 2.45 to 4.46
CPI 2027Q2, 68% 1.38 to 5.81 1.49 to 5.70
CPI 2028Q2 to 2029Q3, 90% e.g. 2029Q3 −2.34 to 8.69 e.g. 2029Q3 −2.11 to 8.45

Added, not replaced

  • /svar/validation: the November 2026 MPR beside the model's 17 Nov run (every Bank central figure for CPI, GDP and unemployment from 2026Q4 to 2029Q4 inside the 68% band; CPI gaps 0.3, 0.7, then 1.2 and 1.3pp), from results/forecast_vs_boe_obr.md. The August 2024 MPR line stays: it is the matched vintage for the frozen 2024Q2 origin.
  • /svar/validation, satellite list: the 28 Jul round forecast 2026Q3 unemployment at 4.96% (90% band 4.85 to 5.08); the outturn was 4.64%, 0.21pp below the lower edge, from results/score_2026q3.md.

Decision log

  • CPI global share stays "partial". The mean is now 2.0pp from the paper, but results/sense_check.md at 17f5d82 still lists it as open ("close on the mean, the band is wide"), so the site says that and does not call it a pass. The status string in the registry changes from "CPI global-shock share about 8.5pp short" to "global share of CPI variance close on the mean, with a wide band; domestic CPI share about 15pp short".
  • Band factors follow the pin, as in the c13a9c9 re-pin. The rounds on the page were produced before Paper polish: no overflowing equations + ours-vs-published comparison sections #34; the file now says so (model_vintage). The alternative, holding the old factors until a post-Paper polish: no overflowing equations + ours-vs-published comparison sections #34 round is archived, would leave the first new round drawn with factors from another model version.
  • current_forecast.json is not regenerated. It mirrors the archived 12 Nov 2026 round, which predates Paper polish: no overflowing equations + ours-vs-published comparison sections #34; the overview and methodology now say so. The next scheduled round replaces it.
  • Drift canary not re-anchored. test_svar_canary_matches_the_committed_forecast_file ties it to the archived round. A local 2,000-draw call at 17f5d82 gave GDP 1.632 and CPI 2.879 against expected 1.567 and 2.971: gaps of 0.065 and 0.092 against a tolerance of 0.1. It passes, but CPI is close, so validate-deployment could fail on sampling noise until a post-Paper polish: no overflowing equations + ours-vs-published comparison sections #34 round is archived. The tolerance is unchanged; the basis text records the gap.
  • Working paper not rebuilt. Its abstract and comparison section are written around the 8.5pp CPI shortfall, so a refresh is a rewrite, not a number swap. The landing page now labels the PDF as the Rename technical identifiers: macromod → policyengine-macro #24 vintage and points to the validation tab for current figures.
  • Figures labelled "pre-refresh, not recounted" are left (4 of 744 stable draws, about 1.6% of GDP per 100bp): no file at 17f5d82 restates them.
  • Compare page untouched (another branch is editing BoE comparison text there).
  • Commit trailer names Claude Fable 5.2, the model that did this work; the round brief asks for Opus 5.5.
  • Working paper not attempted further: the unweighted 3,000-draw run behind its figures (38.1 / 42.3%) would also need re-running for the text to stay consistent.

Checks

  • Every generator --check in site-provenance.yml and forecast-archive.yml, site_nav.py, site_contract.py, data/fetch.py --check, tools/model_hero.py --check: pass locally. tools/prerender_math.py --check needs npm ci, so it is left to CI.
  • python3 -m pytest tests -q: 3279 passed, 87 skipped. forecasts/tests: 50 passed. integration fast suite with boe_var installed at 17f5d82: 495 passed, 79 skipped.
  • No file under data/vintages/ or forecasts/rounds/ is touched.

What is left

  • Working paper (papers/boe-svar/main.tex, sections/*.tex, main.pdf, figures/fig_comparison.pdf): still the Rename technical identifiers: macromod → policyengine-macro #24 vintage. comparison_numbers.json is now ahead of the tex.
  • Compare page: models/compare/index.html, models/compare/build.py and tests/test_compare.py:155-156 still say "765 accepted"; it is 731 at 17f5d82.
  • Dated note notes/2026-07-25-cpi-2026q2/ quotes "smallest q = 0.39"; left as a dated record.
  • First post-Paper polish: no overflowing equations + ours-vs-published comparison sections #34 round: not archived yet, so the current-forecast chart and the canary still rest on the 12 Nov round.
  • Upstream wording: results/summary.md at 17f5d82 still says the replication "falls materially short on UK CPI" beside a 52.0% table. The site quotes the table and sense_check.md.
  • Site scorecard for 2026Q3: not touched here; it scores when forecasts/outturns.json ingests the outturn from the data store.

🤖 Generated with Claude Code

…ov 2026 MPR comparator) and move every BoE number

- Pins: integration/pyproject.toml and modal_app.py BOE_REVISION c827ab86 -> 17f5d824
- Replication (results/summary.md at 17f5d82): 765 -> 731 accepted of 10,000, ESS 368.0 -> 333.3; UK GDP global share 38.7 -> 37.6%; UK CPI global share 41.5 -> 52.0% (paper ~50%; close on the mean, wide band, verdict stays partial); UK CPI domestic share 36.8 -> 24.8% (paper ~40%), now the open gap; monetary policy 11.2% of UK GDP variance against ~20% read off the paper's figure
- Evaluation files copied from 17f5d82 (rolling_evaluation.json, coverage_evaluation.json); charts regenerated; smallest drift q 0.39 -> 0.36; no-change wins after adjustment 3 of 64 -> 1 of 64; UK CPI 90% coverage 78/54 -> 76/56%; 68% mean 70 -> 72% at h1
- Shocks (local run at 17f5d82, 6,000 draws, 430 accepted, ESS 185.6; reproduces results/forecast_summary.md): world supply negative 70% -> no call (55% positive); UK demand 95 -> 90%; UK supply 75 -> 70%; monetary policy 80 -> 70%
- Hosted default re-measured locally: 2,000 draws -> 147 accepted, ESS 55.3 (was 159 / 48.7); 3,500 -> 260 / 106.1
- Band factors follow the pin (band_scale_factors_yoy.production_spec): eight of 32 moved (seven by one step of measured coverage, one by two); forecast record and hero fans redrawn; archived rounds untouched
- Validation page: November 2026 MPR comparator and the 2026Q3 unemployment outturn against the 28 Jul round (4.96%, 90% 4.85-5.08; outturn 4.64%), from results/forecast_vs_boe_obr.md and results/score_2026q3.md at 17f5d82
- FEVD chart description states the direction of each gap from the data
- Not changed: current_forecast.json (the archived 12 Nov round, which predates #34), the working-paper tex and PDF (the #24 vintage, labelled on the landing page), models/compare

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
@vercel

vercel Bot commented Nov 21, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
macromod Building Building Preview Nov 21, 2026 9:41pm UTC

Request Review

This branch was successfully deployed

1 active deployment
Preview — 9be43a53 Deployed Nov 21, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant