From 322d745b452e5e4cdbd3d1fd39e195cd9b98b502 Mon Sep 17 00:00:00 2001 From: igerber Date: Fri, 30 Oct 2026 10:32:33 -0400 Subject: [PATCH 1/2] feat(v4): item 8a of the 4.0 cut - retire the event-study reference-row sentinel (M-093, first half) Through 3.x every native event-study dict marked its normalization (reference) row with a zero-count sentinel: n_groups == 0 on the CallawaySantAnna aggregation, n_obs == 0 on StackedDiD, ImputationDiD, TwoStageDiD and SpilloverDiD. Every producer that materializes that row now marks it explicitly - "is_reference": True and NO count key - and every remaining zero count is a genuine count (a non-estimable horizon: TwoStageDiD's all-filtered horizons, SpilloverDiD's rectangular cells). SpilloverDiD's event-study att_dynamic / spillover_effects frames gain a boolean is_reference column with n_obs = NaN on the reference rows (aggregate-mode frames, which have no reference row, are unchanged). SunAbraham (omitted row, reference_period) and EfficientDiD (PT-Post anchor, an estimated cell named by reference_period) are untouched. The unified builder results_base._from_relative_dict reads only the is_reference key unioned with the named reference_period: the zero-count heuristic is gone, and a keyed row with a finite nonzero effect raises instead of being silently normalized to 0 by __post_init__. A reference-only TwoStageDiD surface now carries n_kind=None (no row records a count). 3.x pickles migrate on load: _deprecation.migrate_event_study_reference_sentinel rewrites a zero-count row whose effect is exactly 0.0 into the keyed shape (NaN-effect zero-count rows stay genuine) and is the single place the retired sentinel is still recognized - run by the __setstate__ of CallawaySantAnnaResults (DMLDiD inherits), StackedDiDResults, ImputationDiDResults, TwoStageDiDResults and a new SpilloverDiDResults.__setstate__ (which also adds the frame column for event-study-mode fits). A fresh pickle round-trips as a no-op. Consumers take the minimal swap: plot_event_study's raw-dict route reads the reference_period attribute, else the is_reference key, else today's -1 fallback; plot_honest_event_study's constraint-row signature and verified-anchor check, HonestDiD's native CS filter / reference detection / _estimate_max_pre_violation, PreTrendsPower's CS and SunAbraham filters and DiagnosticReport's pre-period / post-effect collectors replace the count conjunct with the key (genuine zero counts stay excluded from n_dropped_undefined). Dispatch, admission, rendering and exception types are unchanged. The Stacked / Imputation / TwoStage summary() renderers key their [ref: h] line on the marker - TwoStageDiD's genuine all-filtered horizons therefore print as NaN rows instead of a mislabeled reference line; the bootstrap loops skip the marked row. The native to_dataframe(level='event_study') frames render the reference row's n_obs as NaN (float column) until the 8b schema rewire. Tests: new tests/test_v4_sentinel_retirement.py (M-093 test_ref) pins the native dict and frame shape on every materializing producer, the CS carrier dict (captured at the builder boundary, not reconstructed), the count-agnostic builder (unkeyed zero count marks nothing; keyed row with NaN count; keyed nonzero effect raises; reference-only dict -> n_kind None), the pickle migration on every carrier plus the aggregate-mode Spillover no-op and fresh round trips, the consumers' key reads (plot extractor, honest plot anchor + all-undefined guard, raw/container parity for HonestDiD and PreTrendsPower, DiagnosticReport collector, summary(), bootstrap skip) and the ledger row. tests/helpers/legacy_surface emits the keyed shape; the enumerated count-key pins across the stacked, imputation, two-stage, spillover, staggered, honest, pretrends, consumer, methodology and diagnostic-report suites move to the key; the visualization / DR hand-built fixtures mark the reference by key (one DR fixture keeps the genuine zero-count shape). Ledger: M-093 gains test_ref and a ninth amendment note; it stays planned until 8b (the to_dataframe(level='event_study') schema rewire on every results class, incl. the dCDH native frame's estimand 'ref' row). Spec item 8 records 8a shipped; REGISTRY (SpilloverDiD reference-row shape, StackedDiD / ImputationDiD / TwoStageDiD reference-row Notes, the plotting and relay Notes, the CS post-fit Note), the migration guide, llms-full.txt and docs/api/spillover.rst describe the marking; notebook 13 re-validated with nbmake. --- ...261030-v4-pr8a-m093-sentinel-retirement.md | 7 + diff_diff/_deprecation.py | 56 +- diff_diff/diagnostic_report.py | 47 +- diff_diff/guides/llms-full.txt | 4 +- diff_diff/honest_did.py | 20 +- diff_diff/imputation.py | 1 + diff_diff/imputation_aggregation.py | 10 +- diff_diff/imputation_bootstrap.py | 4 +- diff_diff/imputation_results.py | 21 +- diff_diff/pretrends.py | 8 +- diff_diff/results.py | 55 +- diff_diff/results_base.py | 96 +-- diff_diff/spillover.py | 29 +- diff_diff/stacked_did.py | 6 +- diff_diff/stacked_did_results.py | 19 +- diff_diff/staggered_aggregation.py | 17 +- diff_diff/staggered_results.py | 11 +- diff_diff/two_stage.py | 1 + diff_diff/two_stage_aggregation.py | 6 +- diff_diff/two_stage_bootstrap.py | 4 +- diff_diff/two_stage_results.py | 19 +- diff_diff/visualization/_event_study.py | 30 +- docs/api/spillover.rst | 6 +- docs/methodology/REGISTRY.md | 26 +- docs/migration-4.0.md | 17 +- docs/v4-deprecations.yaml | 3 +- docs/v4-design.md | 7 +- tests/_capture_v4_aggregate_oracles.py | 8 +- tests/helpers/legacy_surface.py | 18 +- tests/test_diagnostic_report.py | 28 +- tests/test_event_study_consumers.py | 10 +- tests/test_event_study_surface.py | 68 ++- tests/test_honest_did.py | 6 +- tests/test_imputation.py | 12 +- tests/test_methodology_callaway.py | 3 +- tests/test_methodology_stacked_did.py | 4 +- tests/test_pretrends_event_study.py | 18 +- tests/test_spillover.py | 36 +- tests/test_stacked_did.py | 7 +- tests/test_staggered.py | 11 +- tests/test_two_stage.py | 12 +- tests/test_v4_sentinel_retirement.py | 572 ++++++++++++++++++ tests/test_visualization_new.py | 2 +- tests/test_visualization_plotly.py | 2 +- 44 files changed, 1091 insertions(+), 256 deletions(-) create mode 100644 changelog.d/20261030-v4-pr8a-m093-sentinel-retirement.md create mode 100644 tests/test_v4_sentinel_retirement.py diff --git a/changelog.d/20261030-v4-pr8a-m093-sentinel-retirement.md b/changelog.d/20261030-v4-pr8a-m093-sentinel-retirement.md new file mode 100644 index 000000000..9f45e8bc2 --- /dev/null +++ b/changelog.d/20261030-v4-pr8a-m093-sentinel-retirement.md @@ -0,0 +1,7 @@ +### Breaking Changes + +- Event-study reference rows are marked explicitly ([M-093], item 8a of the 4.0 cut): every native `event_study_effects` dict that materializes its normalization row (`StackedDiD`, `ImputationDiD`, `TwoStageDiD`, `SpilloverDiD`, the `CallawaySantAnna`/`DMLDiD` universal-base aggregation carrier) now carries `"is_reference": True` on that row and NO count key there - the 3.x `n_obs == 0` / `n_groups == 0` sentinel is retired, and every remaining zero count is a genuine count (a non-estimable horizon). `SpilloverDiD`'s event-study `att_dynamic` / `spillover_effects` frames gain a boolean `is_reference` column with `n_obs = NaN` on the reference rows (the column is float; `to_dict()` records follow). The native `to_dataframe(level="event_study")` frames of `StackedDiD` / bootstrapped `ImputationDiD` / `TwoStageDiD` render the reference row's `n_obs` as NaN (float column) until the 8b schema rewire replaces them. `plot_event_study`, `plot_honest_event_study`, HonestDiD, PreTrendsPower and `DiagnosticReport` locate the reference by the key on their raw-dict routes - dispatch, admission, rendering and exception types are unchanged. `TwoStageDiD.summary()` now prints its genuine all-filtered horizons as NaN rows instead of a mislabeled `[ref: h]` line. 3.x pickles are migrated on load. `SunAbraham` (omitted row, `reference_period`) and `EfficientDiD` (PT-Post anchor, an estimated cell named by `reference_period`) are unchanged. + +### Internal + +- `results_base._from_relative_dict` reads only the `is_reference` key and the named `reference_period` (the zero-count heuristic is gone; a keyed row with a finite nonzero effect raises); `_deprecation.migrate_event_study_reference_sentinel` is the single place the retired sentinel is still recognized, run by the carriers' `__setstate__` (`SpilloverDiDResults` gained one; aggregate-mode frames untouched). Ledger: M-093 gains `test_ref` (`tests/test_v4_sentinel_retirement.py`) and stays `planned` until 8b (the `to_dataframe(level="event_study")` schema rewire on every results class, incl. the dCDH native frame's `estimand 'ref'` row). REGISTRY (SpilloverDiD, StackedDiD/ImputationDiD/TwoStageDiD, the plotting Notes), the migration guide, `llms-full.txt` and `docs/api/spillover.rst` describe the marking. diff --git a/diff_diff/_deprecation.py b/diff_diff/_deprecation.py index 16fcfc429..dbae6d134 100644 --- a/diff_diff/_deprecation.py +++ b/diff_diff/_deprecation.py @@ -42,8 +42,9 @@ 4.0 cut). """ +import math import warnings -from typing import Any, Dict, Mapping +from typing import Any, Dict, Mapping, Optional __all__: "list[str]" = [] @@ -222,3 +223,56 @@ def migrate_renamed_state_keys( value = out.pop(old) out.setdefault(new, value) return out + + +def migrate_event_study_reference_sentinel( + effects: Optional[Mapping[Any, Any]], +) -> Optional[Dict[Any, Any]]: + """Rewrite 3.x zero-count reference rows of a pickled event-study dict. + + Through 3.x the native ``event_study_effects`` dicts marked their + normalization (reference) row with a zero-count SENTINEL (``n_groups == 0`` + on CallawaySantAnna, ``n_obs == 0`` on StackedDiD / ImputationDiD / + TwoStageDiD / SpilloverDiD). Since 4.0 (ledger row M-093, item 8a of the + cut) the reference row carries an explicit ``"is_reference": True`` key and + NO count key, and every remaining zero count is a genuine count. This + helper is the ONLY place the retired sentinel is still recognized: the + containers' ``__setstate__`` run it so 3.x pickles load in the 4.0 shape + (the ``migrate_renamed_state_keys`` precedent). + + Returns a NEW dict (the input is never mutated); ``None`` passes through. + A row is rewritten only when it is a mapping WITHOUT an ``is_reference`` + key whose count key (``n_groups`` or ``n_obs``) equals 0 and whose effect + (``effect``, else ``att``) is finite and exactly ``0.0`` - the + normalization anchor's signature. A NaN-effect zero-count row is a + genuinely non-estimable horizon (TwoStageDiD all-filtered horizons, + SpilloverDiD rectangular cells) and is copied verbatim. + """ + if effects is None: + return None + out: Dict[Any, Any] = {} + for key, row in effects.items(): + if not isinstance(row, Mapping) or "is_reference" in row: + out[key] = row + continue + count_key = "n_groups" if "n_groups" in row else ("n_obs" if "n_obs" in row else None) + if count_key is None: + out[key] = row + continue + try: + is_zero_count = float(row[count_key]) == 0.0 + except (TypeError, ValueError): + is_zero_count = False + effect = row.get("effect", row.get("att")) + try: + effect_f = float(effect) + except (TypeError, ValueError): + effect_f = float("nan") + if is_zero_count and math.isfinite(effect_f) and effect_f == 0.0: + new_row = dict(row) + new_row.pop(count_key) + new_row["is_reference"] = True + out[key] = new_row + else: + out[key] = row + return out diff --git a/diff_diff/diagnostic_report.py b/diff_diff/diagnostic_report.py index 80bf591bc..c7ac943c0 100644 --- a/diff_diff/diagnostic_report.py +++ b/diff_diff/diagnostic_report.py @@ -3363,9 +3363,10 @@ def _collect_effect_scalars(self) -> List[float]: return self._scalars_from_mapping(ge) # 2. event_study_effects: dict keyed by relative time -> dict with # 'effect'. Filter to **post-treatment** horizons (rel_time >= 0), - # exclude reference markers (``n_groups == 0`` on CS/SA; - # ``n_obs == 0`` on Stacked/TwoStage/Imputation/EfficientDiD), and - # exclude entries with non-finite effect. + # exclude the reference row (``is_reference`` - the explicit marker + # since 4.0, row M-093) and genuine zero-count horizons (a + # non-estimable horizon, e.g. TwoStageDiD all-filtered), and exclude + # entries with non-finite effect. es = getattr(r, "event_study_effects", None) if es is not None: # Anticipation-aware post-treatment cutoff: include horizons @@ -3390,7 +3391,11 @@ def _collect_effect_scalars(self) -> List[float]: if rel < post_cutoff: continue if isinstance(entry, dict): - if entry.get("n_groups") == 0 or entry.get("n_obs") == 0: + if ( + entry.get("is_reference") + or entry.get("n_groups") == 0 + or entry.get("n_obs") == 0 + ): continue eff = _extract_scalar_effect(entry) if eff is None or not np.isfinite(eff): @@ -4087,13 +4092,14 @@ def _surface_to_event_study_dict(surface: Any) -> Dict[Any, Dict[str, float]]: Rows are SKIPPED when ``is_reference`` is set OR the row's count is exactly zero (reference rows carry NaN ``n``, so the conditions are - disjoint). The zero-count skip mirrors the raw route's - ``n_groups == 0`` / ``n_obs == 0`` exclusion in - ``_collect_pre_period_coefs`` — the container deliberately preserves - NaN-effect zero-count horizons as NON-reference rows - (``results_base._from_relative_dict``), and without this skip such a - row would inflate ``n_dropped_undefined`` and flip a valid joint-Wald - PT to "inconclusive". NaN se/p on surviving rows pass through so the + disjoint). The zero-count skip mirrors the raw route's genuine + zero-count exclusion in ``_collect_pre_period_coefs`` — a zero count is + a NON-ESTIMABLE horizon (TwoStageDiD all-filtered, SpilloverDiD + rectangular cells), never a reference marker since 4.0 (row M-093), and + the container preserves such NaN-effect rows as NON-reference rows + (``results_base._from_relative_dict``); without this skip such a row + would inflate ``n_dropped_undefined`` and flip a valid joint-Wald PT to + "inconclusive". NaN se/p on surviving rows pass through so the undefined-inference guards behave identically to the raw route. Keys go through ``_event_time_key``: integer-valued horizons become @@ -4154,11 +4160,13 @@ def _collect_pre_period_coefs( Filtering rules (critical for methodology-safe PT tests): - * Entries marked as reference markers (``n_groups == 0`` on CS / SA or - ``n_obs == 0`` on Stacked / TwoStage / Imputation event-study shape) - are excluded. These are synthetic ``effect=0, se=NaN`` rows injected - for universal-base normalization and are NOT counted in - ``n_dropped_undefined`` — they never represented a real pre-period. + * The reference row (``is_reference`` - the explicit normalization-row + marker every materializing producer writes since 4.0, row M-093; the + zero-count sentinel is retired) is excluded, and so is a genuine + zero-count horizon (``n_groups == 0`` / ``n_obs == 0``: a + non-estimable horizon with no contributing observations). Neither is + counted in ``n_dropped_undefined`` — neither represented a real, + estimated pre-period. * Entries whose ``effect`` or ``se`` is non-finite (NaN / inf) or whose ``se <= 0`` are excluded as undefined inference (``safe_inference`` contract, ``utils.py:175``). These ARE real pre-periods whose @@ -4226,10 +4234,11 @@ def _collect_pre_period_coefs( continue if not isinstance(entry, dict): continue - # Drop universal-base reference markers. These are synthetic, - # not a real pre-period, so they do not count toward + # Drop the reference row (``is_reference``) and genuine + # zero-count (non-estimable) horizons. Neither is a real, + # estimated pre-period, so neither counts toward # ``n_dropped_undefined``. - if entry.get("n_groups") == 0 or entry.get("n_obs") == 0: + if entry.get("is_reference") or entry.get("n_groups") == 0 or entry.get("n_obs") == 0: continue # Wooldridge stores ``att`` rather than ``effect`` in its # event-study payloads; accept either (round-16 CI review). diff --git a/diff_diff/guides/llms-full.txt b/diff_diff/guides/llms-full.txt index 734b97525..f05198626 100644 --- a/diff_diff/guides/llms-full.txt +++ b/diff_diff/guides/llms-full.txt @@ -549,7 +549,7 @@ sp.fit( - `covariates=` raises `NotImplementedError` (planned follow-up). Gardner two-stage requires covariate effects estimated on the untreated-and-unexposed Omega_0 subsample at stage 1; appending raw covariates only at stage 2 silently biases `tau_total` / `delta_j` on panels with time-varying covariates. - `survey_design=` is **supported** (Wave E): design-based variance via Binder (1983) Taylor-series linearization for `vcov_type="hc1"` (HC1) and `cluster=` (CR1), and for `vcov_type="conley"` via a panel-aware stratified-Conley sandwich on per-period PSU totals (`conley_lag_cutoff > 0` adds a within-PSU serial Bartlett HAC and requires an effective PSU — supply `survey_design.psu` or inject one via `cluster=`). Accepts `pweight` with strata / PSU / FPC. Replicate-weight variance (BRR / Fay / JK1 / JKn / SDR) is NOT yet supported (`NotImplementedError`): per Gerber (2026) Appendix A the IF-reweighting shortcut does not apply because `gamma_hat` is weight-sensitive, so correct support requires a per-replicate full re-fit. - `vcov_type="classical"` raises `NotImplementedError` (Wave D restriction). Wave D GMM first-stage correction has not been derived for the homoskedastic meat structure `sigma_hat^2 * (X_10' X_10)`. Use `vcov_type="hc1"`, `vcov_type="conley"`, or pair with `cluster=` for CR1 — all three apply the Wave D GMM correction. -- `event_study=True` SHIPPED (Wave C): emits per-event-time `tau_k` and per-(ring, event-time) `delta_jk` as `att_dynamic: pd.DataFrame` (indexed by event-time `k`) plus MultiIndex `spillover_effects: pd.DataFrame` (levels `(ring_label, event_time)`). TwoStageDiD-compatible `event_study_effects: Dict[int, Dict]` alias also emitted for `plot_event_study` consumption — `_extract_plot_data` prefers the new `reference_period` attribute over the legacy `n_obs==0` heuristic. (DiagnosticReport integration: WIRED — `SpilloverDiDResults` routes to parallel_trends (event-study on these direct-effect dynamics), design_effect, and heterogeneity; bacon excluded since spillover identifies off far-away controls, not TWFE 2x2 comparisons.) (schema: `{k: {"effect", "se", "n_obs", "t_stat", "p_value", "conf_int": (low, high)}}` mirroring `two_stage.py:1355-1389`). Reference period `ref_period = -1 - anticipation` (TwoStageDiD `two_stage.py:486` convention); reference row uses `coef=0.0, se=0.0, n_obs=0, conf_int=(0.0, 0.0)`. Scalar `att` field becomes a sample-share-weighted average of post-treatment `tau_k` (`att = sum_{k>=0} w_k * tau_k` with `w_k = n_treated_at_k / total`) with SE from linear-combination inference `Var(att) = w' V_subset w` on the post-treatment vcov block — no separate fit. **Two-clock K_it:** direct-effect clock is `K_direct = t - effective_first_treat(i)` for ever-treated rows; spillover clock is `K_spill = t - earliest-in-range-cohort-onset(i)` (running min across activated cohorts, NaN pre-trigger). `K_spill >= 0` structurally; negative-k spillover cells are rectangularly emitted with `coef = NaN, n_obs = 0`. **`horizon_max` semantics:** bins event-times outside `[-H, +H]` into endpoint pools (no observations dropped — divergence from TwoStageDiD which filters; intentional, per `feedback_no_silent_failures`). With `horizon_max=None`, auto-detects bin set from observed K. **Validation:** `horizon_max < 0` raises `ValueError`; `ref_period < -horizon_max` (i.e., `anticipation > horizon_max - 1`) raises `ValueError` — silently floor-shifting the reference would change identification. **Reduce-to-aggregate:** under constant-tau DGP with `horizon_max=None`, the share-weighted scalar `att` reproduces Wave B's aggregate bit-identically. **Note:** `horizon_max=0` does NOT reduce to Wave B (binning collapses pre-treatment K values to `k=0`, making `D^0 = D_i` ever-treated indicator rather than `D_it`). Per-event-time SEs include the Wave D Gardner GMM first-stage correction (see next bullet). +- `event_study=True` SHIPPED (Wave C): emits per-event-time `tau_k` and per-(ring, event-time) `delta_jk` as `att_dynamic: pd.DataFrame` (indexed by event-time `k`) plus MultiIndex `spillover_effects: pd.DataFrame` (levels `(ring_label, event_time)`). TwoStageDiD-compatible `event_study_effects: Dict[int, Dict]` alias also emitted for `plot_event_study` consumption — `_extract_plot_data` reads the `reference_period` attribute, else the row's `is_reference` key (4.0, M-093: the reference row carries `is_reference: True` and no `n_obs` in the dict; the frames carry a boolean `is_reference` column with NaN `n_obs` on the reference rows). (DiagnosticReport integration: WIRED — `SpilloverDiDResults` routes to parallel_trends (event-study on these direct-effect dynamics), design_effect, and heterogeneity; bacon excluded since spillover identifies off far-away controls, not TWFE 2x2 comparisons.) (schema: `{k: {"effect", "se", "n_obs", "t_stat", "p_value", "conf_int": (low, high)}}` mirroring `two_stage.py:1355-1389`). Reference period `ref_period = -1 - anticipation` (TwoStageDiD `two_stage.py:486` convention); reference row uses `coef=0.0, se=0.0, conf_int=(0.0, 0.0)` with `is_reference=True` (no `n_obs` key in the dict; `n_obs=NaN` in the frames - 4.0, M-093). Scalar `att` field becomes a sample-share-weighted average of post-treatment `tau_k` (`att = sum_{k>=0} w_k * tau_k` with `w_k = n_treated_at_k / total`) with SE from linear-combination inference `Var(att) = w' V_subset w` on the post-treatment vcov block — no separate fit. **Two-clock K_it:** direct-effect clock is `K_direct = t - effective_first_treat(i)` for ever-treated rows; spillover clock is `K_spill = t - earliest-in-range-cohort-onset(i)` (running min across activated cohorts, NaN pre-trigger). `K_spill >= 0` structurally; negative-k spillover cells are rectangularly emitted with `coef = NaN, n_obs = 0`. **`horizon_max` semantics:** bins event-times outside `[-H, +H]` into endpoint pools (no observations dropped — divergence from TwoStageDiD which filters; intentional, per `feedback_no_silent_failures`). With `horizon_max=None`, auto-detects bin set from observed K. **Validation:** `horizon_max < 0` raises `ValueError`; `ref_period < -horizon_max` (i.e., `anticipation > horizon_max - 1`) raises `ValueError` — silently floor-shifting the reference would change identification. **Reduce-to-aggregate:** under constant-tau DGP with `horizon_max=None`, the share-weighted scalar `att` reproduces Wave B's aggregate bit-identically. **Note:** `horizon_max=0` does NOT reduce to Wave B (binning collapses pre-treatment K values to `k=0`, making `D^0 = D_i` ever-treated indicator rather than `D_it`). Per-event-time SEs include the Wave D Gardner GMM first-stage correction (see next bullet). - Stage-2 variance applies the Gardner GMM first-stage uncertainty correction across HC1 / Conley / cluster (Wave D, SHIPPED). The IF outer-product formula `psi_i = gamma_hat' X_{10,i} eps_{10,i} - X_{2,i} eps_{2,i}` is used unconditionally; kernel `K` is path-dependent (identity for HC1, block-indicator for cluster, spatial kernel for Conley). Documented synthesis of Butts (2021) §3.1 + Gardner (2022) §4 + Conley (1999); no reference software combines all three. Point estimates unchanged from Wave B/C; SE values shift upward by 1-few percent. - Only nearest-treated rings supported; `ring_method="count"` (count of treated neighbors in ring) not yet exposed @@ -2028,7 +2028,7 @@ ImputationDiDResults, TwoStageDiDResults, StackedDiDResults, and EfficientDiDRes | `n_treated_units` | `int` | Number of treated units | | `n_control_units` | `int` | Number of control units | -Each event study effect dict contains: `effect`, `se`, `t_stat`, `p_value`, `conf_int`, `n_obs` (or `n_groups`). +Each event study effect dict contains: `effect`, `se`, `t_stat`, `p_value`, `conf_int` and, on estimated rows, `n_obs` (or `n_groups`); on the producers that materialize one, the reference row carries `is_reference: True` and no count (4.0, M-093). **Variance metadata** (`ImputationDiDResults` and `EfficientDiDResults` carry these; other staggered Results may surface a subset): diff --git a/diff_diff/honest_did.py b/diff_diff/honest_did.py index 70d99e12a..cd7e7c948 100644 --- a/diff_diff/honest_did.py +++ b/diff_diff/honest_did.py @@ -955,7 +955,7 @@ def _extract_container_params( stacklevel=4, ) - # Row filter: is_reference supersedes the fit-time n_groups==0 sniff; + # Row filter: is_reference (the same marker the fit-time dict carries since 4.0); # non-finite-SE rows (genuinely non-estimable horizons) drop exactly as # the fit-time branch drops them. se == 0 rows drop too: safe_inference # treats a zero SE as undefined inference (NaN t/p/CI), and admitting @@ -1003,7 +1003,7 @@ def _extract_container_params( ref_rows = surface.event_time[surface.is_reference].tolist() if len(ref_rows) > 1: # No route recommendation here: the native-results path picks the - # first n_groups==0 marker and then fails its own consecutive-grid + # first is_reference marker and then fails its own consecutive-grid # validation on the same gapped layout, so pointing users there # would be a dead end. raise ValueError( @@ -1256,27 +1256,28 @@ def _extract_event_study_params( ) # Extract event study effects by relative time - # Filter out normalization constraints (n_groups=0), non-finite + # Filter out normalization constraints (the ``is_reference`` row; + # 4.0, row M-093 - the n_groups==0 sentinel is retired), non-finite # SEs, and zero SEs (safe_inference treats se <= 0 as undefined # inference - admitting such a row would launder NaN source # inference into finite sensitivity bounds; mirrors pretrends). event_effects = { t: data for t, data in results.event_study_effects.items() - if data.get("n_groups", 1) > 0 + if not data.get("is_reference", False) and np.isfinite(data.get("se", np.nan)) and float(data.get("se", 0.0)) > 0 } rel_times = sorted(event_effects.keys()) # Infer the omitted reference period from the normalization - # marker injected by _aggregate_event_study for universal base. - # The reference has the exact signature: effect=0.0, se=NaN, n_groups=0. - # Other empty bins may also have n_groups=0 but with NaN effect. + # marker _aggregate_event_study writes for the universal base: + # the ``is_reference`` row with effect=0.0 and se=NaN. Empty bins + # with NaN effect carry no marker and are never a reference. ref_period = None for t, data in results.event_study_effects.items(): if ( - data.get("n_groups", 1) == 0 + data.get("is_reference", False) and data.get("effect", None) == 0.0 and not np.isfinite(data.get("se", 0.0)) ): @@ -3363,7 +3364,8 @@ def _estimate_max_pre_violation(self, results: Any, pre_periods: List) -> float: pre_effects = [ abs(results.event_study_effects[t]["effect"]) for t in results.event_study_effects - if t in pre_set and results.event_study_effects[t].get("n_groups", 1) > 0 + if t in pre_set + and not results.event_study_effects[t].get("is_reference", False) ] if pre_effects: return max(pre_effects) diff --git a/diff_diff/imputation.py b/diff_diff/imputation.py index 5a96f0a6c..f20f0a24e 100644 --- a/diff_diff/imputation.py +++ b/diff_diff/imputation.py @@ -961,6 +961,7 @@ def _precompute_psi( for h in event_study_effects: if ( h in bootstrap_results.event_study_ses + and not event_study_effects[h].get("is_reference", False) and event_study_effects[h].get("n_obs", 1) > 0 ): event_study_effects[h]["se"] = bootstrap_results.event_study_ses[h] diff --git a/diff_diff/imputation_aggregation.py b/diff_diff/imputation_aggregation.py index 379276770..f9a1eff4d 100644 --- a/diff_diff/imputation_aggregation.py +++ b/diff_diff/imputation_aggregation.py @@ -1044,14 +1044,16 @@ def _aggregate_event_study( event_study_effects: Dict[int, Dict[str, Any]] = {} - # Add reference period marker + # Add the reference period row: the normalization anchor, marked + # EXPLICITLY (4.0, ledger row M-093 item 8a) - no count key; the 3.x + # ``n_obs == 0`` sentinel is retired. event_study_effects[ref_period] = { "effect": 0.0, "se": 0.0, "t_stat": np.nan, "p_value": np.nan, "conf_int": (0.0, 0.0), - "n_obs": 0, + "is_reference": True, } # Pre-period coefficients via BJS Test 1 lead regression @@ -1749,7 +1751,9 @@ def _replicate_override_aggregates( _sorted_rel_times = sorted( e for e in _es_effects.keys() - if np.isfinite(_es_effects[e]["effect"]) and _es_effects[e].get("n_obs", 1) > 0 + if np.isfinite(_es_effects[e]["effect"]) + and not _es_effects[e].get("is_reference", False) + and _es_effects[e].get("n_obs", 1) > 0 ) _sorted_groups = sorted( g for g in _grp_effects.keys() if np.isfinite(_grp_effects[g]["effect"]) diff --git a/diff_diff/imputation_bootstrap.py b/diff_diff/imputation_bootstrap.py index a9d1540cc..c9436546d 100644 --- a/diff_diff/imputation_bootstrap.py +++ b/diff_diff/imputation_bootstrap.py @@ -164,8 +164,10 @@ def _precompute_bootstrap_psi( ref_period = -1 - self.anticipation for h in event_study_effects: + if event_study_effects[h].get("is_reference", False): + continue # the normalization anchor carries no draws if event_study_effects[h].get("n_obs", 0) == 0: - continue + continue # genuine zero-observation horizon if h == ref_period: continue if not np.isfinite(event_study_effects[h].get("effect", np.nan)): diff --git a/diff_diff/imputation_results.py b/diff_diff/imputation_results.py index 1f063fe8d..3c3cc0228 100644 --- a/diff_diff/imputation_results.py +++ b/diff_diff/imputation_results.py @@ -15,6 +15,7 @@ from diff_diff._deprecation import ( LEGACY_OVERALL_QUINTET, deprecated_field_property, + migrate_event_study_reference_sentinel, migrate_renamed_state_keys, ) from diff_diff.aggregation import ( @@ -147,7 +148,9 @@ class ImputationDiDResults(BaseResults, AggregationMixin): Confidence interval for overall ATT. event_study_effects : dict, optional Dictionary mapping relative time h to effect dict with keys: - 'effect', 'se', 't_stat', 'p_value', 'conf_int', 'n_obs'. + 'effect', 'se', 't_stat', 'p_value', 'conf_int' and, on estimated + rows, 'n_obs'; the reference row carries ``'is_reference': True`` + and no count (4.0, ledger row M-093). group_effects : dict, optional Dictionary mapping cohort g to effect dict. groups : list @@ -264,7 +267,15 @@ def __setstate__(self, state: Dict[str, Any]) -> None: under their canonical names and ``overall_*`` never reappear in ``__dict__``. """ - self.__dict__.update(migrate_renamed_state_keys(state, LEGACY_OVERALL_QUINTET)) + state = migrate_renamed_state_keys(state, LEGACY_OVERALL_QUINTET) + if "event_study_effects" in state: + # 3.x surfaces marked the reference row with the n_obs == 0 + # sentinel (retired at 4.0, row M-093). + state = dict(state) + state["event_study_effects"] = migrate_event_study_reference_sentinel( + state["event_study_effects"] + ) + self.__dict__.update(state) # --- Post-fit aggregation (M-021/M-118) ------------------------------ @@ -417,7 +428,7 @@ def _aggregate_compute( ) # Carrier + shared builder: ImputationDiD is a _from_relative_dict # producer, so the recomputed dict rides the same route as the - # fit-time surface (zero-count-sentinel reference marking, + # fit-time surface (``is_reference``-keyed reference marking, # n_kind="obs"). The carrier's metadata is a copy-on-use of the # KIT's fit-final metadata copy (never the mutable public field); # on a replicate replay its df_survey is the REPLAYED level-matched @@ -717,8 +728,8 @@ def summary(self, alpha: Optional[float] = None) -> str: for h in sorted(self.event_study_effects.keys()): eff = self.event_study_effects[h] - if eff.get("n_obs", 1) == 0: - # Reference period marker + if eff.get("is_reference", False): + # Reference period row (explicit marker since 4.0, M-093) lines.append( f"[ref: {h}]" f"{'0.0000':>17} {'---':>12} {'---':>10} {'---':>10} {'':>6}" ) diff --git a/diff_diff/pretrends.py b/diff_diff/pretrends.py index cd136123d..eed76bde2 100644 --- a/diff_diff/pretrends.py +++ b/diff_diff/pretrends.py @@ -1188,8 +1188,8 @@ def _extract_pre_period_params( # ``anticipation=k``, true pre-periods are ``t < -k``; # ``t ∈ [-k, -1]`` is the anticipation window and must # not be used for pre-trends power. Filter out - # normalization constraints (n_groups=0) and non-finite - # SEs as well. + # normalization constraints (the ``is_reference`` row; 4.0, + # row M-093) and non-finite SEs as well. _ant = getattr(results, "anticipation", 0) or 0 try: _ant = int(_ant) @@ -1204,7 +1204,7 @@ def _extract_pre_period_params( t: data for t, data in results.event_study_effects.items() if t < _pre_cutoff - and data.get("n_groups", 1) > 0 + and not data.get("is_reference", False) and np.isfinite(data.get("se", np.nan)) and float(data.get("se", 0.0)) > 0 } @@ -1283,7 +1283,7 @@ def _extract_pre_period_params( t: data for t, data in results.event_study_effects.items() if t < _pre_cutoff - and data.get("n_groups", 1) > 0 + and not data.get("is_reference", False) and np.isfinite(data.get("se", np.nan)) and float(data.get("se", 0.0)) > 0 } diff --git a/diff_diff/results.py b/diff_diff/results.py index 970a514ba..cbc64c553 100644 --- a/diff_diff/results.py +++ b/diff_diff/results.py @@ -10,6 +10,7 @@ import numpy as np import pandas as pd +from diff_diff._deprecation import migrate_event_study_reference_sentinel from diff_diff.results_base import ( _SUMMARY_ALPHA_MESSAGE, BaseResults, @@ -397,9 +398,11 @@ class SpilloverDiDResults(DiDResults): Total effect on the treated, ``tau_total`` (inherited from DiDResults). spillover_effects : pd.DataFrame, optional Per-ring spillover-on-control estimates. Columns: ``["coef", "se", - "t_stat", "p_value", "ci_low", "ci_high"]``. Index: ring label - like ``"[0, 50)"``; for event-study output the index is a - ``MultiIndex`` over ``(ring_label, event_time)``. + "t_stat", "p_value", "ci_low", "ci_high"]`` plus, in event-study + output, ``"n_obs"`` and the boolean ``"is_reference"`` (True on the + per-ring reference-period rows, whose ``n_obs`` is NaN; 4.0, ledger + row M-093). Index: ring label like ``"[0, 50)"``; for event-study + output the index is a ``MultiIndex`` over ``(ring_label, event_time)``. ring_breakpoints : list of float Sorted distance breakpoints used to construct the rings. d_bar : float @@ -450,8 +453,10 @@ class SpilloverDiDResults(DiDResults): # Wave C event-study fields (None when event_study=False): att_dynamic: Optional[pd.DataFrame] = field(default=None) # Per-event-time direct effects DataFrame indexed by integer k. - # Columns: ["coef", "se", "t_stat", "p_value", "ci_low", "ci_high", "n_obs"]. - # Includes the reference period row with coef=0.0, se=0.0, n_obs=0. + # Columns: ["coef", "se", "t_stat", "p_value", "ci_low", "ci_high", "n_obs", + # "is_reference"]. Includes the reference period row with coef=0.0, + # se=0.0, is_reference=True and n_obs=NaN (4.0, row M-093 item 8a: the + # 3.x n_obs=0 sentinel is retired; a zero n_obs is a genuine empty cell). event_study_effects: Optional[Dict[int, Dict[str, Any]]] = field(default=None) # TwoStageDiD-compatible alias for ``att_dynamic`` consumable by # ``plot_event_study`` (wired in Wave C via the ``reference_period`` @@ -461,9 +466,10 @@ class SpilloverDiDResults(DiDResults): # applicable checks {parallel_trends (event-study on these direct-effect # dynamics), design_effect, heterogeneity}; bacon is excluded (spillover # identifies off far-away units, not TWFE 2x2 comparisons). - # Schema mirrors ``two_stage.py:1355-1389``: + # Schema mirrors TwoStageDiD's ``_stage2_event_study``: # {k: {"effect", "se", "n_obs", "t_stat", "p_value", "conf_int": (low, high)}} - # Reference row uses ``conf_int = (0.0, 0.0)`` (TwoStageDiD parity). + # Reference row uses ``conf_int = (0.0, 0.0)`` (TwoStageDiD parity) and + # carries ``"is_reference": True`` with no ``n_obs`` key (M-093, 8a). horizon_max: Optional[int] = field(default=None) reference_period: Optional[int] = field(default=None) # Wave E.1 survey-design fields (None when survey_design=None on fit). @@ -476,6 +482,41 @@ class SpilloverDiDResults(DiDResults): n_psu: Optional[int] = field(default=None) n_strata: Optional[int] = field(default=None) + def __setstate__(self, state: Dict[str, Any]) -> None: + """Migrate 3.x pickles to the 4.0 reference-row marking (row M-093). + + Through 3.x the event-study reference row was marked by the + ``n_obs == 0`` sentinel in ``event_study_effects`` and in the + ``att_dynamic`` / ``spillover_effects`` frames. Since 4.0 the dict row + carries ``is_reference: True`` (no count key) and the event-study + frames carry a boolean ``is_reference`` column with NaN ``n_obs`` on + the reference rows. Aggregate-mode fits (``reference_period`` None) + have no reference row and pass through untouched; a fresh 4.0 pickle + round-trips as a no-op. + """ + state = dict(state) + if "event_study_effects" in state: + state["event_study_effects"] = migrate_event_study_reference_sentinel( + state["event_study_effects"] + ) + ref = state.get("reference_period") + if ref is not None: + for name in ("att_dynamic", "spillover_effects"): + frame = state.get(name) + if not isinstance(frame, pd.DataFrame) or "is_reference" in frame.columns: + continue + frame = frame.copy() + if isinstance(frame.index, pd.MultiIndex): + is_ref = frame.index.get_level_values("k") == ref + else: + is_ref = frame.index == ref + frame["is_reference"] = np.asarray(is_ref, dtype=bool) + if "n_obs" in frame.columns and bool(np.any(is_ref)): + frame["n_obs"] = frame["n_obs"].astype(float) + frame.loc[is_ref, "n_obs"] = np.nan + state[name] = frame + self.__dict__.update(state) + def summary(self, alpha: Optional[float] = None) -> str: """Extended summary with ATT row, per-event-time direct block, and per-(ring, event-time) spillover block. diff --git a/diff_diff/results_base.py b/diff_diff/results_base.py index fcc92a4ee..74b13f398 100644 --- a/diff_diff/results_base.py +++ b/diff_diff/results_base.py @@ -510,8 +510,8 @@ def __post_init__(self) -> None: # Multiple reference rows are legal: CallawaySantAnna with # base_period="universal" on a gapped period grid materializes each - # cohort's positional base as its own reference-only horizon (all - # n_groups==0), so a valid surface can carry several. `is_reference` + # cohort's positional base as its own reference-only horizon (each + # marked ``is_reference``), so a valid surface can carry several. `is_reference` # is the general marker; `reference_period` is a convenience scalar # DERIVED from it - authoritative only when there is exactly one # reference, None otherwise - so a caller-supplied value can never @@ -1081,26 +1081,30 @@ def _from_relative_dict(results: Any) -> EventStudyResults: (``att`` inner key normalized here), and TripleDifference's staggered design (through the carrier its ``aggregate('event_study')`` builds). - Reference resolution is PRODUCER-AWARE, because a zero count is not a - universal reference marker: - - 1. If the producer names a ``reference_period`` that is PRESENT in the - effects dict (SpilloverDiD materializes a zero row), that row is THE - reference. Other zero-count rows - SpilloverDiD emits genuinely - non-estimable rectangular horizons with ``n_obs == 0`` and NaN - inference - are preserved as NON-reference NaN rows. - 2. If the producer names a ``reference_period`` that is ABSENT (an - OMITTED baseline, e.g. SunAbraham's ``e = -1 - anticipation``), the - anchor row is synthesized ONLY when the estimator reports it was - genuinely observed (``reference_observed``) - never invented on a - gapped grid where that period was never in the data. - 3. Otherwise the in-band zero-count sentinel marks the reference: a - zero-count row is a reference iff its effect is exactly 0 (the - normalization anchor). A NaN-effect zero-count row is a genuinely - non-estimable horizon (TwoStageDiD emits effect=NaN, n_obs=0 for a - horizon whose observations are all filtered) - preserved, never - normalized. CallawaySantAnna universal base can mark several such - rows; ContinuousDiD / Wooldridge carry no count key and yield none. + Reference resolution reads EXPLICIT markers only (4.0, ledger row M-093 + item 8a) - a count is never interpreted as a marker: + + 1. A row carrying ``"is_reference": True`` is a reference row. This is + how every producer that materializes its normalization row marks it + (StackedDiD, ImputationDiD, TwoStageDiD, SpilloverDiD, the + CallawaySantAnna / DMLDiD universal-base carrier - which can mark + several rows on a gapped grid). Such rows carry no count key. + 2. If the producer names a ``reference_period`` that is PRESENT in the + effects dict (SpilloverDiD; EfficientDiD's PT-Post anchor, an + ESTIMATED cell), that row is marked too. If it is ABSENT (an OMITTED + baseline, e.g. SunAbraham's ``e = -1 - anticipation``), the anchor row + is synthesized ONLY when the estimator reports it was genuinely + observed (``reference_observed``) - never invented on a gapped grid + where that period was never in the data. + + A zero count is a GENUINE count: TwoStageDiD's all-filtered horizons and + SpilloverDiD's rectangular cells carry ``n_obs == 0`` with NaN inference + and stay NON-reference NaN rows. The 3.x zero-count reference sentinel is + rewritten to the explicit key at pickle load + (``_deprecation.migrate_event_study_reference_sentinel``) and is never + interpreted here. A marked row whose effect is finite and nonzero is + malformed and raises (``__post_init__`` would otherwise silently rewrite + it to 0). """ effects = getattr(results, "event_study_effects", None) if effects is None: @@ -1138,7 +1142,7 @@ def _from_relative_dict(results: Any) -> EventStudyResults: ci_lo = np.empty(n_rows) ci_hi = np.empty(n_rows) n = np.full(n_rows, np.nan) - zero_count = np.zeros(n_rows, dtype=bool) + keyed_ref = np.zeros(n_rows, dtype=bool) cband_pairs: Dict[int, Tuple[float, float]] = {} n_kind: Optional[str] = None @@ -1156,11 +1160,10 @@ def _from_relative_dict(results: Any) -> EventStudyResults: if "n_groups" in row: n_kind = "groups" n[i] = float(row["n_groups"]) - zero_count[i] = row["n_groups"] == 0 elif "n_obs" in row: n_kind = "obs" n[i] = float(row["n_obs"]) - zero_count[i] = row["n_obs"] == 0 + keyed_ref[i] = bool(row.get("is_reference", False)) cband = row.get("cband_conf_int") if cband is not None: @@ -1170,29 +1173,28 @@ def _from_relative_dict(results: Any) -> EventStudyResults: ref_present = explicit_ref is not None and ( synthesized_ref is not None or explicit_ref in set(event_time.tolist()) ) + # Explicit ``is_reference`` keys (every materialized reference row since + # 4.0) UNIONED with the named reference: materialized (SpilloverDiD, + # EfficientDiD PT-Post anchor) or synthesized (SunAbraham observed). A + # named reference that is absent and NOT synthesized (SunAbraham on a + # gapped grid where the anchor was never observed) marks nothing. Counts + # are never consulted: a zero count is a genuine count. + is_ref = keyed_ref.copy() if ref_present: - # Named reference that is materialized (Spillover) or synthesized - # (SunAbraham observed): mark exactly that row. - is_ref = event_time == explicit_ref - elif explicit_ref is not None: - # Named reference that is absent and NOT synthesized (SunAbraham on a - # gapped grid where the anchor was never observed): no reference row. - is_ref = np.zeros(n_rows, dtype=bool) - else: - # In-band zero-count sentinel. A zero-count row is the REFERENCE only - # when its effect is exactly 0 (the normalization anchor). A NaN-effect - # zero-count row is a genuinely non-estimable horizon - preserved. A - # zero-count row with a finite NONZERO effect is malformed - fail - # loudly rather than silently rewrite it. - finite = np.isfinite(att) - bad = zero_count & finite & (att != 0.0) - if bad.any(): - raise ValueError( - f"{type(results).__name__} event-study surface has a zero-count " - f"row with a finite nonzero effect at {event_time[bad].tolist()}; " - "a reference/normalization row must have effect 0.0." - ) - is_ref = zero_count & finite + is_ref |= event_time == explicit_ref + # A KEYED row with a finite NONZERO effect is malformed - fail loudly + # rather than let __post_init__ silently rewrite it to 0. The NAMED + # reference is exempt: EfficientDiD's PT-Post anchor is an estimated + # bucket that can pool a nonzero fractional pre-period estimate with the + # mechanical zero (REGISTRY fractional-period Note), and the container + # normalizes it by design. + bad = keyed_ref & np.isfinite(att) & (att != 0.0) + if bad.any(): + raise ValueError( + f"{type(results).__name__} event-study surface has an is_reference " + f"row with a finite nonzero effect at {event_time[bad].tolist()}; " + "a reference/normalization row must have effect 0.0." + ) cband_lo, cband_hi = _materialize_cband(cband_pairs, n_rows) diff --git a/diff_diff/spillover.py b/diff_diff/spillover.py index b40913e90..b89a4bd24 100644 --- a/diff_diff/spillover.py +++ b/diff_diff/spillover.py @@ -849,7 +849,8 @@ def _extract_event_study_results( Builds three output surfaces from a single stage-2 fit: - ``att_dynamic`` : per-event-time direct-effect DataFrame indexed by ``k``. - Includes the reference period row with ``coef=0.0, se=0.0, n_obs=0``. + Includes the reference period row (``coef=0.0, se=0.0``, marked by the + boolean ``is_reference`` column with ``n_obs=NaN``; 4.0, row M-093). Rectangular emission across the full direct-effect event-time grid. - ``spillover_effects`` : MultiIndex ``(ring_label, event_time)`` DataFrame with the same columns. Rectangular over the full spillover grid. @@ -891,6 +892,10 @@ def _extract_event_study_results( direct_rows: List[Dict[str, Any]] = [] for k in all_direct_ks: if k == ref_period: + # Reference row: the normalization anchor, marked EXPLICITLY + # (4.0, ledger row M-093 item 8a); ``n_obs`` is NaN there - the + # 3.x ``n_obs == 0`` sentinel is retired, so a zero ``n_obs`` + # is always a genuine empty cell. direct_rows.append( { "k": k, @@ -900,12 +905,13 @@ def _extract_event_study_results( "p_value": float("nan"), "ci_low": 0.0, "ci_high": 0.0, - "n_obs": 0, + "n_obs": float("nan"), + "is_reference": True, } ) elif ("direct", None, k) in per_coef: r = per_coef[("direct", None, k)] - direct_rows.append({"k": k, **r}) + direct_rows.append({"k": k, **r, "is_reference": False}) else: direct_rows.append( { @@ -917,6 +923,7 @@ def _extract_event_study_results( "ci_low": float("nan"), "ci_high": float("nan"), "n_obs": 0, + "is_reference": False, } ) att_dynamic_df = pd.DataFrame(direct_rows).set_index("k").sort_index() if direct_rows else None @@ -927,8 +934,8 @@ def _extract_event_study_results( # period row per ring. The pre-filter in _build_event_study_design drops # `ref_period` from the fitted column set, but the rectangular schema # for spillover must still emit (ring, ref_period) with `coef=0.0, - # se=0.0, n_obs=0` for symmetry with the direct-effect series (which - # emits its reference row at k=ref_period). Without this, consumers + # se=0.0, is_reference=True, n_obs=NaN` for symmetry with the + # direct-effect series (which emits its reference row at k=ref_period). Without this, consumers # iterating `[-H, ..., +H]` would hit a missing (ring, ref_period) # slice — the registry promises rectangular emission over the full # event-time grid. @@ -937,7 +944,8 @@ def _extract_event_study_results( for ring_lab in ring_labels: for k in all_spillover_ks: if k == ref_period: - # Reference-period spillover row: 0-anchored (mirrors direct). + # Reference-period spillover row: 0-anchored (mirrors direct); + # marked by ``is_reference`` with NaN ``n_obs`` (M-093, 8a). spillover_rows.append( { "ring": ring_lab, @@ -948,14 +956,15 @@ def _extract_event_study_results( "p_value": float("nan"), "ci_low": 0.0, "ci_high": 0.0, - "n_obs": 0, + "n_obs": float("nan"), + "is_reference": True, } ) continue key = ("spillover", ring_lab, k) if key in per_coef: r = per_coef[key] - spillover_rows.append({"ring": ring_lab, "k": k, **r}) + spillover_rows.append({"ring": ring_lab, "k": k, **r, "is_reference": False}) else: spillover_rows.append( { @@ -968,6 +977,7 @@ def _extract_event_study_results( "ci_low": float("nan"), "ci_high": float("nan"), "n_obs": 0, + "is_reference": False, } ) spillover_df = ( @@ -980,13 +990,14 @@ def _extract_event_study_results( event_study_effects: Dict[int, Dict[str, Any]] = {} for k in all_direct_ks: if k == ref_period: + # Reference row: ``is_reference`` marks it, no count key (M-093, 8a). event_study_effects[k] = { "effect": 0.0, "se": 0.0, - "n_obs": 0, "t_stat": float("nan"), "p_value": float("nan"), "conf_int": (0.0, 0.0), + "is_reference": True, } elif ("direct", None, k) in per_coef: r = per_coef[("direct", None, k)] diff --git a/diff_diff/stacked_did.py b/diff_diff/stacked_did.py index dd82dd120..61db46ac3 100644 --- a/diff_diff/stacked_did.py +++ b/diff_diff/stacked_did.py @@ -902,14 +902,16 @@ def _refit_stacked(w_r): es_vcov: Optional[np.ndarray] = None es_vcov_index: Optional[List[int]] = None es_df_used: Dict[int, float] = {} - # Reference period (e = -1 - anticipation) + # Reference period (e = -1 - anticipation): the normalization anchor, + # marked EXPLICITLY (4.0, ledger row M-093 item 8a) - no count key; + # the 3.x ``n_obs == 0`` sentinel is retired. event_study_effects[ref_period] = { "effect": 0.0, "se": 0.0, "t_stat": np.nan, "p_value": np.nan, "conf_int": (np.nan, np.nan), - "n_obs": 0, + "is_reference": True, } for h in event_times: idx = interaction_indices[h] diff --git a/diff_diff/stacked_did_results.py b/diff_diff/stacked_did_results.py index 90bb2721d..890521be8 100644 --- a/diff_diff/stacked_did_results.py +++ b/diff_diff/stacked_did_results.py @@ -14,6 +14,7 @@ from diff_diff._deprecation import ( LEGACY_OVERALL_QUINTET, deprecated_field_property, + migrate_event_study_reference_sentinel, migrate_renamed_state_keys, ) from diff_diff.aggregation import AggregationMixin, AggregationResult @@ -50,7 +51,9 @@ class StackedDiDResults(BaseResults, AggregationMixin): Confidence interval for overall ATT. event_study_effects : dict, optional Dictionary mapping event time h to effect dict with keys: - 'effect', 'se', 't_stat', 'p_value', 'conf_int', 'n_obs'. + 'effect', 'se', 't_stat', 'p_value', 'conf_int' and, on estimated + rows, 'n_obs'; the reference row carries ``'is_reference': True`` + and no count (4.0, ledger row M-093). Always populated on 3.9+ fits (the pooled regression always includes the event-time interactions and the surface is always extracted - row M-024); None only on pre-3.9 pickles. View it as @@ -186,7 +189,15 @@ def __setstate__(self, state: Dict[str, Any]) -> None: if "clean_control" in state and "control_group" not in state: state = dict(state) state["control_group"] = state.pop("clean_control") - self.__dict__.update(migrate_renamed_state_keys(state, LEGACY_OVERALL_QUINTET)) + state = migrate_renamed_state_keys(state, LEGACY_OVERALL_QUINTET) + if "event_study_effects" in state: + # 3.x surfaces marked the reference row with the n_obs == 0 + # sentinel (retired at 4.0, row M-093). + state = dict(state) + state["event_study_effects"] = migrate_event_study_reference_sentinel( + state["event_study_effects"] + ) + self.__dict__.update(state) # ------------------------------------------------------------------ # Container-consumer provenance (rows M-024 / M-093). Class-level @@ -420,8 +431,8 @@ def summary(self, alpha: Optional[float] = None) -> str: for h in sorted(self.event_study_effects.keys()): eff = self.event_study_effects[h] - if eff.get("n_obs", 1) == 0: - # Reference period marker + if eff.get("is_reference", False): + # Reference period row (explicit marker since 4.0, M-093) lines.append( f"[ref: {h}]" f"{'0.0000':>17} {'---':>12} {'---':>10} {'---':>10} {'':>6}" ) diff --git a/diff_diff/staggered_aggregation.py b/diff_diff/staggered_aggregation.py index cfa47b796..09db17e93 100644 --- a/diff_diff/staggered_aggregation.py +++ b/diff_diff/staggered_aggregation.py @@ -1000,6 +1000,10 @@ def _aggregate_event_study( agg_effects_list = [] agg_ses_list = [] agg_n_groups = [] + # Reference-only horizons (universal base): marked EXPLICITLY on the + # row dict (``is_reference``, no ``n_groups`` key) since 4.0 (ledger + # row M-093 item 8a) - the ``n_groups == 0`` sentinel is retired. + agg_is_reference: List[bool] = [] agg_effective_dfs = [] # Per-horizon effective df (replicate designs) agg_periods = [] # Relative times that yielded an estimable aggregate row _psi_vectors = [] # Per-event-time combined IF vectors for VCV @@ -1031,7 +1035,10 @@ def _aggregate_event_study( if reference_cells and all(gt in reference_cells for gt in gt_pairs): agg_effects_list.append(0.0) agg_ses_list.append(np.nan) + # Positional bookkeeping only (agg_n_groups is indexed beside + # agg_periods); the row dict omits the count on reference rows. agg_n_groups.append(0) + agg_is_reference.append(True) agg_effective_dfs.append(None) agg_periods.append(e) # No influence-function column for a reference-only horizon (it @@ -1073,6 +1080,7 @@ def _aggregate_event_study( # NaN cells and zero references don't inflate n_groups — matches the # all-NaN early-return which already reports 0. agg_n_groups.append(sum(1 for gt in gt_pairs if gt not in reference_cells)) + agg_is_reference.append(False) agg_effective_dfs.append(eff_df) agg_periods.append(e) _psi_vectors.append(psi_e) @@ -1121,14 +1129,19 @@ def _aggregate_event_study( event_study_effects: Dict[Any, Dict[str, Any]] = {} for idx, e in enumerate(agg_periods): - event_study_effects[e] = { + row: Dict[str, Any] = { "effect": agg_effects_list[idx], "se": agg_ses_list[idx], "t_stat": float(t_stats[idx]), "p_value": float(p_values[idx]), "conf_int": (float(ci_lowers[idx]), float(ci_uppers[idx])), - "n_groups": agg_n_groups[idx], } + if agg_is_reference[idx]: + # The normalization anchor: explicit marker, no count key. + row["is_reference"] = True + else: + row["n_groups"] = agg_n_groups[idx] + event_study_effects[e] = row # (Universal-mode zero reference rows are now materialized per cohort at # their positional base event time e = base - g during the aggregation diff --git a/diff_diff/staggered_results.py b/diff_diff/staggered_results.py index a929945b6..6873163b0 100644 --- a/diff_diff/staggered_results.py +++ b/diff_diff/staggered_results.py @@ -14,6 +14,7 @@ from diff_diff._deprecation import ( LEGACY_OVERALL_QUINTET, deprecated_field_property, + migrate_event_study_reference_sentinel, migrate_renamed_state_keys, ) from diff_diff.aggregation import ( @@ -340,7 +341,15 @@ def __setstate__(self, state: Dict[str, Any]) -> None: under their canonical names and ``overall_*`` never reappear in ``__dict__``. """ - self.__dict__.update(migrate_renamed_state_keys(state, LEGACY_OVERALL_QUINTET)) + state = migrate_renamed_state_keys(state, LEGACY_OVERALL_QUINTET) + if "event_study_effects" in state: + # 3.x fit-time surfaces marked the universal-base reference rows + # with the n_groups == 0 sentinel (retired at 4.0, row M-093). + state = dict(state) + state["event_study_effects"] = migrate_event_study_reference_sentinel( + state["event_study_effects"] + ) + self.__dict__.update(state) def __repr__(self) -> str: """Concise string representation.""" diff --git a/diff_diff/two_stage.py b/diff_diff/two_stage.py index a38b021a4..dff999f3b 100644 --- a/diff_diff/two_stage.py +++ b/diff_diff/two_stage.py @@ -2006,6 +2006,7 @@ def _run_stage( for h in event_study_effects: if ( h in bootstrap_results.event_study_ses + and not event_study_effects[h].get("is_reference", False) and event_study_effects[h].get("n_obs", 1) > 0 ): event_study_effects[h]["se"] = bootstrap_results.event_study_ses[h] diff --git a/diff_diff/two_stage_aggregation.py b/diff_diff/two_stage_aggregation.py index 603fa48e6..d0a31a322 100644 --- a/diff_diff/two_stage_aggregation.py +++ b/diff_diff/two_stage_aggregation.py @@ -475,7 +475,7 @@ def _stage2_event_study( "t_stat": np.nan, "p_value": np.nan, "conf_int": (0.0, 0.0), - "n_obs": 0, + "is_reference": True, } }, None, @@ -541,7 +541,7 @@ def _stage2_event_study( "t_stat": np.nan, "p_value": np.nan, "conf_int": (0.0, 0.0), - "n_obs": 0, + "is_reference": True, } } ref_only.update(prop5_effects) @@ -608,7 +608,7 @@ def _stage2_event_study( "t_stat": np.nan, "p_value": np.nan, "conf_int": (0.0, 0.0), - "n_obs": 0, + "is_reference": True, } for h in est_horizons: diff --git a/diff_diff/two_stage_bootstrap.py b/diff_diff/two_stage_bootstrap.py index 055f71ac5..ab588e638 100644 --- a/diff_diff/two_stage_bootstrap.py +++ b/diff_diff/two_stage_bootstrap.py @@ -561,8 +561,10 @@ def _run_bootstrap( event_study_cis = {} event_study_p_values = {} for h in original_event_study: + if original_event_study[h].get("is_reference", False): + continue # the normalization anchor carries no draws if original_event_study[h].get("n_obs", 0) == 0: - continue + continue # genuine zero-observation horizon if np.isnan(original_event_study[h]["effect"]): continue # Skip Prop 5 and other NaN-effect horizons if h not in horizon_to_col: diff --git a/diff_diff/two_stage_results.py b/diff_diff/two_stage_results.py index 5befa3f12..febbe2aae 100644 --- a/diff_diff/two_stage_results.py +++ b/diff_diff/two_stage_results.py @@ -15,6 +15,7 @@ from diff_diff._deprecation import ( LEGACY_OVERALL_QUINTET, deprecated_field_property, + migrate_event_study_reference_sentinel, migrate_renamed_state_keys, ) from diff_diff.aggregation import ( @@ -143,7 +144,9 @@ class TwoStageDiDResults(BaseResults, AggregationMixin): Confidence interval for overall ATT. event_study_effects : dict, optional Dictionary mapping relative time h to effect dict with keys: - 'effect', 'se', 't_stat', 'p_value', 'conf_int', 'n_obs'. + 'effect', 'se', 't_stat', 'p_value', 'conf_int' and, on estimated + rows, 'n_obs'; the reference row carries ``'is_reference': True`` + and no count (4.0, ledger row M-093). group_effects : dict, optional Dictionary mapping cohort g to effect dict. groups : list @@ -253,7 +256,15 @@ def __setstate__(self, state: Dict[str, Any]) -> None: under their canonical names and ``overall_*`` never reappear in ``__dict__``. """ - self.__dict__.update(migrate_renamed_state_keys(state, LEGACY_OVERALL_QUINTET)) + state = migrate_renamed_state_keys(state, LEGACY_OVERALL_QUINTET) + if "event_study_effects" in state: + # 3.x surfaces marked the reference row with the n_obs == 0 + # sentinel (retired at 4.0, row M-093). + state = dict(state) + state["event_study_effects"] = migrate_event_study_reference_sentinel( + state["event_study_effects"] + ) + self.__dict__.update(state) # --- Post-fit aggregation (M-022/M-119) ------------------------------ @@ -700,8 +711,8 @@ def summary(self, alpha: Optional[float] = None) -> str: for h in sorted(self.event_study_effects.keys()): eff = self.event_study_effects[h] - if eff.get("n_obs", 1) == 0: - # Reference period marker + if eff.get("is_reference", False): + # Reference period row (explicit marker since 4.0, M-093) lines.append( f"[ref: {h}]" f"{'0.0000':>17} {'---':>12} {'---':>10} {'---':>10} {'':>6}" ) diff --git a/diff_diff/visualization/_event_study.py b/diff_diff/visualization/_event_study.py index 6a5aab631..7bec7f93b 100644 --- a/diff_diff/visualization/_event_study.py +++ b/diff_diff/visualization/_event_study.py @@ -972,19 +972,17 @@ def _extract_plot_data( if reference_period is None: reference_inferred = True # We're about to infer it # Prefer an explicit `reference_period` attribute on the result - # (Wave C ``SpilloverDiDResults`` sets this directly). The - # legacy `n_obs == 0` heuristic was ambiguous for rectangular - # outputs like SpilloverDiD's `event_study_effects`, which - # legitimately emits multiple empty non-reference horizons. + # (Wave C ``SpilloverDiDResults`` sets this directly), else the + # row marked ``is_reference`` (the explicit normalization-row + # marker every materializing producer writes since 4.0, ledger + # row M-093 item 8a - the zero-count sentinel is retired, so a + # zero count is a genuine empty horizon and never a marker). explicit_ref = getattr(results, "reference_period", None) if explicit_ref is not None: reference_period = int(explicit_ref) else: - # Detect reference period from n_groups=0 marker (normalization - # constraint). This handles anticipation > 0 where reference - # is at e = -1 - anticipation. for period, effect_data in results.event_study_effects.items(): - if effect_data.get("n_groups", 1) == 0 or effect_data.get("n_obs", 1) == 0: + if effect_data.get("is_reference", False): reference_period = period break # Fallback to -1 if no marker found (backward compatibility). @@ -1224,13 +1222,15 @@ def plot_honest_event_study( reference_period = getattr(original_results, "reference_period", None) if reference_period is None: # HonestDiD's own reference signature (honest_did.py): a - # normalization-constraint row, NOT any row whose inference - # happens to be undefined. No signature match (e.g. a - # marker-less varying-base surface) means no inferred + # normalization-constraint row - the producer's explicit + # ``is_reference`` key (4.0, row M-093; the zero-count sentinel + # is retired) with effect exactly 0.0 and NaN SE - NOT any row + # whose inference happens to be undefined. No signature match + # (e.g. a marker-less varying-base surface) means no inferred # reference - deliberately no -1 fallback here. for t, data in original_results.event_study_effects.items(): if ( - data.get("n_groups", data.get("n_obs", 1)) == 0 + data.get("is_reference", False) and data["effect"] == 0.0 and not np.isfinite(data["se"]) ): @@ -1238,15 +1238,15 @@ def plot_honest_event_study( break # Verified anchor: the resolved reference (attribute, signature # match, or caller label) counts only if its row carries the - # producers' constraint signature - zero group/obs count, effect + # producers' constraint signature - the ``is_reference`` key, effect # exactly 0.0, and no defined SE (NaN for CS, 0.0 for the # Imputation/TwoStage/Stacked-style markers). An estimated row - # (positive count) is never an anchor, even at effect 0, se 0. + # (no key) is never an anchor, even at effect 0, se 0. _anchor_period = None _ref_data = original_results.event_study_effects.get(reference_period) if ( _ref_data is not None - and _ref_data.get("n_groups", _ref_data.get("n_obs", 1)) == 0 + and _ref_data.get("is_reference", False) and _ref_data["effect"] == 0.0 and not (np.isfinite(_ref_data["se"]) and float(_ref_data["se"]) > 0) ): diff --git a/docs/api/spillover.rst b/docs/api/spillover.rst index 6ea09ff67..7f8a926a7 100644 --- a/docs/api/spillover.rst +++ b/docs/api/spillover.rst @@ -200,8 +200,10 @@ and planned follow-up enhancements: DataFrame plus MultiIndex ``spillover_effects``. The ``event_study_effects: Dict[int, Dict]`` alias mirrors ``TwoStageDiD``'s schema for ``plot_event_study`` consumption (the - plotter prefers the new ``reference_period`` attribute over the - legacy ``n_obs==0`` heuristic). ``DiagnosticReport`` routing for + plotter reads the ``reference_period`` attribute, else the row's + ``is_reference`` key; since 4.0 the reference row carries + ``is_reference: True`` and no ``n_obs``, and the event-study frames a + boolean ``is_reference`` column with NaN ``n_obs`` there - [M-093]). ``DiagnosticReport`` routing for ``SpilloverDiDResults`` is wired: parallel-trends (event-study on the direct-effect dynamics), design-effect, and heterogeneity apply; Goodman-Bacon is excluded (spillover identifies off far-away diff --git a/docs/methodology/REGISTRY.md b/docs/methodology/REGISTRY.md index af81c42f8..04990fac6 100644 --- a/docs/methodology/REGISTRY.md +++ b/docs/methodology/REGISTRY.md @@ -1364,7 +1364,7 @@ The multiplier bootstrap uses random weights w_i with E[w]=0 and Var(w)=1: - **Note:** Repeated cross-sections (`panel=False`, Phase 7b): supports surveys like BRFSS, ACS annual, and CPS monthly where units are not followed over time. Uses cross-sectional DRDID (Sant'Anna & Zhao 2020, Section 4): `reg` matches `DRDID::reg_did_rc` (Eq 2.2), `dr` matches `DRDID::drdid_rc` (locally efficient, Eq 3.3+3.4 with 4 OLS fits), `ipw` matches `DRDID::std_ipw_did_rc`. Per-observation influence functions instead of per-unit. All three estimation methods support covariates and survey weights. - **Note:** Panel and RCS influence functions use the library-wide `phi_i = psi_i / n` convention (SE = `sqrt(sum(phi^2))`, algebraically equivalent to R's `sd(psi)*sqrt(n-1)/n`). Leading IF terms are computed on psi scale and divided by n; PS nuisance corrections are computed on psi scale (`score @ solve(Hessian)`) with a single `/n` conversion to phi. - **Note:** Non-survey DR path also includes nuisance IF corrections (PS + OR), matching the survey path structure (Phase 7a). Previously used plug-in IF only. As of v3.7 the non-survey reg and ipw paths carry their corrections too (OR estimation-effect / PS score), so the nuisance-IF treatment is method-uniform. -- **Note (post-fit aggregate() - rows M-020/M-117):** `fit(aggregate=, balance_e=)` was deprecated in 3.9 and REMOVED at 4.0 (PR 3b of the cut: `fit()` computes no event-study/group surface, so the legacy `event_study_effects`/`group_effects` fields - and `cband_crit_value`/`event_study_vcov`/`event_study_vcov_index`/`event_study_df` - are None after EVERY fit, `to_dataframe(level='event_study'|'group')` raises on fresh fits until the M-093 schema rewire, and `survey_design` is positional slot 7) in favor of post-fit `results.aggregate(type, balance_e=)` on the fit-retained aggregation kit ('simple' relays the stored overall inference; 'event_study'/'group' recompute from the kit — on BOOTSTRAPPED fits they REPLAY the fit-time multiplier bootstrap from the kit's `BootstrapReplaySpec` RNG-state snapshot: percentile se/CI/cband matching a fit-time aggregation to BLAS reassociation (~1 ULP, `assert_allclose`, never bit-identity; the discrete percentile p-value is a count statistic outside that claim), df cleared (no analytical provenance beside percentile inference), vcov/vcov_index from the same bootstrap draws (see the bootstrap event-study covariance Note above). **Note (df_survey two-channel contract):** the container-level `df_survey` SCALAR - the fit's resolved scalar inference df (`EventStudyResults.df_survey`) - DELIBERATELY persists on bootstrapped replayed (CS, DMLDiD, EfficientDiD publish it identically) and relayed (ImputationDiD/TwoStageDiD/ContinuousDiD, 4.0 PR 3c) surfaces: it is a live consumer channel (HonestDiD's container branches consume it as the `df=` for their critical values on the containers they admit), and only the PER-ROW `df` column is the per-row inference-provenance channel that percentile bootstrap clears. A bootstrapped ImputationDiD survey fit with `pretrends=True` clears the per-row channel even though the lead rows retained analytical survey-df inference (the bootstrap loop skips pre-period horizons) - the shipped producer convention, backend-stamped and failing closed on a Rust-vs-NumPy weight-backend mismatch or a pre-replay legacy pickle — the M-027 per-level policy, with the former blanket fail-closed retired). `DiagnosticReport` now derives the event-study container internally on plain fits (raw `event_study_effects` absent), so its `parallel_trends`, `pretrends_power`, and `sensitivity` checks run on every CS fit — CS containers are M-093-admitted into `compute_pretrends_power` / `HonestDiD.sensitivity_analysis` with pinned raw-route parity, and the raw field, when present (the requested-but-empty `{}` included), always takes precedence. `heterogeneity` is unaffected (it reads `group_time_effects` on plain fits). Bootstrapped fits now DERIVE successfully via the percentile-bootstrap replay: the derived container carries the covariance of the bootstrap draws as `vcov`, so `parallel_trends` runs the joint Wald test and `pretrends_power`/`sensitivity` use the full covariance (the Bonferroni and diagonal-covariance fallbacks remain where no bootstrap covariance is stored); the replay's re-emitted fit-time warnings are recorded and republished per section. The fail-closed skip remains for kit-less/legacy pickles, backend-mismatched replays, and bootstrapped ImputationDiD/TwoStageDiD/ContinuousDiD fits whose fit-materialized surface was fail-soft cleared (since 4.0 PR 3c those three RELAY the surface computed at fit - see their bootstrapped-fit relay Notes). The recompute levels' replay RE-EMITS the fit-time bootstrap warnings by design (the relay levels' no-re-warn convention is scoped to 'simple'/'total'). +- **Note (post-fit aggregate() - rows M-020/M-117):** `fit(aggregate=, balance_e=)` was deprecated in 3.9 and REMOVED at 4.0 (PR 3b of the cut: `fit()` computes no event-study/group surface, so the legacy `event_study_effects`/`group_effects` fields - and `cband_crit_value`/`event_study_vcov`/`event_study_vcov_index`/`event_study_df` - are None after EVERY fit, `to_dataframe(level='event_study'|'group')` raises on fresh fits until the M-093 schema rewire, and `survey_design` is positional slot 7) in favor of post-fit `results.aggregate(type, balance_e=)` on the fit-retained aggregation kit ('simple' relays the stored overall inference; 'event_study'/'group' recompute from the kit — on BOOTSTRAPPED fits they REPLAY the fit-time multiplier bootstrap from the kit's `BootstrapReplaySpec` RNG-state snapshot: percentile se/CI/cband matching a fit-time aggregation to BLAS reassociation (~1 ULP, `assert_allclose`, never bit-identity; the discrete percentile p-value is a count statistic outside that claim), df cleared (no analytical provenance beside percentile inference), vcov/vcov_index from the same bootstrap draws (see the bootstrap event-study covariance Note above). **Note (df_survey two-channel contract):** the container-level `df_survey` SCALAR - the fit's resolved scalar inference df (`EventStudyResults.df_survey`) - DELIBERATELY persists on bootstrapped replayed (CS, DMLDiD, EfficientDiD publish it identically) and relayed (ImputationDiD/TwoStageDiD/ContinuousDiD, 4.0 PR 3c) surfaces: it is a live consumer channel (HonestDiD's container branches consume it as the `df=` for their critical values on the containers they admit), and only the PER-ROW `df` column is the per-row inference-provenance channel that percentile bootstrap clears. A bootstrapped ImputationDiD survey fit with `pretrends=True` clears the per-row channel even though the lead rows retained analytical survey-df inference (the bootstrap loop skips pre-period horizons) - the shipped producer convention, backend-stamped and failing closed on a Rust-vs-NumPy weight-backend mismatch or a pre-replay legacy pickle — the M-027 per-level policy, with the former blanket fail-closed retired). `DiagnosticReport` now derives the event-study container internally on plain fits (raw `event_study_effects` absent), so its `parallel_trends`, `pretrends_power`, and `sensitivity` checks run on every CS fit — CS containers are M-093-admitted into `compute_pretrends_power` / `HonestDiD.sensitivity_analysis` with pinned raw-route parity, and the raw field, when present (the requested-but-empty `{}` included), always takes precedence. `heterogeneity` is unaffected (it reads `group_time_effects` on plain fits). Bootstrapped fits now DERIVE successfully via the percentile-bootstrap replay: the derived container carries the covariance of the bootstrap draws as `vcov`, so `parallel_trends` runs the joint Wald test and `pretrends_power`/`sensitivity` use the full covariance (the Bonferroni and diagonal-covariance fallbacks remain where no bootstrap covariance is stored); the replay's re-emitted fit-time warnings are recorded and republished per section. The fail-closed skip remains for kit-less/legacy pickles, backend-mismatched replays, and bootstrapped ImputationDiD/TwoStageDiD/ContinuousDiD fits whose fit-materialized surface was fail-soft cleared (since 4.0 PR 3c those three RELAY the surface computed at fit - see their bootstrapped-fit relay Notes). The recompute levels' replay RE-EMITS the fit-time bootstrap warnings by design (the relay levels' no-re-warn convention is scoped to 'simple'/'total'). (8a, [M-093]: the carrier's reference-only horizons carry `is_reference: True` and no `n_groups`.) - **Note (post-fit `aggregate('total')` - the estimator-owned total incremental outcome, 3.10):** (a) **Estimand:** `total = C x overall_att` with `se = C x overall_se`, CI scaled by `C`, `t`/`p`/`df` inherited unchanged - an exact relay CONDITIONAL ON THE REALIZED AGGREGATION MASS (never call it unqualified "exact"): `C` is the design-fixed complete-case treated-observation count on every admitted routing, the inherited SE prices the overall's full inference (WIF/estimated-share terms included) but treats `C` as fixed at its realized value, and the random-mass `att*dC` variance term is deliberately out of scope (the DEFERRED survey/RC remainder). Every SE branch is homogeneous of degree 1 in the scaling, so the relay is exact under analytical AND percentile-bootstrap inference. (b) **Fail-closed routing gates**, in order: (1) repeated-cross-section-routed fits (`panel=False`, or genuinely-unbalanced `allow_unbalanced_panel=True` fits - a balanced panel with the flag stays panel-routed and admitted) raise `NotImplementedError` - the RC cohort mass counts UNITS and weights every kept post period by it, a ~T-fold overcount of treated observations on a T-wave RCS (execution-verified); (2) fits declaring a `survey_design=` raise - the realized-mass relay omits the survey mass-uncertainty term and design-aware population-scale totals are not implemented (retained weight scale differs by design family: analytic pweight resolves normalized to sum = n and CS retains no PER-OBSERVATION raw-weight record - the `survey_metadata` scalars report raw-scale `sum_weights`/`weight_range` provenance, but not the per-obs raw vector a population-scale total would need; CS accepts pweight only - while replicate designs retain raw scale); the gate reads the fit-time `is_survey_fit` KIT snapshot, so bare-`cluster=` fits are ADMITTED (their synthesized all-ones internal design is not a survey) and a post-fit edit of the public `survey_metadata` cannot bypass it; the DECLARATION BOUNDARY is deliberate - an explicit unweighted `SurveyDesign(psu=...)` fit is numerically identical to bare `cluster=` yet fails closed by declaration (declaring survey_design opts into the survey contract; conservative, never wrong numbers; the message points at the `cluster=` route); a combined survey+RC fit deterministically gets the RC message; (3) the coincidence check: on bare-`cluster=` fits `_aggregate_simple` weights kept cells by the synthesized cohort masses (full cohort size per kept cell), and when kept cells have INCOMPLETE treated support that realized mass diverges from the complete-case count - the same overcount class as (1) - so the fit raises rather than publishing an ambiguous mass; clean bare-`cluster=` fits (the normal case) are admitted with `C` equal to both counts. Panel-routed pre-upgrade (legacy) kits missing the fit-time snapshots raise the refit message - never a mutable-field or public-replay fallback. (c) **Filter-replay contract:** `C` replays `_aggregate_simple`'s cell selection verbatim over the immutable `agg_gt_cells` fit-time kit snapshot (EDiD snapshot discipline; the kit builder stashes `(g, t, effect, n_treated)` 4-tuples in dict order) - anticipation filter FIRST (cells with `g - anticipation <= t < g` COUNT as post, the business-facing subtlety; universal-base reference cells are excluded by this filter alone, their base period being `< g - anticipation` by construction), finite-effect mask SECOND, weight = cohort mass where `fixed_cohort_agg_weights` provides one else per-cell complete-case `n_treated` (the source's `agg_weight` fallback is deliberately absent - RC-only, gated out), pairwise `np.sum`; the overall scalars come from the same stored fields the `'simple'` relay reads, so a post-fit mutation of `group_time_effects` moves NEITHER factor and `total == C x aggregate('simple').att` cannot desynchronize (pinned by a mutation test). NEVER use `agg_total_weight` (the RC all-units WIF mass). (d) **NaN rules:** empty post set / all-NaN effects give `C = NaN` and an all-NaN row WITHOUT re-emitting the fit-time UserWarnings (the RELAY-level no-re-warn convention — the NaN row is the signal; the recompute levels' bootstrap replay re-emits by design); with finite `C` the relay is VERBATIM - inherited NaNs (a degenerate fit's `(nan, nan)` CI beside finite att/se = 0) pass through, mirroring `aggregate('simple')` per the repo's non-estimable-row convention (att/n are never blanked by NaN inference fields); the ONLY additional blanking is true float overflow (a finite overall value made non-finite BY the `x C` scaling - unreachable in practice, guarded per this code family's finiteness convention). (e) **Bootstrap:** 'total' is a RELAY level - available on bootstrapped fits with the df column NaN (the M-027 per-level policy). (f) **Container conventions:** single row, `level="total"`, `label=["total"]`, `target=["total"]` (a total incremental outcome, NOT an ATT - the single-non-att neutral "estimate" rendering applies), `n = [C]` with `n_kind="obs"`, `weight=[1.0]`, per-class df carrier (CS: `resolve_inference_df` - finite on admitted bare-`cluster=` fits, NaN on plain analytical panel fits). Per-level `n` semantics are deliberately DIFFERENT: 'total''s `n` is the treated observations entering the support, while 'simple''s `n` stays treated+control units by contract - `n_kind` is per-container and each is documented, satisfying the never-conflate rule. (g) **MMM admission:** both `diff_diff.mmm` exporters accept the total container ALONE - `scale` (numeric or `"auto"`) is rejected as double-counting; for overall-total exports this route supersedes `scale="auto"` (see the MMM section). The total remains meaningful only for outcomes additive in levels - the same unverifiable caveat every scale route carries. @@ -2335,7 +2335,8 @@ where `W_it(h) = 1[K_it = h]` are lead indicators, estimated on `Omega_0` only. - [x] Event study and group aggregation - **Note (post-fit aggregate() - rows M-021/M-118):** `fit(aggregate=, balance_e=)` was deprecated in 3.9 and REMOVED at 4.0 (PR 3c of the cut; `survey_design` is positional slot 7; the `imputation_did()` wrapper that used to forward the shared sentinel was removed per M-070 - PR 2) in favor of post-fit `ImputationDiDResults.aggregate(type, balance_e=)` - a PANEL-BACKED lazy recompute kit (not an EIF-payload kit: ES/group aggregation is a target-specific Theorem-3 recompute - each `balance_e` re-masks which treated observations enter every horizon and re-solves the untreated projection - so no compact influence payload can replace the frame). (a) RETAINED BUFFERS (memory contract): the kit's bookkeeping holds REFERENCES to the SAME per-fit objects `_fit_data` already retains for `pretrend_test()` - the working panel copy (all user columns plus `_tau_hat`/`_rel_time`/`_never_treated`), the Omega masks, `unit_fe`/`time_fe`/`grand_mean`/`delta_hat`/`kept_cov_mask`, the resolved survey design, and `survey_weights` - ZERO marginal memory, and pickles are unchanged via memoization (`_estimator_ref` already ships these objects); plus value SNAPSHOTS for isolation (a `treatment_groups` copy, config scalars, a `dataclasses.replace` copy of `survey_metadata`, `att`, `n_treated_obs`) and TWO df-provenance scalars (`survey_df_seed`, what the analytical aggregators received; `survey_df_final`, what the stored overall inference received). Each `aggregate()` call runs on a fresh throwaway host with a call-local projection cache (the fit-local factorizations are unpicklable and never retained). (b) `balance_e` uses the BALANCED-WINDOW rule: a cohort is retained iff its observed relative-time set - checked against the FULL panel via `_build_cohort_rel_times()` - covers the contiguous window `[-balance_e, max_h]`; the SAME rule TwoStageDiD uses, divergent from CS/EfficientDiD's anchor-horizon rule. A window no cohort satisfies warns and yields the reference-marker-only dict (a legal near-empty container). (c) BOOTSTRAP fits: 'simple' RELAYS the stored overall quintet verbatim (finite safe_inference t included) with a NaN df column; since 4.0 (PR 3c) the ES/group levels RELAY the percentile surface materialized at fit (see the bootstrapped-fit relay Note below; `balance_e` fails closed there - the balanced-window replay via the retained psi machinery is the TODO row); the prior uniform fail-closed rule was superseded 2026-08-05 with the M-027 per-level convergence. (d) CONTAINER ADMISSION into `compute_honest_did`/`compute_pretrends_power` is REJECTED BY DESIGN (both terminal TypeErrors state it): the surface carries no joint event-study covariance - per-horizon conservative SEs only (container `vcov=None`; the PER-ROW df column now threads the FINAL survey df via the results-level `event_study_df` channel - M-092 completion; `survey_df_final` at fit, the level-matched replay value on replicate replays; leads included on survey fits; None on non-survey/bootstrapped/no-surface fits and for the 0 sentinel - beside the scalar `df_survey` channel; see the CS post-fit Note's two-channel contract). (e) RELAY CONVENTIONS: 'simple' relays the stored overall quintet bit-exact with `n = n_treated_obs`, `n_kind="obs"` (the treated/control UNIT sets overlap - a treated unit with pre-periods counts in both - so the CS/EDiD disjoint-units convention cannot apply; |Omega_1| is the population the ATT averages over, of which only finite-tau-hat observations enter the average - `n` reports the raw count, so on partially unidentified fits `n` exceeds the averaged support) and `df = survey_df_final`; 'group' rows carry per-row `df_used` captured at each row's `safe_inference` (the replicate override rewrites it, the bootstrap override clears it, the all-NaN cohort branch writes no key - consumers read via `.get`); 'event_study' rides the shared `_from_relative_dict` builder via a carrier whose metadata is a copy-on-use of the KIT's fit-final metadata copy. REPLICATE-WEIGHT fits replay the extracted `_replicate_override_aggregates` with a LEVEL-MATCHED stack: `compute_replicate_refit_variance` validates replicates JOINTLY (all-finite rows), so `aggregate(L)` reproduces `fit(aggregate=L)` exactly, a `fit(aggregate='all')` surface is NOT the equivalence target when a replicate NaNs on exactly one family's targets, and - the documented migration delta - moving a replicate fit from `fit(aggregate=)` to plain fit + post-fit `aggregate()` can change the public OVERALL row's se/CI/df on such degenerate designs (each surface self-consistent; pinned in the contract tests). `pretrends=True` + replicate: post-fit `aggregate('event_study')` raises NotImplementedError (per-replicate lead refits unimplemented; the unreachable fit-time twin of this gate was deleted at 4.0); 'group'/'simple' still work. Recompute re-emits the fit-time warnings (LSMR, Prop-5, empty-window) with fit-tuned stacklevels - post-fit attribution lands on a library frame, an accepted verbatim-move trade-off. `DiagnosticReport` now derives this container internally on plain fits (raw `event_study_effects` absent), so its `parallel_trends` (`pretrends=True` fits) and `heterogeneity` checks run on analytic plain fits (bootstrapped fits are read from the fit-materialized raw fields - raw-field precedence); the re-emitted recompute warnings are captured and re-published on the consuming report section (record-and-republish), and derivation failures (missing kit, the replicate gate above, a fail-soft-cleared bootstrapped surface) surface as explicit per-check skip reasons. (f) `aggregate('total')` (3.10): the estimator-owned total incremental outcome - an exact relay `C x overall` CONDITIONAL on the realized aggregation mass, with `C` the FINITE-tau complete-case support (`total_support`, a float SNAPSHOT stashed at kit build precisely because the kit's `df` is a live `_fit_data` reference - a post-fit frame edit must not move the total; pre-upgrade kits missing the stash raise the refit message). Since the overall IS the finite-support mean, `C x overall = sum(tau)` exactly - this total uses the FINITE support and thereby fixes the documented raw-|Omega_1| overcount of the MMM `scale=\"auto\"` route for total exports ('simple''s `n` stays the raw count by contract; different levels, different documented n semantics). Zero support (the all-unidentified fit, overall already NaN) -> NaN mass -> all-NaN row (no re-warn); finite `C` relays verbatim - the constant-outcome degenerate fit keeps finite att/se = 0 beside its inherited `(nan, nan)` CI, mirroring 'simple'. Unweighted fits only: declared `survey_design=` fails closed via the kit's `survey_metadata` snapshot (mutation-proof). Bootstrap fits relay with a NaN df column (M-027); df carrier = `survey_df_final`; container: single `target='total'` row, `n_kind='obs'`. MMM: admitted with NO scale (see the MMM section). -- **Note (bootstrapped-fit relay, 4.0):** (i) RELAY: a fit with `n_bootstrap > 0` ALWAYS computes the event-study AND group percentile surfaces at fit (`balance_e=None` - exactly what the 3.x `fit(aggregate='all')` computed; percentile se/p/CI, recomputed t, `df_used`/`event_study_df` cleared), stores them in `event_study_effects`/`group_effects`, and `aggregate('event_study')`/`aggregate('group')` RELAY them (alpha/anticipation/survey metadata from the kit snapshot; per-row `df` column NaN - the `pretrends=True` lead rows show NaN df too, identical to the 3.x bootstrapped surface, by design; `df_survey` scalar from the carrier's survey metadata; `vcov`/`vcov_index` None; reference rows via the zero-count sentinel). `aggregate('event_study', balance_e=k)` on a bootstrapped fit FAILS CLOSED (`NotImplementedError`: the stored surface is unrestricted and the per-target draws are not retained; re-fit with `n_bootstrap=0`). Analytic fits are unchanged (kit recompute; legacy fields None). The gate is `bootstrap_results is not None`: `None` (n_bootstrap == 0, OR a bootstrapped fit that ended analytical) means the analytical kit RECOMPUTE, the pre-4.0 behaviour. (ii) FAIL-SOFT materialization + storage rule: the surfaces are built inside one guard - any exception (e.g. the `pretrends=True` + `rank_deficient_action='error'` lead regression raising `ValueError` on a lead-collinear covariate) leaves them None and the bootstrap runs overall-only (the pre-4.0 plain-fit path, which SUCCEEDS where the 3.x `fit(aggregate='event_study')` raised); a bootstrap attempt that raises (anything but `NotImplementedError`, re-raised) WITH surfaces present clears them and RETRIES ONCE overall-only - `_precompute_bootstrap_psi` + `_run_bootstrap` re-run with `event_study_effects=None, group_effects=None`; `np.random.default_rng(self.seed)` is rebuilt on entry, so a seeded retry reproduces attempt 1's weight draws and the overall quintet equals a plain fit's (an unseeded retry draws a fresh, equally valid realization). The retry keeps the pre-4.0 failure semantics: a psi-precompute failure -> the 'Bootstrap pre-computation failed ... Skipping bootstrap inference.' warning + analytical overall; a `_run_bootstrap` failure -> RE-RAISED (the fit aborts exactly as before). STORAGE RULE: the legacy fields hold a surface IFF `bootstrap_results is not None`; when the fit ends analytical (`n_valid == 0` skipped the psi precompute, total failure) they are cleared and `aggregate()` RECOMPUTES analytically - an analytical surface must never sit on a bootstrapped fit where `summary()`/`to_dataframe()`/the DiagnosticReport raw-field precedence would read it as the fit's inference. The fail-soft `UserWarning` fires only when the fit DOES carry bootstrap inference and a surface was dropped: 'ImputationDiD: the event-study/group surfaces could not be materialized on this bootstrapped fit ({e}); the bootstrap covered the overall ATT only, so aggregate('event_study') / aggregate('group') are unavailable here - re-fit with n_bootstrap=0 for the analytical post-fit recompute.'; the relay then raises the shared absent-surface `ValueError` ('This bootstrapped ImputationDiDResults carries no stored event_study surface: either it was fitted before 4.0 (a bootstrapped plain fit built none) or the surface could not be materialized at fit (see the fit-time warning). Re-fit with diff-diff >= 4.0 for the relayed surface, or with n_bootstrap=0 for the analytical post-fit recompute.'). Net invariant: a bootstrapped plain fit's overall quintet and success/abort outcome are unchanged by the 4.0 removal. (iii) Newly reachable warnings on bootstrapped plain fits: the aggregators' own `UserWarning`s (Prop-5 unidentified horizons, empty horizon after filters, LOO single-unit auxiliary groups) and the per-target bootstrap-stats `RuntimeWarning`s ('Dropping n/N non-finite bootstrap samples in ... event study (h=...)', 'Too few valid bootstrap samples', 'Bootstrap SE is non-finite or zero'). (iv) Pre-4.0 pickle limitation: a 3.9-3.13 bootstrapped pickle fitted with `fit(aggregate=..., balance_e=k)` stores a balance_e-RESTRICTED surface and records no fit-time `balance_e`, so the relay returns that surface as stored - re-fit under 4.0 for the unrestricted one; one fitted plain stores none and the relay raises the refit `ValueError`. (v) The relay reads the MUTABLE public `event_study_effects`/`group_effects` fields (a view by definition - the StackedDiD precedent); only alpha/anticipation/survey metadata come from kit snapshots, so the recompute route's 'kit snapshots only' invariant does not hold for the relayed dict. (vi) Downstream: `summary()` prints the ES/group tables and `to_dataframe(level='event_study'|'group')` returns frames on bootstrapped plain fits; `DiagnosticReport`/`BusinessReport` run `heterogeneity` (on the relayed GROUP percentile dict - `_collect_effect_scalars` reads `group_effects` first) and `parallel_trends` (`pretrends=True`) on the relayed surface instead of skipping with `NotImplementedError` (no sensitivity check applies); `plot_event_study(results)` plots them natively. Replicate-weight designs never meet the always-on surfaces (replicate + `n_bootstrap > 0` is rejected before them), so the fit-time replicate stack is always `[overall]`-only - the degenerate-design overall-row migration delta in (e) stands. +- **Note (bootstrapped-fit relay, 4.0):** (i) RELAY: a fit with `n_bootstrap > 0` ALWAYS computes the event-study AND group percentile surfaces at fit (`balance_e=None` - exactly what the 3.x `fit(aggregate='all')` computed; percentile se/p/CI, recomputed t, `df_used`/`event_study_df` cleared), stores them in `event_study_effects`/`group_effects`, and `aggregate('event_study')`/`aggregate('group')` RELAY them (alpha/anticipation/survey metadata from the kit snapshot; per-row `df` column NaN - the `pretrends=True` lead rows show NaN df too, identical to the 3.x bootstrapped surface, by design; `df_survey` scalar from the carrier's survey metadata; `vcov`/`vcov_index` None; reference rows via the `is_reference` key). `aggregate('event_study', balance_e=k)` on a bootstrapped fit FAILS CLOSED (`NotImplementedError`: the stored surface is unrestricted and the per-target draws are not retained; re-fit with `n_bootstrap=0`). Analytic fits are unchanged (kit recompute; legacy fields None). The gate is `bootstrap_results is not None`: `None` (n_bootstrap == 0, OR a bootstrapped fit that ended analytical) means the analytical kit RECOMPUTE, the pre-4.0 behaviour. (ii) FAIL-SOFT materialization + storage rule: the surfaces are built inside one guard - any exception (e.g. the `pretrends=True` + `rank_deficient_action='error'` lead regression raising `ValueError` on a lead-collinear covariate) leaves them None and the bootstrap runs overall-only (the pre-4.0 plain-fit path, which SUCCEEDS where the 3.x `fit(aggregate='event_study')` raised); a bootstrap attempt that raises (anything but `NotImplementedError`, re-raised) WITH surfaces present clears them and RETRIES ONCE overall-only - `_precompute_bootstrap_psi` + `_run_bootstrap` re-run with `event_study_effects=None, group_effects=None`; `np.random.default_rng(self.seed)` is rebuilt on entry, so a seeded retry reproduces attempt 1's weight draws and the overall quintet equals a plain fit's (an unseeded retry draws a fresh, equally valid realization). The retry keeps the pre-4.0 failure semantics: a psi-precompute failure -> the 'Bootstrap pre-computation failed ... Skipping bootstrap inference.' warning + analytical overall; a `_run_bootstrap` failure -> RE-RAISED (the fit aborts exactly as before). STORAGE RULE: the legacy fields hold a surface IFF `bootstrap_results is not None`; when the fit ends analytical (`n_valid == 0` skipped the psi precompute, total failure) they are cleared and `aggregate()` RECOMPUTES analytically - an analytical surface must never sit on a bootstrapped fit where `summary()`/`to_dataframe()`/the DiagnosticReport raw-field precedence would read it as the fit's inference. The fail-soft `UserWarning` fires only when the fit DOES carry bootstrap inference and a surface was dropped: 'ImputationDiD: the event-study/group surfaces could not be materialized on this bootstrapped fit ({e}); the bootstrap covered the overall ATT only, so aggregate('event_study') / aggregate('group') are unavailable here - re-fit with n_bootstrap=0 for the analytical post-fit recompute.'; the relay then raises the shared absent-surface `ValueError` ('This bootstrapped ImputationDiDResults carries no stored event_study surface: either it was fitted before 4.0 (a bootstrapped plain fit built none) or the surface could not be materialized at fit (see the fit-time warning). Re-fit with diff-diff >= 4.0 for the relayed surface, or with n_bootstrap=0 for the analytical post-fit recompute.'). Net invariant: a bootstrapped plain fit's overall quintet and success/abort outcome are unchanged by the 4.0 removal. (iii) Newly reachable warnings on bootstrapped plain fits: the aggregators' own `UserWarning`s (Prop-5 unidentified horizons, empty horizon after filters, LOO single-unit auxiliary groups) and the per-target bootstrap-stats `RuntimeWarning`s ('Dropping n/N non-finite bootstrap samples in ... event study (h=...)', 'Too few valid bootstrap samples', 'Bootstrap SE is non-finite or zero'). (iv) Pre-4.0 pickle limitation: a 3.9-3.13 bootstrapped pickle fitted with `fit(aggregate=..., balance_e=k)` stores a balance_e-RESTRICTED surface and records no fit-time `balance_e`, so the relay returns that surface as stored - re-fit under 4.0 for the unrestricted one; one fitted plain stores none and the relay raises the refit `ValueError`. (v) The relay reads the MUTABLE public `event_study_effects`/`group_effects` fields (a view by definition - the StackedDiD precedent); only alpha/anticipation/survey metadata come from kit snapshots, so the recompute route's 'kit snapshots only' invariant does not hold for the relayed dict. (vi) Downstream: `summary()` prints the ES/group tables and `to_dataframe(level='event_study'|'group')` returns frames on bootstrapped plain fits; `DiagnosticReport`/`BusinessReport` run `heterogeneity` (on the relayed GROUP percentile dict - `_collect_effect_scalars` reads `group_effects` first) and `parallel_trends` (`pretrends=True`) on the relayed surface instead of skipping with `NotImplementedError` (no sensitivity check applies); `plot_event_study(results)` plots them natively. Replicate-weight designs never meet the always-on surfaces (replicate + `n_bootstrap > 0` is rejected before them), so the fit-time replicate stack is always `[overall]`-only - the degenerate-design overall-row migration delta in (e) stands. +- **Note (reference-row marking, 4.0, [M-093] item 8a):** the `event_study_effects` reference row (`e = -1 - anticipation`) carries `"is_reference": True` and no `n_obs` key (3.x: the `n_obs == 0` sentinel, retired); every remaining `n_obs` is a genuine count. `summary()` keys its `[ref: h]` line on the marker; the bootstrap loop skips the marked row; 3.x pickles are migrated on load (`_deprecation.migrate_event_study_reference_sentinel` in `__setstate__`). The native `to_dataframe(level='event_study')` frame renders the reference row's `n_obs` as NaN (float column) until the 8b schema rewire. - **Note:** `anticipation` is validated at construction (non-negative integer; `bool` rejected) via the shared `utils.validate_anticipation`, and re-checked on the fit path (direct-mutation defense) — see the family-wide adoption note (ledger row [M-144]) in the TripleDifference staggered-mode section. --- @@ -2431,7 +2432,8 @@ Our implementation uses multiplier bootstrap on the GMM influence function: clus - [x] Event study and overall ATT aggregation - **Note (post-fit aggregate() - rows M-022/M-119):** `fit(aggregate=, balance_e=)` was deprecated in 3.9 and REMOVED at 4.0 (PR 3c of the cut; `survey_design` is positional slot 7; the `two_stage_did()` wrapper that used to forward the shared sentinel was removed per M-071 - PR 2) in favor of post-fit `TwoStageDiDResults.aggregate(type, balance_e=)` - a PANEL-BACKED lazy recompute kit: each level is a fresh Stage-2 OLS + joint Gardner-GMM sandwich on a level-specific design, so no compact influence payload exists. (a) RETAINED BUFFERS (memory contract - the FIRST panel retention on TwoStageDiD results, a deliberate break from the CS/EDiD identifier-minimization guarantee, with a `store_kit` opt-out tracked in DEFERRED.md): a COLUMN-SUBSET COPY of the working frame - `unit`/`time`/`outcome`/`first_treat` + covariates + the cluster column (deduplicated: `cluster=` may legally name a core column) + `_never_treated`/`_rel_time`/`_y_tilde` - O(n_obs) on every results object and pickle; the Stage-1 FE model (`unit_fe`/`time_fe`/`grand_mean`/`delta_hat`/`kept_cov_mask`), the Omega masks, the full-domain `keep_mask`, the Wave-E.3-GATED `score_pad_mask`/`cluster_ids_full` values fit actually passed (None unless the always-treated pad was active), `survey_weights`, and the resolved survey design - on replicate designs that adds the O(n_obs x R) replicate matrix; plus value snapshots (`treatment_groups` copy, `ref_period`, `overall_att`, `n_treated_obs`, a `dataclasses.replace` copy of `survey_metadata`) and TWO df scalars (`survey_df_stage2`, the recompute seed; `survey_df_final`, what the stored overall inference received). (b) `balance_e` uses the BALANCED-WINDOW rule (`[-balance_e, max_h]` coverage against the full panel - the ImputationDiD rule, divergent from CS/EfficientDiD's anchor-horizon rule); zero qualifying cohorts warns and yields the reference-row-only dict with `vcov=None`. (c) BOOTSTRAP fits: 'simple' RELAYS the stored overall quintet verbatim (finite safe_inference t included) with a NaN df column, while since 4.0 (PR 3c) the ES/group levels RELAY the percentile surface materialized at fit (see the bootstrapped-fit relay Note below; `balance_e` fails closed there - a balanced-window replay needs retained per-level GMM scores, which are function-locals: the TODO row) - the prior uniform rule superseded 2026-08-05 with the M-027 per-level convergence; a fit whose bootstrap FAILED (`bootstrap_results=None`, analytical inference retained) aggregates normally (analytical recompute). (d) CONTAINER ADMISSION into `compute_honest_did`/`compute_pretrends_power` is DEFERRED, not by-design (both terminal TypeErrors state it): analytical surfaces DO carry the real joint Gardner-GMM covariance (M-092), but the pre-period coefficients are stage-1 residual MEANS - the reference horizon is dropped from the no-intercept Stage-2 design and the zero anchor row is appended mechanically - not contrasts against the advertised reference, while HonestDiD's Delta^RM/Delta^SD arithmetic hard-codes the `delta_0 = 0` normalization into its boundary/bridge constraints; admission awaits a normalization derivation (either re-estimating Stage 2 with the reference horizon in the design or deriving the residual-to-reference mapping) - the DEFERRED.md paper-gated row. (e) RELAY CONVENTIONS: 'simple' relays the stored overall quintet bit-exact with `n = n_treated_obs`, `n_kind="obs"` (overlapping unit sets - the StackedDiD carve-out class; the pre-filter |Omega_1| count, while the ATT's Stage-2 support excludes rows whose `y_tilde` is non-finite - on such degenerate fits `n` exceeds the averaged support) and `df = survey_df_final` (on replicate fits that value came from the `[overall]`-only joint stack - snapshotted, never re-derived); 'group' relays a SCALAR df broadcast (deliberate divergence from ImputationDiD's per-row `df_used`: `_stage2_group` passes one immutable `survey_df` to every row's `safe_inference`, so the scalar is provenance-exact by construction and the moved method stays verbatim); 'event_study' reproduces the M-092 container contract exactly - analytical fits thread the recomputed joint vcov + `vcov_index` + the finite-and->0 df scalar through the carrier, replicate fits thread `vcov=None`/`index=None` with the REPLAYED level-matched df, and the carrier's metadata is a copy-on-use of the KIT's fit-final metadata copy. REPLICATE-WEIGHT fits replay the extracted `_replay_replicate_inference` with a LEVEL-MATCHED stack (the ImputationDiD semantics: `aggregate(L)` reproduces `fit(aggregate=L)`; `fit(aggregate='all')` is not the equivalence target on degenerate designs; the OVERALL-row migration delta on such designs is documented and pinned). Recompute re-emits fit-time warnings with fit-tuned stacklevels - an accepted verbatim-move trade-off. `DiagnosticReport` now derives this container internally on plain fits (raw `event_study_effects` absent), so its `parallel_trends` (`pretrends=True` fits; the recomputed joint vcov drives the joint-Wald path) and `heterogeneity` checks run on analytic plain fits (bootstrapped fits are read from the fit-materialized raw fields - raw-field precedence, no joint vcov there); re-emitted recompute warnings are captured and re-published on the consuming report section, and derivation failures (missing kit, a fail-soft-cleared bootstrapped surface) surface as explicit per-check skip reasons. (f) `aggregate('total')` (3.10): the estimator-owned total incremental outcome - an exact relay `C x overall` CONDITIONAL on the realized aggregation mass, with `C` the POST-FILTER treatment-indicator support the second stage actually regressed on (`omega_1_mask` minus non-finite `_y_tilde` rows, computed inline from the kit's private column-subset frame COPY - snapshot-grade, no fit-time stash needed, and the fit-time masker is never re-called so its warnings are not re-emitted; `score_pad_mask` is irrelevant here - it pads scores/variance only). This is the post-filter count 'simple''s raw pre-filter `n` documents as exceeding on degenerate fits. `D.sum() == 0` -> NaN mass -> all-NaN row (mirrors the fit's `(nan, nan)`); finite `C` relays verbatim. Unweighted fits only: declared `survey_design=` fails closed via the kit's `survey_metadata` snapshot (mutation-proof). Bootstrap fits relay with a NaN df column (M-027); df carrier = `survey_df_final`; container: single `target='total'` row, `n_kind='obs'`. MMM: admitted with NO scale (see the MMM section). -- **Note (bootstrapped-fit relay, 4.0):** (i) RELAY: a fit with `n_bootstrap > 0` ALWAYS computes the event-study (`_stage2_event_study`) AND group (`_stage2_group`) percentile surfaces at fit (`balance_e=None` - exactly what the 3.x `fit(aggregate='all')` computed; percentile se/p/CI; `event_study_vcov`/`event_study_vcov_index`/`event_study_df` None), stores them in `event_study_effects`/`group_effects`, and `aggregate('event_study')`/`aggregate('group')` RELAY them (alpha/anticipation/survey metadata from the kit snapshot; per-row `df` column NaN on both levels - the group relay passes `group_df=None` so no analytical df leaks beside percentile inference; `df_survey` scalar from the carrier's survey metadata; `vcov`/`vcov_index` None; reference rows via the zero-count sentinel). `aggregate('event_study', balance_e=k)` on a bootstrapped fit FAILS CLOSED (`NotImplementedError`: the stored surface is unrestricted and the per-level GMM scores are not retained; re-fit with `n_bootstrap=0`). Analytic fits are unchanged (kit recompute; legacy fields None). The gate is `bootstrap_results is not None`: `None` (n_bootstrap == 0, OR a bootstrapped fit that ended analytical) means the analytical kit RECOMPUTE, the pre-4.0 behaviour. (ii) FAIL-SOFT materialization + storage rule: the surfaces are built inside one guard - any exception leaves them None (the `_es_vcov_ts`/`_es_vcov_index_ts` locals reset too) and the bootstrap runs overall-only (the pre-4.0 plain-fit path); a `_run_bootstrap` attempt that raises (anything but `NotImplementedError`, re-raised) OR RETURNS `None` WITH surfaces present - `_run_bootstrap` returns `None` without an exception when `_compute_cluster_S_scores` hits `_TS_LSMRUnconverged` in the ES or group stage, since the LSMR fallback certifies each RHS column separately and the overall column can succeed while an ES/group column does not - clears them and RETRIES ONCE overall-only (`original_event_study=None, original_group=None, aggregate=None`; `np.random.default_rng(self.seed)` is rebuilt on entry, so a seeded retry reproduces attempt 1's weight draws and the overall quintet equals a plain fit's). A retry failure (a `None` return included) keeps the pre-4.0 catch-all 'Bootstrap failed' warning + analytical overall. STORAGE RULE: the legacy fields hold a surface IFF `bootstrap_results is not None`; when the fit ends analytical they are cleared and `aggregate()` RECOMPUTES analytically. The fail-soft `UserWarning` fires only when the fit DOES carry bootstrap inference and a surface was dropped: 'TwoStageDiD: the event-study/group surfaces could not be materialized on this bootstrapped fit ({e}); the bootstrap covered the overall ATT only, so aggregate('event_study') / aggregate('group') are unavailable here - re-fit with n_bootstrap=0 for the analytical post-fit recompute.'; the relay then raises the shared absent-surface `ValueError` ('This bootstrapped TwoStageDiDResults carries no stored event_study surface: either it was fitted before 4.0 (a bootstrapped plain fit built none) or the surface could not be materialized at fit (see the fit-time warning). Re-fit with diff-diff >= 4.0 for the relayed surface, or with n_bootstrap=0 for the analytical post-fit recompute.'). Net invariant: a bootstrapped plain fit's overall quintet and success/abort outcome are unchanged by the 4.0 removal. (iii) Newly reachable warnings on bootstrapped plain fits: the aggregators' own `UserWarning`s (Stage-1/Stage-2 singular or rank-deficient fallbacks) and the per-target bootstrap-stats `RuntimeWarning`s ('Dropping n/N non-finite bootstrap samples in ... event study (h=...)', 'Too few valid bootstrap samples', 'Bootstrap SE is non-finite or zero'). (iv) Pre-4.0 pickle limitation: a 3.9-3.13 bootstrapped pickle fitted with `fit(aggregate=..., balance_e=k)` stores a balance_e-RESTRICTED surface and records no fit-time `balance_e`, so the relay returns that surface as stored - re-fit under 4.0 for the unrestricted one; one fitted plain stores none and the relay raises the refit `ValueError`. (v) The relay reads the MUTABLE public `event_study_effects`/`group_effects` fields (a view by definition - the StackedDiD precedent); only alpha/anticipation/survey metadata come from kit snapshots, so the recompute route's 'kit snapshots only' invariant does not hold for the relayed dict. (vi) Downstream: `summary()` prints the ES/group tables and `to_dataframe(level='event_study'|'group')` returns frames on bootstrapped plain fits; `DiagnosticReport`/`BusinessReport` run `heterogeneity` (on the relayed GROUP percentile dict - `_collect_effect_scalars` reads `group_effects` first) and `parallel_trends` (`pretrends=True`) on the relayed surface instead of skipping with `NotImplementedError` (no sensitivity check applies); `plot_event_study(results)` plots them natively. Replicate-weight designs never meet the always-on surfaces (replicate + `n_bootstrap > 0` is rejected before them), so the fit-time replicate stack is always `[overall]`-only - the degenerate-design overall-row migration delta in (e) stands. +- **Note (bootstrapped-fit relay, 4.0):** (i) RELAY: a fit with `n_bootstrap > 0` ALWAYS computes the event-study (`_stage2_event_study`) AND group (`_stage2_group`) percentile surfaces at fit (`balance_e=None` - exactly what the 3.x `fit(aggregate='all')` computed; percentile se/p/CI; `event_study_vcov`/`event_study_vcov_index`/`event_study_df` None), stores them in `event_study_effects`/`group_effects`, and `aggregate('event_study')`/`aggregate('group')` RELAY them (alpha/anticipation/survey metadata from the kit snapshot; per-row `df` column NaN on both levels - the group relay passes `group_df=None` so no analytical df leaks beside percentile inference; `df_survey` scalar from the carrier's survey metadata; `vcov`/`vcov_index` None; reference rows via the `is_reference` key). `aggregate('event_study', balance_e=k)` on a bootstrapped fit FAILS CLOSED (`NotImplementedError`: the stored surface is unrestricted and the per-level GMM scores are not retained; re-fit with `n_bootstrap=0`). Analytic fits are unchanged (kit recompute; legacy fields None). The gate is `bootstrap_results is not None`: `None` (n_bootstrap == 0, OR a bootstrapped fit that ended analytical) means the analytical kit RECOMPUTE, the pre-4.0 behaviour. (ii) FAIL-SOFT materialization + storage rule: the surfaces are built inside one guard - any exception leaves them None (the `_es_vcov_ts`/`_es_vcov_index_ts` locals reset too) and the bootstrap runs overall-only (the pre-4.0 plain-fit path); a `_run_bootstrap` attempt that raises (anything but `NotImplementedError`, re-raised) OR RETURNS `None` WITH surfaces present - `_run_bootstrap` returns `None` without an exception when `_compute_cluster_S_scores` hits `_TS_LSMRUnconverged` in the ES or group stage, since the LSMR fallback certifies each RHS column separately and the overall column can succeed while an ES/group column does not - clears them and RETRIES ONCE overall-only (`original_event_study=None, original_group=None, aggregate=None`; `np.random.default_rng(self.seed)` is rebuilt on entry, so a seeded retry reproduces attempt 1's weight draws and the overall quintet equals a plain fit's). A retry failure (a `None` return included) keeps the pre-4.0 catch-all 'Bootstrap failed' warning + analytical overall. STORAGE RULE: the legacy fields hold a surface IFF `bootstrap_results is not None`; when the fit ends analytical they are cleared and `aggregate()` RECOMPUTES analytically. The fail-soft `UserWarning` fires only when the fit DOES carry bootstrap inference and a surface was dropped: 'TwoStageDiD: the event-study/group surfaces could not be materialized on this bootstrapped fit ({e}); the bootstrap covered the overall ATT only, so aggregate('event_study') / aggregate('group') are unavailable here - re-fit with n_bootstrap=0 for the analytical post-fit recompute.'; the relay then raises the shared absent-surface `ValueError` ('This bootstrapped TwoStageDiDResults carries no stored event_study surface: either it was fitted before 4.0 (a bootstrapped plain fit built none) or the surface could not be materialized at fit (see the fit-time warning). Re-fit with diff-diff >= 4.0 for the relayed surface, or with n_bootstrap=0 for the analytical post-fit recompute.'). Net invariant: a bootstrapped plain fit's overall quintet and success/abort outcome are unchanged by the 4.0 removal. (iii) Newly reachable warnings on bootstrapped plain fits: the aggregators' own `UserWarning`s (Stage-1/Stage-2 singular or rank-deficient fallbacks) and the per-target bootstrap-stats `RuntimeWarning`s ('Dropping n/N non-finite bootstrap samples in ... event study (h=...)', 'Too few valid bootstrap samples', 'Bootstrap SE is non-finite or zero'). (iv) Pre-4.0 pickle limitation: a 3.9-3.13 bootstrapped pickle fitted with `fit(aggregate=..., balance_e=k)` stores a balance_e-RESTRICTED surface and records no fit-time `balance_e`, so the relay returns that surface as stored - re-fit under 4.0 for the unrestricted one; one fitted plain stores none and the relay raises the refit `ValueError`. (v) The relay reads the MUTABLE public `event_study_effects`/`group_effects` fields (a view by definition - the StackedDiD precedent); only alpha/anticipation/survey metadata come from kit snapshots, so the recompute route's 'kit snapshots only' invariant does not hold for the relayed dict. (vi) Downstream: `summary()` prints the ES/group tables and `to_dataframe(level='event_study'|'group')` returns frames on bootstrapped plain fits; `DiagnosticReport`/`BusinessReport` run `heterogeneity` (on the relayed GROUP percentile dict - `_collect_effect_scalars` reads `group_effects` first) and `parallel_trends` (`pretrends=True`) on the relayed surface instead of skipping with `NotImplementedError` (no sensitivity check applies); `plot_event_study(results)` plots them natively. Replicate-weight designs never meet the always-on surfaces (replicate + `n_bootstrap > 0` is rejected before them), so the fit-time replicate stack is always `[overall]`-only - the degenerate-design overall-row migration delta in (e) stands. +- **Note (reference-row marking, 4.0, [M-093] item 8a):** the `event_study_effects` reference row (`e = -1 - anticipation`) carries `"is_reference": True` and no `n_obs` key (3.x: the `n_obs == 0` sentinel, retired); the all-filtered horizons keep their GENUINE `effect = NaN, n_obs = 0` rows, which `summary()` now prints as NaN rows (previously mislabeled `[ref: h]`); a REFERENCE-ONLY surface (`balance_e` leaving no cohort, or the `ref_only` return with no Prop-5 rows - Prop-5 rows carry a positive `n_obs`, so an all-Prop-5 surface keeps `n_kind="obs"`) carries `n_kind=None` in the container (no row records a count). The bootstrap loop skips the marked row; 3.x pickles are migrated on load (`__setstate__`). The native `to_dataframe(level='event_study')` frame renders the reference row's `n_obs` as NaN (float column) until the 8b schema rewire. - **Note:** `anticipation` is validated at construction (non-negative integer; `bool` rejected) via the shared `utils.validate_anticipation`, and re-checked on the fit path (direct-mutation defense) — see the family-wide adoption note (ledger row [M-144]) in the TripleDifference staggered-mode section. --- @@ -2584,6 +2586,7 @@ The pooled estimator is `DID^{CBWSDID}_e = Σ_a (N^D_a/N^D_Ω)(Δ̄^D_{a,e} − - **Note:** v1 scope — only `balance="entropy"` with `weighting="aggregate"`. `balance` + `population`/`sample_share` and `balance` + `survey_design=` raise `NotImplementedError`; matching-based balancing and the repeated `0→1/1→0` episode extension are out of scope. - **Note (post-fit aggregate() - row M-024):** `fit(aggregate=)` was deprecated in 3.9 and REMOVED at 4.0 (PR 3b of the cut; its `group`/`all`/unknown ValueErrors died with it - `aggregate('group')` already fails closed in the mixin - and `population`/`survey_design`/`covariates` are positional slots 6/7/8) and the event-study surface is ALWAYS materialized at fit — the pooled regression always included the event-time interactions, so only the extraction and, under `hc2_bm`, the per-event Bell-McCaffrey contrast DOFs were fit-time-gated; both gates are gone (the param had been behaviorally inert since 3.9). `aggregate('event_study')` returns the unified `EventStudyResults` container (a pure view; no recomputation, so every stored inference mode — survey TSL, replicate refit, hc2_bm fail-close — relays faithfully); `aggregate('simple')` relays `overall_att/se/t/p/CI` bit-exactly with `target="att"` (the scalar is the equally-weighted post-period average, NOT the per-event-time trimmed aggregate ATT `θ^e_κ` — `describe_target_parameter` remains the estimand's prose source of truth), `n = n_treated_units` with `n_kind="units"` (TREATED-only scope: under dynamic clean-control definitions a later-treated unit is treated in its own sub-experiment and a clean control in earlier ones, so the treated and control unit sets OVERLAP and no disjoint total exists — cross-container `n` comparisons vs CallawaySantAnna's treated+control convention are out of contract), and `df = inference_df` (the stored overall inference's exact provenance). Consequences of the always-computed surface: plain-fit `summary()` renders the event-study table, `to_dataframe('event_study')` is total on 3.9+ fits, `plot_event_study` renders plain fits, DiagnosticReport's `heterogeneity` (always) and `parallel_trends` (`kappa_pre >= 2`) checks run on plain fits, and the `hc2_bm` overall-ATT contrast now shares the m>1 batch of `_compute_cr2_bm_contrast_dof`, whose degeneracy guard is batch-relative — the noise-floor carve-out on the `Wald_test(test="HTZ")$df_denom` parity claim in the variance-families bullets above widens quantitatively (unchanged behavior when tripped: fail-closed NaN inference plus linalg's loud noise-floor UserWarning; the absolute floor already made this reachable at m=1). - **Note (container admission - rows M-024/M-093):** `compute_honest_did` and `compute_pretrends_power` admit Stacked-sourced `aggregate('event_study')` containers. Provenance: `base_period` is the class-level constant `"universal"` (every sub-experiment normalizes against the single omitted reference `e = -1 - anticipation`) and `reference_event_times` is the singleton `(-1 - anticipation,)`. Requirements: `kappa_pre >= 2` (the default `kappa_pre=1` grid minus the reference has ZERO estimated pre-periods — both consumers reject), and for HonestDiD a non-singular FULL retained event-study covariance — the validated sub-block spans the retained pre AND post rows (`allow_singular=False` per Rambachan-Roth's eigenvalues-bounded-away-from-zero assumption on the full Σ — keep `kappa_pre` small relative to the cluster count; PreTrendsPower keeps its documented singular support). The container's scalar `df_survey` is None on analytical fits — the generic resolver's fallback reads `df_inference`, a name StackedDiD does not carry (`inference_df` is the overall POST-AVERAGE contrast df, not a design-level cluster df, and misusing it for FLCI critical values would be worse than the asymptotic default) — so honest FLCI critical values are NORMAL-THEORY on analytical Stacked fits, diverging from bare-`cluster=` CS fits (which thread `G-1`); survey fits thread the survey df and replicate-undefined designs the fail-closed `0.0` sentinel (NaN FLCI CIs; the identified-set `lb`/`ub` bounds are df-independent and stay finite). Withheld-inference rows (finite `se`, non-finite `p_value` — the hc2_bm BM-DOF fail-close and replicate-undefined designs both emit this shape) are ADMITTED with a source-scoped warning in BOTH consumers: bounds/power consume only the point estimates and covariance, which remain valid; the warning is shape-descriptive because containers are publicly constructible. A rank-dropped interior event-time column (NaN-se row) fails closed via honest's consecutive-grid ValueError with a Stacked-specific remedy (no `balance_e` — Stacked's aggregate() has no such level). REFERENCE-SUPPORT GUARD: `fit()` raises when either omitted-reference cell (treated / control) carries no POSITIVE COMPOSED WLS WEIGHT — Equation 3 is Q-weighted, so raw row presence is not support: a gapped panel can empty the cell outright (IC1 checks only min/max periods), aggregate Q-weights zero a sub-experiment's controls at any event time where that sub-experiment has no treated rows, and survey pweights can zero a cell — without the guard the design rank-drops and QR pivoting silently re-normalizes against an arbitrary horizon while the surface certifies `delta_0 = 0` at the reference. Partial support (some cohorts' own ref period absent while the POOLED omitted category is populated) still fits: the coefficient-vector normalization is real; the per-cohort baseline raggedness is estimand composition, the documented ragged-window behavior. +- **Note (reference-row marking, 4.0, [M-093] item 8a):** the always-materialized `event_study_effects` reference row (`e = -1 - anticipation`) carries `"is_reference": True` and no `n_obs` key (3.x: the `n_obs == 0` sentinel, retired); `summary()` keys its `[ref: h]` line on the marker; `plot_event_study(results)` and `DiagnosticReport` locate the reference by the key; 3.x pickles are migrated on load (`__setstate__`). The native `to_dataframe(level='event_study')` frame renders the reference row's `n_obs` as NaN (float column) until the 8b schema rewire. *Covariate-balancing edge cases:* - Infeasible cohort (treated covariate mean outside the clean-control hull → entropy balancing cannot match the moments): **fail-closed** `ValueError` naming the cohort and worst covariate — NOT silently dropped (dropping a cohort would shift the estimand to an overlap-trimmed ATT, Ustyuzhanin 2026 §3.1). @@ -6973,7 +6976,9 @@ should be a deliberate user choice. non-reference container row previously drew a spurious `(0, 0)` bar there and now draws none. The one retention: on the raw `event_study_effects` dict route, an auto-inferred reference row (`effect` exactly 0, `se` 0) keeps its degenerate constraint bar per the - "retained for auto-inferred" rule above - the `effect == 0` conjunct stops the `-1` + "retained for auto-inferred" rule above (the reference is resolved from the + `reference_period` attribute or the row's `is_reference` key - 4.0, [M-093] item 8a - never + from a count) - the `effect == 0` conjunct stops the `-1` reference fallback from retaining a genuinely estimated zero-SE row (a degenerate estimated row with effect exactly 0 and se 0 at the fallback position retains today's bar - today's behavior preserved, not a new defect). The dCDH route's synthesized @@ -6984,12 +6989,13 @@ should be a deliberate user choice. - **Note (`plot_dose_response` zero-SE gate and band labeling):** the raw-DataFrame `se` branch adopts the same gate: rows with non-positive or non-finite `se` have their reconstructed band masked to NaN with a `UserWarning` (previously a zero-SE row drew a finite zero-width band); the plotly renderer filters non-finite-CI rows out of the `toself` band polygon and bridges the gap (the `plot_group_effects` convention above; CI values only - the dose axis may be non-numeric), and both renderers suppress an all-masked band entirely (no stray legend entry). Band LABELING states the confidence level only where it is knowable: the requested `alpha` on the DataFrame-`se` branch (validated strictly inside (0, 1)), `results.alpha` on `results=` input, and the level-free "CI" for bare-curve and explicit-CI input. Alpha semantics deliberately DIVERGE from `plot_group_effects`: that surface recomputes `effect +/- z*SE` at a requested alpha, while `plot_dose_response` warns-and-ignores an explicit `alpha` on non-`se` input, because stored/explicit-CI dose-response inputs cannot be recomputed. `plot_honest_event_study`'s raw (non-container) routes mirror the container's retained-row semantics: zero/non-finite-SE rows are excluded up front (explicitly requesting one raises), the reference row - now - auto-inferred on raw routes from a `reference_period` attribute or HonestDiD's own - constraint signature, never a bare `-1` fallback - is kept as a normalization anchor, + auto-inferred on raw routes from a `reference_period` attribute or the producer's + `is_reference` key (4.0, [M-093] item 8a; the former zero-count constraint signature is + retired), never a bare `-1` fallback - is kept as a normalization anchor, and an all-undefined surface raises instead of rendering a blank figure. The reference exemption (and honest-bound suppression) applies only to a VERIFIED anchor, never a bare `reference_period=` label: on dict surfaces the labeled row must carry the producers' - full constraint signature (zero `n_groups`/`n_obs` count, `effect` exactly 0.0, no + full constraint signature (the `is_reference` key, `effect` exactly 0.0, no defined SE - NaN for CS, 0.0 for the Imputation/TwoStage/Stacked-style markers); on the container route only `is_reference`-marked rows do (the removed MultiPeriodDiDResults dict surface, where only the result's own `reference_period` metadata verified, went at 4.0). An estimated zero-SE row - even one with @@ -7128,7 +7134,7 @@ where `D_it` is the treatment indicator (1 if unit `i` is treated by time `t`) a ### Event-study mode (Wave C, `event_study=True`) -Replaces the aggregate spec with the per-event-time × ring decomposition from Butts Section 5 / Table 2. Direct effects `tau_k` and per-(ring, event-time) spillover effects `delta_jk` are emitted in `att_dynamic` and a MultiIndex `spillover_effects`. A TwoStageDiD-compatible `event_study_effects: Dict[int, Dict]` alias (mirroring `two_stage_aggregation.py::_stage2_event_study` row schema with `conf_int = (low, high)` tuple) is also emitted for `plot_event_study` consumption — `_extract_plot_data` prefers the new `reference_period` attribute over the legacy `n_obs==0` heuristic. `DiagnosticReport` routing is now wired (see the DiagnosticReport applicability Note below). +Replaces the aggregate spec with the per-event-time × ring decomposition from Butts Section 5 / Table 2. Direct effects `tau_k` and per-(ring, event-time) spillover effects `delta_jk` are emitted in `att_dynamic` and a MultiIndex `spillover_effects`. A TwoStageDiD-compatible `event_study_effects: Dict[int, Dict]` alias (mirroring `two_stage_aggregation.py::_stage2_event_study` row schema with `conf_int = (low, high)` tuple) is also emitted for `plot_event_study` consumption — `_extract_plot_data` reads the `reference_period` attribute, else the row's `is_reference` key (4.0, [M-093] item 8a; no count heuristic). `DiagnosticReport` routing is now wired (see the DiagnosticReport applicability Note below). - **Note (DiagnosticReport applicability):** `SpilloverDiDResults` is registered in `DiagnosticReport`'s `_APPLICABILITY` / `_PT_METHOD` tables with applicable checks {`parallel_trends` (method `event_study`, on the per-event-time DIRECT-effect dynamics — populated when `event_study=True`; Bonferroni fallback across pre-period coefs since there is no `event_study_vcov`), `design_effect` (instance-gated on `survey_metadata`), `heterogeneity` (reads `event_study_effects`)}. This mirrors the `TwoStageDiD` set **minus `bacon`**. `bacon` is **deliberately excluded**: SpilloverDiD identifies the direct effect off FAR-AWAY control observations (`d_it > d_bar`, Assumption 5), not off the TWFE 2×2 comparisons a Goodman-Bacon decomposition enumerates; running the Bacon decomposition on the raw binary treatment would ignore the ring/distance structure and pool spillover-contaminated in-ring units into the control group — the exact SUTVA violation this estimator handles (same rationale that excludes `bacon` for SyntheticControl / TROP / ContinuousDiD). `pretrends_power` / `sensitivity` / `epv` / `estimator_native` are not applicable (no `compute_pretrends_power` adapter, no HonestDiD adapter, no `epv_diagnostics`, not an SDiD/TROP/SCM native path). @@ -7136,7 +7142,7 @@ Replaces the aggregate spec with the per-event-time × ring decomposition from B **Note (endpoint binning, divergence from TwoStageDiD):** `horizon_max` bins event-times outside `[-H, +H]` into endpoint pools (`k ≤ -H` aggregated into one pre-bin dummy; `k ≥ +H` into one post-bin dummy). No observations are dropped. This **diverges intentionally** from `TwoStageDiD`'s `horizon_max` semantic, which **filters** rows with `|K_it| > H` out of the stage-2 sample. SpilloverDiD's bin-into-endpoint behavior honors the no-silent-data-drop policy; the divergence is documented on both estimators' docstrings to prevent future unification. With `horizon_max=None`, the helper auto-detects the event-time bin set from the observed K values (no binning). -**Note (reference period `-1 - anticipation`):** The reference period (the event-time dummy dropped to anchor the level interpretation) is `ref_period = -1 - anticipation`. With `anticipation=0`, `ref_period = -1` (standard event-study convention; coefficients are relative to one period before treatment). With `anticipation > 0`, the reference period shifts to `-1 - anticipation` so the "pre-treatment" anchor sits BEFORE the anticipation window. Mirrors `TwoStageDiD`'s convention at `two_stage_aggregation.py` (`ref_period = -1 - anticipation`). The reference row appears in `att_dynamic` and `event_study_effects` with `coef = 0.0`, `se = 0.0`, `n_obs = 0`, `conf_int = (0.0, 0.0)` (TwoStageDiD parity, `two_stage_aggregation.py::_stage2_event_study`). When `horizon_max` is set and `ref_period < -horizon_max` (i.e., `anticipation > horizon_max - 1`), the fit raises `ValueError` — silently floor-shifting the reference to `-horizon_max` would change identification (rejected per `feedback_no_silent_failures`). +**Note (reference period `-1 - anticipation`):** The reference period (the event-time dummy dropped to anchor the level interpretation) is `ref_period = -1 - anticipation`. With `anticipation=0`, `ref_period = -1` (standard event-study convention; coefficients are relative to one period before treatment). With `anticipation > 0`, the reference period shifts to `-1 - anticipation` so the "pre-treatment" anchor sits BEFORE the anticipation window. Mirrors `TwoStageDiD`'s convention at `two_stage_aggregation.py` (`ref_period = -1 - anticipation`). The reference row appears in `att_dynamic` and `event_study_effects` with `coef = 0.0`, `se = 0.0`, `conf_int = (0.0, 0.0)` (TwoStageDiD parity, `two_stage_aggregation.py::_stage2_event_study`) and is marked EXPLICITLY: `is_reference = True`, with `n_obs = NaN` in the `att_dynamic` / `spillover_effects` FRAMES and NO `n_obs` key in the `event_study_effects` DICT (4.0, [M-093] item 8a; the 3.x `n_obs = 0` sentinel is retired - a zero `n_obs` is now always a genuine empty cell). When `horizon_max` is set and `ref_period < -horizon_max` (i.e., `anticipation > horizon_max - 1`), the fit raises `ValueError` — silently floor-shifting the reference to `-horizon_max` would change identification (rejected per `feedback_no_silent_failures`). **Note (post-finite_mask sample):** `att_dynamic["n_obs"]`, `event_study_effects[k]["n_obs"]`, AND the scalar `att` share weights all reflect the POST-`finite_mask` stage-2 estimation sample — not the pre-mask design built by `_build_event_study_design`. On warn-and-drop fits (baseline-treated units without Omega_0 rows are excluded via `finite_mask`), counts and weights are recomputed from `X_2_fit` so the reported metadata matches the actual stage-2 sample that `solve_ols` sees. @@ -7154,7 +7160,7 @@ share_k = sum_{i: K_direct_i = k AND treated_i = 1} w_i ``` where `w_i` are the Hájek-normalized survey weights. Using raw `n_treated_at_k` shares on weighted WLS horizon coefficients would mix unweighted aggregation with weighted horizons and target the wrong estimand. The same survey-weight totals enter both `att` and `Var(att) = w' V_subset w`, so the lincom variance stays consistent with the point estimate. On the no-survey path, `share_k = n_treated_at_k` (sample-share rule) — backward-compatible with Wave C. -**Note (rectangular schema):** `att_dynamic` and the MultiIndex `spillover_effects` are emitted **rectangularly** across the full event-time × ring grid implied by `horizon_max`. Empty cells (no rows contribute to the corresponding stage-2 design column — pre-filtered to keep `solve_ols` rank warnings quiet) emit `coef = NaN`, `se = NaN`, `n_obs = 0`. This includes negative-k spillover cells (structurally empty: `K_spill ≥ 0`) and outer-ring cells when the panel has no units in that ring band. The `(ring_label, ref_period)` cells are also emitted in `spillover_effects` (one per ring) with `coef = 0.0, se = 0.0, n_obs = 0` to mirror the direct-effect reference row anchor — without this, consumers iterating `[-H, ..., +H]` would hit a missing `(ring, ref_period)` slice. Downstream consumers can iterate the rectangular grid without `KeyError` on missing cells. +**Note (rectangular schema):** `att_dynamic` and the MultiIndex `spillover_effects` are emitted **rectangularly** across the full event-time × ring grid implied by `horizon_max`. Empty cells (no rows contribute to the corresponding stage-2 design column — pre-filtered to keep `solve_ols` rank warnings quiet) emit `coef = NaN`, `se = NaN`, `n_obs = 0`. This includes negative-k spillover cells (structurally empty: `K_spill ≥ 0`) and outer-ring cells when the panel has no units in that ring band. The `(ring_label, ref_period)` cells are also emitted in `spillover_effects` (one per ring) with `coef = 0.0, se = 0.0, is_reference = True, n_obs = NaN` (4.0, [M-093] item 8a) to mirror the direct-effect reference row anchor — without this, consumers iterating `[-H, ..., +H]` would hit a missing `(ring, ref_period)` slice. Downstream consumers can iterate the rectangular grid without `KeyError` on missing cells. **Reduce-to-aggregate equivalence:** Under a **constant-tau DGP** with `horizon_max=None`, the sample-share-weighted scalar `att` reproduces Wave B's aggregate `tau_total` (bit-identical at machine precision in the deterministic limit; small MC noise on realized panels). This is the canonical equivalence path. Note: `horizon_max=0` is **not supported** under `event_study=True` (rejected at validation with a clear remediation message): the single bin `k=0` leaves no event-time pair to anchor the reference period `ref_period = -1 - anticipation` against. Users wanting a single aggregate direct effect should use `event_study=False` (the Wave B static spec); event-study mode requires `horizon_max>=1` or `horizon_max=None`. diff --git a/docs/migration-4.0.md b/docs/migration-4.0.md index 1d1145912..003a24986 100644 --- a/docs/migration-4.0.md +++ b/docs/migration-4.0.md @@ -219,6 +219,21 @@ for heterogeneity. StackedDiD and SunAbraham always populate the raw surface (nothing to derive), and ChaisemartinDHaultfoeuille's pre-period checks read `placebo_event_study` directly. +**Reference rows ([M-093], item 8a).** Since 4.0 every native event-study dict +that MATERIALIZES a normalization row (StackedDiD, ImputationDiD, TwoStageDiD, +SpilloverDiD, the CallawaySantAnna/DMLDiD universal-base carrier) marks it with +`"is_reference": True` and carries no observation/group count there (3.x: +`n_obs == 0` / `n_groups == 0`); SpilloverDiD's event-study +`att_dynamic`/`spillover_effects` frames gain a boolean `is_reference` column +(`n_obs` NaN on those rows); the native `to_dataframe(level="event_study")` +frames of StackedDiD / bootstrapped ImputationDiD / TwoStageDiD show +`n_obs = NaN` on the reference row (the column is float) until the 4.0 schema +rewire replaces them. SunAbraham omits its reference row (`reference_period`) +and EfficientDiD's PT-Post anchor is an estimated cell named by +`reference_period` - both unchanged. Code that located the reference by a zero +count should read the key, or the container's `is_reference` column. 3.x +pickles are migrated on load. + ```{warning} **Bootstrapped fits: every adopter has a post-fit route - two mechanisms.** On `CallawaySantAnna`, `DMLDiD` (which inherits the CS replay channel), and `EfficientDiD`, the post-fit recompute levels REPLAY the @@ -656,4 +671,4 @@ does not mean no action is required, so read the `Fix` cell. | M-046 | renames-robust-drop | `diff_diff:TripleDifference[robust]` | — | Drop it. `TripleDifference` accepts only `vcov_type="hc1"`, so `robust=` never changed its inference - do NOT write `vcov_type="classical"`, which raises. | | M-047 | renames-robust-drop | `diff_diff:HeterogeneousAdoptionDiD[robust]` | — | HAD's legacy default is non-robust: `robust=True` becomes `vcov_type="hc1"`; drop `robust=False`. | | M-115 | renames-robust-drop | `diff_diff:LinearRegression[robust]` | — | Drop `robust=True` (already the `hc1` default); `robust=False` becomes `vcov_type="classical"`. | -| M-093 | results-contract | `diff_diff:CallawaySantAnnaResults.event_study_effects` | `diff_diff:EventStudyResults` | The sentinel is retired - read the unified `EventStudyResults` surface instead of the raw field. | +| M-093 | results-contract | `diff_diff:CallawaySantAnnaResults.event_study_effects` | `diff_diff:EventStudyResults` | The zero-count sentinel is retired (8a): read `is_reference` on the row, or the unified `EventStudyResults` surface; the per-class `to_dataframe(level='event_study')` schema lands in 8b. | diff --git a/docs/v4-deprecations.yaml b/docs/v4-deprecations.yaml index 8da38d381..27f2b3555 100644 --- a/docs/v4-deprecations.yaml +++ b/docs/v4-deprecations.yaml @@ -1043,8 +1043,9 @@ rows: removed_in: null status: planned phase: 5 + test_ref: tests/test_v4_sentinel_retirement.py code_refs: [diff_diff/staggered_results.py, diff_diff/sun_abraham.py, diff_diff/imputation_results.py, diff_diff/two_stage_results.py, diff_diff/stacked_did_results.py, diff_diff/efficient_did_results.py, diff_diff/continuous_did_results.py, diff_diff/wooldridge_results.py, diff_diff/chaisemartin_dhaultfoeuille_results.py, diff_diff/lpdid_results.py, diff_diff/triple_diff_results.py, diff_diff/results.py, diff_diff/had.py, diff_diff/visualization/_event_study.py, diff_diff/honest_did.py, diff_diff/pretrends.py] - notes: "4.0 sentinel retirement + schema enforcement (spec section 5): the n_groups==0 / n_obs==0 reference-row sentinels retire; every estimator's to_dataframe(level='event_study') emits the [M-092] column schema; the plotter / HonestDiD / PreTrendsPower consume the unified surface. Thirteen ES-carrying source modules enumerated file-by-file (results.py covers MultiPeriodDiD + SpilloverDiD) plus the plotter. behavior-at-done requires test_ref. PARTIAL PRE-CUT DELIVERY (2(b) PR-1, with M-026): the consumer half shipped for CS-SOURCED containers - compute_honest_did, compute_pretrends_power (both with SOURCE-SCOPED admission: source == CallawaySantAnnaResults only, rejecting dCDH l1 containers BY DESIGN since their placebo semantics need honest_did's native branch, and every other producer pending its own aggregate() migration) and plot_event_study / plot_honest_event_study (no source scoping - plotting is label-faithful). Admission widening is each later shim PR's methodology decision, not automatic. SECOND PRE-CUT AMENDMENT (2(b) PR-2, with M-024): admission widened to STACKED-SOURCED containers in compute_honest_did + compute_pretrends_power (source in {CallawaySantAnnaResults, StackedDiDResults}; kappa_pre >= 2 required for estimated pre-periods; honest additionally needs a non-singular full retained event-study covariance (pre+post sub-block); withheld-inference rows admitted with a source-scoped warning; the seven producer-derived guard messages now derive the producer from surface.source). Source scoping continues for all other producers. THIRD PRE-CUT AMENDMENT (2(b) PR-3a, with M-023): admission NOT widened to EfficientDiD-sourced containers BY DESIGN (the dCDH precedent) - the PRIMARY ground is the absent joint event-study covariance (container vcov=None; per-row df was all-NaN at that amendment and is finite on analytical survey fits since the 3.12 M-092 completion - the rejection ground is the missing joint covariance, unchanged); reference semantics are regime-dependent (PT-All: no reference row, universal first-period baseline; PT-Post: a materialized mechanical zero anchor at e=-1-anticipation when estimated, marked via the membership-gated reference_period property - never fabricated when absent); both terminal TypeErrors now state the EfficientDiD rejection explicitly; source scoping continues to reject. FOURTH PRE-CUT AMENDMENT (2(b) PR-3b, with M-021/M-022): admission NOT widened by the Imputation/TwoStage migrations - ImputationDiD rejected BY DESIGN (no joint ES covariance; per-horizon Theorem-3 conservative SEs only); TwoStageDiD DEFERRED despite carrying the joint Gardner-GMM covariance on analytical fits (M-092), because its pre-period coefficients are stage-1 residual means (the reference horizon is dropped from the no-intercept Stage-2 design; the zero anchor row is appended mechanically), not contrasts against the advertised reference, while HonestDiD's Delta^RM/Delta^SD arithmetic hard-codes the delta_0=0 normalization into its boundary/bridge constraints - admission awaits a normalization derivation (DEFERRED.md paper-gated row); both terminal TypeErrors state the per-producer grounds. FIFTH PRE-CUT AMENDMENT (2(b) PR-3c, with M-025): admission NOT widened by the ContinuousDiD migration - rejected BY DESIGN on two independent grounds: no joint event-study covariance exists (per-bin IF SEs only), and the binarized bins carry NO reference-period normalization at all (no reference row exists in the surface; each bin is a raw reweighted binarized ATT level), so HonestDiD's delta_0=0 arithmetic has no anchor even in principle without new methodology; both terminal TypeErrors name the ContinuousDiD rejection alongside EfficientDiD/Imputation. Amended pre-cut (2(b) PR-4, with M-027): HeterogeneousAdoptionDiD admission NOT widened by the HAD migration - DEFERRED, not by-design (the TwoStage class): its event-study coefficients ARE reference-normalized (each horizon differences against the F-1 anchor) but the anchor row is omitted from the container (identically zero AND the WAS is not identified there - no reference row exists for the consumer grid) and no joint cross-horizon covariance exists (per-horizon independent sandwiches; the DEFERRED.md row); both terminal TypeErrors name the HAD deferral in that corrected wording. Source scoping continues to reject all other producers. SIXTH PRE-CUT AMENDMENT (Phase 3(a), with M-010): admission widened to TWFE-SOURCED CALENDAR containers in compute_honest_did + compute_pretrends_power via a dedicated calendar route that reconstructs the native MultiPeriodDiDResults-branch inputs from the container's post_periods partition provenance (single-reference gate; HonestDiD additionally GEOMETRY-SCOPED to chronological partitions - suffix post_periods + last-pre reference - with the pre-existing native non-suffix geometry limitation documented in the REGISTRY HonestDiD Note and the DEFERRED.md transform-or-reject row; PreTrendsPower plumbs reference_period through the gamma-unit helper, string-label degradation included); relative-scale admission unchanged. SEVENTH PRE-CUT AMENDMENT (DML PR-B1): admission widened to DMLDiD-SOURCED containers in compute_honest_did + compute_pretrends_power (source in {CallawaySantAnnaResults, StackedDiDResults, DMLDiDResults}): DMLDiDResults subclasses CallawaySantAnnaResults and its aggregate('event_study') container is built by the INHERITED CS kit channel, so it carries the same staggered ATT(g,t) payload, reference semantics, base_period provenance, and (bootstrapped) replay behavior as a CS container - the same-semantics ground the original CS admission rests on; the producer-derived guard messages treat DMLDiDResults like CS (both expose base_period). DMLDiD adds NO builder (M-092 untouched) and never populates fit-time event_study_effects (its sentinel-retirement obligation is discharged by the shared CS code). Source scoping continues to reject all other producers. EIGHTH PRE-CUT AMENDMENT (4.0 cut PRs 3b/3c, with M-020..M-027/M-117..M-120/M-139): the fit-time aggregate=/balance_e= keywords are gone, which fixes WHICH producers populate the legacy event_study_effects/group_effects fields - CallawaySantAnna and EfficientDiD: None on EVERY fit (analytic and bootstrapped; the post-fit replay is the only bootstrap route); ImputationDiD and TwoStageDiD: None on analytic fits, ES + group dicts (percentile se/p/CI, df cleared) on bootstrapped fits IFF bootstrap_results is not None, relayed by aggregate(); ContinuousDiD: None on analytic fits, the ES dict on bootstrapped fits IFF _run_bootstrap ran, relayed by aggregate(); StackedDiD and SunAbraham: always populated (unchanged). CS/EDiD to_dataframe(level='event_study'|'group') therefore raises on every fresh fit until this row's schema rewire lands; admission is unchanged by the removals. This row stays planned: its transition remains the 4.0 sentinel retirement + full-producer schema enforcement." + notes: "4.0 sentinel retirement + schema enforcement (spec section 5): the n_groups==0 / n_obs==0 reference-row sentinels retire; every estimator's to_dataframe(level='event_study') emits the [M-092] column schema; the plotter / HonestDiD / PreTrendsPower consume the unified surface. Thirteen ES-carrying source modules enumerated file-by-file (results.py covers MultiPeriodDiD + SpilloverDiD) plus the plotter. behavior-at-done requires test_ref. PARTIAL PRE-CUT DELIVERY (2(b) PR-1, with M-026): the consumer half shipped for CS-SOURCED containers - compute_honest_did, compute_pretrends_power (both with SOURCE-SCOPED admission: source == CallawaySantAnnaResults only, rejecting dCDH l1 containers BY DESIGN since their placebo semantics need honest_did's native branch, and every other producer pending its own aggregate() migration) and plot_event_study / plot_honest_event_study (no source scoping - plotting is label-faithful). Admission widening is each later shim PR's methodology decision, not automatic. SECOND PRE-CUT AMENDMENT (2(b) PR-2, with M-024): admission widened to STACKED-SOURCED containers in compute_honest_did + compute_pretrends_power (source in {CallawaySantAnnaResults, StackedDiDResults}; kappa_pre >= 2 required for estimated pre-periods; honest additionally needs a non-singular full retained event-study covariance (pre+post sub-block); withheld-inference rows admitted with a source-scoped warning; the seven producer-derived guard messages now derive the producer from surface.source). Source scoping continues for all other producers. THIRD PRE-CUT AMENDMENT (2(b) PR-3a, with M-023): admission NOT widened to EfficientDiD-sourced containers BY DESIGN (the dCDH precedent) - the PRIMARY ground is the absent joint event-study covariance (container vcov=None; per-row df was all-NaN at that amendment and is finite on analytical survey fits since the 3.12 M-092 completion - the rejection ground is the missing joint covariance, unchanged); reference semantics are regime-dependent (PT-All: no reference row, universal first-period baseline; PT-Post: a materialized mechanical zero anchor at e=-1-anticipation when estimated, marked via the membership-gated reference_period property - never fabricated when absent); both terminal TypeErrors now state the EfficientDiD rejection explicitly; source scoping continues to reject. FOURTH PRE-CUT AMENDMENT (2(b) PR-3b, with M-021/M-022): admission NOT widened by the Imputation/TwoStage migrations - ImputationDiD rejected BY DESIGN (no joint ES covariance; per-horizon Theorem-3 conservative SEs only); TwoStageDiD DEFERRED despite carrying the joint Gardner-GMM covariance on analytical fits (M-092), because its pre-period coefficients are stage-1 residual means (the reference horizon is dropped from the no-intercept Stage-2 design; the zero anchor row is appended mechanically), not contrasts against the advertised reference, while HonestDiD's Delta^RM/Delta^SD arithmetic hard-codes the delta_0=0 normalization into its boundary/bridge constraints - admission awaits a normalization derivation (DEFERRED.md paper-gated row); both terminal TypeErrors state the per-producer grounds. FIFTH PRE-CUT AMENDMENT (2(b) PR-3c, with M-025): admission NOT widened by the ContinuousDiD migration - rejected BY DESIGN on two independent grounds: no joint event-study covariance exists (per-bin IF SEs only), and the binarized bins carry NO reference-period normalization at all (no reference row exists in the surface; each bin is a raw reweighted binarized ATT level), so HonestDiD's delta_0=0 arithmetic has no anchor even in principle without new methodology; both terminal TypeErrors name the ContinuousDiD rejection alongside EfficientDiD/Imputation. Amended pre-cut (2(b) PR-4, with M-027): HeterogeneousAdoptionDiD admission NOT widened by the HAD migration - DEFERRED, not by-design (the TwoStage class): its event-study coefficients ARE reference-normalized (each horizon differences against the F-1 anchor) but the anchor row is omitted from the container (identically zero AND the WAS is not identified there - no reference row exists for the consumer grid) and no joint cross-horizon covariance exists (per-horizon independent sandwiches; the DEFERRED.md row); both terminal TypeErrors name the HAD deferral in that corrected wording. Source scoping continues to reject all other producers. SIXTH PRE-CUT AMENDMENT (Phase 3(a), with M-010): admission widened to TWFE-SOURCED CALENDAR containers in compute_honest_did + compute_pretrends_power via a dedicated calendar route that reconstructs the native MultiPeriodDiDResults-branch inputs from the container's post_periods partition provenance (single-reference gate; HonestDiD additionally GEOMETRY-SCOPED to chronological partitions - suffix post_periods + last-pre reference - with the pre-existing native non-suffix geometry limitation documented in the REGISTRY HonestDiD Note and the DEFERRED.md transform-or-reject row; PreTrendsPower plumbs reference_period through the gamma-unit helper, string-label degradation included); relative-scale admission unchanged. SEVENTH PRE-CUT AMENDMENT (DML PR-B1): admission widened to DMLDiD-SOURCED containers in compute_honest_did + compute_pretrends_power (source in {CallawaySantAnnaResults, StackedDiDResults, DMLDiDResults}): DMLDiDResults subclasses CallawaySantAnnaResults and its aggregate('event_study') container is built by the INHERITED CS kit channel, so it carries the same staggered ATT(g,t) payload, reference semantics, base_period provenance, and (bootstrapped) replay behavior as a CS container - the same-semantics ground the original CS admission rests on; the producer-derived guard messages treat DMLDiDResults like CS (both expose base_period). DMLDiD adds NO builder (M-092 untouched) and never populates fit-time event_study_effects (its sentinel-retirement obligation is discharged by the shared CS code). Source scoping continues to reject all other producers. EIGHTH PRE-CUT AMENDMENT (4.0 cut PRs 3b/3c, with M-020..M-027/M-117..M-120/M-139): the fit-time aggregate=/balance_e= keywords are gone, which fixes WHICH producers populate the legacy event_study_effects/group_effects fields - CallawaySantAnna and EfficientDiD: None on EVERY fit (analytic and bootstrapped; the post-fit replay is the only bootstrap route); ImputationDiD and TwoStageDiD: None on analytic fits, ES + group dicts (percentile se/p/CI, df cleared) on bootstrapped fits IFF bootstrap_results is not None, relayed by aggregate(); ContinuousDiD: None on analytic fits, the ES dict on bootstrapped fits IFF _run_bootstrap ran, relayed by aggregate(); StackedDiD and SunAbraham: always populated (unchanged). CS/EDiD to_dataframe(level='event_study'|'group') therefore raises on every fresh fit until this row's schema rewire lands; admission is unchanged by the removals. This row stays planned: its transition remains the 4.0 sentinel retirement + full-producer schema enforcement. NINTH AMENDMENT (4.0 cut item 8a, 2026-10-30): the SENTINEL half shipped - reference rows in every native event-study dict that materializes one carry is_reference: True and no count key (StackedDiD, ImputationDiD/TwoStageDiD, SpilloverDiD - whose event-study att_dynamic/spillover_effects frames gain a boolean is_reference column with NaN n_obs on the reference rows - and the CallawaySantAnna/DMLDiD aggregator carrier); SunAbraham still omits its row (reference_period) and EfficientDiD's PT-Post anchor stays an estimated cell named by reference_period; results_base._from_relative_dict reads the key (the zero-count heuristic is gone, a zero count is a genuine count, a keyed row with a finite nonzero effect raises); 3.x pickles are migrated by _deprecation.migrate_event_study_reference_sentinel in the carriers' __setstate__ (SpilloverDiDResults gained one; aggregate-mode frames untouched); plot_event_study / plot_honest_event_study / HonestDiD / PreTrendsPower / DiagnosticReport read the key on their raw-dict routes (dispatch, admission, rendering and exception types unchanged); the native Stacked/Imputation/TwoStage to_dataframe(level='event_study') reference row renders n_obs=NaN (float column) until 8b; TwoStageDiD summary() prints its genuine all-filtered horizons as NaN rows (no longer mislabeled [ref: h]); a reference-only TwoStageDiD surface carries n_kind=None. test_ref pins it. REMAINING for done (8b): the to_dataframe(level='event_study') schema rewire on every results class (incl. the dCDH native frame's estimand 'ref', n_obs 0 row)." # ---- Behavior policies (schema-tracked, spec-governed; no reality probe) - - id: M-080 diff --git a/docs/v4-design.md b/docs/v4-design.md index 5f6edc295..b7ddc508d 100644 --- a/docs/v4-design.md +++ b/docs/v4-design.md @@ -1239,7 +1239,12 @@ tests-first PR that flips no rows). `tests/test_v4_cluster_policy.py`, notebook 23 re-executed; 7c SHIPPED 2026-10-29: [M-154] SpilloverDiD df_convention, pinned by `tests/test_v4_spillover_df_convention.py`; item 7 complete.) -8. [M-093] sentinel retirement + event-study schema enforcement. +8. [M-093] sentinel retirement + event-study schema enforcement. (8a SHIPPED + 2026-10-30: the sentinel retirement - every materialized reference row carries + `is_reference: True` and no count key, the builder and the consumers' raw-dict + routes read the key, 3.x pickles migrate on load; pinned by + `tests/test_v4_sentinel_retirement.py`. 8b = the + `to_dataframe(level="event_study")` schema enforcement + the row flip.) 9. [M-090] docs reorganization: Bacon to the diagnostics family, catalogue counts, JOSS prose, REGISTRY archive framing for the removed classes. 10. Release: bump to 4.0.0 (closes the cut window), changelog compile, diff --git a/tests/_capture_v4_aggregate_oracles.py b/tests/_capture_v4_aggregate_oracles.py index 48630a68b..38949b197 100644 --- a/tests/_capture_v4_aggregate_oracles.py +++ b/tests/_capture_v4_aggregate_oracles.py @@ -132,18 +132,14 @@ def _ref_labels(r: Any) -> Optional[list]: the container under test (a surface that wrongly marks an estimated horizon as reference must fail, not skip). Where the post-fit route is closed on the capture tree (bootstrapped Imputation/TwoStage/ContinuousDiD) the - legacy zero-count sentinel rows (``n_obs``/``n_groups`` == 0) are the + ``is_reference``-keyed rows (the 4.0 marking, row M-093) are the reference set; a surface with neither yields ``None`` (no membership pin). """ try: es = _quiet(r.aggregate, "event_study") except NotImplementedError: legacy = r.event_study_effects or {} - refs = [ - _key(k) - for k, v in legacy.items() - if ("n_obs" in v or "n_groups" in v) and float(v.get("n_obs", v.get("n_groups"))) == 0.0 - ] + refs = [k_ for k_, v in ((_key(k), v) for k, v in legacy.items()) if v.get("is_reference")] return sorted(refs, key=float) labels = list(es.event_time) flags = np.asarray(es.is_reference, dtype=bool) diff --git a/tests/helpers/legacy_surface.py b/tests/helpers/legacy_surface.py index 2d66bbeb4..ce27b4b9c 100644 --- a/tests/helpers/legacy_surface.py +++ b/tests/helpers/legacy_surface.py @@ -11,13 +11,14 @@ SUBJECT is the native (legacy-pickle) route build a faithful stand-in: ``es_dict(results, balance_e=None)`` - ``aggregate("event_study")`` as the legacy ``event_study_effects`` mapping + ``aggregate("event_study")`` as the native ``event_study_effects`` mapping ``{event_time: {"effect", "se", "t_stat", "p_value", "conf_int", }}``. The count key follows the container's ``n_kind`` (``"groups"`` -> ``n_groups``, - ``"obs"`` -> ``n_obs``, ...); a NaN count becomes 0 (the legacy zero-count - reference sentinel); when the container has NO ``n_kind`` (ContinuousDiD) no - count key is emitted - ``_from_relative_dict`` rejects a zero-count row with a - finite nonzero effect. ``cband_conf_int`` is added where the band is finite. + ``"obs"`` -> ``n_obs``, ...) on estimated rows; a reference row carries + ``"is_reference": True`` and NO count key (the 4.0 marking, ledger row M-093 - + the zero-count sentinel is retired); a NaN count on a non-reference row emits + no count key; when the container has NO ``n_kind`` (ContinuousDiD) no count key + is emitted at all. ``cband_conf_int`` is added where the band is finite. ``group_dict(results)`` ``aggregate("group")`` as the legacy ``group_effects`` mapping (label keys; ``df_used`` from the ``df`` column). @@ -92,9 +93,12 @@ def es_dict_from_container(es: Any) -> Dict[Any, Dict[str, Any]]: "p_value": float(es.p_value[i]), "conf_int": (float(es.conf_int_lower[i]), float(es.conf_int_upper[i])), } - if key is not None: + if bool(es.is_reference[i]): + row["is_reference"] = True + elif key is not None: n = float(es.n[i]) - row[key] = 0 if math.isnan(n) else (int(n) if float(n).is_integer() else n) + if not math.isnan(n): + row[key] = int(n) if float(n).is_integer() else n if es.cband_lower is not None and es.cband_upper is not None: lo, hi = float(es.cband_lower[i]), float(es.cband_upper[i]) if math.isfinite(lo) and math.isfinite(hi): diff --git a/tests/test_diagnostic_report.py b/tests/test_diagnostic_report.py index 805f48a2c..e4be372ff 100644 --- a/tests/test_diagnostic_report.py +++ b/tests/test_diagnostic_report.py @@ -1150,8 +1150,9 @@ def test_sun_abraham_sensitivity_not_applicable(self): assert sensitivity["status"] == "not_applicable" def test_n_obs_zero_reference_marker_filtered(self): - """Stacked / TwoStage / Imputation reference markers use n_obs=0 - (not n_groups=0). ``_collect_pre_period_coefs`` must filter both.""" + """A GENUINE zero-count row (n_obs=0; the 3.x reference sentinel shape, + still a non-estimable horizon today) is excluded by + ``_collect_pre_period_coefs`` and never counted as undefined.""" import numpy as np from diff_diff.diagnostic_report import _collect_pre_period_coefs @@ -1166,13 +1167,14 @@ class StackedDiDResults: "effect": 0.0, "se": np.nan, "p_value": np.nan, - "n_obs": 0, # synthetic reference marker + "n_obs": 0, # a GENUINE zero-count (non-estimable) row, not a marker }, 0: {"effect": 1.5, "se": 0.2, "p_value": 0.0001, "n_obs": 50}, } - coefs, _ = _collect_pre_period_coefs(obj) + coefs, n_dropped = _collect_pre_period_coefs(obj) keys = [k for (k, _, _, _) in coefs] assert -1 not in keys, "n_obs==0 row must be filtered out" + assert n_dropped == 0, "a zero-count horizon is not an undefined pre-period" assert -2 in keys @@ -1182,7 +1184,7 @@ class TestReferenceMarkerAndNaNFiltering: Universal-base CS / SA / ImputationDiD / Stacked event-study output injects a synthetic reference-period row (``effect=0``, ``se=NaN``, - ``p_value=NaN``, ``n_groups=0``). Treating that row as valid + ``p_value=NaN``, ``is_reference``). Treating that row as valid pre-period evidence would inflate the Bonferroni denominator and collapse all-NaN fallbacks to a false-clean verdict. """ @@ -1204,7 +1206,7 @@ class CallawaySantAnnaResults: obj.n_treated = 40 obj.n_control = 160 obj.survey_metadata = None - # Two real pre-period rows + one universal-base reference marker (n_groups=0). + # Two real pre-period rows + one universal-base reference row (is_reference). obj.event_study_effects = { -3: {"effect": 0.1, "se": 0.3, "p_value": 0.74, "n_groups": 5}, -2: {"effect": -0.2, "se": 0.3, "p_value": 0.51, "n_groups": 5}, @@ -1213,7 +1215,7 @@ class CallawaySantAnnaResults: "se": np.nan, "p_value": np.nan, "conf_int": (np.nan, np.nan), - "n_groups": 0, + "is_reference": True, }, 0: {"effect": 1.5, "se": 0.2, "p_value": 0.0001, "n_groups": 5}, } @@ -1230,7 +1232,7 @@ def test_reference_marker_excluded_from_pt_collection(self): coefs, _ = _collect_pre_period_coefs(obj) keys = [k for (k, _, _, _) in coefs] assert -1 not in keys, ( - "Universal-base reference marker (n_groups=0) must not appear " + "Universal-base reference row (is_reference) must not appear " "as a valid pre-period coefficient" ) assert -3 in keys and -2 in keys @@ -1262,7 +1264,7 @@ class CallawaySantAnnaResults: "effect": 0.0, "se": np.nan, "p_value": np.nan, - "n_groups": 0, + "is_reference": True, }, 0: {"effect": 1.5, "se": 0.2, "p_value": 0.0001, "n_groups": 5}, } @@ -2519,11 +2521,11 @@ class CallawaySantAnnaResults: pass stub = CallawaySantAnnaResults() - # Event study: pre horizons (rel<0), reference marker (n_groups=0), + # Event study: pre horizons (rel<0), the reference row (is_reference), # non-finite row, and two valid post rows. stub.event_study_effects = { -2: {"effect": -3.0, "se": 0.2, "n_groups": 15}, - -1: {"effect": 0.0, "se": float("nan"), "n_groups": 0}, # reference marker + -1: {"effect": 0.0, "se": float("nan"), "is_reference": True}, # reference row 0: {"effect": 1.0, "se": 0.2, "n_groups": 15}, 1: {"effect": 2.0, "se": 0.2, "n_groups": 12}, 2: {"effect": float("nan"), "se": 0.2, "n_groups": 5}, # non-finite @@ -3124,7 +3126,7 @@ class _FakeCovBacked: "t_stat": np.nan, "p_value": np.nan, "conf_int": (np.nan, np.nan), - "n_obs": 0, + "is_reference": True, }, 0: { "effect": 0.9, @@ -3266,7 +3268,7 @@ class _FakeSingular: "t_stat": np.nan, "p_value": np.nan, "conf_int": (np.nan, np.nan), - "n_obs": 0, + "is_reference": True, }, 0: { "effect": 0.9, diff --git a/tests/test_event_study_consumers.py b/tests/test_event_study_consumers.py index 18d45c408..ea4617ca8 100644 --- a/tests/test_event_study_consumers.py +++ b/tests/test_event_study_consumers.py @@ -1699,7 +1699,7 @@ def test_bm_failure_fit_warns_in_both_consumers(self, monkeypatch): and np.isnan(v["conf_int"][0]) and np.isnan(v["conf_int"][1]) for h, v in eff.items() - if v["n_obs"] > 0 + if v.get("n_obs", 0) > 0 ) surf = res.aggregate("event_study") with pytest.warns(UserWarning, match="withheld/undefined"): @@ -2209,7 +2209,7 @@ def test_cs_dict_route_filters_and_keeps_signature_reference(self): nan = float("nan") original = SimpleNamespace( event_study_effects={ - -1: {"effect": 0.0, "se": nan, "n_groups": 0}, + -1: {"effect": 0.0, "se": nan, "is_reference": True}, 0: {"effect": 1.0, "se": 0.5, "n_groups": 3}, 1: {"effect": 1.2, "se": 0.0, "n_groups": 3}, } @@ -2256,7 +2256,7 @@ def test_reference_only_surface_raises_not_anchor_figure(self): nan = float("nan") original = SimpleNamespace( event_study_effects={ - -1: {"effect": 0.0, "se": nan, "n_groups": 0}, + -1: {"effect": 0.0, "se": nan, "is_reference": True}, 0: {"effect": 1.0, "se": 0.0, "n_groups": 3}, 1: {"effect": 1.2, "se": nan, "n_groups": 3}, } @@ -2275,7 +2275,7 @@ def test_explicit_reference_only_selection_raises(self): nan = float("nan") original = SimpleNamespace( event_study_effects={ - -1: {"effect": 0.0, "se": nan, "n_groups": 0}, + -1: {"effect": 0.0, "se": nan, "is_reference": True}, 0: {"effect": 1.0, "se": 0.0, "n_groups": 3}, 1: {"effect": 1.2, "se": nan, "n_groups": 3}, } @@ -2348,7 +2348,7 @@ def test_explicit_reference_with_anchor_signature_still_retained(self): nan = float("nan") original = SimpleNamespace( event_study_effects={ - -1: {"effect": 0.0, "se": nan, "n_groups": 0}, + -1: {"effect": 0.0, "se": nan, "is_reference": True}, 0: {"effect": 1.0, "se": 0.5, "n_groups": 3}, } ) diff --git a/tests/test_event_study_surface.py b/tests/test_event_study_surface.py index 9867f1ed6..c3fe06b71 100644 --- a/tests/test_event_study_surface.py +++ b/tests/test_event_study_surface.py @@ -558,9 +558,10 @@ def _expected_reference_labels(label, native): -1-anticipation; dCDH at 0). - LPDiD's horizon==-1 base row. - Wooldridge / HAD omit no baseline (no reference). - - The remaining relative-dict producers mark EVERY zero-count sentinel - row (CallawaySantAnna universal base on a gapped grid can materialize - several); CS varying and EfficientDiD estimate e=-1, so none. + - The remaining relative-dict producers mark EVERY row carrying the + explicit ``is_reference`` key (CallawaySantAnna universal base on a + gapped grid can materialize several; 4.0, row M-093 - the zero-count + sentinel is retired); CS varying and EfficientDiD estimate e=-1, so none. """ if label == "SpilloverDiD" or label in NATIVE_SURFACE_PRODUCERS: return [] if native.reference_period is None else [native.reference_period] @@ -572,14 +573,9 @@ def _expected_reference_labels(label, native): return [-1] if (native.event_study["horizon"] == -1).any() else [] if label in ("WooldridgeDiD", "HeterogeneousAdoptionDiD"): return [] - # Count-sentinel producers: a zero-count row is the reference only when its - # effect is finite (0.0); a NaN-effect zero-count row is non-estimable. - return sorted( - e - for e, row in native.event_study_effects.items() - if (row.get("n_groups", 1) == 0 or row.get("n_obs", 1) == 0) - and np.isfinite(row.get("att", row.get("effect", np.nan))) - ) + # Keyed producers: the explicit ``is_reference`` row(s); a NaN-effect + # zero-count row is a genuine non-estimable horizon, never a reference. + return sorted(e for e, row in native.event_study_effects.items() if row.get("is_reference")) def test_all_producers_built(surfaces): @@ -615,7 +611,7 @@ def test_reference_row_values_normalized(label, surfaces): @pytest.mark.parametrize("label", ALL_PRODUCERS) def test_no_sentinel_in_schema(label, surfaces): - # The retiring sentinels (n_groups==0 / n_obs==0) must not surface as a + # The retired sentinels (n_groups==0 / n_obs==0) must not surface as a # zero count on the marked reference row: it is NaN there. _, surface = surfaces[label] ref = surface.is_reference @@ -754,13 +750,17 @@ def test_spillover_oversized_horizon_single_reference(): ) surface = build_event_study_surface(sp) df = surface.to_dataframe() - # Native has >=2 zero-count rows; exactly one is the reference. + # Native: the reference row is keyed (no count); the rectangular + # non-estimable horizons carry a GENUINE n_obs == 0 (>= 1 of them). zero_native = [k for k, r in sp.event_study_effects.items() if r.get("n_obs") == 0] - assert len(zero_native) >= 2 + keyed_native = [k for k, r in sp.event_study_effects.items() if r.get("is_reference")] + assert len(zero_native) >= 1 + assert keyed_native == [sp.reference_period] + assert "n_obs" not in sp.event_study_effects[sp.reference_period] marked = df[df["is_reference"]]["event_time"].tolist() assert marked == [sp.reference_period] # A non-reference zero-count horizon keeps its NaN att and n=0. - other_zero = next(k for k in zero_native if k != sp.reference_period) + other_zero = zero_native[0] row = df[df["event_time"] == other_zero].iloc[0] assert not bool(row["is_reference"]) assert np.isnan(row["att"]) @@ -881,7 +881,8 @@ def test_cs_universal_gapped_multiple_references(): first_treat="first_treat", ) ) - native_refs = sorted(e for e, r in cs.event_study_effects.items() if r.get("n_groups") == 0) + native_refs = sorted(e for e, r in cs.event_study_effects.items() if r.get("is_reference")) + assert all("n_groups" not in cs.event_study_effects[e] for e in native_refs) assert len(native_refs) >= 2 # the case only matters when >1 surface = build_event_study_surface(cs) marked = sorted(surface.event_time[surface.is_reference].tolist()) @@ -918,36 +919,53 @@ class _NotRequested: build_event_study_surface(_NotRequested()) -def test_zero_count_finite_nonzero_effect_raises(): - # A zero-count row with a finite NONZERO effect is malformed (a reference - # must be exactly 0) - fail loudly rather than silently rewrite to 0. +def test_keyed_reference_finite_nonzero_effect_raises(): + # An ``is_reference`` row with a finite NONZERO effect is malformed (a + # reference must be exactly 0) - fail loudly rather than let the + # container's __post_init__ silently rewrite it to 0 (row M-093, 8a). class _Malformed: alpha = 0.05 event_study_effects = { 0: {"effect": 1.0, "se": 0.1, "n_obs": 40}, - 1: {"effect": 3.3, "se": np.nan, "n_obs": 0}, # nonzero + zero count + 1: {"effect": 3.3, "se": np.nan, "is_reference": True}, # nonzero + marked } with pytest.raises(ValueError, match="finite nonzero effect"): build_event_study_surface(_Malformed()) +def test_zero_count_is_a_genuine_count_not_a_reference(): + # The 3.x zero-count SENTINEL is retired (row M-093, 8a): a row with + # ``n_obs == 0`` / ``n_groups == 0``, effect 0.0 and NO ``is_reference`` + # key is an ordinary zero-count row - the builder marks nothing. + class _Unkeyed: + alpha = 0.05 + event_study_effects = { + -1: {"effect": 0.0, "se": np.nan, "n_groups": 0}, + 0: {"effect": 1.0, "se": 0.1, "n_groups": 40}, + } + + surface = build_event_study_surface(_Unkeyed()) + assert not surface.is_reference.any() + assert surface.n[0] == 0.0 and surface.n_kind == "groups" + + def test_zero_count_nan_horizon_is_not_a_reference(): # TwoStageDiD emits effect=NaN, n_obs=0 for an estimated horizon whose # observations are all filtered (two_stage_aggregation.py::_stage2_event_study) - distinct from - # its effect=0.0, n_obs=0 reference. The count-sentinel adapter path must - # mark ONLY the finite-effect (0.0) row as reference and preserve the - # NaN-effect horizon as a non-reference NaN row (never normalize it to 0). + # its keyed reference row. The adapter must mark ONLY the ``is_reference`` + # row and preserve the NaN-effect zero-count horizon as a non-reference + # NaN row (never normalize it to 0). class _FakeTwoStage: alpha = 0.05 event_study_effects = { - -1: { # the reference: effect 0.0, n_obs 0 + -1: { # the reference: effect 0.0, keyed, no count "effect": 0.0, "se": 0.0, "t_stat": np.nan, "p_value": np.nan, "conf_int": (0.0, 0.0), - "n_obs": 0, + "is_reference": True, }, 0: { # an identified horizon "effect": 1.5, diff --git a/tests/test_honest_did.py b/tests/test_honest_did.py index f1df5885c..055112087 100644 --- a/tests/test_honest_did.py +++ b/tests/test_honest_did.py @@ -713,10 +713,10 @@ def test_max_pre_violation_excludes_reference_period(self): # ``_estimate_max_pre_violation`` reads the CS raw ``event_study_effects`` # dict (the 3.x fit-time shape, now the legacy-pickle route): feed it a - # stand-in carrying the container's surface. Its reference row is the - # zero-count sentinel the method excludes. + # stand-in carrying the container's surface. Its reference row carries + # the is_reference key the method excludes (4.0, M-093). results = legacy_standin(fit) - assert results.event_study_effects[-1]["n_groups"] == 0 + assert results.event_study_effects[-1].get("is_reference") is True # If there are real pre-periods, max_violation should be > 0 # (based on actual pre-period effects, not the reference period's effect=0) diff --git a/tests/test_imputation.py b/tests/test_imputation.py index be291688b..699e7b68c 100644 --- a/tests/test_imputation.py +++ b/tests/test_imputation.py @@ -308,7 +308,7 @@ def test_horizon_max(self): ) for h, eff in es_dict(results).items(): - if eff["n_obs"] > 0: + if eff.get("n_obs", 0) > 0: assert abs(h) <= 3 def test_summary(self): @@ -723,7 +723,7 @@ def test_se_positive_event_study(self): ) for h, eff in es_dict(results).items(): - if eff["n_obs"] > 0 and np.isfinite(eff["se"]): + if eff.get("n_obs", 0) > 0 and np.isfinite(eff["se"]): assert eff["se"] > 0 def test_aux_partition_cohort_horizon(self): @@ -964,7 +964,7 @@ def test_bootstrap_event_study(self, ci_params): relayed = es_dict(results) assert set(relayed) == set(results.event_study_effects) for h, eff in results.event_study_effects.items(): - if eff["n_obs"] == 0: + if eff.get("is_reference"): continue # reference row: the container reports NaN inference assert relayed[h]["effect"] == eff["effect"] assert relayed[h]["se"] == eff["se"] @@ -1019,7 +1019,7 @@ def test_bootstrap_balance_e_rejected_post_fit(self, ci_params): # The unrestricted relay still works and its SEs are the bootstrap SEs. relayed = es_dict(results) for h, eff in relayed.items(): - if eff["n_obs"] > 0 and np.isfinite(eff["effect"]): + if eff.get("n_obs", 0) > 0 and np.isfinite(eff["effect"]): if h in results.bootstrap_results.event_study_ses: assert np.isfinite(results.bootstrap_results.event_study_ses[h]) assert eff["se"] == results.bootstrap_results.event_study_ses[h] @@ -1567,7 +1567,7 @@ def test_nan_propagation_all_nan_horizon(self): # Check that NaN horizons have all-NaN inference for h, eff in effects.items(): - if eff["n_obs"] > 0 and np.isnan(eff["effect"]): + if eff.get("n_obs", 0) > 0 and np.isnan(eff["effect"]): assert np.isnan(eff["se"]) assert np.isnan(eff["t_stat"]) assert np.isnan(eff["p_value"]) @@ -1713,7 +1713,7 @@ def test_plot_reference_with_anticipation(self): # Reference should be at -2 (= -1 - anticipation) es = results.aggregate("event_study") assert es.event_time[es.is_reference].tolist() == [-2] - assert es_dict(results)[-2]["n_obs"] == 0 # legacy zero-count marker + assert es_dict(results)[-2].get("is_reference") is True # explicit marker (M-093) # Test that plot_event_study auto-detects it on the container pytest.importorskip("matplotlib") diff --git a/tests/test_methodology_callaway.py b/tests/test_methodology_callaway.py index 58a44d4c9..fce7dbe9c 100644 --- a/tests/test_methodology_callaway.py +++ b/tests/test_methodology_callaway.py @@ -271,7 +271,8 @@ def test_base_period_universal_includes_reference_period(self): ref = es[-1] assert ref["effect"] == 0.0, "Reference period effect should be 0" assert np.isnan(ref["se"]), "Reference period SE should be NaN" - assert ref["n_groups"] == 0, "Reference period n_groups should be 0" + assert ref.get("is_reference") is True, "Reference period row must carry is_reference" + assert "n_groups" not in ref, "Reference period row carries no count (M-093)" class TestDoublyRobustEstimator: diff --git a/tests/test_methodology_stacked_did.py b/tests/test_methodology_stacked_did.py index 9cb40f72d..5a3e55df6 100644 --- a/tests/test_methodology_stacked_did.py +++ b/tests/test_methodology_stacked_did.py @@ -776,7 +776,7 @@ def test_matches_closed_form_paper_formula(self): fitted = { h: res.event_study_effects[h]["effect"] for h in res.event_study_effects - if res.event_study_effects[h]["n_obs"] > 0 + if res.event_study_effects[h].get("n_obs", 0) > 0 } closed = _closed_form_cbwsdid(df, res.groups, _CB_KP, _CB_KPOST) assert set(fitted) == {h for h in closed} @@ -931,7 +931,7 @@ def test_anchor_and_effective_mass_is_load_bearing(self): fitted = { h: res.event_study_effects[h]["effect"] for h in res.event_study_effects - if res.event_study_effects[h]["n_obs"] > 0 + if res.event_study_effects[h].get("n_obs", 0) > 0 } # control counts genuinely vary across cohorts (else the test is vacuous) ncs = {a: d["n_control"] for a, d in res.balance_diagnostics.items()} diff --git a/tests/test_pretrends_event_study.py b/tests/test_pretrends_event_study.py index c2479baad..debf9b4ee 100644 --- a/tests/test_pretrends_event_study.py +++ b/tests/test_pretrends_event_study.py @@ -125,7 +125,7 @@ def test_pretrends_se_finite_positive(self): for h, eff in es_dict(results).items(): if h == ref_period: continue - if eff["n_obs"] == 0: + if eff.get("n_obs", 0) == 0: continue assert np.isfinite(eff["se"]), f"h={h}: SE not finite" assert eff["se"] > 0, f"h={h}: SE not positive" @@ -146,7 +146,7 @@ def test_reference_period_correct(self): assert -1 in effects ref = effects[-1] assert ref["effect"] == 0.0 - assert ref["n_obs"] == 0 + assert ref.get("is_reference") is True and "n_obs" not in ref def test_backward_compatibility(self): """pretrends=False (default) gives identical results.""" @@ -234,7 +234,7 @@ def test_anticipation_interaction(self): assert -2 in effects ref = effects[-2] assert ref["effect"] == 0.0 - assert ref["n_obs"] == 0 + assert ref.get("is_reference") is True and "n_obs" not in ref def test_get_params_includes_pretrends(self): """get_params includes pretrends parameter.""" @@ -334,7 +334,7 @@ def test_pretrends_se_finite_positive(self): for h, eff in es_dict(results).items(): if h == ref_period: continue - if eff["n_obs"] == 0: + if eff.get("n_obs", 0) == 0: continue assert np.isfinite(eff["se"]), f"h={h}: SE not finite" assert eff["se"] > 0, f"h={h}: SE not positive" @@ -409,7 +409,7 @@ def test_anticipation_interaction(self): assert -2 in effects ref = effects[-2] assert ref["effect"] == 0.0 - assert ref["n_obs"] == 0 + assert ref.get("is_reference") is True and "n_obs" not in ref # ============================================================================= @@ -509,7 +509,7 @@ def test_imputation_survey_weighted_event_study(self): ) for h, eff in es_dict(results).items(): - if eff["n_obs"] == 0: + if eff.get("n_obs", 0) == 0: continue assert np.isfinite(eff["se"]), f"h={h}: SE not finite" @@ -568,7 +568,7 @@ def test_imputation_pretrends_bootstrap_post_only(self): # touch them): the analytical recompute vs the relayed fit-time surface. es_no_boot, es_boot = es_dict(results_no_boot), es_dict(results_boot) for h in es_no_boot: - if h >= 0 or es_no_boot[h]["n_obs"] == 0: + if h >= 0 or es_no_boot[h].get("n_obs", 0) == 0: continue eff_nb = es_no_boot[h] eff_b = es_boot[h] @@ -630,7 +630,7 @@ def test_nondefault_index_analytical(self): es_perm = es_dict(results_perm) es_gap = es_dict(results_gap) for h in es_default: - if es_default[h]["n_obs"] == 0: + if es_default[h].get("n_obs", 0) == 0: continue eff_d = es_default[h] eff_p = es_perm[h] @@ -688,7 +688,7 @@ def test_nondefault_index_bootstrap(self): es_default, es_perm = es_dict(results_default), es_dict(results_perm) for h in es_default: - if es_default[h]["n_obs"] == 0: + if es_default[h].get("n_obs", 0) == 0: continue eff_d = es_default[h] eff_p = es_perm[h] diff --git a/tests/test_spillover.py b/tests/test_spillover.py index aa2b4cea5..478b03eb3 100644 --- a/tests/test_spillover.py +++ b/tests/test_spillover.py @@ -3318,6 +3318,7 @@ def test_event_study_emits_att_dynamic(self): "ci_low", "ci_high", "n_obs", + "is_reference", } def test_event_study_emits_multiindex_spillover_effects(self): @@ -3334,7 +3335,12 @@ def test_event_study_emits_event_study_effects_dict(self): assert all(isinstance(k, int) for k in res.event_study_effects.keys()) # Each entry has the TwoStageDiD schema. for k, entry in res.event_study_effects.items(): - assert set(entry.keys()) == {"effect", "se", "n_obs", "t_stat", "p_value", "conf_int"} + # Estimated rows carry n_obs; the reference row carries the + # explicit is_reference key and no count (4.0, M-093). + count_or_marker = {"is_reference"} if k == res.reference_period else {"n_obs"} + assert set(entry.keys()) == {"effect", "se", "t_stat", "p_value", "conf_int"} | ( + count_or_marker + ) assert isinstance(entry["conf_int"], tuple) assert len(entry["conf_int"]) == 2 @@ -3345,7 +3351,7 @@ def test_event_study_effects_reference_row_matches_two_stage_did(self): ref = res.event_study_effects[res.reference_period] assert ref["effect"] == 0.0 assert ref["se"] == 0.0 - assert ref["n_obs"] == 0 + assert ref["is_reference"] is True and "n_obs" not in ref # 4.0 marking (M-093) assert ref["conf_int"] == (0.0, 0.0) assert np.isnan(ref["t_stat"]) assert np.isnan(ref["p_value"]) @@ -3391,7 +3397,7 @@ def test_reference_row_appears_in_att_dynamic(self): ref_row = res.att_dynamic.loc[-1] assert ref_row["coef"] == 0.0 assert ref_row["se"] == 0.0 - assert ref_row["n_obs"] == 0 + assert bool(ref_row["is_reference"]) and np.isnan(ref_row["n_obs"]) # M-093 class TestSpilloverDiDEventStudyReduceToAggregate: @@ -3832,8 +3838,9 @@ def test_outer_ring_cells_may_be_empty(self): res = _fit_event_study(df, horizon_max=2) if "[50, 200]" in res.spillover_effects.index.get_level_values("ring"): outer = res.spillover_effects.xs("[50, 200]", level="ring") - # All n_obs = 0 (no units in the outer ring in this DGP). - assert all(outer["n_obs"] == 0) + # All n_obs = 0 (no units in the outer ring in this DGP); the + # reference row is excluded (its n_obs is NaN by construction). + assert all(outer.loc[~outer["is_reference"], "n_obs"] == 0) class TestSpilloverDiDEventStudyConleyIntegration: @@ -4015,6 +4022,9 @@ def test_event_study_effects_n_obs_reflects_post_mask_sample(self): res = est.fit(df, outcome="y", unit="unit", time="time", treatment="D") # event_study_effects dict mirrors att_dynamic, must be consistent. for k in res.att_dynamic.index: + if bool(res.att_dynamic.loc[k, "is_reference"]): + assert "n_obs" not in res.event_study_effects[int(k)] + continue es_n = res.event_study_effects[int(k)]["n_obs"] dyn_n = res.att_dynamic.loc[k, "n_obs"] assert es_n == dyn_n, ( @@ -4111,7 +4121,7 @@ def test_ref_period_row_uses_zero_anchor(self): row = res.spillover_effects.loc[(ring_label, ref)] assert row["coef"] == 0.0 assert row["se"] == 0.0 - assert row["n_obs"] == 0 + assert bool(row["is_reference"]) and np.isnan(row["n_obs"]) # M-093 assert row["ci_low"] == 0.0 assert row["ci_high"] == 0.0 assert np.isnan(row["t_stat"]) @@ -8656,7 +8666,11 @@ def test_k3_event_study_subpop_excludes_zero_weight_from_n_obs_metadata(self): # Exact n_obs values captured from the Wave E.3 implementation # post codex-R8 count_mask fix (excluding PSU 3 = first_treat=3 # removes treated-cohort rows at each event-time horizon). - _EXPECTED_EVENT_STUDY_N_OBS = {-2: 22, -1: 0, 0: 26, 1: 26, 2: 56} + _EXPECTED_EVENT_STUDY_N_OBS = {-2: 22, 0: 26, 1: 26, 2: 56} + # The reference row (k=-1) carries the is_reference marker and no + # count since 4.0 (M-093). + assert res_subpop.event_study_effects[-1]["is_reference"] is True + assert "n_obs" not in res_subpop.event_study_effects[-1] for k, expected_n in _EXPECTED_EVENT_STUDY_N_OBS.items(): actual_n = res_subpop.event_study_effects[k]["n_obs"] assert actual_n == expected_n, ( @@ -8666,9 +8680,9 @@ def test_k3_event_study_subpop_excludes_zero_weight_from_n_obs_metadata(self): ) # att_dynamic n_obs matches event_study_effects n_obs (same source) assert res_subpop.att_dynamic is not None - expected_n_list = [22, 0, 26, 26, 56] + expected_n_list = [22.0, np.nan, 26.0, 26.0, 56.0] # NaN at the reference row actual_n_list = res_subpop.att_dynamic["n_obs"].tolist() - assert actual_n_list == expected_n_list, ( + assert np.allclose(actual_n_list, expected_n_list, equal_nan=True), ( f"Wave E.3: att_dynamic.n_obs={actual_n_list}, " f"expected {expected_n_list}" ) # spillover_effects per-(ring, k) n_obs reflects count_mask too. @@ -8676,8 +8690,8 @@ def test_k3_event_study_subpop_excludes_zero_weight_from_n_obs_metadata(self): assert res_subpop.spillover_effects is not None # Sum across ring-k cells (positive horizons only, ref_period excluded). spill_n_obs = res_subpop.spillover_effects["n_obs"].tolist() - _EXPECTED_SPILLOVER_N_OBS = [0, 0, 42, 42, 102] - assert spill_n_obs == _EXPECTED_SPILLOVER_N_OBS, ( + _EXPECTED_SPILLOVER_N_OBS = [0.0, np.nan, 42.0, 42.0, 102.0] # NaN at the reference row + assert np.allclose(spill_n_obs, _EXPECTED_SPILLOVER_N_OBS, equal_nan=True), ( f"Wave E.3: spillover_effects n_obs={spill_n_obs}, " f"expected {_EXPECTED_SPILLOVER_N_OBS}" ) diff --git a/tests/test_stacked_did.py b/tests/test_stacked_did.py index 8b9c25770..646994b1d 100644 --- a/tests/test_stacked_did.py +++ b/tests/test_stacked_did.py @@ -98,7 +98,7 @@ def test_event_study(self, staggered_data): # Reference period effect should be zero ref = results.event_study_effects[-1] assert ref["effect"] == 0.0 - assert ref["n_obs"] == 0 + assert ref["is_reference"] is True and "n_obs" not in ref # 4.0 marking (M-093) # Post-treatment periods should have effects for h in range(0, 3): @@ -571,7 +571,8 @@ def test_anticipation_reference_period(self): # Reference period is -2 (not -1) assert -2 in results.event_study_effects assert results.event_study_effects[-2]["effect"] == 0.0 - assert results.event_study_effects[-2]["n_obs"] == 0 # sentinel + assert results.event_study_effects[-2]["is_reference"] is True # explicit marker (M-093) + assert "n_obs" not in results.event_study_effects[-2] # -1 is NOT the reference; it should have a non-zero estimated effect assert -1 in results.event_study_effects @@ -1860,7 +1861,7 @@ def test_constant_covariate_reduces_to_plain_stacked(self): cb = _cb_fit(df, balance="entropy", covariates=["x"]) plain = _cb_fit(df, balance="none") for h in plain.event_study_effects: - if plain.event_study_effects[h]["n_obs"] > 0: + if plain.event_study_effects[h].get("n_obs", 0) > 0: assert ( abs( cb.event_study_effects[h]["effect"] - plain.event_study_effects[h]["effect"] diff --git a/tests/test_staggered.py b/tests/test_staggered.py index 2f71fd238..42d284e53 100644 --- a/tests/test_staggered.py +++ b/tests/test_staggered.py @@ -2679,7 +2679,7 @@ def test_bootstrap_event_study_vcov_is_draw_covariance(self, base_period, antici assert np.linalg.eigvalsh(vcov).min() > 0 off_diagonal = vcov[~np.eye(len(index), dtype=bool)] assert np.abs(off_diagonal).max() > 0.1 * ses.min() ** 2 - reference = [e for e, d in effects.items() if d.get("n_groups", 1) == 0] + reference = [e for e, d in effects.items() if d.get("is_reference")] assert bool(reference) == (base_period == "universal") assert not set(reference) & set(index) @@ -4100,7 +4100,8 @@ def test_event_study_universal_includes_reference_period(self): assert np.isnan(ref["conf_int"][0]) and np.isnan( ref["conf_int"][1] ), f"Reference period CI should be (NaN, NaN), got {ref['conf_int']}" - assert ref["n_groups"] == 0, f"Reference period n_groups should be 0, got {ref['n_groups']}" + assert ref.get("is_reference") is True, "Reference period row must carry is_reference" + assert "n_groups" not in ref, "Reference period row carries no count (M-093)" def test_event_study_varying_excludes_reference_period(self): """Test that varying base period does NOT artificially add e=-1 with effect=0.""" @@ -4121,10 +4122,10 @@ def test_event_study_varying_excludes_reference_period(self): # Varying mode: no single reference period, e=-1 computed normally or excluded # The key is we don't artificially add a 0-effect entry if -1 in es: - # If it exists, it should be an actual computed effect, not 0.0 with n_groups=0 + # If it exists, it should be an actual computed effect, not a marked reference assert ( - es[-1]["n_groups"] > 0 - ), "Varying mode should not artificially add e=-1 with n_groups=0" + not es[-1].get("is_reference") and es[-1]["n_groups"] > 0 + ), "Varying mode should not artificially add e=-1 as a reference row" def test_event_study_universal_with_anticipation(self): """Test reference period with anticipation > 0.""" diff --git a/tests/test_two_stage.py b/tests/test_two_stage.py index c40e6a665..13e33466c 100644 --- a/tests/test_two_stage.py +++ b/tests/test_two_stage.py @@ -1057,7 +1057,10 @@ def test_balance_e_empty_cohorts_warns(self): # Event study should contain only the reference period assert len(es.event_time) == 1 assert bool(es.is_reference[0]) - assert es_dict_from_container(es)[int(es.event_time[0])]["n_obs"] == 0 + # Reference-only surface: the keyed row records no count (n_kind None). + assert es.n_kind is None + row = es_dict_from_container(es)[int(es.event_time[0])] + assert row["is_reference"] is True and "n_obs" not in row def test_proposition_5_nan_for_long_run_horizons(self): """Prop 5 horizons have n_obs > 0 but NaN inference (unidentified).""" @@ -2264,10 +2267,11 @@ def _assert_full_bootstrap_nan(r): assert all(np.isnan(v) for v in b.event_study_ses.values()) if b.group_ses: assert all(np.isnan(v) for v in b.group_ses.values()) - # Per-horizon event-study inference fields (skip reference-period markers, - # which carry n_obs == 0 and are not real effects). + # Per-horizon event-study inference fields (skip the reference row - the + # explicit is_reference marker, not a real effect - and genuine + # zero-observation horizons). for eff in (r.event_study_effects or {}).values(): - if eff.get("n_obs", 1) == 0: + if eff.get("is_reference") or eff.get("n_obs", 1) == 0: continue assert np.isnan(eff["se"]) assert np.isnan(eff["t_stat"]) diff --git a/tests/test_v4_sentinel_retirement.py b/tests/test_v4_sentinel_retirement.py new file mode 100644 index 000000000..ff1ce4728 --- /dev/null +++ b/tests/test_v4_sentinel_retirement.py @@ -0,0 +1,572 @@ +"""Item 8a of the 4.0 cut - the event-study reference-row sentinel retirement (ledger row M-093, first half). + +Through 3.x every native event-study dict marked its normalization (reference) +row with a zero-count SENTINEL: ``n_groups == 0`` on CallawaySantAnna, +``n_obs == 0`` on StackedDiD / ImputationDiD / TwoStageDiD / SpilloverDiD. +Since 4.0 the reference row carries an explicit ``"is_reference": True`` key +and NO count key, SpilloverDiD's event-study frames carry a boolean +``is_reference`` column (NaN ``n_obs`` on the reference rows), the unified +builder reads the key only (a zero count is a genuine count), the consumers' +raw-dict routes read the key, and 3.x pickles are migrated on load by +``_deprecation.migrate_event_study_reference_sentinel``. + +The second half of M-093 (every results class's ``to_dataframe(level= +"event_study")`` emitting the unified schema) is item 8b; the row flips to +``done`` there. +""" + +import copy +import pickle +import warnings + +import numpy as np +import pandas as pd +import pytest +from legacy_surface import es_dict_from_container, legacy_standin + +from diff_diff import ( + CallawaySantAnna, + ImputationDiD, + StackedDiD, + TwoStageDiD, + compute_honest_did, + compute_pretrends_power, +) +from diff_diff._deprecation import migrate_event_study_reference_sentinel +from diff_diff.prep_dgp import generate_staggered_data +from diff_diff.results_base import build_event_study_surface +from diff_diff.spillover import SpilloverDiD +from tests._dgp_utils import generate_butts_staggered_dgp + +THIS_TEST_REF = "tests/test_v4_sentinel_retirement.py" + +_PANEL_KW = dict(outcome="outcome", unit="unit", time="period", first_treat="first_treat") + + +def _quiet(fn, *args, **kwargs): + with warnings.catch_warnings(): + warnings.simplefilter("ignore") + return fn(*args, **kwargs) + + +# --------------------------------------------------------------------------- # +# Fixtures: one fit per producer that materializes a reference row +# --------------------------------------------------------------------------- # + + +@pytest.fixture(scope="module") +def stacked_fit(): + d = generate_staggered_data(n_units=120, n_periods=12, cohort_periods=[4, 6, 8], seed=42) + return _quiet(StackedDiD(kappa_pre=2, kappa_post=2).fit, d, **_PANEL_KW) + + +@pytest.fixture(scope="module") +def stacked_fit_anticipation_1(): + d = generate_staggered_data(n_units=120, n_periods=12, cohort_periods=[4, 6, 8], seed=42) + return _quiet(StackedDiD(kappa_pre=3, kappa_post=2, anticipation=1).fit, d, **_PANEL_KW) + + +@pytest.fixture(scope="module") +def imputation_boot_fit(): + d = generate_staggered_data(n_units=120, n_periods=8, cohort_periods=[4, 6], seed=42) + return _quiet(ImputationDiD(n_bootstrap=19, seed=1).fit, d, **_PANEL_KW) + + +@pytest.fixture(scope="module") +def twostage_boot_fit(): + d = generate_staggered_data(n_units=120, n_periods=8, cohort_periods=[4, 6], seed=42) + return _quiet(TwoStageDiD(n_bootstrap=19, seed=1).fit, d, **_PANEL_KW) + + +@pytest.fixture(scope="module") +def spillover_fit(): + d = generate_butts_staggered_dgp(seed=42) + est = SpilloverDiD( + rings=[0.0, 50.0, 200.0], + d_bar=200.0, + conley_coords=("lat", "lon"), + conley_metric="haversine", + conley_cutoff_km=200.0, + conley_lag_cutoff=0, + event_study=True, + horizon_max=2, + ) + return _quiet(est.fit, d, outcome="y", unit="unit", time="time", first_treat="first_treat") + + +def _gapped_universal_panel(): + """CS universal base on a gapped period grid: one positional base per cohort.""" + rng = np.random.RandomState(3) + rows = [] + for u in range(40): + g = 3 if u < 15 else (6 if u < 30 else 0) + ufe = rng.randn() * 2 + for t in (1, 3, 6): + post = 1 if (g > 0 and t >= g) else 0 + rows.append( + { + "unit": u, + "period": t, + "outcome": ufe + 0.3 * t + 2.0 * post + rng.randn() * 0.5, + "first_treat": g, + } + ) + return pd.DataFrame(rows) + + +@pytest.fixture(scope="module") +def cs_universal_gapped(): + return _quiet( + CallawaySantAnna(n_bootstrap=0, base_period="universal").fit, + _gapped_universal_panel(), + **_PANEL_KW, + ) + + +@pytest.fixture(scope="module") +def cs_universal_regular(): + d = generate_staggered_data(n_units=120, n_periods=8, cohort_periods=[4, 6], seed=7) + return _quiet(CallawaySantAnna(n_bootstrap=0, base_period="universal").fit, d, **_PANEL_KW) + + +def _ref_rows(effects): + return sorted(k for k, r in effects.items() if r.get("is_reference")) + + +# --------------------------------------------------------------------------- # +# (1) Native dict shape on every producer that materializes a reference row +# --------------------------------------------------------------------------- # + + +@pytest.mark.parametrize( + "fixture_name, expected_ref", + [ + ("stacked_fit", -1), + ("stacked_fit_anticipation_1", -2), + ("imputation_boot_fit", -1), + ("twostage_boot_fit", -1), + ("spillover_fit", -1), + ], +) +def test_native_reference_row_is_keyed_without_count(request, fixture_name, expected_ref): + res = request.getfixturevalue(fixture_name) + effects = res.event_study_effects + assert effects is not None + assert _ref_rows(effects) == [expected_ref] + ref = effects[expected_ref] + assert ref["is_reference"] is True + assert ref["effect"] == 0.0 + assert "n_obs" not in ref and "n_groups" not in ref + for k, row in effects.items(): + if k == expected_ref: + continue + assert "is_reference" not in row, k + assert "n_obs" in row and isinstance(row["n_obs"], (int, np.integer)), k + # The container marks exactly that row, with a NaN count. + es = build_event_study_surface(res) + marked = es.event_time[es.is_reference].tolist() + assert marked == [expected_ref] + assert np.isnan(es.n[es.is_reference][0]) + + +def test_spillover_frames_carry_is_reference_with_nan_n_obs(spillover_fit): + res = spillover_fit + ref = res.reference_period + for frame in (res.att_dynamic, res.spillover_effects): + assert "is_reference" in frame.columns + assert frame["is_reference"].dtype == bool + k = ( + frame.index.get_level_values("k") + if isinstance(frame.index, pd.MultiIndex) + else frame.index + ) + is_ref = np.asarray(k == ref) + assert np.array_equal(frame["is_reference"].to_numpy(), is_ref) + assert np.all(np.isnan(frame.loc[frame["is_reference"], "n_obs"].to_numpy(dtype=float))) + non_ref = frame.loc[~frame["is_reference"], "n_obs"].to_numpy(dtype=float) + assert np.all(np.isfinite(non_ref)) + # Rectangular empty cells (if any) keep a GENUINE zero count. + empty = frame.loc[~frame["is_reference"] & frame["coef"].isna(), "n_obs"] + assert np.all(empty.to_numpy(dtype=float) == 0.0) + # One reference row per ring on the spillover frame. + n_rings = res.spillover_effects.index.get_level_values("ring").nunique() + assert int(res.spillover_effects["is_reference"].sum()) == n_rings + # to_dict() records carry the column. + rec = res.to_dict()["att_dynamic"] + assert all("is_reference" in r for r in rec) + assert sum(1 for r in rec if r["is_reference"]) == 1 + + +@pytest.mark.parametrize( + "fixture_name", ["stacked_fit", "imputation_boot_fit", "twostage_boot_fit"] +) +def test_native_to_dataframe_event_study_reference_n_obs_is_nan(request, fixture_name): + # The 8a visible change on the native level frames (8b replaces them with + # the unified schema): the reference row's n_obs renders NaN, not 0. + res = request.getfixturevalue(fixture_name) + df = res.to_dataframe(level="event_study") + ref_mask = df["relative_period"] == -1 + assert ref_mask.sum() == 1 + assert np.isnan(float(df.loc[ref_mask, "n_obs"].iloc[0])) + assert np.all(np.isfinite(df.loc[~ref_mask, "n_obs"].to_numpy(dtype=float))) + + +# --------------------------------------------------------------------------- # +# (2) The CallawaySantAnna aggregator carrier +# --------------------------------------------------------------------------- # + + +def _capture_carrier_dict(monkeypatch, res): + """Run aggregate('event_study') and capture the PRODUCER dict the carrier + hands to the builder (never reconstructed from the container).""" + import diff_diff.staggered_results as sr + + captured = {} + real = sr.build_event_study_surface + + def _spy(carrier): + captured["effects"] = carrier.event_study_effects + return real(carrier) + + monkeypatch.setattr(sr, "build_event_study_surface", _spy) + es = _quiet(res.aggregate, "event_study") + return captured["effects"], es + + +def test_cs_universal_gapped_carrier_marks_every_positional_base(monkeypatch, cs_universal_gapped): + effects, es = _capture_carrier_dict(monkeypatch, cs_universal_gapped) + refs = _ref_rows(effects) + assert len(refs) >= 2 # one per cohort on the gapped grid + for k in refs: + assert "n_groups" not in effects[k] + assert effects[k]["effect"] == 0.0 and np.isnan(effects[k]["se"]) + for k, row in effects.items(): + if k not in refs: + assert "is_reference" not in row and row["n_groups"] > 0 + assert sorted(es.event_time[es.is_reference].tolist()) == refs + assert es.reference_periods == refs + + +def test_cs_universal_regular_carrier_single_reference(monkeypatch, cs_universal_regular): + effects, es = _capture_carrier_dict(monkeypatch, cs_universal_regular) + assert _ref_rows(effects) == [-1] + assert "n_groups" not in effects[-1] + assert es.reference_period == -1 + + +# --------------------------------------------------------------------------- # +# (3) The builder is count-agnostic +# --------------------------------------------------------------------------- # + + +class _Dict: + alpha = 0.05 + + def __init__(self, effects): + self.event_study_effects = effects + + +def test_builder_ignores_zero_count_without_key(): + es = build_event_study_surface( + _Dict( + { + -1: {"effect": 0.0, "se": np.nan, "n_groups": 0}, + 0: {"effect": 1.0, "se": 0.1, "n_groups": 40}, + } + ) + ) + assert not es.is_reference.any() + assert es.n[0] == 0.0 and es.n_kind == "groups" + + +def test_builder_marks_keyed_row_with_nan_count(): + es = build_event_study_surface( + _Dict( + { + -1: {"effect": 0.0, "se": 0.0, "is_reference": True}, + 0: {"effect": 1.0, "se": 0.1, "n_obs": 40}, + 1: {"effect": np.nan, "se": np.nan, "n_obs": 0}, # genuine empty horizon + } + ) + ) + assert es.is_reference.tolist() == [True, False, False] + assert np.isnan(es.n[0]) and es.n[1] == 40.0 and es.n[2] == 0.0 + assert np.isnan(es.att[2]) and np.isnan(es.se[0]) + assert es.n_kind == "obs" + + +def test_builder_rejects_keyed_row_with_finite_nonzero_effect(): + with pytest.raises(ValueError, match="is_reference row with a finite nonzero effect"): + build_event_study_surface( + _Dict( + { + -1: {"effect": 0.4, "se": 0.0, "is_reference": True}, + 0: {"effect": 1.0, "se": 0.1, "n_obs": 40}, + } + ) + ) + + +def test_builder_reference_only_dict_has_no_n_kind(): + # TwoStageDiD's balance_e no-cohort early return / ref_only with no + # Prop-5 rows: the only row records no count, so n_kind is None. + es = build_event_study_surface(_Dict({-1: {"effect": 0.0, "se": 0.0, "is_reference": True}})) + assert es.is_reference.tolist() == [True] + assert es.n_kind is None and np.isnan(es.n[0]) + legacy = es_dict_from_container(es) + assert list(legacy) == [-1] + row = legacy[-1] + assert set(row) == {"effect", "se", "t_stat", "p_value", "conf_int", "is_reference"} + assert row["is_reference"] is True and row["effect"] == 0.0 + assert np.isnan(row["se"]) and np.isnan(row["t_stat"]) and np.isnan(row["p_value"]) + assert np.isnan(row["conf_int"][0]) and np.isnan(row["conf_int"][1]) + + +def test_spillover_explicit_reference_and_key_agree(spillover_fit): + es = build_event_study_surface(spillover_fit) + assert es.event_time[es.is_reference].tolist() == [spillover_fit.reference_period] + + +# --------------------------------------------------------------------------- # +# (4) Legacy (3.x) pickle migration +# --------------------------------------------------------------------------- # + + +def _legacy_shape(effects, count_key): + """Rewrite keyed reference rows into the 3.x zero-count sentinel shape.""" + out = {} + for k, row in effects.items(): + row = dict(row) + if row.pop("is_reference", False): + row[count_key] = 0 + out[k] = row + return out + + +def test_migration_helper_rewrites_only_the_anchor_signature(): + legacy = { + -2: {"effect": np.nan, "se": np.nan, "n_obs": 0}, # genuine: stays + -1: {"effect": 0.0, "se": 0.0, "n_obs": 0}, # sentinel: rewritten + 0: {"effect": 0.0, "se": 0.2, "n_obs": 30}, # estimated zero: stays + 1: {"att": 0.0, "se": np.nan, "n_groups": 0}, # Wooldridge-style key + } + out = migrate_event_study_reference_sentinel(legacy) + assert out is not legacy + assert out[-2] == legacy[-2] and "is_reference" not in out[-2] + assert out[-1] == {"effect": 0.0, "se": 0.0, "is_reference": True} + assert out[0] == legacy[0] + assert out[1] == {"att": 0.0, "se": np.nan, "is_reference": True} + assert migrate_event_study_reference_sentinel(None) is None + # Idempotent on the 4.0 shape. + assert migrate_event_study_reference_sentinel(out) == out + + +def _reload_from_state(res, state): + obj = res.__class__.__new__(res.__class__) + obj.__setstate__(state) + return obj + + +@pytest.mark.parametrize( + "fixture_name, count_key", + [ + ("stacked_fit", "n_obs"), + ("imputation_boot_fit", "n_obs"), + ("twostage_boot_fit", "n_obs"), + ], +) +def test_setstate_migrates_legacy_dict_sentinel(request, fixture_name, count_key): + res = request.getfixturevalue(fixture_name) + state = copy.deepcopy(res.__dict__) + state["event_study_effects"] = _legacy_shape(res.event_study_effects, count_key) + assert state["event_study_effects"][-1][count_key] == 0 + loaded = _reload_from_state(res, state) + assert _ref_rows(loaded.event_study_effects) == [-1] + assert count_key not in loaded.event_study_effects[-1] + es = _quiet(loaded.aggregate, "event_study") + assert es.event_time[es.is_reference].tolist() == [-1] + + +def test_setstate_migrates_legacy_cs_universal_dict(cs_universal_gapped): + standin = legacy_standin(cs_universal_gapped) # 3.x fit-time fields, 4.0 keyed rows + refs = _ref_rows(standin.event_study_effects) + assert len(refs) >= 2 + state = copy.deepcopy(standin.__dict__) + state["event_study_effects"] = _legacy_shape(standin.event_study_effects, "n_groups") + loaded = _reload_from_state(standin, state) + assert _ref_rows(loaded.event_study_effects) == refs + assert all("n_groups" not in loaded.event_study_effects[k] for k in refs) + es = build_event_study_surface(loaded) + assert sorted(es.event_time[es.is_reference].tolist()) == refs + + +def test_setstate_migrates_legacy_spillover_frames(spillover_fit): + res = spillover_fit + ref = res.reference_period + state = copy.deepcopy(res.__dict__) + state["event_study_effects"] = _legacy_shape(res.event_study_effects, "n_obs") + for name in ("att_dynamic", "spillover_effects"): + frame = state[name].drop(columns=["is_reference"]).copy() + frame["n_obs"] = frame["n_obs"].fillna(0).astype(int) + state[name] = frame + loaded = _reload_from_state(res, state) + assert _ref_rows(loaded.event_study_effects) == [ref] + assert "n_obs" not in loaded.event_study_effects[ref] + pd.testing.assert_frame_equal(loaded.att_dynamic, res.att_dynamic) + pd.testing.assert_frame_equal(loaded.spillover_effects, res.spillover_effects) + + +def test_setstate_leaves_aggregate_mode_spillover_untouched(): + d = generate_butts_staggered_dgp(seed=42) + est = SpilloverDiD( + rings=[0.0, 50.0, 200.0], + d_bar=200.0, + conley_coords=("lat", "lon"), + conley_metric="haversine", + conley_cutoff_km=200.0, + conley_lag_cutoff=0, + ) + res = _quiet(est.fit, d, outcome="y", unit="unit", time="time", first_treat="first_treat") + assert res.reference_period is None and res.event_study_effects is None + assert "is_reference" not in res.spillover_effects.columns + loaded = pickle.loads(pickle.dumps(res)) + pd.testing.assert_frame_equal(loaded.spillover_effects, res.spillover_effects) + assert "is_reference" not in loaded.spillover_effects.columns + + +@pytest.mark.parametrize( + "fixture_name", + ["stacked_fit", "imputation_boot_fit", "twostage_boot_fit", "spillover_fit"], +) +def test_fresh_pickle_round_trip_is_a_noop(request, fixture_name): + res = request.getfixturevalue(fixture_name) + loaded = pickle.loads(pickle.dumps(res)) + a, b = res.event_study_effects, loaded.event_study_effects + assert set(a) == set(b) + for k in a: + assert set(a[k]) == set(b[k]), k + assert a[k].get("is_reference") == b[k].get("is_reference"), k + if "n_obs" in a[k]: + assert a[k]["n_obs"] == b[k]["n_obs"] + if getattr(res, "att_dynamic", None) is not None: + pd.testing.assert_frame_equal(loaded.att_dynamic, res.att_dynamic) + pd.testing.assert_frame_equal(loaded.spillover_effects, res.spillover_effects) + + +# --------------------------------------------------------------------------- # +# (5) Consumers read the key on their raw-dict routes +# --------------------------------------------------------------------------- # + + +def test_plot_extractor_resolves_reference_by_key(stacked_fit_anticipation_1): + from diff_diff.visualization._event_study import _extract_plot_data + + out = _extract_plot_data(stacked_fit_anticipation_1, None, None, None, None) + assert out[5] == -2 and out[6] is True # reference_period, reference_inferred + + +def test_plot_extractor_never_treats_zero_count_as_reference(): + from diff_diff.visualization._event_study import _extract_plot_data + + class _Res: + alpha = 0.05 + anticipation = 1 + event_study_effects = { + -3: {"effect": np.nan, "se": np.nan, "n_obs": 0}, # genuine empty horizon + -2: {"effect": 0.0, "se": 0.0, "is_reference": True}, + -1: {"effect": 0.2, "se": 0.1, "n_obs": 30}, + 0: {"effect": 1.0, "se": 0.1, "n_obs": 30}, + } + + out = _extract_plot_data(_Res(), None, None, None, None) + assert out[5] == -2 and out[6] is True + + class _Keyless: + alpha = 0.05 + event_study_effects = { + -2: {"effect": 0.1, "se": 0.1, "n_obs": 30}, + -1: {"effect": 0.0, "se": 0.0, "n_obs": 0}, # a 3.x sentinel, unmigrated + 0: {"effect": 1.0, "se": 0.1, "n_obs": 30}, + } + + out = _extract_plot_data(_Keyless(), None, None, None, None) + assert out[5] == -1 and out[6] is True # the backward-compatibility fallback + + +def test_honest_plot_anchor_from_key_and_all_undefined_guard(cs_universal_regular): + from diff_diff.visualization._event_study import plot_honest_event_study + + standin = legacy_standin(cs_universal_regular) + h = _quiet(compute_honest_did, standin, M=0.5) + es = cs_universal_regular.aggregate("event_study") + h.event_study_bounds = { + int(t): {"ci_lb": float(a) - 0.5, "ci_ub": float(a) + 0.5} + for t, a, r in zip(es.event_time, es.att, es.is_reference) + if not r + } + fig = plot_honest_event_study(h, show=False) + assert fig is not None + with pytest.raises(ValueError, match="No valid data to plot"): + plot_honest_event_study(h, periods=[-1], show=False) + + +def test_honest_and_pretrends_raw_route_parity(cs_universal_regular): + standin = legacy_standin(cs_universal_regular) + container = cs_universal_regular.aggregate("event_study") + h_raw = _quiet(compute_honest_did, standin, M=0.5) + h_con = _quiet(compute_honest_did, container, M=0.5) + np.testing.assert_allclose([h_raw.lb, h_raw.ub], [h_con.lb, h_con.ub], rtol=1e-10) + p_raw = _quiet(compute_pretrends_power, standin, M=0.5) + p_con = _quiet(compute_pretrends_power, container, M=0.5) + np.testing.assert_allclose(p_raw.power, p_con.power, rtol=1e-10) + + +def test_diagnostic_report_pre_period_collector_skips_reference_and_empty_rows(): + from diff_diff.diagnostic_report import _collect_pre_period_coefs + + class _Res: + anticipation = 0 + event_study_effects = { + -4: {"effect": np.nan, "se": np.nan, "p_value": np.nan, "n_obs": 0}, # empty + -3: {"effect": 0.1, "se": 0.05, "p_value": 0.05, "n_obs": 20}, + -2: {"effect": -0.1, "se": 0.05, "p_value": 0.05, "n_obs": 20}, + -1: {"effect": 0.0, "se": 0.0, "p_value": np.nan, "is_reference": True}, + 0: {"effect": 1.0, "se": 0.1, "p_value": 0.0, "n_obs": 20}, + } + + rows, n_dropped = _collect_pre_period_coefs(_Res()) + assert [k for k, *_ in rows] == [-3, -2] + assert n_dropped == 0 + + +@pytest.mark.parametrize( + "fixture_name", ["stacked_fit", "imputation_boot_fit", "twostage_boot_fit"] +) +def test_summary_marks_reference_row(request, fixture_name): + res = request.getfixturevalue(fixture_name) + assert "[ref: -1]" in res.summary() + + +@pytest.mark.parametrize("fixture_name", ["imputation_boot_fit", "twostage_boot_fit"]) +def test_bootstrap_override_skips_reference_row(request, fixture_name): + res = request.getfixturevalue(fixture_name) + ref = res.event_study_effects[-1] + assert ref["se"] == 0.0 and np.isnan(ref["p_value"]) # no percentile override + for k, row in res.event_study_effects.items(): + if k != -1 and np.isfinite(row["effect"]): + assert np.isfinite(row["se"]) and row["se"] > 0 + + +# --------------------------------------------------------------------------- # +# (6) Ledger +# --------------------------------------------------------------------------- # + + +def test_ledger_row_points_here_and_stays_planned_until_8b(): + from tests.test_v4_matrix import ROWS as LEDGER_ROWS + + row = {r["id"]: r for r in LEDGER_ROWS}["M-093"] + assert row["kind"] == "behavior" + assert row["status"] == "planned" # flips to done in 8b (the schema rewire) + assert row["test_ref"] == THIS_TEST_REF + assert row["old"] == "diff_diff:CallawaySantAnnaResults.event_study_effects" diff --git a/tests/test_visualization_new.py b/tests/test_visualization_new.py index 5f79b93e4..d98e2a20d 100644 --- a/tests/test_visualization_new.py +++ b/tests/test_visualization_new.py @@ -882,7 +882,7 @@ class _Fake: "t_stat": nan, "p_value": nan, "conf_int": ref_conf_int, - "n_obs": 0, + "is_reference": True, }, 0: { "effect": 1.5, diff --git a/tests/test_visualization_plotly.py b/tests/test_visualization_plotly.py index 47cc306ea..87e29d153 100644 --- a/tests/test_visualization_plotly.py +++ b/tests/test_visualization_plotly.py @@ -539,7 +539,7 @@ class _Fake: res = _Fake() res.event_study_effects = { - -1: {"effect": 0.0, "se": 0.0, "conf_int": (0.0, 0.0), "n_obs": 0}, + -1: {"effect": 0.0, "se": 0.0, "conf_int": (0.0, 0.0), "is_reference": True}, 0: {"effect": 1.5, "se": 0.0, "conf_int": (nan, nan), "n_obs": 8}, 1: {"effect": 1.0, "se": 0.5, "conf_int": (0.02, 1.98), "n_obs": 8}, } From a6b5eb2f145214853eb97c593c50ccc3fbf0dcdb Mon Sep 17 00:00:00 2001 From: igerber Date: Fri, 30 Oct 2026 16:54:46 -0400 Subject: [PATCH 2/2] test(v4): skip the honest-plot anchor test without matplotlib The CI test legs install only the core stack (numpy, pandas, scipy, pytest), so test_honest_plot_anchor_from_key_and_all_undefined_guard failed on every leg with the plotter's "matplotlib is required" ImportError. The test now follows the test_event_study_consumers.py::TestPlotting convention - importorskip matplotlib, force the Agg backend, close figures on exit - so it runs wherever matplotlib is present and skips where it is not. The sentinel retirement itself is unchanged. --- tests/test_v4_sentinel_retirement.py | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/tests/test_v4_sentinel_retirement.py b/tests/test_v4_sentinel_retirement.py index ff1ce4728..192f6b18c 100644 --- a/tests/test_v4_sentinel_retirement.py +++ b/tests/test_v4_sentinel_retirement.py @@ -494,6 +494,13 @@ class _Keyless: def test_honest_plot_anchor_from_key_and_all_undefined_guard(cs_universal_regular): + # matplotlib is an optional dependency (the CI test legs install only the + # core stack), so the plotting test skips without it - the + # test_event_study_consumers.py::TestPlotting convention. + matplotlib = pytest.importorskip("matplotlib") + matplotlib.use("Agg") + import matplotlib.pyplot as plt + from diff_diff.visualization._event_study import plot_honest_event_study standin = legacy_standin(cs_universal_regular) @@ -508,6 +515,7 @@ def test_honest_plot_anchor_from_key_and_all_undefined_guard(cs_universal_regula assert fig is not None with pytest.raises(ValueError, match="No valid data to plot"): plot_honest_event_study(h, periods=[-1], show=False) + plt.close("all") def test_honest_and_pretrends_raw_route_parity(cs_universal_regular):