Skip to content

Add experimental native ChainedDiD for incomplete panels - #887

Draft
shawcharles wants to merge 1 commit into
igerber:mainfrom
shawcharles:contrib/chained-did-native
Draft

shawcharles wants to merge 1 commit into
igerber:mainfrom
shawcharles:contrib/chained-did-native

Conversation

@shawcharles

Copy link
Copy Markdown
Contributor

Add experimental native ChainedDiD for overlapping incomplete panels. It combines observed within-unit changes to recover group-time effects using normalised pair IPW, never-treated controls, PT-post and identity GMM. Results retain original-unit population accounting, analytical unit/cluster covariance, canonical aggregation and explicit support diagnostics.

The methodology rests on Bellégo, Benatia and Dortet-Bernadet (2025), The chained difference-in-differences, Journal of Econometrics (DOI: 10.1016/j.jeconom.2024.105783). This contribution derives a restricted native Python estimator from that paper and uses cdid as an external validation reference. Its implementation-specific safeguards and finite-sample limitations are documented below.

The restricted contract includes static complete covariates and zero anticipation. Singleton comparisons are excluded, low support warns, mixed diagnostic event rows exclude structural reference zeros, and constituent cohort/date weights are available. OR/DR, attrition correction, survey designs, optimal GMM, simultaneous bands and bootstrap are outside this contribution.

Validation:

  • Default non-slow suite on the clean contribution branch: 15,619 passed, 291 skipped (CPython 3.14, default auto backend with the local OpenBLAS extension available).
  • Focused estimator/reference, naming, cluster, documentation-dependency and changelog checks: 609 passed. Minimum-version/reference checks: 111 passed on each of Python 3.9 and 3.15.
  • Built and installed a local CPython 3.14 Linux/OpenBLAS wheel; isolated Python/Rust consumers agree exactly for unit/cluster and covariate/unconditional fits, including aggregation, pickle and plotting. This is not a portable manylinux build claim.
  • The affected tutorial executes and Sphinx builds with warnings treated as errors. Black/Ruff pass.
  • Frozen public R output regressions preserve eight matching fixtures and the known incomplete continuous-covariate criterion difference.
  • Source-bound simulations and independent algebraic tests are retained. The latest paired diagnostic has 20,000 native fits without fit failures; oracle and design-weight results remain benchmark-only.

Normal pointwise inference remains experimental: supported sparse pairs can produce materially anti-conservative intervals despite the safeguards, and historical coverage qualification failures remain visible. This contribution makes no general finite-sample calibration or blanket R-parity claim. Hosted/non-Linux CI has not run; the configured PR jobs require the ready-for-ci label.

The statistical contract, derivation and reproducible evidence are in docs/methodology/chained-did-proposal.md, chained-did-pseudocode.md, chained-did-review-remediation.md and chained-did-oracle-diagnostic.md; user-facing behaviour is in docs/api/chained_did.rst. The post-initial-review changes are singleton-pair exclusion, support warnings, mixed-reference aggregation and exposed constituent weights. The user-supplied numerical review of these changes is recorded in docs/methodology/chained-did-review-remediation.md#prepared-branch-review-2-november-2026; the separate packaging review did not assess numerical methodology.

Before merge, maintainer decisions are required:

  1. Inference policy: accept experimental identity GMM with the documented sparse-pair undercoverage, or require precision-aware weighting or another inference policy with further validation first? With two fixed sparse treated units, recorded 95% interval coverage is 0.8005 at n=400 and 0.7260 at n=4,000. Support warnings and experimental status do not resolve this failure. The design-weighted prototype remains benchmark-only and does not repair the historical balanced-panel failure.
  2. Propensity criterion: accept the declared observed-pair logit MLE despite the known incomplete continuous-covariate R difference, or require alignment with the cohort-response-calibrated R criterion, its influence-function derivation and validation? See docs/methodology/chained-did-validation-followup.md.

The contribution is ready for upstream review, but not ready to merge: these decisions and hosted CI remain outstanding. Statistical qualification remains false.

…anels

Context:
- Extend diff-diff to overlapping incomplete panels using within-unit changes,
  with a paper-derived algorithm and cdid as an external validation reference.

Changes:
- Add ChainedDiD and ChainedDiDResults with normalised pair IPW, never-treated
  controls, PT-post and fixed identity GMM; support static complete covariates.
- Preserve original-unit accounting and provide analytical unit/cluster
  covariance, canonical aggregation, constituent weights and support diagnostics.
- Add API/LLM documentation, executable tutorial, public exports, plotting
  integration, algebraic/reference tests and source-bound simulation evidence.
- Retain failed coverage qualification and the known continuous-covariate R
  criterion difference. Sparse-pair pointwise intervals remain experimental.
- Record the subsequent user-supplied numerical review; keep maintainer
  inference-policy and propensity-criterion decisions explicit before merge.
- Exclude OR/DR, attrition correction, survey designs, optimal GMM, bootstrap and
  simultaneous bands. Preserve upstream 4.0.0 release metadata and dependencies.

Verification:
- Clean upstream contribution: default non-slow pytest suite, 15,619 passed and
  291 skipped on CPython 3.14 with the local OpenBLAS backend available.
- Focused estimator/reference/integration checks: 609 passed.
- Minimum-version/reference checks: 111 passed each on Python 3.9 and 3.15.
- Built and installed a local CPython 3.14 Linux/OpenBLAS wheel; isolated Python
  and Rust consumers agree exactly across four configurations, including
  aggregation, pickle and plotting checks.
- Tutorial 34 executed; Sphinx with -W, Black and Ruff passed.
- Independent submission review found no actionable packaging findings.
- Subsequent user-supplied numerical review confirmed the bounded fixes but
  retained sparse-pair undercoverage and requested explicit merge decisions.
- Submission wording and review-record correction only: previous execution
  logs retained; estimator code, tests and frozen evidence unchanged.
- Not run: hosted/non-Linux CI, portable manylinux build, or fresh broad
  Monte Carlo qualification. Existing qualification failures remain documented.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant