Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
51 changes: 49 additions & 2 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,8 +58,30 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
pretrends-power/sensitivity via the diagonal-covariance fallback, replay warnings
republished per section), and `practitioner_next_steps` advises the post-fit
route on bootstrapped CS fits instead of the deprecated fit-time kwarg. The
sibling estimators' (EfficientDiD/ImputationDiD/TwoStageDiD/ContinuousDiD)
bootstrapped recompute gates are unchanged.
sibling estimators' (ImputationDiD/TwoStageDiD/ContinuousDiD) bootstrapped
recompute gates are unchanged (EfficientDiD adopted the replay in this release —
see its own entry).
- **Post-fit `aggregate('event_study')`/`aggregate('group')` now work on bootstrapped
EfficientDiD fits** ([M-023] notes amendment; the CS replay mechanism transplanted).
The recompute levels REPLAY the fit-time multiplier bootstrap from a fit-retained
`BootstrapReplaySpec` (the RNG state captured at weight-stream construction, plus
the run parameters BY VALUE): percentile se/CI/t match a fit-time
`fit(aggregate=...)` aggregation to floating-point reassociation
(`assert_allclose`, ~1 ULP — never bit-identity; the discrete percentile p-value is
a count statistic compared at `2/n_bootstrap`), `seed=None` fits replay (the state
is retained regardless of seeding), post-fit `set_params`/attribute mutation cannot
alter a replay, and pickles carry the state. Replays re-emit the fit-time bootstrap
warnings for the replayed configuration; the `'simple'`/`'total'` relays stay
silent and unchanged. The spec stamps the weight-generation backend at capture —
an artifact replayed under the other Rust/NumPy weight backend fails closed
naming both backends rather than silently regenerating a different realization
(stratified-survey, census-FPC, and single-PSU degenerate fits are stamped
`"portable"` and replay anywhere). Pre-replay legacy pickles fail closed with a
refit message. The ES/group percentile-override appliers moved to
`diff_diff.bootstrap_utils` and are now shared verbatim by the CS and EfficientDiD
fit paths and replays (internal relocation; verified bit-inert).
`practitioner_next_steps` now advises the post-fit route on bootstrapped
EfficientDiD fits instead of the deprecated fit-time kwarg.
- **`LWDiD` (Lee & Wooldridge 2025, 2026 rolling-transformation DiD).** Unit-specific
demean/detrend (plus quarterly `demeanq`/`detrendq`) converts panel data to
cross-sectional transformed outcomes; supports common timing and staggered
Expand Down Expand Up @@ -95,6 +117,31 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
`results.aggregate('event_study')` surface instead.

### Fixed
- **Exactly-constant bootstrap distributions now NaN out instead of leaking a
roundoff SE.** Census-FPC zero-weight draws leave every multiplier-bootstrap
replicate at the original effect; `np.std` of a constant non-zero level can
return a tiny positive value from mean-subtraction roundoff, slipping past the
`se <= 0` guard and publishing an astronomically large, silently "significant"
t-statistic with a degenerate point CI. The shared percentile-statistic helpers
(scalar and batch, used by every multiplier-bootstrap engine — CallawaySantAnna
and EfficientDiD included) now detect exactly-constant distributions and return
the full NaN inference tuple with the existing zero-SE RuntimeWarning. Genuinely
varying draws are unaffected (the check is exact, not a tolerance).
- **EfficientDiD fractional-period event-study bucketing.** The bootstrap ES prep
keyed horizons by raw `t - g` while the analytical aggregator buckets by
`int(t - g)` (truncation toward zero), so on fractional-period panels a strict
sub-aggregate's percentile inference was attached to the pooled analytical row
(and the `balance_e` anchor filter could miss cohorts anchoring at fractional
horizons). The bootstrap prep now keys all three sites by the analytical
expression — a no-op on integer-period panels. Companion changes on BOTH the
analytical and bootstrap paths for fractional panels: `n_groups` now counts
DISTINCT cohorts per bucket (previously a cell count that over-counted once
buckets pool multiple cells per cohort — this also moves the event-study `n`
column on analytical fractional-period fits), and any aggregation that
truncation-buckets a fractional horizon (fit-time, post-fit, replay, and
`hausman_pretest`) now emits a `UserWarning` naming the new REGISTRY truncation
Note, which documents the full convention (double-width bucket 0, the PT-Post
reference collision, cell-mass weighting).
- **`LWDiD` maintainer fix wave** (post-acceptance validation campaign: 43
execution-verified findings, all resolved):
- Estimand: the `tau_omega` composite is complete-case with FIXED cohort
Expand Down
4 changes: 2 additions & 2 deletions TODO.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ Related tracking surfaces:
| Numeric between-period cohorts (e.g. `first_treat=4.5` with integer times) are rejected by LWDiD while CallawaySantAnna estimates them and LWDiD's own datetime/Period cohorts map to the next observed period — close the dtype asymmetry by adopting the next-observed-period mapping for numeric cohorts too (contract documented in REGISTRY cohort-encodings Note + `docs/api/lwdid.rst` Input Contract). Lands only after PR #588 merges | `diff_diff/lwdid.py` | #588 | Quick | Low |
| Implement the LW 2026 eq. 7.9/7.10 unit-average cohort estimand (regress per-unit post-average transformed outcomes on `[1, D_g]` vs never-treated) as an alternative to the documented cell-mass `cohort_effects` convention (REGISTRY within-cohort aggregation Note; the two differ on unbalanced panels, where cell-mass weights units by observed post periods). Needs the 7.10 regression + its covariance on the NT path. Lands only after PR #588 merges | `diff_diff/lwdid_staggered.py` | #588 | Quick | Low |
| Expose cell-mass overall ATT (Stata `Post_avg` convention; = CS-simple on balanced panels) as an aggregate extra on LWDiD results — the fit's `.att` is the paper's `tau_omega` (cohort-mean-then-treated-weight, eq. 7.18); the authors' large-N display uses cell-mass weighting instead, and both are legitimate estimands (see the REGISTRY LWDiD Aggregation note). Lands only after PR #588 merges | `diff_diff/lwdid_results.py` | #588 | Quick | Low |
| Post-fit `aggregate()` for the staggered DDD container: `StaggeredTripleDiffResults` carries no `AggregationMixin`, which is why the phase-3(b) merge had to carry fit-time `aggregate=`/`balance_e=` onto the surviving `TripleDifference` (rows M-140/M-141) as the ONE documented exception to the section-6 aggregate-postfit program. Porting the container onto the M-122 aggregation contract retires both rows; note the bootstrapped-fit recompute levels will need replay or a fail-closed relay — solved for CS via the BootstrapReplaySpec state replay (the container port can adopt the same mechanism); EfficientDiD/ImputationDiD/TwoStageDiD/ContinuousDiD still track theirs. Until it lands, the DDD docs deliberately keep teaching the fit-time kwarg (the canonical route there) | `diff_diff/staggered_triple_diff_results.py`, `diff_diff/aggregation.py`, `docs/api/triple_diff.rst`, `docs/tutorials/08_triple_diff.ipynb` | 3(b) | Heavy | Medium |
| Post-fit `aggregate()` for the staggered DDD container: `StaggeredTripleDiffResults` carries no `AggregationMixin`, which is why the phase-3(b) merge had to carry fit-time `aggregate=`/`balance_e=` onto the surviving `TripleDifference` (rows M-140/M-141) as the ONE documented exception to the section-6 aggregate-postfit program. Porting the container onto the M-122 aggregation contract retires both rows; note the bootstrapped-fit recompute levels will need replay or a fail-closed relay — solved for CS and EfficientDiD via the BootstrapReplaySpec state replay (the container port can adopt the same mechanism); ImputationDiD/TwoStageDiD/ContinuousDiD still track theirs. Until it lands, the DDD docs deliberately keep teaching the fit-time kwarg (the canonical route there) | `diff_diff/staggered_triple_diff_results.py`, `diff_diff/aggregation.py`, `docs/api/triple_diff.rst`, `docs/tutorials/08_triple_diff.ipynb` | 3(b) | Heavy | Medium |
| Staggered-DDD power support: `simulate_power`/`simulate_mde`/`simulate_sample_size` now REJECT a staggered-configured `TripleDifference` (both registered DDD generators emit 2x2x2 data and fit with `(group, partition, post)`, so a staggered config would be simulated under the wrong design). Support needs a staggered DDD DGP profile plus fit-kwargs builder, and a decision on whether the mode is selected by profile or by the estimator's own config | `diff_diff/power.py` | 3(b) | Mid | Low |
| Bootstrap-`seed` provenance on multiplier-bootstrap results containers: neither `StaggeredTripleDiffResults` nor `CallawaySantAnnaResults` carries the `seed` that generated its bootstrap SEs / p-values / sup-t bands, so a serialized result cannot report the random configuration behind its inference. NOT a 3(b) regression - `seed` reaches the engine and `get_params()` correctly (same seed reproduces the SE bit-exactly, a different seed moves it), the gap is results-object observability only, it predates the merge, and both containers inherit it from the shared `CallawaySantAnnaBootstrapMixin`. Add `seed` (and consider `n_bootstrap`/`bootstrap_weights`/`cband`) to BOTH containers plus `to_dict()`, with seeded and unseeded pins; sequence it with the M-014 container unification rather than schema-changing one container mid-merge. Precedent for exposing it: `ContinuousDiDResults`, `EfficientDiDResults`, `SyntheticDiDResults` already do | `diff_diff/staggered_triple_diff_results.py`, `diff_diff/staggered_results.py` | 3(b) | Quick | Low |
| `ContinuousDiD.pscore_trim` still validates `0.0 <= x < 0.5`, i.e. it admits `0`, while `TripleDifference` tightened to `0 < x < 0.5` in phase 3(b) (row M-142) on the grounds that `trim=0` disables the `np.clip(pscore, trim, 1-trim)` overlap guard keeping the `1/(1-p)` weights finite. The same argument applies to ContinuousDiD; aligning it was out of scope for a DDD merge and is recorded in the REGISTRY staggered-mode Note rather than left as silent drift. `TripleDifference` additionally gained a TYPE guard in 3(b) (reject bool/non-real-scalar/non-finite BEFORE the range comparison) because a bare `0 < x < 0.5` raises an incidental `TypeError` on `None`/str/complex/list, an ambiguous-truth error on a multi-element array, and silently ACCEPTS a 1-element array as the parameter; `ContinuousDiD`'s `np.isfinite(self.pscore_trim) and ...` has the same hole. Aligning both is one change - promote the guard to a shared `utils.validate_pscore_trim(value, *, allow_zero)` alongside `validate_n_bootstrap` rather than copying it | `diff_diff/continuous_did.py`, `diff_diff/utils.py` | 3(b) | Quick | Low |
Expand All @@ -36,7 +36,7 @@ Related tracking surfaces:
| `EventStudyResults` inference-provenance fields: the container records no `vcov_type`/`cluster_name`/`n_clusters`/`df_convention`/Conley metadata, so a serialized surface cannot distinguish unit auto-clustering from explicit clustering, survey, Conley, or the one-way carve-out (3(a) R9 review). Adding them is a cross-producer M-092 schema amendment (six builders, to_dict/summary rendering, surface-suite pins) - follow the pre-cut amendment convention (optional fields appended last, ledger note same-diff) rather than bolting onto one producer | `diff_diff/results_base.py` | 3(a) R9 | Mid | Low |
| Opt-in singleton-group pruning for TwoWayFixedEffects (static + event-study mode; reghdfe parity): singleton units/periods are currently RETAINED class-wide - the within-demeaned row is zero so points are unchanged, but N/G/residual-df count it and CR1/finite-sample SEs shift (~0.41019 -> 0.40962 measured; REGISTRY "Deviation from R" Note, R5 review) - reghdfe iteratively drops singletons by default while fixest retains them (diff-diff matches fixest); an opt-in knob needs iterative unit+period pruning with consistent cluster/survey/replicate/Conley array subsetting and a default-flip decision protocol (moves published SEs) | `diff_diff/twfe.py`, `diff_diff/estimators.py`, `diff_diff/utils.py` | 3(a) R5 | Mid | Low |
| Cohort-timing validation input for the simultaneous-adoption event-study family (TWFE `event_study=True` + MultiPeriodDiD through 3.9): an optional `first_treat=`/`cohort=` column so simultaneous adoption becomes checkable under the contract-valid time-invariant `D_i` indicator - today the staggered-adoption advisory derives timing from within-unit 0->1 transitions, so it can only fire on off-contract time-varying `D_it` input, and with valid `D_i` adoption timing is not observable in the inputs at all (REGISTRY "staggered-adoption detection limit" Notes, both sections); design questions: validate-only vs steering error, and interplay with the M-011 removal | `diff_diff/twfe.py`, `diff_diff/estimators.py` | 3(a) R2 | Mid | Medium |
| EfficientDiD `aggregate()` recompute levels (event_study/group) on bootstrapped fits fail closed ('simple' relays since the M-027 per-level convergence); wiring `BootstrapReplaySpec` (the CS mechanism: fit-captured RNG state + backend stamp, replayed post-fit — allclose to fit-time, not bit-identical, with a cross-backend fail-closed gate) would enable post-fit replay of percentile inference | `diff_diff/efficient_did_results.py`, `diff_diff/aggregation.py` | 2(b) PR-3a | Mid | Low |
| Bootstrapped ES REPLAY containers (CS and EfficientDiD) publish the analytical `survey_metadata.df_survey` provenance scalar beside percentile inference on survey fits — a cross-estimator provenance residual (shipped CS behaves identically; the per-row df/inference channels are correctly NaN, the scalar metadata field is the residual). Evaluate clearing/gating it on both estimators together (cross-surface twins) | `diff_diff/staggered_results.py`, `diff_diff/efficient_did_results.py`, `diff_diff/results_base.py` | EDiD-replay review | Quick | Low |
| ImputationDiD/TwoStageDiD `aggregate()` recompute levels on bootstrapped fits fail closed ('simple' relays since the M-027 per-level convergence; M-021/M-022); ImputationDiD's per-target psi machinery makes seeded replay tractable (the panel-backed kit retains everything the psi precompute reads), TwoStageDiD's per-level GMM scores are function-locals and would need retention | `diff_diff/imputation_results.py`, `diff_diff/two_stage_results.py`, `diff_diff/aggregation.py` | 2(b) PR-3b | Mid | Low |
| ContinuousDiD `aggregate('event_study')` on bootstrapped fits fails closed (M-025); a seeded post-fit bootstrap-ES replay is tractable - the multiplier draws are seeded (`np.random.default_rng(self.seed)`) - but needs the FULL per-cell `_bootstrap_info` (bread/ee_treated/Psi_eval/dPsi_*/beta_pred) the pruned kit deliberately drops, so shipping it means a kit-payload change with its own memory contract | `diff_diff/continuous_did_aggregation.py`, `diff_diff/continuous_did_results.py` | 2(b) PR-3c | Mid | Low |
| EfficientDiD, ImputationDiD, ContinuousDiD and HeterogeneousAdoptionDiD are the outstanding M-092 event-study df-provenance holes: the container's per-row df is all-NaN even on survey fits where a finite `_survey_df` governed the p-values (the container-level scalar `df_survey` IS exposed - the hole is the PER-ROW column only; no event_study_df/df_inference field; pre-existing, NOT a regression of the M-023 PR - today's builder output is identical). The kits now retain the scalar (ImputationDiD's since 2(b) PR-3b, ContinuousDiD's since 2(b) PR-3c - same shape: scalar `df_survey` exposed, per-row column all-NaN, identical to each fit-time surface); threading it into the per-row channel is a contained follow-up | `diff_diff/efficient_did_results.py`, `diff_diff/imputation_results.py`, `diff_diff/continuous_did_results.py`, `diff_diff/results_base.py` | 2(b) PR-3a | Quick | Low |
Expand Down
18 changes: 10 additions & 8 deletions diff_diff/aggregation.py
Original file line number Diff line number Diff line change
Expand Up @@ -554,10 +554,10 @@ class AggregationKit:
when no aggregation ran, so it cannot distinguish the two.
bootstrap : AggregationKit.BootstrapReplaySpec or None
Value-bound bootstrap replay description. Populated on
CallawaySantAnna bootstrapped fits (the recompute levels replay the
fit-time multiplier bootstrap from it); ``None`` on analytical fits
and on pre-replay legacy artifacts (whose bootstrapped recompute
levels fail closed with a refit message).
CallawaySantAnna and EfficientDiD bootstrapped fits (their recompute
levels replay the fit-time multiplier bootstrap from it); ``None``
on analytical fits and on pre-replay legacy artifacts (whose
bootstrapped recompute levels fail closed with a refit message).
"""

bookkeeping: Dict[str, Any]
Expand Down Expand Up @@ -585,10 +585,12 @@ class BootstrapReplaySpec:
- ``rebuild()`` reconstructs the plain unit-level stream via
``iter_weight_blocks`` (it does NOT cover the survey/FPC/PSU-expansion
branches).
- CallawaySantAnna's post-fit replay is STATE-ONLY: it consumes
``bitgen_state``/``n_bootstrap``/``weight_type``/``backend`` and lets
``_run_multiplier_bootstrap`` re-derive the generation branch from the
kit bookkeeping - one branch-selection implementation, no drift.
- The CallawaySantAnna and EfficientDiD post-fit replays are
STATE-ONLY: they consume
``bitgen_state``/``n_bootstrap``/``weight_type``/``backend`` and let
each engine's ``_run_multiplier_bootstrap`` re-derive the generation
branch from the kit bookkeeping - one branch-selection implementation
per engine, no drift.

``backend`` records the weight-generation backend identity at capture
(``"rust"``/``"numpy"`` per
Expand Down
7 changes: 4 additions & 3 deletions diff_diff/bootstrap_chunking.py
Original file line number Diff line number Diff line change
Expand Up @@ -79,9 +79,10 @@ def effective_weight_backend() -> str:
the same bit-generator state (Rust draws one base seed and row-seeds
Xoshiro absolutely; the NumPy fallback consumes the PCG64 stream
directly), so a captured RNG state replays bit-identically only within
one backend. Post-fit bootstrap replay (CallawaySantAnna's
``BootstrapReplaySpec``) stamps this value at fit and fails closed on a
mismatch rather than silently regenerating a different realization.
one backend. Post-fit bootstrap replay (the CallawaySantAnna and
EfficientDiD ``BootstrapReplaySpec``) stamps this value at fit and fails
closed on a mismatch rather than silently regenerating a different
realization.
"""
return "rust" if (HAS_RUST_BACKEND and _rust_bootstrap_weights is not None) else "numpy"

Expand Down
Loading