diff --git a/CHANGELOG.md b/CHANGELOG.md index 5a24fe8a..036fb9ae 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -58,8 +58,30 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 pretrends-power/sensitivity via the diagonal-covariance fallback, replay warnings republished per section), and `practitioner_next_steps` advises the post-fit route on bootstrapped CS fits instead of the deprecated fit-time kwarg. The - sibling estimators' (EfficientDiD/ImputationDiD/TwoStageDiD/ContinuousDiD) - bootstrapped recompute gates are unchanged. + sibling estimators' (ImputationDiD/TwoStageDiD/ContinuousDiD) bootstrapped + recompute gates are unchanged (EfficientDiD adopted the replay in this release — + see its own entry). +- **Post-fit `aggregate('event_study')`/`aggregate('group')` now work on bootstrapped + EfficientDiD fits** ([M-023] notes amendment; the CS replay mechanism transplanted). + The recompute levels REPLAY the fit-time multiplier bootstrap from a fit-retained + `BootstrapReplaySpec` (the RNG state captured at weight-stream construction, plus + the run parameters BY VALUE): percentile se/CI/t match a fit-time + `fit(aggregate=...)` aggregation to floating-point reassociation + (`assert_allclose`, ~1 ULP — never bit-identity; the discrete percentile p-value is + a count statistic compared at `2/n_bootstrap`), `seed=None` fits replay (the state + is retained regardless of seeding), post-fit `set_params`/attribute mutation cannot + alter a replay, and pickles carry the state. Replays re-emit the fit-time bootstrap + warnings for the replayed configuration; the `'simple'`/`'total'` relays stay + silent and unchanged. The spec stamps the weight-generation backend at capture — + an artifact replayed under the other Rust/NumPy weight backend fails closed + naming both backends rather than silently regenerating a different realization + (stratified-survey, census-FPC, and single-PSU degenerate fits are stamped + `"portable"` and replay anywhere). Pre-replay legacy pickles fail closed with a + refit message. The ES/group percentile-override appliers moved to + `diff_diff.bootstrap_utils` and are now shared verbatim by the CS and EfficientDiD + fit paths and replays (internal relocation; verified bit-inert). + `practitioner_next_steps` now advises the post-fit route on bootstrapped + EfficientDiD fits instead of the deprecated fit-time kwarg. - **`LWDiD` (Lee & Wooldridge 2025, 2026 rolling-transformation DiD).** Unit-specific demean/detrend (plus quarterly `demeanq`/`detrendq`) converts panel data to cross-sectional transformed outcomes; supports common timing and staggered @@ -95,6 +117,31 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 `results.aggregate('event_study')` surface instead. ### Fixed +- **Exactly-constant bootstrap distributions now NaN out instead of leaking a + roundoff SE.** Census-FPC zero-weight draws leave every multiplier-bootstrap + replicate at the original effect; `np.std` of a constant non-zero level can + return a tiny positive value from mean-subtraction roundoff, slipping past the + `se <= 0` guard and publishing an astronomically large, silently "significant" + t-statistic with a degenerate point CI. The shared percentile-statistic helpers + (scalar and batch, used by every multiplier-bootstrap engine — CallawaySantAnna + and EfficientDiD included) now detect exactly-constant distributions and return + the full NaN inference tuple with the existing zero-SE RuntimeWarning. Genuinely + varying draws are unaffected (the check is exact, not a tolerance). +- **EfficientDiD fractional-period event-study bucketing.** The bootstrap ES prep + keyed horizons by raw `t - g` while the analytical aggregator buckets by + `int(t - g)` (truncation toward zero), so on fractional-period panels a strict + sub-aggregate's percentile inference was attached to the pooled analytical row + (and the `balance_e` anchor filter could miss cohorts anchoring at fractional + horizons). The bootstrap prep now keys all three sites by the analytical + expression — a no-op on integer-period panels. Companion changes on BOTH the + analytical and bootstrap paths for fractional panels: `n_groups` now counts + DISTINCT cohorts per bucket (previously a cell count that over-counted once + buckets pool multiple cells per cohort — this also moves the event-study `n` + column on analytical fractional-period fits), and any aggregation that + truncation-buckets a fractional horizon (fit-time, post-fit, replay, and + `hausman_pretest`) now emits a `UserWarning` naming the new REGISTRY truncation + Note, which documents the full convention (double-width bucket 0, the PT-Post + reference collision, cell-mass weighting). - **`LWDiD` maintainer fix wave** (post-acceptance validation campaign: 43 execution-verified findings, all resolved): - Estimand: the `tau_omega` composite is complete-case with FIXED cohort diff --git a/TODO.md b/TODO.md index d115a51e..291d5b44 100644 --- a/TODO.md +++ b/TODO.md @@ -25,7 +25,7 @@ Related tracking surfaces: | Numeric between-period cohorts (e.g. `first_treat=4.5` with integer times) are rejected by LWDiD while CallawaySantAnna estimates them and LWDiD's own datetime/Period cohorts map to the next observed period — close the dtype asymmetry by adopting the next-observed-period mapping for numeric cohorts too (contract documented in REGISTRY cohort-encodings Note + `docs/api/lwdid.rst` Input Contract). Lands only after PR #588 merges | `diff_diff/lwdid.py` | #588 | Quick | Low | | Implement the LW 2026 eq. 7.9/7.10 unit-average cohort estimand (regress per-unit post-average transformed outcomes on `[1, D_g]` vs never-treated) as an alternative to the documented cell-mass `cohort_effects` convention (REGISTRY within-cohort aggregation Note; the two differ on unbalanced panels, where cell-mass weights units by observed post periods). Needs the 7.10 regression + its covariance on the NT path. Lands only after PR #588 merges | `diff_diff/lwdid_staggered.py` | #588 | Quick | Low | | Expose cell-mass overall ATT (Stata `Post_avg` convention; = CS-simple on balanced panels) as an aggregate extra on LWDiD results — the fit's `.att` is the paper's `tau_omega` (cohort-mean-then-treated-weight, eq. 7.18); the authors' large-N display uses cell-mass weighting instead, and both are legitimate estimands (see the REGISTRY LWDiD Aggregation note). Lands only after PR #588 merges | `diff_diff/lwdid_results.py` | #588 | Quick | Low | -| Post-fit `aggregate()` for the staggered DDD container: `StaggeredTripleDiffResults` carries no `AggregationMixin`, which is why the phase-3(b) merge had to carry fit-time `aggregate=`/`balance_e=` onto the surviving `TripleDifference` (rows M-140/M-141) as the ONE documented exception to the section-6 aggregate-postfit program. Porting the container onto the M-122 aggregation contract retires both rows; note the bootstrapped-fit recompute levels will need replay or a fail-closed relay — solved for CS via the BootstrapReplaySpec state replay (the container port can adopt the same mechanism); EfficientDiD/ImputationDiD/TwoStageDiD/ContinuousDiD still track theirs. Until it lands, the DDD docs deliberately keep teaching the fit-time kwarg (the canonical route there) | `diff_diff/staggered_triple_diff_results.py`, `diff_diff/aggregation.py`, `docs/api/triple_diff.rst`, `docs/tutorials/08_triple_diff.ipynb` | 3(b) | Heavy | Medium | +| Post-fit `aggregate()` for the staggered DDD container: `StaggeredTripleDiffResults` carries no `AggregationMixin`, which is why the phase-3(b) merge had to carry fit-time `aggregate=`/`balance_e=` onto the surviving `TripleDifference` (rows M-140/M-141) as the ONE documented exception to the section-6 aggregate-postfit program. Porting the container onto the M-122 aggregation contract retires both rows; note the bootstrapped-fit recompute levels will need replay or a fail-closed relay — solved for CS and EfficientDiD via the BootstrapReplaySpec state replay (the container port can adopt the same mechanism); ImputationDiD/TwoStageDiD/ContinuousDiD still track theirs. Until it lands, the DDD docs deliberately keep teaching the fit-time kwarg (the canonical route there) | `diff_diff/staggered_triple_diff_results.py`, `diff_diff/aggregation.py`, `docs/api/triple_diff.rst`, `docs/tutorials/08_triple_diff.ipynb` | 3(b) | Heavy | Medium | | Staggered-DDD power support: `simulate_power`/`simulate_mde`/`simulate_sample_size` now REJECT a staggered-configured `TripleDifference` (both registered DDD generators emit 2x2x2 data and fit with `(group, partition, post)`, so a staggered config would be simulated under the wrong design). Support needs a staggered DDD DGP profile plus fit-kwargs builder, and a decision on whether the mode is selected by profile or by the estimator's own config | `diff_diff/power.py` | 3(b) | Mid | Low | | Bootstrap-`seed` provenance on multiplier-bootstrap results containers: neither `StaggeredTripleDiffResults` nor `CallawaySantAnnaResults` carries the `seed` that generated its bootstrap SEs / p-values / sup-t bands, so a serialized result cannot report the random configuration behind its inference. NOT a 3(b) regression - `seed` reaches the engine and `get_params()` correctly (same seed reproduces the SE bit-exactly, a different seed moves it), the gap is results-object observability only, it predates the merge, and both containers inherit it from the shared `CallawaySantAnnaBootstrapMixin`. Add `seed` (and consider `n_bootstrap`/`bootstrap_weights`/`cband`) to BOTH containers plus `to_dict()`, with seeded and unseeded pins; sequence it with the M-014 container unification rather than schema-changing one container mid-merge. Precedent for exposing it: `ContinuousDiDResults`, `EfficientDiDResults`, `SyntheticDiDResults` already do | `diff_diff/staggered_triple_diff_results.py`, `diff_diff/staggered_results.py` | 3(b) | Quick | Low | | `ContinuousDiD.pscore_trim` still validates `0.0 <= x < 0.5`, i.e. it admits `0`, while `TripleDifference` tightened to `0 < x < 0.5` in phase 3(b) (row M-142) on the grounds that `trim=0` disables the `np.clip(pscore, trim, 1-trim)` overlap guard keeping the `1/(1-p)` weights finite. The same argument applies to ContinuousDiD; aligning it was out of scope for a DDD merge and is recorded in the REGISTRY staggered-mode Note rather than left as silent drift. `TripleDifference` additionally gained a TYPE guard in 3(b) (reject bool/non-real-scalar/non-finite BEFORE the range comparison) because a bare `0 < x < 0.5` raises an incidental `TypeError` on `None`/str/complex/list, an ambiguous-truth error on a multi-element array, and silently ACCEPTS a 1-element array as the parameter; `ContinuousDiD`'s `np.isfinite(self.pscore_trim) and ...` has the same hole. Aligning both is one change - promote the guard to a shared `utils.validate_pscore_trim(value, *, allow_zero)` alongside `validate_n_bootstrap` rather than copying it | `diff_diff/continuous_did.py`, `diff_diff/utils.py` | 3(b) | Quick | Low | @@ -36,7 +36,7 @@ Related tracking surfaces: | `EventStudyResults` inference-provenance fields: the container records no `vcov_type`/`cluster_name`/`n_clusters`/`df_convention`/Conley metadata, so a serialized surface cannot distinguish unit auto-clustering from explicit clustering, survey, Conley, or the one-way carve-out (3(a) R9 review). Adding them is a cross-producer M-092 schema amendment (six builders, to_dict/summary rendering, surface-suite pins) - follow the pre-cut amendment convention (optional fields appended last, ledger note same-diff) rather than bolting onto one producer | `diff_diff/results_base.py` | 3(a) R9 | Mid | Low | | Opt-in singleton-group pruning for TwoWayFixedEffects (static + event-study mode; reghdfe parity): singleton units/periods are currently RETAINED class-wide - the within-demeaned row is zero so points are unchanged, but N/G/residual-df count it and CR1/finite-sample SEs shift (~0.41019 -> 0.40962 measured; REGISTRY "Deviation from R" Note, R5 review) - reghdfe iteratively drops singletons by default while fixest retains them (diff-diff matches fixest); an opt-in knob needs iterative unit+period pruning with consistent cluster/survey/replicate/Conley array subsetting and a default-flip decision protocol (moves published SEs) | `diff_diff/twfe.py`, `diff_diff/estimators.py`, `diff_diff/utils.py` | 3(a) R5 | Mid | Low | | Cohort-timing validation input for the simultaneous-adoption event-study family (TWFE `event_study=True` + MultiPeriodDiD through 3.9): an optional `first_treat=`/`cohort=` column so simultaneous adoption becomes checkable under the contract-valid time-invariant `D_i` indicator - today the staggered-adoption advisory derives timing from within-unit 0->1 transitions, so it can only fire on off-contract time-varying `D_it` input, and with valid `D_i` adoption timing is not observable in the inputs at all (REGISTRY "staggered-adoption detection limit" Notes, both sections); design questions: validate-only vs steering error, and interplay with the M-011 removal | `diff_diff/twfe.py`, `diff_diff/estimators.py` | 3(a) R2 | Mid | Medium | -| EfficientDiD `aggregate()` recompute levels (event_study/group) on bootstrapped fits fail closed ('simple' relays since the M-027 per-level convergence); wiring `BootstrapReplaySpec` (the CS mechanism: fit-captured RNG state + backend stamp, replayed post-fit — allclose to fit-time, not bit-identical, with a cross-backend fail-closed gate) would enable post-fit replay of percentile inference | `diff_diff/efficient_did_results.py`, `diff_diff/aggregation.py` | 2(b) PR-3a | Mid | Low | +| Bootstrapped ES REPLAY containers (CS and EfficientDiD) publish the analytical `survey_metadata.df_survey` provenance scalar beside percentile inference on survey fits — a cross-estimator provenance residual (shipped CS behaves identically; the per-row df/inference channels are correctly NaN, the scalar metadata field is the residual). Evaluate clearing/gating it on both estimators together (cross-surface twins) | `diff_diff/staggered_results.py`, `diff_diff/efficient_did_results.py`, `diff_diff/results_base.py` | EDiD-replay review | Quick | Low | | ImputationDiD/TwoStageDiD `aggregate()` recompute levels on bootstrapped fits fail closed ('simple' relays since the M-027 per-level convergence; M-021/M-022); ImputationDiD's per-target psi machinery makes seeded replay tractable (the panel-backed kit retains everything the psi precompute reads), TwoStageDiD's per-level GMM scores are function-locals and would need retention | `diff_diff/imputation_results.py`, `diff_diff/two_stage_results.py`, `diff_diff/aggregation.py` | 2(b) PR-3b | Mid | Low | | ContinuousDiD `aggregate('event_study')` on bootstrapped fits fails closed (M-025); a seeded post-fit bootstrap-ES replay is tractable - the multiplier draws are seeded (`np.random.default_rng(self.seed)`) - but needs the FULL per-cell `_bootstrap_info` (bread/ee_treated/Psi_eval/dPsi_*/beta_pred) the pruned kit deliberately drops, so shipping it means a kit-payload change with its own memory contract | `diff_diff/continuous_did_aggregation.py`, `diff_diff/continuous_did_results.py` | 2(b) PR-3c | Mid | Low | | EfficientDiD, ImputationDiD, ContinuousDiD and HeterogeneousAdoptionDiD are the outstanding M-092 event-study df-provenance holes: the container's per-row df is all-NaN even on survey fits where a finite `_survey_df` governed the p-values (the container-level scalar `df_survey` IS exposed - the hole is the PER-ROW column only; no event_study_df/df_inference field; pre-existing, NOT a regression of the M-023 PR - today's builder output is identical). The kits now retain the scalar (ImputationDiD's since 2(b) PR-3b, ContinuousDiD's since 2(b) PR-3c - same shape: scalar `df_survey` exposed, per-row column all-NaN, identical to each fit-time surface); threading it into the per-row channel is a contained follow-up | `diff_diff/efficient_did_results.py`, `diff_diff/imputation_results.py`, `diff_diff/continuous_did_results.py`, `diff_diff/results_base.py` | 2(b) PR-3a | Quick | Low | diff --git a/diff_diff/aggregation.py b/diff_diff/aggregation.py index a62bf89f..b1804150 100644 --- a/diff_diff/aggregation.py +++ b/diff_diff/aggregation.py @@ -554,10 +554,10 @@ class AggregationKit: when no aggregation ran, so it cannot distinguish the two. bootstrap : AggregationKit.BootstrapReplaySpec or None Value-bound bootstrap replay description. Populated on - CallawaySantAnna bootstrapped fits (the recompute levels replay the - fit-time multiplier bootstrap from it); ``None`` on analytical fits - and on pre-replay legacy artifacts (whose bootstrapped recompute - levels fail closed with a refit message). + CallawaySantAnna and EfficientDiD bootstrapped fits (their recompute + levels replay the fit-time multiplier bootstrap from it); ``None`` + on analytical fits and on pre-replay legacy artifacts (whose + bootstrapped recompute levels fail closed with a refit message). """ bookkeeping: Dict[str, Any] @@ -585,10 +585,12 @@ class BootstrapReplaySpec: - ``rebuild()`` reconstructs the plain unit-level stream via ``iter_weight_blocks`` (it does NOT cover the survey/FPC/PSU-expansion branches). - - CallawaySantAnna's post-fit replay is STATE-ONLY: it consumes - ``bitgen_state``/``n_bootstrap``/``weight_type``/``backend`` and lets - ``_run_multiplier_bootstrap`` re-derive the generation branch from the - kit bookkeeping - one branch-selection implementation, no drift. + - The CallawaySantAnna and EfficientDiD post-fit replays are + STATE-ONLY: they consume + ``bitgen_state``/``n_bootstrap``/``weight_type``/``backend`` and let + each engine's ``_run_multiplier_bootstrap`` re-derive the generation + branch from the kit bookkeeping - one branch-selection implementation + per engine, no drift. ``backend`` records the weight-generation backend identity at capture (``"rust"``/``"numpy"`` per diff --git a/diff_diff/bootstrap_chunking.py b/diff_diff/bootstrap_chunking.py index edc7db49..6ec4a110 100644 --- a/diff_diff/bootstrap_chunking.py +++ b/diff_diff/bootstrap_chunking.py @@ -79,9 +79,10 @@ def effective_weight_backend() -> str: the same bit-generator state (Rust draws one base seed and row-seeds Xoshiro absolutely; the NumPy fallback consumes the PCG64 stream directly), so a captured RNG state replays bit-identically only within - one backend. Post-fit bootstrap replay (CallawaySantAnna's - ``BootstrapReplaySpec``) stamps this value at fit and fails closed on a - mismatch rather than silently regenerating a different realization. + one backend. Post-fit bootstrap replay (the CallawaySantAnna and + EfficientDiD ``BootstrapReplaySpec``) stamps this value at fit and fails + closed on a mismatch rather than silently regenerating a different + realization. """ return "rust" if (HAS_RUST_BACKEND and _rust_bootstrap_weights is not None) else "numpy" diff --git a/diff_diff/bootstrap_utils.py b/diff_diff/bootstrap_utils.py index 43e62f0c..dd08d30a 100644 --- a/diff_diff/bootstrap_utils.py +++ b/diff_diff/bootstrap_utils.py @@ -1,16 +1,18 @@ """ Shared bootstrap utilities for multiplier bootstrap inference. -Provides weight generation, percentile CI, and p-value helpers used by -both CallawaySantAnna and ContinuousDiD estimators. +Provides weight generation, percentile statistics (CI / p-value / per-effect +stats), and the percentile-override appliers shared across the estimator +bootstrap engines. """ import warnings -from typing import TYPE_CHECKING, Optional, Tuple +from typing import TYPE_CHECKING, Any, Dict, Optional, Protocol, Tuple import numpy as np from diff_diff._backend import HAS_RUST_BACKEND, _rust_bootstrap_weights +from diff_diff.utils import safe_inference_batch if TYPE_CHECKING: from diff_diff.survey import ResolvedSurveyDesign @@ -29,6 +31,8 @@ "compute_effect_bootstrap_stats_batch", "warn_bootstrap_failure_rate", "stratified_bootstrap_indices", + "apply_bootstrap_event_study_overrides", + "apply_bootstrap_group_overrides", ] @@ -366,7 +370,12 @@ def compute_effect_bootstrap_stats( se = float(np.std(valid_dist, ddof=1)) # Guard: if SE is not finite or zero, all inference fields must be NaN. - if not np.isfinite(se) or se <= 0: + # An EXACTLY CONSTANT distribution is degenerate too, even when its + # np.std comes back tiny-positive from mean-subtraction roundoff at a + # non-zero constant level (e.g. census-FPC zero-weight draws leave every + # replicate at the original effect): a t built on that roundoff SE would + # be astronomically large and silently "significant". + if not np.isfinite(se) or se <= 0 or float(valid_dist.max()) == float(valid_dist.min()): warnings.warn( f"Bootstrap SE is non-finite or zero (n_valid={n_valid}) in {context}. " "Returning NaN for SE/CI/p-value.", @@ -472,8 +481,13 @@ def compute_effect_bootstrap_stats_batch( batch_p = np.minimum(2 * batch_p, 1.0) batch_p = np.maximum(batch_p, 1 / (n_bootstrap + 1)) - # Guard: SE must be positive and finite - se_valid = np.isfinite(batch_ses) & (batch_ses > 0) + # Guard: SE must be positive and finite, and the distribution must + # not be EXACTLY CONSTANT (a constant non-zero level can produce a + # tiny-positive np.std from mean-subtraction roundoff - e.g. + # census-FPC zero-weight draws - which would otherwise leak a + # roundoff SE and an astronomically large t past the zero check). + is_constant = sub.max(axis=0) == sub.min(axis=0) + se_valid = np.isfinite(batch_ses) & (batch_ses > 0) & ~is_constant n_bad_se = int(np.sum(~se_valid)) if n_bad_se > 0: warnings.warn( @@ -948,3 +962,96 @@ def generate_rao_wu_weights_batch( for b in range(n_bootstrap): result[b] = generate_rao_wu_weights(resolved_survey, rng) return result + + +# ============================================================================= +# Bootstrap override helpers (shared by fit and the post-fit replay) +# ============================================================================= +# Extracted verbatim from CallawaySantAnna.fit()'s inline blocks (and adopted +# by EfficientDiD's fit/replay) so the post-fit aggregate() replay applies +# EXACTLY the same percentile overrides the fit-time path applies — one +# implementation, no twin drift. (The deprecated StaggeredTripleDifference +# keeps its OWN copy of the group replacement loop; unifying it is sequenced +# with the M-014 container port.) Note on warning attribution: when the +# engines run under the post-fit replay their fit-tuned stacklevels resolve +# into library frames rather than the user's aggregate() call — accepted as +# cosmetic (recorded decision). + + +class _BootstrapOverrideSource(Protocol): + """Structural contract for bootstrap containers the appliers consume. + + Both ``CSBootstrapResults`` and ``EDiDBootstrapResults`` satisfy it by + field name; a Protocol keeps this module free of estimator imports. + """ + + event_study_ses: Optional[Dict[Any, float]] + event_study_cis: Optional[Dict[Any, Tuple[float, float]]] + event_study_p_values: Optional[Dict[Any, float]] + group_effect_ses: Optional[Dict[Any, float]] + group_effect_cis: Optional[Dict[Any, Tuple[float, float]]] + group_effect_p_values: Optional[Dict[Any, float]] + + +def apply_bootstrap_event_study_overrides( + event_study_effects: Optional[Dict[int, Dict[str, Any]]], + bootstrap_results: _BootstrapOverrideSource, + alpha: float, +) -> None: + """Overwrite per-event-time se/CI/p with percentile-bootstrap values. + + Mutates ``event_study_effects`` in place; t is recomputed from the + percentile SE via ``safe_inference_batch``. No-op when either side has + no event-study surface. + """ + if ( + event_study_effects is not None + and bootstrap_results.event_study_ses is not None + and bootstrap_results.event_study_cis is not None + and bootstrap_results.event_study_p_values is not None + ): + es_keys = [e for e in event_study_effects if e in bootstrap_results.event_study_ses] + if es_keys: + es_effects_arr = np.array([float(event_study_effects[e]["effect"]) for e in es_keys]) + es_ses_arr = np.array([float(bootstrap_results.event_study_ses[e]) for e in es_keys]) + es_t_stats, _, _, _ = safe_inference_batch(es_effects_arr, es_ses_arr, alpha=alpha) + for idx, e in enumerate(es_keys): + event_study_effects[e]["se"] = bootstrap_results.event_study_ses[e] + event_study_effects[e]["conf_int"] = bootstrap_results.event_study_cis[e] + event_study_effects[e]["p_value"] = bootstrap_results.event_study_p_values[e] + event_study_effects[e]["t_stat"] = float(es_t_stats[idx]) + + +def apply_bootstrap_group_overrides( + group_effects: Optional[Dict[Any, Dict[str, Any]]], + bootstrap_results: _BootstrapOverrideSource, + alpha: float, +) -> None: + """Overwrite per-group se/CI/p with percentile-bootstrap values. + + Mutates ``group_effects`` in place and clears each row's ``df_used`` + (the percentile inference never used the analytical df, so keeping it + would claim a t-reference that governed nothing). No-op when either + side has no group surface. + """ + if ( + group_effects is not None + and bootstrap_results.group_effect_ses is not None + and bootstrap_results.group_effect_cis is not None + and bootstrap_results.group_effect_p_values is not None + ): + grp_keys = [g for g in group_effects if g in bootstrap_results.group_effect_ses] + if grp_keys: + grp_effects_arr = np.array([float(group_effects[g]["effect"]) for g in grp_keys]) + grp_ses_arr = np.array([float(bootstrap_results.group_effect_ses[g]) for g in grp_keys]) + grp_t_stats, _, _, _ = safe_inference_batch(grp_effects_arr, grp_ses_arr, alpha=alpha) + for idx, g in enumerate(grp_keys): + group_effects[g]["se"] = bootstrap_results.group_effect_ses[g] + group_effects[g]["conf_int"] = bootstrap_results.group_effect_cis[g] + group_effects[g]["p_value"] = bootstrap_results.group_effect_p_values[g] + group_effects[g]["t_stat"] = float(grp_t_stats[idx]) + # Same clearing rule the ES df provenance follows: these + # se/p/CI are now percentile-bootstrap values that never used + # the analytical df, so keeping df_used would claim a + # t-reference that governed nothing. + group_effects[g]["df_used"] = None diff --git a/diff_diff/efficient_did.py b/diff_diff/efficient_did.py index 6a6b0e6b..4df2051d 100644 --- a/diff_diff/efficient_did.py +++ b/diff_diff/efficient_did.py @@ -30,7 +30,11 @@ from diff_diff._base import BaseEstimator from diff_diff._deprecation import NOT_SUPPLIED -from diff_diff.aggregation import AggregationKit +from diff_diff.aggregation import AggregationKit, BootstrapReplaySpec +from diff_diff.bootstrap_utils import ( + apply_bootstrap_event_study_overrides, + apply_bootstrap_group_overrides, +) from diff_diff.efficient_did_aggregation import ( _cluster_aggregate, _compute_se_from_eif, @@ -118,6 +122,7 @@ def _build_edid_aggregation_kit( df_survey: Optional[float], alpha: float, anticipation: int, + bootstrap_results: Optional[EDiDBootstrapResults] = None, ) -> Optional[AggregationKit]: """Bundle the retained EIF payload + bookkeeping for post-fit aggregate(). @@ -138,6 +143,21 @@ def _build_edid_aggregation_kit( # Unreachable after fit()'s empty-effects raise; kept for the CS # guard shape (a kit with nothing to re-aggregate is not attached). return None + # STATE-ONLY replay carrier (the CS contract): the spec retains the RNG + # snapshot + generation-branch identity the run recorded, BY VALUE, so + # post-fit aggregate() can replay the fit-time multiplier bootstrap + # through the same engine. Its rebuild() factory is unused here — the + # engine re-derives the generation branch from the kit bookkeeping. + # None on analytical fits, where the recompute levels stay analytical. + replay_spec = None + if bootstrap_results is not None and bootstrap_results._replay_bitgen_state is not None: + replay_spec = BootstrapReplaySpec( + bitgen_state=bootstrap_results._replay_bitgen_state, + n_bootstrap=bootstrap_results.n_bootstrap, + n_units=int(n_units), + weight_type=bootstrap_results.weight_type, + backend=bootstrap_results._replay_backend, + ) return AggregationKit( bookkeeping={ # PRIVATE SNAPSHOTS of the aggregation inputs (CI review P0): @@ -165,7 +185,7 @@ def _build_edid_aggregation_kit( alpha=alpha, anticipation=anticipation, cband=False, - bootstrap=None, + bootstrap=replay_spec, ) @@ -513,20 +533,23 @@ def fit( ``.aggregate('group')`` / ``.aggregate('simple')`` / ``.aggregate('total')``. On bootstrapped fits (``n_bootstrap > 0``) the post-fit - RECOMPUTE levels (``'event_study'``/``'group'``) fail closed - — the deprecated fit-time path remains the supported route - for those bootstrapped aggregated surfaces — while - ``aggregate('simple')`` and, where supported, - ``aggregate('total')`` relay the stored bootstrap - inference and stay available (the per-level policy - converged with row M-027). + RECOMPUTE levels (``'event_study'``/``'group'``) REPLAY the + fit-time multiplier bootstrap from the kit-retained RNG + state (percentile inference, allclose to a fit-time + aggregation; no refit needed), while ``aggregate('simple')`` + and, where supported, ``aggregate('total')`` relay the + stored bootstrap inference (the per-level policy converged + with row M-027). balance_e : int, optional DEPRECATED (3.9, removed in 4.0, row M-120): moves onto post-fit ``aggregate()`` — ``results.aggregate('event_study', balance_e=2)``. EDiD's balance rule is the ANCHOR-HORIZON rule (keep cohorts with a - finite effect at ``e == balance_e``), the same rule - CallawaySantAnna uses. + finite effect at the anchor horizon), the same rule shape + CallawaySantAnna uses — with one keying-granularity + difference: EDiD anchors on the ``int(t - g)`` bucket while + CS keys raw ``t - g`` (identical on integer-period panels; + see the REGISTRY truncation Note). survey_design : SurveyDesign, optional Survey design specification for design-based inference. Applies survey weights to all means, covariances, and cohort @@ -1364,46 +1387,14 @@ def _finalize_cell(g: Any, att_gt: float, eif_vals: np.ndarray) -> Dict[str, Any se = float(group_time_effects[gt]["se"]) group_time_effects[gt]["t_stat"] = safe_inference(eff, se, alpha=self.alpha)[0] - es_cis = bootstrap_results.event_study_cis - es_pvs = bootstrap_results.event_study_p_values - if ( - event_study_effects is not None - and bootstrap_results.event_study_ses is not None - and es_cis is not None - and es_pvs is not None - ): - for e in event_study_effects: - if e in bootstrap_results.event_study_ses: - event_study_effects[e]["se"] = bootstrap_results.event_study_ses[e] - event_study_effects[e]["conf_int"] = es_cis[e] - event_study_effects[e]["p_value"] = es_pvs[e] - eff = float(event_study_effects[e]["effect"]) - se = float(event_study_effects[e]["se"]) - event_study_effects[e]["t_stat"] = safe_inference( - eff, se, alpha=self.alpha - )[0] - - g_cis = bootstrap_results.group_effect_cis - g_pvs = bootstrap_results.group_effect_p_values - if ( - group_effects is not None - and bootstrap_results.group_effect_ses is not None - and g_cis is not None - and g_pvs is not None - ): - for g in group_effects: - if g in bootstrap_results.group_effect_ses: - group_effects[g]["se"] = bootstrap_results.group_effect_ses[g] - group_effects[g]["conf_int"] = g_cis[g] - group_effects[g]["p_value"] = g_pvs[g] - eff = float(group_effects[g]["effect"]) - se = float(group_effects[g]["se"]) - group_effects[g]["t_stat"] = safe_inference(eff, se, alpha=self.alpha)[0] - # Percentile-bootstrap inference has no analytical df; - # clear the provenance key the analytical pass recorded - # (the CS precedent) so bootstrap rows never publish an - # analytical survey df beside percentile p/CI. - group_effects[g]["df_used"] = None + # ES/group percentile overrides via the shared appliers (the same + # implementations the post-fit aggregate() replay runs — one + # code path, no fit-vs-replay drift). The appliers carry the + # availability guards and the group df_used clearing internally. + apply_bootstrap_event_study_overrides( + event_study_effects, bootstrap_results, self.alpha + ) + apply_bootstrap_group_overrides(group_effects, bootstrap_results, self.alpha) # ----- Build results ----- self.results_ = EfficientDiDResults( @@ -1487,6 +1478,7 @@ def _finalize_cell(g: Any, att_gt: float, eif_vals: np.ndarray) -> Dict[str, Any df_survey=_survey_df_post_overall, alpha=self.alpha, anticipation=self.anticipation, + bootstrap_results=bootstrap_results, ) self.is_fitted_ = True return self.results_ @@ -1637,8 +1629,12 @@ def _aggregate_es( ) -> Dict[int, Tuple[float, np.ndarray]]: """Aggregate (g,t) effects to post-treatment ES(e) with WIF-corrected EIF.""" by_e: Dict[int, List[Tuple[Tuple, float, float, np.ndarray]]] = {} + _has_fractional = False for (g, t), d in gt_effects.items(): - e = int(t - g) + raw_e = t - g + e = int(raw_e) + if raw_e != e: + _has_fractional = True if e < -ant: continue if not np.isfinite(d["effect"]): @@ -1653,6 +1649,15 @@ def _aggregate_es( by_e[e] = [] by_e[e].append(((g, t), d["effect"], pg, eif_vec)) + if _has_fractional: + warnings.warn( + "Fractional relative times detected: Hausman pre-test " + "horizons are bucketed by int(t - g) (truncation toward " + "zero). See the EfficientDiD REGISTRY truncation Note.", + UserWarning, + stacklevel=3, + ) + result: Dict[int, Tuple[float, np.ndarray]] = {} for e, items in by_e.items(): if e < 0: diff --git a/diff_diff/efficient_did_aggregation.py b/diff_diff/efficient_did_aggregation.py index 54a6015c..3ed450f5 100644 --- a/diff_diff/efficient_did_aggregation.py +++ b/diff_diff/efficient_did_aggregation.py @@ -22,10 +22,13 @@ is what keeps ``aggregate()`` off an ``_estimator_ref``. The numerical content of every function in this module is byte-identical to -its pre-extraction form, with ONE additive exception recorded in the M-023 -ledger notes: ``_aggregate_by_group`` records a per-row ``df_used`` key (the +its pre-extraction form, with the exceptions recorded in the M-023 ledger +notes: ``_aggregate_by_group`` records a per-row ``df_used`` key (the ``self._survey_df`` value at that row's ``safe_inference`` call) so the -post-fit group relay can publish exact per-row df provenance. +post-fit group relay can publish exact per-row df provenance; +``_aggregate_event_study`` counts DISTINCT cohorts in ``n_groups`` (identity +on integer panels) and warns once when fractional horizons are truncation- +bucketed (see the EfficientDiD REGISTRY truncation Note). """ import warnings @@ -325,7 +328,12 @@ def _aggregate_event_study( cluster_indices: Optional[np.ndarray] = None, n_clusters: Optional[int] = None, ) -> Dict[int, Dict[str, Any]]: - """Aggregate ATT(g,t) by relative time e = t - g. + """Aggregate ATT(g,t) by relative time ``e = int(t - g)``. + + On integer-period panels the ``int()`` is the identity. Fractional- + period panels are truncation-bucketed toward zero (a documented + deviation from the exact-relative-time equation — see the + EfficientDiD REGISTRY truncation Note) and emit a ``UserWarning``. Parameters ---------- @@ -346,15 +354,30 @@ def _aggregate_event_study( unit_cohorts : ndarray, optional Cohort assignment for each unit (for WIF correction). """ - # Organize by relative time + # Organize by relative time. Fractional horizons truncation-bucket + # (int() toward zero) — a lossy, documented convention that must not + # stay invisible to the user (no-silent-failures). + _has_fractional = False effects_by_e: Dict[int, List[Tuple[Tuple[Any, Any], float, float]]] = {} for (g, t), data in group_time_effects.items(): if not np.isfinite(data["effect"]): continue - e = int(t - g) + raw_e = t - g + e = int(raw_e) + if raw_e != e: + _has_fractional = True if e not in effects_by_e: effects_by_e[e] = [] effects_by_e[e].append(((g, t), data["effect"], cohort_fractions.get(g, 0.0))) + if _has_fractional: + warnings.warn( + "Fractional relative times detected: event-study horizons are " + "bucketed by int(t - g) (truncation toward zero), pooling " + "fractional horizons into integer buckets. See the " + "EfficientDiD REGISTRY truncation Note.", + UserWarning, + stacklevel=2, + ) # Balance if requested if balance_e is not None: @@ -441,7 +464,13 @@ def _aggregate_event_study( "t_stat": t_stat, "p_value": p_val, "conf_int": ci, - "n_groups": len(elist), + # DISTINCT cohorts in the bucket (the cohort-count n column): identity + # with len(elist) on integer panels (one cell per cohort per + # bucket); on fractional panels truncation-bucketing pools + # multiple cells per cohort and a raw cell count would + # over-count. Weights above remain per-cell (cell-mass within + # the bucket — see the REGISTRY truncation Note). + "n_groups": len({gt[0] for gt in gt_pairs}), } return result diff --git a/diff_diff/efficient_did_bootstrap.py b/diff_diff/efficient_did_bootstrap.py index 9f0089d0..538fc262 100644 --- a/diff_diff/efficient_did_bootstrap.py +++ b/diff_diff/efficient_did_bootstrap.py @@ -15,6 +15,7 @@ from diff_diff.bootstrap_chunking import ( ReplayableWeightStream, compute_block_size, + effective_weight_backend, iter_survey_multiplier_weight_blocks, iter_weight_blocks, tiled_if_matmul, @@ -48,6 +49,20 @@ class EDiDBootstrapResults: group_effect_p_values: Optional[Dict[Any, float]] = None bootstrap_distribution: Optional[np.ndarray] = field(default=None, repr=False) + def __post_init__(self) -> None: + # Post-fit replay bookkeeping, attached as PLAIN attributes (never + # dataclass fields) so the exported class's __init__ signature, + # dataclasses.fields() and asdict() stay unchanged — the CS + # precedent (CSBootstrapResults). `_replay_bitgen_state` is the RNG + # snapshot that fully determines the weight stream; + # `_replay_backend` is the generation-branch identity + # ("rust"/"numpy", or "portable" for provably backend-independent + # branches). Both are populated by _run_multiplier_bootstrap on + # every run (the single-PSU degenerate path included) and pickle + # via __dict__. + self._replay_bitgen_state: Optional[Dict[str, Any]] = None + self._replay_backend: Optional[str] = None + class EfficientDiDBootstrapMixin: """Mixin providing multiplier bootstrap for EfficientDiD.""" @@ -71,6 +86,8 @@ def _run_multiplier_bootstrap( n_clusters: Optional[int] = None, resolved_survey: Optional["ResolvedSurveyDesign"] = None, unit_level_weights: Optional[np.ndarray] = None, + *, + _replay_bitgen_state: Optional[Dict[str, Any]] = None, ) -> EDiDBootstrapResults: """Run multiplier bootstrap on stored EIF values. @@ -100,6 +117,16 @@ def _run_multiplier_bootstrap( ) rng = np.random.default_rng(self.seed) + if _replay_bitgen_state is not None: + # Post-fit replay: restore the fit-captured state so the weight + # stream below reproduces the fit-time draws bit-for-bit. + rng.bit_generator.state = _replay_bitgen_state + # Snapshot HERE — nothing below consumes the rng before the + # ReplayableWeightStream construction (the survey psu resolution is + # deliberately rng-free), so this value fully determines the weight + # stream within one weight backend. Taken before the single-PSU + # degenerate early return so that path is stamped too. + replay_bitgen_state = dict(rng.bit_generator.state) gt_pairs = list(group_time_effects.keys()) @@ -150,13 +177,21 @@ def _run_multiplier_bootstrap( UserWarning, stacklevel=3, ) - return self._build_nan_bootstrap_results( + nan_result = self._build_nan_bootstrap_results( group_time_effects, aggregate, balance_e, treatment_groups, cohort_fractions, ) + # No weights are ever generated on this path, so the + # artifact is backend-independent: the replay re-runs the + # engine, deterministically re-hits this return (the psu + # resolution is rng-free and kit-determined), re-emits the + # warning above, and reproduces the NaN surfaces anywhere. + nan_result._replay_bitgen_state = replay_bitgen_state + nan_result._replay_backend = "portable" + return nan_result # Build unit -> PSU column map if resolved_survey.psu is not None: psu_id_to_col = {int(p): c for c, p in enumerate(psu_ids)} @@ -213,6 +248,25 @@ def _make_weight_iter( ) -> Iterator[Tuple[int, np.ndarray]]: return iter_weight_blocks(self.n_bootstrap, n_units, self.bootstrap_weights, rng_) + # Generation-branch identity for the post-fit replay: "portable" for + # branches whose draws are provably identical under either weight + # backend — the stratified survey generator draws through the NumPy + # generator unconditionally, and the unstratified census-FPC case + # (fpc[0] <= n_psu, mirroring iter_survey_multiplier_weight_blocks' + # fpc_zero) replaces every block with zeros — else the current + # effective backend, because Rust and NumPy produce DIFFERENT draws + # from the same bit-generator state. (The n_psu < 2 case stamped + # "portable" at its early return above and never reaches here.) + _backend_independent = False + if _use_survey_bootstrap: + assert resolved_survey is not None + _fpc = getattr(resolved_survey, "fpc", None) + _n_psu = len(psu_ids) # bound above in the survey branch + _backend_independent = resolved_survey.strata is not None or ( + _fpc is not None and _n_psu / _fpc[0] >= 1.0 + ) + replay_backend = "portable" if _backend_independent else effective_weight_backend() + # Re-iterable stream: each column tile of the fused perturbation GEMM # below makes its own full pass over the bit-identical weight stream. weight_stream = ReplayableWeightStream(_make_weight_iter, rng) @@ -253,6 +307,23 @@ def _make() -> List[Tuple[Optional[np.ndarray], np.ndarray]]: with np.errstate(divide="ignore", invalid="ignore", over="ignore"): bootstrap_atts = original_atts[None, :] + perturbations + # Degenerate weight streams (census-FPC zeroes every block) leave + # every replicate row IDENTICAL — zero information. The per-cell + # columns are then exactly constant and NaN out via the stats + # guards, but the second-stage re-aggregations below reduce each + # ROW separately, and BLAS kernels may use different reduction + # orders for different row positions: identical rows in, rows + # differing by ~1 ULP out — enough to leak a roundoff SE past the + # zero/constant guards. When rows are identical, compute each + # reduction ONCE and broadcast, keeping the distribution exactly + # constant on every platform. + _rows_identical = self.n_bootstrap > 1 and bool(np.all(perturbations == perturbations[0:1])) + + def _replicate_reduce(cols: np.ndarray, w: np.ndarray) -> np.ndarray: + if _rows_identical: + return np.full(self.n_bootstrap, float(cols[0] @ w)) + return cols @ w + # Post-treatment mask — also exclude NaN effects post_mask = np.array( [ @@ -263,8 +334,8 @@ def _make() -> List[Tuple[Optional[np.ndarray], np.ndarray]]: post_indices = np.where(post_mask)[0] # Overall ATT: fixed-weight re-aggregation of perturbed cell ATTs. - # This matches CallawaySantAnna._run_multiplier_bootstrap - # (staggered_bootstrap.py:281). The analytical path includes a WIF + # This matches CallawaySantAnna._run_multiplier_bootstrap's overall + # aggregation (staggered_bootstrap.py). The analytical path includes a WIF # correction; bootstrap captures sampling variability through per-cell # EIF perturbation without re-estimating weights — this is standard # in both this library's CS implementation and the R did package. @@ -278,7 +349,7 @@ def _make() -> List[Tuple[Optional[np.ndarray], np.ndarray]]: agg_w = pg / pg.sum() if pg.sum() > 0 else np.ones(len(pg)) / len(pg) original_overall = float(np.sum(agg_w * original_atts[post_mask])) with np.errstate(divide="ignore", invalid="ignore", over="ignore"): - bootstrap_overall = bootstrap_atts[:, post_indices] @ agg_w + bootstrap_overall = _replicate_reduce(bootstrap_atts[:, post_indices], agg_w) # Event study: fixed-weight re-aggregation (same pattern as overall). # See note above re: WIF — analytical WIF is not needed in bootstrap. @@ -293,7 +364,7 @@ def _make() -> List[Tuple[Optional[np.ndarray], np.ndarray]]: idx = info["gt_indices"] w = info["weights"] with np.errstate(divide="ignore", invalid="ignore", over="ignore"): - bootstrap_event_study[e] = bootstrap_atts[:, idx] @ w + bootstrap_event_study[e] = _replicate_reduce(bootstrap_atts[:, idx], w) # Group aggregation bootstrap_group = None @@ -305,7 +376,7 @@ def _make() -> List[Tuple[Optional[np.ndarray], np.ndarray]]: idx = info["gt_indices"] w = info["weights"] with np.errstate(divide="ignore", invalid="ignore", over="ignore"): - bootstrap_group[g] = bootstrap_atts[:, idx] @ w + bootstrap_group[g] = _replicate_reduce(bootstrap_atts[:, idx], w) # Compute statistics gt_ses: Dict[Tuple[Any, Any], float] = {} @@ -360,7 +431,7 @@ def _make() -> List[Tuple[Optional[np.ndarray], np.ndarray]]: g_cis[g] = ci g_pvs[g] = pv - return EDiDBootstrapResults( + result = EDiDBootstrapResults( n_bootstrap=self.n_bootstrap, weight_type=self.bootstrap_weights, alpha=self.alpha, @@ -378,6 +449,9 @@ def _make() -> List[Tuple[Optional[np.ndarray], np.ndarray]]: group_effect_p_values=g_pvs, bootstrap_distribution=bootstrap_overall, ) + result._replay_bitgen_state = replay_bitgen_state + result._replay_backend = replay_backend + return result def _prepare_es_agg_boot( self, @@ -386,12 +460,20 @@ def _prepare_es_agg_boot( cohort_fractions: Dict[float, float], balance_e: Optional[int], ) -> Dict[int, Dict[str, Any]]: - """Prepare event-study aggregation info for bootstrap.""" + """Prepare event-study aggregation info for bootstrap. + + Horizons are keyed by the ANALYTICAL aggregator's expression + ``int(t - g)`` (truncation toward zero) so the percentile draws pool + exactly the cells each published analytical bucket pools — on + fractional-period panels a raw ``t - g`` key would attach a strict + sub-aggregate's inference to the pooled row (see the EfficientDiD + REGISTRY truncation Note). + """ effects_by_e: Dict[int, List[Tuple[int, float, float]]] = {} for j, (g, t) in enumerate(gt_pairs): if not np.isfinite(original_atts[j]): continue # Skip NaN cells - e = t - g + e = int(t - g) if e not in effects_by_e: effects_by_e[e] = [] effects_by_e[e].append((j, original_atts[j], cohort_fractions.get(g, 0.0))) @@ -400,14 +482,14 @@ def _prepare_es_agg_boot( groups_at_e = { gt_pairs[j][0] for j, (g, t) in enumerate(gt_pairs) - if t - g == balance_e and np.isfinite(original_atts[j]) + if int(t - g) == balance_e and np.isfinite(original_atts[j]) } balanced: Dict[int, List[Tuple[int, float, float]]] = {} for j, (g, t) in enumerate(gt_pairs): if g in groups_at_e: if not np.isfinite(original_atts[j]): continue # Skip NaN cells even in balanced set - e = t - g + e = int(t - g) if e not in balanced: balanced[e] = [] balanced[e].append((j, original_atts[j], cohort_fractions.get(g, 0.0))) @@ -473,14 +555,16 @@ def _build_nan_bootstrap_results( Used when survey-PSU bootstrap collapses to G<2 PSUs and would otherwise produce ≈0 SE from BLAS roundoff. Each NaN dict is keyed to the same (g,t)/event-time/group reductions the downstream - override loop at ``efficient_did.py:1078-1115`` expects, so the - override finds each key and overwrites analytical SE with NaN. + override appliers (``apply_bootstrap_event_study_overrides`` / + ``apply_bootstrap_group_overrides`` in ``bootstrap_utils``, plus the + inline per-(g,t) loop in ``EfficientDiD.fit``) expect, so each + override finds its key and overwrites analytical SE with NaN. Setting these dicts to ``None`` instead would let the analytical SE leak through, defeating the NaN-propagation contract; keying an empty dict would silently no-op the override for every key. ``event_study_ses``/``group_effect_ses`` are ``None`` (not empty) - when ``aggregate`` does not request them, matching the - ``is not None`` gates at ``efficient_did.py:1090, 1109``. + when ``aggregate`` does not request them, matching the appliers' + ``is not None`` gates. """ gt_pairs = list(group_time_effects.keys()) gt_ses: Dict[Tuple[Any, Any], float] = {gt: np.nan for gt in gt_pairs} diff --git a/diff_diff/efficient_did_results.py b/diff_diff/efficient_did_results.py index afa809ae..36ea67d9 100644 --- a/diff_diff/efficient_did_results.py +++ b/diff_diff/efficient_did_results.py @@ -13,7 +13,13 @@ import pandas as pd from diff_diff.aggregation import AggregationMixin, AggregationResult, build_total_relay_row +from diff_diff.bootstrap_chunking import effective_weight_backend +from diff_diff.bootstrap_utils import ( + apply_bootstrap_event_study_overrides, + apply_bootstrap_group_overrides, +) from diff_diff.efficient_did_aggregation import _EfficientAggregationMixin +from diff_diff.efficient_did_bootstrap import EfficientDiDBootstrapMixin from diff_diff.results import _format_survey_block, _get_significance_stars from diff_diff.results_base import BaseResults, build_event_study_surface @@ -85,6 +91,35 @@ def __init__( self._unit_level_weights = unit_level_weights +class _EDiDKitBootstrapAggregator(EfficientDiDBootstrapMixin): + """Value-bound host that replays the fit-time multiplier bootstrap. + + ``_run_multiplier_bootstrap`` reads exactly five attributes off its + host (the mixin's typed contract): ``n_bootstrap``, + ``bootstrap_weights``, ``alpha``, ``seed``, ``anticipation`` — all + carried here BY VALUE from the kit/spec so post-fit ``set_params`` or + attribute mutation on the estimator can never desynchronize a replay. + ``seed`` is None because the replay injects the fit-captured + bit-generator state directly. Warning attribution note: the engine's + fit-tuned stacklevels resolve into library frames under the deeper + ``aggregate()`` chain — accepted as cosmetic (the CS-recorded + decision). + """ + + def __init__( + self, + alpha: float, + anticipation: int, + n_bootstrap: int, + bootstrap_weights: str, + ) -> None: + self.alpha = alpha + self.anticipation = anticipation + self.n_bootstrap = n_bootstrap + self.bootstrap_weights = bootstrap_weights + self.seed = None # unused — the replay injects the captured state + + @dataclass class EfficientDiDResults(BaseResults, AggregationMixin): """ @@ -298,27 +333,72 @@ def _aggregate_compute( # Per-level bootstrap policy (v4-design section 6, converged with row # M-027): 'simple' is a bit-exact RELAY of the stored overall row - # faithful under any inference regime, bootstrap included - so it - # dispatches BEFORE the bootstrap gate. Only the RECOMPUTE levels - # below fail closed on bootstrapped fits. (This supersedes the - # uniform-conservatism decision recorded with M-023; its rationale - - # never publish analytical provenance beside percentile inference - - # is honored by the relay's NaN df column.) + # dispatches BEFORE the bootstrap branch. On bootstrapped fits the + # RECOMPUTE levels below REPLAY the fit-time multiplier bootstrap + # from the kit's BootstrapReplaySpec state (percentile inference, + # allclose to a fit-time aggregation); the M-023 rationale - never + # publish analytical provenance beside percentile inference - is + # honored by the percentile overrides + the NaN df channels. if level == "simple": return self._aggregate_simple_result(kit) if level == "total": return self._aggregate_total_result(kit) + boot_replay = None if self.bootstrap_results is not None: - raise NotImplementedError( - f"aggregate({level!r}) is not yet available on a bootstrapped " - "fit (n_bootstrap > 0): the per-horizon bootstrap draws are " - "not retained, so post-fit re-aggregation cannot replay " - "percentile inference and analytical inference would " - "misrepresent the fit. aggregate('simple') and, where " - "supported, aggregate('total') relay the stored " - "bootstrap inference and remain available; otherwise re-fit " - "with the aggregation you need, or use n_bootstrap=0." - ) + spec = getattr(kit, "bootstrap", None) + if spec is None or spec.bitgen_state is None: + raise NotImplementedError( + f"aggregate({level!r}): this bootstrapped result predates " + "the fit-time bootstrap replay state (its kit carries no " + "BootstrapReplaySpec) - refit with diff-diff >= 3.10 to " + "enable the post-fit replay. aggregate('simple') and, " + "where supported, aggregate('total') relay the stored " + "bootstrap inference and remain available." + ) + current_backend = effective_weight_backend() + if spec.backend not in ("portable", current_backend): + raise NotImplementedError( + f"aggregate({level!r}): this fit's bootstrap weights were " + f"generated under the {spec.backend!r} weight backend, but " + f"the current install uses {current_backend!r} - the two " + "backends produce DIFFERENT draws from the same RNG " + "state, so replaying here would publish a different " + "bootstrap realization beside the stored fit-time " + "inference. Re-fit under the current backend, or restore " + "the original one (DIFF_DIFF_BACKEND / the Rust " + "extension). aggregate('simple') and, where supported, " + "aggregate('total') relay the stored inference." + ) bk = dict(kit.bookkeeping) + if self.bootstrap_results is not None: + spec = kit.bootstrap + host = _EDiDKitBootstrapAggregator( + alpha=kit.alpha, + anticipation=kit.anticipation, + n_bootstrap=spec.n_bootstrap, + bootstrap_weights=spec.weight_type, + ) + # Per-call cost: O(n_bootstrap x n_units x n_gt) - the fused GEMM + # carries the n_gt per-cell EIF columns; ES/group targets + # re-aggregate cheaply from the dense (n_bootstrap, n_gt) matrix. + # No memoization (the recompute convention; the kit deliberately + # retains no draws). The engine re-derives the generation branch + # from the kit bookkeeping and re-emits the fit-time bootstrap + # warnings for the replayed configuration. + boot_replay = host._run_multiplier_bootstrap( + group_time_effects=bk["group_time_effects"], + eif_by_gt=kit.influence, + n_units=bk["n_units"], + aggregate=level, + balance_e=(balance_e if level == "event_study" else None), + treatment_groups=bk["treatment_groups"], + cohort_fractions=bk["cohort_fractions"], + cluster_indices=bk["cluster_indices"], + n_clusters=bk["n_clusters"], + resolved_survey=bk["resolved_survey_unit"], + unit_level_weights=bk["unit_level_weights"], + _replay_bitgen_state=spec.bitgen_state, + ) agg = _EDiDKitAggregator( alpha=kit.alpha, anticipation=kit.anticipation, @@ -337,6 +417,10 @@ def _aggregate_compute( cluster_indices=bk["cluster_indices"], n_clusters=bk["n_clusters"], ) + if boot_replay is not None: + # Same applier as fit-time (clears each row's df_used, so + # _group_effects_to_aggregation publishes an all-NaN df). + apply_bootstrap_group_overrides(effects, boot_replay, kit.alpha) return self._group_effects_to_aggregation(effects, kit) # level == "event_study" (the mixin validated the vocabulary) es = agg._aggregate_event_study( @@ -351,6 +435,13 @@ def _aggregate_compute( cluster_indices=bk["cluster_indices"], n_clusters=bk["n_clusters"], ) + if boot_replay is not None: + # Same applier as fit-time. The carrier below needs NO field + # clearing: this class has no vcov/cband/df fields for the ES + # surface (its published df column is all-NaN by construction), + # and the non-None bootstrap_results it retains keeps the + # container's inference provenance honest. + apply_bootstrap_event_study_overrides(es, boot_replay, kit.alpha) # Carrier + shared builder: EDiD is a _from_relative_dict producer, # so the recomputed dict rides the same route as the fit-time # surface. The carrier's survey_metadata is a COPY whose df_survey @@ -698,10 +789,10 @@ def to_dataframe(self, level: str = "group_time") -> pd.DataFrame: raise ValueError( "Event study effects not computed at fit time. Use " "results.aggregate('event_study') for the post-fit " - "event-study container (on bootstrapped fits, re-fit " - "with n_bootstrap=0 or use the deprecated fit-time " - "aggregate=); a result unpickled from a pre-3.9 " - "release carries no aggregation kit and must be refit." + "event-study container (bootstrapped fits replay the " + "fit-time multiplier bootstrap - no refit needed); a " + "result unpickled from a pre-3.9 release carries no " + "aggregation kit and must be refit." ) rows = [] for rel_t, data in sorted(self.event_study_effects.items()): @@ -723,10 +814,10 @@ def to_dataframe(self, level: str = "group_time") -> pd.DataFrame: raise ValueError( "Group effects not computed at fit time. Use " "results.aggregate('group') for the post-fit group " - "container (on bootstrapped fits, re-fit with " - "n_bootstrap=0 or use the deprecated fit-time " - "aggregate=); a result unpickled from a pre-3.9 " - "release carries no aggregation kit and must be refit." + "container (bootstrapped fits replay the fit-time " + "multiplier bootstrap - no refit needed); a result " + "unpickled from a pre-3.9 release carries no " + "aggregation kit and must be refit." ) rows = [] for group, data in sorted(self.group_effects.items()): diff --git a/diff_diff/guides/llms-full.txt b/diff_diff/guides/llms-full.txt index c16a7a42..4b13d380 100644 --- a/diff_diff/guides/llms-full.txt +++ b/diff_diff/guides/llms-full.txt @@ -1188,7 +1188,7 @@ results = edid.fit(data, outcome='y', unit='id', time='t', first_treat='first_treat') results.print_summary() # Aggregate post-fit (recomputed from retained EIFs; on bootstrapped fits -# the recompute levels raise while 'simple' relays the stored inference): +# the recompute levels REPLAY the fit-time multiplier bootstrap - no refit): es = results.aggregate('event_study') # EventStudyResults container grp = results.aggregate('group') # per-cohort AggregationResult print(grp.to_dataframe()) @@ -1898,7 +1898,7 @@ Each event study effect dict contains: `effect`, `se`, `t_stat`, `p_value`, `con | `n_clusters` | `int | None` | Number of effective clusters; `None` under survey designs and `None` under EfficientDiD's default unclustered fit | | `df_convention` | `str | None` | On the knob-carrying containers (`StackedDiDResults`, `ImputationDiDResults`, `WooldridgeDiDResults`, `LPDiDResults`, `SunAbrahamResults`): the configured df convention (3.9 / M-127); `StackedDiDResults` additionally carries `inference_df` (the overall-ATT df actually used) | -**Methods:** `summary()`, `print_summary()`, `to_dataframe()`, `to_dict()` (flat dict of headline aliases + `vcov_type` + conditional `cluster_name`/`n_clusters`/`n_bootstrap`/`inference_method`); `aggregate(type, weights=None, *, balance_e=None)` on the shipped post-fit adopters (`StackedDiDResults` views since 3.9/M-024; `EfficientDiDResults` recomputes from retained EIFs since 3.9/M-023; `ImputationDiDResults` and `TwoStageDiDResults` recompute from their PANEL-BACKED kits since 3.9/M-021/M-022 - on bootstrapped fits their recompute levels raise while `'simple'` and, where supported, `'total'` relay the stored bootstrap inference with a NaN df column (the per-level rule, converged with M-027); `'total'` (3.10) is the estimator-owned total incremental outcome on `CallawaySantAnnaResults`/`EfficientDiDResults`/`ImputationDiDResults`/`TwoStageDiDResults` - panel non-survey fits only, fails closed with the reason elsewhere; `ContinuousDiDResults` is MIXED since 3.9/M-025 - `'simple'`/`'dose'` are views over stored fields that work on any fit incl. bootstrapped, `'event_study'` recomputes from a pruned per-cell IF kit and raises on bootstrapped fits; `HeterogeneousAdoptionDiDResults` and `HeterogeneousAdoptionDiDEventStudyResults` are pure views since 3.9/M-027 - `'simple'` on the overall class, `'event_study'` on the event-study class, no kit, work on pickles from any release) +**Methods:** `summary()`, `print_summary()`, `to_dataframe()`, `to_dict()` (flat dict of headline aliases + `vcov_type` + conditional `cluster_name`/`n_clusters`/`n_bootstrap`/`inference_method`); `aggregate(type, weights=None, *, balance_e=None)` on the shipped post-fit adopters (`StackedDiDResults` views since 3.9/M-024; `EfficientDiDResults` recomputes from retained EIFs since 3.9/M-023 - on bootstrapped fits its recompute levels REPLAY the fit-time multiplier bootstrap (percentile inference; M-023); `ImputationDiDResults` and `TwoStageDiDResults` recompute from their PANEL-BACKED kits since 3.9/M-021/M-022 - on bootstrapped fits their recompute levels raise while `'simple'` and, where supported, `'total'` relay the stored bootstrap inference with a NaN df column (the per-level rule, converged with M-027); `'total'` (3.10) is the estimator-owned total incremental outcome on `CallawaySantAnnaResults`/`EfficientDiDResults`/`ImputationDiDResults`/`TwoStageDiDResults` - panel non-survey fits only, fails closed with the reason elsewhere; `ContinuousDiDResults` is MIXED since 3.9/M-025 - `'simple'`/`'dose'` are views over stored fields that work on any fit incl. bootstrapped, `'event_study'` recomputes from a pruned per-cell IF kit and raises on bootstrapped fits; `HeterogeneousAdoptionDiDResults` and `HeterogeneousAdoptionDiDEventStudyResults` are pure views since 3.9/M-027 - `'simple'` on the overall class, `'event_study'` on the event-study class, no kit, work on pickles from any release) ### ContinuousDiDResults diff --git a/diff_diff/guides/llms-practitioner.txt b/diff_diff/guides/llms-practitioner.txt index bd412718..d7b167ef 100644 --- a/diff_diff/guides/llms-practitioner.txt +++ b/diff_diff/guides/llms-practitioner.txt @@ -448,13 +448,13 @@ print(results.aggregate('event_study', balance_e=2).to_dataframe()) # NEW in 3.10 - the estimator-owned TOTAL incremental outcome (exact # relay C x overall; single row; panel non-survey fits only): print(results.aggregate('total').to_dataframe()) -# BOOTSTRAPPED fits: CS's recompute levels (event_study/group) REPLAY -# the fit-time multiplier bootstrap post-fit (percentile inference; no -# refit needed) — but they still RAISE on EfficientDiD/ImputationDiD/ -# TwoStageDiD, where aggregate('simple') and, where supported, -# aggregate('total') relay the stored bootstrap inference and the -# deprecated fit-time aggregation remains the ES/group route: -results = edid.fit(data, ..., aggregate='all') # EfficientDiD et al. only +# BOOTSTRAPPED fits: the CS and EfficientDiD recompute levels +# (event_study/group) REPLAY the fit-time multiplier bootstrap post-fit +# (percentile inference; no refit needed) — but they still RAISE on +# ImputationDiD/TwoStageDiD, where aggregate('simple') and, where +# supported, aggregate('total') relay the stored bootstrap inference and +# the deprecated fit-time aggregation remains the ES/group route: +results = imp.fit(data, ..., aggregate='all') # ImputationDiD/TwoStageDiD only ``` ### For ContinuousDiD (MIXED post-fit `aggregate()`, row M-025) diff --git a/diff_diff/guides/llms.txt b/diff_diff/guides/llms.txt index 01832f5d..f9d686af 100644 --- a/diff_diff/guides/llms.txt +++ b/diff_diff/guides/llms.txt @@ -21,7 +21,7 @@ diagnostic steps produces unreliable results. 4. **Choose estimator** — staggered adoption → CS/SA/BJS (NOT plain TWFE); few treated units → SDiD; factor confounding → TROP; simple 2x2 → DiD. Run `BaconDecomposition` to diagnose TWFE bias. 5. **Estimate** — `estimator.fit(data, ...)`. Always print the cluster count first and choose inference method based on the result (cluster-robust if >= 50 clusters, wild bootstrap if fewer — for DifferenceInDifferences pass `cluster=`; TwoWayFixedEffects auto-clusters at unit level). 6. **Sensitivity analysis** — `compute_honest_did(results)` for bounds under PT violations (MultiPeriodDiD, CS, or dCDH natively; the TwoWayFixedEffects `event_study=True` surface and a StackedDiD `results.aggregate('event_study')` container also admit - Stacked needs `kappa_pre >= 2` so estimated pre-periods exist), `run_all_placebo_tests()` for 2x2 falsification, specification comparisons for staggered designs. -7. **Heterogeneity** — CS: `results.aggregate('group')`/`.aggregate('event_study')` post-fit, no refit (fit-time `aggregate=`/`balance_e=` are deprecated since 3.9, removed in 4.0; `compute_honest_did` / `compute_pretrends_power` / `plot_event_study` all accept the post-fit `results.aggregate('event_study')` container directly; BOOTSTRAPPED CS fits included: the recompute levels `'event_study'`/`'group'` REPLAY the fit-time multiplier bootstrap from the kit's retained RNG state (percentile inference matching a fit-time aggregation; container carries vcov=None), while `.aggregate('simple')` and, where supported, `.aggregate('total')` relay the stored bootstrap inference (NaN df column); only pre-replay legacy pickles and cross-weight-backend artifacts fail closed with a refit message). NEW in 3.10: `.aggregate('total')` on CS/EfficientDiD/ImputationDiD/TwoStageDiD - the estimator-owned total incremental outcome (exact relay C x overall over the finite-masked complete-case support; single target='total' row; bootstrap-safe RELAY; panel non-survey fits only - repeated-cross-section and declared survey_design fits raise NotImplementedError with the reason); dCDH: `results.aggregate('event_study')`/`.aggregate('simple')` post-fit views (bootstrap fits included — pure views); SA: `results.event_study_effects`/`to_dataframe(level='cohort')`; StackedDiD: `results.aggregate('event_study')`/`.aggregate('simple')` post-fit views (the surface is ALWAYS computed at fit since 3.9 - row M-024 - and the container admits into `compute_honest_did`/`compute_pretrends_power` with `kappa_pre >= 2`); EDiD: `results.aggregate('event_study')`/`.aggregate('group')`/`.aggregate('simple')` post-fit, RECOMPUTED from retained EIFs (3.9, row M-023; fit-time `aggregate=`/`balance_e=` deprecated; on bootstrapped EDiD fits the recompute levels raise while `.aggregate('simple')` relays the stored bootstrap inference - use the fit-time aggregation for a bootstrapped ES/group surface; EDiD containers are NOT admitted into honest/pretrends - no joint ES covariance); BJS/TwoStageDiD: `results.aggregate('event_study')`/`.aggregate('group')`/`.aggregate('simple')` post-fit on ImputationDiD and TwoStageDiD too (3.9, rows M-021/M-022; recomputed from panel-backed kits, `balance_e=` on `aggregate('event_study')`; on bootstrapped fits the recompute levels raise while `.aggregate('simple')` relays the stored bootstrap inference - use the deprecated fit-time aggregation for a bootstrapped ES/group surface; their containers are not admitted into honest/pretrends - Imputation by design, TwoStage deferred pending a normalization derivation); CGBS continuous: ContinuousDiD is a MIXED adopter (3.9, row M-025) - `results.aggregate('dose')` (ATT(d)+ACRT(d) rows) and `.aggregate('simple')` (att+acrt rows) are views over the always-computed curves and work on ANY fit incl. bootstrapped, while `.aggregate('event_study')` recomputes the binarized event study from a pruned per-cell IF kit and raises on bootstrapped fits (use the deprecated fit-time `aggregate='eventstudy'` there until 4.0; its container is not admitted into honest/pretrends - no joint ES covariance and no reference normalization); HAD: `results.aggregate('simple')` (overall two-period fits; the target column carries the WAS estimand label) / `.aggregate('event_study')` (multi-period fits) - pure views, work on any fit (3.9, rows M-027/M-139; fit() selects the mode from the panel shape; HAD containers are not admitted into honest/pretrends - no joint cross-horizon covariance, deferred); subgroup re-estimation. +7. **Heterogeneity** — CS: `results.aggregate('group')`/`.aggregate('event_study')` post-fit, no refit (fit-time `aggregate=`/`balance_e=` are deprecated since 3.9, removed in 4.0; `compute_honest_did` / `compute_pretrends_power` / `plot_event_study` all accept the post-fit `results.aggregate('event_study')` container directly; BOOTSTRAPPED CS fits included: the recompute levels `'event_study'`/`'group'` REPLAY the fit-time multiplier bootstrap from the kit's retained RNG state (percentile inference matching a fit-time aggregation; container carries vcov=None), while `.aggregate('simple')` and, where supported, `.aggregate('total')` relay the stored bootstrap inference (NaN df column); only pre-replay legacy pickles and cross-weight-backend artifacts fail closed with a refit message). NEW in 3.10: `.aggregate('total')` on CS/EfficientDiD/ImputationDiD/TwoStageDiD - the estimator-owned total incremental outcome (exact relay C x overall over the finite-masked complete-case support; single target='total' row; bootstrap-safe RELAY; panel non-survey fits only - repeated-cross-section and declared survey_design fits raise NotImplementedError with the reason); dCDH: `results.aggregate('event_study')`/`.aggregate('simple')` post-fit views (bootstrap fits included — pure views); SA: `results.event_study_effects`/`to_dataframe(level='cohort')`; StackedDiD: `results.aggregate('event_study')`/`.aggregate('simple')` post-fit views (the surface is ALWAYS computed at fit since 3.9 - row M-024 - and the container admits into `compute_honest_did`/`compute_pretrends_power` with `kappa_pre >= 2`); EDiD: `results.aggregate('event_study')`/`.aggregate('group')`/`.aggregate('simple')` post-fit, RECOMPUTED from retained EIFs (3.9, row M-023; fit-time `aggregate=`/`balance_e=` deprecated; on bootstrapped EDiD fits the recompute levels REPLAY the fit-time multiplier bootstrap from the kit's retained RNG state (percentile inference matching a fit-time aggregation; only pre-replay legacy pickles and cross-weight-backend artifacts fail closed with a refit message) while `.aggregate('simple')` relays the stored bootstrap inference; EDiD containers are NOT admitted into honest/pretrends - no joint ES covariance); BJS/TwoStageDiD: `results.aggregate('event_study')`/`.aggregate('group')`/`.aggregate('simple')` post-fit on ImputationDiD and TwoStageDiD too (3.9, rows M-021/M-022; recomputed from panel-backed kits, `balance_e=` on `aggregate('event_study')`; on bootstrapped fits the recompute levels raise while `.aggregate('simple')` relays the stored bootstrap inference - use the deprecated fit-time aggregation for a bootstrapped ES/group surface; their containers are not admitted into honest/pretrends - Imputation by design, TwoStage deferred pending a normalization derivation); CGBS continuous: ContinuousDiD is a MIXED adopter (3.9, row M-025) - `results.aggregate('dose')` (ATT(d)+ACRT(d) rows) and `.aggregate('simple')` (att+acrt rows) are views over the always-computed curves and work on ANY fit incl. bootstrapped, while `.aggregate('event_study')` recomputes the binarized event study from a pruned per-cell IF kit and raises on bootstrapped fits (use the deprecated fit-time `aggregate='eventstudy'` there until 4.0; its container is not admitted into honest/pretrends - no joint ES covariance and no reference normalization); HAD: `results.aggregate('simple')` (overall two-period fits; the target column carries the WAS estimand label) / `.aggregate('event_study')` (multi-period fits) - pure views, work on any fit (3.9, rows M-027/M-139; fit() selects the mode from the panel shape; HAD containers are not admitted into honest/pretrends - no joint cross-horizon covariance, deferred); subgroup re-estimation. 8. **Robustness** — compare 2-3 estimators (CS vs SA vs BJS), MUST report with and without covariates (shows whether conditioning drives identification), present pre-trends and sensitivity bounds. After estimation, call `practitioner_next_steps(results)` for context-aware diff --git a/diff_diff/practitioner.py b/diff_diff/practitioner.py index 31e06f74..7dd9b575 100644 --- a/diff_diff/practitioner.py +++ b/diff_diff/practitioner.py @@ -982,21 +982,20 @@ def _handle_efficient(results: Any): "EfficientDiD aggregates post-fit from retained EIFs " "(M-023) - no refit needed." if getattr(results, "bootstrap_results", None) is None else "This fit is BOOTSTRAPPED: the post-fit event-study/group " - "recompute levels raise on bootstrap fits, while " - "aggregate('simple') and, where supported, " - "aggregate('total') relay the stored inference - " - "refit with the deprecated fit-time aggregation (or " - "n_bootstrap=0) to obtain the recomputed surfaces." + "recompute levels REPLAY the fit-time multiplier " + "bootstrap from the retained RNG state (percentile " + "inference, no refit needed), while aggregate('simple') " + "and, where supported, aggregate('total') relay the " + "stored inference." ), code=( "# Aggregate post-fit - no refit needed:\n" "print(results.aggregate('group').to_dataframe()) # Per-cohort ATTs\n" "print(results.aggregate('event_study').to_dataframe()) # Dynamic effects" if getattr(results, "bootstrap_results", None) is None - else "# Bootstrap fit: aggregate at fit time (deprecated kwarg):\n" - "results = edid.fit(data, ..., aggregate='all')\n" - "print(results.group_effects) # Per-cohort ATTs\n" - "print(results.event_study_effects) # Dynamic effects" + else "# Bootstrap fit: post-fit aggregation replays the fit-time bootstrap:\n" + "print(results.aggregate('group').to_dataframe()) # Per-cohort ATTs\n" + "print(results.aggregate('event_study').to_dataframe()) # Dynamic effects" ), priority="medium", # NON-STEPS key (the M-024 "sub_experiment_balance" lesson): diff --git a/diff_diff/results_base.py b/diff_diff/results_base.py index e116b858..b71a687e 100644 --- a/diff_diff/results_base.py +++ b/diff_diff/results_base.py @@ -209,7 +209,8 @@ class EventStudyResults(BaseResults): (e = t - g; first treated period at e=0) or ``"l1_first_switch"`` (de Chaisemartin-D'Haultfoeuille: instantaneous effect at l=1, placebos at negative keys). Horizons are documented, not - renumbered. + renumbered. (EfficientDiD buckets fractional-period horizons by + ``int(t - g)`` — see its REGISTRY truncation Note.) vcov : np.ndarray or None Full event-study variance-covariance matrix where the RESULT CONTAINER exposes one (e.g. CallawaySantAnna, SunAbraham, @@ -717,7 +718,7 @@ def summary(self, alpha: Optional[float] = None) -> str: # materialize the surface (row M-024). "StackedDiDResults": "re-fit with diff-diff >= 3.9, which always computes the surface", "StaggeredTripleDiffResults": "refit with aggregate='event_study' (or 'all')", - "EfficientDiDResults": "call results.aggregate('event_study') (on a bootstrapped fit, re-fit with n_bootstrap=0 or the deprecated fit-time aggregate=)", + "EfficientDiDResults": "call results.aggregate('event_study')", "ContinuousDiDResults": ( "call results.aggregate('event_study') (on a bootstrapped fit, " "re-fit with n_bootstrap=0 or the deprecated fit-time aggregate=)" diff --git a/diff_diff/staggered.py b/diff_diff/staggered.py index c729e6fa..762a73ed 100644 --- a/diff_diff/staggered.py +++ b/diff_diff/staggered.py @@ -17,6 +17,10 @@ AggregationKit, BootstrapReplaySpec, ) +from diff_diff.bootstrap_utils import ( + apply_bootstrap_event_study_overrides, + apply_bootstrap_group_overrides, +) from diff_diff.linalg import ( _check_propensity_diagnostics, _detect_rank_deficiency, @@ -32,8 +36,6 @@ from diff_diff.staggered_bootstrap import ( CallawaySantAnnaBootstrapMixin, CSBootstrapResults, - apply_bootstrap_event_study_overrides, - apply_bootstrap_group_overrides, apply_cband_conf_ints, ) diff --git a/diff_diff/staggered_bootstrap.py b/diff_diff/staggered_bootstrap.py index 7bfef5b1..4e3ea6c3 100644 --- a/diff_diff/staggered_bootstrap.py +++ b/diff_diff/staggered_bootstrap.py @@ -32,7 +32,6 @@ from diff_diff.bootstrap_utils import ( compute_percentile_ci as _compute_percentile_ci_func, ) -from diff_diff.utils import safe_inference_batch if TYPE_CHECKING: import pandas as pd @@ -599,13 +598,37 @@ def _make_weight_iter( # Group aggregation: fixed-weight re-aggregation of the completed # perturbed cell draws (matches at reassociation level). + # Degenerate weight streams (census-FPC zeroes every block) leave + # every replicate row of the cell draws IDENTICAL; this per-row + # matvec can then reduce different row positions in different + # BLAS orders — identical rows in, ~1-ULP-different rows out — + # leaking a roundoff SE past the zero/constant guards. When rows + # are identical, compute each reduction ONCE and broadcast so + # the group distribution is exactly constant on every platform + # (the fused-GEMM cell/overall/ES columns are exact and need no + # such handling). bootstrap_group: Optional[Dict[Any, np.ndarray]] = None if group_agg_info is not None: - bootstrap_group = { - g: bootstrap_atts_gt[:, group_agg_info[g]["gt_indices"]] - @ group_agg_info[g]["weights"] - for g in group_list - } + _gt_rows_identical = self.n_bootstrap > 1 and bool( + np.all(bootstrap_atts_gt == bootstrap_atts_gt[0:1]) + ) + if _gt_rows_identical: + bootstrap_group = { + g: np.full( + self.n_bootstrap, + float( + bootstrap_atts_gt[0, group_agg_info[g]["gt_indices"]] + @ group_agg_info[g]["weights"] + ), + ) + for g in group_list + } + else: + bootstrap_group = { + g: bootstrap_atts_gt[:, group_agg_info[g]["gt_indices"]] + @ group_agg_info[g]["weights"] + for g in group_list + } # Batch compute bootstrap statistics for ATT(g,t) batch_ses, batch_ci_lo, batch_ci_hi, batch_pv = _compute_effect_bootstrap_stats_batch_func( @@ -1011,83 +1034,11 @@ def _compute_effect_bootstrap_stats( ) -# ============================================================================= -# Bootstrap override helpers (shared by fit and the post-fit replay) -# ============================================================================= -# Extracted verbatim from CallawaySantAnna.fit()'s inline blocks so the -# post-fit aggregate() replay applies EXACTLY the same percentile overrides -# the fit-time path applies — one implementation, no twin drift. (The -# deprecated StaggeredTripleDifference keeps its OWN copy of the group -# replacement loop; unifying it is sequenced with the M-014 container port.) -# Note on warning attribution: when these run under the post-fit replay the -# engine's fit-tuned stacklevels resolve into library frames rather than the -# user's aggregate() call — accepted as cosmetic (recorded decision). - - -def apply_bootstrap_event_study_overrides( - event_study_effects: Optional[Dict[int, Dict[str, Any]]], - bootstrap_results: CSBootstrapResults, - alpha: float, -) -> None: - """Overwrite per-event-time se/CI/p with percentile-bootstrap values. - - Mutates ``event_study_effects`` in place; t is recomputed from the - percentile SE via ``safe_inference_batch``. No-op when either side has - no event-study surface. - """ - if ( - event_study_effects is not None - and bootstrap_results.event_study_ses is not None - and bootstrap_results.event_study_cis is not None - and bootstrap_results.event_study_p_values is not None - ): - es_keys = [e for e in event_study_effects if e in bootstrap_results.event_study_ses] - if es_keys: - es_effects_arr = np.array([float(event_study_effects[e]["effect"]) for e in es_keys]) - es_ses_arr = np.array([float(bootstrap_results.event_study_ses[e]) for e in es_keys]) - es_t_stats, _, _, _ = safe_inference_batch(es_effects_arr, es_ses_arr, alpha=alpha) - for idx, e in enumerate(es_keys): - event_study_effects[e]["se"] = bootstrap_results.event_study_ses[e] - event_study_effects[e]["conf_int"] = bootstrap_results.event_study_cis[e] - event_study_effects[e]["p_value"] = bootstrap_results.event_study_p_values[e] - event_study_effects[e]["t_stat"] = float(es_t_stats[idx]) - - -def apply_bootstrap_group_overrides( - group_effects: Optional[Dict[Any, Dict[str, Any]]], - bootstrap_results: CSBootstrapResults, - alpha: float, -) -> None: - """Overwrite per-group se/CI/p with percentile-bootstrap values. - - Mutates ``group_effects`` in place and clears each row's ``df_used`` - (the percentile inference never used the analytical df, so keeping it - would claim a t-reference that governed nothing). No-op when either - side has no group surface. - """ - if ( - group_effects is not None - and bootstrap_results.group_effect_ses is not None - and bootstrap_results.group_effect_cis is not None - and bootstrap_results.group_effect_p_values is not None - ): - grp_keys = [g for g in group_effects if g in bootstrap_results.group_effect_ses] - if grp_keys: - grp_effects_arr = np.array([float(group_effects[g]["effect"]) for g in grp_keys]) - grp_ses_arr = np.array([float(bootstrap_results.group_effect_ses[g]) for g in grp_keys]) - grp_t_stats, _, _, _ = safe_inference_batch(grp_effects_arr, grp_ses_arr, alpha=alpha) - for idx, g in enumerate(grp_keys): - group_effects[g]["se"] = bootstrap_results.group_effect_ses[g] - group_effects[g]["conf_int"] = bootstrap_results.group_effect_cis[g] - group_effects[g]["p_value"] = bootstrap_results.group_effect_p_values[g] - group_effects[g]["t_stat"] = float(grp_t_stats[idx]) - # Same clearing rule the ES df provenance follows: these - # se/p/CI are now percentile-bootstrap values that never used - # the analytical df, so keeping df_used would claim a - # t-reference that governed nothing. - group_effects[g]["df_used"] = None - - +# The shared percentile-override appliers (apply_bootstrap_event_study_ +# overrides / apply_bootstrap_group_overrides) live in +# diff_diff.bootstrap_utils, consumed by both the CallawaySantAnna and +# EfficientDiD fit paths and their post-fit replays. Only the CS-specific +# sup-t band applier remains here. def apply_cband_conf_ints( event_study_effects: Optional[Dict[int, Dict[str, Any]]], cband_crit_value: Optional[float], diff --git a/diff_diff/staggered_results.py b/diff_diff/staggered_results.py index d515e7f9..c856e060 100644 --- a/diff_diff/staggered_results.py +++ b/diff_diff/staggered_results.py @@ -18,6 +18,10 @@ resolve_inference_df, ) from diff_diff.bootstrap_chunking import effective_weight_backend +from diff_diff.bootstrap_utils import ( + apply_bootstrap_event_study_overrides, + apply_bootstrap_group_overrides, +) from diff_diff.results import _format_survey_block, _get_significance_stars from diff_diff.results_base import BaseResults, build_event_study_surface from diff_diff.staggered_aggregation import ( @@ -27,8 +31,6 @@ from diff_diff.staggered_bootstrap import ( CallawaySantAnnaBootstrapMixin, CSBootstrapResults, - apply_bootstrap_event_study_overrides, - apply_bootstrap_group_overrides, apply_cband_conf_ints, ) diff --git a/docs/methodology/REGISTRY.md b/docs/methodology/REGISTRY.md index 3dfb9788..735134e1 100644 --- a/docs/methodology/REGISTRY.md +++ b/docs/methodology/REGISTRY.md @@ -1173,7 +1173,7 @@ Dynamic placebos `DID^{pl}_l` look backward from each group's reference period, - **Note (Phase 2 cost-benefit delta SE):** When `L_max >= 2`, `overall_att` holds the cost-benefit `delta`. Its SE is computed via the delta method from per-horizon SEs: `SE(delta) = sqrt(sum w_l^2 * SE(DID_l)^2)`, treating horizons as independent (conservative under Assumption 8). When bootstrap is enabled, per-horizon bootstrap SEs flow through the delta-method formula, so `overall_se` reflects bootstrap-derived per-horizon uncertainty but the delta aggregation itself uses normal-theory (not bootstrap percentile). This is an intentional exception to the general bootstrap-inference-surface contract: `overall_p_value` and `overall_conf_int` for `delta` use `safe_inference(delta, delta_se)`, not percentile bootstrap, because the delta is a derived aggregate rather than a directly bootstrapped estimand. -- **Note (post-fit `aggregate()` is a view - row M-026):** `results.aggregate('event_study')` and `results.aggregate('simple')` are pure VIEWS over stored fields - nothing is recomputed. `'event_study'` returns the unified `EventStudyResults` container (Phase-1 `L_max=None` fits: the 2-row l=1 view; `L_max >= 1`: the multi-horizon `l1_first_switch` surface); `'simple'` a one-row `AggregationResult` relaying `overall_att/se/t/p/CI` bit-exactly with the estimand-aware `target` label (DID_M / DID_1 / delta, or the trends-linear first-difference label whose overall row is all-NaN by design). Because it is a view, BOOTSTRAP FITS ARE PERMITTED - the library-wide per-level relay rule (since M-027 the recompute adopters' 'simple' relays are permitted on bootstrap fits too, while their kit-based recompute levels stay closed - a recompute could silently substitute analytical inference); here each row relays exactly the inference the fit stored, including the delta's analytical-with-df numbers under bootstrap per the `Note (Phase 2 cost-benefit delta SE)` above (the view's `df` column resolves from the actual inference path, not the bootstrap-cleared `event_study_df` channel). The dCDH event-study CONTAINER is deliberately rejected by `compute_honest_did`/`compute_pretrends_power` - its l1 placebo semantics need HonestDiD's native dCDH branch (mandatory reinterpretation warning + horizon trimming); pass the results object itself. +- **Note (post-fit `aggregate()` is a view - row M-026):** `results.aggregate('event_study')` and `results.aggregate('simple')` are pure VIEWS over stored fields - nothing is recomputed. `'event_study'` returns the unified `EventStudyResults` container (Phase-1 `L_max=None` fits: the 2-row l=1 view; `L_max >= 1`: the multi-horizon `l1_first_switch` surface); `'simple'` a one-row `AggregationResult` relaying `overall_att/se/t/p/CI` bit-exactly with the estimand-aware `target` label (DID_M / DID_1 / delta, or the trends-linear first-difference label whose overall row is all-NaN by design). Because it is a view, BOOTSTRAP FITS ARE PERMITTED - the library-wide per-level relay rule (since M-027 the recompute adopters' 'simple' relays are permitted on bootstrap fits too; their kit-based recompute levels REPLAY the fit-time multiplier bootstrap on CallawaySantAnna and EfficientDiD and stay closed on the remaining adopters - a recompute must never silently substitute analytical inference); here each row relays exactly the inference the fit stored, including the delta's analytical-with-df numbers under bootstrap per the `Note (Phase 2 cost-benefit delta SE)` above (the view's `df` column resolves from the actual inference path, not the bootstrap-cleared `event_study_df` channel). The dCDH event-study CONTAINER is deliberately rejected by `compute_honest_did`/`compute_pretrends_power` - its l1 placebo semantics need HonestDiD's native dCDH branch (mandatory reinterpretation warning + horizon trimming); pass the results object itself. - **Note (dynamic placebo SE - library extension):** Dynamic placebos `DID^{pl}_l` (negative horizons in `placebo_event_study`) now have analytical SE and bootstrap SE when `L_max >= 1`. The placebo IF uses the same cohort-recentered structure as positive horizons, applied to backward outcome differences `Y_{g, F_g-1-l} - Y_{g, F_g-1}` with the dual-eligibility control pool (forward + backward observation required). The paper's Theorem 1 variance result is stated for `DID_l`, not `DID^{pl}_l` - this extension applies the same IF/variance structure to the placebo estimand as a library enhancement. The single-period placebo `DID_M^pl` (`L_max=None`) retains NaN SE because the per-period aggregation path has no IF derivation. @@ -1578,7 +1578,7 @@ where `q_{g,e} = pi_g / sum_{g' in G_{trt,e}} pi_{g'}`. - **Duplicate rows**: Duplicate `(unit, time)` entries are rejected with `ValueError`. The estimator requires exactly one observation per unit-period - **Note:** PT-All index set includes g'=∞ (never-treated) as a candidate comparison group and excludes period_1 for all g'. When g'=∞, the second and third Eq 3.9 terms telescope so all (∞, t_pre) moments produce the same 2x2 DiD value; these redundant moments are damped by the default Omega* ridge (see the Omega* ridge Note below; at `omega_ridge=0`, by the legacy pseudoinverse). When t_pre = period_1, the third term degenerates to E[Y_1 - Y_1 | G=g'] = 0 for any g', adding no information. Valid pairs require only t_pre < g' (pre-treatment for comparison group), not t_pre < g. Same-group pairs (g'=g) are valid and contribute overidentifying moments (Equation 3.9) — except the degenerate PRE-treatment self-pair (g'=g, t_pre=t), which the default ridge path excludes (see the ridge Note). - **Note:** Omega* ridge regularization (`omega_ridge`, default 1e-6; v3.7) — a documented refinement of the Omega* inversion in Eq 3.5/3.13/4.3, in the space the paper leaves open (it assumes Omega* is invertible and does not prescribe finite-sample handling of singularity). Under PT-All the overidentified moment set makes the SAMPLE Omega* numerically singular by construction (the telescoping (∞, t_pre) moments above plus near-duplicate cross-cohort moments): measured cond 1e17–1e22 for 100% of units on realistic panels, with a spectrum of one exact-null direction (the degenerate self-pair, below) plus a cluster of statistically-null directions at relative eigenvalue 1e-5–1e-8, far below the ~1e-2 sampling noise of the covariance entries — and no clean spectral gap. The prior pseudoinverse fallback therefore sat on the rcond-cutoff cliff: ANY floating-point-level change (BLAS reordering, platform change, a 1-ulp data perturbation) redrew per-cell weights and moved per-cell ATT(g,t) at up to ~1e-2 relative (measured 1.2e-4 per-cell rel for a 1-ulp outcome perturbation on the pre-v3.7 code; overall ATT stable at ~1e-9 because the redraw averages out across units and cells). The ridge solves `(Omega* + omega_ridge * max(trace/H, 0) * I) x = 1` (trace-scaled: scale-equivariant, O(H), no SVD) — a smooth regularization with sensitivity bounded by 1/omega_ridge, not a cutoff. **Why the deviation is safe:** every moment individually identifies ATT(g,t), so any fixed weights summing to 1 keep the estimator consistent; the ridge trades a numerically ill-defined efficiency optimum for a stable one, and the plug-in EIF treatment of estimated weights (Remark 4.2) is unaffected. **Calibration evidence (2026-07):** default 1e-6 chosen by 1-ulp stability (per-cell rel <= 3e-9 vs 1.2e-4 legacy; candidates 1e-4/1e-6/1e-8 all pass the <=1e-6 target) with the HRS Table 6 anchors as a hard gate (all anchors within the published-value tolerance at every candidate; worst deviation 0.0257 SE, unchanged from legacy; shift <= 0.0001 SE); Monte Carlo on covariate-confounded DGPs shows bias/RMSE/SE-calibration/coverage statistically identical to legacy (ridge marginally better point metrics). **One-time value shift:** per-cell and event-study values on the covariate path move within the pre-existing indeterminacy band when upgrading (worst observed post-treatment cell shift ~0.6 of its own SE at n=500, shrinking with n: overall-ATT shift 1.6e-2 → 4e-3 → 1.1e-3 rel at n=500/1k/2k); the no-covariates path is essentially unchanged (~1e-7). `omega_ridge=0` restores the ENTIRE legacy code path bit-for-bit (both the omega constructions and the inv/pinv weights, including per-cell condition-number warnings and the legacy O(n^2 H^2) runtime). **Degenerate self-pair:** for PRE-treatment cells (t < g), the pair (g'=g, t_pre=t) telescopes to the identically-zero moment 0=0 (the exact-null Omega* direction). The legacy pseudoinverse truncated it, spreading weight over noisy moments (spurious pre-treatment placebos of ~5% of the effect size, pure noise amplification); a naive ridge would instead load all weight on the zero-variance moment, making placebos deterministically zero and silently disabling the pre-trend diagnostic. The default ridge path therefore drops this zero-information pair (fit-level filter; post-treatment cells never contain it), restoring honest data-driven placebos; `omega_ridge=0` keeps the legacy pair set. **Warnings/diagnostics:** the no-covariates path still computes per-cell condition numbers (cheap at (H,H)) for `results.omega_condition_numbers` and consolidates cells with cond > 1e12 into ONE fit-level warning (count + max cond) instead of the legacy per-cell pseudoinverse warnings; the covariate path intentionally computes NO per-unit condition numbers — they would cost exactly the per-unit SVDs the v3.7 rewrite removes, near-singularity there is structural (~always true under PT-All, so a warning would be always-on noise rather than signal), and the ridge handles it by design (the legacy covariate path likewise had no per-unit diagnostics). The scalability warning threshold moved from n > 5000 (legacy per-cell warning) to n > 50,000 (one fit-level warning; the kernel stage is still intrinsically O(n^2) but with a ~100x lower constant and tile-bounded memory). **Cross-cell table hoisting (2026-07 follow-up):** the tiled implementation now builds the kernel-covariance tables once per comparison group per unit-tile instead of per (g, t) cell — every Omega* term is `s_group * KCov(Y_u1 - Y_v1, Y_u2 - Y_v2 | group)`, keyed only by wide-outcome columns, so the per-cell H(H+1)/2-pair tables dedup to distinct product columns per group (~26x fewer kernel GEMM columns on a PT-All fit), and the kernel weight matrices are built and freed one group at a time (the tile memory budget is governed by the largest single group rather than the sum, giving proportionally fatter tiles at large n). Each cell's Omega* is then gathered from the group tables in the same per-entry operation order as the per-cell construction (value-exact, locked by test); this is a pure implementation change — measured results move only at floating-point reassociation level (post-treatment cells ~1e-12 relative, overall ATT ~1e-13; the no-covariates path is byte-identical), and `omega_ridge=0` still routes the entire legacy path. **Rust-backend batched-Cholesky ridge solve (2026-07 follow-up):** on the Rust backend the batched ridge solve `(Omega* + lam * max(trace/H, 0) * I) x = 1` dispatches to a batched Cholesky kernel (the ridged Omega* is SPD by construction; a non-SPD row — measured zero on realistic panels — falls back to LU in-kernel, and any non-finite row is recomputed through the exact legacy numpy chain including its pseudoinverse backstop, so edge-case semantics are unchanged), parallelized over units. This too is a pure implementation change: Cholesky and LU solutions differ only at the condition-bounded floating-point level, and measured results move at reassociation level (post-treatment cells ~2e-12 relative, overall ATT ~1e-13); the pure-Python backend is byte-identical, and `omega_ridge=0` never reaches the kernel. -- **Note:** Bootstrap aggregation uses fixed cohort-size weights for overall/event-study reaggregation, matching the CallawaySantAnna bootstrap pattern (staggered_bootstrap.py:281 computes `bootstrap_overall = bootstrap_atts_gt[:, post_indices] @ weights`; L297 uses the same fixed-weight pattern for event study). The analytical path includes a WIF correction; fixed-weight bootstrap captures the same sampling variability through per-cell EIF perturbation without re-estimating aggregation weights, consistent with both the library's CS implementation and the R `did` package. +- **Note:** Bootstrap aggregation uses fixed cohort-size weights for overall/event-study reaggregation, matching the CallawaySantAnna bootstrap pattern (`_run_multiplier_bootstrap`'s overall aggregation in `staggered_bootstrap.py` computes `bootstrap_overall = bootstrap_atts_gt[:, post_indices] @ weights`; its event-study block uses the same fixed-weight pattern). The analytical path includes a WIF correction; fixed-weight bootstrap captures the same sampling variability through per-cell EIF perturbation without re-estimating aggregation weights, consistent with both the library's CS implementation and the R `did` package. - **Overall ATT convention**: The library's `overall_att` uses cohort-size-weighted averaging of post-treatment (g,t) cells, matching the CallawaySantAnna simple aggregation. This differs from the paper's ES_avg (Eq 2.3), which uniformly averages over event-time horizons. ES_avg can be computed from event study output as `mean(event_study_effects[e]["effect"] for e >= 0)` *Algorithm (two-step semiparametric estimation, Section 4):* @@ -1637,7 +1637,8 @@ where `q_{g,e} = pi_g / sum_{g' in G_{trt,e}} pi_{g'}`. - **Note:** `vcov_type` is permanently narrow to `{"hc1"}` per the Chen-Sant'Anna-Xie (2025) EIF-based variance achieving the semiparametric efficiency bound. Analytical-sandwich families `{classical, hc2, hc2_bm}` are rejected at `__init__` — the per-unit EIF aggregation has no equivalent single design matrix on which hat-matrix leverage or Bell-McCaffrey Satterthwaite DOF can be defined. `cluster=` invokes Liang-Zeger CR1 on cluster-aggregated EIF (`_compute_se_from_eif` with `cluster_indices`); `survey_design=` invokes TSL on the combined IF (`_compute_survey_eif_se`); both live in `diff_diff/efficient_did_aggregation.py` since the M-023 post-fit aggregate() extraction. `vcov_type='conley'` deferred to the EfficientDiD Conley follow-up row in DEFERRED.md. - **Note:** Default `cluster=None` (no survey design) renders summary label "HC1 heteroskedasticity-robust" because the per-unit EIF SE `sqrt(mean(EIF²)/n)` is methodologically HC1-style (no Liang-Zeger G/(G-1) finite-sample correction). `EfficientDiDResults.cluster_name` and `n_clusters` stay None under unclustered fits. This diverges from `ImputationDiD` which auto-clusters at unit per Borusyak-Jaravel-Spiess (2024) Theorem 3 — there the default summary renders the CR1 unit-clustered label. - **Note:** `set_params(vcov_type=bad)` raises immediately on EVERY estimator: since the shared `BaseEstimator` mixin (`diff_diff/_base.py`, v4 2(c)-i), `set_params` validates transactionally by constructor probe re-init, so it enforces exactly `__init__`'s validation, eagerly, library-wide. The former split — EfficientDiD eager vs `ImputationDiD`/`TripleDifference`/`CallawaySantAnna` (and six more: SunAbraham, StackedDiD, StaggeredTripleDifference, SpilloverDiD, TROP, PreTrendsPower) accepting constructor-rejected values until `fit()` — is retired; the fit-time re-validation layers remain as a second check against DIRECT attribute mutation (`est.vcov_type = ...`), which no setter can see. -- **Note (post-fit aggregate() - rows M-023/M-120):** `fit(aggregate=, balance_e=)` is deprecated (3.9; removed 4.0; joint FutureWarning, warn-and-still-work) in favor of post-fit `EfficientDiDResults.aggregate(type, balance_e=)` - a LAZY RECOMPUTING KIT (the CallawaySantAnna class, not a StackedDiD/dCDH view relay): `fit()` computes nothing extra, the results object retains an `AggregationKit`, and `aggregate('event_study'/'group', balance_e=)` re-runs the extracted `_EfficientAggregationMixin` aggregators on a throwaway host while `aggregate('simple')` relays the stored overall row bit-exact. (a) RETAINED BUFFERS (memory contract; phrased as maxima - optional design fields stay None when unsupplied): the per-(g,t) EIF dict, O(n_units x n_gt), the dominant payload - retained on EVERY fit regardless of `store_eif`, which since 3.9 governs only the public `influence_functions` field; `unit_cohorts` (cohort labels), `unit_level_weights`, factorized cluster codes - O(n_units) each; on ordinary (TSL) survey fits the unit-level `ResolvedSurveyDesign` adds `weights` plus, where supplied, `strata`/`psu`/`fpc` (factorized int codes / float values, never raw labels) - up to four O(n_units) arrays; on replicate designs it adds the O(n_units x n_replicates) replicate matrix plus, where supplied, `replicate_strata`/`replicate_rscales` (O(n_replicates)); per-row dict SNAPSHOTS of `group_time_effects` plus copies of the `groups`/`time_periods` lists and the scalar `pt_assumption`/`n_treated+n_control` provenance (aggregate() recomputes exclusively from these private snapshots, never from the mutable public result fields - a user edit of the public rows cannot mix altered point estimates with the retained EIF variance); scalars `n_units`, `cohort_fractions`, and the POST-OVERALL `df_survey` snapshot (captured after the overall inference and before the ES/group gates: the group pass can degenerate the working df to None on replicate designs with `n_valid <= 1`, and every fit-time aggregation seeds from the post-overall value, so recompute replays the exact seed). The data-minimization guarantee is scoped to unit identifiers - no unit-label container is retained. (b) `balance_e` uses the ANCHOR-HORIZON rule (keep cohorts with a finite effect at `e == balance_e`, then retain all their horizons) - the SAME rule CallawaySantAnna uses, divergent only from ImputationDiD/TwoStageDiD's balanced-window rule; an anchor no cohort reaches warns and yields a legal zero-row container. (c) BOOTSTRAP fits: 'simple' RELAYS the stored overall row verbatim (percentile se/p/CI beside the finite safe_inference t) with a NaN df column, while the RECOMPUTE levels (ES/group) fail closed - per-horizon draws are not retained (exact-replay wiring is a TODO row). The prior uniform-conservatism BY-DECISION rule was superseded 2026-08-05 with the M-027 per-level convergence; its rationale - no level publishes analytical-provenance fields beside percentile inference - is honored by the relay's NaN df column. The fit-time bootstrap override still clears the group rows' `df_used` key. (d) CONTAINER ADMISSION into `compute_honest_did`/`compute_pretrends_power` is REJECTED BY DESIGN (both terminal TypeErrors state it): the primary ground is the absent joint event-study covariance (container `vcov=None`, all-NaN per-row df - the scalar `df_survey` channel is the container's only df provenance; the per-row hole is the tracked M-092-completion TODO row). Reference semantics are regime-dependent: under `pt_assumption="all"` there is NO reference row (universal first-period baseline; e=-1 is a genuine estimate); under `"post"` the per-cohort baseline cell is materialized as a mechanical zero anchor at `e = -1 - anticipation` whenever it is not the panel's first period, and the MEMBERSHIP-GATED `reference_period` property (the SunAbraham rule - never synthesized when the anchor cell was not estimated) marks it `is_reference` in the container and corrects `plot_event_study`'s inferred reference (previously the `-1` fallback) on PT-Post `anticipation>0` fits. (e) 'simple' relay conventions: `target="att"`, `n = n_treated_units + n_control_units` with `n_kind="units"` (DISJOINT by construction - `last_cohort` trimming reassigns before the counts, so a true total exists, unlike StackedDiD's overlapping sets), `df` = the post-overall snapshot (provenance-exact where `survey_metadata.df_survey` can diverge in the degenerate replicate state); 'group' relay: `n_kind="cells"`, `weight=None` (equal within-cohort weights, no cross-cohort mass), per-row `df_used` array captured at each row's `safe_inference` call (exact by construction; a stated divergence from CS's conservative-min scalar broadcast); 'event_study' rides the shared `_from_relative_dict` builder via a carrier whose `survey_metadata` copy carries the snapshot `df_survey`. (f) `aggregate('total')` (3.10): the estimator-owned total incremental outcome - an exact relay `C x overall` CONDITIONAL on the realized aggregation mass, with `C = sum(n_treated)` over the kept post-anticipation FINITE-effect cells of the kit's deep-copied `group_time_effects` snapshot (an exact integer sum; NEVER the routinely-non-integral `n_units x sum(cohort_fractions)` float product). Unweighted fits only: ANY declared `survey_design=` fails closed via the kit's `unit_level_weights`/`resolved_survey_unit` markers (weight-type-agnostic - unweighted psu-only and analytic fweight designs, whose resolved weights stay RAW, gate identically; EDiD never synthesizes an internal design, so the markers are exact declaration provenance and post-fit mutation of the public `survey_metadata` cannot bypass them). No keepers -> NaN mass -> all-NaN row (no re-warn); with finite `C` the relay is verbatim (inherited NaNs pass through; att/n never blanked). Bootstrap fits relay with a NaN df column (M-027); df carrier = the post-overall `df_survey` snapshot; container: single `target='total'` row, `n_kind='obs'`, the per-level n-semantics note in the CS total Note applies. MMM: the container is admitted with NO scale (see the MMM section); the survey/RC totals remainder and its att*dC variance term are the DEFERRED row. +- **Note (post-fit aggregate() - rows M-023/M-120):** `fit(aggregate=, balance_e=)` is deprecated (3.9; removed 4.0; joint FutureWarning, warn-and-still-work) in favor of post-fit `EfficientDiDResults.aggregate(type, balance_e=)` - a LAZY RECOMPUTING KIT (the CallawaySantAnna class, not a StackedDiD/dCDH view relay): `fit()` computes nothing extra, the results object retains an `AggregationKit`, and `aggregate('event_study'/'group', balance_e=)` re-runs the extracted `_EfficientAggregationMixin` aggregators on a throwaway host while `aggregate('simple')` relays the stored overall row bit-exact. (a) RETAINED BUFFERS (memory contract; phrased as maxima - optional design fields stay None when unsupplied): the per-(g,t) EIF dict, O(n_units x n_gt), the dominant payload - retained on EVERY fit regardless of `store_eif`, which since 3.9 governs only the public `influence_functions` field; `unit_cohorts` (cohort labels), `unit_level_weights`, factorized cluster codes - O(n_units) each; on ordinary (TSL) survey fits the unit-level `ResolvedSurveyDesign` adds `weights` plus, where supplied, `strata`/`psu`/`fpc` (factorized int codes / float values, never raw labels) - up to four O(n_units) arrays; on replicate designs it adds the O(n_units x n_replicates) replicate matrix plus, where supplied, `replicate_strata`/`replicate_rscales` (O(n_replicates)); per-row dict SNAPSHOTS of `group_time_effects` plus copies of the `groups`/`time_periods` lists and the scalar `pt_assumption`/`n_treated+n_control` provenance (aggregate() recomputes exclusively from these private snapshots, never from the mutable public result fields - a user edit of the public rows cannot mix altered point estimates with the retained EIF variance); scalars `n_units`, `cohort_fractions`, and the POST-OVERALL `df_survey` snapshot (captured after the overall inference and before the ES/group gates: the group pass can degenerate the working df to None on replicate designs with `n_valid <= 1`, and every fit-time aggregation seeds from the post-overall value, so recompute replays the exact seed). The data-minimization guarantee is scoped to unit identifiers - no unit-label container is retained. (b) `balance_e` uses the ANCHOR-HORIZON rule (keep cohorts with a finite effect at the anchor horizon, then retain all their horizons) - the same rule SHAPE CallawaySantAnna uses, with one keying-granularity difference (EDiD anchors on the `int(t - g)` bucket, CS on raw `t - g` - identical on integer-period panels; see the truncation Note below), divergent from ImputationDiD/TwoStageDiD's balanced-window rule; an anchor no cohort reaches warns and yields a legal zero-row container. (c) BOOTSTRAP fits: 'simple' (and 'total') RELAY the stored overall row verbatim (percentile se/p/CI beside the finite safe_inference t) with a NaN df column and never re-warn, while the RECOMPUTE levels (ES/group, balance_e included) REPLAY the fit-time multiplier bootstrap from the kit's `BootstrapReplaySpec` (the fit-captured RNG state + run params BY VALUE - seed=None fits replay; post-fit `set_params`/attribute mutation cannot alter it; pickles carry it): se/CI/t match a fit-time aggregation to BLAS reassociation (~1 ULP, `assert_allclose` - never bit-identity), the percentile p-value is a count statistic (compared at 2/n_bootstrap), and the replay publishes no analytical provenance (percentile overrides via the shared `bootstrap_utils` appliers; group rows' `df_used` cleared exactly as at fit time; EDiD has no sup-t cband). The spec stamps the weight backend at capture ('rust'/'numpy', or 'portable' for backend-independent branches - stratified survey generation, census-FPC zero weights, the single-PSU degenerate early return); a replay under a different backend fails closed naming both, and legacy pickles without the spec fail closed with a refit message. Replays RE-EMIT the fit-time bootstrap warnings for the replayed configuration (low n_bootstrap, single-PSU, empty balance_e anchor, non-finite-draw RuntimeWarnings); the relay levels stay silent. Per-call cost O(n_bootstrap x n_units x n_gt), no memoization. The M-027 rationale - no level publishes analytical-provenance fields beside percentile inference - is honored by the percentile overrides and NaN df channels. (d) CONTAINER ADMISSION into `compute_honest_did`/`compute_pretrends_power` is REJECTED BY DESIGN (both terminal TypeErrors state it): the primary ground is the absent joint event-study covariance (container `vcov=None`, all-NaN per-row df - the scalar `df_survey` channel is the container's only df provenance; the per-row hole is the tracked M-092-completion TODO row). Reference semantics are regime-dependent: under `pt_assumption="all"` there is NO reference row (universal first-period baseline; e=-1 is a genuine estimate); under `"post"` the per-cohort baseline cell is materialized as a mechanical zero anchor at `e = -1 - anticipation` whenever it is not the panel's first period, and the MEMBERSHIP-GATED `reference_period` property (the SunAbraham rule - never synthesized when the anchor cell was not estimated) marks it `is_reference` in the container and corrects `plot_event_study`'s inferred reference (previously the `-1` fallback) on PT-Post `anticipation>0` fits. (e) 'simple' relay conventions: `target="att"`, `n = n_treated_units + n_control_units` with `n_kind="units"` (DISJOINT by construction - `last_cohort` trimming reassigns before the counts, so a true total exists, unlike StackedDiD's overlapping sets), `df` = the post-overall snapshot (provenance-exact where `survey_metadata.df_survey` can diverge in the degenerate replicate state); 'group' relay: `n_kind="cells"`, `weight=None` (equal within-cohort weights, no cross-cohort mass), per-row `df_used` array captured at each row's `safe_inference` call (exact by construction; a stated divergence from CS's conservative-min scalar broadcast); 'event_study' rides the shared `_from_relative_dict` builder via a carrier whose `survey_metadata` copy carries the snapshot `df_survey`. (f) `aggregate('total')` (3.10): the estimator-owned total incremental outcome - an exact relay `C x overall` CONDITIONAL on the realized aggregation mass, with `C = sum(n_treated)` over the kept post-anticipation FINITE-effect cells of the kit's deep-copied `group_time_effects` snapshot (an exact integer sum; NEVER the routinely-non-integral `n_units x sum(cohort_fractions)` float product). Unweighted fits only: ANY declared `survey_design=` fails closed via the kit's `unit_level_weights`/`resolved_survey_unit` markers (weight-type-agnostic - unweighted psu-only and analytic fweight designs, whose resolved weights stay RAW, gate identically; EDiD never synthesizes an internal design, so the markers are exact declaration provenance and post-fit mutation of the public `survey_metadata` cannot bypass them). No keepers -> NaN mass -> all-NaN row (no re-warn); with finite `C` the relay is verbatim (inherited NaNs pass through; att/n never blanked). Bootstrap fits relay with a NaN df column (M-027); df carrier = the post-overall `df_survey` snapshot; container: single `target='total'` row, `n_kind='obs'`, the per-level n-semantics note in the CS total Note applies. MMM: the container is admitted with NO scale (see the MMM section); the survey/RC totals remainder and its att*dC variance term are the DEFERRED row. +- **Note (fractional-period truncation bucketing):** the event-study equation defines `ES(e)` at an exact relative time, but the implementation buckets horizons by `int(t - g)` - truncation TOWARD ZERO - on the analytical aggregator, the bootstrap ES prep (all three keying sites, the `balance_e` anchor filter included; aligned 2026-08), and `hausman_pretest`'s internal aggregation. On integer-period panels `int()` is the identity and nothing below applies. On fractional-period panels (accepted by `fit()` without an integrality check): (i) bucket 0 is DOUBLE-WIDTH, spanning `(-1, 1)`, so the pre-treatment horizon e=-0.5 pools into the post-treatment e=0 bucket; e=1.0/1.5 pool into bucket 1 and e=-1.5/-1.0 into bucket -1; (ii) under `pt_assumption="post"` the bucket at `-1 - anticipation` pools genuine estimated fractional pre-treatment horizons (e.g. raw e=-1.5) with the mechanical zero anchor, and the container's reference normalization then publishes the WHOLE pooled row as `att=0.0` with NaN inference (`is_reference=True`) - genuine fractional-horizon estimates are subsumed into the reference row (the fit-time `event_study_effects` dict keeps the pooled effect + percentile inference unrewritten; the rewrite is container-only); (iii) WEIGHTING is CELL-MASS within the bucket - the aggregator appends one `(effect, pi_g)` term per CELL and normalizes, so a cohort contributing k cells to a bucket carries k*pi_g mass, whereas the exact-relative-time equation has one term per cohort - while `n_groups` counts DISTINCT cohorts in the bucket (identity with the cell count on integer panels). Every aggregation that truncation-buckets a fractional horizon emits a UserWarning naming this Note. - **Note:** `anticipation` is validated at construction (non-negative integer; `bool` rejected) via the shared `utils.validate_anticipation`, and re-checked on the fit path (direct-mutation defense) — see the family-wide adoption note (ledger row [M-144]) in the TripleDifference staggered-mode section. --- @@ -6289,13 +6290,23 @@ ContinuousDiD, EfficientDiD): are NOT cross-backend reproducible: the Rust generator draws one base seed and row-seeds Xoshiro256++ absolutely, while the NumPy fallback consumes the PCG64 stream directly, so the SAME bit-generator state yields DIFFERENT (equally valid) draw matrices under the two - backends. `bootstrap_chunking.effective_weight_backend()` names the active one, and - CallawaySantAnna's post-fit bootstrap replay (row M-020) stamps it on the kit's - `BootstrapReplaySpec`, failing closed on a mismatch. Branches that never touch the Rust + backends. `bootstrap_chunking.effective_weight_backend()` names the active one, and the + CallawaySantAnna (row M-020) and EfficientDiD (row M-023) post-fit bootstrap replays + stamp it on their kits' `BootstrapReplaySpec`, failing closed on a mismatch. Branches that never touch the Rust generator — stratified/single-PSU survey generation and census-FPC zero weights — are stamped `"portable"` and replay under either backend. Within one backend the chunked weight stream remains bit-identical and replayable per column tile (`ReplayableWeightStream`); downstream GEMMs match to BLAS reassociation (~1 ULP). +- **Note (exactly-constant bootstrap distributions NaN out):** census-FPC zero-weight + draws leave every multiplier-bootstrap replicate at the original effect, an EXACTLY + CONSTANT distribution. `np.std` of a constant non-zero level can come back + tiny-positive from mean-subtraction roundoff, which would slip past a `se <= 0` + check and publish an astronomically large, silently "significant" t. The shared + percentile-statistic helpers (`compute_effect_bootstrap_stats` scalar and batch, + consumed by every multiplier-bootstrap engine) therefore detect + `max(draws) == min(draws)` and return the full NaN inference tuple with the zero-SE + RuntimeWarning — a defensive enhancement of the zero-SE-means-NaN contract, exact + by construction (genuinely varying draws are never affected). **Rao-Wu Rescaled Bootstrap** (SunAbraham, TROP): diff --git a/docs/migration-4.0.md b/docs/migration-4.0.md index 8fd48b60..a450347c 100644 --- a/docs/migration-4.0.md +++ b/docs/migration-4.0.md @@ -155,14 +155,15 @@ surface (nothing to derive), and ChaisemartinDHaultfoeuille's pre-period checks read `placebo_event_study` directly. ```{warning} -**Bootstrapped fits: CallawaySantAnna is covered; the other recompute adopters are not yet.** -On `CallawaySantAnna`, the post-fit recompute levels now REPLAY the fit-time multiplier -bootstrap from the fit-retained RNG state (percentile inference matching a fit-time -aggregation to floating-point reassociation) — no fit-time keyword needed; only pre-replay -legacy pickles and artifacts moved across the Rust/NumPy weight backend fail closed with a -refit message. On `ImputationDiD`, `TwoStageDiD`, `EfficientDiD` and `ContinuousDiD`, the -recompute levels still raise `NotImplementedError` when the fit used `n_bootstrap > 0` — -keep the fit-time call there for now and track their open `TODO.md` rows. +**Bootstrapped fits: CallawaySantAnna and EfficientDiD are covered; the remaining recompute adopters are not yet.** +On `CallawaySantAnna` and `EfficientDiD`, the post-fit recompute levels now REPLAY the +fit-time multiplier bootstrap from the fit-retained RNG state (percentile inference +matching a fit-time aggregation to floating-point reassociation) — no fit-time keyword +needed; only pre-replay legacy pickles and artifacts moved across the Rust/NumPy weight +backend fail closed with a refit message. On `ImputationDiD`, `TwoStageDiD` and +`ContinuousDiD`, the recompute levels still raise `NotImplementedError` when the fit used +`n_bootstrap > 0` — keep the fit-time call there for now and track their open `TODO.md` +rows. `aggregate("simple")` — and, on its four adopters, `aggregate("total")` (3.10) — does relay, and `StackedDiD`, `ChaisemartinDHaultfoeuille` and `HeterogeneousAdoptionDiD` are unaffected — their `aggregate()` is a pure view over stored fields. @@ -303,7 +304,7 @@ does not mean no action is required, so read the `Fix` cell. | M-020 | aggregate-postfit | `diff_diff:CallawaySantAnna.fit[aggregate]` | `diff_diff:CallawaySantAnnaResults.aggregate` | Move `aggregate=` off `fit()` onto post-fit `results.aggregate(...)`. Bootstrapped fits replay the fit-time bootstrap post-fit - see the aggregation section. | | M-021 | aggregate-postfit | `diff_diff:ImputationDiD.fit[aggregate]` | `diff_diff:ImputationDiDResults.aggregate` | Move `aggregate=` off `fit()` onto post-fit `results.aggregate(...)`. On a bootstrapped fit the recompute levels raise today - see the aggregation section. | | M-022 | aggregate-postfit | `diff_diff:TwoStageDiD.fit[aggregate]` | `diff_diff:TwoStageDiDResults.aggregate` | Move `aggregate=` off `fit()` onto post-fit `results.aggregate(...)`. On a bootstrapped fit the recompute levels raise today - see the aggregation section. | -| M-023 | aggregate-postfit | `diff_diff:EfficientDiD.fit[aggregate]` | `diff_diff:EfficientDiDResults.aggregate` | Move `aggregate=` off `fit()` onto post-fit `results.aggregate(...)`. On a bootstrapped fit the recompute levels raise today - see the aggregation section. | +| M-023 | aggregate-postfit | `diff_diff:EfficientDiD.fit[aggregate]` | `diff_diff:EfficientDiDResults.aggregate` | Move `aggregate=` off `fit()` onto post-fit `results.aggregate(...)`. Bootstrapped fits replay the fit-time bootstrap post-fit - see the aggregation section. | | M-024 | aggregate-postfit | `diff_diff:StackedDiD.fit[aggregate]` | `diff_diff:StackedDiDResults.aggregate` | Move `aggregate=` off `fit()` onto post-fit `results.aggregate(...)`. | | M-025 | aggregate-postfit | `diff_diff:ContinuousDiD.fit[aggregate]` | `diff_diff:ContinuousDiDResults.aggregate` | Move `aggregate=` off `fit()` onto post-fit `results.aggregate(...)`. On a bootstrapped fit the recompute levels raise today - see the aggregation section. | | M-026 | aggregate-postfit | `diff_diff:ChaisemartinDHaultfoeuille.fit[aggregate]` | `diff_diff:ChaisemartinDHaultfoeuilleResults.aggregate` | Move `aggregate=` off `fit()` onto post-fit `results.aggregate(...)`. | @@ -311,7 +312,7 @@ does not mean no action is required, so read the `Fix` cell. | M-117 | aggregate-postfit | `diff_diff:CallawaySantAnna.fit[balance_e]` | `diff_diff:CallawaySantAnnaResults.aggregate[balance_e]` | Move `balance_e=` off `fit()` onto post-fit `results.aggregate(...)`. Bootstrapped fits replay the fit-time bootstrap post-fit - see the aggregation section. | | M-118 | aggregate-postfit | `diff_diff:ImputationDiD.fit[balance_e]` | `diff_diff:ImputationDiDResults.aggregate[balance_e]` | Move `balance_e=` off `fit()` onto post-fit `results.aggregate(...)`. On a bootstrapped fit the recompute levels raise today - see the aggregation section. | | M-119 | aggregate-postfit | `diff_diff:TwoStageDiD.fit[balance_e]` | `diff_diff:TwoStageDiDResults.aggregate[balance_e]` | Move `balance_e=` off `fit()` onto post-fit `results.aggregate(...)`. On a bootstrapped fit the recompute levels raise today - see the aggregation section. | -| M-120 | aggregate-postfit | `diff_diff:EfficientDiD.fit[balance_e]` | `diff_diff:EfficientDiDResults.aggregate[balance_e]` | Move `balance_e=` off `fit()` onto post-fit `results.aggregate(...)`. On a bootstrapped fit the recompute levels raise today - see the aggregation section. | +| M-120 | aggregate-postfit | `diff_diff:EfficientDiD.fit[balance_e]` | `diff_diff:EfficientDiDResults.aggregate[balance_e]` | Move `balance_e=` off `fit()` onto post-fit `results.aggregate(...)`. Bootstrapped fits replay the fit-time bootstrap post-fit - see the aggregation section. | | M-139 | aggregate-postfit | `diff_diff:did_had_pretest_workflow[aggregate]` | — | Remove `aggregate=`; the battery is inferred from panel shape - two periods select the overall battery, more than two select the event-study battery. It is not selected post-fit (`HADPretestReport.aggregate` is a metadata field, not a method). | | M-060 | alias-table | `EventStudy` | — | Import `TwoWayFixedEffects(...).fit(..., event_study=True)` - the alias is dropped, not retargeted instead of the `EventStudy` alias. | | M-061 | alias-table | `QDiDResults` | — | Import `ChangesInChangesResults` instead of the `QDiDResults` alias. | diff --git a/docs/tutorials/15_efficient_did.ipynb b/docs/tutorials/15_efficient_did.ipynb index 6f601012..60df470c 100644 --- a/docs/tutorials/15_efficient_did.ipynb +++ b/docs/tutorials/15_efficient_did.ipynb @@ -257,7 +257,7 @@ "source": [ "## Event Study Aggregation\n", "\n", - "Event study effects aggregate ATT(g,t) by relative time $e = t - g$, averaging across cohorts at each horizon. This shows how treatment effects evolve over time since adoption. Pre-treatment coefficients ($e < 0$) serve as a diagnostic for parallel trends." + "Event study effects aggregate ATT(g,t) by relative time $e = t - g$ (fractional-period panels bucket horizons by `int(t - g)` --- see the EfficientDiD REGISTRY truncation Note), averaging across cohorts at each horizon. This shows how treatment effects evolve over time since adoption. Pre-treatment coefficients ($e < 0$) serve as a diagnostic for parallel trends." ] }, { @@ -570,7 +570,7 @@ "7. **Condition numbers** flag potentially unstable weight matrices\n", "8. **Anticipation** shifts the effective treatment boundary for pre-treatment effects\n", "9. **Covariates are supported** via the doubly robust path (sieve outcome regressions + propensity ratios) --- pass `covariates=[...]` to `fit()`\n", - "10. **Aggregate post-fit** (3.9): `results.aggregate('event_study'/'group', balance_e=)` recomputes from retained EIFs --- no refit; `aggregate('simple')` and (3.10) `aggregate('total')` are RELAYS of the stored inference (no `balance_e`; both stay available on bootstrapped fits, where the recompute levels keep the deprecated fit-time route)\n", + "10. **Aggregate post-fit** (3.9): `results.aggregate('event_study'/'group', balance_e=)` recomputes from retained EIFs --- no refit; `aggregate('simple')` and (3.10) `aggregate('total')` are RELAYS of the stored inference (no `balance_e`); on bootstrapped fits the recompute levels REPLAY the fit-time multiplier bootstrap --- percentile inference, no refit needed\n", "11. When in doubt, run both EDiD and CS --- if ATTs agree, report EDiD for tighter CIs\n", "\n", "**Parameter reference:**\n", diff --git a/docs/v4-deprecations.yaml b/docs/v4-deprecations.yaml index 79c046c7..04adb940 100644 --- a/docs/v4-deprecations.yaml +++ b/docs/v4-deprecations.yaml @@ -228,7 +228,7 @@ rows: phase: 5 warning: FutureWarning test_ref: tests/test_aggregate_contract.py - code_refs: [diff_diff/staggered.py, diff_diff/staggered_results.py, diff_diff/staggered_bootstrap.py, diff_diff/bootstrap_chunking.py, diff_diff/aggregation.py, diff_diff/practitioner.py, diff_diff/guides/llms-practitioner.txt, diff_diff/diagnostic_report.py] + code_refs: [diff_diff/staggered.py, diff_diff/staggered_results.py, diff_diff/staggered_bootstrap.py, diff_diff/bootstrap_utils.py, diff_diff/bootstrap_chunking.py, diff_diff/aggregation.py, diff_diff/practitioner.py, diff_diff/guides/llms-practitioner.txt, diff_diff/diagnostic_report.py] notes: "Shimmed in 3.9: fit(aggregate=) warns via a sentinel default (so a plain fit() never warns) and still returns the fully populated legacy surface; results.aggregate(type=) is the successor. balance_e moves alongside it as its own row [M-117] - it was previously tracked only as prose here, which nothing asserted. VOCABULARY: the closed set is library-wide (simple|event_study|group|calendar|total - 'total' promoted 2026-08-16 as the estimator-owned total incremental outcome, an exact relay C x overall over the finite-masked complete-case support); CallawaySantAnna's SUPPORTED SUBSET is simple|event_study|group|total - it has no calendar aggregator (the DEFERRED 'Calendar-time aggregation' row), and aggregate('calendar') raises naming what is supported; 'total' is panel/non-survey only (RC-routed, declared-survey_design, divergent bare-cluster, and pre-upgrade-kit fits raise NotImplementedError naming the reason; the mass replays from the fit-time agg_gt_cells/is_survey_fit kit snapshots). BOOTSTRAP fits: 'simple' and, where supported, 'total' RELAY the stored overall quintet verbatim (percentile se/p/CI beside the finite safe_inference t; 'total' scales att/se/CI by C) with a NaN df column - the relay levels never re-warn; the recompute levels (event_study/group) REPLAY the fit-time multiplier bootstrap from the kit's BootstrapReplaySpec (the fit-captured RNG state + run params BY VALUE, so seed=None fits replay and post-fit set_params/attribute mutation cannot alter it; pickles carry it): se/CI/cband match a fit-time aggregation to BLAS reassociation (~1 ULP, assert_allclose - never bit-identity), the discrete percentile P-VALUE is a count statistic carved out of that claim (compared at 2/n_bootstrap), and the container publishes NO analytical provenance (vcov/vcov_index/df cleared, sup-t cband recomputed from the replayed draws). Each replaying call regenerates the full weight stream and re-runs the fused perturbation GEMM over the per-cell + per-event-time influence columns - O(n_bootstrap x n_units x (n_gt + n_event_times)) FLOPs per call, no memoization (immutability discipline; DR caches its derived surface once per report). The replay re-runs the same warning sites (never suppresses; per-site Python warning-registry semantics govern re-display). BACKEND GUARD: the spec stamps the weight-generation backend at capture ('rust'/'numpy' per bootstrap_chunking.effective_weight_backend, or 'portable' for provably backend-independent branches - stratified/single-PSU survey generation and census-FPC zero weights); Rust and NumPy produce DIFFERENT draws from the same bit-generator state, so a replay under a different backend (DIFF_DIFF_BACKEND flip, missing extension, another machine) fails closed naming both backends rather than silently desynchronizing from the artifact's stored relay quintet. Legacy pickles without the spec fail closed with a refit message. The per-level policy converged with [M-027]; the SDDD engine keeps its own fit-time override-loop copy (unification sequenced with the M-014 container port - twin-drift risk on record). DiagnosticReport now derives the event-study surface via post-fit aggregate('event_study') when the raw field is absent, so its ES-gated checks run on plain fits BOOTSTRAPPED INCLUDED (percentile replay; parallel_trends rides the Bonferroni fallback, pretrends_power/sensitivity the diagonal-covariance fallback - the derived container has vcov=None); derivation failures (kit-less/legacy pickles, backend mismatch, sibling estimators' gates) still surface as explicit skip reasons." - id: M-021 kind: param @@ -270,8 +270,8 @@ rows: phase: 5 warning: FutureWarning test_ref: tests/test_aggregate_contract.py - code_refs: [diff_diff/efficient_did.py, diff_diff/efficient_did_aggregation.py, diff_diff/efficient_did_results.py, diff_diff/aggregation.py, diff_diff/results_base.py, diff_diff/honest_did.py, diff_diff/pretrends.py, diff_diff/practitioner.py, diff_diff/guides/llms-practitioner.txt] - notes: "Shimmed in 3.9: fit(aggregate=) warns via the shared NOT_SUPPLIED sentinel (a plain fit() never warns; supplying ANY value, None included, warns - CS-style joint warning with balance_e [M-120], warn-and-still-work since the params genuinely worked). NO fit-time value validation existed and none is added: unknown strings silently act like None on the deprecated path, unchanged; the post-fit successor fails closed on unknown types via the mixin vocabulary - a behavior improvement. The successor is a LAZY RECOMPUTING KIT (the CallawaySantAnna class, not a view relay): fit() computes nothing extra, the results object retains an AggregationKit referencing the per-(g,t) EIF dict (O(n_units x n_gt) dominant payload; full buffer enumeration in the REGISTRY EfficientDiD Note) plus O(n_units) bookkeeping, PRIVATE per-row snapshots of group_time_effects/groups/time_periods plus the pt_assumption/alpha/anticipation/n-total provenance (recompute and the ES carrier never read the mutable public fields), and the POST-OVERALL df_survey snapshot (captured before the ES/group gates - the group pass can degenerate the working df on replicate designs, and every aggregation seeds from the post-overall value), and aggregate('event_study'/'group', balance_e=) recomputes on demand while 'simple' relays the stored overall row bit-exact (n = treated+control units - disjoint by construction; df = the snapshot, provenance-exact where survey_metadata.df_survey can diverge in the degenerate n_valid<=1 replicate state). SUPPORTED SUBSET simple|event_study|group|total ('total' since 2026-08-16: exact relay C x overall with C = the integer sum of kept post-anticipation finite cells' n_treated from the kit's deep-copied group_time_effects snapshot - never the non-integral n_units x sum(cohort_fractions) float product; fits declaring ANY survey_design - the kit's unit_level_weights/resolved_survey_unit markers, weight-type-agnostic - raise NotImplementedError); calendar and 'all' fail closed via the mixin; weights= rejected. Bootstrap fits: 'simple' and 'total' RELAY the stored overall row verbatim (percentile se/p/CI beside the finite safe_inference t) with a NaN df column, while the recompute levels (ES/group) fail closed - per-horizon draws are not retained (exact-replay wiring is a TODO row). The prior uniform-conservatism BY-DECISION rule (no level publishes analytical-provenance fields beside percentile inference) was superseded 2026-08-05 with the [M-027] per-level convergence; its rationale is honored by the relay's NaN df column. Group rows record per-row df_used at each safe_inference call (additive public row-dict key; the fit-time bootstrap override clears it - CS precedent); the post-fit group relay publishes the per-row array, a stated divergence from CS's conservative-min scalar broadcast. PT-Post reference provenance: the membership-gated reference_period property (SunAbraham rule - never synthesized when the anchor cell was not estimated) marks the materialized mechanical zero anchor is_reference in the container, and plot_event_study's inferred reference correctly shifts to -1-anticipation on PT-Post anticipation>0 fits. store_eif now governs only the public influence_functions field - the kit ALWAYS retains the EIF dict (memory-contract change; a store_kit opt-out is a DEFERRED row). Container admission NOT widened to EfficientDiD (see M-093); balance_e moves as its own row [M-120]." + code_refs: [diff_diff/efficient_did.py, diff_diff/efficient_did_aggregation.py, diff_diff/efficient_did_bootstrap.py, diff_diff/efficient_did_results.py, diff_diff/aggregation.py, diff_diff/bootstrap_utils.py, diff_diff/bootstrap_chunking.py, diff_diff/results_base.py, diff_diff/honest_did.py, diff_diff/pretrends.py, diff_diff/practitioner.py, diff_diff/guides/llms-practitioner.txt] + notes: "Shimmed in 3.9: fit(aggregate=) warns via the shared NOT_SUPPLIED sentinel (a plain fit() never warns; supplying ANY value, None included, warns - CS-style joint warning with balance_e [M-120], warn-and-still-work since the params genuinely worked). NO fit-time value validation existed and none is added: unknown strings silently act like None on the deprecated path, unchanged; the post-fit successor fails closed on unknown types via the mixin vocabulary - a behavior improvement. The successor is a LAZY RECOMPUTING KIT (the CallawaySantAnna class, not a view relay): fit() computes nothing extra, the results object retains an AggregationKit referencing the per-(g,t) EIF dict (O(n_units x n_gt) dominant payload; full buffer enumeration in the REGISTRY EfficientDiD Note) plus O(n_units) bookkeeping, PRIVATE per-row snapshots of group_time_effects/groups/time_periods plus the pt_assumption/alpha/anticipation/n-total provenance (recompute and the ES carrier never read the mutable public fields), and the POST-OVERALL df_survey snapshot (captured before the ES/group gates - the group pass can degenerate the working df on replicate designs, and every aggregation seeds from the post-overall value), and aggregate('event_study'/'group', balance_e=) recomputes on demand while 'simple' relays the stored overall row bit-exact (n = treated+control units - disjoint by construction; df = the snapshot, provenance-exact where survey_metadata.df_survey can diverge in the degenerate n_valid<=1 replicate state). SUPPORTED SUBSET simple|event_study|group|total ('total' since 2026-08-16: exact relay C x overall with C = the integer sum of kept post-anticipation finite cells' n_treated from the kit's deep-copied group_time_effects snapshot - never the non-integral n_units x sum(cohort_fractions) float product; fits declaring ANY survey_design - the kit's unit_level_weights/resolved_survey_unit markers, weight-type-agnostic - raise NotImplementedError); calendar and 'all' fail closed via the mixin; weights= rejected. Bootstrap fits: 'simple' and 'total' RELAY the stored overall row verbatim (percentile se/p/CI beside the finite safe_inference t) with a NaN df column and never re-warn, while the recompute levels (ES/group, balance_e included) REPLAY the fit-time multiplier bootstrap from the kit's BootstrapReplaySpec (fit-captured RNG state + run params BY VALUE - seed=None fits replay, post-fit set_params/attribute mutation cannot alter it, pickles carry it): se/CI/t match a fit-time aggregation to BLAS reassociation (~1 ULP, assert_allclose - never bit-identity), the percentile P-VALUE is a count statistic compared at 2/n_bootstrap, and the replay publishes no analytical provenance (percentile overrides + all-NaN df channels; EDiD has no cband). The ES/group percentile overrides are the SHARED appliers in bootstrap_utils (apply_bootstrap_event_study_overrides/apply_bootstrap_group_overrides, relocated from staggered_bootstrap - one implementation for the CS and EDiD fit paths and replays; verified bit-inert at the swap). Per replaying call the engine regenerates the full weight stream and re-runs the fused perturbation GEMM over the n_gt per-cell EIF columns - O(n_bootstrap x n_units x n_gt) FLOPs, ES/group targets re-aggregated cheaply from the dense (n_bootstrap, n_gt) matrix - no memoization (immutability discipline; DR caches its derived surface once per report). The replay re-runs the same warning sites (re-emits, never suppresses). BACKEND GUARD (the M-020 contract): the spec stamps the weight backend at capture ('rust'/'numpy' per bootstrap_chunking.effective_weight_backend, or 'portable' for provably backend-independent branches - stratified survey generation, census-FPC zero weights, and the single-PSU degenerate early return, which is stamped before any generation); a replay under a different backend fails closed naming both, and legacy pickles without the spec fail closed with a refit message. FRACTIONAL-PERIOD KEY FIX shipped with the replay: the bootstrap ES prep now keys horizons by the analytical int(t - g) truncation at all three sites (the two horizon keys AND the balance_e anchor filter) - previously raw t - g attached a strict sub-aggregate's percentile inference to the pooled analytical row on fractional panels; n_groups now counts DISTINCT cohorts per bucket (identity on integer panels) and fractional truncation emits a UserWarning (see the REGISTRY truncation Note). The prior uniform-conservatism BY-DECISION rule (no level publishes analytical-provenance fields beside percentile inference) was superseded 2026-08-05 with the [M-027] per-level convergence; its rationale is honored by the percentile overrides and NaN df channels. Group rows record per-row df_used at each safe_inference call (additive public row-dict key; the fit-time bootstrap override clears it - CS precedent); the post-fit group relay publishes the per-row array, a stated divergence from CS's conservative-min scalar broadcast. PT-Post reference provenance: the membership-gated reference_period property (SunAbraham rule - never synthesized when the anchor cell was not estimated) marks the materialized mechanical zero anchor is_reference in the container, and plot_event_study's inferred reference correctly shifts to -1-anticipation on PT-Post anticipation>0 fits. store_eif now governs only the public influence_functions field - the kit ALWAYS retains the EIF dict (memory-contract change; a store_kit opt-out is a DEFERRED row). Container admission NOT widened to EfficientDiD (see M-093); balance_e moves as its own row [M-120]." - id: M-024 kind: param group: aggregate-postfit @@ -1410,8 +1410,8 @@ rows: phase: 5 warning: FutureWarning test_ref: tests/test_aggregate_contract.py - code_refs: [diff_diff/efficient_did.py, diff_diff/efficient_did_aggregation.py, diff_diff/efficient_did_results.py, diff_diff/aggregation.py] - notes: "balance_e moves from fit() onto aggregate() with [M-023] (joint FutureWarning; the M-117 twin). Applies to event-study aggregation only (the mixin default vocabulary), so aggregate(type='simple'|'group'|'total', balance_e=...) raises. EfficientDiD's balance rule is the ANCHOR-HORIZON rule - keep cohorts with a finite effect at e == balance_e, then retain all their horizons - the SAME rule CallawaySantAnna uses, divergent only from ImputationDiD/TwoStageDiD's balanced-window rule (their own rows document theirs). An anchor no cohort reaches warns (UserWarning) and yields a legal zero-row EventStudyResults container." + code_refs: [diff_diff/efficient_did.py, diff_diff/efficient_did_aggregation.py, diff_diff/efficient_did_bootstrap.py, diff_diff/efficient_did_results.py, diff_diff/aggregation.py, diff_diff/bootstrap_utils.py] + notes: "balance_e moves from fit() onto aggregate() with [M-023] (joint FutureWarning; the M-117 twin). Applies to event-study aggregation only (the mixin default vocabulary), so aggregate(type='simple'|'group'|'total', balance_e=...) raises. EfficientDiD's balance rule is the ANCHOR-HORIZON rule - keep cohorts with a finite effect at the anchor horizon, then retain all their horizons - the same rule SHAPE CallawaySantAnna uses, with one keying-granularity difference: EDiD anchors on the int(t - g) bucket while CS keys raw t - g (identical on integer-period panels; the REGISTRY truncation Note documents the fractional case), and divergent from ImputationDiD/TwoStageDiD's balanced-window rule (their own rows document theirs). On bootstrapped fits balance_e rides the M-023 replay (the bootstrap ES prep applies the same int-bucketed anchor rule). An anchor no cohort reaches warns (UserWarning, re-emitted on replay) and yields a legal zero-row EventStudyResults container." - id: M-118 kind: param group: aggregate-postfit diff --git a/docs/v4-design.md b/docs/v4-design.md index eb61a87c..d44fdd08 100644 --- a/docs/v4-design.md +++ b/docs/v4-design.md @@ -586,9 +586,10 @@ relay is exact under percentile inference too; all of dCDH/StackedDiD/HAD) is faithful under any inference regime and stays available on bootstrapped fits, with the df COLUMN NaN'd there - no df governs percentile inference, so a relay never publishes analytical -provenance beside percentile statistics; where bootstrap draws are not -retained, a RECOMPUTE level on a bootstrapped fit RAISES rather than -silently returning analytical inference. **View-relay exception (Phase 2b PRs 1-2):** +provenance beside percentile statistics; a RECOMPUTE level on a +bootstrapped fit either REPLAYS the fit-time bootstrap from retained RNG +state (CS [M-020], EfficientDiD [M-023]) or RAISES - it never silently +returns analytical inference. **View-relay exception (Phase 2b PRs 1-2):** estimators whose `aggregate()` RELAYS stored fields without recomputation need no influence-function kit - there is nothing to re-weight. The retention requirement binds RECOMPUTING estimators (the CallawaySantAnna @@ -987,8 +988,8 @@ five recorded deviations: - **No `Available since` column.** Considered and rejected: it cannot be derived. `introduced_in` tracks the dataclass storage flip, so the nine `field-flip` rows say `4.0` while `.att` already resolves today; and a successor that exists can - still raise (the bootstrapped-fit `aggregate()` recompute gates — four families - since CS's percentile-bootstrap replay landed). Availability is + still raise (the bootstrapped-fit `aggregate()` recompute gates — three families + since the CS and EfficientDiD percentile-bootstrap replays landed). Availability is stated in prose where it is verifiable instead. - **§7b "Remaining 4.0 changes"** was added: §§2-8 as skeletoned cover only 102 of the 108 qualifying rows, leaving `obligation-sdid-params`, `constructor-hygiene`, diff --git a/tests/test_aggregate_contract.py b/tests/test_aggregate_contract.py index 95cfca26..d494527e 100644 --- a/tests/test_aggregate_contract.py +++ b/tests/test_aggregate_contract.py @@ -2273,14 +2273,26 @@ def test_survey_metadata_not_mutated(self): res.aggregate("group") assert res.survey_metadata.df_survey == before - def test_bootstrap_recompute_levels_fail_closed(self, efficient_panel): - res = _fit_efficient(efficient_panel, est_kw={"n_bootstrap": 20, "seed": 1}) - for level in ("event_study", "group"): - with pytest.raises(NotImplementedError, match="bootstrap") as exc: - res.aggregate(level) - assert "aggregate('simple') and, where supported, aggregate('total') relay" in str( - exc.value - ) + def test_retention_bootstrapped_spec_no_leak(self, efficient_panel): + # The BootstrapReplaySpec adds only the RNG state dict + scalars to + # the kit; a bootstrapped fit's kit must keep the no-DataFrame / + # no-unit-label retention contract and pickle round-trip its replay. + import pickle + + import pandas as pd + + d = efficient_panel.copy() + sentinel = {u: f"SENTINEL-ID-{u}@example.invalid" for u in d["unit"].unique()} + d["unit"] = d["unit"].map(sentinel) + res = _fit_efficient(d, est_kw={"n_bootstrap": 20, "seed": 1}) + kit = res._aggregation_kit + assert kit.bootstrap is not None and kit.bootstrap.bitgen_state is not None + for v in kit.bookkeeping.values(): + assert not isinstance(v, pd.DataFrame) + blob = pickle.dumps(res) + assert b"SENTINEL-ID" not in blob + res2 = pickle.loads(blob) + np.testing.assert_array_equal(res.aggregate("group").se, res2.aggregate("group").se) def test_bootstrap_simple_relays_stored_quintet(self, efficient_panel): res = _fit_efficient(efficient_panel, est_kw={"n_bootstrap": 20, "seed": 1}) @@ -2386,6 +2398,500 @@ def test_zero_row_balance_e_surface(self, efficient_fitted): assert len(es.event_time) == 0 +# --------------------------------------------------------------------------- # +# EfficientDiD bootstrap REPLAY (the CS BootstrapReplaySpec mechanism): +# post-fit aggregate('event_study'/'group') on bootstrapped fits replays the +# fit-time multiplier bootstrap from the kit-retained RNG state. The parity +# REFERENCE is always the NATIVE fit-time surface (the kit attaches +# unconditionally, so a fit-time-aggregated result's own aggregate() would +# ALSO replay - replay-vs-replay proves nothing). +# Tolerances: se/ci/t at 1e-13 (the replayed draws are bit-identical; only +# GEMM tile-boundary reassociation differs - ~1 ULP, with headroom for +# quantile interpolation); percentile p-values are COUNT statistics, so a +# draw within a ULP of the point estimate could flip one count - compared +# at atol=2/n_bootstrap. +# --------------------------------------------------------------------------- # + +_EDID_NBOOT = 50 + + +def _efficient_boot_fit(data, **fit_kw): + est_kw = fit_kw.pop("est_kw", {}) + return _fit_efficient(data, est_kw={"n_bootstrap": _EDID_NBOOT, "seed": 42, **est_kw}, **fit_kw) + + +def _efficient_boot_fit_time(data, *, aggregate="all", **fit_kw): + est_kw = fit_kw.pop("est_kw", {}) + return _fit_efficient( + data, + est_kw={"n_bootstrap": _EDID_NBOOT, "seed": 42, **est_kw}, + aggregate=aggregate, + **fit_kw, + ) + + +def _assert_edid_es_replay_parity(es, fit_time, n_boot=_EDID_NBOOT): + df = es.to_dataframe() + assert es.vcov is None + assert np.all(np.isnan(df["df"].to_numpy(dtype=float))) + assert len(df) == len(fit_time.event_study_effects) + p_atol = 2.0 / n_boot + for _, row in df.iterrows(): + ref = fit_time.event_study_effects[int(row["event_time"])] + assert row["att"] == ref["effect"] + np.testing.assert_allclose(row["se"], ref["se"], rtol=1e-13, atol=1e-13) + np.testing.assert_allclose(row["t_stat"], ref["t_stat"], rtol=1e-13, atol=1e-13) + np.testing.assert_allclose(row["p_value"], ref["p_value"], rtol=1e-13, atol=p_atol) + np.testing.assert_allclose( + [row["conf_int_lower"], row["conf_int_upper"]], + list(ref["conf_int"]), + rtol=1e-13, + atol=1e-13, + ) + + +def _assert_edid_group_replay_parity(grp, fit_time, n_boot=_EDID_NBOOT): + df = grp.to_dataframe() + assert np.all(np.isnan(df["df"].to_numpy(dtype=float))) + assert len(df) == len(fit_time.group_effects) + p_atol = 2.0 / n_boot + for _, row in df.iterrows(): + ref = fit_time.group_effects[float(row["label"])] + assert row["att"] == ref["effect"] + np.testing.assert_allclose(row["se"], ref["se"], rtol=1e-13, atol=1e-13) + np.testing.assert_allclose(row["t_stat"], ref["t_stat"], rtol=1e-13, atol=1e-13) + np.testing.assert_allclose(row["p_value"], ref["p_value"], rtol=1e-13, atol=p_atol) + np.testing.assert_allclose( + [row["conf_int_lower"], row["conf_int_upper"]], + list(ref["conf_int"]), + rtol=1e-13, + atol=1e-13, + ) + + +def _efficient_fractional_panel(cohorts=(3.0, 2.25), seed=0, n_units=90): + """0.5-spaced panel; cohorts may sit ON the grid or OFF it (e.g. 2.25).""" + import numpy as np + import pandas as pd + + rng = np.random.default_rng(seed) + periods = np.arange(1.0, 5.0, 0.5) + per_cohort = n_units // (len(cohorts) + 2) + rows = [] + for u in range(n_units): + idx = u // per_cohort + g = cohorts[idx] if idx < len(cohorts) else 0.0 + for t in periods: + y = rng.normal() + u * 0.01 + t * 0.1 + (0.5 if g > 0 and t >= g else 0.0) + rows.append((u, t, g, y)) + return pd.DataFrame(rows, columns=["unit", "period", "first_treat", "outcome"]) + + +class TestEfficientBootstrapReplay: + def test_event_study_parity_with_fit_time(self, efficient_panel): + res = _efficient_boot_fit(efficient_panel) + ftime = _efficient_boot_fit_time(efficient_panel) + _assert_edid_es_replay_parity(res.aggregate("event_study"), ftime) + + def test_group_parity_with_fit_time(self, efficient_panel): + res = _efficient_boot_fit(efficient_panel) + ftime = _efficient_boot_fit_time(efficient_panel) + _assert_edid_group_replay_parity(res.aggregate("group"), ftime) + + def test_balance_e_parity_with_fit_time(self, efficient_panel): + res = _efficient_boot_fit(efficient_panel) + ftime = _efficient_boot_fit_time(efficient_panel, aggregate="event_study", balance_e=1) + _assert_edid_es_replay_parity(res.aggregate("event_study", balance_e=1), ftime) + + def test_balance_e_empty_anchor_replay_warns_zero_rows(self, efficient_panel): + # The replay re-emits the fit-time anchor warning and returns the + # LEGAL zero-row surface (the analytical twin of + # test_zero_row_balance_e_surface, now on the replay route). + res = _efficient_boot_fit(efficient_panel) + with pytest.warns(UserWarning, match="anchor horizon"): + es = res.aggregate("event_study", balance_e=99) + assert len(es.event_time) == 0 + + def test_pt_post_parity_with_fit_time(self, efficient_panel): + # PT-Post: the bootstrap prep DOES key the finite effect=0.0 anchor + # at e=-1 and the override NaNs its inference (zero-SE draws); the + # is_reference marking is label-based and survives. Parity holds + # row-for-row, the anchor included. + res = _efficient_boot_fit(efficient_panel, est_kw={"pt_assumption": "post"}) + ftime = _efficient_boot_fit_time(efficient_panel, est_kw={"pt_assumption": "post"}) + es = res.aggregate("event_study") + df = es.to_dataframe() + anchor = df[df["is_reference"]] + assert len(anchor) == 1 + assert float(anchor["att"].iloc[0]) == 0.0 + assert np.isnan(float(anchor["se"].iloc[0])) + # Non-reference rows hit full parity; the reference row's fit-time + # DICT entry is also NaN'd by the override (same applier), so the + # container/dict split is exercised on the fractional fixture below. + _assert_edid_es_replay_parity(es, ftime) + + def test_covariates_parity_with_fit_time(self, efficient_panel): + d = efficient_panel.copy() + rng = np.random.default_rng(3) + xmap = {u: rng.normal() for u in d["unit"].unique()} + d["x1"] = d["unit"].map(xmap) + res = _efficient_boot_fit(d, covariates=["x1"]) + ftime = _efficient_boot_fit_time(d, covariates=["x1"]) + _assert_edid_es_replay_parity(res.aggregate("event_study"), ftime) + _assert_edid_group_replay_parity(res.aggregate("group"), ftime) + + def test_seedless_fit_replays_and_is_idempotent(self, efficient_panel): + res = _fit_efficient(efficient_panel, est_kw={"n_bootstrap": _EDID_NBOOT}) + a = res.aggregate("event_study").to_dataframe() + b = res.aggregate("event_study").to_dataframe() + np.testing.assert_array_equal(a["se"].to_numpy(), b["se"].to_numpy()) + + def test_set_params_and_mutation_immunity(self, efficient_panel): + from diff_diff import EfficientDiD + + est = EfficientDiD(n_bootstrap=_EDID_NBOOT, seed=42) + with warnings.catch_warnings(): + warnings.simplefilter("ignore") + res = est.fit(efficient_panel, **EFFICIENT_KW) + before = res.aggregate("event_study").to_dataframe() + est.set_params(n_bootstrap=5, seed=1, bootstrap_weights="mammen") + est.n_bootstrap = 3 + after = res.aggregate("event_study").to_dataframe() + np.testing.assert_array_equal(before["se"].to_numpy(), after["se"].to_numpy()) + + def test_pickle_round_trip_replays(self, efficient_panel): + import pickle + + res = _efficient_boot_fit(efficient_panel) + before = res.aggregate("group").to_dataframe() + res2 = pickle.loads(pickle.dumps(res)) + after = res2.aggregate("group").to_dataframe() + np.testing.assert_array_equal(before["se"].to_numpy(), after["se"].to_numpy()) + + def test_relays_unchanged_and_order_independent(self, efficient_panel): + res = _efficient_boot_fit(efficient_panel) + s_before = res.aggregate("simple") + res.aggregate("event_study") + s_after = res.aggregate("simple") + assert float(s_before.att[0]) == float(s_after.att[0]) == res.overall_att + assert float(s_before.se[0]) == float(s_after.se[0]) == res.overall_se + + def test_legacy_kit_without_spec_fails_closed(self, efficient_panel): + res = _efficient_boot_fit(efficient_panel) + object.__setattr__(res._aggregation_kit, "bootstrap", None) + for level in ("event_study", "group"): + with pytest.raises(NotImplementedError, match="predates"): + res.aggregate(level) + + def test_backend_mismatch_fails_closed(self, efficient_panel): + import dataclasses as dc + + from diff_diff.bootstrap_chunking import effective_weight_backend + + res = _efficient_boot_fit(efficient_panel) + kit = res._aggregation_kit + current = effective_weight_backend() + other = "numpy" if current == "rust" else "rust" + assert kit.bootstrap.backend == current # plain fits stamp the live backend + object.__setattr__(kit, "bootstrap", dc.replace(kit.bootstrap, backend=other)) + for level in ("event_study", "group"): + with pytest.raises(NotImplementedError, match="weight backend"): + res.aggregate(level) + # None (unknown) also fails closed - a permissive default on a + # safety discriminator would disarm the guard. + object.__setattr__(kit, "bootstrap", dc.replace(kit.bootstrap, backend=None)) + with pytest.raises(NotImplementedError, match="weight backend"): + res.aggregate("event_study") + + def test_low_bootstrap_warning_re_emitted_on_replay(self, efficient_panel): + res = _fit_efficient(efficient_panel, est_kw={"n_bootstrap": 49, "seed": 7}) + with pytest.warns(UserWarning, match="n_bootstrap=49 is low"): + res.aggregate("event_study") + + +class TestEfficientBootstrapReplayDesigns: + def test_cluster_parity(self): + d = _efficient_clustered_panel() + res = _efficient_boot_fit(d, est_kw={"cluster": "cl"}) + ftime = _efficient_boot_fit_time(d, est_kw={"cluster": "cl"}) + _assert_edid_es_replay_parity(res.aggregate("event_study"), ftime) + _assert_edid_group_replay_parity(res.aggregate("group"), ftime) + + def test_weights_only_survey_parity(self): + # Weights-only design: unit weight PATH + survey EIF scaling. + d, _ = _efficient_survey_panel() + sd = _efficient_survey_design() + res = _efficient_boot_fit(d, survey_design=sd) + ftime = _efficient_boot_fit_time(d, survey_design=sd) + assert res._aggregation_kit.bootstrap.backend != "portable" + _assert_edid_es_replay_parity(res.aggregate("event_study"), ftime) + + @staticmethod + def _psu_survey_frame(strata=False, fpc=None): + from diff_diff import SurveyDesign + + d, _ = _efficient_survey_panel() + d = d.copy() + d["psu"] = (d["unit"] // 6).astype(int) + kw = dict(weights="w", psu="psu") + if strata: + d["stratum"] = (d["unit"] // 60).astype(int) + kw["strata"] = "stratum" + kw["nest"] = True + if fpc is not None: + d["fpc_col"] = float(fpc) + kw["fpc"] = "fpc_col" + return d, SurveyDesign(**kw) + + def test_stratified_survey_portable_stamp_and_parity(self): + d, sd = self._psu_survey_frame(strata=True) + res = _efficient_boot_fit(d, survey_design=sd) + assert res._aggregation_kit.bootstrap.backend == "portable" + ftime = _efficient_boot_fit_time(d, survey_design=sd) + _assert_edid_es_replay_parity(res.aggregate("event_study"), ftime) + _assert_edid_group_replay_parity(res.aggregate("group"), ftime) + + def test_fpc_parity(self): + # Non-census FPC (population 10x the PSU count): fpc_scale on top of + # the backend-dependent generator. + d, sd = self._psu_survey_frame(fpc=200.0) + res = _efficient_boot_fit(d, survey_design=sd) + assert res._aggregation_kit.bootstrap.backend != "portable" + ftime = _efficient_boot_fit_time(d, survey_design=sd) + _assert_edid_es_replay_parity(res.aggregate("event_study"), ftime) + + def test_census_fpc_portable(self): + # Census FPC (fpc == n_psu): every weight block is zeroed, so every + # bootstrap distribution is EXACTLY CONSTANT at the original effect + # - the discarded draws' backend is irrelevant (stamped portable), + # and the constant-distribution guard must NaN ALL inference fields + # on BOTH routes (a tiny-positive roundoff np.std at a non-zero + # constant level must never leak a huge finite t - CI review P0). + d, sd = self._psu_survey_frame(fpc=20.0) # 120 units // 6 = 20 PSUs + with warnings.catch_warnings(): + warnings.simplefilter("ignore") + res = _efficient_boot_fit(d, survey_design=sd) + ftime = _efficient_boot_fit_time(d, survey_design=sd) + assert res._aggregation_kit.bootstrap.backend == "portable" + # Fit-time surfaces: full-NaN inference beside finite effects. + for surface in (ftime.event_study_effects, ftime.group_effects): + assert surface + for row in surface.values(): + assert np.isfinite(row["effect"]) + assert np.isnan(row["se"]) and np.isnan(row["t_stat"]) + assert np.isnan(row["p_value"]) + assert np.isnan(row["conf_int"][0]) and np.isnan(row["conf_int"][1]) + # Replayed surfaces reproduce the same degenerate state. + with warnings.catch_warnings(): + warnings.simplefilter("ignore") + es_df = res.aggregate("event_study").to_dataframe() + g_df = res.aggregate("group").to_dataframe() + for frame in (es_df, g_df): + att = frame["att"].to_numpy(dtype=float) + ref_mask = ( + frame["is_reference"].to_numpy(dtype=bool) + if "is_reference" in frame + else np.zeros(len(frame), dtype=bool) + ) + assert np.all(np.isfinite(att[~ref_mask])) + for col in ("se", "t_stat", "p_value", "conf_int_lower", "conf_int_upper"): + assert np.all(np.isnan(frame[col].to_numpy(dtype=float)[~ref_mask])), col + + def test_single_psu_nan_surfaces_and_warning(self): + # n_psu < 2 early-returns the NaN container BEFORE any generation: + # stamped portable; the replay re-hits the return deterministically + # and re-emits the PSU warning; inference NaNs on both levels. + from diff_diff import SurveyDesign + + d, _ = _efficient_survey_panel() + d = d.copy() + d["psu"] = 0 + sd = SurveyDesign(weights="w", psu="psu") + with warnings.catch_warnings(): + warnings.simplefilter("ignore") + res = _efficient_boot_fit(d, survey_design=sd) + assert res._aggregation_kit.bootstrap.backend == "portable" + with pytest.warns(UserWarning, match="n_psu=1"): + es = res.aggregate("event_study") + assert np.all(np.isnan(es.se)) + with warnings.catch_warnings(): + warnings.simplefilter("ignore") + grp = res.aggregate("group").to_dataframe() + assert np.all(np.isnan(grp["se"].to_numpy(dtype=float))) + + def test_anticipation_parity(self, efficient_panel): + # Pins the replay host's anticipation wiring (shifts the engine's + # post-treatment mask and the group prep's inclusion rule); a + # defaulted host attribute would pass every anticipation=0 arm. + res = _efficient_boot_fit(efficient_panel, est_kw={"anticipation": 1}) + ftime = _efficient_boot_fit_time(efficient_panel, est_kw={"anticipation": 1}) + _assert_edid_es_replay_parity(res.aggregate("event_study"), ftime) + _assert_edid_group_replay_parity(res.aggregate("group"), ftime) + + def test_alpha_and_mammen_parity(self, efficient_panel): + # Pins the host/spec wiring of alpha and weight_type - a host that + # hard-codes the defaults passes every other arm. + kw = {"alpha": 0.10, "bootstrap_weights": "mammen"} + res = _efficient_boot_fit(efficient_panel, est_kw=kw) + ftime = _efficient_boot_fit_time(efficient_panel, est_kw=kw) + _assert_edid_es_replay_parity(res.aggregate("event_study"), ftime) + _assert_edid_group_replay_parity(res.aggregate("group"), ftime) + + +class TestEfficientFractionalPeriods: + """Decision-10 pins: int(t - g) truncation-bucketing on fractional panels. + + The published fit-time key set is int-bucketed by the ANALYTICAL + aggregator regardless of the bootstrap prep's keying, so the + discriminating surfaces are (a) the prep's own key set and (b) the + off-grid-onset arm, where pre-fix NO raw key intersected the analytical + buckets and the fit-time rows kept analytical inference. + """ + + def test_prep_keys_match_analytical_buckets_and_warns(self): + from diff_diff import EfficientDiD + + d = _efficient_fractional_panel() + # _fit_efficient suppresses warnings, so fit directly to pin the + # fractional-truncation warning alongside the prep-key assertion. + with pytest.warns(UserWarning, match="bucketed by int"): + with warnings.catch_warnings(): + warnings.simplefilter("ignore", FutureWarning) + res = EfficientDiD(n_bootstrap=_EDID_NBOOT, seed=42).fit( + d, aggregate="all", **EFFICIENT_KW + ) + assert set(res.bootstrap_results.event_study_ses) >= set(res.event_study_effects) + assert all(isinstance(e, int) for e in res.bootstrap_results.event_study_ses) + + def test_offgrid_rows_carry_percentile_inference(self): + # EVERY treated cohort off-grid: raw t-g is never an integer, so + # pre-fix no override would land and the fit-time rows would keep + # ANALYTICAL inference - the end-to-end discriminator. + d = _efficient_fractional_panel(cohorts=(2.25, 3.75)) + with warnings.catch_warnings(): + warnings.simplefilter("ignore") + boot = _efficient_boot_fit_time(d) + ana = _fit_efficient(d, aggregate="all") + for e, row in boot.event_study_effects.items(): + assert row["se"] != ana.event_study_effects[e]["se"] + + def test_replay_parity_on_fractional_panel(self): + d = _efficient_fractional_panel() + with warnings.catch_warnings(): + warnings.simplefilter("ignore") + res = _efficient_boot_fit(d) + ftime = _efficient_boot_fit_time(d) + with warnings.catch_warnings(): + warnings.simplefilter("ignore") + es = res.aggregate("event_study") + _assert_edid_es_replay_parity(es, ftime) + + def test_n_groups_counts_distinct_cohorts(self): + # Fractional buckets pool multiple cells per cohort; n_groups must + # count DISTINCT cohorts, not cells. + d = _efficient_fractional_panel() + with warnings.catch_warnings(): + warnings.simplefilter("ignore") + res = _fit_efficient(d, aggregate="event_study") + for e, row in res.event_study_effects.items(): + assert row["n_groups"] <= 2, (e, row["n_groups"]) + + def test_integer_panel_n_groups_regression(self, efficient_fit_time): + # Identity claim: on integer panels one cell per cohort per bucket, + # so distinct-cohort counting equals the old cell count. Pin the + # exact values on the standard 2-cohort fixture (cohorts 4 and 6 on + # an 8-period panel: both cohorts share buckets -1..1 given cohort + # 6's horizons span -5..1, cohort 4's -3..3). + expected = { + e: len({g for (g, t) in efficient_fit_time.group_time_effects if int(t - g) == e}) + for e in efficient_fit_time.event_study_effects + } + for e, row in efficient_fit_time.event_study_effects.items(): + assert row["n_groups"] == expected[e] + assert max(expected.values()) == 2 # both cohorts pool somewhere + + def test_integer_panel_never_warns(self, efficient_panel): + with warnings.catch_warnings(record=True) as caught: + warnings.simplefilter("always") + res = _fit_efficient(efficient_panel) + res.aggregate("event_study") + assert not any("bucketed by int" in str(w.message) for w in caught) + + def test_fractional_balance_e_offgrid_anchor(self): + # Cohort 2.25 reaches the integer anchor bucket 1 only via raw + # horizons 1.25/1.75 - the :403 anchor-filter pin: the bootstrap + # balanced cohort set must match the analytical one. + d = _efficient_fractional_panel() + with warnings.catch_warnings(): + warnings.simplefilter("ignore") + boot = _efficient_boot_fit_time(d, aggregate="event_study", balance_e=1) + ana = _fit_efficient(d, aggregate="event_study", balance_e=1) + assert set(boot.bootstrap_results.event_study_ses) >= set(ana.event_study_effects) + assert ana.event_study_effects[1]["n_groups"] == 2 # both cohorts anchored + for e, row in boot.event_study_effects.items(): + assert row["se"] != ana.event_study_effects[e]["se"] or np.isnan(row["se"]) + + def test_bucket_pooled_att_cell_mass_weighting(self): + # Hand-computed oracle for the REGISTRY Note's weighting clause: a + # bucket's ATT is the CELL-MASS weighted mean - one pi_g term per + # CELL, normalized within the bucket (a cohort with k cells carries + # k*pi_g mass) - not one term per cohort. + d = _efficient_fractional_panel() + with warnings.catch_warnings(): + warnings.simplefilter("ignore") + res = _fit_efficient(d, aggregate="event_study") + kit = res._aggregation_kit + fracs = kit.bookkeeping["cohort_fractions"] + gt = kit.bookkeeping["group_time_effects"] + for e, row in res.event_study_effects.items(): + cells = [ + (d_["effect"], fracs.get(g, 0.0)) + for (g, t), d_ in gt.items() + if int(t - g) == e and np.isfinite(d_["effect"]) + ] + w = np.array([c[1] for c in cells]) + effs = np.array([c[0] for c in cells]) + w = w / w.sum() if w.sum() > 0 else np.ones(len(w)) / len(w) + np.testing.assert_allclose(row["effect"], float(np.sum(w * effs)), rtol=1e-12) + + def test_fractional_pt_post_reference_collision(self): + # Note clause (iii), PER-ROUTE oracles: reference normalization is + # CONTAINER-only. The bucket at -1-anticipation pools a genuine + # fractional pre-treatment estimate with the mechanical zero anchor; + # the container publishes it as the reference (att=0, NaN + # inference), while the fit-time DICT keeps the pooled effect with + # percentile inference. Parity asserted on NON-reference rows only. + d = _efficient_fractional_panel() + with warnings.catch_warnings(): + warnings.simplefilter("ignore") + res = _efficient_boot_fit(d, est_kw={"pt_assumption": "post"}) + ftime = _efficient_boot_fit_time(d, est_kw={"pt_assumption": "post"}) + es = res.aggregate("event_study") + df = es.to_dataframe() + anchor = df[df["is_reference"]] + assert len(anchor) == 1 and int(anchor["event_time"].iloc[0]) == -1 + assert float(anchor["att"].iloc[0]) == 0.0 + assert np.isnan(float(anchor["se"].iloc[0])) + # The fit-time DICT entry for the same bucket is NOT rewritten. + dict_row = ftime.event_study_effects[-1] + assert dict_row["effect"] != 0.0 or not np.isnan(dict_row["se"]) + for _, row in df[~df["is_reference"]].iterrows(): + ref = ftime.event_study_effects[int(row["event_time"])] + assert row["att"] == ref["effect"] + np.testing.assert_allclose(row["se"], ref["se"], rtol=1e-13, atol=1e-13) + + def test_hausman_pretest_fractional_warns(self): + d = _efficient_fractional_panel() + from diff_diff import EfficientDiD + + est = EfficientDiD() + with warnings.catch_warnings(): + warnings.simplefilter("ignore") + est.fit(d, **EFFICIENT_KW) + with pytest.warns(UserWarning, match="Hausman pre-test horizons are bucketed"): + est.hausman_pretest(d, **EFFICIENT_KW) + + class TestEfficientInternalCallers: def test_hausman_pretest_emits_no_future_warning(self, efficient_panel): # hausman_pretest refits internally; its fit_kwargs no longer pass diff --git a/tests/test_bootstrap_utils.py b/tests/test_bootstrap_utils.py index 6c12280f..92c76318 100644 --- a/tests/test_bootstrap_utils.py +++ b/tests/test_bootstrap_utils.py @@ -33,6 +33,43 @@ def test_bootstrap_stats_single_valid_sample(self): assert np.isnan(ci[1]) assert np.isnan(p_value) + def test_bootstrap_stats_constant_nonzero_distribution_nans(self): + """EXACTLY constant non-zero draws (census-FPC zero weights leave every + replicate at the original effect): np.std can return a tiny POSITIVE + value from mean-subtraction roundoff (measured ~2.8e-17 at level 0.1), + which would slip past `se <= 0` and publish a huge finite t. The + constant-distribution guard must NaN the full tuple instead.""" + for level in (0.1, 1.0 / 3.0, 100.7): + boot_dist = np.full(50, level) + with pytest.warns(RuntimeWarning, match="non-finite or zero"): + se, ci, p_value = compute_effect_bootstrap_stats( + original_effect=level, boot_dist=boot_dist + ) + assert np.isnan(se) and np.isnan(p_value) + assert np.isnan(ci[0]) and np.isnan(ci[1]) + + def test_bootstrap_stats_near_constant_distribution_unaffected(self): + """The guard is EXACT-constant only: genuinely varying draws (even by + 1e-9) keep finite inference.""" + boot_dist = np.full(50, 0.1) + boot_dist[0] = 0.1 + 1e-9 + se, ci, p_value = compute_effect_bootstrap_stats(original_effect=0.1, boot_dist=boot_dist) + assert np.isfinite(se) and se > 0 + assert np.isfinite(p_value) + + def test_batch_constant_column_nans_healthy_column_intact(self): + """Batch twin of the constant-distribution guard: the constant column + NaNs out (with the zero-SE warning) while a genuinely varying column + keeps finite inference.""" + from diff_diff.bootstrap_utils import compute_effect_bootstrap_stats_batch + + rng = np.random.default_rng(0) + mat = np.column_stack([np.full(50, 0.1), rng.normal(size=50)]) + with pytest.warns(RuntimeWarning, match="non-finite or zero"): + ses, lo, hi, pv = compute_effect_bootstrap_stats_batch(np.array([0.1, 0.05]), mat) + assert np.isnan(ses[0]) and np.isnan(lo[0]) and np.isnan(hi[0]) and np.isnan(pv[0]) + assert np.isfinite(ses[1]) and np.isfinite(pv[1]) + def test_bootstrap_stats_all_nonfinite(self): """All non-finite samples: fails 50% validity check -> all NaN.""" boot_dist = np.array([np.nan, np.nan, np.inf]) diff --git a/tests/test_efficient_did.py b/tests/test_efficient_did.py index 8bfce357..82d84988 100644 --- a/tests/test_efficient_did.py +++ b/tests/test_efficient_did.py @@ -1563,8 +1563,8 @@ def test_clustered_bootstrap_aggregate_all(self, ci_params): """Clustered bootstrap with aggregate='all' should produce finite results.""" n_boot = ci_params.bootstrap(99) df = self._make_clustered_panel(n_clusters=60, units_per_cluster=3) - # Bootstrapped fits keep the fit-time kwarg: post-fit aggregate() - # fails closed on n_bootstrap > 0. + # Deprecated fit-time kwarg kept as the parity REFERENCE: post-fit + # aggregate() now REPLAYS the fit-time bootstrap on n_bootstrap > 0. with pytest.warns(FutureWarning): result = EfficientDiD(cluster="cluster_id", n_bootstrap=n_boot, seed=42).fit( df, "y", "unit", "time", "first_treat", aggregate="all" @@ -1837,8 +1837,8 @@ def test_balance_e_with_bootstrap(self, ci_params): """Bootstrap balance_e should produce finite SEs.""" n_boot = ci_params.bootstrap(99) df = _make_staggered_panel(n_per_group=80, n_control=80, groups=(3, 5)) - # Bootstrapped fits keep the fit-time kwargs: post-fit aggregate() - # fails closed on n_bootstrap > 0. + # Deprecated fit-time kwargs kept as the parity REFERENCE: post-fit + # aggregate() now REPLAYS the fit-time bootstrap on n_bootstrap > 0. with pytest.warns(FutureWarning): result = EfficientDiD(n_bootstrap=n_boot, seed=42).fit( df, @@ -1916,8 +1916,8 @@ def test_bootstrap_se_finite(self, ci_params): def test_bootstrap_with_aggregation(self, ci_params): n_boot = ci_params.bootstrap(99) df = _make_simple_panel() - # Bootstrapped fits keep the fit-time kwarg: post-fit aggregate() - # fails closed on n_bootstrap > 0. + # Deprecated fit-time kwarg kept as the parity REFERENCE: post-fit + # aggregate() now REPLAYS the fit-time bootstrap on n_bootstrap > 0. with pytest.warns(FutureWarning): result = EfficientDiD(n_bootstrap=n_boot, seed=42).fit( df, "y", "unit", "time", "first_treat", aggregate="all" @@ -1928,6 +1928,28 @@ def test_bootstrap_with_aggregation(self, ci_params): if np.isfinite(d["effect"]): assert np.isfinite(d["se"]) + def test_bootstrap_override_t_is_effect_over_se(self, ci_params): + # Committed pin for the t-recompute semantics of the shared + # percentile-override appliers (bootstrap_utils): on every ES and + # group row, t == effect/se where se is finite-positive, NaN + # otherwise (the two clauses of the safe_inference contract). + # Portable across OS/backends - it pins the relationship, not the + # draw values - and catches a misaligned or altered t routine. + n_boot = ci_params.bootstrap(99) + df = _make_simple_panel() + with pytest.warns(FutureWarning): + result = EfficientDiD(n_bootstrap=n_boot, seed=42).fit( + df, "y", "unit", "time", "first_treat", aggregate="all" + ) + rows = list(result.event_study_effects.values()) + list(result.group_effects.values()) + assert rows + for d in rows: + se = float(d["se"]) + if np.isfinite(se) and se > 0: + assert d["t_stat"] == pytest.approx(float(d["effect"]) / se, rel=1e-15) + else: + assert np.isnan(d["t_stat"]) + def test_bootstrap_coverage_basic(self, ci_params): """Rough coverage check: true effect should be in CI.""" n_boot = ci_params.bootstrap(199, min_n=49) @@ -2642,8 +2664,8 @@ def test_bootstrap_with_covariates_smoke(self): def test_covariates_pt_all_bootstrap(self): """PT-All + bootstrap + covariates end-to-end.""" df = _make_covariate_panel(n_units=300) - # Bootstrapped fits keep the fit-time kwarg: post-fit aggregate() - # fails closed on n_bootstrap > 0. + # Deprecated fit-time kwarg kept as the parity REFERENCE: post-fit + # aggregate() now REPLAYS the fit-time bootstrap on n_bootstrap > 0. with pytest.warns(FutureWarning): result = EfficientDiD(pt_assumption="all", n_bootstrap=99, seed=42).fit( df, diff --git a/tests/test_practitioner.py b/tests/test_practitioner.py index 8b911ce2..f8b0af32 100644 --- a/tests/test_practitioner.py +++ b/tests/test_practitioner.py @@ -582,14 +582,18 @@ def test_aggregation_step_post_fit_branch(self, mock_efficient_results): assert "no refit needed" in steps[0]["why"] def test_aggregation_step_bootstrap_branch(self, mock_efficient_results): - # Bootstrapped fit: post-fit aggregate() fails closed, so the - # guidance routes through the deprecated fit-time aggregation. + # Bootstrapped fit: the recompute levels now REPLAY the fit-time + # multiplier bootstrap, so the guidance routes through the post-fit + # aggregate() calls, never the deprecated fit-time kwarg. mock_efficient_results.bootstrap_results = object() output = practitioner_next_steps(mock_efficient_results, verbose=False) steps = self._agg_step(output) assert len(steps) == 1 assert "BOOTSTRAPPED" in steps[0]["why"] - assert "aggregate='all'" in steps[0]["code"] + assert "REPLAY" in steps[0]["why"] + assert "aggregate('event_study')" in steps[0]["code"] + assert "aggregate('group')" in steps[0]["code"] + assert "aggregate='" not in steps[0]["code"] def test_aggregation_step_name_is_non_steps_key(self, mock_efficient_results): # The "aggregation" key is deliberately OUTSIDE the STEPS