PurgedKFold::split leaks: test window starts at the first test label's END time - #66
Conversation
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…n test rows split() built its purge window from the END time of the first test label and the end of the last one. Training labels overlapping the first test labels were kept (leakage), and with variable-length labels the window could invert (start after end), purge nothing, and leave the test rows themselves in the training set. Use the first test label's start and the latest end in the fold (AFML 7.3), and drop the test indices explicitly. new() now rejects n_splits < 2 or > n_samples, which previously panicked (division by zero / empty fold). Tests: an exact 3-day-label case, a seeded property test over variable-length spans asserting no train label intersects any test label, and the n_splits bounds. The first two fail before this change. No existing expectation moved. Fixes #65 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
Review notes Worse than filed in #65. The property test showed that with variable-length labels the old window could invert ( Blast radius. Behavior changes
Evidence. Two of the three new tests fail on Merge note. #64 also edits |
|
CI note: |
Scope
[code / P0]
PurgedKFold::splitincrates/openquant/src/cross_validation.rsbuilds its purge window from.1(label END) of the first test sample instead of.0(label START), and takes the window end from the last test sample rather than the max end over the fold. Training samples whose labels overlap the leading test labels are therefore not purged (label leakage), and variable-length labels can under-purge the trailing edge. In scope: correct the window, add a regression property test that fails before the fix, check every other caller/copy of the same window logic in the crate (the freeml_get_train_times-style helper near line 95, feature-importance and hyperparameter-tuning CV paths) and the docs page for the module. Out of scope: Python bindings (#42), re-porting 59ac6fa (#33), changing embargo semantics.Acceptance
Closes #65