S-002Did not hold
Locked forward test of promoted names
Does the edge found in S-001 survive on data the model has never seen?
Result
0 of 93 names held their edge. The model trailed a naive baseline.
- Significant after correction
- 0 / 93
- Mean forward accuracy
- 60.0%
- Majority-direction baseline
- 75.7%
- Names above baseline
- 7 / 93
What it means
The in-sample screen did not carry forward. This Signal is not shipped. Publishing the failure is the point of a locked Study: it stops a good-looking backtest from becoming a Signal anyone reads.
Locked setup
- Universe
- The 93 names promoted by S-001
- Window
- Training cutoff 2026-07-16 to 2026-07-22; frozen 2026-08-13
- Method
- Model and features frozen on 2026-08-13 (xgboost, 5 PCA components, 5-bar horizon). Each name is scored only on bars after its training cutoff. Accuracy is compared with the majority-direction baseline (always guessing the more common direction) using a one-sided binomial test, corrected across all names.
Breakdown
| Universe | Names | Forward bars | Accuracy % | Baseline % | Significant |
|---|---|---|---|---|---|
| S&P 500 | 56 | 780 | 61.4 | 76.7 | 0 |
| Nifty 500 | 37 | 394 | 57.9 | 74.2 | 0 |
Replay it
python scripts/run_locked_forward_replay.pyOutputs: data/research/locked_forward_test_registry.csv, data/research/locked_forward_replay_results.csv