H2 Confirmatory Pre-Registration [REGISTERED 2026-07-11]

This document freezes the confirmatory test of the batter breakout/ fade signals BEFORE any holdout evaluation. The holdout is evaluated EXACTLY ONCE, results published regardless of outcome (miss-inclusive doctrine). Any deviation from this protocol must be disclosed as a deviation in the results.

Data-hygiene disclosure

• All signal development to date (docs/research/h2-exploratory- 2026-07-11.md) used TRAIN predictor seasons only: 2015-2019, 2021.
• H5 stability estimation used year-pairs across 2015-2025 (not outcome-linked; disclosed).
• HOLDOUT predictor-season outcomes have never been analyzed.

Universe

Qualified batters (PA >= 200). HOLDOUT predictor seasons: 2022 and 2023 (the only holdout seasons with complete next-2-season outcomes as of registration; 2024-2025 predictors are excluded until their outcome windows close, and would require an amended registration).

Variables

• Outcome: war_next2 = FanGraphs WAR(t+1) + WAR(t+2) (bat rows, outcomes_war.parquet; players absent in a future season = 0 for that season).
• Baseline predictor: war_t (same-season FG WAR).
• Signals (from quality_bat.parquet, park-adjusted build of 2026-07-11): ev95 (95th-pct exit velocity) and luck_gap_adj (park-adjusted actual-minus-expected value on contact).
• Standardization: z-scores computed with TRAIN-estimated means/SDs (frozen; holdout distribution never used for scaling).

Frozen models (estimated on TRAIN, coefficients locked before

holdout)

• M0: war_next2 ~ b0 + b1*war_t
• M1: war_next2 ~ b0 + b1war_t + b2ev95_z + b3*luck_gap_adj_z Both fitted on TRAIN qualified batters (n=2,128 fitting sample) and locked. Holdout evaluation applies the frozen coefficients.

Pre-registered success criteria

PRIMARY (all three must hold):

1. Out-of-sample R2(M1) − R2(M0) >= +0.01 on holdout.
2. Spearman(M1 prediction, war_next2) > Spearman(M0 prediction, war_next2) on holdout.
3. Re-estimating M1 on holdout: luck_gap_adj coefficient negative with p < 0.05.

SECONDARY (reported, not gating): 4. Breakout check: define war_delta = war_next2/2 − war_t. Top quintile of the signal overlay (b2ev95_z + b3luck_gap_adj_z, frozen coefficients) has higher mean war_delta than the bottom quintile; bootstrap 95% CI (10k resamples) of the difference excludes zero. 5. Fade check: top quintile of luck_gap_adj alone has mean war_delta below the sample mean; bootstrap 95% CI excludes zero. 6. Youth cohort (age 22-24) replication of 1-3; small n, reported descriptively.

Interpretation rules (decided now)

• All PRIMARY pass → breakout/fade signals are validated for publication; product calls may cite the holdout numbers.
• PRIMARY 3 passes but 1-2 fail → fade signal publishable alone; breakout claims held back pending more data/features.
• All fail → the signals are TRAIN artifacts; the product's calls fall back to descriptive framing (no predictive claims) until a revised, re-registered model earns them. Published either way.

Explicitly out of scope (require separate registration)

Pitcher signals (Stuff-lite pending); MiLB/AAA signals; any model using components beyond ev95 + luck_gap_adj; weight sweeps of the CR composite.

Execution

Single script run, added as catalyst/analysis/h2_confirmatory.py at evaluation time, mirroring this document exactly. Recommended: git commit of this document BEFORE evaluation for timestamp integrity.