Does squad quality beat Elo? — the model scorecard

The site's whole premise is that paper pedigree and results diverge. Here it is tested properly: on the six complete tournaments (2002–2022; 2026 is live and held out), leave-one-tournament-out cross-validation predicts each squad's actual outlast_pct, and asks whether squad pedigree improves on the Elo Monte-Carlo baseline. 192 squads.

Verdict

The Elo baseline wins. Adding squad market value and pedigree makes out-of-sample forecasts worse, not better — MAE rises from 0.256 (Elo alone) to 0.273 (Elo + Paper Index). Every candidate that prices in squad value lands at best level with, usually below the ratings-only model. The lock's Elo→Poisson method isn't a shortcut — it's what the data supports.

Out-of-sample prediction — lower MAE is better

Fit on five tournaments, predict the sixth, pooled. The Elo baseline is the raw MC sim (no fitting). Predictions clipped to [0,1].

CSV ↓
ModelFormMAERMSE ρ
Elo baselinethe Monte-Carlo Elo expectation itself — what /leaderboards uses0.25620.3151+0.467
Elo, recalibratedOLS(actual ~ Elo)0.26750.3138+0.465
Elo + Paper IndexOLS(actual ~ Elo + squad-pedigree composite)0.27270.3196+0.443
Elo + pedigree vectorOLS(actual ~ Elo + all five z-components)0.27220.3216+0.393

Why pedigree doesn't add signal

Squad pedigree correlates with the Elo rating at r = +0.80 — Elo, being results-derived, already prices most of it in. What's left is the Elo residual (over/under-performance), and no squad feature explains it — every correlation below is statistically indistinguishable from zero, and faintly negative: the fancier the squad relative to its rating, the more it tends to underperform. That is the paper-vs-performance cliff, quantified.

CSV ↓
Squad featureDefinition ρ vs Elo missn
Value Momentumshare of squad with rising value-0.045148
Rust Shareshare under 900 season minutes-0.042192
Paper Indexsquad pedigree composite-0.021192

Honest limits

Six tournaments is a small sample; leave-one-tournament-out is the fair unit (no within-tournament leakage) but the confidence bands are wide. This says squad value adds no bankable signal over Elo at World Cup scale — not that squad quality is irrelevant to football. 2026's rows settle once the tournament finishes; the fit here never uses them. Method: pure-Python OLS, deterministic — see scripts/model_squad_v2.py.

Elo-baseline over/under → · the paper-vs-performance scatter → · method