How well pitches were actually located.
Next season's command, forecast from this season's locations.
Pitch physics alone: velocity, movement, spin, and release.
Command and stuff combined into one grade.
What drives each pitch type's grade, and how to read it.
How individual pitches add up to a whole arsenal.
Next-season outcomes from historical skill signals.
His earlier seasons were in training, so the model recognizes him instead of grading him.
Reusing the same hidden pitchers for hundreds of keep-or-discard calls turns them into practice data.
A change can look like a win on one run and vanish when the run is repeated.
We pick a random group of pitchers and remove them from every season the model learns from. The model is graded only on them.
A grade and a forecast do different jobs, so they take different tests. Pitch grades are never tested a year ahead, because a pitcher's stuff changes from year to year.
Hidden pitchers' grades vs. what those pitches were worth that season; no credit from location, count, or hitter.
The same hidden-pitcher test; location and count count here, the hitter doesn't.
Each pitch type's grade vs. what that pitch type was worth for hidden pitchers.
Must add something Pitching+ alone misses, on hidden pitchers.
Describes what happened, so it must be steady for the same pitcher and separate from Stuff+.
Learns from earlier seasons only and must predict next year's command better than “same as last year.”
Learns from earlier seasons only and must beat “same as last year” and the standard public projection.
Stuff+ should grade the pitch itself. If a grade looks better only because the pitch came in a good count or against a weak hitter, that isn't stuff, so we check for it.
Every experiment draws its own hidden group, and the old and new versions face the same one. Before a change ships, the locked version takes one final exam on a season nobody tuned on.
Each version is retrained several times on more than one hidden group. In one check, which pitchers were hidden moved the score about ten times more than retraining did. “Not enough evidence” never counts as “no change.”
The current version is re-scored with the same code, data, and pitchers as the new one, so neither gets an easier test.
The tests produce evidence. No model promotes itself.