## Errata: what’s a draft pick worth?

Posted by Doug on April 5, 2007

I found an error in my calculations in Wednesday's post. Fortunately, the important conclusions remain essentially unchanged.

What happened was that, in the regressions I ran, each team was being included several times. So the data was essentially right, but the repetitions made the sample size bigger and therefore made one of the coefficients look significant when in fact it wasn't.

In Wednesday's post, I ran three regressions. I claimed that NFL pick value chart value was not statistically significant in any of them, and that Massey-Thaler draft value was significant in one of them. In actuality, neither variable was significant in any of the regressions.

So my conclusion does change a bit. Here is what it used to say:

Assuming the regression is technically OK, these results simultaneously validate Massey and Thalerâ€™s paper and also my argument against it. More precisely, the data show that M-T value is a more relevant measure of draft value than pick value chart value is. But they also show that the distinction is nearly trivial.

It now says this:

Assuming the regression is technically OK, these results validate my criticism of the Massey-Thaler paper. Namely, teams do not appear to be able to the translate the theoretical surplus value they get from their draft picks into surplus production on the field. That's probably because the difference in theoretical average value between draft picks is so small that it's swamped by other factors. One of those factors, of course, is what the teams actually do with those picks. In other words, it's much, much less important for a team to know that pick #1 is on average less valuable than pick #30 than it is for them to know that Peyton Manning is better than Ryan Leaf (if they're picking #1) or that Chad Johnson is better than Quincy Morgan (if they're picking #30). What Massey and Thaler's paper shows is that the NFL draft is a meritocracy, or maybe a luckocracy, but it's not in its present form a mechanism for promoting parity.

Namely, teams do not appear to be able to the translate the theoretical surplus value they get from their draft picks into surplus production on the field.Doug:

Actually, I think the reason is simpler: Massey and Thaler didn't consider the effects of a fixed roster size, and limited numbers of players at a given position. To them, a draft pick which averages $750K of production and costs $250K is better than a draft pick that averages $2.25M production and costs $2M.

If you're talking about quarterbacks, for instance, you can't take 3 pick-75 quarterbacks and merge them in some sort of sick science experiment into a pick-1 quarterback. And while you could imagine that if you take 3 pick-75 quarterbacks, one of them is bound to be nearly as good as a pick-1 quarterback, you'd never be able to find out in a reasonable time frame, and in the time you're taking to figure out which quarterback is good, the team's probably still losing.