Gridiron Skew Part 1

Something you have already guessed: your humble blogger is a bit of a nerd. And, I really nerd out on football. It is a great sporting frontier of obstinate complexity. Twenty-two men with different, overlapping and constantly fluctuating jobs operate from a multitude of formations on a field that, conceptually, is massive, continental, vast. This is no pitcher-vs-batter game, is not even as simple as basketball. Football is to other sports as chess is to checkers.

So naturally, it's exciting to an amateur mathnerd like myself to see that folks out there on the intertubes are trying to make sense of the pile of data generated by each single football game. Granted, some of this is of the "We calculate this metric from a privileged position" variety, or baldly dismissive of core mathematical concepts¹. But Chris Brown of Smart Football really digs up some gems, and he's more than willing to tantalize us with his methods. He talks about game theory and its application to run/pass balance. He mentions the Sharpe ratio when evaluating the effectiveness of a play. Every step wrings a little more heuristics out of the great game. This, my friends, is awesome.

For instance, the aforementioned Sharpe ratio. As Chris puts it:

The Sharpe Ratio is defined as the ratio of the difference between the expected return of some strategy minus the expected return of some riskless benchmark and the standard deviation of the strategy.

[ed: This can be written as such for those of us who like equations]

Standard deviation is a measurement of the volatility of a set of values. You can read more about standard deviation here. For example, if a given pass play was run four times, and the results (in yards) of the play was 10, 10, 10, and 10, it has a standard deviation of 0. However, if it was run four times and the results were 0, 0, 40, and 0, it has the same average gain (10 yards) but its standard deviation would be 20. We would prefer pass play one to pass play two. It has the same expected gain, but it is less risky than the second.

This is all well and good. What's more, the Sharpe ratio may be used to determine the ratio with which you use one strategy (or one play) over another. If one play's Sharpe ratio is 2 and the other play's Sharpe ratio is 4, you should use the latter play twice as often as the former. The power of this method is clear when you begin sectioning your data ever more finely - the Sharpe ratio allows us to compare any number of situations. How do your different running plays fare on second down and less than five yards? Out of which formation is play-action most effective? At what position on the field is your offense most able to run the HB draw to the near side out of a pro set in the fourth quarter on third down with more than four yards to go? With enough data, the Sharpe ratio can help you answer that question.

However, the Sharpe ratio is weak in a critical aspect: it punishes inconsistency. Why is that a weakness? Take, for example, the first pass play postulated: absolute, robotic 10 yards per attempt. Sounds like a great play. What could be better, though, is a play that results in a 20 yard gain every fourth attempt (10, 10, 10, 20.) It's clear to everyone and their blind, demented uncle that 10, 10, 10, 20 is better than 10, 10, 10, 10. And yet, the Sharpe ratio is misleading. Because we divide by standard deviation, a play's inconsistency - whether the play succeeds wildly or fails miserably - is punished. Essentially, when you calculate the Sharpe ratio, you are implicitly assuming that an offensive big play and a defensive big play are equally horrible. This weakness becomes especially clear when one starts adjusting yardage totals for touchdowns and turnovers. A helpful guy out on the interwebs has run the calculations: a touchdown or a turnover is basically equivalent to 50 yards. But if your goal line play wins you a touchdown every fourth attempt - certainly an effective play - that's 1, 1, 1, and 51. Standard deviation is so high that it will obliterate your statistical decision-making.

I'm picking on the Sharpe ratio because its promises are so tantalizing, and yet I think it asks the wrong question. What we should be ascertaining is the level of skew in the yards gained by each play. Aside perhaps from Jim Tressel, coaches value big-play capability just as much as they dread the defense's ability to ram a big play down their throat. We all want our teams to be explosive. Just, explosive in the downfield direction. To put it in the terms of the shape of a distribution, we want the yards-per-play distribution to have a negative skew, like so (via): Skew answers the question, does a play regularly succeed? If there is negative skew, the answer is yes. So how do we reward negative skew in our analyses?

I should probably put the rest in another post...




1. Direct quote:

I do agree ... that R-squared gives you "the proportion of variance that is in common between NBA team payroll and NBA team performance." But what does that mean? Almost nothing, unless you're a statistician.

Similarly, one could declare that "I do agree that USC hammered the seams in Penn State's overmatched cover 3 defense, but what does that mean? Almost nothing, unless you're talking about football." May as well come out and say "I don't understand how to use this metric but I'm going to criticize it."