GROTUS' ACORN

God girl grill gridiron
Showing posts with label we shall reconvene in the nerdery. Show all posts
Showing posts with label we shall reconvene in the nerdery. Show all posts

Thursday, October 8, 2009

Method for Quantifying Whole-Team Performance

The question is, what’s the best way to look at a team’s performance on the whole? We basically have three important units by which we measure a football squad. The first – and most ultimately important – is the scoreboard. The second is yards, either yards given up by the defense or yards claimed by the offense. And the third is turnovers. Being an engineer and seeing three units, my impulse is to combine the three into some useful metric. According to this article at Sports Quant, fumbles and interceptions both result in about the same amount of lost points or yards. Smart Football reader Brad Eccles makes the case that fifty yards is a statistically-reasonable penalty for either a fumble or interception – that is, if you fumble, subtract fifty from your offensive yardage total. I’ve also seen (though I can not remember where, so let me know if you run across that number or a better one) that fifty yards is an appropriate bonus for plays that result in a touchdown. This seems about right to me. Thus, we can convert turnovers and touchdowns into yards and express a team’s production using a single unit. Moreover, what’s good for me is bad for my opponent and vice-versa: penalties against Auburn should be recorded as negative yards, and penalties against opponents as positive yards.

This includes the offense and the defense: the offense’s forward progress can be expressed as positive yardage, and the defense’s as negative yardage. As we learned last year, the offense and defense are not distinct.

I’ve borrowed another idea from Smart Football. There, he describes a method of evaluating a play’s success called the Sharpe Ratio. Now, I have my own issues with the Sharpe ratio – dividing by standard deviation is not a good idea when, in particular, most running backs’ success depends on their big-play potential. However, one important thing contained in that statistic is the evaluation of a play in comparison to its risk. We could theoretically run a short screen, a slant, or a quarterback draw every single play and gain a dependable, undefendable two yards every single time. While this would not be an effective strategy, all plays must be evaluated against this standard. For instance, an incomplete pass is essentially two yards given up forever. Thus, the yardage that results from each play is adjusted down by two.

I usually make some judgment about what constitutes “garbage time” and ignore that yardage. If one team is clearly bleeding clock, they’ve essentially stopped running their offense and it shouldn’t count. Also, I ignore what I consider to be uncoachable plays or dumb luck, like interceptions returned for touchdowns – these don’t count as an interception and a touchdown, just as one or the other. If the defense gets the ball, they’ve won and a win is worth fifty. Same thing for special teams touchdowns.

By this method, we can chart out our team’s progress over the entire game, in both phases of the game, as a single function. This is just what I’ve done. I copy the play-by-play at ESPN.com (more reliable than Rivals) into a text file, import that text file into a spreadsheet, and use the wonders of MS Excel to semi-automatically create a running list of the plays that are run, the players who run them, the down, the distance, the location and the result. Then, once I have an adjusted yardage for each play, I graph a running sum of offensive and defensive yards – this is essentially a running, risk-adjusted yardage differential. It gives some idea of the arc and flow of the game, and unites the offense’s and defense’s statistics. The goal, obviously, is to be in the positive by the last whistle. Granted, this does not reliably correlate with a team winning the game – field goals, defensive touchdowns, and special teams touchdowns are not included. But that’s not the point – I just want to see which team is most effective when they get down to work.

(I originally posted this at The War Eagle Reader, where you can see the results.)

Wednesday, June 10, 2009

Gridiron Skew Part 1

Something you have already guessed: your humble blogger is a bit of a nerd. And, I really nerd out on football. It is a great sporting frontier of obstinate complexity. Twenty-two men with different, overlapping and constantly fluctuating jobs operate from a multitude of formations on a field that, conceptually, is massive, continental, vast. This is no pitcher-vs-batter game, is not even as simple as basketball. Football is to other sports as chess is to checkers.

So naturally, it's exciting to an amateur mathnerd like myself to see that folks out there on the intertubes are trying to make sense of the pile of data generated by each single football game. Granted, some of this is of the "We calculate this metric from a privileged position" variety, or baldly dismissive of core mathematical concepts1. But Chris Brown of Smart Football really digs up some gems, and he's more than willing to tantalize us with his methods. He talks about game theory and its application to run/pass balance. He mentions the Sharpe ratio when evaluating the effectiveness of a play. Every step wrings a little more heuristics out of the great game. This, my friends, is awesome.

For instance, the aforementioned Sharpe ratio. As Chris puts it:
The Sharpe Ratio is defined as the ratio of the difference between the expected return of some strategy minus the expected return of some riskless benchmark and the standard deviation of the strategy.

[ed: This can be written as such for those of us who like equations]

Standard deviation is a measurement of the volatility of a set of values. You can read more about standard deviation here. For example, if a given pass play was run four times, and the results (in yards) of the play was 10, 10, 10, and 10, it has a standard deviation of 0. However, if it was run four times and the results were 0, 0, 40, and 0, it has the same average gain (10 yards) but its standard deviation would be 20. We would prefer pass play one to pass play two. It has the same expected gain, but it is less risky than the second.

This is all well and good. What's more, the Sharpe ratio may be used to determine the ratio with which you use one strategy (or one play) over another. If one play's Sharpe ratio is 2 and the other play's Sharpe ratio is 4, you should use the latter play twice as often as the former. The power of this method is clear when you begin sectioning your data ever more finely - the Sharpe ratio allows us to compare any number of situations. How do your different running plays fare on second down and less than five yards? Out of which formation is play-action most effective? At what position on the field is your offense most able to run the HB draw to the near side out of a pro set in the fourth quarter on third down with more than four yards to go? With enough data, the Sharpe ratio can help you answer that question.

However, the Sharpe ratio is weak in a critical aspect: it punishes inconsistency. Why is that a weakness? Take, for example, the first pass play postulated: absolute, robotic 10 yards per attempt. Sounds like a great play. What could be better, though, is a play that results in a 20 yard gain every fourth attempt (10, 10, 10, 20.) It's clear to everyone and their blind, demented uncle that 10, 10, 10, 20 is better than 10, 10, 10, 10. And yet, the Sharpe ratio is misleading. Because we divide by standard deviation, a play's inconsistency - whether the play succeeds wildly or fails miserably - is punished. Essentially, when you calculate the Sharpe ratio, you are implicitly assuming that an offensive big play and a defensive big play are equally horrible. This weakness becomes especially clear when one starts adjusting yardage totals for touchdowns and turnovers. A helpful guy out on the interwebs has run the calculations: a touchdown or a turnover is basically equivalent to 50 yards. But if your goal line play wins you a touchdown every fourth attempt - certainly an effective play - that's 1, 1, 1, and 51. Standard deviation is so high that it will obliterate your statistical decision-making.

I'm picking on the Sharpe ratio because its promises are so tantalizing, and yet I think it asks the wrong question. What we should be ascertaining is the level of skew in the yards gained by each play. Aside perhaps from Jim Tressel, coaches value big-play capability just as much as they dread the defense's ability to ram a big play down their throat. We all want our teams to be explosive. Just, explosive in the downfield direction. To put it in the terms of the shape of a distribution, we want the yards-per-play distribution to have a negative skew, like so (via): Skew answers the question, does a play regularly succeed? If there is negative skew, the answer is yes. So how do we reward negative skew in our analyses?

I should probably put the rest in another post...




1. Direct quote:
I do agree ... that R-squared gives you "the proportion of variance that is in common between NBA team payroll and NBA team performance." But what does that mean? Almost nothing, unless you're a statistician.
Similarly, one could declare that "I do agree that USC hammered the seams in Penn State's overmatched cover 3 defense, but what does that mean? Almost nothing, unless you're talking about football." May as well come out and say "I don't understand how to use this metric but I'm going to criticize it."