This is good fodder for the weekend.
Recently, a bunch of sports-related talking heads got up on a stage at M.I.T. for something called the Sloan Sports Analytics Conference. The panel of note, at least for me, was "What Geeks Don't Get: The Limits of Moneyball" which featured Jonathan Kraft and Bill Polian along with Mark Cuban and Bill Simmons.
More interesting? The take of Aaron Schatz, founder of Football Outsiders, a site that most serious football fans are familiar with. Schatz, needless to say, was not very happy with the proceedings. Money quote:
The most annoying part, for me, is when Polian basically says "there's nobody doing analysis worth listening to." This is about an hour after his son had personally told me that everyone in the Colts front office reads FO and they generally feel we're on the right track. So I was a bit pissed off.
OK, first things first.
I've been reading Football Outsiders for just about the entire time they've been on the web. I'm a big fan. They were the first people to give you something more than "you need to establish the run" as commentary. I met Aaron at the Combine, we had dinner and got along famously. He's a passionate football fan, as dedicated to the game as he is his work, which is superb.
But that doesn't mean Polian doesn't have a point, even if he made it in his characteristically abrasive fashion.
Even someone as dedicated to statistical analysis as Schatz is has to recognize that there's a limit to what we can do as fans. No matter how many times we watch our DVR copies of games, no matter how many times we log in to NFL Game Rewind, no matter how many incredibly interesting though thoroughly perplexing stats we come up with on our own, we are essentially hobbyists dabbling in someone else's profession.
That's not to say we don't have things of consequence to say and it doesn't mean we shouldn't be listened to. I understand Aaron's frustration - he's worked long and hard over the course of the last decade and people that matter in NFL circles know his work. I talked to tons of people in Indy who are regular readers, both in the NFL and in the journalistic community. And to be fair, his site isn't strictly numbers - what gets lost is that there is some downright amazing writing on there. (Tanier's Walkthrough and Farrar's Cover-3 are simply weekly must-reads for any football fan)
But all the writing and all the numbers in the world by fans doesn't capture a complete picture of what happens on every play of every football game. The football people have that. Polian himself even eludes to this saying:
There's nobody out there working on quantifying all that goes on, 22 guys, 60 times a game
There's a reason for this - the NFL guards the coaches tape as though it were a state secret, which, for the NFL, it kind of is. The obvious inference here? If the stat geeks ever got a hold of the coaches film, they would pull the curtian back on Oz even more than they already have.
The football people, on the other hand and armed with said coaches tape, have every tendency, every formation, every medical report on every player, every piece of history of every coordinator in the league including, at the press of a button, a cut-up of every third down throw from the right hashmark that the slot receiver caught for a first down, etc.
All of this, and they've been doing it for longer than a lot of these stat guys have been alive.
What drives me to distraction about most stat-driven analysis is how its proponents hold it up as some kind of talisman that they and they alone are capable of understanding. They bandy about numbers as though they are their own facts rather than the part-to-the-puzzle they truly are. Even worse, stat geeks often fail to provide any context to the numbers they hold so dear.
To be fair, I know I sound like I'm completely anti-stats, completely anti-FO. This is far from the truth, and I love this from Schatz:
I take pride in the fact that Football Outsiders leads the league in couching our opinions in caveats
Knowing what you do and don't know is a very important thing - some stat geeks are better at this than others.
Enjoy the video and I'd love to get your thoughts on the subject.