|
The field of serious hockey analysis has been open for roughly a decade now, with detailed stats such as shot distance, takeaways, drawn penalties and hits now available for puckmetricians to delve into. Unfortunately, one of the serious flaws with these stats is that every arena seems to have its own definition of everything. 40 feet is actually 30 feet if you’re in Madison Square Garden, while a missed shot will almost never be recorded as such in Chicago. It seems like the only reliable statistics are goals, penalties and shots, and one of these may not be as reliable as it seems.
With data provided to me for every team in the NHL by Puck Prospectus contributor Gabriel Desjardins, I have compared the number of shots recorded in every team's home games with the number of shots recorded in every team's road games for the four seasons from 2005-06 to 2008-09. The result does not inspire confidence:
Total shots recorded by both teams, 2005-2009
Team Home Road Difference
New Jersey 9171 9963 -792
St. Louis 8746 9516 -770
Minnesota 9094 9689 -595
Dallas 8557 9108 -551
Chicago 9281 9731 -450
Pittsburgh 9536 9901 -365
Vancouver 9179 9535 -356
Carolina 9826 10038 -212
Washington 9986 10174 -188
Los Angeles 9593 9731 -138
Philadelphia 9858 9961 -103
Edmonton 9308 9378 -70
New York I 9949 9992 -43
Atlanta 9955 9905 50
Phoenix 9728 9661 67
Toronto 9880 9800 80
Detroit 9863 9781 82
Calgary 9461 9363 98
Buffalo 9863 9754 109
Montreal 10054 9906 148
Tampa Bay 9662 9497 165
San Jose 9343 9147 196
Boston 10111 9906 205
New York R 9793 9580 213
Columbus 9535 9243 292
Florida 10745 10316 429
Anaheim 9866 9371 495
Ottawa 10182 9609 573
Colorado 9825 9236 589
Nashville 10068 9226 842
Given a total of roughly 19,000 shots, the home-road difference could be expected to be about 140 on luck alone, so differences of 800 are clearly due to some kind of bias. There are clearly arenas in which more and less shots are systematically recorded. This leads me to wonder: are the shots simply being underreported, or are there arenas (or teams) which are simply conducive to less shots?
As an aside, for those who think that the number of shots is a firm, objective number, I invite you to view this video.
How many shots do you count on this play? Watch it again. The game log, from Florida, says 4. In St. Louis, this would have been 2 shots.
To figure out whether the number of shots was actually being undercounted, I figured I’d look at goals. Surely, all things being equal, if less shots were truly being taken, less goals would be scored, right?
Here is the sum of stats for the 8 arenas with a strong “less shots” bias:
Home Road H/R Ratio
Shots 73390 77481 94.7%
Goals 7238 7325 98.8%
Here is the sum of stats for the 8 arenas with a strong “more shots” bias:
Home Road H/R Ratio
Shots 80125 76487 104.8%
Goals 7346 7285 100.8%
In both cases, roughly 5% more or less shots are recorded, but the goal difference is only 1%. This leaves only two possibilities: either some arenas are recording less shots, or somehow the arenas with less shots simultaneously feature more dangerous shots on average. While I find the second possibility remote, it is a distinction without a difference: in both cases shot data is skewed, resulting in biased shooting percentages and save percentages.
Who Does This Affect?
The good news is that, since shots at both ends of the ice are affected, shot differential metrics, such as Corsi, are almost untouched. Player shooting percentages and shots on goal, while interesting statistics, are rarely ever used to judge players, with good reason: because a player controls both how many shots he takes and how well he takes them, it is rarely useful to rank players by these metrics, except when looking for outliers whose stats are likely to mean regress. Where this bias will have the most impact is on goaltender save percentages.
In the “more shots” category, I was not surprised to find Nashville and Florida. I’ve always found something fishy about Florida’s inflated shot totals, both during the old Luongo days and more recently with Vokoun. Likewise, Nashville always seems to have a goaltender with a 0.920 save percentage (Tomas Vokoun, Chris Mason, Dan Ellis and Pekka Rinne). Though the most remarkable bias comes on the “less shots” side and is found, unsurprisingly, in New Jersey.
Whenever any hockey analyst makes any breakthrough, it always seems like it applies to the New Jersey Devils and Martin Brodeur. The Devils are the team that everyone loves to hate, everyone loves to redefine and analyze. Statistical analysts have made it their job to deconstruct Martin Brodeur’s reputation as one of the all-time elite goaltenders; one blogger even named his site after him. I am about to become the first statistician to attempt to enhance Brodeur’s reputation.
One well-known and impressive statistical nugget about the Devils is how well, historically, they have managed to keep shots against to a minimum; the last season where the Devils allowed more shots than the league average was 1989-90. Since then, over the course of 18 seasons, they have averaged about 330 shots against less than the league average per season, despite going through different coaches and massive personnel changes (Brodeur notwithstanding). I believe we now have one explanation for this phenomenon: by my calculations, New Jersey’s home shots against have been underreported by about 100 a year. This would still leave them as a defensive juggernaut (preventing 230 shots against per season is no joke) without making them such a statistical outlier.
If this is true, it could further legitimize Brodeur’s claim of being one of the all-time greats. Brodeur’s career all-time shots against (25,481 as of this writing) could be too low by as much as 1,000 shots, which would boost his career save percentage from 0.914 to 0.918. This would boost his career GVT by roughly 80, which would place him #3 or #4 on the all-time list, neck-and-neck with Jacques Plante and only trailing Dominik Hasek and Patrick Roy.
I would love to have somebody else validate my analysis, if possible using data from other seasons. Until then, I will maintain a healthy skepticism on the reported shot totals in the NHL. As we continue to push back the frontiers of hockey analysis, I realize that we are building our edifice on sand.
Tom Awad is an author of Hockey Prospectus.
You can contact Tom by clicking here or click here to see Tom's other articles.
| 
0 comments have been left for this article. 
Conn Smythe Watch: Lundqvist Leads