|
Following up on last week’s article, I determined that if we can extract the amount of skill in the distribution of goaltending across the league, we can also perform an estimate of the skill of each individual goaltender in the league. Today I will attempt to do just that.
First of all, the methodology: I have used the same 3 seasons of data, 2007-08 to 2009-10, that I used in last week’s article. I decided to perform an estimate for the 40 goaltenders who have seen the most shots at 5-on-5 and 4-on-5, so the cutoff was Alexander Auld’s 2,296 shots. I also assumed that each goaltender had the same “true” talent for the 3 years, and that is what I’m estimating, so I’m not taking into account potential injuries, changes due to personal circumstances (Jean-Sebastien Giguere) or aging (Tim Thomas, Chris Osgood).
I took the distribution of skill from last week’s analysis. The league average save percentage was 0.910, and the amount of skill observed in the actual distribution of Goals Versus Average (GVA) implied that the standard deviation was 0.005. I then calculated, based on this distribution of skill, what was the most likely true talent for each goaltender, and what was the uncertainty of our estimate. The more shots a goaltender faced, the more certain we are of his actual skill level. Because we are looking especially at #1 goaltenders, more than half will be above average.
Note for Math Geeks: I performed a Bayesian analysis to do this. As the a priori distribution, I took a normal distribution with an average of 0.910 and a standard deviation of 0.005. I know the distribution of goaltender talent, especially at the NHL level, is not normal, and certainly not symmetrical, but it made the math easier and didn’t change the results much. I then used a binomial model to calculate what were the odds that each initial save percentage could have yielded the observed results.
One last thing: all I am looking at is shot-blocking ability. Nowhere have I attempted to account for other goaltender skills, such as puck handling or rebound control.
First, the results:
Rank Goaltender True Talent Rank Goaltender True Talent
1 TOMAS VOKOUN 0.9176 21 CRISTOBAL HUET 0.9108
2 TIMOTHY THOMAS 0.9172 22 MANNY LEGACE 0.9107
3 JONAS HILLER 0.9167 23 JASON LABARBERA 0.9106
4 JAROSLAV HALAK 0.9157 24 CAREY PRICE 0.9104
5 HENRIK LUNDQVIST 0.9156 25 JONATHAN QUICK 0.9102
6 MARTIN BRODEUR 0.9144 26 ILJA BRYZGALOV 0.9101
7 CRAIG ANDERSON 0.9137 27 MARTIN BIRON 0.9098
8 TY CONKLIN 0.9135 28 ANTERO NIITTYMAKI 0.9097
9 ROBERTO LUONGO 0.9134 29 JOSE THEODORE 0.9096
10 CAM WARD 0.9133 30 MIIKKA KIPRUSOFF 0.9092
11 RYAN MILLER 0.9131 31 DWAYNE ROLOSON 0.9088
12 PEKKA RINNE 0.9129 32 ALEX AULD 0.9083
13 NIKLAS BACKSTROM 0.9128 33 STEVE MASON 0.9081
14 NIKOLAI KHABIBULIN 0.9126 34 MATHIEU GARON 0.9076
15 CHRIS MASON 0.9121 35 MIKE SMITH 0.9070
16 MARC-ANDRE FLEURY 0.9121 36 CHRIS OSGOOD 0.9069
17 KARI LEHTONEN 0.9121 37 MARTY TURCO 0.9066
18 EVGENI NABOKOV 0.9112 38 JOHAN HEDBERG 0.9053
19 JEAN-SEBASTIEN GIGUERE 0.9109 39 VESA TOSKALA 0.9048
20 DAN ELLIS 0.9108 40 PETER BUDAJ 0.9044
My colleague Rob Vollman calculated the rankings of save percentage since the lockout, and Tomas Vokoun came out 1st by a pretty big margin, so it’s no surprise to find him at #1 here. In the top 5, we see that 2009 playoff hero Jonas Hiller and 2010 playoff hero Jaroslav Halak are not complete flukes, especially given that this is only regular-season data. The top 5 surprise is Tim Thomas, who most people wrote off as a fluke after his numbers came back to Earth this year, but he had 2 excellent seasons (2007-08 and 2008-09) as well as multiple excellent seasons in Europe. During the current “disaster” season, his save percentage was 0.915; we should all be so lucky. It’s quite possible that age is catching up to him, as he is already 36, but his track record is rock solid.
In the following graphics, I have listed every goaltender, as well as the 50% and 90% confidence intervals for the true talent of each. The thick rectangle represents the 50% confidence interval: there is 1 chance out of 2 that a goaltender's true talent lies within that rectangle. The thin extremities are the 90% confidence interval: there are 9 chances out of 10 that a goalie's true talent lies within those points.
You can see that the confidence bars are not the same length for everybody, although since I restricted myself to goaltenders with 2,300 shots against they’re all reasonable. For goaltenders who have seen a lot of shots (Vokoun, Lundqvist, Kiprusoff) we have a pretty good idea of their talent range, while for Halak, Conklin and Auld we’re less certain.
What we see is that, for almost three-quarters of the goaltenders in the NHL, their performance is almost indistinguishable from one another. We know for sure Tomas Vokoun is better than Peter Budaj and Vesa Toskala, but that’s about it. This explains why the NHL goaltender leaderboard changes so much from one season to another, and why nobody agrees whether goalie X is really good or just overrated. When fans were debating whether team Canada should have brought in Marc-Andre Fleury or Cam Ward, it turns out that our best guess is that they’re equivalent.
It also shows that public perception of goaltenders is very much affected by team success and track record and less by actual talent and performance. Of the upcoming goaltender free-agent class, the four currently highest-paid goalies are Evgeni Nabokov, Marty Turco, Jose Theodore and Vesa Toskala. Nabokov is seen as a blue-chip goaltender and Turco as a dependable veteran (granted, Turco also has puck handling skills not quantified here), while the perception of Theodore has much to do with him being pulled in Game 2 against Montreal. But none of these goaltenders is any better than Chris Mason, a less touted name, and they’re not demonstrably better than Dan Ellis either.
In practice, the assumption that there is little skill difference between most NHL goaltenders is correct. You can rarely count on your goaltending to win you a game or a series because Brian Boucher can outplay Martin Brodeur over 7 (okay, 5) games, but there is still some difference. Next week I’ll attempt to establish how much you should pay for goaltending.
Tom Awad is an author of Hockey Prospectus.
You can contact Tom by clicking here or click here to see Tom's other articles.
| 
0 comments have been left for this article. 
Conn Smythe Watch: Lundqvist Leads