May 17, 2010
Howe and Why
Quality Starts

by Robert Vollman

Mid-season last year we introduced the concept of Quality Starts to describe when a goalie was playing well enough to win, regardless of whether or not the team in front of him scored enough goals to actually secure the 2 points. Without further ado, let's take a look at this year's leaderboard.

Legend:

QS: Quality Starts (think of it like Wins)

NQS: Non-Quality Starts (think of it like Losses)

QS%: Quality Start percentage (think of it like Winning Percentage)

Goalie           Team       QS NQS QS%
Tuukka Rask      Boston     27 12 69.2%
Ryan Miller      Buffalo    45 23 66.2%
Evgeni Nabokov   San Jose   45 26 63.4%
Henrik Lundqvist NY Rangers 45 27 62.5%
Jimmy Howard     Detroit    38 23 62.3%
Miikka Kiprusoff Calgary    44 28 61.1%
Ilya Bryzgalov   Phoenix    46 27 60.9%
Mathieu Garon    Columbus   16 11 59.3%
Martin Brodeur   New Jersey 45 31 59.2%
Jaroslav Halak   Montreal   25 18 58.1%
...
Chris Osgood     Detroit     8 13 38.1%
J-S Giguere      Toronto    12 20 37.5%
Manny Legace     Carolina    9 15 37.5%
Alex Auld        NY Rangers  7 13 35.0%
Vesa Toskala     Calgary     7 19 26.9%
*Minimum 20 starts

Last year Tim Thomas topped the list with 70.4%, and won the Vezina, and this year it's his teammate Tuukka Rask topping this list at 69.2%. Henrik Lundqvist slipped from 2nd to 4th, but is the only goalie to appear in the top 5 both of the last two seasons.

On the flip side, Vesa Toskala somehow declined from being the 5th worst in the league in Quality Start % last season to finishing dead last this year. Jean-Sebastien Giguere and Chris Osgood both made return appearances on this list, making you wonder just how far name recognition can take you these days.

Calgary's Miikka Kiprusoff and Ilya Bryzgalov of the Coyotes made surprise appearances on our leaderboard after ho-hum results last season. Here's a list of this year's most improved goalies.

Legend:

08-09: Quality Start % in 2008-09

09-10: Quality Start % in 2009-10

DIFF: Difference between 2009-10 and 2008-09 Quality Start %

Goalie           Team         08-09 09-10 DIFF
Johan Hedberg    Atlanta      33.3% 47.6% 14.3%
Miikka Kiprusoff Calgary      47.4% 61.1% 13.7%
Ilya Bryzgalov   Phoenix      52.4% 60.9%  8.5%
Jose Theodore    Washington   45.5% 53.5%  8.0%
Jaroslav Halak   Montreal     51.5% 58.1%  6.6%
*Minimum 20 starts both seasons

If you include those with fewer starts you could mention Peter Budaj, Jason LaBarbera, Pascal Leclaire, Mathieu Garon, Michael Leighton and Ondrej Pavelec, too. Here's a quick look at those whose careers went in the opposite direction.

Goalie           Team         08-09 09-10 DIFF
Alex Auld        NY Rangers   57.9% 35.0% -22.9%
Tim Thomas       Boston       70.4% 53.5% -16.9%
Vesa Toskala     Calgary      43.4% 26.9% -16.5%
Niklas Backstrom Minnesota    64.8% 48.3% -16.5%
Mike Smith       Tampa Bay    57.5% 41.7% -15.8%
*Minimum 20 starts both seasons

The Methodology

Take Pekka Rinne and Chris Mason as a harmless example of how traditional statistics don't tell the whole story. They both had identical GAA of 2.53 in roughly the same number of starts - 61 for Mason and 54 for Rinne, and their save percentages were only 0.002 apart. And yet, Mason's Quality Start percentage of 50.8% is noticeably better than Rinne's at 46.3%. Why? Because he played more consistently.

Using traditional statistics the only real difference between the two was in shutouts: Rinne had 7 and Mason had 2. Does that make Rinne better? Far from it! For every extra shutout, he also must have had an extra 5-goal night. Would you be better off with a goalie that consistently allows 2.5 goals per game like Mason, or someone that has more shutouts and 5-goal nights like Rinne? Gauging the consistency of a goalie is the primary purpose of Quality Starts.

Technically a Quality Start is defined as any game where the starting goalie stops more shots than the median save percentage (but with one caveat, discussed below). Last season, that median was 0.917, up from 0.913 the year before. I rounded off the save percentage required to 92%, especially since I noticed that it made a big difference in actual winning percentage.

There's a little more to the Quality Start statistic than just save percentage. If the goalie stops between 88.5% and 91.9% of shots, the goaltender is considered to have played well enough for his team to win only if he allows fewer than 3 goals. The reasoning is that if you're only facing 20 shots, stopping all but 2 is usually enough, but if you're being peppered with 40, you'd better let in fewer than 4 if your team is going to win. Observe the difference:

Legend:

GAA: Approximate goals-against average (minutes played can cause it to vary slightly)

WIN%: Team winning percentage in such cases

 GAA   WIN%
~2.00 0.663
~3.00 0.480
~4.00 0.389
~5.00 0.346
*For starting goalies stopping between 88.5% and 91.9% of shots

While the object of Quality Starts is to remove team bias, some of it is inescapable, and that's why you may prefer to define Quality Start based only on save percentage, and sacrifice this closer correlation to actual winning percentage. If so, you may want to use the standard deviation of a goalie's save percentage to measure his consistency (but that's outside the scope of this article).

Wasted Quality Starts

When a goalie plays well enough to earn the victory, but the team's anemic offense fails to produce enough goals, we call that a Wasted Quality Start. Here is a list of the least fortunate goalies, finding themselves on teams unable to consistently pick points off the silver platter.

Goalie        Team         QS WQS WQS%
Martin Biron  NY Islanders 10  6 60.0%
Tomas Vokoun  Florida      34 16 47.1%
Carey Price   Montreal     18  8 44.4%
Tim Thomas    Boston       23 10 43.5%
Brian Boucher Philadelphia 14  6 42.9%
*Minimum 10 Quality Starts

Much has been made of Jaroslav Halak's 26-13-5 regular season record compared with Carey Price's 13-20-5, especially in light of his postseason success. It's true that Halak had a better season, besting Price 58.1% to 46.2% in Quality Start percentage, but the Habs also wasted 44.4% of Price's Quality Starts compared with only 24.0% for Halak, and they also bailed out Halak in a third of his poor starts, but only 14.3% for Price. With comparable support their win-loss records would have been much closer.

As for the more fortunate netminders, Vesa Toskala's teammates knew how important it was to take advantage of his rare Quality outings (7), and didn't waste a single one -- only Chicago's Antti Niemi had more starts without any waste (18).

Bail-Outs

On the flip side, when a goalie plays poorly but the potent offense generates enough scoring to reward such play, the goalie is said to have been bailed out. Here's our list of the most fortunate netminders last season and as you can see, Chicago Antti Niemi appears once again among the most fortunate.

Goalie             Team       NQS BO BO%
Semyon Varlamov    Washington 12  5 41.7%
Pekka Rinne        Nashville  29 12 41.4%
Antti Niemi        Chicago    17  7 41.2%
Jose Theodore      Washington 20  8 40.0%
Nikolai Khabibulin Edmonton   11  4 36.4%
*Minimum 10 Non-Quality Starts

It's no surprise to see a goalie bailed out by a high-octane offense like Washington, Chicago or Edmonton. Wait a minute ... Edmonton??? Jeff Deslauriers was only bailed out of 3 games, despite having 27 non-quality starts (11.1%), and Devan Dubnyk wasn't bailed out of a single one of his 11 non-Quality starts, but somehow Khabibulin was lucky enough to get over half of the team's bail outs. Lucky Khabi!

Dubnyk doesn't hold the distinction of being the least fortunate goalie, as Chris Osgood had 13 non-Quality Starts without a bail-out, though Mike Smith should get mention for only being bailed out once in his 21 non-Quality outings.

Wrap Up

Quality Starts, a concept borrowed from baseball analysts, tells us exactly which goalies are consistently keeping their teams in a position to win. Speaking of consistency, we learned that Lundqvist is a top-5 goalie, that Antti Niemi is the league's luckiest netminder, and Osgood, Giguere and Toskala are riding their big names without living up to expectations.

Coming up next we'll take a look at the goalies coming in for struggling or injured starters. Who is the best relief goaltender in the NHL? Find out next week.

Robert Vollman is an author of Hockey Prospectus. You can contact Robert by clicking here or click here to see Robert's other articles.