December 19, 2010
Benchmarcs
Performing to Expectations?

by Marc Foster

One of the earliest Sabermetric tools adapted for hockey was Bill James’ Pythagorean Analysis. It was based on the idea that a team scoring 800 runs and giving up 800 runs should theoretically finish the season with a .500 winning percentage, and conceived to determine what winning percentage can be expected based on different values for and against. The key here is expected, because it’s the variance that makes this an interesting measure. Teams don’t perform as expected, for which we have to determine why.

But before I jump in, let’s look at how we determine a team’s Pythagorean Winning Percentage (Pyth%):

Where GF = Goals For; GA = Goals Against

You can adjust the exponents based on the era to reduce the average total error, downward as you go back in time, but for general use, there’s no reason to adjust it because it’s the outliers on the bell curve we’re most interested in, and tweaking things isn’t going to change the status of those teams.

Once a Pyth% has been calculated, you can multiply it by GP x 2 to convert to standings points, which is an easier value to compare than percentage points.

Now, when I originally adapted this tool for hockey, times were easier (Rob Vollman was kind enough to remind me that there are now 30 teams in the NHL and that the Rockies play in New Jersey). The reason things were easier was because the standings were simpler. There was none of this three point game nonsense. With three point games, everyone was going to overperform relative to Pythagoras based on how many OT points they received, which doesn’t tell us much. I believe I have a solution for this, however. The base equation stays the same, but when converting to standing points, you use the average value of a game. A third of the way into this season, the average value of a game is approximately 2.224 points.

Now that we have a conversion factor, the idea of three point games actually makes the tool more interesting, because now you can evaluate performance and look at it in the context of how a team is performing beyond regulation.

In his 2010 NHL Review, Alan Ryder examines the idea of lucky and unlucky teams and suggests that it’s better to be an unlucky team in the regular season. I don’t view teams performing above or below Pythagorean expectations as lucky or unlucky. Instead, they are either over- or under-performing.

But let’s go back to Alan’s thought about whether it’s good to be lucky or unlucky. I can see where he’s going with this, especially if teams are unlucky in shootouts and to a lesser degree in regular season overtime. There is no shootout in the playoffs and overtime is a different animal, and it’s probably best to predict that performance based on an even strength Pythagorean analysis (a subject for another column). But I pulled together the data on the Expansion Era prior to the advent of the third point, and based on the top/bottom five performing teams, there’s not much evidence that regular season “luck” is important to postseason success.

Top overachieving teams of the expansion era—pre-OTL point

Top underachieving teams of the expansion era—pre-OTL point

So among the top overachieving teams, we have two in the Finals and one winning a Cup. The 1979-80 Flyers, for what it’s worth, lost to an Islanders team who was within a point of their projected value. The 1974-75 Bruins, on the other hand, lost in the first round to a mediocre Chicago team that finished with 82 points in the very weak Smythe Division, so their underachieving ways under coach Don Cherry continued into the playoffs. I realize that this is a small dataset, but I think there’s some value in exploring this further and analyzing all seasons since expansion started for trends.

All that said, what’s going on today? Here are where things stood through December 9th, exactly one-third into the season.

So Tampa Bay has about three more wins than they should, but that’s not a big surprise given their goaltending. They were a fairly neutral 2-3 combined in OT and SO, so that’s not really impacting their numbers. But looking a few spots below at Edmonton, we see a team 3.4 points above expectation that’s 0-4 in shootouts. I don’t believe in luck, but if they were breaking even in shootouts they’d be close to the Lightning as the most overperforming team thus far.

Looking at the bottom, we find a couple of teams with excellent goaltending in Florida and Boston who are the most underperforming teams. Is there a relationship between quality of goaltending and Pythagorean performance? Perhaps. I do intend to look at the standings at the two-thirds mark to see if things hold. That should give us some time to look for trends in the past.Marc Foster is a frequent contributor to Hockey Prospectus.