Goaltending

Goaltending is one of the hardest positions to look at and compare, this is largely the result of extreme multicollinearity of the goaltender with the team. Any goaltender’s results are largely due to what the team is doing, whether it is goals against average due to a poor defense or large quantity of shots. Even the save percentage is affected by the team via the types of shots the goalie sees and the amount of time the goaltender spends killing penalties or in different score situations or just chance. Brodeur has had close to the most wins for many years largely due to the fact he has always played on a great New Jersey team. Just as a goaltender is a function of the team, one also has to remember that the goaltender affects the team’s results as well, it’s a two way street. If the goaltender can’t stop anything then you’ll lose every game no matter how good you are. The obvious question then is: how on earth can one evaluate goaltending?

Goaltenders have been evaluated on a system of: goals against average, save percentage, shutouts, how well their team does and wins. All of which are not true measures of goaltending, but the best one can do at the present time. So much of the goaltending scouting is probably opinions, because the limitations of the data. This article basically summarizes how poor the data is.

Wins

The most important job for a goaltender is to win a game, if the score it 10-9 and he wins, he wins! The fact that he has 9 goals against and a save percentage around 0.7 doesn’t matter because the game was won. Problem is that the goalie wins with the team and the team wins with the goaltender. For example, Legace was considered to have an inflated win statistic, because he played for Detroit. Cloutier (who has always been considered average in Vancouver) was able to capture 3 30 win season in a row.

In order to analyze wins, I broke them into goaltender factors and team factors. The team controls shot quality and quantity so I can follow the shot quality model based on hockey analytics study, using data from 2002-2006. In order to understand goaltender’s ability to wins games I have to first understand the “non-goaltender” probability of winning games, so I assume both goaltenders are identically average and the team with the highest expected goals for wins. I calculate the probably of winning using a Pythagorean winning percentage using the above expected goals and with a binary logistic regression I can see how similar these results are to the actual results. The regression shows that without considering goaltending one can predict game outcomes based on shot quality 84.6% of the time.

To continue, I use the expected goals from each team, and rescale the expected goals to account for the quality of goaltending on the other end. Using these results I can see if the goaltender in question wins or losses and how that compares to how often the goaltender should be winning, since I’ve removed every factor the goaltender cannot effect (to a certain extent).

Now once you get the results you have to look at how it would compare to random results. Due to small sample size I broke the NHL into 4 groups, 0-30%, 30%-50%, 50%-70% and 70%-100%, where the percentages are probabilities of winning a game. If you look at the errors, you will find that the top group (70-100%) random error explains 95% of the variability as such only 5% can be explained by goaltender quality (or error in standard deviation), even the second to top group (50-70%) has 90% error. The bottom group has approximately 70% random error, meaning that approximately 30% could be attributed to goaltender quality. These results show that goaltending play a small factor in determining whether a game is won or lost. This would also make a losing team (who relies heavily on goaltending to get wins) make a goaltender appear better than they are, because every win requires a stellar goaltending. This basically says goaltenders (in general) don’t cause losses, but can win the odd game for the team (which is understood phenomenon). One can conclude that one cannot build a winning team around a goaltender.

It would appear good strategy would be to build a good team and then you can throw any goaltender in front without too much worry so long as the goaltender is at least average; this was likely the circumstance in Carolina’s run to the finals. If you have a bad team a good goaltender is a great way to get wins (that is if you’re able to figure out which goaltender will get those wins). What does this say about worth of a goaltender, a goalie should be somewhere between 5-30% of salary, typically teams spend about 10% on goaltending (this is the correct amount if you’re planning on being around 50% team). It would appear that Detroit is playing smart by spending very little on goaltending and spending on defense, Boston and Thomas are following similar logic. Of course this is all in the perspective on wins. What this says about the value of a goaltender is hard to say, due to the fact that a goaltender is worth more on a bad team and less on a good team. The best way to find out how good a goaltender appears to be: throw him on a bad team. Take Lalime, hugely successful in Ottawa (a team that doesn’t need a stellar goaltender), but failed miserably in St. Louis.

How does this help the binary logistic regression: it helps a little, the variables “0-30%” winning percentage for the home goaltender and away goaltender get the regression up marginally to 85.2%, both variables are significant to 4 standard deviations, it’s nothing to write home about, but agrees with the fact that there’s a lot of error in these results that makes them a poor measurement.

Save Percentage

What is the real job of a goaltender though? Most would say they’re not responsible to win or lose games, but to simply stop as many pucks as possible (preferably at key times). This is commonly measured by save percentage, or the new shot quality neutral save percentage. There’s not much I can add here, but will say that this measurement is inadequate, due to a number of complexities. For example if a goaltender goes on a losing streak giving up a lot of goals in a few games and then wins ten in a row stopping almost everything, they may end up with a bad save percentage, but they will have won 10 games out of 13. Also, one thing people often forget about save percentages is the amount of error is significant, even for goaltenders who play a lot, for example a goaltender who faces 500 shots has approximately 2.5% error, or has a save percentage of 0.908 ± 0.025 or it has a 95% confidence interval of (.934, 882), Luongo with approximately 2500 shots is (.925, .903), which in terms of quality is a huge difference, in fact that range covers the top 20 goaltenders of 2005-2006. When you consider this it can explain why you see a new goaltender on top every year (Huet, Roloson/Kiprusoff, Turco, Theodore, Dunham). So with this in mind you can learn all about shot quality neutral save percentages and marginal goaltending at hockey analytics.

Goals against Average

Since I am talking about goaltending I should mention goals against average. If you’re interesting in looking at this statistic it’s basically save percentage multiplied by shots against average, now if someone can explain how shots against average has anything to do with a goaltender that would be great, but goals against average is a worthless metric as it provides nothing on top of the current save percentage data.

Shutouts

Shutouts are hard to analyze simply because they contain so much error and team components and are always small in quantity. Auld was highly criticized after this season for not making the playoffs and not getting a single shutout (the Canucks first season without a shutout). The simplest way to calculate expected shutout is assume a team faces 30 shots then the probability is simply save percentage to the power of 30, or 0.908^(30) = 5.5%, so a goaltender should get 3 ± 3 (95% confidence interval). So an average goaltender should have anywhere from 0 to 6 shutouts (this sounds about right). However if you consider 25 shots with an above average goaltender (.920)^25, the odds go up to 12.5% (over double), or 9 ± 6, so shutouts can measure quality, but more importantly there’s an exponential decay of “chance of shutout” for each shot against. So Auld who averaged 29 shots against with a .902 save percentage should have 3 ± 3 shutouts. Kiprusoff has a 12.5% chance of a shutout and is expected to have 9.2 (he had 10), so basically shutouts are functions of your other statistics along with some error, there is little additional information one can get from them.

Conclusion

So now, who is the best goaltender? After doing this study I realized there is technically no good way to determine what makes a good, average or bad goaltender, while each measure has its strengths and weaknesses, there is by far too much error. Goaltenders that spend a lot of time on great teams will appear amazing, when they could be average or well below average, there’s almost no good method for quantifying their skills. One could likely generalize goaltenders into groups of good, average and bad, but claiming you have the best or even a goaltender in the top 3 is extremely hard to determine, and even these groups as you will soon see are completely inadequate.

So here’s where I get my head chewed off here’s my good, average and bad goaltenders of the last 3 years, I realize these goaltenders don’t agree to the norms or current goaltender measure, but that’s ok.

Rankings

Good: Legace, Roy, Ward, Markkanen, Aebischer, Raycroft, Burke, Roloson, Nurminen, Shields, Hasek, Brodeur, Turco.
Average: Vokoun, Kiprusoff, Gerber, Miller, Auld, Cloutier, Toskala, Luongo, Joseph, Denis, Grahame, Johnson, Kolzig, Giguere, Fernandez, Huet, Biron, Snow, Niittymaki, Belfour, Esche, Lundqvist, Hackett, Hedberg, Cechmanek, Garon, Khabibulin.
Below average: Boucher, Thibault, Noronen, Dunham, Conklin, Caron, Dipietro, Salo, Prusek, Weekes, Turek, Osgood, Theodore, Potvin, Fleury, Nabokov, Mclennan, Anderson, Aubin, Emery, Lalime.

Note this list is based on winning percentage and no subjective analysis, they are ordered by their score, so Vokoun and Kiprusoff are borderline cases as are Cechmanek, Garon and Khabibulin. What does this tell me? In terms of goaltending Vancouver has done nothing to help the Canucks win games. If you don’t like these groups you can always check out: LCS Player Ratings: Goaltenders.

2007 will be the year of the goaltenders, with a lot of movement and changes, you can almost count the number of teams with the same starting goaltender on one hand, we’ll see who’s right. We will see who sinks and who swims it will be interesting in the end, because it’s always hard to predict which goaltender will do well.