How can I show that outperforming expected goals is the result of luck and not skill? Year over year correlations are the best way. That is to say, do a regression on how well compared to expected results a given player did in 2005-2006 and compare that to identical player in 2006-2007. If skill is a significant component of the data then we should see that good players in 2005-2006 are good in 2006-2007. If this is not the case then there are other reasons (most of which can be attributed to luck).
The graph below is a scatter plot of the Z scores for each year. Z scores simply allow me to normalize out the quantity of shots and create a unitless measure of skill. The amount of variability in the data is about 5-10% greater than I would expect, assuming binomially distributed data, which is likely the result of problems with the shot quality model.
There are small signs that there are players who can outperform expectations. There are a few regressions that are significant at the 5% level, but these regressions explain 1% of all the variability, so I will consider that other influences, such as luck, have a larger effect on the results. I’m not saying that players can’t outperform expected goals; I’m just saying that outperforming expected goals is more likely a sign of luck than a sign of skill.
Here’s the list
The team listed is one of the team’s the given player has played on. It just helps me figure out who the player is. P = F:forward, D:defense, G = goals (not empty nets), D = difference between actual goals and expected goals, Z = z-score of the difference using binomial standard deviation. [sqrt(npq)].The Z-score represents a unitless value about how much better (or worse) a player did than expected. It just creates a nice value to sort the list by. The D value is the absolute number of "unexpected" goals.
Below I’m going through the best and the worst on the list to demonstrate that it makes sense with the extreme values and that skill doesn’t seem to be the reason players come to the top of the list or the bottom of the list.
Explanations of the top 10.
- Semin: Scored 10 goals in the 52 games previous to this season and wasn’t a goal scorer in
Europe. But managed almost 40 in 77 games. He’ll likely be a 20 goal scorer in the future. His goal:assist ratio is 1:0.9 (it should be 1:1.7), this is also a sign of luck. Semin is likely a 20 goal scorer, which is good, but not amazing.
- Souray: This season he should’ve scored 13, but he scored 26 goals. Last season he scored 12 (in a few less games). I don’t need better evidence than that. He’s a “13 goals per season” goal scorer (excellent for a defenseman).
- Visnovsky: 44 goals in the 348 games (0.13 g/g) prior to this season. 18 goals in 69 games (0.26 g/g) should’ve scored 8 goals (0.12 g/g).
- Kuba: Never surpassed 10 goals his entire career, but was able to score 15 goals this season. I’d bet he’ll closer to 7 than 15 next season.
- Huselius: Many people were most impressed with Huselius this season and for good reason he scored over double his previous season. But he exceeded expectations by about 12. Meaning he should’ve only had about 19 goals and he’s historically been a 20 goal scorer.
- Staal (J): He’s going to be good, but 8 shorthanded goals and a goals:assist ratio > 2:1 is insanely uncommon. However since he’s young, he’ll be better next year so I wont say how well he should do, but I expect it be somewhere between 20 and 30. And I expect a lot more assists.
- Horton: Two almost 30 goal season back to back, you’d think there’d be something going on. Horton is more likely a 20 goal scorer than a 30 goal scorer, but he’s still young so he’ll get better as well.
- Lecavalier: You always need a little luck to win the big trophies and Lecavalier is no exception (still a great player). But scored 16 more goals than expected, which would bring him very close to his previous season’s scoring of 35 goals.
- Roche: Who? 6 goals on 32 shots. I need not say more. I expect him to disappear next year.
is covered with outperforming players and Drury is the leader of the pack. In 2005-2006 Drury scored 30 goals and 18 in 2003-2004 or 48 goals in 156 games for 0.31 goals per game. In 2006-2007 he scored 37 in 77 or 0.48 goals per game. He has maintained this scoring level in the playoffs (7/13), but I wouldn’t bet on it continuing forever. Buffalo
Explanations of the bottom 10.
probably thought they were getting a deal trading away dead weight at the end of the season, but Kobasew should’ve had 9 extra goals, which would have basically tripled his nice total of 5. 14 goals in 50 games would’ve brough Kobasew quite close to his 0.26 goals per game record in 2005-2006. Calgary
- Hoggan: Who? Was demoted after scoring 0 goals in 46 games.
- Cullen: 25 goals in 78 games last season and 16 goals in 80 games this season. My numbers say he should’ve scored 28. Sounds reasonable to me.
- Niedermayer (R): Lowest goals per game in his career and the closest he has come to this horrible scoring total was 1994-1995, when he got 4 goals in 48 games (21 year old). He should’ve had 10, which is close to his normal historical performance level.
- Gomez: 33 goals to 13 goals, need I say more?
- Hollweg: Who? Probably won’t ever be a goal scorer, but probably won’t see such a bad year as this twice.
- Sedin (H): 18 goal season to 10 goal season. Won’t be a goal scorer, but he should’ve scored a lot more
- Svatos: Amazing rookie season, but only scored 15 this year. He should’ve scored about 25, which is great number of goals in 66 games. Probably score 30 goals next season.
- Marchant: Historically scored 0.18 goals per game and this season scored 0.11.
- Robidas: 0 goals on 106 shots. He’s not a goal scorer, but that’s just doesn’t happen very often to forwards.
All this said, being lucky this year doesn’t lower your chances of being lucky next year, so any prediction could be wrong simply because they could be lucky two years in a row (or unlucky for that matter).
I am planning to go through the players and adjust assists based on how lucky the scorer was all season. So, an assist on a luck player will be worth less than an assist on an unlucky player. So Daniel Sedin’s assists on Henrik Sedin’s goals will be worth more than assists on Drury goals. I’ll then see if the results make sense.