Player Contribution Error

A lot of people have criticized player contribution for a number of reasons including the linear approach to equating players to value, which is probably a good reason to explain the high player contribution scores for goaltenders without matching salaries (The extra contribution isn’t worth it due to the leveling off of the Pythagorean curve). Of course the simplest and most fundamental error of the player contribution system is that of error.

This document will criticize player contribution in terms of statistical error that was mentioned in the previous document. I will use this test player (average regular defenseman = 20.5 min):

	SH	PP	EV
GF	4	37	51
GA	27	5	47
TIME	16000	20000	65000
GA-STD	5.19	2.24	6.85
95% %err*	38.4%	89.6%	29.1%
GA/(60*players)	1.52	0.18	0.52
Marginal Rate	1.81	0.46	0.81
Difference	0.29	0.28	0.29
STD-DEV	0.29	0.081	0.075
95% %err*	200%	57%	51%
Marginal GA	1.29	1.55	5.236
Defense Score	4.69	5.63	19.04
Final ±	9.4	3.2	9.69
Goals Created	0.5	8	7
GC-STD	0.25	1.31	0.98
95% %err*	100%	29.5%	28%
GC/60	0.112	1.44	0.33
Marginal Rate	0.104	1.05	0.26
Difference	0.008	0.39	0.06
STD-DEV	0.056	0.24	0.054
95% %err*	1400%	133%	180%
Marginal GF	0.04	2.17	1.08
Offense Score	0.145	7.89	3.93
Final ±	2.0	10.5	7.1

Note the Final ± scores shouldn’t change significantly for a player with the same ice times. A linear sum of the scores would suggest error of 42, a square root of the sum of squares suggests 18. Or you can do a linear sum of each type: EV, PP, SH as they’re independent events, and do square root of the sum of squares of these two values and get 29. Even with an error of 20 the best player scores around 100, suggests that the best you can do in the final result is 20% error. In fact if you compare players with scores of 20 or so you’ll frequently see their scores change significantly every season. St. Louis had a score of 84 in 2003, but then scored 118 in 2004 a difference of 34. Niedermayer had 99 in 2003 and performed at 114. McCabe is a great example of error: 44 in 2003, but 105 in 2004. Pronger’s development brought him from -1 in 2003 to 105 in 2004? I doubt it. Elias went from 83 to 103. Brashear went from a marginal 1 to -18. Most of these guys I’ve chosen without special selection (if you don’t believe me look at the PC data at hockey analytics).

What you’re basically doing with player contribution is taking a portion of the GA/60min and subtracting off some constant and then for effect you multiply by a ice time (assumed to not have error) constant to get the marginal value of that goal, but since that error from subtraction is large it propagates throughout the multiplications and becomes just as large as a percent leaving you with essentially worthless player data.

Goalies

However, 10% of the players are goaltenders and player contribution includes these players, how does their error compare. The error is much easier to calculate to the extent that the average goalie sees around 1500 shots and as such has a standard deviation of 0.0077, marginal save percentage is around 0.885, (.900-.885) = 0.015 and is 51% (103% if you’re looking at a 95% confidence interval). Of course if you calculate this difference in terms of plus minus of actual marginal goals it works out to 23.2 marginal goals, this equivalent to a player contribution score of 84, although this makes some sense when you look at Luongo’s score in 2003 of 243 and 346 in 2004 or Andrew Raycroft’s score of 199 in 2004 (don’t ask how he did in 2006), Aebischer in 2003 scored 173 and 49 in 2004. As you can see there is virtually no agreement between the values as my error here suggests, of course these shots are not actually binary events because of shot quality, but its close enough I guess.

* 95% %err: this is the percentage error using the range produced by a 95% confidence interval.