August 8, 2006

Strength of Opposition: The Question of Existence

Hockey sets itself apart from many sports in the sense that coaches can choose, to a reasonable extent, which players play against other players. In baseball you can’t pick your pitcher, in basketball you can’t pick your opponents as you don’t have shifts. In soccer the same 11 guys play for almost the entirety of the game. In individual sports your opponent’s strength is determined by how far you get or by seeding. Football there are no lines, just offense and defense and the players on either side stay relatively constant (ok, I don’t know much about football). In fact I’d be hard-pressed to think of a sport with similar dynamics in terms of shifts and coaches determining the strength of opposition they face, but is this really the case, can coaches really determine opponents, or do things occur as a random mixture.

Naturally with such a phenomenon lazy-boy hockey fans will often claim a player is doing poorly because they face too difficult of opposition, or do well because they have easy opposition. You will often here fans site: “difficult minutes” as a determining factor in raising or lowering a player’s value. I will attempt to figure out to what extent opposition effects players’ performance. Personally I would like to see results where opposition is irrelevant as it would make hockey studies a lot simpler; however I expect to see some players being protected and others being “abused”.

A few examples of people discussion strength of opposition.

  • “Donovan's recent three year average is 1.5 ESP/hr. So Friesen's an upgrade if you assume they have comparable GAA and strength of opposition. However, that rate upgrade costs a marginal $975K. I'm not sure that's worth it, unless Friesen can do that against tougher opposition or if he can keep down the GA at a better rate than Donovan.” (RiversQ).
  • Horcoff appeared to have considerably more difficult strength of opposition” (RiversQ).
  • “All this number crunching is nice, but it's not clear that the arbitrators are looking at things like strength of opposition” (Robert Cleave).
  • MacT creates a Samsonov-Stoll-Hemsky line that gets all the easy minutes.” (Dennis).
  • “Well, he has been getting lot of easy minutes compared to Salei and Vish, i.e. he plays more against opponents 3rd and 4th lines.” (Pepper).
  • “That PPV game was worth every nickel. It was Pronger's signature game this year, IMO. He played a ton of difficult minutes in a place the Oilers hadn't done well, and was, even by his standards, a calming presence” (Robert Cleave).

My methodology is probably not rigorous enough, but it should be simple enough for most to understand. I rated offense via shooting percentage and defense using expected goals against per minute (given an average goalie). I redistributed these values on a 0 to 100 scale, using 50 as average with approximately 20 as the standard deviation (so 68% of the players are within 30 and 70). I would expect to see the most protected player seeing offense of about 25 and defense well near 50. So I processed every second of every players time (took 43000 seconds) and scored their opposition using a linear scale: sum of opposition score divided by five (goalie not counted). So offense example: Ohlund (56), Baumgartner (58), Morrison (67), Naslund (60), Bertuzzi (68) would score 61.8. So if you only played against this line you should have an opposition offensive score around 62. To find out a player’s score you just sum up all the opposition scores for every second on the ice and divide by amount of time on ice. Method to calculate defense is identical. Note: I will use short forms for offensive opposition score (OOS) and defensive opposition score (DOS) for the rest of this article.

After spending hours processing the data, the results were pretty innocuous. I got distributions with much smaller standard deviations (1.5) than the players’ individual scores (20). This basically says that players play against a diverse group of players some of whom are good others who are bad, in other words most strength of opposition is averaged out to nothing significant. There are two very informative graphs: The first is OOS vs. ev ice-time; the second is DOS vs. ev ice-time. These graphs indicate collapsing of error and there is basically not much, if anything can be gained here.

With the differences, mentioned above, showing poor if anything useful about strength of opposition, I needed to find out how much error was reasonable in this problem. Now assuming a shift length is 54 seconds (average shift length), I ran a script that played a player against random opponents for 500 shifts a thousand times. The standard deviation was 0.4, compared to 1.5 for OOS and 2.3 for DOS. With this in mind I can only explain approximately 30% of OOS and 80% of DOS (the rest is going to be random error).

Looking at some individuals, many phoenix players were all given low DOSs, as such I remembered that results in this years NHL were largely due to teams players played against as such, over 60% of the variability of defense can be attributed to which division you are in, the rest may depend on the teams you played against in the other conference or injuries. The regression can be improved by adding plus minus and/or the individuals’ defensive score, this is likely just the caused the OOS and DOS effecting things like plus minus. This simplifies strength of opposition analysis a lot, because you can just look at the caliber of the teams played against to determine if a player is under or over rated.

Doing the same for the offensive OOS is much less effective (7% of variability explained); using more or less variables doesn’t help. It would appear regressions may be the wrong technique here; there must be a better way to look at this. When you remove the team factor you’ll quickly notice the error drop substantially. If you only include players that played in more than 80 games on the same team you will see the error match that of randomly generated player data.

This likely wont convince you, but I should add that I did a number of regressions and there were a few "notable variables, in general it appeared to be the team you played against and not the players.

Players do face varied opposition, but this variability stems from the teams they play and not a result of being protected by a coach. There is no data showing significant coaching effect on difficulty of opposition, either offensively or defensively. This means that the concept of “strength of opposition” can only relate to team they’re playing and is not affected by playing on a different line. Coaches don’t protect players by using line matching (they may line match for other purposes).

In the future, I hope to furthur analyze whether coaches "protect" players by using other players on their own team to create "balance".


Jeff J said...

Very interesting. The notion that a coach will try to protect weak players from tough opposition completely ignores the fact that there is another coach on the other bench theoretically trying to get his top players on the ice against those same weak players. It should be a wash.

Showerhead said...

Wow, I have just recently discovered your blog and quite enjoy it. I wouldn't be surprised if the limited responses are more from a lack of general ability to discuss the math than because people don't find your posts interesting.

I have a couple of issues with what you've done here though. First off, what is your rational for using shooting percentage as an indicator of offensive ability? I assume you used a minimum number of shots and sorted your data appropriately; however, without knowing your rationale I can not defend shooting shooting percentage over total points, points per some given icetime, or even ice time per game.

With a top 10 shooting % from 05/06 including names such as Prucha, Carter, Holmstrom and Eaves it shouldn't be hard to see that the numbers don't mesh with the hockey reality in this case. I toyed with the idea of finding a way to combine Pts/hr with TOI/game but found that all I was really doing was normalizing points with games played - perhaps you'll have better success here but I think that from a hockey perspective your measure of offense has to include scoring rates. Ideally you would separate ES and PP as well but I'm rambling and you've long since gotten my point.

The other issue I have is that by putting all 30 teams into the same melting pot, you may be diluting the tendencies of some coaches and exaggerating the tendencies of others. Bloggers such as Vic Ferrari and RiversQ have shown strong tendencies in Edmonton for players like Horcoff and Peca to get much higher percentages of Thornton/Sakic/etc minutes than other Oilers. I can't speak for other teams/coaches but Craig MacTavish absolutely matches lines.

If you were going from the perspective of sheltering players only, then the above paragraph misses your point. I would think that the type of player you need to shelter is also the type to be at the bottom of the heap in minutes played. If you were a coach who matched lines, would it not be more effective to try hardest to get the matchups you considered ideal for your top minute eaters?

It's true that some coaches go power-vs-power but it's just as true that others go checking-vs-power and try to exploit weaker opposition with their own power lines.

Anyhow, as a final note and again using Edmonton as an example... My observation of MacT's 4th line strategy was that not only did he try to ensure his 4th line wasn't going up against the other team's stars but he also did his best to ensure his 4th line was getting on the ice with possession or with an offensive draw. These are two factors that it occurs to me are difficult to measure with current published NHL stats.

Unfortunately for the reader, I've thought of an additional point since declaring the previous paragraph to be my last one. As Jeff J points out, what if every coach in the league had exactly the same success with line matchups? Indeed you'd find a wash. You could get into an interesting debate as to whether it was best to try so hard and simply run in place or let it ride or even game theory it up a little.

I have typed too much and said only an adequate amount. Phew.

Earl Sleek said...

As I indicated at my site, I also have some difficult-to-articulate issues with this opposition line-neutral argument. Surely there must be a difference between playing vs. McDonald/Selanne or playing vs. Pahlsson/Moen over the long haul.

As we would expect these two tandems to focus on different aspects of the game ("different sides of the rink", if you will), why would players playing against these varying lines not feel any effect?

It might be an issue with metrics. Expected goals are already an average of 1,200+ games' experience, but in measuring an opponents' effectiveness in affecting goal-differential, perhaps it is better to use actual scoring in this exercise.

Maybe a 30-foot slap shot against Pahlsson has worse success than it does against McDonald, because Pahlsson's got his stick on your stick and his hip on your hip. Also I would expect a 30-foot slap shot from Pahlsson to be different than one from McDonald's stick.

Like showerhead, all I can spit out is a lot of text with little cohesive thought, but given a choice, an NHL player would have a preference of which line to play against in a game.

Your stuff here seems to suggest that he is a fool for thinking that might matter.

JavaGeek said...

These are good questions.

First off, what is your rational for using shooting percentage as an indicator of offensive ability?
Offense is hard to measure, mainly there are two things for scoring:
quality and quanitity if both mattered even strength goals for per minute would be the best measure, but in my offense article I showed that shots are the result of a given team and not a players decision (this is a little strange), meaning that the only thing that matters is quality: shooting percentage, I use "line shooting percentage" (gives credit for every goal and shot while on the ice) where this is the list of the best NHLers. I should note: Carter is ranked 176th/406, or a bit better than average. The point is to seperate offense from defense (possession from not possession), if you never have the puck you'll have lots of shots against and no shots for, that doesn't make you necissarily poor offensively, but extremely poor defensively. If you scored on 5 times on 10 shots, you'd be great offensively, but so terrible defensively your offensive would be worthless.

Ideally you would separate ES and PP as well
I rarely combine them. In this study I only looked at even strenght, I should have made that more clear

The other issue I have is that by putting all 30 teams into the same melting pot, you may be diluting the tendencies of some coaches and exaggerating the tendencies of others.
I did look at team based scales, interestingly Edmonton had a low amount of difference between players for offensive and defensive scores. It was hard to base anything on the results (as teams that I would not consider line matching teams came on top and others like Edmonton were in the middle). So you can see some "attempted" sheltering that results in a zero net effect.

players like Horcoff and Peca to get much higher percentages of Thornton/Sakic/etc minutes than other Oilers.
I should hope so he plays more in general. Take Sakic and Horcoff. Horcoff: 8465s vs Col., Sakic VS. Edm. 8265s, Sakic and Horcoff: 2200s, which works out to 47%, 45% and 26.6% is approxemately the 21% expected time together. That 3 minutes wont make a huge statistical difference. Also, you forget that a team can have extremely good competitive second lines, if fact what differentiates Sakic for Laperiere, for example is their defensive game. So Sakic gets more shots and thus more goals.

Indeed you'd find a wash.
So I guess were in agreement...

Like showerhead, all I can spit out is a lot of text with little cohesive thought, but given a choice, an NHL player would have a preference of which line to play against in a game.
No question there, obviously a player would choose the worse defensive player to play against to maximize their own goals, but the fact is each player gets about the same ammount of time with each group and thus it balances out.

I should state that this article is more about opposition affecting stats, than not playing vs. varying oppositions

JavaGeek said...

I recently discovered a small bug in my script, meaning every player was only being compared to the first period's oppositions (so if you're on the ice at 19:38 in the third I would compare that to the opposing players playing at 19:38 in the first period - Oops.). This means my analysis was correct, ie. coaches don't base their third period ice time decisions on the first period (that will be all random).

However, this means this article has to be re-done! When it is I will delete this one...