This is the full description of the research that I presented at the Vancouver Hockey Analytics Conference on January 16, 2016. The slides for that presentation can be found here, where readers should note that the term "pressure" was used instead of "leverage" as here. Some of the figures still say "pressure".
Intuitively we know that some moments in some hockey games matter more than others. The final seconds of a close game seem very important, where the early moments of a game allow both teams plenty of times to adjust an ill-thought-out strategy or recover from an unlucky goal or injury. I have made a model to measure just how much each moment in a hockey game matters; letting us talk about moments of "high leverage" and moments of "low leverage". I show that some players see appreciably more leverage than other players, relative to their icetime. This excess appears to be the deliberate action of their coaches.
To measure how important each moment in a hockey game is, I made an expected points model. That link explains the model in detail but the central idea is to measure how many points the home and the road team are likely to take from a game given the game time, the score, and how many skaters each team has on the ice. By comparing how many points a team is likely to take and how many points they will take if a goal is scored we can isolate moments of high pressure. When a team will likely gain a large number of points by scoring (for instance, when tied or trailing by one late in a game) we say that they are in a high offensive leverage situation. Conversely, when a team will likely lose a large number of points by conceding a goal (for instance, when tied or leading by one late in a game) we say that they are in a high defensive leverage situation.
Leverage varies from game to game
Not all games have the same amount of overall pressure. Here we measure the average leverage of both types over the course of a game:
Here are the games from the 2015-2016 season, displayed by regulation win (blue), regulation loss (red), and games that were tied after regulation (black). Mostly winning teams see higher defensive leverage (they have leads to defend), and losing teams see higher offensive leverage (they have deficits to overcome). The points close to the origin are snoozers, games where one team jumps out to an early lead and then piles on goals. Unsurprisingly, games that are not settled in regulation generally are higher in both types of leverage; also surprising is that the effective upper limit of total leverage seems to around one standings point per second, on average, suggesting that plenty of games are as tight as theoretically possible.
Leverage varies within games
Of course, pressure varies considerably over the course of a game:
Unsurprisingly, pressure of both types for both home and away teams largely increases as the game goes on. Defensive leverage is larger than offensive leverage almost all the time for both home and road teams. This unfortunate discrepancy pushes teams to defend more than they attack in a way which is very reasonable: teams mostly have more to lose by conceding a goal than they have to gain by scoring. I suspect that the cause of this may be the point system, specifically, the extra "winner" point that is given to one of the two teams despite them both failing to win in regulation. The curious exception to this is the home team incentives very early on in games, when offensive pressure is momentarily higher. It seems that a very early goal for the home team is worth more than the cost of conceding early---the so-called 'wake up call' goal may well wake up home teams.
Leverage is mostly determined by icetime
The variation in leverage from time to time is not large enough, however, to defend saying that
a player who plays two very important shifts is under the same pressure as a player who plays
ten less important shifts. The relationship between the amount of icetime a player sees and the
fraction of the game's leverage that they experience is very, very high:
We do see that there is slightly more spread in defensive leverage, however. As we shall see, even after controlling for icetime, we will see that some players see more leverage than others.
To measure what sort of leverage a player is seeing, we simulate all of the other ways that a coach could have played a player in a given game while keeping their icetime the same. So, for instance, if a forward plays fifteen minutes, I consider all of the ways that they might have played those fifteen minutes---with most of their shifts early on, with perfectly spaced gaps, with most of their shifts late in the game, or any other pattern of shifts. By simulating a great many such patterns of shifts, all adding up to fifteen minutes, and measuring the leverage observed, we can compare the leverage a "typical fifteen minute player" would see to the leverage that player actually saw.
High-leverage areas of games generally cluster together, since goals are rare, and thus we expect that many of the least realistic simulated deployments (such as the ones featuring five shifts in six or seven minutes, which we not observe) will have some of the most extreme leverages. This means that the method we use is of low power and likely understates the deliberateness of actual deployments.
For instance, let's take a look at a game of Alex Chiasson's, versus Boston on November 16, 2015:
The observed leverages largely follow the full league pattern, with the higher minute players (Turris and Stone) seeing the highest leverage and the low-minute players (Smith, Neil) seeing much less. However, Chiasson had an unusual fifteen minutes. The blue regions show the expected leverages for a "randomly distributed" fifteen minutes of icetime in this game; the dark region (where Pageau coincidentally appears) is the most likely region. Chiasson himself appears in a very faint region, suggesting an unusual deployment, it seems as though he was kept away from situations where Ottawa needed to score and given minutes instead at times when Ottawa had likely points to defend.
To determine if the variations we see are more than we would expect from random variation, we compute z-scores for offensive and defensive leverage for each player-deployment in each game, and form an aggregate z-score by summing the individual scores and then dividing by the square root of the number of game measurements. This aggregated z-score gives us an estimate of how likely the data points are to be drawn from a normal distribution; in this case we obtain p-values of 0.058 for offensive leverage and 0.004 for defensive leverage. Given the low power of our method we feel encouraged that we have identified a non-trivial effect.
We can make a density plot of the observed deviations from "typical" deployment:
We have already seen from the low p-values that this data is likely not normal -- the bulk of the non-normality comes from the fact that the overall spread of the data is larger than we would expect. However, we can also see from the above that the peak is slightly negative both defensively and offensively. This suggests that, in aggregate, coaches view some of their players as defensively or offensively weak more than they view others of their players as strong. It seems that coaches prefer to hide certain players from high-leverage situations instead of deliberately exposing certain players to high-leverage situtations, consistent with general risk aversion that we see in many coaching decisions. It may also be that well-trusted players are simply given more icetime, where there is less scope for deliberate leverage deployment.
Conclusion
I do not know yet how to properly analyse which players are succeeding in high-leverage situations and which ones are failing; nor can I suggest when players in low-leverage deployments should be moved to higher-leverage ones. However, we can be sure that:
- Leverage is a measurable effect that varies non-trivially between games and in-games;
- Players who play more are more exposed to high-leverage situations;
- Even controlling for icetime, some players see varying levels of leverage; and
- This variation appears to be the deliberate choice of coaches.
Appendix: Leverage Deployments per team
Below I have tabulated the observed leverage deployment of all thirty teams for the 2015-2016 season. Keep in mind that the leverages are relative to given icetime. Players with very deliberate patterns of deployment in few minutes are under much less pressure than players with neutral deployment but who play more.
