Introduction
The national hockey league has for years recorded shot locations for goals, saves, and misses,
and unsurprisingly people like me have done a great deal with the data so recorded. However,
doubts have arisen about how accurate these shot locations have been at various times and
places. For instance, consider the New York Rangers shots taken (on top) and allowed
(on the bottom) at 5v5 during
the 2007-2008 season, broken out by home (on the left) and away (on the right).
Red regions show areas with more shots than league average, blue areas with fewer. Immediately we can see that the pattern of shots at Madison Square Garden (where the Rangers play their home games) is dramatically closer to the net. Furthermore, the "tops of the circles" are defined with an unusual clarity on both of the MSG maps, a feature replicated (somewhat) in the rangers shots away from home but not at all in their shots conceded on the road.
Even more unsettlingly, the goaltending results vary substantially. The ratio of goals to expected goals at MSG is around 0.7, both for the rangers goaltenders (mostly, by volume, the in-his-prime future hall-of-famer Henrik Lundqvist; fair enough, perhaps) and also for visiting netminders. On the road, the ratio of goals to expected goals is much closer to one, suggesting that somehow goaltenders consistently perform better at MSG, regardless of which team they play for. One plausible explanation is that the scorers at this rink during these games systematically recorded the shots (for both teams) as closer to the goal than they actually were.
It is a common notion (I am among the people who has popularized this notion) that scorer
bias is an old problem, and that modern shot locations are much improved. This is true,
to an extent, as we will see, but even modern rinks have concerning patterns, such as
the current cup champions:
Here the pattern of the ratios of goals to expected goals is in the other directions; goals are easier to come by, relative to their seeming quality, in Tampa Bay than in Tampa games elsewhere. More generally, the striking similarity of shot locations for both teams in Tampa home games is itself concerning.
Over the years various attempts have been made to measure this scorer bias on shot locations, and to artificially modify the listed coordinates, with the aim of producing a more accurate data set. Most important among them were:
- Krzywicki in 2009;
- Appendix I of Schuckers and Curro (2013); and
- Schuckers and MacDonald comprehensive approach to adjustments for counts.
- Home and road performances are confounded with changes in many other factors, including rest and playing rosters, especially goaltenders. Especially salient here are score effects, the phenomenon of teams playing differently because of the score, especially the leading team and especially late in games. This confounding of score variation with home/road variation is known to not be random, since home teams have won and play more while winning than the road team.
- The above point explains how home/road variation can arise from other factors that correlate with home/road distinctions. However, home and road performances may be different for that reason, that is, because players (perhaps unwittingly) and coaches (somewhat more wittingly) employ different stratagies. For instance, one coach may prefer a riskier style at home, wisely or unwisely; another may prefer to use their home-advantage of last-change to mitigate risk instead.
- Random variation.
Method
To estimate the scorer bias, I used a modified version of my "flagship" shot rate model, magnus that I use for estimating, among other things, player impacts on shot generation and shot suppression at 5v5. I assume that scorer bias at 5v5 can be extrapolated to all situations. Briefly, the model in its "usual" form contains terms for every player on offence, for every player on defence, for home-ice advantage, for the various score states, for shift starts by zone, and for coaches. To this "stock" model I add the following terms for today's purpose:
- For each team, a term for their "home behaviour" on offence, and another for defence.
- For each team, a term for their "away behaviour" on offence, and another for defence.
- For each rink a term for that rink, meant to proxy for the bias in locations introduced by the scorers who work at that rink. I identified the rink with the home team in every case, even when there are multiple physical rinks (such as Nassau Coliseum and Barclay's Center for the New York Islanders), expecting as I do that these distinctions do not affect staffing. Not knowing precisely what to do with special venue games (winter classics, etc.), I decided to assume for simplicity that the rink scorer bias for such games was to be attributed to the home team rink.
Fitting
It is a vital feature of magnus that it is fit with zero-bias penalties, that is, values different from league average are discouraged. Without them the estimates for player impact would be overfit, conforming too tightly to the random peculiarities of happenstance instead of the innate ability of the aspects of the terms that interest us.
I have applied similar penalties to the new terms in this model. The numerical value of the penalty is ten thousand, matching the penalties used for players in magnus. (There is no suitable canonical choice for this value, here ten thousand arises in part because of my normalization of many covariates to have values near one and my use of a hundred-by-hundred grid for discretizing the half-rink.)
Turning Bias Distribution Into Bias Vectors
The "coefficient" values output from fitting magnus are shot maps, that is, one-hundred-by-one-hundred grids of numbers describing the rate of shots at a given point relative to league average. What I want, however, is a vector field, associating to every point on the rink a different point, such that moving all shots in this way will correct for the scorer bias, in total. The road from the one to the other is somewhat hairy. First, I am considering at this time only modifications of location, not the possibility that the overall rate of shots might possibly change. Hence, I first normalize each rink adjustment covariate to have mean zero. Then, for each rink, I form two distributions, each defined on the half-rink: the shots per time as recorded, and the shots per time as recorded plus the bias covariate. Both of these distributions are strictly positive and we want to find the simplest way to describe a point-by-point movement that will turn one pile of shots into the other pile of shots.
In addition to normalizing the bias term to have zero net change, I also symmetrize both shot distributions, that is, averaging them with their left-right-flipped versions, so that the results I get are left-right symmetric. Since the direction of play changes every period but the scorers' equipment does not, I expect any bias corrections should be symmetric in this sense. Furthermore, I clip the distributions to be zero below the goal line and outside of the blue line, since these are the shots in which I am most interested.
Described in this way, the problem fits the pattern of so-called "optimal transport" problems, that of moving a pile of dirt of a given shape to instead have a different shape. Specifically, if it "costs" \(M(p,q)\) dollars to move a unit of dirt from location \(p\) to location \(q\), I would like to find a function \(f(p,q)\) which describes the amount of dirt to be moved from \(p\) to \(q\), and I would like \(f\) to be optimal in the sense that I want both:
- To move as little dirt as possible; if we are going to alter data we should do so gently; and
- For the movements \(f\) to vary smoothly as its inputs vary; since we expect that this matches the behaviour of scorers.
I used a "cost" function \(M(p,q)\) as the square of the distance from \(p\) to \(q\), to encourage many small movements instead of fewer larger movements.
I used a python library (Python Optimal Transport) to find the \(f\) which moves the one shot distribution to the other while minimizing the sum over all locations \(p\) and \(q\) of the following quantity: $$ f(p,q) M(p,q) + f(p,q) \ln f(p,q) $$ The first term \(f(p,q) M(p,q)\) is the cost of the movement, the second term is so-called "entropic regularization", which encodes our desire that the function be smooth; the reader will notice the reapparance of the function \(x \mapsto x \ln x\) which features in the definition of log-loss and other information-theoretic notions.
In practice (because computers) this function \(f\) is known through its values on a grid, and for a given \(p\) there are generally many different values of \(q\) for which \(f(p,q)\) is non-zero; that is, shots recorded at \(p\) are moved by optimal transport to many different locations, some here, some there, and so on. I want to extract for each \(p\) the average movement to as to obtain a single location to which I will move shots recorded at \(p\). Thus, for every shot recorded at a location \(p_0\) I find the four closest locations \(p\) in the grid for which I have \(f\) values and compute the weighted sum of every value of \(f(p,q)\), where each value is weighted according to the exponential of the negative of the distance between \(p\) and \(p_0\).
Results
The full results for each rink for each season can be found in the appendix below. As you can imagine, an adjustment for every location for every rink for every season is a lot of information. To summarize this, I've compared the xG for each unblocked shot taken by either team at the indicated rink using the existing coordinates with the xG for the same shot at the adjusted location. Other shot characteristics (rush, rebound, strength state, etc.) are unchanged.
Goalies who played a lot in rinks in the top part of the graph are not quite as good as their unadjusted results would indicate, and goalies who played a lot in rinks at the bottom of the graph are better than we previously thought. Conversely (and to a lesser extent by virtue of ice-time differences), shooters who played a lot of minutes in the upper rinks are better than we thought, and those who played a lot in the lower rinks are not as good as we thought.
Of course, the unavoidable, extraordinarily salient feature of this graph is the deviation of Madison Square Garden, where the Rangers play, from all of the other teams, although the Islanders are no better recently. All of the other rinks, broadly, are in a tight range. In fact, it seems as though shot location bias in the greater New York metropolitan region has been powerful enough to be visible relative to league average. Perhaps a truly sophisticated approach could compute adjustments as I have done here in a way that satisfies a suitably chosen global condition.
I mentioned earlier that it is conventional wisdom that scorer bias is less of an issue in recent years; this seems somewhat true (certainly it is true for MSG) but the full-league trend seems modest. Also, the early seasons are a little more chaotic, with less year-to-year variation of bias in recent years.
Computing per-shot changes (as above) is my preferred way to look at this adjustment; you may
instead prefer cumulative xG changes:
Note of course that not every season has 41 games for every rink; the first covid season (19-20) and the second lockout season (12-13) have fewer.
Conclusions
The next step is to perform the adjustments described here, computing new location coordinates for each shot in my hockey database. Then, I will need to re-train my xG model using the new coordinates; I expect that this will change the shooting estimates for some players and the goaltending saving estimates for quite a few others. Then, I will use the new xG model to re-train my shot rate model (without the new terms described here) to obtain new estimates for shot rate impact for skaters and coaches. Then, I will regenerate nearly every single image on my website.
Appendix: Home/Road 5v5 shot rate Splits and Adjustments
The "Splits" links show the road vs home performance for every team, using the existing data. The "Adj" links show the adjustment computed here, where the arrows show the movement of each shot and the red/blue colouring shows where we estimate that there are more/fewer shots recorded than there ought to be.
