APBRmetrics

**jsill** · Joined: 19 Aug 2009 Posts: 73

Schtevie:

Thanks very much for your feedback.

It's hard to say for sure how big the APM estimates should be for the best players, but let's suppose for the sake of the argument that you're right. Suppose that some all-knowing being told us that the average APM for the top 10 players in the league ought to be 9 instead of what I have (which I think averages to about 5.5 for my 3-year top 10 from RAPM, although I'm just eyeballing it).

Suppose you have the results of another APM- one that doesn't use regularization- which has an average magnitude for the top 10 that's "correct", i.e., around 9. That doesn't necessarily mean that this second APM analysis is more accurate than the one with the values which are "too small". The second analysis could have the right magnitude but still be off on an individual player-by-player basis to such an extent that it's not as accurate as RAPM.

A related point is the distinction between the true parameter magnitudes and the magnitudes which can be reliably estimated given a limited amount of highly noisy data and significant evolution over time of player abilities (what statisticians sometimes call nonstationarity). If we had 50,000 games each year, it might well be that the models would converge on larger magnitude APM values. We don't, though, and my experiments suggests that given the limited, noisy data that we do have, penalizing large APM values improves accuracy.

Regarding the plausibility of the magnitudes for the top players, it's worth remembering that the number is computed relative to a minutes-weighted average APM player. When you hear "average NBA player" casually, you might tend to think of a simple average over everyone on an NBA roster, which would mean "average NBA player" might correspond roughly to a sixth or seventh man on an average team. However, since it's minutes-weighted, that means the "average NBA player" is a good bit better than that. I haven't looked at the numbers carefully, but it probably roughly corresponds to the third or fourth best player on an average team- in other words, a fairly decent player. That's why the majority of players have scores below zero.

So if the model says Chauncey Billups gets you 3 extra points in margin of victory (given that he plays 35 mpg) relative to an average NBA player, that's relative to a fairly decent player. That's about a 3 points per game boost relative to Andre Miller or Mike Bibby (just to take a couple of players for whom I get an RAPM close to 0).

Also, let's remember that 3 points a game in average margin of victory over an entire season is quite a lot. 3 points a game is the difference between Denver (+ 3.4 ppg) and Philadelphia (+ 0.1 ppg) last year. I realize that there's a Pythagorean model for translating margin of victory into expected wins, but from eyeballing it, it looks like a rough rule of thumb is that 1 ppg in margin of victory over a whole season maps to about 3 extra wins on average. So that's 9 extra wins if you replace a minutes-weighted average NBA player- which again, is not the second guy off the bench but a decent starter- with Billups.

Finally, can you clarify what you mean by this?

**jsill** · Joined: 19 Aug 2009 Posts: 73

Aaron:

**jsill** · Joined: 19 Aug 2009 Posts: 73

deepak_e:

deepak · Joined: 26 Apr 2006 Posts: 664

Thanks for this work.

I'd be very interested to see the APMs split into offensive and defensive impact. Also it would be useful to have 1-year APMs in each of the last 3 seasons. If there is good year to year consistency (for vets, in particular) compared to other APM methods, it would be good to show.

gabefarkas · Joined: 31 Dec 2004 Posts: 1311 Location: Durham, NC

schtevie · Joined: 18 Apr 2005 Posts: 400

DSMok1 · Posted: Wed Nov 04, 2009 5:31 pm Post subject:

schtevie--you're saying the same things as I was... basically, it appears that the reason everything is compressed is that all of the players are being "regressed" toward 0, which is why this is a better projection of the future/out of sample numbers. It should be far better, I think, if all players were not being regressed toward 0, but some better Bayesian prior used.

gabefarkas · Joined: 31 Dec 2004 Posts: 1311 Location: Durham, NC

gabefarkas · Joined: 31 Dec 2004 Posts: 1311 Location: Durham, NC

Crow · Joined: 20 Jan 2009 Posts: 773

DLew · Joined: 13 Nov 2006 Posts: 222

Given that a point per game in differential is worth roughly 2.5 wins, then a player who is a +8 per 100 possesions is worth about +6 per 40 minutes which is what most stars play in a game. So that's 15 wins better than average and roughly 25 wins better than replacement. To me that does not seem unreasonably low for the best player in the league and that seems to be what the numbers suggest.

One possible check might be, and I could be wrong on my theory here but it sounds right, to look at his home court advantage coefficient and see how it compares to coefficient found in other places. If the numbers are too strongly biased towards zero then perhaps his home court advantage coefficient will show this. If it has a similar magnitude as reported else where then this would suggest that little or no signal has been lost, just noise.

DSMok1 · Posted: Thu Nov 05, 2009 2:42 pm Post subject:

Crow · Joined: 20 Jan 2009 Posts: 773

Minutes alone is useful but isn't enough to guide that well on player quality for big minute players on bad teams or a lot of bench players on good teams; hence I suggested minutes somewhat adjusted by team win performance.

mtamada · Joined: 28 Jan 2005 Posts: 375

Excellent stuff. The ridge regression seems to improve things substantially. And if I understand your cross-validation correctly, you're basically using the Retrodiction technique (meaning, using the actual player minutes for the Out-Of-Sample data) -- but rather than trying to retrodict the entire 2008-09 season, you're looking at the last few months' of the 2008-09 season?

Looks like really good work. Three ideas occur to me as possible next steps.

It's too bad the the optimal cutoff was M = 1200 minutes. Maybe the Generalized Ridge Regression technique that tpryan mentioned could be useful here ... larger lambdas for players with small minutes played.

So the optimal value of D was 0.25, i.e. pretty small, and no wonder the one-year results don't differ that much from the multi-year results. Several people have mentioned the notion of adding career curves to the model, i.e. players at the peak (or more accurately plateau) of their career would have relatively large values of D, whereas young players or clearly declining old players would have small values of D -- or might even have their next-season's parameters extrapolated, rather than directly based on current and recent seasons with no attempt to predict possible changes for next season.

I also like RyanP's random effects hierachical regression ... he seemed to get an ever larger reduction in RMSE. I don't know if someone has worked out the estimator (or if it can even be done), but how about combining the two: a regularized or ridge regression of a random effects hierarchical model?

Oh a fourth thought: multicollinearity reduces our ability to get good estimates of the effects of any one predictor variable (or player), due to their high correlation. But even in a collinear situation, one can get good estimates of the impact of a combination of variables.

An intuitive example: if Stockton and Malone always started and subbed out at the same time, and never played without each other, obviously we'd never be able to tell what Stockton's APM was, distinct from Malone's, and vice-versa. But what we could do is estimate the APM of the Stockton-Malone duo.

Which fits in nicely with the notion of looking at lineups and lineup effects that has been floating around. Maybe we can't estimate whether Stockton was a better PG than Isiah Thomas, because Stockton's value is inextricably tied up with Malone's -- but if Jerry Sloan never plays Stockton without Malone anyway, that's not such a big issue, from the perspective of team evaluation, predicting won-loss records, etc.

That fourth thought was where my thoughts had been drifting the last several weeks, but RyanP's random effects model and JoeS's regularized regression results make me think there could still be new and useful stuff to extract from the good ol' Adjust Plus-Minus model, suitably upgraded with more sophisticated estimation techniques.

Crow · Joined: 20 Jan 2009 Posts: 773

Another idea could be to run "1 on 1 games" between Mr. Laker Lineup Factors playing against the counterparts of Mr. Celtic Lineup Factors and all the other match-ups. That would get Adjusted Lineup Factors. Maybe that would help predict the future even better?

You could go with the straight 4 Factors on offense and defense but I might divide it up as 3 pt shooting, non-3 point shooting, free throw shooting, turnovers and rebounding to get 5 to match the 5 when you use players for the heck of it and to add a bit more detailed breakout of scoring activity. Run the regression just looking at the play by play for a single of these factors at a time. Turnovers and offensive rebound findings would be converted into points based on regression findings or value of possession.

This would help illuminate which Factor is influencing a lineup most toward winning or losing their match-ups. And give you better measurements / more obvious clues about what you might try to fix about them than the raw data itself since the quality of lineup match-up and quality of Factor matchup among lineups will probably vary a lot and make surface comparisons often misleading.