
APBRmetrics The statistical revolution will not be televised.

View previous topic :: View next topic 
Author 
Message 
DSMok1
Joined: 05 Aug 2009 Posts: 611 Location: Where the wind comes sweeping down the plains

Posted: Wed Jul 14, 2010 4:39 pm Post subject: Advanced Statistical Plus/Minus 


Advanced Statistical Plus/Minus
I've been working on deriving a new SPM regression based purely upon "advanced" stats (like TS% and OR%) for some time now. I feel comfortable enough with the results thus far to release the first iteration of this SPM.
The data used: Neil Paine's collection of 1Yr APM's (unfortunately without std err's; I estimated the standard errors for weighting purposes), Joe Sill's 4 Year RAPM's, with the regression toward 0 backed out, and finally (and most importantly) Steve Ilardi's 6Year APM's posted on this forum. These 6 Year APM's had quite low errors, and provided the groundwork for this regression. I weighted each player in the regression by 1/stderr^2, where stderr is their APM standard error.
I then compiled the advanced metrics from the Basketball Reference Play Index for each player, and weightedaveraged the multiyear data (including playoffs, for the APM's that included those). Thus I have 3 APM data sets and the associated advanced statistics.
I experimented with a number of constructions for the rebounding and especially the scoring parts of this regression. Finding a good way to relate turnovers, shooting, usage, and assists proved illusive for some time. I finally now have a construction I am comfortable with, though (like with any construction) there are a few holes.
To avoid overweighting steals and blocks for defense, I also included offensive rating and defensive rating of the teams. This is not included in the final SPM, because the team adjustment (to make the teams sum to their efficiency differentials) already accounts for this.
Here are the factors in this regression:
Code:  Factor Value
TRB% 1.33823090
TRB^2 0.08918572
TRB^3 0.00219790
STL% 1.43951052
BLK% 0.35237880
MPG 0.10099403
TO% Coeff 0.66920540
PPP Threshold 1.64758151
PPP USG Scale 0.01394727
PPP AST Scale 0.01005596
Scoring 0.55728095
USG Const 4.67604494
Intercept 6.90680060 
Let me explain.
First of all, note the rebounding terms. I discovered that the value of splitting rebounding into offensive and defensive was much less than that of adding this nonlinearity (which didn't work when ORB and DRB were split). Basically, in the neighborhood of 10%, there isn't a huge amount of change. A player that gets very few rebound hurts the team a lot, and a player near 20% rebounds helps quite a bit. Here's a quick table:
Code:  TRB% Pts
0 0.00
2.5 2.82
5 4.74
7.5 5.95
10 6.66
12.5 7.09
15 7.42
17.5 7.89
20 8.67
22.5 10.00
25 12.06 
Next: steals, blocks and MPG. These are all straightforward, linear terms. Be aware, though: I'm inputing these percentages throughout in their wholenumber forms, like BasketballReference outputs them.
Charges taken would be added into the steals termother research I've done shows them to be equivalent in SPM terms (1 ChgTkn = 1 Steal). I'm trying to make this SPM able to be applied historically; thus I've left that out.
Here's the complicated part: the scoring term. First the actual formula:
Code:  {TS%*2*(1TO%/100)  TO%Coeff*(TO%/100)  (PPPThreshold  PPPUSGScale*USG%  PPPASTScale*AST%)}*(USG% + USGConst)*Scoring 
What's going on here? First of all, this is basically an efficiency*USG term. It takes into account TS%, USG%, TO%, and AST% to create a composite scoring value.
Now, term by term. The True Shooting term is very basic. It gives the number of points scored per possession used by the player. Next, the turnover term provides the penalty for each turnover. These terms make up the efficiency side of the equation.
Next, the PPP (Points per Possession) threshold and modifiers. The threshold is just a baseline constant. Then usage is subtracted out, indicating from the regression that there is a clear benefit to having a higher usagein fact, .1 PPP per 7 %USG increased. Finally, the assist modifier. This is the ONLY place in the regression that has assists included. It was not significant anywhere else I tried it, compared to this location in the regression. Assists also modify the PPP; when everything is multiplied through the assists basically go to the form AST%*(USG%+Constant), which is a reasonable construction.
Finally, the whole (PPP  PPPThreshold) term is multiplied by (USG% + USGConst). Again, we're using whole percentages, everywhere but with TS% (I'm following Basketball Reference on this). Because of the USGConst, even if a player has NO usage, he still gets some credit for assists. Just not very much. In other words, Steve Blake just isn't that great.
Finally, after compiling the RAW SPM, the team adjustment must be applied. This can range from negligible (Cleveland, Boston, and Utah had 0 team adjustments this year) to quite large (+1.36 for ORL, 1.43 for GSW). Mostly defense is what is accounted for by the team adjustment since it is not captured well by the regression.
Here is a sample of the resultsthe top 20 in SPM, minimum 1000 minutes:
Code:  Rnk Tm Player G MP SPM
1 CLE LeBron James 76 2966 12.16
2 MIA Dwyane Wade 77 2792 9.69
3 NOH Chris Paul 45 1712 6.51
4 ORL Dwight Howard 82 2843 6.31
5 SAS Manu Ginobili 75 2150 5.57
6 LAL Kobe Bryant 73 2835 5.38
7 OKC Kevin Durant 82 3239 5.32
8 SAS Tim Duncan 78 2438 5.21
9 BOS Rajon Rondo 81 2963 4.82
10 LAC Marcus Camby 51 1596 4.81
11 UTA Deron Williams 76 2802 4.42
12 ATL Josh Smith 81 2871 4.32
13 DAL Dirk Nowitzki 81 3039 4.07
14 LAL Pau Gasol 65 2403 4.01
15 UTA Carlos Boozer 78 2673 3.99
16 WAS Gilbert Arenas 32 1169 3.90
17 DEN Nene Hilario 82 2755 3.77
18 TOR Chris Bosh 70 2526 3.62
19 CHA Gerald Wallace 76 3119 3.53
20 DEN Carmelo Anthony 69 2634 3.51 
The full results for 20092010 regular season are here: Google Spreadsheet: Advanced SPM 0910
EDIT: See later in this thread for revisions to this method and a complete spreadsheet to play with.
Last edited by DSMok1 on Tue Oct 26, 2010 12:11 pm; edited 1 time in total 

Back to top 


Ilardi
Joined: 15 May 2008 Posts: 265 Location: Lawrence, KS

Posted: Wed Jul 14, 2010 4:55 pm Post subject: 


Nice work, DSM: this looks like an important contribution.
A couple of quick questions:
a) Can you provide standard error (se) estimates for the SPM values?
b) Did you consider using any of the advanced metrics from 82games? I've always thought eFG% Allowed would be quite useful in an SPM model . . .
c) What is the correlation between your SPM values for each player and his corresponding APM value? (i.e., the zeroorder correlation for the entire league)
d) Any plans for "outofsample testing" on this new SPM metric (a la Joe Sill)? 

Back to top 


DSMok1
Joined: 05 Aug 2009 Posts: 611 Location: Where the wind comes sweeping down the plains

Posted: Wed Jul 14, 2010 5:38 pm Post subject: 


Ilardi wrote:  Nice work, DSM: this looks like an important contribution.
A couple of quick questions:
a) Can you provide standard error (se) estimates for the SPM values?
b) Did you consider using any of the advanced metrics from 82games? I've always thought eFG% Allowed would be quite useful in an SPM model . . .
c) What is the correlation between your SPM values for each player and his corresponding APM value? (i.e., the zeroorder correlation for the entire league)
d) Any plans for "outofsample testing" on this new SPM metric (a la Joe Sill)? 
Good to see you around, Ilardi!
a) How would I go about developing them for a nonlinear model? I would love to, but haven't figured out how. Another issue with the standard errors is that the APM against which we are regressing has error within it (which I think biases the error on the regression upwards).
b) I wanted to make this metric as useful historically as possible. Basketball Reference has all of the stats used in this regression back to 1977. A more intricate SPM is possible, using things like eFG% allowed, location of assists, etc.
c) I can run that... should I do it just on the lowerror six season sample?
d) That would be tough for me to do. I don't have a lot of samples to work with.
Thanks for the input! 

Back to top 


Ilardi
Joined: 15 May 2008 Posts: 265 Location: Lawrence, KS

Posted: Wed Jul 14, 2010 6:01 pm Post subject: 


Thanks: and most guys on the forum call me 'Steve'.
I'd have to get a consult to figure out how to calculate se's on a nonlinear metric like that, but I know it must be doable. Perhaps someone on this forum can point the way to a workable approach?
As for the correlation between SPM and APM, I might suggest using the 0809 season, for which you have my 6season estimates (weighted heavily toward 0809), as well as your own SPM values.
On the outofsample test: presumably it would be possible to calculate SPM values for each player based on games through, say, the first 4 months of last season, and then use those estimates to predict results of the final 2 months. (Same basic approach Joe used with his ridge regression APM numbers.) It would be a fair amount of work, but should be easily doable, at least in principle. 

Back to top 


DSMok1
Joined: 05 Aug 2009 Posts: 611 Location: Where the wind comes sweeping down the plains

Posted: Wed Jul 14, 2010 11:47 pm Post subject: 


Ilardi wrote:  Thanks: and most guys on the forum call me 'Steve'.
I'd have to get a consult to figure out how to calculate se's on a nonlinear metric like that, but I know it must be doable. Perhaps someone on this forum can point the way to a workable approach?
As for the correlation between SPM and APM, I might suggest using the 0809 season, for which you have my 6season estimates (weighted heavily toward 0809), as well as your own SPM values.
On the outofsample test: presumably it would be possible to calculate SPM values for each player based on games through, say, the first 4 months of last season, and then use those estimates to predict results of the final 2 months. (Same basic approach Joe used with his ridge regression APM numbers.) It would be a fair amount of work, but should be easily doable, at least in principle. 
I'd love to figure out how to do standard errors on nonlinear metrics.
I'll look into the correlation for the data you suggested, when I have time.
I still have issues with the out of sample test, because it is replacing a descriptive stat with a predictive statwhich is why the ridge regression technique provided the best outofsample results. It's basically regression to the mean. When I do regression, I'm going to use the samples, with their error, and regress in a Bayesian manner toward a prior based on peripheral data. 

Back to top 


Ilardi
Joined: 15 May 2008 Posts: 265 Location: Lawrence, KS

Posted: Thu Jul 15, 2010 12:18 pm Post subject: 


DSMok1 wrote:  I still have issues with the out of sample test, because it is replacing a descriptive stat with a predictive statwhich is why the ridge regression technique provided the best outofsample results. It's basically regression to the mean. When I do regression, I'm going to use the samples, with their error, and regress in a Bayesian manner toward a prior based on peripheral data. 
But isn't the utility of any metric linked in large part to its predictive ability? Certainly, in the natural sciences, the valid prediction of phenomena is regarded as the sine qua non of the entire enterprise, so I'm admittedly a bit biased, but suffice it to say that even NBA decision makers realize that it's much more valuable to have a stat that gives accurate prediction than one that merely provides accurate description.
Also, although ridge regression makes use of 'regression to the mean', it does so in a limited way  essentially by simply reining in outlier values via an a priori (Bayesian) determination that they are unlikely. In my view, it's an extremely clever technique for enhancing the 'signal' of player APM values via tamping down the 'noise' of extreme variations in efficiency from one lowminute lineup to another. 

Back to top 


DSMok1
Joined: 05 Aug 2009 Posts: 611 Location: Where the wind comes sweeping down the plains

Posted: Thu Jul 15, 2010 1:00 pm Post subject: 


Ilardi wrote:  DSMok1 wrote:  I still have issues with the out of sample test, because it is replacing a descriptive stat with a predictive statwhich is why the ridge regression technique provided the best outofsample results. It's basically regression to the mean. When I do regression, I'm going to use the samples, with their error, and regress in a Bayesian manner toward a prior based on peripheral data. 
But isn't the utility of any metric linked in large part to its predictive ability? Certainly, in the natural sciences, the valid prediction of phenomena is regarded as the sine qua non of the entire enterprise, so I'm admittedly a bit biased, but suffice it to say that even NBA decision makers realize that it's much more valuable to have a stat that gives accurate prediction than one that merely provides accurate description.
Also, although ridge regression makes use of 'regression to the mean', it does so in a limited way  essentially by simply reining in outlier values via an a priori (Bayesian) determination that they are unlikely. In my view, it's an extremely clever technique for enhancing the 'signal' of player APM values via tamping down the 'noise' of extreme variations in efficiency from one lowminute lineup to another. 
I'm not disputing the value of prediction. However, I'd like to do that AFTER the SPM is calculated. In other words, construct a SPM, THEN apply the Bayesian regression to estimate "true talent", then combine with previous years to create a projection. I simply want the SPM itself to not be "biased" with information outside of actual production numbers.
I agree that RAPM works very well, but it does have a few quirks. Like Anderson Varajao getting very highly rated because it is so unlikely that Lebron is really a +11 player. 

Back to top 


Ilardi
Joined: 15 May 2008 Posts: 265 Location: Lawrence, KS

Posted: Sat Jul 17, 2010 10:40 am Post subject: 


I've also had Varajao rated highly using a more traditional APM approach . . . 

Back to top 


DSMok1
Joined: 05 Aug 2009 Posts: 611 Location: Where the wind comes sweeping down the plains

Posted: Sat Jul 17, 2010 12:27 pm Post subject: 


Ilardi wrote:  I've also had Varajao rated highly using a more traditional APM approach . . . 
15th is pretty high. That's what the 4year RAPM had him. Don't you think there is possibility of using the Bayesian in such a way causing some odd effects like that?
Alsowould it be possible to get from you a 4 year, regular season only APM, though 0910? Then I could use the advanced stats collected by Hoopdata in that time to run a more comprehensive SPM. _________________ GodismyJudgeOK.com/DStats
Twitter.com/DSMok1 

Back to top 


Neil Paine
Joined: 13 Oct 2005 Posts: 774 Location: Atlanta, GA

Posted: Sat Jul 17, 2010 2:50 pm Post subject: 


Great work, DSMok!! I'm trying to replicate your work, and I had a question: how are you doing the team adjustment? What I always did was to find the minuteweighted average of each team's SPM and multiply by 5, then subtract that from the team's actual efficiency differential and divide the result by 5. But when I do that, my team adjustments don't match yours (ORL is +1.26, GSW is 1.70). Is it a rounding issue (I'm using the full, calculated versions of the BBR stats, while you used rounded versions), or is my team adjustment method incorrect? _________________ http://www.basketballreference.com/blog/ 

Back to top 


DSMok1
Joined: 05 Aug 2009 Posts: 611 Location: Where the wind comes sweeping down the plains

Posted: Sat Jul 17, 2010 4:02 pm Post subject: 


Neil Paine wrote:  Great work, DSMok!! I'm trying to replicate your work, and I had a question: how are you doing the team adjustment? What I always did was to find the minuteweighted average of each team's SPM and multiply by 5, then subtract that from the team's actual efficiency differential and divide the result by 5. But when I do that, my team adjustments don't match yours (ORL is +1.26, GSW is 1.70). Is it a rounding issue (I'm using the full, calculated versions of the BBR stats, while you used rounded versions), or is my team adjustment method incorrect? 
I didn't use the team efficiency precisely. I summed to 2/3 SRS 1/3 Efficiency differential. Is the SRS calculated from efficiency differentials or point margins? If it is calculated off of efficiency differentials per game, it should be the best thing to sum to. I think it's point differential, which is why I used the average I did. But whatever you choose to sum to, that's up to you.
I'm glad you're doing this! You've got all of the data for compiling a full list and actually doing the team adjustments correctly.
I'm hoping this doesn't undervalue great centersbecause there weren't any in the time period I used for the regression, I don't know if the top end of the regression can capture them. Then again, I don't know how much a great center truly contributed, either. _________________ GodismyJudgeOK.com/DStats
Twitter.com/DSMok1 

Back to top 


Ilardi
Joined: 15 May 2008 Posts: 265 Location: Lawrence, KS

Posted: Sat Jul 17, 2010 4:59 pm Post subject: 


[quote="DSMok1"] Ilardi wrote: 
Alsowould it be possible to get from you a 4 year, regular season only APM, though 0910? Then I could use the advanced stats collected by Hoopdata in that time to run a more comprehensive SPM. 
I haven't run it yet, but maybe your request will be just the catalyst I need. Is it really the case that no one else out there has put out any publicly available multiyear APM stats? Are Aaron's 2year APM stats on basketballvalue.com all there is? If that's the case, I really will try to carve out the time to work on this . . . 

Back to top 


DSMok1
Joined: 05 Aug 2009 Posts: 611 Location: Where the wind comes sweeping down the plains

Posted: Sat Jul 17, 2010 5:28 pm Post subject: 


Ilardi wrote:  DSMok1 wrote: 
Alsowould it be possible to get from you a 4 year, regular season only APM, though 0910? Then I could use the advanced stats collected by Hoopdata in that time to run a more comprehensive SPM. 
I haven't run it yet, but maybe your request will be just the catalyst I need. Is it really the case that no one else out there has put out any publicly available multiyear APM stats? Are Aaron's 2year APM stats on basketballvalue.com all there is? If that's the case, I really will try to carve out the time to work on this . . . 
I don't know of any other APM's out there, now that the RAPM was taken down. The one I could use would be an "average" APM over the last 4 years. _________________ GodismyJudgeOK.com/DStats
Twitter.com/DSMok1 

Back to top 


DSMok1
Joined: 05 Aug 2009 Posts: 611 Location: Where the wind comes sweeping down the plains

Posted: Mon Jul 19, 2010 9:53 am Post subject: 


Neil Paine wrote:  Yeah, SRS is actually just SOSadjusted point differential per game, which means it's not tempoindependent (we do it that way because we only have game possessions going back to 198687). If we had historical SOSadjusted efficiency differential, that would definitely be the thing to sum to, but since we don't, I'm probably going to just sum to efficiency differential (which is what APM does anyway).
In case I didn't say so, like this new regression a lot! The most glaring problem with the old regression was that it drastically overvalued assists (and therefore PGs  I found that the average PG was +1 or so while every other position was near zero), but it looks like you fixed this by tying AST% to the scoring term instead of having it stand alone. I would imagine this retrodicts better than the old regression as a result.
...
One troubling result at a first glance is that Dennis Rodman's 1995 is +10.91, one of the greatest seasons of all time... Maybe the rebounding term needs to be reevaluated?
 (Posted from an email)
It looks like the rebounding term will need some more work. The cubic works for just about everyone, except Rodman. He breaks the regression. The 30% TRB% is way out into the nonlinear term, and is worth like 9 points more than DHoward's 22% TRB%.
Here are a few possibilities:
The cubic is the best fit, but only by a hair. The power + Sq is very close in terms of fit, and would probably be my pick to use. The pure power curve can't capture the desired upturn at higher TRB% rates, but in terms of fit is still very close to power+Sq (most of the difference is out where there aren't any observations). The linear is here for reference (incidentally, TRB% by itself is a BETTER fit than ORB% and DRB% split out). _________________ GodismyJudgeOK.com/DStats
Twitter.com/DSMok1 

Back to top 


Mike G
Joined: 14 Jan 2005 Posts: 3618 Location: Hendersonville, NC

Posted: Mon Jul 19, 2010 10:19 am Post subject: 


DSMok1 wrote:  ...
I'm hoping this doesn't undervalue great centers... 
This is reminiscent of a discussion starting with Nazr Mohammed (this year), that many of his rates resembled prime Moses, Gilmore, and others. Low versatility index the likely culprit.
It was asked then whether those known greats were also undervalued by the (then most current) SPM method.
If versatile lessthangreat centers (Divac, Daugherty) seem to be better than Moses or Artis, then maybe it needs to be fixed? _________________ `
36% of all statistics are wrong 

Back to top 




You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum

Powered by phpBB © 2001, 2005 phpBB Group
