 |
APBRmetrics
The statistical revolution will not be televised.
|
| View previous topic :: View next topic |
| Author |
Message |
Dan Rosenbaum
Joined: 03 Jan 2005
Posts: 541
Location: Greensboro, North Carolina
|
 Posted: Thu Jul 28, 2005 3:15 pm Post subject: Defensive Adjusted Plus/Minus Ratings |
 |
|
On my blog I have posted a few of the results from my latest adjusted plus/minus ratings analysis.
http://danrosenbaum.blogspot.com/2005/07/defense-for-big-men-what-do-adjusted.html
| Quote: | Defense for the big men: what do the adjusted plus/minus ratings say?
By Dan T. Rosenbaum
If I had a dime for every time I heard that you can't measure defense with stats, I would be a rich man. (Well maybe not rich, but I might have enough money for a nice dinner.)
Steals, blocks, and defensive rebounds - they give us only a snapshot of what a player does on defense. We would like to have more and better data to measure defense. One direction is to collect better defensive statistics, an effort that is being spearheaded by Roland Beech at 82games.com.
Another approach is to use plus/minus statistics to measure how a team defends when a player is in the game versus when he is not. It would seem odd to say that a player was a good defender when his team defended better when he was on the bench than when he was in the game.
Now, of course, it is important to account for who a player is playing with and against. Playing beside Ben Wallace might even make me appear to be a good defender. For that reason I compute adjusted plus/minus ratings that account for who a player is playing with and against. These adjusted plus/minus ratings can then be broken down into their offensive and defensive components.
In "Measuring How NBA Players Help their Teams Win" I describe the gory details of how I compute these adjusted plus/minus ratings. (I have made a few changes since then, along with adding another year of data.) It takes a lot of data for adjusted plus/minus ratings to tell us anything useful and for that reason it is useful to ask another question. What is the average adjusted plus/minus rating of players similar to a given player? Answering this question can give me a second estimate of a players' defensive productivity and help combat errors from adjusted plus/minus ratings due to lack of data.
So combining ratings of defense from a players' own adjusted plus/minus rating and that of players similar to him, which players are the best defenders? I list the best and worst by position among players playing 1,000 or more minutes in 2004-05. These ratings are predictions for the 2005-06 season assuming that younger players will improve their defense and older players may see a decline in their defense. |
I apologize for not providing more details about this (including the full lists), but I want to let a situation play out before I give away the store. But I can talk more about the methodology and respond to questions and complaints. |
|
| Back to top |
|
 |
HoopStudies
Joined: 30 Dec 2004
Posts: 705
Location: Near Philadelphia, PA
|
| Posted: Thu Jul 28, 2005 4:03 pm Post subject: Re: Defensive Adjusted Plus/Minus Ratings |
|
|
| Dan Rosenbaum wrote: | On my blog I have posted a few of the results from my latest adjusted plus/minus ratings analysis.
http://danrosenbaum.blogspot.com/2005/07/defense-for-big-men-what-do-adjusted.html
| Quote: | ....
So combining ratings of defense from a players' own adjusted plus/minus rating and that of players similar to him, which players are the best defenders? I list the best and worst by position among players playing 1,000 or more minutes in 2004-05. These ratings are predictions for the 2005-06 season assuming that younger players will improve their defense and older players may see a decline in their defense. |
I apologize for not providing more details about this (including the full lists), but I want to let a situation play out before I give away the store. But I can talk more about the methodology and respond to questions and complaints. |
I like that you did what I highlighted in bold. In my proactive job, I feel like forecasting is more valuable than explaining the past. At the same time, however, there are always the questions of "How did he adjust for it?" For now, I'm glad that you did it. It's a mentality that should be emphasized. This is a business and forecasts are important in business, not debating whether Jordan was better than Magic (though that's fun).
A couple things.
- You should give some rough sense of the magnitude of difference between the best and worst. Is it worth 10 ppg or 1 ppg?
- When you say that Nick Collison is uncertain because of one year of data, you should say the same thing for Matt Bonner.
- You might want to say that the results are based upon as many as 3 years (is it 3?) of data for the players when you state that they are projections.
- What is your general confidence interval on these guys? Could #1 also be #5? Could #1 be #30?
- What is the relative magnitude above average for the best D guys? Is it more or less or about the same as the relative magnitude above average for the best O guys? (related to the first point, but I'm thinking off the cuff)
- It's a little weird that so many Blazer fans tuned in. But as I look at it, it is weird to have the disparity in Blazer #s. Ratliff and Przybilla near the top. Abdur-Rahim and Randolph near the bottom.
_________________
Dean Oliver
Author, Basketball on Paper
The postings are my own & don't necess represent positions, strategies or opinions of employers. |
|
| Back to top |
|
|
kbche
Joined: 19 Jul 2005
Posts: 51
Location: washington d.c.
|
| Posted: Thu Jul 28, 2005 6:46 pm Post subject: Defensive Player Ratings |
|
|
Hi Dan,
How do you explain the absence of Shaq? |
|
| Back to top |
|
|
Eli W
Joined: 01 Feb 2005
Posts: 402
|
| Posted: Thu Jul 28, 2005 9:49 pm Post subject: |
|
|
I don't really understand the theory behind combining adjusted plus/minus with statistical plus/minus. I see SPM as a way of estimating APM from boxscore stats. Why would you want to add an imperfect estimate into your ultimate ranking?
What does it mean if a player has a higher APM than SPM? That he does positive things that the boxscore doesn't capture? If so, then adding in SPM would seem to unfairly penalize this player.
What does it mean if a player has a higher SPM than APM? Do certain players consistently have higher SPM than APM (or vice versa)?
When you have lots of data to work from (several years), will SPM be unneccesary? |
|
| Back to top |
|
|
Dan Rosenbaum
Joined: 03 Jan 2005
Posts: 541
Location: Greensboro, North Carolina
|
| Posted: Thu Jul 28, 2005 10:24 pm Post subject: |
|
|
| John Quincy wrote: | I don't really understand the theory behind combining adjusted plus/minus with statistical plus/minus. I see SPM as a way of estimating APM from boxscore stats. Why would you want to add an imperfect estimate into your ultimate ranking?
What does it mean if a player has a higher APM than SPM? That he does positive things that the boxscore doesn't capture? If so, then adding in SPM would seem to unfairly penalize this player.
What does it mean if a player has a higher SPM than APM? Do certain players consistently have higher SPM than APM (or vice versa)?
When you have lots of data to work from (several years), will SPM be unneccesary? |
Thank you very much for this question. I get it a lot, and I don't think I have ever answered it that well. I will give it another try.
In terms of the issues, you have hit the nail on the head. The problem is that plus/minus ratings are noisy and they bounce around a lot from game to game or month to month or even season to season. This is true of regular plus/minus ratings and adjusted plus/minus ratings.
This is a particularly big issue with players that tend to play together a lot. Detroit's starters play a lot of minutes together, so statistically it is hard to break apart the individual ratings. For the Bulls Ben Gordon and Tyson Chandler played together a lot. During the few times when Gordon was in but Tyson was not, the Bulls usually played very good defense. For that reason Ben Gordon is going to get most of the credit for the good defense during the times when they were playing together, even if it was Chandler who was responsible for the good defense. It may be the case that Gordon truly is an elite defender, but it also possible that he just got lucky in the small sample of observations without Chandler.
This is where the statistical plus/minus ratings can come in. The issue here is that we just don't have a large enough and varied enough sample of games for Chandler and Gordon. So what can we do to supplement that data? We can look for players like Chandler and Gordon and see what their adjusted plus/minus ratings look like. If we see that the adjusted plus/minus and statistical plus/minus ratings give us roughly the same result, we can be pretty confident in our results. If not, we need to ask more questions. Does Ben Gordon do a lot of good defensive things that are not picked up in the box score? Could he have just gotten lucky?
In a lot of cases the statistical plus/minus ratings might be a more accurate predictor of future defensive performance. If all I wanted to do was characterize past defensive performance, then adjusted plus/minus is what I want. But I want to use these data to tell me which players are going to be good or bad defenders in the future. For that a combination of the adjusted and statistical plus/minus ratings will do a better job.
For the players that made 1st and 2nd All-Defensive Teams, here are the average adjusted and statistical plus/minus rating percentiles.
1st Team
Adjusted (82), Statistical (91)
2nd Team
Adjusted (58), Statistical (77)
The fact that the statistical percentiles are on average higher could be because the coaches who vote for these teams are too overly concerned with stats like steals and blocks. Or it could mean that the adjusted plus/minus ratings are pretty noisy and the statistical plus/minus ratings do a better job dealing with that noise.
I do not think more years of data will make the statistical plus/minus ratings suprefluous. It will help in getting more precise estimates of adjusted plus/minus ratings, but players change over time and some players aren't in the league that long, so there are limits to how much this will help.
One other point that I would like to make is that I do not just use defensive stats in my statistical plus/minus ratings. I use the offensive stats as well. Some of them are not very important, but assists often are pretty important. It appears that especially among big guys, players who can pass tend to be better defenders - perhaps because they tend to do a better job on help defense.
The point here is not that assists have any significant direct effect on defense, but I am using them to identify a particular type of player. And what the statistical adjusted plus/minus ratings are measuring are the average adjusted plus/minus ratings of various types of players. This results in players like Bruce Bowen and Tayshaun Prince, players without gaudy defensive stats, having higher statistical plus/minus ratings than adjusted plus/minus ratings.
Now if I found that a player consistently had a higher statistical plus/minus rating than adjusted plus/minus rating, then I probably would start to attribute that to the player being overrated by the statistical plus/minus rating. But the results are rarely that clean as the adjusted plus/minus ratings tend to bounce around quite a bit. |
|
| Back to top |
|
|
Kevin Pelton
Site Admin
Joined: 30 Dec 2004
Posts: 979
Location: Seattle
|
| Posted: Thu Jul 28, 2005 11:32 pm Post subject: |
|
|
I find Jarron Collins' consistent adjusted plus-minus performance to be utterly fascinating. Hollinger called him "extremely limited" last year based on his individual statistics, and I frankly don't find him all that impressive when watching. Yet without the benefit of blocking many shots, he rates as a better defender than many big-time shot blockers.
Here is where I think the data Roland plans to collect will be valuable -- not so much for determining which players are good, but explaining discrepancies in the data. Watching Collins play a lot would probably do the same, but this way is a lot more efficient for me. |
|
| Back to top |
|
|
Dan Rosenbaum
Joined: 03 Jan 2005
Posts: 541
Location: Greensboro, North Carolina
|
| Posted: Fri Jul 29, 2005 3:14 am Post subject: Re: Defensive Adjusted Plus/Minus Ratings |
|
|
| HoopStudies wrote: | | - You should give some rough sense of the magnitude of difference between the best and worst. Is it worth 10 ppg or 1 ppg? |
It is about 6.6 points per 40 minutes for the centers and 6.7 points per 40 minutes for the power forwards.
| Quote: | - When you say that Nick Collison is uncertain because of one year of data, you should say the same thing for Matt Bonner.
- You might want to say that the results are based upon as many as 3 years (is it 3?) of data for the players when you state that they are projections. |
Yes and yes.
| Quote: | | - What is your general confidence interval on these guys? Could #1 also be #5? Could #1 be #30? |
These estimates are typically not precise enough to distinguish #1 and #5, but are precise enough to reasonably distinguish #1 and #30.
| Quote: | | - What is the relative magnitude above average for the best D guys? Is it more or less or about the same as the relative magnitude above average for the best O guys? (related to the first point, but I'm thinking off the cuff) |
The variances for defensive and offensive ratings appear to be about the same.
| Quote: | | - It's a little weird that so many Blazer fans tuned in. But as I look at it, it is weird to have the disparity in Blazer #s. Ratliff and Przybilla near the top. Abdur-Rahim and Randolph near the bottom. |
Miles is also going to end up towards the top and Stoudamire and Van Exel are near the bottom. That is one weird team. |
|
| Back to top |
|
|
Dan Rosenbaum
Joined: 03 Jan 2005
Posts: 541
Location: Greensboro, North Carolina
|
| Posted: Fri Jul 29, 2005 3:18 am Post subject: |
|
|
Shaq is the top center overall and he is a good but not great defensive center. He struggles with the pick and roll and getting up and down the court.
It is Jason Collins of the Nets and not Jarron Collins of the Jazz who is the good defender. Jarron rates as a little bit below average defending center. |
|
| Back to top |
|
|
Kevin Pelton
Site Admin
Joined: 30 Dec 2004
Posts: 979
Location: Seattle
|
| Posted: Fri Jul 29, 2005 9:33 am Post subject: |
|
|
| Uh-oh, I've been watching too much WNBA if I'm getting my Collinses mixed up. |
|
| Back to top |
|
|
Dan Rosenbaum
Joined: 03 Jan 2005
Posts: 541
Location: Greensboro, North Carolina
|
| Posted: Fri Jul 29, 2005 12:42 pm Post subject: |
|
|
I had some time to look into how "noisy" these adjusted plus/minus ratings are.
Using players who played 1,000 or more minutes in both of the last two seasons, here is the distribution of the absolute value of the difference in these ratings in the last two seasons.
Overall Ratings:
25th Percentile: 0.57 points per 40 minutes
50th Percentile: 1.44 points per 40 minutes
75th Percentile: 2.88 points per 40 minutes
90th Percentile: 4.43 points per 40 minutes
Offensive Ratings:
25th Percentile: 0.53 points per 40 minutes
50th Percentile: 1.23 points per 40 minutes
75th Percentile: 2.09 points per 40 minutes
90th Percentile: 3.09 points per 40 minutes
Defensive Ratings:
25th Percentile: 0.54 points per 40 minutes
50th Percentile: 1.14 points per 40 minutes
75th Percentile: 1.71 points per 40 minutes
90th Percentile: 2.55 points per 40 minutes
These ratings are more stable than I have often given them credit for. Remember not all of this above is "noise." Some of it is the natural progression and digression of players' skills as they age and gain experience.
With these results I am reconsidering putting more weight on the adjusted plus/minus ratings in my combined rating. |
|
| Back to top |
|
|
JasonNapora
Joined: 06 Jan 2005
Posts: 6
|
| Posted: Fri Jul 29, 2005 1:53 pm Post subject: |
|
|
Dan: Great stuff. Any chance you'll be releasing your complete lists?
Also, I think it makes a lot of sense that guys that pass well are better on defense (especially help defense), since both skills are subsets of the same core skill - court awareness. It's certainly something I'd never thought of before, though, which is one of the great things about this community... |
|
| Back to top |
|
|
HoopStudies
Joined: 30 Dec 2004
Posts: 705
Location: Near Philadelphia, PA
|
| Posted: Fri Jul 29, 2005 2:05 pm Post subject: |
|
|
| Dan Rosenbaum wrote: | I had some time to look into how "noisy" these adjusted plus/minus ratings are.
Using players who played 1,000 or more minutes in both of the last two seasons, here is the distribution of the absolute value of the difference in these ratings in the last two seasons.
|
To clarify, these are differences from one season to the next?
_________________
Dean Oliver
Author, Basketball on Paper
The postings are my own & don't necess represent positions, strategies or opinions of employers. |
|
| Back to top |
|
|
Dan Rosenbaum
Joined: 03 Jan 2005
Posts: 541
Location: Greensboro, North Carolina
|
| Posted: Fri Jul 29, 2005 2:27 pm Post subject: |
|
|
| HoopStudies wrote: | | Dan Rosenbaum wrote: | I had some time to look into how "noisy" these adjusted plus/minus ratings are.
Using players who played 1,000 or more minutes in both of the last two seasons, here is the distribution of the absolute value of the difference in these ratings in the last two seasons.
|
To clarify, these are differences from one season to the next? |
Yes. |
|
| Back to top |
|
|
Dan Rosenbaum
Joined: 03 Jan 2005
Posts: 541
Location: Greensboro, North Carolina
|
| Posted: Fri Jul 29, 2005 4:08 pm Post subject: |
|
|
| JasonNapora wrote: | Dan: Great stuff. Any chance you'll be releasing your complete lists?
Also, I think it makes a lot of sense that guys that pass well are better on defense (especially help defense), since both skills are subsets of the same core skill - court awareness. It's certainly something I'd never thought of before, though, which is one of the great things about this community... |
I need to let a situation play out before I decide how I am going to put more information out there.
Thinking about stats as an indicator of a skill set is a big departure from the accounting-type approach that thinks about stats as a contribution. Tendex, PER, DeanO's offensive efficiency, and Bob's simulation method all work from a base assumption that what we need to do is account for the value of an assist, rebound, steal, made shot, etc. and work from there. This methodology is rooted in what our baseball brethren have done for years and so it is a fundamental aspect of our methodologies.
But that is why what I am suggesting is a big change. Basketball is not like baseball. Lots of key contributions are not captured in our statistics, so what we want to know when evaluating players is not the average value of an assist, but the average value of an assister. If assisters tend to be horrible defenders (they aren't), it is possible that players with more assists would be less valuable than players with fewer assists (after accounting for the other statistics).
That may seem wrong since assists surely have value, but that is not the point. The point is we want to measure the value of the assist plus everything else that differs between players with different numbers of assists. And that is not a theoretical question; it is an empirical question that can only be estimated using data. Also, these relationships could change over time, so the value of an assister may change over time.
I would love to hear what folks have to say about this. |
|
| Back to top |
|
|
Kevin Pelton
Site Admin
Joined: 30 Dec 2004
Posts: 979
Location: Seattle
|
| Posted: Fri Jul 29, 2005 4:37 pm Post subject: |
|
|
I think that line of thinking connects well with our discussion of valuing statistics in another recent thread.
To quote myself (I know, how asinine):
| Quote: | | he unfortunate thing about the Moneyball comparison is that isolating these effects in basketball is levels of magnitude more difficult. Obviously you want to balance, say, guys who get on base versus power hitters, but it's not like having too many power hitters would actually cause them to play worse, whereas that's a possibility in the NBA. |
You can't add five guys' PERs and really find out that much about how they'll fare together because that's just not how basketball works. There's way too much interplay, and that's where DanVAL comes in handy. Ultimately, when the sample size becomes larger, I think the analysis can extend to similar five-man lineups as well as similar players.
The other factor is there are some skills (shot creation, for one) that can't really be measured except in the context of team performance.
I'm forgetting the most difficult skill to value by traditional statistics, which is passing. PER, Dean's work ... all just guesses when it comes to assists. Logical ones perhaps, but guesses nonetheless.
I guess that's a long way of saying I agree. |
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
Powered by phpBB © 2001, 2005 phpBB Group
|
FAQ
Search
Memberlist
Usergroups
Register
Profile
Log in to check your private messages
Log in
