|
APBRmetrics The statistical revolution will not be televised.
|
View previous topic :: View next topic |
Author |
Message |
Mountain
Joined: 13 Mar 2007 Posts: 1527
|
Posted: Wed Feb 18, 2009 3:43 pm Post subject: |
|
|
If pure +/- is the main solution I think it will be pure +/- with an array of many splits and parts.
As I've noted before Dan Rosenbaum introduced pure +/- and then seemed to immediately move to "overall +/-" with his statistical as about an 80% and pure at just 20%. I would really appreciate hearing from any the next pure adjusted stat producers if they considered following with a blended product or why they choose not to.
And at this time I am no longer a fan of a 80% statistical 20% pure split because I tend to think this ends up shorting defense again.
I wonder if something like 60% statistical / 40% pure or 50-70% statistical on offense depending on usage and therefore 30-50% pure offensive adjusted and maybe 40-50% statistical on defense and 50-60% pure defensive adjusted would be "better". A fuller blend feels more promising to me. |
|
Back to top |
|
|
Ilardi
Joined: 15 May 2008 Posts: 265 Location: Lawrence, KS
|
Posted: Wed Feb 18, 2009 3:48 pm Post subject: |
|
|
bastillon wrote: | Quote: | (The two component errors always turn out to be virtually identical for any given player, with a correlation above r=.99, presumably because a player's lineup companions on offense and defense are almost exactly the same). |
what about one-sided players like Nash(great offensively, liability defensively) or Ben Wallace(the opposite) ?
if it's about companions, D and O should be completely separated because players' skills on both sides of the court can be completely different. |
It's all about the lineups - estimation error generally results from heavily intercorrelated player minutes. Except for relatively infrequent, not-heavily-weighted one-possession lineups (where a lineup is used only for a single offensive or defensive possession), lineups tend to be identical for offensive and defensive possessions, so the multicollinearity issue for each player is roughly the same when estimating his offensive and defensive APM components. |
|
Back to top |
|
|
Ilardi
Joined: 15 May 2008 Posts: 265 Location: Lawrence, KS
|
Posted: Wed Feb 18, 2009 3:56 pm Post subject: |
|
|
Mountain wrote: | If pure +/- is the main solution I think it will be pure +/- with an array of many splits and parts.
As I've noted before Dan Rosenbaum introduced pure +/- and then seemed to immediately move to "overall +/-" with his statistical as about an 80% and pure at just 20%. I would really appreciate hearing from any the next pure adjusted stat producers if they considered following with a blended product or why they choose not to.
And at this time I am no longer a fan of a 80% statistical 20% pure split because I tend to think this ends up shorting defense again.
I wonder if something like 60% statistical / 40% pure or 50-70% statistical on offense depending on usage and therefore 30-50% pure offensive adjusted and maybe 40-50% statistical on defense and 50-60% pure defensive adjusted would be "better". A fuller blend feels more promising to me. |
Dan's original "statistical +/-" relies entirely on traditional boxscore stats, and I seem to recall that it correlates pretty strongly with offensive APM (just as we'd expect) and not at all with defensive APM.
I won't purport to speak for the others who've published APM analyses at some point (Aaron Barzilai, Dave Lewin, Eli Witus), but I have chosen to focus exclusively on "pure" APM for now because I believe the "statistical +/-" (SPM?) approach, as currently conceived, introduces unnecessary bias into the estimation process. However, as we've discussed in the past, I wouldn't rule out refining and augmenting the SPM approach in the future to make it more useful. (Knowing how innovative Dan is, he probably did all this back during the Clinton administration and just hasn't been able to make it publicly available.)
Ultimately, however, my money is still on the pure APM measure (using multi-season databases to get estimation noise down to a reasonably low level), augmented with: (a) analysis of significant player-by-player interaction effects (probably rare, but potentially invaluable in the case of outlier APM results like CP3's), and (b) year-by-year time-trend analyses that compute individualized APM growth curve trajectories for each player.
Last edited by Ilardi on Wed Feb 18, 2009 8:03 pm; edited 1 time in total |
|
Back to top |
|
|
gabefarkas
Joined: 31 Dec 2004 Posts: 1313 Location: Durham, NC
|
Posted: Wed Feb 18, 2009 5:35 pm Post subject: |
|
|
Ilardi wrote: | 2) The team's improvement in defensive efficiency with Paul on the court this season has also occurred despite a dramatic downturn in the team's defensive performance with Paul off the court. Last year, the team was better defensively when Paul was off-court (101.4) than when he was on (107.6); this year, the situation is reversed: they're much worse defensively when he's off-court (111.8 ) than when he's on (106.1). That's a swing of over 10 point per 100 possessions in Paul's teammates' off-court performance. And remember, most of these off-court teammates also share a high percentage of their minutes with Paul on-court. (We'll come back to this point in a moment). |
It's circular logic. Either the team's OffEff and DefEff have changed because CP is a much better defender (as evidenced by his APM), or Paul's APM is much better because his team's OffEff and DefEff have changed. It's one or the other; you can't have both. |
|
Back to top |
|
|
gabefarkas
Joined: 31 Dec 2004 Posts: 1313 Location: Durham, NC
|
Posted: Wed Feb 18, 2009 5:38 pm Post subject: |
|
|
schtevie wrote: | Perhaps it was the case that Paul, being the notionally better defender, was guarding the 2, and taking one for the team. 82games counterpart data says he didn't do even a second of that, but that can't be right. |
No, it definitely can't be right! Especially if it doesn't support our pre-supposed conclusions! |
|
Back to top |
|
|
Ilardi
Joined: 15 May 2008 Posts: 265 Location: Lawrence, KS
|
Posted: Wed Feb 18, 2009 5:51 pm Post subject: |
|
|
gabefarkas wrote: | Ilardi wrote: | 2) The team's improvement in defensive efficiency with Paul on the court this season has also occurred despite a dramatic downturn in the team's defensive performance with Paul off the court. Last year, the team was better defensively when Paul was off-court (101.4) than when he was on (107.6); this year, the situation is reversed: they're much worse defensively when he's off-court (111.8 ) than when he's on (106.1). That's a swing of over 10 point per 100 possessions in Paul's teammates' off-court performance. And remember, most of these off-court teammates also share a high percentage of their minutes with Paul on-court. (We'll come back to this point in a moment). |
It's circular logic. Either the team's OffEff and DefEff have changed because CP is a much better defender (as evidenced by his APM), or Paul's APM is much better because his team's OffEff and DefEff have changed. It's one or the other; you can't have both. |
I don't follow you at all. What does the team's OffEff have to do with Paul's defensive APM? (That's what I was referring to, after all. His offensive APM is almost completely unchanged from 07-08 to 08-09.)
And please recall that my contention in the passage you've quoted - admittedly somewhat subtle - is not about overall team DefEff, but with DefEff as it pertains to Paul's on-court and off-court minutes across the 07-08 and 08-09 seasons.
If you can restate your objections, I'll be happy to try to respond . . . |
|
Back to top |
|
|
Ilardi
Joined: 15 May 2008 Posts: 265 Location: Lawrence, KS
|
Posted: Wed Feb 18, 2009 5:53 pm Post subject: |
|
|
gabefarkas wrote: | schtevie wrote: | Perhaps it was the case that Paul, being the notionally better defender, was guarding the 2, and taking one for the team. 82games counterpart data says he didn't do even a second of that, but that can't be right. |
No, it definitely can't be right! Especially if it doesn't support our pre-supposed conclusions! |
Are you saying you believe Paul, who spent 40% of his 07-08 minutes playing in tandem with Pargo or Jackson, was never assigned to guard the 2-position? |
|
Back to top |
|
|
schtevie
Joined: 18 Apr 2005 Posts: 409
|
Posted: Wed Feb 18, 2009 6:43 pm Post subject: |
|
|
I think Gabe's in a mood.
What must be the case is that the counterpart positions on 82games are defined by offensive designation. No doubt that CP was the offensive point guard 100% of the time. |
|
Back to top |
|
|
Mountain
Joined: 13 Mar 2007 Posts: 1527
|
Posted: Thu Feb 19, 2009 3:06 am Post subject: |
|
|
Alright Steve. Thanks again for the thoughtful replies. |
|
Back to top |
|
|
gabefarkas
Joined: 31 Dec 2004 Posts: 1313 Location: Durham, NC
|
Posted: Thu Feb 19, 2009 6:55 am Post subject: |
|
|
Ilardi wrote: | gabefarkas wrote: | schtevie wrote: | Perhaps it was the case that Paul, being the notionally better defender, was guarding the 2, and taking one for the team. 82games counterpart data says he didn't do even a second of that, but that can't be right. |
No, it definitely can't be right! Especially if it doesn't support our pre-supposed conclusions! |
Are you saying you believe Paul, who spent 40% of his 07-08 minutes playing in tandem with Pargo or Jackson, was never assigned to guard the 2-position? |
No, I'm just saying that it seems Schtevie is doing the same thing you and he accused Bob of doing a few days ago -- splitting hairs and twisting around conclusions.
If it were me, and I found that 82games doesn't have data that supports my conclusion, I would keep looking elsewhere for other data that did, instead of just automatically assuming it's true, and stating something like "it must be true because I say so" (essentially what he wrote).
Look, I'll lay my cards out on the table. I think APM is fascinating, and I want to learn more about what goes into it. My major gripe is twofold:
(1) It's probably not as much of a panacea as its proponents seem to think it is.
(2) Calling it "plus/minus" when it's split out into two parts, offensive and defensive, seems, for lack of a better word, silly. The very notion of a "plus/minus" implies that it accounts for the net difference from both ends of the court. Once you separate it out into two components, it ceases to contain that defining property. |
|
Back to top |
|
|
schtevie
Joined: 18 Apr 2005 Posts: 409
|
Posted: Thu Feb 19, 2009 8:19 am Post subject: |
|
|
Gabe,
I did not recently accuse Bob of "splitting hairs and twisting around conclusions". In fact I didn't say anything on the matter. OK, I thought about it, but I didn't.
The fact remains that there is an interesting contrast between general perceptions of CP's defensive abilities and what APM says. It is an anomaly worth discussing. One factor explaining the gap might be the one I described. The numbers will be crunched at Steve's convenience and the relevance of the resulting estimates may or may not be confirmed by someone who could say whether in fact CP did in fact spend his time guarding 2s.
Regarding APM generally, who says it is a panacea? Is there a better single metric, in concept, that is its superior? No, and by definition there cannot be. Underline it. Can the metric be improved by including age profiles, position adjustments, etc. Absolutely, and that is the (hopefully near-term) future.
Finally, I don't understand your objection to dividing APM into Offensive and Defensive components. I just don't. |
|
Back to top |
|
|
gabefarkas
Joined: 31 Dec 2004 Posts: 1313 Location: Durham, NC
|
Posted: Thu Feb 19, 2009 9:16 am Post subject: |
|
|
I appreciate your candor. If it appears our positions are intractable, as you describe, and you just don't understand my objection, then let's put this matter to rest. For the record, I'm not an opponent of APM. I'm just in general a skeptic. |
|
Back to top |
|
|
bchaikin
Joined: 27 Jan 2005 Posts: 687 Location: cleveland, ohio
|
Posted: Thu Feb 19, 2009 9:20 am Post subject: |
|
|
The fact remains that there is an interesting contrast between general perceptions of CP's defensive abilities and what APM says. It is an anomaly worth discussing.
not sure where the anomaly exists. you have a person who watched hornets games (in 07-08 ) tell us this about chris paul:
I really don't understand all of this statistical stuff,but as someone who has actually watched most of the Hornets games I'm somewhat confused by the repeated claim that Jannero Pargo is a good defender or a better defender than Paul. Pargo is probably one of the worst defenders in the NBA and as a player he's utterly useless when his shot isn't falling.
In fact I'm relatively sure that a lot of this discussion is a result of conflating Pargo's defense with Paul's. Before Pargo got hot and began to actually contribute I looked at 82games.com and Paul's defensive PER was around 14.5 or something. His differential PER was in the top five in the NBA because he was producing at better than twice the rate of opposing PG's. At that point Pargo was basically his backup and the Hornets hadn't yet given up on Morris Peterson. After Pargo began to produce he took much of Peterson's minutes,Pargo and Paul would probably play around 15 to 20 minutes per night together. Paul always guarded the stronger player because Pargo is completely incapable of dealing with larger players. That is to say that Pargo nearly always guarded the opposing point guard. Immediately Paul's defensive PER began to skyrocket.If you watched the games you'd see that teams go right at Pargo at every opportunity and that he offers very little resistance.
and a person who charted hornets games tell us this about chris paul:
Here are some numbers; The guy he was guarding scored .87 points per offensive action, (on 1203 actions). The best number on his team was .80 held by Rasual Butler (313 actions). Pargo's numbers were 1.01 on 620 actions.
The numbers I gave were for 07-08. His defensive numbers for 07-08 were above average for the PG position, his numbers for this year are also above average for the PG position.
and the people that quite possibly saw him play the most (coaches/sportswriters) vote him onto the all-D 2nd team and 2nd overall in MVP voting...
but another person who neither watches chris paul ("extensively") nor charts hornets games tell us this:
virtually everybody has trouble believing that a guy with eye-popping boxscore stats and horrific defense is really as big a liability as adjusted plus-minus says he is (think: Al Jefferson, Carmelo Anthony, CP3, etc.)...
that's why adjusted plus-minus is so potentially valuable - by unmasking the fact that players like Calderon and Carmelo and Al Jefferson and Richard Jefferson and Amare Stoudemire don't have nearly the positive impact that people think they do, because they give away on defense much of what they add on offense (if not more, in some cases).
Although Chris Paul was one of the Top 3 offensive players in the league (+9.24), he was actually a liability on the defensive side of the ball (-4.54). As a result, his overall contribution (+4.69), while impressive, was not commensurate with that of a top MVP candidate.
yet later tries to modify his comments:
In the thread on CP3's defense last year... I still concluded that he was a liability on defense in 07-08... but not to the extent that this completely "negated" his off-the-charts offensive contributions.
it would be a mistake to look at a player's defensive APM number from a given season and conclude: this is a valid rating of Player X's defensive ability. Rather, it's merely an indicator of his defensive contribution over the period in question
even if this interaction effect is found to account for much of CP3's improvement on defensive APM this year, it would not fundamentally call into question the validity of the APM approach to player evaluation.
if anything the anomaly is the methodology, which i believe we all want to work...
but when claims like the above are made, or:
http://blogs.wsj.com/numbersguy/mark-cubans-surprising-player-performance-numbers-580/
that "...sebastien telfair is more valuable than dirk nowitzki...", i can see why some may be skeptical or why someone might say adjusted +/- is "...one of the most overhyped player rating systems..."... |
|
Back to top |
|
|
schtevie
Joined: 18 Apr 2005 Posts: 409
|
Posted: Thu Feb 19, 2009 11:01 am Post subject: |
|
|
Bob. To my surprise, I find that to be a rather reasonable summary of the discussion. I look forward to seeing what additional numbers Steve comes up with regarding interaction effects.
As for APM being "...one of the most overhyped player rating systems..." however, I don't follow. First, I don't see much hype. Second, if you share this view, then you are at least implicitly of the belief that, overall, APM has shortcomings relative to another rating system. Is this true, or does your dissatisfaction relate only to particular estimates, such as CP's defensive APM in '07-'08 that don't pass your laugh test? |
|
Back to top |
|
|
deepak
Joined: 26 Apr 2006 Posts: 665
|
Posted: Thu Feb 19, 2009 11:17 am Post subject: |
|
|
gabefarkas wrote: |
Look, I'll lay my cards out on the table. I think APM is fascinating, and I want to learn more about what goes into it. My major gripe is twofold:
(1) It's probably not as much of a panacea as its proponents seem to think it is.
(2) Calling it "plus/minus" when it's split out into two parts, offensive and defensive, seems, for lack of a better word, silly. The very notion of a "plus/minus" implies that it accounts for the net difference from both ends of the court. Once you separate it out into two components, it ceases to contain that defining property. |
I don't see the problem you're describing in (2). Plus/minus refers, generally speaking, to how the score changes with the player on the floor. Adjusted plus/minus is describing how the score changes assuming everyone else on the floor is an average player. Logically, this is the same as defining it as how the score changes with the player on the floor compared to if he was replaced by an average player, and everyone else stayed the same.
So, if player A is +10 with everyone else on the floor being considered average, then that means if he replaces an average player and you make no other changes on the floor, he'd make the team +10 better. So, "adjusted plus/minus" and "on/off, replacing with an average player and holding everyone else constant" are really two sides of the same coin. |
|
Back to top |
|
|
|
|
You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum
|
Powered by phpBB © 2001, 2005 phpBB Group
|