 |
APBRmetrics
The statistical revolution will not be televised.
|
| View previous topic :: View next topic |
| Author |
Message |
bchaikin
Joined: 27 Jan 2005
Posts: 690
Location: cleveland, ohio
|
 Posted: Fri Feb 20, 2009 7:01 pm Post subject: |
 |
|
Adjusted plus-minus... when compared to all other overall player rating systems... is clearly superior when done properly.
clearly? might you care to substantiate this?...
i ask as aside from the adjusted +/- chris paul poor defender in 07-08 claims, i seem to remember these claims from 8/2008 in the "shane battier and bobby jackson" thread concerning brad miller and amare stoudemire, this one from your compatriot:
Yes, Miller has consistently (for several years) been a high adj +/- guy; Stoudemire, not so much (poor defense, alas). Accordingly, I believe Phoenix would have been better last year with Miller playing in place of Stoudemire.
and this one from you:
I just want to clarify that I am not arguing that Miller is better than Stoudemire. I am not at liberty to argue that one way or the other. I was merely commenting that, based upon publicly available data, Miller has consistently been one of the 3-4 best centers in the league over the period for which adjusted plus-minus has been available.
one says miller is better than stoudemire in 07-08, the other no. are you two getting different results for the same player(s) from your adjusted +/- methodologies?...
and miller in the top 3-4 Cs?...
amare stoudemire was voted all-NBA 2nd team in 07-08. i don't believe brad miller got even one single vote. plus i don't know what publicly available data you might be referring to, all the offensive stats i see for stoudemire for all intent and purpose dwarf those of miller for 07-08, and the defensive counterpart data for the C spot for when each was on the floor for their respective teams in 07-08 at 82games.com:
http://www.82games.com/0708/07SAC16C.HTM
http://www.82games.com/0708/07PHO15C.HTM
seems to indicate high offensive opponent production when each played, inferring neither was a good defender...
Last edited by bchaikin on Fri Feb 20, 2009 7:18 pm; edited 2 times in total |
|
| Back to top |
|
 |
Mountain
Joined: 13 Mar 2007
Posts: 1527
|
| Posted: Fri Feb 20, 2009 7:17 pm Post subject: |
|
|
| Based on APM I understand and agree with the generally positive view of Miller. Brad versus Amare is a hypothetical and I don't want to dwell on it but I think Miller on the Suns would fit and work pretty well. Amare / Porter didn't work but Amare / 7 seconds did. But now he's out so back to other stuff for the time being. But long-term I don't know that you go Nash-Amare-7 secs further except as last gasp hope in the face of nothing better. I think it is probably time to blow the whole thing up and start over, if it is championship or nothing. Dumars did that but it was Kerr that should have. He just added the Big Cactus and changed the coach and that wasn't enough or right. |
|
| Back to top |
|
|
gabefarkas
Joined: 31 Dec 2004
Posts: 1313
Location: Durham, NC
|
| Posted: Fri Feb 20, 2009 8:47 pm Post subject: |
|
|
| schtevie wrote: | Important roster decisions are necessarily based upon past as well as anticipated performance. With respect to offering him a five year contract, the probable 1.5 standard error on his then-to-date 1ish APM was well within the bounds of useful and actionable information. More generally, if individual player estimates are indeed independent of one another, multi-year estimates with standard errors around 1.5 provide a non-noisy basis for contemplating the effect of multiple roster moves.
My point is, that in terms of how APM has always been represented as being relevant, it can't be written off for the noisiness of its estimates. This was the basis for my saying that the Celtics had a good thing coming in 2008, and this is a basis for it being the preferred metric for similar types of analysis. What GMs do is important, right? |
So if you were assembling a team from scratch, who would you reasonably select to be on it? I include "reasonably" to take into account salary and playing time (ie, the 12th man almost never sees floor time) considerations. |
|
| Back to top |
|
|
Mountain
Joined: 13 Mar 2007
Posts: 1527
|
| Posted: Fri Feb 20, 2009 11:06 pm Post subject: |
|
|
| DLew, I agree with your post. |
|
| Back to top |
|
|
DLew
Joined: 13 Nov 2006
Posts: 224
|
| Posted: Sat Feb 21, 2009 4:09 am Post subject: |
|
|
Bob,
clearly? might you care to substantiate this?...
No, I have contractual obligations not to, and frankly I wouldn't care to if I was allowed. If you choose to make an effort to understand adjusted plus-minus then you'll likely come around, but I suspect you've already made up your mind about it.
and miller in the top 3-4 Cs?...
Again, I can't comment on this. However, I will say that questioning along these lines indicates that you still don't understand what we have been saying about the noise associated with adjusted plus-minus. Because of the large standard errors on the coefficient estimates in all the results that are publicly available it is pointless to engage in most player vs. player debates. This is not to say that you can't empirically test the validity of adjusted plus-minus using the publicly available numbers, you can, but not with a sample size of two players. |
|
| Back to top |
|
|
schtevie
Joined: 18 Apr 2005
Posts: 413
|
| Posted: Mon Feb 23, 2009 2:45 pm Post subject: |
|
|
| HoopStudies wrote: | | schtevie wrote: |
My point is, that in terms of how APM has always been represented as being relevant, it can't be written off for the noisiness of its estimates. This was the basis for my saying that the Celtics had a good thing coming in 2008, and this is a basis for it being the preferred metric for similar types of analysis. What GMs do is important, right?
Can we agree on that? |
I agree that GMs are important.
If you're saying that picking the Celtics to win the title in 2008 based on APM is proof that it is not noisy, I don't agree. I hope no one would believe one instance of an accurate prediction as proof.
You're generally talking about making predictions as evidence of a method's value. There are flaws with that approach (I've been told). But even if it is accepted, you gotta do more than 1. Or even do retrodictions, where at least you know player minutes after a season but use the prior season's best estimate of per-minute performance (or, essentially equivalently, per-possession stats). Since you now have APM on the O side and D side, predict team O Rtgs and D Rtgs, not just for one team. And then compare to other methods. (There are important nuances in doing this, but I'll let you figure those out.) |
Dean, I am not sure that you are reading what I was trying to write. Picking the Cs to win the title in 2008 based on APM was motivated by the expectation that the sum of the individual APMs would only be slightly biased when aggregating to the team level. The confidence I had that the estimate would be reasonably precise was based on the expectation that in summing individual APMs a lot of the noise of individual estimates would be expected to drop by the wayside. The latter point is related to econometric theory and has nothing to do with empirical confirmation.
As for there being flaws with the approach of using predictions or retrodictions as the metric of a method's value, what are these exactly and why are they particularly relevant to the exercise in question?
Yes of course "you gotta do more than 1", but at the same time, all experiments are not equal. This example was not cherry-picked. It was perhaps the largest net transfer of apparent talent to one franchise in the history of the league - two future HOFs for prospects. (My apologies to the historians for any egregious oversights.) If one's preferred scheme for appraising such momentous events doesn't get this kind of story straight, this is a big problem, no? It would not surprise me much if for the bulk of more or less average teams, with no conspicuous roster changes, that a variety of techniques would yield largely similar results. And furthermore that comparing these measures would persuade no one of anything. It is the outliers where the action is.
But you are right, "you gotta do more than 1". I would hope that I could persuade someone with the data at hand to address the issue conclusively. I see that Neil has some interesting results with statistical +/- on basketball reference. Perhaps he would like to take the ball and run with it. Or Steve, he clearly has a lot of free time.
But for fun, I did look at another team from the same season that had another kind of outlier status, the Chicago Bulls. Though here, this was a team that woefully underperformed relative to expectations, and as I recall there were no exceptional injury-related issues to explain their fall from grace. What the exercise in retrodiction shows (again, individual APMs summed in proportion to minutes, as opposed to possessions, played; rookies assigned actual APMs; replacement players assigned -7) was that using '06-'07 data, the predicted team APM for '07-'08 was 0.68 as opposed to the observed -2.94 (courtesy of basketballvalue). So, the prediction would be for a roughly 42 win team, instead of a 52 to 55 win team, courtesy of Dave Berri and John Hollinger. So APM does OK, not great, though I think that when it comes to teams in turmoil and free-fall, stemming from unmet expectations, one needs to cut all ratings systems some slack.
Performing the same exercise, but using previous, single-season data, the prediction changes dramatically. Using numbers from '05-'06 and '04-'05 the prediction is a team APM of 7.53 and 6.61.
And one final point which may be of some interest. Looking at the particular members of the Bulls roster and trying to tell myself a tentative story of what was going on, there was one individual contribution that was particularly notable: the Roland-ridiculed estimate of Ben Wallace in '05-'06 where his APM was 17.92. Of course, the rules are the rules for these prediction/retrodiction games, but this fact did lead me to finally read the fine print in Steve's 82games article on the '06-'07 season, and it appears that to lower standard errors, he blended in weighted '05-'06 data. Taking out the back of my envelope, I created a modified, interpolated set of values for '06-'07. The effect of this was to correspondingly reduce Ben Wallace's '06-'07 APM (and alter every other team member's as well). The overall effect of such tampering was to offer a revised '06-'07-based retrodiction of the '07-'08 Chicago Bulls' team APM of.....-1.68.
Such shenanigans however do oblige full-disclosure. Doing the same modification of the Celtics data implies that they should have done even better than I actually predicted. And maybe they should have. According to '07-'08 APM, Ray Allen was actually a big disappointment. Perhaps the real story is what could have been had his off-season surgery not hampered his performance. No need to tamper with conventional wisdom too much however. |
|
| Back to top |
|
|
Mountain
Joined: 13 Mar 2007
Posts: 1527
|
| Posted: Mon Feb 23, 2009 5:15 pm Post subject: |
|
|
Unweighted the positional averages of all positions fell short of Neil's projected statistical plus minus, the most at SG (-3) and the least at PG (less than -1).
Weighted by minutes PGs outperformed projection significantly and Centers outperformed too but by a bit more than half as much. The SG-SF-PF middle guys underperformed with PFs and SGs underperforming by a third more than the PGs out performed.
Altogether, minutes weighted players underperformed the projection at a rate I would consider significant. This in spite of a team efficiency improvement? Something is going on or the projection method needs adjustment. |
|
| Back to top |
|
|
Mountain
Joined: 13 Mar 2007
Posts: 1527
|
| Posted: Mon Feb 23, 2009 5:47 pm Post subject: |
|
|
| Looking at top lineups (raw) and everything else I'd give the edge to the Cavs over their main rivals. To beat Cavs somebody is probably going to have at least 2 good games guarding / limiting James. Pierce, Ariza and Turkoglu all had one regular season so there is reason to hope it can be done. |
|
| Back to top |
|
|
trk
Joined: 26 Feb 2009
Posts: 12
|
| Posted: Sat Feb 28, 2009 3:17 am Post subject: The effect of Chris Paul’s backups on his defensive Adj +/- |
|
|
On of the problems I have seen with both raw and adjusted +/- ratings is that the ratings are often better at showing how good a player is compared to other players on the same team who play the same position than it at showing how good a player is compared to the league as a whole.
Lets say there is a starter Aaron and a backup Bob who play the same position on the same team. Either Aaron or Bob is almost always on the court, but very rarely are both of them on the court at the same time. Aaron is a lot better than Bob, so the team tends to do better when he is on the court. As long the starters aren’t all subbed into and out of the game at the same time (which would add a bunch of new problems for any +/- metrics), adjusted plus minus should be able to easily figure out that Aaron is better than Bob. However, even after you figure that out, you still don’t know how good Aaron and Bob really are compared to the rest of the league. It could be that Aaron is an elite player and Bob is an average player, or Aaron could be the average player and Bob could be an exceptionally bad player. Since you rarely see them both on the bench or both on the court at the same time, it is very difficult to determine their value in absolute terms rather than just in comparison to each other.
I think this effect might explain most of Chris Paul’s dramatic improvement in defensive adjusted +/- compared with last year. In 2007-2008, the PG when Chris Paul wasn’t on the court was usually either Bobby Jackson or Jannero Pargo. This year most of the backup point guard minutes are going to Antonio Daniels, with some minutes also going to Devin Brown. Now, I’m not an expert on the Hornets, but what I have hear of Antonio Daniels is that he is a poor defender who is getting worse as he ages. A check of Daniels’ production by position data from 82games shows that he has an unusually poor opponent PER of 20.0 when he plays at point guard, further backing up the idea that he is a poor defender. Is it possible that Chris Paul didn’t suddenly jump from being a very poor defender to being one of the league’s best, but that it just looks that way because he is now being compared with Daniels and Brown rather than Jackson and Pargo?
The Hornets are allowing 105.2 points per 100 possessions while Chris Paul is playing this year, which is just slightly better than the 106.6 points per 100 possessions they allowed last year. This year the Hornets allow 110.6 points per 100 possessions without Paul (meaning Daniels or Brown was the PG) vs. only 100.6 last year (when Jackson or Pargo was the PG). Rather than concluding that “everyone got a lot worse on defense except Chris Paul who got a lot better”, isn’t it more logical to conclude that Paul and most of the other players haven’t changed that much but Daniels & Brown are much worse at defending the PG position than Jackson & Pargo? |
|
| Back to top |
|
|
Mountain
Joined: 13 Mar 2007
Posts: 1527
|
| Posted: Sat Feb 28, 2009 4:04 pm Post subject: |
|
|
| I think what you say is a big part of the story but might not be the whole story. |
|
| Back to top |
|
|
Django
Joined: 22 May 2008
Posts: 4
|
| Posted: Sat Feb 28, 2009 9:22 pm Post subject: |
|
|
| The biggest cause for higher opponent scoring when Paul is off the floor would be turnovers.Paul has an assist to turnvover ratio around 3.8 to 1 while his teammates turn the ball over at a much higher rate,especially when he's not on the floor.In fact their basic gameplan is to keep the ball in Paul's hands,minimising turnovers to keep their opponents from getting easy baskets.Since they are an offensively challenged team with only two reliable scorers at this point it's critical for them to make their opponents work hard to score,because they themselves aren't well equipped to win high scoring games. |
|
| Back to top |
|
|
Crow
Joined: 20 Jan 2009
Posts: 821
|
| Posted: Wed Oct 21, 2009 2:41 pm Post subject: |
|
|
My previous skepticism about Carmelo Anthony was based on past individual Adjusted +/- data to support . He turned that number around this regular season and post season. Apparently they finally got him in contexts where he worked well, on average. Finally had all the ingredients around him to make it work- Nene's defense and rebounding, Billup's leadership and shot making, etc. Maybe it stays there, maybe not. I'd guess it probably will stay better than before but not sure how elite it will stay. But individual ratings are only an ingredient and is not the real destination, which is lineup performance.
On Adjusted +/- at the lineup level in the playoffs the Nuggets 2 most used lineups came out a bit negative and more importantly far worse than the next 5 (all with Chris Andersen- perhaps he should have been used more?). The regular season Adjusted +/- at the lineup level suggested Anthony, - Billups- Nene - Jones - Martin wasn't good or even positive and the playoff data said almost the exact same thing. Should that lineup have been de-emphasized ? I'd think so (and said it several times last season) given the much stronger performance of other lineups but instead it went from being used about 1 in 7 minutes in the regular season to being used twice as frequently in the playoffs. Seems like a big disappointment- or error- there that could have been avoided, at least in terms of not doubling its use. If you gave weight to Adjusted Lineup data.
But to be fair and to note the complexity of the challenge- the regular season data said Anthony, - Billups- Nene - Smith - Martin was great but the playoff data said it was a bit below neutral. The Nuggets may have listened to the regular season data (or got to the same decision by other means) because they increased the use of this lineup from 1 in every 24 minutes to 1 in every 9. A good idea but not all good ideas work. Was there a point where they should have seen it wasn't working that well? Aaron's game by game evolving lineup Adjusted data was available. If you did look I think you might have been able to see these lineups were really providing edge - on this measure- and could perhaps shift to trying other things more that were working better. Trend chasing can often disappoint but not noticing or chasing them can be a problem too. Should they have been content to be slightly negative on these lineups? Were they trying to win with other lineups? It is an available strategy but didn't quite work against the very best. Particular match-ups matter- team, position and where the rubber meets the road lineup vs lineup.
One lineup tracked in the playoffs nearly exactly to what it did regular season and the other was wildly different. You can throw up your hands and ignore it as not a reliable guide with too large errors or go to the splits (regular season data against playoff level teams or "similar" teams to a specific match-up) and dig hard and ask why and see if you can find things or maybe even find ideas about things that might be useful in combination with other information.
It still comes down to interpretation and blending all the input though. |
|
| Back to top |
|
|
Crow
Joined: 20 Jan 2009
Posts: 821
|
| Posted: Fri Nov 27, 2009 11:39 pm Post subject: Re: The effect of Chris Paul’s backups on his defensive Adj |
|
|
Warning: Old thread but seemed appropriate to me to put these comments here since they pull on previous comments in this thread:
| trk wrote: | On of the problems I have seen with both raw and adjusted +/- ratings is that the ratings are often better at showing how good a player is compared to other players on the same team who play the same position than it at showing how good a player is compared to the league as a whole.
...
I think this effect might explain most of Chris Paul’s dramatic improvement in defensive adjusted +/- compared with last year. In 2007-2008, the PG when Chris Paul wasn’t on the court was usually either Bobby Jackson or Jannero Pargo. This year most of the backup point guard minutes are going to Antonio Daniels...
Is it possible that Chris Paul didn’t suddenly jump from being a very poor defender to being one of the league’s best, but that it just looks that way because he is now being compared with Daniels and Brown rather than Jackson and Pargo?
The Hornets are allowing 105.2 points per 100 possessions while Chris Paul is playing this year, which is just slightly better than the 106.6 points per 100 possessions they allowed last year. This year the Hornets allow 110.6 points per 100 possessions without Paul (meaning Daniels or Brown was the PG) vs. only 100.6 last year (when Jackson or Pargo was the PG). Rather than concluding that “everyone got a lot worse on defense except Chris Paul who got a lot better”, isn’t it more logical to conclude that Paul and most of the other players haven’t changed that much but Daniels & Brown are much worse at defending the PG position than Jackson & Pargo? |
This season Paul without Chandler, the Scott firing. other stuff and backed up at PG by Collison and B. Brown:
Paul on the court team Defensive Rating 115.3
Collison 110.6.
B Brown 111.6
It is real early of course but wonder what kind of Defensive Adjusted rating Paul gets this season.
Last edited by Crow on Sat Nov 28, 2009 2:51 pm; edited 3 times in total |
|
| Back to top |
|
|
Crow
Joined: 20 Jan 2009
Posts: 821
|
| Posted: Fri Nov 27, 2009 11:43 pm Post subject: Re: The effect of Chris Paul’s backups on his defensive Adj |
|
|
3 of the 4 factors worse on than off including shot defense and forced turnovers. Paul has below average counterpart defensive data right now too.
Last edited by Crow on Sat Nov 28, 2009 1:52 am; edited 1 time in total |
|
| Back to top |
|
|
Crow
Joined: 20 Jan 2009
Posts: 821
|
| Posted: Sat Nov 28, 2009 12:21 am Post subject: |
|
|
DLew wrote earlier in this thread:
"To me, there are three main things that we want to know about a player:
1. How good has he been?
2. Why was he effective/ineffective? i.e. How did he achieve his level of performance?
3. How good is he going to be?
...
The second question is not really addressed well by adjusted plus-minus, as Roland and many others have pointed out..."
Perhaps 4 Factors Adjusted will help with Question 2.
And more attention to lineup Adjusted +/- and sub-unit Adjusted (e.g. player pairs).
Are there top teams with analytic shops not using Adjusted or the full range of Adjusted versions and splits at all or hardly at all? It seems like a lot to give up or push to the periphery.
Or do they use it but just not promote or acknowledge as highly as others?
What are they are using instead beyond the known discrete action tracking tools (items in the boxscore or the additions that are easily identified whether they are available from PBP parsing, Synergy video or manual, insider-informed compilations- what was supposed to happen vs what did) or products built off them? Are they making better decisions mainly with these other tools than those also using Adjusted tools or using them a lot more?
Paul is a contentious test case but I believe digging into test case can potentially help.
Anthony is another. He looked good by Adjusted last season, far better than most of the time previously. Believe it or not- then or for thinking about the future? This season his early small sample Adjusted is back at -5.
Complicated. I tend to think that further study of the role of lineups and sub-unit / player interactions might help improve explanatory power and perhaps predictive power beyond where it often gets left right now in public. Whether it be by Adjusted techniques and / or otherwise.
By Adjusted lineups the simplest stories for contenders are being written in Dallas, Denver, Phoenix and with the Lakers. Whatever you take from that. None of these big believers in Adjusted? Different tools got them to the desired place as measured by lineup Adjusted?
Virtually everywhere the Player level Adjusted is complicated right now though= to my eyes.
Including Cleveland and Houston where there is also less consistently good lineup Adjusted estimates for top lineups. I am guessing these teams use Adjusted techniques the most? Issues related to more and more important newcomers hard to predict with Adjusted or fix quickly? Seems like last season was smoother sailing at the lineup level.
What to make of the use or non-use of tools and the results by different levels of Adjusted or simply point differential or win %?
I sometimes want to inject "What about the Lakers' approach" or more generally "What about systems / coaches?" how much does that explain by these top contenders (including Dallas, Denver, Phoenix) using Adjusted techniques heavily or not? |
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
Powered by phpBB © 2001, 2005 phpBB Group
|
FAQ
Search
Memberlist
Usergroups
Register
Profile
Log in to check your private messages
Log in
