APBRmetrics

Kevin Pelton · Posted: Tue Feb 17, 2009 5:46 pm Post subject:

My subjective perspective is that we're probably spending a lot of time arguing about something that is largely noise, along the lines of what Neil suggests.

Steve, could you calculate the statistical significance of the difference between Paul's defensive ratings? I don't believe you've posted error terms broken out by offense and defense for him those two seasons, plus my grasp of formal statistics is such that I would probably screw up that calculation myself with two variable distributions.

Westy · Joined: 15 Nov 2007 Posts: 15 Location: Chicago, IL

mtamada · Joined: 28 Jan 2005 Posts: 377

Mountain · Joined: 13 Mar 2007 Posts: 1527

Using 82 games data. the opponent eFG% with Paul on the court improved this season compared to last season by 2 percentage points but the comparison of the offs showed 3 times the change to the negative. Team defensive rebounding with Paul on the court actually fell 1% but team defensive rebounding % without Paul showed 5 times the change to the negative. Year to year opponent free throw attempts with Paul are up by 5 per 48. without just 1. Turnovers are unchanged for with or without Paul. Paul is slightly better this season than last on counterpart data but given most of these factor results I have trouble seeing where he improved on help or team defense net or that much. As Bob asked before, is the off data having too much impact?

Chandler's sharp fall - from injuries- from =1 to -9 but also Peja's fall from +13 to +6 and West's fall from +6 to 0 coincide with Paul's rise from =1 to +17. Playing with these same 3 teammates heavy minutes Paul has improved so much that he takes 85% of their collective credit losses? I've wondered before if adjusted overly polarizes and in this case I continue to wonder.

Whatever value adjusted has, actual value derived from a player is heavily affected by the lineups he is used in. Players are not the same player all the time, they are different depending on what opportunities / competition and support or lack of necessary support (different players different types of support needed, to different degrees and different impacts) they have in those lineups. I think somebody or several somebody's (including Mark Cuban) said that a strong adjusted is a sign of a player succeeding in a role / in the lineups he was used in. I agree with that and would add that the polarizing that seems to occur on teams may make the very best role achievers shine too brightly and the rest suffer some as a consequence. So given that and errors unless you are -4 I wouldn't be that critical.

Looking at Paul's five most used lineups, good for almost half his total minutes, I only find about 35 total minutes where Daniels played with the exact other 4. Not much for direct comparison of similar contexts. Paul's 6th biggest minute lineup was -7 per 46, in 34 minutes of use without Paul and +14 with. This and the overall raw Paul on/off do square with adjusted. So maybe it is on target. Hard to say. Would Daniels be better with these foursomes if he got more than scant minutes run with them? Probably. How much better? Would have to see it to know.

**Ilardi** · Joined: 15 May 2008 Posts: 265 Location: Lawrence, KS

So, Chris Paul's defensive adjusted plus-minus (APM) has improved substantially from 07-08 to 08-09 (partial season), a finding that's somewhat counter-intuitive, and we're left pondering the interpretive possibilities:

Maybe it's just the noisiness of the APM measure? (suggested by Kevin Pelton and others) Although the noise level in Paul's APM ratings is considerably higher than that of most other players in the league with comparable minutes (reflecting a relative lack of variability in Hornets' lineups), this cannot be the whole story. Paul's defensive APM estimate last season was -4.54 with a standard error of 1.52, so we can say with 95% confidence that his true rating last year was no better than -1.5 (since standard errors are roughly normally distributed). Conversely, his defensive APM this year (through Feb 3) is 6.76 with an se of 1.57, so we can say with 95% confidence that his true rating this year is no lower than roughly +3.5. So, despite the noise, his defensive APM rating this year is markedly improved, at a high level of statistical significance.

So maybe it's an inherent flaw in the APM metric itself? Maybe, but let's review again the evidence in favor of an alternative hypothesis - that the metric is valid, and Paul is actually having a more favorable impact on defense this year:

1) During Paul's on-court minutes, the Hornets' defensive efficiency has improved from 107.6 last season to 106.1 (pts per 100 poss) this season. That improvement has occurred despite a clear deterioration in the play (and minutes) of Tyson Chandler, who was widely regarded as the anchor of the team's defense last season.

2) The team's improvement in defensive efficiency with Paul on the court this season has also occurred despite a dramatic downturn in the team's defensive performance with Paul off the court. Last year, the team was better defensively when Paul was off-court (101.4) than when he was on (107.6); this year, the situation is reversed: they're much worse defensively when he's off-court (111.8 ) than when he's on (106.1). That's a swing of over 10 point per 100 possessions in Paul's teammates' off-court performance. And remember, most of these off-court teammates also share a high percentage of their minutes with Paul on-court. (We'll come back to this point in a moment).

3) Paul's opposing PG eFG% has dropped sharply from 51.8% last year to 46.7% this year (again, without the benefit of Chandler playing in top form). Improvements are also seen this year in opposing PG turnovers, assists, iFG%, and PER.

Now, it is certainly true, as Mountain and others have suggested, the APM estimates rely on the assumption that player effects on offense and defense remain reasonably constant across various lineup permutations. (This assumption is not strictly necessary, however, since the APM model can also test various player-by-player interaction effects, i.e., the possibility that the impact of Player A varies according to the presence/absence of Player B; however, no one to my knowledge has ever systematically tested such player-by-player interaction effects - a tedious, though highly worthy, task for another day).

Anyway, let's consider this key assumption of "constancy of player impact", since I believe our interpretation of Paul's defensive APM improvement hinges on it. As we've seen, Paul's teammates are much worse defensively this year when Paul is off-court, while the team was actually better without Paul defensively last year. This could either reflect the fact that: (a) Paul's teammates greatly improve on defense whenever they play alongside Paul this year, whereas (for some mysterious set of reasons) they did the exact opposite last year, and got much worse on defense whenever they played alongside Paul; or (b) Paul's teammates have exerted a reasonably constant defensive effect regardless of Paul's on-court or off-court presence, but Paul has stepped up his impact on defense since last season while his teammates in aggregate (especially Chandler) have gotten worse.

I happen to find the latter alternative the more parsimonious (and thus the more compelling), but I accept that tastes can certainly differ on this point. Perhaps no clear resolution will occur until I go back and test all the 2-way, 3-way, 4-way, and 5-way player interaction effects in an attempt to untangle the mystery in more systematic fashion. (Such an analysis will tell us for certain the extent to which Paul's teammates actually played better without him on d last year but better with him this year.) Since I've got lots on my plate for the next couple weeks, however, it may take until Spring Break before I'm able to carve out ample time to take this on. Stay tuned . . .

schtevie · Joined: 18 Apr 2005 Posts: 411

Steve, there is one particular story of player interaction, raised early on in this string, that seems to me the most compelling to investigate. And this has to do with the interaction of Chris Paul playing alongside other small guards. In 2007-08, in almost 40% of his minutes, he was running with Pargo and Jackson. (And these same guys were his teammates in '06-'07.) Chris Paul is rated by one year Def. Adj+/- as being a liability compared to these two. Perhaps it was the case that Paul, being the notionally better defender, was guarding the 2, and taking one for the team. 82games counterpart data says he didn't do even a second of that, but that can't be right.

This year, according to your data, Paul has played but 9 seconds with a guard smaller than 6' 4". This might be an important part of the turnaround.

Would a precise way to address this issue be to divide CP into two players: Small Paul and Large Paul, where the former would be his minutes running with small 2s and the latter when he was not at the same risk of being abused by taller opponents?

gabefarkas · Joined: 31 Dec 2004 Posts: 1313 Location: Durham, NC

**Ilardi** · Joined: 15 May 2008 Posts: 265 Location: Lawrence, KS

**Ilardi** · Joined: 15 May 2008 Posts: 265 Location: Lawrence, KS

Kevin Pelton · Posted: Wed Feb 18, 2009 2:29 pm Post subject:

Mountain · Joined: 13 Mar 2007 Posts: 1527

Lineup permutation and player interaction effects would be valuable new research tracks. Look forward to what you do / find / share Steve.

**Ilardi** · Joined: 15 May 2008 Posts: 265 Location: Lawrence, KS

**Ilardi** · Joined: 15 May 2008 Posts: 265 Location: Lawrence, KS

**Ilardi** · Joined: 15 May 2008 Posts: 265 Location: Lawrence, KS

bastillon · Joined: 04 Nov 2008 Posts: 55