APBRmetrics

Mountain · Joined: 13 Mar 2007 Posts: 1527

I came back across Ryan's Points Added again.

http://tinyurl.com/cfclf9

I found myself thinking there are some advantages of a positive 0-1 scale. Focus on the scoreboard producing positive rather than trying to pin down blame at the same time or really wrestle with the causation/correlation question about that "blame". Of course data is lost too. This cut is probably more similar to statistical +/- than pure adjusted.

I'd be hearing more from Ryan about his model and comparison of the results to adjusted and comment from the adjusted modelers on it.

And from anyone about the possible value of running a negative scale -1 to 0. Doing so would give another view different from what the positive scale does (does this essentially focus on the non-boxscore?) and similar to what adjusted does to get offense/defense splits.

And if you still split out offense and defense you could see how many points added and subtracted a player causes on offense and defense instead of just the net so you could see who still has work to do minimizing negatives even with their positives on a side of play or recognize the low mistake guys even if they aren't doing the positive work (sometimes they don't have to with others in roles and with high abilities for that.)

Having pure adjusted, statistical and points added positive and negative would give at least 4 perspectives.

In the end it is about gathering information for consideration, multi-dimensional (matter/energy) understanding building and not one number settles everything.

Ryan J. Parker · Joined: 23 Mar 2007 Posts: 707 Location: Raleigh, NC

As DLew pointed out, my formulation is really just a translation of +/-. The estimates are clearly going to be different, but for the most part it's the same thing. The published work on adjusted +/- doesn't spell out the weighting very clearly (consider this my naivety with fitting the data with a classical linear regression model), so I went this route to try and grasp the concept.

Also, See the latest discussion on Chris Paul to see the difficulties underlying the affects of a player's coefficients. Does the methodology really do a good job of rating players versus the rest of the league?

Lets suppose we rate a player's rebounding %. Is it safe to assume that players playing on teams with non-normal rebounding strategies are worse than the other players in the league? How do we measure this affect? I mean clearly they go smaller which has an affect, but I'd think all that leaking out has an important role as well.

There are some important things we don't have a good grasp on that affect these ratings. That is what I'm studying. Smile

_________________
I am a basketball geek.

Mountain · Joined: 13 Mar 2007 Posts: 1527

Ok "my formulation is really just a translation of +/-". A statistical player translation of raw team +/-? I think that is what you mean. right?

If so then a -1 to 0 scale isn't going to work here without a shot defense blame assignment. And total blame by matchup isn't going to justify and other split rules aren't going to justify many.

0 to 1 and -1 or 0 scale runs with the adjusted method could be done though, I think and might be worthing do, I think.

**Ilardi** · Joined: 15 May 2008 Posts: 262 Location: Lawrence, KS

bchaikin · Joined: 27 Jan 2005 Posts: 681 Location: cleveland, ohio

With all due respect, it appears you've mischaracterized my position.

did you or did you not start this thread with this statement:

Chris Paul's defensive rating is dramatically higher than it was last year (improving from -4.54 to +6.78 this season).

and did you or did you not reference your article on 82games.com for 07-08, in which chris paul and his defensive adjusted +/- -4.54 rating is listed at the bottom of the defensive adjusted +/- list among other supposed poor defenders like michael redd and al jefferson? i'm assuming a high negative number there represents a poor defender and a high positive number represents a very good defender - or is this a bad assumption?...

in the "chris paul +/-" thread you also made this statement:

Bottom line: the Hornets give up more points with Paul on the court than they do without him (over 6 more points per 100 possessions). Because this on-off value is the worst on the team, it does not appear to be a mere artifact of his playing with other weak defenders, nor does it appear to be a fluke of sampling (since his on-off defensive rating was virtually identical during the 06-07 season: -6.8 pts/100 possessions).

despite the fact that chris paul played 3 times as many minutes compared to what the hornets played without him, you insisted his defense was poor based on what the team did without him...

and did you or did you not in the "new adjusted plus minus ratings for 07-08" thread make this statement:

Although Chris Paul was one of the Top 3 offensive players in the league (+9.24), he was actually a liability on the defensive side of the ball (-4.54). As a result, his overall contribution (+4.69), while impressive, was not commensurate with that of a top MVP candidate.

saying in no uncertain terms that the PG with the most votes for the nba all-D team was a poor defender (or does "liability on the defensive end" mean something other that what it sounds like - i'd hate to mischaracterize you here), and that a player that was voted to the all-D 2nd team and the all-nba 1st team was not a viable mvp candidate?...

and now you make this statement:

Further, I didn't say Chris Paul is "dramatically better on defense now"; what I said was "his defensive rating is dramatically higher". It's perhaps a subtle point, but an important one. One might infer the former from the latter, but a better interpretation would be "his overall impact on the Hornets' defensive efficiency is markedly more positive this year.... there is only one way to account for the fact that the Hornets have improved in defensive efficiency with Paul on-court this year in comparison with last: Paul is having a more positive impact..."

in the span of a few paragraphs you have gone from "dramatically" to "markedly more positive" to just "more positive". so just how big is a jump from -4.54 to +6.78 in defensive adjusted +/-? is that an awful to an excellent defender, a poor one to a very good one, or is it just a below average defender to a just above average defender? your 07-08 listing shows only one player better than +6.78 (kevin garnett at +7.59) in adjusted defensive +/- and only four players with a rating less than -4.54 (al jefferson, al thornton, leandro barbosa, and kevin martin) out of a list of some 120+ players who played over 2000 minutes. so is this range of paul's from 07-08 to 08-09 very large or not?...

Your analysis regarding team offensive/defensive efficiency with Paul on-vs-off court does not strike me as persuasive.

it wasn't meant to be persuasive, just factual. those numbers came straight off the 82games.com website...

Evidence consistent with this interpretation is found in the fact (noted earlier) that Paul's opposing PGs have an eFG% that's dropped from 51.8% last year to 46.7% this season.

perhaps you should re-read the "chris paul +/-" thread...

let's do the math again. the league average PG in 07-08 playing about 3000 minutes took about 1000 shots, so the average PG playing 3000 minutes defended about 1000 shots. the difference between defending at 46.7% vs 51.8% would be 467 vs 518 FGM or 51 FGM over a season. if his improvement this year is in forcing another 51 missed FGAs over the span of a season, then those missed FGAs have to be rebounded by the defense to be a defensive stop. assuming 70% are rebounded by the defense that's about 51 x .70 = 36 more defensive stops over the span of a season due to better defense on FGAs...

but we are talking about a player who got 217 steals in 07-08 (each of which is a defensive stop) and is on pace to get another 215-220 steals this season. that's 215/36 or about 6 times as many defensive stops due to steals than the additional stops he forces due to better defense on shot attempts...

so you can quote those opposing eFG%s all you want, but we are talking about a player here who gets a ton of defensive stops from steals that dwarfs any difference in his eFG% allowed from last season to this season - in other words a player who was a good defender both last year and this year...

Likewise, this year opposing PG assists are down, turnovers up, iFG% down, and PER down.

again last season the PER for hornets opposing PGs when paul was on the floor was 18.5, but in 08-09 its 16.9. i don't see alot of difference here. better defensively? yes, but not by much, nothing that is "dramatic" or "markedly more positive"...

it does not strike me as implausible to assert that Paul is having a considerably more positive impact on the Hornets' defensive efficiency this year.

where's the evidence? adjusting a player's "defense" dramatically (saying he's a liability one season but then markedly more positive the next) based on what happened when he did not play, even though the team defense when he did in fact play changed little, does not seem like plausible evidence at all...

"...some such players ultimately decide to take it to the next level on d: LeBron has been highly vocal this season on that very point, and his markedly improved defensive APM backs him up. My sense is that Paul has done likewise this season, but having watched the team only a couple times this year, I cannot verify it. Hence, my earlier query..."

are you seriously trying to equate paul's supposed better defense this season compared to last with the better defense of lebron james from last year to this year? the hornets allowed 107.6 pts/100poss with paul in 07-08, 106.1 pts/100 poss with paul so far in 08-09. that's a difference of just 1.5 pts/100poss, enough to move a team up the defensive rankings about 1-5 notches in rank...

however in 07-08 with james cleveland allowed 107.7 pts/100poss, yet so far in 08-09 just 100.8 pts/100poss. that's a difference of almost 7 pts/100poss, which right now is the difference in defensive ranking between the best defensive team in the league and about the 18th best (13th worst) team in the league defensively in terms of pts/100poss allowed...

you might want to try another more analogous comparison for paul other than james...

DLew · Joined: 13 Nov 2006 Posts: 222

Bob,

Let's discuss the 82games information which you provided, as I feel that you did not make the appropriate inference.

(1) The Hornets are much worse on defense this year with Chris Paul off the court, as compared to last year with Chris Paul off the court.

From this we learn that players on the Hornets not named Chris Paul are collectively playing worse defensively this year than last year.

(2) The Hornets are better on defense this year with Chris Paul on the court as compared to last year with Chris Paul on the court.

Given that we know that his teammates are playing worse defensively than last year, this piece of information strongly suggests that Chris Paul is playing far better defensively.

Please let me know if you feel this line of reasoning is valid. It is the one that the 82games numbers suggest to me, but obviously leads to a different conclusion than the one you came to.

cherokee_ACB · Joined: 22 Mar 2006 Posts: 157

deepak · Joined: 26 Apr 2006 Posts: 664

Mountain · Joined: 13 Mar 2007 Posts: 1527

In rough terms Paul on the court is with West 80% of the time, Peja 70%, Butler 60%, Posey 50% and Chandler 50%. The rest of team fills 90% of one slot.

Paul off the court has 85% Posey, 45% West, 60% Butler, 60% Daniels, 50% Brown, 50% Armstrong, 30% Marks, 30% Wright, just a bit over 10% for Chandler, 20% Peterson and 40% other.

So Paul has 6 pretty solid pieces on team defense for about 78% of the available total time.

The team with Paul off plays those same guys just 50% of total time. That edge in having familiar decent pieces around him on defense probably helps Paul.

Maybe B Scott needs to sort out a non-Paul roster that does better on defense. Maybe that group needs more continuity than 10+ guys checking in and out. Maybe they need more Chandler or a better Chandler substitute or a better non-Chandler defensive scheme. Or better backcourt protection / court leadership without Paul. Needs something(s).

bastillon · Joined: 04 Nov 2008 Posts: 55

**Ilardi** · Joined: 15 May 2008 Posts: 262 Location: Lawrence, KS

There's much to reply to in recent posts, so I'll do my best to proffer a response on the most salient points:

1) Chris Paul's defensive APM (adjusted plus-minus) has improved markedly over last season, and there are converging lines of evidence regarding the validity of these observed APM changes, including: (a) the improved on-court defensive efficiency of the 08-09 Hornets (versus 07-08 Hornets) with Paul on-court, despite marked deterioration in the team's defensive efficiency with Paul off-court (and, as Mountain noted, the 6 players most commonly on-court with Paul in 08-09 are still on-court 50% of the time when Paul is out, so there is substantial personnel overlap across the team's Paul-on and Paul-off minutes); (b) an improved 08-09 Hornets defensive efficiency with Paul on-court despite the stark drop in effectiveness (and minutes) of the team's 07-08 defensive centerpiece, Tyson Chandler; and (c) a substantial drop in the effectiveness of Paul's opposing PGs in 08-09. (Bchaikin attempts to dismiss this latter point, waving away, for example, the impact of the drop in eFG% - down to 46.7% from last year's 51.8% - by calculating a net of roughly 36 more defensive stops per season, but he ignores the fact that much of a player's defensive efficiency lies in his contribution to team defense - rotations, double-teams, transition coverage, traps, etc.; if a player's on-ball defense is improved, presumably it's not a stretch to infer that his off-ball defense has experienced commensurate improvements; bchaikin might also ponder what Paul's opposing PGs would have yielded in collective FG% this season had the Hornets benefited from the 07-08 version of Chandler.)

2) Bastillon raises interesting questions about the validity of defensive APM ratings when they're derived from a multi-season database, given the apparently high potential for variability in individual performances from one season to the next. My own take is that there's one sense in which the concern is warranted, and one in which it's not. Let me explain. My primary goal in APM analyses is to provide a metric of a player's actual contribution to the team's bottom line in a given season, not to provide an indicator of a player's potential contribution (e.g., if he were to start playing harder, adopt a different role in a different system, etc.). What I'm discovering is that offensive APM (which correlates quite heavily with traditional boxscore stats, and metrics like PER that are derived from them) is more stable from year to year than is defensive APM. This isn't terribly surprising, but I think it's an important discovery with myriad implications. It suggests, for example, that many players have the potential to greatly increase their on-court impact by giving greater effort (and playing smarter) on the defensive side of the ball.

Now, is it a problem for the APM analysis that some players (like Paul and LeBron) experience major changes in defensive APM from one season to the next? Not really. The model uses 6 seasons' worth of data on the premise that cross-season regularities in performance (which decidedly exist) help reduce estimation error for any given season, and this is simply an empirical question that has been answered in the affirmative. For example, the standard error in Paul's 08-09 defensive APM estimate (through ~50 games) is only 1.57 (using 03-08 seasons, with very low weightings, to reduce model error), compared with an implied standard error for Paul's single-season 07-08 estimate (in Eli W's single-season analysis) of over 7.0. In other words, using multiple seasons' worth of data, we can still get low-noise estimates of each player's performance in any given season, despite the fact that some players will vary quite a bit on the defensive end from year to year.

Nevertheless, as Bastillon implies, it would be a mistake to look at a player's defensive APM number from a given season and conclude: this is a valid rating of Player X's defensive ability. Rather, it's merely an indicator of his defensive contribution over the period in question.

3) Bastillon also wonders about the apparent threat posed by a whole-team turnaround in defensive effort, and he proffers the 08-09 Nuggets as Exhibit A. Well, first, the Nuggets weren't that bad defensively in 07-08 (in fact, they were above average). They've improved from #12 in the league in 07-08 (106.2 ppg) to #6 this season (102.4) - a nice jump, but hardly a revolutionary metamorphosis. Regardless, the APM model can handle such cross-season fluctuations just fine: it merely reflects the fact that some players (perhaps most) on a given team experience improvement in defensive APM from one season to the next. While such team-wide improvements might add a bit more noise to the overall model, the effect is not terribly large - and besides, it's an effect that's readily observable in the standard error terms associated with each player's estimates, so the model will actually tell us if there's a problem worth worrying about.

Ok, there's lots more to say, and more questions I haven't yet addressed, but I need to go grade some papers, so hopefully this is at least a decent start . . . more anon.

P.S. Bastillon, I'm having trouble interpreting your question about Posey: perhaps you could re-state it? For what it's worth, I have his defensive APM dropping from 0.57 last year in Boston (slightly above-average defensive contribution) to -1.43 this season. This drop is about what we might expect of a 32-year-old player (i.e., past his prime) adjusting to a new system.

bchaikin · Joined: 27 Jan 2005 Posts: 681 Location: cleveland, ohio

...this piece of information strongly suggests that Chris Paul is playing far better defensively.

to whom? to you? the hornets team defense gets a bit better (not insignificant, but not significantly better) when paul was on the floor from one season to the next (107.6 to 106.1 pts/poss allowed), but because of something that has absolutely nothing to do with chris paul himself, and based on a sample size of data that is just 1/3 the size of the data that when paul played and is thus much more apt to be skewed, adjusted +/- comes to the conclusion that paul "...was actually a liability on defense..." one season (07-08 ) but is "...far better defensively..." the next (08-09)?...

this despite the fact that the defensive pts/poss allowed when he actually did play changed little? and despite the fact that the 07-08 season that adjusted +/- finds paul to be a liability on defense there are many others who have found his defense to be quite good? after all he received not only the most votes among all PGs for the all-D team, but also the 7th most votes among all players in the entire league for DPOY? and this from the people that have actually seen him play alot - how much did you actually see chris paul play last year?...

tell us then - knowing that the number of steals paul got both seasons is about the same (and each is a defensive stop), and the data for eFG% allowed seems to show at most 1 additional defensive stop through missed FGAs forced every 2 games, just what exactly has chris paul - himself - done this year that he did not do last year such that he is "...playing far better defensively..."?...

do you have any evidence whatsoever that paul has forced significantly more non-steal turnovers, or that he was personally responsible for forcing more missed FGAs by the players guarded by his teammates (that might not be reflected in data such as that found on 82games.com) this year than last year?...

"...a substantial drop in the effectiveness of Paul's opposing PGs in 08-09. Bchaikin attempts to dismiss this latter point, waving away, for example, the impact of the drop in eFG% - down to 46.7% from last year's 51.8% - by calculating a net of roughly 36 more defensive stops per season, but he ignores the fact that the bulk of a player's defensive efficiency lies in his contribution to team defense - rotations, double-teams, transition coverage, traps, etc.; if a player's on-ball defense is improved, presumably it's not a stretch to infer that his off-ball defense has experienced commensurate improvements, as well;..."

oh, i get it. let's just completely ignore the actual data that does in fact exist, and then infer what we will to validate our conclusions. i'll ask again, how much have you actually seen chris pual play? because in the opinions of those who have seen him play the most (the coaches and media) he was a very good defender in 07-08...

"...bchaikin might also ponder what Paul's opposing PGs would have yielded in collective FG% this season had the Hornets benefited from the 07-08 version of Chandler...."

oh, so now not only something that happened when chris paul was not on the floor, but something that has not happened period, is futher validation of adjusted +/- conclusions?...

Neil Paine · Joined: 13 Oct 2005 Posts: 774 Location: Atlanta, GA

Neil Paine · Joined: 13 Oct 2005 Posts: 774 Location: Atlanta, GA

Kevin Pelton · Posted: Sun Feb 15, 2009 1:53 pm Post subject: