A week ago, I discussed overrated and underrated players over the past 12 years, evaluated by comparing popular box-score stats with Jeremias Engelmann’s 12 year average Regularized Adjusted Plus/Minus (RAPM) dataset. For a primer on the virtues of RAPM (in large samples), see my article reviewing the state-of-the-art of adjusted plus/minus and stabilization.

This time, I am looking at offensive and defensive RAPM over the same 12 years, and comparing them to Offensive and Defensive Win Shares and PER. Again, I am limiting this to very large sample sizes (20,000+ possessions) where RAPM should be both accurate and stable.

Offense

Underrated

First of all: the R^2 of PER on ORAPM is 0.38 and the R^2 of OWS/48 on ORAPM 0.48. This indicates that Offensive Win Shares do a better job of measuring offensive value than does PER.

However, both box score stats underrate a similar set of offensive players: excellent point guards and creators. The most underrated players by both stats are Steve Nash and Baron Davis, both excellent passers/creators. Steve Nash is rated by ORAPM as the best offensive player of the past 12 years by a significant margin, rating out at +7.0, or 7 points per hundred possessions better than average. Second is D-Wade at +6.2; third is Kobe at +5.9.

Looking at the players most underrated by both box-score stats: Nash, Davis, Mike Conley, Ray Allen, Toni Kukoc, Damon Jones, Paul Pierce, Vladimir Radmanovic, Gilbert Arenas, and … Shaq.

I guess the exception proves the rule?

Overrated

Conversely, most of the overrated offensive players are post players. The most overrated player on offense by both systems is Andrew Bynum. Both box score stats rated him well above average, but ORAPM says he has been pretty bad. I am not totally sure what to make of this; it could be related to issues with ORAPM as well as him simply being overrated.

The most overrated players on offense: Andrew Bynum, Hakim Warrick, Yao Ming, Rajon Rondo (!), Marcin Gortat, Aaron Williams, Craig Smith, Dikembe Mutombo, Tony Battie, Eddy Curry, and Michael Curry. Outside of Rondo, that is a who’s who of “not creators” on offense! It appears that assists (at least with these implementations) are not enough to capture the effect of great creators on offense, and the effect of poor creators on offense.

Defense

Okay, let’s look at the other side of the ball.

First of all: note the confirmation of something John Hollinger has said often, but most people ignore: PER does not measure defense. At all. So I won’t even discuss PER for rating defensive players. It doesn’t.

Underrated

That leaves us to look at Defensive Win Shares. It is well known that the box score doesn’t really measure defense well. All we’ve got to go on are blocks, steals, rebounds, and the overall points allowed to the opposing team. That said, DWS/48 still manages an R^2 of 0.48 on DRAPM for this sample, oddly enough the same as was found for OWS/48 on ORAPM! When I previously looked at similar R^2 numbers with a different data set, I found OWS on ORAPM of 0.59, and DWS on DRAPM of 0.45. Some more investigation is needed on these subjects…

The most underrated defensive players of the last 12 years: Andrew Bogut, Brendan Haywood, Jason Collins, Amir Johnson, LaMarcus Aldridge, Dikembe Mutombo, Kevin Garnett, Shawn Bradley, Quinton Ross, Casey Jacobsen, and Nick Young (!!). It looks like Defensive Win Shares underrates the excellent rim protectors and mobile defensive bigs.

Overrated

On the other hand, let’s look at who is overrated by Defensive Win Shares: Carlos Boozer, Karl Malone, Andres Nocioni, Troy Murphy, Drew Gooden, J.J. Hickson, Chris Webber, Malik Rose, Steve Smith, and Jeff McInnis.

So the most overrated by Defensive Win Shares are again bigs–offensive minded bigs who are known for their poor defense. In other words–bigs MATTER on defense. Both the underrated and overrated lists for defense are populated mostly by bigs. Box score stats are not going to tell you how good your bigs are on defense with any certainty, but your bigs will definitely determine how good your team’s defense is.