APBRmetrics

[dogra]

Hi everyone. I am new to this forum. I used to read Dean's articles many years back when I was working for an Internet company and looking for interesting things to make my day go faster.

My interest in statistics is strong, but my math background is very limited. I am joining this forum to learn more.

I have a question. Lately, a lot of basketball fans I know seem to be in love with +/- statistics for a player. They claim that this tells us how valuable a player REALLY IS to their team. Because it shows how many points the team scores, and gives up, when said player is on the floor.

Am I missing something, or is this wildly simplistic? It would seem to be clear that there are so many other factors and complex relationships at play.

What don't I understand?

[radio]

I'm not sure you're missing anything. I like +/- to help figure out optimal starting 5s for teams with set rosters (and even there, they're fraught with selection errors and small sample sizes), but I don't know that many people that think the numbers are applicable across teams.
_________________
Ankur Desai
Amateur Hoops Junkie

[dogra]

A friend of mine was talking up Yao Ming's negative plus/minus rating. And I thought there could be an enormous number of reasons for that -- given the situation/current roster in Houston. And only one of those reasons was that Yao Ming, alone, was hurting the team.

Then I thought of the complexity of parsing out these different variables, and my head started spinning.

At which point, I began to think that these +/- ratings don't mean too much. That they're just an eyeball stat.

Maybe I'm overstating it, though.

[Golabki]

One problem with +/- is the fact that bench players tend to play with other bench players and starters tend to play with other starters. You can see this by glancing threw a few game flows (see "linkage" thread). To my knowledge there has been no effort to deal with this problem (although I could be wrong as I am new here). Without an adjustment +/- seems seriously flawed.

A second issue is that these numbers may have too much noise in them to be useful. Looking at raw point differential is probably is too blunt a tool to rate players in any meaningful way (this is me being totally subjective and very possibly wrong). I'll bet someone has done a study of this, but I don't know where.
_________________
Rock over London, Rock on Chicago.

[Mike G]

I too was disappointed in the seeming randomness of +/-. The biggest single factor seems to be Who plays in your place when you're out. If you back up Nowitzki, you better be damn good.

Forum member Dan R is the one person I'm aware of who has mathematically sorted thru all the combinations. It's called "regression", I think. There's fewer obvious departures from common sense; and when you do find one, he blames it on "noise".



Carry on.

[dogra]

If Dan R. could post or point to something on this subject, I'd be greatful.

What do you guys mean when you use the term "noise" in this [stats] context?

[dogra]

I also noted this quote from an article link posted by the Admin --

[HoopStudies]

[HoopStudies]

[Dan Rosenbaum]

The big advantage of adjusted plus/minus ratings are that they are the closest we can come to an "unbiased" measure of a player's effectiveness. By unbiasedness, I mean that if we could observe a player in an infinite number of games matched up with and against lots of combinations of players, adjusted plus/minus ratings would be a near perfect measure of a player's effectiveness.

Regular plus/minus statistics and any rating based upon traditional statistics is not unbiased. Even if we observed an infinite number of games these ratings would still be a good deal off from measuring a player's effectiveness.

So that means adjusted plus/minus ratings are the best rating system, right? Well, no, not necessarily. Plus/minus ratings, both adjusted and unadjusted, are very "noisy." What I mean by "noisy" is that if we measure a player's adjusted plus/minus ratings over two different 20 game stretches, there is a very good chance that the ratings will differ a lot - even if the player has not really gotten a lot better or worse. Ratings based upon traditional statistics would vary a lot less over these two 20 game stretches.

So adjusted plus/minus ratings are almost unbiased but have a high variance.

Ratings based upon traditional statistics (PER, TENDEX, nba.com efficiency) are biased but have a relatively small variance.

Regular plus/minus ratings are biased and have a high variance.

One way to measure the variance of a rating is to present its standard error. Suppose a player has a rating of 5.0 and a standard error of 3.5. For this player, the rating of 5.0 says that if we replaced 40 minutes of play by an average NBA player with the 5.0 player, the team would improve its performance by 5.0 points.

But standard errors are useful because about 95% of the time a "confidence interval" equal to 5.0 +/- 2*3.5 = (-2.0, 12.0) should include the "true" effectiveness of the player. In other words, we are 95% confident that this player is between 2.0 points per 40 minutes less and 12.0 points per 40 minutes more effective than an average NBA player.

So you see the standard error is pretty important. Suppose the standard error was 10.0, which is sometimes is for players that don't play much. In that case the 95% confidence interval would be (-15.0, 25.0), which is so huge that it includes pretty much any value that we might have thought was reasonable. What it tells us that we cannot really use these data to say anything useful about the effectiveness of this player.

Winston and Sagarin made a big deal about Mitchell Butler last year with Wizards, claiming he was the best player on the Wizards. But his 95% confidence interval was pretty large and basically said his effectiveness was somewhere between that of Kevin Garnett and that of a replacement player. In my opinion, such an estimate is pretty noisy and certainly not worthy of saying much about.

It takes work, but it is possible to compute standard errors for pretty much any rating. The standard errors for plus/minus ratings, both regular and adjusted, are quite large. Those for ratings based upon traditional statisitics tend to be much, much smaller. It generally takes about 2 to 4 times as many games for plus/minus-based ratings to have the same standard error as traditional statistics-based ratings.

Thus, in small samples the additional "noise" of adjusted plus/minus ratings may make them less useful than traditional-statistic-based ratings. It is the old bias versus variance tradeoff that is a recurring theme in statistics.

For this reason I have often argued that one needs to use a couple seasons to get reasonable results from plus/minus ratings. Also, I have tended to focus a lot of my time trying to see how traditional statistics translate into adjusted plus/minus statistics. I can then use these estimated relationships to devise a traditional statistic-based rating that is closer to being unbiased than other traditional statistic-based ratings. This hybrid ratings, IMO, combines the best of both worlds.

[gabefarkas]

what about Roland Ratings, comparing +/- on-court to that when the player is off-court? i'm definitely a big fan of those.

[Golabki]

Dan R.

It's clear that relating traditional stats to adj. +/- is a fairly obvious next step. When you say, "I have tended to focus a lot of my time" what do you mean exactly? That would be interesting to see.
_________________
Rock over London, Rock on Chicago.

[Dan Rosenbaum]

I would include Roland Ratings in the unadjusted plus/minus category - even though it does make a small adjustment from raw plus/minus ratings.

So far, I have talked about my results relating traditional statistics and adjusted plus/minus ratings in a variety of posts here and elsewhere. However, the most significant collection of comments on this are in my original piece on adjusted plus/minus ratings.

http://www.uncg.edu/bae/people/rosenbaum/NBA/winval2.htm