APBRmetrics Forum Index APBRmetrics
The statistical revolution will not be televised.
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

+/- Statistics: Golden Egg or Wooden Nickel?

 
Post new topic   Reply to topic    APBRmetrics Forum Index -> General discussion
View previous topic :: View next topic  
Author Message
dogra



Joined: 25 Feb 2005
Posts: 10
Location: Brooklyn, NY

PostPosted: Fri Feb 25, 2005 5:11 pm    Post subject: +/- Statistics: Golden Egg or Wooden Nickel? Reply with quote

Hi everyone. I am new to this forum. I used to read Dean's articles many years back when I was working for an Internet company and looking for interesting things to make my day go faster.

My interest in statistics is strong, but my math background is very limited. I am joining this forum to learn more.


I have a question. Lately, a lot of basketball fans I know seem to be in love with +/- statistics for a player. They claim that this tells us how valuable a player REALLY IS to their team. Because it shows how many points the team scores, and gives up, when said player is on the floor.

Am I missing something, or is this wildly simplistic? It would seem to be clear that there are so many other factors and complex relationships at play.

What don't I understand?


Last edited by dogra on Fri Feb 25, 2005 5:21 pm; edited 1 time in total
Back to top
View user's profile Send private message Visit poster's website
radio



Joined: 17 Feb 2005
Posts: 13
Location: New York

PostPosted: Fri Feb 25, 2005 5:18 pm    Post subject: Reply with quote

I'm not sure you're missing anything. I like +/- to help figure out optimal starting 5s for teams with set rosters (and even there, they're fraught with selection errors and small sample sizes), but I don't know that many people that think the numbers are applicable across teams.
_________________
Ankur Desai
Amateur Hoops Junkie
Back to top
View user's profile Send private message
dogra



Joined: 25 Feb 2005
Posts: 10
Location: Brooklyn, NY

PostPosted: Fri Feb 25, 2005 5:26 pm    Post subject: Reply with quote

A friend of mine was talking up Yao Ming's negative plus/minus rating. And I thought there could be an enormous number of reasons for that -- given the situation/current roster in Houston. And only one of those reasons was that Yao Ming, alone, was hurting the team.

Then I thought of the complexity of parsing out these different variables, and my head started spinning.

At which point, I began to think that these +/- ratings don't mean too much. That they're just an eyeball stat.

Maybe I'm overstating it, though.
Back to top
View user's profile Send private message Visit poster's website
Golabki



Joined: 25 Feb 2005
Posts: 7
Location: Boston

PostPosted: Fri Feb 25, 2005 5:38 pm    Post subject: problems with +/- Reply with quote

One problem with +/- is the fact that bench players tend to play with other bench players and starters tend to play with other starters. You can see this by glancing threw a few game flows (see "linkage" thread). To my knowledge there has been no effort to deal with this problem (although I could be wrong as I am new here). Without an adjustment +/- seems seriously flawed.

A second issue is that these numbers may have too much noise in them to be useful. Looking at raw point differential is probably is too blunt a tool to rate players in any meaningful way (this is me being totally subjective and very possibly wrong). I'll bet someone has done a study of this, but I don't know where.
_________________
Rock over London, Rock on Chicago.


Last edited by Golabki on Fri Feb 25, 2005 5:41 pm; edited 1 time in total
Back to top
View user's profile Send private message
Mike G



Joined: 14 Jan 2005
Posts: 1521
Location: Delphi, Indiana

PostPosted: Fri Feb 25, 2005 5:40 pm    Post subject: Reply with quote

I too was disappointed in the seeming randomness of +/-. The biggest single factor seems to be Who plays in your place when you're out. If you back up Nowitzki, you better be damn good.

Forum member Dan R is the one person I'm aware of who has mathematically sorted thru all the combinations. It's called "regression", I think. There's fewer obvious departures from common sense; and when you do find one, he blames it on "noise".

Confused

Carry on.
Back to top
View user's profile Send private message Send e-mail
dogra



Joined: 25 Feb 2005
Posts: 10
Location: Brooklyn, NY

PostPosted: Fri Feb 25, 2005 5:44 pm    Post subject: Reply with quote

If Dan R. could post or point to something on this subject, I'd be greatful.

What do you guys mean when you use the term "noise" in this [stats] context?
Back to top
View user's profile Send private message Visit poster's website
dogra



Joined: 25 Feb 2005
Posts: 10
Location: Brooklyn, NY

PostPosted: Fri Feb 25, 2005 5:49 pm    Post subject: Reply with quote

I also noted this quote from an article link posted by the Admin --

Quote:
In Seattle and around the NBA, statistics haven't made the same kind of impact on the sidelines as they have in front offices. That's to be expected. Coaching ultimately comes down to feel, and, unlike general managers, coaches need only evaluate players they watch on a daily basis, not hundreds throughout the NBA and countless more amateur prospects.

Still, there are two primary ways in which the Sonics use statistics: Evaluating their own players and lineup combinations with plus-minus statistics, and scouting upcoming opponents by looking at their statistics.

[emphasis mine]

I would hope these are adjusted plus/minus stats.

Otherwise I'm confused again.
Back to top
View user's profile Send private message Visit poster's website
HoopStudies



Joined: 30 Dec 2004
Posts: 534
Location: Near Philadelphia, PA

PostPosted: Fri Feb 25, 2005 6:18 pm    Post subject: Reply with quote

Mike G wrote:
I too was disappointed in the seeming randomness of +/-. The biggest single factor seems to be Who plays in your place when you're out. If you back up Nowitzki, you better be damn good.

Forum member Dan R is the one person I'm aware of who has mathematically sorted thru all the combinations. It's called "regression", I think. There's fewer obvious departures from common sense; and when you do find one, he blames it on "noise".

Confused

Carry on.


Dan's base column is here:

http://www.uncg.edu/bae/people/rosenbaum/NBA/stats1.htm

I explain the concept in Basketball on Paper, too.

What it does is separate out the fact that starters tend to play against better guys, so if you're a starter but get moved to the bench, you could benefit in raw +/- by playing against weaker guys and by replacing a generally worse guy on your own team. Dan's method accounts for all of this.

But yes, there is noise. Noise means that Dan comes out with estimates of how far off he could be from "truth" (I hate using the word). So his estimate of one guy's value may be +6 points with an error of 8. That would mean that the guy could be adding 0 points or adding a lot more than +6. You see larger errors for players who don't play much or who play only with a small set of players.

Noise happens. People aren't consistent. Nor will numbers representing them be consistent. No one should make a decision based on a single number.

Raw +/- numbers are interesting, but again just one number. On their own, they're just an indicator. As with almost everything we do here, the story is more complex than one number can represent. I think we do a good job here of balancing the story by looking at several different numbers.
_________________
Dean Oliver
Author, Basketball on Paper
http://www.basketballonpaper.com
Back to top
View user's profile Send private message Visit poster's website
HoopStudies



Joined: 30 Dec 2004
Posts: 534
Location: Near Philadelphia, PA

PostPosted: Fri Feb 25, 2005 6:27 pm    Post subject: Reply with quote

dogra wrote:
I also noted this quote from an article link posted by the Admin --

Quote:
Still, there are two primary ways in which the Sonics use statistics: Evaluating their own players and lineup combinations with plus-minus statistics, and scouting upcoming opponents by looking at their statistics.

[emphasis mine]

I would hope these are adjusted plus/minus stats.

Otherwise I'm confused again.


Not adjusted, just in context. Seattle uses +/- numbers as indicators, as part of the full story. As Nate said, he didn't get a big bonus for having the highest +/- when he was a player. It's just an indicator. Decisions (such as playing time) are much more complex than looking at the #, as he says in the article.

Do you hire people based solely on IQ? Or just on SAT/GRE score? No. Real life, which to the surprise of some people includes basketball, means decisions based on a broad variety of indicators. In Seattle, we actually choose to use those indicators, whereas the NBA as a whole has historically not done much with them (beyond ppg, rpg, apg). It's changing, though. We here in this discussion group are making a difference.
_________________
Dean Oliver
Author, Basketball on Paper
http://www.basketballonpaper.com
Back to top
View user's profile Send private message Visit poster's website
Dan Rosenbaum



Joined: 03 Jan 2005
Posts: 497
Location: Greensboro, North Carolina

PostPosted: Sat Feb 26, 2005 12:29 am    Post subject: Reply with quote

The big advantage of adjusted plus/minus ratings are that they are the closest we can come to an "unbiased" measure of a player's effectiveness. By unbiasedness, I mean that if we could observe a player in an infinite number of games matched up with and against lots of combinations of players, adjusted plus/minus ratings would be a near perfect measure of a player's effectiveness.

Regular plus/minus statistics and any rating based upon traditional statistics is not unbiased. Even if we observed an infinite number of games these ratings would still be a good deal off from measuring a player's effectiveness.

So that means adjusted plus/minus ratings are the best rating system, right? Well, no, not necessarily. Plus/minus ratings, both adjusted and unadjusted, are very "noisy." What I mean by "noisy" is that if we measure a player's adjusted plus/minus ratings over two different 20 game stretches, there is a very good chance that the ratings will differ a lot - even if the player has not really gotten a lot better or worse. Ratings based upon traditional statistics would vary a lot less over these two 20 game stretches.

So adjusted plus/minus ratings are almost unbiased but have a high variance.

Ratings based upon traditional statistics (PER, TENDEX, nba.com efficiency) are biased but have a relatively small variance.

Regular plus/minus ratings are biased and have a high variance.

One way to measure the variance of a rating is to present its standard error. Suppose a player has a rating of 5.0 and a standard error of 3.5. For this player, the rating of 5.0 says that if we replaced 40 minutes of play by an average NBA player with the 5.0 player, the team would improve its performance by 5.0 points.

But standard errors are useful because about 95% of the time a "confidence interval" equal to 5.0 +/- 2*3.5 = (-2.0, 12.0) should include the "true" effectiveness of the player. In other words, we are 95% confident that this player is between 2.0 points per 40 minutes less and 12.0 points per 40 minutes more effective than an average NBA player.

So you see the standard error is pretty important. Suppose the standard error was 10.0, which is sometimes is for players that don't play much. In that case the 95% confidence interval would be (-15.0, 25.0), which is so huge that it includes pretty much any value that we might have thought was reasonable. What it tells us that we cannot really use these data to say anything useful about the effectiveness of this player.

Winston and Sagarin made a big deal about Mitchell Butler last year with Wizards, claiming he was the best player on the Wizards. But his 95% confidence interval was pretty large and basically said his effectiveness was somewhere between that of Kevin Garnett and that of a replacement player. In my opinion, such an estimate is pretty noisy and certainly not worthy of saying much about.

It takes work, but it is possible to compute standard errors for pretty much any rating. The standard errors for plus/minus ratings, both regular and adjusted, are quite large. Those for ratings based upon traditional statisitics tend to be much, much smaller. It generally takes about 2 to 4 times as many games for plus/minus-based ratings to have the same standard error as traditional statistics-based ratings.

Thus, in small samples the additional "noise" of adjusted plus/minus ratings may make them less useful than traditional-statistic-based ratings. It is the old bias versus variance tradeoff that is a recurring theme in statistics.

For this reason I have often argued that one needs to use a couple seasons to get reasonable results from plus/minus ratings. Also, I have tended to focus a lot of my time trying to see how traditional statistics translate into adjusted plus/minus statistics. I can then use these estimated relationships to devise a traditional statistic-based rating that is closer to being unbiased than other traditional statistic-based ratings. This hybrid ratings, IMO, combines the best of both worlds.
Back to top
View user's profile Send private message Send e-mail Visit poster's website Yahoo Messenger
gabefarkas



Joined: 31 Dec 2004
Posts: 881
Location: Durham, NC

PostPosted: Sun Feb 27, 2005 12:16 am    Post subject: Reply with quote

what about Roland Ratings, comparing +/- on-court to that when the player is off-court? i'm definitely a big fan of those.
Back to top
View user's profile Send private message Send e-mail AIM Address
Golabki



Joined: 25 Feb 2005
Posts: 7
Location: Boston

PostPosted: Sun Feb 27, 2005 1:20 pm    Post subject: Reply with quote

Dan R.

It's clear that relating traditional stats to adj. +/- is a fairly obvious next step. When you say, "I have tended to focus a lot of my time" what do you mean exactly? That would be interesting to see.
_________________
Rock over London, Rock on Chicago.
Back to top
View user's profile Send private message
Dan Rosenbaum



Joined: 03 Jan 2005
Posts: 497
Location: Greensboro, North Carolina

PostPosted: Sun Feb 27, 2005 5:15 pm    Post subject: Reply with quote

I would include Roland Ratings in the unadjusted plus/minus category - even though it does make a small adjustment from raw plus/minus ratings.

So far, I have talked about my results relating traditional statistics and adjusted plus/minus ratings in a variety of posts here and elsewhere. However, the most significant collection of comments on this are in my original piece on adjusted plus/minus ratings.

http://www.uncg.edu/bae/people/rosenbaum/NBA/winval2.htm
Back to top
View user's profile Send private message Send e-mail Visit poster's website Yahoo Messenger
Display posts from previous:   
Post new topic   Reply to topic    APBRmetrics Forum Index -> General discussion All times are GMT - 5 Hours
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Powered by phpBB © 2001, 2005 phpBB Group