Joined: 25 Feb 2005 Posts: 10 Location: Brooklyn, NY
Posted: Fri Feb 25, 2005 5:11 pm Post subject: +/- Statistics: Golden Egg or Wooden Nickel?
Hi everyone. I am new to this forum. I used to read Dean's articles many years back when I was working for an Internet company and looking for interesting things to make my day go faster.
My interest in statistics is strong, but my math background is very limited. I am joining this forum to learn more.
I have a question. Lately, a lot of basketball fans I know seem to be in love with +/- statistics for a player. They claim that this tells us how valuable a player REALLY IS to their team. Because it shows how many points the team scores, and gives up, when said player is on the floor.
Am I missing something, or is this wildly simplistic? It would seem to be clear that there are so many other factors and complex relationships at play.
What don't I understand?
Last edited by dogra on Fri Feb 25, 2005 5:21 pm; edited 1 time in total
I'm not sure you're missing anything. I like +/- to help figure out optimal starting 5s for teams with set rosters (and even there, they're fraught with selection errors and small sample sizes), but I don't know that many people that think the numbers are applicable across teams. _________________ Ankur Desai
Amateur Hoops Junkie
Joined: 25 Feb 2005 Posts: 10 Location: Brooklyn, NY
Posted: Fri Feb 25, 2005 5:26 pm Post subject:
A friend of mine was talking up Yao Ming's negative plus/minus rating. And I thought there could be an enormous number of reasons for that -- given the situation/current roster in Houston. And only one of those reasons was that Yao Ming, alone, was hurting the team.
Then I thought of the complexity of parsing out these different variables, and my head started spinning.
At which point, I began to think that these +/- ratings don't mean too much. That they're just an eyeball stat.
Posted: Fri Feb 25, 2005 5:38 pm Post subject: problems with +/-
One problem with +/- is the fact that bench players tend to play with other bench players and starters tend to play with other starters. You can see this by glancing threw a few game flows (see "linkage" thread). To my knowledge there has been no effort to deal with this problem (although I could be wrong as I am new here). Without an adjustment +/- seems seriously flawed.
A second issue is that these numbers may have too much noise in them to be useful. Looking at raw point differential is probably is too blunt a tool to rate players in any meaningful way (this is me being totally subjective and very possibly wrong). I'll bet someone has done a study of this, but I don't know where. _________________ Rock over London, Rock on Chicago.
Last edited by Golabki on Fri Feb 25, 2005 5:41 pm; edited 1 time in total
Joined: 14 Jan 2005 Posts: 1521 Location: Delphi, Indiana
Posted: Fri Feb 25, 2005 5:40 pm Post subject:
I too was disappointed in the seeming randomness of +/-. The biggest single factor seems to be Who plays in your place when you're out. If you back up Nowitzki, you better be damn good.
Forum member Dan R is the one person I'm aware of who has mathematically sorted thru all the combinations. It's called "regression", I think. There's fewer obvious departures from common sense; and when you do find one, he blames it on "noise".
Joined: 25 Feb 2005 Posts: 10 Location: Brooklyn, NY
Posted: Fri Feb 25, 2005 5:49 pm Post subject:
I also noted this quote from an article link posted by the Admin --
Quote:
In Seattle and around the NBA, statistics haven't made the same kind of impact on the sidelines as they have in front offices. That's to be expected. Coaching ultimately comes down to feel, and, unlike general managers, coaches need only evaluate players they watch on a daily basis, not hundreds throughout the NBA and countless more amateur prospects.
Still, there are two primary ways in which the Sonics use statistics: Evaluating their own players and lineup combinations with plus-minus statistics, and scouting upcoming opponents by looking at their statistics.
Joined: 30 Dec 2004 Posts: 534 Location: Near Philadelphia, PA
Posted: Fri Feb 25, 2005 6:18 pm Post subject:
Mike G wrote:
I too was disappointed in the seeming randomness of +/-. The biggest single factor seems to be Who plays in your place when you're out. If you back up Nowitzki, you better be damn good.
Forum member Dan R is the one person I'm aware of who has mathematically sorted thru all the combinations. It's called "regression", I think. There's fewer obvious departures from common sense; and when you do find one, he blames it on "noise".
I explain the concept in Basketball on Paper, too.
What it does is separate out the fact that starters tend to play against better guys, so if you're a starter but get moved to the bench, you could benefit in raw +/- by playing against weaker guys and by replacing a generally worse guy on your own team. Dan's method accounts for all of this.
But yes, there is noise. Noise means that Dan comes out with estimates of how far off he could be from "truth" (I hate using the word). So his estimate of one guy's value may be +6 points with an error of 8. That would mean that the guy could be adding 0 points or adding a lot more than +6. You see larger errors for players who don't play much or who play only with a small set of players.
Noise happens. People aren't consistent. Nor will numbers representing them be consistent. No one should make a decision based on a single number.
Raw +/- numbers are interesting, but again just one number. On their own, they're just an indicator. As with almost everything we do here, the story is more complex than one number can represent. I think we do a good job here of balancing the story by looking at several different numbers. _________________ Dean Oliver
Author, Basketball on Paper
http://www.basketballonpaper.com
Joined: 30 Dec 2004 Posts: 534 Location: Near Philadelphia, PA
Posted: Fri Feb 25, 2005 6:27 pm Post subject:
dogra wrote:
I also noted this quote from an article link posted by the Admin --
Quote:
Still, there are two primary ways in which the Sonics use statistics: Evaluating their own players and lineup combinations with plus-minus statistics, and scouting upcoming opponents by looking at their statistics.
[emphasis mine]
I would hope these are adjusted plus/minus stats.
Otherwise I'm confused again.
Not adjusted, just in context. Seattle uses +/- numbers as indicators, as part of the full story. As Nate said, he didn't get a big bonus for having the highest +/- when he was a player. It's just an indicator. Decisions (such as playing time) are much more complex than looking at the #, as he says in the article.
Do you hire people based solely on IQ? Or just on SAT/GRE score? No. Real life, which to the surprise of some people includes basketball, means decisions based on a broad variety of indicators. In Seattle, we actually choose to use those indicators, whereas the NBA as a whole has historically not done much with them (beyond ppg, rpg, apg). It's changing, though. We here in this discussion group are making a difference. _________________ Dean Oliver
Author, Basketball on Paper
http://www.basketballonpaper.com
Joined: 03 Jan 2005 Posts: 497 Location: Greensboro, North Carolina
Posted: Sat Feb 26, 2005 12:29 am Post subject:
The big advantage of adjusted plus/minus ratings are that they are the closest we can come to an "unbiased" measure of a player's effectiveness. By unbiasedness, I mean that if we could observe a player in an infinite number of games matched up with and against lots of combinations of players, adjusted plus/minus ratings would be a near perfect measure of a player's effectiveness.
Regular plus/minus statistics and any rating based upon traditional statistics is not unbiased. Even if we observed an infinite number of games these ratings would still be a good deal off from measuring a player's effectiveness.
So that means adjusted plus/minus ratings are the best rating system, right? Well, no, not necessarily. Plus/minus ratings, both adjusted and unadjusted, are very "noisy." What I mean by "noisy" is that if we measure a player's adjusted plus/minus ratings over two different 20 game stretches, there is a very good chance that the ratings will differ a lot - even if the player has not really gotten a lot better or worse. Ratings based upon traditional statistics would vary a lot less over these two 20 game stretches.
So adjusted plus/minus ratings are almost unbiased but have a high variance.
Ratings based upon traditional statistics (PER, TENDEX, nba.com efficiency) are biased but have a relatively small variance.
Regular plus/minus ratings are biased and have a high variance.
One way to measure the variance of a rating is to present its standard error. Suppose a player has a rating of 5.0 and a standard error of 3.5. For this player, the rating of 5.0 says that if we replaced 40 minutes of play by an average NBA player with the 5.0 player, the team would improve its performance by 5.0 points.
But standard errors are useful because about 95% of the time a "confidence interval" equal to 5.0 +/- 2*3.5 = (-2.0, 12.0) should include the "true" effectiveness of the player. In other words, we are 95% confident that this player is between 2.0 points per 40 minutes less and 12.0 points per 40 minutes more effective than an average NBA player.
So you see the standard error is pretty important. Suppose the standard error was 10.0, which is sometimes is for players that don't play much. In that case the 95% confidence interval would be (-15.0, 25.0), which is so huge that it includes pretty much any value that we might have thought was reasonable. What it tells us that we cannot really use these data to say anything useful about the effectiveness of this player.
Winston and Sagarin made a big deal about Mitchell Butler last year with Wizards, claiming he was the best player on the Wizards. But his 95% confidence interval was pretty large and basically said his effectiveness was somewhere between that of Kevin Garnett and that of a replacement player. In my opinion, such an estimate is pretty noisy and certainly not worthy of saying much about.
It takes work, but it is possible to compute standard errors for pretty much any rating. The standard errors for plus/minus ratings, both regular and adjusted, are quite large. Those for ratings based upon traditional statisitics tend to be much, much smaller. It generally takes about 2 to 4 times as many games for plus/minus-based ratings to have the same standard error as traditional statistics-based ratings.
Thus, in small samples the additional "noise" of adjusted plus/minus ratings may make them less useful than traditional-statistic-based ratings. It is the old bias versus variance tradeoff that is a recurring theme in statistics.
For this reason I have often argued that one needs to use a couple seasons to get reasonable results from plus/minus ratings. Also, I have tended to focus a lot of my time trying to see how traditional statistics translate into adjusted plus/minus statistics. I can then use these estimated relationships to devise a traditional statistic-based rating that is closer to being unbiased than other traditional statistic-based ratings. This hybrid ratings, IMO, combines the best of both worlds.
It's clear that relating traditional stats to adj. +/- is a fairly obvious next step. When you say, "I have tended to focus a lot of my time" what do you mean exactly? That would be interesting to see. _________________ Rock over London, Rock on Chicago.
Joined: 03 Jan 2005 Posts: 497 Location: Greensboro, North Carolina
Posted: Sun Feb 27, 2005 5:15 pm Post subject:
I would include Roland Ratings in the unadjusted plus/minus category - even though it does make a small adjustment from raw plus/minus ratings.
So far, I have talked about my results relating traditional statistics and adjusted plus/minus ratings in a variety of posts here and elsewhere. However, the most significant collection of comments on this are in my original piece on adjusted plus/minus ratings.
You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum