|
APBRmetrics The statistical revolution will not be televised.
|
View previous topic :: View next topic |
Author |
Message |
deepak
Joined: 26 Apr 2006 Posts: 664
|
Posted: Sat Jun 10, 2006 2:25 pm Post subject: Berri: NBA's Secret Superstars (NYT) |
|
|
http://www.nytimes.com/2006/06/10/opinion/10berri.html?_r=1&oref=slogin
Thoughts?
Also, could someone who's knowledgable about his methods also explain how/why his results differ from individual Net Wins via Offensive and Defensive Ratings. Are they trying to answer the same question, or are they actually describing two different concepts?
Thanks. |
|
Back to top |
|
|
Dan Rosenbaum
Joined: 03 Jan 2005 Posts: 541 Location: Greensboro, North Carolina
|
Posted: Sat Jun 10, 2006 11:16 pm Post subject: |
|
|
[Let me rewrite some of this to add clarity.]
I have not yet picked up the book, but their "simple" formula is the following.
Win Score = PTS + REB + STL - FGA - TO + 0.5*(AST + BLK - FTA - PF)
They also have a simpler formula that they use in several of their academic papers.
Old Win Score = PTS + REB + STL - FGA - TO - 0.5*FTA
As far as I can tell, all of the stats are per-minute (or per-48 minute or per-40 minute) measures. I don't know if they are pace-adjusted.
But Win Score is a simplified version of Wins Produced. And here is the difference between the two.
http://dberri.wordpress.com/2006/05/21/simple-models-of-player-performance
Quote: | To get at Wins Produced you have to use the exact values, and make a few adjustments – such as adjusting for position played – which we note in the book. Still, Win Score is sufficient to give you a quick snapshot of a player’s performance. And it is especially useful if you wish to know if a player is playing better or worse than he did before. |
By the "exact values" they mean they are using multipliers different from 1, -1, 0.5, and -0.5. I would imagine this helps, but in their academic papers with their Old Win Score metric, they suggested that using the exact regression parameters did not matter much. Also they write the following.
Quote: | To keep it simple, one can argue that each of these latter stats is worth ½ a point, rebound, etc… Now the ½ value is not exact, but using ½ keeps it simple and we find one gains very little using the exact relative value. In other words, player rankings do not change very much when you use the exact values. |
Making adjustments by position could be more helpful, but I would be suprised if it helped a lot. And by arguing that Win Score does a pretty good job, they in essence argue something similar.
So what have I done with these two metrics?
I have related these two metrics to my adjusted plus/minus ratings, which in theory capture practically everything of value that a player does while he is on the floor. It may not be a perfect metric to measure future productivity, but to gauge the usefulness of metrics like these, it is probably ideal.
So I run a regression with my adjusted plus/minus measure as the dependent variable and various metrics as the independent variable. I report the R-squared values from these regressions. R-squared measures the fraction of the total variation explained by the regression. A value of 0 means that the regression explained nothing and a value of 1 means the regression explained everything. Here are my results.
Win Score - R-squared = 0.2009
Old Win Score - R-squared = 0.0843
NBA Efficiency - R-squared = 0.2209
Putting in each of the variables in the Berri1 formula in the regression model separately - Rsquared = 0.3740
My statistical plus/minus rating - R-squared > 0.45
This may not be the best way to gauge which metric is best, but I cannot think of a better way. And so my results suggest that the Win Score is a little worse than the NBA efficiency metric. (It is a big improvement, however, over the metric they use in their academic papers. That metric explains almost nothing.)
So, according to these results, there is not a lot to recommend the Win Score as a big improvement over the metrics that folks here use.
Now again Wins Produced is a different animal but from what I can gather, I suspect it might be a little better than NBA efficiency but that's about it. If it was A LOT better than Win Score, I would think the authors would be hesitant to ever use Win Score.
But I suspect it is still worthwhile to read the book. Many of the points they make are still valid - even if they are not using an ideal metric. Also, they touch on a lot of topics that get little play here (such as how salaries relate to stats), and I suspect folks might find that interesting.
Last edited by Dan Rosenbaum on Sun Jun 11, 2006 5:49 pm; edited 4 times in total |
|
Back to top |
|
|
edijorj
Joined: 21 Apr 2006 Posts: 30
|
Posted: Sun Jun 11, 2006 12:47 am Post subject: |
|
|
Dan Rosenbaum,
I havn't read the book either (and I probably won't), but I think Berri1 is what they call Win Score, not Wins Produced. Win Score is supposedly similar to Wins Produced if player position is accounted for. (Berri apparently realizes that his metric is biased against guards.) So it would make more sense if you added some dummy variables for player position.
It seems like the problem with Berri's analysis is that he believes that his metric is good if it explain team wins. To give an extreme example, the equation Y=1.78*FG-1.49*FGA+1.78*Th-0.57*ThA+0.51*FT-0.41*FTA+1.46*Orb+1.41*Drb+0.06*ast+1.76*stl+0.13*blk-1.52*to+0.13*pf explains team wins very well, but is clearly wrong for evaluating a player. |
|
Back to top |
|
|
deepak
Joined: 26 Apr 2006 Posts: 664
|
Posted: Sun Jun 11, 2006 10:22 am Post subject: |
|
|
Dan Rosenbaum wrote: | I have not yet picked up the book, but I think their formula is the following.
Berri1 = PTS + REB + STL - FGA - TO + 0.5*(AST + BLK - FTA - PF)
They also have a simpler formula that they use in several of their academic papers.
Berri2 = PTS + REB + STL - FGA - TO - 0.5*FTA
As far as I can tell, all of the stats are per-minute (or per-48 minute or per-40 minute) measures. I don't know if they are pace-adjusted.
|
Is their metric really that simple? For all the press the book is getting, somehow I thought it would be a tad more sophisticated than that.
They describe their approach in this way:
Quote: |
The approach we took in The Wages of Wins is simply to utilize regression analysis – a common technique in economics – to determine the relative impact of each statistics on team wins. We had three objectives in constructing our model. Ultimately we wanted a measure that was simple, complete, and accurate. In the end, we think each of these objectives was met. The Wins Produced model is not hard to understand, it incorporates each of the statistics tracked for individual players, and it connects accurately to team wins.
|
From here. they claim:
Quote: |
In particular, they argue, traditional talent evaluation over-rates the importance of points scored, and under-rates the importance of turnovers, rebounds and scoring percentage. Wages of Wins also obliterates the so-called NBA Efficiency rating, which is the official algorithm used by the league and many basketball experts to rank the statistical performance of players. The Efficiency rating, they argue, makes the same error. It dramatically over-rewards players who take lots and lots of shots.”
...
“Okay: part two. Is the Wages of Wins algorithm an improvement over the things like the NBA Efficiency system? To make the case for their system, the authors “fit” their algorithm to the real world. For the 2003-04 season, they add up the number of wins predicted by their algorithm for every player on every team, and compare that number to the team’s actual win total. Their average error? 1.67 wins. In other words, if you give them the statistics for every player on a given team, they can tell you how many wins that team got that season, with a margin of error under two wins. That’s pretty good.”
|
So, apparently, they believe their metric is much more accurate than NBA efficiency.
Considering that PER has been become much more publicly well known of late (through ESPN, 82games, here, etc.), I wonder why they don't make any mention of it? |
|
Back to top |
|
|
Kevin Pelton Site Admin
Joined: 30 Dec 2004 Posts: 978 Location: Seattle
|
Posted: Sun Jun 11, 2006 11:48 am Post subject: |
|
|
deepak_e wrote: | Is their metric really that simple? For all the press the book is getting, somehow I thought it would be a tad more sophisticated than that. |
No. As edijorj said in the previous post, that is Win Score, as opposed to the more detailed Wins Produced. |
|
Back to top |
|
|
Mark
Joined: 20 Aug 2005 Posts: 807
|
Posted: Sun Jun 11, 2006 11:51 am Post subject: |
|
|
We can't compute error on individual player level but it seems like there could be higher levels of player error that cancel each other out. Error reported at team level really just measures error for formula for combined team level stats doesnt it? Several other make this same case in this thread http://gladwell.typepad.com/gladwellcom/2006/05/basketball_by_t.html
This would apply to the simple version of Berri, would it to the regression-based version too?
I noticed that 3 of the 10 most underrated were Memphis Grizzlies. http://gladwell.typepad.com/gladwellcom/2006/05/basketball_by_t.html
Only one brought in by Jerry West but the presence of the other two may have been some attraction to him and he did give them 6 year extensions. But they can't get out of the first round even with Gasol . Even win a playoff game over last three years. I thought the bench was supposed to have been a strength. Is it the coaching? Is it the level of "clutchness"? (Dan gave clutch time extra weight and drew an apparent criticism for doing so from edijori earlier viewtopic.php?p=8403&highlight=#8403, I tend to support a clutchtime weight but not sure how much. I mention in case either or others wish to expand of whether clutch weights should or should not be in best practice methods.) Just the toughness of the opponents, not getting a higher playoff seed? But why not higher seeding or better playoff performance? Is it that they are strong on other things that make them appear underrated but not high enough as a group on the "overrated" category of scoring?
I wonder how different the ratings would be if the weights were selected based on team playoff performance instead of regular season.
This year's Knicks had for a time 4 of the bottom 10 "difference makers" on this 2004 list from Dan http://www.82games.com/comm29.htm
Single examples don't validate or demonish a formula, I just noticed these cases. I believe the two differnet systems produce ratings for Kobe Bryant and Ray Allen that are close to each other. But that is surface level result similarity only and folks will write in different explanations of it.
Last edited by Mark on Mon Jun 12, 2006 7:13 am; edited 8 times in total |
|
Back to top |
|
|
deepak
Joined: 26 Apr 2006 Posts: 664
|
Posted: Sun Jun 11, 2006 12:17 pm Post subject: |
|
|
More explanation on Win Score and Wins Produced
Win Score is a simple, approximate formula that can be used to quickly assess how good a game a player had. I supposed it's analogous to John Hollinger's game scores -- though even simpler to calculate.
Also, I wonder what their rationale is in adjusting for positions, beyond making their list more credible.
Here are the top 20 players, in terms of Win Score per 48 minutes:
Code: |
Rank Player WinScore/48min
1 garnett,kevin 19.7
2 marion,shawn 18.8
3 camby,marcus 18.5
4 wallace,ben 17.5
5 hayes,chuck 16.9
6 foster,jeff 16.3
7 brand,elton 15.8
8 ming,yao 15.5
9 nowitzki,dirk 15.4
10 howard,dwight 15.3
11 duncan,tim 15.2
12 wallace,gerald 14.6
13 boozer,carlos 14.5
14 o'neal,shaq 14.2
15 mutombo,dikembe 14.2
16 gooden,drew 14.0
17 mourning,alonzo 14.0
18 chandler,tyson 13.9
19 evans,reggie 13.7
20 kidd,jason 13.7
|
I love Chuck Hayes, but number 5? And no LeBron anywhere.
BTW, I just noticed that this book was being discussed in a thread down below. Sorry about making a duplicate. |
|
Back to top |
|
|
Mark
Joined: 20 Aug 2005 Posts: 807
|
Posted: Sun Jun 11, 2006 1:08 pm Post subject: |
|
|
18 big men and 2 perimeter by this method? I'm all for giving full deserved credit for big men but is that the main discovery?
Last edited by Mark on Wed Jun 28, 2006 12:42 am; edited 4 times in total |
|
Back to top |
|
|
bchaikin
Joined: 27 Jan 2005 Posts: 685 Location: cleveland, ohio
|
Posted: Sun Jun 11, 2006 1:23 pm Post subject: |
|
|
I love Chuck Hayes, but number 5?
what chuck hayes produced statistically in just 535 minutes in 05-06 was excellent. i can easily see why a rating system (not just this one) would have him rated very high....
just looking at his numbers (and not noting that he may have accrued them solely against other teams' reserves and not their starters, which may or may not be true but if true could make a world of difference), if you multiplied his stats by 5 so instead of 535 minutes he played 2675 minutes he'd have scored 18.5 pts/g on a ScFG% of over 56% with 11 rebs/g playing 33 min/g - with 130 steals, 70 bs, and just 60 turnovers (uh... and also 380 fouls committed)....
would he do this if giving the playing time? who knows? but his stats in those 500+ minutes were outrageously good, kind of like anderson varejao in 04-05... |
|
Back to top |
|
|
HoopStudies
Joined: 30 Dec 2004 Posts: 705 Location: Near Philadelphia, PA
|
Posted: Sun Jun 11, 2006 4:23 pm Post subject: |
|
|
Dan Rosenbaum wrote: |
But I suspect it is still worthwhile to read the book. Many of the points they make are still valid - even if they are not using an ideal metric. Also, they touch on a lot of topics that get little play here (such as how salaries relate to stats), and I suspect folks might find that interesting. |
I just got the book and read some. Being pretty familiar with sports economics literature, I knew of many of these things, but I have been finding them interesting for years, which is why I'm familiar. If you do like issues on salaries vs winning, competitive balance, impacts of labor strikes on attendance, the value of having a star on your team -- you'll find it interesting.
Or if you want to hear him call me a Marxist, that's also in there.
The NBA GMs do seem to be reading it, as well, as I heard mention of it from a few people down in Orlando this week.
(It's not really my job to do so, but I will say that the method in this book is definitely NOT the one that he used in the 1999 paper. I walked him through a few things that caused him to change it pretty significantly. As to whether it's "good" or "bad" -- it's a rating system, there are a million of them, and the distinctive aspect of this one is that it penalizes poor shooters more. Dan says it's not necessarily better or worse than the NBA efficiency -- well, that's what most of them are.) _________________ Dean Oliver
Author, Basketball on Paper
The postings are my own & don't necess represent positions, strategies or opinions of employers. |
|
Back to top |
|
|
Mark
Joined: 20 Aug 2005 Posts: 807
|
Posted: Sun Jun 11, 2006 5:52 pm Post subject: |
|
|
HoopStudies wrote: |
Or if you want to hear him call me a Marxist, that's also in there.
|
Since you brought it up, I am curious on what grounds or in what context does he say this and what is your response? |
|
Back to top |
|
|
Mark
Joined: 20 Aug 2005 Posts: 807
|
Posted: Sun Jun 11, 2006 5:57 pm Post subject: Another question |
|
|
edijorj, what is the source of the linear equation in your last post? How good is the fit? |
|
Back to top |
|
|
HoopStudies
Joined: 30 Dec 2004 Posts: 705 Location: Near Philadelphia, PA
|
Posted: Sun Jun 11, 2006 6:42 pm Post subject: |
|
|
Mark wrote: | HoopStudies wrote: |
Or if you want to hear him call me a Marxist, that's also in there.
|
Since you brought it up, I am curious on what grounds or in what context does he say this and what is your response? |
Apparently, the concept of assigning credit based on difficulty of accomplishing a portion of a task is Marxian. _________________ Dean Oliver
Author, Basketball on Paper
The postings are my own & don't necess represent positions, strategies or opinions of employers. |
|
Back to top |
|
|
Ed Küpfer
Joined: 30 Dec 2004 Posts: 785 Location: Toronto
|
Posted: Sun Jun 11, 2006 6:51 pm Post subject: |
|
|
HoopStudies wrote: | Apparently, the concept of assigning credit based on difficulty of accomplishing a portion of a task is Marxian. |
It had nothing to do with your balaclava and Che Guevara underoos? _________________ ed |
|
Back to top |
|
|
Mark
Joined: 20 Aug 2005 Posts: 807
|
Posted: Sun Jun 11, 2006 7:40 pm Post subject: |
|
|
HoopStudies wrote:
"Apparently, the concept of assigning credit based on difficulty of accomplishing a portion of a task is Marxian."
I'd have to hear more of that explanation to understand what was meant. Maybe you could say that at a level of technical Marxian economic analysis, of value accounting- stressing the importance of the labor input.
But at the simplest philosophical level communism is "from each according to their abilities, to each according to their needs" and that seems contrary to the practice in the quote. I would think a classical political Marxist would likely have a VOP attitude of equal worth of different activities that produce equivalent results to me. And would probably be against moneyball and a skewed freemarket bidding war distribution of salary heavily to the top performers. The quote sounds like it has more in common with a capitalist attitude than a political Marxist one to me.
But I guess economic Marxian economic analysis can be considered somewhat distinct from that and it was initially applied to understanding capitalism, how the system actually works according to the existing rules and market values of different inputs in producing outputs-wins.
Berri is giving all rebounds defensive and offensive equal weight (at least in the simple version)? And you dont view the world that way. Is that part of what makes you Marxian to him? Hmmm... Not sure the label is particularly useful if that is the issue, though this debate is important. I guess if difficult, perhaps especially physical labor (like off. rebounding, defense, blocked shots) was seen as deserving more credit than shooters (and in some systems it is) that might perhaps be called Maoist Gang of 4 peasants vs the elite rhetoric.
Team +/- ratings for players could certainly seem pure communist but for the opposite rationale than the one asserted in the quote. Everybody gets the same score for that time on court together! Does he pull away from that by just using individual boxscore data, not give stress to contextual data, opposed that line of thinking? Extra weight for crunch time is that opposed too?
The general idea of value being added by high level managers, traders, consultants thru their intellectual capital contributions still seems patently capitalist, though I guess you could call it central planning and get away with the same thing as in historical communist state practice. The workers/players continue to be steered to some degree. But compared to football, basketball is a player's game.
Last edited by Mark on Mon Jun 12, 2006 7:10 am; edited 3 times in total |
|
Back to top |
|
|
|
|
You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum
|
Powered by phpBB © 2001, 2005 phpBB Group
|