APBRmetrics

[Dan Rosenbaum]

At long last here is a working paper version of my paper with Dave Lewin that we presented at the NESSIS conference at the end of September. As we mention in the acknowledgments, this paper has benefited greatly from comments from many of you. And this is still a working paper, so we would love to hear reactions to the paper. Feel free to post your comments here or e-mail them to me. Please let me apologize in advance if I am not able to respond quickly to comments, but each of my three jobs right now keep me pretty busy (and that doesn't include my two year-old).

http://www.uncg.edu/eco/rosenbaum/nessis.pdf

The Pot Calling the Kettle Black: Are NBA Statistical Models More Irrational than “Irrational” Decision-Makers?

Abstract
Recent research suggests that “statistics and analysis” typically lead to better decisions than “intuition and human intellect” in diverse areas such as choosing which students to admit to college and assessing mortality risks among cancer patients. Sports economics, with its rich data and abundant decisions to analyze has provided a fertile laboratory for studies of the efficiency of decision-making. In fact, researchers of the National Basketball Association (NBA) have used statistical models of player productivity to make strong claims about the “rationality” of NBA decision-makers. Yet these statistical models have rarely been subjected to any rigorous examination of their ability to forecast the future. We examine how well several player productivity metrics, including (a) John Hollinger’s Player Efficiency Rating, (b) Wages of Wins Wins Produced, and (c) the NBA Efficiency metric, do in predicting future team wins and future player productivity (the latter as measured by plus/minus statistics). In addition to a comprehensive examination of the player productivity metrics used by NBA statistical analysts, this paper is the first academic presentation of plus/minus statistics. Our findings provide a counterweight to much of prevailing literature and suggest that models that assume simplistic NBA decision-making often outperform more sophisticated statistical models.

[mikez]

Even ignoring its discussion of the "Is Wins Produced Flawed" question, this is an important paper and a significant contribution to the community. Thanks guys.

(Also thanks Cavs, for allowing them to continue to publish despite the work they're doing for the team.)

-MZ

[Westy]

Hmm, I'm very interested to see the response this paper receives.

With the post from TrueHoop, you can certainly anticipate it will have some readers.

What is the plan for future publication (i.e. how long will it remain a 'working' paper)?

[Dan Rosenbaum]

My guess is that the reaction will be less than you think. The issues here are subtle and technical, which makes them easy to ignore. There is the potential for a really interesting and substantive discussion to take place, but based upon past experience I highly doubt that is what will take place. But we'll see. Hopefully I am wrong.

I also want to point out that Berri should get some credit for the way in which his work has motivated deeper thinking about how to evaluate players. I sincerely mean that.

[Harold Almonte]

That's true. A lot of detractors think like his book is some kind of Davinci code of basketball metrics, some followers think it's really the apocriphal gospels, but it's just science procedures from different hiphotesis that needed to be tested. I think this paper is the test.

[NickS]

[Eli W]

[HoopStudies]

[Eli W]

[Harold Almonte]

What they say about Dan including the +/- stuff in the end could be valid, it hurts a bit the paper, but it's his paper.

[Dan Rosenbaum]

Here apparently is the argument that Berri has against what we are doing in our paper.

[Dan Rosenbaum]

[Dan Rosenbaum]

Berri makes claims like this all of the time.

[Analyze This]

I like the pot calling the kettle black because it's not your typical I have found a metric and will defend it to dead article. You look critical to some metrics including your own and try to be objective.

However I find the division between "irrational" traditional decision makers and "new"decision makers/statistical analysts a bit to strong. I think that you have a third party namely the nba decision maker who uses his traditional experience/intuition on one hand and stat analysis/metrics on the other hand while he realizes the strenghts and limitations of the metrics he uses and the context he needs to see it within and also the advantages and shortcomings of the more traditional view. I think that if you combine that traditional approach with what the best of new stat analysis has to offer and you place it in the right context, you see what it tells you and what not, that you are a better decision maker than when a) you only lean on the traditional way of evaluating talent or b) you only use statistical analysis to judge players productivity.

With the article you also risk that people who are against stat analyis will use your words to attack stat analysis as a source. They can say look the analyst himself shows that complicated stat models are sometimes less accurate than an easy/ traditional way of looking at it. You see this stat anlysis is worthless. Of course that is not what your article is about but in a world were slogans mostly prevail above text with nuance it could happen. However that should never stop you to write a critical article like you did. After basketball on paper it was the best thing i read.
_________________
Where There's a WilT There's a Way

[Dan Rosenbaum]