Ok thanks for setting aside both these questions. I didnt realize the clutch adjustment was so limited in application. Appreciate the sharing of the method results.
Joined: 03 Jan 2005 Posts: 413 Location: Greensboro, North Carolina
Posted: Sat Jul 29, 2006 10:37 am Post subject:
One of the main points of Wages of Wins is that according to the evidence presented in the book, it appears that NBA decision-makers are irrational. They write: "It is not that people in the NBA are lazy or stupid. It is just that the tools at their disposal do not allow them to see the value of the various actions players take on the court."
The argument that supports this conclusion is they show that Wins Produced explains wins much better than NBA Efficiency, but NBA Efficiency is much more closely tied to salaries and All-Rookie Team voting (done by coaches). Thus, according to their evidence, it is irrational for teams to not be using something like Wins Produced to make their decisions.
OK, but the difficulty with this argument is that it hinges on Wins Produced better explaining wins than NBA Efficiency. Wins Produced does a great job explaining wins because of their team defense adjustment, but the authors admit that this adjustment has very little effect on their relative rankings of players. So if it doesn't matter much for the relative rankings of players, I just don't see how it can be used as a justification for the methodology. To me, that whole exercise raises a big red flag about the validity of using the prediction of team wins as a barometer for a metric for individual players.
But then if we move to another barometer - adjusted plus/minus ratings - we see that Wins Produced only performs better than NBA Efficiency if position adjustments are used for Wins Produced but not for NBA Efficiency. That significantly changes the story of much of their book. Instead of a story about NBA teams overvaluing scorers, their story becomes one that NBA decision-makers are irrational because they don't properly position adjust.
Moreover, the authors provide little justification for their position adjusting, especially in relation to how important it is to their metric. They argue that big players would have difficulty filling the roles of guards; i.e. a team could not play all centers. But if centers truly are worth more than guards as their unadjusted Wins Produced suggests, this would not be the only reaction of NBA decision-makers. Instead of playing centers at guard, what would happen would be that they would pay centers more than guards - which is precisely what does happen. So rather than proving conventional wisdom wrong, maybe the authors have provided justification for conventional wisdom.
Now I am not necessarily trying to defend NBA decision-makers as being super-rational. Lots of points made in Wages of Wins are points I agree with wholeheartedly. And I highly recommend that everyone in this APBRmetric community read this book. You will find lots that you agree with in this book and lots that will force you to think more deeply about things.
But when we go to cast stones at NBA decision-makers, we need to be sure that our own house is in order. And that is really my biggest complaint about Wages of Wins. My experience has been that NBA people often do a much better job than we give them credit for. I work for Cleveland, and last year when we traded for Flip Murray I was against it. Flip was the lowest rated two guard in my system. (And I am sure he would not be rated too highly in Wins Produced.)
But you know what? Flip did not play too badly for us. He was not a star or even a good starter, but he made some changes to how he played in Seattle and he contributed to us doing well down the stretch and in the playoffs. So in that situation if Danny had listened to me (or probably consulted Wins Produced), the team would have won fewer games.
There is a lot about this game that we in this community know well, but there is a lot we don't know well. I strongly believe that good stats work can play an important role in a well-run organization. But I vehemently disagree with those who are ready to start calling NBA people irrational because of some results from a possibly mistaken empirical analysis. We all can work on being better than that.
I find it strange that your posting doesn't get more replies from people who are mentioned in your post or other leading apbr metricians. (only Marc is heavenly responding) The fact that you are sometimes answering your own posts shows how important you find the remarks that you made in the original post of this topic. I've read Berri his book, just like I've done with Oliver his work. (and Hollinger). The problem I have with remarks like “And I have tended to find that the weights for these stats differ a lot from the logic-based approaches of Dean Oliver, John Hollinger, and Berri and his co-authors.” or the other judgements you make about different methods is that I can’t give a value to your work because you are not making public how it works. If you decide to make something for the money and don't give comment about " your tricks" you are not helping the stat community as a whole, something that Oliver, Berri and others did, but you are just helping the Cavs and are basically just cashing in. Don't get this the wrong way. I've read from different people that you are a really nice guy. So you probably are. But don’t start telling how much bettter your results are without making your method public. How can the other analysts respond if they don't know how your method works exactly.
Joined: 14 Jan 2005 Posts: 971 Location: Delphi, Indiana
Posted: Sat Jul 29, 2006 11:12 am Post subject:
Dan Rosenbaum wrote:
... My experience has been that NBA people often do a much better job than we give them credit for. .
Thanks for taking the high middle ground, Dan. While some among us fit the description, others (I consider myself one of these) presume that competency is the norm in such a megadollar business as NBA ownership.
The various (and radically different) forms of analysis seen in this forum are proof positive that there is not a single "Us" position. (See any thread involving Antoine Walker.) I suppose the evolution of all efforts will someday provide such things as salary that's commensurate with value, formulaic team-building, etc.
There really should be a good number of dead-end routes per each new breakthrough, or else we aren't turning every stone.
Meanwhile, I've been doing eWins, which seems to be unique in one aspect: individual eWins are not meant to add up to team Wins. It seems the sum of individual departure from average is only half the team departure: A squad of players totalling 51 eWins (41+10) will win 61 games (41+20).
W = eW*2 - 41
My multiplier seems to be 2, but it might really be 2.03, or 1.98. Or something more complex. Other systems might find 1.4 or 2.2 to be appropriate. In any event, since a scoring advantage of 10% over the course of the season, between team A and team B, will surely produce much more than 10% difference in their W/L%, doesn't it seem reasonable that individual Wins Contributed (or whatever name one chooses) should not add up to Team Wins? _________________ 40% of all statistics are wrong.
Last edited by Mike G on Sat Jul 29, 2006 11:48 am; edited 1 time in total
Joined: 03 Jan 2005 Posts: 413 Location: Greensboro, North Carolina
Posted: Sat Jul 29, 2006 11:23 am Post subject:
Analyze This wrote:
I find it strange that your posting doesn't get more replies from people who are mentioned in your post or other leading apbr metricians. (only Marc is heavenly responding) The fact that you are sometimes answering your own posts shows how important you find the remarks that you made in the original post of this topic. I've read Berri his book, just like I've done with Oliver his work. (and Hollinger). The problem I have with remarks like “And I have tended to find that the weights for these stats differ a lot from the logic-based approaches of Dean Oliver, John Hollinger, and Berri and his co-authors.” or the other judgements you make about different methods is that I can’t give a value to your work because you are not making public how it works. If you decide to make something for the money and don't give comment about " your tricks" you are not helping the stat community as a whole, something that Oliver, Berri and others did, but you are just helping the Cavs and are basically just cashing in. Don't get this the wrong way. I've read from different people that you are a really nice guy. So you probably are. But don’t start telling how much bettter your results are without making your method public. How can the other analysts respond if they don't know how your method works exactly.
Point well taken, but my piece at 82games http://www.82games.com/comm30.htm lays out the basic points of my methodology in more detail than most books or academic papers would allow - certainly moreso than Wages of Wins. Also, I do not answer every question about my methodology, but I have spent a lot of time in a lot of different forums talking about details about my work. Now I cannot give out my data or my results, because they are not mine to give out (they never have been even before I worked for the Cavs). But I agree that without others replicating my work, it is hard to put it into perspective. But I have laid out the methodology in a lot of detail; I can't be responsible for others not replicating my work.
I agree that listening to, looking for the intelligence of others is wise as well as trying to find the faults. Refining one's own knowledge both thru self-directed research and awareness of and absorption of what is useful from the research and observations of others.
With regards to Flip Murray both sides had arguments before the acquisition and then afterwards both sides could point to time periods and parts of the game where the choice worked (regular season, with Snow, including clutch shooting regular season) and where it didnt (playoffs). Snow PG / Murray SG combo worked pretty well regular season. Flip had third highest win % on court but that can be read two ways, and both be part of the story, Flip working and Flip put in good context to be safe.
According to 82 games Cavs went from playing Flip at point almost not at all regular season to 20% in playoffs, to try to capture his historical ability to really score on PGs but he didnt have it going and their quickness and craftiness burned him back threefold on points per 48 minutes (including 10 FTAs per 48 at PG compared to only half that at SG). Murray's poor performance in limited floortime in 2004-5 playoffs could have read different ways (his offense game doesnt work as well against playoff's higher level of defense vs he didnt get enough run to judge) but it was a key showdown dataset on Murray. Acquiring him has to be put in context of immediate regular season need, who else they could get, and prospects for getting Hughes back for playoffs and in what condition. Perhaps they could have spent more to get someone else, proven in playoffs.
I agree with deepak that it would be useful to see unadjusted +/- and roland ratings added to the comparison mix. The Heroes of the Hardwood book too (and protrade) if time and interest was there for the fullest possible comparison. Might you publish in an academic journal about this type of method performance comparison and the surrounding context?
With regards to replication of adjusted +/-, others previously talked about attempting to do so and bring the method into the public view. I hope they will do so but it would be a huge task to replicate the database and then navigate all the analysis so I doubt it will happen.
The other methods all seem to bump up against a limit on how much they can explain, a limit on correlation of results. This may show that using the same dataset of the commonly tracked, the seen has a max regardless of method.
Dan's method capturing team play and the hidden uncounted contributions to it has the lead and should get more study from us as best we are able to gather from him or independently. Available unadjusted player pair data might allow some questions to be asked, answered approximating insights available thru adjusted +/- datasets.
I assume Dan's comment about not being free to release the adjusted +/- data even before the Cavs work is because the data came from 82games under conditions limiting how much could be revealed? Although it is easy and understandable to want revelation, I accept business need for confidentiality that will withhold much of what teams might pay for. Really we are very fortunate 82games shares so much, so thanks again for that and thanks Dan for sharing as much as you feel you can about method and implications. Although I ask a lot of questions, reply to and use as you will. I ultimately respect your limits and trying to be a leading, considerate participant in really 3 worlds.
I remain interested in Mike's Ewins because of the different approach he notes. It is more traditional use of the tracked stats and individual based. More knowledge about transference of players and transferability of their win producing value from one team to another needs more work to aid trade and free agent analysis and even the draft.. Methods that are good at capturing contributions to team play and individual based methods are both important to consult.
Identifying different roles, different successes levels in those roles of the same player on same team also needs more precise data sorting and analysis. All the methods cited in this thread could with unlimited time and resources be broken into data subsets to evaluate players by different positions and roles. Teams certainly should.
Mike, do you plan any further enhancements to eWins in near future? To more fully capture defense, including one on one defense?
Would you want to give Dan the information necessary to score your method the same way, against the others if he were inclined? Or Dan is there anything more you could say that would allow others to pursue accurate comparable scoring of other methods similar to your scoring if you arent inclined to score more or all possible method contenders and variations? Rating the methods and putting light on the best elements and the weaknesses seems like a very timely and important thing to do, to help kick off the next era in player and team evaluation.
What should be done? Capture team and individual contributions. Defense as fully as offense. Use many methods, refine them but to some extent preserve their uniqueness. Consider a best practices grand theory or blended meta-ratings. Count as much as possible, beyond traditional stats using tape and also see the game wholistically. Compare players by position and role but be very flexible about what you do with position and role given the talents of the players you have, the mix and the potential of various lineups against the range of challenges by opponents. Try to measure and rate the coaches input more. It can be an endless search to know more and use that knowledge...
Last edited by Mark on Fri Aug 04, 2006 11:07 pm; edited 5 times in total
Joined: 30 Dec 2004 Posts: 410 Location: Near Philadelphia, PA
Posted: Sat Jul 29, 2006 1:02 pm Post subject:
Dan Rosenbaum wrote:
...
There is a lot about this game that we in this community know well, but there is a lot we don't know well. I strongly believe that good stats work can play an important role in a well-run organization. But I vehemently disagree with those who are ready to start calling NBA people irrational because of some results from a possibly mistaken empirical analysis. We all can work on being better than that.
As Dan knows, the concept of rationality is an important issue in economics. There is an assumption of rational decision makers and economists are often looking for evidence of that rationality, if not using the assumption to show something else. The Malcolm Gladwell book, Blink, discusses how experts can come to a more rational conclusion in the first two seconds of thought than others can with some analysis. An expert's "gut feel" may not be easily explained or easily justified, but it is often shown to be justifiable after very extensive follow up. The simple analysis often used to justify a decision when experts are saying it "seems wrong" can very definitely be wrong.
We are here to do thoughtful analysis, not to do quick and dirty things that fly in the face of an expert's blink.
My phrasing of this concept was as follows about a year ago:
hoopstudies wrote:
My null hypothesis is usually traditional coaching or management wisdom. So a hot hand exists, defense wins championships, and statistics are irrelevant until I prove otherwise (which I think I've done in many cases). Others may choose a different null hypothesis, but I think mine makes sense because I work with coaches and management and I'm not [in the position of] Billy Beane -- it is my burden to prove things, not theirs.
Bottom line, though, is that it does go both ways and I feel a significant burden of proof for anything that strays from traditional wisdom.
That being said, there is irrationality in sports (people in sports have said that to ME). There is irrationality within every decision-making group I have ever seen. There is irrationality among the discussions here. Any GROUP will have its irrational decisions because of competing interests and significant inefficiencies in negotiation to reach a group decision.
(Or at least apparently irrational. There are reasonable justifications for some apparently excessive salaries, for instance. It is an interesting economic phenomenon associated with the limited supply of players and extreme non-commoditized market - rational short and moderate term thoughts can make for an apparently irrational long-term or steady-state market. Highly skewed distributions of value makes it much easier to have mistakes, too. Are mistakes irrational? Not necessarily.)
Wages of Wins definitely makes you think about this kind of stuff, whether you agree with the specifics of the methodology or not. _________________ Dean Oliver
Author, Basketball on Paper
http://www.basketballonpaper.com
Joined: 03 Jan 2005 Posts: 413 Location: Greensboro, North Carolina
Posted: Sat Jul 29, 2006 1:02 pm Post subject:
Mark wrote:
With regards to replication of adjusted +/-, I hope the others who previously talked about attempting to do so and bring the method into the public view will do so.
I want to address this topic, because it came up with Analyze This, as well. Replicating adjusted plus/minus ratings, at least doing it the way I do it, is never going to be something that can be done on a wide scale.
Most of this community is comfortable working with datasets numbering in the thousands in something like MS Excel. The dataset that I use for my adjusted plus/minus ratings numbers in the hundreds of thousands and it would be bigger if I had not condensed it.
So that means computing adjusted plus/minus ratings requires a person to learn a programming language like SAS or Stata (or something more basic). That is something that most people do not have the time to do.
It also requires becoming very proficient at working with complex datasets. This is what I teach and while I am NOT a very good economist, I am probably in the upper echelon of economists in terms of putting together complex datasets.
After putting the data together and getting the model to run, understanding how to make sense of the results requires very good intuitive econometrics skills. Again, this is one of my strengths relative to other economists, since I do a lot of teaching of econometrics at the undergraduate, MA, and Ph.D levels.
So the point here is not that I am sort of genius. I certainly am not. It just so happens that my skill set is pretty rare (even for economists) and just so happens to be a good fit for computing adjusted plus/minus ratings. It is really just a right place at the right time kind of thing.
To put this into perspective, even with my skill set, taking my programs and converting them from one program to another (SAS to Stata) took me several weeks this Spring and Summer. And that is with the help of a graduate student of mine - one who has been trained by me to do this kind of work and who is very smart and very good at this kind of stuff. I think he could attest to how difficult this was. And this was only taking a set of programs that I had already written and converting them into another program.
I don't write this to scare people off from replicating my work. I encourage others to do so, and Winston and Sagarin are not the only folks to compute adjusted plus/minus ratings. So it can be done. But I thought this was as appropriate of a place as any to warn folks to be prepared that doing is no small feat.
That said, hopefully someone can figure out how to do what I do but in a much simpler fashion. Obviously I am not smart enough to do that.
So the point here is that it is going to be very difficult to bring adjusted plus/minus ratings into the mainstream in the same way as PER or Dean's work. And I understand the frustration and skepticism that results from this.
I revised my remark about hoping for replication by following it with skepticism it will be done, but perhaps someone passionate about one team might try it for 1/30th of the full league. That could be a way to start and have data in public and further method discussion. Probably some pitfalls compared to doing all 30 but probably still enough benefits to look at it.
Joined: 13 Jan 2005 Posts: 168 Location: Iowa City
Posted: Mon Jul 31, 2006 1:40 pm Post subject:
Mark wrote:
I revised my remark about hoping for replication by following it with skepticism it will be done, but perhaps someone passionate about one team might try it for 1/30th of the full league. That could be a way to start and have data in public and further method discussion. Probably some pitfalls compared to doing all 30 but probably still enough benefits to look at it.
I still think the big problem is getting the data (you need the full league's data). There are enough people out there with the skills in applied statistics and the popular statistics programs that I'm confident the ratings would get done.
Joined: 03 Jan 2005 Posts: 413 Location: Greensboro, North Carolina
Posted: Mon Jul 31, 2006 4:56 pm Post subject:
I like to tout this group as a pseudo-academic community, so I would love to hear any thoughts folks have about Wages of Wins. I suspect that if a larger swath of our community chimes in about the book, there would be a real chance of getting the authors to maybe respond. If it is just me and a couple others, I doubt they will.
It would be really great to see this group interacting with the sports economists that are interested in the same topics we are. There would be a great benefit for all groups if there was more interaction.
It has only been out 2 months. It is summer. Apparently half or somewhat more on basketball- perhaps if 100% basketball it might have move even faster into this market. The reviews, the news articles and the blog may have taken some of the mystery out of it. On Berri's blog I see he has responded to some questions and comments. Further discussion could be beneficial.
Last edited by Mark on Fri Aug 04, 2006 11:08 pm; edited 5 times in total
Joined: 13 Jan 2005 Posts: 168 Location: Iowa City
Posted: Tue Aug 01, 2006 11:25 am Post subject:
Dan Rosenbaum wrote:
I like to tout this group as a pseudo-academic community, so I would love to hear any thoughts folks have about Wages of Wins. I suspect that if a larger swath of our community chimes in about the book, there would be a real chance of getting the authors to maybe respond. If it is just me and a couple others, I doubt they will.
It would be really great to see this group interacting with the sports economists that are interested in the same topics we are. There would be a great benefit for all groups if there was more interaction.
It could be that most haven't read the book, but I also think disagreements provide the most active threads. My guess is most people find your analysis convincing.
You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum
FAQ
Search
Memberlist
Usergroups
Register
Profile
Log in to check your private messages
Log in
