but Hollinger justifies his assertion that the PER is superior by saying that it confirms his pre-existing conceptions of how players should rank -- after all, those other methods must be hogwash, because they don't have Shaq as the number one center, or they have David Robinson ranking ahead of Michael Jordan (I've got news for you, John, PER has been known to do the same thing). It's called confirmation bias, and Berri's right -- that's not how science works.
I see what you're saying. Two comments though:
I never really think of PER as a scientific attempt to rate players. I think you're right that it doesn't compare to some of these other projects. For me, it's always just been a summary tool. It gathers all these box score numbers from all over the place -- more than I could ever keep inside my head -- and gives me a single number summary of what a player has achieved statistically. And for that, it's extremely useful. Going from statistical dominance to true value is another jump, but that doesn't mean that there is no merit to trying to map out the former.
Secondly, I don't think the primary appeal of PER is that it confirms pre-existing conceptions. You yourself pointed out how often it doesn't. I think its strength is that all the weights for each stat seem very well crafted and thought out. Assists, of course, stand out as being particularly arbitrary, but there's not much that can be done there within the scope of this project. The 2/3 - 1/3 split seems like a reasonable compromise.
"If you take ten shots at a below average rate you hurt your team. If you take twenty shots at a below average rate you hurt your team even more."
1) Is this statement true?
2) If it is true, how do we know that? If it is not true, how do we know that?
3) Is this universally true, or is it context-dependent?
I still have trouble believing that efficiency should be judged independently, as opposed to being judged in the context of shooting frequency. I also have been trained to think that using the average as the baseline of acceptable performance is wrong. But if there's no measurable correlation between efficiency and shot volume... what's correct?
Joined: 03 Jan 2005 Posts: 413 Location: Greensboro, North Carolina
Posted: Mon Nov 20, 2006 5:20 pm Post subject:
By the way, somebody who at least lurks here at APBRmetrics is posting comments at the Wages of Wins blog relating some of the arguments of mine in this thread and others. Please be a little careful in those posts; in a couple cases they have overstated arguments that I have made.
It is good, I guess, that these comments have resulted in the closest thing to a conversation about Wins Produced that we have to date. It is frustrating that it is the ONLY way we can have a conversation in anywhere near real time.
I apologize for prolonging this, but I still think this conversation - despite how strained it is - is productive and interesting. Some of the digressions have been particularly interesting. And, as I have said before, I have learned a lot from the conversation in this thread.
Joined: 03 Jan 2005 Posts: 413 Location: Greensboro, North Carolina
Posted: Mon Nov 20, 2006 5:48 pm Post subject:
94by50 wrote:
From the post linked above:
"If you take ten shots at a below average rate you hurt your team. If you take twenty shots at a below average rate you hurt your team even more."
1) Is this statement true?
2) If it is true, how do we know that? If it is not true, how do we know that?
3) Is this universally true, or is it context-dependent?
I still have trouble believing that efficiency should be judged independently, as opposed to being judged in the context of shooting frequency. I also have been trained to think that using the average as the baseline of acceptable performance is wrong. But if there's no measurable correlation between efficiency and shot volume... what's correct?
I actually sent Dave an e-mail on this point just last night. It can be shown that replacing a player with an otherwise identical player, except that the number of true shot attempts per 40 minutes is one standard deviation higher, results in about 3 more wins a season. This is holding the number of "extra" points constant (approximately points minus true shot attempts), so it corresponds to the thought experiment above.
This result is statistically significant with a p-value of less that 0.0000 (which means there is less than in a 1 in 10,000 chance that this is due to random chance). The 95% confidence interval rules out that this effect is smaller than 2 wins.
I generally provide Dave with far more detail about the nature of this evidence than I can provide publicly, but still no response. After probably more than a dozen attempts publicly and privately to provide evidence to bear on the claims that Wages of Wins makes, the only response has been that such evidence is "hogwash." I think he actually has used the word "silly," but I think the point is the same.
This is a bitch of a subject to study, and their results differ from the results that Dean reported here. I don't think the methodology they describe really gets at the issue effectively because of their focus on single game performances. If a guy is shooting well on a given night, odds are, he's going to shoot more. The phrase that relationship differently -- saying repeatedly, "...the more he shoots, the better he shoots."
But, they're addressing an argument that I don't think Conventional Wisdom is actually making. Rather, CW says that changing a player's role from low usage to higher usage will often bring about lower efficiency. And vice versa -- going from higher usage to lower usage will often bring about higher efficiency. Looking at game-to-game fluctuations in shot attempts and shooting percentage don't address this issue. As I recall, Dean's findings do address this issue, though I don't know what methods Dean used to reach the findings. (I recall from Dean's book that his skills curves rely heavily on the notion that efficiency tends to decline as usage increases. Was Dean merely finding fiction in the numbers?) _________________ If you can't explain it simply, you don't understand it well enough.
-- Albert Einstein
Last edited by WizardsKev on Tue Nov 21, 2006 9:40 am; edited 1 time in total
Joined: 03 Jan 2005 Posts: 413 Location: Greensboro, North Carolina
Posted: Mon Nov 20, 2006 5:53 pm Post subject:
Improving the shooting percentage of teammates is not the only way that scorers can affect their teammates. In fact, it may not even be the primary way. Difficulty in creating a shot often leads not to bad shots, but to no shots at all due to increased turnovers.
I actually sent Dave an e-mail on this point just last night. It can be shown that replacing a player with an otherwise identical player, except that the number of true shot attempts per 40 minutes is one standard deviation higher, results in about 3 more wins a season. This is holding the number of "extra" points constant (approximately points minus true shot attempts), so it corresponds to the thought experiment above.
This result is statistically significant with a p-value of less that 0.0000 (which means there is less than in a 1 in 10,000 chance that this is due to random chance). The 95% confidence interval rules out that this effect is smaller than 2 wins.
At some point, I'd love to see the details of the study that gives this conclusion. For some time now, I've tried to figure a reasonable method to "credit" increased efficiency to players who are responsible for more than their share of their team's offense (and vice versa), but none of it would be empirically or statistically valid - it's just guesswork, so it's been more or less fruitless.
How many true shot attempts per 40 minutes is one standard deviation?
Dan Rosenbaum wrote:
Difficulty in creating a shot often leads not to bad shots, but to no shots at all due to increased turnovers.
Absolutely. As long as there's a shot clock, anyone who can create shots should be more valuable than a player who can't, all other things equal. Any missed shot attempt may still be rebounded by the offense. A turnover can't be reversed in any way. A missed field goal attempt has to be "worth more" to a team than a turnover. I don't see how it can be otherwise.
Joined: 03 Jan 2005 Posts: 413 Location: Greensboro, North Carolina
Posted: Mon Nov 20, 2006 8:47 pm Post subject:
94by50 wrote:
Dan Rosenbaum wrote:
Difficulty in creating a shot often leads not to bad shots, but to no shots at all due to increased turnovers.
Absolutely. As long as there's a shot clock, anyone who can create shots should be more valuable than a player who can't, all other things equal. Any missed shot attempt may still be rebounded by the offense. A turnover can't be reversed in any way. A missed field goal attempt has to be "worth more" to a team than a turnover. I don't see how it can be otherwise.
First off, a standard deviation in true shot attempts is about 4.
Second, here is another way to think about the formula for a possession (ignoring the 0.96 multiplier that I use to account for team rebounds).
where TO is turnovers,
TSM is true shots made (field goals made + 0.44*free throws made)
TSA is true shot attempts
OREB is offensive rebounds
oppDREB is opponent's defensive rebounds
This equation works for any value of alpha. Berri and his co-authors actually talk about this formula using two different values of alpha, i.e. when it equals one and zero.
Here is the formula when alpha equals one.
POSS = TO + TSA - OREB
Here is the formula when alpha equals zero.
POSS = TO + TSM + oppDREB
But if Berri and his co-authors are going to use these two formulas, they need to add them up to get the proper weights.
Note that this will result in rebounds being worth less than in Wins Produced. Also, it changes the relative value of points versus shot attempts, because it accounts for the fact that missed shots may be rebounded. (That is why TSMissed ultimately comes in as a positive.) In other words, a proper application of the Wages of Wins logic results in a model very much at odds with what Wages of Wins claims. Rebounders are worth less. Shot creators are worth more.
But the bigger problem is that any value of alpha will make this possession formula work. In other words, it needs to be estimated.
In essence, what they have done is take an empirical question (the relative weight for offensive rebounds relative to defensive rebounds) and assumed it away. For no good reason (although I must admit that I had not thought it through like this until now).
In econometric terms what they have done is take an equation that is not identified and through an arbitrary assumption simply assumed away the identification problem. (And made a mistake in doing so.) Sometimes we HAVE to do that in economics to make a model tractable. But this is not one of those cases.
I am sorry for not noticing this until now. Noticing this several months ago would have saved a lot of trouble. I really was not expecting to find something like this, but some of the posts here and at the Wages of Wins blog got me thinking about this differently. It is that type of collaboration that makes a forum like this so valuable.
P.S. Let me make something clear that may not be so. If you are only wanting to calculate the number of possessions, it doesn't really matter much what value you use for alpha. But if you want to use these possession equations to derive the relative value of rebounds, then it is critical to estimate alpha. So for what most of us use these possession formulas for, this is not an issue. Not so for Wins Produced, where this is the foundation of their whole methodology.
Last edited by Dan Rosenbaum on Tue Nov 21, 2006 2:02 am; edited 1 time in total
It's interesting that Berri's crew [url=http://dberri.wordpress.com/2006/06/11/the-law-of-diminishing-returns-in-the-nba}says there's no relationship between usage and efficiency.[/url]
This is a bitch of a subject to study, and their results differ from the results that Dean reported here. I don't think the methodology they describe really gets at the issue effectively because of their focus on single game performances. If a guy is shooting well on a given night, odds are, he's going to shoot more. The phrase that relationship differently -- saying repeatedly, "...the more he shoots, the better he shoots."
But, they're addressing an argument that I don't think Conventional Wisdom is actually making. Rather, CW says that changing a player's role from low usage to higher usage will often bring about lower efficiency. And vice versa -- going from higher usage to lower usage will often bring about higher efficiency. Looking at game-to-game fluctuations in shot attempts and shooting percentage don't address this issue. As I recall, Dean's findings do address this issue, though I don't know what methods Dean used to reach the findings. (I recall from Dean's book that his skills curves rely heavily on the notion that efficiency tends to decline as usage increases. Was Dean merely finding fiction in the numbers?)
I did skill curves for all Belgium players (game by game). What I found out was that some players (an important %) actually have a higher efficiency when they use a high amount of the possessions (between 20 and 25%) than when they use a low/medium amount of the total possessions. But with most the efficiency goes down when the % of possessions goes up. This was just a small remark, now carry on with the discussion.
This is a bitch of a subject to study, and their results differ from the results that Dean reported here. I don't think the methodology they describe really gets at the issue effectively because of their focus on single game performances. If a guy is shooting well on a given night, odds are, he's going to shoot more. The phrase that relationship differently -- saying repeatedly, "...the more he shoots, the better he shoots."
But, they're addressing an argument that I don't think Conventional Wisdom is actually making. Rather, CW says that changing a player's role from low usage to higher usage will often bring about lower efficiency. And vice versa -- going from higher usage to lower usage will often bring about higher efficiency. Looking at game-to-game fluctuations in shot attempts and shooting percentage don't address this issue. As I recall, Dean's findings do address this issue, though I don't know what methods Dean used to reach the findings. (I recall from Dean's book that his skills curves rely heavily on the notion that efficiency tends to decline as usage increases. Was Dean merely finding fiction in the numbers?)
I did skill curves for all Belgium players (game by game). What I found out was that some players (an important %) actually have a higher efficiency when they use a high amount of the possessions (between 20 and 25%) than when they use a low/medium amount of the total possessions. But with most the efficiency goes down when the % of possessions goes up. This was just a small remark, now carry on with the discussion.
It's a good "small remark". I think we can all agree that some players would get more efficient with more possessions -- to a point -- while others would see a drop off in efficiency. Dean posted on the boards a couple times an average effect, which was a decline in efficiency as usage increases. Do you see something similar using players from Belgium? _________________ If you can't explain it simply, you don't understand it well enough.
Joined: 03 Jan 2005 Posts: 460 Location: Washington, DC
Posted: Wed Nov 22, 2006 11:49 am Post subject:
What does Malcolm Gladwell say about PER in his latest blog entry? http://gladwell.typepad.com/gladwellcom I'd read it myself, but my workplace has a firewall blocking access to many blogs. Gladwell's unfortunately is one of them. _________________ If you can't explain it simply, you don't understand it well enough.
What does Malcolm Gladwell say about PER in his latest blog entry? http://gladwell.typepad.com/gladwellcom I'd read it myself, but my workplace has a firewall blocking access to many blogs. Gladwell's unfortunately is one of them.
"I've long been a fan of John Hollinger, who writes about basketball for espn.com, in large part because of Hollinger's statistical system for analyzing NBA players. Hollinger calls it PERs, and I like it chiefly because I'm in favor of any system that tries to improve on what I think are our woefully inadequate intuitive judgments of basketball ability.
Now David Berri--whose book, "The Wages of Wins", I wrote about a few months back--has critiqued Hollinger's methodology.
Berri's argument is quite simple. As those of you who have read "Wages of Wins" know, Berri's big problem with the way we judge pro basketball players is that we over-rate the importance of how many points a player scores, and vastly under-rate the importance of things like turnovers, rebounds, and shooting percentage.
Now comes Berri's critique of Hollinger: he says that Hollinger makes the same mistake. Here's the critical section:
In discussing the NBA Efficiency metric – which the NBA presents at its website – I argued that this measure fails to penalize inefficient shooting. The regression of wins on offensive and defensive efficiency reveals that shooting efficiency impacts outcomes in basketball. The ball does indeed have to go through the hoop for a team to be successful.
The same critique offered for NBA Efficiency also applies to Hollinger’s PERs, except the problem is even worse. Hollinger argues that each two point field goal made is worth about 1.65 points. A three point field goal made is worth 2.65 points. A missed field goal, though, costs a team 0.72 points.
Given these values, with a bit of math we can show that a player will break even on his two point field goal attempts if he hits on 30.4% of these shots. On three pointers the break-even point is 21.4%. If a player exceeds these thresholds, and virtually every NBA played does so with respect to two-point shots, the more he shoots the higher his value in PERs. So a player can be an inefficient scorer and simply inflate his value by taking a large number of shots.
I'd be interested to see how Hollinger replies to this.
As I recall from the last time I posted on Berri, some readers have a problem with Berri's conclusions, mostly because his system ends up highly valuing players like Ben Wallace and Dennis Rodman and Kevin Garnett and dismissing the value of players like Allen Iverson. But the more Berri's fleshes out in arguments, the more convinced I become.
If you're a skeptic, I urge you to start reading Berri's blog.
One more point: one of the fascinating things about this argument is how similar it is to the argument currently going on in medicine about "clinical" versus "acturial" decision-making. One study after another has demonstrated that in a number of critical diagnostic situations, the unaided judgment of most doctors is substantially inferior to a diagnosis made with the assistance of some kind of algorithm or decision-rule. Doctors don't like to admit this. But it happens to be true.
A lot of the huffing and puffing about Berri's ideas, it strikes me, is just basketball's version of the same defensiveness and close-mindedness."
You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum
FAQ
Search
Memberlist
Usergroups
Register
Profile
Log in to check your private messages
Log in
