APBRmetrics

[Fire]

So I don't know if anybody on this follows Dave Berri's blog recently, but in a recent response Berri said:

"The critiques you noted were supposed to be published in an academic article. That never happened. If you read — and understood– the article where the critiques were presented, you would see why these were not published.

Although the criticisms were never published, I did respond. You can see the response in an article I published with JC Bradbury. And also in Stumbling on Wins. So I suggest you go buy a copy of the book and look at a few of the end notes. You might discover that much of what the on-line stats community was telling you over the years didn’t stand up very well to scrutiny"

I was wondering if somebody could summarize the arguments against the WoW model and tell me how legitimate they are. To be honest Dberri's model seems to be gaining steam recently, especially in the eye of the public with his new book coming out

[magicmerl]

What was Dennis Rodman's Wins Produced again?

[DLew]

There are many theoretical arguments that can be made against Berri's model, but these are often difficult for people to understand. Really the most effective criticism of his model is empirical.

The best way to test whether a statistical model is accurate capturing player ability is to test whether the model can predict out of sample how a lineup will perform based on the ratings of the players in that lineup and the ratings of their opponents. When tested, Wins Produced fared significantly worse than many other metrics (PER, Win Shares, etc) at out of sample predicting the results of each shift [a shift is a period of a game between substitutions, so just two five man lineups playing against each other].

This is a damning result. Basically, it seems, that Wins Produced's high season level fit is due to overfitting and mis-attribution of credit and that when each shift of the season is considered, in order to disentangle player collinearity, Wins Produced does a poor job of predicting how teams will perform.

[DLew]

Also, could you please link to where he wrote that? Thanks.

[Fire]

[quote="DLew"]Also, could you please link to where he wrote that? Thanks.[/quote]


Sure. http://dberri.wordpress.com/2010/04/22/aaron-brooks-is-not-the-mip-in-2009-10/#comment-70280

The thread has turned into a long-winded debate about usage and efficiency, looking at the case of Ariza. Berri makes a few other interesting comments on this post as well, which is much more than his standard no reply.

[Mike G]

'Stumbling' is about right.
Wherever you stumble, there you are.
_________________
`
36% of all statistics are wrong

[Fire]

more fightin words from berri...

"You are correct that I do not go to the Apbrmetrics forum to discuss topics. I am already here on the Internet. It seems silly to say that someone should dismiss an argument because it wasn’t left in the proper forum.

The Apbrmetrics people are free to leave comments here. In the past they have and we discovered their arguments were not quite as sound as the Apbrmetrics community apparently thinks.

I would add…. I have also already addressed much of what the Apbrmetrics community has said. And this was done in a peer reviewed journal and in Stumbling on Wins. My sense is that most members of this community have read none of this. I also suspect, few ever read The Wages of Wins."

[Mike G]

This is the first appearance of commentary which avers that " the Apbrmetrics community apparently thinks" a certain thing.

We think all kinds of things here. Nobody agrees with anyone else about everything. About Berri, almost everyone substantially agrees: There's nothing worth reading.
_________________
`
36% of all statistics are wrong

[Ryan J. Parker]

[Kevin Pelton]

Berri ought to know that I (and Roland and Dave) read Wages of Wins, since we reviewed it. I have started Stumbling on Wins, but the arrogance of the tone has made it a difficult read for me.

Berri's version of addressing the issues we've raised is to slightly restate the same points. Take usage, for example. It wasn't mentioned, if I recall correctly, in Wages of Wins. After the APBRmetrics community questioned placing no value on usage, Berri relied on a Stacey Brook study that found no relationship between usage and efficiency on a game-by-game basis.

In Stumbling on Wins, Berri uses a study that looks at the relationship between usage and efficiency for players season over season. This continues to be flawed because of the fact that it does not account for player development/aging. Berri has not, to my knowledge, ever addressed Eli Witus' findings--though it's true, I don't read everything he writes.

[DLew]

Yeah, to me it is very troubling that he has not addressed Eli's findings (replicated by Ryan Parker). That is a very well constructed, sound study, and if Berri were actually interested in the truth he should have at least addressed why he thinks it is flawed. Instead he has refused to address it because it contradicts his viewpoint. That is not good science.

[BobboFitos]

[basketballvalue]

[Mike G]

Who has time to write a book?
More to the point, who can write a book fast enough that it's not outdated when it gets to print?

Conversely, what has one gained by putting it into a book?
One can't readily alter one's printed conclusions, because that disparages the value of the book.
It's an albatross of sorts, in that you have to defend what was printed, rather than investigate the ongoing universe of possibilities.

A book does generally fit in the bathroom better than a computer.
_________________
`
36% of all statistics are wrong

[Ed Küpfer]