APBRmetrics

Ryan J. Parker · Joined: 23 Mar 2007 Posts: 708 Location: Raleigh, NC

I believe it's defeatist to worry about what a lineup of 5 Wallace-like players would look like. No one really wants the answer to that, as it's really not practical.

That said, if we examine the performance of teams that play small ball, an uptempo pace, a slow pace, etc. we should be able to determine how this sort of coaching affects the players and hopefully parametrize it in some form or another.

The common theme is that regardless of what we do or the data we have access to, we're simply trying to approximate reality. At first these approximations are probably going to suck. But overtime hopefully we will be able to refine them into something that makes good basketball sense and can help aid the decision making process.

Either way they will never be a "perfect" model of reality. No model is.
_________________
I am a basketball geek.

schtevie · Joined: 18 Apr 2005 Posts: 408

Dean,

I think get the new religion. So, say a small forward for the three years preceding his possible contract extension produces APMs of 1.27, -0.49, and 0.86 (something below the positional average of about 1.0). Never mind that. I still shouldn't have confidence that this player is not worth 5 years and $80 million (where after he produces -5.08 and 1.61 - as an on-going two year average).

There indeed must be a bigger, more useful truth that is beyond reach.

And it is a job to answer this the right way.

Neil Paine · Joined: 13 Oct 2005 Posts: 774 Location: Atlanta, GA

Dude, are you seriously taking a potshot at Dean because Mark Warkentien extended Carmelo Anthony's contract almost 3 years ago, coming off a season in which he scored 27 PPG on 48% shooting as a 21-year-old?

Mountain · Joined: 13 Mar 2007 Posts: 1527

Yeah, metrics can sorta evolve into religions, especially in these public discussions,,, as opposed to how you treat them in personally directed evaluations / work plans. I think most people snap out of it enough see them as just analysis tools.

'...there is no way that one number - any of them - can adequately tell the bigger more useful truth."

That's why you, in part, turn adjusted into a series of products. I laid out how adjusted can so much richer with splits.

As a summary tool lineup adjusted seems a darn fine summary tool for capturing the bigger more useful truth though. Study the entire production unit.

Have you produced or tasked it for the Nuggets? Or used Eli's first rough cut? If not, how do you know how far it falls from telling the bigger more useful truth"? If not, are you just a priori convinced that smaller projects combined yield more or know that they are quicker / more likely to be applied for gain in your organization?

Saying there is no Holy Grail has it is place...but then you get back to finding as much meaning as you can. As big and fine a chalice as you can. A lot of shot glasses can fuel the analytic review too but I'll take the product of a big cask or a big gulp and put it in the middle. A responsive one-liner: "Integrity is wholeness." Redundant? Yeah, on purpose to make its point. I'd never completely give up including that macro-perspective. You end up trying to cobble it back together from the dozens of micro-metrics but they never add up neatly to the whole. Pixelation and pretty grainy at that.

And how is adjusted (split to offense and defense and taken together) at player or lineup level philosophically different than "offensive rating" or "defensive rating" or point differential? Do you still use these or only look at parts? 4 Factors tell useful / necessary details of the "truth" but don't you still put them together? Are you opposed to synthesis by numbers, not as the final score but simply as an organized way to be sure to include a lot of things, about the way you think it should be included to get most of the way there then philosophize, trade-off and guess and decide? To do the synthesis all in your head or in words... I am not sure that is "better". But whatever way you want to do it. I like to see the pieces and best representations of the whole side by side, in numeric form as much as you have and feel good about and bounce between them.

Ilardi · Joined: 15 May 2008 Posts: 262 Location: Lawrence, KS

Since I had the temerity to start this spirited thread - and since I'm in the apparent minority of APBRistas that regards adjusted plus-minus (APM) as an enormously useful evaluative tool - I feel some responsibility to address the many cavils and critiques that appeared on the thread yesterday. Unfortunately, I don't have the time to get to them all right now (that's not a dodge; I'm just on a tight deadline to return copy edits to a publisher). But I'll do my best to respond to some of the main issues in brief, and will be happy to weigh back in as soon as I can next week if anyone's still interested in this discussion by then . . .

I guess the best place to start is with addressing Roland's points, since he's been kind enough to publish a few of my articles on 82games. He begins:

Neil Paine · Joined: 13 Oct 2005 Posts: 774 Location: Atlanta, GA

If APM, using interaction effects, can truly make a credible prediction for the point differential of a lineup of 5 Ben Wallace clones against an average NBA team, then I'd be sold. Because sans said interaction effects (which is sort of the only way the layperson has been able to make predictions using APM), a lineup of 5 prime Wallaces would supposedly be like +25 or +30 per 100 possessions -- which is insane, seeing as how they'd never get the ball over halfcourt, much less be able to shoot or really do anything offensively.

But if interaction effects can take this into account and reflect that Wallace and his 4 clones, while 5 great role players for championship teams, would get destroyed on the court by themselves (an absurd example, to be sure, but a necessary thought experiment for any system such as this), then it would greatly increase my faith in APM.

Westy · Joined: 15 Nov 2007 Posts: 15 Location: Chicago, IL

Great post Steve!

HoopStudies · Posted: Fri Feb 20, 2009 1:34 pm Post subject:

Ilardi · Joined: 15 May 2008 Posts: 262 Location: Lawrence, KS

schtevie · Joined: 18 Apr 2005 Posts: 408

I like a group hug as much as the next guy, but before we agree to agree, we should agree on what we never disagreed about.

By my reading of this and my recollection of all previous versions of this conversation, there has never been any controversy surrounding the notion that "APM cannot exist in a vacuum" or that APM's "lack of ability to say why it comes up with what it comes up with forces use of other tools." Neither has there ever been an assertion of the constancy of APM results (absolutely or relatively, in comparison to other metrics). Nor can I recall there ever having been the claim that APM can accurately predict out of sample wackiness, such as what a team of Ben Wallaces, or Shaquille O'Neals or Allen Iversons would look like. And certainly, there has never been the claim that APM provides the answers to many questions relating to the day to day management and coaching of a basketball team.

But I am sure that there are areas of disagreement that remain. Dean refers to the noisiness of APM estimates being a problem. I am curious if Steve really agrees with this.

To help make the joint points that all rating systems are equal but that some are more equal than others AND that a single, boring statistic can be highly relevant, I offered up the case of Carmelo Anthony. Admittedly, this is perhaps clearer than the truth, in that he is a conspicuous outlier in terms of the gap between offensive-focused conventional wisdom and APM, but it speaks to the issue of noisiness as well.

Important roster decisions are necessarily based upon past as well as anticipated performance. With respect to offering him a five year contract, the probable 1.5 standard error on his then-to-date 1ish APM was well within the bounds of useful and actionable information. More generally, if individual player estimates are indeed independent of one another, multi-year estimates with standard errors around 1.5 provide a non-noisy basis for contemplating the effect of multiple roster moves.

My point is, that in terms of how APM has always been represented as being relevant, it can't be written off for the noisiness of its estimates. This was the basis for my saying that the Celtics had a good thing coming in 2008, and this is a basis for it being the preferred metric for similar types of analysis. What GMs do is important, right?

Can we agree on that?

Ilardi · Joined: 15 May 2008 Posts: 262 Location: Lawrence, KS

Dean refers to the noisiness of APM estimates being a problem. I am curious if Steve really agrees with this.

Yeah, I agree - it's definitely a problem, but one that can often be overcome to a large extent.

Think of it this way . . . using 5 seasons' worth of data to reduce noise, it's possible to get the standard errors on weighted seasonal APM offensive/defensive ratings for high-minutes players down around 1.0. That means we can be confident that 68% of the time any single player value we've estimated has a true value that's +/- 1 point of the estimated rating itself, and 95% of the time it's +/- 2 points. The noise hasn't been eliminated: the level's just a lot lower than it used to be.

Now, don't get me wrong: I'm happy to be able to get the confidence intervals that narrow, as this is what allows the metric to be quite useful in its present form. But I'll be the first to admit that there's still enough noise in the system to prevent us from making fine-grained distinctions - at a high level of confidence - between players whose estimated ratings fall within a point or two of one another.

I think that's why Mountain and others are still interested in augmenting APM with statistical regressions ("statistical plus-minus"), as Dan R. showed that it can, in principle, bring the errors down even further. There's still ample room for improvement.

P.S. Ok, now I've really spent way too much time here today (it's so much more enjoyable than editing and writing end notes), but I need to go become a hermit until next Wednesday.

DLew · Joined: 13 Nov 2006 Posts: 222

This has been a very interesting thread.

As most of you know I am a proponent of Adjusted Plus-Minus. I have published articles on it, and generally feel that it is useful. Please allow me to explain why.

To me, there are three main things that we want to know about a player:
1. How good has he been?
2. Why was he effective/ineffective? i.e. How did he achieve his level of performance?
3. How good is he going to be?

The first question is always the starting point to me. You can't know what is going to happen until you know what has happened. And I feel that adjusted plus-minus represents the truth in this regard, although it comes with a lot of noise (at least in some formulations).

The second question is not really addressed well by adjusted plus-minus, as Roland and many others have pointed out, and is probably more important.

Answering the third question involves combining the information from the answers to the first two questions in order to understand what type of performance can be expected from a player as his situation changes because of coaching changes, teammate changes, or just getting older and more experienced.

I think that Roland makes a good point about how the linear model of APM assumes that a player performs the same across the period being considered with the only deviations from this being due to random variation. If this assumption is untrue then the model is invalid. However, I believe that although player performance varies significantly depending on the way in which the player is used, that in general this variation is not so large that it renders the technique useless.

Adjusted plus-minus is a blunt instrument, and should be regarded as far from perfect, but I think that most of the opposition to it stems from the idea that it is supposed to be perfect. It is not, and when compared to all other overall player rating systems (which, as I've said before are not the be all and end all of analysis) it is clearly superior when done properly.

The arguments that are brought against adjusted plus-minus, like that player performance is context dependent can be applied to any overall player rating system, and therefore should not be part of this discussion. If we are going to debate its merits, we should consider it a given that it has the flaw of summarizing player performance in one number (or two if we want to split it by offense and defense). This is not open for debate. Instead intelligent debate will center on whether it is better or worse than other metrics that share this same property.

schtevie · Joined: 18 Apr 2005 Posts: 408

Steve - who is not there - my question to you about noise not being the problem some fear it to be had less to do with using APM for considering the value of an individual player and more to do with using APM as a tool for evaluating hypothetical rosters. If the player estimates are uncorrelated, the variance of the sum decreases with the increasing number of players considered.

HoopStudies · Posted: Fri Feb 20, 2009 4:51 pm Post subject:

Mountain · Joined: 13 Mar 2007 Posts: 1527

The top 4 playoff teams the last 2 years have been lead by top 3 players who were all positive on the initial multi-season adjusted study Steve and Aaron published, except the 2007 Cavs. The Cavs changed.

Melo's APM is relevant for contention. But, it is not terrible- and must be better this season- and he can probably work in his specialist role in a big 3 or 4 where the others handle the make the team better stuff and that was attempted last season and this season, and right now looks better.

Still I'd trade him for a good offer. Previously. Or this summer. But I'll let you figure that out.