APBRmetrics

[Ben F.]

I'd like to raise this question again, after we've had some time off from talking about it, because I think that it is a very fundamental question that we have not yet answered - Does efficiency drop as usage rises? And if so, how much?

Obviously it's player specific - the players we traditionally think of as "stars" are those that can (usually) maintain high levels of efficiency at high usage rates. The best "role players" are those that are efficient at low usage rates.

The question is, if you gave that role player a much larger role, so that his usage climbed drastically, would he be able to maintain his efficiency? And if you limited the usage of the star, would his efficiency rise?

I looked into this a little bit, and looked at the 100 most efficient players of the last 5 years, according to ORating at b-r.com. I don't know that much statistically, but I found a very slight correlation (-.3) between usage and efficiency that suggests that as usage rises efficiency decreases.

But that's only very slight. I believe if you just take the top 100 from this last year you actually find the opposite trend, although again, it's VERY slight.

It seems to me that theoretically, the reason "stars" help the team so much are for the reasons Dean O layed out in BoP, in his chapter on skill curves - that their usage at a fairly high efficiency helps the other players on the team maintain their ideal usage, and therefore their highest efficiency. That was his theory on Iverson - that although he didn't seem that efficient, he could use very large numbers of possessions at a slightly below average efficiency, but in turn the players around him could maximize their efficiency because they didn't have to force anything. In this way the team was built almost perfectly around him, and it carried them to the championship.

That all makes very logical sense - but as Dean also said, it's very hard to prove statistically. There's really not a large sample size to be able to view what happens when role players take on possessions and stars lose them. And it also seems that you will almost never see a "role player" take on a lot of possessions for extended periods of time - they default to what is most comfortable, I suppose. So it's very hard to study.

I suppose this is all a plea to say: Prove it to me! (I also suppose this is confirmation bias, but hell, it seems so damn logical.) Where is the proof that Efficiency drops as usage rises? How do we go about studying this? How can we find that point that Dean described, where all the players are using right around the perfect amount of possessions?

[Mike G]

[Kevin Pelton]

I think the biggest obstacle to any study is the natural optimization. When a role player does take on a lot more possessions in a given game, it's probably because he's hot or has a favorable matchup or something that should improve his efficiency as well.

Now maybe if we got to be coaches and were willing to sacrifice some regular-season games so we could force Brent Barry to shoot 20 times a night that would be different, but otherwise I don't think this will ever be "provable." That's the nature of studying something you can't control.

[Dan Rosenbaum]

KevinP's post is very much on the mark. This relationship between usage and efficiency is very hard to estimate empirically. Well, here is a strategy and a fairly advanced econometrics lesson.

We basically have the regression equation.

EFF = B1 + B2*USAGE + e,

where EFF is efficiency, USAGE is possession usage, e is a random error term, and B1 and B2 are regression parameters.

But the problem is that in games where players are more efficient, the coach will encourage them to use more possessions. Thus, there is positive correlation between USAGE and the error term. This will result in an estimate of B2 using data from games during a season to get an estimate of B2 that is too high, i.e. that is upwardly biased.

One trick commonly used in economics is to use a technique called instrumental variables. The idea is to find an instrument - Z - that would be correlated with USAGE (either positive or negative correlation will do), but would not be correlated with the error term, i.e. the part of efficiency not explained by possession usage.

Here is a candidate for an instrument - the average possession usage in the previous season of a player's current season teammates. In computing this average I would weight by the minutes played in the current season.

The logic behind this methodology is that if a player gets stuck with a bunch of teammates who in the past did not use many possessions, he will be forced to up his own possession usage. It allows us to use "natural experiments" in the data where players are forced into new roles.

That is better than just looking at players who change a lot in possession usage, because that change could be caused by the player getting a lot better. Using the previous season possession usage of the teammates gets around that issue.

Puting the data together and running this instrumental variables regression is quite a bit of work, so I am not quite sure when I will be able to do this. (This upcoming semester is going to be very busy for me.) So if anyone wants to try this, the results are likely to be pretty interesting.

[Kevin Pelton]

Dan points out a way to do it season-to-season. We could also look game-to-game at situations when a high-possession player was out of the lineup, for example.

Hopefully, these kinds of situations can provide some solid evidence (whether it proves our theory or not). I don't wish to be overly pessimistic on the issue, but I do think that often the notion of "proof" we're left with from the lab sciences (and social sciences) is something of a burden at times.

[Ben F.]

Well, then, fine. We don't have to "prove" it, but I think it's something very important to study. Ideally, if Dean is right about this, it gives a very interesting lesson on chemistry, how a group of players might function together on the offensive end.

I think it's something that we should look into more, and thus was asking perhaps for ways to look at it (if it's even there). Dan's suggesting was very interesting, and although I understood the logical reasoning, I don't know the nuts and bolts of the statistics, otherwise I'd do the work.

[bchaikin]

Does efficiency drop as usage rises?

we tried this once before, and one of the problems was what defines usage? touches/min? possessions as defined in BoP?....

There's really not a large sample size to be able to view what happens when role players take on possessions and stars lose them.

there's tons of data using touches/min, all you need do is look at players who play significant minutes annually and who's touches/min change alot from one season to the next...

again use iverson as an example. he had his highest touches/min in 04-05 and also one of his most efficient seasons. i just don't know if his "possessions" as defined by BoP were the highest of his career....

[HoopStudies]

Quick comment as I'm out of the country and my girlfriend didn't allow me to bring my computer and only gives me a couple minutes per day to digest the outside world. And it is good that way. Trust me. I needed this vacation.

I think the burden of proof should be on those saying that such an effect doesn't exist. If you've played hoops, you have some sense that you can't be efficient if you try to score all the time. Or if someone really good tries to score all the time.

I haven't tried to "prove" it myself, but I've seen the effect in numerous game level studies. I haven't looked across seasons, which is more what Bob is suggesting. I have run skill curves for different player seasons and seen differences, but usually they are similar from season to season. I remember the break out Elton Brand season (first year with Clips) really bugging me because it was sooo different than other things he'd done. But really that was an anomaly that he hasn't duplicated. Most players don't show big differences.

It is tough to get at because of reasons mentioned by KP and Dan. But there are a lot of different game level studies that see it. Getting skill curves, though, is tougher and obviously something I'm keeping to myself until I can put food on the table doing this job (those were a part of BoP that proved frequently useful this past season).

Uh, gotta go have fun.
_________________
Dean Oliver
Author, Basketball on Paper
The postings are my own & don't necess represent positions, strategies or opinions of employers.

[Dan Rosenbaum]

I think the instrumental variables approach that I am suggesting could work at the game-level or the season-level. It would be interesting to do both.

I had an idea that might be fun. There is the Journal of Quantitative Analysis in Sports that DeanO and Roland are helping start. Done right, I think this would be a nice article for that journal.

http://www.bepress.com/jqas/

I am very busy this Fall and probably only have time to serve as the organizer of this, but I was wondering who might be interested in being part of this project. We probably can't have 10 co-authors, but I suspect that even four or five would not be completely unworkable. Here is what I think needs to be done.

(1) putting together the season-level data
(2) putting together the game-level data
(3) devising the empirical strategy
(4) running the regressions
(5) putting together the tables
(6) writing up the results/writing the paper
(7) revising the paper
(8) organizing the process
(9) handling the submission

With the limited time I have this Fall, I think that I am best employed doing (3), (4), part of (5), some guidance on (6), (7), (8), and (9). If there were folks who were interested in helping with (1), (2), (5), (6), and (7), that would be great.

Let me know either through e-mail, PM, or in this thread if you might be interested. If we dozens of people interested in helping out, we may need to make some choices about whose name goes on the paper, but it would be good to get many people involved in this. (But it would look bad to have a list of co-authors longer than the article itself - although I have seen that done with medical journals.)

[Ben F.]

What response, if any, Dan, have you gotten from this?

[bchaikin]

I think the burden of proof should be on those saying that such an effect doesn't exist.

this is the 2nd time this arguement of "...i don't have the statistical evidence, so guess what? the burden of proof is on someone else who doesn't agree with it..." has been used concerning this supposed tenet of efficiency-falls-off-as-usage-increases. if you are going to make a statement and declare it as fact, shouldn't you then substantiate it with statistical evidence, as again this is a stats analysis group?....

i for one have seen the statistical evidence go both ways on this issue. i can show stats for players whose efficiency decreases with usage (touches/min), and for those whose efficiency increases with usage. but if one is going to claim something as fact, then it is up to that who claims it to provide the evidence for it....

i broached this topic earlier when i mentioned players like allen iverson and derek fisher (both used as examples in BoP), who had increased touches/min in 04-05 and with greater efficiency than they had in recent seasons with less touches/min, but never got a response - just a discussion of how possessions were different than touches/min, but not anything specific on iverson and fisher...

but in using touches/min there are numerous examples that can be looked at...

If you've played hoops, you have some sense that you can't be efficient if you try to score all the time. Or if someone really good tries to score all the time.

i don't doubt this for a second - but to quantify it with stats or statistical analysis is another story. statements like "...well it must be true otherwise team A would give the ball to player B all of the time..." isn't enough imho to quantify it...

This relationship between usage and efficiency is very hard to estimate empirically.

exactly - that is why we use stats, and there are plenty of those to look at historically. allen iverson and dwyane wade and dirk nowitzki and larry hughes increased their touches/min from 03-04 to 04-05 and were more efficient, damon jones and jalen rose and jamal crawford and jason kidd decreased their touches/min and were more efficient. there are multiple examples for both. i just don't know if their possessions as defined by BoP reflect this...

I think it's something that we should look into more, and thus was asking perhaps for ways to look at it (if it's even there). Dan's suggesting was very interesting, and although I understood the logical reasoning, I don't know the nuts and bolts of the statistics, otherwise I'd do the work.

again all player touches/min stats for the past 28 seasons are in the free stats database at www.apbr.org or www.bballsports.com. the evidence is there for anyone who wishes to look at it....

[jkubatko]

First, let me state that this argument is somewhat similar to the argument in baseball that there is no such thing as a clutch hitter. You're going to have people on both sides who refuse to budge one bit, and at the end of the day no progress will be made.

Getting back to basketball, I believe that some players will show little or no loss in efficiency (and perhaps even a gain) with increased usage, while others will see a loss in efficiency. Determining a priori who those players are, however, is another matter.

I decided to examine this issuse by doing a small study. Using true shooting attempts (TSA = FGA + 0.44*FTA) as my measure of usage and true shooting percentage (TS% = 0.5*PTS / TSA) as my measure of efficiency, I found players who met the following crtieria:

1) Were at least 28 years old in the target season. (I thought that players in this age range would be less likely to see an improvement in TS% due to improvement of skills.)

2) Saw an increase of at least 1/3 in their TSA per 40 minutes played from the prior season to the target season.

3) Played at least 1000 minutes in the prior season and target season.

A total of 43 players met the criteria above. Here they are:

[Mike G]

Studies of consecutive player-seasons that don't incorporate some knowledge of Player X's "need to shoot" don't address the causality question: Did he shoot more because he was shooting better? or did he shoot better because he shot more?

Offhand, I'd say unless 80-90% of all major (+25%, say) increases in TSA/min are accompanied by improved TS%, the indication is that just increasing your shots tends to make you miss More of them (all else being equal). In other words, 80-90% of those TSA/min jumps would be the Result of (perceived) better shooting habits/effectiveness, rather than the Cause.

Of course, the opposite would also be true: the great majority of players whose shots and TS% both Drop is explained by their taking fewer shots Because their shot went south, and not vise versa.

[badgerbucco]

Since I'm still in the process of teaching myself statistics, so I can contribute more fully to the analysis here, I'll just stick to what I do know.

I agree with Dean on the burden of proof. In a legal sense, the burden is on the person trying to prove an presumption wrong. For example, in a criminal case, the state has the burden of proof of guilt since the presumption is innocence.

In the law, the presumptions are generally based on policy, such as that we REALLY want to make sure that a person is guilty before we take away his/her liberty. Dean's presumption is based not on policy but on the conventional wisdom that efficiency declines as usage increases. Although there isn't great empirical evidence of this, I think anyone who has played the game feels that it is true. This is as good a reason as any to come up with a presumption.

Without the presumption and burden of proof, you have nothing against which to push. The burden of proof in this case is therefore similar to the scientific method since it essentially sets something up as a hypothesis which the analyst community must work to prove or disprove.

We believe that statistical analysis can disprove conventional wisdom but that fact doesn't mean conventional wisdom has no basis to be believed. Conventional wisdom is based on experience that should be given its due. For every piece of conventional wisdom that is disproven with analysis, there are 10s pieces that are shown to be true. Its track record is better than the credit it is given.

[marc200]

Just joined...I am another economics professor who does applied micro, I actually teach undergrad metrics. This sounds like an interesting idea and I'm familiar with the methods Dan is proposing to use. I don't have that much time this semester but might be able to fit in some work on this. Especially as we move into 06, when I should free up more. I'll email Dan directly. I might also be able to find some undergrad sports fans who would be interested in working on this as a metrics project.

It seems to me using injuries to major high-usage stars as an instrumental variable would also work here. Unless people are being put on injured reserve to give them rest for unimportant games, injuries should be exogenous to the expected skills, "hot hands", or convenient matchups of role players on the team. Injuries should also result in increased usage for other players.

I actually think this could be a better IV than the one Dan proposes, as it seems to me that the usage patterns of new teammates the management brings in could well be endogenous to expected changes in usage by current players. For example, if you think someone is ready to take a step from being a role player to a more central player on the team who gets more touches, you might surround him with people who have more role-type skills. Conversely, if you think someone should be playing a more limited role than they are you might go out and search for a star-type player to replace them as the team's offensive motor. I'm not saying Dan's IV wouldn't work, just that one must be careful.

As Dan knows, applied micro people can argue about endogeniety of instruments all day (or all seminar) long, it's very boring for those outside the tribe...sorry to those who had to suffer through the post...