APBRmetrics

kjb · Joined: 03 Jan 2005 Posts: 864 Location: Washington, DC

Ahh heck, you mean we agree?!

Mad

**Dan** **Rosenbaum** · Posted: Thu Jan 13, 2005 4:17 pm Post subject:

Here a little something that I posted on a message board the other day that I think pertains to this discussion the one brought up by Dean in APBR Analysis. It was in reaction to a query about some of my research suggesting that GMs are overly focused on glory statistics.

One advantage of statistics is that it allows a person to "observe" a lot more games. Relying on actual observation can result in biases simply because folks are only able to observe a fraction of the games a statistical analyst observes.

Any analyst, statistical or non-statistical, is bombarded with lots of information. How they organize that information is really important. I think that folks who rely on mostly non-statistical analysis often fall into the trap of being overly impressed by super-impressive plays (the shot that no one else could hit or the rebound that no else could get) and they miss the host of small contributions that one way or another get picked up in statistical analyses. It is not that these scouts are dumb or not well-trained. The problem is they are human and it is very hard to make sense out of hundreds of thousands of observations at games, especially in a dynamic game like basketball. Thus, the super-impressive, I think, gets magnified and the grinding effectiveness that often is critical to winning often gets lost in the shuffle. But statistics can pick this stuff up, not perfectly, but it can shed light on things that GMs and scouts should take a closer look at.

So I do not believe that GMs and scouts are poring over spreadsheets saying they will only pick guys with great glory stats. It's just that I think that is what they tend to remember when they are watching games. So it isn't really something they are doing on purpose, but is a common result of not being systematic in how they are collecting and organizing the information that they observe.

That said, basketball IS a dynamic game that is not captured well by statistics. And I think that will lead to a different dynamic in how statistical analysts are incorporated. I think the statistical consultant model used in baseball, where an analyst sits in his office and pores over numbers, will not be successful in basketball. There is too much that the stats guys need to learn from the scouts and the scouts from the analysts, so I think what will develop in basketball is a much closer relationship between scouts, coaches, and stats guys. This is the Dean Oliver/Seattle model, and I think that is where the NBA is heading. Stats guys who benefit from the perspective of scouts and coaches, and scouts and coaches who benefit from the perspective of systematic analysis - both groups helping guard against the excesses and pitfalls of the other group.

I think it will be critical that teams train their stats guys to be good scouts and help their scouts become more conversant in statistics. I also think this is why Dean is the model for how this can happen. Last summer at the Pre-Draft camp, I was very impressed with Dean not because of his statistical analysis, but because of his scouting ability. He was able to see things in those games that I was not able to see. And I think that helps lead him to better statistical analysis. And also helps him do a better job translating statistical analysis to the unitiated.

This qualitative/quantitative divide is very common in academic circles. What is odd about baseball is how far you can go ignoring the qualitative side. That makes it very, very unusual and it is a result of the isolatedness of performance in baseball. Basketball is much more like the academic circles that I am accustomed to, where real progress is a mix of good qualitative and good quantitative work.

kjb · Joined: 03 Jan 2005 Posts: 864 Location: Washington, DC

Excellent points, Dan. The way I think of it is that very often we as observers get caught up in how a guy looks while he's on the court vs. what he's accomplishing. Haywood is an example of this in Washington. He looks stiff, awkward, uncoordinated, and sometimes downright weak. But he makes the Wizards nearly 10 pts per hundred possessions better defensively when he's on the court. Stats don't care how a guy "looks". It doesn't matter if a guy can jump over a backboard or can't jump over a phone book. The rebound looks the same on a spreadsheet.

Kevin Pelton · Posted: Thu Jan 13, 2005 11:57 pm Post subject:

Kevin Pelton · Posted: Tue Jan 25, 2005 12:30 am Post subject:

Alan Schwarz's follow-up:
http://www.baseballamerica.com/today/features/050123debate2.html

**HoopStudies** · Posted: Wed Jan 26, 2005 2:18 am Post subject: When scouts are better

I've been thinking a lot about what scouts add and how in the world they can compete with stats. Not that I've ever doubted it, but I think it's valuable to think hard about it.

In a recent conversation with some people, I didn't say it very well, I realized. But the one bright spot out of the recent Seattle loss to Minnesota was that it reminded me of one reason scouts are quite helpful at times. Minnesota in that game made a ton of shots that they really shouldn't make. Good scouts see that. They see that Troy Hudson doesn't usually make 28' contested jump shots, but he did that night. They see when loose balls that are up for grabs luckily go one way. These things can also be evaluated statistically, but not easily (no one is tracking loose balls very systematically). A brain that sees a lot of games builds its own statistics without the numbers and knows when watching just one game the value of some plays. In other words, a human brain is getting more out of makes and misses than the the numbers 0 for miss and 1 for make represent.

People know this, I think. But it is important to recognize. There are times when I don't trust numbers, especially when there are small sample sizes. And small sample sizes don't happen just at the start of the season. They happen when a star is injured and everyone else needs to play together in a way they don't usually do so. They happen when a team is playing a lineup it doesn't usually play. Human brains may do better in these small sample situations. Nothing is going to do great, so using as much info as possible is better, but it's just as reasonable in such cases to question the stats as it is to question a scout, even if the numbers completely dispute a scout.

As we collect more stats, the jobs of scouts evolve. I think they must. But they don't dissolve.
_________________
Dean Oliver
Author, Basketball on Paper
The postings are my own & don't necess represent positions, strategies or opinions of employers.

KnickerBlogger · Joined: 30 Dec 2004 Posts: 180

CrazyFromTheHeat · Joined: 21 Jan 2005 Posts: 31

I don't know if someone else has said this yet, but I see the battle of scouts vs. stats as really a question of process vs outcome. If I want to know if Shaq is being successful, I'll look at stats. But if I want to know how to stop Shaq, or how to increase Shaq's productivity, or what type of player might best complement Shaq, I'd ask a scout.

I think baseball has provided enough evidence that what people can see and know in their heart can be far from the truth. Millions of people have watched baseball for a hundred years, and yet you still know that come April, some team will have a leadoff hitter with a low on-base percentage because he runs fast. All of us who pay any attention now know that you want guys who get on base in front of your power hitters, but even though scouts had a century's head start, it was stat geeks who figured it out.
_________________
The Best Miami Heat Coverage
http://heat.mostvaluablenetwork.com/

**Dan** **Rosenbaum** · Posted: Wed Jan 26, 2005 5:41 pm Post subject:

**HoopStudies** · Posted: Wed Jan 26, 2005 6:57 pm Post subject:

CrazyFromTheHeat · Joined: 21 Jan 2005 Posts: 31

[quote="HoopStudies"]

Ed **Küpfer** · Joined: 30 Dec 2004 Posts: 785 Location: Toronto

CrazyFromTheHeat · Joined: 21 Jan 2005 Posts: 31

Ed **Küpfer** · Joined: 30 Dec 2004 Posts: 785 Location: Toronto