APBRmetrics

[acollard]

[kjb]

Here's one of my encounters with an NBA front office. The team expressed interest in my defensive tracking. I proposed to track the 2nd half of the season and provide written reports every 5th game for less than $10k. They told me that they thought the info was valuable and that it would help them, but that it wasn't in their budget (at least not in their front office/scouting budget area).

So, I offered to do it for less. They poor-mouthed again, and then questioned my commitment to breaking into the league when I refused to do it for free. In the following weeks, they went out and signed a guy to two 10-day contracts, and then for the remainder of the season. Including playoffs, the player was on the floor for 145 minutes and totaled 0.1 win shares and a PER below 10.0.

For those contributions, they paid him more than $100k. I think my data would have helped the team win more games, and it would have cost them very little. Oh well.

[schtevie]

[mtamada]

[schtevie]

Asking any relevant party for an estimate of his own value in terms of his effect on wins (or points scored, allowed, differential, whatever) can easily be formulated in a way (and asked at a time) that it is not disallowed by league rules or directly provides bulletin board fodder. Indeed, what about posing the question to agents who in representing their clients are advocating precisely on these grounds?

I understand why individuals might be uninterested in providing direct answers, but not why the questions aren't publicly asked (or maybe they are?)

Anyway, getting back to the point at hand, I would be quite surprised if the informed belief is that the value of the marginal, competent analyst (from a starting point of none) would be less than one in the win column. And I would be really shocked if that value in terms of revenue - never mind the owner's consumer surplus - were only $270,000. What is the expected profit from just the gate receipts of one playoff home game?

[acollard]

I agree my methods probably weren't best but based on these numbers and the wins that each team had in that year, that's the coefficient I come up with. I apologize I'm not familiar with linking to graphs, but I'd show you the distribution, its all over the place.

Here are the numbers I was using for operating income: http://www.forbes.com/lists/2009/32/basketball-values-09_NBA-Team-Valuations_Income.html

I agree that it probably underestimates marginal wins, and obviously that one win that makes a difference from playoffs to staying home matters a lot. The point I was trying to make is that I think we are biased and over-valuing our own craft a bit.

If profitability is the goal, market, marketing, media rights and arena cost/financing seem to be a bigger deal than winning. If winning is the goal, then sure, get some analysts. But if you really think one guy being paid 70k a year will somehow come up with data that is so much more valuable to a team than the data that already is in the public domain or instinctively known by the coach, I disagree. To create a valuable metric, convince a front office of its value, then convince the coach to implement it, and then get the coach to get the players to do it seems much more than trivial, and that entire process seems like it would cost so much more in terms of time and money than 70k.

[bchaikin]

But if you really think one guy being paid 70k a year will somehow come up with data that is so much more valuable to a team than the data that already is in the public domain or instinctively known by the coach, I disagree. To create a valuable metric, convince a front office of its value, then convince the coach to implement it...

this was during the 02-03 season:

http://www.nytimes.com/2003/04/27/sports/pro-basketball-mavericks-new-math-may-be-an-added-edge.html

and apparently based on what mark cuban has mentioned at the sloan conference the mavs made some lineup decisions based on their data. that was $100,000 for two guys - eight seasons ago...

[gabefarkas]

[schtevie]

Bob, excellent get! I had forgotten about that article. And it, in combination with the recent remarks at the MIT Sloan Conference the essential points can be inferred.

The most interesting and relevant paragraphs for the topic at hand are found bridging pages 3 and 4:

"Sagarin and Winston have tried to sell their system to other teams besides the Mavericks. But only the Seattle SuperSonics have bought it, and they did so for just one month last season.

''It's a real nice system, it's just real expensive,'' Rich Cho, the Sonics' assistant general manager, said. ''And it's only as good as what you make of it. In general, coaches don't have a lot of time to look at the analysis. They're not going to sit there during a game and look at a spreadsheet. They coach by feel a lot.''

Though I am guessing that Rich Cho doesn't think $100k is too much money right now, consider the context of his remarks, being made at the end of the 2002-03 season. The Sonics finished oh-so-close to the last playoff spot in the West. An improvement of a touch over 1 point per game, over the season, and they would have been expected to match Phoenix in strength. And the minimum of two home play-off games and all related income could have been theirs. Though they certainly had a statistical analyst on staff, could WinVal have put them over the hump?

I am not sure of its capacities back then, but cue the tape from last year's Basketball Analytics Panel to the 19:30 mark. Mark Cuban implicitly asks himself the question "what is the value of an analyst?" (or at least a lower bound in terms of a subset of potential services) in discussing the fact that he can tell when teams are using advanced techniques (in the instance, implicitly referring to what WinVal produces) by the basic line-up errors he observes. And just thereafter, at 19:42, Dean says "he is right about that". Cuban's follow-up quote, regarding the consequences of these errors, is: "...nasty, when it comes to results".

Now, maybe I am reading these emphatic remarks incorrectly, but they suggest to me that the opportunity cost of analytics is greater than the aformentioned 1+ point per game, which relative to a 41 win baseline, is about 4 additional expected wins.

I wonder what Rich Cho's opinions are these days about not having cut a check to Winston and Sagarin.

[acollard]

Thanks for sharing that article Bob. I didn't mean to get into this big debate, I just think that this board has a lot of like-minded people who obviously value analytics a lot, and it might be worthwhile to think of it from another prospective. Two things I noticed about the article.

By my understanding, Winston and Sagarin were not full time analysts and they did not have exclusivity rights. They shared a database with the Mavs, but could have shared it with other teams, too. If their data was exclusive or they worked full time for the Mavs, I'm guessing it would be more expensive. In a few years their winval calculations (~2005 when adjusted +/- was on places like 82games) were essentially available in different forms for free.

Secondly, Dallas had the second worse operating income in 2009, and has had a negative operating income for 10 of the last 11 seasons. So they don't necessarily make decisions at that franchise based on profits. I don't think a profit is the end all be all of running a sports franchise, but I also don't think that just because Dallas/Cuban paid someone a certain amount of money, that amount is the market value. Front-office people are paid a lot of money to do their job well, and if it was as easy as hiring an analytic guy and making millions of dollars, I think every team would have a department. My guess is a lot of it comes down to institutional philosophy, if your coach is receptive, and if you want to win vs. make money.

[kjb]

I ran into the same "the coaches won't use it" comments when I briefly consulted for a team. I sorta understood, because the coaches are incredibly busy. And, there are a lot of voices trying to get in the coach's ear and tell him how to do his job.

But, I found that when I could actually get a coach to engage in the conversation, there was learning on both sides. My stat work often challenged the coach's thinking; and his superior experience and hoops knowledge challenged me to dig deeper into the numbers and work harder in the analysis.

Even fairly basic stat work would contradict what a coach believed about a player. Probably because the coach is human and is as subject to confirmation bias as anyone else.

My experience working with coaches is limited, but I've thought that just having a guy on the staff and in the meetings who can test what the coaches are saying and thinking would have value. Perhaps that's part of Roland's roll with Dallas.

[Marver]

If the goal as an analyst is to increase wins (assuming a static player payroll), and solely to increase wins, the sources of that improvement, via analytics, would be one of the following:
- Player acquisitions/roster composition (draft, free agency, trade, etc.)
- Player usage (minutes, lineups)
- Playing emphasis (facets of the game the players should actively stress while on the floor)

With respect to the 'coaches won't use it' comment:
It is very unlikely that a team would hire an analyst only to not value their input regarding the first bullet, roster composition, so I think we can make the fair assumption that the analyst's worth in that area would be fully manifested upon the team. Moving down the list to player usage you reach analyst/coach interaction, though with the proper voices in his ear and sound logic, I would assume the analyst's worth in this area would be at least partially manifested. Finally, you reach an area that I would imagine would be nearly impossible to convince a coach -- any coach -- to use, since it is intrinsically changing how players would move and interact on the floor, ie. how they are coached.

So while I see some limitation to the analyst's worth reaching full manifestation, due to coaching stubbornness, I am somewhat surprised that it would be a factor in hiring an analyst; I believe the overwhelming majority of the analyst's worth is in roster composition.

However, I strongly agree with one of acollard's earlier points about weighing the difficulty of landing (and keeping) an analyst job in the NBA against the subsequent payoff. If you're like me, and I imagine many of you are, jumping through a myriad of hoops to reach an insecure job as another sort of analytic scientist, with the same pay, seems like a strange exercise.

[acollard]

Marver, this comes back to the main problem I have with all this. If hiring an analyst were such a surefire way to increase wins, and coaches/front offices were receptive to the input of an analyst, why don't all teams have analysts working for them? Its a very competitive environment, with a lot of smart people. I just figure there has to be a logical reason about half the teams don't have one (16/30 based on the teams with an analytical staff thread, if you're including consultants).

[Ryan J. Parker]

Here's another way to pose the problem: are there actually 30+ quality analysts hanging around that can do quality work for an NBA team? Like basketball players, analysts have varying skill sets with their own strengths and weaknesses.
_________________
I am a basketball geek.

[kjb]