APBRmetrics

[B Purist]

Statistical analysis is gaining speed on the NBA radar screen. Computer data, rating systems, mathematical representations and the theories of APBRmetricians are altering the way organizations look at the game. Teams are seeing the usefulness of statistical tools as ingredients in decision-making about on-court strategies and roster composition. Researchers are breaking new ground and a number of teams have hired leading analysts.

The movement toward embracing the new formulas and approaches is gradual. The statistical revolution in the NBA must overcome the skepticism and resistance that new ideas always encounter. The obstacles to acceptance will be a featured topic of discussion at the 2007 New England Symposium on Statistics in Sports. On September 29, 2007 at Harvard University, Allan Schwarz will make the first presentation which is titled, “Cant we all just get along: Overcoming the impasses between sports insiders and outsiders. Schwarz describes the insiders (management) vs. the outsiders (academics) as the Hatfields and McCoys. In Dan Rosenbaum’s segment of the program, he asks the question, “are NBA statistical models more irrational than “irrational” decision makers.”

Part of the difficulty is that tens of thousands of 12 year olds can digest box scores and site the deficiencies and unmet needs of NBA teams. Those with basic math skills can manipulate equations and speak to the performance of individual players, teams and coaches. Casual fans can identify problems and there are amateur Red Auerbachs on many a street corner. Youthful enthusiasts grow up to become “experts” that pontificate from bar stools about the strategic nuances of basketball. Many individuals can systematically dissect and interpret data that attributes a loss or losses to sub-optimal coaching. Members of the sports media are becoming more sophisticated, knowledgeable, and powerful as Monday morning quarterbacks. They have access to coaches and organizations and the ability to vilify NBA employees in public forums. Owners, presidents, general managers, coaches and others that are scrutinized have reason to be suspicious and defensive. When the wagons are circled, they can choose to hunker down and place the entire stat community in the category of critics and second guessers. The noise from antagonists can drown out individuals and concepts that deserve to be heard.

Coaches and other basketball professionals have a level of competency with analytics that can cause them to tune out the statistical revolution. Numbers have been a component in decision making from day one. Statistics are routinely used to evaluate a player, the team, and the opposition. When a general manger, coach and scout have different views on a draft candidate, a potential trade, or a free agent signing, statistics are always part of the discussion. Coaches can look at a stat sheet and know with absolute certainty that it tells an imperfect, incomplete or misleading story. In some cases the stat sheet is thrown away. This ability to refute data adds to the confidence that they have a firm grasp on the use of statistics and a belief that they do not need outside help. That logic often leads to a habitual disinclination to speak with members of the analytics community. Overconfidence can cause concepts to be overlooked and underestimated. New ideas can be dismissed without investigation or abandoned too soon, when they are explored.

There are often problems and breakdowns in getting information across the communication barrier. The languages of coaching and metrics are drastically different. One speaks of regression theory, conditional expectations and forecasting models while the other deals in the realm of X Steps for fronting the post and the best positioning for success with staggered screens. Coaches are rarely accomplished authorities on computers, data-mining and laboratory science. There are not many statisticians that qualify as advanced basketball technicians and masters of X’s and O’s. Very few people have ever had the aptitude, enthusiasm and available time to achieve a high level of expertise in both fields. Bridging the divide is made more complicated by the fact there are philosophical differences within each group. Members of the coaching fraternity do not always agree with each other and individuals in the stat community have internal debates about the merits and drawbacks of the “best” formulas.

There are reasons for optimism. The complex relationship that exists between statistical data and the activities of players on the court cannot be disputed. Instances where a coach sees one thing and a numbers person sees something else present opportunities to connect the two worlds and make progress. Occasionally the twain will meet and common ground will put all parties on the same page. That agreement will lead to the acceptance and adoption of new standards of conventional wisdom.

It is easy to know and define how players and teams should perform. The difficulty for coaches is an issue of getting the players to perform and execute in the desired manner. Coaches know that statistics can quantify benefits and point to why something should be done. What they need is a guide on how to accomplish the objective. Coaches wonder where data can aid them with respect to basketball mechanics and elements of instruction and development.

The list of player contributions that do not show up on the stat sheet is growing smaller as the stat world defines and quantifies more elements of the game. New categories of statistics and analysis will help remove barriers to constructive conversations between outside researchers and NBA personnel. The future will defy current norms but it is going to take time.

"Basket Pressure (offense), Ball Pressure (defense), Backboard Pressure (rebounding)"

"More Hoops, More Stops, More Boards = More Wins"
John De Francisco
The Basketball Purist Inc.

BP

[Mountain]

Public analytics focuses on individual and team stats. In between there is some attention to lineups and pairs, etc. A lot can be gained from these levels of inquiry but to bridge gap with coaches I think analytics should also be applied, as you suggest, to plays.

I assume team analysts do this a fair amount with good videotape resources and perhaps knowledge of what plays were run and what the options were and what went right and what went wrong on offensive execution and what the defensive strategy was and what went right or wrong for them. This level of analytics doesnt get to the public from the team and with a few exceptions doesnt get produced by outside observers. It is a crucial level. How players fit into plays and execute plays has to be weighed along with overall individual and team impact stats.

If you broke a play down in whatever number of micro-actions or even just the 5 most important that take some combo of however many skills and physical performances and scored performance of every player on your roster at each of those microsteps maybe analysts after doing this to every play could say:

Running play 27 against the Spurs you probably need a guy with a grade B or better skill level (and consistency) at setting a certain kind of pick and a ballhandler with an A level type of a particular penetration move to produce points at or better than league average efficiency for all options that come off it and either tape shows we have that based on past track records or we don't. And a review of all our plays shows these plays are the best ones we have the talent to run against them.

You probably could score which choreography (order of plays, options used, fakes emphasized) appears to work best at improving results. Work this stuff hard I'd bet you could go pretty far and creating valuable information. Coaches do this but a fully skilled statistician with a competent basketball mind and eye probably could perhaps take some of it further than the coaches have traditionally. I'd guess.

What percent of possessions run close enough to design to score against that called play I don't know. 70%? 50, 30? Certainly improv skills are important and get back to underlying individual productivity and lineup efficiency.

Right now public databases can score plays based on type of shot and location and time of shot but it doesnt address the how you get there as BP states. As a professional that stuff leading up to the shot is crucial. From a far, I'll acknowledge it as another layer that sets their task apart from outsider stat analysis and would be interested to hear more about process getting to the shot but few can catch up fully with them on that. Scoring skills and physical tools that contribute to individual and team results might be easier for outside analysts to add value on to fans. I'd be quite interested in more build up of analytic (and observational) work on that. Good player writeups in guides get into some of that.

[HoopStudies]

[Dan Rosenbaum]

I don't have a lot to add here, but the point that I will be making in my presentation at NESSIS is NBA decision-makers make much better decisions that we sometimes give them credit for.

If you fairly and carefully try to write down a model that approximates their evaluation of players, that model does a pretty damn good job predicting the future - better than many of the models that stats folks, especially those in academia, have written down.

Like John says, there is chorus of people that criticize NBA decision-makers, yet precious little evidence demonstrating that they could do any better. Even with us stats folks, here and in academia, I do not recollect much evidence ever being presented that shows that we better predict the future than NBA decision-makers. And when it is done, it is often a straw man argument projecting a very simple and naive decision-making rule on NBA decision-makers that very poorly characterizes their decisions.

For me the lesson learned from this exercise is that there is a lot that we still don't know. We can make contributions to NBA decision-making, but we fool ourselves if we think we have all the answers or even if we think we clearly have better answers than NBA decision-makers.

In general, the folks in this group have been humble in touting our expertise and this paper, in my opinion, will reinforce the wisdom of that approach.

[Mountain]

Model vs model is interesting but not the whole story. Dan's above comment broadens the topic into the general topic of how good NBA decisionmakers are vs what outsiders think should be done or could do with similar resources. If this is to be discussed it will benefit from detail and rigor.

There are plenty of cases of smart decisionmaking that could be highlighted including unsung ones in favor of that point but there are also a good number of cases where the majority outside view is that the team's decisionmakers didnt do well at the time of the decision. What are the proportions? Will enough data and analysis be presented to offer or support a summary judgment?

If you highlighted the 10 GMs on teams who have done the worst over recent years how many should be regarded as "still good, but somebody had to lose in a tough league" and how many would you say arent good? I'd say at least 6 aren't good by any standard I have information to apply. Of those finishing in the middle 10 are there some cases where others (assistant GMs, consultants or some of the best outsiders) could have done better? I'd think so.

But this doesnt really get us beyond the usual background buzz. With the rise of blogs which provide history on fan feeling, a researcher has some opportunities to collect data on fan reactions to draft picks, free agent signings and trades. Obviously you will find a diversity of opinion and pros and cons in blog discussions probably too diffuse and of uneven quality to interest or sway this discussion but you could look at poll question responses to major decisions. Or at the outside "expert" level, you could target select voices with fuller records and compare their views to actual GM decisions and look at results. You could also compare NBA pay (give by GMs based on forecast of future performance) to average national fantasy basketball draft rankings (fan forecast of the current season). I have wondered if any teams are making casual or serious use of neural network information (say analyzing the real GM trade records for ideas or quality fantasy games for trade volumes to give signals about "movers". (Some sophisicated gamblers are doing it with fan based movie information- star like/dislike, degree of interst in movie titles- to bet on box office and information from sports trading sites could help understand players or just the things that move other bettors.)

Other comparisons could be formulated and tested in future. There might be modest value to comparing the annual NBA GM survey answers to responses from fans collected at roughly same point in time (perhaps prior to GM results release to make fan responses unbiased) to gauge predictive power or compare all-NBA teams from each source with statistical evidence (not that this will settle it but it could add texture).

[mikez]

[B Purist]

Dan-

I will concede that “NBA decision-makers make much better decisions than we sometimes give them credit for.” There are definitely cases where the record warrants clarification and defending. What are the categories of decisions (allocation of cap money, draft, hiring of coaches, trades, free agent signings) where NBA decision makers have been incorrectly criticized and outperformed statistical models.

You talk about a comparison of their (NBA) models to the models of “stats folks.” Examining the results of decisions is a far cry from evaluating the method and process used to arrive at the decision. The models of the stat community are known and precisely described. What are the NBA models that you have “fairly and carefully” written down? NBA organizations and decision makers do not all behave in the same manner. Idiosyncrasies are such that organizations can use different variations of the same formula. Two problem solvers could look at identical data and reach opposing conclusions by attaching more or less importance to a factor or factors. It would be very difficult to obtain accurate information on the internal operations and preferred decision-making methods of thirty organizations.

Speaking in support of what NBA decision makers do well takes the spotlight off their shortcomings and lets bad decision makers off the hook. Stat models attempt to provide value by improving areas where NBA decision makers can do a better job. There are organizations with flawed models that are in desperate need of help.

A comparison of the decision-making models will generate three findings.
A) No appreciable difference
B) Current and conventional NBA methods do a better job
C) New and different statistical models do a better job

Category A is the largest of the three. Time spent on A reinforces that status quo and misses the point. Items B and C are the exceptional cases and deserve the most attention. Talking about B does not promote scientific advancement and progress from the innovations of outsiders. I think that the cases of C outnumber B but changing my mind will not alter the larger point. When you use the term “irrational” decisions, the instances in C are the ones that best fit the criteria. It may only be a small percentage of the time, but emphasis should be placed on areas where it can be shown that NBA decision makers are “irrational“. Your lecture will concentrate on areas A and B but it is C that really needs a spokesperson and endorser. The problems of C represent the best opportunities for NBA decision makers to improve and distinguish themselves. There are outsiders and non-traditional basketball analytics that can contribute to improved decision making. An upgraded process would be more rational.

Any evaluation of decision-making should take the ramifications of the decision into account. There are major decisions and minor decisions. Minor decisions involve very little risk management and hardly ever produce a big score or cause a lot of damage. Accuracy, precision and good judgment are essential for major decisions because they come with sizable positive and/or negative consequences. Errors, oversights and omissions in major decisions can set an NBA organization back for years. When somebody swings for the fences, they need to get the bat on the ball. What does your analysis and comparison of NBA decision making and statistical models say about success with major decisions?

The decision-making models of a steel manufacturer, a hospital, a financial services firm, and almost any other non-sports venture differ from the way the NBA conducts business. NBA teams have a business side and a basketball side that are run by different executives. What are the similarities and differences in how they conduct their diligence and make decisions?

To ensure future success, organizations must be willing to embrace change and cultivate progress through the evolution of their systems. In seeking the ideal formula and striving to out-perform and out-manage their competitors, NBA teams should rigorously explore every reasonable opportunity that each day presents. The exclusion of outsiders is a big mistake. Once upon a time, you were an outsider that knocked on NBA doors and advocated for the adoption of statistical methods that were not be utilized by NBA decision makers.

My post made negative mention of 12 year olds, amateur Red Auerbachs and experts on bar stools. They blow a lot of smoke but once in a blue moon a forward thinking pioneer could rise from the group and become the next John Wooden. Thousands of high school, college, and AAU coaches have devoted their lives to basketball. Could any of these outsiders do a better job than the worst NBA coaches?

Since the start of last season, there have been 9 coaching changes in the NBA and some of the coaches that are still employed are hanging by a thread. It doesn’t take a crystal ball to know that teams will struggle and more coaches will bite the dust every year. What does that say about NBA decision making?

Your presentation scares me Dan. When you pat the NBA decision makers on the back they will love the vote of confidence but your position could hinder and slow the statistical revolution. I sincerely hope that you can find a way to prevent that from happening.

[kjb]

I thought Dan's point was that for there to be a statistical revolution, there has to be a showing that a particular statistical approach is better than business as usual. I'd be interested to see the model that approximates NBA decision-making, though I suspect there's going to be an incredible amount of variation. In my limited interaction with front office execs, I've found them to be extremely thorough in their process of player evaluation -- including the use of various statistical tools.

Talking with these execs about what they're thinking and why they're making certain moves has been educational for me. Before last season, they made a couple free agent signings. They signed these guys not because of their abilities to produce certain stats, but because of their abilities to play certain roles. These free agents had specific skills and temperaments that would fit nicely into the group they already had in place.

On one of the signings, I objected pretty strenuously because every stat said the guy was a bad player. (To be fair to myself, I did tell the team that my objection to signing the player decreased as his salary decreased. They were rumored to be offering him a lot of money; they ended up signing him for very little.) The team wanted him to play a specific role -- play defense and shoot open jumpers. He did exactly that, and had the best season of his career.

The stats were of limited use in this case because the new team was going to ask this player to take on a role that was different than what he'd done in the past. Traditional scouting did a much better job of assessing the impact he'd make on his new team.
_________________
My blog

[Mountain]

Roles ... in plays. 4 Factor roles and unscored tactical roles.

[Mountain]

NBA Coaches and GMs can be evaluated on a number of measures. Here are some that could be researched or if they have been already please share:


1. Percent of first time coaches who do not win 35% of games. Same for GMs.
2. Win % and longevity of coaches given a second contract with same team. Same for GMs.
3. Average and distribution of number of years it takes for a team who fire a coach or GM to get to .500.
4. Average and distribution of number of years it takes for a team with a bottom 5 or 10 offensive or defensive efficiency to get it above average.
5. Average and distribution of number of years that a team stays above .500.
6. Average and distribution of number of playoff series wins for team making playoffs before falling out 3 or more consecutive years.
7. Amount of contracts waived per year over last 2 decades in nominal dollars and % of league payroll.

How current NBA decisionmakers compared on these measures to their counterparts from past decades and other pro sports could be measured.

Results would require thoughtful analysis to evaluate but would give more talking points for the discussion of managerial performance.

[Mountain]

1. Percent of first time coaches who do not win 35% of games.

For coaches starting since 1987 and coaching at least 10 games 34% did not win more than 35% of games in total NBA career. Only 12% got chance to coach past 3rd year and the max was 5, so losses were cut. How many of these were "mistakes"? Some teams weren't salvageable of course.

40% won between 35-50% career. 35% of these got 5 years or more. Were these lucky ones treated fairly or too kindly?

Only 25% of new coaches won 50%+ for career and only 15% over 55%. It is tough biz to win in above average over time for sure but these success rates still seem low and not that impressive for the new coaches or the GMs making the choices.

I am not suggesting that a large share of coaches and GMs are not well-informed and talented. But I am asking if these results don't weaken the case some that they are broadly strong and unfairly criticized. How much better they are than the the talent pools of alternatives is hard to say, some chances are given and the data is incremental.

If a 100 college coaches got NBA jobs would the % who would win 55% be a lot less than 15%? (Given the overall record of new NBA coaches, expecting 50% or higher success of college coaches going NBA would be unrealistic standard.) If informed fans picked the coach would they pick winners far less than NBA GMs? This is half in jest but there isnt a lot of room to go much lower, at least on picking such a strong coaching success story.


Related to 3. Of the teams below .500 in 97-98 they averaged 4 years to get to .500+. Some did it in 2 additional years or less, some 6 or more years. I didnt tabulate the number of preceeding years of below .500 play but it would add to length in a number of cases. I think saying some seatholders do not grade well on turnaround management is reasonably supported.


4. Number of years it takes for a team with a bottom 5 or offensive or defensive efficiency to get it above average. In a very small sample to get a first look, I checked 7 teams who appeared in bottom 5 on offensive or defensive efficiency in 01-02 or 02-03. The average time to get above average was a bit over 2 years. That's reasonable and shows abillity to fix problems, Though in 3 of the 14 cases the team was still below average on that efficiency measure after 4 or 5 years. And 20-30% of teams yo-yoed between fixing one side of the ball and ending up with a large weakness on the other side of the ball.

5. Average number of years that a team stays above .500.
Thius may not be representative year but of teams above .500 in 97-98 the average streak above .500 was a little short of 8 years. That is very impressive for the best GMs and coaches. But this achievement benefits from the quality gap to the lower half. When folks criticize NBA decisionmakers they are referring mostly to the bottom 1/3rd or half.


7. Amount of contracts waived- 6 of top 25 salaries last season have been waived at a cost of over $150 million. Second max contracts are vulnerable to that.

[B Purist]

On-court and Off-court Decisions

The titles and job descriptions of NBA executives (Pres, GM) and coaches define different roles and organizational responsibilities that are interconnected. Coaches and executives co-exist in a complex relationship that has boundaries and gray areas. Chemistry can be god or bad. Some are progressive thinkers and others are old school. A focus to build for the future or to win now can help one group and harm the other.

Decisions can be separated into categories of On-court (Coach) and Off-court (Executive) with each side providing a wish list for the other. Coaches want senior management to upgrade the skill, depth and flexibility of rosters to fit their plans and programs. Executives want to see rosters shaped and utilized to deliver maximum progress in the form of player development, better execution and more wins. The GM sets the table, the coach serves the meal, and then they judge each other.

Some management teams (executive and coach) are joined at the hip and some are not. In the best of cases (highly efficient), there can be instances where one or the other walks on eggshells. Dysfunctional marriages fail and on occasion some succeed. The two decision makers can logically or illogically have different agendas and flourish harmoniously. Individuals on the same page can battle contentiously. Personalities that are more dominant may have less positional power and influence in the hierarchy. Third parties can interfere or smooth the way and the quality of relationships with owners and star players can have major bearing.

On and off-court decisions can be analyzed before, during and after they are made. When rosters are finalized and injuries are not a consideration, “qualified” experts and prognosticators can do a fine job of predicting how many games a team should win in a season. Teams will meet, fall short of, or surpass reasonable expectations. At year-end, internal and external reviewers can grade on-court decisions, off-court decisions and the methods of forecasters. Good bad or indifferent, owners that keep their finger on the pulse might compare the performance of their decision makers to the rest of the league. Results that are surprisingly positive or negative warrant special attention. There are gains to be made from determining how and why groups underachieve and overachieve. Analysis can be used to allocate credit and blame.

[B Purist]

Problems With the Advantages of NBA Decision Making Models

My prior reference of Dan Rosenbaum’s NESSIS presentation, did not give its complete title, THE POT CALLING THE KETTLE BLACK: ARE NBA STATISTICAL MODELS MORE IRRATIONAL THAN “IRRATIONAL” DECISION-MAKERS?

Comparing NBA decision-making models to the stat models of outsiders is unfair. NBA models have many advantages over the stat models of outsiders and should do a better job. NBA decision makers have an “insider’s” advantage. NBA staffs are supposed to know their own personnel better than every other organization in the league. They have insights into why stats attached to one of their players can be flawed. Familiarity enables them to do a good job predicting the future development of someone on their roster. NBA insiders have access to tools and information that outsiders are not privy too. NBA organizations draw on input from departments and professionals that excel at their assigned tasks.

The position that NBA decision makers outperform models in the stat community is too general. It excludes the fact that some NBA organizations and decision makers are behind the times and do not fare well. Stat models are fighting an uphill battle but NBA decision makers are not always right. There are cases where NBA decision making models can be correctly labeled as “irrational”. Good stat models can provide better answers.

I may be way off base here, but I am not on a crusade. The pot calling the kettle black strikes me negatively and could be viewed as harsh and excessive rhetoric. It could adversely impact a statistical revolution by making it harder for outsiders to gain a foothold.

NBA decision makers are a tough nut to crack. Experts in the field possess a wealth of knowledge, experience and proven records of accomplishment. NBA decision makers have quality people and systems in place and they are not easily dazzled by science. NBA staffs have seen and heard almost everything in basketball. Almost is the operative word because it dispels the belief that there are no secrets in basketball. New and better methods do exist and the future will bring more sophisticated tools and advancements. An NBA decision maker that is alert and at the ready can beat his competitors to the punch and gain a competitive advantage that results in a home run.

Many an old dog that is set in his ways will not bother exerting the energy to attempt to learn new tricks. If the old dog is complimented on his approach, he will be even less open to ideas that are contrary to his own. Any thing or anyone that lobbies against quality inputs from outsiders is an impediment to progress and positive change. I hope Dan’s presentation does not slow the road to acceptance.

[Mountain]

The Rockets will be the best case yet of outsider with power (and assistants) and new methods though my impression is that Adelman has traditionally run his own show and I am not sure how receptive to advice from new methods he will be for on court decisionmaking. But it will still take work to see Morey and Adelman affects separately.

I talked previously about Morey possibly using a value/cost expectation ratio with short Aaron Brooks. Stats based analysis might be giving various data supporting the value of smart energy bigs (Hayes, Lee, Biedrins, Garbojsa, Varejao, Moore, the young Detroit bigs, Milsap, etc.) in the league and Scola also fits that.

Unclear to me how heavy Presti is on truly new methods or if he is just a new force rigorously working a little of everything including all the old methods. A computerized draft database in itself isnt really that new or significant unless traits are being formally correlated with success and followed to a large degree.

Dallas and Denver are the teams without the fewest 4 Factor flaws in the league. I assume their consultants may have helped with that in some manner but can't know from outside and the traditionalists could be similarly motivated. They werent much different from year before. Denver, aided by return of Nene, improved FG% allowed to remove it as a weakness.

Dallas' eliimination of big flaw on eFG% allowed coincided with Nash's depature (and Walker and Jamison) and return to more playing time for 7 footers (Bradley and LaFrentz which also worked in 02-03 and 00-01. Stat analysis presumably very involved with all this. Not sure what they thought would make the 03-04 lineup work better (seems like a repeat of the 99-00 failure when they injected several more "scorers" with Ceballos and Strickland). But they changed dramatically in response to that playoff failure back to more of what had worked in 02-03 by adding Dampier then Diop. Run n gun with no defense doesnt work. Phoenix has paired Nash leading the offense with a defense that while not strong is at least only slightly below average.

Cavs let go or reduced playing time for everyone with a negative raw +/- in 05-06 but these were fringe players anyways. Looking at adjusted +/- from 05-06 you might think Snow would persist on the court and Gooden and Marshall's time should shrink. Snow only shunk a little and Marshall shrunk a moderate amount but Gooden persisted (though trade whispers at out there). 2 of 3 consistent with that metric but how much decision was influenced by the stats and how much it was just based on contract reality/politics of that and lack of bench alternatives or trade opportunities I don't know. There wasnt a strong adjusted +/- season signal for Pavlovic but maybe near neutral was considered a good mark for his stage of development or splits or something else added to the case for a bigger role this season. Or scouting or both.

So many questions can't be answered from outside when insiders don't reveal the real details often. Sometimes parts of the story come out. Mostly we are left to make our own assessments.

[Dan Rosenbaum]

I think I may have overstated how rigorous the modeling of NBA decision-making will be in the paper. We propose three statistical player evaluation models based upon (1) minutes per game, (2) points per game, and (3) NBA efficiency per minute - all with team adjustments - as proxies for NBA decision-making.

We often talk about the limitations of these crude proxies of NBA decision-making, but there is very little evidence that more sophisticated metrics do that much better predicting things like future team wins or future adjusted plus/minus statistics. This paper will examine that evidence and the upshot is that these crude proxies do pretty damn well. As my earlier work in this forum has shown, Wins Produced, in particular does particularly poorly compared to these three metrics, but the point of the paper will be broader than just picking on Wins Produced.

John (BP) has expressed reservations about whether a paper like this is good for basketball analytics. If what we are doing is so fragile that it cannot stand up to some self-evaluation, then we are in a whole heap of trouble. Long-term progress is not made by going around patting ourselves on the back and hiding our limitations. Instead I think progress is made by rigorous and careful (and when possible public) examination of the evidence.

I am sorry if I there are later posts that I don't respond to, but I am really squeezed for time with my new job.