There is a lot of resistance to stats amongst the general basketball consuming public. Well, not stats. New stats. We love the old stats. Points, assists, points…did I say points yet. We just don’t like people telling us what they mean. We like the power to control the narrative and we resent it deeply when nerds with quiet certainty wrest that power away from us. “Why don’t you watch the games?”, is the only counter we can muster in the face of their overwhelming facts.
People hate, absolutely loathe, being proved wrong. These same people have uniquely strong opinions about sports. And those opinions come from a position of knowledge.
Sports are probably the most complicated thing that the general public understands well, and for a long time, it was the subject area where the knowledge of the average follower most closely rivaled that of the experts, the practitioners. I’ve read a little about Afghanistan, but I obviously don’t have anything resembling the comprehension of the place that, say, David Petraeus has. I could though, I’m pretty sure, coach or GM a basketball team better than a lot of the guys who get paid millions to do it. A lot of people could.
So getting proved wrong about sports, a thing we understand nearly as well as we give our selves credit for, makes people crazy. One of the most enthusiastic perpetrators of this insanity is Dave Berri.
Berri –economist, professor, author, columnist, blogger– is the architect of win score, wins produced, wp48, and a host of other handy tools for understanding the why and how of basketball outcomes. A couple months ago he answered some of our questions on the Sixers(fyi, he saw this start coming) and now he’s provided some A’s for our Q’s on some of the nitty gritty of his methods.
Berri makes us crazy after the jump…
1. Are you trying to find a clever way to express man-to-man defense in your formula? Have you considered a system where you determine and include the win score differential between a player and the guy he was covering in each game?
Defense is part of the calculation of Wins Produced (something that is not well understood). But defense is treated as a team activity. So however your team performs on defense is noted in the measurement of your performance (this is similar to what Dean Oliver does in the calculation of Win Shares).
Such an approach does make some sense. Even when teams exclusively played man-to-man defense, it was still the case that team’s had defensive schemes. Now that zones are legal, treating defense as a team activity makes even more sense.
That being said, there are alternatives. Ty Willihnganz – at Courtside Analyst – has measured player performance by noting the Win Score of the player and the Win Score of the opponent at the player’s position. The problem with this approach is that the opponent’s Win Score appears to be inconsistent across time. And this is probably because defense is a team activity (hence returning to our original position). That being said, Ty’s approach is a reasonable alternative if you do not like treating this as team activity.
And if you prefer something else…. I offered another approach to incorporating defense was offered back in 2007. Specifically, I looked at allocating the team defensive factors (i.e. opponent’s field goals made, opponent’s turnovers that are not steals, etc…) according to the individual defensive measures reported at 82games.com. The results didn’t appear much different for the one team I examined. But this is something that might be expanded upon the future.
2. With the “European style” (ie. guys are trained in a uniform way, monolithically skilled, little demarcation in playing style or responsibilities between bigs and littles) gaining a foothold in the NBA, how are you going to adjust position-adjustments? Is it becoming more difficult to assign guys to those positions in the first place?
In evaluating a player one needs to consider position played. Big men tend to rebound and don’t often turn the ball over while little guys (i.e. guards) tend to do the opposite. In measuring performance, this aspect of player productivity needs to be taken into account.
Actually, let me amend that statement. If you create a measure of performance that emphasizes scoring and/or de-emphasizes rebounds, then you don’t need to worry about position played. Such approaches (see the Player Efficiency Rating, NBA Efficiency, Points Created, TENDEX), though, don’t produce results that are highly correlated with team wins.
Wins Produced, though, is correlated with team wins. But it does require that you note that different positions have different responsibilities. Of course, the problem with this is figuring out exactly what position everyone plays. For most players I think this is fairly easy. For example, clearly Shaq and Chris Paul are playing very different positions. But there are players where this is less clear. And when that happens, all you can do is state your position assignment (and note this process is not perfect). If people think this is incorrect, they are free to re-calculate performance with the position they think is correct (I offered a brief post that mentioned this issue on November 1).
3. What are the aspects of a player’s game that seem to be the most affected by coaching or other factors? What are the aspects that are most set in stone?
In our study of coaching we looked at how specific coaches impacted the performance of players. Across 30 years of data – and more than 60 coaches – we only found 14 coaches that had any impact on performance. In other words, the vast majority of coaches had no statistically significant impact on the performance of players.
The study, though, only considered a player’s overall performance. We really didn’t look at how coaching impacts the performance of individual statistics. We can say that most NBA statistics are quite consistent across time. In other words, most stats – on a per-minute basis – have a correlation coefficient of 0.7 or higher (while most stats in football — and most stats in baseball–have correlation coefficients that are lower than 0.5). So it doesn’t appear that coaches have much of an impact on most aspects of performance in basketball. If they did, performance in the NBA would be less consistent (since coaches in the NBA are changed very frequently).
4. We advocated hiring you as a special consultant in a recent post. Have you been offered consulting work by NBA teams? Do you have any desire to move your ideas from theory to application?
Let me discuss the second question first. It is important to note that our work is not theoretical. Our research involves studying actual data from the NBA. So what we are saying has been tested and applied with real world data (and therefore, is not purely theoretical).
As for your first question… one needs to remember that both The Wages of Wins and Stumbling on Wins explicitly criticize decision-making in professional sports (especially the NBA). And I imagine people are not anxious to hire consultants who have publicly criticized their ability to make decisions.
That being said, I have actually had conversations with a number of executives in a variety of professional sports across the past few years (in the NBA I think I have spoken to people on at least six different teams). And I have done some paid consulting in sports. But this is not something I actively pursue.
Essentially, I am willing to answer my phone (in other words, in the past few years executives have contacted me but I have not called them). Executives, though, tend to find me to be an unwilling consultant (or at least, not a very enthusiastic consultant).
Furthermore, I also think the hourly fee I charge (an hourly fee that I charge non-sports entities and that I think is fairly standard for Ph.D.s in economics) to do consulting tends to be higher than teams are willing to pay. From what I understand, the stats people teams hire are not generally paid very well (and earn an hourly wage well below the standard consulting fee economists charge).
5. Why are front offices so resistant to advanced statistical methods of player evaluation like your own? Is that resistance starting to buckle?
I think teams are recognizing that traditional methods of evaluating players are not adequate. So teams increasingly understand that a greater effort has to be made to utilize statistical analysis to evaluate players.
But understanding there is a problem is only the first step in finding a solution. There are a number of statistical methods out there. Metrics like Adjusted Plus-Minus and Player Efficiency Rating are not particularly good measures of player performance.
By the way, for a comment on the problems with adjusted plus-minus see:
For a comment on the problems with PERs see:
Wins Produced is a better measure. But of course… the books where this measure is described also criticizes NBA decision-makers (and again, people tend to get unhappy when they are criticized). So I think that has diminished enthusiasm for Wins Produced.
In the end, I think a big problem teams have is objectively and accurately evaluating statistical measures. If you don’t have much training and/or experience analyzing statistics, then identify which measures actually help is going to be difficult.
The way I have put this to students is that it is a bit like someone who didn’t go to medical school trying to do open-heart surgery. Even if a surgeon is standing behind them giving them advice, if you didn’t go to medical school you really won’t know why you are doing what you the surgeon advises. Similarly, when stats people advise decision-makers in sports, it is hard for the latter to understand the advice. If you don’t understand the math involved in the various metrics, it is hard to understand the advice that is given.
6. Who are the teams who seem to “get it,” or make decisions that are most in line with their interests?
My first reaction to this question is “none”; or at least I am not sure we can identify one team.
But then… well, the Utah Jazz get something. Larry Miller – the long-time owner of the Jazz (who recently passed away)—offered a measure called the “Miller Metric”.
Miller Metric = Points + Rebounds + Steals + Blocked Shots + Assists – Turnovers – Shot Attempts – Personal Fouls
This measure is not tremendously different from Win Score (the simplified version of Wins Produced).
Win Score = PTS + REB + STL + ½*BLK + ½*AST
– FGA – ½*FTA – TO – ½*PF
And I know the Jazz often talk about how players need to focus less on points scored and more on how efficiently shot attempts are utilized.
Unfortunately – and this is a point made in Stumbling on Wins – NBA teams do more than focus on shooting efficiency. In other words, I think NBA decision-makers try and consider “everything”. And that might be the problem.
Here is what we said in Stumbling on Wins (in reference to our analysis of the NBA draft):
It may seem somewhat surprising to hear that decision-makers are better off considering less. This argument can be illustrated if we consider what was uncovered with respect to rebounds. Rebounds don’t impact where a player is chosen on draft day, but are found to be related to future productivity in the NBA. Such results suggest that decision-makers are not aware of the importance of rebounds. Such a suggestion, though, is hard to believe. Rebounds have been tracked for NBA players since 1950 and we can be fairly certain that decision-makers in the NBA understand that better rebounders help teams win games.
We also suspect, though, that decision-makers believe a vast list of factors is connected with winning basketball games. Unfortunately, the size of the list is the problem. People are taught to consider everything before making a decision. Such advice would be good to follow if the human mind had unlimited computing power. The human mind, though, has clear limits. Too much information has actually been shown by researchers to result in declines in the quality of decisions.
We believe this is what’s happening on draft day. Decision-makers try to consider everything, but the limits of the human mind undermine this effort. In order for a decision to be made, the human mind has to simplify the vast list of factors considered. The simplification process ends up emphasizing the factors that are most conspicuous. In other words, the final decision is dominated by scoring, age, height, and Final Four appearances; a list of factors unrelated to future productivity in the NBA.
One can probably expand this argument to all decisions made in the NBA. People in the NBA probably understand that players help when the player shoots efficiently, grab rebounds, and avoid turnovers (the factors Wins Produced emphasizes). But they also probably think a player helps who “looks athletic” or has a “winning attitude” or is able to “take the big shot.” In other words, a host of factors end up being considered.
Ultimately the human mind has to sort through this list and make a decision. And because the list is not examined systematically, the informal decision-making process tends to be biased in favor of scorers. In other words, the one factor that stands out the most when you watch a player ends up dominating the list of factors considered.
One last note on this issue… a number of teams have hired statistical consultants. An incomplete list would include the Denver Nuggets, Houston Rockets, Dallas Mavericks, Boston Celtics, Portland Trail Blazers, New Jersey Nets, Oklahoma City Thunder, and Cleveland Cavaliers. Again, though, the metrics actually employed may not actually help.
And there is still another problem. Even with a statistical consultant on the payroll, there is no real guarantee that the consultant actually impacts decision-making. At times one senses that these consultants are hired to simply give fans the impression that teams are “doing everything they can” to make the right decision. But in the end – consultant or not – the decision reached is essentially the same.