Seems like a step back compared to similar measures as applied to baseball and I would assume that the missing step adds some variation in to the expected winning percentage. Clay Davenport's system does that and comes up with much more logical results.
Now obviously the range between the best and worst baseball teams is absolutely smaller than the range between the best and worst NBA teams, but some of the odds just don't even come close to passing the smell test. And that seems mainly due to lack of imagination in the metric.
Two division leaders at 100% odds to stay that way? Boston with a roughly 4995 out of 5000 chance to win theirs? Chicago at less than a 3 in 5000 chance to win their division or the East? The Bulls are a great example since they started at 3-9 as recently as last season. Orlando (with its 14-6 start in 2007) is another. The Cavs, likewise come out as divison or East champs in fewer than 3 of 5000 simulations.
Colorado and Philly, despite poor records of 14-19 each had a better than 8% chance to eventually make the playoffs and Philly was at ~7% to win their division. This, IMHO, captures the uncertainty of the sport at such an early stage of the season, while still trying to offer something valuable analytically (an idea of how good these teams ACTUALLY are).
Not sure how Hollinger's metric adds anything of value to the discussion, since common sense tells us that teams that are good now will probably finish with good records (and vice versa). Even the basic media narrative (which takes injuries into account) would be a better basis in my book for assessing current vs. predicted team strength.
There are some obvious outliers that regression to the mean alone would probably not explain (Clev/Miami in 2004, Jersey/Chicago/Orlando/Memphis/Houston in 2005, Jersey/Minnesota/GS in 2006, Orlando/Toronto/LAL last year). No metric is going to be perfect here, but I think Hollinger's misses out on a lot by interpreting SSS noise as valuable signal.
Joined: 13 Jan 2005 Posts: 219 Location: Iowa City
Posted: Sat Dec 08, 2007 1:14 pm Post subject:
I agree with your criticisms, but I like how his predictions extend throughout the playoffs. I wish Baseball Prospectus and others would do that. The Celtics having a 60% chance of winning the title does seem problematic though.
I agree with your criticisms, but I like how his predictions extend throughout the playoffs. I wish Baseball Prospectus and others would do that. The Celtics having a 60% chance of winning the title does seem problematic though.
Well, the projector also has them winning 67 games, and being the only 60-win team in the league. I can see how they might be the overwhelming favorite if that happened.
I wonder if the regression to the mean isn't strong enough. It looks modestly reasonable, but not fully reliable...
To me the regression dilemma depends on which part of the equation you're more focused on -- the final W/L record or the % odds. The coolstandings produce a much more compact league, to the point that a team that plays a quarter of the season with the best point diff of all time still barely squeaks past the 60-win barrier, even though at least one team and usually more clears 60 wins every year; meanwhile it say s nobody will win less than 25, which doesn't seem plausible. That's with a quarter-season of data in hand, 10 games ago it was even more compressed. You can say their % odds are more reasonable because they factor in that things can and do change, but I think the W-L is much less reasonable.
That said, both these tools are built more for March than December. The real idea behind this tool is to evaluate what a team's playoff chances are based on differences in remaining schedules; be interesting to see how the two models converge as the year goes on.
Joined: 13 Jan 2005 Posts: 219 Location: Iowa City
Posted: Mon Dec 10, 2007 5:18 pm Post subject:
John Hollinger wrote:
To me the regression dilemma depends on which part of the equation you're more focused on -- the final W/L record or the % odds. The coolstandings produce a much more compact league, to the point that a team that plays a quarter of the season with the best point diff of all time still barely squeaks past the 60-win barrier, even though at least one team and usually more clears 60 wins every year; meanwhile it say s nobody will win less than 25, which doesn't seem plausible. That's with a quarter-season of data in hand, 10 games ago it was even more compressed. You can say their % odds are more reasonable because they factor in that things can and do change, but I think the W-L is much less reasonable.
That said, both these tools are built more for March than December. The real idea behind this tool is to evaluate what a team's playoff chances are based on differences in remaining schedules; be interesting to see how the two models converge as the year goes on.
John, have you considered 5th, and 95th percentiles instead of best and worst? It might be more instructive.
Nobody winning less than 25 might be implausible - but how about Boston with less than a 1 in 5000 chance at finishing with fewer than 57 games? What is your goal at this point if it were a contest? Lowest root mean square error in predicting wins?
The current leader in expected wins would presumably have fewer expected wins than the expected wins of the regular season "champ" (assuming that one team doesn't win 100% of simulated seasons.) That could lead to predicted standings closer to what we're use to even if you don't get that by looking at the expected values of the teams.
I will be visiting the page regularly - it's quite fun. And if the 60% is right, then there's a lot of money to be made in the futures markets...
The coolstandings produce a much more compact league, to the point that a team that plays a quarter of the season with the best point diff of all time still barely squeaks past the 60-win barrier, even though at least one team and usually more clears 60 wins every year;
True, but those teams with the top records invariably have an element of luck involved (obviously they have a lot of talent and quality in addition to the luck). There probably could indeed be more than one team with 60+ wins, but that does not mean that it is good practice to pick several teams to do that. Good predictions this early in the season SHOULD be "compact".
For baseball, all of the predict playoff odds that I see still seem insufficiently regressed to the mean. I haven't looked in detail the the basketball playoff odds, the ones at CoolStandings.com do indeed appear to be too extreme. A lot can, and will, happen between now and the end of the season.
I'm sure there is a lot of variability here, but I don't see that quantified anywhere on the page.
Yeah, that's why we need to do a fairly large amount of regression to the mean. If we KNOW the Celtics will win 80% of their games the rest of the way, it's fairly easy to calculate their expected wins and probability of making the playoffs.
But we don't know that the Celts will win 80% of their games the rest of the way. We can only ESTIMATE what their true probability is.
A good practice when reporting estimates is to report their standard error. With probabilities, an often easier, more straightforward calculation is to regress the estimated probabilities to the mean, with the amount of regression based on the degree of uncertainty about the true probability. If we had zero uncertainty, we'd do no regression at all and use p = 0.80. If we had total uncertainty (e.g. we were dealing with coins that we knew were fair), we'd take the Celtics' "90% heads" and regress it totally, to p = 0.50.
Obviously the optimal estimate will be somewhere in between ... where that optimum is, I don't know, but most of the playoff estimates that I see at CoolStandings (and in baseball, at BaseballProspectus.com) make early-season estimates that seem way too extreme to me, and insufficiently regressed to 0.50.
Joined: 23 Mar 2007 Posts: 111 Location: Charleston, SC
Posted: Tue Dec 11, 2007 12:26 am Post subject:
mtamada, I don't understand what you mean by regressing this data to the mean. I understand the concept of regression to the mean, but what sort of process do you speak of here? I don't want to get this thread off track, so maybe a link or two would help me.
I agree that the regression factor is a key ingredient. For our MLB odds we've gone back to 1903 to tune our model, trying to optimize several parameters, one of them being the regression coefficient and how it changes throughout the season.
For the NBA model we've only gone back a few years, but are currently adding previous years to the model to improve it, while taking into account the scoring changes over the past decades. The regression coefficients for the NBA and MLB are of course quite different.
We look at both expected wins and playoff percentages as criteria, since neither one alone is sufficient. For example, our "dumb" mode assumes every team is a .500 team and has a 53.3% chance of making the playoffs at the start of the season. And at the end of the year, sure enough, 53.3% of all teams made the playoffs! But of course, the expected wins are way off. Our "smart" mode (the default mode) tries to get both the expected wins and playoff odds closer to reality.
At the beginning of the season we use last year's numbers as a starting point, and gradually lessen their impact as the season moves on. At this point there is little residue left from last year, but for the first few games of the season the playoff odds are not accurate for teams undergoing huge roster changes (thank you Danny Ainge!).
It will definitely be interesting to see how our numbers converge with John's as the year goes by.
Do his odds include the probabilities of injuries to star players?
According to Hollinger's odds the Magic have a 100% chance of making the playoffs. What if Dwight Howard has a season ending injury tomorrow? Are they still guaranteed to make the playoffs? Doesn't seem intuitively right to me for any one team to have a 100% chance of making the playoffs this early in the season.
Joined: 13 Jan 2005 Posts: 219 Location: Iowa City
Posted: Tue Dec 11, 2007 10:23 am Post subject:
tenkev wrote:
Do his odds include the probabilities of injuries to star players?
According to Hollinger's odds the Magic have a 100% chance of making the playoffs. What if Dwight Howard has a season ending injury tomorrow? Are they still guaranteed to make the playoffs? Doesn't seem intuitively right to me for any one team to have a 100% chance of making the playoffs this early in the season.
To my mind, that's part of the reason for the regression to the mean. Some argue that Hollinger doesn't go far enough, but nobody's really done the empirical work to suggest what the percentages should be.
mtamada, I don't understand what you mean by regressing this data to the mean. I understand the concept of regression to the mean, but what sort of process do you speak of here? I don't want to get this thread off track, so maybe a link or two would help me.
I plan to discuss regression to the mean in my blog at some point, but I'm not sure when that will be. _________________ Eli W. (formerly John Quincy)
CountTheBasket.com
You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum
FAQ
Search
Memberlist
Usergroups
Register
Profile
Log in to check your private messages
Log in
