Well here’s a model for ya…economist Daniel Johnson is getting a lot of press for a model that predicts Winter Olympic medal counts by country. A friend sent me the story about it that appeared in Forbes, and even the Freakonomics blog has taken up the cause. From the article:
Over the past five Olympics, from the 2000 Summer Games in Sydney through the 2008 Summer Games in Beijing, Johnson’s model demonstrated 94% accuracy between predicted and actual national medal counts. For gold medal wins, the correlation is 87%.
Sounds impressive! But is it really? I mean, this guy is just basically running a regression to fit medal counts to a function of some available macroeconomic variables, and the reported accuracy refers to how well the model fits the data. I would say that the impressive part is not so much in the analysis itself but rather in the choice of what variables to look at.
His forecast model predicts a country’s Olympic performance using per-capita income (the economic output per person), the nation’s population, its political structure, its climate and the home-field advantage for hosting the Games or living nearby. “It’s just pure economics,” Johnson says. “I know nothing about the athletes. And even if I did, I didn’t include it.”
Spoken like a true economist, for better or for worse. So how did he do this time around? His predictions, from the article:
Johnson, 40, grew up in the Vancouver area and says he’s looking forward to attending the upcoming Games, with the opening ceremonies scheduled for Friday, Feb. 12. But he points out that sentimental attachment has no bearing on his prediction that Canada will lead the medals table when the Olympic flame is extinguished at the conclusion of the 2010 Games. If events play out as his economic model predicts, it would be the first time for Canada to be atop the final medals scoreboard at an Olympic venue. Johnson’s forecast calls for Canadian athletes to collect 27 medals altogether, five of them gold.
The U.S. team, according to Johnson’s economic crystal ball, will hear “The Star-Spangled Banner” played five times at the medal ceremonies in British Columbia, compared with nine times in Torino. Johnson thinks the U.S. contingent will accumulate 26 medals in total at the Vancouver Games, one more than the team’s haul in Torino.
The professor’s forecast also has Norway tying the U.S. at 26 medals overall, with the Norwegians earning four golds, double their Torino take. Among the nations Johnson looked at this time in his calculations, Finland is the only other country he predicts will win more golds in Vancouver (four) than in Torino (none).
Germany, if Johnson’s predictions unfold correctly, won’t lead the Vancouver medal tables as its athletes did in Italy four years ago. German athletes this time around, according to the model, will win 20 medals, seven of them gold (compared with 29 and 11, respectively, in 2006).
And the actual medal counts? Drumroll please…
Hm. In fairness, Johnson was pretty spot on about Canada’s medal count, but since he vastly underpredicted the number of gold medals going to each of the top countries, I have to wonder where he thought all of those gold medals were going to go. (On that note, I am curious as to whether the model is restricted in such a way that the combined predictions actually add up to the total number of medals available.) More importantly, however, Johnson has inadvertently illustrated an important point: Just because you can come up with a model that describes the past accurately doesn’t mean that you can predict the future. Predicting the future is a form of extrapolation, which is something I’ve written about before. (That post is getting a lot of mileage nowadays.) To refresh your memory:
There is nothing actually wrong with Johnson’s analysis- his only problem is that he isn’t in possession of the all important crystal ball. Using his model to predict the future implicitly assumes that the future is going to look like the past. Since we don’t know what the future is going to look like, I would argue that the whole continuing the status quo thing is the best guess we can make, but that guess often turns out to be incorrect.
I doubt that the world is going to implode because this guy’s medal predictions were less than stellar, but this concept of extrapolation is present in a lot of more serious and economically-relevant contexts. (Financial crisis, anyone?) Economic models that describe past behavior are not useless, but their limitations should be understood and acknowledged. On a related note, the next time someone says “oh, you’re an economist…so when is the economy going to turn around?” (this actually happens a lot) they are getting punched in the face. Oh, and yes, I studied computer science, but no, I cannot fix your slow Internet or your busted hard drive.
P.S. I could not resist including the following photo in an Olympic-themed post. I would like to think that I would celebrate in a similar fashion, given the opportunity.