Economists Do It With Models

Warning: “graphic” content…

Bookmark and Share
Things Economists Do When No One’s Looking, Olympics Edition…

March 1st, 2010 · 6 Comments
Fun With Data · Sports

Well here’s a model for ya…economist Daniel Johnson is getting a lot of press for a model that predicts Winter Olympic medal counts by country. A friend sent me the story about it that appeared in Forbes, and even the Freakonomics blog has taken up the cause. From the article:

Over the past five Olympics, from the 2000 Summer Games in Sydney through the 2008 Summer Games in Beijing, Johnson’s model demonstrated 94% accuracy between predicted and actual national medal counts. For gold medal wins, the correlation is 87%.

Sounds impressive! But is it really? I mean, this guy is just basically running a regression to fit medal counts to a function of some available macroeconomic variables, and the reported accuracy refers to how well the model fits the data. I would say that the impressive part is not so much in the analysis itself but rather in the choice of what variables to look at.

His forecast model predicts a country’s Olympic performance using per-capita income (the economic output per person), the nation’s population, its political structure, its climate and the home-field advantage for hosting the Games or living nearby. “It’s just pure economics,” Johnson says. “I know nothing about the athletes. And even if I did, I didn’t include it.”

Spoken like a true economist, for better or for worse. So how did he do this time around? His predictions, from the article:

Johnson, 40, grew up in the Vancouver area and says he’s looking forward to attending the upcoming Games, with the opening ceremonies scheduled for Friday, Feb. 12. But he points out that sentimental attachment has no bearing on his prediction that Canada will lead the medals table when the Olympic flame is extinguished at the conclusion of the 2010 Games. If events play out as his economic model predicts, it would be the first time for Canada to be atop the final medals scoreboard at an Olympic venue. Johnson’s forecast calls for Canadian athletes to collect 27 medals altogether, five of them gold.

The U.S. team, according to Johnson’s economic crystal ball, will hear “The Star-Spangled Banner” played five times at the medal ceremonies in British Columbia, compared with nine times in Torino. Johnson thinks the U.S. contingent will accumulate 26 medals in total at the Vancouver Games, one more than the team’s haul in Torino.

The professor’s forecast also has Norway tying the U.S. at 26 medals overall, with the Norwegians earning four golds, double their Torino take. Among the nations Johnson looked at this time in his calculations, Finland is the only other country he predicts will win more golds in Vancouver (four) than in Torino (none).

Germany, if Johnson’s predictions unfold correctly, won’t lead the Vancouver medal tables as its athletes did in Italy four years ago. German athletes this time around, according to the model, will win 20 medals, seven of them gold (compared with 29 and 11, respectively, in 2006).

And the actual medal counts? Drumroll please…

Hm. In fairness, Johnson was pretty spot on about Canada’s medal count, but since he vastly underpredicted the number of gold medals going to each of the top countries, I have to wonder where he thought all of those gold medals were going to go. (On that note, I am curious as to whether the model is restricted in such a way that the combined predictions actually add up to the total number of medals available.) More importantly, however, Johnson has inadvertently illustrated an important point: Just because you can come up with a model that describes the past accurately doesn’t mean that you can predict the future. Predicting the future is a form of extrapolation, which is something I’ve written about before. (That post is getting a lot of mileage nowadays.) To refresh your memory:

There is nothing actually wrong with Johnson’s analysis- his only problem is that he isn’t in possession of the all important crystal ball. Using his model to predict the future implicitly assumes that the future is going to look like the past. Since we don’t know what the future is going to look like, I would argue that the whole continuing the status quo thing is the best guess we can make, but that guess often turns out to be incorrect.

I doubt that the world is going to implode because this guy’s medal predictions were less than stellar, but this concept of extrapolation is present in a lot of more serious and economically-relevant contexts. (Financial crisis, anyone?) Economic models that describe past behavior are not useless, but their limitations should be understood and acknowledged. On a related note, the next time someone says “oh, you’re an economist…so when is the economy going to turn around?” (this actually happens a lot) they are getting punched in the face. Oh, and yes, I studied computer science, but no, I cannot fix your slow Internet or your busted hard drive.

P.S. I could not resist including the following photo in an Olympic-themed post. I would like to think that I would celebrate in a similar fashion, given the opportunity.

Tags: Fun With Data · Sports

6 responses so far ↓

  • 1 dWj // Mar 1, 2010 at 1:30 pm

    A few months ago I remarked to a macroeconomist who does micro-foundations stuff that “all predictions of the future are out-of-sample extrapolation.” She got a little bit defensive about that, and it does suggest even less justification for economic prediction than is fair; I mostly regard it as justification for the theoretical content she was working on for why this particular out-of-sample extrapolation might be credible anyway.

    A few years ago I thought that most people thought “economics” was just macroeconomics. Now I think that most people think that “economics” is just business cycle theory.

  • 2 Steve // Mar 1, 2010 at 4:23 pm

    He should have also included the number of events in each sport and how successful a country was in those sports. Then extrapolated this to the number of events. perhaps a cumulative index of medals in each sport could be included. I think he wants a proxy for the economic signals which lead to success at the olympics rather than just a prediction model but in regressing it against historical data it suggests that the sports don’t change, but they have significantly over time. you see this in the article as the prediction ability of the model gets weaker.

    I would suggest the reason for the United States success is that a significant number of new sports were invented in America or developed in America. Therefore historical data simply doesn’t cut it

  • 3 Philip // Mar 1, 2010 at 9:41 pm

    Good point Steve,
    I wonder how his model fared at predicting the outcome of only those events that were also contested in Torino.

  • 4 Steve // Mar 2, 2010 at 5:25 am

    or especially only those events at all previous winter olympics. I think it would be a pretty simple regression.

  • 5 Dan L // Mar 2, 2010 at 5:55 pm

    pseudo-scientific garbage

  • 6 econgirl // Mar 2, 2010 at 6:10 pm

    @ dWj: I took your last point and included it in an introductory discussion in my book draft. I think people in general have very little idea of what economists do.

    @ others: Were there really enough new events since Torino for it to matter? *curious*

    @ Dan L: Well, I might put it a little more tactfully, but I *am* curious as to what the (adjusted) R-squared of the regression is.

Leave a Comment