This is from a book on climate change, but the principle definitely holds more widely…(click for larger)
(thanks to EconLog for the images!)
This concept- that researchers don’t have the luxury of always running randomized controlled experiments (i.e. randomized controlled trials, or RCTs) to determine what affects what- is what leads a growing number of microeconomic researchers to search for good instrumental variables in order to, to a degree, simulate some form of randomization. In addition, researchers in microeconomics have been increasingly able to go out and perform field experiments in order to have a good deal of control over the data that they create and collect- just ask John List or Esther Duflo, for example.
In macroeconomics, however, it’s hard to even conceive of how experiments to answer most questions would be carried out. Say you want to analyze tax policy- do we tell people that if their social security number ends in an even number then they get a tax break (and that if it ends in an odd number they don’t) and then see what happens? In an immediate sense, I don’t need to do the experiment to tell you what’s going to happen- some combination of rioting and a deluge of of those petitions that the White House has to respond to if they get enough signatures, depending on how much initiative the nation is feeling at the time. Even in an economic sense, though, this experiment wouldn’t be very useful- after all, in order to get a clean result, we’d basically have to have the tax break consumers only interact with one another and the non-tax-break consumers to only interact with one another. (Just imagine the strife this would cause in households where one spouse has an even social security number and the other has an odd one.)
The experimental logistics problem only gets worse when we look to answer questions regarding economic growth- can we form an econ army and take over a country, divide it randomly in two, subject the two parts to different institutions or capital investment and observe what happens over time? Probably not, though the idea does dovetail nicely with my suggestion of an Economist political party. Luckily, we don’t have to, since countries like Korea have pretty much done this for us, in a way, and they provide interesting natural experiments to study.
If you don’t think that being limited to observational data is bad enough, just consider the fact that government economic policy is usually endogenous to the economic situation at hand- people think I’m crazy when I advocate for a government that conducts monetary and fiscal policy in an unpredictable manner, but that would actually be a big win, for research purposes at least. (Insert lame joke about the government seeming to want to help out researchers here.)