Economists Do It With Models

Warning: “graphic” content…

Bookmark and Share
Causal Friday: On The Limitations Of Experimental Research…

December 19th, 2014 · 2 Comments
Causal Friday · Just For Fun

Apparently Randall Munroe also has a thing for causal Friday:

On one hand, randomized experiments with a treatment and a control group are a very clean and effective way to determine the effects of a treatment. On the other hand, addressing the logistical caveats necessary in order to get a proper result is not always possible, as shown above. So let’s ruin the joke:

  • Blind trials- trials where the subjects don’t know whether they are receiving the treatment or a sugar pill- are important in order to control for the placebo effect– after all, you wouldn’t want to falsely attribute cardiovascular benefits to sex when they could actually be caused by the belief that you’re having sex!
  • Double-blind trials (now we’re getting fancy)- trials where neither the subjects nor those administering the experiments know whether a subject is receiving the treatment or a sugar pill- are important to control for the possibility that the experimenters inadvertently treat subjects differently based on whether they are receiving the treatment or the placebo. If a trial isn’t double-blind, researchers can’t rule out the possibility that the observed effect is due to the differential treatment on the part of the experiment administrators and not the treatment itself.

Technically, we don’t know whether the experiment in the cartoon is in fact double-blind, since, well, we don’t know whether the experimenters can tell who they are giving lot of sex to and who they are giving a placebo. (So many bad ways to envision this experiment…) There, joke ruined. But I’ll try to make it up to you…

Obviously, not very trial can be double-blind or even blind, but sometimes scientists can be super creepy in order to achieve scientific validity. For example, I bring you…wait for it…placebo surgeries!

A total of 180 patients who had osteoarthritis of the knee were randomly assigned (with their consent) to one of three groups. The first had a standard arthroscopic procedure, and the second had lavage. The third, however, had sham surgery. They had an incision, and a procedure was faked so that they didn’t know that they actually had nothing done. Then the incision was closed.

The results were stunning. Those who had the actual procedures did no better than those who had the sham surgery. They all improved the same amount. The results were all in people’s heads.

I guess I’m in favor of this, though I think that I’d be so pissed if I was in the placebo group…upon further reflection, however, I guess I wouldn’t really know so I wouldn’t have a reason to be mad, so now my brain hurts a little just thinking about it. (Perhaps it was the headache resulting from pondering the placebo surgery philosophy that distracted people from the knee pain!) That said, I definitely don’t want to think about how researchers could make this sort of experiment double-blind.

Tags: Causal Friday · Just For Fun

2 responses so far ↓

  • 1 Phil Shaffer // Dec 21, 2014 at 12:43 pm

    Jodi – precisely the same study design was done for another medial procedure – vertebropalsty. In this procedure old people who are having very painful compression fractures in the spine have bone cement injected into the vertebrae. Same result as above – sham surgery and real surgery had the same amount of pain relief. I pulled the author aside and asked if this meant they were no longer doing vertebroplasties. They were continuing to do them. Hmmm.

  • 2 Anonymous // Dec 29, 2014 at 10:48 am

    “So many bad ways to envision this experiment…”

    Couldn’t they just use some sort of visual barrier…?

    *ba DUM KRISH*

Leave a Comment