Causal Friday: The Dumbest Differences-in-Differences Ever, Viral Video Edition…

It’s causal Friday, so I’m poking around in some data trying to make a case for cause and effect. More specifically, I’m drowning in a differences-in-differences analysis trying to construct a proper control group. (In this case, identifying a control group isn’t conceptually difficult, it’s just really annoying to pull the data.)

So what is a differences-in-differences analysis? It’s…well, pretty much exactly what it sounds like- it’s kind of nice when that happens. Here’s a little thing I wrote up a while ago for my team at work (who had more of a data science than social science background), or you can consult Wikipedia. (note: you will never convince me it’s “difference-in-differences” and not “differences-in-differences”) The general principle behind differences-in-differences is that you can’t just do a before-after comparison to identify the effect of an event, since you’d also need to know what the before-afters look like for stuff that wasn’t subjected to the event. For example, consider the following hypothetical question:

Fairlife milk initiated a new marketing campaign at the beginning of 2018, and so far sales of Fairlife milk are 5 percent higher than for the same portion of 2017. Should the marketing campaign be considered a success?

Hopefully it’s at least somewhat intuitive that the answer should be “I dunno, how do the sales of non-Fairlife milk compare to last year?” If milk sales more generally are, say, also up 5 percent, it’s not particularly likely that the marketing campaign is doing much. On the other hand, if sales for the milk industry were generally down compared to last year, the marketing campaign should be viewed much more favorably.

So this is what I was trying to do, but with music sales. Remember this?

Is this the best way ever to quit your job? Marina Shifrin resigns with Kanye West dance video

Is this the best way ever to quit your job? Marina Shifrin resigns with Kanye West dance video

Ever wanted to quit your job? Why don’t you take a leaf out of video producer Marina Shifrin’s book and do it through the medium of “interpretive dance”?

Source: www.telegraph.co.uk/news/newsvideo/viral-video/10344179/Is-this-the-best-way-ever-to-quit-your-job-Marina-Shifrin-resigns-with-Kanye-West-dance-video.html

For context, I’m using this as a motivating example for a larger analysis on music sampling. So I dutifully went through and identified two other songs from the same Kanye album that were about as popular as ‘Gone” before the video above went viral, and then I looked up the sales of all three songs (this is way more annoying than you’d think it should be) before and after the video’s posting date so I could do a very careful and nuanced analysis. Clearly I didn’t think things through, since, well…

This is only for the song used in the viral video, so I don’t technically have a comparison group (yet), but I mean COME ON…nonetheless, I persisted and added my control group:

(I changed the scale of the graph so it looked just wonky rather than useless.) You’ll be pleased to know that my confidence in the video causing a sales bump has not decreased…but let’s calculate some differences in differences anyway (it’s not really possible to run a regression here). So here are the numbers for 4 weeks before and 4 weeks after:

These numbers are pretty easy to interpret- an effect is positive if the differences-in-differences numbers are positive and vice versa. (An effect is nonexistent if the difference is close to zero.) Now I guess technically I should run a test to see whether the differences are *statistically* different from zero, but 1. that’s kind of hard with 3 data points, and 2. I mean come on.

The real punch line in all of this is the fact that the video has been taken down on copyright grounds…I’m, um, not sure you’re doing it right, record label…

Leave a Reply

Your email address will not be published. Required fields are marked *