Posted on

Causal Friday: The Dumbest Differences-in-Differences Ever, Viral Video Edition…

It’s causal Friday, so I’m poking around in some data trying to make a case for cause and effect. More specifically, I’m drowning in a differences-in-differences analysis trying to construct a proper control group. (In this case, identifying a control group isn’t conceptually difficult, it’s just really annoying to pull the data.)

So what is a differences-in-differences analysis? It’s…well, pretty much exactly what it sounds like- it’s kind of nice when that happens. Here’s a little thing I wrote up a while ago for my team at work (who had more of a data science than social science background), or you can consult Wikipedia. (note: you will never convince me it’s “difference-in-differences” and not “differences-in-differences”) The general principle behind differences-in-differences is that you can’t just do a before-after comparison to identify the effect of an event, since you’d also need to know what the before-afters look like for stuff that wasn’t subjected to the event. For example, consider the following hypothetical question:

Fairlife milk initiated a new marketing campaign at the beginning of 2018, and so far sales of Fairlife milk are 5 percent higher than for the same portion of 2017. Should the marketing campaign be considered a success?

Hopefully it’s at least somewhat intuitive that the answer should be “I dunno, how do the sales of non-Fairlife milk compare to last year?” If milk sales more generally are, say, also up 5 percent, it’s not particularly likely that the marketing campaign is doing much. On the other hand, if sales for the milk industry were generally down compared to last year, the marketing campaign should be viewed much more favorably.

So this is what I was trying to do, but with music sales. Remember this?

Is this the best way ever to quit your job? Marina Shifrin resigns with Kanye West dance video

Is this the best way ever to quit your job? Marina Shifrin resigns with Kanye West dance video

Ever wanted to quit your job? Why don’t you take a leaf out of video producer Marina Shifrin’s book and do it through the medium of “interpretive dance”?

Source: www.telegraph.co.uk/news/newsvideo/viral-video/10344179/Is-this-the-best-way-ever-to-quit-your-job-Marina-Shifrin-resigns-with-Kanye-West-dance-video.html

For context, I’m using this as a motivating example for a larger analysis on music sampling. So I dutifully went through and identified two other songs from the same Kanye album that were about as popular as ‘Gone” before the video above went viral, and then I looked up the sales of all three songs (this is way more annoying than you’d think it should be) before and after the video’s posting date so I could do a very careful and nuanced analysis. Clearly I didn’t think things through, since, well…

This is only for the song used in the viral video, so I don’t technically have a comparison group (yet), but I mean COME ON…nonetheless, I persisted and added my control group:

(I changed the scale of the graph so it looked just wonky rather than useless.) You’ll be pleased to know that my confidence in the video causing a sales bump has not decreased…but let’s calculate some differences in differences anyway (it’s not really possible to run a regression here). So here are the numbers for 4 weeks before and 4 weeks after:

These numbers are pretty easy to interpret- an effect is positive if the differences-in-differences numbers are positive and vice versa. (An effect is nonexistent if the difference is close to zero.) Now I guess technically I should run a test to see whether the differences are *statistically* different from zero, but 1. that’s kind of hard with 3 data points, and 2. I mean come on.

The real punch line in all of this is the fact that the video has been taken down on copyright grounds…I’m, um, not sure you’re doing it right, record label…

Posted on

Experimental Pitfalls Are Generally Dumber Than We Imagine, Now With More Dogs

Ok, story time, so go grab your blankie and a beverage…

A number of years ago, I taught an econ tutorial entitled “Sex, Drugs, and Rock & Roll”…it was a neat class- as the name would suggest, we looked at papers on birth control, drug legalization, the music industry, and so on. For reasons I’m still not sure I understand, it was a somewhat peculiar group of students- basically the football team, the kid who as far as I could tell tutored the football team, and a female transfer from Tulane (because of hurricane Katrina). (I actually confirmed with her beforehand that she was ok with the gender ratio, given the subject matter…turns out she wasn’t easily fazed and later explained to everyone how to make meth.)

Anyway, I digress…the first paper I assigned was authored by a combination of friends and classmates- it wasn’t so much directly relevant to the subject matter but it has “Sex, Drugs, and Rock & Roll” in the title so I figured it would be a good place to start getting students used to reading academic research. The discussion…well, didn’t go as I expected.

Basically, the football team came in and explained that it was funny to read the paper because they participated in the experiment. So, PSA: if you use the CLER Lab at Harvard your subject pool is the Harvard football team I guess. (I don’t mean to imply this is positive or negative, just that it might affect generalizability.) Given this information, I found it far more interesting to have them talk about their experience than to jump right into talking about the results of the paper. The main thing I wanted to know was essentially “did you take the task seriously and think things through?”

The responses were “meh” at best, from what I remember. I pushed back, asking something along the lines of “but you get paid more if you performed better at the task, did that not serve as an incentive?” The response was basically this:

Again, I’m not criticizing the students- I completely believe that effort isn’t costless even when a person doesn’t have anything better to do, so it’s entirely possible that the kids were in fact maximizing their utility. But it gave me something interesting to think about regarding experimental design and how to interpret experimental results. In general, economists are wary of asking hypothetical questions about preferences and behaviors, since there’s good evidence that what people say they would do or what they think they would do doesn’t always coincide with the choices that they actually make. To mitigate this problem, experimental economists try to give subjects tasks where payoffs depend on behavior (usually in addition to giving a payment for showing up), since this supposedly gives the subjects proper incentives to treat the task as though it’s real. Economists have even been known to do their experiments in developing countries, since that enables the research budgets to create stakes that are more real than trivial.

The discussion with my students makes me think that we need to be more critical regarding whether our experiments give sufficient incentives to elicit true behaviors and preferences. In addition, economists could probably do more to assess both during and after the experiments how seriously the subjects took their assigned task. In any case, the bottom line is that the success of an experiment depends crucially on subjects taking the task seriously and behaving as expected in a logistical sense, and these characteristics can’t be taken for granted.

Or, put more succinctly, here’s a helpful warning:

I’ve been watching this all day and I can’t stop laughing. (probably related: Gizmo looks a little worried at the moment) For context, there’s this thing going around Twitter where people try to see whether their dogs understand object permanence by holding up a sheet and then running away as it falls. I mean, what could possibly go wrong with this clearly rock solid experimental design?

Update: There’s a point brought up in one of the comments below that I think warrants discussion…I certainly didn’t intend for the takeaway from this post to be “experiments are bad, burn them,” but I see how it could be taken that way. If anything, I’m suggesting the opposite, so allow me to explain. If subjects aren’t fully engaged in an experiment, this probably biases the experiment towards “no effect” rather than creating effects where none exist. In the dog scenario, for example, maybe the chihuahua does in fact understand object permanence, but we can’t document the phenomenon because the dog’s behavior caused the experiment to be garbage. Therefore, I’m not advocating for questioning results so much as questioning “non results.” We talk about a “replication crisis” in economics, where a surprisingly large percentage of documented effects can’t be re-documented in subsequent experiments. These numbers are general used to call the initial findings into questions, and I guess I’m suggesting that we should go easier on this suspicion until we can confirm that such lack of replicability isn’t driven by the sorts of issues I describe above. I guess I’m also trying to help researchers make their budgets go further, since publication bias means that non results often just get shoved in the file drawer, never to see the light of day.

Posted on

How Monopsonies Work, Now With More Numbers…

You guys, I love monopsonies…for starters, it’s a fun word to say, and it makes people do a double take when they initially think you said “monopoly.” Definitionally, a monopsony is a market with only one buyer- basically the economic counterpart to a monopoly, which is a market with only one seller. Not surprisingly, the buyer in a monopsony has all of the market power in much the same way that the seller in a monopoly does, and largely for a mathematically analogous reason.

Economists have been talking about monopsonies more than usual as of late, most often in the context of labor markets. In labor markets, companies are the buyers (or, perhaps more accurately given the 13th amendment, the renters) and people are the sellers of labor. When the number of employers in an area competing for workers is small, firms are generally thought to have “monopsony power.” This is a little different from literally being a monopsony, but you can think of monopsony power as the ability to act like a monopsony but to a lesser degree.

So what’s interesting about monopsonies in labor markets? Here’s a quick summary:

  • In competitive markets, i.e. markets with large numbers of companies and workers, minimum wages are thought to decrease employment.
  • If a labor market is a monopsony (or if employers have monopsony power), minimum wages could actually increase rather than decrease employment.

This distinction is obviously important from a policy perspective, since minimum wages look way more appealing if they don’t cause people to get laid off. Now, I could try to explain to you why minimum wages work this way using some fun graphs and whatnot, but I’ve come to learn that not everyone’s brain wants to operate at an abstract conceptual level, so let’s work through a simple example instead.

Ready? I’m gonna try to do this without using any econ-specific terms (they’re not as fun to say as monopsony), and we’re going to talk about a company that makes widgets- yes, I know this is a tired econ thing, but Widget is the name of my mom’s cat so I’m asserting it’s fair game.

First, here are some numbers on how many widgets workers can produce:

You’ll note that the “incremental widgets” numbers are decreasing as more workers are added- this is generally reasonable, since most production faces a diminishing returns or “too many cooks in the kitchen” problem. Now let’s say that the company can sell as many widgets as it wants at a price of $10 each- now we can see how much each worker would add to the company’s revenue (does revenue count as an econ term?)

If we want to figure out how many workers the company wants to hire, we need to know how much the workers cost. The thing about workers is that some of them will work more cheaply than others (even if they’re of equivalent quality), and this is relevant for a monopsony, since they’re not so small that they can increase their hiring without pushing up the going market wage. (In fact, this feature is the crux of why monopsonies look different from competitive labor markets!) Let’s assume the wage required to get each worker to work looks like this: (you can think of everything in per hour terms, per day, whatever, it doesn’t really matter)

We can now calculate how much it would cost to hire 1 worker, 2 workers, etc., and we can also calculate the incremental cost of each additional worker:

I feel like this requires some explanation since it’s not really intuitive…if you want to hire the first worker, things are pretty simple, but if you want to hire the second worker, you not only have to pay the second worker the $20 to get him to work, but you also have to pay the first guy $20 rather than $10- there are a number of reasons for this, but you can just imagine that the first guy will get really pissed if he makes less money than second guy for doing the same job. This means that the incremental cost of hiring an additional worker includes not only his wage but also the additional compensation that has to be given to everyone who came before. (This is seriously the most important point so keep staring at it until it sinks in. I’ll wait.)

So take a look at the relevant numbers so far- how many workers does the company want to hire?

Does the company want to hire the first worker? Sure, since that worker brings in $120 and only costs $10. (just assume that labor is the only cost of production) Does the company want to hire the second worker? Sure, since the second worker brings in an additional $100 but only adds $30 to the company’s costs. If we keep analyzing in this way, we can conclude that the company will want to stop hiring after it hires 3 workers. (and will pay a wage of $30)

Now how do things change if a minimum wage is imposed? Let’s assume that a minimum wage of $40 is set. (Now I look real generous to those people who interpreted the numbers as being per hour.)

Workers 1, 2, 3, and 4 are happy to work for the minimum wage, so the company can hire up to 4 workers at $40 each. After 4 workers are hired, however, things change dramatically- in order to hire the 5th guy, the company not only has to pay him $50 but also has to pay each of the 4 other workers $50 rather than $40. How many workers does the company want to hire now? (This is the last set of numbers, I swear.)

Now the company would find it profitable to hire the first 4 workers but not the 5th worker. BOOM- I just showed you a case where a minimum wage would increase rather than decrease the number of workers hired.

So why did this happen? Put simply, the minimum wage took away the “well if I hire you I have to pay everyone else more too” feature of the company’s analysis, at least for the first 4 workers. That said, the minimum wage still creates somewhat of a double-edged sword in terms of employment, since it also made it incrementally more expensive to hire the 5th guy than it was before. As a result, minimum wages placed on monopsonistic markets can both encourage companies to hire more workers and discourage companies from setting wages just above the minimum wage. (This is relevant for companies that pay just above what would be the minimum wage, since it could actually cause them to decrease wages!)

I’m not trying to convince you that minimum wages are good, I’m just walking you through some numbers so that you don’t do the “I took one econ class once and models of competitive markets are always applicable” thing that causes economists to call you an ass clown behind your back. One last thing- in case you (understandably) have the follow-up question of “how prevalent is monopsony power in labor markets?”, here’s some reading material for you to peruse. Spoiler alert: the answer is “too prevalent to ignore.”

Update: You can see a version of this story on Medium if you prefer.

Posted on

Economist Beverages Just Got A Major Upgrade…

Every Friday at about 5pm or so, Justin Wolfers declares it beer o’clock on Twitter:

This is nice and all (and, honestly, a helpful reminder to maybe stop working even though I generally ignore it), but I’ve always felt like we could do better with the econ-associated beverages. And it turns out we can…

Maybe, as a behavioral economist, I’m biased, but this is just so good. Also, here’s the reference in case you’re not familiar, so you even get some weekend reading material with your weekend booze suggestions.

Posted on

July 4th May Be Over, But The Fireworks Discussions Continue…

Okay I know we just talked about fireworks, at least indirectly, but I think we need to have another conversation since i’m not convinced that we’re doing them right:

I guess the good news is that my concerns regarding free riding leading to underprovision of fireworks are pretty much abated. The bad news is that I’m not sure people have caught on to the whole nonrival consumption thing. Here’s the basic idea: a good is nonrival in consumption (or has low rivalry in consumption, or is simply nonrival) if one person consuming the good doesn’t prevent others from consuming the same unit of the good. In this instance, this is a fancy way way of saying “you can all watch the same fireworks my eyes don’t somehow absorb them and prevent your ocular enjoyment.”

When goods are rival in consumption (the opposite of nonrival, as you would expect), we determine how many of the good the market wants via a process of “horizontal addition.” (This makes physical sense if you take a quick look at a demand curve.) Let’s say that there’s a market that consists of me and Mr. Econgirl, and at a price of $5 we each want to consumer 1 ice-cream cone. This means that market demand is 2 at a price of $5 since we can’t both fully consume the same ice cone. (Even if we physically could…well, ew. I guess my dad shared a Dairy Queen cone with our rottweiler once and didn’t die but I’m still not recommending it.) But, as I said, this isn’t how nonrival goods work.

Let’s say instead that Mr. Econgirl and I are each willing to pay $5 to watch a fireworks display. (He’s not, but let’s pretend.) This is sort of a hypothetical payment since obviously we would both try to free ride, so it’s more accurate to say that we would each get $5 of benefits from watching a fireworks display. This implies that the market values a fireworks display at $10 (arrived at via “vertical addition,” in case you’re curious), in which case it’s efficient to provide a fireworks display if it costs $10 or less.

This last point gets at why I”m so annoyed at the individual fireworks people- if you believe that a fireworks display that costs $10 is cooler than 2 fireworks displays that each cost $5 (I’m not an expert but am pretty confident that this is true), then having each house do their own fireworks is an incredibly inefficient use of resources. I mean, wouldn’t you rather all coordinate and have this?

I guess there are a couple of explanations for the outcome that we see. One is simply that the households don’t want to coordinate- either they just don’t feel like it or are worried about the free-rider problem creeping up. Another, more interesting, potential explanation is that the households get utility from producing the fireworks, whereas I’ve been assuming that only consumption is enjoyable. (probably related: I have no desire to set off fireworks myself) In general, economic models don’t take this possibility into account, but they probably should, at least in some cases. For example, if we take into account the fact that musicians specifically like producing music, a lot of what we see in the music industry coincides far more nicely with our economic models. Unfortunately, one of the things that becomes explainable is the fact that prices pretty quickly get driven to zero. But yay, free* fireworks at least?

* do not start with the “there’s no such thing as free fireworks here” since the thing about nonrival goods is they do in fact have zero marginal cost

Posted on

So Do We Still Have Unions Now or No? Economics Can (Sort of) Answer That!

Yes, I know titles of this form are kind of stupid. No, I don’t care. Instead, I find it super interesting how many seemingly unrelated matters actually have a fairly substantial economic component. Sometimes there are multiple economic components, and here’s a discussion of one:

So Do We Still Have Unions Now or No? Economics Can (Sort of) Answer That!

So Do We Still Have Unions Now or No? Economics Can (Sort of) Answer That!

Earlier this week, the Supreme Court released a number of decisions, including one that can basically be summarized as follows:

Source: medium.com/@jodiecongirl/so-do-we-still-have-unions-now-or-no-economics-can-sort-of-answer-that-7454e4aa0e24

Along similar lines, I’ll give you a bonus and just leave this here:

Posted on

I Told You My TV Watching Habits Extend Past Law And Order…

Ok, something you should know about my family…they *love* Jeopardy! No seriously, my parents have basically watched it every day for…well, ever. And they’re pretty good at it- I think the fact that my dad has read the entire 1988 World Book Encyclopedia (which they got me as a gift, sigh) at least once…and in order, not “I feel like looking up penguins today.” I’m ok at it I guess, but I doubt I could do well enough on the online test to actually make it on the show. In contrast, some of you may have met Jarret- he helped me with research stuff and at conferences for a while:

Anyway, he won money whereas I sat at home yelling “how can you not know what a spinnaker is” at the television.

Given this background, it’s not particularly surprising to get the following from mom:

She’s right in that yep, there was an economics category a few days ago (and yes there is a comprehensive online archive you’re welcome), though looking at the questions I’m not sure how much I specifically had to do with my parents’ success. (I swear I don’t just lecture them about economics at family dinners. Well, not always.) I was curious to see how the players did, however, since there’s a great video that seems to no longer be available where, after the first clue in an economics category, the players just stared at the board and didn’t answer correctly for the rest of the category. (I think it was this game, but the visual was so much better.) This episode was also pretty good:

(I don’t understand how people who are perfectly fine with memorizing almanacs are scared of a silly little econ category.) I’ll let you just look through the clues, but let me just say that it started pretty much the way I expected, given the other instance I was familiar with. It’s actually pretty interesting to see the clues for a field that you know well, since it gives good insight into how the clues are constructed and chosen. In related news, I’m curious how many economics-related clues ended up in here:

I assume this is also what my advisor did with paper proposals he didn’t find interesting.

Posted on

How Not to Sound Like an Idiot About the Trade Deficit, Part 1

Today (I guess technically yesterday) in “I just can’t”…

So, uh, we gotta talk, because that’s not even remotely close to how it works. Here’s me talking a bit:

How Not to Sound Like an Idiot About the Trade Deficit, Part 1

How Not to Sound Like an Idiot About the Trade Deficit, Part 1

Ok, we need to talk about the trade balance…not so much whether trade deficits are bad, since I fear that we’re not even ready for that…

Source: medium.com/@jodiecongirl/how-not-to-sound-like-an-idiot-about-the-trade-deficit-part-1-746f2fbee376

Posted on

Economic Consulting, Or “Why If You Think You Can Make $750 An Hour Cleaning Data You Will Be Very Disappointed”…

If you’ve been following the news, you’ve likely noticed that there’s a pending lawsuit against Harvard University regarding discrimination against Asian-American students in the admissions process. I’m going to do a post on the substance of the case soon, but for now I want to talk a bit about the case logistics. (If you’re going to read about the case, take my usual disclaimer of “the media is not always great at reporting scientific findings properly” with you.)

More specifically, people in my circles are discussing David Card’s $750 billing rate, so let’s talk about “economic consulting.” (note: this has a particular meaning and is different from “economist who happens to be acting as a consultant”) “Economic consulting” most often translates to “litigation support”- damages calculations for product liability cases, impact reports for M&A, etc. Most of the time this involves putting an expert in the courtroom, and it’s thought to be helpful if they sound fancy, so they work with high-profile academics and such to be the actual experts. In fact, the first part of the expert reports is usually a pretty salesy bio of the expert, followed by how much the expert is getting paid- hence the knowledge of the $750 figure. (Expert reports generally aren’t made easily publicly available, but Card posted his in this instance, which is why people are aware of the number.)

It’s important to know that Card is not sitting there getting paid $750 an hour to clean data, run regressions, and whatever other tasks are involved in producing expert testimony. Instead, Card is affiliated with an economic consulting firm called Cornerstone Research, so they were probably involved in much of this work. (I’m pretty sure this has been acknowledged publicly in the expert report and/or other.) The expert billing rate is specific to the expert, however, so it’s likely not the case that the client is paying $750 an hour for most of the work done. ($750 also doesn’t seem unreasonable for Card’s level of fancy- one of the experts I worked for in grad school wasn’t nearly as fancy as Card and billed at $675.) That said, Card’s contract with the firm could take a number of forms (eg. does he get a percent of billings) and generally depends crucially on whether he brought the business to the firm or vice versa.

The “client,” in most cases, is one of parties in the lawsuit, meaning that the expert isn’t sought out or assigned by the judge in the case. As such, there are generally two experts for a given case, and each expert’s mandate is to make a case for their side. (This is similar to the principle that it’s not a lawyer’s job to make the opposition’s case for them, but it’s a little more awkward if they think the opposition is right than it would be for a lawyer.) Here’s another fun part- technically speaking, only what the expert knows is “discoverable” in court, so technically the consultants could do all of the analysis and just not share results that are inconvenient to the case with the expert. I would hope this doesn’t happen very often just on principle, but I don’t have good data. Also, this leads to some interesting questions as to at what point in the process the actual expert is brought in, since it’s not really possible to know beforehand what the answer is going to look like when the question involves actual data.

Speaking as one of the economic consultants that supported the experts, the tasks can be really boring or pretty interesting- for example, how would you try to show that banks knew about Enron’s troubles and kept lending to them anyway? (My contention is that it involves interest-rate spreads. Oh, and a PSA: if you ever take such a job, make sure you are familiar with how to run a Hausman test.) Some tasks are just, well, weird…one time I had to call a phone sex line to try to reverse engineer how its automated cell center technologies function- for the record, I was not mad about the task so much as I was about the fact that we were representing a huge patent troll.

Here’s an interesting article about one such economic consulting firm…I’m not endorsing the tone per se, but there’s a lot of detail here on the history of the industry and whatnot:

These Professors Make More Than a Thousand Bucks an Hour Peddling Mega-Mergers — ProPublica

These Professors Make More Than a Thousand Bucks an Hour Peddling Mega-Mergers — ProPublica

The economists are leveraging their academic prestige with secret reports justifying corporate concentration. Their predictions are often wrong and consumers pay the price.

Source: www.propublica.org/article/these-professors-make-more-than-thousand-bucks-hour-peddling-mega-mergers

Lastly, and perhaps most hilariously, remember when I said that what the expert doesn’t see generally isn’t discoverable to the opposing side? Well, the reverse is also true- all relevant work product must be provided to opposing counsel upon request. But…you know those episodes of Law and Order where Michael Cutter requests a document and gets 30 boxes in response along with a flippant “well it’s in there somewhere”? That practice has a nerd equivalent, and it’s pretty amazing (click for the full thread):

I’ll state for the record that I didn’t do any of this, but I guess that’s partly because I didn’t think of it. Also, there really should be a Law and Order spinoff that focuses on the economic consultants, right after we get CSI: Regression Analysis off the ground.

Posted on

Adventures In Misleading Data, Housing Markets Edition…

So there’s a new study out that apparently has a good PR team, since I saw all sort of articles with such headlines today:

Well that’s alarming…except it’s likely not true, unless you define “afford” as “can’t rent a place that’s more expensive than 40 percent of similar places without spending more than 30 percent of your income.” (The 40th percentile thing seems to come from how HUD calculates “fair market rents.”)

Anyway, I rant about it more here:

Not to alarm you, but here’s a headline that’s been popping up a bunch:

Not to alarm you, but here’s a headline that’s been popping up a bunch:

Anywhere? Are you sure? Because, not gonna lie, my backup plan for when I get sick of the world is to go take a minimum wage job and live…

Source: medium.com/@jodiecongirl/not-to-alarm-you-but-heres-a-headline-that-s-been-popping-up-a-bunch-9246716a963e

…and yes, I’m mad about forgetting to put a title on the article and don’t want to talk about it.