Causal Friday: Some Real Effects of the Flint Water Crisis…

On Fridays, we examine a research paper that uses (or fails to use) a clever method to perform causal inference, i.e. to tease out cause and effect.

In case you haven’t been keeping up, I’ll start by noting that Flint, Michigan is still having problems with its water supply. (Technically, residents are being told that the lead prevalence is down to acceptable levels but also that they should keep using bottled water anyway for the next 3 years until the pipes are replaced, which, uh…ok.) We know on a general level that lead consumption is bad, but it’s worth thinking about what specific problems can arise, so here’s a fun synopsis on lead poisoning. Oh, and lead is also being blamed for Legionnaries’ Disease in the area, so there’s that. We’re also learning that the water problems appear to have had significant effects on fertility and infant mortality:

Flint changed its public water source in April 2014, increasing lead exposure. The effects of lead in water on fertility and birth outcomes are not well established. Exploiting variation in the timing of births we find fertility rates decreased by 12%, fetal death rates increased by 58% (a selection effect from a culling of the least healthy fetuses), and overall health at birth decreased (from scarring), compared to other cities in Michigan. Given recent efforts to establish a registry of residents exposed, these results suggests women who miscarried, had a stillbirth or had a newborn with health complications should register.

Woah if true. On principle, I’m talking about this study because I feel like this matter needs attention in order for government officials to be willing to act responsibly. As a data analysis matter, I’m talking about this paper because it illustrates an important component of causal analysis. So let’s take a look at one of the pieces of data that the authors used to reach their conclusion:

Ok, that doesn’t look great. BUT…it’s actually not enough information to conclude that the water itself is causing a problem- maybe there was something else going on more generally that resulted in decreasing fertility at the time. (In other words, maybe we’re falling for the post hoc ergo propter hoc fallacy– it’s not just the title of the second episode of The West Wing!) To do a proper analysis, we need a counterfactual- theoretically, a world where everything is the same except there is no water crisis. In practice, a counterfactual approximation is usually constructed by looking at comparable areas that didn’t undergo the “treatment”- in this case, didn’t have a water crisis.

If you looked closely at the above graph, you may have noticed that I cheated in order to fit my narrative- the graph actually looked like this:

Oh. Yeah, that doesn’t look great either for the people of Flint, but it at least looks better from a data analysis perspective. Economists refer to this sort of analysis as “differences-in-differences”- i.e. a comparison of before-after comparisons. In picture form, something like this:

Before After Difference
Treated (i.e. Flint) A B B-A
Untreated (i.e. comparison group) C D D-C

We can then analyze whether the treatment had an effect by investigating whether the incremental difference of the treatment group (B-A) – (D-C) is different from zero. (If the difference is positive, the treatment had a positive effect and vice versa, assuming that larger outcomes are better.) In order to be more rigorous in the analysis, the next logical step would be to test whether this difference in differences is statistically significantly different from zero. To do this, economists run regressions with various interaction terms that get at this “difference in differences.”

Like I’ve said before, causal analysis generally aims to be as close to the middle-school science project as possible- control group, experimental group, the only difference between the groups being the treatment, and so on. In this case, the causal interpretation of the data presented in the paper rests crucially on whether Flint really is like the comparison group in all ways other than the water supply (or at least those ways that can’t be controlled for). In addition, it’s crucial that the treatment that is being analyzed (lead in the water, in this case) is random, meaning that it doesn’t pop up in response to something about the treatment group that researchers can’t observe/quantify.

From what I’ve read about the history of the Flint water crisis, I feel pretty comfortable ruling out that latter concern- in other words, I really don’t think Flint did anything to invite the water crisis that would also affect fertility. (I guess I also don’t think Flint did anything to invite the water crisis more generally, so there’s that.) As to the former concern, the comparison group is comprised of the 15 largest non-Flint cities in Michigan, so they could be different from Flint in various demographic ways that are not controlled for in the picture above. That said, those differences can be controlled for in the regression analysis, which still does find a significant difference in fertility. On the other hand, it could be the case that some of the women who wanted to have children moved out of Flint to do so for precautionary reasons- we wouldn’t be able to see this easily in the data, and it gives an explanation for why fertility rates could have dropped even if the water wasn’t actually making women infertile. While this explanation is initially plausible, we would also have to believe that the women who stayed in Flint were predisposed to have much sicker babies, since there is also an observed difference in infant mortality between Flint and the comparison cities.

The general idea is as follows: use techniques at your disposal to perform causal analysis as best as possible from a mathematical perspective, try to come up with alternative explanations for your findings, and then try to use the data to rule them out. These last two parts can get interesting, largely because different people think of different alternative explanations. For example, the authors of the paper conjecture that people in Flint might just have decided to have less sex rather than leave the area, and they actually use data from the American Time Use Survey to argue that this is not the case, which I personally find hilarious. Overall, I’m not sure whether to feel happy that we’re doing rigorous analysis or depressed that we’re finding that the water supply has significant negative effects.

New Videos! Starting with One on Gains From Trade…

Sooo…I have a minor confession to make- I’ve established a bit of a cottage industry tutoring students in the course I taught while I was in grad school. Not gonna lie, it’s pretty nice to be seen as an advocate as opposed to the thing between the student and the grade that the student wants, in part because students are more willing to admit what they find confusing to me than to their “real” instructors.

As a related project, I figured it would make sense to create videos for the items that students find to be particularly confusing or challenging. The first one is about gains from trade, since we specifically teach that the “price” of a trade has to be right in order to make all parties better off from trade, but we kind of gloss over the fact that a trade also has to be of a size that makes sense for everyone as well.

Hopefully that was helpful! You can see more information on all of the videos here. If you’re an instructor, you might find the Econ 101 Database, listed under “Other Projects”, to be useful as well.

Update: My customer pool doesn’t appear to be dwindling anytime soon. =P

Yet Another Reason We Need a Consistent Definition of “Money”…

Also, as a related matter, never say “give me all of your money” when mugging an economist.

“So, like, do you mean only M1 or do I need to hand over M2 as well? Are you only counting items officially recognized as currency or are you demanding all items that could function as money? Technically speaking, fiat money has no intrinsic value so is there any chance I can convince you that this is not worth your time?”

(Don’t get it? See here for a brief explainer.)

Causal Friday: Fun with Gender Discrimination, Now with More Bad Econometrics…

On Fridays, we examine a research paper that uses (or fails to use) a clever method to perform causal inference, i.e. to tease out cause and effect.

Disclaimer: I’m kind of stretching the definition of both “causal analysis” and “research paper” here, but I guess you could interpret the analysis as relating to the causal impact of being female.

In case you haven’t heard, Google is the target of a class-action lawsuit based on gender discrimination. (Shocking, I know, given what we know about Silicon Valley more generally. =P) Part of the impetus for the lawsuit is an employee-led effort to collect compensation data that shows that men are paid more than women at the company:

At Google, Employee-Led Effort Finds Men Are Paid More Than Women

At Google, Employee-Led Effort Finds Men Are Paid More Than Women

A spreadsheet created by employees to share salary information shows pay for women is falling short of what men make at various levels.


From a data perspective, proving discrimination can be somewhat difficult- for example, we hear the often-quoted “women make 77 cents for every dollar a man makes” statistic, but this in itself doesn’t really tell us anything about discrimination. It could instead be the case that women sort into lower-paying occupations and jobs of their own volition, choose to work fewer hours, and so on. (On the other hand, we can’t rule out the discrimination hypothesis either.)

Ideally, what one would do to look for discrimination would be to compare otherwise equivalent men and women and see whether compensation differences still exist within the matched groups. Mathematically, this is essentially what economists do when they run a regression with “control variables”- variables that suck up the differences that are accounted for by stuff other than gender.

Google employees seem to be up on their applied math, since they put together an analysis so that they could make the following statement:

Based upon its own analysis from January, Google said female employees make 99.7 cents for every dollar a man makes, accounting for factors like location, tenure, job role, level and performance.

On the surface, this seems to suggest that significant gender discrimination just doesn’t show up in the data. BUT…and this is important…this example highlights the difference between doing math and doing data analysis (or, more charitably, data science)- while this conclusion may be mathematically correct, it’s basically a “garbage in, garbage out” use of econometric tools. Simply put, if you’re trying to isolate gender discrimination, you can’t just blindly control for things that themselves are likely the result of gender discrimination! It’d be like looking at the impact of diet on health and using weight as a control variable- sure, you’d get an “all else being equal” sort of result, but it wouldn’t make sense since weight is likely a step in the chain between diet and health outcomes.

In this way, Google tipped its hand quite a bit regarding the particular nature of gender discrimination at the company- if men and women are paid the same once job title and performance reviews are taken into account, then gender discrimination (if it exists) is taking place either by herding women into jobs with different roles/levels or showing anti-female (or pro-male) bias in performance reviews. (Also, if the “levels” have set pay bands, which the article kind of suggests, doesn’t controlling for level largely amount to assuming your conclusion?)

Turns out my suspicions are pretty on point, given the specific claim of the lawsuit:

Google ‘segregates’ women into lower-paying jobs, stifling careers, lawsuit says

Google ‘segregates’ women into lower-paying jobs, stifling careers, lawsuit says

Exclusive: Women say Google denied them promotions, telling the Guardian they were forced into less prestigious jobs despite qualifications


It’s amazing what you can learn from data IF you look at it properly. In a semi-previous life, I worked as an economic consultant, which basically means that I helped prepare expert testimony to be used in lawsuits involving economic matters. What I wouldn’t give to be the expert witness who gets to offer up a rebuttal to Google’s crap econometrics here.

Update: This is amazing:

In case you’re curious, the excerpt is from this book, which I highly recommend.

Just a Couple More Things on Price Gouging and Then I’ll Shut Up (For Now)…

Look, I get it, negative supply shocks suck. They’re not as good as everyone getting what they want at low prices. Sometimes economists are too flippant about high prices as a rationing mechanism. We’ve been over this. I do feel a little like I’m screaming into the void though, especially when I see, um, interesting takes come from places that should know better:

Why Businesses Should Lower Prices During Natural Disasters

Why Businesses Should Lower Prices During Natural Disasters

It helps your customers, which helps your brand.


I’m sorry, come again? Fine, I’ll reserve judgment until I finish reading the article…

Ok done. What the article is saying without being terribly explicit about it is that companies should engage in completely untargeted disaster charity in the form of low prices since it will make customers so happy that they’ll be super loyal afterwards. Maybe it’s just me, but I’m not going to be terribly loyal to a company that made their stuff cheaper so that it sold out before I could get what I needed. To be fair, the article’s recommendation seems to be that companies both lower prices and satisfy whatever level of demand exists at that price, which at best would be very expensive and at worst logistically infeasible.

While I do get that public relations is a thing and that customers aren’t robots, two things still bug me. First, the article asserts that lowering prices and satisfying demand at the lower price would be a low-cost tactic to generate goodwill, but, unless you’re running a zero marginal cost business, it’s really not. (For example, it’s far cheaper to offer free phone service than free plane flights.) Second, it’s far from clear that the rewards in terms of customer loyalty are strong enough to warrant such an investment- in fact, using Jetblue as an example is particularly bad since it’s pretty well known that part of why airline service is so bad is that many customers focus on price to the exclusion of all other considerations. So sure, maybe it would be a nice thing to do, but don’t pretend like it’s long-term profitable without even trying to estimate the costs or benefits.

Moving on…look, I tried to warn you that below equilibrium prices lead to sub-optimal allocation of goods, but you didn’t listen, and now we have this:

Sneaky car dealer takes free Hurricane Irma garage spaces, city says

Sneaky car dealer takes free Hurricane Irma garage spaces, city says

Hollywood opened its garages so people in flood-prone areas could park for free in a dry spot. Many of the spots were quickly filled with cars with price tags and no license plates.


I really hope this snaps some people out of their fantasy world where low prices get goods to the nice, deserving but perhaps not high income people. I guess it also highlights the need for some sort of non-price allocation rule if you’re not going to allocate via price- straight-up rationing is generally not great, but in this case perhaps require a license plate or local address? Geez. (Sidenote: This is where my parents live and let’s just say they are not surprised by this outcome.) Come to think of it, this is even an example of the lowering of prices that the HBR article recommends, but I’m not convinced that the City of Hollywood got a whole lot of positive PR in return for its largesse.

Last but not least, apparently the small slice of the world that is the economics profession is heartless. *headdesk* I don’t think I particularly like being referred to as if I’m a quirky zoo animal or something. Unless it’s a panda, hen I’ll allow it. (Also, the article actually says that the voucher idea I presented as a joke earlier is actually a thing. GUYS, I WAS JUST KIDDING, IT’S MOSTLY ABSURD. Mostly.)