When to Control for X
At Vox, Ezra Klein raises an important meta-theoretical point in social science analysis, which is that it is very hard to distinguish mechanisms from controls. If there is a zero-order correlation between X1 and Y, but it drops out when you introduce X2, this in of itself does not tell you if the X1~Y relationship is spurious or if it is mediated. If being shot at is correlated with death, but the effect falls out of significance once you control for exsanguination, one would not say that the probabilistic effect of being shot at on death is spurious. Rather you would say that losing a lot of blood is the mechanism by which being shot at often kills you.
I have been attentive to this issue at least since I read Lieberson’s Making it Count in grad school. In practice this sometimes comes out as fighting with peer reviewers who demand that I throw controls in a model, when the controls are closely related to the mechanism I had posited and so my model would predict that some or all of the effect would drop out when the controls/mechanisms are introduced. So I was glad to see Klein take up this issue at Vox. Klein’s particular case was that when you introduce a lot of controls, a lot of of ascriptive inequality effects drop out. He argues, very plausibly, that this is missing the point since this does not mean there is no effect, only that the effect is mediated by causal pathways.
There are two issues with this.
First, as Klein appreciates, but which I feel may be lost, identifying mechanisms is crucial for crafting a critique and a solution. Or for that matter, deciding whether a solution would be too intrusive and clumsy to be worthwhile. Take the women make 77 cents on the dollar factoid. This is often implicitly or explicitly taken as the boss ashing his cigar, laughing, and saying “A raise? Haven’t you heard? It’s a man’s world.” And this model underlies things like the Ledbetter Act and political mobilization premised on it. But if social science can show us that almost all of the gender gap is really a) a mommy gap and b) occupational sorting, then this is profoundly misleading and all we could expect of the Ledbetter Act is to make a few lawyers rich and maybe at best push the figure from $0.77 to $0.78. Working on the other $0.22 would require stuff like family-friendly workplaces and comparable worth laws, and even that probably would only get you halfway there in the absence of fairly radical changes in gender roles. Personally, I think things like comparable worth laws would be absurdly intrusive manipulations of labor markets that are likely to create innumerable absurdities and distortions, but at least they’d be addressing one of the key mechanisms instead of targeting a caricature of minor contemporary relevance and in practice do little more than give a handout to trial lawyers and an extra bullet point to the Life of Julia slide deck.
As more sophisticated critics of ascriptive inequality appreciate, the vast bulk of the action is structural not discretionary, let alone discretionary and volitional. Indeed under some circumstances they will craft theories that argue that even discretionary inequality is ultimately structural insofar as counterfactual structures would have reduced the scope of discretion. And yet when somebody is writing a story on racism the editor goes for a stock photo of a Klan rally, not a heat map of public transit commuting times to geo-tagged entry level retail positions.
Second, I expect any day now Vox will update this piece “ Debunking the most pervasive myth about black fatherhood, “ which shows that black men are great dads controlling for whether they live with their kids. The article hilariously acknowledges a few paragraphs in that black men are much less likely to live with their kids than white men, but then hand waves at how this is in turn probably caused by mass incarceration. Curiously, the “myth” piece on how the racial disparity in parenting goes away (and maybe even reverses sign) when you control for living with your kids was written after the piece on how it doesn’t matter that most of the racial disparity in criminal justice goes away when you control for offending rates, allocation of policing resources, etc. because controls are better conceptualized as mechanisms. Maybe Lopez doesn’t read the boss’s copy. How embarrassing.
Now it’s tempting to leave it at pundit, voxsplain thyself. However I have a broader point. To paraphrase Schmitt, the intellectual is he who decides on the control variables. Whether we call something a control or a mechanism is not a statistical issue in the sense that there is no R function or Stata postest command that will give you the answer. Rather, distinguishing a control from a mechanism is a theoretical issue. And in practice, theoretical issues are often political issues and so control versus mechanism is yet another instance of our old friend, that depending on how we feel normatively about a factual premise our epistemological standard shifts from “can we believe it” to “must we believe it.” If you find a zero-order association consistent with your worldview, then anything that threatens to explain it away is a mechanism. And if you find a zero-order association inconvenient for your worldview, then anything that promises to explain it away is a control.
