Can We Prove That Marriage Improves Behavior?
Amanda Marcotte (via Andrew Sullivan) questions whether the evidence often trotted out for the social desirability of marriage – basically that married people score better on various health, economics and similar indicators – really demonstrates that marriage causes these differences because:
[T]hese kinds of studies lump all nonmarried people into one group. People who are in long term, committed relationships without that piece of paper are put in the same group as people who’ve never held a relationship together.
Marcotte is not quibbling here, but is pointing to a substantial objection. It is a special case of the problem of selection bias – i.e., do married people tend to do better because of the effects of marriage, or because of what kinds of people tend to get married?
Her proposed solution:
I want to see apples to apples comparisons. How do unmarried people who’ve been together for five or 10 years hold up next to people who have been together that long but tied the knot in their first year or two together?
Marcotte’s proposed approach of using long-term cohabitating couples as a control group doesn’t solve the problem. Whether we discover that married couples do better, worse or the same as this control group on some indicator, how do we disentangle the selection bias effect from the causal effect of marriage? By example, assume we find that married couples do worse on average than cohabitating couples for some outcome. How would we know that they would not have done even worse had not the beneficial effects of marriage improved them, and that therefore, marriage actually has a positive causal effect on the outcome? We wouldn’t.
Of course, Marcotte’s selection bias issue has occurred to countless social scientists, and they have therefore created a huge body of analysis that attempts to create exactly such apples-to-apples comparisons, using various case-control matching, regression and other methods. For reasons that I’ve gone into in detail elsewhere, these are all unreliable in evaluating the direct causal link between marriage and outcomes for the couple under consideration, never mind addressing the broader issue of the social effects of a change in the marriage norm on other people outside this couple. There is no practically-available social scientific method that can provide the kind of proof that Marcotte demands in a non-totalitarian society.
But it doesn’t follow that those who advocate continued legal support for traditional marriage are, as Marcotte puts it, “marriage chauvinists.” As I’ve written about elsewhere at length, I think the general principles for political action that flow from such a lack of scientific knowledge on a topic like this are: (1) a loose status quo preference, and (2) a regime of subsidiarity.
Wrong question.
Study the children from married and unmarried couples, perhaps starting with how many.
— Tony Comstock · Dec 9, 10:37 PM · #
There are plenty of other selection biases in Marcotte’s solution. The most obvious is “What qualifies as a long-term, committed relationship?” Is that 3 years? 5? 10? Does it include people who have been dating exclusively for 3 years but live separately? And how do you account for the skewing effect that the difficulty of divorce would make people stay in a bad marriage longer than a bad committed relationship?
Mike
— MBunge · Dec 9, 10:50 PM · #
Changes in welfare payments and regulations regarding unmarried mothers in the early 1960s in some Northern states constitute one naturally-occurring experiment. The rise in illegitimacy rates and in homicide rates both began to accelerate around 1964, suggesting (although, certainly, not proving) that to see the negative effects of a decline in marriage at the bottom of society, you don’t have to wait around a generation for the kids to grow up, you can see it immediately in the behavior of poor young men.
— Steve Sailer · Dec 10, 04:57 AM · #
Tony hit the nail on the head. And when we look at the reams of social science data lo and behold what do we find: children who grow up with two married parents do better on just about any measure you can think of. Plus, as Tony also hints at, married couples (vs. unmarried couples) actually bother to have kids, although in the West at an increasingly low rate. Unforunately, so do unmarried women who are poor or less well-off.
— Jeff Singer · Dec 10, 04:51 PM · #
Anyway, do you think that an America populated by over 50% minorities, specifically black and Hispanic minorities would have given us the internet/web/blogs for you to both be posting your nonsense here? Quick, name all the wonderful contributions (besides food) that Hispanics have made to American culture? Heck, name all the wonderful contributions to world culture from all of Latin America? [Off the top of my head I would say Octavio Paz, Mario Vargas Llosa, the economist Hernando de Soto, and maybe Shakira — just kidding about her).
— Jeff Singer · Dec 10, 05:35 PM · #
Jeff – corn and therefore all corn-derived products.
— Chet · Dec 10, 06:03 PM · #
What the new control group does is rule out one possible mechanism of causation and point you in a new direction. Or marriage holds true as a good thing and then requires a further refinement in controls to search for other possible mechanisms. It’s a never ending cycle that never proves anything… but that’s science. Even controlled experiments are ultimately meaningless. Inductive reasoning is inherently logically flawed.
What you’re trying to do is come up with some sort of rationality based on an epistemology that you refuse to build or contend with. You just want to present the superiority of controlled experiments as inherently better. It would be fine if you didn’t throw around words like “prove.” What you really mean is that some forms of evidence (experimentation) give one more reason to ASSUME causality than others. But you never quite explain why that is. Even given the track record scientific theories in the hard sciences have for getting thrown out and rewritten, you still have faith in extrapolating from controlled experiments. Hell, the example in your article “What Social Science Does—and Doesn’t” deals with assuming the Law of Gravity… even though we both know that newton’s law of gravity doesn’t hold up in all situations. Induction is flawed and when applying inductive reasoning to novel situations, you’ll get nowhere arguing that one form of induction is better or more rational than another.
In short, you are the definition of an engineer.
— Console · Dec 10, 06:16 PM · #
The last paragraph in this post is weird. Manzi suggests that what he’s written before about ceding policy decisions to the status quo and state governments challenges Marcotte’s charge of marriage chauvinism.
How so? Even if Manzi believed in another approach— that, say, federal legislation should sweep in to address most issues, trumping the slower, uneven process of states-as-labs— that would also be irrelevant to Marcotte’s charge.
If she decides that legal marriage is unjustly privileged, then it follows that she sees its supporters as practicing chauvinism, however mild. Perceptions of injustice need not be established in proofs, and measuring them by the status quo is potentially dubious.
Was a suffragist of the nineteenth century wrong to claim chauvinism simply because she was out of step with the status quo preference of her time? Or because state governments had not been given the chance to explore the issue of the woman’s vote?
I realize that Manzi disagrees with Marcotte about the perceived chauvinism. I just don’t think his proposed policy recipe, where proofs can’t be relied upon, has any real bearing on Marcotte’s charge.
— Morly Grey · Dec 10, 07:05 PM · #
Obviously, a democratic society can’t do an RCT here. We can’t compel some people to marry and other people not to marry. The best we can do is try to find natural experiments or do regressions, and there will be irreducible methodological problems with the result.
I think the scientific response is to acknowledge the uncertainty, and do the best with the knowledge we’ve got. Marcotte is a partisan, so she will do this when it is in her interests, and demand absolute certainty when the best knowledge we got suggests facts she would rather are not true. Jim’s trying to be more consistent, but I think he ends up labelling knowledge he likes as good and practical, and reserving methodological criticism for social science or climate science that leads to conclusions his team is against.
— Pithlord · Dec 10, 07:17 PM · #
“Perceptions of injustice need not be established in proofs”
Perceptions of injustice are largely useless without reference to established norms.
Mike
— MBunge · Dec 10, 08:28 PM · #
“Perceptions of injustice are largely useless without reference to established norms.”
Allright, but that doesn’t undermine my comment. Closing out his post, Manzi suggests that where scientific knowledge does not rise to a rigorous standard, we should defer to established norms. In his argument, it “doesn’t follow” that those norms are/could be chauvinistic, because we can’t prove with scientific rigor that such norms unfairly privilege one group over another. Without such proof, how can we know the norms are unjust/chauvinistic?
If perceptions of injustice are only valid when the law says so, then yes, I guess they are simply handmaids of the status quo. But this doesn’t resonate with me. Marcotte’s charge of chauvinism could be a vanguard sentiment of an unfolding consensus on the subject. The law may or may not catch up at some point. Her charge may be impotent or unpersuasive for the time being, but it is not unwarranted simply because it lacks current legal support.
— Morly Grey · Dec 10, 09:50 PM · #
cw,
Are you stalking me? If so, I like it! Anyway, that old post you reproduced here for no good reason was talking about the kind of human accomplishment Charles Murrary cataloged in his famous book of the same name:
http://mangans.blogspot.com/2010/12/human-accomplishment-west-versus-rest.html#comments
Morley,
I’m a proud marriage chauvinist who thinks Marcotte is nuts. Manzi wasn’t responding to Marcotte’s charge of chauvinism, but rather her claim that studies showing the supposedly good effects of marriage are flawed. And further, what Manzi is saying to anyone who is proposing radical social change, is basically — slow down, try it out first (locally), and then get back to me.
— Jeff Singer · Dec 10, 10:26 PM · #
“Marcotte’s charge of chauvinism could be a vanguard sentiment of an unfolding consensus on the subject.”
1. You’re pretty safe is assuming that nothing Marcotte says is a vanguard of anything except whatever ax she’s grinding.
2. The distinction between married and unmarried couples has been around since probably before recorded history. If you’re going to attack that distinction as “chauvanistic”, the burden is on you to make the case and not on others to defend the distinction.
Mike
— MBunge · Dec 10, 10:29 PM · #
Hah! I just remembered! Some years ago, a single, sex-positive feminist, not taken with the treacly eroticism of our fourth film (a couple married 20 years and still very much in love) accused it of/dismissed it as being “propaganda for marriage.”
Good times!
— Tony Comstock · Dec 10, 11:04 PM · #
Jeff singer-
Did I comment on this post? DO I know you? I’m all confused.
— cw · Dec 11, 01:06 AM · #
To Jeff Singer,
You claim, “Manzi wasn’t responding to Marcotte’s charge of chauvinism.” C’mon, man, you can do better than that— read the last paragraph of his post, again.
Maybe what you mean to say is that the chauvinism charge plays a minor role in his reply. This is true, but even where the thrust of his response is concerned, you aren’t reading closely. Manzi is not declaring Marcotte wrong to suspect that many current studies of marriage suffer from selection bias. He is saying that she is in fact right to suspect this. However, she is wrong to think that her proposed fix would overcome it. The selection bias problem in this case is probably insurmountable in a free, humane society.
Yet, in your first reply to Manzi, you defend traditional marriage using social science research that is inevitably plagued by selection bias, showing you missed the essence of the original post entirely.
To MBunge,
1. She may well be an axe-grinder. Manzi’s style is something else— sort of a slow, relentless hone.
2. The question of burden is interesting. If the science of this issue cannot overcome selection bias, then we have no real evidence that marriage is a beneficial institution. So, the notion that a married couple deserves legal privileges— ancient though it is— may be completely unfounded, based on prejudice. Why should it be the skeptic’s burden to prove the accepted wisdom wrong, if supporters never bothered to prove it right to begin with? After all, it’s not the advocate of the unmarried who lobbies for legal advantage. Why should she be obliged to a greater burden of proof, when it’s the marriage advocate who expects continued privilege?
— Morly Grey · Dec 11, 02:49 AM · #
one of the major issues with marriage versus cohabitation is that marriage is much more stable than cohabitation, with the latter having a very high rate of either dissolution or formalization into marriage. thus paradoxically comparing people who have been married for five years to people who have been cohabiting for five years is not an apples to apples comparison since people who have been married for five years are typical of marital formations whereas people who have been cohabiting for five years are unusual for cohabitation formations. what you’d really want to do is use longitudinal data to compare people from the point of union formation moving forward and measure their status five years out (using propensity score matching to isolate the treatment effect on the margin), with the population averaged effect of marriage vs cohabitation being an average across stable unions and dissolved unions (the latter being a higher proportion of cohabitation formations than marital formations).
more generally, whether marriage is causally important was debated for decades but around the mid-1990s social science developed a pretty strong consensus that in the United States marriage is crucial, especially for children. at this point its one of the most established findings in sociology — which is noteworthy since it goes against the ideological priors of the modal sociologist.
— gabriel · Dec 11, 05:00 AM · #
The cost of weddings has gone way up, which works as a barrier to entry to marriage. People used to get married much more cheaply. For example, there’s a church about a ten minute walk from my house that’s basically a medium-sized shack. It’s made out of wood, and can hold maybe 80 people. The only decoration is a small picture of the most famous couple to get married there: Ronald and Nancy Reagan.
These days, a Hollywood movie star and a starlet would never get married in such an unpretentious place.
That’s just one of the self-imposed financial barriers to entry to marriage these days.
— Steve Sailer · Dec 11, 11:05 AM · #
Thanks to everybody for the thoughtful comments.
Tony / Jeff:
As later comments point out, I think the same selection bias issues would undermine analysis of legitimate versus illegitimate children.
Steve,
That does form something like a natural experiment. But it is very much like the natural experiment that Leavitt et al used to “show” that abortion legalization in the 70s caused a crime reduction in the 90s (which you ably attacked years ago). I made this point about both this natural experiment argument, and the contrary argument by Lott that proper analysis of this natural experiment shows the opposite, in a book review in NR. In my upcoming book, I go into a lot of detail on this exact example, and show why “these states did X and other states did not, so we can use this to measure the causal effect of X” is not a reliable method.
Jeff,
That’s a fair, quick characterization.
Morly,
I accept this. The point in my last paragraph is that others disagree with her – so how do we reconcile these competing beliefs in a free society?
I accept that status quo preference is “potentially dubious”, but in my view, it is superior to “the intuition of person X” (even when X is me). The evolutionary process that has produced current arrangements is statistical, glacial and crude, but it is often the best we have. Experiments are valuable because they create reliable knowledge that can trump this in some narrow contexts. Barring this, subsidiarity is valuable because it both encourages evolution and learning, and also provides space for sub-societies that allow people who want to live differently to actually live differently.
Console,
I don’t think that you understood my City Journal article that you reference.
— Jim Manzi · Dec 11, 01:19 PM · #
Jim,
RE: Selection Bias
That’s why I suggested starting with how many. Nature has a selection bias against organisms that don’t reproduce.
— Tony Comstock · Dec 11, 05:47 PM · #
cw,
I apologize — my first comment was supposed to be direct to Chet. I don’t know what I typed your name (Freudian slip?)
Jim,
Do we have any natural experiment studies on kids raised as bastards and those who are not? How would you even create such a study?
— Jeff Singer · Dec 13, 01:03 AM · #
Jeff Singer,
I would be interested to hear how Chet and I overlap in your mind.
By the way, Chet, where have you been?
— cw · Dec 13, 02:18 AM · #
“But it is very much like the natural experiment that Leavitt et al used to “show” that abortion legalization in the 70s caused a crime reduction in the 90s (which you ably attacked years ago).”
No, Levitt’s best-selling abortion-cut-crime theory was generated by his own incompetence, as demonstrated by Foote and Goetz in 2005 when they attempted to replicate his state level analysis and found he’d made two major mistakes. As I pointed out to Levitt in 1999 in Slate, the burden of proof was on him to show why his state level analysis was utterly contradicted by simple national level analysis showing that the homicide rate among 14-17 year olds tripled from the last cohort born before legalization of abortion the first cohort born after.
— Steve Sailer · Dec 13, 04:54 AM · #
These are wonderful! <a href=http://www.caps2011.com>red bull hats</a>Thank you for sharing
— red bull hats · Dec 13, 10:36 AM · #
Steve,
Well aware of the dueling regressions debate between them and others. In the book, and more important the paper on which it is based, Leavitt calls out exactly the natural experiment analysis. In the book it actually gets most of the space.
— Jim Manzi · Dec 13, 01:24 PM · #
Morley Grey http://theamericanscene.com/2010/12/09/can-we-prove-that-marriage-improves-behavior#c033611
Exactly! The status quo is just as much a leap of faith as is a change. Just because slavery, or misogyny, or “faith,” or bourgeois values, or individual-versus-group selection has “stood the test of time” does not demonstrate its superiority. One could argue, quite the contrary.
Choosing to do nothing, change nothing, has no a priori claim to superiority. Just look at the Republicans and health care 2000-2008.
I also know of no empirical proof that distributed power yields superior well-being. The correlation of the rise of large state societies with the rise of well-being is profound. Not proof of causation, but a darned good demonstration of lack of destructive causation…
— Steve Roth · Dec 14, 05:40 AM · #