I was fascinated to read a story this morning discovering that “Mistakes in Scientific Studies Surge.” Research journals are reporting an epidemic of retractions. Scientific researchers are, alas, inclined to the same sorts of statistical errors and silly mistakes the rest of us make and, in an era when academics must publish or perish, the tendency to seize on a significant result, even if it doesn’t pass the smell test, is high.
I’ve been thinking about this because I’m also dealing with the fall-out from a study error, in this case in the area of social science. In my chapter on “The Marginal Cost of Children” in All The Money In The World, I cite a study from Scottish economist Luis Angeles showing that overall life satisfaction rises, among married couples, with the number of children.
This result made a fair amount of intuitive sense. Children do not boost moment-by-moment happiness; changing a diaper is less fun than watching TV. People stay in jobs they don’t like in order to feed, clothe, house and educate their children. But overall life satisfaction is a different matter. Among married couples in a society with widespread access to birth control (Angeles was looking in the UK), having a growing brood (up to the 3-4 kid level Angeles examined) tends to be a sign of a stable marriage, good finances, and a general sense that things are going well enough to bring another child into the world. Why wouldn’t that correlate with overall life satisfaction?
But Angeles later discovered a coding error in his data which removed any statistical significance to this finding, and muddled it completely. So he retracted the paper. I didn’t learn about that until now, so now I’m re-writing the chapter.
I think there is a circumstantial case that parents benefit from having more children (one of the tenets of economist Bryan Caplan’s book Selfish Reasons to Have More Kids; I’m now realizing the fact that Caplan didn’t tout Angeles’s study should have been a red flag!) For starters, people who have fewer children often regret not having more, as shown in this study released recently of male and female scientists at top research universities. There is also the well-documented fact that a strong social network increases happiness. Kids can’t be that network when they are little, but later in life they can be. Given that cost data from the USDA finds that the third child is much cheaper than the first two, this may tip the cost-benefit analysis. And some re-evaluations of the Day Reconstructive Method used to show that childcare doesn’t make people happy have found that, when you take into account what people think about what they are doing, they rate time with kids much higher (see Mathew White and Paul Dolan’s “Accounting for the Richness of Daily Activities,” in the August 2009 issue of Psychological Science, unfortunately not available in full text online).
But, of course, using something like social science data to look at what is a profoundly personal question is always a bit questionable. My own decision to go for a third kid was not about whether, averaged over thousands of individuals, a third child increases life satisfaction. It was more that I think my two little boys are awesome people and while past performance is no indication of future returns, I suspect I’ll like any other kids I have too, and on the margin they will make the world a better place. My main concern with motherhood was that it would come at the price of my writing career; since that has not been the case, that tipped the balance toward a bigger brood. It would be nice if there were social science data to back this up, but life is sometimes too messy to be quantified to a level of statistical significance.