Skating on thin ice

Some of the potential pitfalls of science applied to same-sex parenting.

Walter R. Schumm | Jan 28 2016

- See more at: http://www.mercatornet.com/articles/view/skating-on-thin-ice/17511#sthash.8LJoG8U9.dpuf

My journey dealing with controversial research may have begun with my older brother's dissertation in mechanical engineering. When I was 15, he told me that probably one-third of what scientists had thought they knew about the research topic had been incorrect.

Later researchers, of course, have shown that this is quite wrong. According to some, the true figure is closer to two-thirds.

Even well-known historical events can be deconstructed statistically to show that things did not occur as we have been led to believe. In the RMS Titanic disaster, the lowest survival rates for men and the highest survival rates for children were among the middle class passengers, suggesting a new nonlinear theory of social class and compliance with social rules – and that James Cameron’s Hollywood epic was a bit misleading.

I recently summarised a career-long engagement with controversial research in a wide range of topics in a 40-page article in the journal Comprehensive Psychology.

Readers may have heard of my research on same-sex parenting but may not be aware that I have taken on powerful interests in a number of other areas, which are detailed in the same article. However, here, allow me to present findings on same-sex parenting research. Here are some of the highlights. However, the larger theme behind this paper was how science really works behind the scenes – it’s not as pretty as some might lead you to believe!

Overlooking relevant research

Many literature reviews on same-sex parenting seem to have overlooked a number of relevant research articles. First, despite sworn testimony in some court cases to the contrary, same-sex couples appear to have lower levels of relationship stability if they have children (not so much if they don’t have children) than heterosexual couples. As far as I am aware, no longitudinal research has tested or controlled for the stepfamily effect, which might account for the difference in stability.

Second, despite over a hundred published scholarly comments to the contrary, it does appear that same-sex parents are more likely to have older children who have either experimented with or have identified with same-sex sexual behavior or identity. For example, one recent meta-analysis claimed that they could only find 4 (four) studies that dealt with associations between parental and child sexual orientation, which was remarkable because the same author had found five such studies in an earlier similar meta-analysis. Yet, I found 38 useful studies.

Third, although research is more limited in this area, it does appear in most studies that underage children of same-sex parents are more likely to use substances that are illegal, at least for their age, than are children of heterosexual parents. One 2015 literature review cited Sarantakos (1996) as evidence of the positive or neutral outcomes of same-sex parenting, even though even Herek (2014) recognized otherwise with respect to Sarantakos’s (1996) article. A similar pattern was found for two articles published in the same journal, same data, same authors, same time, same university; the article with results favorable with respect to same-sex parenting was cited many times more often by other scholars than the article with results not as favorable to same-sex parenting. My general conclusion is that many literature reviews appear to have been biased or uneven in coverage with respect to the research they did or did not include in their summaries. My guess is that this may occur more often when research is controversial and has political or legal overtones.

Lack of attention to detail

Sometimes the details really do matter when doing statistical analyses. For example, I challenge every reader to consult their favorite statistician and see if your statistician says the following protocols are statistically optimal. Start with 50 cases with three measures over time. That sounds like a repeated measures analysis over time but, no, the researchers turn it into 150 cases at one point in time. Then they use 50 independent variables to control for the 50 separate cases and another 15-20 variables as independent variables, so the ratio of cases to variables is barely better than 2:1 for their regression analysis. Then they have only one case in which, for their most critical independent variable, the score was one; the other 49 were zero.

If you run a Fisher’s Exact Test with 2/49 having an outcome and 1/1 having the same outcome, the result will not be statistically significant even though the effect size is 1.37 from r = .565. Thus, the situation represents a very low degree of statistical power. When it appeared that there was no difference in dependent variables as a function of that one independent variable, they concluded that that variable made no difference rather than possibly attributing their finding to methodological issues or low statistical power.

This study was used in at least one major court case in the USA involving same-sex marriage laws and their impact on society. If you can find a professional statistician who will swear that turning a repeated measures analysis into three times as many cases, or having a ratio of only 2:1 for cases to variables, or using an independent variable where only one case has any different value than all the other 50 cases are optimal statistical procedures, please let me know! I would be delighted to hear the explanation for why this would be an ideal way to set up or perform statistical analyses.

Lack of statistical power

Rosenfeld in an article in the prestigious Journal of Marriage and Family claimed that his research proved that same-sex parents had as stable relationships if they were married as heterosexual parents, but if you cull out mixed-orientation couples from the heterosexual couples and cull out the nearly 100 couples who were counted as stable because one partner had died during the 4-year study, then there were only four same-sex married parent couples and 488 married heterosexual parent couples with instability rates of 25% and 7.8%, respectively, over the four years of the longitudinal study. 25% is three times greater than 7.8% but not significant because of the low N for the same-sex couples.

Exaggeration

Some amazing estimates of the number of children being raised by same-sex parents have been reported in nearly 100 law reviews or social science articles since 1976. One report estimated that 14 million children had been adopted by same-sex parents when a quick check of the internet suggests that perhaps a million adopted children are living with all U.S. parents today and with only about 100,000 new adoptions a year for all U.S. parents.

One report seemed to suggest that same-sex parents were raising 28 million children in the USA when there are only about 70 million children under age 18 in the USA.

Most recently, Gary Gates of the Williams Institute has estimated that the truth is closer to 200,000 children being raised by same-sex couples. But these far-fetched estimates were seldom challenged and no article of which I am aware focused, as a primary concern, on this issue until 2004 when both Paul Cameron and Lynn Wardle did so in separate reports. This might seem a minor issue to some but at least two U.S. courts considering the legality of same-sex marriage were presented with and seemed to accept as factual such estimates.

Lack of insight

Another interesting discovery was that the best predictor of support for same-sex marriage and parenting in data from the NFSS was not the participant’s sexual orientation, education, political orientation (conservative versus progressive), gender, having been bullied as a child, early quality of family life, or if one’s parents had ever engaged in a same-sex romantic relationship but rather personal support for pornography, one-night stands, and premarital cohabitation (essentially casual sex). That is to say, not some noble ideal of human equality and dignity but rather an ideal of relatively unconstrained human sexuality, not tied to marriage or commitment or long-term dedicated love.

Some might say that, so what – casual sex is a good thing with many positive long-term outcomes. However, those who indicated belief in casual sex also tended to score much worse on many positive indicators, which seemed to contradict the presumed positive benefits of belief in casual sex.

Confusion over significance

Very few of the studies that have looked at outcomes of same-sex versus heterosexual parenting have controlled for background factors or potential social desirability (I provide suggested scales for social desirability in an appendix to the article). A couple of studies featured substantial advantages for same-sex parents in terms of socioeconomic status and mental health and yet when the children were asked about their physical or cognitive competence, the children of the hugely disadvantaged heterosexual parents said they were doing better! One study had claimed that family function was no different between the two types of families but in the article I showed that if education had been controlled statistically, it was likely that same-sex parents would have shown significantly lower reports of family functioning. I also showed how in some studies results were claimed to be significant when they were not and in others, results were claimed to be non-significant when they were significant statistically.

Science and controversy

My overall conclusion was when dealing with areas of controversy, you are more likely to find greater bias and more problematic research protocols and statistics than in less controversial areas. Results that should be challenged are less likely to be, at least for the more popular side of the argument. I think the standard scientific method is at greater risk for abuse the more controversial the area of study. Not only is science vulnerable to “attack” in terms of its credibility because of weak research methods, I note many ways in which those on the less popular side are often treated to ad hominem attacks, even possibly physical attacks (e.g., homes being firebombed).

If you hear a scholar say that there is only one answer and it’s their answer and it’s their way or the highway, you can bet they are probably wrong. Because a wise scholar is unlikely to say they have the only credible research and that everyone else is wrong in every way. For myself I think I have some credible research (as do many other scholars) and that some other scholars have made some mistakes (as have I).

Dr Walter Schumm is Professor of Family Studies in the School of Family Studies and Human Services at Kansas State University.

Note

* Schumm, W. R. (2015) Navigating treacherous waters – one researcher’s 40 years of experience with controversial scientific research. Comprehensive Psychology, 4, 24 (online, open access, 40 pages).

- See more at: http://www.mercatornet.com/articles/view/skating-on-thin-ice/17511#sthash.8LJoG8U9.dpuf

There’s a well-worn saying about statistics that suggests we should put very little faith in them. Yet in the big controversies of the day, numbers really count, even when the people using them apparently cannot.

That is the gist of today’s article by sociologist Walter Schumm on the fraught subject of same-sex parenting. It summarises part of a much longer journal article dealing with other controversial topics as well, and illustrates how scientists can count and weigh the same things differently depending, it appears, on the results they want.

Researchers on both sides of the gay parenting debate have been accused of this, but only those with negative findings have been pilloried in the press. Read Dr Schumm’s piece and see what the other side have been up to.

There's a great video on the front page of the annual Right to Life March in Washington a few days ago. Courtesy of Mary Cooney.