Several papers have been quick off the mark to run claims that abortion increases the risk of mental health problems based on a new study by Priscilla Coleman, Professor of Human Development and Family Studies at Bowling Green State University, which has been published this week by the British Journal of Psychiatry (BJPsych).

As you might expect, given that the study claims, amongst other things, that women who had undergone an abortion exhibited an 81% increase in their risk of developing a mental health problem and that close on 10% of the overall incidence of mental health problems is directly attributable to abortion, this study is being heavily promoted by the anti-abortion lobby in support of its efforts to foist so-called ‘independent’ counselling on women seeking an abortion. In practice, ‘independent’ means trying to propel women into so-called crisis pregnancy centres run by anti-abortion organisations, including Life and CareConfidential, while excluding existing, highly regarded, providers like BPAS and Marie Stopes International from providing the same services on the back of spurious and wholly unevidenced allegations that the latter are actively operating under a conflict of interest and inducing women to have abortions to boost their own ‘profits’ – both BPAS and MSI are not profit-making charities.

Over the last few days, I’ve even seen claims that the publication of this study in BJPsych amounts to an endorsement of the study by the Royal College of Psychiatrist, who are the publishers of this journal – it isn’t. Publication in BJPsych means only that the study has passed the first stage of peer review, i.e. that its not so obviously or egregiously flawed as to be entirely unsuitable for publication, which in itself does not mean that the study is actually any good or that its claims stand up to close scrutiny.

So, the question we need to address here is that of whether this study lives up to the hype, or whether its just another load of anti-abortionist bullshit?

Does it really provide the ‘strongest evidence yet’ of a causal link between abortion and subsequent mental health problems and the ‘largest estimate of mental health risks associated with abortion available in world literature’, as its author claims, and is it really based on only the ‘strongest, most rigorous studies’ and therefore free of bias?

No. Not even close – in fact those claims are so far wide of the mark that they are not just wrong, they’re not even wrong.

The study, ‘Abortion and mental health: quantitative synthesis and analysis of research published 1995–2009‘ is a meta-analysis of data extracted from 22 studies published between 1995 and 2009, i.e. it attempts to pool the results of existing studies into one extremely large, statistically powerful, study.

In the right hands, meta-analysis is a powerful tool, one that’s capable of finding important and valuable evidence that smaller, individual studies, have failed to pick up. The classic example of this is a 1996 study by Patricia Crowley, ‘Prophylactic corticosteroids for preterm birth‘, which was published by the Cochrane Collaboration. Crowley’s study pooled the data from 18 seemingly unsuccessful trials of the use of steroids to promote lung maturation in severely premature neonates, most of which had failed to generate a statistically significant postive result, and found that, in reality, the use of steroids was significantly improving premature neonates chances of survival. Fifteen years on, there are many thousands of children out there who owe their lives to Crowley’s study and the improvements in the treatment of severely premature neonates that stemmed from its findings.

However, meta-analyses are not, in themselves, rigourous scientific studies. They are statistical examinations of existing studies and the methods used to conduct meta-analyses do not, of themselves, incorporate controls for common and rather obvious sources of bias. A well designed meta-analysis of a series of methodologically weak, badly designed studies will still generate poor quality statistics or, to use a common data processing aphorism, if you put garbage into your analysis then you’ll get garbage out no matter how rigorous your analytical methods might appear to be.

In order to assess whether or not abortion is associated with an elevated risk of subsequent mental health problems and , in particular, whether this association indicates a causal relationship between the two, our ideal study should:

a. be based on a fair comparison; and in the case of abortion the fairest comparison would be between the prevalence of mental health problems in women who aborted an unwanted pregnancy and the prevalence of the same problems in women who have carried an unwanted pregnancy to term.

Failing that, we could compare our abortion group to women who have an unplanned pregnancy to term on the assumption that at least some of these pregnancies may also have been unwanted as well, even though this is a less fair and less valid comparison that that of comparing the abortion group to a group of women who, we know, carried an unwanted pregnancy to term. If we don’t have any data on whether a pregnancy was planned on not, then we could compare out abortion group to women who simply carried a pregnancy to term, even though this means our results will be less reliable; and as a last resort we could compare our abortion group to a group of women who we know have never had abortion without taking in account whether or not these women have even been pregnant, even though this would give the least fair comparison and, therefore the least reliable set of results. However,  most diligent researchers, including those responsible for conducting RCPsych’s systematic review of the evidence relating to abortion and mental health would, quite rightly, consider a simple no abortion group to be an invalid comparison and would therefore exclude data derived from such a comparison from their analysis, which is precisely what RCPsych has done.

Unsurprisingly, none of the 22 studies included in Coleman’s meta-analysis are based on our optimum comparison (abortion vs unwanted pregnancy to term). 8 of the studies used unplanned pregnancy as their comparison group, generating 13 of the 36 results included in the meta-analysis. 7 studies used a ‘delivery’ group as their comparator, accounting for 10 of the 36 results included in the meta-analysis, and this leaves with a balance of 7 studies, and 13 results, which were based on comparisons to ‘no abortion’ groups and which really should have been excluded from Coleman’s analysis, not least because these studies will tend to give an unacceptably inflated and inaccurate estimate of the risks associated with abortion by including women who have never been pregnant in the comparison group.

b. exclude or control for women who have a history of mental health problems prior to having an abortion;

This should be screamingly obvious, but its still worth restating – the largest single predictor of mental health problem in women who have had abortion is a prior history mental health problems before the abortion took place. So, it follows that unless you either exclude women with a prior history of mental health problems from your analysis or control for this, statistically, then the inclusion of women with such a history in your data will result in an inflated estimate of the prevalence of post abortion mental health problems – and as you might expect, studies which inflate the prevalence of mental health problems in women who have had an abortion by failing to control for women’s prior history of mental health problems will inevitably provide an inflated and unreliable estimate of women’s risk of developing a mental health problems after having an abortion.

Based on the analysis undertaken by RCPsych in its systematic review, studies which neither exclude nor control for the inclusion of women with a prior history of mental health problems will, generally speaking, estimate that the prevalence of mental health problems in women who have had an abortion is around two to three times higher that the prevalence estimates generated by studies which do exclude and/or control for this major source of confounding.

Of the 22 studies included in Coleman’s analysis, eight do not include appropriate controls for women’s prior history of mental health problems at all and there is one further study which used such controls for only one  of the two sets of results included in Coleman’s analysis – these eight and a half studies contribute 12 of the 36 results included in Coleman’s analysis.

Sticking with the recording of mental health problems, our ideal study should also –

c. use a reliable, validated, method of assessing and recording whether women  included in the study do, or do not, have a mental health problem;

Three of the studies included in Coleman’s meta-analysis used inappropriate measures of mental health, based on RCPsych’s assessment of these studies, and these contributed four of the 36 results included in her analysis.

A prior history of mental health problems is not the only potential source of confounding.

We know, from numerous studies, that a wide range of socio-economic and other factors are associated with an increased risk of women developing mental problems, including age, ethnicity, socio-economic class/status, income, employment, education, marital/relationship status, family background, etc.

We also know, from other studies, that there are a wide-range of socio-economic and other factors that are associated with either an increase or a decrease in the prevalence of unwanted pregnancies and abortion including, age, ethnicity, socio-economic class/status, etc.

You get the picture… there is a degree of overlap between some of the known risk factors for mental health problems and the known risk factors for unwanted pregnancy and abortion and so. ideally, we should:

d. control for as wide a range of potential confounding factors as possible and, in particular, for confounding factors which are known to be associated with both an increased risk of mental health problems and an increased risk of unwanted pregnancy/abortion.

For our purposes here, we needn’t go to the trouble of assessing the extent to which each of the individual studies included in Coleman’s analysis does, ro does not, successfully control for known source of confounding. Its enough to know that these studies vary considerably in the extent to which they do, or don’t, include appropriate controls.

What should be obvious by now, however, is that there are a number of issues here with the quality of data included in Coleman’s analysis, issue which, when she claims to have found that women who have had an abortion exhibited an 81% increase in their risk of development mental health problems as a result of the abortion, she more or less ignores completely.

Fortunately, for our purposes, the RCPsych systematic review is nothing like as slapdash in its approach to the issue of data quality.

Each of the studies included in the three analyses contained in the RCPsych review (prevalence of mental health problems in women who have had an abortion, risk factors associated with an increase in mental health problems and whether or not an increased risk was evidence when compared to an appropriate control group) was systematically graded for study quality on a six point scale ranging from ‘Excellent’ to ‘Very Poor’ with studies which failed to make any of the these grades or which failed to provide RCPsych with any usable data being excluded from the analysis.

As luck would have it, 21 of the 22 studies included in Coleman’s paper were assessed and graded by RCPsych in its review – one study, a 2009 paper by Priscilla Coleman, does not appear anywhere in the RCPsych review, not even in the list of excluded studies, which suggests that it failed to meet any of RCPsych’s inclusion criteria.

Of the 21 studies that were graded, only 1 study was graded as ‘Very Good’ and this contributed only 1 of the 36 results included in Coleman’s paper. Nine studies were graded as being of ‘Fair’ quality, five were graded as being ‘Poor’, one was graded ‘Very Poor’ and five were ungraded and excluded from all three analyses because their either failed to provide any usable data or because they used an inappropriate method of assessing/measuring mental health throughout.

If we focus solely on the big question of whether women who have an abortion are at a greater risk of developing mental health problems than women included in an appropriate comparison group, 13 of the studies included in Coleman’s paper were excluded from this analysis by RCPsych due to their use of inappropriate controls, an invalid comparison group or because they provided no useable data, with the result that only 14 of the 36 datasets included in Coleman’s analysis were deemed suitable for inclusion in their own analysis by RCPsych.

61% of the data used by Coleman to generate her claim that women who had undergone an abortion exhibited an 81% increase in their risk of developing a mental health problem and that close on 10% of the overall incidence of mental health problems is directly attributable to abortion was not found to be of sufficient quality or reliability to merit its inclusion in RCPsych’s own analysis.

For this key analysis in the RCPsych review, the researchers separated the studies that did make the grade into two groups; studies that did not taken into account whether pregnancies were unplanned or unwanted, and studies which did and after comparing the results for both groups, RCPsych found that:

Studies that do not control for whether or not the pregnancy was planned or wanted suggest that there are increased risks of psychiatric treatment, suicide and substance misuse for women who undergo abortions compared with those who deliver a live birth. Findings for depression, anxiety disorders and PTSD did not indicate an increased risk.

1. Where studies control for whether or not the pregnancy was planned or wanted, there is no evidence of elevated risk of mental health problems and some evidence of lower rates of psychotic illness for women who have an abortion compared with those who deliver the pregnancy.

2. Data for all outcomes is still limited by a number of factors including a lack of comparable data across a range of diagnostic categories. Adequate control of confounding factors was shown to impact results.

3. Rates of psychiatric contact did not increase following an abortion, whereas there was a significant increase in psychiatric treatment following a pregnancy (without controlling for whether or not the pregnancy was planned).

4. Rates of psychiatric contact were found to be significantly higher in the abortion group 9 months prior to the abortion (MUNK-OLSEN2011). This suggests that women who have an abortion may differ from those who deliver a live birth and that any propensity towards mental health problems may be present before the abortion.

There is one final issue to address here – bias.

Perhaps the major weakness inherent in meta-analysis is the fact that because it does not control for source of bias it is a method which is wide open to abuse by ideologically motivated researchers who are pursuing a specific economic, social or political agenda.

Priscilla Coleman is one of small, tightly-knit group of abortion researchers associated with David C Reardon of the Elliot Institute – eight of the 22 studies included in Coleman’s meta-analysis have both Coleman and Reardon listed as either the lead author or as a co-author.

Reardon is, to put in mildly, is a complete and utter fucknut and, in academic terms, a fraud in addition to being – until 2004/5 – perhaps the most vocal member of this group in terms of his advocacy of the fictitious psychiatric condition, ‘post-abortion syndrome’.

I’ve dealt with both in previous posts but to recap:

Reardon was, until 2005, a relatively prolific author/co-author of papers which purported to show a causal link between abortion and an increased risk of subsequent mental health problems but has since slipped quietly into the background as a result of an exposé by Chris Mooney in which it was revealed that Reardon has obtained his PhD is bioethics from what was, at the time, an unaccredited correspondence college.

However, to get a real appreciation of the degree of outright batshit lunacy that’s lurking in the background of Dorries’ ongoing campaign to restrict abortion rights, you need to see what Reardon has to say on the subject of abortion when he’s not pretending to be an academic – oh, and the full ‘argument’ from which this short extract is taken can be read via this link to one of my earlier articles about Reardon:

The similarity between Nazi manipulations of the Jews and the abortionists’ manipulation of women faced with crisis pregnancies is striking. Just as the victim-Jews were forced to choose between losing everything, or just a little, so abortion counselors encourage the victim-woman to view “this pregnancy” as a threat to everything she has, her relationships, her family, her career, her entire future. She is assured that by sacrificing this one thing (a tiny unborn child), she can save the rest. During this process, the victim-woman is urged to view the abortion decision not as a moral choice, but as a rational choice of “saving what you can.”

But in fact, just as those who reluctantly cooperated with the Nazis discovered, the bargain is a false one. The demands on ghetto leaders to sacrifice more and more victims never stopped. And so it is with the post-aborted woman. After her child is destroyed, she faces self-condemnation, lower self-esteem, difficulty with relationships, substance abuse, career problems, a cycle of repeat abortions, and more. Often she experiences an intense desire for replacement pregnancies to atone for her lost child, and she becomes a single parent, the very problem she sought to avoid in the first place – but now she also has to deal with the emotional scars of an abortion.

With Reardon having taken a back seat, Coleman seems to have emerged as the leader – in terms of publications – of this coterie of anti-abortion researchers so its worth noting that twelve of the 22 studies included in Coleman’s paper have either Coleman, Reardon or both cited as one of the authors and these studies contribute 18 of the 36 results included in her meta-analysis.

It should also be noted that both Coleman and Reardon have, in the past, been heavily criticised for the methodological and statistical shortcomings of their work, in fact I looked in detail at one of Coleman’s more recent papers earlier this year and from my own analysis of the same dataset used by Coleman found that it would appear that Coleman had applied a large positive modifier to the data, on the pretext of a weighting the dataset to give a nationally representative sample. As a result of applying this modifier, Coleman’s figures suggested that the percentage of women included in the dataset with current (1-month) depression was 40% higher than the percentage of women with a lifetime history of depression, which is – of course – impossible.

So, to sum up, what Coleman’s study demonstrates is not that women who had undergone an abortion exhibited an 81% increase in their risk of developing a mental health problem.

Rather, its the case that Coleman’s study validates, yet again, the adage ‘Garbage In, Garbage Out’; so much so, in fact, that its maybe time we all stopped treating GIGO as a mere aphorism and, instead, started treating it as one of the core axioms of the scientific method.