One of things I’ve been watching out for over the last few weeks has been the response of the Christian Medical Fellowship (CMF) to the NCCMH/RCPsych consultation on its draft systematic review of the evidence relating to the mental health impact of induced abortion.

The CMF has now published its detailed response and its summary guidance for its members and supporters, neither of which are particularly impressive. In fact, the most appropriate summation of the CMF’s position on the RCPsych review I can think of would go something along the lines of…

We don’t like where you’ve put the goalposts, so we want them moved to a position more of our own liking.

That’s pretty much the situation here; the evidence doesn’t support the CMF’s religious/moral position on abortion for the simple reason that reality itself is insufficiently biased in favour of the anti-abortion movement’s efforts to marshal scientific evidence in favour of prohibiting legal induced abortions and, so, for that reason they have no alternative but to try and manufacture a reality all of their own, complete with its own dubiously provenanced ‘evidence’, much of which relies either on the use of poor quality research methodologies, the cherry-picking and torturing of available datasets and the misinterpretation and misrepresentation of what is otherwise credible research evidence.

From the CMF’s summary guidance, its ‘findings’ on the review are given as follows:

Overall, we have concerns that:

– There is insufficient transparency in the selection, exclusion and rating of research papers;

– A key paper by Fergusson was re-analysed by the reviewers, producing a finding that contradicted his published paper;

– Several summary evidence statements are more definitive and stronger than the actual evidence suggests;

– Claims that there is ‘no evidence’ of an elevated risk of mental health post-abortion compared to post pregnancy are not justified by the data presented.

None of these findings actually stand up to close scrutiny and one in particular, relating to re-analysis for a ‘key paper by Fergusson’ appears to be a concern largely because the CMF have overlooked the obvious and failed to understand both how and why the findings of this paper were re-evaluated.

Taking these ‘concerns’ in order, the claim that there is ‘insufficient transparency in the selection, exclusion and rating of research papers’ is a complete and utter nonsense. The methodology section of the review paper is commendably clear and contains all the elements one would expect from a  rigorously conducted systematic review; i.e. a clear statement of the search terms and libraries used to identify papers for inclusion in the study, a clear exposition of the inclusion and exclusion criteria (with a rationale) and an explanation of the grading method used to assess the papers cited in the study against the inclusion/exclusion criteria use which clearly identifies the standard grading system used by the review team.

Any problems identified by the CMF in regards to selection and grading process stem entirely from the reviews choice of inclusion/exclusion criteria, which serve to exclude almost all of the poor quality ‘evidence’ that anti-abortion groups, like the CMF, have been hawking around the internet in an effort to promote the idea that abortion can be linked to a significant increase in the incidence of mental health problems in women who have had an abortion.

To summarise the inclusion criteria chosen by the NCCMH/RCPsych review team, studies included in the review should be:

– Quantitative studies which provide statistical evidence of the incidence/prevalence of DSM-IV psychatric conditions in women who have had an abortion.

– Studies which provide evidence of mental health problems which persist for more than 90 days after an abortion, or occur after that 90 day limit.

– Studies which provide  comparative data for the incidence/prevalence of post-abortion mental health problems in women against the incidence/prevalence of the same problems in women who have carried one or more pregnancies to term.

As the review correctly points out, the ideal study for inclusion in this review would be one based on data from the UK only which compared the mental health outcomes of women who had an unwanted pregnancy and who the chose to have an abortion against mental health outcomes of women who had an unwanted pregnancy and who we then compelled to carry the pregnancy to term against their wishes, but as abortion is legal in the UK and we don’t force women to continue with an unwanted pregnancy provided that they make their decision to have an abortion before 24 weeks gestation, there were no ideal studies that could be included in the review.

This, coupled with the general shortage of UK only studies, forced RCPsych to widen its inclusion criteria to admit studies from overseas – if published in English – provided that they offered comparative data for incident/prevalence of mental health problems for abortion again for pregnancies carried to term and/or terminating by way of a spontaneous abortion (i.e. miscarriage).

This is a perfectly reasonable and appropriate set of inclusion criteria for a study of this kind and yet its one about which the CMF have lodged several complaints, e.g.

By limiting the criteria used for inclusion of research in this study to only those that measure outcomes occurring more than 90 days post-abortion excludes a large body of evidence and literature on mental health disorders (see for example our comment on p45 below). For example, for just one review question this accounted for 27 studies being excluded (p27). While there is some evidence that mental health may improve in the short term after abortion, there are also many women who suffer mental health disorders in the two months post-abortion, and this group is excluded. Outcomes will vary with time post-abortion. The limitations of these inclusion criteria and the evidence that the consultation cannot therefore consider should be made clear.

The main reason that studies looking at short-term post-abortion mental health outcomes are excluded is because such studies are easily confounded by the entirely natural grieving process that many women undergo after having an abortion. In the weeks immediately following an abortion it is perfectly natural and normal that some/many women will feel a bit ‘down’ and experience feelings of sadness, regret and even guilt at having had an abortion, and as the language of mental health has permeated wider society, it would not be at all uncommon for women in that situation to describe themselves as being ‘depressed’. Most women will, however, quickly bounce back from those feelings and, within a reasonably short space of time, start to feel much more like their old selves, i.e. the ‘depression’ and other feelings they experienced were transient, short-lived and, therefore, fall short of amounting to a full blown psychiatric problem and, consequently, one cannot make any reliable assessments of the general risk of post-abortion psychiatric sequelae from dataset which include these women.

Put simply, if a woman feels depressed for a few weeks after an abortion, we cannot be sure whether than indicates that she has development depression or whether she’s merely going through an entirely natural grieving and recovery process. If, on the other hand, her depression lasts for more 90 days after the abortion than we can be pretty sure that we do have a significant psychiatric issue on our hands, which is why it only data the relating to medium-long term psychiatric sequelae that will provide a reliable indication of any causal links between abortion and any subsequent mental health problems.

The CMF also complain – somewhat absurdly – about the exclusion of qualitative studies from the review, despite the fact that these almost always provide no usable evidence for a review of this kind. What we can gain from good qualitative studies is valuable information about the thought processes and feeling of women during the period in which they make their decision on the future of their unwanted pregnancy and, of course, how they feel after they’ve had an abortion, all of which can help considerably to inform the work of support workers and counsellors working with women who do struggle to come to terms with their decision. But this does not tell us anything about how common, or uncommon, such issues might be and as no one is actually denying that some women do struggle to come to terms with having had an abortion even to the extent of developing diagnosable psychiatric problems after the procedure then the inclusion of such studies would add nothing whatsoever to the review.

For all that, perhaps the most ridiculous and desperate argument deployed by the CMF is this one:

One problem with measurement is that many people with mental illness do not seek treatment. Women who have negative reactions to abortion are less likely to return to the clinic. The eligibility criteria therefore will be likely to have excluded many women who do not return to the health professionals who were involved in the abortion process. Poor follow up post-abortion compounds this problem. Clearly this would underestimate prevalence of mental health disorders. Compounding this, women delivering will be more likely to have regular contact with health professionals than those having a termination and so a higher reporting of their mental health problems will be likely, again introducing a bias in the groups.

In short, we can’t count what we don’t know about… Duh-huh!

If the only data recorded related to women who reported mental health problems back to the clinic where they had the abortion then this would indeed be a major problem, but some of the evidence included in the review comes from general population and cohort studies which draw their data from medical records, self-report questionnaires and other sources which do pick up on women who experience mental health problems that weren’t reported directly to the clinic which carried out the abortion. It also has to be noted that just because women who do deliver tend to have more, and more regular contact, than women who have had a termination, it doesn;t follow that there aren;t significant levels of underreporting in this group. There is still, unfortunately, a considerable stigma attached to mental health problems, for all that issues such as depression and anxiety are extremely common, and for some women who do deliver, anxieties about the stigma attached to mental health problems can readily be compounded by fears that seeking help for a possible mental health problem may cast doubts on their ability to safely/adequately care for and raise their own child and/or amount to an admission of failure. The whole notion of motherhood is still very much bound up in the popular myth that it’s the ‘most natural thing in the world’ for women – well, it isn’t; its hard work, sometime extremely frightening and a hell of lot to come to terms with in a very short space of time and anyone who suggests otherwise is either lying, a man, hasn’t got kids or just flat-out deserves to be slapped repeatedly until they get the message that motherhood – and fatherhood, for that matter – is anything but a breeze.

The CMF’s underreporting argument smacks of nothing more than desperation in the face of a review that’s going entirely against their position, i.e. a spurious argument advanced only for the sake of providing the organisation with a means of trying discredit the review when their arguments are dismissed for their lack of credibility.

In so far as the transparency of the process goes, the CMF has this to say:

Although this is not an entirely unusual rate of exclusion of studies, there is a concerning lack of transparency in the inclusion and exclusion process. The authors exclude studies if they do not contain ‘useable data’ or did not use a ‘validated measure of mental health’ but they fail to explain what these actually constitute. There is insufficient transparency regarding the reasons for excluding hundreds of peer-reviewed studies, many of which may have failed in just one or two criteria but could still provide useful findings.

As comments go, file this one under ‘does not understand how systematic reviews are conducted’.

This isn’t difficult at all. The review paper clearly set out its inclusion/exclusion criteria, which includes its data requirements, so…

No useable data = The paper didn’t contain any data that met the inclusion criteria or data requirements for the review and was, therefore, excluded.

Did not use a ‘validated measure of mental health’ – There are several fully validated diagnostic scales which are widely used to measure the prevalence, longevity and severity of psychiatric conditions. Papers that did not use any of these measures cannot be reliably evaluated and were, therefore, excluded.

What is being excluded from the review here are poor quality studies which fail to provide useful or reliable evidence, and the mere fact that the CMF is complaining about their exclusion should give you more than a few clues as to the kind of quality of evidence on which the anti-abortion lobby has been basing its arguments.

As for the grading process, the CMF puts foreard this argument:

Similarly, it is not clear on what basis the gradings were made for quality. Which criteria were more important than others? How did the reviewers reach conclusions about the quality of studies? Which criteria were met or not met? For example, we are concerned that Fergusson 2009 is rated as ‘fair’ while Steinberg study 2 is rated as ‘very good’, which is different to previous reviews. Fergusson 2009 (and 2008) is a longitudinal study, a primary analysis and controls well for confounders. In contrast, Steinberg 2008 study 2 is a secondary analysis, it is cross-sectional and it uses data from a pre-existing database. It should not be graded as very good. More justification and transparency on the ratings is necessary here.

Again, the real problem here is one of a lack of understanding on CMF’s part – that or they’re wilfully throwing in an argument they know to be spurious.

When grading papers for a systematic review, at least part of the grading assessment will depend heavily on the inclusion criteria and data requirements specified in the review. What this inevitably means is that, in some case, a paper graded as ‘Good’ in one review, which is looking for answers to one set of questions, will go on to be graded only as ‘Fair’ in a different review, which is asking a very different set of questions. This is often no reflection of the overall quality of a particular paper or of its methodological rigour, rather it stems largely from the extent to which the data provided by the paper is relevant to the question that a systematic review is seeking to assess.

Somewhat ironically, given that last complaint, one of the main reasons why the Fergusson paper was only graded as ‘Fair’ is directly relevant to their next recommendation, which questions the use of that paper in the review. So, moving on to item two which relates directly te the Fergusson paper, the CMF’s summary complaint goes as follows:

A key paper by Fergusson was re-analysed by the reviewers, producing a finding that contradicted his published paper;

And the detailed complaint is outlined as:

The authors state that they received additional figures from Fergusson, leading them to reanalyse Fergusson 2008 data and reach a conclusion that is different to his published paper. However the authors do not provide these new figures, nor describe how the new analysis was undertaken, and nor do they state what the original findings clearly showed. Since this new ‘evidence’ actually contradicts the original evidence in the Fergusson paper, more rationale must be provided to explain this conclusion, along with the new and original ‘evidence’. This is an important point to rectify as Fergusson’s 2008 findings have been widely cited to indicate a higher relative risk for those having an abortion.

[The original paper states: “…women exposed to induced abortion had risks of mental health problems that were about 30% higher than women not exposed to abortion.” Fergusson, D. et al, 2008]

And the answer to the CMF’s question is readily apparent if only you read both the Fergusson paper and the Draft review paper side by side. In fact, somewhat bizarrely, the CMF provides half the answer to its own question in its bracketed comment.

So, here’s the exact text of one of Fergusson’s findings from his original 2008 paper, based on data from his 5-year lagged model…

The results of the 5-year lagged model were in most respects very similar to the findings from the concurrent model. Exposure to induced abortion was associated with consistently increased risks of mental health problems, with women who had had abortions having overall rates of mental health problems that were 1.48 (95% CI 1.18–1.85) times the rates for those who had not become pregnant (P<0.001).

And here’s the RCPsych review reporting from Fergusson’s data on the same issue, based on the same 5-year lagged model.

Similarly, FERGUSSON2008 indicated that women who had an abortion were not at an increased risk of a higher number of mental health problems compared with those who deliver an unwanted pregnancy (IRR 0.79; 95% CI, 23 0.51 – 1.23, p> 0.05)7. Note that this comparison was not made by FERGUSSON2008, however, figures were provided by the authors which 25 informed this analysis.

I’ve highlighted the key difference between the two statements. In the original paper, the comparator used was women who had never become pregnant. In the RCPsych review, the comparator is women who delivered an unwanted pregnancy – the figures aren’t the same because the comparison group used in the RCPsych review is not the same as the one used in Fergusson’s original paper. Indeed, thanks to the raw data supplied by Fergusson et al. what RCPsych have been able to do here is make an ideal, or near ideal comparison between women who have had an abortion and women who carried an unwanted pregnancy to term, and the finding show there is no increased risk.

What’s missing here, because its outside the scope of the review, is a direct comparison between the mental health risks for women who carried an unwanted baby to term and women who carried a baby to term that they did want. Hopefully, that comparison will be made in future paper either by RCPsych or by Fergusson and his co-authors but, even with direct statistical confirmation, what we can infer from this result is that because the risk of subsequent mental health problems is higher im women who have had an abortion than its is women who carry a wanted baby term then its highly likely that women carrying an unwanted baby to term will also show a higher risk of subsequent mental health problems, an result which supports the continued provision of legal abortions to women on grounds of a risk to their mental health and wellbeing while blowing out the water the CMF’s claim that abortions carried out in the UK on this basis, which is the overwhelming majority of abortions, are technically illegal.

It is hardly surprising, therefore, that the CMF are complaining about this finding and about the manner in which the data from Fergusson et al. has been used given that it has generated just about the last possible result that the CMF would wish to see, let alone acknowledge.

If you’re still with me, then we’re on the home stretch as the next item is easily disposed of…

– Several summary evidence statements are more definitive and stronger than the actual evidence suggests;

No, and to illustrate the point, here’s what the CMF have to say about one such statement, which relates to an evidence table (table 17) on page 80 of the review:

This table compares like with like groups. It reveals weak evidence of a higher risk of anxiety disorder and self-harm outcomes for women post-abortion. It also shows weak evidence of higher risk of psychotic illness for women post-birth than post-abortion (but see our comment on p81, line 37-40).

Whilst only weak evidence, the authors should not conclude, page 81, line 38, that ‘there is no evidence of elevated risk of mental health problems’ post-abortion if they feel able to conclude that there is ‘some evidence of lower rates of psychotic illness’ post-abortion. As it stands, this evidence statement thus favours (cites) only the one outcome that demonstrates a positive effect (post-birth) whilst ignoring the two outcomes that show a negative effect (post-abortion).

The evidence statement should be amended to either state: ‘there is some evidence of elevated risk of mental health problems and some evidence of lower rates of psychotic illness for women who have an abortion compared with those who deliver a pregnancy’ OR, there is no evidence for an elevated risk for either.

Table 17 lists and grades the evidence for an increased risk of range of mental health outcomes for women who had an abortion when compared to women who carried an unplanned/unwanted pregnancy to term, including anxiety disorder. All the evidence is graded ‘Very Low’ with relative risk increases being given as either and odds ratio or risk ratio and range at a 95% confidence interval. If you look at the table yourself then the key piece of information you need to make sense of it is that if the range includes 1 (e.g. 0.5-1.3) then the odds/risk ratio is not statistically significant and you have no evidence.

In the table, all but two of the outcomes have range values of which only one, for psychotic illness, is statistically significant.

For anxiety disorder, no range values are given and the most likely reason for this is simply that this s typographical error, an accidental omission which will no doubt be corrected in the final version of the review – and that is the sole basis of the CMF’s claim that there is weak evidence for an increased risk of anxiety disorder, even though, looking at the grading and odds/risk ratios for other outcomes, it seems highly likely, if not certain, that the inclusion of the missing range values will show that this result is, like the others, non-significant.

As a general rule of thumb, if you have to resort to picking on typos, then you haven’t any kind of argument, least of all a worthwhile one.

Last, and most definitely least, we come to the claim that:

– Claims that there is ‘no evidence’ of an elevated risk of mental health post-abortion compared to post pregnancy are not justified by the data presented.

And the CMF’s ‘detailed’ argument for this claim begins with this statement:

Note our comments on Table 14, p73 above. Despite the limitations of the evidence, which are detailed in the review on p73-74, Table 14 clearly shows that the risks of many mental disorders are increased in women who have abortions, compared to those giving birth. Therefore this does not justify the claim in evidence statement 1 (p81 line 38) that there is ‘…no evidence of elevated risk of mental health problems…’. We are highly concerned about this statement which does not reflect the evidence.

What the full statement on p81 actually says is…

Where studies control for whether or not the pregnancy was planned or wanted, there is no evidence of elevated risk of mental health problems and some evidence of lower rates of psychotic illness for women who have an abortion compared with those who deliver the pregnancy.

This relates only to the studies which provide comparative evidence for whether a pregnancy was planned or wanted and relates solely to the evidence listed in Table 17 (p80) – it has nothing whatsoever to do with the data given in Table 14.

So, what the CMF have done here is…

a) Cherry-pick half a sentence from the review,

b) Present that statement out of context, and

c) Link the statement to data given in entirely the wrong evidence table (table 14) when it actually relates to and accurately reflects the evidence given in a different table. (Table 17).

This is either rank incompetence or outright dishonesty, but either way their argument is complete and utter nonsense.

This argument is followed up by one or two other equally invalid and misconceived attempts to cast doubt on the validity of the reviews evidence statements, including one based on their own interpretation of the Fergusson paper, which we already know is completely wrong, all of which leads up to this goal=post shifting ‘suggestion’.

As we note above, (see p45 and p85) the rates of mental health problems are higher post-abortion than in the general population, even when controlling for mental health problems.

Therefore we suggest that on p89 an extra evidence statement is added to clarify that when prior mental health problems are controlled for, rates of post-abortion mental health problems occur at higher rates than the general population.

As I noted in discussion the inclusion criteria for the review, the optimum comparator for assessing the mental health outcomes of women who have had an abortion are women who carried an unwanted pregnancy to term for reasons which the review paper clearly notes:

Both the APA and Charles reviews looked at studies that used a “never pregnant‟ comparison group (Pedersen, 2008; Rees & Sabia, 2007). Although it was useful from a research perspective to compare abortion with outcomes like miscarriage or not being pregnant, these would not be viable options for a woman facing the decision of whether to have an abortion or not. This issue is summarised effectively by Cameron (2010) who claimed that “once a woman is in the situation of having an unwanted pregnancy, there is no magical state 16 of “un-pregnancy”.

As such, the CMF’s suggested statement has no value whatsoever in the context of the RCPsych review nor as information for pregnant women who may be considering an abortion. All they seeking here is the unmerited legitimisation of one of their own prohibitionist arguments by way of a bit of absurd goalpost shifting, and I’ve no doubt that RCPsych will have none of it.

Based on the evidence, the CMF’s submission to the RCPsych review can best be summed up in just seven words;

… a pile of tendentious and ill-informed crap.