As might well be expected, the published findings of a major Food Standards Agency commissioned systematic review of the alleged health benefits of organic produce have put a few noses out of joint, not least that of the Soil Association, after it concluded that:

…there are no important differences in the nutrition content, or any additional health benefits, of organic food when compared with conventionally produced food.

What the review is, in effect, affirming is more or less the view set out by Ben Goldacre in chapter six of Bad Science, ‘The Nonsense du Jour‘, which deals, in broad terms, with the relationship between nutrition (and the nutrition industry) and health. Ben’s argument, for those who’ve not ready the book, is that for the vast majority of people, i.e. anyone who doesn’t have an underlying medical condition that requires some sort of controlled diet, the key to continuing good health is no more than eating a healthy, well-balanced diet and that pretty much all the advice and information anyone could need on that particular subject is available, free of charge, from your GP.

Everything else, from vitamin supplements and so-called ‘superfoods’ to Gillian McKeith’s unedifying (and utterly unscientific)  obsession with poo is nonsense – not just any kind of nonsense, mind, but a very specific kind of carefully contrived and constructed nonsense that wilfully abuses and misuses science in order to  massively overcomplicate what is otherwise a pretty simple and straightforward subject in the pursuit of profit…

Basic, uncomplicated dietary advice is effective and promotes health. Overly complicated, confusing, tinkering nutritionism is poorly evidenced, because it’s a branch of the entertainment industry – it’s there to make money, to create a new market for a new profession, to soup up a recipe show, to titillate, to distract us from social inequality and the real lifestyle causes of ill health, and to pander to our collective modern obsession with food. It tarnishes and undermines the meaningful research work of genuine academics studying nutrition.

The FSA’s organic review, which was carried out by a research team from the Nutrition and Public Health Intervention Research Unit of the London School of Hygiene and Tropical Medicine (LSHTM), comes in two parts, a review of the evidence for differences in nutrient content in organic and non-organic food, which weighs in at a pretty hefty 209 pages in total, and supplementary review of the evidence for health benefits associated with eating organic produce over and above any benefits attributed to conventional foodstuffs.

Neither report found any significant evidence that would support the suggestion that organically produced food is intrinsically ‘healthier’ that food produced by more conventional farming methods. The report on nutrient content concluded that:

No evidence of a difference in content of nutrients and other substances between organically and conventionally produced crops and livestock products was detected for the majority of nutrients assessed in this review suggesting that organically and conventionally produced crops and livestock products are broadly comparable in their nutrient content.

And where differences in nutrient content were identified, the report went on to note that:

There is no good evidence that increased dietary intake of the nutrients identified in this review which are present in larger amounts in organically than in conventionally produced crops and livestock products, would be of benefit to individuals consuming a normal varied diet, and it is therefore unlikely that these differences in nutrient content are relevant to consumer health.

As for he second review, its most significant finding turns out to have been the general paucity of relevant research:

Despite an extensive search, the systematic review process identified only eleven articles relating to nutrient content that met our inclusion criteria. The studies included in the review were very heterogeneous in terms of study design, exposure and health outcome. Taken together, the eleven articles do not currently provide any evidence of health benefits from consuming organic compared to conventionally produced foods.

Although the report does go on to note that it does not address issues relating to contaminant content or environmental impact, both of which are certainly relevant to the wider public debate but not to the question of whether organic produce is, generally, healthier than food produced using conventional farming methods.

Bearing all that in mind, it should come as no great surprise to find that the Soil Association is ‘disappointed‘ with these findings, not least when one considers that the claim that organic produce has health benefits that either exceed or are absent in conventionally farmed produce plays a major role in justifying the premium prices that consumers typically have to pay for organic foodstuffs.

What some may find rather surprising is that, having expressed its disappointment, the Soil Association then goes on to make a fairly blatant attempt to devalue, if not discredit, the findings of the review using methods more commonly associated with practitioners of so-called ‘complementary and alternative medicine’ (CAM).

So it is that we find the Soil Association complaining that:

The review rejected almost all of the existing studies of comparisons between organic and non-organic nutritional differences. This was because these studies did not meet particular criteria fixed by the London School of Hygiene and Tropical Medicine, which carried out the review.

As one might expect, the Soil Association’s statement makes no reference at all to the specifics of the exclusion criteria used by the LSHTM, even though these are set out clearly in the research as follows:

Articles were excluded if they:

– were not peer-reviewed

– did not have an English abstract

– did not address composition of nutrients and other substances

– did not present a direct comparison between organic and conventional production
systems

– were primarily concerned with impact of different fertiliser regimes

– were primarily concerned with non-nutrient contaminant content (cadmium, lead and
mercury)

– were authentication studies describing techniques to identify food production methods.

If those criteria did result in the exclusion of  ‘almost all of the existing studies of comparisons between organic and non-organic nutritional differences’ then one would have to conclude that almost all of these existing studies are either of rather poor quality or largely, if not entirely irrelevant to matter at hand and, consequently, provide little or no reliable evidence for the claimed health benefits of organic produce.

Having kicked things off with an intellectually dishonest knock that the reports methodology, the Soil Association’s next gambit is to go for a liberal dose of statistics abuse…

“Although the researchers say that the differences between organic and non-organic food are not ‘important’, due to the relatively few studies, they report in their analysis that there are higher levels of beneficial nutrients in organic compared to non-organic foods. For example, the mean positive difference between the following nutrients, when comparing organic to non-organic food, was found in the FSA study to be:

– Protein 12.7%
– Beta-carotene 53.6%
– Flavonoids 38.4%
– Copper 8.3%
– Magnesium 7.1%
– Phosphorous 6%
– Potassium 2.5%
– Sodium 8.7%
– Sulphur 10.5%
– Zinc 11.3%
– Phenolic compounds 13.2%

The researchers also found higher levels of beneficial polyunsaturated fatty acids in organic meat and dairy products (between 2.1% – 27.8% higher) compared to non-organic meat and dairy.

And this all means what, exactly?

This list exactly what it appears to be – a meaningless stream of decontextualised percentages which one would presume were included in the Soil Association’s press statement by a press office idiot who firmly believes that journalists, and the general public, are stupid, lazy and/or gullible enough to simply assume that more is better.

Sadly that’s a low opinion of both journalists and the general public that’s probably justified if the parlous state of health reporting in the British press is anything to go by but, strictly for the record, it should be noted that in the case of the minerals listed (Zinc, Magnesium, Phosphorus, etc.) there are no direct health benefits associated with any of them but for cases in which an individual is deficient in a particular mineral. Such cases are, of course, few and far between in affluent, well-fed countries like the UK, which is why the report concludes that:

A health benefit of increased dietary intake of these minerals is unlikely in adequately nourished populations.

The same is true of Beta-carotene, which is metabolised in the body into vitamin A.

While vitamin A plays an important role in a number of biological and biochemical processes, including vision, gene transcription and the immune system, an increased intake of either vitamin A or Beta-carotene is only likely to beneficial in individuals who are deficient in vitamin A.

However, its also very much the case that more is not necessarily better when it comes to Beta-carotene given that excessive consumption can result in carotenodermia, an harmless condition in which the skin develops a noticeable orange tint arising from the deposition of carotenoid in the outermost layers of the epidermis.

That leaves us with the phytochemicals (i.e flavonoids and phenolic compounds) which have been linked to a range of health benefits due to their anti-oxidant properties. Unfortunately, the evidence on anti-oxidants is derived primarily from cohort studies and not from randomised controlled trials and, when reviewed in any detail, turns out to be far less compelling than the media publicity surrounding them.

To pick out just a couple  of the many interesting and illustrative  studies in this area, a paper published in  2007 by researchers from the Linus Pauling Institute found that, within the human body, flavonoids have little or no antioxidant value – only around 5% of amount of flavonoids present in foodstuffs were found to be absorbed by the body while the rest (95%) was either rapidly metabolised or simply excreted without generating any beneficial effect. The study concluded that the human body treats flavonoids as a foreign compound and attempts to get rid of them.

Turning to the ‘Big C’ – cancer – where all manner of claims are routinely reported which suggest that flavonoids present in everything from tea and red wine to citrus fruits, sprouts and onions, may have preventative benefits, a study by researchers at UCLA did find evidence to support such a contention in relation to lung cancer in smokers but added this note of caution:

“Since this study is the first of its type, I would usually be hesitant to make any recommendations to people about their diet,” Zhang said. “We really need to have several larger studies with similar results to confirm our finding. However, it’s not a bad idea for everyone to eat more fruits and vegetables and drink more tea.”

So, the jury remains out on the subject of the preventative benefits of flavonoids and even if the UCLA findings are confirmed by large studies there is no guarantee that this will then demonstrate that organic foodstuffs will offers any beneficial effects over and above those associated with conventional produce.

Before we leave the statistics abuse behind, it also needs to be noted that the Soil Association have been indulging is a bit of cherry picking when it comes to the figures they’ve chosen to quote in their statement.

As is commonly the case in systematic reviews, the mere fact that a study meets the core inclusion criteria offers no guarantee that the study has also been conducted in a satisfactory manner – such reviews frequently include data from studies that are graded as being less than satisfactory but not so seriously flawed as to render them wholly unreliable and where this practice is followed the study will typically provide two sets of results, one for all included studies and one for only those studies that were found to be conducted in a satisfactory manner.

The upshot of this, when you look at the detailed results, is that, in certain cases, there is marked disparity between the results for all studies and those produced by limiting the analysis to only those studies deemed ‘satisfactory’ – and to make thing even more interesting, where these disparities arise in this particular review, they do not necessarily fall on one side of the equation only.

So, for example, the review of studies of the phosphorus content found higher levels in organic produce when restricted to just the satisfactory studies but no difference when all studies were included in the analysis, while for Beta-carotene, the opposite occurred, such that the satisfactory studies alone showed no difference while taking all studies showed a significant difference in the quantities present in organic produce.

What this creates is a situation that’s ripe for cherry picking and that’s precise what the Soil Association have done – quoted the highest figure regardless of both whether it derived from an analysis of all studies or just those deemed satisfactory and, to compound things even further, also regardless of the whether the quoted figure was statistically significant.

So, for Beta-carotene, the figure of 53.6% derives from the analysis of all included studies where limiting the analysis to satisfactory studies only gives a figure of only 20.7% – and neither figure is statistically significant.

However, for phosphorous, the higher figure of 8.1% quoted in the press statement derives from the analysis of satisfactory studies, including all studies reduces this figure to 6.0% and, in this case, both figures are statistically significant.

In short, this entire section of the Soil Association’s press release is, at best, meaningless and, at worst, a contrived attempt to manufacture unjustified and unwarranted doubt as the validity of the LSHTM’s findings – and it doesn’t stop there as the press release goes on to make the following spurious point…

The Soil Association is also disappointed that the FSA failed to include the results of a major European Union-funded study involving 31 research and university institutes and the publication, so far, of more than 100 scientific papers, at a cost of 18million Euros, which ended in April this year.

The European Union research programme concluded that:

• ‘Levels of a range of nutritionally desirable compounds (e.g. antioxidants, vitamins, glycosinolates) were shown to be higher in organic crops’
• ‘Levels of nutritionally undesirable compounds (e.g. mycotoxins, glycoalkaloids, Cadmium and Nickel) were shown to be lower in organic crops’.

In addition, levels of fatty acids, such as CLA and omega 3 were between 10 – 60% higher in organic milk and dairy products, and levels of Vitamin C were up to 90% higher in leafy vegetables and fruits.

The research programme in question is the QualityLowInputFood programme the purpose of which is to improve the quality and safety of organic produce and reduce production costs, reducing, in turn, the costs to the consumer.

This programme may well have cost the EU a small fortune and generated a veritable plethora of research papers but, as should be obvious from the programme’s stated objectives, none of the research was actually directed towards the question of whether there are any actual health benefits to eating organic produce. As such, it seems highly unlikely that the inclusion of data from this programme would make the slightest bit of difference to the LSHTM’s findings.

And, just the drive the point home, the fact that certain minerals and compounds are identified by this programme as ‘nutritionally desirable’ in no sense indicates that ingesting them in greater quantities will confer any health benefits, it merely indicates that a deficiency in one or more of them will have a negative impact on an individual’s health, and as the LSHTM report clearly indicates, such deficiencies are, by and large, rarely seen in our already affluent and well-fed population.

To sum up, there is absolutely nothing in the Soil Association’s statement which provides any kind of credible challenge to the LSHTM’s conclusion that there is no credible evidence to show that eating organic produce produces health benefits over and above those produces by eating a healthy, well-balanced diet consisting entirely of conventionally farmed produce.

What there is, however, is an intellectually dishonest attempt to manufacture doubt as to the validity of these findings where no such doubt exists save for those limitations (i.e. the absence of evidence on long-term exposure to pesticides) that the reports freely and openly acknowledge.

There are plenty of good reasons for buying organic produce, from concern for the environment and animal welfare, to supporting small farms and farmers who make up the bulk of the organic sector, to simply acting on the belief that organic produce taste better and is of a higher quality that conventionally farmed produce but the idea that that it will make you healthier than other people around you is not one of them.

That’s what the evidence says and no amount of contrived obfuscation and misdirection is going to prove otherwise.