Although the Internet is, today, positively swimming with consumer review websites, the online consumer review model has its flaws. One can never be entirely certain about the personal motives or degree of experience that product reviewers bring to the table nor how robust a review sites’ systems when it comes to dealing with the risk of unscrupulous vendors resorting to the use of sockpuppets to artificially inflate their review ratings. More often than not, the best you can hope for is that a particular product can attract enough reviews for you to trust that the wisdom of the crowd effect will give you a reasonably honest picture of a product’s quality and performance but this is by no means a certainty.

Independent organisations and magazines that carry out and publish systematic reviews of competing products continue to serve an important function in the marketplace, even in the face of the rapid growth of the crowdsourced end-user review industry, and if you live in the the UK, the acknowledged ‘daddy’ of the consumer review sector is Which?, the magazine of the Consumers’ Association, a charitable organisation that has built up an unrivalled reputation as a champion of consumer rights over more than half a century.

In its latest issue, Which? has turned its attentions towards the alt-med sector and carried out an investigation into the services provided fifteen ‘high street nutritional therapists’ producing what are striking, if not exactly unsurprising results:

A Which? investigation has found that some advice given by high street nutritional therapists could seriously harm patients’ health.

We sent undercover researchers – posing as patients with a range of health problems – to 15 consultations with nutritional therapists.

The Which? panel of medical experts rated six of the 15 consultations as ‘dangerous fails’ – where the advice given could potentially endanger the health of the researcher.

Eight consultations were rated as ‘fails’ and one as a ‘borderline pass’.

The detail of Which? investigation and report has already been ably covered by the ever reliable Quackometer and by David Colquhoun, who’s also tracking media coverage of the Which? report – both articles should be filed under ‘must read’.

As David notes in his coverage of responses to the Which? investigation, BANT – British Association for Applied Nutrition and Nutritional Therapy – has responded by sticking out a press release that fails to address any of the serious issues that the report raises, despite the fact a number of the nutritionists whose dodgy ‘advice’ is featured in the report are members of this organisation. Although BANT claims to have many of the trappings of an industry regulator; i.e. a register of members, an ethical code, etc. its actually nothing more than trade association and lobbying organisation which has no real regulatory powers or authority and, if its press release is anything to go by, no real inclination to to use what little authority is does have, i.e. the ability to suspend or cancel memberships, even when some of its members have been exposed as outright quacks.

That, sadly, should come as no great surprise because, for its pretensions and talk of ‘the science of nutrition’ and ‘nutrition science’ most of what BANT and its members have to peddle to the general public is overblown, confusing and contradictory nonsense. That’s not to say that there isn’t genuinely a science of nutrition, because there is, rather its a function of the nutritionist’s parasitical and exploitative relationship with real science.

Before looking at BANT’s response in some detail, its worth noting the main findings of the Which? investigation. Of the fourteen nutritionists who were mystery shopped – one was visited twice – only one consultation resulting in the mystery shopper being given advice that was rated as borderline acceptable by a panel of medical experts which included a Professor of Pharmacology, David Colquhoun, a GP, Margaret McCartney, who is also a blogger and a regular contributor to the British Medical Journal and Catherine Collins, a principle dietician at St George’s Hospital NHS Trust. Each of these consultations was covertly recorded, transcribed and then graded by panel members based on the quality of the nutritional advice, the nutritionist’s medical knowledge, whether a condition was diagnosed and whether or not the client was advised to see their GP.

Of the remaining fourteen consultations, eight were rated ‘fail’ and six were considered to be dangerous fails including one in which the nutritionist advised a client, whom they were told had cancer, against going ahead with surgery and radiation therapy and telling them, instead, that they could rid their body of cancer by diet alone:

The nutritional therapist said: ‘Cancer lives off sugar; if you feed it sugar it’s going to thrive. If we starve the cancer of sugar then you have a better opportunity of the cancer going away’.

Thirteen of the fourteen nutritionist’s who were mystery shopped were members of BANT, making that organisations response to Which? particularly pertinent, not least as it begin with a statement which it alleged is factually incorrect:

BANT has the following comments to make on the WHICH report –

BANT did not decline to comment on this article but was unable to comment for the print edition because WHICH did not provide all the promised transcripts/questionnaires in a timely fashion.

Now, according to David Colquhoun, this is simply untrue:

BANT moans that that Which?  ”did not provide all the promised transcripts/questionnaires in a timely fashion”.   In fact they got them before Christmas and were given until January 15th to respond.

BANT had at least three weeks to come up with something better than ‘and we would have got away with it if it weren’t for those meddling kids’ and came up with nothing, and even after being stung into a response by the publication of the report their response remains distinctly disappointing:

It is disappointing that WHICH appears to have little interest in conducting a genuine review of the effectiveness of nutritional therapy preferring instead to use fictitious consultations and a biased panel of ‘experts’.

So, far from addressing the evidence of quackery and bad practice, BANT has chosen instead to a) try and move the goalposts away from the activities of its members, and b) mount an ad-hominem attack on Which? and the panel that evaluated their members’ ‘advice.

It’s an interesting line of defence but not one that I can see catching on in other environments:

Barrister: “In summing up for the defence I would urge the jury not to give too much weight to the evidence which shows that my client was discovered in possession of the murder weapon, or that they were covered from head to foot in the victim’s blood or even the CCTV footage clearly shows them dismembering the victim’s body and feeding it, piece by piece, into a woodchipper.

“Rather I would ask them to think carefully on the evidence presented to the court which shows that the latest statistical evidence from the Home Office shows a clear year-on-year decline in the homicide rate, not to mention the investigating officer’s striking admission that he has never, himself, committed murder and therefore cannot reasonably be considered to be an expert in such matters, and render the only possible verdict in this case..”

…that my client is not guilty.”

As for BANT’s apparent disquiet over ‘fictitious consultations’, again this an attempt to move the goalposts and deflect attention away from the issue that Which? investigation, which isn’t whether there’s any good evidence for the efficacy of nutritional therapy, generally, but whether or not supposedly trained and qualified therapists are capable recommending an appropriate and effective course of treatment for a particular condition or even accurately identify particular conditions, or likely conditions, from a client’s reported symptom, and the answer in almost all cases appears to be ‘No’.

Moving switftly on, BANT’s next port of call is what can best be described as the ‘Wot, no Witch Doctor’ defence.

We would have hoped that the panel would have included qualified and experienced nutritional therapists who would have been able to assess the performance of the targeted practitioners against the National Occupational Standard, (Skills for Health). As in other professions, assessment of practitioner performance would normally involve experts from that profession.

You can read the National Occupational Standard for yourself here but, be warned, it really is nothing much more than an exercise in superficial bureaucratese which fails, entirely, to address the core problem that lies at the heart of nutritional therapy and other CAM modalities; the absence of a scientifically credible – and in many cases even plausible – body of knowledge to underpin the activities of CAM practitioners.

If you attach qualifications to bullshit then they only thing you’ll get is qualified bullshitters, and if you then go on to build an entire pseudoacademic edifice on top of the those qualifications what’ll you’ve eventually end up with a few expert bullshitters who, lets be frank, are about as much use as a chocolate teapot because, as the saying goes, if you put garbage in, you’ll get garbage out not matter how rigorous your processes are.

Concern was expressed about lack of referrals to GPs but our review reveals frequent reference to working with the client’s GP or consultant. However, several of the clients made up stories that they were either dissatisfied with their GP and did not wish to make contact, would not provide details or said that they were moving to a new GP.

It would help matters considerably were Which? to release the full transcripts of these consultations so we can judge this issue for ourselves but nevertheless is not at all uncommon for CAM practitioners to be faced with clients who’ve turned to so-called alternative therapies out of dissatisfaction with whatever conventional medicine has had to offer The test here is not just that of whether nutritionists are willing to make referrals to GPs but whether they are prepared to insist on making such a referral when its clear that this is the best course of action, and particular when they are legally or ethically obliged to do so and it seems clear from at least some of the practitioners whose activities were scrutinised managed to fail this test.

As the professional body for nutritional therapists, BANT is dedicated to the advancement of nutrition science and the safe, evidence-informed practice of nutritional therapy.

And they’re clearly doing a bang up job of this aren’t they?

Assuming that Which? followed standard practice and chose the therapists they mystery shopped at random, thirteen of the fourteen therapists – and fourteen of the fifteen consultations – failed to live up to this ideal of evidence-informed practice, with six failing to manage safe, and although we’re dealing with a very small sample group relative to BANT’s total membership of around 2,400 practitioners, these are not good odds at all, as David Colquhoun points out:

17 January 2012 The discussion is going on at the Which? web site. A Chris James cast doubt on the reaults because ot the small sample size, and asked for confidence limits, so I gave them.

Chris James

It is easy enough to calculate limits yourself -you don’t even need to be able to do the maths -there are web calculators that do it for you, e.g. http://www.causascientia.org/math_stat/ProportionCI.html

14/15 = 93% failed. 95% confidence limits for this are 69.8% to 98.4%

6/15 = 40% gave dangerous advice 95% confidence limits 19.7% to 64.6%

So despite the small sample size we can say that it’s likely that at least 70% (and possibly 98%) of nutritional therapists fail.

And it is likely that at least 20% (and possibly 65%) of nutritional therapists give dangerous advice.

These results give real cause for concern, despite the small sample size.

Unfortunately, because the CAM industry’s own pseudoregulators and trade organisations don’t tend to publish any data on data on disciplinary sanctions against members we’ve got no way of comparing Which? evidence of the extent of bad practice in the sector against that which might have been collated by BANT.

Instilling public confidence and offering consumer protection is of primary importance to BANT. BANT members are bound by a strict code of ethics designed to protect patient interests and procedures are in place to deal with any complaint brought against a BANT member.

Instilling consumer confidence is, of course, no less important if you’re attempting to sell the Forth Bridge to an obscenely rich American or running a Ponzi scheme – all CAM trade bodies profess to take ethics serious and have procedures to deal with complaints, but very few deal with such matters in an open and transparent manner – typically only those, such as the General Chiropractic Council, that have statutory status and, therefore, little or choice in the matter.

BANT would welcome the opportunity to discuss the future of nutritional therapy regulation to further develop safe and effective practice. We agree that practitioners should come under statutory regulation.

Well, of course they do – statutory regulation would provide nutritionist and other CAM practitioners with both an un-deserved veneer of legitimacy and go some considerable way to opening up the doors to El Dorad, or as we prefer to call it, the NHS and a nice steady, tax-payer funded, income stream, providing that its done under their own preferred terms.

That’s just one of many reasons why statutory regulation of most CAM modalities is an absolutely lousy idea, particularly under the current government with its marked preference for ‘light touch’ self-regulation.

While I’ve been writing this article, BANT have issued an updated statement on the Which? report, most of which amounts to allegations that the report contains ‘inaccuracies and inconsistences’ most of which appear to be based on nothing much more than hairsplitting over definitions and interpretations, plus the usual ad-hominem complaints of bias, e.g.

Furthermore we propose that this report is heavily biased on the grounds of:-

a) Fundamental differences of opinion between dietary guidelines outlined in the Manual of Dietetics (2007) and those that Nutritional Therapy promotes.

b) At least two of the panellists are well known detractors of Nutritional Therapy

c) There was no attempt to have representation for Nutritional Therapy on the panel and thus no attempt to allow discussion or understanding of the Nutritional Therapy process. This can only lead to a conclusion that it was intended that Nutritional Therapy would be seen to be an apparent fail.

Again, without access to the full transcripts, its impossible to assess the validity or accuracy of any of BANT’s counter allegations, although there does appear to be a fairly high degree of selectivity in this issues that BANT have chosen to address. On the headling cancer issue, the Which? report states:

Sarah, posing as a patient diagnosed with DCIS, visited a nutritional therapist who advised her to delay treatment recommended by her oncologist (a lumpectomy and a course of radiotherapy). The therapist suggested that Sarah follow a no-sugar diet for three to six months and told her, ‘cancer lives off sugar; if you feed it sugar it’s going to thrive. If we starve the cancer of sugar then you have a better opportunity of the cancer going away’.

When Sarah asked whether the cancer could progress during this time the therapist said it was a ‘gamble’.

Dr Margaret McCartney, from our panel of experts, says: ‘If cancer treatment were as simplistic as cutting out sugar, surely we would have discovered a cure. This advice is highly irresponsible.’ Our experts rated this consultation as a ‘dangerous fail’.”

To which BANT has given this response.

We have looked carefully at the transcripts provided by WHICH?, alleging that a practitioner recommends a client refrain from pursuing cancer treatment for a period of time rather just follow a change in diet. Whilst the information could have been better presented we have found no evidence in the transcripts that the practitioner directly made this recommendation, and are thus unable to agree with this finding. The practitioner made it quite clear that the client’s oncologist must be involved and that the client should refer the suggestions to them to seek their opinion and agreement.

So BANT isn’t rejecting this particular finding they’re just ‘unable to agree with it’ or – to give the correct term for this response – they’re just weaselling.