I’ve had a couple of comments  on yesterday’s article about the slogan ‘ATOS kills‘ which rather serve to illustrate the problems you invariable run into when you try to inject  a bit of evidence and reason into a narrative based largely, if not entirely, on anecdotes and emotive outbursts.

One directs me to a comment at CIF that apparently either answers my question or adds to the argument, or maybe a bit of both. I’m not entirely sure that it does, but to save time I’ll reproduce and analyse the comment in full.

I need to correct something about this ’32 people a week’.

Originally this was reported in an article by the Mirror, by taking the total number who died in a certain amount of time and dividing it by the number of weeks. But they got it wrong- it was not the number who died after being found ‘fit for work’ because the DWP collects virtually no information about them. The figure was for those put into the WRAG.

So far, so good – the figure quoted by the Mirror do indeed relate the WRAG.

Why was this significant? Because these are people which the process has deemed can not work currently but who are expected to when appropriate support is given or their condition is expected to improve.

Yes, this is a fair description of what the work related activity group is for.

This is worse than if they had been simply found fit for work because a certain number of people found fit for work are bound to die anyway because there is a mortality rate for the general population for any age and regardless of diagnosis. Some astonishing failures of math were used in response to the Mirror article, claiming that the mortality for those in the figures were no worse than the general population. They unfortunately didn’t check the Mirror’s figures properly though because the Mirror was talking about the WRAG, not those declared fit for work, among other things.

Again, correct, but not an error that appears in my own article which doesn’t suggest that we should treat mortality rates as an appropriate comparator for evaluating the claim that ‘ATOS kills’.

What I do note is that one would fully expect to see a higher mortality rate amongst claimants placed in the support group, which is for the most ill and/or disabled claimants, than we would for claimants in the WRAG, who are still unwell and/or disabled but, if the system is assessing claimants correctly, less so than than claimants placed in the support group.

Were I to have commented on the morality rates in these group compared to the general population, I would, of course, have noted that one would expect both groups to exhibit higher mortality rates than the general population as the latter will include a substantial number of people who are neither ill nor disabled and whose risk of dying, in any given period, will therefore be significantly lower that people in either of these groups, all other things being equal.

If the process says a group of people are going to get better, fails to recognise when they are actually going to get worse, ignores all the previous existing evidence that a person claiming these benefits is more likely to die whilst claiming than come off it and go to work- and then a disproportionate number of them die, well that’s bad. It’s worse than if they had simply been called ‘fit for work’ because that doesn’t claim that they are going to get better, that they are less likely to die than not or require and can be supported into work.

Yes, if the system fails to provide an accurate assessment of people’s condition and prognosis, that is bad but certainly not worse than labelling them ‘fit for work’ is that is equally based on inaccurate assessment of their condition.

Being placed in WRAG rather than the support group means that claimants get a slightly lower rate of ESA – £96.75 per week rather than £103.35 per week at current rates – and are required to attend six ‘work-focussed interviews’ with an advisor over the course of a year to discuss the kind of work you think they could do and received help with all the usual job search gubbins, such as knocking up a CV. If and when the government put a one-year limit on claiming contributory ESA then things get a fair bit stickier for people placed in WRAG as, after 12 months, they will be automatically be pushed down onto JSA at the standard rate – £71 a week for a single adult – assuming that they’re eligible to claim income related JSA in the first place, unless their condition has worsened sufficiently to justify moving them up into the support group. Claimants in the support group, however, only have their condition reassessed when their claim comes up for renewal or if they either notify the DWP of a change in circumstances or even choose to submit a fresh claim for whatever.

To suggest that that’s somehow worse than being incorrectly labelled as ‘fit for work’. which means an immediate transfer to JSA and a drop or, in some cases, complete loss of benefits and a return to the treadmill of weekly signings at the Jobcentre, with all that entails, could be considered to be worse than being incorrectly placed in WRAG as a result of inaccurate or inadequate assessment is, to day the least, a bit of a reach.

Whether or not the system itself is fair or unfair in its construction is, of course, and entirely separate question from it is being implemented fairly, and outside the scope of this article.

If a doctor says a person is healthy and they drop dead, the doctor has the evidence to fall back on to justify their judgement at the time even if it turned out to be wrong- there is an unavoidable minimum percentage of this happening because of general mortality rates in the working-age population.

Not quite – it all depend on whether the cause of death is related to something that the doctor should reasonably identified and provided treatment for at the time of the examination – there is such a thing as malpractice.

If the doctor says a person is ill but they will get better, then that is a problem because if that happens more than a certain number of times, the evidence base has changed and therefore diagnostic patterns should change and with some urgency.

Sorry old boy, don’t quite understand your banter, and I’m really not sure you understand how this whole diagnosis business works in the first place.

First, the majority of diagnoses made by doctors are based on probabilities. If you turn up to an appointment with a GP complaining of a runny nose, sore throat and a bit of headache then  you’ll probably be told that you’ve got a cold because that is by far the most common, but not the only, cause of those symptoms. If those symptoms don’t abate after a few weeks and you head back to see your GP then they’ll start to consider the possibility of other causes for your symptoms, starting with the next most common condition on the list of those with the same kind of symptoms. So maybe you’re first trip to the doctors was at the beginning or March, but now its the beginning of April, so if you don’t have a cold, the time of year of your visit may well suggest that your problem is likely to be hayfever, and so on.

Unless you under go a specific test, or set of tests, for a specific condition then your doctor is almost always assessing your health in terms of probabilities, and even if you do undergo some sort of tests, more often than not the interpretation of your results will also rely heavily on probabilities – there aren’t that many clinical tests out there than have been show to be 100% accurate, most have a margin of error and at least some risk of generating both false positives and false negatives.

By the same token, any assessment of a patient’s prognosis once their condition has been diagnosed correctly will still be a matter of probabilities. Your doctor will tell you that your broken leg should heal up just fine after being set correctly and immobilised in a cast because that’s what the vast majority of broken legs do – but in a small number of cases that doesn’t happen for reasons that your doctor cannot predict in advance and, as a result, some people wind up having their their leg amputated or worse, when it doesn’t heal up as expected because…

… well, because shit happens. You may have contracted an infection or you may have an underlying medical condition that only became apparent when your broken leg didn’t heal up as expect, contingencies that doctors tend not to mention at the time that they’re putting your leg in cast because they really don’t crop up very often and, in any case, doctors are, for the most part, nice people who don’t really want to be scaring the shit out of their patients by regaling them with the full of ins out of everything that could possibly go wrong when the chances of shit actually happening are extremely remote.

The odds of someone with a particular condition getting better may well alter over time in response to new treatments and procedures coming to play, or those new treatments may well have no overall affect on the odds. New drugs come onto the market all the time that don’t have any appreciable impact on the big, top level, indicators like mortality or morbidity, but nevertheless receive marketing approval because, on the whole, any side effects associated with the new treatment are a bit less shitty that the one’s associated with existing treatments.

And, of course, the most unfair question question any doctor can face from a patient is ‘how long have I got, Doc?’. At the level of an individual patient, its an unanswerable question – your doctor can look at the data and tell that 505 of people with your condition survive past two years but neither they, nor you, have anyway of knowing an advance whether you’re going to get lucky and find yourself amongst the 50% who do survive that long. Many people’s statistical illiteracy and inability to grasp the extent to which medicine relies on probabilities lead, of course, to all sorts of nonsensical situations based on fallacious reasons from people who blow life savings on a desperate attempt to complete as much of their bucket list as possible, only to end up flat broke when they don’t die at the appropriate hours, to the lucky dipshits who beat the odds and ascribe their survival past the zero hour their doctor gave them to  a regular diet of dingleberries and three good shits a day.

In short, it’s fucking complicated – there are any number of good reasons why doctors and policy makers get their numbers wrong – and, admittedly, a lot of bad ones too – so I’m finding it just a bit difficult to see where this whole ‘diagnostic patterns’ argument is going or even where its author thinks its going.

The fact that this isn’t being treated seriously when it is a matter well within the control of those who set policy is a damning sign that they never had an evidence base to start with and don’t care.

Maybe they didn’t have an evidence base to start with, or maybe the evidence base just wasn’t very good or, perhaps, the evidence base was just fine, and the problem lies in they way its been interpreted by policy makers or the faulty assumptions they’ve made from their flawed interpretation of the evidence or maybe – probably – the design of this system is predicted far more on a desire to cut the welfare budget, by hook or by crook, than it is on assessing people’s needs correctly. And maybe, right now, the policy makers genuinely don’t care so long as the figures on the balance sheet add up at the end of the year to the kind of saving that the Treasury is demanding.

None of this answers or adds to the question of whether or not the new system has any material impact on mortality rates amongst the people within the system or the failure of campaigners, to date, to supply evidence to support the assertion that ‘ATOS kills’.

Comment number two is shorter and in many respects more straightforward, although it still takes a bit of unpicking.

People placed in WRAG die at 2.5 times the expected rate for people of working age.

Again, this is hardly surprising.

So on average, people expected to be able to return to work in the near-ish future are very unwell compared to the working population.

Well yes, but being unwell relative to the working population is not the same as being unfit for any kind of work – there are, I guarantee, people who’ve been placed into WRAG who are not only fit to undertake some types of work but would cheerfully bite your hand off for job.

The moral question there, is not whether or not they should be put into this group but whether, if and when the one year rule does come into effect, its reasonable to penalise these people for being unable to secure employment  to their being faced with barriers over which they have little or no control. It is right to penalise people, financially, because their health limits their work options to job for which they lack the right skills and, perhaps, more to point where gaining those skills might be beyond either their intellectual or financial means or require them to undertake a level of training that cannot be completed in a matter of twelve months, or less? Should we penalise them if they can’t find work because many employers still discriminate against them – yes, the law says this shouldn’t happen but it does and it came nigh on impossible to prove discrimination when you’re already on the outside of the labour market looking in. After all, if the additional money people receive while on ESA, over and above standard JSA rates, is intended to reflect the fact that they have additional – and more expensive – needs due to their poor health, why should that additional money disappear after 12 months if their health needs remain the same and they cannot, for whatever reason, be held to be responsible for failing to get a job in that time?

These are important questions but still ones that have little or nothing to do with the question of mortality rates and as the comment introduces a point of comparison between people in WRAG and the rate expect for ‘people of working age’, the first question we have to ask is whether or not this is actually a fair comparison.

Who are these ‘people of working age’?

Well for the time being, at least, anyone aged between 16 (or possibly 18, depending on exactly how the statistics are compiled) and either 59 (for women) or 64 (for men).

Now, if you pull the figures and plot the number of people in that group by age at any given time, you’ll get a very specific age distribution graph, on that reflects changes in patterns of births and deaths over time and, if you do the same thing just for people in WRAG you’ll get another age distribution graph, one that I can guarantee will look very different to the age distribution for the working age population and the one thing you’ll certainly notice is that the distribution on the WRAG graph will be heavily skewed to the right, showing that, overall, there are a greater proportion of older people in WRAG than their are in the working age population as a whole.

So when someone tells you that people in WRAG die at 2.5 times the expected rate for people of working age, the first question you need to ask is whether that’s an age-standardised figure, i.e. one adjusted for the difference in the age distributions in each group, or not. I fit isn’t based on age standardised rate then a proportion of that 2.5 times figure, and almost certainly a high proportion, will be due to nothing more than the difference in the age distributions between the two two groups.

That’s why I made the point, albeit in much briefer manner, that if you want to stand the claim that the new system is having an adverse impact on mortality rates in any of these groups then what you have to do is compare the mortality rates amongst people on comparable benefits and with comparable levels of disability/ill health from before the news system was introduced with those for people under the new system. This will take quite a bit of work as you’ll need the mortality data and medical records for a good sized, randomly chosen but age standardised sample of benefit claimants from under the old system, people who were alive and in receipt of benefits at the beginning of the year on which your comparison is based, and preferably from the most recent year for which you can this data in order to avoid the results being skewed by any new treatments that may have on-stream since. You’ll also need to use the medical records for this sample group to run everyone through the new assessment to identify which of the new groups each individual person in the sample group would, most likely, have been assigned to had the current system been in place at the time, although you might be able to ‘cheat’ a little by cross-referencing the sample data with the current system and using people’s current classification, where people in the sample group are still alive and have been assessed under the new system, as long as their condition has not altered significantly over time, either for better or for worse.

Do all that and we’re cooking on gas – we’ll have the baseline data we need to assess the truth value of the claim that ‘ATOS kills’. Without it, the slogan is no more than a piece of empty political rhetoric and can reasonably be dismissed as such. The plural of anecdote is still not data no matter how much time and effort some people put in to avoiding that awkward fact.

The issue isn’t just, if at all, about causality.  It’s about very sick people – people who die from their illnesses – being told that they can work, or are close enough to being able to work to be able to attend interviews, work placements and experience, training, skills assessments and so on.

Which issue are you talking about?

The truth value of the claim that ‘ATOS kills’ and the validity of the article that Mirror ran and the figures within it, or the broader issue of the fairness, or otherwise, of the system as a whole?

I know perfectly well what I’m talking about and I also know, perfectly well, when someone’s trying to shift the goalposts because they don’t actually have an answer to the point I’ve made which is that the ‘evidence’ as it relates to mortality rates in these groups does not support your argument because it isn’t actually evidence, just a bunch of content, and context, free numbers that contain no useful, or even meaningful, information. There may be, and are, other sources of  evidence on which you can hang you argument, not least the large number of assessments that are currently being overturned on appeal, but until someone put in the time and effort to generate the baseline mortality data necessary to make a fair ‘before and after’ comparison of the impact of this new system on claimants’ health and well-being then there’s no point pursuing the mortality rates angle because you just don’t have the evidence to stand it up.

People didn’t suddenly start dying from their illnesses when the new system was introduced – they died from their illnesses under the old system as well and its only when you can show a difference in mortality rates under the two system that you can make anything of the fact that people are dying.  Whether or not some of these people are well enough, before they die,  to attend interviews, undertake training or even hold down a job of some description is a different matter.

And yes, it is about causality.

The slogan ‘ATOS kills’ is quite obviously intended to suggest that the stress of dealing with the system is shortening the lives of some claimants and if not actively precipitating their death at least acting a contributory factor. This may indeed by true in some cases but still highly unlikely to be the case across the board. You can’t automatically blame ATOS or the system if a claimant accidentally walks in front of bus, or gets beaten to death by their partner or even if their cancer suddenly takes an aggressive turn. Yes people in the WRAG group are ill and/or disabled and, yes, some of them die but again these two facts are not necessarily connected in every single case. Some sick and/or disabled people die of causes that are entirely unrelated to the reasons why they’re sick and/or disabled or for reasons in which stress cannot reasonably be considered to be risk factor. Ideally, we’d exclude these people when we run the numbers but as that’s often an  impractical proposition, the best we can do is assume, not without good reason,that in genuinely random samples, the number of people dying of wholly unrelated causes will be near enough the same in both groups and, therefore, cancel each other out.

However much you might want to skirt around this issue, you cannot make the mortality argument stick on any level without producing credible evidence to show that mortality rates have increased under the new system, just as evidence that mortality rates haven’t changed wouldn’t automatically mean that its reasonable to make sick people jump through hoops the for pathetic sums of money that the state insists they need to live on. You have compare apples with apples and oranges with oranges if your evidence, and your arguments, are going to stand up to scrutiny.

These are different issues and different arguments. Standing them up requires different types, levels and standards of evidence – to suggest otherwise just isn’t honest and, in the long run, it doesn’t help the credibility of your arguments when you get caught bandying around statistics that you can’t stand up.