When stories start appearing about the government’s plans for a nationwide rollout of the use of so-called ‘lie detectors’ to trap ‘benefit cheats’ there seems to little else for a diligent blogger to do but run the whole idea through their own personal bullshit detector to see what shakes loose.

So, lets start at the beginning by knocking a large hole in the biggest lie of all, the idea that what the government are rolling out here is actually a lie detector, or as its manufacturer seems to prefer, ‘voice risk analysis technology’.

The science which underpins systems of this kind is nothing particularly new.

Voice stress analysis (VSA), which is the basic foundation of these systems, has been around since the 1970s and, in its standard form, works by detecting involuntary psychophysiological modulations in a subsonic component of the voice called the ‘Lippold Tremor’ at a frequency of between 8Hz and 12Hz. As such, there is a reasonable body of research available relating to the use of this technology, most of which validates the claim that VSA systems can and do reliably detect whether an individual is stressed.

The basic problem with this, however, is that people can feel stressed for many different reasons, including getting the third degree from some twat of a council officer about their benefit claim.

The actual system in use in this case is not a straightforward VSA system but one based on what’s called ‘Layered Voice Analysis’, which is of a much more recent vintage and purports to assess the subject’s voice across a much wider frequency spectrum than conventional VSA, a distinction that we’ll come back to in a moment.

So far as this system’s use by local authorities is concerned, this comes in two parts.

In the first instance, the technology is used as a threat in order to persuade some of those who may be cheating the system to turn over a new leaf and not risk trying it on, and its in this mode that the system has delivered most of what has been claimed of it by way of successes as, according to Harrow Council, which ran the first pilot of the system on housing benefit and council tax benefit claimant, the number of existing claimants who voluntarily declared a change of circumstances which meant that they no longer needed to claim one of these benefits doubled in the first three months of the pilot.

So that’s obviously a good thing, then?

Maybe, but then again maybe not…

…because when a benefit claimant withdraws their claim the local authority is under no obligation to enquire after or ascertain whether they withdrew the claim for good reason or whether they were merely so intimidated by the threat of taking this test that they withdrew their claim even though they are perfectly entitled to the benefit they were claiming. I’ve no doubt that, as far as the council are concerned, they don’t give a toss about this fine distinction as each withdrawn claim is money saved (and also money lost due to the manner in which local authority revenue support grants are calculated but that’s another story) but as a matter of basic justice this does actually matter as it means that there is a clear possibility that genuine claimants may have been frightened off claiming money they are legally entitled to, money that they actually need.

As ever, when bullying and intimidation is deployed by the apparatus of the state it is the most vulnerable members of society who are the first to suffer, and when it comes to a system of this type and the manner in which its being, the big concern has to be the potential impact it may have on claimants who have a history of mental illness. Laugh as we might at the paranoid image of a guy wearing a tinfoil hat to prevent the government/aliens beaming signals directly into their brain, that image carries an underlying grain of truth. Some people are genuinely paranoid about such things and the prospect of being subjected to an electronic test that will purported reveal their innermost thoughts is more than enough to make such individuals run for cover and withdraw their claim despite it being a legitimate one.

In piloting this technology it is not enough for the government, Harrow Council or any of the local authorities who’ve been used this technology simply to assume that every additional claimant who withdrew their claim over and above the number who did so before this system was implemented must automatically have been cheating the system. An assessment of the extent to which it may also act to deprive genuine claimants of benefits to which they are legally entitled should also have been a feature of the pilot but, on what little published information is available, I can find nothing to suggest that this has been the case.

The second use to which this system is put is, naturally enough, that of actually administering the test to claimants and that, of course takes us on the question of just exactly reliable is this system given that a ‘positive’ result for voice stress places an individual under suspicion that they may be defrauding the system…

…and the answer to that question if one looks at both the research literature and at the data from the Harrow pilot, doesn’t seem at all promising.

Before getting into the detail, it’s worth noting that the patent holder/developer of the LVA system that has been used in the pilot, Nemesysco, makes some pretty extravagant claims about its technology which, in its latest iteration, can allegedly identify:

various types of stress, cognitive processes, and emotional reactions.

Stress..? Okay, although I’m a touch dubious about the ‘various types’.

Emotional reactions..? Perhaps, but how accurately?

Cognitive processes…? No, I really don’t think so.

Although it’s being used over here, in a system provided by a company called Digilog, purely for assessing (and threatening) benefit claimants, Nemesysco’s main sales pitch in the US is directed towards Law Enforcement (hence the company’s seriously twatty name) and the more paranoid arms of corporate America, all of which ensure that the more you read about this ‘wonder’ system the more your personal bullshit detector goes right of the scale.

For example, the big new feature is the latest version of the system (6.50) is:

Added SAF (Sexual Arousal Factor) analysis – suitable for sexually oriented crimes investigations. Find the motive behind the crime!

All of which conjures nothing more than a series of rather uncomfortable mental images of Alex the Droog being subjected to Ludovico’s Technique.

So that’s the hype, but about the evidence base?

Despite the heavy law enforcement sales pitch, the technology has not fared particularly well when put to the test.

A field study funded by the US Department of Justice and published in March this year, in which Nemesysco’s LVA system was one of two systems evaluated for its ability to detect lies about drug use found that:

their ability to accurately detect deception about recent drug use was about 50 percent.

In which case, a coin flip would do just as well.

Elsewhere, a 2005 evaluation study presented to the annual Hawaii International Conference on System Sciences provided somewhat better results – 63% accuracy overall across 516 tests…

…but this was a ‘data only’ study in which analysts with years of training behind them made the evaluations based on raw data provided by the test system and the study still generated a false positive rate of 23%.

By comparison, those ‘analysing’ the tests in the Harrow pilot will have had nothing like the same level of training and will be relying heavily on the processing software to distinguish the ‘cheats’ from the genuine claimants, all of which nicely explains why the false positive rate after the first three months of the Harrow pilot was running at around 64%. Of the 998 individuals tested in Harrow, 119 (12%) were flagged as having a ‘high risk’ of dishonesty and only 43 of these were then found to be on ‘incorrectly paid benefits’, which is a rather ambiguous turn of phrase and one which raises distinct suspicions that at least some of the ‘incorrect’ payments to these claimants may have stemmed from genuine errors and or misunderstandings rather than from dishonesty or fraud.

Fortunately, our criminal justice system has more sense than to let a system like this and the ‘evidence’ it generates anywhere near a court room as you wouldn’t wish the experience that Michael Crowe, of Escondido CA, went through on anyone…

After Michael Crowe’s 12-year-old sister, Stephanie, was found stabbed to death in her bedroom, the Escondido, Calif., police department brought him into the station for questioning and hooked him up to the CVSA in the middle of the night.

From tapes recorded during his questioning, Crowe answered “Yes” when the detective asked, “Is today Thursday?” But when Crowe replied “No” when asked whether he took Stephanie’s life, the detective told him that he had failed the test.

“I started to think that, you know, maybe the machine’s right, especially when they added on top of it that the machine was getting my subconscious feelings on it, that I could be lying and not even know it,” Crowe, now 21, told “Primetime.” “They said the machine is more accurate than the polygraph and is the best device for telling the truth, for finding the truth.”

Once the detective told him that he had failed the test, Crowe said he began to doubt his own memory and wonder whether he might have killed his sister.

“I didn’t want to go to prison, and I just wanted to be out of that room,” Crowe recalled. “So my only option was to say, ‘Yeah, I guess I did it,’ and then hope for the best.”

Luckily for Michael, DNA evidence which identified his sister’s real killer was found, but only a week before he was due to stand trial and in the shit-storm that blew up afterwards, one VSA expert admitted that the system was routinely used as a scare tactic to elicit confessions.

On of the most striking things one finds on reading research evaluations of these systems is the extent to which studies that fail to deliver better than chance results are routinely written off with the explanation that the conditions under which the evaluation was undertaken failed to place test subjects under a sufficient ‘threat’ to deliver the expected response. Most studies have this kind have evaluated these systems with an eye of their use in law enforcement rather than as a means of uncovering benefit ‘cheats’ – and the Digilog system in use in Harrow and other councils is actually design for and marketed to insurance companies and used to ‘assess’ customers making claims – but the fact remains that what these studies appear to indicate is that intimidation is not merely a side-effect of these systems but a necessary component of their use – a latter day rubber hose.

There’s nothing in the case of Michael Crowe to suggest that he was anything other than a normal teenager at the time he was subjected to a VSA test and then intimidated into giving a false confession on the basis of the test having decided, falsely, that he was lying, which raises pertinent questions about just exactly where that might leave those benefit claimants who have a lower than average intelligence and/or a diagnosed learning disability.

It’s worth pointing out here that I’m not alone in harbouring serious reservations about the use of these systems. The Disability Alliance have produced a factsheet which is well worth a read and which notes that, in addition to the point I made about the potential impact of these system on individuals with mental health problems, the system that’s being rolled out by the DWP:

has not been tested on those who have speech/language impairments – about 700,000 people in the country.

And that:

It appears that the DWP does not have any way of identifying those claimants with disabilities who could innocently fall foul of this system.

They also raise perfectly valid questions about record-keeping and whether individuals who have been classified as being ‘high risk’ by the system only to be found to to be ‘low risk’ following a personal visit/interview will have their records amended accordingly and that those classified as high risk by the system may face a much slower claims process through no fault of their own.

The TUC is, similarly, unhappy with the introduction of the system, pulling out the classic if, almost certainly apocryphal, police tale of the Xerox ‘lie detector’ to illustrate its concerns:

“I remember the Xerox machine being a valuable investigative tool as well. All the detective had to do was push the magic button and out came a piece of paper with the word LIE or TRUTH on it. ‘What’s your name?’ was the first question. ‘Sam Jones’ was the reply. Push the button and out came the answer ‘TRUE.’ ‘Okay Sam, you’ve done pretty well so far. Now, did you steal your Bill Doe’s bike this morning?’ asked the detective. ‘No,’ says Sam. Push the Xerox button, and here comes the answer, ‘LIE.’

As the Snopes article notes, the only verified ‘sighting’ of this particular story is a fictional one which appeared in the series ‘Homicide: Life on the Streets’ – and if I recall the episode in question correctly, I think it was Richard Belzer’s character, Det. John Munch, who pulled off the photocopier stunt.

That fact that the TUC have chosen to illustrate its point with what is almost certainly an urban myth in no sense negates its argument that innocent people can. and do, confess to ‘crimes’ they haven’t committed and while Harrow Council and the DWP are, naturally, keen to talk up the systems presumed ‘successes’ there is nothing whatsoever to suggest that either have shown any interest in conducting any kind of objective evaluation of the system of the kind which examines the possible extent of its ‘failures’ in terms of the extent to which legitimate claimants may have been denied benefits to which they were, and are, entitled as a result of their having been intimidated into withdrawing a valid claim.

So, what all this amounts to is bad science?

Yes… and no.

In so far as we have sound scientific evidence to go on, the basic premise that its possible to evaluate whether and individual is under stress by assessing tonal micro-modulation in their voice is basically sound and, when confined to that task, and that task alone, the success rate of this technique stands up to scrutiny and gives consistently high success rates in region of 95-98%.

So, underpinning these systems is an element of good science, but where it becomes bad science is when, as in the this case, it is being hopelessly misapplied to a task for which the best evidence we have available indicates it is ill-suited and, at best, no more effective in detecting benefit ‘cheats’ than flipping a coin. In fact, when one looks at the number of false positives evident in the data from Harrow trial, it’s actually somewhat less successful than flipping a coin.

In the broader civil liberties debate there are some occasions where rather too much is made of the idea that a particular policy or initiative somehow alters the relationship between the citizen and the state, a theme which is typically accompanied by a sizeable amount of hyperbolic shouting about ‘Stalinism’ and the use ‘Stasi-like tactics’ but in this case there is considerable merit in deploying this line of argument – the one of the changing relationship not the ‘Stalin’ thing, which is becoming hopelessly overused to the point at which its losing all meaning and impact in much the same way that the left’s reflexive characteristion of anything it disgrees with as ‘fascist’. The use of these system does implicitly treat all benefit claimants as suspects whose personal honesty and integrity is there to be proven – make a benefit claim with any one of the councils who have taken up this system and you are automatically treated as being ‘potentially’ guilty of fraud until the little black box in the corner says otherwise, and that is a wholly unacceptable state of affairs.

That said, before anyone goes off an starts riffing on the subject of the secret police and tractor factories, not least because the mental image this conjures is nothing more than of Rik (from the Young Ones) time-shifted twenty-five years into the future but still ranting at the world in a situation where he lives the surburban life as a bloated Daily Mail-reading middle manager with two kids, a mortgage and a put-upon wife who only succeeds in getting through the day with the assistance of Prozac and white wine while praying desperately for the day that he kids leave home so she can also walk out on the marriage, this ‘you’re all suspects now’ mentality is by no means only a feature of the state. So far as one can see, the insurance companies who’ve adopted this system and for whom, in fact, the system was designed, are no more inclined to exhibit any concern as to the extent to which it might be generating false positives any denying legitimate claimants redress for their losses.

It’s not socialist authoritarianism driving forward the introduction of these systems, but capitalist authoritarianism, a point which too many self-styled right-wing libertarians are prone to overlook.

And with that observation I’ll leave thing there for now but for putting up one important general question for the ‘Cry Stalinism and Let Slip the Dogs of Free Market Capitalism’ brigade to ponder.

If socialism genuinely lies behind the systematic ratcheting up of state authoritarianism across the globe then how was it that when the IMF and World Bank were swanning around th heavily endebted developing nations of the world during the 80s and 90s and demanding that those counties make savage cuts in public expenditure and privatise public utilities in return for the munificence of being bailed out of their financial difficulties, they were also privately advising those same countries to increase expenditure on policing and the armed forces and bring in laws which dramatically increased the power and authority of the state at the same time?

And the effects of this? Well, for that, one need only read any of the following…

Who Shot Argentina? The Finger Prints On the Smoking Gun Read ‘I.M.F.’
Sell The Lexus, Burn The Olive Tree
Failures Of The 20th Century: See Under I.M.F.
Eyes-Only Memos Show Who Done It