If anyone’s unsure as I why I put so much time and effort into chasing down the evidence from the DWP’s failed trial of Nemesysco’s pseudoscientific ‘voice risk analysis’ system then the simple juxtaposition of these two articles should explain things very nicely:

Article 1

Domodedovo International Airport’s Clarifications on GK-1 Voice Profiling Technology Application – 14.04.2006

In response to some publications, where GK-1 multilevel voice analysis system that is being implemented in Domodedovo International Airport was erroneously referred to lie detector systems, press service of EAST LINE Group states that these systems cannot be treated as analogous.

GK-1 that is being implemented by certified specialists of Domodedovo Aviation Security is based on the latest computer technologies and is a further development of the traditional profiling system.

Profiling is a preliminary interviewing by trained specialists, who analyze a person’s look, responses and behavior. This method is actively used by security services of the world largest airports and airlines in order to prevent terrorist attacks and transportation of illegal items and substances.

GK-1 interviewing is performed in an automatic mode. Questions will not be of personal nature and will not prejudice the person’s rights. Such interviewing is related only to prevention of terrorist attacks and transportation of illegal items and substances…

GK-1 multilevel voice analysis system was developed by NEMESYSCO (Israel). The company’s representative in Russia is Urbis Management Systems Software. The testing which has been held by Domodedovo Aviation Security, Areopag-Center, and Drug Smuggling Prevention Department of Domodedovo Customs Authorities at Domodedovo Airport for several months, confirmed that the system can be effectively used in the security zones to prevent:

– terrorist attacks;

– illegal traffic of arms, ammunition, explosives, poisonous substances and drugs…

Note.

The GK-1 system referred to in the above article is based on the same pseudoscientific principles, patents and underlying ‘technology’ as the voice risk analysis system that the DWP unsuccessfully tested on benefits claimants in the UK at a cost of £2.4 million over two years.

Article 2

Domodedovo airport hit by deadly bombing

At least 35 people were killed and 130 injured in a suspected suicide bombing today at Moscow’s biggest airport, Domodedovo, Russian officials said.

A spokeswoman for the healthcare ministry said a further 20 people injured in the bombing were in a critical condition in hospital. Sources said three men were suspected of plotting the explosion, which had the power of 7kg of TNT.

The attack is the most deadly in Russia since last March when two female suicide bombers from Russia’s Mulsim-majority Dagestan region set off explosives on the subway system, killing 40 people. It was Moscow’s worst attack for six years.

It may well be that there is no direct connection between these two stories as its entirely possible, if not likely, that the suicide bomber responsible for this week’s attack on Domodeovo Airport didn’t get as far as entering an area in which the GK-1 system was in use.

That said, a second Guardian article on this attack provides the relevant background that links the two articles:

Domodedovo is the biggest of Moscow’s three international airports and the hub of choice for Russians and tourists, handling more than 22 million passengers a year. Unlike frustration-prone Sheremetyevo airport, home to the state carrier Aeroflot, Domodedovo has a reputation for being relatively efficient.

Its security regime was questioned in 2004 when two female Chechen suicide bombers were able to carry dev-ices on to two passenger planes and detonate them in midair, killing 89.

However, according to one western security expert, the airport reacted “positively” to those incidents.

“When they had the black widows attack, they beefed up security quite significantly,” said Norman Shanks.

So, one of the measures introduced to beef up security was Nemesysco’s worthless piece of pseudoscientific junk, which the airport claims is a ‘development’ of the equally useless ‘traditional profiling system’.

For anyone trying to understand why Domodedovo Airport has suffered such a catastrophic security failure, including the Russian authorities, one need only look to the comments of Phillip Baum, the editor of Aviation Security International – which ran an utterly credulous cover story on Nemesysco’s system in February 2008 – and understand that what you’re actually viewing is yet more evidence of the Dunning-Kruger effect at work:

Philip Baum, editor of Aviation Security International, said the attack had confirmed the industry’s worst fears. “The aviation security community has been talking for years about the threat of suicide bombers detonating themselves at airports so it is no surprise that this has happened.”

Baum said passenger profiling, where people deemed to be behaving suspiciously are singled out for further checks, could help locate terrorists in public areas in airports.

To be absolutely clear on this matter, there is no credible scientific evidence to show that offender profiling works – forget everything that you might have seen in films (‘Silence of the Lambs’) or on the TV (‘Criminal Minds’), that is fiction, pure and simple.

We need to be careful , here, to differentiate between fact and fiction.

It is a fact that many criminals are creatures of habit and adopt the same modus operandi when committing their crimes. In some cases, that modus operandi is sufficiently distinctive as to reasonably be considered to constitute a ‘signature’. As such, if investigators do find such a signature when investigating a crime, they can reasonably infer that the crime was committed by the offender(s) that had previously committed other documented crimes in which the same signature was found and, if the identity of the offender(s) who committed those earlier crimes is known to the police, direct their suspicions, and investigation, accordingly.

This use of this particular type of ‘profiling’ is well supported by empirical evidence and, indeed, a staple element of investigative police work. Depending the physical evidence available to the police at the crime scene, it may even provide the police with some limited information about personal, and particularly physical, characteristics of the offender(s). Basic traits, such as whether the offender is left- or right-handed, their approximate weight and, sometimes height, and even their gender can predicted by a forensic scientist with a reasonable degree of accuracy if the physical evidence is there to support such predictions.

What cannot be done, with any kind of reliability, is the kind of ‘profiling’ one sees in the series ‘Criminal Minds’ in which the investigation team’s resident geek rattles off a series of offender statistics and probabilities from memory prior to giving an unfeasibily detailed – and invariably accurate – prediction of the offender’s age and physical appearance, educational and family background, comprehensive employment and medical histories, place of residence and the precise colour of their underwear.

That kind of profiling is complete and utter fiction, and the research evidence indicates that it’s still fiction even when its practiced in the real world under the guise of forensic psychological profiling.

Naturally enough, that’s an assertion that I’m happy to back up with evidence, starting with Snook et al (2007) [1] which includes both a narrative review and two meta-analyses of the published literature on criminal profiling:

The evidence generated from this research confirms the perceptions of those who have concluded that the CP [Criminal Profiling] field relies on weak standards of proof and that profilers do not decisively outperform other groups when predicting the characteristics of an unknown criminal (e.g., Alison, Bennell, Mokros, & Ormerod, 2002; Muller, 2000). Based on the results of the narrative review and meta-analytic reviews presented herein, profiling appears at this juncture to be an extraneous and redundant technique for use in criminal investigations. CP will persist as a pseudoscientific technique until such time as empirical and reproducible studies are conducted on the abilities of large groups of active profilers to predict, with more precision and greater magnitude, the characteristics of offenders.

Snook et al’s narrative review evaluated 130 articles on criminal profiling from a range of source, including articles authored by law enforcement professionals and psychologists. Amongst the more interesting findings in this part of the paper is the fact that scientific evidence came only joint-third (42%)  alongside appeals to authority (also 42%) in terms of the frequency that particular sources of knowledge were used in the articles under review. The two most frequently cited source of knowledge in these articles were anecdotal evidence (60%) and testimonials (45%) with intuition trailing in last at 23%.

Overall, Snook et al found that articles about criminal profiling relied far more heavily on ‘commonsense rationales’ than empirical evidence when citing sources of knowledge and applying analytical processes to the ‘evidence’; the five most commonly used analytical processes identified in the review were self-serving bias, post-hoc ergo propter hoc arguments, the availability heuristic, illusory correlates and hindsight bias.

Only when it came to integrating evidence into the narratives did empirical methods predominate over commonsense rationales, the most common being references to complex causality and expectations that the theory behind criminal profiling with be revised at some unspecified juncture. Although not a point explicitly addressed by Snook et al, the prevalence of these two arguments does suggest that where these empirical arguments were deployed, it was often for the purpose of explaining away discrepancies between the offender characteristics predicted by the profiler and those subsequently found when the actual offender was identified and caught.

Unsurprisingly, perhaps, Snook et al found that the dominant use of commonsense rationales over empirical methods was most often to be found in articles supportive of the use of criminal profiling and, in particular, in articles authored by law enforcement professional.

Of the two meta-analyses, it need only be noted that both found that supposedly expert criminal profilers were no more successful at identifying the personal traits and characteristics of unknown offenders than laymen given access to the same information.

These findings, together with a critical response from Dern et al (2009) [2] prompted Snook et al to give careful considerations to some of the assumptions that underpin the belief that criminal profiling works, giving rise to two further papers, ‘The criminal profiling illusion: What’s behind the smoke and mirrors?‘ [3]and ‘A failure to find empirical support for the homology assumption in criminal profiling‘ [4] that, in my view, systematically dismantle the argument for criminal profiling.

Note – I should acknowledge the fact that Brent Snook helpfully provides a comprehensive (and free) archive of his full published articles and papers at the ‘Bounded Rationality and Law‘ website of the Memorial University of Newfoundland. If you’re at all interesting is this field, then Brent’s archive – and his work – is well worth reading, particularly his introductory article to a special issue of the journal ‘Criminal Justice and Behaviour’ on ‘Pseudoscientific policing practices and beliefs‘ [5].

If you’re a fan of primary sources then, hopefully, I proved my point about criminal profiling, however, I do appreciate that not everyone is fan of hacking through academic papers so if you’ve only skim-read the last bit without following any of the links, but still want some evidence to back up my point, then can I recommend that you try Malcolm Gladwell’s 2007 New Yorker article, ‘Dangerous Minds: Criminal Profiling Made Easy‘ for starter before moving on to Perri and Lichrenwald’s analysis of the Timothy Masters case [6]:

This paper offers an analysis of the series of events that occurred when a homicide detective contacted an international expert in forensic psychology to assist in the arrest process and the prosecution of a targeted sexual homicide suspect. The forensic psychologist developed a psychological profile of a killer using narrative and drawings made by the suspect to conclude that the suspect’s fantasy was the motive and behavioral preparation for the sexual murder, regardless of the fact that the forensic psychologist knew that there was no direct or physical evidence linking the suspect to the crime. In this article, the authors examine the case of Timothy Masters, who was arrested and convicted of sexual murder based on the testimony of a forensic psychologist while the opinion of a criminal investigative analyst was ignored.

Profiling is not effective – so says the empirical evidence – and the kind of ‘passenger profiling’ referred to by Phillip Baum, in which security officers are trained to look for people behaving suspiciously, isn’t actually profiling at all and has more in common with kind of cold reading used legitimately by stage magicians (and illegitimately by self-styled medium and psychics). Neither practice is a substitute for good quality, intelligence-led, policing, least of all when backed up by the kind of pseudoscientific junk technology peddled by Nemesysco.

At best, these things are a waste of time and money. At worst, an over-reliance on pseudoscientific technology – and psychology – will breed complacency and over-confidence amongst security professionals, the kind of complacency and over-confidence that, sadly, contributes to the tragic events that took place, this week, at Domodedovo Airport. An erroneous and wholly misplaced belief in pseudoscience as a ‘security tool’ is not, of course, directly responsible for the murder of 35 people at a Russian Airport – that responsibility lies squarely with those who planned and carried out the attack. However it can – and I would argue does – make it just that bit easier for those who harbour murderous intentions to turn their intentions, successfully, into actions and, consequently, bears some indirect responsibility for those deaths.

References

[1] Snook, B., Eastwood, J., Gendreau, P., Goggin, C., & Cullen, R. M. (2007). Taking stock of criminal profiling: A narrative
review and meta-analysis. Criminal Justice and Behavior, 34, 437-453.

[2] Dern, H., Dern, C., Horn, A., & Horn, U. (2009). The fire behind the smoke: A reply to Snook and colleagues. Criminal Justice and Behavior, 36(10), 1085-1090.

[3] Snook, B., Cullen, R. M., Bennell, C., Taylor, P. J., & Gendreau, P. (2008). The criminal profiling illusion: What’s behind the smoke and mirrors? Criminal Justice and Behavior, 35 (10), 1257-1276

[4] Doan, B., & Snook, B. (2008). A failure to find empirical support for the homology assumption in criminal profiling. The Journal of Police and Criminal Psychology, 23, 61-70

[5] Snook, B. (2008). Introduction to special issue: Pseudoscientific policing practices and beliefs. Criminal Justice and Behavior, 35 (10), 1211-1214

[6] Perri, Frank S. and Lichtenwald, Terrance G. (2009). “When Worlds Collide: Criminal Investigative Analysis, Forensic Psychology and the Timothy Masters Case”,Forensic Examiner, 18:2