Title: What did Bain really say? A
forensic analysis of the disputed utterance based on data, acoustic analysis, statistical
models, calculation of likelihood ratios, and testing of validity

The David Bain legal
case is (in)famous in New Zealand. In 1995 Bain was convicted of murdering his
family. He maintained his innocence and a retrial was eventually held in 2009.
He was found not guilty. Prior to the retrial, a police officer thought he
heard the words “I shot the prick” in the telephone call that Bain made to the
emergency services shortly after the murders. The defence contended that these
were not the words spoken. An alternative hypothesis which emerged was that
Bain had said “I can’t breathe”.

A number of forensic experts offered their
opinion, but all except one focussed on what they heard. Rose, in contrast,
pointed out that what anyone hears is irrelevant, what matters is what
Bain said, and the latter should be assessed via acoustic analysis not
auditory perception. Further, the proper way for a forensic scientist to
evaluate the strength of evidence is via a likelihood ratio: What is the
probability of getting the acoustic properties of the disputed utterance if
Bain had said “I shot the prick” versus what is the probability of getting the
acoustic properties of the disputed utterance if he had said “I can’t breathe”.

The key segment of the recording which Rose focussed on was the part
immediately following “I”. Were the acoustics of this segment more likely if
what had been said was a postalveolar fricative (a realisation of the first
phoneme in “shot”) or were they more likely if what had been said was a palatal
fricative (a realisation of the first phoneme in “can’t”)? Rose concluded that
the likelihood of the latter was greater than that of the former, but at the
time he did not go as far as building a database, and performing acoustic and
statistical analyses to calculate a numeric likelihood ratio, and to test the
validity of such procedures.

I have now conducted a preliminary version of such
an analysis and in this presentation provide a step-by-step exposition of how
to calculate and test the validity of a numeric likelihood ratio for the
disputed utterance in the Bain case.

Dr Geoffrey Stewart Morrison received his PhD from the University
of Alberta in 2006. His dissertation focussed on statistical modelling of
speech data. He began working on forensic voice comparison in 2007 as a
Research Associate with Dr Philip Rose at the Australian National University.
Since 2010 he has been a Senior Researcher and the Director of the Forensic
Voice Comparison Laboratory, School of Electrical Engineering &
Telecommunications, University of New South Wales. He is also an Invited
Lecturer in the Judicial Phonetics Specialisation, Master in Phonetics and
Phonology Programme, Spanish National Research Council / Menéndez Pelayo
International University, an Adjunct Associate Professor in the Department of
Linguistics, University of Alberta, and is Chair of the Forensic Acoustics
Subcommittee of the Acoustical Society of America. In collaboration with the
Australian Federal Police, New South Wales Police, Queensland Police, National
Institute of Forensic Sciences, Australasian Speech Science and Technology
Association, and the Guardia Civil, he is the lead researcher on an Australian
Research Council Linkage Project on making demonstrably valid and reliable
forensic voice comparison a practical everyday reality in Australia. He has
published on forensic voice comparison and on the evaluation of forensic
evidence in leading journals in acoustics, speech processing, forensic science,
and law. (Any opinions expressed by Dr Morrison are his own and do not
necessarily represent the policies or opinions of any of the above-mentioned
organisations or individuals.)

21 March 2012

Research Seminar on the Bayes and Law network at the Jill Dando Institute for the Forensic Sciences

Professor Norman Fenton will be giving a seminar on the work of the Bayes and Law Seminar at
UCL’s Jill Dando Institute for The Forensic Sciences. For more
information or to register to attend, click here.

As part of the department of Electronic Engineering and Computer Science's Distinguished Lecturer Seminar series, network member Professor David Balding will speak on DNA profiling at Queen Mary, University of London on 29 February, 2012

Recently, forensic DNA profiling has been used with far smaller volumes of DNA than was previously thought possible. This "low copy number" profiling enables DNA to be recovered from the slightest traces left by touch or even merely breath, but brings with it serious interpretation problems that courts have not yet adequately solved. These problems have contributed to important cases collapsing or convictions being overturned, for example in R v Hoey in Northern ireland, and the case of Knox and Sollecito in Italy. The most important challenge to interpretation arises when either or both of "dropout" and "dropin" create discordances between the crime scene DNA profile and that expected under the prosecution allegation. Stochastic artefacts affecting the peak heights read from the electropherogram (epg) are also problematic, in addition to the effects of masking from the profile of a known contributor. I will outline a framework for assessing such evidence, based on likelihoods that involve dropout and masking by stutter and other artefacts, and discuss possible options for modelling dropin. I will apply it to casework examples and reveal serious deficiencies in some reported analyses.

David Balding was educated in his native Australia before coming to the UK to study for a PhD in mathematics at the University of Oxford. He then held a junior academic post at Oxford for a year before moving successively to Queen Mary London, the University of Reading, and Imperial College London. Since October 2009, he has been Professor of Statistical Genetics at the Institute of Genetics, University College London. David researches a broad range of mathematical and statistical problems in genetics – evolutionary, population and medical. He has also developed widely-adopted methods of analysis for the interpretation of forensic DNA profiles, summarized in his monograph Weight-of-Evidence for Forensic DNA Profiles (Wiley, 2005). On occasions, he acts as an expert witness for cases involving complex DNA profile evidence, and he has until recently been a member of the External Advisory Group of the UK Forensic Science Service. He is a Fellow of both the Royal Statistical Society and the Society of Biology.

Dec 2011

Where next for Bayes? Can we agree on the role of
Bayesian reasoning in Criminal Justice?

Brief report on the first meeting of the Bayes and Law
network, 2pm-5pm, Wednesday 7 December, 2011, Queen Mary, University of London.

Almost 40 mathematicians, computer scientists, legal
scholars and practitioners, forensic scientists and civil servants attended the
meeting at Queen Mary, University of London chaired by Professor Norman Fenton. Hosted by the School of Law's Criminal Justice Centre, the meeting was welcomed by Professor Fenton and Amber Marks of QMUL and addressed by the Forensic Science Regulator,
Andrew Rennison, Strategic Policy Advisor to the Crown Prosecution Service,
Karen Squibb-William, and barrister Richard Thomas (who was Counsel in the R v
T case).

The second half of the meeting consisted of a discussion of
the network’s research goals and issues arising from the speakers’ talks.
Research ideas put forward included creating databases of different types of
forensic evidence to provide a statistical basis for Bayesian calculations, a
comparative law survey of the rules on how statistical evidence is analysed and
presented in other countries and jurisdictions, and empirical experiments in
which different experts would be asked to analyse the same case evidence, first
on their own and then together, to compare and contrast their results.

Karen Squibb Williams answered many questions on the
practicalities of different research ideas and where Bayesian reasoning might
be most successfully deployed in the legal system. She saw great potential for
its use at the case management stage, in which the CPS decides which cases to
bring to court, and urged researchers to target their ideas there. It was
further remarked by several speakers that an inquisitorial rather than
adversarial approach between experts evaluating evidence at this pre-trial
stage would be useful.

Another key theme to come out of both the talks and the open
discussion was the need for transparency by forensic scientists using Bayes to
evaluate evidence. Rather than approach the question of when it is appropriate
to use Bayesian reasoning by defining which types of forensic evidence were
suitable for Bayesian analysis, the general consensus was that any approach to
evidence evaluation could be used, as long as it was done transparently and
explained clearly in court. There was some debate over the usefulness of verbal
scales for presenting expert opinions in court, with some people finding them useful
and others thinking they were less informative than a numerical scale and had
too great a potential to be misinterpreted by the jury.

Discussing what could be achieved over the next 6 months, it
was suggested that creating a couple of examples of how Bayesian analysis
worked in practice, by retrospectively analysing the evidence in a couple of
appropriate recent trials would be helpful. Work on how best to present
Bayesian arguments to lay people was also put forward. Lastly, Norman Fenton
said that his recent experience as an expert witness led him to believe that in
practice there was a widespread perception among trial lawyers that R v T meant
it was too risky to admit Bayesian evidence in court. He therefore suggested
that the network draft and publish a public statement expressing its consensus
that this was incorrect.