Statistics

Statistics

More posts

An epidemiology-of-representations solution to a WWII shipwreck mystery

The Australian Cruiser HAMS Sidney

After a shameful lull in the activities of the ICCI (Sorry, folks!), we need something sensational – something, say, like Urbain Le Verrier’s famous conjecture that there had to be a yet unknown planet and his calculation of the location of Neptune that led to its actual sighting in 1846. Well, my story is not quite as sensational but I hope it will kick start a return to ICCI full speed. It involves two psychologists, John Dunn and Kim Kirsner, using cognitive and mathematical analyses of old testimonies to locate a German and an Australian warship that, in 1941, had been engaged in a firefight somewhere off the west coast of Australia and had both sunk. While none of the 645 men onboard the Australian HMAS Sydney survived, 317 sailors from the German cruiser Kormoran did, were picked up by the Australian navy, and interrogated. About 70 of them gave some indications of the location of the event. The locations they indicated however were spread out over hundreds of miles. Even assuming that the prisoners were not trying to deceive their captors, their testimonies seemed impossible to exploit.

“We approached the analysis of the survivor reports as a problem of how information is stored and transmitted by people and how it might, through that process, become distorted or degraded. Because of the technical nature of location information, it is likely that the original source material was known to relatively few people such as the captain, the navigator and the signals operator, and that this material, or versions of it, was communicated to other survivors either directly and indirectly through a network of individuals over the period of time leading up to individual interrogations. We proposed therefore that the set of survivor reports may be viewed as variants of a relatively small number of original location statements that we called source statements. Our analysis of the reports consisted of three stages. First, given the controversial nature of the survivor reports, we needed to show that the observed variation could be attributed to the distortion of relatively few source statements due to inter-individual communication and intraindividual memory. Second, we attempted to identify these source statements and, third, to use them to deﬁne a feasible search area for Kormoran.”

Their work is explicitly based on the transmission chains first studied by Bartlett (1932) and recently in the study of cultural evolution (Mesoudi & Whiten, 2008), the evolution of language (Smith & Kirby, 2008), iterated learning (Kalish, Grifﬁths, & Lewandowsky, 2007), and reconstruction from memory (Hemmer & Steyvers, 2009). It contains statistical comparisons between Bartlett’s experimental evidence – in particular regarding his best known example, the transmission of the “War of the Ghosts” legend – and the testimonies provided by the German prisonners, and many subtle and surprising methodological points that cannot be easily summarized.

I will quote extensively from Dunn and Kirsner’s conclusion:

“Our involvement in the search for Sydney revealed to us that the domain knowledge and methods of cognitive psychology offered a unique perspective on a problem that was not shared by other individuals and groups whose expertise lay in very different ﬁelds such as oceanography, engineering, and oral and military history. […] our approach is unique in drawing upon a perspective offered by cognitive psychology informed by the following three aspects:

(1) The capacity to account for variability in human data in a systematic manner. It is noteworthy that other discipline-based approaches were unable to offer a systematic account of human variability. While oceanographic analyses that pre-dated our involvement dealt with variability in physical quantities such as wind speed and direction, none of this domain knowledge could be applied to variability in survivor reports. Similarly, historical reconstruction was also unable to address this issue and resorted either to selecting one or two statements as veridical and ignoring the remainder or to dismissing the entire corpus. The unique perspective offered by cognitive psychology allowed us to generate a principled account of this variability that supported the view that the entire corpus could contain much previously neglected information.

(2) A model of information integration as a basis for decision making. Cognitive psychology offers many different models of information integration and decision making. We employed a linear integration model that is particularly simple and robust and has been shown to yield good results in a variety of contexts (Dawes, 1979). This model, ultimately based on Brunswik’s Lens Model (Brunswik, 1957), allowed us to be reasonably conﬁdent that an unweighted average of error distances would provide a satisfactory if not optimal prediction of location.

(3) Commitment to a quantitative approach. Cognitive psychology often relies on quantitative models of unseen mental processes. We applied this principle in developing a model of the relational structure of the survivor reports and in evaluating this model against the statistical properties of the data. It also informed our aim of integrating all of the available information quantitatively in the form of a series of constraints that provided a single ‘goodness of ﬁt’ measure which we used to evaluate each candidate location. Our analysis revealed a high level of internal consistency among a set of reports that had previously been regarded as being too diverse to be of value.”

In 2004 Dunn and Kirsner indicated the location of the Kormoran, with, as it turned out when the boat was eventually found in 2008, an approximation of less than 5 km. Not quite Le Verrier’s discovery of the existence and location of Neptune, but a highly original demonstration of the fact that the flow of information through social networks can be studied with remarkable precision and insight provided that the memory and communication processes involved are seriously discussed rather than merely postulated. Note that this work would not have been possible if the distortions that occur in the transmission of information were similar to 'random mutation'. This would make it impossible to reconstruct the chains of testimonies and, in this case, to properly identify the source statements. (For the relevance of this to the epidemiology of representations, see this paper).

PS. For a non-technical presentation of Dunn and Kirsner’s work, read this excellent NPR report.

2 Comments

That’s a pretty cool paper, Dan, thanks for the link. It’s good to see quantitative methods in psychology being applied in such an Indiana Jonesey setting, but it’s not clear from the paper how actually useful they were. From what I can tell, it could well be that the most important bit was the sleuth-work they did in figuring out what the source statements were, and the quantitative methods only came in as support. A delight to read nonetheless.

Lovely. I want to see the movie (may I suggest Neal Stephenson as screenwriter?). That said, making this a triumph for cognitive and quantitative approaches to cultural transmission might be imprudent.
Dan (and, often, the authors) are claiming that, because information gets importantly distorted in a non-random way when passed along diffusion chains, we need statistical heavy-lifting to retrieve it. In the case at hand, both claims seem quite wrong to me.

The majority of survivors (31/72; those 31 are by far the absolute majority of those who gave coordinates) agreed on the location of the ship, with a degree of approximation that is as good as Dunn and Kirsner’s estimations (before they took physical evidence into account). In spite of the dramatic circumstances, the passing of time and the opportunity of deception, most of the german sailors who overheard Linke or Detmers repeated what they heard with perfect accuracy. A similar result could had been reached by assuming that most people committed what they heard to a faithful and unbiased memory, and applying a simple majority rule (and then taking into account the crucial physical evidence that really explains the breakthrough, but has nothing to do with cognition and transmission). The sophisticated cladistic analysis played no role in identifying the ship (by the authors’ own admission), and many other analyses look like post-hoc rationalisation of an empirical hunch. Nothing wrong here, but as Simon remarks, careful historical detection seems to do most of the work later attributed to cognitive and quantitative tools.

Update (2015): On reading a book chapter written by the researchers (https://books.google.hu/books?id=gGqIBAAAQBAJ&dq=26S+111E&source=gbs_navlinks_s), I feel I have to revise my judgment on this wonderful study. One needs to read that chapter to understand how terribly difficult the search was. Everything seems easy in retrospect. For instance, applying a (relative) majority rule to the testimonies would have been anything but trivial, given the degree of controversy and suspicion that surrounded the whole affair. Also, it is a shame that the two discoverers did not get the credit they deserved for this fantastic work.