History

Scores of researchers have contributed to scientific breakthroughs at Haskins Laboratories since its founding. All of them are indebted to the pioneering work and leadership of Caryl Parker Haskins[2], Franklin S. Cooper[3], Alvin Liberman[4], Seymour Hutner [5] and Luigi Provasoli [6]. This history focuses on the research program of the main division of Haskins Laboratories that, since the 1940s, has been most well known for its work in the areas of speech, language and reading. [1]

1940s

The U. S. Office of Scientific Research and Development, under Vannevar Bush asked Haskins Laboratories to evaluate and develop technologies for assisting blinded World War II veterans. Experimental psychologist Alvin Liberman joined the Laboratories to assist in developing a "sound alphabet" to represent the letters in a text for use in a reading machine for the blind. Luigi Provasoli joined the Laboratories to set up a research program in marine biology. The program in marine biology moved to Yale University in 1970 and disbanded with Provasoli's retirement in 1978.

1950s

Franklin S. Cooper invented the pattern playback[8][9], a machine that converts pictures of the acoustic patterns of speech back into sound. With this device, Alvin Liberman, Cooper, and Pierre Delattre [10] (later joined by Katherine Safford Harris[11], Leigh Lisker and others), discovered the acoustic cues for the perception of phonetic segments (consonants and vowels). Liberman and colleagues proposed a "motor theory" [12] of speech perception to resolve the acoustic complexity: they hypothesized that we perceive speech by tapping into a biological specialization, a speech module, that contains knowledge of the acoustic consequences of articulation. Liberman, aided by Frances Ingemann[13] and others, organized the results of the work on speech cues into a groundbreaking set of rules for speech synthesis by the Pattern Playback[14].

1960s

Franklin S. Cooper and Katherine Safford Harris, working with Peter MacNeilage [15], were the first researchers in the U.S. to use electromyographic techniques, pioneered at the University of Tokyo, to study the neuromuscular organization of speech. Leigh Lisker and Arthur Abramson[16] looked for simplification at the level of articulatory action in the voicing of certain contrasting consonants. They showed that many acoustic properties of voicing contrasts arise from variations in voice onset time, the relative phasing of the onset of vocal cord vibration and the end of a consonant. Their work has been widely replicated and elaborated, here and abroad, over the following decades. Donald Shankweiler[17] and Michael Studdert-Kennedy[18] used a dichotic listening technique (presenting different nonsense syllables simultaneously to opposite ears) to demonstrate the dissociation of phonetic (speech) and auditory (nonspeech) perception by finding that phonetic structure devoid of meaning is an integral part of language, typically processed in the left cerebral hemisphere. Liberman, Cooper, Shankweiler, and Studdert-Kennedy summarized and interpreted fifteen years of research in "Perception of the Speech Code," still among the most cited papers in the speech literature. It set the agenda for many years of research at Haskins and elsewhere by describing speech as a code in which speakers overlap (or coarticulate) segments to form syllables. Researchers at Haskins connected their first computer to a speech synthesizer designed by the Laboratories' engineers. Ignatius Mattingly[19], with British collaborators, John N. Holmes [20] and J.N. Shearme [21], adapted the Pattern playback rules to write the first computer program for synthesizing continuous speech from a phonetically spelled input. A further step toward a reading machine for the blind combined Mattingly's program with an automatic look-up procedure for converting alphabetic text into strings of phonetic symbols.

1970s

In 1970 Haskins Laboratories moved to New Haven, Connecticut and entered into affiliation agreements with Yale University and the University of Connecticut. Isabelle Liberman, Donald Shankweiler, and Alvin Liberman teamed up with Ignatius Mattingly to study the relationship between speech perception and reading, a topic implicit in the Laboratories' research program since its inception. They developed the concept of phonemic awareness, the knowledge that would-be readers must have of the phonemic structure of their language in order to be able to read. Under the broad rubric of the "alphabetic principle," this is the core of the Laboratories' present program of reading pedagogy. Patrick Nye [22] joined the Laboratories to lead a team working on the reading machine for the blind. The project culminated when the addition of an optical character recognizer allowed investigators to assemble the first automatic text-to-speech reading machine. By the end of the decade this technology had advanced to the point where commercial concerns assumed the task of designing and manufacturing reading machines for the blind[23].

Building on earlier work, Philip Rubin developed the sinewave synthesis program, which was then used by Robert Remez, Rubin, and colleagues to show that listeners can perceive continuous speech without traditional speech cues from a pattern of sinewaves that track the changing resonances of the vocal tract. This paved the way for a view of speech as a dynamic pattern of trajectories through articulatory-acoustic space. Philip Rubin and colleagues developed Paul Mermelstein's anatomically simplied vocal tract model [25], originally worked on at Bell Laboratories, into the first articulatory synthesizer [26] that can be controlled in a physically meaningful way and used for interactive experiments.

1980s

Studies of different writing systems supported the controversial hypothesis that all reading necessarily activates the phonological form of a word before, or at the same time, as its meaning. Work included experiments by George Lukatela [27], Michael Turvey[28], Leonard Katz [29], Ram Frost [30], Laurie Feldman [31] and Shlomo Bentin [32], in a variety of languages. Various researchers developed compatible theoretical accounts of speech production[3], speech perception and phonological knowledge. Carol Fowler[33] proposed a direct realism theory of speech perception: listeners perceive gestures not by means of a specialized decoder, as in the motor theory, but because information in the acoustic signal specifies the gestures that form it. J. A. Scott Kelso and colleagues demonstrated functional synergies in speech gestures experimentally. Elliot Saltzman [34] developed a dynamical systems theory of synergetic action and implemented the theory as a working model of speech production. LinguistsCatherine Browman[35] and Louis Goldstein[36] developed the theory of articulatory phonology[37], in which gestures are the basic units of both phonetic action and phonological knowledge. Articulatory phonology, the task dynamic model, and the articulatory synthesis model are combined into a gestural computational model of speech production. [38] Saltzman and Rubin started the IS group to explore cutting edge developments in science and technology and foster collaboration across institutions and disciplines. The group, not formally affiliated with Haskins Laboratories, continues to meet.

2000s

Anne Fowler[51] and Susan Brady[52] launched the Early Reading Success (ERS) program [53], part of the Haskins Literacy Initiative[54] which promotes the science of teaching reading. The ERS program was a demonstration project examining the efficacy of professional development in reading instruction for teachers of children in kindergarten through second grade. The Mastering Reading Instruction program [55], which combines professional development with Haskins-trained mentors, was a continuation of ERS. David Ostry[56] and colleagues explored the neurological underpinning of motor control using a robot arm to influence jaw movement. Douglas Whalen and Khalil Iskarous [57] pioneered the pairing of ultrasound, used here to monitor articulators that cannot be seen, and Optotrak[58], an opto-electronic position-tracking device, used here to monitor visible articulators. Donald Shankweiler[59] and David Braze [60] developed an eye movement laboratory that combines eye tracking data with brain activity measures for investigating reading processes in normal and disabled readers. In March 2005 Haskins Laboratories moved to a new state-of-the-art facility on George Street in New Haven.