Reliability and Validity of Real-Time Response Measurement: A Comparison of Two Studies of a Televised Debate in Germany.

Abstract:

This paper investigates the reliability and validity of real-time response measurements (RTR). It is based on a comparison of two quasi-experimental studies independently conducted on the second televised debate of the two major candidates for chancellor in the 2002 German federal election campaign. Participants in Bamberg and Mainz--two mid-sized German cities--followed the debate on a large-scale screen. The viewers’ immediate reactions to the candidates were measured in real-time. In terms of technicalities and substance, both quasi-experiments differed in several important respects. For example, the system used in Mainz was based on a control unit with a 7-point scale and yielded one merged metric dimension for both candidates. The Bamberg system measured positive and negative impressions of the candidates independently, yielding categorical data. Despite these operational differences, the results show that both methods render reliable results that also meet the criteria of face, construct, and criterion validity. Hence, RTR measurements provide valuable, unique insights into subjective immediate reactions to candidates in televised debates and help to explain post-debate perceptions and attitudes.