4 Principal Author Original signed by Peter Suedfeld, Ph.D Peter Suedfeld, Ph.D Department of Psychology, University of British Columbia Approved by Original signed by David R. Mandel, Ph.D David R. Mandel, Ph.D Group Leader, Thinking, Risk, and Intelligence Group Approved for release by Original signed by Joseph V. Baranski, Ph.D Joseph V. Baranski, Ph.D Chair, Document Review and Library Committee Her Majesty the Queen in Right of Canada, as represented by the Minister of National Defence, 2010 Sa Majesté la Reine (en droit du Canada), telle que représentée par le ministre de la Défense nationale, 2010

5 Abstract.. Measures of the cognitive complexity of leaders have been used to infer the flexibility, openendedness, and information-orientation of their decision-making in international and non-state confrontations. At present, there are two major methods of assessment at a distance used in this context. One uses computer scoring to develop personality profiles of leaders; the other uses a more labour- and time-intensive human scoring system to track changes in the target s thinking to predict the outcome of a particular confrontation. If computer scoring were able to make eventspecific predictions, the saving in time and work would be substantial. This study compared the two systems to establish (a) whether the computer-scored system could replace human scoring, and (b) using the example of the South Ossetia War between Georgia and Russia, which method was a better predictor of rising and falling tension, The data confirmed the relevance of integrative complexity measurement in a new context, that of an ongoing confrontation with changing levels of tension, up to and including war, between a major and a minor national power. The correlation between scores from the two methods was low; at high levels of cognitive complexity, it was essentially zero. The human scoring of integrative complexity, which tracks changes in complexity over the duration of a particular event, was closely tied to the course of the confrontation; the computer scoring of cognitive complexity, which profiles complexity as a stable personality characteristic, was not. Thus, although computer scoring has significant advantages in cost and time, it does not accomplish the same goals. This finding has important implications for security/intelligence applications, as the computer scoring approach is an attractive way to process large amounts of information. However, our data indicate that the negative trade-off between speed and accuracy is serious enough to opt for the more laborious human tasking if the goal is the prediction of crisis outcome. Résumé... Nous avons eu recours à des mesures de la complexité cognitive chez les leaders pour prévoir le caractère conciliant, ouvert et axé sur l information du processus de prise de décision dans les situations d affrontement internationales et non étatiques. À l heure actuelle, on utilise principalement deux méthodes «d évaluation à distance» dans ce contexte. La première établit un profil de la personnalité des chefs en ayant recours à la cotation informatisée; la deuxième fait appel à une méthode de cotation par des humains plus exigeante en main-d œuvre et en temps pour effectuer un suivi des changements dans le raisonnement du sujet cible afin de prévoir l issue d un affrontement en particulier. Si la cotation informatisée permettait de faire des prévisions relativement à un événement précis, l économie de temps et de travail ainsi réalisée serait considérable. Dans le cadre de la présente étude, nous avons comparé les deux systèmes dans le but de a) déterminer si le système de cotation informatisé pouvait remplacer le système de cotation par des humains; b) déterminer quelle méthode permettait le mieux de prévoir l exacerbation et le relâchement de la tension, à l aide de l exemple du conflit Géorgie-Russie en Ossétie du Sud. Les données ont confirmé la pertinence de la mesure de la complexité intégrative dans un contexte nouveau, celui d un affrontement en cours entre une puissance nationale majeure et une puissance nationale mineure, affrontement caractérisé par un niveau de tension DRDC Toronto CR i

8 Executive summary The Complexity Construct in Political Psychology:: Personological and Cognitive Approaches Peter Suedfeld; DRDC Toronto CR ; Defence R&D Canada Toronto; February Introduction or background: The analysis of the cognitive processes of leaders has been a useful tool in the prediction of crisis outcome in international and non-state confrontations. Because leaders of nations or violent non-state groups are rarely if ever available for first-hand psychological studies, such forecasts are typically made at a distance. Quantitative methods for these assessments can be categorized as either profiling, whose goal is to generate a multidimensional picture of the individual s personality that in turn can be used to predict his or her behavioural tendencies in general, or as tracking, a method that uses repeated measures of changes in the state of a particular variable as a specific situation unfolds to predict future decisions related to that situation. Cognitive complexity, a measure of flexible, open-ended, information-oriented thinking and decision-making, has been used in both profiling and tracking modes to predict international crisis outcomes. Both approaches use content analysis of spoken and verbal outputs of relevant leaders as indices of their level of complexity. The question of whether one mode is a more reliable predictor than the other is an important one in the assessment of adversarial intent; it is made even more salient by the fact that the most widely used profiling system, Margaret G. Hermann s Profiler Plus software measuring conceptual complexity (CC), is a much less labourintensive method than the most widely used tracking system, Peter Suedfeld s manual for handscoring situation-related changes in integrative complexity (IC). If the two methods yield similar results, abandoning the hand-scoring system would facilitate research in this area. Results: The current study, carried out by Suedfeld s research group with the cooperation of Margaret Hermann, correlated scores derived by the two methods. Texts originally scored by the IC system were re-scored using Profiler Plus, and vice versa. A low but reliable correlation was found at low levels of complexity. At medium and high levels, the relationship eroded, eventually to zero. Analysis of a third sample, tracking the statements of Georgian President Saakashvili before and during the 2008 Georgia-Russia War over Abkhazia and South Ossetia, found that IC scoring showed a close relationship to rising and falling tensions between the two states; CC scoring showed no such relationship. Significance In view of the vast amount of information that must be rapidly accessed, analyzed, and understood in developing intelligence forecasts, it would be highly desirable if computerized CC scoring could replace the IC scoring method. However, the data obtained in this study show that only IC scoring closely monitors the level of tension and therefore usefully assesses the probability of violence during a specific international crisis. However attractive rapid computer scoring may appear, the trade-off in predictive power would make such a substitution inadvisable. Future plans: The next Subproject, now being completed, applies IC scoring and other methods of thematic content analysis (TCA) to the prediction of conflict course and outcome to four iv DRDC Toronto CR

9 confrontational situations: The Russian-Georgian War (including the scoring of statements by Russian President Medvedev). Domestic political violence in Zimbabwe, terrorist attacks by al Qa ida, and the international confrontation concerning Iran s nuclear program (scoring the leaders of Iran, the USA, and Israel). DRDC Toronto CR v

17 1 Introduction The assessment of leader personalities and decision patterns has been an important component of political psychology even before the field acquired a label and identity of its own. In the very first Handbook of Political Psychology (Knutson, 1972), at least three of the 16 chapters -- by Knutson, Glad, and Katz -- were to a great extent focused on reviews of publications linking such characteristics with the political ideology and behaviour of a diverse list of individuals and groups. Assessment at a distance became one of the hallmark activities of the field, several of whose major contributors combined innovative methodological and theoretical ideas with practical applications for diplomacy and negotiating strategy. 1.1 Cognitive Style as a Personality Variable Among the best-established and most frequently used variables in the study of the personalitypolitics link have been authoritarianism (Adorno et al., 1950) and its variants, such as dogmatism, rigidity (Rokeach, 1960), and right-wing authoritarianism (RWA; Altemeyer, 1981). The original version, whose stated goal was to identify people who would be susceptible to Fascist ideology, was explicitly content-oriented, assuming that for the most part (although not universally) high levels of authoritarianism would be found on the right wing of the political spectrum. The trait itself was theoretically the outgrowth of child-rearing and societal ideologies, and was manifested in respect for parents and traditional authorities, unwillingness to change or even to examine one s ideas, ethnocentrism, and derogation of outgroups whether ideological or racial. The authoritarian personality was conceived as a Gestalt incorporating cognitive, emotional, and unconscious components. Later approaches, however, concentrated more on stable ways of thinking, labelled cognitive styles. Among the earliest, the concepts of dogmatism and rigidity (Rokeach, 1960) presented a reaction to the political bias of the authoritarianism work, proposing a theory and scales that dealt with closed-mindedness and inflexible thinking regardless of any specific political position. RWA, despite its label, is also content-free, at least in conception: it is defined by acceptance of the conventional mores of one s society, obedience to its leaders, and punitiveness toward those who dissent from either. Obviously, such a pattern can exist in both conservative and liberal or radical political systems. 1.2 Cognitive Complexity Another construct, cognitive complexity, has similarities and overlaps with some aspects of authoritarianism, but even more so with dogmatism, rigidity, and a number of later cognitive style formulations such as need for cognition (Cacioppo & Petty, 1982), orientation toward uncertainty (Sorrentino et al., 1992), and need for closure (Kruglanski & Webster, 1996). Although cognitive complexity has been defined and measured in a number of ways by different theorists and researchers (see, e.g., Goldstein & Blackman, 1978; Schroder & Suedfeld, 1971), some common threads run through all of these versions. In general, high complexity is thought to be marked by openness to and searching for new information; the ability to modify or even abandon problemsolving strategies or decisions if the new information indicates that they are inappropriate; DRDC Toronto CR

18 tolerance for high information load, uncertainty, and lack of closure; the integration of decisions with each other both temporally and across domains; the perception of nuances and fine distinctions; and a willingness to consider alternative and opposing viewpoints. As the above list implies, cognitive complexity in any version is a content-free variable; it focuses not on what people think, but on how they think. Thus, any position on any topic can be held at any level of complexity. This is not to deny that in practice there are fairly reliable correlations between some ideological positions and complexity: in general, convictions occupying extremes on an attitudinal dimension tend to be held more dogmatically than those nearer the center, and therefore to be associated with lower cognitive complexity. But the relationship is with how the views are held, not with their content; highly complex arguments can underlie some people s extremist views in politics, religion, economics, science, etc. and, just as easily, simple ones can support centrist views. Among the best-known cognitive style approaches is the conceptual complexity theory of Schroder, Driver, and Streufert (1967). That theory posits that conceptual complexity is a stable personality trait that governs how people process information, draw conclusions, and make decisions. Actual information processing and decision making may vary in complexity in response to environmental factors such as information under- or overload, but trait complexity is like trait intelligence: it imposes a limit on the extent to which any individual can process information at a complex level. Complexity itself is defined and measured in terms of two components: differentiation, the ability to perceive alternative dimensions in a stimulus object or alternative legitimate attitudes toward it, and integration, the ability to perceive relationships among the differentiated dimensions or attitudes. Integration, therefore, cannot occur in the absence of differentiation. Schroder et al. describe several ways in which conceptual complexity can be measured, but the one most frequently used by them and others is the Paragraph Completion Test (PCT). The PCT is a semi-projective technique in which the respondent completes a topic sentence stem and continues to write on the topic, up to a time limit, which is the signal to do the same with the next stem. The specific stems can vary (usually according to the age, literacy, etc. of the subject sample or the context of the study); but generally there are two stems tapping each of three important psychosocial dimensions: orientations to uncertainty, attitudes toward authority, and reactions to interpersonal rejection. Each completion is scored on a seven-point scale; the six scores are then averaged to obtain the person s total score. The seven levels of scoring are shown in Table 1. Table 1: Seven Levels of Integrative Complexity Integrative Complexity Score Score of 1 Explanation of Score There is no sign of either conceptual differentiation or Prototypical Example of Score Enforced retirement at 65 years is most certainly beneficial to the workings of our 2 DRDC Toronto CR

19 integration at this scoring level. The author relies, without qualification, on a simple, one-dimensional rule for interpreting events or making choices. society. We must realize that work is the job of the young, and that the elderly should leave their work positions in order to make room for the next generation. The elderly must resign themselves to let go of their family domination and employment in order that the new generation may explore its ideas and promote growth. Some would argue that the elderly have much to contribute because they have lived through so much, but this is clearly wrong. With the rapid rate of technological change, the elderly are the least able to adapt to the new conditions. Score of 2 The critical indicator for a score of 2 is the potential or conditional acceptance of different perspectives or dimensions. The author does not explicitly develop the alternate dimension or perspective; nor is it necessary that it be explicitly stated or named. Simple qualification, without elaboration, is sufficient evidence for a score of 2. Enforced retirement at 65 is probably beneficial to our society. To be sure, it is difficult to force the resignation of long-time employees in order to make room for the upcoming generation. We should, however, give highest priority to creating opportunities for the new generation to explore their ideas. This need may be especially critical in view of the requirements of running a modern economy. More than ever, our country appears to require new ideas and fresh perspectives. Score of 3 The critical indicator for a score of 3 is the recognition of alternative perspectives or different dimensions, and the acceptance of these as being relevant, legitimate, justifiable, valid, etc. Enforced retirement has recently engaged the interest of the popular press. Some think that the elderly should give up their jobs in order to make room for the younger generation. Such an action would bring fresh vision to business and give our society the new ideas we clearly need. On the other hand, there are those who think that the experience of the elderly is potentially our society s deepest resource, and that their hardwon knowledge can help business to adapt to these changing times. Of course, we rarely see either vision or wisdom in its pure form; each school of thought tends to ignore a bit of the other s perspective. Score of 4 The author must indicate that multiple perspectives or dimensions exist, and also For society fully to benefit from its members skills and to survive the technological revolution, a new philosophy must develop DRDC Toronto CR

20 that they could interact. which unites the youth and the elderly of the nation. The technological skills of the young are important if we are to keep pace with the other industrialized countries. However, we have important social decision-making ahead of us if we are to improve our quality of life in the long term. Our future success depends upon our ability to realize the potential of our citizens, whether they be young or old. Score of 5 The critical indicator of a score of 5 is that alternative perspectives or dimensions are not only held in focus simultaneously but also are viewed interactively. The author is able to see that multiple alternatives are all to some degree legitimate, and combines them to produce a result that none of the alternatives could have produced alone. For society to benefit fully from its members skills and to survive the technological revolution, a new philosophy must develop which unites the youth and elderly of the nation. The technological skills of the young are important if we are to keep pace with the other industrialized countries. However, we have important social decision-making ahead of us if we are to improve our quality of life in the long term. Thus, we must be concerned equally with developing technology as with developing a plan for its use. It is in this latter area that the elderly, with first-hand knowledge of history, must be consulted. With this philosophy in mind we are faced with the economic challenge of employing both young and senior citizens. Score of 6 For a paragraph to be given a score of 6, the author must be working across several levels of schemata and at least one of the indicators noted above must be explicitly delineated. Thus, there may be an explicitly presented global overview with only an implicit indication of the specific dynamics of the alternatives. Conversely, there may be explicitly stated details about the dynamic interaction between alternatives and only an implicit communication of the global overview. Their experiences with war and depression during the thirties created in many members of our parents generation a drive to create some form of security for the future that was not available for them to enjoy in earlier years. By continuously building upon their gradually increasing assets while still maintaining the conservative lifestyles they had been pressed to follow during hard times, they created economic stability for themselves. This economic stability, enjoyed by many approaching old age, lends greater power to seniors increasingly vocal demands for an improved quality of life for the elderly. Their offspring, not having faced the same hardships as their parents, have had opportunity and cause to be somewhat reflective about issues pertaining to the quality of life in general, including the plight of the elderly. 4 DRDC Toronto CR

21 Score of 7 1. An overarching viewpoint is presented, which contains an explanation of the organizing principles (e.g., temporal, causal, theoretical) of the problem or concept. 2. There is a discussion of the ways in which levels of the problem or concept interact and thus demonstrate the validity of the overarching perspective. The description of the ways in which levels of the system interact must be both specific and dynamic, demonstrating how each level is affected by the other. While these indicators are distinct, they are inextricably linked. The global overview encompasses the components of a system, and in fact may have developed as a result of the author s simultaneous consideration of these levels or components. We must view this problem from the very broad perspective which involves the kind of society we see ourselves to be and the kind of society we strive to be. Do I assume correctly that we are, and that we want to be, civilized? It may be that our heavy emphasis on individualism, productivity and self-sufficiency has resulted in a view of the elderly as basically useless members of society. On a larger scale, should this opinion prevail, we could hardly have a view of ourselves as belonging to an integrated and harmonious community. At a more specific level of intergroup relations, if this attitude flourishes, the older segment of the community may feel forced to exert their increasing economic and social power to the detriment of others in society, such as younger individuals in need of their employment positions. This would only contribute to an adversarial relationship (albeit reversed), where resources are still not distributed with social and economic equity among a heterogeneous population. We must all take our fair share of responsibility for the state we live in. Our treatment of all individuals, regardless of age, depicts the state of our society; a civilized community treats all its members in a civilized manner. These examples are drawn from Baker-Brown et al. (1992). DRDC Toronto CR

22 2 The Use of Complexity in Political Psychology Although the PCT has been used in experimental research on political attitudes and decisionmaking in politically relevant scenarios, it has a serious drawback. As is the case with projective as well as paper-and-pencil instruments as a class, it requires the research subject to be willing to respond to the tasks or questions in the test. Even with current mechanisms such as web-based administration, which free researcher and subject from having to arrange face-to-face meetings, the need for time out from the subject s normal daily activities in order to participate remains. This requirement, unfortunately, leads to some potential subjects of interest, such as historical figures and contemporary high-level political leaders, being inaccessible; perhaps even worse, it severely diminishes the external or ecological validity of the study. People who agree to participate in research that they know has the goal of uncovering their attitudes, beliefs, ideology, thinking processes, rationale for particular decisions, and so on, may give answers contaminated by motives such as self-enhancement, conformity, desire to please the researcher, or lack of selfinsight. The study of political leaders at a distance overcomes these handicaps by using as its databases information available in archives, and materials produced by the study subjects in the course of their ordinary activities. With such methods, neither the agreement of the subject to participate nor subject expectancy/bias is an issue. Not surprisingly, researchers interested in cognitive styles and other aspects of information processing have addressed themselves to developing ways in which such variables can be studied at a distance and in a reliable manner. In applications to political psychology, the study of cognitive complexity has branched in two major directions. Both stem from the same tree, the theory and measurement technique of Schroder et al. (1967). Perhaps ironically, the more direct descendant has moved to a more divergent role than the less direct one. The two variants have different names, emphases, and scoring systems. One, integrative complexity, belongs to a family of tracking procedures; i.e., repeated measures are taken during a particular time period to monitor changes in the subjects level of complexity and forecast decisions in the context of problems occurring during that period. The other, conceptual complexity, belongs to a family of profiling procedures, which aim at producing a personality profile that allows researchers and policy makers to understand and predict the subject s ideas and predispositions across situations. Despite these differences, because both originate from the same conceptual starting point, prominently use the term complexity for their core variable, and seek to explain and predict leader behaviour, the question of how much they overlap is both important and inescapable. 2.1 Integrative Complexity Integrative complexity (IC; e.g., Suedfeld, Tetlock, & Streufert, 1992) was developed by collaborators of the Schroder et al. (1967) group and their students. The perspective is that complexity is a cognitive process. This approach differs from Schroder et al. s personological version in the following ways: 1. Theoretical Emphasis. IC research, while acknowledging that there is a stable level of trait complexity as posited by Schroder et al., focuses on state complexity: the levels of 6 DRDC Toronto CR

23 differentiation and integration that characterize the subject s information processing in a particular situation. Schroder and his students (Driver, 1962; Streufert & Swezey, 1986) demonstrated that information load and feedback valence (success or failure) affect the complexity of decisions aside from the underlying level of conceptual complexity as measured by the PCT; Suedfeld and his students (e.g., Suedfeld, Leighton, & Conway, 2005) have concentrated on how a wide range of both external and internal conditions (e.g., stress, fatigue, time pressure, issue domain, accountability), interacting with trait complexity, determines the level of functional complexity at any given time. Complex thinking is thought to be a process that takes up resources such as time, energy, and attention. The cognitive manager model (Suedfeld, 1992) hypothesizes that good decision makers conserve such resources by engaging any problem with only as much complexity as is merited by its importance and difficulty, and by the total resources available. Both insufficient and excessive complexity may lead to poorer decision outcomes and negative implications for other decisions. Thus, unlike most cognitive complexity theories, IC theory does not value highly complex thinking above a simpler alternative; which is preferable depends on the situation. 2. Methodology. IC research uses the same 7-point scale as PCT scoring. As with the PCT, thematic content analysis (TCA) is used: that is, scoring depends on the scorer s assessment of the level of differentiation and integration in each passage. The appearance of particular words or phrases ( content flags ) may alert the scorer to the likelihood of a particular score, particularly at the lowest levels, but can never be used as full justification of that score. Databases of material used in scoring IC are archival or concurrent materials produced by individuals of interest. These materials can be written, oral, or electronically recorded. As with the PCT, almost any connected verbal passages can be scored. Paragraphs to be scored are randomly identified from the whole available data base (or all available passages are included); as far as possible, identifying information regarding the source, issue, and time of production is removed and the paragraphs scrambled in random order; and the material is then scored by qualified scorers using a standard scoring manual. Qualification as a scorer requires passing a standard training course in a 3-day workshop or on line, with a reliability of at least kappa = 0.85 with expert scorers on a test set of paragraphs. Research scorers were not involved in the preparation of the material for scoring. Between 10% and 20% of the paragraphs are then re-scored independently by another qualified scorer. Qualified scorers can usually score 2-3 paragraphs per minute. These procedures are relatively elaborate and time-consuming, but they minimize the danger of selective attention to content and scorer bias, and make it possible to measure interscorer reliability and to use standard significance tests and power calculations to analyze the data. Regrettably, several attempts by experts to develop computerized IC scoring systems have failed. 3. Research Goals. IC research does not attempt or claim to assess leader personality or cognitive style. Rather, its aim is to understand and predict decisions made by leaders in specific situations, such as international confrontations. Repeated sampling of leader speeches and writings during the course of such an event is used to track changes in IC, DRDC Toronto CR

24 which in turn are considered to be indicators of the source s information processing at each data point and to predict the decisions that will emerge. Although research topics and results cover a wide range, one of the most firmly established is the finding that the resolution of international confrontations can be predicted from the changing IC patterns of major national leaders or leadership groups. Levels of IC that remain steady or rise during negotiations precede peaceful compromises at the end of the process; substantial drops in IC are a sign that the crisis is likely to lead to war (for a review, see Conway, Suedfeld, & Tetlock, 2001). The cognitive manager model (Suedfeld, 1992) proposes that IC changes under stress follow a curvilinear pattern: when a problem first arises that cannot be solved in a simple way, IC rises as the individual increases information search and processing to deal with it. Eventually, an asymptote is reached; but if the problem becomes too severe and as the solver exhausts his or her cognitive resources, IC drops ( disruptive stress ) and a simple strategy is sought to escape the continued demands. In international crises, for example, that simple strategy may be to attack the opponent. 2.2 Conceptual Complexity The version of conceptual complexity (CC) that is best known in political psychology (e.g., Hermann, 1999/2002) is a personological construct that uses the same name as the construct of Schroder et al. (1967), but differs from the original in several important ways. 1. Theoretical Emphasis. Hermann s definition of CC is entirely in terms of differentiation. The focus is on specific words that show a perception of different dimensions in the environment. The measurement technique (see below) does not allow for an assessment of integration. CC scores are computed as part of a multidimensional measurement of several trait characteristics, from which a particular leader s stable personality is inferred ( Leadership Trait Analysis, Hermann, 1999/2002). Because personality traits by definition are stable characteristics, CC researchers generally develop one profile for each subject rather than tracking possible changes over time (e.g., Dyson, 2009; Thies, 2009). 2. Measurement. Although the scoring was originally done by hand, as in the case of IC, currently the material taken from archived speeches or writings is analyzed by a software package (Profiler Plus), with a recommended minimum of 5,000 words (in practice, the samples range into the millions). This is a much larger sample size than is customary in IC scoring. Like other content analysis programs, Profiler Plus uses a dictionary developed by the research team. The program counts how often a passage mentions words that according to the dictionary indicate either high or low complexity. High CC is indicated by the more frequent appearance of words such as possibly, and approximately; low CC, by such words as certainly and absolutely. Technically, scores can range from.00 (no word or phrase in the material reflects high complexity) to 1.00 (every word reflects high complexity). Higher scores indicate higher CC. Because all scoring is computerized once the dictionary is prepared and entered into the program, the method is inexpensive and fast. 8 DRDC Toronto CR

25 3. Research Goals. The goal of profiling research is to develop an accurate picture of the personality of the individual being studied. His or her general orientation to life and to the leader role, perceptions of the international system, willingness to negotiate or, alternatively, to use armed force, personal motivations and goals, and similar variables, are measured. In any particular application, the researcher must weigh these various characteristics to infer their relative importance and to predict the decisions that the person is likely to make. In combination with self-confidence, the CC score is considered to be a measure of openness to information; whether it is higher or lower than self-confidence (which Profiler Plus also measures) differentiates pragmatic, democratic leaders from ideologues. 2.3 Goals of the Current Study The current study was designed to investigate whether IC and CC scoring would yield similar results when applied to same body of materials. The key question is whether tracking and profiling methods would be equally accurate in predicting a leader s decisions in a particular situation i.e., whether the assessment of trait complexity, a stable long-duration characteristic -- would suffice for such predictions, or whether state complexity, measuring cognitive processing in a situation-specific, time-limited way would significantly improve forecasting. In addition, an important implication for researchers was at stake. As noted previously, IC scoring is relatively labour-intensive and time-consuming, both in training scorers and in their actual work. The interpretation of some passages can require nuanced analysis and may lead to disagreement between scorers. CC scoring, by contrast, has all of the advantages of computer analysis: rapidity, low time requirements, and clarity. If the scores obtained by the two methods were closely similar, considerable time and work could be saved by substituting computer scoring for the current human scoring of IC. DRDC Toronto CR

26 3 Method and Results Several samples of material were scored by both methods. These included passages previously scored as part of IC studies in Peter Suedfeld s laboratory and others previously scored in the course of CC studies in Margaret Hermann s laboratory. All of the correlations were computed in the former. Passages that had been scored for CC were scored for IC following the standard procedures for such scoring; the agreement between the two qualified IC scorers on this material was above 85%, the level required for reliable scoring. Sample 1. From examples in the IC scoring manual (Baker-Brown et al., 1992) and from the paragraphs used in training scorers, for all of which there are scores assigned by a consensus of experts, 223 passages were extracted. The passages were processed using the Profiler Plus software, and Pearson product-moment correlations were calculated. The overall correlation barely missed statistical significance, r(210) = 0.13, p <.06. The next step was to compare the correlation between the two coding systems separately for extracts coded as low vs. high in complexity according to the IC scoring system. The former category included 135 extracts that had been scored as 1, 2, or 3 (ranging from no differentiation to differentiation with no integration); the latter included 88 extracts coded as 4, 5, 6, or 7 (ranging from implicit evidence of integration to high level integration within a superordinate schema). The correlation between the two scoring systems for low complexity passages was positive and significant, r(125) = 0.30, p <.05, while that for the high complexity extracts was essentially zero, r(85) = 0.01, NS. Sample 2. The second part of the study involved passages that had been previously used in research by the research group of Margaret Hermann. The dataset included 13 speeches (244 paragraphs) by Muqtada al-sadr, the leader of the Sadrist Movement and Mahdi Army, the violent Shi a resistance to the Coalition and newly elected government in Iraq, and 12 speeches (420 paragraphs) by Sayyid Hassan Nazrallah, the leader (Secretary General) of Hezbollah since These passages were then scored by trained scorers using the IC scoring system, following the usual procedures. Reliability between the primary and secondary scorers was 87%. The overall correlation was r = (p < 0.001) between the two scoring methods. In order to get a finer-grain disaggregated analysis than with Sample 1, separate correlations were calculated for passages scored as IC 1 or 2 (no clear differentiation), 3 and 4 (clear differentiation only or clear differentiation with implied integration), and 5-7 (clear integration). The respective Pearson product-moment correlations were r = (p <.02), (NS), and (NS). Sample 3. It may be argued that because of the prevalence of relatively low-complexity materials in many (although by no means all) datasets, the lack of significant correlations at the higher end of IC do not really matter. If that were the case, Profiler Plus might be adequate for tracking the course of an event and forecasting its outcome despite its omission of integration as a component of complexity. To test this possibility, we used data from a recent study (Frisch, 2009) that assessed IC at various points relevant to the onset and outcome of the South Ossetia (Russo-Georgia) War 10 DRDC Toronto CR

27 of The entire set of speeches and interviews of the Georgian president, Mikheil Saakashvili, from April through September, a total of 1,255 paragraphs from 49 documents, was scored. Four nodal stages were identified: Baseline (April 5-19); the Russian downing of a Georgian reconnaissance drone and the growing hostility arising from that event (April 20-May 18); negotiations to try to resolve the issue peacefully, with international mediation (May 19-August 6); and the increasing tension leading to the war itself, beginning with the Russian invasion of South Ossetia and Abkhazia, and ultimately including fighting over previously undisputed sections of Georgia (August 7-September 31). Figure 1 shows the IC and CC score curves during the entire period. IC scoring followed the usual pattern: steep decreases as tension increased, higher IC when negotiations were proceeding, and a steep and statistically significant drop during the period of armed conflict. Normally, we might have expected an increase in IC when the fighting stopped, but in fact tensions remained high as Russian forces continued to occupy both breakaway provinces and the other areas they had invaded. Profiler Plus CC scores showed a significant increase from baseline through negotiations, not reflecting the initial period of tension, and a nonsignificant drop at the final stage. Figure 1: IC and CC Scores for Mikheil Saakashvili DRDC Toronto CR

28 4 Discussion Both data analyses indicate that the across-the-board correlation between IC hand-scoring and CC scoring using Profiler Plus is quite low, reaching significance as a function of the large sample size. That overall correlation level is driven by the agreement between the two systems on identifying the lowest complexity: i.e., passages that do not even reach the level of differentiation. For passages in the differentiated range, the correlation fell below.04, and in the integrated range, it was essentially zero in both samples. The fundamental theory of IC ensures that the measure is not useful for personality profiling; in that context, the low correlations do not matter. The question of whether CC scoring is useful for tracking is also somewhat moot, as in practice CC researchers consider that they are studying a stable personality factor so that repeated measurement within a short time period is unnecessary and useless. The low level of compatibility between the two systems may be somewhat ameliorated by the fact that in many IC studies, the mean score of paragraphs is at the 1-2 level. However, that mean is the product of undifferentiated passages being averaged with scores above that level. Many passages and complete speeches or writings reach the level of clear differentiation, and some go considerably higher. To ignore the levels of clear differentiation to high-level integration would be to miss important aspects of thinking and decision-making. It would also make it impossible, or at least very difficult, to identify the onset and impact of disruptive stress, which presages a simplifying, often violent outcome. As a last point on this issue, using CC as a tracking rather than profiling tool did not work out well in the case of the Russo-Georgian War. Thus, theoretical, methodological, and empirical arguments all point to the conclusion that neither method can be used to substitute for the other. Our conclusion has to be that both methods are appropriate for their respective purposes; but, regrettably, the computerized scoring of integrative complexity is not yet on the horizon. 12 DRDC Toronto CR

Promotion of bio-methane and its market development through local and regional partnerships A project under the Intelligent Energy Europe programme Contract Number: IEE/10/130 Deliverable Reference: W.P.2.1.3

Single photon source made of single nanodiamonds This project will consist in studying nanodiamonds as single photon sources. The student will study the emission properties of such systems and will show

GCSE Bitesize Controlled Assessment Model 2 (for A/A* grade) Question 3 Subject: Topic: French Writing In this document you will find practical help on how to improve your grade. Before you start working

GCSE Bitesize Controlled Assessment Model 1 (for C/C+ grade) Question 2 Subject: Topic: French Speaking In this document you will find practical help on how to improve your grade. Before you start working

COMMISSION FOR THE COMPENSATION OF VICTIMS OF SPOLIATION RESULTING FROM THE ANTI-SEMITIC LEGISLATION IN FORCE DURING THE OCCUPATION (Decree 99-778 of September 10, 1999) Case Number : (to remind in all

eid Trends in french egovernment Liberty Alliance Workshop April, 20th 2007 French Ministry of Finance, DGME Agenda What do we have today? What are our plans? What needs to be solved! What do we have today?

Sent: 11 May 2011 10:53 Subject: Please find attached a revised amendment letter, extending the contract until 31 st December 2011. I look forward to receiving two signed copies of this letter. Sent: 10

David Marsden Labour market segmentation in Britain: the decline of occupational labour markets and the spread of entry tournaments Article (Accepted version) (Refereed) Original citation: Marsden, David

AccessLearn Community Group: Introductory Survey Introduction The W3C Accessible Online Learning Community Group (aka AccessLearn) is a place to discuss issues relating to accessibility and online learning,

The potential of the building sector in sustainable and lowcarbon strategies Arab Hoballah, UNEP SUSTAINABLE AND COMPETITIVE HOTELS THROUGH ENERGY INNOVATION - NEZEH 2015 L'INNOVATION ÉNERGÉTIQUE AU SERVICE

Version officielle en français disponible ici POSTING APPLICATION OF THE PAY EQUITY ACT (SECTIONS 76.3 AND 76.4) PAY EQUITY AUDIT WITHIN THE PAY EQUITY PLAN OF THE CONSEIL DU TRÉSOR FOR ALL EMPLOYEES IN

DISS.ETH NO. 15802 Forthcoming Database A Framework Approach for Data Visualization Applications A dissertation submitted to the SWISS FEDERAL INSTITUTE OF TECHNOLOGY ZURICH for the degree of Doctor of

The ISIA Minimum Standard Why, What, How Budapest 10-11 October 2011 Vittorio CAFFI 1 Questions to address Why a Minimum Standard? Who is concerned? When has to be implemented? What is it? How can it be

CONVENTION FOR THE JOINT SUPERVISION OF THESIS - SAMPLE CONVENTION DE COTUTELLE DE THESE (Note: This sample agreement will normally be initiated by the European/other partner institution on their proforma)

LA PERSONNE SPÉCIALE These first questions give us some basic information about you. They set the stage and help us to begin to get to know you. 1. Comment tu t appelles? What is your name? Je m appelle

GRAPHIC STANDARDS MANUAL CHARTE GRAPHIQUE This Graphic Standards Manual is aimed at the relays of the Europe Direct information network. They are members of a single family dedicated to the same aim: the

RULE 5 - SERVICE OF DOCUMENTS General Rules for Manner of Service Notices of Application and Other Documents 5.01 (1) A notice of application or other document may be served personally, or by an alternative

Improving the breakdown of the Central Credit Register data by category of enterprises Workshop on Integrated management of micro-databases Deepening business intelligence within central banks statistical

Design and creativity in French national and regional policies p.01 15-06-09 French Innovation policy Distinction between technological innovation and non-technological innovation (including design) French

The Outsiders No Longer: Embracing Non-Traditional Candidates to Grow Your Development Team First Last Position, Firm Carmen Charette Vice President, External Relations, University of Victoria Colm Renehan

#4 EN FRANÇAIS CI-DESSOUS Preamble and Justification This motion is being presented to the membership as an alternative format for clubs to use to encourage increased entries, both in areas where the exhibitor