The Role of Beliefs and Attitudes in Learning Statistics: Towards an Assessment Framework

Iddo Gal & Lynda Ginsburg
University of Pennsylvania

Journal of Statistics Education v.2, n.2 (1994)

Copyright (c) 1994 by Iddo Gal and Lynda Ginsburg, all
rights reserved. This text may be freely shared among
individuals, but it may not be republished in any medium
without express written consent from the authors and advance
notification of the editor.

Key Words: Affective issues; Assessment
instruments; Anxiety.

Abstract

While many teachers of statistics are likely to focus on
transmitting knowledge, many students are likely to have
trouble with statistics due to non-cognitive factors, such as
negative attitudes or beliefs towards statistics. Such factors
can impede learning of statistics, or hinder the extent
to which students will develop useful statistical intuitions
and apply what they have learned outside the classroom. This
paper reviews the role of affect and attitudes in the learning
of statistics, critiques current instruments for assessing
attitudes and beliefs of students, and explores assessment
methods teachers can use to gauge students' dispositions
regarding statistics.

1. Introduction

1 In recent years, statistics educators have focused attention
on rethinking the process of statistics education at both the
college and pre-college levels. Calls for reform of college level
statistics education now urge faculty to update their materials and
methods and to involve students in more hands-on activities. At
the elementary and secondary levels, attempts are being made to
integrate statistics education into the mathematics and science
curricula.

2 While statistics educators have focused on improving the
cognitive side of instruction, i.e., the skills and knowledge that
students are expected to develop, little regard has been given to
non-cognitive issues such as students' feelings, attitudes,
beliefs, interests, expectations, and motivations. This paper aims
to examine selected non-cognitive aspects of statistics
instruction, especially those related to affective reactions and
attitudes towards statistics. We believe that further attention
to such factors is warranted, as they may contribute to students'
difficulties in learning basic concepts in statistics and
probability. These difficulties have been widely documented over
the last two decades (e.g., Shaughnessy
1992).

3 Statistics educators routinely mention that many students
enter statistics courses with negative views or later develop
negative feelings about the domain of statistics.
Perney and Ravid (1991, p. 2) describe a familiar scenario:

"Statistics courses are viewed by most college
students as an obstacle standing in the way of
attaining their desired degree. It is not uncommon
to see students who delay taking the statistics
courses until just before graduation. . . College
professors who teach the research and statistics
course are all too familiar with the high level of
anxiety exhibited by the students on the first day
of the term."

"Few college students escape taking an
introductory course in statistics. But for many of
these students, the lessons don't seem to stick.
They remember the pain but not the substance.
`Such initial courses tend to turn students off,'
says Barbara A. Bailar, executive director of the
American Statistical Association."

5 Simon and Bruce (1991, p. 22), explaining
their motivation for developing the resampling method (the process
of generating new samples from a given data set or from a
data-generating mechanism for the purpose of providing access and
meaning to the study of probability and inferential statistics),
offer this lament:

"Prob-stats continues to be the bane of students,
most of whom consider the statistics course a
painful rite of passage---like fraternity
paddling---on the way to an academic degree. At
the end of the semester, most of those who study it
happily put prob-stats out of their minds forever."

6 Some readers may be familiar with students who nickname the
statistics course "sadistics." Rosenthal (1992,
p. 281) adopted the term in an editorial for the UMAP
Journal titled, "No more sadistics, no more sadists, no
more victims."

"I come to announce...a future for the elementary
statistics course, one utopian possibility for
exorcising the malevolent spirits of the standard
errors in our statistics teaching...I appropriate
the pejorative "sadistics" from student culture, to
implore our community to acknowledge and legitimize
students' perceptions of the quality of life in the
course we create for them...[and] reflect the
reality that unintended human suffering takes place
under our watch."

7 We take these and other related views (Benson
1989; Harvey, Plake and Wise 1985; Roberts and Bilderback 1980) as strong indicators that
students' feelings about statistics education, and the effects of
these feelings on resulting learning, knowledge and further
interest in statistics, should occupy a more central role in the
minds of statistics educators.

8 This paper is organized in three sections. First, we
examine the justification for attending to non-cognitive issues
within the larger context of the goals of statistics education.
Next, we briefly review and critique existing approaches to
research on students' beliefs and attitudes towards statistics.
Finally, we explore implications for assessment practices in
statistics education and directions for further research. (By
research we refer both to "academic" research done to increase
general knowledge, as well as to local research that individual
teachers or statistics departments can, and in our view should,
undertake in order to be informed about their students' views
and beliefs and to be able to provide a better service to
learners.)

2. The Logic Behind Attention to Affective and Attitudinal Issues in Statistics Education

9 The above discussion suggests two separate yet related
motivations for attending to non-cognitive aspects of statistics
education: the outcomes we desire and the educational process
itself.

2.1 Outcome considerations

10 In general, statistics courses, especially the first course,
have one or more of the following goals:

a. Prepare students to take higher-level statistics
courses. This is most relevant for college students
(who may consider) majoring in statistics and related
fields.

b. Prepare students for academic or professional careers.
Usually, college students majoring in certain
disciplines (e.g., biology, engineering, education,
business) are required to take a statistics course to
enable them to handle, use, or interpret research or
statistical data in their content area. For quite a
few students, the first statistics course may be the
only statistics course they will ever take.

c. Prepare students to deal effectively with statistical
aspects of the world outside the classroom, as they
relate to students' personal lives or to their
community or civic activities (e.g., be able to
interpret charts, graphs, and statistical claims in a
newspaper article or on TV, improve personal decisions,
etc.). This may be less of a declared goal at the
college level, but more characteristic of the goals of
high-school statistics instruction.

11 The above goals of statistics education imply that students'
encounters with statistics instruction, particularly their
first encounter, should encourage further statistical experiences.
It follows that,

a. A statistics course should not block demand for
further statistics instruction. Students should
emerge from statistics classes without apprehension or
negative feelings about learning more statistics.
Thus, statistics teachers should aim to engender in
students a positive view of statistics and an
appreciation for the potential uses of statistics and
its role in future personal and professional areas
relevant to *each* student.

b. A statistics course should facilitate statistical
thinking. Students should emerge from statistics
classes with an appreciation for when and how the
application of statistics in their professional or
personal lives is warranted, and with a willingness to
think statistically (or probabilistically) in relevant
situations. If, on the other hand, students leave
statistics classes with a sense that statistics is not
relevant to their everyday lives, that it leads to
faulty or biased conclusions, or that they are not
capable of thinking statistically (i.e., "I'm not good
with statistics"), then one is hard pressed to point to
the benefits of taking a statistics class.

12 To ensure they have done their job well, statistics
educators should be able to assess students' standing on
non-cognitive factors, including: (1) interest or motivation for
further learning, (2) self-concept or confidence regarding
statistical skills, (3) willingness to think statistically in
everyday situations, and (4) appreciation for the relevance of
statistics in their personal and vocational lives. (The first of
these factors---interest in further learning---may be an obvious
and easy one to assess, yet is only one of several factors to
which educators must attend, and not necessarily the most
important one, given the three goals for statistics courses
suggested above.)

2.2 Process considerations

13 An altogether different set of reasons for attention to
affective and attitudinal factors concerns the impact such factors
may have on the learning and teaching process in statistics.
Increasingly, one of the stated goals of statistics education is
to develop flexible problem-solving and data-analyzing skills, as
opposed to merely imparting computational and procedural skills (Moore 1990). The creation of a problem-solving environment
for learning statistics requires that teachers build an
emotionally supportive atmosphere where students feel
safe to explore, conjecture, hypothesize, and brainstorm; are
motivated to struggle with and keep working on problems which may
not have right or wrong solutions and may require extended
investment of energy; feel comfortable with temporary confusion or
a state of inconclusive results; and are not afraid to experiment
with applying different (statistical) tools or methods.

14 Many students, however, do not come to statistics classes
fully ready to embrace and function within a learning environment
oriented toward problem solving. Instead, many carry baggage that
includes negative or unconstructive beliefs about themselves in
relation to the learning of quantitative and mathematical issues,
including math anxiety (McLeod 1992);
apprehension about taking tests, including math tests (Hunsley 1987); beliefs about the relevance (or lack thereof)
of statistics for their future career or job plans (Roberts and Saxe 1982), and more.

15 Beliefs and attitudes related to math may play a powerful
role in affective responses to statistics since students often
expect that the study of statistics will include a heavy dose of
mathematics, including complex algebra and formulae (
Simon and Bruce 1991). The frequent appearance of statistics
courses within mathematics departments (or as part of a high
school math class) reinforces this perception. Since practically
all students have studied some high school level mathematics
before starting a formal statistics class, their affective
reactions to those math-learning experiences may affect how they
relate to statistics learning. Students' predispositions,
beliefs, and expectations may interact with aspects of the
learning environment created by the teacher in ways that would
work against what the teacher is attempting to accomplish.

16 Affective and attitudinal factors are also likely to become
important during a class once students begin to experience
difficulties or frustrations with their statistics studies.
Research on adults' memories of learning math during their school
days (Allan and Lord 1991; Tobias
1994) suggests a sequence of events often triggered when
students experience an initial failure to understand. An initial
confusion is followed by a failure to receive adequate
explanations or assistance from the teacher, leading to loss of
confidence and panic over the sense of lack of control of one's
own comprehension. Eventually, students become bored or
disinterested, "switch off" and disengage from what they perceive
as a futile learning process. This process is often accompanied
by students' development of negative views of their mathematical
skills and their ability to handle quantitative problems (Tobias 1994). We hypothesize that similar processes happen
in statistics classes; therefore statistics educators need to be
sensitive to students' emotional and attitudinal status. It is
thus important that statistics educators have access to assessment
instruments that enable an initial diagnosis of
their students' attitudes and beliefs and that also enable
monitoring of the status of such attitudes and beliefs during
a course.

3. Research on Attitudes Towards Statistics

17 The body of research on students' attitudes, beliefs, and
affect related directly to statistics education is very small and
problematic. To date, practically all studies
that have been published (either in journals or conference papers
available through ERIC) have been based on data gathered from
paper-and-pencil Likert-type scales. Two measures worthy of
attention, the Statistics Attitude Survey (Roberts and Bilderback 1980) and the Attitudes Toward
Statistics (Wise 1985), are described below.
Educators interested in using the SAS, ATS, or similar instruments
for diagnostic purposes, and researchers interested in
understanding factors affecting statistics learning, should be
aware of several fundamental problems with the SAS and ATS, and
also with the studies in which they were developed or employed.
These problems are identified later in response to four questions
of concern to statistics educators. Finally, a summary of the
state of attitude assessment in science education provides a
larger context for our evaluation of statistics attitude
assessment frameworks.

3.1 Existing attitude assessment instruments

18 The SAS was developed by Roberts
and Bilderback (1980) to improve the prediction of success in
statistics courses beyond what was offered by general measures of
affective reactions to mathematical domains. The instrument
appears to have been designed as an "affective scale couched in
statistical jargon" (p. 236) in line with the general item
format and item phrasing employed by measures of math anxiety and
attitudes towards mathematics (Fennema and
Sherman 1976; Richardson and Suinn 1972).
Roberts and Saxe (1982) suggest
administering the SAS at the beginning and/or end of a statistics
course. Students are given 33 statements regarding the perceived
usefulness of statistics ("Statistics will be useful to me to test
the superiority of one method over another"); personal competence
in solving statistical problems ("I normally am able to solve
statistics problems without too much difficulty"); beliefs about
statistics ("I find statistics to be very logical and clear"); and
affective responses to statistics ("The thought of taking another
statistics course makes me feel sick"). Students respond using a
1-5 Likert-type scale ranging from Strongly Agree to Strongly
Disagree. Roberts and Bilderback
(1980) reported that the questionnaire is highly homogeneous
(alphas in three samples all exceeded 0.93), and
Roberts and Saxe (1982) corroborated
that all items in the questionnaire load high on a single general
factor.

19 The ATS was developed by Wise (1985) in
an attempt to improve on perceived limitations of the SAS, mainly
that many of the SAS items seemed to Wise to measure prior
achievement in or exposure to statistics, rather than attitudes.
(For example, one SAS item states, "I make a lot of errors when I
calculate statistics problems.") The ATS scale is composed of two
subscales: a 9-item Course subscale, measuring attitudes towards
the course in which students are enrolled, and a 20-item Field
subscale, measuring attitudes towards the use of statistics in
their fields of study. As with the SAS, the students respond to
each of the items using a five-point Likert scale, ranging
from Strongly Disagree to Strongly Agree. Wise reported a
correlation of 0.33 between the two sub-scales. He argued that
since the ATS focuses on attitudes rather than students' previous
experience with statistics, it is more appropriate than the SAS as
a measure of changes in students' attitudes from the beginning to
the conclusion of a statistics course.

20 A few other instruments relating to statistics anxiety have
been developed but have not been widely tested or evaluated,
and their quality is yet to be established. A 24-item instrument,
Students' Attitude Toward Statistics (STATS) has been developed by
Sutarso (1992), and a small-scale pilot
study indicates that this instrument does not particularly differ
from the SAS and ATS. Another instrument, the Coping Strategies
Inventory for Statistics (CSIS) has been designed by Jarrell and Burry (1989). It appears to have no particular
relevance to statistics, as all of its items evaluate general
test-taking skills and coping strategies. The word "statistics" is
never mentioned in any of the CSIS items and is only used in the
title of the instrument.

3.2 What constructs are being measured by instruments such
as the ATS and SAS?

21 As mentioned earlier, items in both the SAS and ATS deal
with respondents' self-evaluation of several different issues:
competence in dealing with statistical problems and calculations,
competence in and attitudes towards dealing with mathematical
tasks, interest in learning statistics, beliefs about the
usefulness of statistics, and expectations regarding the relevance
of statistics for their careers. While all these topics appear
important and are aligned with the goals of statistics education,
it is difficult to accept that students' views, feelings, or
attitudes towards these diverse issues are all combined into one
(SAS) or two (ATS) global scores. The findings reported by
Roberts and Reese (1987) regarding
the similarity of the SAS and ATS scales (a reported correlation
of 0.88 between the two scales), their homogeneity in terms of
both high internal consistency (alpha coefficients exceeding
0.90), and their simple factorial structure (a single general
factor, though Wise [1985] earlier reported
two factors), do not sit well with the diverse content of the
items.

22 If students appear to answer such diverse items in more or
less the same way, there may be underlying sources that cause
responses to converge. As mentioned earlier, many students expect
a statistics course to have a heavy mathematical bent (and such
courses usually do; see Simon and Bruce 1991),
or they may expect that "doing" statistics later on in their
career will involve lots of math and computations. Thus,
students' responses to the ATS or SAS may reflect mainly their
attitudes towards mathematics, or beliefs about their own ability
or knowledge in mathematics, which may then pass for attitudes
towards statistics.

23 Not only do instruments such as the ATS and the SAS ignore
the possibility that students who report negative attitudes
towards statistics are actually reacting to the perceived
mathematical component of statistics, they also do not enable
their users to distinguish the effects of (a) generalized anxiety,
(b) test anxiety, and (c) mathematics anxiety on students'
responses. The nature of mathematics anxiety and its relationship
to generalized anxiety and test anxiety have been the focus of
much research in the mathematics education and educational
counseling communities (Hembree 1990; McLeod 1992), yet are not reflected in the development of
the ATS or the SAS. It is entirely possible that students' scores
on the SAS and ATS, and the homogeneity of these scales, reflect
influences of other types of anxieties or attitudes which are not
unique to statistics.

24 An altogether different problem lies with the fact that the
SAS, ATS and similar questionnaires never ask students to
explain their answers to Likert-type items. This
limits interpretability of scores as it is unclear what motivates
students to answer as they do, whether answers represent a
negative attitude towards the domain in general or reflect a local
(classroom) reality to which the student reacts. Assume, for
example, that a student circled "Strongly Disagree" for the SAS
item "I would like to study advanced statistics." What does this
indicate? Should this response be taken as indicating a
negative attitude towards statistics? If so, is
the student afraid of the topic of statistics in general, or only
of "advanced" statistics? In this case, what does "advanced"
mean? Is it possible that the student is uninterested because she
believes (whether rightfully or not) that advanced statistics are
not required to graduate from college, or perhaps the student does
not think statistics would be required in her professional
career after graduating from college? Alternatively,
could the student be uninterested because she has heard
rumors about the difficult exams given by a professor whose class
is the only one available at present? We argue that very little
can be learned from responses to Likert-type scales without the
use of an open-ended procedure enabling respondents to elaborate
on their initial answers; further suggestions in this area are
made below.

25 We also have concerns about the meaning of the construct of
"usefulness of statistics" being measured by SAS items such as
"Statistics will be useful to me when I describe my professional
activities to other people" or by the ATS Field subscale. (Items
about the usefulness of mathematics in
out-of-school contexts similarly appear in many inventories of
attitudes towards mathematics.) Consider that students come to
statistics classes with different career goals and expectations
which may be at different levels of crystallization when the
assessment is administered. Some students have little or no
information about the actual content or requirements of their
future occupations. (See Dick and Rallis 1991
for a discussion of career development issues in making course
selections in mathematics.) Depending on their stage of career
development and the amount of factual information they have,
respondents might not think (rightfully or not) that statistics
will be useful in their future profession. Thus, a student's
responses to items assessing usefulness-of-statistics issues might
have little to do with
feelings or attitudes towards statistics as a
subject; instead, they may only reflect the student's vocational
development (Osipow 1973) or knowledge about
requirements or content of certain jobs. In sum, we argue that
it is difficult to establish the construct validity of Likert-type
items or scales measuring usefulness issues without further
information about the process or factors that account for a
response.

3.3 What do beginning students understand the term
"statistics" to mean?

26 As mentioned earlier, one reason for developing the SAS and
the ATS was to aid in initial diagnosis of
students' attitudes towards statistics. In this context any
examination of the meaning of the scores generated by the ATS and
SAS (or other instruments using similar items) needs to consider
that all the items on these instruments include
statements using the word
statistics. (Examples from the SAS include "It
takes me a long time to understand a statistical concept" and
"There are so many statistical concepts to learn that I get
confused.")

27 How the word "statistics" is interpreted by respondents to
surveys of attitudes towards statistics is a major point for
concern. While the word "statistics" should be imbued with some
meaning for students who are finishing a statistics
course, many students who are just beginning their
first statistics course have little sense of what "statistics"
includes; to the extent that students attach some meaning to the
term "statistics," our concern is that students' generalized
expectations may reflect stereotypical, distorted, or partial
perceptions which may differ from student to student.

28 The potential fuzziness of the term "statistics" for many
beginning students is illustrated by results from a preliminary
study conducted as part of Project STARC, an NSF-funded project on
the acquisition of statistical skills conducted at the University
of Pennsylvania (Gal 1993; Gal
and Wagner 1992). A group of twelfth graders from a
prestigious private school in the Philadelphia area, all of whom
were college bound and in the process of applying to high-level
universities or colleges, were asked, "What do people study when
they take a statistics course? What comes to your mind when you
hear the term statistics?" The following quotes illustrate the
range of responses obtained:

"When I hear the term statistics, I usually think of
basketball statistics (% of shooting, number of
rebounds, etc.) or survey statistics (as in 40% of
teenagers hate peanut butter). I'm not exactly sure
what people study when they take a statistics course,
but I think it's along the lines of percentage and
graphing, etc."

"I really am not sure what you learn when you take
statistics. I guess it has to do with taking averages
of things."

"Numbers and figures of surveys come into my head. I
think of people having a boring life if they make a
profession of it, because I know it's a lot more
complicated than what I said."

"Although I have never taken a statistics course, I
hear they are very difficult. Being a huge sports fan,
when I think of statistics I think of how many goals or
touchdowns someone has."

"I imagine a statistics course as boring and factual as
math. Statistics are gathered data (1000 people live
in PA), information good for newspapers, writers, and
lawyers."

"Statistics is when someone takes the scores of many
things, such as baseball statistics. Math is used a
great deal in finding statistics."

29 Some of the above statements contain elements that
reasonably portray some of what actually happens in statistics
classes, but others give tenuous or incorrect information. Most
convey some fuzziness regarding what "statistics" might be about,
or a narrow view regarding life domains where statistics may be
used. It is difficult to see how these students, who are
positioned to begin a high-school or college-level statistics
class, would effectively relate to attitude items which repeatedly
use the word statistics, thus raising concerns regarding the
diagnostic value of attitude surveys if used before the onset of
statistics instruction. In fact, the attitude questionnaire
itself may unnecessarily cause some students to begin to believe
that statistics is very computationally-oriented and difficult.

3.4 How sensitive are the ATS, SAS, or other instruments in
detecting changes in students' attitudes from the beginning
to the end of a course?

30 As suggested above, one goal of statistics education is to
engender in students a positive outlook about statistics and its
uses, and confidence in themselves as (intuitive, fledgling)
statisticians. It is thus important to examine the ability of
current assessment instruments to detect changes
in students' standing on such issues. Considering the
"slipperiness," or subjectivity of some of the above target
constructs, or the multiplicity of factors affecting achievement
in a statistics class, it is surprising that researchers have so
far paid little attention to instrument sensitivity in this
regard.

31 We have argued earlier that students' standing on some of
the constructs being measured by attitude instruments, such as
beliefs about the usefulness of statistics, may change over time
as a result of vocational maturation or increase in career-related
knowledge. There is reason to suspect that such changes, if they
occur, will develop gradually (Osipow 1973).
In contrast, students' standing on some of the other facets being
measured, such as affective responses or self-confidence regarding
statistics, may be more labile and likely to fluctuate depending
on changing circumstances and classroom events. Thus,
interpretation of score changes needs to take into account the
expected stability over time of the constructs being measured.
Unfortunately, the developers of statistics attitude measures
currently in use have not provided data about the instruments'
test-retest reliability.

32 Even when score changes may be noticed from the beginning to
the end of a course, there is little information on how to
interpret such changes. A key question relates to the magnitude
and nature of a score change that would be considered meaningful
from an educational perspective. For example,
Roberts and Saxe (1982) report that SAS scores rose from
105.29 to 109.94 with a standard deviation of approximately 15 and
a maximum scale score of 165. Presently, no large-scale data exist
which describe score patterns within the general population of
students, and there are no norms comparable to those available for
instruments measuring math anxiety, such as the
MARS (Richardson and Suinn 1972) or the
Fennema-Sherman scales (Fennema and Sherman
1976). In the absence of such information, it is difficult
for teachers or researchers using statistics attitude surveys to
know if score changes (e.g., due to an intervention designed to
affect students' attitudes) are educationally, rather than
statistically, significant.

33 Assume, for example, that the pre-instruction
class mean on SAS items such as "I would like to study advanced
statistics," or "I find it easy to explain a statistics topic
to someone else" was 3.5 (given a 1-5 scale, where 1 is "strongly
disagree" and 5 is "strongly agree") and that the
post-instruction mean was 3.9. Assume further that
this pre-post difference was calculated for a large
introductory statistics class and is statistically significant.
What does the finding mean? Is it good news, bad news, or no news
for the instructor? (Even after the change, scores may still be
in a range indicative of negative attitudes or affect towards
statistics.)

34 Studies using statistics attitude surveys usually report
correlational data or mean score changes, but not absolute score
levels. Such data, while perhaps showing the existence of
relationships between attitude scores and variables of interest,
provide insufficient information about the nature of that which is
changing over time. A basic requirement from studies of attitudes
towards statistics is that they report absolute scores in addition
to any other statistical data, so that the educational
implications of any score change can be evaluated. Correlational
data, as well as reported mean score changes, may be of limited
value if they are not accompanied by data about trends of
educational significance, such as the proportion of students whose
scores stayed the same, improved (i.e., attitudes became more
positive), or worsened (i.e., attitudes becomes more negative);
such figures are masked when a single statistic is calculated on a
whole sample.

3.5 What do responses to Likert-type scales tell us about
students' concerns about learning and using statistics?

35 There has been a long tradition of using Likert-type items
in questionnaires measuring attitudes towards mathematics and
science (Helgeson 1993). Instruments such as
the SAS or ATS, which build on this tradition, yield scores that
are easily reportable and are convenient to use for a broad
description of the outcomes of statistics education. Such
instruments have not been designed to provide diagnostic
information that can point to particular issues of concern to
individual students; therefore, we argue that present
measures of attitudes towards statistics have very limited
capability to inform teachers interested in improving the process
or content of (statistics) education through
either remedial interventions or preventive measures.

36 A key deficiency discussed earlier is that responses to
Likert-type scales reveal little about the causes for answers.
Especially when it comes to mathematics or statistics anxiety,
which may negatively influence students' interest, motivation, and
comprehension, it appears that Likert-type scales have very
limited usefulness for identifying what individual students are
anxious about, their beliefs about learning statistics that might
be counter-productive, and what types of support or educational
experiences might be useful for students.

37 Mathematics educators are beginning to recognize this
fundamental deficiency in the design of measures of students'
beliefs and attitudes towards mathematics; some (see McLeod 1992 for a recent review) are beginning to conduct
interviews, lead focus group discussions, or ask students to write
journals or histories of their present or past mathematical
experiences to gain a closer look at the factors underlying
students' outlook on their educational experiences in quantitative
fields.

3.6 Comparison with attitude assessment instruments used in
science education

38 Above we expressed reservations about the quality of the
instruments presently used to assess beliefs, attitudes, and
affect towards statistics, and raised questions about the
interpretability of results of such instruments. To put these
comments in perspective, it might be useful to examine the state
of the art in science education, a related, and presumably more
mature, field of attitude assessment.

39 In contrast to about 12 studies and four instruments so far
published in the area of assessment of statistics education, in
a recent review, Helgeson (1993) notes that
more than 700 studies have been published so far on students'
attitudes towards science (including both attitudes towards
science and scientific beliefs) and that more than 50 different
instruments have been developed over the years. Major reviews
of these studies and instruments repeatedly point to two problems:
lack of conceptual clarity in defining attitudes towards science
and other constructs used by researchers, and "technical"
limitations of instruments used to assess attitudes. Helgeson (1993, p. 6) cites Germann
(1988):

"First, the construct of attitude has been vague,
inconsistent, and ambiguous. Second, research has
been often conducted without a theoretical model of
the relationship of attitude with other variables.
Third, the attitude instruments themselves are
judged to be immature and inadequate."

40 Helgeson (1993) further reviews
results of an evaluation by Munby (1983) of
the Scientific Attitude Inventory (SAI), then the most popular of
the attitude instruments used in science education research. The
SAI uses 60 items with a four-point, Likert-type response scale.
An examination of the instrument itself and of 30 studies which
used it by 1983 revealed that the instrument's validity was highly
questionable. Munby (1983) further
commented that the field of measuring attitudes was replete with
instruments, but that these were being used in a rather "cavalier
fashion," without attention to the various aspects of their
validity and reliability.

41 We believe that much of what has been said about assessment
of attitudes in science education is directly applicable to the
emerging field of assessment of attitudes in statistics education.
Work on assessing statistics attitudes so far has proceeded with
little attention to the meaning of the complex constructs being
measured (e.g., what do "attitudes towards statistics," or
"statistics anxiety" mean?). Few visible attempts have been made
to benefit from accumulating experience and from methodologies
developed in other disciplines with similar assessment needs, such
as in mathematics and science education. Research designs used by
researchers studying attitudes towards statistics show
little sophistication, and statistical analyses and presentation
of results are often ill-suited to the analytic task. Studies so
far do not appear to have produced information of much educational
value about students' affective reactions to statistics learning,
in part due to exclusive reliance on Likert-type scales. In fact,
studies conducted so far serve as rather unimaginative and
disheartening models for the skills we would like our statistics
students to develop.

4. Conclusions and Future Directions

43 We find that current approaches towards assessing attitudes
towards statistics are ill-suited for the tasks identified
above, due to: (1) exclusive use of Likert-type scales, (2) the
inclusion of items that are not appropriate for students who have
not had extended experience with statistics, or who are not at a
very advanced stage of their career development, (3) the tendency
not to seek explanations from subjects for their answers, (4) the
practice of using total scores which aggregate responses to
different item types, and (5) inattention to the links between
attitudes towards statistics and other constructs, such as
attitudes towards mathematics, when interpreting results. These
problems severely limit interpretability of obtained scores at
both the personal and group level.

44 The important progress made by mathematics educators in
clarifying the differences among emotions, attitudes, and beliefs,
and between these and other related constructs such as anxiety,
confidence, self-doubt, self-concept, or self-efficacy (see McLeod 1992), could provide an excellent starting point for
statistics educators seeking to understand factors affecting their
students' performance and learning. Further, the emerging
literature on the role of non-cognitive factors in cognitive
performance, including metacognitions, affect, and beliefs (Dweck 1986; Schoenfeld 1992),
highlights the practical importance of attending to non-cognitive
factors.

45 Progress in understanding the role of non-cognitive issues
and their effects on statistics learning must be supported from
two different but related directions. First, we must improve our
ability to assess students' attitudes toward statistics so that we
can understand the meaning as well as the existence of anxious or
uncomfortable feelings. Second, we must explore the interaction
between these negative attitudes and students' beliefs
about statistics as a field of study and about themselves as
learners and users of statistics. These issues are discussed
below.

4.1 Improving assessment

46 Given the above criticisms, a necessary first step is to
create a better instrument which would include fine-tuned
subscales with an acceptable factorial structure. The instrument
should enable separation of students' attitudes towards
mathematics, test-taking, statistics, and perceived usefulness of
statistics. Reliability and validity data for this instrument
should be presented separately for students at different stages of
academic careers, and for those majoring in different fields,
where the role and relative importance of statistics can be
estimated. Our analysis of deficiencies in current practices
suggests, however, that little would be gained from additional
development of "improved" assessment instruments, if these
instruments continue to rely exclusively on Likert-type items.

47 We believe that as a minimal requirement, an assessment
instrument for initial diagnosis of students' attitudes and
beliefs towards statistics should combine the use of Likert-type
items with open-ended questions. This design
should enable students to explain what feelings,
attitudes, expectations, or beliefs underlie their responses to a
Likert scale; describe the intensity and frequency of specific
emotional responses; and elaborate on their source(s) or causes.
So that the validity of the scale will not be compromised by
interspersed requests for response explanations, students could
first respond to a series of statements using the Likert scale and
then, on a separate page, be directed to explain each of their
responses. Examples of common five-point Likert-type items
(Strongly Disagree to Strongly Agree) and suggestions to guide
students' explanations of their ratings are:

I think statistics will include a lot of mathematics.
(How do you know? What mathematical skills, if any, do
you expect to use in this course? Would this create a
problem for you?)

Solving math problems in math class makes me anxious.
(Why? What types of problems, if any, make you
anxious? How anxious?)

I was successful in the last math course I took. (What
percentage of the material did you feel you understood?
What did "success" mean for you?)

I am nervous about taking statistics. (Why? What
aspect(s), if any, of the course make you feel this
way?)

I am looking forward to taking statistics. (Why? What
aspect(s), if any, of the course make you feel this
way?)

I think that statistics will be useful in my
profession. (How do you know? Why do you assume
statistics will or will not be useful? What profession
are you considering?)

48 Alternatively, open-ended items in an initial assessment
could ask students to express concerns they have about taking a
statistics course and about the extent to which their prior
academic background may assist or impede their learning of
statistics; relate factors that may cause poor performance in this
course; describe feelings about learning mathematical topics in
general; outline expectations for the extent to which the course
will involve mathematical work; explain motivation(s) for taking
the course; or describe how the course may fit into future career
plans, if any (after all, the course may be a requirement forced
upon the student...). Possible open-ended "sentence completion"
items might include:

I expect that for me, personally, statistics may be
useful later for... (write "not at all" if you so
feel)

When I think about this course, I'm concerned that ...
(write "not at all" if you so feel)

49 A different format which has been used to explore students'
feelings about learning mathematics and which could also be
used as part of an assessment of attitudes towards statistics,
includes a series of 12 to 15 pictures depicting faces with
various emotions (or just a list of emotion words), such as
"anxious," "puzzled," "fearful," "frozen," "interested,"
"indifferent," "confused," etc. Individuals can mark these to
express their feelings about mathematical or statistical
situations. (See Allan and Lord 1991for an
application with adult learners of math in basic skills programs.)
This type of assessment provides limited information about
students' feelings, but is useful to break out of the mold of
perceiving students' attitudes as lying across linear
paths, and of "attitude change" as moving students "higher" or
"lower" along such paths, as is the case when five-point Likert
scales are used. This form of assessment can be easily adapted to
assess students' feelings at various points during the class and
to detect students who develop high degrees of anxiety or
frustration. Such an assessment is especially useful if students
are asked to explain their answers and be specific whenever they
circle faces or labels that indicate some distress or
difficulty. (These responses can also serve as a useful source of
in-class data on which students can practice various data analysis
techniques.)

50 Statistics educators interested in a deeper understanding of
how their students perceive statistics and statistics courses
could opt for the use of structured interviews (Tobias
1994) or discussions in focus groups. A closer look at
clinics for treating math anxiety among college students or adults
engaged in continuing education, which are now emerging on various
campuses around the United States (Hembree
1990 cites recently published reports), can point to
additional formats for assessing students' attitudes and affective
reactions to statistics learning. The information garnered may
help a statistics department form policies regarding interventions
or support for students with strong negative feelings.

4.2 Identification and influence of beliefs

51 The research on students' beliefs about statistics is much
more sparse than that concerning attitudes towards statistics.
Other than the commonly held belief that statistics is heavily
mathematical, students' beliefs about statistics remain
unexplored. This gap in our knowledge is disturbing since recent
work on the relationship between beliefs and attitudes in
mathematics learning (McLeod 1992) suggests
that beliefs may be filters through which experiences and events
are interpreted by learners.

52 Since students' beliefs towards statistics have generally
not yet been explored, and given that many students identify
statistics with mathematics, it is informative to first examine
related research on beliefs about mathematics and mathematics
problem solving. Schoenfeld (1992, p. 359)
lists some of the typical student beliefs about the nature of
mathematics and mathematical activity:

Mathematics problems have one and only one right
answer.

There is only one correct way to solve any mathematics
problem---usually the rule the teacher has most
recently demonstrated to the class.

Ordinary students cannot expect to understand
mathematics; they expect simply to memorize it and
apply what they have learned mechanically and without
understanding.

Mathematics is a solitary activity, done by individuals
in isolation.

Students who have understood the mathematics they have
studied will be able to solve any assigned problem in
five minutes or less.

These beliefs could be similar to students' beliefs about
statistics, but there may also be beliefs unique to statistics and
statistics education that have not been recognized so far.

53 The relationship between beliefs and attitudes such as
anxiety is also being explored in the field of mathematics
education. Carter and Yackel (1989) argue that
mathematics anxiety is an appropriate response when certain
beliefs are present. For example, if an individual believes that
mathematics is a collection of rules and procedures, then
success in mathematics is determined by one's ability to memorize
the rules and procedures and produce them at appropriate moments
in the problem-solving process. For routine exercises and
practice problems, this belief system allows success and comfort.
If an appropriate rule or solution path is not apparent during a
problem-solving situation, however, then the learner is at a
standstill since there is no mechanism in place for modifying
and/or developing rules or procedures. This situation causes
feelings of panic, inadequacy, and anxiety. On the other hand,
some individuals believe that mathematics is "relational," that
is, mathematical knowledge is an interconnected, meaningful
network. These individuals are not afraid to enter their
"mathematical network" and try to derive or develop an appropriate
solution when the solution to a mathematical problem is not
immediately apparent. Relational mathematics learners realize that
there are many points from which to enter their networks of
mathematical knowledge and will thus feel comfortable using their
experiences and emerging knowledge as problem-solving tools.

54 To make the learning of statistics less frustrating, less
fearful, and more effective, further attention by both statistics
educators and researchers should be focused on beliefs, attitudes,
and expectations students bring into statistics classrooms or
develop during their educational experiences. Instructors can use
assessments of attitudes and beliefs to understand students'
presuppositions and, with continuous monitoring, to identify areas
of frustration for individual learners, guiding the provision of
supportive interventions. The discussion above suggests that the
development of assessment instruments capable of providing
relevant information of value for instructional and research
purposes involves many challenges, both conceptual and
methodological. Unless significant progress on such fronts is
achieved, however, the vision of statistical literacy for all may
remain out of reach for too many learners.

Carter, C. S., and Yackel, E. (1989), "A
Constructivist Perspective on the Relationship Between
Mathematical Beliefs and Emotional Acts," paper presented at
the Annual Meeting of the American Educational Research
Association, San Francisco (ERIC document ED309076).

Dick, T. P., and Rallis, S. F. (1991),
"Factors and Influences on High School Students' Career
Choices," Journal for Research in Mathematics
Education, 22,
281-292.

Fennema, E., and Sherman, J. A. (1976),
"Fennema-Sherman Mathematics Attitudes Scales: Instruments
Designed to Measure Attitudes Towards the Learning of
Mathematics by Females and Males," Journal for Research in
Mathematics Education, 7, 324-326.

Gal, I., and Wagner, D. A. (1992),
"Project STARC (Statistical Reasoning in the Classroom):
Annual Report," University of Pennsylvania, Graduate School of
Education.

Gal, I. (1993), "Reaching Out: Some Issues and
Dilemmas in Expanding Statistics Education," in Introducing
Data-Analysis in the Schools: Who Should Teach It and
How?, ed. L. Pereira-Mendoza, Voorburg, Holland:
International Statistics Institute.

Germann, P. J. (1988), "Development of the
Attitude Toward Science in School Assessment and Its Use to
Investigate the Relationship Between Science Achievement and
Attitude Toward Science in School," Journal of Research
in Science Teaching, 25, 689-703.

Harvey, A. L., Plake, B. S., and Wise, S. L.
(1985), "The Validity of Six Beliefs About Factors Related to
Statistics Achievement," paper presented at the Annual
Meeting of the American Educational Research Association,
Chicago, IL.