Department of Physics, University of Maryland, College Park,
MD 20742

V. STUDENT EXPECTATIONS: DISTRIBUTION AND EVOLUTION

In this section, we discuss the results obtained from giving the
MPEX survey at the beginning and end of the first term of introductory
calculus-based physics at six different institutions. In each
case, the subject covered was Newtonian mechanics. The schools
involved include the flagship research institutions of three large
state universities: the University of Maryland (UMCP), Ohio State
(OSU), and Minnesota (UMN); plus three smaller schools: Dickinson
College (DC), a small public liberal arts college (PLA), and a
public two year college (TYC). At the named colleges, we have
data from multiple instructors. In the case of the last two institutions,
data was only collected from a small number of instructors and
students. These are included in order to demonstrate how the MPEX
survey can be used as a diagnostic tool, but are kept anonymous
to protect the identity of the instructors and institutions involved.

At Maryland, Ohio State, and Minnesota, classes were presented
in the traditional lecture-lab-recitation framework with some
modifications. At Maryland, there is no laboratory in the first
semester and some of the recitation sections were done with University
of Washington style tutorials.[26] Results for tutorial and recitation classes were comparable. At Minnesota, the laboratory and
recitations involve carefully designed problem-solving group work.[27]
At Ohio State, lectures are traditional but are enhanced by use
of various interactive elements, while recitation and laboratory
are done in a group problem-solving format somewhat similar to that developed
at Minnesota. At Dickinson College and at the public liberal arts
institution, the classes surveyed were done in the Workshop Physics
environment which replaces lectures with a combined lab and class
discussion.3d The two-year college used a purely traditional
lecture-recitation framework. Like Maryland, they have no lab
in the first semester. The schools involved, the structure of
their courses, and the number of students in our sample are summarized
in Table 1.

In order to eliminate the confounding factor of differential drop-out
rates, we only include students who completed the survey both
at the beginning and at the end of the term. We say that the data
is matched. Our results show some differences among different
classes at the same institution, but the variation is statistically
consistent with the sample size. Therefore, we have combined results
for similar classes at a given institution.

The overall survey results for the six schools are presented in
an A-D plot in Fig. 2. In order to simplify the reading of the
graphs, we have displayed the results from the three large research
universities in one part of the figure (Fig. 2a) and those from
the smaller schools in another (Fig. 2b). The pre-course results
are shown by filled markers and the post-course results by open
markers. The results of the expert group are shown by a cross.

Fig. 2a: A-D plot for large schools, average of all items......
Fig. 2b: A-D plot for small schools, average of all items.

We make two observations.

The initial state of the students at all the schools tested
differs substantially from the expert results. The expert group
was consistent, agreeing on which survey responses were desirable
87% of the time. Beginning students only agreed with the favorable
(expert) responses about 40-60% of the time, a substantial discrepancy.
What is perhaps more distressing is that students explicitly supported
unfavorable positions about 20-30% of the time.

In all cases, the result of instruction on the overall survey
was an increase in unfavorable responses and a decrease
in favorable responses (though some changes were not significant).
Thus, instruction produced an average deterioration rather
than an improvement of student expectations.

The overall survey includes items that represent a variety of
characteristics, as displayed in Table 2. In order to better understand
what is happening in the classes observed, let us consider the
initial state and the change of student expectations in our various
clusters. The results are presented in Table 4.

Table 4

Overall

Indepen-dence

Coherent

Concept

Reality Link

Math Link

Effort

Experts

87/6

93/3

85/12

89/6

93/3

92/4

85/4

College

80/10

80/8

80/12

80/8

94/4

84/9

82/6

HS

73/15

75/16

62/26

71/18

95/2

67/21

68/13

POT

68/18

81/12

79/8

73/13

64/20

85/8

50/34

UMCP pre

54/23

54/25

53/24

42/35

61/14

67/17

67/13

UMCP post

49/25

48/27

49/27

44/32

58/18

59/20

48/27

UMN pre

59/18

59/19

57/20

45/27

72/9

72/11

72/11

UMN post

57/20

58/20

61/17

46/28

69/10

72/12

63/16

OSU pre

53/23

51/24

52/21

37/36

65/10

65/13

66/16

OSU post

45/28

46/28

46/26

35/35

54/17

55/20

44/30

DC pre

61/15

62/14

58/17

47/23

76/4

70/10

75/7

DC post

60/19

67/14

66/18

58/23

72/9

71/12

57/26

PLA pre

57/23

57/27

57/26

38/46

71/13

74/11

72/8

PLA post

49/31

52/22

47/33

45/34

52/25

54/19

48/30

TYC pre

55/22

41/29

50/21

30/42

69/16

58/17

80/8

TYC post

49/26

42/32

48/29

35/41

58/17

58/18

65/21

Table 4: Percentages of students giving favorable/unfavorable
responses on overall and clusters of the MPEX survey at the beginning
(pre) and end (post) of the first unit of university physics.

A. The Independence Cluster

One characteristic of the binary thinker, as reported by Perry
and BGCT, is the view that answers come from an authoritative
source, such as an instructor or a text, and it is the responsibility
of that authority to convey this knowledge to the student. The
more mature students understand that developing knowledge is a
participatory process. Hammer classifies these two extreme views
as "by authority" and. "independent." Survey
items 1, 8, 13, 14, 17, and 27 probe students' views along this
dimension. On this cluster, students' initial views were favorable
in a range from 41% (TYC) to 62% (DC). All groups showed essentially
no significant change as a result of one term of instruction.
For comparison, the USIPOT showed favorable views on these items
81% of the time.

Survey items 1 and 14 are particularly illuminating and show the
largest gaps between experts and novices.

#1: All I need to do to understand most of the basic ideas
in this course is just read the text, work most of the problems,
and/or pay close attention in class.

#14:. Learning physics is a matter of acquiring new knowledge
that is specifically located in the laws, principles, and equations
given in the textbook and in class.

The expert group was in 100% agreement that students should disagree
with item 1 and in 84% agreement that they should disagree with
item 14. Disagreeing with these items represents a rather sophisticated
view of learning, but favorable shifts on these items are exactly
the sort of changes that indicate the start of a transition between
a binary and a more constructivist thinker. The interviews strongly
support this view. Students who disagreed with these items were
consistently the most vigorous and active learners.

This cluster of items, and items 1 and 14 in particular, appear
to confirm that most students in university physics enter with
at least some characteristics of binary learners, agreeing that
learning physics is simply a matter of receiving knowledge in
contrast to constructing one's own understanding. We would hope
that if a university education is to help students develop more
sophisticated views of their own learning, that the introductory
semester of university physics would begin to move students in
the direction of more independence. Unfortunately, this does not
appear to have been the case. In the touchstone items of 1 and
14, the only significant improvement was DC on item 14 (26% to
53%), and overall, only DC showed improvement.

B. The Coherence Cluster

Most physics faculty feel strongly that students should see physics
as a coherent, consistent structure. A major strength of the scientific
worldview is its ability to describe many complex phenomena with
a few simple laws and principles. Students who emphasize science
as a collection of facts fail to see the integrity of the structure,
an integrity that is both epistemologically convincing and useful.
The lack of a coherent view can cause students many problems,
including a failure to notice errors in their reasoning and an
inability to evaluate a recalled item through cross-checks. Survey
items 12, 15, 16, 21, and 29 have been included in order to probe
student views along this dimension.

Our expert group was in agreement as to what responses were desirable
on the elements of this cluster 85% of the time. The initial views
of students at our six schools were only favorable between 50%
and 58% of the time. Most classes showed a small deterioration
on this cluster, except for UMN (slight improvement from 57% to
61% favorable responses) and DC (improvement of 58% to 66% favorable
responses).

Two specific items in this cluster are worthy of an explicit discussion.

#21: If I came up with two different approaches to a problem
and they gave different answers, I would not worry about it; I
would just choose the answer that seemed most reasonable. (Assume
the answer is not in the back of the book.)

#29: A significant problem in this course is being able to
memorize all the information I need to know.

Item 21 is a touchstone. Coming up with two different answers
using two different methods indicates something is seriously wrong
with at least one of your solutions and perhaps with your understanding
of the physics and how to apply it to problems. Our expert group
and USIPOT students feel strongly that students should disagree
with item #21 at the 85% level. Initially, only 42-53% of students
produced a favorable response for this item, and only DC showed
any significant improvement on this item (52% to 59%). One school
(PLA) showed a substantial deterioration (42% to 17%).

The interpretation of item #29 may depend significantly on the
details of the examination structure of the course being probed.
A sophisticated student will realize that the large number of
different equations and results discussed in a physics text can
be structured and organized so that only a small amount of information
needs to be memorized and the rest can be easily rebuilt as needed.
Item #29 is part of a probe into whether or not students see this
structure or are relying on memorizing instead of rebuilding.
However, if students are permitted to use a formula sheet or if
exams are open book, they may not perceive memorization as a problem.
This does not mean that they see the coherence of the material.[28]
If extensive information is made available to students during
exams, item #29 needs to be interpreted carefully. A variety of
examination aids were used for the classes of this study, ranging
from open-book exams (DC) to no aids (UMCP). Omission of item
#29 does not change the distributions in this cluster significantly.

C. The Concepts Cluster

The group of items selected for the concepts cluster (items 4,
19, 26, 27, and 32), are intended to probe whether students are
viewing physics problems as simply a mathematical manipulation
of an equation, or whether they are aware of
the more fundamental role played by physics concepts in complex
problem solving. For students who had high-school physics classes
dominated by simple "problem solving" (find the right equation,
perhaps manipulate it, then calculate a number), we might expect
largely unfavorable responses on our items. We would hope, however,
for substantial improvement, even as the result of a single college
physics course.

Our experts agree on their responses to the items of this cluster
89% of the time. The initial views of the students at the six
schools were favorable between 30% (TYC) and 47% (DC) of the time.
All schools showed some improvement on this cluster except OSU
which showed a small deterioration (37% to 35% favorable responses).
The two Workshop Physics schools showed the largest gains in favorable
responses (DC 47% to 58%, PLA 38% to 45%).

Within this cluster, the results on items 4 and 19 are particularly
interesting.

#4: "Problem solving" in physics basically means
matching problems with facts or equations and then substituting
values to get a number.

#19: The most crucial thing in solving a physics problem is
finding the right equation to use.

While these items are similar, they are not identical. Agreeing
with item 4 indicates a naive view of physics problems or a lack
of experience with complex problems. A more experienced student
could reject 4 but still agree with 19 because of the phrase "most
crucial". One would, however, hope that increased experience
with complex physics problems would lead a student to disagree
with this item as well. For example, 54% of the USIPOT students
gave a favorable response on this item as compared to only 22%
of beginning students at UMCP. Our personal observations of these
students indicate that as expected, the USIPOT students have considerably
more experience with complex problem solving than the typical
beginning engineering student.

Notes

C. McDermott et al., Tutorials in Introductory
Physics, to be published. (See refs. 4b, c, and e for published
descriptions of the method.)

Indeed, some student comments lead us to suspect
that formula sheets may have the tendency of confirming
student expectations that formulas dominate physics. Their interpretation
is that although memorizing lots of formulas is important for
professionals, they do not need to do so for the current course.
Thus, many faculty may be encouraging precisely that attitude
they hope to discourage when they permit the use of formula sheets
on exams. We are not aware of any research that shows the effect
of formula sheets on student perceptions of the coherence of the
material.