Learning Outcomes

Terminology Tangle

We have all heard of terms like course objectives, goals, and outcomes in association with syllabi, course planning, and accreditation. In recent years, the great "eye of Gondor" (think Tolkien) has come to rest on learning outcomes.

If we google the term, 5.6 million sites magically assemble. So do hundreds of viewpoints and definitions.

So, let's explore a bit.

"Learning outcomes are statements that specify what learners will know or be able to do as a result of a learning activity. Outcomes are usually expressed as knowledge, skills, or attitudes." Source

Institutional Outcomes

1. There are learning outcomes that are very general, like those from Kansas State University which lists five goal areas (Knowledge, Critical Thinking, Communication, Diversity, and Academic and Professional Integrity) at the institutional level. For each, there are learning outcomes. Read this one:

"Students will demonstrate the ability to access and interpret information, respond and adapt to changing situations, make complex decisions, solve problems, and evaluate actions."

Can you tell to which goal area this belongs? Check your guess

2. The Association of American Colleges and Universities commissioned a Liberal Education and America's Promise (LEAP) campaign to investigate 21st century skills and solidify general outcomes and ways to assess them.

LEAP was organized around Essential Learning Outcomes, which were developed with input from hundreds of colleges and universitites.

The LEAP web site offers information about High Impact Educational Practices and Authentic Assessments.

Changing Landscape

So, learning outcomes in a very general way can tell the intended result of an experience, program, or a course of study.

Look through the Institutional Outcomes for State Fair Community College in Missouri:

Here are the areas of WNMU focus:

The goals of the General Education Program at the University (the goals for student learning and development are consistent across the curriculum) are as follows:

1. Provide each student with opportunities for critical thinking and reasoning, communication of ideas and information to others, numerical analysis and decision-making, and insure personal and emotional well-being;

2. Ensure that graduates of the University possess the skills necessary to interact positively and productively in society;

3. Develop personal, social, and cultural awareness that values cultural diversity and recognizes the commonalities that bind peoples of the world;

4. Instill an appreciation for the variety of perspectives that are used to interpret the world in which we live and that provide the foundation for solving the problems that confront society.

Based on these goals, the following objectives are incorporated into the General Education Program, with emphasis placed on integrating an appreciation of cultural diversity throughout the curriculum:

a. Critical thinking;

b. Creativity;

c. Communication (written, oral, and visual);

d. Multicultural perspectives;

e Social responsibility and cooperation;

f. Literacy of all types (reading, numbers, consumerism, technology);

g. Intellectual curiosity and wonder (continued learning);

h. Environmental responsibility.

A Directional Shift

For many years, the focus on learning objectives or outcomes was on what instructors did. In traditional lesson plans, the teacher was the "doer" of lesson objectives. Today, the focus is on the learner. What will the learner be able to know or do? How will institutions know that students have successfully learned the concept?

Western Washington University talks about a paradigm shift from what teachers do to what students do.

As the assessment and accountability movements in higher education have converged on student learning as the center of the educational universe, ideas about what constitutes a high-quality education have shifted from the traditional view of what teachers provide to a practical concern for what learners actually learn, achieve, and become.

In the traditional "teacher-centered" model, the focus has been on inputs: the credentials of faculty, the topics to be presented, the sequencing of presentations, and so forth.

In the "student-centered," or "learner-centered" model, the focus is on outputs: what knowledge have students actually acquired, and what abilities have they actually developed? Implicit in the student-centered model is the idea that instructors arefacilitatorsof learning. It is not enough to construct a syllabus and present information; the job of instructors now involves creating and sustaining an effective learning environment based on a wide range of "best practices" in teaching and learning. The fundamental role of assessment is to provide a complementary methodology for monitoring, confirming, and improving student learning.

We start here, at institutional objectives, in the journey. Those are "mountaintop" views.

These general outcomes stand on peaks and point west or south.

We can tell the direction we need to go to end up at the destination, but we don't really know the roadmap to get there or how we will know we have arrived.

Look through this presentation on learning outcomes.

Mountaintop to Roadmap

Consider the institutional Student Learning Outcomes (SLOs) from Cascadia Community College in Washington. They are grouped into four areas: Think, Learn, Interact, Communicate.

Collaborate with others in complicated, dynamic, and ambiguous situations

Practice civility, empathy, honesty, and personal responsibility

Let's look, for example, at the middle bullet point.

That is a noble learning outcome, since we live in a world of "complicated, dynamic and ambiguous situations."

So, how would we determine whether two seniors have met that outcome as part of their college experiences, courses, or programs of study?

We could have seen Gilda giving directions to a fellow student as they cooperated on an assignment, heard her explain a concept to a study group in front of the coffee shop, noticed that she worked on a service project with community members and other students. Can we check off that learning outcome as completed on Gilda's records?

Another student, Harvey, was argumentative with the class while working on a project, disruptive in a service learning activity with community members, and earned low peer evaluations in a group assignment. When we consider marking him incomplete on the learning outcome, he points out that the learning outcome wording does not say students had to be good at collaborating, but only that they did it. He demands a checkmark on that goal.

We can probably agree that using broad general goals and outcomes serves a purpose in planning and vision of a college or university. They can also serve a purpose when determining the goals of a department or program.

However, we need to travel down from the mountain vista to get somewhere more definite than east or south.

Narrowing the Route

From institutional learning outcomes, we create and align program outcomes. Then, we specify what each department's goals and outcomes are. Finally, we focus on a course, what it will teach and how students will demonstrate learning.

In each of these areas, we have to then navigate the jungle of goals, objectives, and outcomes.

Western Washington University has these guidelines for all program learning outcomes:

First, student-centered programs are output- oriented. The primary measure of program success is what graduates actually know and are able to do.

Second, student-centered programs are competency-based. Learning objectives and learning outcomes are tied to the most important skills and knowledge in a program.

Third, learner-centered education is dedicated to continual improvement throughongoing assessmentof student learning. By monitoring the effects of program changes on learning outcomes, program faculty are enabled to identify problem areas and to design improvements.

The College of Alameda in California describes the difference between course objectives and courseSLOs like this:

How Do Outcomes Differ From Objectives?

Objectives address the details in a course and are related to the specific course content that will be covered by the instructor. There will be many individual objectives for any class. This objectives list must be exhaustive enough to meet the requirements of equivalent classes [at other univerisities] so that the course will articulate. Objectives tell students what supporting skills, knowledge, and attitudes they will learn during a course that lead to mastery of the course SLOs.

Whereas, SLOs (student learning outcomes) indicate what students will be ABLE TO DO after completing the course. Typically, there will be between three and eight student learning outcomes for any particular course or program. An SLO also provides a context for learning and moves toward a means to evaluate the student's performance. The primary rule to be applied when formulating SLOs is that they must be assessable; there must be some way to measure student success in achieving those goals. Include within the SLO the criteria that must be met in order for students to demonstrate to you that they have achieved the desired outcomes.

One argument might arise: Isn't it obvious what students will learn in a College Algebra course? Can't students intuitively guess what they are required to do in an Introduction to Psychology course?

Well, yes and no.

If you are a junior student looking for a course, how would you know the distinctions between Introduction to Sociology and Social Problems? Looking at a syllabus and learning outcomes should tell you.

Purpose and Benefits of Course Learning Outcomes

We know one purpose of general learning outcomes at institutional, course of study, and program levels: to point the direction from a high vantage point, from a panoramic view.

As we move down into the trip itself, we must have more specific directions to guide us. From institutional and program intentions, we must formulate very specific goals at the department and at the course level.

What purpose do these outcomes have?

Goucher College outlines main purposes and benefits of outcomes:

These are important aspects!

They communicate what the course will teach to students, parents, advisors, faculty, departments, other institutions, and accreditation organizations.

The Roadmap

In a course, learning outcomes are the specific indicators of where the traveler will go, what road and mile markers he will pass along the way, and the exact distance the course will take him to the end destination. Because of their purpose in the course, outcomes should be specific, explanatory, and directed at how learning will be demonstrated.

While "Students will understand the principles of chemistry" might be a course objective, it is not a learning outcome. How would we measure or test it? A student could memorize well and ace all the tests and not really understand the principles. What demonstrates understanding?

Passing the test?

Checking off that he understood?

Doing all the homework?

Completing the labs?

If someone needed to see evidence that the student met that objective, could you provide undeniable proof?

"It may be best to start with what learning objectives aren't: They aren't simply a list of the topics to be covered in the course.

Certainly, there will be a body of knowledge that students should know and understand by the time the course is complete. But if the goals for what students should achieve stops there, there may be many missed opportunities for providing them with a more productive learning experience.

A learning objective should describe what students should knowor be able to do at the end of the course that they couldn't do before. Learning objectives should be about student performance.

Good learning objectives shouldn't be too abstract ("the students will understand what good literature is"); too narrow ("the students will know what a ground is"); or be restricted to lower-level cognitive skills ("the students will be able to name the countries in Africa.").

The following are examples of learning objectives drawn from several courses at MIT:

From a physics course on electromagnetism

The overall goal is to be able to explain the enormous variety of electromagnetic phenomena in terms of a few relatively simple laws.

From the introductory course in the department of Aeronautics and Astronautics

Students will be able to demonstrate an understanding of the fundamental properties of linear systems, by explaining the properties to others.

From a course in managerial communication

By the time you complete the course, you should be able to formulate an effective communication strategy by selecting appropriate content, organizational structure, and media.

Ideally, learning objectives should be accompanied by measurable outcomes, which describe ways in which students will be asked to demonstrate that they have achieved the learning objectives.

Methods of assessment of student learning can take many forms—exams (written or oral), papers, oral presentations, team projects.

Criteria for success (often called rubrics)should be developed so that students understand what is expected of them, and so that they canuse feedback to see where they need to strengthen their performance."(emphasis mine)

Examples of Good Map Directions

Carnegie Mellon lists these course outcomes:

Course: Interpretation & Argument--Communication Globally: The World Is Your Audience

The following objectives emphasize the connection between perceptive analysis and effective writing:

Articulate the thesis of each reading in the same way that you must articulate the thesis in your own written work.

Analyze the selection and effectiveness of support from other sources in the course readings, and support your own ideas by drawing on course readings and other approved sources.

Describe and analyze different types of organization in the readings—from transition sentences to introductions and conclusions—and compare these with the organization you use in your own written work.

Analyze the readings in terms of the common aspects that determine genre, such as purpose and audience, and consider these aspects in your own written work.

Practice analysis in oral and written form through class discussions, group work, and individual conferences.

Provide constructive feedback to your peers on their written work, and address issues identified by your instructor and peers when revising your own written work.

Correctly use MLA style (or, with permission, the preferred citation style of your college or department).

Use standard written English's conventions or grammar, spelling, and punctuation.

Notice the emphasized words. All are things students can do and things we, as evaluators, can prove they did.

Can you look at each of these outcomes and imagine an artifact or specific project or process that could be documented for each?

Moving in the Right Direction

So, a student learning objective needs to:

be specific

be observable

be measurable in some way

contain action verbs

be focused on high-priority outcomes (contained to a manageable number...no more than 10 if possible)

The University of Rhode Island has these guidelines:

Look through the guidelines.

Assessment Quickies

Keeping Your Mind on the Road

When you create Student Learning Outcomes (SLOs), include the:

performance

criterion

condition

Watch the video!

Laying Out the Map

Look at this handout from Eastfield College from Mesquite, Texas. Try the exercises toward the bottom!

Handout from Eastfield College.

Bloom's Classification of Cognitive Skills

One tried-and-true way to formulate active learning outcomes is to use Bloom's Taxonomy as a resource. Bloom organized cognitive processes into "levels." At the lowest, or simplest, level was Knowledge...facts memorized and recalled without much more involved (think of the ABC's). At the other end of the spectrum is Evaluation, in which the learner categorizes the worth, or value, or an array of possibilities using criteria.

Knowing what we want students to end up with is often the beginning place of determining what they will do. If we want them to acquire deeper understanding, then we need to have them accomplish more complicated tasks.

Identifying Earth's continents calls for simpler processes than those involved in determining which continent Earth could do without, with the least physical and economic impact.

Category

Definition

Related Behaviors

Knowledge

recalling or remembering something without necessarily understanding, using, or changing it

After Bloom

Lorin Anderson, a former student of Bloom, revisited the cognitive domain in the learning taxonomy in the mid-nineties and made some changes, with perhaps the two most prominent ones being, 1) changing the names in the six categories from noun to verb forms, and 2) slightly rearranging them (Pohl, 2000).

This new taxonomy reflects a more active form of thinking and is perhaps more accurate:

Category

Example and Key Words (verbs)

Remembering: Recall previous learned information.

Examples: Recite a policy. Quote prices from memory to a customer. Knows the safety rules.

Creating: Builds a structure or pattern from diverse elements. Put parts together to form a whole, with emphasis on creating a new meaning or structure.

Examples: Write a company operations or process manual. Design a machine to perform a specific task. Integrates training from several sources to solve a problem. Revises and process to improve the outcome.

The Affective Domain is gaining some educational notice today. Proponents say that, without the "buy in" of affective components, the cognitive process is hindered. For instance, students studying Algebra often ask, "Why should I learn this?" They see no value in learning it, which hinders their motivation to do so. Until they are willing to receive, hear, and respond to new information, they must involve themselves in establishing why they need to learn it.

"Affect has been neglected in education and this neglect reduces the engagement of both students and teachers in their studies… For the past 200 years, philosophers have emphasized reason over affect… Emotion has, for the most part, been dismissed as unreliable." (Noddings, 1996; P. 435-6)

The affective domain (Krathwohl, Bloom, Masia, 1973) includes the manner in which we deal with things emotionally, such as feelings, values, appreciation, enthusiasms, motivations, and attitudes. The five major categories are listed from the simplest behavior to the most complex:

Responding to Phenomena: Active participation on the part of the learners. Attends and reacts to a particular phenomenon. Learning outcomes may emphasize compliance in responding, willingness to respond, or satisfaction in responding (motivation).

Valuing: The worth or value a person attaches to a particular object, phenomenon, or behavior. This ranges from simple acceptance to the more complex state of commitment. Valuing is based on the internalization of a set of specified values, while clues to these values are expressed in the learner's overt behavior and are often identifiable.

Examples: Demonstrates belief in the democratic process. Is sensitive towards individual and cultural differences (value diversity). Shows the ability to solve problems. Proposes a plan to social improvement and follows through with commitment. Informs management on matters that one feels strongly about.

Organization: Organizes values into priorities by contrasting different values, resolving conflicts between them, and creating an unique value system. The emphasis is on comparing, relating, and synthesizing values.

Examples: Recognizes the need for balance between freedom and responsible behavior. Accepts responsibility for one's behavior. Explains the role of systematic planning in solving problems. Accepts professional ethical standards. Creates a life plan in harmony with abilities, interests, and beliefs. Prioritizes time effectively to meet the needs of the organization, family, and self.

Internalizing values (characterization): Has a value system that controls their behavior. The behavior is pervasive, consistent, predictable, and most importantly, characteristic of the learner. Instructional objectives are concerned with the student's general patterns of adjustment (personal, social, emotional).

Examples: Shows self-reliance when working independently. Cooperates in group activities (displays teamwork). Uses an objective approach in problem solving. Displays a professional commitment to ethical practice on a daily basis. Revises judgments and changes behavior in light of new evidence. Values people for what they are, not how they look.

Assessment of Learning Outcomes...Are we There, Yet?

So, what do you do with those learning outcomes once they are compiled, organized, and distributed?

At every level, SLOs help with quality control and systems improvement.

If we have a list of program SLOs that are not relevant, not informative for students who are considering entrance, don't really describe the results of the program, are not being met, and don't align with institutional outcomes, then we know that we are wandering without direction. Outcomes are the roadsigns, mileage markers, and highway numbers to get us to the destination.

The National Institute for Learning Outcomes Assessment

PRINCIPLES OF GOOD PRACTICE FOR ASSESSING STUDENT LEARNING

1. The assessment of student learning begins with educational values.

2. Assessment is most effective when it reflects an understanding of learning as multidimensional, integrated, and revealed in performance over time.

3. Assessment works best when the programs it seeks to improve have clear, explicitly stated purposes.

4. Assessment requires attention to outcomes but also and equally to the experiences that lead to those outcomes.

5. Assessment works best when it is ongoing, not episodic.

6. Assessment fosters wider improvement when representatives from across the educational community are involved.

7. Assessment makes a difference when it begins with issues of use and illuminates questions that people really care about.

8. Assessment is most likely to lead to improvement when it is part of a larger set of conditions that promote change.

9. Through assessment, educators meet responsibilities to students and to the public.

If we aren't using measures of student success in meeting learning outcomes in our courses, programs, departments, and institutions, we are missing the point. The student is the pivot upon which they all turn.Without evidence of student learning, we are failing to educate.

"Assessment is an ongoing process aimed at understanding and improving student learning. It involves making our expectations explicit and public; setting appropriate criteria and high standards for learning quality; systematically gathering, analyzing, and interpreting evidence to determine how well performance matches those expectations and standards; and using the resulting information to document, explain, and improve performance."