issue of our new magazine: Assessment in Action. We have
some specific goals for this publication. They include:
•Encouraging conversation around various assessment
topics;
•Encouraging the continuing development of a culture
of assessment and a culture of evidence in the
division;
•Providing resources related to assessment practices;
•Introducing and explaining the Mission Assessment
and Alignment Planning (MAAP) program; and
•Highlighting and learning from the variety of effective
assessment practices from throughout the division.
I would add to this list—having some fun. As some of our
writers point out, assessment tends to have a bad rap—it
is dull, onerous, an unnecessary addition to our work,
and something people avoid unless instructed to do it.
Within these pages we intend to challenge these beliefs.
We argue that assessment can be fun, energizing, and even
inspirational! You can let us know if we succeed.
Of course, we welcome feedback (what kind of assessment
magazine would we be if we didn’t?!) and hope you will
participate in an ongoing conversation about assessment.
Our intention is to publish Assessment in Action (AiA)
five times each year – once in the summer and twice each
semester. We welcome the active participation of anyone in
the division wishing to assist in the production of this new
resource.
Thank you and happy assessing!
Patrick Love
Associate Vice President for Student Affairs

Assessment in Action

in This
Issue

Never Stop Assessing:
An Interview with Gavin
Henning
By Catrina Gallo

page

04

Weight Loss
& Assessment?
By Anne Newman

page

05

Michael’s Musings
By Michael Miragliotta

page

07

5 Tips for Running
Focus Groups
By Bridget Turner-Kelly

page

11

Undergrad in Action
By Ian Ragsdale

page

The Dreaded “A” Word
By Sarah Grun

page

13

14

Cover Story
“MAAPING” the Future of
Undergradute Education at
Rutgers University
page
By Ashley Nickelsen
08
Brent Ruben and Susan Lawrence
(cover photo) are co-chairs of the
MAAP Committee, a university
wide assessment project that
started on the back of a napkin.

Summer 2012

page

03

Never Stop Assessing:
Gavin
An Interview With Henning
By Catrina Gallo

I

had the pleasure of
interviewing Dr. Gavin
Henning about his
assessment experiences. Gavin is a
national expert on assessment who
recently left the position of Senior
Research Analyst at Dartmouth
College to become Associate
Professor of Higher Education at
New England College. Gavin is
founder and chair of Student Affairs
Assessment Leaders and he also
chaired the ACPA Commission on
Assessment and Evaluation.
After speaking with Gavin, it was
evident that despite his background
and experience he has had his fair
share of both successful and failed

wished to gain an understanding of
the college experience for students
of color. Many factors contributed to
the initiative falling short. One of the
main things Gavin came to realize
was that the topic he chose was too
broad. The “college experience” does
not refer to one specific thing; it has
a vast definition. He subsequently
realized that he should have spent
more time narrowing the topic on
particular aspects of the college
experience for students of color.
Basically, what we might think is
one issue can often be several issues
rolled into one. We need to peel back
those layers and get to the root issue
of concern.

use the resources available to us,
such as experienced professionals in
other units or faculty with skills we
can use in our assessment work.
3. Avoid a mixed-methods approach
Gavin had chosen a mixedmethods approach for this initiative,
combining a survey with focus
groups. “Poor decision,” he says
now! He came to realize that an
online survey was not the most
effective way to reach students of
color. Furthermore, a survey was
not the best approach for assessing
something that turned out to be
much more of a qualitative issue.

1. Be sure to narrow and define the focus of assessment
2. Use all available resources
3. Avoid a mixed-methods approach
4. Learn and use additional assessment techniques
5. Regularly review assessment action plans
6. Understand the potential impact of you as a person and you in your
position
7. Recognize that assessment takes place within a political environment
8. Never stop assessing
assessment experiences, and from
the stories he shared, and especially
the stories of failure, there were
a number of lessons for anyone
attempting to practice effective
assessment. This was a wide-ranging
interview on a number of topics,
but the focus of this article is on
the lessons he has learned through
experience that he wishes to share
with student affairs professional.
1. Be sure to narrow and define the
focus of assessment
Gavin shared a story with me
about a failed assessment. In this
assessment, he and a colleague
page

04

2. Use all available resources
Gavin explained that the
Commission for Assessment and
Evaluation was supporting him
and his colleague during this
failed initiative, but he did not
utilize the commission as much as
he should have. Had he involved
the commission more, he and
his colleague would not have
encountered as many obstacles
and they may have made different
decisions that could have led to a
more positive and effective outcome
for the project. When we conduct
assessments on campus we need to

Assessment in Action

Students were more willing to talk
about their experiences in focus
groups that had other students of
color than respond to an anonymous
survey written by people they did
not know.
4. Learn and use additional
assessment techniques
When I asked Gavin for his best
piece of advice for Student Affairs
professionals doing assessment,
he simply stated: “Learn more
techniques, because a survey is a
blunt instrument.” A survey can
often be tedious for those requested
to take it. Also, people who are

asked to take the survey often
have no commitment to the overall
focus of the assessment effort. More
creative or interactive methods,
such as individual interviews, focus
groups, and participant observation
will often lead to better and more
accurate data. Gavin believes that we
need to shift away from the surveyfirst mindset.
5. Regularly review assessment
action plans
Gavin asserted that it is critical to
examine and assess our action plans
on an ongoing basis. He explained
that in his failed assessment example,
he and his colleague should have
been monitoring and following up
with the students they were trying to
reach with their survey, but this was
another one of their shortcomings.
Oftentimes, speed bumps can arise
during what was thought to be a
foolproof plan. Gavin now knows

he will make changes to his plan as
needed.
6. Understand the potential impact
of you as a person and you in your
position
Another mistake from his
experience that Gavin highlighted
was that both he and his colleague
were White, heterosexual
individuals, and having a
homogenous group interpreting
assessment responses can potentially
hinder a holistic and accurate view of
results.
7. Recognize that assessment takes
place within a political environment
When Gavin reported the
results of this assessment, the
recommendations that he made
were actually the opposite of the
recommendations that another
unit had made to the president
on the same topic. Of course, it is
completely acceptable to present

contradicting results if that is what
the project yielded. Gavin, however,
was completely unaware of what
the other group had originally
recommended and did not address
this discrepancy. He vowed to never
let that happen again!
8. Never stop assessing
One of the most important things
that Gavin Henning pointed out to
me is that assessment should never
stop. We need to get in the habit of
constantly improving our practices,
and learning how to perform
effective assessment.
Look for stories from my
interview with Gavin in future issues
of Assessment in Action.
Catrina Gallo is a 2012 graduate of the
Rutgers Ed.M. College Student Affairs
Program. She is currently a Residence
Life Educator at Rutgers University.

Weight Loss & Assessment?
By Anne Newman

T

he new school year is
almost upon us. This is
the perfect time to think
about assessment plans for the
upcoming year. Assessment plans
can provide your office with a
wealth of information and can make
planning more intentional. While
assessment is worthwhile, I know
some student affairs professionals
dread the thought of putting together
a thoughtful and useful assessment
plan. Assessment dredges up
thoughts of numbers, complicated
surveys, analysis, and judgment.
Assessment also means extra work
and the fear of finding out that
what you are doing is not effective.
Student affairs professionals struggle
to find the proper motivation to
begin working on an assessment

plan. Many of us search for a better
understanding of the importance of
assessment in order to get started on
an assessment plan.
I happen to love assessment. I
can admit this in public. Iâ&#x20AC;&#x2122;m one of
those fortunate people that have an
affinity for numbers. I like figuring
out what makes things tick and how I
can make things better. I like coming
up with questions and using data to
answer those questions. I also love
using assessment to budget and plan
for my office. Assessment helps us
bring about change, and we all know
change is necessary for student affairs
programs to thrive and survive. I
love watching my office change and
develop based on the information we
collect every year.
There are a lot of things in which I
do not excel and do not come easy for

me. Take weight loss for instance. It
is time consuming to follow a plan, to
keep track of progress, to write down
everything you eat and to exercise
on a consistent basis. Because it was
important to me to lose weight and
get healthy I started a weight loss
plan last October and have been able
to lose 55 pounds so far. However, I
have been challenged during every
step of my journey and know I have a
long way to go before I meet my long
term goals.
So why talk about weight loss in
an assessment article? It makes a lot
of sense to anyone who has struggled
to lose weight. I canâ&#x20AC;&#x2122;t tell you the
anxiety I feel at the beginning of a
diet. Losing weight also requires
a great deal of persistence and
motivation. In addition, successful
weight loss also requires an

Summer 2012

page

05

understanding of the science behind
successful weight loss.
In their book, Assessment in Student
Affairs: A Guide for Practitioners, Lee
Upcraft and John Schuh provide nine
principles of good practice for student
affairs assessment. As I was rereading
these assessment guidelines, I was
struck by some of the similarities
between good practice for student
affairs assessment and the principles
of successful weight loss.
Upcraft and Schuh point out
that assessment works best when
a program lists clear and explicitly
stated goals. Outcomes for your
assessment plan need to be both clear
and measurable. When I set out to
create measurable goals for our office

to weight goals, I also set fitness
goals for myself. I decided to set a
goal to complete a marathon by the
time I turn 45. Each year before the
marathon, I have a shorter distance
to complete. I have completed a 5K
race and I’m now training for an
8K this fall. These fitness goals are
understandable and attainable. This
keeps me motivated to continue
working toward my goal.
Upcraft and Schuh also note that
assessment works best when it is
ongoing. A lot of times we look at
assessment as an “all or nothing”
proposition. Those successfully using
assessment know that assessment
is cyclical. Assessment occurs, and
then changes are made, which are

on.
Numbers are important. You have
to have objective information about
your programs in order to know
what you’re doing well and what
you need to improve. How can you
plan for the future if you don’t know
where you stand now? It’s the same
for weight loss. As much as I can’t
stand my scale, I know that in order
to be successful at losing weight, I
have to weigh myself on a regular
basis. I also needed to weigh in and
take measurements at the start to
accurately measure progress. I am
better able to know where I’ve been
and where I am going because I have
accurate data about myself and my
progress.

Those successful at weight loss must set measurable goals.
Having a measurable outcome makes the goal seem more
attainable.
I understood that saying, “We want to
be the best student conduct program
in the country” was not good enough.
How would anyone know what that
means? What makes a good student
conduct program? Realistic outcomes
related to conduct include “reducing
recidivism rate by 10 percent” or
“increase student knowledge of
the Code of Student Conduct by 15
percent.” Assessment is much more
tangible and understandable to staff
members when there is something
concrete we can measure. It’s the
same with any weight loss program.
Those successful at weight loss must
set measurable goals. Those that are
successful don’t just say, “I want to
be skinny” or “I want to be healthy.”
They instead say, “I want to lose 30
pounds”, or “I want a BMI under
22”, or “I want my blood pressure to
be 120/80.” Having a measurable
outcome makes the goal seem more
attainable. When I started my weight
loss plan last October I tried to come
up with measureable short term
and long term goals. In addition
page

06

then assessed again. This is how an
organization keeps track of how it is
doing and can respond when changes
need to be made. It is the same with
weight loss. Individuals who rely
on fad diets or quick fix schemes
often find themselves frustrated
when they can’t lose weight. Those
that do lose weight on fad diets are
often frustrated when they regain
the weight. It certainly is tempting
to purchase “infomercial” products
advertised on Saturday morning
that promise success with little
change or effort. Those successful at
weight loss know that weight loss
is a journey and it takes changes in
one’s lifestyle to maintain weight loss.
The weight loss program I follow
has you reassess how much you eat
every time your weight drops below
a certain level. This helps me reassess
my plan in order to restart weight loss
and maintain progress. By constantly
monitoring progress and altering my
weight loss plan when needed, I was
able to successfully lose weight and
keep the weight from coming back

Assessment in Action

Weight loss and assessment have
a lot in common. Both are dreaded
by individuals who are daunted by
the perception of the enormity of
the task. Both require a great deal of
motivation and commitment in order
to produce successful results. Both
can also provide rewards if done
correctly. I can attest to the rewards
of both weight loss and assessment.
I know I am personally and
professionally better by completing
both to the best of my ability.
The start of every new school year
brings with it the opportunity to
create new “school” year resolutions.
I hope one of your resolutions
for the upcoming year includes
incorporating assessment into your
programs and services. While it might
not be easy, the rewards will be worth
it. If you happen to see me eating
something fattening, please remind
me of my goals.
Anne Newman is the Director of
Student Conduct at Rutgers University.

Michael’s
Musings
By Michael Miragliotta
Assessment.
Many readers have
probably stopped
reading after my
first sentence.
Assessment can
be a daunting
task and conjures
images of surveys,
complicated
statistics, and
hours of work.
As the chair of
the Division’s
Assessment
Committee, I am
here to provide
assurance that anyone in this Division can do
assessment everyday without needing any extra tools
other than what she/he already has.
How does Dining Services know how to best staff
its dining halls? How does Student Life know if a
program is successful as it is happening? How do you
know if the presentation you are giving is engaging
the audience? All of these questions can be answered
with the term “spotlight assessment.” Spotlight
assessment is the act of assessing your work every
day and making adjustments with the observations
you collect. Dining records the amount of swipes in
every dining hall and uses this data to create staffing
patterns that ensure the dining halls run smoothly.
This is a form of spotlight assessment that uses easily
collected data to make decisions.
Student Life staff can assess a program as it is
happening by monitoring the audience. They can
count the number of audience members, the number
of audience members engaged in the program, and the
number of positive comments they hear as audience
members are leaving the program. All of these are
examples of spotlight assessments helping Student
Life gauge the relative success of a program.
I use spotlight assessment when engaging in a
presentation. As I speak to my audience I am using
spotlight assessments to gauge the level of attention,
engagement, and comprehension from my audience.
An audience full of people on their phones, yawning,
or not making eye contact demonstrates to me that I

am not engaging the audience. Audience members who
are making eye contact, nodding, smiling and laughing
(when appropriate) shows me that I am engaging the
audience. Spotlight assessments allow me to make
adjustments to my presentation, review content that
may not have been presented clearly, or change my
presentation style on the fly to engage my audience.
Spotlight assessment is used in many other aspects
of our work every day. It should be used as a first step
to assessing our work. If, for example, Student Veteran
Services notices dozens of students starting to ask about
financial aid opportunities, that office can implement
various initiatives to proactively get financial
information out to veterans. This spotlight assessment
could lead to a more complicated assessment of the
types of financial questions students are asking or a
program evaluation of a newly created financial aid
program geared toward veterans. Spotlight assessments
are the first step in using the information that is at
our fingertips to make Rutgers a better place for our
students.
As we begin this new academic year, I hope that you
can begin to look at the copious amounts of information
we all have at our fingertips. Use spotlight assessments
to begin to understand how your services are affecting
our students. Start small and ease into assessing your
work every day. The Assessment Committee will be
providing tools and help as we begin to create a culture
of evidence based decisions. Good luck, and remember
that assessment is not a scary word!
Each issue of Assessment in Action will feature Michael’s
Musings. In this column, Michael will address some of
the common misperceptions about assessment, reader
questions, and interesting assessment strategies. Michael
Miragliotta has worked at Rutgers for three years in
various roles in Housing and Residence Life. He completed
his B.A. in Psychology and Sociology at The College of
New Jersey and his M.A. in Psychology at New York
University. Michael structured both of his degrees to
focus on research methodology and statistics. He was
an adjunct professor of research methodology in the
Psychology Department at The College of New Jersey for
two years. His passion lies in using appropriate assessment
methodology to help understand how to do our jobs better
for the benefit of our students.

Summer 2012

page

06
07

T

By Ashley Nickelsen

wenty-seven months ago in the Busch Campus Center dining area, the Mission Alignment Assessment
and Planning (MAAP) system came to life as a simple sketch on the back of a Gerlandaâ&#x20AC;&#x2122;s napkin. Brent
Ruben, Executive Director of the University Center for Organizational Development and Leadership, and
Susan Lawrence, Dean for Educational Initiatives and the Core Curriculum, serve as the co-chairs of the MAAP
Task Force. With a team of more than 10 representatives from throughout Rutgers University, the MAAP Task
Force brainstormed, discussed, and discerned ways to assess undergraduate education at Rutgers University.
After sharing hundreds of ideas and conducting months of research, the MAAP Task Force transformed the once
rudimentary sketch on a napkin into a massive visual matrix ready for use.

Meet MAAP
MAAP provides a framework to meet both
external and internal assessment needs. The visual
matrix consists of seven mission-critical goals
that were developed based on university strategic
planning documents and the Rutgers University
mission. These goals for undergraduate education
include: 1. Student Recruitment, 2. Rutgers Support
and Pride, 3. Personal and Professional Development,
4. Academic Degree Goals, 5. Progress to Degree, 6.
Post-Graduation Success, and 7. Operational Support
for Faculty and Staff. Each department and unit
strives to align their programming services with these
mission-critical goals as well as the more specific subgoals within each category [see side bar]. The matrix
page

08

Assessment in Action

serves as a visual aid that demonstrates how each
unitâ&#x20AC;&#x2122;s goals align with the overarching university
goals.
Overall, the success of MAAP relies on the
assessment of these goals, the use of these data to
measure university attainment in meeting these
mission-critical goals, and the subsequent planning
to improve programs and services to better align with
goals.
A Collaborative Perspective
On June 9, 2012, I had the pleasure of interviewing
Susan Lawrence, co-chair of the MAAP Task Force.
During the interview, I learned about MAAP as a
collaborative assessment model that helps individual
units and departments critically think about their

Susan compared the simplicity of MAAP to
purpose within the university and their goals
creating
the dinner menu for a summer barbecue.
to contribute to the undergraduate educational
“MAAP
is fundamentally a fairly simple concept;
experience. Susan emphasized the importance
assigning
goals to columns and unit services to rows.
of assessing the undergraduate experience by
[see
the
image
on page 8]. It’s about finding the
comparing MAAP to a matryoshka, also known as
match;
much
like
you would run a potluck dinner at
a Russian nesting doll. A Russian nesting doll has
a
barbecue.
Who’s
bringing the salads, burgers, and
been traditionally used to tell stories, and each doll
desserts?
Does
fruit
salad fit into the salad or dessert
contains increased levels of detail and decoration.
category?”
The
fruit
salad could possibly contribute
Every doll has its purpose and place within the
to both the dessert and salad
storytelling process, and
categories depending on the
within the MAAP construct,
type of barbecue, but the
each doll represents a
potluck system helps people
specific unit or department
avoid overlap and assess
at Rutgers. Every unit and
the purpose of each dish.
department contributes a
Similarly, MAAP provides
piece of their unique story
the framework for units
based on their specific
and departments to better
goals. Susan explained
understand what they are
that MAAP is a process as
bringing to the “university
much as a result, “MAAP is
table.” There are a number
not only about the process
of outstanding programs,
of getting everybody
initiatives, and efforts
on board, but it’s also
Susan
compares
MAAP
to
a
matryoshka.
occurring
within each unit and
about having a cohesive,
Every
doll
has
its
purpose
and
place
within
department
at Rutgers; MAAP
coherent, and tellable
the
storytelling
process,
and
within
the
MAAP
provides
a
framework
for
story.” Susan hopes that
construct,
each
doll
represents
a
specific
unit
or
these
units
and
departments
MAAP will change peoples’
department at Rutgers.
to tell their stories better, to
way of thinking, and that
coordinate
their
goals
to
align with the university’s
more faculty and staff will begin to critically think
overarching
mission,
and
to collect assessment data to
about “mapping out” the goals for their individual
determine
unit
and
institutional
effectiveness.
department. Susan is particularly excited by the
The
Future
of
MAAP
synergistic approach within the university as a result
As Rutgers changes, MAAP will continue to
of MAAP; both staff and faculty have been working
evolve.
With a few departments already in the
in a collaborative and cohesive process to tell their
process
of utilizing MAAP to evaluate their goals,
individual stories. Each part contributes to the whole,
the
Task
Force hopes for MAAP to become the “new
and people have begun to see the impact they make
normal”
across the university. The institutional goals
on the student experience. Susan emphasized that in
may
change,
but the basic framework will persist,
order to understand our specific roles and how each
because
MAAP
provides a comprehensive visual
of our roles fit within the largest nesting doll, or in
story
to
guide
university
strategic planning and to
this case the greater university community, each unit
tell
our
story
to
the
public.
Because the concept is so
is encouraged to develop their own goals, specifically
simple,
MAAP
can
be
adapted
to other institutions
aligned to the undergraduate mission goals.

Summer 2012

page

09

and to a variety of programs. Susan is excited for the
future of MAAP because her passion for assessment is
driven by her belief in the university mission to help
students through the most important transformative

that story-telling process, but MAAP continuously
urges staff and faculty to reflect on and assess the
purpose of their stories as well as their story-telling
techniques.

MAAP is not only about the process of getting
everybody on board, but it’s also about having a
cohesive, coherent, and tellable story.
-Susan Lawrence
years of their life. Susan explains, “We need to be able
to tell our story about the value of the undergraduate
learning experience we provide at a university
that has deep historic roots and a university that is
committed to meeting the challenges of education
for the 21st century.” MAAP not only facilitates

More
Information

Ashley Nickelsen is a 2012 graduate of the Rutgers
Ed.M. College Student Affairs Program. She is currently a
Residence Hall Coordinator at the University of Delaware.

A little lost?
Here is some MAAP information to help you along the way.

The members of MAAP are

Brent Ruben, Professor II
(Distinguished Professor) of
Communication; Executive Director,
University Center for Organizational
Development and Leadership
Susan Lawrence, Director of Core Curriculum, Office of the Vice President
for Undergraduate Education; SAS Dean for Educational Initiatives and
the Core Curriculum; Associate Professor of Political Science
Barbara Bender, Associate Dean, Graduate School-NB
Richard DeLisi, Dean, Graduate School of Education; Professor of
Educational Psychology
Gary Gigliotti, Associate Vice President, Teaching and Assessment
Research, Executive Director of the Center for Teaching Advancement and
Assessment Research, and Professor of Economics
Carol Goldin, Associate Dean for Assessment, School of Pharmacy
Robert Heffernan, Director of Institutional Research & Planning
Kate Immordino, Director of Organizational Research and Assessment at
the University Center for Organizational Development and Leadership
Patrick Love, Associate Vice President for Student Affairs
Don Smith, Vice President, Information Technology and Chief
Information Officer, Associate Professor of Computer Science

page

10

Assessment in Action

In addition to Student Affairs, MAAP
has also been doing pilots with:

University Office of Undergraduate Education
Division of Instructional Support
Division of Academic Engagement
The Libraries
Administration and Public Safety
Information Technology
Core Curriculum
The School of Arts and Sciences
Academic Services
Honors Program

Online Resources

http://bit.ly/N59OiF - MAAP website

Special
Contributor

5 Tips for Running
Focus Groups

By Bridget Turner-Kelly

Y

ou’ve seen it on television, in political campaigns, and other media
advertising—focus groups are the way to gauge what consumers want.
Focus groups are a relatively recent technique used in higher education
to understand consumers of the college environment. Their strength lies in the
knowledge gained through interactions. Many of us have not known what we
wanted to say until someone else expressed their thought. Gretchen Rossman
and Sharon Rallis argue that “People often need to listen to others’ opinions and
understandings to clarify their own.” In assessments where you need to know what
people really think and why they think that way in order to improve practice, focus
groups can provide rich, efficient data that teases out people’s attitudes and beliefs as
they develop through group interaction and exchange.

Here are my top 5 tips for facilitators on Running Focus Groups:

1. Be sure to include people in the focus group who have unique experiences with the topic being
assessed. In other words, you need to include the students, faculty, staff, or parents that are the most
informed about whatever it is you want to know.
2. Establish a safe environment where people can trust group members, examine their own ideas and
feelings, and openly and honestly express their responses. Your challenge is to carefully handle issues
that people are reluctant to discuss in public and make sure you avoid forcing people to disclose
embarrassing or sensitive information about themselves or the university.
3. Use a structured interview guide with 4-7 questions for an hour to an hour-and-a-half session
with around 6-10 people. Allow for many follow-up and group-generated questions so most of the
discussion occurs among the participants rather than between facilitator and participants.
4. Focus on and return to the central assessment questions throughout the focus group so you can be
sure to obtain the data you need. Gently steer the group back from areas unrelated to the study while
making sure everyone has a chance to contribute to the conversation.
5. Conduct focus groups in such a way that people can publically disagree with each other and discuss
their views and you can view how committed people are to their views. Encourage people to defend
or provide further explanation for their beliefs so you can interpret the meaning behind the words.
Below I provide a real university
example to tease out these tips.
HBU and PWI Campus Contexts
Case:
Professor Sharon Fries-Britt and
I sought to examine the influence
of a historically Black university
(HBU) and a predominantly White
institution (PWI) on successful Black

students. Utilizing tip #1 (include
people who are most likely to
know what you need to know), we
believed the students themselves
were the most knowledgeable about
their experience. We conducted
four focus groups, two at the HBU
campus and two at the PWI campus,
to obtain data about academic

and social programs as well as
experiences that facilitated or
blocked their success. Utilizing tip
#3 (use an interview guide for the
6-10 people in focus group), we had
approximately 6 students in each
focus group and asked them each
structured questions from an
interview guide, such as:

Summer 2012

page

11

(a) What challenges have you
encountered to your academic and
social life on campus?
(b) Has a topic ever come up in class
that you felt you couldn’t express
your opinion on?
(c) How have you connected socially
to the campus? And, what do you
do when you need academic help?
One theme evident during the
focus groups conducted at the HBU
was the close connection and sense
of family-like atmosphere that the
students exuded in their interactions
with each other. Individual
interviews may have communicated
the degree of closeness the Black
students felt with each other at the
HBU. However, the focus group
setting allowed the facilitators to
witness the looks of pride, smiles,
high-fives, and genuineness
exhibited as the students responded
to comments other participants
made. Thus, utilizing tip #5 (find
the meaning people make of their
words), the group interaction
allowed the facilitators to interpret
the feelings and attitudes expressed
in the actual stories of support
and involvement Black students
shared. The findings of strong
support and high involvement at
the HBU were in stark contrast to
the findings of dispersal of energy at
the PWI. Our use of tip #4 (focus on
assessment questions to get needed
data) allowed us to keep coming
back to our central question of
what facilitated or blocked student
success at the PWI. The students
at the PWI described their energy
being taken away from their studies
in the classroom and into efforts to
combat isolation resulting from the
lack of racial and ethnic diversity.
They reported feeling as if most of
the activities on the PWI campus
were primarily geared for White
students. We worked hard to adhere

page

12

to tip #2 (establish a comfortable
environment) by using strategies to
create a safe and open environment
in the focus group and saw the fruits
of our labor in how comfortable
the group was trusting each other
with their experiences and feelings.
During the PWI focus group
interviews, knowing glances of
shared frustration were exchanged.
Heads nodded in agreement about
different coping strategies used
by students to defray the effects of
racist comments or situations they
encountered at the PWI.
As advertised, the focus group
method provided rich data about
the phenomenon under study
and did so in an efficient way.
The strength of the focus group
interaction showed during the
HBU focus group interviews where
we observed a distinct “uplifting”
atmosphere and a contrasting
“coping” atmosphere during the
PWI focus group interviews. The
focus groups helped us more fully

Assessment in Action

understand the varying experiences
students described on each campus.
As difficult it is for educators to use
assessment to guide practice, focus
groups provide a convenient way
to gather thick, descriptive data
necessary to answer questions about
complex or not easily-accessed
information about the college
environment. We would do well in
higher education to utilize focus
groups to ask the right people what
we want to know to guide better
practice.
Bridget Turner Kelly is the Associate
Professor and Program Co-Director of
the Educational Leadership program at
Loyola University. Dr. Kelly obtained
her Ph.D. from the University of
Maryland, College Park in Social
Foundations of Education. She is an
expert in assessment, and she has
numerous publications on the topic of
assessment and focus groups.

Summer 2012

page

02

Undergrad

in
Action
By Ian Ragsdale

A

s a student worker for
Housing and Residence Life
during the summer break, Luis
Fernandez ‘15 helped the marketing
team research how other schools market
and communicate to their student
residents. This initiative plays into
division-wide goals of self-assessment
and bringing innovation and clarity to
branding, marketing, and discovering
new technologies and processes that
connect with students.
For Luis, a first-generation college
student who just finished his first year
at Rutgers, the experience was far
more than a few paychecks over the
summer. It was also a deep immersion
into the world of higher education
and prepared him for his new role as
Advocacy Director for the Residence
Hall Association (RHA).
“At the end of this experience,
I’ll have something I can present to
Housing and Residence Life, but also
something that I can take back to
RHA to improve the organization’s
communications,” Luis says.
Luis’ primary task was immense:
research every Association of American
Universities (AAU) institution to find
out how other housing operations or
residence life groups use technology to
market and communicate to students.
Much of the research was conducted
online—he visited countless websites,
social media profiles, and YouTube
channels—but Luis also corresponded
directly with residence life professionals
at other AAU schools. These e-mail
surveys and phone interviews gave Luis
a deeper understanding of how other
departments approach communications.
The results of his research had
immediate impact. A multimedia

campaign surrounding new student
move in was completely rethought after
Luis surveyed the successes and failures
of other schools’ move in videos. Luis’
input also supported the marketing
team’s creation of an exhaustive FAQ
document for the department’s website
and forthcoming mobile site.
“We had completed an AAU
residence life communications survey
previously, but having a student conduct
the research and focus explicitly on
new technologies allowed us to get a
lot of new value,” says Ian Ragsdale,
Student Communications Coordinator
for Housing and Residence Life.
“His research allowed us to create a
benchmarking document and generated
a long idea list of things that Luis, as
a student, thinks would be good to
implement at Rutgers.”
The immersive nature of the project
has given Luis a rich perspective and
allowed him to make a number of
specific recommendations based on
his assessment. These include: making
the presence of professional and
graduate student Residence Life staff
better known and understood; more
multimedia to relieve first year student
anxiety; more material on sustainability
in the residence halls; and contests that
can support communications while also
engaging students.
Ultimately, Luis hopes that his work
for Housing and Residence Life and
his newfound knowledge can help
bridge the gap between students and
professionals in Housing and Residence
Life. “I want to help students get more
involved in life on campus,” he says.
“Learning how to communicate better
will allow me to do that.”
The department’s marketing team is

Summer 2012

page

13

also counting on Luis’ work to inform
promotional item intended to
and guide projects as another school
generate buzz about key return.
year approaches.
Summer ends for Luis with a
“It has been really valuable to have
study abroad experience in Peru, but
a student be able to come to us and say
he claims that he has already done a
how students would like us to market
lot of traveling this summer—albeit
to them,” says Michael Miragliotta,
virtually.
Assistant Director of Marketing,
“It’s like I took a national tour
Assessment, and University Relations
of colleges,” he says. “It has
for Housing and Residence Life.
been fun seeing cool things that
“No matter how hard we work to
other colleges have to offer—like
understand our students, nothing beats
mountaintop residence halls or
their direct input and guidance on our
dorms on the beach—but at the end
work.”
of the day it has also allowed me to
Recent innovations, such as offering
better appreciate how distinct and
quiet floors and incentivizing
important my undergraduate
key return with foldable
experience at Rutgers has
Luis Fernandez ‘15 helped the marketing
water bottles, were the direct
team research how other schools market and been. The size of Rutgers, the
results of student feedback.
students, the location, all these
communicate to their student residents.
Luis and other members
have given me the opportunities
of RHA are also regularly asked to review new
I was looking for.
student communications initiatives, including an
“This work has made me see that Rutgers is really
advertisement in the Rutgers University Visitor’s
the right place for me.”
Guide, postcards sent to first year and transfer
students, and contests that enhance social media
Ian Ragsdale is the Student Communications
engagement.
Coordinator for Rutgers Housing & Residence Life.
A central feature of the new process was a

The Dreaded A word
A

By Sarah Grun

wise and innovative
man once said, “I’m
always thinking of what’s
wrong with the thing and how it
can be improved.” Much like Walt
Disney, we, as higher education
professionals, should also be thinking
of ways to improve our work. One
way we can intentionally improve
our work is through assessment.
According to one assessment guru,
John Schuh, assessment should be
an “essential element of student
affairs practice.” Ideally, assessment
demonstrates how we, as Student
Affairs professionals, contribute
to the lives, learning, growth, and
development of our students. But
assessment is often considered the
page

14

dreaded “A” word. It is a process
that some of us in student affairs
avoid.
Recently, one Rutgers assessment
advocate, Krista Kohlmann, shared
with me her take on assessment,
why it is important to the work she
does, and the value of starting small.
Krista is a Program Coordinator
within Student Life-Student
Involvement. She has incorporated
learning outcomes and assessment of
those outcomes into her role advising
the Student Volunteer Council at
Rutgers, a 15 member student group
that plans community service events
for the Rutgers Community. As part
of advising the students, she wanted
to create learning outcomes from

Assessment in Action

scratch to specifically gear training
activities to improve their learning
over the course of the academic year.
To do this, she spent a few weeks
over the summer reading literature
on service learning, including
Council for the Advancement of
Standards in Higher Education (CAS)
for Service Learning, Community
Service Attitudes Scale, and various
other sources. She then defined her
own “areas of competence” based on
the research, each of which had 3-5
learning objectives, and she created
a pre-test to give to the students
prior to their August training.
Throughout the year, Krista made
an intentional effort to include the
areas of competence and the learning

objectives in her on-going training
sessions, and administered a post-test
at the end of the year. After analyzing
the results and compiling a report for
the department, Krista has already
started to use the data to inform her
work in planning next year’s training
for her students.
When asked about assessment
Krista said, “Assessment is not
something to do once and then put
aside for the rest of the year. This
is challenging for people who have
multiple responsibilities and do not
have the time and energy…that is
why I specifically started on a small
scale to keep track of the learning
of my students so that I could be

of her student affairs background,
nor is it part of her job description,
Krista finds it to be “extremely
informative and guides the work
that [she does] to make it more
meaningful.”
Innovation, intentional change,
and improvements take time and
effort. They require us to assess our
current practices and look forward
to better programs and initiatives
for our students. However, it is
not enough to say that we do
assessments or evaluations. Doing
them is really just one step in the
assessment process. To be effective,
we must learn from assessment,
integrate it into everything we do,

intentional and doing assessment are
essential. We, like Walt Disney and
Krista Kohlmann, should embrace
assessment and in doing so become
better higher education professionals
and people.
Sarah Grun is a 2012 graduate of the
Rutgers Ed.M. College Student Affairs
Program.

Assessment is not something to do
once and then put aside for the rest
of the year.
-Krista Kohlmann
(Pictured Right)

thorough and intentional about their
learning.” Krista also said that the
assessment process keeps her on her
toes, keeps her feeling invigorated,
and keeps her constantly working
harder at her job. Her work with the
Student Volunteer Council is one
small effort within the division of
Student Affairs to assess and validate
the work we are doing to improve
the services and programs we offer
here at Rutgers.
While assessment can appear to
be complex, time consuming, labor
intensive, and/or overwhelming, it is
possible to start small. For example,
a unit may begin with an assessment
to measure participation, satisfaction,
cost effectiveness, and/or the needs
of current students. For units to claim
that they contribute positively to
students’ lives, there must be data to
support those claims. But, you don’t
need to have a formal background
in assessment to get started. Just ask
Krista! While assessment is not part

and make it continuous. It must
become a mindset and we must
remember to enact intentional
changes. Much of Walt Disney’s
success came from being a creative
innovator who was not afraid to
dream and improve upon his own
work. He also continuously assessed
his products and parks saying,
“It’s something that will never be
finished. Something that I can keep
developing…and adding to.” Disney
understood that there is always room
for improvement, but that being
Rutgers University does not discriminate on the basis of race, color, national origin, sex, sexual
orientation, gender identity or expression, disability, age or any other category covered by law in its
programs, activities, or employment matters. The following people have been designated to handle
inquiries regarding the non-discrimination policies:
Judy Ryan, Title IX Coordinator for Students &
ADA/Section 504 Compliance Officer
Office of the Vice President for Student Affairs
83 Somerset Street, Suite 101, CAC
p. 848-932-8576
ryan@oldqueens.rutgers.edu

For further information on notice of non-discrimination, visit http://wdcrobcolp01.ed.gov/CFAPPS/
OCR/contactus.cfm for the address and phone number of the Office for Civil Rights that serves your
area, or call 1-800-421-3481.

Summer 2012

page
page

15

All assessment is a perpetual work in
progress.
-Linda Suske

Without deviation from the norm,
progress is not possible.
- Frank Zappa
Self-assessment is the first step to
all assessment.
-Unknown

USA Today has come
out with a new survey.
Apparently, three out
of four people make up
75% of the population.
- David Letterman

Experience is a hard teacher because she gives the
test first, the lesson afterwards.
-Vernon Law