Monthly Archives: January 2013

One clear trend among college students during the past several decades is the increasing proportion of students who maintain a job while attending school. At Augustana, more than half of our students work on campus, while many more hold jobs off campus. Typically, this phenomenon has been cast as a detriment to the college student experience since – as the argument goes – the obligations of work take away from the time that students might spend involved in co-curricular activities or studying for their courses. I have sometimes heard folks talk about the ideal student employment as a position where the student can do their homework while sitting at a desk. However, I’d like to suggest that work – especially if it is conceived as an educational experience – can be powerfully beneficial to our students’ development.

A few weeks ago I met with our Student Affairs senior staff to talk about ways that we can use our student data to support their work. Soon our conversation turned to the possible educational impact of the Community Adviser (CA) position on the students who hold these jobs. It’s time-intensive work that can sometimes be especially challenging when sorting through the whims and wiles of first year students. And even though this position might seem to be a hybrid of co-curricular involvement and student employment, the requirements of the position obligate CAs to forego other opportunities on and off-campus. So we thought it would be useful to test whether or not students who hold CA positions gain some unique educational benefit from the experience.

We chose to compare responses of CAs and non-CAs on one question from the senior survey that asks students to respond to the statement, “My co-curricular involvement helped me develop a better understanding of my leadership skills.” The response options ranged from “strongly disagree” (1) to “strongly agree” (5). The CA’s average response score was a 4.63, while the non-CA’s average response was a 4.26. Statistically, this difference proved to be significant despite the small number of CAs (20) out of the total number of responses (511). This suggests that there might indeed be something about a CA position that provides a unique educational benefit for those students.

While this specific finding might not qualify as the most rigorous quantitative analysis, it replicates other research on the educational benefits of student employment. After examining the impact of work across the 2006 cohort of the Wabash National Study, my colleagues and I found that students who worked made gains on several aspects of leadership skills that non-working students did not (you can read the full study here). Furthermore, the more hours per week that students worked, the larger the educational gain. This held true even after we accounted for students’ other co-curricular involvement.

Now I’m not suggesting that co-curricular involvement is somehow frivolous. There are lots of powerful educational benefits that can come from involvement in a variety of activities. But these findings suggest that maybe work shouldn’t be considered a detriment to the student experience. In fact, I would suggest that each of us who oversee student workers have an opportunity to uniquely influence their development in important ways. We only miss that opportunity if we don’t conceive of the employment opportunity as a learning experience. In the same way that we would like to develop our students as autonomous learners, we should hope to develop our student employees as autonomous workers. That means giving them more than a simple checklist of things to do and instead, asking them to help solve problems and contribute to the quality of the working environment.

So I hope that you will take the time to think about your student workers as students, and see your role in overseeing their work as an educational one.

While we consider a variety of measures to assess our educational effectiveness, we focus on our retention rate (the proportion of full-time first year students who return for a second year) for some pretty crucial reasons. First, it’s a legitimate proxy for the quality of our educational and socially-inclusive environment. Second, as a tuition-dependent institution every student we lose represents lost revenue; and there is real truth to the old adage that it costs more to recruit students than it does to retain them. So every year we calculate our retention rate, hold it up next to the last five or ten years-worth of numbers and ask ourselves:

Did we do a good job of retaining students?

Most of the time, we end up telling ourselves that our retention rate falls somewhere between “decent” and “pretty good” – especially considering all of the things we can’t control. But this conversation always leads us to the next question; one that is substantially more difficult to answer:

What should our retention rate be?

And that is where people in charge start to daydream and folks in the trenches start to cringe. Because it’s all too common for a small group of folks – or even one folk – to arbitrarily decide on the institution’s goal for 1st-to-2nd year retention without any sense of whether or not that number is a reasonable goal. And there’s nothing more corrosive to an educational organization’s long-term quality than assigning an unrealistic goal to the people you depend on to accomplish it. So over the last few months, I’ve been wondering how we could get closer to figuring out what Augustana’s ideal retention rate should be. I don’t know if I have an answer yet – or if there really is a right answer – but I’d like to share some numbers and consider their implications.

Since research on retention suggests that a primary predictor of student success is a student’s incoming academic ability or preparation, it seems reasonable to use our students’ ACT score as a starting point to test whether or not we could realistically expect to improve our retention rate. If most of the students that we lose are also those who enter with low ACT scores, it suggests that the students we lose depart because they are academically unprepared and it’s therefore more likely that we’re already pretty close to our optimum retention rate. However, if most of the students we lose enter with ACT scores comparable to our average freshman ACT score, then it’s likely that we still have room to improve. And if this latter possibility proves to be so, we could consider a few additional factors and come closer to identifying a “ceiling” retention rate from which we could begin to choose a plausible goal.

To begin this process, we took the two most recent cohorts for which we can calculated retention rates (2010 and 2011) and broke down the students who departed before the beginning of their second year by incoming ACT scores. The table below shows the number of students in each of three different categories – the bottom quartile (<22), the middle 50% (22-28), and the top quartile (>28) – that departed before the second year.

cohort

<22 ACT

22 – 28 ACT

>28 ACT

2010

28

54

13

2011

17

72

15

Clearly, in both of these cohorts the majority of the students who left entered with ACT scores in the middle 50% rather than the bottom quarter. Thus, to the degree that ACT score is a proxy for pre-college academic preparation, it appears that there might be some room for us to realistically improve our 1st-to-2nd year retention rate.

However, ACT score doesn’t necessarily reflect the degree to which a student has the personality traits and personal habits (persistence, time management, motivation, etc.) to succeed in college. And there are plenty of students who enter with low ACT scores and thrive at Augustana. So another way to explore this data is to consider the number of students who left in good academic standing. Even though good academic standing at Augustana is a 2.0, in an effort to be conservative in this analysis, I set the bar at a GPA of 2.5.

From the 2010 cohort, 48 of the students who left departed with a GPA above a 2.5. From the 2011 cohort, 58 students fit into this category. Again, both of these numbers suggest some degree of opportunity for improvement. I emphasize caution here because there are many reasons why students depart that are beyond our control (health issues, financial exigency, or family emergencies). In addition, some students leave for non-academic reasons that aren’t accounted for in this rudimentary analysis. So we would be wise to estimate a number substantially below the 48 or 58 students noted above.

Where does that leave us? Well, I would suggest that a reasonable starting point would be to build out from the 2010 cohort. As it stands, our retention rate with that group was 87.6% – the highest on record. If we assume that, with some combination of improved programming , advising, and student support, half of those 48 students could have been retained, that means that we could estimate an additional 24 students – or an increase of about 3 percentage points in our retention rate. That would put us at an optimum retention rate – a best possible scenario – of between 90% and 91%.

How does that compare to colleges like to us? A 90% retention rate would be significantly higher than colleges like Augustana that enroll a similarly student profile. What kind of financial investment would this require? Although that is an even more difficult question to answer, the comprehensive effort necessary to improve our relatively strong retention rate would not be free and would likely require some tradeoffs.

Two final thoughts stick out in my mind. First, while we might have some room to improve, I’d suggest that the we aren’t that far away from our optimum rate. Second, since there are as many moving parts in this equation as there are students at risk of departure, effective change may result from subtle shifts in institutional culture just as much as it might be influenced by a new program or policy.

So can we improve our average retention rate? Probably. Will it be easy? Probably not. Is it the right thing to do? Of course. But we had better not assume that we will see a surge in revenue even if we are successful.

Before the holiday break, I described the evidence from our overall IDEA scores that our students’ Progress on Relevant Objectives (PRO) scores had increased substantively in the past year. It is clear from looking at our data that this didn’t happen by accident and I hope you have taken a moment or two to take pride in your colleagues. Admittedly, it is gratifying to see that all of the effort we have put toward maximizing our use of the new IDEA course feedback forms pay off. So in the spirit of that effort, I want to highlight one other piece of data from our most recent overall report – the low proportion of courses that selected “Developing Creative Capacities” as an essential or important learning objective – and to advocate for more emphasis on that objective.

Of the 12 different learning objectives on the IDEA faculty forms, “Developing Creative Capacities” was selected by only 16% of the courses offered during the fall term – the least common selection (by comparison, 69% of courses indicated “gaining factual knowledge” as an essential or important learning objective). As you might expect, “developing creative capacities” was chosen almost exclusively by fine arts courses, seemingly reflecting a traditional conception of creative capacities as something reserved for artistic expression.

Yet, as a liberal arts college, it seems that “developing creative capacities” should represent a central element of our educational goals and the culmination of a liberals arts education. The parenthetical description of “creative capacities” in that objective includes “writing,” “inventing,” and “designing.” Of course, these skills transcend any specific discipline. Every time a student tries to make an argument with language, portray a concept visually, solve a problem that doesn’t have a singular solution, or articulate the implications of multiple sources of information on a particular point, their ability to do so hinges on these skills.

Moreover, in the updated version Bloom’s Taxonomy, “creating” is the highest cognitive domain. Not unlike synthesizing, creating requires each of the skills listed in the preceding levels of the taxonomy (remembering, understanding, applying, analyzing, and evaluating). It strikes me that this broadened definition of creating could apply to virtually all senior inquiry projects or other student work expected of a culminating experience. For a more detailed discussion of creating as a higher-order skill, I’d suggest the IDEA paper that examines Objective #6.

So how do we infuse “developing creative capacities” more fully into our students’ educational experience? I regularly hear faculty talk about the difficulty that many students exhibit when trying to synthesize disparate ideas and create new knowledge. It’s complicated work, and I’ll bet that if we were to look back on even the best of our own undergraduate work, we would likely cringe in most cases at what we might have thought at the time was the cutting edge of genius. Thankfully, this objective doesn’t say, “Mastering Creative Capacities.” This learning outcome is developmental and will likely be something that most students miss at least as often as they hit. But three ideas come to mind that I’d like to propose for your consideration . . .

Students need practice. This starts with simple experiences connecting ideas and deriving insights from those connections. Students will surely be less capable of successfully wielding this key skill when it is needed if they haven’t explicitly been asked to develop it through previous courses and experiences.

Students won’t take risks if they don’t trust those who ask them to do it. Developing creative capacities requires learning from all manner of failure. Students won’t take the kinds of risk necessary to make real progress if there isn’t space for them to fall down and get back up – and a professor who will help them to their feet.

Eventually, you just have to jump. If nothing else, we are experts at paralysis by analysis. Although there is always a critical mass of information or content knowledge that students must know before they can begin to effectively connect ideas or form new ones, we sometimes get caught trying to cover more material at the expense of developing thinking skills in students. Often, it is through trying to integrate and connect ideas without having all of the pieces that teaches the importance of seeking new knowledge and the awareness that there might be details critical to the development of an idea that we don’t yet know.

As you look at the role of your courses in the collective scheme of our students’ growth, I hope you’ll consider the possibility of adding this learning objective. You may find that you are already doing many of the things in your course that make this happen. You may find that you need to take a few risks yourself in the design of your course. Whatever you decide, I hope you will consider the ways that you help students develop creative capacities as complex, higher-order thinking skills. For our students to succeed in the world they will inherit, I would suggest that our collective future depends on the degree to which we develop their creative capacities to solve problems that we have not yet even seen.

Welcome back to the second half of winter term! As nice as it is to walk across campus in the quiet calm of a fresh new year (ignoring the giant pounding on top of the library for the moment), it’s a comfort to see faculty and students bustling between buildings again and feel the energy of the college reignited by everyone’s return.

Over the last several weeks, I’ve been trying to read the various higher ed opinionators’ perspectives on MOOCs (Massive Open Online Courses) and the implications they foresee for colleges like Augustana. Based on what I’ve read so far, we are either going to 1) thrive without having to change a thing, 2) shrivel up and die a horrible death sometime before the end of the decade, or 3) see lots of changes that will balance each other out and leave us somewhere in the middle. In other words – no one has a clue. But this hasn’t stopped many a self-appointed Nostradami (Nostradamuses?) from rattling off a slew of statistics to make their case: the increasing number of students taking online courses, the number of schools offering online courses, the hundreds of thousands of people who sign up for MOOCs, the shifting demographics of college students, blah blah blah. After all, as these prognosticators imply, historical trends predict the future.

Except when they don’t. A recent NYT article, Sure, Big Data Is Great, But So Is Intuition, highlights the fundamental weakness in thinking that a massive collection of data gathered from individual behaviors (web-browsing, GPS tracking, social network messaging, etc.) inevitably holds the key to a brighter future. As the article puts it, “The problem is that a math model, like a metaphor, is a simplification. This type of modeling came out of the sciences, where the behavior of particles in a fluid, for example, is predictable according the laws of physics.” The article goes on to point out the implications of abiding by this false presumption, such as the catastrophic failure of financial modeling to predict the world-wide economic collapse of 2008. I particularly like the way that the article summarizes this cautionary message. “Listening to the data is important, they [experts interviewed for the article] say, but so is experience and intuition. After all, what is intuition at its best but large amounts of data of all kinds filtered through a human brain rather than a math model?”

This is where experience and intuition intersect with my particular interest in improvisation. When done well, improvisation is not merely random actions. Instead, good improvisation occurs when the timely distillation of experience and observation coalesces through intuition to emerge in an action that both resolves a dilemma and introduces opportunity. Improvisation is the way that we discover a new twist in our teaching that magically “just seemed to work.” Those moments aren’t about luck; they materialize when experience meets intuition meets trust meets action. Only after reflecting on what happened are we able to figure out the “why” and the “how” in order to replicate the new innovation onto which we have stumbled. Meanwhile, back in the moment, it feels like we are just “in a zone.”

Of course, improvisation is no more a guarantee of perfection than predictive modeling. That is because the belief that one can somehow achieve perfection in educating is just as flawed as the fallacy of predictive modeling. Statisticians are taught to precede findings with the phrase “all else remaining constant . . . ” But in education, that has always been the supremely ironic problem. Nothing remains constant. So situating evidence of a statistically significant finding within the the real and gnarly world of teaching and learning requires sophisticated thinking borne of extensive experience and keen intuition.

Effective improvising emerges when we are open to its possibilities – individually and collectively. It’s just a matter of letting our experience morph into intuition in a context of trust that spurs us to act. Just because big data isn’t the solution that some claim it to be doesn’t mean that we batten down the hatches, pretend that MOOCs and every other innovation in educational technology don’t exist, and keep doing what we’ve always done (only better, faster, smarter, more, more, more . . . ). Effective improvising is always preceded by intuition that is informed by some sort of data analysis. When asked why they did what they did, successful improvisers can often explain in detail the thought processes that spurred them to take a particular action or utter a particular line. In the same way, we know a lot about how our students learn and what seems to work well in extending their learning. Given that information, I believe that we have the all of the experience and knowledge to improvise successfully. We just need to flip the switch (“Lights, Action, Improv!”).

Early in the spring term, I’ll host a Friday Conversation where I’ll teach some ways to apply the principles of improvisation to our work. Some of you may remember that I did a similar session last year – although you may have repressed that memory if you were asked to volunteer for one of the improv sketches.

In the mean time, I hope you’ll open yourself up to the potential of improvisation. Enjoy your return to the daily routine. It’s good to have you back.