In which a veteran of cultural studies seminars in the 1990's moves into academic administration and finds himself a married suburban father of two. Foucault, plus lawn care. Comments are welcome. Comments for general readership can be posted directly after the blog entry. For private comments, I can be reached at deandad at gmail dot com. The opinions expressed here are my own (or those of commenters), and not those of my (unnamed) employer.

Wednesday, November 23, 2005

Ask the Administrator: How to Measure Advisement?

Bardiac, true to form, asked a great question. If performing academic advisement is part of a professor’s job description (which it is, at my cc), how do we measure whether they do it well or not?

Some measures are fairly easy, if indirect. Does the professor usually show up for office hours? Do we ever see students there? Does the professor show up on in-person registration days? Do we ever get that professor’s signature on change-of-major forms or course-substitution forms?

The problem with these measures, of course, is that they’re vague at best. It’s possible for office hours to consist of social chatter, rather than real advisement. It’s possible that advisement can be erroneous, causing more problems than it solves. Change-of-major and course-sub forms could show active advisement, earlier mistakes, or even new mistakes.

(We also have a group of faculty who get stipends for showing up at in-person advisement sessions during the off-contract times of year. Presence in that group is generally smiled upon, although one could certainly argue that extra pay is its own reward.)

These measures worked tolerably well when advisement and registration were entirely in-person. Now that registration is increasingly on the web, though, measuring advisement is trickier. I certainly don’t want to start snooping through faculty email accounts to see how much they’re helping students that way – the issues there are too many, and too ugly. But without some sort of monitoring, I have no way of knowing which professors are actually carrying the load, and which free-riding. (Confessions of free-riding are few and far between.)

Failing to monitor, effectively, leads to failing to reward. Failing to reward, over time, leads to failure to bother in the first place. I’m concerned that we’re kind of running on historical momentum, force of habit, from the days when everything was in person. That’s fine, for now, but it’s probably not sustainable.

Does anyone out there have a system that effectively rewards faculty for active, distance advisement? How does it work?

I know you don't have grads at your school, but advice on how to advise graduate students would also be welcome. It seems like that's where the success of the student is even more of a reflection on a professor's competence as a faculty member.

Email has indeed added a new dimension to advisement. Even our on-campus students are only in academic buildings before and after class. And, because of budget cuts, it's harder and harder for students to enroll much after pre-enrollment times.

Then there's ongoing advisement throughout the semester. I do a great deal more advisement since the advent of email and also have to occasionally explain that we do get breaks from that, too. :-)

This is a great topic to discuss as the traditional model of "in office" hours doesn't work anymore. However, that can be a good thing for those of us who prefer not to measure productively simply by who can spend the most hours in the office when job demands vary by time of year and by position (ie I'm out in the schools a great deal with student teachers and grant teachers).

How about asking students when they register for classes to fill out a brief on-line evaluation form that asks questions like:

"What professors have provided helpful advice to you in choosing your classes this semester?"

"If your advisor wanted feedback to help him/her become more helpful to advisees in the future, what would you tell him/her?"

"What questions do you still feel uncertain about?"

"What was the most useful advice your advisor gave you?"

It would also be useful to ask students to fill out this questionnaire again when they log on to find out their grades at the end of the semester. (Kind of retrospective reality check.)

And you could also ask students to respond to the questionnaire again years down the road when they log on to request a transcript to be sent when they are transferring, applying to graduate or professional school, need one for employment, etc.

This isn't just about rewarding evaluation--it's also about giving professors the information they need to improve their advising.

I agree that a questionnaire would be a good idea--perhaps every year, perhaps on graduation (although the problem with questionnaires to graduating seniors is that they're not likely to get returned unless the students are malcontents).

Re. email/distance advising, surely an easy way to do it would be to ask faculty to cc/forward those emails to whoever is in charge of giving them 'credit' for advising? Of course, the down side is, more work for whoever it is reading all the emails.

Or assessment could also be included as part of a year-end review, e.g., having faculty print out representative email exchanges, solicit and provide student letters re. advising, something like that.

Finally, surely gathering certain statistics could help; are advisors assigned to particular students? If so, one could do "outcomes assessment" (ha!) on the students--e.g., do Dr. X's students, the ones whose last names end in A-G, graduate in time? Do they have problems fulfilling requirements in a timely fashion?

In terms of checking the academic outcomes of certain advisors, make sure to compensate for differential students' backgrounds. Even something as theoretically random as assignation by first letter of last name can lead to varying cultural and class origins.

Still, sounds like the kind of thing a grad student with the right data and a good statistics package could make quick work of.

"surely an easy way to do it would be to ask faculty to cc/forward those emails to whoever is in charge of giving them 'credit' for advising? Of course, the down side is, more work for whoever it is reading all the emails."

I'm not sure I follow this, would it be so they can evaluate the content of the email? or your response to the student? or just the fact that the emails exist?

It seems like another downside would be the privacy of the student involved, and of the advisor too. I mean I guess all emails to professors are property of the school anyway, but I feel like I'd want to warn students that anything they write to me would also be seen by (whoever). Maybe it would affect what I wrote, as well. Maybe that wouldn't be a bad thing? I don't know.

I guess I'm a little wary of sharing the contents of emails between two people to a third party..

Fair enough, but in terms of documentation, I suppose it wouldn't be that difficult to strip the email/identifying info of the student, so that the faculty could demonstrate that they're doing the advising, possibly demonstrate the kind of advice they're giving (e.g., do they seem aware of university policies, etc.).

I'd say one step at a time, why not start by asking faculty to report how many student advising requests (and of what nature -- easy registration questions? program questions? career questions?) and the format of those (office hours visit; request for appointment at another time; email exchange), just to see what they say is going on in a non-threatening manner. If you ask from the standpoint of "we'd like to know how students access their advisors" I think you'll get some fairly honest replies.

Also, depending on the students and the advisor, the load may be different. I'm great about answering emails. As a result, NO ONE comes to my office hours. I mean, NO ONE. It is at the point where I wonder why I bother. Huge waste of time. Much easier to arrange a meeting with students as needed at a mutually convenient time.