Library assessment case study: chat reference

From the Spring 2015 Newsletter

As part of our ongoing assessment process, in early December 2014, Library faculty reviewed measures of Sealy Library use that we had been collecting for many years. One figure that jumped out at us was the decrease in the number of reference questions (as measured by questions asked in a typical week) over the years, even as the student body has grown and the use of the physical library (measured by gate count) has varied but not significantly dropped. After considerable discussion and research, we concluded:

This trend was not unique to John Jay but had been widely observed across academic libraries

Students were not more library knowledgeable than in the past and needed our help just as much

We needed to do more to encourage students to ask questions of the reference librarians in person, and

Since students were using the library’s electronic resources more heavily than print, and since 60-70% of the use of our electronic resources tends to be from offcampus, we should try to encourage the use of our new “chat” reference service (see Classified Information, Fall 2014). We needed to go where the students are.

To try to increase chat reference—where students (or faculty!) exchange typed questions and answers in a chat box—we took two steps for the spring 2015 semester: we increased the number of hours we are offering chat by adding an extra hour to 6 pm, and we placed a Library chat widget on as many external library database sites as we could where we thought students might need help.

The results were impressive. Even though library activity of all kinds—including reference questions asked—tends to decrease from fall to spring*, the number of chat reference questions went up—from 120 in the first 9 weeks of the service in the fall (9/8/14-11/7/14) to 141 in the first 9 weeks of the spring semester. Even more interesting, 41% of the chat traffic was now coming from two of the sites where we had newly placed our chat widget (compare charts below).

* (Reasons for the drop in library activity include the general drop in enroll- ment from fall to spring and the reduction in the number of English 101 classes, in all of which students have research assignments.)

EBSCOhost, which figures so prominently in the graph on the lower right and which provides the Academic Search Complete database, was the only database vendor that enabled the placement of our chat widget directly on their search results page. The other vendors merely provided links from their pages to our “Ask-us” page; we have not been able to track whether this resulted in more traffic to our Ask-us page. Our EZproxy login page also became a major source of chats. We thought that our instructions on how to log in to eresources were clear but, clearly, our students still needed help.

Also of great interest is that the places where students are chatting from has shifted (see charts below). In the fall, 22% of the students were actually initiating chat sessions from within the Library. In the spring, this had dropped to 10%. In the Fall, only 29% of the users of chat were contacting us from off-campus. In the spring over 62% were. This is exactly the group we were hoping to reach.

Even more important to us, though, is that even a cursory review of the chat transcripts reveals that we are helping students succeed: “How can I find information on the NYPD gun buyback program?” “Where can I find laws from the former British colonies?” “How can I find articles that aren’t against stop and frisk?“ “I don’t understand what the name of the periodical is” [spelling and punctuation normalized, of course!].

So, yes, students do need our help and yes, if we build an online reference platform they will come. Although they might ask, as one student did “You’re a human?”!