An Evaluation of an Upper-Division, General Education Information Literacy Program

Volume 2, Issue 2, Fall 2008 AN EVALUATION OF AN UPPER-DIVISION, GENERAL EDUCATION INFORMATION LITERACY PROGRAM Elizabeth Spackman Hopkins Brigham Young University Suzanne Julian Brigham Young University ABSTRACT The Advanced Writing library instruction program at Brigham Young University s Harold B. Lee Library (HBLL) is intended to teach junior-level students advanced information literacy and research skills. The university general education curriculum requires students to participate in the program as part of their Advanced Writing course. When anecdotal feedback from librarians and students identified problems with the program, the authors conducted a qualitative evaluation of the program in order to identify problems and possible solutions. The evaluation included a student survey and focus groups with students, librarians, and English faculty. This paper describes the HBLL Advanced Writing instruction program, identifies the problems with the current model, and conveys recommendations from stakeholders for improvement. It also presents observations about the viability of such programs at other institutions. INTRODUCTION AND LITERATURE REVIEW Ideally, library instruction programs build information literacy skills in a sequential and logical way. Finding a way to systematically deliver progressively more advanced research skills to students as they move from their freshman to their senior year is challenging. Librarians struggle to find the right combination of one-shot, course-integrated, and creditbearing information literacy classes to meet the diverse needs of faculty and students. Some libraries incorporate information literacy into the curriculum through multi-year instruction programs, which take various forms. 67 The University of Rhode Island uses a creditbearing course to teach information literacy competencies (MacDonald, Rathemacher, & Burkhardt, 2000). The University of Guelph in Ontario provides a mentoring program that starts with students in their first year and continues as they move through their university experience (Harrison & Rourke, 2006). Other library instruction programs create formal relationships with English composition courses to deliver information literacy instruction. A number of studies describe library instruction programs associated with freshman English (for example: Sult & Mills, 2006; Kennedy, 2005). University of Arizona librarians collaborate with English instructors to ensure that students receive information literacy instruction. Inexperienced English instructors bring their students to the library for a librarian-taught session, whereas experienced English instructors teach students using activities, assignments, and other tools created by librarians (Sult & Mills, 2006). As described by Kennedy, Northern Kentucky librarians teach 50-minute sessions for an advanced writing class called a Research Paper class (2005, 1). A Utah State University study reports on library integration with a sophomore-level English course (Holliday & Fagerheim, 2006). The English 2010 class is considered an intermediate writing class for sophomores and juniors (2006, p. 170). Students receive library instruction in two English classes, English 1010 and 2010; the second class builds on skills learned in the first. The tension between general and subjectspecific information literacy skills creates problems for libraries. Ann Grafstein (2002) says the primary purpose of the library instruction session is to develop critical thinking skills. However, she says, research is conducted differently in the humanities, the social sciences, the physical sciences, and the formal sciences There are essential aspects of the ability to think critically that develop within the context of an understanding of the research concerns in particular disciplines. Each approach described above has benefits and challenges. One perfect solution to the problem of building information literacy skills in a systematic and effective program may not exist. Brigham Young University (BYU), a doctorategranting university with 30,000 FTE students, also struggles to find the perfect combination of instruction activities to build students information literacy skills. BYU uses a variety of one-shot and courseintegrated instruction sessions to teach information literacy skills as students progress through their university experience. Students receive library instruction in their freshmanlevel writing classes, in their junior-level writing classes, and in a number of disciplinebased courses with research assignments. See Table 1 for a complete description of the current library instruction program. The library instruction program began in the 1940s. In 1962, librarians developed a strong program for first-year students in their general education writing classes. In 1981, the program expanded with the addition of library instruction in Advanced Writing, a junior-level general education English class. Library instruction in discipline-based courses has grown over the last 10 years, as subject librarians have worked with their respective departments to meet student needs. BYU S ADVANCED WRITING PROGRAM When the Advanced Writing (AW) program began in 1981, its goals were to build on the skills taught in the freshman sessions and to introduce students to discipline-specific resources. Unfortunately, the practical application of these goals was problematic. Class assignments did not always match the library instruction students received. Students selected research topics unrelated to their majors but received library instruction focused on their majors. In 1994, partly in response to this Hopkins & Julian, An Evaluation Communications in Information Literacy 2(2), Fall 2008 68 Hopkins & Julian, An Evaluation Communications in Information Literacy 2(2), Fall 2008 69 TABLE 1 LIBRARY INSTRUCTION PROGRAM AT BYU Freshman Sophomore or Junior Recommended (only one class required) Sophomore through Senior First-Year Writing (GE Requirement) Engl 115 College Writing and Reading Engl 150/H Writing and Rhetoric Honrs 150 Honors University Writing Phil 150/H Reasoning and Writing E Lang 105 First-Year Writing, International Advanced Written and Oral Communication (GE Requirement) ArtHC 300 Art Historical Methodology, Research, & Intensive Writing Chem 391 Technical Writing Using Chemical Literature *Engl 311/H Writing about the Arts and Humanities *Engl 312/H Persuasive Writing *Engl 313 Expository Writing for Elementary Education Majors *Engl 314 Writing About Literature *Engl 315/H Writing in the Social Sciences *Engl 316/H Technical Writing Germ 340/H Writing about Literature Hist 200+490 The Historian's Craft plus Historical Research and Writing Honrs 300R Advanced Writing IAS 360+361 Int'l Field Study plus Post-Field Analysis and Writing MCom 320/H Writing in Organizational Settings Nurs 320+339 Scholarly Inquiry in Nursing plus Ethics in Nursing: An Advanced Writing Course Phil 300/H Philosophical Writing Phscs 416(A+B) Writing in Physics PlSc 200+capstone seminar Political Inquiry plus one capstone seminar chosen from: PlSc 400, 410, 430, 450, 470, or MESA 495R RMYL 487 Research and Evaluation Discipline-based Research Courses In most departments on campus Number per major varies Two 50-minute instruction sessions Sessions are scheduled to match the class research assignment General librarians and library staff teach basic research skills Class comes as a group during normal class time One 50-minute instruction session Sessions are offered between fourth and eighth week of the semester Subject librarians introduce students to the databases and research techniques appropriate for their major Students individually attend a session outside assigned class time Sessions vary in length and in number based on needs of discipline-specific classes; some are semester-long Sessions offered as needed throughout the semester Subject librarians teach advanced research skills The class may come as a group or outside of class time Oversight through library instruction unit. Oversight through library instruction unit. Maintained by individual subject librarians. *Study focused only on these classes problem, the English composition administrators approached librarians to suggest changing the instruction from discipline-specific to assignment-oriented. The librarians decided to continue the program as originally designed. Since 1994, the program has been relatively static. The most significant modifications have occurred in the general education curriculum documents, which require all AW classes to provide library instruction based on students majors. In addition, the General Education Council has approved discipline-specific courses that receive Advanced Writing credit. Typically, these are research and writing courses taught by faculty in the department instead of by English faculty. Professors work directly with the department s subject librarian to incorporate information literacy instruction. The library instruction unit has no oversight for AW courses outside of the English department. In the current implementation of the Advanced Writing library instruction program, students in an AW English course attend a 50-minute library session based on their major, outside the scheduled class time. They receive points toward their English course grade. Students attend sessions with a variety of skill levels, topics, and types of projects, which include research papers, group projects, or technical manuals. Subject librarians teach the Advanced Writing sessions. Librarians at BYU have faculty status. Their instructional objective is to prepare students to successfully conduct research in their major field of study. Generally, subject librarians introduce students to resources and advanced research skills relevant to their discipline. No formal and few informal relationships exist between subject librarians and the AW English faculty. Librarian and student dissatisfaction with the program, as well as concerns with university accreditation, led to the formal evaluation of the Advanced Writing library instruction program. For years, librarians have suspected that the library sessions were not producing the desired outcome of creating competent and proficient student researchers. In addition, library instructors felt frustrated by low student motivation and mixed student expertise in basic searching skills. Anecdotally, students reported dissatisfaction with the material taught. Some students waited until their senior year to take the AW English class. By that time, many had learned their discipline-specific research skills by trial and error or through another courserelated instruction session. At a recent visit, university accreditors urged the campus to articulate learning outcomes. Based on the resulting accreditation report, library Hopkins & Julian, An Evaluation Communications in Information Literacy 2(2), Fall 2008 70 Standard Performance Indicator 1 2.e. Differentiates between primary and secondary sources, recognizing how their use and importance vary with each discipline 2 1.d. Selects efficient and effective approaches for accessing the information needed from the investigative method or information retrieval system 2 2.e. Implements the search strategy in various information retrieval systems using different user interfaces and search engines, with different command languages, protocols, and search parameters 2 2.f. Implements the search using investigative protocols appropriate to the discipline 2 5.c. Differentiates between the types of sources cited and understands the elements and correct syntax of a citation for a wide range of resources 5 3.a. Selects an appropriate documentation style and uses it consistently to cite sources TABLE 2 ACRL STANDARDS FOR ADVANCED WRITING faculty and staff identified the Association of College and Research Libraries (ACRL) s Information Literacy Standards (Association of College and Research Libraries, 2000) as critical to the Advanced Writing program. Advanced Writing students are expected to have a basic understanding of the research process and performance indicators defined by several of the Standards, as shown in Table 2. The library faculty and staff identified these performance indicators, but wanted a full understanding of stakeholder perceptions of the program before formalizing and assessing learning outcomes. METHODS The study of the Advanced Writing (AW) library instruction program was designed to gather information about the perceptions of students, faculty, and librarians. After reviewing possible assessment tools to determine how to improve the AW program, an online survey and focus groups were selected as the two best methods to gather this type of data. The online survey provided a quick and efficient way to gather information from many students. However, focus groups presented an opportunity to delve into participants attitudes and behaviors. Several studies in recent years have successfully used focus groups to elucidate student and faculty perceptions about information literacy and information literacy programs (Spackman, 2007; Carter, 2002; Morrison, 1997). Focus groups allow participants freedom to express their thoughts and clarify ideas. The BYU study was conducted during the Winter 2007 semester, from January to April. An online survey is sporadically administered to students who complete the Advanced Writing library instruction sessions. For this study, library instructors were asked to solicit survey responses at the end of each instruction session. In addition, students were e-mailed a link to the survey following the completion of all Advanced Writing instruction sessions for the semester. The survey asked students to identify their comfort level with the library, the research skills they learned, and the quality of the instruction received (see Appendix 1 for survey questions). Students and English faculty participated in the focus groups. Facilitators were subject librarians with administrative responsibilities for AW who were able to ask appropriate follow-up questions because of their understanding of the program. Four groups of Advanced Writing students from the Fall 2006 and Winter 2007 semesters were recruited by e-mail; the students who responded first were included in the sessions. One additional group consisted of English faculty teaching AW in the Winter 2007 semester. Focus groups met in small conference rooms in the library and lasted approximately 50 minutes. Sessions were recorded and transcribed. Each student participant received a $10 gift certificate for the BYU Bookstore, and each faculty participant received a $20 gift certificate. Students completed a consent form and a brief questionnaire (see Appendix 2). Participants were encouraged to be open and honest with their comments, and respectful of others responses. The librarian facilitator prompted feedback by encouraging responses from all participants and asking follow-up questions when necessary. A complete list of focus group questions can be found in Appendix 3. Subject librarians also participated in informal focus groups in their department meetings. The meetings were less structured than the focus groups, and began with several broad questions about successes and failures of the AW library instruction program. These were not recorded, but the two librarians conducting the sessions took extensive notes. RESULTS Survey In the Winter 2007 semester, 193 of the 882 Advanced Writing students registered for a library session responded to the online survey, providing a 22% response rate sufficient to highlight general trends and identify potential problems. Students rated their knowledge of five skills both before and after the library Hopkins & Julian, An Evaluation Communications in Information Literacy 2(2), Fall 2008 71 session (see Table 3). The survey used a 5-point Likert scale, with 1 representing no knowledge and 5 representing completely comfortable knowledge. The average response on these items increased from approximately 2 to approximately 4, with the exception of the skill using the library catalog to find books on your topic. Here the students confidence in their skills was 2.95 before the session and 1.81 after the session, implying that the library session negatively impacted student understanding of the library catalog. One possible explanation for the decrease is that most AW library instructors do not include the library catalog in their instruction, because it is covered in the freshman writing library instruction series. FOCUS GROUPS Demographics Thirty-one students participated in the focus groups. Sixteen of these students took Advanced Writing (AW) in Fall 2006, and the remaining 15 had enrolled in AW during Winter 2007. Campus-wide representation was evident, with students majoring in the sciences, social sciences, and humanities (see Table 4 for a Hopkins & Julian, An Evaluation Communications in Information Literacy 2(2), Fall 2008 72 Skill Before Session After Session Narrowing your topic down to a research question/ statement 2.04 4.06 Finding background information about your topic 1.68 4.19 Using online databases to find journal articles on your topic 1.94 4.09 Being able to find electronic or hard copy versions of journal articles in the library 1.83 4.01 Using the library catalog to find books on your topic 2.95 1.81 Likert scale: 1=no knowledge, 5=completely comfortable knowledge TABLE 3 SELF-REPORTED STUDENT KNOWLEDGE ON FIVE LIBRARY SKILLS College Number of Students Engineering and Technology 5 Family, Home, and Social Sciences 7 Fine Arts and Communication 1 Health and Human Performance 5 Humanities 5 International and Area Studies 1 Life Sciences 5 Education 2 TABLE 4 DECLARED MAJORS OF STUDENT FOCUS GROUP PARTICIPANTS, BY COLLEGE breakdown of majors by college). Table 5 shows the demographics of the student focus-group participants. The typical participant was a female senior who had taken First-Year Writing, had not transferred from another institution, and chose her AW session by both major and paper topic. Students who had not taken First-Year Writing received credit either from another institution or through advanced placement tests. Of 14 participating students majoring in the sciences, 10 were seniors and 4 were juniors. Five members of the English faculty, three female and two male, participated in the focus group (see Table 6). Four completed the demographics survey, and they had an average of 6.75 years of teaching experience at BYU. Three of the four had attended an AW session at some point in their career. Although faculty members do not normally attend library sessions with their students, the librarians hoped to learn whether the professors were familiar with the material taught in AW sessions. Three allowed their students to choose their AW session by either major or paper topic, and one recommended that students attend a session determined by major. The third group of research participants included subject librarians in each of the three departments, Science/Maps, Social Sciences, and Humanities. Overall, approximately 75% of the library s 30 subject librarians were surveyed. Table 7 lists the number of sessions the subject librarians taught during the 2006 2007 academic year. Hopkins & Julian, An Evaluation Communications in Information Literacy 2(2), Fall 2008 73 Gender Number Male 9 Female 22 Junior 13 Senior 17 Taken FYW Yes 17 No 13 Transfer Students Yes 4 No 27 AW Session Choice Major 12 Topic 1 Both 13 Other 6 Class TABLE 5 STUDENT DEMOGRAPHICS Gender Number Male 2 Female 3 Average Years Teaching at BYU 6.75 Attended AW library session? Yes 3 No 1 AW Session Choice Major 1 Topic 0 Both 3 Other 0 TABLE 6 FACULTY DEMOGRAPHICS Fall 2006 Winter 2007 Number of Sessions 100 115 Number of Students 602 882 TABLE 7 NUMBER OF AW SESSIONS AND STUDENTS Successes and Challenges The researchers obtained a wide range of feedback from student and faculty focus groups as well as from subject librarian department meetings. Students shared general impressions, discussed their responses to session content, and deliberated the value of the library session. Students, faculty, and librarians shared problems with the program and made a number of excellent suggestions for improvement. Overall, research participants felt that the AW program is generally successful in teaching content. Sessions equip students with the research skills they need to find information for their advanced classes. One student described her experience with the AW library session this way: I can t even tell you it changed everything for me, because in all of my classes at this point in college, all of them I m required to research journal articles and to do critiques on them and write papers, and that s what I do in every class. And so it just saved me so much time and took away so much stress because I feel like I can go there and confidently find what I need to find to support my thesis. I was so grateful that I was able to come. Students reported that they learned a number of search techniques, including both introductory and advanced skills. Students mentioned the following skills, which are considered introductory because they are covered in the library s freshman instruction sessions: How to use Boolean operators How to use truncation How to narrow a search How to find the full text of a journal article from a citation Students stated they learned the following advanced search techniques: Using a thesaurus to identify appropriate search terms Subject searching Using the search history Combining searches Student participants mentioned that having access to librarians expertise and becoming familiar with interlibrary loan and RefWorks (a bibliographic management software program) were particularly helpful aspects of the library session. Students also reported that they had used or would use these skills in other classes, and several predicted they would apply their new knowledge in their careers after their time at the university. Librarians also noted positive aspects of the program. The AW program allowed them to reach students in majors whose faculty members resist library instruction in discipline-specific courses. In addition, librarians expressed satisfaction with the library instruction classrooms in which each student has a computer. Although some participants discussed positive aspects of the AW program, others detailed problems with the current implementation. One student summed up his experience with the AW library session this way: I figured all that out just clicking around on the Web site before I thought that it was a waste of time. Problems with the AW program fell into several categories, which included scheduling, variety of student skill levels, library session content, teaching methods, lack of communication between English faculty and librarians, and discrepancies between major field of study and paper topic. Research participants mentioned three separate scheduling problems. First, students and faculty reported difficulties in finding sessions that fit students daily schedule during the four weeks that AW sessions were offered. Second, participants reported that the timing of the AW sessions is not matched to the research paper assignment in the course: Library sessions are offered from the beginning of the fourth week through the end of the eighth week of the semester, and in many English classes the Hopkins & Julian, An Evaluation Communications in Information Literacy 2(2), Fall 2008 74 research project is assigned later. Students are not familiar with their assignment, and as a result derive less value from the instruction sessions. Librarians felt frustrated when students asked for individual sessions after missing the deadline. Third, the scheduling of the AW class within students college career frustrated students as well as librarians. On the survey, one student wrote the presentation and info was wonderful. The timing was ridiculous; I m graduating in a month and a half and most of my class is too. Like this student, many wait until their last year or even last semester at the university to take an AW English course. Several focus group participants reported that they wished they had attended the Advanced Writing library session earlier in their university studies. Students come to the library AW session with varying skill levels, causing difficulties for library instructors, who then must decide whether to teach introductory skills or cover advanced material. Four scenarios explain the variety of skill levels in Advanced Writing students. First, some students have attended course-integrated library instruction, and some have written research papers that have given them experience with the library s research tools. Second, transfer students and those who tested out of First-Year Writing lack formal experience with library research. Third, students acquire and retain different skills over the course of their time at the university. Some students, for example, were exposed to general search tools as freshmen, but have neglected or forgotten their skills. Finally, pre-college experience varies; an international student in one focus group commented that multicultural students might not have the same experience with computers or libraries as their counterparts. Participants raised two problems related to the library session itself: inconsistency in content taught and teaching methods. Thirty subject librarians teach AW library sessions with no standard curriculum. As mentioned previously, instructors teach a variety of skills, including advanced searching in the library catalog, following a research process, using thesauri, and searching in discipline databases. As students discussed their AW sessions, it became evident they had received dissimilar training from different library instructors. Both librarians and students mentioned teaching methods. Librarians reported that there was too much information to cover and not enough time. Many students commented that the material was covered so quickly that they were unable to absorb it for later use. Students expressed frustration about the lack of hands-on time and interactivity in the library sessions. Research participants revealed the lack of communication between AW English instructors and library instructors. Librarians were not aware of the assignments given to AW students, and some English instructors were not aware of the purpose or content of the library sessions. As a result, some English instructors didn t see the importance of the library experience, as demonstrated by this student s comment: I think one thing for me is my teacher seemed to really downplay the importance of every like the tour and...she s like it s [three points] but just go do it, you re required to do it for me going into it, I was like, ok I have to do this because I have to, not because she even thinks it s important or anyone thinks it s important. It s just I need to do this. So I came into it with the mindset that it was a waste of my time. I had my books out. You know like So it would have taken a lot to even grab my attention to make it useful to me because of how it was presented to me. Students do not value the library session when their instructors do not value the library portion of AW. Subject librarians noted the common discrepancy between major and paper topic. Library sessions are taught based on major, but students are not required to write papers on topics within their majors. Students sometimes Hopkins & Julian, An Evaluation Communications in Information Literacy 2(2), Fall 2008 75 attend a session based on their major and sometimes on their paper topic, so librarians often have a mix of student majors in their sessions. Participating students and English faculty did not raise this point in the focus groups, and might not be aware of the problem. Participant Recommendations Focus group participants and survey respondents made a number of recommendations to address problems in the current Advanced Writing implementation, particularly mentioning issues in scheduling, curriculum, teaching methods, and resources. In addition, librarians specifically urged university advocacy for discipline-specific Advanced Writing courses. Participants recommended a number of modifications regarding the myriad problems with AW scheduling. Focus group subjects proposed that the library offer more sessions at a variety of times, send e-mail reminders to students before their sessions, and communicate with AW instructors to determine the appropriate timing of the sessions during the semester. Students also suggested that the university require the English AW course to be taken earlier in students college careers. Students made several suggestions for the curriculum content and teaching methods in the AW library sessions. They advised library instructors to teach advanced material, instead of repeating the same content from First-Year Writing library instruction sessions. Students proposed that the library offer tiered sessions, at basic, intermediate, and advanced levels, to meet students at their skill level. Regarding teaching methods, focus group participants asked library instructors to slow down, to provide more hands-on time, and to make the session more interactive. Students requested resources to help support their learning. They asked for handouts to be distributed at the session to help them remember the material covered. They also requested an interactive online tutorial to help them learn the skills and resources available in their subject area. Interestingly, most students valued the inperson instruction they received and were not interested in online instruction only; they wanted in-person instruction with online support instead of an entirely online instruction experience. However, a handful of students expressed a preference for online-only instruction. Librarians made a more sweeping recommendation for the Advanced Writing program to engage in top-level advocacy for discipline-specific Advanced Writing courses in every department on campus. Subject librarians felt that existing discipline-specific AW courses (see Table 1 for list) were much more effective than the formal AW instruction program for several reasons. Students write papers related to their majors, so librarians know the skills they teach will be relevant to student work. Students come to the library during class time, with their professor, which underscores the importance of the library session. In some cases, students attend multiple library sessions instead of just one, which allows librarians to cover more material and offer more hands-on time. Librarians work directly with the professors to tailor sessions to the needs of the specific department. Because of the effectiveness of these classes, subject librarians recommended that the library work with the university to incorporate Advanced Writing courses into every department on campus. DISCUSSION Several overall trends relating to different aspects of the Advanced Writing instruction program became evident. In instruction, participants identified increased interaction, less content, and personal connection as essential to success. English faculty attitudes impacted student perceptions of the session, and junior students were much more satisfied with the AW library session than seniors. In addition, mismatches between students declared majors and paper topics led to less effective sessions. Comments from students about the content and structure of the library session provided Hopkins & Julian, An Evaluation Communications in Information Literacy 2(2), Fall 2008 76 important insights about potential teaching improvements. Students asked for increased interaction in the session, in the form of handson time and other activities. This supports literature that has detailed the learning styles of students in the Millennial generation (Oblinger, 2003). Increased interaction will lead to improved student learning. Both students and librarians reported that they felt rushed during the sessions because too much material was forced into 50 minutes. Students were not able to remember the material because so much was covered and they were given no time to practice. In this case, it appears that less is more. Library instructors need to choose the most important things for students to learn, and cut out repetitive or peripheral material. In particular, library instructors should not repeat material covered in the freshman sessions, like truncation, Boolean operators, basic searching in the library catalog, and using the journal finder. Students did not talk specifically about what they wanted to learn, but they did request advanced material, which varies by discipline. For example, in the life sciences, students need to understand citation data; in history, advanced catalog searching is important. In psychology, using database thesauri is key. A particularly interesting finding is that most students did not want to lose the personal connection with the library. They desired online tools and resources to help support their learning, but they were also interested in meeting their librarian face-to-face in an inperson session. Several students specifically mentioned becoming acquainted with their subject librarian as an important benefit of the AW session. This finding is particularly salient in today s digital age, with the ever-increasing availability of online resources and information. The attitude of English faculty toward the AW session impacted student perceptions of the session. One student reported that his teacher s negative view of the session led him to believe that the session was going to be a waste of time. Although some studies have examined faculty attitudes towards information literacy and library instruction (for example, McGuinness, 2006; Gullikson, 2006), few have studied the effect of faculty attitudes about a library instruction session on student attitudes toward the session. This subject deserves further follow-up, both at institutions and in research studies. Class standing largely determined student satisfaction with the AW session. Students who enroll as juniors learn major research skills that they can use for their remaining time at the university, and generally those students reported more satisfaction with the library session. Students who took AW as seniors derived less value from the library session because they had almost completed their university studies. Students are encouraged, but not required, to take Advanced Writing as juniors. Until this issue is resolved at the university level, seniors will continue to regret that they did not take the AW library session earlier. Discrepancies between students majors and paper topics lead to less relevant, and therefore less successful, sessions. When students select a topic unrelated to their major, but attend a session for their major, they do not learn about library resources for their papers and are unable to immediately apply discipline-specific information. If students attend the session related to their paper topic, they get help that is immediately applicable, but they miss out on material related to their majors that they might not get elsewhere. The common mismatch between major and paper topic is the most challenging problem with AW library sessions. It relates directly to the intended purpose of the Advanced Writing program, which librarians decided in 1994: that students receive major-based library instruction. This goal is not achieved for every student, and the effectiveness of the session decreases. In some cases, the English curriculum is only loosely tied to the students declared major, and consequently, the English faculty place little emphasis on research skills within the students majors. In addition, students are free to choose Hopkins & Julian, An Evaluation Communications in Information Literacy 2(2), Fall 2008 77 from a wide range of topics and assignments. Variations in the traditional research paper model, such as the technical manual and group paper, do not always lend themselves to topics in the students major field of study. Since students and English faculty did not mention this problem, a dialogue with the English department is needed. FUTURE DIRECTIONS The researchers felt that the AW program at BYU is broken, and that much can be done to increase its effectiveness. Librarians have limited influence on university curriculum decisions, the timing of student enrollment in an Advanced Writing class, and the discrepancy between major and paper topic. However, they have full control over some aspects of the program, such as library curriculum, session scheduling, and communication with the English faculty. Since conducting this study, the researchers have made progress on several of these problems, and have formulated a plan for implementing additional improvements. Although librarians have limited influence on the university curriculum, they intend to engage in advocacy for change regarding the AW program. Currently, the Assistant University Librarian for Public Services serves as a nonvoting member on the university s General Education Council, which makes decisions about the Advanced Writing curriculum. The librarians will coordinate with this library representative to advocate for an increased number of departmental AW classes outside of the English department. They will also continue to suggest that the council require students to take the Advanced Writing class by the end of their junior year. According to library administration and English faculty, this proposal has failed in the past; however, new administrators or future groups may be open to the change. In addition, librarians will work with the English faculty to advocate for paper topics that match students majors in order to ensure relevant and meaningful instruction. In the future, librarians might be able to work with the English department to change the AW foundation documents to mandate library instruction based on paper topic instead of discipline. In attempts to make positive, concrete changes, librarians tackled scheduling and teaching methods first. During the Fall 2007 semester, subject librarians were asked to incorporate 15 minutes of hands-on time in their AW sessions so that students could practice research skills. In the Winter 2008 semester, a new scheduling system was implemented and tested. The system allows librarians to schedule sessions based on student availability, instead of guessing what times would be best for students. Next, the librarians considered how to improve communication with the English faculty. During the 2007 2008 academic year, they met with English faculty in three groups: AW program administrators, course coordinators, and English 316 instructors. They presented a brief summary of the assessment and raised several discussion points, including the major/paper mismatch and the negative instructor attitude reported by a focus group student participant. The faculty administrators of the AW program maintained the need for major-based instruction, based on the learning outcomes of the program. On the other hand, AW program coordinators, who teach classes and work with other faculty, recognized the problematic major/paper topic discrepancies but made no concrete suggestions. Both administrators and program coordinators were surprised to hear of the negative attitude reported by the student focus group participant. English 316 instructors expressed their desire that the AW library session prepare students for dealing with information throughout their careers, not simply for their AW course or their major courses. Views were varied, and until further consensus is reached, the library is moving forward with major-based sessions. To maintain communication lines with English faculty, formal liaison relationships will be formed between the library and each of the English Advanced Writing courses. Librarians assigned to each course will act like subject librarians assigned to a campus department, Hopkins & Julian, An Evaluation Communications in Information Literacy 2(2), Fall 2008 78 serving as the main contact with the library and determining the best way for the library to serve the course. Increased communication may lead to differences in program administration; for example, AW students might come to the library as a class instead of individually. The plan for the future involves more significant changes to the library curriculum. AW session content will be standardized to include no more than 20 minutes lecture time, and librarians will write more specific learning outcomes for both the freshman program and the Advanced Writing program. They will create a list of skills students will need before they come to their AW session, and will provide Web tutorials and open research labs for students who need reminders. Subject librarians will teach one or two tools or principles that are particularly important in their discipline, and then students will have at least 30 minutes to complete an assignment tailored to their discipline and related to their paper topic. Librarians and students will be able to work together during the hands-on time to facilitate meaningful student learning. Subject librarians will have the flexibility to waive the AW requirement for students who have previously received course-integrated instruction. As the plan is implemented, a number of assessments may be conducted to determine the success of the new program. These will include measures of student, librarian, and English faculty perceptions as well as performance measures to determine student learning. OBSERVATIONS AND ADVICE Here are some suggestions for others who are considering library instruction in an upperdivision English course: If possible, focus the instruction on the paper assignment instead of the students majors. Avoid repetition in the library sessions; do not turn the upperdivision program into a repeat of the freshman-level program, either intentionally or unintentionally. However, provide remedial help in some form for those who need it. Teach students meaningful, advanced research skills. Create a standard curriculum and write learning outcomes for the program as well as for subject area sessions, if applicable. If students attend sessions outside of their normal class time, consider what you will offer for makeup credit. Decide how your program will integrate with other programs, such as library instruction integrated into discipline-specific courses. As you develop the program, communicate frequently with the English department, teaching faculty, and appropriate decisionmaking bodies on campus, like BYU s General Education Council. Establish a formal relationship between the library instruction unit and significant university stakeholders, such as curriculum councils and English coordinators. Involve library administration to ensure those with more influence are supportive of the plan and clear about their roles in developing your program. Clear program goals from the outset are essential to success. Consider the correct placement of the program in the university curriculum, and the optimal time for students to take the related course. If necessary, advocate for a university requirement mandating that students must take the course by a certain point in their coursework. Maintain communication with the English department as you implement the program and as it runs independently. CONCLUSION A program such as the one at BYU, in which subject-based information literacy instruction is administered through an upper-division generaleducation English course, creates obvious problems. The library is happy to retain a place in the university s General Education curriculum, but librarians have found subject- Hopkins & Julian, An Evaluation Communications in Information Literacy 2(2), Fall 2008 79 based instruction in a general course challenging. Despite the challenges, BYU s librarians are confident that they can improve logistics, increase student learning, and ensure a better experience for all involved. REFERENCES Association of College and Research Libraries. (2000). Information literacy competency standards for higher education. Retrieved March 21, 2008 from http://www.ala.org/ala/mgrps/ divs/acrl/standards/standards.pdf Carter, E. W. (2002). Doing the best you can with what you have : Lessons learned from outcomes assessment. Journal of Academic Librarianship, 28(1), 36 41. Grafstein, A. (2002). A discipline-based approach to information literacy. The Journal of Academic Librarianship, 28(4), 197 204. Gullikson, S. (2006). Faculty perceptions of ACRL s Information Literacy Competency Standards for Higher Education. The Journal of Academic Librarianship, 32(6), 583 592. Harrison, J., & Rourke, L. (2006). The benefits of buy-in: integrating information literacy into each year of an academic program. Reference Services Review, 34, 599 606. Holliday, W., & Fagerheim, B. (2006). Integrating information literacy with a sequenced English composition curriculum. Portal: Libraries and the Academy, 6 (2), 169 184. Kennedy, C. (2005). Teaching information literacy to the advanced writing class in three sessions. Electronic Journal of Academic and Special Librarianship, 6(1 2). Retrieved Oct 25, 2 0 0 7 , f r o m h t t p : / / southernlibrarianship.icaap.org/content/v06n01/ kennedy_c01.htm MacDonald, M. C., Rathemacher, A. J., & Burkhardt, J. M. (2000). Challenges in building an incremental, multi-year information literacy plan. Reference Services Review, 28, 240 247. McGuinness, C. (2006). What faculty think exploring the barriers to information literacy development in undergraduate education. The Journal of Academic Librarianship, 32(6), 573 582. Morrison, H. (1997). Information literacy skills: An exploratory focus group study of student perceptions. Research Strategies, 15, 4 17. Oblinger, D. (2003). Boomers, Gen-Xers, and Millenials: Understanding the new students. Educause Review, 38(4). Retrieved Oct 15, 2007, from http://www.educause.edu/ir/library/ pdf/ERM0342.pdf Spackman, E. (2007). Using focus groups to evaluate an information literacy program in a general biology course. Science and Technology Libraries, 27 (3), 3 28. Sult, L., & Mills, V. (2006). A blended method for integrating information literacy instruction into English composition classes. Reference Services Review, 34, 368 388. Hopkins & Julian, An Evaluation Communications in Information Literacy 2(2), Fall 2008 80 APPENDIX 1 STUDENT SURVEY QUESTIONS Hopkins & Julian, An Evaluation Communications in Information Literacy 2(2), Fall 2008 81 Please answer the following questions about your library SESSION. Strongly Disagree Disagree Neutral Agree Strongly Agree 1. After this presentation, I feel more knowledgeable and comfortable about using the library. ( ) ( ) ( ) ( ) ( ) 2. I see the material covered in the sessions as important to my college career. ( ) ( ) ( ) ( ) ( ) 3. The pacing of the session was appropriate to the material covered. ( ) ( ) ( ) ( ) ( ) 4. I know where to go to get further help when I have questions about doing research and using the library. ( ) ( ) ( ) ( ) ( ) Please answer the following questions about your library INSTRUCTOR. Strongly Disagree Disagree Neutral Agree Strongly Agree 5. The Library Instructor presented the material in an organized, easy-to-follow manner. ( ) ( ) ( ) ( ) ( ) 6. The Library Instructor explained concepts and answered questions in a clear and concise manner. ( ) ( ) ( ) ( ) ( ) 7. The Library Instructor made the research concepts, principles and programs interesting to me. ( ) ( ) ( ) ( ) ( ) 8. I felt like the Library Instructor cared about me individually. ( ) ( ) ( ) ( ) ( ) 9. Overall, how would you rate this presentation? ( ) Excellent ( ) Good ( ) Average ( ) Below Average ( ) Poor 10. Did you take the Library Instruction course as part of a introductory writing course or an advanced writing course? ( ) Introductory Writing Course 11. ( ) Advanced Writing Course OTHER COMMENTS OR SUGGESTIONS about the Library Sessions or Library Instructor? 12. 13. Class Name (ex. Engl 150, Phil 150 etc) Library Instructor s Name: Thank you for taking the time to take this survey and give us valuable feedback. We want the Library Instruction program to be as useful to students as possible and your honest answers will help us do that. Please know that your answers will be completely anonymous. APPENDIX 2 FOCUS GROUP QUESTIONNAIRES Advanced Writing Focus Groups Student Questionnaire Please list the Advanced Writing class you took during Fall 2006: Please answer the following questions. All answers will be confidential. Male_____ Female_____ Class: Freshman_____ Sophomore______ Junior_____ Senior_____ Major: ______________________________ Have you taken a First-Year Writing class here at BYU? Yes_____ No______ If you answered no please explain why not. Are you a transfer student? Yes_____ No_____ If you answered yes , what was your class standing when you arrived at BYU? Freshman_____ Sophomore______ Junior_____ Senior_____ Did you leave BYU for more than one semester and come back? Yes_____ No_____ If you answered yes , how long were you away? Advanced writing students are required to attend a one hour session in the library with a librarian who is a subject specialist. Please answer the following questions based on your experience. Did you attend an Advanced Writing Library Session? Yes______ No______ How did you choose which library session to attend? (please mark one) _____The library session related to my major _____The library session related to my paper topic _____The library session related to both my major and my paper topic _____Other. Please explain: Advanced Writing Focus Groups Student Questionnaire Please list the Advanced Writing class you took during Winter 2007: Please answer the following questions. All answers will be confidential. Male_____ Female_____ Class: Freshman_____ Sophomore______ Junior_____ Senior_____ Major: ______________________________ Have you taken a First-Year Writing class here at BYU? Yes_____ No______ If you answered no please explain why not. Are you a transfer student? Yes_____ No_____ If you answered yes , what was your class standing when you arrived at BYU? Freshman_____ Sophomore______ Junior_____ Senior_____ Did you leave BYU for more than one semester and come back? Yes_____ No_____ If you answered yes , how long were you away? Advanced writing students are required to attend a one hour session in the library with a librarian who is a subject specialist. Please answer the following questions based on your experience. Did you attend an Advanced Writing Library Session? Yes______ No______ How did you choose which library session to attend? (please mark one) _____The library session related to my major _____The library session related to my paper topic _____The library session related to both my major and my paper topic _____Other. Please explain: Advanced Writing Focus Groups Faculty Questionnaire Please answer the following questions. All answers will be confidential. Male_____ Female_____ Status (e.g. faculty, graduate student, etc.): Number of years teaching Advanced Writing at BYU: Have you ever attended an Advanced Writing library session? Yes_____ No_____ How do you recommend to your students the library session they should attend? (please mark one) _____The library session related to my major _____The library session related to my paper topic _____The library session related to both my major and my paper topic _____Other. Please explain Hopkins & Julian, An Evaluation Communications in Information Literacy 2(2), Fall 2008 82 APPENDIX 3 FOCUS GROUP QUESTIONS Focus Group Questions: Current AW Students 1. Have you written your advanced writing research paper yet? If so, please tell us about the advanced writing research paper you have written. Describe the process you went through to find information. 2. What did you learn in the Advanced Writing library session? Prompts (only if necessary): Databases Search techniques Full-text of the articles RefWorks (Did you know prior to Advanced Writing? If yes: from where) 3. Are these skills ones you will use in other classes or in a profession? If so, how? 4. What did you know prior to coming to the session that was repeated in the class? 5. What do wish you had learned in the session that wasn t taught? 6. Is this a valuable class? Should it continue? 7. What suggestions do you have for improving the instruction you received? 8. Are there other ways you could receive this instruction that would be more useful to you? 9. What other library instruction have you received? How did Advanced Writing library instruction fit with the other instruction you received? Focus Group Questions: Fall 2006 AW Students 1. Please tell us about the advanced writing research paper you wrote last semester. Describe the process you went through to find information. 2. What did you learn in the Advanced Writing library session? Prompts (only if necessary): Databases Search techniques Full-text of the articles RefWorks (Did you know prior to Advanced Writing? If yes: from where) 3. Are these skills ones you will or have used in other classes or in a profession? If so, how? 4. What did you know prior to coming to the session that was repeated in the class? 5. What do wish you had learned in the session that wasn t taught? 6. Is this a valuable class? Should it continue? 7. What suggestions do you have for improving the instruction you received? 8. Are there other ways you could receive this instruction that would be more useful to you? 9. What other library instruction have you received? How did Advanced Writing library instruction fit with the other instruction you received? 10. A semester after taking the class, do you have any additional perspective or comments about the library portion of Advanced Writing? Focus Group Questions: AW Instructors 1. Does the AW library session help your students with library research? If so, how? If not, why? 2. What do your students tell you about the AW library session? 3. What do you wish your students would learn in the sessions that isn t currently taught? 4. What suggestions do you have for improving the instruction your students receive? 5. Are there other ways your students could receive this instruction that would be more useful to them? Hopkins & Julian, An Evaluation Communications in Information Literacy 2(2), Fall 2008 83

Click tabs to swap between content that is broken into logical sections.

The Advanced Writing library instruction program at Brigham Young University's Harold B. Lee Library (HBLL) is intended to teach junior-level students advanced information literacy and research skills. The university general education curriculum requires students to participate in the program as part of their Advanced Writing course. When anecdotal feedback from librarians and students identified problems with the program, the authors conducted a qualitative evaluation of the program in order to identify problems and possible solutions. The evaluation included a student survey and focus groups with students, librarians, and English faculty. This paper describes the HBLL Advanced Writing instruction program, identifies the problems with the current model, and conveys recommendations from stakeholders for improvement. It also presents observations about the viability of such programs at other institutions.

Volume 2, Issue 2, Fall 2008 AN EVALUATION OF AN UPPER-DIVISION, GENERAL EDUCATION INFORMATION LITERACY PROGRAM Elizabeth Spackman Hopkins Brigham Young University Suzanne Julian Brigham Young University ABSTRACT The Advanced Writing library instruction program at Brigham Young University s Harold B. Lee Library (HBLL) is intended to teach junior-level students advanced information literacy and research skills. The university general education curriculum requires students to participate in the program as part of their Advanced Writing course. When anecdotal feedback from librarians and students identified problems with the program, the authors conducted a qualitative evaluation of the program in order to identify problems and possible solutions. The evaluation included a student survey and focus groups with students, librarians, and English faculty. This paper describes the HBLL Advanced Writing instruction program, identifies the problems with the current model, and conveys recommendations from stakeholders for improvement. It also presents observations about the viability of such programs at other institutions. INTRODUCTION AND LITERATURE REVIEW Ideally, library instruction programs build information literacy skills in a sequential and logical way. Finding a way to systematically deliver progressively more advanced research skills to students as they move from their freshman to their senior year is challenging. Librarians struggle to find the right combination of one-shot, course-integrated, and creditbearing information literacy classes to meet the diverse needs of faculty and students. Some libraries incorporate information literacy into the curriculum through multi-year instruction programs, which take various forms. 67 The University of Rhode Island uses a creditbearing course to teach information literacy competencies (MacDonald, Rathemacher, & Burkhardt, 2000). The University of Guelph in Ontario provides a mentoring program that starts with students in their first year and continues as they move through their university experience (Harrison & Rourke, 2006). Other library instruction programs create formal relationships with English composition courses to deliver information literacy instruction. A number of studies describe library instruction programs associated with freshman English (for example: Sult & Mills, 2006; Kennedy, 2005). University of Arizona librarians collaborate with English instructors to ensure that students receive information literacy instruction. Inexperienced English instructors bring their students to the library for a librarian-taught session, whereas experienced English instructors teach students using activities, assignments, and other tools created by librarians (Sult & Mills, 2006). As described by Kennedy, Northern Kentucky librarians teach 50-minute sessions for an advanced writing class called a Research Paper class (2005, 1). A Utah State University study reports on library integration with a sophomore-level English course (Holliday & Fagerheim, 2006). The English 2010 class is considered an intermediate writing class for sophomores and juniors (2006, p. 170). Students receive library instruction in two English classes, English 1010 and 2010; the second class builds on skills learned in the first. The tension between general and subjectspecific information literacy skills creates problems for libraries. Ann Grafstein (2002) says the primary purpose of the library instruction session is to develop critical thinking skills. However, she says, research is conducted differently in the humanities, the social sciences, the physical sciences, and the formal sciences There are essential aspects of the ability to think critically that develop within the context of an understanding of the research concerns in particular disciplines. Each approach described above has benefits and challenges. One perfect solution to the problem of building information literacy skills in a systematic and effective program may not exist. Brigham Young University (BYU), a doctorategranting university with 30,000 FTE students, also struggles to find the perfect combination of instruction activities to build students information literacy skills. BYU uses a variety of one-shot and courseintegrated instruction sessions to teach information literacy skills as students progress through their university experience. Students receive library instruction in their freshmanlevel writing classes, in their junior-level writing classes, and in a number of disciplinebased courses with research assignments. See Table 1 for a complete description of the current library instruction program. The library instruction program began in the 1940s. In 1962, librarians developed a strong program for first-year students in their general education writing classes. In 1981, the program expanded with the addition of library instruction in Advanced Writing, a junior-level general education English class. Library instruction in discipline-based courses has grown over the last 10 years, as subject librarians have worked with their respective departments to meet student needs. BYU S ADVANCED WRITING PROGRAM When the Advanced Writing (AW) program began in 1981, its goals were to build on the skills taught in the freshman sessions and to introduce students to discipline-specific resources. Unfortunately, the practical application of these goals was problematic. Class assignments did not always match the library instruction students received. Students selected research topics unrelated to their majors but received library instruction focused on their majors. In 1994, partly in response to this Hopkins & Julian, An Evaluation Communications in Information Literacy 2(2), Fall 2008 68 Hopkins & Julian, An Evaluation Communications in Information Literacy 2(2), Fall 2008 69 TABLE 1 LIBRARY INSTRUCTION PROGRAM AT BYU Freshman Sophomore or Junior Recommended (only one class required) Sophomore through Senior First-Year Writing (GE Requirement) Engl 115 College Writing and Reading Engl 150/H Writing and Rhetoric Honrs 150 Honors University Writing Phil 150/H Reasoning and Writing E Lang 105 First-Year Writing, International Advanced Written and Oral Communication (GE Requirement) ArtHC 300 Art Historical Methodology, Research, & Intensive Writing Chem 391 Technical Writing Using Chemical Literature *Engl 311/H Writing about the Arts and Humanities *Engl 312/H Persuasive Writing *Engl 313 Expository Writing for Elementary Education Majors *Engl 314 Writing About Literature *Engl 315/H Writing in the Social Sciences *Engl 316/H Technical Writing Germ 340/H Writing about Literature Hist 200+490 The Historian's Craft plus Historical Research and Writing Honrs 300R Advanced Writing IAS 360+361 Int'l Field Study plus Post-Field Analysis and Writing MCom 320/H Writing in Organizational Settings Nurs 320+339 Scholarly Inquiry in Nursing plus Ethics in Nursing: An Advanced Writing Course Phil 300/H Philosophical Writing Phscs 416(A+B) Writing in Physics PlSc 200+capstone seminar Political Inquiry plus one capstone seminar chosen from: PlSc 400, 410, 430, 450, 470, or MESA 495R RMYL 487 Research and Evaluation Discipline-based Research Courses In most departments on campus Number per major varies Two 50-minute instruction sessions Sessions are scheduled to match the class research assignment General librarians and library staff teach basic research skills Class comes as a group during normal class time One 50-minute instruction session Sessions are offered between fourth and eighth week of the semester Subject librarians introduce students to the databases and research techniques appropriate for their major Students individually attend a session outside assigned class time Sessions vary in length and in number based on needs of discipline-specific classes; some are semester-long Sessions offered as needed throughout the semester Subject librarians teach advanced research skills The class may come as a group or outside of class time Oversight through library instruction unit. Oversight through library instruction unit. Maintained by individual subject librarians. *Study focused only on these classes problem, the English composition administrators approached librarians to suggest changing the instruction from discipline-specific to assignment-oriented. The librarians decided to continue the program as originally designed. Since 1994, the program has been relatively static. The most significant modifications have occurred in the general education curriculum documents, which require all AW classes to provide library instruction based on students majors. In addition, the General Education Council has approved discipline-specific courses that receive Advanced Writing credit. Typically, these are research and writing courses taught by faculty in the department instead of by English faculty. Professors work directly with the department s subject librarian to incorporate information literacy instruction. The library instruction unit has no oversight for AW courses outside of the English department. In the current implementation of the Advanced Writing library instruction program, students in an AW English course attend a 50-minute library session based on their major, outside the scheduled class time. They receive points toward their English course grade. Students attend sessions with a variety of skill levels, topics, and types of projects, which include research papers, group projects, or technical manuals. Subject librarians teach the Advanced Writing sessions. Librarians at BYU have faculty status. Their instructional objective is to prepare students to successfully conduct research in their major field of study. Generally, subject librarians introduce students to resources and advanced research skills relevant to their discipline. No formal and few informal relationships exist between subject librarians and the AW English faculty. Librarian and student dissatisfaction with the program, as well as concerns with university accreditation, led to the formal evaluation of the Advanced Writing library instruction program. For years, librarians have suspected that the library sessions were not producing the desired outcome of creating competent and proficient student researchers. In addition, library instructors felt frustrated by low student motivation and mixed student expertise in basic searching skills. Anecdotally, students reported dissatisfaction with the material taught. Some students waited until their senior year to take the AW English class. By that time, many had learned their discipline-specific research skills by trial and error or through another courserelated instruction session. At a recent visit, university accreditors urged the campus to articulate learning outcomes. Based on the resulting accreditation report, library Hopkins & Julian, An Evaluation Communications in Information Literacy 2(2), Fall 2008 70 Standard Performance Indicator 1 2.e. Differentiates between primary and secondary sources, recognizing how their use and importance vary with each discipline 2 1.d. Selects efficient and effective approaches for accessing the information needed from the investigative method or information retrieval system 2 2.e. Implements the search strategy in various information retrieval systems using different user interfaces and search engines, with different command languages, protocols, and search parameters 2 2.f. Implements the search using investigative protocols appropriate to the discipline 2 5.c. Differentiates between the types of sources cited and understands the elements and correct syntax of a citation for a wide range of resources 5 3.a. Selects an appropriate documentation style and uses it consistently to cite sources TABLE 2 ACRL STANDARDS FOR ADVANCED WRITING faculty and staff identified the Association of College and Research Libraries (ACRL) s Information Literacy Standards (Association of College and Research Libraries, 2000) as critical to the Advanced Writing program. Advanced Writing students are expected to have a basic understanding of the research process and performance indicators defined by several of the Standards, as shown in Table 2. The library faculty and staff identified these performance indicators, but wanted a full understanding of stakeholder perceptions of the program before formalizing and assessing learning outcomes. METHODS The study of the Advanced Writing (AW) library instruction program was designed to gather information about the perceptions of students, faculty, and librarians. After reviewing possible assessment tools to determine how to improve the AW program, an online survey and focus groups were selected as the two best methods to gather this type of data. The online survey provided a quick and efficient way to gather information from many students. However, focus groups presented an opportunity to delve into participants attitudes and behaviors. Several studies in recent years have successfully used focus groups to elucidate student and faculty perceptions about information literacy and information literacy programs (Spackman, 2007; Carter, 2002; Morrison, 1997). Focus groups allow participants freedom to express their thoughts and clarify ideas. The BYU study was conducted during the Winter 2007 semester, from January to April. An online survey is sporadically administered to students who complete the Advanced Writing library instruction sessions. For this study, library instructors were asked to solicit survey responses at the end of each instruction session. In addition, students were e-mailed a link to the survey following the completion of all Advanced Writing instruction sessions for the semester. The survey asked students to identify their comfort level with the library, the research skills they learned, and the quality of the instruction received (see Appendix 1 for survey questions). Students and English faculty participated in the focus groups. Facilitators were subject librarians with administrative responsibilities for AW who were able to ask appropriate follow-up questions because of their understanding of the program. Four groups of Advanced Writing students from the Fall 2006 and Winter 2007 semesters were recruited by e-mail; the students who responded first were included in the sessions. One additional group consisted of English faculty teaching AW in the Winter 2007 semester. Focus groups met in small conference rooms in the library and lasted approximately 50 minutes. Sessions were recorded and transcribed. Each student participant received a $10 gift certificate for the BYU Bookstore, and each faculty participant received a $20 gift certificate. Students completed a consent form and a brief questionnaire (see Appendix 2). Participants were encouraged to be open and honest with their comments, and respectful of others responses. The librarian facilitator prompted feedback by encouraging responses from all participants and asking follow-up questions when necessary. A complete list of focus group questions can be found in Appendix 3. Subject librarians also participated in informal focus groups in their department meetings. The meetings were less structured than the focus groups, and began with several broad questions about successes and failures of the AW library instruction program. These were not recorded, but the two librarians conducting the sessions took extensive notes. RESULTS Survey In the Winter 2007 semester, 193 of the 882 Advanced Writing students registered for a library session responded to the online survey, providing a 22% response rate sufficient to highlight general trends and identify potential problems. Students rated their knowledge of five skills both before and after the library Hopkins & Julian, An Evaluation Communications in Information Literacy 2(2), Fall 2008 71 session (see Table 3). The survey used a 5-point Likert scale, with 1 representing no knowledge and 5 representing completely comfortable knowledge. The average response on these items increased from approximately 2 to approximately 4, with the exception of the skill using the library catalog to find books on your topic. Here the students confidence in their skills was 2.95 before the session and 1.81 after the session, implying that the library session negatively impacted student understanding of the library catalog. One possible explanation for the decrease is that most AW library instructors do not include the library catalog in their instruction, because it is covered in the freshman writing library instruction series. FOCUS GROUPS Demographics Thirty-one students participated in the focus groups. Sixteen of these students took Advanced Writing (AW) in Fall 2006, and the remaining 15 had enrolled in AW during Winter 2007. Campus-wide representation was evident, with students majoring in the sciences, social sciences, and humanities (see Table 4 for a Hopkins & Julian, An Evaluation Communications in Information Literacy 2(2), Fall 2008 72 Skill Before Session After Session Narrowing your topic down to a research question/ statement 2.04 4.06 Finding background information about your topic 1.68 4.19 Using online databases to find journal articles on your topic 1.94 4.09 Being able to find electronic or hard copy versions of journal articles in the library 1.83 4.01 Using the library catalog to find books on your topic 2.95 1.81 Likert scale: 1=no knowledge, 5=completely comfortable knowledge TABLE 3 SELF-REPORTED STUDENT KNOWLEDGE ON FIVE LIBRARY SKILLS College Number of Students Engineering and Technology 5 Family, Home, and Social Sciences 7 Fine Arts and Communication 1 Health and Human Performance 5 Humanities 5 International and Area Studies 1 Life Sciences 5 Education 2 TABLE 4 DECLARED MAJORS OF STUDENT FOCUS GROUP PARTICIPANTS, BY COLLEGE breakdown of majors by college). Table 5 shows the demographics of the student focus-group participants. The typical participant was a female senior who had taken First-Year Writing, had not transferred from another institution, and chose her AW session by both major and paper topic. Students who had not taken First-Year Writing received credit either from another institution or through advanced placement tests. Of 14 participating students majoring in the sciences, 10 were seniors and 4 were juniors. Five members of the English faculty, three female and two male, participated in the focus group (see Table 6). Four completed the demographics survey, and they had an average of 6.75 years of teaching experience at BYU. Three of the four had attended an AW session at some point in their career. Although faculty members do not normally attend library sessions with their students, the librarians hoped to learn whether the professors were familiar with the material taught in AW sessions. Three allowed their students to choose their AW session by either major or paper topic, and one recommended that students attend a session determined by major. The third group of research participants included subject librarians in each of the three departments, Science/Maps, Social Sciences, and Humanities. Overall, approximately 75% of the library s 30 subject librarians were surveyed. Table 7 lists the number of sessions the subject librarians taught during the 2006 2007 academic year. Hopkins & Julian, An Evaluation Communications in Information Literacy 2(2), Fall 2008 73 Gender Number Male 9 Female 22 Junior 13 Senior 17 Taken FYW Yes 17 No 13 Transfer Students Yes 4 No 27 AW Session Choice Major 12 Topic 1 Both 13 Other 6 Class TABLE 5 STUDENT DEMOGRAPHICS Gender Number Male 2 Female 3 Average Years Teaching at BYU 6.75 Attended AW library session? Yes 3 No 1 AW Session Choice Major 1 Topic 0 Both 3 Other 0 TABLE 6 FACULTY DEMOGRAPHICS Fall 2006 Winter 2007 Number of Sessions 100 115 Number of Students 602 882 TABLE 7 NUMBER OF AW SESSIONS AND STUDENTS Successes and Challenges The researchers obtained a wide range of feedback from student and faculty focus groups as well as from subject librarian department meetings. Students shared general impressions, discussed their responses to session content, and deliberated the value of the library session. Students, faculty, and librarians shared problems with the program and made a number of excellent suggestions for improvement. Overall, research participants felt that the AW program is generally successful in teaching content. Sessions equip students with the research skills they need to find information for their advanced classes. One student described her experience with the AW library session this way: I can t even tell you it changed everything for me, because in all of my classes at this point in college, all of them I m required to research journal articles and to do critiques on them and write papers, and that s what I do in every class. And so it just saved me so much time and took away so much stress because I feel like I can go there and confidently find what I need to find to support my thesis. I was so grateful that I was able to come. Students reported that they learned a number of search techniques, including both introductory and advanced skills. Students mentioned the following skills, which are considered introductory because they are covered in the library s freshman instruction sessions: How to use Boolean operators How to use truncation How to narrow a search How to find the full text of a journal article from a citation Students stated they learned the following advanced search techniques: Using a thesaurus to identify appropriate search terms Subject searching Using the search history Combining searches Student participants mentioned that having access to librarians expertise and becoming familiar with interlibrary loan and RefWorks (a bibliographic management software program) were particularly helpful aspects of the library session. Students also reported that they had used or would use these skills in other classes, and several predicted they would apply their new knowledge in their careers after their time at the university. Librarians also noted positive aspects of the program. The AW program allowed them to reach students in majors whose faculty members resist library instruction in discipline-specific courses. In addition, librarians expressed satisfaction with the library instruction classrooms in which each student has a computer. Although some participants discussed positive aspects of the AW program, others detailed problems with the current implementation. One student summed up his experience with the AW library session this way: I figured all that out just clicking around on the Web site before I thought that it was a waste of time. Problems with the AW program fell into several categories, which included scheduling, variety of student skill levels, library session content, teaching methods, lack of communication between English faculty and librarians, and discrepancies between major field of study and paper topic. Research participants mentioned three separate scheduling problems. First, students and faculty reported difficulties in finding sessions that fit students daily schedule during the four weeks that AW sessions were offered. Second, participants reported that the timing of the AW sessions is not matched to the research paper assignment in the course: Library sessions are offered from the beginning of the fourth week through the end of the eighth week of the semester, and in many English classes the Hopkins & Julian, An Evaluation Communications in Information Literacy 2(2), Fall 2008 74 research project is assigned later. Students are not familiar with their assignment, and as a result derive less value from the instruction sessions. Librarians felt frustrated when students asked for individual sessions after missing the deadline. Third, the scheduling of the AW class within students college career frustrated students as well as librarians. On the survey, one student wrote the presentation and info was wonderful. The timing was ridiculous; I m graduating in a month and a half and most of my class is too. Like this student, many wait until their last year or even last semester at the university to take an AW English course. Several focus group participants reported that they wished they had attended the Advanced Writing library session earlier in their university studies. Students come to the library AW session with varying skill levels, causing difficulties for library instructors, who then must decide whether to teach introductory skills or cover advanced material. Four scenarios explain the variety of skill levels in Advanced Writing students. First, some students have attended course-integrated library instruction, and some have written research papers that have given them experience with the library s research tools. Second, transfer students and those who tested out of First-Year Writing lack formal experience with library research. Third, students acquire and retain different skills over the course of their time at the university. Some students, for example, were exposed to general search tools as freshmen, but have neglected or forgotten their skills. Finally, pre-college experience varies; an international student in one focus group commented that multicultural students might not have the same experience with computers or libraries as their counterparts. Participants raised two problems related to the library session itself: inconsistency in content taught and teaching methods. Thirty subject librarians teach AW library sessions with no standard curriculum. As mentioned previously, instructors teach a variety of skills, including advanced searching in the library catalog, following a research process, using thesauri, and searching in discipline databases. As students discussed their AW sessions, it became evident they had received dissimilar training from different library instructors. Both librarians and students mentioned teaching methods. Librarians reported that there was too much information to cover and not enough time. Many students commented that the material was covered so quickly that they were unable to absorb it for later use. Students expressed frustration about the lack of hands-on time and interactivity in the library sessions. Research participants revealed the lack of communication between AW English instructors and library instructors. Librarians were not aware of the assignments given to AW students, and some English instructors were not aware of the purpose or content of the library sessions. As a result, some English instructors didn t see the importance of the library experience, as demonstrated by this student s comment: I think one thing for me is my teacher seemed to really downplay the importance of every like the tour and...she s like it s [three points] but just go do it, you re required to do it for me going into it, I was like, ok I have to do this because I have to, not because she even thinks it s important or anyone thinks it s important. It s just I need to do this. So I came into it with the mindset that it was a waste of my time. I had my books out. You know like So it would have taken a lot to even grab my attention to make it useful to me because of how it was presented to me. Students do not value the library session when their instructors do not value the library portion of AW. Subject librarians noted the common discrepancy between major and paper topic. Library sessions are taught based on major, but students are not required to write papers on topics within their majors. Students sometimes Hopkins & Julian, An Evaluation Communications in Information Literacy 2(2), Fall 2008 75 attend a session based on their major and sometimes on their paper topic, so librarians often have a mix of student majors in their sessions. Participating students and English faculty did not raise this point in the focus groups, and might not be aware of the problem. Participant Recommendations Focus group participants and survey respondents made a number of recommendations to address problems in the current Advanced Writing implementation, particularly mentioning issues in scheduling, curriculum, teaching methods, and resources. In addition, librarians specifically urged university advocacy for discipline-specific Advanced Writing courses. Participants recommended a number of modifications regarding the myriad problems with AW scheduling. Focus group subjects proposed that the library offer more sessions at a variety of times, send e-mail reminders to students before their sessions, and communicate with AW instructors to determine the appropriate timing of the sessions during the semester. Students also suggested that the university require the English AW course to be taken earlier in students college careers. Students made several suggestions for the curriculum content and teaching methods in the AW library sessions. They advised library instructors to teach advanced material, instead of repeating the same content from First-Year Writing library instruction sessions. Students proposed that the library offer tiered sessions, at basic, intermediate, and advanced levels, to meet students at their skill level. Regarding teaching methods, focus group participants asked library instructors to slow down, to provide more hands-on time, and to make the session more interactive. Students requested resources to help support their learning. They asked for handouts to be distributed at the session to help them remember the material covered. They also requested an interactive online tutorial to help them learn the skills and resources available in their subject area. Interestingly, most students valued the inperson instruction they received and were not interested in online instruction only; they wanted in-person instruction with online support instead of an entirely online instruction experience. However, a handful of students expressed a preference for online-only instruction. Librarians made a more sweeping recommendation for the Advanced Writing program to engage in top-level advocacy for discipline-specific Advanced Writing courses in every department on campus. Subject librarians felt that existing discipline-specific AW courses (see Table 1 for list) were much more effective than the formal AW instruction program for several reasons. Students write papers related to their majors, so librarians know the skills they teach will be relevant to student work. Students come to the library during class time, with their professor, which underscores the importance of the library session. In some cases, students attend multiple library sessions instead of just one, which allows librarians to cover more material and offer more hands-on time. Librarians work directly with the professors to tailor sessions to the needs of the specific department. Because of the effectiveness of these classes, subject librarians recommended that the library work with the university to incorporate Advanced Writing courses into every department on campus. DISCUSSION Several overall trends relating to different aspects of the Advanced Writing instruction program became evident. In instruction, participants identified increased interaction, less content, and personal connection as essential to success. English faculty attitudes impacted student perceptions of the session, and junior students were much more satisfied with the AW library session than seniors. In addition, mismatches between students declared majors and paper topics led to less effective sessions. Comments from students about the content and structure of the library session provided Hopkins & Julian, An Evaluation Communications in Information Literacy 2(2), Fall 2008 76 important insights about potential teaching improvements. Students asked for increased interaction in the session, in the form of handson time and other activities. This supports literature that has detailed the learning styles of students in the Millennial generation (Oblinger, 2003). Increased interaction will lead to improved student learning. Both students and librarians reported that they felt rushed during the sessions because too much material was forced into 50 minutes. Students were not able to remember the material because so much was covered and they were given no time to practice. In this case, it appears that less is more. Library instructors need to choose the most important things for students to learn, and cut out repetitive or peripheral material. In particular, library instructors should not repeat material covered in the freshman sessions, like truncation, Boolean operators, basic searching in the library catalog, and using the journal finder. Students did not talk specifically about what they wanted to learn, but they did request advanced material, which varies by discipline. For example, in the life sciences, students need to understand citation data; in history, advanced catalog searching is important. In psychology, using database thesauri is key. A particularly interesting finding is that most students did not want to lose the personal connection with the library. They desired online tools and resources to help support their learning, but they were also interested in meeting their librarian face-to-face in an inperson session. Several students specifically mentioned becoming acquainted with their subject librarian as an important benefit of the AW session. This finding is particularly salient in today s digital age, with the ever-increasing availability of online resources and information. The attitude of English faculty toward the AW session impacted student perceptions of the session. One student reported that his teacher s negative view of the session led him to believe that the session was going to be a waste of time. Although some studies have examined faculty attitudes towards information literacy and library instruction (for example, McGuinness, 2006; Gullikson, 2006), few have studied the effect of faculty attitudes about a library instruction session on student attitudes toward the session. This subject deserves further follow-up, both at institutions and in research studies. Class standing largely determined student satisfaction with the AW session. Students who enroll as juniors learn major research skills that they can use for their remaining time at the university, and generally those students reported more satisfaction with the library session. Students who took AW as seniors derived less value from the library session because they had almost completed their university studies. Students are encouraged, but not required, to take Advanced Writing as juniors. Until this issue is resolved at the university level, seniors will continue to regret that they did not take the AW library session earlier. Discrepancies between students majors and paper topics lead to less relevant, and therefore less successful, sessions. When students select a topic unrelated to their major, but attend a session for their major, they do not learn about library resources for their papers and are unable to immediately apply discipline-specific information. If students attend the session related to their paper topic, they get help that is immediately applicable, but they miss out on material related to their majors that they might not get elsewhere. The common mismatch between major and paper topic is the most challenging problem with AW library sessions. It relates directly to the intended purpose of the Advanced Writing program, which librarians decided in 1994: that students receive major-based library instruction. This goal is not achieved for every student, and the effectiveness of the session decreases. In some cases, the English curriculum is only loosely tied to the students declared major, and consequently, the English faculty place little emphasis on research skills within the students majors. In addition, students are free to choose Hopkins & Julian, An Evaluation Communications in Information Literacy 2(2), Fall 2008 77 from a wide range of topics and assignments. Variations in the traditional research paper model, such as the technical manual and group paper, do not always lend themselves to topics in the students major field of study. Since students and English faculty did not mention this problem, a dialogue with the English department is needed. FUTURE DIRECTIONS The researchers felt that the AW program at BYU is broken, and that much can be done to increase its effectiveness. Librarians have limited influence on university curriculum decisions, the timing of student enrollment in an Advanced Writing class, and the discrepancy between major and paper topic. However, they have full control over some aspects of the program, such as library curriculum, session scheduling, and communication with the English faculty. Since conducting this study, the researchers have made progress on several of these problems, and have formulated a plan for implementing additional improvements. Although librarians have limited influence on the university curriculum, they intend to engage in advocacy for change regarding the AW program. Currently, the Assistant University Librarian for Public Services serves as a nonvoting member on the university s General Education Council, which makes decisions about the Advanced Writing curriculum. The librarians will coordinate with this library representative to advocate for an increased number of departmental AW classes outside of the English department. They will also continue to suggest that the council require students to take the Advanced Writing class by the end of their junior year. According to library administration and English faculty, this proposal has failed in the past; however, new administrators or future groups may be open to the change. In addition, librarians will work with the English faculty to advocate for paper topics that match students majors in order to ensure relevant and meaningful instruction. In the future, librarians might be able to work with the English department to change the AW foundation documents to mandate library instruction based on paper topic instead of discipline. In attempts to make positive, concrete changes, librarians tackled scheduling and teaching methods first. During the Fall 2007 semester, subject librarians were asked to incorporate 15 minutes of hands-on time in their AW sessions so that students could practice research skills. In the Winter 2008 semester, a new scheduling system was implemented and tested. The system allows librarians to schedule sessions based on student availability, instead of guessing what times would be best for students. Next, the librarians considered how to improve communication with the English faculty. During the 2007 2008 academic year, they met with English faculty in three groups: AW program administrators, course coordinators, and English 316 instructors. They presented a brief summary of the assessment and raised several discussion points, including the major/paper mismatch and the negative instructor attitude reported by a focus group student participant. The faculty administrators of the AW program maintained the need for major-based instruction, based on the learning outcomes of the program. On the other hand, AW program coordinators, who teach classes and work with other faculty, recognized the problematic major/paper topic discrepancies but made no concrete suggestions. Both administrators and program coordinators were surprised to hear of the negative attitude reported by the student focus group participant. English 316 instructors expressed their desire that the AW library session prepare students for dealing with information throughout their careers, not simply for their AW course or their major courses. Views were varied, and until further consensus is reached, the library is moving forward with major-based sessions. To maintain communication lines with English faculty, formal liaison relationships will be formed between the library and each of the English Advanced Writing courses. Librarians assigned to each course will act like subject librarians assigned to a campus department, Hopkins & Julian, An Evaluation Communications in Information Literacy 2(2), Fall 2008 78 serving as the main contact with the library and determining the best way for the library to serve the course. Increased communication may lead to differences in program administration; for example, AW students might come to the library as a class instead of individually. The plan for the future involves more significant changes to the library curriculum. AW session content will be standardized to include no more than 20 minutes lecture time, and librarians will write more specific learning outcomes for both the freshman program and the Advanced Writing program. They will create a list of skills students will need before they come to their AW session, and will provide Web tutorials and open research labs for students who need reminders. Subject librarians will teach one or two tools or principles that are particularly important in their discipline, and then students will have at least 30 minutes to complete an assignment tailored to their discipline and related to their paper topic. Librarians and students will be able to work together during the hands-on time to facilitate meaningful student learning. Subject librarians will have the flexibility to waive the AW requirement for students who have previously received course-integrated instruction. As the plan is implemented, a number of assessments may be conducted to determine the success of the new program. These will include measures of student, librarian, and English faculty perceptions as well as performance measures to determine student learning. OBSERVATIONS AND ADVICE Here are some suggestions for others who are considering library instruction in an upperdivision English course: If possible, focus the instruction on the paper assignment instead of the students majors. Avoid repetition in the library sessions; do not turn the upperdivision program into a repeat of the freshman-level program, either intentionally or unintentionally. However, provide remedial help in some form for those who need it. Teach students meaningful, advanced research skills. Create a standard curriculum and write learning outcomes for the program as well as for subject area sessions, if applicable. If students attend sessions outside of their normal class time, consider what you will offer for makeup credit. Decide how your program will integrate with other programs, such as library instruction integrated into discipline-specific courses. As you develop the program, communicate frequently with the English department, teaching faculty, and appropriate decisionmaking bodies on campus, like BYU s General Education Council. Establish a formal relationship between the library instruction unit and significant university stakeholders, such as curriculum councils and English coordinators. Involve library administration to ensure those with more influence are supportive of the plan and clear about their roles in developing your program. Clear program goals from the outset are essential to success. Consider the correct placement of the program in the university curriculum, and the optimal time for students to take the related course. If necessary, advocate for a university requirement mandating that students must take the course by a certain point in their coursework. Maintain communication with the English department as you implement the program and as it runs independently. CONCLUSION A program such as the one at BYU, in which subject-based information literacy instruction is administered through an upper-division generaleducation English course, creates obvious problems. The library is happy to retain a place in the university s General Education curriculum, but librarians have found subject- Hopkins & Julian, An Evaluation Communications in Information Literacy 2(2), Fall 2008 79 based instruction in a general course challenging. Despite the challenges, BYU s librarians are confident that they can improve logistics, increase student learning, and ensure a better experience for all involved. REFERENCES Association of College and Research Libraries. (2000). Information literacy competency standards for higher education. Retrieved March 21, 2008 from http://www.ala.org/ala/mgrps/ divs/acrl/standards/standards.pdf Carter, E. W. (2002). Doing the best you can with what you have : Lessons learned from outcomes assessment. Journal of Academic Librarianship, 28(1), 36 41. Grafstein, A. (2002). A discipline-based approach to information literacy. The Journal of Academic Librarianship, 28(4), 197 204. Gullikson, S. (2006). Faculty perceptions of ACRL s Information Literacy Competency Standards for Higher Education. The Journal of Academic Librarianship, 32(6), 583 592. Harrison, J., & Rourke, L. (2006). The benefits of buy-in: integrating information literacy into each year of an academic program. Reference Services Review, 34, 599 606. Holliday, W., & Fagerheim, B. (2006). Integrating information literacy with a sequenced English composition curriculum. Portal: Libraries and the Academy, 6 (2), 169 184. Kennedy, C. (2005). Teaching information literacy to the advanced writing class in three sessions. Electronic Journal of Academic and Special Librarianship, 6(1 2). Retrieved Oct 25, 2 0 0 7 , f r o m h t t p : / / southernlibrarianship.icaap.org/content/v06n01/ kennedy_c01.htm MacDonald, M. C., Rathemacher, A. J., & Burkhardt, J. M. (2000). Challenges in building an incremental, multi-year information literacy plan. Reference Services Review, 28, 240 247. McGuinness, C. (2006). What faculty think exploring the barriers to information literacy development in undergraduate education. The Journal of Academic Librarianship, 32(6), 573 582. Morrison, H. (1997). Information literacy skills: An exploratory focus group study of student perceptions. Research Strategies, 15, 4 17. Oblinger, D. (2003). Boomers, Gen-Xers, and Millenials: Understanding the new students. Educause Review, 38(4). Retrieved Oct 15, 2007, from http://www.educause.edu/ir/library/ pdf/ERM0342.pdf Spackman, E. (2007). Using focus groups to evaluate an information literacy program in a general biology course. Science and Technology Libraries, 27 (3), 3 28. Sult, L., & Mills, V. (2006). A blended method for integrating information literacy instruction into English composition classes. Reference Services Review, 34, 368 388. Hopkins & Julian, An Evaluation Communications in Information Literacy 2(2), Fall 2008 80 APPENDIX 1 STUDENT SURVEY QUESTIONS Hopkins & Julian, An Evaluation Communications in Information Literacy 2(2), Fall 2008 81 Please answer the following questions about your library SESSION. Strongly Disagree Disagree Neutral Agree Strongly Agree 1. After this presentation, I feel more knowledgeable and comfortable about using the library. ( ) ( ) ( ) ( ) ( ) 2. I see the material covered in the sessions as important to my college career. ( ) ( ) ( ) ( ) ( ) 3. The pacing of the session was appropriate to the material covered. ( ) ( ) ( ) ( ) ( ) 4. I know where to go to get further help when I have questions about doing research and using the library. ( ) ( ) ( ) ( ) ( ) Please answer the following questions about your library INSTRUCTOR. Strongly Disagree Disagree Neutral Agree Strongly Agree 5. The Library Instructor presented the material in an organized, easy-to-follow manner. ( ) ( ) ( ) ( ) ( ) 6. The Library Instructor explained concepts and answered questions in a clear and concise manner. ( ) ( ) ( ) ( ) ( ) 7. The Library Instructor made the research concepts, principles and programs interesting to me. ( ) ( ) ( ) ( ) ( ) 8. I felt like the Library Instructor cared about me individually. ( ) ( ) ( ) ( ) ( ) 9. Overall, how would you rate this presentation? ( ) Excellent ( ) Good ( ) Average ( ) Below Average ( ) Poor 10. Did you take the Library Instruction course as part of a introductory writing course or an advanced writing course? ( ) Introductory Writing Course 11. ( ) Advanced Writing Course OTHER COMMENTS OR SUGGESTIONS about the Library Sessions or Library Instructor? 12. 13. Class Name (ex. Engl 150, Phil 150 etc) Library Instructor s Name: Thank you for taking the time to take this survey and give us valuable feedback. We want the Library Instruction program to be as useful to students as possible and your honest answers will help us do that. Please know that your answers will be completely anonymous. APPENDIX 2 FOCUS GROUP QUESTIONNAIRES Advanced Writing Focus Groups Student Questionnaire Please list the Advanced Writing class you took during Fall 2006: Please answer the following questions. All answers will be confidential. Male_____ Female_____ Class: Freshman_____ Sophomore______ Junior_____ Senior_____ Major: ______________________________ Have you taken a First-Year Writing class here at BYU? Yes_____ No______ If you answered no please explain why not. Are you a transfer student? Yes_____ No_____ If you answered yes , what was your class standing when you arrived at BYU? Freshman_____ Sophomore______ Junior_____ Senior_____ Did you leave BYU for more than one semester and come back? Yes_____ No_____ If you answered yes , how long were you away? Advanced writing students are required to attend a one hour session in the library with a librarian who is a subject specialist. Please answer the following questions based on your experience. Did you attend an Advanced Writing Library Session? Yes______ No______ How did you choose which library session to attend? (please mark one) _____The library session related to my major _____The library session related to my paper topic _____The library session related to both my major and my paper topic _____Other. Please explain: Advanced Writing Focus Groups Student Questionnaire Please list the Advanced Writing class you took during Winter 2007: Please answer the following questions. All answers will be confidential. Male_____ Female_____ Class: Freshman_____ Sophomore______ Junior_____ Senior_____ Major: ______________________________ Have you taken a First-Year Writing class here at BYU? Yes_____ No______ If you answered no please explain why not. Are you a transfer student? Yes_____ No_____ If you answered yes , what was your class standing when you arrived at BYU? Freshman_____ Sophomore______ Junior_____ Senior_____ Did you leave BYU for more than one semester and come back? Yes_____ No_____ If you answered yes , how long were you away? Advanced writing students are required to attend a one hour session in the library with a librarian who is a subject specialist. Please answer the following questions based on your experience. Did you attend an Advanced Writing Library Session? Yes______ No______ How did you choose which library session to attend? (please mark one) _____The library session related to my major _____The library session related to my paper topic _____The library session related to both my major and my paper topic _____Other. Please explain: Advanced Writing Focus Groups Faculty Questionnaire Please answer the following questions. All answers will be confidential. Male_____ Female_____ Status (e.g. faculty, graduate student, etc.): Number of years teaching Advanced Writing at BYU: Have you ever attended an Advanced Writing library session? Yes_____ No_____ How do you recommend to your students the library session they should attend? (please mark one) _____The library session related to my major _____The library session related to my paper topic _____The library session related to both my major and my paper topic _____Other. Please explain Hopkins & Julian, An Evaluation Communications in Information Literacy 2(2), Fall 2008 82 APPENDIX 3 FOCUS GROUP QUESTIONS Focus Group Questions: Current AW Students 1. Have you written your advanced writing research paper yet? If so, please tell us about the advanced writing research paper you have written. Describe the process you went through to find information. 2. What did you learn in the Advanced Writing library session? Prompts (only if necessary): Databases Search techniques Full-text of the articles RefWorks (Did you know prior to Advanced Writing? If yes: from where) 3. Are these skills ones you will use in other classes or in a profession? If so, how? 4. What did you know prior to coming to the session that was repeated in the class? 5. What do wish you had learned in the session that wasn t taught? 6. Is this a valuable class? Should it continue? 7. What suggestions do you have for improving the instruction you received? 8. Are there other ways you could receive this instruction that would be more useful to you? 9. What other library instruction have you received? How did Advanced Writing library instruction fit with the other instruction you received? Focus Group Questions: Fall 2006 AW Students 1. Please tell us about the advanced writing research paper you wrote last semester. Describe the process you went through to find information. 2. What did you learn in the Advanced Writing library session? Prompts (only if necessary): Databases Search techniques Full-text of the articles RefWorks (Did you know prior to Advanced Writing? If yes: from where) 3. Are these skills ones you will or have used in other classes or in a profession? If so, how? 4. What did you know prior to coming to the session that was repeated in the class? 5. What do wish you had learned in the session that wasn t taught? 6. Is this a valuable class? Should it continue? 7. What suggestions do you have for improving the instruction you received? 8. Are there other ways you could receive this instruction that would be more useful to you? 9. What other library instruction have you received? How did Advanced Writing library instruction fit with the other instruction you received? 10. A semester after taking the class, do you have any additional perspective or comments about the library portion of Advanced Writing? Focus Group Questions: AW Instructors 1. Does the AW library session help your students with library research? If so, how? If not, why? 2. What do your students tell you about the AW library session? 3. What do you wish your students would learn in the sessions that isn t currently taught? 4. What suggestions do you have for improving the instruction your students receive? 5. Are there other ways your students could receive this instruction that would be more useful to them? Hopkins & Julian, An Evaluation Communications in Information Literacy 2(2), Fall 2008 83