Page 3

The question the Virginia Tech AiA Team sought to answer was how effectively do graduate students use Summon? At Virginia Tech Summon is taught extensively in First Year Experience courses. It provides a simple interface conducive to interdisciplinary, undergraduate research. At the graduate level, students are expected to already have well developed, sophisticated library research skills tailored closely to their discipline of study. Therefore, graduate level information literacy courses do not focus on Summon search skills but instead emphasize research skills that are targeted toward discipline-specific databases. Interdisciplinary research is increasingly more prominent at the graduate level and this type of research requires skillfully refined searches that are well suited for a discovery layer. Our AiA team sought to examine graduate students use of Summon by comparing their skill sets with undergraduate students and faculty.

Executive Summary (150 words open)

How does the project align with your institution’s priorities and needs?

Why did you choose the outcome and library factor as areas to examine?

Why was the team composition appropriate?

Providing transformative graduate education is an initiative that helps graduate students build an inclusive interdisciplinary academic community. And assessing our graduate student’s use of Summon aligns with one of our strategic goals; to collect and analyze the learning and research needs of Virginia Tech students and faculty. We know that our graduate students’ academic and research needs are quite different from the needs of their undergraduate student counterparts. Therefore, a thorough assessment of Summon from the perspective of our graduate students’ should yield practical data that can inform our actions in better aligning library services, tools and training programs that meet the expectations and needs of these students’ in order to increase their level of success.

What are the significant contributions of your project?

What was learned about assessing the library’s impact on student learning and success?

What was learned about creating or contributing to a culture of assessment on campus?

What, if any, are the significant findings of your project?

The project was developed to study our Summon discovery tool and to analyze its alignment with our graduate students needs as researchers. The assessment indicated that overall graduate students were adept at using Summon more effectively than faculty or undergraduate students. However, many graduate students demonstrated some difficulty with finding relevant full-text articles and e-books, which requires the appropriate use of filters and subject terms. Additionally, Graduate students only used the Advance Search features 8% of the time, the least among the three groups tested. This may be significant because using appropriate subject terms in the Advance Search fields provides the most targeted, relevant results. An area for future research would be to study whether the effective use of Summon correlates to the type of library instruction that graduate students receive.

What will you change as a result of what you learned (– e.g., institutional activities, library functions or practices, personal/professional practice, other)?

How does this project contribute to current, past, or future assessment activities on your campus?

Data-driven decision making is a vital part of the Virginia Tech culture of continuous improvement. We are heavily invested in the uses of assessment for our continued growth and excellence in University Libraries. As a result of participation with AiA, we will continue to assess our students' success with the Summon discovery layer. This will enable us to further engage students in scholarly inquiry and provide students with the tools needed in conducting research using innovative technologies in research, scholarship, and learning. This assessment of our Summon service has challenged our assumptions of graduate student research methods and the tools needed for success. As a commitment to our student’s success, we will continue the assessment of our discovery layer in order to transform the ways our patrons discover, access, and experience information.

Assessing graduate student use and skill with the Summon discovery service

How effectively do graduate students use Summon? Twenty-three graduate and undergraduate students participated in a usability study to measure skill factors that impact research. The study indicates that graduate students were adept at using Summon more effectively than faculty or undergraduate students. Future research would be necessary to know if this correlates to frequency and type of library instruction.