INLS 782: Develop an Assessment Project

The UNC libraries has a Assessment Team. This Team always has several projects underway, and many more that they would like to undertake but don't have the time or staffing to execute. Here are some potential projects that the Assessment Team currently has underway or would like to undertake:

Evaluation of the benefit of in-depth cataloging of materials in the rare books collection. How do users use the cataloging records? What benefit do users gain from in-depth cataloging records?

Evaluation of the collection development policies and practice for the Area Studies collection.

Evaluate tools that may be used to analyze the diverse data collected by the UNC Libraries, and develop a prototype dashboard and/or visualization using the tool of choice.

For both of these projects: identify patterns of user and/or staff behavior, and/or make recommendations for modifications to the existing service model

Student groups will consult to either the UNC Assessment Team or to the NCSU Director of Learning Commons Services on one or more project(s). This may be one large project across the whole class that can be subdivided into several components that each student group can address, or several smaller projects. Either way, the class will divide up into working groups of approximately 4-6 students (depending on course enrollment).

Select one of these projects, or come up with your own idea and pitch it to the Assessment Team. If they give it a thumbs-up, then probably so will I.

This project will be divided up into several deliverables, due throughout the semester. See the course schedule for the due dates. The deliverables will be as follows:

First draft project plan. This must contain the following:

The library service(s) or function(s) that your group plans to evaluate.

The evaluation questions that will guide the project.

The stakeholder group(s) for whom you are conducting the evaluation.

The data to which you would ideally like to have access.

The evaluation metrics and methodologies you plan to use.

A description of the deliverable(s) from the project and its component parts.

A timeline for the project as a whole and component parts.

The tools you will use to manage the project.

Presentation on prior evaluations similar to your project. In preparing your project plans, you must look to the library literature, both published and gray, to find examples of evaluations similar to your project that have been done in the past. This prior evaluation work should inform your project: Perhaps there are existing measures or data collection instruments that you can use. Perhaps you believe that all prior work is flawed and that you can design a better evaluation. This presentation should systematically present the themes running through this prior evaluation work: What methodologies and analyses have been used in the past? What data and metrics have been used? What assumptions have been made? What was the outcome of this prior work? How can this prior work inform your project? This presentation must be no more than 15 minutes long.

Second draft project plan. This must contain the following:

The data to which you actually have access about the service(s) or function(s).

A revision of any of the following:

Evaluation questions

Metrics and methodologies

Deliverables

Timeline

Tools

Progress report. This will be due approximately halfway between the due dates of the Second draft project plan and the Final report, and will be submitted as brief (~500 words) posts. This should be a formative evaluation of your group's project, and must contain the following:

Description of the work your group has conducted so far on your project.

Description of any changes that have become necessary to your project plan, and explanation for why those changes were necessary.

Revised deliverables, timeline, etc.

Final report. In the last week of the semester, groups will write up your project. This writeup will have 3 parts: the evaluation report, reflections, and an evaluation plan.

By the end of the semester, you will have seen several evaluation reports; you may want to model your report on those. The report must contain the following:

Evaluation questions

Decisions about how you scoped your project

Methodology & data collection instruments developed

Data collected & analyses conducted

Findings

Preliminary recommendations

Etc.

In the second section of your report, your group must reflect on the process of working on your project. Some questions to consider as you write this post:

Were you able to answer your evaluation questions? Were your evaluation questions answerable?

Were the deliverable(s) you articulated realistic?

How did the data to which you actually had access differ from the data you needed, or ideally would have liked? How did that affect your evaluation?

Were the metrics you used appropriate? Were the methodologies?

How successful was your group in meeting the deadlines you set in your timeline? Why?

Were the tools you used to manage the project appropriate? Easy to use?

Evaluation plan. This must be a middle- to long-range plan for continuing the evaluation of the library service(s) or function(s) into the future. The evaluation plan must contain the following:

Who (individuals or stakeholder groups) must be involved in the evaluation?