Minutes of August 28, 2002 Meeting

Committee members were asked to read the revised report on e-journal survey before the September meeting. The committee
will determine if it is necessary to have a discussion of the expanded or added sections at the next meeting. Sherrie
Tromp, who co-authored several publications on process improvement and root cause analysis in higher education, will
attend our next meeting to discuss key concepts and steps in her books. Such a discussion is deemed useful for our work
in the coming year.

Samson reported that two of the goals the Public Service Council has formulated for the current fiscal year are related
to committee's work: "Assess Reference" and "Assess the Web Pages." Last year, as part of a proposed review of
reference services, PSC wanted to assess either the quality and/or the outcomes of reference services, in addition to
longer-range planning and discussion of relevant issues. While the intended objectives later evolved, some members of
PSC apparently are still interested in having a formal assessment done. Assessment of library web pages will also
require considerable discussion and planning. There are a few good sources on how to prepare and implement a usability
test/study and results of some web site usability tests recently done at other schools have been reported via the
library literature. We will work with Web Advisory Committee to make sure the methodology and procedures used are
adequate for the stated objectives and purposes.

Susan reported briefly on the research project she did during her sabbatical leave. The focus of her project was on
assessment and evaluation activities and decision making at ARL libraries. She had a chance to interview many
individuals, ranging from university librarians, functional directors, department heads, to team leaders, at 9 ARL
libraries. To conduct these interviews, she formulated a large number of questions on assessment impact on decisions,
assessment process goals, decision factors, new data measures, and technological impact on assessment process. Susan
agreed to share with the Committee the list of people she talked to and the specific questions she used during her
interviews. Susan is in the process of further analyzing her findings and preparing a final report.

The committee discussed the activities it would like to focus on in the coming year and set the following goals: 1)
Refine the role of the Assessment Committee and identify library colleagues who possess analytical and assessment
expertise; 2) Test the process improvement tool being considered with several functional areas for its applicability in
the Libraries; 3) Develop criteria for assessing library services and collections; and 4) Coordinate the Libraries'
participation in the ARL E-Metrics Project. In preparation for some of these activities, the Committee will look at
special assessment and management techniques used by other research libraries to determine success in different service
areas, e.g. the Balanced Scorecard at the University of Virginia (see
http://www.lib.virginia.edu/bsc/ ).

To prepare for our participation in the E-Metrics Project, the Committee reviewed again the ARL proposed measures for
different areas, e.g. use of networked resources, library digitization activities. Since a number of proposed measures
are interrelated, it was agreed that we will not designate at this time coordinators for different areas to make sure
required data are collected according to the definitions and established timeframe. We will revisit the necessity of
doing this at a later time.

Our September meeting will be held on Wednesday, September 25 at 9:30 am
and the October meeting will be held on 10/16/02 at 9:30 am.