The next installment of the ACRL OnPoint chat series, “What Really are Student Outcomes,” will take place at 2:00pm CST on Wednesday, January 14. Peter Hernon, Graduate School of Library and Information Science, Simmons College, will convene this month’s chat.

At the upcoming ACRL 14th National Conference in Seattle, Hernon will address student learning outcomes, which reflect changes in students (e.g., knowledge, skills, abilities) over the duration of a program of study. The focus is on moving beyond course evaluation and assessment and developing a picture at the program level and institutional levels. Both regional and program accreditation organizations support (demand) this broader view of outcomes assessment, while still encouraging the application of student outcomes (for accountability purposes). This focus on accountability, combined with using the results to improve the educational experience, has implications for creating a culture of assessment in libraries.

Transcript archive:

margotconahan Hi all from cold Illinois! This is Margot from ACRL. We'll be starting in a few minutes. Thanks for joining us this afternoon.

page2page here in South Dakota we have actual temp of -5, don't know the wind chill

InfoLitGal who should be asessing? us? in-class profs? students? administration?

Simmons_Hernon Clss level assessmentis different from program level. Apples to oranges

InfoLitGal true, I'm at a community college

page2page assessment should be done by a designated office on campus

lrichel How do you assess a program level when you only do one-shot instruction and IL is not required by any programs...?

cilibertia Our GenEd program is undergoing revision and key info lit outcomes are included in the draft

amurrell are there standards of testing for prog level?

ECalderon outcome assessments in various levels to me produce different outcomes at program, insti. etc... right?

lisalibrarian we use our classroom assessments to "wrap up" to the program level

Simmons_Hernon Librarians and faculty together shoukld build an asessment plan. Yu are assessing student learning over time

Amy_Gustavson I agree with lrichel's question.

graycath anyone that teaches shoudl at least assess their instruction for the particular session, but how much applies to the whole semester when you just do a one-time presentation?

South7th The question about the development of information literacy that interests me is whether librarians can be demonstrated to have any impact on its development. I don't believe there has been a well controlled study that has done so.

InfoLitGal How do you assess a program level when you only do one-shot instruction and IL is not required by any programs...? --THAT"S why I asked who is to assess?

Simmons_Hernon How do you wrap up class assesment to program level?

deeter11 Yes, Irichel's last question most applies to us here as well

lisalibrarian programs are made up of classes + other out of class experiences/requirements

Simmons_Hernon Program assessment involves a new culture drivne by accreditation and others

lisalibrarian so, just like our campus faculty take assessments from across student courses and experiences to "wrap up into" program level, we do too

lrichel Program level assessment needs buy in from administration -- what can we do, in the meantime, if we don't have that?

Simmons_Hernon Sure, I'll be discussing this at ACRL in March. Any way yiout translate an outcome into steps of what you want people to accomplish over time: novice, etc. The key is that you develop therubirc with faculty.

lisalibrarian interesting, we dont have an accreditation mandate but we're finding other ways

LuLibrarians is info lit defined by the accrediation agencies? do they all use the same definition?

cohenarl I think focusing on critical thinking is crucial, it goes beyond the library and the programs and focuses on future success

Laurel Littrell It is embedded in critical thinking.

Simmons_Hernon Checuk Google for scoring rubirc or see publciations from publisher--Stylus in VA

ECalderon thanks

InfoLitGal Our VP aof academic affairs says there's NO room for one measly credit for an info lit course - we need to look into embedding in another course

znewell Right now we are doing self-assessment and trying to make a case with core curriculum committees to look at a larger role for assessment. Assessment is now in the hands of separate accreditations by various departments

Simmons_Hernon I have a lsit of about 14 outcome areas: as critical thinking and communciation skills. You can also connect outcomesx

lucretiamcculley our science librarian is working with the biology dept to use the iSKILLS test from ETS and add "biology-specific" IL questions to the test.

graycath i think IL is taught best when embedded in other courses

InfoLitGal yup

LuLibrarians I agree.

aq8554 So, it's a discrete set of skills that can each be isolated into steps -- how do you assess their skill level on this?

sbrasley some of our campuses engage in curriculum mapping at the department level to help with creating rubrics for courses within the department

keris2103 Agree - IL has to be incorporated in the course

InfoLitGal continuity of learning & linking skills with education

znewell Yes, but can still be assessed even as part of a larger course

Simmons_Hernon I am not king about a course. The key is integration of outcomers--with librarians participating--throughout the whole curriculumf a program-usng scoring rubirc

Laurel Littrell This is what we are trying to do, Peter, but it is sure difficult!

kathy.labadorf Peter said: I am not king about a course. The key is integration of outcomers--with librarians participating--throughout the whole curriculumf a program-usng scoring rubirc

mollyanswers Does ACRL have a info lit rubric?

kathy.labadorf Yes!!!

Simmons_Hernon I do not t plug my writings., OH well, I hasve two books on outcomes assessment published by Libraries Unlimited that addresses our questions

aq8554 So, a student's "score" on the rubric would increase as they went along, being explosed to and learning specific skills.

aq8554 Would various prof's score each student on the rubric over time?

page2page isn't the use of rubrics often time consuming if done for an entire program?

cilibertia you could do just a sample

Simmons_Hernon You can use the information literacy guidelines of ACRL to refin th outcomes an then to think about a scoring rubirc. In my book Outcomes asessement in higher educatin, there is a set of outcome and rubrcs used at one university in NJ

pwatkins1 how do you measure the results demonstrated evidence of what students have learnd?

kathy.labadorf Would you develop one rubric for the entire program? Or are there stages of development with different rubric measures

Simmons_Hernon o can reduce rubircs as time consuming by working with existing ones and collectively modifying them

graycath assessment of any kind is time-consuming, but rubrics take less time than many others, and they are easy to write and grade objectively

page2page are you applying the rubric to one assignment

breitena Is anyone familiar with an assessment management system such as TracDat where each dept on campus including the library lists their learning outcomes?

Simmons_Hernon You might have one outcome but different programs could have different rubircs

ECalderon exactly

cohenarl what is the title of your books?

Simmons_Hernon Trac Data does not work in the context of progams--where faculty and librarians develop0 rubircs..

lisalibrarian I think you can record the data from the rubric in TracDat

kmcrowe WE use WEAVE for program assessment

Simmons_Hernon See iSills, Project Sails, etc. they need a scoring rubric for them to be truly useful

LuLibrarians Sails is very advanced for some.... we had issues using it.

pwatkins1 info literacy @ erau is creeping into Gen Ed program

page2page we are looking at developing rubrics and portfolios in our campus assessment, but there seems to be trouble in getting faculty to see the importance of having the students create a portfolio

Simmons_Hernon Again what is (are) the outcomes settled on by faculty, where does libraries fit in, and what are the rubrcis. The key then is how are the dfata gathered used to improve the educational experience

kathy.labadorf InfoLit is a competnecy in our GenEd program here

Simmons_Hernon A portfolio s a good source for outcomes and again liink it to a rubric

kathy.labadorf We've used SAILS and find it is more a bibliographic test than an infolit test.

page2page that is a big issue. we gather so much data but then it MUST be used to improve instruction, programs, learning

kmcrowe This isn't a program level but I'm working on a pilot with a class this semester using a rubric. The faculty member & I are developing it together

cohenarlSimmons_Hernon: What are the titles of your books?

Laurel Littrell Any advice about how to gather the data to be the most useful and efficient?

breitena I am thinking about TracDat as a place for all learning outcomes together, then faculty and librarians together working on the rubrics.

Simmons_Hernon Right with Project SAILS. I only mentin it as an exaple of on of the many instruments ou there. The key is how do you relate uch instruments tomeet your needs

lrichel New Jersey Inst of Technology has a rubric to assess portfolios from their Humanities program...

South7th Having information literacy listed as a competency in Gen Ed to be taken care of by taking one course or another is not the same as measuring sophomores and seniors to see if they have appreciably different levels of information literacy.

pwatkins1 we use Survey Monkey for pre/post tests in info literacy classes

lmaddengsu We're thinking about targeting our critical thinking through writing and writing across the curriculum programs for developing our IL program. Any advice on getting buy in from them?

Simmons_Hernon Data gathering: direct and indirect methods. Right now accreditation does care. Start with soemthing and use it to improve the eudcaitonal experience. You can build

mollyanswers Clickers have worked great for us. But pretest posttest model is a challenge

nefrazier3 and nice to use the clickers in the beginning of class to guage students' experience w/particular databases, etc

Simmons_Hernon At the course level I ony use the minute assessment. I do not focus much at the course level,. Ican read students and learning witout ore involvment. So more pre and post testing to prgam level (and advancing rubrcis)

cohenarl When will the transcription of this discussion be available; I'm getting alot of new info?

page2page i think we may be caught up in the course level because it seems like the most imediate way to prove our usefulness

lisalibrarian you can get out of the pre-post trap by not thinking so much about how does the student change from where they were to what level has the student attained

aq8554 Actually co-developing the rubric with faculty would level their expectations somewhat about what their students should know.

South7th I would be more interested in knowing the degree to which a skill or knowledge is retained at the end or beyond the end of an undergraduaute program.

sbrasley We need to educate ourselves more about activities associated with course-level assessment, program and institutional. I'm glad the the ACRL Immersion program now has an assessment track, so we can be up to speed.

kmcrowe working with faculty and also with key administrators on campus e.g. assessment peopel, curriculum people,

aq8554 It would also help faculty look to the library to improve discrete areas of their rubric?

Simmons_Hernon I agree ... My books are editied ones and invlve a numer of players. Will gie you something to think aobot

nefrazier3 co-developing rubrics w/faculty sounds ideal. working and interacting w/faculty on campus committees helps promote our abilities to help in this area

kmcrowe assessment immersion focuses on program assessment I highly recommend it

Simmons_Hernon Yes, work collaboratively with faculy on outcomes and rubrics

nefrazier3 it sure helps to have a place at the table on assessment and/or curriculum committees

pwatkins1 easier said than done

Simmons_Hernon Customer service (as satisaction) is an output. Accrediators would be interested. Butths is not outcomes assessment

meganoakleaf Ditto Peter. Working with faculty on rubrics is great fun! And really illuminating for both parties. Great too to include students in the process.

Simmons_Hernon Easiersaid than done. Well, a lot s being ne. Dive in

rouxlibrary Yes, I have to admit that I still don't quite understand what constitutes program level assessment

pwatkins1 how do studnets get involved?

Laurel Littrell Getting to the table is the first challenge and sure helps.

aq8554 In fact, isn't "assessment" basically the technique here. In other words, you're not focusing so much on the outcome of the assessment itself, but on where the holes int he rubric would tell you to address first?

Simmons_Hernon program level-look at te courses from ore to eective. Look t the progression of student learning. What shoudl they knwo and be able to do t this stge, next stage, etc

aq8554 Then if you try something else to incrase their info literacy to address X box in the rubric, you could assess the effectiveness of that 'improvement' by whether the score in that particular box went up?

Laurel Littrell Program level is seeing how the learning in all the classes fit together.

Simmons_Hernon Outcomes has different parts--your goal (outcome), whih one

Simmons_Hernon Then what is the corespndign rubirc,.the how use the information to mprove

kmcrowe and fits into the whole curriculum (program level)

Simmons_Hernon Rigt, the ooutcome and rubirc are accepted by the whole progam .. You are goind beyond whatis learned at ocurse levl

jonesphi Does program level mean for an institution? A major?

Simmons_Hernon major

Laurel Littrell That is why there is such pressure on the teaching departments, to have a rubric and plan for every single degree program.

Laurel Littrell They seem to be grateful for whatever partnership we can do with them!

Simmons_Hernon To compicate matters som places have institutonal outcvomes. Simmons has sai it is leadershp. So here beocme an area to developoutcme

cohenarl I guess this is like a prerequiste?

nefrazier3 do you think that suggested outcomes for basic vs. advanced levels would work? or would you recommend say, all 4 yrs of a 4-yr program?

Simmons_Hernon theree would still be program outcomes as criticla thinking and better communication skills

Simmons_Hernon rubric factors in basic and advanced--you are progressing studentd

ECalderon info lit skills are imprt

c_hammer How wide/extensive is program level? Post-graduation assessments?

BobPh_ftw have you discussed course mapping of program outcomes? if so, I will find it in the transcript.

Simmons_Hernon So for an undergraduate in their major, wat are the desired outcomes and programs may have different rubirics

Simmons_Hernon The mistake I see is that to many people try to do too mcuh

ECresdesk Esp when administrators are pushing for assessment

jonesphi What are they doing that they shouldn't?

cohenarl about prerequisites; does this create a progressive learning path?

Laurel Littrell Some try to do too much and some completely ignore the requirements! :-)

Simmons_Hernon The prblem also is the literature. there is a lot of fiocus on LibQUAL, which is an output not anoutcome

nmurillo1 I don't think our faculty see us as legitimate players in the development of learning outcomes in their departments.

Simmons_Hernon No, in doing too much I am saying be selctive in getting stted with outcomes and rubrcs. You can build lter

breitena some programs offer capstone course at the end. this would a natural place to perform an assessment of info lit learning.

margotconahan **This is a fast and furious conversation, everyone! We told Peter to plan on about 45 minutes so I just wanted to chime in with a 5 minute warning. You are all welcome to continue the chat, but wanted to give a heads up on the time.***

Simmons_Hernon Rght, LibQUAL is that. BUT a lot of the literature moves it to the other side

Laurel Littrell Getting faculty to accept us as partners is critical to our participation.

kmcrowe absolutely!

nefrazier3 have others impelemented the Research Practices Survey? HEDs/NITLE, originated I think w/St. Olaf's and other midwestrn. colleges?

InfoLitGal amen

Laurel Littrell We get on better with some departments than others!

Simmons_Hernon I am having a hard time keeping up wih all of the exchanges I am delighted to introduce the topic. If Marogt wans m back, happy to advance our chat

South7th LibQUAL seems built on the assumption that libraries are service/support organizations and not educative.

lisalibrarian LibQual does have questions though about how users perceive the library's impact on their info lit skills

guest7601925 At Binghamton University we are currently doing our second round of a Research Practices survey

Simmons_Hernon Again start by reading the Middle States text and ahve a chat within your libraries about what we are discussing.

InfoLitGal it always boils down to the politics of your inst. How well your chair gets on with the other chairs & admin

lisalibrarian it isn't student learning assessment but is good data for then using to say librarians have something to contribute

kmcrowe I'l like more info on Research Practices

page2page but perception is well perception

nefrazier3 we just did our first round--to FY students and seniors

pwatkins1 true

guest7496612 I am fuzzy what is the diffrence between an outcome and an out put?

Simmons_Hernon A urvey may have program use, I'd need to know more.

InfoLitGal Ours has a rodney dangerfield approach with all other chairs

lisalibrarian yes, but if students perceive the library as helping them develop info lit skills - then that can help in saying why librarians should be part of the conversation

nefrazier3 Link to more info about development of the Research Practices survey: