Assessment Report 1999 Appendix C

1999 Annual Report

Appendix C

This appendix details examples of the effects of outcomes assessment at UNL. However, because the purpose of this section is only to provide a broad summary, the identification of the college/department/program involved is not included. This anonymity also emphasizes that assessment information is to be collected by a college/department/program to inform their own efforts for improvement and not as an accountability measure to outside constituents unless the college/department/program chooses to use it as such. It should also be noted that this listing provides only a representative sampling rather than a comprehensive list of assessment activities. Therefore, conclusions based on this narrative should be used to qualitatively evaluate the progress of outcomes assessment at UNL rather than quantitatively evaluate UNL's progress.

Effects are summarized within five categories. These five categories represent the different types of actions resulting from assessment activities. The five categories are:

Instructors created and enhanced scoring rubrics for courses used to assess learning outcomes after attending a Teaching and Learning Center workshop that assisted them in establishing clearer definitions of objectives and measurements.

Because assessment activities produced a lot of information about the process of student learning but very little information on the outcomes of student learning, the assessment plan was modified so that evidence about student learning outcomes would also be collected.

Faculty who attempted to use seniors' job portfolios for assessing learning outcomes realized that this product did not provide the kind of information needed to improve student learning. Therefore, they considered using a one-hour portfolio class to modify the contents of the portfolio to accomplish both.

Faculty reconsidered how they collect information from graduating seniors because they are unsure that one-on-one interviews with faculty were inhibiting students from being open in their responses.

Past assessment results were used to determine how to improve an assessment measure. The percentages of students who selected incorrect responses on multiple-choice questions were used to create a more logical set of "foils". The content of the exam is also frequently updated to represent the currency of the curriculum.

Educational Testing Service approached a college using one of their standardized exams for assessment purposes about a joint project. The joint project would allow the publisher to obtain information needed for validating the exam and the college would be able to administer the exam at no cost. In addition, the college would obtain additional background data collected by the publisher for use in studying factors that may affect students' performance.

Detailed protocol for assessment that outlined assessment methods, when and how those methods would be administered, steps in using the evidence collected, and who is responsible for each step was used to clarify the process for faculty.

II. HIGHLIGHTED ISSUES AND PROMPTED DISCUSSIONS

Responses on senior exit interviews indicated that the professional development of students was a skill needing to be addressed.

Evaluation of students' final papers indicated a need for a course highlighting research methods and grant writing.

Students responses on senior exit interviews indicated they did not understand the philosophy behind the structure of the curriculum and indicated a need to communicate that philosophy to students so they better understand why they are required to take some courses.

Less than positive responses on senior exit surveys, indicated that actions needed to be taken that promoted faculty and student interactions, improved students' educational experiences, and encouraged the involvement of student in activities outside of class.

III. CONSIDERATION OF ACTIONS TO IMPROVE STUDENT LEARNING

Faculty discussions about student deficiencies exhibited in their portfolios led to the conclusion that these skills may not be covered adequately in the existing curriculum. Therefore, a new course was considered for the major.

The feasibility and potential benefits of offering a research seminar to sophomores, in addition to the research seminar offered to seniors, was considered due to students' request in senior exit surveys to encounter these skills earlier in their education. In addition, evidence from assessment measures indicated that seniors who took classes that focused on research skills earlier in their education had higher quality research papers.

Faculty generated several ideas for addressing the poor ratings given on student exit surveys regarding opportunities to interact with faculty. These ideas included email distribution list and formal meetings that provided an opportunity for faculty and students to get to know one another.

Students' dissatisfaction with advising led to the consideration of organizing advising seminars and the creation of advising packets.

A strategy for exploring how a senior seminar could be changed to improve student performance and perceptions was considered. This strategy would involve bringing together all faculty members who have taught the seminar to discuss its content, structure, and format.

Student responses on exit interviews led faculty to reconsider the core curriculum by determining how that core fits together and whether the core meets the students future needs after they graduate.

Exit interview responses encouraged the consideration of a capstone course that synthesized the curriculum of the program in the student's final semester.

IV. ACTIONS TAKEN TO IMPROVE STUDENT LEARNING

A new major was created after realizing that student's inadequate performance on a standardized exam highlighted areas in which the curriculum did not adequately cover the breadth of the discipline.

Student responses in exit interviews encouraged a change in the advising structure to improve communication about the program and its requirements.

The hosting of an awards ceremony was prompted by students' request in exit interviews for more student recognition.

To improve seniors' performance on theses a non-credit course in identifying problems for a thesis and preparing a thesis was initiated.

Discrepancies between the ratings of faculty and students regarding each student's achievement of the program's learning objectives prompted one program to conduct a curriculum review to determine which courses emphasized the skills needed to achieve the objectives in question. This review led to addition of a new course to the major.

New and existing courses that could contribute to the completion of requirements for a major were identified and added to the program of studies because students requested more course variety and increased frequency of offerings.

Student learning deficiencies in a core course prompted the identification of a set of course objectives that would be documented in all syllabi for that core course. This requirement helped insure that the material covered in each core course remained consistent across instructors.

To improve students' written work, criteria for evaluating the writing and examples of good performance were publicized in the syllabi.

In an effort to clearly articulate the objectives for core courses in a program, teaching circles of instructors for each core course were created. These teaching circles discussed what was being taught and how it is taught so that common learning objectives for that core could be clearly articulated. This discussion led to a reevaluation of the final class project and experimenting with ways to connect information in one course with subsequent courses.

Student input in senior exit interviews prompted the establishment of a resource library with the assistance of private companies.

Students' requests in senior exit interviews for more opportunities to work with students in related majors prompted the establishment of a learning community with students from each of the related majors. In addition, faculty discussed projects that would require students from all three majors to work together.

A focus on enhancing research skills and written communication through a review and modification of the curriculum was instigated by senior and alumni survey responses.

V. FOLLOW-UP ON ACTIONS TAKEN TO IMPROVE STUDENT LEARNING

In response to students' request on exit surveys for a better sense of community, faculty arranged several programs/groups to address that request. This included a mentoring program and revitalization of an honor society. The contribution of this action was monitored in the first year by assessing the amount and quality of participation. Although participation was low the quality of the participation by those involved (i.e. enthusiasm and feedback) provided some ideas on how participation might be improved in the future.

The assessment of products from the second course of a two course sequence indicated that changes made in the first course led to improvement on objectives in the second course.

Because using peer review improved the quality of senior papers, a more structured framework for students to engage in the peer review process was created.

How well students continue to acquire content and behavioral outcomes will be monitored with an annual assessment measure. This measure provides a benchmark for knowledge and skills before changing the curriculum from a paper based environment to a multimedia environment.

A follow-up review of student portfolios after a new core curriculum had been instituted indicated that students illustrated a higher level of creativity and had better technical skills for the discipline. Faculty expect that after the new core curriculum be better established additional improvements in student proficiencies will be obtained.

The full participation of all faculty members in a portfolio review process resulted in faculty obtaining a better understanding of the skills and knowledge gained by students throughout the program (rather than just the faculty member's individual course). This participation also provided faculty a better sense of they each could enhance student learning. This led to improvement in students' written communication and increased sophistication in their use of technology.

Continual dialogue among faculty has assisted in the assessment of learning objectives because faculty members have reinforced the criteria for assessment in their individual course assignments.