Formative Assessments: In Development

for Test-Enhanced Learning

Aquifer is developing new formative assessments for test-enhanced learning. As the non-profit leader in clinical learning, Aquifer is pleased to create high-quality assessments for learning to accompany clinical rotations for Family Medicine, Internal Medicine, Pediatrics, and Radiology.

Research shows that focused, repetitive practice spaced over time improves recall and transfer of knowledge. Aquifer is combining the concepts of test-enhanced learning and distributed practice to create shorter, more frequent assessments that will help build more efficient, effective learners.

Development & Validation

Aquifer Consortium Assessment Leads for each course, led by Valerie J. Lang, MD, MHPE, Aquifer Academic Director for Assessment, worked to recruit and train medical educators from across the country to develop single best-answer multiple-choice questions for our new assessments. All new questions are mapped to key learning objectives and teaching points in Aquifer’s cases for each course. Because Aquifer’s courses cover the national curriculum (STFM, AUR, CDIM, and COMSEP), this provides a strong correlation between the new formative assessments and national curriculum standards.

Question authors were trained by assessment experts, and content was extensively peer-reviewed and edited during regular calls and day-long seminars at national meetings. These new questions will be added to Aquifer’s existing multiple-choice exams to provide psychometric validation prior to piloting formative assessments. Valid questions from our existing exams will also be used in the new formative assessments.

Pilot Testing & Schedule

Family Medicine, Pediatrics, and Radiology formative assessments will be available for pilot testing in Spring 2020. Internal Medicine is planned to join the pilot testing later. If you are interested in learning more about pilot testing these innovative new exams, please complete our survey.

Existing Exam Update: Available Through July 2021

Our summative 100-question multiple-choice exams for Aquifer Family Medicine, Pediatrics, and Radiology will continue to be available in their current format until June 2021.Aquifer will no longer offer a summative exam option beyond June 2021.

The Aquifer Internal Medicine Clinical Decision-Making Exam, formerly the Key Features Exam, will not be impacted and will continue to be available at no cost for subscribers of Aquifer Internal Medicine. In addition, formative assessment tools to accompany our Internal Medicine course are in development.

Summative Assessment Status Update

Initially, we planned to offer a summative option that requires a proctor and uses our formative assessment questions. As our formative assessment plans evolved over the last few months, it became clear that we could no longer plan to offer a summative exam option beyond June 2021.

Aquifer has decided to extend the availability of our existing 100-question multiple-choice exams for Family Medicine, Pediatrics, and Radiology through June 2021 to ease the transition. Summative exam users should be aware that some exam questions will also appear on the formative exams.

Frequently Asked Questions

What did Aquifer learn from surveying subscribers about assessment?

In August 2018, we surveyed both current Aquifer exam users and subscribers who don’t currently use our exams. We learned that:

Our subscribers would like more high-quality formative assessment tools that can be used as assessments for learning, providing immediate feedback to students to help them focus their learning. About one-half of the respondents to our surveys told us that they would require their students to take formative assessment if a high-quality option was available.

About 80% of respondents agreed that offering formative assessment with granular feedback would promote students’ learning regarding patient care as well as improve student outcomes on the Shelf and National Board Exams

Immediate feedback, comprehensive curriculum coverage, and customization were highly valued as the most important aspects of formative assessment.

As a non-profit organization, Aquifer’s mission is to enable learning that matters by building students’ essential knowledge, clinical reasoning and decision making skills. Our Aquifer Family Medicine, Pediatrics, and Radiology exams were originally developed to provide alternatives to NBME Shelf and AMSER exams that more closely aligned with each discipline’s curricula. As the quality of NBME Shelf exam coverage of national curricula has improved over the years, the value Aquifer contributes to the community through our own summative exams has diminished. In addition, clerkship directors told us that they want high quality, rigorous formative assessment for their students, but there are no resources that are directly linked to the national curriculum.

How do Aquifer cases prepare my students for other summative exams?

Aquifer courses are developed in collaboration with our national partners (STFM for Family Medicine, COMSEP for Pediatrics, AUR/AMSER for Radiology, AAIM/CDIM for Internal Medicine) to provide comprehensive coverage of their national curriculum. NBME Subject exams, like Aquifer courses, are designed to test a broad range of knowledge within the subject. Also like Aquifer courses, the Subject exams concentrate heavily on application and integration of knowledge rather than on recall of isolated facts. Students using Aquifer courses gain significant experience in applying knowledge in a virtual clinical context. Additionally, each Aquifer case is followed by a set of self-assessment questions to provide students with additional practice answering multiple-choice questions and to reinforce key concepts from the case.

How can I learn more?

Be sure to view our latest update and share your feedback on assessment format, administration, and reporting—and read about Aquifer’s evidence-based approach to building formative assessments for test-enhanced learning.

Feel free to reach out to Dr. Bridie Napier, Aquifer’s Director of Curricular Operations, at [email protected], with questions about this project. Please let us know if you would like to be included in helping us develop and pilot test these new tools.

Engage

Our website uses cookies to help us remember you and understand how you use our site. You can learn more by reading our privacy policy. By continuing to use our site, you are agreeing to our cookies policy.Accept & Continue