Team-Based Learning: 2016 JGME-ALiEM Hot Topics in Medical Education

As a follow-up to last year’s inaugural JGME-ALiEM Hot Topic in Medical Education on the Resident as Teacher role, this week we will be conducting a cross-disciplinary discussion about a unique instructional strategy called team-based learning (TBL). Originally developed by Dr. Larry Michaelson, a professor of Business at the University of Oklahoma, over the past 15-20 years TBL has been increasingly incorporated in health professions education. Prominent in undergraduate medical curricula, TBL focuses on active learning, collaboration, and application to real-world problems. As educators consider its value in postgraduate education, TBL is our “hot topic” for 2016.

Whether you are hearing about TBL for the first time, considering incorporating it into your practice or just curious to stay on top of what’s hot in meded, we invite you to engage in the discussion of the JGME publication entitled “Use of Team-Based Learning Pedagogy for Internal Medicine Ambulatory Resident Teaching” by Balwan et al. using the Twitter hastag #JGMEscholar [free article PDF].

Similar to previous ALiEM-Annals Journal Clubs, a live Google Hangout will be held with the authors and selected experts. Ultimately, a curated summary from discussions (ALiEM blog, Twitter, Google Hangout) will be published back in JGME. Some of your best tweets and blog comments will be featured.

What is Team Based Learning (TBL)?

TBL is an active learning and small group instructional strategy that provides students with opportunities to apply conceptual knowledge through a sequence of activities that includes individual work, teamwork, and immediate feedback.1 The concept is that sequential activities allow participants to scaffold their learning while tackling problems similar to real practice.

Image from http://www.bradford.ac.uk/educational-development/team-based-learning/TBL-new-diagram.png

1. Advanced, Pre-Class Preparation

The instructor must choose and provide learners with materials (reading assignments, podcasts, videos) that specifically address the learning objectives for the session. Learners are expected to study and review the materials in preparation for the TBL session.

2. Individual Readiness Assurance Test (IRAT)

At the beginning of the instructional session, learners complete a knowledge assessment (typically a multiple choice test) that is directly related to the advance preparatory material. This assessment induces an educational effect,2 where learners are motivated and accountable for advanced preparation.

3. Team Readiness Assurance Test (TRAT)

After completing the IRAT, the learners split off into their teams and complete the same set of questions previously answered individually in the IRAT. During this exercise the team must develop a consensus about their answers. Ideally there is a mechanism in place that allows team members to know as immediately-as-possible whether or not they have selected the correct answers. This immediate feedback helps them to refine their decision-making processes.

4. Instructor Clarification and Review

Typically this involves a very brief clarification of concepts that a significant number of learners struggled with throughout the readiness assurance tests. Learners must feel confident that they are adequately prepared to use their newly-acquired knowledge during the subsequent the team application exercise.

5. Application Exercise

The application exercise is the most important step: learners are given a problem similar to what they might face in clinical practice. This must be a problem that challenges them and forces them to interpret, calculate, predict, or analyze. At the end of the exercise, all teams must report their responses (near)simultaneously to the group and explain or defend their choice to the group.

What are the perceived benefits of TBL?

Unlike other small group learning strategies, it allows for large groups of learners to be taught by one content-expert, thus making it scalable. It is appealing to educators who want to promote active and engaged learning without the need for additional instructors.

It helps to equip learners to function in a team-oriented environment common to Medicine.

The integrated assessments hold each individual accountable for their preparation and participation. The assessments also allow for early identification of poor performers, while supporting and engaging them through extensive peer teaching.

A recent BEME review acknowledges the potential of TBL to significantly increase knowledge scores in health professions education.3 Although the evidence does not clearly demonstrate superiority over other instructional strategies, it is at least as effective.

BACKGROUND: Team-based learning (TBL) is used in undergraduate medical education to facilitate higher-order content learning, promote learner engagement and collaboration, and foster positive learner attitudes. There is a paucity of data on the use of TBL in graduate medical education. Our aim was to assess resident engagement, learning, and faculty/resident satisfaction with TBL in internal medicine residency ambulatory education.

METHODS: Survey and nominal group technique methodologies were used to assess learner engagement and faculty/resident satisfaction. We assessed medical learning using individual (IRAT) and group (GRAT) readiness assurance tests.

RESULTS: Residents (N = 111) involved in TBL sessions reported contributing to group discussions and actively discussing the subject material with other residents. Faculty echoed similar responses, and residents and faculty reported a preference for future teaching sessions to be offered using the TBL pedagogy. The average GRAT score was significantly higher than the average IRAT score by 22%. Feedback from our nominal group technique rank ordered the following TBL strengths by both residents and faculty: (1) interactive format, (2) content of sessions, and (3) competitive nature of sessions.

CONCLUSIONS: We successfully implemented TBL pedagogy in the internal medicine ambulatory residency curriculum, with learning focused on the care of patients in the ambulatory setting. TBL resulted in active resident engagement, facilitated group learning, and increased satisfaction by residents and faculty. To our knowledge this is the first study that implemented a TBL program in an internal medicine residency curriculum.

Hot Topics Questions

Remember to respond using #JGMEscholar hashtag if joining the discussion via Twitter.

If knowledge is socially constructed (i.e. how an individual organizes, perceives and attends to information is influenced by the interaction of other individuals in their environment) then the idiosyncrasies of the make-up of a particular team may lead to different learning outcomes between teams. Should the organization of a team be random or specifically cultivated?

The TBL evaluation data presented conflicting results. The engagement survey, which used individual responses, indicated that 93% of residents and 88% of faculty agreed or strongly agreed that “I/residents contributed my/their fair share to session discussions.” Yet, the nominal group technique, which used a quasi-consensus process, noted an imbalance of resident participation from both faculty and resident evaluation groups. How do you explain this inconsistency?

While the study did not compare the required resources or time necessary to run a TBL session, the discussion implied that this instructional method was more resource intensive for faculty. How can we motivate faculty to invest in learning methods that require more work than their current practice?

How does the team-based decision-making process improve a learner’s independent decision-making typically required of clinical practice?

Disclaimer: We reserve the right to use any and all tweets to #JGMEscholar and comments below in a commentary piece for a Journal of Graduate Medical Education publication as a curated conclusion piece for this hot topic discussion. Your comments will be attributed. We thank you in advance for your contributions.

WashU in St. Louis @wustl_em has been using TBL for a few years now. One of our faculty (Rebecca Bavolek) runs the majority of the classes. It seems to me that faculty investment might be a bit higher in the beginning, until you get the hang of how to construct the sessions. We tend to have Rebecca act as the TBL expert moderating the session, while another content expert drives the information. We split our residents into 6 groups (families) with 6 or so per group. 3 of the groups do Sim while the other 3 do TBL. At 2 hrs w/ flip. Thus 2 hrs of content takes up about 2 hrs of faculty time. So while prep time might be a bit more, the cost of faculty/hr is about the same. Although, you double the effort if you use the TBL “Expert” model; it decreases the perceived “Buy in” cost for other faculty.

One of the benefits of TBL touched on by the paper is retention. It seems residents are more likely to pre-read and prep for the group TBL, knowing they may have to contribute to their families performance. On the surface at least; this kind of prep doesn’t seem to happen to the same extent when the learner only disappoints themselves.

Another likely benefit to the “social construction” of knowledge is the conceptualization of knowledge. You are asking for more than rote memorization on content. In order for the families to educate their peers; they have to attain some level of competence with the information. In developing these conceptual frameworks, they are likely in turn reinforcing neural pathways for retrieval. Research demonstrates that interconnection of concepts measurably increases myelin density between the connections (often considered a surrogate for increased knowledge, skill, ability). Just as students who learn math as a rote skill have difficulty solving word problems, students taught facts instead of frameworks have trouble with clinical application of knowledge. The former is proven while the latter is conjecture on my part.

I guess that’s enough for now. Looking forward to everyone’s input!

jeff riddell

Interesting thoughts Jason. Good to see that your program has been doing this for a few years.

To your point about preparation, I like the idea that the peer pressure of knowing you’re going to have to contribute to the group is more incentive that simply preparign for oneself. I wonder if that same pressure could work for us with PBL, flipped classroom, or other active learning models.

Regarding social construction, you’re right on the money. The AMEE guide listed as a reference above (http://www.ncbi.nlm.nih.gov/pubmed/22471941) says, “… gone also are the days in which medical education programs can be satisfied if their graduates have successfully mastered the content of their courses. Current and future practitioners need to be able to both use the content and to be able to solve problems by working effectively with and accessing the combined knowledge of a diverse team of healthcare professionals.”

I have a question about TBL experience, Jason. I have no experience with TBL but definitely see the benefits of engaged, active, small group learning with educator-framed, higher-order application learning objectives. In the AMEE paper (http://www.tandfonline.com/doi/pdf/10.3109/0142159X.2012.651179), it lists PEER EVALUATION as another component of the TBL experience to book end the in-classroom learning activities at the end. So:

Before class: learning material on own
During class: IRAT, TRAT, Instructor review, Group application
After class: peer evaluation

The AMEE paper suggests having each learner get comfortable with giving constructive feedback especially in the prevalence of team-based patient care today. Such questions to ask might include:

– ‘‘What is the single most valuable contribution this person makes to your team?’’
– ‘‘What is the single most important thing this person could do to more effectively help your team?’’

Do you do this peer evaluation, and do you have an orientation on these elements of all the TBL components for the learners. All this newness may be quite jarring, I imagine, if you haven’t done TBL before.

jason wagner

We have done the full TBL w/ the Peer eval, but to be honest we don’t do it every time. It is often uncomfortable for the learners. The exception to this are when members contribute very little. Peers then appreciate a formal way to let them know this. It’s also nice, as it provides more consistent feedback to residents about their perceived teamwork. This is more beneficial than only hearing this during semi-annuals. While we have not used this every time, I feel it’s a component we need to enforce more routinely.

At times, I think the iRAT and gRAT sometimes make it feel like I’m “checking up” on the residents. I think more and more as you build a culture of preparation, you need less and less of these to fulfill the promise of TBL.

At McMaster University, the predominant framework is Problem-based learning (PBL), which shares many common elements of TBL, but is less regimented. Group learning likely does improve understanding (via a community of practice sort of model, especially if you use mixed-level groups and have the senior residents guiding the juniors), but you can also imagine how this technique might favour outgoing folks (just like PBL can).

For answering Q1: As I stated before, I think there is inherent benefit to having mixed-year groups. Having the senior residents in a group allow them to develop teaching/mentoring skills, and also serve as a sounding board for junior colleagues – but these expectations should be laid out from the get go. I think that intentionally mixing groups is quite important – and I would say that awkwardly waiting for residents to select their own small groups can often take away key classroom time. I would suggest that for purposes of efficiency, it might be best to simply pre-assign the teams, and post this on the wall so people can sit in teams right off the bat. There may also be merit in having the same teams for a whole chunk of time, as based on Tuckman’s model (http://www.businessballs.com/tuckmanformingstormingnormingperforming.htm), high performing teams may take time to go through the typical steps of team formation in order to hit their stride.

jeff riddell

Agree with your answer to Q1 – specifically cultivated. And diverse if possible.

Yup, it makes sense that teams would be fairly stable over time and diverse. I’m not sure who said this but “diversity trumps ability”. Having a diverse set of foundational knowledge, perspectives, and backgrounds lends itself to a broader range of opinions and potentially more creative solutions/ideas — more so than having 4 homogenous superstars. Breaking up into residency “families” — one from each year like Jason’s example makes sense.

Catherine Patocka

Great thoughts on the iRAT/gRAT concept Teresa- I think the intention of these exercises is to check up on learners and ensure they are arriving prepared. Perhaps this concept worked better in it’s original setting of business school where competition was appreciated and encouraged. Also, these assurance tests were used in the determination of final grades (hence further incentive for students to prepare), I’m not sure how this translates into graduate medical education where residents don’t get formal grades for such activities and thus need different sources of motivation.

There is indeed many similarities between PBL ad TBL (both are small group instructional strategies and promote active learning to name a few), however one of the key differences is that in PBL learning goals are student-directed based on what they decide they need to know about a topic, whereas in TBL the learning objectives are determined by the instructor and the focus for the learner is on the application of what they have learned to solve a real-world problem. I think this difference is reflected in some of the the other comments regarding the need for faculty investment initially.

jason wagner

I alluded to it, but didn’t outright state it.
At WashU we have families. Each family consists of 2 residents from each class (4 yr program) along w/ 2 parents (faculty). These families progress as a unit throughout residency, so the residents get a fairly consistent experience throughout residency as far as family goes. Our TBL, Sim, PBL, etc (really any small group) are based on the family model. Sometimes we will modify families if there are big conflicts, but in several years we’ve only done this once or twice.

jeff riddell

Great idea. Would you (or do you) disclose to the learners exactly how the teams or families are chosen?

jason wagner

Our families are chosen well in advance. Each incoming intern is assigned a family after orientation is over. We try to select families to compliment strengths & weaknesses, backgrounds, behavioral traits. The goal is to create a dynamic team.

TBL being used at national conferences. Interesting twist, with many challenges.

Eric Morley

Great topic. We have used different approaches over the years at Stony Brook and Kings County before that. Here are some of my thoughts:

1. We used to put people in small groups made of of at least 1 resident from each PGY and 1 faculty. Generally I would have them work through a case with pre-defined questions/scenarios to work through. We used a tiered approach where PGY 1 would be responsible for basic core content, PGY 2 for more advanced, and PGY 3 would typically be responsible for presenting cutting edge literature. Faculty need to be adequately prepped for this. They need to get the reading well in advance and have clear learning objectives. When done well faculty and residents love this type of activity. The best sessions hit the core learning objectives but also leave room for faculty to discuss their own nuanced approach to certain clinical issues.

2. I feel the iRAT and gRAT were counterproductive and wasted valuable time. I’m not sure what to do with a resident that didn’t prepare. Remove them from the session? If not, does it only serve to make the resident feel a bit humiliated? That being said I haven’t figured out a way to ensure residents prepare prior for the session. I find 70% will prepare. I think one of the keys is making sure you pick high-yield really well done material for them. This generally takes the most time in preparing one of these sessions, but can make or break the experience. You also have to temper your enthusiasm and not select too many resources for them to read.

This spring I will likely be changing our “Sim” days to look a lot like what Jason Wagner described. Limit the TBL/PBL station to 1 group at a time (therefore only 1 faculty has to prep for this activity). They will rotate through other stations like sim, oral boards, and procedures that reinforce what they are learning in the case-based discussion. I think this is the only way to balance the clear need for this type of activity (in my opinion) and time commitment it requires. I’m not convinced yet that I need to add a RAT though.

Thanks for sharing your experience with TBL, Eric. Same question as for Jason above. Do you include a peer evaluation of the process? This may be a rich opportunity for learners to get feedback about how they perform on a team — which is the crux of our day-to-day medical career which often goes overlooked.

Eric Morley

I haven’t, but would consider it. Not sure it would add any more information than what the faculty group leader already knows. Becomes clear pretty quickly who didn’t prepare. However, I could see how trending this kind of data could be used to measure professonalism.

Would love to hear from others if they are doing this and what they do with the data.

Agree with many of the ideas already shared. We typically do a 1 hour small-group session each month (outside of simulation workshops) where a single faculty member (usually me or one of the other APDs) discusses a case to frame a discussion. Then questions are posed to the group to discuss within their small groups. We then bring all the groups together to discuss the “answers.” Residents are given resources to read in advance.

I’m not a fan of the assessment tests. Agree with Eric that they waste time (mine in preparing + live time in conference). I find the residents are highly engaged in these sessions and most prepare before coming in. Those that don’t are often shamed by their peers instead of me doing it.

I also struggle with the post-assessment part. Spaced learning with simulation? Haven’t tried it but I like the idea.

I think any effort to drop group size will lead to increased advanced preparation and involvement. These workshops work particularly well because they are not faculty intensive (the live part) though the prep work in advance can be quite time consuming. Works great for residencies with limited core faculty with protected teaching time.

I’m interested to hear other’s thoughts particularly on the post-workshop evaluation of learning.

jeff riddell

Thanks for your thoughts Swami. You mention some of the challenges of TBL that may be more specific to graduate medical education.

Given that those with experience like some, but not all, of TBL, I wonder if it is (to borrow @Weir_Alec ‘s analogy) like Early Goal Directed Therapy for sepsis. We know it works in the bundle. But what are the key parts of the bundle that really matter?

Can we get rid of CVO2 monitoring and still show the benefit? Do we have to do all the work to see the benefit? Or are there key pieces (early antibiotics, fluids) that really make the key mortality difference?

What are the early antiboitcs and fluids in TBL? Or maybe we really do need the whole package to see the learning gains.

There is a significant body of literature that shows retrieval practice (answering questions from memory) works. Not just to assess readiness, but to make the info sticks. Like when you study for a board exam – you do lots of questions. So I wonder if the testing, though time-consuming, are key to the overall bundle. What do you all think?

Catherine Patocka

Great analogy Jeff, I agree it may be important to try and understand what part (if any) of the TBL bundle is most useful in graduate medical education.

On the other hand I can appreciate the time-consuming and tedious nature of preparing these assessments, it sounds like people are always developing these sessions from scratch… does anybody have experience in repeating the same session (for example doing it for the PGY-2s every year)? and if so, does it get easier to facilitate/prepare for repeated TBL sessions.

Eric Morley

Catherine. What are your thoughts on post small group testing through related sim or oral board style cases? I’m finding myself wanting to move more in this direction.

Eric Morley

I’m also considering using sim as a kind of “pre-test” which will set up the small group exercise that will drive home learning objective from prep material.

Catherine Patocka

Great ideas Eric, if you were a purist and wanted to stick with the key components of TBL, then doing the sim pre-test (replacing the Irat/grat) would be the way to go ensuring that the sim scenario addresses the key learning objectives you wanted the learners to get from the preparation material, for example if your reading was about sepsis, then you might give them a typical pneumosepsis case to go through as a pre-test, after this you would debrief and clarify concepts that the learners were unclear on and after this move to the application exercise (where you could put your more complicated, real-world scenario) for the group to discuss/address.

With regards to the testing effect, testing pre or post should have the beneficial effects mentioned above (improved retention and storage) and although I am not familiar with any literature specifically looking at the testing effect in simulation (though I seem to recall some work being done on testing prior to simulation), active learning in general was found to have a more robust testing effect than passive learning.

Doug Franzen

I looked in to TBL a few years back after learning about it in an instructional methods course. One thing that is missing from the summary above and is only briefly mentioned in the paper is team accountability. In the initial description of team based burning, teams were aborted a number of points towards their final grade in a course. Teams assigned points to individual members based on each member’s participation/contribution. The motivation of a final grade helped ensure that all team members contributed. From Parmelee DX, Michaelsen LK. Twelve tips for doing
effective team-based learning (TBL). Med Teach.
2010;32(2):118–122. (reference 3 in the paper for discussion)

“Tip 7: Highlight accountability as the cornerstone of TBL”

The only thing I can see in the paper under discussion that looks like accountability is that on the third day of each exercise, team members “were asked to comment on 1 thing
they appreciated and requested of each group
member.” (p644) As is well described in team literature, lack of accountability can lead to “social loafing” which may very well have led to the inconsistent participation noted under negative feedback (p646) – a few residents may have “dominated” the discussion because other residents weren’t participating.

Unfortunately, grades aren’t a part of residency and so cannot be used as a mechanism to build accountability. I suppose that the “shame” of knowing you didn’t contribute to a team might motivate some, more so if they were “called out” for their lack of participation in the feedback/peer comments. It would be interesting to see if 1) there were any comments along those lines and 2) if members that did receive such feedback increased their participation in subsequent sessions.

When you have good group buy in, TBL can be fun/enjoyable/successful. But when some residents consistently don’t prepare for the sessions, it can build resentment among the other team members who did the pre-work, ultimately leading to team dysfunction and dislike of the overall TBL format.

jeff riddell

Doug – appreciate the point about the role of grades in TBL.

You’re correct, most of the literature around TBL does involve points and grades and it really can’t be an accountability issue for GME. And you’re right to point out this mixed response to feedback issue from the community.

I think the lack of preparation for sessions is an issue that many program faculty have been wrestling with as many move to active learning. Flipped classroom and small group learning proponents have faced uphill battles on this front as well, though it seems that many have found residents prepare better as the culture slowly gets built.

So then I wonder how to build that culture as expediently as possible so we don’t love hope and dislike formats like TBL during the “getting used to it” phase.

“I saw that you were having a problem with the TBL use in residents and what to do if a resident didn’t do the preparatory work for the class.

Having had a lot of experience in this, here are my two cents:

1. The Individual RAT should count towards residency evaluations, making the resident motivated to prepare properly.

2. The resident’s performance with their group on both the Group Rat AND the Application Exercises should be incorporated in the peer evaluation portion of the exercise, which should become part of the resident’s portfolio of competencies, specifically teamwork.”

Hope that this helps you. My experience using TBL was in a pass fail EBM course that was in a mostly graded curriculum. I had to make a lot of modifications, but generally have a good idea of what works and what doesnt.

Also, are you in the TBL Collaborative? There are a lot of good resources in that group, although most of the medical ones are in medical student education rather than GME.”

Doug Franzen

Using the prep work towards residency evaluations is a good thought, although in my opinion doesn’t have quite the same motivating power as grades do at the UG or UME level. As an APD and member of the CCC, I really can’t see putting a resident on a remediation plan simply because they didn’t do their pre-reading; however, it could be used along with other data points (poor teamwork skills, late paperwork, etc).
I think Jeff’s question about building a culture of preparation is a good one to consider – especially with the flipped classroom becoming such a popular thing. I think creating teams with residents from each PGY (as Eric Morley mentions) is one way to do this – as new interns come in, they see that PGY2’s and 3’s are prepared and absorb that expectation. (Of course, that means ensuring the senior residents are indeed coming prepared…As with any cultural expectation, it’s hard to get started but easy to maintain once it’s in place) I also think it’s important to consider resident workloads when setting up any sort of session that requires out-of-class preparation. Sessions that require prep should have plenty of advance notice and reasonable prep requirements so that residents can incorporate the prepwork into their busy schedules.

jeff riddell

Thank you all so much for joining the conversation. It has been a great discussion here and on Twitter.

Stay tuned for a curated summary of the discussion coming soon.

Pik Mukherji

Thanks to all for letting me follow along w a great discussion.

Andy Little

We do a similar but not exact version of this @DoctorsEMres We do 2 hours of group discussion each week over the readings from Rosen’s. We break up into 4-6 groups (with learners from years 1-4 and attendings), where we work through 5 cases that have been made (we use Google forms as our format to make/administer these), each case covers high yield topics from the reading with questions designed for 1st years up to 4th years.The answers for each case are also reviewed in these small groups, with a portion of the google form allowing for an “ask an expert” section, where the group is able to ask a faculty member to discuss point brought up in there discussions (management, diagnosis, etc.). This has been a huge improvement to just getting death by powerpoint.

Ali R. Jalali

Great to see all the interest and discussions.
Here are few interesting links:

ALiEM Affiliates

The future of health professions education at ALiEM's own e-course site at ALiEMU.com, featuring the CAPSULES and AIR series.

NEW POST NOTIFICATION

About

This blog aims to disrupt how medical providers and trainees can gain public access to high-quality, educational content while also engaging in a dialogue about best-practices in EM and medical education. We strive to reshape medical education and academia in their evolution beyond the traditional classroom.

WestJEM journal partnership

We are proud to partner with the Western Journal of Emergency Medicine to add another layer of legitimate peer review to existing and original ALiEM content to advance the specialty of EM and health professions education worldwide.