In this Dialogue, Diana Dolmans, Professor in the School of Health Professions Education (SHE) at Maastricht University, and David Gijbels, Professor in the Department of Training and Educational Sciences at the University of Antwerp, conduct an e-mail discourse about current thinking and future approaches and challenges to research in, and application of, problem-based learning (PBL) in the learning environment.

Diana Dolmans:

Many claims are made for problem-based learning (PBL), which has been implemented worldwide across many schools. But what are these claims and what evidence is available so far? Does PBL enhance domain-specific knowledge acquisition? Does PBL enhance the application and transfer of domain-specific knowledge? Does PBL enhance the development of generic lifelong learning competencies? Does PBL enhance deep learning? Under which conditions is PBL effective and for what kinds of students? What are the working ingredients of PBL and why does PBL work or not work? There are many questions to be answered about PBL. You have conducted several reviews that shed light on some of the issues raised here and therefore I wonder what your opinions are on some of the claims made for PBL and on how to advance research in this domain.

David Gijbels:

Thank you for inviting me to this discussion and for sharing your questions with me. In a contribution on student achievement in PBL,1 we looked at the numerous reviews of research into PBL that have been conducted since the influential review by Albanese and Mitchell2 and came to the same conclusion as Strobel and van Barneveld3 in their meta-synthesis of meta-analyses comparing PBL with learning in conventional classrooms: (i) traditional learning approaches tend to produce better outcomes on assessments of basic science knowledge, but don’t always do so, and (ii) PBL approaches tend to produce better outcomes in terms of clinical knowledge and skills. The more recent meta-analyses point to the important roles of several mediating variables such as the type of problem tasks4 and the level of assessment5 in explaining the effects of PBL. In my opinion, when implementing or investigating PBL, much more attention should be paid to the conditions under which the effects of PBL can be maximised. This also links to the question of what should be considered as PBL and what learning environments should be labelled otherwise. In other words: what kind of learning environment are we talking about when we use the PBL label? When writing reviews on PBL myself, I noted that although a learning environment may be labelled as PBL in a research paper, important information about the details of the learning environment is often lacking. (For example: what kinds of problems are used? How many students are enrolled in a group? What is the role of the tutor? How much time is allocated to discussion? How are students assessed?) Furthermore, individual differences in students are not taken into account enough in discussions of the effects of learning environments such as PBL.6

To conclude, assessing a PBL versus a conventional lecture-based learning environment (or, more broadly, assessing whether the claims of PBL are justified) is probably less important than the following question: what kind of learning environment works for what kind of student at what moment in his or her career and why? It is my opinion that at present we need more research that takes the variety in essential elements in the learning environment and in students’ learning into account in order to answer this question.

I wonder whether or not you agree with my analyses and what kind of research you would like to see on the research agenda in order to foster our understanding of students’ learning in PBL and beyond.

DD:

Yes, I fully agree that it is important to define what type of learning environment should be labelled as PBL. But I assume that we both agree that PBL can be characterised by: (i) learning in small groups; (ii) a teacher facilitating learning in the group; (iii) learning by means of problems that are first discussed in the group, and (iv) learning by means of self-study after which a discussion in the group follows. Problem-based learning is a student-centred instructional approach in which the problem is the stimulus for learning, as is emphasised by Barrows and Tamblyn.7 The problem is the trigger for students’ learning, for raising questions, searching for information and for self-study, and the ultimate aim is to better understand the problem.

And indeed, we need many more studies on the conditions under which the effects of PBL can be maximised. Your review5 is a good example and clearly demonstrates that PBL has positive effects on tests that measure understanding of principles that link concepts (tests in which students have to solve problems) and probably also on tests that measure the application of knowledge (simulations or performances in practice), although the number of studies is limited for the latter type of assessment. The review by Walker and Leary4 is another example of a review focusing on mediating variables such as the differential effects of different problem types within PBL. However, the data about the effects of different problem types within this review must be interpreted with a great deal of caution, as the authors4 themselves argue, because only a limited number of outcomes are available.

Other examples of studies focusing on the conditions under which PBL is effective were published in the early 1990s. These studies were aimed at investigating the conditions under which a tutor’s subject matter expertise influences student achievement. The results demonstrated that if a PBL unit lacks structure or if a PBL unit does not fit well to the students’ level of prior knowledge, students guided by a subject matter tutor perform better than students guided by a non-subject matter expert tutor.8 In a similar study conducted somewhat later in another PBL curriculum, these findings could not be replicated.9 A closer comparison of the data of the two studies demonstrated that the structure of the PBL units and the degree to which the units fit with students’ level of prior knowledge showed less variation in the latter study than in the previous study. Thus, the effect of a tutor’s subject matter expertise on student performance seems to differ across situations. In a well-structured curriculum, a tutor’s subject matter expertise seems to be less influential in terms of student achievement. If the PBL unit is well designed, the PBL problems work more or less on their own, which means the subject guidance offered by the tutor is less needed. Of course, a tutor preferably has some subject matter expertise and some capacity to facilitate the learning process,10 but the context or structure of a curriculum also plays a role. So, reviews in which we analyse whether PBL has positive or negative outcomes and whether this differs under various conditions are of importance, but it is also important to explain why some studies demonstrate contradictory findings, as illustrated in the example just given.

Furthermore, we need studies that go beyond measuring the outcomes of PBL in terms of student achievement. For example, what do we know about how students perceive the PBL environment? Does PBL encourage students towards deep learning? Does PBL stimulate students towards less rote learning and more self-directed learning, under which conditions, and why? I wonder what in your opinion is the evidence so far on whether PBL does lead towards deeper learning, under which conditions and why.

DG:

Yes, indeed, we both agree on a broad definition of what kind of learning environment could be labelled as PBL. The issue I wanted to raise is that in several research papers, very little attention is paid to the description of the learning environment and that in order to strengthen our understanding of student learning in PBL, such a description would be useful. For example, at the American Educational Research Association conference in Vancouver last year, I noticed that, when asking for more clarification of the learning environment, several contributions from the Special Interest Group for PBL would not fit with the definition of PBL you describe and I agree upon. This is only a problem if the authors fail to describe the learning environment and simply call it ‘PBL’ (although even a term like ‘small group’ is ambiguous as it might refer to a group of 100 or more students…)

The question of whether PBL leads towards deeper learning, under which conditions and why, is an interesting one. In a recent article,11 we included a small review of the literature on the way students approach their learning in PBL. So far, research related to the effects of PBL on student approaches to learning (SAL) seems to be rather inconclusive: some studies report that PBL can foster desirable approaches to learning, whereas others do not.12,13 Based on my reading of the literature, it seems that creating curricula that foster and encourage deep approaches to learning is not an easy task. I want to make some remarks that relate to the research that has produced these results so far. A lot of the research is cross-sectional in nature14 and as a consequence cannot resolve whether or not PBL can foster desirable approaches to learning. In order to establish whether or not PBL fosters deep approaches to learning, we need to look at research with a longitudinal design and this research is much more scarce and the results of those studies that do exist are inconclusive.

A promising approach from this perspective is the so-called ‘person-oriented approach’,15 which takes individual differences between students into account. It seems fair to assume that students with various learning characteristics are differentially affected by a learning environment such as PBL and, therefore, differ in the way they develop their learning approaches throughout this learning environment. It could be hypothesised that this variety of differential trends may to a degree oppose one another, resulting in unexpected effects at the whole-group level or a lack of effect altogether (which might explain the inconclusive results in the research so far). Nijhuis et al.16 demonstrated, for instance, that students in PBL could be divided into subgroups according to variability in their learning approaches. Some students were found to be more variable in their learning approaches across courses, while others consistently stuck to a similar approach. In research taking a person-oriented approach, it is argued that the way students engage in learning is not necessarily constant.6 Students might also adopt specific approaches to learning that combine aspects that are indicative of both the (theoretical) deep approach and a surface approach. Moreover, how and to what degree students combine these approaches may be a relevant issue. Therefore, combining students’ scores on various learning approaches and their interrelations into a kind of ‘learning profile’ may provide a nuanced and multifaceted picture of how students approach their learning in PBL. These profiles can then be used in explaining differential developments in learning approaches. In other words, the effects of PBL on students’ approaches to learning (and by extension on other learning outcomes) could be different for students with different profiles. It would be worthwhile to further investigate the interaction between variations in the implementation of PBL, and variation in students’ profiles.

DD:

Whether PBL will foster a deep approach to learning will be dependent on various factors. The alignment between the assessment and the learning environment is crucial, as is how students perceive the assessment and the learning environment. For example, does the environment encourage application of knowledge? Does the assessment provide feedback?17 Baeten et al.18 reported in their review paper different factors that hinder and stimulate deep learning. If a teacher reports the use of a conceptual change in his or her approach to teaching, students report studying more deeply, whereas if teachers report the predominant use of an information transmission approach, students use a more surface-based approach to learning. An excessive workload is negatively related to deep learning. However, this review also mentions the characteristics of students as being influential, such as whether students are intrinsically motivated. Investigating variations in students, as you propose, is worthwhile, especially if it gives insight into how we can support students to become deep and intrinsically motivated self-regulating learners. Ten Cate et al.19 argue, based on the theory of self-determination of Ryan and Deci,20 that autonomy-supportive teaching behaviour as opposed to controlling teacher behaviour is crucial. They argue that students’ perceptions about PBL might be positive because PBL may create a certain amount of autonomy (i.e. students generate their own learning issues), because PBL may stimulate relatedness (i.e. students work together in a group), and because PBL might enhance feelings of competence (i.e. students can explain something in their own words to others).19 Yet what do we know so far about how PBL enables students to become intrinsically self-regulating learners? Not that much, in my opinion. Rotgans and Schmidt21 demonstrated that PBL problems trigger students’ situational interest, probably because students become aware of their own lack of knowledge and therefore develop curiosity. However, much more evidence is needed about the process of PBL and how this influences student learning. A more microanalytic approach, as it is called by Schmidt et al.,22 to studying PBL is needed. What actually happens to the learner in PBL? How does PBL bring students to work autonomously, to feel competent and to feel related to others and to become intrinsically self-regulating and deep learners? What is your perspective on the importance of microanalytic approaches in investigating the process of PBL?

DG:

Self-determination theory provides an interesting framework from which to take a more microanalytic approach to investigating the process of PBL. Take, for example, the argument that PBL may stimulate relatedness because students work together in a group. I agree that working together in a group is an important distinctive element of a PBL environment. In a study in which we compared students’ perceptions of the learning environment in PBL with students’ perceptions of the learning environment in a conventional lecture-based environment, being confronted with different ideas and being able to share ideas with others appeared to be the two aspects in which students’ perceptions of the learning environments differed the most.14 Nevertheless, although most practitioners and researchers would agree that working in small groups is essential for PBL, I dare to doubt whether much attention is paid to the underlying principles that make working in small groups potentially beneficial for a large group of students (e.g. by fostering a beneficial interpersonal context for sharing ideas with others or by introducing problem-based tasks that require interdependence between students [such as in the work of van den Bossche23]). So, I support the idea that a microanalytic approach to investigating the process of PBL is needed. These questions seem much more necessary and sensible to me than the question of which learning environment is better. Which microanalytic aspects should be studied first, do you think?

DD:

Examples of microanalytic studies are studies in which different types of interaction in groups are investigated, such as in a study conducted by Visschers-Pleijers et al.24 This observational study demonstrated that, although task involvement was very high in the groups, relatively little time was spent on exploratory questioning (i.e. open questions, critical questions, verification questions) and handling cognitive disagreements (i.e. giving counter arguments) within the tutorial groups. More recently, Aarnio et al.25 demonstrated in an observational study that conflicts or discussions about disagreement took only a very small percentage of the time in tutorial groups and were very short. These studies make it clear that the quality of interactions in PBL requires further improvement and that both students and tutors need further training and feedback on how to enhance a deep group discussion.

In order to improve the quality of the discussions in the group, Kamp et al.26 investigated whether peer process feedback (i.e. in which students give each other feedback on their contributions to the discussion in the group) has a positive effect on the students’ contributions to the group discussion. Kamp et al.26 demonstrated that peer feedback does work for students with below-average scores (i.e. the quality of their contributions improved 4 weeks after the feedback was received). The qualitative data from this study demonstrated that peer feedback increased the students’ sense of belonging to the group and that they felt closer to others, which might have positive effects on their learning. These studies were all conducted in tutorial groups with homogeneous student populations, so how does this work in tutorial groups in which students from diverse and heterogeneous backgrounds collaborate, and groups in which students differ in terms of language, academic background and cultural background? What about students who are less used to speaking up in a group? Not much is known yet about PBL with diverse student teams and students with diverse backgrounds. And, of course, it isn’t only tutorial group processes that need further study; studies focusing on how tutors can scaffold students’ learning in the group are also required. And last but certainly not least, studies focusing on the role of PBL problems are likely to be important. As I argued before, the PBL problem should preferably do the work on its own. Thus, I think we need not only studies in which we investigate whether using a real patient problem leads to better student outcomes than a paper-based or video-based problem, but studies in which, for example, the types of knowledge that are investigated are discussed along with the types of learning interactions that take place in a tutorial group when using different types of problem. Do you have any thoughts or suggestions on studies that are needed in this area?

DG:

Research on self-directed learning (SDL) in PBL is badly needed because it refers to the claim that PBL is able to foster students to become lifelong learners. From this perspective, it seems not only relevant to investigate which microanalytic aspects of the learning environment are linked to the development of SDL, but also to look at the effects in the longer term. If we find evidence that a certain approach can stimulate students’ SDL (the microanalytic approach), we must still establish whether these effects survive the transition from higher education to learning in the workplace and beyond. Several studies are possible from this perspective, but it would probably be wise to start with a good state-of the-art article on SDL in PBL.

DD:

Yes, great idea. Hopefully others will also be inspired to conduct research on how the different elements of a PBL environment can be optimised for what kind of student, under which conditions and why.

Contributors: this manuscript is a transcription of an original e-mail correspondence that took place between DD and DG.