The Hard Problem of Teaching

Tomorrow, we have our final classes of the year in IdeaMath, a weekend contest math program run by former US International Math Olympiad team coach Zuming Feng that I’ve been teaching at for the last five years. It’s always tough to say goodbye, having spent over a dozen Saturday afternoons with these middle and high school students, helping to teach them problem solving skills and having some fun along the way.

I’m not yet sure whether I’ll be returning to the program in the fall during my last year of grad school here, so this could be my last regular teaching opportunity in grad school, or possibly ever. I joined IdeaMath in my first year partly as a way to give back to the math contest community that I grew up in, and partly as a way to keep up my involvement with teaching while in a graduate program with a light teaching load.

My teaching experiences at MIT were also very positive — in fact, between the teaching I’ve done online with the Art of Problem Solving, Caltech, and MIT, I’ve somehow managed to help teach five different calculus classes. Some were aimed at the strongest students, and some at the weakest (albeit the weakest Caltech and MIT students). Some attempted to be fully rigorous, while others simply provided an upgraded version of Calculus BC. All were very rewarding personally, as I got to see students grasp the material for their first time.

Given my experiences and passion for teaching, I’ve often been asked if teaching is a career I’d consider. The question makes sense; I like to teach. I enjoy being able to inspire another generation of students with neat tricks, clever ideas, and powerful results. Even more than inspiring, I enjoy bringing clarity, helping students better grasp important concepts and form appropriate intuitions around them.

That passion and experience has made me into quite a proficient teacher, if I do say so myself. The MIT Math department administrator was quite impressed with my teaching ratings from students at MIT and my senior year at Caltech, I received a teaching prize meant for graduate students for my TA work there. In a surreal turn of events, one day the Caltech math department head called me into his office, wondering if he could pay me to essentially rescue a statistics class that had gone awry from poor teaching by holding a bunch of recitations and office hours. (I turned him down since I was too busy at that point, and suggested that he make the same offer to a few of the TAs instead.) Given my false starts with various research projects in grad school, it seems pretty clear that I’m generally better at teaching than research.

And yet, despite this passion, experience and success, I actually don’t see myself continuing to teach full-time or long-term. Why not?

It’s not because teaching is less lucrative or prestigious than other career directions. There’s definitely a widespread sentiment at the schools I’ve been to (though Caltech more than MIT) that research is where the most important work is done, and teaching is either secondary or a distraction. I find that attitude selfish and short-sighted. Yet even out of that sort of environment, I’ve seen close friends who have chosen to teach, an honorable decision.

But I could never do it myself.

What it’s like to be in charge

This realization began with my last official teaching experience at MIT. I was put in charge of a month-long remedial calculus class specifically for students who fell just short of passing in their first semester. There were only six students in the class, and it was up to me to decide everything: when and for how long we met, what material I would cover and focus on, how the tests would be structured, and how well they’d need to do to pass. The wide range of choices was daunting.

Fortunately, I had a bunch of help in the form of materials and schedules from previous years, as well as the lecture notes and tests for the calculus class that these students had just taken in the fall. I ended up mimicking much of the previous schedule, with practice problem packets and tests every week.

I did choose to rearrange the topics, since the class seemed to more neatly divide into three parts, which let me use the last week for review and a big final test rather than covering more material. As I did that, though, I realized that I would have to shorten the coverage of some topics over others. Did the students really need to learn how to calculate the surface area of a volume of revolution? How much time should I spend on integration techniques like partial fractions?

The other big question was the pass line: How well did the students need to do for me to sign off on them “knowing calculus?” I decided it would only be fair to announce a line at the beginning of the month, and let them land where they did. Fortunately, in the end, it was pretty clear to me based on effort which students “should” have passed, and the criterion I set gave me the correct result.

But the experience still raised major questions in my mind going forward. As I discussed this with my adviser, he explained that everyone who teaches starts to wonder those things, and being in full charge of that class gave me that experience a little bit earlier than most.

Why do they need to learn?

As I reflected more on this experience, my concerns about the pass line faded — tests might not be perfect measures of ability, but they’re probably the best we can work with, especially in a month’s time. By contrast, my concerns about how we choose what to teach only kept growing.

I realized that I still didn’t have answers to any of my previous questions. Are students ever going to need to calculate the surface area of volumes of revolution again? Even if they will, do they need to know how to derive the formula, or just get the intuition that it exists and they could look it up if they needed to?

Once that can of worms is opened, there’s no going back. Do students really need to be able to integrate by hand when they have Wolfram Alpha? Do they even need to know the limit definition of a derivative, or will they be just fine understanding it intuitively? Do these somewhat silly word problems we write about how the height of a cone changes if its volume is decreasing at a certain rate actually bear any significance to problems they’ll face in the future?

The heuristics of tradition and feedback are rather useless to answering this question in practice. Tradition only helps keep a curriculum the same; it doesn’t adapt to a world where Wolfram Alpha is now accessible to anyone with an internet connection.

Feedback and communication do sound more promising in theory. In an ideal world, professors would spend their copious hours of free time comparing curricula and asking these questions of each other. Alumni would give feedback about which classes ended up being most useful to their careers, and what they wish they’d learned but hadn’t. Prospective employers would tell departments what classes they’d like to see taught. Departments would conduct regular curriculum audits to make sure that class time was spent on the most valuable topics possible.

But in reality, very little of that ideal world exists. I actually got a better glimpse of this than most during my time at Caltech, where I served on the undergraduate-led Academics and Research Committee for a couple years. It was encouraging to be part of an organization that played a small role in improving academics at Caltech, primarily by voicing student frustration with notorious classes and pressuring professors and departments to improve them.

At the same time, although I didn’t realize it then, serving on the ARC showed me how little of that ideal world existed. One of the major initiatives of the Caltech administration comes to mind: They wanted to encourage us to write more, mainly (if I recall correctly) by adding an additional requirement to take a certain number of writing-intensive humanities classes. This was a big deal and occupied a bunch of their time as they had to get this approved by the faculty board.

The motivation behind this change, however, was rather flimsy at best. I saw multiple presentations on the subject, and it all came back to a single question on an alumni survey. On a scale, alumni were asked to rate how much they improved while at Caltech in various ways. Compared with understanding their field of science or how to do research, many fewer alumni said that Caltech improved their ability to write.

This would seem to be a clear-cut example of the ideal world I’ve described, but there were several problems. For one, the question hadn’t clearly distinguished between scientific writing (research papers) and humanities essay writing, and given this, it wasn’t clear that adding more humanities classes with essays would even solve this “problem.” More importantly, the question hadn’t asked whether this lack of writing advancement had actually been detrimental for the alumni in any respect — it could just have reflected the balance of classes that most Caltech students took and still take. Even as a regular blogger myself, I don’t expect that additional humanities essay writing would have significantly helped me.

It got worse. The much bigger shift that started while I was on the ARC was the gutting transition of Caltech’s core requirements down to one year of math and one year of physics (from five trimesters, i.e. 1 2/3 years). This motivation was similarly well-justified: Students in some of the newer (or newly popular) majors like biology and computer science didn’t need to know quantum physics, as nerdy-cool as that requirement was to be able to advertise. Yet as I saw this debate unfold, the main drivers seemed to be essentially emotional: What should it mean to get a degree from Caltech?

Those emotions are certainly valid, and it’s appropriate to weigh the cost to the brand. But tradition is only useful so long as it preserves something of value, and there seemed to be very little attempt to actually gauge the value of a biologist learning quantum mechanics. It’s not that measuring that value would be easy, but it didn’t even seem to occur to some of the smartest people in the world to even try.

Pure versus Applied Education

Reflecting on these events from years ago has also highlighted a broader difference in my own thinking. While I was at Caltech, I thought about academics from a more or less pure perspective: I wanted to learn for interest’s sake, not with any purpose in mind. This led me to take and audit a wide range of classes and even double major in chemistry “just for fun.”

Upon getting to graduate school, though, my thinking matured, especially when I found myself asking for funding to study my own curiosities. I realized that all along the links to applications that would have motivated and utilized my research in pure math were actually much more tenuous than I had expected. This motivated my journey down the chain of potential applications, a journey that very well might end up taking me straight out of academia.

My story and message about teaching are very similar. As I’ve matured and advanced in the teaching world as well, I’ve begun to see that the links and constraints holding curricula together and improving them are also much more tenuous than I had expected or hoped. My observations are obviously limited to my personal experience, but the bit I have seen is not exactly encouraging.

As with pure versus applied research, I don’t think that my motivation to teach has changed so much as I’ve better understood that motivation in light of experience and unflinching self-criticism. To illustrate this, I’d like to share an excerpt from an article by Richard Rusczyk, co-founder of Art of Problem Solving, which speaks to much of my motivation and passion for teaching, then and now:

We use math to teach problem solving because it is the most fundamental logical discipline. Not only is it the foundation upon which sciences are built, it is the clearest way to learn and understand how to develop a rigorous logical argument. There are no loopholes, there are no half-truths. The language of mathematics is precise, as is ‘right’ and ‘wrong’ (or ‘proven’ and ‘unproven’). Success and failure are immediate and indisputable; there isn’t room for subjectivity. This is not to say that those who cannot do math cannot solve problems. There are many paths to strong problem solving skills. Mathematics is the shortest.

Problem solving is crucial in mathematics education because it transcends mathematics. By developing problem solving skills, we learn not only how to tackle math problems, but also how to logically work our way through any problems we may face. The memorizer can only solve problems he has encountered already, but the problem solver can solve problems she’s never seen before. The problem solver is flexible; she can diversify. Above all, she can create.

While inspirational, this message spoke to me as a fundamentally applied motivation for teaching math. And yet, just as I’ve reasoned through in my research, it’s struck me that Rusczyk’s beautiful language should be more like the beginning to the discussion than the end. How do we know that “problem solving skill” generalizes like he claims? Can we actually demonstrate that math is actually the shortest path to it? What’s second-shortest, and does this vary by student? It seems that we could attempt to test this sentiment with a lot more than Rusczyk’s single anecdote preceding this about how those with and without this problem solving skill approached organic chemistry at Princeton.

Again, this isn’t to say that these questions are easy to answer. My point is mainly to highlight how rare such questions even are.

Is the system working?

I don’t mean to suggest that every teacher needs to understand the usefulness of everything they teach. If the system was functioning like it should, this would be the responsibility of department heads, administrators, and education researchers. I’d be happy to just be a teacher if I was confident in the system around me to make sure that what I’m teaching is ultimately useful. It’s precisely because the system has offered me such little confidence that I’d be hesitant to commit my life to it.

Actually, this is one reason why I’ve stuck with the IdeaMath program: Within the self-contained world of math competitions, the goal of improving student performance on that style of problem under time pressure is fairly clear. This isn’t to say that I focus exclusively on this goal, just that it helps to answer these sorts of questions and keep the proper focus of the class. But while it’s been a well-motivated side job, I want the bulk of my life to contribute to more than the arms race around high school math competitions these days.

In that world of math competitions, there’s already a broad understanding that all of the time we spend learning to solve olympiad-level plane geometry problems is basically useless after we leave the competition scene, no matter what we do. Imagine if we raised the same skepticism towards the whole of the typical high school math curriculum: Is trigonometry just a holdout from the days of navigation by compass rather than GPS? Is geometry really the best way to learn logical reasoning (applying a second level of Rusczyk’s argument above)? And why does no one learn statistics?!?

I’m reminded of a high school math teacher who vented to me and others about students who complained about having to learn the quadratic formula. “Why do we need to know how to solve these equations?” they asked, not seeing any practical applications in their lives. He ultimately justified it to them by saying that it would help them get a high-paying job, apparently not very convincingly.

If I became a teacher, those are the sorts of questions I’d ask myself every day. And it would bug me to teach, not knowing their answers. It would haunt me to think of all of the effort I put in just to cram some useless technique into a teenager’s brain. Even if I succeeded and he remembered the quadratic equation for the rest of his life, what difference would it make?

I don’t think I can overstate the significance of this realization. Education is a gigantic sector of the economy and world, affecting every single person for over a decade of their lives, and yet it still hasn’t even tried to answer some of the basic questions about why we do what we do. This feels similar to my realization a couple weeks ago that from a pro-life perspective, a third of all humans are dying before they’re born and, you know, maybe we should try to do something about it.

It’s also similar to the realization that Holden Karnofsky and Elie Hassenfield made a decade ago, when they realized what a mess charitable giving was in, leading them to found the charity evaluator GiveWell. Education is arguably even more long-term and complicated, and no, I’m not looking to start an effective education consulting company.

Is experience the key?

As I’ve concluded that teaching is not for me, I’ve always kept one small caveat: I’m still open to returning to teach some time later in life, after I’ve experienced enough (first- and second-hand) to be able to motivate substantial parts of the curriculum.

How cool would that be to be able to tell a student that I used precisely this technique to solve a problem in a real-world industry? This is related to another outlandish proposal of mine: that academia itself should be populated primarily by late career professionals and retirees rather than recent college graduates with minimal relevant experience. Without going that far, though, the first baby steps towards addressing this problem would be to simply increase the interaction between the professional world and schools, both at the high school and college levels.

Instead, in our current system, those who are responsible for deciding what should be taught typically haven’t spent much time outside academia or the classroom. Career academics and educators are naturally excellent at giving advice and picking curricula that will be relevant for others that follow that narrow career path, but the vast majority of students don’t.

What would this look like in practice? A good friend of mine working at a startup has told me that he’s used something from every engineering class he took in college on just his current project. Clearly some parts of our education system are working! After the project is done, he’s talked about giving a talk at schools which could serve both as motivation for the students and a recruiting pitch for his company.

Even those who do plan to go into academia or education full-time should consider spending some time on the outside. On a similar note, one of my pastors worked at Apple for a year; he’s told me how he values that experience in helping him empathize with those in the more typical working world.

Ultimately, though, that experience will only solve part of the problem. No one can feasibly go out and experience everything, so it will be very hard to know if a piece of curriculum can safely be cut. We’d like to know if a biologist is going to go out and reinvent the trapezoid rule because we stopped teaching it in calculus. Besides, I haven’t even begun to discuss the constraints introduced by standardized testing…

Where this leaves me

“Doesn’t this realization make you want to teach more? You see a problem with the system, and you don’t want to go and fix it? If you don’t start the GiveWell for education, then who will?”

Those are certainly natural questions to ask. But some clarification is in order. When I say that I’m good at teaching, this primarily means two things: that I’m able to find effective methods for transmitting material I’m given, and that I’m good at making students happy (to be honest, that’s all teaching scores measure). It doesn’t mean that I’d be remotely effective at curriculum review, the social science of education research, or building networks of working professionals, some of the ingredients I’ve identified that could potentially fix this mess.

The best I can do for teaching from my current position is to point to what seems like a major problem with education that no one is even trying to address, and hope that someone reading this can do something about it.

Like this:

6 responses to “The Hard Problem of Teaching”

I entered teaching with many of the same questions and concerns. At Stanford and the University of Chicago, I found that I was good at getting pre-med students ready for the MCAT. Physics education research had begun to have an impact at the university level, and the new tools were extremely useful to me and my students. After graduating, I turned it into a nice tutoring business while my wife was in medical school.

After we moved to Fort Collins and my son entered elementary school, I started looking at career options. While physics education research was improving physics instruction at the university level, I couldn’t see much improvement in K-12 education. I got involved with the education and outreach programs for some big science projects, and quickly learned of the complete futility of that endeavor.

Poudre High School seemed like the right place to start looking for answers. I believed that I could only learn to identify and address the problems if I got into a classroom and faced them myself. After ten year of teaching high school (and working with some fantastic students) I am convinced that this was the right thing for me to do.

I’ve moved to Liberty Common High School, where all students are required to take physics. I’m learning how to connect physics with the other areas of their education: music, art, language and history. I’ve redesigned the sequence of topics to address the issues of what students need to know, and to explore the things that many students yearn to know. I’m sorry to hear that quantum mechanics is on the chopping block at Caltech. I teach it to high school kids, as you probably remember!

I have a plan now, which starts with writing a new textbook for algebra based physics. There is a lot to do after that, and I certainly need help. If you, or anyone else reading this post, wants to get involved, contact me (contact info at the Liberty Common High School web page). The challenge still looks big, but it no longer looks too big. Every day I am excited to be working on it.

I’m going to answer a few of the specific issues you raise as my time allows. The most discouraging issue is the one I want to address first. Assuming we figure out what should be taught and how it can be taught, how can we possibly get it out there to make a difference?

The solution to this has become completely clear to me: produce quality resources for a well structured course at a price that schools can afford. Schools are absolutely desperate for good materials and they are being fleeced by the textbook publishing industry. Book prices are ridiculous, the content is often questionable and always outdated, the writing is terrible, and the supplemental resources are mostly useless.

I found a few really great books and they all had similar features:
– An actual author who cares about the content, rather than a committee churring out frequent new editions.
– Less clutter, because they focused on the central content rather than distracting sidebars on careers and unconvincing applications.
– Lower cost, in part due to the reduced size and less dependance on glossy pictures.

In physics there is growing research showing that these sorts of books are actually much more effective, in addition to being cheaper. Two example of calculus-based physics books of this type are “Matter and Interactions” and “Six Ideas that Shaped Physics.” I have not found any books like this for an algebra-based physics course, which is why I am writing my own.

When I taught Calculus III at LCHS, I used a pair of books by Alan Macdonald that really hit these points: “Linear and Geometric Algebra” and “Vector and Geometric Calculus.” They are terrific, each is about 200 pages, and they cost about $30 each. No, I did not forget a zero. I have had many conversations with the author. He was able to keep costs down by self publishing. I am convinced that this is the way to go.

I intend to produce a physics book that is effective and inexpensive, along with good problem sets, assessments, instructor resources, and a lab manual. Then, I intend go to teacher conferences to sell it. If all of that works, it will have a huge impact on how people understand the physical world.

I never would have thought of this without getting into teaching. Aren’t there plenty of good books? Nope. They stink. Can’t teachers get what they need off of the internet? Nope. The deluge of content on the internet is totally incoherent. Teachers don’t have the time or the expertise to find the diamonds in the gravel pile.

At this point, I’ve moved from near despair to confidence. Lots of great teachers are asking for answers to the same questions, and they need those answers in the form of usable materials. In order to produce a good book and good materials I need to be in a classroom using them myself. I am very luck to have the patience of the LCHS community while I pull this together.

When I get a chance I will say something about what we should teach in a physics or math course.

I teach students the truth, to the extent that we understand it, about how the world works. I show them how to apply that truth to understand the world around them right now. I never teach something because it’s going to be practical in some career.

Of course, there are limits. The two great theories we have right now, General Relativity and Quantum Field Theory, are not very accessible using high-school algebra. However, I use these two theories as guides in everything I do, constantly asking myself if what I am teaching is moving my students in the right direction, or leading them astray. This has driven some big changes. For example, I do not teach F=ma. It’s not true for relativistic objects, and photons, which are part of are every-day experience, are always relativistic. I teach that force is the rate of change of momentum. This is correct, and it is also how Newton originally stated his second law.

Bruce Sherwood and Ruth Chabay, the authors of “Matter and Interactions,” gave me the confidence to ditch F=ma. They apply the how-to-teach lessons of physics education research, but they also made bold decisions about what to teach. They wanted a thoroughly modern approach. They teach the four fundamental forces from the beginning. Special relativity is introduced before acceleration. Statistics and quantum mechanics are used to derive the second law of thermodynamics. All of this is in a one semester, calculus-based mechanics course. Obviously, they had to make some choices. Waves, to give one example, are completely gone.

I’m making some different choices in my book, but nothing goes in unless there are multiple reasons for it. It must be part of the story about how we came to understand the universe. It must be current or absolutely necessary for getting to the current understanding. It must connect to things around them right now. For example, we hit music and color hard because they are doing music and art right now and because these reveal deep connections between frequency and energy.

Teachers in other disciplines have been very helpful in answering the what-to-teach question. I have asked them what physics they want students to know, and they have a lot to say. History teachers want them to understand the scientific revolution and the atomic bomb. The music teacher wants that to understand what sound is and how it is made. The Latin teacher is helped me with a lab that uses the Galileo’s original 1613 observations of the moons of Jupiter. The data tables are written in Latin! Kids love these connections, and they aren’t forced because I always connect them to the main ideas of my course.

Now that I’ve got my plan in place, we are doing some Quantum Field Theory. My students are learning the particles and interactions and drawing Feynman diagrams to understand the how weak interactions drive key reactions that power the sun. We do a little GR too, so we can understand the detection of gravitational waves from merging black holes.

The what-to-teach question is a hard one that I have to answer every day, but the goal of connecting current physics knowledge to my student’s experience is a good guide for answering that question.

I haven’t though about math as much, but I have thought about it quite a bit. We need a geometry course based on vectors. I can’t use Euclidian geometry in my class, and if I don’t use it, nobody will. As a string theorist, I’ve spent some time with mathematicians, and even they don’t use Euclidian geometry. Vectors are more practical. They can be manipulated graphically or algebraically. They easily generalize to more dimensions. They can be taught with the same devotion to axioms, theorems and proofs. They are the foundation for many great ideas, like Fourier decomposition. They have a great story with interesting characters and surprising twists. This is on my to-do-list, but it is several years away. If anyone wants it, go for it…and send me a copy when it’s done.

Thank you for this excellent article! I agree that identifying what to teach is probably the hardest problem in education. (In comparison, we know lots and lots about _how_ to teach; but even there, most education systems are slow to take the extant evidence into account.) It also seems widely neglected, at least if you expect something more of an improvement than just stacking additionall skills and requirements on top of existing curricula and syllabuses, more in reaction to pressure from special-interest groups than through any evidence-informed process.

I also think you correctly identified two of the most promising levers for improvement: establishing better feedback loops (either through institutional change or by providing the necessary incentives) and involving professionals from various walks of life in education.

How to achieve this in a short timeframe may be the second hardest question. A dedicated and rigorous educational consulting operation modeled after GiveWell might even work these days, when so much educational content is available globally and ripe for in-depth evaluation and curation. My own approach at the moment is creating a scalable open-source model for specialized micro-schools catering to specific age groups and needs that has short and long feedback loops and community/expert integration built into its basic structure. The idea there is to circumvent the inertia of public school systems by enabling as many people as possible to start their own locally-integrated schools, and at the same time to create incentives for curating and creating high-quality teaching materials that can be shared and evaluated collectively. So far we’re in the intermediate stages of implementing the pilot project (a school for 12-15-year-olds in Austria) and working on making the model accessible and flexible enough to serve different needs up to university level. Your article has been a valuable piece of input for that last part, so thanks again!