Once upon a time there were two farmers, Simplicio and Salvatio. They
were very similarly situated. They operated neighboring tomato farms.
Their fields were nearly the same size.

On harvest day, Simplicio said “I’m going to work really hard
today. I’m going to work so hard that I harvest twice as many
tomatoes as the other guy. I planted twice as many plants, so there
should be no problem. I’m going to turn a huge profit.”

However, there is one slight problem. Because the plants were too
close together, they weakened each other. Then the worms came along
and finished them off. As a result, there are hardly any tomatoes in
Simplicio’s field. It doesn’t matter how hard he works on harvest
day; he can’t harvest tomatoes that aren’t there.

Problem-solving day is like harvest day. You get to harvest and
profit from facts and ideas that are in your brain. The process is
subject to a crucial constraint: You cannot harvest stuff that isn’t
there. If you didn’t put ideas into your head in the proper way, you
won’t be able to get them out when needed, no matter how hard you work
at it.

Let’s be clear: In large measure, your success at problem-solving
depends on what you did during previous days, months, and years. You
have to learn stuff over time. There are right ways and wrong ways to
do this; see e.g. item 1.

When reading this document – or any other writeup on problem-solving
– please be aware of its limitations. There is nothing wrong with
what is said here; the problem is what is not said. There is some
value in problem-solving techniques per se, but you must look
elsewhere to find the other 95% of what you need for effective
learning, thinking, and problem-solving.

Here’s another limitation: A checklist such as this one (or
reference 1) can be useful as a reminder of things you
already mostly know. However, it’s hard to learn new ideas from a
checklist. It’s too condensed. In some cases, an individual item on
this list (e.g. item 46) would become a book chapter
or an entire book if spelled out. If you want to learn, it helps to
have more detail. It helps to see the ideas applied to examples.
There are quite a few good examples in reference 2. An
additional nice example is discussed in reference 3.

Whenever you encounter a new idea, mull it over. Think
about all the ways in which it is connected to – or inconsistent
with – things you already know.

This applies all day every day, whether you are trying to solve a
particular problem or not. It applies to every new idea you
encounter, whether it comes from something you read, or an experiment
you did, or a calculation you did, or whatever. This is discussed in
some detail in reference 4.

The connections are necessary if the idea is to be useful,
because they are what allow you to recall the idea when needed. To
say the same thing the other way, if you put an idea into your brain
but have no way of recalling it when needed, it might as well not be
there. An idea that can be recalled in many ways is vastly more
useful than one that can only be recalled in one way.

As a related point: Even when you’re not working on
a specific problem, look for patterns in the world around you. This
might lead you to formulate a good question or a hypothesis that
needs testing. Conversely, always be on the lookout for anomalies,
for deviations from the expected pattern.

This is important for two reasons:

On a day-to-day basis, it is a
crucial part of the learning process mentioned in item 1 and
discussed in reference 4.

On more rare but more important occasions, it contributes to
scientific creativity, perhaps to the discovery of something new and
important.

Following this advice may slightly increase the workload in the short
run, but it greatly reduces the workload in the long run, especially
when dealing with complex problems. For details on this, see
reference 5.

Checking the work at each step along the way greatly increases the
odds of getting the right answer. In a complex calculation, there
will be mistakes, and showing the work makes it easier to find and fix
the mistakes. Showing the work and showing the checks makes it easier
to check and re-check the work. It is particularly valuable in a
teamwork situation, because it helps other people check the work and
learn from the work. For more on this, see section 4 and
item 15.

Write out a glossary, spelling out what your notation
means, and what your terminology means. Do not rely on the name of
an idea encode the full meaning the idea. We routinely give short
names to complex things; that’s what names are for. For more on
this, see reference 6.

This is particularly important when you are working as part of a
problem-solving team. You are free to make up whatever notation and
terminology you like, but don’t expect other people to guess what it
means.

If it’s an electrostatics problem, sketch the field lines
and/or the equipotential contours.

If it’s a statistics problem, make scatter plots and
diaspograms of the data.

If it’s a scheduling problem, draw the PERT diagram.

et cetera

There exist thousands of different types of diagrams; reference 7 lists more than 100 examples. If you cannot find a
pre-existing way of representing the data, invent something. If this
is not easy, do it anyway. Remember the sardonic proverb:

Sometimes real-world problems start out underdetermined. In
such a case:

You might need to find the entire solution set.

You might need to turn it into an optimization problem,
i.e. to find the optimal element of the solution set. If the
statement of the assignment didn’t specify the optimization
objective, you may need to ask questions to figure that out.

You might need to track down some additional information,
to make the problem fully-determined or (better) overdetermined.

Real-world problems are very often overdetermined in a good way.
That allows you to solve the problem more than one way and check for
consistency. If it isn’t overdetermined, see if you can make it
overdetermined, to facilitate checking, as discussed in section 4.

Sometimes the assignment is overdetermined in a bad way,
i.e. not consistent with itself, or not consistent with the laws of
nature. In that case it may be necessary to turn it an optimization
problem, to see how close you can come to satisfying all the
stated objectives. Again, if the statement of the assignment didn’t
specify the optimization objective, you may need to ask questions to
figure that out.

Sometimes you get an assignment with just barely enough
information to permit a solution. That is typical of textbook
problems, but very atypical in the real world. It should make you
suspicious that there’s something you’re not being told.

Sometimes the assignment is not ill-posed in the technical
sense, but merely ill-conceived. Negotiate with the person who
posed the question, so that you can figure out what question
should have been asked ... and answer that instead.

Procedures for dealing with ill-posed problems are discussed in
reference 8.

Formalism can be very powerful, and sometimes it
delivers a valid result whether you understand it or not.

A lot of people know how to count reliably, but couldn’t give
a formal, rigorous definition of “integer”.

A lot of people know how to perform long division reliably,
but couldn’t really explain how it works.

A lot of people know how to play the piano proficiently, but
don’t fully understand how a piano works inside. They couldn’t
repair one, much less build one from scratch.

Much of quantum mechanics is in this category. Our ability to
calculate super-accurate results using the quantum mechanical
formalism far outstrips our ability to visualize and understand
what’s going on. Richard Feynman said he didn’t really understand
quantum mechanics ... even though he probably understood it better
than anyone else in the world at the time.

You are absolutely not required to fully understand every element of
what’s going on before starting the analysis and calculations. It is
common to understand a problem well enough to solve it, and then
achieve a much deeper understanding by pondering the solution. See
also the discussion of formalism in section 5.

Sometimes experiment delivers a valid result,
whether you understand it or not. It is fairly common to obtain a
experimental result and not fully understand it until later.
There are layers upon layers of understanding. There is such a thing
as serendipity.

When Penzias and Wilson observed excess noise in their antenna
system, that was a valid experimental result. They did not
understand its origin – or its cosmic significance – until later.
1

During the process of solving the
problem, build up an understanding of what’s going on (as far as
possible). That is to say, don’t blindly follow the formalism (any
more than necessary).

Analysis helps with the understanding ... just as understanding helps
with the analysis. You can leverage one against the other, again and
again, recursively and iteratively, like the itsy-bitsy
spider.

Even though it is sometimes possible (as mentioned in item 11 and item 12) to obtain a
formally-correct result without understanding it, this should be
considered only a first step. It is not something to be proud of.
Don’t walk away from the result until you have made a serious effort
to understand it. See also item 14 and item 17.

After you have found one solution, see if you can
find another solution to the same problem. Try to make the solutions
independent, to the extent possible. If they have some steps in
common, that’s not tragic, but try to minimize the overlap.

Example: There are two remarkably dissimilar solutions to the
Mississippi Flow problem (reference 4).

Example: There are two remarkably dissimilar solutions for finding the
Johnson noise in a parallel combination of resistors (reference 9).

Obviously this overlaps quite a bit with the suggestion to Check The
Work (item 4) but it is worth mentioning, because it is a
specific way of checking the work. The obvious approach is to
carry out the same calculation a second time, checking and re-checking
each step along the way.

The non-obvious approach is to carry out a completely different
calculation that produces the same result. This has the tremendous
advantage that it can catch conceptual errors, not just arithmetic
errors. (The downside is that if there is an error, this will not
tell you exactly where to find it.)

Intermediate example: If you use trig functions to find the sides of a
triangle, the answer had better uphold the Pythagorean theorem and
other trig identities. Similarly, any physics answer had better
uphold the various conservation laws.

Fancy example: Given the task of estimating (closed book) how much
Mississippi River water flows past New Orleans in a year, there are at
least two completely different ways of approaching the problem,
leading to completely independent estimates.

For students who have never seen the problem before,
finding one solution is hard ... and finding two
independent solutions is even harder, much worse
than twice as hard.

If you don’t immediately see how to get started on the
assigned problem, it may help to do some warm-up exercises. That is,
attack a similar but simpler problem.

Try doing some numerical examples, i.e. instances drawn from the
more general class. This may involve a simple spreadsheet or a fancy
numerical simulation.

Conversely, if given a problem with numerical values, try
working out the general case. It’s amazing how often the general
case is easier to handle than ten special cases ... or even one
special case.

Here’s an example that recently came across my desk:
Suppose you have to multiply (20.14 + 20.04) × (20.14 −
20.04). You could grind it out, and obtain 4.018. On the other
hand, if theory tells you that 20.14 is really 2cosh(ρ) and
20.04 is really 2sinh(ρ) where ρ=3, then you know the
answer has to be 4, exactly, independent of ρ. So that’s a
solution to the current problem and innumerable others, because it
works for all ρ.

Furthermore, you get to go looking for a good physics reason why the
answer had to be independent of ρ, which makes you smarter in
the long run.

Try doing an analogous problem. Try introducing approximations.

What do we mean by analogous? That could be almost anything you
know that has some features in common with the current problem.
Here are some hints on places where you might look for analogies
and/or approximations you can introduce:

If you have a problem involving distance in three
dimensions, see if there is an analogous problem in two
dimensions or one dimension. For that matter, see if there is an
analogous problem in zero dimensions (involving no distance at
all, just counting). Sometimes such analogous problems can be
found, and sometimes not.

Try a version of the problem with more symmetry and less
detail, such as the proverbial spherical cow.

More generally, you might try approximating things by some
sort of series, perhaps by expanding everything to lowest order.
You can get better and better approximations by including more and
more terms in the series.

There are lots of ways of doing this: Taylor series, Fourier
series, et cetera. For example, the sphere mentioned in the
previous item is the lowest-order term in an expansion based on
spherical harmonics.

If the problem involves a random distribution, consider
approximating it by a uniform distribution. You can always go back
later and put in the fluctuations as a correction term.

Sometimes your first approxmation turns out to be good enough. If
not, maybe you can redo the analysis using slightly better
approximations. Sometimes the approximation is off by a factor of
infinity and cannot be repaired, but presumably you learned something
from the process of doing the analysis. Remember the spirit here try
something and see if it works. If not, try something else.

The difference between an approximation and an analogy is that you
don’t even pretend that the analogy is the right answer. After
doing the analogous problem, you set it aside and start attacking
the original problem, using what you learned from the analogy as a
roadmap.

After doing the warm-up exercise, sometimes you have to set it
aside and start working on the main problem from the beginning
... but sometimes not. Sometimes you can convert a simple, crude
model into a fancier, nuanced model by a process of successive
refinement. This is discussed in reference 10.

After you have solved the assigned problem, see if
you can generalize it. This is an important part of building up an
understanding, as discussed in item 13.

On the input side: Try to obtain the same result
with less-strict assumptions.

On the output side: Given a result, explore is ramifications,
implications, and applications. Check whether it is consistent (or
inconsistent!) with other things you know. This may or may not help
you with today’s problem, but it will certainly help you with other
problems down the road. It will make you smarter in the long run.

Given a word problem, find a way to formalize it.
This often involves choosing a set of variables, and then writing some
equations that express relationships between the variables. This in
turn may involve choosing a suitable frame of reference, system of
coordinates, vector basis, et cetera.

Don’t hesitate to break one hard problem into a
great many easy problems. You don’t need to be Superman. That is to
say, you don’t need to be able to leap tall buildings at a single
bound if you can walk up the stairs. There are a lot of steps, but
each one is easy. See also section 3.

For example, balancing a complicated chemical reaction equation would
be well-nigh impossible if you didn’t have a systematic way of doing
it. Gaussian elimination might require 20 steps or even 100 steps,
but each step is cut-and-dried and incredibly easy, so you just grind
it out. Or tell the computer to do it. Problem solved.

Attack the problem from all angles. If necessary,
chip away at the corners. Try something and see if it works.
If not, try something else. Use your intuition to help decide what
is a plausible line of attack. See also section 3.

Feynman compared knowledge to a grand tapestry. Something you don’t
know is like a hole in the tapestry. You can fix it by weaving
upward from the bottom, downward from the top, and/or inward from the
sides. Sometimes it pays to start in the middle of nowhere and
construct something from scratch, and then patch it into the main
tapestry later.

Beware that a great many people who ought to
know better emphasize the straight-backwards attack, i.e. starting
with the conclusion and working backwards. This is no worse than
many of the other possible lines of attack, but it is no better, and
there are thousands of possibilities.

The straight-backward approach works particularly well for some of
the multiple-guess tests that plague our classrooms today, but does
not work nearly so well in the real world.

When you use one equation to derive
another, you should not do it by revising the initial equation in
place. Instead, leave the original equation alone, and write the
derived equation on a new line. This is important for making the work
checkable. See reference 11.

Generally, you should avoid erasing anything in your
working notes. If it’s wrong, draw a tidy diagonal line through it,
rewrite it correctly if necessary, and proceed from there. Ditto for
seemingly-worthless dead ends. Anybody who is reading your work
knows what a mistake is and what a dead end is, and won’t be confused
or surprised to find such things in your work. Contrast item 26.
For additional details, see reference 11.

Rationale: In the best case, erasing isn’t the trouble. In the worst
case, you run the risk of erasing something valuable.

If you are keeping your notes on a computer, use a version
control system. This allows you to “tidy up” the current version
as much as you like. Meanwhile, previous versions can be retrieved
at the touch of a button.

Keep track of the difference between the main goal
(solving the maze) and the various sub-goals (reconnoitering some
branch of the maze). Often making progress toward the main goal
requires giving up on a sub-goal (i.e. backing out of a dead end).
See reference 4.

The “scientific method” is not nearly as methodical as
non-scientists seem to think it is.

Thinking of it as a step-by-step process suggests stepping along a
road, moving steadily forward along a one-dimensional path. That’s
not how problem-solving actually works. That may be a good way to
explain the solution once it is found, but that’s not a good way
to find it.

The reality is more like searching a maze (rather than following a
clear one-dimensional path). It goes something like this:

Start with an intuitive idea. Draw a qualitative
diagram. That provides motivation to write some equations, which can
be turned into software code, which produces a vastly more accurate
quantitative diagram. Looking at this diagram indicates that the
original intuitive idea is not entirely satisfactory. Checking the
limiting cases shows that the idea works in certain limits, but not in
the general case. This suggests a more elaborate idea. That
motivates writing some new equations and some new code. That produces
a new diagram. Checking the limiting cases and the symmetries
suggests that the “elaborate” model can be made much more
streamlined and more intuitive. That motivates yet more equations.
Alas the equations show that the streamlined model is not actually
correct. So we have to back up and come up with yet another model. A
new concept is brought to bear. It is sophisticated but not
complicated. This motivates an entirely new approach, including new
equations, new code, and new diagrams. The result is elegant and
provably correct. It is intuitive in retrospect. There is an
algebraic proof and a non-algebraic geometric proof, which can be
checked against each other. An analysis of the symmetries, limiting
cases, and analogies makes it even more plausible and intuitive.

One lesson to take from this story is that a check-list is not a
do-list. You should check each of the items enumerated in section 2, but you don’t necessarily need to do them all. You
certainly don’t need to do them in any particular order. Some of them
will get done multiple times at various stages of the game.

Very commonly you chip away at one corner of the problem and go as far
as you can until you get stuck. Then you set that aside and start
chipping away at another corner.

Very commonly a seemingly-promising line of attack turns out to be a
dead end, and a big chunk of preliminary work has to be abandoned.
Abandoning something is painful and difficult for psychological
reasons, but it has to be done.

Consider the following styles:

The way ideas are presented in textbooks and in the scientific
literature. This is very polished, logical, systematic,
sophisticated, and elegant.

The way problems are actually solved.

The way problem-solving skills are actually learned.

For most of the last 2300 years, there has been a tendency to pretend
these are all the same thing ... when in fact they are not. Not even
close.

I like elegance and polish as much as the next guy, but not when
it requires pulling rabbits out of hats in such a way that the
students cannot understand the process.

There will always be enormous temptation to polish things, for some
good reasons and some not-so-good reasons.

One semi-justifiable reason is efficiency: Real problem solving
involves a lot of back-tracking out of dead ends. It is more
efficient if you can skip the dead ends, with the help of 20/20
hindsight.

The polished method works fine if you are using the
solution for the third or fourth time.

The polished method isn’t at
all representative of the process of discovering the solution
for the first time.

If all you want is a solution, the elegantly polished
solution is fine.

If you want to learn how to solve problems, you
need to see the problem-solving process, which is rarely (if ever)
elegant.

When teaching, or when writing a textbook, you have to decide whether
you are training solution-users or educating solution-discoverers. It
isn’t practical to show the gory details every time, but I suggest
showing them at least occasionally.

Another factor is that people always like to pretend they are
smarter than they really are. The temptation is to say “here is my
elegant solution to the problem” without mentioning that the elegant
solution was found only after 13 less-elegant solutions. The smartest
person in the world might have been able to find the elegant solution
the first time, but most of us aren’t that smart.

There’s even a name for this: hindsight bias. That means you
are much more likely to see how somebody thinks they should have
solved the problem, not how they really solved the problem.

As a means to this end, leave breadcrumbs (or
whatever) to mark your path. Very often, the solution requires a
large number of steps. Document things in enough detail that each
step can be checked locally, independently of the others. This
makes checking vastly more efficient and vastly more reliable,
compared to a situation where the only way to check the solution is
to re-do the whole thing from scratch.

Solving a hard problem often includes a process that is equivalent
to searching a maze. Mathematically this corresponds to enumerating
the corners of a high-dimensional hypercube. In computer science,
it is called walking the tree. In any case, when you return to a
given node, it is important to know what’s been tried before;
otherwise you risk gross inefficiency, up to and including infinte
loops. Therefore you need to leave bread crumbs to mark the path.

If you solved the problem in a flash of divine
inspiration, go back and figure out a way it could have been solved
methodically, step by step. This is necessary so that mere mortals
can check your work. Also, you may discover that the inspired
solution is only part of a larger solution-set.

Also: Inspiration is a way of obtaining an answer very quickly ...
but beware that all too often, it is a way of obtaining a
wrong answer very quickly.

There are other things that are technically
not dimensions, but can be checked in the same manner. An example
is vector character. If the answer is supposed to be a vector, and
you’ve got a scalar, you’ve done something wrong.

For example, if you prove that a2 + b2 = c2 for «all»
triangles, you’ve proved something that isn’t true. The statement
should be restricted to right triangles in the Euclidean plane
... and your proofe of the statement should be clearly dependent on
these restrictions.

Solve the problem in more than one way. Any
problem worth solving is worth solving twice. For example, if a
physics problem can be solved in terms of forces, solve it again in
terms of energy, i.e. the principle of virtual work. If you
can apply Newton’s third law in terms of equal-and-opposite
forces, apply it again in terms of momentum conservation.

Maybe for an easy, non-tricky problem you can get away with solving
it only once, but if you are dealing exclusively with easy,
non-tricky problems you wouldn’t be reading this document.

As mentioned in item 11, formalism can
be very powerful. Learn to use the formalism.

Formalism is a tool. Any tool can be abused ... but that doesn’t mean
you are obliged to abuse it. Powerful tools, when properly used, make
it easier (not harder) to understand what’s going on. For example:
Consider the following contrast:

Maxwell’s paper on electrodynamics is almost unreadable.
That’s because he didn’t use vector formalism. Vectors were not
invented until many years later.