Thursday, July 30, 2009

I'm trying to hone my problem solving skills.
Do you know any good books written on how to solve complex
problems? Did school, i.e., teachers, books, labs, peers,
help define the process? How do you approach a problem? How do you
break it apart — the fluff from the stuff? How do you formulate
a plan?

Wow. These are not easy questions.

Years ago I had a book called How to Solve It by George Pólya. I
remember not giving it the attention that I think it deserves. I
recall that some of the advice seemed rather obvious. In retrospect,
I wish I had payed more attention to it. When someone like Pólya
suggests something obvious, it probably isn't as obvious as you think
and it is probably worth pondering.

College was a huge influence. The whole focus of the undergraduate
curriculum was to “teach you how to think”. The detail of
the coursework was almost secondary. It was there in order to be
stuff to think about. It had the benefit of being topical. You got
to know the jargon of the field and get familiar with the kind of
day-to-day problems you would encounter, but that was secondary.

Graduate students would teach the recitations, but the professor was
usually a tenured researcher. That made a difference. I found that
the professors were amazingly sharp people. They also enjoyed
tremendously what they were doing and their enthusiasm showed.

Nearly every course would start with a lecture where the professor
would say “we're here to teach you how to think&rdquo. Then
they would start in on the “first principles”. For
example, in electrical engineering we started with Kirchoff's Laws.
Now Kirchoff's Laws are really obvious: all the current flowing into
and out of a node has to add up to zero (otherwise it would pile up
somewhere or come out of nowhere), and all the voltages around a loop
had to add up to zero (or you'd have made power out of thin air). But
the professor would always start from “first principles&rdquo.

The next thing the professor would do is introduce a model.
Invariably the professor would take a pretty simple and
straightforward problem and point out that it is essentially
impossible to understand and solve. Again, to use the example from
electrical engineering, the action of any electrical circuit
whatsoever follows Maxwell's equations (it has no choice), but no one
is going to solve those (what's the B field around an irregular lump
of solder?) The professor would show us the basic model and
how to use it to approximate solutions to real problems.

Finally we'd go over the building blocks. There were a selection of
primitive elements and a handful of ways to combine them.

What was interesting is that virtually every course took this
approach. It didn't matter whether it was chemistry, electronics,
physics, or computer science. It came down to “first
principles&rdquo, models, and the basic elements.

Another thing that I noticed was the almost absurdly naive way that
the professors would approach things. They always wanted to mention
their research, so we'd hear about some of the cutting edge of
science. The problems that the professors were working on could
usually be summed up in a sentence or two: “Why do things fall
down instead of up?” “How about if I made a computer chip
with really tiny mirrors on it and computed with photons?”
“If I run my computer backwards, can I get back the energy I put
in?”

Of course someone would object and say “That wouldn't
work.” or “You can't do that.” and the professor
would immediately counter with “Why not?” and start
showing that his ideas were actually plausible if you went back to
first principles and used some imagination.

I found that the best professors seemed to have what seemed to be the
most naive questions. I also found that they usually had a long, long
record of posing interesting questions and developing astounding
solutions. And they were fun to talk to and hang around with.

I really enjoyed the way that certain professors thought about things
and I tried to emulate that. I keep telling myself to start with
first principles and make a model. (I have to keep reminding myself
of this, too. It's too easy to get wrapped up in the detail.)

There's one more thing these professors did: teach. Some of them
taught undergrad courses that weren't their specialty because they
wanted to learn the material themselves. If you can teach a
non-expert enough to understand what it is you are doing, then you
probably have a good grasp of it. Otherwise, you might not even
understand what problem you are trying to solve. (That's one reason I
want to write more blog entries. It helps me organize my thoughts by
trying to describe what I'm doing to other people.)

How do you approach a problem? How do you formulate a plan?

This question gave me a lot to think about. I kept trying to think
how I formulate a plan and I realized just this moment that I don't.
I think maybe I disagree with Pólya on this point. He gives these
steps:

First, you have to understand the problem.

After understanding, then make a plan.

Carry out the plan.

Look back on your work. How could it be better?

The interesting problems are the ones
where I have no idea at all how to go about solving. Then there is a
good opportunity to learn something (and I love learning things).

I usually don't understand the problem, and that's the biggest
hurdle. Of course I often think that I do
understand it and get off on the wrong path. I have found myself with
a ‘solution’ to something I didn't really want to solve,
or the answer to a question that wasn't worth asking. I have to keep
reminding myself of what question I want the answer to.

I either don't have good plans, or I can't recognize the good plans
amongst the bad ones. I look at a lot of different approaches and try
one that looks promising. If that doesn't work, I'll try another.
I'll keep refining the plan on the way.

When I get really into a problem, I'm constantly explaining it to other
people to try to get some insight. Often I find I don't have a clear
enough concept to be able to explain it to someone. Usually the
person I'm explaining it to gives me some irrelevant suggestions, but
it helps me understand how they are perceiving or
understanding what I am asking. Sometimes they say words that trigger
an idea.

I also like the oddball approaches. For a lot of problems there seem
to be one or two people on the ‘fringe’. They eschew the
standard approach to something and advocate something that seems to
come out of left field. Sometimes, however, a non-standard approach
gives you some insight into the problem that you wouldn't have gotten
otherwise. A good example is using Clifford Algebras for computer
graphics.

That's about all I can think of right now. I hope I answered your
questions.