I don’t know how people take it, but here’s what I meant by it. Sometimes you can find a smarter way to work, and if you can, I assume you’re doing that. Don’t drive nails with your shoe if you can find a hammer. But ultimately the way to get things done is hard work. You might see some marginal increase in productivity from using some app or another, but there’s nothing that’s going to magically make you 10x more productive without extra effort.

Many people have replied on Twitter “I think you mean ‘work smart.'” At some point “work smarter” wasn’t a cliché, but now it is. The problem of our time isn’t people brute-forcing their way with hard, thoughtless work. We’re more likely to wish for a silver bullet. We’re gnostics.

Smart work is a kind of hard work. It may take less physical work but more mental work. Or less mental work and more emotional work. It’s hard work to try to find a new perspective and take risks.

One last thought: hard work is not necessarily long work. Sometimes it is, but often not. Hard creative work requires bursts of mental or emotional effort that cannot be sustained for long.

]]>http://www.johndcook.com/blog/2016/11/21/hard-work/feed/7Group projectshttp://www.johndcook.com/blog/2016/07/01/group-projects/ http://www.johndcook.com/blog/2016/07/01/group-projects/#commentsFri, 01 Jul 2016 18:28:20 +0000http://www.johndcook.com/blog/?p=22904The best teams have people with complementary skills, but similar work ethic. Academic assignments are the opposite. There’s not much variation in skills, in part because students haven’t yet developed specialized skills, and in part because students are in the same class because they have similar interests. The biggest variation is likely to be work ethic. It’s not uncommon for the hardest working person in a group to do 10x as much work as the laziest person in the group. The person doing most of the work learns that it’s best to avoid working with teams.

Working with people with complementary skills is a blast, but you’re unlike to experience that in an academic project. You might get some small degree specialization. Maybe one of the mechanical engineers on a project has more artistic ability than the other mechanical engineers, for example. But this is hardly like the experience of working with a team of people who are all great at different things.

]]>http://www.johndcook.com/blog/2016/07/01/group-projects/feed/2Grateful for failureshttp://www.johndcook.com/blog/2016/06/06/grateful-for-failures/ http://www.johndcook.com/blog/2016/06/06/grateful-for-failures/#commentsMon, 06 Jun 2016 14:48:25 +0000http://www.johndcook.com/blog/?p=22738I’ve been thinking lately about different things I’ve tried that didn’t work out and how grateful I am that they did not.

The first one that comes to mind is my academic career. If I’d been more successful with grants and publications as a postdoc, it would have been harder to decide to leave academia. I’m glad I left when I did.

When I was in high school I was a fairly good musician. At one point decided that if I made the all state band I would major in music. Thank God I didn’t make it.

I’ve looked back at projects that I hoped to get, and then realized how it’s a good thing that they didn’t come through.

In each of these examples, I’ve been forced to turn away from something I was moderately good at to pursue something that’s a better fit for me.

I wonder what failure I’ll be grateful for next.

]]>http://www.johndcook.com/blog/2016/06/06/grateful-for-failures/feed/4How about one good one?http://www.johndcook.com/blog/2016/05/18/how-about-one-good-one/ http://www.johndcook.com/blog/2016/05/18/how-about-one-good-one/#commentsWed, 18 May 2016 17:51:56 +0000http://www.johndcook.com/blog/?p=22655I’m no fan of tobacco companies or their advertising tactics, but I liked the following story.

When the head of a mammoth [advertising] agency solicited the Camel Cigarette account, he promised to assign thirty copywriters to it, but the canny head of R. J. Reynolds replied, “How about one good one?” Then he gave his account to a young copywriter called Bill Esty, in whose agency it has remained for twenty-eight years.

One really good person can accomplish more than thirty who aren’t so good, especially in creative work.

]]>http://www.johndcook.com/blog/2016/05/18/how-about-one-good-one/feed/1Well, F = ma.http://www.johndcook.com/blog/2016/04/05/f-equals-m-a/ http://www.johndcook.com/blog/2016/04/05/f-equals-m-a/#commentsWed, 06 Apr 2016 00:39:36 +0000http://www.johndcook.com/blog/?p=21393Three or four very short stories on the difficulty of learning to use simple things. Depends whether you count the last section as a story.

* * *

When I was taking freshman physics and we were stuck on a problem, the professor would say “Well, F = ma.”

True, but absolutely useless. Yes, we know that F = ma. (Force equals mass times acceleration.) Nobody thought “Oh, that’s it. I was thinking F = ma2. That explains everything.” Newton’s laws are simple (in a sense) but subtle to apply. The difficult part isn’t the abstract principles but their application to concrete problems.

* * *

The heart of Bayesian statistics is not much more complicated than F = ma. Its the statement that

posterior ∝ likelihood × prior.

It takes a few years to learn how to apply that equation well. And when people try to help, their advice sounds about as useless as “Well, F = ma.”

* * *

Learning to use Unix was hard. When I asked for help getting started, a lab assistant said “Go read the man pages.” That’s about as hostile as saying “Want to learn English? Read a dictionary.” Fortunately I knew other people who were helpful. One of them told me about the book.

But still, it took a while to get the gestalt of Unix. I knew how to use a handful of utilities, and kept thinking everything would be fine once I knew maybe 10x as many utilities. Then one day I was talking with a friend who seemed fluent working with Unix. I asked him how he did a few things and realized he used the same tools I did, but used them better. It was almost as if he’d said “I just use F = ma” except when he said it things clicked.

* * *

The motivation for this post, the thing that brought these stories to mind, was listening to a podcast. The show had some good advice, things that I know I need to do, but nothing I hadn’t heard many times before. The hard part is working out what the particulars mean for me personally.

It often takes someone else to help us see what’s right in front of us. I’m grateful for the people who have helped me work out the particulars of things I was convinced of but couldn’t see how to apply. Sometimes I have the pleasure of being able to do that for someone else.

Maybe the thing is indeed easy, and has been done before. Then someone was the first to do it. The warning that it had been done before didn’t apply to this person, even though it would apply to the subsequent people with the same idea.

This reminds me of the story of two economists walking down the street. They notice a $20 bill on the sidewalk and the first asks “Aren’t you going to pick it up?” The second replies “No, it’s not really there. If it were, someone would have picked it up by now.”

Sometimes a solution is easy, but nobody has had the audacity to try it. Or maybe circumstances have changed so that something is easy now that hasn’t been before.

Sometimes a solution is easy for you, if not for many others. See how much less credible the opening sentence sounds with for you inserted: “If it were easy for you, someone would have done it.”

]]>http://www.johndcook.com/blog/2016/03/08/if-it-were-easy/feed/2Structure in jazz and mathhttp://www.johndcook.com/blog/2016/02/07/structure-in-jazz-and-math/ http://www.johndcook.com/blog/2016/02/07/structure-in-jazz-and-math/#commentsSun, 07 Feb 2016 21:03:14 +0000http://www.johndcook.com/blog/?p=20636Last night I went to a concert by the Branford Marsalis Quartet. One of the things that impressed me about the quartet was how creative they are while also being squarely within a tradition. People who are not familiar with jazz may not realize how structured it is and how much it respects tradition. The spontaneous and creative aspects of jazz are more obvious than the structure. In some ways jazz is more tightly structured than classical music. To use Francis Schaeffer’s phrase, there is form and freedom, freedom within form.

Every field has its own structure, its tropes, its traditions. Someone unfamiliar with the field can be overwhelmed, not having the framework that an insider has to understand things. They may think something is completely original when in fact the original portion is small.

In college I used to browse the journals in the math library and be completely overwhelmed. I didn’t learn until later that usually very little in a journal article is original, and even the original part isn’t that original. There’s a typical structure for a paper in PDEs, for example, just as there are typical structures for romantic comedies, symphonies, or space operas. A paper in partial differential equations might look like this:

Motivation / previous work

Weak formulation of PDE

Craft function spaces and PDE as operator

A priori estimates imply operator properties

Well posedness results

Regularity

An expert knows these structures. They know what’s boilerplate, what’s new, and just how new the new part is. When I wrote something up for my PhD advisor I remember him saying “You know what I find most interesting?” and pointing to one inequality. The part he found interesting, the only part he found interesting, was not that special from my perspective. It was all hard work for me, but only one part of it stood out as slightly original to him. An expert in partial differential equations sees a PDE paper the way a professional musician listens to another or the way a chess master sees a chess board.

While a math journal article may look totally incomprehensible, an expert in that specialization might see 10% of it as somewhat new. An interesting contrast to this is the “abc conjecture.” Three and a half years ago Shinichi Mochizuki proposed a proof of this conjecture. But his approach is so entirely idiosyncratic that nobody has been able to understand it. Even after a recent conference held for the sole purpose of penetrating this proof, nobody but Mochizuki really understands it. So even though most original research is not that original, once in a while something really new comes out.

]]>http://www.johndcook.com/blog/2016/02/07/structure-in-jazz-and-math/feed/8Get rid of something every Thursdayhttp://www.johndcook.com/blog/2015/12/31/turnover-thursday/ http://www.johndcook.com/blog/2015/12/31/turnover-thursday/#commentsThu, 31 Dec 2015 16:13:09 +0000http://www.johndcook.com/blog/?p=19944I heard of someone who had a commitment to get rid of something every Thursday. I don’t know anything about how they carried that out. It could mean throwing out or donating to charity a physical object each Thursday. Or maybe it could be handing over a responsibility or letting go of an ambition. It could be a combination, such as getting rid of an object that is a reminder of something intangible that you want to let go of.

This may mean reducing your total inventory of objects or obligations, or it could be simply turnover, making room for new things.

Getting rid of an obligation is not necessarily irresponsible, nor is letting go of an ambition necessarily lazy. Letting go of one obligation to take on another could be very responsible. Letting go of one ambition to pursue another could be a lot of work.

]]>http://www.johndcook.com/blog/2015/12/31/turnover-thursday/feed/3Retoolinghttp://www.johndcook.com/blog/2015/12/18/retooling/ http://www.johndcook.com/blog/2015/12/18/retooling/#commentsFri, 18 Dec 2015 14:18:35 +0000http://www.johndcook.com/blog/?p=19890I was listening to a classic music station yesterday, and I heard the story of a professional pianist whose hand was injured in an accident. He then started learning trumpet and two years later he was a professional trumpeter. I didn’t catch the musician’s name.

I was not surprised that a professional in one instrument could become a professional in another, but I was surprised that he did it in only two years. It probably helped that he was no longer able to play piano; I imagine if he wanted to learn trumpet in addition to piano he would not have become so proficient so quickly.

If you go by the rule of thumb that it takes about 10 years to master anything, this professional pianist was 80% of the way to becoming a professional trumpeter before he touched a trumpet. Or to put it another way, 80% of being a professional musician who plays trumpet is becoming a professional musician.

Transferable skills are more difficult to acquire and more valuable than the context in which they’re exercised.

]]>http://www.johndcook.com/blog/2015/12/18/retooling/feed/6Taking away a damaging toolhttp://www.johndcook.com/blog/2015/09/24/taking-away-a-damaging-tool/ http://www.johndcook.com/blog/2015/09/24/taking-away-a-damaging-tool/#commentsThu, 24 Sep 2015 13:47:14 +0000http://www.johndcook.com/blog/?p=19467This is a post about letting go of something you think you need. It starts with an illustration from programming, but it’s not about programming.

Bob Martin published a dialog yesterday about the origin of structured programming, the idea that programs should not be written with goto statements but should use less powerful, more specialized ways to transfer control. Edsgar Dijkstra championed this idea, most famously in his letter Go-to statement considered harmful. Since then there have been countless “considered harmful” articles that humorously allude to Dijkstra’s letter.

Toward the end of the dialog, Uncle Bob’s interlocutor says “Hurray for Dijkstra” for inventing the new technology of structured programming. Uncle Bob corrects him

New Technology? No, no, you misunderstand. … He didn’t invent anything. What he did was to identify something we shouldn’t do. That’s not a technology. That’s a discipline.

Huh? I thought Structured Programming made things better.

Oh, it did. But not by giving us some new tools or technologies. It made things better by taking away a damaging tool.

The money quote is the last line above: It made things better by taking away a damaging tool.

Creating new tools gets far more attention than removing old tools. How might we be better off by letting go of a tool? When our first impulse is that we need a new technology, might we need a new discipline instead?

Few people have ever been able to convince an entire profession to let go of a tool they assumed was essential. If we’re to have any impact, most of us will need to aim much, much lower. It’s enough to improve our personal productivity and possibly that of a few peers. Maybe you personally would be better off without something that is beneficial to most people.

What are some technologies you’ve found that you’re better off not using?

]]>http://www.johndcook.com/blog/2015/09/24/taking-away-a-damaging-tool/feed/19The Mozart Mythhttp://www.johndcook.com/blog/2015/04/20/the-mozart-myth/ http://www.johndcook.com/blog/2015/04/20/the-mozart-myth/#commentsTue, 21 Apr 2015 02:11:18 +0000http://www.johndcook.com/blog/?p=18534I don’t know how many times I’ve heard about how Mozart would compose entire musical scores in his head and only write them down once they were finished. Even authors who stress that creativity requires false starts and hard work have said that Mozart may have been an exception. But maybe he wasn’t.

In his new book How to Fly a Horse, Kevin Ashton says that the Mozart story above is a myth based on a forged letter. According to Ashton,

Mozart’s real letters—to his father, to his sister, and to others—reveal his true creative process. He was exceptionally talented, but he did not write by magic. He sketched his compositions, revised them, and sometimes got stuck. He could not work without a piano or harpsichord. He would set work aside and return to it later. … Masterpieces did not come to him complete in uninterrupted streams of imagination, nor without an instrument, nor did he write them whole and unchanged. The letter is not only forged, it is false.

Economic forecasting is useful for predicting the future up to about ten years ahead. Beyond ten years the quantitative changes which the forecast accesses are usually sidetracked or made irrelevant by qualitative changes in the rules of the game. Qualitative changes are produced by human cleverness … or by human stupidity … Neither cleverness nor stupidity are predictable.

]]>http://www.johndcook.com/blog/2015/04/03/looking-ten-years-ahead/feed/0Confidencehttp://www.johndcook.com/blog/2015/01/28/confidence/ http://www.johndcook.com/blog/2015/01/28/confidence/#commentsWed, 28 Jan 2015 16:39:57 +0000http://www.johndcook.com/blog/?p=18034Zig Ziglar said that if you increase your confidence, you increase your competence. I think that’s generally true. Of course you could be an idiot and become a more confident idiot. In that case confidence just makes things worse [1]. But otherwise when you have more confidence, you explore more options, and in effect become more competent.

There are some things you may need to learn not for the content itself but for the confidence boost. Maybe you need to learn them so you can confidently say you didn’t need to. Also, some things you need to learn before you can see uses for them. (More on that theme here.)

I’ve learned several things backward in the sense of learning the advanced material before the elementary. For example, I studied PDEs in graduate school before having mastered the typical undergraduate differential equation curriculum. That nagged at me. I kept thinking I might find some use for the undergrad tricks. When I had a chance to teach the undergrad course a couple times, I increased my confidence. I also convinced myself that I didn’t need that material after all.

My experience with statistics was similar. I was writing research articles in statistics before I learned some of the introductory material. Once again the opportunity to teach the introductory material increased my confidence. The material wasn’t particularly useful, but the experience of having taught it was.

[1] See Yeats’ poem The Second Coming: … The best lack all conviction, while the worst Are full of passionate intensity. …

]]>http://www.johndcook.com/blog/2015/01/28/confidence/feed/4Prevent errors or fix errorshttp://www.johndcook.com/blog/2014/09/13/prevent-errors-or-fix-errors/ http://www.johndcook.com/blog/2014/09/13/prevent-errors-or-fix-errors/#commentsSun, 14 Sep 2014 00:40:21 +0000http://www.johndcook.com/blog/?p=14711The other day I was driving by our veterinarian’s office and saw that the marquee said something like “Prevention is less expensive than treatment.” That’s sometimes true, but certainly not always.

This evening I ran across a couple lines from Ed Catmull that are more accurate than the vet’s quote.

Do not fall for the illusion that by preventing errors, you won’t have errors to fix. The truth is, the cost of preventing errors is often far greater than the cost of fixing them.

]]>http://www.johndcook.com/blog/2014/09/13/prevent-errors-or-fix-errors/feed/16What would Donald Knuth do?http://www.johndcook.com/blog/2014/08/12/what-would-dondald-knuth-do/ http://www.johndcook.com/blog/2014/08/12/what-would-dondald-knuth-do/#commentsWed, 13 Aug 2014 01:17:07 +0000http://www.johndcook.com/blog/?p=14671I’ve seen exhortations to think like Leonardo da Vinci or Albert Einstein, but these leave me cold. I can’t imagine thinking like either of these men. But here are a few famous people I could imagine emulating when trying to solve a problem

What would Donald Knuth do? Do a depth-first search on all technologies that might be relevant, and write a series of large, beautiful, well-written books about it all.

What would Alexander Grothendieck do? Develop a new field of mathematics that solves the problem as a trivial special case.

What would Richard Stallman do? Create a text editor so powerful that, although it doesn’t solve your problem, it does allow you to solve your problem by writing a macro and a few lines of Lisp.

What would Larry Wall do? Bang randomly on the keyboard and save the results to a file. Then write a language in which the file is a program that solves your problem.

What would you add to the list?

]]>http://www.johndcook.com/blog/2014/08/12/what-would-dondald-knuth-do/feed/25Sometimes definitions are enoughhttp://www.johndcook.com/blog/2014/05/16/sometimes-definitions-are-enough/ http://www.johndcook.com/blog/2014/05/16/sometimes-definitions-are-enough/#commentsFri, 16 May 2014 16:03:11 +0000http://www.johndcook.com/blog/?p=14484Sometimes you can apply math just by raiding it for vocabulary. You may not need to apply a single theorem.

This has been a surprise to me. I’m more used to creating a mathematical model so you can compute something or apply some theorem. But sometimes you can move a project along just by providing a name for a concept. A meandering discussion can snap into focus because someone has a name for an idea everyone vaguely understands.

Sometimes it may be clear that only part of a mathematical definition applies. In this case math can guide the discussion by asking whether the rest of the definition applies. “It sounds like we’ve got a widget here. A widget has to have these five properties and clearly we have the first three. Let’s think about whether the last two hold.” The answers don’t have to be positive to be useful. You might realize something important in the process of explaining why your thing is not a widget.

Sometimes a definition may not apply at all and still be useful! “This reminds me of a widget. It’s not a widget in any strict sense. But if it were, this is what we’d do next. I wonder whether we can do something like that.”

… I’m older and, I hope, more able to cope with stress: just as carpenters get calloused hands that make them insensitive to small abrasions, I like to imagine that academics get calloused minds that allow them not to be bothered by small stresses and strains.

Mental callouses are an interesting metaphor. Without the context above, “calloused minds” would have a negative connotation. We say people are calloused or insensitive if they are unconcerned for other people, but Leinster is writing of people unperturbed by distractions.

You could read the quote above as implying that only academics develop mental discipline, though I’m sure that’s not what was intended. Leinster is writing a personal post about the process of writing books. He’s an academic, and so he speaks of academics.

Not only do carpenters become more tolerant of minor abrasions, they also become better at avoiding them. I’m not sure that I’m becoming more tolerant of stress and distractions as I get older, but I do think I’m getting a little better at anticipating and avoiding stress and distractions.

]]>http://www.johndcook.com/blog/2014/02/17/mental-callouses/feed/3“I got the easy ones wrong”http://www.johndcook.com/blog/2013/11/06/i-got-the-easy-ones-wrong/ http://www.johndcook.com/blog/2013/11/06/i-got-the-easy-ones-wrong/#commentsWed, 06 Nov 2013 18:42:57 +0000http://www.johndcook.com/blog/?p=14102This morning my daughter told me that she did well on a spelling test, but she got the easiest words wrong. Of course that’s not exactly true. The words that are hardest for her to spell are the ones she in fact did not spell correctly. She probably meant that she missed the words she felt should have been easy. Maybe they were short words. Children can be intimidated by long words, even though long words tend to be more regular and thus easier to spell.

Our perceptions of what is easy are often upside-down. We feel that some things should be easy even though our experience tells us otherwise.

Sometimes the trickiest parts of a subject come first, but we think that because they come first they should be easy. For example, force-body diagrams come at the beginning of an introductory physics class, but they can be hard to get right. Newton didn’t always get them right. More advanced physics, say celestial mechanics, is in some ways easier, or at least less error-prone.

“Elementary” and “easy” are not the same. Sometimes they’re opposites. Getting off the ground, so to speak, may be a lot harder than flying.

]]>http://www.johndcook.com/blog/2013/11/06/i-got-the-easy-ones-wrong/feed/6Hard-working lazy peoplehttp://www.johndcook.com/blog/2013/11/04/hard-working-lazy-people/ http://www.johndcook.com/blog/2013/11/04/hard-working-lazy-people/#commentsMon, 04 Nov 2013 21:11:03 +0000http://www.johndcook.com/blog/?p=14098“It was a favorite theme of C. S. Lewis that only lazy people work hard. By lazily abdicating the essential work of deciding and directing, establishing values and setting goals, other people do it for us; then we find ourselves frantically, at the last minute, trying to satisfy a half dozen demands on our time, none of which is essential to our vocation, to stave off the disaster of disappointing someone.” — Eugene Peterson

]]>http://www.johndcook.com/blog/2013/11/04/hard-working-lazy-people/feed/2Remove noise, remove signalhttp://www.johndcook.com/blog/2013/10/28/remove-noise-remove-signal/ http://www.johndcook.com/blog/2013/10/28/remove-noise-remove-signal/#commentsMon, 28 Oct 2013 12:06:52 +0000http://www.johndcook.com/blog/?p=14067Whenever you remove noise, you also remove at least some signal. Ideally you can remove a large portion of the noise and a small portion of the signal, but there’s always a trade-off between the two. Averaging things makes them more average.

Statistics has the related idea of bias-variance trade-off. An unfiltered signal has low bias but high variance. Filtering reduces the variance but introduces bias.

If you have a crackly recording, you want to remove the crackling and leave the music. If you do it well, you can remove most of the crackling effect and reveal the music, but the music signal will be slightly diminished. If you filter too aggressively, you’ll get rid of more noise, but create a dull version of the music. In the extreme, you get a single hum that’s the average of the entire recording.

This is a metaphor for life. If you only value your own opinion, you’re an idiot in the oldest sense of the word, someone in his or her own world. Your work may have a strong signal, but it also has a lot of noise. Getting even one outside opinion greatly cuts down on the noise. But it also cuts down on the signal to some extent. If you get too many opinions, the noise may be gone and the signal with it. Trying to please too many people leads to work that is offensively bland.

I had botched a great many pieces of wood before I mastered the right angle with a saw, botched even more before I learned to miter a joint. The knowledge of these things resides in my hands and eyes and the webwork of muscles, not in the tools. There are machines for sale—powered miter boxes and radial arm saws, for instance—that will enable any casual soul to cut proper angles in boards. The skill is invested in the gadget instead of the person who uses it, and this is what distinguishes a machine from a tool.

Sometime that’s how I feel about computing. I think of messes such as having to remember that arc tangent is atan in R and Python, but arctan in NumPy and a in bc. Or that C, Python, and Perl use else if, elif, and elsif respectively. Or did I switch those last two?

These trivial but innumerable messes keep us from devoting our full energy to bigger problems.

One way to reduce these messes is to use fewer tools. Then you know less to be confused about. If you only use Python, for example, then elif is just how it is. But knowing more tools is worth the added mess, up to a point. Past some point, however, new tools add more mental burden than utility. You have to find the optimal combination of tools for yourself, and that combination will change over time.

To use fewer tools, you may need to use more complex tools. Maybe you can replace a list of moderately complex but inconsistent tools with one tool that is more complex but internally consistent.

In both questions it is conceptually necessary to compute a*b, but not logically necessary. Both are a question about a*b, so computing the product is conceptually necessary. But there is no logical necessity to actually compute a*b in order to answer a question about it.

In the first question, there’s an obvious short cut: multiply the last two digits and see that the last digit of the product must be 1.

In the second question, it is conceivable that there is some way to find the median digit that is less work than computing a*b first, though I don’t see how. Conceptually, you need to find the digits that make up a*b, sort them, and select the one in the middle. But it is conceivable, for example, that there is some way to find the digits of a*b that is less work than finding them in the right order, i.e. computing a*b.

I brought up the example above to use it as a metaphor.

In your work, how can you tell whether a problem is more like the first question or the second? Are you presuming you have to do something that you don’t? Are you assuming something is logically necessary when it is only conceptually necessary?

When I’m stuck on a problem, I often ask myself whether I really need to do what I assume I need to do. Sometimes that helps me get unstuck.

… applied science, purposeful and determined, and pure science, playful and freely curious, continuously support and stimulate each other. The great nation of the future will be the one which protects the freedom of pure science as much as it encourages applied science.

Writing novels is hard, and requires vast, unbroken slabs of time. Four quiet hours is a resource I can put to good use. Two slabs of time, each two hours long, might add up to the same four hours, but are not nearly as productive as an unbroken four. … Likewise, several consecutive days with four-hour time-slabs in them give me a stretch of time in which I can write a decent book chapter, but the same number of hours spread out across a few weeks, with interruptions in between them, are nearly useless.

I haven’t written a novel, and probably never will, but Stephenson’s remarks describe my experience doing math and especially developing software. I can do simple, routine work in short blocks of time, but I need larger blocks of time to work on complex projects or to be more creative.

]]>http://www.johndcook.com/blog/2013/04/09/slabs-of-time/feed/5Not complex enoughhttp://www.johndcook.com/blog/2013/01/09/not-complex-enough/ http://www.johndcook.com/blog/2013/01/09/not-complex-enough/#commentsWed, 09 Jan 2013 17:33:59 +0000http://www.johndcook.com/blog/?p=12683One time a professor asked me about a problem and I suggested a simple solution. He shot down my idea because it wasn’t complex enough. He said my idea would work, but it wasn’t something he could write a paper about in a prestigious journal.

I imagine that sort of thing happens in the real world, though I can’t recall an example. On the contrary, I can think of examples where people were thrilled by trivial solutions such as a two-line Perl script or a pencil-and-paper calculation that eliminated the need for a program.

The difference is whether the goal is to solve a problem or to produce an impressive solution.

For until this moment he had lived in a state of pure possibility, not knowing what sort of man he was or what he must do, and supposing therefore that he must be all men and do everything. But after this morning’s incident his life took a turn in a particular direction. Thereafter he came to see that he was not destined to do everything but only one or two things. Lucky is the man who does not secretly believe that every possibility is open to him.

As Lawler summarizes,

Without some such closure — without knowing somehow that you’re “not destined to do everything but only one or two things” — you never get around to living.

It’s taken me a long time to understand that deliberately closing off some options can open more interesting options.

]]>http://www.johndcook.com/blog/2012/11/17/pure-possibility/feed/1Nobody's going to steal your ideahttp://www.johndcook.com/blog/2012/11/02/nobody-will-steal-your-idea/ http://www.johndcook.com/blog/2012/11/02/nobody-will-steal-your-idea/#commentsFri, 02 Nov 2012 10:51:43 +0000http://www.johndcook.com/blog/?p=12356When I was working on my dissertation, I thought someone might scoop my research and I’d have to start over. Looking back, that was ridiculous. For one thing, my research was too arcane for many others to care about. And even if someone had proven one of my theorems, there would still be something original in my work.

Since then I’ve signed NDAs (non-disclosure agreements) for numerous companies afraid that someone might steal their ideas. Maybe they’re doing the right thing to be cautious, but I doubt it’s necessary.

I think Howard Aiken got it right:

Don’t worry about people stealing your ideas. If your ideas are any good, you’ll have to ram them down people’s throats.

One thing I’ve learned from developing software is that it’s very difficult to transfer ideas. A lot of software projects never completely transition from the original author because no one else really understands what’s going on.

It’s more likely that someone will come up with your idea independently than that someone would steal it. If the time is ripe for an idea, and all the pieces are there waiting for someone to put them together, it may be discovered multiple times. But unless someone is close to making the discovery for himself, he won’t get it even if you explain it to him.

And when other people do have your idea, they still have to implement it. That’s the hard part. We all have more ideas than we can carry out. The chance that someone else will have your idea and have the determination to execute it is tiny.

]]>http://www.johndcook.com/blog/2012/11/02/nobody-will-steal-your-idea/feed/70Maybe you don’t need tohttp://www.johndcook.com/blog/2012/11/01/maybe-you-dont-need-to/ http://www.johndcook.com/blog/2012/11/01/maybe-you-dont-need-to/#commentsThu, 01 Nov 2012 12:00:16 +0000http://www.johndcook.com/blog/?p=12349One life-lesson from math is that sometimes you can solve a problem without doing what the problem at first seems to require. I’ll give an elementary example and a more advanced example.

The first example is finding remainders. What is the remainder when 5,000,070,004 is divided by 9? At first it may seem that you need to divide 5,000,070,004 by 9, but you don’t. You weren’t asked the quotient, only the remainder, which in this case you can do directly. By casting out nines, you can quickly see the remainder is 7.

The second example is definite integrals. The usual procedure for computing definite integrals is to first find an indefinite integral (i.e. anti-derivative) and take the difference of its values at the two end points. But sometimes it’s possible to find the definite integral directly, even when you couldn’t first find the indefinite integral. Maybe you can evaluate the definite integral by symmetry, or a probability argument, or by contour integration, or some other trick.

Contour integration is an interesting example because you don’t do what you might think you need to — i.e. find an indefinite integral — but you do have to do something you might never imagine doing before you’ve seen the trick, i.e. convert an integral over the real line to an integral in the complex plane to make it simpler!

What are some more examples, mathematical or not, of solving a problem without doing something that at first seems necessary?

Usefulness comes not from pursuing it, but from patiently gathering enough of a reservoir of material so that one has the quirky bit of knowledge … that turns out to be the key to unlocking the problem which someone offers.

Holmes was speaking specifically of theology. I edited out some of the particulars of his quote to emphasize that his idea applies more generally.

Obviously usefulness can come from pursuing it. But there’s a special pleasure in applying some “quirky bit of knowledge” that you acquired for its own sake. It can feel like simply walking up to a gate and unlocking it after unsuccessful attempts to storm the gate by force.