Andrew Koenig

Dr. Dobb's Bloggers

Optimization and Context

January 18, 2012

Programmers who think about how — and whether — to try to optimize a piece of code should keep in mind the context of both the code and the optimization.

Programmers who think about how — and whether — to try to optimize a piece of code should keep in mind the context of both the code and the optimization. Context can push decisions in either direction: Depending on context, an optimization that seems questionable might turn out to be essential, a waste of effort, or anything in between.

For example, a friend of mine was once asked to help a commercial data-processing shop in which its daily processing was taking 28 hours. In this example, there were only two choices: Figure out how to make the processing faster, or buy a faster computer. Accordingly, anything they could do to get the processing time below 24 hours would be worthwhile — provided only that it cost less than upgrading the computer.

My friend told me that he quickly found one problem. Part of the daily processing was to merge two large files into a third file, and then to sort that file. The usual merge algorithm assumes that the input files have already been sorted, and this was no exception. As a result, the output file was already sorted; so sorting it a second time was unnecessary. The files in question were all on magnetic tape, and they were so large that simply eliminating this redundant sorting step reduced the overall execution time to less than 24 hours.

Having found such an elementary mistake, my friend was confident that he could find others that were equally bad. Nevertheless, he was told to stop looking: Once the execution time was less than 24 hours, there was no need to make it run any faster. The same context that made the first optimization essential made further optimization unnecessary.

Another example of the importance of context comes from a journal paper I once read that described a system that was built for newspaper production. The paper began by pointing out that nearly every aspect of the system's design came from two fundamental properties of newspaper production:

It is inconceivable that the newspaper should fail to be produced for any reason.

The presses start rolling on schedule, no matter what.

This system, then, had a lot of "this result must be ready by a given time" constraints, because any text that wasn't ready in time would result in a newspaper with blank spaces where articles should have been.

Yet another contextual aspect that often goes unappreciated is whether or not a human is waiting for the program's output. If you're running something overnight, you probably don't care much whether it takes one hour or four; but if you're waiting for your screen to change in response to a key you just pressed, a fraction of a second will affect your perception of how pleasant the program is to use.

For example, I once used an email-reading program that stored all of its messages in a single data file. When the program started, it would read the entire file into memory, and then run through the contents of that memory to locate the beginning of each line of text. The code worked by defining pointers named buf and endbuf that pointed to the first character and one past the last character, respectively, of the buffer that contained the contents of the file. Once the file had been read, the program did something like this (except that being a C program, it didn't literally call push_back):

At the time I was working with this program, it took about three seconds to read and process the biggest mail files that we had on our system. This three-second delay was enough to make the program feel sluggish.

I hit on the idea of making the buffer one character longer than the actual size of the file, thereby making the character located at *endbuf accessible. Not only was it accessible, but I could modify it freely without changing the contents of the file. This extra character allowed me to rewrite the loop:

Although I replaced one loop with two nested loops, the inner loop had less work to do than the original loop. In fact, on the machine on which this program ran, the original version executed five instructions each time through the loop (compare p with endbuf; conditional jump; dereference p; compare *p with '\n'; conditional jump). The revised version's inner loop made the first two of these five instructions unnecessary. As a result, the revised program ran noticeably faster on large mailboxes.

Under many other circumstances, this particular optimization would be a complete waste of effort. However, like the earlier examples, this particular piece of code ran in a context that made it worth rewriting for extra speed. Although the actual speed gain was modest, it happened in a psychologically important setting and required relatively little effort.

More generally, I think that in order to know how much effort is worthwhile to put into tuning a piece of code, it is important to understand how that code is going to be used and what kind of effect that effort is likely to have.

Dr. Dobb's encourages readers to engage in spirited, healthy debate, including taking us to task.
However, Dr. Dobb's moderates all comments posted to our site, and reserves the right to modify or remove any content that it determines to be derogatory, offensive, inflammatory, vulgar, irrelevant/off-topic, racist or obvious marketing or spam. Dr. Dobb's further reserves the right to disable the profile of any commenter participating in said activities.

Video

This month's Dr. Dobb's Journal

This month,
Dr. Dobb's Journal is devoted to mobile programming. We introduce you to Apple's new Swift programming language, discuss the perils of being the third-most-popular mobile platform, revisit SQLite on Android
, and much more!