Having encountered errors in technical documentation that almost certainly came from incomplete editing after copying from earlier documentation, I am curious about what techniques can be used to avoid this problem.

Sadly, it seems difficult to strictly apply the programming principle of Don't Repeat Yourself since much of the documentation is written in a human language which is less friendly to automated macro insertion.

(Automatic consistency might be possible for tables and diagrams with a regular structure, and often such features could be generated automatically from lower level data. This would not necessarily guarantee consistency with the actual product, but [barring bugs in the translation software] descriptions would be consistent from the lower level upward. If this extended down to the hardware description language, only translation bugs would be an issue, but I do not know if that is practical and such would apply less to architectural documentation than to implementation documentation.)

Using macros as much as practical would help (e.g., many numerical substitutions could be automated), and it might be possible to include dependency information to highlight when changes in certain parts of the document (or system of information) might require changes in other parts of the document. In some cases, automatic generation of documentation that only needs modest modification for readability might be practical and minimize the use of copy-paste.

What practical techniques help avoid incomplete changes to documentation?

Just to check, you're asking about cases where, say, some feature in the documented software changes, you update three places in the doc that are affected, and you miss a fourth place that also needed to change? (Because you can't just click "refactor" like in your IDE...)
–
Monica Cellio♦Apr 6 '14 at 3:48

@MonicaCellio That would be one case. One case I encountered was in documentation of an instruction set variant where the text description of an instruction was apparently copied from the documentation of the earlier instruction set and the change in immediate size was not included in the text description (but was documented in the instruction format diagram). Inconsistency is slightly better than invisible errors (for the end user) but reflects poorly on the organization.
–
Paul A. ClaytonApr 6 '14 at 11:08

2 Answers
2

This is a hard problem. Unless your company has the resources to do full reviews of all the documentation on each release -- and if they do, I wonder how they stay competitive -- then you are at risk here. In my experience you can do some things to address the problem when it happens, and some things to reduce its chance of occurring.

Here are techniques I've used to find and fix missing updates:

Quick visual scan of the most-relevant documentation. This isn't a full review (you probably don't have time), but I try to at least pass my eyeballs over each page in related documents/sections to allow for the chance that something will jump out at me. This actually happens.

If a feature (API function, GUI element, schema element, etc) was changed or removed, I search the documentation for relevant keywords like the name of the function/element (however we talk about it in the documentation). This helps find stray references, such as the documentation of some other function that compares it to the one that changed. This has helped me catch cases like "The X function does blah blah blah; it's similar to the Y function except..." where Y changed.

If behavior changed then test cases probably also changed. I try to chat with the testers who'll be testing the newly-changed feature to find out what other test cases they had to change. (Note: even though I have access to their test-case repository, I find it easier to spend a little time talking with a tester instead. The testers know their test cases way better than I ever will. For similar reasons, they generally ask me what significant changes I made in the documentation.)

On a project with multiple writers, I choose peer reviewers who've worked on parts of the documentation that I haven't spent as much time on. They'll notice "oh if that changed then this other thing in 'my' doc needs to change too" before I will.

Here are some techniques I've used to prevent, or at least reduce the likelihood of, this problem. Think of this as defensive writing:

In the spirit of "don't repeat yourself" (DRY), if I need to include the same information in more than one place, I try to abstract it out into one blob that I can include by reference in all places it needs to go. Writing fragmented documentation like this is harder (you have to be careful to make sure that your blob remains valid in all contexts), but this level of repetition doesn't happen that much in my experience. But it happens some, so use include tools when it does.

Sometimes you know when you're writing something that it's interconnected with some other part of the documentation. Use comments (or whatever annotation mechanism your tool supports) to add findable reminders to yourself. Comments aren't just for code; they can be useful in doc source too. (My source is XML; your mileage may vary.)

When it comes to your bugs database, be nosy. I am automatically copied on all bugs in certain components, and I review new bugs often and add myself to the CC list on any that look like they might affect me. Yes, this generates a lot of email, but it's worth it to catch notes from testers, and also ones from developers who found they had to change something else in order to implement this. I get a lot of my best information from Bugzilla notifications.

Be more thorough in initial project estimation. At the beginning of the release-planning cycle, when the product manager, development team, et al are laying out the work to be done, you probably -- if you're like most people -- added a line to the project plan that said something like "update doc (N weeks)". But if you do a little more up front, you can avoid some problems for yourself and for the testers.1 While the developers are doing their analysis of what will change if they do such-and-such, you can be doing the same for the doc. So instead of that "update doc" line you might have items like "document new feature X", "update feature Y (that changed because of X)", "update doc of example that uses Y and Z", and "review doc of Y for impact". While you're talking with your friendly tester about this, you might learn that your tester knows that some part of Z is going to have to change too, so you can start thinking about that earlier. In other words, the more your team talks about the impact early on, the easier it will be for you to make your doc changes systematically and with less of a panicked rush at the end.

1 You may be getting the idea by now that I think the testers are your natural allies. They are. You and they are often at the end of the information pipeline, not because anybody meant to exclude you but because people focused on a narrow task don't always realize the implications themselves. Work together.

The code is already a form of documentation of what the system does. That is rather much the point of high level language code, to ease the understanding of systems for human readers of it.

This is a hint to a solution. Ask yourself, "who is the documentation for?" Is it for programmers to communicate to other programmers within your organization? Then do what you can to make the code clear, concise, not surprising, and DRY. DRY is about the future more than it is about the present.

And there is no point to keeping broken documentation. You would be better off discarding the docs and eliminating any chance someone might make a technical error by trusting potentially erroneous documentation. Bad documentation isn't known to be bad until it blows up in your face. Until then, it provides a false sense of security.

Is it for a program manager? Then seek to write documentation that is clear, concise, and doesn't hide behind technical buzzwords and faux legalese. Most people don't keep up with documentation because they find it difficult to understand what is already written, difficult to emulate the style, and despise doing make-work. There is little point in putting minute technical detail into documentation. Otherwise, we would have designed a programming language that uses MS Word documents as source files.

Is it for users? Then your documentation is part of your product. You need to dedicate specific time, and probably specific people who make careers out of writing use documentation, to the tasks. The project iteration is no more complete without the user documentation--a form of user interface--than the features it would cover. I tend to prefer to put such explanatory text on the screen next to the feature.

And that is generally a problem with all parts of a system: how closely located are tightly coupled components? You will see it happen in you database schema definitions if you manage them through raw SQL scripts. It will happen to reports generated in your system if the report code is "far away" from the report UI. Keeping related bits of the system located together is the only way to avoid them drifting apart, like some two samples of a species diverging over time after getting separated by an ocean.

Also realize that you don't have to follow the "obvious" process for writing documentation that program managers always seem to love. The last time I worked in a heavily documented environment, I spent at least 50% of my time on documentation. But though my program manager wanted the documentation written before the code, I did no such thing. I wrote them in tandem. I needed to write the code to figure out what would go in the documentation. I was lauded for having the best documentation and deadline rate in the company.

But the moral of the story is not "do it my way". It is that every company has its own culture that will value certain things over others. Use that knowledge to your advantage. Steal coding time from documentation time or vice versa. Write whatever code you have to to automate repetitive tasks. Question fundamental assertions and seek to minimize the work load.