Slashdot videos: Now with more Slashdot!

View

Discuss

Share

We've improved Slashdot's video section; now you can view our video interviews, product close-ups and site visits with all the usual Slashdot options to comment, share, etc. No more walled garden! It's a work in progress -- we hope you'll check it out (Learn more about the recent updates).

dnnrly asks: "I've been writing software professionaly for a couple of years now for more than 1 company and I've noticed a recurring pattern: I get put on a new project that already has a bit of history and I get told to read the design documents and then implement XYZ. What happens is I read the document, find that it gives me a lot of information about certain aspects of the system we are building, but leaves huge gaps in others. We're going to be rewriting some of the procedures very soon and I'll be able to influence the software side so I wanted to ask Slashdot readers what sort of things have they seen in design documents that they've liked/thought are a good idea? What have they found works and what doesn't? If all else fails, where's a good place to find all this stuff out?"

tester data"There's usually a very defined and rigid format for every design document and the writers have obviously tried very hard to make sure that procedure has been followed, generally leading to an almost unreadable doc or a design for the sake of it. Part of the issue is that these guys have written the design after 2 or more years exposure to the problem so they tend to forget just how much they know."

Reeves' article really is classic, and hugely valid even today. However, one of his most important points is left dangling.

The software design is not complete until it has been coded and tested.

100% true. The problem is, designers in traditional or semi-traditional development teams usually get only the most rudimentary feedback from test, usually just along the lines of "This doesn't work very well". In some places I've worked in (I'm freeelance), designers pretty much sat in their ivory towers almo

"The software design is not complete until it has been coded and tested."

I would have to disagree with this; a design is complete when it meets all requirements. A design can only be shown to be complete when it passes all testing; a subtle difference, but one that's come to my attention as being very important.

I have to agree with the statement about traceability though. One of the worst things I've had to deal with is getting a specification document that does some stuff and then have to write the requirements "that would result in this design if we started with those requirements." This is endemic to all industries: ill-formed or ill-stated requirements. I'm always amazed at how many "requirements" are either designs or implementations or are not testable or aren't requirements at all. Even more entertaining are tests that don't test in a way that proves requirements are met...

If I had to enumerate what makes good design the list would have to include:

DO NOT START DESIGN UNTIL YOU HAVE THE REQUIREMENTS (yes, I know there are exceptions, but you should at least have *most* of the requirements first).

Have a good requirements document: everything is actually a requirement (not a design or implementation) and is testable (if you have the word "not" in it, it's not testable, for instance).

Make sure the design is design and not implementation (design is "sum two numbers and check for overflow"; implementation is "temp32 = x16+y16; if temp32 > MAX16 then result16 = MAX16 else result16 = x16+y16;). Put another way, despite the popular writings I've seen lately to the contrary, CODE IS NOT DESIGN (any more than a car is the design of the car).

Make sure you can test that THE DESIGN MEETS THE REQUIREMENTS (which is subtly different than "does the design do what I designed it to do?"). If the design doesn't satisfy the requirements, it can never be a "good design".

Separation of responsibility. You mentioned XP above where "the designer is the coder is the tester" which is just bad practice in my experience. My company *strongly* discourages the designer to be the coder, or the coder to be the tester (the designer can be the tester though, and this often makes sense). This is because there are inevitable blinders that you put on when coding your own design or testing your own code.

Ok, that wasn't an exhaustive list by any means, but I think those are the key points. To summarise: have good requirements, separate requirements from design from implementation, and have many eyes looking at things throughout the whole process.

They're all important points. If I had to rank them in order, I'd sort them as follows:

Items 1, 2, and 4 are the most critical, followed by 5 and then 3.

I'd add subitem 4a: During design, don't add features that aren't in the requirements, unless you can show a derived requirement (usually from one what I call the "X-ability" requirments -- scalability, maintainablilty, testability, and reliability). But don't add it because it would be "a cool feature to have".

The five steps enumerated here are, sadly, rather untenable. In particular, points one and two make a fundamental assumption that I believe is invalid - get your requirements first. If we have learned nothing in the past 40 years of software, and especially in the last decade, it's that we will never have "the requirements." We will have subsets, or views of the requirements, true. The fact, however, is that requirements change during the life of the product development.

Skipping to point four for a moment: I would argue that the requirements should be expressed in the form of tests - functional, unit, or otherwise, and that these tests be executable. The design MUST therefore meet the requirements. This then means that point number three may or may not apply.

I agree that code is not design. Design is abstract, and code is not. And pure-code-no-design folks go overboard in my view, but if you can't have complete requirements (and you can't) then you certainly should document that which is not obvious from a structural breakdown of the code itself. Particularly, documenting risky or unusual features of the code, such as tortured relationships and their semanitcs. To me that is the limit. Document what is not obvious, do not document what is structurally implicit, or otherwise explicit. Documentation covers risk, and risk mitigation points (ie. person hours) are best spent making better software, except when documenting something that, if left unsaid, might lead other developers to an incorrect assumption or conclusion.

As to point number five, your experience may vary. In my experience, it's quite the reverse. But in the example I'm thinking about, we didn't have each individual designing - the whole team designed together, and the whole team coded and tested, and never alone. Also, design changes, or any changes to assumptions (or APIs, or etc.) were well published throughout the team, so that there was ongoing restatement of context. If anyone thought that something was an odd or interesting design feature/artifact, they mentioned it, and the author clarified it and documented it. I don't think having individuals altering design outside of a process of consultation is wise, and I think this "cowboy" flavour of agile developent is abortive and a throwback to no-process development. However, small teams that are engaged collectively produce, in my experience, excellent design. You then need to capture what is not obvious, or somewhat obfuscated, and especially capture assumptions.

Where ThosLives and I would agree, however, I think, is that you need to trace design aspects to requirements. You may not have final requirements, but you always have some, and you shoulnt' make design decisions without a requirement to trace to. In this sense, you absolutely must have tracability from requirement to design to implementation to proof (test). And if requirement can be expressed as a test (not all requirements can), then that is, in my view, the best way.

I'm a CEO, and my company is profitable since 1995 -- I know the risks and the opportunities of projects and associated contracts.

But I also know that non-insistance on precise, predefined requirements got us lots of contracts; in especially the circumstances that our customer doesn't know exactly what he wants and that it is part of our job to find that out.

There are other ways than technical or functional requirement specs to bring this shared understanding in a contract and manage the risk for both s

I've been on projects where the RTM became God and we ended up spending more time documenting how X -> Y -> Z throughout the document tree than we did coding it. There is some sort of happy medium.

OTOH, I'm working on a project now with:No design docs.No requirements in written form unless my company writes them for the client, and even then they are abbreviated severly.The great habit of getting a bug report from QC, wanting to say "Show me the requirement" then realizing there isn't o

Program does not dump if an alpha value is entered where a numeric is expected

Okay, I'll make the program erase all your files, hack into your bank, and transfer the balance to my offshore accounts when I get an alpha value when a numeric was expected.

This meets your requirement; the program didn't "dump". *grin* Hopefully that illustrates the dangers of most "not" requirements.

The best way to write the requirement you're looking for in this instance is something like:

"Ignore alpha values where a numeric is expected."

Or, "If an alpha is given where a numeric was expected, ignore all following input up to [some delimeter].

Or, perhaps, "If an alpha value is given when a numeric is expected, display an error and get the input again." The general idea is you want to define the requirement and the bounds of operation; "not" requirements are unbounded.

I will grant, however, that sometimes you want a "not" requirement; this discussion [slashdot.org] has more details on 'not requirements' and rjh [slashdot.org] points out that sometimes folks use not requirements to prohibit things but purposefully allow anything else; in my opinion this is very dangerous and should only be used very rarely.

Okay, but they aren't functionally equivalent. In the first spec, are users with such-and-such authorization allowed to change property yada? Although it implies it, the specification doesn't actually say that they can. The permission is actually undefined; the "unboundness" that the parent is talki

I think people misunderstand what happens when code is compiled. Compiling is not "the compiler uses the source code as instructions to produce an executable"; it's merely a translation of human-readable instructions into computer-readable instructions. I can "read" code and produce the same results by stepping through by hand; the executable code does the same thing (generally much faster). In this sense, the code *IS* the final product; the Reeves article incorrectly equates compiling to building. The "building" is the writing of the code. Compiling is more like translating a book from English to some other language.

When I use the blueprints for an automobile to build the automobile, I get something different than the blueprints. I cannot use the blueprints to emulate the functionality of the vehicle. The blueprints are instructions to build the vehicle.

In software, the design is the thing which tells me how to write the code (have you ever tried to read code and find out what the function is supposed to do? generally you can only know what it does, not what it's supposed to do. Sometimes you get lucky and the design is written in the comments, but that's rare; generally comments just say "we watch out for overflow here" or something like "use pointer rather than array lookup to avoud some tricky thing" - comments are usually descriptive of what is going on, not why). If I have a good design I can write it in any language and meet the same requirements. That's a good litmus test for software: if it depends on a language, it's not a pure design. If you have to figure out what's going on to translate into another language, you're in implementation rather than design.

I will grant, however, that there are some difficult situations when you have a requirement "you shall use language X for this project", or "we decided to use language X to meet these requirements" - what happens if that requirement changes halfway through and your "design" was language specific? You're hosed. Good designs are language transparent.

Remember, the goal isn't to produce an application, the goal is to meet the requirements. The design tells you how you are going to meet the requirements ("have a text grid input box" and, at the next level, "have a function that draws a text grid input box") and the code fulfills the design ("textgrid.draw()"). (Note that some design decisions become requirements for a lower level, such as function calls, interfaces, etc.). Code should only be implementation.

I think people misunderstand what happens when code is compiled. Compiling is not "the compiler uses the source code as instructions to produce an executable"; it's merely a translation of human-readable instructions into computer-readable instructions. I can "read" code and produce the same results by stepping through by hand; the executable code does the same thing (generally much faster).

This viewpoint seems to be largely tautological - two forms of an algorithm are equivalent because their input/outpu

I'm still taking CS classes after ten years in the business. The books my college uses present an assortment of different design techniques and various documents that go along with them. They do not present an absolute solution.

I dare say that I haven't seen much in the way of design documents in anything I do. A "design" to me, typically means a braindump committed to email. It's not too surprising either, I doubt many software engineers got into the business to write documents.

"I doubt many software engineers got into the business to write documents."

No, that's just it. People often ask the difference between a Software Engineer and a Computer Scientist. This pretty well covers it. Ideally, a Software Engineer is an Engineer like any other. A major portion of their work is designing and documenting. Like how a Structural Engineer would create blueprints. Also ideally, a Computer Scientist is a scientist like any other. He investigates the world of computer software - experimenting and recordign the results.

In the real world, the roles of both blurr into "the customer wants this, code it." But to live up to the title of Software Engineer, you need to be much more proactive and be very involved in the non-programming aspects such as requirements gathering, documenting, designing, documenting, prototyping, documenting, and documenting.

Software Engineers are pretty thin on the ground. I don't know of that many. Well, truth be told, I've not seen one since moving to the US. Real design work is rarely done, and documentation is something added on at the end.

Getting back to the original question, a design document should be divided into three primary sections. The first section should always say what the problem is that the design document is trying to address. Any given design document should address only a limited set of problems, or it will become too complex. Ideally, any given document should be small, compact and address one or (at most) two issues.

The second section should describe how to use the solution presented to solve the problem described. In programming terms, this is your API. That is all it should describe.

The third section should then cover how the solution goes about solving the problem. Again, that is all it should cover.

When projects or modules within those projects are updated, it should be possible to update/replace any one section in any one document without touching any other section or any other document.

Documentation should also always stick to the level for which it is intended. Ideally, references should be to ever-more specific information, never the other way round. So, a project overview might point to the components of that project. Each component might then point to documentation on the low-level mechanics. On the other hand, a discussion on low-level mechanics is not the right place to talk about a specific project that uses those mechanics. That gets in the way of understanding and obstructs code reuse.

That, I think, is the key to writing good documentation. Does it positively assist those who would need it? If the answer is "no", or is even a "not sure", then it doesn't matter what standards it follows, it is useless.

The ideal length for a document is going to vary, project to project, but by building documents in a modular fashion it should be rarely necessary to have a document longer than about 20 sides of A4. Most should be around 10 sides. Anything longer likely covers unrelated topics and can be split up. (Remember, it is easier to open 20 books to one page each, than to open one book to 20 different pages.)

This limit gives you about 3-7 sides per section per report. If you need more than 7 sides, the design is too complex and unmaintainable over the long-haul. On the other hand, any less than 3 sides likely means that the project has been over-designed with lots of redundancy, plenty of overhead and no possibility of meeting the requirements.

Going by this description, virtually all documentation in existance is ghastly beyond all imagining. Which, by and large, is exactly how most people see it.

The ideal length for a document is going to vary, project to project, but by building documents in a modular fashion it should be rarely necessary to have a document longer than about 20 sides of A4. Most should be around 10 sides. Anything longer likely covers unrelated topics and can be split up. (Remember, it is easier to open 20 books to one page each, than to open one book to 20 different pages.)

Yes. Tetris and MS Office have the same target complexity, to within fifty percent. Also, generalization

Anyone idiotic enough to put MS Office in a single document is asking for trouble. In the same way that programmers learn quickly to break things down into managable procedures and functions, documentation writers need to look at one small compartment at a time.

In the case of MS Office, for example, the requirements for loading and saving documents has nothing to do with the requirements for handling macros, and none of the above has anything to do with the way that the GUI is structured.

"But to live up to the title of Software Engineer, you need to be much more proactive and be very involved in the non-programming aspects such as requirements gathering, documenting, designing, documenting, prototyping, documenting, and documenting."

You make an excellent point. It's the difference between being a developer vs being a "programmer", something that Eric Sink (founder of a small ISV called SourceGear) wrote a very nice article [ericsink.com] about.

Figure out which class from your local university deals with software engineering, find the book(s) for the class, and buy it.

Then burn it. The methods are all nice in theory, but in practice it's often a crapshoot. Then again, I have a bit of history on this one, as my software engineering class was more of a class on how to deal with the PHB (clueless professor).

We learned how to make all the pretty diagrams, while we discovered that such pretty diagrams could never have relevance to our particular project. It ended up a game of "make the diagrams the customer wants, then make much simpler and more sensible ones for yourself that don't follow proper software engineering protocol but actually tell us how the darn thing works."

This is where my classmates coined the phrase UML Nazi. Some professors were so obsessed with producing "proper UML" that the diagrams the professor asked for didn't really explain how things worked, and the documents that explained how things actually worked, got shot down as not being proper UML.

A lot of the time, we just put diagrams that made sense to us, and that worked a lot better than trying to make proper UML documents.

That's why you go to school. Much of the learning is formative, you'll use little of it in "the real world", but you'll learn "how to learn". Do I use all those years of integrals and derivatives? No, but they trained my mind into the proper engineering mode. While in school, follow the excercises, once you are outside you'll quickly figure out what works and what doesn't, and sometimes, you'll find yourself coming back to things you learn in school a long time ago.

Indeed. For me, the most important thing to remember about UML, and other methodologies, is this: They are tools to be used, not rules to be followed. Too many PHBs (and colleagues) don't seem to understand this.

The most important thing for a document is to communicate. UML is a very good way to communicate that sort of information (when everyone understands it). But it's not the only way. Use whatever's best. If a few lines of text on a scrap of paper, or an ad-hoc diagram you've just invented, will communicate better (with your memory, with your immediate colleagues, and with anyone who might be maintaining the system in future), then by all means use it! Anyone who insists on UML or whatever for the sake of it is missing the point.

The main problem with these methodologies is that there's no way to force people to think clearly. You can encourage it, certainly; you can make it easier to do, easier to communicate, and easier to spot its absence; but you can't force it. Following the procedure is no guarantee of clear thinking -- and it's that clear thinking which gets projects designed and built.

It ended up a game of "make the diagrams the customer wants, then make much simpler and more sensible ones for yourself that don't follow proper software engineering protocol but actually tell us how the darn thing works."

The person writing the document should have enough software/hardware resources. So many engineering specs are written too early in the phase when there is nothing tangible. It just becomes a fairy tale document.

The design document was rumored to be a cross between a unicorn and the dodo bird. What would a bird need with a big pointy horn on it's head? I don't know, but then, I've never seen one of these rumored documents, so I can't say for sure.

I don't know, but then, I've never seen one of these rumored documents, so I can't say for sure.

Sure, you have. RFCs are my idea of what makes a good design document: tell me what I have to do to make this software interoperable with the rest of the world, what you want this software to accomplish (including sample gozintas and gozoutas), and that's it. Short and sweet. Now get out of the way and let me code.:^)

An RFC specifies what behavior MUST, MUST NOT, SHOULD, SHOULD NOT, MAY, MAY NOT exist. It doesn't say jack about how that behavior is supposed to come into being. I could write an OpenPGP application that did all of its work by hiring Bruce Schneier to manually do the RSA computations, and it'd pass the RFC.

This is not an insult. I want to make that very clear, right up front. That said:

Go take a software engineering course. There's a lot of wisdom that's been accumulated about how to write reliable software, and while (IMO) about two-thirds of it is utter rubbish, one-third is really good advice. Learning software engineering is like eating pufferfish: really cool, but be careful what you swallow. You'll learn about requirements analysis, drafting a coherent requirements document, architectural analysis

Write them before you write the new features, to test for the new features.

This will save you eons when you are actually coding. Way too many testers are still wasting their time with manually going through the ssame maotion again and again to make sure it all works as expected.

Talk to the various stakeholders. Hold meetings. Get everyone's input on what's the Right Thing To Do.

To the degree these ideas are not the Wrong Thing, do them, even if they're less efficient than you'd like, or are less fun to code. You're going to be giving them a prostate exam with a cheese grater in a couple of steps, so soothe their egos proactively by letting their ideas make it into the final product.

Take the draft to your dev team. Circulate copies, have everyone read it, then have a short meeting--one hour, tops--not to discuss how to do things, but which parts of the design will require a lot of experimentation and fiddling.

If your dev team doesn't already have someone fluent in Corporate Weaselspeak, then get one.

Give your translator this sentence: "We will use our magic powers to accomplish this part of the design document." Have him turn it into a five-page monstrosity that lets every stakeholder think these difficult parts are going to be done their way, without really committing your dev team to anything.

Take the weaselized design doc back to the stakeholders. Your Corporate Weasel's job is to make the stakeholders sign off on it.

The easy and routine parts of the job get done the way the stakeholders want, assuming their way isn't completely braindamaged. The hard parts of the job will be solved by your development team's magic powers. It's right there in the design document.

Bring the project to completion. As you're doing the hard part, write This Is How It Really Works documentation for engineers who are coming after you.

When your project is ready for handoff, make sure to praise the (easy, routine) parts for which you used Marketing's ideas of how the software ought to be written.

Gloss over the fact that you did the hard part via magic powers. The other stakeholders probably don't care. You're giving them a beautiful bullet point for their end-of-year performance eval. That's what they care about at this point.

Move on to the next project.

... Is all this weasel office politics? Damn straight. On the other hand, it's weasel office politics meant to shield your development team from unnecessary weasel office politics. As much as we hate weasel office politics, sometimes it's necessary.

Be sure your DD includes:- Sitemap (web) or screen map (desktop app)- Feature matrix with columns for features, rows for pages/screens and indicators where a page/screen has a feature.- Detailed feature specs need to be written kinda like function documentation - "preconditions" that state what is expected as in the user is authenticated or the DB has products and "post conditions" that say what gets set or hwat the user can do.- Style documents greated by graphics designers who understand the medium (web/Windows API/etc) are invaluable for refering back to when someone doesn't like a font size.

All this documenting is no fun, but it is more fun then dealing with the lake of the document.

- add a button to screen A that does function X- change the layout of screen B so that it shows ALL of the widgets in chronological order

etc.

To do that, you want to start off with a nice diagram of the screens showing which UI elements come from which classes/controls/include files/etc. so that you know where to go and start looking at code.

90% of software design is about putting code in the right place. That means architecture consists, most

I document software for end users, and get some of my information in the early stages of development from design docs. I have to say that the most useful design documents that I've seen are ones that contain diagrams, flowcharts, and other "pictures" of how the system (or the module, component, whatnot) functions.

If I have to choose between a 50-page design document and three pages of clear diagrams, I'd pick the pictures.

I have to say that the most useful design documents that I've seen are ones that contain diagrams, flowcharts, and other "pictures" of how the system (or the module, component, whatnot) functions.

Amen. I do Quality Assurance (and for those who don't know, that isn't just testing). I use design docs to figure out how something is supposed to work before I get it. Pictures are good. You can (intentionally) bury information in a 50 page document. It is hard to bury information in pictures. I say "intentionally" above because in the past I worked with a guy who was the director of a development group. He didn't like to design things, or tell people about how things were going to work. So his requirements and design documents were vast containers of information. His standard answer to questions was: It's in the document.

Me: What is the flow of events from beginning to end?
Him: It is in the document.
Me: I couldn't find it. Where is it?
Him: It's in there, you just have to find it. See, here on page 3, and on page 10, and...ummm... You just have to piece the information together, but it is all in there.

Talk about information hiding. In meetings, people would ask questions, and he would say "It's all in the document, should I go get it?" Nobody wanted to spend meeting time sifting through it for answers. And sometimes, the answers weren't there, and we would always get the "I'll add it". Of course, nobody ever checked to make sure he added it.

I fought for months to get him to add a flow diagram in a doc. He kept insisting that all the information was there and that a diagram was useless. After months and months, he finally added it. The FIRST thing that someone said at the next meeting was how useful that diagram was, and they pointed out some improvements to it. It turned out those comments sparked conversations that led to the discovery of flaws that went unnoticed for months. I'll leave it up to the reader to guess who got credit for the diagram in the document. (hint: Senior QA person or director of development)

Overly rigid structure and formatting are not helpful. Often times I think programmers are so used to writing for compilers that they forget that humans are much better parsers.

Design documents should be easilly written and easilly updated. I prefer a text editor to something where I can have 'really nice' formatting. Its just easier and quicker, and that leads to the documents being more likely to be kept up to date.

If the docs aren't up to date, no matter how well written or well designed they are, they are misleading and unhelpful.

Unfortunately, templates and standards for design documents rarely are left to the hapless development staff but are instead left to the people who don't have to read them... Oh, they have to look at them, bind them, distribute them, and related "business" machinations, but they don't really have to READ them, and thus far too often technical teams are left having to shoehorn designs into inappropriate, rigid, and non sequitor documents. If this were an only-one-time experience in my career, I'd have more

Basic communications 101 says the purpose of any document is to communicate. And you, the poor schmoe left with maintenance, is exactly the guy the design doc is supposed to be talking to.
Now a lot of times businesses create documents just to "check off the boxes" in which case they want some big, heavy monstrosity to deliver. If that's what you're getting, good luck.
People (especially programmers) tend to think of design docs as some kind of ultimate bible in the sky that's going to answer all questions and be a completely accurate guide to what's in the code. That's fine in theory, but in the real world the best design doc you're going to find is one that tells you what the design team was thinking when they started down the path of building this program. What's the patterns that were used? What constraints kept them from doing things differently? It's just a technical memo from them to you that's supposed to help you get oriented and work more efficiently.
Sometimes those memos aren't done so well. My advice is to get what general information you can from them, and then talk to the coders who worked on the project.
That's my two cents.

The source, especially the comments. Assuming there are comments, and they don't read (I really saw this once) "Why did I do this?"

A good design document often duplicates the actual programming process. It starts at a high level (What are we trying to do? How do we do it?) and drills down to the level of what each function (or class method) should do. It should define acceptable inputs and outputs, both overall and for each method. It should also define the unacceptable ones. You should be able to generate the unit tests from it as well as the actual program.

It should leave room for changes if you find out that using methodology Bar to implement Foo doesn't work.

The Final Document should be based on a reading of the code, and any differences between that and the pre-code spec should be documented as to their cause.

The biggest problem I've had with design documents is that they aren't updated.

When first written, they are good and complete.

As the project moves along and the design changes, the design documents aren't updated. If you are lucky, new features will be covered in errata documents, but old features that were found to be not feasable remain in the document, without any indication of why they were removed from the project.

I've noticed a recurring pattern: I get put on a new project that already has a bit of history and I get told to read the design documents and then implement XYZ.

1. Write a design document containing information about the design document you're trying to create.
2. Read the design document you wrote in (1) describing the design document that you are trying to write.
3. Write actual design document as described in design document written in (2).

I've seen alot of design documents. Most of them fit into a sort of cookie-cutter as to what the company expects. If UML is standard, expect to see alot of crappy UML with substandard explanations.

The best design documents are ones that aren't trying as much to fit the cookie cutter as much as they clearly convey the idea of the design.

I'm not saying to throw design standards to the wind. I'm saying that your standard as a design document generator is to create something that is readable, decent to look at, and clearly conveys what is going on.

Try to use vernacular vocabulary and language style when explaining what things do, and try to make your pictures look as pleasing and simple as possible.

There are lots of things that can, and arguably should, be documented in a design document. One category that should never be omitted is the interfaces. Whenever a system or subsystem interacts with something else, you need to know how. This includes:

Database schemas. Unless your DB is so well hidden that no one will ever find it, someone will want to read it some day.

APIs you support or use. If there's a standard or a 3rd party product, just list it and the version.

Command line parameters for those little maintenance tools you don't tell the users about. This is a design doc, not a users' guide.

What happens is I read the document, find that it gives me a lot of information about certain aspects of the system we are building, but leaves huge gaps in others.

If you can already identify gaps in previous design documents, then you are already qualified to write the next design document.

Apart from that, talk to some other experienced people in your organization and get their take on previous projects failures and delays. Then, see if there is any way to preemptively incorporate measures into the design document to improve your projects chances for success.

[Although, a lot of project success really boils down to getting the right people on the team.]

There are a lot of things that can be said, but I think the biggest problem with design docs is that they aren't kept up-to-date.

If you're really serious about having useful design docs, the only way to do it is to have a dedicated staff whose job it is to keep it current, accurate and useful. Coders are rarely good writers, and even if they are, it's almost impossible to get them to keep docs in sync.

To be honest, I find that customers have a hard time visualizing how something will work unless they can interact with it. I'll take some internal design notes for myself, then whip up something and reiterate.

This approach doesn't work well with some projects, but for the small web applications I make it works great.

But most 'old-school' outfits using heavy waterfall processes think that prototyping is evil if you do any of it prior to writing your 50 to 100 page 'design document' (I worked at a place like this a few years back).

Of course the company that uses Agile processes and encourages prototyping will probably get to market way before the dinosaur company.

If your product has a UI, then the design document should start by describing the UI and how all the features work.

If you don't have a UI, then start from the configuration file or command line switches, and do the same thing.

Then you're done. If you can't configure a feature, or can't trigger it from the UI, then it shouldn't exist...unless there are other interaction points.

If you have a network app, document the wire protocol, range of input values, and expected behaviors (of your app and its clients) and outputs.

If it's a distributed system, figure out the different states and document them.

It's actually a pretty simple, though tedious, process. Starting from the UI is a great place, because it shows you instantly how complicated your app is. If it's too complicated, this type of document will show that pretty quickly.

Instead, document why you chose PostgreSQL over [foo]. Why you chose to roll your own templating system. Why you fork off jobs in a separate process rather than doing them in one process. Why you wrote this particular thingy as a C extension.

Documenting that stuff will be helpful to folks down the road when requirements/environments/whatever changes and they wonder why things were done this way in the first place.

My preference is to get a list of features from the customer in the form of User Stories [c2.com], and turn those directly into Acceptance Tests [c2.com].
Tests tell you what to do, and when to stop.
I also put a rough estimate on the various user stories, and then let my client prioritize the stories so the work I do first is what they want most.

In many cases, clients don't know what they want until you don't give it to them. The best approach I've found for those cases is to whip up a tiny prototype to show to the client

The primary problem is that there's too much information to document. I could create a document which included every design decision and every little facet of the project, but the document would wind up so huge, it'd be impossible to work with. The best resource I know of is the project guru. Every project has one -- the one guy who seems to know everything or at least can tell you where to find the details. Find this guy and pick his brain as much as you can. That'll carry you a lot farther than the documentation itself. However, if the guru is no longer around, you're up a creek without a paddle.

A Good Product Manager
This person should follow the development of a product from beginning to end, and document as well as be responsible for the production of the Design Requirements.
The Product Manager should work in the business/marketing side of the company, as ultimately the customer will determine the success of the product, however they should have a strong enough background in the technologies that they aren't some caricature from a Dilbert cartoon. It isn't necessary to define individual segme

[diety] save me from documentation that consists of links upon links upon links. I certainly find this disconcerting, and I imagine I'm not the only one in that execution based on these documents often seems disjointed with significant factors going lost in the process.

Whatever you put together, try to ensure it's not blown into thousands of pieces. Documentation requires some effort; updating should be more than just glomming on a paragr

Once your model contains a good description of these three domains, expressing a design document from the model is straightfoward (indeed, many of good modellers will provide excellent document generators). XDE works fine, but my pa

In my experience, I'll be honest and tell you that design documents are a pile of crap. They're usually written from a limited point of view with the assumption that you understand all the nuances of the business. Most of the time, that is not case.

You're a lot better off talking to the key people involved in either the business process or decision making. If you're working for a smaller company, this is relatively easy to do; however, if it's a larger company, I would think it's more difficult (ie less a

1.1.1.1.1.1.1.1.1 The system shall not make tacos on demand.
1.1.1.1.1.1.1.1.1.1 The system shall not make tacos on demand with sour cream.
1.1.1.1.1.1.1.1.1.2 The system shall not make tacos on demand with guacomole.
1.1.1.1.1.1.1.1.1.3 The system shall not make tacos on demand with cheddar cheese.
1.1.1.1.1.1.1.1.1.3.1 The system shall not make tacos on demand with sharp cheddar cheese.
1.1.1.1.1.1.1.1.1.3.2 The system shall not make tacos on demand with mild cheddar cheese....

Yep. Reference the requirements document from the design document, showing how aspects of the design address each particular requirement.

You may also find it useful to split the design document into a hierarchy of many; one or more high-level documents for module-level descriptions, each having one or more low-level design documents for stuff like unusual algorithms or other non-obvious aspects of the design of a particular module.

A good design document, like any technical document, should be well written. All too often, something that could take a sentence to write is expanded into multiple paragraphs or even pages, because the writer likes to feel important. Keep it relevant, and keep it simple.

Also, despite what they tell you in academic paper-writing classes, it's *all right* to write in everyday language. A statement such as "The user moves pointing device A over the activity area of the button in question, intending to act

The basic idea behind writing anything right is to know your audience. You've used these documents before -- write out a list of all the information you, as a user of these documents, need to get out of them to do the work properly. The format, etc., should support the answering of these questions -- don't decide on a format/structure and then shoehorn the information into it.

The best thing you can do is compose an outline or draft and show it to as many people as possible. Get feedback, criticism, and suggestions, and update the outline. Then get feedback again. The more eyes, the fewer mistakes.

No matter what the official titles may be, the real design document is the test plan.

This is because if the product should accepted if and only if it satisfies the test plan. A good test plan will thoroughly exercise the conditions that the product will encounter, and specify the expected response. Once you know those things, generating a traditional design document is trivial (plus or minus nontrivial graphic design).

Because PHBs expect something called a Design Document, I suggest you build a database o

I think that Joel Spolsky addresses software design docs well. Among other things, at one time he was responsible for writing the software spec for Microsoft Excel.

Summary: This series of articles is about functional specifications, not technical specifications. People get these mixed up. I don't know if there's any standard terminology, but here's what I mean when I use these terms.

A functional specification describes how a product will work entirely from the user's perspective. It doesn't care how the thing is implemented. It talks about features. It specifies screens, menus, dialogs, and so on.

A technical specification describes the internal implementation of the program. It talks about data structures, relational database models, choice of programming languages and tools, algorithms, etc.

When you design a product, inside and out, the most important thing is to nail down the user experience. What are the screens, how do they work, what do they do. Later, you worry about how to get from here to there. There's no use arguing about what programming language to use before you've decided what your product is going to do. In this series, I'm only talking about functional specifications."

The design documents which have driven you mad probably weren't incomplete. They were probably quite complete design documents. But a complete design document isn't supposed to cover everything. If you had a design document that was fully specified in every significant detail, you could run the design document through a compiler and generate your code. (This isn't as far-fetched as it sounds; there are tools to automatically generate large amounts of code just from simple UML diagrams. That's an example of design documents being translated directly into running code.)

A design document can best be viewed as the development team's prejudices regarding the best way to solve the problem. There will be holes in the design document, mostly in those areas where the programming team doesn't really have a good grasp on what the best thing to do is.

A good design document is a like a good steak; they're best when served a little bit rare. You want to give the guy who comes after you a game plan, but you don't want to commit yourself to doing things in one particular way when you don't know if that one particular way is going to work. After all, once the design document has had every stakeholder sign off on it, going back to the drawing board and saying "uh, this isn't going to work, let's try something else" means all the stakeholders get back on board again. But if the design document has some room to move--what you think are "holes"--then that gives the programmers freedom to get the job done without having to go through the entire design approval bureaucracy again.

Most serious software engineering shops worship at the altar of requirements and architecture documents. Hackers in the trenches add the "this is how it really works" document to that list. There's nothing quite as valuable as stumbling across some prior hacker's notes when you're trying to grok the system.

Stop looking at the design document for the 'missing' stuff. It might very well have been deliberately omitted. Start asking around for the "this is how it really works" documentation, instead.

Every design involves choices and tradeoffs.Each of these requires a decision.

In order to intelligently maintain and extendthe project, the owners need to know why eachdecision was made: what were the design goals,what were the available alternatives,what were the constraints, and what might bedone under different alternatives or constraints.

That is, explain why the design is the way it is.

"This algorithm has n-squared behavior, but
was easy to implement and test. N-squared
behavior is toler

You can write the best design document around, but if it doesn't address the requirements, it won't produce a good result.

I think that most requirements gathering activities don't involve the right people. Most importantly, they should involve the developer. I have seen requirements sessions involving only the business analyst, an IT lead, and the project manager. They leave out the developer! Usually it is done because the developer is busy with other things.

I like to describe why I did things in a particular way. Much of what ends up in a design document describes what the system does (I can run the app to see that), or how it does it (that's what the code is there for and I guarantee that the code is more recent than the design document).

What always gets me is when I show up and ask "Why didn't you use XXXX?" No one knows. Often there is a really good reason that I find out after weeks of unsuccessfully attempting to implement XXXX.

Having someone that can speak and write English do it will go a long way. My current project is working from a design generated by an Indian guy that has no clue what half the words in the language mean. I resent having to rely on half-assed work when I'm not in a position to advise on the design. It's not hard to run the grammar checker.

If the design document is underweight, it was good enough to trick the bean counters into giving up the first check. But you really can't use it to build test cases out of it since the developer can add or subtract whatever they want.

If the design document is overweight, you can build test cases with a fair degree of accuracy over the life of the project. The problem is that some developers will bristle at being held accountable for every detail promised and don't like their milestone checks being held up until they deliver the goods.

The ideal design document that lays out what to expect without too much overwhelming detail and the developer delivering to full spec on time probably doesn't exist. At least, not in the video game industry.

Much of software engineering is received wisdom. It involves little engineering and even less science.

I draw the analogy to medicine in the 19th century because at that time physicians were finally trying to investigate the causes of disease and developing insight that, more than a century later, would lead to their interventions improving rather than diminishing life expectency.

Nobody knows what "best practice" should be, yet we're codifying a process. The design document is one aspect of this process. Rarely is the purpose of a design document to convey design. Rather, it is a "deliverable" presented to a PHB or client as evidence of progress. To this end, the larger and prettier it is, the better.

A good design document starts with good requirements. It also requires that your process have some way to capture and fold changes back into the documentation as necessary.

A good place to look is "Software Requirements--Revision" by Alan Davis (1993). I don't agree with everything Davis has to say, but the book is full of good ideas and potential "gotchas" to watch out for. Another good reference is DO-178B, the guidelines used for the development and testing of safety-critical software in commercial aerospace applications. It is available from the RTCA [rtca.org] for about $50.

If you're doing an OO project, then you might want to look at Booch's book: "Object Oriented Analysis and Design" and the UML 2.0 specification.

But, most importantly, you MUST have some kind of design documentation (requirements, at a minimum) and that documentation needs to be flexible enough to accomodate changes without causing everything else to grind to a halt while the revisions happen. Expecting to get good software with inadequate formal documentation, minimal planning and insufficient requirements is why 70% of all commercial software projects end in failure (documented and published statistic).

Anyone who says you can do good software by shooting from the hip is nuts. And they don't work for me.

I'm not totally sure what you mean by Design document, I've seen many software shops. Some call the requirements document design, while others define the design document as the document that describes the high level description of the internal architecture of the system (system diagram, major modules, client/server decomposition, class diagram).

Either way, I like to start with some templates I created based on IEEE standards, a few come to mind, Here's the list:

Some of these are very documentation intensive, but I find that at least reading through them when starting a new project helps me direct my thoughts and make sure I don't forget anything that might be relevant to the phase in question (what? I need to think about the maintainability? the stability? the robustness?) pick and choose those things that apply to your project.

There's usually a very defined and rigid format for every design document and the writers have obviously tried very hard to make sure that procedure has been followed

Exactly. I used to work at a company that had a 50+ page document that was supposed to be used as a template for design documents - a sort of meta-design document, I suppose. Problem was that things rarely fit into their preconceived ideas. It definately led to unreadable documents. The other problem was that the procedure to change the design document was so draconian that everyone was hesitant to change anything even though it would become clear that things needed to be changed when the real world crept in. (BTW: this company allowed absolutely no prototyping prior to finishing the almighty design document - for those of us who like to prototype, it was frustrating)

I'm thinking that wikis are the best way to both communicate and develop a 'design document'. The design document should be flexible and wikis fit the bill. The design document should show the history of thinking about the design and again, Wiki's fit the bill quite nicely as you can look back and see the discussions that took place that led up to various decisions.

In addition to a design doc, which you may or may not need to update, depending on how I interpret your immediate task, you really should have a maintenance document.

Design Docs tell the reader how you intend to build something. In many shops (depending on the SDLC version, if any, that they use), it has little value after the design review.

As a roving consultant, I've seen a lot of crappy documents and SDLC (et.al.) procedures, and what has the greatest value for the programmers who come later is a Maintenance document that specifies:

1. The overall philosophy of the design: Why specific design choices were made.

2. What things did you want to implement (and why) but were postponed to a later version.

3. What design decisions were specifically rejected (and the reason).

4. Where are the system's weaknesses?

5. What are the coding standards specific to this particular project?

The Maintenance Doc provides guidance to the people who have to work on this stuff after you're gone (and to you, unless your memory is perfect). More than anything else, they will need to know the why of the design, not the what.

In my perfect world, the maintenance doc would be the main appendix to the design doc.However, in most shops, the SDLC & its policies & procedures will prohibit that, and usually acknowledge neither the existance nor desirability of a Software Maintenance Document.Good Luck.

I really really enjoyed Joel Spolsky's series on writing Painless Functional Specifications [joelonsoftware.com] back in 2000. Granted, this doesn't refer to design docs (or technical specifications as Joel calls them), but I think some of the ideas carry over.

You should read the entire series, but I'll give some of the highlights:

You should write the spec before you start the project, not during.

It should have one and only one author

You need to flag Open Issues and Side Notes by themselves so you can search for them later.

The functional spec should be a living document.

Be funny: specs are only good if they are read. The more funny anecdotes the better.

Be pithy and understandable.

Lots of screenshots, tables and diagrams to break up long pages of text.

The single biggest mistake most design documents make is that they document the design.

That's nice, to a small extent, but generally of relatively little use.

It falls down completely when- the designer made bad assumptions that subsequently don't hold- the users change the requirements- someone actually writes the system- the system goes through years of maintenance

So what is of use in these circumstances? The ideas, concepts, approaches and general thrust of the design.

Where did this design come from? Why has this approach been taken. What are the concepts embodied here?

Don't tell me that the Widget is round and talks to the Doodah.

Tell me _why_ the Widget is round (and why square wans't good enough), explain what the Doodah does and why the Widget needs to talk to it, what the contract (informal or formal) between the two is.

If the Doodah works with hexagonal Thingamies then explain that. If there aren't any Thingamies yet but it's possible they may be added give the guidance on where they'll be added.

A good piece of software design is a vision, a pure and beautiful concept in the mind of its creator. What gets written on paper has to share that vision with others, so that they can understand it, and share it going forwards.

Then you have design documentation that makes sense, that outlives the initial implementation, that's useful to people in years to come.

It's simple, make sure that each requirement that is stated in your design document meets the testablility test. If you can't think of a simple way to test the requirement, than it isn't properly defined.

Along with this simple idea, the person who is specifying the system has to be willing to put in the time to make sure all requirements are testable. You also have to have a good programming manager, one who will make sure each new requirement is checked for testability and that all changes are checked to make sure they don't mess things up.

With this combination of factors, I was able to reduce the number of errors discovered after release in one system from over 400 (taking 4 months to fix) to 4, which took three days to fix.

Make note of the tests you envisage for each requirement. Ideally, this should be done by a very sharp-eyed QA Analyst.

Finally, build code reviews into your schedule. That way you have a good chance of meeting your deadlines. The code reviews not only find many bugs, they are also good places for mentoring members of the programming team who have less expertise.

First trick: Empathy. You have to understand your target audience and write to inform them and fulfill their needs. Don't try to appease every audience with the same document--engineers need to see a much different document than marketing.

Second trick: Brevity. Put NOTHING in there unless it communicates the design and is required by the target audience. Get rid of Cut & Paste boilerplate just as you would in your code.

I guess finally I'd have to say --be complete. If you find yourself saying "We'll work out the details of that later", it's going to be one of the more difficult parts of your project.

The problem is that these standards are difficult to quantify, so the most important point would be Hire a good architect. Look at design docs they have written and see how many questions you might have if you were implementing that project. Let them train the rest of your staff.

The difference between a typical programmer and a good architect is about the same difference as that between a house painter and a classic artist. Even if a group of house painters could paint the Sistine Chapel, they would have to have some pretty good instructions to follow--and they would be completely incapable of making those instructions themselves.

I write seperate functional specifications. If the implementation requires new machines and installation by our operations department, then I might do a presentation showing why we need it, and how to hook it up (for most things this is pretty much standard boilerplate) - and perhaps how to pay for it in an appendix to the spec.

The software design documentation is in the code in the form of a detailed comment at the begining of the main code module explaining my design choices for each piece. I like to use languages, such as Perl, that allow me to format and extract documentation directly from my source code. I also like it to be inside of the code so I can make changes to my documentation at the same time that I make changes to the code. This is much more efficient for me - things don't get lost in the shuffle.

Going forward my goal is to generate XML documentation for not only the design document, but also the user manual, and other documentation from the same source code. I'll have a makefile sitting in my revision control archive that will generate all of my documentation for all apps in one command. BAM! I'm done, and can then read and edit my docs at my leisure.

The hard part is getting everyone else who touches my code to follow the same procedures.

Reading through some of the comments it seems that other engineers and those that havn't progressed to that level, don't indicate who should be reading the design documents. The audience for design documents are your collegues. Before you begin your document you should start by reading theirs. A good PHB should force you to.

A good design document should describe what you want to build, what you need to change in the existing system and what your dependancies are to name a few. It should spell out how you want to order you classes (UML!), most importantly how you will use the global classes (like an user class).

Therefore, if any one of your colleagues can read your document and think one of the following thoughts your document was a success:

"Oh Joe wants to use the user class like this, I guess I won't change it to stop that from happening"

"I don't think Joe knows that I'm building a Foo class since he thinks he's going to build one, I'll email him and we'll decide who's going to do it"

"I see that Joe has this great system designed, but he seems to be forgetting the requirement that we have to let the end-users configure it through the app and he's only using property files. I'll email him and make sure he didn't forget that."

"Uh... what the hell is Joe thinking, there's an existing class that does exactly what he wants to do, why did he waste his time writing this document?"

Assuming that you have separate documents that include the
requirements that you've agreed with the users/client, the system test
battery, how the system will be operated/managed/installed, you should
include the following in your high level design document (hint:
high-level desgin documents include pictures but not pseudocode)

Not all of these things will be appropriate for all systems.
This is not a table of contents!

Purpose & Context

why are we doing this?

helps low-level designers (who read the high-level design) to
make the inevitable trade-offs in the right way, and also
helps them to recognise a gap when they see one

how this fits in with everything else

scope (that is, what are we not trying to do?)

References

list your 'upstream' input documents

sources of extra information

ways to arbitrate conflicting statements or to help drive the
process of feeding back reports of defects in this document

what is the baseline? It's important to work from a consistent
set of documents:Know what you are trying to build; otherwise you
won't know when you haven't managed to build it

Assumptions

Check these! Often the users needs to agree to these (but make sure they actually understand the implications!)

System Overview

The Big Picture

what are the interfaces?

what smaller components make up the overall system?

within the components, what are the layers? e.g.

presentation layer

application-specifc ("business") logic

data access, protocol handling, etc.

migration to/from (this verstion of) the system

Explain the end-to-end processing of requests (or whatever)

Main body

pehaps one section per interaction
or per group of web forms,
or per batch process
or per... whatever is appropriate for the things your system does...

include mapping to significant parts of other documents; for example, if you have a database, show how each thing described in your functional spec is stored in the database.

Error handling, exception processing, resilience issues

user input validaiton

validation of other inputs

error conditions; how do we distinguish

data/business errors - the functional spec demands that these
errors must occur when the system gets this input

technical errors - might be fixed by trying them again later

handling environment problems

database went away

pneumatic drill through network cable

no response from remote system

cluster node failure

out of disk space

server obliterated by thunderbolt

...

But not an error catalogue (since you will not yet know the full
list of error conditions; don't try to put one in here otherwise when
the code is written people will represent a new kind of error as some weird variant of an old one because the old one is in the error catalogue and the catalogue is carved in stone - bad idea!)

likely failure modes - and how the design prevents things getting
worse in each case

trust boundary (which things do we trust, what information are we trying to keep safe?)

authentication

authorisation

data integrity

Compliance matrix

Shows how the design meets each agreed functional (or non-functional) requirement; cross-references the functional spec to each section of the design document

Handy when you change the design document - it shows you which functional spec requirement might be affected and hence which
of the tests will need to be redone (you do have a test battery, don't you?)

All of this talk seems to assume that there's at least some excess manpower to be thrown at the problem, that the customer knows what they need at the time of specification, etc. It doesn't allow the softare to become what it was meant to be, but only what the potentially narrow mind of one designer allows for early in the process.

I work in control systems. We have in-house motion control software that we use to do weird jobs. This software is not as featureful or complete as the packaged solutions, but it is cheap to deploy, and designed to be expandable and flexible. A project might entail anywhere from a few hundred to a few thousand lines of custom code that sits on top of the 30-40klocs of core code.

When a customer requests a CNC retrofit of an old or weird machine, they seldom have any idea how the machinist on the floor actually uses the equipment. They see the problem as a basic one--the machine takes too long at X or the machine lacks capability Y. The machinist is not really a programmer and isn't mentally prepared to make the leap to writing a specification.

The solution to the problem was for me (a programmer) to go into work with the machinist for a few days when the software was about half done and help him make the parts he needed to. Even though the software was incomplete, I could at least step around any immediate problem by throwing a little bit of prototype code at the problem and get the operator's feedback regarding the process as a whole and how it could be changed to make his life easier.

I guess the point I'm making isn't fully formed, but all this talk about design processes seems to be missing the essence of that project, which was only possible by allowing the sole user of the software and the programmer to spend time collaborating. For much of the coding, I brought my workstation to the factory floor and worked right there. It made testing much easier and let me ask the operator questions whenever an issue arose that hadn't been forseen by my boss or his when they planned out this task and designed the system. I would, of course, run major changes by my boss first, but he pretty much trusted what the operator and I were figuring out.

We've been providing custom software to this company for several years. It's pretty much run without issues and has saved them huge amounts of money, essentially by optimizing a handful of machines in there shop. The project would never have gotten off of the ground if we'd been caught up in these sorts of formal processes. We don't have the manpower, or the timeframe.