Fog Creek Software
Discussion Board




Documentation

Had a talk with one our senior developers. I was asking him for the design docs of the product I was newly put into. He said there were no design docs. And that design docs was for wimps. He said that any developer who is unable to understand the code by reading thru it is not fit for the company.

He underlined his argument saying that "successful products rarely or never have any documentation - take for example MS WORD. Its just pick your feature and start punching. Programmers are in a race against time. We need to bring out the product fast." He tells me that his friends in Microsoft don't care about design docs and wasn't the company successful?

Even when I have the impulse to code right-away, I take some time in putting down in words what I want to implement. And many times I do give a brief write up on the design before plunging in.

Now this developer is one of the star and senior developers in our company and my manager consults with him often. I am not sure whether I should continue with my inclination towards writing something about a feature before coding it.

Bob
Wednesday, December 18, 2002

If I recall correctly, wasn't the first version of Word for Windows something like 18 months behind schedule when it shipped?  Can't remember where I read that, it's probably in a McConnell book somewhere ... anyways ...

Well, I think your senior dev's attitude is motivated by a very real and healthy fear -- the fear of spending inordinate amounts of time writing docs that serve no useful purpose and which will never be read by anyone.  In some groups, the writing of "design docs" before coding is a cargo-cult rain dance ceremony, causing even the most straightforward coding project to succumb to paralysis by analysis.  That said, though, my impression is that this dev has reacted by taking things to the exteme in the other direction.  I don't think this is any more of a healthy approach.

I'm amused by his "just read the code" attitude, it's the sort of he-man macho view that you'd expect of a senior developer.  But okay, sure, you're a smart programmer.  You ARE capable of reading the code and getting a rough overview of it, given enough time.  But why should you have to?  Is it really the most efficient usage of your time?  Probably not, after all, there is a lot of code, and it's probably a little sloppy, and it's hard to extract the details from the architecture sometime.  And most importantly, code only tells you WHAT the program does, it doesn't explain WHY it was done that way.  Wouldn't it be easier if he took, say, half a day and prepared a walkthough of the major features?  And if there are more new people on the project, wouldn't it make sense to write a design overview down in a document?  I think so.  But he doesn't want to do that, because, like all good programmers, he's lazy.  Why should he use a few hours of his own time to save a week of your own?

I, like you, benefit from writing my thoughts down on paper before implementing them in code.  Sometimes I can even show these scribblings to developers whose opinions I respect to get some feedback before coding.  But I try not to go overboard with the details, because often many of the details need to be reworked during coding.  (I am especially suspicious of any design methodology which demands that all the details be set in stone before coding commences).  Once the module is code complete, I'll go back and write a "design document", after the fact, explaining what the code does and why.  This document gets read if new people come on the project, or if I have been away from the project for a while.  And I try to keep this document in sync with the code, for example, after major refactoring sprees.  This practice seems to work well for me, your milage may vary.

So, to answer your question, I think you *should* continue with what you're doing, if it works well for you.  Just be careful not to sink more time into it than you'd expect to recoup elsewhere in the development cycle.  As far as your senior dev goes, I don't know if it would be fruitful to argue this with him any more -- he seems pretty stuck in his ways.  But if you make yourself a reputation as a fast coder who's easy to work with because you document everything, that's probably good enough revenge ...

Alyosha`
Wednesday, December 18, 2002

Design docs aren't the problem.

Heavyweight design docs that can't change after people have signed off are the problem.

The problem is that your senior developer is prone to hysterics.

w.h.
Wednesday, December 18, 2002

Most of the time a good deal of software work is maintenance. Thus, often no design docs are available.

Thus, a intelligent developer will often explain that we don’t have, nor is the cost of a design document required. Also, arguments can be made that the source code it self is the design document.

However, if the developer is arguing that design docs are not to be used for the creating of software, then would recommend the developer be fired on the spot.

I have no problem with a developer arguing that design doc is not worth the effort, that is fair. This is especially in the case of existing systems that have been in use for years (I am NOT saying this is good, but I can understand the argument put forth). Often, the creating of some of the design doc means additional work for the developer, and their seems to be no benefits. However, to argue that design docs are no good is just plain stupid. There are lots of hungry developers out there, and my industry does not need that type UN-professional developer working.

If you have a developer that has worked on a project every day for two years (adding new features..and doing maintenance), then it seems to be a complete waste of time to have a ER diagram of the tables used. These people will know the table and field names even when intoxicated at the annual Christmas party. Suggestions to these people to document will result in total blank stares from these people. (I had this happen on one of my over seas consults this year!!).

I have developed lots of applications over the years, and many did not have design specs, or even any documentation beyond a few scribbled notes on a table napkin. All developers have done this. Once much of the core of the project is functional then everything else is the easy stuff like reports, and minor changes to the product. It is often during the design part that thoughts and ideas need to be put on paper. Once the software is up and running, those cool scribble notes have little value.

I want to stress that you thus can get a way without having a design doc for a lot of software. However, arguing that you don’t need designs doc is complete brain dead idea. I cannot respect any developer who says that design docs have no value.

Further, if your company has no documents then when that developer gets sick, or quits, then the ramp up time for the next person can be VERY STEEP indeed. However, the next new developer that comes along can generally pick up the system in about a month or two anyway. I mean if you hire a developer once every 4 years, then that one month of cost is not so bad as compared to the developer having to take four months of time to really document the system correctly!.

However, the second you get beyond 1, or 2 developers, then the issues of lack of docs becomes VERY important. It will start to cost more. Again, we have to distinguish between a lone programmer maintain some software for a company with 100 employees.

Further, it depends on the type of coding work done. I am a consultant, and thus may not go back to work for a particular client up to a year at a time. When I go back, I will FOR SURE NEED some design docs. This is because I am not working on the software every day. Hence, the frequency of how often the code is worked on is a factor here. Thus, for each product I have a binder with the meeting notes, screen shots, ER table diagrams, and the functional specs. I also often have sample invoice forms, or what ever type of form that the software creates.  I need these docs, since I cannot remember something from a year ago!

(by the way, these binders of mine are absolute killer docs for perspective new clients!). 9 out of ten times I ask the prospect company what kind of existing software docs do you have. Most of the time they don’t have any. I pull out one, or two binders and usually get hired right on the spot. Those binders blow client after client away. It also showcases my work, and how well organized I am. I also suggest to the client that I am willing to leave a copy of the binder on-site if they hire me (for some reason, the clients never ask for those binders once they hire me). I also do bring the binder to all meetings for that company, and it certainly is used to explain ramifications of new changes. We have often pulled a page out of the binder and photo copied it for the meeting. I love those cool screen shots, and my favorite are those spider web looking ER diagrams when you have lots of tables!!

Here is two screen shots from some docs that exist in that binder. In fact, the first one is actually slipped under the plastic on the binder, and is the “cover” for that client:

http://www.attcanada.net/~kallal.msn/Articles/PickSql/Appendex1.html

http://www.attcanada.net/~kallal.msn/Articles/PickSql/Appendex2.html

Many developers don’t see the need for some docs. The whole idea of some docs is much like asking the developer to create a map on how to drive from home to work. The first few times driving to work it might be hard to find the way. After doing it every day for a year, the suggestion to document, or create a map to get to work is laughable. This is the same for a developer. On the hand, if you have a bus company, then you can bet that the investment is made into providing drivers with maps (despite some drivers on the same route for the last 10 years!). Thus, it is question of what type of software development the company is involved in. It also depends on how large the software project is.

Another real import thing is that often these developers do not realize how helpful some docs are. Good documents also allow a developer to create better software, and often again this is not realized (even by some very talented developers). Even for very small projects I now spend some time on documents because it allows one to MAKE BETTER SOFTWARE. I always feel sad that some developers have not yet learned this.

I really have a passion for good software. If you want to write first rate software, then you will realize that documents for the software improves the process. If you don’t care about good quality work, then you will not see the need for documents.

I think I just came up with a new additional question to ask perspective developers: What do you think of documentation!

Since having documents allows me to make better software, then you can bet you life that I use them. 

I want to write REALLY good software, don’t you folks?


Albert D. Kallal
Edmonton, Alberta Canada
Kallal@msn.com

Albert D. Kallal
Wednesday, December 18, 2002

ER diagrams can be pulled out of any existing database at any time.  You can dissolve that down to a conceptual model as well if you want.

The code that runs is the system, any documentation of that system patently is not.  The map is not the territory.

This is not to say that documentation is irrelevant or bad, it depends upon its purpose.  There should be a requirements document, there should be some functional specification.  Both of those are documents of intent, not documents about the running system.

The documentation of the running system is the code and the bug database.  If there isn't a bug database of some kind then there are problems of a different colour.

If there are critical decisions in the code then they should be marked in the code.

/* CAVEAT PROGRAMMER
Screw with this and you will die, don't even think about it, move on, nothing you can do here.
*/

And so on.

Simon Lucy
Thursday, December 19, 2002

Of course there are finer details that are best documented in code and for which it makes no sense to duplicate in external documentation.

But, many systems are more complex than one piece of code (=programme) or a single database.
Would you agree that not everything about the design of a system can ever be extracted from the (collective) code with similar effort as it would take to document it? In other words, to find your way around in the system quickly, you need at least some sort of a map to introduce you to the "territory"? And that you need to keep that map up-to-date? And that if keeping it up-to-date seems elaborate, you might be thinking to document at the wrong level of detail?

And of course that as with any kind of wisdom, you need to analyse the situation first, before you try to apply wisdom that might not be relevant :-)

Practical Geezer
Thursday, December 19, 2002

It is so tempting isn't it? "Just read the code". But if your senior developers have given into this self delusion for your companies mission critical stuff, you are in serious trouble.
Now I am sure that most of us here have at least gone trough one project that never got of the ground because of design documentation requirement overload. This happens, but your take home message from that should not be "docs suck, code rocks". It should be that the doc requirements should match the project type and lifecycle stage.

The docs you write are never of most benefit to yourself. It is a responsibility towards others. If the whole dev team is on a teambuilding rafting weekend and the party bus drives into a ravine, will the new hires not benefit immensely from an updated basic documentation set that describes the systems in gross detail?

Just me (Sir to you)
Thursday, December 19, 2002

You can't document intelligence and documentation can't be used in place of it.

You can treat the system as a living document and all of the surrounding documentation should be used as part of that understanding of what is required.

But in many real life systems domain critical information isn't documented within the system itself but is part of the environment,

If its an accounting system, where within it is going to be the documentation about what a 'credit' or 'debit' means and in what contexts?

The answer is that it won't be, its domain critical information to the system but it isn't documented within the system at all.  I've lost count of the number of times I've explained debits and credits (and other abstruse accounting terms) to programmers who didn't care and found it difficult to understand why it was important.

Simon Lucy
Thursday, December 19, 2002

I'm a big fan of documentation, but can easily appreciate the amount of time it can take to note down everything in intricate detail.

Basically, I ask myself the following questions :

* Will anybody else need to know how this code works?
* Will I be able to remember how this code works in six month's time?

If I answer "yes" to the first, or "no" to the second, it goes in a document.

Most design docs I write are just simple word documents with a pictorial diagram at the top and a brief description about what each bit on the diagram does. I might include one or two diagrammatic paths of execution (what UML calls "use case scenarios" iirc) but usually just write a description in English.

The point is, they don't take long to write (no more than an hour or two) and they aren't complicated to understand - the basic idea is they should be easier to read than the code (otherwise what's the point?). If you really haven't got the time, wait until the deadline's passed and do them then while the project's fresh in your head.

The last project I got involved in had no documentation, so as part of my learning curve, I wrote a design document for it. I found that by doing that I understood everything a lot better than if I'd just sat down and read the code.

By contrast, a customer is currently screaming blood at another development team because it's full of hard-to-fix bugs due to developer after developer just bolting bits on one after another from the original (poorly designed) product. If even a simple design document had been in place, everybody would have worked out what direction the product was heading in. Now, nobody has a clue.

Better than being unemployed....
Thursday, December 19, 2002

So on balance, there issue that are best documented close to where it is applied, the code, and there are issues that are best documented in terms that don't apply to code.

The difficulty then is to determine which is which.

Just a thought...
Thursday, December 19, 2002

Isn't that just the difference between a technical design document and a functional spec?

(I was talking about technical design documents in my last post).

Better than being unemployed....
Thursday, December 19, 2002

Design documents document the design, not the system as it exists, they may coincide, they certainly document the intent of the system but the system may, for one reason or another, behave differently.

Which is why the system that manages bugs is so important, its a fundamental document as to the system's behaviour.

Simon Lucy
Thursday, December 19, 2002

"Isn't that just the difference between a technical design document and a functional spec?"

To some degree I suppose.
But consider bubble sort for instance. Others might benefit greatly from a verbal explanation, maybe with a picture or two. Instead of having to rely on reverse engineering from the code.

Now, I am sure someone is going to say that good developers already understand bubble sort and recognise it when they see it. Still, you don't start off an accomplished developer, and there are other situations where the algorithm might not be well known. And as an expert, a reminder can be helpful too.

So novices and experts might benefit from some documentation over none, albeit for different reasons.
No documentation at all (even if the code is commented), or too much documentation IMHO is rarely, if not never, a good thing.

Practical Geezer
Thursday, December 19, 2002

I should probably add that in my experience no design documents usually means no design. Arguments are usually retrofited to defend why there is no documentation, and it is carefully neglected to mention that there was no design in the first place to be documented.

Too much documentation usually indicates another pathology, obsessing over process instead of achieving goals.

You milage may vary.

Practical Geezer
Thursday, December 19, 2002

Writing design documents is in the best interests of the developer.

There will eventually come a point when you want to forget about the current project and go do something different (like work on another project or go on vacation). Design documents make that process easy. Without design docs, you run the risk of getting stuck because "we couldn't manage without you, nobody else really understands the system."

To put it more simply: every detail that's not documented is one more thing that someone can call you and nag you about long after you no longer have any interest in remembering it.

Beth Linker
Thursday, December 19, 2002

Beth,

That might just be the best way to convince "non-believers". Unless they like others being dependent on them and have no interest in vacations :-)

Might give you something to think about though. What if the developer who claims to not need documentation is going on a vacation, is secretely keeping documentation to get himself back up-to-speed when he returns, but denies it to maintain your dependence?

Hmm, might just be a bit to paranoid. Still, isn't the alternative plain stupid? I wonder...

Practical Geezer
Thursday, December 19, 2002

IMO, there are two kinds of design documents:  Those written by developers to clarify code before writing the code, and documents used by developers after the code is written to communicate the intent of the code with other people.

Some people hate design documents, and don't need them to write code.  Fine.  But many others need to be able to read design documents to understand the code, and part of working with a team is accomodating the needs of the rest of the team.

Look at the Truck Factor, which says, "What is the smallest number of people in a project such that, if one of them got hit by a truck, the project would be in trouble?"  If that senior developer got hit by a truck, what would happen to the project?  Would it be in serious jeopardy?  If so, that senior developer needs to leave some record of how the system works.

So, no, design documents are not for wimps.  Arg.  I hate that sort of attitude.

FWIW, if this starts to seriously impact the development effort, consider raising your concerns with the senior developer's boss.  This is classic arrogant Cowboy Coder behavior, and it's killed many, many projects.

Brent P. Newhall
Thursday, December 19, 2002

"FWIW, if this starts to seriously impact the development effort, consider raising your concerns with the senior developer's boss.  This is classic arrogant Cowboy Coder behavior, and it's killed many, many projects."

To be fair, you have to be as concerned about the other end of the spectrum as well, and even more difficult, the "Cargo Cult"-type of documents.

Practical Geezer
Thursday, December 19, 2002

"Too much documentation usually indicates another pathology, obsessing over process instead of achieving goals."

Or it could indicate a packrat loathe to throw out anything, no matter how obsolete. (Which is my problem - eventually I move obsolete docs into a special 'important gems' folder, never to be seen again, but safely tucked away 'just in case'.)

X. J. Scott
Thursday, December 19, 2002

Too much documentation is actually an interesting question.  We seem to agree that there is such a thing, but how do you know you have arrived at the tipping point when you're writing it?  The 6 month rule is a good one, does anyone else have one?

I think one of mine, from recent experience, is to not write a document in a format that's more formal or authoritative than its contents.

We've had a two week long adventure here with a terrible design document done two months ago.  Why?  Because its format, passing itself off as a proper design, wasn't compatible with the incomplete and contradictory content.  There was nothing wrong with the content -- that's what we knew or suspected at the time, and it's certainly a good thing we wrote it down or we'd have forgotten it while working on other projects.  But, when we came back to it and some people thought we had a finished or mostly finished design doc because it *looked* right on the surface, it caused a bit of a confrontation.  And a bit of difficulty in trying to deal with the ego of the writer.  It's hard to clarify that said document is terrible and unusable as what it seemed to be, while simultaneously being necessary information as something else.

Mikayla
Thursday, December 19, 2002

Bob,

Unfortunately, your senior design guy is obnoxious and its going to suck working for/with him.

You should realize though, that this is culture you find yourself in and is unlikely to change.  Its also very likely that the team members can maintain a high level of code quality and communication without a formal design process or documentation.

You'll have to adapt to that environment.  Get used to "grep" and hopefully you have really good tag functionality built into your code editor.

FWIW,

Nat Ersoz
Thursday, December 19, 2002

Its already beginning to suck.

Bob
Thursday, December 19, 2002

"Or it could indicate a packrat loathe to throw out anything, no matter how obsolete. "

Actually, that's a different kind of too much.
I meant endless writing in too much detail on a subject, before actually starting to work on it. Like the kind of documentation that tries to cover every single detail even if feasibility isn't even determined yet, for instance.

Your kind of pathology seems to be of the benign kind :-)

Practical Geezer
Thursday, December 19, 2002

*  Recent Topics

*  Fog Creek Home