Fog Creek Software
Discussion Board




How to do design?

I can write code like there's no tomorrow. I have no problem writing code in any language (if I don't know it, I can learn it).

But I can't go to the next level: design. I can write the code, but deciding what to write in a manner that results in code that makes sense, that's easy to read, that is not horribly bug-prone.

My usual strategy was to start banging out code. I usually think about it some, write a bit, diagram a bit, but if I need another feature, it's like pounding a square peg into the round hole, and I wind up with ugly, buggy code.

The most recent try, I tried writing a spec. I'm still working on it, but it brought up an interesting question: How do you decide how to organize things? What classes do I use? What functions should I have in a given class?

I believe that the organizational problem is the fundamental problem of programming. It's one I have not a clue as how to solve in even the most trivial program, and I can't find any documentation on it. What do you guys do? What works for your programs?

Mike Swieton
Sunday, October 28, 2001

You've probably already read a bunch of the stuff about writing specs on JoelOnSoftware.  Thats a good start.  Its funny how little programmers are taught to write while getting a CS degree.  In fact at probably almost all the colleges that offer degrees in Computer Science you could probably get away with not taking a single writing class.

I'm a horrible writer, but you just have to force yourself to do it - over and over and over again.  My brother teaches philosophy at Harvard and when he started out in college, he admittedly couldn't write very well either.  He was a big math and science geek, but writing philosophy papers isn't creative at all.  Now he writes ALL THE TIME, and he has to be really good at it.  I told him once that I wished I could write well, and he shook his head.  Apparently a lot of his students say that to them, and he says they are missing the whole point.  Its just a process and the more you practice the process, the better you get.  Its the same for writing specs.  Its not an art (like writing fiction) and you don't need to be "creative" or "inventive".  Its just as deterministic as writing code.  Read his Guidelines on Writing a Philosophy paper... they are very relevant to spec writing I think..
http://www.people.fas.harvard.edu/~jpryor/general/writing.html

The other thing that is so important for writing is to document everything (not necessarily in your code, but if you have to do something like a ten step process where you move your ftp site from one machine to another, make a web page on how to do it while you are doing it). 

Joel had to add an email address to one of our FogBUGZ customer databases and it entails like a 4 step process because we need it in a couple of places and I watched him do it the other day. The first thing he did was open up a file on his disk that had all the steps written out.  I was thinking, if something happened to him, and I had to get some information, it would be so easy.  He has a culture of documenting everything and its amazing how easy it makes it for you to do a task that he's already done.  He doesn't have to say "um, i'm the only one who can do it because you need to telnet to port 384 on machine A and then passive ftp back to port 33 on machine B".  If its too hard to remember, then write it down.

You can tell when software companies don't have a culture of documentation.  Just look at their help file.  If you choose help on a dialog and the help screen tells you that the name field is where you type your name, then they don't know how to document.  We've been trying to set up these DELL servers for the past week and they come with all this software that monitors all the hardware and has all these bells and whistles.  Joel just wanted the server to email him when the disk failed (the RAID array knows -  or is supposed to know - when this happens).  Impossible.  The documentation was written by an idiot who knew absolutely nothing about what a user would need from the documentation.  It was full of the "type your name in the name field" documents, and nothing, absolutely nothing on anything task related.  For instance, you couldn't look up something like "How do I do x?" or "What happens if my RAID disk fails?".  You need to write with your reader in mind, not yourself.  You already know how the flux capacitor works, but your reader if they are looking at the docs, might not even know what a capacitor is...  They are going to come to your documents with questions on how to get things done, not how to fill out your form...

Michael Pryor
Sunday, October 28, 2001

It's worth noting, I think, that in large organizations design, coding, and documentation are activities performed by three separate groups of people. While many of us do move back and forth from one role to another, there are in fact many specialist skills to learn.

As far as design goes, though, there are a number of strategies out there. Worth reading on the subject, I think, is Steve McConnell's CODE COMPLETE: A PRACTICAL HANDBOOK OF SOFTWARE CONSTRUCTION. He discusses the design of small routines and their assembly and testing, rather than architectural design. That may well mean that he misses the part of the story you need.

Despite the variety of design methodologies you'll find out there, I personally don't think we've got anything much better than traditional top-down design: start with what the program as a whole needs to do, and break that down into half a dozen functions. Break each of those down into smaller pieces. Iterate until you've got things small enough to code, then stick them together.

No doubt someone will be along presently to accuse me of being naive and out of touch.

Mike Gunderloy
Monday, October 29, 2001

Thank you very much for the input, and it helps. The thing is, you didn't quite nail the same nail I'm looking at, which is a very ethereal and difficult to explain nail. I am writing a spec right now, which is what brought up the issue. Designing what it will DO is easy, took me an hour to write out in clear terms, but I did it.



But now I started documenting the implementation and APIs, and it's like, how do I know what functions I'll need before I need them? That has been my constant problem: I anticipate a certain subset, but that's never complete: The missed ones that i only see later, because they are needed in some helper function to some algorithm, far down the chain like that, and maybe that requires access to a certain data, and at that point i wind up adding some boolean flag or some integer variable to some class, even though it isn't in spec.



So there's where my personal problem is, and I'm assuming its fairly common. What it comes down to is this: I can accept that I need a good spec, but how do I write a good spec without knowing what I'll need exactly? And how do I know what I'll need exactly, without first writing the code and seeing what happens?



Sorry that didn't come out very clear at all in the original post, but if it were that simple of a concept, I probably wouldnt be asking about it :)

Mike Swieton
Monday, October 29, 2001

I think your problems are only part design-related, but also construction-related. There's a long way to go from being capable of programming the computer to do anything you want, to actually constructing solid code. I'm going to throw in a few completely unproven claims, use my advice at your own peril.

It's healthy to be sceptical towards adding flags to a class, but there's no reason why your whole design should be considered broken because you need to add a flag. The key to survival is documentation (and sensible naming of the flag).

A key to avoid buggy code is to design minimalistically. Begin a new class by assuming it knows nothing about the rest of the system. Minimize the public methods as much as possible (and document them especially well). Limit formal parameters as well. Don't pass a class if the function only needs an integer. As you add responsibilities, ask yourself if the class knows what it needs. Be wary of adding dependencies. Not only does it make your class easier to use (and reuse, for whatever that might be worth), it also reduces the potential scope of bugs. And of course, whenever you're about to assume something, put in an Assert statement first (assuming your language has one).

Also, if you find it hard to add a new feature without introducing bugs, it sounds like your functions are too large, and need to be split up. Someone invented a "complexity factor" determined by the number and types of control flow keywords in your function, and said that if it exceeds a certain number, your function is too complex. I usually design each function to do just one "thing." That usually keeps its size under one page (choose your own page size). It's a great practice to separate the "get" part of a "set" method, if the "get" part is more than one line of code.

And then of course, there are design patterns. Some developers claim to be immensely inspired by pattern books when doing their software design. You may find them useful, too.

And don't aim for perfection and expect to hit. In real life, it's about shipping working software on time, not about being proud of your elegant code, and although these acts are not completely contradictory, you'll probably find yourself sacrificing elegance for getting things done on a daily basis.

Johannes Bjerregaard
Monday, October 29, 2001

From my experience it is very difficult to design your project in detail using one approach : top-down or bottom-up and then go forward to pure implementation. It has to be an iterative process : designing->coding->redesigning the problem modules->back to implementing.. It is better if this up-down process is practised using the feature driven approach of completing the project , ie , implementing each feature set one-by-one completely else it will turn chaotic.

shailesh kumar
Monday, October 29, 2001

The analogy with writing can be useful.  It is for me because systems, coding have always had a grammar about them, a syntactical approach.

A familiar blockage for those that write is not knowing where to start, the breadth the depth of what they want to achieve is too amorphous, starting is the hardest point for them.  Often as a result they will jerk into writing, each sentence painful. Perhaps at some point they write more easily they have a better idea of what they are writing, sometimes though they are just following their nose and eventually they hit a different brick wall.

In other words they begin at the beginning and go on to the end.  A surprising amount of software seems to get written this way, for all the UML and documentation that ornaments it.

So what is the alternative?  There are a number.  The top down method is to divide the logical cake into different slices and then progressively subdivide into smaller segments making each division communicate with the containing segment and perhaps its siblings.

Or you can use the lifecycle.  Take an instance of whatever data object you are playing with and follow it from birth, through all the transformations, the intermediate chrysalis stages, until it becomes the adult butterfly, caddis or frog and finally dies (or perhaps it dies somewhere along the way).  Each individual stage of the lifecycle has its own environment and its own requirements, indeed at any particular stage you can use top down to design that state.

Whatever process you use to break down the whole into a structure that you can communicate to someone else (and this is a critical test of any design), you will discover concordances, similarities of process, the management of strings, a particular transformation of data, the selection of data, whatever it may be.  These will be needed at any level and can be probably be abstracted from the rest of the design, made available in some kind of utility library or support services. 
From what you said originally about finding that you write code easily but often find that you need to jam functionality into something that wasn't designed to do it that way, I'd advise this.  Throw whatever the original function, class, was away and rethink it together with all the new functionality it needs to have.  If that means the rest of the structure is wrecked (and the investment in that structure is such that you can't afford to have that change ripple outward), then build interfaces to make this new class or function look the same to those functions that need it so, and look new for those that need it to look new.  The circulatory system of the body knows nothing of what the pancreas does only that it handles sugar. If the pancreas needs the help of extra insulin that doesn't affect how the blood flows.
In the end the actual method of analysis and synthesis used doesn't much matter so long as all those involved understand the same one and that it documents the process and the reasons why this or that design decision was made (hence the importance of software such as FogBugz and similar).

Simon Lucy
Monday, October 29, 2001

I think I have the same issues as Mike (who initiated this thread) but before I go on, I would like to check what his problem is. In this thread I see people writing about two problems:
Problem 1: How do I write a functional specification of a program?
Problem 2: How do I design (organize) the software I am building when I am implementing a specification?
When I read Mike's initial posting, I thought that his problem was Problem 2, but reactions to his posting addressed also Problem 1. Let's decide what we are talking about or we'll have a very broad, unclear and very non-conclusive thread here since both problems are complicated enough when adressed seperately (and in my humble opinion only loosely coupled).

Jan Marten Visser
Monday, October 29, 2001

In reply to Jan's message, I'd like to clarify: The question is: how do you get from knowing what you want the piece of software to do, to knowing what code to do it with? The spec I see as a tool to get from A to B, and as documentation, but that wasn't what I was asking about.

Mike Swieton
Monday, October 29, 2001

> The question is: how do you get from knowing what you want the piece of software to do, to knowing what code to do it with?

That's a non-deterministic problem (there are many, uncountably many, possible solutions): which is why you use a human rather than an automated programmer to do this.

There may be three 'levels' to the problem: 1) design follows function, e.g. if the functional spec. says that the software is dealing with driver's license records, then there's probably some kind of 'driver's license record' software object in the eventual software design; 2) software architecture, e.g. what you might see if you remove the the body of every class or subroutine and are left with only their names ... how do these related, how do they interact; 3) low-level (near code-level) design, rules of thumb like 'minimize tramp parameters into a subroutine', 'minimize a class' public interface'. etc.

I have three general suggestions:

Re. 2) and 3), read books which describe architecture (aka design patterns), and low-level design (aka construction techniques). Approximately 10 of the books that I listed in the "Good resources for learning programming?" thread earlier on this board are in this category. I consider them as useful to a designer as a catalog of recipes might be to a chef. Historically, I learned: C (including Code Complete); Thinking in C++ (ie syntax); Effective C++ (ie mistakes to avoid); then when I read Design Patterns, my reaction was: "So this is why I learned C++: to have a language that supports design techniques like these!" Code Complete is perhaps my all-time favourite, though it discusses low-level design ... I'd had years of practical experience before reading it, but reading it was the first time I had seen rules laid out explicitly, so that you may have an understood justification for your feeling that one design is better than another. It's funny, too.

Re. the fact that design is, in my experience, iterative or spiral instead of waterfall, read the Refactoring book.

Thirdly, there's no substitute for practice, and feedback.

Christopher Wells
Monday, October 29, 2001

I find it useful to think outside-in. This can be done at 2 levels.

At the architectural level, after you have decomposed the system into broad modules, you could use the "Facade" design pattern to implement the module. By trying to define a single interface - the Facade - that hides the complexity of the module from it's clients, you can toy with providing the simplest, cleanest  interface to the module's clients. It really helps in refining the high level contract of the module.

At the class level, it helps to think "Test First" (from Xtreme Programming). The effort of writing a test that exercises the main elements of a class, even before implementing the class, clears a lot of issues.

Also see the excellent sections on "Reasons for creating a function" and "Information Hiding" in Code Complete, by Steve McConnell. He has another article on the topic at http://www.construx.com/stevemcc/ieeesoftware/bp02.htm.

You could also explore xtreme programming. Bill Wake's notes are quite insightful. At http://users.vnet.net/wwake/xp/.

Warning: Some XP practices seem to go a little overboard with the xtremeness, but behind the hype there is some real value in its key ideas.

Yogi
Tuesday, October 30, 2001

Most useful to me is wondering what is the crucial meaning of something.  For example, when dealing with XML, people get tripped up on the hype.  But it's just a tree structure.  So it's important to get away from dealing with XML's syntax, and abstracting it away until you're just dealing with trees.  (Or whatever subset of trees you want to deal with.)

Maybe it's best not to "bang out code."  Instead, sit down with a pencil and paper, or use a whiteboard.  Flesh out your ideas on a conceptual level.  Erase when you realize there's a better conception.

From such a high-level perspective, you'll see the things that modules will need for intercommunication.  And eventually you'll have a feel for what useful things can be done for optimization (which should come later), such as adding a cache in the right place.

I too like Code Complete.  It even has little debates on what the proper amount of commenting should be, and if comments are good at all.  If I were to start a company, I would have a learning period, say a month, and during that time, they would get a copy of this book to own.

forgotten gentleman
Tuesday, October 30, 2001

I have a question regarding the next step in the process: actually writing the technical specification for a piece of software.  What process do people normally use?  We've got a reasonable functional spec, but now that I've drawn the short straw, how do I go about turning it into something that my co-workers can use to crank out code?  Any good resources I should consult?

jw
Tuesday, October 30, 2001

I'm a big fan of Jesse Liberty's books and "Beginning Object-Oriented Analysis and Design" would be a good read for you now. From the "How this book is Different" -
"I sometimes fall asleep reading methodology books...This book aims to be an antidote to MEGO [My Eyes Glaze Over]...does offer is a working programmer's guide to building commercial software"

I particularly like the way it includes issues like persistence, concurrency and has a full real-world example.

We have our own software process which is evolving, the heart of which is recording design decisions (see one-page summary at <http://www.oofile.com.au/adsother/sse.html>) because I found for myself and many people I employed or worked with, making choices and justifying them to yourself and your peers is not only difficult but badly supported in most software processes.

Finally, when stuck for how to design something:
1) imagine a GUI
2) imagine you are explaining it to your mother/uncle/postman, whatever and then sitting back and watching them use it.

What information do they need to see or enter at the same time?

What behaviours must the program provide (react to mouse, validate data entry, print a list...)?

3) now imagine you are trying to sell them an upgrade - what did you need to add or (more importantly) what annoying behaviours did you need to change?


I know it sounds funny, but very often we have an idea of how the outside of the program behaves, just not how to factor the implementation. Considering what things a user of the program needs to have happen can really help a model of how it is behaving.

Thinking about what you could sell someone as an upgrade, and how it would change the existing software, will help you identify where to partition the design - what's going to need to be flexible for change?

Andy Dent
Tuesday, October 30, 2001

"I have a question regarding the next step in the process: actually writing the technical specification for a piece of software. What process do people normally use?"

I could write a book answering this simple two-line question!  Fortunately Steve McConnell has already written three or four books already on this subject so I'd suggest, if you haven't done so already, read his books, primarily _Rapid Development_.  I can't suggest a "process", yeah processes are good and all but they're like the Tao, the more you try to codify what it is, the more you realize the inadequacy of trying to boil an artistic endeavor into a cookbook solution.

Here are some tips though:

First of all, technical writing is just another sort of programming.  The language used is your plain old standard English, and the target platform is wetware instead of hardware, but the task is the same: you're trying to instruct a (mostly) ignorant agent to do some complicated task.

Fortunately humans are a bit more lenient in the processing instructions they will accept.  In general one statement in English in the technical spec translates out to maybe ten to twenty lines of code.  So expect for technical specs will be an order or two magnitude less in size than the actual code.
Aim for that level of detail.  If your English spec is actually LONGER than the code it describes, you're going too far; you're not writing a spec, you're programming in COBOL.  If on the other hand each line of your spec covers several hundred lines of code, then really your spec is just a functional requirements doc.

You almost need to be as detailed and thorough as you are programming, but still stop at about one layer of abstraction above
actual C or (for the Delphi nuts here) Delphi
code.  A technical spec is done when there is no more "and then a miracle happens" sections of the implementation plan.  You need not nail down every single parameter and return value but should have a general idea of what each module requires as input and generates as output.  If you are working with unfamiliar APIs or third-party code, it may be necessary to do some prototyping and research in order to figure out exactly how those APIs and third-party libs will be used.  What you don't want to happen is get halfway into implementation and then realize the Foo library doesn't implement, oh, say, passive FTP.

Prototypes are just a good thing in general.  If you can't prototype it, stands to reason you can't build it either.  Prototypes are the best way to keep the tech writer honest and ensure that he has an implementation plan for every feature.

Expect that some problem domains will be more difficult than others.  If you're writing an email client, there's nothing hugely sophisticated going on there, it's largely data manipulation and glorified library science.  If you're writing a natural language translation program, okay, expect to spend a year or so getting current in AI research in both academia and in the industry, and accept that your product will be experimental in a lot of ways.

Politics: like Joel said, a spec should have an author.  ONE author.  *Maybe* a small group of authors if the writers have worked well together in the past.  But resist at all costs the "design by committee" approach.  First, what happens is that everyone looks at the problem for 15 minutes and has tons of neat ideas but no clue how they would fit in the big picture and no discipline to look beyond the first-level analysis of the problem.  Ten people looking at a problem for six minutes is not equivalent to one person looking at it for an hour.  Second, it slows decision-making to a crawl.  Yes, you should always solicit input from coworkers, but if you always need group affirmation to make a decision, you'll get nothing productive done.  Gather input, make an executive decision, and then be prepared to defend and potentially modify it when someone comes questioning.  Centralized responsibility for the design is a must.

And lastly ... well, there's no substitute for good old practice, trial, error, and experience!

Alyosha`
Wednesday, October 31, 2001

Two things vastly improved my design skills.  First, early in my career I learned real object-oriented programming from the Digitalk Smalltalk/V for Macintosh tutorial.  "Programming by sending messages to objects" mapped really well to the way I thought about problems.

The best introduction now is the section on OOP in Apple's "Object-Oriented Programming and the Objective-C Language" which is online at <http://developer.apple.com/techpubs/macosx/Cocoa/ObjectiveC/index.html>.  Even if you don't read the parts on Objective-C the parts on OOP will be useful.

Once you have a better understanding of object-oriented programming, check out "Design Patterns" by Gamma, Vlissides, et al ("the Gang of Four book").  I had been using OOP daily for years when I read it and it was still an epiphany.  What's useful about it is that it provides a standard vocabulary for talking about design problems.

I've found that design is a lot like learning a musical instrument.  At first you worry a lot about low-level technique: Fingerings, notes, scales, and so on.  But once you get the technique down, you can start concentrating on music.  So learn the tools and vocabulary and practice, practice, practice.

Chris Hanson
Friday, November 02, 2001

I found three dead links in the above posting, very annoying, specially when u r interested.

jan marten visser
Friday, November 02, 2001

> I found three dead links in the above posting, very
> annoying, specially when u r interested.

You've just found some bugs in the way the discussion board handles links.

The solution is just to fix the addresses, by removing bad periods and greater-than signs.

forgotten gentleman
Saturday, November 03, 2001

*  Recent Topics

*  Fog Creek Home