Fog Creek Software
Discussion Board

Fundemental Laws of Software Development Part 1

I have been working in the industry for several years. And over the years I have been exposed to the good and the bad in the industry as a software engineer. As doer, a listener, and an observer, these are some of the things about production I have learned. These are complementary and in some cases precursors to success in Joel's test. I throw these out for feedback:


The developer should not develop in an environment that is not same as which the customer of the product uses. This blinds the developer to the problems that a customer will see, and thus, turning the customer into a tester. If the test environment is only environment that is the same as the customer, then testing finds the problems, but it cripples the developers' ability to debug and fix these problems, which should have been caught in development to begin with.
The consequences:
  Loss of product quality
  Loss of productivity
  Loss of customer confidence

The process that creates the software product must be written, followed, and improved. Representatives of the people involved in the process must be the process owners (the process group), not the management. And all of the people involved in the process must be able to submit change to the process group. The processes that interact with product are requirements process (requirements capture procedures, requirements tracking procedures, etc), development process (Coding standards, designing, modeling, version control, etc.), build and release process (build procedure, release procedure, etc.), and testing (test procedure, reporting procedure, etc.). The inability to do this leads to chaotic processes in which one release of a product does not happen in the same manner as the last. This alone does not guarantee failure, however it does reduce the probability of repeated successes. Additionally, it increases training time, since another individual must pass down the information, which may or may not be communicated correctly.
The consequences:
  Loss of product quality
  Loss of productivity

The process when first established is not always ideal. Making goals for the process gives a direction in which to proceed. Without this, processes will stagnate. Software process fall into this problem due to the fact that, the majority of the tools, which process the product, are people, not machines. Also, every software version is a one off, not a repeat of the same product version over and over again, as in manufacturing of hardware. This make software processes harder to control and to implement. From these difficulties, processes may be thought of a extras resulting in process improvement usually being put off or forgotten. In the software industry, the processes that can be automated should be in order to capture productivity gains and process repeatability.
The consequences:
  Loss of gain product quality and productivity

Making major process changes prior to a major release invalidates the process's repeatable success, since this version of the process has not been followed once before. This introduces Risks, which can negatively impact the product or the ability to produce the product. Only small incremental changes should be made at this point after verifying the change works in a test case environment.
The consequences:
  Loss of product quality
  Loss of productivity
  Inability to meet schedule

Software production is not an assembly line, and assembly line tactics do not produce quality products. Groups based on function communicate poorly and tend to treat the function as the product, not the actual product. This also leads to people who either lose touch with the product, or people who lose touch with the big picture. In the first case, the loss of actual ground level technical experience can lead to uninformed or unreal decisions. In the latter case, people without the big picture may not process in the correct direction for the project. These are problems of communication due to a lack of common perspective or experience with that perspective. Teams should consist of a spectrum of people, and people should fulfill more than one responsibility.
The consequences:
  Loss of product quality
  Loss of productivity

Ill constructed base products will cause defects in the product, unless addition work is done to mitigate the base product's failings. Poorly documented base products will cause excessive time spent to understand and use the base product. Poorly supported base products will require addition time to overcome problems, when the problems occur.
The consequences:
  Loss of product quality
  Loss of productivity
  Inability to meet schedule

David Hickerson
Friday, December 6, 2002

#1 is wrong.  You should TEST in the environment that you expect your users to have. 

Developers have different needs than users.  We run tools and programs that users never see.  I was once on a project where management insisted that user and developer machines be the same.  The result?  Compiling took half an hour or more - every time we needed to compile.  It could have been a fraction of that.  Naturally, the project was not on time.

Bruce Perry
Friday, December 6, 2002

>#1 is wrong.  You should TEST in the environment [...]
>I was once on a project where management insisted that
>user and developer machines be the same.  The result? 

I thought the original poster meant to develop in the same environment platform wise, not to say, yeah our low-end customers are still on PII133s, lets develop on PIIs. That would be insane.

However, if your client has say an Oracle database on their top-of-the-line Sun Enterprise box where you  deploy your appliacation, you better not develop it using your Oracle NT box you have sitting at the office, or be prepared for a huge surprise the day you are migrating the application.

It is not uncommon that cross-plattform products display different behaviours and have a different set of bugs on different plattforms.

So I think this is a valid point.

Friday, December 6, 2002

Hmmm. Yes, agree with these points.

Are you writing a book? Its really not easy to read. Can you not express these principles in simpler language?

I forget the article, but there's one here somewhere about writing such that people find things easy to read. "Painless specs" or something. Joel, where has this gone? It was the one about sending people on a creative writing course.

Friday, December 6, 2002

Don't think #1 is inherently wrong. Sometimes you need to build a machine with specs different from your normal development machine in order to catch some pesky bug that some clients are having (ie, strange hardware related problems).

This is particularly true in the graphics industry (games, etc)

Leonardo Herrera
Saturday, December 7, 2002

Regarding number 1,  "DEVELOP IN THE ENVIRONMENT THAT THE PRODUCT DELIVERS" an obvious exception is embedded system software.  If my software resides in a digital camera, then must I develop my software using a digital camera?  Is there a GNU C++ compiler that runs on a Fuji A303?

Sorry, I just did a nit-pick.

The rule is a good rule-of-thumb, but not an absolute.  I've done development that targeted three different operating systems equally: VAX VMS, Unix, and WIN NT.  I always prefered to do the development on the fastest comptuter I had available.

A. Coward
Monday, December 9, 2002

I was going to mention that I do all my communications satellite software development in a vacuum while being bombarded by solar radiation, but somebody already beat me to the "embedded systems" challenge to rule #1.

But in general, I believe that if the target environment is poor in development tools, then you try to work in a tool-rich environment but test in the target environment.

Monday, December 9, 2002


That idea is a pipe dream. I guess several years in the business haven't taught you enough.


This is not very practical when a product is delivered on multiple platforms. What if your product is sold on Windows, Unix, Linux, OpenVMS, etc.? What if it's imbedded? Which one should be the development environment then?

The development process then involves the extra step of  porting and the testing process also becomes far more complex.

Monday, December 9, 2002

Of course fundamental rules are a pipe dream, but that doesn't mean that debating these rules won't make us better software developers ;-)

That being said, here's a conjecture:

In crafting any collection of rules, the interaction between the rules is more important than the identity of the individual rules.

I believe some types of rules are axiomatic: they are neither true nor untrue, but once you choose to adopt them as truths, certain other rules follow as logical consequences.

So a determined reader could probably attack any of these rules. But what interest me is, do these rules taken together describe a productive software development philosophy?

Reginald Braithwaite-Lee

Reginald Braithwaite-Lee
Monday, December 9, 2002

Actual I pick up 1. at MuSE Technologies from the chief architect, we delievered a 3d visualization API on 4 platforms (SGI, Sun, Windows, HPUX, and working on Linux when management sold us out for the money) with a unified code architecture (OS dependencies were in abstraction layer, otherwise the same application code compiled and ran on all platforms without a change). Though I agree it was stated from an API developer prespective.  The message is basically correct, but it needs help. It needs to imply that easy and quick access to the environment is necessary for maximum productivity. I know if I have to wait long time to get my code on a test platform; I sometimes lose the flow and have to spend time trying get back again.

Additionally, it needs to mean that developing in a simulated environment all the time and throwing the code over the wall to the testers is a bad thing. You need to get on platform configure it, run it and test it yourself before the testers do.

The reason I am raising these issues, is that I am changing positions and I am summerizing what I have learned that I can actually use from the past 2 years. My company is trying to start a process improvement system to satisfy a certification. It has been my experience, that top down support is imperative, but implementation needs to be at the ground level. If the process does not touch the product, it is window dressing and means nothing. I plan on helping to get the process going in right direction without it becoming a nightmare of paperwork and procedures that mean nothing.

David Hickerson
Tuesday, December 10, 2002


"...It has been my experience, that top down support is imperative, but implementation needs to be at the ground level. If the process does not touch the product, it is window dressing and means nothing. ..."

I certainly concur with this. I have seen companies completely fail in both these directions.

One place I was that deservedly died in the dotcom bust (though not before making the top-most corporate folks with all the options who bailed many tens of millions) started beating their breasts about some new xyz process (not real name of course) that was going to do wonderful things for bidding, project management, and development. It was nothing more (as we discovered much, much later) than a re-hashed (and relabeled) FDD.

Not that there's anything wrong with FDD, but the folks 'on high' were burning dollars flying all around the country/world to the various corporate offices peddling their collection of power point slides claiming to have discovered the holy grail. OK, fine, about that time I remember having had to sit with a PM who was trying to figure out how the f**k to shoe-horn his next project into this mysterious, poorly-defined magic process. **Everything** below the 'slide-people', including our core time tracking and billing sytems, were organized to support the 'old' way of doing things. A lone PM cannot on his own change all that crap from the bottom up in a company spanning oceans.

People at the top implementing change, completely failing to coordinate with those at the bottom. Once again.

In the Army, when we rolled out new doctrine or new weapon systems, though not perfect, there was a coherent fielding plan for doing it, because if it didn't have any good effects on how our troops did in the field, it was at best useless. Geeze! How could those super-expensive people be so damned stupid. Well, that bunch went away in the end.

A different organization I was in, actually several, screwed up in the other direction--the top preserved the malfunctioning status-quo, and squashed any needed improvements.

We found that unless top management had a massive stroke or something and just went away, there wasn't going to be any way in hell anybody was going to do anything to disturb the status quo (i.e. make any improvements at all). It first became clear to me when I was a company commander in the late 80s, that there's a critical threshold in any hierarchical organization such that  below that threshold, a leader can screw up an otherwise good 'command environment' as we used to call it, but just didn't have the juice to make a bad one good, if the cause of the environment being bad was from somebody above the threshold. (this directly echoes your point about senior level support being essential--absolutely so).

If you were above the magic threshold, then you had the juice to make the command environment a good one if it was bad when you took over.  My personal observations were that the threshold was about at battalion command level -- the other three companies and mine collectively made up a 750 soldier battalion. Where the battalion commander had things screwed up, we just didn't have the authority or resources to fix them; so yes, get senior mgmt support. Or, perhaps first, try to see if that's even possible.

If you have some psychotic idiot running the show, or very high up in things, you're probably screwed.

Another thing that will toast your efforts out of the gate is a critical mass of people who've been so beaten down by living through repeated efforts by various souls (invariably no longer there anymore) to actually make some improvements, that they simply won't budge on the way things have been getting done. So long as they get paid, can keep their heads down in their respective foxholes, and stay anonymous, they'll keep right on doing exactly what they're doing (classic 'Office Space' syndrome--but for real), no matter how useless it is, or how economically screwed up the company's getting.

At one such organization, I took over as Director over 5 departments: QA, Lab Services, Deployment Engr & Homologation, CM, Documentation. Right at the start, I talked about things with my managers, found out what they thought needed fixing, gave them my points of view on things I thought we should do with our sister directorate that owned all the developers to fix problems we were having.

A very telling comment came back when I looked into my manager's skeptical faces and pressed them for what was on their minds: "Well," they said, " we really like what you say needs to be fixed; we agree, and have thought so ourselves." "OK, go on." I responded. "But," they continued, " we've heard all that before from the previous guys who had your job and they're all gone now." That was red flag number 1.

I told them that I had not taken the job to not make a difference for the better in the organization; that had been my mandate from the VP who hired me. I lasted about four months before both the VP who hired me and I were both "in a 'manpower reduction'". So the poor folks who ended up staying had to continue under the same totally f**ked process they'd been barely scraping by with for several years, while the company stock (and their options) went into the toilet (company was not in the web-space, and this was well before the peak of the internet boom, so for the company stock to be toilet-bound then really said something).

So absolutely yes. The head-honchos have got to support the changes, and the troops have to be ready and willing to change too. Otherwise, either leave, or just get yourself comfortable in your foxhole, keep your head down, punch your clock, and collect your check, because your organization is bent on suicide and you won't be able to help it.

It's not cynicism, it's experience.

Best of luck to you; you've chosen a very laudable objective.

Tuesday, December 10, 2002

*  Recent Topics

*  Fog Creek Home