Fog Creek Software
Discussion Board

Construction sorrows...

>Well, what I've learned from my first large construction >project is that this is hogwash. The building industry >doesn't know how to do anything on schedule or on >budget, either.

I have two years experience supervising and managing building construction projects. Add to that five years in road construction. My experience is that General Contractors are very good at scheduling their projects. 

Along the way, they have to often work in the schedules of many subcontractors, as well as weather concerns, hour restrictions due to noise bylaws and traffic patterns, material delays and availability problems, material cost changes, machinery breakdowns, environmental delays, union requirements, and more.

I think if software projects had to deal with some of these complexities, the current situation would look like a dream.  I think if we are going to learn anything from the construction world, it's that a firm plan is required from the outset.

Perhaps it's best not to judge someone's expertise based on one obviously poor experience.

Thursday, September 18, 2003

Spam - point taken, but if my Project Manager for my last software project were to list what caused the schedule to be delayed (business subject matter expert schedule availability, time to order and install hardware from vendors, evaluation of new tools and resource ramp up time, legacy code integration problems) they all sound like similar complexities. The person who pays for something gets three related things: the estimate, the completed work at a point in time and the bill. When the latter two don't jive with the first, dissatisfaction will result regardless of the specific problems or excuses. I agree a plan is a valuable tool, but any type of project is subject to unexpected issues to arise.

Thursday, September 18, 2003

Absolutely right, m. I certainly don't think that software scheduling doesn't have it's own complexities. I've been a software engineer for 8 years, and it's something I struggle with daily.

My point is that in my experience contractors deal very well with unexpected delays; they plan on them occuring, and have a work around available. Obviously there are different qualities of contractors out there, and it seems that Joel didn't get what he thought he was getting.

Just wish people wouldn't dismiss an entire industry out of hand, based on one small experience.

Thursday, September 18, 2003

I haven't heard many cases of what you describe.  From every experience I've heard, things are always more expensive, take longer than necessary, etc. when you are dealing with construction.

I'd like to reference a blog on this one:

Check the Feb 16th entry.  Joel's not the only one who's had problems.

Flamebait Sr.
Thursday, September 18, 2003

The difference between road construction and software is the that in road construction you can come up with the final cost.

Sure, there is  billion variables like cost of gas, trucks, cement, pavement etc. People getting hurt, people getting sick etc. 

However, after you build a road or two, then the numbers can be crunched as to what 1 mile of paving costs. That cost of 1 mile of paving, or 2 miles or 5 miles remains reasonable constant after that. This is why so many builders can easily quote what it cost per square foot.

That final number of a cost of 1 mile of pavement might have as mentioned  zillion variables, but who cares? All we care about is the cost of 1 mile of pavement. The other thing is that we can also schedule how long this will take...barring the weather issues. Again, the previous history and experience will dictate this number.

The problem is that software HAS NONE of the above luxuries. You can’t determine the cost of building a data entry screen based on the last time you created a data entry screen! (the time will be different each time)

Scheduling of software development and determining the time required is many orders of magnitude harder then building roads or building buildings!

Further, some developers produce at 100 times the rate as the developer sitting in the next office. While some drivers might be a bit quicker driving the cement don’t as general rule see one worker outputting 100 times the work as the next person. In software this is a common occurrence.

With software, you are dealing with a product of the human mind, and things like physical constraints like how fast the truck is, or how fast a person can walk does NOT apply to software development.

Sorry, estimating of construction is a trivial problem as compared to estimate software costs.

Albert D. Kallal
Edmonton, Alberta Canada

Albert D. kallal
Thursday, September 18, 2003

Spam - I don't think that there is much correlation between building construction / road construction and the level that Joel is dealing with.  When it comes to residential construction firms and remodeling companies, project management is largely unheard of. It consists mainly of a cell phone and a calendar.

Thursday, September 18, 2003

Nick, that could well be. Doesn't that make it even more strange for someone like Joel to say that the industry knows nothing about scheduling then?

Thursday, September 18, 2003

"You can’t determine the cost of building a data entry screen based on the last time you created a data entry screen! (the time will be different each time)"

You can't determine the exact cost, but this is true of construction projects, and any other project as well.  Even something as seemingly straightforward as putting up a wall will never be exactly the same two times in a row.

What you can do is gather data for a number of screens, walls, pavement-miles, or whatever, and develop a distribution of times/costs/scope/whatever.  This distribution is a reasonably accurate predictor for the next screen, wall, or pavement-mile.  It may not, however, be very precise (the distribution may be wide). 

For precision, you have to systematically remove sources of variation from your process.  For example, use design rules to limit and control the range of possibilites for designing screens.  Use code templates to reduce variation in the code.  To the extent you can eliminate these variations, the precision of your estimator will improve.

Once you have data to allow you to estimate individual tasks, the next step is to estimate chains of tasks.  Monte Carlo simulation of the chain is one way to determine a good approximation of the distribution of the chain.  Critical chain methods are another. 

Over large numbers of samples, this method will yield very good results for any reasonably stable development process.  Manufacturing textbooks dealing with process control and quality give excellent guidance in this area.

Thursday, September 18, 2003

The thing is with software, if you want to make two identical data entry screens, the second one will take zero time. Not true for a house, office, etc. I know that it's not typically exactly the same, but if its' 80% the same then you can probably cut 50% off the estimate.

Thursday, September 18, 2003

I think the primary issue related to software that makes it seem more unstable than many other human efforts is you duplicate the exact same code and send it to possibly millions of people.

Sometimes buildings collapse, sometimes the electrical systems go haywire, etc, etc.  But when this happens it is a local effect, so you don't get a hundred million people scratching their head thinking this is a huge issue because the exact same problem is happening to everyone else.

If you could somehow duplicate and deliver the exact same house to millions of people, you'd wind up with a lot of people finding the same defect in the construction and the housing industry would seem just as "bad" as the software industry.  But since each house is different, even if you're talking about tract/pre-fab ones, the little problems you see with them that are rougly analgous to software bugs are overlooked as bad luck and part of life.

Mister Fancypants
Thursday, September 18, 2003

>What you can do is gather data for a number of screens, walls, pavement-miles, or whatever, and develop a distribution of times/costs/scope/whatever. This distribution is a reasonably accurate predictor for the next screen

This is exactly the problem. In fact, previous history it is NOT a reasonable predictor.

Why not then go and publish a book with how long it takes to do things? All of the auto mechanic shops have suggested rates and times for completing a given job.

Why then is there not a book published for software developers telling them how it will take to compete a task?

Putting up a wall will NORMALLY only vary by about 20 to 40% in time. Perhaps even less.

Building a data entry screen will in general vary by 3000%

As mentioned, once you build a few walls, then you get a average number. That same technique CAN NOT be applied to building a data entry screen. One data entry screen might be to enter a simple mailing list name. This screen will only take about 10 minutes to build.

The next data entry screen might be a air traffic control system that takes one year to develop, and the code behind the screen may cost several million dollars. We only got one screen, but it is far cry from the first screen.

And, of course, you might have a developer team fill with some developers that crank out code at 100 times the average. Or, perhaps the developer team is 100 times below the averages. These kinds wild swings don’t occur when building a wall. There are exceptions, and things can go wrong when you build a wall, but in general, the building the wall takes a very similar time as to what it took build the wall the last time.

In software, there is little relation to what the last screen built takes. You can certainly build up a set of metrics to help with the estimating process (each developer should, and must do this). However, it is no where the same as physical things. Developers might walk the roughly the same speed, but they don’t produce code at the same rate at all. 

In fact, the only reliable way to get a estimate is to let the actual developers who are creating the code to make the estimate. You can NOT make the estimate FOR the developers.

With construction, one group of people building a wall does not take much different then the other group. With construction, YOU CAN MAKE estimates for the workers since physical things are constrain by the laws of nature.  You can only move so fast.

The mind is no such creature, and does not have physical constraints.

I mean, please, point me to that book, or manual that gives a suggested list of how long it will take developers to complete given tasks.

I certainly think that such a book would have been published many years ago if such a thing was possible. Our industry has no such book.

Albert D. Kallal
Edmonton, Alberta Canada

Albert D. kallal
Thursday, September 18, 2003

It's my experience that due to high competition in that market the construction business requires a large degree of trickery, i.e. "unforeseen events" make the bills pop up everywhere. Once the software market gets on the same track (cf. outsourcing) we're going to see that same market behavior.

Johnny Bravo
Thursday, September 18, 2003

Hi Albert,

I don't disagree with you that our industry has no book like the one you describe.  I also agree with you that at this point in time, a handbook like this for the entire software industry is not tractable.  We aren't mature enough yet.  For a given team, though, I have reason to believe that an approach like the one I described earlier will work well.  Let me talk a little about why.

One thing that helps all these other industries is the existence of large numbers of high quality standards.  These standards cover everything from materials to interfaces to training.  There are entire sub-industries devoted to the creation and maintenance of these standards.  These standards are also enforced by the industry and by external entities.

The effect of this is to dramatically reduce the variation seen by practitioners in these fields.  They also actively pursue the goal of reducing variation in the things that are not covered by these standards.

I know software has some standards.  Sort of.  These aren't even in the same league as some of these other industries, however.

Another thing these folks have done is to develop robust classification systems for the work they do.  This means that each classification describes a class of very similar kinds of objects.  As you noted, no two screens are the same, so "Screen" is probably not a good classification - it admits too much variation.  (Also, I mistakenly assumed when you said "screen" that you meant just the gui controls and whatnot.  My mistake).

What might be more appropriate is to define a class around things like "Input Field" or "Table" or "Command" since these have a much higher degree of similarity.  Then estimate the screen as the composition of these more homogenous classes. 

Similarly, behind the GUI you can count things like classes, entities, relationships, files, etc, whatever makes sense to you.  This is essentially what function point counting is all about, and done right this technique produces good results.

I don't want to belabor the point, so I'll stop there.  In general, there are a large number of things that more mature industries have done in order to make their processes/products more predictable, and this shows up in those handbooks.  (Note that I don't believe that this is always a good thing - often you can trade a little predictability risk for a strong competitive advantage in the marketplace.  Let's leave economics out of it for now).

Finally, even with handbooks that give good data for individual tasks, many of these industries still have problems when they have to combine the tasks into networks of interacting tasks (i.e. projects).  The better organizations that find themselves in situations like this make plans to accomodate the uncertainty which remains.  For instance, most mature manufacturing concerns have quantified the uncertainty inherent in their processes using statistical process control.  They can tell with a high degree of confidence whether a given specification is achievable by their process or not.  Less sophisticated organizations resort to confidence tricks like the one's Joel describes.

Anyhow, software could take a similar route as an industry or especially at the individual team level.  We might not get to the level of precision seen in the metalworking industry, for example, but we certainly could improve things by several orders of magnitude if we'd pay closer attention to these issues and take the time to see how others have solved similar problems.  It won't come for free, though.

As an aside, I spoke with Steve McConnell a while back and he's apparently writing a book on software estimation called "The Black Art of Software Estimation" or some such.  From what I know about Steve and his work, he will very likely cover all of these issues in great detail and many more. 

Thursday, September 18, 2003

Sorry for the long postings, by the way.  It's hard to talk about this stuff and simultaneously be brief.

I wanted to add to my earlier posting that I don't necessarily believe that a high degree of precision is a desirable thing in this industry.  In order to achieve high precision outputs from a good estimation process, you almost have to have high precision inputs to start with.  This means the requirements have to be precise and not subject to change, you have to have a whole pack of interchangeable resources, and so on.  This is what many development approaches have been striving to achieve, lo these many years.

The problem is that this approach tends to drive the softness out of software.  One of the main advantages of software is that is it malleable (or at least can be).  You can shape it as you go.  The level of responsiveness that is possible with software is enormous when compared to just about anything else.  I don't think I'd want to let go of this advantage very easily.

Thursday, September 18, 2003

I thought one of the problems in software industry were the assertion that managers tend to believe software development is just another manufacturing task.

And about those processes/handbooks:

Six Sigma - Alstom (global player in power plants/transportation, 100.000+ employees) is _the_ success story for Six Sigma. This summer they nearly went bankrupt.

ISO 900x - Some 10 ys. ago managers thought it was unapropriate to handle their workers just like machines, so they came up with those ISO standards. "Look, we need a competitive advantage over the Asians, so let's get an ISO 900x certification ... Oh, now that we've employed our ISO 9001/2 quality assurance programs, you [the employee] are required to follow exactly the rules described on pages 1019 to 1274 of book 12. We don't want to loose our certification, do we?" Actually, nowadays it's appropriate even w/o that ISO quality disguise to squeeze the last bit of soul and dedication out of your workers.

Johnny Bravo
Thursday, September 18, 2003

Having said that software is much different then construction, I do agree that the more formalized the software development process, the better the estimates are.

So, I take well heed to the concept that adoption of standards, and a more formalized process DOES help the estimating  of software costs. However, the ability to quote on a square foot basis like contractors is not likely to appear soon. They just figure out the square feet, and then multiply by their square foot cost (for whatever type of floor they are making for example).

The problem still remains that a portion of software is a creative process. We are also are dealing with the fact this product is a NON physical thing.

This is a interesting debate, and one that has occurred many times. I mean, no doubt the architect who designs the building is creating something from the mind. It is then built, or manufactured.

The problem with software is that the design process is the very process that results in software! I suppose we could think of the coding part as the building process “after” the design is made. For sure, if you have a detailed design, then you can get very good estimates of the coding time.  However, the coding part on large projects only represents 15% of the total cost. In other words, the exact opposite of construction projects.

So, the design stage represents a major part of the software development process. It is this design part that is not formalized. In fact, it is not formalized for architects either, but for construction that does not represent the bulk of the costs, and we don’t care very much. The bulk of construction cost is when building starts.

Once you have very good detailed design, then the coding part is really the easy part in software anyway. The hard part of software is the design part...not the coding, or construction part.

Albert D. Kallal
Edmonton, Alberta Canada

Albert D. kallal
Thursday, September 18, 2003

"Once you have very good detailed design, then the coding part is really the easy part in software anyway. The hard part of software is the design part...not the coding, or construction part."

It might be true for a rigid framework (J2EE and the like) where you can work with a nice IDE and design with UML, MDA, etc. I don't think that statement is valid for a large percentage of software development projects.

You write 100 lines of code - you get bugs. You write 100.000 lines of code - you get bugs. But it's even worse: a bad programmer might produce more bugs in 100 loc than an excellent one in 100.000 loc.

I'd rather say the hard part of the whole software development process is ... debugging, and this is where most of the money is spent. Debugging the specification, debugging the design, debugging the code. We have yet to see a non-futile approach for cost-estimating that part. I tend to believe we will not see it ever.

Johnny Bravo
Thursday, September 18, 2003

The problem with Six Sigma is that it is (to my mind) very tactical.  It's a tool in the toolbox (a powerful one, but still just a tool).  Where it fails is in its inability to focus attention tightly where it will do the most good.  What difference if you achieve a 100% yield of your product if your sales force cannot sell them?  It's a local optimizer.

ISO (again in my opinion) also fails to live up to its promise, for similar reasons.  In addition, it is not responsive to the natural variation inherent in all physical processes.  It is the dreamchild of folks that live in an ideal world where all things unfold exactly as planned over and over again.  The approach is far too static.

These are tools that are useful as a part of a true continuous improvement process, but they aren't a sufficient solution on their own.

Thursday, September 18, 2003

From Yourdon’s Book Decline and Fall of the American Programmer:

page 179

For large projects, coding represents 12%, debugging and defect removal represented about 35%

For small projects, coding represents 75% of the code, and defect removal represented about 4%

In the middle we get 45% for coding, and 18% or defect removal.

It is not clear if defect removal is debugging process, but one would assume so.

Most other sources have stated about 10 to 15% for coding.

Of course that book is dated 1992, and we now develop with a GUI. There is also the adoption of COM, and OO. I suspect that the coding part has dropped a good deal since then. (we don’t have to code as much as the UI as we did back then). Further, with increased use of databases and GUI developer tools, again less code is needed.

Also, with a more formalized design processes, again the coding part would represent less of the  total time. (with really good designs...the code practically writes it self!).

Does anyone have some newer numbers on this breakdown?

Albert D. Kallal
Edmonton, Alberta Canada

Albert D. kallal
Thursday, September 18, 2003


I agree with your last post just about 100%.  I don't think we'll see process capabilities like those in high volume manufacturing or service industries for a very very long time.  Maybe never.  I do think a study of their methods would be fruitful for improving our own work, though.

I don't think construction is a good analogy for what we do for reasons you listed above, either.  I think that studying other kinds of work, though, would show some similarities.  For instance, look at how newspapers get published.  Or look at how movies get made.  Just about any kind of job shop that does custom work will compare well.  Even writing a book bears some similarities to the kinds of issues we see regularly in software.  These are mature industries that we can learn from and improve upon.

Thursday, September 18, 2003

> Look at how newspapers get published ...

Yes, if you told journalists to write their stories together, one person navigating, one person typing, they would tell you to f off.

If managements started saying there was a shortage of journalists, the journalists would go on strike. If managements tried to cut journalists' pay, there would be a strike.

Thursday, September 18, 2003

To make comparisons with other industries, the first and most important observation is that roles are well defined and sufficient people are hired to perform them.

You don't hire six people, give them shovels and tell them to excavate 1 million cubic meters by the end of the month. You don't also expect them to erect the formwork, pour the concrete, install the wiring and also manage deliveries to the site. Yet that's how a lot of software is expected to get written.

Increase staffing by a factor of ten and ensure appropriate specialists are on hand for all relevant tasks, and you would start to approach mature industries.

Friday, September 19, 2003

I used to work in the printing industry for about 10 years, and got pretty good at estimating how long a project would take to complete. Then I switched over to software, and after about 5 years now, I still can't give very good estimates.

The major reason IMO is that new things (technologies, platforms, programming languages etc.) are constantly appearing at a pace which leaves too little time to gather enough data of projects.

What I meanis , in printing we would use a technology that had changed very little in a hundred years, so of course I have seen most of the different types of jobs so many times, that I know how long they take. I have enough data to calculate averages that hit close enough to mark to be described as "accurate".

But in software, we get new platforms and new programming languages and new technologies etc. etc. so much, that none of them has enough time to gather data enough. If I were to program for DOS using C language, and only build apps that communicate via a serial port I/O, by now I would probably be able to give very good estimates of new projects. But now I'm constantly studying new things which I need to use in my projects, so of course I have only a vague idea about how productive I can be. Let alone someone else. I don't know what exactly they know, how much and well can they learn and so on.

It will be very interesting to see what happens in, say, 20 years time from now. Will some technologies mature and become so commonplace, that the time it takes to produce software can be estimated like the time it takes to pave a road today? I don't think so, but in any case it is an interesting progress to watch.

Antti Kurenniemi
Friday, September 19, 2003

I've worked with software for more than 30 years, and in the last few years I've been involved with several construction projects including, now, my own house. The estimating and scheduling issues are similar in both types of projects.

Software projects and construction projects have unknown risks. Remodeling projects are famous for surprises when walls are opened up. I talked to a builder last night who recently found during a remodel that his own house was balloon framed, which was a surprise to him. These risks can be reduced at the cost of early investigation. Many software project risks can be reduced through simple prototypes or proofs-of-concept. However, the cost of reducing risk may be too high. The foundation of our new house requires good soil conditions, which we expected several feet below the surface. The city plan reviewer asked for soil samples. Our geotechnical engineer and structural engineer pointed out that this was too expensive, and wouldn't be definitive. We elected to discover the soil conditions during excavation. As it turned out, the good soil was closer to the surface than expected, except in one area, in which we had to dig down an additional three feet for footings. It could have gone the other way. Many houses in our Seattle neighborhood have cracked foundations because they are not on good soil.

A problem for software projects is that developing for the single most popular target environment (Microsoft Windows) resembles a remodeling job more than a new construction job. The total API is huge, poorly defined, unstable, and unpredictable. Even experienced Windows programmers seem to experience surprises on almost every project. Construction projects have problems with new techniques and materials. For example, many condominium buildings around here had siding problems. The siding manufacturer claims the problems are due to improper installation. For our house, we had to look long and hard to find builders who had experience with structural steel. Our selected builder immediately found large cost savings through small changes in the plans. Experience and knowledge really do count, and they are hard to get in some environments.

Construction projects and software projects both have problems with low bidders who expect to make up the difference on change orders and surprises. Many software supervisors I've encountered give management the answers they want to hear about schedule and cost, knowing that management won't really cancel the project once it has started. They are very good about giving excuses as to why changes and surprises have altered the schedule. If there is a difference, I think in software, many people lie to themselves, convinced that they are one of the elect few software engineers with productivity orders of magnitude above that of other programmers. This causes them to create a meaningless schedule. Most builders don't seem to lie to themselves as much, even if they do lie to the customer.

If it is any consolation, I was talking to a builder early this morning, who told me that Bill Gates ran into the same problem when choosing a builder for his house. The builder bid on a house, but basically a bare one, without the finishes that the Gates family wanted. This was not caught until construction, even though Bill Gates had his own project manager working for him. It resulted in large cost increases.

My experience working with a builder and coming up with a schedule for a small project (my house) is that it does work. It is important not to lie to yourself and put down numbers you can't meet. It is also important to understand that just because Microsoft Project says that the house will take seven months to build doesn't mean it won't really take nine.

What I found interesting about building a construction schedule was learning when decisions had to be made. This seems to be a rare experience in software projects, which in my experience are based on either very rigid specifications or poor specifications. For example, we learned that we have to decide on specific plumbing fixtures as of a certain point in the project. It becomes very clear how making small changes during construction can result in a cascade of problems later.

The oddest experience I've had so far is discovering that it was too late to remove a window in the house foundation. The foundation forms were being erected, and removing the window would have required only a few minutes, since all that was present were the forms that kept concrete out of the window opening. However, removing the window would have resulted in changes to how the rebar (reinforcing steel) went in. The rebar wasn't up, but people were already cutting and placing rebar in other parts of the foundation. Communicating the trivial change among the 5-6 people working on the site would have led to confusion. or at least a delay as everybody was informed. We decided to leave the window, and keep going.

Software engineers and managers claim that customers and management do not want to hear the truth about schedules and cost. Builders complain about the same problem - customers want a certain level of finish, but are unwilling to pay for it. In many cases, the solution is the same: lie about the schedule and cost, knowing that the customer will make changes later. Builders seem to do this by proposing a low level of finish, that the customer will change later. Software project plans seem to be built with unrealistic schedules that are ignored once the project is going.

Dan Brown
Friday, September 19, 2003

Yup, It's microsoft's fault.

Friday, September 19, 2003

Comparing software to building can be fun, but often far from enlightning.
e.g. here in Europe we often build houses from brick. A single house takes thousands and thousands of bricks, all identical. These bricks have been tested for forces, and simple sampeling in production can assert the conformance of the bricks to spec. Those same type bricks have been used in hundereds of buildings before, and the general brick priciple hasn't changed much in the last 600 years.
Appart from some decorations, there is hardly much unique parts nescessary as the complete "building space" can be covered by reasonably simple and wel understood configurations of a few basic parts.

In software there is never a single duplicate part (well, there shoudn't be). Every single component is unique. Furthermore, most components are one-off hadcrafted pieces. All compositions of the components are also unique, and mostly non redundant. The "software space" is vast and can not be covered by simple combinations of a few basic parts. Interactions are complex and non-localized.

Just me (Sir to you)
Saturday, September 20, 2003

I'm sorry, but that's just pure hyperbole.

"In software there is never a single duplicate part..."  Two instances of the same class are as similar as two bricks cut from the same mold. 

"Every single component is unique. Furthermore, most components are one-off hadcrafted pieces."  Yes, and why is this?  Conceptually, software is reuseable the same way bricks are.  The same concepts appear over and over and over, especially within a given problem domain. 

Somehow, though, when it comes to implementation we can never seem to pull it off.  Each time someone builds a new accounting program, they invariably have to come up with their own definition of the "Account" concept.  Do you think this is an inherent problem with software?  What other explanation might there be?

"All compositions of the components are also unique, and mostly non redundant. The "software space" is vast and can not be covered by simple combinations of a few basic parts. Interactions are complex and non-localized. "

Same with building.  Look at the almost limitless variations on the concept of "House" for example.  The "building space" is just as vast as the "software space"  Interactions are even more complex and nonlocalized in building too.  The force of gravity on the brick at the top affects the pressure felt by the brick at the bottom, for example.  Dynamic forces from wind interact with building materials in enormously complex ways.  We ain't got nothing on those guys in that quarter.

These kinds of arguments always look at building in an abstract way - "a wall is a wall is a wall."  At the same time you are looking at software from the point of view of a practitioner - "of course a bubble sort is different from a mergesort, you fool!"  This is probably why we are agog when business guys look at a GUI and think "how hard could it be, for pete's sake?"

Saturday, September 20, 2003

anon, the variations in house design are not as deep as those in software. Different configurations of rooms, different finishes, and and different components are all accommodated within standard, well known procedures.

Software is not bound like that, just as the character of companies and organisations is not bound like that. If you think software complexity is of the same order as house complexity, you must be building web sites, and simple ones at that.

Saturday, September 20, 2003

A better comparison to software is writing a novel or a screen play or composing songs or an opera.  It's intellectual work, with fuzzy criteria for what constitutes good software.

So I wonder how accurate schedules are for these fields, and how much variation there is between practitioners in productivity, etc.  Also, what kind of metrics do you have for these people?  Do you pay a novelist based on words generated per hour/day/week?  A composer per note?

Software Engineering is a huge misnomer, engendering all the wrong ideas.  Software Composition, maybe?

Jim Rankin
Sunday, September 21, 2003

"anon, the variations in house design are not as deep as those in software."

I am sure that any competent architect could produce at will architectural problems that are easily as difficult to resolve as the stuff we see in software, so I have to disagree with your statement. 

"...components are all accommodated within standard, well known procedures."

The point I am trying to make is that a large class of software problems is solvable by the exact same means.  Current software practice most often fails to achieve this, but it does not seem impossible to me.

As far as the kinds of systems I build, you've actually gotten it backward.  The systems I've worked on are mind-bogglingly complex.  This has forced me to learn a lot about how to manage complexity and variation in order to make the problems I work to solve tractable. 

Sunday, September 21, 2003


I mentioned "writing" in an earlier posting as a place to look for concepts that might be applied to software, so I agree in part with your thesis. The big difference for me, though, is the precision of the goals.  In writing novels and screenplays, most often the goals are very fuzzy - irreducibly so.  For instance, it would be very hard for an author to describe precisely the state he wants to put his reader's mind into, after the reader finishes the work.  Even if he could, though, variation in the people consuming the work would defeat the author's attempt.  The author or composer has only a very vague model of his "problem space."  This imprecision introduces irreducible variation into the work.

Software programs, on the other hand, usually have more precise, business-oriented goals.  It's true that human factors play a part, but to me these seem more like necessary conditions than goals.  What's more, these things are usually measureable.  It should be much easier for the business analyst or product manager or whoever does this work at your location to describe the state of the business or target market after the software product is in place.  In my experience, this precision provides opportunity for reducing variation in our work - enough to admit an engineered solution in a large number of cases.

This is why I continue to use the Software Engineering appellation for the work we do.  With the current state of affairs, though, I think the "craftsman" zealots are being more accurate.  We're not there yet.

And before everyone gets out the flamethrowers, I know there are certain segments of software that require enormous amounts of creativity and which pursue nebulous goals.  I'm just saying that a lot of what is common practice in our industry is needlessly complex.  Further, there's a non-trivial amount of work that requires essentially zero creativity.  I'll bet everyone reading this knows at least one person that spends the day scanning clipping snippets and pasting them into your complany's next enterprise application.

Sunday, September 21, 2003

One more note.  I think the biggest gap between some of these other fields and software is the economics of the situation.  It costs a whole lot of money to build a copy of a house, but building a copy of a program is very cheap.  Economics have an enormous impact on the processes that get selected whether you are building the next version of Tetris or the next Taj Mahal.  Look for a mature industry that operates in a similar economic environment as software, and I'll wager it is a rich source of useful ideas and concepts you can apply in your own work.

Sunday, September 21, 2003

"anon, the variations in house design are not as deep as those in software. Different configurations of rooms, different finishes, and and different components are all accommodated within standard, well known procedures."

While we toil away inventing the two-by-four, the nail, the woodscrew, and drywall, each time we build something new.  Plus we do all our work with a hammer.  Sometimes a claw hammer, sometimes a ball-pien hammer, but always a hammer.

Tuesday, September 23, 2003

*  Recent Topics

*  Fog Creek Home