Fog Creek Software
Discussion Board

When the software industry is mature

That is a excellent article by as to why software is NOT like the construction industry. The point about each software project being different then the previous project is also why us developers still have work!

However, there is a major industry where things change from project to project, but as a whole still seems to deliver on time, and on a budget.

What industry are we talking about?

Well, the movie industry of course. Pop in any dvd like the Matrix, tomb raider etc. and view the "notes" on creation of the move. It is a amazing process, and reminds me much of software. The sets, ideas and construction that occurs in movie usually has not been done before. Many movies create special effects and the like that have not been done before.

While the movie industry is full of failures, and over budget does seem to control, and deliver a project in a fashion somewhat better than the software industry. Perhaps the software industry has much to learn from the movie industry.

Or perhaps those movies just throw money at their problems.....

How come they people can make movies on time...and not software?

Albert D. Kallal
Monday, December 31, 2001

Movies don't carry any risk to the viewer when they fail, beyond the money spent buying the ticket/video.  Software carries businesses, livelihoods, personal correspondence, hours of time, as its burden.

That's why movies look good by comparison - they don't have to *do* anything.

Neil Stevens
Monday, December 31, 2001

Don't that that all (or most) movies get made on schedule or on budget - just look at Waterworld or Titanic for a couple of extreme examples.

Jeff Pleimling
Monday, December 31, 2001

>How come they people can make movies on time...and not software?

Games don't have any bugs when released. Game developers do not really use any software engineering techniques described in the standard best practices books. Yet  the gaming industry at large does not miss deadlines.  Sega is known for never ever missing a deadline. Of course, there are game companies that don't make deadlines, but they don't exist for very long.

Of course, writing a content-management system isn't like writing a game....I just wanted to point out that there are forms of software that are consistently released on time.

I'm not sure if a company like Fog Creek can learn much from a game development company, but it might be interesting to see what can be learned.

Rolf Hanson
Monday, December 31, 2001

Unless you are writing the new Quake 3D engine, most computer games reuse a well-tested game engine, which is programmed/parameterized with game data and graphics. There is not nearly as much new code as in Microsoft Word 12.

Monday, December 31, 2001

You must not have ever worked in the movie industry :-)

The various creative shops which do work on time and within budget use extensive project management tools - schedules, resource forcasts, etc.  They have a good modl of how long it takes to do certain stuff, and they are always improving.

So the "secret", I guess, is to plan and manage.

Dave Liebreich
Monday, December 31, 2001

movies aren't mission critical...i remember reading somewhere that majority of movies lose money (i think it was like 40% that turn a profit).  the movie industry makes money on big blockbusters like LoTr or Harry Potter to make up for the duds.  imagine a world where 60% of software is so buggy as to not be usable.

also, in my experience (and from what others have told me as well in their own projects), a lot of the time is spent getting the application to work 99.9% (add or subtract whatever decimal place suits you-you get my point), rather than 90% of the time.  on the other hand, a lot of movies have parts that are filler, or slow, or where the execution just isn't that great.  the rest of the movie can make up for it, because the whole of the movie isn't totally contingent on each part.

razib kahn
Monday, December 31, 2001

>Games don't have any bugs when released.

Ummm.  Yes they do.  Its not unknown for a patch to become available within days of release.  Games are often developed by very small teams doing very specific things.  Even if the engine isn't an existing one your average game is considerably simpler in structure to a piece of software with arbitrary inputs .

Simon Lucy
Monday, December 31, 2001

Software is, at it's core, built on code: it's a computer following instructions. These instructions are written, typically at a much higher level than the computer itself sees, by people. An altogether trivial change (lets call it a mutation) can result in a catastrophic failure of the software. In fact I'd say this is the typical result of making small "mutations" to a piece of software.

Now look at a bridge. Civil engineers don't "program" a bridge. A bolt missing here or there won't cause the structure to collapse. The strength of the bridge design can be modelled to determine how it will act in normal use. Random small "mutations" in bridge design probably won't cause the structure to collapse.

I think this difference is fundamental to software and isn't likely to be going away any time soon. The movie industry isn't really a good model either: actors can flub their lines, dialogue can be changed, scripts can be rewritten, scenes can be reshot, edits happen; but a movie won't cause the movie theatre to go up in flames though it may cause the audience to run screaming. Are there any better metaphors for what mature software development would look like? I'm not really sure there are, actually; given how recently programmable machinery has arrived on the scene, I wouldn't expect there to be, either...

Oliver Seiler
Monday, December 31, 2001

Of course there is a 'Tile Floor Template Library (TFTL)', I don't think Herman the Handyman made the tiles in your own home himself, I am willing to bet everything I have they were bought made.

It is very naive to tink that contruction engineering is always the same and
hasn't been doing anything new in years, they also like doing the
tallest building ever, or the wierdest bridge or try that new material.

Somewhere I read about this, it
is about the contruction of a state-of-the-start bridge, something that had
never been done before. Yet the bridge stands.

Civil engineering does have an advantage, one that a building is up,
it is very hard to make it come down. In software a minor mistake can
show up in a big way though.

Andres Garcia
Monday, December 31, 2001

>>Games don't have any bugs when released.

>Ummm. Yes they do. Its not unknown for a patch to become available within days of release.

My apologies. I had console games in mind when I
made my comments. Console games typically don't have
any noticeable bugs.

Rolf Hanson
Monday, December 31, 2001

> imagine a world where 60% of software is so buggy as to not be usable.

Try ... ;-)

Rolf Hanson
Monday, December 31, 2001

This is actually a deep question, I think.  After all, when I was immersing myself in things like Literate Programming, I learned commenting style from those DVD director's commentaries.  So some brainstorming:

I understand the movie industry is about maximizing business and not creativity.  Therefore it is harder to get investment for a movie, and it's slanted towards controlled risk. 

Special effects are extremely expensive.  I often hear figures of hundreds of thousands of dollars for short effects.  There's the trinity:  fast, stable, cheap; pick two.  Good quality under good schedules is possible, provided you are willing to pay.

A principle of innovation is to limit it to one area.  Lord of the Rings is a perfect movie, since the story is already time-tested, the special effects are beautiful but spare, and the main innovation comes from storytelling skill.  Software companies do this as well; it minimizes R&D.

Richard Jenkins
Monday, December 31, 2001

Oh, I forgot one until I looked at Rolf's note on console games.  All the special fx machines are completely controlled, to a degree not possible with PC's.  That kills many bugs dead; no Packard Bell machines causing humorously twisted errors.

In fact, the users (artists) are often expected to go through a steep learning curve, so small glitches and holes can be answered with "don't do that."

Richard Jenkins
Monday, December 31, 2001

I think Richard sort of has it with that controlled environment bit.

And now to carry on the construction analogy:

If you drive over a bridge and there's a wierd noise because the engineer decided to make the roadbed a grate so he didn't have to worry about ice building up on it (or whatver it is that causes that), this is like a bug that can be worked around--it's a little bit of a nusence, but you can deal with it.

A security hole, however, is like someone started taking the bolts off of your bridge.  Civils don't generally have to worry about malicious users, programmers do.

Another advantage that Civil Engineers have is that they can add a safety margin.  The bridge won't fall apart if you forget a couple bolts because it was designed to take a load several times that which it actually carries.

Michael Chansky
Tuesday, January 01, 2002

[imagine a world where 60% of software is so buggy as to not be usable.]

Oddly, I don't find this all that hard to imagine :-)

[Of course there is a 'Tile Floor Template Library (TFTL)', I don't think Herman the Handyman made the tiles in your own home himself]

Ah, but the TFTL Joel was talking about would not just make the tiles, it would them in.  All of the cost would be in figuring out how to make and install them, but once that's done, it would cost little to make and install tile in 10,000 homes instead of 1 home.  Software is like that.  Tile is not.

[Civils don't generally have to worry about malicious users]

Unfortunately, this is not true, as events earlier this year pointed out.

Kyle Cordes
Wednesday, January 02, 2002

Anyone who has had a home built knows that in that portion of the construction industry, there's a lot of similarity to the imperfection of software development. Smart buyers do daily walkthroughs, and can tell many stories about the "bugs" that can be detected -- walls misplaced, ceilings framed at the wrong height, windows missing or added, slabs having to be pulled up and repoured, this time _after_ the sewer line is installed, etc. 

Having office space finished out is much the same, only with limited ability to change the shell.  I'd guess that commercial buildings and civil engineering projects (bridges, roadways) show many of the same characteristics.  There is underlying engineering, then there's actual construction.  Yet there seems to be general agreement that the construction industry is mature, with well-understood, documented, and (nearly) standardized techniques.

Good engineering recognizes the fallibility of the construction process, and attempts to accomodate it through simplification, redundancy, "overengineering," and other techniques.

Daily builds, bug databases, code reviews, and the like are tools that can be used to keep the construction process on track, so that the final artifact is close to the initial intent.

Bob Bechtel
Wednesday, January 02, 2002

One difference between construction and software is that a general contractor doesn't hire a cement man to do framing or drywall or roofing.

I can't count the number of times I've landed a programming contract only to end up doing something outside my primary skill set (e.g. requirements gathering, testing, report writing, etc.).  Sure I can do these tasks, but there are people out there better at them than I am.

Isn't specialization of labor a basic trait/requirement of civilization?

Brandon Knowle
Wednesday, January 02, 2002

Several posts on this thread suggest that a mature industry produces fewer bugs than an immature one, perhaps as few as zero bugs. The notion that the construction industry has zero  bugs might be giving too much credit to the construction industry. Ask the Seattle Mariners or Milwaukee Brewers, who had their ballparks "shipped" to them 6 months and 1 year late, respectively (the Mariners' home ballpark was also overbudget by more than $100m. And their are plenty of two-bit contractors who don't produce a quality product (and the money they spend isn't large enough for a breach-of-contract suit to be cost effective)

Nick Beaudrot
Tuesday, January 08, 2002

Software industry is complex because we are trying (note, trying not doing) modeling the organization itself, I think.
Any organization has two worlds, two organizations inside: one formal and another non-formal.
Formal structure have rules, positions, policies, procedures, some times witten, other times not, but they are what we see and perceive as "the real organization" ("the would be" organization), but really, it is the "ideal organization" ("the being" organization).
The non-formal structure is that which is not witten, is built on power and relationships, and many times it doesn´t fits with the formal structure.  This non formal organization is which really operates the organization.
So, when a systems analyst is trying to capture the organizational requirements he (she) is looking for the formal structure organization. But, if he (she) doesn´t is trust with final users (or whom are doing this role)  he (she) doesn't capable to get this non formal structure.
This is the in depth cause because software projects are failed. These projects are developed for the "formal structure" which is not  the really structure which moves the  organization.
¿What do you think about?

Jesus Zavala
Sunday, November 16, 2003

*  Recent Topics

*  Fog Creek Home