Fog Creek Software
Discussion Board




Objection to XP (extreme programming)

I have recently read a lot about XP, SCRUM, FDD (feature driven development), etc.

Most agile methodologies develop a product incrementally, and make sure quite often that the product works and is in shipable state.

Let's make an analogy with making a car.


Metod 1 (traditional):

You design a car.
You make all the pieces one by one.
You assemble the pieces into larger parts (for example the gear box, etc).
You assemble the larger parts into a car.

A problem often appears while doing this: you discover that your design is flawed, and have to trace back a few steps, redesign some things, etc.


Method 2 (similar to agile methodologies):

You first make a bicycle.
Then you make an engine.
You put the engine on the bike and make it a motorcycle.
You add to the motorcycle a stronger frame, and 2 additional wheels.
You add chairs, etc.
You make a larger engine.
... and so, you have a car!


Method #2 may work, but isn't it A LOT MORE EXPENSIVE than method #1?

:-)

George
Wednesday, July 07, 2004

Nice troll. I'm still waiting for the day when people realize that making cars and making software is quite different.... (Regardless of what methodology you use)

Just to make that clear - designing a car involves numerous models and simulations that you constantly improve. (You build your software). Then, you mass-produce it. (You compile and link it. Or, if you prefer it simpler, you copy it).

Same analogy, completely different approach to software design, both meaningless. Can we *please* stop thinking "creating software is just like..... "? It's not - it's a very unique process.

Robert 'Groby' Blum
Wednesday, July 07, 2004

It depends on how you define "expensive" - it's a good analogy, but expensive in terms of building software basically equates to: how many man-hours does it take to build, and how stable is it.

It's not clear from your analogy whether one approach takes longer or leads to a better product than the other.

Joe
Wednesday, July 07, 2004

> Can we *please* stop thinking "creating software is just like.....


I cannot until someone points out how exactly it is different

Ignore my ignorance
Wednesday, July 07, 2004

While the original poster's argument may be flawed, I'm sick and tired of people justifying things in software management by saying "it's just different."  Hogwash.  Some things are different but a lot of issues are similar to other industries.

Please be more specific about why you don't agree.

I would argue the following.... big design up front (option 1) is necessary in auto manufacturing because changing the design later is extremely expensive (parts, tooling) and you're going to build a lot of units so the cost averages out.

This is an oversimplification, but in software, we only need one working copy of the item and then the marginal cost of producing another unit is low (pressing another CD/DVD), so there is some value in using a different approach than traditional manufacturing.  That one unit may have to work on a variety of different systems but the point is about the distribution of marginal versus fixed costs.

Jeremy
Wednesday, July 07, 2004

I'll bite.  Let's have more fun with analogies.  XP seems like it is a lot like rock climbing where you tie off as you climb.  Sometimes you choose the wrong holds and have to back track.  Sometimes you fall, but luckily you only fall to the level of your last tie off.

Top-down would have someone go out to the mountain and look at it.  Then he comes back and tells the climber what it looks like and the path to take.  Then he says here's $100 to start, I'll give you $100 more when I see you at the top.  Oh, and by the way, your map only has one waypoint on it to gauge your progress and you have half as many tie-offs.

cheeto
Wednesday, July 07, 2004

"I cannot until someone points out how exactly it is different"

Since you're asking: We're creating something intangible here. So writing software (if it is comparable at all), is comparable to creating intangible products. Music, writing, meditating on the true nature of software.

Since we're also rooted deeply in mathematics, music is probably closest, if you *have* to compare.

Robert 'Groby' Blum
Wednesday, July 07, 2004

Does somebody want to dig up that "If Microsoft Made Cars" bit that's floating around the Internet? It's pretty funny.

Building cars is nothing like building software. XP is intented to be applied to the building of software and might be very expensive for building cars as you described. An assembly line works really well for building cars, but it is not the way that software is built.

Each person working on an assembly line does a single sequence of things repetitively. Software developers working on a project solve a different problem every day.

You can easily replace an assembly line worker with another, train them in a short time, and they'll be doing the same job as the one before them. Not the case with software developers who build up a lot of knowledge over a long period of time, and when they leave, the take it with them, at a great loss to the company.

I worked for a very large manufacturing company, doing software for about 10 years. They never understood how software isn't simply a manufacturing process. They never had a clue how to manage software.

Mark S
Wednesday, July 07, 2004

"You design a car."

Are you thinking that somebody just sits down and draws up the designs off the top of there head?

The first thing you do when building a car is that you build a prototype. 

Take  a look at this page for example:

http://www.gom.com/En/Applications/Digitizing/rpm/rivage.html

See that guy carving a model out of wood, that isn't going to be the final car, that just one of the iterations on the way to the final car.  They don't start with bicycles, but they don't start with finished designs either.  It won't have everything a finished car has, but it will have everything necessary at this stage.  For example, they can use it in a wind tunnel.

You could build the whole car, then test it in the wind tunnel, but that would be much, much more expensive.

In an agile methodology you start with a simple model.  Few companies have the equivalent of a wind tunnel, so you have to produce working prototypes.  These working prototypes don't have all of the features of the finished product, but they do have the necessary features to use it for a given purpose.

Ged Byrne
Wednesday, July 07, 2004

....We're creating something intangible here....

Hm, software is very real to me. Software feels different if it crashes than if it doesn't. Software that isn't snappy feels worse than software that responds fast. Software that I can rely on feels better than software I don't trust.

Indeed, software is very much like a car. I want a car that takes me from A to B and doesn't fail on me. I want software that delivers data B when I enter data A and that doesn't fail when doing so.

Lastly, software, although weightless, is tangible because there are buttons, menus, input fields, dialog boxes etc.

.....Each person working on an assembly line does a single sequence of things repetitively.....

A person working on an assembly line correspondents to a person working in a company that burns existing software on a CD or prints manuals for the software. The assembly line workers are different from the car designers. And designing a car is like programming.

René

René Nyffenegger
Wednesday, July 07, 2004

"Method #2 may work, but isn't it A LOT MORE EXPENSIVE than method #1?"

This is a very important point.

For a car it is a lot more expensive.  Chassis and engines are not plyable.

With software the raw material are intangable bytes.  They can be quickly and cheaply altered and changed.

Ged Byrne
Wednesday, July 07, 2004

The problem is that analogies between software and manufacturing are often misleading, not to say fallacious. A lot of software refactoring does not involve major changes to things like supply lines, equipment, etc., which you'd need to change.

That is, it's not just what it costs you to rearrange your own components and processes, but what does it cost you to arrange to get other components you need from suppliers? With software, it's easier -- you can evaluate many libraries without incurring cost for shipment, etc., because cost of copying information is zero. These really add up if you decide to spend a month how your car would end up if you used bicycle seats vs. dental chairs :)

DEBEDb
Wednesday, July 07, 2004

Thank you very much for your replies.

I'm not trying to troll. In fact, I'm trying to understand agile methodologies in order to apply them.

What I don't understand is this:

I have to write a program with features F1, F2, F3 ... F10.

So let's start the agile way:

I write a program only with features F1 and F2.

It works, it's a working prototype, everybody's happy, maybe it can even be used in production.

Then I have to add F3.

For adding F3, I have to rewrite 40% of the code, and then add 15% more code for F3.

Then I get a program which has features F1, F2, F3, which works, yada, yada, yada.

For every feature I add to my working program, I have to rewrite SOME code - probably a LARGE portion of code, and then ADD some code for the feature I am currently adding.

If I have many features to implement, because of the incremental way the code is developed, and because of the requirement that the program must always work, I have to rewrite and rewrite and rewrite and rewrite.. for each feature.

In a traditional model, where everything is designed upfront, rewrites may occur when I discover unexpected difficulties with the design.

So, when one uses agile methodologies, one has to write code for all the features, and also, for most features one adds incrementally, rewrite a significant amount of the existing code code.

When one designs everything at the beginning, rewrites also occur, but there may be a lot less rewrites.

Please don't take my post as a troll. I'm simply trying to understand something.

Because of the facts I explained above, I fear that a project done using agile methodology would take something like:

100 hours coding of the features
120 hours rewriting code due to the incremental nature of the process and the need to have the program in a working state often (every week, for example)

TOTAL TIME: 220 man*hours

whereas, using a classic methodology, it would be something like this:

100 hours coding of the features
60 hours rewriting existing code because of unexpected problems, design problems, etc.

TOTAL TIME: 160 man*hours

Project done using agile methodology: 220 hours

The same project done using design up-front: 160 hours

I'm not trolling. The above is simply what my (probably wrong) intuition tells me it would happen.

So, can you explain to me why agile methods are faster?

Thank you!

George
Wednesday, July 07, 2004

> With software the raw material are
> intangable bytes. They can be
> quickly and cheaply altered and
> changed.

The problem is exactly the fact that they can't be cheaply altered and changed.

Software development is slow and developer rates are high.

Once you write a program, changing it is often difficult and time-consuming.

George
Wednesday, July 07, 2004

I'm not arguing one way or another (I can believe Waterfall is the most efficient for perfectly specced code), just your analysis doesn't take into account changes to spec, or an incremental model which requires a customer to look at and suggest course corrections.

Perhaps an analogy is what you'd use for a perfect straight road with no other drivers, vs. course corrections based on traffic.

Tayssir John Gabbour
Wednesday, July 07, 2004

"Once you write a program, changing it is often difficult and time consuming."

Of course, that might be true with traditional software development, but part of the XP philosophy is TDD. When you have good tests, you can refactor with reckless abandon. Don't forget a good refactoring plugin. That makes your software extremely pliable.

Brad Wilson (dotnetguy.techieswithcats.com)
Wednesday, July 07, 2004

All of software development can summed up in one point above:

"Method 1 (traditional):
You design a car."

In software development, all you do this step.  There is nothing else. 

So although a car is built by assembling pieces together to form larger pieces and so on -- that's not relevant.  What you need to know, is how the car is DESIGNED and nothing else.  As people have pointed out, it's probably pretty similar to software development.

Almost Anonymous
Wednesday, July 07, 2004

So if you use the big design up front method you tend to miss what the customer REALLY wanted. So you deliver the system but then the customer says sorry that isn't quite what I wanted and then you spend another 200 hours fixing things.  Using XP you find out about these design changes a lot sooner which makes it cheaper and the chances of the customer getting what they really wanted increase dramatically.

DJ
Wednesday, July 07, 2004

You know, this is supposed to be a "scientific" discipline. Why do so many people say "I've never done this, but my feelings tell me this is the outcome we'll see"?

How about you try it out and see if it works for you? And if you don't want to try it out anyways, what's your interest in it?

Robert 'Groby' Blum
Wednesday, July 07, 2004

Also, the rewriting of code is correct, but the orig XP book claims that improved tools and languages have decreased the cost of modifications.

Again, I'm not arguing one way or another. I think Java is not that agile a language, but is kind of pushed in that role, but I haven't tried it with a refactoring ide like IntelliJ.

Tayssir John Gabbour
Wednesday, July 07, 2004

Extreme Programming is just some bullshit some guy pulled out of his ass one day to try to sell books on a 'process du jour'.

It has no merit.

Mr. Fancypants
Wednesday, July 07, 2004

Are you supposed to use Hungarian notation in Xtreme Programming?  And what is the best language to use with Xtreme Programming: C#, VB.NET, or Java?  Also are you supposed to use exception handling or check return codes?

I would like to learn the answers to these important questions so I can begin using XP properly.  It is only once or twice in a career that something as new and exciting as XP comes along and I can't wait to use XP everyday in my projects.

free(malloc(-1))
Wednesday, July 07, 2004

HA HA HA HA HA HA HA HA HA HA

free(malloc(-1))
Wednesday, July 07, 2004

George,

Using the traditional method, don't forget to take into account the amount of time required to do the design process.  Often, this involves many people and many whiteboard sessions.  Then it has to be approved, and any developers who weren't involved with the design need to be educated on it.

If the design was flawed and needs refactoring, or the requirements change (and they always do!), then the whole design process starts over again, albeit possibly only for a subset of the application.

We recently began discussions with a new potential client who brought with a detailed project spec that they had evidently payed a *very* large sum of money for.  But when they came to us, they said "This design is useless.  It took months to put together, and doesn't really reflect what we're trying to accomplish."  Effectively, without knowing anything about programming, they were telling us they would prefer an agile methodology, so that they can see interim results and change course direction if necessary.

Remember, it's a cost-benefit analysis, not just a cost analysis.  And there's no one-size-fits-all answer, either.

Joe
Wednesday, July 07, 2004

Creating a new piece of software is exactly like designing a car.  First you come up with an overall design, given the requirements.  Then you take the design and create the finished product.  What's the finished product? 

In the case of software, it's the source code (along with other files).  The source code and these additional files (icons, help text, etc.) form a specification.  This specification is turned into a working product by using a build system, whether it's simply typing "gcc" or "javac" on the command line, or a complex automated system that takes several hours to run.

In the case of a car, the finished product is the engineering drawings and other documents which describe the materials to use, and how to cut and form and assemble them.  These documents form a specification.  This specification is turned into a working product -- a car -- by people and robots working in factories.

The OP's analogy is fatally flawed.  Here's a better one:

Traditional:
1. Design up-front every detail of the product to be built, based on the requirements.
2. Build the product.  Discover that assumptions were made, things were left out, there are various other problems with the design.
3. Work out the problems as best you can, doing what's necessary to "make it work" while fulfilling as much of the original design as possible.

The result is still a car, but it's got some problems.  Maybe the original design called for a 400HP vehicle, but there wasn't enough room in the engine compartment for that big an engine, so the car tops out at 340HP.  Or maybe the engine got built first, and in order to fit it in they had to make a big bulge in the hood.  Etc.

The XP/Agile approach:
1. Choose a requirement or three.  Create something that fulfills those requirements, just barely.
2. Pick another requirement or three.  Create something that fulfills those requirements, modifying what you just created in step 1 so that it all works well.  If you have a serious problem, something that makes it extremely difficult to be able to meet some of those requirements simultaneously, you know up front, and can either modify one (or both) or perhaps make a special effort.  (It's gotta be 400HP _and_ fit in this small engine compartment?  Okay, let's look into advanced composites....)
3. Keep going until all requirements have been satisfactorily implemented (designed).  (Okay, we simply can't get 400HP in this space... but we could get 380HP into a slightly larger space.  Not the original plan, but it still satisfies the higher-order requirements of "powerful, small package".)

The agile approach is a more fluid and flexible approach to software design.  However, it does require more flexibility on the part of the customer.  They need to be willing to sacrifice 20HP.  They need to be able to defer some things until the future.  They need to be willing to live with a degree of uncertainty.

Few are willing to do that.  It's a lot easier to say "you will provide this software implementing X, Y, and Z features by September 1" and let the developers get stressed out and panic over trying to fit that 400HP engine into a too-small engine compartment.

Should be working
Wednesday, July 07, 2004

George, XP was never designed to be a cheaper alternative.  What you get is 100% customer satisfaction.  The reason is, 90% of customers don't know exactly what kind of software they want written until halfway through the project.  Sure, if you give me a huge spec and let me chug away at something using the waterfall methodology, it will be cheaper then if I get no spec, and only hear each feature request 1 by 1.  BUT, if I'm using the waterfall methodology, and it turns out when client X said I needed to sort the databse, she really didn't mean sort the database at all, what she meant was export an XML representation, sort that, and then send it off somewhere awaiting a response, we're kinda screwed.  In a perfect world where you know *exactly* what the end product should be, a waterfall type approach is superior assuming you have a good designer.  But, how often is the world perfect?

vince
Wednesday, July 07, 2004

I'll bite.

Agile methodologies emphasize the incremental aspect of software development so there is a working, deliverable product every couple of weeks: it builds cleanly, your customer can install it, and provide feedback.  Each iteration "fills out" functionality more and more.  When you can't think of any more tests to write, you're done.  If the project is cut short, yoru customer simply has a less-functional system.

Traditional methodologies are basically one huge iteration.  The whole thing doesn't have to build until the very end, so you better make sure your specification was very thorough, and that everybody (including the customer) interpreted it the same way.  If you aren't correct, it's back to the drawing board.  Except in the real world, where there is a need for heroic action (see "death march") to save the project, and to quell an angry customer.

Partially because of this, agile methodologies are more predictable.  As the work progresses, it is easy to see how far behind you are.  This is far less transparent with the waterfall model.  So even though it seems like XP and friends encourage more code to be written and refactored, you know this up-front, where this is hidden with traditional models.

I don't think the particluar language chosen has much impact as to whether a project can be built using XP, but object-oriented languages tend to work well, given the clean seperation of implementation and interface, as well as the other OO goodies like polymorphism and encapsulation.  There are good refactoring tools for Smalltalk and Java, which is why they are often associated with XP.  There are also Python-based refactoring tools on the horizon as well. But you can do XP with C, C++, C#, Delphi, assember, whatever.

As for free(malloc(-1)), take a look at http://www.extremeprogramming.org/ and decide for yourself if it makes sense for your projects.  HA HA HA indeed.

joev
Wednesday, July 07, 2004

"The software is the design, not the product."

dot for this one
Wednesday, July 07, 2004

> Few are willing to do that.  It's a lot easier to say "you
> will provide this software implementing X, Y, and Z
> features by September 1"

But in your example, the customer didn't even get X, Y, and Z using the traditional methodology, did they?  They got X, half of Y, and half of Z (or something like that).  At least w/ agile, they got X, Y, and 3/4 of Z.

So what makes people cling to the old methodology then?  Not trying to criticize the example, just wondering...

Joe
Wednesday, July 07, 2004

So, from what I understand:


XP

- more man*hours than waterfall

- lower risks

- client gets a working product sooner, even if it doesn't have all the features


waterfall

- less man*hours than XP

- high risk of misunderstanding specification or of project getting somehow "stuck" because the initial design was bad, etc.

- client gets working product at the end of the development cycle

George
Wednesday, July 07, 2004

Building a car is like using a very detailed reusable framework, as most mass produces cars, basically use the same materials, parts, paint, tires, radios, frames.

As soon as you use something like the MFC, JET, ADO and third party libraries you are getting close. Most of the other stuff you need is a new database scheme and user interface. All the logic should be basically the same.  Be sure to save code snippets of routines.

Also most cars can be prototyped in CAD software and they will be only truely build a "beta" version if the computer model works.

somemorone
Wednesday, July 07, 2004

- high risk of misunderstanding specification or of project getting somehow "stuck" because the initial design was bad, etc.

This shouldn't be a real problem is you allow plugins to enhance functionality and seperate logic routines from each other (dll's) and from the GUI. If so a small patch should fix the problem.

somemorone
Wednesday, July 07, 2004

> waterfall
>
> - less man*hours than XP
>
> - high risk of misunderstanding specification or of project
> getting somehow "stuck" because the initial design was
> bad, etc.
>
> - client gets working product at the end of the
> development cycle

I'm still not sure where this blanket assessment is coming from that XP requires more man-hours.  Anybody have any real experience in comparing this?

Also, you are making an assumption that just because there was a spec, the final product will be feature-complete.  In reality, the waterfall method may also create a product that only meets 90% of the customers needs, because those needs changed partway through development and it was determined that 10% of them could not be reasonably factored into the current spec (because of budget, time, complexity, etc).

Joe
Wednesday, July 07, 2004

The best analogy to producing software that I have seen so far is making a movie.

Every product has some commonality (a hero, a villian, etc) yet every product is dramatically different from the ones that went before.

Planning the different pieces up-front is important, yet flexibility in implementation is necessary also.  You need control of many different pieces (actors, equipment, scripts). 

Movies come in large expensive productions, and small, low-cost indy versions.

But yes, there has never been a product made by man which really shares all the attributes of software.  Infinitely modifiable, it supports an incredible amount of complexity.  We're still deciding what language to use to create the beast -- and each language, language construct, and application domain changes dramatically what the final product looks like.

And 'Tangeable' literally means you can put your fingers on it.  A Code Listing is tangeable.  The Code, as it runs in the computer, is not -- and it is the running code and the products and interface it produces, that is the product.

AllanL5
Wednesday, July 07, 2004

http://alistair.cockburn.us/crystal/articles/cpanfocisd/characterizingpeopleasnonlinear.html

genius
Wednesday, July 07, 2004

Writing software is not like building cars because there is no mechanical manufacture step to writing software.  Programming *is* design.  The false separation of coding from design was conceived and perpetuated by people who don't understand creative work, because they haven't done any themselves.

And somehow swarms of us folk who actually write software bought that line of crap.  I'll save my theories about that for another day, but they aren't kind theories.

The essence of the power behind agile methods comes down to two really important truths.

(1) If you explore your design notions as they occur to you, you'll not go to far down blind alleys.  By working through it in an unforgiving medium, you'll notice most faults and foibles right away.  In the wishy-washy world of paperwork, you don't get solid feedback.  In short, you *learn* and adjust simultaneously.

BDUF sillinesses pretend that you know everything a priori.  Agile knows that's crap, and acutally leverages the fundamental truth that you'll *learn* new and important things as you progress in writing the software.

Imagine scribbling down the solution to a maze before you've looked down most of the corridors, and you'll have a useful mental picture.

(2) People talking to one another, face-to-face, beats the piss out of any other form of communication.  You get better fidelity, quicker transmission, richer interaction, less clumsiness, and more excitement than any other conveyance physically possible.

Spec-tossing wastes gobs of time and dehumanizes the creative people involved.  It drains your soul.

Of course, the agile methods have quite a few other powerful qualities, but in my experience, these are the Big 2.

Now.  Getting back to the original post...  Go back up and reread his "Method 2".  (Before continuing, I'll wait here for you to go read it and return.)

Ok.  A BMW M Roadster is pretty much the same thing as a Chevy Tahoe.  They're both the result of our original poster's Method 2, but not at BMW during the M design process, or at GM during the Tahoe design process.  The period over which Method 2 happened was the evolution of machinery that culminated in the automobile itself.

Method 2 IS ALMOST EXACTLY HOW THE CAR WAS INVENTED!

If your level of understanding of software lets you think the Mars Rover motion control software has as much in common with a telecom bill processing system as the M has with the Tahoe, then you'll probably be able fall for this whole design/code dichotomy rather easily.  The more you understand how close writing a new software system is to the invention of the automobile, the better equipped you are to understanding the situation, and knowing why approaches based on that false dichotomy are crap.

Cabby
Wednesday, July 07, 2004

George,

FYI, your subject line mentions one thing (objection to Extreme Programming) and your post talks about something else (various agile methodologies).

Anyway, while I haven't really studied many of the popular agile methodologies (SCRUM, FDD, Crystal, etc.), I have read quite about Extreme Programming (read books and articles and visited various forums) and in my opinion this methodology has a lot of problems with it. If you are interested in learning more about the current problems with this particular methodology you might want to do the following:

* Read a book titled "Extreme Programming Refactored The Case Against XP".

* Visit a website called softwarereality.com and read some of the articles that discuss Extreme Programming.

One Programmer's Opinion
Thursday, July 08, 2004

I find agile programming is more efficient, for a few reasons.

The projects I've worked on always have new requirements arriving on an ongoing basis. That forces me to keep modifying my code.

Also, I also have a working version of the code ready to ship, with the higher-priority tasks finished. That takes the pressure off and lets me include as much as possible in each release.

Even if I have the requirements in advance, agile methodologies are more productive for me. The rate-limiting step in coding is thinking, not typing. Adding features individually is faster and easier than figuring it all out in advance, even if more typing is necessary as I re-arrange my code. Besides, the abence of design docs probably reduces my total typing.

Julian
Thursday, July 08, 2004

George,

Why is software development slow?

Why do you need to rewrite large portions of existing code?
    
I'm sure free(malloc(-1)) is being sarcastic, but he highlights a very important point: what language are you using.

The key proponents of the Agile Methodologies usuallly come from the school of agile OO languages.  Kent Beck is a Small Talker, The Pragmattic Programmers are Rubyists.

These are agile languages that allow for rapid rewriting of code.
    
They also support strong OO, which allows for the easy growing of code.
    
For the first few iterations core components will need refactoring, but this will soon settle.  The processes of frequent iterations and rewrites 'softens' the code, making it more mallable.  Read what Ward Cunningham has to say here:

http://www.artima.com/intv/clay.html

Just applying the principles of frequent releases and regular refactoring without the other principle elements such as automated unit testing and a suitably agile languages will prove expensive and get you into trouble.

Regarding the expense, with big up front design you do end up with an application that satisfys the requirements.  As you say, however, it is extremely hostile to change.  As time goes on the application will become less and less relevant to the working environment.  The code is difficult and expensive to change so it will be left alone.  People will cope as it grows old and cranky until finally enough is enough and the whole thing is scrapped.  A replacement is written from scratch.
    
With an agile approach you finall application is far more flexible.  The constant additions and refactoring during development have forced the development of an extensible architecture that does not resist change.

As the code lives on, it grows further and adapts.  Features grow and over time it become more and more useful and relevant.

Remember that he average code spends most of it's life in support, not development. 

Ged Byrne
Thursday, July 08, 2004

so then...

which of these methodologies: (1) Traditonal (2) XP
would you recommend for: 

(a) Shrinkwrap (software that needs to be used by a large number of people)
(b) internal (software that has to work in one situation, i.e. one company)
(c) embedded
(d) games

(source: five worlds by Joel Spolsky http://www.joelonsoftware.com/news/20020507.html )

Throwaway
Thursday, July 08, 2004

Traditioinal waterfall spends a huge deal of money before there is any tangible result. This implies a huge risk. Furthermore, since the "ananlyses and design" is further away in time from the "delivery", it is increasing the risk again by increasing the chance of requirements drift.
Agile methods do not show any cost or risk reduction from a traditional approach in the "best case" scenario. They may even take a bit more cost. Where the difference comes is in the risk reduction. In my experience this far FAR outstrips the slightly higher cost in the VERY few cases where everything would have been okay either way.

Just me (Sir to you)
Thursday, July 08, 2004

Waterfall design suffers from a major flaw - the assumption that you can know/predict every eventuality. This, of course, is false.

What *really* happens in waterfall design is that huge assumptions are made. Each assumption relies on previous assumptions. You end up with guesswork tacked onto estimates based on outright lies - a house of cards.

As a developer, how do you answer the question "Which would work better in this case, a treeview or a wizard?" Nine times out of ten, if you're intellectually honest, the answer is "I don't know - let me fiddle with it for a while and I'll get back to you"

Most of the assumptions made in waterfall design are schroedingerish - they can only really be answered by performing the task. What this translates to is that you can't really present a reliable waterfall design/estimate until you've actually finished the project.

Oops.

The idea behind iterative design is that you plan milestones where each milestone delivers a working product. This gives your users (remember them? they're often forgotten in waterfall design) a chance to truly interact with it and give feedback on the design. It's better to find out that [x] basic assumption is wrong when all you've delivered is Edit and Print capabilities than when you've spent three years building Word.

Obviously you can't always do the iterative design thing - it works best in corporate solutions where you have captive guinea pigs, as opposed to shrinkwrap where people just want the software to work.

Philo

Philo
Thursday, July 08, 2004

Ok, I'll admit I only skimmed the previous posts, but I didn't distinctly see anyone point out the obvious fact that cars have been around a heck of a lot longer than software.  Cars are pretty much a solved problem, as far as I can tell, unless you are coming up with a brand spanking new type of engine, but overall automotive engineering is, well, nothing new.  It's a mature discipline.

Software engineering, on the other hand, is far from mature.  There are a lot of people still doing the equivalent of tinkering in their garages.  To be certain, there are places where people are refining it into a science.  It's almost like we're still arguing over which is better, AC or DC, or gasoline engines versus steam engines.  I wouldn't expect software engineering to even remotely resemble automotive engineering at this stage, if for that reason alone.  Nevermind the fact that apples are not oranges.

Aaron F Stanton
Thursday, July 08, 2004

Another intangible benefit is against featuritis.

With big projects, theres a habit for managers to sit around a table dreaming up cool features.  These guys will never use the system, and probably don't have a clue what is really needed.  They do, however, have lots of ideas they consider very desirable.

The managers come up with features a, b, c, d .... x, y and z.

With the waterfall method you have to implement all features, from a to z, before go live.  There are always those nasty features that eat up weeks of development time and are never used.

In an agile methodology a, b and c are implemented.  The c and d.  Keep going to n.  Then every body is happy with the system so when you say 'time for o and p' they say 'to be honest we really don't need those.  Just give us q and r and forget about the rest.

Oh, we have though of a1, b1 and c1.  These are features that we really would find useful.

Ged Byrne
Thursday, July 08, 2004

"Most of the assumptions made in waterfall design are schroedingerish " (philo)

"Schrodingerish" -- what an excellent word!  And it applies very well to the chicken-and-egg problem of software. 

You don't know if the cat is alive or dead until you open the box to find out -- the Schrodinger's Cat paradox.  The action you take to find out something actually changes what you find out.

AllanL5
Thursday, July 08, 2004

I liked the rock climbing anology; I'm surprised no one else has chimed in on it.  I constantly feel like the second rock climber with regards to a current project that is on a difficult path set long ago by a heavy investment in complex up front analysis and design.

If you're sole responsibility was only to code to a spec whose accuracy you had no accountability for would that be like top roping?

MacSqueeb
Thursday, July 08, 2004

First a quick defense of the Waterfall model: While it is true that the waterfall model originally had no iteration, the spiral model came along almost immediately afterward. It's basically Waterfall, but incorporates iteration through multiple cycles. VERY old idea.

Second: My understanding of XP leads me to think it would be a cost-effective method for certain small, ill-defined projects. These, however, are only a fraction of all the projects done in our industry. Different domains probably require different methodologies. So the initial comparison to building a car was probably a poor pick because of a domain-based impedance mismatch in the examples.

Third: I also think that to be effective, XP requires more engineering discipline than most other methods because it leaves much more room for flexibility. That's the tradeoff. Unfortunately, I fear lots of people on the XP/Agile bandwagon are simply there to avoid discipline.

Fourth: There isn't really anything special about software engineering compared to other engineering or manufacturing methodologies, other than the fact that software engineers too often prefer hacking to engineering. If you just equate compiling to manufacturing, the rest of the steps preceding them actually correspond pretty well.

Fifth: I'm bothered by the statement that software is "creative". Not that I don't think creativity is necessary, but in the implicit assumption that it is more like the arts than like engineering. To me this indicates a misunderstanding of both art and engineering.

Interesting discussion...

Jeff Kotula
Thursday, July 08, 2004

"Fifth: I'm bothered by the statement that software is "creative". Not that I don't think creativity is necessary, but in the implicit assumption that it is more like the arts than like engineering. To me this indicates a misunderstanding of both art and engineering."

Especially since engineering also is part creative.  Deciding what a piece of software will do, or how it will look should be creative and "arty", but implementation? Come on! "How will I walk this array? I know! I'll use a FOR loop!" Implementing a design should be a science.  If you have two choices on how to do something, the one you choose isn't based on "prettiness" of the code, it should be base on an analysis of performance and usability.

Steamrolla
Thursday, July 08, 2004

IMHO there is NO silver bullet. It really depends on the project. 

I personally, like Staged Delivery (aka Waterfall Model with Planned versions). I sometimes feel like XP and Agile methods lend itself to users not thinking at all about what they want. That may not be the methodologies fault, but the people running the project and allowing the user to walk all over the project.

What I like about Staged Delivery is that you say an overall goal. But, you say exactly what will be built next (ie tires). In the next version, the tires might be enhanced. But, you delivery in stages what was agreed.

For me I need that contract and agreement.

Patrick
Thursday, July 08, 2004

First of all, car design is incremental. It is cheaper this way, most probably because you have the chance to catch most problems before you increase complexity.

An exception is the space shuttle. This was a design from scratch, not a delta from an existing design (The R. Feynmann's report on the Challenger disaster).

Secondly, software design is different from all other types of product design: no two programs are alike. This is intrinsic to the fact that programs do not behave statistically nor deterministic (e.g. Turing's halting problem)

Agile methodologies are an attempt to producing software in a cost efficient way.

Dino
Thursday, July 08, 2004

In a talk like this, y'all ought to prefix your posts or the "Full name" field with "(full-time programmer)" or "(nonprogrammer)" so we all get a rough sense of whether your opinions on the act of making software are well-informed or not.

(full-time programmer) Jerry
Thursday, July 08, 2004

"The action you take to find out something actually changes what you find out."

No, that's heisenbergish.

As for XP only being for "small" projects - show me a "large" project and I'll show you a collection of small projects. ;-)

Philo

Philo
Thursday, July 08, 2004

Jeff,

Just picking up on a few of your points:

First:

The real problem with the Big methodologies, not just the Waterfall, is the devision between the designing and coding.  The designer wants to be like an architect, drawing diagrams but never laying bricks.  Ideally the programmer is an unskilled labourer who can be outsourced if necessary.  With Agile methodologies the programmer/designer is merged.  The Agile programmer does not start coding before they have a design, they use the code to express the design.  For example, instead of a Use Case diagram you have an Unit Test.  This is why tests must be written before the code, because creating the test is where the design work happens.

Second:
Large projects are actually in the minority, you just see more of them because they are so big.  There are countless number of small, internal projects being carried out.  Agile methods suits these projects.

Third:
Jeff, you are exactly right here.  Agile development requires stricter discipline not less.  Too many people use it as an excuse to sit down and churn out code like they did in school. 

Fourth:
There is on key difference between software engineering and physical engineering.  In software engineering there is nothing physical.  If you are building a bridge, you will require hundreds of workers to actuall construct the final product.  It is very, very expensive so you want to make sure that every little detail exactly right before any work begins. 

In manufacturing they have been using automation for the last few decades, but that is also a hard process.  Imagine if it was possible to consrtuct an automated production line in a day, how would this transform the manufacturing industry?

Fifth:
I totally agree with you.  I spend most of my time trying to eliminate the creativity in favour of robustness and reliability.  In my book creative = likely to fail.  Again, Agile Methodologies done right seek the same goal.  In XP the unit test is written first, the programmer is restricting themselves as much as possible before writing code.

Big Methodologies also fall into the trap of creativitiy, with the designer going power crazy and constructing some grand, impractical vision.

The other problem is that lack of control in the construction phase.  Programmers are months away from coming up with the goods, so they are free to unleash their corrupt imaginations.  With an Agile methodology you are restricted by the need to produce a working solution in a short timescale.

I often think that the arguments between Big and Agile methodologies are rather like the contrast between Communism and Facism.  They seem to be poles appart, yet beneath both extremes hide the same evils.

Ged Byrne
Friday, July 09, 2004

*  Recent Topics

*  Fog Creek Home