The Old Joel on Software Forum: Part 3 (of 5) - CMM

CMM - How relevant is NASA example?

Most CMM enthusiasts quote the example of NASA's space shuttle software to prove how CMM helps improve quality. Did NASA have the time-to-market pressures or budgetory constraints similar to a commercial software product? I think it is more relevant if they talk about a well known comercial software product that is drastically better than the competition _and_ while turning out such a great product the team had a healthy work-life balance. All because they adopted CMM!!!(I don't know if one can come across such an example)

Plover
Tuesday, October 28, 2003

You don't think NASA is a high pressure environment or has budgetary constraints?

You should've seen the interviews I saw after the Challenger tragedy about how they borrowed parts from anywhere they could find them to help save costs.

Everyone seems to think that the other company or industry really has their stuff together and that they're the only ones facing problems. It just ain't true.

www.MarkTAW.com
Wednesday, October 29, 2003

In any case, arguing from a specific example to a general assumption isn't valid logic.

www.MarkTAW.com
Wednesday, October 29, 2003

NASA's procedures and quality existed independently of any CMM rating and were not dependent on it.

They existed that way because they have enormous budgets to spend on their software, which translates into lots of time and programmers, compared with most businesses.

Anyone adducing the NASA experience as a benefit delivered by CMM is distorting the situation.

Wednesday, October 29, 2003

"In any case, arguing from a specific example to a general assumption isn't valid logic."

Why? Isn't that the base for induction?

Induction: "The act or process of reasoning from a part to a whole, from particulars to generals, or from the individual to the universal; also, the result or inference so reached."

http://www.hyperdictionary.com/dictionary/induction

uncronopio
Wednesday, October 29, 2003

This is an article about the space shuttle software that's been posted here before:
http://www.fastcompany.com/online/06/writestuff.html

John Topley (www.johntopley.com)
Wednesday, October 29, 2003

Perhaps inductive logic works like this:

1) John is American
2) Americans are hard-working
3) Therefore John is hard-working

Invalid inductive non-logic would be like this:

1) John is American
2) John is hard-working
3) Therefore Americans are hard-working

Christopher Wells
Wednesday, October 29, 2003

No, NASA's example isn't valid. For the Shuttle flight software, every single *line* of changed code had to be pre-approved and later reviewed by about eight different people in total. The process for proposing & implementing a change in the software was equally thorough/onerous.

But has this process worked? In general yes, as there have been very few in-flight software failures.

But is it practical in a business environment? No (except of course for certain life- and property-safety applications, of course). Go ask your boss to hire 8 more code reviewers/managers, plus the administrative support to manage all the paperwork, and see what the response is.

Mark Newman
Wednesday, October 29, 2003

"Perhaps inductive logic works like this"

I believe that you are wrong. Inductive reason is, as was stating, going from a particular truth to a universal assumption - John is hardwork and American, therefore all Americans are hardworking.

Deductive reasoning is going from a universal truth to a specific example - Americans like green beans, therefore John likes green beans.

I could be wrong though.

Dennis Forbes
Wednesday, October 29, 2003

Deductive logic:
1. Americans are hard-working
2. John is an American
3. John is hard-working

Inductive logic:
1. John is hard-working.
2. Everyone who works with a hard-working person will become a hard-working person themselves.
3. Everyone in John's office will eventually be hard-working.

We now return you to your regularly scheduled NASA/CMM discussion, already in progress.

Wednesday, October 29, 2003

As I recall, a valid deductive argument is an argument in which the conclusion necessarily must be true if all the premises are true (a sound argument is one in which the premises are true). Note that validity is with respect to the structure of the argument, not the content.

An inductive argument is valid where the conclusion is probably true if the premises are true (it is implied, in the non technical sense of the word). This can often be the case where moving from the specific to general, but it is by no means always the case.

God, this sounds pedantic!

Mongo
Wednesday, October 29, 2003

Yes, but being pedantic can be fun :)

I think Mongo has it right.

My copy of "A concise introduction to logic" states:

"A deductive argument is an argument in which the premises are claimed to support the conclusion in such a way that if they are assumed true, it is _impossible_ for the conclusion to be false."

and

"...an inductive argument is an argument in which the premises are claimed to support the conclusion in such a way that if they are assumed true, then based on that assumption it is _improbable_ that the conclusion is false (that is, it is _probable_ that the conclusion is _true_)"

There are lots of examples of inductive reasoning, including "an inductive generalization", which is arguing from knowledge of a selected sample to some claim about the whole group.

This is a perfectly valid logical argument technique - an example is inferring that results from a small statistical sample apply to the larger population - a survey or poll, for example.

Mike Treit
Wednesday, October 29, 2003

The reason I've never put much credit in NASA's CMM as a viable point of comparison is their platform.

They code for three identical CPU's that they have full hardware and revision control over.

If I was coding for that kind of platform, I'd consider "zero bugs" an attainable goal as well.

If you tried to apply NASA's processes and goals to shrinkwrap, you would have to have several dozen if not hundreds of QA machines for all the possible combinations of crap that you might encounter, and you'd have to use all of Oracle or Microsoft (or both) as your QA department.

I'm not saying QA isn't important or that CMM can't add quality. I just think NASA is not a valid example.

Philo

Philo
Wednesday, October 29, 2003

http://www.satisfice.com/articles/cmm.htm

Jim S.
Wednesday, October 29, 2003

I remember reading an article about it, and it was written that the cost per line of code for the development in CMM 5 for NASA was around $5000 (or was it $2000?).

Google was no help to find the source, does anyone can confirm it ?

R Chevallier
Thursday, October 30, 2003