|
CMM Level 5 - does it mean anything anymore?
http://www.cio.com/archive/030104/cmm.html
T. Norman
Friday, March 12, 2004
Well... By rights, it never did.
It's really a flawed system, from the most basic levels. It assumes that there's one proper way to engineer quality software that works.
And, heck, that doesn't even work for the fully-engineering fields of engineering. There's the military-and-big-contractor way to build aircraft, with documentation and everything. Then there's the Skunk Works in their heyday, who would put out equally good (if not better) designs with a much more lightweight process that resulted in better stuff faster. There's Scaled Composites now, same deal.
The problem is, it's Cargo Cult process. By doing this and that and shaking the talisman at the right points, quality software is produced. It's just as easy to fake CMM5 for a few weeks while the inspector's there as it is to fake happieness when you've got an unwanted house guest.
I'm predicting that 10 years down the road, CMM Level 5 produce the same sort of snickers that Ada, Fourth Generation Languages, PL/1, and others produce today.
Flamebait Sr.
Friday, March 12, 2004
10 years down the road? How about *already*?
veal
Friday, March 12, 2004
When I look at a company, I try to make sure that the letters CMM aren't in their methodolgy vocabulary. If I hear CMM, I run the other way (snickering).
Elephant
Friday, March 12, 2004
I worked for a company that had a single division -- a location with about 300 developers - assessed at Level 3. (Like the article said, most places only have a division assessed but speak of their rating as if it applied to the whole company).
I went through rounds of CMM training and read the 300+ page handbook. We had an SQA team that audited all projects for process compliance before software was released. From my experience, in my opinion I have to say that in large organizations the CMM is quite useful -- if actually followed.
For example, Level 2 includes (but isn't limited to) Requirements Management and Software Configuration Management. Most decent software organizations would have a good handle on those.
Level 3 includes Peer Reviews (code reviews for programs, document reviews for designs and requirements documents) and having a training program. Again, sensible practices that are hard to disagree with. But like the third normal form, I don't think it is useful to go beyond CMM level 3.
However, the SEI is so slack with the people who do the assessment and the people doing the assessment are so slack when they do it. The SEI is also very slack with allowing companies to make unqualified declarations of their CMM level (there is no accessible list of every company that was assessed and the result, scope or date of their assessment) and there is no need to be reassessed.
All that slackness means that it is unlikely that anybody at level 3 or higher is actually following the CMM at that level. Getting your own company assessed might be useful for gauging where your own company stands internally, like having my brother time me with a stopwatch is useful for a personal judgement of how fast I can run a mile. But you wouldn't put money on my ability to run a mile unless you timed me yourself or you read of my performances in strictly officiated races.
T. Norman
Friday, March 12, 2004
Here's a story of a software disaster involving an Indian offshorer:
http://www.nwc.com/shared/article/printFullArticle.jhtml?articleID=15201900
And, no, CMM-5 doesn't mean much unless you've got business toads running your business.
JM
Friday, March 12, 2004
One of the unspoken truths about CMM and ISO type efforts is that the organizations that tend to pursue such things tend to be truly, deeply, astonishingly screwed up at the start. So it's not all that surprising when they make some modest gains, which they and their CMM or ISO brothers-in-arms will declare to be sensational. When you're face-in-the-mud, you only have one direction available: up.
If you haven't enjoyed the pleasure, tee-ball is a sport featuring very tiny baseball players, absurdly young ones to be specific, who hit a stationary -- rather than tossed -- baseball off a post called the tee. Now a really awful tee-ball team might take some modest benefit in batting advice from a pimply 15-year-old skateboarder not talented enough for his high school's baseball team. Yet from this we can make no positive predictions about whether same fourteenibopper might improve the batting of Sammy Sosa or Barry Bonds.
I'll suppose I could end here and merely leave as an exercise delivery of the moral of that metaphor back to software. But I won't. I'll toss in yet another aspect I like to describe with metaphor.
Driving a racecar requires insight, stamina, sharp reflexes, and courage. Bumper cars at the amusement park are more forgiving of horrid ability, primarily because they travel so much slower, and to some extent owing to their abusable construction. Likewise CMM style process efforts tend to slow things down to the speed of the participant's sluggish wit, thereby reducing the velocity of mistakes. But to try to put David Coulthard in a bumper car at Monaco would be a travesty.
veal
Friday, March 12, 2004
"It assumes that there's one proper way to engineer quality software that works."
I'm not certain that this is really correct. CMM is about building processes, not software. The goal is to build processes in such a way that they can be systematically improved over time. When applied intelligently and correctly, there are some good learnings to be had. I think a lot of the trouble comes from incorrect/incomplete understanding of what CMM tries to do (on my part as well no doubt) and when the advice it offers applies to a given situation. In and of itself, I don't think it's a bad thing. What's bad is all the hype, misunderstanding, and so on that surrounds it. Some of these issues have been addressed in the newer CMMi work, but as long as it holds buzzword status, it's going to be difficult to separate the wheat from the chaff.
perturbed
Friday, March 12, 2004
I tend to agree with Peopleware that "process improvement" initiatives carry with them a sort of Heisenberg effect. That is, companies that drive to improve their CMM level, tend towards taking on projects that help them maintain it. It's just basic politics: if CMM is so important to a company, then noone wants to take on risky projects that might endanger it.
indeed
Friday, March 12, 2004
CMM 5 isn't about any particular methodology or process. It's about having a meta-meta-process. You don't just have a process. And you don't just stop at having a process that codifies and measures the process. You have a process that measures the process that is measuring the process.
By the way, I have CMM level 9 certification so I know what I am talking about. I defy anyone to prove that I am not level 9 certified. Oh, you say there is no level 9? Well it is so advanced that it is kept secret.
Anyway, is Microsoft CMM certified? Is Joel? Is Borland? Is a single development shop that produces well known quality innovative software certified? No? Well then what does that suggest to you about having a meta meta process?
Now every single two bit backwater joint in India is CMM 5 certified.
And retards like the CIO of that insurance firm mentioned in the article won't accept anything but the finest firms that are fully CMM 5 certified. What an absolute wanker.
If he wants results, he should hire someone who is level 9 certified. We are the only people who know anything at all.
Dennis Atkins
Saturday, March 13, 2004
CMM is something for those bullshit oriented data processing shops.
... and for those fuckheads that judge other people
by whether they have a degree or not ;-)
On the other end, networking shops generally don't bother about CMM, they have enough technical specs and standards to bother about.
Michael Moser
Saturday, March 13, 2004
In explanation of my previous rant -
Data processing is much about politics. You really don't have to know SUCH A LOT in order to create a application with a form interface + SQL database backend.
(VB, Delphi etc)
- the remaining time that you don't have to spend programming is therefore better spent with politics.
(and you would have guessed that people would bother about having a live ?)
Michael Moser
Saturday, March 13, 2004
"Data processing" ??
Wow. Haven't heard that one for a while. DP-MIS/IS-IT. In my experience, those were each decades. DP was a much better term than MIS. The BPR phase/craze at least removed the friggin' "M" from MIS. (yes, in know, BPR is still truckin'. Keeps lot's of us employed )
CMM certification, along with other development processes/methodologies, may be most important in the coming liability crakdown on the technology industry.
Not CMM certified? Didn't use Six Sigma ? Not even XP? Guilty as sued.
fool for python
Sunday, March 14, 2004
...yes, I know...
fool for python
Sunday, March 14, 2004
That's actually one of the concerns about standard methodolgies such as CMM-5 - that they will come to be seen as the "correct" way to develop software, and that anyone not using them will thus be vulnerable if sued.
I don't have the URL's, but several good people have discussed this in more detail.
So, a load of crap becomes the defacto standard, and those who don't use it suddenly become the "cowboys," when it's actually the other way around.
JM
Sunday, March 14, 2004
JM et al,
You have to appreciate the provenance of CMM and ISO 9001/TickIT (which is the software profile for ISO). They come from management's desire to achieve a degree of reliability in software construction that comes anywhere near to the level of design and manufacturing quality that is required in the fields of civil, automotive, aerospace, chemical or any other field of engineering.
Business Managers want reliable software. They don't know squat about how its written and, generally, they don't want to learn, anymore than they want to know or to learn how their car works. But what business managers do think they understand is how to manage processes. Software development is just another process in their eyes. So it must succumb to general principles of process and quality management. That's all that CMM / ISO are about, trying to control processes so that you can see how they work and were they're going wrong.
Now I really don't think that most businesses give a brass monkey about exactly what process software developers use, but they are beginning to get mightily hacked off about software quality and reliability. If the industry doesn't get its act together soon, then my estimation is that product liability will be stuffed down our throats. If you are a development shop there is an increasingly strong business argument to adopt CMM or ISO simply as due diligence.
On a slightly different tack - but still quality related - I'm waiting for the first big, European law suite against one of the major vendors (probably Microsoft) concerning security and the cost of failures. In the UK it is certainly arguable that the limitations on liability in most software licences are void under the Unfair Contract Terms Act. Under this legislation they must be 'fair and reasonable' in the circumstances of the contract, and a there is plenty of precedent to show that the relative bargaining strengths of two parties is a factor in determining 'fair and reasonable'. When you're up against a virtual monopoly that means the little guy is going to get the benefit of the doubt.
David Roper
Sunday, March 14, 2004
> If you are a development shop there is an increasingly strong business argument to adopt CMM or ISO simply as due diligence.
That would be fine if CMM certified shops were known to produce higher quality software but every study done so far shows that software developed by CMM-certified shops has a greater rate of defects than non-CMM certified shops.
So a reasonable thing from a product liability view would be to penalize shops that were CMM certified since this seems to be counterproductive against software quality.
I'd love to hear YOUR method that ensures high quality software since you seem to believe that software is of lower quality than civil engineering projects.
Truth be Known
Sunday, March 14, 2004
David Roper, I'm passionately interested in high quality software myself.
That's why I don't believe Business Managers (TM) should be in charge of software, if they don't understand it.
Without realising it, you've put your finger on the fraud that's in CMM-5. It's a mechanism for incompetent people to pretend they understand something they don't.
Rather than admiring CMM, we should legislate to ensure only real software professionals are in charge of software projects.
JM
Sunday, March 14, 2004
The real problem is not CMM per se; it's the idiots who claim CMM while not actually following it.
And the companies that go as far as Level 5 aren't interested in actually improving their software; they are only interested using the Level 5 for marketing to CIOs who don't do due diligence.
T. Norman
Sunday, March 14, 2004
I'd say whether they're performed correctly or not makes little difference. Not only does CMM or ISO have almost no chance of helping organizations that are likely to ever do well, but as my second metaphor above suggests they can be harmful too, if only by wasting time and attention on pointless gyrations.
When my petrol tank runs bone dry, changing my tires sets me back even further.
veal
Monday, March 15, 2004
If you get CMM (or ISO9001) just to put it on your letterhead, then it will show in your product and you deserve everything you get.
On the other hand, if you are committed to quality and you use these models as tools to help improve, then you will reap the benefits.
If I was Microsoft, I wouldn't want to be hostage to an assessor or assessment institute. These things break down for really big players.
Woodentongue
Monday, March 15, 2004
CMM was a research step in understanding what factors lead to well developed software. There were clear correlations between good requirements management and good software, good XZY processes and good results.
Then it became a business... The goal became getting CMM-3 certification, where the original goal was "repeatable quality" and the CMM-3 gave you well researched clues on ways to achieve that. Not everything applies to every business, but it provided some well grounded data on what has worked before (and what hasn't).
The main problem with CMM (and most methodologies) is when people forget the problem/goal the methodology is supposed to help with, and instead the goal ends up being conformance to the methodology itself.
The point of the discipline is to achive certain goals, not to just become more disciplined.
Chris Kessel
Monday, March 15, 2004
I never laugh so hard when I call company in India and they tell me they CMM Level 5. OOOhhhh I laugh and laugh. After I hang up, they call me back and asking me why I'm laughing and then I start laughing again. Ooooohhhh, I cant stop. CMM Level 5 too funny.
sudafed
Monday, March 15, 2004
Truth Be Known,
If I KNEW how to guarantee the type of performance seen in the aerospace industry I would be making a fortune, but I don't, so I'm not. I do know that's there's no way on God's Earth that I'd get on an aircraft that is as unreliable as much software, and I don't just mean expensive civil aircraft, but small GA 'planes that don't cost much more than a decent size database.
The key differences are (i) much more aggressive product liability, (ii) a process of certification that means that both designs and prototypes are rigorously and verifiably tested, (iii) a more cautious and conservative approach to development (new is NOT better until proven) and (iv) a history of learning from mistakes and NOT repeating them (due to point (i), which calls making the same mistake twice negligence).
JM,
Exactly. Business managers are simply just covering their backs by demanding CMM or ISO compliance.
David Roper
Monday, March 15, 2004
 Recent Topics
Fog Creek Home
| 
