Fog Creek Software
Discussion Board




Measurement of Joel's Preaching

Hello,

I was wondering if Joel could show us (his avid readers) if his ideas are working.  By his ideas I mean that by giving programmers perfect working conditions and hiring only the best should result in excellent quality software.

As a way of seeing if it works one could measure the ratio between bugs generated and LOC count.  And if the schedules set forth in the beginning of the project are a long way from the resulting product.  If anyone has more ideas on how to measure the quality of software development, please reply to this thread.

It would be very interesting to see if Joel's ideas on software management are actually working.  I hope he wouldn't be giving up too sensitive business details if he gives us some feedback on the success of his ideas.

Best regards,

Anonymous

Anonymous
Monday, November 12, 2001

Anonymous wrote:

> If anyone has more ideas on how to measure the quality of software development, please reply to this thread.

Quality: the extent to which product meets user/customer requirements.

Measure quality: measure requirements (e.g. count the number of items in a list of all requirements); test (e.g. run) product to see which/how many requirements it meets, and how well it meets them.

Christopher Wells
Monday, November 12, 2001

<quote>
By his ideas I mean that by giving programmers perfect working conditions and hiring only the best should result in excellent quality software.
</quote>

You shouldn't take this too literally.  If you ask a CEO of any relatively small company, they will tell you that they only hire the best people.

Some facts:

- 99.99% of the best people do not work for you

- Having a team of elite people is not always enough to make a company succeed

--
Cedric

Cedric
Monday, November 12, 2001

<You shouldn't take this too literally. If you ask a CEO of any relatively small company, they will tell you that they only hire the best people.>

And I am sure they all did hire the best people - that they could find.

Robert Moir
Monday, November 12, 2001

C. Wells said

> Quality: the extent to which product meets user/customer > requirements.

> Measure quality: measure requirements (e.g. count the
> number of items in a list of all requirements); test (e.g.
> run) product to see which/how many requirements it
> meets, and how well it meets them.

I don't think this is the whole truth.
First The requirements could be bad. If you have bad requirements, the measurement above is absolutely
useless.
(As we know from Nielsens "Usability Engineering",
often users do not know how much they would like a feature before they have seen it. Nor do they know what they would need or what they could need. So it is very hard
to get the requirements right in the first place.)

Second: A program that meets its requirements 100% and is unmaintanable spaghetti code is in my opinion worse than a program which meets requirements 95% but is well
factored and maintanable.

It is not so easy to judge the quality of a piece of software.

What is easier is to judge the quality of a company that develops software. My measurement
would be a measurement of how much value they build.
I'd propose the (sum of the earnings of all employees plus earnings of the company) divided by the number of employees.

Andreas Wicker
Monday, November 12, 2001

Andreas,

Thank you for your feedback.

> First The requirements could be bad. [...]

That's true.

In another discussion on this subject elsewhere, someone recently added that this process is iterative.

A user might see a feature, decide that they don't want it (so, the written requirement are changed).

Or, the requirements may be badly expressed (need clarification). Or the requirement may not be a requirement at all (it might instead be a bad guess of what a *design* or solution should be ... a design, looking for a requirement or customer).

Nevertheless: I'm stating as axiomatic a mission to "meet requirements"; I'm not sure how to 'measure requirements' except in writing; perhaps the equation continues to be true or useful, at any given moment (e.g. a project is 100% finished-quality, when it meets 100% of the requirements; on a next day, if 50% of the requirements change, then the project/product is now 50% finished-quality).

> A program that meets its requirements 100% and is unmaintanable spaghetti code is in my opinion worse than a program which meets requirements 95% but is well
factored and maintanable.

Well, true, that is, unless, the 100% perfect but unmaintainable speghetti code is to be burned into ROM for an embedded system and forgotten.

Still, the question re. 'unmaintainability' is only an 'exception which tests the rule': I could reply that one of your 'customers' or 'users' is whoever it is that owns it, and that (if maintainability is a requirement), this person could easily pencil-in "Maintainable (must!!)" on the list of all (measurable) requirements.

> What is easier is to judge the quality of a company that develops software.

There are so many ways or criteria from which a person can choose, by which to judge them! <tossing a coin>

> My measurement would be a measurement of how much value they build. I'd propose the (sum of the earnings of all employees plus earnings of the company) divided by the number of employees.

That would be the "value equals price" theory (definition of 'value').

Christopher

Christopher Wells
Monday, November 12, 2001

> That would be the "value equals price" theory (definition of 'value').

My mistake: more like the "value equals monetary earnings (possibly 'profit')" definition of 'value'.

Christopher Wells
Monday, November 12, 2001

> My mistake: more like the "value equals monetary earnings (possibly 'profit')" definition of 'value'.

Which sounds like an appropriate way to measure the quality of software development, if the only 'requirement' is "maximize profit".

One requirement, with one corresponding metric, looks a bit simplistic.

I'm not sure whether it's part of requirements-gathering or part of the design (I suspect the latter), to define subrequirements of each requirement: for example, maximize earnings, minimize cost, and minimize the number of employees; and then measure implementation against each subrequirement; and perhaps somehow measure the extent to which each subrequirement or design component (e.g. 'minimize number of employees') is successfully a part of the overall major requirement (e.g. 'maximize profit').

Christopher Wells
Monday, November 12, 2001

I think this misses the point of Joel's preaching.  Joel has been kind enough tell us his strategy and why he chose it. And then, he executes that strategy in public for us to see.

Joel's passive FTP incident in CityDesk is a great example.  Joel not only told about the tools he used, he also told us why he chose them.  He didn't hide when he hit a brick wall caused by an omission his tool set.

Terry Kearns
Tuesday, November 13, 2001

I agree with Terry.

Prakash S
Tuesday, November 13, 2001

I think the best "measurement" of Joel's techniques is the fact that FogCreek stays in business! Wasn't it was Joel himself who said that "average" performance for a small startup is bankruptcy... So, as long as this site is still up, and as long as Joel and co. are still making a living doing the things they love to do, I'd consider it pretty good proof that his strategies are successful =).

Dan Maas
Tuesday, November 13, 2001

Re. Dan: Do they really make a living with FogCreek?
(I presume, as Joel worked for MS, he's got enough
cash to live without FogCreek)
So the test if Joel's "preaching" works is just:
Do they earn considerably more than other companies?

Re. Christopher:
You wrote:
> perhaps the equation continues to be true or useful, at any given moment (e.g. a project is 100% finished-quality, when it meets 100% of the requirements; on a next day, if 50% of the requirements change, then the project/product is now 50% finished-quality).

I consider such a formula, that allows to change the quality from one day to the next so much, as totally useless.
Of course, if requirements change 50% from one day to the other, they were garbage.

So I'd state as axiomatic: You cannot judge the quality of a software (or anything, by the way) without judging the requirements.

Andreas Wicker
Wednesday, November 14, 2001

*  Recent Topics

*  Fog Creek Home