Fog Creek Software
Discussion Board




Looking for JOS Projected Bug Count Comment

i've got a manager who wants us to predict bug counts by this formula:

  projected bugs = .046 * number of lines of code changed

after i commented that this was plain silly, my co-worker summed it up best: not all lines of code are created equal.

anyhow, as i recall, joel made a comment in an article about this once.  i've been searching around for the article, but can't find it.  anybody know what it was?

Scott
Thursday, August 26, 2004

1/42 would be a better indicator.


Thursday, August 26, 2004

I don't know.  It might be a useful heuristic (sp?).  I mean, perhaps its totally bogus, but perhaps not.  One way to find out is to test the theory.

Without knowing for sure, but just based on my experience, I'd guess that bug count is probably linearly proportional with LOC changed (or created - I think they should be counted similarly if not the same).

I think your boss's mutlple is far too high.  That would be a number that you'd get when developers did test their code at all (a bug per every 20 LOC?).

Since you mention it... (reading old code...)  One really painful bug I remember was in a module that consisted of 400 LOC.  Since that module was nicely contained, and the bug was isolated to that example, for my sample of one, 1/400.

If I took more time, I could add up more bugs & LOC.  If I had an LOC counter, and access to old bug logs, that would make for an interesting experiment.

hoser
Thursday, August 26, 2004

errr... that should be developers that do NOT test their code.

hoser
Thursday, August 26, 2004



I think it depends entirely on how coupled the various classes/modules are.

If they're all highly dependent on one another where one error can have a cascading effect, then error checking/handling may be required throughout.  This would tend to increase that factor.

If they're all loosely couple and (as hoser mentioned) errors are isolated into their own areas, then this might be high.


Regardless, unless you have historical data to back it up, it's pretty arbitrary.

Be careful about citing a number though as it could easily be said: "well, this group's CER (code-error-rate) is half of your group's... what's going on?"

KC
Thursday, August 26, 2004

I like the comment "not all lines of code are created equal".  That is very true.  Also remember that not all bugs are created equal.  Some are 5 minute fixes where it was just a lapse in concentration that caused the error.  Others can take a day to fix. 

Historic data is probably the best way to try and guage how long testing will take.

You also need to take into account the skill of the programmer and how complex each part of the program is.

Steven
Thursday, August 26, 2004

*gauge

The article you are looking for is:
http://www.fogcreek.com/FogBUGZ/WhyFogBUGZWorks.html

About the fifth paragraph down.

Steven
Thursday, August 26, 2004

The problem with this 0.046 thing is that you need at least 20 lines of code (changed code) in order to have 1 bug. That's stupid.


Thursday, August 26, 2004

In response to above comment -

Or it means that one should only make small changes (19 lines or less ;-) and then run your unit tests.


Friday, August 27, 2004

I'm still "young" as a programmer, but it seems to me that knowing the number of projected bugs is, itself, pointless because, just as not all lines of code are equal, not all bugs are equal.

So I know when we get to QA we're going to have approximately 143 bugs to address.  Are they bugs that will take two minutes to find and fix, or two days?  Moreover, out of those 143 projected bugs, which ones will we *need* to fix and which ones can we let slide?

I guess that's why the OP started out explaining that this is a requirement being imposed by his boss.

OffMyMeds
Friday, August 27, 2004

I just remembered that this was about predicting bugs in code before making *changes*, not before creating new code.  In that case, I guess the metric is slightly more useful than it would be in the case of new code, perhaps to asses the risk vs. payoff before deciding to make changes.  Still seems a little weak in meaning though.

OffMyMeds
Friday, August 27, 2004

*  Recent Topics

*  Fog Creek Home