Fog Creek Software
Discussion Board




XP and Bug Databases (from Five Worlds article)

"Last week Kent Beck made a claim that you don't really need bug tracking databases when you're doing Extreme Programming, because the combination of pair programming (with persistent code review) and test driven development (guaranteeing 100% code coverage of the automated tests) means you hardly ever have bugs."

Sadly, you didn't provide a reference to that claim. The one I know of coming closest is a posting in news:comp.software.extreme-programming from 26. April 2002 under the subject "Is XP Suitable Under these Conditions?".

There he isn't making any general claims but simply reporting from a project:

>>>This is the XP project that couldn't. It contains all the elements doubters
and supporters point to as being inimical to XP:
  * Embedded system
  * Customer hardware development (a cool 2048x1 color camera, FPGAs for
low-level processing)
  * Life critical (well, finger or arm critical anyway)
  * Multi-site (including a business merger)
  * Disengaged customer

And no bug database because they have essentially no bugs.<<<

"Lo and behold, I discovered that very few of the bugs in there would have been discovered with pair programming or test driven development. Many of our "bugs" are really what XP calls stories -- basically, just feature requests. We're using the bug tracking system as a way of remembering, prioritizing, and managing all the little improvements and big features we want to implement."

So you are basically saying that you are using a bug database to manage feature requests. Did you ever try the XP way of managing them?

Ilja Preuß
Tuesday, May 07, 2002

Isn't a feature request a bug too? Does it matter if it's called a bug or a story or a pain in the left toe - it still means the same.

If i want to do something with a product that to my mind i ought to be able to do, then isn't that a bug? What if not being able to do it has a *serious* impact on my work flow?

What about if I download the demo and upon finding it doesn't do what i want, and i contact my salesdroid and explain i won't be placing that big $$$$$$$$$ order until it does, doesn't that make it a bug?

Robert Moir
Tuesday, May 07, 2002

Unfortunately, I don't follow Kent's claims so closely, neither I read all new XP books. To my outdated understanding (got from Kent's book "XP eXplained"), XP teams are supposed to write test for each reported bug. This requirement could easily be extended to feature requests. It seems to me, this way test base becomes a sort of bug tracking system. 

Igor Bazarny
Tuesday, May 07, 2002

Igor, you are correct.

In fact, XP requires the Customer to define automated acceptance tests for every feature she wants to see in the system.

Ilja Preuß
Tuesday, May 07, 2002

Robert,

"Isn't a feature request a bug too? Does it matter if it's called a bug or a story or a pain in the left toe - it still means the same."

Well, I would differentiate the two by the implied hole in the development process:

A bug happens when the development team didn't develop what they thought they developed.

A badly missing feature means that the team didn't analyze the requirements of the users correctly.

YMMV.

"What about if I download the demo and upon finding it doesn't do what i want, and i contact my salesdroid and explain i won't be placing that big $$$$$$$$$ order until it does, doesn't that make it a bug?"

For me, it makes it a feature request with big business value. Of course you might call it a bug. Why would you need a database to organize it?

Ilja Preuß
Tuesday, May 07, 2002

My first impression of Joel's essay is that he simply proved Kent's point - he doesn't need Bug Tracking software, he needs Story Tracking software - and really, with only 106 stories sounds like he could use index cards.

Brett McHargue
Tuesday, May 07, 2002

Thanks for the thoughtful reply Ilja,

I think where you say "YMMV" you sum it up perfectly. There isn't  a "hard" right or wrong answer to this sort of thing, it's what works for you.

For me, tracking bugs and feature requests together does make sense because they are both feedback from testing, they are both changes to currently functioning code (well this might be stretched depending on what the feature request is), and if you have a good process for tracking bugs in place, and you do consider missing features to be bugs  then why *not* use the bug tracking process to track feature requests as well?

The issue is whether or not you see feature requests as being the same, or nearly the same, as bugs, which is again down to the workplace culture, again not a right or wrong answer thing.

Robert Moir
Tuesday, May 07, 2002

Robret,

With loving respect, I suspect you've gone off the deep end here. You're not to blame, because "bug" is such a fuzzy word that it can infect any conversation with all sorts of poisonous assumptions.

I prefer to speak of "defects". A defect is an error in your program;  it is much like a typo in writing, as when I write 'Robret' above instead of 'Robert' - you might have spotted the typo and felt a little annoyed with it.

A defect usually causes (and is usually spotted due to) a failure in the software system under consideration. If you felt annoyed at the typo, this parallels how users feel when they experience a failure.

A "feature" is something different - it actually adds to the semantic content of the system under consideration. In most cases this also has the effect of adding to the system's usefulness, though not always - this is one of the reasons XP recommends "customer tests", because they make sure all semantic content contributes to usefulness.

Yes, these precise distinctions *are* down to workplace culture. But, as people smarter than me have observed, culture is an important factor in whether we are successful at shipping software or not. Workplaces where the culture favors such precision write software faster for less.

Cheers,

Laurent Bossavit
Tuesday, May 07, 2002

Hi Robert!

"if you have a good process for tracking bugs in place, and you do consider missing features to be bugs then why *not* use the bug tracking process to track feature requests as well?"

On the other hand, why not use the feature tracking process (that is, index cards) for bugs too? ;-)

BTW, I think bugs that have major impact on code quality should be fixed rather immediatly, so that they shouldn't be handled by a long term tracking process (that is what I think Ron Jeffries means when he says you shouldn't plan to have so many bugs that you need a database for tracking them... ;-)

Thanks for the interesting discussion!

Regards, Ilja

Ilja Preuß
Tuesday, May 07, 2002

Hi Laurent,

so, is "does not work under Polish Windows" a defect, or "has to work under Polish Windows" a feature?

How do you determine the difference and what impact does the decision have on development?

Curious, Ilja

Ilja Preuß
Tuesday, May 07, 2002

"On the other hand, why not use the feature tracking process (that is, index cards) for bugs too? ;-)"

Oh it absolutely stretches the other way too but I think from the smiley you knew that :)

I think having the one way of tracking this sort of thing, be it index cards, a fancy bug database like mozilla or fogbugz, or a whiteboard everyone on the project can see is more important than which method you choose.

From my point of view, looking at the work flow, having just one system - assuming it's at least basically usable for all your needs - is more important than what the system actually is.

Robert Moir
Tuesday, May 07, 2002


I believe that XP simply is not the panacea. Also, I find the zealotry of XP followers a very dangerous thing.

Somebody (sorry, didn't remember the name) is talking about index cards instead of bug databases. I'm kind of stunned, because databases were invented to replace index cards, among other things.

I really would like to know how a XP guru would have applied XP to the development process of a shrinkwrap product like Citydesk. Given the fact that we know a lot of the process (Joel has talked about it a lot) probably somebody could teach something that I don't know, because, frankly, I believe XP is foobar.

Regards,

Leonardo Herrera
Tuesday, May 07, 2002

Ilja: I have written an article on this and posted it to the XP mailing list. Your feedback would be gratefully appreciated.

The other folks here will have to look wait until it's published elsewere, *or* subscribe to the XP list. (Hard choice, I know. Life is tough.)

Robert: "Somebody (sorry, didn't remember the name) is talking about index cards instead of bug databases. I'm kind of stunned, because databases were invented to replace index cards, among other things."

I'm one of these crazy people. I would agree with you that databases were indeed invented to replace index cards. And Palm Pilots were invented to replace paper notebooks. For some strange reason, I find that index cards work better in some situations, and paper notebooks also work a lot better in some situations.

For me, given my own skills and limitations, software management happens to be one of the situations where index cards and paper notebooks carry the day. Your mileage may vary.

I ought to mention, too, that nowadays I ship software with a defect rate five times lower than I was used to before I learned about XP and seriously looked into it.

Leonardo: "I find the zealotry of XP followers a very dangerous thing."

Yes, zealotry is a dangerous thing, and you are right to denounce it as such.

Did you have some specific XP followers in mind ? Or are you saying that *all* XP followers are zealots ? (Zat would strike me az a zomewhat zealous thing to zay.)

I'm an XP practitioner, and I like to thing of myself as a reasonable enough person not deserving the label "zealot". You be the judge, though. What impression did I make on you ?

"I really would like to know how a XP guru would have applied XP to the development process of a shrinkwrap product like Citydesk."

I'm not an XP guru, but I would have written user stories on index cards, estimated them, written unit tests, refactored... The whole shtick.

The XP process works for me. If you have a process that works, I'm interested in learning about it. If you're interested in learning about mine, we can always talk.

Cheers,

Laurent Bossavit
Tuesday, May 07, 2002

"If you're interested in learning about mine, we can always talk."

I didn't mean that the way it came out. Even if you're *not* interested in learning about "my" process, we can talk.

I want to learn from others more than I need to inflict my knowledge on others. (Although, like a lot of people, I do in fact succumb to the temptation to inflict my knowledge on others from time to time.)

Laurent Bossavit
Tuesday, May 07, 2002

>>Somebody (sorry, didn't remember the name) is talking about index cards instead of bug databases. I'm kind of stunned, because databases were invented to replace index cards, among other things.<<

I'm pretty sure that the "index cards" comment was a joke, and moreover, was a comment on the paucity of bugs in Fog Creek's bug database, not an Extreme Programming practice.

Also, there's nothing inherently wrong with index cards.  A lot of people still use physical Rolodexes.

Brent P. Newhall
Tuesday, May 07, 2002

"I'm pretty sure that the "index cards" comment was a joke"

Actually, I am quite sure it wasn't.

Shaun Smith
Tuesday, May 07, 2002

Many XP shops put their stories written on index cards (whether those stories are "bugs" or "features") on a whiteboard or wall of their team programming room.

For a small team, the number of stories to be done in one two-week iteration could be around 10 or 20, so that the are easily managed as index cards. The other 100 or so stories to be done are kept by the customer to be looked at during Release Planning and Iteration Planning meetings.

Since a story implementation is not considered complete until it passes all of its acceptance tests (which the customer has specified when describing the requirements), any bugs that slip through indicate a lack of testing or thinking, which provides feedback that testing or thinking needs to be improved.
Since a story is by defition not complete until it passes all of its tests, there are few bugs to keep track of.

Also, since the Customer is on-site, he can observe the program in operation, and indicate things that need to be changed (like spelling mistakes in the GUI) -- the programmers can fix those things faster than it would take someone to write up a bug report, put it into a bug database and route it to someone, have that someone read a bug database, and so on.

C. Keith Ray
Tuesday, May 07, 2002

By the way -- when I wrote "last week Kent Beck said..." I didn't provide a link because he actually said it to me in person.

Joel Spolsky
Tuesday, May 07, 2002

>>Many XP shops put their stories written on index cards (whether those stories are "bugs" or "features") on a whiteboard or wall of their team programming room.<<

Wow, cool.  I had no idea.

Brent P. Newhall
Tuesday, May 07, 2002

>>>Also, since the Customer is on-site, he can observe the program in operation, and indicate things that need to be changed (like spelling mistakes in the GUI) -- the programmers can fix those things faster than it would take someone to write up a bug report, put it into a bug database and route it to someone, have that someone read a bug database, and so on. <<<

If I write tests for everything I think can break, and don't consider that Swedish Windows does something different than English Windows why would I even consider writing a test? Never mind the customer having an acceptance test, never mind an automated acceptance test.

And what does one do with the report from the "wild"?

I postulate that you handle it like any other story. The Customer decides when if ever to put cycles into it. If it makes sense, write a story. But in the meantime, you need a place for it to be tracked, and I think it helps the Customer as well to have a database of stories.

Daniel Berlinger
Tuesday, May 07, 2002

>>>
"I'm pretty sure that the "index cards" comment was a joke"

Actually, I am quite sure it wasn't.
<<<

Yes, it wasn't.

Ron Jeffries recently posted a very interesting comment at usenet. I hope he doesn't mind that I found it interesting enough to conserve it at http://www.wikiservice.at/dse/wiki.cgi?KarteiKarten/Experiment ...

Ilja Preuß
Tuesday, May 07, 2002

Uh. Most of the questions brought up here could be answered by simply going to barnes and noble and reading through some of the XP books...

stating the obvious
Tuesday, May 07, 2002

*  Recent Topics

*  Fog Creek Home