|
Another critique of Joel's talk in Davis...
I attended Joels talk and felt so strongly about one of his points that I was compelled to write a critique of some kind. I urge those that disagree with my thoughts to enlighten me or show where I missed the mark. You can find the critique at www.foggymyst.com
Steve
Saturday, January 31, 2004
If it ain't broke don't fix it. Regd your "windows USB screen" thought, the ROI for MS would have been 0, since for 95% of the folks using MS there is no alternative.
Read Joel's essay on Bicultarism.
The Marketing bit, reminded me of the database called Ingres (I think) which was about 10 times better than the early versions of Oracle. While Ingres was busy fixing bugs Oracle was busy marketing....we all know who won!
Prakash S
Saturday, January 31, 2004
You seem rather judgmental, letting your emotions reject information a bit too quickly. What if he said that the calculation of whether a bug is "worth fixing" includes taking into account a backlash? And of course there are the bugs that early programmers realized were intrinsically difficult to solve.
A macho notion of bugs assumes that if there's an error in your program, you put it there. But that completely ignores buggy compilers, platforms and machines. Programmers (read: computer users) build upon sand. Even the innocuous + operator in many languages doesn't do what many people think: Gray/Reuter in the "TP bible" tried writing a perfect 1-liner add() function, and noted their first version had two bugs! They cite in 1992, US spaceshuttle software cost ~$5000 per line of code, and pilots were still handed a buglist to work around. No one has more time and money than the US government, they argue.
Businesses are economic entities. They play by a different set of rules than "real" programmers. Spending too many resources on something that isn't worthwhile, means less resources toward what users truly want. Going out of business or having insufficient resources are violent acts against your customers. Of course, this brings many people to conclude that the business world isn't cut out for them.
[source: Gray, Reuter _Transaction Processing_, ch 3.6 "Software is the Problem". Also see Bentley _Programming Pearls_ column 4 "Writing Correct Programs" for statistics on the difficulty of writing what seems to be a simple binary search.]
Tayssir John Gabbour
Saturday, January 31, 2004
If fixing the $90 bug brings in $300 of revenue, you can surely use some of it to fix the $110 bug.
Saturday, January 31, 2004
That's a good rant. Of course Joel is right. Geeks do way too little marketing and in commercial software one should not fix bugs if the fixing costs are higher than the bug costs. But it would have been wise if he immediately mentioned afterwards that even small bugs and annoyances can be incredibly expensive. Bugs are negative marketing.
In the same speech he talks about bugs/annoyances in Windows and the ipod. How many geeks will now rethink their possible ipod purchase, because it does not come with replaceable batteries? How much will it cost MS having a famous ex-employee indicating windows flaws? How many of the listeners will retell his anecdotes? You just did it in your rant. I mention it again here. We are looking at a pyramid scheme of bad marketing. That $ 0.001 the bug costs each time it shows its ugly head or gets mentioned, quickly adds up.
So I am not saying that for the best ROI one should fix all bugs, just that the costs of bugs are generally grossly underestimated.
And Joel knows it, as he says in his own bug article "indeed, we fixed every known bug in FogBUGZ, not just the big bang ones". So someone is not following his own preachings.
Maybe he just wants to lower the quality of the competition by telling them that bugs are OK, really ;)
Jan Derk
Saturday, January 31, 2004
Hmm, I hope that didn't come out sounding like an attack. Entirely possible I could be more judgmental than you. But being overly judgmental is bad for both of us, isn't it? ;)
Tayssir John Gabbour
Saturday, January 31, 2004
I would presume that all the bugs were fixed in Fogbugz because they all satisfied the criteria of 'it would be more expensive not to fix it and it won't break anything else'.
The second point is just as important. I've seen engineering/development departments change some innocuous behaviour in software that had such ripple effects that it destabilised the whole product. Or sometimes they made changes for 'good technical reasons' but without any demand to make them and ended up with a product that did less than it did before.
Those that have seen version 4 of a product somehow do less than version 3 will recognise that kind of behaviour.
Simon Lucy
Saturday, January 31, 2004
Read this:
http://headblender.com/joe/blog/archives/microsoft/001280.html
chris
Saturday, January 31, 2004
Steve, where you really missed the mark is your assumption about the impact of the bug on the users.
The impact of the bug is going to depend on the kind of product, your users and the nature of the bug itself.
1) Kind of product: Some products are so "vital" and/or "useful" that even fairly serious bugs aren't going to make the users consider switching. Maybe switching is hard, or maybe there aren't alternatives. The users might whine and moan, but they don't switch.
One example is Intuit's QuickBooks - a program that has many "quirks", which range from incredibly slow searching of invoices once you have any significant volume to incorrectly displaying reports under some non-obvious circumstances. Switching accounting systems is a lot of work, especially for the target audience of that software. Many times, users don't even know whether the problems are with the software or something they "did wrong". Which leads into 2)
2) Users: Before we embark on any project, we first complete a full-blown user needs analysis including user interviews. On one of our recent projects, the (aging) vertical software package in use routinely corrupted the data. This information was then imported into their accounting package (Simply Accounting) - causing all kinds of headaches. The users didn't trust the accuracy of the data (no wonder, it had to be sent for "repair" once every two weeks) - and many created their own spreadsheets of information which they used to compare to the results of the application. Yet, these users felt most of the problems they encountered were a result of not understanding how to use it properly. If only they could figure out the magic key-combinations, there wouldn't be any problems.
This attitude is actually more common than you might think. The opposite attitude is communicated more frequently and more loudly, but it is not the only user attitude.
3) The nature of the bug. If the bug in question is obscure, or is not well understood to be a bug by your users, whether you fix it or not likely won't impact your users' perception of the software. Obviously, there are those display only "problems" and even the odd trivial "feature" that your users are going to complain long and loudly about - these are bugs you should fix as soon as possible. But the bugs that your users don't care at all about are generally much less urgent.
Regular Poster
Saturday, January 31, 2004
The most depressing thing about this is that, apparantly, merely being a competent professional and actually wanting to produce a high quality product that's easy to use and works well seems to be a bad thing because it will reduce a company's profits.
Maybe it's time to take up an honest trade, if there are any left.
Saturday, January 31, 2004
I tend to agree that the cost of bugs is underestimated, from the customer perspective. What happens is that companies forget who's making the decisions.
Let's take as an example the idea of a database package which has a record entry bug. Sometimes, when you enter a record -- about one in every five hundred -- it will mysteriously disappear and not end up in the database. Nobody knows why.
You estimate that it will take two weeks of searching the code just to find the bug, let alone fix it. You estimate that fixing the bug will not get you any more software sales, because nobody finds the bug until after they buy the software, and they usually work around it somehow. Should you fix the bug?
There's a missing variable in there. By fixing the bug, you save all of your customers a certain amount of time and money. Consider a company that enters 250,000 records a day. Without the bug, they wouldn't need to identify and reenter 501 missing records a day. If it takes 30 seconds to enter a record, that's over four hours a day that they lose because your software has a bug.
For you, the company producing the software, this bug is not worth fixing. But for the customer, the bug clearly *is* worth fixing. The question is, should you demand money from your customers because the software is important to them, or just go ahead and fix the bug at your own expense because the customers are important to you?
Not an easy question.
Caliban Tiresias Darklock
Saturday, January 31, 2004
The cost of bugs is prone to underestimation, because you don't know how many people did NOT buy your product after hearing about or observing the problems.
T. Norman
Saturday, January 31, 2004
T.Norman-- you stated the point of my entire rant in one clear sentence. Nice job.
Steve
Saturday, January 31, 2004
Let's take the opposite view for a moment. Joe Sixpack coder works at our software company, "LittleBucks". LittleBucks sells a product at 300USD per license and hopes users upgrade at 150USD each year. LittleBucks currently has sales of 2000 licenses, and an annual budget for salaries of 500,000 USD (~7 programmers), leaving a marketing and support budget of 100,000 USD.
Joe Sixpack, while coding a module which is expected to increase sales by 10% by allowing a new category of users to use the product, discovers a bug. While calling an obscure (read little used) key command the product may freeze for 30 seconds and not perform the action. A quick search of some software forums finds that several users have noted this and are complaining about it.
Joe Sixpack decides to fix the bug immediately, if users are complaining that must be bad. So he starts coding. It takes him a month to find the bug and fix it.
In the mean time the new version of LittleBucks program ships. What was the cost of finding and fixing that bug?
Opportunity cost of 10% of 2000users = 60000USD
But we kept 5% of existing customers for upgrade = 15000
Note that the cost of the programmer's time is not factored in here, it's actually irrellevant from a business perspective as it is a sunk cost either way.
If that 5% were dramatically higher, or our upgrade cost were higher, or our opportunity cost was lower, we should probably investigate the bug and fix it. But with these numbers, a manager will be quite upset, we missed out on 45000USD, that's nearly 10% of our annual sales.
One needs to clearly define the opportunity cost (the profit we lose by doing something else) for fixing a bug. It's inexact, and bad marketing does compound. But that's not for Joe Sixpack to determine, he needs to go to his boss and ask if it's worth it - present some information, and do his assigned job so the company can succeed and give him a nice holiday bonus.
Lou
Sunday, February 1, 2004
We've ranted on two threads about this to arrive at this gem of knowledge.
Q: Should the bug be fixed?
A: It depends.
Crusty Admin
Sunday, February 1, 2004
This thread makes me think of the Five Worlds article. I think that a shrinkwrap bug has much more potential to snowball into bad marketing than corporate software has. IMO.
Shodan
Sunday, February 1, 2004
Joel might have mentioned what he does to mitigate problems. For example, I don't remember seeing a shrinkwrap company with its finger closer to the pulse of its customers. (Kind of weird, I'm sure there must be one.) Its ez 2 use forums allow them to see users going nuts. Presumably someone interested in forking over the price but angry at a bug would still have enough energy to post on the forum or send an email. A reasonable percent at least.
If joel is to be believed, fogcreek put enough effort into bugfree install programs and basic functionality that users who perceive bugs will be invested enough to complain loudly. so iamboringmyselftypingthisshit whydoicarewhattheydo
Tayssir John Gabbour
Sunday, February 1, 2004
"We've ranted on two threads about this to arrive at this gem of knowledge.
Q: Should the bug be fixed?
A: It depends. "
Yep, and still most of the oh so intelligent nerds (cfr.PG) can't grasp something this simple.
Just me (Sir to you)
Monday, February 2, 2004
As someone who actually makes a living selling shrinkwrap software, I am rather interested in the "It depends" part. The "How expensive is this bug" question is an issue I have to deal with on a daily basis, as it directly influences the amount of money I make.
In my humble opinion Joel's bug article too easily provides the impression that bugs are OK. I would have preferred him adding something about bug costs. And what factors make up that cost.
I always, apparently wrongly, assumed that the JoelOnSoftware forum was meant for discussions about software development. I hereby offer my sincere apologies if this isn't the case or if no one is allowed to be critical about one of Joel's articles.
Jan Derk
Monday, February 2, 2004
One thing that hasn't been mentioned is the risk associated with fixing a bug. You may introduce a more serious bug while fixing the first bug. You can't certain that your analysis of the bug was correct.
There is a non-zero risk of adding a more serious bug. If you early in a development cycle where there will be lots of testing to find the problem, it might be worth correcting. If the bug is found the day before the final code freeze, the risks go up substantially.
pdq
Monday, February 2, 2004
 Recent Topics
Fog Creek Home
| 
