Fog Creek Software
Discussion Board




What to do on comments from users?

What is a good policy to deal with usability and bug comments from users? Should you issue the generic postcard response ('your issue is appreciated. we will look into it when it falls out of the round file.')? Post the issues publically so others can see them ('me too')? Keep it totally quiet? Discuss future plans? (We plan to fix the top 5 most reported usability issues?) Claim perfection? ('this software has no bugs or usability problems, never mind what you just saw and the hair you just ripped out'?)

This seems to be a damned if you do or damned if you don't issue; if you're quiet, but produce regular and good updates, people will be happy. If you have full and honest discussion, people will be happy but the trolls will come. In both cases, expectations may be raised too high (releases every 6 weeks means a 3 month lag is unbearable; response to some but not all comments angers and frustrates people).

I haven't had to deal directly with a large audience for software bugs before, so I can't really say what's worked for me. What plans have worked for you, general forum people? What about Fog Creek, which promotes both good usability, sells bug tracking/customer issue tracking software, and runs message boards?

mb
Thursday, February 27, 2003

If you give users an interface that lets them describe their problem, or their suggestion easily, without making them jump through so many hoops they give it up in disgust, and they can see the status of that issue and others, then users will appreciate it and you will get higher quality reports.

Bugzilla would not be a good choice for a non-developer audience.  FogBugz might well be. 

The combination of collecting data silently (but in a known and approved way), and a context free descriptive form I think would be the best solution.

Simon Lucy
Thursday, February 27, 2003

It is usually difficult to deal with bug reporters. Some of them are professional: they send you a clean report with all the steps to reproduce a bug. But the problem is with the wide majority of people that "believe" they found a bug.
If you are too open to users' feedback, average users will see bugs everywhere. Some of them will even credit each Windows misbehavior to your software although it is unrelated.
My advice, after more than 2 years of software support, is to never make public the bug reports. Try as much as possible to restrict their submission. For example, let them require a subscription to a betatesting program. Besides, let each user feel that its feedback was important for you: reply by a standardized email customized with their name. But never give the information about the ETA of a new release.

dtr
Thursday, February 27, 2003

Ah. The difference between a reported bug and a bug... or in ITIL terms, the difference between an incident, a problem, and a known error.

An incident is a case where 'the product does not appear to be behaving as I expect it should'

A problem, which may be identified from one or more related incidents is a case where 'there really is a problem, the root cause of which is unknown'

A known error, which may be the root cause of one or more problems is a case where 'a failing CI (part of the system) has been identified'

So, Only the reporter should be able to see incidents that he has submitted (for tracking).

I would suggest that definitely known errors, and probably problems, should be publicly available.

And how is an error fixed? Why, via. a (company internal) change request of course.

Oh, and if the users want enhancements then these are change requests too and the user community should probably have visibility to the requests that other users have raised, as they can add weight to a particular requirement which may help you focus your development effort.

'Proper' incident/problem/error/change management cannot be trivialised to 'bug tracking' and nor is it easy to try and manage it all via a lightweight bug tracking tool, e.g. FogBugz; you inevitably end up with customisation and compromise which has a cost associated with it.

Gwyn
Thursday, February 27, 2003

I would only add that if you have a support website (and IMHO you should), then have a FAQ/KB section where you document the major bugs - either "this is by design, here's how you deal with it" or "this is a known bug, here's the workaround" or "this is a known bug"

KEEP IT UPDATED

this kind of thing keeps your users confident that you're listening to them and paying attention to your product. This encourages them to spend money on you...

Philo

Philip Janus
Thursday, February 27, 2003

I think it depends on what your product is.  Where I work we sell a java library, so all of our users are programmers.

We have email and phone support that you must have support contract to use.  We do not send out a form letter.  A real person works with the customer to resolve their problem.

If it is not a problem, we update documentation and FAQs so we don't get the problem again.

If it is a bug, we fix it and release a patch.  We typically release a new version quarterly and release patches more frequently depending on how serious/how many bugs we have.

If it is a new feature, we tell them we will think about it.  If more than 1 customer asks for a feature it will bubble its way up our future feature list.  When we decided to implement a requested feature we typically try to contact the customers who requested it.

This type of setup may not work for shrink wrap/non-developer customers, but for our customers and us it has worked really well.

Adam
Thursday, February 27, 2003

I already hate your product. :-)

I buy your product and install it. I find a bug. Nothing in the FAQ (it's a bug, and my impression was that you don't put bugs in the FAQ), I can't call support because I didn't pay for it. How do I find out if it's a bug or a feature? Do you know about it? When will it be fixed?

Whether I get a refund or not, I can't see buying from that company again...

Philo

Philip Janus
Thursday, February 27, 2003

Phillip,

You make some good points.  Somethings I did not mention
1. We have a 30 day trail period.  This includes support.
2. First year support is included with purchase.

Would the above allow you to condier doing business with us? ;)

Adam
Thursday, February 27, 2003

We put all the errors we know about onto our website, immediately. Immediately may mean later that week if the error is a small one.

Why do this? New customers know *exactly* what they're getting, and can see that there's a pattern of things being fixed in a timely manner. This makes for fewer support calls, less refunds and happier customers.

One company I know of and no longer deal with
has a page saying "'The know issues lists archive issues which have been reported" on their website. In actual fact this is really a list of what got fixed in the /last/ version of their software, not the refund-worthy problems in the current release!

Thats the way the money goes
Thursday, February 27, 2003

To address the original question --

I have found that for every given possible improvement in our product, we hear about it from dozens of customers, each of whom feels that they have just discovered America.

I can't tell you how many emails I've gotten suggesting that CityDesk have a feature to insert tables. OK, message received!

Given that I hear this every other day, but the customer who has taken the time to tell me about it thinks it's quite new and important, there is a fundamental imbalance there, and it becomes hard for me to say graciously, "thank you for your 7 page letter explaining how tables should be implemented."

Another risk, albeit minor, is that someone will suggest a feature that was obvious and that you already knew about, and then try to sue you when you implement it. So you have to be very clear in replying to people that you already know about that idea. (99% of the time, you do.)

Joel Spolsky
Thursday, February 27, 2003

Offtopic, but it's amazing to think that individuals would suggest a feature and then sue you when you implemented it.  Absolutely amazing.

Konrad
Thursday, February 27, 2003

Weird people with funny delusions of grandeur have been known to propose various semi-obvious ideas that they think they invented to large companies, and then spend the rest of their lives suing the large company, usually acting as their own attorney, in various inconvenient and ridiculous jurisdictions. It got bad enough that a lot of large companies simply refuse to listen to customers' product ideas in order to protect themselves. (Car companies are famous for this, after they finally lost the lawsuit by the guy who claims to have invented intermittent windshield wipers.)

IBM for example has a policy that if you want to present a new product idea to IBM, you have to present it first to a retired ex-IBM employee. Only if that ex-employee judges it valuable somebody inside the company will take a look.

Joel Spolsky
Thursday, February 27, 2003

Joel,

What's Microsoft's policy on this?

Prakash S
Friday, February 28, 2003

If you had a list of 'upcoming features', you might not hear about it from so many customers. But is that over-promising? Or leaking info to your competitors?

Open source projects are often cool, so long as they're below critical mass and have an active mailing list and archive. Sometimes you can propose a solution when you report a problem. And the to do list is often public and in fact a request for outside help.

But for closed source teams, it seems there are two responses here: keep the 'wish list' up to date and public, and hope for 'me too' responses. And send a form reply, 'your message has been received and will be considered'. Between the two lies danger and reality.


Prakash:
Microsoft has a defined 'mswish' program. I don't know exactly how it works, but it ideas do get disseminated internally. They also have relationships with consultants and good customers (e.g. large corps) which provide feedback.

MSWish: http://register.microsoft.com/mswish/suggestion.asp . First line: "By offering suggestions through this page, you give Microsoft full permission to use them freely."

mb
Friday, February 28, 2003

Adam,

Like Philo I am also very wary of software that seems to generate a substantial percentage of its revenue from "support" or "consultancy" by the company that publishes it. If you simply have a revenue stream from selling the software, it s absolutely clear that making your product better is good for you in every respect. Trow generous servings support and consultancy into the mix and suddenly the brew gets more murky.
"Sure we might loose some sales by not directly fixing this installer issue, but it is more than compensated for by the additional S&C revenue. Let's just wait a few weeks and cash in on the calls in the mean time."
It is a dangerous play, but it is clear your optimal revenue point no longer coincides with your optimal product point.

Just me (Sir to you)
Friday, February 28, 2003

A couple of things ...

I sent some ideas to that Microsoft Wish site, and as far as I can tell, the suggestions vanished into a black hole. I'd love to know if any developers actually read them.

Also, since Joel mentioned the folks asking for table handling in CityDesk, what other enhancements might we be seeing in the next release?

Dana Hoffman
Monday, March 03, 2003

*  Recent Topics

*  Fog Creek Home