Fog Creek Software
Discussion Board




UI bugs

In another thread someone said:
[quote]
here is what happened...
  - Incentives for every bug closed.  The GUI presentation people loved this idea.  The A/R people hated it.  While GUI had more requests outstanding, the A/R peoples were far more time consuming to fix.[/quote]

Is this normal? Are (G)UI bugs more common than bugs in other parts of a system (and if so, why), or is it just that the UI is more visible (no pun intended) than those other parts and therefore bugs in the UI are more obvious?

I have a couple of ideas about this myself, but I don't want to poison the water so I will post again a little later when people have had time to think about it themselves.


Tuesday, April 01, 2003

As has been said a number of times, to the user the UI _is_ the application, its behaviour and presentation of that behaviour.

The underlying guts of whatever an application does is frequently logically simpler with simpler data states and logic paths than a UI which has to sift, select and validate all the possible combinations of user input into something which matches the user's desires and the application's requirements.

That's not true for all applications but its likely to be true for any application with a UI more complicated than a text box and a button that says 'OK'.

Simon Lucy
Tuesday, April 01, 2003

Simon - your answer is along the lines that I have been thinking.

Here are my thoughts, for what they are worth.

1) The interactions between all application features come together at the UI, and those interactions might well not be "designed", which results in "undefined behaviour". This is basically your logic paths comment, Simon. Nice to see that I am not the only one who believes this to be the case.

2) Testing GUIs is _hard_ (I found http://c2.com/cgi/wiki?GuiUnitTesting this morning, which seems to back this up). I don't mean things like the random mouse click test, I mean:

  a) Writing automatic unit tests is so difficult as to be almost impossible. JFCUnit looks like it might be useful, but since it is for Java & swing and I use C++ & MFC, not in my case. In fact the link I provided above seems to suggest that there is not really a complete answer to this problem, but I'm still reading.

  b) If the we believe 1, then the chances of imagining and (manually) testing a greater number of possible actions and interaction than in non-UI software is slim to none.

3) As a result of 2a, refactoring is fraught with danger (because one cannot automatically test those small steps).

4) Reliance on external libraries is almost obligatory, yet they are often not flexible enough to account for user requirements (because, for example, they try to force a particular model on the implementation and that model is flawed - yes, I am thinking of MFC almost-but-not-quite MVC here amongst other things).

So much for that. It doesn't help me with the next step though, which is what to do about it. Maybe I should just stop implementing UIs.


Tuesday, April 01, 2003

Brute-forcing every perm of operations to test completely is a daunting task.  Rearranging n operations and trying them in every combination is n!


Tuesday, April 01, 2003

As someone mentioned above, it's much harder to do automated testing of GUIs. You can do basic smoke tests but it's hard to get the same coverage you can get for a business logic componet with a set of unit tests. I've used httpunit extensively on a previous project and found that benefit/cost ratio for GUI testing was very low. I believe that this is because code interfaces tend to be more stable than UI.

The other reason UIs tend to have more bugs is that you wind up having spelling/grammar/wording issues that in code are caught by the compiler (sort of). Usability is also a more subjective area than software design. Everyone's got an opinion on how UI should look and flow.

Finally, I've found that a lot of managers think UI work is much easier than business functionality. So these managers will assign their best developers to the "back end" work and put less experienced people on UI work. The end result is that UI code winds up having more bugs.

igor
Tuesday, April 01, 2003

I think it's relatively easy to get a UI up and running and relatively hard to get all the little details right.  To some degree, this is true for all parts of an application, but particularly so for the UI, because there are so many visible details and so many interactions between the parts.

For example, you might hook some toolbar buttons to some actions in a few minutes, but then spend days getting the buttons to appear enabled or disabled at the appropriate times.  There are scores of situations like this.

What's more, the straightforward UI implementation generally doesn't result in the best experience for the user. I'm thinking "multi-threaded" here.

In my application, the UI accounts for half the 50000 lines of code (refactored continuously) and two thirds of the bugs.

Rob Daugherty
Tuesday, April 01, 2003

UI has something that the other levels of system don't: user expectation.

The user has no expectation of the interaction between two core system components, because they don't know they exist. They can be tied together in whatever way is most convenient for the engineer.

The UI, on the other hand, has the issue of user interaction. The user expects it to work a specific way, and when it doesn't, that's more than usually considered a bug. It could be that something needs to be changed, additional methods of interaction need to be developed, etc.

A polished, usable UI is almost always the most difficult part of a project -- triply so with the web (not just user interactions, but working around the technology, and testing with a variety of clients with sometimes vastly different capabilities and interpretations of the "rules").

Brad (dotnetguy.techieswithcats.com)
Tuesday, April 01, 2003

If you design your interfaces correctly, and apply something like the Observer pattern (or Model/View/Controller etc) you can almost completely automate the UI testing.

There may always be bugs in the actual UI components (widgets), but if you can ignore that fact as they will be picked up pretty quicksmart by users (and there's not a lot you can do about it besides code work arounds).

You can safely assume that the bulk of UI widgets will work. Eg. Textbox, Combobox, Listbox, ListView etc. Most of the bugs have been found and the known ones have (in most cases) documented work arounds.

Geoff Bennett
Wednesday, April 02, 2003

Geoff - regarding your first paragraph: Post Your Proof! (Seriously, if you can provide something like CPPUnit for testing UIs then I think you could make a bucket of money.)


Wednesday, April 02, 2003

There are various products to automate GUI testing - e.g. WinRunner.

punter
Wednesday, April 02, 2003

Those tools are not for automatic unit tests, they are for system testing.


Wednesday, April 02, 2003

And most importantly, they do not test the UI against the user's expectations.

Practical Geezer
Wednesday, April 02, 2003

Geezer - nothing does that. And the expectations seem to change depending on the phase of the moon anyway.


Wednesday, April 02, 2003

It's important to differentiate unit testing -- the attempt to validate that the code is working as expected with no bugs -- vs. acceptance testing -- the human part where someone looks at it and says "move this 3 pixels to the left, make this come before that, etc.".

You can't test the latter automatically.

Brad (dotnetguy.techieswithcats.com)
Wednesday, April 02, 2003

Well, waaay back in the day I used a product SQA TeamTest(?) or some part of that family that would allow the automated testing of GUI screens.

i.e. you could have a test script for a screen to enter data into the various controls and press the buttons, twidle the knobs, basically fully exercise the screen.

My impression was that the SQA set of products was bought and repackaged as WinRunner / LoadRunner, etc.

So, whatever it's called now, you could certainly use it for unit testing - i.e. check that all fields have the correct behaviour on invalid input, etc.

punter
Thursday, April 03, 2003

That sort of "exercizing the controls" testing isn't really GUI testing, or at least it's only a part of it. (Or if you like, it's UI testing, but not GUI testing.)

GUI bugs tend to be the sort where label text doesn't fit, perhaps only in some languages, or maybe depending on the system font settings. Or the programmer decided to be creative and mixed checkboxes and radio buttons in an option group (or some similarly absurd misuse of controls--don't laugh, you know you've seen these). Or he neglected to run spellcheck. Or there are their/there or our/out type mistakes that even spellcheck can't catch. Or the more subtle consistency and alignment problems, where some screens have controls x pixels from the edge, while others leave kx room; or buttons are y pixels wide, except for that one OK button on that one screen.

Lots of GUI bugs are the result of the programmer failing to pay enough attention to the nitpicky details. But he shouldn't have to, right? Nitpicky details are supposed to be an ideal task for a computer, yet this seems to be impossible to program. It comes down to 'spellcheck will never be perfect', which is rather hard to accept when you've met the umpteenth stupid to/too mistake.

Martha
Thursday, April 03, 2003

"Geoff - regarding your first paragraph: Post Your Proof! (Seriously, if you can provide something like CPPUnit for testing UIs then I think you could make a bucket of money.)"

Best example I can come up with at short notice: http://www.codeproject.com/dotnet/tdd_in_dotnet.asp

Scroll down the bottom of that article, and there's a little explanation about it. I'm not talking about excersising controls, or whether the contents of label captions fit. I'm talking about the code that handles grabbing input from UI controls and jamming it into business objects.

Most people seem to shove that into the event handlers. If you have designed a halfway decent interface to your business object, you could create a view class that encapsulates all this and makes it unit testable.

Like I said, you can't use this technique to check the "G" part of GUI, but you can at least use it to test the app from as close as you can get. You can simulate bad data right from the highest point in the chain.

We've started using this around here now and while I can't say there's a tremendous productivity increase at this stage, with clear requirements you can definately see an improvement in code. Especially when you merge stuff back into the mainline.

Geoff Bennett
Friday, April 04, 2003

Thanks Geoff. Actually that looks pretty similar to CPPUnit and that kind of thing is in principle exactly what I am after. My experience with CPPUnit agrees with what you said - these things are not so good for testing the G part of the GUI which is what I am interested in. I can test many of the individual methods in a class (those that do not directly affect the presentation for example) but that is not the same.


Friday, April 04, 2003

*  Recent Topics

*  Fog Creek Home