The Old Joel on Software Forum: Part 4 (of 5)

Automated testing of UIs?

If I may ask another "testing" related question (and thanks for the replies on unit testing - I think I'm getting the drift now, I can see some areas in my apps where it could be applied, which leads me to ...)

What strategies/tools exist out the for building automated tests of the UI portions of an app? Preferable inexpensive or free! ;-)

In a particular app I work on, we would dearly love to be able to have some standard tests we could automatically run before each monthly update to validate core UI functionality still behaves as it should.

Aussie Poster #7
Wednesday, May 12, 2004

only simulate input to the ui.
Take screenshots
test the data

The Artist Formerly Known as Prince
Wednesday, May 12, 2004

We work in an XP kind of environment, so automated testing plays a big part of our work. Our tests are run as part of our build.

Our tests are written in the Fitnesse (www.fitenesse.org) framework, which allows you to create Fit tests in a wiki. Its a bit under-documented but works well once you get going.

Rhys Keepence
Wednesday, May 12, 2004

That should be http://www.fitnesse.org

Rhys Keepence
Wednesday, May 12, 2004

try this group:

http://groups.yahoo.com/group/TestFirstUserInterfaces/

Giovanni Corriga
Wednesday, May 12, 2004

Automated testing of UIs is a bit like coverage testing (getting a report of all the paths of execution of a process). It tells you where all the big holes are but it doesn't tell you that what isn't a hole is truly right.

I guess in the kind of applications you're talking about the business rules significantly affect the expected behaviour of the user interface.

I haven't found a good automated way of improving on getting real world context with real world users. Hence, if you have the time to do it use beta testing.

Simon Lucy
Wednesday, May 12, 2004

Check out the freeware package AutoIt.

It sends keystrokes to Windows and comes with a well documented DLL. I've used it several times to test various parts of the UI with pretty good results.

I think it works best for data-driven tests, like a user registration, where you want to test the program with all kinds of different data to make sure you can't brake it. You would specify the procedure (Open file, tab three times, enter the first name, tab, enter last name, etc etc) and loop it. Each loop throw in some random characters into each field and see what happens. It works well for this because you can loop it 10,000 times and just watch it and if there aren't many problems you can probably get that done in a day.

The biggest downside is that you have to keep an eye on it, there's no way to use the program to detect what's going on with your app and if your app crashes AutoIt will keep sending keystrokes to the screen.

Shane Harter
Wednesday, May 12, 2004

Silk Test -

http://www.segue.com/products/functional-regressional-testing/silktest.asp

genx'er
Wednesday, May 12, 2004

Not free, but TestComplete is supposed to be one of the better testing tools out there. It includes support for "automated functional testing", which may be most useful for GUI's: http://www.automatedqa.com/products/tc.asp

Herbert Sitz
Wednesday, May 12, 2004

If you're in java, we've written a small handy framework that extends Swing's capabilities to allow easy GUI-based tests: http://haste.sourceforge.net

We use this internally on some large projects with great success.

Mike Swieton
Wednesday, May 12, 2004

oh yeah,
for web based ui's http unit is pretty good too

The Artist Formerly Known as Prince
Wednesday, May 12, 2004