Fog Creek Software
Discussion Board




Recommendations for GUI testing tools?

Does anybody have experience with GUI automated testing tools? I'm going to take advantage of the CityDesk beta to develop an automated test suite (so we don't keep breaking 2 things for every 1 thing we fix!)

If you have any recommendations, post them here. It needs to support Windows apps and be the sort of thing that doesn't involve a salesperson visiting your office with a $6000 projector for the demo (in other words, not too expensive.)

Joel Spolsky
Tuesday, October 16, 2001


The AutomatedQA products seem to be pretty popular:
http://www.turbopower.com/products/sleuthqa/

And for $399 they cannot afford to send you the salesperson.

Jan Derk
Wednesday, October 17, 2001

I've used some of the rational testing tools before, rational robot, I believe, fits the bill for what you are looking for.
Some of the rational stuff is a bit overpriced though.
I hope you never have to speak to a rational salesperson.

Tony McConnell
Wednesday, October 17, 2001

I think Jan Derk made a mistake his program name and url didn't match.

The correct ones are:

AQTest ($349.99/su) from AutomatedQA at http://www.automatedqa.com/

SleuthQA ($399/pu) from Turbopower at
http://www.turbopower.com/products/sleuthqa/

Our tester has looked at and tested AQTest from AutomatedQA and was pleased with it (but whe haven't bought it yet afaik).

But you could always give it a try yourself.

Goodluck

Tjipke A. van der Plaats
Wednesday, October 17, 2001

Just another vote for AutomatedQA. AutomatedQA's stuff is absolutely top-notch. Well worth using, very powerful and feature rich. It's also reasonably priced.

Tim Sullivan
Wednesday, October 17, 2001

Definitely AutomatedQAs'
AQTest

http://www.automatedqa.com/products/aqtest.asp

Roman Eremin
Wednesday, October 17, 2001

I forgot to tell you that we using it from the earliest beta times - for almost 2 years, so feel free to ask me specific questions.

Roman Eremin
Wednesday, October 17, 2001

Oops, I left wrong e-mail. Use romae@salesexplosion.com

Roman Eremin
Wednesday, October 17, 2001


<<I think Jan Derk made a mistake his program name and url didn't match.>>

Yep, you're right. Thanks for correcting. That'll learn me to copy and paste links without checking.

Jan Derk
Wednesday, October 17, 2001

Rational Visual Test is pretty good if you don't need to perform performance testing.

As you might know, it was originally called MS Test, and was a tool that MS created internally for its own use. As more and more people outside MS were asking for a copy, MS finally released it as a commercial product.

At some point, I assume it figured it was using too much resources to keep developing it, so sold it to Rational... which only came up with one release of their own in 3 years.

Pretty good, Basic tool, though. A trial version is available here:
http://www.rational.com/products/visual_test/index.jsp

FF.

Frederic Faure
Wednesday, October 17, 2001

I think you should consider "rolling your own".  You might find that you are happier in the long run with that level of control.

Anon Lover
Wednesday, October 17, 2001

<font size=20>Hmm, well Joel doesn't think QA tools are a <br>
"core business function", so therefore he won't do it himself..<br>
<br>
also rolling his own won't work since he is pressed for time. Finish City Desk asap! :) <br></font>
<br>
<br>

Jasper Jonze
Wednesday, October 17, 2001

Perhaps I was referring to cigarettes?

Anon Lover
Wednesday, October 17, 2001

We tried a couple of the big packages with the GUIs and the week-long training courses and the yearly support contracts, but they didn't work with the UIs we were building (custom HTML hosting).

Instead, we settled on a free tool called AutoIt [1] combined with the JScript development and debugging tools built into Windows. It worked like a charm.

[1] http://www.hiddensoft.com/AutoIt/

Chris Sells
Thursday, October 18, 2001

Tony McConnell writes:
> I hope you never have to speak to a rational salesperson.

I must say, that's one for the quote book.  I don't think I've ever had the above problem.

Paul Brinkley
Thursday, October 18, 2001

Another vote for AQTest and AQTime (or whatever they're calling them this week) from AutomatedQA.

Top notch tech support from a small company who give a damn.

Mark Smith
Thursday, October 18, 2001

Yes, my comment, "I hope you never have to speak to a rational salesperson" does seem a bit unfair. I dont mean to be unpleasant. All the ones I have spoken to are nice people. What I really meant was that they seem to sell so many tools that its hard to get a handle on just which toolset you need for what it is exactly that you are trying to do. My apologies to any rational people.

Tony McConnell
Thursday, October 18, 2001

My guess is he actually meant the term "rational salesperson" is an oxymoron. :)  (not referring to the company Rational I meant..)

Jim Buck
Thursday, October 18, 2001

Yes, you're right, I realised this about a nano second after I hit the submit button. A rational salesperson is indeed a rare thing. Never mind, its Friday, I'm losing the plot anyhow, time to go home. I checked out the AutomatedQA web site, it certainly looks pretty good.

Tony McConnell
Friday, October 19, 2001

I most heartily endorse Rational Test (or whatever it's called now). Its discovery of your gui's object model is brilliant (or perhaps nowadays it is commonplace) and really makes defining test conditions wonderful. You manually do it for one set of inputs, then go to the code produced, and define a multi-dim array with the other inputs and expected results and just loop it through them. Very similar to using Excel to unit test COM objects, but the interface is your gui.

Woody
Wednesday, October 24, 2001

http://c2.com/cgi/wiki?VbUnit

VBUnit is free or cheap depending on version,
open source, written in VB.

Bob Haugen
Wednesday, October 24, 2001

I used the demo AQTest software to do some testing of a VB6 app I'm developing at the moment, and it had a few problems repeating row selection on a grid. Somehow the recorded 'click' on a row would not occur, which was quite annoying as the software is certainly easy to use. The test I was trying to perform was:
1.Start the application
2.Open a form with a grid containing a list of names
3.Add a new name
4.Select the added name (row) from the grid
5.Delete it
6.Close the application.

Step 4 would not occur as it was meant to. I'm still looking at it, the software certainly has merits, maybe its just me...

Tony McConnell
Thursday, October 25, 2001

Joel,

"Not too expensive" sounds strange...  especially when it comes from you, - developer and  project manager.

Talking about pricey stuff, - why do those comapnies survive if their stuff is so expensive?!

The answer is very simple, - do not look at the product's cost only. It means nothing! If the product doesn't fit your needs and you will spend 2 hours more per week using a $5K cheaper tool rather than getting your job done more effectively, - c'mon: calculate how much your working hour costs. Now, rethink.

My point is: initial tool's cost means not that much. There is much more money wasted using the wrong tool in the wrong way. And that might happen during _years_ ! Unfortunately, I've seen that :-(

BTW, do you really _know_ when do you need to use such tool? (I mean product life cycle) Do you know how to estimate what should and what shouldn't be automated?

Good Luck,
QA automation veteran

Ilia Bromberg
Sunday, October 28, 2001

We ahve has only limited success using Rational Robot (and once you purchase a copy the saleperson will keep calling - unless you have a problem with it). The problem I have found is that most of these automated test tools do not do well with customized components, so for you to do regression tests you will need to do screen scrapes and bitmap compares. We did get this working and it certainly takes the drudgery out of the necessary but tedious regression tests, but I keep thinking there must be a better way. I think the right way is to embed self-test into the application rather than have a third-party tool try and figure out what we are trying to do

Sam Cirka
Friday, November 02, 2001

Joel,

What did you go with?  How do you like it?

Chris
Friday, April 26, 2002

*  Recent Topics

*  Fog Creek Home