Fog Creek Software
Discussion Board

Costs of testing on every platform


Just read the latest article and it makes a good point, I've been guilty of only testing software on my setup in the past. My actual question is do you actually need to get licenses for all those versions of windows you were running?

I've switched over to Linux for a while now so getting a hold of different distros to test isn't a problem, but it seems like testing on Windows would be.

Miles Barr
Tuesday, November 20, 2001

You can get a subscription from Microsoft. We have something called MSDN Universal which costs around $2k a year and includes a subscription to all operating systems (going back to DOS) in every language and version, the complete development environment, and developer editions of Office and all the Backoffice server software.

Joel Spolsky
Tuesday, November 20, 2001

Yeah MSDN is great.

Matthew Lock
Tuesday, November 20, 2001


I made a program using only Javascript (backend will be ASP). Netmmercial ( shows a banner between pages.

I thought, well Javascript is even more ubiquitious than Windows, we're talking about a language that works in almost any browser! Great! Man how wrong was I... I didn't do my homework, developed in IE5, and just couldn't hold the enthusiasm to let the world know about it.

At the first discussion list, lots of problems pointed. I had to install Win95 on another machine, and start the proccess of testing IE2 thru IE6. Functions that claimed to work in Javascript1.1 and didn't. Methods implemented only in version 1.2.

When I finished IE, it was time of Netscape. Four days to make it support NS4.5-6.1 e Mozzila. Had to give up in NS3. And in Opera 6 didn't work either.

Now I know how there is a theory and a practice about multiple platforms. You can only be sure if you see it working. The same applies in part to Java, beware folks of everything you read in marketing and even docs.

Mauricio Macedo
Tuesday, November 20, 2001

No offence but I hope your Netmmercial idea doesn't take off. ;)

Matthew Lock
Tuesday, November 20, 2001

In addition to MSDN, remember VMware when you're thinking of setting up testing environments. Joel has recommended this in the past and it's a great way to go -- set up undoable virtual machines of every platform you care to test with and toss them on a fast machine with plenty of RAM.

Mike Gunderloy
Tuesday, November 20, 2001

>You can get a subscription from Microsoft. We have
>something called MSDN Universal

As far as I know, you have to get a copy of MSDN Universal *per developer or tester* who uses it, since Universal is not by default a site license -- it is licensed to individuals. And there are quite a few insane rules -- no more than n installs, where 3 < n and n < 10. No "stress testing" with 10+ connections. And so on.

So I'd think Win32 platform testing can still be a factor. Especially if you -- unlike CityDesk -- have localized versions.

Ranajit Ray
Thursday, November 22, 2001

We go through this pain barrier every time we do a new lot of install scripts. (95, 95OSR2, 98, 98SE,NT4,Win2K,WinXP)

It is worthwhile getting your software certified compatible for Win2K. Sorting out the install & uninstall requirements is painful and you have to spend some $ on the testing (less than $500).

However you can then become a microsoft certified partner, well worthwhile as you then get FIVE lots of MSDN univ. along with ten copies of XP & Office (all non product activation copies) all for around $1500 a year.

Certified compatible IS much easier than the old designed for Win2K program which used to require that you use the MS installer and supported a ton of difficult to retro fit stuff (like self repairing installations)

Well worth while. Saved us two full universal subs, so we'll be up $8000 next year.

Peter Ibbotson
Monday, November 26, 2001

*  Recent Topics

*  Fog Creek Home