Fog Creek Software
Discussion Board

Personal Experiences in Testing/Documenting


I am curious to know how you like to test and document your software, or more correctly the methodology you follow. Also, please feel free to comment on mine. 

I have a fairly complex piece of freeware around 10,000 lines of code written in a tool similiar to Visual Basic and 80 screens (major and minor). I dislike writing elaborate test cases in Word documents.

This is how testing it is done.

1) I keep using my software for 1-3 months till bugs are identified. I can release my software whenever i want - as its freeware.

2)I make sure i debug almost every important function/class. Setup breakpoints and see how the software behaves.

3)Continue Steps 1 and 2 till a day comes when no more significant (minor or major) bugs are found. After that i release it to my friend for beta testing.

For documentation, i read the Microsoft Manual for writing help files (Yes Philo, I am a hypocrite). Usually revise (Reread) documentation around 2 times to make sure there are no spelling mistakes or grammar errors. The entire documentation is around 90 pages.

For usability, again depend on the Microsoft usability standards (Yes Philo, i am a  hypocrite #1) . See whether hotkeys confirms to standards etc. etc. Usability testing is around 1 week. Reading the entire manual is sometimes a pain, as i tend to forget. Also, i use both Windows XP and classic settings to see whether it looks allright on other OS. Sometimes,  also take a look at Joels online articles or books to get some ideas.

After that it gets released to a friend of mine who tests it and reports bugs. The product has around 500-600 downloads for the past 1 year.


Friday, June 11, 2004

My company has a 100,000 LOC software (mostly in C).

We do 4 levels of testing:
* Every function or procedure is tested individually. The code is reviewed by other developers. The original coder is asked to review what he wrote 2 weeks later. This seems odd but actually this helps finding bugs or code weaknesses that weren't "visible" to him immediately after finishing his "homework".
* We create automated testing sequences for each module.
* "We eat our dog food": our software is used on a daily basis by our developers and by most of our company staff.
* 3 or 4 months before the official release date, the software is available for beta testers.

We use the feedback of our non-techie staff to write the documentation: They test an early version of the software (or one of its modules) without giving them any documentation. A silent developer will monitor them and report what they found immediately, what was confusing for them, and what features they haven't understood (or even found.) This report will help us know on what features we should focus in the documentation.
After writing the first draft, we redo the test with people that do not know any thing about the software (usually relatives or friends) and we give them the documentation. Their feedback helps us fix usability problems and evaluate the clarity of the documentation.

Saturday, June 12, 2004

*  Recent Topics

*  Fog Creek Home