Fog Creek Software
Discussion Board




Unit Testing

Joel wrote about not writing unit tests until after the code was at least functional.  I'm trying to use unit tests more consistently, at least where it makes sense (in embedded systems, some code is impossible to test outside the lab), but generally don't do things the "XP" way.

As I understand XP, you should write one test at a time, watch it fail, write the simplest code that will get the test to pass, verify the test passes and that all other tests still pass, then go on to do the same thing with the next test (which presumably represents another "user story").  Of course, all this is done with a coding buddy.

I'm far more likely to write the tests in batches, and just as likely to implement them up after writing my code as before. Thinking about what tests I'll need to convince me my code is correct helps me plan it out better, but I haven't noticed that it really makes a difference what order the code and tests are actually written in.

I'd like to hear other folks' experiences with unit testing.  Any advice on C testing frameworks would also be welcomed.

Michael Gauland
Thursday, December 20, 2001

Basically "code unit test first" rule leads to optimal thinking before coding.
XP does not value long design period before coding (to avoid architecture austronaut syndrome). So in order to prevent opposite effect of writing code without thinking it have some light rules. One of them is "code unit test first".
My experience is that when I writing unit test first, I am creating a better system:

1. I'm coding only those features that I need right now.

2. I'm creating less coupled system, because I'm forced to test my classes separately.

You may want to check a brilliant example: http://www.objectmentor.com/publications/xpepisode.htm

Roman Eremin
Thursday, December 20, 2001

Joel,

I recommend a read of this article by Kent Beck

http://www.computer.org/software/homepage/2001/05Design/

It sounds like point 'c' in your article is ideal for a regression test.

Shaun Smith
Thursday, December 20, 2001

Hi Michael.

Definitely you should give the "proper way" a try. I think it sometimes seems difficult to do tests one at a time before the code because we're so used to thinking of tests as coming after the code. If you write your tests first, you end up with highly testable (and therefore nicely compartmentalized) code. Conversely, if you haven't been doing that, you end up with legacy code that isn't very testable.

An example of this is Joel's code that returns...HTML? I hope that's just a misunderstanding. If he has small functions that return simple results, then they are the ones you would run the UT's on. If his small functions are returning HTML, there's probably a design problem.

As for your C code...we're running UT's for Perl code, which is basically the same in that it's not OO.  The solution is simple: asserts. Create a test function for each .c file that does something like this:

void mod1_test() {
  assert(return_one() == 1);
  asserT(return_notzero() != 0);
}

etcetera. You might also want to build convenience functions like:

AssertEquals(return_one(), 1);
AssertNotEquals(return_notzero(), 0);

...which can print out more useful assert complaints.

Steve Willer
Thursday, December 20, 2001

As someone else pointed out, Joel could test his HTML before he wrote the code. Unit tests are NOT feature tests.

Starting at the bottom layer, Joel could write tests for his SQL statements (which already existed because FogBugz is a released product). Once the SQL unit tests are complete and passing, he can work on changing the HTML UI without worrying about UI bugs caused by the underlying SQL code.

cop
Thursday, December 20, 2001

I was pleased to see Joel write:
"If you do things in the XP order you have to figure out character-by-character what your output HTML is supposed to look like in advance, which strikes me as mind-numbingly tedious and a serious waste of time".
I have worked in places where it is impossible to convince development teams that unit testing can be incredibly time consuming and it is possible that it is not delivering enough bang for the buck when considering the time and effort to work out the test, produce the doco, in some cases - team review the results - etc. My experience is that far too often teams unit test too trivial bits of the application and that unit testing is most effective when attention is paid to carefully selecting the appropriate bits to unit test.
I'm a bit fan of putting tests into the application itself as well, so that the code is self testing.
I recommend "Refactoring" - by Martin Fowler (Addison Wesley).

Tony McConnell
Thursday, December 20, 2001

Good point. When writing his code, Joel needed to decide what the output HTML should look like. Instead of doing it once and creating a unit test first, he wrote his code first, making tiny undocumented decisions along the way.

k0d3 p03t
Thursday, December 20, 2001

If you use XHTML then you can use DTDs and XPath to unit test your code.

The DTD will ensure that the HTML is following the correct general form. (Or you could use schema).

The values of specific elements in the code can be tested with simple xpath expressions.

No need to think about it on a character by character basis at all.

Ged Byrne
Friday, December 21, 2001

Tony:

Unit tests aren't time-consuming if you write them before code. I'm sorry, but they're just not, in my experience.

Successful unit test writing depends on you doing them consistently . Once you have a bunch of code that hasn't been written test-first, it becomes hard to test. It's one of those self-perpetuating situations that you can only avoid by being consistent. If you're consistent, it's fast and easy.

And it's also fast and easy later! No longer do you have to worry "am I breaking obscure functionality with my minor change?" You just run the test.

This is probably the easiest aspect of XP to implement. All it requires is that you try it and use it consistently. I can't think of any excuse for doing test-first programming on new code.

Steve Willer
Friday, December 21, 2001

oops...I had meant I can't think of any excuse for *not* doing test-first programming on new code.

Steve Willer
Monday, December 24, 2001

*  Recent Topics

*  Fog Creek Home