Fog Creek Software
Discussion Board




Unit testing in n-tier app

We are building 3-tier app. Client - mid-tier - database.
Client app unit testing is "simple" (we have idea how to do that).
But how do you write unit tests for middle-tier bussiness rules?
Each procedure have complex preconditions (data in pretty complex datasets).
This is COM+ app, so there are transactions involved, etc.
We cannot find any reasonably large(complex) unit test procedure.
Any hints? Experiences? Online resources?

drazen
Tuesday, March 11, 2003

Unit tests should be small. Is it possible to break down the dataset handling to smaller, more test friendly code?

Thomas Eyde
Tuesday, March 11, 2003

I was thinking about something in similar terms - but this isn't unit testing, this is end-to-end testing.  OK, so scratch the notion of unit test here, unit test is testing upon the small quantities of code - like a math library, algorithm, device driver, user widget, ...

But suppose you want to automate the testing of the end product, client against server.  Has anyone successfully created a test harness which tests a user intensive client? 

Just thinking aloud, you'd need:
1. A method of automating user input (we've done this with a UDP socket handler in the past, which we send simulated user key input via UDP).  This is easy part.
2. A method for determining if the application responds properly.  I'm thinking that hook into the server for correct client application response to the UDP key input is required here.
3. For every human test script item, you'll need an machine method for determining success or failure.  The question is can this testing be reliable or not.  Can you successfully test a GIF rendering library without viewing the GIF?

If you aren't convinced that the machine can test the application reliably, then is it a waste of time to try?  I don't think its a waste of time to try - because you'll likely create a tool that you'll wonder how you ever lived without.  But it likely won't eliminate the human testing.  That's my guess, and nope, I've never successfully seen it done.

Nat Ersoz
Tuesday, March 11, 2003

Could you somehow script the input and the expected output?

For tips, see
http://www.xprogramming.com/xpmag/acsIndex.htm
www.fitnesse.org

Thomas Eyde
Tuesday, March 11, 2003

For example:
Simplified bussines rules for invoices: add taxes to items (some items have no any taxes). No taxes at all for export.
If you want to test tax adding procedure (really small) you have to do some really big setup. You cannot have item record if you do not have invoice record. So for setup I have to create two datasets and fill at least two records. Now there is COM+ in play. Object expects transaction in progress, so my test proc should prepare MTS just to be able to check if procedure will correctly add 5% to item amount.

This is biiiig overkill. All TDD examples and tutorials I found on internet are more/less academic. I could not find one that could be applied in n-tier scenario. Even more, I cannot see any of those examples to be aplicable in any kind of database application.
What am I missing here?

drazen
Tuesday, March 11, 2003


1) Create a test environment
2) Create modules that pass what you need back and forth
3) Create unit tests that walk the main routine

For example:

  You want to transfer some money from account 1 to account 2, then print a receipt.

Test the data interface layer:

1) Transfer.  Pass it -$1.  Check for error
2) Transfer.  Pass it an invalid from account. Check for error.
3) Transfer.  Pass it an invalid to account.  Check.
4) Transfer.  Pass it an invalid to and from account.
5) Transfer.  Pass it valid $ amount, to and from.  Check for success.  Get the transaction ID.

6) Print.  Pass it an invalid transaction ID.  Check for error.
7) Print.  Pass it a valid transaction ID. Check for success.  Check the output against a specific flag.

Business Layer:
  Similar to 1-7.  Slightly different parameters.

UI and system:
  You know how to test this.

Does that help?


Matt H.
Tuesday, March 11, 2003

I don't know jack about COM, but in Java I try to make my business objects dumb (aka, POJOs, Plain Old Java Objects) so that I can test them in a mocked up environment and not even need a database. Read the data I need from XML files (or whatever data format you choose) and operate on it in-memory -- makes it easy to keep the data consistent because there's nothing stored between test runs. A nice side benefit: you can create a separate applet that allows your customers, or your implementation team (however you do that) to create scenarios of the data using a separate utility. These scenarios are then stored in the data files and used by your testing environment.

Barring that, some sort of framework to load a known set of data into your database (and clean it out) is immensely helpful. In Java there is the dbUnit [1] utility to help out with this.

[1] http://dbunit.sourceforge.net/

Chris Winters
Tuesday, March 11, 2003

Cover All Adjoining Tiers with Test Software

Bill Tomlinson
Tuesday, March 11, 2003

Sorry, accidentally posted.

One alternative I'd suggest is to write unit tests that use test code for all tiers (other than the one being tested), not just the ones "above".

So to test your middle tier code, you'd write test code that behaved like your "upper" tier; but rather than actually using the database "lower" tier, you would also write accompanying test code that simulates the appropriate responses from the lower tier. In other words, your test case code sandwiches the tier under test and you don't exercise other tiers.

You need a high degree of decoupling in your code to achieve this, but it does let you test each tier in isolation.

And, of course, this only applies to unit testing. Don't neglect your integration testing.

Bill Tomlinson
Tuesday, March 11, 2003

Thanks for advices.
Question is if unit testing in this kind of application is cost effective?
We do full QA, and these tests discover bugs, reports are sent back to developers, they do fix, we do new release, then we repeat this until there is acceptable release.
I hoped that we could minimize number of bugs by introducing unit testing, so that we can lower number of  bugs as much as possible before QA. As it looks now, unit testing in n-tier would require much more work than QA procedure we now have.
Not using data access objects directly would require large amount of new code that will probably generate number of new bugs.
"Sandwiching" of objects is possible, but needs too much effort to be cost effective.
Is there any book or online resource that describes agile methods approach to n-tier application (and example is not trivial )? 

drazen
Tuesday, March 11, 2003

End to end testing is functional testing.  If you have figured out how to Test your front end (in an automated fashion) do the same with the whole thing.  I assume this is a windows app, so use an app (WinRunner?) to walk it through it's paces.  For Htp use can use HttpUnit or something similar.  Just go a gainst a clean, testing database (not live data). Check the results of what appears on your screen with what you expect.

Adam Young
Tuesday, March 11, 2003

Nat Ersoz:

I realize this is a bit late in the conversation, but just to reply to your early comment/question -- I currently work on a product where we do automated user input testing for the product I work on.

We have both text input validations (where we script possible inputs from our command pane) and a GUI validation, where we use various text commands to simulate GUI actions (open menus, select menu items, display windows, close windows, etc).

In both cases, we write a validation by preparing a script and an expected output. Our validation system then compares the output from our program with the expected otuput (using diff), and complains if they mismatch. Pretty simple, really, but we can catch a pretty significant amount of functionality using this system.

Steven C.
Tuesday, March 11, 2003

Don't really know anything about it, but here you go:
http://www.iunittest.co.uk/

pUnk
Tuesday, March 11, 2003

"This is COM+ app, so there are transactions involved, etc.
We cannot find any reasonably large(complex) unit test procedure.  Any hints? Experiences? Online resources?"

I have NOT read any the responses you have received so far, however, if you are looking for example(s) that you can follow and emulate at work then my suggestion is that you visit a local bookstore and look for books that among other things covers testing/debugging MTS components. You might also want to re-post your question on a few online programming newsgroups (such as Microsoft's) that have a COM+ or a MTS forum.

Let us know if you find anything useful that you would recommend to others.

One Programmer's Opinion
Wednesday, March 12, 2003

Steven,

This is interesting to me.  Has your automated testing reduced your reliance on human testing, and by how much (i.e. it used to take us 16 hours to fully run a test pass, and now it only takes 3 hours).  The more I think about this, the more it seems to make sense to me.

If you don't mind me asking, what does the end result of your GUI app generate (server interaction, text, ... )?

Thanks,

Nat Ersoz
Wednesday, March 12, 2003

I work on a debugger/IDE my company sells. As such, the end result (pre se) is a little hard to quantify, but most individual actions have a quantifiable change --  we change the displayed text, we change the status of some breakpoints, etc. These individual changes are what we attempt to validate.

Originially, I belive our validation system was designed to validate our compilers (which is a very easy thing to do, relatively -- since there is a single correct answer for every source code, you just compile it and see if it does the right things).

Only recently did we finish extending it enough to handle pure GUI functionality and even then, we handle that by exposing the callbacks to command pane interaction.

Its a bit hard to say how much this setup has saved us in human interaction -- but I'm pretty sure "tons" is close to accurate. Particularly since I work on a large project in a decent size group, the validation system we have has given me a number of warnings when a change over HERE broke something over THERE (especially when "THERE" was something I didn't even know existed).

Steven C.
Thursday, March 13, 2003

Thanks!

That is very encouraging.  I love the full on test, which is part of the development process.  I'd say your's is a somewhat rare case of professionalism at that level.  Very elegant.

Thanks,

Nat Ersoz
Friday, March 14, 2003

*  Recent Topics

*  Fog Creek Home