Fog Creek Software
Discussion Board




Help managing test code

In my current project we have a "large" code base of about 100K SLOC and a set of unit test code of about 50K SLOC. Our challenge now is to maintain the correlation between the test code and the project code. Most of the test code specifically tests a single class/method.

I'm looking for a way to:
1) Quickly determine for a project class/method which test code covers it and vice-versa.
2) Using the above detect where there are "holes" in our testing.

Several options we have considered are:
A) Maintaining an external to the source code reference (spreadsheet, etc.) - This is difficult since it is a separate cross reference to maintain.
B) Documenting directly in our comments using DOxygen's \test tag the relation from project to test code - here, if we document in the project code, a change to the test code would make us change the comments in the project code.
C) Continue attempting to use a code coverage tool like DevPartner Studio's TrueCoverage to generate this data. Unfortunately TrueCoverage's output format doesn't seem conducive to this type of reporting. For example, it doesn't distinguish overloaded methods.

Any tools, procedures, and options are appreciated.

GStark
Wednesday, July 07, 2004

Before anyone questions my "large" code base comment:  I mean that this code base is large for my particular department, which is why we are having a challenge on how to manage the test code.  I recognize that 100K SLOC is not large compared to many projects.  :)

GStark
Wednesday, July 07, 2004

You need a list of requirements, and a list of bugs.  Map each test to a requirement, it usually takes several tests to completely verify a requirement.  Also ensure you have at lease one test for each bug that's been closed.

Snotnose
Wednesday, July 07, 2004

Thanks for the input.

We have a good traceability of requirements to tests for functional requirements.  However we have a number of internal libraries for which we would not have "requirements" to trace unit tests to.  We have, for example, container classes and need to ensure the add, remove, contains, etc. methods are all tested.  We could use our requirement management tool to trace each class method to it's test, but this seems inconvenient and error prone because it is another place to have to put this traceability.

GStark
Wednesday, July 07, 2004

Have a look at gcov:

http://gcc.gnu.org/onlinedocs/gcc-3.0/gcc_8.html

Tom Payne
Thursday, July 08, 2004

*  Recent Topics

*  Fog Creek Home