Fog Creek Software
Discussion Board




Should developers see the test specs?

Well, should they? On the one hand it might make it easier to code - "oh, I missed that possibility for this feature" - but on the other maybe it could lead to complacency and the code just satisfying the tests - "but the tests all run fine! Yes, but look, if I wiggle the mouse like this the system crashes." What do you think? XP players maybe shouldn't answer this question :-)


Wednesday, August 14, 2002

Well, we do unit tests for all our new code.  They don't have to be big and complex, but they do have to be done.

Whenever QA writes a bug report, we add a fix and a unit test, so that particular bug should never happen again.

Mind you, at my company we have a good relationship between development and QA, so there's none of the "gotcha" stuff I've seen elsewhere.

I'd be happy to see what QA plans to do, but I've got so much going on that simply designing and developing to requirements takes a fair amount of time.  QA input might be useful a priori, but adding fixes and unit tests is working well for us so far.

Patrick Carroll
Wednesday, August 14, 2002

I've always thought that good testing requires a mixture of structured and unstructured tests. There needs to be a formal test plan, which must be signed off. However, unless your application has a very tightly defined set of inputs and outputs, it's impossible to include every possibility in these tests. So there also needs to be a certain amount of "mouse wiggeling", as you say. Bugs reported from the informal tests should have the same weight as those from the formal tests (and in both cases prioritized according to severity).

And to answer your question, yes, I think developers should have access to the test scripts. I've never seen a developer say "I can get away with sloppy code here because the test is weak". They are more likely to suggest how the test could be made stronger.

James

James Shields
Wednesday, August 14, 2002

James - I wasn't thinking that developers might make a conscious decision to "write sloppy code", rather that they might miss some other case that they might otherwise have caught if they hadn't been concentrating on satisfying the testing requirements.


Wednesday, August 14, 2002

"I'd be happy to see what QA plans to do, but I've got so much going on that simply designing and developing to requirements takes a fair amount of time"

Patrick - aren't the tests that QA run based on the same requirements? To me QA tests are a different "view" onto the same "model" (the requirements) as code (an altogether different view than the QA tests). In fact I doubt that either view is likely to be 100% correct in its interpretation of the model, and I believe that this is why many bugs are caused, so I was wondering if by having access to both views developers could create a more faithful view, at least on the code side.


Wednesday, August 14, 2002

Anonymous :- I agree on the "single model, different views" idea. 

I'd add, however, that a development view different, and divorced, from a QA view might work the same way as a thesis, antithesis approach: when the two have fought it out, as it were, the resulting synthesis is all the better.

If there's perfect communication, and perfect understanding, and perfect implementation, then what comes out of development should pass QA the first time, with no defects.

That will never happen: people bring their assumptions to the table, people descope stuff based on time constraints; stuff happens.

That being the case, access to QA's plans might also lead to a gaming of the system - implement only what they'll test.

I don't know.  I could argue it both ways, and I'm no sophist.

Patrick Carroll
Wednesday, August 14, 2002

"a development view different, and divorced, from a QA view might work the same way as a thesis, antithesis approach"

Yes, I agree. I hadn't thought of that.

"a development view different, and divorced, from a QA view might work the same way as a thesis, antithesis approach"

That's my problem with it as well.

I am going to go away and have a think about what you have said. Thanks.

"I'm no sophist."

For a moment there I thought you were saying you weren't a lesbian :-)


Wednesday, August 14, 2002

B*gger. The second "a development view different..." quote in the above post should have been "access to QA's plans might also lead to a gaming of the system - implement only what they'll test"

Bah! Humbug!


Wednesday, August 14, 2002

Do you tell QA what to test for, beyond what the spec calls for? Do you tell them about any weak points or areas of concern?

I tell 'em because QA doesn't know what's inside the black box. I know I haven't thought up all the ways something can go wrong, and I want anything that's not minor discovered before release.

I can see this might not be the practice in an antagonistic setting.

tangram
Thursday, August 15, 2002

*  Recent Topics

*  Fog Creek Home