Fog Creek Software
Discussion Board

QA Teams - do we need them?

My boss has recently asked me to restructure our dev team into a much more reactive role concentrating on small 2-6 week projects rather than a single dev build project team (we have now begun to rollout a system that has taken the last two years to write.)

When discussing my proposed restructure he has just dropped a bit of a bombshell by suggesting that we don't need to have a QA team who are a seperate functional unit (as it is currently), but to just include QA as part of our design teams concentrating on user testing. This assumes that our dev team has carried out all unit and system testing activities before handover to the users for beta. 

In all my previous employment I have fought long and hard for a dedicated QA function who are trained specialists concentrating on making sure that the business is exposed to minmum risk when deploying new products and system.

My boss's behaviour is not unusual when talking to non-software development managers who don't see the need for QA, but what's different here is that he has a distinguished career in IT and software development behind him - it's made me think, am I right or is there really another way? Does anyone have any opinions???

Friday, May 23, 2003

You are right on!  Don't let your boss convince you other wise.  Developers make the worst QA people that you can have. 

Matt Watson
Friday, May 23, 2003

True QA is performing an internal audit of the state of the system and reporting to senior managment.

In my experience, lots of people think QA = testing.  This is wrong - if the requirements are screwed up, if the spec is bad, if the coding is bad --- testing isn't going to save you.  You need to catch the errors earlier.

And, sadly, most coders have our own work to do, are overly optimistic, and feel bad "picking on" other coders who aren't making the grade.  Worse, most supervisors and lower-level managers have tremendous pressure on them.  By Bonus and incentive, they are allmost forced to be overly optimistic.

As a result, senior management often has no clue what's really going on in the trenches.  That's where QA comes in.

Sadly, this expanded role of QA often threatens the supervisor and lower-level manager.  After all, the more people that can tell Sr. MAnagement what's going on, the greater the chance that Sr. Management will hear a different story - which may make the line manager look wrong, silly, incomepetant - or - at the very least - disconnected.

So yes, it is common for folks to feel threatened and fight against and expanded and empowered QA department.

That doesn't make it right.


Matt H.
Friday, May 23, 2003

I believe that a separate QA structure is useful, QA needs the same political clout as the Development group, however QA doesn't just belong with a separate department its a function that is part of everyone's involvement down to and including the CEO's secretary.  (actually the CEO's secretary is usually very useful for QA).

It isn't necessarily true in small organisations for there to be an entirely separate group but if there isn't one then it means that the releasing authority has to have even greater moral integrity.

Simon Lucy
Friday, May 23, 2003

James, I don't think that you explained why your boss wanted to eliminate QA. What is the rationale that he's using? It would help to understand his argument against having QA.

obJoel: here's the link to Joel's article "Top Five (wrong) Reasons You Don't Have Testers", in case anyone missed it:

Bill Tomlinson
Friday, May 23, 2003

It really depends on your team as to whether you need dedicated QA staff or not.

You can't test in quality, so don't think of QA as "quality insertion". If the code is crap, there's nothing QA can do about it except to point it out (way too late, I might add).

You also can't test in human factors. The QA department shouldn't be the first time an "end user" is giving the product a run through. That's something that has to be planned out much earlier on.

So you're left with QA doing two things: making sure we're sticking to the plan, and finding small bugs or inconsistencies that made it through development.

If your team is using Extreme Programming, you may find it's much easier for them to do occasional QA. There is less of an individual ownership feeling in XP teams, so it doesn't feel like finding bugs is picking on one developer. Everyone owns all the code, so a bug affects everyone.

It also depends on whether you have enough people so that everybody can test something that they didn't have a direct hand in implementing. It's sort of critical to black box test, and if you know the underlying system too well, you can't do it.

So, yeah, I'd say it's quite possible to get by without dedicated testers, perhaps even indefinitely. That's not the same as saying you won't be testing. You absolutely need someone to check the quality of the product, but as long as all the developers are willing to do it, you're fine w/o a dedicated QA staff.

Brad Wilson (
Friday, May 23, 2003

There is another way, though it may not help.

Extreme Programming works the way you described your boss wanting to work.  That doesn't mean that this will work in your organization, of course.  Here's how the XP folks do it:

The customer has defined a particular feature that's to be implemented.  The developers first sit down and write out (automated) tests that ensure that that feature exists and works as defined by the customer.  The developers then run the tests, which of course fail, because the feature doesn't exist yet.  The developers then write code implementing the feature, so that the tests pass.  They essentially code to the tests.

This requires:

1) Very clear requirements.
2) Strong support from the developers.  This is an unusual method of development, so the developers need commitment to it or else they'll be even less productive than they otherwise would be.
3) Automated tests.  Developers are good at writing code, and thus can write code to test code, but they're not necessarily good at writing physical test procedures.
4) Pair Programming.  When you're alone, it's too easy to skimp on tests.  When you're coding with somebody else, you keep each other honest.

From your post, I think the above four requirements do not hold in your organization.  So, I think there will be problems.

And, of course, it's always good to have a trained QA professional to test for human-factors issues and other problems that developers just don't think about.

Do you want our suggestions on how to fix this?  Or do you just want to know that you're not crazy?

Brent P. Newhall
Friday, May 23, 2003

My company hasn't had a dedicated QA department since October of 2000.  Since then we've released probably 20 different releases of varying degrees.

Our QA department is our development team.  As Brent said, this is easier since there is less ownership of parts of our software.  Our team is also fairly mature and doesn't give into having our feelings hurt when someone points out problems with our code.

For some larger projects with tigher deadlines, we'll bring in support and other volunteers from the rest of the company.

Based on my experience, having developers do QA is great.  Part of the problem we experienced with a separate QA team was that it was much easier to "code, compile, forget" and have the project suffer multiple rounds of needless extra testing.  It only takes a few iterations for developers to realize that skimping on unit testing is only going to hurt them in the end.  That's contributed to much of the teams's maturation over the last few years.

The next step I'd like to take is to add more XP-based techniques, namely development of test plans which aren't as rigidly enforced as we probably should.  Usually I'm the one who ends up writing/revising much of the tests in the final week of coding.

One of our major problems is QA scheduling.  While we track some data like how many tests can be run per day given X number of individuals, we've always been fairly horrible about the actual scheduling.  Usually we don't have a clue as to how big the test suite will be until coding is complete and once we get into it, can put our overall project schedule off a week or more.  I think that with a better integration between our PR system and test center (and PTO calender) that we can have better tracking on a day-to-day basis as to what we're facing in terms of testing and the associated scheduling.

Chris Blaise
Friday, May 23, 2003

One of the key tasks of QA is to measure software against specifications and sometimes (actually quite often), those specifications may be outside the knowledge of the domain of the developer.

For instance, taxation rules, accountancy, professional regulation....

In that case its essential to have a separate sign off authority.

Simon Lucy
Sunday, May 25, 2003

QA != testing

In particular, QA != people who test after the programmers have finished coding.

QA is part of the whole process. Software designed using the waterfall method can be Quality Assured. So can XP. So yes, your programmers (along with everyone else) should be part of the QA process.

Testing, especially end user/usability testing is a whole different ballgame, and you definitely do need people who are good at testing. This usually means people who are not your developers.

Sunday, May 25, 2003

*  Recent Topics

*  Fog Creek Home