Fog Creek Software
Discussion Board




Who does your QA report to?

I am the director of software development for a Sales & Marketing company. We have 10-12 developers working on a variety of in-house software products. Our QA department consists of one dedicated person and another part time person. The QA folks report to another department in the IS group. I'm arguing with my boss that the QA people should report into software development. He wants them outside of development so they are not pressured to pass an app just to meet a deadline.

For those of you with dedicated QA people, where do they report and why? What's your ratio of QA people to developers? Any issues with QA reporting into software?

Michael Kenyon
Tuesday, February 12, 2002

I agree with your boss -- although I would want them to report to the same people that your Software Development department report to. 

Software Development and QA (If you have it <g>) should be on equal footing ;)

Rick
Tuesday, February 12, 2002

Your boss's point is a legitimate concern, although this is more a function of corporate culture than it is a function of corporate organization.  Frankly, I'm a practicing anarchist.  It really doesn't matter who QA reports to as long as they're lead by someone who can take a principled stand against pressure to release the software early, and as long as the testing gets done.  If that's not happening, then all stakeholders involved need to sit down and negotiate and bang out a compromise.

With 1.5 FTE for 10-12 developers, you sound like you're a bit light on QA.  McConnell, I think, recommends one QA for every two dev.  Some projects might need less, some projects might need more -- it mostly depends on whether you're building a skyscraper or a quonset hut.  But if your devs seem a bit swamped with testing duties, it may be more cost-effective to hire more testers.  See Joel's article on the subject:

[http://www.joelonsoftware.com/articles/fog0000000067.html]

Alyosha`
Tuesday, February 12, 2002

>For those of you with dedicated QA people, where do they report and why?

We have several types of "dedicated QA people".

First a hope that among these are developers themselves.

And, we have an "Internal Test Group" (ITG) who run the lab equipment next to developers' office. Developers submit builds to ITG. ITG report results to developers, and forward the software to "elsewhere".

"Elsewhere" may include: beta sites; our "System Test Group" (described below); and (sometimes) to other developer groups (with whom we're sharing prerelease builds); and occasionally to other people (e.g. potential customers) if these need pre-release demos or whatever.

ITG therefore also report to (ie communicate results and builds etc. with) all of the people in the "elsewhere" list above; but they formally "report to" the Head of Development.

Then there is "STG", the System Test Group ... they don't report to our development group, they work for company head office. They test release candidates. Their primary customers (the people who are most interested in their work) are the field engineers / customer support people. Again, customer support don't work for development ... and they may refuse to support something which hasn't been through STG.

>What's your ratio of QA people to developers?

>Any issues with QA reporting into software?

Christopher Wells
Tuesday, February 12, 2002

>What's your ratio of QA people to developers?

Let's say 20 people in our development group, of whom 1 is the Head of Development (who doesn't do programming), 1 is head of QA, and maybe two to five people (depending) in QA ... so a ratio of between 1/10 and 1/3.

If you don't have enough QA people now, then depending on your current metrics (how many bugs you have, how quickly you're finding them, what the severity is etc.) I expect you are free to assign QA-style duties to your developers: tell them to do unit tests, code walkthroughs and inspections, peer reviews, ...

We have more QA people employed in ITG (e.g. 5 of 15, 1 in 3) when the product is pre-release (of course). Now that it's released and we're beginning a new project, the number of people in ITG is down to only 1 or 2: they're acting as liason between maintenance programmers' builds, and the field; also, learning the equipment they'll need to test the next release. Other QA people have been reassigned to other development projects, as developers.

I don't know how many are in STG; you have to book time with them months ahead, e.g. say "We'll be releasing version 6.2 in April, please allocate 15 days of testing then."

Plus, there's a person called Project Manager, who works with one or more Development Heads, to advise them on following procedure, which can be described as a sequence of documents: various specifications (requirements and design), documented code inspection results, test results, etc., which are eventually auditted for ISO certification.

>Any issues with QA reporting into software?

QA do a lot for me: integration tests (verify my build works with others'); regression tests (can be time-consuming); setting up test equipment; communicating with "elsewhere"; maintaining the bug-list, so the Head of Development needn't ask me when he wants to know about that; writing test plans (we train by word of mouth and email our ITG to use our software, ITG write the test plan[s], and STG needs a written test plan); reproducing in our lab, for me to come and look at with a debugger, some problem reports from the field (which is better for me than debugging on remote customer equipment, or striving to reproduce the alleged problem on my own machine).

There's a well-equipped QA lab. It would be pointlessly expensive to replicate that lab 15 times, one for each developer, so there's one QA lab shared by all developers, and ITG run it, and report back to developers.

QA produce information. That information is useful to all groups in the company, Development included. I'd hope that any Brass who talk to QA do so to inquire (what is the status?) and to direct (test this next, send it to so-and-so), but not to pressure (what do you mean it's buggy, you're in charge of Quality, what's the matter with you).

Christopher Wells
Wednesday, February 13, 2002

I agree that a QA group should not report to the person who controls the development of software products.  A very bad conflict of interest not to mention that the QA will sooner or later feel that they will be pressured to pass "acceptable" bugs to meet deadlines.

Chris Woodruff
Wednesday, February 13, 2002

Worked in the past for a company with three founders.  One founder was head of development.  QA reported to a different founder.  Kept deadline pressures from influencing QA.  However, it took the head of development, the product manager and the QA manager to agree to ship a product.

But I agree that it depends on corporate culture. If testing is happening quickly enough, and the committment to software quality is high enough, then the development manager should have no problems if QA says WAIT!

Bob

Bob Crosley
Wednesday, February 20, 2002

*  Recent Topics

*  Fog Creek Home