Fog Creek Software
Discussion Board




Pair Programming Metrics

One of the strongest arguments against Pair Programming is that there are no objective metrics proving that Pair Programming is better than solo programming.

What sort of objective metrics would you like to see applied?

I'm in a situation which would allow me to apply such metrics of pair programming vs. solo programming, and will be happy to do so.

Brent P. Newhall
Tuesday, March 11, 2003

The problem with coming up with a metric is that the benefit of Paired Programming is not quantitative, but qualitative.

My partner and I have used our own variant of XP, and we often pair programming.  But the results are not necessarily fewer work hours.  Most often the benefit is reflected in the quality of the underlying design and the number of bugs.

However, we believe that it has helped reduce production times.

Always remember that PP isn't oriented to all tasks.  We use it most often when designing the code on the board, and implementing complex code.  Small simple code and bug tracking are not as beneficial using PP.

Walt
Tuesday, March 11, 2003

http://www.stsc.hill.af.mil/crosstalk/2003/03/jensen.html

Thomas Eyde
Tuesday, March 11, 2003

That analysis used "lines/person-month" as a metric. Of course, "lines of code" being absolutely the worst possible metric...

I would think the best evaluation of Pair Programming would be to take a small, complex task (like "Build a parsing engine to do [strict set of processing guidelines]"), and take 30 people - ten work as individual programmers, the others work as ten pair programming teams.

When they're done, score the 20 projects - a subjective blind score on coding style and an objective score from a test harness (run 1000 documents through the engine, score the results). Average the scores and rank by manhours taken to complete the task.

Philo

Philo
Tuesday, March 11, 2003

I agree that lines of code is a bad metric in general, but assume the coding style is the same before and after a methodology change, then a higher line count must mean a higher productivity.

But that's of minor interest. The real interesting part is the one about quality. Did you notice the reduced bug count?

Thomas Eyde
Tuesday, March 11, 2003

Yeah, and i don't doubt it - I find these days a *lot* of my bugs are "head smackers" - obvious once they're pointed out.

I think I could work pair, and would probably enjoy it with the right person. It's finding the right person that makes me question PP as an absolute.

Philo

Philo
Tuesday, March 11, 2003

I don't think anyone doubts working in pairs is beneficial in some situations. I'd be more interested in what situations it works best, and what team configurations (skill levels etc) work bests in these situations. I think if 'those parts' of the pp advocacy movement would stop calling people social rejects, and maybe drop the 'silver bullet for all situations' mantra, there would be a lot less people calling for 'proof'.

When I've worked in pairs with pleasure and success is on design, esp the type that crosses boundires, and debugging. I haven't done a lot of mentoring in programming, but in other fields I've often paired up with the learner (or pro) with the learner at the controls. Learning to drive for example. I think that is obvious and outside the whole pp thing...

For straight programming of a module though, I think a lot of the benefits of xp are still there if you:

1) use a good ide that catches bugs, has intellisense so you don't have to memorize so much, does quick partial builds etc.

2) Take short breaks every 15-20 mins, depending on you

3) write comments that explain your intentions, always

4) Follow a (any, but a) standard coding practice (style and architecture)

5) Be conscientious. Don't surf the net or sleep when you should be working. Don't show up tired, drunk, or with a hangover. (ok, not regularily)

6) Write tests, don't take shortcuts on things like exceptions or type saftey

7) talk with others when you need to, obviously. Work in pairs, or a full on orgy when it makes sense. Don't be dogmatic.

I see this has veered a little off topic though - sorry. Tests I would like to see (or observations are fine) would try to figure out what exactly are the advantages of pairing when it is an advantage (eg how much of it is taking breaks? How different are individuals with their optimum time between breaks and length of break?). Then use this to figure out what situations pp makes the most sense, and also (I would guess) to improve your productivity when alone or in larger groups.

I would guess that each aspect of it would have to be separated somehow, and measured on people working alone and in pairs. There is as much to glean looking into people who can't program in pairs too. Probably they have a very different path to productivity, much like visual thinking vs analytic vs social. It is important to look at the individual if you want to get maximum productivity out of them rather that push them into a box. Maybe by the end, a sort of aptitude test that finds your best work enviroment. If you are an 'pp only' shop, such a test would still be useful - at the interview ; ).

Robin Debreuil
Wednesday, March 12, 2003

I wouldn't agree that a higher line count is a valid measure of productivity, even in this limited context. One of the benefits of pair programming is smoother transfer of project knowledge -- which can include things like "you don't need do that in X, we've already handled that case in Y".

Daryl Oidy
Wednesday, March 12, 2003

If we introduce pair programming and the process results in more reuse of existing code, then the loc count will drop.

If we know we are better on reuse, create higher qualitu code, and the loc count still increase, what would that mean?

I would say increased productivity.

Thomas Eyde
Wednesday, March 12, 2003

...or your first two assertions are in error. :-)

Philo

Philo
Wednesday, March 12, 2003

(DeveloperSatisfaction + ManagementSatisfaction) / 2

Big B
Wednesday, March 12, 2003

Since the last tread on pair programming I have been tinking about it, and have come to the conclusion that I am actually keen to give it a try. My intitial gut reaction of "No way!" stems more from an instinctive fear of being forced to pair up with "that monster", as opposed to a reasoned evaluation of the method itself.
Unfortunately in my current position there is no opportunity for pairing, but should it arise in the future, I am going to test the waters a bit.

Just me (Sir to you)
Thursday, March 13, 2003

Glad to hear it, Sir!  I hope you find it as productive and enjoyable as I have.

FWIW, you might want to consider pairing with a friend off-hours, just for testing purposes.  I've had a lot of fun coding with buddies.

Brent P. Newhall
Thursday, March 13, 2003

*  Recent Topics

*  Fog Creek Home