Fog Creek Software
Discussion Board




Extreme Programming and the Hawthorne Effect

I was reading some case study articles in Software about extreme programming and one thing that I noticed is that all the articles were about the first project a company/group did using extreme programming. This led me to think about other XP case studies I've read and they were all about the first implementation (including the infamous C3 project).

This led me to thinking about the Hawthorne Effect (which is, from "Peopleware": "Loosely stated, it says that people perform better when they're trying something new."), particularly in regards to claimed productivity increases seen because of XP.

What I'd like to see is an article that starts out something like: "We've been using XP for two years, on all projects, day-in day-out and here's what we've found..."

Does anyone have any personal experience along these lines or point to any articles along these lines?

I'd like to avoid a degeneration into "XP sux" vs. "XP rocks" so how about limiting posts to the theme of: XP after the initial novelty wears off.

Bill Tomlinson
Thursday, May 22, 2003

I can't offer any experience on this score, but I will suggest that a post on the XP mailing list would undoubtedly be fruitful:

http://groups.yahoo.com/group/extremeprogramming/

One point:  XP hasn't been around for a very long time.  Only a very few people were willing to try XP even three years ago.  Few enough are willing to try it now....

Brent P. Newhall
Thursday, May 22, 2003

Extreme Programming makes sense to me, since I know that there's a synergy between two people working together that produces a "sum greater than the parts".

I keep thinking of one time in the Navy when a friend and I were on a boring assignment together - we did the Washington Post crossword as a team every morning, and finished it in under an hour. I know neither one of us could have finished it on our own, and I strongly suspect if one of us had done his best then simply handed it to the other, we still wouldn't have finished it.
It was the back-and-forth of bits and pieces of ideas that got us through it.

Also think about design bull sessions - creating a design (in my experience) goes faster when there are multiple minds at work:
a) Problem-solving goes faster
b) People find design holes more readily (then (a) helps find the solution)
c) With professionals, the pressure of wasting someone else's time keeps things moving.

(C) also comes into play with XP - let's face it, most people can't sit and focus for eight hours; you're gonna goof off. But when someone you respect is sitting next to you, you're gonna work.

I don't recall seeing this in XP articles, but I'd think you would have to take breaks - something like 15 minues every 60-90 minutes to clear your mind, since XP should be fairly draining. However, again since you've got someone waiting on you, that 15 minute break doesn't turn into two hours. So I would expect the process to be more focused and structured.

The biggest problem I see with XP is the cargo cult effect - the belief that simply putting two people at a terminal will make things go faster. XP must take some seriously award-winning management to make it work. You've got to have comparable people who can work together sitting next to each other. They have to be there at the same time together every day (in metro areas this can become problematic). You have to manage their tasking appropriately, and you have GOT to be a moderator as well as a manager.

I *think* the best application of XP in larger organizations would be to make some of your programmers try it for a while, then ease back and let those that want to volunteer to work XP ("Hey, John and I are gonna XP this segment together") instead of making it a mandate.

Philo

Philo
Thursday, May 22, 2003

Philo:  Very cogent analysis; thank you.

FWIW, XP pairs often swap members several times a day, which provides for the necessary breaks.  However, since a pairing session is typically more dynamic than a solo programming session (you're discussing, designing, writing tests, implementing, experimenting...all in the same hour!), it can be a lot less mentally draining.  Or, put this way:  Did you feel more or less drained at the end of that crossword hour?

Regarding the Cargo Cult issue:  XP highly recommends the use of all its Core Practices together, because like you say, simply decreeing "All developers shall now pair!" won't necessarily improve productivity.

Brent P. Newhall
Thursday, May 22, 2003

Okay, I'm going to be a bit of a jerk and try to direct the thread back to the topic I wanted, sorry.

Brent, Philo, I'd like to avoid having yet another discussion about whether XP works or not, in the theoretical sense. That dead horse has been quite efficiently beaten.

I'm trying to get some information about whether the reported productivity gains from XP might simply be related to the novelty of programmers trying something new and fun, rather than the inherent benefits of the new procedures. Hence my request for longer term case studies and personal experiences.

Feel free to reply: "piss off, we'll talk about whatever we damn well please".

Bill Tomlinson
Thursday, May 22, 2003

Hmmm..interesting point about the Hawthorne Effect.

I first heard of XP in '99, so surely there are some companies out there with at least 2-3 years of experience in using it.

I still can't get used to the idea of pair programming...I just can't. Lots of joint design and architecture meetings in the beginning, lots of communication, lots of code reviews, but I just can't see myself working day in and day out side by side with another programmer. There are plenty of times when I simply need to research something, or try something out. I don't want to have to explain to my "buddy" what is going through my head every second.  There are times when I have an idea and I just wanna try something out for a few minutes. Alone.

Mark Hoffman
Thursday, May 22, 2003

Oh.

Sorry, Bill. I posted my opinions of XP after your other post.

Mark Hoffman
Thursday, May 22, 2003

Bill, your description of the Hawthorne Effect isn't entirely correct.

The Hawthorne Effect was discovered (I believe) when they were studying factory workers making cars. They tried dimming the lights, and making the environment brighter, and for some reason both made the workers more efficient.

Then then realized that it wasn't what they were changing, it's that something changed AND they were being watched. It's a variation of the Placebo Effect. I don't think that raising the lights without them knowing they were being watched would've had quite the same effect.

Couldn't part of the effectiveness of Pair Programming be pressure to make Pair Programming work? If you have 15 developers who say "let's try pair programming" wouldn't the pressure be on to perform better?

www.marktaw.com
Thursday, May 22, 2003

Yeah, I know that I'm not using the strictly correct definition of the Hawthorne Effect. Sorry, I should have been more precise. I guess the more precise definition might be the novelty effect or something like that.

Basically, I know that when I'm given something new to "play" with at work, I tend to be more productive then when I'm doing/using the "same old thing". And this is especially true if I'm interested in the new thing.

And wanting to prove to doubters that my cherished new something works is also a factor in this.

So I'm wondering if the reports of XP giving productivity gains might simply be this novelty effect rather than something inherent to XP itself.

So how well does XP work when it's no longer the new thing and you've got nothing left to prove.

Bill Tomlinson
Thursday, May 22, 2003

Bill:  Fair enough.  When I read "What I'd like to see is an article that starts out something like: 'We've been using XP for two years, on all projects, day-in day-out and here's what we've found...'" I assumed that you wanted a more generic discussion.  FWIW, I wasn't trying to pull the discussion away from its topic.

Again, I have no experience with this, so I can't comment authoritatively.

However, I would *expect* that there's another side to this effect:  expertise.  As programmers become used to pairing, they'd become more efficient and more productive at it.  I expect that this would more than offset the initial enthusiasm as it wears off.

When I've read case studies of developers beginning pair programming, the ones I recall have reported the process as initially awkward.  Most programmers weren't initially enthusiastic about trying this new pair programming thing.  However, as they continued (at least, in the short term), they became better at pair programming.

Brent P. Newhall
Thursday, May 22, 2003

Bill T. wrote, "What I'd like to see is an article that starts out something like: "We've been using XP for two years, on all projects, day-in day-out and here's what we've found..."

Bill, you might want to read "Pair Programming Illuminated" which is published by Addison-Wesley. The review I read said it contained two case studies that are discussed in depth.

You might want to checkout the Usenet and Yahoo! newsgroups on Extreme Programming.

After doing all of the above, you are probably going to have also to do your own web searching to find the type of articles you are looking for.

One Programmer's Opinion
Thursday, May 22, 2003

I'd like to throw out another tangent on the combination of the "Hawthorne Effect" and XP's paired programming:

It's long been a theory of mine that paired programming works (when it does -- I've never done it myself) exactly *because* of the Hawthorne Effect.

Essentially, the Hawthorne Effect -- to loosely translate -- says that workers are more productive when they know that they're being watched/observed/studied. Under paired programming, you're constantly being watched by your partner and therefore because you're being watched, whatever built-in psychological mechanism we've got that manifests itself in the Hawthorne Effect decides to kick in, ultimately resulting in increased productivity. The beauty of this is that there is increased productivity in *both* members of the paired team, as you're both being observed by the other team member.

Comments? Thoughts? Ideas? Anyone ... Anyone ... Beuler ...

Sargent Sausage
Thursday, May 22, 2003

Just to defend my 'net cred, I did search a fair bit all over the web including the google newgroup archive, various XP sites, and various XP mail archives before asking my question in this forum. I didn't find anything much that talked about long term experiences with XP.

I didn't check out books, which I'll have a look at (but I'm not going to buy a new book just for this, the degree of my curiousity doesn't warrant exploratory purchases).

I also didn't ask the question in the various XP forums, as Brent earlier suggested. But that was intentional. I find that XP forums tend to contain a lot of zealots and I expected to get massively flamed with my somewhat critical question. I deliberately choose to ask on JoS because it's fairly neutral ground. But, it is true that I'm more likely to get the answers I seek in an XP forum, even if I have to wade through a lot of angry rhetoric to find them.

Bill Tomlinson
Thursday, May 22, 2003

Bill, I apologize - I wasn't trying to hijack your question; I was trying to answer it tangentially.

To wit - the theory behind XP appears valid. However, it also appears that it would take superhuman management and top .1% coders to make it work.

So in effect my point was "you won't find any long-term success stories." [grin]

Philo

Philo
Thursday, May 22, 2003

I thought XP stemmed from years of development @ one of the big auto makers?

victim, jr.
Thursday, May 22, 2003

Bill,

No promises, but when I have time I will see if I can come up with something for you.

Philo wrote, "To wit - the theory behind XP appears valid. However, it also appears that it would take superhuman management and top .1% coders to make it work."

Why do you believe this is the situation?

How is XP (or any of the other Agile methodologies) any worse or more difficult than what actually goes on in many small and medium size organizations today (code, fix, and pray)?

One Programmer's Opinion
Thursday, May 22, 2003

Bill, it's not clear whether your seeking confirmation of a belief that XP is wonderful, or trying to satisfy a genuine query.

I think the answer to the implied query is obvious - XP is just a cultish fad, with no genuine benefits for good developers.


Thursday, May 22, 2003

OPO - read my first reply where I detail *why* I consider XP a serious management challenge. I think it takes a lot of understanding from management, and a lot of working together. In addition, you need two programmers of equal talent working on one project. I'll venture your average manager will look at that situation and think "if they were working on two separate projects, they could get twice as much done"

Basically, I think that you need a FogCreek ideal to get XP to work, and those are very, very, very rare.

Philo

Philo
Thursday, May 22, 2003

It's even stranger than the Hawthorne Effect.  How do you distinguish XP's effect from simply being more productive because you learned something new and shattered some old biases?  (It's like when a C person learns Haskell and permanently becomes better overall.)

I'd probably not confine my search to pure XP, but also its components.  For example, pair programming occurs naturally in the wild; my first and my best coding happened under these conditions.  Incremental releases + unit testing are easily found in opensource projects.  Whlie XP makes a big deal of needing everyting for a good stew, things like pair programming can have reasonable substitutions like code reviews.

Tj
Thursday, May 22, 2003

I think this might be a case of new innovative techniques attracting a proportionally greater number of good developers.

Same thing happened in the early days of comp.lang.c++ . When it was new almost everyone doing it was pretty clueful and the level of discourse was high. As it became more popular the level of discourse dropped to that of the average developer.

XP is pretty new, so it's attracting people who are interested in the craft of software construction. These people IMO are going to generally be a cut above the average programmer. Programming is just a job to most people (present company excepted, natch). I'd be surprised if more than half the people at my current job had even _heard_ of XP.

Andrew Reid
Thursday, May 22, 2003

I like what Andrew said.
As a consequence, it may be hard today to find 'average' work environments in which XP has been used for 2 years, to test the initial hypothesis.

GP
Friday, May 23, 2003

We found that one version of pair programming works very well.
We use "pair check-in"(tm), programmers work on their own but aren't allowed to merge their own code back into the build.  They have to sit with a partner who does the merge line by line and they have to justify the advantage / correctness / understandability of each change to the other programmer who then accepts / rejects that line.

As well as learning language + system features from each other we found that you think more about your code if you have to justify it to another programmer.
It also catches more of the errors where the programmer is too close to the code to see the bigger picture which XP pair programming doesn't help with.

Martin Beckett
Friday, May 23, 2003

The real answer is that a development manager should use change creatively to get that 'first time' experience.  You can do it for yourself as well.  Even if you're doing something you've done tens or hundreds of times before think about it and consider doing it a different way, a tweak here or a tweak there.

Actually I wouldn't confine the idea of introducing change to be just the manager's responsibility but the group, department, company.

Naturally those changes should be rational even if counter-intuitive and they shouldn't be introduced in ways that are oppressive or bullying.  And, the culture of change shouldn't become its own Cargo Cult.

Simon Lucy
Friday, May 23, 2003

We've had a bundle of people turn up to enthuse the population about XP. They've been doing it for a couple of years and every single one of them says they're having a great time and they'll go back to other methodologies only if you kill them first.

I thought they seemed pretty convinced about the idea and it's long-term viability.

Katie Lucas
Friday, May 23, 2003

Maybe I'm reading too much into this, but why do so many people in this thread seem to think "Pair Programming" and "Extreme Programming" are interchangeable terms?

Brad Wilson (dotnetguy.techieswithcats.com)
Friday, May 23, 2003

Because it is the most scary part for most developers. Personally I have still not found an opportunity to do pair programming since this board convinced me to give it a go some months ago, but I am still looking forward to it and will jump with both feet should the occasion arise.

If it happens I will sure let you all know the score.

Just me (Sir to you)
Friday, May 23, 2003

"Pair Programming" is the most visible XP practice, so it tends to get argued the most.  However...

Bill wrote, "XP forums tend to contain a lot of zealots and I expected to get massively flamed with my somewhat critical question."  I read almost every post to the XP mailing list, and it's remarkably tolerant.  Those who post are extremely mature.

Tell you what; I'll post your question to the XP mailing list and see how they respond.

Brent P. Newhall
Friday, May 23, 2003

Getting back to the Bill's original inquiry, I am aware of a company in Denver, CO that I hear has been doing XP practices since about 2001.

I do not know how far you are willing to shuffle for this information but I would expect these people to be a good starting place.  However, you would be basically creating the case study you are looking to locate. :)

As a quick side node, I have personally attempted to bring XP practices to a past company.  I won't bore people with the details in this forum; however, I found it to be a mixed bag of pros and cons.

My biggest problem was getting the rest of the developers truely interested and prepared to try it with an open mind.

XP is an ideal with a set of tools to approach it.  You can take these tools to different degrees of rigor and still call it XP.

And one last note, I do find it unforunate that some persons on this forum has expressed the believe they can not imagine doing pair programming.  To me, it says "I can not find a way to connect with another person".  Your very presents on this forum suggests otherwise and I hope you find the courage to try it.

Dan Miner
Friday, May 23, 2003

Bill wrote, "...I also didn't ask the question in the various XP forums, as Brent earlier suggested...."

Yes, the XP forums do tend to contain SOME zealots and as well as people (Kent Beck, Ron Jeffries, Robert Martin, etc.) who are promoting XP because it is financially beneficial for them to do so. That said, I have visited XP newsgroups such as  http://groups.google.com/groups?hl=en&group=comp.software.extreme-programming on several occasions and the zealots and promoters who participate there rarely flame doubters or people who ask critical questions. In fact, I found the exact opposite to be true. The doubters seem to be the ONLY ones doing the flaming there! 

Philo wrote, "OPO - read my first reply where I detail *why* I consider XP a serious management challenge..."

First, a possible correction to what you wrote. I believe you meant to say "...if they were working on two separate tasks, ..." rather than "...if they were working on two separate projects, ...". Am I correct?

I agree with your management resistance comments, however, I am not so sure about your comments regarding the talent level of programmers. Yes, programmers working on a XP project will need to be talented, but do they need to be equally talented?  Also, are you saying that this fact (the need for talented programmers) is somehow unique to Extreme Programming and the other Agile methodologies?

I have never worked on an XP project and I have never tried pair programming, however, after working on several small and problematic software projects with no PM only 1-2 other developers who ONLY had the ability to perform basic coding tasks (i.e. couldn't do anything beyond what they were told to code), I am willing to give XP and even pair programming a try if the opportunity ever presents itself.  


Philo wrote, "Basically, I think that you need a FogCreek ideal to get XP to work, and those are very, very, very rare."

Could you expand on what you are talking about here? I think what you are saying is, "Unless you own your own development company or you hold a position that allows you to make or influence policy -- you (a grunt worker with no power) -- probably don't have a chance in hell of getting something like this implemented". If I interpreted what you wrote correctly, then I have to say that in most situations you are probably right.

Personally, I believe that choosing to use one of the Agile methodologies -- is not "the answer" but simply one answer -- for many small and medium size organizations. One simple but hard to accomplish approach seems to me to be -- Companies need to hire: 

* Managers who are willing to relinquish some control over the decision making process
* Employees who are capable and willing to work together for a long period of time (for many years on many projects)
* Employees who are capable and willing to participate in every aspect of the software development process

Given the right conditions just about any development process can work for an organization. The problem is that decision makers (those that hold or guard the gold) have to be willing to show some patience and spend the money needed to make whatever process they choose to use work. All project teams require tools and support to do their work efficiently and effectively, however, some receive more of the above than others.


Someone who didn't bother using a nickname wrote, "I think the answer to the implied query is obvious - XP is just a cultish fad, with no genuine benefits for good developers."

I don't know about you, but I work in a very immature industry where I am expected to deliver predictable results, so please tell me if XP really is just a cultish fad what does work? 

Don't bothering offering some cryptic response, such as, "a good developer is a master of his craft rather than a pedantic follower of some process". That type of response doesn’t offer anything meaningful or useful to this discussion.

One Programmer's Opinion
Friday, May 23, 2003

Thanks to everyone for their comments.

I was probably over-reacting about flame concerns in XP forums. So thanks for the offer Brent, but I'll ask the XP experts myself about long terms XP experiences.

But since we're talking about XP in general now....

My main problem with XP is trying to convince the customers (internal or external, I've worked in both types of situations) to accept what is essentially a fixed time, fixed cost, but *optional scope* contract.

Now, of course, this is stupid because we (software developers) all know that the project plan prepared, for example, six months before the end of the project isn't worth much. But customers always seem to want it anyway.

I know, I know, the answer is to educate the customer in the realities of software development and explain that in reality all software development contracts are optional scope in the end so we might as well embrace the truth. But they don't want to learn.

Bill Tomlinson
Friday, May 23, 2003

OPO, the question of "what does work" is a much bigger topic that I am not going to address in a JOS posting.

If we could stay on the subject of XP, though, I have seen no evidence that XP provides any sustained benefits.


Friday, May 23, 2003

"I have seen no evidence that XP provides any sustained benefits."

Have you been using it for a sustained period of time, then?

Brad Wilson (dotnetguy.techieswithcats.com)
Saturday, May 24, 2003

*  Recent Topics

*  Fog Creek Home