Thursday, June 2, 2011

Our move to pair testing

Three sprints ago the team decided that the way were handling testing wasn't working well. There were a number of issues that I'm not going to get into right now, but suffice it to say that we needed to do something differently. I also want to point out that this was nothing to do with the quality of the testing that was being done.

An action item that came out of that sprint's retrospective was as follows:

When a developer has finished coding they should try to pair test if our only tester was available to do so.
This action item was the top most voted for item with regards to how much energy each member had for making this change.

Over the following sprint there wasn't really a wholesale up take with the new idea until the last few days of the sprint. There were a couple sessions that happened before that, but something changed in the last few days that would altered how we viewed pair testing.

Before I move on I'll briefly describe what we consider pair testing.

To us pair testing is performed with the developer, who has written the code for the story to be tester, and our tester. During this session any bugs that are found and are related to the story being tested are fixed and tested again immediately. When the pair feels that the story is good enough to release then they stop. Any bugs that are found that are unrelated are noted down to be added to our bug wall later.

Anyway back to my story.

Unbeknownst to me our tester was in training for two days, three days before the end of the sprint. On finding out, which was the first day he was out, I asked the team how they wanted to handle this situation.

At this time we had a good number of stories that needed to be tested and there was no way we would have made our sprint commitment if we didn't do something.

The unanimous decision was that the developers would pair test themselves.

What happened then was a series of pair testing sessions with two developers. As one of my team mates put it, having two developers test kept them both honest. Also as bugs were being fixed as they were found, we shortened our feedback loop. With two developers testing we probably weren't as effective as pairing with our tester, but when each pair finished they were happy that the result was releaseable. Testing and bugfixing was no-longer taking days, but taking hours. There was little or no miscommunication and if there was it was cleared up immediately.

In addition, an observation I made about my own code was that I wanted few bugs to be found so I made sure it was really ready to test before exploratory testing.

All of this really seemed to bring home the value of testing as a pair to everyone, as well as removing a bottleneck that had been a problem for us. As it happens our tester was ill on the last day of the sprint so we couldn't have waited anyway.

At the next retrospective the team decided that we should pair test on every story and agreed to add it to our team working agreements.

During this last sprint our tester was back and everything is going well.  We still had at least one developer pair testing sessions to alleviate a bottleneck, but that's fine.

Overall I feel that this has been one of the largest improvements, if not the largest, the team has made. That is from a performance and a team cohesiveness level.

I'd fully encourage other teams to try this for themselves especially if there's a lot of to and fro between developers and testers or if testing is a bottleneck.

5 comments:

  1. Hi James,

    Excellent post, and I can certainly relate with the bottleneck feeling of having a test team, who do only re-active testing.

    When I first started as a tester this is exactly what we did (re-active testing), hunting for bugs and finding lots of them!

    I probably looked like the worlds best tester to management, but also the biggest pain in the arse as well.

    I can certainly say back then, that our team was a bottleneck to the development process. We'd test too late in the cycle, and wouldn't look at ways to improve the quality of code coming from our developers.

    I got tasked with thinking up of proactive ways to test the product, and initially came up with a couple which completely changed things around. The pair testing as you put it, is very similar to one of the techniques I'd used.

    To be honest the real game changer though, was not just the techniques applied themselves, it was the raising of awareness, and goal setting in developers minds. It made a big difference for us.

    You can read about those techniques here if you like: http://www.bettertesting.co.uk/content/?p=123

    Thanks for sharing, and keep up the excellent articles. I'm really enjoying your style of writing.

    Cheers,

    Darren.

    ReplyDelete
  2. I love the quote about "keeping Developers honest".

    I am a tester and two sprints ago I way pair programming with a developer (which I normally do not do), I was happily pointing out typos, missing ';' as he was typing and asked questions about why he had coded some logic from tester viewpoint.

    I also like your teams fast turn around on bug fixes; fix then retest.

    ReplyDelete
  3. Hi Darren, thanks for the link, your perspective and encouragement, I'll definitely have a read. Raising awareness and educating both developers and testers on why this is an effective way of working is critical.

    Hi Chris, thanks for the feedback and your experience with pairing with a developer. Would you choose to do it again? How did the developer feel afterwards?

    Darren & Chris, I'm glad that I'm getting positive feedback from the testing community which makes me feel that we are on the right track. I'll be interested to see how this evolves over time

    Thanks!
    James

    ReplyDelete
  4. Please do keep us updated on this in the future, I've already stuck you on my Google Reader so I don't miss out on any of your posts. You've got a really nice, honest style of writing, keep it up :-)

    ReplyDelete