Background: Scrummerfall

Brad Wilson and others have blogged about a persistent “scrummerfall” pattern in agile transformations wherein, despite good-faith efforts on the part of teams to blast away cube walls, silos, and artificial role divisions, we still have programmers and testers working in phases. I’ve seen this repeatedly myself. Each little iteration is (at best) a two-phase waterfall project: programmers test-drive story implementations in isolation, then toss the stories over a tiny wall to testers in the same room, who write automated tests and do manual testing. And the testers find bugs not caught by programmers, in all the little nooks and crannies between systems, projects, frameworks, architectural layers, web service end points, yadda. This is not the only way in which crappy conveyance muda and role preservation shows up in iterations, but it is typical, and fresh in my mind.

Note that I am not arguing against the official role of tester on agile teams. To the contrary, I think the agile community at large tends to undervalue and misunderstand the value testers bring to agile teams and software development generally.

And on a side note, we could argue here about how useful non-isolation-testing is or is not, when we are supposed to be test-driving all of our code. We could argue about whether the need for automated integration tests are a measure of inadequate test-driving skill (which I do not actually believe).

But let’s not argue. Let’s instead agree that regardless of the story, its definition of done should include testing on which programmers and testers do some pairing. The story is not done unless programmers and testers have worked together on all the non-isolation testing that needs done. In a web app, this might include SeleniumRC or Watir testing. In any sort of app, it might be Story Testing of the kind we could do with Fit or Slim or RobotFramework or JBehave or EasyB or whatever. Let’s then also agree that our programmers should be learning more about testing generally, and we should be hiring testers who can program.

Most Good Agile Testers are Good Agile Programmers

Let me be more plain. Too few software testing professionals can program well. Agile Testing requires that most of its agile testers be programmers and, in my experience, pretty darned good ones. So, decent object modelers, pretty creative manual-step-automaters, pretty good build system massagers. Agility demands that testers evolve in the direction of solid, real-world development skill.

Hire Testers Who Are Good Programmers

So, please, stop hiring testers who can only do record/playback testing, manual Word doc test plan creation, etc, and have no programming ability. We have plenty of data now that whether your testers are in-house or off-shore, whether they have 2 years or 20 years of experience, most of them are going to need to be able to actually code once they join your agile teams.

This is not to say that programming is the most important or central skill a tester possesses. As Elisabeth Hendrickson recently commented, “Seems to me that professional testers do 4 things well: observe, notice what can be varied, predict risks, & explore methodically.” I agree. She and I and others wish to grow those skills as well.

But again, programming is one of the things good testers do well. And it is a means toward the above ends of noticing what can be varied, of predicting risks, of observing and exploring. And when not a means toward those things, it is an adjunct skill.

And in order to determine whether or not an agile testing candidate can actually code, you will need to test that candidate’s level of skill, using some kind of programming exercise.

And, presuming you are working in Java, I can help you with that.

More Background: an Agile Developer Coding Exercise

A bit of context: At Pillar Technology, I created a TicTacToe coding exercise (available on google code) for candidate agile Java programmers a couple of years ago. It has evolved and evolved, and we have seen several dozen programmers thrown through it. Those of you who know it, know, as I repeat constantly, that it has its flaws. Further, this is not the only hiring tool or technique we use, though we do try to use it for every programmer.

And I tell you what. Our ability to hire programmers with actual skill and experience in TDD, Refactoring, Simple Design, Clean Code, Design Patterns, and Software Craftsmanship generally has skyrocketed simply by throwing every candidate at this little exercise, and by having the right little community of Pillar programmers evaluate the code that the candidates write. And our cost per programmer hired has plummeted.

Where we had very little determinism and precision before, we now have some.

Finally: a Draft Agile Tester Coding Exercise

It turns out we also needed the ability to test how well agile testing candidates can code, at least in Java, in a particular context (Selenium RC web app GUI testing). I’ve now created another little google code project, this one for testing how well agile testers can write Java code for Selenium RC tests. It’s a very first draft (as of this writing), and it will need plenty of love and evolution and crusty criticism of the kind the other codebase got. I hereby solicit any and all, friendly or not. I just don’t want to get into conversations about whether this kind of automated testing is useful in the first place, since my operating premise is that it is.

The exercise currently uses as its System Under Test a fledgling Triangle Testing website under development by Elisabeth Hendrickson, to whom I owe thanks. Thanks, Elisabeth! As her triangle site SUT evolves, so may my codebase.

Anyone who you want to put through the exercise will need to be instructed to download and install both Eclipse (for example) and a Subversion client (like the Subclipse plugin for Eclipse). You’ll then need to instruct them to check out the coding exercise, and then finally to open and read the README file.  My intention is that from there on out, the exercise instructions are clear, starting with instructions for downloading and installing the TestNG plugin for Eclipse.

If you would like some instruction in how to interpret and assess the code your candidates write, please contact me separately.