Software Execs: Do You Have Toxic Code Assets?

Simple “Clean Code” Metrics for C-Level Execs

A recent Twitter thread I was involved in goes something like this. Someone claimed that  software managers and executives should not have to care whether their developers are test-driving Clean Code.  They should be able to presume that their developers are always courageous, disciplined, and skillful enough to produce that level of quality. Ultimately that quality turns into least Total Cost of Ownership (TCO) for any codebase asset.

Sigh. Well, would that that were true. To my mind, it’s like saying that all consumers should presume that all cars are built as well as Hondas. Sadly, they are not.

So, yes, of course, developers should take full ownership of the extent to which they create  Clean Code. Developers should own their own levels of skill, discipline, knowledge, passion, and courage. Absolutely so. Developers should refuse to cave to pressure to hack junk out the door to meet deadlines. That’s not what I am debating. I am debating whether or not the industry has accountability systems and quality monitoring systems to ensure that developers are in fact doing all of that.  My premise is that something like the opposite of that is going on.

If managers and developers are still being rewarded for hacking junk out the door, and executives and managers cannot and do not measure the TCO consequences, well then no wonder our culture of Software Craftsmanship is not spreading. We have a crappy incentive structure.

Are Hondas still better made than GM cars, all these years later, and despite quality gains on both sides? Of course. The car industry does, in fact, have accountability systems in place to measure asset quality, duty cycles, TCO. Too much money is at stake.

As a responsible car buyer, I inform myself with exactly the least info necessary and sufficient to determine whether I am about to buy a great car or a lemon. I have data to help me predict the outcome.

Managers and executives in software cannot expect that every codebase is as good as a Chevy, nor even a Yugo. Most enterprise software these days, 10+ years into an agile movement and Software Craftsmanship movement, is still gunk that continuously deteriorates.

And managers and executives cannot see that. They are buying lemon after lemon, not knowing any better.

We want managers of developers to insist on Clean Code, so we want them to be able to tell the difference between Clean and Mud, and to hire programmers who code Clean. And we want executives to hire managers like that. These inevitably will be executives who can distinguish between managers who know Clean Code and those who do not. I posit that these executives will in turn need to know how, at a very basic level, to distinguish between Clean and Mud. Only then can they preserve their asset value, and hire delegates who can.

Two Metrics You Can Use to Measure Code Asset Deterioration

At my current client, each team self-governs several kinds of objective and subjective Clean Code quality measures, including test coverage, cyclomatic complexity per module, average module size, coupling, etc. There are all kinds of details here around issues like automated deployment, test quality and semantics, etc. They don’t publish it all, they use most of it tactically, within the team boundaries, to hold themselves and each other accountable for continuous code improvement. The teams can and should own that stuff, and they do.

But you know what?  Each of these teams is also publishing at least two metrics to other teams and straight up the management chain for their codebase assets: test coverage and cyclomatic complexity per method.  The Continuous Integration plugins publish these metrics for all to see. And all any team is held accountable for is this: do not let these numbers slip between iterations. Anyone can see historical trend graphs for these numbers for any of the projects/codebases currently covered (there are several so far, and more each month).

Yes, these two measures are imperfect and can be gamed. Yes, test coverage is a one-way metric. But let’s presume for a moment that we are not hacking the coverage config to exclude huge whacks of yucky code, and we have good-faith participation on developers’ part. If average complexity per method goes from 4 to 6 over a two-week iteration, and if test coverage slips from 80% to 60%, does that not often mean that the codebase, as an asset, probably deteriorated?  My experience has been that it does.  As an owner of such an asset for which numbers had slipped like that, would you not care, and would you not want some answers?  I would, and I counsel others to care and dig in. I hereby counsel you, if you own such assets, to care if those two numbers are slipping from week to week. If they are, I bet you dollars to donuts your software asset is getting toxic.

A Culture of Accountability

So at this client, if those two metrics slip, teams hold each other accountable, and execs are learning to hold dev team managers accountable. Why not? Every car buyer understands MPG these days. Why not educate executives a little bit about how to monitor their code asset health?

Could there be a better 2 or 3 metrics to publish upstairs?  You guys are smart; you tell me. So far, these 2 are working pretty well for me. The published metrics are not sufficient to protect against asset deterioration, but so far they sure seem necessary.

So guess how this is turning out?  We are growing a top-to-bottom, side-to-side culture of Clean Code accountability in what once a garden-variety, badly-outsource-eviscerated, procedural-hacking sort of culture.  Partly by hiring lots of Been-There-Done-That agile coders, and partly with these metrics. Suddenly, managers who were only measuring cost per coding-hour (and slashing FTE jobs to get to LOW $/hour numbers) are measuring more meaningful things. Could we do better? Doubtless. Stop by, see what we are doing, and help improve it.

What metrics would you publish up the reporting chain, and between teams?  How would you help executives detect when their code assets are slipping into that horrible BigBallofMud state?

Speak up, all you shy people. ;)

Hiring Testers Who Can Code: An Exercise

Background: Scrummerfall

Brad Wilson and others have blogged about a persistent “scrummerfall” pattern in agile transformations wherein, despite good-faith efforts on the part of teams to blast away cube walls, silos, and artificial role divisions, we still have programmers and testers working in phases. I’ve seen this repeatedly myself. Each little iteration is (at best) a two-phase waterfall project: programmers test-drive story implementations in isolation, then toss the stories over a tiny wall to testers in the same room, who write automated tests and do manual testing. And the testers find bugs not caught by programmers, in all the little nooks and crannies between systems, projects, frameworks, architectural layers, web service end points, yadda. This is not the only way in which crappy conveyance muda and role preservation shows up in iterations, but it is typical, and fresh in my mind.

Note that I am not arguing against the official role of tester on agile teams. To the contrary, I think the agile community at large tends to undervalue and misunderstand the value testers bring to agile teams and software development generally.

And on a side note, we could argue here about how useful non-isolation-testing is or is not, when we are supposed to be test-driving all of our code. We could argue about whether the need for automated integration tests are a measure of inadequate test-driving skill (which I do not actually believe).

But let’s not argue. Let’s instead agree that regardless of the story, its definition of done should include testing on which programmers and testers do some pairing. The story is not done unless programmers and testers have worked together on all the non-isolation testing that needs done. In a web app, this might include SeleniumRC or Watir testing. In any sort of app, it might be Story Testing of the kind we could do with Fit or Slim or RobotFramework or JBehave or EasyB or whatever. Let’s then also agree that our programmers should be learning more about testing generally, and we should be hiring testers who can program.

Most Good Agile Testers are Good Agile Programmers

Let me be more plain. Too few software testing professionals can program well. Agile Testing requires that most of its agile testers be programmers and, in my experience, pretty darned good ones. So, decent object modelers, pretty creative manual-step-automaters, pretty good build system massagers. Agility demands that testers evolve in the direction of solid, real-world development skill.

Hire Testers Who Are Good Programmers

So, please, stop hiring testers who can only do record/playback testing, manual Word doc test plan creation, etc, and have no programming ability. We have plenty of data now that whether your testers are in-house or off-shore, whether they have 2 years or 20 years of experience, most of them are going to need to be able to actually code once they join your agile teams.

This is not to say that programming is the most important or central skill a tester possesses. As Elisabeth Hendrickson recently commented, “Seems to me that professional testers do 4 things well: observe, notice what can be varied, predict risks, & explore methodically.” I agree. She and I and others wish to grow those skills as well.

But again, programming is one of the things good testers do well. And it is a means toward the above ends of noticing what can be varied, of predicting risks, of observing and exploring. And when not a means toward those things, it is an adjunct skill.

And in order to determine whether or not an agile testing candidate can actually code, you will need to test that candidate’s level of skill, using some kind of programming exercise.

And, presuming you are working in Java, I can help you with that.

More Background: an Agile Developer Coding Exercise

A bit of context: At Pillar Technology, I created a TicTacToe coding exercise (available on google code) for candidate agile Java programmers a couple of years ago. It has evolved and evolved, and we have seen several dozen programmers thrown through it. Those of you who know it, know, as I repeat constantly, that it has its flaws. Further, this is not the only hiring tool or technique we use, though we do try to use it for every programmer.

And I tell you what. Our ability to hire programmers with actual skill and experience in TDD, Refactoring, Simple Design, Clean Code, Design Patterns, and Software Craftsmanship generally has skyrocketed simply by throwing every candidate at this little exercise, and by having the right little community of Pillar programmers evaluate the code that the candidates write. And our cost per programmer hired has plummeted.

Where we had very little determinism and precision before, we now have some.

Finally: a Draft Agile Tester Coding Exercise

It turns out we also needed the ability to test how well agile testing candidates can code, at least in Java, in a particular context (Selenium RC web app GUI testing). I’ve now created another little google code project, this one for testing how well agile testers can write Java code for Selenium RC tests. It’s a very first draft (as of this writing), and it will need plenty of love and evolution and crusty criticism of the kind the other codebase got. I hereby solicit any and all, friendly or not. I just don’t want to get into conversations about whether this kind of automated testing is useful in the first place, since my operating premise is that it is.

The exercise currently uses as its System Under Test a fledgling Triangle Testing website under development by Elisabeth Hendrickson, to whom I owe thanks. Thanks, Elisabeth! As her triangle site SUT evolves, so may my codebase.

Anyone who you want to put through the exercise will need to be instructed to download and install both Eclipse (for example) and a Subversion client (like the Subclipse plugin for Eclipse). You’ll then need to instruct them to check out the coding exercise, and then finally to open and read the README file.  My intention is that from there on out, the exercise instructions are clear, starting with instructions for downloading and installing the TestNG plugin for Eclipse.

If you would like some instruction in how to interpret and assess the code your candidates write, please contact me separately.