Agile Programming: Lesson One (Part Three)
At the start of this series of posts I suggested that those who really, really want to become Agile Programmers start with a couple of refactoring exercises. In another post I explained why I suggest this as a starting point. I’ll reiterate here that I am suggesting that you perform those exercises several times, several ways, before you attempt the exercises coming up next.
All done? OK. Cool. Now, three posts in, I’ll introduce the ideas of microtests and Test-Driven Development (referring you to other sources for deeper dives into these large topics).
If you have been following along in code, as well as in your reading, you may by now have discovered and learned several things. Here are a few, any one or two of which may have occurred to you:
- You would not really be safe trying to do those original refactoring exercises without the handful of tests that cover them.
- It’s not hard to break existing tests if you keep running them after each little refactoring step
- The refactoring steps in the exercises can be pretty hard to do, even in those really small amounts of code.
- As you extract behavior into lots of new methods, and perhaps into a few new classes, it would be really cool to have new, separate JUnit tests that cover those specific little behaviors.
- It can be tricky trying to determine how to keep test code itself clean and concise.
I suspect that after enough repetitions of those first exercises I suggested, one or more of these does occur to you. Let’s dig into testing in the context of these kinds of observations.
By Microtests, which term was coined by Mike (@GeePawHill) Hill, I mean the same thing that people like Mike Feathers and J.B. Rainsberger (and many other TDD thoughtleaders) mean by unit tests or isolation tests or programmer tests. So in Java, a JUnit test is a Microtest if it:
- Tests a single discrete behavior/responsibility (“in isolation”)
- Runs completely in memory: does not touch databases, GUI frameworks, file systems, or networks. Those other tests are frequently useful, but they are not microtests.
- Runs really, really fast. Fast enough means somewhere between 10 and 100 per second, or faster.
Vital Ends: Enough Microtests for a Production Codebase
Every production codebase desperately needs to have enough microtest coverage.
Your codebase has enough microtests if your code coverage (as previously discussed) is at least 85%, and your median test-method length and production method-length is really small (as previously discussed), and each test class and test method is itself SRP-compliant, and very experienced microtesters have looked over the entire test suite and pronounced it healthy.
There are leaps of faith for you to take here, admittedly. And that last clause is really fuzzy and subjective. I get it.
Nevertheless, I’ll emphasize this:
Aside from the production code itself, there is no more important artifact for any codebase than its suite of microtests. A codebase without enough microtests, as Feathers has pointed out, is by definition Legacy Code. Agile Programmers are careful about microtests the way veteran skydivers are careful about packing parachutes.
Again, an exhaustive suite of microtests are ends, worthy in their own right, for protecting us from introducing defects when we change code.
But as is very frequently discussed in agile circles, Test-Driven Development (TDD), as a means to those ends, is important in other ways.
Definition: Test-Driven Development (TDD)
Test-Driven Development is the practice and craft of writing tiny little microtests for production code before you write the production code itself. The practice has been covered exhaustively in other sources, so I’ll only sketch it here, starting with its benefits. Again, prepare to make leaps of faith:
- TDD is the most efficient, least-expensive way to produce high-ROI, low-TCO suites of exhaustive microtests.
- TDD is the most efficient, least-expensive way to drive production code in the direction of high-ROI, low-TCO Clean Code.
- TDD and microtesting have layers of sophistication, auxiliary practices and techniques, and otherwise a world of richness of their own.
- Skilled TDD (and refactoring) completely change how you feel about programming, for the better.
- Only after you have written a few thousand microtests using TDD (and other practices) do those first four points above become experientially clear.
- For many programmers, TDD seems at first (perhaps for months) to be a maddening, fanatical waste of time. It feels (to some) like walking backward up a staircase, blindfolded, carrying a tray of expensive flute glasses full of expensive champagne. Senseless, in other words. But that does pass.
Yes, you can write tests last, after you have written production code. But in exercises coming up soon here, I hope to provide you an opportunity to learn that this slows you down more than you can tolerate. I’ll also explain why that seems to be true.
One more thing: it can be very, very difficult to do real TDD, or to add or change code at all, in existing codebases with very low code coverage. Before continuing with this post/lesson, if you doubt what I just said, please consider attempting this exercise I created a few years ago. Then come back here and continue.
The TDD References
Aside: My fave, fave intro to TDD online is @JamesShore ‘s “Let’s Play” series. James models to us all expertise, poise, fun, and best of all, supreme openness and vulnerability. Also the desire for a pair! You can follow these videos and learn enormous amounts (the pace of learning is different from what I am attempting here — it’s OK if you find yourself lost by him pretty fast; if you revisit these next year you likely won’t be). You can also hear his cat whining about TDD in the background, if you listen carefully.
In all seriousness, in the opinion of some, James dives quickly into “TDD by the book,” which is to say with very, very little up-front design. Note that, as James proceeds, given his level of skill, this is not a serious problem for him. Master that he is, he can refactor himself out of any design misstep. This is not TDD at a novice level. Also notice in this series of videos how deeply James dives into mastering the problem domain at hand. It is his deepening mastery of that domain, coupled with his TDD mastery, that allows his design to emerge in increments that might at first seem ungainly, but are in fact elegant.
TDD is also covered in a slew of great books. Good ones in Java include (at least)
No, you need not purchase and read all of these books. Any one of them is likely sufficient to challenge you with Java TDD exercises and thinking for some time. And you need not purchase any of them this minute. But you will need at least one of them soon, both for tutorial and for reference. There is a completely different slew of books for test-driving in other languages, BTW. [More later.]
The Classic Red-Green-Refactor TDD Cycle
The classic TDD cycle is covered well, in all seriousness, in the wikipedia TDD write-up. When you are learning TDD, taking this grand leap of faith, stick to this cycle like glue. Follow it blindly for a good long while. Internalize it.
If you need a little humor to keep you on track, you can use this handy on-line “pair partner” to remind you where you are in the cycle.
A New Rule to Follow in Our Challenges
We’ve been working in the BankOCR Kata for awhile now. We’ve been refactoring existing code. As we go, we’ve been adhering to certain rules (paraphrased here, for our next challenges):
- Keep all tests running green as much as possible. Also try to purposefully break them sometimes, to see what kinds of coverage they give you in the code (more on that later, too).
- Rename each entity (project, package, class, method, variable) at least twice, as you learn more and more about what it should be doing
- By the time you are done, no method should contain more than 8 lines of code (this also applies to test methods), and most methods should be in the neighborhood of 4 or 5 lines of code, including declarations and return statements. This includes test methods.
- Don’t let code coverage fall below 85%.
As we switch to test-driving brand-new code, the rules above still apply. To this I’ll add a new rule, for the test-driven code:
- Test-drive classes that contain only a single public method, no more than 3 fields, and no more than 4 private helper methods. Try to avoid accessors (getters and setters). Can you avoid them? If not, why not?
Your Next Challenges
Start by ignoring the design you have been working with in the first exercises. Empty your mind of design ideas for the BankOCR Kata. Start with this new, nearly fresh version of an implementation, which can now help you with the special love provided by the Whack-A-Method testing mechanism.
Try test-driving just one part of the BankOCR Kata at a time, and in this order (for now):
- Test-drive code that parses a bunch of raw OCR-like String data (represented however you like) into separate OCR digits. Remember the input data looks like this:
_ _ _ _ _ _ _ _
| | | _| _||_||_ |_ ||_||_|
|_| ||_ _| | _||_| ||_| _|
- How many classes does this need? I mean, if they are really, really small, SRP-compliant classes?
- Next, test-drive code that converts those individual OCR digits into integers. Again, how many classes do you need? How do you know?
- Finally, test-drive code that converts groups of those OCR digits into proper account numbers.
Now, after having done that the above way, throw away all the code, and do it in reverse. Is that harder? Easier?
Remember to follow all of the above rules. And stay tuned, next, for some more deep dive into emergent design: the magic thing that happens as we begin to master TDD, Clean Method Hierarchies, and Clean Code generally.