An open letter to the Editor in Chief of Dr. Dobb's

Here is the letter I just sent to Andrew Binstock at Dr. Dobb's regarding this editorial:

http://www.drdobbs.com/architecture-and-design/the-corruption-of-agile/240166698

Hello Mr. Binstock,

I agree with your conclusions regarding the productization of Agile, and the zealotry regarding various sets of practices.  Within every community or body of knowledge, it seems, strong practices become the dogma of the enthusiastic, and then the core value gets lost in illogical debates where everyone is trying to be “right."

I’d like to take a moment of your time to provide some of my experience with the practice that you selected (likely not arbitrarily) as an example of the zealotry:  Test-Driven Development (TDD). Indeed, there is a lot of confusion and passion and zealotry regarding this practice.  I feel no need to defend, exactly, but I believe I understand much of the enthusiasm, and much of why most people can’t really articulate their love for TDD.  One reason why it’s so confusing even for those passionately in favor of TDD is that the real value appears as a “sub-practice” of TDD. (I’ll come back to that.)

I’ve been writing code since 1976, and in the “Agile” space since 1998, when I first experienced TDD (or “test-first” as it was known then).  Unlike many of my bright-eyed young Agile colleagues, I spent half my career crafting quality, nearly-defect-free code before that point, without TDD.  Here’s what I’ve noticed over time:


1. From “All in your head” to "fast feedback."

Pre-TDD, I spent a lot more time running various branches and permutations through my mind, or in a flowchart, or in a sequence diagram, and in review with others on my team.  All great practices, and necessary in the days when it took an hour to compile the code.

What I do with test-first coding is describe the outcomes of small pieces of behavior in my code, then allow those passing tests to provide confirmation of my investment in that behavior.  I no longer have to keep the permutations and possibilities in my head.  Great burden lifted.  We’ve long had compilers to provide a red flag when I make a syntactic mistake, and now fast tests provide similar near-instantaneous feedback when I mistakenly change run-time behavior.  To me, TDD feels like a natural progression of good practices for the computer scientist, and the professional developer.


2. From slow and expensive to fast and cheap.

The code I was building pre-TDD was probably a little more complex than Internet protocols, but not much. We did supercomputer protocol stacks for a variety of machines, some of them with 36-bit words and 9-bit bytes. Two modes: Character and binary, but character was also subdivided into translations between any two of ASCII, EBCDIC, and BSD.  Oh, how I would have loved a (really fast) implementation of something like Strategy or maybe there’s even a Bridge hiding in there?

Nowadays I find our products to be, in the aggregate, far more complex than in the 80’s: Voice-recognition, search, security, multi-platform, interesting business rules regarding financial transactions, regional laws, internationalization…  Far too much changing (or potentially changing) far too quickly for us to do more than paint broad-stroke architectures at the high level; and far too complex and dynamic for our developers to be doing code-and-fix-and-update-the-UML.-that-no-one-reads.

Overall, TDD now feels far more natural to me.  I play a game with my code:  “You can’t do this yet,” I say. My code replies “You’re right, I fail at that.” I then tell my code how to do that, and together we confirm that it works, and simultaneously confirm that I haven’t broken anything my code has learned over the years. Then I see if the introduction of that implementation is asking to be blended more cleanly into the code: Refactoring is the oft-neglected sub-practice of TDD. Each time I reshape the design to accommodate this new behavior or the next, I run the tests to let my code confirm that it hasn’t lost any capability.

In fact, “relentless refactoring,” i.e., the constant application of good design practices, is the real “must have” for Agile software development. How else adapt to fresh new unexpected requirements without being able to reshape code? But without some form of fast, automated, deterministic safety-net of tests, relentless refactoring is mere dangerous unprofessional hacking. Ergo, test-first coding.

I’m not attached to TDD or even “Agile" as static, dogmatic sets of practices for the very reason that I already know it was once less efficient than pure human brain-power. Someday we will look back and laugh at our zeal. But I think TDD has some good life left to it, because of the outcomes:  Comprehensive, executable engineering documentation describing all the discrete behaviors of the system in a non-combinatorial fashion; and the ability to change the code rapidly and confidently.

Thank you for your time and kind consideration.

Rob Myers