Something--probably my general principle about "never being an absolutist"--bothered me about the statement. We had a private discussion later, and we agreed that I could summarize here, and we can openly debate, if necessary, right here on my blog. And you are welcome to weigh in. It could turn into a hotly debated topic, or just a tempest in a teapot.
I'm also breaking this into multiple parts, because I think there are three layers to the decision:
- The choices available regarding iteration-wise scope-cutting.
- How to use process and practices to help you make the right choices.
- The process-coach's role in these choices.
A polite disagreement with a colleague with as many years of Agile-related experience was no surprise to me. I suspect that we may both learn something. If you’re new to Agile, well…you may learn something too, just have some antacid handy. And rest assured, I believe Agile Bob and I agree on the core principles involved.
“We’re Behind!”
Let me set up a common scene for you: The team is half-way through a two-week iteration. To make the math easy, let’s say the team’s velocity was previously measured at 20 points. The last story scheduled for this iteration is called “Plugh” and it’s been estimated at 4 points.
On that fateful 6th day of the iteration (we try not to count weekends!), the team sees that they’ve completed 8 story points. The burn-up graph clearly shows the trouble ahead: At this rate, the team would complete 16 points, with a shortfall of 4 points. Plugh would not get done.
What are the team’s options? Two possible options come immediately to mind:
A. We acknowledge this and keep going.
This course of action results in two possible outcomes: Perhaps Plugh fails to get completed, or the team works (perhaps heroically) to get all commitments finished. (This is perhaps the “default” Scrum approach. This is also Agile Bob’s preferred choice.)
B. We ask the Product Champion (Scrum "Product Owner" or XP "Customer") to drop Plugh (or split it, but we’ll avoid shades of gray for simplicity’s sake).
Again, two similar outcomes: The iteration completes without Plugh, or the team finishes early and pulls Plugh back in and completes it. (This may be the “default” XP approach.)
You’ll notice that both options end up having the same two possible outcomes, at least on the surface. Either Plugh is done, or it’s not. The reality of whether or not Plugh gets completed in this iteration is not obviously affected by our choice.
The choice could affect the outcome of the iteration indirectly: Will the team choosing Option A work effectively, keeping quality in mind, if they opt for a heroic effort? Will the team choosing Option B remember to ask for more work if they finish early?
The choice could also have longer-term effects on the quality of the release, and team morale and trust. To me, those are the more significant considerations for long-term team performance.
Agile Bob’s Argument
At the heart of Agile Bob’s argument for Option A is a legitimate concern over the message we (management and coaches) are sending the team, and what habits the team is forming. In the course of our brief conversation, Bob put forth two arguments, one in favor of Option A, and one against Option B:
In support of Option A, Bob explains that no one likes failure: Let the failure to implement Plugh be examined in the retrospective, and the team will adjust to avoid over-commitment in future iterations.
Cutting scope (Option B) relieves the team of responsibility for Plugh for two weeks. In the next iteration, they may complete Plugh, but the “Plover” story may suffer. If this continues unabated, scope is slowly sliding away from the release plan due to the habit of sheer laziness.
I agree with both of these as concerns, or considerations. The issue I have with this is the suggestion that one choice works as an absolute rule for all teams. Perhaps Option A is the right choice for his team at this time. But I see enough value in Option B to resist choosing one over the other prior to considering more context. I want to encourage the team to reflect honestly on their own habits and tendencies.
Both of Agile Bob’s arguments can, by my direct experience, be falsified. I’ve worked with (and on) teams who handled iteration-wise scope-cutting as a rare but useful technique, and I’ve worked with a number of teams who have made a bad habit of failure, at least until the real cause of dysfunction was identified and corrected.
“A Habit of Failure?”
Yep.
First, let's not confuse this with a good habit of failure. The technique called "fail fast" is inherent in Test-Driven Development, and in Agile processes: If you're headed in the wrong direction, you want to know it as soon as possible, ergo "fail fast!"
Perhaps a more accurate description of what we're talking about here would be: Making a habit of over-promising and under-delivering, without any real learning or improvement.
Any habit of over-committing is a habit of failing the iteration.
Consider "velocity" (the number of story points completed per iteration). Anything that artificially increases velocity can become a habit of over-commitment.
I recall a conversation with a team after their first iteration. They had scheduled 20 points but completed 12. I suggested that they simply schedule 12 points for the next iteration.
"But that was our first iteration." In the next iteration, they scheduled 20 again, and completed 12.
"But the architect was out sick for half the iteration!" They scheduled about 20, and got 12.
"But the test-system was out for repair..."
The PM looked at me and asked, "How am I supposed to plan for all these [expletive deleted] surprises?" and I began with "Well, your velocity is 12..."
Any time a team makes a plan that assumes there will be no unpleasant surprises, or the team talks about "increasing the velocity," or tries to "get some points" for a partially-completed story, there is already a subtle habit of failure in place.
And that's when everyone is trying to do the right thing! Sometimes we're not so fortunate. Many teams in transition from a waterfall-flavored process to an Agile-flavored process will discover that they've developed a taste for failure. More accurately, a fear of success. A single fearful person can do some hefty damage to the organization, whether or not it's intentional.
Comfortably Numb
This unconscious sabotage arises from a small part of the team (perhaps one or two people) who have not yet reached a sufficient comfort-level with all of their“Agile Transition” responsibilities. People who are used to doing something their way (e.g., “I must see all the requirements before I can begin design”), and truly do not realize their negative impact on the team or the organization, and thus on themselves.
So these individuals think (or feel): “The sooner this 'Agile' stuff fails, the sooner I can get back into my comfort zone."
Using our example of iteration failure, we can add more detail: "If we let the iteration fail, we will then commit to less next time, and I can have more time to do my work using the [inefficient and cloaked] practices I’ve been using for years. What’s the hurry? We’re not really releasing in October. They’ll slip the date, just as they’ve always done…”
This same argument could be used with either of our options, A or B, but there’s a difference in perception between a tool for "sullying the opinion of Agile within the organization" (a possible perception of Option A), and a tool for "relieving some of the temporarily heightened pressure of an over-committed iteration" (a possible perception of Option B).
Perhaps it's just my brain's preference for bizarre analogies, but Option B strikes me as a "safety valve" and Option A looks more like a "toilet about to overflow." Either can be handled responsibly or otherwise, but as far as analogies go, I have my preferences...
Whatever their perceptions or analogies, the activities of these subtle subconscious saboteurs cannot hide in a highly-iterative agile environment. We will notice the effects in the stand-up meetings and the retrospectives.
"So...Fire Them?"
On one of my favorite XP teams, we had an intentional agile-saboteur: I have no doubt that he consciously knew what he was up to.
This gentleman was a senior developer who had been working on the legacy product for a decade. He was nearing retirement age, and would be able to pull in a comfortable pension. His expanded team was rewriting the system in Java using the XP process.
Can you imagine his feelings? He had poured his heart, soul, and career into "The Product" only to have a team of young know-it-alls come in and rewrite it in a language named after a beverage using some newfangled "Extreme Snowboarding" process (where no one would leave him alone long enough to think). He probably resented the fact that he was going to be forced to learn a whole set of new techniques that he would never use again. And if he didn't play along, he was worried they would find some reason to fire him, and he would lose his pension.
(He never expressed any of this. Did I mention that being an "Agile Coach" requires a small amount of empathy?)
More fear, rather than making transition easier, makes people do stupid things. Like sitting with his arms folded, literally rolling his eyes (and letting out an occasional loud sigh), but otherwise saying not a single word during a two-hour pair-programming session.
Right now, you think I exaggerate! Nope. I endured two hours of sighs and eye-rolls, I kid you not! I was trying to outlast him. (In hindsight, it was the wrong strategy.) At first I tried very hard to engage him in conversation, to bring out some of the wisdom locked away in his brain, and (frankly) to get him to like me, just a little. After that I just test-drove my code, and perhaps looked a little closer at the screen whenever I heard a sigh.
Unsalvageable relationship? Fire him? Actually, it was the PM who mentioned this option during a private conversation about the man, but only to dismiss it because he was very well liked and respected in the organization, and he knew the system better than she did.
And then the lightbulb appeared above our heads, hers and mine, simultaneously.
Legacy code, for all its faults, contains a lot of business knowledge (obscured and obfuscated, but it's in there somewhere). Giving this guy the "XP customer" role gave us access to that expertise, gave the PM more time to do her work, and gave him a heightened sense of participation and control over the robustness of the product.
Actually, he and the PM shared the customer role, because she would weed out frivolous stories and prioritize the rest. But he would frequently point out business rules that everyone else had forgotten, and we'd quickly write down another user-story. Sure, we had to put up with the occasional snarky "Well, obviously you've forgotten about this failure case..." (Have I mentioned that an Agile Coach has to be part diplomat?)
"Just Give Them Whatever They Want?"
We should try to give people the benefit of the doubt. When the organization is transitioning to the win-win-win game of Agile software development, it can take a while for people to see the benefits to themselves as employees and teammates (one of the “wins”). People aren’t trying to hurt themselves when they resist change. They’re mostly either confused, skeptical, or tired of changes that have generated no tangible improvement. We have to be clear about the potential benefits to employees, teams, and the organization; and we have to set reasonable expectations about our progress towards particular levels of agility.
We have to look for the right changes for the right person, at the right time. Find out what motivates negative behavior, and refocus that nervous, destructive energy towards positive results. This will result in a much happier and productive team.
I’m not one to promote coddling the team, or limiting constructive feedback, during a transition. Imagine if we chose Option B and the Product Champion simply said “Every iteration is a success! You completed less than last time, but you did a great job!” Ack! That's a watered-down definition of "success" that will lead to dysfunction. We don't congratulate the team for cutting iteration scope, but neither do we punish them by similarly overusing the word "failure." Let's save the word "failure" for real, impending failures.
Let's give the teams the tools they need, and help them decide when and how to use them. One of those tools is velocity. Velocity is measured to plan, and to diagnose, like body temperature. We neither scold nor reward our children for having a fever. Fever is a symptom, and “my kid is sick” is just today’s reality.
Besides, the team is not a collection of people playing the role of children; with managers, executives, and coaches playing parents. The Agile team is the whole team of professionals (including developers, testers, tech writers, analysts, managers, execs, coaches) who work together to identify problems early and to adjust accordingly.
Stay Tuned...
That's enough to start with. Part II (working title "Making the Lean Choice") is just a week or two away. Here's further related reading, from people I really trust:
Agile Bob's blog post mentioning iteration-wise scope-cutting as a form of mediocrity.
Jim Shore's blog post on the three forms of success. He talks about sabotage, as well. Jim despises mediocrity, too.
Esther Derby wrote an article for Better Software Magazine that talks about self-organizing teams. It seems relevant to this post.
Rob, I think part of our difference of opinion was also based on our coaching styles. You tend to embed with the team enabling you to monitor the iteration in a more hands-on fashion. I tend to do more touch-point coaching which means the team is expected to do the right thing without me and then at touch-points (planning meetings, retrospectives, etc.) we sort things out.
ReplyDeleteBecause of my touch-point coaching, when the team takes option B I tend not to see how the crisis evolved. However, when they take option A I can examine their results with them and help them understand whether they ended up delivering maximum value.
I think this has more impact than either of us realized during our initial discussion. It obviously leads to another discussion about which coaching method is most effective, but that gets to be a circular argument because both work when done properly.
I'm probably opening a bigger can of worms than I want with this next question, but is it important to management to know the team didn't actually make their commitment? If so, choosing option B can easily hide that fact in future metrics. I know many executives like to see how their teams are performing, and if they have a velocity of 20 and suddenly have a "successful" iteration with only 16 points it might raise some questions.
I will add one more issue about option B before ending this comment. I wonder whether Parkinson's Law comes into play when a team chooses option B. You wonder if the team will remember to ask for more work under option B and my concern is they will not even have the ability to ask for more because what they have left will fill all available time. I don't know the answer, but I do know option A doesn't have it as a potential problem :-)
- Bob -
Agile Bob,
ReplyDeleteI'll defer comparing touch-point (aka fencepost) coaching to embedded coaching for a later post. I, too, felt that it was a related, but significantly separate, issue.
If the team chooses to cut scope, there will be a drop in the top-line of the burn-up chart, and a visible drop in the velocity. There's no place for Plugh to hide. Much of this may ride on the size of stories. Is it a crisis if Plugh doesn't get done, or just a small dip in the velocity. I set up the hypothetical so that Plugh was a significant portion of the velocity, so there is certainly some reflection required.
And, again, if the execs note a 20% drop in velocity, that's the same indicator whether or not we call it a "success" or "failure." I'm suggesting we keep the terms neutral in order to foster trust. Again, this may be something to consider on a team-by-team basis, but teams eventually deserve to have planning metrics and performance metrics measured separately, and words like "commitment," "success," and "failure" be used at the release level, but not necessarily at the iteration level. It strikes me as too much emotional baggage around a week or two of work.
That's not to say that we don't examine and evaluate the iteration during the retrospective. And though we'd prefer that the transition from one iteration to the next be a celebratory time, there's no reason to fake it.
Yes, Parkinson's Law can become a hidden habit of failure with Option B. But overcommitment can become a habit of failure with Option A: I worry that corners may be cut if there's too much pressure. That's why teams need us coaches as external observers. I like to employ other techniques for heightening urgency, such as short iterations, continuous integration, and even pair programming. People tend not to let the work fill the time if that means having to watch someone else spend an hour responding to his e-mail. You can literally hear the difference between a team that's running at capacity, and one that's lost its momentum, or is overworked.
And if it's not we the coaches who are there to hear that change, perhaps we need to coach the PM/ScrumMaster to learn the various personalities of their team.
Rob,
ReplyDeleteI prefer Option A (without any heroic effort) mostly because it's simpler. It's simpler because it doesn't require any extra decisions on the part of the product owner.
Both options reveal important information information, namely that "our velocity went down."
Many Agile practitioners seem to think that the primary goal of estimating is to maintain a constant velocity. I don't agree. The main purpose of estimating is to provide the customer with a predictable schedule. The schedule is predicted in the same way whether the velocity changes or not.
It's interesting to note that the same result could have been achieved in this example if the team had only committed to 16 points in the first place. Call that Option C.
Options A, B and C all lead to the same fact: The velocity went down. I like the idea of letting the velocity speak the truth of the situation.
Steve, you are absolutely right: All options have approximately the same results. I might call Option C the "prescient" option in this hypothetical case, because something made the team choose not to put Plugh into the iteration despite their velocity's "yesterday's weather" prediction that they could get it done.
ReplyDeleteOriginal, "Dogmatic" Scrum, as I understand it, suggests that the team commits, and then is left alone until the end of the sprint. "Dogmatic" XP suggests that the Customer (aka Product Owner/Product Champion/Decider) is always readily available. I used "dogmatic" in quotes to suggest that insisting on either is really a good way for the coach to get booted from the team: The reality for actual Agile teams falls somewhere inbetween. If the Customer is present, she will see the same information as the rest of the team, and may suggest a course of action. As a team, we should react to the information at hand. We should certainly wait until the last responsible moment, but then act decisively. A 20% drop in velocity indicated 50% of the way through an iteration seems to be a red flag to me, but that's my opinion. If the team sees it as a concern, I think they should not ignore that concern.
If I implied that I have a preference for Option B, that may certainly be the case, but I mostly wanted to defend it as an option. I prefer that the team decide, and since the PO/Customer is part of the team, Option B could be initiated simply by the Customer looking at the charts and saying "Hey, don't worry about Plugh," and walking up to the board and taking it down. If I implied there was more formality than that, forgive me. People can't *not* make decisions. Coaches have to help teams gather good information, and make informed decisions.
A lot of this also depends on how well the team is adapting to agile disciplines. Agile Bob mentioned Parkinson's Law ( http://c2.com/cgi/wiki?ParkinsonsLaw ). A team that is running like an Agile Bullet Train will occasionally finish early and ask "What's next?" A team whose train is just leaving the station, so to speak, may have an iteration where nothing gets measurably done. The recommendations I would make to these two teams could vary considerably.
I worked for a client who gave me a dozen teams to work with, including mostly their best and their worst. One of the PMs I worked with was managing both a "very best" and a "very worst" team, and she noticed that my suggestions for these teams would sometimes be near-polar opposites. But she also realized that she was probably the only one who would notice. ;-)
Steve, you already know that we have to go back to basic Lean principles and the Theory of Constraints when tweaking the process for our teams. No single set of Agile rules will apply in all situations.
Hi Rob,
ReplyDeleteI hate the comment engine on your site. There. I said it.
Here's the comment I can't post, because the textbox will not allow cut-and-paste. I cut-and-paste because I wrote the comment in a text editor. I wrote it in a text editor because my first comment was eaten. :-b
----
Rob,
(My previous comment seems to have been eaten. Grr.)
To briefly repeat myself: I prefer option B, even when I'm doing "touch-point consulting" of the sort Bob describes, because I want the team to remain in control of their iteration at all times. If they aren't going to make it, I want them to have a frank discussion with their on-site customers and adjust the plan so they still release something of value. Maybe that means dropping story "Plugh," but sometimes it means more significant changes.
Furthermore, my teams don't miss their iteration commitments because they're lazy; they miss their commitments because they overcommit. One of the hardest things they have to learn is how to let go of their fervent hope and wish that things will go perfectly. Replanning the iteration based on real-world evidence is a lesson towards this goal.
Jim
Sorry, Jim. Apparently you can copy/paste if you have one of the Blogger-approved profile logins. I have a gripe out to Blogger. I think copy/paste should always work, since I have that word-verification stuff activated, too.
ReplyDeleteAgile Bob mentioned that my blog occasionally crashes IE. (Or, more accurately, IE cannot handle the script from one of my Blogger widgets.) There are many reasons why I'm considering a transition to WordPress. That story just hasn't fallen into an iteration yet! ;-)
I'm going to suggest a more nuanced interpretation of Scrum's position and then come down that I like option B too.
ReplyDeleteI think of Scrum using a two-level commitment. The deeper commitment is to the sprint goal. The surface commitment is to the particular stories. Like any commitment, if we hit a point where we don't believe we can meet the commitment, we need to take action. In the best case, a team will seek to find a creative solution to the problem. If they can't find a way to make the stories go, they'll try to find a way to still meet the sprint goal. This is in collaboration with the Product Owner. If they can't mutually work out something, cancel the sprint and report the failure to management.
In the best cases, the team responds to the pressure with creativity. If their default response is to fake that by cutting quality or cutting quality of life, they'll be in trouble in the medium to long term.
My default stance is like your option B, with an emphasis on splitting. But I've also worked with teams that only work on a story or two at a time, and just treat iteration boundaries as planning/retrospective waypoints. (In effect their commitment is to work steadily rather than to a particular set of work.)
--Bill Wake, xp123.com
I say option B is better than A as explained well by Bill Wake.
ReplyDeletePeter
CSC, CST
www.scrumsense.com
Here's a related post by Ron Jeffries, examining kanban:
ReplyDeletehttp://tinyurl.com/llr55t
(...on his blog named after the 2nd Puppeteer ship to visit Ringworld. Ah, the geekiness! ;-)
Isn't "never be an absolutist" an absolutist philosophy?
ReplyDeleteInteresting topic!
ReplyDeleteI think it's generally the team's decision, but I encourage them to decide based on what they actually think will happen, without a lot of regard to what they wish would happen.
In practice, I think that means picking option B a lot. As Brooks says, all programmers are optimists. If they are starting to worry that the plan isn't correct, they should default to making the plan more accurate.
I also believe that teams generally do better when they experience a series of small wins, so dropping back to 16 has benefits. If they hit the 16, then they win for fixing the plan as early as possible. If they make 20, then they've pulled something in and won by stretching a little.
If a company pushes for option A because they like to keep pressure on, I think they're making a big mistake. If developers aren't excited to work on things, then there's an underlying problem that should be exposed and addressed, not covered up through the use of fear and pressure.