The following post was originally published by the Cutter Consortium as an e-mail advisor for the Data Insight and Social BI practice  in December, 2011. Hence, the reference to the holiday season. That publication can be found here.

In honor of the holiday season, this month’s topic is focused on an all-too-familiar problem: the common hangover. No, I’m not speaking of an “adult beverage” hangover, although if I find a cure for that I’ll be sure to share it with you. The type of hangover I’m speaking of is the routine failure of your agile delivery team to complete the user stories it commits to during sprint planning.

I’m surprised at what a common occurrence this is for many agile analytics teams. Here are a few example scenarios:

1. The team completes development but testing hangs over into the next sprint.
2. The team completes the data integration elements of a story, but the demonstrable corresponding BI features hang over to the next sprint.
3. The story is too big to be completed in a single sprint, so the team just allows it to hang over into the next sprint.

There are other examples, but you get the idea. A hangover is not the same as a BI feature that is technically done, but the customer review calls for further modification or refinement (it isn’t “Done! done!”). A hangover is simply a failure to have the completed story ready for customer review at the end of the sprint.

Back in 1975, software engineer Fred Brooks observed that, “A project gets late one day at a time.”¹Similarly, an agile project gets late one iteration at a time, when teams fail to honor their sprint commitments. During project chartering/planning the entire project community speculates on an anticipated project outcome. While this outcome is subject to the routine fluxuations of requirements uncertainty, technical uncertainty, and risk, it is hindered by fluxuations caused by the failure of the team to meet shared expectations.

Agile development works in large part because we are continuously realigning the expectations of management stakeholders, customers, and delivery team members. These three constituencies collaborate during sprint planning to ensure that everyone has a shared understanding of the expected sprint outcome. They also collaborate at the end of every sprint to validate that the results are valuable and acceptable. Many times they collaborate during midsprint to provide guidance and clarity.

Delivery teams must accept that customers and product owners know what is important and can focus the team on the next most important stories. Customers and product owners must accept that the sprint outcome is constrained by the team’s limited capacity. However, when delivery teams fail to honor their end of the bargain or when the customer team puts unreasonable pressure on the delivery team, the trust relationship between customer community and delivery team erodes. The customer community and product owner can no longer rely on the team to meet their mutual expectations, and the process loses predictability and reliability.

The root of the problem is what Jim Highsmith refers to as “wish-based planning” instead of “capacity-based planning.”² The team succumbs to wishful thinking, overly optimistic estimating, and the pressure to do more than is realistic. Moreover, they fail to base their plans on previously demonstrated actual capacity. The problem snowballs when the team does this routinely sprint after sprint after sprint. Anticipated project outcomes are no longer valid, but they cannot be replaced with any degree of certainty.

So, now to the cure for the common hangover:

Ingredients Include

• Two parts acknowledgement
• One part self-discipline
• Two parts effective user story decomposition
• Two parts shared responsibility
• One part team honor
• Two parts celebration

Recipe

1. The customer team (including product owner) and the delivery team collaboratively acknowledge the symptoms of hangover. The delivery team acknowledges the undesirable consequences of wish-based planning and routine hangover. The customer team acknowledges any undue pressure placed on the team to plan beyond its capacity. Both teams recommit to the principle of continuous delivery of business value at a sustainable pace, and within capacity.
2. The delivery team explicitly recognizes that with the freedom of self-management comes the responsibility of self-discipline. Agile team self-discipline calls for moderation and self-control. The delivery team recommits to more thoughtful estimating and more realistic sprint planning. This involves pushing back on the pressure to overcommit, while still focusing on the delivery of high-valued, production quality working BI features.
3. Both the delivery team and the customer team seek to improve the art of user story decomposition. The customer team learns to see value in small, simple, immature, early working BI features rather than always expecting fully embellished and mature BI reports. The delivery team uses story-driven development to slice the architecture as thinly as possible, doing the minimal amount of data modeling, database development, ETL development, BI development, and so on, needed to complete the story. The delivery team does no more than is necessary for each story, and learns that each incremental change in the implementation is not rework, but instead is an opportunity to evolve a high quality DW/BI system.
4. The customer team takes responsibility for helping the delivery team plan within its capacity, and the delivery team takes responsibility for making realistic sprint commitments and keeping those commitments. Both teams take responsibility for the mutual goal of establishing a steady and effective cadence of successful sprints.
5. Despite Steps 1 through 4, the delivery team may occasionally still overcommit during sprint planning. When this happens, the delivery team must make every effort to honor those commitments, even if it means working at an unsustainable pace for a short period of time. Afterwards the team must seek to rebalance its planning for the next sprint to achieve a more realistic commitment.
6. Finally, every successful sprint should end in a celebration by both the customer team and the delivery team. When both teams get a taste for celebrating success rather than bemoaning failure, the agile project takes on a positive and energetic tone that carries forward into successive iterations. This practice builds a powerful camaraderie between customers and developers and a shared sense of purpose. But don’t celebrate too hard; that can cause hangovers of a different sort.

Finally, I wish each of you a happy New Year and continued growth and improvement in your agile analytics efforts. As agile luminary Mike Cohn says, “Agile is not something you become. It is something you become more of.” Best wishes on curing your project hangovers in the process of becoming more agile. I’ll see you in 2012.

Note

¹Brooks, F.J. Mythical Man Month: Essays on Software Engineering. Addison-Wesley, 1975.

²Highsmith, Jim. “A Recipe for Success, Part 3.” Cutter Agile Product & Project Management Advisor, 8 March 2007.

2 Responses to “Curing the Common Hangover”