Is Your Sprint Pipeline Running Well?

Most agile teams, when starting out new on their agile transformation road, conduct all sprint related activities in the same timebox, i.e., sprint planning, sprint execution, sprint review and sprint retrospective.  This is illustrated in Figure 1, where each sprint (from Sprint 0 through N+1, represented by each row) is mapped onto a single sprint timebox (Timebox 0 through N+1, represented by each column), and successive sprints are executed in successive timeboxes in a sequential order (Sprint 0 in Timebox 0, Sprint 1 in Timebox 1, and so on).

 Figure 1: Sequential execution of sprints

For example, all implementation tasks for Sprint 2 (analysis, design, code development and unit testing, acceptance test case development and acceptance testing, defect fixing – done concurrently (not as a mini-waterfall) by a self-organized, cross-functional team — are completed in Timebox 2.  Sprint 2 planning is completed (before Sprint 2 starts) in approximately 4 hours for a two-week sprint (or 8 hours for a four-week sprint); and sprint review and retrospectives are completed in approximately 1 hour each for a two-week sprint (or about 2 hours each for a four-week sprint).

Each light pink vertical thin box separator between two successive sprint timeboxes represents a small timebox to complete all these ceremonies, i.e., sprint review and retrospective for the previous sprint as well as sprint planning for the next sprint.   Thus all these tasks for successive sprints are carried out in a sequence of successive timeboxes.  This is a simple and straightforward model where work goes in each timebox sequentially, sprint by sprint, without any overlap.

What are the advantages of and the challenges for the simple model of sequential sprint execution shown in Figure 1?

Advantages of the sequential sprint execution model:

1AS.    Simple to teach, understand and learn – and hence covered by all agile text books, training courses, and certification programs.

2AS.    Conceptually simple to execute — but fraught with challenges as explained below.

Challenges for the sequential sprint execution model:

1CS.     Analysis issues may block the team and reduce throughput: Analysis of backlog items (or stories, as they are often called) must involve intense conversations among the product owner and team members, and confirmation of each story by defining its acceptance criteria.  Stories can be properly understood only through conversation and confirmation parts of the story metaphor of card, conversations, and confirmation.   The product owner should answer the questions and clarify the issues raised by the team members before design and code development work can start.     This goal – clarifying all stories to reach a collective, common, shared understanding – may become very challenging if all stories are analyzed during the same timebox in which they are scheduled for design, code development, testing, defect fixing and delivery.   This is so because the product owner may not know all the answers to team members’ questions without consulting users, customers and stakeholders, and performing necessary market and customer research and analysis.  This entire analysis effort is often difficult to compress in the same timebox when design-development-testing-defect fixing work is also going on concurrently because the actions of many actors responsible for providing answers to the analysis questions may be beyond the control of the product owner (for example, some stakeholders or customers may not be available to provide specific clarifications within the timebox time constraints).   The net effect of this challenge is often reduced team velocity (throughput) as the team is still waiting for stories to be clarified while sprint execution is going on, and team members may be even blocked waiting for clarification.  Finally the sprint timebox may be over before some planned stories could be completed by the team members and accepted by the product owner.

2CS.     Sprint planning may be less effective and efficient: Without clear and shared understanding of each story by all team members and the product owner, the team will find it difficult to estimate the work effort and complete all sprint planning tasks in the allocated time for sprint planning (approx. 4 hours for two-week sprints and 8 hours for four-week sprints).   Needless to say poorly planned sprints contribute to poor (ineffective and inefficient) sprint execution and reduced throughput.

A better model to overcome the challenges of the sequential sprint model is to execute sprints in a pipeline fashion as illustrated in Figure 2, where the Analysis task is performed one timebox ahead of the Design-Code-Test timebox for each sprint.  For example, development and analysis of Sprint 2 backlog is performed during Timebox 1 (one timebox ahead of Timebox 2), while the actual design, code development and unit testing, acceptance test cases development and testing, regression testing and defect fixing tasks for Sprint 2 are performed in Timebox 2.  This same pattern repeats for each sprint, i.e., the work for each sprint proceeds in a pipeline manner, and as a consequence, the work for each sprint is spread over two consecutive timeboxes in an overlapping manner with the next sprint.

Figure 2: Sprint pipeline

 What are the advantages of and challenges for the pipeline model of sprint execution shown in Figure 2?

Advantages of the pipelined sprint execution model: 

1AP.   Analysis work proceeds smoothly without blocking the team: As analysis work is carried out one timebox ahead of design-development-testing-defect fixing work for each sprint, the stories are substantially clearer by the time the team enters the Sprint Planning Workshop for each sprint.  Note that the product owner has a full sprint timebox (typically 2 to 4 weeks) to consult with the necessary users, customers and stakeholders in order to answer team members’ questions on stories.  And while this clarification of stories for the next sprint is going on, the team members are not held up or blocked as they are implementing the stories for the current sprint.   Effort estimation and other sprint planning work proceeds more smoothly and can be more easily completed during the Sprint Planning Workshop.

2AP.   Team experiences higher throughput with well understood stories and well-planned sprint: Note that although each sprint work is spread over two timeboxes (as shown by the blue oval in Figure 2), the throughput is not adversely impacted because the team is still delivering working software at the end of each timebox.   In fact, as sprint planning effectiveness and efficiency goes up and stories become clearer, there is a lot less uncertainty about stories as the sprint starts, which reduces impediments, improves team morale and dynamics, and improves team’s throughput compared to sequential execution of sprints.

Challenges for the pipelined sprint execution model: 

1CP.   In each timebox, the team needs to work on two sprints: Although most of the time the team is focused on design-code development-testing-defect fixing work for the current sprint, some of its time must be set aside to analyze the backlog items prepared by the product owner for the next sprint.  This is indicated by the red oval in Figure 2, where the team is spending most of its effort on design-code development-testing-defect fixing for Sprint 2 in Timebox 2, while at the same time spending some effort in analyzing the stories for Sprint 3.   Some teams — especially when they are new to agile — may find working on two sprints in the same timebox challenging.

2CP.   Small risk of doing wasteful work: There may be a small risk of analyzing few stories one timebox ahead of their actual implementation that may end up not being part of the next sprint commitment due to change in priorities as the next sprint planning is completed.   Some people may even object that doing analysis one timebox ahead of the actual implementation could be somewhat wasteful, and it goes against the “just-in-time” agile work philosophy.

I will now explain how to deal with both of these challenges, 1CP and 2CP.

Solution to the challenge of working on two sprints in the same timebox (1CP): The ScrumMaster for the team can help manage work on both sprints in the same timebox (i.e., design-development-testing-defect fixing work for the current sprint as well as analysis of backlog items for the next sprint) by establishing and following a Sprint Cadence Calendar as illustrated in Figure 3.

 Figure 3: Two-Week Sprint Cadence Calendar

The two-week Sprint Cadence Calendar has 10 work days, with workday starting at 8:00 AM (0800 hours) and ending at 5:00 PM (1700 hours). The team should allocate about 30 minutes for preparation for the Daily Scrum (DS) meeting and actual attendance by each team member.  These DS meetings should be held every day of the sprint at the same time and place.   In Figure 3, these DS meetings (preparation and actual meeting) are shown as starting at 9:00 AM and ending around 9:30 AM every day.  If you are interested in understanding and implementing highly effective and efficient Daily Scrums, please take a look at my Making-daily-scrum-meetings-really-effective-efficient blog on the subject.  The other ceremonies (Sprint Planning, Sprint Review, and Sprint Retrospective) are also illustrated in Figure 3. If you are interested in understanding and implementing really effective Sprint Retrospectives, please take a look at my Making-sprint-retrospectives-really-effective blog on the subject.

As shown in Figure 3, I recommend that the analysis of backlog items for the next sprint should be scheduled on a regular cadence, where a two-hour meeting is held on every Wednesday (3 PM to 5 PM), as an example.  In these two meetings in a two-week sprint (a total of 4 hours or approximately 5% of the time in the two-week timebox), the product owner and the entire team converse about each story and also confirm each story with its acceptance criteria.  If the product owner cannot answer some questions in the meetings, he/she has adequate time left in the two-week sprint timebox to get those questions answered in consultation with users, customers and stakeholders, without blocking any team member.

I also recommend that the product owner grooms the product backlog and prepares a draft of backlog items for the next sprint on a regular cadence, shown on every Tuesday (3 PM to 5 PM) in Figure 3.  The cadence or pattern for sprint planning, sprint review, sprint retrospective, analysis for the next sprint, and product backlog grooming (shown in Figure 3) is for illustrative purposes only.  Your team should discuss and experiment various sprint cadence options to find the cadence that most suits your needs and choose the cadence that best works for you.  For example, a team may hold Sprint Review and Retrospective for the previous Sprint on Week-1 Monday morning (9 AM to Noon)  followed by Sprint Planning Workshop for the current sprint from 1 PM to 5 PM.  Another team may start the Timebox on a Wednesday (instead of Monday), and two weeks later end it on a Tuesday (instead of Friday), i.e., stagger the start and end of the timebox shown in Figure 3 by two work days to avoid Sprint Planning, Sprint Review and Sprint Retrospective falling adjacent to a weekend day (and tempting some team members to miss them due to their long weekend vacation!).

Regular cadence for all the activities mentioned above minimizes coordination, scheduling and transaction costs; increases predictability through a disciplined schedule published ahead of time for several release cycles (6 to 12 months), and helps an agile team to become a well-oiled machine.  Regular cadence or pattern also reduces unplanned, unexpected, interrupt-driven work that is very damaging due to sudden, unplanned context switches with resulting loss of productivity.

For all these activities shown on a regular cadence in Figure 3, the ScrumMaster working with the team members must set aside adequate capacity: 4 hours for sprint planning, 4.5 hours for 9 Daily Scrum meeting (preparation and attendance), 4 hours for Analysis of the next sprint, 3 hours for Sprint Review, Sprint Retrospective and Celebration – a total of 15.5 hours of capacity is needed for each member for these activities, and that capacity for each member is not available for the design-development-testing-defect fixing work for the current sprint.  If you are interested in calculating the agile capacity of a team after allocating capacity for all these activities, please see my Agile-capacity-calculation blog on the subject.

Solution to the challenge of managing the risk of doing wasteful work (2CP): An important reason why teams fail to deliver stories or backlog items in a sprint is that they were not understood properly when they were committed to the sprint plan during Sprint Planning.  As Dean Leffingwell explains in his Agile Software Requirements book (page 211), from a timing perspective there are three opportunities to do this conversation and confirmation for each story.

• Prior to the sprint: This is what I have advocated in this blog by stipulating to engage in this conversation and confirmation only one timebox ahead.  Doing it more than one timebox ahead will increase the risk that some of those stories may not get into any sprint commitment as they get trumped by higher priority stories, or they may be removed from the product backlog altogether due to changed business conditions.
• During the Sprint Planning Workshop:  This workshop is limited to approximately 4 hours for a two-week sprint.  In this short time box, the team has to not only converse and confirm the stories, but do story effort estimation and all other tasks related to sprint planning.  The team may find it difficult to get the time needed to complete the conversation and confirmation for each story, or the product owner simply may not have answers to some of the questions and most likely there is no time in the workshop timebox (only 4 hours) to get the answers by contacting users, customers and stakeholders.
• During the actual sprint: If the team feels sufficiently confident that conversation and confirmation about a story can wait until the actual sprint due to uncertainty about the story’s business needs, then the conversation and confirmation for that story may be completed along with its implementation in the actual sprint if the story is of sufficiently lower priority.   However, keep in mind that the story was committed to the sprint during its sprint planning without proper conversation and confirmation (and hence with either no estimate or a very rough estimate).  This situation carries some risk and is not ideal.

I recommend that you complete the analysis of as many high priority stories as possible one timebox ahead of the sprint in which they will be implemented, try to complete as much analysis as possible for some of the lower priority stories during the Sprint Planning Workshop, and leave very small number of analysis questions to be resolved during the actual sprint if you have to.

In my coaching engagements, I have seen agile teams transitioning from sequential sprint execution model to the pipelined sprint execution model after 2 to 4 sequential sprints under their belt, and then with experience (inspect and adapt) getting better at the pipelined model.  This is kaizen (continuous improvements) in work, resulting into higher throughput, improved quality, and increased team morale and work satisfaction.

Have you tried the sprint pipelined model?  What is your experience? Are you interested in starting your own sprint pipeline?

I would love to hear from you either here or by e-mail (Satish.Thatte@VersionOne.com) or on twitter (@smthatte).