Measuring Sprint Progress

9 May 2011

I often find that teams implementing Scrum have questions about how best to track progress. There are several different commonly used options. Some teams divide each story into tasks of one-day duration. Well, they try to; actually, I've seen teams run into a lot of impediments that way. That's natural, because some activities demand much less time, while many require two days or more.

Other teams explode each story in different activities and make a significant effort to estimate how many story points go into each of those activities. This is done during the planning meeting, which therefore lasts four to six hours. Let's name this scenario "Story Points Distribution" for future reference.

A third approach is the one I like best. I consider it the most beneficial, and it doesn't require a large amount of effort. This makes it consistent with other aspects of Scrum (such as use of the Fibonacci sequence or powers of two sequences during estimation) that help teams avoid investing too much effort trying to get to an apparent precision.

This approach, which I recommend every time I have the opportunity, is to measure progress based on a continually updated ETC (estimate-to-complete) for each story. You can use the approach at the end of every daily stand-up meeting -- or during the meeting itself, if timeboxing is something you've already mastered. The idea is basically this: Let the team estimate the number of story points remaining to get a certain user story completed. Do this every day.

Let's look at an example. Say we have a two-week sprint. After the planning meeting, the committed backlog is seven user stories, from A to G (Figure 1).

                                                                             Fig. 1

 

After making some progress during Monday afternoon, the team gathers on Tuesday morning for the daily stand-up meeting. The group estimates the ETC (remaining work) for the stories that are still in process. As seen in Figure 2, stories A and B are in progress and have been reestimated. The basis for this is what remains of the entire story, not the activities or tasks identified for it. It is this estimate that the team uses to render the burn-down chart (Figure 3).

Every day continues in this manner, with the team estimating a new ETC for each in-progress story. Figure 4 illustrates the resulting estimates, while Figure 5 shows the burn-down chart at the end of the sprint.

One of the arguments I often hear is, "But the ETCs are not 100-percent accurate." That's absolutely true. No estimate is 100-percent accurate. But estimating the ETCs of in-process stories every day gets you closer to the actual effort required than does dividing the story points among all the activities at the beginning of the sprint and adding up the remaining activities every day. Why is that?

We can explain it with the Cone of Uncertainty Theory: "In project management, the Cone of Uncertainty describes the evolution of the amount of uncertainty during a project. At the beginning of a project, comparatively little is known about the product or work results, and so estimates are subject to large uncertainty. As more research and development is done, more information is learned about the project, and the uncertainty then tends to decrease." (http://en.wikipedia.org/wiki/Cone_of_Uncertainty)

As you can guess, the Cone of Uncertainty is strongly based on traditional project management. Nonetheless, its principles apply to Agile project management.

Remember the Story Points Distribution scenario? (Divide the estimate of a user story down into its activities.) We can say that we already have a good amount of certainty -- let's say +/-25 percent (1.25/0.75). Of course, this effort doesn't give back estimates that are 100-percent accurate. While the team develops the product, they'll note the completed activities and summarize the ones still in progress, in order to quantify the remaining effort needed. The problem here is that, without a reestimation, we get stuck at the +/-25 percent point of the cone, although we now have lot more information than when we first estimated the story and activities.

But what happens if we don't distribute the story points among the activities? We won't have activities to add up in order to find the ETCs. However, we will have to estimate the remaining effort needed to complete each in-process story. This way we guarantee that we aren't carrying forward that +/-25 percent error we get when estimating the stories. We'll be moving to the right of the cone, getting closer to 0 percent. Estimating the ETCs while developing the product is much more accurate than simply maintaining the estimates done during the planning meeting.

I encourage teams to give this approach a try on the next sprint. If everything goes as it should, you'll quickly notice the benefits of continually evaluating story ETCs.


Opinions represent those of the author and not of Scrum Alliance. The sharing of member-contributed content on this site does not imply endorsement of specific Scrum methods or practices beyond those taught by Scrum Alliance Certified Trainers and Coaches.



Article Rating

Current rating: 4 (1 ratings)

Comments

Fernando Di Bartolo, CSM, 5/9/2011 5:56:32 PM
Good approach, will try to give it a shot.
Thanks a lot for sharing.
Ani S Menon, CSM,CSPO, 5/12/2011 3:31:23 PM

Looks as a good approach but it has its own practical difficulties. It would be too tier some for the teams to do re-estimation on a daily basis. Also as per the scrum rules before sprinting we are committing with clear understanding of the requirements. So break of tasks will help in measuring the progress in much more detailed level than at user story level.
Julie Hendry, CSP,CSM, 5/20/2011 3:17:11 AM
Good idea - I'd love to see an example of a story that increases in size as the team find out more? Also, how do we tell which days the team worked on that story or did nothing?
Kevin McLellan, CSM, 5/24/2011 9:21:40 PM
Thanks for that. I think the daily ETC is a great idea. This is the process that my team uses quite successfully (although the temptation is to just take off a day every day). As well managing the burndown it is also a great tool for driving conversation and exposing impediments. One trap to avoid is using the daily ETC as a ΓÇ£status updateΓÇ¥ to the Scrum Master.

Is there not a 4th widely practiced technique. Whereby, at Sprint Planning each Story is estimated in Story Points (or hopefully reestimated as each Story should already carry a point value from Release Planning), then the story is broken into Tasks and those tasks are estimated (In Dev Days or Hours). Then the Tasks are burntdown and tracked through the sprint with ETC given each day.
Jaakko Kaski, CSM, 5/25/2011 5:56:46 AM
We have reached a similar type of solution as Kevin, with a little twist that saves us updating any estimates on daily Scrums. As we split stories to tasks in sprint planning, we split them to as small as possible (though one task must be somehow valuable on its own). A rule of thumb is that no single task should take more than 4 hours, one day at tops. Some of the teams I work with manage to split 2 week sprint into 50 to 80 tasks. Then we'll draw a burndown chart from day 0 to day 10 and the first value is the number of tasks. Then on every daily scrum all we do is count the tasks on the task board. Of course some tasks are more demanding than other, but we have good experiences that the time to accomplish a random task tends to average up and we get an accurate enough idea of how our sprint is doing every day.
Phil DiBello, CSM, 6/2/2011 8:29:03 AM
The question is always "What do you do with the data?" In my experience the completion of stories is *not* linear, and frequent sprint reviews have the desired consequence of teams pushing to complete stories before the review. I would really emphasize planning around a sustainable velocity and spend more time pruning the backlog that gather mid-sprint statistics.

The truth is that you may convince yourself that you need to "know how the team is doing" mid sprint, but I am reminded of the Apollo 13 scene where the astronauts rip off their bio sensors for more freedom.

Trust your team, and act like it.
Wait for the Sprint review.
Andreea TOMOIAGA, CSM, 6/3/2011 6:03:09 AM
Excellent article, I think that this method is very useful for tracking progress, control delays and mitigate when necessary in a suitable way. I also see this method compatible to the traditional management point of view when for example you need to deliver an MPP with accurate percent complete status. If you think at User Story level and also estimate and control progress at user story level, then you can build the bridge between Scrum and traditional reporting methods since...in some companies where the culture is very traditional managers want these reports.
Aditya Tandon, CSM, 6/30/2011 11:06:42 AM
Excellent article , It is worth giving a try .I am going to try it tomorrow.

Chia Wei Cheng, CSM, 7/2/2011 4:30:58 AM
Thanks for sharing. It is a good approach but the immediate drawback that I can think of is the team will be re-estimating the story point of each story everyday which some of the developer do not like doing. Estimating story point oftenly linked to 'commitment' for some developers which always cause the estimation session taken long time than it should. There are always some disputes among developers about the complexity of the user stories.
Thomas Guarino, CSM, 7/22/2011 2:17:57 PM
Its a good concept and if it works for you I would recommend to keep using it.

Story points are not actually a piece of scrum, its simply a value to help the team not over commit to work. Once the sprint begins, I feel the "meat" of the work is in the tasks. At this point, we dont care about story points. Those are for planning a sprint.

Tracking hours left todo on a task would result in much more precise tracking, and stop the re-estimation of stories.

I appreciate the article, it was very informative.
Patrick Heller, CSM, 10/5/2011 3:29:23 AM
We used this method and found it be faulty. Here's why: the 20-80 rule applies most of the time to work on a user story: a developer gets 80% of the work done in 20% of the time, and the last 20% of the work is done in 80% of the time (especially bug fixing). Therefore the initial estimates are always very positive and that looks very good on the burndown chart. Alas, that turns sour at the end of work on each user story when estimates are going up again. That feels bad, even if the end-result is good and in time. So, instead, we switched to another way of keeping track: at the stand-up meeting we don't quantify the progress with hours or percentages or story points or whatever, but merely say if things are going as expected or very smooth, or really bad. All that really matters actually is whether the entire user story is done in about the amount of time that was estimated. So we lower the burndown line only when an entire user story is really done - which is all that really, really matters in the end.
Warren Singzon, CSM, 9/12/2012 8:18:29 PM
Interesting approach. I'd like to give this a try but our company is using Rally. I'm still figuring out how to do this, right now I don't know how to do it.
Olivier LEDRU, CSP,CSM,CSPO, 11/7/2012 2:16:38 PM
My team tried 2 methods : the "regular", with a burndown chart of remaining short tasks. And the same as Patrick Heller, just above : a burndown of the remaining not-done stories. I like this method because it's easy, clear and because it's closer to the idea of finishing the stories, not the tasks. The burndown chart is not so nice, not so smooth, but does it really matter ?
Sanjit Tiwari, CSM, 12/30/2012 8:04:56 PM
One question which arises is. If we are already doing a burndown chat with number of hours left in a story, why do we actually need to do another one with estimated points.
One point which comes out in most of the retro sessions are , there are too many calls/meeting in Agile. I fear doing a daily re-estimation will only add to this.
Konstantinos Kastrounis, CSM, 5/22/2013 8:14:21 AM
really like the idea, going to give it a try
Walt Campbell, CSM, 10/23/2013 5:45:24 PM
Interesting idea, but I wonder how teams can credible decide when they're halfway done with an 8-point story. My hunch is that teams who use this approach find it takes many days to finish off the last few remaining points of a story. In other words, I suspect it takes about the same amount of calendar time to go from 2 points to 0 that it took to go from 8 points to 2.

Reminds me of the point that for most projects 90% of the work accounts for the first 90% of the allotted time and the remaining 10% accounts for the other 90% of the allotted time ;-)
Anonymous, 7/31/2014 5:22:42 AM
Liked the idea. I would like to implement. Thanks for sharing.

You must Login or Signup to comment.