Transitioning From Time-Based to Relative Estimation

3 August 2011

Congratulations! You’ve finally convinced the team that relative story point estimation is a great way to move forward and you’re now ready to jump into your first planning poker session.  So where do you start? What’s a 1-point story? What’s a 3-point story? What’s a 13-point story? Your team is looking to you and this process is almost as new to you as it is to them.

One simple way that I’ve seen used to help set these initial values is to find a small story in your product backlog and somehow obtain a general group agreement that it’s in fact a 1 point story. From there you would work your way down the list of stories and allocate 2 points for a story that is roughly double the 1 pointer, 5 points for every story that’s roughly double a 2 pointer and so on. There’s nothing wrong with this approach but you have to bear in mind that your team is new to this process, which means that you really want to try and reduce as much ambiguity and subjectiveness as you can. Any estimation by nature is already ambiguous so the last thing that we want is to add more fluffiness to the mix. It is for this reason that I use an approach that offers the team something more tangible to help build their estimation building blocks and base foundation.

A different approach

Just because the team is starting a new method of estimation it doesn’t mean that the old time based estimations should be treated like a bad dream and quickly forgotten. At some stage in the past, work got done. Cases, issues, bugs, features – whatever they were or whatever you used to call them were completed and the team had some idea of how long it took to do that work (perhaps this time was even documented *gasp*). The idea is that we use a subset of this historical work to create a mapping between these known quantities (old completed work) to these funky new story point values. So the process goes something like this:
1)    Create a set of time buckets that map to the relevant story point values (personally I drop the 20, 40, 100 values to keep things simpler, faster and tighter). The buckets that I use are as follows:

Feel free to modify the ranges for each point value as appropriate. The important thing is to ensure that as the point value increases, the range between the upper and lower values for the bucket needs to become more significant (i.e. it shouldn’t be linear). This is due to the fact that there is more uncertainty involved when dealing with larger stories and this needs to be acknowledged.

2) As a team, we go through our previous work and determine (either through documentation or memory) how many total man hours each old feature took. Remember this includes all necessary tasks to move the feature to a state of ‘Done’ i.e. development, review, test design, test execution, deploy etc.

3) Based on the story point time buckets, we then categorize the old features based on the times we identified in step (2) i.e. old “Feature XYZ” now becomes a reference story with a point value of 8.

4) We continue going through our old work until we have a corresponding feature for each value in our planning poker deck i.e. “Feature A” = 1 point, “Feature B” = 5 points. These become our reference stories.

Here comes an important point. Once we’ve calibrated the points using our old work, I throw away the time calibrations and replace them with the reference stories instead. We can now start playing planning poker with the new user stories by comparing them to the reference stories as opposed to the buckets that we set earlier. It’s tempting to leave the time buckets up there and continue estimating based on those ranges but that’s not what we want. If we keep the time buckets up on the board, we’re just disguising the old time based method of estimation in story point clothing; we end up wasting time trying to break the new stories down rather than simply comparing their size to the reference stories (which is much, much faster). If anyone asks for the old time calibrations, fake amnesia and tell them you can’t remember what they were and the only thing you DO remember is that the calibration is complete and they should be comparing the stories, not trying to estimate time. The exception to this is if someone new joins your team and is unfamiliar with the reference stories, they’ll get to use the time buckets as their training wheels for a session or two.

Not over quite yet…

If you’ve followed the shortcuts above, you should feel pretty satisfied that you’ve completed your first planning poker session relatively unscathed and that your first sprint is underway. But you’re not done yet. There are still some minor tweaks that you should do.

In Scrum, you are guided to capture all tasks’ remaining time on a daily basis (and this of course is reflected within the daily burndown chart); but what I like to also track are the hours of work actually completed on each user story. Once a user story is finished, I add up the hours worked to see how close we were to the original calibration time bucket (OK, so I might have thrown away the calibration hours in the planning session but I always have a copy at my desk Γÿ║).

If the story went fairly smoothly but it actually fell into a 13 point bucket instead of the 8 (that was originally estimated) I’ll adjust it for the sake of reference and use it as an example of a 13 point story for any future product planning or grooming sessions. I simply do this to ensure that the estimation foundations remain solid and that we’re not building an estimation model on top of a house of cards (pardon the pun).

I also make it a point to tell the team why I’m asking them to track their hours completed (and not just the time remaining). It’s important that they don’t feel like their individual performance is being measured. They need to know that I do it to inspect and adapt the estimation process going forward and that it’s not about micro-management.

So there you have it. You’ve recycled your historical work to give you a basis for relative estimation with story points. If you found this technique helpful or if you have an alternative method please post your comments as I’d love to hear from you.


Opinions represent those of the author and not of Scrum Alliance. The sharing of member-contributed content on this site does not imply endorsement of specific Scrum methods or practices beyond those taught by Scrum Alliance Certified Trainers and Coaches.



Article Rating

Current rating: 3.3 (6 ratings)

Comments

Ariel Kogan, CSPO, 8/3/2011 10:52:53 PM
Great article, Ilan! I've worked with teams that really struggled to think of stories in terms of relative complexity rather than time, but I'm currently working with a team that introduced relative estimation in a very similar way to this and now I hear things like, "Is this bigger than implementing and testing two new scripted fields? If so than it has to be more than 3 points!".

I think you mentioned a really good point about new members to the team. Making sure they are clear on the relative sizes is important, and sometimes recalibrating the sizes as team members change is a worthwhile process - especially if the new members have come from other Scrum teams and don't realise that relative sizes are determined by the team rather than being a universal standard.

Keep up the good work!
Colin Tan, CSM,CSPO, 8/8/2011 1:58:38 AM
I think your article is an important one because I can certainly reinforce the fact that getting your story point values right can be quite daunting in your very first relative estimation. Calibrating the story points with past features seems to be a solid way to the get the ball rolling.

BTW, I checked out your website and I do have to say that the images youΓÇÖve got on there are hilarious! Who would have thought that Scrum was the methodology chosen by Darth Vader to rebuild the Death Star! Nicely done.
master blaster, CSM, 8/9/2011 10:29:33 PM
Hello, I have been struggling on time estimation in my project, Can you please advise.

We are implementing a software modernization project for a public sector customer. Customer follows Agile methodology and Scrum framework with some customization. Our team is made up of a Product owner, a scrum master, developers, a architect, business analysts and a tester. Neither our product owner or scrum master is technical and it is on developers discretion to bring forth developments items with respective hours allocation for a particular Sprint. Similarly analysts, architect and tester plan their respective sprint tasks.

Initially few sprints we were not able to meet sprint goals due to frequent change in requirements, deviation from already planned tasks, incorrect time estimation and infrastructure problems. I believe our product owner thus in retrospection is now keeping a strict vigilance of Sprint backlog on daily basis. There is a strong emphasis on team members to update sprint backlog and track daily 8hrs of work getting reflected against any sprint item. I as a Team member see a major flaw in this approach.

Since there is so much increased focus on Sprint backlog ,team members have overestimated hours which otherwise could take less time. Also if a team member completes his work much before originally estimated time, he/she just sits on that time until the estimated time runs out. Since product owner and scrum master relies on hours given my team members and just looks at Sprint backlog everyday just to see those hours getting reduced, I donΓÇÖt this is a true reflection of actual work getting completed. Also now as there are less chances of time slippage for a particular work which is already over estimated, I donΓÇÖt understand how a product owner/scrum master would be able to correct the time estimation of work in future sprints.

As a scrum master what should be my approach to take a corrective measure to bring things on track and make it a self sustainable process.
Ilan Goldstein CST PMP ACP, CST,CSP,CSM,CSPO, 8/10/2011 12:47:17 AM
@Arik ΓÇô glad you enjoyed the article. Indeed I find that this method works best when spinning up a new team or getting a new member of the team acclimated to the stories.

@Colin – yep that’s right, the Death Star was rebuilt using Scrum ;) I recently posted a new entry (with associated Scrumtrooper image) that I’m sure you’ll enjoy if you haven’t seen it already…

@master blaster ΓÇô thanks for you in-depth question. I will attempt to briefly answer it below however if you would like additional detail then please let me know:
- Estimates are exactly that i.e. they are NOT a guarantee and bottom line is that it will take a while to get estimates close to reality. The most important thing to remember when tracking estimates is to understand why they might be inaccurate so that you can constantly improve this process. This needs to be strongly communicated to the team so that they donΓÇÖt feel pressured to pad the numbers. Scrum is all about inspecting and adapting our processes and not getting everything right straight away.

- If you focus on the above point and remove the fear of estimate inaccuracy (whilst identifying why you might be off target) then I assure you that estimates will eventually become more accurate rather than artificially accurate (as they might be now)

- If you feel that there is ambiguity with your teamΓÇÖs estimation accuracy you could potentially get an external senior developer/team lead/architect in to observe sprint planning to ensure that there is qualified questioning of potentially inaccurate estimates

- if ALL of the planned sprint work is genuinely completed to the expected ΓÇÿdefinition of doneΓÇÖ then simply go back to the product owner and add more work to the current sprint (as opposed to finishing the sprint early or simply sitting on your hands waiting) ΓÇô this is a good problem to have. Big point here is that if team member X feels that his/her work is complete whilst others are still working on the current sprint then he/she needs to do whatever they can to assist the rest of the team to finish ΓÇÿtheirΓÇÖ work ΓÇô remember that the sprint goal is a team goal not an individual one.

- My last little tip is to reduce your team memberΓÇÖs daily capacity to 6 hours of dedicated project work as 8 hours is typically very unrealistic due to disruptions, meetings/collaboration time, breaks etc.
Aaron Corcoran, CSP,CSM,CSPO, 9/6/2011 9:00:08 AM
This is a great article. I appreciate the time you took to write it, as it is helps us that are used to a time based (non-effective) scheduling process to more agile approach.

In the end, management always wants how long will this take (how much will this cost). When transitioning, it is a difficult process as it is transitioning not only the analysts who were responsible for this estimate in the past, but now including more of the development body, as well as upper management.

When you do such a drastic transition (let's say you're mid-way in a project that is a bit behind) and you propose a new scheduling technique, you'll have to get ready for a lot of questions from upper management...e.g, why are you changing, why didn't you do this for the initial schedule, etc.

Some great tips in this article to help transition teams from a more traditional (read: waterfall) approach to the relative complexity approach.
Anita Upadhyay, CSM, 9/8/2011 11:51:02 PM
In some of the early adaptions we tried this approach and it worked well. We identified reference stories from past experience of point sizes 1, 2, 3 and 5. We mapped animals to fibonacci series and asked team to map stories size to relevant animals - rat, cat, dog, sheep, horse, camel, elephant, whale. During initial days we mapped these to range of days to identify how much to fit into the backlog for current sprint. Later as our velocity became clear we moved away from mapping days to size and also moved away from animal estimates to fibonacci estimates.
We also experienced when the team was not pressurized they were more close to estimates than otherwise.
Ilan Goldstein CST PMP ACP, CST,CSP,CSM,CSPO, 9/12/2011 6:03:29 AM
@Aaron - thanks for your comments and glad you enjoyed the article. As you point out, implementing a fundamental change is never easy so timing is important; as such I would recommend trying this technique from the very beginning of a short project (as a pilot/trial) rather than trying to change mid-large project (especially if you are already encountering issues).

@Anita thanks for your thoughts and good to hear that story points are working for you. I have heard of teams doing the 't-shirt size' mapping before but never animals - nice one :) My only concern when mapping to 'symbols' rather than numeric story points is that you cannot divide your aggregate product backlog point value by velocity to get your expected project duration otherwise I think it is a good transitional tip.
Rajesh Francis, CSM, 9/27/2011 7:12:20 AM
Hi, I had this problem of explaining the usage of poker cards; infact i was thinking to never talk about this anymore and stick to the WBS way of estimating. Let the team estimate, question and reach to some conclusion. Now , with your way i am in a better position to use poker cards. Thanks for your suggestion.
Ilan Goldstein CST PMP ACP, CST,CSP,CSM,CSPO, 10/6/2011 10:26:33 PM
Thanks for your feedback Rajesh - I definitely find that this approach helps bridge the communication gap. Let me know how you go.
Kirk Bryde, CSM, 12/27/2011 10:29:59 PM
Ilan, I think your approach to converting to story-points is solid, except ...

I don't think it's very useful to ask team members to track their actual time spent on each task once the conversion to story-points is done.

The reason I think this is because people are almost as inaccurate in estimating time SPENT on a story as in estimating the time TO BE SPENT on a story (which, after all, is why you converted away from time estimates in the first place).

Here's a few issues: (1) The story consists of several tasks, and these tasks are worked on by varying numbers of people - for varying amounts of time. The accuracy of the total time spent on a story is only as good as the sum of the time spent by all of these people; (2) There may be meetings held to discuss a story - certainly during the standups, but also possibly during special meetings that are specific to a story. And other more general meetings where the story is discussed in the course of a more general discussion. Some people may include this time, whereas others may ignore it - leading to an inconsistent way of recording actuals (realize that the actuals they record are actually only "estimates of actuals"!); (3) there may be a bias (conscious or subconscious) where team members have the amount of time left at the START of a story, which influences how much time they say they actually SPENT on a story after it's done; (4) there may be a bias towards reporting what they think you want to hear - rather than true ACTUAL time spent; (5) people have very poor memories for how they spent their time; (6) stories aren't usually started and finished in a single chunk, so there's going to be bits and pieces of time here and there that's either forgotten, underestimated, or overestimated; (7) ... and so on ...

My suggestion is that once the team has bitten the bullet and gone to story-point estimating, then they (and you) shouldn't fret over whether there's any slight inaccuracies. Remember that a story-point estimate is merely to categorize a story into a size relative to other stories. Inaccuracies will "come out in the wash" and will even out for all intents and purposes.

Kirk Bryde, CSM, CSP, PMP

Manish Gupta, CSM, 3/9/2012 12:46:59 AM
I have a different opinion to this article. Scrum does not suggest mapping story points to efforts directly/indirectly. I think initially team will have confusion about Story Points, but in due course, may be after 4-5 iterations/sprints, they will understand the benefit and usages of Story Points. Naturally, team will give proper points to story and naturally efforts spent on stories will be bucketed by the team - we need not change the story points once it is given in planning meeting.
Henri Stegehuis, CSP,CSM, 5/31/2012 4:42:18 AM
Great article! I train and coach teams for customers and the first thing I try to tell them is to get away from time estimates. I deliberately do not use hours (in a bucket system). We use 'bananas'. In the initial Product Backlog identify together a Product backlog Item that takes 1/3 of your sprint, one that takes 2/3 and one that takes 4/3. Assign points in such a way that they are linear but also no traceable to hours. For example 200, 500 and 1000 bananas (in smaller projects use smaller numbers; maybe the poker cards can be used. If you have 100 'eights' then you might want to choose a larger scale). Mark them as reference stories (Ref1, Ref2 and yep: Ref3) and all others are referenced against these. "This PBI is more then Ref1 but lesser then Ref2, closer to Ref1 so we give it: 300". With big projects 5 reference PBIs are recommended.

With respect to Sprint Planning ... we tried reference points again but this failed for most teams. It took too long to get used to this for most team members. Especially on shorter projects it was a complete chaos; too much discussions on trivial assignments. This has bordered me for a while until it hit me: ideal-hours can be understood by any one. Relative estimations are effective in teams working together for a longer time; these teams have learned and grown to understand the common value of the 'banana'. For short projects with different team members (in a software development organizations very common) the use of ideal hours is commonly understood, maybe not perfect but has a solid basis in common understanding.

Keep this restriction in mind: strong cooperating teams working together for longer times can definitely go for points and will benefit; teams assembled for the occasion and with a short duration will not benefit because their understanding of bananas is not aligned.
Shivraj Bhushan, CSP,CSM, 7/19/2012 4:17:30 PM
Ilan , Very good article. I saw you are CSP . Can i ask one question ? I am also planning to take CSP. Can you please recommend what all books you went thru and how much is passing score. Thanks for your help.
Ilan Goldstein CST PMP ACP, CST,CSP,CSM,CSPO, 8/3/2012 4:03:09 AM
@ Manish, Henri - thanks for your feedback and for sharing your thoughts.
@Shivraj - I recommend that you check out the CSP requirements here:
http://www.scrumalliance.org/resources/2733
Willem-Jan Jacobs, CSP,CSM,CSPO, 10/3/2012 8:20:50 AM
Nice article, a few remarks:
If you ask team memebers to track their time on a per task basis, in my experience you can be sure that the time spend will match the time estimated, as one of the previous commenters mentioned above. In my own teams, we started out with ideal time estimation, since we had no experience in actual story points and no idea about our potential velocity, but now I notice that it hinders some team members in making the transition to more abstract relative story points. They still want to see a direct relationship between scope and effort, and have difficulty to accept that some points will be delivered cheap and others more expensive. So you get discussions like "this story took more time than we thought so now we have to raise the amount of story points for it". I try to emphasize that Story Points are a measure of scope, and that it is related to actual effort by means of the velocity. And indeed this is easier for teams that are a while together already than fro teams just starting out, because the latter have no idea about their velocity and the value of a Story Point.
Willem-Jan Jacobs, CSP,CSM,CSPO, 10/3/2012 8:20:55 AM
Nice article, a few remarks:
If you ask team memebers to track their time on a per task basis, in my experience you can be sure that the time spend will match the time estimated, as one of the previous commenters mentioned above. In my own teams, we started out with ideal time estimation, since we had no experience in actual story points and no idea about our potential velocity, but now I notice that it hinders some team members in making the transition to more abstract relative story points. They still want to see a direct relationship between scope and effort, and have difficulty to accept that some points will be delivered cheap and others more expensive. So you get discussions like "this story took more time than we thought so now we have to raise the amount of story points for it". I try to emphasize that Story Points are a measure of scope, and that it is related to actual effort by means of the velocity. And indeed this is easier for teams that are a while together already than fro teams just starting out, because the latter have no idea about their velocity and the value of a Story Point.
Bandana Pandey, CSM,CSPO, 2/24/2013 6:32:54 PM
Very good article. Do any body have the article with funding.. How we should be doing that..??
Alon Tvina, CSM,CSPO, 4/14/2013 6:46:35 PM
Great article Ilan! Really helpful to me and the team.

You must Login or Signup to comment.