The Value of PERT in Agile Estimation and Risk Management

18 December 2013

Sriramasundararajan Rajagopalan
Physicians Interactive


Problem statement
Estimation techniques are critical to planning exercises in both traditional and Agile project management. Whether it is a project manager drawing a schedule based on estimates given by the team or a self-organized Agile team calibrating the velocity in the sprint, the importance of timely delivery of the agreed features to the customer is a critical component in any development method. Therefore, there is special emphasis on knowledge of estimation techniques, with the client-facing role in managing the estimated delivery of the features.

Both traditional project management and Agile development approaches rely on several techniques, such as bottom-up estimation, order of magnitude, planning poker, affinity estimation, relative sizing, and parametric estimation at various stages. But the most important derivative of all these techniques is to ensure the central tendency of the estimate as perceived by the team closest to the implementation. When team members vary on the perceived requirements of user stories, this is an indication of ambiguous requirements or unclear scope leading to questionable "done" criteria. Here is where the proven PERT method can be applied even in an Agile environment.

Approach
This approach was experimented on by the author, who acted as a product owner, and who found it to be effective with one of the major internal products that was developed with a distributed team. As the release cycle was developed with the features listed in the product backlog, the sprint planning sessions began with the product owner describing the product backlog. The team provided story point estimates based on a planning poker exercise. Where the points differed vastly, instead of just explaining the story, the product owner listed the smallest, the largest, and the story point that was most likely based on the mean closest to the Fibonacci sequence. This exercise was repeated, explaining the stories to eliminate ambiguity, and then the PERT approach was applied to arrive at a better risk-adjusted estimate for the velocity of the sprint and to put together a cadence.

Example
For instance, consider five user stories with the story point estimates tracked as follows. The most likely story points of 128 differ greatly from the optimistic (44 points) and pessimistic (201 points) extremes. By using PERT, the team can go with 126 as the best cumulative story point.

Table 1: Round 1, estimation applying PERT
Planning Optimistic Most Likely Pessimistic Variance Average PERT
User Story 1 8 20 40 32 22.67 21.33
User Story 2 1 3 8 7 4.00 3.50
User Story 3 2 5 13 11 6.67 5.83
User Story 4 13 20 40 27 24.33 22.17
User Story 5 20 80 100 80 66.67 73.33
Est. Velocity 44 128 201 157 124.33 126.17

The results are tabulated in Table 1, demonstrating the central tendency of the most likely, average, and PERT estimates. By extending the same approach and using another round in which the product owner explains the user stories, particularly focusing on large variances, the estimates can be further refined. Table 2 summarizes these findings, assuming a second round in which the same user stories are refined with the team's input. This approach of using PERT to triangulate and eliminate ambiguity helps the team to be safe rather than sorry and also helps manage the risks.

Table 2: Round 2, refining PERT estimates and identifying risky stories
Planning Optimistic Most Likely Pessimistic Variance Average PERT
User Story 1 8 13 40 32 20.33 16.67
User Story 2 1 3 8 7 4.00 3.50
User Story 3 2 5 8 6 5.00 5.00
User Story 4 8 13 20 12 13.67 13.33
User Story 5 20 40 80 60 46.67 43.33
Est. Velocity 39 74 156 117 89.67 81.83

Comparing the variance between the user stories from Round 1 and Round 2, the risk associated with the potential probability of User Stories 1 and 5 is high due to the high variance. While the approach may help eliminate ambiguity, the inherent assumptions that may no longer be true as expected by the team may introduce a risk to their completion, and reducing the sprint velocity enables the team or the project manager to better manage risk and maintain a consistent delivery of features.

Conclusion
Estimation and delivery are important elements of successful movement of value in project management and product development. While a mature, self-organized team can address the variations in the estimation, an evolving team or a team embracing Agile may find the PERT approach a value-add to help build trust in their estimation approaches and help Agile implementation succeed.

Article Rating

Current rating: 4 (1 ratings)

Comments

Andreas Schliep, CST,CSC,CSP,CSM,CSPO, 12/18/2013 8:11:59 AM
Interesting approach. Although the examples would suggest that you just need to go with the most likely estimate.
Zach Bonaker, CSP,CSM,CSPO, 12/18/2013 10:52:45 AM
I found your idea to be creative, but I'm confused on a couple points:

1. Your use of variance in the examples doesn't appear correct. I think you mean the range, instead (max value - min value)?

2. As noted by Andreas, the method to generate the "most likely" point value is - generally - the same as the PERT estimate.

3. The scenario presented does a nice job of illustrating the technique, but I'm not convinced it's suitable in a real-world situation. Team member estimates that vary to such a degree (e.g., 8 and 40) should be a warning sign the story might not satisfy the "estimatable" aspect of INVEST. Does it really make sense to try and generate a "best estimate" for a story that is clearly not well understood by the team (or well written)? Given we assume a normal distribution for each point on our scale, I feel you're actually "safer" going with an epic-sized point estimate early on (e.g., 40, 100) and allow grooming to decompose it naturally.
Sriramasundararajan Rajagopalan, CSP,CSM,CSPO, 12/31/2013 8:51:38 AM
Zach, Andreas,

Great points. As the central tendency suggest in normal distribution, yes, the idea is to get towards the most likely estimate that eliminates surprises as much as possible. But I meant range and not necessarily variance in probabilistic terms.

As for the estimate following INVEST or generating a best estimate for a story not understood, I disagree somewhat. In general terms, such a wide range definitely seeds ambiguity and doesn't follow INVEST and I agree that product owner should consider this as an indication of ambiguity and explain the story further. But, I disagree because of the Pharma industry I operate where software delivery is as important as the documentation that goes with it for pharmaceutical audit trail. Still, from a MMF point of view, software delivery is more important. Often, therefore unless there is a need to decouple documentation from software delivery, they are done together as noted by Mike Cohn as well in the importance of documentation. In my experience working in this regulated industry, the examples of the low and high extremes are seen only when documentation can be decoupled from the software delivery. I think this is an important pointing I should have captured so that the story can be split as well when extended generically.

Please let me know if any question.

You must Login or Signup to comment.