Get certified - Transform your world of work today

Close

Order Your Product Backlog

4 December 2014

Sameer Chudaman Patil
Cognizant Technology Solutions


Ordering the product backlog is important for an Agile team's success. It provides a flexibility to the product development process that allows delaying decisions to the last responsible moment, and it decentralizes control from product management to Scrum teams, giving those teams time to evaluate options, gather feedback from customers, and acquire knowledge.

The prime responsibility for ordering the product backlog rests on the product owner's shoulders. The product owner should prioritize the backlog in consultation with all the other members of the project, since there could be dependencies. On initial assessment, one item may seem to take priority based on the business need, but when you take the whole picture, that item may have to be pushed back due to a dependency. It's best not to make story points a highly affecting factor while prioritizing the product backlog; value and dependencies must be accounted for. However, there is no standard process around prioritization, and each PO needs to determine what works for him or her. A bit of coaching from the ScrumMaster or an Agile coach on what "value" means for the product backlog can be helpful here.

We should use the term value instead of priority, as ordering the backlog depends on more than just priority but also factors in risk, business value, return, and dependencies. How do we order the product backlog according to business objectives? I generally ask POs and SMs whether any of them actually understand the business objectives for their projects. No one does! This seems to be an issue at many companies; prioritizing seems to be done by the product owner and teams without a true understanding of the relevant business Value. To avoid this, the business objective must be defined clearly in advance and continuously maintained as part of the ongoing project.

When ordering your backlog, you as can ask some powerful questions of the PO, customer, team, and all other stakeholders before you begin:
  • What feature/stories will give the highest ROI?
  • What feature/stories will solve the biggest pain first?
  • What risks can be minimized to meet the release objective?
  • Use the 80/20 principle: Which 20 percent of features are going to make 80 percent of the users happy?
  • What is minimal and what can be delayed?
  • What is going to have the biggest impact?
  • If the budget were drastically cut, what would the stakeholders choose to keep first?
  • What areas of technology could jeopardize the project, and what will mitigate these?
  • What level of quality is needed?
Here are some guidelines I would like to recommend for when you are ordering your backlog.

Business value: How much business value will the story deliver? The most valuable items should always be placed first in order to ensure that the product delivers what the customer desires. Jeff Sutherland has noted that for a successful product, 80 percent of the revenue will flow from 20 percent of the features. Identify those features and keep them at the top of the product backlog. In short, how much money will the organization gain after developing that particular feature? Basically, when you are identifying business value, you have to prepare a tentative forecast for a few quarters -- maybe the next six quarters, say -- and then compare the feature against each other. If you can't do this, then ask the executives and stakeholders how they rate the value of each feature by playing Planning Poker or a money game; with this you can get a tentative idea of the value of the feature from the organization, executive, and stakeholder viewpoints.

Catherine Louis has written about ways to identify this 20 percent of value and has listed some powerful questions:
  • Can you name the 20 percent of the product in your portfolio that delivers 80 percent of the value?
  • Do you know the 20 percent of epic that delivers 80 percent of the value in the current project?
  • When you break an epic story into smaller bits, does your team know the critical 20 stories that will deliver the 80 percent for the current iteration?
Risk: How much risk is associated with the feature? The sooner risky stories are developed, the sooner the team will be able to mitigate the unknowns, thus removing uncertainty, thus leading to a higher-quality product. When we were developing a PC companion app a few years back, we came across a security story that was added at the end of the product backlog. We discussed the matter and concluded that If we developed a security subfeature of the PC log-in/log-out feature today, that would probably cost less than adding the same security to the application later. Sometime you can identify the size of the risk in story points, as you would for feature estimation. Then you can make a relative comparison in terms of total feature points of "story risk." With this approach you can identify and address the tentative size of the rework and its proportional investment.

Cost to develop: How much is the cost to develop new features (to calculate ROI)? I remember when I was consulting to an organization where we took the approach of refactoring and implementing a unified graphic user interface (GUI), so the feature developed on one product line could be easily imported onto another product line. Initially, before moving to the unified GUI, we were putting a lot of effort into the same feature development that had been developed for the other product line, and this was costing tremendous effort and rework again and again. But once we put the unified GUI product backlog item at the top (which was challenging, risky, and even questionable), the life of the product development become easy to deliver, with a lower cost of development. In fact, with this approach we reduced the cost of development by 22 percent. In short, if you have an item that might lower product development costs, then that item should be at the top of your product backlog. Ask your stakeholders and Scrum team how they would size the cost of development, including the amount of effort involved from idea to deployment. Sometime a particular feature looks amazing and may attract your attention, but once you know the amount of effort involved in building it, then you, the customer, and the organization may decide to move it lower down the priority list.

Dependencies: Are there any dependencies affecting the development of the feature or story? At times you may need to order one product backlog against another if there are potential dependencies. Let's say we are ordering the product backlog for release and we have Feature A, which has three substories, one of which is dependent on Feature B -- and Feature B itself has two substories. The customer has made Feature A its priority. If you think you should develop the first two (non-dependent) substories out of the three for Feature A, talk to the PO and customer and discuss whether that will deliver sufficient value. If they say yes, then deliver feature A with two substories. But if delivering two substories from Feature A is not delivering value to the customer, then move Feature B to the top of the product backlog, which will remove the dependency of that third Feature A substory.

Learning and new knowledge: If you have a few stories and features in the product backlog that help bring the team's knowledge about the project to the next level and remove feature development uncertainty, then such stories should be brought to the PO's attention and moved to the top of the product backlog. Sometime a few stories related to a GUI can be good candidates to move to the top of the product backlog, because they will help the team understand product behaviors, work flow, and user attributes; this will help the team build its knowledge about the project and reduce uncertainty.

Ordering the backlog based on value, risk, rewards/ROI, and success

Now that I've discussed core factors that influence product backlog ordering, I would like to give ideas for ordering your product backlog based on factors such as value, risk, rewards/ROI, and success. This method can be use while ordering the product backlog at the portfolio and program levels. Many organizations want to implement Agile in their product/service development. Some are doing excellently in terms of their quarterly and yearly financial numbers, and such organization always try to focus on innovation and new product development. Such organizations should focus on ordering their product backlogs, program backlogs, and portfolio backlogs as in the diagram below. Organizations that are struggling to meet their financial goals should look at their product backlogs carefully, especially item 2 (the red box below).


When it comes to ordering your product backlog under any project, you can go with the following order. Here when you are taking high-value, high-risk features to the top of the product backlog, you are removing uncertainty about the product and unknown dependencies.



When you consider the above factors in prioritizing, consider the value of the feature it delivers to the customer compared to what it would cost to develop. This will give you an initial level of ordering the product backlog. Once you get a number for this calculation, then you can further think about the "learning and new knowledge" factor. If your team and PO think there is great uncertainty about product behavior, then respective stories should move to the top of the product backlog, which will help remove uncertainty; it will help development run smoothly and the cost of changes will be at a minimum. Once you order the product based on the factors above, then the final factor you should look at is the "risk." You can look at identified risk and move related stories that are creating risk to meet the objectives of the release, sprint, and customer to the top of the product backlog.

Three-layer approach for product backlog ordering

First level of ordering

If you observe below, Feature A has more priority for its value than Feature B. However, if you see the cost of development for those two features, then you can see that the amount of effort involved in developing feature A is higher than for B. Hence the priority of feature B is higher than that of A.


Second level of ordering

While ordering based on learning and new knowledge factors acquired by developing new features, then it looks as though developing new features will help the team learn more about the future product and remove uncertainty.

Third level of ordering

Once you find features/epics that bring value to remove risk and dependencies or that meet project team objectives, such features should be reordered in the product backlog after taking a look at first- and second-level ordering. If the team thinks it makes sense to develop Feature E first and then Feature D, as that will help remove risks that may create difficulty in meeting project or sprint objectives, and the team has discussed this with the PO, the final product backlog order could be as below:


After ordering, the PO and stakeholders can make the final call on approval of the product backlog order so that the team can start release planning.

Two-field approach to product backlog ordering

The three-layer approach discussed above can used when the organization is planning an immediate release. But often, as at the portfolio and program level, planning is done at a very high level and ordering the product backlog with the above approach is time-consuming and tedious.

So for large-grain ordering, we can set up the product backlog with two fields: Importance and Urgency. The Importance field allows the PO and stakeholders to aggregate customer value and satisfaction. The Urgency field adds the time component to the ordering. Separating these two pieces of data allows multiple views of the ordering to be done and trade-offs to be made -- for example, what is the most urgent story to do versus the most important. As the ScrumMaster, you meet with the PO at least weekly to review the product backlog and order any newly added items.

Our two fields are Importance (1 = low value, nice to have; 50 = high value, must have) and Urgency (1 = low urgency, 21 = customer may face danger/product shutdown/organization losing revenue). There is also a third, which is simply the sum of the two. With these numbers, the highly important, highly urgent stories move to the top. We can then look at tentative high-level story point estimates to make the final, tie-breaking decision, and the PO and product management are able to make sure that the backlog is always ordered just right to run business.

Ordering based on the Kano model

The Kano customer satisfaction model was developed by Dr. Noriaki Kano. The Kano model effectively analyzes the importance of attributes in driving product value for the customer. The Kano model is used to address these questions:
  • What about my product or service is important to my customers?
  • What are things that my company must do well in order to succeed?
  • Which improvements will have the greatest overall impact on customer satisfaction?
  • What are our strengths relative to our competitors?
The Kano model focuses on three states of satisfaction. First are "threshold" or "mandatory" features -- these features are the basic ones needed for the customer to be satisfied. Second are the "linear" features, which increase the satisfaction of the customer to the next level; this increases as you deliver more linear/desired features. Third are "exciter/delighter" feature. which the user doesn't imagine until they see them.

Mike Cohn has given a technique to order the product backlog in his book Agile Estimation and Planning. He suggests that to best use the Kano model, there is a set of questions and a survey you should conduct before you come to any conclusion. To decide on this you can take relative guess or you can survey 25 to 30 customers and/or stakeholders and then categorize.

Below are two tables that give functional and dysfunctional questions that can be asked in a user survey. The collective data will determine where a feature lives under the relevant quadrants.




Let's take an example where I want to access my feature epic of "Backup and restore contacts and phonebook." Every user response needs to be categorized in the format below:

M: Mandatory L: Linear E: Exiciter I: Indifferent R: Reverse Q: Questionable

The diagram below shows a sample survey report from one user. You can collect the same data from different users and then collect the number around each E/I/R/Q/L/M category. The count for each letter will give you the feature position under the Kano model. This is a very effective technique that you can use for prioritization at the portfolio, program, and user story levels.



Ordering feature based on cost of delay

This is most effective and recommended approach for product backlog prioritization. This highlighting is based on the mathematics and underlying economics of Lean product development, explained in Reinertsen's Principles of Product Development.

Now, how to calculate cost of delay includes three components: "user value," "time value," and "risk reduction value."
  1. User value: What is the potential value of the feature in the eyes of the user/customer.
  2. Time value: Another relative estimate based on how user value decays over time; many features have high value when they are delivered early in the market, and that value is reduced over time.
  3. Risk reduction: If we develop this feature, how it will help reduce risk for users or reduce risk in meeting the release objective and goals or sprint objective and goals?
Finally, efforts are numbers that indicate the amount of effort it will take to develop and deploy that particular feature. These are just relative numbers taken from story points or team judgment. The feature-ordering matrix based on the above points will appear as below.

After developing this chart, we have to order the feature/epic ranking under which it is expected to develop and deliver more value and economic benefits to users. CoD gives us the value of that feature, and it means the value is ROI (Feature C has more value as compared to Feature A). But if we look again at the efforts column, then to deliver Feature C you may need more resources than another Feature A (due to feature criticality), which has low CoD. Feature C will give you less ROI though it has higher CoD value than A, as the amount of resources you will burn to develop the particular feature is more. Here features should be developed in order B-A-C.

This is a tested and fast way to prioritize features, and this technique can be used to order your backlog at portfolio and program levels, and it can even apply at the iteration level.

Ordering the product backlog financially

Net present value, internal rate of return, and payback period are three major factors to consider when ordering the product backlog on financial factors.

Basing the order on net present value is fast, but the primary disadvantage to NPV is that comparing the values of two different cash flows can be misleading. So the next factor to consider is the internal rate of return (IRR), the measure of how quickly the money invested in a project will increase in value. In the table below, most teams will try to prioritize Feature B before A, as it has higher IRR.

Another factor is payback period, the time when the organization will start earning positively from the investment.

We add the discounted cash flow factor under each cash flow, and then you can calculate the discounted payback period. When you are prioritizing, you have to consider these three factors: NPV, IRR, and discounted payback period. Last but not least, you should keep the rest of the factors (risk, resource & uncertainty, etc.) in mind as well. If you include all three factors for financial prioritization and you have any tentative numbers, you may get data as below:

Even doing all this detailed analysis, it not certain that your product backlog is correctly ordered. You can think of several factors such as the amount of effort, risk reduction, customer wish list, etc., that must be taken into consideration before you announce your potential release/product.

Ordering using the MoSCoW model

When dealing with business value and prioritization, many product owners don't understand how to define and stick to a definition of Business value. One way to prioritize your product backlog is using the MoSCoW method. The acronym are "Must have," "Should have," "Could have," and "Won't have."

Every requirement documented must be tracked back to an objective. If a requirement doesn't trace directly to the initial objectives, then this is a "Should to have" or a "Could have" requirement. This process is bit faster than the others, but when you are categorizing detailed responses, then you need to access features very carefully.

Finally, when you are done ordering your product backlog, then the next job is to launch your release planning event. Before you jump to release planning, you can do a quick assessment of your product backlog; this will make the PO, stakeholders, and development team members more confident. Here is quick way to determine the maturity of the product backlog before you move to your release planning event.

In the diagram below we have Features A, B, C, D, and E, ordered based on one of the processes discussed above. The Order column shows the order of that feature. The Detail column shows how product backlog items are detailed appropriately; for example, the higher number 8 shows that it is detailed and there are fewer chances to modify it. The third column shows how the product backlog is estimated in story points and how confident the team is about this estimate; higher numbers mean more confidence. The fourth column shows how emergent product the backlog is, which means what are the chances of it evolving and that content changes will be frequent. The higher the number in the emergent column, the lower the chance the PB will emerge. And finally, the fifth column shows the prioritization field.

Now looking at the total you can an overview of your product backlog. The customer, PO, and team members can do some refinement and improve on areas where they see fewer points.

Let's consider the above chart, in which you can see that Feature B is the top priority. However, that feature is not detailed and it has a high chance of emerging, so the team is less confident about its estimates. You can see clear improvement items on Feature B, so the PO can act proactively before team starts working. Once you modify required sections of any feature or user story to meet deep attributes, then you are ready to plan and commit to your release and sprints.

References

Cohn M. Succeeding with Agile.

Pichler, R. Agile Product Management with Scrum: Creating Products that Customers Love.

Rubin K. Essential Scrum: A Practical Guide to the Most Popular Agile Process.

Sutherland J, van Solingen R, Rusenberg E. The Power of Scrum.




Opinions represent those of the author and not of Scrum Alliance. The sharing of member-contributed content on this site does not imply endorsement of specific Scrum methods or practices beyond those taught by Scrum Alliance Certified Trainers and Coaches.



Article Rating

Current rating: 5 (12 ratings)

Comments

Amol Munde, CSM,CSPO, 12/23/2016 2:43:52 AM
This is a very detailed article and had covered all views for how to order the product backlogs. A very nice and informative article. I got the information which I'm looking for it...!
Sameer Chudaman Patil, CSP,CSM,CSPO, 3/16/2017 9:14:33 PM
Thanks

You must Login or Signup to comment.


The community welcomes feedback that is constructive and supportive, in the spirit of better understanding and implementation of Scrum.


 

Newsletter Sign-Up

Subscribe