The product backlog is the master list of desired functionality for projects being developed using Scrum. Prioritising it is notoriously difficult. A number of different approaches have been tried without any panacea being found. We don’t claim to have found a cure-all but we have devised another tool for teams to consider, especially when faced with numerous stakeholders.
We prioritise the product backlog to enable us to determine the optimal order in which to deliver functionality: we want to complete those stories with the highest business value first. One way to determine a story’s business value is to calculate the return on investment (ROI) of the functionality. In its simplest form, ROI is a ratio of a story’s value to its implementation cost. Measuring a story’s value is harder than it seems, though, especially when more than one person is asked to value something; value is often personal or, at least, context specific.
Typical methods of prioritization include:
- Whomever shouts the loudest
- Averaging priorities
- Forced ranking
- Kano model
All of these models involve human interpretation, so none of them are perfect, but they are all useful in certain situations. We tried almost every existing way of prioritising our backlog before we decided we needed to try something different.
We used a market-based approach, where each stakeholder is given some virtual money in the form of development dollars, which they can use to bid on backlog items.
The method is fair and democratic in that all stakeholders get a voice in the prioritisation process. It is also open, in that all participants can see how each backlog item was prioritised. The method is also relatively efficient to administer and scales well to large numbers of stakeholders and arbitrarily long lists of backlog items.
Step 0: Set up
We’ll assume that the project already has a backlog of development items. For our example we have six backlog items with differing development costs.
The first step is for the product owner to identify the different stakeholders who will be involved in setting the development priorities. The PO should then assign each stakeholder a relative weighting of their importance to the project.
For the sake of simplicity in our example, we will assume four stakeholders ( John, Paul, George and Ringo) with an equal weighting of 1.0 each.
Now we introduce the concept of development dollars. Development dollars will be used by the stakeholders to bid for and buy changes in the system.
We first create a “bank” of unallocated development dollars. To start the process, we inject an initial number of development dollars into the system. The exact number is not critical; practice shows that around $10 per stakeholder gives a reasonable result.
At the start we then have a table as shown in Figure 1:
Figure 1. Six backlog items and four stakeholders of equal weight.
Step 1: Dollar distribution
Now we distribute the unallocated development dollars to the stakeholders based on their weights. After the distribution, each stakeholder has a number of unspent development dollars.
Figure 2. Start with a certain number of unallocated dollars ($40).
Figure 3. Divide the unallocated dollars among stakeholders according to weight ($10/each).
Step 2: Bidding
The stakeholders now bid their development dollars on the backlog items as per their preferences. Development dollars that have been bid are deducted from their unspent dollars. Imagine the stakeholders allocate their dollars as follows:
Figure 4. Each stakeholder bids on the PBIs that have the highest priority for him or her.
Step 3: Priority calculation
We next determine the relative value of each item by calculating the total bid of development dollars allocated to it. The developers can then estimate the development cost of each backlog item that has a bid. From there we can calculate the return on investment (ROI) of each backlog item by dividing the total bid by the development cost. Ordering the backlog by largest to smallest ROI gives us our priorities:
Figure 5. Use the total bid and development cost to determine the ROI for each PBI. Priortize based on ROI.
Some notes to make on the priorities:
- Ordering by ROI tells us to deliver items in terms of best-bang for the buck. Although Change C has the largest total bid, it also has a large development cost and so is further down the priority list.
- Change F is at the top of the priorities even though it is important to only a single stakeholder. Given that Ringo allocated $7 to it, Change F is clearly critical to him.
- Change A has had no development dollars bid against it and is thus at the bottom of the list. In a typical project there will be a large number of items that have no Dollars bid against them. This is actually an advantage of the method, as the stakeholders can concentrate their attention on a limited number of critical items on the Product Backlog. Also note that it is not critical to have a development cost estimate at this time for any items that received no bids.
Step 4: Develop for an Iteration
The prioritised backlog is then taken by the developers so they can schedule work for an iteration.
Say that the developers have the development capacity to complete the first two items, Change F and Change D, in the next iteration. These changes are then implemented.
Step 5: Reallocation of Dollars
Change F and Change D have been completed and can be archived. The dollars spent on them are now be reallocated. The two backlog items had a total of $7 + $9 = $16 spent on them. This amount is first put into the unallocated bin. The completed features (F and D) are removed from the bidding table.
Figure 6. Completed features F & D are removed from the table. The dollars allocated to them are put back in the unallocated bin.
For the next round of prioritisation, we then reallocate these dollars among all the stakeholders (note that the dollars do not go back to the original bidders) as we did in Step 1.
Figure 7. The unallocated dollars are divided among the stakeholders. They can add those dollars to existing bids or reallocate all of their allotted dollars.
We then proceed as before. Each stakeholder bids on items in the product backlog using their unspent dollars (in our example each stakeholder has $4). The amount bid is added to dollars bid before. Alternatively, a stakeholder can choose to reallocate the dollars he has bid previously if her priorities have changed. For any bid-upon items that do not already have a development cost, a cost is calculated. ROI is then calculated by dividing the total bid by the development cost. A new prioritised list is available for the next iteration.
By redistributing development dollars and allowing the stakeholders to change previous bids, we are agile in response to changes in the stakeholders’ priorities.
That’s it! The process then repeats iteration by iteration. Dollars spent on each iteration are reallocated and re-bid.
Determining stakeholder weightings
The example above assumed equal weighting for each stakeholder. In a real-life project, some stakeholders will typically be less important than others. For example, stakeholders that only require a few reports from a system may be less important than the stakeholders that are responsible for data entry.
This can be handled in the Prioritisation Market by assigning different weighting to the different stakeholders. This will affect the number of development dollars each stakeholder is allocated.
Let’s modify our previous example with different weightings and then allocate our initial $40 in development dollars according to these values:
Figure 8. Dollars are allocated based on stakeholder weighting for more accurate ROI results.
Here we see that John is the most important stakeholder, George is the least and Paul and Ringo are considered equal. Their allocations reflect their relative importance.
Weightings will have a very direct outcome on how much development effort is devoted to each stakeholder. In this example, John will receive 50% more development effort than George.
Determining who the stakeholders are and their individual weightings can be a politically challenging exercise. If the stakeholders cannot agree on a weighting among themselves, this decision should be made by the product owner, or senior product owner, if applicable. Whoever makes this decision must be someone sufficiently high in the organisational hierarchy. In many regards, it is analogous to a departmental budgeting process. An advantage of this approach is that management can make a single upfront decision about the relative priorities of the stakeholders. This avoids them having to micromanage the relative merits of individual backlog items and empowers individual stakeholders with control of the priorities.
Benefits of Priority Markets
Priority Markets offer additional benefits to all project participants.
Firstly the method is scalable by the number of backlog items. In projects with hundreds of outstanding backlog items, the priority market allows the stakeholders and the development team to focus only those changes that are of immediate relevance. This avoids the trap that many teams fall into of vigorously debating every item of a very long backlog. Items that do not have development dollars allocated to them stay in the backlog for consideration in future iterations.
Priority Markets also scales well with the number of stakeholders. Each stakeholder can bid independently of the others. A stakeholder can quantify the importance of their change request by bidding on them. This neatly avoids long discussions about why one backlog item may be more important than another.
Backlog prioritisation done through Priority Markets is an open process. At any time, each stakeholder can know the priorities of the other stakeholders and see how development dollars have been spent. Project priorities are openly set and transparent to all.
The process is also fair. If a stakeholder hasn’t had any of their backlog items implemented in recent iterations they will have proportionally more dollars to spend in future iterations.
The Psychology of the Market
By monetising the prioritisation process, some interesting behaviours can emerge.
Bargaining: Stakeholders can see one another’s bids and adjust their bidding accordingly.
Trading: One stakeholder can “sell” their development dollars to another stakeholder for in-kind benefits.
Tactical voting: Based on others bids, a stakeholder may allocate more or fewer dollars to certain items.
Governmental intervention: When the free market generates priorities that are not in the strategic interests of the project, the Product Owner may step in and allocate additional dollars to a particular item. As with any free market, governmental intervention can have a number of negative consequences: extra dollars injected into the system reduces the relative value of other dollars, resulting in inflation; stakeholders may also feel disenfranchised by having their priorities overridden.
While there never will be a perfect solution to the difficult process of prioritisation, Priority Markets provides an efficient and effective way to allow multiple stakeholders to participate in a prioritisation process. Stakeholders can better focus on their immediate priorities, leaving inefficient negotiation and bickering behind.
This work is licensed under a Creative Commons Attribution-Noncommercial-No Derivative Works 3.0 Unported License.
If shared, please attribute by noting author name and providing a link back to this article on our site.