Priority Markets

A Free Market Approach to Managing the Product Backlog

4 February 2009

Geoff Watts
Inspect & Adapt

Jason Haines
Man Investments

Background

The product backlog is the master list of desired functionality for projects being developed using Scrum. Prioritising it is notoriously difficult. A number of different approaches have been tried without any panacea being found. We don’t claim to have found a cure-all but we have devised another tool for teams to consider, especially when faced with numerous stakeholders.

We prioritise the product backlog to enable us to determine the optimal order in which to deliver functionality: we want to complete those stories with the highest business value first. One way to determine a story’s business value is to calculate the return on investment (ROI) of the functionality. In its simplest form, ROI is a ratio of a story’s value to its implementation cost. Measuring a story’s value is harder than it seems, though, especially when more than one person is asked to value something; value is often personal or, at least, context specific.

Typical methods of prioritization include:

  1. Whomever shouts the loudest
  2. Averaging priorities
  3. Forced ranking
  4. MoSCoW
  5. Kano model

All of these models involve human interpretation, so none of them are perfect, but they are all useful in certain situations. We tried almost every existing way of prioritising our backlog before we decided we needed to try something different.

Prioritisation Markets

We used a market-based approach, where each stakeholder is given some virtual money in the form of development dollars, which they can use to bid on backlog items.

The method is fair and democratic in that all stakeholders get a voice in the prioritisation process. It is also open, in that all participants can see how each backlog item was prioritised. The method is also relatively efficient to administer and scales well to large numbers of stakeholders and arbitrarily long lists of backlog items.

Step 0: Set up

We’ll assume that the project already has a backlog of development items. For our example we have six backlog items with differing development costs.

The first step is for the product owner to identify the different stakeholders who will be involved in setting the development priorities. The PO should then assign each stakeholder a relative weighting of their importance to the project.

For the sake of simplicity in our example, we will assume four stakeholders ( John, Paul, George and Ringo) with an equal weighting of 1.0 each.

Now we introduce the concept of development dollars. Development dollars will be used by the stakeholders to bid for and buy changes in the system.

We first create a “bank” of unallocated development dollars. To start the process, we inject an initial number of development dollars into the system.  The exact number is not critical; practice shows that around $10 per stakeholder gives a reasonable result.

At the start we then have a table as shown in Figure 1:

Figure 1. Six backlog items and four stakeholders of equal weight.
Step 1: Dollar distribution

Now we distribute the unallocated development dollars to the stakeholders based on their weights. After the distribution, each stakeholder has a number of unspent development dollars.

Figure 2. Start with a certain number of unallocated dollars ($40).

Figure 3. Divide the unallocated dollars among stakeholders according to weight ($10/each).
Step 2: Bidding

The stakeholders now bid their development dollars on the backlog items as per their preferences.  Development dollars that have been bid are deducted from their unspent dollars.  Imagine the stakeholders allocate their dollars as follows:

Figure 4. Each stakeholder bids on the PBIs that have the highest priority for him or her.
Step 3: Priority calculation

We next determine the relative value of each item by calculating the total bid of development dollars allocated to it. The developers can then estimate the development cost of each backlog item that has a bid. From there we can calculate the return on investment (ROI) of each backlog item by dividing the total bid by the development cost. Ordering the backlog by largest to smallest ROI gives us our priorities:

Figure 5. Use the total bid and development cost to determine the ROI for each PBI. Priortize based on ROI.

Some notes to make on the priorities:

  • Ordering by ROI tells us to deliver items in terms of best-bang for the buck.  Although Change C has the largest total bid, it also has a large development cost and so is further down the priority list.
  • Change F is at the top of the priorities even though it is important to only a single stakeholder. Given that Ringo allocated $7 to it, Change F is clearly critical to him.
  • Change A has had no development dollars bid against it and is thus at the bottom of the list.  In a typical project there will be a large number of items that have no Dollars bid against them. This is actually an advantage of the method, as the stakeholders can concentrate their attention on a limited number of critical items on the Product Backlog. Also note that it is not critical to have a development cost estimate at this time for any items that received no bids.
Step 4: Develop for an Iteration

The prioritised backlog is then taken by the developers so they can schedule work for an iteration.

Say that the developers have the development capacity to complete the first two items, Change F and Change D, in the next iteration. These changes are then implemented.

Step 5: Reallocation of Dollars

Change F and Change D have been completed and can be archived. The dollars spent on them are now be reallocated. The two backlog items had a total of $7 + $9 = $16 spent on them. This amount is first put into the unallocated bin. The completed features (F and D) are removed from the bidding table.

Figure 6. Completed features F & D are removed from the table. The dollars allocated to them are put back in the unallocated bin.

For the next round of prioritisation, we then reallocate these dollars among all the stakeholders (note that the dollars do not go back to the original bidders) as we did in Step 1.


Figure 7. The unallocated dollars are divided among the stakeholders. They can add those dollars to existing bids or reallocate all of their allotted dollars.

We then proceed as before. Each stakeholder bids on items in the product backlog using their unspent dollars (in our example each stakeholder has $4). The amount bid is added to dollars bid before. Alternatively, a stakeholder can choose to reallocate the dollars he has bid previously if her priorities have changed. For any bid-upon items that do not already have a development cost, a cost is calculated. ROI is then calculated by dividing the total bid by the development cost. A new prioritised list is available for the next iteration.

By redistributing development dollars and allowing the stakeholders to change previous bids, we are agile in response to changes in the stakeholders’ priorities.

That’s it! The process then repeats iteration by iteration. Dollars spent on each iteration are reallocated and re-bid.

Determining stakeholder weightings

The example above assumed equal weighting for each stakeholder.  In a real-life project, some stakeholders will typically be less important than others. For example, stakeholders that only require a few reports from a system may be less important than the stakeholders that are responsible for data entry.

This can be handled in the Prioritisation Market by assigning different weighting to the different stakeholders. This will affect the number of development dollars each stakeholder is allocated.

Let’s modify our previous example with different weightings and then allocate our initial $40 in development dollars according to these values:


Figure 8. Dollars are allocated based on stakeholder weighting for more accurate ROI results.

Here we see that John is the most important stakeholder, George is the least and Paul and Ringo are considered equal. Their allocations reflect their relative importance.

Weightings will have a very direct outcome on how much development effort is devoted to each stakeholder. In this example, John will receive 50% more development effort than George.

Determining who the stakeholders are and their individual weightings can be a politically challenging exercise. If the stakeholders cannot agree on a weighting  among themselves, this decision should be made by the product owner, or senior product owner, if applicable.  Whoever makes this decision must be someone sufficiently high in the organisational hierarchy.  In many regards, it is analogous to a departmental budgeting process. An advantage of this approach is that management can make a single upfront decision about the relative priorities of the stakeholders. This avoids them having to micromanage the relative merits of individual backlog items and empowers individual stakeholders with control of the priorities.

Benefits of Priority Markets

Priority Markets offer additional benefits to all project participants.

Firstly the method is scalable by the number of backlog items. In projects with hundreds of outstanding backlog items, the priority market allows the stakeholders and the development team to focus only those changes that are of immediate relevance. This avoids the trap that many teams fall into of vigorously debating every item of a very long backlog. Items that do not have development dollars allocated to them stay in the backlog for consideration in future iterations.

Priority Markets also scales well with the number of stakeholders. Each stakeholder can bid independently of the others. A stakeholder can quantify the importance of their change request by bidding on them. This neatly avoids long discussions about why one backlog item may be more important than another.

Backlog prioritisation done through Priority Markets is an open process. At any time, each stakeholder can know the priorities of the other stakeholders and see how development dollars have been spent. Project priorities are openly set and transparent to all.
The process is also fair. If a stakeholder hasn’t had any of their backlog items implemented in recent iterations they will have proportionally more dollars to spend in future iterations.

The Psychology of the Market

By monetising the prioritisation process, some interesting behaviours can emerge.

Bargaining: Stakeholders can see one another’s bids and adjust their bidding accordingly.

Trading: One stakeholder can “sell” their development dollars to another stakeholder for in-kind benefits.

Tactical voting: Based on others bids, a stakeholder may allocate more or fewer dollars to certain items.

Governmental intervention: When the free market generates priorities that are not in the strategic interests of the project, the Product Owner may step in and allocate additional dollars to a particular item. As with any free market, governmental intervention can have a number of negative consequences: extra dollars injected into the system reduces the relative value of other dollars, resulting in inflation; stakeholders may also feel disenfranchised by having their priorities overridden.

Conclusion

While there never will be a perfect solution to the difficult process of prioritisation, Priority Markets provides an efficient and effective way to allow multiple stakeholders to participate in a prioritisation process. Stakeholders can better focus on their immediate priorities, leaving inefficient negotiation and bickering behind.

 

Creative Commons License
This work is licensed under a Creative Commons Attribution-Noncommercial-No Derivative Works 3.0 Unported License.

If shared, please attribute by noting author name and providing a link back to this article on our site.


Opinions represent those of the author and not of Scrum Alliance. The sharing of member-contributed content on this site does not imply endorsement of specific Scrum methods or practices beyond those taught by Scrum Alliance Certified Trainers and Coaches.



Article Rating

Current rating: 0 (0 ratings)

Comments

Andy Brandt, CSM,CSPO, 2/11/2009 5:55:12 AM
Great idea, Geoff. Could be helpful in many large projects where numerous stakeholders are norm. I think this should be also supported by any Scrum/agile tool used by organizations to make the process as easy to manage as possible.
Geoff Watts, CST,CSC,CSP,CSM,CSPO,REP, 2/17/2009 2:00:10 PM
Thanks Andy
Most of the credit should go to Jason and it is quite a simple process that can be supported by tools as simple as a whiteboard/spreadsheet/wiki.
Jeff Allen, CSM, 3/3/2009 11:40:11 AM
I love the free market dynamic at play here. It gets people engaged and gets the desired end result of a prioritized backlog without the dry project weighting discussion that no one wants to have.
Jeffrey Hoffer, CSM, 3/23/2009 2:15:30 PM
I think it's quite brilliant in its simplicity. I've been working (off and on; more off than on) about prioritization mechanisms which tie requests back to dollars, and this seems the simplest route. In that work, I've understood dollars to be more of a relative than real measure. I like the use of dollars because it speaks the language of the stakeholders, be they sales, marketing, etc.

One question is how do Technical Debt items get prioritized in this fashion?
Would you allow the dev team a budget of dollars to allocate towards those items?
Geoff Watts, CST,CSC,CSP,CSM,CSPO,REP, 3/27/2009 11:14:33 AM
Thanks for the comments Jeff & Jeff.

Jeff H - I would have no problem with someone representing the dev team and bidding on things. However, I would say this would be something I would favour less than making the Product Owner aware of some of the issues of technical debt and making them aware of the benefits and consequences (there are limitations coupled with a certain conflict of interests). I would also prefer that a certain degree of the technical debt be automatically included in the next development features either as acceptance criteria or an implicit assumption that the team will always improve the state of the system they are working on. I would also consider a process of a partnership and/or influence/lobbying.
Having said all that, I would still consider your suggestion a valid one that should lead to a very healthy relationship amongst the various stakeholders and hopefully not an adversarial one.
Juan Lopez, CSM, 6/23/2010 7:04:21 AM
I just found this article through a LinkedIn post, great material! It kind of reminds me of the SEI's QAW, but the use of virtual money to cast votes makes the process more relevant.
Geoff Watts, CST,CSC,CSP,CSM,CSPO,REP, 6/23/2010 12:05:25 PM
Thanks Juan - I am glad you liked the article
Karen Smiley, CSM, 7/8/2010 4:09:00 PM
Hi Geoff,

Thanks for this interesting article. I'm currently looking for prioritization methods that have been successfully used with large industrial products and large Product Backlogs (eg hundreds of items). I have three questions.

1) Regarding scalability, with large products, a stratification step could help to reduce the pool of PBI's to consider for dollar-voting. Have you seen or done this kind of pre-voting stratification (using eg MoSCoW or Kano, which you noted above) in practice, and if so could you share your experiences with it?

2) Even with stratification on a large product, there might still be large classes or a big 'must' list to consider at sprint planning time. What is the largest number of PBI's on which you have seen this dollar voting method work well in practice? And were any adaptations needed to make it effective? (eg increase the number of voting dollars per stakeholder, in some proportion to the number of items)

3) Based on your experiences, do you think there is a practical limit on the number of stakeholders with which this method can be effective?

Thanks!
Karen :)

You must Login or Signup to comment.