This post is from the CollabNet VersionOne blog and has not been updated since the original publish date.
Agile prioritization: A comprehensive and customizable, yet simple and practical method
There is rarely enough time or resources to do everything. Therefore, agile teams must use prioritization to decide which features to focus on and which lowest rank-order features could be pushed out of scope when close to the end of a timeboxed sprint or release. For agile development projects, you should linearly rank-order the backlog, rather than coarse-graining prioritization where features are lumped into a few priority buckets, such as low, medium, high, and critical priorities. Linear rank ordering (i.e., 1, 2, 3, 4 ….n) avoids inflation of priority, keeps everyone honest, and forces decisions on what is really important. It discourages the “kid-in-a-candy-shop” behavior when the business side clamors that everything is high priority or equally important.
For agile and scrum projects, the sprint backlog contains backlog items including stories, defects, and test sets. The scope of work for each backlog item is small enough to fit inside a typical short two- to four-week sprint. Epics (large stories) present in a release backlog may not have been decomposed into stories during release planning; they will be decomposed during sprint planning.
The responsibility of agile prioritization is shared among all members of a team; however, the prioritization effort is led by the product owner. Here, I will use the generic term “feature” to denote an epic or a story. The context will make it clear if “feature” means “story” (in the context of a sprint backlog) or “epic” (in the context of a program or a portfolio, and release backlog). Note that an epic rank order is separate from a backlog item rank order. Epics and backlog items are conceptually different, and should not be mixed or aggregated while developing a rank order.
In this blog post, I will present a simple but fairly comprehensive method to linear rank-ordering a backlog. The method is extensible, customizable, and very practical.
Let me start with a simple example. Table 1 shows a sample backlog of five features (F1 – F5) along with each feature’s “total value” (column B) and “total effort” (column D).
Table 1: Sample backlog with total value, total effort, total ROI, and rank order
The total value for features is estimated by the agile team; it is a relative number and not absolute (such as dollars). Similarly, the total effort for features is estimated by the agile team; it is also a relative number, not absolute (such as ideal days or hours). Relative numbers are easier to estimate than absolute numbers, and are adequate for agile prioritization. The total value for all five features is 1,791, and the total effort is 14. Percent total value (column C) is calculated with simple math; for example, % total value of feature F1 is (190/1791) = 10.61%. Similarly, % total effort (column E) is calculated similarly; for example, % total effort of feature F3 is (3/14) = 21.43%. Figure 1 (below) illustrates the % total value and % total effort for each of five features, F1-F5, as two pie charts. Note that, by definition, the full pie of either total value or total effort always represents 100% of total value or total effort.
Figure 1: % Total value and % total effort distribution of five features in the sample
Column F of Table 1 calculates total return on investment (TROI, a relative number) for a feature as the ratio of % total value of the feature (column C) to % total effort of the feature (in column E). This is an economic model that tells us how valuable a feature is based on its TROI. Column G of Table 1 calculates % TROI for each feature; for example, for feature F2, its % TROI is (0.87/5.72) = 15.18%. Column H of Table 1 shows the rank order of five features based on their % TROI data. F4 is rank-ordered # 1, F3 as #2, F2 as #3, F5 as #4 and F1 as #5.
What is the total value and how do you calculate it?
I now present a comprehensive definition of “total value,” yet simple to calculate and very practical to use for agile projects. First let’s start with the notion of value of a feature to its provider (the organization developing the product or solution). This “provider value” consists of parameters, such as:
- Revenue increasing parameters: Examples of these parameters are number of new sales, increase in market share, cross-sales or upsell opportunities, etc.
- Other benefits parameters: Examples of these parameters are alignment with the product strategy, intellectual property (patents) creation, etc.
- Cost savings parameters: Examples of these parameters are operational cost savings, customer support cost savings, etc.
Depending on your organization’s business and product, some of these parameters may or may not be applicable to you. If you are an IT organization developing solutions for internal use, “revenue increasing parameter” is not very applicable or may need suitable modifications.
Donald G. Reinertsen has proposed a rigorous economic framework to prioritizing value delivery (“Principles of Product Development Flow, 2009”). He points out that, “If you quantify one thing, quantify the cost of delay (CoD).” As Dean Leffingwell has pointed out, CoD of a feature is an aggregation of three factors: user value, time value and opportunity enablement, risk reduction value (“Agile Software Requirements,” Chapter 12, pp. 266-267).
- User value: The potential value of a feature (expressed in relative terms) in the eyes of the users.
- Time value: A relative estimate based on how the user value decays over time. Features deliver higher value when delivered early, and a lower value when delivered later by the time the feature has become commoditized. As an example, time value for “implement new UI standard with new corporate branding” may be at best modest; on the other hand, “implement new 2014 federal tax law provisions by Dec. 31, 2014” has an extremely high time value prior to Dec. 31, 2014 if you are developing a tax package for tax year 2014.
- OR value: Opportunity enablement, risk reduction value:
- Opportunity enablement: Work done on a capability for one or more features may be more valuable to us depending on how it helps us exploit new opportunities. As an example, “LDAP for distributed directory information service” capability implemented with one or more features may open up new opportunities in the future.
- Risk reduction: Risk is anything that hasn’t happened yet, but might, and would jeopardize the success of the project. As an example, work done on “use an alternate component to deliver required performance” for one or more features may be a good risk reduction approach.
Two works done independently by Noriaki Kano and Karl Wiegers are very interesting and can help us in agile prioritization by modeling agile value. The Kano model, illustrated in Figure 2, shows how customer satisfaction changes with absence or presence of features, and the impact of feature enhancements on customer satisfaction.
Figure 2: The Kano model of customer satisfaction
- Must-have, mandatory feature: There is low to moderate customer satisfaction for including the feature, but a high penalty for excluding it. When this feature is present, user satisfaction remains low until a threshold is reached. Thereafter, enhancing the feature produces a less-than-proportional reward. The center point (the present line in Figure 2) of this curve gives rise to the minimal marketable feature (MMF). For a solution to be viable, it must contain some requisite set of MMFs. However, enhancing an MMF will not produce a proportional economic return; i.e., gold plating this feature does not offer much value. Typically, these features have either become a commodity (all competitors offer them), or they are required for some compliance or regulatory reasons.
- Game changer, exciter, and delighter feature: This feature is unique, compelling, and differentiated; even a small investment produces high customer interest and satisfaction. There is high customer satisfaction for including the feature, but no or low penalty for excluding it. Additional investment produces higher proportional satisfaction. These features provide the greatest leverage for investment. As an example, consider a car navigation device that gives accurate navigation guidance based on real-time traffic conditions, congestion, and accidents using “crowd-sourced” intelligence.
- Linear performance feature: Investment in this feature returns proportionally higher user satisfaction; i.e., the performance is linear (see the diagonal line in Figure 2).
Note that with passage of time, game changers, exciters, and delighters as well as linear performance features may become must-have features in a market that is intensely competitive, such as the mobile device market. What is a game changer today (e.g., voice recognition or maps) may become a commodity feature in few years.
Karl Wiegers’ model (“First Things First: Prioritizing Requirements,” published in Software Development, September 1999) recommends an approach that is similar to the Kano model because it considers both the positive benefit of having a feature and the negative impact of its absence.
The user value, as proposed by Reinertsen and Leffingwell, can be calculated based on the following two parameters inspired by the Kano and Wiegers models:
- Delighter parameter: User delight for including the feature
- Penalty parameter: Penalty for excluding the feature
The information presented so far gives us a good foundation to define the total value. Note that all factors or parameters in the equation below are estimated, relative numbers (not absolutes).
Total value = provider value + user value + time value + OR value
Furthermore, total value can be based on parameter weights; i.e., it can be weighted total value. Table 2 illustrates how to combine all the above parameters into a weighted total value model.
Table 2: Weighted total value model
Each parameter can be assigned a parameter weight in the range of zero to ten. Weight value zero means the parameter should not be considered (it has no weight or impact). Weight value one means the parameter has the least weight (or impact), while ten means the parameter has the most weight. As an example, if the parameter “increase market share” has a weight of nine, while parameter “LDAP for distributed directory information service” has a weight of three, then the weight of “increase market share” is three times the weight of “for distributed directory information service.” If all parameters have the same non-zero weight (such as three or eight), the impact of the weight is identical and, effectively, parameters are not weighted relative to each other because they all have the same weight.
Parameter values reflect an agile team’s collective judgment or consensus based on the discussions between the product owner and the cross-functional agile team members. Parameter value zero means the parameter is not relevant for that feature in the row (the parameter has no importance). Parameter value one means the parameter has the least importance, while ten means the parameter has the most importance.
The weighted total value for each feature is the weighted sum of products of parameter weight and parameter value. For example, for feature F237, the weighted total value = (7*5) + (9*0) + (8*0) + (7*5) + (8*3) + (6*6) + (9*4) + (10*9) + (10*10) + (3*0) + (6*9) = 410. The weighted total value of all five features is 1,748. Therefore, % weighted total value for feature F237 is (410/1,748) = 23.46%. Note that the weighted total value for each feature is still relative.
Note that the weighted shorted job first (WSJF) method used by the Scaled Agile Framework® (SAFe™) is subsumed by the TROI method proposed earlier. WSJF captures only user value, time value, and OR Value. It does not take into consideration provider value, an important part of the total value. It neither requires nor precludes the weighted parameter model proposed in Figure 2. It also does not specify how to calculate the user value, and is not tied to the Kano or Wiegers model. Thus, the total value model is more general than the WSJF model of SAFe. You may choose a subset of the total value model to create the WSJF model, if that’s what you decide to do.
The TROI method can be subset to a very simple method of ROI = (user value / effort) or ROI = (provider value / effort). But it would be too simplistic to do so; ROI simply ignores the time value. A higher ROI feature may be less sensitive to schedule delays than a lower ROI feature that is more sensitive; in this case, a simple ROI method will yield a wrong rank order. So avoid using simple ROI method. Both TROI and WSJF capture the cost of schedule delays. As explained above, TROI subsumes WSJF.
What is the total effort and how do you estimate it?
Agile teams usually take into account only development and testing effort for each feature, and estimate relative effort in story points. I suggest that the team (consisting of cross-functional skills) takes into account not only development and testing effort, but also training (customer support staff training and customer/end-user training), delivery, deployment, operations, and customer support for each feature. Total effort needs to include not just development and testing effort, but all of the efforts listed above. If a project is a small project of a single team that is largely stable, using story points will suffice. Story points represent the relative estimated effort and serve as a good, reliable proxy for incurred cost or investment needed to develop, deliver, operate, and support a feature.
For large projects with multiple teams (organized perhaps into programs and portfolios), story points estimated by each team must be normalized so they have the same scale and semantics; otherwise, combining story points across teams — or doing any math based on story points — does not have a well-defined meaning. You may use the calibrated normalization method to normalize story points across multiple teams of a project. This method is applicable to backlog items as well as epics, even when epics are not broken down into stories. Using the calibrated normalization method, the estimated relative effort for all features (epics and stories) and work items (defects, spikes, regression testing, reducing technical debt, etc.) can be expressed in a single, common currency of normalized story points, and has the same meaning across the whole enterprise.
- All prioritizations are local and temporal: Note that, by my definitions here, the total value of a feature is a global property of the feature, while total effort is a local property of the team implementing the feature. A feature with a lower-weighted total value may require less relative total effort from a specific team, and therefore, should be implemented ahead of another higher-weighted total value feature, if that feature requires more total effort from the team. All priorities are inherently local.
- Prioritization changes as time goes on: Rank order of a backlog may need to be readjusted or recalculated at the end of a sprint, release cycle, or major event. As features are added, deleted, revised, refined, groomed, and as more knowledge becomes available, it will be necessary to recalculate the rank order of a backlog. Game changer features may become basic-commodity, must-have features. Make sure that a rank order indicating linear priority does not create rigidity or reduce agility. It should only provide guidance to the team in deciding what to focus on and what may be pushed out as the team approaches the end of the timebox. Also, don’t prioritize too early; prioritize as needed, closer to “just in time.”
- Keep the TROI model simple: Pay attention to the parameters influencing the provider value, user value, time value and OR value as appropriate to your business and your situation. But don’t overdo it. You may even subset the model to match it exactly with the SAFe WSJF if you are working on SAFe projects, and need to follow the WSJF-based prioritization.
- Financial calculations are not required: Often, business managers ask for specific financial information, such as how much expected revenue the product will generate, or expected cost savings. Those questions are much more difficult to answer, and are not essential to doing agile prioritization. Relative total value and relative total effort numbers are adequate and are easier to estimate as shown in this blog post.