How to Reduce Agile Risk with Monte Carlo Simulation

I love to gamble, but I prefer to gamble with dice, not software delivery. Precisely predicting what can be completed in a release, and accurately accounting for risk, is one of the most important responsibilities of a program manager, IT leader, project manager or executive; yet the majority of our predictions have the same odds as winning the lottery. Given those odds, can you improve the accuracy of predicting when software will be delivered? Why Linear Forecasting Isn’t Optimal for Reducing Agile Risk For decades the people managing software projects have attempted to predict software delivery dates with mixed results. In most cases, these predictions relied on linear forecasting techniques. While these can be useful, they fail to consider the risk associated with human involvement. Linear techniques assume a fixed future velocity based on an average historical velocity. As any investment broker or financial advisor will tell you “past results do not guarantee future performance.” Consider a simple scenario where two teams both have an average velocity of 7.5. Using this average, either team could deliver the next 45 points in 6 iterations. However, suppose Team A’s historical velocities values were 10, 4, 5, and 11, and Team B’s historical velocities values were 7, 8, 8, and 7. The average is the same, but because the velocity values for Team A have a higher variance, there is greater risk when predicting future results. Furthermore, as stated above, the historical velocity values are not indicators of future performance. Either team could encounter a scenario that significantly improves or degrades velocity. Another problem with linear forecasting is the fact that it yields a single date; nothing more. Yes, you can forecast a cone of uncertainty using the best and worst historical values, but in the end you only have dates; the best case date, the worst case date and the average date. There is no information to help you determine why any given date is better or worse than another. Therefore, when asked to commit to a delivery date, you’re left with “trusting your gut” based on your knowledge of the team and other perceived trends. Unfortunately, many people would look at the upward trend of Team A and conclude “velocity is going up; that’s a good thing; they must be getting better.” The significant drop from 10 to 4 in the second iteration would be considered an anomaly “because the team was just getting started.” These factors result in a false confidence that leads to a more optimistic date commitment. Given nothing more than a forecasted delivery date, and limited insight into the probability of hitting any date, you are left negotiating a contract, the delivery date, with limited information. Therefore, it is not a surprise that we consistently pick the wrong date and scramble at the end to “deliver something”. There are better techniques for forecasting delivery dates that account for risk and give us the information necessary to effectively collaborate on a delivery date. Reducing Agile Risk using a Monte Carlo Simulation A Monte Carlo Simulation is a forecasting technique that can be used for predicting software delivery dates and accounting for risk. A Monte Carlo Simulation is “a problem solving technique used to approximate the probability of certain outcomes by running multiple trial runs, called simulations, using random variables.” The technique gets its name from the city of Monaco, a place renowned for its casinos long before the likes of Las Vegas or Atlantic City. Monte Carlo methods have been effectively used for years in the physics, engineering, financial application and even predicting election results. Douglas Hubbard, author of “How to Measure Anything”, states, “running Monte Carlo is the only way to analyze big uncertain decisions.“ A Monte Carlo Simulation considers the historical values of control variables, in our case velocity, as opposed to relying on a single value. It then simulates the completion of remaining work and produces a histogram showing the distribution of possible delivery dates. From those results, you can calculate various percentiles, or confidence levels, on each possible delivery date. You’re probably thinking, “Whoa! That sounds really complicated!” Well, maybe if you were building the simulation yourself, but there are solutions that make it very simple. VersionOne is the first agile lifecycle management solution to offer predictive project modeling through Monte Carlo Simulation. The solution leverages quantitative information and risk factors to help forecast project completion dates at various confidence levels so that project managers understand when all of the planned work is most likely to be completed. Going back to the example presented earlier. When asked to determine when Feature X is going to be available, you can use a Monte Carlo Simulation to have an informed conversation about various dates and the confidence level of delivering on those dates. And now you are collaborating with the business on a delivery date rather than negotiating. Even if you made customer commitments that Feature X will be included in a given release, you can use the Monte Carlo Simulation to determine, much earlier in the delivery cycle, the likelihood of the feature being delivered on time. By having this information earlier, you can make better business decisions, rather than waiting until the end of a delivery cycle. Start Accurately Accounting for Agile Risk In closing a word of caution: the Monte Carlo Simulation is not a panacea nor is it a crystal ball that lets you see into the future. Rather, the results of a Monte Carlo Simulation provide a more informed forecast that accounts for risk. And with the Monte Carlo Simulation capabilities in VersionOne, these simulations are faster and easier than you might imagine. In fact, it probably takes more time to deal with dissatisfied stakeholders after a missing a date because of a poor prediction. I hope this article has inspired you to more precisely predict what can be completed in a release and more accurately account for risk using a Monte Carlo Simulation. In addition to Monte Carlo Simulation, find out how VersionOne provides better insights for making better decisions.