How Unified Software Development and Delivery Makes the Vision of DevOps a Reality

 

Are you finding that DevOps is more vision than reality? Here’s how you can unify the systems that DevOps workflows depend upon to help make your DevOps vision a reality. DevOps Can Be More Vision Than Reality The DevOps movement has provided organizations building software with a vision of increased deployment frequency, product quality and mean time to recovery gained from improved collaboration and automation. While propagating that vision has been a success, executing against it often remains challenging – especially in the enterprise. Ultimately the DevOps movement seeks to tightly unify Dev and Ops workflows, but so far two systemic barriers have kept these functions from becoming truly unified. 2 Barriers to Unifying Dev and Ops I believe successfully unifying plan, develop, validate, deploy and run workflows is still challenging for two fundamental reasons:

1. Plan and develop work items (features, fixes, stories, etc.) are not directly linked to operational outputs (builds, artifacts, environments, etc.)
2. Lots of fragmented automation make it difficult to orchestrate and creates many pockets of siloed data.

1. Development Workitems Are Not Directly Linked to Operational Outputs In any software delivery process, there is an inherent disconnect between development workitems and delivery outputs. The image above highlights a common pattern that organizations adopting DevOps face regardless of their level of DevOps maturity. This platform disconnect between functional workitems and delivery outputs makes it very difficult to truly unify development and operations. Starting with the green box on the left, you have a simple representation of the agile development process. The main units of flow moving through the development organization’s storyboards have traditionally been workitems such as features, fixes, stories, epics, etc… However, once these development initiatives get converted into builds or artifacts and deployed into environments, the linkage gets muddy. At that point, “release” or “deployment” units of flow are only loosely affiliated with their corresponding workitems back in the agile storyboard on the left. Feature attributes such as cycle time and current status can be tracked accurately while moving within context of the development storyboard, but manual updates to that data are required during downstream delivery. This creates a very weak understanding of the real-time flow of value once you get beyond the planning tool and into the downstream and more “operational” software delivery process. According to a recent DevOps Survey conducted by VersionOne, more than 87 percent of respondents indicated that multiple systems are required to manually cross-reference features and fixes with their corresponding builds, artifacts and environments. This problem then gets magnified as functional changes “queue up” in later stage environments between release events. This lack of automated manifest reporting makes it increasingly difficult to express with certainty which workitems are included within specific artifacts and deployed into specific environments at any given point in time. Here are a few questions that are typically difficult to answer with absolute certainty: It will continue to be difficult for all stakeholders across the end-to-end delivery pipeline to collaborate at the highest level if Dev and Ops platforms are not truly unified. Building DevOps maturity mandates a tight linkage between functional workitems and corresponding delivery outputs to streamline the flow of value and simplify cross functional collaboration. 2. Automation Processes and Tools Are Fragmented A clear and positive outcome of the DevOps movement is the emergence of a plethora of point process automation tools. These tools have been important enablers of DevOps practices and have dramatically reduced the amount of time required to validate, deliver and manage new software. However, the primary data models of these DevOps automation tools are wholly unaware of concepts such as features and fixes. Since these workitems represent the actual “content” flowing thru automation, visibility and traceability at the feature/fix level is critical to driving efficiency in a DevOps setting. The image above depicts the fragmented delivery environment that frustrates our ability to link delivery outputs with functional workitems. This graphic was shared with me recently by an organization trying to enhance their ability to track the flow of value, in real-time, thru their delivery pipelines. If DevOps is a priority at your organization, this example is probably similar to what you have now or what you will have in the not too distant future. As this very busy diagram indicates, the DevOps automation tools we depend upon to move value from the initial commit all the way out to production are continuously generating important audit, test and deployment data at every stop across the delivery pipelines. However, this data is often under-leveraged and buried deep inside tools completely unaware of the features and fixes flowing thru them. Because of this fragmentation and lack of context, it is very difficult to provide critical status and audit data back to DevOps stakeholders. Without a unified development and delivery platform, correlating data generated through delivery pipelines back to specific features and fixes will continue to be a largely manual, error prone and time-consuming process. 4 Costs of Dev and Ops Not Sharing a Unified Platform The costs of development and delivery not being unified is a missed opportunity. While small and incremental gains toward end-to-end unification have yielded progress, the reality is that most enterprise software development organizations are still struggling to improve: 1. Value Stream Efficiency Because of the units of flow problem, stakeholders don’t have automated visibility into the status or and/or deployed location of the features and fixes flowing thru a delivery pipeline. As a result, manual effort is required to perform continuous business-to-operational cross-reference reporting and analysis that introduces material and unnecessary overhead into the software delivery value stream. 2. Opportunities for Continuous Improvement The plethora of fragmented point automation generates siloed data that is difficult to access and correlate back to a discrete set of features and fixes without significant human intervention. This fragmentation makes it difficult to collect meaningful statistics that can identify bottlenecks across the entire software delivery chain. This data is the crucial fuel required to drive the kind of continuous process improvements needed to materially increase delivery frequency and shorten time to market. 3. Software Quality & Failure Rate of New Releases The lack of end-to-end visibility into the entire value stream makes it difficult to know with absolute precision which functional changes have been included in any given build or artifact. This reconciliation process is almost always manual and is susceptible to errors that increase the odds of deploying unstable or incomplete “work-in-progress” into critical environments. 4. Mean Time to Recovery & Slower Analysis The lack of detailed end-to-end delivery accounting and audit history, at the business level, frustrates the ability to find root cause and issue repairs for issues and defects once uncovered. Additionally, this un-correlated data makes it difficult to perform the detailed analysis needed to identify system or process failures that caused the introduction of critical production defects in the first place. What Is a Unified Software Delivery Platform? In order to make the vision of DevOps a reality, a truly unified platform that supports the end-to-end delivery stream – from idea to production is a primary requirement.  A crucial capability to achieve platform unification is the ability to link together all of the data generated throughout the delivery process. If data can be gathered and correlated at the time of creation, a comprehensive dashboard can be created that supports real-time collaboration across stakeholders. Most organizations that have multiple agile teams are already using some sort of agile lifecycle management platform to manage priorities and coordinate development activities. By reimagining our storyboards as development, validation, and deployment orchestration hubs, we can unify the planning and development platforms with the infrastructure required to support downstream automation – without ripping out or replacing any of the tools and technology you’ve already implemented. By leveraging centralized pipeline orchestration, you can better track work items as they move from one stage to the next in your storyboard. Because the orchestration layer understands automation in context of the features and fixes flowing thru it, stories can now be directly associated with the artifacts, builds, config files or deployments, linking these two traditionally decoupled platforms. When your storyboard is linked with all the DevOps automation tools that move changes from the first commit all the way out to production you can begin to capture and associate the important audit, test and deployment data generated at each and every point within your delivery pipelines.This is the type of unified software delivery platform that can help make the vision of DevOps a reality. Here are a few characteristics of a Unified Software Development and Delivery Platform:

• Unified DevOps repository that can support robust cross-referencing between business value (features/fixes) and operational objects (builds, artifacts, deployments).
• Ability to visualize, measure and optimize the journey of features and fixes from idea all the way to production deployment.
• Robust pipeline orchestration that leverages existing DevOps automation and eliminates or minimizes the need for manual handoffs.

The 5 Benefits of Unified Software Development and Delivery 1. Increased Collaboration Across All Disciplines Product Owners, Project Managers, Developers, Testers, and Ops team members can more easily collaborate because business work items are linked to delivery outputs providing visibility, traceability and clarity across the entire value stream. 2. Increased Automation and Streamlined Value Streams The plethora of fragmented point DevOps automation tools are now orchestrated by the unified DevOps orchestration engine, reducing the need for human intervention. 3. Increased Deployment Frequency & Shorter Time to Market With clear visibility of the entire value stream it is much easier to make the continuous process improvements that can increase delivery frequency and time to market. 4. Improved Software Quality & Reduce Failure Rate for New Releases The ability to automatically cross-reference any build or binary to the features and fixes included within – with absolute precision – greatly reduces the chances of testing in the wrong environment, accidently promoting works in progress or items with unmet dependencies. This capability results in higher release quality with less wasted manual effort. 5. Shorter Mean Time to Recovery & Faster Analysis Unified audit and traceability throughout the entire software delivery process – from idea to production – will make it much easier to uncover issues prior to deployment. When defects do reach end-users, post-mortem root cause analysis can occur in minutes instead of weeks, uncovering root cause and prevent issues from recurring. Conclusion The independent evolution of planning platforms, build automation, testing and release management tools has created a profound and systematic data division between Dev planning platforms and Ops automation. As long as these disconnects persist, achieving the key DevOps ideal of cross functional collaboration and streamlined process flow is challenged. Unified Software Development and Delivery is the process of merging these two universes to provide a comprehensive and end-to-end value stream that documents the flow of business value from idea to production.