This post is from the XebiaLabs blog and has not been updated since the original publish date.
The Do’s and Don’ts of Cloud DevOps
If you’re in software development, or you’re in IT Operations and support software developers, you’re probably familiar with DevOps and cloud. The two are quickly becoming intertwined, leaving folks scrambling to figure out how to make them work together.
As organizations implement DevOps best practices and improve how they build software, the need for better and faster ways to deploy their applications has become essential, so they’re turning to the cloud to speed things up. It seems to makes sense. Cloud platforms like Amazon Web Services (AWS), Microsoft Azure, and Google Cloud Platform (GCP) hold a lot of promise to help organizations drive innovation and accelerate software development by continuously providing new technologies, as well as better ways to run applications and consume infrastructure.
But beware. When moving to the cloud, you can’t do DevOps the same way you’ve been doing it on-premises. There’s a whole new set of challenges to face, and while evolving your apps to the cloud might not go as fast as you like, the rewards of quicker delivery with lower risk and more control are worth the effort.
Challenges of Doing Cloud DevOps
Based on years of experience working with companies of all sizes on their cloud journeys, I’ve come across a set of common challenges that organizations run into when moving their development practices to the cloud.
Lack of cloud expertise. Even though cloud has been around for over 10 years, the amount of skilled resources is still very limited. Cloud is new for a lot of Operations and Development people, as they’re focused on their day-to-day jobs and lack the time to experiment and learn the ins and outs of cloud technologies. So how do they begin to make the transition? How do they navigate all the options, such as different cloud platforms, containers and serverless architectures, machine learning, IoT, and much more? Without cloud expertise, it can be daunting for IT teams to answer these questions.
Understanding the new cost model. Cost is more than a side effect and must be planned for upfront. Many cloud implementations have failed because cost was an afterthought, causing costs to sky rocket and cloud adoption to come to a standstill.
Scaling and onboarding. Implementing cloud on a small scale can often be accomplished using the cloud platform’s tools. However, scaling out a cloud implementation to different teams requires more standardization and automation than today’s cloud platforms offer.
Security and Compliance. Keeping your applications safe in the cloud is critical for a successful cloud implementation. Many traditional security practices, such as SAST and DAST, are still very relevant in the cloud. But as many organizations adopt open source tools and move to cloud native services, additional security practices must be added to the mix.
Bad Practices that Hamper Cloud Adoption
Let’s go over a few things you should NOT do when implementing cloud DevOps.
Make your developers create and maintain plumbing work to keep pipelines together. Expecting developers to write and manage endless ad-hoc scripts takes them away from doing what they’re good at — innovating. Cloud and microservices will only make pipelines more difficult to maintain. The time is right to look at tools built for automating and managing end-to-end delivery.
Think that your current tools will “just work” in the cloud. Not getting the people and tools you need to make DevOps work in the cloud will likely lead to failure. Leveraging vendor expertise where it’s available can expedite cloud adoption and minimize some of the technology hurdles.
Believe that cloud providers have all the services and tools you need. It’s not a good idea to lock yourself into a single cloud platform. Applications should be deployable on multiple clouds. A well-designed software development process is cloud agnostic.
Think that your cloud resources are endless.Governance is often overlooked on both the DevOps and cloud sides of the equation. This can result in blindness to how much cloud services are being used. Bad practices around cleaning up resources and rolling back after failures leaves resources abandoned inside your cloud, incurring costs month after month — a serious problem that has derailed many implementations.
Best Practices for Successful Cloud DevOps
Now on to the good stuff. Here are few practices I recommend for maximizing your chances of success in the cloud.
Learn and practice with new technologies. Succeeding at DevOps requires continuous learning and improvement. There are lots of cloud training classes and workshops available to help you in this regard. Also, some vendors provide pre-built templates that make it easy and quick to learn about and test different cloud technologies and incorporate best practices learned from working with numerous companies. Leveraging this knowledge removes common constraints that hinder cloud adoption and shortens the learning curve.
Scale with DevOps as Code. “Everything as Code” has quickly become a common part of the DevOps practice as companies try to automate as much as possible to speed up the delivery process. CI/CD, Infrastructure as Code, Configuration Management, and Container Management all leverage some type of code to manage the complexity of the function. The same should be applied to full software delivery pipelines. DevOps as Code “blueprints” allow you to create templates from proven YAML files built by your experts. The blueprints define and standardize best practices across your entire release pipelines and take input parameters that can be applied on a per-configuration basis (environment, app, or team). With DevOps as Code, you can build consistent pipelines across your organization and control quality as you scale.
Standardize security and governance practices. Many organizations struggle to move to the cloud because they don’t put controls in place to track access, usage, and cost. Fortunately, there are tools on the market that let you do all that. You can also implement automated cleanup and rollback procedures directly into the CI/CD pipeline to ensure resources are cleared as soon as their purpose is complete, preventing those resources from becoming abandoned. Security should also be automated and built into your pipelines — automation is key here. For example, SAST and SCA should be run on every check-in, DAST should be run on every build, and application and DB tests should be run on every deployment.
Get visibility across your entire software delivery process. DevOps is a shared responsibility across many roles in the organization (developers, IT operations, security, product managers, and executives). DevOps generates a lot of data that can be used to optimize the software delivery process. It’s imperative that everyone has access to this data to get visibility into what’s happening anywhere in the release pipeline. Chain-of-custody, for example, should provide full traceability and auditability for every product in the organization. And, at any point, anyone in the delivery cycle should be able to see where a feature is in the pipeline, who’s working on it, and what the next phases are. Predictive analytics based on machine learning models can also help DevOps teams increase release success rate, detect bottlenecks, and improve their ability to fix problems across a wide variety of tools. The more visibility teams have, the better their chance for success.
A Few Best Practices Go a Long Way Toward Cloud Success
To realize the full benefits of the cloud, it’s worth investing time upfront to avoid common challenges many organizations face. A few best practices: learning new technologies, incorporating DevOps as Code, standardizing security and governance, and giving visibility into the entire pipeline to everyone in the delivery process will all help make your cloud DevOps journey successful.