Using Open Source to Scale Enterprise DevOps

I’m a huge fan of open source DevOps tools. I’m also a huge fan of scaling open source tools for the enterprise. But having talked with my fair share of companies over the years, one important thing I’ve learned is that you can’t scale your release process using open source tools alone. They simply require too much scripting and maintenance when used that way.

Scripting may be fine for smaller organizations, but it’s not ok in an enterprise environment that includes many independent teams and tons of applications. You’ll end up with an unmanageable spaghetti factory. And who maintains those scripts? Your developers, some of the most talented and expensive folks in your organization. This is bad for the company, of course, but also for developers, who are much happier building code than doing maintenance work. Standardization is the answer and that’s where commercial tools come in.

The Benefits of Standardization

The DevOps Handbook by Gene Kim and team states that, to scale DevOps, you need to standardize. To do that, you need a repeatable process. Leverage your open source tools, but combine them with tools that allow you to model the release process. When changes are needed, just update the model, which automatically propagates changes across all releases. Keep your open source tools and gain an efficient way to scale. This is not just me saying this. We recently worked with a very large company that was using 300 jobs inside Jenkins to implement a Continuous Delivery pipeline. And the thing was, they weren’t just using Jenkins, they were also using AWS and Puppet. They were trying to orchestrate all this using Jenkins, and it was a massive nest of scripts that was costing them a ton of developer time. (See my last blog post to learn just how expensive using open source tools alone can get.) By standardizing their release process, they were able to keep all their open source tools, while reducing their 300 jobs down to 8 with 4 pipelines. Today, they can scale out to 150 microservices. Their ROI is huge because they no longer need to maintain all those scripts. Plus, they get visibility into the end-to-end process, even with all their different tools in play. So, to reiterate, it’s not about getting rid of open source. It’s about leveraging those tools for what they’re good at and using commercial products to scale, while improving efficiency and ROI. It’s a win-win for your IT staff, customers, and your company’s bottom line.