What We Learned from Fully Automating our Own Release Process with XL Release
At XebiaLabs we build products to help companies visualize, automate and control the process of releasing their software. Of course, we have our own release process too – and like pretty much everyone’s release process, there is room for improvement. Naturally, we turned to our own tools to make that happen. In this post I am going to explain how XL Release has helped us to visualize the release process of XL Deploy, our product for deployment automation, and how we optimized the release process by automating it.
Step 1: Model the process with manual tasksOur customers use XL Deploy on premise. Releasing XL Deploy for us means compiling and testing all components and plugins, assembling the documentation and other resources, putting everything in a distribution and finally making everything available online for download, including the announcement. When we first started to use XL Release, we modeled our release flow with only manual tasks. That's not to say nothing was automated: we had a lot of individual scripts, but there was no flow connecting everything together. Having only manual tasks in XL Release felt a bit disappointing back then, but right now I think that was perfectly okay – maybe even desirable – as a start. How can you automate something without understanding the bigger picture? A release plan with only manual tasks is very flexible. We reordered and changed quite a lot of tasks since then, and learned quite a lot. I think the most important things we learned are:
- What is the exact scope of the release? Including what actions belong to which people? How do we break up larger tasks into more concrete actions?
- What is the order of events? Which tasks depend on each other? Which tasks can be done in parallel? Which phases do we have?
- Which systems do we need to interact with? How do we integrate with those systems?
Step 2: Start automating and wire it togetherOur goal is to have a release flow that takes our source code and turns it into a downloadable distribution on the website, with the click of a button. The release process is actually quite technical by nature, so we started by automating the release process of individual components. We ended up with many small automated subtasks. From there on we started wiring everything together. It is important to understand that we manage our release process at different levels. Let’s look at them from the bottom up:
- Gradle is our build tool and focuses just on releasing one specific component. It fetches dependencies, compiles the source code and runs tests, but Gradle is also involved in the release itself. We created Gradle plugins for our low level release tasks like bumping version numbers and creating tags in Git.
- Jenkins is our middle layer and is basically where all the hard work happens during the release, like executing the actual build process and delegating the work to build slaves. For the release process, there is not a lot of control or logic in this layer. We like to keep this layer as thin as possible.
- XL Release sits on top to orchestrate the flow of the release and delegates work to Jenkins and Gradle. It also interacts with other systems involved in the release process like JIRA and our download server. XL Release is in full control of the release process.
Step 3. Where are we nowCurrently most of our high-level release tasks are automated tasks in XL Release. After a couple of releases and many improvements, this is what our release template currently looks like:To explain in a bit more detail, we distinguish the following phases in our release:
- Release dependencies: First we make sure to have all dependencies of the core product available.
- Release preparation: In this phase, we do a final check and we put everything in order. This is a very important phase because we try to gather all human input, allowing the rest of the release to proceed in an unattended way.
- The core release: If all the builds are green, we build the source code and create the distribution.
- Smoke test: We run a final smoke test that integrates all released artifacts together and tests them one last time. A final quality check before we go live.
- Go live: After this phase there is no going back. When the product owner accepts the release, everything is published on the website.
- Post release: A list of tasks that need to be done to wrap up the release and to prepare development for next release.
- Triggering builds in Jenkins: To trigger the release build in Jenkins.
- Running remote shell scripts: Triggers a remote shell script to start syncing the distribution from nexus to our download server.
- Verifying distributions are downloadable: A custom tasks that checks if the HTTP status code of a specific URL is ok.
- Marking the released version as released in JIRA with current date: Yet another custom task that does rest calls to the JIRA API.