Skip to main content
DevOps icon showing cogs

This post is from the XebiaLabs blog and has not been updated since the original publish date.

Last Updated Feb 16, 2015 — DevOps Expert

11 Things to Consider When Evaluating Application Deployment (ARA) Tools


Based on our experience we have found there are several land mines that you need to be aware of when evaluating your Application Deployment (or Application Release Automation/ARA) tools. As a Senior Principal Architect of a large midwestern bank, I had the opportunity to evaluate XL Deploy and other tools. Some of the key differences and important design considerations that we found were as follows:

  1. Are you planning on implementing the DevOps best practice of treating Configuration as Code? If yes, then do you prefer the approach of copying configurations from one environment to another or would you prefer to define the configuration required independent of the target?XL Deploy helps you define your deployable CIs independent of the target when possible. In this way, you can always discover, compare, report and apply versioned configurations across your deployment pipeline.
  2. Will the tool's reporting support continuous improvement?You should evaluate the reporting capabilities carefully. Having useful reporting is critical to any continuous improvement process.
  3. What is the future of the tool?Will you need to rebuild your entire CD environment in 6 months?
  4. How hard will it be to manage workflows?Other tools use workflows. We know that workflows are problematic and an approach that is not scalable. XebiaLabs has a white paper explaining why Workflows are not a suitable solution. XL Deploy uses a model-based approach to define your application deployment process. Since XL Deploy uses a model it can more efficiently deploy only the application components that need to be changed.
  5. Will OS level patching cause agent problems?Since XL Deploy does not have agents, there is nothing to patch on your target servers. It is less likely that an OS patch on your target servers will break you CD process.
  6. Do you want an agent on your target servers?XL Deploy is agentless, we use the same methods you currently use to log on to your servers to deploy your configuration items. We don’t require additional software. Since XL Deploy uses the same methods that you would use to install your CIs manually, you don’t need to install any additional code. XebiaLabs also has a white paper describing why agents are not an ideal approach.
  7. How much Network Traffic will be driven by the agent?Agents can consume a lot of network traffic calling home to the master server.
  8. How well will the tool integrate with the rest of your environment?Large software organizations try to lock you into their solutions. Therefore, you may experience problems trying to integrate with other tools. If you are looking for “best of breed” solutions some tools are going to have problems keeping up with your integration points.
  9. Are you comfortable with tool “magic”?With XL Deploy, you can see our deployment scripts that will be used during a deployment. If you don’t like our approach you can override our behavior. Transparency is a better approach (see my blog post "Transparency In Your App Deployment Tools Is A Good Thing"). XebiaLabs is much more agile than large organizations and can quickly extend our product to support your needs. Furthermore, there is an active community and plenty of examples to guide users on how to customize XL Deploy.
  10. Is the level of complexity of the tool worth the effort?Because some vendors try to wedge their tool into a broader suite of tools, the level of complexity will go up. We believe the better approach is to choose tools with open APIs that solve delivery challenges across all of your tools.
  11. Do you want to use an app deployment tool that has weak support for state of the art CI tools like Jenkins, Bamboo or TFS?Integrating Jenkins with some tools is awkward and involves using workflows that inject an unbelievable amount of complexity into a task. Integrating with XL Deploy is simple.

The IT Manager's Guide to Continuous Delivery

Continuous Delivery allows you to get new features and capabilities to market faster and more reliably. This ebook helps managers understand the principles behind Continuous Delivery, explains the transition to a Continuous Delivery organization, and gives practical advice on how to start benefiting from the dramatic improvements Continuous Delivery provides.

More from the Blog

View more
Ascension Launch Banner
Apr 26, 2022

Get ready for peak performance with’s newest AI-Powered DevOps Platform Ascension Release

Today, is excited to announce our latest AI-Powered DevOps ...
Read More
Jan 24, 2022 Value Stream Delivery for SAFe®: The key to amazing business outcomes

The Scaled Agile Framework (SAFe) is the world’s leading framework for ...
Read More
Dec 09, 2021

How SaaS and cloud-based solutions helped the U.S. Department of Veterans Affairs achieve digital transformation

Modernizing legacy systems was an ongoing goal for the U.S. Department ...
Read More
Nov 29, 2021

Increase velocity and reduce risk with AI and machine learning

Artificial Intelligence (AI) and machine learning (ML) have proven use ...
Read More
Contact Us