Skip to main content
Which Continuous Integration Environment is best for you?

This post is from the Experitest blog and has not been updated since the original publish date.

Last Updated Apr 10, 2019 — Continuous Testing Expert

Which Continuous Integration Environment is best for you?

Continuous Testing

Nowadays, developing new software is usually a task for a team of professional developers. As a result, working as a team on one project usually involves coordination challenges. With one repository to store your code and several developers working with it, it's only natural that conflicts sometimes arise in the committed code. Each time the team is facing such conflict, it takes some time to resolve it. That puts emphasis on continuous testing which is the purpose of this article.

Additionally, the more infrequently merges to a repository branch occur, the more work is necessary to resolve conflicts. If repository commits happen relatively often, then it's easier to solve code conflicts when the trail is still hot, in order to avoid "integrational hell" later.

Continuous Testing resolves code conflicts

Continuous Integration (CI) is a software development practice of merging changes into a repository as often as possible to reduce the time required to resolve conflicts. Usually, CI also includes regular automatic builds and the running of automated tests. Certainly, continuous integration proved its usefulness and is now considered necessary to be included in any relatively complex project. Continuous Integration tools are software that helps manage integration jobs, build automatization, test and deploy.

From the quality assurance viewpoint, Continuous Integration is a great opportunity to introduce automated testing. Most importantly CI tools support automatic test execution for triggered builds. As for the Experitest SeeTest integration, it can be done seamlessly by using SeeTest RESTful API to mass deploy applications on target devices, and then use build triggers of your Continuous Integration environment to launch automated tests on your SeeTest devices.

There are a number of different Continuous Integration tools available. In this article we will cover the following:

  • TeamCity
  • Hudson
  • Jenkins
  • Bamboo
  • HP ALM

The following workflow can be used with any CI environment:

continuous integration flow


teamcity - continuous integration

TeamCity is a CI tool developed by JetBrains, famous for its IntelliJ IDEA development environment. The main advantages of TeamCity are:

  • Great integration with IntelliJ IDEA;
  • Very easy to configure builds out-of-box;
  • Integrated code coverage, inspections and duplicates search.

TeamCity is a classic example of a Continuous Integration environment. It supports different version control systems, such as Git, Mercurial, TFS and others. It supports running commits through automated tests in order to spot bugs early. Being, probably, the easiest CI tool to create your first build job, it also has some hidden layers, that are difficult to master without a proper DevOps background.

JetBrains offers TeamCity for a number of pay plans, varying from the somewhat limited free version to enterprise-level solutions.


hudson - continuous integration

Hudson is an open source continuous integration tool. It has multiple advantages listed below:

  • Free cross-platform tool, distributed under the MIT license.
  • Quick installation and simple settings make this tool attractive for development teams.
  • Hudson has plugins architecture, which extends the functionality of the tool. The start page lists the available plugins.
  • Hudson has integrations with bug tracking and project management systems such as Bugzilla and JIRA.
  • Numerous integrations with version control systems, such as subversion, SCM, Git, SVN, Mercurial and other.
  • Plugins for popular builders such as Maven, Ant, Gradle, MSBuild and other.
  • Integrates with testing frameworks, code coverage, and code analysis tools.
  • Plugins for virtual environments and for communication applications.

With these functions, Hudson can provide a full cycle for building, deploying, testing and reporting as a continuous integration tool.

For a long time, the Eclipse company developed and supported Hudson. But the development of the tool stopped in 2017 and currently the last version of Hudson is 3.3. In 2011 on the basis of Hudson another famous CI tool - Jenkins was created. Jenkins can be considered as the evolution of Hudson. Both tools look similar and have similar features and flow. So it's better to use Jenkins as a more advanced tool than Hudson.


jenkins - continuous integration

Jenkins is a direct successor to the Hudson continuous integration tool. It is an open-source continuous integration and continuous delivery tool, that runs in servlet containers such as Apache Tomcat. It has inherited most of the merits of Hudson, mainly - great customization by numerous plug-ins, allowing Jenkins to be compatible with almost any development environment.

Main advantages of Jenkins:

  • It's the most popular CI tool.
  • Open source (MIT-license).
  • An astonishing number of plug-ins - mailing tools, Javadoc publishing, and various third-party tools integrations.
  • Great community support.
  • Good support of the coded build pipelines.

However, some of the pros can also be cons. With a great number of third-party plug-ins comes great responsibility. Some of them may cause severe compatibility issues.

Also, customization and open-source nature of Jenkins may be a problem, if you need one particular solution for your needs right out-of-box.


bamboo - continuous integration

Bamboo is a continuous integration tool from Atlassian. The main feature of Bamboo is great integration with other Atlassian tools - JIRA, Bitbucket. It supports all classical continuous integration workflows:

  • Flexible build configuration.
  • Integration with popular services such as AWS CodeDeploy, Docker, and Amazon S3.
  • Dedicated agents for your builds.

The pricing for Bamboo varies from 10 USD for the basic "Small Teams" plan to thousands of USD for complex plans with an unlimited number of build jobs and access to multiple remote build agents.

Also, there are a lot of plug-ins for Bamboo available in the Atlassian marketplace, offering easy integration with most third-party tools and development environments.


hpalm - continuous integration

HP ALM (also known as Micro Focus Quality Center) is an Application Lifecycle Management program by HP, that is supposed to serve not only as CI environment but provide a whole platform of tools to manage the application lifecycle. It provides dedicated tools for requirements management, test planning, defects management, load testing and more.

Features of HP ALM:

  • Out-of-box compatibility with HP tools;
  • Enterprise-level solutions are available for almost every development/testing/management task.

Being a very powerful tool, HP ALM has its own limitation - one of the most severe of which is that HP ALM is only available for Windows operating systems, and only on Explorer 10 or 11. Also, it's worth noting it's a rather rigid UI with minimum customization.


Choosing a Continuous Integration environment for your project is a complex process, in which you have to consider your existing tech stack, the skills of your team and desired build tasks. When you make the right choice, it greatly improves the whole development and testing process. With Experitest SeeTest, invoking the RESTful API to mass deploy applications on target devices as a part of your build step, you can use Continuous Integration practices to test your application on your SeeTest devices. In order to do so, you should upload your application to the cloud, then add a step to your CI build, that makes a call to Experitest API to install the application to your cloud devices. For example, the following cURL request installs an application to all of your cloud devices:

curl --request POST {yourhost}/api/v1/applications/{applicationId}/install -d "AllDevices=true"

Then, if you wish, on a clean-up step of your build you can uninstall this application by adding a step, that invokes:

curl --request POST {yourhost}/api/v1/applications/{applicationId}/uninstall -d "AllDevices=yes"


Konstantin Tonkov - Performance Engineer ISSArt QA team


More from the Blog

View more
Jul 29, 2022

The automation testing advice that management most frequently ignores

Continuous Testing
No one likes to be told “I told you so” but sometimes the people doing ...
Read More
ios 16
Jun 07, 2022 Continuous Testing, first in market to support iOS 16 (Beta)

Continuous Testing is proud to announce that our Continuous Testing solution i ...
Read More
May 31, 2022

How automated/continuous testing solutions help create more secure mobile apps

Continuous Testing
Mobile app security risks climb as remote work increases Security o ...
Read More
May 13, 2022

What tips would you give testers on how to ask better questions?

Continuous Testing
The nature of testing is not about confirmation. It is about questioni ...
Read More
Contact Us