Skip to main content

This post is from the XebiaLabs blog and has not been updated since the original publish date.

Last Updated Jun 30, 2015 — DevOps Expert

Test Management = Scalability


As many companies make the transition to automated testing, the way they manage the growth of testing data needs to scale as well. Spreadsheets and inbox analytic from testing tools aren’t enough to report trends or support quick test reruns anymore, enterprises need more and they need it as quickly as possible. The ability to scale your testing automation within an agile environment will allow your enterprise further growth without every change costing you a disproportionate amount of time. This is how, a major online retailer in the Netherlands and Belgium, did it.

bol.comWith over 9 million products and more than 1 million visitors per day, is a large company that needs the ability to scale quickly and effectively. The functionality of the website is built by a 35 person development team, and in order to safeguard the quality of the site, they build out more than 6,000 automated functional tests in FitNesse to continuously verify correct software operations. On the performance testing side, uses JMeter. The Test Automation (TA) Team manages the quality and functionality of the automated tests, as well as the testing infrastructure, the tests themselves and the reporting. When tests are failing or the test duration increases, the team fulfills a role that signals the responsible development teams. To facilitate this process, the TA team developed an in-house solution that acted as a short-term fix to a long term problem. The poor scalability and manageability of their in-house solution ended up costing them an immense amount of time and money for every change and expansion they needed. With increasing growth and an unscalable solution, needed a test management system that could quickly integrate with their existing tools and allow them to leverage new tools in the future. adopted XL TestView to provide real-time quality awareness, visualization and test management by handling complex test management functions  
  • Organizes the existing large test suite into multiple parts that are run heavily in parallel using multiple test slaves.
  • Identifies the tests that have an above-average duration and need speed-up actions.
  • Provides quick support for re-execution of tests, reporting on a per-team level, and links to other relevant information about the test specification runs.
  • Offers extensive reporting and dash-boarding capabilities with trends across teams, applications, and components.
product-xl-test  XL TestView is the first test management and analysis tool that enables you to manage complex test setup scenarios by defining, configuring, and executing tests across a full spectrum of test tools. It offers an aggregated overview across all test results by seamlessly integrating with your already existing test tools. IT Manager at, Ralph Hopman says “In a smooth and painless process, XebiaLabs replaced our legacy code with a modern solution to run, monitor and analyze our tests. XL TestView fits perfectly in our existing technology stack and allows us to leverage new tools and technologies without having to worry about test management.”’s open source technology stack integrates easily with XL TestView, providing assurance that future technical or usage needs will be supported without any problems – either out of the box or with minimum effort through the tool’s extensibility support. Finally Hopman added, “The current growth figures of require flexible support in test automation execution, management and analytics. XL TestView provides just that and paves the way to align continuous delivery with further growth.”

More from the Blog

View more
Feb 22, 2021

Reckoning DevOps’ role in the enterprise value stream

If you’re a software or digital solutions company, you may use DevOps ...
Read More
Feb 10, 2021

Customer spotlight: Schneider avoiding bumps in the road with DevOps adoption

Everyone wants to deliver software faster and more reliably. Companies ...
Read More
Jan 06, 2021

How testing automation can build a culture of QA while accelerating continuous delivery

An organization’s level of automated test coverage is quickly emerging ...
Read More
Jul 30, 2020

Part 2: Is Technology Slowing Down Your Digital Transformation?

In part one of this post, we shared insights from Andreas Prins’ webin ...
Read More
Contact Us