This post is from the XebiaLabs blog and has not been updated since the original publish date.
Test Management = Scalability
As many companies make the transition to automated testing, the way they manage the growth of testing data needs to scale as well. Spreadsheets and inbox analytic from testing tools aren’t enough to report trends or support quick test reruns anymore, enterprises need more and they need it as quickly as possible. The ability to scale your testing automation within an agile environment will allow your enterprise further growth without every change costing you a disproportionate amount of time. This is how bol.com, a major online retailer in the Netherlands and Belgium, did it.
With over 9 million products and more than 1 million visitors per day, bol.com is a large company that needs the ability to scale quickly and effectively. The functionality of the website is built by a 35 person development team, and in order to safeguard the quality of the site, they build out more than 6,000 automated functional tests in FitNesse to continuously verify correct software operations. On the performance testing side, bol.com uses JMeter. The bol.com Test Automation (TA) Team manages the quality and functionality of the automated tests, as well as the testing infrastructure, the tests themselves and the reporting. When tests are failing or the test duration increases, the team fulfills a role that signals the responsible development teams. To facilitate this process, the TA team developed an in-house solution that acted as a short-term fix to a long term problem. The poor scalability and manageability of their in-house solution ended up costing them an immense amount of time and money for every change and expansion they needed. With increasing growth and an unscalable solution, bol.com needed a test management system that could quickly integrate with their existing tools and allow them to leverage new tools in the future. Bol.com adopted XL TestView to provide real-time quality awareness, visualization and test management by handling complex test management functions
- Organizes the existing large test suite into multiple parts that are run heavily in parallel using multiple test slaves.
- Identifies the tests that have an above-average duration and need speed-up actions.
- Provides quick support for re-execution of tests, reporting on a per-team level, and links to other relevant information about the test specification runs.
- Offers extensive reporting and dash-boarding capabilities with trends across teams, applications, and components.