This post is from the Experitest blog and has not been updated since the original publish date.
Insights into The Forrester Wave™: Omnichannel Functional Test Automation Tools, Q3 2018 Report
The Forrester Wave™ reports are good for both business and customers. Since Experitest is featured among the best automated functional testing tools on the market we thought we would share some insights from the report.
Forrester has released its latest Forrester Wave™ report and Experitest is featured as a strong performer. To make matters even sweeter we were top rated in the category of market presence with the largest install base.
It is a terrific achievement to be considered among the world's best automated functional testing tools and one that we are proud of.
The Forrester Wave™: Omnichannel Functional Test Automation Tools report is much more than a popularity contest for test automation companies. There is a lot to unpack so let's take a closer look at some of the interesting results of Forrester's research.
First a bit about Forrester. Their first report came out in 1983 on the subject of "Professional Automation" Since then they have become a leading and respected member of the BI community.
In evaluating various vendors and service providers they provide not only valuable information for not only those looking to purchase a product or service but also to the vendors themselves. With a Forrester Wave™, report vendors can see how they are positioned in their respective market, and gain insights into the movement of their competitors.
The 2018 Forrester Wave™ OFTA report
Forrester analyst Diego Lo Giudice, a principal analyst in the Application Development & Delivery market, has spearheaded this report analyzing the 15 best omnichannel functional test automation (OFTA) providers. Each company in the report was subjected to a 26-criteria evaluation.
These criteria were grouped into several categories of analysis:
- Current Offering
- Market Presence
Experitest's SeeTest Continuous Testing Platform for Web & Mobile Apps was ranked as a "strong performer" among the other solutions tested. With an exciting endorsement like that, it seems like a slam dunk to think that such praise would instantly win any vendor some new customers.
The realities of the mobile test automation market are far beyond mere platitudes. It is true that people come to a Forrester Wave™ report for the rankings and company overviews, but they stay for the real market research.
What App Dev pros are looking for
Forrester reports that application development officials are looking for omnichannel functional test automation. In the era of digital transformation, QA professionals face an array of challenges that an OFTA provider can address. App developers increasingly operate in Agile and DevOps inspired environments and the way that applications are being tested has needed to adapt. OFTA providers offer services that help to extend testing to include the customer experience, design, performance and functional testing.
Digital Transformation and Automated testing
These two concepts truly work hand-in-hand, or to be more precise as an effect and cause. Digital Transformation is here to stay and it has caused a need for better and more comprehensive automated testing environment. Forrester reports that "Nearly three-quarters of global services decision makers tell us they are investigating or implementing a digital transformation".
Web and mobile apps must be delivered quickly and with flawless performance, and app testing is a central factor in making that a reality. When developing apps using Agile and DevOps methodology manual testing simply won't cut it. Testing cycles need to be shortened in order to speed up the feedback loop and bring new releases to market quickly.
In Forrester's Customer Experience Index (CX Index™) it was reported that improved CX leads to "increased revenue and even stock valuation". With the quality of a well-implemented Customer Experience bearing that much weight in a customer's mind, it is a necessity that functional testing is carried out on any web or mobile app. Forrester reports that in 2018 "38% of global developers told us that their development teams measure success and status by quality." Outside of the need for speed quality is key, and encompasses test coverage as well as bugs detected and fixed. The volume of testing that this requires simply cannot be performed manually.
Breathe the Pressure
The need to increase speed and quality has increased the pressure on testers who as of 2013 are increasingly finding themselves as members of product teams. Even with improvements on the development side, there are still some significant challenges.
Test automation has a long way to go. As reported by the Forrester Wave™ in 2017 only 44% of functional tests are automated.
The evolving automated functional testing tools market
All of this has led to a dramatic evolution in the Testing tool market. A market in which The Forrester Wave™ recognizes Experitest as a Strong Performer.
In last year's report, Forrester put emphasis on the trends like the meeting of functional and performance testing. Also discussed was the shift towards selective design automation. This year there are some new criteria used by The Forrester Wave. One of those deals with Enterprises that are beginning to think mobile first when designing and developing new applications. Experitest is strongly represented here due to our expansion from mobile-only, to an omnichannel platform.
With all of that being said Forrester looked at the following when doing their research:
- Omnichannel testing capabilities
- Design, UI, and API omnichannel functional test automation
- Customer references
- Presence in client inquiries and/or a tool that put it on Forrester's radar
So let's see how all that adds up to Forrester's evaluation of Experitest.
Most importantly Experitest is the best in market presence with the highest installed base.
It means that people in the know recognize and use our SeeTest Platform. It is a gratifying feeling.
Beyond that Forrester went on to say that: "Experitest has strong UI and scripting capabilities for web and mobile. Experitest is well known for its mobile testing platform. But it also offers a complete testing environment for both web and mobile, a robust mobile device cloud, and extensive cross-browser testing. Record and replay for both web and mobile are solid and give developers a strong code scripting option. Experitest's integrations for maintaining and reusing tests and leveraging real-time app monitoring are also excellent. It is one of the few players to offer a unified platform for both mobile and web on its cloud and is introducing a differentiated visual testing tool for cross-browser and mobile testing that manages visual layouts (besides functional test) in an automated and precise way. Experitest shows strong market growth, and all reference clients claimed to reach more than 60% automation through Experitest in the past 12 months."
It is an exciting time for Experitest. We have opened new data centers with real mobile devices for better global testing access. We have also initiated several partnerships that will expand our user base.