Skip to main content
How to Improve CX by using Smart Continuous Testing

This post is from the Experitest blog and has not been updated since the original publish date.

Last Updated Dec 05, 2018 — Continuous Testing Expert

How to Improve CX by using Smart Continuous Testing

Continuous Testing

Continuous testing and digital testing have been gaining attention as of late. It is an area of business that is growing exponentially. With steady growth have come great innovations that have both disrupted and improved the market. Using artificial intelligence in the mobile test development and execution process is one such disruption.


Continuous and smart testing trends


Digital and the customer experience are some of the buzzwords and buzz-concepts we see written about. The world seems captivated by customer experience. So let's dive in and see why.


Digital Leaders are obsessed with CX


Research firm Forrester has a customer experience index that they update every year. The index is comprised of a number of unique drivers and measurements, like user experience, and quality. What we can learn from the index is that an improvement of as little as 1% will increase company revenue.


The index is divided by industry so you can see the impact a CX improvement will have by each industry.

Experitest - CX industry - continuous testing

For example, let's take a look at the auto manufacturing industry. An improvement of 1% in CXC equals an increase of 48 dollars per customer. When we multiply that over the average number of customers per company we are talking about a potentially huge increase in company revenue. The point is that positive customer experiences add up to growth in existing revenue as well as a growth in new customers.


Tied to customer experience is the user experience which can be defined as what clients see. We are all tied to UX in the way we interact with the apps on a daily basis. What we see makes us engage with what we are using. If we enjoy what we see we will keep using those applications.


The world is changing. With the advent and growth of AI, the user experience will be based on what we see, what we hear, and what we say. We can see this already taking shape in systems like Siri and Alexa that we can talk and listen to. Many large enterprises are also implementing chatbots that can reply, and graphical user interfaces that have voice capabilities. There are still some kinks to work out, but as the technology gets better, usually with rigorous test development and execution, the use of such systems will increase as well.


Customer Obsessed - DNA


Enterprises that are focused on giving a constantly improving customer experience are referred to as customer obsessed. The DNA of such companies is different from a more traditional company. A customer obsessed company plans new products and applications. These plans are based not on what the customer needs but on what the customer wants or even will want. It is business development that is driven by insights rather than data.


We live in a data and the analytics-driven world. Companies that have a ton of data cannot always make the correct insight-driven decisions. Customer-obsessed companies have three desires:


  • Use customer feedback to build a flawless customer experience.
  • Deliver new apps and updates at speed.
  • Shift from a siloed to a more connected mentality as technical leads collaborate with marketing and sales counterparts.


Interestingly, the area in which enterprises say that they struggle the most is speed. It is true that speed matters but we also know that speed needs to come with quality as well.


Continuous Testing at the heart of the revolution


We are discussing the revolution that is currently taking place around digital customer experiences and it centers around testing. Enterprises know that they need to deliver great CX with great quality and at great speed. One of the most important ways that they make this happen is via continuous testing.


One sort of side note about continuous testing. In a world that expects to continuous delivery, how could you ever hope to achieve that without testing continuously?


What we mean by continuous testing is testing constantly in an integrated fashion between development and delivery. That is why we say that testing is at the heart of this revolution. This should bring joy to testers whose jobs have become more strategic than they used to be. Once mobile app testing was an afterthought and testers were isolating themselves out of business. The role of testing has expanded in importance especially when employing shift left methodologies. Testing needs to be involved in each step from planning and ideation through deployment and even the feedback stage.


Diving a bit deeper into what continuous testing really is, we can say that basically, continuous testing is the enabler of the quality at speed approach. Similar to the way Motorsports includes pit stops in order to change a race car's tires or add fuel during a race, the point is to make sure that the car is in the best position to win. So too with testing. Pit stops, like testing, is part of the process of making either a successful race strategy or a successful application release.


This means that continuous testing and all testing activities must run constantly in an integrated fashion and not as an isolated afterthought. Testers' involvement stars early in the process. Product and business owners decide the testing, security and automation strategy that they will need for their specific product and development. This way code is verified on the go and the steps of maintenance and development all happen at the same time.


Quick debugging as part of the Continuous Testing process


As part of this process, bugs need to be fixed right away. Simply identifying bugs is only part of the process. Once identified we must figure out what the problem is if it is a problem and who is responsible for fixing it. We also need to make sure to prioritize any identified bugs in the developers' workload. When we say prioritized we mean in terms of the severity of the bug and not any other factor. In today's high-performance testing teams, the concept is that we build and fix bugs as we go. This changes the job of testing in that while we still want to favor speed, test environments need to be provisioned instantly. It's about running unit functional and non-functional tests with as much automation as possible in the earliest stages of development.


When you are in a continuous testing loop you are using a CICD environment that orchestrates all of the steps. So your unit, functional, non-functional, performance and security tests all get kicked off as part of the process. If there are bugs then the tests revert back so that there is still a quality gate working. As we execute continuous testing, we must eliminate manual testing as much as possible. For example, we need to find ways to automate user acceptance testing (UAT), as well as customer experience testing.


Enter AI


Web and mobile app testing are going through a revolution. Today the concept is to try to imitate human behavior in a way that the testing reflects the way a human would interact with a web or mobile application.


This is where AI also comes in. The role of AI in testing is not exactly to create more automation but to augment the testing personas. Business people need a smart tool that helps then overcome any technical knowledge they are missing, by having information placed in more of a business view. Also augmented would be technical testers. They usually sit somewhere between business and development and will benefit from better understanding the test data in order to make better decisions. The third benefit of AI goes to the developers by being able to generate more unit testing.


In the image below you can see three ways where the industry is trying to enable this type of testing.

Experitest - industry enables type testing - continuous testing

  • Smart Testing - Using AI to interact with a UI as a human brain would. This is a trend that is increasing rapidly in popularity.
  • Visual Testing - The thought here is not only to test that something works but to test that it looks right when it displays on the screen. Technologies already exist to imitate the logic of the human eye, in terms of layout testing.
  • Intuitive Testing - The idea here is to empower a person who is not a coder, and give them the ability to create automated tests. Intuitive Testing is based on the idea of codeless automation via a simplified environment.


How AI is changing the continuous testing game


We can illustrate this with a cool example. The main way that AI is making headway is via something called deep neural networks. Simply put a deep neural network uses sophisticated mathematical modeling to process data in complex ways. This poses the largest difference in AI technology from now compared to 30 years ago. One of the main reasons for this is that we have more data than ever available to us. The second is because of the improvements in algorithms. Third, is because of digital transformation.


One of the areas in which deep neural networks are the most successful is in the visual space. Deep neural networks have gotten much better at recognizing images and videos to the point where they are able to come up with their own new algorithms. Generative adversarial networks are algorithms that train other networks. They do this by generating images that are difficult for the deep neural networks to distinguish between what is real and what is fake.


Take a look at the image below. This clever mashup of muffins and chihuahuas was created by a generative adversarial network to confuse the real neural network.

Experitest - big challenge - continuous testing

It looks funny and yes, in this case, it is. The point is that through the test of identifying which is the dog and which is the confection the deep neural network is being trained to improve its image recognition algorithms.


AI and visual testing


We now have the ability to leverage machine learning and AI capabilities within a visual testing tool. What that tool can do is to basically interact with a web or mobile application in the same way a human can. Visual testing tools can recognize images, videos, and text, and it can also capture slight differences that a human eye might not be able to. Using those capabilities an AI can support the human side of application testing by helping to decide what is right and what is wrong in User Interfaces.


Visual testing also can take the place of Manual testing in the sense that the AI can interact with an application in the same way a human does only faster and with more accuracy. This can be expanded to be included in a continuous delivery project and also can work across thousands of different devices and browsers. Think about the impact this will have on testing when you can deploy an AI to test an application on any combination of device, operating system, or version. With the smartphone market more fragmented than ever this will have a huge impact on testing at large.


Achieving this kind of scale is really difficult which is why we at Experitest offer a cloud-based testing environment. It makes it so much simpler to test on different devices in the cloud.


Visual Testing emerging, augmenting and automating


AI fuels visual testing to augment testers and automate more. This will speed up many activities during the software development and delivery process in a nice and consistent way. In terms of speed visual testing can change how you identify changes to your UI, during the development process. Debugging will pick up in speed by identifying errors more quickly and even fixed automatically. The main point is to use the visual testing method to keep the consistency between the UI and the CX.


We are starting to enter the realm of Science fiction with some interesting experiments that take visual testing into the realm of Science Fiction. Some companies are deploying brainwave scanning equipment to test users' emotions as they operate an app. This gives testers a real insight into the reactions of a person's brain and can be used to pinpoint pain points and better understand the experience that a user is having from the interface they are interacting with.


One of these systems could watch me interact with an app and tell me that while the UI is totally functional and has no bugs, perhaps I was exhibiting signs of frustration while interacting with certain parts of it. These are only some of the new areas in which we can increase automation compared to what we have been doing until now. These types of tests augment our testers compared to past testing capabilities.


AI is still in its early days


AI is still in its infancy. For the past few years, we have seen a lot of activity among system integrators and consultants in the testing arena. They are leveraging machine learning and predictive algorithms in testing.


Visual testing, however, is already here and with SeeTest Visual Testing you can integrate visual testing into your CI/CD flow for responsive web testing that will detect and fix design issues before production.


We still need to separate the myth from reality. Using natural language processing might seem like a farfetched idea. It is certainly possible to be used to create some simple and very specific tests. If you would try to carry out something like that on a more generalized level it probably would not work. This is simply due to the sheer amount of data you would need from the get-go. That said there are some startups that are using NLP to generate unit testing and are making excellent headway into it.


It is essential that we distinguish the fact from the fiction and do not take everything for granted. We are on the right path and moving this type of AI continuous testing forward. It is starting to disrupt the market but the major breakthroughs are still ahead of us.


More from the Blog

View more
Jul 19, 2021

Manual Testing is not dead, All Hail Manual Testing

Continuous Testing
In 1897 legendary American Satirist Mark Twain was reported to have di ...
Read More
Jul 08, 2021

Good practice demands not just testing automation, but testing of automation itself

Continuous Testing
Digital transformation and continuous testing continue to accelerate, ...
Read More
Jul 05, 2021

When automation works: metrics that measure success

Continuous Testing
How do organizations know when automation in their software delivery p ...
Read More
Jun 17, 2021

Smash the mobile app release barriers hindering your release velocity

Continuous Testing
Your new app version is ready for release. It has been dreamed up, dev ...
Read More
Contact Us