We Give this Advice to Teams trying to Achieve Test Automation Success

In the early days of NASA astronauts like Gus Grissom, and Gordon Cooper felt that their considerable piloting talent was going to waste. The amount of automation involved in the NASA space shuttle program, had them feeling like little more than “spam in a can”. Their sarcastic comparisons to trained monkeys were not unfounded as they entered the training area one day to see a literal monkey training on a simulator.

Similarly today in the world of test automation success. The feelings of “insert chimp here” that many QA and testers feel comes from certain CIOs and managers who share an unspoken belief that talent and experience are superfluous when it comes to test automation success. They expect the developer’s code to have full coverage, in an automated system that will also find 100% of the bugs.

Where they make a key error in judgment is in neglecting to consider that test automation helps amplify testing which in turn helps testers work smarter, and takes the developer’s code where no one has gone before.

So what do they want?

Business leaders want test automation success, and they want it quickly. The thing is that absent of a set of “success criteria” they might not even be sure what success means or looks like. Success criteria for automated testing are not set in stone either.  Company goals drive success criteria, and as such, they are highly personal to each organization. Instead of simply getting tools in place, business leaders need to understand the issues they are trying to solve first.

That said there are several test automation success criteria that seem to be universal.

• The test automation system works.
• It tests your apps.
• While testing it finds bugs.
• Test Automation saves time.

If you can answer yes to these questions chances are you have already reached test automation success. Keep reading though. There’s more.

Painting a picture of Test Automation Success

If you are looking to achieve test automation success here is some important advice to give your teams and help make it happen.

Automation by default

There are two ways to look at this. Teams that say “Everything must be automated” and teams that say “Tell us why this should be automated“. The key difference is in a situation where the default is to automate everything. Sometimes this works, and other times it is equally helpful to understand what should NOT be automated in your testing suites.

The reason for this is that creating automation standards across enterprises is a practice fraught with challenges. There is no one way to create a definition that works across all teams, so any automation standards you come up with will likely show only minimal practicality. It is also important to note that the tools that you use will improve your automation process. It is not the tools themselves that define the process. A bus driver, even with the most advanced machinery still needs to drive the bus. He cannot simply put it on autopilot and kickback. (Not yet at least.)

You want to know how to automate different scenarios while remaining open to the fact that not everything can be automated. If your automated testing is a mess it will become an even larger mess. It is important to get the entire process in order first. The tools you use will not help fix any process issues you might have either. It will simply automate an already messy process. We advise not building on messy testing projects. Rather you Automate in parallel deleting redundant tests as you go.

A resilient process leads to well-written tests which lead to better results. Along with that comes the understanding that it’s not necessary to force everything into an automation suite. When you do decide what to specifically automate your automation tools will help you conform to the criteria you have set in place that deems your apps ready for release.

Reruns if you haven’t seen it it’s new to you

One of the best ways to create test automation success within your company is with rerunnable automated tests. If a test can only be performed a few times that is something best left for the manual team. The best-automated test cases run frequently and use tons of data to perform the same actions.

Perfect for automation are tests that:

• Repeat and run for multiple builds
• Cause human error
• Require multiple data sets
• Cannot be performed manually
• Run-on different platforms or configurations
• Take time and effort when done manually

Aside from this, you need to plan carefully. While we advise companies to develop an automation plan remember we said above that these are not the be-all and end-all. The main point is to define your goals clearly and figure out which tests to automate. Repeatable tests are a great way to expand your test coverage and speed up regression testing, but it is not good to force all testing into an automation framework.

Once that is complete divide your tests up. A test environment is vastly more efficient when tests are manageable. It will also make sharing test code, data, and processes a breeze.

Testing Early, and Testing Often

Another piece of advice we give is that testing should start as early in the process as possible. By extension then your tests should run as often as possible. It fits with the growth of Shift-Left testing and the idea that unit testing should start early in the development process. It also empowers developers and testers to work together, in order to test earlier. What should go without saying is that when you test more you will discover more bugs. Shift-left practices are easier than ever to integrate with your IDE so validating app quality can be done from within the ecosystem. It means that by shifting left your apps will be delivered faster and with fewer bugs.

The Right People for the Job

Some managers think that since automation tools do the work that interns can run the software. By that understanding, you should be able to operate a self-driving car without a license. Not so much, however, as it is imperative that you get the right people in the job. Your testers need to know how the application they test is built. They also need to know how to write test code and execute the tests. Test automation success is a complex process it pays to have the right people in the right positions.

If your organization has detailed and specific test automation projects dedicated testers are essential. Developers do not have the time or ultimately the motivation to split their time between manual and automated testing. If you try to force them into that position then your automated testing will never really get off the ground.

What does Test Automation Success look like?

With the proper planning, implementation, people, and tools everyone in your organization should understand what test automation success means to them. Their specific goals might change but the high-level understanding will be unified.

The advice we give teams for test automation success is to create plans and implement measures that give timely and reliable insights to their apps. It goes like this a high-quality app leads to happy team members from testers up to product owners. And you know who benefits the most from that? Your customers.

Jonny Steiner – Content Manager