How testing automation can build a culture of QA while accelerating continuous delivery

An organization’s level of automated test coverage is quickly emerging as a competitive differentiator. The goal of continuous delivery cannot be achieved if the process is hindered by legacy practices, like considering manual testing the default. A lack of test coverage can also lead to things breaking in production, resulting in unplanned work and unexpected downtime. 

As digital transformation expert and author Isaac Sacolick notes, “Automation let teams develop tests against individual or groups of application components and services, making it easier to isolate issues before they become production defects.”

By automating tests either on different areas of the software or at varying levels (e.g., unit level, integration), it is easier to identify issues and fix them early on. It becomes much more difficult to isolate root cause and fix an issue in production rather than in development.

Testing automation also gives organizations the agility needed to deliver changes fast while minimizing risks. Furthermore, test automation creates an infrastructure within development so that every stage of work is checked for integrity and performance, as well as priority issues, such as security, governance, and compliance. With automated testing in place, the results of each stage of development can be verified before advancing to the next stage.

Like coding, test automation must adhere to best practices. Diligence ensures the desired, value-producing outcomes both at speed and at scale. Some recommended approaches include:

• Incorporating test automation into your organization’s DevOps pipeline
• Letting developers drive test creation to foster ownership of the process
• Setting criteria for test coverage to balance risk management with agility, ensuring that testing focuses on specific areas or software features

Entwine automated testing into DevOps 

Some see automated testing merely as a way to replace manual tests, which QA specialists often find to be slow, frustrating, and repetitive – leading to errors. 

But automation also represents a more fluid movement of work and value for DevOps. Testing itself becomes an automatic process rather than a step that has to be deliberately enacted or encouraged.

Automated testing can also mean quicker testing at more frequent intervals, potentially catching defects or errors before they begin to generate unplanned work. It also leaves an automatic paper trail, and it eliminates much of the human error that comes along with manual steps and processes.

Best practices must be followed to meet the goal of continuous testing alongside continuous delivery. A prime example: test scripts should not be built in isolation but rather as part of a test suite. When the first script is written for a new feature or function, developers should immediately create a test suite that others can add to as needed. This will reduce the friction generated at each stage of new work, setting a model for subsequent changes in the same area.

A solid testing architecture can mean the difference between added freedom or frustrations for your development team. This is why many organizations encourage the use of testing frameworks, which carry benefits including: 

• Faster test creation
• Easier maintenance 
• Preventing defects, instead of detecting them in production
• Focus on user stories, not tech functions (e.g., prioritizing the need for fewer user clicks over the need for validation checks at multiple stages of platform use)

Make testing a part of developer culture 

The best automated test scripts are generated from within the development team, not outside of it.

According to DevOps Research and Assessment (DORA) data, “developers are primarily responsible for creating and maintaining suites of automated tests, and when it is easy for developers to fix acceptance test failures, this drives improved performance.”

Developers need to understand the tests and should also be able to tweak pass criteria to optimize testing towards given sprint goals. 

There are cases where tests can be built outside of the development team, as long as developers are still responsible for the quality of their own code, fixing bugs, and are accountable for performance. They should also have the ability to requisition optimizations based on their intuition.

Still, developers will need to be guided to ensure that everyone involved is following the testing practices that will increase the chances of a quality end-product. This diligence gives QA experts the opportunity to move into more consultative roles as organizations rely less on manual testing processes. Such a transition not only gives QA experts more autonomy, it also helps developers learn how to write better tests and improve their approach when screening for quality issues.

It’s important to note that simply creating test suites is not enough to ensure the desired results. Development teams must be capable of proactively monitoring test results, interpreting them, and then using feedback to improve build quality.

One popular training module on test automation suggests the following factors will indicate when teams have a positive automated testing culture in place: 

• Teams are guiding development with business facing tests
• Testing and coding activities are integrated
• Team immediately addresses test failures 
• Whole team engages in monitoring and test production
• Tests cover performance, security, reliability, and resilience
• Teams generate testing suites alongside new features

Be strategic about automated testing implementation

Test automation can help organizations achieve their CI/CD quality goals, including improving release frequency as manual steps are removed. But automation by itself is not a magic wand. Organizations must create goals for test automation, including what they hope to achieve while using introspection to determine whether changes are needed. 

The first step is to audit the DevOps process itself in order to find processes that aren’t working and to learn how to improve them, while also highlighting optimization opportunities.

This evaluation process is a critical first step. According to David Linthicum, Chief Cloud Strategy Officer at Deloitte Consulting, “automation is about taking once manual processes and placing technology around them so they’re inherently repeatable. If your processes are bad or flawed, then you’re just making bad processes happen faster.”

Linthicum also cautions against the approach of automating simply for automation’s sake. Organizations that chase “automation without purpose end up with too many tools and must adjust their DevOps processes to accommodate those tools, not the other way around.” It’s imperative to apply automation in order to achieve specific goals, such as increasing release frequency, reducing manual steps, or improving release quality and/or reliability. Then, once goals are defined, start with specific use cases to get acclimated to the automated process. 

One testing consultancy suggests test automation in specific scenarios or when tests fit the following criteria:

• Repetitive in nature and have to run frequently
• Prone to manual errors 
• Expensive and time-consuming when performed manually
• Labor intensive and require a heavy resource investment
• Run on various software or hardware platforms and configurations
• Require large amounts of data

Organizations should run fast tests first in order to catch feature-breaking errors earlier – and before more time-consuming measures are taken. Faster tests will also be more focused and isolated, for example, in the case of unit testing of a specific feature. 

In an ideal environment, tests should be completed in the following order:

• Base level: Unit tests – Unit tests are fastest and can quickly catch specific errors
• Mid-level: Service tests – Service tests have a broader net and can catch more pervasive errors; however, resolving failed tests can be more time-consuming. 
• Top level: UI tests – UI testing is extremely time-consuming, with or without automation, and should only be completed once unit and service tests have been passed. 
• Cherry on top – Manual testing, if needed, can be enacted once automated testing is complete.

Testing automation can make continuous delivery a reality 

The goal of achieving faster releases while managing risks requires a fluid process where test automation is fully integrated into the overall DevOps pipeline.  

As a prominent DevOps outsourcer states, “the natural progression for a DevOps process that is already highly automated is to transition more of the QA process to fall in line, either through test driven development (TDD) or automated scripts running continuously during testing of newly developed features.”

Organizations focused on their use of automation and that encourage buy-in from development teams will see the most success, as test automation facilitates quicker releases with better integrity, security, governance, and compliance than ever before. 

Learn more in this on-demand webinar where Guy Arieli, Digital.ai CTO of QA, shares continuous testing best practices in digital transformation and demonstrates how continuous testing platforms can help you release your applications faster and save you time and money.