The automation testing advice that management most frequently ignores

No one likes to be told “I told you so” but sometimes the people doing the telling take a perverse amount of joy in reiterating how they were right about something and knew it all along.  

We are not here to tell you to act that way, though the desire is understandable. Members of testing and QA teams are often caught between the product team, developers, and business team members. The role of testers in an organization is more than simply the main aspect of testing. They must be able to communicate effectively and bring teams together for a common goal.  

The goal is web and mobile application delivery at speed and at scale. 

QA leaders, and test team managers need to apply their communication skills in several ways.

• Convincing business leaders of the value of test automation
• Understanding the user perspective and identifying gaps to management
• Working with the developers to fix defects 

However, as we know there are communication breakdowns in companies, and it is often that the best laid suggestions by QA or testing team members can be disregarded. 

So, let’s look at some of the most relevant advice that QA and testers give but is often ignored.

More automated testing does not automatically equal a better test plan

Business leaders like to hold on to this one. Of course, automated testing is good and of course more is better. However, there is more to the picture here. 

We know the benefits of automated testing. So why not automate everything? The short answer is that it is not only difficult, but also unnecessary. In the past you would only automate tests once your platform and user flow were consistent and stable. This made test creation and maintenance much more time-consuming.  

The truth is that these days with our tools like our new Test Editor capability creating tests manually and validating them takes less time than ever. So, we are left with the original question: why not automate everything? 

You might be able to get up to 80% automation but there are still scenarios that require a human interaction to execute. While you might think that customer and business pressure add up to trying to automate up to 100% there is still a human interactive part of software testing that is important to preserve.

Business teams make impractical estimates and promises to customers

When preparing to test software there are two key issues that companies face. How long will testing take and how much will it cost? Often in organizations with poor internal communication, these answers can be given with impractical estimates and promises. The issue is that software test estimates are created by management, and they do not always understand the technicalities of continuous testing.  

Estimation is essential in making sure that testing teams do not exceed their time limits or budgets. It helps in test project planning as it gives you the power to allocate resources and optimize testing activities.  

So how can business leaders make better software testing estimates without throwing their testing and QA teams under the bus? 

There are four core aspects to making a good continuous testing estimate. Time, resources, cost and skills. Let’s take a closer look at each of them.  

• Time: It starts with the team managers calculating the optimal time it should take for each part of a testing project. They can then prepare schedules to make sure the test execution happens within the parameters they set. The success of these teams will be determined by their ability to meet their deadlines.
• Resources: These include team needs like infrastructure, tools, and specialists. If the team is not empowered with the items that they need for a successful test execution it will not be completed in time.  
• Cost: This is partially included in the resources section. A successful testing estimate has teams understanding what they need and preparing a budget accordingly. The ideal situation is one where the budget is set beforehand, and the testing project is aligned with that.
• Skill: This refers to the technical and professional skills of team members. If they are lacking or deficient, then the overall testing process will slow.

Implementing shift-left and become testing centric

The purpose of shift-left methods is to discover defects in the development pipeline as early as possible. It means testing earlier in the process which requires better collaboration and communication between teams.  

There is pushback on adopting these methods from management because of the cost in both time and money to get the process implemented. They still see testing as the last step in the SDLC which leads to problems. Most concerning is that testing starts to become a bottleneck which delays teams that are trying to speed up their software delivery.  

The point of shifting left is that yes, testing can still happen at the end of the cycle but having already tested ahead of time means it will be a smaller and faster execution. In that sense shift left is not exactly moving testing closer to the beginning of the cycle but it is more of a seasoning sprinkled over the entire development process.  

More QA does not mean more defects

Some business leaders look at QA as a necessary evil. They know QA needs to implement the processes associated with releasing quality software, but the process often takes place late in the release cycle and can feel like a bottleneck. They believe erroneously that more QA will lead to more defect discoveries and thereby delayed releases.  

Where testers are identifying defects and errors, QA is looking at more usability issues that are potentially not even technical. Hence the feelings of QA and delays. They are considering defects outside of the realm of bugs. QA focuses on the end user experience and more performance related issues like bad navigation, or slower loading times.  

So, while more QA will lead to a better app with improved user experience, it does not mean that an app will be discovered to be any more broken than the testing teams have already discovered.  

The bottom line

When we talk about business leaders ignoring advice, what it often seems to boil down to is power and intuition. On the one hand they often want to preserve their own sense of power and ownership over company processes. On the other hand, they might overly trust their gut when it comes to making decisions as it is often that gut feeling that initially got the leader into their place of power. 

When it comes to software testing and QA the notion of more testing or increasing QA often raises eyebrows as it can lead to delays in the SDLC. Aside from being open and honest with business teams companies need a way to not only move testing early in the development process, but also find a reporting tool that helps to open the dialogue and provide empiric evidence as to the viability of their software testing processes.

Learn more about Digital.ai Continuous Testing solution here.