How to expertly manage the risks of Quality and Velocity

We have discussed many methodologies in this blog before. From the best methods for agile VSM to BDD and more.  Let’s not forget continuous testing, integration, and delivery. I’m not trying to show off. The point is that these methodologies all help you enable a DevOps forward culture in your organization. While DevOps means different things to different companies, and even separate teams within, the main goal of DevOps is to make the time between committing changes to your code and getting that code to production as short as possible. Simply put you need quality and velocity to release an amazing app.

The entire process is like modifying a sports car. The car right off the lot is probably a decent machine. However, all the tinkering and upgrading you do might get you 1000 horsepower. If not properly tested your engine will explode.

High-speed software delivery often comes at a similar loss of quality. Yes, everything might be working fine in your nightly builds. Your unit testing might look like a string of green lights stretching as far as the eye can see. There are always speedbumps between you and the horizon. If your eagerness to reach high velocity leads to less than high-quality releases, your web or mobile app will have poor reviews and high abandonment. We have seen this situation many times and it is hard to recover from.

The risks of Velocity versus Quality

Looking at the chart above you can plainly see that you can increase quality by reducing velocity. This is usually due to taking longer with each feature for testing and improvement.

The opposite is also true. Increase velocity by reducing quality. The strategy is to release apps quickly, warts and all, in order to use your users to help discover problems and report them back.

The long way is to automate everything, but as we will see it is very difficult. Let’s look at some other examples.

Waze has chosen the route of velocity. They frequently release beta versions to users to get them working on the latest versions, but also in order to receive user feedback which helps them improve functionality.

Similarly, Booking.com uses its customers to test new versions and traffic. They monitor logs and activity as they go and continuously improve their apps that way.

Companies in the financial sector, on the other hand, are willing to place their app’s quality over velocity. They understand the sensitive nature of their customer’s information and the high-stakes nature of their industry.

Companies often hold their web and mobile apps in “beta” stages for a long time because as quality increases velocity goes down. By having your software listed as “beta” you can release a lower quality app and increase your velocity without worrying about customers savaging your “beta” app.

It almost feels like flipping a coin, whichever side it falls on, be it velocity or quality the other will suffer. Sometimes it seems like organizations need to choose one or the other rather than make them work together.

The bottom line is that the debate surrounding velocity versus quality is constant. While you will find plenty of people on either side of the argument, what you will not find is a standard solution.

Mobile App Testing Risk Management

Every QA tester has some risk management responsibility. They must identify specific areas in which to prioritize testing. They also need to discover risk early in each sprint to keep the development team meeting their deadlines. Every decision regarding quality and testing has a certain degree of risk attached to it. We encourage our QA team to be aware of any factor that can cause a project to fail. Testing is their weapon against that.

In agile development, new version deployments live and die by testing. The earlier in the sprint that QA can implement their app testing the better. Falling behind means that your app velocity will suffer. Unit testing is the best way to remove this risk.

The most important aspect of risk management in mobile app testing is uncovering app bugs early on. This will help the collaborative fast-paced agile environment you are trying to build, work. There are 4 main types of risk that affect your quality and by extension your velocity.

• Project risks – These risks come from issues with communication, and knowledge transfer. The human element of software development comes in here.
• Product risks – Literally the bugs or poorly functioning elements of a new release. These are the simplest for testers to find.
• Technical risks – These are more infrastructure-related risks, like integrating with external web services, or security and performance issues.
• Business risks – These are features that are so important to customers that their failure would be catastrophic.

The adoption of Agile and DevOps

As organizations all over the world adopt Agile and DevOps practices velocity has become a priority. It makes sense on the surface, with new features coming almost daily and a global culture that forgets things as quickly as the day rolls over to tomorrow. Speed in 2020 is of the essence.

Instead of trying to make velocity a matter of choice or preference we need to look deeper to see how to achieve both. The perfect SDLC with Agile and DevOps practices would combine the two. Management and the developer teams get the speed they constantly work toward, while QA and testers attain the level of quality that they know is vital to app success.

Quality and Velicity Combined into Automated and Parallel Testing

The closest we can get to a standard practice giving you the best of both velocity and quality is via test automation. Most obviously is the fact that since automation enables testers to rerun test scripts as part of their regression testing it means that they do not have to manually test their apps over and over. Automated testing opens the testers’ ability to test new features instead of constantly reviewing old ones.

Nothing is perfect, however, as we know that automated testing is a process that can take a lot of time. It is equally important to your testing to consider the browser versions device types, and OSes necessary to enable your testing. That is a time-consuming process.

In order to build a balanced approach and mitigate the velocity and quality risks you need Parallel test execution. This solution allows you to leverage a Selenium Grid with a device lab like our SeeTest Mobile device lab, or our Cross Browser testing tool. With parallel testing enabled testers and QA can run their test scripts simultaneously across thousands of devices and browsers. That gives developers and testers the ability to test across environments quickly and effectively.

Where testing tasks (unit, smoke, regression, cross-browser) have been time-consuming, and unwieldy, with parallel testing these tasks go from taking days to hours. Not only will this speed up your deployments, but it will also boost your test coverage. What it boils down to is that you will address your velocity and quality simultaneously.

What this means is that your business, developer, testing and QA teams no longer need to argue the merits of either speed or quality. They can focus on their tasks, and deliver great web and mobile apps to your customers.

It’s a Bird It’s a plane its QA

Let’s talk manpower too. The heroes behind the success of your DevOps culture are QA. Their job is to bring reality to the entire process. In other words, rather than taking a perfect app and breaking it, QA is more responsible for showing the developers and stakeholders that perhaps the web or mobile app they are testing was not as perfect as they thought.

They are involved in making sure your app works during different phases of the SDLC:

• Development
• Continuous Integration
• Quality Assurance
• Staging

The different types of testing (unit, acceptance, sanity, database, smoke, integration, functional, usability, compatibility, regression, security, performance, and more) fall under the purview of QA. In companies where they demand DevOps but have not updated their tools or environments, there is a growing concept. It is helping QA teams make a difference in mitigating the risks between velocity and quality.

QA as Code

This is a new concept that gives QA teams the ability to work on their environments as easily as their counterparts on the development teams do. With velocity as the means to the end goal of high-quality, QA as code reimagines QA as agents of change, instead of agents of chaos.

We work in a software world where “code” has become the default term for each piece of the SDLC. QA needs to have that same respect given to their work. With the right respect given to QA teams, they can lead the DevOps quality revolution which is critical when trying to achieve the highest quality releases in the least amount of time. When you treat QA as code you enable your QA teams to be the single source of truth for your testing suites the same way you enabled your development teams.

Summary

It seems like the challenge companies in the DevOps world are facing is trying to combine speed and quality. The conversations about whether it is either or are becoming a thing of the past with new tools and methodologies available to support both velocity and quality. By enabling your testing and QA teams to treat their work as important as developers do their part of the process will be treated with more respect. Additionally, tools like SeeTest and other automated and parallel testing platforms are blazing a trail that is allowing organizations to release high-quality web and mobile apps faster than ever.

Jonny Steiner – Content Manager