The More Software Development Changes, the More the Testing Challenges Stay the Same

By Jonny Steiner, Product Marketing Manager

 

As Professor Barnhardt and Klaatu discuss in the 2008 remake of “The Day the Earth Stood Still,” the doctor asks, “There must be alternatives. You must have some technology that could solve our problem”. Klaatu blithely responds, “Your problem is not technology. The problem is you. You lack the will to change.”

It is almost as though screenwriters David Scarpa and Edmund H. North sat in on one of my recent customer meetings. In a world where businesses are undergoing rapid digital transformation, I have been approached more than once by development and testing professionals looking to solve their problems with technology. They ask me what we do, and inevitably, I respond by asking them about the maturity of their testing process. “We can meet you where you are,” I confidently explain.

It seems like a simple option. Get a tool and get digitally transformed. However, as we know in practice, it takes much more than purchasing a shiny new tool to help your business transform.

Evolving With the Times

The rise of personal computing in the 1980s enabled individual productivity and connectivity. The 90s internet revolution further connected people and businesses worldwide. Once the aughts set in, e-commerce was proliferating while mobile devices began to emerge. These days as we harness the power of data and cloud computing, digital transformation has become an essential part of the evolution of organizations. Companies must adapt to the times and use digital technologies to innovate, increase efficiency, drive innovation, and enhance the customer experience.

Software development testing and delivery have become more dynamic over that period and evolve incrementally each year. Driving the development of software development methods is the need for a faster time-to-market, aiming to improve customer satisfaction. Agile and DevOps practices are replacing legacy methodologies like Waterfall as they emphasize continuous delivery, frequent feedback, and close collaboration between development, testing, and operations teams.

Similarly, digital transformation has become a key focus for organizations, and the result has technology integrated into every piece of the business, from Agile planning to deployment. However, with the evolution of software development testing and delivery, it is crucial to recognize that the fundamental challenges have remained unchanged despite technological advancements and methodology. These persistent challenges affect global organizations daily, and while they might not be new, the methods used to mitigate them can brush on technology so unique it might look like Science Fiction.

The Testing Challenges Remain the Same

It seems antithetical, but it is true. Many challenges plaguing testing organizations worldwide are the same as eight to ten years ago. Modern software systems are complex, while the development and testing cycle time grows ever shorter. Organizational silos can crush communication and collaboration between stakeholders, developers, and testers while these teams continue to produce vast amounts of data. Managing that data, and testing environments, is an issue and is just as relevant as ensuring that you have enough test coverage and scalability to truly meet your needs. That makes adapting to the times essential. Though as we will see, those times have not changed as much as you would think.

Testing Challenges

Test Automation

The challenge here is that automating a continuous testing environment requires skilled resources to develop and maintain the automation framework. Unfortunately, a global lack of skilled QA and testing professionals has kept this challenge at the top of the list. Some of the other details inherent in the challenge of test automation include:

• Skills – Test automation requires practitioners to be skilled in programming, scripting languages, and test automation frameworks. Those skills are employed in designing, developing, and maintaining frameworks while keeping them aligned with organizational needs.
• Learning – The team’s skills are important, but equally important is leveling up those skills to the latest technologies, tools, and best practices. Continuous learning is crucial for teams to leverage the full scope of their automated testing.
• Scalability – As software evolves, so must the tests with it. Updated automation scripts must support any web or mobile application changes.

The framework itself must also scale with the growing complexity of the application.

Environment Management

The challenge here is that developers and testers must access multiple environments to ensure the code works across various scenarios. Further exacerbating the issue is the increasing complexity of web and mobile apps which adds to the difficulty in managing these environments. Drilling down into the challenge, we find that teams struggle with the following:

• Access – Developers and testers must access development, staging, and production environments for code validation purposes. Each of these environments will likely have unique configurations, databases, and third-party integrations.
• Complexity – Web and mobile apps often have multiple components and services with their own dependencies and configurations. Keeping all components connected, available, and configured for each environment is more important than ever.
• Data – Managing the data generated by testing is a huge task. The data must be protected while also synchronized across the organization. Managing data dependencies and consistency while handling data migrations across environments takes care and effort.

Test Data

The challenge here is simple. Testing generates data. Tons of it. Once generated, data needs to be managed and analyzed to help make it actionable. Some of the more detailed points about this are:

• Realism – Test data needs to mimic real-world scenarios to ensure accuracy. Having a diverse data set is important but also complex and time-consuming. Test data must represent the production environment as accurately as possible to unearth defects effectively.
• Security – Sensitive information often makes it into test data. This includes personally identifiable data or proprietary business data. Privacy is key, as is staying compliant with global regulations.
• Actionability – The data generated by continuous automated testing needs to be presented in a way that helps identify issues and track results. Data must be organized and structured to enable the best possible analysis of results.

Fast Feedback

Every organization needs this, and yet it falls to the wayside. The importance is to enable developers to identify and address defects quickly and protect the development and testing process from breaking down. What is most relevant to this challenge is:

• Identification – Defects need to be unearthed as early in the development and testing process as possible. The longer it takes to mitigate a defect, the harder that mitigation will be in terms of the cost and time needed to fix it.
• Resolution – Once discovered, a defect needs to be mitigated at once. Leaving this unresolved will cause bottlenecks and harm the process. Adding to the challenge are the tools, resources, and expertise developers and testers need to investigate and fix defects efficiently.
• Reporting – It boils down to communication. Testers must report defects and include as much information as possible to developers. They, in turn, need to understand and prioritize the testers’ findings so they can mitigate defects promptly.

Cultural Change

Where the previous challenges have remained essentially unchanged for years, this may be the newest challenge added recently. To change a culture, you must break down silos, improve collaboration, and change your development approach to a more iterative one. What is most relevant to that challenge are:

• Silos – Even in organizations with defined continuous testing practices, teams are still siloed. Developers, testers, and QA all work in isolation while the processes they share languish. Breaking down these silos requires sharing responsibility and work to help improve communication.
• Communication – Speaking of which, the need for better collaboration and communication is essential to define requirements, set quality goals, and align efforts better. Equally important is to share feedback and knowledge as well.
• Support – This must come from management’s strong support and leadership. Team leaders need to champion the adoption of continuous testing while providing teams with the resources, tools, and know-how they need to move forward.

Overcoming the Challenges

The challenge of test automation is met by an organizational investment in training and improving automation resources. However, using automation tools and tech that help foster ease of use, maintainability, and scalability is a good start.

When we look at managing testing environments, what is needed here are effective environment management practices. You can ensure consistency across all environments with better version control and change management practices. We will discuss communication more, but it is relevant in the context of environment management in that it can help development, testing, and operations teams streamline the process.

It leads straight to the challenge of test data in that these processes and environments generate massive amounts of data. Data management tools are helpful here, providing features like data generation, data versioning, and data synchronization. However, guidelines for test data management must be clear and processes well defined.

That data comes in the form of fast feedback, which is easier to digest when working in an Agile environment that promotes collaboration and feedback. With the right tools in place for defect discovery and resolution, you will improve the velocity of the feedback loop.

It all adds up to a cultural change across the organization as you establish cross-functional teams that encourage collaboration through regular meetings and workshops and promote knowledge-sharing and learning initiatives. Of course, management needs to lead by example but with tools that can give test automation powers to business users, this is a massive development in that direction. Changing a company’s culture takes time, persistence, and clear communication, but it is a process that pays off in many ways.

The Payoff

It was a shocking realization to discover how immature testing processes are globally. Many large organizations still need to implement continuous testing and are bogged down by legacy processes and manual work. Additionally surprising is how fragmented testing tooling is even within mature organizations with automated testing practices and processes. It is a testament to the speed of the industry and the stubbornness of some leaders who seemingly cannot implement a culture change that unifies tooling, and processes, breaks down silos, and increases communication across stakeholder, developer, and testing teams.

Business leaders need a greater stake in their development and testing process. Not in terms of responsibility or overall authority. Instead, they need the ability to dig in the trenches and gain the perspective of their teams. We discussed tooling through this blog post, and this might be the key. First, an analytics tool that gives everyone in the organization a complete view of their development and test executions will help all teams understand each other’s work. In contrast, hundreds of AI-powered insights and analytics will help drive smarter decisions. Lastly, low code tools that allow business users to create automated tests further help to break down silos, encourage communication, and create a lasting and positive organizational shift.