Build Better Parallel Testing with Multithreaded Execution

 

In today’s fast-paced world, a faster time to market is a key requirement for businesses of all sizes. As such, businesses expect QA teams to quickly provide feedback on their software products. While test automation significantly reduces test time and improves operational efficiencies, there are times when certain code takes longer to execute and stalls your testing automation projects. Parallel execution comes to your rescue here. Parallel testing is a process of simultaneously running test scripts on multiple browsers and platforms.

While parallel execution significantly reduces test times, it comes with certain challenges. Separating test scripts for parallel execution is a complex task. You should also consider thread safety in multithreaded execution as concurrent processes run at the same time. You should have a clear understanding of how threads interact with each other. So, multithreaded execution is a key component of software testing projects.

An Overview of Multithreaded Execution

Integration testing is a part of software testing wherein software modules are logically integrated as a group and tested to see how they interact with each other. It focuses on how multiple different modules communicate data. Multithreaded execution is a part of integration testing that tests specific functionalities of a thread. A thread is the smallest unit of work executable by a machine. It is commonly performed at the beginning of integration testing and is mostly used for testing client-server architecture.

In the initial stage, integrated threads are first tested as subsystems and then tested as a whole system. As this integration testing is done in an incremental model, testing sequentially processing tasks, it is also known as Thread Interaction Test. It is best suited for object-oriented real-time systems. Generally, Multithreaded execution starts with testing a single and individual thread and then proceeds to test multithreaded execution.

You can carry out Threaded execution in two ways:

• Single thread testing: Single Thread testing is about testing one application transaction at a time to see how the application logic works, how to carry out input and output data transactions as desired.
• Multithread testing: Multithreaded testing is where concurrent transactions of an application are running at the same time. It can run on a single machine or distribute across multiple machines. Thread testing is critical in the client-server architecture because a single thread might work well when it is not interacting with other objects held by other threads. However, the same thread might fail when it is running in a multithreaded environment. So, thread interaction is a key aspect to check out.

While performing multithreaded testing, writing reproducible code for unit tests is a big challenge. Similarly, you should take into consideration the fact that resources such as memory, storage, and speed. are different for each hardware. So, carefully writing your code is the key here. Once you take care of these challenges, you can reap all the benefits offered by thread testing. You can check the capabilities, performance, and functionality of client-server–based applications with ease.

Parallel Execution for thread testing

By running parallel testing, you can reduce test execution time while being able to test multithreaded code in the applications. TestNG is a popular test automation framework that helps you run parallel testing for multithreaded execution.

Here is an example to check thread ID:

[java] public void ExperitestThreadOne() { long id = Thread.currentThread().getId(); System.out.println(“Experitest Thread id: ” + id); } [/java]

Now, let us create a small test script that will run two methods and check the thread IDs.

Open Eclipse IDE and create a Java project and create a class in that project and write the following code:

[java] package ExperitestMultiThread; import org.testng.annotations.Test; public class ExperitestMTDemo { @Test public void ExperitestMethod1() { long id = Thread.currentThread().getId(); System.out.println(” Experitest Method 1 Thread id is: ” + id); } @Test public void ExperitestMethod2() { long id = Thread.currentThread().getId(); System.out.println(“Experitest Method 2 Thread id is: ” + id); } } [/java]

Now, run this program and check the output.

If you check the output, both methods are running in a single thread in sequential order.

Now, let us run both methods in parallel. To do so, create a new file and name it TestNG.xml and add the following code to it.

  [xml] [/xml]

Alternatively, you can right-click on your project and click on TestNG -> Convert to TestNG

Now, choose Parallel mode -> methods and thread count as 2 and save the file.

Run the TestNG.xml file as TestNG suite:

You can see that each method is running in a separate thread. (Thread 14,15,14)

Similarly, you can run parallel testing based on tests, suite, etc.

TestNG also allows you to run parallel testing in a multithreaded environment using the ‘threadPoolSize’ and ‘invocationCount’ attributes.

Here is an example of how to run parallel tests in a multithreaded environment.

[java] package ExperitestMultiThread; import org.testng.annotations.Test; public class ExperitestMT { @Test(threadPoolSize = 4, invocationCount = 8, timeOut = 1000) public void ExperitestMTEnvironment() { Long id = Thread.currentThread().getId(); System.out.println(“Experitest multithreaded testing is now running on Thread: ” + id); } } [/java]

In this example, we set the threadPoolSize to 4. It means the program will run on 4 different threads. Similarly, set the invocationCount to 8 which means the method will execute 8 times.

Now, create a testng.xml file and add the following code:

[xml] [/xml]

When you run the testng.xml, it will execute the same method 8 times using 4 threads.

Here, you can see that the program is executed in 4 different threads (Thread 14, 15, 16, 17)

Multithreaded Execution on SeeTest Platform

To run multiple test cases in parallel, you need adequate computing resources. SeeTest offers a powerful test automation platform that will enable you to run parallel tests on multiple browsers and platforms.

Here is an example that runs test classes in a parallel mode:

Create a Java project and create two classes in that project. In the first class (ExperitestSearch.java), add the following code:

[java] private static final String ACCESS_KEY = “Enter your access key here”; private RemoteWebDriver driver; private URL url; private DesiredCapabilities dc = new DesiredCapabilities(); @BeforeMethod public void setUp() throws Exception { url = new URL(“https://cloud.seetest.io/wd/hub”); dc.setCapability(CapabilityType.BROWSER_NAME, BrowserType.CHROME); dc.setCapability(CapabilityType.PLATFORM, Platform.ANY); dc.setCapability(“accessKey”, ACCESS_KEY); dc.setCapability(“testName”, “Quick Start Chrome Browser Demo”); driver = new RemoteWebDriver(url, dc); long id = Thread.currentThread().getId(); System.out.println(” Experitest Search Setup Method Thread id is: ” + id); } @Test public void browserTestGoogleSearch() { driver.get(“https://www.google.com”); new WebDriverWait(driver, 10).until(ExpectedConditions.presenceOfElementLocated(By.name(“q”))); WebElement searchBar = driver.findElement(By.name(“q”)); searchBar.click(); searchBar.sendKeys(“Experitest”); searchBar.sendKeys(Keys.ENTER); long id = Thread.currentThread().getId(); System.out.println(” Experitest Search Test Method Thread id is: ” + id); } @AfterMethod public void tearDown() { System.out.println(“Report URL: “+ driver.getCapabilities().getCapability(“reportUrl”)); driver.quit(); long id = Thread.currentThread().getId(); System.out.println(” Experitest Search Teardown Method Thread id is: ” + id); } [/java]

private static final String ACCESS_KEY = “Enter your access key here”;

This program will log in to the SeeTest platform and opens a Google Browser and search for the keyword ‘Experitest’. After the search results are displayed, it will close the browser. It will display the thread ids for each method.

Now, create another class (SeeTestSearch.java) and add the following code:

[java] package ExperitestMultiThread; import org.openqa.selenium.*; import org.openqa.selenium.remote.BrowserType; import org.openqa.selenium.remote.CapabilityType; import org.openqa.selenium.remote.DesiredCapabilities; import org.openqa.selenium.remote.RemoteWebDriver; import org.openqa.selenium.support.ui.ExpectedConditions; import org.openqa.selenium.support.ui.WebDriverWait; import org.testng.annotations.AfterMethod; import org.testng.annotations.BeforeMethod; import org.testng.annotations.Test; import java.net.URL; public class SeeTestSearch { private static final String ACCESS_KEY = “Enter your access key here”; private RemoteWebDriver driver; private URL url; private DesiredCapabilities dc = new DesiredCapabilities(); @BeforeMethod public void setUp() throws Exception { url = new URL(“https://cloud.seetest.io/wd/hub”); dc.setCapability(CapabilityType.BROWSER_NAME, BrowserType.FIREFOX); dc.setCapability(CapabilityType.PLATFORM, Platform.ANY); dc.setCapability(“accessKey”, ACCESS_KEY); dc.setCapability(“testName”, “Quick Start Firefox Browser Demo”); driver = new RemoteWebDriver(url, dc); long id = Thread.currentThread().getId(); System.out.println(” SeeTest Search Setup Method Thread id is: ” + id); } @Test public void browserTestGoogleSearch() { driver.get(“https://www.google.com”); new WebDriverWait(driver, 10).until(ExpectedConditions.presenceOfElementLocated(By.name(“q”))); WebElement searchBar = driver.findElement(By.name(“q”)); searchBar.click(); searchBar.sendKeys(“SeeTest”); searchBar.sendKeys(Keys.ENTER); long id = Thread.currentThread().getId(); System.out.println(” SeeTest Search Test Method Thread id is: ” + id); } @AfterMethod public void tearDown() { System.out.println(“Report URL: “+ driver.getCapabilities().getCapability(“reportUrl”)); driver.quit(); long id = Thread.currentThread().getId(); System.out.println(” SeeTest Search Teardown Method Thread id is: ” + id); } } [/java]

This program will log in to the SeeTest cloud platform, opens the Firefox browser, opens the Google search site, and searches for the keyword ‘SeeTest’.

Now, we have created two classes. The next step is to run them in parallel. So, create a TestNG.xml file and add the following code to it.

[xml] [/xml]

Now, run the TestNG.xml file.

You can see that both classes are simultaneously executed in different threads (Thread 16 & 15). Similarly, you can run methods, tests, and suites in parallel using the TestNG framework.

Summary

Multithreaded execution is a powerful way to run parallel testing. Using the powerful SeeTest test automation framework and TestNG framework, you can build better parallel tests and easily execute them in a multithreaded environment without worrying about computing resources. However, it is important to note that scripts written for thread testing should be independent of each other and also be reproducible. They should deliver the same results every time when using the same inputs and application versions. It is also important to use thread-safe variables in your tests. By following thread testing best practices, you can not only build better parallel tests for multithreaded execution but also optimize resources while significantly reducing test times.

Guy Arieli – CTO