TestNG Listeners – An End-to-End Overview and Demo

TestNG Overview

TestNG stands for ‘Test Next Generation’. It is a popular test automation framework that enables you to create easily-to-read code scripts and generate customized test reports. They initially designed TestNG to perform unit testing. Lately, they extended TestNG to cover all types of testing, including integration system testing and unit testing. Cedric Beust designed the tool in 2003, taking inspiration from JUnit and NUnit. With powerful features such as annotations, TestNG listeners, data providers, etc. incorporated into the tool, TestNG is now more popular and easy to use compared to JUnit.

Why is TestNG popular?

TestNG enables developers to easily write test cases using annotations that can be efficiently prioritized, grouped, or easily executed. It supports data-driven testing and parameterization while easily integrating with IDE tools such as Eclipse and IntelliJ and build-tools such as Ant, Maven, and Gradle. You can perform parallel execution of test cases as well as group test cases and execute a specific test group. TestNG generates test reports in HTML with details of test cases that skipped, failed, and passed. It also generates logs along with reports.

What are TestNG Listeners?

One of the important features of the TestNG framework is listeners. It is an interface that listens to predefined events in test scripts and modifies the default behavior of the TestNG tool. Whether you want to customize reports or generate logs for specific tests, TestNG listeners help you to do so. Testers would also want to define prerequisites and clean-up configurations for each test project. Automate the process of setting up the prerequisites before starting your testing process. Performing clean-up operations after the testing finishes with a TestNG listener.

There are different types of Listeners in TestNG.

• ITestListener
• ISuiteListener
• IReporter
• IAnnotationTransformer
• IAnnotationTransformer2
• IMethodIntercepter
• IHookable
• IConfigurable
• IConfigurationListener
• IExecutionListener
• IInvokedMethodIntercepter
• IInvokedMethodIntercepter2

While multiple listeners exist, not every listener is regularly used. Other popular TestNG listeners include ITestListerner, ISuiteListener, and Ireporter.

Each TestNG Listener contains different methods invoked to perform a specific task. For instance, when a test case fails or is skipped, you might want to take a screenshot of that particular test case to see what went wrong. In that case, you can invoke the onTestFailure() method within the ITestListener and move the test execution to a specific block and execute a specific event.

For instance, the ITestListner contains the following methods.

• onStart(): Invoked at the start of a test.
• onTestFailure(): Invoked when the test fails
• onTestSuccess(): Invoked when a test passes
• onTestSkipped(): Invoked when skipping tests.
• onFinish(): Invoked when finishing tests.
• onTestFailedButWithinSuccessPercentage(): Invoked when the test fails but the results lie within the success percentage.

IReporter Listener

Generate customized reports with the IReporter Listener. It invokes the generateReport() method when executing all tests that take the following arguments:

• suites: The object that contains the suite information such as package, test method, class, test results, etc.
• xmlSuite: It is the XML file containing the list of suites that are to be executed.

IInvokeMethod:

This is a TestNG listener interface, used to perform specific actions before and after tests.

• beforeInvocation(): Invoked before each test.
• afterInvocation(): Invoked after each test.

Installation Procedure

Here are the prerequisites to use a TestNG Listener.

1. Java Development Kit (JDK)
2. Selenium
3. IDE to create code (Eclipse or IntelliJ)
4. TestNG
5. Build tool (Gradle or Maven)

Step 1: Install Java

To download Java, visit the following link:

https://www.oracle.com/in/java/technologies/javase-downloads.html

Click on the JDK download option and accept the license agreement.

The installation of Java is pretty simple. After the software gets downloaded, double-click the package to begin the installation.

After the installation is completed, it will show a message that the software is successfully installed.

Step 2: Configure Environment Variables for Java

To use Java, you should specify the Java path to the system. To do so,

• Open System Properties-> Advanced System Settings – > Advanced -> Environment Variables

Select the ‘User variables for user’ tab, and click on the path and then click on Edit. Now, add the Java path.

• C:Program FilesJavajdk-14.0.2

Under the ‘system variables’ path, enter the Bin folder path

• C:Program FilesJavajdk-14.0.2bin

Now Java is configured and ready for use.

Step 3: Install Eclipse IDE

To create and edit the test code, you need an IDE tool. You can either use Eclipse or IntelliJ. To install Eclipse, visit the following link:

https://www.eclipse.org/downloads/

Download the Eclipse IDE. The latest version is Eclipse IDE 2020-06

After the software is downloaded, double-click on it to begin the installation procedure.

1. Choose Eclipse IDE for Java Developers.
2. If you see a notification at the right top corner, you can click on it and choose ‘update’ so that the installer gets updated.
3. Choose the installation folder and click Install.
4. Now, the installer begins to install Eclipse.
5. After the installation is completed, click on Launch.
6. Choose a default Eclipse workspace.
7. Now, Eclipse is successfully installed. When you open it for the first time, it will display a welcome screen.

Step 4: Install Selenium WebDriver

To use WebDriver and perform tests, you should install the Selenium Server. You can download Selenium Standalone Server here:

https://www.selenium.dev/downloads/

After the zip file is downloaded, extract it and store the contents in a separate folder.

Step 5: Install TestNG

To install TestNG, open the Eclipse IDE, and click on Help.

Now, type TestNG in the search box and press go.

1. The latest version of Eclipse doesn’t show TestNG in the search results. Alternatively, visit this site and manually download TestNG. https://dl.bintray.com/testng-team/testng-eclipse-release/
2. Click on the ‘zipped’ option. On the next screen, click on the latest version and download the zip file.
3. Save the zipped file in a separate folder. Now, open the eclipse IDE and click on ‘Help’ and choose ‘Install New Software’.
4. Click on ‘Add’ and provide the path where you have stored the TestNG zip file. Give a name as well.
5. Click on ‘Add’. On the next screen, select TestNG and click on Next.
6. On the next screen, review the items to be installed and click Next. The next screen displays a license agreement. Accept it and click finish. It will prompt you to restart Eclipse.
7. Once the Eclipse is restarted, click on the new file -> Project.
8. Give a name for your project and click next.
9. On the next screen, click on library tab and click on ‘add library’
10. Click on ‘Add Library’ and select ‘TestNG’. Click Next and then click finish. It will set the TestNG lib as the default lib for this project.
11. Now, open Eclipse in package explorer views and right-click on the project you have created earlier. Choose New -> other
12. Click on TestNG and select TestNG class and click next.
13. Specify a class name and click finish.

Now, TestNG is ready for use.

Step 6: Install a Build Tool (eg; Gradle)

Build tools to automate the process of building executable applications from the source code. Gradle and Maven are the most popular build tools.

Visit the following link and download the latest Gradle release.

https://gradle.org/releases/

Unzip the files into a separate folder. Now, add the Gradle bin folder path to the environment variables. (Follow the steps mentioned in Step 2)

• eg: C:Gradlegradle-6.6bin

To use Gradle in Eclipse, open the Eclipse IDE -> New -> Other -> Gradle

For the mobile test automation demo, we will use the following code for Gradle.

[java]
apply plugin: ‘java’
repositories { mavenCentral() maven { url “https://cloud.experitest.com/repo/” }
}
test { useTestNG() “20.7”
}
dependencies { testCompile group: ‘org.testng’, name: ‘testng’, version: ‘6.11’ testCompile group: ‘com.experitest’, name: ‘SeeTest-client’, version: “20.7”
}
[/java]

So, your test environment is ready now!

TestNG Listeners – First Test Demo

To check how a TestNG Listener works, run this simple project.

Create a new Java project in Eclipse.

Give a name to the project and click next.

Now, click on the library tab and then click on Add external jar files.

Select the path where you have stored the Selenium jar files and click finish.

Now, you can see the selenium jar file listed under ‘Referenced Libraries’ in the package explorer tab in Eclipse.

Now, right-click on ‘src’ and create a new package and a new class. Give a name for the class. For instance, we will create two classes here. One is a parent class that has listeners and the other one is the child class that runs the test cases.

So, the first class is created with the name ExperitestParent.java.

In the parent class, we use two TestNG listeners.

1. onTestSuccess() method that will run when the test case is successfully executed.
2. onTestFailure() method that will run when the test case fails.

Here is the code used for ExperitestParent.java class:

[java]
package ExperitestListener;
import org.testng.ITestListener;
import org.testng.ITestResult;
public class ExperitestParent implements ITestListener
{
@Override public void onTestSuccess(ITestResult expresult) {
System.out.print(“Test passed. Details are “+expresult.getName()); } @Override public void onTestFailure(ITestResult expresult) {
System.out.print(“Test Failed. Details are”+expresult.getName()); }
}
[/java]

Now, create a new class and name it as ExperitestChild.java

This class will contain the test cases. One test case returns the Experitest website title and the second test case verifies if the title matches the predefined title. It will also contain the TestNG Listener added as follows:

• @Listeners(ExperitestListener.ExperitestParent.class)

Here is the code used for ExperitestChild.java:

[java]
package ExperitestListener;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.testng.annotations.Listeners;
import org.testng.annotations.Test;
@Listeners(ExperitestListener.ExperitestParent.class)
public class ExperitestChild { @Test
public void ExperitestTitle()
{
System.setProperty(“webdriver.chrome.driver”, “C:\chromedriver_win32\chromedriver.exe”); WebDriver driver = new ChromeDriver(); driver.get(“https://www.experitest.com”); System.out.print(driver.getTitle()); driver.quit();
} @Test public void ExperitestTitleName() {
System.setProperty(“webdriver.chrome.driver”, “C:\chromedriver_win32\chromedriver.exe”); WebDriver driver = new ChromeDriver(); driver.get(“https://www.experitest.com”); String Actual = driver.getTitle(); String Expected = “Digital.ai”; if (Actual.equals(Expected)) { System.out.println(“Test Passed!”); } else { Assert.assertTrue(false); } driver.close(); }
}
[/java]

When you run the test suite, TestNG listeners listen to the events and react when a test passes or fails.

Here, the test case that returns the Experitest website title passed. The test case that verified the title failed.

TestNG – Mobile Test Automation

Here is another example that runs mobile test automation. Selenium doesn’t support mobile testing. So, you should install a mobile test automation framework such as Appium. However, the configuration and management of Appium involve a steep learning curve. Experitest solves this challenge by offering Appium Studio that leverages Appium but removes all the complexity challenges. Experitest also offers an Appium Studio plugin for Eclipse so that you can run test automation right from your eclipse IDE.

Go to Eclipse Marketplace and search for Appium Studio to install the plugin.

After the plugin is installed, the Appium Studio tab will appear in your Eclipse IdE.

• Now, click on the Windows menu and go to Perspective -> Open Perspective -> Other.
• Choose ‘Mobile’ and click ‘open’.
• The eclipse IDE opens the mobile perspective.

Now, you can run your mobile test automation from your Eclipse IDE. However, you need to choose a SeeTest cloud subscription from Experitest to use real-time mobile devices for test automation projects. You can use the free trial option too.

Now, create a new project and then create a package and two classes in that package. The two classes are named SeeTestparent.java and SeeTest child.java. The SeeTestparent class will contain the listeners while the SeeTestchild contains the test cases.

SeeTestparent.java class code:

[java]
package com.experitest.auto;
import org.testng.ITestContext;
import org.testng.ITestListener;
import org.testng.ITestResult;
public class SeeTestparent implements ITestListener{ @Override public void onTestSuccess(ITestResult expresult) { System.out.print(“Test passed. Details are “+expresult.getName()); } @Override public void onTestFailure(ITestResult expresult) { System.out.print(“Test Failed. Details are”+expresult.getName()); } @Override public void onFinish(ITestContext arg0) { // TODO Auto-generated method stub } @Override public void onStart(ITestContext arg0) { // TODO Auto-generated method stub } @Override public void onTestFailedButWithinSuccessPercentage(ITestResult arg0) { // TODO Auto-generated method stub } @Override public void onTestSkipped(ITestResult arg0) { // TODO Auto-generated method stub } @Override public void onTestStart(ITestResult arg0) { // TODO Auto-generated method stub }
}
[/java]

SeeTestchild.java code:

[java]
package com.experitest.auto;
import io.appium.java_client.ios.IOSDriver;
import io.appium.java_client.ios.IOSElement;
import io.appium.java_client.remote.IOSMobileCapabilityType;
import io.appium.java_client.remote.MobileCapabilityType;
import org.openqa.selenium.By;
import org.openqa.selenium.ScreenOrientation;
import org.openqa.selenium.remote.DesiredCapabilities;
import static org.testng.Assert.assertTrue;
import java.net.MalformedURLException;
import java.net.URL;
import org.testng.Assert;
import org.testng.annotations.*;
@Listeners(com.experitest.auto.SeeTestparent.class)
public class SeeTestchild { private String host = “https://cloud.SeeTest.io”; private String accessKey = “enter your access key here”; protected IOSDriver driver = null; DesiredCapabilities dc = new DesiredCapabilities(); @BeforeTest public void setUp1() throws MalformedURLException { dc.setCapability(“testName”, “Quick Start iOS Native Demo”); dc.setCapability(“accessKey”, accessKey); dc.setCapability(“deviceQuery”, “@os=’ios’ and @category=’PHONE'”); dc.setCapability(MobileCapabilityType.APP, “cloud:com.experitest.ExperiBank”); dc.setCapability(IOSMobileCapabilityType.BUNDLE_ID, “com.experitest.ExperiBank”); driver = new IOSDriver<>(new URL(“https://cloud.SeeTest.io/wd/hub”), dc); } @Test public void SeeTestDemo1() { driver.rotate(ScreenOrientation.PORTRAIT); driver.findElement(By.xpath(“//*[@id=’usernameTextField’]”)).sendKeys(“company”); driver.hideKeyboard(); driver.findElement(By.xpath(“//*[@id=’passwordTextField’]”)).sendKeys(“company”); driver.findElement(By.xpath(“//*[@id=’loginButton’]”)).click(); driver.findElement(By.xpath(“//*[@id=’makePaymentButton’]”)).click(); driver.findElement(By.xpath(“//*[@id=’phoneTextField’]”)).sendKeys(“0541234567”); driver.findElement(By.xpath(“//*[@id=’nameTextField’]”)).sendKeys(“Jon Snow”); driver.findElement(By.xpath(“//*[@id=’amountTextField’]”)).sendKeys(“50”); driver.findElement(By.xpath(“//*[@id=’countryButton’]”)).click(); driver.findElement(By.xpath(“//*[@id=’Switzerland’]”)).click(); driver.findElement(By.xpath(“//*[@id=’sendPaymentButton’]”)).click(); driver.findElement(By.xpath(“//*[@id=’Yes’]”)).click(); driver.findElement(By.xpath(“//*[@id=’logoutButton’]”)).click(); } @Test public void SeeTestDemo2() { Assert.assertTrue(false); } @AfterTest public void tearDown1() { driver.quit(); } }
[/java]

Here is the report. Out of the two tests, one test has failed.

The reports shows the failed test details here:

Here, you can see the TestNG Listener reporting the details of the test passed and the test failed.

Conclusion

TestNG Listener is a handy feature for testing and QA teams to easily automate testing projects and easily identify how each is performing. For a failed test or a skipped test, you can take the screenshot and identify why the test failed or skipped.

SeeTest cloud enables you to leverage TestNG listeners to easily run your test automation projects from your favorite IDE while being able to customize reports and logs for realtime reporting and analytics purposes. See more from Digital.ai Continuous Testing.

Guy Arieli – CTO