How to Set Up and Run Selenium Tests, Using Various Builders and Java Test Frameworks

Suppose that you need to write Selenium tests. Your plan is to write them in Java. First, you will need to choose the test framework you will use for the code of your tests. Next, you will need a build automation tool for your project. Maybe you already know the framework and builder you will use, but you don’t know how to configure them for your project.

In this article, we are going to consider TestNG and JUnit. Both are popular Java test frameworks. We will also take a look at some popular builders, such as Apache Maven, Gradle, and Apache Ant. We selected TestNG and JUnit, as they are the most popular Java test frameworks. Although it is true that TestNG has an advantage over JUnit, JUnit is more often used in software development projects. We will demonstrate how to create and run Selenium tests, using these frameworks and builders.

Before you start writing tests, and using some frameworks and build tools, it’s necessary to make sure that JDK is installed on your workstation and to check the browser you have installed. You need to install an IDE and a browser driver, which corresponds to a browser, in which you will execute your tests. You may need to make or change some environmental settings.

If Java JDK is not installed on the workstation, where you are going to develop and run tests, you need to install it. At the time of writing this article, the latest available version of Java is 11.0.2. You can download it here After installation you should set Java_HOME as an environment variable. For example:

set Java_HOME=C:\Program Files\Java\jdk-11.0.2

In the examples demonstrated in this article, we will use the IntelliJ IDEA. It’s enough to install the latest Community version. The dropdown menu enables you to choose an installation package for a certain Java version. Since you have installed JDK 11, select the IDEA version for Java 11. https://www.jetbrains.com/idea/. If you have another version of JDK installed on your workstation, select the appropriate IDEA package. The way to do it is shown in the picture below.

Install the driver for the corresponding type and version of the browser, you are going to use for tests. In the examples presented in this article, we use the Google Chrome browser. Drivers for Google Chrome can be downloaded here. Unzip it inside a directory of your choice.

After the basic set of tools is installed and configured we may get down to writing demo tests.

To demonstrate the usage of a test framework with a build tool we need an example of some tests. In the example test, we will open the Experitest website, find the link “Webcasts” on the experitest home page, click on it and then check the title of the page, which will then open up. The example test is represented by 2 files: TestBase.Java and ViewTest.Java. The first file is the class responsible for the test preparation, starting browser driver, and its configuration. The second file is the class, which contains the test itself. Here is the code:

[java] public class TestBase { protected static WebDriver driver; @BeforeClass public static void setUp(){ System.setProperty(“webdriver.chrome.driver”, “D:\\chrome-driver-73\\chromedriver.exe”); driver = new ChromeDriver(); driver.manage().timeouts().implicitlyWait(10, TimeUnit.SECONDS); } @AfterClass public static void tearDown(){ driver.close(); driver.quit(); } } System.setProperty(“webdriver.chrome.driver”, “D:\\chrome-driver-73\\chromedriver.exe”); – This method sets path to the chrome driver on your workstation. The contents of the ViewTest.Java class: public class ViewTest extends TestBase { @Test public void test() { driver.get(“https://experitest.com/”); WebElement linkToWebinars = driver.findElement(By.linkText(“Webcasts”)); linkToWebinars.click(); WebElement headerForWebinar = driver.findElement(By.xpath(“//div[@class=’papers2 image’]/p[@class=’title1′]”)); Assert.assertEquals(headerForWebinar.getText(), “From Tester to Developer: Tips and Insights”); } } [/java]

Files have annotations @BeforeClass, @AfterClass, and @Test annotations, that both TestNG and JUnit test frameworks include. The Assert.assertEquals method is also used in both frameworks and is included in the test, though it has a different order of parameters in these frameworks. In the Assert.assertEquals method of the TestNG framework, the asserted parameter comes first and represents the actual value. The expected value is the second parameter. On the contrary, the Assert.assertEquals method of JUnit framework uses the reverse order of actual and expected values parameters: expected value goes as the first parameter and the actual value as the second:

  [java]Assert.assertEquals(“From Tester to Developer: Tips and Insights”, headerForWebinar.getText());[/java]

In the following sections, we’ll demonstrate how to create and run tests with a certain framework and builder chosen for this article. We assume ahead of time that you already have JDK, IDEA and a browser driver installed and configured.

TestNG framework and Apache Maven

Apache Maven is a dependency management and build automation tool. Maven relies on the conventions, provides predefined commands and prescribes strict project structure. The configuration file of Maven is pom.xml.

Step 1

Install the Apache Maven tool for building Java project. In order to do it download the latest version of maven. At the time of writing this article, the latest available version of Maven was 3.6.0. Unzip the archive in any directory. Set environment variables:

  [java] set M2_HOME=C:\apache-maven-3.6.0 set PATH=%M2_HOME%\bin. [/java]

Step 2

Create a new Maven project in Idea IDE, add the information about groupID, artifactID and the version of the created project. Specify the project’s name and location. After these steps, the project structure will be created and pom.xml file, which controls the build process will be placed into the project root. The screenshot below displays the project structure and contents of the project build file pom.xml. It has parameters, defined during the project creation flow. Actually, the dependencies have to be defined in the build file in order to start working with automated tests and using a test framework. Besides dependencies, it’s necessary to define other artifacts. All parameters, artifacts, and dependencies are added in the corresponding sections of the pom.xml file.

Step 3

Specify the list of libraries to create tests. Add the main plugin – the compiler. It is used by default and there is no need to specify it, but you can specify its version. The contents of the plugin section of the build file are shown here.

  [html] <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-compiler-plugin</artifactId> <version>3.8.0</version> </plugin> [/html]

In order to work with the TestNG framework, it’s necessary to add its definition in the dependencies section of the pom.xml file. The dependency section of the pom.xml file for the connection TestNG framework to the project looks as it’s shown here.

[html 1=”https://mvnrepository.com/artifact/org.testng/testng” 2=”–>” 3=”<dependency>” 4=”<groupId>org.testng</groupId>” 5=”<artifactId>testng</artifactId>” 6=”<version>6.14.3</version>” 7=”</dependency>” 8=”[/html” language=””] &nbsp; <p class=”abstract”>The groupId, artifactId and version parameters can be found in the <a href=”https://search.maven.org/”>maven central repository</a> Just perform search for TestNG in it, select latest version of TestNG and the dependency section with these parameters will be displayed. Copy and paste them into the pom.xml file of your project.</p> &nbsp; <p class=”abstract”>In the same way, the selenium-api library is connected to your project. <a href=”https://github.com/jonnysteiner/How-to-set-up-and-run-Selenium-tests-using-various-builders-and-Java-test-frameworks/commit/9748889d6f039aa6eb52ae5d3dc38d43645c5018″>The dependency section for connection to the selenium is shown here.</a></p> &nbsp; [html] <!– https://mvnrepository.com/artifact/org.seleniumhq.selenium/selenium-Java –> <dependency> <groupId>org.seleniumhq.selenium</groupId> <artifactId>selenium-api</artifactId> <version>3.14.0</version> </dependency> [/html]

For working on WebDriver with Chrome browser the selenium-chrome-driver library is necessary. Add it to the dependency section of pom.xml file.

[html] <!– https://mvnrepository.com/artifact/org.seleniumhq.selenium/selenium-Java –> <dependency> <groupId>org.seleniumhq.selenium</groupId> <artifactId>selenium-chrome-driver</artifactId> <version>3.14.0</version> </dependency> [/html]

Besides maven central, there’s Maven main repository you can get all necessary information from. The pom.xml file configured for our project is shown below.

  [html] <project xmlns=”http://maven.apache.org/POM/4.0.0″ xmlns:xsi=”http://www.w3.org/2001/XMLSchema-instance” xsi:schemaLocation=”http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd”> <modelVersion>4.0.0</modelVersion> <groupId>Template</groupId> <artifactId>Template</artifactId> <version>1.0-SNAPSHOT</version> <dependencies> <!– https://mvnrepository.com/artifact/org.seleniumhq.selenium/selenium-Java –> <dependency> <groupId>org.seleniumhq.selenium</groupId> <artifactId>selenium-api</artifactId> <version>3.14.0</version> </dependency> <!– https://mvnrepository.com/artifact/org.testng/testng –> <dependency> <groupId>org.testng</groupId> <artifactId>testng</artifactId> <version>6.14.3</version> </dependency> <dependency> <groupId>org.seleniumhq.selenium</groupId> <artifactId>selenium-chrome-driver</artifactId> <version>3.14.0</version> </dependency> </dependencies> <build> <plugins> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-compiler-plugin</artifactId> <version>3.6.0</version> </plugin> </plugins> </build> </project> [/html]

All libraries, that are defined in the dependency section of the pom.xml file are supposed to be downloaded from the maven repository. In order to pull the necessary libraries, you need to update your project.

To do that, click on the Re-import button in the Maven Projects tool window of IDEA IDE. All the specified dependencies will be downloaded and during the test creation, the required libraries will be accessed via Java import directives.

Step 4

Add your tests to the test.Java folder. In this article, we added test examples, the code of which is shown in the first paragraph.

Step 5

If you plan to run all tests sequentially, you can skip this step. But if you plan to manage the test suite afterward, then you need to use the testng.xml file. Add the resources subdirectory to the test directory and create a testng.xml file inside of it.

Specify in your testng.xml file the tests you are going to launch. In our case, according to the example test, there’s one suite with one test class. The contents of the test.xml file for the example test is shown here.

  [html] <!DOCTYPE suite SYSTEM “https://testng.org/testng-1.0.dtd”> <!–Suite–> <suite name=”Suite1″> <!–Test–> <test name=”My Example test” > <!–Test Class–> <classes> <class name=”SearchTest” /> </classes> </test> </suite> [/html]

Since we define tests for launch in the test.xml file, it’s necessary to furnish Maven with information about it. Add the following lines to pom.xml to the plugin section. The contents of the corresponding plugin section is shown here.

  [html] <plugins> … <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-surefire-plugin</artifactId> <version>3.0.0-M3</version> <configuration> <suiteXmlFiles> <suiteXmlfile>src/test/resources/testng.xml</suiteXmlfile> </suiteXmlFiles> </configuration> </plugin> … </plugins> [/html]

Step 6

Now the tests can be executed in the IDE console using Maven. Enter the mvn clean test command. The parameter clean instructs Maven to remove all previous builds, if any exist, the parameter test instructs Maven to run tests.

If you followed the instructions, your example test will be successfully executed. The result of the execution is shown in the picture below.

JUnit framework and Apache Maven

The steps for configuring the JUnit framework with Apache Maven for projects are nearly the same, as for the case, when the TestNG framework is used. The only difference is in step 3 when test framework dependencies are added to the pom.xml file. For JUnit dependencies section of the build file will be here.

  [html] <!– https://mvnrepository.com/artifact/junit/junit –> <dependency> <groupId>junit</groupId> <artifactId>junit</artifactId> <version>4.12</version> <scope>test</scope> </dependency> [/html]

Enter the mvn clean test command in the terminal window and launch tests from the project. The results of the execution will be the one shown on the screenshot below.

JUnit + Gradle:

Gradle is a dependency management and build automation tool which was built upon the concepts of Ant and Maven. Gradle is that it’s not using XML files, unlike Ant or Maven. Gradle is using a domain-specific language (DSL), based on Groovy. This leads to smaller configuration files. The configuration file of Gradle is by convention called build.gradle.

We assume that JDK, IDEA, and browser drivers are both installed and configured on your workstation.

Step 1

Install Gradle. The current Gradle release is 5.4

Download the latest version from https://gradle.org/releases/.

Unzip the archive, for example to the local disc C.

Set the next environment variables:

GRADLE_HOME=C:\gradle-5.4,

PATH=%GRADLE_HOME%\bin

Step 2

Create a Gradle project in IDEA IDE, select Gradle builder, specify the SDK version and additional libraries and frameworks. Add information about groupID, artifactID, and the version of the created project. Define Gradle options. Specify the project name and location. After these steps are completed, the project structure will be created and gradle.build file, which controls the build process will be placed to the project root. The screenshot below displays the project structure and contents of the project build file – build.gradle. Actually, the dependencies have to be defined in the build file in order to start working with automated tests and using the test framework. Besides dependencies, it’s necessary to define other artifacts. All parameters, artifacts, and dependencies are added in the corresponding sections of the build.gradle file.

Step 3

By default, Gradle has reference to the JUnit framework in the dependency section. You can see it on the screenshot above. In order to add references to the selenium-api library and selenium-chrome-driver library in the Gradle dependency section, we have to search for these libraries in the maven central repository. Then in the search results, we have to select the reference for Gradle builder. The reference for selenium-API in the dependency section will look like this:

[java] //https://mvnrepository.com/artifact/org.seleniumhq.selenium/selenium-api compile group: ‘org.seleniumhq.selenium’, name: ‘selenium-api’, version: ‘3.14.0’ The reference to the selenium-chrome-driver library will look like: //https://mvnrepository.com/artifact/org.seleniumhq.selenium/selenium-chrome-driver compile group: ‘org.seleniumhq.selenium’, name: ‘selenium-chrome-driver’, version: ‘3.14.0’ [/java]

After all of the dependencies are configured, update the project by clicking on the refresh button in the Gradle Toolbar, before declaring referred classes in import directives.

Add your tests to the test.Java folder. In order to run the test, click Execute Gradle Task button on the Gradle toolbar.

In the Command line field of the dialog window which appears after, define the task to be performed and click on the OK button. The test will start and the log of the successful test will print out to console and look like it’s shown on the picture below.

TestNG + Gradle:

The steps for configuration TestNG framework with Gradle builder for projects are nearly the same, as for the case, when the JUnit framework is used. The only difference is at step 3 when test framework dependencies are added to the build.gradle file. For connection to TestNG, it’s necessary to replace the reference to JUnit with the reference to the TestNG framework.

Add a reference to the TestNG Framework, as it’s shown below. This string refers to the latest version of the TestNG framework in the maven repository.

  [java] // https://mvnrepository.com/artifact/org.testng/testng testCompile group: ‘org.testng’, name: ‘testng’, version: ‘6.14.3’ [/java]

Add your tests to the test.Java folder. If you need to manage your tests, create testng.xml file. Add it to src/test/resources/ folder. The example structure of testng.xml file is shown here.

  [html] <!DOCTYPE suite SYSTEM “https://testng.org/testng-1.0.dtd” > <suite name=”Suite1″ verbose=”1″ > <test name=”Nopackage” > <classes> <class name=”SearchTest” /> </classes> </test> </suite> [/html]

In order that Gradle searches and runs testng after compilation, you need to add the test task at the end of the file and specify where the testng.xml is.

In order to run the test, click the Execute Gradle Task button on the Gradle toolbar. In the Command line field of the dialog window which appears after, define the task to be executed and click on the OK button. The test starts and the log of the successful test is output to console and looks like it’s shown in the picture below.

TestNG + Apache Ant

Apache Ant (“Another Neat Tool”) is a Java library used for automating build processes for Java applications. Additionally, Ant can be used for building non-Java applications. Ant build files are written in XML, and by convention, they’re called build.xml. The main benefit of Ant is its flexibility. Ant doesn’t impose any coding conventions or project structures.

We suppose, that JDK, IDEA, and browser driver are installed and configured.

Step 1

Install Apache Ant. It can be downloaded from the official site. Download the Apache Ant 1.10.5, because it supports Java 8 and above. Unzip the downloaded archive, for example to your local disc C. Set the next environment variable:

set ANT_HOME=C:\apache-ant-1.10.5

set PATH=%Java_HOME%\bin;%ANT_HOME%\bin

Step 2

Create a new project in IDE. Since IDEA does not offer a choice of Ant project, create a simple Java project. Skip project creation from a project template. Select the project name and location. After the configuration steps the project in IDE will look like the screenshot below.

In order to form the structure of the project, add subfolders test.Java and test.resources

Step 3

Then you need to add corresponding libraries. For this download libraries and put them to the lib folder and connect to the project. Create lib folder in your project, download libraries to the lib folder from the main Maven repository:

common-exec-1.3.jar

guava-27.0.1-jre.jar – set of useful libraries.

jcommander-1.72.jar – the library for parsing the command line.

okhttp-3.14.1.jar – the library that Selenium uses to work with driver and server.

okio-1.17.3.jar – one more additional library which complements Java.io and Java.nio.

selenium-api-3.14.0.jar – the library for the creation of Selenium WebDriver scripts.

selenium-chrome-driver-3.14.0.jar – the library, that Selenium uses to work with Google Chrome web-driver.

selenium-remote-driver-3.14.0.jar – the library for the creation of Selenium WebDriver scripts.

testng-6.14.2.jar – TestNG framework.

After all of the libraries are added, the contents of the lib folder will look like the following:

After that these libraries should be connected to the project. In Project Settings of IDEA IDE choose Modules entry. On the Dependencies tab, add path to folder with libraries.

Step 4

Create build file build.xml in the root folder of the project. Define tasks: clean, compile, test. The example of the build.xml file with defined tasks is shown below.

  [html] <?xml version=”1.0″ encoding=”UTF-8″?> <project name=”TestNGAnt” default=”testng”> <!– Set global properties for this build –> <dirname property=”basedir” file=”${ant.file}”/> <property name=”test.classes.dir” value=”${basedir}/out”/> <property name=”report.dir” location=”report” /> <!– Set libraries –> <path id=”library.lib.classpath”> <fileset dir=”${basedir}/lib”/> </path> <!– Property sources –> <path id=”test.source.dir”> <dirset dir=”${basedir}”> <include name=”src”/> </dirset> </path> <!– Register Custom Compiler Taskdefs –> <taskdef name=”testng” classname=”org.testng.TestNGAntTask”> <classpath location=”lib/testng-6.14.2.jar” /> </taskdef> <patternset id=”compiler.resources”> <exclude name=”**/?*.Java”/> </patternset> <!– Clean task –> <target name=”clean” description=”cleanup task”> <delete dir=”${test.classes.dir}”/> <delete dir=”${report.dir}”/> </target> <!– Compile –> <target name=”compile” description=”Compile module TestNGAntTemplate”> <mkdir dir=”${test.classes.dir}”/> <Javac destdir=”${test.classes.dir}”> <classpath refid=”library.lib.classpath”/> <src refid=”test.source.dir”/> </Javac> <copy todir=”${test.classes.dir}”> <fileset dir=”${basedir}/src”> <patternset refid=”compiler.resources”/> <type type=”file”/> </fileset> </copy> </target> </project> [/html]

Step 5

Add your tests to the test.Java folder.

Step 6

Create a file to manage your tests. Add testng.xml to src/test/resources/ folder.

  [html] <!DOCTYPE suite SYSTEM “https://testng.org/testng-1.0.dtd” > <suite name=”Suite1″ verbose=”1″ > <test name=”Nopackage” > <classes> <class name=”SearchTest” /> </classes> </test> </suite> Add rules for building and running tests to the build.xml file : <!– Run testng –> <target name=”testng” depends=”clean, compile” description=”build and run tests”> <testng classpathref=”library.lib.classpath” outputdir=”${report.dir}” haltonfailure=”true”> <classpath location=”${test.classes.dir}” /> <xmlfileset dir=”${test.classes.dir}/test/resources” includes=”testng.xml” /> </testng> </target> [/html]

Step 7

After that, add the build file to the Ant Build Tool:

Select the task and run it:

On the screenshot below there’s the task execution result.

JUnit + Apache Ant:

The steps for a Java test project creation and for its execution are the same as the ones, that were described above. The difference is at the first step. JUnit library is downloaded and installed instead of TestNG:

junit-4.12.jar – JUnit framework.

hamcrest-all-1.3.jar – an additional library that is necessary for JUnit work.

The build.xml file for JUnit framework will look in the following way.

  [html] <?xml version=”1.0″ encoding=”UTF-8″?> <project name=”JUnitAntTemplate” default=”junit”> <!– Set global properties for this build –> <dirname property=”basedir” file=”${ant.file}”/> <property name=”test.classes.dir” value=”${basedir}/out”/> <!– Set libraries –> <path id=”library.lib.classpath”> <fileset dir=”${basedir}/lib”/> </path> <!– Property sources –> <path id=”test.source.dir”> <dirset dir=”${basedir}”> <include name=”src”/> </dirset> </path> <path id = “classpath.test”> <pathelement location = “${test.classes.dir}” /> </path> <patternset id=”compiler.resources”> <exclude name=”**/?*.Java”/> </patternset> <!– Clean task –> <target name=”clean” description=”cleanup task”> <delete dir=”${test.classes.dir}”/> </target> <!– Compile –> <target name=”compile” depends=”clean” description=”Compile module JUnitAntTemplate”> <mkdir dir=”${test.classes.dir}”/> <Javac destdir=”${test.classes.dir}” modulepath=”${basedir}” includeantruntime=”true”> <classpath refid=”library.lib.classpath”/> <src refid=”test.source.dir”/> </Javac> <copy todir=”${test.classes.dir}”> <fileset dir=”${basedir}/src”> <patternset refid=”compiler.resources”/> <type type=”file”/> </fileset> </copy> </target> </project> The rules for building and running tests are added to the build.xml file in the following way: <!– Run JUnit test –> <target name=”junit” depends=”clean, compile” > <junit> <classpath> <path refid=”library.lib.classpath”/> <path refid=”classpath.test”/> </classpath> <batchtest fork=”yes”> <fileset dir=”src” includes=”**/*Test.Java”/> </batchtest> </junit> </target> [/html]

Add the build file to Ant Build Tool. Select JUnit task and run it.

The log of compilation and running tests is shown in the picture below.

In this article, we have discussed and described how to use JUnit and TestNG Java frameworks with popular builder tools. Now you know how to start your automation project and successfully accomplish it.