Table of contents
- What is test Priority in TestNG?
- Syntax for using test Priority in TestNG
- What is default Priority in TestNG?
- How to set negative Priority in TestNG?
- How to test with the same Priority in TestNG?
- How to set test case Priority in TestNG using Selenium?
- Implementing test case Priority when methods are without Priority in TestNG
- Implementing test case Priority when methods are with Priority in TestNG
- Implementing test case Priority when methods are with the same Priority in TestNG
- Implementing test case Priority with both Prioritized and Non-Prioritized methods in TestNG
- TestNG issue when running tests with same Priority and in Parallel
- Conclusion
TestNG is an open-source and powerful test automation framework with a wide range of features, including annotations, data providers, group testing, prioritization, and much more.
Prioritization in TestNG is an easy way to set test case priority and run your test case in the exact order you want. This feature is particularly useful when defining a sequence for the test case execution when you have multiple test cases or assign precedence to a few methods over others. For example, if you want your login scenario to run first, followed by adding an item to your cart, you don’t need to worry as TestNG has got you all covered.
](https://cdn-images-1.medium.com/max/2000/1*haNpmKo6wzLnZtAkF1Xf9A.gif)
In our previous blogs of this Selenium TestNG tutorial series, we have learned how to install TestNG, how to use annotations in TestNG, and much more. In this blog, we will cover how to set test case priority in TestNG while performing Selenium automation testing.
Let’s get started!
Note: MD2 Hash Calculator —Free online tool that computes MD2 hash of text and strings.
What is test Priority in TestNG?
In TestNG, Priority is an attribute that helps the users define the order in which they want the test cases to be executed.
When you have multiple test cases and want them to run in a particular order, you can use the Priority attribute to set test priority in TestNG. The test cases get executed following an ascending order in the priority list, and hence, test cases with lower priority will always get executed first.
Syntax for using test Priority in TestNG
To set test case priority in TestNG, we need to add annotation as @ Test (priority=X). In the below-shown example, we have given a priority of 1 to the test case.
@Test(priority = 1)
public void myTestCaseWithPriority() {
try {
System.out.println("I am in my first testcase with priority=1");
} catch (Exception e) {
}
}
What is default Priority in TestNG?
The default test priority in TestNG is 0. So, when you don’t set test priority in TestNG explicitly to the test cases, TestNG assigns them a priority of 0.
For example, the below-shown test case doesn’t have any priority set, and hence, it assumes a priority of 0.
@Test
public void myTestCaseWithDefaultPriority() {
try {
System.out.println("I am in my first testcase with default priority");
} catch (Exception e) {
}
}
With TestNG certification, you can challenge your skills in performing automated testing with TestNG and take your career to the next level.
Here’s a short glimpse of the TestNG certification from LambdaTest:
How to set negative Priority in TestNG?
If you wonder whether we can assign a negative priority value to any test case, the answer is Yes!! We can assign a negative priority to the test methods when we want them to take precedence over the default ones.
The below-shown example has two test methods, one with a default priority 0 and the other with negative priority, i.e., -1. Hence, the test method with negative priority will always be executed first, followed by the test method with default priority.
@Test
public void myTestCaseWithDefaultPriority() {
try {
System.out.println("I am in my first testcase with default priority");
} catch (Exception e) {
}
}
@Test(priority = -1)
public void myTestCaseWithNegativePriority() {
try {
System.out.println("I am in my first testcase with negative priority");
} catch (Exception e) {
}
}
Console Output:
How to test with the same Priority in TestNG?
You must wonder what happens if you have a test case with all or some of the methods having the same priority. In such cases, TestNG follows an alphabetical order while executing them.
The below example has two test methods (a and b), and both have the same default priority that is 0. Therefore, the order of execution will be method a followed by b.
@Test
public void a() {
try {
System.out.println("A");
} catch (Exception e) {
}
}
@Test
public void b() {
try {
System.out.println("B");
} catch (Exception e) {
}
}
Console Output:
How to set test case Priority in TestNG using Selenium?
In this blog on how to set test case priority in TestNG, we have learned the fundamentals of priority in TestNG. In this section of the Selenium TestNG tutorial, we will see how test priority in TestNG works or how to set test case priority in TestNG.
Problem Scenario:
We will automate the main page of Selenium Playground offered by LamdaTest and print the headers of all the sections sequentially.
LambdaTest is an automated cross browser testing platform. With over 2000 online browser and OS combos, it lets you manage your manual and automated tests in one place with ease.
Implementation:
We will implement the problem scenario in four different ways, as listed below:
When all the methods are without priority
When all the methods are with priority
When all the methods are with the same priority
When there are both methods with and without priority
If you are getting started with running TestNG scripts using Selenium WebDriver, you can refer to our blog on Create TestNG Project In Eclipse & Run Selenium Test Script.
Note: Lines Count — Free online tool to calculate the number of lines in any given text.
Implementing test case Priority when methods are without Priority in TestNG
In this section of the Selenium TestNG tutorial, we will see how to set test case priority in TestNG when all the methods are without any priority using a sample Selenium automation code.
package LamdaTest;
import org.openqa.selenium.By;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.remote.DesiredCapabilities;
import org.openqa.selenium.remote.RemoteWebDriver;
import org.testng.Reporter;
import org.testng.annotations.AfterClass;
import org.testng.annotations.BeforeClass;
import org.testng.annotations.Test;
import java.net.MalformedURLException;
import java.net.URL;
class SeleniumPlaygroundTestsWithoutPriority {
public String username = "YOUR USERNAME";
public String accesskey = "YOUR ACCESSKEY";
public static RemoteWebDriver driver = null;
public String gridURL = "@hub.lambdatest.com/wd/hub";
@BeforeClass
public void setUp() throws Exception {
DesiredCapabilities capabilities = new DesiredCapabilities();
capabilities.setCapability("browserName", "chrome");
capabilities.setCapability("version", "94.0");
capabilities.setCapability("platform", "win10"); // If this cap isn't specified, it will just get the any available one
capabilities.setCapability("build", "TestNGWithoutPriorityTests");
capabilities.setCapability("name", "TestNGWithoutPriorityTestsSample");
try {
driver = new RemoteWebDriver(new URL("https://" + username + ":" + accesskey + gridURL), capabilities);
} catch (MalformedURLException e) {
System.out.println("Invalid grid URL");
} catch (Exception e) {
System.out.println(e.getMessage());
}
Reporter.log("Logging into Selenium Playground", true);
driver.get("http://labs.lambdatest.com/selenium-playground/");
}
@Test
public void getFirstOptionName() {
try {
WebElement option1 = driver.findElement(By.xpath("(//div[@class='contentlearn']//h2)[1]"));
Reporter.log("The First Option Is: " + option1.getText(), true);
} catch (Exception e) {
}
}
@Test
public void getSecondOptionName() {
try {
WebElement option2 = driver.findElement(By.xpath("(//div[@class='contentlearn']//h2)[2]"));
Reporter.log("The Second Option Is: " + option2.getText(), true);
} catch (Exception e) {
}
}
@Test
public void getThirdOptionName() {
try {
WebElement option3 = driver.findElement(By.xpath("(//div[@class='contentlearn']//h2)[3]"));
Reporter.log("The Third Option Is: " + option3.getText(), true);
} catch (Exception e) {
}
}
@Test
public void getFourthOptionName() {
try {
WebElement option4 = driver.findElement(By.xpath("(//div[@class='contentlearn']//h2)[4]"));
Reporter.log("The Fourth Option Is: " + option4.getText(), true);
} catch (Exception e) {
}
}
@Test
public void getFifthOptionName() {
try {
WebElement option5 = driver.findElement(By.xpath("(//div[@class='contentlearn']//h2)[5]"));
Reporter.log("The Fifth Option Is: " + option5.getText(), true);
} catch (Exception e) {
}
}
@Test
public void getSixthOptionName() {
try {
WebElement option6 = driver.findElement(By.xpath("(//div[@class='contentlearn']//h2)[6]"));
Reporter.log("The Sixth Option Is: " + option6.getText(), true);
} catch (Exception e) {
}
}
@AfterClass
public void closeBrowser() {
driver.close();
Reporter.log("Closing the browser", true);
}
}
You can use the below testng.xml file for running the above test class.
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="TestNGPriorityTest">
<test name="TestNGPriorityTest" >
<classes>
<class name="LamdaTest.SeleniumPlaygroundTestsWithoutPriority">
</class>
</classes>
</test>
</suite>
The below pom.xml can be used for installing all the necessary dependencies.
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>org.example</groupId>
<artifactId>LambdaTest</artifactId>
<version>1.0-SNAPSHOT</version>
<dependencies>
<dependency>
<groupId>org.seleniumhq.selenium</groupId>
<artifactId>selenium-api</artifactId>
<version>4.0.0-alpha-7</version>
</dependency>
<dependency>
<groupId>org.testng</groupId>
<artifactId>testng</artifactId>
<version>6.14.3</version>
</dependency>
<dependency>
<groupId>org.seleniumhq.selenium</groupId>
<artifactId>selenium-remote-driver</artifactId>
<version>4.0.0-alpha-7</version>
</dependency>
</dependencies>
<properties>
<maven.compiler.source>8</maven.compiler.source>
<maven.compiler.target>8</maven.compiler.target>
</properties>
</project>
Code Walkthrough:
- Imported Dependencies: Here, we have imported all the necessary classes, including the Desired Capabilities, Reporter Class, etc., for using the corresponding methods.
Reporter Class is an inbuilt class in TestNG and is used for logging messages in the HTML Reports and Standard Output.
You can refer to How to Use TestNG Reporter Class In Selenium that deep dives into the details of Reporter Class.
2. Global Variables: Here, we have used an online Selenium Grid like LamdaTest to perform parallel testing.
Here, you can populate the values for your corresponding username and access key, which can be collected by logging into your LamdaTest Profile Section. You can copy the Username and the Access Token to be used in the code. However, the grid URL will remain the same, as shown below.
3. @ BeforeClass(Setup Method): Here, we have used the LamdaTest Desired Capabilities Generator and have set the necessary capabilities of browser name, version, platform, etc., for our Selenium Remote Web Driver.
Do refer to our detailed blog that deep dives into the important TestNG annotations that are used for web automation testing.
Along with setting the desired capabilities, we are logging into the Selenium Playground in the Before Class Method to use the platform for all the following test methods by logging in once.
We are also logging the text message “Logging into Selenium Playground” in our HTML Report as well as standard output by making use of the Reporter Class Method.
4. @ Test(getFirstOptionName): In this method, we are getting the Web Element of the header of the first section of the Selenium Playground highlighted below. Then, we are printing the text of the same Web Element, using Reporter Class on the console and HTML Report. Here, the text of the web element printed on the console will be Input Forms.
Since there is no priority explicitly set, the test method will assume the default priority of 0.
Here, we have made use of XPath in our test case for all the web elements. You can get the XPath of any element by simply doing a right-click on the Element→ Inspect Option. You can also refer to the complete guide on XPath in Selenium for understanding how you can make the most out of XPath for accessing WebElements in the DOM.
5. @ Test(getSecondOptionName): In this method, we are getting the Web Element of the header of the second section of the Selenium Playground. Then, we are printing the text of the same Web Element, using Reporter Class on the console and HTML Report. Here, the text of the web element printed on the console will be Progress Bars & Sliders. The test method here again assumes the default priority of 0.
6. @ Test(getThirdOptionName): In this method, we are getting the Web Element of the header of the third section of the Selenium Playground. Then, we are printing the text of the same Web Element, using Reporter Class on the console and HTML Report. Here, the text of the web element printed on the console will be Alerts & Modals. The test method assumes the default priority of 0.
7. @ Test(getFourthOptionName): In this method, we are getting the Web Element of the header of the fourth section of the Selenium Playground. Then, we are printing the text of the same Web Element, using Reporter Class on the console and HTML Report. Here, the text of the web element printed on the console will be Date Pickers.
8. @ Test(getFifthOptionName): In this method, we are getting the Web Element of the header of the fifth section of the Selenium Playground. Then, we are printing the text of the same Web Element, using Reporter Class on the console and HTML Report. Here, the text of the web element printed on the console will be Table.
9. @ Test(getSixthOptionName): In this method, we are getting the Web Element of the header of the sixth section of the Selenium Playground. Then, we are printing the text of the same Web Element, using Reporter Class on the console and HTML Report. Here, the text of the web element printed on the console will be List Box.
10. @ AfterClass(closeBrowser): In this method, we are simply closing the browser launched before class once all the test methods are executed.
Execution Results:
Since no priority was explicitly mentioned for all the above-shown test methods, TestNG will assign them a default priority of 0. Now, as explained in Tests With Same Priority In TestNG, to tie break between all these methods having the same priority, TestNG will follow alphabetical order while executing them. As a result, console output will look something like this:
You can further check the execution results of your test published at LambdaTest Automation Dashboard.
While your test is running, you can also see the live video streaming of your tests and various other details such as logs or exceptions raised.
Note: MD5 Hash Calculator — Free online tool to generate MD5 Hash values.
Implementing test case Priority when methods are with Priority in TestNG
In the previous section of this blog on how to set test case priority in TestNG, you saw the test methods getting executed alphabetically since they had a default priority assigned. Now, let’s see how the same test case would behave when we explicitly assign priority to the test methods while performing Selenium automation testing.
package LamdaTest;
import org.openqa.selenium.By;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.remote.DesiredCapabilities;
import org.openqa.selenium.remote.RemoteWebDriver;
import org.testng.Reporter;
import org.testng.annotations.AfterClass;
import org.testng.annotations.BeforeClass;
import org.testng.annotations.Test;
import java.net.MalformedURLException;
import java.net.URL;
class SeleniumPlaygroundTestsWithPriority {
public String username = "YOUR USERNAME";
public String accesskey = "YOUR ACCESSKEY";
public static RemoteWebDriver driver = null;
public String gridURL = "@hub.lambdatest.com/wd/hub";
@BeforeClass
public void setUp() throws Exception {
DesiredCapabilities capabilities = new DesiredCapabilities();
capabilities.setCapability("browserName", "chrome");
capabilities.setCapability("version", "93.0");
capabilities.setCapability("platform", "win10"); // If this cap isn't specified, it will just get the any available one
capabilities.setCapability("build", "TestNGWithPriorityTests");
capabilities.setCapability("name", "TestNGWithPriorityTestsSample");
try {
driver = new RemoteWebDriver(new URL("https://" + username + ":" + accesskey + gridURL), capabilities);
} catch (MalformedURLException e) {
System.out.println("Invalid grid URL");
} catch (Exception e) {
System.out.println(e.getMessage());
}
Reporter.log("Logging into Selenium Playground", true);
driver.get("http://labs.lambdatest.com/selenium-playground/");
}
@Test(priority = 1)
public void getFirstOptionName() {
try {
WebElement option1 = driver.findElement(By.xpath("(//div[@class='contentlearn']//h2)[1]"));
Reporter.log("The First Option Is: " + option1.getText(), true);
System.out.println("I am in my first testcase with priority=1");
} catch (Exception e) {
}
}
@Test(priority = 2)
public void getSecondOptionName() {
try {
WebElement option2 = driver.findElement(By.xpath("(//div[@class='contentlearn']//h2)[2]"));
Reporter.log("The Second Option Is: " + option2.getText(), true);
} catch (Exception e) {
}
}
@Test(priority = 3)
public void getThirdOptionName() {
try {
WebElement option3 = driver.findElement(By.xpath("(//div[@class='contentlearn']//h2)[3]"));
Reporter.log("The Third Option Is: " + option3.getText(), true);
} catch (Exception e) {
}
}
@Test(priority = 4)
public void getFourthOptionName() {
try {
WebElement option4 = driver.findElement(By.xpath("(//div[@class='contentlearn']//h2)[4]"));
Reporter.log("The Fourth Option Is: " + option4.getText(), true);
} catch (Exception e) {
}
}
@Test(priority = 5)
public void getFifthOptionName() {
try {
WebElement option5 = driver.findElement(By.xpath("(//div[@class='contentlearn']//h2)[5]"));
Reporter.log("The Fifth Option Is: " + option5.getText(), true);
} catch (Exception e) {
}
}
@Test(priority = 6)
public void getSixthOptionName() {
try {
WebElement option6 = driver.findElement(By.xpath("(//div[@class='contentlearn']//h2)[6]"));
Reporter.log("The Sixth Option Is: " + option6.getText(), true);
} catch (Exception e) {
}
}
@AfterClass
public void closeBrowser() {
driver.close();
Reporter.log("Closing the browser", true);
}
}
The below testng.xml file can be used for running the above Java class.
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="TestNGPriorityTest">
<test name="TestNGPriorityTest" >
<classes>
<class name="LamdaTest.SeleniumPlaygroundTestsWithPriority">
</class>
</classes>
</test>
</suite>
Code Walkthrough:
In this test case, the scenario remains the same as what we saw in our first example, except that all the methods have been assigned a priority here.
Execution Results:
Since all the test methods had their respective priorities assigned, TestNG will execute them in ascending order of their priorities. Hence, the method getFirstOptionName with priority=1 will be executed first, followed by getSecondOptionName with priority=2, and so on. As a result, console output looks like this:
You can further check the LamdaTest Automation Dashboard to check your test results and logs.
Implementing test case Priority when methods are with the same Priority in TestNG
In the previous section of this blog on how to set test case priority in TestNG, we saw how easily TestNG picked up our test methods based on their priority in ascending order, but what if the methods had the same priority assigned. So let’s see how our test result would look if the same test methods run with an equal priority assigned.
In this method, we will also assign a negative priority to a few of our test methods.
package LamdaTest;
import org.openqa.selenium.By;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.remote.DesiredCapabilities;
import org.openqa.selenium.remote.RemoteWebDriver;
import org.testng.Reporter;
import org.testng.annotations.AfterClass;
import org.testng.annotations.BeforeClass;
import org.testng.annotations.Test;
import java.net.MalformedURLException;
import java.net.URL;
class SeleniumPlaygroundTestsWithSamePriority {
public String username = "YOUR USERNAME";
public String accesskey = "YOUR ACCESSKEY";
public static RemoteWebDriver driver = null;
public String gridURL = "@hub.lambdatest.com/wd/hub";
@BeforeClass
public void setUp() throws Exception {
DesiredCapabilities capabilities = new DesiredCapabilities();
capabilities.setCapability("browserName", "chrome");
capabilities.setCapability("version", "93.0");
capabilities.setCapability("platform", "win10"); // If this cap isn't specified, it will just get the any available one
capabilities.setCapability("build", "TestNGWithSamePriorityTests");
capabilities.setCapability("name", "TestNGWithSamePriorityTestsSample");
try {
driver = new RemoteWebDriver(new URL("https://" + username + ":" + accesskey + gridURL), capabilities);
} catch (MalformedURLException e) {
System.out.println("Invalid grid URL");
} catch (Exception e) {
System.out.println(e.getMessage());
}
Reporter.log("Logging into Selenium Playground", true);
driver.get("http://labs.lambdatest.com/selenium-playground/");
}
@Test(priority = 1)
public void getFirstOptionName() {
try {
WebElement option1 = driver.findElement(By.xpath("(//div[@class='contentlearn']//h2)[1]"));
Reporter.log("The First Option Is: " + option1.getText(), true);
} catch (Exception e) {
}
}
@Test(priority = 1)
public void getSecondOptionName() {
try {
WebElement option2 = driver.findElement(By.xpath("(//div[@class='contentlearn']//h2)[2]"));
Reporter.log("The Second Option Is: " + option2.getText(), true);
} catch (Exception e) {
}
}
@Test(priority = 2)
public void getThirdOptionName() {
try {
WebElement option3 = driver.findElement(By.xpath("(//div[@class='contentlearn']//h2)[3]"));
Reporter.log("The Third Option Is: " + option3.getText(), true);
} catch (Exception e) {
}
}
@Test(priority = -1)
public void getFourthOptionName() {
try {
WebElement option4 = driver.findElement(By.xpath("(//div[@class='contentlearn']//h2)[4]"));
Reporter.log("The Fourth Option Is: " + option4.getText(), true);
} catch (Exception e) {
}
}
@Test(priority = -2)
public void getFifthOptionName() {
try {
WebElement option5 = driver.findElement(By.xpath("(//div[@class='contentlearn']//h2)[5]"));
Reporter.log("The Fifth Option Is: " + option5.getText(), true);
} catch (Exception e) {
}
}
@Test(priority = -3)
public void getSixthOptionName() {
try {
WebElement option6 = driver.findElement(By.xpath("(//div[@class='contentlearn']//h2)[6]"));
Reporter.log("The Sixth Option Is: " + option6.getText(), true);
} catch (Exception e) {
}
}
@AfterClass
public void closeBrowser() {
driver.close();
Reporter.log("Closing the browser", true);
}
}
You can use the below testng.xml file for running the above Java class.
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="TestNGPriorityTest">
<test name="TestNGPriorityTest" >
<classes>
<class name="LamdaTest.SeleniumPlaygroundTestsWithSamePriority">
</class>
</classes>
</test>
</suite>
Code Walkthrough:
In this test case, the scenario remains the same as what we saw in our first and second examples, except that all the methods have been assigned a priority here, two of them having the same priority and few being assigned a negative priority.
Let’s decode how our results would look like. Here, the test method getSixthOptionName is assigned the lowest priority of -3, and hence, it will be picked up first, followed by getFifthOptionName with priority = -2, and then getFourthOptionName with priority = -1.
After these, there are two test methods getFirstOptionName and getSecondOptionName running with the same priority = 1. Hence, out of these two, getFirstOptionName will be executed first, since it comes first alphabetically followed by getSecondOptionName. The last method to be executed will be getThirdOptionName running with priority = 2.
Execution Results:
As expected, the console output will look like:
You can further check the LamdaTest Automation Dashboard to check your test results and logs.
Implementing test case Priority with both Prioritized and Non-Prioritized methods in TestNG
In the previous section of this blog on how to set test case priority in TestNG, we had either all the test methods without any priority set or all running with priority. But can we have a combination of both prioritized and non-prioritized methods in TestNG? The answer is YES.
Let’s explore a sample test case and understand how TestNG would execute our test methods in such scenarios.
package LamdaTest;
import org.openqa.selenium.By;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.remote.DesiredCapabilities;
import org.openqa.selenium.remote.RemoteWebDriver;
import org.testng.Reporter;
import org.testng.annotations.AfterClass;
import org.testng.annotations.BeforeClass;
import org.testng.annotations.Test;
import java.net.MalformedURLException;
import java.net.URL;
class SeleniumPlaygroundTestsWithAndWithoutPriority {
public String username = "YOUR USERNAME";
public String accesskey = "YOUR ACCESSKEY";
public static RemoteWebDriver driver = null;
public String gridURL = "@hub.lambdatest.com/wd/hub";
@BeforeClass
public void setUp() throws Exception {
DesiredCapabilities capabilities = new DesiredCapabilities();
capabilities.setCapability("browserName", "chrome");
capabilities.setCapability("version", "93.0");
capabilities.setCapability("platform", "win10"); // If this cap isn't specified, it will just get the any available one
capabilities.setCapability("build", "TestNGWithAndWithoutPriorityTests");
capabilities.setCapability("name", "TestNGWithAndWithoutPriorityTestsSample");
try {
driver = new RemoteWebDriver(new URL("https://" + username + ":" + accesskey + gridURL), capabilities);
} catch (MalformedURLException e) {
System.out.println("Invalid grid URL");
} catch (Exception e) {
System.out.println(e.getMessage());
}
Reporter.log("Logging into Selenium Playground", true);
driver.get("http://labs.lambdatest.com/selenium-playground/");
}
@Test
public void getFirstOptionName() {
try {
WebElement option1 = driver.findElement(By.xpath("(//div[@class='contentlearn']//h2)[1]"));
Reporter.log("The First Option Is: " + option1.getText(), true);
} catch (Exception e) {
}
}
@Test
public void getSecondOptionName() {
try {
WebElement option2 = driver.findElement(By.xpath("(//div[@class='contentlearn']//h2)[2]"));
Reporter.log("The Second Option Is: " + option2.getText(), true);
} catch (Exception e) {
}
}
@Test(priority = 1)
public void getThirdOptionName() {
try {
WebElement option3 = driver.findElement(By.xpath("(//div[@class='contentlearn']//h2)[3]"));
Reporter.log("The Third Option Is: " + option3.getText(), true);
} catch (Exception e) {
}
}
@Test(priority = 1)
public void getFourthOptionName() {
try {
WebElement option4 = driver.findElement(By.xpath("(//div[@class='contentlearn']//h2)[4]"));
Reporter.log("The Fourth Option Is: " + option4.getText(), true);
} catch (Exception e) {
}
}
@Test(priority = 0)
public void getFifthOptionName() {
try {
WebElement option5 = driver.findElement(By.xpath("(//div[@class='contentlearn']//h2)[5]"));
Reporter.log("The Fifth Option Is: " + option5.getText(), true);
} catch (Exception e) {
}
}
@Test(priority = 2)
public void getSixthOptionName() {
try {
WebElement option6 = driver.findElement(By.xpath("(//div[@class='contentlearn']//h2)[6]"));
Reporter.log("The Sixth Option Is: " + option6.getText(), true);
} catch (Exception e) {
}
}
@AfterClass
public void closeBrowser() {
driver.close();
Reporter.log("Closing the browser", true);
}
}
You can use the below testng.xml for running the Java class.
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="TestNGPriorityTest">
<test name="TestNGPriorityTest" >
<classes>
<class name="LamdaTest.SeleniumPlaygroundTestsWithAndWithoutPriority">
</class>
</classes>
</test>
</suite>
Code Walkthrough:
In this test case, the scenario remains the same as what we saw until now, except that some methods have been assigned a priority here while some are running with default priority.
Let’s see how our results would look like, Here, the test methods getFifthOptionName has been assigned a priority =0, but it still falls at the same level as methods getFirstOptionName and getSecondOptionName since even though they are not assigned any priority explicitly, they will run with the default priority which is 0. Hence, out of all these three methods, getFifthOptionName will run first, followed by getFirstOptionName and getSecondOptionName, considering their alphabetical order. After these, the test method getFourthOptionName will be executed, followed by getThirdOptionName as both of them have the same priority set, which is 1. In the end, getSixthOptionName will be executed since it is running with a priority of 2.
Execution Results:
As expected, the console output will look like:
You can further check the LamdaTest Automation Dashboard to check your test results and logs.
TestNG issue when running tests with same Priority and in Parallel
We saw how to set case priority in TestNG and how the TestNG priority worked in different scenarios when they had the same or different priority set or even when they had no priority set. However, the scenarios where the same priority assigned methods could behave a little differently if you would run them in parallel.
You can refer to Create TestNG XML File & Execute Parallel Testing that deep dives into the details of parallel testing in TestNG using TestNG XML file.
For example, If we use our same test case where all the methods had no priority explicitly defined, which means they run with a default priority, i.e., 0 but, instead of running the test methods sequentially, let’s run them in parallel and see how the result would look like.
package LamdaTest;
import org.openqa.selenium.By;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.remote.DesiredCapabilities;
import org.openqa.selenium.remote.RemoteWebDriver;
import org.testng.Reporter;
import org.testng.annotations.AfterClass;
import org.testng.annotations.BeforeClass;
import org.testng.annotations.Test;
import java.net.MalformedURLException;
import java.net.URL;
class SeleniumPlaygroundTestsWithoutPriority {
public String username = "YOUR USERNAME";
public String accesskey = "YOUR ACCESSKEY";
public static RemoteWebDriver driver = null;
public String gridURL = "@hub.lambdatest.com/wd/hub";
@BeforeClass
public void setUp() throws Exception {
DesiredCapabilities capabilities = new DesiredCapabilities();
capabilities.setCapability("browserName", "chrome");
capabilities.setCapability("version", "94.0");
capabilities.setCapability("platform", "win10"); // If this cap isn't specified, it will just get the any available one
capabilities.setCapability("build", "TestNGWithoutPriorityTests");
capabilities.setCapability("name", "TestNGWithoutPriorityTestsSample");
try {
driver = new RemoteWebDriver(new URL("https://" + username + ":" + accesskey + gridURL), capabilities);
} catch (MalformedURLException e) {
System.out.println("Invalid grid URL");
} catch (Exception e) {
System.out.println(e.getMessage());
}
Reporter.log("Logging into Selenium Playground", true);
driver.get("http://labs.lambdatest.com/selenium-playground/");
}
@Test
public void getFirstOptionName() {
try {
WebElement option1 = driver.findElement(By.xpath("(//div[@class='contentlearn']//h2)[1]"));
Reporter.log("The First Option Is: " + option1.getText(), true);
} catch (Exception e) {
}
}
@Test
public void getSecondOptionName() {
try {
WebElement option2 = driver.findElement(By.xpath("(//div[@class='contentlearn']//h2)[2]"));
Reporter.log("The Second Option Is: " + option2.getText(), true);
} catch (Exception e) {
}
}
@Test
public void getThirdOptionName() {
try {
WebElement option3 = driver.findElement(By.xpath("(//div[@class='contentlearn']//h2)[3]"));
Reporter.log("The Third Option Is: " + option3.getText(), true);
} catch (Exception e) {
}
}
@Test
public void getFourthOptionName() {
try {
WebElement option4 = driver.findElement(By.xpath("(//div[@class='contentlearn']//h2)[4]"));
Reporter.log("The Fourth Option Is: " + option4.getText(), true);
} catch (Exception e) {
}
}
@Test
public void getFifthOptionName() {
try {
WebElement option5 = driver.findElement(By.xpath("(//div[@class='contentlearn']//h2)[5]"));
Reporter.log("The Fifth Option Is: " + option5.getText(), true);
} catch (Exception e) {
}
}
@Test
public void getSixthOptionName() {
try {
WebElement option6 = driver.findElement(By.xpath("(//div[@class='contentlearn']//h2)[6]"));
Reporter.log("The Sixth Option Is: " + option6.getText(), true);
} catch (Exception e) {
}
}
@AfterClass
public void closeBrowser() {
driver.close();
Reporter.log("Closing the browser", true);
}
}
Let’s use the below testng.xml and run the same class in parallel with a thread count of 6.
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="TestNGPriorityTest" parallel="methods" thread-count="6" >
<test name="TestNGPriorityTest">
<classes>
<class name="LamdaTest.SeleniumPlaygroundTestsWithoutPriority">
/class>
</classes>
</test>
</suite>
Execution Results:
Now ideally, the test methods should have been executed in alphabetical order since they all had a default priority of 0 assigned. So, getFifthOptionName should run first, followed by getFirstOptionName, and so on.
Actual Results:
The test methods get executed in a random sequence instead of alphabetical order since we are running them in parallel.
The console output shows that the methods have been executed in a random sequence.
Note: The same works fine if we simply remove the parallel and thread-count attribute from our testng.xml file.
One thing important to consider here is that you would only face this problem when the methods have the same priority. The execution order remains exactly as expected when those methods run with different priorities assigned and in parallel.
You can follow the LambdaTest YouTube Channel by scanning the below QR code and stay up to date with the latest tutorials around Selenium automation testing, cross browser testing, and more.
Conclusion
In this blog on how to set test case priority in TestNG, we learned about the Priority attribute in TestNG and how we can use it while performing Selenium.
](https://cdn-images-1.medium.com/max/2000/1*37CzKsE-9N2w4rIQCTdVWA.gif)
We saw how TestNG handles any negative priority, or when all methods have the same priority and when all methods have different priorities. We also came across the limitation of TestNG, which happens when the methods have the same priority assigned, and they are being run in parallel.
I hope you enjoyed reading this Selenium TestNG tutorial on how to set test case priority in TestNG and understood about setting test case priority in TestNG.
Happy Testing !!