TestNG Data Provider - Data Driven Automation Testing

TestNg DataProvider is used for Data Driven testing in a Test Automation Framework. As the name implies DataProvider supplies data to our test methods. It helps us to execute a single test method with multiple sets of data.
DataProvider can also be used for getting data from file or database and passing to our test methods, we will see that in another post.

A DataProvider is a method of our class that must returns an array of array of objects(object[][]). This method is supposed to be annotated with @DataProvider and a 'name' attribute.
In our test method with @Test annotation, we specify the DataProvider with the 'dataProvider' attribute with a name that corresponds to our 'name' attribute in the DataProvider method.

The DataProvider method is usually defined and looked up in the same class as the test method. But we find it more helpful if we define it in another class, in that case it needs to be a static method. If we define DataProvider in a separate class, we specify the class name with 'dataProviderClass' attribute in our @Test annotation. See example (1) below.

A DataProvider method can return one of the follwing:
- An array of array of objects(Object[][]) where the first dimension's size is the number of times the test method will be invoked and the second dimension size contains an array of objects. The second dimension's array is what we define as parameter in our test method and must be compatible with the parameter types of the test method. See example (1) below for more clarity.
- An Iterator <Object[]>. The java.util.Iterator used in here lets us create our test data lazily. If we have a lot of parameter sets to pass to the method and we don't want to create all of them upfront, which is done incase we are using double array Object, we should use an Iterator. TestNg allows us to use an Iterator on the DataProvider so that the set of parameters are instantiated and returned when the test method gets invoked each time. This is called lazy initialization, and the idea is to create an object when required and not before. Example (5) below.

Here we will see a few examples using DataProviders. Just to make this post solely for TestNg DataProviders, we have not included any Selenium code here.

1) Lets start with passing simple parameters - we will pass 2 parameters in one test method and 3 parameters in another test.
We will define the DataProvider method in separate class.

package com.testng.test;
import org.testng.annotations.DataProvider;

public class Providers {
	@DataProvider(name = "provider1")
	public static Object[][] provideData() {
		return new Object[][] { 
			{ 1, 100 }, 
			{ 5, 500 }, 
			{ 10, 1000 } 
		};
	}
	
	@DataProvider(name = "provider2")
	public static Object[][] provideSomething() {
		return new Object[][]{
				{1,"Delhi","DEL"},
				{2,"Mumbai","MAH"}
		};
	}
}

The class which uses the providers:

package com.testng.test;
import org.testng.annotations.Test;

public class CheckDataProvider {
	
	@Test(dataProvider = "provider1", dataProviderClass = Providers.class)
	public void testData1(int inNum, int expect) {
		System.out.println("Number: " + (inNum * 100) + " Expected: " + expect);
	}

	@Test(dataProvider = "provider2", dataProviderClass = Providers.class) 
	public void testData2(int cityId, String city, String state) {
		System.out.println("City Id: " + cityId + " City: " + city + " State: " + state);
	}
}

Run the CheckDataProvider class as TestNG Test. Below is the output:

[TestNG] Running:
  C:\Users\sarats\AppData\Local\Temp\testng-eclipse--714784067\testng-customsuite.xml

Number: 100 Expected: 100
Number: 500 Expected: 500
Number: 1000 Expected: 1000
City Id: 1 City: Delhi State: DEL
City Id: 2 City: Mumbai State: MAH
PASSED: testData1(1, 100)
PASSED: testData1(5, 500)
PASSED: testData1(10, 1000)
PASSED: testData2(1, "Delhi", "DEL")
PASSED: testData2(2, "Mumbai", "MAH")

===============================================
    Default test
    Tests run: 5, Failures: 0, Skips: 0
===============================================

===============================================
Default suite
Total tests run: 5, Failures: 0, Skips: 0
===============================================

We see that each of the test method is run the number of times in our Object[][] array.


2) Passing a List: Returning a List from the DataProvider is usually done when we read input data from a file and DataProvider is used to pass on the data to the test method. Method for reading data from files are better placed in another class and then from within the DataProvider method we can just call the method.
We can iterate over the list in the DataProvider method(e.g. listProvider2) and return an Object[][], or we can pass on the List to the test method.
Below we will see both the ways, for the test method "listTesting1" we are just populating a List in the DataProvider method(listProvider1) and passing the list to the test method. We can also pass other collection objects like maps.
For test method "listTesting2" it is getting values from DataProvider(listProvider2) one at a time. Here the DataProvider method takes care of iterating over the list, which is usually the way we read data from a file and DataProvider passes on the data to test method one after another.

 package com.testng.test;  
 import java.util.ArrayList;  
 import java.util.List;  
 import org.testng.annotations.DataProvider;  
 import org.testng.annotations.Test;
 
 public class ListDataProvider {  
      @DataProvider(name = "listProvider1")  
      public static Object[][] provideListData() {  
            List<String> list = new ArrayList<String>();  
           for (int i=1; i<=5; i++) {  
                list.add("data" + i);  
           }  
           return new Object[][] { { list } };  
      }  
 
     @DataProvider(name = "listProvider2")  
      public static Object[][] provideListData2() {  
            List<String> list = new ArrayList<String>();  
           for (int i=1; i<=5; i++) {  
                list.add("data" + i);  
           }  
           String[][] ret = new String[list.size()][];  
           for (int i=0; i<list.size(); i++) {  
                ret[i] = new String[]{list.get(i)};  
           }  
           return ret;  
      }  

      @Test(dataProvider = "listProvider1")  
      public void listTesting1(List<String> list) {  
           System.out.println("#### " + Thread.currentThread().getStackTrace()[1].getMethodName() + " ####");  
           for (String str : list) {  
                System.out.println(str);  
           }  
      }  

      @Test(dataProvider = "listProvider2")  
      public void listTesting2(String str) {  
           System.out.println("#### " + Thread.currentThread().getStackTrace()[1].getMethodName() + " ####");  
           System.out.println("test data: " + str);  
      }  
 }  

Run the ListDataProvider class as TestNG Test, and below is the output:

[TestNG] Running:
  C:\Users\sarats\AppData\Local\Temp\testng-eclipse-788397593\testng-customsuite.xml

#### listTesting1 ####
data1
data2
data3
data4
data5
#### listTesting2 ####
test data: data1
#### listTesting2 ####
test data: data2
#### listTesting2 ####
test data: data3
#### listTesting2 ####
test data: data4
#### listTesting2 ####
test data: data5
PASSED: listTesting1([data1, data2, data3, data4, data5])
PASSED: listTesting2("data1")
PASSED: listTesting2("data2")
PASSED: listTesting2("data3")
PASSED: listTesting2("data4")
PASSED: listTesting2("data5")

===============================================
    Default test
    Tests run: 6, Failures: 0, Skips: 0
===============================================

===============================================
Default suite
Total tests run: 6, Failures: 0, Skips: 0
===============================================

3) Provide data depending on the calling test method: If same DataProvider is used to supply data to several test methods we can use "java.lang.reflect.Method" and provide data according to the calling method. DataProvider method can take a java.lang.reflect.Method as first parameter, in this case TestNG will pass the calling test method for this first parameter.

 package com.testng.test;
  
 import java.lang.reflect.Method;  
 import java.util.ArrayList;  
 import java.util.List;  
 import org.testng.annotations.DataProvider;  
 import org.testng.annotations.Test;  

 public class DataProviderMethodParam {  
      @DataProvider(name = "ProviderMethodParam")  
      public static Object[][] provideListData(Method method) {  
            List<String> list = new ArrayList<String>();  
           for (int i=1; i<=5; i++) {  
                list.add("data" + i);  
           }  
           if (method.getName().equals("listTesting1")) {  
           return new Object[][] { { list } };  
           }  
           else if (method.getName().equals("listTesting2")){  
                String[][] ret = new String[list.size()][];  
                for (int i=0; i<list.size(); i++) {  
                     ret[i] = new String[]{list.get(i)};  
                }  
                return ret;  
           }  
           return null;  
      }  

      @Test(dataProvider = "ProviderMethodParam")  
      public void listTesting1(List<String> list) {  
           System.out.println("#### " + Thread.currentThread().getStackTrace()[1].getMethodName() + " ####");  
           for (String str : list) {  
                System.out.println(str);  
           }  
           System.out.println("**********************************");  
      }  

      @Test(dataProvider = "ProviderMethodParam")  
      public void listTesting2(String str) {  
           System.out.println("#### " + Thread.currentThread().getStackTrace()[1].getMethodName() + " ####");  
           System.out.println("test data: " + str);  
      }  
 }  

Run the class DataProviderMethodParam as TestNG Test, and below is the output:

[TestNG] Running:
  C:\Users\sarats\AppData\Local\Temp\testng-eclipse--1473820315\testng-customsuite.xml

#### listTesting1 ####
data1
data2
data3
data4
data5
**********************************
#### listTesting2 ####
test data: data1
#### listTesting2 ####
test data: data2
#### listTesting2 ####
test data: data3
#### listTesting2 ####
test data: data4
#### listTesting2 ####
test data: data5
PASSED: listTesting1([data1, data2, data3, data4, data5])
PASSED: listTesting2("data1")
PASSED: listTesting2("data2")
PASSED: listTesting2("data3")
PASSED: listTesting2("data4")
PASSED: listTesting2("data5")

===============================================
    Default test
    Tests run: 6, Failures: 0, Skips: 0
===============================================

===============================================
Default suite
Total tests run: 6, Failures: 0, Skips: 0
===============================================

4) Provide data depending on ITestContext: We can use 'org.testng.ITestContext' in the DataProvider method to determine the runtime parameters of the calling test method. Depending on the parameter we can choose what data to provide. Here we will see how to pass data depending on TestNg's 'groups' attribute.

 package com.testng.test;  
   
 import java.util.Arrays;  
 import org.testng.ITestContext;  
 import org.testng.annotations.DataProvider;  
 import org.testng.annotations.Test;  
   
 public class DataProviderITestContext {  
    
  @DataProvider(name = "dpContext")  
  public Object[][] provided(ITestContext context) {  
   Object[][] result = null;  
   int numTimes = 5;  
   String[] groups = context.getIncludedGroups();  
   System.out.println(context.getName());  
   System.out.println(Arrays.toString(groups));  
     
   for (String grp: groups) {  
    if (grp.equalsIgnoreCase("smoke-test")) {  
     numTimes = 2;  
     break;  
    }  
   }  
   
   result = new Object[numTimes][];  
   for(int i=0;i<numTimes;i++) {  
    result[i] = new Object[] {new Integer(100 + i)};  
   }  
   return result;  
  }  
    
  @Test(dataProvider = "dpContext", groups = {"smoke-test"})  
  public void smokeTesting(int num) {  
   System.out.println("Data provided: " + num);  
  }  
    
  @Test(dataProvider = "dpContext", groups = {"regress-test"})  
  public void regressTesting(int num) {  
   System.out.println("Data provided: " + num);  
  }  
   
 }  

Below is the testng.xml:

 <?xml version="1.0" encoding="UTF-8"?><!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd" >  
 <suite name="test-Dataprovider" verbose="1">  
  <test name="example-DP">  
   <groups>  
     <run>  
       <include name="smoke-test" />  
     </run>  
   </groups>  
   <classes>  
     <class  
     name="com.testng.test.DataProviderITestContext" />  
   </classes>  
  </test>  
 </suite>  

Below is the output:

[TestNG] Running:
  D:\eclipse_workspace\TestngWDFramework\testng.xml

example-DP
[smoke-test]
Data provided: 100
Data provided: 101

===============================================
test-Dataprovider
Total tests run: 2, Failures: 0, Skips: 0
===============================================

5) Lazy Data Provider using Iterator: A DataProvider method returns an Iterator instead of an array of arrays of objects. Whenever TestNg require the next set of parameters, it can instantiate the parameter and return it to the test method.
We will see the example in another post.