What is Data Driven Testing? How to Create a Data Driven Automation Framework

What is Data Driven Testing?

Data Driven Testing is an Automation framework where we can alliteratively run multiple data set from an external source like a table for the same test script-instead of hard coding.

Multiple data sets and test environment can be controlled and run without hard coding and results obtained can be compared.

If data is hard coded it is highly tedious, monotonous and inefficient to run the same test script for different data set.

This becomes easier when data from different sources can be fed as input, this input gets verified and result outputs obtained can be compared.

Types of Data Driven Testing

Data-driven testing can be broadly classified into three parts:

• Data-driven scripts: Data-Driven Scripts are application-specific scripts (like JavaScript) that are coded to include variable data sets.
• Keyword-driven test automation: Keyword Driven Test Automation is also known as Table Driven Test Automation. In keyword Driven test automation a data table using keyword id deployed for testing
• Hybrid Test automation: It is a blend of Data-Driven and Keyword Driven Automation Frameworks.

Why Data Driven Testing?

Data-Driven testing tests application with multiple data sets ensuring extensive testing. It also allows in organizing Test data and validation data in a single file.

Example:

For example, we have a login system that has to be tested for multiple input fields with different data sets.

We can test it using different approaches:

Approach 1) For every data set creates separate scripts and execute each of it one by one.

Approach 2) Every time you have to run the test case for different data set, annually change it in the test script and execute it for all required number of data sets.

Approach 3) Import the data in an excel sheet and fetch this data one by one from the excel and run the script.

Executing the test scripts using approach 1 and 2 are very time consuming and lengthy process, the third method or what we call data-driven framework is ideal for such scenarios.

 Importance of  Data Driven Testing?

In real life applications with frequent changes in data set such as addition, modification and deletion of data, hard coding data makes it difficult or almost impossible to run each time.

Data Driven Testing makes it much easier to use and maintain voluminous data individually in tables.

Manually to cover all the scenarios, a tester may have to design multiple test scripts or edit existing test scripts multiple times and run it individually.

Thus, making this difficult or almost impossible task. For a Manual Tester, it is humongous and monotonous.

That’s exactly when Data Driven Testing comes to the rescue. Test Data can be fed from a table or other sources, independently without making changes to the test script.

All these data run for the same test script iteratively and results obtained can be compared.

Procedure Simplified

Test data are stored in an easily maintainable manner like tables or rows and columns and then passed as input variables to the AUT.

Test scripts and Data Provider are two separate entities having no direct impact on each other. Verified results from each test execution are then compared from expected results of the Data Provider.

Data Provider

Data can be fed in different forms. Few popular ones are

• Internal table
• Spreadsheet
• CSV file
• XML file

Data Driven Script

Scripts containing hard-coded data can be tough to maintain and sometimes they can break the execution.

In Data Driven Testing Test scripts and Test Data are separated and made into separate entities.

Hence Data Provider is independent of test script and both can be modified individually without impacting each other.

Real Life Citation

We shall take an example of online registration form. Data from Registration forms increase frequently sometimes every day. Each User registration add data rows in database.

1.Let’s take a simple form with Name, Email, Password and Confirm Password

2. Test Scenarios are identified and test cases are designed.

3. For now, consider only happy path – validating that data in table and actual result are correct.

4. Now let’s feed multiple data from a spreadsheet, csv file or the like as a table as below

5. During Test execution test script is run and data is taken from the table.Each data is executed one after the other in iteration row after row.

6. Once script is executed, actual results and expected results are compared.

7. Notice that data has been modified – added newly at the end of the table, edited existing in the middle row and deleted in-between the table.

This can be easily done by modifying the Data table like below without touching the script.

This can be easily done by modifying the Data table like below without touching the script.

8. Data has been modified easily without changing the script.

What is the Difference between Keyword Driven Testing and Data Driven Testing?

In Data driven testing test scripts are executed for a different set of data to validate proper working of application with different variable values. Here data is used as inputs to your test script. Every data contributes to a test case and hence with every different test data you have a different test case.

On the other hand, in Keyword-driven testing, a keyword represents action. A set of keywords drives a script. These keywords build test scripts.

Example of Data Driven Testing

Let us understand data driven testing more deeply by taking an example of a Login Page of a Flight Reservation

1) Create a test data file in Comma Separated Values and name it as TestData.csv

2)  Inputs for driver script and expected results are stored in this file.

3) Create a driver script for the data file as

data = open(‘TestData.csv’).read()

 lines = data.splitlines()

4) Steps for data script will be

• Read Value1
• Read Value2
• Read Operator

5) Calculate the result

6) Compare the expected result with the actual result.

Automation Framework For Data Driven Testing

This method can be used integrating with various Test Automation Tools like Selenium, QTP, TestComplete, TestNG etc    

How to Create a Data Driven Automation Framework?

Let us take Test Login functionality. 

1) Classify the Test Cases 

• Right username and password 
• Wrong username and the right password 
• Right username and wrong password 

2) Make Test Steps for Test Cases 

Test Case#	Description	Test Steps	Test Data	Expected Results
1	Check Login for valid credentials	Launch the application Enter Username password Click Okay Check Results	Username: valid password: valid	Login Success
2	Check Login for invalid credentials	Launch the application Enter Username password Click Okay Check Results	Username: invalid password: valid	Login Fail
3	Check Login for invalid credentials	Launch the application Enter Username password Click Okay Check Results	Username: valid password: invalid	Login Fail

3) Develop Test Script

// This is Pseudo Code

// Test Step 1: Launch Application

driver.get(“URL of the Application”); 

// Test Step 2: Enter Username

txtbox_username.sendKeys(“valid”);

// Test Step 3: Enter Password

txtbox_password.sendKeys(“invalid”);

// Test Step 4: Check Results

If (Next Screen) print success else Fail

4) Make excel/csv for Input Test Data

5) Alter the Script for various Input Test Data.

// This is Pseudo Code

// Loop 3 Times

for (i = 0; i & lt; = 3; i++) {

    // Read data from Excel and store into variables

    int input_1 = ReadExcel(i, 0);

    int input_2 = ReadExcel(i, 1);

    // Test Step 1: Launch Application

    driver.get(“URL of the Application”);

    // Test Step 2: Enter Username

    txtbox_username.sendKeys(input_1);

    // Test Step 3: Enter Password

    txtbox_password.sendKeys(input_2);

    // Test Step 4: Check Results

    If(Next Screen) print success

    else Fail

}

Advantages of Data Driven Testing

• Test Data can be stored in a single file making it easy to maintain, monitor and track.
• Test Data and Test scripts are separate entities. Hence changes in one does not impact the other.
• Large volumes of data can be executed thereby improving Regression testing and better coverage
• Smoother maintenance of test scripts making addition, editing and deletion of data rows easy and hence less time consuming
• Manual effort involved is less and hence it is cost effective

Disadvantages of Data Driven Testing

• Requires great expertise of scripting language
• Each time a new test case is designed and a new driver script will be required with different data so that the changes made to the test cases should reflect in the driver script or vice versa.

Conclusion

Data Driven Testing a very good strategy if we have huge volumes of data to be tested for same scripts provided there is a highly skilled testing team. And it does not suit projects that do not have much work with data.