How to address customer issues with better test data?
Have you ever been in a situation where your manager or stakeholder in a project complain that a customer tried something in your software and it did not work? You never expected the customer to use the software the way they used it. If only you had gauged the right test data you would have caught the issue before it was released to production right?
Test data in short is the data that a test engineer uses to identify what a customer could use and test it against the system to ensure the requirement is satisfied.
Creating test data is a creative approach that could include collecting data from a given requirement, understand usage of the software by a customer in production using analytics, build data using customer issues and also sometimes you may even talk to the customer.
If we have a good set of test data, then half of our testing difficulties are ruled out. A good test data will help in identifying the coverage of the testing.
Let us look at the types of test data mainly used in testing:
a) Positive test data – This data is used to verify that a given set of input to software produces as an expected result.
b) Negative test data – This data is used to verify the invalid behavior of the system using our test data.
Test data can be used for Manual Testing or Automation testing. It also plays a vital role in performance testing, security testing, and black box testing.
Test data is created by test engineer during the planning cycle. It always dependent on the test cases that are planned for execution.
Test data will not be always constant, it may vary based on what and where are we going to perform the testing. Understanding user context is paramount to creating test data.
What can constitute test data?
1. Blank files
2. Valid set of test data
3. Invalid set of test data
4. Huge test data
5. Test data to check all the boundary conditions
Test data generation is an art and is always a time-consuming practice.
The test data creation is depended on the feature or test cases that we are going to test.
Based on our testing, the data can be generated:
1. Manually
2. Automatically for bulk data testing
3. Can also be copied in bulk from a test environment or production environment (given the customer is fine to copy and data is not subjected to privacy risks)
Examples of test data
Example 1 #:
For performance testing, we might need to replicate a huge set of test data in test environment. To deal with it, the easy method is to copy the data from live environment to test environment or use a tool to generate test data.
Example 2 #:
Let say we are testing an e-commerce website. For testing, we might need various details like:
Type of Merchants (in case of a marketplace)
Type of Products
Type of shipment
Type of payment
Type of shipment
Best practices for test data generation:
1. It is always a best practice to plan ahead and create the test data before the testing cycle starts, otherwise, it might end up eating in to your testing schedule.
2. Understand the context and collect information from all possible artefacts before generating test data.
To conclude, better the test data, better coverage and helps you avoid unforeseen customer issues in the future.
– Harikrishnan, Senior QA Engineer, Coviam.
How much time take generally to create test data and after testing, is it works for ? % in real life.How do you go about distribution model?
The answer depends on the complexity of the project and the type of testing required. For example: Manual Testing a web form requires minimal set of test data while performance testing the same form may require a test script that would generate huge amount of data.