Are you find difficulty when identify Test Case Combinations?

As a Test Lead or Test Analyst, they need to come up with an effective testcase design and planning techniques in order to attain maximum test coverage. It is nearly impossible to achieve 100% test coverage unless product has simple requirements. Many defects only occur when a combination of inputs or events occur that interact with each other. Hence, it becomes critical to come up with a ‘subset of combinations’ which provide inputs to the test cases in achieving the best results.

Common Manual Testing drawbacks

• Time Consuming & Cost driven
• More numbers of Resources are required for testing
• Multiple times of testing the same set or subset of test cases in a different phase.
• Tester does not conclude that they cover the application completely or not (Not sure about coverage of the application)

So, test design techniques should be very effective to identify best possible test cases/combinations that can help in testing the core functionalities of the product, thereby achieving the best coverage results and high defect yield rate.

Pairwise testing is a black box testing technique which is an alternative to testing all possible combinations of a set of variables. This testing method helps to reduce the number of test cases by testing all pairs of variables rather than attempting to test all of the combinations for all the values for all the variables. Also, this method works well when the number of possible parameter values is large. As a result, the number of test-cases used is lowered while the overall functional coverage does not decrease significantly.

· Sources of defects are usually due to unexpected interactions between parameters

· Testing all possible combination is infeasible due to budget and time constraints

· Suitable approach is to test all two-way between parameters

· If a test plan covers all the two -way interactions between parameters, then we are balancing of completing the testing within reasonable budget vs. risk of defects due to interactions among three or more parameters

· Good way of measuring a test plan coverage is to find how much percentage of pair wise combinations are covered in the test plan

Simplest bugs in a program are generally triggered by a single input parameter. Next category of Bugs consist of dependent interactions between pairs of parameters can be caught with less effort using All-pairs testing.

Suppose, you have a scenario of a software application for testing as follows;

Assume that there is a function with a list box that contains 5 elements, text box that can accept 1 to 10 characters, checkbox and OK button.

1. Check Box — Checked or Unchecked
2. List Box — 0, 1, 2, 3, 4
3. Text Box — Number of alphabets between 1 to 10.

Calculation of all the possible combinations:

The total number of test cases, including negative test cases, is 100.

Test team has to perform exhaustive testing and it is very long and time-consuming process if team going to test all 100 positive and negative cases.

Therefore, testing team has to reduce the number of test scenarios. Inputs values for list box in such a way that the first value is 0, and the other value can be any numeric number, neither positive nor negative. Five values are now converted into 2 values.

Input values for the checkbox can be identified as only 2 combinations. At last, input values of the text box can be categorized into three input categories valid integer, invalid integer, and alpha-special character.

So total number of combinations can be reduced to 2*2*3 = 12 testcases

12 Test Combinations can be identified as follows in conventional software testing

Now, the task is to reduce combinations using all pair technique, which guarantees to test best possible combinations of a Test Scenario.

Let’s take T1 test combination and all three pairs of this scenario ({Checked ,0} {Checked, Valid Integer} {0, Valid Integer}) covered using different test combinations as described below. Therefore, we can remove T1 combination from the Testcases.

Get rid of Testcase (T1)

(Checked, 0, -) Covered in T2

(Checked, -, Valid Integer) Covered in T4

(-, 0, Valid Integer) Covered in T7

Test cases/Test combinations can be reduced to only 6 pair test cases by using pair wise testing instead of testing all 100 possible Test cases or 12 testcases using conventional software testing method.

Available tools for Pair wise testing

¡ Hexawise

¡ PICT (Pairwise Indepedent Combinatrial Testing) tool

¡ Automatic Efficient Test Generator or AETG

Advantages

· Guarantees the testing of pair-wise combinations of all the selected variables

· Efficient and effective way of creating fewer test cases from all combinations of all variables

· simpler to generate and less error prone test sets created by hand

· Exercises some of the complex combinations of all the variables

Limitations

· Selecting the wrong input: Problems of equivalence partitioning are made worse with a pair of wrong inputs

· High probability combinations get too little attention

· Not knowing the interdependencies between variables

Reference

https://www.tutorialspoint.com/software_testing_dictionary/all_pairs_testing.htm

Pairwise Testing or All-Pairs Testing Tutorial with Tools and Examples

About the Author

Sameera Palansooriya works as a Senior Quality Assurance Engineer at 99X Technology.