Explain Pairwise Testing

In pairwise testing, all pairs of values have to be exercised during testing. If there are n parameters, each with m values, then between each two parameter we have m*m pairs. The total number of pairs is m* m* n * (n − 1)/2.
The objective of pairwise testing is to have a set of test cases that cover all the pairs. As there are n parameters, a test case is a combination of values of these parameters and will cover (n−1)+(n−2)+... = n(n−1)/2 pairs. In the best case when each pair is covered exactly once by one test case, m2 different test cases will be needed to cover all the pairs.
For example, if there is a software product being developed for multiple platforms that use the browser as an interface. Suppose the software is being designed to work for three different operating systems and three different browsers. In addition, as the product is memory intensive there is a desire to test its performance under different levels of memory. So, we have the following three parameters with their different values:
Operating System: Windows, Mac, Linux
Memory Size: 128M, 256M, 512M
Browser: IE, Safari, Mozilla
For discussion, we can say that the system has three parameters: A (operating system), B (memory size), and C (browser). Each of them can have three values

which we will refer to as a1, a2, a3, b1, b2, b3, and c1, c2, c3. The total number of pairwise combinations is 9 * 3 = 27. The number of test cases, however, to cover all the pairs is much less. A test case consisting of values of the three parameters covers three combinations (of A-B, B-C, and A-C).
Hence, in the best case, we can cover all 27 combinations by 27/3=9 test cases.
Test cases for pairwise testing:
A        B              C                        Pairs
a1       b1           c1                        (a1,b1)(a1,c1)(b1,c1)
a1       b2           c2                        (a1,b2)(a1,c2)(b2,c2)
a1       b3           c3                        (a1,b3)(a1,c3)(b3,c3)
a2       b1           c2                        (a2,b1)(a2,c2)(b1,c2)
a2       b2           c3                        (a2,b2)(a2,c3)(b2,c3)
a2       b3           c1                        (a2,b3)(a2,c1)(b3,c1)
a3       b1           c3                        (a3,b1)(a3,c3)(b1,c3)
a3       b2           c1                        (a3,b2)(a3,c1)(b2,c1)
a3       b3           c2                        (a3,b3)(a3,c2)(b3,c2)
Often, it will not be possible to generate the minimum set of test cases,
particularly when the number of values for different parameters is different.


For more on this see:
A Concise Introduction to Software Engineering by Pankaj Jalote

Unearthing the Power of IBM – Rational Functional Tester - RFT

Most of the professionals in the Software Testing & Quality Assurance domain must be aware of IBM’s Rational Functional Tester, popular by its nick name - RFT. 

This article is aimed at providing an overview of this most powerful next generation functional-testing and automation tool from Rational and is based upon object-oriented technology. Earlier it was known as RobotJ and XDE Tester. 

Overview of Rational Functional Tester:

RFT provides us choice among two scripting languages and two development environments i.e. Java under Eclipse framework, alternatively Visual Basic.NET under Microsoft Visual Studio. One of the great plus point of RFT is that irrespective of the scripting language and the development environment deployed for development, we can easily integrate RFT with them while developing the tests for automation.

RFT helps us in quickly testing many variety types of applications. It enables testers to create scripts at a much rapid pace by doing script recording in the AUT & enables testing of all the objects and data conveniently.

Benefits of Rational Functional Tester at a Glance:

The current version of IBM Rational Functional Tester 7.0 integrating with ClearQuest provides strong test management functionality which includes; Planning of tests, Execution of tests followed by analysis of the test results. Integration with ClearQuest is a major mile stone, which offers new features much, demanded by most of the progressive QA enterprises of today’s time. 

Few of the benefits offered are:

1) Sound web interface

2) Support for Microsoft .NET Framework 2.0

3) Support for testing HTML applications using Mozilla Firefox 2.0

4) Seamless integration with Eclipse

5) Support for Eclipse Test and Performance Tools Platform (TPTP) logs

6) Support for Siebel 7.8 and SAP

7) Support for playback on Red Hat Linux 9.0, Red Hat Enterprise Linux 3.0 and SuSE Linux Professional 9.0.

8) Greater defect integration

9) Enhanced database support

10) Powerful test management & reporting

11) Greater querying capabilities

12) Extension of support to geographically distributed teams

13) Process advice using IBM Rational Process Advisor

How do we kick-start our work with Rational Functional Tester:

This involves a four-step process:

1) Creation of a Project: For storing various assets of the test. 

2) Configuration of the environment: Prior to beginning the automation of tests we need an environment which is conducive for Testing using RFT. The environment refers to system of communication of information like process of development for the application under test to the Rational Functional Tester, to have a proper handshake between the AUT & the test automation tool.

3) Configuration of the application under test: is the next step to suit the above two. This can be done with the help of user friendly wizard, wherein we define the application name, its type, its location & associated files.

4) Ultimately - Recording of test Scripts: is done to record all user actions against our application, like keystrokes and mouse clicks etc. We can insert verification points as well into the test data or properties of any objects in our application. We can do selective recording by omitting some of the process elements from the tests by quickly manipulating the preferences. RFT offers great flexibility of operations during the playback of the recorded script.

STLC - Software Testing Life Cycle

There is a systematic cycle to software testing, although it varies from organization to organization


Software Testing Life Cycle: Software testing life cycle or STLC refers to a comprehensive group of testing related actions specifying details of every action along with the specification of the best time to perform such actions. There can not be a standardized testing process across various organizations, however every organization involved in software development business, defines & follows some sort of testing life cycle.


STLC by & large comprises of following Six Sequential Phases: 
1) Planning of Tests
2) Analysis of Tests
3) Designing of Tests
4) Creation & Verification of Tests
5) Execution of Testing Cycles
6) Performance Testing, Documentation

7) Actions after Implementation 

Every company follows its own software testing life cycle to suit its own requirements, culture & available resources. The software testing life cycle can’t be viewed in isolation, rather it interacts with the every phase of Software Development Life Cycle (SDLC). Prime focus of the software testing life cycle is on managing & controlling all activities of software testing. Testing might be manual testing or an automated testing using some tool.


1) Planning of Tests: In this phase a senior person like the project manager plans & identifies all the areas where testing efforts need to be applied, while operating within the boundaries of constraints like resources & budget. Unless judicious planning is done in the beginning, the result can be catastrophic with emergence of a poor quality product, dissatisfying the ultimate customer. Planning is not limited just to the initial phase, rather it is a continuous exercise extending till the end. 
During the planning stage, the team of senior level persons comes out with an outline of Testing Plan at High Level. The High Level Test Plan comprehensively describes the following:

• Scope of Testing : Defining the areas to be tested, identification of features to be covered during testing 
• Identification of Approaches for Testing: Identification of approaches including types of testing
• Defining Risks: Identification of different types of risks involved with the decided plan
• Identification of resources : Identification of resources like man, materials & machines which need to be deployed during Testing
• Time schedule: For performing the decided testing is aimed to deliver the end product as per the commitment made to the customer.
  
  Involvement of software testers begins in the planning phase of the software development life cycle. During the design phase, testers work with developers in determining what aspects of a design are testable and with what parameters those tests will work.

2) Analysis of Tests: Based upon the High Level Test Plan Document, further nitty-gritty’s covering the following are worked out.

• Identification of Types of Testing to be performed during various stages of Software Development Life Cycle.
• Identification of extent to which automation needs to be done. 
• Identification of the time at which automation is to be carried out. 
• Identification of documentation required for automated testing

The Software project can’t be successful unless there is frequent interaction among various teams involved in Coding & Testing with the active involvement of the Project Managers, Business Analysts or even the customer. Any deficiencies in the decided test plans come to the surface, during such meetings of cross-functional teams. This provides an opportunity to have a rethinking & refining the strategies decided for testing.

Based upon the customer requirements a detailed matrix for functional validation is prepared to cover the following areas:

• Ensure that each & every business requirement is getting covered through some test case or the other.
• Identification of the test cases best suited to the automated testing

• Identification of the areas to covered for performance testing and stress testing
• Carry out detailed review of documentation covering areas like Customer Requirements, Product Features & Specifications and Functional Design etc.
  

3) Designing of Tests: This phase involves the following:

• Further polishing of various Test Cases, Test Plans
• Revision & finalization of Matrix for Functional Validation.
• Finalization of risk assessment methodologies. 
• In case line of automation is to be adopted, identification of test cases suitable for automation.
• Creation of scripts for Test cases decided for automation. 
• Preparation of test data.
• Establishing Unit testing Standards including defining acceptance criteria 
• Revision & finalization of testing environment.

4) Construction and verification: This phase involves the following:

• Finalization of test plans and test cases
• Completion of script creation for test cased decided for automation.
• Completion of test plans for Performance testing & Stress testing.
• Providing technical support to the code developers in their effort directed towards unit testing.
• Bug logging in bug repository & preparation of detailed bug report. 
• Performing Integration testing followed by reporting of defects detected if any.

5) Execution of Testing Cycles: This phase involves the following:

• Completion of test cycles by executing all the test cases till a predefined stage reaches or a stage of no detection of any more errors reach. 
• This is an iterative process involving execution of Test Cases, Detection of Bugs, Bug Reporting, Modification of test cases if felt necessary, Fixing of bugs by the developers & finally repeating the testing cycles.

6) Performance Testing, Documentation & Actions after Implementation: This phase involves the following:

• Execution of test cases pertaining to performance testing & stress testing.
• Revision & finalization of test documentation 
• Performing Acceptance testing, load testing followed by recovery testing
• Verification of the software application by simulating conditions of actual usage.

7) Actions after Implementation: 
This phase involves the following:

• Evaluation of the entire process of testing. 
• Documentation of TGR (Things Gone Right) & TGW (Things Gone Wrong) reports. Identification of approaches to be followed in the event of occurrence of similar defects & problems in the future.
• Creation of comprehensive plans with a view to refine the process of Testing.
• Identification & fixing of newly cropped up errors on continuous basis.

Winding up of the test environment & restoration of all test equipment to the original base line conditions.

Life Cycle of Software Testing ( STLC )
Phase	Activities	Outcome
Planning of Tests	($) Creation of a Test Plan of High Level	Refined Test Plans & Specifications
Analysis of Tests	($) Creation of fully descriptive Test Plan ($) Creation of Matrix for Functional Validation ($) Creation of Test Cases	Refined Test Plans, Test Cases & Matrix for Functional Validation
Designing of Tests	($) Revision of Test Cases ($) Selection of Test Cases fit for automation	Refined Test Cases, Input Data Sets & Documents for Assessment of Risk
Creation & Verification of Tests	($) Creation of scripts suitable for Test Cases for automation	Detailed Procedures for Testing, Testing Scripts, Test Reports & Bug-Reports
Execution of Testing Cycles	($) Completion of Cycles of Testing	Detailed Test Reports & Bug-Reports.
Performance Testing, Documentation	($) Execution of Test Cases related to performance tests & Stress Testing ($) Detailed documentation	Test Reports, Documentation on various metrics used during testing
Actions after Implementation	($) Evaluation of all Processes of Testing	Detailed Plans for improving the process of testing