Quality Assurance and Quality Control

Explain a.) Quality Assurance b.) Quality Control c.) Test Case d.) Test Condition e.) Test Script f.) Test Data g.) Test Bed

a.) Quality Assurance
Quality assurance is an activity that establishes and evaluates the processes that produce products. Quality assurance would measure processes to identify weaknesses and then correct them to continually improve the process: Eg. Code review.
b.) Quality Control
Quality control is a process in which the product's quality is compared with standards and corrective actions taken if required. Quality control activities focus on identifying defects in the actual products produced. Eg. Testing.
c.) Test Case
A test case is a unit level document describing the inputs, steps of execution and the expected result of each test condition for every requirement from the BRD. Testers determine whether the application is working correctly or not based on the test case that is being executed. A test case is marked as "Pass" if the application works as expected and is marked as "Fail" if otherwise. Test cases also aide in generating test status metrics.
d.) Test Condition
Test condition is the condition on which the test execution evaluates to pass or fail. Eg of a test condition: Set Password field should accept a mix of Alphanumeric and Special characters on mandatory basis and throw error message otherwise.
e.) Test Script
Test script is the set of instructions performed on the application to verify its functions.
f.) Test Data
Test data is the input data used to test a particular functionality. Eg: Username and password to test a login screen.
g.) Test Bed
Test bed is the environment on which the testing is done and includes the software and hardware set up that is required for testing the application.

how to perform Risk Analysis

Describe how to perform Risk Analysis during software testing.

While a test plan is being created, risks involved in testing the product are to be taken into consideration along with possibility of their occurrence and the damage they may cause along with solutions; if any. Detailed study of this is called as Risk Analysis.
Some of the risks could be:
New Hardware
New Technology
New Automation Tool
Sequence of code delivery
Availability of application test resources
Identify and describe the risk magnitude indicators: High, Medium and Low
High magnitude means the effect of the risk would be very high and non-tolerable. Company may face severe loss and its reputation is at risk. Must be tested.
Medium: tolerable but not desirable. Company may suffer financially but there is limited liability or loss of reputation. Should be tested.
Low: tolerable. Little or no external exposure. Little or no financial loss. Company’s reputation unaffected. Might be tested.
Three perspectives of Risk Assessment: Effect, Cause and Likelihood.
To assess risk by Effect, identify a condition, event or action and try to determine its impact.
To asses risk by Cause is opposite of by Effect. Begin by stating an undesirable event or condition and identify the set of events that could have permitted the condition to exist.
To asses risk by Likelihood is to determine the probability that a requirement will not be satisfied.

What is risk analysis? What does it have to do with Severity and Priority?

Risk analysis is a method to determine how much risk is involved in something. In testing, it can be used to determine when to test something or whether to test something at all. Items with higher risk values should be tested early and often. Items with lower risk value can be tested later, or not at all. It can also be used with defects. Severity tells us how bad a defect is: "how much damage can it cause?" Priority tells us how soon it is desired to fix the defect: "should we fix this and if so, by when?"
Defects with High Severity and Priority are tested first.

Difference between build and release

What is the difference between build and release?

A “build” is given by dev team to the test team. A “release” is formal release of the product to its customers.
A build when tested and certified by the test team is given to the customers as “release”. A “build” can be rejected by test team if any of the tests fail or it does not meet certain requirements. One release can have several builds associated with it.

How do you differentiate the roles of Quality Assurance Manager and Project Manager?

Quality Assurance Manager (QA Manager) defines the process to be followed at each phase of SDLC. He defines the standards to be followed, the documents to be maintained and sets the standard for the product.
Where as it is the Project Manager’s responsibility to ensure that the things defined by QA manager are being implemented. He develops the product from start to finish with his team and ensures that the product which is to be rolled out is Defect free and it reaches the standards and views defined by QA Manager.
QA managers can audit the process for certain time periods which are being handled by the Project Manager.

What's the role of CMM Level in Testing?

What's the role of CMM Level in Testing?

Capability Maturity Model (CMM) is a model of 5 levels of process 'maturity' that determine effectiveness in delivering quality software. The 5 levels of CMM are described as below:
Level 1: Initial: - characterized by chaos, periodic panics, and heroic efforts required by individuals to successfully complete projects. Very few or none of the processes are in place.
Level 2: Repeatable: - Software Project Tracking, Requirements Management, Realistic Planning and Configuration Management Processes are in place; successful practices can be repeated.
Level 3: Defined: - Standard Software Development and Maintenance Processes are integrated throughout an organization. A Software Engineering Process Group is in place to oversee software processes, and training programs are used to ensure understanding and compliance.
Level 4: Managed: - Metrics are used to track productivity, processes, and products. Project performance is predictable, and quality is consistently high.
Level 5: Optimizing: - The focus is on continuous process improvement. The impact of new processes and technologies can be predicted and effectively implemented when required.
Any organization can start from any level, but its motto is to reach level 5; where the focus is continuous process improvement. By doing this, high quality s/w delivery is assured.
From this, we can see that the whole essence of CMM or CMMI is to produce quality software. It targets the whole organizational practices (or processes), which are believed to be the best across industries. Testing is part of Quality Assurance. CMM levels play an important role in an organization’s Quality Assurance effort. Thus testing plays an important role in determining CMM level.

How do you develop a test plan and schedule?

How do you develop a test plan and schedule? Describe bottom-up and top-down approaches.

A test plan is contract between the testers and the project team describing the role of testing in the project. The purpose of test plan is to prescribe the scope, approach, resources and schedule of the testing activities. To identify items being tested, the feature to be tested, the testing task to be performed, the personnel responsible for each task and the risks associated with the plan. From this, it is imperative that test plan is made by taking inputs from the product development team, keeping in consideration the project deadlines and risks involved while testing the product or components of the product.
The steps in creating test plan are:
1. Identifying requirements for Test: This includes tests for Functionality, Performance, and Reliability etc.
2. Assess Risk and Establish Test Priorities: In this step risks are identified and risk magnitude indicators (high, medium, low) are assigned.
3. Develop Test Strategy: This includes following things:
i. Type of test to be implemented and its objective
ii. Stage in which test will be implemented
iii. Test Completion Criteria
When all these 3 steps are completed thoroughly, a formal document is published stating above things which is known as “Test Plan”.
Bottom up Integration Testing:
The program is combined and tested from the bottom of the tree to the top. Each component at the lowest level of the system hierarchy is tested individually first, then next component is to be tested. Since testing starts at the very low level of software, drivers are needed to test these lower layers. Drivers are simply programs designed specifically for testing that make calls to these lower layers. They are developed for temporary use and need to be replaced when actual top level module is ready.
Eg: Consider a Leave Management System. In order to approve leave, there has to be a module to apply leave. If this module for apply leave is not ready, we need to create a driver (which will apply for leave) in order to test the approve leave functionality.
Top down Integration Testing:
Modules are tested by moving downwards in control hierarchy, beginning with main control module. A module being tested may call another that is not yet tested. For substituting lower modules, stubs are used. Stubs are dummy modules developed to test the control hierarchy. Stubs are special purpose programs that simulate the activity of missing component.
Eg: In Leave Management System, once leave is approved, the leave status can be seen in leave report. So we need to create a dummy implementation of a leave report (stub).

Bug leakage and bug release

What is "bug leakage?" and what is "bug release?"

A bug leakage results when a bug is detected which should have been detected in earlier builds/versions of the application.
A defect which exists during testing yet unfound by the tester which is eventually found by the tester/end-user is also called bug leakage.
A bug release is when a particular version of s/w is released with a set of known bug(s)/defect(s). These bugs are usually low severity and/or low priority bugs. It is done when the company can afford the existence of bug in the released s/w rather than the time/cost for fixing it in that particular version. These bugs are usually mentioned in the Release Notes.

Bug Leakage

Testing - Key challenges of software testing

What are the key challenges of software testing?

Key Challenges of s/w testing:
1. Testing considered late in project
2. Requirements not testable
3. Integration is done after all components have been developed: This might result into full testing not being covered.
4. Complete testing is not possible

What are the roles of glass-box and black-box testing tools?

Glass Box (or white box) testing is the process of giving i/p to the system and checking how the system processes i/p to generate o/p
Black Box testing is the process of giving i/p to the system and checking if the system is giving correct o/p without bothering how the o/p is generated.
As we can see from the definitions, the role of black box testing is to ensure that the o/p generated is correct. And role of white box testing is to ensure that methods used to generate the o/p are correct.

What is the difference in writing the test cases for Integration testing and system testing?

Integration testing is done at module level when various modules are integrated with each other to form a system or sub-system. Its main purpose is to ensure that interfaces between various modules are working properly; i.e. modules which are working individually are also working correctly together.
System testing is done on a complete, integrated system to evaluate the system’s compliance with its specified requirements. It validates that the system meets its functional and non-functional requirements.
From these definitions, it is clear that the purpose of Integration and purpose of System Testing are different. Therefore, Integration test cases focus more on the interfaces between modules (interface integrity) - the data transfer and their interaction with each other. System test cases focus on testing the product as a whole; i.e. whether the functional, non-functional requirements of the System are met or not. Since System Testing is the final phase before delivery of the product, System test cases should pinpoint configuration related errors along with testing for performance, security, reliability etc.

What methodologies have you used to develop test cases?

I have used following 4 types of Methodologies:
1. Boundary value analysis
2. Equivalence partitioning
3. Error guessing
4. Cause effect graphing

common problems with software automation

What are the common problems with software automation?

The biggest concern is the cost incurred for test automation. Also, often s/w test automation requires skilled personnel and authentic test automation tools. Purchasing license of such tools is a costly affair.
Also the time required for test automation is more. Automation often requires recording the sequence of operations you need to perform in order to execute a particular test case and comparing the final o/p with the expected o/p. Functions/macros need also be written sometimes for some test cases. All this could be a time consuming job.

What are the parameters of Quality cost?

The parameters of Quality Cost are:
Speed
Accuracy
Efficiency
Performance
Customer Satisfaction

What is the role of QA in a project development?

The word assurance means ‘guarantee’. So the Quality Assurance Group’s role is to guarantee that the product is of high quality. They examine the overall s/w development process and their responsibility is to create and enforce standards and methods to improve it with the goal of preventing bugs from ever occurring.

Benefits of Software Quality Assurance to the organization

The main task of Software Quality Assurance Group is to examine the overall s/w development process and to create and enforce standards and methods to improve it with the goal of preventing bugs from ever occurring. With this definition, it is imperative that the QA helps an organization in continuous performance improvement and strive for perfection.

What is the value of a testing group? How do you justify your work and budget?

Testing is a process used for revealing defects in s/w and for establishing that the s/w has attained a specific degree of quality.
The s/w testing group’s main objective is to ensure that the s/w is doing what it is supposed to do as well as the s/w is not doing what it is not supposed to do. With this objective, it is clear that it is necessary to have a separate group for testing the s/w other than the group which developed it.
Also, bugs in the s/w can cause severe monetary as well as other loss to the organization. So it is important to allocate some budget for the testing group.

 

How to read XML file from QTP

I'm going to show how to parse XML file and read its values from QTP (QuickTest Professional).
For example, we will use this XML file:

This XML file describes a bookstore and books located there. 

Note: You can download this XML file here.

I will use Microsoft's XML Parser also known as Microsoft.XMLDOM. This XML parser allows running XPath queries.

The loading of XML file in QTP is simple enough:

Const XMLDataFile = "C:\TestData.xml"

Set xmlDoc = CreateObject("Microsoft.XMLDOM")

xmlDoc.Async = False

xmlDoc.Load(XMLDataFile)

Several comments on this code:

1. Microsoft.XMLDOM is a name of COM object of Microsoft's XML parser
2. Async is a property of Microsoft.XMLDOM.
   The property specifies whether asynchronous download of the document is permitted.
   Processing of asynchronous operations is more complex, that's why I've disabled it (xmlDoc.Async = False). For datailed info see here.
3. Load method loads an XML document from the specified file.
   Also, you can use LoadXML method, which loads an XML document using the supplied string.

After that we can use SelectSingleNode or SelectNodes of Microsoft's XML parser to execute XPath query.

You can use this approach in QTP to get data from XML file.

Check the above bookstore XML file again and let's see:

1. How to get number of books in a bookstore?
   The QTP script is:
   Set nodes = xmlDoc.SelectNodes("/bookstore/book")

   
   

   MsgBox "Total books: " & nodes.Length
   And its result is:
   
   As you can see, "/bookstore/book" XPath expression selects all "book" nodes in a "bookstore" nodes. As a result, we get the list of all books.
   
   
   I hope, that's clear. Let's complicate the task a bit.
   
   
   
   
2. How to get titles of all books in a bookstore?
   QTP script gets list of books from XML file and iterates them through:
   ' get all titles

   Set nodes = xmlDoc.SelectNodes("/bookstore/book/title/text()")

   
   

   
   

   ' get their values

   
   

   For i = 0 To (nodes.Length - 1)

   
   

       Title = nodes(i).NodeValue

       MsgBox "Title #" & (i + 1) & ": " & Title

   Next
   The result is:
   
   
   
   
3. How to get title of the first book?
   QTP script is:
   Set node = xmlDoc.SelectSingleNode("/bookstore/book[0]/title/text()")

   
   

   MsgBox "Title of 1st book: " & node.NodeValue
   Please see how QTP parses XML file and gets the required value:
   
   Note: Pay attention that nodes are zero-indexed, i.e. first book is book[0], second book is book[1], and so on.
   
   
   
   
4. How to get titles of all John Smith's books?
   QTP script is:
   ' get list of John Smith's books 

   
   

   Set nodes = xmlDoc.SelectNodes("/bookstore/book/title[../author = 'John Smith']/text()")

   
   

   
   

   
   

   ' get their titles

   
   

   For i = 0 To (nodes.Length - 1)

   
   

       Title = nodes(i).NodeValue

       MsgBox "Title #" & (i + 1) & ": " & Title

   Next
   Note: We use square brackets to apply a filter. So, [../author = 'John Smith'] means 'to get only those books whose author is John Smith'.
   
   
   Since Mr. Smith wrote two books, we get the following result:
   
   
   
   
   
5. How to get titles of all books published after 2003?
   QTP script is:
   ' get list of books published after 2003

   
   

   Set nodes = xmlDoc.SelectNodes("/bookstore/book/title[@published > 2003]/text()")

   
   

   
   

   
   

   ' get their titles

   
   

   For i = 0 To (nodes.Length - 1)

   
   

       Title = nodes(i).NodeValue

       MsgBox "Title #" & (i + 1) & ": " & Title

   Next
   Note: And again, we use square brackets to apply a filter - [@published > 2003].
   At sign (@) is used to address an attribute of a node. For example:
   
   Here, title node contains one attribute - published.
   So, [@published > 2003] means 'to get only those books which were published after 2003'.
   
   
   The result of above QTP script is:

You can check initial XML file and make sure that QTP parses it correctly.

Summary:

• Use Microsoft.XMLDOM to read and parse XML file from QuickTest Professional (QTP)
• XPath allows creating different queries to extract required data
• The present visual tutorial explains how to process XML file from QTP

LoadRunner Analysis - tabbed world

Do you like this dialog from LoadRunner Analysis? Not too many tabs? :)

No words!

It seems, HP should hire a professional UI designer :)