Software Testing framework

Software Testing framework

Testing plays an important role in today's System Development Life Cycle. During Testing, we follow a systematic procedure to uncover defects at various stages of the life cycle.

This framework is aimed at providing the reader various Test Types, Test Phases, Test Models and Test Metrics and guide as to how to perform effective Testing in the project.
Here is an outline of the Framework:

1. Introduction.

2. Varification and Validation Strategies.

3. Testing Types.

4. Test Phases.

5. Metrics.

6. Test Models.

7. Defect Tracking Process.

8. Test Process for a Project

9. Deliverables.

Bug Life Cycle

Life cycle of a BUG:

As defects move through the system they are given various states. At each state there are a number of possible transitions to other statesThe following figure is the life cycle of a bug

to ASSIGNED by acceptanceto RESOLVED by analysis and maybe fixingto NEW by reassignment

ASSIGNED The owner, i.e. the person referenced by Assigned-To has accepted this bug as something they need to work onto NEW by reassignmentto RESOLVED by analysis and maybe fixing

REOPENED Was once resolved but has been reopenedto NEW by reassignmentto ASSIGNED by acceptanceto RESOLVED by analysis and maybe fixing

RESOLVED Has been resolved (e.g. fixed, deemed unfixable, etc. See "resolution" column)to REOPENED by reopeningto VERIFIED by verificationton CLOSED by closing

VERIFIED The resolution has been approved by QAto CLOSED when the product shipsto REOPENED by reopening

CLOSED Over and done withto REOPENED by reopening

Difference between Bug Tracking and Testing

Bug Tracking:

Receiving and filing bugs reported against a software project, and tracking those bugs until they are fixed. We use BTS ( Bug tracking System) to track them properly. Most major software projects have their own BTS, the source code of which is often available for use by other projects.

Testing:

Software Testing is the process used to help identify the correctness, completeness, security, and quality of developed computer software.

Quality Assurance Software Testing levels:

Testing is applied to different types of targets, in different stages or levels of work effort. These levels are distinguished typically by those roles that are best skilled to design and conduct the tests, and where techniques are most appropriate for testing at each level. It's important to ensure a balance of focus is retained across these different work efforts.

Developer Testing
Developer testing denotes the aspects of test design and implementation most appropriate for the team of developers to undertake. In most cases, test execution initially occurs with the developer-testing group who designed and implemented the test, but it is a good practice for the developers to create their tests in such a way so as to make them available to independent testing groups for execution.

Independent Testing
Independent testing denotes the test design and implementation most appropriately performed by someone who is independent from the team of developers. In most cases, test execution initially occurs with the independent testing group that designed and implemented the test, but the independent testers should create their tests to make them available to the developer testing groups for execution.

The other levels include:
Independent Stakeholder Testing- testing that is based on the needs and concerns of various stakeholders
Unit testing - Unit testing focuses on verifying the smallest testable elements of the software.
Integration testing - to ensure that the components in the implementation model operate properly when combined to execute a use case.
System Testing - Usually the target is the system's end- to-end functioning elements
Acceptance Testing - to verify that the software is ready, and that it can be used by end users to perform those functions and tasks for which the software was built.

Key Measures of Quality Assurance Software Testing

The key measures of a test include Coverage and Quality. Test Coverage is the measurement of testing completeness. It is based on the coverage of testing expressed by the coverage of test requirements and test cases or by the coverage of executed code. Test coverage includes requirements based coverage and code based coverage. Quality is a measure of the reliability, stability, and performance of the target-of-test (system or application-under-test). Quality is based on evaluating test results and analyzing change requests (defects) identified during testing.

Advantages of Quality Assurance Software Testing

The most effective way to reduce risk is to start testing early in the development cycle and to test iteratively, with every build. With this approach, defects are removed as the features are implemented. The testing of the application is completed shortly after the final features are coded, and as a result the product is ready for release much earlier. Additionally, the knowledge of what features are completed (i.e. both coded and tested) affords management greater control over the entire process and promotes effective execution of the business strategy. Testing with every iteration may require some additional upfront planning between developers and testers, and a more earnest effort to design for testability; but these are both inherently positive undertakings, and the rewards are substantial.
There are several key advantages gained by testing early and with every build to close the quality gap quickly:
Risk is identified and reduced in the primary stages of development instead of in the closing stages. ·
Repairs to problems are less costly.
The release date can be more accurately predicted throughout the project.
Results will be given by the way of requirement.
The product can be shipped sooner.
The business strategy can be executed more effectively.
Transparency established.
Artifacts can be reused for regression testing.
Not bound to any particular vendor.

Software Testing Procedure

A good test effort is driven by questions such as:
How could this software break?
In what possible situations could this software fail to work predictably?
Software testing challenges the assumptions, risks, and uncertainty inherent in the work of other disciplines, and addresses those concerns using concrete demonstration and impartial evaluation.
Testing focuses primarily on evaluating or assessing product quality, which is realized through the following core practices:
Find and document defects in software quality.
Advise on the perceived software quality.
Validate and prove the assumptions made in design and requirement specifications through concrete demonstration.
Validate that the software product works as designed.
Validate that the requirements are implemented appropriately.

The Need for Quality Assurance Software Testing

There are two major industry trends adding to the pressure. The first is accelerated release cycles. As Business Week put it, "A year's worth of change happens in a couple of months-a pace known as 'Internet Time.' .And that's the problem. The whole industry is operating on Internet Time...". Second, while releases are more frequent and cycles shorter, the cost of failure has increased dramatically. Just a few years ago, when client server products were at the cutting edge, releases were, perhaps, annual and the expected number of users was known well in advance, because all the users were employees. An organization could mitigate a system failure with a manual backup; for example, orders could be taken by hand while the system was down.
But as huge portions of the business were overhauled, and these systems addressed larger user populations, releases became more frequent, and system failures commonly meant that no orders could be taken at all. Today with E-commerce applications, releases can occur two or three times per month. Now, the user base is a large but unknown number of customers, not employees. System failures are highly visible and can cause customers to run to the competition.
These trends have several serious implications for project managers. The high cost of failure means that deploying untested software is simply not an option. Additionally, every aspect of quality needs validation, not just one or two. Accelerated release cycles drive the need for automated testing in which tests are easy to create, maintain and reuse. There is just too much to do in too little time to rely on manual methods.
A continuous approach to quality, initiated early in the software lifecycle, can lower the cost of completing and maintaining the software significantly. This greatly reduces the risk associated with deploying poor quality software.

The Importance of Quality Assurance Software Testing

Software Quality Assurance Testing is an integral and critical phase of any software development project. Developers and testers must ensure that newly developed products or product enhancements meet functional and performance requirements and that those products are reliable and able to operate consistently under peak loads. The risks of releasing a product that is not yet ready are greater now than ever before because of the change in expectations and demands of the end users.
According to the Standish group's research report on Project Failure and Success nearly three out of four software projects are either delivered late, over budget or are cancelled before being completed. This is true despite the involvement of experienced managers, developers and testers in the project. This is a problem that continues till date. The anxiety induced by the question "Are we ready to release?" affects every member of the team. Management dreads having to ask it for fear of hearing an unqualified "No" or even worse.

Test Director Related Interview Questions

Q: What is TestDirector?
TestDirector is a test management tool produced by Mercury Interactive. Its four modules - Requirements, Test Plan, Test Lab and Defects Manager - are integrated to enable information to flow smoothly between different stages of the testing process. Completely Web-enabled, TestDirector supports communication and collaboration among distributed testing teams. TestDirector has been classified in the following categories:Defect TrackingTesting and AnalysisDebuggingAutomated Software Quality (ASQ)

Q: What is the use of Test Director software?
TestDirector is Mercury Interactive's software test management tool. It helps quality assurance personnel plan and organize the testing process. With TestDirector you can create a database of manual and automated tests, build test cycles. TestDirector is not used for executing any actual test activity but it is a test management tool used for Requirements Management, Test Plan, Test Lab, and Defects Management. Even if the individual test cases are not automated, TestDirector can make life much easier during the test cycles.

Q: How you integrated your automated scripts with TestDirector?
When you work with WinRunner, you can choose to save your tests directly to your TestDirector database or while creating a test case in the TestDirector we can specify whether the script in automated or manual. And if it is automated script then

Q: Is there any possibility to restrict duplication of defects being created in TD?
No Way. The only thing we can do is to find the similar defects and delete or close them.

Q: What is Quality Center( Test Director)?
You can use Quality Center to create a project (central repository) of manual and automated tests and components, build test cycles, run tests and components, and report and track defects. You can also create reports and graphs to help you review the progress of test planning, runs, and defect tracking before a software release. When you work in QuickTest, you can create and save tests and components directly to your Quality Center project. You can run QuickTest tests or components from Quality Center and then use Quality Center to review and manage the results. You can also use Quality Center with Business Process Testing support to create business process tests, comprised of the components you create in either QuickTest or Quality Center with Business Process Testing support.
Q: After creating the test cases in excel and exported to TD. How does test director know the headings?
To export the test cases from spreadsheet to TD there are 8 steps. In 6th step we need to map the Td fields with corresponding Spreadsheet columns. Hence you are the mapping so you can map according to your specifications.

Q: How to use TestDirect like a Dashboard?
The new version of TD (TestDirector for Quality Center) should provide you with that. If you not want to upgrade, you have to design your own "start page", include the apps and bits you want to display, and use code to extract data from TD.

Q: Can you retrieve a test case once you have deleted them in Test Director ?
In Quality Center, if you delete entire folder with tests, the tests get stored in Unattached folder. But if you just deleted 'individual' tests, they are GONE! and can't be retrieved. Not sure if Test Director has the same behaviour/functionality. There are 2 options in QC. one is remove and another is delete. the diffrence is that .once it is remove it is removed from test set but avail in QC directory. If it is deleted, you can't retrive. If you press delete that will delete from that QC directory also.

Q: How do we import testcases written in Excel to Test Director
Use Mecury Interactive Microsoft Excel Add-in for importing test cases written in excel sheet. It is available on Add-ins page. Select the rows in excel which you want to upload to TD Then select export to TD option under tools menu

Q: Is it necessary to learn Test Director for beginners
Test director is a test mangement tool, it is used across all major organizations and is generally used for management of all test activities in organization. It is important to learn this tool, but for beginners it is enough to understand how to log defects into it and how to run tests using it.

Q: Can you please explain the procedure of connecting TestDirector in QTP?
To connect to TD from QTP follow the steps...Open Qtp ==> Tools ==> Select TestDirector Connection ==> In Server Connction Box Enter TD address(URL of TD) ==> Click Connect==> In project Connection Box Enter the Details Domain,Project,User name and Password ==> Click Connect If you want to reconnect on startup check the reconnect on startup and save password for reconnection on startup. Then close.

Q: What are the various types of reports in TestDirector?
For each and every phase we can get reports, like for requirements, test cases, and test run. There are some types of reports also available like report summary, progress report and requirements coverage report. Each and every test director client tool consists of a menu bar Analysis. By using this menu you can crate reports in table format. You can generate graphs. All graphs options in maths are supported. And you can create various types of charts too.

Q: TD (Quality Center 9.0) how can you run automated test cases?
While designing your test steps in QC for automation tests in test plan module, Test Script tab is availble. You can generate script here or copy from your automatioon tool. While running your tests, it will ask for on which host you want to run. You need to select the system in your network. Then run it. Before going to run your script in a system, the automation tool, like WinRunner, must be installed on that system. Otherwise you will get an error.

Q: Can we add user defined fields to Test Director?
Yes. We can add the user defined fields using TD 8.0, But you need to have admin priviliges to this.

Q: How do we attach Excel sheet with test director?
This function is for getting datatable(excel sheet) in test director. Try to use it and as vbs file and call this function to get ur datatable. GetAttachment(FileName, OutPath) FileName The name of the attachment that needs to be copiedOutPath The folder location where the file needs to be storedReturn value The full path where the file has been copied on the local file systemExample:FilePath = GetAttachment("test.pdf", "C:")MsgBox "Your file is here:" & FilePathThe GetAttachmentFromTest finds the attachment associated to the given test name and stores it in a local folder.GetAttachmentFromTest(TestName, FileName, OutPath)TestName The name of the test where the attachment is locatedFileName The name of the attachment that need to be copiedOutPath The folder location where the file needs to be storedReturn value The full path where the file has been copied on the local file systemExample:FilePath = GetAttachmentFromTest("Attachment", "hello.vbs", "C:aa")MsgBox "Your file is here:" & FilePath

Q: What is the use of Test Lab in Test director?
Test Lab can be used to create a test set. You can add one or many test cases into a test set. Then run all test cases in a test set together and change the status as pass/fail.

Q: Can we map the Defects directly to the requirements (not thru the test cases) in the Test Director?
Yes. Create your req. structure.Create the test case structure and the test casesMap the testcases to the apr. req.Run and report bugs from your test cases in the test lab module.The database structure in TD is mapping testcase to defects, only if you have created the bug from the apr. test case. Maybe you can update the mapping by using some code in the bug script module (from the customize project funktion), as fare as I know, its not possible to map defects directly to an req.

Q: How do I run reports from Test Director?
This is how you do it:1. Open the test director project.2. Display the requirements module.3. Chose reportAnalysis > Reports > Standard Requirements Report.

Q: Can we export the files from Test director to Excel Sheet? If yes then how?
Design tab -- Right click -> go to save as -> select excel and save it Requirement tab -- Right click on main req/ click on export/ save as word, excel or other template. This would save all the child requirement. Test plan tab-- only individual test can be exported. No parent--child export is possible.Select a test script. click on the design steps tab. right click anywhere on the open window. click on export and save as.... Test lab tab-- select a child group. Click on execution grid if it is not selected. right click anywhere . default save option is excel. but can be saved in doc and other formats. select 'all' or 'selected' option. defects tab -- right click anywhere on the window, export all or 'selected' defects and save excel sheet or document

Q: Can we upload test cases from an excel sheet into Test Director?
Yes, you can do that. Go to Add In menu in TestDirector, find the Excel add in, and install it in you machine. Now open excel, you can find the new menu option export to Test director. Rest of the procedure is self explanatory

Q: How can we map a single defect to two test scripts? Is there a way in test director so that we can state that defect defect X is same for test script A and test script B?
No way. When you run a script, you find and generate a defect report. In other words, every defect report is unique to a single test script.

Q: How can we create our own Defect Template from Test Director? Is it possible in Test Director? If possible how we can Create our Own Template?
You can not create your own template for defect reporting in Test Director but you can customize the Template in Test Director

Q: How can we export multiple test cases from TD in a single go?
Open any test and click on the tab 'design step'. Once it opens, you can right click on the first cell and export into any format .

Q: How to customize the reports generated?
This depends a lot of what you are interested in "reporting on". You have to combine both SQL and VB script to extract data from TD to Excel. Its also possible to "customize" the standard reports given from the "analyze" tab, this is written in XML if you are familiar with this language. If you log in to Mercury support you will be able to find a lot of code examples.

Q: How many tabs in TestDirector and explain them?
there are 4 tabs available in Testdirector.1. Requirement -> to track the customer requirenments2. Testplan -> to design the testcases & to store the testscripts3. Testlab -> to exectue the testsets & track the results4. Defect -> to log a defect & to track the logged defects

Q: How to map requirements with testcases in TestDirector?
1. In requirements TAB select coverage view.2. Select requirement by clicking on Parent/Child or grand Child.3. On right hand side (in Coverage View Window) another window will appear. It has two TABS (a) Tests Coverage (b) Details. Test Coverage TAB will be selected by default or you click it.4. Click on Select Tests Button. A new window will appear on right hand side and you will see a list of all tests. You can select any test case you want to map with you requirement.

Q: How to use TestDirector in real time projects?
Once completed the preparing of the test cases.1. Export the test cases in to Test Director. ( it will contain total 8 steps).2. The test cases will be loaded in the Test Plan module.3. Once the execution is started, we move the test cases from Test Plan tab to the Test Lab module.4. In Test Lab, we execute the test cases and put as pass or fail or incomplete. We generate the graphs in the test lab for daily report and sent to the onsite (where ever you want to deliver).5. If you got any defects and raise the defect in the defect module. When raising the defect ,attach the defect with the screen shot.

Q: How can we add requirements to test cases in Test Director?
You can add requirements to test cases in two ways; either from Requriements tab or Test Plan tab. Navigate to the appropriate Requirement and right click, you can find the menu to map the test case and the vice versa is available in Test plan tab.

Q: What does Test Grid contains ?
The Test Grid displays all the tests in a TestDirector project. The Test Grid contains the following key elements: Test Grid toolbar, with buttons of commands commonly used when creating and modifying the Test Grid. Grid filter, displaying the filter that is currently applied to a column. Description tab, displaying a description of the selected test in the Test Grid. History tab, displaying the changes made to a test. For each change, the grid displays the field name, date of the change, name of the person who made the change, and the new value.

Q: How to generate the graphs in Test Director ?
An1:Open test director and then click the Analysis you will find three type of graphs Planning Progress Graphs Planning Summary Graphs Defect Age Graph Click any of one and you can generate the graphs..
An2:The generation of graphs in the Test Director that to Test Lab module is :1. Analysis2. Graph3. Graph Wizard4.Select the graph type as Summary and click the Next button.5.Select the show current tests and click the next button.6.Select the Define a new filter and click the Filter button.7. Select the test set and click the Ok button.8.Select the Plan : subject and click the ok button.9. Select the Plan: Status10 Select the test set as x- Axis11. Click the Finish button.

Q: What is the difference between Master test plan and test plan?
Master test plan is the doccumaent in which each and every functional point is validated. Test case docuument contains test cases, Test case is the perception with which probability of finding the defect is more.

Q: What is the main purpose of storing requirements in Test Director?
In TestDirector(Requirement Tab) We Stores Our Project Requirement documents according to our modules or functionality of the applications. This helps us to makes sures that all requirements are covered when we trace developed Test Case/Test Script to the requirements. This helps QA Manager to review what extent the requirements are covered.

Q: What are the 3 views and what is the purpose of each view?
The 3 views of requirment are:1)Document View-tabulated view2)Coverage View-establish a relationship between requirement and the test assosiated with them along with their execution status.Mostly the requirements are written in this view only.3)Coverage analysis view-show a chart with requirementassociated with the test,and execution status of the test.

Q: How many types of reports can be generated using TestDirector?
Reports on TestDirector display information about test requirements, the test plan, test runs, and defect tracking. Reports can be generated from each TestDirector module using the default settings, or you can customize them. When customizing a report, you can apply filters and sort conditions, and determine the layout of the fields in the report. You can further customize the report by adding sub-reports. You can save the settings of your reports as favorite views and reload them as needed.

Q: How will you generate the defect ID in test director? Is it generated automatically or not?
The Defect ID will be generated automatically after the submission of the defect..

Q: How do you ensure that there are no duplication of bugs in Test Director?
In the defect tracking window, at the top we can see the find similar defect icon. If we click after writing our defect, if any of the tester already added the similar defect it will tell. Else we can add.

Q: Difference between WinRunner and Test Director?
WinRunner: Its an Automation Testing tool, used for automation of manual written Test Cases to Test Scripts and Regression Test also. Test Director: Its an Testing Management tool, used from Creating Test Plan,Preparation of testCases, execution of testcases and generating defect report.Also used for maintaining Test Scripts.

Q: How to add Test ID to TestPlan?
Create an object with a type = Number. Name it something like "Test_ID" in the Customize Entities area. Then go into the Workflow Script Editor to "TestPlan module script/TestPlan_Test_MoveTo" and insert the following: if Test_Fields.Field("Your Object Name").Value <> Test_Fields.Field ("TS_TEST_ID").Value then Test_Fields.Field("Your Object Name").Value = Test_Fields.Field("TS_TEST_ID").Value end if This will put an object on each test thet displays the Test ID Number.