:: QC Boss :: Testing, Independent Software Testing, Manual Testing, Website Testing, Functionality Testing, Usability Testing, QC, QA, UAT

829 Standard for Software Test Documentation, is an IEEE standard that specifies the form of a set of documents for use in eight defined stages of software testing, each stage potentially producing its own separate type of document. The standard specifies the format of these documents but does not stipulate whether they all must be produced, nor does it include any criteria regarding adequate content for these documents. These are a matter of judgment outside the purview of the standard. The documents are:

Test Plan: a management-planning document

• How the testing will be done (including SUT configurations).
• Who will do it?
• What will be tested?
• How long it will take (although this may vary, depending upon resource availability).
• What the test coverage will be, i.e. what quality level is required

Test Design Specification

• Explaining test conditions and the expected results as well as test pass criteria.

Test Case Specification

• Specifying the test data for use in running the test conditions identified in the Test Design specification

Test Procedure Specification

• Detailing how to run each test, including any set-up preconditions and the steps that need to be followed

Test Item Transmittal Report

• Reporting on when tested software components have progressed from one stage of testing to the next

Test Log

• Recording which tests cases were run, who ran them, in what order, and whether each test passed or failed

Test Incident Report

·         Detailing, for any test that failed, the actual versus expected result, and other information intended to throw light on why a test has failed. This document is deliberately named as an incident report, and not a fault report. The reason is that a discrepancy between expected and actual results can occur for a number of reasons other than a fault in the system. These include the expected results being wrong, the test being run wrongly, or inconsistency in the requirements meaning that more than one interpretation could be made. The report consists of all details of the incident such as actual and expected results, when it failed, and any supporting evidence that will help in its resolution. The report will also include, if possible, an assessment of the impact upon testing of an incident.

Test Summary Report

·         A management reports providing any important information uncovered by the tests accomplished, and including assessments of the quality of the testing effort, the quality of the software system under test, and statistics derived from Incident Reports. The report also records what testing was done and how long it took, in order to improve any future test planning. This final document is used to indicate whether the software system under test is fit for purpose according to whether or not it has met acceptance criteria defined by project stakeholders.

• Review all requirements and design documents.
• Attend system reviews presented by Development and Analysis Team members.
• Create and maintain a detailed System Test Project Plan.
• Divide the FRAM Requirements into logical groupings or scenarios. These scenario should reflect the business or operational approach to the system.
• Define any necessary regression tests.
• Create a System Test Plan.
• Where possible, create scripts to automate the execution of a test case.
• Ensure the System Test Plan is reviewed by appropriate parties (Development and Quality Assurance).
• Verify that the System Test Environment has been created.
• Conduct the test as specified in the test cases.
• Identify any problems that are encountered or where the actual results do not agree with the defined expected results and complete a Problem Report.
• Record in the Tracking document the steps executed, relevant PRs, and test cases completed.
• Once all problems have been resolved, re-run the necessary tests.
• Update test plans after the testing is complete.
• Produce Post Project System Testing Reports

Testing Process should start straight away when the requirements & wireframe of the software application is finalised.

So that any serious flaws which may come due to some misinterpretation/ignorance of customer requirements, can be avoided.

Testing should begin with the design phase of a project and continue all the way through. Software testing is ultimately about validating the design requirements, but to be effective at that, the testing process must begin with the design stage itself.

When testing is started later, it can be less effective. Designers must sync with testers, developers must sync with testers, and valuable time is wasted getting teams and technology expectations to mesh. This is a recipe for delays, confusion, and budget overruns.

Starting the testing process with design lets testers be involved from the beginning. They can contribute early, get to know the product early, and be involved early. That makes for better testing and that makes for a better product.

Some useful points/tips to follow while testing software

Check whether you’re testing the latest version of the software application

Try to do unusual stuff with the software or test it in unusual environments, such as especially slow hardware or uncommon platforms.

Check for duplicates before filing a bug in bug report

Include enough information in your report so the issue can be reproduced

Your impressions are critical – report everything you see

Describe as completely as possible the conditions leading up to a fault

Don’t apologize for your language

Use a bug tracking system of some sort

Become familiar with the tools that will be used to compile, link, and test the code

If possible, write automated unit tests

If possible, use the code you’re testing under real-life conditions

If possible, maintain a separate environment for testing

Keep a few backup of the earlier versions of application to track when errors appear

And last but not the least, be little bit patient with developers

 

• It is the policies that define the objective for the workbenches.

• The objective of the workbench is to produce the defined output products in a defect-free manner. The procedures and standards are designed to assist in this objective.

• The worker performs defined procedures on the input products in order to produce the output products. The procedures are step-by-step instructions that the worker follows in performing his/her tasks.

• The standards are the measures that the worker uses to validate whether or not the products have been produced according to specifications. If they meet specifications, they are quality products or defect-free products. If they fail to meet specifications or standards they are defective and subject to rework.

• If tools are to be used, they are incorporated into the procedures. Tool usage becomes mandatory, not optional.

All pages should have links external to the page; there should be no dead-end pages.

What are the expected loads on the server (e.g., number of hits per unit time?), and what kind of performance is required under such loads (such as web server response time, database query response times).

What kinds of tools will be needed for performance testing (such as web load testing tools, other tools already in house that can be adapted, web robot downloading tools, etc.)?

Who is the target audience? What kind of browsers will they be using? What kinds of connection speeds will they by using? Are they intra- organization (thus with likely high connection speeds and similar browsers) or Internet-wide (thus with a wide variety of connection speeds and browser types)?

What kind of performance is expected on the client side (e.g., how fast should pages appear, how fast should animations, applets, etc. load and run)?

Pages should be 3-5 screens max unless content is tightly focused on a single topic. If larger, provide internal links within the page.

The page layouts and design elements should be consistent throughout a site, so that it’s clear to the user that they’re still within a site.

Pages should be as browser-independent as possible, or pages should be provided or generated based on the browser-type.

A method aiming to measure the size of the functionality of an information system. The measurement is independent of the technology. This measurement may be used as a basis for the measurement of productivity, the estimation of the needed resources, and project control.

Static Testing

The Static testing refers to testing that is not running. It is analyzing and reviewing it. The specification is a document and not an executing program, so it’s considered as static. It’s also something that was created using written or graphical documents or a combination of both.

High-level Reviews of specification

• Pretend to be the customer.
• Research existing Standards and Guidelines.
• Review and Test similar software.

Low-level Reviews of specification

• Specification Attributes checklist.
• Specification terminology checklist.

Dynamic Testing

Techniques used are determined by type of testing that must be conducted.

• Structural (usually called “white box”) testing.
• Functional (“black box”) testing.

Structural testing or White box testing

The Structural tests check the structure of the software itself and require full access to the source code. This is known as ‘white box’ testing because you are into the internal workings of the code. White Box tests make sure that the software structure itself contributes to proper and efficient program execution. Complicated loop structures, common data areas, 100,000 lines of spaghetti code and nests of ifs are evil. Well-designed control structures, sub-routines and reusable modular programs are good.

White Box testing strength is also its weakness. The code needs to be examined by highly skilled technicians. That means that tools and skills are highly specialized to the particular language and environment. Also, large or distributed system execution goes beyond one program, so a correct procedure might call another program that provides bad data. In large systems, it is the execution path as defined by the program calls, their input and output and the structure of common files that is important. This gets into a hybrid kind of testing that is often employed in intermediate or integration stages of testing.

Functional or Black Box Testing

Black Box (or Functional Testing) testing examines the behavior of software as witnesses by its outputs without reference to internal functions. Thus it is also called ‘black box’ testing. If the program consistently provides the desired features with acceptable performance, then specific source code features are irrelevant. It’s a pragmatic and down-to-earth assessment of software. Black Box (or Functional Testing) testing better addresses the modern programming paradigm. As object oriented programming, automatic code generation and code re-use becomes more prevalent, analysis of source code itself becomes less important and functional tests become more important. 

The Black box tests also better attack the quality target. Since only the people paying for an application can determine if it meets their needs, it is an advantage to create the quality criteria from this point of view from the beginning. Black box tests have a basis in the scientific method. Like the process of science, Black box tests must have a hypothesis (specifications), a defined method or procedure (test plan), reproducible components (test data), and a standard notation to record the results. One can re-run black box tests after a change to make sure the change only produced intended results with no inadvertent effects.

• In any project 50 % of the project costs goes in testing related activities.
• The cost to company in solving errors reported by the client is huge.
• The legal implications following the delivery of an under tested product is unimaginable.