:: QC Boss :: Testing, Independent Software Testing, Manual Testing, Website Testing, Functionality Testing, Usability Testing, QC, QA, UAT

IEEE:-

· Institute of Electrical and Electronics Engineers.

· Founded in 1884.

· Have an entire set of standards devoted to Software.

· Testers should be familiar with all the standards mentioned in IEEE.

IEEE STANDARDS: That a Tester should be aware of…..

1. 610.12-1990 IEEE Standard Glossary of Software Engineering Terminology

2. 730-1998 IEEE Standard for Software Quality Assurance Plans

3. 828-1998 IEEE Standard for Software Configuration Management Plan

4. 829-1998 IEEE Standard for Software Test Documentation.

5. 830-1998 IEEE Recommended Practice for Software Requirement Specification

6. 1008-1987 (R1993) IEEE Standard for Software Unit Testing (ANSI)

7. 1012-1998 IEEE Standard for Software Verification and Validation.

8. 1012a-1998 IEEE Standard for Software Verification and Validation – Supplement to 1012-1998 Content Map to IEEE 122207.1

9. 1016-1998 IEEE Recommended Practice for Software Descriptions

10. 1028-1997 IEEE Standard for Software Reviews

11. 1044-1993 IEEE Standard classifications for Software Anomalies

12. 1045-1992 IEEE Standard for Software Productivity Metrics (ANSI)

13. 1058-1998 IEEE Standard for Software Project Management Plans

14. 1058.1-1987 IEEE Standard for Software Management

15. 1061-1998.1 IEEE Standard for Software Quality Metrics Methodology.

Other Standards:

· ISO – International Organization for Standards.

· SPICE – Software Process Improvement and Capability Determination.

· NIST - National Institute of Standards and Technology.

· DoD – Department of Defense.

The cost of quality (COQ) is the money spent beyond what it would cost to build a product right the first time. If every worker could produce defect-free products the first time, the COQ would be zero. Since this situation does not occur, there are costs associated with getting a defect-free product produced.

There are three COQ categories:

• Prevention – Money required preventing errors and to do the job right the first time is considered prevention cost. This category includes money spent on establishing methods and procedures, training workers and planning for quality. Prevention money is all spent before the product is actually built.

• Appraisal – Appraisal costs cover money spent to review completed products against requirements. Appraisal includes the cost of inspections, testing and reviews. This money is spent after the product or subcomponents are built but before it is shipped to the user.

• Failure – Failure costs are all costs associated with defective products. Some failure costs involve repairing products to make them meet requirements. Others are costs generated by failures, such as the cost of operating faulty products, damage incurred by using them and the costs incurred because the product is not available. The user or customer of the organization may also experience failure costs.

Experience has shown that a small group of knowledgeable people can develop estimates for the COQ categories. The estimate does not have to be highly precise because the amounts will be so large that even errors of plus or minus 50% would not affect identifying the actions that need to be taken to reduce the COQ.

The cost of building a product is comprised of the cost of production, which is the cost if the product could be built defect free, plus the three COQ categories. Added together, the production cost and COQ become the cost to build a product. The three COQ categories are sometimes called the cost of nonconformance, meaning COQ is the failure to conform to a process that enables defect-free products to be produced.

The quality function attempts to reduce the cost of quality. This is usually accomplished by increasing the prevention and/or the appraisal costs in order to reduce the failure costs more than the increase in the prevention and appraisal costs.

To end with don’t consider cost of quality as a pain, instead consider it as a cure.

“Quality is free, but it is not a gift.”

- Philip B. Crosby (Quality is Free)

Defect tracking is an important part of Software Testing. Mostly in many organizations defect tracking process will be the same. The steps below describe a sample defect tracking process. Depending on the size of the project or project team, this process may be substantially more complex.

1. Execute test and log any discrepancies.

The tester executes the test and compares the actual results to the documented expected results. If a discrepancy exists, the discrepancy is logged as a “defect” with a status of “open.” Supplementary documentation, such as screen prints or program traces, is attached if available.

2. Determine if discrepancy is a defect.

The Test Manager or tester reviews the defect log with an appropriate member of the development team to determine if the discrepancy is truly a defect, and is repeatable. If it is not a defect, or repeatable, the log should be closed with an explanatory comment.

3. Assign defect to developer.

If a defect exists it is assigned to a developer for correction. This may be handled automatically by the tool, or may be determined as a result of the discussion in
step 2.

4. Defect resolution process.

When the developer has acknowledged the defect is valid, the resolution process begins. The four steps of the resolution process are:

• Prioritize the correction – Three recommended prioritization levels are: “critical”, “major”, and “minor”. “Critical” means there is a serious impact on the organization’s business operation or on further testing. “Major” causes an output of the software to be incorrect or stops or impedes further testing. “Minor” means something is wrong, but it does not directly affect the user of the system or further testing, such as a documentation error or cosmetic GUI error. The purpose of this step is to initiate any immediate action that may be required after answering the questions: Is this a new or previously reported defect? What priority should be given to correcting this defect? Should steps be taken to minimize the impact of the defect before the correction, such as notifying users, finding a workaround?

• Schedule the correction – Based on the priority of the defect, the correction should be scheduled. All defects are not created equal from the perspective of how quickly they need to be corrected, although they may all be equal from a defect-prevention perspective. Some organizations actually treat lower priority defects as changes.

• Correct the defect – The developer corrects the defect, and upon completion, updates the log with a description of the correction and changes the status to “Corrected” or “Retest”. The tester then verifies that the defect has been removed from the system. Additional regression testing is performed as needed based on the severity and impact of the correction applied. In addition, test data, checklists, etc., should be reviewed and perhaps enhanced, so that in the future this defect will be caught earlier. If the retest results match the expected results, the tester updates the defect status to “closed.” If the problem remains, the tester changes the status back to “Open” and this step is repeated until closure.

• Report the resolution – Once the defect has been corrected and the correction verified, appropriate developers, users, etc., need to be notified that the defect has been corrected, the nature of the correction, when the correction will be released, and how the correction will be released. As in many aspects of defect management, this is an area where an automated process would help. Most defect management tools capture information on who found and reported the problem and therefore provide an initial list of who needs to be notified. Computer forums and electronic mail can help notify users of widely distributed software.

• Acceptance testing is performed by user personnel and may include assistance by software testers. System testing is performed by developers and/or software testers. The objective of both types of testing is to assure that when the software is complete it will be acceptable to the user. Testing conducted to enable a user/customer to determine whether to accept a software product. Normally performed to validate the software meets a set of agreed acceptance criteria.

• System test should be performed before acceptance testing. There is a logical sequence for testing, and an important reason for the logical steps of the different levels of testing. Unless each level of testing fulfills its objective, the following level of testing will have to compensate for weaknesses in testing at the previous level. System testing of software or hardware is testing conducted on a complete, integrated system to evaluate the system’s compliance with its specified requirements.

All projects one way or the other will surely benefit from some kind of testing.

If the size of the project is very small and with low risks and having well experienced developers with good knowledge of unit testing, then there might not be any need for testers for the successfully completing the project.

If it’s a small IT firm or a start up company where there are no testing staff then it is always better to request for help from organizations, which do testing as an outsourcing job. These days there are a lot of third party testers available like us who give good value to the money, which you have spent.

Sometimes the big guns in the organizations gamble by skipping the testing process or trust their developers for their skills, which will at last become a very costly gamble.

It is always best to leave the risks to the testers who are specialized in finding them; this way the organization can be successful in large and difficult projects.

There should always be contribution from developers as well as testers, because the developers think ‘how to make a functionality work’ and testers think ‘how will the functionality fail and does it work as per the requirements?’

It is always difficult for a developer to think in the way a tester thinks. That’s why I say that every Software project needs Testers.

Friends, below listed are major certifications which plays a vital role in tester’s career

CSTP – Certified Software Test Professional

CTM – Certified Test Manager

CSQE – Software Quality Engineer Certification

CSQA Certification – Certified Software Quality Analyst

CSTA – Certified Software Test Analyst

SQA – Software Quality Assurance Testers

SCSPE – Segue Certified SilkPerformer Engineer

SCSTE – Certified SilkTest Engineer

SCSVPM – Segue Certified SilkVision Project Manager

Brainbench – http://brainbench.com/

Empirix Certification

Mercury Interactive Certification

Rational Certification

It is mandatory that any QA/Testers should have at least any one of the above mentioned certificates, it is essential that QA/Testers are certified due to the increasing complexity of modern software development projects.It is vital for any company that their QA Team possess a thorough understanding of the ‘Software Development Process’, and how it fits into the business approach and goals of the organization.

Ineffective Testing/Testers/QA might result in costing the company billions of dollars

Traceability matrix:

The purpose of traceability matrix is to identify the entire business requirement and to trace each requirement through the project completion.

Each business requirement must have an established priority is outlined in the business requirement document. They are,

Essential – Must satisfy the requirement to be accepted by the customer

Useful Value – Added requirement inflecting the customer decisions

Nice to have – Cosmetic non – essential conditions. Make product more appealing

The traceability matrix will change and evolve throughout the entire project life cycle.

Test Metrics (How effective the testing is)

A metric is a mathematical number that show relationship between two variables. Software metrics are the measures that are used to quantify the software, software development resources and software development process

There are two types in metrics

1. Process metrics

2. Product metrics

Process metrics:

A metric used to measure the characteristic of methods, techniques and tools. Employed in developing implementing and marinating the software system.

Product metrics:

A metric used to measure the characteristic of documentation and code.

Test Metrics:

It is the number of defects found new in customer tests, which indicate the effectiveness of the test process itself.

Informal or Peer Review:

In this type of review generally a one-to one meeting between the author of a work product and a peer, initiated as a request for input regarding a particular artifact or problem. There is no agenda, and results are not formally reported. These reviews occur as need-based through each phase of a project.

Semiformal or Walkthrough Review:

The author of the material being reviewed facilitates this. The participants are led through the material in one of the two formats: the presentation is made without interruptions and comments are made at the end, or comments are made throughout. Possible solutions for uncovered defects are not discussed during the review.

Formal or Inspection Review:

An inspection is more formalized than a ‘walkthrough’, typically with 3-8 people including a moderator, reader, and a recorder to take notes. The subject of the inspection is typically a document such as a requirements spec or a test plan, and the purpose is to find problems and see what’s missing, not to fix anything.

Attendees should prepare for this type of meeting by reading thru the document; most problems will be found during this preparation. The result of the inspection meeting should be a written report. Thorough preparation for inspections is difficult, painstaking work, but is one of the most cost effective methods of ensuring quality.

Three rules should be followed for all reviews:

1. The product is reviewed, not the producer.

2. Defects and issues are identified, not corrected.

3. All members of the reviewing team are responsible for the results of the review.

• QA: Quality assurance. The main responsibility is to decrease the defect through attend the process of analysis/design/development, so it need a process that go through the whole software development life cycle and supported by the other team such as design/development. QA should detect the potential defects and help the developer to decrease them in the earlier phase of the software development life cycle.

• QC: Quality control. The main responsibility is to detect bugs and report them to the developer. QC helps the developer to decrease bugs before issuing the product. This process will be last phase of the software development life cycle.

Finding defects in a system is not easy. Some are easy to spot, others are more subtle. There are at least two reasons defects go undetected:
 

• Not Looking
  Tests often are not performed because a particular test condition was unknown. Also, some parts of a system go untested because developers assume software changes don’t affect them.

• Looking, But Not Seeing (Ignoring)
  This is like losing your car keys, only to discover they were in plain sight the entire time. Sometimes developers become so familiar with their system that they overlook details, which is why independent verification and validation is used to provide a fresh viewpoint.

Defects typically found in software systems are the results of these circumstances:

• IT improperly interprets requirements
  IT staff misinterprets what the user wants, but correctly implements what the IT people believe is wanted.

• Users specify the wrong requirements
  The specifications given to IT are erroneous.

• Requirements are incorrectly recorded
  IT fails to record the specifications properly.

• Design specifications are incorrect
  The application system design does not achieve the system requirements, but the design as specified is implemented correctly.

• Program specifications are incorrect
  The design specifications are incorrectly interpreted, making the program specifications inaccurate; however, it is possible to properly code the program to achieve the specifications.

• Errors in program coding
  The program is not coded according to the program specifications.

• Data entry errors
  Data entry staff incorrectly enters information into your computers.

• Testing errors
  Tests either falsely detect an error or fail to detect one.

• Mistakes in error correction
  Your implementation team makes errors in implementing your solutions.

• The corrected condition causes another defect
  In the process of correcting a defect, the correction process itself injects additional defects into the application system.