:: QC Boss :: Testing, Independent Software Testing, Manual Testing, Website Testing, Functionality Testing, Usability Testing, QC, QA, UAT

What is Isolation ?

This is a process of examining the causes of a defect. The exact root cause might not be determined, it is important to try and separate the symptoms of the problem from the cause. Isolating a defect is generally done by reproducing it multiple times in different situations to get an understanding of how and when it occurs.

What is Generalisation ?

This is a process of understanding the broader impact of a defect. Because developers reuse code elements throughout a program a defect present in one element of code can manifest itself in other areas. A defect that is discovered as a minor issue in one area of code might be a major issue in another area. Individuals logging defects should attempt to extrapolate where else an issue might occur so that a developer will consider the full context of the defect, not just a single isolated incident.

A defect report that is written without isolating and generalising it, is a half reported defect.

Usability testing is a technique used to evaluate a product by testing it on users. It is the process of observing users’ reactions to a product and adjusting the design to suit their needs. The product can be a Web site, Web application, or any other product. It does not have to be a finished product Marketing knows usability testing as “focus groups” and while the two differ in intent many of the principles and processes are the same.

In usability testing a basic model or prototype of the product is put in front of evaluators who are representative of typical end-users. They are then set a number of standard tasks, which they must complete, using the product. Any difficulty or obstructions they encounter are then noted by a host or observers and design changes are made to the product to correct these. The process is then repeated with the new design to evaluate those changes.

There are some fairly important tenets of usability testing that must be understood:

• Users are not testers or designers – you are not asking the users to make design decisions about the software. Users will not have a sufficiently broad technical knowledge to make decisions that are right for everyone. However, by seeking their opinion the development team can select the best of several solutions.

• You are testing the product and not the users – all too often developers believe that it’s a ‘user’ problem when there is trouble with an interface or design element. Users should be able to ‘learn’ how to use the software if they are taught properly. Maybe if the software is designed properly, they won’t have to learn it at all.

• Selection of end-user evaluators is critical –You must select evaluators who are directly representative of your end-users. Don’t pick just anyone off the street, don’t use management and don’t use technical people unless they are your target audience.

• Usability testing is a design tool – Usability testing should be conducted early in the lifecycle when it is easy to implement changes that are suggested by the testing. Leaving it till later will mean changes will be difficult to implement.

Team Building

There are eight guidelines that are helpful in developing compatibility and motivation of a software testing team:

1. Communicate the vision, objectives, and goals of the project to each and everyone in the team.

2. Define roles and responsibilities of team members.

3. Empower team members to manage their responsibilities.

4. Hold team members accountable for their assigned responsibilities in the team process.

5. Ensure that all the required skills are present on the team.

6. Provide the necessary technical and team training to everyone.

7. Share everything among the team members.

8. Award successes and celebrate achievments.

Team Member Interaction

1. Know communication and work preference styles of staff and assure that the team complements those communication and work preference styles.

2. Set clear, measurable work requirement standards.

3. Delegate authority to staff members that empowers them to perform the tasks in the manner they deem most effective and efficient.

4. Exact responsibility and accountability for team members for completing their work tasks in an effective efficient manner with high quality work products.

5. Give immediate and objective feedback to team members on the performance of their individual and team tasks.

6. Communicate, communicate, and communicate with each and every team members about any event that may impact team performance.

What is Object-Oriented?
This is an interesting question, answered by many scientists. I will just give a brief of the same.
“Objects” are re-usable components. General definition of “Object-Oriented Programming” is that which combines data structures with functions to create re-usable objects. 

What are the various Object Oriented Life Cycle Models?
What is mainly required in OOLife Cycle is that there should be iterations between phases. This is very important. One such model which explains the importance of these iterations is the Fountain Model.
This Fountain Model was proposed by Henderson-Sellers and Edwards in 1990. Various phases in the Fountain Model are as follows: 

1.  Requirements Phase

2.  Object-Oriented Analysis Phase

3.  Object-Design Phase

4.  Implementation Phase

5.  Implementation and Integration Phase

6.  Operations Mode

7.  Maintenance 

The Object Oriented Methodology

Let us look at the Object Modelling Technique(OMT) methodology: 

   Analysis:Starting from the statement of the problem, the analyst builds a model of the real 

  world situation showing its important properties. 

System Design:The system designer makes high-level decisions about the overall architecture. During system design, the target system is organized into subsystems based on both the analysis structure and the proposed architecture. 

Object Design:The object designer builds a design model based on the analysis model but containing implementation details. The designer adds details to the design model in accordance with the strategy established during system design. 

Implementation:The object classes and relationships developed during object design are finally translated into a particular programming language, database, or hardware implementation.

The Three Models
The OMT methodology uses three kinds of models to describe the Object Oriented System:

1. Object Model, describing the objects in the system and their relationships, which are static structures.
2. Dynamic Model, describing the interactions among objects in the system, which change over the time.
3. Functional Model, describing the data transformations of the system.

In this tutorial you will learn about Risk Analysis, Technical Definitions, Risk Analysis, Risk Assessment, Business Impact Analysis, Product Size Risks, Business Impact Risks, Customer-Related Risks, Process Risks, Technical Issues, Technology Risk, Development Environment Risks, Risks Associated with Staff Size and Experience.

Risk Analysis is one of the important concepts in Software Product/Project Life Cycle. Risk analysis is broadly defined to include risk assessment, risk characterization, risk communication, risk management, and policy relating to risk. Risk Assessment is also called as Security risk analysis.

Technical Definitions:

Risk Analysis: A risk analysis involves identifying the most probable threats to an organization and analyzing the related vulnerabilities of the organization to these threats.

Risk Assessment: A risk assessment involves evaluating existing physical and environmental security and controls, and assessing their adequacy relative to the potential threats of the organization.

Business Impact Analysis: A business impact analysis involves identifying the critical business functions within the organization and determining the impact of not performing the business function beyond the maximum acceptable outage. Types of criteria that can be used to evaluate the impact include: customer service, internal operations, legal/statutory and financial.

Risks for a software product can be categorized into various types. Some of them are:

Product Size Risks:

The following risk item issues identify some generic risks associated with product size:

• Estimated size of the product and confidence in estimated size?
• Estimated size of product?
• Size of database created or used by the product?
• Number of users of the product?
• Number of projected changes to the requirements for the product?

Risk will be high, when a large deviation occurs between expected values and the previous experience. All the expected information must be compared to previous experience for analysis of risk.

Business Impact Risks:

The following risk item issues identify some generic risks associated with business impact:

• Affect of this product on company revenue?
• Reasonableness of delivery deadline?
• Number of customers who will use this product and the consistency of their needs relative to the product?
• Number of other products/systems with which this product must be interoperable?
• Amount and quality of product documentation that must be produced and delivered to the customer?
• Costs associated with late delivery or a defective product?

Customer-Related Risks:

Different Customers have different needs. Customers have different personalities. Some customers accept what is delivered and some others complain about the quality of the product. In some other cases, customers may have very good association with the product and the producer and some other customers may not know. A bad customer represents a significant threat to the project plan and a substantial risk for the project manager.

The following risk item checklist identifies generic risks associated with different customers:

• Have you worked with the customer in the past?
• Does the customer have a solid idea of what is required?
• Will the customer agree to spend time in formal requirements gathering meetings to identify project scope?
• Is the customer willing to participate in reviews?
• Is the customer technically sophisticated in the product area?
• Does the customer understand the software engineering process?

Process Risks:

If the software engineering process is ill defined or if analysis, design and testing are not conducted in a planned fashion, then risks are high for the product.

• Has your organization developed a written description of the software process to be used on this project?
• Are the team members following the software process as it is documented?
• Are the third party coders following a specific software process and is there any procedure for tracking the performance of them?
• Are formal technical reviews are done regularly at both development and testing teams?
• Are the results of each formal technical review documented, including defects found and resources used?
• Is configuration management used to maintain consistency among system/software requirements, design, code, and test cases?
• Is a mechanism used for controlling changes to customer requirements that impact the software?

Technical Issues:

• Are specific methods used for software analysis?
• Are specific conventions for code documentation defined and used?
• Are any specific methods used for test case design?
• Are software tools used to support planning and tracking activities?
• Are configuration management software tools used to control and track change activity throughout the software process?
• Are tools used to create software prototypes?
• Are software tools used to support the testing process?
• Are software tools used to support the production and management of documentation?
• Are quality metrics collected for all software projects?
• Are productivity metrics collected for all software projects?

Technology Risk:

• Is the technology to be built new to your organization?
• Does the software interface with new hardware configurations?
• Does the software to be built interface with a database system whose function and performance have not been proven in this application area?
• Is a specialized user interface demanded by product requirements?
• Do requirements demand the use of new analysis, design or testing methods?
• Do requirements put excessive performance constraints on the product?

Development Environment Risks:

• Is a software project and process management tool available?
• Are tools for analysis and design available?
• Do analysis and design tools deliver methods that are appropriate for the product to be built?
• Are compilers or code generators available and appropriate for the product to be built?
• Are testing tools available and appropriate for the product to be built?
• Are software configuration management tools available?
• Does the environment make use of a database or repository?
• Are all software tools integrated with one another?
• Have members of the project team received training in each of the tools?

Risks Associated with Staff Size and Experience:

• Are the best people available and are they enough for the project?
• Do the people have the right combination of skills?
• Are staffs committed for entire duration of the project?

There are many sources of ambiguity in software design and development.

• In wording or interpretation of specifications or standards
• In expected response of the program to invalid or unusual input
• In behavior of undocumented features
• In conduct and standards of regulators/auditors
• In customers’ interpretation of their needs and the needs of the users they represent
• In definitions of compatibility among 3rd party products

Whenever there is ambiguity, there is a strong opportunity for a defect (at least in the eyes of anyone who understands the world differently from the implementation).

The following methods may be used to uncover ambiguity in requirements:

1. Determine how well individuals recall a requirement by asking them to repeat it from memory. Often, the parts that are forgotten are illogical or ambiguous.
2. Determine if the meaning of a requirement is clear to individuals by reading a requirement aloud several times.  Each time emphasize different words in the requirement, and see if the meaning of the requirement changes when different words are accentuated.
3. Determine if the meaning of a requirement changes for individuals when key words are substituted with synonyms.

Performing risk analysis during test planning is a three-step process as follows:

1. Form the risk analysis team

2. Identify risks

3. Select testing priorities

Form the Risk Analysis Team

The key to a successful risk analysis is the establishment of the correct risk team, whose responsibility will be to identify risks and prioritize the use of test resources. The objective is to reduce the risks to an acceptable level.

The risk team may be part of the requirements team, or part of the test team, or it may be a team specifically selected for the purpose of completing risk analysis. The team should be comprised of three to six members and at a minimum possess the following skills:

• Knowledge of the user application

• Understanding of risk concepts

• Ability to identify controls

• Familiarity with both application and information services risks

• Understanding of information services concepts and systems design

• Understanding of computer operations procedures

Identify Risks

The objective of the risk team is first to identify the application-oriented, not environmental, risks associated with the application system. For example, the risks that relate to all applications equally (i.e., environmental risks) need not be identified unless they have some special relevance to the applicants. There are two methods to identify risks there are:

• Risk analysis scenario

In this method, the risk team “brainstorms” the potential application risks using there experience, judgment, and knowledge of the application area. It is important to have the Synergistic effect of a group so that group members can challenge one another to develop a complete list of risks that are realistic for the application.

• Risk checklist

The risk team is provided with a list of the more common risks that occur in automated applications. From this list, the team selects those risks that are applicable to the application. In this method, the team needs fewer skills because the risk list provides the stimuli for the process, and the objective of the team is to determine which of the risks on the list are applicable to the application.

Select Testing Priorities

The risk team needs to rank the risks as an input to the test planning process. Risks are normally ranked by the severity of the risk. However, other considerations may impact the prioritization such as:

• Compliance required to laws and regulations

• Impact on competitiveness

• Impact on ethics, values and image

Acceptance testing is a formal testing conducted to determine whether a software system satisfies its acceptance criteria and to enable the buyer to determine whether to accept the system.

Acceptance testing at delivery is usually the final opportunity for the buyer to examine the software and to seek redress from the developer for insufficient or incorrect software.

To reduce the risk of problems arising at delivery or during operation, the buyer must become involved with software acceptance early in the acquisition process.

Software Acceptance is an incremental process of approving or rejecting software systems during development or maintenance, according to how well the software satisfies predefined criteria.

Acceptance decision occur at pre-specified times when processes, support tools, interim documentation, Segments of the software and finally the total software system must meet predefined criteria for acceptance.

The Final Acceptance Decision occurs with verification that the delivered documentation is adequate and consistent with the executable system and the complete software system meets all buyer requirements.

Formal Final software Acceptance testing must occur at the end of the software Development process. It consists of test to determine whether the developed system meets predetermined functionality, performance, quality and interface criteria.

Regression testing is the process of testing changes to computer programs to make sure that the older programming still works with the new changes.

Usage:

• All aspects of system remain functional after testing.
• Change in one segment does not change the functionality of other segment.

Objective:

• Determine System documents remain current
• Determine System test data and test conditions remain current
• Determine Previously tested system functions properly without getting effected though changes are made in some other segment of application system.

How to Use

• Test cases, which were used previously for the already tested segment is, re-run to ensure that the results of the segment tested currently and the results of same segment tested earlier are same.
• Test automation is needed to carry out the test transactions (test condition execution) else the process is very time consuming and tedious.
• In this case of testing cost/benefit should be carefully evaluated else the efforts spend on testing would be more and payback would be minimum.

When to Use

• When there is high risk that the new changes may effect the unchanged areas of application system.
• In development process: Regression testing should be carried out after the pre-determined changes are incorporated in the application system.
• In Maintenance phase : regression testing should be carried out if there is a high risk that loss may occur when the changes are made to the system.

Example

• Re-running of previously conducted tests to ensure that the unchanged portion of system functions properly.
• Reviewing previously prepared system documents (manuals) to ensure that they do not get effected after changes are made to the application system.

Disadvantage

• Time consuming and tedious if test automation not done

Why Test for Regression?

Webster’s Dictionary defines regression as “a trend or shift toward a lower or less perfect state,” and indeed, regression testing in its most basic form is simply testing done to determine whether a product has regressed to a less functional state than in the previous build. Any time you rerun a test after the code has been changed, you are performing regression testing. For most test teams regression testing takes up a large percentage of their time, especially near the end of the development cycle.

The uninitiated may well wonder “Why spend so much time testing something that has already been looked at?” but the answer is clear: regression testing finds many bugs. Studies have shown that changes and error corrections tend to be much more error prone than the original code in the program, in much the same way that most copy errors that make it to print were introduced during edits rather than in the original drafti. Indeed, according to Hetzelii, for some instances of software maintenance, the probability of making an error while introducing a change is between 50 and 80%. This statistic is shocking and clearly underscores the need for extensive regression testing.

Many testers don’t realize that regression testing has such a broad definition. Some of the most common types of regression tests are discussed below.

Regression tests confirm that an application under test works on this run the same way it worked on the last run. If it doesn’t, then you *do* have a problem.

Bug Verification Tests

The phrase “I need to regress my bugs” is commonly used to mean, “I need to verify that the bugs assigned to me have been fixed.” Bug fix verification is one of the most important and common types of regression testing that is performed. Many fixes fail, for a variety of reasons. Perhaps the developer didn’t understand your bug report and fixed something else instead: perhaps the developer didn’t test his fix before he checked in the code; or perhaps the fix did not make it into the build. It’s also common to find new bugs while verifying bug fixes. Sometimes the fix breaks nearby functionality, or sometimes the new bug was not observable until the previous error was corrected. Sometimes bugs are fixed in such a way that you would still consider the behavior a problem (although usually not as serious a problem as the original bug). Bug verification testing should be among the first tests you do when you receive a new build of a product.

Build Acceptance Tests

A build acceptance test (sometimes also called build verification test, smoke test, quick check, or the like) is a set of tests run on each new build of a product to verify that the build is testable before the build is released into the hands of the test team. The build acceptance test is generally a short set of tests, which exercises the mainstream functionality of the application. Any build that fails the build verification test is rejected, and testing continues on the previous build (provided there has been at least one build that has passed the acceptance test). So build acceptance tests are a type of regression testing that is done every time a new build is taken. Build acceptance tests are important because they let developers know right away if there is a serious problem with the build, and they save the test team wasted time and frustration.