:: QC Boss :: Testing, Independent Software Testing, Manual Testing, Website Testing, Functionality Testing, Usability Testing, QC, QA, UAT

The web is a powerful medium, through which any person or organization can publish its message for consumption or use by anyone who has access. Part of the value of the web is that content can be easily modified or added to as a message grows or changes. However, the downside to this non-permanence is the idea that the message or site does not need to be perfect before it is presented to the intended audience and participants. This is not a productive school of thought. We must realize that a website should be treated similar to any other marketing or informational piece and thoroughly reviewed for correct content and functionality through formal quality control procedures.

If you want a great Web site, you must test its usability. Just because no one has complained about your site doesn’t mean that all your visitors are using your site effectively, efficiently and to their full satisfaction.

Visitors won’t make the effort to complain unless you have outraged them in some way, or have provided an easy way for them to in touch with you from every page about the problems they may be having.

Who should be testing the website? The client? The development team? A third party? The answer is, “Yes, to all three.” Quality control personnel should consist of the client, the development team members and other personnel who are not involved with the creation of the site. This ensures that the site is reviewed by a variety of different people who have different relationships to – and expectations of – the site.

Life Cycle Models

A variety of life cycle models have been proposed, most of which focus exclusively on the development processes.

Code and Fix

A simple and widely used software life cycle model: Repeat the following steps until the solution is good enough.

1. Code
2. Compile and execute
3. Detect errors
4. Fix

Waterfall Model

The waterfall model prescribes a sequential execution of a set of development and management processes, with no return to an earlier activity once it is completed. Some variants of the waterfall model allow revisiting the immediately preceding activity (“feedback loops”) if inconsistencies or new problems are encountered during the current activity.

V-Model

Another variant of the waterfall model — the V-model — associates each development activity with a test or validation at the same level of abstraction. Each development activity builds a more detailed model of the system than the one before it, and each validation tests a higher abstraction than its predecessor.

Spiral Model

Barry Boehm devised the spiral model to address the weaknesses of the waterfall model, especially its lack of resilience in the face of change. The spiral model focuses on addressing risks incrementally by repeating the waterfall model in a series of cycles or rounds:

• Concept of Operation
• Software Requirements
• Software Product Design
• Detailed Design
• Code
• Unit Test
• Integration and Test
• Acceptance Test
• Implementation

Each round consists of four phases (as illustrated in this figure):

1. Determine objectives: product definition, determination of business objects, specification of constraints, generation of alternatives
2. Evaluate alternatives: risk analysis, prototyping
3. Develop product: detailed design, code, unit test, integration
4. Plan next cycle: customer evaluation, design planning, implementation, customer delivery

The spiral model is an improvement on the waterfall model, as it provides for multiple builds and provides several opportunities for customer involvement. However, it is elaborate, difficult to manage, and does not keep all workers occupied during all phases.

Boehm subsequently proposed a modified version of the spiral model — the WinWin spiral model — which adds activities to the start of each cycle to identify the key stakeholders and their “win” conditions.

Prototyping — the Sawtooth and Shark Tooth Models

As a software product is being developed, the developers’ view of the system diverges from the client’s view, since the developer’s focus on design and implementation while the client remains focused on requirements.

The sawtooth model provides checkpoints during the development process in order to check that development is proceeding in a direction that will eventually meet the client’s requirements. Typically, the checkpoints involve demonstrating a prototype to the client. For example, two prototypes may be developed:

1. a revolutionary prototype is an illustrative model of the system, representing only a small fraction of the required functionality
2. an evolutionary prototype, shown late in the development process, is based on a completed design and implements some of the required functionality

The shark tooth model adds management reviews and demonstrations to the sawtooth model. Since these may be seen to be at an intermediate level of abstraction, a diagram of the model includes large “teeth” and small “teeth”.

Rapid Application Development and the Time Box Model

Rapid application development (RAD) is an approach rather than a model. Its proponents view formal life cycle models as inherently inefficient, due to the large amount of documentation and the number of reviews required. The formality of such models is seen as interfering with customer communication.

Instead, RAD focuses on developing a sequence of evolutionary prototypes which are reviewed with the customer, both to ensure that the system is developing toward the user’s requirements and to discover further requirements.

The process is controlled by restricting the development of each integration to a well-defined period of time, called a time box. Each time box includes analysis, design, and implementation of a prototype.

One problem with RAD is reaching closure — the customer always has feedback, and the developers and the customer may never agree that the project is done.

Synch-and-Stabilize

Synch-and-stabilize is Microsoft’s attempt to scale-up a loosely structured small-team (“hacker”) style of product development.

• many small teams (3 – 8 developers per team) work in parallel
• changes are synchronize frequently so components will work together
  • developers check-in their code by a particular time so a new build (complete recompile) is done by the end of the day or the next morning
  • a defect that “breaks” the build must be fixed immediately
• features are evolved incrementally, with occasional innovations
  • start with a “vision statement”
  • select features and establish priority of features with user input
  • developers are free to innovate or adapt to unforseen competitive opportunities or threats
  • continual testing during development
• the product is stabilized at 3 or 4 milestone junctures in the project lifetime
  • thorough internal and external testing (beta sites)
  • fix almost all errors detected
  • “zero-bug” release at the last milestone

The process is also called a “milestone”, “daily build”, “nightly build”, and “zero-defect” process.

The overall strategy is to quickly introduce products that are “good enough” to capture a mass market, then improve the product, selling multiple product versions and upgrades.

Iterative and Incremental Development

The Unified Process is an example of an iterative software development process. Its designers — Ivar Jacobson, Grady Booch, and James Rumbaugh — characterize the process [1] as:

• use-case driven — The use case model describes the complete functionality of the system. It replaces the traditional functional specification of the system, extending the question “What is the system supposed to do?” with the words “for each user?”

The use cases drive the development process: developers create design and implementation models to realize the use cases, and testers test the implementation to ensure that the use cases are correctly implemented.

• architecture-centric — The software architecture represents the most significant static and dynamic aspects of the system — the platform on which the software is to run, reusable components and frameworks available, deployment considerations, legacy systems, and nonfunctional requirements.

The architect selects the use cases which represent the key functions of the system (5% to 10% of all the use cases), specifies them in detail, and realizes them in terms of subsystems, classes, and components.

• iterative and incremental — The software development project is divided into mini-projects, each of which is an iteration that results in an increment. Each iteration deals with the most important risks and realizes a group of use cases that together extend the usability of the product as developed so far. In the early phases, a superficial design might be replaced by a more detailed or sophisticated one; in later phases, increments are typically additive.
  

Entity-Based Models

Issue-Based Life Cycle Model

Bruegge and Dutoit present (textbook, pp. 483–485) an issue-based life cycle model, in which the project is driven by a set of issues such as “How do we set up the initial teams?” and “What software architecture shall we use?” Issues are classified as open or closed, but closed issues can be reopened as changes occur in the application or solution domain. Issues are maintained in an issue base accessible to all project participants.

Issues do not directly correspond to risks, as some issues are design problems, not risks.

The management problem in issue-based development is to keep the number of open issues small and manageable. By organizing the project around issues, all life cycle activities may proceed concurrently, using dependencies between issues to determine which activities can be performed concurrently and what the impact of reopening an issue will be.

Workbook-Centered Development

IBM’s Object-Oriented Technology Center uses an iterative and incremental process, but organizes it around work products — documents, models, and software — and uses a structured workbook to ensure that the work products are available and organized for access by the entire project. Work products are seen as the primary means of project communication.

Capability Maturity Model

The Software Engineering Institute (SEI) has defined five levels to characterize the maturity of a software development organization:

1. Initial — ad hoc activities; dependence on the heroic efforts and skills of key individuals.
2. Repeatable — each project has a well-defined software life cycle, but different models are used for different projects; success is predictable for similar projects.
3. Defined — uses a documented model for all activities; model is customized at the beginning of each project.
4. Managed — metrics are defined for activities and deliverables; data is collected during the project to quantify progress
5. Optimized — measurement data are used to improve the model.

System Development Life Cycle Model

This is also known as Classic Life Cycle Model (or) Linear Sequential Model (or) Waterfall Method. This has the following activities.

1. System/Information Engineering and Modeling

2. Software Requirements Analysis

3. Systems Analysis and Design

4. Code Generation

5. Testing

6. Maintenance

System/Information Engineering and Modeling

As software is always of a large system (or business), work begins by establishing requirements for all system elements and then allocating some subset of these requirements to software. This system view is essential when software must interface with other elements such as hardware, people and other resources. System is the basic and very critical requirement for the existence of software in any entity. So if the system is not in place, the system should be engineered and put in place. In some cases to extract the maximum output, system should be re-engineered and spiced up. Once the ideal system is engineered or tuned up, the development team studies the software requirement for the system.

Software Requirements Analysis

This is also known as feasibility study. In this phase, the development team visits the customer and studies their system. They investigate the need for possible software automation in the given system. By the end of the feasibility study, the team furnishes a document that holds the different specific recommendations for the candidate system. It also includes the personnel assignments, costs, project schedule, and target dates. The requirements gathering process is intensified and focussed specially on software. To understand the nature of the program(s) to be built, the system engineer (“analyst”) must understand the information domain for the software, as well as required function, behavior, performance and interfacing. The essential purpose of this phase is to find the need and to define the problem that needs to be solved .

System Analysis and Design

In this phase, the software’s overall structure and its nuances are defined. In terms of the client/server technology, the number of tiers needed for the package architecture, the database design, the data structure design etc are all defined in this phase. Analysis and Design are very crucial in the whole development cycle. Any glitch in the design phase could be very expensive to solve in the later stage of the software development. Much care is taken during this phase. The logical system of the product is developed in this phase.

Code Generation

The design must be translated into a machine-readable form. The code generation step performs this task. If design is performed in a detailed manner, code generation can be accomplished with out much complication. Programming tools like Compilers, Interpreters, Debuggers are used to generate the code. Different high level programming languages like C, C++, Pascal, Java are used for coding. With respect to the type of application, the right programming language is chosen.

Testing

Once the code is generated, the program testing begins. Different testing methodologies are available to unravel the bugs that were committed during the previous phases. Different testing tools and methodologies are already available. Some companies build there own testing tools that are tailor made for there own development operations.

Maintenance

Software will definitely undergo change once it is delivered to the customer. There are many reasons for the change. Change could happen because of some unexpected input values into the system. In addition, the changes in the system could directly affect the software operations. The software should be developed to accommodate changes that could happen during the post implementation period.

Prototyping Model

This is a cyclic version of the linear model. In this model, once the requirement analysis is done and the design for a prototype is made, the development process gets started. Once the prototype is created, it is given to the customer for evaluation. The customer tests the package and gives his/her feed back to the developer who refines the product according to the customer’s exact expectation. After a finite number of iterations, the final software package is given to the customer. In this methodology, the software is evolved as a result of periodic shuttling of information between the customer and developer. This is the most popular development model in the contemporary IT industry. Most of the successful software products have been developed using this model – as it is very difficult (even for a whiz kid!) to comprehend all the requirements of a customer in one shot. There are many variations of this model skewed with respect to the project management styles of the companies. New versions of software product evolve as a result of prototyping.

Rapid Application Development (RAD) Model

The RAD is a linear sequential software development process that emphasizes an extremely short development cycle. The RAD model is a “high speed” adaptation of the linear sequential model in which rapid development is achieved by using a component-based construction approach. Used primarily for information systems applications, the RAD approach encompasses the following phases:

Business modeling

The information flow among business functions is modeled in a way that answers the following questions:
What information drives the business process?
What information is generated?
Who generates it?
Where does the information go?
Who processes it?

Data modeling

The information flow defined as part of the business modeling phase is refined into a set of data objects that are needed to support the business. The characteristic (called attributes) of each object share identified and the relationships between these objects are defined.

Process modeling

The data objects defined in the data-modeling phase are transformed to achieve the information flow necessary to implement a business function. Processing the descriptions are created for adding, modifying, deleting, or retrieving a data object.

Application generation

RAD assumes the use of the RAD tools like VB, VC++, Delphi etc rather than creating software using conventional third generation programming languages. The RAD works to reuse existing program components (when possible) or create reusable components (when necessary). In all cases, automated tools are used to facilitate construction of the software.

Testing and turnover

Since the RAD process emphasizes reuse, many of the program components have already been tested. This minimizes the testing and development time.

Component Assembly Model

Object technologies provide the technical framework for a component-based process model for software engineering. The object oriented paradigm emphasizes the creation of classes that encapsulate both data and the algorithm that are used to manipulate the data. If properly designed and implemented, object oriented classes are reusable across different applications and computer based system architectures. Component Assembly Model leads to software reusability. The integration/assembly of the already existing software components accelerate the development process. Nowadays many component libraries are available on the Internet. If the right components are chosen, the integration aspect is made much simpler.

Traceability matrix is used to trace the Functional Specification with the corresponding Test case to check whether all the Functionality has been carried throughout the SDLC and tested or not. The matrix which verifies that whether the written test cases covers all the functionalities or not. In simple terms whether there is a test case for every functionality or not.

 

Requirements Traceability Matrix (RTM) – The RTM is a key document. It provides the information linking the contractually invoked requirements; both stated and derived, to all of the design documentation. The RTM is the tool that enables the IPT to verify that all of the design requirements have been addressed.

 

USEFUL TRACEABILITY MATRICES

 

Various traceability matrices may be utilized throughout the system life cycle. Useful ones include:

 

- Functional specification to requirements document: The example illustrates a Requirement to Function Cross-Reference Matrix. It shows that each requirement (obtained from a preliminary requirements statement provided by the customer or produced in the Concept Definition stage) has been covered in an appropriate section of the functional specification.

 

- Top-level configuration item to functional specification: For example, a top-level configuration item, Workstation, may be one of the configuration items that satisfy the function Input Order Information. On the matrix, each configuration item would be written down the left hand column and each function would be written across the top.

 

- Low-level configuration item to top level configuration item: For example, the top-level configuration item, Workstation, may contain the low-level configuration items Monitor, CPU, keyboard, and network interface card.

 

- Design specification to functional specification verifies that each function has been covered in the design.

 

- System test plan to functional specification ensures you have identified a test case or test scenario for each process and each requirement in the functional specification.

 

Although the construction and maintenance of traceability matrices may be time-consuming, they are a quick reference during verification and validation tasks.

 

A collection of possible scenarios between the system under discussion and external actors, characterized by the goal the primary actor has toward the system’s declared responsibilities, showing how the primary actor’s goal might be delivered or might fail. Use cases are goals (use cases and goals are used interchangeably) that are made up of scenarios. Scenarios consist of a sequence of steps to achieve the goal; each step in a scenario is a sub (or mini) goal of the use case. As such each sub goal represents either another use case (subordinate use case) or an autonomous action that is at the lowest level desired by our use case decomposition.

 

 

This hierarchical relationship is needed to properly model the requirements of a system being developed. A complete use case analysis requires several levels. In addition the level at which the use case is operating at it is important to understand the scope it is addressing. The level and scope are important to assure that the language and granularity of scenario steps remain consistent within the use case.

 

There are two scopes that use cases are written from: Strategic and System. There are also three levels: Summary, User and Sub-function.

Scopes: Strategic and System

 

Strategic Scope:

 

The goal (Use Case) is a strategic goal with respect to the system. These goals are goals of value to the organization. The use case shows how the system is used to benefit the organization. These strategic use cases will eventually use some of the same lower level (subordinate) use cases.

 

System Scope:

 

Use cases at system scope are bounded by the system under development. The goals represent specific functionality required of the system. The majority of the use cases are at system scope. These use cases are often steps in strategic level use cases

 

Levels: Summary Goal, User Goal and Sub-function.

 

Sub-function Level Use Case:

 

A sub goal or step is below the main level of interest to the user. Examples are “logging in” and “locate a device in a DB”. Written always at System Scope.

 

User Level Use Case:

 

This is the level of greatest interest. It represents a user task or elementary business process. A user level goal addresses the question “Does your job performance depend on how many of these you do in a day”. For example “Create Site View” or “Create New Device” would be user level goals but “Log In to System” would not. Written always at System Scope.

 

Summary Level Use Case:

 

Written for either strategic or system scope. They represent collections of User Level Goals. For example summary goal “Configure Data Base” might include as a step, user level goal “Add Device to database”. Written either at System of Strategic Scope.

 

 

In computer programming, a unit test is a method of testing the correctness of a particular module of source code. The idea is to write test cases for every non-trivial function or method in the module so that each test case is separate from the others if possible. The developers mostly do this type of testing.

 

Benefits

 

The goal of unit testing is to isolate each part of the program and show that the individual parts are correct. It provides a written contract that the piece must satisfy. This isolated testing provides four main benefits:

 

Encourages change

 

Unit testing allows the programmer to re-factor code at a later date, and make sure the module still works correctly (regression testing). This provides the benefit of encouraging programmers to make changes to the code since it is easy for the programmer to check if the piece is still working properly.

 

Simplifies Integration

 

Unit testing helps eliminate uncertainty in the pieces themselves and can be used in a bottom-up testing style approach. By testing the parts of a program first and then testing the sum of its parts will make integration testing easier.

 

Documents the code

 

Unit testing provides a sort of “living document” for the class being tested. Clients looking to learn how to use the class can look at the unit tests to determine how to use the class to fit their needs.

 

Separation of Interface from Implementation

 

Because some classes may have references to other classes, testing a class can frequently spill over into testing another class. A common example of this is classes that depend on a database; in order to test the class, the tester finds him writing code that interacts with the database. This is a mistake, because a unit test should never go outside of its own class boundary. As a result, the software developer abstracts an interface around the database connection, and then implements that interface with their own Mock Object. This results in loosely coupled code, thus minimizing dependencies in the system.

 

Limitations

 

It is important to realize that unit testing will not catch every error in the program. By definition, it only tests the functionality of the units themselves. Therefore, it will not catch integration errors, performance problems and any other system-wide issues. In addition, it may not be trivial to anticipate all special cases of input the program unit under study may receive in reality. Unit testing is only effective if it is used in conjunction with other software testing activities.

Usage:

• To ensure that system performs correctly
• To ensure that correctness can be sustained for a considerable period of time.
• System can be tested for correctness through all phases of SDLC but incase of reliability the programs should be in place to make system operational.

Objective:

• Successfully implementation of user requirements,
• Correctness maintained over considerable period of time Processing of the application complies with the organization’s policies and procedures.
• Secondary users needs are fulfilled:
  • Security officer
  • DBA
  • Internal auditors
  • Record retention
  • Comptroller

 How to Use:

• Test conditions created
• These test conditions are generalized ones, which becomes test cases as the SDLC progresses until system is fully operational.
• Test conditions are more effective when created from user’s requirements.
• Test conditions if created from documents then if there are any error in the documents those will get incorporated in Test conditions and testing would not be able to find those errors.
• Test conditions if created from other sources (other than documents) error trapping are effective.
• Functional Checklist created.

When to Use:

• Every application should be Requirement tested
• Should start at Requirements phase and should progress till operations and maintenance phase.
• The method used to carry requirement testing and the extent of it is important.

Example:

• Creating test matrix to prove that system requirements as documented are the requirements desired by the user.
• Creating checklist to verify that application complies to the organizational policies and procedures.

In the fast changing and highly competitive web-based business environment, it is critical for organizations to test their web sites and web applications using an automated testing tool. Automated web testing should ensure that the web applications/web sites/web services usual functionality works correctly, provides the ability to reuse and extend the tests across multiple browsers/platforms/languages/databases/servers and ensure that all the users accessing the web applications get results in an acceptable time.

Ultimately, web testing large or small Web applications/Web Sites requires some kind of expertise. Today, most of the organizations contact dedicated QA professionals, testing labs and testing consultants to get this done. Also, the need of the day is to do remote web testing instead of traveling to a particular site for teams who are geographically distributed. This helps to cut costs, minimize the effort required to test web applications/web sites, increase software quality, reduce time-to-market and use reusable test cases.

• Functionality Testing – Test Web Applications/Web Sites to ensure that all aspects of your site functions properly such as images, text, form elements, dialogs, menus appear correctly, HTML links work, form submissions succeed, etc.
• Compatibility Testing – Test Web Applications/Web Sites to ensure that it is compatible with different browsers, operating systems, languages, databases and servers.
• Localized Testing – Automate localization testing of your web applications/web sites to test multi-language deployments of your applications without re-recording.
• Stress Testing – Test Web Applications/Web Sites to determine the system’s breaking point based on predefined failure criteria.
• Load Testing – Test Web Applications/Web Sites to ensure that it is capable of handling a predetermined peak load by generating 1000’s of users on the Web site.
• Web Services Functional Testing – Test Web Services to ensure that the functionality works correctly by validating the responses sent to each SOAP/HTTP request.
• Web Services Performance Testing – Test Web Services to gather the performance and stability information of web services by realistically simulating the load on your web services.
• Regression Testing – Test Web Applications/Web Sites/Web Services by running all day and night to ensure that it is still working between builds or releases.

What is Quality Assurance?
-Quality Assurance is the process that demonstrates your product is able to satisfy your users

What is the aim of Quality Assurance?
-When good Quality Assurance is implemented there should be improvement in usability and performance and lessening rates of defects

Why is QA needed?
In the past JISC had standards-based philosophy but:
-Approach of “let a 1,000 flowers bloom”
-Diversification as no killer application
Things are different now:
-The Web is the killer application
-There is now a need for interoperability (now self-contained independent Web sites)
-There is a need for publicly-funded services to be seen to provide value for money

There are so many definitions for Quality.
Quality is…..

- Based on judgments by an individual or organization
- Fitness for purpose
- Corresponds to a rating
- Based on acceptable performance
- Meeting goals
- Meeting requirements

Fifteen years ago, “user-friendly” software was any application that had a menu or allowed a user to correct an input error. Today, usability engineering is a distinct professional discipline in its own right, where researchers and practitioners strive to develop and implement techniques for making software systems more user-friendly.

In the meantime, the sustained growth of the World Wide Web has resulted in the creation of literally millions of Web sites — only a small percentage of which are reader-friendly. Fortunately, many of the principles from usability engineering can be easily applied (or adapted) to Web development.

Reader-friendliness can be summed up in a simple axiom:

Always and in everything, remember your reader.

Technical Considerations

Web authors who want to build reader-friendly Web sites confront certain technical constraints:

• Understand the medium

Part of the beauty of hypertext documents is that they allow an author to create a virtual world composed of small pieces of information, which can then be interwoven in an endless variety of ways. It is not necessary, nor is it desirable, to try to cram everything onto a single page (or even a single site).

Understand, too, that the Web is not a WYSIWYG medium. There is simply no way to build a page that looks exactly the same to every reader.

• Keep it small

Speed is the single biggest determinant of reader satisfaction. The key to fast-loading pages is simple:

Send less data

Small, fast-loading pages make for more and happier visitors at your Web site. Large, slow-loading pages simply invite your visitor to browse elsewhere.

• Keep it accessible and compatible

There are many techniques available for creating pages that are browser-independent and hardware-independent. A reader-friendly Web site acknowledges that, on a truly world-wide Web, Lynx is, at worst, a lesser among equals.

Stylistic Considerations

Reader-friendly Web sites share some common stylistic characteristics:

• Use familiar, natural language

A reader-friendly Web site understands who its intended readers are, and it targets them directly. Speak to your readers using terms they find familiar. Using your readers’ language, without apology nor pretense, helps them to feel “at home” at your Web site.

Always remember that you are your reader’s host in their virtual tour of your Webspace. Be conversational. Be polite. Use complete sentences. And don’t nag them about their choice in Web browsers.

Structural Considerations

Reader-friendly Web sites also share certain structural characteristics in common:

• Be consistent

Humans are driven to seek out structure and pattern. By implication, readers will learn the “flow” of your site — but only if you let them.

Consistency is a key element to learnability in any Web site. From predictable navigational conventions, to a consistent “look-and-feel,” a reader-friendly Web site makes it easy for readers to develop reliable expectations about the site and its pages.

• Provide a solid navigational base

Hypertext systems allow readers to define and traverse their own paths through your Web site. Thus it is absolutely critical that you provide them with a means of maintaining and regaining their bearings during their virtual visit.

Perhaps the most critical (and sadly, most often lacking) elements in a solid navigational infrastructure are:

• • Multiple modes of navigational support, to accommodate readers ranging from the casual surfer to the focused searcher.
  • Easily accessed shortcuts to the paths that you believe people will want to follow most often
  • Easy-to-find exits, just in case you’re wrong!

Fringe Benefits

Building a reader-friendly Web site is a worthwhile endeavor in its own right. After all, satisfied readers are the key to a truly successful Web site. But there are also certain fringe benefits that go along with genuine reader-friendliness. Reader-friendly Web sites are also:

• Browser-friendly
• Bandwidth-friendly
• Server-friendly

Maintainers of reader-friendly Web sites even receive occasional fan mail!

A Final Note

It’s been said, but it bears repeating:

Always and in everything, remember your reader.