:: QC Boss :: Testing, Independent Software Testing, Manual Testing, Website Testing, Functionality Testing, Usability Testing, QC, QA, UAT

Familiarity with HTML and CSS, the ability to appreciate the unique challenges faced by users with various disabilities, and an understanding of the W3C Accessibility Guidelines is enough to test a web page.

CSS and HTML validation

This step may come as a surprise to many. After all, wouldn’t invalid code either not work or leave a visible bug? Actually, the answer is not necessarily.

The reason can be that some WISIWIG editors generate invalid code and hard-core programmers who write their code by hand can easily omit some bit of HTML or CSS “grammar”. This doesn’t mean non-functioning code it just means it doesn’t meet the standards. I won’t go into specifics here, just think of it as sort of similar to formal collegiate writing. There is a particular standard which is expected. A paper could be written differently, more “free form”, it could contain all the ideas and arguments, and it could be just as well thought out – but because it doesn’t meet the standard it would not get a top grade.

Validating your code has a number of advantages. It decreases the probability of cross browser problems, it tends to eliminate or reduce so called code bloat, and valid code tends to be easier to maintain as well as being compatible with a broader range of assertive technologies used by people with disabilities.

Accessibility Testing – Automated
Automated accessibility testing is an often misunderstood step in the overall process. To some it is everything that needs to be done. “My site is Bobby compliant. Doesn’t that mean it’s accessible?” To others it’s a red herring and should be avoided all together. My take is that it’s an invaluable step. When writing an article I rely on the spellchecker to catch my typos even though I know I still need to go through and check out the copy myself to make sure I have written “Dave” and not “Cave”, for instance. Automated testing finds many issues which could easily be missed by reading the code and so I always begin with it.

Depending on the scale of your project you might be able to use one of several free web based validators or you may opt to buy one of the testing packages available on the market.

The report you will get will include tests which cannot be run by the validator but which it flags for manual examination. Make sure to go through these as well. Most of the tools will describe the issue enough for someone with the above mentioned prerequisites to test.

And lastly, make sure to have any issues raised, fixed before continuing. Doing so will greatly reduce the time required for the remainder of testing.

And please, I cannot emphasize enough that automated testing alone cannot assure accessibility. Please continue with the steps below.

Keyboard Testing
This a simple but very important step. Hide you mouse and navigate your web site using only your keyboard. If you have never done this then you are likely to learn something.

Various groups of people can’t or don’t want to use a mouse. For some it’s just confusing or difficult, especially those with certain motor control problems or sometimes seniors. For others, like blind web users, it’s impossible. Making sure every link, form field, button, or any other functionality in the page is accessible via the keyboard is a basic necessity of web accessibility – but you may also find that to get to the main content or primary form on the page you need to click the Tab Key many times. Though technically accessible, this is extremely inconvenient.

Again, be sure to make any changes required which this phase of testing brought up before continuing.

Screen Reader Testing
To conduct screen reader testing you will need to install the necessary software. It will take some time to get used to and configure your screen reader so be patient. Begin by simply turning off your monitor and listening to your page. Does it make sense? Many web designs depend on visual cues and can get close to unintelligible when those cues aren’t available.

Next, try to carry out one or more of the tasks your website was built for. If it’s an online store, find a product and make a purchase. If it’s an informational site then find key information. Remember – this is the reason you built the site and it is the reason you are making it accessible. If it core functionality depends on a complex form, can you tell which fields are required? If it’s a shopping cart, can you see how much you have spent before making the purchase?

Audience Testing
Various conventions of web design have emerged in the course of the World Wide Web’s short existence which we have grown used too and even depend on to help us navigate a new site. Links appear in a different color (often blue) and underlined. Site wide or global navigation is usually found along the top of the page. Small pictures can often be clicked to get a bigger one. Similarly, there are conventions used in quality accessible design but naturally those of us who aren’t dependant on accessible design may not be aware of them. These might include links, sometimes invisible, along the top of a page which allow the user to skip to various parts of the page, colors with high contrast values or just consistent design throughout the site.

Web accessibility isn’t just fulfilling a set of requirements or validating against predefined checkpoints. It also means quality design. And just as it’s best to leave questions of browser based user interface design to an expert it’s best to have your site checked over by an expert in screen reader user interface design when considering accessibility. And though in theory, there is no reason a sighted specialist couldn’t become such an expert, one who is dependant on screen readers will more likely be intimate with their functions and use, the frustrations of poor web site design and solutions which ease or eliminate those frustrations in practice, not just in theory.

Conclusion
Website accessibility is something every company and organization needs to consider if they have or are planning to have a web presence. It’s not only morally correct but it increases your potential customer base and it is increasingly becoming the law.

Testing doesn’t need to be difficult or expensive. Depending on the scope of your site compliance can be achieved entirely in-house by following the steps above. If you haven’t got the available resources or feel outside, expert help is required there are a number of consultants from which to choose. One should be aware, however, that the services rendered can range from automated testing only (perhaps appropriate for a web development team which is not particularly knowledgeable in accessibility issues but able to understand and implement post test recommendations) to full, regular audits costing 10’s of thousands (possibly appropriate for a company with a large and regularly changing website such as an on-line auctioneer or a newspaper). It is vital, therefore, to understand one’s needs and know which service to seek.

For any larger testing effort target audience testing is key. Unfortunately, most accessibility testing companies don’t offer the service, and some don’t even offer screen reader testing.

Verification process is done through Reviews. The reviews brings visibility from the development process throughout the SDLC, and help the teams to determine whether to proceed development activity at various checkpoints or milestones in the process. It’s executed in term to identify defects in a product early in the life cycle.

Types of Reviews

In-process Reviews.

They look at the product during a specific time period of life cycle, such as during the design activity. They are usually limited to a segment of a project, with the goal of identifying defects as work progresses, rather than at the close of a phase or even later, when they are more costly to correct.

Decision-point or phase-end Reviews.

This type of review is helpful in determining whether to continue with planed activities or not. They are held at the end of each phase.

Post implementation Reviews.

These reviews are held after implementation is complete to audit the process based on actual results. Post-implementation reviews are also know as “Postmortems”, and are held to assess the success of the overall process after release and identify any opportunities for process improvements.

Classes of Reviews

Informal or Peer Review.

In this type of review generally a one-to one meeting between the author of a work product and a peer, initiated as a request for input regarding a particular artifact or problem. There is no agenda, and results are not formally reported. These reviews occur as need-based through each phase of a project.

Semiformal or Walkthrough Review.

The author of the material being reviewed facilitates this. The participants are led through the material in one of the two formats: the presentation is made without interruptions and comments are made at the end, or comments are made throughout. Possible solutions for uncovered defects are not discussed during the review.

Formal or Inspection Review:

An inspection is more formalized than a ‘walkthrough’, typically with 3-8 people including a moderator, reader, and a recorder to take notes. The subject of the inspection is typically a document such as a requirements spec or a test plan, and the purpose is to find problems and see what’s missing, not to fix anything. Attendees should prepare for this type of meeting by reading thru the document; most problems will be found during this preparation. The result of the inspection meeting should be a written report. Thorough preparation for inspections is difficult, painstaking work, but is one of the most cost effective methods of ensuring quality.

Three rules should be followed for all reviews:

1. The product is reviewed, not the producer.

2. Defects and issues are identified, not corrected.

3. All members of the reviewing team are responsible for the results of the review.

In a website generally, the usability problems fall into two categories:

• Site-level usability issues

home page;

information architecture, navigation, and search;

linking strategy;

internally vs. externally focused design;

overall writing style;

page templates, layout, and site-wide design standards;

graphical language and commonly used icons.

 

• Page-level usability issues

specific issues related to the individual pages;

understandability of headlines, links, and explanations;

intuitiveness of forms and error messages;

inclusion or exclusion of specific information;

individual graphics and icons.

 

A usability test with 5 users will typically uncover 80% of the site-level usability problems plus about half of the page-level usability problems on those pages that users happen to visit during the test. The reason for the lower coverage of page-level problems is that different users will visit different pages, so most pages will be tested by less than the 5 users.

Of course, on a large site, most individual pages will not be visited by any of the test users. Thus, the main goal of user testing a site should be to find the site-level problems.

Also, you should obviously take note of the page-level problems so that they can be fixed on those pages that the users happened to visit. The most important outcome of finding page-level usability problems is a better understanding of the extent of page-level problems on your site. Also, the specific set of problems can serve as a starting point for developing a list of typical page-level design shortfalls that need special attention in the design of future pages.

It is possible to run specialised user tests of particularly important pages to increase coverage of the page-level usability problems. For example, it is often a good idea to test registration pages and the pages where users actually buy products or download software. It is possible to substantially increase the number of users who are able to complete their transactions.

Structural testing is white box testing, not black box testing, since black boxes are considered opaque and do not permit visibility into the code. Structural testing is a dynamic technique in which test data selection and evaluation are driven by the goal of covering various characteristics of the code during testing. Assessing such coverage involves the instrumentation of the code to keep track of which characteristics of the program text are actually exercised during testing. The inexpensive cost of such instrumentation has been a prime motivation for adopting this technique. More importantly, structural testing addresses the fact that only the program text reveals the detailed decisions of the programmer. For example, for the sake of efficiency, a programmer might choose to implement a special case that appears nowhere in the specification. The corresponding code will be tested only by chance using functional testing, whereas use of a structural coverage measure such as statement coverage should indicate the need for test data for this case. Structural testing is also known as clear box testing, also known as glass box testing. Structural testing is a way to test software with knowledge of the internal workings of the code being tested. Structural coverage measures for a rough hierarchy, with a higher level being more costly to perform and analyze, but being more beneficial, as described below.

• Statement Testing
• Branch Testing
• Conditional Testing
• Expression Testing
• Path Testing

Question: Name the 5 errors tester makes?

• Not Looking
• Looking but not seeing
• Improper interpretation of requirements
• Wrong selection of testing techniques
• Unable to report a defect clearly
• Creation of wrong test environment
• Execution of test case without sufficient knowledge

Question: Name the causes from which most of the testing problems occur?

1. Failing to define testing objectives

2. Testing at the wrong phase in the life cycle

3. Using ineffective test techniques

Question: You find that the senior testers are making more mistakes then junior testers; you need to communicate this aspect to the senior tester. Also, you don’t want to loose this tester. How should one go about the constructive criticism?

Providing Constructive Criticism

1. Do it privately.

2. Have the facts

3. Be prepared to help

4. Be specific on expectations

5. Follow a specific process

• State the positive first
• Criticize the work, not the individual
• Get an agreement that there is a problem
• Seek advice on how he thinks he can improve his performance
• If individual is unable, suggest a course of action
• Make a specific contract and define expectations

Motivation, Mentoring & Recognition

Motivation

Motivation has sometimes been defined as getting individuals to do work tasks they do not want to do or to perform those work tasks in a more efficient or effective manner. 4 most common motivators are:

1. Personal Challenge

2. Respect

3. Rewards

4. Recognition

Mentoring

Mentoring is helping or supporting an individual in a non-supervisory capacity. Mentors can be peers, subordinates, or superiors. Mentoring can occur in 3 areas:

1. Career counseling

2. Work tasks

3. Professional advancement

Recognition

• Recognition by an IT manager at a formal IT meeting.
• Group luncheons/group celebrations.
• Tokens of appreciation such as a coupon for dinner out or a movie.
• Time off if completing a task involved an excess of work hours.
• Lunch with the boss.

“An effective way to test code is to exercise it at its natural boundaries” — Brian Kernighan

“Testing is the process of comparing the invisible to the ambiguous, so as to avoid the unthinkable happening to the anonymous.”– James Bach

“Testing is organised skepticism.”– James Bach

“Program testing can be used to show the presence of bugs, but never to show their absence!”– Dijkstra

“Beware of bugs in the above code; I have only proved it correct, not tried it.”– Knuth

Software Testers: “Depraved minds, usefully employed.” — Rex Black

Best Testing Practices to be followed during testing:-

• Testing and evaluation responsibility is given to every member, so as to generate team responsibility among all.

• Develop Master Test Plan so that resource and responsibilities are understood and assigned as early in the project as possible. 

• Systematic evaluation and preliminary test design are established as a part of all system engineering and specification work. 

• Testing is used to verify that all project deliverables and components are complete, and to demonstrate and track true project progress. 

• A-risk prioritized list of test requirements and objectives (such as requirements-based, design-based, etc) are developed and maintained. 

• Conduct Reviews as early and as often as possible to provide developer feedback and get problems found and fixed as they occur.

Some Basic Principles of Testing:

• Define the expected output or result.
• Don’t test your own programs.
• Inspect the results of each test completely.
• Include test cases for invalid or unexpected conditions.
• Test the program to see if it does what it is not supposed to do as well as what it is supposed to do.
• Avoid disposable test cases unless the program itself is disposable.
• Do not plan tests assuming that no errors will be found.

ISO 9001 2000 Quality Management Standard

ISO 13485 2003 Medical Device Quality Standard

ISO 14001 2004 Environmental Management Standard

ISO 22000 2005 Food Safety Management Standard

ISO 27001 2005 Information Security Management Standard

ISO 27002 2005 Information Security Management Standard

ISO 90003 2004 Software Quality Management Standard

NFPA 1600 2007 Business Continuity Standard

OHSAS 18001 2007 OH&S Standard

Test engineers should provide evidence to company that the software operates properly and correct. They also assure that the successful launch of your product by discovering bugs and design flaws, before users get discouraged, before shareholders loose their temper, and before your employees get stuck. They save your company money by discovering defects EARLY in the design process, before failures occur in production.

Roles : 

• Reviewing of Requirement and Functional specifications
• Definition of Test Cases based on the Requirements Specifications and the Functional Specs
• Integration and system testing of the product
• Assisting in the delivery of the product [eg. System Pre-configuration and Test Checks]
• Automation of the testing
• Performance Testing and Test Tools implementation
• Development of Quality procedures Documenting of Test objectives, Test Cases, Test reports etc