Archive for November 2008
Approach for Load Testing.
The basic steps that are involved in Load Testing of a Web Application:
Step 1 – Identify Performance Acceptance Criteria.
Step 2 – Identify Key Scenarios.
Step 3 – Create a Workload Model.
Step 4 – Identify the Target Load Levels.
Step 5 – Identify Metrics.
Step 6 – Design Specific Tests.
Step 7 – Run Tests
Step 8 – Analyse the Tesults.
Summary:
- Identify the performance-critical key scenarios;
- Identify the workload profile for distributing all the load among the key scenarios;
- Identify metrics that you want to collect in order to verify them against your performance objectives;
- Create test cases that will be used to simulate the load test;
- Use tools to simulate the load according to the test cases and capture the metrics;
- And finally, analyse the metrics data captured during the tests.
Economics of Testing
“Too little testing is a crime – Too much testing is a sin”. The risk of under-testing is directly translated into system defects present in the production environment. The risk of over-testng is the unnecessary use of valuable resources in testing systems that have no defects, or very few defects that the cost of testing far exceeds the value of detecting the system defect.
Most of the problems associated with testing occur from one of the following causes:
-
Failure to define testing objectives
-
Testing at the wrong phase in the cycle
-
Use of ineffective test techniques
System Administrator?
Test Build Managers, System Administrators, Database Administrators deliver current software versions to the test environment, install the application’s software and apply software patches, to both the application and the operating system, set-up, maintain and back up test environment hardware. Depending on the project, one person may wear more than one hat. For instance, a Test Engineer may also wear the hat of a System Administrator.
Test Build Manager?
Test Build Managers deliver current software versions to the test environment, install the application’s software and apply software patches, to both the application and the operating system, set-up, maintain and back up test environment hardware. Depending on the project, one person may wear more than one hat. For instance, a Test Engineer may also wear the hat of a Test Build Manager.
Basic Checklist of GUI Testing
- Test each toolbar and menu item for navigation using the mouse and keyboard.
- Test window navigation using the mouse and keyboard.
- Check that proper formatted masks are used. i.e., the drop-down boxes should be sorted.
- Date entry should also be properly formatted and sorted.
- Make sure to check the colors, fonts, and font widths are to standard for the field alerts and displayed text.
- Test that the color of the field prompts and field background is to standard in read-only mode.
- Make sure that vertical scroll bars or horizontal scroll bars do not appear unless required.
- Test that the various controls on the window are aligned correctly.
- Make sure that the window is resizable.
- Check the spellings of all the text displayed in the window, such as the window caption, status bar options, field prompts, pop-up text, and error messages.
- Test that all character or alphanumeric fields are left-justified and that the numeric fields are right-justified.
- Check for the display of defaults if there are any.
- In case of multiple pages or windows, check that they all have the same look and feel.
- Check that all shortcut keys are defined and work correctly.
- Check for the tab order. It should be from top left to bottom right. Also, the read-only/disabled fields should be avoided in the TAB sequence.
- Check that the cursor is positioned on the first input field when the window is opened.
- Make sure if any default button is specified, it should work properly.
- Check for proper functioning of ALT+TAB.
- Ensure that each menu command has an alternative hot key sequence and that it works correctly.
- Check that there are no duplicate hot keys defined on the window.
- Validate the behavior of each control, such as push button, radio button, list box, and so on.
- Test to make sure that the window is modal. This will prevent the user from accessing other functions when this window is active.
- Test that multiple windows can be opened at the same time.
- Check to make sure that there is a Help menu.
- Check to make sure that the command buttons are grayed out when not in use.
Stress testing
Stress testing is a form of testing that is used to determine the stability of a given system or entity. The objective of stress testing is to simulate a production environment for the purpose of determining that:
-
Normal or above-normal volumes of transactions can be processed through the transaction within the expected time frame.
-
The application system is structurally able to process large volumes of data.
-
System capacity, including communication lines, has sufficient resources available to meet expected turnaround times.
-
People can perform their assigned tasks and maintain the desired turnaround time.
- The system is not expected to process the overload without adequate resources, but to behave (e.g., fail) in a decent manner (e.g., not corrupting or losing data).
Understanding the Characteristics of Software
The test team should investigate the project characteristics in order to evaluate the potential magnitude of the risk. During that investigation, the testers should at least do the following:
1. Define what it means to meet the project objectives. These are the objectives to be accomplished by the project team.
2. Understand the core business areas and processes. Focusing on core business areas and processes is essential to the task of assessing the impact of the problem and for establishing the priorities for the program.
3. Assess the severity of potential failures. This must be done for each core business area and its associated processes.
4. Identify the components for the system:
· Links to core business areas or processes
· Platform languages, and database management systems
· Operating system software and utilities
· Telecommunications
· Internal and external interfaces
· Owners
· Availability and adequacy of source code and associated documentation
5. Assure requirements are testable. Effective testing cannot occur if requirements cannot be tested to determine if requirements are implemented correctly.
6. Address implementation schedule issues:
· Implementation checkpoints
· Meetings
· Identification and selection of conversion facilities
· Time needed to put converted systems into production
· The conversion of backup and archival data
7. Address interface and data exchange issues:
· Development of a model showing the internal and external dependency links among core business areas, processes, and information systems
· Notification of all outside data exchange entities
· Data bridges and filters
· Contingency plans if no data is received from an external source
· Validation process for incoming external data
· Contingency plans for invalid data
8. Evaluate contingency plans for this system and activities. These should be realistic contingency plans, including the development and activation of manual or contract procedures, to ensure the continuity of core business processes.
9. Identify vulnerable parts of the system and processes operating outside the information resource management area. Include telephone and network switching equipment and building infrastructure systems. Develop a separate plan for their testing.
SEI? CMM? CMMI?
SEI = ‘Software Engineering Institute’ at Carnegie-Mellon University; initiated by the U.S. Defense Department to help improve software development processes.
CMM = ‘Capability Maturity Model’, now called the CMMI (‘Capability Maturity Model Integration’), developed by the SEI. It’s a model of 5 levels of process ‘maturity’ that determine effectiveness in delivering quality software. It is geared to large organizations such as large U.S. Defense Department contractors. However, many of the QA processes involved are appropriate to any organization, and if reasonably applied can be helpful. Organizations can receive CMMI ratings by undergoing assessments by qualified auditors.
Level 1 – characterized by chaos, periodic panics, and heroic efforts required by individuals to successfully complete projects. Few if any processes in place; successes may not be repeatable.
Level 2 - software project tracking, requirements management, realistic planning, and configuration management processes are in place; successful practices can be repeated.
Level 3 – standard software development and maintenance processes are integrated throughout an organization; a Software Engineering Process Group is is in place to oversee software processes, and training programs are used to ensure understanding and compliance.
Level 4 – metrics are used to track productivity, processes, and products. Project performance is predictable, and quality is consistently high.
Level 5 – the focus is on continuous process improvement. The impact of new processes and technologies can be predicted and effectively implemented when required.
WHY THIS FAILURE ?
Software project failures are most common now-a-days in IT industries L, and the biggest projects fail most often. There are always many excuses for these failures, but there are a few common symptoms.
-
Inadequate project definition
-
Inadequate Brief
-
Poor Planning
-
Poor Estimates
-
Reusing the same plan for different projects
-
Incorrect Resource Allocation
-
Wrong Prioritize
-
Accepting to do the work in short span
-
Project workers being overloaded with work
Finally ———————————— LACK OF QUALITY MANAGEMENT
……….
Test Engineer?
Test Engineer is an engineer who specializes in testing. Test engineers create test cases, procedures, and scripts and generate data. They execute test procedures and scripts, analyze standards of measurements, and evaluate results of system/integration/regression testing.