...
- Production-like test environment
- Production-like setup and settings
- Fully operational test tools
- Involvement of subject matter experts (SMEs) for all stages of the testing effort
- Adequate Non-functional Requirements
- Sufficient access to systmes systems in order monitor (This includes any necessary server side scripts which may need to be developed in order to capture desired metrics.)
If any of the above items are not available the testing effort will not reach all of its goals. If these items are latent the testing effort will take longer than expected.
If we encounter any substantial issue which requires significant modifications to the application or re-configuration of the system some testing may need to be re-done, possibly including re-scripting etc.
3.0 Approach
3.1 Testing Strategy
The overall strategy for performance testing the Touchstone project is goal based. There are four main goals whe hope to acheive:
- Performance - Benchmark the system to ensure it meets all non-functional requirements related to performance.
- Stress - Push the system to it breaking point and beyond to identify how and under what level of load the system fails as well as the ramifications of such a failure.
- Endurance - Place the system under a heavy, yet manageable, load for a protracted period of time to identify any performance degradation and/or memory leaks.
- Fail-over - Place the system under a heavy, yet manageable, load, wait for it to stabilize and then disconnect the servers from their network connections to identify how the system handles the sudden loss of a server. This will help satiate any up-time SLAs or non-functional requirements.
Scripts will be designed to model various user interactions with the system. While most of the user interactions will be scripted, some may be omitted according to the 80/20 rule and/or any time constraints which may exist.
3.2 Tools
The tools we will employ are yet to be determined.
3.3 Environmental Needs
We will need the following:
- Stable production like system to test against.
- Stable hardware and software to use to generate load.
- Adequate rights and privileges to capture server side metrics (monitoring) as well as any server side scripts necessary to accomplish any needed monitoring.
4.0 Scripts
4.1 CAMS Account Creation
...