...
MIT Touchstone is a new suite of technologies for authenticating a variety of web applications, being introduced by IS&T. MIT Touchstone does provide provides a single sign-on solution for applications that have been coded and configured to use the system. Within the context of Touchstone enabled applications, users will be able to seamlessly transition between systems without being prompted for additional authentication information.
The intended audience of this document includes all IT personnel involved in the development, testing, and support of Touchstone.
...
Risk | Contingency |
Production-like test environment not available | Utilize development or production environment. Results may not be indicative of production and therefore cannot be used as a benchmark. Production performance issues may not be identified during testing. |
Production-like setup and settings not available. | Use the closest setup and settings we can. Results may not be indicative of production and therefore cannot be used as a benchmark. Production performance issues may not be identified during testing. |
Fully operational test tools not available. (i.e. tools capable of generating the desired load and capturing the desires metrics) | Wait until the test tools are available or find and use another test tool(s). This will extend the time required to perform testing. |
Test time increases due to changes in scope requiring additional test analysis and/or test case creation | If test time cannot be increased, reduce/cut performance testing scenarios and execute highest priority scenarios initially followed by lower priority tests until test time runs out |
Involvement of subject matter experts (SMEs) for all stages of the testing effort not sufficient. | If test time cannot be increased, reduce/cut performance testing scenarios and execute highest priority scenarios initially followed by lower priority tests until test time runs out Extend testing time. |
Inadequate Non-functional Requirements | Missing pass/fail criteria invalidates based on stated non-functional requirements will invalidate performance benchmarking. Missing load modeling invalidates will invalidate all test scenarios. Perform The contingency is to perform only a brute stress test to try and flush out major bottlenecks and functionality under load issues. Additionally an endurance test can be run to attempt to identify memory leaks. All , although these tests will be less indicative of real world usage scenarios. |
Insufficient access to systems in order to monitor the tests (This includes including any necessary server side scripts which may need to be developed in order to capture desired metrics). ) | Root cause analysis will be difficult is if possible. Testing time will most likely need to be extended and scenarios may be abbreviated due to time constraints. |
Substantial issue(s) which requires significant modifications to the application or re-configuration of the system are encountered. | Some testing may need to be re-doneexecuted, possibly including re-scripting etc. This would extend testing time. |
Excessive number of bottlenecks encountered and/or issue correction time. | Extend testing time. |
Test time increases due to changes in scope requiring additional test analysis and/or test script/scenario creation . | If test time cannot be increased, reevaluate priorities and risk risks and test execute tests according to new priorities. |
Conflicting information provided. | Project manager will work to resolve any conflicting information, goals, and/or instructions received. |
...
- Performance - Benchmark the system to ensure it meets all documented non-functional requirements related to performance.
- Stress - Push the system to it its breaking point and beyond to identify how and under what level of load the system fails as well as the ramifications of such a failure.
- Endurance - Place the system under a heavy, yet manageable, load for a protracted period of time to identify any performance degradation and/or memory leaks.
- Fail-over - Place the system under a heavy, yet manageable, load, wait for it to stabilize and then disconnect the servers from their network connections to identify how the system handles the sudden loss of a server. This will help satiate any up-time SLAs or non-functional requirements.
...
Tool | Purpose | Used By |
Atlassian Jira | Web-based defect tracking system accessed by http://mv.ezproxy.com.ezproxyberklee.flo.org/jira | Touchstone Project Team (MIT & Questcon) |
StressTester | Performance testing / load generation tool | (MIT & Questcon) |
3.3 Environmental Needs
We will need the following:
...
The following scripts will be used during the performance testing effort.
When the design steps have been provided by MIT all of the to be determined (TBD) values will be replaced with the actual values.
4.1 CAMS Account Creation
Precondition: TBD
Data Needed: TBD
Transaction Name | Step(s) | Expected Result | 95th % Response Time |
|---|---|---|---|
TBD |
|
|
|
We will be able to specify the precondition, data needs and steps once these the design steps have been provided by MIT.
4.2 CAMS Association - OpenID
Precondition: TBD
Data Needed: TBD
Transaction Name | Step(s) | Expected Result | 95th % Response Time |
|---|---|---|---|
TBD |
|
|
|
We will be able to specify the precondition, data needs and steps once these the design steps have been provided by MIT.
4.3 CAMS Association - Kerberos
Precondition: TBD
Data Needed: TBD
Transaction Name | Step(s) | Expected Result | 95th % Response Time |
|---|---|---|---|
TBD |
|
|
|
We will be able to specify the precondition, data needs and steps once these the design steps have been provided by MIT.
4.4 Site Access - Kerberos w/ticket
Precondition: TBD
Data Needed: TBD
Transaction Name | Step(s) | Expected Result | 95th % Response Time |
|---|---|---|---|
TBD |
|
|
|
We will be able to specify the precondition, data needs and steps once these the design steps have been provided by MIT.
4.5 Site Access - Web Auth
Precondition: TBD
Data Needed: TBD
Transaction Name | Step(s) | Expected Result | 95th % Response Time |
|---|---|---|---|
TBD |
|
|
|
We will be able to specify the precondition, data needs and steps once these the design steps have been provided by MIT.
4.6 Site Access - CAMS Account
Precondition: TBD
Data Needed: TBD
Transaction Name | Step(s) | Expected Result | 95th % Response Time |
|---|---|---|---|
TBD |
|
|
|
We will be able to specify the precondition, data needs and steps once these the design steps have been provided by MIT.
4.7 Site Access - OpenID
Precondition: TBD
Data Needed: TBD
Transaction Name | Step(s) | Expected Result | 95th % Response Time |
|---|---|---|---|
TBD |
|
|
|
We will be able to specify the precondition, data needs and steps once these the design steps have been provided by MIT.
4.8 Password Reset
Precondition: TBD
Data Needed: TBD
Transaction Name | Step(s) | Expected Result | 95th % Response Time |
|---|---|---|---|
TBD |
|
|
|
We will be able to specify the precondition, data needs and steps once these the design steps have been provided by MIT.
4.9 Admin - Password Reset
Precondition: TBD
Data Needed: TBD
Transaction Name | Step(s) | Expected Result | 95th % Response Time |
|---|---|---|---|
TBD |
|
|
|
We will be able to specify the precondition, data needs and steps once these the design steps have been provided by MIT.
4.10 Admin - De-Activate Account
Precondition: TBD
Data Needed: TBD
Transaction Name | Step(s) | Expected Result | 95th % Response Time |
|---|---|---|---|
TBD |
|
|
|
We will be able to specify the precondition, data needs and steps once these the design steps have been provided by MIT.
4.11 Admin - Delete Account
Precondition: TBD
Data Needed: TBD
Transaction Name | Step(s) | Expected Result | 95th % Response Time |
|---|---|---|---|
TBD |
|
|
|
We will be able to specify the precondition, data needs and steps once these the design steps have been provided by MIT.
4.12 Admin - Activate Account
Precondition: TBD
Data Needed: TBD
Transaction Name | Step(s) | Expected Result | 95th % Response Time |
|---|---|---|---|
TBD |
|
|
|
We will be able to specify the precondition, data needs and steps once these the design steps have been provided by MIT.
** Design steps for the following scripts have not been provided by MIT yet.
Script | Objective | Priority |
|---|---|---|
CAMS Account Creation | End user creates a Touchstone account. | High |
CAMS Association - OpenID | End user associates their Touchstone account with an OpenID account. | Medium |
CAMS Association - Kerberos | End user associates their Touchstone account with a Kerberos account. | High |
Site Access - Kerberos w/ticket | End user access a site using an MIT machine that has a Kerberos ticket. | High |
Site Access - Web Auth | End User access a site using Web Auth. | High |
Site Access - CAMS Account | End user access a site using their CAMS accont only. | High |
Site Access - OpenID | End user access a site using their OpenID. | Medium |
Password Reset | End User resets their password. | Low |
Admin - Password Reset | Administrator resets an end user's password. | Low |
Admin - De-Activate Account | Administrator de-activates an end user's account | Low |
Admin - Delete Account | Administrator deletes an end user's account. | Low |
Admin - Activate Account | Administrator activates a de-activated account. | Low |
5.0 Scenarios
5.1 Performance Testing Scenarios
...
The objective of this scenario is to check how both IdPs handle a sudden interruption in connectivity by pulling the network plug from 1 from one of the servers ( at a time).
5.4.1.1 Load Model
Desired Transaction Rate: TBD
...
Architectural diagrams will be reverenced referenced here as MIT proved provides them.
8.1 Physical
Touchstone Production Physical Architecture
...
Key Deliverables | Description | Expected Delivery Date | Resource |
Performance Test Plan | This document. | After all non-functional requirements and other needed data is delivered | Questcon |
Performance Test Scripts | Automated scripts used to deliver load. | 30 business days after test plan finalization and environmental needs are met. | Questcon |
Performance Test Scenarios | Automated execution designs used to conduct performance tests. | 5 ~5 business days after scripts are developed. | Questcon |
Execute Performance Tests | Running of scenarios. | 10 business days | Questcon |
Status Reports | Accomplishments, issues and plans. | Weekly | Questcon |
Defect Reports | Entered in Jira as they are discovered. | Ongoing during test execution | Questcon |
Performance Test Summary Report | Details the results of the testing effort. | 3 business days after the last performance test is completed. | Questcon |
...
Milestone | Target Timeframe | Summation of Activities |
Develop performance test plan | 01/15/2008 - 02/05/2008 |
|
Review performance test plan | 02/05/2008 - 02/11/2008 |
|
Build Performance test scripts | //2008 - //2008 |
|
Build Performance test scenarios | //2008 - //2008 |
|
Setup test data | //2008 - //2008 |
|
Execute performance tests | //2008 - //2008 |
|
Create test summary | //2008 - //2008 |
|
TOTAL: | 73 Business Days |
|