...
- Production-like test environment
- Production-like setup and settings
- Fully operational test tools
- Involvement of subject matter experts (SMEs) for all stages of the testing effort
- Adequate Non-functional Requirements
- Sufficient access to systems in order monitor (This includes any necessary server side scripts which may need to be developed in order to capture desired metrics.)
If any of the above items are not available the testing effort will not reach all of its goals. If these items are latent the testing effort will take longer than expected.
...
4.1 CAMS Account Creation
Precondition: TBD
Data Needed: TBD
Transaction Name | Step(s) | Expected Result | 95th % Response Time |
|---|---|---|---|
TBD |
|
|
|
4.2 CAMS Association - OpenID
Precondition: TBD
Data Needed: TBD
Transaction Name | Step(s) | Expected Result | 95th % Response Time |
|---|---|---|---|
TBD |
|
|
|
4.3 CAMS Association - Kerberos
Precondition: TBD
Data Needed: TBD
Transaction Name | Step(s) | Expected Result | 95th % Response Time |
|---|---|---|---|
TBD |
|
|
|
4.4 Site Access - Kerberos w/ticket
Precondition: TBD
Data Needed: TBD
Transaction Name | Step(s) | Expected Result | 95th % Response Time |
|---|---|---|---|
TBD |
|
|
|
4.5 Site Access - Web Auth
Precondition: TBD
Data Needed: TBD
Transaction Name | Step(s) | Expected Result | 95th % Response Time |
|---|---|---|---|
TBD |
|
|
|
4.6 Site Access - CAMS Account
Precondition: TBD
Data Needed: TBD
Transaction Name | Step(s) | Expected Result | 95th % Response Time |
|---|---|---|---|
TBD |
|
|
|
4.7 Site Access - OpenID
Precondition: TBD
Data Needed: TBD
Transaction Name | Step(s) | Expected Result | 95th % Response Time |
|---|---|---|---|
TBD |
|
|
|
4.8 Password Reset
Precondition: TBD
Data Needed: TBD
Transaction Name | Step(s) | Expected Result | 95th % Response Time |
|---|---|---|---|
TBD |
|
|
|
4.9 Admin - Password Reset
Precondition: TBD
Data Needed: TBD
Transaction Name | Step(s) | Expected Result | 95th % Response Time |
|---|---|---|---|
TBD |
|
|
|
4.10 Admin - De-Activate Account
Precondition: TBD
Data Needed: TBD
Transaction Name | Step(s) | Expected Result | 95th % Response Time |
|---|---|---|---|
TBD |
|
|
|
4.11 Admin - Delete Account
Precondition: TBD
Data Needed: TBD
Transaction Name | Step(s) | Expected Result | 95th % Response Time |
|---|---|---|---|
TBD |
|
|
|
4.12 Admin - Activate Account
Precondition: TBD
Data Needed: TBD
Transaction Name | Step(s) | Expected Result | 95th % Response Time |
|---|---|---|---|
TBD |
|
|
|
5.0 Scenarios
5.1 Performance Testing Scenarios
...
Desired Transaction Rate: ??? TBD
Script | % of Load |
|---|---|
Site Access - Kerberos w/ticket | 50% TBD |
Site Access - Web Auth | 50% TBD |
5.1.2 IDPe Only
The objective of this scenario is to benchmark just the exzternal IDP.
...
Desired Transaction Rate: ??? TBD
Script | % of Load |
|---|---|
CAMS Account Creation | 20% TBD |
CAMS Association - OpenID | 20% TBD |
CAMS Association - Kerberos | 20% TBD |
Site Access - CAMS Account | 20% TBD |
Site Access - OpenID | 20% TBD |
5.1.3 Integrated IDP External & Internal
...
Desired Transaction Rate: ??? TBD
Script | % of Load |
|---|---|
CAMS Account Creation | 10% TBD |
CAMS Association - OpenID | 10% TBD |
CAMS Association - Kerberos | 10% TBD |
Site Access - CAMS Account | 10% TBD |
Site Access - OpenID | 10% TBD |
Site Access - Kerberos w/ticket | 25% TBD |
Site Access - Web Auth | 25% TBD |
5.2 Stress Testing Scenarios
5.2.1 IDPi Only
The objective of this scenario is to stress only the internal IDP. We plan to push it gradually up to its breaking point and then beyond to determine how and at what load it fails.
5.1.3.1 Load Model
Desired Transaction Rate: OPEN
Script | % of Load |
|---|---|
Site Access - Kerberos w/ticket | TBD |
Site Access - Web Auth | TBD |
5.2.2 IDPe Only
The objective of this scenario is to stress only the external IDP. We plan to push it gradually up to its breaking point and then beyond to determine how and at what load it fails.
5.1.3.1 Load Model
Desired Transaction Rate: OPEN
Script | % of Load |
|---|---|
CAMS Account Creation | TBD |
CAMS Association - OpenID | TBD |
CAMS Association - Kerberos | TBD |
Site Access - CAMS Account | TBD |
Site Access - OpenID | TBD |
5.2.3 Integrated IDP External & Internal
The objective of this scenario is to stress both IDPs concurrently. We plan to push it gradually up to its breaking point and then beyond to determine how and at what load it fails.
5.1.3.1 Load Model
Desired Transaction Rate: OPEN
Script | % of Load |
|---|---|
CAMS Account Creation | TBD |
CAMS Association - OpenID | TBD |
CAMS Association - Kerberos | TBD |
Site Access - CAMS Account | TBD |
Site Access - OpenID | TBD |
Site Access - Kerberos w/ticket | TBD |
Site Access - Web Auth | TBD |
5.3 Endurance Testing Scenarios
...