Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

IS&T web applications contain complex browser-based user interfaces created with JavaScript, HTML and CSS.  The functional/regression test tool we currently use, QTP, only works in Internet Explorer, so it does not test IS&T’s set of supported web browsers. Therefore QTP cannot test individual browser issues.  Also, IS&T needs to evaluate its web applications against new web browsers and changes to the IS&T web application infrastructure (new database, new application server, new VM, etc.)

What we did:

  1. We convened a committee with representation from the Help Desk, the Quality Assurance Team, Web Services, DCAD and Student Systems, and began studying the current web application testing landscape.
  2. We looked at a list of approximately 82 different automated Test Tools (http://www.softwareqatest.com/qatweb1.html#FUNC) and choose 6 worthy of further evaluation, based on the following criteria:
    1. Support for all MIT operating systems
    2. Support for all MIT browsers
    3. Ability to playback tests in a browser (instead of testing via browser emulation)
    4. Ability to record tests (to make test creation easier)
    5. Some sort of gui (versus code only framework)
    6. Decent documentation
    7. Some sort of name in the industry
  3. We created a test plan to evaluate these Test Tools (the test plan is here). The test plan was heavily weighted with DOM manipulation, AJAX and other JavaScript functionality. (expand)
  4. We broke up into groups of 1 to 3 users to try each of the 6 Test Tools.
  5. We modified the Test Plan as needed.
  6. We found the following:

criteria

AppPerfect

EggPlant

FuncUnit

QTP 11

Selenium

Squish

adequate documentation  (1 to 10 scale)

3

-2

5

6.5

6.5

4

longevity/viability of vendor (1 to 10 scale)

 

8/unknown

5yrs/

20yrs/10

9yrs/9

unknown

recorded test plan

yes

yes

no

yes

yes

yes

ran test plan in IE 9/Win

no

yes

did not test because it could not record

yes

yes

 

ran test plan in Firefox 8/Win

no

yes

did not test because it could not record

no

yes

 

ran test plan in Firefox 8/Mac

no

did not test

did not test because it could not record

no

yes

 

ran test plan in Safari 5/Mac

no

did not test

did not test because it could not record

no

no

sort of

Conclusions

 

 

 

 

 

 

...