Versions Compared

Old Version 33

changes.mady.by.user Michael B Berger

Saved on

compared with

New Version 34

changes.mady.by.user Michael B Berger

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

IS&T web applications include many complex browser-based user interfaces created with JavaScript, HTML and CSS.  The functional/regression test tool we currently use, QTP, only works in Internet Explorer, so it does not test IS&T’s set of supported web browsers. Therefore QTP cannot test individual browser issues.  IS&T needs to evaluate it's web applications against new web browsers and changes to the IS&T web application infrastructure (new database, new application server, new VM, etc.). Therefore we convened a cross-directorate team to evaluate functional web testing tools to see if any would fit IS&T's needs.

What We Recommend:

Roll out Selenium to web developers with the following requirements and caveats:

  1. No support for current version of Safari, may be restored in future.
  2. Hardware requirements: need to be able to get to the all the OS/brower combos
  3. Need someone who is the product evangelist, creates the docs, builds some baseline tests, shows handle problem items (type versus type keys), trains developers and help desk
  4. Need dedicated hardware/software and test users (Touchstone and certificates)
  5. Need someone to coordinate and maintain the environment
  6. Management buyin and continued support
  7. A project and a project plan and project manager

What we did:

  1. We convened a committee with representation from the Help Desk (Lisa Robinson), the Quality Assurance Team (Sean Velury and Don Flanders), Web Services (Michael Berger and Ed Orsini), Stellar (Judith McJohn) Student Systems (Felicia Young), and SWRT (Alex Kozlov) and began studying the current web application testing landscape.
  2. We looked at a list of approximately 82 different automated Test Tools (http://www.softwareqatest.com/qatweb1.html#FUNC) and choose 6 worthy of further evaluation, based on the following criteria:
    1. Support for all MIT operating systems
    2. Support for all MIT browsers
    3. Ability to playback tests in a browser (instead of testing via browser emulation)
    4. Ability to record tests (to make test creation easier)
    5. Some sort of gui (versus a code only framework)
    6. Decent documentation
    7. Some sort of name in the industry
  3. We created a test plan to evaluate these Test Tools (view the test plan here). We created the test plan based on an existing administrative web application called APR Hires. The test plan involved data input of text, numbers, dates and dollar amounts. It included testing AJAX calls, JavaScript Validation of user input, insertion of new DOM elements. The test plan contains positive test assertions (checks in the software to confirm that the expected result occurred) for most user input. The test plan did not contain negative assertions (tests that unexpected things did not happen).
  4. We broke up into groups of 1 to 3 users to try each of the 6 Test Tools.
  5. We modified the Test Plan as needed.
  6. We found the following:

...

Therefore supporting constant rollout of the latest browsers requires manual testing resources. However, automated Testing Tools do improve the quantity and coverage and reduce time for testing Test Tool-supported browsers.

Recommendations:

Roll out Selenium to web developers with the following requirements and caveats:

...

.

...

Further Study:

Once QTP 11 final ships, retest the Test Plan in QTP 11 for both IE 9 and Firefox.

...