...
- We convened a committee with representation from xxxxxxx, and began studying the current web application testing landscape.
- We looked at a list of X amount of Test Tools (http://www.softwareqatest.com/qatweb1.html#FUNC
) and choose 5 worthy of further evaluation, based on the following criteria:
...
- We created a test plan to evaluate these Test Tools. The test plan was heavily weighted with DOM manipulation, AJAX and other JavaScript functionality. (expand)
- We broke up into groups of 1 to 3 users to try each of the 5 6 Test Tools.
- We modified the Test Plan as needed.
- We compared notes and removed EggPlant, AppPerfect and FuncUnit because:
- AppPerfect: Could not parse an html select and we were not able to get support from the vendor, (as Ed for details)
- EggPlant: Because it is image based, it does not directly test JavaScript and it was maintenance intensive.
- FuncUnit would not record in newer versions of FireFox than 3.X (confirm) and otherwise was similar to and based on Selenium
Lessons Learned
Conclusions:
Test Tools, by their nature, are one step behind the rapidly changing current browser versions. Therefore Test Tools will never be the ONLY testing solution if MIT is going to support current browsers.
The current browser space is expanding and changing and churning rapidly right now, making it even harder for Test Tools to catch up.
Also user and management expectations around browser support and platform are constantly expanding. Chrome, mobile in general and iPhone are the current issues, Windows 8 is looming on the horizon.