...
- We created a test plan to evaluate these Test Tools. The test plan was heavily weighted with DOM manipulation, AJAX and other JavaScript functionality. (expand)
- We broke up into groups of 1 to 3 users to try each of the 6 Test Tools.
- We modified the Test Plan as needed.
- We compared notes and removed EggPlant, AppPerfect and FuncUnit because:
- AppPerfect: Could not parse an html select and we were not able to get support from the vendor, (as Ed for details)
- EggPlant: Because it is image based, it does not directly test JavaScript and it was maintenance intensive.
- FuncUnit would not record in newer versions of FireFox than 3.X (confirm) and otherwise was similar to and based on Selenium
criteria | AppPerfect | EggPlant | FuncUnit | QTP 11 | Selenium | Squish |
|---|---|---|---|---|---|---|
adequate documentation (1 to 10 scale) | 3 | -2 | 5 | 6.5 | 6.5 | 4 |
longevity/viability of vendor |
| 8/unknown | 5yrs/ | 20yrs/10 | 9yrs/9 | unknown |
recorded test plan | yes | yes | no | yes | yes | yes |
ran test plan in IE 9/Win | no | yes | did not test because it could not record | yes | yes |
|
ran test plan in Firefox 8/Win | no | yes | did not test because it could not record | no | yes |
|
ran test plan in Firefox 8/Mac | no | did not test | did not test because it could not record | no | yes |
|
ran test plan in Safari 5/Mac | no | did not test | did not test because it could not record | no | no | sort of |
Conclusions |
|
|
|
|
|
|
Lessons Learned
Record and play is a myth; it turned out that programming was needed with Every tool to be able to reproduce the test case.
...