Evaluation

Users

The pool of test users are quite representative of the target audience. We built the website mainly targeting college students, and because of this the bulk of the test users were college students. We have two other users that are apart of different populations; while the site was not directly targeting to them, we wanted to make sure that the site was still usable. 

The users we tested on are the following:

• course 5 sophomore at MIT
• course 6 junior at MIT
• course 7 junior at MIT
• high school CS teacher
• course 6 MIT alum '06

Briefing

Hi, I'm _____ and these are my partners _____ and _________. Thanks for helping us out! We're testing out a system to help people manage a job search. The system, named JobTracker, is a website that people can use to manage documents and keep track of tasks related to their job search.

We'll show you a prototype of the website. You'll get a few index cards with tasks written on them. Try to execute these tasks on our prototype. After we start, my partners and I will be taking notes on how we can improve our interface design.

Remember, this is a test of the interface and not a test of you. The interface is in a preliminary stage and might have problems that make it difficult to use. Your input is important to fixing these problems. While you execute the tasks, feel free to think out loud so we can understand your thought process. Also, you are free to stop the test at any time. Before we get started, do you have any questions for me or my partners?

Tasks

The tasks we gave the test users are listed below:

1. Log in using your email address
2. Add a new Company, "BBN"
3. Add a new Document, "resume" (which is a file on your computer) and link it with BBN
4. Add a new Contact to BBN, "Jane Doe jdoe@bbn.com"
5. Add a Task called "Cover Letter" for BBN due 5/15/11
6. View all upcoming tasks
7. Delete the "Cover Letter" task for BBN

Demo

Users were not given a demo. We felt a demo would hurt learnability testing for this site, and we wouldn't find out if design choices we made had or had not improved learnability. Since the target audience would not normally get a demo from the designs of the site, we felt that the best way to find out any issues was let test users approach the site the same way we though new users would.

Usability issues

1. While not directly a design issue, "View All Tasks" task seems to be universally difficult. It is possible that since there was only one company in the testing environment that all tasks were on that company. One approach to test this would be to add some information on the site, however since we wanted to recreate what a new user would see that would not be the best approach. I think the best approach would have been to add multiple companies and add multiple tasks before given them this task, but since the test users' time was precious we felt that this was probably the best way. One user was confused by the word "upcoming", thought that might be different than just viewing all the tasks, tried to use the company selection box, and ended up more confused.
2. It wasn't clear which fields on all the forms (Add Company, Add Task, Add Contact) was required, this caused some users to make up information, while others just tried submitting without them. We could resolve this issue by adding (Required) in the placeholder text, or we labels with asterisks  could be re-added.
3. The icons on the upper left hand corner (Home, Add Document) were hard to locate, and took a long time. One user went to the company details page to add a document, while the majority eventually found the add document button. To fix this issue, it would make sense to move the logo to the top-left corner and move all the other buttons to the top right corner creating a navigation bar. Since most sites use this design, it will be easier for users to find them.
4. It was not apparent what the "home" icon did for one user, and the thought the "logout" icon was to get back to the main page. One solution would be to replace the home icon with the Logo and have the logo link back, or to add a back icon on the bottom of the page like some sites.
5. The dialog alerting users about unsaved data, confused most users while others just ignored it and clicked through. One possible solution would be to save changes on leaving the page, or to just not save it. The alert itself didn't seem to help convey that something was unsaved.
6. "Details" link is not visible enough for most users. It's hard to find, small,and did not line up with the delete icon. Users expanded the company view attempting to get to the company page. A solution would be to make the details link larger, and have the company name link to the companies page also. 
7. Many users were unclear what a "task" was. SOLUTION? One possible solution could be to change the title of the boxes to "To do list" or something to that effect.
8. One user attempted to click on the filler text to add a company. One solution to this problem would be to let the text be able to add a company.
9. The submit button remains disabled until the text boxes are no longer selected. This was confusing to several users, to fix this we could have the event trigger when enough data is added, or to just not disable the submit button.
10. One user forgot the purpose of the "Name" field in the "Add Contact" form after selecting it, and since it was selected the placeholder text was no longer visible. One solution would be to add tooltips, so that hovering over the field would also give that information, or to add fields for the form back.
11. One user entered dates in directly to the text box and didn't use the date picker. He ended up entering in 5/15/11, which will confuse the database ordering. A solution to this would be to sanitize the inputs before inputting them into the database.'
12. In the expanded company box, the notes text area looks editable when it isn't. A solution would be to just use a div instead of a textarea object for that part of the widget.