DESIGN
IMPLEMENTATION
EVALUATION
Method
We conducted our testing using the same briefing and tasks from GR3 Paper Prototyping. Throughout GR4 and GR5 the intended purpose of our application and it's features did not change, so we decided that the same briefing and tasks would still be suitable and sufficient to test our finished implementation.
During pilot and user testing our users were given the briefing and then began completing the tasks one by one, which were dictated to them. There was no demo given since our testing involved tasks that highlighted every main feature of the application. Therefore, any sort of demo would have caused us to bias user performance on some of our tasks. We opted to have more thorough testing with all of the nine original user tasks, rather than give a demo and have to omit some of the tasks from the testing process.
In addition to the briefing, during our actual user testing, users were given instructions to think out loud, to not worry about how quickly they completed the tasks, and to feel free to give up and ask for the solution to a task if they got frustrated.
In both situations our users were completely representative our our target audience, as they were all current MIT students.
Pilot Testing Results
Pilot testing was conducted on four 6.813 / 6.831 students on Demo Day. All four users had no trouble completing each of the 9 tasks but provided feedback that brought to light the issues detailed below. More important however, our pilot testing assured us that our testing method was clear and concise enough to be used for our final user testing.
1. Cosmetic (graphic design): Two users pointed out that the add/delete group button looks too similar to the Google maps navigation buttons.
Buttons for adding and deleting a group attached along the sidebar
This issue did not affect their ability to figure out what the buttons were for, so it was deemed cosmetic. To alleviate this issue, we could have affordances for those buttons other than a ' + ' and ' - '. For example, the button for adding a group could read ' + Group ' instead of just ' + '.
The status text field says "your status message" when it's blank, but once a status message is submitted, there are no other hints what that text field is for. To alleviate this issue, there is enough room on the left-hand side of the status bar to include the label "Status: "
Room for a "Status" label on the left
html: Security restricted macro is not allowed. An edit restriction is required that matches the macro authorization list. <BR></BR><BR></BR>
3. Major(visibility): Managing friends and hiding / un-hiding from a friend is delayed with no feedback.
The response time for these features are too long, and to fix this issue we would have to modify our backend. Local variables would get modified when information is submitted to the server, so that the user can see the result of their actions right away as opposed to having to wait for a response from the server before changed are made visible.
User Testing Results
User testing was conducted on three students who have never taken either course and have never been educated in user interface design through any other sources. These students were recruited simply by asking friends if they would like to participate in our user testing and letting them know what the time commitment would be and what would be expected of them before they agreed. The results of our user testing proved to be much more interesting than our pilot testing. Some of the issues our users had were unexpected, and it was surprising that each had the same exact intuitions. Our user testing brought to light the following issues.
1. Minor (learnability): When logging in, none of our three users clicked on the mood icons to login.
Our users all simply typed in the given username and password and hit enter. Our application allows this and will default the user's mood to "Adventurous" if they do not pick one. Upon being told how the login was intended to work, one user replied that she would have never thought that those icons had a purpose other than decoration. When mousing over the icons, the mouse changes from an arrow to a hand, indicating that these icons ca be clicked on and have a purpose, however this is only beneficial if the user never even thinks to mouseover the icons.
It is possible here to see the mood icons as decoration, as opposed to functioning buttons
A possible solution could be to leave things as is, because if the user does not select a mood upon login, they can change their mood afterwards in the status bar. Another option would be to disable the fact that clicking "Enter" will log a user in, and instead require that a mood icon be clicked on. In this case, if the user does hit "Enter" to try and login, the mood icons could be highlighted to show the user they must choose one to login.
2. Minor (learnability): When tasked with hiding from a friend, all three users initially clicked in the hiding from area, two of them specifically attempting to click within the placeholder.
When asked why that was their first intuition, two users stated that the placeholder made it look as if they could type in a name. However, when clicking in the "Hiding from" area] in the bottom bar, all three users went to click on the "Hide from user" eye next to the designated friend, successfully completing the task without too much hassle. state possible solution
3. Major (learnability): All three users had issues discovering how to create a group.
Two of the users took significantly longer to complete the task "Create a group called Study Buddies" than any other task, and one user opted to be told how to do it. All three users commented that they thought the ' + ' and ' - ' signs meant that the buttons were meant for zoom functionality on the map. On user also commented that it did not help that the buttons were separated from the group tabs, as opposed to just being right below. This issue was acknowledged in pilot testing, however the fact that these users had significantly more trouble with the task and one user having to ask how to do it, we now consider this issue a major issue. The solution to this issue remains the same as described above