Final Writeup

Final link: http://mv.ezproxy.com.ezproxyberklee.flo.org/collegeroute/

Design

Over the course of the semester, our design has changed and adapted accordingly to the many iterations of paper prototyping and user testing.

...

We tested our interface on three users. Our two original user classes were students and parents who were applying to colleges and planning to visit those schools. The three users tested were broadly representative of those user classes. The users in the evaluation were personal contacts of our team members (either either family or friends).

• MIT undergraduate who has several siblings currently looking at colleges
• Mother of three children with one child currently looking at and visiting colleges and two children who have already completed the college search process
• Mother of two children who have already completed the college search process

Tasks

Our users were given the same tasks that had been given to those testing our paper prototype. Namely:

1. Imagine you are a high school junior with a 3.6 GPA and the following SAT scores: Math: 640, Reading: 690, Writing: 710. Find schools in New England that might be a good fit.  Note: for this prototype the SAT, GPA, and ACT data for each school was randomly generated and does not reflect the actual data associated with that school.
2. Select a few schools that you would like to visit during summer vacation (June 1-8^th, 2013).
3. Plan an itinerary for visiting the schools selected in Task 2.

Users 1 and 3 were briefed on the task in person; User 2 was briefed on the task over Skype (screen-sharing was used to allow for remote observation).

Usability Problems

• (was already familiar with tasks and was not administered standard briefing)

Briefing

Purpose:

• Parents often plan trips for their high school age children to visit colleges, but there are a lot of challenges along the way that we are trying to alleviate.
• Find good fit colleges in a certain geographic area
• Balance multiple campus tours/info session times in one easy place

Tasks:

• A few tasks as if you were a college junior planning visits to schools

Disclaimers:

• We’re not testing you, we’re testing the interface, feel free to ask questions, but also try to explore and see what happens

Conclusion:

• Do you have any questions? If I can’t answer them now because it will interfere with the test I will definitely answer all unanswered questions at the end.

Tasks

Our users were given the same tasks that had been given to those testing our paper prototype. Namely:

1. Imagine you are a high school junior with a 3.6 GPA and the following SAT scores: Math: 640, Reading: 690, Writing: 710. Find schools in New England that might be a good fit.  Note: for this prototype the SAT, GPA, and ACT data for each school was randomly generated and does not reflect the actual data associated with that school.
2. Select a few schools that you would like to visit during summer vacation (June 1-8^th, 2013).
3. Plan an itinerary for visiting the schools selected in Task 2.

Users 1 and 3 were briefed on the task in person; User 2 was briefed on the task over Skype (screen-sharing was used to allow for remote observation).

Usability Problems

1. Calendar view does not default to selected trip dates - Major (Fixed)
   Both Users 2 and 3 could not find events on the calendar originally because the calendar view did not default to the selected trip dates. Therefore, users had to navigate through an unfamiliar calendar interface to the dates they desired. This issue was not fully apparent to the developers because the start date of the trip defaults to today. This issue was easily fixed by changing a setting in the calendar initialization.
2. Users must hit refresh button to apply new date range - Major (Fixed)
   After selecting the dates, Users 2 and 3 struggled to apply their changes; they expected that the new dates would take effect immediately. They did not realize they had to hit "Refresh Events" in order to apply their changes. This issue has been resolved; now, once the start or end dates are changed, the calendar refreshes to reflect the new events.
3. Users did not understand the map affordances - Major
   Users 2 and 3 did not select schools from the map or click on the map in their initial run through the tasks; older users may be less familiar with viewing and clicking on map clusters. User 1 discovered the affordances associated with the clusters and the map pins by accident. Help text would be a convenient way to allow users to discover these affordances.
4. Users did not immediately understand the relationship between displayed search results and map display - Minor
   The displayed search results only include those search results that are within the viewport of the map. Text which explicitly labeled the list of results could help explain this (e.g., "Results in map view") 
5. Users did not understand the ordering of the search results - Minor
   User 3 did not realize that the schools were ranked according to how well they matched the user's search criteria. Using more explicit text for the results list woud also clarify this issue; a header like "Top Results in this Area" could be an effective fix.
6. Users did not initially understand calendar affordances - Minor
   Despite helptext, some users attempted to double-click on calendar events, which has the effect of selecting and deselecting the event. More explicit instructions could resolve this issue, like "Click once on an event to add it to the trip".
7. Users could not figure out how to go back and start a new trip - Minor
   The logo in the top-left hand corner could be bigger or become highlighted upon hovering to better indicate a link functionality.
8. Users creating profiles have to delete the pre-filled values (fragile text does not work) - Minor
   We could change the event handling of clicks inside the input box.
9. After entering scores for a given test (SAT or ACT), section labels (Math, Reading, Writing, etc.) disappear in the user profile page - Minor
   We could add a set of labels for each of these boxes, or add tooltip text
10. Calendar view does not default to selected trip dates - Major (Fixed)
    Both Users 2 and 3 could not find events on the calendar originally because the calendar view did not default to the selected trip dates. Therefore, users had to navigate through an unfamiliar calendar interface to the dates they desired. This issue was not fully apparent to the developers because the start date of the trip defaults to today. This issue was easily fixed by changing a setting in the calendar initialization.
11. Users must hit refresh button to apply new date range - Major (Fixed)
    After selecting the dates, Users 2 and 3 struggled to apply their changes; they expected that the new dates would take effect immediately. They did not realize they had to hit "Refresh Events" in order to apply their changes. This issue 
12. Users did not understand the map affordances - Major
    Users 2 and 3 did not select schools from the map or click on the map in their initial run through the tasks; older users may be less familiar with viewing and clicking on map clusters. User 1 discovered the affordances associated with the clusters and the map pins by accident. Help text would be a convenient way to allow users to discover these affordances.
13. Users did not immediately understand the relationship between displayed search results and map display - Minor
    The displayed search results only include those search results that are within the viewport of the map. Text which explicitly labeled the list of results could help explain this (e.g., "Results in map view") 
14. Users did not understand the ordering of the search results - Minor
    User 3 did not realize that the schools were ranked according to how well they matched the user's search criteria. Using more explicit text for the results list woud also clarify this issue; a header like "Top Results in this Area" could be an effective fix.

General Feedback

All in all, the feedback for CollegeRoute was positive. Users were generally excited about the idea and thought that the overall experience was well-designed and implemented. Very little comments were negative in such a way that required immediate and drastic change or improvement; instead, any constructive criticism seemed to point to features that could be implemented and added on top of the current design.

...