GR6 - User Testing

Design

Final Design

In our final design, we divided our website into three pages: Search, Results, and Friends. These pages were navigated with a tab interface. The search page allowed users to input their search criteria. It included a widget that allowed users to efficiently select times for which they were busy, as well as a drop down menu for selecting specific departments. The results page displayed the results of their search in a gigantic table which was organized by starting-time of the class. Finally, the friends page allowed a user to search for their friends and see what classes they were taking. 

Our final design was very graphical. We used colorful post-it notes to represent classes. These post-it notes were draggable, and we had several windows in a sidebar on the right which users could drag the classes into. These windows represented classes the user was considering, that they planned on taking, and that they were not interested in. The contents of these windows were saved for the user, so they could return to the site and view their selections at any time. Users could drag both the classes on the Results Page, and classes their friends were taking on the Friends page. We also had a feature that allowed users to print their selected classes.

Search Page

Results Page

Friends Page

Design Decisions

One of our biggest design decisions was to make our interface very visual and to incorporate drag-and-drop operation. Our reasoning was that this scheme has been growing in popularity, so it would be familiar and comfortable for many reasons. Our alternatives were a more conventional interface that simply displayed all the results in a list instead of providing an elaborate graphical interface, and an interface entirely based on text commands for efficiency. The results from our paper prototyping and our heuristic evaluation made us more confident about this design decision, because everyone seemed to like the colorful, draggable interface. In addition, we found that a common use pattern was that people selected a small subset of courses that they were interested in and then wanted to compare them side-by-side to make their final course decisions. With a tradition interface that simply listed all results vertically, such as the current Course Catalog, users reported that they often opened two tabs and switched back and forth between them to accomplish this; by representing courses as manipulable objects we enabled users to physically group the courses they were interested in in just one window.

Much of our website's layout and organization was also guided by our evaluations. Through our paper prototyping, we decided to have a small drop-down menu for department-selection instead of taking up the whole page. We were originally going to have a "My Classes" page which had features like printing classes, but we instead just moved that to the right sidebar. 

Other design decisions focused on visibility of information. We chose to make pop-up info boxes appear when the user clicked on classes, include the class names on the post-it notes, and include a legend for the HASS categories. We had originally not included these for sake of simplicity, so that the interface would look cleaner. However, an application that works with classes inherently handles lots of information, and through our evaluations, we found that users unanimously requested the information we had hidden. So we included it at the risk of making the page appear more cluttered. 

There were many other minor decisions, like making the mouse a finger-icon when hovering over the draggable post-its. Needless to say, the evaluations were invaluable in improving our design and finding flaws.

Implementation

• We implemented the backend with php and a mySQL database. All of the class data is stored in the database. Using mySQL was very useful because it makes searching for different criteria very quick and efficient.
• We chose scripts.mit.edu to take advantage of features such as certificates login. The certificates allow us to save the sidebar data to a particular username, so that no login is needed to use the site.
• We also used php and mySQL to implement the persistency of classes in the right sidebar. Every time the user added or removed a class from a balloon, we would use an AJAX call to update a table in the database. This increased simplicity of the interface by removing the need for a "save" button. It also increased reliability of the website because the changes were always saved as soon as the user made them.
• Friends were also implemented with a mySQL database. The friends page is pre-populated with the names of everyone who has an entry stored for "classes I'm taking" in our database. This allows for fast,efficient searching of friends.
• We implemented the drag-and-drop functionality for courses using JQueryUI's Draggable and Droppable interfaces.

Evaluation

We tested our interface on four MIT students who had never seen our interface before. Three of them were asked to complete the tasks listed below, while the fourth was simply instructed to "Use this website to select your HASS classes for next semester." This allowed us to get feedback on not only the usability of all features in our interface, but also the discoverability of those features and on which features were most interesting to users.

We selected our test users from our friends and dorm-mates, trying to get a set of users that was as representative of our user population as possible. We feel that our test users were fairly representative of our target user base; they included both males and females, upperclassmen and underclassmen, students who enjoy HASS classes and students who simply want to fulfill their graduation requirements. One notable sub-group that was missing from our test users was freshmen, who are on the new HASS system. However, we felt that the difference in HASS classifications would not significantly impact usability or use patterns of our site.

We did not provide any demo as part of our user testing.

Briefing

I Can Has HASS is a schedule planning and course selection tool aimed at helping MIT students to select humanities courses each semester.

The site allows students to search for courses matching certain criteria, look at what courses their friends are taking, and keep track of courses that they are considering taking. Students can also construct a projected schedule based on their selections.

As you complete the tasks given to you, please vocalize your thought process as much as possible. This is especially important if you become confused or frustrated, because that means you have identified a problem in our design, and telling us what you are thinking will help us fix it.

Remember that you are free to quit the experiment for any reason, at any time.

Thank you for helping us to improve I Can Has HASS!

Tasks

1. Search for HASS classes of the following criteria:
   1. All HASS categories except 4 and Elective.
   2. No final exam
   3. Don't care about CI-H
   4. Not interfering with your other class, which meets 1-4pm on Wednesdays
   5. From departments STS, 21F, and 9
2. Select two classes and mark them as a classes that you are considering.
3. Compare the details of these classes side-by-side
4. Add Mark Zhang as a friend. Add one of his classes to classes you're taking.
5. Decide you no longer want to take one of the classes you were considering.
6. Decide you definitely want to take the other class you were considering.
7. Remove Mark from your list of friends.
8. Print the classes that you have selected.

Observations

Our interface received a lot of positive feedback in user testing. Some particularly noteworthy points were:

• users easily completed the tasks
• users instantly knew how to interact with non-traditional elements like our calendar and the draggable items -> good affordances
• when the search results page with the post-its first came up, several users exclaimed “Oooh!” or “Wow!” --> good user satisfaction and aesthetics

However, we also identified a number of points that our users struggled with:

• users didn’t know what all the department abbreviations stood for
• user tried to click on “I can has HASS” logo to return to Search page
• users did not immediately notice the legend at the top, as a result did not realize what colors meant
• users had trouble finding the “Compare” feature
• users were very confused that “print” link only appeared on results page
• some users had trouble closing the course details bubbles
• users were confused by the fact that when you add a friend’s class, the post-it disappears from the friend’s set of classes until page is reloaded. Asked, does he not have that class anymore?
• users weren’t sure whether friends’ classes were guaranteed to match their search criteria
• users expected results page to remain populated with results after they navigated away and then navigated back
• users were not sure what the difference was between putting a class in the “Trash” and dragging it out of the sidebar

Based on these observations, we have compiled the following table of usability issues that could be improved in the next iteration of design.

Reflection

We've learned a lot over the course of the iterative design process. Being forced to sketch out three user interfaces that were significantly different from each other showed us that there really are multiple approaches to arrange the inputs and outputs to our software. The paper prototypes showed us serious usability problems early on in the process, before we became attached to any code that we might have written. Our first paper prototype had a lot of a problems, which were easily fixed in the second and subsequent versions.

If we had the project to do over, we might explore alternatives to the organization of the post it notes and the post its themselves. We chose to organize them by time of day, which is useful, but as a result, they cannot be sorted any other way. Also, it's hard to display a lot of information about individual classes because all of the info has to fit on a tiny post it note. The small square shape makes it even more difficult to wrap text intelligently and such.

One challenging aspect was the drag and drop for the post it notes. In our paper prototype, it was easy for the user to pick up post it notes and move them around the screen...coding this was much harder, of course. But the interface of dragging and dropping was very intuitive to all of our users, so we decided it was essential for our interface.

We took some design risks in our project. For instance, we designed a calendar interface which allows users to select and deselect blocks of time for when they are busy. The interface is completely unique - we have not seen one like it. It's similar to Google Calendar, but simplified because we don't need all of the functionality that they provide. Our users loved it! Everyone found it extremely intuitive to use and was able to perform the necessary actions to complete our tasks.

Another risk that we took was a compare for the details of multiple classes. We allowed users to open multiple info boxes by clicking on the post its. Unfortunately, it wasn't possible to get the info boxes to organize themselves in a visually useful way, so after the computer prototype we disabled opening multiple info boxes, and instead added a compare feature for classes added to two of the boxes on the sidebar.

For the paper prototype, we decided to do an iterative approach for our two designs, instead of two parallel designs. This benefited us greatly, as we had drastic changes between our two paper prototypes. We changed what content went where, as well as renaming navigation tabs (because our users couldn't find what they were looking for), and the organization of the pages themselves. User feedback was key in our design decisions. We made sure to address all of the issues/features that the users complained about. The computer prototype was useful in seeing what worked and didn't when used on a computer. We got most of our website working in the computer prototype, but we were far from having a usable site at that point - we made a lot of important changes, such as making the right sidebar be offset visually and fixed position, so that when the user scrolls down on the page, the sidebar stays put.

Over the course of this term, our interface has gone through many stages of improvement. It's amazing how an interface can really affect how well users are supposed to examine data. The registrar's course listing has a bad interface for selecting hass classes - even though it has the exact same data as we do. Luckily, with I Can Has HASS, students are saved, and can now figure out what hass classes they want to take!