Group Members
- Stephen Suen ssuen
- Kathy Fang kfang
- Yi Wu wuyi
- Connie Huang connieh
- TA: Katrina Panovich
GR1 - User and Task Analysis
GR2 - Designs
GR3 - Paper Prototyping
GR4 - Computer Prototyping
GR5 - Implementation
GR6 - User Testing
GR6 - User Testing
In this group assignment, you will evaluate your interface with a small user test and write a final reflection.
Find at least 3 representative users from your target population. None of your users should be enrolled in 6.813/6.831. All should be willing to participate voluntarily.
Prepare a briefing and tasks. These may be the same ones that you used in paper prototyping, but you may need to improve them based on feedback from the paper prototyping.
You may, if you wish, also prepare a short demo of your interface that you can use to show your users the purpose of the system. The demo should be scripted, so that you do and say the same things for each user. It should use a concrete example task, but the example task should be sufficiently different from the test tasks to avoid bias. The demo option is offered because some interfaces are learned primarily by watching someone else use the interface. Think carefully about whether your interface is in this category before you decide to use a demo, because the demo will cost you information. Once you've demonstrated how to use a feature, you forfeit the chance to observe how the user would have learned to use it alone.
Pilot test your briefing, demo, and tasks, before you test your real users. You can use another member of the class for your pilot testing, if you wish.
Conduct a formative evaluation with each user:
- Provide your briefing and (optionally) demo.
- Then provide the tasks one at a time, observe, and take notes.
One member of your group should be the facilitator of the test, and the rest should be observers. Watch and record critical incidents.
Collect the usability problems found by your user tests into a list. Assign each problem a severity rating (cosmetic, minor, major, catastrophic), and brainstorm possible solutions for the problems.
Design
Describe the final design of your interface. Illustrate with screenshots. Point out important design decisions and discuss the design alternatives that you considered. Particularly, discuss design decisions that were motivated by the three evaluations you did (paper prototyping, heuristic evaluation, and user testing).
Implementation
Describe the internals of your implementation, but keep the discussion on a high level. Discuss important design decisions you made in the implementation. Also discuss how implementation problems may have affected the usability of your interface.
Evaluation
Describe how you conducted your user test. Describe how you found your users and how representative they are of your target user population (but don't identify your users by name). Describe how the users were briefed and what tasks they performed; if you did a demo for them as part of your briefing, justify that decision. List the usability problems you found, and discuss how you might solve them.
Reflection
Discuss what you learned over the course of the iterative design process. If you did it again, what would you do differently? Focus in this part not on the specific design decisions of your project (which you already discussed in the Design section), but instead on the meta-level decisions about your design process: your risk assessments, your decisions about what features to prototype and which prototype techniques to use, and how you evaluated the results of your observations.
The iterative design process taught us the importance of being flexible with our design, prototyping, and testing with users.
Flexible Design
Throughout the prototyping and user testing process, we all realized that we need to continually adapt our design based on feedback from other team members and users. Initially, we each had different design ideas and prototype sketches. After evaluating the pros and cons of each design alternative, we created a paper prototype that incorporated the most user-friendly aspects of our different designs. This process showed us the power of brainstorming as a group and coming up with a collaborative design. Similarly, each round of user testing showed us issues that we had previously not considered. For example, users struggled to use certain features (such as dragging events around on the timeline) and suggested that we build additional features (such as a list view of events). As a result, we changed our design to increase learnability, safety, and efficiency at each iterative step. This required many frequent changes to our design. Hence, we learned that we should adjust our design (making both small and large changes at each iteration), rather than be set on a specific design.
Paper Prototyping
In addition, we learned about the usefulness of paper prototyping. None of us had ever used rounds of prototyping to build an application, so we were initially skeptical of how effective prototyping could be. However, after our first paper prototyping round (with other 6.813 students in Walker Memorial), we were convinced that prototyping could generate many useful comments. Even though our paper prototypes were primitive, in that a group member served as the back-end and some features were roughly sketched out, we still received valuable user feedback on the layout and progression of our screens. We found the most difficult part of paper prototyping to be creating a realistic experience for the user. Many features easily implemented on the computer (such as drag/drop, drop down menus, etc), were difficult to replicate using paper. As a result, we found that paper prototyping was the most useful for helping us make broad design choices, such as choices on the overall setup and layout of key features (i.e, comments). In the paper prototyping stage, we chose to prototype all the layouts of our main pages, including the registration & select events page, timeline, event page with comments, and edit event page. However, we did not fully prototype certain features that required more extensive back-end. These un-prototyped features were (1) adjusting the timeline after the user edited an event, (2) showing dates as the user dragged an event around on the timeline, and (3) populating the timeline based on the user’s selection of which tasks to include.
Computer Prototyping
Computer prototyping, on the other hand, provided useful feedback on both broad features and detailed design issues. We prototyped all of the front-end for our design, but gave users canned responses because we did not prototype the back-end. Specifically, as mentioned in GR4, we allowed users to edit events and add comments, but these edits were not stored in the back-end and thus disappeared after the user refreshed the page. We also did not prototype the back-end of the registration system, so users who created logins could not login with those credentials. Our timeline was pre-populated with events, instead of populated with user selected events. We did not implement much of the back-end for our computer prototype because we wanted to focus on testing the feel of the user interface.
Many of the heuristic evaluations pointed out small design inconsistencies that detracted from the user experience (i.e, font and color). Computer prototype testers also pointed out large design issues (i.e, an overly complex registration system). Computer prototypes were much more costly to implement than paper prototypes, and thus made sense to implement only after paper prototyping refined our design. We found that computer prototyping generated more useful comments than paper prototyping because our computer prototypes had a higher fidelity in feel, leading to a more representative user experience.
User Feedback and Evaluation
As mentioned above, users provided valuable insight into the learnability, efficiency, and safety of our application. Our prototype users helped us modify the layout of certain screens, decide which features to eliminate and add, and found inconsistencies in our design. Users generally understood the main features of our application (timeline, events, comments). They became more familiar with the timeline, the most complex feature, as they interacted with it. Hence, we believe that our user interface is highly learnable. Our users were also able to recover from any mistakes, such as editing an event with the wrong time or selecting the wrong wedding date. We purposely incorporated the safety features into our design in anticipation of such user mistakes. Finally, users were able to complete tasks in a short time period (1-2 minutes), so we believe that our design is efficient, although the efficiency could be improved even further with additional design changes. Overall, we believe that our application has low risk in terms of learnability, safety, or efficiency.
One of the most challenging issues that we encountered was dealing with user feedback regarding the scope of our application. During our briefing, we told users that our application was designed to guide engaged couples through the wedding planning process by helping them visualize the timing of important tasks. Even though we focused on the timeline aspect of wedding planning, some users suggested that we increase the scope of our application. For example, users wanted to ability to link to their Pinterest sites, post pictures, or pick items for their registry. While we agree that having a comprehensive wedding planning site would be highly useful, we did not think we would have enough time to implement such a site. We compromised by keeping the comments feature (enabling users to share ideas with friends/family), but not implementing other features (such as photo boards).
What We Would Do Differently
One of the main things we would do differently is present users with several different initial paper prototypes. As mentioned above, we initially each drew different sketches for paper prototypes. As a group, we decided on a paper prototype, taking elements from our individual sketches. For example, in our individual designs, we represented events using a timeline, to-do list, and calendar. After debating the pros and cons of each design, we decided to use the timeline. However, we think it would have been useful to create several several different paper prototypes to test on users. This way, we could see if users agreed that the best design was the timeline, rather than the to-do list or calendar.
If we had more time, we would implement more features, such as the photo board, suggested by users. This would allow our application to be a more robust wedding planning tool.