...
- Sophia Teacher: A teacher for middle school students with special needs in a low-income community in Philadelphia (sister of group member)
- Tricia Teacher: An elementary school teacher at a school in Barcelona (aunt of group member)
- Emily Teacher: An educational coordinator at the Stata Childcare Center (visited in-person)
Prior to user testing we introduced our users to our problem statement, told them they would be testing our website as teachers - which is their profession - and provided users with a website briefing (below). We then presented them user tasks (below) to complete, one at a time. We encouraged them to talk out loud and explain what they were doing & provide feedback during testing, which we took notes of. This helped us identify usability problems with our interface and suggest potential solutions in an evaluation (below).
...
Panel |
---|
You are the fourth grade elementary teacher, Tammy. As a teacher, you have many things to do before the weekend. Please, complete the following tasks: 1. Greg, Elisa, and Barbara have a group project on ancient empires. You found the following webpage link: http://en.wikipedia.org/wiki/List_of_largest_empires and you thought it might be useful to share this link with (the parents of) Greg, Elisa, and Barbara. Your task it to share this website with their group. 2. Sarah and several other students recently failed your math midterm. As a usually high performing student, Sarah's mom, Brenda is concerned and messaged your about Sarah's performance. You want to schedule a study session for the students who have C's in the class and respond to Brenda's message. Your task is to: 3. You've finished preparing grades for all but a couple of students. You still want to send out grades but also want to let the parents who receive no reports that their children's grades will come soon. Your task is to: |
Evaluation:
Heuristic | Severity Rating | Problem | Solution | |||||||
---|---|---|---|---|---|---|---|---|---|---|
Learnability & Efficiency | Minor | In our Messages tab, users are required to select a conversation before writing a new message. Although a grey banner with "Select a conversation to the left" is displayed to inform users of this requirement, during testing, two of our users tried to click inside the message text area box to begin writing their message before they realized they first had to choose a conversation. One user commented that this threw her off a bit, and she felt like this may derail some teachers attempting to get their tasks done very quickly. | Although we attempted to provide instructions by placing a simple grey banner inside the message content pane, this was not sufficient for the users we observed in testing. Rather than requiring users to select a conversation themselves, a solution would be to have the top conversation (which contains all of the users that are selected on the left contact menu) automatically selected would indicate to users that a conversation must be selected to send a message. Because selecting a conversation selection causes it to be highlighted, this would give the affordance to users that they have to highlight a different conversation to select it. A negative effect of this solution may be decreased safety in sending messages, since users do not have to intentionally pick their conversation. | |||||||
Safety & Learnability | Major | Pressing enter in the text area for messages causes the message to be sent rather than creates a new line. While trying to complete a task, one user accidentally pasted the link she was instructed to send to the parents of several students and then pressed the enter key, which sent her message. She commented that she was trying to include some context with the link and the behavior of the enter key in our UI was not externally consistent with most other message UI's (e.g.: Gmail). | A solution to this problem would be to make the text area for messages more externally consistent by making the enter key create a new line rather than send the message. Although the original intention of making the enter key send the message automatically was to make messaging more efficient, we realized that some users who are familiar with other messaging UI's would also make these slips which decreases safety. | |||||||
Learnability | Major | The new message icon that is displayed on a user's row in the contact selection menu gives the affordance that clicking on it will direct the user to that new message. Two of our users tried to click on the message icon itself within a different context (i.e.: calendar, grades) to see the message. They said it was confusing that the message icon immediately went away after clicking on the corresponding row but the conversation was not selected, leaving users bewildered about what happened to the message. | The obvious solution to this problem would be to cause the new message icon to automatically direct users to the message context (despite what context they are currently in) and select the corresponding conversation, which would display the new message that the icon corresponded to. This would prevent users from getting confused over whether or not they missed or lost a message, and it would also make reading new messages much simpler and safer. | |||||||
Learnability | Minor | Conversations in the Message tab are displayed in groups according to the contacts that are included in that conversation. All conversations that a selected contact has been involved in (individually or in different groups) display in the conversation tab when that contact is selected. One user commented that displaying all of these conversations at once is being confusing. When more than a couple users are selected, the label of a conversation says the first couple of student names, then "+ X more" (with X being the additional number of selected students). All conversations also contain the icons of all the students selected. We received feedback that these icons are too small to identify who is in the conversation, and names would be less helpful than subject lines or some other label. | One solution would be to allow users to include a subject for each new conversation that is created. This would be helpful for more efficient idenfication of which group of students displayed in the conversation pane corresponds to which task. However, one negative aspect of this solution would be to cause additional clutter in the conversations pane by allowing for multiple subjects for the same group of contacts, which would remove a limit from how many conversations would belong to the same group or individual contact selected. A better solution might be to allow for group creation, giving users a way to give names to groups of students that a user needed to frequenty communicate about (e.g.: Greg, Steve, and Sarah belong to Reading Group A). | |||||||
Efficiency | Minor | One user commented that she worked alongside other teachers in one classroom, as it was her job to work with a specific subset of students at once, and not all of them. Another user provided feedback that although she teaches an entire class, she often organizes lesson plans around different groupings of students according to their level and behavior. These users suggested that having a way to create and select groups of students under labels or titles would be more efficient than having to select all of the users in a group every time they need to be contacted, which could introduce more errors (e.g.: including the wrong student, leaving a student out). | As mentioned in the previous solution, a way to solve this problem would be to allow for group creation, giving users a way to give names to groups of students that a user needed to frequenty communicate about (e.g.: Greg, Steve, and Sarah belong to Reading Group A). | |||||||
Learnability Efficiency | Minor | Tried to send grade report without selecting SMS or Email (not internally consistent with messages, need to fix) Teachers time is very pressed, its good to have a simple layout and not have too much customization (SMS or Email, for example should already be checked), needs to be quick and simple |
| Major | All of our users commented that the way conversations are displayed, labeled, and scaled (height of a conversation's box) was not intuitive and was hard to figure out. Conversations are sorted by people, which users commented was useful after we explained it to them, but all of them said they were expected it to sort by recency, which would be externally consistent with other messaging UI's. | To make the way messages are sorted more externally consistent, we could simply order the conversations (beneath the dynamic one that displays all currently selected users at the top) in descending order by the most recent message that was sent/received. This would not remove the diplay of conversations by people involved, but it would order them in a way that was more relevant and intuitive. | Learnability | Major | the way conversations are displayed, labeled, scale them to how many messages have been sent, and sort them in a more intuitive way (would have expected to see most recent message listed first, not by people) Confusing to have the most recent conversations that person was involved in displaying , useful but not intuitive, make it more clear with a title or something |
|
Reflection
As our group completed our development of CheckMark, we had to make significant choices that changed the outcome of our project. Over the course of the iterative process, we learned getting through each iteration cycle as quickly as possible allowed us to move quickly and incorporate the maximum number of important changes into our design. By continually and constantly challenging our design, we improved it in look and functionality as well as developed a thorough understanding of why every piece of our design existed and was placed where it is. Although we are extremely happy with our finished product, in hindsight, we may have done a few things that made our lives a little bit tougher when it came to the implementation. Below are a few sections on how we made decisions with regard to aspects of this project, and what we would've done differently.
...