...
- *Prototype photos.*Digital photos of the pieces of your prototype. Show the prototype in interesting states; don't just show a blank window. Although you will iterate your paper prototype during this assignment, the photos only need to show one iteration.
- *Briefing.*The briefing you gave to users.
- *Scenario Tasks.*The tasks you gave to users, as you wrote them on the cards.
- Observations.*Usability problems you discovered from the testing. Describe critical incidents encountered by the users, but *don't record users' names. Record these as a series of high-level takeaways, focusing on the usability problems you saw, rather than what each participant did. For instance, you might describe how you had some learnability issues with your prototype, as evidenced by users B and C clicking all of the menus to try to find option X.
- *Prototype iteration.*You did two rounds of paper prototyping. Describe how your prototype changed between those two rounds.
Briefing
...
Thanks so much for helping us test our website.
...
The task is for you to enter these five ballots. We will give you the tasks as you go, here is your first one.
Scenario Tasks
...
Round 1
...
Task 1:
- Task: “Begin entering votes. Go through one full ballot at a time.”
...
Task 6: After 4 ballots have been entered, at any point:
- “Finish entering votes”
Round 2
...
(We changed the wording slightly to make the usability test process faster)
INSERT TASKS HERE!
Observations
...
Round 1
...
User 1:
- Asked if they were buttons
- No problem finding buttons
- Assumes race means same ballot -> strange because different levels of seriousness (could accidentally hit wrong help button)
- Knew restart previous ballot because it was in previous ballot summary
- Interesting - why fix mistake over back?
- Safety for restart all auditing? "Are you sure" button?
- Perhaps keep party lines for top/bottom over frequency
...
- Are these buttons or drag and drop?
- What happens if you make a mistake? -> jumped ahead in tasks
- Successfully fixed a mistake without hesitation
- "Prototype is very thorough"
- Restarted previous race without hesitation
- "Race" -> last race or previous ballot?
- Do you need to record who the write in was?
- "Does the progress bar measure races in a ballot or across all ballots?"
Round 2
...
User 1:
- Where do you enter write-in?
- Doesn't see candidate names as buttons to click on
- Restarted previous ballot opposed to just clicking race/candidate name
- Understood the second time that you can click a name and not just reset whole ballot
- Like the fix mistake button.
...
- weird to click on the left panel after getting used to just using that as feedback
- confused about reset and clicking on a race
- wants to see animation of current moving down to previous
- wants heading to be at top, buttons closer together
- didn't read help menu carefully
- read help menu sequentially, as if things were in order
- wants to fix mistakes two ballots ago
- really doesn't understand what the reset ballot button does
Prototype Iterations
...
Most users found the main screen easy and intuitive to use, but had lots of trouble understanding the "Fix Mistake" menu. As a result, we focused on changing that menu in our second iteration. The terminology of auditing is confusing, but our testers never chose the "Help" menu. As a result, we now just display the instructions on the right hand-panel to make sure all information is shown from the start. In addition, instead of lots of buttons requiring understanding of auditing terms, the edit button now shows up beside each ballot, and each race/candidate name becomes a clickable button. Now users can just click on what they need to edit without knowing the name for it. The bottom buttons of the left panel include "cancel" and "restart whole audit" to make these two items distinct. We also added a confirmation page so any choice informs the user what proceeding will do the audit, and gives them a chance to cancel. This improves the system safety.
Paper Prototype
...
Photos
Images (Round 1) | Caption |
|---|---|
| This is the first screen the user was shown - we skipped the log in screens, etc. |
| This is the menu that popped up when users selected "Fix Mistake" |
| If users chose "Help" on the "Fix Mistake" menu, they would see these instructions. This state was never encountered in user testing. |
Images (Round 2) | |
|---|---|
| The main screen didn't change since we got such good user feedback on it. |
| This is what the users see now when the select "Fix Mistake" |
| If the user chooses a specific candidate name or race, they get a confirmation screen. The screen confirms the element they selected, and also warns that all information from that point will be reentered (by system design). |
| There is a similar confirmation page for restarting a ballot. |
| This is the confirmation for restarting the full audit. Now users are informed that all information will be reentered. |
