Requirements (Delete before submitting):

Update your group's wiki page so that it contains a section GR3 Paper Prototyping, containing the following subsections:

• *Prototype photos.*Digital photos of the pieces of your prototype. Show the prototype in interesting states; don't just show a blank window. Although you will iterate your paper prototype during this assignment, the photos only need to show one iteration.
• *Briefing.*The briefing you gave to users.
• *Scenario Tasks.The tasks you gave to users, as you wrote them on the cards. *<-- SO I ONLY INCLUDED WHAT THE CARDS SAID
• Observations.*Usability problems you discovered from the testing. Describe critical incidents encountered by the users, but *don't record users' names. Record these as a series of high-level takeaways, focusing on the usability problems you saw, rather than what each participant did. For instance, you might describe how you had some learnability issues with your prototype, as evidenced by users B and C clicking all of the menus to try to find option X.
• *Prototype iteration.*You did two rounds of paper prototyping. Describe how your prototype changed between those two rounds.

Briefing

Thanks so much for helping us test our website.

This website is designed to improve the efficiency of the election auditing process. This is the process by which ballots are manually recounted to verify election results. Note that most election auditors are retired and elderly.

The way that ballot auditing happens is as follows:

• Ballot auditors work in pairs of people - the reader and the recorder. You will be the recorder
• One of us will serve as your reader. That person takes each ballot and reads out the results to you, the recorder.
• The recorder enters this result into the system. This process continues until all ballots have been entered. In your case, the total number of ballots is five.

The task is for you to enter these five ballots. We will give you the tasks as you go, here is your first one.

Scenario Tasks

Round 1

The users received the following list of tasks as they went along the prototype test:

1. Begin entering votes. Go through one full ballot at a time.
2. Restart audit.
3. Begin entering votes. Go through one full ballot at a time.
4. Correct the entry for the previous race to “Obama and Biden".
5. Correct the presidential race on the previous ballot to “Stein and Honkala”.
6. Finish entering votes.

Round 2

The users received the following list of tasks as they went along the prototype test:

1. Begin entering votes. Go through one full ballot at a time.
2. Correct the presidential race on the previous ballot to “Stein and Honkala".
3. Continue entering votes.
4. Correct the entry for the previous race to “Obama and Biden”.
5. Restart audit.

Observations

Round 1

User 1:

• Asked if they were buttons
• No problem finding buttons
• Assumes race means same ballot -> strange because different levels of seriousness (could accidentally hit wrong help button)
• Knew restart previous ballot because it was in previous ballot summary
• Interesting - why fix mistake over back?
• Safety for restart all auditing? "Are you sure" button?
• Perhaps keep party lines for top/bottom over frequency

User 2:

• Wants candidates in a list to easily select
• "I don't actually say who the write-in was?"
• Hesitation restarting all audits
• Likes that the buttons are big and far apart
• Confused about what "previous" means
• Wants to be able to click left side of the screen to correct

User 3:

• []

• "Don't you have to write who the write-in was?"
• Lots of time reading the menu
• Lots of hesitation before selecting an option (bu didn't click 'Help')
• Chose "restart previous ballot" instead of "restart previous race" -> was confused about wording
• "Fix previous race" instead of "restart previous ballot"
• Then got "fix this ballot"
• Really likes the giant screen, easy to use and less chance for mistakes
• Why previous ballot summary? Could get confusing
• Confused between "race" and "ballot" in the menu
• What do you do with more candidates?
• Tradeoff between big screen and changing screens frequently, but good call
• How does this look on the screen?

User 4:

• Are these buttons or drag and drop?
• What happens if you make a mistake? -> jumped ahead in tasks
• Successfully fixed a mistake without hesitation
• "Prototype is very thorough"
• Restarted previous race without hesitation
• "Race" -> last race or previous ballot?
• Do you need to record who the write in was?
• "Does the progress bar measure races in a ballot or across all ballots?"

Round 2

In Round 2, we focused our testing on the "Fix Mistake" menu, so our critical incidents were centered on that part of the prototype. 

Users liked the large "Fix Mistake" button and found it easy to enter that menu. After fixing one mistake, users were generally able to use the menu accurately and efficiently. 

Even though the help menu was displayed constantly when the user went to the "Fix Mistake" menu, many users did not read it carefully. There was some confusion about the reset button next to the ballot heading; some users thought they would correct an error and then hit "Reset" to save their changes. Some users clicked the "Reset" button on the whole ballot instead of correcting one individual race. Users were also surprised by the sudden input mechanism on the left. Previously, they had been using the right panel to enter in the ballots. When that panel became informational and the user was expected to click on the left panel, there was some confusion. In addition, the help menu was written vertically and some users read the items sequentially, as if they had to complete every step in order to fix their mistake. 

Prototype Iterations

Most users found the main screen easy and intuitive to use, but had lots of trouble understanding the "Fix Mistake" menu. As a result, we focused on changing that menu in our second iteration. The terminology of auditing is confusing, but our testers never chose the "Help" menu. As a result, we now just display the instructions on the right hand-panel to make sure all information is shown from the start. In addition, instead of lots of buttons requiring understanding of auditing terms, the edit button now shows up beside each ballot, and each race/candidate name becomes a clickable button. Now users can just click on what they need to edit without knowing the name for it. The bottom buttons of the left panel include "cancel" and "restart whole audit" to make these two items distinct. We also added a confirmation page so any choice informs the user what proceeding will do the audit, and gives them a chance to cancel. This improves the system safety.

Paper Prototype Photos