Requirements (Delete before submitting):

Update your group's wiki page so that it contains a section GR3 Paper Prototyping, containing the following subsections:

• *Prototype photos.*Digital photos of the pieces of your prototype. Show the prototype in interesting states; don't just show a blank window. Although you will iterate your paper prototype during this assignment, the photos only need to show one iteration.
• *Briefing.*The briefing you gave to users.
• *Scenario Tasks.*The tasks you gave to users, as you wrote them on the cards.
• Observations.*Usability problems you discovered from the testing. Describe critical incidents encountered by the users, but *don't record users' names. Record these as a series of high-level takeaways, focusing on the usability problems you saw, rather than what each participant did. For instance, you might describe how you had some learnability issues with your prototype, as evidenced by users B and C clicking all of the menus to try to find option X.
• *Prototype iteration.*You did two rounds of paper prototyping. Describe how your prototype changed between those two rounds.

Briefing

...

Thanks so much for helping us test our website.

...

The task is for you to enter these five ballots. We will give you the tasks as you go, here is your first one.

Scenario Tasks

...

Task 1:

...

Round 1

The users received the following list of tasks as they went along the prototype test:

1. Begin entering votes. Go through one full ballot at a time.

...

2. Restart audit.
3. Begin

Task 2: After 4 ballots have been entered:

• Reader: “Oops, we should have entered 5 ballots, but we’re only at 4. I think we need to start over”. -hands next task to them-
• Task: “Restart audit.

Task 3:

...

1. entering votes. Go through one full ballot at a time.

...

Task 4: Enter 2 ballots correctly. On the second race of ballot #3:

...

2. Correct the

...

1. entry for the previous race to “Obama and

...

Task 5: After 4 ballots have been entered, at any point:

...

1. Biden".
2. Correct the presidential race on the

...

1. previous ballot to “Stein and Honkala”.
2. Finish entering votes.

Round 2

The users received the following list of tasks as they went along the prototype test:

1. Begin entering votes. Go through one full ballot at a time.
2. Correct

...

1. the presidential race on the previous ballot to “Stein and

...

1. Honkala".
2. Continue entering votes.
3. Correct the entry for the previous race to “Obama and Biden”.
4. Restart audit.

Observations

Round 1

For Round 1, we tested our main candidate button interface as well as different error detection. None of the users had difficulty with using the main button entering interface. Entering ballots seemed pretty intuitive and no major critical accidents were recorded with ballots that had names on them. Almost all users were confused by the write-in option. All selected the button correctly but then were confused that they did not have to enter the write-in name. 

All of the users were confused with using the error menu. While most easily found the Fix Mistake menu, one tried to click on the candidate names in the sidebar to change them. Almost all of the users spent a very long time reading the menu, and not a single one pressed the help button for further instructions. Multiple users were also very confused by the difference between race and ballot and one user even chose the wrong option and ended up performing the incorrect task. The second user was also confused with what "previous ballot" referred to in the help menu. The first user was also very surprised that there was no confirmation popup when "Restart Audit" was clicked.

Round 2

In Round 2, we focused our testing on the "Fix Mistake" menu, so our critical incidents were centered on that part of the prototype. 

Users liked the large "Fix Mistake" button and found it easy to enter that menu. After fixing one mistake, users were generally able to use the menu accurately and efficiently. 

Even though the help menu was displayed constantly when the user went to the "Fix Mistake" menu, many users did not read it carefully. There was some confusion about the reset button next to the ballot heading; some users thought they would correct an error and then hit "Reset" to save their changes. Some users clicked the "Reset" button on the whole ballot instead of correcting one individual race. Users were also surprised by the sudden input mechanism on the left. Previously, they had been using the right panel to enter in the ballots. When that panel became informational and the user was expected to click on the left panel, there was some confusion. In addition, the help menu was written vertically and some users read the items sequentially, as if they had to complete every step in order to fix their mistake. 

Prototype Iterations

Most users found the main screen easy and intuitive to use, but had lots of trouble understanding the "Fix Mistake" menu. As a result, we focused on changing that menu in our second iteration. The terminology of auditing is confusing, but our testers never chose the "Help" menu. As a result, we now just display the instructions on the right hand-panel to make sure all information is shown from the start. In addition, instead of lots of buttons requiring understanding of auditing terms, the edit button now shows up beside each ballot, and each race/candidate name becomes a clickable button. Now users can just click on what they need to edit without knowing the name for it. The bottom buttons of the left panel include "cancel" and "restart whole audit" to make these two items distinct. We also added a confirmation page so any choice informs the user what proceeding will do the audit, and gives them a chance to cancel. This improves the system safety.

Paper Prototype Photos

Task 6: After 4 ballots have been entered, at any point:

• “Finish entering votes”

Observations:

Round 2:

User 1:

• Where do you enter write-in? 
• Doesn't see candidate names as buttons to click on
• Restarted previous ballot opposed to just clicking race/candidate name
• Understood the second time that you can click a name and not just reset whole ballot
• Like the fix mistake button.

User 2:

• Can tell that in the main screen, right is for selection and left is for verification
• "You have to reenter all information?" when going back to fix mistake
• Expected to change just one race and return to same location
• Expected to see the panel on the right - "You work on the left and see a review on the right"

...

Images (Round 1)	Caption
	This is the first screen the user was shown - we skipped the log in screens, etc.  This is the main screen users see throughout the project: it is the screen which allows them to enter ballot information.
Image Added	This is the menu that popped up when users selected "Fix Mistake"
	If users chose "Help" on the "Fix Mistake" menu, they would see these instructions. This state was never encountered in user testing.

Images (Round 2)

...

Caption
	The main screen didn't change since we got such good user feedback on it.
	This is what the users see now when the select "Fix Mistake" The left panel now is full of click-able buttons in addition to full ballot "reset" fields. The bottom has "restart whole audit" and "cancel" buttons. The right panel displays instructions how to use this menu.
	If the user chooses a specific candidate name or race, they get a confirmation screen. The screen confirms the element they selected, and also warns that all information from that point will be reentered (by system design).
	There is a similar confirmation page for restarting a ballot.
	This is the confirmation for restarting the full audit. Now users are informed that all information will be reentered.