...
All of the users were confused with using the error menu. While most easily found the Fix Mistake menu, one tried to click on the candidate names in the sidebar to change them. Almost all of the users spent a very long time reading the menu, and not a single one pressed the help button for further instructions. Multiple users were also very confused by the difference between race and ballot and one user even chose the wrong option and ended up performing the incorrect task. The second user was also confused with what "previous ballot" referred to in the help menu. The first user was also very surprised that there was no confirmation popup when "Restart Audit" was clicked.
Round 2
In Round 2, we focused our testing on the "Fix Mistake" menu, so our critical incidents were centered on that part of the prototype.
...
Most users found the main screen easy and intuitive to use, but had lots of trouble understanding the "Fix Mistake" menu. As a result, we focused on changing that menu in our second iteration. The terminology of auditing is confusing, but our testers never chose the "Help" menu. As a result, we now just display the instructions on the right hand-panel to make sure all information is shown from the start. In addition, instead of lots of buttons requiring understanding of auditing terms, the edit button now shows up beside each ballot, and each race/candidate name becomes a clickable button. Now users can just click on what they need to edit without knowing the name for it. The bottom buttons of the left panel include "cancel" and "restart whole audit" to make these two items distinct. We also added a confirmation page so any choice informs the user what proceeding will do the audit, and gives them a chance to cancel. This improves the system safety.
Paper Prototype Photos
Images (Round 1) | Caption |
|---|---|
| This is the first screen the user was shown - we skipped the log in screens, etc. |
| This is the menu that popped up when users selected "Fix Mistake" |
| If users chose "Help" on the "Fix Mistake" menu, they would see these instructions. This state was never encountered in user testing. |
Images (Round 2) | |
|---|---|
| The main screen didn't change since we got such good user feedback on it. |
| This is what the users see now when the select "Fix Mistake" |
| If the user chooses a specific candidate name or race, they get a confirmation screen. The screen confirms the element they selected, and also warns that all information from that point will be reentered (by system design). |
| There is a similar confirmation page for restarting a ballot. |
| This is the confirmation for restarting the full audit. Now users are informed that all information will be reentered. |
