Prototype Pictures

Paper Prototype	Description
	In the initial prototype, the user was presented with the following dashboard. Tabs on the side provide the user the ability to filter the result set (not implemented in the first paper prototype). The center column provides a list of jobs separated into 3 categories: unassigned, assigned, and recently completed. Jobs have the ability to be clicked ("focused"), or starred. The rightmost column shows the currently selected or focused job. A description of the job, the ability to mark it complete, assign it to a worker, and create updates on the focused job.
	The user is able to star jobs in the center column by clicking on the star to the left of the job title. The UI is simply updated to reflect the starred job.
	Users can assign a focused job to a list of workers. The assignement is done in the right column by clicking the dropdown list of users.
	When a user assigns a job to a particular user, the UI is updated to reflect this by showing the assigned worker on the right side of the assignment widget.
	Notifications are presented to the user in the upper left. The notification widget is updated with a count of the number of notifications. In the paper prototype, the mechanic created an update on a particular job (see the focused job update section), which created a new notification in "real time".
	The address book, opened from the main dashboard, presents the user with an unfiltered alphabetically ordered listing of contacts. The user has the affordance of searching (with autocomplete and auto-filtering), and scrolling. The paper prototype did not support scrolling, however.
	By typing the beginning of a search term, the results are dynamically updated to reflect the best fit results.
	The second version of the paper prototype does not have a job selected by default. This forces a user to select a job before any information is displayed in the right column. The rest of the initial dashboard is unchanged.
	The second version paper prototype gives feedback to the user as to which job is currently focused. An arrow on the right of the focused job as well as an indicator on the left of the focused job is displayed.
	The second version of the paper prototype includes a new way of starring jobs. The updated job information includes the star in the upper right of focused job. This allows users two places to acheive the same functionality of marking a job as "starred".
	The second version of the paper prototype includes the ability to filter results based on the left-most column. When the tab in the left column is selected, the center list is filtered to list only the matching jobs. Jobs may then be selected as previously.

Briefing

Application purpose:

MIT House Managers and mechanics have to coordinate to complete house repairs. Currently:

House Managers get bombarded with emails requesting house repairs.
House Managers then delegate those requests to mechanics via phone or walkie-talkie.
House Managers coordinate repair job progress via phone or walkie-talkie.
All repair job progress is managed in House Managers' heads/makeshift record books.

Our application speeds up this repair process by being a centralized, organized hub for House Managers to identify/prioritize important repair jobs, to delegate said jobs to mechanics, and to then monitor repair job progress.

Background information for user:

For this user test, you are the House Manager (Michael McIntyre) of McCormick Hall. You’ve just arrived to work, and you open up our application to begin working on house repairs.

Scenario Tasks:

Identify the date, time, and location of the most recently completed repair job.
View more information about the McCormick light bulb repair job.
Star the McCormick light bulb repair job.
Assign the light bulb repair job to Jenks Jenkinson, the house mechanic.
View any new notifications and read them.
Look up contact information for Home Depot in order to place an order for light bulbs.
Make a non-urgent note on the light bulb repair job indicating that supplies have been ordered and will be available Wednesday afternoon.

Observations:

Iteration #1:

Users A, D, E, and F were overwhelmed by data and were not sure where to look when they first saw the page.
Users A, B, and F were confused about the relationship between the middle and right panels
Users B and D had trouble marking a job as important. They saw a “Mark complete” button in the right panel and expected there to be a comparable button in that panel for marking importance, but there was not one.
A couple users assumed an update listed in the right side panel was a “notification” (the term “notification” is used in task 5). Users did not seem to be completely aware of the notification alert widget in our interface.
User A tried to assign a job to a worker by clicking the “unassigned” icon on the job listing in the middle panel
User A was confused about why, after she assigned a job to a worker, details about the job still appeared in the right panel
User E was confused why when he selected a worker with the drop-down selector in the right panel the worker wasn’t automatically assigned (our system requires the user to press the “Assign” button to confirm the selection).
User E felt that the search bar and scrollbar in the address book should be more obvious.
User F thought there were too many panels on the screen, and that the left tab panel especially was too much. She said she would have instead prefered a drop-down menu at the top of the screen for filtering.

Iteration #2:

Users G, H, and I had a hard time finding the notification alert widget at the top left because there were too many items on the page and the alert widget is small. Instead, users looked to the unassigned sections.
User I was concerned about the internal/relative ranking of the tasks.
User G expressed interest in having a number appear next to ALL like in Gmail to represent the number of updates/notifications.
User I suggested that the house manager should be able to see all of the tasks assigned to a particular mechanic.

Prototype Iterations:

Change	Iteration #1	Iteration #2
In Iteration #1, users felt bombarded with text/information when they first looked at the application. So when the application is first opened in Iteration #2, no repair job entry in the middle panel is selected and so the right panel (which provides details of a selected job) is empty (with just the text "No job selected" showing in the right panel). This approach not only spares the user from the initial text/information overload, but it also emphasizes the connection between the middle/right panels by conveying to the user that nothing is displayed in the right panel since "No job selected".
In Iteration #1, users did not understand the connection between the middle and right panels. In addition to the aforementioned fix, Iteration #2 features new UI components that better indicate that the content in the right panel are details of the job selected in the middle panel. In particular, we added a vertical rectangle at the left of a selected job entry to indicate which job has been selected and an arrow at the right of a selected job entry to indicate where more information about the job can be found.
Users had difficulties finding how to mark a task as important. Users attempted to find the "Star" widget in the right panel to no avail. We realized this was because there was a "Mark Completed" button in the right panel and so users may have thought a "Mark Important"/star button was also located in the right panel. To fix this, in Iteration #2, we added a "Star" button/widget to the right panel.
Users wanted to use the tabs on the left. During the first round of user testing, users would often click through the tabs in the left panel when searching for a particular job's details. However, we had not implemented this functionality for Iteration #1's Paper Prototype. So for Iteration #2, users were able to search through jobs via clicking through the tabs in the left panel.

Studio Feedback

It seemed like prioritization of tasks was one of your main need that you found in GR1

We decided to scrap notion of relative priority and implement this notion of two tiered ordering – important or unimportant.

How did you identify important/unimportant

This was done through the notion of starring.
There was a lot of trouble figuring out how to "mark important"; this is why there was a change to "starred".

I wish the task actually hadn't been called "starred". If you're making the change to the task you're not actually testing ithe important feature.

Good point.

Were you looking for gmail metaphor or is this how things are playing out?

Good design characteristics are found in gmail and/or asana.
People are finding that the external consistency makes features easier to learn.
Some poeple, however, are looking for certain aspects that we have not implemented, such as notifications appearing in the tab bar as (#) where # is the number of new incoming jobs.

It sounds like the second prototyping went well, maybe a third round can be done before you start implementing your product.

Noted.

Appendix (raw user test observations):

Iteration #1:

User Test A

Didn’t realize the entries in the middle panel were job listings at first.
Unsure about the link between middle panel and right panel.
To assign a job to a mechanic, attempted to click the “Unassigned” icon on the job listing.
Was confused why details about the lightbulb job were still open on the screen after the job was assigned to a work

User Test B

Scanned the right panel furiously to find out how to mark job as important. Really wanted a way to mark job as important in right side panel.
need physical link between job listings (one suggestion was an arrowed panel divider between a job listing and details)

User Test C

UI suggestions

What happens when the screen is very small (w/ 13-inch hardware or mobile)? Does text just get scrunched up or are certain things not displayed?
Wrench logo can be mistaken for ‘Settings’ icon.
Feature suggestions

Questions

Is there any way of tagging things? Is there any way of setting defaults for/identifying recurrent job problems?
How am i going to assign more than one worker to a particular job?
What is going on on the mechanic side of things? How is priority reflected on the mechanic side of things?
Can I aggregate job listings to increase efficiency of doing one operation over multiple jobs?
Thrown off by fact that address book information would be in this app.
Don't know how to add contacts to our address book.
Any way housing manager can make private updates/notes?

User Test D

Maybe put a “mark important” button or star in the right pane as well? (right now it’s only on the central pane with the main list of jobs
Was confused about the difference between notifications and updates (I think this was more just an issue with the prototype and wouldn’t be an issue on the computer)
At first a little overwhelmed with all the text

User Test E

“assign should be automatic, no? why click “assigned”?”
information overload → especially for beginning user
search bar and scrollbar for address book should be more obvious
how to filter the jobs when I type in search bar

User Test F

info overload? any way to only display relevant info?
right side panel not easily linked to left side panel?
change tabs on left side to be drop down menu at the top

Iteration #2

User Test G

tried to drag a job into the right pane when it said “No job selected”
When trying to find notifications, wanted to look in unassigned section
Tried to use scrollbar
Was expecting a number to appear next to ALL like in gmail

User Test H

could not notice the notifications button
confused about the update instruction <-- lack of clarity on the update instruction

User Test I

wanted to look in unassigned area of notifications
thought it would be cool if by clicking on Home Depot you could be redirected to the purchasing website
wanted to ask about long term to for area or label
might be cool if the house manager could see, for a given worker, which jobs are assigned to him