GR6 - User Testing

Design & Implementation

High level design decisions:
- A primary design decision we made was to ground the app around user-contributed images, that is, pictures and thumbnails they take and provide of products that they own. This decision, we found, was effective in that it gave users flexibility as to how they wanted specific items organized, and gave users their own mental model of what the images actually were.
- Simplicity was a key feature of our prototype -- we received positive feedback from all levels of testing that the simplicity of our user interface made it much easier to use. We were initially afraid that the interface would be too simple in that it would effect learnability; however, a reduced layout actually contributed to learnability and efficiency.

High level implementation decisions:
- We chose to implement this software on Android because the picture taking functionality translated best on a mobile application. We utilized the Android SDK, coding primarily in Java. The frontend was handled with Android's own XML view layer. The backend was handled with SQLite databases.
- In general, we followed a Model-View-Controller pattern, where the ContentProvider and SQLite abstracted the database information into data models, the XML layer contained the View, and the bulk of the Java code was the controller.
- One major implementation problem we encountered was simply lack of familiarity with the platform. None of our team members had ever programmed in Android before, and found the learning curve sufficiently steep enough to be problematic.

GUI Section	Design	Implementation
Home Page	Users are first greeted by our home page, which is a simple layout of buttons. Relative button size corresponds with frequency of use -- that is, the larger buttons are more frequently used than the smaller ones. This data was found from user polling and testing.	Many elements of the application, including this particular page, operate solely in the view. This view does not have access to the backend data of the application, allowing for abstracting and safety.
Collection View	We opted to make collection view laid out with images and with names of the item underneath it. Initially we toyed around with text or image only, but we found during our testing that users responded best when both were available. Clicking the menu button brought up a bottom menu. Long-pressing on an item in the collection would bring up a more detailed menu; however, we found this to be among the most difficult menu for users to find (more detailed in evaluation).	Collections are a model of their own, which contain links to other images.
New Item/Item Properties	A simple, non-obtrusive item adding layout was most effective for our purposes. We feared, initially, that users would find it not revealing enough and would have understanding problems; however, users seemed to pick up fairly quickly on how to add items to their collections.	Items were a model that were linked to Tags and Collections. Tags were utilized primarily for searching.
New Collection/Collection Properties	Adding a new collection has a similar unobstructive interface. This also adds for consistency across our application.	Collections also contained extra information such as name, description, thumbnail data, and who shared with.
Sharing Manager	To share a collection, we enter the sharing menu and input the name and editing privileges. This sharing menu was one of our most difficult design decisions, and the current iteration of it is thanks to input from paper and computer prototyping. Our previous design decisions were too complicated, and we found this current one simple and easy for users to use.	Sharing is primarily executed from an external server, which the application contacts to touch base with contacts you share with.
Filtering	Searching for items in our application is as simple as clicking on filter and typing.	Filtering functions on demand -- as the user types, the application quickly sorts through tags and shows the application whose substring matches the text apparent in the filter box.

Evaluation

In general, we found that users found the application easy to use and navigate. Users enjoyed the simplicity of the application and the ability to do the same task through multiple, but natural, avenues.
The users we chose were representative of our population in that we chose users from different age ranges, who also generally found the need to organize large amounts of items.

Usability problems discovered from testing:

MAJOR: Learnability, Efficiency - Users almost all found the task: "Setting a thumbnail the collection thumbnail" difficult to do. The thumbnail was not editable from the Collection Properties page, which was where most users first looked to execute the task. Though long pressing is a normal thing in applications, users did not find long-pressing an obvious feature of our application. There is no feedback that suggests that a long-press does something (in other applications that we looked at, most long-presses cause vibration or something).
MINOR: Efficiency -- Users generally did not find naming an item a necessity, which made the collection view page unaesthetically pleasing because it was cluttered with Untitled1s. We can solve this by making removing the name labels from Collection View.

Procedure for user testing:

Brief users with following brief:

Collector’s Catalogue allows users to view and organize their collections on the go. By tagging and taking pictures of items, users can keep track of and search their large collections through visual recognition, without having to recall solely from memory.

Remember that we are testing the application, not you. Feel free to explore, and think your actions out aloud. If you find yourself confused, or wishing for something extra, let us know, and again, feel free to comment out loud with any observations you make.

This session should be fun, so relax and make yourself comfortable. If at any time you want to stop, you are free to do so. Thank you for taking the time to help us user test the application.

Ask users to preform following tasks:

1)      Create a new collection, and add some items with photos and tags.
2)      Share your new collection with a friend for viewing.
3)      Starting from the Home screen, quickly search for a crab.
4)      Delete the “imaginary” animal item, since it doesn’t belong.
5)      You decide you need some help managing your board games, so you share the collection with a friend, and let them co-manage it with you.
6)      Go back to the collection you created, and set one of your items to be the collection thumbnail.
7)      From the Home screen, make a new board game item and add it to the app.
8)      Finally, you’ve decided that you have too many shoes, so you want to delete your entire shoe collection

User testing notes

User Test

Notes

Debriefing and Conclusions

1

Tasks:

From home, clicked on new collection button. Says he’s bad at phone keyboards.
1. Name: Test collection, Description: This is a test, Tries to push thumbnail image, doesn’t respond
2. “Maybe I can change it later” (gives the appearance of affordance)
3. Public/Private/Public/Private (assumes network will do stuff later like use the public/private), clicks done!
4. Thinks that new item is what he wants, but checks out “edit properties” to see what it does, goes back
5. Item1, takes a picture of shoe, but shoe doesn’t show up (orientation problem? camera is in landscape mode, problem), assumes just lag, and adds some tags (footwear, white). What’s that blue line (hiddennote: it’s the tag scrollbar)? Has a bit of trouble with keyboard text input, adding spaces. “Probably has fixed space for each tag”
6. OH HELLO: discovers that the tag gallery scrolls, and the blue line is a scrollbar.
7. Makes more items. Takes picture of backpack, picture shows up. Thinks maybe he didn’t take a picture of shoe, goes back to take that picture.
8. “Can I rename the item?” click click click, “Probably not”
9. Has two items, makes a third Cereal: food, basic, blue
10. Goes back to shoe: “Picture present, thumbnail obtained” “That’s good”
11. Tries out filter: “Blue” Yay it filters by tags, oh but not by names, goes back to previous collections screen.
Goes to “Edit properties”, makes it public. Am I done? No.
1. Tries to figure out what to do. Filter? Menu? Search? Woops (application exits out, usability problem of non-persistence of memory)
2. (Finds the sharing menu!, in edit properties)
3. (Finds a bug going from Edit Properties, going back to click on an item, bug crashes!)
4. “I’ll avoid this bug then”
5. New Contact! Oh, acces privileges, want View. contact name: meeee
6. “Done. oh woops that’s a different button” (conflicting affordance)
Tries the hardware search. No response, ok. goes to browse collections, searches for Crab. Oh its in emoticons. A crab! V.v.V, I found it, makes sense! Takes a picture, replaces the crab default picture.
Goes to animal collection. Oh i’m still filtering for crab. Clear’s the filter, oh lochness, click. How do I delete? (usability, not clear affordance), searches for a delete option. Tries the menu option, oh cool, but it’s not there. Tries to drag and drop, oh so it’s not that.
1. OH the longtap. longtaps, oh i found it, ok delete the item.
2. oh, the items are renumbered after I delete.
Ok. games. properties. share menu. adds a friend with edit privileges. pushes cancel. oh woops. does again but with done. Can I see it again. Tries longpress (item space is too small for easy longpress!), edit sharer. What happened (pause) oh it’s on the edit bar again I see (not a clear metaphor) Ok, so add is for editing a current sharer as well (not clear change)
1. List of sharers awkward
Takes a picture, longpresses it no response, goes to collectionscreen and longpresses the item. Oh found it, hooray I’m done! How do I get out of a context menu without doing anything? ohno, oh the back button (Should have easier optout context).
1. Oh the name got squished with the picture (consistency usability issue visibility) can’t see item name anymore
Clicks “New Item”. Takes a picture of Risk, tags a few tags. (the + button for adding tags is a little small). Changes collection to Games. Oh wait now the spinner menu is messed up (Animals twice, no games, which its already selected to. bug). Yay you finished!
oh there are a lot of shoes! longpresses the shoe collection, deletes it. Now I have no shoes! (no undo action, what if i want my shoes back, safety problem)

Misc: thumbnail removing seems consistent, when you remove collection selections.
Share menu, the keyboard return item does not respond. “No way to get rid of keyboard? Oh there are ways! swipe, return, etc.”
Debriefing: Desired: Sharing: an easier way to see in the list what their level is. an easier way to do edits.
Long time to figure out how to do long touch
Not sure why the first picture didn’t show up? might have pressed cancel (affordance to say so would be nice)
Can you reorder items? No
Filtering worked as expected! thought it might also filter by name, but ok, might be an option
Adding delete actions to be within items or collections, instead of just as longpress context dependent.
Can’t rename items, would like to.
What is blue bar for? it disappears as a scroll bar, so didn’t understand why it was showing up.

2

Tasks:

Making a new collection, added name and description, checked out sharing manager. asked if private or public mattered. started a new item, took a picture. started a second item with a picture. Didn’t add tags yet. Goes back to add some tags.
Presses the menu button, accesses sharing from the share menu option. adds a sharer and goes back.
searches crab from the filter, finds the crab emoticon successfully
immediately goes and starts longpressing the animal collection, then going back, and finding lochness and longpressing it, finds the delete option and deletes it.
Adds victoria to the list of sharing manager, used the menu button from the bottom
Tries to click the picture from the item screen, no response. tries the edit properties screen. tries to longpress the thumbnail, no response. “I have no idea.” difficult task. Finally, longpresses the thumbnail item from the collectionview, and sets the thumbnail!
Clicked on “New Item” but didnt make an item there. Instead goes to browse collections, and makes an item directly from the games collection.
Longpresses the shoe collection, and deletes it.

Debriefing: I think it’s pretty simple to use. Does the filter search by tag and text? I don’t think it’s bad to use, I got stuck on trying to set thumbnails for the collection, but other than that I think it’s pretty intuitive! There’s multiple ways of doing a lot of actions, I just noticed I can make a new item from the home screen, but it makes more sense to go from a collection to item, than directly to an item.
The recent items button doesn’t need to be so big? I think it’s easy to use, it could be a little bigger and prettier.

3

Tasks:

New Collection -> Title - “hello world”, Description - “test collection”, done.
New Item -> Title - “room”, Tag - “room”
Take Photo -> Takes a picture of the room
New Item -> Title - “desk”, Tag - “desk”, “wood”
Take Photo -> Takes a picture of the desk
Clicks Item -> “oops” -> back. Clicks Edit -> Share -> Adds “Thomas” to list, clicks on Thomas and presses Notify with Email.
First -> Browse -> Animals -> “huh...? no crab...” -> back -> Emoticons -> Crab... “Haha”
“Ooooh...” Goes back to Browse screen -> Types in crab in the filter, and clicks on emoticons collection, then on the v.V.v.
Browse -> Animals -> Click and hold on Imaginary -> Delete
Browse -> Games -> Edit -> Share -> Types in Thomas, changes setting to can Edit -> Add
Browse -> hello world -> Click and hold on the desk -> Set as Thumbnail
Browse -> Games -> New Item -> “Ohhh....”
Take 2 from Home Screen: New Item
Browse -> Click and Hold on Shoes -> Delete

When picture is being taken, the Cancel/Retake/Done menu is landscape as opposed to portrait.
When tagging, when you press “+”, and the text box dialog pops up, it would be convenient if the keyboard automatically showed up. Alternatively, there could be suggested tags that you could drag’n’drop into the tag part to add tags.
It would be convenient if there were a quick way to get to the Home Screen instead of pressign back a lot.
No difficulties!
The user liked that there were multiple ways to do each task - it makes it very convenient!

Reflection

In general, we found this class highly rewarding to learning about user-centered software design. Though the Spiral Model is frequently preached in other software development classes, only in this class is it truly emphasized. Cheap iterations played a large role in honing out developed UI ideas, preventing us from spending the effort to implement a feature that was not well-received.
In the future, one thing I would find particularly useful is another round of paper prototyping, especially evaluating design decisions we were not totally sure about. In user testing stage, we should be more persistent about clarifying what exactly was difficult about a particular task, if any difficulty is found.
One very core thing we learned as a team was the discrepancy between the mind of the user and the mind of the designer. Things that were immediately clear to us was not necessarily the case for our users. In this respect, user testing was invaluable. Even our hardest efforts to think from the user's standpoint did not reveal nearly as much as a paper prototype in front of an user.
One thing we would change in the future is to draft more realistic paper prototypes. Templating tools have readily available Android UI toolkits, which would easily give us a more realistic paper prototype. This may trade the perceived cheapness of the prototype, but we believe that a prototype that visually looks more similar to an actual app has more perceived affordance than a hand-drawn one.