...
We tested three different options for a file browser with our paper prototypes, and settled on the most traditional. This isn't really surprising since a file browser is an interface we expect all of our user population to be familiar with, so external consistency is a big advantage here. We did sacrifice some potential gains in efficiency to be had in our other two designs, but the learnability tradeoff was not worth it.
Figure 1. SETENTS front page with fewer notes and folders than the length of the page.
If you click on Figure 1, you can see the first page that users see. Folders and notes are displayed, with folders on top. Notes are sorted so that the most recently modified notes are on top. Modification time has no meaning for folders in our system, so folders can only be sorted alphabetically or reverse alphabetically. When sorting Notes by modification time, folders are always displayed alphabetically. Sorting alphabetically or reverse alphabetically results in sorting both folders and notes, although the two categories still remain segregated. Notice that the New Note and New Folder buttons stay close to the bottom of the short list, rather than being stuck to the bottom of the window. This change was in response to critiques received during our heuristic evaluations, saying that it took too long to locate the buttons when they were far away from anything else of interest.
Figure 2. SETENTS file browser with enough files and folders that you have to scroll.
Figure 2 shows a folder containing many more items. In this case the New Note and New Folder button remain stuck to the bottom of the page while items scroll behind them. We would have preferred to put the buttons near the top, where they would be more easily spotted. However, between the search box, the bread crumbs, and the affordances to sort the files and folders by name or date modified, it would have made the top of the page too cluttered.
Figure 3. New folder creation.
...
Any file or folder can be moved into any other folder or up a level by dragging the item and dropping it on top of the destination. The interface for this can be seen in Figure 5. The dragged item has its opacity reduced, and valid targets turn blue underneath it. If the item is dropped somewhere that is not a valid destination, it reverts back into its original position. If a folder is dropped somewhere that already has a folder with the same name, the move is cancelled and the user is alerted.
Figure 6. Document search with autocomplete.
...
In our initial paper prototype for our webcam recording design, we had two separate modes for editing notes and reviewing notes. This was very unpopular with our test users, who wanted the ability to change something in their notes while watching playback of the video. Although this made our time-stamping implementation more complex, we changed our design to have a single Edit and Review interface, allowing users to edit notes while recording video or while playing it back.
Figure 7. The Edit and Review Interface immediately after creating a new file.
...
After pressing stop, instead of the black box / intended live view, a frame from the video that has just been recorded is shown. The standard record icon changes to an Append Recording icon with a plus sign, to hint to the user that it will add more video rather than overwriting existing video. This change was made in response to feedback we got from our heuristic evaluations. Playback controls are shown and enabled for the existing video. This can be seen in Figure 9.
Figure 10. Pressing the "Take Snapshot" button will cause the current frame of video to be inserted at the location of the cursor, at full resolution.
...
To insert an image from the video into the lecture, users can press the "Take Snapshot" button, which will insert the current frame of video (whether being played back or recorded live) into the notes, fully sized. This can be seen in Figure 10. Users can drag the video (oh hover, the icon changes to the "move" cursor with the tool-tip "Drag Me!"). Hovering over the inserted image also reveals the delete/crop buttons, which can be seen in Figure 11.
Figure 12. Crop window allows you to crop out a portion of the image. Crop box is resizeable by dragging the corners, and can be moved. Again the grabby hand cursor disappears from screenshots.
Clicking the crop icon brings up a crop window that lets you crop the image using standard controls. After cropping an image, if you click to crop it again you get the whole frame back, which is good from a safety perspective in case users accidentally crop out data they need. In our paper prototype, we had this crop window popping up immediately after you pressed the take snapshot button, and the image was not inserted until you cropped it. Our paper prototype testers complained about the drag on efficiency that this caused, though. Therefore we revised our design to insert the frame directly, and allow users to come back and crop it after they were done taking crucial notes.
Figure 13. Green highlighting of notes corresponds highlights notes that were taken at the point you're at in the video.
When reviewing notes, green highlights the notes that were taken at the current point in the video, as seen in Figure 13. The highlighting advances as the video plays, or as the user uses the scrubber bar to fast-forward through the video. The user can also click on notes to highlight them and cause the video to jump to the corresponding location. One of our heuristic evaluators found the green highlighting confusing, since we also have normal blue highlighting when users select text for text editing purposes. However, since it is important to distinguish between the two types of highlighting, and since our other heuristic evaluators considered it to be a positive feature, we did not change this from our computer prototype.
Figure 14. Video player can be enlarged.
...
Our first test user is a senior, majoring in course 16.
Task 1: The user create created a new note in the 6.813 folder and titled it "Animation." He did not immediately start recording video: first he tried pressing the disabled play button. He clicked the enlarge video button, and began to take notes.
...
Aside from initial confusion with the play and record button, the user did not make any errors in using the interface. He said that the most irritating part of it was how enlarging the video while taking notes made you have to scroll sideways after inserting a large snapshot, and pointed out that the smaller video size was not useful to see what was going on while taking notes. However, since in a functional implementation the video on screen would match up with what was happening in the real world, enlarging the video while taking notes would probably not be a common case. He also was surprised that the enlargement button did not make the video go full screen.
User 2
User 3
...
Most of the usability problems he encountered had to do with our video resizing behavior. Perhaps eliminating the confusing enlarge button (which is too similar to a full-screen button), in favor of making the Record and Review pane resizeable as a whole so the users can control the size of the video would be better.
User 2
Our second user was a sophomore majoring in course 11.
Task 1: The user created a new note in the 6.813 folder and titled it "Animation". She clicked the play button when trying to record, rather than clicking the record button. She was very confused by the idea that she was supposed to pretend she was in a lecture with the ability to record a video of the lecture, because it looked to her like she was just watching a video. This probably wouldn't be an issue if the user was actually in a lecture with the ability to record the live video feed.
Task 2: The user enlarged the video to take a snapshot, likely in the hopes of getting a larger picture to insert. She kept the enlarged version of the video player for the rest of the duration, but this wouldn't really make sense in a real usage scenario because the lecturer would be visible and the video would not need to be large. She had no problems cropping the image.
Task 3: This task was easy because the user had already created the note in the correct folder. This user also had issues with the "Home" button taking her to the root rather than the parent folder.
Task 4: When trying to find the notes, she initially didn't try to sort the notes by title, but she did eventually discover the search bar. However, searching for "Lecture 19" didn't find anything, because the search only works for note contents. She then discovered the sort function, but the sort order was broken because it sorts "Lecture 1, Lecture 10, Lecture 11, ... , Lecture 2, Lecture 20, ..." Instead of increasing order. When she finally got to the note, she didn't click on the relevant section of text to skip to that part of the video, she instead dragged the scrub bar until the right section was highlighted, which took quite a while because it was hard to be accurate enough.
Overall, the user didn't have serious issues with the core functionality of the interface, it was just supplementary features that were confusing. Most of the issues with the core functionality were learnability issues, so the next time she used the application she would probably better understand how it worked. Resizing was also an issue with this user, so maybe eliminating the enlarge video button and replacing it with a resizable Review pane would be a good idea. Since the user wanted to go back to the parent folder from the note and couldn't do it with the "Home" button, she was worried that clicking the browser back button would lose all of her changes to the note.
User 3
Our final user was a junior majoring in course 6.
Task 1: The user made a new note in the root folder, hoping to be able to move it into the 6.813 folder and was surprised that it went directly to the edit screen. This user had no issue with starting the recording, noting that the disabled "play" button was helpful in letting him know what he could and couldn't do.
Task 2: While this user didn't resize the video player to take a picture, he had much more trouble actually cropping it. Part of the inserted image was hanging off the bottom of the screen, and since our "crop" and "delete" buttons always show up just below the image, they weren't visible so he didn't think there was an option to crop at first. He also got annoyed at the "crop" and "delete" toolbar moving when he scrolled, rather than remaining attached to the bottom part of the image.
Task 3: Since the user didn't create the note in the 6.813 folder to begin with, he had to move it to the correct location. He had no problem dragging it into the correct folder.
Task 4: This user also tried to use the search bar to find "Lecture 19", which didn't work because it only searches based on note contents. Once he found the note, he was unsure of how to find the relevant section of notes. He started watching the video at 2x speed for a few seconds, before realizing that the scrub bar could be dragged. Once he learned (by mistake) that the correct section of video could be skipped to by clicking on the corresponding notes, he was able to navigate the notes and video much more easily.
Again, most of the issues involved seemed to be caused by learnability problems. Once he used the application a few times, the user would likely have no problem accomplishing these tasks.
Usability Problems Summary
- Users clicked the disabled "play" button rather than the "record" button when trying to record new video.
- Category: Learnability
- Severity: Minor
- Proposed Solution:
- The "expand video" button looks like a "fullscreen" button
- Severity: Cosmetic
- Proposed Solution: Replace with a different icon that looks less like fullscreen
- Sizing issues arose when the user made the video bigger. Snapshots in the text editor became too big for the smaller editing window.
- Severity: Major (Visibility suffered greatly, and navigating the editor window became difficult)
- Proposed Solution: Rather than having an "expand video" button, allow the division between the editor pane and the playback pane to be draggable to resize the two areas.
- The "Home" button takes users back to the root of the file tree, rather than to the parent folder like they might expect. To go back to the parent folder, they might use the browser's "back" button but they worry that their changes will be lost.
- Severity: Minor
- Proposed Solution: Provide a back button within the interface (separate from the browser back button) so that the user feels safer going back one level. Since this button would be part of the application, rather than part of the browser, they might not feel like their changes would be lost by using it.
- The search bar in the file browser doesn't search by lecture title, only by note contents.
- Severity: Major
- Proposed Solution: Include the lecture titles in the searchable fields
- The sort option sorts numerical fields in the wrong way.
- Severity: Minor
- Proposed Solution: Correct the sort order for numbers.
- The "Crop" and "Delete" toolbar for cropping images always displays at the bottom of the image, sometimes off the bottom of the page. Also, this toolbar scrolls with the page rather than remaining attached to the bottom of the image.
- Severity: Minor
- Proposed Solution: Fix the positioning of this toolbar, and make it reposition correctly when the window scrolls.
Reflection
Overall, the way that we did our initial designs and paper prototypes had some pros and cons. We came up with three very different ways to bring images into our note-taking program: the webcam solution we implemented, taking photos with a smartphone, and drawing diagrams with a smartphone. We paper prototyped all three, since they were so different, before settling on the webcam option since we liked the extra review features it let us implement. Considering very different alternatives for how to handle our input was useful, but it also meant that we only came up with one design option for the input method we ended up using. Paper prototyping all three alternatives let us be a lot more confident in our decision to narrow in on the webcam alternative, but it also meant we spent less time developing and iterating our paper prototype of the webcam design. In summary, the broadness of our scope in the beginning of our design and paper prototyping process helped us come upon a design idea that we might not have decided on otherwise. However, it left us with less time to tweak and iterate on that idea once we had settled on it. The ideal solution would be to start with a broad scope, but spend more time iterating later; however, time constraints prevented us from doing this. It's difficult to say whether we made the right choice in that regard. However, even with less iteration on our paper prototype of the webcam design, we still got very good feedback from users, and made several major design decisions based on their feedback, detailed in the Design section.
Our computer prototyping focused more on the editing interface than the file browser interface, since the file browser was not very necessary for our tasks and was in our opinion a less interesting interface for us to develop, since we were keeping it fairly externally consistent with similar interfaces. However, our heuristic evaluators had a lot to say about the file browser, and if we had spent more time implementing its behavior fully for the computer prototype we could have gotten more useful feedback from them in that regard. We probably missed out on some critiques they might have made of our final file browser design. However, since the Editing interface is far more important to our project, we don't regret cutting out prototype features from the editing interface to focus on the file browser.
Overall, we're satisfied with the tradeoffs we made when constructing our paper and computer prototypes for users and heuristic evaluators. However, in an ideal world, we would have been able to devote more time to our prototypes: as it is, we're sure we missed out on some useful critiques people weren't able to give us because of incomplete prototype implementations.