User Testing
Design
Based on the response to the three design ideas that we tested on the first day of paper prototyping, (smartphone photo notes, smartphone diagram notes, and webcam video notes), we decided to implement webcam video notes with a fairly standard file browser. We did not implement separate user accounts.
File Browser Interface
We tested three different options for a file browser with our paper prototypes, and settled on the most traditional. This isn't really surprising since a file browser is an interface we expect all of our user population to be familiar with, so external consistency is a big advantage here. We did sacrifice some potential gains in efficiency to be had in our other two designs, but the learnability tradeoff was not worth it.
Figure 1. SETENTS front page with fewer notes and folders than the length of the page.
If you click on Figure 1, you can see the first page that users see. Folders and notes are displayed, with folders on top. Notes are sorted so that the most recently modified notes are on top. Modification time has no meaning for folders in our system, so folders can only be sorted alphabetically or reverse alphabetically. When sorting Notes by modification time, folders are always displayed alphabetically. Sorting alphabetically or reverse alphabetically results in sorting both folders and notes, although the two categories still remain segregated. Notice that the New Note and New Folder buttons stay close to the bottom of the short list, rather than being stuck to the bottom of the window. This change was in response to critiques received during our heuristic evaluations, saying that it took too long to locate the buttons when they were far away from anything else of interest.
Figure 2. SETENTS file browser with enough files and folders that you have to scroll.
Figure 2 shows a folder containing many more items. In this case the New Note and New Folder button remain stuck to the bottom of the page while items scroll behind them. We would have preferred to put the buttons near the top, where they would be more easily spotted. However, between the search box, the bread crumbs, and the affordances to sort the files and folders by name or date modified, it would have made the top of the page too cluttered.
Figure 3. New folder creation.
Clicking the New Folder button results in a folder being added to the top of the folder list, with and input with "New Folder" selected for pending delete. Pressing enter of blurring the input box results in the new folder sorting into place with a brief highlight animation so the user's eye is drawn to the new location. If the user attempts to name the folder with a name that already exists, an alert pops up and the new folder disappears. Clicking on the New Note button goes straight to the editing interface for the new note, which is shown later in Figure 7.
Figure 4. Hovering over an item reveals the delete and rename buttons.
In our computer prototype, we did not implement or mock up the ability to delete, rename, or move items within the file browser. While we recognized that these abilities are very important from the perspective of flexibility and safety, we did not think they were the most interesting or unique parts of our interface and left them unimplemented. However, our heuristic evaluators uniformly pointed this out as a major issue. One of our evaluators also brought up the point that right-click menus, which we had been considering, can be confusing in a web-app since users expect the right-click menu to contain only browser functionality, not app functionality. For this reason, we settled on hover affordances. Figure 4 shows our delete and rename affordances, which are revealed when you hover over an item. Clicking on the delete button brings up a confirmation dialogue. Clicking on rename brings up the rename interface, which is the same as the new folder naming interface shown in Figure 3 where the folder's current name is replaced with an input, with the current name selected for pending delete. Clicking anywhere else on the gray highlighted area results in opening the note or folder in question. Taking the screenshots unfortunately got rid of the pointer, but hovering over most of the item has a pointing finger. Hovering over the note or folder icon results in a grabby hand, and you can in fact click and drag anywhere in the highlighted area and drop on any folder or the "Up A Level" section to move folders or notes up or down one level in our file hierarchy. This behavior is shown in Figure 5.
Figure 5. Moving a folder into a different folder.
Any file or folder can be moved into any other folder or up a level by dragging the item and dropping it on top of the destination. The interface for this can be seen in Figure 5. The dragged item has its opacity reduced, and valid targets turn blue underneath it. If the item is dropped somewhere that is not a valid destination, it reverts back into its original position. If a folder is dropped somewhere that already has a folder with the same name, the move is cancelled and the user is alerted.
Figure 6. Document search with autocomplete.
The last feature of our file browser, seen in Figure 6, is the ability to search through all of the notes. We use the jquery UI autocomplete to populate the search's drop-down menu with matches, showing both the relative path of the Notes title and a snippet of text surrounding the first match. Clicking on one of the results opens the notes.
Edit and Review Interface
The Edit and Review interface is the main interface of our application. In this window, you can type notes in our rich text editor while recording (or not recording) video. If you type notes while recording video, the software keeps track of what time your notes were created. Then when watching playback of the video, you can see what notes you were typing at that moment in time. Users also have the ability to insert snapshots from the live recording or from already recorded video into the body of their notes, and crop, resize, or move the snapshots.
In our initial paper prototype for our webcam recording design, we had two separate modes for editing notes and reviewing notes. This was very unpopular with our test users, who wanted the ability to change something in their notes while watching playback of the video. Although this made our time-stamping implementation more complex, we changed our design to have a single Edit and Review interface, allowing users to edit notes while recording video or while playing it back.
Figure 7. The Edit and Review Interface immediately after creating a new file.
Figure 7 shows the interface a user sees immediately after clicking the New Note button in the file browser. The default title of "Untitled Note" is highlighted for pending delete, and pressing enter or tab will give focus to the rich text editor. Since no videos are associated with a new file, the video playback controls are disabled or hidden, and a standard recording button is shown. In our planned design, a live view from the web cam would have been shown in the black box in Figure 7, and taking snapshots would have been allowed even without the recording of video. Initially, we had four different playback/recording control buttons: play, pause, record, and stop buttons. These buttons were enabled or disabled based on the application state. The stop button also served a dual purpose of stopping a recording, if the system was recording, or stopping the video and going back to the beginning, if a recorded video was playing. Our heuristic evaluators pointed out that this was inconsistent with modern web video interfaces, which have a combined play/pause button and do not have a stop button to go back to the beginning. We carried over the combined play-pause button idea to our combined record/stop button, and removed the second functionality from the stop button.
Figure 8. Recording video brings down a recording overlay showing how long recording has lasted.
Since we were unable to implement video recording, pressing record starts playing a pre-recorded video that you have to pretend is currently being recorded. An overlay indicates how long the recording has been going on, seen in Figure 8.
Figure 9. After video has been recorded, the controls switch to allow playback of the existing video and appending of new recordings.
After pressing stop, instead of the black box / intended live view, a frame from the video that has just been recorded is shown. The standard record icon changes to an Append Recording icon with a plus sign, to hint to the user that it will add more video rather than overwriting existing video. This change was made in response to feedback we got from our heuristic evaluations. Playback controls are shown and enabled for the existing video. This can be seen in Figure 9.
Figure 10. Pressing the "Take Snapshot" button will cause the current frame of video to be inserted at the location of the cursor, at full resolution.
Figure 11. Inserted image hover affordances, allowing move, delete, and crop.
To insert an image from the video into the lecture, users can press the "Take Snapshot" button, which will insert the current frame of video (whether being played back or recorded live) into the notes, fully sized. This can be seen in Figure 10. Users can drag the video (oh hover, the icon changes to the "move" cursor with the tool-tip "Drag Me!"). Hovering over the inserted image also reveals the delete/crop buttons, which can be seen in Figure 11.
Figure 12. Crop window allows you to crop out a portion of the image. Crop box is resizeable by dragging the corners, and can be moved. Again the grabby hand cursor disappears from screenshots.
Clicking the crop icon brings up a crop window that lets you crop the image using standard controls. After cropping an image, if you click to crop it again you get the whole frame back, which is good from a safety perspective in case users accidentally crop out data they need. In our paper prototype, we had this crop window popping up immediately after you pressed the take snapshot button, and the image was not inserted until you cropped it. Our paper prototype testers complained about the drag on efficiency that this caused, though. Therefore we revised our design to insert the frame directly, and allow users to come back and crop it after they were done taking crucial notes.
Figure 13. Green highlighting of notes corresponds highlights notes that were taken at the point you're at in the video.
When reviewing notes, green highlights the notes that were taken at the current point in the video, as seen in Figure 13. The highlighting advances as the video plays, or as the user uses the scrubber bar to fast-forward through the video. The user can also click on notes to highlight them and cause the video to jump to the corresponding location. One of our heuristic evaluators found the green highlighting confusing, since we also have normal blue highlighting when users select text for text editing purposes. However, since it is important to distinguish between the two types of highlighting, and since our other heuristic evaluators considered it to be a positive feature, we did not change this from our computer prototype.
Figure 14. Video player can be enlarged.
Users can enlarge the video by clicking on the enlargement button to the right of the playback controls. This feature was confusing, since the button we used is more commonly associated with making a video full-screen. This was pointed out by our heuristic evaluators and our GR6 test users. However, HTML5 video does not support full-screen viewing, and we were unable to find or invent on icon that conveyed making a video larger but not full-screen. The video can be enlarged for ease of viewing while in review mode, or in recording mode. Our test users tended to enlarge the video in recording mode, since we did not implement live video recording and had users imagine that the video corresponded to what they saw in front of them. However, we expect that this usage would be less common in a fully implemented version of SETENTS.
Implementation
After deciding on the webcam-based design, we had to choose which platform we were going to use for our implementation. We all preferred a web app in terms of creating a user interface, but capturing webcam output from a web app proved challenging. We hoped to be able to use the HTML5 getUserMedia API to capture webcam output. However, this is a very new technology -- only in the latest version of Chrome, and only if you enable experimental features. We eventually realized that chrome's implementation didn't have all of the features we needed yet: most importantly the ability to save video, but also the ability to differentiate between webcams plugged in to a laptop to access a front-facing webcam pointing at a lecturer. In the end, we were not able to use getUserMedia to implement live webcam capture for SETENTS, and had to resort to using a pre-recorded video and pretending that the playing video corresponded to a camera live view. This is the implementation limitation that has the single largest impact on the usability of our project, since it can be somewhat confusing having to pretend that a pre-recorded video corresponds to what you're seeing during a user test. And of course, it means that the killer feature of our app is not useful in the real world.
We used a Python flask backend with a sqlite server to power SETENTS. The way we structured our database allowed for notes having the same title, but did not allow folders to share a title: internal consistency suffers some as a result.
Evaluation
Since our target population is college students, test users were recruited from among our friends (who had not yet seen the project). Our users were briefed using the same briefing we used for the paper prototype of our design, which follows:
"You’re a student at MIT, and you were annoyed with taking notes on paper, so you’re trying out an application called SETENTS.
SETENTS primarily advertises the ability to record video with your webcam that’s synced up with the notes as you type them. It also should let you take snapshots from the video to insert into your notes.
As a good student, it’s very important that you take good notes - including all diagrams that the professor draws on the board - and then take time to review those notes later (before the test)."
Our tasks were modified somewhat from our paper prototype tasks. The test users were asked to perform the following tasks:
1. "Create new notes for your 6.813 lecture, "Animation." Take notes on the lecture and make sure you have video to review later.
2. There's an interesting figure on the lecture slides. Insert it into your notes and make sure the figure itself is prominent.
3. The lecture is over. Make sure your notes are organized so you can find them later.
4. Review 6.813 Lecture 19 for the test. Make sure your original notes didn't miss anything about the topic of "Message Files."
We did not present our users with a demo.
User 1
Our first test user is a senior, majoring in course 16.
Task 1: The user create a new note in the 6.813 folder and titled it "Animation." He did not immediately start recording video: first he tried pressing the disabled play button. He clicked the enlarge video button, and began to take notes.
Task 2: The user took a snapshot and cropped it. He had trouble locating the cursor after inserting the image but before cropping it, since the enlarged video screen made the text area smaller than the width of the inserted image.
Task 3: Since the user had created his notes while already in the 6.813 folder, he did not feel any additional organization was necessary. He expressed annoyance that clicking on the "Home" button in the editing interface took him back to the root of the file browser instead of the 6.813 folder.
Task 4: The user went back into the 6.813 folder and opened Lecture 19. He played the video, and after watching for a few seconds clicked on the part of the notes headed "Message Files" to jump to it.
Aside from initial confusion with the play and record button, the user did not make any errors in using the interface. He said that the most irritating part of it was how enlarging the video while taking notes made you have to scroll sideways after inserting a large snapshot, and pointed out that the smaller video size was not useful to see what was going on while taking notes. However, since in a functional implementation the video on screen would match up with what was happening in the real world, enlarging the video while taking notes would probably not be a common case. He also was surprised that the enlargement button did not make the video go full screen.