Robo-Monitor - User Testing

Design

In our project we created an Android application called Robo-Monitor. Android applications consist of a number of activities, where each activity has its own GUI. The Robo-monitor consists of 3 activities:

Channels overview screen

This is the initial screen which opens when the user opens the application. It shows all channels for which there is data arriving, with some statistics about the data like bandwidth, frequency, and values.

Since there could be many channels, the user can "star" certain channels and hide all the rest. When the user stars a certain component, the text label of the component gets bolded - to increase visibility.

Main screen with no items starred and the control channel collapsed (left). Starring items makes their entry bolder (right).

A menu allows the user to view only starred items. Often many channels have various timestamps and debug information that is helpful to hide. Once selected, the list is much shorter (right).

Graph overview screen

This screen presents a graph widget which allows the user to observe how data has changed over time.

It allows the user to use touch gestures to scroll and zoom. We also provided "Track" functionality so that they can watch the data stream as it appears in real time.

The graph axis have small labels - we received feedback on GR4 that these may be hard to read. We experimented with bigger font sizes, however, they were taking too much space on the graph and were very distracting, so we decided to stick with the small axis labels.

Graph view and view with the menu option. When the user drags the graph, controls appear allowing for a zooming and tracking the current plots. Note that multi-touch zooming is also implemented.

During testing, some users had trouble finding the menu option on this screen, so we are considering moving it or adding another method of access.

Plot with multiple traces. The legend in the upper right displays the channel and variable names. We chose to use a font similar to the axis labels to be as small as possible but still readable.

Graph traces selection

The graph widget can display multiple graphs at the same time.

We provide a utility for the user to select which traces are shown and easily change that

When this screen opens, channels that currently have variables being plotted are automatically expanded. In all tests, users quickly understood the purpose and usability features of this screen.

Implementation

Our implementation uses combination of standard and custom widgets.

• Graph widget: we based our main visualization tool -- the graph widget -- on a open source calculator app called Arity. We added improvements to their widget and fixed several bugs. We plan to submit patches upstream for enhanced muti-touch zoom and infinite scrolling. The graph widget is implemented as a canvas screen which is redrawn periodically depending on a software timer.  The plot itself is drawn with the versatile Android Path object.

• Star widget: the star widgets on the channel screen are implemented with an ImageView and a corresponding boolean variable. Their functionality is almost identical to a checkbox, but it was easier to recreate this functionality that to modify the checkbox object's layout and integrate with expandable lists.

• Expandable lists: To create the main channel views, we use expandable lists.  We have custom list adapters which create each view (or row) in the table.  These list adapters each have their own layout and internal click listeners which allows us full functionality in their look and feel.
  • A drawback of this method is that when the data is updated, each row in the expandable list is redrawn, meaning that the onClick listeners for the stars are recreated. If the user has clicked during this time, that input will be lost.  This factor prevented us from exploring table update rates faster than 1Hz.  We may be able to work around the issue by providing a custom update function.

• Add trace screen: The add trace screen is implemented in a similar manner to the main screen's expandable lists, with different custom list adapters to reflect the need for checkboxes.

Backend

• We run a background service that generates data and stores it in the main Application class (which is shared among all activities.)  We do this for two reasons: (1) we wanted the data must be consistent when the user switches between activities, and (2) we plan to hook real LCM data streams into the application so we wanted an architecture that could support this.

• To show real-time information about the data we have a periodical, background asynchronous thread which "pings" the main GUI thread of the channel overview screen. This way we assure that the GUI will remain responsive even if there is a lot of data to be processed.

Evaluation

We tested our interface with 3 robotics researches from MIT. All of them were possessed Android phones. We think this is important, so that they are familiar with Android's app conventions (such as using the menu button to explore what you can do).

Briefing

Since the users have the background knowledge that channels are dependent on the robot, and since we had fake data we had the following briefing:

"We have built an interface for visualization of LCM streaming data. We support only few types of channel data structures: those, which have only one layer of sub-components which take float values. Try using the app in order to explore the data stream."

We didn't create very specific tasks - instead we wanted to see which parts of the app the users will discover and depending on that we gave them few high-level directions (in case they didn't try it on their own):

• Use the stars.
• Explore a channel.
• Get multiple graphs to show.

User 1:

We estimate this user's initial Android skills as HIGH:

Observations:

The user didn't use stars right away, but rather selected a channel component and started looking at the graph. He commented that the graph seems very intuitive, and liked the zoom out and track ideas. He mentioned that clicking on a component and getting the graph immediately is a very direct way to observe the data and liked that it feels fast. He explored the menu in the graph which allowed him to select traces to be shown in the graph. We didn't observe any difficulty for him when he added traces.

When he returned to the main screen we asked him to use the stars but he wasn't really sure what the stars are for. He commented that it would be nice if there was a way to add traces using the star functionality. He also suggested that one can hold-tap over a channel to see possible options (such as plotting all components, or starring all components, etc.)

Interpretation:

User 2:

We estimate this user's initial Android skills as MEDIUM:

Observations:

This user spent more time than the first one on the channel overview screen. He noticed the channel statistics and the component values and tried opening and closing different channels. He was starting to cycle through his actions and asked us if there is a way to plot some of the data in 3D when he clicked on a channel component (called "x") and that brought him to the graph. He used the buttons to track the graph and to zoom-in / zoom-out. He also moved the graph up and down but didn't discover that it is possible to use multitouch to zoom in and out. He didn't observe that it is possible to add multiple traces to the graph so we asked him to do it. He tried tap-holding the graph thinking that this will do it but this didn't work. He eventually looked at the stars and figured out that you can only show a subset of the channels. He suggested that holding a starred item should draw all starred items.

Interpretations:

User 3

We estimate this user's initial Android skills as HIGH

Observations:

This user looked quickly at the channel overview screen and easily got to see a graph. He was very active, trying a lot of interactions in quickly. He turned the phone on the side to look at the graph. He seemed lost in the graph and then discovered "Oh, there's the track button." We then asked the user to plot several traces at the same time. He first tried to star the traces in the channel overview screen and hold - this didn't work. He then tried holding a channel - this didn't work either. He then went to the graph and tried the menu option which worked. When browsing the graph after that he tried using the "Search" button on the side of the phone, but this didn't do anything.

This user suggested that we allow for plotting quantities against each other, allow for mixed scale Y axis (where 10x change can be applied to certain traces).

We asked the user to try using the stars, and he was able to hide certain channels. He mentioned that this will be worthwhile when there are more channels.

Interpretations:

Reflection