Prototype Photos
Touch Interface
Command Line Interface
Briefing
Thank you for volunteering to test our prototype for recording volleyball statistics.
The MIT men's varsity volleyball team is playing against Harvard. Our assistant coaches are on the sidelines recording everything that happens in the game.
SETistics is an application designed for the recording and analysis of information captured during volleyball matches. You are tasked with recording the various statistics that arise during MIT’s match.
Below is a list of terminology that we use during this test. For those interested, each is defined as follows (from Wikipedia):
Spike - the act of scoring a point by slamming the ball over the net into the opposing court effectively and aggressively
Block - when one team makes a move trying to get the ball over the net and the other team reaches up and blocks the ball from coming onto their side
Dig - the act of stopping the ball from hitting your side of the court after the opposing team has either spiked or served it to your team
Serve - A player stands behind the inline and serves the ball, in an attempt to drive it into the opponent's court.
You will be presented with a series of tasks that you will use the SETistics prototype to complete. We would appreciate any and all feedback that you can provide during and after the tasks, so we encourage you to talk as you complete each action.
Thank you again for your participation! Do you have any questions before we get started?
Scenario Tasks
- Record that Player 1, who is standing on the front-right side of the court, spiked the ball to the back-left side of the opposing court for a point.
- If you were to repeat this task to perform it faster, how would you do it?
- Record that Player 99, who is standing on the back-middle side of the court, served the ball to the back-right side of the opposing court.
- Substitute Player 1 for Player 7. (only on the touch interface)
- Modify the stat you just entered to record the following: Player 98, who is standing on the back-middle side of the court, served the ball to the back-right side of the opposing court. (only on the command line interface)
Observations
Touch Interface
- 1 of 3 users did not know where to start
- 1 of 3 users could not find where to enter shot type
- 1 of 3 users could not find where to indicate which player made the current shot
- 1 of 3 users could not understand mapping between gesture and shot type
- 1 of 3 users tried to enter shot direction by tapping on the shot's start and end points in the shot direction view
- 1 of 3 users could not figure out what to do with the drawing view
- 1 of 3 users forgot to enter whether the current shot ended the current point
Command Line Interface
- 1 of 3 users tried to enter the start position of the shot without indicating which team made the shot
- 1 of 3 users was confused by the direction of the arrows for the menus on each of the input fields
- 1 of 3 users tried entering data about the shot by clicking on the space for the picture above the input field
- 1 of 3 users was confused by the unrealistic players numbers used on the prototypes
- 1 of 3 users had trouble efficiently mousing through the player selection menu because it was too thin
- 1 of 3 users was confused by the picture of the spike that came up after indicating that the current shot was a spike
- 1 of 3 users typed 1 instead of w for the point scored input field
- 1 of 3 users thought that se was the shortcut for serve
- 1 of 3 users was confused about the usefulness of the video feed
Note that, for both interfaces, all the issues we found were concerned with learnability, even though we included a test that asked users to perform a task as quickly as they could. This may be due to the fact that the testing environment had less pressure than an actual game and that the user only had to enter a single stat quickly, not many. We plan to gather more information about efficiency and safety issues in the future by performing more tests with actual game footage.
Prototype Iteration
Touch Interface
- We used to have buttons on the drawing pad that could be selected to indicate whether the current shot won or lost the point. We removed these and added them next to the drawing pad with clearer labels and also included a selection to indicate that the current shot did not end the point.
- We had a drop down menu on the drawing pad that was used to show possible gestures, and we replaced it with a help button.
- When a user selected a shot type from the help menu on the drawing pad, the gesture for that shot simply appeared on the pad. We then made it such that a small hand was drawing the gesture on the pad.
- We added labels to all the major sections of the interface, such as Video, Shot Type, Team A, Team B, and Shot Direction.
- We added labels to all the players to make it clearer that the numbers referred to a player.
- Substitutions used to be triggered by a long press on a player number, but now there is a substitution button corresponding to each team that brings up a substitution menu.
- Made it such that the direction of the shot could be indicated by a touch to the endpoints rather than just a swipe.
- The video used to run continuously. It now pauses while the user is entering a shot and continues when the user is done. This allows the user to not miss any action and catch up between points.
Command Line Interface
- We added labels to all the major sections of the interface, such as Video, Player, Shot, Direction, Point Outcome, and Previous Stats.
- Labeled the submit button.
- A small onscreen court is shown to indicate the names of all court positions. Input feedback regarding shot direction is now drawn on this court rather than on an unlabeled court that came up when a direction was input.
- The point outcome now defaults to the point continuing after the current shot.
- The video used to run continuously. It now pauses while the user is entering a shot and continues when the user is done. This allows the user to not miss any action and catch up between points.
- We used to have a single text box for inputting both endpoints of the shot, and now we have two separate, labeled ones.