Usability Testing Report (Excerpt)
This is an excerpt from a group project report, listing some findings from a usability test of the Squeezebox, a networked digital music player. The full report is available upon request.
The test was conducted at the Design and Usability Center at Bentley College over the course of three weeks beginning on November 11, 2006. Two groups of 5 participants were recruited based on their ownership of an mp3 player, age and familiarity with the Internet and digital music. The participants completed a set of 9 tasks and gave their feedback in the form of open ended questions and a post-test questionnaire. In addition, the think aloud method was used to obtain participant comments and task times and assists were measured for each task.
A. Global Findings
Participants found the remote control layout to be confusing and difficult to master.
Severity: Level 2
The remote control was not positively received, and many participants cited issues with particular buttons or the remote layout overall. When asked what could be improved with the product, most participants cited the clarity of the functions on the remote. Many found it to be excessively complicated and/or confusingly laid out.
"The remote is just like any other TV remote. For first time use it's just too confusing. It should be very simple just like the simple scrolling they have on the iPod. A few buttons and concentrating more functions in less number of buttons. Interface is just too confusing. Could be made much more simpler." [P1]
"I'm surprised at how much it expects the user to master the remote. (...) The remote, for its size - it's pretty basic." [P9]
Specific feedback about the remote's functionality included:
- button labels did not seem clear
- location of the volume controls (in top area of the remote) is not intuitive
- 'add' button does not behave as expected
- play button location is not intuitive
- form factor of remote does not facilitate use of lower area
"The thumb is so close to the scroll buttons that I tended to use the scroll buttons for what I was doing." [P8]
Participants also noted the absence of functions that they deemed to be central, including an 'enter' or 'select' button, a 'browse' button, and a 'menu' button. Of note here is the fact that some first-level navigation options such as 'search' and 'favorites' offer buttons on the remote, yet another key first-level navigation option - 'browse' - is not accessible via a button on the remote. This may be the main reason most participants used the search method to complete most tasks, even when browse would have been more appropriate.
"I didn't see a browse button" [Pilot]
"I'm trying to figure out the ENTER button." [P7]
Recommendations:
- Re-design the layout of remote to facilitate common tasks and to be consistent with user expectations (see Figure 4)
- Add an 'enter/select' button in the middle of the arrow navigation (to replace the 'play' button)
- Add a 'browse' button to the remote
- The 'add' button should function to add songs to the favorites list, much like adding a bookmark within a web browser
B. Local Findings
Participants found the Brightness/Size controls easy to find and use.
Positive Finding
All participants found these remote buttons to be easy and intuitive to use. Participants easily comprehended that subsequent presses of the buttons cycled the user through various brightness/size settings. One participant (P5) did question whether the inclusion of those two buttons on the remote was necessary, as he did not believe them to be as essential as other functions (i.e. browse or mute). He considered them more as "system settings", something that would most likely be set once during initial system setup.
Participants had great difficulty using the Favorites/Playlists functionality.
Severity: Level 1
All participants found the favorites/playlists functionality to be confusing. In fact, task 6, which asked users to save a song to favorites/playlist, had the longest task times (over 4 minutes for mp3 player owners, and over 7 minutes for non-owners), and was one out of a total of 3 tasks that required moderator assistance. Most participants did not see a clear distinction between the use of the 'favorites' vs. 'playlist' terminology. Participant 4, for example, noted that she would prefer to have just a playlist functionality, and would then create her own 'favorites' playlist.
Most participants attempted to use the 'add' button to add their song to favorites. The fact that there is no 'playlist' button on the remote led them to associate 'add' with 'favorites', while in fact the 'add' functionality is associated with playlists. After using 'add,' participants were uniformly confused when they accessed the favorites and found them to be empty.
"There should be something here to help me."
[hits add on remote] "Adding to end of playlist"...."what playlist"? (P1)
Those that were able to use the menu to add to favorites found this option only through trial and error, and found it difficult to repeat the process a second time. Even when participants completed this task, they were not sure that they had in fact succeeded.
"How did I do that before?" (P2)
Some participants attempted to briefly press the favorites button to add to favorites. Only one participant was able to infer that holding down the 'favorites' button served to add an item to favorites. In addition, participants did not intuit that hitting the 'add' or 'favorites' button a second time deleted the item from favorites. Those who saw a 'deleting from...' message were confused by it.
Recommendations:
- Consider combining the favorites and playlist functionality into one
- Include 'add to favorites' functionality in the top level 'Favorites' menu
- Whatever terminology is chosen, associate the 'favorites' or 'playlist' button on the remote with the 'add' functionality