- \n
- Scan path length and duration<\/li>\n
- Convex hull area, for example the size of a circle enclosing the scan path<\/li>\n
- Spatial density of the scan path.<\/li>\n
- Transition \tmatrix, or the number of movements between two areas of interest<\/li>\n
- Number of saccades, or sizable eye movements between fixations<\/li>\n
- Saccadic amplitude<\/li>\n<\/ul>\n
Measures of processing:<\/h3>\n
- \n
- Number of Fixations<\/li>\n
- Fixation duration<\/li>\n
- Fixation\/saccade ratio<\/li>\n<\/ul>\n
\tIn general, longer, less direct scan paths indicate poor representation (such as bad label text) and confusing layout, and a higher number of fixations and longer fixation duration may indicate that users are having a hard time extracting the information they need (Renshaw, Finlay, Tyfa, and Ward, 2004). Usability studies employing eye tracking data may employ measures that are context-independent such as fixations, fixation durations, total dwell times, and saccadic amplitudes as well as screen position-dependent measures such as dwell time within areas of interest (Goldberg, Stimson, Lewenstein, Scott, and Wichansky, 2002).<\/p>\n
\tBecause of the time frame of this investigation, the nature of the study tasks, and the researcher’s inexperience with eye tracking hardware and software, eye tracking data was compiled into \u201cheat maps\u201d based on the number and distribution of fixations. These heat maps are interpreted as a qualitative measure.<\/p>\n
<\/p>\n
Methods<\/h2>\n
\tThe goal of the study is to evaluate the usability of the web site and identify problem areas that might be improved.<\/p>\n
\tParticipants were solicited from a population of Kent State University graduate students in the School of Library Science (SLIS) and the Information Architecture Knowledge Management (IAKM) program via email. Five students volunteered, although one dropped out of the study at the last minute. Participants ranged in age from 25 to 35 with half female, half male. All had more than five years experience using computers and the web.<\/p>\n
\tParticipants were invited to the SLIS usability lab where they were informed of the procedure and asked for their consent. Once demographic data was collected, participants were calibrated on the eye tracking system and then asked to complete eight tasks using the web site. Participants were encouraged to \u201cthink out loud\u201d while they attempted to complete each task, and their comments were recorded. Interaction with the web site was recorded by screen capture software and an unobtrusive eye-tracking device. Finally, participants were asked to fill out a short survey gaging their opinion of the site and it’s usability. Participants were free to stop at any time, and no incentive was offered for participation other than the opportunity to see the new lab and equipment.<\/p>\n
\tQuantitative measures of usability include task completion, time required for task completion, and number of steps to completion. Participant comments and eye tracking data are used as qualitative measures.<\/p>\n
Testing Difficulties<\/h3>\n
\tDuring the course of testing a number of problems arose with the eye tracking software, ClearView. At the conclusion of one of the four tests, the software locked up, making the computer unresponsive. Eventually it crashed, losing all test data for that participant. Consultation with a faculty familiar with the software confirmed that this is an unpredictable, but common bug in ClearView with no known workaround.<\/p>\n
\tIn addition, a deficiency was discovered in ClearView’s scanpath and hot spot visualization tools. Both overlay fixation data on top of screen shots of the web page the user was viewing. Unfortunately, the screen shots are not taken as the participant proceeds through each task. Instead the software remembers the URL of each page and then retrieves the screen shot at a later time. Many pages in Mealographer (and virtually any non-trivial web application) look different depending on whether or not a user is logged in, what the user has submitted in a form, and the presence or absence of cookies for session information. Therefore, many of the hot spot maps are laid over a version of the page dissimilar to what the participant had actually seen.<\/p>\n
Findings<\/h2>\n
Task Completion Analysis<\/h3>\n
\tOverall, participants were able to complete the tasks 70% of the time. For the purpose of this evaluation, a task is considered complete only when the participant has found and used the expected feature for each task \u2013 often times, users were satisfied with their outcome, even though they had not used the most appropriate features. For example, two participants approached Task 1A by searching for a food they already knew was high in protein and low in fat, instead of finding and using the Healthy Food Search form. Table 1 displays the task list and completion rates. The tasks with the lowest completion rate were 1B, 6B, 7 and 8. It is interesting to note that the best-performing tasks include food search, meal entry, and user account functionality, which are the top items seen on the homepage when not logged it. This may be an indication of poor discoverability.<\/p>\n
\tIn general, the task completion rate indicates plenty of room for improvement.<\/p>\n
\n\n
Task<\/strong><\/td>\n Completion Rate<\/strong><\/td>\n 1. A.<\/strong> Think of a food that you like. Use this website to find out how much protein is in that food.<\/td>\n 4\/4<\/td>\n<\/tr>\n \n 1. B. <\/strong>Try to find a food that is high and protein but low in fat.<\/td>\n 1\/4<\/strong><\/td>\n<\/tr>\n \n 2.<\/strong> Please find your way back to the Mealographer homepage without using the browser’s Back button.<\/td>\n 4\/4<\/td>\n<\/tr>\n \n 3. <\/strong>Use this website to figure out how much fat and fiber you had at lunch today.<\/td>\n 3\/4<\/td>\n<\/tr>\n \n 4.<\/strong> Sign up for an account.<\/td>\n 4\/4<\/td>\n<\/tr>\n \n 5.<\/strong> Now that you have an account, please enter in what you had for breakfast today. How many calories did you have at breakfast?<\/td>\n 4\/4<\/td>\n<\/tr>\n \n 6. A.<\/strong> Did you have more Calories at breakfast or lunch today? Find a way to figure this out.<\/td>\n 3\/4<\/td>\n<\/tr>\n \n 6. B.<\/strong> Let’s say that you wanted to see a summary of your nutrition each day this week. Try to find a report that shows a week’s worth of information and look up your calcium intake.<\/td>\n 2\/4<\/strong><\/td>\n<\/tr>\n \n 7.<\/strong> Let’s say you wanted to try to have more than 25 grams of fiber each day. What’s the best way to make sure that you are with this web site?<\/td>\n 2\/4<\/strong><\/td>\n<\/tr>\n \n 8.<\/strong> If you were going to use this web site often, what would you do to make entering meals easier? If you can think of something, go ahead and do it.<\/td>\n 1\/4<\/strong><\/td>\n<\/tr>\n \n Total<\/strong><\/td>\n 70%<\/td>\n<\/tr>\n<\/table>\n Table 1: Task Completion<\/em><\/p>\n
\n\n
\n <\/p>\n<\/td>\n
\n Completed Tasks<\/strong><\/p>\n<\/td>\n
\n Incomplete Tasks<\/strong><\/p>\n<\/td>\n
Number (recorded)<\/strong><\/td>\n 19<\/strong><\/td>\n 11<\/strong><\/td>\n<\/tr>\n \n Average Time<\/strong><\/td>\n 1:56<\/td>\n 2:37<\/td>\n<\/tr>\n \n Average Path Length<\/strong><\/td>\n 3.79<\/td>\n 6.27<\/td>\n<\/tr>\n<\/table>\n Table 2: Task Completion vs Performance Measures<\/em><\/p>\n
<\/p>\n
\tNote that for the three participants with recorded times per task and path lengths, completed tasks took less time and involved a shorter path (see Table 2). This is logical, but with only three participants the correlation is not significant.<\/p>\n
\tIndividual completion rates, average times per task, and average path lengths are given in Table 3 along with user ratings of the site from the post-test questionnaire. It is interesting to note that user performance on task does not seem to correlate with user ratings (although with so few participants, correlations would not be statistically significant). The user with the highest completion rate gave the site the lowest ease of use score and lowest score overall, while the highest rating was given by a participant in the middle of the field for each performance measure.<\/p>\n
\tOverall, the user ratings of the site were positive, but not outstanding. Although the ratings don’t correlate to the measures, many of the specific usability problems noted later in this report were noticed by the participants themselves as they used the site. Presumably changes to correct these problems would result in higher ratings.<\/p>\n
\n\n
Participant<\/strong><\/td>\n Completion Rate<\/strong><\/td>\n Avg Time per Task<\/strong><\/td>\n Avg Path Length<\/strong><\/td>\n Organization Rating <\/strong><\/td>\n Ease of Use Rating<\/strong><\/td>\n Design Rating<\/strong><\/td>\n<\/tr>\n \n 1<\/strong><\/td>\n 4\/10<\/strong><\/td>\n 1:33<\/strong><\/td>\n 2.9<\/strong><\/td>\n 4<\/strong><\/td>\n 4<\/strong><\/td>\n 3<\/strong><\/td>\n<\/tr>\n \n 2<\/strong><\/td>\n 6\/10<\/td>\n 3:35<\/td>\n 8.0<\/td>\n 3<\/td>\n 3<\/td>\n 4<\/td>\n<\/tr>\n \n 3<\/strong><\/td>\n 10\/10<\/td>\n –<\/td>\n <\/td>\n 3<\/td>\n 3<\/td>\n 3<\/td>\n<\/tr>\n \n 4<\/strong><\/td>\n 8\/10<\/td>\n 1:36<\/td>\n 3.2<\/td>\n 4<\/td>\n 5<\/td>\n 4<\/td>\n<\/tr>\n \n Total<\/strong><\/td>\n 70%<\/td>\n 2:15<\/td>\n 3.7<\/td>\n 3.5<\/td>\n 3.75<\/td>\n 3.5<\/td>\n<\/tr>\n<\/table>\n Table 3: Participant Performance and Site Ratings<\/em><\/p>\n
<\/p>\n
Table 4 shows the task time and path length for each recorded participant broken down by task. Except for tasks 2 and 8, target path lengths for each task were set to the smallest path that would achieve the goal plus one, and target task times were set somewhat arbitrarily to 2 minutes. Task 2 was much simpler than the other tasks and task 8 was open-ended, allowing participants to enter favorites, usuals, or both. It is difficult to set a task time goal for an interactive web site without data from previous tests. The amount of time a user might want to spend on a particular activity before becoming frustrated or giving up may very based on a number of factors including user motivation, user enjoyment, and web site stickiness. The target of two minutes was chosen because it was thought to be fairly aggressive, meeting the goal of the project to make diet tracking quick and easy.<\/p>\n
<\/p>\n
![\"Hot](\"http:\/\/farm3.static.flickr.com\/2224\/1954450693_597cad62d9.jpg\")
<\/a><\/p>\n![\"Hot](\"http:\/\/farm3.static.flickr.com\/2253\/1955277596_43843c6f82.jpg\")
<\/a><\/p>\n![\"Hot](\"http:\/\/farm3.static.flickr.com\/2199\/1954452273_5838626294.jpg\")
<\/a><\/p>\n![\"Scanpath](\"http:\/\/farm3.static.flickr.com\/2273\/1954452661_dcdec11d09.jpg\")
<\/a><\/p>\n![\"Scanpath](\"http:\/\/farm3.static.flickr.com\/2193\/1954453501_4d41a22895.jpg\")
<\/a><\/p>\n![\"Scanpath](\"http:\/\/farm3.static.flickr.com\/2352\/1955279230_646fc53514.jpg\")
<\/a><\/p>\n