I’ve been kicking around the idea of redesigning my homepage and blog, though I’m not sure I really have the free time to do it. To start, I thought I would to put down a few thoughts about applying usability principles when designing blogs.

When you starting thinking about usability it’s temping to jump right into lists of principles and rules of thumb. It’s a little silly applying Fitt’s Law when you haven’t even established what you want your site to accomplish in the first place. So what, generally, do you want your blog to do?

Personal Goals

• Share thoughts and work with others
• Collect a body of work to represent myself (like a portfolio)
• Collect information for later discovery (by myself and others)
• Provide an outlet to continue practice writing
• Allow others to communicate with me and comment

If you’re creating or redesigning a blog for a company, the goal set may be very different. Below are some examples that don’t actually apply in my case.

Business goals

• Communicate with customers
• Build long term relationships with customers
• Produce quality content to drive search traffic
• Generate revenue through advertising
• Etc.

Many projects don’t even get this far before the graphic designers and web developers are already making mock-ups, but we still have one more important step to do. We know why you’re building a blog, but why are users coming to it?

This is tougher than you might think. User-centered design requires you to put yourself in other people’s position and suspend your own ego a bit.

User goals

• Get help with a problem
• Read about a topic they’re interested in for entertainment
• Keep in touch with me, see what I’m up to
• Wait until I slip up and reveal internal Google secrets

Just kidding about that last one. For a larger site with a bigger potential audience, you can see how it might be useful to break up users into logical groups – in my example, the number of people who want to keep in touch is much smaller than the number of people with problems.

Some usability experts will even build fictional personas to organize the way they address the need of different user groups.

A final step I like to take is to think a bit about the goals of spammers, hackers, and other abusive users. This has less bearing on visible things like graphic design but is an important consideration for any public-facing site or web app.

Abusive goals

• Post spammy content and links
• Hack the server or blog software and insert malware, links, content, or new pages
• Scrape my content to generate spam

The reason I like to think about this now is that you sometimes need to balance usability against security measures. Users probably don’t enjoy filling out captchas or waiting for their comment to be moderated, but those are probably okay tradeoffs to mitigate comment spam.

From goals to design

Speaking of tradeoffs, you’ll never design the perfect blog and there’s no way you can apply every single usability principle that’s ever been posited. Try to figure out how you can support both your goals and your users’ goals.

For example, I have a goal to share information with others and my potential readers want to find information and do some reading. One commonality here is readability – I should make it as easy as possible to read my blog. This probably sounds pretty obvious, but site redesigns rarely start with typography. It’s a lot more fun to design a logo than think of the best way to present text. Readibility is a complex topic with mixed findings in the literature that deserves it’s own post, coming later.

Another common point – I want to share and collect information so people can find it, and my readers want to get help or read about the topics I tend to write about. This means that I should pay special attention to my blog’s information architecture, navigation system, and search engine friendliness. For example, most blogging systems and CMSs implement a few basic built-in navigational systems, and one that’s very common is the timeline view. You can see it on the right-hand column of my blog now. But when was the last time I used to it look up information? Do my users ever come looking for help designing a WiFi interstitial page and think to themselves, “I should click here where it says January 2009″? Maybe I should nix that section in the redesign.

We’re on our way to designing a successful, usable blog but there’s still plenty of work to do. Stay tuned for more.

In the mean time, I’d like to hear what you did when you designed (or redesigned) your blog. Did you put any emphasis on usability and audience or just come up with an cool idea and implement it?

After a few years of Youtube showing the world how to do video on the web, lots of traditional broadcasters and studios have started putting their content online. Part of the reason is to try to steal YouTube’s thunder – a more market-friendly tactic than just lawsuits. Many of these sites are trying to figure out an advertising model and make money, while others are obviously trying to get viewers more engaged by joining social networks, making mash-ups, etc.

But enough about their goals, what about user goals and experience? In web video the content may be king but usability is almost as important. If your user interface is difficult, confusing, or unpleasant, users will leave your site to get the content elsewhere.

So I’m going to try to do a usability review of various web video sites over the next few weeks. These won’t be formal reviews with user tests and cool eye-tracking heatmaps. Instead I’ll point out some user goals and hold up each site to the same rubrick.

The first site: SouthParkStudios.com

So, what do users want out of web video? I can think of a number of scenarios: finding a particular clip or episode, watching recent episodes, sending a link to a friend or embedding a clip in a blog, and , well, just enjoying the show.

Selection

Score: 4 out of 4 points. This site has everything – every show from every season.

Finding Particular Videos

Method: I’m taking a cue from the creators of Friends – people don’t remember episode names. So I’ll be doing a Google search for the show name and “the one where” and taking the first relevant result. In this case it’s “the one where Ben Affleck has a relationship with Cartman’s Jennifer Lopez hand” (without quotes).

Score: 2 out of 4 points. The search fails, but a simpler query for “Ben Affleck” leads us right to the clips. The full episode is available.

Watching Videos

How easy is it to watch videos? What’s the quality?

Score: 4 out of 4. It’s immediately apparent what to click on to see an episode or clip. You can watch videos full screen and South Park’s animation lends itself well to compressed video. The navigation between episodes is pretty nice, with thumbnails of all episodes for that season along the bottom of the window.

Linking to Videos

Score: 3 out of 4 points. The URL for each clip and episode is available by clicking the “Share” button. Clips open up in the main window so if you can get the link like any other web page. The only lost point is the fact the episodes open in new windows – what is the point? It takes away my browser toolbar and any social bookmaking toolbars or extensions I might normally use.

Embedding videos

Let’s give it a try:

Score: 3 out of 4 points. Once again use the Share button to get the embed code. They lose a point for not allowing embedding of full episodes – they probably have good reasons for not wanting users to do so, but we’re only concerned about the user’s side of things right now.

Advertising

Score: 3 out of 4 points. Ads are shown before the video (for clips) or at two break points about halfway through (for full episodes). Commercials are short and don’t obscure video or interrupt the show more than normal TV commericals would. They lose a point, though, because of the lack of variety – I watched a few episodes and plenty of clips and only saw two different commercials, over and over again.

Audio Experience

I’m going in go with a slightly different scale this time: introducing the patented Bleeding Ear Scale of Web Video Volume.

You may have noticed that some TV stations play their commercials a little louder than the show. The theory I’ve always heard is that they want you catch your attention even if you get up to go to the fridge.

Score:

Unfortunately, most people don’t watch web video the same way they watch TV – they’re usually sitting much, much closer to the speakers or wearing headphones. The bone-shattering difference in volume between the video and the commercials on SouthParkStudios.com earned the site four bleeding ears.

Total score: 19 out of 24 points, with a special note to dive for the volume button whenever an ad is coming up.

Usability and design aren’t just concerns for web developers.  They can make a real impact in the use and usefulness of physical products as well.

Warning labels are a great example – you can’t buy anything these days without some kind of warning label, and they are visual design elements intended to convey important information to buyers and users.  I ran into two great examples in the course of packing up and clearing out our house.

Example number one is from a big plastic storage tub.  It’s a great example of both usability and design, though the actual message might seem a bit silly.  Do people really need to be warned not to seal their children inside airtight containers?

It’s great from a design standpoint because it is clean, puts clear emphasis on the important diagram, and uses bright, attention-grabbing colors.  Any parents poised to place their toddler in the bin will not doubt see the label before recklessly replacing the lid.  It’s a good usability case because it conveys information very clearly and effectively-the silhouette kid is immediately recognizable and it uses common conventions such as the red circle and slash to mean “NO!”

The second example is…  well, strange and off-putting.  We might laugh at the thought of tupperwared toddlers but fireworks obviously pose some danger.  This series of warning messages from the back of a box of fireworks is, well…  take a look for yourself.  I recommend clicking on the image to zoom in in Flickr.

So what could have been improved from a design standpoint?  For one thing, it would help if the coach and the two children in the second panel weren’t wearing what appears to be ghoulish, grimacing deathmasks.   They look like a cross between some misguided ventriloquist’s dummy and the clown that haunts the nightmares of every five-year-old child.  Cartoons caricatures can be very effective in warnings because we can remove unneeded visual detail to focus on what’s important and because people are accustomed to following short narratives in the style of comic strips.  But not if they are so ugly.

What are the usability problems?  Let’s start at the top.  The phrase “Common sense coach reminds you to…” isn’t quite as clear and gripping as “Warning: suffocation risk” in the first example.  The idea of using cartoons to illustrate each point is good, but the actual illustrations miss the point.  Without the text, do you think you could figure out the meaning of each of these?

For more examples of how usability and information design impact the real world, see The Design of Everyday Things by Don Norman.

The Cleveland Plain Dealer ran a story today about the origin of guns used in crimes in the city. This is an issue that people are concerned about and it deserves coverage. Rather than present information about gun laws in various states and numbers of crime gun recovered as a boring list, the PD provided a helpful infographic.

Maps and bar charts can be really useful tools to help people make sense of information. But look closely and you’ll see a problem – the bar chart showing the relative number of crime guns recovered is wrong:

At first glance it looks like about as many guns are recovered from Cleveland as from Cincinatti and Columbus. But the Cleveland number is really about 65% of the Columbus number.

This is probably just an error, akin to misspelling someone’s name in an article. But it’s a good example of a bad graphic, sometimes called chartjunk. Ignoring the error, a bar chart like this might conceal more than it conveys. The top three cities are much larger than the rest, so wouldn’t we expect them to have more guns seized? Maybe a measure per 1000 persons would be better. We also need to think about what this chart implies to readers – is a higher number worse, because it correlates to more crimes, or better, because it means police departments are doing a good job of taking guns off the streets?

If you’re interested in reading more about how to design good graphics and communicate large amounts of data effectively, take a look at the books of Edward Tufte.

Okay, so I’ve had my iPhone for a while now, but back when I got it I took a few notes about my first impressions.  I thought I’d clean them up a bit and post my thoughts for anyone who still on the fence about buying one.

I, like a lot of you, have been following the iPhone since it was just a twinkle in Steve Jobs’ eye.  Hundreds of bloggers and journalists have written about the device.  Now that I’ve had one for two days, does it meet the hype?

Before I write down a few thoughts, I have to say that my wife got me the iPhone as an anniversary present.  My Treo 650 has become increasingly frustrating, freezing up silently and making it impossible to get in touch with me.  I’ve also been informed that this counts as Christmas 2007 as well, which is fine by me.

My first experience with the iPhone was a bit frustrating.  My main desktop is still on Windows 2000.  Unfortunately, even though I had the latest version of iTunes, I needed Windows XP, Vista, or OSX to sync with the iPhone.  I had to activate the phone using my wife’s iBook.  Activation was very quick and painless – as a current AT&T customer who already has a data plan and iTunes account I would imagine I’m the ideal case.

The iPhone does a lot of things very well.  Safari is a great web browser, with one caveat I’ll talk more about below.  The large, high resolution screen makes web surfing a much better experience than my Treo.  The screen is amazingly bright – I have it set at the default, halfway setting and could still read everything easily in the bright sun.  I love the way it picks up nearby wifi networks and then remembers once you’ve okayed a particular one – at home, web surfing is very fast.  Surfing on the AT&T network is noticeably slower but usable.  At least once or twice it seemed to stall completely.

I put a few mp3s and photos on it and the process is pretty painless.  So far iTunes seems a lot easier to use than the Palm Desktop software for my Treo which always seemed a little odd to me.

How does it work as a phone?  Very well.  The speakerphone is loud and clear and everyone on the other end has told me I sound great.  I even took a work call on a Sunday night, and it seemed everyone else on the bridge had background noise problems but me.

The biggest frustration for me so far (other than the incompatibility issues) has been that Safari is so much like a real browser that it tricked me into thinking it was a real browser.  I’ll explain.  I have some photos up on Flickr and my wife was using her iBook so I thought I would just grab photos online instead of syncing them.  No dice.  There’s no way to actually save pictures, or anything for that matter, from the web.  Now I know Safari can save things, that’s how web browsers work, they download and cache files to display them to you.  So why is it impossible to save a photo to my photos?  I wonder if this is Apple trying to make it simpler for novice users or AT&T trying to keep people from skipping services somehow.

Either way it’s disappointing.  It shows you why so many people are rushing to hack the iPhone – there’s a lot of untapped potential there.

I mentioned that iTunes was easy to use, but the syncing process does have one fatal flaw: I can’t seem to figure out how to do a real backup, other than syncing again to a recent version of Outlook.  I really just want a file system I can copy to a CD (or better yet, let Mozy automatically back up).

Anyone else have an iPhone?  Please let me know your thoughts in the comments below.

The New York Times had a great article a few months ago about redesigning the font used on highway signs.  You’ve probably seen the current font, Highway Gothic, a million times without ever thinking about it – it’s been in use for more than 50 years.

Worrying about the font on the signs seems pretty silly compared to all the engineering and resources that go into a single bridge, let alone the entire highway system.  Why does this merit an article in the Times and what does this have to do with programing, web development, social software, or any of the topics many visitors to this blog are interested in?

Programmers and analysts sometimes doubt the value of design.  It’s hard work gathering all the requirements and writing all the code – we don’t have time to worry about how pretty it looks.  A lot of projects we work on, though, involve creating user interfaces, often web applications.  That means that, when it comes down to it, your job is to support the user’s tasks.

Now if someone wanted to add 10% to your hours to pick just the right shade of chartruse, you would be justifiably miffed.  But good interface designers will have good, empirically-tested reasons for their work and the guidelines they live by.  Highway signs are a great example of this kind of empirical benefit:

Intrigued by the early positive results, the researchers took the prototype out onto the test track. Drivers recruited from the nearby town of State College drove around the mock highway. From the back seat, Pietrucha and Garvey recorded at what distance the subjects could read a pair of highway signs, one printed in Highway Gothic and the other in Clearview. Researchers from 3M came up with the text, made-up names like Dorset and Conyer ? words that were easy to read. In nighttime tests, Clearview showed a 16 percent improvement in recognition over Highway Gothic, meaning drivers traveling at 60 miles per hour would have an extra one to two seconds to make a decision.

A one or two second gain in legibility matters a lot when your life depends on it.  Few web applications present the same kind of physical danger, but multiply a small gain over an application with 10,000 users, operating 24 hours a day for a year, and you can see how this can impact the business.  Good design is part of the larger concept of usability.  As anyone who’s done any usability testing can tell you, most applications have many small, easy-to-change pitfalls that can quickly add up to huge wastes of time and effort.

When I was a kid, computers were slow. They had little or no storage, the most reliable connection was sneakernet, and having more than four colors on the screen at the same time was amazing. So the user experience looked a lot like this…

For quite some time now we really haven’t had those same excuses, so why does this video remind you of the point-of-sale system at work, or the database HR is still using, or the software that came with your MP3 player?

I had the chance to catch a presentation by Matt Bailey about web analytics and usability. He made a great point – a lot of the kinds of problems that we look for with usability testing should show up in your web log data too, if you know how to analyze it.

Usability Testing

Usability tests can be seen to fall into two general categories, based on their aim: tests which aim to find usability problems with a specific site, and tests which aim to prove or disprove a hypothesis. This test would fall into the former category. A search of the literature will reveal that tests looking to uncover specific usability problems often use a very small number of participants, coming from Nielsen’s (2000) conclusion that five users is enough to find 85 percent of all usability problems. Nielsen derived this formula from earlier work (Nielsen and Landauer, 1993). Although there is much disagreement (Spool and Schroeder, 2001), this rule of thumb has the advantage of fitting the time and money budget of many projects.

Use of Eye-Tracking Data

In terms of raw data, eye tracking produces an embarrassment of riches. A text export of one test of Mealographer yielded roughly 25 megabytes of data. There are a number of different ways eye tracking data can be interpreted, and the measures can be grouped into measures of search and measures of processing or concentration (Goldberg and Kotval, 1999):

Measures of search:

• Scan path length and duration
• Convex hull area, for example the size of a circle enclosing the scan path
• Spatial density of the scan path.
• Transition matrix, or the number of movements between two areas of interest
• Number of saccades, or sizable eye movements between fixations
• Saccadic amplitude

Measures of processing:

• Number of Fixations
• Fixation duration
• Fixation/saccade ratio

In general, longer, less direct scan paths indicate poor representation (such as bad label text) and confusing layout, and a higher number of fixations and longer fixation duration may indicate that users are having a hard time extracting the information they need (Renshaw, Finlay, Tyfa, and Ward, 2004). Usability studies employing eye tracking data may employ measures that are context-independent such as fixations, fixation durations, total dwell times, and saccadic amplitudes as well as screen position-dependent measures such as dwell time within areas of interest (Goldberg, Stimson, Lewenstein, Scott, and Wichansky, 2002).

Because of the time frame of this investigation, the nature of the study tasks, and the researcher’s inexperience with eye tracking hardware and software, eye tracking data was compiled into “heat maps” based on the number and distribution of fixations. These heat maps are interpreted as a qualitative measure.

Methods

The goal of the study is to evaluate the usability of the web site and identify problem areas that might be improved.

Participants were solicited from a population of Kent State University graduate students in the School of Library Science (SLIS) and the Information Architecture Knowledge Management (IAKM) program via email. Five students volunteered, although one dropped out of the study at the last minute. Participants ranged in age from 25 to 35 with half female, half male. All had more than five years experience using computers and the web.

Participants were invited to the SLIS usability lab where they were informed of the procedure and asked for their consent. Once demographic data was collected, participants were calibrated on the eye tracking system and then asked to complete eight tasks using the web site. Participants were encouraged to “think out loud” while they attempted to complete each task, and their comments were recorded. Interaction with the web site was recorded by screen capture software and an unobtrusive eye-tracking device. Finally, participants were asked to fill out a short survey gaging their opinion of the site and it’s usability. Participants were free to stop at any time, and no incentive was offered for participation other than the opportunity to see the new lab and equipment.

Quantitative measures of usability include task completion, time required for task completion, and number of steps to completion. Participant comments and eye tracking data are used as qualitative measures.

Testing Difficulties

During the course of testing a number of problems arose with the eye tracking software, ClearView. At the conclusion of one of the four tests, the software locked up, making the computer unresponsive. Eventually it crashed, losing all test data for that participant. Consultation with a faculty familiar with the software confirmed that this is an unpredictable, but common bug in ClearView with no known workaround.

In addition, a deficiency was discovered in ClearView’s scanpath and hot spot visualization tools. Both overlay fixation data on top of screen shots of the web page the user was viewing. Unfortunately, the screen shots are not taken as the participant proceeds through each task. Instead the software remembers the URL of each page and then retrieves the screen shot at a later time. Many pages in Mealographer (and virtually any non-trivial web application) look different depending on whether or not a user is logged in, what the user has submitted in a form, and the presence or absence of cookies for session information. Therefore, many of the hot spot maps are laid over a version of the page dissimilar to what the participant had actually seen.

Findings

Task Completion Analysis

Overall, participants were able to complete the tasks 70% of the time. For the purpose of this evaluation, a task is considered complete only when the participant has found and used the expected feature for each task – often times, users were satisfied with their outcome, even though they had not used the most appropriate features. For example, two participants approached Task 1A by searching for a food they already knew was high in protein and low in fat, instead of finding and using the Healthy Food Search form. Table 1 displays the task list and completion rates. The tasks with the lowest completion rate were 1B, 6B, 7 and 8. It is interesting to note that the best-performing tasks include food search, meal entry, and user account functionality, which are the top items seen on the homepage when not logged it. This may be an indication of poor discoverability.

In general, the task completion rate indicates plenty of room for improvement.

Table 1: Task Completion

Table 2: Task Completion vs Performance Measures

Note that for the three participants with recorded times per task and path lengths, completed tasks took less time and involved a shorter path (see Table 2). This is logical, but with only three participants the correlation is not significant.

Individual completion rates, average times per task, and average path lengths are given in Table 3 along with user ratings of the site from the post-test questionnaire. It is interesting to note that user performance on task does not seem to correlate with user ratings (although with so few participants, correlations would not be statistically significant). The user with the highest completion rate gave the site the lowest ease of use score and lowest score overall, while the highest rating was given by a participant in the middle of the field for each performance measure.

Overall, the user ratings of the site were positive, but not outstanding. Although the ratings don’t correlate to the measures, many of the specific usability problems noted later in this report were noticed by the participants themselves as they used the site. Presumably changes to correct these problems would result in higher ratings.

Table 3: Participant Performance and Site Ratings

Table 4 shows the task time and path length for each recorded participant broken down by task. Except for tasks 2 and 8, target path lengths for each task were set to the smallest path that would achieve the goal plus one, and target task times were set somewhat arbitrarily to 2 minutes. Task 2 was much simpler than the other tasks and task 8 was open-ended, allowing participants to enter favorites, usuals, or both. It is difficult to set a task time goal for an interactive web site without data from previous tests. The amount of time a user might want to spend on a particular activity before becoming frustrated or giving up may very based on a number of factors including user motivation, user enjoyment, and web site stickiness. The target of two minutes was chosen because it was thought to be fairly aggressive, meeting the goal of the project to make diet tracking quick and easy.

Table 4: Performance Measures by Participant

Items in Table 4 in bold denote tasks in which a participant had not met the target, and those with gray backgrounds represent successfully completed tasks. Looking at average performance, only tasks 1B and 2 completely met all objectives. It would be reasonable to look for specific usability problems in the tasks for which each participant missed one or more target, and many are addressed in the Specific Usability Problems section below.

Hot Spot Analysis

In Illustrations 5, 6, and 7, hot spot maps of the Mealographer home page are shown for three participants (one, two and four, respectively). These images show the relative number of fixations by color, with green meaning at least one fixation and red meaning three or more, with fixations length set to a maximum of 100ms.

Illustration 5: Hot spot map for participant 1

Illustration 6: Hot spot map for participant 2

Illustration 7: Hot spot map for participant 4

These hot spot maps illustrate some of the observations that can be made with eye tracking that might be missed by other usability testing methods. For example, participant one did not read the light blue help boxes at the bottom of the page whereas two and four did. The participants fixated on many of the red “* required” labels, but only briefly – this may mean that they served their purpose in alerting users without causing confusion or additional concentration. Note the large red areas under the drop-down boxes for “Month” and “Question.” Users had to fixate more on these controls than on many of the text fields.

Illustration 8: Scanpath for participant searching for “coffee”

Illustration 9: Scanpath for expert searching for “tea”

Illustration 10: Scanpath for participant searching for “coffee”

Scanpath Analysis

The scanpaths in Ilustrations 8, 9 and 10 show the result of users entering in a search term on the meal entry form and searching for the proper item to add to their meal. The two on the left represent searches for “coffee” by two participants, novice users of the site. The one on the right is a search for “tea” by the researcher, and can be considered an example of expert use. Note that the paths of both novice users are both much longer than the expert, and that the convex hull area would be larger as well, despite the fact that the desired item is located in a similar position on all three. The novices need to gaze at each item on the list in order, and require more gazes on likely items in order to make a decision. In all three cases, the long, spread out gaze paths indicate that desired items are not near the top of the results, and are not readily apparent. Improvements should be made to the search engine and the results display, perhaps highlighting search terms within each line.

Specific Usability Problems

A list of specific usability problems has been compiled from the participant performance results in Table 4, observations made during the test by the researcher, and participant comments. Problems are organized by task.

Task 1A – Food search

Search results do not always match user expectations. Multiple participants expresses confusion or had to refine their search to get to the food they were looking for. As the tagging system grows, it will help to address this issue.

Search results are hard to scan. Participants seemed lost in all the text in the results. This can be addressed by highlighting the search terms within the results and visually dividing items from each other.

The ordering of nutrients on food pages might not be intuitive. The food pages were designed to match the ordering used on product packages, but some users needed extra time to find what they were looking for. One participant said she expected them to be in alphabetical order.

Task 1B – Healthy food search

Users did not notice the healthy food search function at first. Three users tried using the food search to search for a food they already knew was high in protein and low in fat, at least at first, and another was satisfied with the food he had just searched for by name. The healthy food search could be given more prominent placement, perhaps above the scroll on the search results page.

Task 3 – Meal entry using the quick form on the homepage

One user did not find the meal form on the homepage. Participant 1 used the food search instead, explaining that she had only had one thing to eat. Since the other found the form easily, action might not be needed to correct this.

Results for each food entered in the form do not match user expectations. One participant, for example, entered “rice” but did not find plain rice in the dropdown on the next page. This is another search engine difficulty.

Users could have refined their terms, but did not see a way to do so. In fact they would have had to go back to the homepage and submit the new terms. An entry box could be added to the results page to allow users to refine their terms.

Some users left some fields blank, and did not get useful results. One user left all the “Units” fields blank and the results showed a meal with no nutrition. It took some time for that user to figure out the cause of the problem. Validation hints could be added to the form, and units could default to one instead of blank or zero.

Task 4 – Account creation

Meals entered by visitors could be lost if they do not go directly to create a new account. One user missed the link to create an account from the meal page and went to the homepage first. This is a perfectly valid action. The ID of the temporary meal record in the database should be associated with the user session so visitors do not have to re-enter meals later.

Task 5 – Meal entry

It is not immediately clear how to use the meal entry form. Three users tried to click on the notepad at first, expecting it to work similarly to the quick meal form. Although both did figure out to use the search form to find and then add items, this could be improved. Currently when the form first loads, the search results frame is blank. An arrow graphic and some explanatory text might make use of this form more clear. It might also make sense to have only one meal entry form, mixing the strengths of each.

Users did not always try to modify search terms to improve results. At least one user suggested good modifiers when thinking out loud, but did not try any of them. There is text in the results that suggest trying again, but it could be made more clear.

Food search results were difficult to scan. Participants seemed to take a long time finding the item they wanted on the list (see Illustrations 8, 9, and 10 for scanpath diagrams). One participant suggested adding bullet points to better mark items in the list. Search terms could also be highlighted in the results.

Users are not sure how to enter complex or compound food items. One user tried “salad” and “caesar salad” but compound items with many variable ingredients like salads and sandwiches are left out of the database. Some instructions could be added. This might also be solved by the addition of a food or recipe entry system.

Some users left some fields blank, and did not get useful results. Like the quick meal form, validation hints could be added to the form.

Task 6A – Use of daily report

Some users didn’t think to look for a report system. Three users used pen and paper to compare meal totals instead, although two did later find the daily reports. The existence and functionality of the reports could perhaps be made more clear, and additional links might make the reports more discoverable.

Reports (and many other sections of the site) are not identifiable from browser history.

Participant 2 had a particularly difficult time finding the reports using the browser’s history function. Page titles were very similar, and he tried nine different links before trying another tactic. This is important because it means many pages would not make good bookmarks either. Better, more succinct page titles need to be used on interactive pages.

Reports cannot be accessed when not logged in. In the course of using the browser history, participant 2 accidentally logged himself out without noticing. This made it almost impossible to find the reports, and when he did find the reports they did not have any data. Care was taken to ensure users could not alter other users data, or make entries without logging in, but dynamic pages such as the reports should clearly remind users to log back in rather than displaying zeros. Once that is done links to the reports could be added to the tour page.

Task 6B – Use of weekly report

Users were not immediately aware of the weekly report. The control to switch to the weekly report is located below the scroll on the report page. It could be moved up or a simpler control could be added to the top of the page.

Task 7 – Goal creation

Users did not think to look for goal-setting functionality. Two users reported being happy just using the reports to track progress, though one later did find the goal functionality. Another user overlooked the link to create a goal at least once before finding it. Goals should be better integrated into the report pages.

The list of goals was confusing when no goals were listed. A new user looking at the list before entering sees the column headings but no indication that this is list with zero items. The list should not appear when no goals exist.

Some terms are unfamiliar to users. One participant did not know what “DRI” and “Daily Value” meant when entering a goal. Tooltips or links to a FAQ could be used to help inform users.

Task 8 – Use of usuals and/or favorites

The list of favorites was confusing when no favorites were listed. This is also true for usuals. The list should not appear when no favorites exist.

Favorites and usuals were not discovered by most participants. Three participants either did not think to look for such options or did notice them while using the site. Favorites and usuals need to be better integrated into the rest of the site. It might be helpful to indicate existence of favorites on meal page, even if user has no favorites. Also, an “add this food to my favorites” link could be added to food pages.

References Cited

Gibson, J. J. (1979). The Ecological Approach to Visual Perception. Hillsdale, NJ:Lawrence Erlbaum Associates.

Goldberg, J.H., Kotval, X.P., (1999). Computer interface evaluation using eye movements: methods and constructs. International Journal Of Industrial Ergonomics 24, 631–645.

Goldberg, J.H., Stimson, M.J., Lewenstein, M., Scott, N., Wichansky, A.M. (2002). Eye Tracking in Web Search Tasks: Design Implications. Proceedings of the 2002 symposium on Eye tracking research & applications. New York: ACM Press. pp. 51-58.

Golder, Scott A., Huberman, Bernardo A. (2006). Usage patterns of collaborative tagging systems.

Journal of Information Science, Vol: 32, Issue: 2, April 2006, pp. 198-208

Lindgaard, Gitte, Fernandes, Gary, Dudek, Cathy, and Browñ, J. (2006).

Attention web designers: You have 50 milliseconds to make a good first impression. Behaviour and Information Technology, Volume 25, Number 2, Number 2/March-April 2006, pp. 115-126(12).

Nielsen, Jakob. (2000). Why You Only Need to Test With 5 Users. Alertbox March 19, 2000. Retrieved May 15, 2006, from http://www.useit.com/alertbox/20000319.html

Nielsen, Jakob and Landauer, Thomas K. (1993). A mathematical model of the finding of usability problems. Proceedings of the SIGCHI conference on Human factors in computing systems. April 24-29, 1993, Amsterdam, The Netherlands. pp. 206-213.

Norman, Don. (2004). Affordances and Design. Don Norman’s jnd.org. Retrieved May 15, 2006, from http://www.jnd.org/dn.mss/affordances_and.html

Renshaw, J.A., Finlay, J.E., Tyfa, D., Ward, R.D. (2004). Understanding visual influence in graph design through temporal and spatial eye movement characteristics. Interacting with Computers. Vol: 16, Issue: 3, June, 2004. pp. 557-578.

Spool, J. and Schroeder, W. (2001). Testing Websites : Five Users is Nowhere Near Enough. CHI ’01 extended abstracts on Human factors in computing systems. New York: ACM Press. pp. 285-286.

Stafford, Tom and Webb, Matt. (2005). Mind Hacks: Tips and Tools for Using Your Brain. Sebastopol, CA:O’Reilly Media, Inc.

Palmer, J.W. (2002). Web site usability, design, and performance metrics. Information Systems Research, 13(2), 151-167.

In this study Palmer looks at three different ways to measure web site design, usability and performance. Rather than testing specific sites or trying out specific design elements, this paper looks at the validity of the measurements themselves. Any metrics must exhibit at least construct validity and reliability—meaning that the metrics must measure what they say they measure, and they must continue to do so in other studies. Constructs measured included download delay, navigability, site content, interactivity, and responsiveness (to user questions). The key measures of the user’s success with the web site included frequency of use, user satisfaction, and intent to return. Three different methods were used: a jury; third-party rankings (via Alexa), and a software agent (WebL). The paper examine the results of three studies, one in 1997, on in 1999, and one in 2000, involving corporate web sites. The measures were found to be reliable, meaning jurors could answer a question the same way each time, and valid, in that different jurors and methods agreed on the answers to questions. In addition, the measures were found to be significant predictors of success.

This is an interesting article because in my experience, usability studies are often all over the place, with everything from cognitive psychology and physical ergonomics to studies of server logs to formal usability testing to “top ten usability tips” lists. Some of this can be attributed to the fact that it is a young field, and some of it is due to the different motive fueling research (commercial versus academic). One thing in the article I worry about, however, is any measure of “interactivity” as a whole. Interactivity is not a simple concept to control, and adding more interactivity is not always a good idea. Imagine a user trying to find the menu on a restaurant’s web site—do they want to be personally guided through it via an interactive Flash cartoon of the chef, or do they want to just see the menu? Palmer links interactivity to the theory of media richness, which has a whole body of research behind it that I am no expert on. But I would word my jury questionnaires to reflect a rating of appropriate interactivity.

The most important impact of this study is that it helps put usability studies on a more academically sound footing. It is very important to have evidence that you are measuring what you think you are measuring. It would be interesting to see if other studies have adopted these particular metrics because of the strong statistical evidence in this study.

The most straight-forward metric, download delay, is also one that has been discounted lately. The thought is that with so many users switching to broadband access, download speed is no longer the issue it used to be. This is especially false for sites with information seeking interfaces, which are often very dynamic and rely on database access. No amount of bandwidth will help if your site’s database server is overloaded.