JasonMorrison.net

I do a bit of amateur photography.  I’m not very strong technically and I don’t have particularly good equipment, but I enjoy finding interesting angles and compositions.  I’ve been putting up photos on Flickr for a while to share them with friends and the public.  I also have an account on Panoramio with some photos that show up in Google Earth.

No matter the particular photo site used, sharing photos online has been a great experience.  I’ve had a number of encouraging comments on my photos and people have emailed me to ask if they could use a photo in a report for school or a pamphlet for their non-profit.

When I signed up with Flickr I noticed they had options to add Creative Commons licenses to photos by default.  I’m more than happy to let people use my photos for noncommercial purposes, so why didn’t turn on Creative Commons licensing from the start?

Part of it was the number of options available.  Creative Commons licensing allows other people to share your work but it’s not the same thing as releasing the copyright or putting photos in the public domain.  You have some options:  do you want people to be able to make money off your work, or do you just want it available for non-profits, educational, and personal use?  Do you want people to be able to alter and remix your work or just present it as-is?

So I was a bit struck by the paradox of choice and decided to skip ahead and start uploading photos.  In retrospect, that was a mistake.

There’s a great page at the Creative Commons site that explains the options.  I am going to license my photos with an Attribution Non-commercial (by-nc) license.  That license covers my default attitude about my amateur photography – everyone is welcome to use my photos for non-commercial purposes, so long as they give me credit. This is, of course, in addition to fair use rights that people already have.

Another important point:  it doesn’t mean people can’t use it commercially, they just have to contact me and get permission.  Depending on the use, I might put a price on it.  And I can always sell prints or make products myself.

I might even switch over to allow commercial use as well, if I can get over my delusions of being the next Ansel Adams.

The abuse and incessant extension of copyright might not seem like a life-or-death issue, but it’s one of those issues where technology and public policy are inextricably linked.  It’s like the problem of software and business method patents.  There’s a great story by Spider Robinson that illustrates what happens if taken to extremes.

So take a look at the licenses and consider applying the appropriate copyleft to your work.

  The issue has been our for a little while now, but I thought I would note that I have an article about The use of tagging systems in this month’s issue of the ASIST Bulletin. Take a look at Why Are They Tagging, and Why Do We Want
Them To?

Almost everyone has a tagging system the web is facing serious weather with tag clouds on every site. I think it’s interesting to explore the uses of folksonomies and why users bother tagging things in the first place. Here’s an excerpt:

When thinking about adding tagging to a site, the first question should be: What do we want to get out of this? Does the site need something to improve search results or a new navigational facet to better connect related pages? Is the goal to classify lots of multimedia objects with minimal cost or to get users to interact with the site a little more?

To complete my MS in Information Architecture and Knowledge Management at Kent State I did some research on folksonomies and how the can support information retrieval.  I compared social bookmarking systems with search engines and directories.  I’m hoping to see the results published in an academic journal.   In the mean time, you can see a pre-publication copy of my results:

Tagging and searching [pdf, 989K]

Usability Testing

Usability tests can be seen to fall into two general categories, based on their aim: tests which aim to find usability problems with a specific site, and tests which aim to prove or disprove a hypothesis. This test would fall into the former category. A search of the literature will reveal that tests looking to uncover specific usability problems often use a very small number of participants, coming from Nielsen’s (2000) conclusion that five users is enough to find 85 percent of all usability problems. Nielsen derived this formula from earlier work (Nielsen and Landauer, 1993). Although there is much disagreement (Spool and Schroeder, 2001), this rule of thumb has the advantage of fitting the time and money budget of many projects.

Use of Eye-Tracking Data

In terms of raw data, eye tracking produces an embarrassment of riches. A text export of one test of Mealographer yielded roughly 25 megabytes of data. There are a number of different ways eye tracking data can be interpreted, and the measures can be grouped into measures of search and measures of processing or concentration (Goldberg and Kotval, 1999):

Measures of search:

• Scan path length and duration
• Convex hull area, for example the size of a circle enclosing the scan path
• Spatial density of the scan path.
• Transition matrix, or the number of movements between two areas of interest
• Number of saccades, or sizable eye movements between fixations
• Saccadic amplitude

Measures of processing:

• Number of Fixations
• Fixation duration
• Fixation/saccade ratio

In general, longer, less direct scan paths indicate poor representation (such as bad label text) and confusing layout, and a higher number of fixations and longer fixation duration may indicate that users are having a hard time extracting the information they need (Renshaw, Finlay, Tyfa, and Ward, 2004). Usability studies employing eye tracking data may employ measures that are context-independent such as fixations, fixation durations, total dwell times, and saccadic amplitudes as well as screen position-dependent measures such as dwell time within areas of interest (Goldberg, Stimson, Lewenstein, Scott, and Wichansky, 2002).

Because of the time frame of this investigation, the nature of the study tasks, and the researcher’s inexperience with eye tracking hardware and software, eye tracking data was compiled into “heat maps” based on the number and distribution of fixations. These heat maps are interpreted as a qualitative measure.

(more…)

Abstract

Diet can have a great effect on health, but few people keep track of what they eat each day, let alone how much fat, protein, Calcium, or other nutrients. Although most food items have nutrition information printed on the packaging, few people can tell you whether or not the 10 grams of fat in their candy bar is acceptable, or whether it has put them over the edge.

In this project the author assumes that a big part of the reason people do not keep track of their diet is that there is no easy way to do so. The final product of this project is Mealographer, a simple interface that allows users to enter in the foods and meals they eat each day, set simple nutrition goals, and see reports of their progress. Mealographer was created by implementing a large number of improvements to the product of a previous investigation, WhatYouEat. A usability test was conducted to evaluate Mealographer and find specific usability problems.

Previous Work – The WhatYouEat Project

Mealographer inherits much of its functionality from a previous project, titled WhatYouEat, part of an individual investigation from fall, 2005. The original project had two goals: to create an application that allows users to track their dietary intake, and to make the application as easy to use as possible.

WhatYouEat allowed users to record their meals, set simple goals for different nutrients, and

track their diet through simple reports. Supporting functionality included a simple user sign up and login system, and systems allowing users to indicate favorite foods and “usuals” – foods eaten on a regular basis.

WhatYouEat was demonstrated informally to several groups and an informal usability test was run with four participants. Although this style of evaluation was not rigorous, users were asked to use the site and comment on any confusion or difficulties. Many users also commented on design and additional functionality. Usability issues included difficulty in:

Targeting

• Even with a large screen size and large font, it was hard for one subject to click on fields before entering text.
• Field labels were used to enlarge the clickable area. It may be possible to have the cursor will default to the first field.

Layout

• Two users were a little confused about the two-column layout of input forms.
• A thin line was added to help make the grid more clear.

Forms

• Three users forgot to set the meal date at least once. The submit button was easy to miss. One user hit enter to submit the search form and didn’t expect the entire meal to be submitted. There were problems using the back button.
• The submit button was made more visible
• The forms were be broken up so that the submit button for a particular field only submits that field.
• Required fields could be made more clear with a symbol and some JavaScript.

Labeling

• Some labels were unclear or hard to read. In particular, dates presented in yyyy-mm-dd format and names of nutrients.
• The labels should be changed to reflect user expectations.

Measurements

• Many users had a hard time determining how much they had eaten, or understanding how much food each measurement amount actually represented. Few of them knew what an ounce or gram of a given food looked like, or how much of non-fluid items made up a cup.
• Some graphic representation of food amounts should be available in the system, as well as a conversion application. See Future Plans for more information on the approach to this problem.

Missing items

• Users more than once looked for food items that did not appear to be in the database at all. This included brand-name items or items from specific restaurants. This is a limitation for the USDA database.
• There is no simple or quick solution to this problem. See Future Plans for more information on the approach to this problem.

(more…)

Note: This was originally created for an information architecture class – the project was to redesign the Kent State School of Library Science web site. You can also see a usability study of the site.

Executive Summary

The current Kent State University School of Library Science (SLIS) does not meet the needs of the department. This project outlines a plan and strategy for designing a new site. The new site will better communicate the department’s image and core attributes to the outside world and better meet the needs of users. This report covers the entire process, from research and project goals, through the development of a new design and how to measure success. Major recommendations include the use of a simple content management system (CMS), a new navigation structure and graphic design, and a few new content elements such as news, video, and podcasts.

Introduction

This report will cover the overall strategy for the redesign of the Kent State University SLIS web site, including the site’s audiences, the vision for the site, and analysis of the content and maintenance. Finally, recommendation are made for the content, information architecture, and design of the new site. The ultimate goal of this project is to create a coherent analysis and plan for the SLIS department to execute. The result will be a site that better projects the image of the department, better serves the users, and, if possible, makes the staff’s job a little bit easier.

Site content has been updated, but the organization and design of the site has been the same since 2000. The web has changed a great deal in the last 5 years, and the Kent SLIS site look and feel is not exactly cutting edge. The faculty and staff have voiced a desire to update the site, and there is anecdotal evidence that at least some students find the site lacking. Any new design must better address the needs of the site’s audiences and should better project the image of the department to the outside world. Also, the process used to update the current site is slow and unwieldy. The new site will solve three main problems: poor ease of use, an image that does not fit the department, and difficulty updating the site and communicating with users.

The process followed in creating this report has included requirements-gathering meetings with SLIS faculty and staff, content analysis of the current site, analysis of server logs, brainstorming sessions with Information Architecture Knowledge Management (IAKM) students, analysis of similar sites, academic usability research, the creation of persona, card sorting exercises, wireframing, prototyping and other techniques. The report will recommend additional steps such as formal usability testing be taken as well.

(more…)

Friedman, B., & Nissanbaum, H.  (1996). Bias in computer systems.  ACM Transactions on Information Systems, 14(3), 330-347.

In this article Friedman and Nissenbaum look at bias in software systems. Although the word bias can cover a number of related concepts, the definition used here is something that systematically and unfairly discriminates toward one party or against another. The authors see three main classes of bias in computer systems: Preexisting bias, when an external bias is incorporated into a computer system, either through individuals who have a hand in designing the system or via the society the software was created in; Technical bias, where technical considerations bring about bias (from limitations, loss of context in algorithms, random number generation, or formalization of human constructs); and Emergent bias, where bias emerges after design when real users interact with the system (for example, when new information is available but not in the design, or when systems are extended to new user groups). A number of illustrative examples are given, and the authors look at a number of specific software systems and point out existing or potential biases. One of the systems is the The National Resident Match Program (NRMP), used to match med school graduates to hospitals. In this system, if a student’s first choice of hospital and hospital’s first choice of student do not match, the students’ second choices are run against the hospitals’ first choices. Overall, the result favors the hospitals. Two steps are proposed to rectify bias – diagnosis and active minimization of bias.

This is an extremely interesting subject, and and I doubt most users and programmers are any more aware of it now than they were in 1996. One more recent article, (http://web.mit.edu/21w.780/Materials/douglasall.html) which sought to turn literary criticism toward video games by pointing out cultural biases, also mentions the lack of study in this area. With so many people spending so much of their day interacting with software, why do these kinds of articles seem so few and far between? On the other hand, the particular examples chosen are illustrative but not very current. All three of the systems were large-scale, mainframe-type software that users interacted with in a very small sense. Would the risk of bias be even greater for a system which is largely a user interface?

One clear implication is shown in the diagnosis stage of removing bias—to find technical and emergent bias, designers are told to imagine the systems as they will actually be used and as additional user groups adopt them, respectively. So the charge is one-third ‘know thyself’ and two-thirds ‘know the users.’ The very notion of looking for bias is probably foreign to many user interface designers (in fact, few of the programmers I’ve met are even aware that accessibility guidelines exist for blind, deaf, and other users). The authors’ proposal that professional groups offer support to those designers who detect bias and wish to fight it is a nice thought but doubtful. Few programming or UI organizations can exert any kind of pressure or drum up much bad publicity, or if they can, I haven’t heard of it (which I suppose means they can’t).

Palmer, J.W. (2002). Web site usability, design, and performance metrics. Information Systems Research, 13(2), 151-167.

In this study Palmer looks at three different ways to measure web site design, usability and performance. Rather than testing specific sites or trying out specific design elements, this paper looks at the validity of the measurements themselves. Any metrics must exhibit at least construct validity and reliability—meaning that the metrics must measure what they say they measure, and they must continue to do so in other studies. Constructs measured included download delay, navigability, site content, interactivity, and responsiveness (to user questions). The key measures of the user’s success with the web site included frequency of use, user satisfaction, and intent to return. Three different methods were used: a jury; third-party rankings (via Alexa), and a software agent (WebL). The paper examine the results of three studies, one in 1997, on in 1999, and one in 2000, involving corporate web sites. The measures were found to be reliable, meaning jurors could answer a question the same way each time, and valid, in that different jurors and methods agreed on the answers to questions. In addition, the measures were found to be significant predictors of success.

This is an interesting article because in my experience, usability studies are often all over the place, with everything from cognitive psychology and physical ergonomics to studies of server logs to formal usability testing to “top ten usability tips” lists. Some of this can be attributed to the fact that it is a young field, and some of it is due to the different motive fueling research (commercial versus academic). One thing in the article I worry about, however, is any measure of “interactivity” as a whole. Interactivity is not a simple concept to control, and adding more interactivity is not always a good idea. Imagine a user trying to find the menu on a restaurant’s web site—do they want to be personally guided through it via an interactive Flash cartoon of the chef, or do they want to just see the menu? Palmer links interactivity to the theory of media richness, which has a whole body of research behind it that I am no expert on. But I would word my jury questionnaires to reflect a rating of appropriate interactivity.

The most important impact of this study is that it helps put usability studies on a more academically sound footing. It is very important to have evidence that you are measuring what you think you are measuring. It would be interesting to see if other studies have adopted these particular metrics because of the strong statistical evidence in this study.

The most straight-forward metric, download delay, is also one that has been discounted lately. The thought is that with so many users switching to broadband access, download speed is no longer the issue it used to be. This is especially false for sites with information seeking interfaces, which are often very dynamic and rely on database access. No amount of bandwidth will help if your site’s database server is overloaded.

Marchionini, G., & Komllodi, A.  (1998). Design of interfaces for information seeking. Annual Review of Information Science and Technology (ARIST), 21, 89-130.

In this chapter Marchionini and Komlodi examine the state of user interfaces for information seeking. Interfaces are defined as the conjunctions and boundaries where different physical and conceptual human constructs meet, and is at the center of information science in fields such as human-computer interaction (HCI and human factors. The chapter looks at advances in technology and research, summarizes the developments of the first two generations of user interfaces, and examines current (as of 1998) developments in the field. One way to look at the chapter is shown in figure 1, with technology, information seeking, and interface design research and development shifting from mainframes to PCs to the web, from professionals to literate end users to universal access, and from ASCII characters to graphics to multimedia respectively. Some early developments remain important today, such as the components of an interactive system – task, user, terminal and content (with context added later). Another milestone was the development of the GOMS (goals, operators, methods and selection) model, the first formal model of of HCI. Two themes throughout the chapter are the interdependent nature of research in this area and the importance of human-centered concepts and design.

This is a really good summary of the history of HCI with an eye specifically toward searching and information use. It’s not surprising the many of the names we have seen on articles this semester show up here as well. The only real regret I have is that there are no pictures. User interfaces often rely on visual display for interaction, so in addition to all the description it would be really interesting to see examples of the different generations of user interfaces. One other criticism is that little attention is paid the the interfaces of video games—I have read a lot of articles about interface design that ignore this field as well.

Although it is a little out of date, there’s a lot to be taken from this chapter’s historical perspective. I found three things in particular that were talked about in relationship to third-generation user interfaces that were particularly interesting. First was the move toward universal access or ubiquitous computing. It is in some ways a measure of success that researchers now worry about the lack of computers in Sub-Saharan Africa—this wouldn’t be a problem if information seeking computer interfaces were not so available, useful, and approachable. Second was the notion that the advance of the web in some ways slowed the advance of user interface design, although the apparent disadvantage quickly disappeared. This is something I’ve run into in a different form as a web designer—clients complaining that their web site did not look exactly like their brochure. Again, in some ways this was an embarrassment of riches—the web site cost nothing to distribute, could be found by search engines, acted as a storefront, but the lack of a particular font face was a step backward? Finally, the notion that the whole field is really interdisciplinary is important to always keep in mind.

Can, F., Nuray, R., & Sevdik, A. B. (2004). Automatic performance evaluation of web search engines. Information Processing & Management, 40(3), 495-514.

Although virtually all Internet users utilize search engines to find information on the web evaluation of search engines is often difficult. A large number of searches would need to be tested and each one would need to be judged subjectively by human participants. The authors of this paper have devised a new way to test search engines and have tested their method against evaluations done by human judges, and found their automatic Web search engine evaluation method (AWSEEM) significantly predicted the subjective judgments. In the human-evaluation control, users were given a list of resources called up by the various search engines with no idea which engine each came from and were asked to rank the relevance of each. In AWSEEM, each query was run and the top 200 results for each engine were compiled into a collection of vectors which are then ranked by their similarity to the “the user information-needs” (including the question, the query, and a description of the need). The system then looks at the top 20 ranked pages for each engine and counts how many are in the top s (50 and 100 are used) commonly retrieved pages. These are assumed to be relevant.

One possible issue with this system is that it requires a little more human interaction than first assumed—the query providers must provide more than just a query. A bigger issue, though, is the choice of measure for relevancy. AWSEEM assumes that if a result appears in the results of multiple engines, it is relevant. This may be reasonable, but does raise the question—what if all the engines studied are wrong? For a simple example, searching for my own name online will retrieve a large number of results that are the same in many search engines but have nothing to do with the particular Jason Morrison who sits here typing this. Another interesting thing to note is that they did not find much of a statistically significant difference between the performance of the different search engines using either method (although more so with the human-judgment method). Very few scholarly articles (and even fewer popular press articles) bother to do this when pitting search engines against each other. Is it possible that the very notion of the “best” search engine has been statistically meaningless for some time?

The authors make a good point about the difficulty in using real users for search engine evaluation. An automated approach is one answer, but there is another—the problem is that too much time and effort is required of a small number of users. Instead, if tiny amounts of time and effort were spread across thousands or millions of users, similar results could be achieved while still using subjective measures. For example, if every time a user got results on any search engine they were presented with a simple “rate these results on a scale of 1 to 5 stars” input, they could quickly and effortlessly contribute data toward a shootout-type study. Cooperation of the search engines would not necessarily be needed, if one could use a university’s proxy to substitute or add the input for popular search engines, for example, or if a generic search page was set up to produce results from randomized (double-blind) engines. It would be interesting to try this, AWSEEM, and individual evaluation in one study to see if there was a statistical correlation.