JasonMorrison.net

First, a disclaimer: the title of this post is obviously exaggerated. Proof is an awfully big word to throw around, and although I employed pretty good experiment design practices and statistical checks, I can’t really prove that Reddit should do this or that. But I can show that what they are doing now is not working, at least when it comes to search.

So, I got an email the other day letting me know that my article, Tagging and Searching: Search Retrieval Effectiveness of Folkonsomies on the World Wide Web, is being published in the July 2008 issue of Information Processing and Management (here’s the official DOI link to the article). In the study I compared search performance between traditional search engines (like Google), subject directories (like Open Directory), and social bookmarking systems (like Reddit) and their folksonomies.

What’s a folksonomy? The word is a play on the term taxonomy - a taxonomy is a system of organizing and categorizing things, like the Dewey Decimal System. Taxonomies usually follow very strict rules and are controlled by experts. A folksonomy is a system of organization built by large numbers of regular users, who add things to the collection, evaluate them, and usually tag them with keywords.

In my study, the social bookmarking systems with tagging systems did surprisingly well - Del.icio.us was more precise than Open Directory, and at a cut off of 20 results it’s precision was fairly close to that of the search engines.

Reddit, however, did not fare so well. It consistently had the lowest precision, meaning that searches returned very few relevant results. There could be many reasons for this, but the biggest difference between Reddit and the others is the lack of tags.

Now, it’s possible that the folks at Reddit have no interest in search, or information retrieval in general. I think Reddit is very effective at bringing out new and interesting links on a daily basis and encouraging commentary (just my opinion, no stats to back that up). But I think it’s a big missed opportunity not to add tagging and see where it leads.

(One last disclaimer: this post is my personal opinion as someone who enjoys using Reddit and does not reflect on my employer. This post refers to research done independently as a grad student.)

To complete my MS in Information Architecture and Knowledge Management at Kent State I did some research on folksonomies and how the can support information retrieval.  I compared social bookmarking systems with search engines and directories.  I’m hoping to see the results published in an academic journal.   In the mean time, you can see a pre-publication copy of my results:

Tagging and searching [pdf, 989K]

Earlier I mentioned that I have some photos uploaded to Panoramio. I’ve also played with Flickr off and on, and have recently started uploading some photos there as well. To add to the confusion, I use Picasa to manage photos on my hard drive, and my wife uses iPhoto on her Mac. Picasa has a web albums feature, and I’m sure iPhoto has something similar with a .Mac account.

Why use four different services that overlap each other to such a degree? Picasa and iPhoto both do the important job of managing photos locally, Flickr seems to have the largest community and the most widgets written for it, and Panoramio integrates with Google Earth. Since I want to do all those things, I have to use them all.

There are ways to make them play nice together. You can use a Gmail account to email photos from Picasa to Flickr, and so far it seems to work fairly well. There are a few iPhoto plugins to upload to Flickr and you can use iPhoto to subscribe to Flickr photostreams. Google just bought Panoramio, so I’m sure there will be more integration there soon as well.

Even with all these options, there are some annoyances. Picasa’s keyword tagging is not very useful, it only allows one-word tags. I tried creating multi-word tags with dashes or by enclosing them in quotes, but Picasa ate the special characters. There’s also the complication of managing public photos vs private photos.

Still, it is amazing how well these different websites and programs work together, through the magic of RSS, web API, and plain old email.

If you’d like, you can see my Flickr photos here. You can also see my photos in Panoramio, or just look close enough in Google Earth, since a few of my photos now how up there.

Here’s something interesting: the Last.fm mainstream-o-meter. Apparently my music tastes are 41.48% mainstream, at least within the Last.fm community. The biggest boost to my mainstreamness is Radiohead, which is listened to by an astounding 103.56% of Last.fm users.

Last.fm no doubt attracts a skewed population, but I do have to say I’m surprised that it continues to differ from radio playlists and CD sales. Radiohead is a perfect example - from my sampling of commercial radio over the past few years I would say they are almost completely absent. Yet a large number of people listen to Radiohead on their PCs.

Next up is a page that tells you which movie reviewer has tastes that best match yours. I’m sure we’ve all read reviews online or in the local paper and wondered if the reviewer saw the same movie. With sites like Rotten Tomatoes and Metacritic, you’re n longer limited to the opinions of a few writers. The average scores on those sites are interesting, but still don’t always match my tastes or your tastes. This will give you some names to look out for.

Panoramio is yet another photo sharing site like Flickr. What sets them apart is their integration with Google Earth. As you pan around the globe, photos from Panoramio users will appear in as icons. The also support tagging and some social networking features as any Web2.0 site should.

I’ve uploaded some photos, you can see my page here. I’m a little disappointed that I don’t see an easy way to embed my photostream directly into a blog post.

Even more interestingly, you can see my photos in Google Maps or (if you have it installed) on Google Earth.