JasonMorrison.net

I’m a fan of Twitter – it can be really useful. But status update services and microblogging are relatively young technologies. Twitter is the frontrunner now, but it’s still possible that everything could go south really fast. Here are five people (or more accurately, types of people) who could destroy Twitter and what can be done to stop them.

The list is in no order, except I’ve saved the most dangerous for last.

1. Spammers

Twitter spam is growing, and my guess is it’s a profitable business to be in. Spammers are getting crazy refollow-rates with very little effort put into their fake profiles. Part of this is a technical problem, with Twitter playing catchup to the collective innovative power of the greediest jerks on the internet. The more difficult part is social – users’ trust barriers are too low. Either Twitter finds ways to deal with this, or people will start treating reply tweets, direct messages, and invites the same way they do unsolicited emails now. One of the reasons I stopped logging in to MySpace was a flurry of fake friend requests that followed every session. Twitter runs that risk, in addition to the risk of service degradation.

What can be done? The good news is that no communication medium can be considered successful until someone has tried to send you unsolicited marketing and scams over it. But the Twitter team needs to redouble their efforts and head off potential problems proactively. For example, there are lots and lots of apps built on top of Twitter’s API – and almost all of them ask for your username and password. How long until one of those apps is compromised, or worse scammers make password-phishing apps of their own? Twitter needs to implement strong API keys or something like OpenID.

2. Anyone who uses url shortening services.

It’s hard to fit both a witty observation and a url in 140 characters, especially given url inflation. Bit.ly, Tinyurl, and the like perform the valuable service of giving you more space. They also cloak the destination of almost all the links on Twitter and get everyone used to following links blindly. I’ve already had friends whose accounts were hacked in order to send out a tweet like: “Check out this hilarious video: http://tiny/innocuousgibberish”. The New York Times’ account has been hacked, among others. Twitter can work on improving security and removing spam, but the more everyone uses url shorteners the more we train our friends to click recklessly. I’m as guilty on this one as anyone.

What can be done? People post links to Twitter frequently enough that maybe it should be separate field with it’s own character limit. If that’s too much complication for the brilliantly simple interface, maybe url previews should be enforced. Clients can do this now, but to be safe it should be done by Twitter.

3. Pirates, ninjas, zombies, and mafia thugs

Ah, I remember logging into Facebook the day I got my first “robots vs. hobos vs. Chuck Norris vs. etc.” request. “Ha,” I thought, “that’s a somewhat entertaining way to extend an internet meme into a social networking site.” Little did I know the horror that was about to unfold.

In all seriousness, the “tag, you’re it” games and gratuitous survey apps didn’t ruin Facebook, but they did make everything a bit more tedious. Those apps still fit within the umbrella of social networking – they don’t work at all in Twitter’s use model. When I log in, I want to see, very quickly, what the people I’m interested in are doing or reading. I don’t want to weed through their halves of various games I’m not interested in.

What can be done? This one is up to us – just don’t do it. Twittering with a hashtag for an event, a theme, etc. is fun and useful to others. Sending around vampire bites is not.

4. Chinese government officials

Think periodic fail whale sightings is bad for Twitter’s reliability? China can (and does) just block the whole site, most recently in advance of the Tienanmen Square anniversary. Why does this matter? China is a huge market, and growing. The days where being big in the U.S. meant major marketshare on the whole web are running short. What’s worse countries with theoretically free speech like Australia are following the Chinese model, proposing national internet content control (i.e. censorship).

What can be done? Many American companies just give up. Even Google has had to bend to government pressure. This is not easy to remedy. Perhaps there’s a way to take advantage of the small byte size of tweets, decentralize serving, and wrap access with something like Tor to get it through the Great Firewall. Let’s hope there’s a grad student or genius hacker out there with the right idea and Twitter is smart enough to hire them.

And finally, the absolute worst, most pressing threat the Twitter’s survival is…

(drumroll….)

5. Your mom

Despite the allure of turning this into one big “your mom” joke, I am completely serious. What happens when your mom joins Twitter? Do you censor yourself? Take your tweets private? Delete off-color tweets from your recent past?

There’s no right answer. Just about any social software eventually runs into this dilemma where the very different ways you communicate personally, professionally, and publicly collide.

What can be done? Some of the problem might fade as the userbase of sites like MySpace, Facebook and Twitter ages. But that will take years, so what can Twitter do now? It might help to have better relationship management. You could at least put your friends in one group and family in another. But in general, this strikes me as the toughest problem of them all – I don’t think there are any real solutions for the general possibility of parental embarrassment, or all efforts of every teenager in the world has yet to reveal discover them.

Disagree? Any threats I missed? Please post in the comments below.

It was inevitable. As Twitter has grown and started pushing into the mainstream, spammers have started ramping up abuse. At first glance, Twitter isn’t the most obvious target – you actually have to follow someone to get content from them, users don’t generally search it for high-cpc stuff like meds and lawyers, and how much spam can you really get into 140 character messages?

But I’m seeing more invites from users like the one below:

First: What is Twitterspam? How do I know this is a spammer?

When it comes to spam, most people “know it when they see it,” but it’s helpful to look at the specific signals that this user might not be worth talking to. First off, they have 180 followers and yet haven’t posted a single update. The photo is a dead giveaway. The bio is actually pretty well-done, it’s in English and it’s not outlandish, but the homepage link (http://my-pictures.no.tp/tlow/) – she’s in Portuguese Timor?

Second: Why spam Twitter?

Spammers have two reasons to abuse Twitter: monetary payoff, and because it works.

How can they make money by tweeting a bunch of random people? Well in this case they aren’t, at least not yet. The payoff has to be through the homepage link, which I’m not following and you shouldn’t either. You get a friend invite on a system that, so far, has been a medium of immediate, short, personal communication. Your trust barriers thus weakened, you at least want to see who it is. They don’t have any updates yet, so you click the homepage link and… Virus. Or a maze of PPC affiliate pages and redirections.

Above I said spammers are hitting Twitter because it’s working. How do I know? Look at the number of followers, and the ratio of people followed to followers. About 22 percent of the people spammed so far have responded. I don’t know how many click through to the home page link, but if half the people bother to go that far they’ve got an amazing success rate for spam.

I wish Twitter luck. I know a few people over there, they’ve got their work cut out for them. This sort of thing isn’t easy to fight, it’s an ongoing process. They’ve already taken some visible steps, like using rel=”nofollow” on the Bio link, which at least keeps away blackhat SEOs looking for sources of pagerank. They’ll probably have to do more, most of it on the backend where you and I will never be the wiser. Happy spamfighting!

I read two reports today about large-scale botnets that really pointed out that security is still an open problem on the web. Recently, researchers got access to a nasty botnet, Torpig (original paper: Your Botnet is My Botnet: Analysis of a Botnet Takeover). A few months earlier researchers hijacked the Storm Worm and looked at its profitability (original paper: Spamalytics: An Empirical Analysis of Spam Marketing Conversion). Both papers are fascinating, but terrifying reads.

Some findings:

• In 10 days, a botnet running on 160,000 machines stole credentials for over 8,000 bank accounts.
• About 1 in 10 people who open a spam email click through to get infected by the malware.
• 350 million spam emails resulted in only 28 sales, but the average purchase was $100.

How do these botnets get control of machines? How do they make money? Whether it’s a spammer who needs to get someone to make a purchase on a website or a scammer stealing credit card numbers, passwords, and other information, ultimately you need to get someone to a bad website. Think about all the paths you might take to different sites during the day:

• Via a web search
• Clicking on a link in an email
• Going directly to a favorite site
• Clicking through an ad

Spammers and scammers try to take advantage of all of those methods, and given the huge volumes of machines at their disposal, it’s a wonder search engines, spam filters, and advertising systems protect users as well as they do now. Between the first and third bullet point above, there’s a huge motivation to hack otherwise good sites to inject drive-by download malware – it can happen to anyone.

So what can we do about it? I think it ultimately comes down to a combination of smarter automated methods, better ways to establish trustworthiness, and removing the economic incentives for spamming, identity theft, and hacking. I have a few posts in mind about some current tools that help with the trust issue and how we might be able to build a social web of trust.

This isn’t a new discussion, Tim Berners-Lee has been writing about the web of trust since the 1990s. But all the work done since then has yet to really solve these problems. And really, so long as a few people are willing to click on a malware link or buy drugs via a spam email, it will never stop.

Blogs are great because they give you a creative outlet and let your readers comment on you posts, making it a much more social experience.  But spammers take advantage of comment forms, using scripts and bots to fill the web with links back to their site.

What can you do about it?  Even with captchas, systems like Akismet, and other automatic techniques (you can read more about these here), some spam will slip through.  Specifically, compliment spam.

What is compliment spam? Spammers know you and I like to be told what great writers we are, how helpful our posts are, and that we are brilliant geniuses.  So they set their bots to spam you with complimentary comments that just so happen to link back to their crappy blog, online casino, or fake viagra store.  Here’s an example:

Typolight
http://www.typolight-blog.de | info@typolight-blog.de | 82.146.49.61

Thanks, you nice post that helped me alot.

From Keep your WordPress site from being hacked with automatic upgrades, 2008/09/06 at 9:27 AM

So, at first glance this looks like a legit comment.  The post in question was a “how-to”, so it would be nice to hear that someone found my instructions helpful.  But, do a Google search with the comment in quotes (an exact phrase search) and you’ll see the problem:

http://www.google.com/search?q=%22Thanks%2C+you+nice+post+that+helped+me+alot.%22

At the time of this writing, we see 168 instances of this exact comment.  By this same Typolight person.

So that’s my tip – if a comment seems a bit too randomly complimentary, throw it in quotes and do a Google search. Then, if it’s spam, make sure to spam it – systems like Akismet only work because we’re all reporting spam.

If you really want to go after the spam poster, you can also give their site a bad rating on Web of Trust, StumbleUpon, and other reporting systems.

Maybe if I get some time I’ll throw together a WordPress plugin to make this easy to do.  If you’d like a plugin like this (or have other tips), drop me a comment and it will help motivate me.

I happened across NewsTrust.net, a new social news aggregation site.  I’m a big fan of other sites in the category like Reddit, despite their flaws, and NewsTrust includes a tagging system so I feel obligated to investigate it like any other folksonomy.

So I created an account to give it a try.  The big difference between this site and others is the emphasis on quality journalism.  NewsTrust asks for your real name, and in addition to giving weight to users who write good reviews and get votes from other users, it adds factors like experience as a journalist to the mix.  It makes specific disticntions between mainstream media sources and altenrative media sources.

It’s an interesting idea, and it’s good to see journalists working together with programmers and web developers to make use of some of the social software techniques that newspaper websites so often catch on the trailing edge.  The site’s features seem geared toward providing users with the best that professional journalism has to offer with a dash of brilliant amateur writing thrown in – even the page layout looks more like a newspaper site than a Digg or Del.icio.us clone.

But I’m not sure it will work, at least not without some tweaking.  I don’t know if they put a lot of weight into the “experience” of users, but it didn’t require any verification of my 5-9 years of journalism experience (for the record, that’s four years in college plus more than a year of stringing here and there).  Here’s the problem of trust again, though hopefully mitigated by fellow users’ reviews.

The other issue is interaction design.  The widgets and buttons all work just fine, but when you rate a story you’re asked to score on six dimensions: Recommendation, Trust, Information, Fairness, Sources, and Context.  Only the first is required, but give users options and they are bound to feel obligated to exercise them.  Give them too many tasks and they will tend to give up.  So the simple interaction model of Reddit, where users don’t even have to click through to rate a story, might be information-poor but participation-rich in comparison.

Still, I will play with the site more and I wish them luck, I think they have some promising ideas.  For example, in their blog they talk about gathering sources from other countries based on big world news events, specifically the Russian invasion of Georgia.  Reddit is only fleetingly so reflective and few sites use temporary peaks in interest to get long-term data on source credibility.

A few months ago I attended a really interesting talk by Eric Meyer where he touched on the use of microformats.  You might know Eric from his excellent O’Reilly Press CSS books.

What are microformats?  Before giving an example, I’ll give a little context.  When Tim Berners-Lee created the web, he tried to make HTML simple, flexible, and meaningful.  He succeeded on the first two counts but the third was quickly left by the wayside – many designers didn’t care what a particular tag meant, so long as it could be used for page layout.  The use of tables to arrange graphic elements instead of holding tabular data is a perfect example.

So Berners-Lee has been talking for years about the next step – the semantic web.  In the semantic web, tags are used to say what a particular piece of content is, with all styling done with stylesheets.  There is, of course, more to the semantic web than just separating content and presentation, after all you can work that way with HTML and CSS now.  One other key component is the web of trust, where people and web sites are able to describe relationships to each other so that search engines can help you find trustworthy content automatically.

Unfortunately, the semantic web has not really taken off.  There have been lots of meetings and XML schemas but it’s all too complicated, the process is too bureaucratic, and everything is being designed from the top down.

This is where microformats come in.  Let’s say you have a blog and you’ve tagged all your articles.  You’d like to let search engines and aggregators like Technorati know what your tags are.  But HTML doesn’t have anything like this:

<tag>semantic web<tag>

So what do you do?  Simple, use the rel-tag microformat:

<a href=”http://example.com/tag/semantic+web” rel=”tag”>semantic web</a>

The microformat makes use of existing html tags and attributes and just follows simple conventions.  But now that this little bit of meaning can be interpreted by spiders and other programs, we’ve actually added a pretty powerful bit of functionality to the web.

Most blog software, including WordPress, includes does microformatting for you.  If install my tag cloud plugin Altocumulous, and view source, you can see for yourself.

For intranet purposes, the hCard and hCalendar microformats look promising.  Take a look at microformats.org to see why I think so.  I’ll write more on it later.

Although you don’t hear about it much, trust is an extremely important issue in the software world.  A common example is eBay – how could eBay stay in business if millions of anonymous buyers and sellers didn’t have a certain level of trust?

Andy Brice, a software developer, gives a really interesting example of the problem of trust in his blog.  He became concerned that his software products were getting a ridiculous number of awards and 5-star ratings from shareware download sites.  He devised an experiment: if you create a text file, change the file extension to .exe, and submit it to 700 download sites, how many award would you get?

It turns out you would get tons of awards.  A large percentage of these sites, which ostensibly provide users the service of evaluating shareware and freeware, are in reality just trying to skim adwords revenue.

Social software, if applied correctly with enough participation, can help to solve this problem.  It is much harder to fake 1000 del.icio.us bookmarks than it is to make an authoritative-looking award banner.

Many of us work on projects internal to companies where we don’t confront these issues directly on a day-to-day basis.  Large companies can generate billions of pages of documents and code each year.  Add to that the billions of external web pages we use as reference material.  Tools such as social bookmarking can help build up this network of trust and sift through the less useful resources even on intranets.

So now that we have the tools available, all we need is participation.  You’re reading this, so I’m probably already preaching to the choir.  Trust is a really interesting issue, though, so I’ll be writing about it here and there in the future.

Here’s something interesting: the Last.fm mainstream-o-meter. Apparently my music tastes are 41.48% mainstream, at least within the Last.fm community. The biggest boost to my mainstreamness is Radiohead, which is listened to by an astounding 103.56% of Last.fm users.

Last.fm no doubt attracts a skewed population, but I do have to say I’m surprised that it continues to differ from radio playlists and CD sales. Radiohead is a perfect example – from my sampling of commercial radio over the past few years I would say they are almost completely absent. Yet a large number of people listen to Radiohead on their PCs.

Next up is a page that tells you which movie reviewer has tastes that best match yours. I’m sure we’ve all read reviews online or in the local paper and wondered if the reviewer saw the same movie. With sites like Rotten Tomatoes and Metacritic, you’re n longer limited to the opinions of a few writers. The average scores on those sites are interesting, but still don’t always match my tastes or your tastes. This will give you some names to look out for.