Random blog-like rambling from Rachel's brain. A mixed up mess of usability posts, fiction, and travel.

On Rating Systems and User Motivations

I've been a neglectful blogger and now, I'm going to spurn writing about tagging in favor of discussing ratings systems. Primarily because I recently encountered this

really interesting blog post

by the fine people at YouTube. The sum of what you'll learn in that post is this: the vast majority of videos at YouTube are rated 5 stars.

"Great videos prompt action; anything less prompts indifference. Thus, the ratings system is primarily being used as a seal of approval, not as an editorial indicator of what the community thinks about a video."

The general intention of designers when they put together a rating style system such as the one at YouTube is with the idea that ratings will be used as a way users can judge content. Unfortunately, if all your content is either not rated at all or rated 5 stars, users won't get much out of those ratings.

Kittens, riding a vacuum.

Kittens, riding a vacuum.

This isn't a universal issue though. Some sites use star ratings with great success. Take Amazon and Netflix for example. Users of those sites are diligent about rating the content and sometimes will spend time on the site for the sole purpose of rating. Why are users willing to spend time rating movies on Netflix but won't take a couple of seconds to rate videos on YouTube? The answer has to do with user motivation and the inherent selfishness of users. Now, do not take umbrage, there's nothing wrong with being a selfish user. It's the job of a site designer to encourage you to act otherwise, and if they fail, it's no skin off your teeth. Carry on.

What's important for us, the designers, is to recognize and be aware that users are inherently selfish and will be highly unlikely to engage in activities that do not offer them a benefit of some kind. No one rates videos on YouTube, because the act of rating a YouTube video does not provide you with any benefit. The only people it could, in theory, assist are the submitter of the video and some small percentage of the user community when they browse for the video. But you the rater? You don't get anything.

In contrast, if I go to Netflix and spend some time rating the movies I've watched I can actually see the recommendations improve. The more movies I rate, the better Netflix's recommendations turn out to be. Thus, I have a good motivator for continuing to rate the content. Luckily for Netflix's user base, my ratings also benefit the rest of the community. Amazon works the same way. Ratings work out exactly the way we want because there's a clear benefit to everyone when it comes to contributing.

So what if YouTube took a page from that book? What would be interesting to see is if they instituted a recommendation system that actually used those video ratings to pull up similar videos relevant to you, the selfish user, would people start using the rating system differently?

Not easy to test without some significant work, but my inclination is that yea, you'd see some changes over time.

We could probably extrapolate the general theory at work here to a lot of other features and applications. The key take away is that if you want a user to take an action, you have to provide reasonable motivation no matter how small and quick the interaction appears. For example, most discussion forums are populated mostly by lurkers, only a small percentage contributes. Designing better motivators into your design could increase participation and lead to a better overall experience.

It's food for thought, in any case.

On Karma, Oh What is it Good For?

I don't believe in karma. At least, I don't believe that people demonstrably get what's coming to them based on their past behavior. Still, that's not what we're here to talk about. We are here to discuss internet karma.

Karma is that elusive number, setting, hidden voodoo that many sites of the Reddit and Digg variety use to elevate certain users above the wild fray. Karma, in theory, encourages users to submit quality content with the hopes that quality will equal higher karma. Higher karma, in turn, can also be used by the site itself to push content submitted by those users up higher than those submitted by newcomers or trolls.

Sounds pretty good doesn't it? Well, many many things have a tendency to sound good in theory and to then fall apart when us irrational human beings actually get our hands on it. Internet karma is no different.

Karma is intended to work as an incentive system, and for a lot of people it certainly does just this. That little number can become an obsession. Getting it higher, getting to be the highest, can turn into a goal that undermines the essential point of a site like Reddit. How so you ask? Well, it's the karma whore issue you see.

karma whore: originally coined at slashdot, a karma whore plays to the prejudices of the masses to get positive moderation on their comments (via urban dictionary).

There are, of course, folks who take that definition to the very extremes, but to small degrees almost every member of an online community is going to end up at least a little susceptible to this phenomenon. The reason is, after awhile posting content that doesn't see a lot of traction and never makes it to the front page, a user is likely to take one of two paths:

1. Leave

2. Start posting content they know the community likes.

So, thusly, the community feeds it's own interests and only those who are willing to play along see their karma increase.

This isn't that different from how we interact with other people offline of course. Like minds hive together, that's human nature, but what if we wanted to see something different happen in cyberspace? What if we wanted to create a community that instead of feeding our existing interests and beliefs expanded and challenged them? Karma, the way I've seen it used today, is an ideology that keeps that from happening.

On Reddit, karma accumulates if the net up votes on your submitted content goes up. Imagine a situation in which instead, the level of controversy on your content resulted in a karma increase. Instead of incentivizing the user to submit content they know will appeal to the beliefs of the community, this encourages the user to submit content that will be polarizing in some way. Net result will be a very different picture of the overall content submitted to the site. Certainly, you can view the controversial items on Reddit, but there's no system that outright encourages users to submit that kind of content.

Maybe we can go the other direction entirely. After all, the best way to firm up your beliefs is to have them challenged. Try this site idea on for size: instead of positive karma, we encourage negative karma. The more down votes you get the higher your score. 

Clearly, there's still a failing in all of these systems. That failing is that it is still always possible to game the system. So what if we abandon the idea entirely, at least as a visible, measurable entity. Hide the karma from users and tweak the algorithm on the back end to get your desired results. Will users still submit content if they aren't 'rewarded' in some fashion? I think so, provided your algorithm still works well enough that interesting, varied content crawls its way to the top. 

So, karma, be it good or bad or controversial, certainly produces interesting dynamics in an online community. I'd love to see it used in a more varied or dare I say, backward fashion.

Keep it real guys, and keep your karma whoring to a minimum.