We Already Suffer From The Paradox Of Choice, Now We're Paralyzed By The Ubiquity Of User Reviews
from the analysis-paralysis dept
Though you may be finishing your holiday shopping online, away from the mobs at the malls, ubiquitous user reviews on most merchant websites make shopping online slightly less lonely. It’s nice to know that you’re not the first one to try out some product, but just how helpful are user reviews? Even the most cursory pondering on this subject leads to a whole slew of questions and concerns. Just who are these reviewers and can you trust their opinions? Are they influenced by unseen motives (like free meals)? Perhaps their criteria for judging something is completely different than what you would judge on? Before you know it, you’ve spent hours chasing user reviews down a rathole, and you’ve ended up where you started — no closer to finishing your shopping. So, do user reviews actually make shopping online better, or are they just a waste of time? We are already plagued with the paradox of choice: where an increase in the number of choices available has been shown to actually decrease satisfaction (since by choosing 1 thing, we are not choosing 89 others, and that makes us sad). So, add to this a dizzying array of user opinions, and all of a sudden, even choosing a box of tissues ceases to be a mundane task. Do I want the normal “Puffs” (average rating, 4 stars) or do I want the “Puffs Plus Lotion” (average rating, 5 stars) for more money? Where in this process does the fact that I use my tissues to clean my monitor once in awhile come in to play (in which case the Lotioned tissue would definitely not be a 5 star product)? User reviews have been around for years now, yet not much has been done since they were first launched to make them more useful. Perhaps the true role of user reviews is to serve as a sort of a sanity check — it’s the health rating posted at the front of a restaurant. As long as nobody has seen any cockroaches running around the kitchen, it’s safe to eat.
Filed Under: user reviews
Comments on “We Already Suffer From The Paradox Of Choice, Now We're Paralyzed By The Ubiquity Of User Reviews”
I find reviews usefull
I usually have a specific product in mind, and find the guy who says the door release is weak, for instance, is saving me time and money. The reviews on NEW EGG are usually spot on, I seldom buy a less than 4 egg product, unless the reviews are obviously from lightweights who just can’t configure a complicated product. Still a lot of reviews that say the “first one had to go back” is as big of a concern as a 79% positive on EBAY.
Re: I find reviews usefull
I was about to post about newegg too. I think that they do a really good job of soliciting and recommending good products. You don’t get the people pissed about a slow delivery bashing a good product, or something like that. If you buy a 4 or 5 star item and it has a good number of reviews you can bet that it will be a decent product. Also lending confidence to this is the fact that I can see my own reviews, positive and negative, on their site right now.
PEBKAC
I think one of the biggest problems I see in user reviews boils down to stupidity. Sometimes I see reviewers describing problems that are obviously user error. Often, however, it is difficult to tell whether the reviewer is dumb or has raised a legitimate issue with the product.
It’s for this reason that I think user reviews are best taken in aggregate. If even a sizeable minority of people love the product, I think it’s probably decent.
They’re no different from pro reviews. Some are good, some are bad, and you have to search through them to figure out if it’s the right product.
But user reviews and a pro reviews comes from different places.
A user review speaks to passion, either they liked it enough to sing its praises or hated it enough to complain to someone.
A pro review is more instructive, more thorough and (hopefully) less biased.
Reviews are fine as long as unmoderated.
Some reviews systems are horridly moderated, to the point that any negative review is removed. Which completely defeats the purpose, so I usually look for negative reviews as an indicator of this nasty practice. And while more times the negative review is “PEBKAC” related, sometimes they help.
For example I was thinking about getting RPG game called Witcher but after reading user review about the lame button mashing combo system I decided against getting it. Something the “professional” reviewers neglected to notice/mention.
I generally treat online reviews as little more than light entertainment. Maybe, under some circumstances, you might use them in aggregate as a rough guideline, but that’s about it.
Even as a cynic I take the time to look at user reviews when they are available. At least for me they may bring to my attention things I might have forgotten to consider (battery life, noisiness, compatibility issues, etc.) That way I don’t have to rely on the independence or the skill of the reviewer, they just help double check my list of important features. In my experience the pro reviews are not as useful.
A couple of thoughts
1. Maybe sites should evolve into specialized review sites, instead of the current aggregators (summize.com), or embedded in the purchase sites (amazon.com). An example of this is tryphone.com which lets you actually test the phone before you buy it.
2. It might be interesting to have a review site that has a human editor that creates a criteria, and when a user reviews a product, he can give both weights and marks to each one along with the free text review. This way, not only does the site have more information, but users using it can enter their criteria weights, and narrow the reviews to like-minded individuals.
A couple of thoughts
1. Maybe sites should evolve into specialized review sites, instead of the current aggregators (summize.com), or embedded in the purchase sites (amazon.com). An example of this is tryphone.com which lets you actually test the phone before you buy it.
2. It might be interesting to have a review site that has a human editor that creates a criteria, and when a user reviews a product, he can give both weights and marks to each one along with the free text review. This way, not only does the site have more information, but users using it can enter their criteria weights, and narrow the reviews to like-minded individuals.
User Reviews
I’ve quit using user reviews as a factor in my purchases for some time now. I’ve noticed an increased trend in these reviews containing information that boils down to price than product.
Take, for example, this:
“I really enjoy this MP3 player and for the price, you can’t beat it.”
That’s a review? What really stuns me are the “5 out of 5” rating systems that appeared all of a sudden. These ratings often don’t reflect the actual review at all, making me wonder what in the world people are thinking.
I was looking for an HDTV online to get an idea of specs for my choice. The reviews I’ve read were absolutely worthless while the “stars” were given high marks.
One reviewer wrote about a Panasonic LCD tv, complaining how the picture wasn’t clear, the sound was choppy, and ended up rating it with 4.5 stars.
My advice to people is simply ignore the user reviews and ensure the website offering the product gives detailed “specs” of the product.
Because in the long run, it’s your money. Would you really be happy chunking down $1999 for a TV that someone rated a 4.5 but had problems with?
People are stupid and the internet shows this every day.
I like them
User reviews give me some extra perspective that can sometimes be lacking, and often are able to reveal a lot more information about products and features that aren’t always mentioned, even by professional reviews.
Past their prime
I am increasingly skeptical about many categories of reviews:
The “we got it first” blogger — these characters post ultra fast reviews of new gadgets and gizmos months before they hit the stores. Unfortunately these reviews are seldom anything more than races to be first. I bought a Blackberry 8300 from AT&T and found the sides creaked unbelievably. Soon I was joined in my complaining by several other people on message forums. Yet the very prominent sites that posted picture after picture and youtube videos — NEVER once mentioned this. Phooey.
The “amazon cheerleader” – these are people who respond to the newest fad which is authors offering spiffs to their loyal fans to buy (and presumably review) their books. You recognize these because the reviews are all written within a day or two – and all gush. Then there are no reviews for weeks.
The “new toy glazed review” – if I never read another review about a cell phone or smartphone that includes the words “beautiful screen” it will be too soon. Who doesn’t think their shiny new toy is the best. Why not actually use the gizmo for a week and THEN write about it. It cracks me up when I read the gushing review in a forum and then in a month you read the person listed the item on eBay. This happens all the time.
The “helpful forum member” – reviews all sorts of items. Always a 5 star gushing review. Lots of praise from other members. But wait, is that reviewer paying for the items or do they use the forum as a way to build up an inventory of free stuff sent to them by vendors. I have no proof of this but boy am I suspicious.
So I’ve started to follow only a select few Blog sites where I regularly read. I’ll also look for more than one confirming review as well as more detail than simply “it’s great”. I want to know the reviewer has used it in daily practice for more than a day or two.
Two minds...
I’ve had a delightful dinner in a restaurant that I later learned was panned by 5 of the 6 reviewers on a restaurant reviewing site.
OTOH, I sure wish I’d seen the reviews on the Palm LifeDrive(tm) before I wasted my money on one.
Before I make a major purchase, I do try my best to research it, but reviews are iffy. If all of the reviews are positive, I consider that suspect, especially if the product isn’t actually publicly available yet. I hesitated to buy an iMac because of the over-the-top evangelism I saw (BTW, I did finally buy one, and I’m satisfied with it, so far. But after over 20 years in the PC world, I’m going through some adjustments. Maybe after I get used to it, I’ll be one of those over-the-top evangelists myself. Or maybe not.).
If all the reviews of a particular item are negative, but no articulate description of specific problems are given, I also consider that suspect, because negative reviewers are usually more motivated to provide their opinions than are folks for whom a product or service simply worked.
Even Consumer Reports’ reviews are sometimes unhelpful, because their criteria differ substantially from mine — especially on electronics. I do carefully read the auto reviews when I’m in the market for another car, though. But even there, my requirements for an automobile appear to be somewhat out of the mainstream (e.g., economy is much more important to me than acceleration or comfort), and I’ve had good experiences with cars that they have panned.
Most of the dedicated ‘review’ sites that I have seen have turned out to be questionable. Not sure what to do about all that, but I’m open to suggestions.
—
Without the 2nd Amendment, the rest of the document is just wishful thinking.
Generally Worthless
When trying to make a purchasing decision I find myself more frustrated and confused after I read user reviews.
You can just go by star ratings, because, like someone else said, people don’t always rate the product consistent with the review. I’ve seen 1 star ratings where the only complaint in the review was that they thought the price was too high. Similarly, a 5 star rating where the only good thing was that it arrived in a timely manner.
I find it’s much better to find a reviewer you can trust (we’re usually talking big media here) and go with what they say. The trouble is that they can’t possibly review everything.
Netflix Reviews
“User reviews have been around for years now, yet not much has been done since they were first launched to make them more useful.“
Netflix has done a decent job of making the reviews of movies more meaningful by indicating how “similar” the reviewer is to yourself, so when someone loves a film, yet are only 28% similar to you, and someone else hates the same film, but are 72% similar, it’s probably more likely that you’re going to agree with the negative review.
Reviews by verified buyers
Reviews are definitely helpful, but only if you can trust that the people leaving the reviews have actually used the products.
On Buzzillions.com for example, the majority of the product reviews are left by ‘verified buyers.’ Every single vendor listed on Buzzillions is a client to Buzzillions, so that way Buzzillions can verify that the person leaving the review has actually purchased the product.
Another cool thing is that it will show both the pros AND cons about a product, instead of just the generic 1-5 star rating.
Epinions
My personal favorite for reviews is Epinions.com. Usually I’ll look for the negative reviews (1 or 2 stars) because these are usually more realistic, especially when it comes to movies.
But I still look to see if the same complaints are common between them. If not, then I figure the complainers are the type that will gripe about anything.
I’m leery of 5 star reviews because they are usually fluff. But I also read the reviews to see if the content matches the star value. If not, then I ignore that author.
Plus, the forms to fill out for a review pretty much require people to consider both pros and cons so hopefully it will slow them down from 5 star and 1 star posts.
Something else that bugs me about user reviews is when you get to one of those sites that doesn’t require a user to post an actual review in order to rate a product (in which case the overall rating is often quite different than what you find yourself reading). People think nothing of rating something a 4/5 or 5/5 without saying anything helpful about it.
Also, ordinary people are much more likely to post a review when they’ve had problems with something than they are when they really liked it, unless they’ve got a good amount of free time.
These phenomena go for both products and services. I work in a hotel and many weeks we get good overall satisfaction scores, but the only people who actually add comments are those who had a problem.
Re: Re:
Wow. That first sentence was incredibly awkward and wrong, and I apologize to anyone who actually tried to make sense of it.
It should have read “I’m bugged by the sites that don’t require a user…”
Reviews are useful if descriptive
The way that I use a review is that I look for the more descriptive reviews. These give you an idea about the product and made by genuine users. Reading a few of these gives an idea about the product.
A suggestion for a review system is to have ratings for reviews where the reviews are shown sorted by rating. So the best rated review gets read the most and if good enough keeps it’s rating.