Looking for feedback: Ratings, Reviews, Quick Reviews-- Oh, my!

 
Start a
Jolie Chicago, IL posts: 177
 Community Manager
posted on October 29, 2008 at 09:47AM Inappropriate? Quote Reply
Hi all.

We wanted to get your feedback on a change we are considering....

As you all know, we are dealing with a challenge around ratings/reviews/quick reviews naming. People are kind of confused by ratings & quick reviews and how they are different from regular reviews. They aren't certain what constitutes a rating/quick reviews and how to upgrade it, etc.

We want to clean this up, but we are struggling with the best way to do it. So, we thought we would come to you, our trusty community!

Here is something we are considering... what do you think?

  • Make the move to call a review, a review.... No matter what form it takes- short or long, it is a review. Most people on the site don't understand the difference. So, why call it something different? This will make it easier for readers to understand.
  • Sort the reviews by completeness.... So that the most comprehensive reviews come to the top. However, if reviews with less information get lots of helpful votes, they can move higher up on the page.
  • Only count certain reviews toward your reputation. In order for a review to count toward your reputation, it must have: a soundbite, a rating, I AM tags, and at least 250 characters of text. So, for example, if you and I both wrote 25 reviews, but yours were ‘complete' and mine weren't, you would have status as Writer 1, but I would not. And, your reviews would appear above mine.
  • Lastly, we would add a filter to the reviews tab... allowing you to sort by quick reviews. This will allow you to upgrade any review that doesn't count toward your status.

What do you guys think?  Are we crazy?

replies: 60 latest post: November 06, 2008 at 11:18PM by vivasuzi
2009 VIP
posted on October 29, 2008 at 10:03AM
 

I REALLY like the idea of being able to filter and sort by quick reviews, especially from my home page.

I like the idea of calling a rating a review.

How do you currently sort the reviews? What comes to the top currently?

Would pros/cons be considered more complete if both are listed? For example, if I review a product and cannot find anything negative to say, would the review appear more complete if I listed "I cannot think of any" under cons?

If you are sorting the reviews by completeness, does this mean that you would give priority to those who list higher numbers under each category? For example, if ten people have "complete" reviews, would someone who lists 25 "I am" tags be placed higher? Or would you have another way of determining placement?

I would really like to see a pop-up when I am ready to publish a review that would ask if I wanted to add elements such as an "I am" tag or pros/cons if I forgot to put those in before I publish a review.

2009 VIP
posted on October 29, 2008 at 10:07AM
 
I like it!  I think this system will allow the difference to work itself out.  Thanks, Angela
 Moderator
posted on October 29, 2008 at 10:32AM
 

I like those changes a lot. The only suggestion I have is that reviews that count toward your reputation should have 500 characters, not 250. If you consider that the average word has five characters, a 250-character review has 50 words, or about three sentences. I sure wouldn't want my reputation built on reviews of three sentences.

Otherwise, all your other suggestions are excellent!

--Bob

2009 VIP
posted on October 29, 2008 at 12:51PM
 
  1. I love this idea  In order for a review to count toward your reputation, it must have: a soundbite, a rating, I AM tags, and at least 250 characters of text. So, for example, if you and I both wrote 25 reviews, but yours were ‘complete' and mine weren't, you would have status as Writer 1, but I would not. And, your reviews would appear above mine.  However now that I see Bob did the numbers that's not right. Did you mean words?  
  2. Sort the reviews by completeness.... So that the most comprehensive reviews come to the top. However, if reviews with less information get lots of helpful votes, they can move higher up on the page.  I like the completeness but an "incomplete" review can move up higher for which I won't go into here but it is a concern of mine.
  3. Lastly, we would add a filter to the reviews tab... allowing you to sort by quick reviews. This will allow you to upgrade any review that doesn't count toward your status.  So would this bring us back to the rating review or just be an internal thing?
  4. I can't get rid of the 1:)
 Community Manager
posted on October 29, 2008 at 02:13PM
 
In response to Katrena's post from October 29 2008 10:03AM
Completeness would simply be if they filled out the 4 key fields... not be the volume in each... make sense?
2009 VIP
posted on October 29, 2008 at 03:09PM
 
I feel dumb for asking this, but.....what is a soundbite?
 Moderator
posted on October 29, 2008 at 03:34PM
 
In response to jasyjen's post from October 29 2008 03:09PM
jasyjen said…
I feel dumb for asking this, but.....what is a soundbite?

That's the title of your review.

--Bob

2009 VIP
posted on October 29, 2008 at 04:26PM
 
I like all of those ideas, Meredith.
2008 VIP
posted on October 29, 2008 at 05:25PM
 
In response to Jolie's post from October 29 2008 09:47AM

edit.....i took out this part about completeness after I see Jolie's next statement that it's about having everything filled in, not about length......

My idea would be to make a clear distinction between a Review and a Rating.  Either "rate this" or "review this".  Because my problem has been that I am trying to do a full review but I forget something, like I forgert the I Am Tags, and then suddenly my review isn't counted in my main count and I have to go find the one I messed up.  If there were 2 distinct links/choices, then you could require everything in a review and I wouldn't have to worry about forgetting anything.

2008 VIP
posted on October 29, 2008 at 05:30PM
 
In response to Jo's post from October 29 2008 12:51PM
I like that the incomplete reviews can move up.  I look at it this way, a "incomplete" review would need probably a lot of Helpful votes in order to appear up top, which means it must be really helpful.  Second, it gives everyone of every writing status a chance to rise to the top - equal opportunity.
2009 Advisor
posted on October 29, 2008 at 05:34PM
 
In response to Jolie's post from October 29 2008 09:47AM
I think this sounds like a good way to make the distinction, and that it will be easier for newer writers to understand it without having to do any research. 
2009 VIP
posted on October 29, 2008 at 09:56PM
 
In response to Jolie's post from October 29 2008 02:13PM
Jolie said…
Completeness would simply be if they filled out the 4 key fields... not be the volume in each... make sense?
Thanks for clarification on that.
 Community Manager
posted on October 29, 2008 at 10:58PM
 
In response to vivasuzi's post from October 29 2008 05:25PM
You would still be able to sort reviews and quick reviews... so you know directly which ones need to be updated.
2009 Advisor
posted on October 29, 2008 at 11:11PM
 
In response to bkovacs's post from October 29 2008 10:32AM
bkovacs said…

I like those changes a lot. The only suggestion I have is that reviews that count toward your reputation should have 500 characters, not 250. If you consider that the average word has five characters, a 250-character review has 50 words, or about three sentences. I sure wouldn't want my reputation built on reviews of three sentences.

Otherwise, all your other suggestions are excellent!

--Bob


I'm with you on this Bob, I like all the suggestions with the caveat you mentioned.  I really don't think someone should move up in reputation writing tiny reviews that really tell us nothing.  Building your reputation should take some effort, right?
2008 VIP
posted on October 30, 2008 at 08:10AM
 
In response to Jolie's post from October 29 2008 10:58PM
That's fine, but I'd rather have it right the first time rather than having to go back and fix them.
 Moderator
posted on October 30, 2008 at 11:09AM
 
In response to Jolie's post from October 29 2008 09:47AM

I like most of the new changes to the site, but I do not like the Quick reviews, I personally do not find them useful at all.  One or two sentences on a product do not a useful review make.  In order to make an informed buying decision on a given product I need more information about it other than "I like this product, it worked really well," or "This thing sucked, don't buy it." 

I think in order for a review to be labeled as such, some time and consideration should given to the product and or service being reviewed, and that means at least 500 characters.  And each review should include a sound-bite, photos (if they can be found), pros, cons, and "I AM" tags.  These things to me show a commitment to the community and the wiliness to share your knowledge and experience about the product.  

And if Viewpoints is going to continue to allow the posting of Quick Reviews, they should not be called reviews, nor should they count towards review totals, or be considered in the awarding of "Writer" titles.  And they should be at the bottom of the sort list on any given product or service being reviewed.         

2009 Writer
posted on October 30, 2008 at 11:12AM
 
I would like to see more of a distinction between ratings and reviews on the product listing itself.  Say there are 10 total listings and 5 are "complete" reviews and 5 are ratings, it would be nice for it to say that at the top instead of saying there are 10 reviews.  Also maybe a way to list the ratings as separated from the reviews... maybe at the bottom of the listing of reviews (if that makes sense.)  Like a "reviews" section and then at the bottom (or anywhere else on the page) a "ratings" section.
2008 VIP
posted on October 30, 2008 at 01:12PM
 
In response to lilsquibb's post from October 30 2008 11:12AM

lilsquibb said…
I would like to see more of a distinction between ratings and reviews on the product listing itself.  Say there are 10 total listings and 5 are "complete" reviews and 5 are ratings, it would be nice for it to say that at the top instead of saying there are 10 reviews.  Also maybe a way to list the ratings as separated from the reviews... maybe at the bottom of the listing of reviews (if that makes sense.)  Like a "reviews" section and then at the bottom (or anywhere else on the page) a "ratings" section.


I agree with this. I would like to see reviews differentiated from ratings on the product pages - whether you call them ratings or reviews.  

In fact, if you have a section for ratings, the heading also serves to define the distinction for users (such as,  "Ratings - Quick Reviews"  or  "Reviews - Ratings Only") etc. 

I don't think 250 characters is nearly enough as a review minimum if you are working toward a Writer designation.  It isn't too difficult to come up with a 200 word review, in my opinion. 

Writers could still write ratings and super short reviews, but it doesn't seem that it should count towards writer status.  Otherwise, it brings down the value of the honor if you can make it to Writer 4 by writing 200 3-sentence reviews.  JMHO

 

2008 VIP
posted on October 30, 2008 at 01:17PM
 
I'm a little at a loss. So if a person puts a name of a place or product in the system. A one word title like Great! and he gets credit for "writing" a review?  I don't find those helpful.  I can't even figure out what they are telling me about many times.

What exactly are you looking to get? A bunch of new listings in the database?  I'm afraid I don't understand. ;>
 Community Manager
posted on October 30, 2008 at 02:39PM
 

We want to encourage anyone to participate on the site. The problem is for readers.... they dont understand all this fancy language-- ratings, reviews, quick reviews. So, for their sake, we just figured we could call everything a "review" but only those with 250 words (or 250 characters) would actually count toward your status. Also, we would always default to showing the more compelte reviews first... so that the reader always sees the good stuff first. Im not certain a reader cares what something is called. They just want to read the reviews and pray that they get the good ones first. Make sense?

2009 VIP
posted on October 30, 2008 at 03:16PM
 
In response to bkovacs's post from October 29 2008 03:34PM
I told you I felt dumb!  LOL  Thanks.
2008 VIP
posted on October 30, 2008 at 04:18PM
 
In response to Jolie's post from October 30 2008 02:39PM
No it doesn't, to be frank. The one word or three word entry fills up one more space in your data base. Is that it? But I can't open it because there is nothing there except the title.  I'm in agreement with the members who stated the 250 WORD (not character) minimum to be counted as a review. I think the review ought to cover the who, what, why, when, and how about whatever it's about also.  If you demand a little quality, people will rise to the occasion.  JMTC
2009 VIP
posted on October 30, 2008 at 05:09PM
 
I agree with you George but as I wrote on another thread (I think I started it) we were looking for a GPS and among the dozens of places I went the reviews I found myself going to and using to make my decision could have been 2000 words long but I just read the first and last paragraph. Everything in between didn't mean a lot to me NOT because I couldn't understand it but because we wanted an easy to use, basic, inexpensive GPS system. I do think that there is a place for more detailed reviews and if someone is into the thing they are looking at (let's say I was going to buy a piece of fitness equipment) and knows a lot about it then they would have to have a lot more detail then what I was looking for.

I guess what Viewpoints wants is both. Jolie I recently read your most recent review and I noted that it was much shorter i.e. a paragraph that what you have written. Perhaps that was done intentionally; perhaps not. I am creeping into doing a paragraph on our DVD and my husband's cell phone for those who want to know - do I like it and why or don't I? I'm wayyyy OT here ...sorry.
 Moderator
posted on October 30, 2008 at 09:39PM
 
 

George you and I are in agreement here (don't faint); and Jo, I agree the long reviews about the meat and potatoes of a product may not be helpful to some, but are to others.  If what was after here at Viewpoints is a body of (quality) reviews that not only inform, but are written by everyday people, then we cannot be afraid of getting detailed. 

I state again that I believe the Quick Reviews are not informative and like George alluded to they take up valuable database space.  I think I understand why they might be appealing, but I think you would get the same result from longer more detailed reviews.  After all one of the purposes of Viewpoints is to make it feel like a community, and it does. 

The reviewers who post here regularly all know each other-more or less-and that is what makes Viewpoints so enjoyable.  I'm afraid the Quick Reviews will attract those who are not really vested in the site, or its mission, and what you may end up with is a plenitude of Quick Reviews by people who have little interest in the site as a whole.  More comprehensive reviews demand more time at the site, and more investment in its mission and ultimately its success.  

2008 Advisor
posted on October 30, 2008 at 10:13PM
 
As a reader of reviews, I hate to open a review and find "I hate this", or "it's garbage".  I really do value the opinion of the reviews posted on VP.  I recently needed a new dishwasher and read Rudixeno's guide to buying a new one, and then went and searched for dishwashers.  Some of the reviews were no help whatsoever and others were quite helpful.  What I am getting at is the quick reviews just give someone an opportunity to post and run.  I don't believe they care about the quality of their review; they more or less just get it off their chests.  The 250 word minimum is a good way to deter some of these bootleg reviews.  I think boosting the minimum to 500 would be too much because as Jo said, some reviews just shouldn't be long. 
2009 Advisor
posted on October 30, 2008 at 10:18PM
 
In response to TheBard's post from October 30 2008 09:39PM
TheBard said…
 

George you and I are in agreement here (don't faint); and Jo, I agree the long reviews about the meat and potatoes of a product may not be helpful to some, but are to others.  If what was after here at Viewpoints is a body of (quality) reviews that not only inform, but are written by everyday people, then we cannot be afraid of getting detailed. 

I state again that I believe the Quick Reviews are not informative and like George alluded to they take up valuable database space.  I think I understand why they might be appealing, but I think you would get the same result from longer more detailed reviews.  After all one of the purposes of Viewpoints is to make it feel like a community, and it does. 

The reviewers who post here regularly all know each other-more or less-and that is what makes Viewpoints so enjoyable.  I'm afraid the Quick Reviews will attract those who are not really vested in the site, or its mission, and what you may end up with is a plenitude of Quick Reviews by people who have little interest in the site as a whole.  More comprehensive reviews demand more time at the site, and more investment in its mission and ultimately its success.  


What he said.

I surely do understand the "takin' all comers" approach, but, if I'm an outsider looking for a good review and find a lot of stuff of dubious quality, then I'm turned off to that site.   Look, there should be NO elitism here.  But, nothing works without a workable standard.  To write a review and have it called a review, it should meet a standard higher than a couple of sentences, pros cons, stars and I am tags.   Mine tend to be a little long and I'm not advocating that.  Just that the review should tell a reader

A: What the product is, in some detail.
B: An informed opinon as to the quality of it, more than "it's great!" or "it sucks!"
C: Does the reviewer recommend it?

I really don't like the Quick Reviews and can never see myself writing one, for better or worse :)

Everyone should write in their own style and feel comfortable here.  However, if it looks like a duck but meows like a kitten, it's not really a duck.
2009 VIP
posted on October 30, 2008 at 10:24PM
 

I am following many of you on this.  It could be helpful to rate -- just stars and have it clearly called a rating -- but in many cases I don't really see the need for ratings when you can have quality reviews.  Mike sums it all up as far as my opinion on quick reviews I think it's easy to write a quality review and keep it short and sweet (400 words) and give a person enough information to make an informed product decision.

As Mike said, the short two sentences of I really love this product don't reflect well and to me sometimes I feel like it jeapordizes the credibility of the site.  I don't want elitism either, but there should be some form of guidelines for a review to count as a review.   

 

2009 Advisor
posted on October 31, 2008 at 02:25AM
 
I'm in agreement with what everybody has been saying about the Quick Reviews. I haven't tried a Quick Review yet and to be honest I probably won't do one. The short sentences like "This is a great product" or "I don't like it" just isn't a review in my opinion. I think a review should have some requirements to be considered a review. A good solid review should be a minimum of 250 words (not characters) and include the usual soundbite, I AM tags, stars, pros, and cons. 
 Staff
posted on October 31, 2008 at 07:30AM
 

I hear everyone on their preference for longer reviews.  I agree with you that in general a longer review with more detail is more helpful.  However, as we look to engage more people to provide insight into their personal experiences and as we attract more readers we are finding that not everyone has the time or inclination to either write or read the complete review.  Ideally we will be able to design the interface so that it highlights the more complete reviews while still "backfilling" with the shorter reviews. 

Does that make sense to all of you?

2008 Advisor
posted on October 31, 2008 at 09:04AM
 
I get what you are saying....how about keeping the quick reviews but don't allow the person to write the review.  Instead of using tags is there a way to use a button/matrix instead....the person can enter the title of the review but they won't be able to write the review...they would have to click on pre managed buttons/matrix to review the product...ie..have buttons like: Not so good product or Awesome Product.  Automate the Quick Review System.  When people are looking for Quick Reviews you can then distinguish the two by putting that a Quick Review is Automated and does not contain detailed information.  Maybe then you can keep both types of reviews.  Also, I don't know if it's possible, but have Quick Reviews delete automatically after 6 months, especially if the writer has no furher contributions.
1  2  
Buttons_next
 
Your Comment
 

General Discussions

displaying 5 of 771 discussions
displaying 1 to 5 of 771
Sticky sticky topic Locked locked topic New Posts new posts
Topic Author Views Replies Latest Post
D
Sticky Highest Rated Reviews Feed
Meredith 40 1
Meredith »
D
Sticky Sweepstakes Winners
Meredith 1773 101
SpokaneMan »
D
Sticky Viewpoints needs your VOTE now!
Meredith 333 33
Vetsy »
D
Sticky Let's talk about Status badges and what you want to follow you!
Meredith 1024 104
Reviewer11 »
D
Sticky Locked Daily Dose of Review Ideas
Meredith 1331 59
Meredith »

Start a New Discussion