Computer Program to Spot Fake Reviews?

Computer Program to Spot Fake Reviews?

Acknowledgment to Cornell University ChronicleOnline

As you probably know, opinion and review sites like Yelp and TripAdvisor are littered with fake reviews.  Review sites are regular targets for phony reviews – both positive reviews created by owners and managers and negative reviews to denigrate competitors.

Although many people claim they can spot fakes, recent research by Cornell University showed that in reality we are very poor at differentiating true from false reviews – human judgment proved no better than tossing a coin.

However, the same Cornell researchers have developed a computer program that is much better than humans at differentiating true from false reviews.

The work was reported in June 2011 at the Association for Computational Linguistics in Portland, Ore., by Claire Cardie, professor of computer science, Jeff Hancock, associate professor of communication, and graduate studentsYejin Choi and Myle Ott.

The team employed a group of people to deliberately write 400 false positive reviews of 20 Chicago hotels. These were compared with an equal number of genuine positive reviews for the same hotels.

Human judges – volunteer Cornell undergraduates – scored no better than chance in identifying fake reviews. They did not even agree on which reviews they thought were false, reinforcing that they were doing no better than chance.

According to the research team,  humans suffer from a “truth bias,” and assume reviews to be true until they find evidence to the contrary.  However, when people are trained at detecting deception they tend to become overly sceptical, swinging to far the other way and reporting deception too often, but still scoring no better than chance at telling true from false.

Computer analysis based on the text of known true and false reviews revealed, amongst other things, that truthful hotel reviews were more likely to use concrete words relating to the hotel, like “bathroom,” “check-in” or “price.”  Fakes included more context setting words like “vacation,” “business trip” or “my husband.”

Using this and other text analyses as algorithms, the researchers trained a computer on a set of true and false reviews, then tested it against the rest of the database.  By combining keyword analysis with the ways certain words were combined in pairs deceptive reviews were identified with 90% percent accuracy by the computer program.

Further research needs to be undertaken to see if a similar analysis can be equally successful at spotting true and false negative reviews.

This sort of software might be used by review sites as a “first-round filter,” Ott suggested. If, say, one particular hotel gets a lot of reviews that score as deceptive, the site should investigate further. 

While this is the first study of its kind, and there’s a lot more to be done, I think our approach will eventually help review sites identify and eliminate these fraudulent reviews.

Unfortunately, once everyone knows what the computer program looks for, the fake review writer will also know how to trick it.  Back to square one …

Advertisements

8 responses to “Computer Program to Spot Fake Reviews?

  1. The guy “Jigna Patel”. is just another independent spammer who spams thousands of websites email in the hope of getting on persons business.
    He sent me the same rubbish, telling me my site is not highly ranked in the search engines, even though it gains between 7000 to 10,000 page views a day

    In order to find out more details just reply and say “please send me your company website as I would like to know whom I’m working with”. Then the person will either reply or the person they are working for will reply with their company website. You will realise that the same company have already sent you an email from their official email in the past. Then if you want just post their spam on forums so everyone else can find out how they spam.

    The way guy like that works is not even the tip of the iceberg. Some of the ways they try to get sites high is actually listed by google as illegal and they state they will either lower sites working with them or even blacklist. So always beware whom you arrange your SEO, Webdesign with.

    Down with SEO / Link Exchange spammers

  2. It’s a shame really. Review sites are such a valuable resource. Why does TripAdvisor not take their power more seriously?

    What do you think to this article? http://newmediatravel.com/2011/12/10/battle-of-the-review-giants-holidaycheck-vs-tripadvisor/

    • I’m not sure yet – the number of places reviewed, and the number of reviews, are obviously a lot smaller. Whether the model for reviews will be more reliable in practice remains to be seen. It needs to grow a lot bigger to be a serious TA competitor but if it can then that could be for the good.

      • Have you got your property listed there? Have you had any first hand experience? I’m looking for a viable alternative to TA.

      • I’ve just listed on HolidayCheck in order to find out what it is like – of course, you don’t have any choice whether yo’re listed on TripAdvisor or not! But, my impression is that it is (not surprisingly) heavily geared in favour of places with which it has an affiliation / financial connection.

      • Would be good to hear how it goes once you start receiving reviews on the site and stuff. Let me know how it goes, hadn’t heard of them before I read that article.

  3. 2) We will also assist you in getting more positive reviews on all top portals.

    I think I can guess what that would involve. The review system lends itself abuse.

    Phil

  4. And on the same day as your post, this is what I get in my inbox:

    From: Jigna Patel [mailto:jignacgs@gmail.com]
    Sent: 29 July 2011 10:38
    To:
    Subject: Negative Reviews are killing your reputation.
    Dear Hotel Owner,
    Reviews about your hotel can make or break your hotel’s success.
    If you don’t enough positive reviews on Expedia, Google, Tripadvisor, Yahoo Travel and other such sites, you are losing the game.
    If you have negative reviews about your hotel, you are at serious disadvantage.
    Allow us to manage your reviews:
    1) Our US-based, native English writers will create most effective management response to the negative reviews about your property.
    2) We will also assist you in getting more positive reviews on all top portals.
    3) We will monitor your reputation to ensure that you get more positive reviews.
    Respond today to protect your reputation on Internet.
    Regards,
    Jigna Patel.

    I would let TA know direct (to help them of course!!), but they are still blocking me from using their website . . .
    Prohibition drove drinking underground; don’t those who control the system ever learn?
    Whatever the computer software does, there will be thousands of inventive internet savvy minds finding ways round the problem generated by anonymous reviews.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s