How Google Addresses Duplicate Content Due To Scrapers

June 18, 2008

Stumble it!

Enter your email address & Get Updates:

This is a topic that many bloggers and many webmasters ask, I am one of them. Sometimes, I found many websites and blogs copying my content like hell. This is one of the reason why my rss is just summary. In an interesting article, Sven Naumann from Google’s Search Quality Team, helps clarify the issues surrounding duplicate content. From Google Webmaster Central Blog:

Before diving in, I’d like to briefly touch on a concern webmasters often voice: in most cases a webmaster has no influence on third parties that scrape and redistribute content without the webmaster’s consent. We realize that this is not the fault of the affected webmaster, which in turn means that identical content showing up on several sites in itself is not inherently regarded as a violation of our webmaster guidelines. This simply leads to further processes with the intent of determining the original source of the content—something Google is quite good at, as in most cases the original content can be correctly identified, resulting in no negative effects for the site that originated the content.

Generally, we can differentiate between two major scenarios for issues related to duplicate content:

• # Within-your-domain-duplicate-content, i.e. identical content which (often unintentionally) appears in more than one place on your site
• # Cross-domain-duplicate-content, i.e. identical content of your site which appears (again, often unintentionally) on different external sites

Here are some webmaster points of view, read them all carefully and share with us yours:

• I am not sure id agree with that. Why do scrapers still appear above the original site? Why do sites that have their content duplicated find they return to old positions when they get the offending sites to remove the duplicated text? They may get it right a lot of the time, perhaps within parameters they find acceptable, but the above suggests they get right almost all the time, im not so sure that’s accurate. Maybe i should qualify and say theres a big problem IMHO with large chunks of copied content rather than site for site or page for page.
• Yes, there’s a good bit of happy face in the message - but given the amount of scraping that actually exists (have you looked at the amount of bot access that goes on in your server logs?)

  Google does get a big bunch of it right. But the situation is something like killing 99% of the mosquitos in a room and then hoping to take a rest. That 1% is still going to buzz in your ear! As I read the comments, there are three main reasons why the original might be outranked by a scraper:

  1. Unintentionally blocking googlebot from parts of your content
  2. Changes to the content after it was scraped
  3. The original is weakened in Google through some guidelines violations

  That second one is not often talked about - but I assume it can make your page look like the derivative page instead of the original.

• Our biggest problem was (is) with scrapers that take our content and embed it on their site wrapped around their navigation, etc. They make a very good effort to masquerade as a legitimate site, which makes it very difficult for Google. Particularly when they are taking stories from our site on the same day we post them, and posting them on their site.

What is your experience about duplicate content and Google? Does Google detect Duplicate content in seconds? Or should they improve their algorithm a lot more than this?

Want One of the Cheapest and Affordable Hosting?

What Next?

 Subscribe To GoogleLady

 Digg It

 Save This Page

 Sphinn It

 Stumble it!

 Favorite This Post

 

Get Updates In Your Email

Posted in Search Engine Optimization