When Google’s “Panda” Algorithm Collides with Duplicate Content

If you’re a Webmaster, it’s likely you’ve heard about the Google latest search algorithm — “Panda” — and all the benefits and implications of this update. Today, we wanted highlight what happens when Google Panda collides online with duplicate content. There have been plenty of opinions written about Google Panda and duplicate content, but we want to provide some background and examples to help you better understand how Panda and duplicate content might affect you.

What is Duplicate Content?

Duplicate content is a term used in the field of search engine optimization to describe content that appears on more than one web page, within the same web site. When multiple pages within a web site contain essentially the same content, search engines such as Google can penalize/not display that site in any relevant search results.

Should you be Concerned?

When Google released Panda, there was a significant outcry from legitimate business and publishers who were either downgraded overnight in their search engine page rank or dropped all together. For many of the businesses, the Panda algorithm reduced SEO rank and decreased visitors, site revenue and online market awareness. Some websites even experienced damage to their brand, as their customers and prospects questioned whether they were still in business.

We’ve spoken with Cult of Mac, Digital Trends and several Fortune 1000 businesses, and they’ve all said the same thing: They were penalized and downgraded as a result of the Panda release as a result of unauthorized duplication of their content. They had done everything to comply with Google in optimizing their SEO configurations, but the third-party websites scraping and duplicating their content (outside of their control) caused their page ranks to fall.

Google’s Official Stance on Duplicate Content:

“We do a good job of choosing a version of the content to show in our search results.”

“In rare situations, our algorithm may select a URL from an external site that is hosting your content without your permission. If you believe that another site is duplicating your content in violation of copyright law, you may contact the site’s host to request removal. In addition, you can request that Google remove the infringing page from our search results by filing a request under the Digital Millennium Copyright Act.”

http://support.google.com/webmasters/bin/answer.py?hl=en&answer=66359

Where is This “External” Duplicate Content Coming From?

Sometimes, it’s not clear how third-party sites obtain copies of legitimate work. Typically, they either steal it by manually or automatically scraping the content. The scraped content is then republished onto their sites, providing no credit or link to the original work.

What does that look like? It’s not difficult to find examples, but I tracked one down that seemed particularly ironic. Here’s an original article by PC World on Google’s War Against Scraper Sites:

Here’s a duplicate copy of the same story that doesn’t give any credit to the original PC World article:

It’s clear that we’re not looking at a coincidence here. The title, article content and images are all identical. The scraping site didn’t even attempt to mask their plagiarism with synonym changes. Why would they do that? Just take a look at the ads on the scraper site … They want to profit from the keywords and traffic driven by PC World’s content.

What Can You Do About It?

• Listen to Google
  Google provides a list of tips for using rel=”nofollow” and canonicalization to ensure they are able to identify you as the original author of content and avoid penalizing or downgrading your business’s search ranking results.
• Learn About DMCA and Use It
  If your content has already been duplicated by unauthorized publishers, you should learn more about the Digital Millennium Copyright Act (DMCA) and how it can help get help remove your content from infringing websites. Two helpful resources to start learning the law and your rights are Google’s official DMCA policy page and the United States Copyright Office.
• Be Proactive About Stopping Scrapers
  We believe the best solution is to implement practices and or services to proactively prevent people or web scrapers from harvesting or scraping your content in the first place. Although web scrapers can be difficult to detect, there are tactics and/or services that can be implemented to limit certain behaviors on your website(s). Some of the quickest ways to make strides in the right direction are to implement rate limiting rules, to block traffic from blacklisted IP addresses and to use Captcha to help reduce automated web scrapers.

While none of these tactics are fool-proof ways to completely prevent your content from being duplicated, the more barriers to entry you have, the more difficult it will be for web scrapers to repeatedly duplicate your content. Distil built an enterprise-ready platform to monitor and prevent site scraping, so if you want some help in the protecting your content, try our our service. Whatever route you take, the key is to make sure that whatever tactics or services you implement, you don’t forget about your legitimate traffic … You don’t want to throw out the baby with the bathwater. Be proactive, but make sure you keep your priorities on the user-experience and quality of your site(s).

-Sean Harmer, Distil

DDoS Protection: Do you need it?

In the last couple of years, you’ve probably seen a references to denial-of-service attacks in the news and how the fallout from those attacks can leave businesses ‘picking up pieces’ for weeks or months after they occur. Think about the helplessness you’d feel if the business you poured your heart and soul into is shut down by some malicious person or group’s attack on your web presence. Worse yet, those attacks are usually for that person’s or group’s own monetary gain or to satisfy some ego-driven urge to punish you for being successful in either your business or a cause you believe in.

It happens all too often, and most people don’t realize that it can actually happen to them. On a weekly basis, I speak to at least one person that tells me, “We’re small, and we really don’t have any competitors … Our website is down. If we can’t stop this attack, I am going to have to send all of my employees home and close down!”

The truth is that denial-of-service protection providers normally sell “fear.” They do this because people don’t have answers to a few key questions about DDoS protection:

• What are the real statistics?
• What is the probability that my website will be hit with an attack?
• What is the real cost of impact?
• What about my data center? I’m sure they already have protection, right?
• We’ve never been hit before, so why should we consider it a priority?

Many of the causes for hesitation regarding the purchase of denial-of-service protection revolve around the lack of education and valid, statistical data. Most know about Distributed-Denial-of-Service (DDoS), but the details are hard to come by. Most people don’t have experience with attacks, so many assume it’s only the big companies or governments that need to worry.

The untold truth is that DDoS attacks occur on a daily basis, and as many as 2500 attacks occur every 24-hour period throughout the world. In the first 6 months of 2011, ServerOrigin saw 2.3 times the number of attacks we observed and mitigated over the course of the 2010 calendar year.

Let’s say you’re considering protection, and you want to ensure that you remain in control and your business continues operating even if you’re the target of extortion or a collective political or religious movement … and that’s assuming there is a reason behind the attack. Over 86% of attacks occur with no explanation! Considering that statistical tidbit, maybe fear isn’t just a manufactured, marketing gimmick to get you buying.

One of the biggest roadblocks in proactive DDoS mitigation in the past has been cost. The average cost for 12 months of a DDoS mitigation appliance or service to protect 1000Mbps is $78,942.00 – just for the equipment/service. Then you have to factor in the variable cost of the bandwidth USED during an attack.

ServerOrigin created ethProxy as a service that overlays your current server platform at an affordable price point. SoftLayer provides one of the best dedicated hosting environments, and we’ve built our reputation on DDoS protection, not hosting, so we bring our service to you.

How do I get protected? How time consuming is this process?
Contrary to the belief that DDoS mitigation is some mystical technology that is painful to implement, our ethProxy mitigation service works on the same premise as a Global Load Balancer or reverse-proxy. Setup is as simple as changing your website’s DNS record to a protected IP in our ethProxy filtering cloud, and once that is done, inbound connections are filtered so only clean traffic is sent to your web server. Transition to the ethProxy service is transparent for the end-user and requires no downtime to implement.

The average deployment time is less than 1-hour and the ethProxy protected IP becomes your public interface to the world. Not only does our service pass rigorous PCI certifications, it guarantees your hosted infrastructure is no longer vulnerable to attack, it allows for upgrading your bandwidth/protection in seconds, and it removes the need for additional web application firewalls or accelerators. On average, customers save around 71% by going with ethProxy when they compare us against the cost of traditional filtering methods.

Our ethProxy service is the combination of many different features or services that you may already pay for separately. This allows businesses to transition to our protection service by replacing one of their current providers which would be prove redundant with the ethProxy subscription. Why go through a budgeting process again when you can simply use a different provider that offers you DDoS protection in addition to the service you’re already paying for?

Included in Every ethProxy DDoS Mitigation Package

• Global Load Balancing
• DDoS Protected AnyCast DNS Services
• Multiple US locations / Complete Datacenter Redundancy
• Instant Scalability – Powered by ServerOrigin’s Cloud Network
• Global Content Delivery Network (CDN)

The options and overall value of these services provide protection that no website should be without, while saving you a ton of money … Especially when you consider that running all of these services separately could cost as much as $10,000/mo.

ServerOrigin Communications services more than 1,200,000 million domains worldwide. Our ethProxy service is the single largest globally deployed mitigation service worldwide, and we protect everyone from non-profit organizations to entire governments and some of Wall Street’s largest online providers (No, we’re not allowed to tell you which ones).

Let us show you why our expertise has saved hundreds of businesses and how we can ensure you never have to ‘pick up the pieces.’

- Kevin Hatfield, ServerOrigin Communications

Anti-virus:

This is a must on systems open to the net now days. There are always nasty little things floating around looking to take your server apart from the OS out. For windows servers there are a multitude of choices and I’ll just mention a few that can help protect your goods. You can use several programs such as avast (which offers a free edition), ClamWin (open source), Kaspersky , and Panda just to name a few. I would suggest before installing any of these you check links such as http://en.wikipedia.org/wiki/List_of_antivirus_software to name one that provides a list of several choices and their compatibility. You may also want to read reviews that compare the available options and give you an idea of what to expect from them. This will allow you to make an informed choice on which one works best for you. Now with linux there are also several options for this including the well known clamav which from personal experience works really well and can be installed on a variety of linux disro’s(aka distributions). It’s very simple to use and may prevent you from headache later on down the road.

Firewalls:

Firewalls are a double edged sword but are most defiantly needed. When it comes to firewalls you can protect yourself from quite a bit of headache however if setup to strict you can block positive traffic and even yourself from reaching your server but in the long run a defiant way to help protect your server from unwanted visitors. A lot of firewalls also have modules and add-ons that further assist in protecting you and securing your server. If in doubt it’s always a good idea to have a security company do an audit and even a security hardening session with your server to make sure you are protected the best way possible.

Passwords:

This is probably one of the most important this you can do to secure your server. Use strong passwords (no using password or jello is not a secure password even if it is in all caps) and if you are worried about not being able to come up with a secure one there are several password generators on the web that can come up with secure ones to assist. Passwords should contain caps letters, numbers, symbols, and should be at minimum 8 – 10 characters (the more the better). It’s the easy to remember and easy read passwords that get you into trouble.

Armed with this information and so much more on security that can be located on the web using the great and all powerful Google should be a good start to making sure you don’t have to worry about data loss and system hacks. Also remember no matter how secure you think you are make regular backups of all your important data as if you server could crash at any time.