Duplicate content is a problem every webmaster will be confronted with at some point. For Google and other search engines, duplicate content is initially not a negative signal. However, identical and similar content from different URLs can cause confusion and lead to ranking problems.
To avoid duplicate content, you can either make use of canonical tags or 301 redirects. Both methods can be very effective when used in a targeted manner. This article will show when to use which approach.
Duplicate Content Can Affect Any Website
Measures taken for onpage optimization are usually clearly definable for website owners. There are standard recommendations for meta titles or descriptions, as well as clear rules on which heading structure is most suitable for users and search engines. Likewise, a clear keyword focus can significantly affect the position in the SERPs.
However, when it comes to the distinction between the need for a canonical tag, or a redirect, many webmasters and SEOs hit problems. Due to the uncertainty, there is the danger of wrongly implementing one or the other. This may ultimately lead to undesired results and the complete opposite of the planned optimization. For instance, when avoiding duplicate content, but rendering sections of the website not indexable or crawlable.
For that reason, it is critical to clarify the function of redirects and canonicals:
Canonical Tag
Canonical Tags are used to signal search engines the preferred version of a web page to be indexed. If there are two identical web pages with different URLs, the second version will refer to the original web page via canonical tags. Thus, telling Google and other search engines which version is to be indexed. The referring web page will still be crawled.
The canonical tag is implemented into the head section of web page. Alterations at the server level are not necessary. As soon as a crawler visits the web page, it reads the tag and indexes the canonical web page. A canonical tag may also refer to the same web page to prevent plagiarized websites.
Example: www.page.com/a and www.page.com/b are pages with similar content, but different URLs. On page /b, the following canonical tag is implemented:
SEOs use canonical tags when content is being repeated or when unique URLs can not be generated. Good examples would be accessibility via different URLs (www.domain.de, domain.de, www.domain.de/index.html, etc.), accessibility with or without trailing slashes (“/”), or if the content can be displayed in different formats (e.g. printable version, as well as a PDF version, etc.).
301 Redirect
A 301 redirect is a permanent redirect of a web page to a different URL. Usually, users do not notice such redirects. Every 301 redirect passes on link strength of almost 100 percent.
To implement 301 redirects, website owners will need to adjust the .htaccess file, as permanent redirects need to be deposited onto the server. At first, a browser or crawler sends a URL request to the server. The server then matches the request to the .htaccess file and redirects the URL to a different URL, given that there is a relevant listing.
301 redirects can be used in different ways. Usually, they are utilized when URLs change and user-friendliness would suffer due to error pages. Likewise, the 301 redirect can be used to avoid duplicate content.
Example:
An entry in the .htaccess file could look like this if the website http://page.com is to be permanently redirected to http://www.page.com.
301 Redirect and Canonical Tag in Action
To make use and differentiation of canonical tags and 301 redirects more comprehensible, here are some examples.
Scenario 1: Similar Products with Similar Descriptions in an Online Shop
This is an issue that many online shop owners and operators face. Given the vast inventory of many online stores, there are many similar or almost identical products (e.g. same color or model number). Hence, these products will have similar product descriptions but will be indexed with differing URLs.
By having an indication of the product in the URL, users know what to expect when clicking on the link. On the other hand, multiple URLs for similar products also lead to duplicate content in search engines.
The solution: To avoid duplicate content, it is important that only one URL is indexed by Google. The easiest way to do this is to use the canonical tag. This way, Google knows, which URL to index.
Example: There are two websites for similar products
myshop.com/shoe1-yellow and myshop.com/shoe1-blue
If Google should index only one URL and define /shoe1-yellow as the original, then the following canonical tag must be inserted into the section on the /shoe1-blue website:
Important: If possible, use only complete URLs within the canonical tag.
Scenario 2: A New Version of a Web Page Is Launched
You run a blog and have written an article that is now being updated and renewed. As a result, not only does the content change but the URL as well. To prevent losing existing traffic to the old article, the old URL is ideally redirected to the new one. The link strength will be almost entirely passed on. That is especially important if the first article has frequently been linked to by other websites.
The solution: A 301 redirect will securely redirect the old URL to the new one without creating any disadvantages. It is advisable to drive traffic through fresh links (internal and backlinks) to the new URL as well, so that search engines can replace the new URL with the old URL as soon as possible.
Scenario 3: Conversion of the Website from HTTP to HTTPs
Many websites are being converted to the encrypted https version. This increases the user’s security, and the change may give webmasters a ranking benefit. However, this conversion can lead to duplicate content, as Google indexes both the http and the https version.
To avoid this problem, a 301 redirect can be used once again. The http version of the website will be redirected entirely to the https version. Important: The permanent redirects may increase the loading time significantly for large sites.
- The solution for smaller websites: All web pages are redirected to the https version via the 301 redirect.
- The solution for very large websites: The rel=canonical refers to the https version as the original URL. This way, crawlers are encouraged to index the https version.
Scenario 4: A Product is No Longer Carried in the Online Shop
Product offerings in online shops regularly change. A product might be sold out, or is removed for other reasons. If the URL is deleted upon removal of a product, the user experience suffers as the server can no longer find the requested URL, and shows an error page instead.
The solution: To avoid weakening the user experience, a 301 redirect to a new URL may be helpful. That could be a redirect to the main category, the newer version of the product, or a similar product.
Yet, if a product is sold out temporarily, a 302-redirect could also be used. It notifies the user that the redirect is only temporary.
Suggestion: If you wish to deliberately display an error page, you should optimize your 404 page. When a user reaches this page, he not only receives an error message but also receives information on which products he can find instead. It is also possible to integrate a search function into the 404 error page.
Figure 1: Search function on a 404 error page at Airbnb.
Scenario 5: Relaunch of a Website with an URL Alteration
If a website is at the top of search engine rankings before its relaunch, website operators will want to maintain its position. However, that should not be taken for granted. If the new site is simply launched without having made any preparations, old URLs are no longer accessible. A search engine indexes the exact URL of an individual web page, and if the structure of the website is changed, almost none of the web pages will keep their URL.
The solution for the domain transfer: The old domain URL can be forwarded to the new domain URL via a 301 redirect. This way, users can be led directly to the new URL, without having to deal with error pages, or receiving browser messages that the complete website is unavailable. It is important to ensure all pages are correctly redirected.
This is why all of the “old” URLs of the website must be captured before the relaunch: To assign them to the new domain. It is also important to continuously monitor the redirects after the relocation. If necessary, webmasters can quickly react and minimize ranking losses.
Conclusion
The canonical tag and the permanent 301 redirect are powerful tools that are not only useful to avoid duplicate content but can also be used to improve the user experience. Knowing when to use which, will ensure user satisfaction and positive results in search engines.
Image Credits
Featured Image: Image by OnPage.org. Used with permission.
All screenshots by Irina Hey. Taken September 2016.
(68)
