How to quickly find and export all subdomains indexed by Google

An SEO audit is rarely limited to the www (or non-www) version of a website. When looking for potential duplicate content, it’s often important to know how many subdomains exist and, more importantly, how many of them are indexed by Google.

The good old search operators

An easy way to find indexed subdomains is to use search operators.

1. Start with “site:” and the root domain.

2. One by one, remove each subdomain (including: “www”) from the results with the “-inurl:” operator.

3. When there are no more results for Google to return, your query with the search operators should include all subdomains indexed.

However, this technique has its limits. It’s unlikely that the site you’re auditing has as many subdomains as wordpress.com, but you may come across a site with several dozen subdomains. This can potentially cause the following issues:

• The process can be long, especially if it needs to be done for several domains.


• You might get Google “captchas” along the way.


• The size of queries is limited (around 30 keywords). Thus, if your query is too long (too many -inurl operators), you will get a 400 error page from Google.


• Once you’re done, you still need some editing to create a nice list of subdomains to work with.

The solution: a simple Chrome extension, by Google

This extension, Personal Blocklist (by Google), will make your life easier. It allows you to “block” domains from appearing in your search results.

The key here is that the extension operates at the subdomain level and stores the domains in a list.

1. Once added to Chrome, start with the same “site:domain.com” search command.


2. Under each result now appears a “Block subdomain.domain.com” link.

3. Click on each link until your result page is empty.

4. You’re almost done! Simply click on the extension icon, then “export,” then copy/paste into Excel.

Enjoy!

[Article on Search Engine Land.]

Some opinions expressed in this article may be those of a guest author and not necessarily Marketing Land. Staff authors are listed here.

About The Author

Max Prin

Max Prin is a Technical SEO Manager at Merkle. As part of the SEO technical team, Max ensures the accuracy, feasibility and scalability of the agency’s technical recommendations. He supports Merkle’s SEO department by developing SEO best practices, tools and processes through research and testing. Max is also in charge of developing and publishing TechnicalSEO.com.