How to prevent Google from indexing the /wp-content/ directory

Here’s the thing

There are multiple ways to do what you want, from adding meta tags, to passing headers, but because, you tagged your question with robots.txt So i consider it off-topic to discuss any other solutions.

Considering your demand you need to have this as your robots.txt This disallowed access to wp-admin, but depending on use case, you may need ajax, so I gave it an exception, and then disallowed wp-content, and wp-includes.

Change your robots.txt situated in root directory, like this

User-agent: *
Disallow: /wp-admin/
Allow: /wp-admin/admin-ajax.php
Noindex: /wp-content/
Noindex: /wp-includes/

This shall ask google to not index anything in wp-content and wp-includes, Google is slow and it would take its bot some time, to realize he is going at wrong place, so it would eventually remove it from its index.

The below HTACCESS METHOD is completely optional, the Robots.txt can do the trick with work, but HTACCESS is much more consistent method, therefore I couldnt resist myself from telling it

Incase you are willing to edit ht-access file, then that would be a better approach.

I myself use it on my site. My code below if put in htaccess files, blocks all PHP and backend specific files, but allows all images, videos, pdfs and various similar file formats to be indexed by Google and others.

# Serves only static files
RewriteCond %{REQUEST_FILENAME} -f
RewriteRule ^wp-(content|includes)/([^/]+/)*([^/.]+\.)+ (jp(e?g|2)?|png|gif|bmp|ico|css|js|swf|xml|xsl|html?|mp(eg[34])|avi|wav|og[gv]|xlsx?|docx?|pptx?|gz|zip|rar|pdf|xps|7z|[ot]tf|eot|woff2?|svg|od[tsp]|flv|mov)$ - [L]
RewriteRule ^wp-(content|includes|admin/includes)/ - [R=404,L]

Be the first to comment

Leave a Reply

Your email address will not be published.