Discourse/discourse
Discourse/discourse
2
0
Fork 0
mirror of https://github.com/discourse/discourse.git synced 2025-09-07 12:02:53 +08:00
Code Packages Activity Actions 5

FEATURE: let Google index pages so it can remove them

Browse source 
Google insists on indexing pages so it can figure out if they
can be removed from the index.

see: https://support.google.com/webmasters/answer/6332384?hl=en

This change ensures the we have special behavior for Googlebot
where we allow indexing, but block the actual indexing via
X-Robots-Tag
This commit is contained in:
Sam Saffron 2020-05-11 12:14:21 +10:00
parent 4a74f18e95
commit bb4e8899c4
No known key found for this signature in database
GPG key ID: B9606168D2FFD9F5
4 changed files with 18 additions and 2 deletions
Download patch file Download diff file Expand all files Collapse all files

8
app/controllers/application_controller.rb
View file

@ -806,7 +806,13 @@ class ApplicationController < ActionController::Base
end
def add_noindex_header
response.headers['X-Robots-Tag'] = 'noindex' if request.get?
if request.get?
if SiteSetting.allow_index_in_robots_txt
response.headers['X-Robots-Tag'] = 'noindex'
else
response.headers['X-Robots-Tag'] = 'noindex, nofollow'
end
end
end
protected