discourse/lib/crawler_detection.rb

module CrawlerDetection
  def self.crawler?(user_agent)
    !/Googlebot|Mediapartners|AdsBot|curl|Twitterbot|facebookexternalhit|bingbot|Baiduspider/.match(user_agent).nil?
  end
end
REFACTOR: Rename `GooglebotDetection` to `CrawlerDetection` because we will likely whitelist more crawlers in the future. 2014-02-20 16:07:02 -05:00			`module CrawlerDetection`
			`def self.crawler?(user_agent)`
Improved crawler detection: add Twitterbot, Facebook, curl, Bing, Baidu. 2014-03-15 20:01:46 +05:30			`!/Googlebot\|Mediapartners\|AdsBot\|curl\|Twitterbot\|facebookexternalhit\|bingbot\|Baiduspider/.match(user_agent).nil?`
Detect Googlebot from user agent and use a different layout that doesn't load javascript 2014-02-14 17:10:08 -05:00			`end`
			`end`