discourse/lib/crawler_detection.rb

module CrawlerDetection
  def self.crawler?(user_agent)
    !/Googlebot|Mediapartners|AdsBot|curl|Twitterbot|facebookexternalhit|bingbot|Baiduspider|ia_archiver/.match(user_agent).nil?
  end
end
REFACTOR: Rename `GooglebotDetection` to `CrawlerDetection` because we will likely whitelist more crawlers in the future. 2014-02-20 16:07:02 -05:00			`module CrawlerDetection`
			`def self.crawler?(user_agent)`
Add archive.org to crawler list to serve no-js to 2014-11-02 16:51:23 -08:00			`!/Googlebot\|Mediapartners\|AdsBot\|curl\|Twitterbot\|facebookexternalhit\|bingbot\|Baiduspider\|ia_archiver/.match(user_agent).nil?`
Detect Googlebot from user agent and use a different layout that doesn't load javascript 2014-02-14 17:10:08 -05:00			`end`
			`end`