mirror of
https://github.com/discourse/discourse.git
synced 2026-08-09 21:45:25 +08:00
2cf226d90fPreviously, `CrawlerScorer` relied only on bot-positive heuristics (automation user agents, ASN, velocity, churn, referrers), so sessions with clear human interaction could still be scored as crawlers. This change joins session engagement into the scorer as a `-40` discount when a session has any trusted interaction, and hardens the signal against spoofing by not tracking automation-driven browsers (`navigator.webdriver`, Cypress, etc.), counting only trusted events, and returning the velocity tiers.
17 lines
452 B
Ruby
Vendored
17 lines
452 B
Ruby
Vendored
# frozen_string_literal: true
|
|
|
|
module Jobs
|
|
class DetectCrawlerPageviews < ::Jobs::Scheduled
|
|
every 10.minutes
|
|
|
|
LOOKBACK = 1.hour
|
|
WINDOW_DELAY = BrowserPageviewSessionEngagement::BEACON_SETTLE_PERIOD
|
|
|
|
def execute(args)
|
|
return if !SiteSetting.experimental_detect_crawler_pageviews
|
|
|
|
window_end = Time.now - WINDOW_DELAY
|
|
CrawlerScorer.score!(window_start: window_end - LOOKBACK, window_end: window_end)
|
|
end
|
|
end
|
|
end
|
