mirror of
https://github.com/discourse/discourse.git
synced 2026-08-05 19:38:04 +08:00
437ab337d2This commit adds two new cards to the redesigned admin dashboard's Site
Traffic section: top countries and top referrers, both sourced from
`browser_pageview_events`.
Key technical decisions:
1. Gate the cards on the `persist_browser_pageview_events` site setting.
The cards have no data source unless browser pageview events are being
persisted, so they are omitted from the dashboard.
2. Normalize referrers at write time. A new `normalized_referrer` column
on `browser_pageview_events` is populated by
`BrowserPageviewReferrerInspector`, which strips scheme, `www.`, port,
fragment, trailing slashes, and common tracking query params. Doing this
at insert time avoids per-row string operations at query time.
3. Count browser pageviews by country and by referrer in two new report
concerns. `Reports::TopCountriesByBrowserPageviews` groups by
`country_code` and `Reports::TopReferrersByBrowserPageviews` groups by
`normalized_referrer`. Both compute share of total browser pageviews and
rank the top 5 in SQL. The country report drops MaxMind reserved codes
(unknown, anonymous proxy, satellite). The referrer report drops
same-host referrals. Both also exclude anonymous browser pageviews
(`user_id IS NULL`) when the `login_required` site setting is enabled,
since only logged-in browser pageviews are meaningful on a closed forum.
4. Fetch each report through the existing dashboard service.
`AdminDashboardSiteTraffic#build` returns one entry per card with a `{
rows:, error: }` shape, e.g.:
```ruby
{
top_countries: {
rows: [
{ country_code: "US", count: 142, percent: 35 },
{ country_code: "GB", count: 89, percent: 22 }
],
error: nil
},
top_referrers: {
rows: [
{ normalized_referrer: "news.ycombinator.com/item?id=1", count: 47,
percent: 12 },
{ normalized_referrer: "reddit.com/r/discourse", count: 31, percent: 8 }
],
error: nil
}
}
```
On report failure, `rows: []` and `error: :timeout` (or another symbol).
This lets the UI render rows, error, or empty state independently.
Healthy responses are cached via `Report.find_cached`.
`SiteSetting.login_required` and `Discourse.current_hostname` flow into
`opts[:filters]` so toggling either invalidates the cache. Timeouts skip
the cache so the next request retries.
5. Use `Intl.DisplayNames` for country names instead of locale files.
`Intl.DisplayNames` is a built-in browser API that returns a localized
country name for an ISO 3166-1 alpha-2 code, avoiding ~250 translation
strings per locale.
73 lines
2.2 KiB
Ruby
Vendored
73 lines
2.2 KiB
Ruby
Vendored
# frozen_string_literal: true
|
|
|
|
# Normalizes referrer URLs captured by the browser pageview middleware so the
|
|
# same logical referrer groups consistently in the top-referrers report. It
|
|
# strips scheme, `www.`, port, fragment, trailing slashes, and common tracking
|
|
# query params, converts the host to lowercase punycode, and truncates the
|
|
# result to 200 bytes.
|
|
class BrowserPageviewReferrerInspector
|
|
# TODO: consider vendoring DuckDuckGo's Tracker Radar tracking-parameter list
|
|
# (https://github.com/duckduckgo/tracker-radar) for broader, maintained
|
|
# coverage instead of this hand-curated subset.
|
|
TRACKING_PARAMS = %w[
|
|
utm_source
|
|
utm_medium
|
|
utm_campaign
|
|
utm_term
|
|
utm_content
|
|
fbclid
|
|
gclid
|
|
mc_cid
|
|
mc_eid
|
|
ref_src
|
|
_hsenc
|
|
_hsmi
|
|
].to_set.freeze
|
|
|
|
MAX_LENGTH = 2000
|
|
|
|
def self.normalize(raw)
|
|
return nil if raw.blank?
|
|
|
|
# Scheme is intentionally dropped: `http://example.com/x` and
|
|
# `https://example.com/x` collapse to the same key so the report groups
|
|
# cross-protocol traffic together.
|
|
uri = Addressable::URI.parse(raw.to_s.strip)
|
|
return nil if uri.nil?
|
|
|
|
host = normalize_host(uri.host)
|
|
return nil if host.blank?
|
|
|
|
path = uri.path.to_s.sub(%r{/+\z}, "")
|
|
filtered_query = filter_query(uri.query)
|
|
query_str = filtered_query.empty? ? "" : "?#{filtered_query}"
|
|
|
|
"#{host}#{path}#{query_str}".byteslice(0, MAX_LENGTH).scrub("")
|
|
rescue Addressable::URI::InvalidURIError, ArgumentError, TypeError
|
|
nil
|
|
end
|
|
|
|
def self.normalize_host(host)
|
|
return nil if host.blank?
|
|
normalized = Addressable::URI.parse("http://#{host}").normalized_host
|
|
return nil if normalized.blank?
|
|
normalized.delete_prefix("www.").delete_suffix(".")
|
|
rescue Addressable::URI::InvalidURIError
|
|
nil
|
|
end
|
|
|
|
# Filters the raw query string so original percent-encoding is preserved
|
|
# (avoids %20/+ duplicate groupings for rows pointing at the same URL).
|
|
def self.filter_query(query)
|
|
return "" if query.blank?
|
|
|
|
query
|
|
.split("&")
|
|
.reject do |pair|
|
|
key = pair.split("=", 2).first.to_s
|
|
TRACKING_PARAMS.include?(key)
|
|
end
|
|
.join("&")
|
|
end
|
|
private_class_method :filter_query
|
|
end
|
