Discourse/discourse
Discourse/discourse
0
0
Fork 0
mirror of https://github.com/discourse/discourse.git synced 2026-08-09 21:45:25 +08:00
Code Packages Activity Actions
discourse/app/jobs/regular
History
Exact
Mark VanLandingham c85593eda8
 PERF: Add fast nested reply stats preparation (#41870) 
## Problem

Nested reply statistics currently start backfilling only after nested
replies are enabled. At the default 100 topics every five minutes, that
is 28,800 topics per day—about 174 days for a site with five million
topics. Large sites therefore have to expose the feature before its
historical stats are ready.

## Solution

Add a backend-only `nested_replies:prepare_stats` rake task. For each
site, it:

- enables the existing realtime stats callbacks while the nested-replies
UI remains disabled;
- captures the current regular-topic high-water mark;
- enqueues low-priority, keyset-paginated batches which immediately
enqueue the next batch instead of waiting five minutes;
- reuses the existing per-topic backfill SQL; and
- isolates an individual failed topic into its own job, where normal
Sidekiq retries apply, so later topics can continue.

The task leaves the hidden maintenance flag enabled, so replies created,
deleted, or reparented after preparation begins keep the same stats
current before nested replies are enabled. On multisite it prepares
every database by default; `RAILS_DB` scopes it to one database.

The existing scheduled backfill keeps its original feature gate, topic
selection, batch schedule, and calculations. This adds no UI,
migrations, stats version, readiness marker, durable cursor setting,
lock, or watchdog.

Run it with:

```sh
RAILS_ENV=production bin/rake nested_replies:prepare_stats
```

Completion is logged with the captured high-water topic ID. Re-running
the task is safe because it uses the existing idempotent upsert
behavior.

Tests cover disabled-mode preparation, bounded continuation and
high-water behavior, isolated retries, rake-task enqueueing,
preservation of the old scheduler gate, and realtime maintenance while
the UI is disabled.
2026-07-21 10:45:53 -05:00
..
admin_confirmation_email.rb DEV: Apply syntax_tree formatting to app/* 2023-01-09 14:14:59 +00:00
anonymize_user.rb FIX: Block associated account emails on user delete and anonymize (#37638) 2026-02-09 18:26:35 +01:00
automatic_group_membership.rb DEV: Apply syntax_tree formatting to app/* 2023-01-09 14:14:59 +00:00
backfill_badge.rb PERF: Set cluster_concurrency 1 for Jobs::BackfillBadge (#32121) 2025-04-02 11:23:32 +08:00
backfill_sidebar_site_settings.rb FIX: Set Jobs::BackfillSidebarSiteSettings cluster concurrency to 1 (#22316) 2023-06-28 13:07:46 +08:00
backup_chunks_merger.rb DEV: Apply syntax_tree formatting to app/* 2023-01-09 14:14:59 +00:00
bulk_grant_trust_level.rb PERF: Add Users to Group Pt. 1 (#38737) 2026-03-26 09:11:22 -05:00
bulk_invite.rb DEV: Update bulk-invite logs and PM template (#27057) 2024-05-17 12:21:21 -06:00
bulk_user_title_update.rb DEV: Apply syntax_tree formatting to app/* 2023-01-09 14:14:59 +00:00
bump_topic.rb FEATURE: Allow topic_timers_allowed_groups to manage topic timers (#41126) 2026-06-23 12:46:56 -05:00
change_display_name.rb FIX: Sync category description when post content changes outside PostRevisor (#39184) 2026-05-11 16:13:23 +02:00
check_video_conversion_status.rb DEV: Update mediaconvert to use a subdirectory for output (#36049) 2025-11-20 14:03:14 -07:00
cleanup_acls_for_deleted.rb FEATURE: Support users in DAccessControl and backend (#41358) 2026-07-15 09:18:06 +10:00
clear_slow_mode.rb DEV: Change Topic Timer from enqueue_at scheduled jobs to incrementally executed jobs (#11698) 2021-01-19 13:30:58 +10:00
close_topic.rb FEATURE: Allow topic_timers_allowed_groups to manage topic timers (#41126) 2026-06-23 12:46:56 -05:00
confirm_sns_subscription.rb SECURITY: Prevent any signed AWS SNS TopicARN from being accepted via webhooks 2026-06-30 16:10:52 +02:00
convert_video.rb FEATURE: Add support for aws MediaConvert (#33092) 2025-07-23 11:58:33 -06:00
crawl_topic_link.rb DEV: always load the excon gem 2026-01-15 17:13:11 -05:00
create_avatar_thumbnails.rb DEV: Apply syntax_tree formatting to app/* 2023-01-09 14:14:59 +00:00
create_backup.rb DEV: Apply syntax_tree formatting to app/* 2023-01-09 14:14:59 +00:00
create_linked_topic.rb DEV: Update to lastest rubocop-discourse 2024-05-27 18:06:14 +02:00
create_user_reviewable.rb DEV: Update to lastest rubocop-discourse 2024-05-27 18:06:14 +02:00
critical_user_email.rb DEV: Fix various rubocop lints (#24749) 2023-12-06 23:25:00 +01:00
delete_inaccessible_notifications.rb DEV: Apply syntax_tree formatting to app/* 2023-01-09 14:14:59 +00:00
delete_replies.rb FIX: DeleteReplies should use the guardian instead of checking for staff (#35443) 2025-10-16 18:26:41 +02:00
delete_topic.rb DEV: Apply syntax_tree formatting to app/* 2023-01-09 14:14:59 +00:00
delete_user_posts.rb FIX: return nil correctly for nil check (#37460) 2026-02-02 15:43:56 -08:00
deliver_push_notification.rb DEV: Centralize push notification delivery into a single job (#39207) 2026-04-14 11:55:45 -03:00
download_avatar_from_url.rb DEV: Apply syntax_tree formatting to app/* 2023-01-09 14:14:59 +00:00
download_backup_email.rb DEV: Apply syntax_tree formatting to app/* 2023-01-09 14:14:59 +00:00
download_profile_background_from_url.rb DEV: Apply syntax_tree formatting to app/* 2023-01-09 14:14:59 +00:00
emit_web_hook_event.rb DEV: always load the excon gem 2026-01-15 17:13:11 -05:00
export_csv_file.rb FIX: re-validate permissions when running CSV export job (#41464) 2026-07-06 16:50:06 +05:30
export_user_archive.rb DEV: Enable Style/RedundantBegin rubocop rule (#40096) 2026-05-19 18:44:54 +02:00
feature_topic_users.rb DEV: Update to lastest rubocop-discourse 2024-05-27 18:06:14 +02:00
generate_topic_og_image.rb FEATURE: generated OpenGraph images for topics (#39194) 2026-06-26 10:37:10 -04:00
generate_topic_thumbnails.rb DEV: Apply syntax_tree formatting to app/* 2023-01-09 14:14:59 +00:00
group_pm_alert.rb DEV: Apply syntax_tree formatting to app/* 2023-01-09 14:14:59 +00:00
group_pm_update_summary.rb DEV: Apply syntax_tree formatting to app/* 2023-01-09 14:14:59 +00:00
group_smtp_email.rb FIX: use next instead of return in sidekiq_retry_in blocks (#39529) 2026-04-24 16:59:49 +02:00
index_category_for_search.rb FIX: don't error-index category job when missing category 2020-12-01 09:30:53 +08:00
index_user_fields_for_search.rb FIX: Move user reindexing into a job (#26753) 2024-04-25 20:58:34 +08:00
invite_email.rb DEV: Tighten protections when disabling allow_email_invites (#37453) 2026-02-02 11:40:04 -05:00
invite_password_instructions_email.rb DEV: Update to lastest rubocop-discourse 2024-05-27 18:06:14 +02:00
make_embedded_topic_visible.rb FIX: Better tracking of topic visibility changes (#26709) 2024-04-29 10:34:46 +10:00
mass_award_badge.rb FEATURE: Add option to grant badge multiple times to users using Bulk Award (#13571) 2021-07-15 05:53:26 +03:00
merge_user.rb DEV: Fix assigned but unused variable Prism warnings (#39436) 2026-04-22 12:42:14 +02:00
notify_category_change.rb FIX: Notify mailing list subscribers on category change (#28811) 2024-10-11 14:47:39 +02:00
notify_mailing_list_subscribers.rb FIX: use next instead of return in sidekiq_retry_in blocks (#39529) 2026-04-24 16:59:49 +02:00
notify_moved_posts.rb DEV: Hand-pick Rails/WhereNot autofixes (#35117) 2025-10-03 13:29:22 +02:00
notify_post_revision.rb DEV: Apply syntax_tree formatting to app/* 2023-01-09 14:14:59 +00:00
notify_reviewable.rb FIX: Resolve all of a deleted spammer's reviewables (#40780) 2026-06-15 13:34:15 +08:00
notify_tag_change.rb FIX: Allow "watching first post" notification to fire when tags edit notifications are disabled (#37935) 2026-02-20 16:59:53 +08:00
notify_users_added_to_group.rb PERF: Add Users to Group Improvements Pt. 2 (#38901) 2026-04-06 09:31:50 -05:00
open_topic.rb FEATURE: Allow topic_timers_allowed_groups to manage topic timers (#41126) 2026-06-23 12:46:56 -05:00
post_alert.rb DEV: Apply syntax_tree formatting to app/* 2023-01-09 14:14:59 +00:00
post_update_topic_tracking_state.rb FEATURE: Exclude small actions from topic counters and unread tracking (#40481) 2026-06-08 08:03:55 +02:00
prepare_nested_reply_stats.rb PERF: Add fast nested reply stats preparation (#41870) 2026-07-21 10:45:53 -05:00
process_bulk_invite_emails.rb DEV: Apply syntax_tree formatting to app/* 2023-01-09 14:14:59 +00:00
process_email.rb DEV: Apply syntax_tree formatting to app/* 2023-01-09 14:14:59 +00:00
process_localized_cooked.rb FEATURE: Localize local oneboxes (#40493) 2026-06-08 11:49:55 +08:00
process_post.rb FIX: Sync category description when post content changes outside PostRevisor (#39184) 2026-05-11 16:13:23 +02:00
process_sns_notification.rb SECURITY: Prevent any signed AWS SNS TopicARN from being accepted via webhooks 2026-06-30 16:10:52 +02:00
publish_group_membership_updates.rb DEV: Apply syntax_tree formatting to app/* 2023-01-09 14:14:59 +00:00
publish_topic_to_category.rb FEATURE: Allow topic_timers_allowed_groups to manage topic timers (#41126) 2026-06-23 12:46:56 -05:00
pull_hotlinked_images.rb SECURITY: Authorize secure-upload hotlink downloads against post user 2026-06-30 16:10:52 +02:00
pull_user_profile_hotlinked_images.rb DEV: Apply syntax_tree formatting to app/* 2023-01-09 14:14:59 +00:00
push_notification.rb DEV: Centralize push notification delivery into a single job (#39207) 2026-04-14 11:55:45 -03:00
rebake_custom_emoji_posts.rb DEV: Upgrading Discourse to Zeitwerk (#8098) 2019-10-02 14:01:53 +10:00
rebake_posts_for_upload.rb FIX: Mark secure media upload insecure automatically if used for theme component (#8413) 2019-11-28 07:32:17 +10:00
rebake_quoted_posts_for_user.rb FIX: Move quoted post rebaking on avatar upload to a background job (#40231) 2026-05-22 13:27:52 -03:00
refresh_users_reviewable_counts.rb DEV: Exclude system users when calculating group user count (#25400) 2024-01-25 08:13:58 +08:00
remap_category_hashtag.rb FIX: Remap category hashtags when slugs change (#41075) 2026-06-23 09:28:30 +10:00
remove_banner.rb DEV: Update rubocop (#38721) 2026-03-20 00:39:52 +01:00
retrieve_topic.rb DEV: Update to lastest rubocop-discourse 2024-05-27 18:06:14 +02:00
run_heartbeat.rb FIX: Redo Sidekiq monitoring to restart stuck sidekiq processes (#30198) 2024-12-18 12:48:50 +08:00
run_problem_check.rb FIX: Fix scheduled targeted problem checks (#35696) 2025-11-10 10:09:14 +08:00
send_email_login_code.rb DEV: Introduce enable_local_logins_via_code as an alpha upcoming change (#41624) 2026-07-10 10:19:27 -07:00
send_push_notification.rb DEV: Centralize push notification delivery into a single job (#39207) 2026-04-14 11:55:45 -03:00
send_system_message.rb DEV: Update to lastest rubocop-discourse 2024-05-27 18:06:14 +02:00
site_setting_update_default_categories.rb FIX: Timeout issue when updating a large collection of users when changing the default_categories_* and default_tags_* SiteSettings (#33665) 2025-08-20 12:55:53 -05:00
site_setting_update_default_tags.rb FIX: Timeout issue when updating a large collection of users when changing the default_categories_* and default_tags_* SiteSettings (#33665) 2025-08-20 12:55:53 -05:00
suspicious_login.rb DEV: Apply syntax_tree formatting to app/* 2023-01-09 14:14:59 +00:00
sync_access_control_for_uploads.rb DEV: Enable Style/RedundantBegin rubocop rule (#40096) 2026-05-19 18:44:54 +02:00
sync_topic_user_bookmarked.rb DEV: Update to lastest rubocop-discourse 2024-05-27 18:06:14 +02:00
toggle_topic_closed.rb FEATURE: Allow topic_timers_allowed_groups to manage topic timers (#41126) 2026-06-23 12:46:56 -05:00
topic_action_converter.rb DEV: Apply syntax_tree formatting to app/* 2023-01-09 14:14:59 +00:00
topic_timer_base.rb DEV: Change Topic Timer from enqueue_at scheduled jobs to incrementally executed jobs (#11698) 2021-01-19 13:30:58 +10:00
truncate_user_flag_stats.rb DEV: Update to lastest rubocop-discourse 2024-05-27 18:06:14 +02:00
unpin_topic.rb DEV: Update rubocop (#38721) 2026-03-20 00:39:52 +01:00
update_gravatar.rb DEV: Apply syntax_tree formatting to app/* 2023-01-09 14:14:59 +00:00
update_group_mentions.rb DEV: Apply syntax_tree formatting to app/* 2023-01-09 14:14:59 +00:00
update_hotlinked_raw.rb DEV: Update rubocop-discourse to 3.13 and autofix issues (#35073) 2025-10-06 16:11:01 +02:00
update_post_uploads_secure_status.rb DEV: Apply syntax_tree formatting to app/* 2023-01-09 14:14:59 +00:00
update_top_redirection.rb DEV: Apply syntax_tree formatting to app/* 2023-01-09 14:14:59 +00:00
update_topic_upload_security.rb DEV: Apply syntax_tree formatting to app/* 2023-01-09 14:14:59 +00:00
update_username.rb FIX: Sync category description when post content changes outside PostRevisor (#39184) 2026-05-11 16:13:23 +02:00
user_email.rb DEV: Enable Style/RedundantParentheses rubocop rule (#40095) 2026-05-19 15:48:09 +02:00