2
0
Fork 0
mirror of https://github.com/discourse/discourse.git synced 2025-09-06 10:50:21 +08:00
discourse/lib/tasks
Martin Brennan faf5b4d3e9
PERF: Speed up secure media and ACL sync rake tasks (#16849)
Incorporates learnings from /t/64227:

* Changes the code to set access control posts in the rake
  task to be an efficient UPDATE SQL query.
  The original version was timing out with 312017 post uploads,
  the new query took ~3s to run.
* Changes the code to mark uploads as secure/not secure in
  the rake task to be an efficient UPDATE SQL query rather than
  using UploadSecurity. This took a very long time previously,
  and now takes only a few seconds.
* Spread out ACL syncing for uploads into jobs with batches of
  100 uploads at a time, so they can be parallelized instead
  of having to wait ~1.25 seconds for each ACL to be changed
  in S3 serially.

One issue that still remains is post rebaking. Doing this serially
is painfully slow. We have a way to do this in sidekiq via PeriodicalUpdates
but this is limited by max_old_rebakes_per_15_minutes. It would
be better to fan this rebaking out into jobs like we did for the
ACL sync, but that should be done in another PR.
2022-05-23 13:14:11 +10:00
..
add_topic_to_quotes.rake
admin.rake DEV: Use strings for :user_email job type argument 2022-02-04 18:28:18 +00:00
annotate.rake DEV: Add annotate rake tasks, and enforce via GitHub actions 2021-07-06 10:11:06 +01:00
api.rake
assets.rake DEV: Fix production sourcemaps with Ember CLI (#16707) 2022-05-11 10:23:32 +01:00
auto_annotate_models.rake DEV: Allow annotations to work in symlinked plugins, add binstub (#13635) 2021-07-05 15:43:10 +01:00
autospec.rake DEV: Add API docs for uploads and API doc watcher (#15387) 2021-12-23 08:40:15 +10:00
avatars.rake
backfill.thor
categories.rake FIX: rake categories:move_topics shouldn't move category description topic (#14797) 2021-11-03 11:27:51 +08:00
cdn.rake DEV: Fix rubocop issues (#14715) 2021-10-27 11:39:28 +03:00
db.rake FIX: Show error message if extensions cannot be created (#16719) 2022-05-11 15:46:49 +03:00
destroy.rake
dev.rake DEV: move discourse_dev gem to the core. (#13360) 2021-06-14 20:34:44 +05:30
docker.rake DEV: Add SKIP_DB_CREATE option to docker:test (#16225) 2022-03-21 13:21:06 +01:00
emails.rake DEV: Correctly tag heredocs (#16061) 2022-02-28 20:50:55 +01:00
emoji.rake DEV: Upgrade to Rails 7 2022-04-28 11:51:03 +02:00
export.rake FEATURE: Rake task to export groups (#9450) 2020-04-17 14:59:54 -07:00
groups.rake
i18n.rake DEV: Update i18n:check rake task to detect invalid Markdown links (#13728) 2021-07-14 13:26:12 +02:00
images.rake DEV: Upgrade to Rails 7 2022-04-28 11:51:03 +02:00
import.rake FEATURE: Add last visit indication to topic view page. (#13471) 2021-07-05 14:17:31 +08:00
incoming_emails.rake
integration.rake DEV: Remove hints of jshint (#14828) 2021-11-08 09:12:37 +08:00
javascript.rake FIX: uses tippy for popover (#15409) 2022-05-02 17:10:26 +02:00
log.rake FIX: whoops, linting 2020-09-03 12:10:11 +01:00
maxminddb.rake
plugin.rake DEV: Improve plugin:versions task (#16391) 2022-04-06 18:29:39 +02:00
populate.rake DEV: Add ability to populate private messages (#16705) 2022-05-12 14:29:49 -04:00
populate.thor DEV: Correctly tag heredocs (#16061) 2022-02-28 20:50:55 +01:00
posts.rake FIX: LocalJumpError in rake tasks (#14874) 2021-11-10 16:53:55 +01:00
qunit.rake DEV: Allow running theme-qunit tests via testem (#16540) 2022-04-22 15:04:01 +01:00
redis.rake FIX: add support for pipelined and multi redis commands (#16682) 2022-05-10 08:19:02 +10:00
release_note.rake DEV: Fix methods removed in Ruby 3.2 (#15459) 2022-01-05 18:45:08 +01:00
rspec.rake
s3.rake DEV: Ensure Ember CLI sourcemaps are uploaded to S3 (#15893) 2022-02-10 19:00:47 +00:00
scheduler.rake
search.rake
site.rake DEV: Fix methods removed in Ruby 3.2 (#15459) 2022-01-05 18:45:08 +01:00
site_settings.rake
smoke_test.rake DEV: Minor linting fixes (#16359) 2022-04-02 21:15:38 +02:00
svg_icons.rake
tags.rake FEATURE: Support append when bulk tagging via rake (#9978) 2020-06-04 09:33:48 -04:00
themes.rake DEV: Only raise rake themes:update errors when flag provided (#16254) 2022-03-22 17:02:14 +00:00
topics.rake DEV: Upgrade to Rails 7 2022-04-28 11:51:03 +02:00
turbo.rake
typepad.thor FIX: use allowlist and blocklist terminology (#10209) 2020-07-27 10:23:54 +10:00
uploads.rake PERF: Speed up secure media and ACL sync rake tasks (#16849) 2022-05-23 13:14:11 +10:00
users.rake