mirror of
https://github.com/discourse/discourse.git
synced 2026-08-11 02:59:07 +08:00
1338f05312This PR fixes several bugs in the bulk import scripts that affect both
regular imports and multi-source `MERGE_IMPORT` runs.
**`import.rake` - skip virtual auto-groups in
`insert_automatic_group_users`**
The `anonymous` and `logged_in_users` groups are virtual groups with no
real membership and no valid SQL condition. Iterating over them caused
errors during import post-processing.
**`generic_bulk.rb` / `base.rb` - poll and MERGE_IMPORT re-run safety**
- **`closed_at` column name mismatch** (`import_polls`): the
intermediate DB schema uses `close_at` but the importer was reading
`closed_at`, causing all poll close dates to be silently imported as
`nil`.
- **`import_poll_options` dedup guard was a no-op**: the `next if` was
inside an `each` block, so it only skipped to the next iteration of the
inner loop - never the outer `create_poll_options` block. Poll options
would always be re-inserted on any re-run. Fixed with `next if
option_ids.all? { ... }` on the outer block.
- **`load_index` cross-source ID collision under `MERGE_IMPORT`**: when
running a second or third source with `MERGE_IMPORT=true`,
`migration_mappings` entries from all prior sources were loaded into
shared maps (`@uploads_mapping`, `@poll_mapping`, etc.). A Woltlab
upload with `id=123` could incorrectly match against a vBulletin upload
`id=123` and be skipped. The fix scopes the query to `original_id LIKE
'<prefix>:%'` and strips the prefix when building the in-memory map, so
each source's dedup is fully isolated. Regular (non-MERGE_IMPORT) runs
are unaffected — `@import_prefix` is `nil` and the original `NOT LIKE
'%:%'` path is taken.
- **Missing dedup guards on `import_chat_threads` and
`import_chat_messages`**: unlike every other entity type, chat threads
and messages had no `next if already_exists` guard. Re-running would
duplicate all chat content. Guards added using the existing
`@chat_thread_mapping` / `@chat_message_mapping` maps, already correctly
prefix-scoped by the `load_index` fix above.
|
||
|---|---|---|
| .. | ||
| benchmarks | ||
| bulk_import | ||
| demon_test | ||
| import_scripts | ||
| .gitignore | ||
| alice.txt | ||
| analyse_message_bus.rb | ||
| analyze_sidekiq_queues.rb | ||
| assemble_ember_build.rb | ||
| backport.rb | ||
| bench.rb | ||
| biggest_objects.rb | ||
| boot_mem.rb | ||
| cache_critical_dns | ||
| check_forking.rb | ||
| check_reproducible_assets.rb | ||
| copyright-deposit | ||
| db_timestamps_mover.rb | ||
| diff_heaps.rb | ||
| discourse | ||
| docker_test.rb | ||
| i18n_lint.rb | ||
| install_minio_binaries.rb | ||
| list_bundled_plugins | ||
| local_minio_s3.rb | ||
| memory-analysis | ||
| memstats.rb | ||
| micro_bench.rb | ||
| mwrap_sidekiq | ||
| profile_db_generator.rb | ||
| promote_migrations | ||
| publish_built_assets.rb | ||
| rails | ||
| redis_memory.rb | ||
| require_profiler.rb | ||
| silence_successful_output | ||
| spawn_backup_restore.rb | ||
| start_test_db.rb | ||
| switch_container_ruby | ||
| test_email_settings.rb | ||
| test_mem.rb | ||
| test_memory_leak.rb | ||
| test_pretty_text.rb | ||
| thread_detective.rb | ||
| user_simulator.rb | ||
