mirror of
https://gh.wpcy.net/https://github.com/discourse/discourse.git
synced 2026-05-23 02:18:23 +08:00
7ecb945ec4
## Overview
This PR introduces comprehensive search functionality for chat messages,
enabling users to search through their chat history both globally across
all accessible channels and within specific channels.
### Search Capabilities
**All-Channel Search**: When no channel is specified, users can search
across all channels they have access to. The search respects channel
permissions through `ChannelFetcher.all_secured_channel_ids`, ensuring
users only see results from channels they can view.
**Per-Channel Search**: Users can scope their search to a specific
channel by providing a `channel_id` parameter, useful for finding
messages within a particular conversation context.
**Search Features**:
- Full-text search using PostgreSQL's tsvector/tsquery
- Advanced filters: `@username` to filter by author, `#channel` to
filter by channel slug
- Sort options: relevance (default) or latest
- Pagination support
- Search data weighted by relevance
## Site Setting: `chat_search_enabled`
This feature is gated behind the `chat_search_enabled` site setting,
which is currently:
- **Default**: `false`
- **Hidden**: `true`
- **Client-accessible**: `true`
### Deployment Strategy
Due to the need for chat messages to be indexed before search becomes
useful, we're implementing a two-phase deployment:
**Phase 1 (Initial Merge)**:
- `chat_search_enabled` remains `false` and hidden
- The `register_search_index` uses default (true) instead of `chat_search_enabled` value
- This allows the reindexing infrastructure to begin indexing existing
chat messages even if we don't show the UI yet
**Wait Period**:
- Wait at least one week after Phase 1 deployment
- `Jobs::ReindexSearch` runs every 2 hours and will progressively index
all chat messages
- This ensures most sites have a significant part of their chat history indexed
**Phase 2 (Follow-up Merge)**:
- Set `chat_search_enabled` default to `true` and unhide it
- Update the `register_search_index` enabled proc uses the default
(true) instead of using the `chat_search_enabled` setting
- Users can now access search with pre-indexed data
**Rationale**: Without this phased approach, users would see the search
UI immediately but receive no results until the reindexing job runs,
creating a confusing experience. By pre-indexing while the UI is hidden,
we ensure search works immediately when enabled.
## New Plugin API: `register_search_index`
This PR introduces a new plugin API that allows plugins to register
custom search indexes that integrate seamlessly with Discourse's search
infrastructure.
### API Signature
```ruby
register_search_index(
model_class:, # The ActiveRecord model to index
search_data_class:, # The model for storing search data
index_version:, # Version number for re-indexing
search_data:, # Proc that returns weighted search data
load_unindexed_record_ids:,# Proc that finds records needing indexing
enabled: # Optional proc to enable/disable (default: -> { true })
)
```
### How It Works
**Integration with SearchIndexer**: When `SearchIndexer.index(obj)` is
called, it checks registered search handlers for the object's type. If a
handler matches, it:
1. Calls the `search_data` proc with the object and an `IndexerHelper`
instance
2. Receives weighted search data (`:a_weight`, `:b_weight`, `:c_weight`,
`:d_weight`)
3. Updates the corresponding search data table with PostgreSQL's
tsvector
**Integration with Jobs::ReindexSearch**: The scheduled job (runs every
2 hours) calls `rebuild_registered_search_handlers`, which:
1. Iterates through all registered search handlers
2. Skips handlers where `enabled` proc returns `false`
3. Calls `load_unindexed_record_ids` to find records needing indexing
4. Indexes up to `limit` records per handler (default: 10,000)
### Chat Implementation Example
```ruby
register_search_index(
model_class: Chat::Message,
search_data_class: Chat::MessageSearchData,
index_version: 1,
search_data: proc { |message, indexer_helper|
{
a_weight: message.message,
d_weight: indexer_helper.scrub_html(message.cooked)[0..600_000]
}
},
load_unindexed_record_ids: proc { |limit:, index_version:|
Chat::Message
.joins("LEFT JOIN chat_message_search_data ON chat_message_id = chat_messages.id")
.where(
"chat_message_search_data.locale IS NULL OR
chat_message_search_data.locale != ? OR
chat_message_search_data.version != ?",
SiteSetting.default_locale,
index_version
)
.order("chat_messages.id ASC")
.limit(limit)
.pluck(:id)
}
)
```
Co-authored-by: Martin Brennan <mjrbrennan@gmail.com>
Co-authored-by: Loïc Guitaut <5648+Flink@users.noreply.github.com>
|
||
|---|---|---|
| .. | ||
| 20210225230057_create_chat_tables.rb | ||
| 20210403025854_add_action_code_to_topic_chat_message.rb | ||
| 20210706214013_rename_topic_chats_to_chat_channels.rb | ||
| 20210729134042_create_chat_message_revisions.rb | ||
| 20210730134847_create_user_chat_channel_last_read.rb | ||
| 20210812145801_create_direct_message_tables.rb | ||
| 20210813141741_add_timestamps_to_chat_channels.rb | ||
| 20210819202912_create_incoming_chat_webhooks.rb | ||
| 20210823160357_create_chat_webhook_events.rb | ||
| 20210901130308_create_user_chat_channel_membership.rb | ||
| 20210930144333_add_chat_enabled_to_user_options.rb | ||
| 20211022151713_create_chat_message_post_connections.rb | ||
| 20211029145508_add_chat_isolated_to_user_options.rb | ||
| 20211104141254_add_only_chat_push_notifications_to_user_options.rb | ||
| 20211119142000_add_cooked_to_chat_messages.rb | ||
| 20211129171229_create_chat_uploads.rb | ||
| 20211201171813_create_chat_reactions.rb | ||
| 20211210191830_create_chat_mentions.rb | ||
| 20211213150607_add_chat_sound_to_user_options.rb | ||
| 20211217221026_add_name_to_chat_channel.rb | ||
| 20211222153716_add_description_to_chat_channels.rb | ||
| 20220104051326_change_chat_channels_timestamp_columns_to_timestamp_type.rb | ||
| 20220119170535_add_chat_retention_fields_to_user_options.rb | ||
| 20220203204002_create_chat_drafts_table.rb | ||
| 20220203204003_migrate_drafts_to_chat_drafts.rb | ||
| 20220218023859_add_status_to_chat_channel.rb | ||
| 20220228051724_create_chat_channel_archive_table.rb | ||
| 20220308165620_add_user_count_to_chat_channel.rb | ||
| 20220309174820_add_last_message_created_at_to_chat_channels.rb | ||
| 20220321235638_drop_chat_message_post_connections_table.rb | ||
| 20220324062937_ignore_channel_wide_mention_to_user_options.rb | ||
| 20220328142120_create_user_chat_message_statuses.rb | ||
| 20220504080457_drop_old_chat_message_post_id_action_code_columns.rb | ||
| 20220516142658_remove_email_statuses_table.rb | ||
| 20220518140004_track_last_unread_mention_when_emailed.rb | ||
| 20220518180642_remove_user_option_last_emailed_at.rb | ||
| 20220526135414_remove_corrupted_last_read_message_id.rb | ||
| 20220531105951_drop_user_chat_channel_last_reads.rb | ||
| 20220629190633_auto_join_users_to_channels.rb | ||
| 20220630074200_drop_chat_isolated_from_user_options.rb | ||
| 20220701195731_convert_chatable_topics_to_categories.rb | ||
| 20220706114835_add_join_mode_to_channel_memberships.rb | ||
| 20220729032237_add_index_to_chat_message_created_at.rb | ||
| 20220802014549_disable_chat_uploads_if_secure_media_enabled.rb | ||
| 20220901034107_add_user_count_stale_to_channel.rb | ||
| 20221004122254_delete_reviewables_targetting_deleted_chat_messages.rb | ||
| 20221005143622_add_type_to_chat_channel.rb | ||
| 20221014005208_add_slug_column_to_chat_channel.rb | ||
| 20221018091412_migrate_chat_channels.rb | ||
| 20221027090832_migrate_dm_channels.rb | ||
| 20221101061319_add_last_editor_id_to_chat_messages.rb | ||
| 20221104054957_backfill_channel_slugs.rb | ||
| 20221107034541_make_chat_editor_ids_not_null.rb | ||
| 20221117052348_truncate_chat_messages_over_max_length.rb | ||
| 20221117142910_delete_orphaned_channels.rb | ||
| 20221118104708_add_allow_channel_wide_mentions_to_chat_channels.rb | ||
| 20221122070108_save_chat_allowed_groups_site_setting.rb | ||
| 20221201024458_make_channel_slugs_unique_with_index.rb | ||
| 20221201032830_drop_tmp_chat_slug_tables.rb | ||
| 20221201035918_add_slug_unique_index_for_chat_channels.rb | ||
| 20221202032006_add_chat_message_count_to_chat_channels.rb | ||
| 20221202043755_update_chat_channel_message_counts.rb | ||
| 20230116090324_drop_chat_drafts_over_max_length.rb | ||
| 20230123020036_move_chat_uploads_to_upload_references.rb | ||
| 20230123025112_move_chat_uploads_to_upload_references_post.rb | ||
| 20230130053144_add_threading_enabled_to_chat_channels.rb | ||
| 20230201012734_create_chat_threading_models.rb | ||
| 20230227172543_make_chat_mention_notification_id_nullable.rb | ||
| 20230228062442_add_chat_header_indicator_preference.rb | ||
| 20230403012844_drop_chat_uploads.rb | ||
| 20230411012630_add_thread_not_deleted_index_chat_messages.rb | ||
| 20230411023246_add_chat_message_replies_count_to_chat_threads.rb | ||
| 20230411023340_update_thread_reply_counts.rb | ||
| 20230510142249_add_user_chat_thread_memberships.rb | ||
| 20230607091233_backfill_thread_memberships.rb | ||
| 20230627044755_add_last_viewed_at_to_user_chat_channel_memberships.rb | ||
| 20230707025733_add_last_message_id_to_channel_and_thread.rb | ||
| 20230707082645_backfill_chat_channel_and_thread_last_message_ids.rb | ||
| 20230710040640_backfill_chat_channel_and_thread_last_message_ids_post_migrate.rb | ||
| 20230721025249_remove_experimental_site_setting_for_threads.rb | ||
| 20230722124044_add_chat_separate_sidebar_mode_user_option.rb | ||
| 20231006160650_add_group_field_to_direct_message_channels.rb | ||
| 20231006161051_set_multiusers_direct_message_channels_as_group.rb | ||
| 20231110214451_adds_thread_id_to_chat_drafts.rb | ||
| 20231207135641_add_user_chat_thread_memberships_on_thread_id_user_id_index.rb | ||
| 20231214180000_add_chat_mention_notifications.rb | ||
| 20231214180001_update_relationship_between_chat_mentions_and_notifications.rb | ||
| 20231214180002_update_relationship_between_chat_mentions_and_notifications_post_migrate.rb | ||
| 20231227160001_add_type_and_target_id_to_chat_mentions.rb | ||
| 20231227160002_set_type_and_target_id_on_chat_mentions.rb | ||
| 20231227160003_add_and_remove_indexes_on_chat_mentions.rb | ||
| 20231227160004_set_type_and_target_id_on_chat_mentions_post_migrate.rb | ||
| 20231227160005_make_type_on_chat_mentions_non_nullable.rb | ||
| 20240118120825_add_threads_enabled_site_setting.rb | ||
| 20240213175713_add_streaming_to_message.rb | ||
| 20240214135517_fix_chat_channel_slug_index.rb | ||
| 20240301100413_add_force_to_threads.rb | ||
| 20240408140000_drop_notification_id_from_chat_mentions.rb | ||
| 20240409060201_add_thread_title_prompt_to_user_chat_thread_memberships.rb | ||
| 20240409093348_add_show_thread_title_prompts_to_user_options.rb | ||
| 20240410130000_drop_user_id_from_chat_mentions.rb | ||
| 20240422042830_add_excerpt_to_chat_messages.rb | ||
| 20240425133407_drop_chat_channels_last_message_sent_at.rb | ||
| 20240516145911_update_user_options_for_thread_title_prompts.rb | ||
| 20240711153837_add_created_by_sdk_to_chat_messages.rb | ||
| 20240711154622_update_chat_messages_created_by_sdk.rb | ||
| 20240827040131_add_new_notification_id_to_chat_mention_notifications.rb | ||
| 20240827040550_copy_chat_mention_notifications_notification_id_values.rb | ||
| 20240827040810_copy_chat_mention_notifications_notification_id_indexes.rb | ||
| 20240827040811_swap_bigint_chat_mention_notifications_notification_id.rb | ||
| 20240829140227_drop_chat_mention_notifications_old_id_column.rb | ||
| 20241003122030_add_notification_level_to_user_chat_channel_memberships.rb | ||
| 20241029192512_alter_chat_ids_to_bigint.rb | ||
| 20241031050638_add_custom_fields_to_chat.rb | ||
| 20241104053309_add_icon_upload_id_to_chat_channel.rb | ||
| 20241110120303_add_blocks_to_chat_messages.rb | ||
| 20241111022618_create_chat_message_interactions.rb | ||
| 20241120182858_add_index_to_chat_messages_on_chat_channel_id.rb | ||
| 20241226162229_add_chat_send_shortcut_preference.rb | ||
| 20250220090521_add_last_unread_message_when_emailed_id_to_chat_thread_memberships.rb | ||
| 20250307185912_add_chat_quick_reaction_preferences.rb | ||
| 20250903091723_create_chat_message_search_data.rb | ||
