Exact

Exact Union RegExp

Joffrey JAFFEUX 7ecb945ec4 FEATURE: Add full-text search for chat messages (#34704) ... ## Overview This PR introduces comprehensive search functionality for chat messages, enabling users to search through their chat history both globally across all accessible channels and within specific channels. ### Search Capabilities **All-Channel Search**: When no channel is specified, users can search across all channels they have access to. The search respects channel permissions through `ChannelFetcher.all_secured_channel_ids`, ensuring users only see results from channels they can view. **Per-Channel Search**: Users can scope their search to a specific channel by providing a `channel_id` parameter, useful for finding messages within a particular conversation context. **Search Features**: - Full-text search using PostgreSQL's tsvector/tsquery - Advanced filters: `@username` to filter by author, `#channel` to filter by channel slug - Sort options: relevance (default) or latest - Pagination support - Search data weighted by relevance ## Site Setting: `chat_search_enabled` This feature is gated behind the `chat_search_enabled` site setting, which is currently: - **Default**: `false` - **Hidden**: `true` - **Client-accessible**: `true` ### Deployment Strategy Due to the need for chat messages to be indexed before search becomes useful, we're implementing a two-phase deployment: **Phase 1 (Initial Merge)**: - `chat_search_enabled` remains `false` and hidden - The `register_search_index` uses default (true) instead of `chat_search_enabled` value - This allows the reindexing infrastructure to begin indexing existing chat messages even if we don't show the UI yet **Wait Period**: - Wait at least one week after Phase 1 deployment - `Jobs::ReindexSearch` runs every 2 hours and will progressively index all chat messages - This ensures most sites have a significant part of their chat history indexed **Phase 2 (Follow-up Merge)**: - Set `chat_search_enabled` default to `true` and unhide it - Update the `register_search_index` enabled proc uses the default (true) instead of using the `chat_search_enabled` setting - Users can now access search with pre-indexed data **Rationale**: Without this phased approach, users would see the search UI immediately but receive no results until the reindexing job runs, creating a confusing experience. By pre-indexing while the UI is hidden, we ensure search works immediately when enabled. ## New Plugin API: `register_search_index` This PR introduces a new plugin API that allows plugins to register custom search indexes that integrate seamlessly with Discourse's search infrastructure. ### API Signature ```ruby register_search_index( model_class:, # The ActiveRecord model to index search_data_class:, # The model for storing search data index_version:, # Version number for re-indexing search_data:, # Proc that returns weighted search data load_unindexed_record_ids:,# Proc that finds records needing indexing enabled: # Optional proc to enable/disable (default: -> { true }) ) ``` ### How It Works **Integration with SearchIndexer**: When `SearchIndexer.index(obj)` is called, it checks registered search handlers for the object's type. If a handler matches, it: 1. Calls the `search_data` proc with the object and an `IndexerHelper` instance 2. Receives weighted search data (`:a_weight`, `:b_weight`, `:c_weight`, `:d_weight`) 3. Updates the corresponding search data table with PostgreSQL's tsvector **Integration with Jobs::ReindexSearch**: The scheduled job (runs every 2 hours) calls `rebuild_registered_search_handlers`, which: 1. Iterates through all registered search handlers 2. Skips handlers where `enabled` proc returns `false` 3. Calls `load_unindexed_record_ids` to find records needing indexing 4. Indexes up to `limit` records per handler (default: 10,000) ### Chat Implementation Example ```ruby register_search_index( model_class: Chat::Message, search_data_class: Chat::MessageSearchData, index_version: 1, search_data: proc { |message, indexer_helper| { a_weight: message.message, d_weight: indexer_helper.scrub_html(message.cooked)[0..600_000] } }, load_unindexed_record_ids: proc { |limit:, index_version:| Chat::Message .joins("LEFT JOIN chat_message_search_data ON chat_message_id = chat_messages.id") .where( "chat_message_search_data.locale IS NULL OR chat_message_search_data.locale != ? OR chat_message_search_data.version != ?", SiteSetting.default_locale, index_version ) .order("chat_messages.id ASC") .limit(limit) .pluck(:id) } ) ``` Co-authored-by: Martin Brennan <mjrbrennan@gmail.com> Co-authored-by: Loïc Guitaut <5648+Flink@users.noreply.github.com>		2025-10-22 11:30:35 +02:00
..
locales	FEATURE: Add full-text search for chat messages (#34704)	2025-10-22 11:30:35 +02:00
routes.rb	FEATURE: Add full-text search for chat messages (#34704)	2025-10-22 11:30:35 +02:00
settings.yml	FEATURE: Add full-text search for chat messages (#34704)	2025-10-22 11:30:35 +02:00