mirror of
https://ghfast.top/https://github.com/discourse/discourse-ai.git
synced 2026-07-17 11:46:28 +08:00
5e80f93e4c
This PR introduces several enhancements and refactorings to the AI Persona and RAG (Retrieval-Augmented Generation) functionalities within the discourse-ai plugin. Here's a breakdown of the changes: **1. LLM Model Association for RAG and Personas:** - **New Database Columns:** Adds `rag_llm_model_id` to both `ai_personas` and `ai_tools` tables. This allows specifying a dedicated LLM for RAG indexing, separate from the persona's primary LLM. Adds `default_llm_id` and `question_consolidator_llm_id` to `ai_personas`. - **Migration:** Includes a migration (`20250210032345_migrate_persona_to_llm_model_id.rb`) to populate the new `default_llm_id` and `question_consolidator_llm_id` columns in `ai_personas` based on the existing `default_llm` and `question_consolidator_llm` string columns, and a post migration to remove the latter. - **Model Changes:** The `AiPersona` and `AiTool` models now `belong_to` an `LlmModel` via `rag_llm_model_id`. The `LlmModel.proxy` method now accepts an `LlmModel` instance instead of just an identifier. `AiPersona` now has `default_llm_id` and `question_consolidator_llm_id` attributes. - **UI Updates:** The AI Persona and AI Tool editors in the admin panel now allow selecting an LLM for RAG indexing (if PDF/image support is enabled). The RAG options component displays an LLM selector. - **Serialization:** The serializers (`AiCustomToolSerializer`, `AiCustomToolListSerializer`, `LocalizedAiPersonaSerializer`) have been updated to include the new `rag_llm_model_id`, `default_llm_id` and `question_consolidator_llm_id` attributes. **2. PDF and Image Support for RAG:** - **Site Setting:** Introduces a new hidden site setting, `ai_rag_pdf_images_enabled`, to control whether PDF and image files can be indexed for RAG. This defaults to `false`. - **File Upload Validation:** The `RagDocumentFragmentsController` now checks the `ai_rag_pdf_images_enabled` setting and allows PDF, PNG, JPG, and JPEG files if enabled. Error handling is included for cases where PDF/image indexing is attempted with the setting disabled. - **PDF Processing:** Adds a new utility class, `DiscourseAi::Utils::PdfToImages`, which uses ImageMagick (`magick`) to convert PDF pages into individual PNG images. A maximum PDF size and conversion timeout are enforced. - **Image Processing:** A new utility class, `DiscourseAi::Utils::ImageToText`, is included to handle OCR for the images and PDFs. - **RAG Digestion Job:** The `DigestRagUpload` job now handles PDF and image uploads. It uses `PdfToImages` and `ImageToText` to extract text and create document fragments. - **UI Updates:** The RAG uploader component now accepts PDF and image file types if `ai_rag_pdf_images_enabled` is true. The UI text is adjusted to indicate supported file types. **3. Refactoring and Improvements:** - **LLM Enumeration:** The `DiscourseAi::Configuration::LlmEnumerator` now provides a `values_for_serialization` method, which returns a simplified array of LLM data (id, name, vision_enabled) suitable for use in serializers. This avoids exposing unnecessary details to the frontend. - **AI Helper:** The `AiHelper::Assistant` now takes optional `helper_llm` and `image_caption_llm` parameters in its constructor, allowing for greater flexibility. - **Bot and Persona Updates:** Several updates were made across the codebase, changing the string based association to a LLM to the new model based. - **Audit Logs:** The `DiscourseAi::Completions::Endpoints::Base` now formats raw request payloads as pretty JSON for easier auditing. - **Eval Script:** An evaluation script is included. **4. Testing:** - The PR introduces a new eval system for LLMs, this allows us to test how functionality works across various LLM providers. This lives in `/evals` |
||
|---|---|---|
| .. | ||
| 20230224165056_create_classification_results_table.rb | ||
| 20230307125342_created_model_accuracy_table.rb | ||
| 20230314184514_migrate_discourse_ai_reviewables.rb | ||
| 20230316160714_create_completion_prompt_table.rb | ||
| 20230320122645_delete_duplicated_seeded_prompts.rb | ||
| 20230320185619_multi_message_completion_prompts.rb | ||
| 20230320191928_drop_completion_prompt_value.rb | ||
| 20230322142028_make_dropped_value_column_nullable.rb | ||
| 20230406135943_add_provider_to_completion_prompts.rb | ||
| 20230424055354_create_ai_api_audit_logs.rb | ||
| 20230519003106_post_custom_prompts.rb | ||
| 20230710171141_enable_pg_vector_extension.rb | ||
| 20230710171142_create_ai_topic_embeddings_table.rb | ||
| 20230710171143_migrate_embeddings_from_dedicated_database.rb | ||
| 20230727170222_create_multilingual_topic_embeddings_table.rb | ||
| 20230831033812_rename_ai_helper_add_ai_pm_to_header_setting.rb | ||
| 20231003155701_create_bge_topic_embeddings_table.rb | ||
| 20231031050538_add_topic_id_post_id_to_ai_audit_log.rb | ||
| 20231109011155_create_ai_personas.rb | ||
| 20231117050928_add_system_and_priority_to_ai_personas.rb | ||
| 20231120033747_remove_site_settings.rb | ||
| 20231123224203_switch_to_generic_completion_prompts.rb | ||
| 20231128151234_recreate_generate_titles_prompt.rb | ||
| 20231202013850_convert_ai_personas_commands_to_json.rb | ||
| 20231227223301_create_gemini_topic_embeddings_table.rb | ||
| 20231228213036_create_ai_post_embeddings_tables.rb | ||
| 20240104013944_add_params_to_completion_prompt.rb | ||
| 20240119152348_explicit_provider_backwards_compat.rb | ||
| 20240126013358_create_openai_text_embedding_tables.rb | ||
| 20240202010752_add_temperature_top_p_to_ai_personas.rb | ||
| 20240207144910_fix_llm_backed_setting_defaults.rb | ||
| 20240209044519_add_user_id_mentionable_default_llm_to_ai_personas.rb | ||
| 20240213051213_add_limits_to_ai_persona.rb | ||
| 20240309034751_add_shared_ai_conversations.rb | ||
| 20240309034752_create_rag_document_fragment_table.rb | ||
| 20240313165121_embedding_tables_for_rag_uploads.rb | ||
| 20240322035907_add_images_to_ai_personas.rb | ||
| 20240404000838_add_metadata_to_rag_document_frament.rb | ||
| 20240409035951_add_rag_params_to_ai_persona.rb | ||
| 20240410170000_add_embeddings_tablesfor_bge_m3.rb | ||
| 20240424220101_add_auto_image_caption_to_user_options.rb | ||
| 20240429065155_add_consolidated_question_llm_to_ai_persona.rb | ||
| 20240503034946_add_allow_chat_to_ai_persona.rb | ||
| 20240503042558_add_chat_message_custom_prompt.rb | ||
| 20240504222307_create_llm_model_table.rb | ||
| 20240514001334_add_feature_name_to_ai_api_audit_log.rb | ||
| 20240514171609_add_endpoint_data_to_llm_model.rb | ||
| 20240527054218_add_language_model_to_ai_audit_logs.rb | ||
| 20240528132059_add_companion_user_to_llm_model.rb | ||
| 20240528144216_seed_open_ai_models.rb | ||
| 20240531205234_seed_claude_models.rb | ||
| 20240603133432_seed_other_propietary_models.rb | ||
| 20240603143158_seed_oss_models.rb | ||
| 20240606151348_create_ai_summaries_table.rb | ||
| 20240606152117_copy_summary_sections_to_ai_summaries.rb | ||
| 20240609061418_tool_details_and_command_removal.rb | ||
| 20240609232736_drop_commands_from_ai_personas.rb | ||
| 20240610232040_copy_summarization_strategy_to_ai_summarization_strategy.rb | ||
| 20240610232546_copy_custom_summarization_allowed_groups_to_ai_custom_summarization_allowed_groups.rb | ||
| 20240611170904_upgrade_pgvector_070.rb | ||
| 20240611170905_move_embeddings_to_single_table_per_type.rb | ||
| 20240611170906_drop_old_embeddings_tables.rb | ||
| 20240618080148_create_ai_tools.rb | ||
| 20240619193057_choose_llm_model_setting_migration.rb | ||
| 20240619211337_update_automation_script_models.rb | ||
| 20240624135356_llm_model_custom_params.rb | ||
| 20240624202602_add_provider_specific_params_to_llm_models.rb | ||
| 20240703135444_llm_models_for_summarization.rb | ||
| 20240704020102_reset_identity_on_ai_summary.rb | ||
| 20240708193243_fix_vllm_model_name.rb | ||
| 20240719143453_llm_model_vision_enabled.rb | ||
| 20240726164937_fix_ai_summaries_sequence.rb | ||
| 20240807150605_add_default_to_provider_params.rb | ||
| 20240909180908_add_ai_summary_type_column.rb | ||
| 20240912052713_add_target_to_rag_document_fragment.rb | ||
| 20240913054440_add_rag_columns_to_ai_tools.rb | ||
| 20241008054440_create_binary_indexes_for_embeddings.rb | ||
| 20241009230724_add_forced_tool_count_to_ai_personas.rb | ||
| 20241014010245_ai_persona_chat_topic_refactor.rb | ||
| 20241020010245_add_tool_name_to_ai_tools.rb | ||
| 20241023033955_add_feature_context_to_ai_api_log.rb | ||
| 20241023041242_add_unique_constraint_to_ai_tools.rb | ||
| 20241025135522_alter_ai_ids_to_bigint.rb | ||
| 20241028034232_add_unique_ai_stream_conversation_user_id_index.rb | ||
| 20241031145203_track_ai_summary_origin.rb | ||
| 20241031180044_set_origin_for_existing_ai_summaries.rb | ||
| 20241104053017_add_ai_artifacts.rb | ||
| 20241125132452_unique_ai_summaries.rb | ||
| 20241126033812_rename_ai_gist_batch_setting.rb | ||
| 20241128010221_add_cached_tokens_to_ai_api_audit_log.rb | ||
| 20241129190708_fix_classification_data.rb | ||
| 20241130003808_add_artifact_versions.rb | ||
| 20241206030229_add_ai_moderation_settings.rb | ||
| 20241206051225_add_ai_spam_logs.rb | ||
| 20241217164540_create_embedding_definitions.rb | ||
| 20241230153300_new_embeddings_tables.rb | ||
| 20250102035341_add_llm_quota_tables.rb | ||
| 20250110114305_embedding_config_data_migration.rb | ||
| 20250114160417_backfill_topic_embeddings.rb | ||
| 20250114160446_backfill_post_embeddings.rb | ||
| 20250114160500_backfill_rag_embeddings.rb | ||
| 20250115173456_add_highest_target_number_to_ai_summary.rb | ||
| 20250121162520_configurable_embeddings_prefixes.rb | ||
| 20250122003035_add_duration_to_ai_api_log.rb | ||
| 20250122131007_matryoshka_dimensions_support.rb | ||
| 20250125162658_fix_broken_open_ai_embeddings_config.rb | ||
| 20250127145305_clean_unused_embedding_search_indexes.rb | ||
| 20250210024600_add_rag_llm_model.rb | ||
| 20250210032345_migrate_persona_to_llm_model_id.rb | ||
| 20250211021037_add_error_to_ai_spam_log.rb | ||
