mirror of
https://github.com/discourse/discourse.git
synced 2026-08-04 10:39:43 +08:00
5b6d87a4c9Previously, every SQL query finding translation candidates (admin progress report and backfill jobs) filtered on `LENGTH(posts.raw)`, which forced detoasting `raw` for every scanned row and made those queries slow, while the limit itself no longer guarded anything — long posts are already split into chunks sized to the model's `max_output_tokens`, and core's `max_post_length` bounds raw size. This change removes the setting and its query filters, deletes any stored override via a post-deploy migration, and bumps the translation progress cache versions since eligibility counts change. Note for review: on existing sites, posts previously excluded by the limit become eligible for backfill, so pending counts (and translation spend on long-form content) will increase accordingly.
45 lines
1.5 KiB
Ruby
Vendored
45 lines
1.5 KiB
Ruby
Vendored
# frozen_string_literal: true
|
|
|
|
module DiscourseAi
|
|
module Translation
|
|
class PostLocalizer
|
|
include LocalizableQuota
|
|
|
|
def self.localize(post, target_locale = I18n.locale, llm_model: nil)
|
|
if post.blank? || target_locale.blank? ||
|
|
LocaleNormalizer.is_same?(post.locale, target_locale) || post.raw.blank?
|
|
return
|
|
end
|
|
target_locale = target_locale.to_s.sub("-", "_")
|
|
|
|
translated_raw =
|
|
PostRawTranslator.new(text: post.raw, target_locale:, post:, llm_model:).translate
|
|
|
|
localization =
|
|
PostLocalization.find_or_initialize_by(post_id: post.id, locale: target_locale)
|
|
|
|
localization.raw = translated_raw
|
|
localization.cooked = post.post_analyzer.cook(translated_raw, post.cooking_options || {})
|
|
localization.post_version = post.version
|
|
localization.localizer_user_id = Discourse::SYSTEM_USER_ID
|
|
localization.save!
|
|
|
|
cooked_processor = LocalizedCookedPostProcessor.new(localization, post, {})
|
|
begin
|
|
cooked_processor.post_process
|
|
localization.update!(cooked: cooked_processor.html)
|
|
rescue => e
|
|
# Log but don't fail translation if post-processing (oneboxes, images) fails
|
|
Rails.logger.warn(
|
|
"Post-processing failed for localization of post #{post.id} to #{target_locale}: #{e.class} - #{e.message}",
|
|
)
|
|
end
|
|
localization
|
|
end
|
|
|
|
def self.model_name
|
|
"post"
|
|
end
|
|
end
|
|
end
|
|
end
|
