Discourse/discourse
Discourse/discourse
2
0
Fork 0
mirror of https://github.com/discourse/discourse.git synced 2025-09-05 08:59:27 +08:00
Code Packages Activity Actions 11

Improvements to mbox importer

Browse source 
* store time it took to index message in DB (to find performance issues)
* ignore listserv specific files
* better examples for split_regex
* first email in mbox shouldn't contain the split string
* always lock the DB in exclusive mode
* save email within transaction
* messages can be grouped by subject and use original order (for Listserv)
* adds option to index emails without running the import
This commit is contained in:
Gerhard Schlager 2018-01-17 12:03:57 +01:00
parent 5d7a33cd6d
commit bb54eb1192
7 changed files with 134 additions and 49 deletions
Download patch file Download diff file Expand all files Collapse all files

9
lib/email/receiver.rb
View file

@ -239,7 +239,7 @@ module Email
if text.present?
text = trim_discourse_markers(text)
text, elided_text = EmailReplyTrimmer.trim(text, true)
text, elided_text = trim_reply_and_extract_elided(text)
if @opts[:convert_plaintext] || sent_to_mailinglist_mirror?
text_content_type ||= ""
@ -255,7 +255,7 @@ module Email
markdown, elided_markdown = if html.present?
markdown = HtmlToMarkdown.new(html, keep_img_tags: true, keep_cid_imgs: true).to_markdown
markdown = trim_discourse_markers(markdown)
EmailReplyTrimmer.trim(markdown, true)
trim_reply_and_extract_elided(markdown)
end
if text.blank? || (SiteSetting.incoming_email_prefer_html && markdown.present?)
@ -265,6 +265,11 @@ module Email
end
end
def trim_reply_and_extract_elided(text)
return [text, ""] if @opts[:skip_trimming]
EmailReplyTrimmer.trim(text, true)
end
def fix_charset(mail_part)
return nil if mail_part.blank? || mail_part.body.blank?