mirror of
https://github.com/discourse/discourse.git
synced 2026-08-05 19:53:47 +08:00
ca5a7e0616
Add a default-deny ImageMagick security policy at `config/imagemagick/policy.xml`, loaded via `MAGICK_CONFIGURE_PATH` which is set in `config/initializers/003-imagemagick.rb`. Child processes inherit the env var, so every `identify`/`magick`/`convert` we shell out to is covered, including from plugins. Discourse previously shipped an imagemagick policy in production (see `discourse/discourse_docker` repository). This change brings the policy into the core repository to make it more visible, easier to manage, and applicable for development environments. Resource limitations remain the same. The main change is to move the allowed coders to an allowlist instead of a denylist. This ensures consistent safety, regardless of the set of modules available on the system. `Upload#fix_dimensions!` is updated to explicitly set the `MSVG:` decoder. This is the decoder which Imagemagick was selecting previously for `-ping` calls. But now, since the `svg` coder isn't in our allowlist, we have to specify msvg explicitly. The pdf-to-image logic in discourse-ai (behind an experimental, hidden, and default-disabled setting) is removed, since it depended on processing pdfs via Imagemagick/ghostscript. The spec for svg rasterization via OptimizedImage is also removed. This was not used in any application code. --- https://github.com/discourse/discourse/security/advisories/GHSA-7wq5-jgww-5rw3
65 lines
1.6 KiB
Ruby
Vendored
65 lines
1.6 KiB
Ruby
Vendored
# frozen_string_literal: true
|
|
|
|
class DiscourseAi::Utils::PdfToText
|
|
MAX_PDF_SIZE = 100.megabytes
|
|
|
|
class Reader
|
|
def initialize(upload:, user: nil, llm_model: nil, execution_context: nil)
|
|
@extractor =
|
|
DiscourseAi::Utils::PdfToText.new(
|
|
upload: upload,
|
|
user: user,
|
|
llm_model: llm_model,
|
|
execution_context:,
|
|
)
|
|
@enumerator = create_enumerator
|
|
@buffer = +""
|
|
end
|
|
|
|
def read(length)
|
|
return @buffer.slice!(0, length) if !@buffer.empty?
|
|
|
|
begin
|
|
@buffer << @enumerator.next
|
|
rescue StopIteration
|
|
return nil
|
|
end
|
|
|
|
@buffer.slice!(0, length)
|
|
end
|
|
|
|
private
|
|
|
|
def create_enumerator
|
|
Enumerator.new { |yielder| @extractor.extract_text { |chunk| yielder.yield(chunk || "") } }
|
|
end
|
|
end
|
|
|
|
attr_reader :upload
|
|
|
|
def self.as_fake_file(upload:, user: nil, llm_model: nil, execution_context: nil)
|
|
Reader.new(upload: upload, user: user, llm_model: llm_model, execution_context:)
|
|
end
|
|
|
|
def initialize(upload:, user: nil, llm_model: nil, execution_context: nil)
|
|
@upload = upload
|
|
@user = user
|
|
@llm_model = llm_model
|
|
@execution_context = execution_context
|
|
end
|
|
|
|
def extract_text
|
|
pdf_path =
|
|
if upload.local?
|
|
Discourse.store.path_for(upload)
|
|
else
|
|
Discourse.store.download(upload, max_file_size_kb: MAX_PDF_SIZE)
|
|
end
|
|
|
|
raise Discourse::InvalidParameters.new("Failed to download PDF") if pdf_path.nil?
|
|
|
|
require "pdf/reader"
|
|
|
|
PDF::Reader.open(pdf_path) { |reader| reader.pages.each { |page| yield page.text } }
|
|
end
|
|
end
|
