0
0
Fork 0
mirror of https://github.com/discourse/discourse.git synced 2026-08-11 02:59:07 +08:00
discourse/migrations/core/lib/migrations.rb
Gerhard Schlager 552868bc37 MT: Add files_db schema configuration and generated models
This is part of the uploads rework series. `disco upload` currently writes its
results into `uploads.db`, a hand-written schema where each upload is stashed as
a JSON blob (`uploads.upload`, `optimized_images.optimized_images`). That's hard
to query and doesn't line up with the schema DSL the other databases use.

The replacement is `files.db`, and I want it to store the real Discourse upload
columns instead of a blob, so an ad-hoc `SELECT` actually tells you something. So
I added a `files_db` schema config under the schema DSL and generated its SQL and
models from it.

A few things are deliberately different from `intermediate_db`:

- The conventions are minimal: no `id -> original_id` rename and no
  `*upload*_id -> text` rule. In `files.db` the ids are the real staging
  `Upload#id`/`optimized_images.id` and stay integers, and `upload_id` is a real
  integer foreign key, not a content hash. I only ignore `updated_at` globally and
  make `created_at` not required.
- `uploads` and `optimized_images` mirror the live Discourse tables. On `uploads`
  I ignore `user_id` (always the system user on the staging site; the import step
  assigns the real owner), `access_control_post_id` (staging post id, not
  portable) and `retain_hours` (not meaningful for imported files).
- `upload_results` is synthetic: one row per IntermediateDB `upload_sources` row,
  keyed by the same XXH3 hash. It carries the precomputed per-source markdown,
  the `upload_id` (nullable, since several results dedup onto one upload via sha1,
  and it's NULL when a source was skipped or failed), plus `status` and
  `skip_reason`. Those two are string enums on purpose so `skip_reason =
  'download_error'` stays readable in plain SQL.
- `downloads` is the synthetic download-cache index.

Nothing consumes `files.db` yet. A later PR switches `disco upload` over and
removes the old `uploads_db` schema; here I only add the new schema alongside it.

The generated SQL, models and enums come straight from
`migrations/bin/disco schema generate --db files_db`.
2026-07-10 21:03:14 +02:00

107 lines
2.8 KiB
Ruby
Vendored

# frozen_string_literal: true
require "active_support"
require "active_support/core_ext"
require "zeitwerk"
module Migrations
class NoSettingsFound < StandardError
end
# Each gem registers its `config/locales` directory so that `enable_i18n`
# loads the union of all translations (and an isolated gem still sees its own).
def self.locale_load_paths
@locale_load_paths ||= []
end
def self.register_locale_path(dir)
locale_load_paths << dir if locale_load_paths.exclude?(dir)
end
# Root of the `migrations-core` gem. Other gems expose their own root.
def self.root_path
@root_path ||= File.expand_path("..", __dir__)
end
# Root of the host Discourse application (the repository root). Used to lazily
# boot Rails and to discover private converters.
def self.host_app_root
@host_app_root ||= File.expand_path("../../..", __dir__)
end
def self.load_rails_environment(quiet: false)
message = "Loading Rails environment..."
print message if !quiet
rails_root = host_app_root
# rubocop:disable Discourse/NoChdir
Dir.chdir(rails_root) do
ENV["DISCOURSE_DEV_ALLOW_HTTPS"] = "1" # suppress warning
require File.join(rails_root, "config/environment")
rescue LoadError => e
$stderr.puts e.message
raise
end
# rubocop:enable Discourse/NoChdir
if !quiet
print "\r"
print " " * message.length
print "\r"
end
end
def self.loader
@loader ||=
begin
loader = Zeitwerk::Loader.new
loader.log! if ENV["DEBUG"]
configure_inflections(loader)
loader.push_dir(File.join(__dir__, "migrations"), namespace: Migrations)
configure_collapses(loader)
loader
end
end
def self.configure_inflections(loader)
loader.inflector.inflect(
{
"cli" => "CLI",
"id" => "ID",
"files_db" => "FilesDB",
"intermediate_db" => "IntermediateDB",
"mappings_db" => "MappingsDB",
},
)
end
# Collapse `common/` into the root `Migrations` namespace, matching the previous
# flat layout (e.g. `common/enum.rb` => `Migrations::Enum`). Nested directories
# such as `common/set_store/` keep contributing a namespace segment
# (`Migrations::SetStore::*`).
def self.configure_collapses(loader)
loader.collapse(File.join(__dir__, "migrations", "common"))
end
def self.setup_loader
loader.setup
end
def self.enable_i18n
require "i18n"
locale_load_paths.each { |dir| I18n.load_path += Dir[File.join(dir, "**", "migrations.*.yml")] }
I18n.backend.load_translations
# always use English for now
I18n.default_locale = :en
I18n.locale = :en
end
def self.apply_global_config
Regexp.timeout = 2
end
end
Migrations.register_locale_path(File.join(Migrations.root_path, "config", "locales"))
Migrations.setup_loader