06b32204c0Previously, the migrations tooling was a single flat `migrations/` tree, autoloaded by one global Zeitwerk loader and driven by a Thor CLI, so each planned next step had nowhere clean to land. This change splits it into four `path:`-referenced gems — `migrations-core`, `migrations-tooling`, `migrations-converters`, and `migrations-importer` — served by a single Samovar-based `disco` binary, without rewriting any domain logic. ### Why now The DSL refactor that replaced the IntermediateDB YAML config just landed, which is the cheapest moment to do this. Everything queued behind it — column coverage verification, the `discourse-migrations` validation plugin, the transformer framework, and private converter isolation — either has nowhere clean to land in the flat tree or would have to be retrofitted into a gem layout later. Doing the split now, while it's still a pure move (suite green, no domain logic touched), is far cheaper than after another round of features has built on the flat layout. ### What changes - **Four gems under `migrations/`**, all `path:`-referenced from the root `Gemfile` (nothing is published to RubyGems): `core` (CLI framework, UI, DB infrastructure, IntermediateDB, and the conversion framework), `tooling` (schema DSL and `schema` commands), `converters` (implementations and source adapters), and `importer` (row and uploads import). - **A single CLI binary:** `migrations/bin/cli` (Thor) becomes `disco` (Samovar), with each gem registering its own commands. Same surface — `convert`, `import`, `upload`, `schema generate|validate|…` — and Rails is still booted lazily. - **Isolated test suites:** each gem runs its own no-Rails specs in a new CI job, while the existing job keeps running the Rails-integration specs. |
||
|---|---|---|
| .. | ||
| bin | ||
| converters | ||
| core | ||
| docs | ||
| importer | ||
| tooling | ||
| .gitignore | ||
| .reek.yml | ||
| .rubocop.yml | ||
| AGENTS.md | ||
| CLAUDE.md | ||
| README.md | ||
Migrations Tooling
The migrations/ directory is split into four path-referenced gems:
core/—Migrations::*: CLI framework, UI, SQLite schemas, DB infrastructure, IntermediateDB models, and the conversion framework (Migrations::Conversion::*).tooling/—Migrations::Tooling::*: the schema DSL,disco schemacommands, benchmarks.converters/—Migrations::Converters::*: public converter implementations + source adapters.importer/—Migrations::Importer::*: the row importer and the uploads importer.
All four are wired into the root Gemfile via path: in the optional :migrations group.
Command line interface
The single binary is migrations/bin/disco (commands register dynamically via
Migrations::CLI::Registry). Run it without arguments — or with --help — for the
authoritative, always-current list of commands:
migrations/bin/disco --help
Rails is booted lazily: only commands that declare requires_rails! (import, upload, schema)
load the Discourse app.
Converters
Public converters live in converters/lib/migrations/converters/. To run a private
(closed-source) converter, put its code in a subdirectory of private/converters/
(or point MIGRATIONS_PRIVATE_CONVERTERS_PATH at it).
Schema DSL
The schema DSL lives in migrations/tooling/lib/migrations/tooling/schema/dsl/. Config sources
are in migrations/tooling/config/schema/. Generated artifacts (SQL, models, enums) are written
into migrations/core/.
Key files:
table_builder.rb- DSL for defining table configsschema_resolver.rb- Resolves DSL config + DB introspection into final schemaconventions_builder.rb- Global column conventions (renames, type overrides)generator.rb- Generates SQL, models, and enums from resolved schemavalidator.rb- Validates DSL configresolved_schema_validator.rb- Validates resolved schema before generation
Development
Installing gems
bundle config set --local with migrations
bundle install
Updating gems
bundle update --group migrations
Running tests
Each gem has an isolated, no-Rails suite, run from the gem directory:
cd migrations/core && bundle exec rspec
cd migrations/tooling && bundle exec rspec
cd migrations/converters && bundle exec rspec
cd migrations/importer && bundle exec rspec
Specs that need a booted Rails environment are tagged :rails. They are excluded by default and
run from the host app's bundle:
cd migrations/<gem> && BUNDLE_GEMFILE=../../Gemfile MIGRATIONS_RAILS=1 bundle exec rspec --tag rails
Linting
bin/lint path/to/file
bin/lint --fix path/to/file
Uses both rubocop and syntax_tree. Always lint changed files.
