mirror of
https://github.com/discourse/discourse.git
synced 2026-08-05 19:38:04 +08:00
d062789b0e
This is the first of a couple of PRs that add deferred post embeds (uploads, polls, events, quotes, links and mentions) to the v2 migrations tooling. The v1 bulk importer (`script/bulk_import/generic_bulk.rb`) already handles these; this builds the same thing for v2 and improves on it. These embeds can't be rendered while a post is converted. They need the `original_id -> discourse_id` maps, and those only exist later, at import time. So it's split in two. The converter drops a token into `post.raw` and stores a linkage row with the same token. The importer swaps the token for real Markdown once the maps are there. The only thing that makes this work is that the token in `raw` and the token on the row are exactly the same, so the importer just does a `gsub`. No more of the v1 whitespace padding. This PR is only the contract, schema and helpers. The Posts converter step and the Markdown converter that feeds the buffer come in a follow-up, so nothing writes these tables yet. They're held out of the reference coverage check on purpose: the rows go in through the shared `EmbedBuffer#write_for`, which the per-converter scanner can't see, so it would never count them. `EmbedBuffer`'s own spec checks that it fills every column instead. What's in here: - `Migrations::Placeholder` (core) owns the token format. A token is wrapped in a Private Use Area character (`U+E000`) that never shows up in real content, plus a random nonce per run, so it can't clash with user text or be faked. - `Migrations::Converters::EmbedBuffer` (converters) collects the embeds found while a post body is converted to Markdown and hands back a token for each one; `write_for` then inserts them into the linkage tables. It also checks the mention type against a shared `Migrations::MentionType` list, so the converter and the importer can't disagree on a spelling. - Six synthetic linkage tables, one per embed kind. - `Migrations::Importer::PlaceholderResolver` (importer) loads the linkage rows once per batch (only for the posts whose raw actually carries a token, so plain-text posts cost no queries) and does the substitution in a single `gsub` pass. Uploads, polls and events that the maps can't resolve, plus orphan tokens that have no row, get stripped and collected in sinks so they can be reported at the end of the run. The report itself lands with the Posts step. |
||
|---|---|---|
| .. | ||
| config | ||
| lib | ||
| spec | ||
| .rspec | ||
| Gemfile | ||
| migrations-importer.gemspec | ||
