mirror of
https://gh.wpcy.net/https://github.com/discourse/discourse.git
synced 2026-05-26 01:04:55 +08:00
0a54093115
By default, Rails maintains a `db/structure.sql` file in the repository. This contains the 'current' database schema, and is intended to quickly provision new database without having to run all migrations from scratch. Historically we've avoided this for a few reasons: 1. Our migrations are sometimes used to insert **rows** into the database. These are not captured in the schema 2. When Rails restores a schema, it adds all the included migrations in `schema_migrations`, but it did not include our custom `schema_migration_details` table, which is critical to parts of our app 3. Plugins make things more complicated. We cannot use automatic dumping of the structure in development environments, because people may have different sets of plugins. This PR enables the use of `structure.sql`. Essentially: 1. Adds a `db:dump_structure` task which creates a clean `structure.sql` in a temporary database (similar to our existing `annotate:clean` for annotations). 2. Adds a `db:check_structure_dump` which runs migrations from scratch and verifies they do not introduce any rows in the database. This will be run in CI. 3. Updates CI to remove our custom database caching. The `structure.sql` loading is approximately as fast The `db:check_structure_dump` detected a number of issues, but these have now been resolved one-by-one in separate PRs. The bulk of `db:migrate` is now the 'seed' step, and the majority of that is the installation of system themes. Performance improvements there could be explored in future.
115 lines
4 KiB
YAML
Vendored
115 lines
4 KiB
YAML
Vendored
name: Migration Tests
|
|
|
|
on:
|
|
pull_request:
|
|
paths:
|
|
- ".github/workflows/migration-tests.yml"
|
|
- "migrations/**"
|
|
push:
|
|
branches:
|
|
- main
|
|
- stable
|
|
paths:
|
|
- ".github/workflows/migration-tests.yml"
|
|
- "migrations/**"
|
|
|
|
concurrency:
|
|
group: migration-tests-${{ format('{0}-{1}', github.head_ref || github.run_number, github.job) }}
|
|
cancel-in-progress: true
|
|
|
|
permissions:
|
|
contents: read
|
|
|
|
jobs:
|
|
tests:
|
|
if: github.event_name == 'pull_request' || github.repository != 'discourse/discourse-private-mirror'
|
|
name: Tests
|
|
runs-on: ${{ (github.repository_owner == 'discourse' && 'cdck-linux-8-core') || 'ubuntu-latest' }}
|
|
container: discourse/discourse_test:release
|
|
timeout-minutes: 20
|
|
|
|
env:
|
|
RAILS_ENV: test
|
|
PGUSER: discourse
|
|
PGPASSWORD: discourse
|
|
LOAD_PLUGINS: 1
|
|
|
|
steps:
|
|
- name: Set working directory owner
|
|
run: chown root:root .
|
|
|
|
- uses: actions/checkout@v6
|
|
with:
|
|
fetch-depth: 1
|
|
|
|
- name: Setup Git
|
|
run: |
|
|
git config --global user.email "ci@ci.invalid"
|
|
git config --global user.name "Discourse CI"
|
|
|
|
- name: Start redis
|
|
run: |
|
|
redis-server /etc/redis/redis.conf &
|
|
|
|
- name: Start Postgres
|
|
run: |
|
|
chown -R postgres /var/run/postgresql
|
|
sudo -E -u postgres script/start_test_db.rb
|
|
sudo -u postgres psql -c "CREATE ROLE $PGUSER LOGIN SUPERUSER PASSWORD '$PGPASSWORD';"
|
|
|
|
- name: Symlink vendor/bundle from image
|
|
run: |
|
|
ln -s /var/www/discourse/vendor/bundle vendor/bundle
|
|
|
|
- name: Setup gems
|
|
run: |
|
|
bundle config --local path vendor/bundle
|
|
bundle config --local deployment true
|
|
bundle config --local without development
|
|
bundle config --local with migrations
|
|
bundle install --jobs $(($(nproc) - 1))
|
|
|
|
- name: pnpm install
|
|
run: pnpm install --frozen-lockfile
|
|
|
|
- name: Create and migrate database
|
|
run: |
|
|
bin/rake db:create
|
|
script/silence_successful_output bin/rake db:migrate
|
|
|
|
- name: Validate IntermediateDB schema
|
|
run: |
|
|
# Show structured differences between DSL config and database
|
|
migrations/bin/cli schema diff --db=intermediate_db || true
|
|
|
|
# Regenerate schema from DSL config
|
|
migrations/bin/cli schema generate --db=intermediate_db
|
|
|
|
# Check if generated output matches what's committed
|
|
if [ ! -z "$(git status --porcelain migrations/db/intermediate_db_schema/ migrations/lib/database/intermediate_db/ migrations/lib/database/intermediate_db/enums/)" ]; then
|
|
echo ""
|
|
echo "=========================================="
|
|
echo "IntermediateDB schema is not up to date."
|
|
echo "=========================================="
|
|
echo ""
|
|
echo "The generated schema files differ from what is committed."
|
|
echo "This usually means the schema DSL config or the database"
|
|
echo "structure changed without regenerating."
|
|
echo ""
|
|
echo "To fix:"
|
|
echo " 1. Run: migrations/bin/cli schema diff"
|
|
echo " to see what changed between config and database"
|
|
echo " 2. Update config files in migrations/config/schema/intermediate_db/"
|
|
echo " - New tables: migrations/bin/cli schema add <table>"
|
|
echo " - Ignore tables: migrations/bin/cli schema ignore <table> --reason '...'"
|
|
echo " - New columns: add to the table's include list or ignore with reason"
|
|
echo " 3. Run: migrations/bin/cli schema generate"
|
|
echo " 4. Commit the updated config and generated files"
|
|
echo ""
|
|
echo "Generated file diff:"
|
|
git -c color.ui=always diff -- migrations/db/intermediate_db_schema/ migrations/lib/database/intermediate_db/ migrations/lib/database/intermediate_db/enums/
|
|
exit 1
|
|
fi
|
|
|
|
- name: RSpec
|
|
run: bin/rspec --default-path migrations/spec
|
