WordPress-Coding-Standards/.github/workflows/unit-tests.yml
jrfnl 01c27cbe37
GH Actions: test against different versions of all CS dependencies
As things were, when testing against "dev" versions, only PHPCS "dev" was used, while WPCS now has more dependencies, i.e. PHPCSUtils and PHPCSExtra.

This commit updates the workflows to take that into account better.
It also improves the readability of the workflows by using multi-line command in a few places.

Notes:
* This renames the `phpcs_version` matrix variable to `dependencies` in all workflows to make it clear this is not just about PHPCS itself, but about all dependencies.
* The CS check for the own codebase is always run against "dev" versions of all dependencies as an early detection system for upstream issues.
* The "Ruleset" checks are run against low/stable/dev.
* The fixer conflict check is run against dev only, as conflicts detected cannot be fixed anymore in already released versions.
* The "quick check" now only runs against low/stable.
* The "test" run run against both low/stable by default and now has some extra builds against "dev" versions.
    - These extra builds could be set to "continue on error", but I've chosen not to do so as if any issues are detected, they should be fixed asap as they will impact WPCS sooner rather than later.
    - Code coverage will now be run against low/high PHP with low/stable dependencies and no longer against dev.
* Also note that, while the "quick test"/"test" runs shouldn't need PHPCSExtra, as that is a dependency used only in the ruleset, not in our sniff code, I've still included PHPCSExtra in the upgrade/downgrade block for consistency.

These changes do mean that the "required build"/branch protection will need to be updated (again) before the PR can be merged.
I will do so once the PR has been approved.
2023-12-13 12:17:44 +01:00

147 lines
4.8 KiB
YAML

name: Unit Tests
on:
push:
branches:
- main
pull_request:
# Allow manually triggering the workflow.
workflow_dispatch:
# Cancels all previous workflow runs for the same branch that have not yet completed.
concurrency:
# The concurrency group contains the workflow name and the branch name.
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
env:
PHPCS_DEV: 'dev-master'
UTILS_DEV: 'dev-develop'
EXTRA_DEV: 'dev-develop'
jobs:
# Runs the test suite against all supported branches and combinations.
# Linting is performed on all jobs run with dependencies `stable`.
test:
runs-on: ubuntu-latest
strategy:
matrix:
php: [ '5.5', '5.6', '7.0', '7.1', '7.2', '7.3', '7.4', '8.0', '8.1', '8.2', '8.4' ]
dependencies: [ 'lowest', 'stable' ]
extensions: [ '' ]
coverage: [false]
include:
- php: '7.4'
dependencies: 'stable'
extensions: ':mbstring' # = Disable Mbstring.
coverage: true # Make sure coverage is recorded for this too.
# Run code coverage builds against high/low PHP and high/low PHPCS.
- php: '5.4'
dependencies: 'stable'
extensions: ''
coverage: true
- php: '5.4'
dependencies: 'lowest'
extensions: ''
coverage: true
- php: '8.3'
dependencies: 'stable'
extensions: ''
coverage: true
- php: '8.3'
dependencies: 'lowest'
extensions: ''
coverage: true
# Test against dev versions of all dependencies with select PHP versions for early detection of issues.
- php: '5.4'
dependencies: 'dev'
extensions: ''
coverage: false
- php: '7.0'
dependencies: 'dev'
extensions: ''
coverage: false
- php: '7.4'
dependencies: 'dev'
extensions: ''
coverage: false
- php: '8.3'
dependencies: 'dev'
extensions: ''
coverage: false
# Add extra build to test against PHPCS 4.
#- php: '7.4'
# dependencies: '4.0.x-dev as 3.9.99'
name: PHP ${{ matrix.php }} on PHPCS ${{ matrix.dependencies }}
continue-on-error: ${{ matrix.php == '8.4' }}
steps:
- name: Checkout repository
uses: actions/checkout@v4
# With stable PHPCS dependencies, allow for PHP deprecation notices.
# Unit tests don't need to fail on those for stable releases where those issues won't get fixed anymore.
- name: Setup ini config
id: set_ini
run: |
if [ "${{ matrix.dependencies }}" != "dev" ]; then
echo 'PHP_INI=error_reporting=E_ALL & ~E_DEPRECATED, display_errors=On' >> $GITHUB_OUTPUT
else
echo 'PHP_INI=error_reporting=-1, display_errors=On' >> $GITHUB_OUTPUT
fi
- name: Set up PHP
uses: shivammathur/setup-php@v2
with:
php-version: ${{ matrix.php }}
ini-values: ${{ steps.set_ini.outputs.PHP_INI }}
coverage: ${{ matrix.coverage && 'xdebug' || 'none' }}
tools: cs2pr
- name: "Composer: set PHPCS dependencies for tests (dev)"
if: ${{ matrix.dependencies == 'dev' }}
run: >
composer require --no-update --no-scripts --no-interaction
squizlabs/php_codesniffer:"${{ env.PHPCS_DEV }}"
phpcsstandards/phpcsutils:"${{ env.UTILS_DEV }}"
phpcsstandards/phpcsextra:"${{ env.EXTRA_DEV }}"
- name: Install Composer dependencies
uses: ramsey/composer-install@v2
with:
# Bust the cache at least once a month - output format: YYYY-MM.
custom-cache-suffix: $(date -u "+%Y-%m")
- name: "Composer: downgrade PHPCS dependencies for tests (lowest)"
if: ${{ ! startsWith( matrix.php, '8' ) && matrix.dependencies == 'lowest' }}
run: >
composer update --prefer-lowest --no-scripts --no-interaction
squizlabs/php_codesniffer
phpcsstandards/phpcsutils
phpcsstandards/phpcsextra
- name: Lint PHP files against parse errors
if: ${{ matrix.dependencies == 'stable' }}
run: composer lint -- --checkstyle | cs2pr
- name: Run the unit tests without code coverage
if: ${{ matrix.coverage == false }}
run: composer run-tests
- name: Run the unit tests with code coverage
if: ${{ matrix.coverage == true }}
run: composer coverage
- name: Send coverage report to Codecov
if: ${{ success() && matrix.coverage == true }}
uses: codecov/codecov-action@v3
with:
files: ./build/logs/clover.xml
fail_ci_if_error: true
verbose: true