[schlessera]

Addresses #392.

Scenarios that cannot run in an environment, or that belong to another configuration of the test matrix, were filtered out silently, so a green run said nothing about what was not tested.

Before the run, the scenarios that are left out are now listed together with the reason, in two groups:

• Not available in this environment, such as scenarios needing a PHP extension that is not loaded, or a database socket.
• Not part of this configuration, such as scenarios for another database, WordPress or PHP version, operating system, or known to be broken.

On GitHub Actions, the list is also added to the job summary.

Setting WP_CLI_TEST_STRICT_REQUIREMENTS stops the run before any scenario starts when the environment lacks something the scenarios need, or when the database cannot be reached, instead of leaving scenarios out or silently falling back to SQLite. This is meant for CI, where a missing requirement is a problem with the environment.

While testing this, it turned out that the database check before a run needed the test database to exist. As the scenarios drop that database themselves, a run could silently switch to SQLite just because the previous run left no test database behind. The check now verifies that the test user may create the database instead.

Example from media-command without the imagick and exif extensions:

Not running 17 of 83 scenarios:
  Not available in this environment:
    @require-extension-exif, as the PHP extension "exif" is not loaded: features/media-fix-orientation.feature:177, ...
    @require-extension-imagick, as the PHP extension "imagick" is not loaded: features/media-regenerate.feature:214, ...
  Not part of this configuration:
    @less-than-wp-5.3, as WordPress 7.1.4 is used: features/media-fix-orientation.feature:16, ...
    @broken, as they are known to be broken: features/media-regenerate.feature:1434

[coderabbitai]

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

• Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!