Visual testing reporter

When e2e visual testing fail, screenshots have to be updated. To facilitate the update of the screenshots directly from the CI, we have created a GitHub action AmadeusITGroup/otter/tools /github-actions/create-branch-e2e-screenshots to create a commit on the branch with updated screenshots if the visual tests have failed. To do so, we have created a Playwright reporter to retrieve the actual and expected screenshots paths when using toHaveScreenshot

Usage

** Prerequisite **

• Install @o3r/testing as a dev-dependency.

Update the reporter in your Playwright configuration.

export default defineConfig({
  reporter: [
    // ...
    ['@o3r/testing/visual-testing-reporter']
  ]
});

In case you use Yarn PnP, you have to resolve the path to the reporter.

export default defineConfig({
  reporter: [
    // ...
    [require.resolve('@o3r/testing/visual-testing-reporter')]
  ]
});

You can specify the outputFile, the default value is playwright-reports/visual-testing/report.json.

export default defineConfig({
  reporter: [
    [require.resolve('@o3r/testing/visual-testing-reporter'), { outputFile: 'visual-testing-report.json' }]
  ]
});

Limitations

The reporter will only contain the screenshots for the failed toHaveScreenshot evaluations. If you chain several ones, don't forget to use expect.soft().

[WARNING] If you use test.step(), it will stop the test execution as soon as one step is marked as failed, even if you use expect.soft().