Table of contents

Visual comparison & snapshot testing

Playwright Test gives you the ability to do visual comparison testing (sometimes called visual regression testing) and a snapshot testing. This is useful for testing the visual appearance of your application. The TL;DR is that you can:

• Use the .toHaveScreenshot() assertion to visually compare a screenshot of your page to a golden image / reference snapshot.
• Use the .toMatchSnapshot() assertion to compare any string or Buffer value to a golden image / reference snapshot.

Starting with visual comparison testing takes just three easy steps:

1. Add a single expect(page).toHaveScreenshot() line of code to your browser check script, like the example below.

   Typescript

   import { test, expect } from '@playwright/test';
   
   test('Playwright homepage', async ({ page }) => {
      await page.goto('https://playwright.dev')
      await expect(page).toHaveScreenshot()
   })
   

   Javascript

   const { expect, test } = require('@playwright/test')
   
   test('Playwright homepage', async ({ page }) => {
      await page.goto('https://playwright.dev')
      await expect(page).toHaveScreenshot()
   })
   

2. Run your browser check. The first time you run it, you will get an error indicating that no golden image / reference snapshot exists yet.

   A snapshot doesn't exist at /tmp/19g67loplhq0j/script.spec.js-snapshots/Playwright-homepage-1-chromium-linux.png.
   

3. Generate a golden image / reference snapshot by clicking the “Run script and update golden image” option in the “run script” button.

   This will generate the golden image, which you can inspect in the “golden files” tab in the editor. You can now save your check and on each check run the golden image will be compared to the actual screenshot.

   Checkly CLI tip

   If you are using the Checkly CLI, you can also generate a golden image / reference snapshot by running the following command in your terminal:

   npx checkly test --update-snapshots
   

   Get started with the Checkly CLI ->

Now, when your check fails due to a visual difference, you will see a diff of the golden image and the actual screenshot in your check result.

To create accurate and actionable screenshot comparisons, Playwright gives you a ton of options to tweak how the .toHaveScreenshot() should behave. What are acceptable differences in color, size, position, etc.? Do you want to match the full screen, or ignore some dynamic elements that might screw up your comparison?

Let’s look at some examples.

You can control the margin of error you find acceptable between your golden image and the actual image using the following options:

• maxDiffPixelRatio: An acceptable ratio of pixels that are different to the total amount of pixels, between 0 and 1
• maxDiffPixels: A total acceptable amount of pixels that could be different.
• threshold: An acceptable perceived color difference in the YIQ color space between the same pixel in compared images, between 0 (strict) and 1 (lax).

import { test, expect } from '@playwright/test';

test('Playwright homepage', async ({ page }) => {
   await page.goto('https://playwright.dev')
   await expect(page).toHaveScreenshot({ maxDiffPixelRatio: 0.2 })
   await expect(page).toHaveScreenshot({ maxDiffPixels: 1000 })
   await expect(page).toHaveScreenshot({ threshold: 0.2 })
})

const { test, expect } = require('@playwright/test')

test('Playwright homepage', async ({ page }) => {
   await page.goto('https://playwright.dev')
   await expect(page).toHaveScreenshot({ maxDiffPixelRatio: 0.2 })
   await expect(page).toHaveScreenshot({ maxDiffPixels: 1000 })
   await expect(page).toHaveScreenshot({ threshold: 0.2 })
})

A typical homepage can have dynamic elements that change on each page load, or change based on the geographical region. Think of a “latest blog posts” section, a cookie banner or a region / language selector. Playwright allows you to ignore these elements when doing a visual comparison using the mask option and using one or more page.locator() selectors.

The example below hides the cookie banner on the Checkly docs pages.

import { test, expect } from '@playwright/test';

test('Ignore cookie banner', async ({ page }) => {
   await page.goto('https://docs.checklyhq.com')
   await expect(page).toHaveScreenshot({ mask: [page.locator('.optanon-alert-box-wrapper')] })
})

const { test, expect } = require('@playwright/test')

test('Playwright homepage', async ({ page }) => {
   await page.goto('https://docs.checklyhq.com')
   await expect(page).toHaveScreenshot({ mask: [page.locator('.optanon-alert-box-wrapper')] })
})

In some cases, any ongoing animations can cause a visual difference between your golden image and the actual screenshot. You can disable any CSS animations and transitions using the animations option.

import { test, expect } from '@playwright/test';

test('Disable animations', async ({ page }) => {
   await page.goto('https://playwright.dev')
   await expect(page).toHaveScreenshot({ animations: 'disabled' })
})

const { test, expect } = require('@playwright/test')

test('Disable animations', async ({ page }) => {
   await page.goto('https://playwright.dev')
   await expect(page).toHaveScreenshot({ animations: 'disabled' })
})

Reference the official Playwright documentation for all the options and their defaults.

Snapshot testing, using the expect(value).toMatchSnapshot(snapshotName) assertion, is a great way to test the output of any arbitrary string or Buffer value. Note that it is not optimized for visual comparison testing.

import { test, expect } from '@playwright/test'

test('Match hero text', async ({ page }) => {
   await page.goto('https://playwright.dev')
   expect(await page.textContent('.hero__title')).toMatchSnapshot('hero.txt')
})

const { test, expect } = require('@playwright/test')

test('Match hero text', async ({ page }) => {
   await page.goto('https://playwright.dev')
   expect(await page.textContent('.hero__title')).toMatchSnapshot('hero.txt')
})

Creating or updating the golden image / reference snapshot works the same as with visual comparison testing.

• Checkly currently only supports the Chromium and Chrome browsers.

• The official Playwright guide on visual comparison and snapshot testing
• The .toHaveScreenshot() API reference
• The .toMatchSnapshot() API reference

You can contribute to this documentation by editing this page on Github