Scraping & asserting on page elements

On this page

Any standard Node.js script that successfully finishes an execution is a valid, passing browser check. However, in many cases, you want to validate whether the session shows the right stuff to the user.

This is a two-step process:

Identify the relevant element on the current page.
Assert that it shows what you expect.

Playwright Test offers many ways to scrape elements like buttons, forms or any arbitrary HTML element. We’ve listed the most common ones below, together with some tips on how to use them effectively.

Check out this video for a quick explainer:

When it comes to assertions, Playwright Test uses Jest’s expect library. Playwright also extends the expect library with its own, recommended web-first assertions. These will re-fetch an element and check it until the condition is met or the test times out. The full list of web-first assertions can be found here.

Scraping text values

For the text assertions, you can use expect().toHaveText() or expect().toContainText(). The first one will look for an exact match, while the latter will check for a substring match. Both methods accept regex patterns too. The example below shows how these could be used in a real-world scenario:

TypeScript

scraping.spec.ts

import { test, expect } from '@playwright/test'

test('Checkly API Docs search input has a keyboard shortcut info', async ({ page }) => {
 await page.goto('https://developers.checklyhq.com/') // 1

 const searchBox = page.getByRole('button', { name: 'Search' }) // 2

 await expect(searchBox).toContainText('CTRL-K') // 3
})

JavaScript

scraping.spec.js

const { test, expect } = require('@playwright/test')

test('Checkly API Docs search input has a keyboard shortcut info', async ({ page }) => {
 await page.goto('https://developers.checklyhq.com/') // 1

 const searchBox = page.getByRole('button', { name: 'Search' }) // 2

 await expect(searchBox).toContainText('CTRL-K') // 3
})

Ok, let’s break this down:

We require @playwright/test library, declare a test block and go to the Checkly docs page.
We use page.getByRole() and pass it two arguments:

A role of the element
An object with name property - it will narrow the search to an element with a matching text, or in this case, aria-label attribute

We expect the searchBox element to contain CTRL-K text (a keyboard shortcut info)

Scraping a list of values

Playwright Test makes it easy to work with lists of elements. The getByX() methods will return a list, if multiple elements match the parameters. You could then assert the count, text or perform additional filtering on the elements:

TypeScript

scraping-list.spec.ts

import { expect, test } from '@playwright/test'

test('Find "Advanced" sections on playwright docs page', async ({ page }) => {
  await page.goto('https://playwright.dev/docs/intro')

  const advancedSections = page.locator('.menu__link').filter({ hasText: /Advanced.+/i }) // 1, 2

  await expect(advancedSections).toHaveCount(2) // 3
  await expect(advancedSections).toHaveText(['Advanced: configuration', 'Advanced: fixtures']) // 4
})

JavaScript

scraping-list.spec.js

const { expect, test } = require('@playwright/test')

test('Find "Advanced" sections on the playwright docs page', async ({ page }) => {
  await page.goto('https://playwright.dev/docs/intro')

  const advancedSections = page.locator('.menu__link').filter({ hasText: /Advanced.+/i }) // 1,2

  await expect(advancedSections).toHaveCount(2) // 3
  await expect(advancedSections).toHaveText(['Advanced: configuration', 'Advanced: fixtures']) // 4
})

We select all elements that have the CSS class menu__link.
In the same line, we filter these elements to those that contain Advanced word
We assert the elements count
We assert that the exact text of these elements (expect().toHaveText() accepts arrays too!)

Scraping form input elements

Text input fields

You can use the locator.inputValue() method to get the text from a standard text input field, or even better, an expect(locator).toHaveValue() to assert it right away.

TypeScript

input.spec.ts

import { expect, test } from '@playwright/test'

test('Search "playwright" on the Duckduckgo search page', async ({ page }) => {
  await page.goto('https://duckduckgo.com/')

  const searchInput = page.locator('#search_form_input_homepage')

  await searchInput.type('Playwright')
  await expect(searchInput).toHaveValue('Playwright')
})

JavaScript

input.spec.js

const { expect, test } = require('@playwright/test')

test('visit page and take screenshot', async ({ page }) => {
  await page.goto('https://duckduckgo.com/')

  const searchInput = page.locator('#search_form_input_homepage')

  await searchInput.type('Playwright')
  await expect(searchInput).toHaveValue('Playwright')
})

Scraping the values of other common form elements is pretty similar to scraping text inputs. See the examples below for specific use cases.

Checkboxes:

TypeScript

checkbox.spec.ts

import { expect, test } from '@playwright/test'

test('Test Bootstrap checkbox element', async ({ page }) => {
  await page.goto('https://getbootstrap.com/docs/4.3/components/forms/#checkboxes-and-radios')

  const checkbox = page.getByLabel('Default checkbox')

  await expect(checkbox).not.toBeChecked()
  await checkbox.check()
  await expect(checkbox).toBeChecked()
})

JavaScript

checkbox.spec.js

const { expect, test } = require('@playwright/test')

test('Test Bootstrap checkbox element', async ({ page }) => {
  await page.goto('https://getbootstrap.com/docs/4.3/components/forms/#checkboxes-and-radios')

  const checkbox = page.getByLabel('Default checkbox')

  await expect(checkbox).not.toBeChecked()
  await checkbox.check()
  await expect(checkbox).toBeChecked()
})

Radio buttons:

TypeScript

radio.spec.ts

import { expect, test } from '@playwright/test'

test('Test Bootstrap radio element', async ({ page }) => {
  await page.goto('https://getbootstrap.com/docs/4.3/components/forms/#checkboxes-and-radios')

  const defaultRadio = page.getByRole('radio', { name: 'Default radio', exact: true })
  const secondDefaultRadio = page.getByRole('radio', { name: 'Second default radio', exact: true })

  await expect(defaultRadio).toBeChecked()
  await secondDefaultRadio.check()
  await expect(secondDefaultRadio).toBeChecked()
})

JavaScript

radio.spec.js

const { expect, test } = require('@playwright/test')

test('Test Bootstrap radio element', async ({ page }) => {
  await page.goto('https://getbootstrap.com/docs/4.3/components/forms/#checkboxes-and-radios')

  const defaultRadio = page.getByRole('radio', { name: 'Default radio', exact: true })
  const secondDefaultRadio = page.getByRole('radio', { name: 'Second default radio', exact: true })

  await expect(defaultRadio).toBeChecked()
  await secondDefaultRadio.check()
  await expect(secondDefaultRadio).toBeChecked()
})

TypeScript

select.spec.ts

import { expect, test } from '@playwright/test'

test('Test Bootstrap select menu', async ({ page }) => {
  await page.goto('https://getbootstrap.com/docs/4.3/components/forms/#select-menu')

  const select = page.locator('.bd-example > select.custom-select.custom-select-lg.mb-3')

  await expect(select).toHaveValue('Open this select menu')
  await select.selectOption('1')
  await expect(select).toHaveValue('1')
})

JavaScript

select.spec.js

const { expect, test } = require('@playwright/test')

test('Test Bootstrap select menu', async ({ page }) => {
  await page.goto('https://getbootstrap.com/docs/4.3/components/forms/#select-menu')

  const select = page.locator('.bd-example > select.custom-select.custom-select-lg.mb-3')

  await expect(select).toHaveValue('Open this select menu')
  await select.selectOption('1')
  await expect(select).toHaveValue('1')
})

Key takeaways are:

You can use locator.check() and expect(locator).toBeChecked() methods to interact with checkboxes and radio buttons
locator.selectOption() will select a new value. expect(locator).toHaveValue() will assert that the desired value is chosen in a select menu component.

Built-in shortcuts

Playwright Test offers some built-in shortcuts to access common elements of a typical webpage which you can use in Checkly. For the full details see the Playwright docs on Page class.

TypeScript

viewport.spec.ts

import { test } from '@playwright/test'

test('Go to google.com', async ({ page }) => {
 await page.goto('https://www.google.com/')

 // grabs the page title
 const title = await page.title()

 // grabs the page's URL
 const url = page.url()

 // return an object with the page's viewport setting
 const viewport = page.viewportSize()
 console.log('Page width is:', viewport.width)
})

JavaScript

viewport.spec.js

const { test } = require('@playwright/test')

test('Go to google.com', async ({ page }) => {
  await page.goto('https://www.google.com/')

  // grabs the page title
  const title = await page.title()

  // grabs the page's URL
  const url = page.url()

  // return an object with the page's viewport setting
  const viewport = page.viewportSize()
  console.log('Page width is:', viewport.width)
})