Testing Applications with Playwright

How to set up end-to-end testing of a Vaadin application with Playwright.

Set Up Playwright
First Playwright Test
Selecting Components
Running the Test
More about Playwright

Playwright is an open source end-to-end testing library developed by Microsoft. Similarly to Selenium, it provides Java API to simulate user interactions via browsers. Compared to TestBench it lacks a high-level API for Vaadin components and it doesn’t automatically wait for the client-server communication to stabilize after user interaction.

Still, this page describes how to set up Playwright for a Spring Boot-based Vaadin application. It also shows how to create a simple Playwright test and the basics of finding and interacting with components in a Playwright test.

Although these instructions are Spring Boot specific, the same principles can be adapted to other architectures. The primary difference is that you need to control the starting and stopping of the server in a different manner. For example, you can use the pre-integration-test and post-integration-test phases — as shown in Getting Started with End-to-End Testing. You can also adopt the @SpringBootTest based approach described here for Testing with Selenium.

Set Up Playwright

To use Playwright in a Vaadin application, you need to perform a few steps. First, add the Playwright dependency to your project’s pom.xml file like so:

Source code

pom.xml
<dependency>
    <groupId>com.microsoft.playwright</groupId>
    <artifactId>playwright</artifactId>
    <version>1.39.0</version> <!-- Check the latest version via https://playwright.dev -->
    <scope>test</scope>
</dependency>

Next, make sure you have the spring-boot-starter-test module in your project. Spring Boot’s test starter module brings in, for example, JUnit 5 and Spring’s own tools that you may use to control starting the server for tests. Below is an example of how you would add this dependency:

Source code

pom.xml
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-test</artifactId>
    <scope>test</scope>
</dependency>

You might already be enabling this dependency, though. Don’t duplicate it.

First Playwright Test

To create your first Playwright test, you’ll first need a view to test. Add or modify your main view to be like this:

Source code

MainView.java
@Route
public class MainView extends VerticalLayout {

    public MainView() {
        var msg = new Paragraph("");
        // Add id to make testing easier
        msg.setId("msg");

        Button button = new Button("Click me", e -> {
            msg.setText("Clicked!");
        });

        add(button, msg);
    }
}

The above view is the same as generated by vaadin-archetype-spring-application. However, it also includes an identifier for the added paragraph. Adding identifiers to components generally helps with writing and maintaining end-to-end tests.

Next, add a Java class to your test directory (e.g., srs/test/java/org/example/) that tests the MainView works properly. You would do that like this:

Source code

TrivialPlaywrightTest.java
package org.example;

import com.microsoft.playwright.Browser;
import com.microsoft.playwright.Page;
import com.microsoft.playwright.Playwright;
import org.junit.jupiter.api.Tag;
import org.junit.jupiter.api.Test;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.boot.test.context.SpringBootTest.WebEnvironment;
import org.springframework.boot.test.web.server.LocalServerPort;

import static com.microsoft.playwright.assertions.PlaywrightAssertions.assertThat;

@SpringBootTest(webEnvironment = WebEnvironment.RANDOM_PORT) 1
@Tag("playwright") 2
public class TrivialPlaywrightTest {

    // This will be injected with the random free port
    // number that was allocated
    @LocalServerPort 3
    private int port;

    static Playwright playwright = Playwright.create(); 4

    @Test
    public void testClicking() {
        Browser browser = playwright.chromium().launch(); 5
        Page page = browser.newPage();
        page.navigate("http://localhost:" + port + "/");

        assertThat(page.getByText("Click Me")).isVisible(); 6

        page.locator("//vaadin-button[contains(text(),'Click me')]").click(); 7

        assertThat(page.locator("#msg")).containsText("Clicked!"); 8

    }
}

Instructs Spring Boot to instantiate a full context with the web server before running this test. A random port for the web server is used so that you can keep a development server running and so it doesn’t disturb the execution of end-to-end tests.
End-to-end tests may be slow to execute. It’s a good idea to tag them with the standard JUnit 5 annotation so that you can include or exclude them when you want.
Inject the random port to a variable so that it’s available for tests.
Playwright environment initialization only needs to be executed once.
Launch a new Chromium browser, open a page and navigate to the local port where Spring Boot started the application.
Playwright’s assert methods do "implicit wait", automatically. When Playwright gets to the Click Me text locator, the element probably won’t be there yet as loading of the single-page Vaadin web application takes some milliseconds. The assertion still passes as Playwright will wait a while for the text to be present.
Use an XPath selector to check for the element name and that it contains the text, "Click Me". On the locator, simulate user action with the click method.
Asserts that there is an element with id msg in the page that contains the text, Clicked!. If you get instead the text using page.locator("#msg").textContent() and assert using standard JUnit API, it might fail as the server round-trip response might not yet be completed. Again, using the assertion method from Playwright helpers gives a bit of time for a single-page web application to render the response. Alternatively, you could add, for example, a page.getByText("Clicked!").waitFor(); line before the assertion to ensure the server round-trip has been completed.

Selecting Components

Vaadin components are Web Components, so part of their structure lives in shadow DOM and part in slotted light DOM. Because Playwright has no Vaadin-specific element API, a couple of cases trip up hand-written selectors.

Dialog Action Buttons Are Slotted

A [classname]`ConfirmDialog’s confirm, cancel, and reject buttons are slotted light-DOM children, not part of its shadow DOM. Target the slotted button directly:

Source code

Javapage.locator("vaadin-confirm-dialog vaadin-button[slot='confirm-button']").click();
// also: [slot='cancel-button'] and [slot='reject-button']

A visibility check based on offsetParent skips slotted content and misses these buttons, and synthesizing a click() on a guessed wrapper element is unreliable — especially when a confirm dialog is stacked over another overlay, such as an editor, where you need the real button. See Confirm Dialog styling for the full list of slot selectors.

Grid Cell Content Is Recycled

Grid renders each cell’s content into a vaadin-grid-cell-content element and reuses these elements as rows scroll or the grid refreshes. Detached copies of previously rendered content can linger in the document, so a broad query such as document.querySelectorAll('vaadin-grid-cell-content') may report rows that the user no longer sees.

Read grid state from the live cells only — scope the query to the grid and let Playwright match attached, visible elements — rather than enumerating every cell-content element in the document:

Source code

Java// Matches only attached, visible cell content within the grid
Locator cells = page.locator("vaadin-grid vaadin-grid-cell-content:visible");

Alternatively, wait for the refresh to settle (for example, with an explicit waitFor()) before asserting, or reload the page to clear detached content.

Running the Test

As the test is annotated with the JUnit 5 @Test annotation, the most natural way to run it is via an IDE. Also, the test is picked up by convention if you call it like so:

Source code

terminalmvn test

If you had previously written some unit tests for your project, you probably noticed that execution time increased by a couple of seconds. This is natural as a full server is started and Playwright launches a browser to execute the test.

You can use standard JUnit 5 and Maven features to include or exclude tests. As there is the playwright tag in the test, your can execute only the fast unit tests by executing the following:

Source code

terminalmvn test -DexcludedGroups="playwright"

More about Playwright

For more information about using Playwright, check out these pages:

Check out the Flow-Hilla hybrid example project on GitHub which provides hands-on examples for testing a view written in Java and another written in TypeScript.

A8496E86-4D72-11EE-BE56-0242AC120002

Docs

Docs

Testing Applications with Playwright

Set Up Playwright

First Playwright Test

Selecting Components

Dialog Action Buttons Are Slotted

Grid Cell Content Is Recycled

Running the Test

More about Playwright