🎭Playwright - Web Apps E2E Testing Tool

Prepared by: Rufo

Abstract

Essentially, as the HOV-SQA department is growing with a bunch of people learning and working with Cypress. Which is really great news in the sense that a lot of people are stepping up doing automation. I think we're limited to 10 users currently with the subscription of the Team sort of bundle. Or even if we got the Business bundle kind of subscription.

Cypress offers:

Naturally, cost is a real problem in the long run. So probably, exploring opportunities with a totally free and open source tech stack might be very advantageous for us.

However, don’t get me wrong, Cypress is still my first love with test automation but eventually we might be needing to move on to a cost-effective one, and you got it right I meant the free one. Who doesn’t love free?😆

Yet still, we should still keep in mind that not everything that is free is great, unless it is developed and maintained by Microsoft. 😅So yeah, I guess it's a wrap for me here. Looking forward to some opportunities in application with this.

​What is Playwright?

• Playwright framework is an open-source, Nodejs based automation framework for end-to-end testing. It is developed and maintained by Microsoft. Its main goal is to run across the major browser engines – Chromium, Webkit, and Firefox.

Setup & Installation

How to Install and run Playwright Test Script

Step 1: Create a fresh new directory (ex: playwright-experiment) in VSCode

Step 2: Open Directory in Visual Studio Code. From VS code,

Click on File > Open Folder > Choose newly Created Folder (playwright-experiment)

Step 3: From the VS Code, Click on Terminal Menu > Click on New Terminal

Step 4: Enter the below command to start the Playwright installation

npm init playwright@latest

Note: The above command asks a set of questions. Please provide appropriate inputs. In this tutorial, we are using typescript language.

The above command does the following operation:

• Creates Package.json

• Installs npm library

• Sets up basic Files and Folders

  • tests folder: This folder contains actual test scripts. By default, an example.spec.ts file will be created inside this folder.

  • .gitignore: This file helps if you are using git repository

  • package.json and package-lock.json: This file helps to track dependencies, create a shortcut for running tests, etc.

  • playwright.config.ts: This is the global configuration file for the Playwright, which you can configure with available options.

Step 5: Install Browsers

However, Playwright is configured to run on existing browsers, which might create issues while running tests, so it is recommended to use the Playwright browsers. Using the below command, you can install all different browsers in Playwright.

npx playwright install

The above command installs the Playwright version of Chromium, Firefox, and Webkit browser.

Installation Process is all done, now we can start on creating some test scripts.

Writing your playwright test

Scenario: Login and Logout Scenario of Identifi

import { chromium, expect, test } from "@playwright/test";

test("login and logout test", async () => {
  const browser = await chromium.launch({ headless: false });
  const context = await browser.newContext();
  const page = await context.newPage();

  await page.goto("https://app.identifi.com/login");

  await page.fill(
    'input[data-test="email.input"]',
    "[e-mail]"
  );

  await page.click("'Sign in'");

  await page.fill('input[data-test="password.input"]', "password");
  await page.click("'Sign in'");

  await expect(page).toHaveURL(/.*dashboard/);
  await page.locator("text=Logout").click();
  await expect(page).toHaveURL("https://app.identifi.com/login");
});

Debugging Tool within the VSCode:

They uniquely offers some in-built debugging tool extension in vscode, which is very cool as well to explore.

Summary of Tests/Dashboard (Freely accessible)

They offer some recordings as well:

Trace Runner feature is among my favorites with playwright:

Sample

Playwright vs Cypress (A bit of comparison)

Notable Learnings

Features	Cypress	Playwright
Multi Language Support	JavaScript, TypeScript	JavaScript, TypeScript, Java, Python, .NET
Browser Support	Chrome, Firefox(dev, nightly), Chromium based browsers, Edge, Beta & Canary Releases, Electron	Chrome, Chromium, Firefox, Webkit(Safari)
Flaky Test Detection	Dashboard(Pricing)	Configure test retry strategy, capture execution trace, videos, screenshots to eliminate flaky tests.
Community	Matured	New and smaller community
Cross-Origin/Third Party Login	NA	Very well supported
Webkit(Safari)	NA	Very well supported
Fast	Slow(Runs inside the browser, uses CDP APIs)	Much Faster (Uses Browser Context)
Page Object Pattern	Not Recommended by Cypress.	Very well supported
Parallel Browser Testing	Needs License	Very well supported using workers on single machine and on CI(Remote) - No License
Parallel Support for Tests	Needs individual test files, split tests into different files	Supports parallel tests within a single test file
Async/Await Issues	Overhead, complex method chaining, difficult to understand code, no await	Playwright fully supports async/await syntax for clean, readable code.
Edge Case Scenarios	Needs third party plugins	In Built Features
Test Grouping and Test Suites	NA	Supported
XPath	Rely on XPath Plugins	In Built Features
Dependency	Depends more on third party plugins, Overhead, Not sure long-term support, Version Upgrade, Maintenance, for almost everything, there is a plugin.	Most of the features are supported in Built. Less Dependencies.
Debugging	Removed VSC debugging, need to debug in developer tool or own debugger	Playwright inspector Paywright Trace Viewer Run in headed mode Browser developer tools Run in Debug Mode Selectors in Developers Tools Console VS Code debugger (Node JS) Verbose API logs
Automatic waits	Does not suppport in many scenarios	Claims in Built Auto Wait and Event-based mechanism
Upload and Download Files	Need third party plugins	In Built Features
User Authentication	NA	Supports reuse of authentication state
Multiple Window/Tab Support	NA	Very well supported
Pricing	Dashboard Pricing/LIcense	Free
IFrame	Limited	Very well supported

Which one is preferred?

While both of the test frameworks offer unique features and capabilities. When choosing the right testing framework, functionality remains the core focal point. Cypress offers extensive support with the matured community proactively sharing documentation and all, but still Playwright might catch up, who knows when.

We could still say though that having playwright as a totally free and open-source testing tool with their unique dashboard that is out of the box, and some other in-built features they got is a very great thing to have.