Playwright vs Puppeteer vs Cypress vs Selenium (E2E testing)

A smooth and engaging user experience is the key to success in today's competitive software landscape. But how do you ensure your app delights users from the first tap to the last? This is where end-to-end (E2E) testing comes in. It simulates real-world user interactions, testing how different parts of your application work together.

The effectiveness of E2E testing hinges on selecting the right tool. Several options are available, including Playwright, Puppeteer, Cypress, and Selenium, each with its strengths and limitations. Selecting the most appropriate tool is crucial as it greatly influences the benefits you derive from E2E testing.

This guide aims to provide a comprehensive comparison of these tools and help you make an informed decision about optimizing your application's testing process.

Let's get started!

What is Playwright?

Playwright is a modern testing framework developed by Microsoft. It was released in 2020 and quickly gained popularity. Its remarkable traction is evident from its impressive following of over 61,000 stars on GitHub and more than 4 million weekly downloads on NPM, indicating a solid and supportive community.

Playwright operates through the DevTools Protocol, which allows smooth browser control. It supports all the major browser engines, such as Chromium, WebKit, and Firefox, and works on multiple operating systems, including Windows, Linux, and macOS.

Playwright stands out for its multi-language support, complete test isolation, and robust debugging tools.

What is Cypress?

Cypress is a testing framework designed to write, run, and debug tests efficiently. It focuses primarily on end-to-end testing and provides quick and reliable solutions for testing web applications. Cypress enables you to write tests in JavaScript, which are executed directly within your application event loop, offering direct access to elements like the Window, DOM, document, and the application instance itself.

Over the years, Cypress has built a strong reputation, achieving over 4 million weekly downloads and attracting 46,000 stars on GitHub. It supports web browsers such as Google Chrome, Mozilla Firefox, and Microsoft Edge.

What is Puppeteer?

Puppeteer is a robust end-to-end testing framework introduced in 2018. It currently has over 3 million weekly downloads on NPM.

As a Node.js library, Puppeteer allows developers to use JavaScript to create detailed tests and effectively automate browser operations. It uses the DevTools Protocol to control browsers.

Puppeteer offers a wide array of high-level APIs for tasks like capturing screenshots, debugging performance, navigating SPAs, and managing automated form submissions and keyboard inputs.

What is Selenium?

Selenium is one of the oldest testing frameworks, first introduced in 2004. It is an open-source suite of tools dedicated to end-to-end testing. It supports scripting in various languages, such as Java and Python, and all major browsers, such as Chrome, Firefox, Webkit (Safari), and Opera.

Selenium integrates with popular testing frameworks like JUnit, TestNG, and Cucumber. Its suite includes key components:

• Selenium IDE for recording user interactions
• Selenium RC for simulating interactions across browsers
• Selenium WebDriver for direct browser communication
• Selenium Grid for parallel test execution on multiple machines.

In the upcoming sections, we will compare the testing tools according to the following criteria:

1. Easy Installation
2. Performance
3. Learning curve and multilingual support
4. Debugging
5. Browser Support
6. Auto waiting
7. Retries
8. Multiple tabs
9. Test isolation
10. Scalability
11. CI/CD integration
12. Visual comparison testing
13. Test Runner Support
14. Record and Playback Support

✖ - does not support ✔ - partial support ✔✔ - full support

1. Ease of installation

Below is an evaluation of how easy it is to install each testing tool:

Playwright simplifies its installation process by requiring only the relevant programming language, like Node.js for JavaScript or Python for Python implementations. A simple command installs Playwright and automatically downloads the necessary browsers. When finished, it provides a sample test in tests directory and automatically creates a Playwright configuration file for you.

Cypress requires Node.js and offers a relatively straightforward installation process. After installing Cypress using npm, the Cypress launchpad guides you in selecting testing types (e.g., end-to-end) and browsers. Cypress then automatically generates configuration files and sample tests.

Puppeteer, similar to Cypress, requires Node.js and can be installed using the npm command. This command installs Puppeteer along with a recent version of Chrome together with a chrome-headless-shell binary. Once installed, you can quickly write the tests with your preferred editor and start running tests. Unlike Playwright or Cypress, Puppeteer does not automatically set up the directory structure; you must manually configure this aspect.

Selenium has significantly streamlined its installation process, especially since version 4.6.0. This update introduces a Selenium manager that automatically handles the downloading of browsers and drivers when you install Selenium using your programming language's package manager. Like Puppeteer, Selenium does not automatically create the directory structure; you must manually set this up.

Overall, each testing framework provides a user-friendly installation process. However, Playwright and Cypress further enhance user convenience through automated configuration setups and sample test generation.

2. Performance

Performance is key in testing, as faster execution times lead to shorter feedback loops. Based on my benchmarks, I've compiled a comparative table showing the mean execution times for various testing tools:

The benchmarks involved navigating a local application I built and checking for a specific title. Here's an example of how the test looked like using Playwright:

const { test, expect } = require("@playwright/test");

test("has title", async ({ page }) => {
  await page.goto("http://localhost:3000/");
  // Expect the title to contain a substring.
  await expect(page).toHaveTitle(/Book List/);
});

I replicated the same test across Playwright, Cypress, Puppeteer, and Selenium, using Hyperfine to run each test 20 times.

Playwright emerged as the fastest, maintaining an average execution time of about 4.513 seconds. Selenium was close behind at 4.590 seconds, followed by Puppeteer with an average of 4.784 seconds. Cypress was the slowest, averaging 9.378 seconds, which is more than double the execution time of Playwright.

If you prioritize speed, Playwright is the most efficient choice with Selenium and Puppeteer close behind.

3. Learning curve and language support

The ease of learning and support for multiple languages is essential for testing frameworks as they reduce training needs.

Playwright features an intuitive API that's incredibly user-friendly for developers. Its documentation is detailed, covering its capabilities like support for web components, shadow DOM, and iframes. Additionally, Playwright's versatility is highlighted by its compatibility with multiple programming languages, including JavaScript, Python, .NET, and Java.

Cypress also provides an easy-to-use API tailored to JavaScript, simplifying test writing. However, its limitation to JavaScript can prove limiting to teams invested in other languages. Despite this, Cypress offers a wealth of sample tests, detailed documentation, and a user-friendly launchpad for configuration.

Puppeteer provides a straightforward API and shares similarities with Playwright, making the transition between the two tools smoother. However, Puppeteer only supports JavaScript, and its documentation could be more comprehensive in certain areas.

Selenium is notable for its straightforward API and broad support for multiple programming languages, including C#, Python, Ruby, Kotlin, JavaScript, and Java. This extensive language compatibility makes it an ideal choice for varied projects, helping to ease the learning curve across different development environments.

For an end-to-end testing tool that is easy to learn and supports multiple languages, Playwright and Selenium emerge as excellent choices.

4. Debugging tools

Here is a comparison of debugging capabilities across different testing tools:

Playwright offers extensive debugging tools, including integration with the VSCode debugger, enabling direct test step-through in the popular code editor for efficient troubleshooting.

Additional advanced debugging tools enhance Playwright's debugging process:

• Playwright Inspector: A GUI that allows you to step through your tests, live edit locators, or view actionability logs.
• Trace Viewer: This GUI tool lets you examine recorded traces of your tests in Playwright.
• Browser Developer Tools: Running Playwright in debug mode enables the playwright object, which is useful for inspecting the DOM, viewing console logs, or checking network activity.
• Verbose API Logs: By setting the DEBUG environment variable, Playwright logs detailed API interactions.
• Headed Mode: Tests can run visually, with an option to slow down execution.

Cypress is also equipped with a comprehensive set of debugging tools. The Cypress app prints detailed error information and stack traces, and allows real-time command execution viewing. It also integrates pause functionality, letting users step through code or resume commands in the command log.

Additionally, the .debug() command and using a debugger statement in the Chromium browser dev tools improve its debugging capabilities.

Puppeteer offers fewer debugging tools than Playwright and Cypress. It allows users to turn off headless mode, enabling visual tracking of test executions. Tests can be slowed down to observe interactions more clearly. Additionally, integration with the Node.js debugger facilitates traditional debugging techniques, and the dumpio option can be enabled during browser launch to capture verbose output and logs.

Selenium debugging tools also pale in comparison. It primarily relies on its logging capabilities to provide extra information to aid in troubleshooting. Developers can use standard debugging techniques, such as setting breakpoints in their IDEs/editors, to step through tests, much like debugging regular programs.

When it comes to debugging, Playwright leads with its diverse range of debugging tools, which cater to various debugging needs.

5. Browser support

Stable browser support is essential for comprehensive end-to-end testing by ensuring applications work seamlessly across various browsers and viewport configurations.

Playwright offers strong cross-browser support, including for mobile viewports, with flexibility in browser updates and installations, making it suitable for testing across diverse environments.

Cypress provides stable support for Chromium browsers and Firefox, with experimental Webkit support. It's effective when Webkit compatibility isn't essential and supports easy mobile viewport setup.

Puppeteer, originally Chromium-focused, now has experimental Firefox support but lacks Webkit support, potentially limiting cross-browser compatibility testing. However, it does support device emulation and custom viewports for supported browsers.

Selenium supports all major browsers, including older ones like Internet Explorer, but mobile viewport setup requires manual coding.

For projects that need extensive cross-browser and mobile compatibility, Playwright is the best choice.

6. Auto waiting

Auto-waiting in testing frameworks ensures that actions pause until elements become interactable (clickable, visible, enabled) before proceeding. Without auto-waiting, testers rely on implicit waits (global settings) or explicit waits (specific conditions), offering varying degrees of control.

Playwright offers robust auto-waiting via locator methods like getByRole(), page.getByLabel(), and page.getByAltText(). It checks for element visibility, stability, enabled state, and editability before acting, aborting if conditions aren't met within a timeout.

Cypress also supports auto-waiting, actively monitoring page load/unload events, element animations, visibility changes, and coverage. It intelligently waits until necessary conditions are met before proceeding.

Puppeteer's current auto-wait capabilities are limited, often necessitating manual condition implementation using methods like waitForSelector(). Its experimental locators API includes auto-waiting, but it needs more stability.

Selenium lacks built-in auto-waiting, relying on implicit waits set globally. While these reduce errors, they require manual configuration and can lead to less predictable results.

Playwright and Cypress excel in auto-waiting, ensuring elements are fully prepared for interaction before continuing tests.

7. Retries

Retries are invaluable in testing frameworks, aiding auto-waiting and detecting flaky tests (those inconsistently passing/failing due to race conditions or setup bugs). Retries mitigate these issues by attempting tests multiple times.

Playwright supports automatic retries for locator methods like page.getByRole() and page.getByLabel(), which feature auto-wait. For flaky tests, global retries can be configured, or specific retry settings overridden with the --retries flag. Further control is possible by configuring retries at group (describe) and individual test (it) levels. Playwright categorizes outcomes as "passed" (successful initially), "flaky" (fails initially, succeeds on retry), and "failed" (doesn't pass after all retries).

Cypress also supports locator methods with built-in retries and allows retries to detect flaky tests. By default, tests don't retry; this must be enabled in configuration settings or for individual tests/suites.

Puppeteer lacks built-in retry convenience, requiring custom logic. While its experimental locators offer auto-retry, this doesn't extend to configuring retries for test blocks or suites.

Selenium similarly lacks built-in retry mechanisms, making handling flaky tests challenging.

For auto-retry capabilities, Playwright and Cypress are excellent choices, supporting auto-retry in locator methods and offering configurable options for effective flaky test management.

8. Multiple tabs

Handling multiple pages or tabs during testing is essential for accurately simulating real-world user interactions.

Playwright enables multiple tabs within browser contexts, simplifying concurrent interactions with various pages. For example:

// Create two pages
const pageOne = await context.newPage();
const pageTwo = await context.newPage();

// Retrieve all pages of a browser context
const allPages = context.pages();

Puppeteer offers similar capabilities, allowing for the navigation and control of several pages simultaneously within the same browser instance:

const pageOne = await browser.newPage();
const pageTwo = await browser.newPage();

// Retrieve all pages of a browser context
const allPages = browser.pages();

Selenium also has multi-tab support, though it does not differentiate between tabs and windows. You can create them using the newWindow() method:

// Opens a new tab and switches to the new tab
await driver.switchTo().newWindow('tab');
// Opens a new window and switches to it
await driver.switchTo().newWindow('window');
const browserTabs = await driver.getAllWindowHandles(); // Retrieves handles for all open tabs/windows

Cypress lacks support for multiple browser tabs, which restricts its ability to test complex user scenarios. However, support for this feature is planned for future updates.

Playwright, Puppeteer, and Selenium are the most suitable options for scenarios requiring multiple tabs.

9. Test isolation

Test isolation, crucial in end-to-end testing, ensures each test runs in a separate environment, preventing interference. This typically includes separate local storage, session storage, and cookies for each test.

Playwright excels in test isolation through its use of BrowserContexts, functioning like separate user profiles (similar to incognito mode) but lighter and faster. This allows each test to run in a fresh environment, making Playwright ideal for scenarios demanding strict separation between tests.

Cypress supports test isolation by automatically resetting browser contexts before each test. This reset ensures that every test starts with a clean slate, as outlined in their documentation. This approach helps prevent issues from one test from affecting the outcomes of another.

Puppeteer also provides Browser Contexts, which ensures isolation as elements like cookies or local storage are not shared between contexts.

In contrast, Selenium do not provide built-in support for test isolation. While it allow for the management of multiple pages, these do not automatically ensure isolation between test runs. Users typically need to implement custom solutions to achieve isolation.

10. Scalability

Scalability is crucial for testing tools, especially when managing large test suites. Here’s how each testing tool fares in terms of scaling capabilities:

Playwright automatically runs test files in parallel across available CPU cores by spawning worker processes. It also allows parallelizing tests within a single file. For larger suites, Playwright supports test sharding, distributing tests across multiple machines to reduce execution time.

Cypress also supports parallel testing. To use this feature, you must distribute your tests across multiple files. Cypress then assigns each spec file to the available machines using a load balance strategy. It's also possible to run tests in parallel on a single machine. However, Cypress documentation discourages this due to significant resource consumption.

Selenium can be scaled using Selenium Grid, which allows tests to be executed in parallel across various remote machines. Selenium Grid routes commands from the test client to different browser instances located on remote servers, optimizing the performance and speed of test execution.

Puppeteer, in contrast, lacks built-in support for parallel test execution on the same or different machines. While running tests in parallel using third-party tools or custom implementations is possible, this functionality is not supported natively.

If you need scalability, either on the same machine or across multiple machines, Playwright, Cypress, and Selenium are strong choices.

11. CI/CD integration

Integrating end-to-end testing tools into CI/CD pipelines is crucial for automating tests as it enhances software quality and ensures stability throughout development cycles. Here’s how various testing tools support CI/CD integration:

Playwright supports execution in various CI environments, with documentation providing sample configurations for major Continuous Integration providers like GitHub Actions, Azure Pipelines, CircleCI, Jenkins, and GitLab CI. This makes setting up Playwright in CI environments more straightforward. It allows for parallel testing and merging reports from multiple files into one consolidated report in CI environments. Additionally, Playwright supports environment debugging with trace recording.

Cypress is well-supported across major CI environments, such as GitHub Actions, Bitbucket Pipelines, GitLab CI, CircleCI, and AWS CodeBuild. The documentation includes real-world examples and setup guides for each environment, simplifying configuration and reporting setup.

Puppeteer integrates with CI providers like GitHub Actions, Travis, CircleCI, and GitLab CI. However, the setup documentation is brief, and practical implementation might require additional research and effort.

Selenium is compatible with nearly all Continuous Integration providers, including GitLab CI, CircleCI, and GitHub Actions. Despite its wide compatibility, the documentation lacks sample configurations or detailed guides, which may make setting up Selenium more time-consuming.

Playwright and Cypress are the best options for quickly starting with CI and easily setting up reporting and parallelizing tests.

12. Visual comparison testing

Visual comparison testing verifies the visual aspects of applications, ensuring the UI displays correctly to users. It usually involves comparing screenshots pixel-by-pixel. Here's how various testing tools support visual comparison testing:

Playwright natively supports producing and visually comparing screenshots. It generates reference screenshots that are used for comparisons in subsequent test runs and can also be updated as needed.

Cypress lacks built-in visual testing capabilities, but it can be enhanced with third-party plugins like Percy and Happo. These plugins add visual comparison functionality but require additional setup.

Puppeteer does not have built-in visual comparison capabilities but can be paired with third-party packages such as jest-image-snapshot to achieve similar results.

Selenium does not offer native support for visual comparison testing. Implementing visual testing with Selenium involves integrating third-party tools, which can be complex and time-consuming.

Overall, Playwright is the most capable and straightforward tool for visual comparison testing, offering comprehensive support for this testing method natively.

13. Test runner support

Test runners are essential for automating and managing test executions. Their compatibility with testing tools significantly impacts testing efficiency and flexibility.

Playwright comes with its test runner called Playwright Test, which includes methods like test and expect. It allows users to start testing without needing a third-party test runner. However, if desired, it can also be used with popular third-party runners like Jest, Mocha, and Pytest.

Cypress is bundled with Mocha, a popular Node.js test runner. This integration allows users to quickly start testing without the need to search for an additional test runner.

Puppeteer does not include a built-in test runner; users must integrate third-party libraries such as Jest or Mocha into their projects to manage test executions.

Selenium also lacks a built-in test runner. Like Puppeteer, users must use third-party test runners like JUnit, TestNG, NUnit, Mocha, or Jasmine.

If you are looking to get started quickly with minimal setup, Playwright leads the way as it comes with a built-in test runner. Cypress is also a strong option as it includes Mocha out of the box.

14. Record and playback support

Record and playback functionality allows you to interact with an application manually and then generate executable scripts based on those interactions, simplifying the process of creating automated tests. Here's how each testing tool supports this feature:

Playwright offers robust support for record and playback through its codegen feature, which generates executable scripts based on user interactions with the application.

Cypress provides record and playback functionality through an experimental feature called Cypress Studio. While promising, this feature is still in development and may not be as mature as other options.

Puppeteer allows for record and playback by initiating recording using the Chrome DevTools' Sources panel, enabling users to capture and replay their interactions with the application.

Selenium also offers record and playback support through the Selenium IDE, a browser extension that allows users to record interactions and generate Selenium WebDriver code.

While all frameworks support record and playback to some extent, Playwright, Puppeteer, and Selenium offer more mature and stable solutions.

Final thoughts

This article compares automation tools for end-to-end testing, including Playwright, Cypress, Selenium, and Puppeteer. Ultimately, Playwright stands out as the tool with the most features, offering high performance and versatility in nearly all scenarios. If you're interested in learning more about Playwright, we have a comprehensive guide available here .

Thanks for reading, and happy testing!

Article by

Stanley Ulili

Stanley is a freelance web developer and researcher from Malawi. He loves learning new things and writing about them to understand and solidify concepts. He hopes that by sharing his experience, others can learn something from them too!

Got an article suggestion? Let us know

Explore more

10 Best Cypress Alternatives in 2024

Cypress is a powerful front-end monitoring tool, but it's not for everyone. If you're looking for a Cypress alternative, then there are plenty of options out there.

End-to-End Testing with Playwright

Learn to learn to develop and execute Playwright test scripts, utilize its time travel debugging capabilities, and proactively identify visual regressions in this hands-on guide

The Top 10 Playwright Alternatives in 2024

Playwright is a powerful, open-source, front-end monitoring tool, but it's not right for everyone. Check out these Playwright alternatives.

Playwright Testing Essentials

Explore the basics of Playwright in this guide covering its key features and setup, with examples demonstrating end-to-end web app testing

9 Playwright Best Practices

Explore 9 essential Playwright best practices to enhance the reliability, efficiency, and effectiveness of your end-to-end tests

This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.