Revolutionizing Web & API Performance Testing: A Deep Dive into Artillery and Playwright

4 min readMay 28, 2024

In the fast-paced world of web development, ensuring your application can handle substantial user traffic is crucial. Performance testing is not just an option but a necessity to maintain reliability and user satisfaction. Enter Artillery and Playwright — two powerful tools that, when combined, offer a robust solution for load and performance testing.

Understanding the Basics: Artillery

Artillery is an open-source, modern, and powerful load-testing toolkit for developers and DevOps engineers. It is designed to test the scalability and performance of your application by simulating high traffic loads. Artillery’s key features include:

Scalability: Capable of simulating thousands of users.
Flexibility: Supports various protocols including HTTP, WebSocket, and Socket.io.
Realistic Testing: Allows the creation of complex user scenarios that mimic real-world usage.

Getting Started with Artillery

To begin using Artillery, you need to install it via npm

npm install -g artillery

Once installed, you can create a basic load test script:yml

config:
  target: "https://yourwebsite.com"
  phases:
    - duration: 60
      arrivalRate: 10
scenarios:
  - flow:
      - get:
          url: "/"

This script configures Artillery to target your website, sending 10 users per second for 60 seconds to the home page.

Enhancing Tests with Playwright

Playwright, another tool in our arsenal, is designed for end-to-end testing of web applications. It supports all modern browsers and provides a reliable way to simulate user interactions. Playwright excels in scenarios where you need to test not just the backend but the entire user experience.

Integrating Playwright with Artillery

Combining Artillery and Playwright leverages the strengths of both tools — the load generation capabilities of Artillery and the detailed user interaction simulations of Playwright. Here’s how you can integrate them:

Install Playwright:

npm install -D @playwright/test

Set Up a Playwright Test Script:

Create a Playwright script to simulate user interactions:

import { Page } from "@playwright/test";
import { loginToTheApplication, logoutFromApplication } from "../commands/login";
import loginData from "../data/loginData.json"

export async function LoginAndLogoutScript(page: Page, vuContext, events, test) {
    await test.step("Login To The Application", async () => {
        await loginToTheApplication(page, loginData.user, loginData.pass)
    });
    await test.step("Logout From The Application", async () => {
        await logoutFromApplication(page)
    });
}

Here are the login and logout command functions:

import { Page, expect } from '@playwright/test';

const usernameInputIDSelector = `#user-name`
const passwordInputIDSelector = `#password`
const loginButtonIDSelector = `#login-button`
const cartIconSelector = `[data-test="shopping-cart-link"]`
const menuButtonSelector = `#react-burger-menu-btn`
const logoutButtonSelector = `#logout_sidebar_link`

export async function loginToTheApplication(page: Page, user: string, pass: string) {
    await page.goto('https://www.saucedemo.com/');
    await page.locator(usernameInputIDSelector).fill(user)
    await page.locator(passwordInputIDSelector).fill(pass)
    await page.locator(loginButtonIDSelector).click()
    await expect(page.locator(cartIconSelector)).toBeVisible();
}

export async function logoutFromApplication(page: Page) {
    await page.locator(menuButtonSelector).click()
    await page.locator(logoutButtonSelector).click()
    await expect(page.locator(loginButtonIDSelector)).toBeVisible();
}

Use Artillery to Execute Playwright Tests:

Modify your Artillery script to execute the Playwright test:

config:
  target: https://playwright.dev/
  engines:
    playwright:
      launchOptions:
        headless: false
  processor: "./tests/artillery/loadTest.ts"
  phases:
    - name: constantArrival
      duration: 10
      arrivalRate: 2
scenarios:
  - name: Login-Logout Function Load Test
    engine: playwright
    testFunction: "LoginAndLogoutScript"

API Load Testing with Artillery

Artillery can also be used for API load testing. Here’s an example of an API load test script:

config:
  target: 'https://simple-books-api.glitch.me'
  phases:
    - name: constantArrival
      duration: 10
      arrivalRate: 1
scenarios:
  - flow:
      - post:
          url: '/orders'
          headers:
            authorization: 'Bearer be17c38da6d004328262c6f3456b7ea2ec2991b6eb673d9088eb9961b97fa0d3'
          json:
            bookId: '4'
            customerName: 'auntor'
          capture:
            json: $.orderId
            as: orderID
      - delete:
          url: '/orders/{{ orderID }}'
          headers:
            authorization: 'Bearer be17c38da6d004328262c6f3456b7ea2ec2991b6eb673d9088eb9961b97fa0d3'

Managing Scripts in a Node.js Project

This setup can be managed within a Node.js project, with all scripts maintained in the package.json file. Here’s an example package.json:

{
  "name": "webloadtesting",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "artillery run functional-artillery.yml",
    "apitest": "artillery run api-artillery.yml",
    "generateJson": "artillery run functional-artillery.yml --output results.json",
    "report": "artillery run functional-artillery.yml --output results.json && artillery report results.json",
    "dashboard": "artillery run functional-artillery.yml --record --key a9_FGcPQzaiFuRPEROfYLtw5hZZTN5X",
    "apireport": "artillery run api-artillery.yml --output results.json && artillery report results.json"
  },
  "author": "Auntor Acharja",
  "license": "ISC",
  "dependencies": {
    "@faker-js/faker": "^8.4.1",
    "@playwright/test": "^1.44.1",
    "@types/node": "^20.12.12",
    "artillery": "^2.0.13"
  }
}

Reporting and Dashboard Features

Artillery comes with built-in reporting capabilities, which include generating detailed HTML reports that provide insights into your test results. You can generate a report using the following script in package.json:

"report": "artillery run functional-artillery.yml --output results.json && artillery report results.json"

Additionally, Artillery offers a dashboard feature that allows you to view the execution of your tests in real-time with a visually appealing interface. This is particularly useful for monitoring ongoing tests and quickly identifying any performance issues. You can use the dashboard feature with the following script:

"dashboard": "artillery run functional-artillery.yml --record --key a9_FGcPQzaiFuRPEROfYLtw5hZZTN5X"

Why Combine Artillery and Playwright?

Combining Artillery and Playwright allows you to:

Simulate Realistic Load: Artillery handles the load, while Playwright manages realistic user interactions.
Identify Performance Bottlenecks: Understand how your application performs under load from both a backend and frontend perspective.
Improve User Experience: By testing actual user flows, you ensure that users receive a seamless experience even under high traffic.

Conclusion

Performance testing is critical for any web application aiming for high availability and user satisfaction. Artillery and Playwright together provide a comprehensive solution for load and performance testing, ensuring your application can handle real-world traffic and interactions effectively. By integrating these tools, you can catch potential issues early and deliver a robust, user-friendly web experience.

For more detailed information, you can check out the Artillery website. To see a practical example of this integration, watch this video of the project or visit the GitHub repository.