GitHub Actions: Part 1 - Run your playwright tests on GitHub

Execute and see the results of your test with GitHub

I will post articles explaining different options for executing your playwright test with GitHub actions. In this first article, I will explain how to create a pipeline to perform tests in GitHub and publish the HTML reporter.

What are GitHub actions?

GitHub actions allow you to automate your code's build, test, and app deployment with a virtual machine in GitHub or your machine.

You can use predefined templates, such as deploying a node application. You can also create your steps with a YAML file (workflow). These steps are executed automatically or with some predefined triggers. If the steps are automated, the errors with manual steps are reduced.

What are workflows?

A workflow is an automated process with steps that can be executed automatically in a yml file in some of the following scenarios:

• With every pull request

• On deployment

• On a schedule, for example, daily at 8:00 am UTC

• With an API post

You must add predefined steps to a YAML file and store it in your repository's .github/workflows directory.

You can define as many jobs and steps as you need to execute some jobs in parallel or different virtual machines and publish the results in GitHub actions or as a file that can be downloaded from the GitHub actions section.

The GitHub marketplace contains all the steps you can add, with descriptions of how to use it.

You can watch the following video with an intro to GitHub actions.

It’s also best practice to define environment variables, such as the BASE_URL, to execute your tests in the test, stage, or prod URL server. You shouldn’t store the username and passwords in your automation code for security reasons.

• A secret environment variable to store password

• An environment variable to store data that changes between test, stage, or prod servers.

You create a .env file with the variable=value. Usually, the name of the variable is in uppercase. One example is this:

BASE_URL='https://www.saucedemo.com'
USER_NAME='standard_user'
PASSWORD='secret_sauce'

You need to define these variables and secret them in Git Hub. To exclude this file, you must create a .gitignore file in the root of your test repository.

playwright-report/
test-results/
test-annotations/
report/
*.env

How to set environment and secrets variables in Github

With the following steps, I will add a variable with the base_url to change in the future to set the URL for stage, test, and prod. To the user, the password should be secret. This way, it is encrypted, and you can’t see the value again.

1. Click on settings in your GitHub repository

2. Click on Secrets and Variables

3. Click in Actions

4. Click in the Variables Tab

5. Click on the New repository variable to add the BASE_URL variable

6. Write BASE_URL as the name and https://www.saucedemo.com for the value

7. Click on Add Variable

8. Click on the Secrets Tab

9. Click on Repositorytory secret

10. Write USER_NAME in the name and standard_user for the value.

11. Repeat steps 9 and 10 and change to store the password.

To access the environment variable in your code procces.env.VARIABLE_NAME

await loginPage.loginWithUser(process.env.USER_NAME, process.env.PASSWORD);

Workflow to execute your test in Playwright.

To add a workflow to run your playwright test after each push on an Ubuntu virtual machine, you must create the main.yml file in the .github/workflows. You must checkout the code, install dependencies, execute your test, and save the reporter as an artifact.

The visual diagram is:

Diagram with a trigger event and job steps to execute your playwright tests

The yml for those steps is:

name: Tests HTML Reporter
on: [push]
jobs:
  Test:
    name: Run Tests
    runs-on: ubuntu-latest
    env: #define your environment variables
      BASE_URL: ${{vars.BASE_URL}}
      USER_NAME: ${{secrets.USER_NAME}}
      PASSWORD: ${{secrets.PASSWORD}}
    steps:
      - name: Check out repository
        uses: actions/checkout@v4

      - name: Install Node.js
        uses: actions/setup-node@v4

      - name: Install dependencies
        run: npm ci

      - name: Install playwright browsers
        run: npx playwright install --with-deps

      - name: Run playwright tests
        run: npx playwright test

      - name: Upload html report as artifact
        uses: actions/upload-artifact@v4
        if: always()
        with:
          name: html-report #artifact name
          path: playwright-report/ # relative path for the html report
          retention-days: 5 # save the report for 5 days

After the file is committed, the test will be executed each time you create a PR or commit to the main branch, and the results will be published in the Artifacts section.

You can check my sample demo with more pipelines that I will explain in future posts:

Playwright Example

Thank you for reading. Feel free to suggest any topics or questions. If the article is helpful, you can share it. Enjoy testing!!