Run your playwright tests with Github Actions (Part 1)
Execute and see the results of your test with GitHub
I will post articles explaining different options for executing your playwright test with GitHub actions. In this first article, I will explain how to create a pipeline to perform tests in GitHub and publish the HTML reporter.
What are GitHub actions?
GitHub actions allow you to automate your code's build, test, and app deployment with a virtual machine in GitHub or your machine.
You can use predefined templates, such as deploying a node application. You can also create your steps with a YAML file (workflow). These steps are executed automatically or with some predefined triggers. If the steps are automated, the errors with manual steps are reduced.
What are workflows?
A workflow is an automated process with steps that can be executed automatically in a yml file in some of the following scenarios:
On a schedule, for example, daily at 8:00 am UTC
You must add predefined steps to a YAML file and store it in your repository's .github/workflows directory.
You can define as many jobs and steps as you need to execute some jobs in parallel or different virtual machines and publish the results in GitHub actions or as a file that can be downloaded from the GitHub actions section.
The GitHub marketplace contains all the steps you can add, with descriptions of how to use it.
You can watch the following video with an intro to GitHub actions.
It’s also best practice to define environment variables, such as the BASE_URL, to execute your tests in the test, stage, or prod URL server. You shouldn’t store the username and passwords in your automation code for security reasons.
A secret environment variable to store password
An environment variable to store data that changes between test, stage, or prod servers.
You create a .env file with the variable=value. Usually, the name of the variable is in uppercase. One example is this:
BASE_URL='https://www.saucedemo.com'
USER_NAME='standard_user'
PASSWORD='secret_sauce'
You need to define these variables and secret them in Git Hub. To exclude this file, you must create a .gitignore file in the root of your test repository.
playwright-report/
test-results/
test-annotations/
report/
*.env
How to set environment and secrets variables in Github
With the following steps, I will add a variable with the base_url to change in the future to set the URL for stage, test, and prod. To the user, the password should be secret. This way, it is encrypted, and you can’t see the value again.
Click on settings in your GitHub repository
Click on Secrets and Variables
Click in Actions
Click in the Variables Tab
Click on the New repository variable to add the BASE_URL variable
Write BASE_URL as the name and https://www.saucedemo.com for the value
Click on Add Variable
Click on the Secrets Tab
Click on Repositorytory secret
Write USER_NAME in the name and standard_user for the value.
Repeat steps 9 and 10 and change to store the password.
To access the environment variable in your code procces.env.VARIABLE_NAME
await loginPage.loginWithUser(process.env.USER_NAME, process.env.PASSWORD);
Workflow to execute your test in Playwright.
To add a workflow to run your playwright test after each push on an Ubuntu virtual machine, you must create the main.yml file in the .github/workflows. You must checkout the code, install dependencies, execute your test, and save the reporter as an artifact.
The visual diagram is:
The yml for those steps is:
name: Tests HTML Reporter
on: [push]
jobs:
Test:
name: Run Tests
runs-on: ubuntu-latest
env: #define your environment variables
BASE_URL: ${{vars.BASE_URL}}
USER_NAME: ${{secrets.USER_NAME}}
PASSWORD: ${{secrets.PASSWORD}}
steps:
- name: Check out repository
uses: actions/checkout@v4
- name: Install Node.js
uses: actions/setup-node@v4
- name: Install dependencies
run: npm ci
- name: Install playwright browsers
run: npx playwright install --with-deps
- name: Run playwright tests
run: npx playwright test
- name: Upload html report as artifact
uses: actions/upload-artifact@v4
if: always()
with:
name: html-report #artifact name
path: playwright-report/ # relative path for the html report
retention-days: 5 # save the report for 5 days
After the file is committed, the test will be executed each time you create a PR or commit to the main branch, and the results will be published in the Artifacts section.
You can check my sample demo with more pipelines that I will explain in future posts:
Playwright Example
Thank you for reading. Feel free to suggest any topics or questions. If the article is helpful, you can share it. Enjoy testing!!