AI-Powered Development: Generating Ideas for your Test Cases, Bugs Reporting, and Ensuring Security
TestCraft, BetterBugs and Snyk
AContinuing with my testing of different AI tools now is the turn of 3 AI tools:
TestCraft: A browser extension that can suggest test cases, generate the e2e code, and check the Accessibility of the selected HTML element.
BetterBugs: A browser extension to capture screenshots and record videos and generates some steps to create the bug
Snyk: An AI tool to check your Git Repos and have some extensions to the most popular IDE to check your code and detect security vulnerabilities
TestCraft
With this code extension, you can set up your:
Preferred language model, for example: gpt-4o, gpt-4-turbo and gpt-3.5-turbo.
Your Automation Framework: Playwright, Cypress and Selenium,
Your favorite languages, for example, for Playwright, are JS, TS, Java, C#, and Python.
After setting up, you can open the website you want to test and Pick an element. I tested with my website for testing, Including a login with company, user, and password. I select the company.
The buttons for Generate Test Ideas, Automate, and Check Accessibility are enabled. Clicking on Generate Test Ideas opens a new window with Positive Tests, Negative Tests, and Creative Test Scenarios.
You can select some test ideas, edit the suggestions, and click Automate. It will open a new window with the code generated in your previous selection. In this case, Playwright TS. I selected the first 2 ideas, and this is the code:
// companyPage.ts
import { Page } from '@playwright/test';
export class CompanyPage {
readonly page: Page;
readonly companyInput: string;
constructor(page: Page) {
this.page = page;
this.companyInput = '#company';
}
async navigateToLogin() {
await this.page.goto('https://effizientedemo.azurewebsites.net/login');
}
async enterCompanyName(name: string) {
await this.page.fill(this.companyInput, name);
}
async getCompanyInputValue() {
return await this.page.inputValue(this.companyInput);
}
async isCompanyInputValid() {
return await this.page.evaluate(() => {
const inputElement = document.querySelector('#company');
return inputElement?.classList.contains('ng-invalid') === false;
});
}
}
// company.spec.ts
import { test, expect } from '@playwright/test';
import { CompanyPage } from './companyPage';
test.describe('Company Name Input Tests', () => {
test.beforeEach(async ({ page }) => {
const companyPage = new CompanyPage(page);
await companyPage.navigateToLogin();
});
test('Enter a valid company name and verify that the input is accepted without errors', async ({ page }) => {
const companyPage = new CompanyPage(page);
await companyPage.enterCompanyName('ValidCompany');
const inputValue = await companyPage.getCompanyInputValue();
expect(inputValue).toBe('ValidCompany');
const isValid = await companyPage.isCompanyInputValid();
expect(isValid).toBeTruthy();
});
test('Ensure that the input field accepts a variety of company name formats, including those with spaces and hyphens', async ({ page }) => {
const companyPage = new CompanyPage(page);
const testNames = ['Company Name', 'Company-Name', 'CompanyName'];
for (const name of testNames) {
await companyPage.enterCompanyName(name);
const inputValue = await companyPage.getCompanyInputValue();
expect(inputValue).toBe(name);
const isValid = await companyPage.isCompanyInputValid();
expect(isValid).toBeTruthy();
}
});
});When you click to check Accessibility, it only shows some WCAG guidelines for the element. This is an extract.
To assess the given HTML input element for accessibility issues according to WCAG 2.1, we'll evaluate it against relevant success criteria. Here's a step-by-step analysis:
### Issues####
Conformance Level A
- **Issue:** The input element lacks an associated label.
- **Criteria:** 1.3.1 Info and Relationships
- **Solution:** Ensure that the input field has a descriptive label element associated with it, either using a `<label>` element with the `for` attribute matching the input's `id` or by using `aria-labelledby`.BetterBugs
With this extension, you can record your manual test with the website with an error, and the AI assistant tool will create the bug description. If you set up the Jira integration, it will create a bug in Jira.
I logged in with an invalid user, but the page shows a loading icon and returns to the login page. I was curious about how the extension would describe the bug. And generates a summary of the steps to reproduce the bug. It includes a link to see the details of the bug, and in that link, you can see the console and network tabs, which are handy for developers.
When you upload the bug, it will create the bug in Jira and your BetterBugs account.
I updated the description of the bugs because the bug was added with many extra blank lines. And show the description and links generated.
When you open the bug on the BetterBugs website, you can see the AI debugger that helps with some ideas about the bug.
This is the text auto-generated
Suggested Solutions:
Implement stronger password validation on the server-side to ensure that only valid passwords are accepted
Enhance the login process to provide more informative error messages to the user, such as specifying that the password is incorrect
Review the login functionality and identify any potential security vulnerabilities or weaknesses that could be exploited
Test Cases:
Verify that the login process correctly rejects invalid passwords
Ensure that the error message displayed to the user clearly indicates that the password is incorrect
Test the login process with a variety of valid and invalid password combinations to ensure consistent behavior
Perform security testing to identify and address any potential vulnerabilities in the login functionality
Conclusion:
The root cause of the issue appears to be that the login functionality is accepting incorrect passwords, leading to a poor user experience and potential security concerns. Implementing stronger password validation, enhancing the error messaging, and thoroughly testing the login process will be necessary to resolve this bug. Maybe because the login shows a loading icon, the AI detects that it shows one page and returns to the login as a valid login. Although is still needed that you need to check all the tests manually, it can help with the description of the bug and some ideas about the root cause.
The price starts with the free unlimited session, and Pro starts at $4 per user per month.
Snyk
The last tool in this article is Snyk. This tool can help check vulnerabilities in your code dependencies and the code you are writing.
I tested the option to check my GitHub Repos, and it sent me a weekly email with the vulnerabilities in the dependencies of my repos.
If there is a solution, like updating your package, it will help you create a PR to fix the issues.
You can allow Snyk to create a PR to fix your vulnerabilities. This a sample
Also, you can install the VS Code extension, which will check your code in real time.
I have an API project for development, and I intentionally commented on a security practice to debug something quickly. Snyk detected this error and suggested my code comment as a solution.
On my Playwright sample, Snyk detects only a quality issue with the uft8.
So, I think it is important to try different AI tools, and although some manual verification is still needed, it can help to suggest some ideas to test, generate code, and review your code. You can use it for free as an individual developer, and the price for a team starts at $25.
You can also check this website https://www.aitesting.tools for more AI tools.
Thank you for reading my article; feel free to share and suggest new topics.