2 Copilots AI tools for testing

Github Copilot and Qodo

Image by vectorjuice on Freepik

With Meta AI included in WhatsApp and the 2024 Stack Overflow survey showing that 76% of the respondents are using or plan to use AI tools, I want to share my experience with some AI tools I tested in the last months.

I believe that AI tools, like the internet, are a significant change in the world and will be included in all software. It’s important to start using them to be more efficient. In this first article, I will focus only on the copilots I’ve used.

Benefits of AI tools

The 2024 Stack Overflow survey shows some of the benefits :

• Increase productivity/Greater efficiency: Some stats show that productivity increases because AI tools can generate data for testing, suggest code, unit test, explain code, and generate documentation. Because developers can finish faster, it is a challenge for testers who are not using some AI tool to have enough time to test all the new code, especially if there is only manual testing. It’s helpful to start with automation tests autogenerated.

• Speed up learning: Copilot tools can explain any code, or you can ask them to explain some concepts to a teenager from 13 years old. You can get answers to your specific doubts.

• Improve accuracy in code: Also, because some tools can add unit tests, the errors in the code can be detected early when it is cheaper to fix them. Other tools can review your PR automatically and suggest how to fix your code.

I use GitHub Copilot at my job, but I have also explored another tool in my personal time. Here is my summary of the tools that I am using.

Github Copilot

I have more experience as a developer than QA. As a developer, I focused more on developing software than unit testing, and it took me some years to understand the importance of unit tests.

Some developers don’t want to or don’t know how to create unit tests, so I created some unit tests in Angular and another in .net to increase the test coverage of one project and explain it to the developers in the team. I told them I would not approve any PR without good unit testing coverage because focusing on delivering increased the number of bugs and developing as fast as possible. With less experience in unit testing and I didn’t write unit tests from 5 or 6 years ago, I needed some time to:

• Research about some code errors.

• I searched Google and StackOverflow for some code syntax I didn’t remember.

• Read angular documentation, my previous angular course, and my previous notes for unit tests with .net.

I did, and I need around 10 hours to create unit tests manually and increase the coverage after comparing with Github Copilot.

I asked Github copilot to create some unit tests but only added happy paths and basic unit tests. I asked to make some negative test cases and have better test coverage.

After I asked to explain my Playwright code, it detected that it was now an E2E test with tPlaywrightght and explained it very well.

I started using Azure Data Studio for SQL, and maybe with more training, you can suggest better SQL queries.

If you are asking about different technologies or languages, it is a good idea to open a new context because sometimes, when I ask in the same chat, many questions suggest some unit test instead of Playwrightght.

Qodo (formerly CodiumAI)

I developed a hobby project to practice my coding skills and tried the new versions of .Net and Angular. Since I didn’t add any unit tests and Github Copilot can’t create good ones, I researched other tools and found CodiumAI, which has different agents for specific tasks.

Codiumate (IDE Plugin)

Add a copilot chat to VS Code with commands to /ask, /explain /enhance, /improve /docstring,

It includes a special section that creates unit tests, which you can execute and get some explanation for; it suggests a happy path and some edge cases and commands to improve, enhance, or explain your code.

I tested to create unit tests for my Angular code and improve and enhance my code. I liked that I could make some edge cases and that some options were included for free.

PR Agent (Git Plugin)

With some setup in GitHub, the primary function is to add a description to the PR and automatically review it.

For development, the plugin added some of the main files to check and some labels to classify the PR, for example, Bug Fix, and another for the effort to review.

PR Agent added description and relevant files

It also created a PR Reviewer Guide with some tips for the reviewer:

• Estimated effort to review

• Check the relevant tests

• Security concerns identified

• Key issues to review

  • There are some possible bugs.

  • Detected that PR was about code refactoring, and with a lot of refactoring, code.

  • Suggested how to improve the performance using async operations

PR Reviewer Guide

The last section is PR Code Suggestions, with the following sections:

• Best practice with the explanation and some code to fix, you can apply the fix, and if the fix is solved, it will show a green check mark

• Possible issue

• Performance

• Enhancement

PR Code Suggestions

As QA, this helps me a lot because the agent can detect any bug since PR review; if the bug is detected soon, and with some suggestions to auto-fix, the number of bugs can be reduced earlier.

I also tested with my public frameworks (Playwright, Cypress, and XCUITest) to add some changes, and the agent added the test results and explained the test error.

They will include a new agent for E2E tests in the future.

I felt like a girl with a new tool when I tested these tools.

In my next article, I will continue with some tools to create test cases, such as BlinqIO and QABunch.

Thanks for reading this article. I hope that you enjoyed it. Feel free to share if it is useful and add another tool in the comment. Enjoy the testing.