AI QA Automation: From Manual Testing to a 20-Minute Release
AI QA automation is the use of AI-assisted tools to remove the repetitive, manual steps around software testing: report generation, test case creation, documentation, and release communication. At First Line Software, one QA engineer used it to rebuild a two-week release routine that used to take hours into a workflow that runs in about 20 minutes. The tool is called the Release Workflow, and it now handles regression tracking, Confluence publishing, and Zephyr test cycle creation for an ERP customer portal built on PHP and Angular. What follows is the story of how it got built, and why it took three attempts and eighteen months to get there.
What Problem Was the QA Team Trying to Solve?
The project sits in the web-to-print segment of the ERP market. Releases go out every two weeks, and the defect count stays under ten per release, thanks to a mix of manual and automated regression testing. The automated suite runs on Playwright, covering both UI end-to-end tests and API tests.
The gap was not in the tests themselves. It was everywhere around them.
Every suite ran from one laptop. Reports lived nowhere but that machine. Configuration files sat scattered, and one QA engineer carried the entire process in their head. Before every release, someone had to build a Confluence page by hand, check manual and automated numbers against each other, write an email, upload every report, and create test cases in Zephyr. None of it was difficult work. All of it was slow, and all of it had to happen again in two weeks.
Why Didn’t AI Solve This the First Two Times?
The engineer tried to build a fix alone first, without AI, and got as far as a basic prototype: a few inputs, a few dropdowns, a report panel. The project needed more ongoing time than the role allowed, so it went on the shelf for a year.
The second attempt used the SpecKit framework, and it was also the first real attempt at building with AI. The output looked far more polished than expected, with a UI that resembled a finished product. It did not work the way the engineer had planned, and daily project work pulled attention away again.
Between the second and third attempts, the engineer kept using AI, on side projects and inside daily QA work, without trying to rebuild the portal. That gap mattered. Learning how to direct an AI tool through smaller, lower-stakes tasks is what made the third attempt different.
What Changed During Innovation Week?
Some time later, a new opportunity opened up to look at the problem again. This time, the engineer had enough hands-on AI experience to see the project through. In one focused stretch, the app gained environment configuration, test statistics, and multiple ways to trigger test runs. The core reporting tool worked, but the biggest source of manual effort, the release routine itself, was still untouched.
That gap became the next target: the Release Workflow.
How Does the Release Workflow Automate a QA Release?
The Release Workflow starts from a single input: a Jira link with the release version. From there, it pulls the release date and issue count automatically and can trigger the Playwright suites directly.
The workflow handles four release tasks that used to be manual:
- Regression sync with Zephyr. One action creates a new test cycle and copies the default regression cases into it.
- Test case generation. For release stories, it creates test cases automatically and copies over reports from the related QA sub-tasks.
- Recipient management. A saved distribution list in settings lets the team add or remove people per release.
- Publish and send. The team reviews the assembled page, publishes it to Confluence, previews the email, and sends it.
| Release task | Before the Release Workflow | After the Release Workflow |
|---|---|---|
| Confluence page | Built by hand, checked manually | Assembled automatically, reviewed once |
| Numbers (manual vs. regression) | Cross-checked by one person | Pulled directly from Jira and test runs |
| Zephyr test cycle | Created and populated manually | One action copies the regression set |
| Release email | Written and sent by hand | Drafted, previewed, and sent from the tool |
| Total time per release | A few hours | About 20 minutes |
What Does This Mean for Other Engineering Teams?
The time saved is the easy number to point to, and it is not the only one that matters. Consistency is the second gain: every release page now follows the same structure, and the numbers come from the same source every time, which removes the chance of a copy-paste error creeping into a customer-facing report.
The pattern behind this story is not unique to QA. McKinsey’s research on generative AI and developer productivity found that developers using AI-based tools completed coding tasks up to twice as fast as those working without them, with the largest gains on tasks people already knew how to do but found tedious to repeat. That matches what happened here: the engineer already knew how to build a Confluence page and a Zephyr cycle by hand. AI removed the repetition, not the judgment.
First Line Software applies the same principle across its automation testing services and QA services, where AI-assisted testing has helped clients cut regression cycles while scaling test coverage. The company’s AI Lab team has documented similar patterns in reducing the cost of software testing across other client projects: the biggest savings usually sit in the process around testing, not in the test scripts themselves.
FAQ
What is First Line Software’s Release Workflow?
The Release Workflow is an internal AI-assisted tool built at First Line Software that automates QA release reporting. It pulls release data from Jira, runs Playwright test suites, syncs regression cases into Zephyr, and publishes a release page to Confluence. It replaced a manual process that took a few hours with one that runs in about 20 minutes.
How much time does it actually save?
The team’s release routine used to take a few hours per two-week cycle: building a Confluence page, checking numbers, creating Zephyr test cases, and sending an email. With the Release Workflow, the same routine takes about 20 minutes, based on the team’s own before-and-after comparison.
Which tools does the Release Workflow connect to?
It connects Jira (for release data), Playwright (for automated test execution), Zephyr (for manual test cycles and cases), and Confluence (for the published release report), along with an email step for stakeholder notifications.
Do QA engineers still need manual testing skills?
Yes. The Release Workflow automates reporting, documentation, and test cycle setup. It does not replace the judgment needed to design test cases, interpret failures, or decide what regression coverage a release actually needs.
Can a team without a dedicated developer build something similar?
The engineer behind the Release Workflow is a full-stack developer and tech lead, which shortened the build. Teams without that background can still start smaller: automating one repetitive task, such as report generation or test case creation, using AI-assisted low-code tools before attempting a full release pipeline.
The Takeaway
A senior developer with years of experience needed three attempts and about eighteen months of stalled progress to finish a side project alone. AI did not remove the need for expertise. It removed the wall that time and repetition had built around a problem the team already understood.
Every engineering team carries at least one process like this: understood, necessary, and repeated so often that nobody has time to fix it. First Line Software’s QA story shows what happens when a team finally points AI at that exact problem. Teams looking to apply the same approach to their own release or testing bottlenecks can start with First Line Software’s QA and automation testing services to see where the biggest time savings are likely to sit.
August 2026