# How a 22-person QA team brought monthly regression from 5 days to 2

*An HR technology company with 550 employees and a 22-person QA team*

A workforce-management platform had a Selenium suite only six people could touch and a manual regression pass that took a week. This worked example shows how recorded flows, an imported suite and reusable steps change the effort, and where the limits are.

## Company profile

- **Company:** HR technology, workforce management (onboarding, time-off, performance reviews, payroll integrations)
- **Size:** About 550 employees, roughly 140 in engineering
- **QA team:** 22 people: 14 manual and functional testers, 6 SDETs, 2 QA leads
- **Release cadence:** Monthly release train, plus hotfixes
- **Before:** Selenium (Java) suite of 620 tests on a self-hosted Grid, plus 980 manual regression scenarios
- **Browsers:** Chrome and Edge in the pipeline; Firefox and Safari checked by hand

The product is a multi-tenant HR platform sold to mid-market and enterprise employers. Customers configure it heavily: country-specific leave policies, custom approval chains, role hierarchies and localized payroll exports. A change to a shared component, such as a table or a date picker, can surface in dozens of screens across modules.

The QA organization was split in two. Six SDETs owned the Selenium suite and the Grid. Fourteen functional testers owned the manual regression pack, which lived in a test-management tool and was executed in full before every monthly release.

## Challenges

### Automation was a six-person job in a 22-person team

The 14 functional testers knew the product and its edge cases best, but could not contribute to the Java suite. Coverage grew only as fast as six SDETs could write and fix tests, so 980 of 1,600 regression scenarios stayed manual.

### A noisy suite that people stopped trusting

On a typical nightly run, 68 of the 620 tests (11%) failed for reasons unrelated to the product: stale locators, timing on slow grids and shared test data. By midweek the team habitually re-ran failures and treated red as normal, which is how real regressions slipped through.

### UI change meant weeks of repair

A table-component upgrade the previous quarter broke about 140 tests. Three SDETs spent most of three weeks repairing locators and waits instead of adding coverage. Maintenance consumed roughly 45% of SDET time, around 432 hours a month.

### A five-day manual regression pass

Executing 980 scenarios, logging defects and retesting fixes took about 240 person-hours, plus another 40 for triaging the Selenium run. The pass took five working days, pulled testers off sprint work and pushed the release candidate date later every month.

### Thin coverage of tenant configurations

Only three of the twelve highest-volume customer configurations were exercised in regression, because covering the rest by hand was not feasible. Configuration-specific defects were mostly found by customers.

### Infrastructure to run and babysit

The self-hosted Selenium Grid cost about $1,900 a month in compute and needed regular attention from an SDET when nodes degraded.

## Why they considered TestOwl

The team did not want a second code framework. The deciding requirements were that functional testers could build tests themselves, that the existing Selenium investment could be brought across rather than rewritten, and that a shared step such as logging in as a given role could be maintained in one place.

They evaluated TestOwl against a Playwright rewrite. The rewrite would have kept automation with the six SDETs and was estimated at two quarters of their time. A trial on the onboarding and time-off modules, run by two SDETs and four functional testers, was the basis for the decision.

## How the team uses TestOwl

1. **Functional testers record the journeys they already run by hand.** A tester walks through a scenario such as submitting a leave request and approving it as a manager, and TestOwl captures the steps. They add assertions in the editor and tag the test by module and tenant configuration. SDETs review the recorded tests rather than writing them.
2. **Shared steps live in reusable Flows.** Logging in as a given role, switching tenant configuration and creating an employee record became Flows. The login Flow is used in over 900 tests. Because each Flow shows where it is used, an SDET can see the blast radius before changing it, instead of editing 900 tests.
3. **The Selenium suite was imported, not rewritten.** Of the 620 Selenium tests, 410 were imported through the import wizard after previewing what would be created. The other 210 were rebuilt as recorded flows because their original steps no longer matched how the product is used.
4. **Generated data replaced shared fixtures.** Tests create the employees, leave balances and approval chains they need through test data rules, which removed a class of failures caused by tests sharing and mutating the same records.
5. **Scheduled runs and a weekly review of healed steps.** The full suite runs on weekdays on TestOwl Cloud with 16 parallel workers, about 25,000 of the plan’s 35,000 monthly cloud runs, and ad-hoc and release-candidate runs use local execution. When the UI changes, self-healing adjusts the locator and the report shows what healed. An SDET reviews healed steps once a week and accepts or corrects them.

## Implementation

- **Pilot (Weeks 1–2):** Two SDETs and four testers recorded 60 flows across onboarding and time-off, built the first shared Flows and compared results with the Selenium run for the same scenarios.
- **Import and convert (Weeks 3–6):** Imported 410 Selenium tests, rebuilt 210, and converted the highest-value manual scenarios into recorded flows. Testers were paired with an SDET for their first three flows, then worked independently.
- **Remaining modules and cutover (Weeks 7–10):** Covered the remaining modules, set up the weekday schedule, ran TestOwl in parallel with the old process for one release to compare results, then retired the Grid.

## What changed

- Fourteen testers who could not contribute to automation now build and maintain tests, so automated coverage rose from 620 to 1,150 scenarios (39% to 72% of the regression pack) without hiring.
- Shared steps are maintained once in Flows, and locator breakage is handled by self-healing plus a weekly review, rather than by weeks of repair after a UI change.
- Manual regression shrank from 980 to 450 scenarios, the ones that need human judgement: layout, document output and unusual configurations.
- The self-hosted Grid was retired.

## Measurable outcomes

| Measure | Before | After | Change |
| --- | --- | --- | --- |
| Automated regression scenarios | 620 of 1,600 (39%) | 1,150 of 1,600 (72%) | +530 scenarios |
| Manual regression scenarios per cycle | 980 | 450 | −54% |
| Regression cycle, elapsed | 5 working days | 2 working days | −60% |
| Person-hours per regression cycle | 280 h | 122 h | −56% |
| Automated suite runtime | ~4 h (620 tests, 8 Grid nodes) | ~3 h (1,150 tests, 16 workers) | 1.85× the tests in less time |
| Non-product failures per nightly run | 68 of 620 (11%) | 28 of 1,150 (2.4%) | −78% rate |
| Test maintenance, hours per month | 432 h | 150 h | −65% |
| Maintenance hours per test per month | 0.70 h | 0.13 h | −81% |
| SDET time spent on maintenance | ~45% | ~16% | −29 pts |
| Customer configurations in regression (of 12) | 3 | 9 | +6 |
| Self-hosted test infrastructure | $1,900 / month | $0 | Grid retired |

### How it was measured

- **Measurement period:** Baseline: the six monthly releases before rollout. After: the six monthly releases following the 10-week rollout. Figures are per-cycle or per-month averages, annualized over 12 releases.
- **Team measured:** 22 QA staff (14 functional testers, 6 SDETs, 2 leads). Regression hours cover testers and SDETs; QA leads are excluded.
- **Regression hours, before:** 980 scenarios × 12 min = 196 h execution, + 44 h data setup, defect logging and retest = 240 h, + 40 h triaging the Selenium run = 280 h.
- **Regression hours, after:** 450 scenarios × 12 min = 90 h execution, + 20 h setup and retest = 110 h, + 12 h triaging failures and reviewing healed steps = 122 h.
- **Maintenance hours:** 6 SDETs × 160 h/month = 960 h. Before: 45% = 432 h. After: 150 h (about 16%), including a weekly review of healed steps.
- **Rates and capacity:** $58 blended fully loaded hourly cost; 1,800 productive hours per FTE. Hours saved are capacity redeployed to other QA work, not headcount removed.
- **Not included:** About 420 person-hours of one-time rollout effort (about $24,400 at $58/h) are excluded from the annual figure.

### How the savings were calculated

| Item | Calculation | Annual |
| --- | --- | --- |
| Regression effort | (280 h − 122 h) = 158 h × 12 cycles | 1,896 h |
| Maintenance effort | (432 h − 150 h) = 282 h × 12 months | 3,384 h |
| Total capacity freed | 1,896 h + 3,384 h = 5,280 h (≈ 2.9 FTE at 1,800 h) | 5,280 h |
| Value of freed capacity | 5,280 h × $58 | $306,240 |
| Selenium Grid retired | $1,900 × 12 months | $22,800 |
| TestOwl subscription | Scale $1,999 + 7 extra seats × $59 + 4 extra workers × $79 = $2,728 × 12 | −$32,736 |
| Estimated net annual value | $306,240 + $22,800 − $32,736 | $296,304 |

Only the $22,800 of Grid spend is cash. The rest is capacity valued at the blended hourly rate. The subscription uses list pricing for the Scale plan plus seat and worker add-ons, so check current add-on rates before reusing this model. One-time rollout effort is recovered in roughly the first month of steady state.

## Business impact

- The release candidate is ready about three working days earlier each month, and the date has held. The team chose to use that time for a longer bug-fix window rather than to ship more often.
- About 2.9 FTE of capacity moved from repetitive execution and test repair to exploratory testing and tenant-configuration coverage, which is why nine of twelve high-volume configurations are now covered.
- Nightly results are credible again: with a 2.4% non-product failure rate, a red run is investigated rather than re-run.
- In month four of the measurement window a navigation redesign touched about 38% of screens. 412 tests had at least one affected step; 355 (86%) healed without an edit, and the remaining 57 were corrected by three people in about two days. The comparable component upgrade before rollout had taken three SDETs most of three weeks.

## Lessons learned

- **Do not import everything.** A third of the old suite tested flows that no longer matched the product. Rebuilding 210 tests as recorded flows cost less than carrying them forward.
- **Put SDETs in a reviewer role early.** The biggest change was moving SDETs from authors to reviewers of Flows and healed steps. Teams that keep SDETs as the only builders will not see the coverage gain.
- **Run old and new in parallel for one release.** Comparing results for a full cycle gave the QA leads evidence to retire the Grid and trim the manual pack with confidence.
- **Budget time for the weekly healed-step review.** Self-healing reduces repair work but does not remove the need to confirm that a healed step still checks the right thing. About an hour per SDET per week is included in the 150 maintenance hours.

### What did not improve

- The 450 remaining manual scenarios are real work. Printed payslips, PDF output and some country-specific edge cases still rely on human review.
- Cross-browser coverage for Firefox and Safari remained a periodic check rather than part of every nightly run.
- The rollout itself took ten weeks and about 420 person-hours, and the first two weeks produced little visible saving.

## Related TestOwl pages

- [Import test scripts and suites](https://testowl.ai/features/test-case-import): How existing Selenium tests are previewed and converted.
- [Self-healing tests](https://testowl.ai/features/self-healing): What heals, what is reported and what still needs review.
- [Automated regression testing](https://testowl.ai/automated-regression-testing): What to automate first and why suites decay.
- [Selenium alternatives](https://testowl.ai/selenium-alternatives): The locator maintenance cost and a migration roadmap.