Regression suite for a retail point-of-sale rollout

A retail group was replacing the point-of-sale software in several hundred stores, in waves, over about eighteen months. Each release touched pricing, promotions and payment, and a defect that reached a store meant either wrong prices at the till or a store that could not trade. Testing had been manual — a team of four working a spreadsheet of about six hundred cases over five days per release — which was both the bottleneck and, by the end of the week, not very careful. They needed the regression pass to take hours instead of days without lowering the bar.

Test automation lead, contracted for eighteen months across the rollout. I owned the suite, the test environment strategy, and the release gate criteria, and I sat in the go/no-go call for every wave.

I built the suite from nothing: the framework, the page objects, the fixtures, and the payment terminal simulator that made card flows testable without hardware. I owned the test data problem, which was most of the work — a point-of-sale test needs a store, a product catalogue, active promotions and a trading day, and all four have to be reset between runs. I defined the gate criteria and, more importantly, wrote down what a failure meant so a red build produced a decision rather than an argument. The daily report that goes to the rollout manager is mine.

I automated the six hundred manual cases down to about two hundred and forty automated ones, and I made the case for deleting the rest rather than converting them — most were variations that tested the same code path with different nouns. That was unpopular with the manual team until we mapped coverage properly and it held. I chose to build the payment terminal simulator instead of using shared physical hardware, which cost three weeks up front and removed the single worst source of flakiness. I also insisted the suite run against a freshly provisioned environment every time rather than a long-lived one, because a long-lived test environment accumulates state and then nobody trusts a failure.

Store rollouts were scheduled around trading calendars and could not move, so the suite had to be ready before it was comfortable. The point-of-sale vendor's software was closed and had no test hooks at all, so everything was driven through the interface. Two payment providers had sandbox environments with different reliability, and one of them was down often enough that I had to build retry semantics that did not mask real failures — which is a genuinely uncomfortable line to draw.

The regression pass went from five days to about three and a half hours, which let the group move from monthly to fortnightly releases mid-rollout. Defects reaching stores dropped from an average of just under four per wave to under one, and the two that did get through were both in areas we had consciously left manual. The suite ran for the full eighteen months and finished with a flake rate around one and a half percent, which I care about more than the pass rate.

The four manual testers moved onto exploratory testing and case design, and two of them ended up writing automated cases better than mine. The rollout manager set the gate criteria with me and defended them in calls I was not in. A developer on the vendor side quietly told me which internal endpoints existed, which made the fixtures possible.