Multi-tenant rebuild of a training records system

The provider ran safety training for industrial employers and kept the records of who was certified in what, which expires and must be renewed. Their system had been built for their largest client and then copied per customer, so there were eleven deployments drifting apart, each needing separate upgrades, and a support team that had to remember which one behaved how. Selling to a new client meant a deployment and a week of setup. The business could not grow that way, and an auditor had started asking how they guaranteed one client's records could not appear in another's report.

Contract developer for about ten months, one of six on the engineering team, reporting to a technical lead who owned the architecture and to a product manager who owned the sequencing. I worked mostly on the data layer and the migration of existing customers, and I ran the per-customer cutovers with their support lead. Two-week sprints, a fortnightly steering meeting with the client's operations director, and a change board for anything touching certification records.

I owned tenant isolation in the data layer — the scoping rules, the enforcement that makes an unscoped query fail rather than return everything, and the test suite that tries to break it. I built the migration tooling that took each of the eleven deployments into the shared system, which was eleven different shapes because they had drifted, so it was less one migration than eleven with a common spine. I owned the certification expiry engine, including the rules for grace periods and for qualifications that renew on a different cycle. I did not own the front end or the reporting module.

I pushed for isolation enforced at the data-access layer with a deliberate escape hatch rather than separate schemas per tenant. Separate schemas were the safer-feeling option and the technical lead initially preferred them, but with a hundred tenants it moves the pain to migrations, and this system changes shape often. What made it acceptable was making the unscoped path explicit and loud: it exists, it is used in exactly four places, and each one is reviewed. I also argued for migrating customers smallest-first rather than largest-first, so that the tooling was well exercised before it met the client with three hundred thousand records.

Certification records are evidence — an employer may need to prove someone was certified on a date years ago — so nothing could be destructively transformed and every migration kept a verifiable copy of the source. An external audit was scheduled mid-project and we lost about three weeks of two people's time to it. Customers had to approve their own cutover window, which meant the schedule was set by eleven separate organisations' calendars rather than ours, and two of them moved twice.

All eleven customers were on the shared system inside seven months, and onboarding a new client went from a deployment plus a week of setup to a form and about an hour. Upgrades went from eleven separate exercises to one. The isolation test suite has caught two genuine mistakes in code review since, which is the number I actually care about. The eleventh migration, the big one, ran twice — the first attempt was aborted after a mismatch in the verification step, which was exactly what the smallest-first ordering had bought us the confidence to do calmly.

The technical lead disagreed with me on the isolation approach and then made the version we shipped better by insisting on the loud escape hatch. Their support lead knew every customer's quirks and wrote the per-customer checklists that made the cutovers boring. A QA engineer built the verification comparison that aborted the eleventh migration, which I would like on the record as a good afternoon for QA.