Six weeks. That was the entire runway between the day we were brought in and the Friday morning a national campaign would aim close to twenty times the platform's usual traffic at a system that already breathed hard on an ordinary payday. The airtime was booked, the budget was spent, and the launch date was fixed in a contract that would not move. The only thing still open to change was the architecture underneath — and only if we were both careful and quick.
The client asked us one plain question: will it hold? On the day we walked in, the honest answer was no. So the assignment narrowed from ambition to triage — deciding what to strengthen, what to postpone, and what simply to protect at all costs.
The situation
The client was a mid-sized consumer platform in the GCC — a few million registered users, a recognizable regional brand, and a small but capable engineering team that had built the product well for the load it had, not the load that was coming. The system was a single application backed by one primary database, with the checkout and account flows running synchronously against it. On a normal day this was perfectly adequate. The campaign, though, would not spread demand across a week; it would compress it into a few hours over a single weekend — and concentration, not raw volume, is what breaks systems.
The challenge
Every real constraint pointed the same way. There was no time for a rewrite and no appetite for one. The date could not move. The team was small enough that whatever we introduced had to be operable by the people already on call, not by a platoon of specialists we would leave behind. And we could not regress the experience for the customers already using the product every day. We were not being asked to build the platform the company would need in three years. We were being asked to get one weekend right without breaking the fifty-one that followed.
Our approach
We treated the launch as a capacity problem with a hard deadline, and worked it in a deliberate order rather than fixing whatever felt most urgent in the moment.
- Measure before touching anything. We built a load-test harness that replayed realistic traffic and pushed until things broke, so we knew the platform's true ceiling — not the one on the dashboard — and exactly which component failed first.
- Protect the money path. We drew a hard line around checkout and payment, and treated everything else as negotiable under pressure.
- Design for graceful degradation. The goal was never to serve every request at full richness; it was to keep the core working while non-essential features quietly stepped aside.
- Rehearse the day, not just the code. A run-book and a live game-day mattered as much as any single technical change.
What we did
With the ceiling mapped, the interventions were targeted rather than sweeping. We put a caching layer in front of the heavy read paths and moved static and media assets onto a CDN, which alone lifted a large share of load off the origin. We added read replicas so browse and reporting traffic stopped competing with writes. The checkout writes we kept synchronous; almost everything else — notifications, analytics, loyalty updates — we moved behind a queue to be processed just after the moment of peak. We wrapped fragile downstream calls in circuit breakers, and put feature flags on the non-essential parts of the experience so we could shed load in seconds instead of shipping code under fire. The stateless tier was made to scale out automatically. Then we rehearsed: a full game-day against production-like load, with the whole team watching the same dashboards they would watch on launch morning.
The outcome
| Metric | Before | After |
|---|---|---|
| Peak traffic absorbed without failure | Buckled near 2–3× normal | Held through roughly 20× normal |
| p95 response time at peak | Several seconds | Under a second |
| Error rate during the spike | Double digits | Well under 1% |
| Checkout success under load | Unreliable (~70%) | Above 99% |
| Recovery after a surge | Manual, hours | Automatic, minutes |
The campaign weekend passed as an ordinary busy day rather than an incident. Traffic arrived roughly where marketing had predicted, the platform absorbed it, and the on-call channel stayed quiet enough that the team spent the night watching graphs instead of firefighting. Just as important, nothing we added degraded the everyday experience in the weeks before or after.
What made the difference
Three things carried more weight than the rest. First, we measured before we built; the load test showed the real bottleneck was the database, not the application tier where the team's instinct had pointed — and that saved weeks. Second, graceful degradation beat brute-force capacity: the ability to shed non-essential features with a flag was cheaper and far more reliable than provisioning for a peak we could only hope to guess correctly. Third, the rehearsal was the product; the game-day surfaced two failure modes we would otherwise have met live, and it turned a nervous team into a calm one.
The architecture we left behind is not the platform's final form — but it is one that can now meet its next spike as a planned event rather than a gamble.