Software Engineer in E-commerce: Resolving API Latency
Diagnose severe API latency and checkout timeouts during flash sales to restore platform reliability.
Context
You are a backend software engineer at an e-commerce platform. During recent promotional flash sales, the checkout service experienced response latency spikes from a normal 180 ms to over 4,200 ms, leading to a 34% drop in completed transactions and widespread gateway timeout errors (HTTP 504). The engineering lead wants a definitive resolution plan before the upcoming holiday campaign next week. You have 30 minutes to review the telemetry and propose an architectural and database remediation plan.
Your Task
Provide your technical assessment and plan in three parts:
- Recommendation — one concrete engineering decision to resolve the API latency and stabilize database performance.
- Impact — expected latency improvement, system throughput, and infrastructure cost effect.
- Implementation Plan — technical deployment phases, ownership, testing verification, and timeline before the upcoming holiday campaign.
Available Data
Service Performance During Peak Traffic (14:00 - 15:00 UTC)
| Endpoint | Normal RPS | Peak RPS | Normal P99 Latency | Peak P99 Latency | Peak Error Rate |
|---|---|---|---|---|---|
| `POST /api/v1/cart/checkout` | 320 | 1,850 | 185 ms | 4,250 ms | 18.2% |
| `GET /api/v1/orders/history` | 450 | 2,100 | 120 ms | 3,890 ms | 12.4% |
| `GET /api/v1/inventory/{sku}` | 1,200 | 8,500 | 45 ms | 65 ms | 0.2% |
| `POST /api/v1/payment/charge` | 210 | 950 | 310 ms | 1,150 ms | 3.1% |
Primary database CPU utilisation reached 96% throughout the peak window. The DevOps team initially reported that write throughput exceeded provisioned IOPS, recommending an immediate migration to a clustered database tier costing an extra $8,500 monthly. However, network egress from the primary relational database to the backend microservices remained below 35% of allocated bandwidth.