During this incident, customers experienced a degradation in performance for their Views in Support, as well as dropped calls in Talk. As part of preparation work for some infrastructure improvements, we saw an increase in the number of API requests to some nodes on Pod 3, causing degraded performance for our customers on that pod. To resolve the incident, we increased the number of nodes available to better handle the influx of API requests. This action, combined with a code tweak, allowed services to recover. To prevent this from reoccurring, we're improving our the API rate limiting for this particular infrastructure work. We're also ensuring better cross-training between our infrastructure and operations engineers so we can respond more quickly to this type of issue.

FOR MORE INFORMATION

For current system status information about your Zendesk, check out our system status page. During an incident, you can also receive status updates by following @ZendeskOps on Twitter. The summary of our post-mortem investigation is usually posted here a few days after the incident has ended. If you have additional questions about this incident, please log a ticket with us.