On February 11, 2019, we identified an issue that was causing customers to experience long, intermittent delays in processing and delivering emails. Customers across all filtering sets were impacted. While no customer emails were lost, many emails were delayed during high traffic volume times. During the typical surge traffic periods from February 11 through February 15th, our Dev team continued to troubleshoot the issue and we ultimately determined the delays were caused by DKIM test timeouts and the inability of the thread manager to recover expired DKIM test threads for further processing.
The code fix was deployed in the afternoon of 15 February 2019 and finally validated once the typical surge traffic resumed on the morning of 18 February 2019.