ATL data center issue
Incident Report for AppRiver LLC
Postmortem

At 7:30 AM (Central), one of our load balancers failed. The redundant load balancer took over immediately, but a bug in the load balancer code prevented the redundancy from working properly. We had limited throughput for an extended period as we worked with the load balancer vendor to correct the issue. The extended downtime caused a large backlog of mail that took a while to process.

To prevent this issue going forward, we fixed the issue causing the load balancer to fail and tested the failover to ensure it is working properly.

Posted 7 months ago. Mar 06, 2018 - 08:47 CST

Resolved
This incident has been resolved.
Posted 7 months ago. Mar 05, 2018 - 16:16 CST
Update
Our team has isolated and resolved the issue with the ATL data center and redundancy measures have been reinforced going forward. Both inbound and outbound mail has resumed, however, delays can still be expected.
Posted 7 months ago. Mar 05, 2018 - 16:16 CST
Monitoring
Both inbound and outbound mail flow has been re-established, however, delays are to be expected due to the large volume.
Posted 7 months ago. Mar 05, 2018 - 12:42 CST
Update
Mail flow has been re-established for some customers. Our team is continuing to work to re-establish mail flow across the board.
Posted 7 months ago. Mar 05, 2018 - 11:51 CST
Update
Affected users may also be unable to access the Customer Portal during this time.
Posted 7 months ago. Mar 05, 2018 - 10:54 CST
Update
Our team is still investigating the issue. This page will be updated as soon as we have more information.
Posted 7 months ago. Mar 05, 2018 - 10:13 CST
Investigating
Our team is currently investigating an issue with one of our ATL data centers that may prevent some users from receiving email. Customers with SmartHost servers in this data center may also be unable to connect and deliver outbound mail. This issue could also affect SecureSurf customers using our Atlanta data center, including beta customers. We will continue to update this page as more information becomes available.
Posted 7 months ago. Mar 05, 2018 - 09:19 CST