ATL data center issue

Incident Report for Zix | AppRiver

Postmortem

At 7:30 AM (Central), one of our load balancers failed. The redundant load balancer took over immediately, but a bug in the load balancer code prevented the redundancy from working properly. We had limited throughput for an extended period as we worked with the load balancer vendor to correct the issue. The extended downtime caused a large backlog of mail that took a while to process.

To prevent this issue going forward, we fixed the issue causing the load balancer to fail and tested the failover to ensure it is working properly.

Posted Mar 06, 2018 - 08:47 CST

Resolved

This incident has been resolved.

Posted Mar 05, 2018 - 16:16 CST

Update

Our team has isolated and resolved the issue with the ATL data center and redundancy measures have been reinforced going forward. Both inbound and outbound mail has resumed, however, delays can still be expected.

Posted Mar 05, 2018 - 16:16 CST

Monitoring

Both inbound and outbound mail flow has been re-established, however, delays are to be expected due to the large volume.

Posted Mar 05, 2018 - 12:42 CST

Update

Mail flow has been re-established for some customers. Our team is continuing to work to re-establish mail flow across the board.

Posted Mar 05, 2018 - 11:51 CST

Update

Affected users may also be unable to access the Customer Portal during this time.

Posted Mar 05, 2018 - 10:54 CST

Update

Our team is still investigating the issue. This page will be updated as soon as we have more information.

Posted Mar 05, 2018 - 10:13 CST

Investigating

Our team is currently investigating an issue with one of our ATL data centers that may prevent some users from receiving email. Customers with SmartHost servers in this data center may also be unable to connect and deliver outbound mail. This issue could also affect SecureSurf customers using our Atlanta data center, including beta customers. We will continue to update this page as more information becomes available.

Posted Mar 05, 2018 - 09:19 CST