Status Page
Identified -

Unless you are using a third-party app on your tenant that is configured with Azure AD Graph API, you likely will only see Microsoft apps listed on your tenant regarding this notice, such as “Microsoft Office”. If you do have a third-party app listed, you will need to work with that vendor who supplies that app to confirm they have an upgrade available to Microsoft Graph API. If you only see “Microsoft Office” as the listed app, see the information below:



 



Please see the following Microsoft blog for full details:



Action required: Azure AD Graph API retirement | Microsoft Community Hub



Statement from Microsoft Blog -



Some of the service principals using Azure AD Graph in your tenant may be provided by Microsoft. Most of these have an update available that uses Microsoft Graph, instead of Azure AD Graph APIs, such as:





Some Microsoft applications, including Microsoft Office, Microsoft Visual Studio Legacy, and Microsoft Intune, do not yet have an update available without Azure AD Graph API usage. For these, we will provide future Azure AD Graph API retirement blog updates when a replacement version is available. These apps will be granted extended access for Azure AD Graph and sufficient time will be given to update the applications when an update is made available.


Dec 09, 2024 - 11:08 CST
Archive Services ? Operational
Information Archive / ZCP Operational
Erado Archiving ? Operational
Legacy Archive / ZixArchive Operational
CloudAlly Backup Operational
Encryption Services Operational
EMP/DeliverySlip Operational
Erado SecureMail (ESM) Operational
Hosted ZixGateway Operational
Hosted SMB Operational
Secure Cloud Email Encryption (AEE/ZEE) Operational
ZixPortal Operational
Customer Portal Interface Operational
Customer Portal Operational
Partner Portal Operational
Billing Area Operational
ZixCentral ? Operational
Secure Hosted Exchange Operational
Exchange 2013/2016+ (EXG7) Operational
Office 365 Operational
DNS Hosting ? Operational
Support Infrastructure Operational
Phone System Operational
Live Chat Operational
Support Ticketing System Operational
Threat Protection Operational
Email Threat Protection ? Operational
Zix Protect/Spam Stops Here ? Operational
Email Continuity Operational
Secure Cloud Email Continuity Operational
Legacy Continuty Operational
Datacenter Operational
90 days ago
100.0 % uptime
Today
Operational
Degraded Performance
Partial Outage
Major Outage
Maintenance
Major outage
Partial outage
No downtime recorded on this day.
No data exists for this day.
had a major outage.
had a partial outage.
Scheduled Maintenance
ZixPortal Upates Dec 15, 2024 02:00-04:00 CST
Our engineers will be updating ZixPortal. No expected interruptions of service expected.
Posted on Dec 04, 2024 - 15:54 CST
Past Incidents
Dec 10, 2024
Resolved - Update from Microsoft:

We’ve confirmed after a period of monitoring service telemetry that the issue is now resolved.

Dec 10, 09:11 CST
Update - Update from Microsoft:

We've disabled proactive caching to provide some relief. In parallel, we've tested and deployed a fix, which we believe will remediate impact. We anticipate that the fix will take approximately two hours to complete.

Dec 10, 07:59 CST
Investigating - Microsoft Service Incident:

Users may see the following message when viewing or accessing their Microsoft 365 apps using a web browser:

"We're experiencing a service outage. All of your open files have been saved. It may be some time before the outage is resolved."

As a workaround, users may be able to access their Microsoft 365 desktop applications and documents via the desktop applications.

Latest Updated from Microsoft:

While we continue to investigate the root cause, we’ve identified a token generation issue which may be contributing towards the impact. We’re testing a fix to revert to an alternate token generation flow, which we believe will resolve the issue.

Dec 10, 07:07 CST
Dec 9, 2024
Dec 8, 2024
Completed - The scheduled maintenance has been completed.
Dec 8, 04:00 CST
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
Dec 8, 02:00 CST
Scheduled - Hello,

On 12/08/2024 at 2:00 AM Central time, OpenText network engineers will be starting a circuit resiliency test between Dallas and Atlanta data centers to test a downed circuit scenario. During this time, services will continue to operate, however, there may be a blip while current sessions are moved from one trunk to another.

This is expected to be completed by 12/08/2024 4:00 AM Central time. Should there be any changes to this schedule we will send an update.

Thank you

Nov 20, 18:11 CST
Dec 7, 2024

No incidents reported.

Dec 6, 2024
Resolved - This incident has been resolved.
Dec 6, 14:57 CST
Monitoring - We are thrilled to announce the launch of our updated Secure Cloud Platform on 10/22, designed to enhance your experience with a modernized user interface. This update streamlines your operations and supports growth with features like enhanced analytics, tailored product recommendations, and an integrated dashboard for seamless management of tasks, customer accounts, and orders.

With the simplified order history page, you can manage all subscriptions in one place, and the customer event calendar will keep you on top of expiring subscriptions and trials. These enhancements, along with API integrations and comprehensive subscription management, will save you time, reduce operational costs, and allow you to focus on growing your business.

For more information, please see our video tutorials at: https://securecloud.hubs.vidyard.com/

Oct 17, 06:51 CDT
Resolved - This incident has been resolved.
Dec 6, 14:57 CST
Monitoring - We are excited to announce a new and improved way to order products and bundles for your customers! This release includes ordering features that will save you time, an order history page and to see all your customers subscriptions in one spot, and a customer event calendar to see when subscriptions or trials are expiring. Learn more about these exciting enhancements by watching this product video.

https://securecloud.hubs.vidyard.com/watch/rWTSjEXMy3V2bLs9Se252S

Aug 15, 08:59 CDT
Resolved - This incident has been resolved.
Dec 6, 14:51 CST
Monitoring - Microsoft Service Incident:

Updates from Microsoft -

User Impact - Users' outbound Exchange Online email messages may have been incorrectly flagged as spam.

More info - Impact was specific to senders protected by the default outbound spam policies.

Admins may have received copies of the flagged email messages.

Scope of Impact - Any user attempting to send outbound Exchange Online email messages was impacted.

Root Cause - A recent change to our model for filtering outbound email messages as spam caused impact.

Next Steps - A recent change to our model for filtering outbound email messages as spam caused impact.

Dec 4, 2024, 5:22 PM CST
We've disabled our recent changes to the model for filtering outbound email messages as spam and confirmed through our mail filtering telemetry that the impact is remediated.

Dec 5, 12:28 CST
Dec 5, 2024
Dec 4, 2024
Resolved - Update from Microsoft:

Dec 2, 2024, 4:33 PM CST
While we continue to investigate the request calls to narrow down the root cause for the increase in traffic to the user profile service cache, our service health monitors indicate that our previously implemented performance optimizations and protection methods have restored the service and remediated the impact.

This is the final update for the event.

Dec 4, 12:18 CST
Investigating - Update from Microsoft:

Some users may be encountering multiple issues in the Microsoft Teams service

User impact -
Users may be encountering multiple issues in the Microsoft Teams service.

More info -
Impacted scenarios include, but aren't limited to:

- View any or up to date profile data
- Update, create or view calendar meetings
- Send or read messages
- Add new users to teams
- Known users appearing as unknown

Scope of impact -
Impact is specific to some users who are served through the affected infrastructure.

Root cause -
A cache overload is resulting in impact.

We're continuing to apply performance optimizations on the affected infrastructure to remediate impact, while we continue to investigate what is causing the cache overload. We're also implementing protection measures to protect downstream dependencies, in order to help alleviate impact and prevent this problem from happening again.

Dec 2, 11:07 CST
Resolved - This incident has been resolved.
Dec 4, 08:28 CST
Update - We are continuing to monitor for any further issues.
Dec 3, 17:02 CST
Monitoring - Microsoft Update:

The fix has been deployed, and we've confirmed the databases have returned to a healthy state and are functioning within acceptable thresholds. After monitoring to ensure service health was stable, we've verified impact has been remediated.

- We're continuing to monitor the affected databases for any signs of unhealthy service.
- We're developing wider, preventative measures to ensure this issue doesn't return.

Dec 3, 17:02 CST
Update - Microsoft Update:

We're preparing to deploy our fix and anticipate that this process should be underway by our next scheduled communications update. We'll provide a precise remediation timeline once available.

Dec 3, 08:16 CST
Update - Microsoft Update:

The aforementioned fix which was developed to manually update the metadata is undergoing the final stages of testing, and we expect to have a deployment timeline by our next update.

Dec 2, 15:51 CST
Update - Microsoft Update:

We've determined the error impacting certain portions of infrastructure is resulting from a table responsible for updating indexes on calls to the database being deleted and metadata not updating to reflect this table being We've determined the error impacting certain portions of infrastructure is resulting from a table responsible for updating indexes on calls to the database being deleted and metadata not updating to reflect this table being deleted. We suspect this table may have been deleted twice in two separate but concurrent transactions, leaving the metadata in a dysfunctional state. We’ve developed a fix to manually update the metadata so the replication mechanism can correct itself and are testing it to determine its effectiveness.

Dec 2, 12:55 CST
Identified - Microsoft Update:

We've identified an error which is occurring in the affected portions of infrastructure, and are developing a fix to address the issue while we investigate the cause of this error.

Dec 2, 10:38 CST
Investigating - Microsoft ID: EX946693

User impact
Users may be unable to send or receive email messages in Exchange Online using Outlook on the web or desktop client.

Scope of impact
Impact is specific to some users who are served through the affected infrastructure.

Root cause
A table responsible for updating indexes on calls to the database was deleted, as intended, by an internal process. However, the corresponding metadata to reflect that deletion did not update correctly due to an unexpected service error that occurred on a subset of infrastructure, which resulted in the impact.

Dec 2, 09:45 CST
Dec 3, 2024
Dec 2, 2024
Dec 1, 2024

No incidents reported.

Nov 30, 2024

No incidents reported.

Nov 29, 2024

No incidents reported.

Nov 28, 2024

No incidents reported.

Nov 27, 2024

No incidents reported.

Nov 26, 2024
Resolved - This incident has been resolved.
Nov 26, 17:04 CST
Update - Microsoft Update:

After a period of extended monitoring, we’ve confirmed through customer reports and service health telemetry that the issue is resolved.

Nov 26, 14:10 CST
Update - Microsoft Update:

We’re continuing to conduct additional optimizations to further improve service availability, which remains in a healthy state. We're monitoring telemetry closely while this work progresses to ensure service availability is fully restored.

Nov 26, 13:24 CST
Monitoring - Microsoft Update:

We’re continuing our period of monitoring service telemetry, which shows the service availability has remained healthy.

Nov 26, 08:12 CST
Update - Microsoft Update:

We’ve successfully restored functionality for all previously impacted services and users with the exception of Outlook on the web, which is showing prolonged impact for a small number of users. We’ll continue carefully monitoring the service health and focus on troubleshooting this persisting impact to fully recover for the remaining affected users. We'll provide a new timeline within the next update.

Next update by: Tuesday, November 26, 2024, at 5:30 AM UTC

Nov 25, 21:51 CST
Update - Microsoft Update: We identified a change in the environment that resulted in an influx in request retries routed through affected servers. Our optimizations, which enhanced the infrastructure's processing capabilities, continue to provide incremental relief. We're monitoring the service and continuing our work to perform any follow-up actions or opening additional workstreams needed to fully resolve the problem.

We understand the significant impact of this event to your organization, we're treating this issue with the highest priority, and we're working to provide relief as soon as possible.

Nov 25, 16:22 CST
Update - Microsoft Update: Services are showing significant signs of restoration following our infrastructure optimizations to redistribute traffic and restart affected servers, upwards of 90 percent recovery as we continue to increase processing capability. We're continuing to perform appropriate optimizations on the subset of remaining affected machines to resolve the issue for all users.
Nov 25, 15:16 CST
Update - Microsoft Update: Our mitigative actions haven't provided relief as expected, and a portion of infrastructure remains in an unhealthy state. We determined that some of the targeted server restarts did not succeed due to processing issues, which are under investigation. We’re currently focused on spreading traffic to healthy infrastructure, and we're seeing some recovery.
Nov 25, 13:29 CST
Update - Microsoft Update: We're continuing to reroute traffic to alternate infrastructure and have reinitiated targeted server restarts to ensure the fix takes effect as expected. We're monitoring to confirm the restarts proceed successfully. We don't yet have an estimated time to resolution; however, we'll provide one as soon as it becomes available.
Nov 25, 12:00 CST
Update - The patch has deployed 90 percent, and according to telemetry, service availability is recovering. An estimated time to completion of the fix is not yet available. Targeted server restarts are underway to address routing service issues, with priority on customers who are currently in business hours or beginning their business day.

This quick update is designed to give the latest information on this issue.

Nov 25, 10:18 CST
Investigating - The affected scenarios are as follows:

Users may be unable to access Exchange Online using the following impacted connection methods:
- Outlook on the web
- Outlook desktop client
- Representational State Transfer (REST)
- Exchange ActiveSync (EAS)

Users may be unable to use the following features within Microsoft Teams:
- Users are unable to create or update Virtual Events, including webinars and Town Halls.
- Users may be unable to access or modify their calendar in Microsoft Teams. This would include loading calendar, viewing meetings, creating/updating meetings and joining meetings.
- Users are unable to create chat, add users and create or edited meetings.
- Users are unable to create or modify new teams and channels.
- Users may be unable to update presence.
- Users may be unable to use the search function.
- Users may not see updated list of files and links failing to load within the Chat shared tab.

Users may experience the following issues with Microsoft Purview:
- Users may be unable to access the Purview Portal, or Purview Solutions.
- Users may experience delays in policy stamping and with Adaptive Scope Evaluations.

Users may be unable to export content or set and view labels within Microsoft Fabric.

Some Microsoft Fabric users with Purview Information Protection Policies with sensitivity labels enabled, may be unable to use interactive operations on Power BI Desktop format files and reports, including export operations on Fabric artifacts with Sensitivity labels applied.

Users may be unable to use the search feature within SharePoint Online.

Users may be unable to perform the following actions within Microsoft Defender for Office365.
- Users may be unable to create simulations, simulation payloads or end user notifications.
- Users may experience issues with delivery for end user notifications and simulation messages
- Some users may experience failures in manual or AIR approved Remediation Actions submitted through ThreatExplorer, Advanced Hunting or the Action Center.
- Users may experiences issues with viewing simulation reports, and content.

Users may also be unable to print via Universal Print.

Users may experience errors running flows that utilize cloud connectors in Power Automate for Desktop.

Users may be unable to access their bookings within Microsoft Bookings.

Users may be unable to use some Microsoft Copilot features including:
- Users are unable to use the personal Copilot panel in meetings and post meetings.
- Users are unable to see historic Copilot conversation history in meetings and post meetings.

Nov 25, 10:16 CST