Title: Some users intermittently receive a security certificate revoked message prompt in the Outlook desktop and mobile apps
User Impact: Users intermittently received a security certificate revoked message prompt in the Outlook desktop and mobile apps.
More info: Users could have clicked on the Ok button in the prompt or restarted the affected app to bypass the message without encountering any impact to app functionality.
Final status: Our analysis of the Outlook client diagnostic and memory process logs identified a subset of network infrastructure that processes Transport Layer Security (TLS) requests wasn't functioning as expect, resulting in impact. We've disabled this problematic network path, allowing traffic to flow through alternate routes and remediating impact.
Scope of impact: Some users may have received a security certificate revoked message prompt in the Outlook desktop and mobile app platforms.
Start time: Wednesday, October 26, 2022, 9:00 AM (2:00 PM UTC)
End time: Tuesday, November 8, 2022, 4:00 PM (10:00 PM UTC)
Root cause: A subset of network infrastructure that processes TLS requests wasn't functioning as expect, resulting in impact.
Next steps: - We're identifying methods of bolstering the reliability of TLS infrastructure to prevent similar impact from occurring in the future. - We're analyzing our findings from this event's investigation to improve our network infrastructure monitoring to more quickly detect and address issues of this nature.
This is the final update for the event.
Posted Nov 15, 2022 - 10:29 CST
Update
Title: Some users intermittently receive a security certificate revoked message prompt in the Outlook desktop and mobile apps
User Impact: Users intermittently receive a security certificate revoked message prompt in the Outlook desktop and mobile apps.
More info: While we're focused on remediation, users can click on the Ok button in the prompt or restart the affected app to bypass the message without encountering any impact to app functionality.
Current status: We're continuing to review Outlook diagnostic logs and Outlook memory process dump logs depicting reproductions of impact, which will allow us to isolate where errors are being encountered during the chain building process. In parallel, we're also continuing to work with some affected users to gather additional information that could provide insight into what checks are being made when the client performs calls to the authentication service.
Scope of impact: Some users may receive a security certificate revoked message prompt in the Outlook desktop and mobile app platforms.
Next update by: Wednesday, November 9, 2022, 5:30 PM (11:30 PM UTC)
Posted Nov 09, 2022 - 07:18 CST
Update
Title: Some users intermittently receive a security certificate revoked message prompt in the Outlook desktop and mobile apps
User Impact: Users intermittently receive a security certificate revoked message prompt in the Outlook desktop and mobile apps.
More info: While we're focused on remediation, users can click on the Ok button in the prompt or restart the affected app to bypass the message without encountering any impact to app functionality.
Current status: Our investigation has successfully ruled out that recently provisioned databases are contributing to impact. Our next step for determining the root cause of impact is reviewing recently collected Outlook diagnostic logs, CAPI2 logs, and Outlook memory process dump logs captured during a reproduction of impact to identify where errors are encountered during the chain building process. In parallel, we’re also reaching out to a subset of affected users to request they perform specific commands on machines where impact was observed to confirm the state of the cache on those particular machines for further insight into what checks are being made when the client performs calls to the authentication service.
Scope of impact: Some users may receive a security certificate revoked message prompt in the Outlook desktop and mobile app platforms.
Next update by: Tuesday, November 8, 2022, 5:30 PM (11:30 PM UTC)
Posted Nov 08, 2022 - 07:51 CST
Update
Title: Some users intermittently receive a security certificate revoked message prompt in the Outlook desktop and mobile apps
User Impact: Users intermittently receive a security certificate revoked message prompt in the Outlook desktop and mobile apps.
More info: While we're focused on remediation, users can click on the Ok button in the prompt or restart the affected app to bypass the message without encountering any impact to app functionality.
Current status: Due to both the complexity and intermittent nature of impact, our analysis so far has been inconclusive. We're continuing to work with affected users to gather Outlook client logs and a live reproduction of the issue with more verbose logging enabled to better assess the impact, and have determined that the aforementioned proxy server logs aren't needed at this time. In parallel, we're attempting to internally gather Outlook client logs for our investigation and we're continuing to look into the possibility of recently provisioned databases causing impact.
Scope of impact: Some users may receive a security certificate revoked message prompt in the Outlook desktop and mobile app platforms.
Next update by: Monday, November 7, 2022, 5:30 PM (11:30 PM UTC)
Posted Nov 07, 2022 - 09:44 CST
Identified
Title: Some users intermittently receive a security certificate revoked message prompt in the Outlook desktop and mobile apps
User Impact: Users intermittently receive a security certificate revoked message prompt in the Outlook desktop and mobile apps.
More info: While we're focused on remediation, users can click on the Ok button in the prompt or restart the affected app to bypass the message without encountering any impact to app functionality.
Current status: We've requested additional data from a subset of affected users, including proxy server logs to determine IP addresses in use when the revoked message is sent to further narrow down potential sources of impact. Due to the intermittent nature of the issue, this data has been difficult to capture and attempted reproductions have been unsuccessful. In parallel, we're looking into the possibility of recently provisioned databases causing impact.
Scope of impact: Some users may receive a security certificate revoked message prompt in the Outlook desktop and mobile app platforms.
Next update by: Friday, November 4, 2022, 5:30 PM (10:30 PM UTC)
Posted Nov 04, 2022 - 08:54 CDT
Investigating
Title: Some users receive a security certificate revoked message prompt in the Outlook desktop and mobile app platforms
User Impact: Users receive a security certificate revoked message prompt in the Outlook desktop and mobile app platforms.
More info: While we're focused on remediation, users can click on the Ok button in the prompt or restart the affected app to bypass the message without encountering any impact to app functionality.
Current status: Our review of IDNA and Fiddler trace logs appears to be inconclusive in isolating the source of the issue. We’re reviewing our code requests within headers as well as reviewing authentication component configurations involved in the process in search of what is causing impact.
Scope of impact: Some users may receive a security certificate revoked message prompt in the Outlook desktop and mobile app platforms.
Next update by: Thursday, November 3, 2022, 5:30 PM UTC