We are now considering this incident resolved. A recent code change generated unexpectedly large database queries that saturated CPU resources, leading to query timeouts and subsequent connection failures. We initiated a rapid rollback once the cause was identified. Since mitigating we have reverted the change and are working to prevent similar issues in the future.
Posted Sep 08, 2025 - 12:15 PDT
Update
We have completed rolling out the mitigation for this incident and are now in the monitoring phase.
Posted Sep 08, 2025 - 11:29 PDT
Identified
We have identified the source of the issue and are in the process of rapidly reverting a change to resolve.
Posted Sep 08, 2025 - 11:02 PDT
Investigating
Our monitoring picked up a spike in errors and we have begun getting reports from users. The team is investigating on highest priority