Delayed webhooks and scenario on US1-enterprise zone
Incident Report for Make
Resolved
This incident has been resolved.
Posted Oct 21, 2024 - 19:27 CEST
Update
All systems are back to normal operation.
We expect the queue of scenario activity logs to be fully processed within 4 hours and we will keep monitoring the situation until then.
Posted Oct 21, 2024 - 18:28 CEST
Monitoring
The cluster’s scenario execution functionality has been restored and all queued executions have been processed. The backlog of scenario execution log entries is being processed at a slower rate and activity logs for completed executions will be displayed in the scenario history with a delay.
Posted Oct 21, 2024 - 18:07 CEST
Update
We are continuing to work on a fix for this issue.
Posted Oct 21, 2024 - 17:41 CEST
Identified
We have identified the likely root cause of the problem and started implementing corrective action. We will provide another update in the next 30 minutes.
Posted Oct 21, 2024 - 17:15 CEST
Update
We are continuing to investigate this issue.
Posted Oct 21, 2024 - 16:50 CEST
Investigating
We are investigating delays with webhook processing and scenario processing on US1-enterprise zone. We will update this incident in 30 min with more details.
Posted Oct 21, 2024 - 16:49 CEST
This incident affected: Website (us1.make.celonis.com), Responsive Webhooks (us1.make.celonis.com), and Scenario Executions (us1.make.celonis.com).