Message Logs Delays
Incident Report for Courier
Resolved
The data stream is unblocked, and the message logs queue is resolved and flowing normally.
Posted Oct 21, 2024 - 11:51 PDT
Monitoring
The fix has landed in production and the team is monitoring the message log queue. Message event logs should be flowing normally.
Posted Oct 21, 2024 - 11:30 PDT
Update
The team encountered an issue with testing the fix and reverted the update. We are publishing a new update that should resolve the backlogged message logs.
Posted Oct 21, 2024 - 10:19 PDT
Identified
The team is testing out a fix to reduce the bottlenecked log lines before releasing to production.
Posted Oct 21, 2024 - 09:07 PDT
Update
We are continuing to investigate this issue.
Posted Oct 21, 2024 - 08:56 PDT
Investigating
The Courier team is investigating an issue with the event logger for message event logs hitting a bottleneck. The team is actively investigating. Messages are still sending.
Posted Oct 21, 2024 - 08:56 PDT
This incident affected: Courier (Observability).