Push notifications in Zulip are done by a background worker, which talks to Apple and Google servers to notify applications on users' mobile devices. On Monday, the rate of push notifications generated by the Zulip Cloud service surpassed the rate which a single worker could send those notifications, leading to a backlog of notifications. This was further worsened by other load on the system, which compounded the backlog, leading to delays of up to 10 minutes between when a push notification was triggered, and when it was sent to users' mobile devices.

We have since split the workers which deliver these notifications, allowing us to process many more in parallel.