Reason For Outage – SIP Trunk Outbound Call Failures 27th March 2022

Issued 31/03/2022

For external use

‌

• Time: 02:00 – 13:05 GMT
• Duration: 11 hours, 5 minutes
• Description of outage: Around 11:00 on the 27th of March 2022, Gradwell support were alerted to outbound call failures on the SIP trunk platform. All outbound calls made over a Gradwell SIP trunk simply failed to route with no distinct error message returned. Gradwell Engineers were fully engaged on the incident which was given a priority one status.
• Cause & resolution: Our engineers identified the cause of the issue and restored service by rebooting the affected hardware at 13:05. No further failures were observed after the restart and we observed calls passing over the platform without error.
• Root Cause: Following a routine supplier maintenance window the trunk servers failed to reconnect causing the outage. The maintenance is a monthly occurrence and this failure is an extraordinary event.
• Prevention of recurrence: Gradwell engineers have deployed a means to force reconnection where required and to log and alert to any failures should they recur.