No Audio on inbound calls via a Specific Carrier
Incident Report for Gradwell Communications Ltd
Postmortem

Description of outage and impact:

On Saturday, 18th May 2024, we received reports from our customers experiencing no audio on inbound calls. While the calls appeared to connect, there was no audio present. Outbound calls were unaffected. Upon investigation, we identified the issue with a specific carrier and escalated it directly to them. During their initial investigation, they believed the issue was specific to calls made to a particular route, seeming to affect only ported numbers through BT IPEX. However, as the incident progressed, we observed that it also impacted native numbers associated with the carrier. Gradwell promptly provided examples of the issue to the carrier for their review.

Root Cause & Resolution:

The root cause was identified by the carrier implementing a new route for Gradwell traffic. However, the media IP addresses were not fully migrated, resulting in no audio on inbound calls because the RTP traffic wasn't properly routed. To resolve this, the carrier migrated the media IPs to the new route, ensuring the RTP traffic was routed correctly.

Prevention of recurrence:

We are working closely with the carrier to ensure they adhere to our standards regarding our Change Management process and confirm they notify Gradwell before they roll out any changes to their platform that could impact the way we present calls to our customers. Please accept Gradwell’s apologies for the service disruption and the impact it has had on your business.

Posted May 30, 2024 - 10:00 BST

Resolved
Hello,

We are aware of an incident currently effecting a subset of numbers from a specific carrier, resulting in failed inbound audio. Following an update from our upstream supplier, they have identified that this issue is only impacting ported numbers through BT. These numbers are categorised by our supplier as Type A numbers that route through BT IPEX.

The issue has been escalated to BT, and they are treating it as a priority 1 incident. We are actively working on this and will provide further updates as soon as we have them.

Gradwell Apologies for the inconvenience, we are treating this as high priority.
Posted May 18, 2024 - 16:00 BST