On the morning of the 11th of January 2018, a subset of Gradwell customers experienced total loss of inbound call routing. We appreciate that this disruption caused significant frustration to customers, and apologise sincerely for this.
This loss was due to a server failure, causing DNS queries to gradwell.com and gradwell.net not to complete.
We pride ourselves on having redundancies built in to our infrastructure to reduce the risk of service disruption. In the case of this incident, the routing of a subset of phone numbers relied on the resolution of our DNS. On Thursday the 11th of January, the servers performing this action suffered a significant failure that caused all queries to subdomains or hosts on gradwell.com and gradwell.net domains to fail to resolve.
Our agents reacted swiftly to identify the cause of the failure and return service, culminating in restarting the affected servers. While this work was being undertaken the servers were unable to resolve DNS queries, resulting in 28 minutes of service disruption.
Moving forward we have implemented a number of changes to ensure service reliability. DNS for gradwell.com and gradwell.net domains has been moved off Gradwell infrastructure to add another layer of protection to the services and reduce the risk of recurrence of this issue.
Gradwell are more committed than ever to continually improving the quality or our services, and would like to apologise for the inconvenience caused by this incident.