Our mail file servers v-mail-file-3 & 4 are currently offline running a file system consistency check due to unexpected issues which were automatically detected. We anticipate the consistency checks completing shortly and the file systems will be brought back to full service as soon as possible.
Update 13:00 Our mail file server v-mail-file-3 is now fully operational, however, v-mail-file-4 is currently still running consistency checks. We will update customers again as soon as any further information is received from our server technicans and apolopgise for the ongoing inconvenience.
Update 13:45: We are currently experiencing a further problem with v-mail-file-8 which shares an underlying SAN with the other affected virtual file servers. This server is currently running consistency checks. This issue has also temporarily affected our ‘mail state’, which are some temporary files used for speeding up mailboxes, which may have caused an interruption to service for a few minutes for customers on servers not affected by this issue. The mail state issue has now been resolved and our server team are working on correcting the issues with v-mail-file-8 as quickly as possible. v-mail-file-4 has been brought back into service.
Update 14:45: This issue has now been resolved. If there are any outstanding issues, please contact our customer services team who will be happy to help.
We are currently experiencing an issue with one our file storage networks, our server team are investigating this problem and are working to restore the service as soon as possible. This is likely to cause an interruption to customers on VoIP Centrex servers lon-web-6,7 and 14 and some customers using our clustered proxy natproxy.gradwell.net. We apologise for any inconvenience.
This issue has now been resolved and all services appear to be up and functioning normally with minimal interruption to service.
We have seen this morning that customers may not have been able to make calls through this pbx server. Our server admins have investigated the issue and have restarted asterisk. We can now see calls being passed through the server correctly. If you are still seeing this issue please contact our support team.
We apologise for any inconvenience caused.
November 13th, 2008 by Thomas Causer VoIP
We experiencied a brief interruption in service for customers with mailboxes on v-mail-file-7 at around 16:00. This server has been restarted and has completed file -system checks.
We have received reports that customers connected to the THAME exchange are experiencing a loss of ADSL connectivity.
BT Engineering have been dispatched to the exchange and are now investigating the cause of the problem.
Dialing codes affected ; 01844 (THAME)
Further updates will follow once information has been received.
15:05 - A problem has been reported with intermittent problems with IAX trunks. Our team are currently investigating this and will provide more information as soon as it is available.
Alternative outbound servers are available at the following addresses:
lon-asterisk-4.gradwell.net
lon-asterisk-6.gradwell.net
lon-asterisk-7.gradwell.net
16:45 - This issue has now been resolved. This issue has been isolated to a problem with our IAX outbound load balancer and will not have affected inbound calls. The IAX load balancer has been restarted and we have gathered diagnostic information which will be returned to our development team. We apologise for any inconvenience caused during this outage.
11:10 - During the last few minutes we have seen a temporary issue with our mail load balancer which will have prevented customer outbound and mail collection facilities. Inbound customer mail was unaffected. This issue has now been resolved.
We have been informed that users connected to the HARTLEPOOL exchange will be experiencing issues with their DSL service.
This is due to a major fibre break, BT engineers are currently onsite and are working to restore service.
UPDATE: BT eningeering have now received replacement parts and works will continue at 12:30
Further updates will be posted once information becomes available.
Some customers may have experienced duplicate delivery of very old mail if mailboxes are stored on our virtual mail server v-mail-file-7 today and some email may be missing in some mailboxes between July 3rd and November 5th on this server only. The majority of these mailboxes have been repaired, however, if you have experienced this problem and it has not yet been addressed, please let us know by submitting an incident to support, ensuring the mailbox is stated and we will investigate for you.
This issue has now been resolved and our admin team are monitoring and correcting some a small number of remaining customers.
