Gradwell Service News

Mail server consistency checks

Our mail file servers v-mail-file-3 & 4 are currently offline running a file system consistency check due to unexpected issues which were automatically detected.  We anticipate the consistency checks completing shortly and the file systems will be brought back to full service as soon as possible.

Update 13:00 Our mail file server v-mail-file-3 is now fully operational, however, v-mail-file-4 is currently still running consistency checks.  We will update customers again as soon as any further information is received from our server technicans and apolopgise for the ongoing inconvenience.

Update 13:45: We are currently experiencing a further problem with v-mail-file-8 which shares an underlying SAN with the other affected virtual file servers.  This server is currently running consistency checks.  This issue has also temporarily affected our ‘mail state’, which are some temporary files used for speeding up mailboxes, which may have caused an interruption to service for a few minutes for customers on servers not affected by this issue.  The mail state issue has now been resolved and our server team are working on correcting the issues with v-mail-file-8 as quickly as possible.  v-mail-file-4 has been brought back into service.

Update 14:45: This issue has now been resolved.  If there are any outstanding issues, please contact our customer services team who will be happy to help.

PSTN number migrations

Starts: 2008/12/16 00:00 Ends: 2009/01/28 08:00

We will be changing the routing of some of our Geographic Number blocks so that inbound calls from the BT, Mobile and other PSTN networks is no longer handled by a 3rd party supplier, but is handled by our own telephone switching platform over the next couple of months.  Our platform has been in use for over 18 months and already handles over half our traffic.  Following other migrations we have done, we do not anticipate any problems with this migration, but there may be a small period of down time in the early hours of the mornings on migration dates, whilst the number routing changes are reflected in the BT Wholesale network.  This work is to be completed in three stages:

On 16th December 2008 the following number blocks will be migrated:

  • 0163782
  • 0164645
  • 0170268
  • 0193444
  • 0192532
  • 0137635
  • 0156576
  • 0190368
  • 0191580
  • 0127026
  • 0138363
  • 0160861
  • 0155375
  • 0170630
  • 0192495
  • 0129634
  • 0132370
  • 0132980
  • 0162171
  • 0135286
  • 0143830
  • 0169170
  • 0190450
  • 0125256
  • 0138680
  • 0193535
  • 0160442
  • 0195278
  • 0127630
  • 0162820
  • 0147376
  • 0142050
  • 0148876
  • 0120924
  • 0192680
  • 0130372
  • 0156261
  • 0144280
  • 0161850
  • 0162580
  • 0198950
  • 0179272
  • 0180350
  • 0182260
  • 0142945
  • 0149787
  • 0144430
  • 0148224
  • 0123448
  • 0128326

On 6th January 2009 the following number blocks will be migrated:

  • 0156863
  • 0126760
  • 0189280
  • 0122725
  • 0145161
  • 0134359
  • 0145556
  • 0160080
  • 0143470
  • 0160334
  • 0184422
  • 0124265
  • 0124940
  • 0160981
  • 0198525
  • 0130259
  • 0135378
  • 0124447
  • 0147650
  • 0127141
  • 0142862
  • 0125077
  • 0132443
  • 0159070
  • 0186969
  • 0122883
  • 0198322
  • 0130530
  • 0134880
  • 0131510
  • 0162232
  • 0124533
  • 0145226
  • 0238000
  • 0128561
  • 0148026
  • 0183750
  • 0125353
  • 0197828
  • 0137380
  • 0162969
  • 0292009
  • 0157040
  • 0120724
  • 0179484
  • 0190570
  • 0128083
  • 0123525
  • 0124320
  • 0186592

On 20th January 2009 the following number blocks will be migrated:

  • 0179325
  • 0188422
  • 0172332
  • 0120670
  • 0125479
  • 0163353
  • 0122558
  • 0127926
  • 0118324
  • 0133265
  • 0142770
  • 0117325
  • 0113320
  • 0121285
  • 0122375
  • 0115824
  • 0116326
  • 0122661
  • 0147273
  • 0114303
  • 0183485
  • 01202237
  • 01224515
  • 01273358
  • 01274288
  • 01483608
  • 01642688

Backend SAN issue

We are currently experiencing an issue with one our file storage networks, our server team are investigating this problem and are working to restore the service as soon as possible.  This is likely to cause an interruption to customers on VoIP Centrex servers lon-web-6,7 and 14 and some customers using our clustered proxy natproxy.gradwell.net.  We apologise for any inconvenience.

This issue has now been resolved and all services appear to be up and functioning normally with minimal interruption to service.

lon-pbx-5 unable to make calls

We have seen this morning that customers may not have been able to make calls through this pbx server. Our server admins have investigated the issue and have restarted asterisk. We can now see calls being passed through the server correctly. If you are still seeing this issue please contact our support team.

We apologise for any inconvenience caused.

v-mail-file-7 filesystem checks

We experiencied a brief interruption in service for customers with mailboxes on v-mail-file-7 at around 16:00.  This server has been restarted and has completed file -system checks.

BT Exchange Outage

We have received reports that customers connected to the THAME exchange are experiencing a loss of ADSL connectivity.

BT Engineering have been dispatched to the exchange and are now investigating the cause of the problem.

Dialing codes affected ; 01844 (THAME)

Further updates will follow once information has been received.

Load-balanced IAX issue

15:05 - A problem has been reported with intermittent problems with IAX trunks. Our team are currently investigating this and will provide more information as soon as it is available.

Alternative outbound servers are available at the following addresses:
lon-asterisk-4.gradwell.net
lon-asterisk-6.gradwell.net
lon-asterisk-7.gradwell.net

16:45 - This issue has now been resolved.  This issue has been isolated to a problem with our IAX outbound load balancer and will not have affected inbound calls.  The IAX load balancer has been restarted and we have gathered diagnostic information which will be returned to our development team.  We apologise for any inconvenience caused during this outage.

Temporary mail load balancer issue

11:10 - During the last few minutes we have seen a temporary issue with our mail load balancer which will have prevented customer outbound and mail collection facilities.  Inbound customer mail was unaffected.  This issue has now been resolved.

LLU Exchange Outage

We have been informed that users connected to the HARTLEPOOL exchange will be experiencing issues with their DSL service.

This is due to a major fibre break, BT engineers are currently onsite and are working to restore service.

UPDATE: BT eningeering have now received replacement parts and works will continue at 12:30

Further updates will be posted once information becomes available.

v-mail-file-7 missing mail

Some customers may have experienced duplicate delivery of very old mail if mailboxes are stored on our virtual mail server v-mail-file-7 today and some email may be missing in some mailboxes between July 3rd and November 5th on this server only. The majority of these mailboxes have been repaired, however, if you have experienced this problem and it has not yet been addressed, please let us know by submitting an incident to support, ensuring the mailbox is stated and we will investigate for you.

This issue has now been resolved and our admin team are monitoring and correcting some a small number of remaining customers.