We are seeing some problems with our speech to text service.
Some users have been seeing incorrectly converted or garbled email notifications, our engineering team are working on implementing a fix for this and hope to resume normal service as soon as possible.
Our apologies for any problems this is causing
Update: This issue was resolved by 27th May and was caused by our supplier unexpectedly changing the format of emails sent inbound to our system. We have altered our logic to support both formats in case of the unlikely event of this happening again.
Starts: 2009/03/03 22:00 Ends: 2009/03/03 23:00
On Tuesday evening, we are upgrading our IMAP servers, affecting all email customers. During this period, email will continue to be delivered to customers inboxes as normal, but customers may be unable to access their email via IMAP for short periods of time.
We apologise in advance for any inconvenience caused, and will endeavour to keep downtime to a minimum.
Starts: 2009/01/21 23:00 Ends: 2009/01/21 23:59
Tonight we need to temporarily shutdown the v-mail-file-3 server to finish the migration onto a new storage array. This will result in mailboxes on this server being completely inaccessible for the duration of the work, which we expect to take no more than 1 hour. We will try to avoid the work affecting mailboxes on other servers however is likely all mailboxes will see some impact.
Please accept our apologies for any inconvenience caused by the ongoing maintenance.
Starts: 2008/11/25 22:00 Ends 2008/11/26 01:00
We are performing maintenance tonight on our iSCSI-3 storage unit, to increase the size of the internal queues on this storage unit. During this maintenance window, some customers may be unable to access their email via POP3 or IMAP. All incoming email will be queued, and successfully delivered after the maintenance has completed.
We apologise for any inconvenience this maintenance causes.
Starts: 2008/11/16 10:00 Ends 2008/11/17 06:00
After reliability problems with one of our storage SANs on Thursday, we need to migrate our email storage onto a new storage device. This work will start at 10am on Sunday, and during this period customers may be unable to access their email using POP3 or IMAP. No incoming email will be lost, but delivery to your inbox may be delayed by several hours.
We apologise for the short notification for this change, and for any inconvenience this emergency maintenance may cause.
Thanks to the customers who pointed out that the initial date on this update was October. It has now been modified to November.
Our mail file servers v-mail-file-3 & 4 are currently offline running a file system consistency check due to unexpected issues which were automatically detected. We anticipate the consistency checks completing shortly and the file systems will be brought back to full service as soon as possible.
Update 13:00 Our mail file server v-mail-file-3 is now fully operational, however, v-mail-file-4 is currently still running consistency checks. We will update customers again as soon as any further information is received from our server technicans and apolopgise for the ongoing inconvenience.
Update 13:45: We are currently experiencing a further problem with v-mail-file-8 which shares an underlying SAN with the other affected virtual file servers. This server is currently running consistency checks. This issue has also temporarily affected our ‘mail state’, which are some temporary files used for speeding up mailboxes, which may have caused an interruption to service for a few minutes for customers on servers not affected by this issue. The mail state issue has now been resolved and our server team are working on correcting the issues with v-mail-file-8 as quickly as possible. v-mail-file-4 has been brought back into service.
Update 14:45: This issue has now been resolved. If there are any outstanding issues, please contact our customer services team who will be happy to help.
We experiencied a brief interruption in service for customers with mailboxes on v-mail-file-7 at around 16:00. This server has been restarted and has completed file -system checks.
Starts: 2008/10/29 22:00 Ends: 2008/10/29 23:59
We will be carrying out emergency maintenance on our email service on Wednesday 29th October. We’re adding additional SMTP relay servers to the cluster.
During this maintenance, customers may experience short periods where they are unable to send email from their email client. Any email that customers attempt to send should queue up in their email client’s outbox, and be automatically sent once the maintenance is complete.
Starts: 2008/10/08 23:45 Ends: 2008/10/09 00:15
We will be performing a number of important system updates at around midnight tonight. Services affected will be:
- VoIP calls/registrations for some NAT customers - however this does not affect customers on nat.gradwell.net which is the default for most customers.
- SSL sites hosted on uk-ssl.com
- Control Panel logins.
- SMTP/POP3/IMAP email services (should be very minimal impact because only 1 server in the cluster is being updated)
- Zimbra email services.
The services above may all have intermittent outages of a few minutes per service. Please accept our apologies for any inconvenience this may cause.
Starts: 2008-09-12 01:00 Ends: 2008/09/12 02:00
Due to a memory error, we have lost access to the management unit in our main cisco switch stack in Telehouse and therefore need to reboot one of the switch nodes.
Although the switches contain redundant controllers, we cannot be sure that the backup will take over, and there may be a short outage to our core network whilst it reboots.
As a precaution, we plan to shutdown most of our virtual servers to a bare minimum, and specifically we will switch off those providing the email system (as their hard disks are accessed via network based storage), and therefore no email collection or delivery will be available during this maintenance window.
We do expect our voip platform to continue working, although some calls may be dropped.
We expect this work to take under 15 minutes to complete, but please accept our apologies for this interuption to service.
Update - 03:15
The switch was successfully rebooted at 01:30 and nearly all services were fully functional by 02:00. At the time of writing (03:15) we believe all services are back to normal.
Thank you for your patience during this maintenance and we apologize for any inconvenience caused.
