Gradwell Service News » Completed Infrastructure Maintenance

End of Life - Ochre and Newred

David Palmer — Mon, 23 Mar 2009 14:27:31 +0000

This announcement is intended to serve as an End of Life (EOL) notice
against two of our legacy shell servers, which are running FreeBSD.
Removal from service will occur on 30th April 2009 and all customers are
hereby advised to migrate to ’shell.gradwell.net’ in the next 4-5 weeks.

End of Life
30th April 2009

Affected Hosts:
ochre.gradwell.net
newred.gradwell.net

Any scheduled jobs (via cron) or screen sessions running on these hosts
must be migrated over prior to the removal date, customers should note
that the new shell servers are running CentOS 5.x and not FreeBSD.

Any data which is not located in your home directory on these servers
will *not* be migrated, so please ensure you do not use other locations
on the filesystem (ie: /tmp) for essential data.

Please contact the support team if you have questions.

Back-end storage issues

David Palmer — Tue, 20 Jan 2009 15:47:01 +0000

We are aware of an issue with our back-end systems including PHP clusters, affecting live sites and test domains, and are working hard to resolve the problems as quickly as possible. All members of our technical team are currently dedicated to resolving this problem for customers as quickly as possible and drawing up contingency plans for getting services back up and running as quickly as possible. We will be updating customers with a full report as soon as possible.

Update 21 Jan 09 6:30am: All websites and domains were brought back online overnight. If you are still experiencing problems, please contact Gradwell Customer Services.

ESX host problem

David Palmer — Tue, 20 Jan 2009 07:35:46 +0000

We are currently investigating a problem with one of our virtual server hosts which may be affecting some network services including customers on v-mail-file-8, lon-pbx-6 and 14. We will customer customers with more details as soon as they are available.

09:31 This issue has been resolved, although our server team will be working on the network throughout the day to ensure services are stabilised as soon as possible.

Network issue

David Palmer — Mon, 19 Jan 2009 13:35:50 +0000

We are currently aware of an issue which is affecting our virtualised network in London. Our server team are investigating and we will update customers as soon as possible with further information.

Update 14:30 Our server team are currently restarting some of the key servers within our Telehouse cluster and we are working closely with our platform vendors to identify the cause of the network issue we are currently seeing. We hope to provide a resolution to this problem as soon as possible.

Update 15:00 We are currently recovering all affected services and expect all services to be fully online shortly.

Update 15:40 Whilst the majority of services are now back in operation, including PHP 4.4 and 5.2 services, and our VoIP NAT proxies, we are experiencing some issues with some of our email services, which our administrators are working to bring back online as soon as possible. Our virtual PBX lon-pbx-14 is not yet back in operation. We apologise for the ongoing inconvenience to customers using these services.

Update 17:35 Our administrators have brought out IMAP,POP3 and SMTP services back online, however, v-mail-file-1,5,6,7,8,9 remain offline and we are working to bring them back online as soon as possible. Customers using these file servers may encounter errors when attempting to send email, however, this will be due to the email client saving the email to the remote folder, the email itself will be sent out.

Update 18:10 Our VoIP Centrex server lon-pbx-14 has now been brought back online and is handling calls correctly. We apologise for the length of time this server was unavailable.

Update 19:30 All known issues with our servers have now been resolved. If you are experiencing any remaining problems, please contact our customer services team, who will be able to advise you and inform our server team of any problems which may remain. Apologies for the extended problems today. Our server team are working through a lot of changes which should make our network more resilient in future.

COMPLETED: VMWare ESX Platform Maintenance

Stuart Herbert — Tue, 13 Jan 2009 16:54:02 +0000

Starts: 2009/01/14 22:00 Ends: 2009/01/15 02:00

Overnight Wednesday / Thursday this week, we will be rotating servers one by one out of our VMWare ESX cluster for essential maintenance. We are adding an additional 4 CPU cores per server, to increase the overall capacity of the ESX cluster.

During this time, our services will continue to run, but there may be small amounts of disruption.

We apologise for the short notice for this change.