The emergency maintenance on server NC027 has started. All services on this server will be unavailable for the duration of the work.
We will post progress updates here. Thank-you for your patience.
The emergency maintenance on server NC027 has started. All services on this server will be unavailable for the duration of the work.
We will post progress updates here. Thank-you for your patience.
We have scheduled emergency maintenance this week for two of our servers to address a yet-to-be-published security vulnerability in the underlying virtualisation software these servers use.
The schedule is as follows:
Please click on the dates and times to convert the time of the maintenance into your local time zone. Server NC023 is commonly referred to by those that use it as the “relay server”, and server NC027 is our primary mail server.
The maintenance window will be approximately two hours long, but often ends up being much shorter. During the maintenance window the servers and everything hosted on them will be inaccessible. Any incoming mail for domains hosted on the mail server will be queued on the sending mail servers until our mail server is back online; email will be delayed during this time, but not lost. You will be unable to send email or access the webmail during the maintenance window. No websites will be affected.
While we are aware that this maintenance is scheduled during part of the work day for some of our clients, we are unable to change that due to the demands of this effort. We sincerely apologise for that.
We will post updates here and in alerts on our website before, during and after the maintenance.
We appreciate your patience with our efforts to keep our infrastructure and yours secure. If you have any questions, please let us know. Thank-you.
Further to our previous posts, we will be processing credits to the accounts of all affected clients for the outage on NC020.
After some research, combing of the logs on the server, and reading of documentation, we believe we found a reason that the web server stopped working, but we believe the cause evaporated when, in our desire to get everything working again as quickly as possible, we just rebooted the machine after other options to restart the web server failed. However, even the suspected cause does not fully jibe with the observed symptoms.
The good news is that we have recently commissioned a new server (NC030) to take on the websites hosted on NC020. We are still working on setting up the control panel on that, but we hope to have it in production as soon as possible.
We apologise for this incident, and thank you for your patience and understanding.
The web server on NC020 was found to be unresponsive today, but would also not respond to commands to restart. As a result we rebooted the machine and this seems to have fixed the problem.
We’re busy analysing the logs to determine the cause, and will post an update here if we find anything.
Over the weekend of 22-23 August 2015 we will be migrating all email accounts still on server NC018 to server NC027. Server NC027 has been our “new” mail server for the last two years, and all new domains have been hosted on it since it was set up in 2013. Server NC018 has outlived its useful life and needs to be replaced. The new server is an improvement over the old server in every respect.
We will be posting additional details here over the next week before the migration. However, you should be aware of the following initial details:
If you have any questions, please reply to the email you will receive from us with details. Thanks for your patience as we work to improve the service that we provide to you.
There were some file consistency issues on server NC020 that started at 17:47 UTC today.
Websites on the server remained online during the problem, but no logging would have happened during that time. Incoming email to aliased domains was delayed during the issue as well.
A file system consistency check was done and the server was rebooted 30 minutes later at 18:17. This solved the issue.
Some of our servers use a technology called Xen. Like all software, it is subject to the discovery of bugs and security vulnerabilities.
Over the last week, all of our servers that use Xen have undergone “patching” to address some newly-discovered — but not yet publicised — security vulnerabilities. This patching required servers to be rebooted, and finished this morning.
We thank you for your patience this week during these upgrades.
Maintenance on server NC027 has been complete. The server was down between 01:59 and 02:31 UTC. Any email sent during those 32 minutes will be redelivered.
Thank-you for your patience and understanding.
Work on server NC027 is underway. It started at 01:59 UTC and should be complete by about 02:45 UTC.
Systems at a Glance:
Loc. | System | Status | Ping |
---|---|---|---|
NC023 | Internal | Up? | |
NC028 | Internal | Up? | |
NC031 | Internal | Up? | |
NC033 | Operational | Up? | |
NC034 | Internal | Up? | |
NC035 | Operational | Up? | |
NC036 | Operational | Up? | |
NC040 | Internal | Up? | |
NC041 | Operational | Up? | |
NC042 | Operational | Up? |
Subscriptions:
Search:
Recent Posts:
Archives:
Categories:
Links
Tags:
Resources:
On NinerNet: