Further to our previous posts, we will be processing credits to the accounts of all affected clients for the outage on NC020.
Further to our previous posts, we will be processing credits to the accounts of all affected clients for the outage on NC020.
After some research, combing of the logs on the server, and reading of documentation, we believe we found a reason that the web server stopped working, but we believe the cause evaporated when, in our desire to get everything working again as quickly as possible, we just rebooted the machine after other options to restart the web server failed. However, even the suspected cause does not fully jibe with the observed symptoms.
The good news is that we have recently commissioned a new server (NC030) to take on the websites hosted on NC020. We are still working on setting up the control panel on that, but we hope to have it in production as soon as possible.
We apologise for this incident, and thank you for your patience and understanding.
The web server on NC020 was found to be unresponsive today, but would also not respond to commands to restart. As a result we rebooted the machine and this seems to have fixed the problem.
We’re busy analysing the logs to determine the cause, and will post an update here if we find anything.
There were some file consistency issues on server NC020 that started at 17:47 UTC today.
Websites on the server remained online during the problem, but no logging would have happened during that time. Incoming email to aliased domains was delayed during the issue as well.
A file system consistency check was done and the server was rebooted 30 minutes later at 18:17. This solved the issue.
Some of our servers use a technology called Xen. Like all software, it is subject to the discovery of bugs and security vulnerabilities.
Over the last week, all of our servers that use Xen have undergone “patching” to address some newly-discovered — but not yet publicised — security vulnerabilities. This patching required servers to be rebooted, and finished this morning.
We thank you for your patience this week during these upgrades.
Maintenance on server NC027 has been complete. The server was down between 01:59 and 02:31 UTC. Any email sent during those 32 minutes will be redelivered.
Thank-you for your patience and understanding.
Work on server NC027 is underway. It started at 01:59 UTC and should be complete by about 02:45 UTC.
We are about to perform urgent maintenance on server NC027 to address an upcoming capacity issue. As soon as the maintenance starts and we have an estimate for how long it will take, we will post an update here.
During the work the server will be unavailable. Incoming email will be delayed, but not lost.
If you have any concerns, please contact support. Thank-you for your patience.
Server NC027 was rebooted at 05:43 UTC after being found unresponsive. We are looking into the cause of the issue.
Further information from the data centre on the reboot of server NC027 indicates that there was a hardware management issue in the data centre that necessitated a reboot of this server to maintain and improve stability of this machine. The issue was addressed with a software upgrade during the downtime, and we have been assured that this addresses the issue and will not happen again.
If you have any questions about this, please contact NinerNet support. Thanks again for your patience and understanding.
Systems at a Glance:
Loc. | System | Status | Ping |
---|---|---|---|
![]() | NC023 | Internal | Up? |
![]() | NC028 | Internal | Up? |
![]() | NC031 | Internal | Up? |
![]() | NC033 | Operational | Up? |
![]() | NC034 | Internal | Up? |
![]() | NC035 | Operational | Up? |
![]() | NC036 | Operational | Up? |
![]() | NC040 | Internal | Up? |
![]() | NC041 | Operational | Up? |
![]() | NC042 | Operational | Up? |
Subscriptions:
Search:
Recent Posts:
Archives:
Categories:
Links
Tags:
Resources:
On NinerNet: