We have rebooted server NC020 after it was overwhelmed. We’re looking into it right now and will post details here as soon as we have them.
We have rebooted server NC020 after it was overwhelmed. We’re looking into it right now and will post details here as soon as we have them.
After spending some time on this issue yesterday and today — and receiving a complaint from only one client who later informed us that the issue no longer existed — we’re closing this matter and considering it a transient network issue, probably closer to the client’s ISP than to server NC027.
Apologies for the false alarm, but we do take your complaints seriously.
If you have any questions or concerns, please let us know by contacting NinerNet support. Thank-you.
We are aware of an issue with server NC027, where most client email is hosted. However, we have not yet determined exactly what the problem is. Some services on the server are accessible, while some are not.
We are currently working with the data centre to determine the issue, and will post further updates here.
Further to our previous posts, we will be processing credits to the accounts of all affected clients for the outage on NC020.
After some research, combing of the logs on the server, and reading of documentation, we believe we found a reason that the web server stopped working, but we believe the cause evaporated when, in our desire to get everything working again as quickly as possible, we just rebooted the machine after other options to restart the web server failed. However, even the suspected cause does not fully jibe with the observed symptoms.
The good news is that we have recently commissioned a new server (NC030) to take on the websites hosted on NC020. We are still working on setting up the control panel on that, but we hope to have it in production as soon as possible.
We apologise for this incident, and thank you for your patience and understanding.
The web server on NC020 was found to be unresponsive today, but would also not respond to commands to restart. As a result we rebooted the machine and this seems to have fixed the problem.
We’re busy analysing the logs to determine the cause, and will post an update here if we find anything.
Further information from the data centre on the reboot of server NC027 indicates that there was a hardware management issue in the data centre that necessitated a reboot of this server to maintain and improve stability of this machine. The issue was addressed with a software upgrade during the downtime, and we have been assured that this addresses the issue and will not happen again.
If you have any questions about this, please contact NinerNet support. Thanks again for your patience and understanding.
We have determined, through log analysis, that server NC027 was shut down at 14:03 and restarted at 14:24 UTC. So the server was unavailable for 21 minutes this afternoon.
Any email sent to addresses on this server would have been queued on the sending servers until this server was back online, and would have been delivered at that time or shortly thereafter. You will have received any such email by this time.
When we receive more details from the data centre we will post them here. We apologise for any inconvenience caused and will post additional details here as soon as we have them. Thanks for your patience.
We received a report from the data centre that there was a hardware issue on this server this afternoon (UTC) that necessitated a reboot at 14:24 UTC. We are awaiting more details and will report them here as soon as we know more.
We have determined that the issue with downloading email from NC027 last week only affected POP email accounts, and was likely related to the limit on POP3 connections configured on the server. We have raised that limit and don’t expect this to be an issue in the future.
This did not affect those who were connecting via IMAP. Our apologies to those who were affected.
Systems at a Glance:
| Loc. | System | Status | Ping |
|---|---|---|---|
| NC023 | Internal | Up? | |
| NC028 | Internal | Up? | |
| NC031 | Internal | Up? | |
| NC033 | Operational | Up? | |
| NC034 | Internal | Up? | |
| NC035 | Operational | Up? | |
| NC036 | Operational | Up? | |
| NC040 | Internal | Up? | |
| NC041 | Operational | Up? | |
| NC042 | Operational | Up? |
Subscriptions:
Search:
Recent Posts:
Archives:
Categories:
Links
Tags:
Resources:
On NinerNet: