Server NC027 has a large and growing mail queue, but the server is up. We are investigating and will post updates here.
Server NC027 has a large and growing mail queue, but the server is up. We are investigating and will post updates here.
The web server on server NC018 was not responding this evening (UTC). The problem was addressed and the server restarted at 19:36 UTC. Other services on the server (including email, FTP and DNS) were not affected.
We apologise for any inconvenience caused during this time.
We have identified the particular websites that are being attacked and have made adjustments to try and deflect the attack. In the meantime we have restarted the web server and will continue to monitor the attack.
Additionally, the mail queue has now cleared.
We have temporarily stopped the NC018 web server in an attempt to fight this attack.
As the title says, it appears that server NC018 is again the target of an attack. We are working to address the issue as quickly as possible.
Server NC018 suffered a denial of service attack beginning at approximately 07:40 on 23 August, UTC. This continued until the server succumbed under the load at about 12:40. This affected all functions of the server: websites, sending and receiving emails, databases, etc.
Unfortunately we did not respond to this situation in a timely manner. This was partially because automated monitoring of server functions had been turned off due to an increasing number of false alarms from the system.
Once the server was rebooted at 13:47 and the attack mitigated, the server then still had a backlog of several thousand emails to process. This took several hours, with the queue being cleared by about 18:35. While emails were “missing” for several hours, no emails should have been lost; they were just delivered several hours late.
We sincerely apologise for the disruption caused by this event. We will reinstate monitoring and the associated actions that are taken when monitoring alarms go off.
We will shortly be responding to all emails and phone messages regarding this incident. If you have any further questions about this event, please contact us.
Server NC018 appears to be suffering the effects of a denial of service attack. We are working to address the issue.
On Thursday the 18th we set up a new server in a new data centre. We’re excited about the innovations in security that this new configuration will bring to your hosting account.
We are working diligently to configure and test this server, and test the migration process to minimise the time it will take. Due to issues on the old server, our priority is to get the migration done as soon as possible. While we had hoped to do this over this weekend, we may need to do it during the week. Either way we will ensure that any disruption will be kept to a minimum.
We will post further updates here over the course of the next few days.
Contrary to previous information, the replacement server will not be ready until mid to late next week.
The good mews is that we have taken further steps to mitigate the performance issues, and we have seen no evidence of performance problems in the last 48 hours.
We’ll post another update here when we have firmer time lines to tell you about.
Server load today has been within tolerance, and email has not been backed up for more than about a couple of minutes at any point.
We’ll post an update later today on the acquisition of new hardware.
Systems at a Glance:
Loc. | System | Status | Ping |
---|---|---|---|
NC023 | Internal | Up? | |
NC028 | Internal | Up? | |
NC031 | Internal | Up? | |
NC033 | Operational | Up? | |
NC034 | Internal | Up? | |
NC035 | Operational | Up? | |
NC036 | Operational | Up? | |
NC040 | Internal | Up? | |
NC041 | Operational | Up? | |
NC042 | Operational | Up? |
Subscriptions:
Search:
Recent Posts:
Archives:
Categories:
Links
Tags:
Resources:
On NinerNet: