Server and System Status

Server NC027 performance degraded

23 January 2014 09:07:10 +0000

Server NC027 has a large and growing mail queue, but the server is up. We are investigating and will post updates here.

Tags: email delivery, performance
Categories: Email, Incidents, NC027
Comments Off

Server NC018 web server restarted

24 August 2013 08:21:57 +0000

We have identified the particular websites that are being attacked and have made adjustments to try and deflect the attack. In the meantime we have restarted the web server and will continue to monitor the attack.

Additionally, the mail queue has now cleared.

Tags: connection issues, dos attack, email delivery, performance
Categories: Incidents, NC018
Comments Off

NC018 web server stopped

24 August 2013 07:24:12 +0000

We have temporarily stopped the NC018 web server in an attempt to fight this attack.

Tags: connection issues, dos attack, down time, performance
Categories: Incidents, NC018
Comments Off

Server NC018 under attack again

24 August 2013 07:08:51 +0000

As the title says, it appears that server NC018 is again the target of an attack. We are working to address the issue as quickly as possible.

Tags: connection issues, dos attack, down time, email delivery, performance
Categories: Incidents, NC018
Comments Off

Server NC018 denial of service attack

23 August 2013 20:55:10 +0000

Server NC018 suffered a denial of service attack beginning at approximately 07:40 on 23 August, UTC. This continued until the server succumbed under the load at about 12:40. This affected all functions of the server: websites, sending and receiving emails, databases, etc.

Unfortunately we did not respond to this situation in a timely manner. This was partially because automated monitoring of server functions had been turned off due to an increasing number of false alarms from the system.

Once the server was rebooted at 13:47 and the attack mitigated, the server then still had a backlog of several thousand emails to process. This took several hours, with the queue being cleared by about 18:35. While emails were “missing” for several hours, no emails should have been lost; they were just delivered several hours late.

We sincerely apologise for the disruption caused by this event. We will reinstate monitoring and the associated actions that are taken when monitoring alarms go off.

We will shortly be responding to all emails and phone messages regarding this incident. If you have any further questions about this event, please contact us.

Tags: connection issues, dos attack, down time, email delivery, performance, reboot
Categories: Incidents, NC018
Comments Off

Server NC018 issues

23 August 2013 15:54:34 +0000

Server NC018 appears to be suffering the effects of a denial of service attack. We are working to address the issue.

Tags: connection issues, dos attack, performance
Categories: Incidents, NC018
Comments Off

Server NC018 replacement: Update 10

8 May 2013 01:41:27 +0000

It took 20 hours, but we have transferred all of the data from the old server (NC018) to the new server (NC026). However, now we have some manual configuring to do to address the errors in the automated process.

As always, we’ll post further updates here.

Tags: performance
Categories: Maintenance, NC018, Upgrades
Comments Off

Server NC018 replacement: Update 9

7 May 2013 07:12:49 +0000

Apparently the first 50% was the quick part. The next 50% is going to take much longer. However, the good news is that most of the processing has now moved to the new server, and none of the operations on the current (old) server have been affected.

Tags: performance
Categories: Maintenance, NC018, Upgrades
Comments Off

Server NC018 replacement: Update 8

7 May 2013 06:07:22 +0000

We are just over 50% of the way through our test. At the current rate, we should be finished by about 06:55 UTC.

Tags: performance
Categories: Maintenance, NC018, Upgrades
Comments Off

Server NC018 replacement: Update 7

7 May 2013 05:04:51 +0000

We are currently running a migration test on server NC018. You may experience intermittent access to services.

We will provide updates here as the test progresses, and when it is concluded.

Tags: hardware, performance
Categories: Email, Maintenance, NC018, Upgrades
Comments Off

System	Status	Ping
NC023	Internal	Up?
NC028	Internal	Up?
NC031	Internal	Up?
NC033	Operational	Up?
NC034	Internal	Up?
NC035	Operational	Up?
NC036	Operational	Up?
NC040	Internal	Up?
NC041	Operational	Up?
NC042	Operational	Up?