Server and System Status

NC027: Hardware issue

8 December 2014 19:35:33 +0000

We received a report from the data centre that there was a hardware issue on this server this afternoon (UTC) that necessitated a reboot at 14:24 UTC. We are awaiting more details and will report them here as soon as we know more.

Tags: connection issues, down time, email delivery, hardware, reboot, smtp
Categories: Email, Incidents, NC027
Comments Off

NC027: Upgrade complete

22 November 2014 20:15:20 +0000

The maintenance on server NC027 to increase the RAM, increase bandwidth capacity and move to solid-state drives has been completed. It took far less time than we expected, which is a good thing of course. The server was taken offline at 19:20 and was back online at 19:41 UTC.

Systems have been checked and everything appears to be operating normally. If you have any questions about this upgrade, please contact NinerNet support to let us know and we will be happy to answer your questions. Thanks very much for your patience and understanding.

Tags: down time, email delivery, hardware, reboot, smtp
Categories: Email, Maintenance, NC027, Upgrades
Comments Off

NC027: Upgrade underway

22 November 2014 19:39:32 +0000

The upgrades to server NC027 are currently underway. They started at 19:20 UTC, which was 20 minutes later than planned. We apologise for the delay in starting, but we still expect to be done within the 90 minute window.

Tags: down time, email delivery, hardware, reboot, smtp
Categories: Email, Maintenance, NC027, Upgrades
Comments Off

NC027: Weekend maintenance reminder

22 November 2014 04:12:32 +0000

This is a reminder that we will be performing upgrades on server NC027 during our maintenance window this weekend. This maintenance will start at 19:00 UTC on Saturday, 22 November, and will take approximately 90 minutes.

During that time NC027 will be offline. No websites are hosted on this server, but this will affect all email accounts on this server. Your email is hosted on this server if you do not have web hosting with NinerNet, or you have a WordPress website hosted with us.

Incoming email will be stored on the sending mail servers while NC027 is down; it will not bounce, and email will not be lost.

If you have any questions or concerns about this maintenance, please contact NinerNet support and we’ll be happy to help you.

Tags: down time, email delivery, hardware, reboot, smtp
Categories: Email, Maintenance, NC027, Upgrades
Comments Off

NC027: Server back online

21 November 2014 04:23:48 +0000

We have completed adding additional storage to server NC027. The server was taken down at 03:13 and was back online 28 minutes later at 03:41.

However, we have decided to postpone the other upgrades until this weekend, during our weekly maintenance window starting at 19:00 UTC on Saturday, 22 November.

Thanks again for your patience and understanding.

Tags: down time, email delivery, hardware, reboot
Categories: Maintenance, NC027, Upgrades
Comments Off

NC027: Emergency maintenance

21 November 2014 01:30:13 +0000

We will be performing emergency maintenance on server NC027 today during our daily maintenance window starting at 03:00 UTC. We are allowing a 90-minute window for this maintenance to add more RAM, bandwidth and storage, but we expect that it will take less time. We will post an update here as soon as the maintenance is complete.

If you have any questions or concerns about this, please contact NinerNet support. Thank-you.

Tags: down time, email delivery, hardware, reboot
Categories: Maintenance, NC027, Upgrades
Comments Off

NC020: Web server restarted

10 November 2014 13:24:48 +0000

The web server on NC020 was down between approximately 11:21 and 12:33 UTC today. We are investigating and will post a report here later today.

Tags: down time, web server
Categories: Incidents, NC020
Comments Off

NC026 emergency maintenance complete

2 October 2014 12:14:12 +0000

The emergency maintenance on server NC026 described earlier has been completed successfully. Thank-you for your patience.

Tags: down time, hardware, reboot, web server
Categories: Maintenance, NC026
Comments Off

NC026 emergency maintenance

1 October 2014 04:40:48 +0000

To deal with an as-yet unpublished security vulnerability, server NC026 will undergo emergency maintenance today (1 October) starting shortly after 15:00 UTC. Down time should be brief, but will be noticeable for websites hosted on this server during the maintenance. Most of the websites hosted on this server are newer ones using more recent versions of WordPress. If you are not sure whether or not your website is hosted on this server, you can contact support to find out.

No email is hosted on NC026, so this maintenance will not affect email flow at all.

Tags: down time, hardware, reboot, web server
Categories: Maintenance, NC026
Comments Off

NC020 outage report

25 August 2014 00:05:53 +0000

The problem with server NC020 actually started before the outage on Friday, about midday UTC. A client’s website was compromised, and the cracker downloaded scripts to cause the sending of spam. This took place over the course of a day or so, and eventually the spammer consumed all of the resources of the server. Although the server was still up, it was unresponsive.

Most websites are authorised to use the mail server on a hosting server, so we can’t block that ability completely. However, we will make a concerted effort to improve the monitoring of abuse (intended by the client or not) of this function.

In this case, we cleaned up all of the generated spam, removed the offending scripts, and will work with the client on the issues with their website security.

For the record, there are two levels of compromises: at the root level or the user level. At the root level is very bad, and requires a brand new server to be provisioned. This was a user-level compromise which, while bad, is easier to fix and recover from.

We again apologise for the downtime. We will be contacting and crediting affected accounts. We are always working to prevent security issues like this, but it’s an ongoing task. Thank-you for your patience and continued business. If you have any questions, please feel free to contact support.

Tags: down time, web server
Categories: Incidents, NC020
Comments Off

System	Status	Ping
NC023	Internal	Up?
NC028	Internal	Up?
NC031	Internal	Up?
NC033	Operational	Up?
NC034	Internal	Up?
NC035	Operational	Up?
NC036	Operational	Up?
NC040	Internal	Up?
NC041	Operational	Up?
NC042	Operational	Up?