NinerNet Communications™
System Status

Server and System Status

Server NC018 replacement: Update 3

21 March 2013 18:59:03 +0000

Contrary to previous information, the replacement server will not be ready until mid to late next week.

The good mews is that we have taken further steps to mitigate the performance issues, and we have seen no evidence of performance problems in the last 48 hours.

We’ll post another update here when we have firmer time lines to tell you about.

Server NC018 replacement: Update 2

20 March 2013 09:50:48 +0000

Server load today has been within tolerance, and email has not been backed up for more than about a couple of minutes at any point.

We’ll post an update later today on the acquisition of new hardware.

Server NC018 replacement: Update 1

19 March 2013 23:32:01 +0000

A contributory cause of the performance issues on server NC018 is one of the anti-spam systems on that server. Unfortunately, we have had to deactivate part of the anti-spam system, and we’re already seeing the results in the form of spam that would normally be blocked reaching mail boxes. We apologise for this temporary issue. On the plus side, the anti-spam systems on the new server will be an improvement over the one on the old server.

Update on server NC018

19 March 2013 12:58:28 +0000

As we informed you on 6 March, we had intended to increase the memory (RAM) on server NC018 on 9 March. (Server NC018 hosts the majority of websites, and all email except specialised email services.) This maintenance would have solved performance issues that were happening with more regularity. However, the maintenance failed due to hardware compatibility issues.

When the maintenance failed, we immediately began preparing to acquire new hardware to replace this server. During the week of 11-15 March we obtained a number of proposals and advice on a plan we had begun developing some months ago.

However, on 18 March the mail server was overwhelmed. The most obvious result of this was that email delivery was delayed during peak periods by as many as several hours. While it was not clear exactly why the mail server was overwhelmed, there was an obvious way to deal with the problem. We implemented this, and the mail queue was cleared in a short amount of time. We believed that this approach should have worked to prevent the mail server from being overwhelmed if the mail server again experienced heavy volume, but it did not when the same thing happened on 19 March.

We are therefore urgently preparing to set up new hardware. I don’t have an exact time line for you at the moment, but it is our goal to have the hardware in place and online by the weekend, and we will endeavour to transfer everything as quickly as we can over the weekend.

We will update you on major events by email, but please check our status page once or twice a day at or for progress reports.

We apologise for the inconvenience that this transition is causing, and ask for your patience as we go through this upgrade.

Web server on NC018 back online

19 March 2013 12:25:06 +0000

We have cleared the mail queue on server NC018, and the web server has been brought back online.

Web server on NC018 currently shut down

19 March 2013 11:54:00 +0000

We have shut down the web server on server NC018 temporarily in order to facilitate the processing of email on the same physical machine. We will restore web hosting as soon as the emails have been processed.

We apologise for the inconvenience.

Mail processing on server NC018 back to normal

18 March 2013 20:58:04 +0000

The mail queue on server NC018 has been cleared, and anti-spam services have been reinstated. We apologise for the inconvenience, and are working on this server to bring these problems to an end.

Mail queue on server NC018 diminishing

18 March 2013 19:11:36 +0000

We have temporarily disabled some of the ant-spam services on the server to allow the mail queue to be processed as quickly as possible.

Mail queue on server NC018

18 March 2013 16:23:27 +0000

The mail queue on server NC018 — the mail server for all clients who host their email with us — is very large at the moment, and message delivery is being delayed. We’re working on this, and will post updates here.

Maintenance on server NC018 has failed

9 March 2013 17:24:36 +0000

The RAM upgrade on server NC018 has failed. We are assessing the situation and will post further updates here.

In the meantime, the server is back online and functioning normally.

NinerNet home page

Systems at a Glance:

Server NC023, London, United Kingdom (Relay server), INTERNAL.NC023InternalUp?
Server NC028, Vancouver, Canada (Monitoring server), INTERNAL.NC028InternalUp?
Server NC031, New York, United States of America (Web server), INTERNAL.NC031InternalUp?
Server NC033, Toronto, Canada (Primary nameserver), OPERATIONAL.NC033OperationalUp?
Server NC034, Lusaka, Zambia (Phone server), INTERNAL.NC034InternalUp?
Server NC035, Sydney, Australia (Secondary nameserver), OPERATIONAL.NC035OperationalUp?
Server NC036, Amsterdam, Netherlands (Mail server), OPERATIONAL.NC036OperationalUp?
Server NC040, Toronto, Canada (Web server), INTERNAL.NC040InternalUp?
Server NC041, New York, United States of America (Web server), OPERATIONAL.NC041OperationalUp?
Server NC042, Seattle, United States of America (Status website), OPERATIONAL.NC042OperationalUp?


RSS icon. RSS

Twitter icon. Twitter



Recent Posts:




Tags: domains domains domains back-up bounce messages browser warnings connection issues control panel database dns dos attack dot-zm domains down time email email delivery error messages ftp hardware imap mail mailing lists mail relay mail server microsoft migration nameservers network networking performance php phplist pop reboot shaw shaw communications inc. smtp spam spamassassin ssl ssl certificate tls tls certificate viruses webmail web server


On NinerNet: