Server and System Status

Server NC018 under attack again

24 August 2013 07:08:51 +0000

As the title says, it appears that server NC018 is again the target of an attack. We are working to address the issue as quickly as possible.

Tags: connection issues, dos attack, down time, email delivery, performance
Categories: Incidents, NC018
Comments Off

Server NC018 denial of service attack

23 August 2013 20:55:10 +0000

Server NC018 suffered a denial of service attack beginning at approximately 07:40 on 23 August, UTC. This continued until the server succumbed under the load at about 12:40. This affected all functions of the server: websites, sending and receiving emails, databases, etc.

Unfortunately we did not respond to this situation in a timely manner. This was partially because automated monitoring of server functions had been turned off due to an increasing number of false alarms from the system.

Once the server was rebooted at 13:47 and the attack mitigated, the server then still had a backlog of several thousand emails to process. This took several hours, with the queue being cleared by about 18:35. While emails were “missing” for several hours, no emails should have been lost; they were just delivered several hours late.

We sincerely apologise for the disruption caused by this event. We will reinstate monitoring and the associated actions that are taken when monitoring alarms go off.

We will shortly be responding to all emails and phone messages regarding this incident. If you have any further questions about this event, please contact us.

Tags: connection issues, dos attack, down time, email delivery, performance, reboot
Categories: Incidents, NC018
Comments Off

Server NC018 issues

23 August 2013 15:54:34 +0000

Server NC018 appears to be suffering the effects of a denial of service attack. We are working to address the issue.

Tags: connection issues, dos attack, performance
Categories: Incidents, NC018
Comments Off

Email delays to some domains

9 June 2013 11:06:16 +0000

Over the last few days we have noticed that email to addresses on the zamtel.zm domain have been delayed. In fact, at this point it appears that the Zamtel mail server is down completely.

We also continue to see, as we have for a year or two or more, email to addresses on the iwayafrica.com delayed considerably, usually for as many as several days. This is due to issues with their mail server.

Tags: email delivery
Categories: Email
Comments Off

Server NC018 replacement: Update 14

9 June 2013 11:00:54 +0000

Our apologies for our silence on this issue. Work continues on this migration, but not at the pace at which it was happening a few weeks ago. We will, when we are finally in a position to finalise the migration, post an update here.

Thank-you for your patience and understanding.

Tags: migration
Categories: Maintenance, NC018, NC026, Upgrades
Comments Off

Spam defences on server NC018

9 June 2013 10:58:16 +0000

Contrary to the last update on this topic, spam defences on server NC018 are indeed active most of the time. We monitor the server load and turn them off if the load gets too high. However, most of the time the spam defences are up 24 hours a day, 7 days a week.

Tags: email delivery, spam
Categories: Email, NC018
Comments Off

Spam defences on server NC018

20 May 2013 06:43:46 +0000

We have disabled the spam defences on server NC018. We re-enabled them over the weekend as server loads are down over the weekends.

Tags: email delivery, spam
Categories: Email, NC018
Comments Off

Spam defences on server NC018

18 May 2013 02:46:22 +0000

We have re-enabled the most effective spam defences on server NC018.

Tags: email delivery, spam
Categories: Email, NC018
Comments Off

Server NC018 replacement: Update 13

17 May 2013 13:35:56 +0000

The migration stalled on Sunday due to complications caused by differences in the way the new control panel hosts websites. The control panel on the new server is the same control panel we have been using on the old server, but there are significant differences between the new and the old versions.

The result is that we have to check every single website manually to ensure that they’re all working before we can proceed with the email migration. We have been working on this this week, but it’s going to take a considerable amount of time.

Thank-you again for your patience.

Tags: migration
Categories: Maintenance, NC018, NC026, Upgrades
Comments Off

Server NC018 replacement: Update 12

12 May 2013 05:48:07 +0000

We will shortly send an email to all email users advising that we will be migrating all email accounts this weekend. Unfortunately, circumstances do not permit a longer lead time advising of the down time that is necessary. We have explored all avenues for doing the email migration in stages, but it must be done all in one go.

Just so you can be assured that the email you received was legitimate, it was sent from info1305@niner.net with the subject, “[NinerNet] Your email account to be migrated this weekend”. If you have any questions, please contact the person referenced in the email.

We do not have a precise time for the start of the migration yet, but based on the initial migration test, we estimate that it will take about 16 hours. Once we have a time, we will post it here. However, it will be on Sunday, of course, given the time this is being posted

During the migration all incoming email will be held on a secondary server until all email from the old server has been migrated to the new primary server. At that point, the new primary server (NC026) will go live, and the email backlog will be delivered to it. You will not be able to log into your email account during the migration. You will, however, be able to send email during the migration using your existing email settings, as long as they are the recommended settings on our website. Webmail will be unavailable during the migration.

During the migration, email on the niner.net domain will be unavailable as well. To that end we have set up a temporary email address at which you can contact us during the migration. That email address is:

migration-may-2013@ninernet.com

Our apologies for this drawn-out migration, but we’re working hard to ensure that there will be minimal complications during and after the move.

The next update will provide a time at which the email migration will start. Thank-you for your patience.

Tags: migration
Categories: Maintenance, NC018, NC026, Upgrades
Comments Off

System	Status	Ping
NC023	Internal	Up?
NC028	Internal	Up?
NC031	Internal	Up?
NC033	Operational	Up?
NC034	Internal	Up?
NC035	Operational	Up?
NC036	Operational	Up?
NC040	Internal	Up?
NC041	Operational	Up?
NC042	Operational	Up?