NinerNet Communications™
System Status

Server and System Status

NC023: Relay server stopped to clean up spam

26 January 2014 13:28:29 +0000

The relay server has been stopped while we clean up a spam outbreak, and identify and block the source. We will post updates here. We expect to have the server back online within the next 30-40 minutes.

Tags: , ,
Categories: Email, Incidents, NC023
Comments Off on NC023: Relay server stopped to clean up spam

NC027: Mail queue almost clear

23 January 2014 14:48:04 +0000

The mail queue will be cleared within the next 20-30 minutes.

Once the queue is clear we’ll be looking for a cause for this incident. There was a great spike in activity on this server around 04:00 UTC, and it’s quite possible that the initial back-up started around that time and didn’t quite manage to catch up.

Tags: ,
Categories: Email, Incidents, NC027
Comments Off on NC027: Mail queue almost clear

NC027: New mail delivered immediately

23 January 2014 13:25:21 +0000

Since reconfiguring the mail server we’ve noticed that new email is being delivered right away. It is now only the backlog that is still slowly being processed.

Tags: ,
Categories: Email, Incidents, NC027
Comments Off on NC027: New mail delivered immediately

NC027: Performance improvement

23 January 2014 11:55:41 +0000

We have made some adjustments to the configuration of the mail server on NC027 and as a result the mail queue seems to be clearing, albeit very slowly. We’re continuing to work on this to speed up delivery.

Tags: ,
Categories: Email, Incidents, NC027
Comments Off on NC027: Performance improvement

Server NC027 performance degraded

23 January 2014 09:07:10 +0000

Server NC027 has a large and growing mail queue, but the server is up. We are investigating and will post updates here.

Tags: ,
Categories: Email, Incidents, NC027
Comments Off on Server NC027 performance degraded

Web server on NC018

13 December 2013 20:21:00 +0000

The web server on server NC018 was not responding this evening (UTC). The problem was addressed and the server restarted at 19:36 UTC. Other services on the server (including email, FTP and DNS) were not affected.

We apologise for any inconvenience caused during this time.

Tags: , ,
Categories: Incidents, NC018
Comments Off on Web server on NC018

Server NC018 web server restarted

24 August 2013 08:21:57 +0000

We have identified the particular websites that are being attacked and have made adjustments to try and deflect the attack. In the meantime we have restarted the web server and will continue to monitor the attack.

Additionally, the mail queue has now cleared.

Tags: , , ,
Categories: Incidents, NC018
Comments Off on Server NC018 web server restarted

NC018 web server stopped

24 August 2013 07:24:12 +0000

We have temporarily stopped the NC018 web server in an attempt to fight this attack.

Tags: , , ,
Categories: Incidents, NC018
Comments Off on NC018 web server stopped

Server NC018 under attack again

24 August 2013 07:08:51 +0000

As the title says, it appears that server NC018 is again the target of an attack. We are working to address the issue as quickly as possible.

Tags: , , , ,
Categories: Incidents, NC018
Comments Off on Server NC018 under attack again

Server NC018 denial of service attack

23 August 2013 20:55:10 +0000

Server NC018 suffered a denial of service attack beginning at approximately 07:40 on 23 August, UTC. This continued until the server succumbed under the load at about 12:40. This affected all functions of the server: websites, sending and receiving emails, databases, etc.

Unfortunately we did not respond to this situation in a timely manner. This was partially because automated monitoring of server functions had been turned off due to an increasing number of false alarms from the system.

Once the server was rebooted at 13:47 and the attack mitigated, the server then still had a backlog of several thousand emails to process. This took several hours, with the queue being cleared by about 18:35. While emails were “missing” for several hours, no emails should have been lost; they were just delivered several hours late.

We sincerely apologise for the disruption caused by this event. We will reinstate monitoring and the associated actions that are taken when monitoring alarms go off.

We will shortly be responding to all emails and phone messages regarding this incident. If you have any further questions about this event, please contact us.

Tags: , , , , ,
Categories: Incidents, NC018
Comments Off on Server NC018 denial of service attack
« Older Entries
Newer Entries »

NinerNet home page

Systems at a Glance:


Loc.SystemStatusPing
Server NC023, London, United Kingdom (Relay server), INTERNAL.NC023InternalUp?
Server NC028, Vancouver, Canada (Monitoring server), INTERNAL.NC028InternalUp?
Server NC031, New York, United States of America (Web server), INTERNAL.NC031InternalUp?
Server NC033, Toronto, Canada (Primary nameserver), OPERATIONAL.NC033OperationalUp?
Server NC034, Lusaka, Zambia (Phone server), INTERNAL.NC034InternalUp?
Server NC035, Sydney, Australia (Secondary nameserver), OPERATIONAL.NC035OperationalUp?
Server NC036, Amsterdam, Netherlands (Mail server), OPERATIONAL.NC036OperationalUp?
Server NC040, Toronto, Canada (Web server), INTERNAL.NC040InternalUp?
Server NC041, New York, United States of America (Web server), OPERATIONAL.NC041OperationalUp?
Server NC042, Seattle, United States of America (Status website), OPERATIONAL.NC042OperationalUp?

Subscriptions:

RSS icon. RSS

Twitter icon. Twitter

Search:

 

Recent Posts:

Archives:

Categories:

Links

Tags:

.co.zm domains .com.zm domains .zam.co domains back-up bounce messages browser warnings connection issues control panel database dns dos attack dot-zm domains down time email email delivery error messages ftp hardware imap mail mailing lists mail relay mail server microsoft migration nameservers network networking performance php phplist pop reboot shaw shaw communications inc. smtp spam spamassassin ssl ssl certificate tls tls certificate viruses webmail web server

Resources:

On NinerNet: