NinerNet Communications™
System Status

Server and System Status

NC027: SSL issue update

24 March 2014 06:29:06 +0000

We’re now getting emails from clients telling us that the issue has apparently resolved itself. We’re continuing to investigate, but we have not done anything to the server, either before this issue started or since it was reported to us.

Updates will continue to be posted here.

NC027: SSL issue

24 March 2014 06:06:57 +0000

We are getting reports from clients that they are getting SSL-related error messages when connecting to this mail server. We have not yet been able to reproduce these errors ourselves, but we are investigating to find out what’s happening.

We will post further updates here.

NC018: Investigation findings

8 March 2014 00:57:23 +0000

The web server on server NC018 sustained a denial-of-service attack between approximately 14:00 and 18:00 UTC on 7 March. This is why websites on this server were unavailable during this period. No other services (email or nameservers) were affected.

We have implemented improved monitoring of this activity on this server. Additionally, we are in discussions with the data centre to move this server to an improved network infrastructure, a move we hope to implement before the end of March.

We apologise for this disruption.

NC018: Web server outage

7 March 2014 18:22:18 +0000

We have had reports that the web server on server NC018 was down in the late afternoon UTC. We restarted the web server at 18:07 UTC and it is up right now.

We’re looking into this and will report back here shortly.

NC027: Investigation findings

26 January 2014 16:00:09 +0000

Server NC027 (the new mail-only server) experienced a great and very brief increase in activity at about 04:00 UTC on 23 January, and we suspect that this lead to the server getting behind in scanning and processing email. The hard drives simply couldn’t keep up.

In order to allow the server to catch up, we disabled virus and spam scanning. Over the course of the coming week we’ll be re-enabling each of these (inbound and outbound) one at a time per day. This will allow us to determine which activity is the bottleneck and make finer adjustments accordingly. We also have a back-up plan to move the accounts on this server to one with faster (solid-state) drives, if necessary.

We apologise again for the inconvenience of significantly delayed email on Thursday. If you have any questions about this incident, please contact NinerNet support. Thank-you.

NC023: Spam cleaned up

26 January 2014 15:45:01 +0000

The spam on server NC023 (the relay server) has been cleaned up and the offending IP address blocked. Most of the legitimate email to destination servers that had been deferred as a result of the spam has also now been delivered; however, some mail servers are still not accepting email from this server, so that email will continue to be delayed a while longer until their systems are automatically updated to realise that this server is no longer spamming. This is currently only affecting about twenty messages.

We apologise for this interruption. If you have any questions or concerns, please contact NinerNet support. Thank-you for your patience.

NC023: Relay server stopped to clean up spam

26 January 2014 13:28:29 +0000

The relay server has been stopped while we clean up a spam outbreak, and identify and block the source. We will post updates here. We expect to have the server back online within the next 30-40 minutes.

NC027: Mail queue almost clear

23 January 2014 14:48:04 +0000

The mail queue will be cleared within the next 20-30 minutes.

Once the queue is clear we’ll be looking for a cause for this incident. There was a great spike in activity on this server around 04:00 UTC, and it’s quite possible that the initial back-up started around that time and didn’t quite manage to catch up.

NC027: New mail delivered immediately

23 January 2014 13:25:21 +0000

Since reconfiguring the mail server we’ve noticed that new email is being delivered right away. It is now only the backlog that is still slowly being processed.

NC027: Performance improvement

23 January 2014 11:55:41 +0000

We have made some adjustments to the configuration of the mail server on NC027 and as a result the mail queue seems to be clearing, albeit very slowly. We’re continuing to work on this to speed up delivery.

NinerNet home page

Systems at a Glance:


Loc.SystemStatusPing
Server NC023, London, United Kingdom (Relay server), INTERNAL.NC023InternalUp?
Server NC028, Vancouver, Canada (Monitoring server), INTERNAL.NC028InternalUp?
Server NC031, New York, United States of America (Web server), INTERNAL.NC031InternalUp?
Server NC033, Toronto, Canada (Primary nameserver), OPERATIONAL.NC033OperationalUp?
Server NC034, Lusaka, Zambia (Phone server), INTERNAL.NC034InternalUp?
Server NC035, Sydney, Australia (Secondary nameserver), OPERATIONAL.NC035OperationalUp?
Server NC036, Amsterdam, Netherlands (Mail server), OPERATIONAL.NC036OperationalUp?
Server NC040, Toronto, Canada (Web server), INTERNAL.NC040InternalUp?
Server NC041, New York, United States of America (Web server), OPERATIONAL.NC041OperationalUp?
Server NC042, Seattle, United States of America (Status website), OPERATIONAL.NC042OperationalUp?

Subscriptions:

RSS icon. RSS

Twitter icon. Twitter

Search:

 

Recent Posts:

Archives:

Categories:

Links

Tags:

.co.zm domains .com.zm domains .zam.co domains back-up bounce messages browser warnings connection issues control panel database dns dos attack dot-zm domains down time email email delivery error messages ftp hardware imap mail mailing lists mail relay mail server microsoft migration nameservers network networking performance php phplist pop reboot shaw shaw communications inc. smtp spam spamassassin ssl ssl certificate tls tls certificate viruses webmail web server

Resources:

On NinerNet: