NinerNet Communications™
System Status

Server and System Status

NC027: Investigation findings

26 January 2014 16:00:09 +0000

Server NC027 (the new mail-only server) experienced a great and very brief increase in activity at about 04:00 UTC on 23 January, and we suspect that this lead to the server getting behind in scanning and processing email. The hard drives simply couldn’t keep up.

In order to allow the server to catch up, we disabled virus and spam scanning. Over the course of the coming week we’ll be re-enabling each of these (inbound and outbound) one at a time per day. This will allow us to determine which activity is the bottleneck and make finer adjustments accordingly. We also have a back-up plan to move the accounts on this server to one with faster (solid-state) drives, if necessary.

We apologise again for the inconvenience of significantly delayed email on Thursday. If you have any questions about this incident, please contact NinerNet support. Thank-you.

NC023: Spam cleaned up

26 January 2014 15:45:01 +0000

The spam on server NC023 (the relay server) has been cleaned up and the offending IP address blocked. Most of the legitimate email to destination servers that had been deferred as a result of the spam has also now been delivered; however, some mail servers are still not accepting email from this server, so that email will continue to be delayed a while longer until their systems are automatically updated to realise that this server is no longer spamming. This is currently only affecting about twenty messages.

We apologise for this interruption. If you have any questions or concerns, please contact NinerNet support. Thank-you for your patience.

NC023: Relay server stopped to clean up spam

26 January 2014 13:28:29 +0000

The relay server has been stopped while we clean up a spam outbreak, and identify and block the source. We will post updates here. We expect to have the server back online within the next 30-40 minutes.

NC027: Mail queue almost clear

23 January 2014 14:48:04 +0000

The mail queue will be cleared within the next 20-30 minutes.

Once the queue is clear we’ll be looking for a cause for this incident. There was a great spike in activity on this server around 04:00 UTC, and it’s quite possible that the initial back-up started around that time and didn’t quite manage to catch up.

NC027: New mail delivered immediately

23 January 2014 13:25:21 +0000

Since reconfiguring the mail server we’ve noticed that new email is being delivered right away. It is now only the backlog that is still slowly being processed.

NC027: Performance improvement

23 January 2014 11:55:41 +0000

We have made some adjustments to the configuration of the mail server on NC027 and as a result the mail queue seems to be clearing, albeit very slowly. We’re continuing to work on this to speed up delivery.

Server NC027 performance degraded

23 January 2014 09:07:10 +0000

Server NC027 has a large and growing mail queue, but the server is up. We are investigating and will post updates here.

NinerNet home page

Systems at a Glance:


SystemStatusPing
NC018OperationalPing
NC019InternalPing
NC020OperationalPing
NC023OperationalPing
NC024InternalPing
NC027OperationalPing
NC028InternalPing
NC031OperationalPing

Subscriptions:

RSS icon. RSS

Twitter icon. Twitter

Identi.ca icon. Identi.ca

General Information:

This blog provides information about the status of NinerNet Communications systems. Dates and times of posts to this blog are in the UTC time zone, and dates and times given for events are also in the UTC time zone, although conversions may be offered for some time zones common to our clients. Please use the World Time Server to ensure accurate conversion of dates and times to your own time zone.

Search:

 

Recent Posts:

Archives:

Categories:

Links

Tags:

.co.zm domains .com.zm domains .zam.co domains browser warnings connection issues dns dos attack dot-zm domains down time email delivery ftp hardware mailing lists mail relay migration nameservers performance phplist reboot smtp spam spamassassin ssl ssl certificate tls tls certificate web server

Resources:

On NinerNet: