NinerNet Communications™
System Status

Server and System Status

NC027: Blacklist update

27 April 2018 12:53:35 +0000

Good news! NC027‘s IP address has been removed from the blacklist it was in, so at 12:41 UTC we switched mail processing back to the primary server. NC027 is in no blacklists of which we are aware.

Please note that the events of the last couple of days have pushed the notices we issued of upcoming scheduled maintenance on three of our servers down the page. If you have missed them please see:

Thank-you for your patience over the last 24 hours. If you have any questions or concerns, please do let us know.

NC027: Blacklist situation

27 April 2018 06:13:40 +0000

As explained yesterday, a client’s compromised email account sent out thousands of spam emails before it was detected and stopped. This has happened before, but the circumstances this time are different.

Most blacklists are automated, both in adding IP addresses to the blacklist and in removing them. This is a double-edged sword. On the plus side, IP addresses that are the source of spam are quickly added, making it less likely that spam will get through in subsequent attempts from the same IP address. Most, if not all, automated blacklists then remove the bad IP address fairly quickly after the spam stops. They realise that stuff happens, and when the spam stops they assume the problem is fixed and remove the IP address. There is short-term pain, but it’s measured in hours and the block is generally removed within your business day.

On the negative side, organisations and people that run blacklists are generally unwilling to manually remove IP addresses before they automatically expire. In and of itself this isn’t actually a bad thing; many blacklist wouldn’t be able to function if they had to field pleas and demands that IP addresses be removed. Quick, automated removal when the problem that caused the listing in the first place is fixed is the cure.

Unfortunately this situation has exposed a blacklist that actually seems to be designed to punish mail servers that have had a temporary problem, even after the problem has been stopped. This is unusual in our experience, as it makes the blacklist less useful, by blocking legitimate email long after the problem has been addressed. Information on their website states that it could be “a week or more” before an IP address is removedif they determine the spam outbreak to be severe enough — without defining “severe” — even though it has stopped. And since the addition and removal of IP addresses is automated, “you cannot” get your IP address removed manually. Full stop.

Since this blacklist is still blocking our mail server’s IP address almost 24 hours later, at 05:01 UTC we started relaying all mail sent by clients through our relay mail server (NC023), which has a different IP address. We will continue to monitor the blacklist in question and reverse this once our IP address is removed.

It seems that most of the mail servers we’ve seen using this blacklist are in South Africa. Mail bounced using this blacklist will show a message like the following, using real email addresses, domains and IP addresses of course:

<destination@example.com>: host something.co.za[1.2.3.4] said:
    550-rejected because 212.71.255.195 is in a black list at
    truncate.gbudb.net 550 http://www.gbudb.com/truncate/ [212.71.255.195]
    (in reply to RCPT TO command)

If you’ve seen this, we suggest that you contact the person to whom you sent the email and suggest that they tell their hosting company that they should stop using blacklists that don’t operate within the norms of most blacklists. Feel free to point them to this blog post.

With all of the above said, we will be setting up a new mail server and migrating all accounts to it within the next couple of weeks. The new server will be better equipped to spot and stop these outbreaks automatically before they become “severe”.

As always, we appreciate your patience, and we also appreciate those clients that keep their anti-virus software up to date. If you have any questions, please feel free to contact us. Thank-you.

NC027: Spam cleaned up

26 April 2018 11:22:13 +0000

We have cleaned up the mail server (NC027) after an email account was compromised. This has resulted in the mail server being placed in at least one blacklist. The email account in question has been disabled pending resolution by the client of the root cause of this issue, but it will be a few hours before restrictions on our primary mail server’s IP address put in place by this blacklist expire.

These incidents usually arise after a client’s computer has been infected with a virus. The virus then sends the email password back to the person or organisation controlling the virus, and they then use that information to compromise that email account on the mail server, using it to send thousands of spams from the account. Please ensure that you install, use and update an anti-virus program on your computers and any other devices to ensure that this doesn’t happen to your email account.

We apologise for this incident. Please contact us if you have any questions. Thank-you.

NC027: Spam clean-up update

28 December 2017 12:27:23 +0000

We have cleaned up the mail server (NC027) after yet another email account was compromised. This necessitated shutting down the mail server between 22:39 and 22:52 UTC yesterday (27 September) while we cleaned up the mess.

This has resulted in the mail server being blacklisted by at least one large mail provider and restrictions put in place by others. The email account in question has been disabled pending resolution of the root cause of this issue, and we are diverting outgoing email to some major mail providers via our relay server until restrictions on our primary mail server’s IP address expire. However, it may still be a few hours more until some outgoing mail is delivered normally without delay.

These incidents usually arise after a client’s computer has been infected with a virus. The virus then sends the email password back to the person or organisation controlling the virus, and they then use that information to compromise that email account on the mail server, using it to send thousands of spams from the account. Please ensure that you install, use and update an anti-virus program on your computers and any other devices to ensure that this doesn’t happen to your email account.

At this time NC027’s IP address is not listed in any of the major blacklists (which operate on an automated basis to remove blacklisted IP addresses once no spam is seen from them), but we will (as always) monitor this and, where necessary, make manual submissions to the smaller, niche blacklists and to ISPs and other mail providers to have our IP address de-listed where that is possible. Manual processes like these can take a couple of days, however.

NC027: Yet another spam outbreak

27 December 2017 22:41:47 +0000

We have temporarily shut down the mail server (NC027) while we clean up tens of thousands of spams from another compromised email account. We will have it back online as soon as possible.

NC027: Spam and delayed delivery

23 December 2017 03:11:45 +0000

Two email accounts on two separate domains on the mail server (NC027) were compromised in the last 24 hours and were used to send out thousands of spam emails. This has resulted in the mail server being blacklisted by at least one large mail provider and restrictions put in place by others. The two email accounts in question have been disabled pending resolution of the root cause of this issue, and we are diverting outgoing email to major mail providers via our relay server until restrictions on our primary mail server’s IP address expire. However, it may still be a few hours more until some outgoing mail is delivered normally without delay.

These incidents usually arise after a client’s computer has been infected with a virus. The virus then sends the email password back to the person or organisation controlling the virus, and they then use that information to compromise that email account on the mail server, using it to send thousands of spams from the account. Please ensure that you install, use and update an anti-virus program on your computers and any other devices to ensure that this doesn’t happen to your email account.

At this time NC027’s IP address is not listed in any of the major blacklists (which operate on an automated basis to remove blacklisted IP addresses once no spam is seen from them), but we will (as always) monitor this and, where necessary, make manual submissions to the smaller, niche blacklists and to ISPs and other mail providers to have our IP address de-listed where that is possible. Manual processes like these can take a couple of days, however.

NC027: Spam cleaned up

7 September 2017 13:23:50 +0000

We have cleaned up the spam on server NC027 and managed to funnel most of the delayed email through our relay server. Although the primary mail server’s IP address is not in a couple of the major blacklists, we are still in some niche ones with major ISPs and mail providers. We have been working to have the IP address de-listed where that is possible, but manual processes like these can take a couple of days.

In the meantime, at the moment most email is flowing normally, but email to most domains will still be delayed for the next few hours.

NC027: Mail server issues

7 September 2017 11:39:46 +0000

We have cleaned up server NC027 after a client’s email account was compromised, resulting in thousands of spams being sent from the mail server. This is having a negative impact on email being sent from the server, as its IP address is now in anti-spam blacklists. This means that outgoing email sent from our clients will be bounced or delayed until the blacklist realise that the spam from our mail server has stopped.

We are working on implementing a workaround through one of our other, clean mail servers, but unfortunately it will be several hours before outgoing email to most domains is delivered normally.

These incidents usually arise after a computer has been infected with a virus. The virus then sends the email password back to the person or organisation controlling the virus, and they then use that information to compromise that email account on the mail server, using it to send thousands of spams from the account. Please ensure that you install, use and update an anti-virus program on your computer to ensure that this doesn’t happen to your email account.

We apologise for this incident. Please contact us if you have any questions. Thank-you.

NC027: Outgoing mail may be delayed

31 July 2017 04:29:48 +0000

An email account on server NC027 was compromised in the last few hours, and it was used to send spam. This problem has been addressed by suspending the account, but the result is that outgoing email from the server may be delayed for the next few hours until automated anti-spam systems around the Internet react to our cleaning up the problem.

We apologise for this problem, and remind everyone to ensure that their computers are scanned regularly for viruses.

If you have any questions about this, please contact support. Thank-you.

NC027: Mail server back online

23 November 2016 07:28:27 +0000

The mail server (NC027) has been restarted after cleaning up after a spammer. It was down for eight minutes between 07:09 and 07:17 UTC.

As a result of this incident, our mail server is being blocked by some mail services. We will work to have those blocks removed before they expire, but this will take some time.

These incidents usually arise after a computer has been infected with a virus. The virus then sends the email password back to the person or organisation controlling the virus, and they then use that information to compromise that email account on the mail server, using it to send thousands of spams from the account. Please ensure that you install, use and update an anti-virus program on your computer to ensure that this doesn’t happen to your email account.

We again apologise for this incident. Please contact us if you have any questions. Thank-you.

NinerNet home page

Systems at a Glance:


Loc.SystemStatusPing
Server NC020, Chicago, United States of America, OPERATIONAL.NC020OperationalPing
Server NC023, London, United Kingdom, OPERATIONAL.NC023OperationalPing
Server NC027, London, United Kingdom, OPERATIONAL.NC027OperationalPing
Server NC028, Vancouver, Canada, INTERNAL.NC028InternalPing
Server NC031, Newark, United States of America, OPERATIONAL.NC031OperationalPing
Server NC033, Toronto, Canada., OPERATIONAL.NC033OperationalPing
Server NC034, Lusaka, Zambia, INTERNAL.NC034InternalPing
Server NC035, Sydney, Australia, PROVISIONING.NC035ProvisioningPing
Server NC036, London, United Kingdom, PROVISIONING.NC036ProvisioningPing

Subscriptions:

RSS icon. RSS

Twitter icon. Twitter

General Information:

This blog provides information about the status of NinerNet Communications systems. Dates and times of posts to this blog are in the UTC time zone, and dates and times given for events are also in the UTC time zone, although conversions may be offered for some time zones common to our clients. Please use the World Time Server to ensure accurate conversion of dates and times to your own time zone.

Search:

 

Recent Posts:

Archives:

Categories:

Links

Tags:

.co.zm domains .com.zm domains .zam.co domains browser warnings connection issues database dns dos attack dot-zm domains down time email delivery ftp hardware mailing lists mail relay migration nameservers performance phplist reboot smtp spam spamassassin ssl ssl certificate tls tls certificate web server

Resources:

On NinerNet: