NinerNet Communications™
System Status

Server and System Status

NC023 and NC027: Emergency server maintenance

18 January 2018 02:54:25 +0000

Per our previous blog post, the next of our servers to undergo updating are NC023 (the relay server) and NC027 (the primary mail server). The maintenance windows for each are two hours’ long, but we expect that each server will be down for much less than that.

The maintenance window for both starts at:

Please click the link above to determine when this will be in your time zone.

During the maintenance you will not be able to send or receive email. Incoming email to your domain will be held on the sending servers until our mail server is back online, and will then be delivered to your accounts. No email will be lost.

Please note that email to NinerNet will also be down during the maintenance on server NC027.

NC027: Spam clean-up update

28 December 2017 12:27:23 +0000

We have cleaned up the mail server (NC027) after yet another email account was compromised. This necessitated shutting down the mail server between 22:39 and 22:52 UTC yesterday (27 September) while we cleaned up the mess.

This has resulted in the mail server being blacklisted by at least one large mail provider and restrictions put in place by others. The email account in question has been disabled pending resolution of the root cause of this issue, and we are diverting outgoing email to some major mail providers via our relay server until restrictions on our primary mail server’s IP address expire. However, it may still be a few hours more until some outgoing mail is delivered normally without delay.

These incidents usually arise after a client’s computer has been infected with a virus. The virus then sends the email password back to the person or organisation controlling the virus, and they then use that information to compromise that email account on the mail server, using it to send thousands of spams from the account. Please ensure that you install, use and update an anti-virus program on your computers and any other devices to ensure that this doesn’t happen to your email account.

At this time NC027’s IP address is not listed in any of the major blacklists (which operate on an automated basis to remove blacklisted IP addresses once no spam is seen from them), but we will (as always) monitor this and, where necessary, make manual submissions to the smaller, niche blacklists and to ISPs and other mail providers to have our IP address de-listed where that is possible. Manual processes like these can take a couple of days, however.

NC027: Spam and delayed delivery

23 December 2017 03:11:45 +0000

Two email accounts on two separate domains on the mail server (NC027) were compromised in the last 24 hours and were used to send out thousands of spam emails. This has resulted in the mail server being blacklisted by at least one large mail provider and restrictions put in place by others. The two email accounts in question have been disabled pending resolution of the root cause of this issue, and we are diverting outgoing email to major mail providers via our relay server until restrictions on our primary mail server’s IP address expire. However, it may still be a few hours more until some outgoing mail is delivered normally without delay.

These incidents usually arise after a client’s computer has been infected with a virus. The virus then sends the email password back to the person or organisation controlling the virus, and they then use that information to compromise that email account on the mail server, using it to send thousands of spams from the account. Please ensure that you install, use and update an anti-virus program on your computers and any other devices to ensure that this doesn’t happen to your email account.

At this time NC027’s IP address is not listed in any of the major blacklists (which operate on an automated basis to remove blacklisted IP addresses once no spam is seen from them), but we will (as always) monitor this and, where necessary, make manual submissions to the smaller, niche blacklists and to ISPs and other mail providers to have our IP address de-listed where that is possible. Manual processes like these can take a couple of days, however.

NC023: Server migration complete

19 November 2017 01:32:47 +0000

The migration of the relay server to new hardware has been completed. The server was offline between 20:20 and 20:39 UTC. Additionally, it now has more disk space and is accessible over IPv6 as well.

NC023: Sever migration starting now

18 November 2017 20:21:05 +0000

The relay server migration will be starting within the next few minutes.

NC023: Migration of relay server to new hardware

17 November 2017 07:22:19 +0000

Server NC023 (the relay server) will be migrated to new hardware during our regular weekly maintenance window this weekend. This maintenance will start at 20:00 UTC on Saturday 18 November 2017, and should conclude within an hour. During this time the relay server will be offline.

This maintenance only affects clients using the relay mail server; it does not affect operation of the regular email server that processes the vast majority of the mail carried by NinerNet.

We will post an update here shortly before, at the start of and following the maintenance.

If you have any questions or concerns, please contact support to let us know. Thank-you.

NC027: Spam cleaned up

7 September 2017 13:23:50 +0000

We have cleaned up the spam on server NC027 and managed to funnel most of the delayed email through our relay server. Although the primary mail server’s IP address is not in a couple of the major blacklists, we are still in some niche ones with major ISPs and mail providers. We have been working to have the IP address de-listed where that is possible, but manual processes like these can take a couple of days.

In the meantime, at the moment most email is flowing normally, but email to most domains will still be delayed for the next few hours.

London network restored

27 December 2015 10:17:45 +0000

The data centre informs us that connectivity on their network was restored at 09:24 UTC.

We have been able to confirm this by accessing all of our servers in this data centre:

  • Incoming mail is once again being processed immediately by server NC027, although any incoming mail sent during the disruption may be delayed a little longer until the sending mail servers try again. This is not under our control.
  • Server NC023 is also processing email immediately.

We apologise for this interruption. Denial of service attacks (especially distributed denial of service attacks) are extremely difficult and expensive to fight. We have confidence that the data centre did everything they could as quickly as they could to combat this attack.

London data centre DDoS update

27 December 2015 08:30:17 +0000

We have been updated that the denial of service attack against the London data centre is actually a large distributed denial of service (DDoS) attack, and it is still underway. Efforts are being made at multiple levels on the network to combat this attack, and we are seeing some movement of email through server NC027, although it took an hour and forty-seven minutes for a test message to get through. This indicates, as mentioned earlier, that the server is up and processing email, but that other servers (and our clients, obviously) are having problems connecting to the server.

We apologise for these problems, but this is affecting a wide range and large number of companies hosted at this data centre, not just NinerNet.

Updates will continue to be posted here as we learn more.

Connectivity issue at London data centre

27 December 2015 06:52:18 +0000

The London data centre has been suffering from the effects of a couple of denial of service (DoS) attacks over the last 12 or so hours. The attack late yesterday (26 December) lasted about an hour and a half. Today’s attack started at 05:37 UTC and is currently ongoing. The data centre staff assure us that they have identified the problem and are implementing a fix for it.

This affects server NC023 (the relay server) and server NC027 (the main mail server), as well as virtual private servers hosted in London. As far as we know, the servers themselves are up and connected to the network, but the attack is causing them to be inaccessible.

We’ll update here as soon as we know more.

NinerNet home page

Systems at a Glance:


Loc.SystemStatusPing
Server NC023, London, United Kingdom (Relay server), INTERNAL.NC023InternalUp?
Server NC028, Vancouver, Canada (Monitoring server), INTERNAL.NC028InternalUp?
Server NC031, New York, United States of America (Web server), INTERNAL.NC031InternalUp?
Server NC033, Toronto, Canada (Primary nameserver), OPERATIONAL.NC033OperationalUp?
Server NC034, Lusaka, Zambia (Phone server), INTERNAL.NC034InternalUp?
Server NC035, Sydney, Australia (Secondary nameserver), OPERATIONAL.NC035OperationalUp?
Server NC036, Amsterdam, Netherlands (Mail server), OPERATIONAL.NC036OperationalUp?
Server NC040, Toronto, Canada (Web server), INTERNAL.NC040InternalUp?
Server NC041, New York, United States of America (Web server), OPERATIONAL.NC041OperationalUp?
Server NC042, Seattle, United States of America (Status website), OPERATIONAL.NC042OperationalUp?

Subscriptions:

RSS icon. RSS

Twitter icon. Twitter

Search:

 

Recent Posts:

Archives:

Categories:

Links

Tags:

.co.zm domains .com.zm domains .zam.co domains back-up bounce messages browser warnings connection issues control panel database dns dos attack dot-zm domains down time email email delivery error messages ftp hardware imap mail mailing lists mail relay mail server microsoft migration nameservers network networking performance php phplist pop reboot shaw shaw communications inc. smtp spam spamassassin ssl ssl certificate tls tls certificate viruses webmail web server

Resources:

On NinerNet: