NinerNet Communications™
System Status

Server and System Status

NC027: Hotmail delivery update 3

19 April 2016 10:04:35 +0000

As I was hitting the “publish” button on the last update, another email came in from Microsoft that contradicted the email we had received just 68 minutes earlier. The latest email states, “Our investigation has determined that the [NC027 IP address does] not qualify for mitigation.”

We will continue to jump through Microsoft’s hoops, but in the meantime the workaround is still in place.

NC027: Hotmail delivery update 2

19 April 2016 09:47:19 +0000

After literally begging for a review of the situation and making three submissions to Microsoft, they have finally granted the IP address of server NC027 a “conditional mitigation”. This means that they have removed the block on this mail server’s IP address, but will put it back the second they see any reason to do so. Because they say that it will take 24-48 hours for this to propagate throughout their system we will leave the workaround in place for at least that long.

Two things have become clear from this and the Yahoo situation:

  • Dealing with megalithic entities like the all-powerful, large, free webmail providers — Yahoo, Hotmail, Gmail and their ilk — and their drone-like employees who are only allowed to copy and paste canned responses, is supremely frustrating.
  • We have a growing problem on our hands as a result of their huge footprint and their lack of concern for the many other companies like NinerNet on the Internet.

On the latter point, as stated in our first post on this issue with Hotmail, if you’re running a mailing list by blind-copying (or, even worse, simply cc’ing!) any more than about a dozen email addresses (of close friends or family or colleagues), we suspect that this may be the cause of the recent problems. Within the next 30 days — and hopefully much sooner than that — we will be implementing limits on the number of email addresses that can be cc’d or bcc’d in a single message. If you’re doing this, please contact us as soon as possible for options to ensure that your mailing list can continue to operate. It would be better to contact us as now in order to avoid disruptions to your mailing lists later.

Thank-you for your patience.

NC027: Hotmail delivery update

19 April 2016 01:39:45 +0000

We are still waiting for a response form Microsoft on this issue. In the meantime we have implemented a workaround that we developed during the Yahoo situation that will get email through to Hotmail and their domains, but there is a high likelihood that email to Hotmail domains will end up in the recipients’ “junk” or “spam” folders.

We will post further updates here as they become available.

NC027: Delivery issues to Hotmail and Outlook.com

18 April 2016 20:48:35 +0000

We have become aware that server NC027 has again been placed in the private block list of a major email provider. About three weeks ago Yahoo blocked this mail server for no good reason — and refused to reveal the nature and source of the alleged complaints that they received — and now Hotmail/Outlook.com has done the same. This means that all email to various Hotmail domains (including outlook.com) is being bounced immediately. Here is what the bounce message looks like for the hypothetical email address example@hotmail.com:

<example@hotmail.com>: host mx1.hotmail.com[207.46.8.167] said: 550 SC-001
    (BAY004-MC5F27) Unfortunately, messages from 212.71.255.195 weren't sent.
    Please contact your Internet service provider since part of their network
    is on our block list. You can also refer your provider to
    http://mail.live.com/mail/troubleshooting.aspx#errors. (in reply to MAIL
    FROM command)

This mail server’s IP address is — as usual — NOT in the major block lists / blacklists, all of which can be checked using a website like MXToolBox. This service checks 105 known blacklists, and we are listed in exactly 0 (zero) of them.

I will echo the advice that we posted during the Yahoo problem: If you’re forwarding email from your domain to a Hotmail account, we strongly suggest that you stop doing that. If you’re doing that or using personal, free accounts for business use on Hotmail or other free email providers, we also strongly suggest you stop doing that. If you need help achieving — with your own domain and the hosting services you’re already paying for — what you’re currently doing with free email accounts with third-party providers, talk to us and we’ll help you do it a better way.

Additionally, if you’re running a mailing list by blind-copying (or, even worse, simply cc’ing!) any more than about a dozen email addresses (of close friends or family), we suspect that this may be the cause of the recent problems. Within the next 30 days we will be implementing limits on the number of email addresses that can be cc’d or bcc’d in a single message. If you’re doing this, please contact us for options to ensure that your mailing list can continue to operate.

We are attempting to resolve this issue with Microsoft and will post updates here. Thanks for your patience.

Yahoo delivery working

31 March 2016 07:29:23 +0000

Currently email to Yahoo domains is getting through. We are closely monitoring this situation, but only expect to provide major updates from this point forwards.

Thank-you again for your patience with this issue.

Yahoo delivery update

31 March 2016 04:42:03 +0000

Yahoo finally replied to our request for information about why they are blocking us by providing a canned response which didn’t tell us anything we didn’t already read on their website. But they did apologise: “I want to apologize for any inconvenience this issue has caused you.”

Shortly after 01:30 UTC on 31 March we did see all emails still queued on our server for addresses on Yahoo and Yahoo-hosted domains clear — i.e., they were delivered. However, a few hours after that the block at Yahoo based on “user complaints” seems to have been reinstated, and so we are back to square one.

We have again written to Yahoo and are continuing to research possible workarounds … while we wait another day or two for their possible response.

In addition to researching workarounds, we’re taking some of Yahoo’s reasonable advice from their canned email and moving up certain changes to mail server configuration policies that they suggest. However, these cannot be implemented overnight across all clients’ domains without causing even more problems for our clients, so they will be introduced gradually.

We sincerely apologise for this inconvenience, but we cannot take responsibility for Draconian policies implemented at a very large email provider that has drastic effects on small hosting companies like NinerNet and our clients, and that are seemingly out of step with comparable large email providers like Gmail, Hotmail, AOL, etc. We’ve been in business for two decades — only two years less than Yahoo — and are not an upstart spamming operation that opened business last night and will be gone by next week. Clearly this kind of information is not being taken into account by Yahoo.

This is yet another example of why — on a network that requires cooperation between disparate entities, which is the very definition of the Internet — we shouldn’t do business with massive companies that can steamroll their way over the ability of smaller entities to participate. Signing up for a service from Yahoo (free or paid) or the parent companies of any of the other corporations mentioned in this post might be quick and easy (and probably even “free” of monetary cost), but there is always a price to be paid down the road. We and you are paying it today.

The workaround we’re working on may solve this issue, but there is still the risk that Yahoo will notice what we’ve done and block that avenue too. If you’re forwarding email from your domain to a Yahoo account, we strongly suggest that you stop doing that. If you’re doing that or using personal, free accounts for business use on Yahoo or other free email providers, we also strongly suggest you stop doing that. If you need help achieving — with your own domain and the hosting services you’re already paying for — what you’re currently doing with free email accounts with third-party providers, talk to us and we’ll help you do it a better way.

Further updates will continue to be posted here as they become available.

Delivery issues to Yahoo domains

27 March 2016 12:11:55 +0000

We have become aware of the fact that Yahoo is either delaying or, in come cases, outright refusing email from our primary mail server. This seems very odd, as our mail server’s IP address is not in any of the major anti-spam blacklists, and other major email providers (e.g., AOL, Gmail, Hotmail, Apple, etc.) are not blocking us.

We have been unsuccessful in trying to contact Yahoo about this as, like most massive companies, they put up massive barriers to communication from small companies or individuals.

An obvious exception to this that we have happily dealt with before to solve similar issues is AOL, who provide an excellent website where companies like NinerNet can look up the “reputation” of their mail servers’ IP addresses, and initiate communication with people who are willing and able to help address issues. Interestingly, this is AOL’s current report on the reputation of the mail server that Yahoo is blocking:

212.71.255.195 [nc027.ninernet.net] Reputation is Good

What does a good reputation mean?

The IP address has a good reputation. This means AOL has enough data to determine that the IP address is a known mail sender with a good reputation. You should not be experiencing bounce or reject errors when sending mail from this IP.

Reputation of NinerNet mail server IP address with AOL.

Reputation of NinerNet mail server IP address with AOL.

We will monitor this situation and make further attempts to contact Yahoo and have them explain or remove this block. In the meantime, this serves as yet another example of why using “free” email on monolithic systems run by massive companies is a bad idea.

London network restored

27 December 2015 10:17:45 +0000

The data centre informs us that connectivity on their network was restored at 09:24 UTC.

We have been able to confirm this by accessing all of our servers in this data centre:

  • Incoming mail is once again being processed immediately by server NC027, although any incoming mail sent during the disruption may be delayed a little longer until the sending mail servers try again. This is not under our control.
  • Server NC023 is also processing email immediately.

We apologise for this interruption. Denial of service attacks (especially distributed denial of service attacks) are extremely difficult and expensive to fight. We have confidence that the data centre did everything they could as quickly as they could to combat this attack.

London data centre DDoS update

27 December 2015 08:30:17 +0000

We have been updated that the denial of service attack against the London data centre is actually a large distributed denial of service (DDoS) attack, and it is still underway. Efforts are being made at multiple levels on the network to combat this attack, and we are seeing some movement of email through server NC027, although it took an hour and forty-seven minutes for a test message to get through. This indicates, as mentioned earlier, that the server is up and processing email, but that other servers (and our clients, obviously) are having problems connecting to the server.

We apologise for these problems, but this is affecting a wide range and large number of companies hosted at this data centre, not just NinerNet.

Updates will continue to be posted here as we learn more.

Connectivity issue at London data centre

27 December 2015 06:52:18 +0000

The London data centre has been suffering from the effects of a couple of denial of service (DoS) attacks over the last 12 or so hours. The attack late yesterday (26 December) lasted about an hour and a half. Today’s attack started at 05:37 UTC and is currently ongoing. The data centre staff assure us that they have identified the problem and are implementing a fix for it.

This affects server NC023 (the relay server) and server NC027 (the main mail server), as well as virtual private servers hosted in London. As far as we know, the servers themselves are up and connected to the network, but the attack is causing them to be inaccessible.

We’ll update here as soon as we know more.

NinerNet home page

Systems at a Glance:


Loc.SystemStatusPing
Server NC023, London, United Kingdom (Relay server), INTERNAL.NC023InternalUp?
Server NC028, Vancouver, Canada (Monitoring server), INTERNAL.NC028InternalUp?
Server NC031, New York, United States of America (Web server), INTERNAL.NC031InternalUp?
Server NC033, Toronto, Canada (Primary nameserver), OPERATIONAL.NC033OperationalUp?
Server NC034, Lusaka, Zambia (Phone server), INTERNAL.NC034InternalUp?
Server NC035, Sydney, Australia (Secondary nameserver), OPERATIONAL.NC035OperationalUp?
Server NC036, Amsterdam, Netherlands (Mail server), OPERATIONAL.NC036OperationalUp?
Server NC040, Toronto, Canada (Web server), INTERNAL.NC040InternalUp?
Server NC041, New York, United States of America (Web server), OPERATIONAL.NC041OperationalUp?
Server NC042, Seattle, United States of America (Status website), OPERATIONAL.NC042OperationalUp?

Subscriptions:

RSS icon. RSS

Twitter icon. Twitter

Search:

 

Recent Posts:

Archives:

Categories:

Links

Tags:

.co.zm domains .com.zm domains .zam.co domains back-up bounce messages browser warnings connection issues control panel database dns dos attack dot-zm domains down time email email delivery error messages ftp hardware imap mail mailing lists mail relay mail server microsoft migration nameservers network networking performance php phplist pop reboot shaw shaw communications inc. smtp spam spamassassin ssl ssl certificate tls tls certificate viruses webmail web server

Resources:

On NinerNet: