NinerNet Communications™
System Status

Server and System Status

NC036: Planned data centre maintenance

15 February 2024 05:14:21 +0000

There is planned network maintenance, “to improve performance and scalability”, for the data centre in Amsterdam where server NC036 (our primary mail server) is located. This maintenance is not expected to result in any downtime, but if any issues arise “affected [servers] may experience increased latency or a brief disruption in network traffic.” This will take place on Tuesday 27 February 2024 from 16:00 to 20:00 UTC. This is 08:00 to 12:00 Pacific Time and 18:00 to 22:00 Central Africa Time on the same date.

If you’d like to calculate this for any other location, please use the World Time Server.

We only post this to ensure you’re aware of the reason for any possible issues in advance. However, based on previous experience, we don’t anticipate any issues.

If you have any questions, please do contact NinerNet support. Thank-you.

Email messages from Xneelo (formerly Hetzner South Africa) senders blocked

19 January 2024 01:23:15 +0000

We’ve been meaning to post about this for several years (since 2020), but we haven’t because there’s always something more important to do and it’s not our job to highlight how badly a competitor is running their mail servers. However, after the issue had gone away for a time we thought that perhaps Xneelo (formerly Hetzner South Africa) had resolved their problem. But now it’s back.

The problem is that one of the multiple mail servers they run is in multiple anti-spam blacklists/blocklists. We don’t know how many outbound mail servers they run, but let’s say (for the sake of example) it’s ten. If the IP address of one of those servers is in a blacklist, and all of their outbound mail is equally spread among those ten servers, then 10% of their outbound messages will not be delivered to anyone that is using the blacklists in which that server is listed.

The problem for us is that we end up devoting a significant percentage of our support resources to answering questions from clients who don’t understand what is going on, and who think that we are the problem. But we’re not!

Getting into an anti-spam blacklist is a significant event for any company, but the bigger you are and the more servers you run the less of an issue it is. NinerNet is not big, and so if one of our servers gets into a blacklist it’s a big deal, and we jump around to fix the problem and have our IP address removed from the blacklist. Considering how long one of Xneelo’s mail servers has been in a major blacklist, and how many of their clients we (or our clients) have told to take the problem to Xneelo, it’s shocking to us that they don’t seem to have done a thing about it. So we’re making this blog post to try and bring their attention to it publicly and to point our clients here whenever they have a question about the problem.

If one of your correspondents tells you that their email messages to you are not getting through to you, and they’re a Xneelo customer, this is almost certainly the reason. Your correspondent — the Xneelo client — needs to go to Xneelo to demand that Xneelo resolve their years-old problem in order to serve their own clientele properly.


Update, 2024-03-01: A Xneelo client that was failing to correspond with one of our clients engaged Xneelo support on 28 February 2024, and we are in ongoing discussions with Xneelo for them to fix this problem. Xneelo have acknowledged that at least two of their outgoing mail servers (197.189.244.82 and 197.189.244.90) are in six blacklists between them, so the situation is actually far worse than the hypothetical example given above. When the issue is resolved we will post an update here.

Update, 2024-03-05: Almost a week later and Xneelo are still spouting excuses and fabrications, and assuming that the people they are addressing (including us!) are too stupid to understand how email works. This may or may not be resolved at some point, but at this point it seems it likely won’t be. Email service providers should cooperate with one another to reduce spam across the Internet, and they should communicate truthfully with one another, but that seems to be too much of a challenge for Xneelo. We’d love to run our mail servers without having to consult blacklists of spammers, but your email account would become instantly useless if we did. All we can suggest you do is to advise your correspondents to use email hosting providers that take care of the reputations of their IP addresses, as otherwise their email will be tainted with the same negative reputation. This is how the combined actions of hosting clients are supposed to drive the bad actors out of business.

Update, 2024-03-14: It has emerged that Xneelo doesn’t even handle greylisting properly! One of the parties involved in the ticket with Xneelo decided to communicate with us directly (to become a client, ironically), but because Xneelo’s mail servers don’t respond correctly to NinerNet’s use of the well-established anti-spam technique greylisting, their messages to us are now bouncing! Apparently this client of Xneelo has done the right thing and told Xneelo that they will be moving their hosting business away if they don’t fix the problem. It blows us away how badly Xneelo is running their mail servers!

Update 2, 2024-03-14: We have added various IP addresses, server names and domains to various whitelists on our mail servers. Adding the blacklisted IP address will not have any effect on messages from Xneelo’s blacklisted servers that are listed in the blacklists our anti-spam system consults, but these additions are, as far as we can tell so far, having a positive effect on Xneelo’s inability to handle greylisting. If you communicate with a South African correspondent who is hosted by Xneelo and their emails to you are bouncing or taking hours to come through, please log into the mail server control panel and add their domain at Domains and Accounts -> YOUR_DOMAIN -> Greylisting -> Do not apply greylisting on listed senders. If your correspondent is susie@example.com, you need to add “@example.com” (with the leading @ symbol) to that text box, and click the green “Save changes” button.

All servers: Company SSL/TLS certificates on all servers renewed and updated

15 January 2024 04:24:46 +0000

The *.niner.net wildcard SSL/TLS certificate has been renewed and updated across our entire infrastructure, although we apologise for being about four hours late. This should be seamless for all our clients. However, should you run into a situation in the next 24 hours where you’re told that the certificate has expired, please log out and reload whatever it is you’re trying to do. You may need to reboot, but this would be very unusual. Simply forcing a reload should clear things up. Via FTPS you may need to re-trust our certificate depending on your FTP client.

This doesn’t affect certificates on your own domain on your website, just NinerNet services such as control panels and connecting to the mail and FTP servers.

If you have any questions or concerns, please do contact NinerNet support with any error messages you may be seeing. Thank-you.

NC036: Network issue at data centre

9 January 2024 04:55:27 +0000

Server NC036, our primary mail server, was apparently down for 34 minutes yesterday, 8 January. Our server itself was not down, but the data centre experienced what seems to have been a partial connectivity issue. In fact, you may not even have noticed it, as we didn’t receive any complaints about the downtime.

According to the data centre’s report, the issue was resolved and it is not expected to recur. We apologise to anyone who was affected.

NC041: Web server rebooted

19 June 2023 18:53:54 +0000

Server NC041 has been rebooted. This appears to have dealt with the database server issue. We will continue to look for the root cause to ensure this doesn’t happen again.

NC041: Web server having some database issues

19 June 2023 17:35:03 +0000

The database server on server NC041 appears to be having some issues. We are looking into it and will post an update about what we find shortly.

NC036: Connectivity problems in Zambia

18 May 2023 19:54:44 +0000

Today we received a number of complaints of connectivity problems to our mail server. However, they were all from Zambia, from the Luangwa Valley to the Copperbelt to Lusaka; nobody else had any issues.

Below is a screenshot of NinerNet’s live status page that was taken moments ago that shows that the mail server (NC036) has been up for 100% of the last 90 days.

NinerNet live status screenshot, 2023-05-18.

NinerNet live status screenshot, 2023-05-18.

In the absence of any other information we can only assume that there was some sort of widespread, country-wide connectivity problem in Zambia late today, for which NinerNet is clearly not responsible. The NinerNet mail servers continued to process inbound email during this time, so even if you couldn’t connect to the mail server, incoming email to you was still collecting on the mail server for your eventual collection.

If you have any questions or concerns, please feel free to contact NinerNet support. Thank-you.


Update, 2023-05-19: One of our clients contacted a Zambian ISP and the problem was blamed on Zambia’s international gateway. Sorry for that!

All servers: Company SSL/TLS certificates on all servers renewed and updated

14 January 2023 23:27:28 +0000

The *.niner.net wildcard SSL/TLS certificate has been renewed and updated across our entire infrastructure. This should be seamless for all our clients. However, should you run into a situation in the next 24 hours where you’re told that the certificate has expired, please log out and reload whatever it is you’re trying to do. You may need to reboot, but this would be very unusual. Simply forcing a reload should clear things up. Via FTPS you may need to re-trust our certificate depending on your FTP client.

This doesn’t affect certificates on your own domain on your website, just NinerNet services such as control panels and connecting to the mail and FTP servers.

If you have any questions or concerns, please do contact NinerNet support with any error messages you may be seeing. Thank-you.

NC036: Messages to Outlook-/Microsoft-/Hotmail-hosted domains delayed

6 July 2022 09:12:00 +0000

Please note that we are seeing messages from NC036 destined for private domains hosted by Microsoft delayed at this time. The cause is not clear, but it’s clearly on Microsoft’s end.

This is not in any way related to the internal server issue that happened yesterday.

NC036: Issue resolved

5 July 2022 23:54:13 +0000

The issue now appears to be completely resolved. If you’re having any email issues, please do contact NinerNet support and we will look into them. Thank-you.

NinerNet home page

Systems at a Glance:


Loc.SystemStatusPing
Server NC023, London, United Kingdom (Relay server), INTERNAL.NC023InternalUp?
Server NC028, Vancouver, Canada (Monitoring server), INTERNAL.NC028InternalUp?
Server NC031, New York, United States of America (Web server), INTERNAL.NC031InternalUp?
Server NC033, Toronto, Canada (Primary nameserver), OPERATIONAL.NC033OperationalUp?
Server NC034, Lusaka, Zambia (Phone server), INTERNAL.NC034InternalUp?
Server NC035, Sydney, Australia (Secondary nameserver), OPERATIONAL.NC035OperationalUp?
Server NC036, Amsterdam, Netherlands (Mail server), OPERATIONAL.NC036OperationalUp?
Server NC040, Toronto, Canada (Web server), INTERNAL.NC040InternalUp?
Server NC041, New York, United States of America (Web server), OPERATIONAL.NC041OperationalUp?
Server NC042, Seattle, United States of America (Status website), OPERATIONAL.NC042OperationalUp?

Subscriptions:

RSS icon. RSS

Twitter icon. Twitter

Search:

 

Recent Posts:

Archives:

Categories:

Links

Tags:

.co.zm domains .com.zm domains .zam.co domains back-up bounce messages browser warnings connection issues control panel database dns dos attack dot-zm domains down time email email delivery error messages ftp hardware imap mail mailing lists mail relay mail server microsoft migration nameservers network networking performance php phplist pop reboot shaw shaw communications inc. smtp spam spamassassin ssl ssl certificate tls tls certificate viruses webmail web server

Resources:

On NinerNet: