NinerNet Communications™
System Status

Server and System Status

Blackberry email status update

24 June 2010 22:28:32 +0000

Early this week we started receiving reports from Blackberry users with different wireless carriers on different continents, that late last week they started having problems using their Blackberries to access their email on their domains hosted with NinerNet. However, they could access their email accounts through regular means — i.e., POP (with an email program) or webmail.

An initial look at the problem suggested that the two common denominators were NinerNet and Research in Motion (RIM), the latter being the “Blackberry company” through which all email on Blackberry devices is handled. It was (and is) unlikely that individual carriers were the cause of the problem.

After considerable research and consultation over the course of a couple of days, we did not have (and still do not have) any definitive reason for the problem. One reason for the lack of a definitive cause is the complete lack of information from RIM, who seem not to have any useful information about their email processing methods on their website for hosting companies like NinerNet, and who make it very difficult for people to contact them. (One good thing that can be said about a behemoth like AOL is that they maintain a website and a channel for hosting companies like NinerNet to contact them in the event of email delivery issues, as happened last week. That issue was successfully resolved thanks to dialogue with AOL.)

In the spirit of trial and error we tried two different configuration changes to our mail server based on the connection activity we were seeing from RIM servers. In hindsight we should have implemented these separately to see which one was actually responsible for fixing the issue, but the focus was, of course, on fixing the issue, not messing around. That said, a third issue was that, after making the configuration changes, we restarted the mail server. This had the effect of closing down about two hundred connections to our mail server from RIM servers. About fifty connections were quickly re-established, and we expected this number to grow again to about two hundred … but it didn’t. After more than twenty-four hours the number of connections remains steady at about fifty. This is interesting in itself, as simply restarting the server and not making the configuration changes may actually have fixed the problem anyway, making us wonder why RIM had so many connections open to our server in the first place.

So for now the issue appears to be resolved. We’re continuing to monitor connections to our mail server by RIM servers, and we ask that, if you have any further problems with accessing your NinerNet-hosted email using your Blackberry, you contact us with the details as soon as possible. Thanks very much.

Progress report on Blackberry email issue

23 June 2010 12:13:47 +0000

We’ve made a couple of configuration changes to the mail server and restarted it. By all accounts this appears to have fixed the problem, but we’re not convinced that this is the end of the issue. We’ll post another update here within the next 48 hours, and we’ll also be sending an email to all clients about this when that update is posted.

Thanks for your patience. As mentioned previously, we’re still looking for more information from Blackberry users about this, so if there is any information you can provide to us that you think might be useful, we’re all ears.

Email issue for Blackberry users

22 June 2010 03:46:31 +0000

We’ve had reports from Blackberry users with different networks on different continents that they are unable to access email on their domains hosted with NinerNet via their Blackberry devices. We’re still looking into this issue to determine if the problem is on our end or with the Blackberry / Research in Motion (RIM, the Blackberry company) systems. Based on our understanding of the relationship between Blackberry/RIM and individual network carriers, this is probably not an issue with the carriers themselves. However, we don’t know whether or not individual subscribers are able to contact Blackberry/RIM directly for support.

In the meantime, we’ve been informed by one client that they were able to regain access to their email account through their Blackberry by changing their configuration to use IMAP on port 143 and a non-SSL connection. If this issue is affecting you, please try this.

We are interested in feedback and diagnostic information on this issue from clients. If you are able to send us copies of error messages or information directly from Blackberry/RIM (the company) support or your carrier, we would appreciate it. Thanks.

We’ll post updates here as we work on this issue.

AOL email update

22 June 2010 03:32:41 +0000

We’ve had the following response from AOL to our query about incoming email from AOL addresses being bounced back to the senders:

We had few issues with our Outbound IP’s timing out with few mail servers last week. Are you still seeing reports of our users not able to connect to your servers? Please send me the headers and I can check the logs at our end. With regards to Spam Flag Yes- we are seeing lots of AOL accounts being compromised forcing us to send mails through our aberrant IP’s. These mails might be rejected at our end too if we suspect compromise but do issue a bounce message.

This would seem to indicate that AOL has solved the issue of legitimate email sent to NinerNet clients being returned as undeliverable. We’ve had no reports of incoming email from AOL senders being bounced for over a week now, so we’ll consider this matter closed.

Server NC020 rebooted

20 June 2010 15:10:59 +0000

Server NC020 was rebooted at 14:34 UTC after it was reconfigured with more RAM.

AOL and Realtime email updates

17 June 2010 06:06:50 +0000

While we have not heard from AOL yet, in recent experiments all email from AOL got through. However, these experiments were on a very small scale, so may not accurately reflect reality.

Domains hosted by Realtime appear to be back up again after being down for about five days. That email backlog has been cleared.

Email to the domain appears to be having problems. There isn’t much of this backlogged on the mail servers, but there is some. Same with a couple of domains hosted by Coppernet.

If you’re trying to communicate with people with unreliable email, why not refer them to NinerNet? We pay referral fees!

Issues with incoming email from AOL

14 June 2010 07:43:43 +0000

We have had some reports of incoming email from AOL users being bounced back to the senders. After some extensive research over the last few days we believe we have managed to address this problem, adjusting the configuration of one of our anti-spam measures (greylisting) to allow for the fact that AOL’s handling of outbound email doesn’t play well with greylisting.

If you are a NinerNet client and have any reports from correspondents with AOL email addresses indicating that email from them to you has been returned as undeliverable, please contact support.

Update 2010-06-15 07:28 UTC: Apparently this issue has not yet been resolved. We contacted AOL yesterday to find out why some of their mail servers are bouncing email prematurely and without reason, but their auto-response says it could take seven days or more for them to respond, it at all. We will post updates as they become available.

Zambian email delays

14 June 2010 07:36:10 +0000

Clients sending email to domains hosted by Realtime are advised that messages to these domains have been backing up on our mail servers over the weekend, and are still not being delivered. A cursory analysis would seem to indicate that there is a routing problem at or near the Realtime mail servers. If you have concerns about this, please contact the intended recipients via a medium other than email and have them contact Realtime for an explanation.

Update 2010-06-15 07:26 UTC: This issue seems to be ongoing. Domains hosted by Realtime appear to be down still, and email to them continues to be queued on our mail servers.

NinerNet home page

Systems at a Glance:

Server NC023, London, United Kingdom (Relay server), INTERNAL.NC023InternalUp?
Server NC028, Vancouver, Canada (Monitoring server), INTERNAL.NC028InternalUp?
Server NC031, New York, United States of America (Web server), INTERNAL.NC031InternalUp?
Server NC033, Toronto, Canada (Primary nameserver), OPERATIONAL.NC033OperationalUp?
Server NC034, Lusaka, Zambia (Phone server), INTERNAL.NC034InternalUp?
Server NC035, Sydney, Australia (Secondary nameserver), OPERATIONAL.NC035OperationalUp?
Server NC036, Amsterdam, Netherlands (Mail server), OPERATIONAL.NC036OperationalUp?
Server NC040, Toronto, Canada (Web server), INTERNAL.NC040InternalUp?
Server NC041, New York, United States of America (Web server), OPERATIONAL.NC041OperationalUp?
Server NC042, Seattle, United States of America (Status website), OPERATIONAL.NC042OperationalUp?


RSS icon. RSS

Twitter icon. Twitter



Recent Posts:




Tags: domains domains domains back-up bounce messages browser warnings connection issues control panel database dns dos attack dot-zm domains down time email email delivery error messages ftp hardware imap mail mailing lists mail relay mail server microsoft migration nameservers network networking performance php phplist pop reboot shaw shaw communications inc. smtp spam spamassassin ssl ssl certificate tls tls certificate viruses webmail web server


On NinerNet: