Server and System Status

NC036: Messages to Outlook-/Microsoft-/Hotmail-hosted domains delayed

6 July 2022 09:12:00 +0000

Please note that we are seeing messages from NC036 destined for private domains hosted by Microsoft delayed at this time. The cause is not clear, but it’s clearly on Microsoft’s end.

This is not in any way related to the internal server issue that happened yesterday.

Tags: email, email delivery, mail, mail server, microsoft
Categories: Email, NC036
Comments Off

NC036: Issue resolved

5 July 2022 23:54:13 +0000

The issue now appears to be completely resolved. If you’re having any email issues, please do contact NinerNet support and we will look into them. Thank-you.

Tags: connection issues, email, imap, mail, mail server, performance, smtp, webmail
Categories: Email, Incidents, NC036
Comments Off

NC036: Server is up, but some issues

5 July 2022 23:09:28 +0000

We have been informed by the data centre where server NC036 is located that, due to “an effect from ongoing data centre maintenance”, there are connectivity issues. We have been noticing intermittent issues with both direct email connections and the webmail, notably slower transactions (e.g., deleting messages, scanning multiple folders, etc.). However, email is still flowing as usual, perhaps with minor delays.

We have opened a ticket with the data centre to determine why the issues are ongoing. We will update you as we receive more information.

Tags: connection issues, email, imap, mail, mail server, performance, smtp, webmail
Categories: Email, Incidents, NC036
Comments Off

NC036: Disk space issue resolved

8 May 2022 06:19:09 +0000

Briefly between Saturday and Sunday the disk storing emails on server NC036 became full. This was resolved immediately upon discovery, but some incoming emails were delayed. Some may also have bounced. However, this issue is now fully resolved.

If you have any questions, please contact NinerNet support.

Tags: bounce messages, email, email delivery, mail, mail server
Categories: Email, Incidents, NC036
Comments Off

NC036: Maintenance port-mortem

16 October 2021 10:41:00 +0000

As stated previously, the maintenance on server NC036 is complete, and has been successful.

That said, it had to be carried out earlier than scheduled because disk space on the server was the issue that was causing a problem. To be clear, the disk space used by email accounts was not and is not the issue (that’s on a separate disk), it was the disk space used by the operating system and the logs in particular. When this problem was noticed earlier this week we projected that addressing the issue during our regular weekend maintenance window on Saturday evening UTC would be sufficient. Well, we were wrong. As soon as it became clear that our temporary mitigation efforts were failing to keep up with the rate at which disk space was being consumed (mostly by the aforementioned logs), we immediately implemented the maintenance that we had planned and already practised on a test server. This practice paid off, as instead of allowing an hour for the maintenance to complete, we were done in four minutes, including a server reboot.

That said, the lack of disk space on the server did cause disruption in the rate at which mail was processed, that led to some of you getting errors when sending email, and some incoming email being delayed. For this we apologise. The backed-up mail queue was fully processed by 09:18 UTC.

Had our prediction held up and the maintenance been carried out this evening as scheduled, we wouldn’t even have needed to post this detail. However, in order to clear up any concern, we have.

Early in 2022 server NC036 is due to be considered for replacement. The experience of this issue and other issues we have run into in recent months will inform our decisions about where to locate the replacement server (hint: it won’t be in the current data centre) and various configuration improvements we can make. We’ll also, of course, be using a more current version of the control panel.

If you have any questions at all about today’s maintenance, please let us know by contacting NinerNet support. Thank-you for your patience, and continued patronage.

Tags: down time, email, mail, mail server, reboot, webmail
Categories: Email, Incidents, Maintenance, NC036
Comments Off

NC036: Maintenance complete

16 October 2021 09:05:46 +0000

The maintenance on server NC036 (the primary mail server) is complete and the server is back online. It was unavailable between 08:28 and 08:32 UTC. According to some reports there have been issues with delayed emails and “lookup failure” errors, but these have now cleared.

We’ll be posting some further information shortly with more details.

Tags: down time, email, mail, mail server, reboot, webmail
Categories: Email, Maintenance, NC036
Comments Off

NC036: Weekend maintenance moved up, starting in the next few minutes

16 October 2021 08:23:31 +0000

The situation on server NC036 needs to be addressed immediately. We will be doing this maintenance in the next few minutes. We will post updates here.

Tags: down time, email, mail, mail server, reboot, webmail
Categories: Email, Maintenance, NC036
Comments Off

NC036: Mail server disk space issue

15 October 2021 11:34:08 +0000

Server NC036 (the primary mail server) briefly ran out of disk space on Friday 15 October at 11:02 UTC. This was immediately rectified, and the server was running normally again at 11:10.

This is the reason for this weekend’s scheduled maintenance. This problem won’t happen again after the maintenance. We apologise for this temporary problem.

If you have any questions, please contact NinerNet support. Thank-you.

Tags: email, email delivery, error messages, mail, mail server, webmail
Categories: Email, Incidents, NC036
Comments Off

NC036: Maintenance this weekend

13 October 2021 11:30:19 +0000

Server NC036 (the primary mail server) will be down briefly this weekend during the weekly maintenance window while we again increase the disk space available on the server. The downtime should be less than an hour during the least busy part of the weekend on Saturday evening UTC.

During the downtime you will not be able to send or check email, and incoming email will be held on the sending servers until server NC036 is back online. Once the server is back online mail will be delivered in line with standard retry rates. This will mean that some incoming mail may be delayed briefly, but no mail will be lost.

We will post updates here shortly before the maintenance begins, and again once it has concluded.

If you have any questions or concerns, please contact NinerNet support. Thank-you.

Tags: down time, email delivery, mail, mail server, reboot, webmail
Categories: Email, Maintenance, NC036
Comments Off

NC036: Emergency maintenance explanation

2 September 2021 09:23:31 +0000

The emergency maintenance on server NC036 earlier today was to add additional disk space for email storage. Normally this maintenance is scheduled in advance, but today a number of factors combined to require us to take action immediately. We will work to avoid this situation in the future, but the cause of the situation was one of the aforementioned factors.

Thank-you for your patience. If you have any questions or concerns, please contact NinerNet support. Thank-you.

Tags: down time, email, email delivery, imap, mail, mail server, pop, smtp
Categories: Email, Maintenance, NC036
Comments Off

System	Status	Ping
NC023	Internal	Up?
NC028	Internal	Up?
NC031	Internal	Up?
NC033	Operational	Up?
NC034	Internal	Up?
NC035	Operational	Up?
NC036	Operational	Up?
NC040	Internal	Up?
NC041	Operational	Up?
NC042	Operational	Up?