Server and System Status

Updates will now be sequentially numbered.

We’ve marked server NC036 as “degraded”, because that’s what it is. Email within the group of clients for which we are responsible works. With the outside world, it doesn’t.

Our live monitoring indicates that our uptime is 99.998%, but as we’ve indicated before, this is not 100% accurate. Even AI is “accurately inaccurate”.

You still have access to your account(s) on the server. Please do with that information what you consider to be best for your business.

We implemented the permanent fix yesterday. However, the result seems to be the same: NO EMAIL.

I half expected that this would reveal to me that I had made a stupid mistake, and all this palaver was for nothing. That would have been supremely uncomfortable, but at least we’d all be online. But we’re not.

I know you, our clients, have struggled for the last two days. So have we, there is no doubt. I literally feel your pain.

We have two options at this point, but I want to make clear at the outset that your data — your 5.1 million emails, 2.9 terabytes — are safe. Additionally, I am still alive, which could be a positive or a negative for you, depending on how you view the situation.

Option one, and the option I’d prefer under normal circumstances, is to start fresh with a new mail server, and even just verbalising that fills me with that feeling you get at the beginning of a footrace. You don’t know if you’ll come first, or 101st, but goddam, you’re going to race your heart out.

Option two, to put it succinctly, is to know when to quit. It’s not the end I envisioned when I was supposed to hand over the business and our name in May last year, but nothing lasts forever.

Already I hear most of you leaning towards option two. I don’t blame you. Just get out of the way, Craig, and let me get my email and entrust it to someone with a bigger marketing budget.

If I choose option one, it’s going to take at least a week … and that’s being optimistic. So you will not be able to send or receive email for all of next week, and if everything works, you’ll be online by Monday, 30 March 2026 … I hope.

If I choose option two, that would be like giving the order to “abandon ship”; you don’t want to be in the cold water, but the only other option is even less palatable.

I’ve decided, after much soul searching and personal conversations today, to take option one.

Who knows what NinerNet will look like in a week or a month. However, I know what’s most important for me, and that is that you have your access to the 5.1 million emails you have entrusted to us. You will have access to them, and what you do with them will be up to you.

We will, as always, be communicative, and post updates here. We’re not ignoring your emails — which we ironically continue to receive — but our focus must be and is on keeping everyone as informed as possible.

We continue to receive reports of continued problems from a very few clients.

We essentially addressed the problem on the evening (UTC) of 19 March. We have yet to put in place a permanent fix. The permanent fix is to attach a secondary volume, and move some back-ups onto the secondary volume, to free up space on the primary hard drive to store and process email. We have not done this yet.

However, we have freed up a huge amount of space, so the server is comfortably handling all the email thrown at it. This is the temporary fix.

The reported problems seem to focus on two areas:

• Bounced email, and
• Expected but not received emails.

We have been working on the mail server non-stop for about 21 hour now. We have rebooted the server once (in the morning UTC) and restarted applications a few times. During that time foreign mail servers may have been disconnected in the middle of trying to deliver an email, resulting in their bouncing the email with an error that refers to “losing the connection while receiving the initial server greeting”. Given how often this would have happened (not very much) we’re surprised to see this bounce, but it’s obviously happened. We can only expect these events to lessen as things return to normal.

We ourselves have noticed that emails from external/foreign sources have not been arriving lately. We can only expect that these emails are delayed, and we expect to receive them again in the next 24 hours. Please be patient.

If you check your domain at mxtoolbox.com you will find that it should comply with all or most requirement to have your email delivered. Not complying with every requirement is not necessarily a bad thing.

We expect to have the permanent fix in place within the next 12 hours, but the temporary fix is more than enough to keep the email flowing. Thank-you.