The database on server NC031 was restarted at 21:25 UTC and is online and working again. We are investigating the cause of this issue.
The database on server NC031 was restarted at 21:25 UTC and is online and working again. We are investigating the cause of this issue.
We are investigating a database issue on server NC031 that is affecting database-driven websites. We will report details here.
The move of NinerNet‘s primary nameserver to a new machine (NC033) has been completed, and our testing over the weekend shows that the new DNS information has propagated to the rest of the Internet.
The new nameserver has its own control panel, but we have not set up all client accounts on it yet. When that task is complete we will be sending you an email with your log-in information, and instructions for using the new control panel. If you have a need to manage your domain’s DNS and have not received that email yet, please contact support and we will assist you.
Thank-you for your patience during our recent migrations.
No sooner had I clicked “publish” on that last post than the server was put back online. Again, we apologise for the service interruption.
Somebody at our data centre jumped the gun, and has taken server NC018 offline two days earlier than it should have been taken offline. This is (or will shortly be) causing a problem for all services on most domains, including our own.
We are waiting for the data centre to reactivate the server, and will update this ticket as it’s possible to do so.
We very sincerely and humbly apologise for this situation.
We will be shutting down server NC018 within the next 24-72 hours. Normally the closure of a server would not be an event we’d dwell on, but server NC018 has been around for a long time and the migrations off of it have been dragged out for longer than they should have been.
We want to assure all hosting clients that the email that used to be hosted on this server has long since been moved to server NC027, and all websites that used to be hosted on NC018 have been moved to servers NC020 and NC031. No email or websites remain on server NC018.
What does remain on this server at the moment is the DNS (domain name system) service. A new server (NC033) has been commissioned and configured to take over this service, and the DNS information for all domains has been transferred to the new server.
All that said, completely removing NC018 from the NinerNet ecosystem at the flick of a switch will probably reveal one function or another still connected to and relying on a server that has been around for as long as it has. While this is possible, any such function will be of minor significance — it won’t be anything vital to the operation of your email or website — and we’re doing this at the start of a weekend so that we can quickly address any issues that may arise outside of your business hours. We will, of course, post any necessary updates here on our status page (status.niner.net), to which you can subscribe.
One long-standing item that we will address after NC018 has been removed is the matter of control panels. The reason we have multiple control panels will be the subject of a future blog post, but there is a reason we don’t have one grand, unifying control panel … besides the fact that we’ve been there and done that and don’t want to do it again. We have three main servers (with control panels) providing our three main hosting services:
Once server NC018 has been shut down we’ll redirect cp.niner.net to a page that explains the above. (You’re reading it now.) In the future we’ll implement a more elegant way of unifying (to a certain extent) the way you access the three control panels.
We thank you for your patience as we work very hard to improve our infrastructure to serve you better. As always, if you have any questions or concerns, please don’t hesitate to contact us.
Craig
We have cleaned up the spam on server NC027 and managed to funnel most of the delayed email through our relay server. Although the primary mail server’s IP address is not in a couple of the major blacklists, we are still in some niche ones with major ISPs and mail providers. We have been working to have the IP address de-listed where that is possible, but manual processes like these can take a couple of days.
In the meantime, at the moment most email is flowing normally, but email to most domains will still be delayed for the next few hours.
We have cleaned up server NC027 after a client’s email account was compromised, resulting in thousands of spams being sent from the mail server. This is having a negative impact on email being sent from the server, as its IP address is now in anti-spam blacklists. This means that outgoing email sent from our clients will be bounced or delayed until the blacklist realise that the spam from our mail server has stopped.
We are working on implementing a workaround through one of our other, clean mail servers, but unfortunately it will be several hours before outgoing email to most domains is delivered normally.
These incidents usually arise after a computer has been infected with a virus. The virus then sends the email password back to the person or organisation controlling the virus, and they then use that information to compromise that email account on the mail server, using it to send thousands of spams from the account. Please ensure that you install, use and update an anti-virus program on your computer to ensure that this doesn’t happen to your email account.
We apologise for this incident. Please contact us if you have any questions. Thank-you.
Work to increase the RAM on server NC031 has completed successfully.
If you have any questions or concerns, please let us know. Thank-you for your patience.
Work on the server upgrade started at 02:30.
Systems at a Glance:
Loc. | System | Status | Ping |
---|---|---|---|
![]() | NC023 | Internal | Up? |
![]() | NC028 | Internal | Up? |
![]() | NC031 | Internal | Up? |
![]() | NC033 | Operational | Up? |
![]() | NC034 | Internal | Up? |
![]() | NC035 | Operational | Up? |
![]() | NC036 | Operational | Up? |
![]() | NC040 | Internal | Up? |
![]() | NC041 | Operational | Up? |
![]() | NC042 | Operational | Up? |
Subscriptions:
Search:
Recent Posts:
Archives:
Categories:
Links
Tags:
Resources:
On NinerNet: