Server and System Status

As we approach a week with this issue, we are forced to make a decision. We continue to evaluate a number of options, and we’ll implement more than one, but the bottom line for us is a usable email system for our clients, or at least the vast majority of you.

In order to allow the vast majority of our clients to carry on business as usual, at 04:15 UTC today, 12 August 2020, we removed the attachment restriction we put in place late last week. According to reports, this will result in difficulty for one, or possibly two clients who have been overwhelmed recently by spam email that contained these attachments. On the other hand, we have heard those clients that have told us that their businesses have been stalled by these restrictions.

However, that isn’t the last or only action we have taken or will take on this matter. These are others:

• The lack of speed with which our anti-virus vendor (ClamAV) has picked up this virus means we will be looking at options for either internal or external secondary virus scanning of our incoming and outgoing email streams. If this was something we already had in place, this disruption would have been hardly noticed by more than a few clients. This is the priority with which we are currently seized.
• We continue to look into ways to apply certain restrictions to some domains and not others in a virtual-hosting environment.
• Virus samples have been submitted to ClamAV and we anticipate that they will add the recently received viruses to their virus-detection database in due course.
• We will make an offer to the client most affected by this virus outbreak to move their hosting to a virtual private server of their own.
• We have blocked several thousand IP addresses that have been the sources of the problem attachments over the last week, and we will block more.
• Server NC036 is approaching its scheduled replacement time frame. Had our plans been further along when this happened we’d have executed them immediately. However, there are always potential issues when implementing half-baked plans too far ahead of schedule, so we didn’t.

There are potential issues that you need to be aware of:

• Some anti-virus scanners, not the least of which is ClamAV, are not detecting some of the recent viruses we have seen. You must ensure that you have anti-virus software installed on all of your machines and devices, and you must ensure that this software is automatically updated at least daily. Please remember that the responsibility for the safety of your computer(s) and your data is ultimately yours.
• You and your employees need to be aware of the risks of opening attachments, and need to be aware of how to evaluate that risk. The risks are to both your machines and devices, and to your organisation and employees.

What have we learned from this experience? Other than our surprise at the degree to which some businesses rely significantly on Microsoft Word documents flying around the Internet, we have learned that the anti-virus vendor on which we have relied without issue for about a decade is not, in fact, infallible. We need, and will obtain, redundancy in this area.

Something else we have decided to act on is SPF (Sender Policy Framework) records. All domains we host have SPF records that tell all mail servers on the Internet that they should accept email only from our servers. The records have all ended with “~all”, but this weekend we will update all records to read “-all”. The difference is that the old records with the tilde (~) allowed receiving servers to act with some leeway, a so-called softfail; the hyphen (-) will enforce that all mail received from domains we host must come from our servers, or any others that are designated in the domain’s SPF record. What will this accomplish? One of the things we have seen is the domain of one of our clients being extensively and aggressively “spoofed”. This is when emails are sent purporting to come from a domain other than their real origin. SPF is designed to prevent this, but the directive with the tilde allows leeway that, it seems, can be too easily abused.

If you believe this may be an issue for you, or if you have any questions at all, please contact NinerNet support and we will assist.

Thank-you very much for your patience.

After three updates to the virus database on NC036 since Thursday we expected that the anti-virus scanner would detect and block the trojan that is currently overwhelming some of our clients. However, that is not the case.

For that reason we are once again blocking the attachments that are the cause of this problem. We sincerely apologise for this situation.

Again though, if you need to send a blocked attachment type, you can still do so if you compress the file into a .zip document. Your correspondents can also do the same.

If you are getting an error message when trying to send email, the first thing you need to do is check to see if you have attached a file to your message. If you have, and it is one of Microsoft’s typical Word documents, please remove it, compress it, attach the compressed .zip file, and then send that attachment.

If you have any questions or concerns, please contact NinerNet support. Thank-you.