Why was my email flagged as spam?
Dec 8th
“Why was my email flagged as spam?” This is a very common question, and while it looks like a simple one on the surface, it’s actually not as easy to answer as you might think. Common misconceptions are;
1) If I’ve sent and received email from my friend for years, it shouldn’t get flagged as spam.
2) If I have their email address in my address book, their email won’t get flagged as spam.
3) If I avoid using certain words, my email won’t get flagged as spam.
None of these things are true. To understand why this is a tricky question to answer, it’s helpful to know a bit about what ISP’s are doing to filter spam. Most ISP’s have their own “custom blend” of what they do to filter spam, but it more or less boils down to using a combination of one or more of the following: Blacklists, Greylisting, enforcing RFC’s, and more traditional Content Filters.
Blacklists can be based on all kinds of things. They can be lists of IP addresses that have been reported as sources of spam, lists of mail servers that have been found to be capable of being used as open mail relays, lists of URL’s that have been “spamvertised”, or any number of other things. Not all blacklists are the same. Some are very aggressive in what they list, and some are very conservative. The aggressive lists might block a lot of spam, but they are also more likely to have “false positives” – as in they blocked something that the recipient really did want to receive. Whereas the conservative lists might not have many false positives, but they’re likely to let more spam through.
Greylisting is when a receiving mail server issues a temporary error, which causes the sending mail server to re-queue the email and send it once more. Being able to re-queue an email is something that any RFC compliant mail server ought to be able to do. Greylisting can drastically reduce spam sent through “spam zombies” – home computers compromised by viruses that send spam out directly from the PC instead of through a mail server capable of re-queuing email.
RFC’s are, in a nutshell, the basic minimum standards for anything Internet related. Enforcing RFC compliance for mail can cut down on mail sent out from compromised PC’s/servers, and cut down on spam sent out from “sketchy” mail servers.
And lastly, content filters are the more traditional form of analyzing the content of an email to determine the “spamyness” of the email. Each spam filter system has its own “custom blend” of techniques to identify spam. Some of these criteria include; spammy words/spelling (\/1agra), format of an email (lot’s of CAPITAL/BOLD/etc lettering), lists of “spamvertised” websites, know spammer addresses, etc. Some filters use a feedback system that allows end users to submit examples of spam to train the filter.
Because blacklists and content filters are dynamic in nature, it can be very difficult to determine what it was at that exact moment that caused a particular email to be tagged as spam.
Anti-Spam Part 1, Greylisting
Dec 5th
I’ve occasionally gotten calls from system administrators about a “mail bouncy thing” they notice in their logs when they send mail to us. They find it weird and sometimes frustrating and many consider it a silly anti-spam technique. Well, that would be greylisting, and while it’s weird, it also drops a lot of spam getting through to our customers.
It’s also our first line of defense against spam.
Greylisting is a very simple technique. It is a daemon attached to database that keeps track of who externally sent mail to whom internally, including from what IP address. When a new sender/recipient/IP-address (or triplet as it is called) combination pops up, it bounces the transaction with a temporary, 450/451 response code. This is per the RFC and any properly implemented SMTP server should adhere to it, re-queue the message, and send it again later. If the server sends it before a specified “too early” window (in my case on my personal server, 2 mins, but that’s fairly aggressive) it’s temp-failed (tech term for try again later) again. If the message comes back after this “too early” window, but before a 24 hour expiration window, the message is passed through, and an entry is made in the database allowing that triplet to send mail unhindered for a few days (depending on configuration). If enough messages come from the same ip address and domain pass Greylisting, that whole domain can be automatically white-listed through the check.
The goal of greylisting is not to penalize legitimate mail servers but only to stop non-compliant botnets from getting through.
Greylisting is very effective because it keeps non-compliant SMTP servers from sending mail to our (or even your) servers. Most virus infected computers that send or relay spam won’t re-queue messages, or will re-queue them for only the briefest amount of time. Why? Their goal is to blast as much email/virus payload as possible, and any slowdown or long retry time is very counterintuitive to this goal.
Problems with greylisting are legitimate, by mis-configured SMTP servers either not re-queuing the messages because they are set to treat 400 series bounces as 500 series (permanent) bounces. Or they re-queue the messages, but report to the original sender that the message bounced.
Yahoo implements a more esoteric set up, where they have 4 servers listed in the MX record, and at any time, any of them will bounce messages. This is another way to test for non RFC compliant servers, as a server is supposed to try all of the MX entries in turn, by weight value. Most virus infected computers won’t do that. At least that is what it looks like from the outside.
Because some of our users may have problems with receiving mail, our web-based interface, ipMom, gives you the option to disable greylisting. If you log into ipMom with your email address and password, you’ll notice a “Greylist” option . Set it to off, and greylisting is no longer affecting your mail. Keep in mind that this does let more spam into the system, although our other anti-spam protections may still catch them.
I hope that helps!
So who hosts what in the where now?
Nov 20th
One of the most common points of confusion for our customers domain registration and DNS hosting. DNS isn’t exactly the easiest thing to understand, nor is domain registration, so it’s natural that a lot of people would find the whole thing baffling.
The first thing to understand is the Registrar. A registrar is a company that is accredited, and allowed to work with ICANN ( The Internet Corporation for Names and Numbers). ICANN actually maintains domain names and their information. You pay a registrar for your domain registration, and they, in turn, pay ICANN and provide them with the domain’s information. There are other parties and services involved, but in the interest of keeping it simple, that’s how it works.
Once the registrar has the domain name registered and reserved, actual Domain Name Servers need to be assigned to it. DNS is the service that turns the domain name, say, example.com, into its associated IP address. Humans can remember words far better than numbers. And computers deal with numbers. This is why we have DNS.
It also tells servers where to send email and what IP addresses various services may use. Each service normally has an A (or address) record. For example, an ftp server, ftp.example.com, may be on one IP address while the website, let’s say www.example.com, may be on another. Mail routing is controlled through the MX (mail exchange) record, which must point to a host name, like mail.example.com instead of an IP address. Canonical Name records and TXT record are used for more specialized purposes.
Usually, a registrar defaults to using their in-house nameservers. You can use these servers, putting in the information from your hosting company, usually by using a web based interface. However, we feel it is a better idea to switch the nameserver to your hosting company’s. That way, if your hosting company makes a change then they can update your DNS automatically alleviating your need to worry about these technical details.
DNS can be tricky and it is absolutely critical that your information is correct. One small mistake can cause you to not receive email, or not be able to view your own website. Many outages are associated with DNS problems and can easily be avoided by making sure the right information is in the right place.
Now, there are PTR (or rDNS) records that often are confusing. Most DNS queries are like looking up a name in a phone book directory, but you look up a server name to get the IP address (instead of a phone number). There are a few instances where you want to check the IP address and see who it belongs to, kind of like a reverse directory search of a phone number to see who’s calling. This is called a reverse DNS check.
This most often come into play when mail servers want to check the legitimacy of the SMTP servers that are sending them messages. If they query the IP address and get mail.example.com (or some other, similar Fully Qualified Domain Name) they let it pass. If they get something like 192-168-123-45-adsl-dynamic-customer.isp.net, they may reject or quarantine messages. These records are maintained by the ISP who provides the IP address, so if you run a server out of colocation space or off your Internet connection, you’ll want to contact the ISP to update that information.
Well, this was a very basic summary, hopefully it helps!
“It Worked Yesterday!”
Nov 6th
The three most dreaded words a support technician can hear are “It Worked Yesterday.”
That phrase has a hidden implication trailing after it “…so what did you do?”
Well, usually, we didn’t do anything. People can accept that a car, or a furnace, or a TV can break someday out of the blue. We all know that those are machines, and we have some idea of how they work, and that they do break. The problem is, not a lot of people realize that computers are machines too. Sure, If you ask someone if a computer is a machine, they’ll say yes. However, if you show them the inside of a computer tower, they’ll usually shake their head and say “Is that it?”
Do you want a Kindle?
Oct 29th
ipHouse has been getting out to events more in the past 5 months than we have in the past 5 years. We are having fun meeting customers face-to-face and learning about how you use ipHouse services.
At some of these events we have held a drawing for a free Amazon Kindle. Personally, I have a Kindle and I love it. Do you folks feel the same way? Do you want a chance to win a free Kindle? Should we be giving something else away? Please send us your opinions and ideas.
Our next event is the 55th Annual Minnesota CPA Tax Conference on November 16th and 17th at the Minneapolis Convention Center. This is a well-organized and educational event put on by the Minnesota Society of Certified Public Accountants. We look forward to seeing you there.
