…Stupid mail servers.
You see, while being a reasonably well balanced person, I tend to see through things quickly, particularly advertising, marketing, PR, promotions, special offers and those guys with the throwing games at the fair ground and shit songs, irritating adverts and crap, mindless TV programmes piss me off really badly so much so that I have to turn over the TV or radio to make it all just go away.
As you may have read here and here, I like tinkering with mail servers, there seem to be endless permutations to solving any number of problems. So, I implemented greylisting a few weeks back and it has worked well, I received compliments on how much the spam rate had dropped, in fact to pretty much zero. I estimate that at least 90% of email connections my main mail server receives are spam and we drop about 40% of that with spam blacklists, Spamassassin was pretty ineffective and caused problems under heavy load so I dropped it and replaced it with greylisting. Greylisting refuses every email connection the first time round and tells you to come back in a short while.
A core feature of mail servers is the ability to spool message to retry later. When the retry comes , if it is after a certain period of time, the mail server looks up in the greylisting table whether it has seen mail from you recently and if it has, lets the mail through. Simple huh? And a pretty common tactic. Most spam comes from virus infected Windows machines and the spam software on them (which you won’t notice apart from your Internet connection being slow) isn’t normally intelligent enough to handle retries, they are normally optimised for sending as much spam as possible, regardless of whether it reaches it’s destination.
So, what has pissed me off today is a client complained that they aren’t getting some mail. I look into it and find that in each case the sending mail server can’t handle retries. WHY NOT??? Because, probably, their administrators don’t know what they’re doing. In one case, I guess it’s a policy decision. That’s YOU Blueyonder, spam capital of the UK. But in the other case, it looks like a home hosted mail server run by people that don’t know what they’re doing.
In this case, I had to whitelist (that means automatically allow, for the non-techies) the servers in question to allow my customers to receive the mail.
So, aside from my other rants on the topic of mail administration above, if you run a mail server, make sure you can handle reties within a reasonable period, unless you have a specific technical reason not to. As a hint, make the first retry within 10 to 15 minutes and then another one after an hour.
Oh yes, in completely unrelated news, I have given up smoking again. Completely unrelated, as I say.
On the good front, it amazes me how useful David Watson seems to be. I’m almost certain I’ve never met him, but he seems to keep popping up with solutions to my quandaries. Such as here and here. And he’s a fellow bass player. Keep on being one of the most useful people in world dude :).
Wow, thanks dude. I did meet you briefly outside of LRL06 just before the doors opened, I’ll introduce myself at the next LRL (I’ll just find the person with the loudest suit 🙂 ).
Hello, I have a problem putting blueyonder.co.uk into the whitelist of greylistd. It just does not work. Mails are still being rejected to sender :-(.
What’s more, I sent a test one from my hotmail account, it was being deterred 3 times before it was finally received. This causes long delay of emails. don’t know whether you have the same problem
OK, you need the IP addresses of all of the Blueyonder/NTL/Virgin Media mail servers, don’t know whether they are all the same physical servers or whether they all have the same no retry policy. On Linux or Unix you can do
dig blueyonder.co.uk mx
And you can get the IP addresses for the mail servers from the results. You put each of these on a new line in /var/lib/greylistd/whitelist-hosts. Not sure whether you have to restart greylistd or not afterwards but I do anyway.
I’ve found that greylistd will continue to greylist everything if there is an error in the whitelist-hosts file. Write you’re network ranges, if you’re using any, in 123.234.345.0/24 notation.
One other thing. Most normal people will retry within a few mins and again within 10-15 mins. There’s no problem with retries being greylisted, provided they don’t try twice and then don’t try for another 8 hours.
My configuration retries every 15 mins for 2 hours, then in increasing intervals, starting at 1 hour and increasing by a factor of 1.5 for up to 16 hours, then every 6 hours for up to 4 days since the first failure. I think everybody else should do the same 🙂 My config in Exim 4.x is:
* * F,2h,15m; G,16h,1h,1.5; F,4d,6h
My customers can’t wait longer than around half an hour for a mail to go through and so, I reduced /etc/greylistd/config’s retryMin value to 600, which allows retries through after 10 minutes. This defeats the ‘make them wait an hour and then they’ll be blacklisted if it’s spam’ idea, but the screams from users would be deafening if they had to wait for over an hour every time someone sent them a mail for the first time.
Hope that helps you out.