Milter-greylist

Emmanuel Dreyfus wrote on Tue, 4 Apr 2006 14:50:27 +0000:

> 1) The context 
> greylisting have been a good protection against spam, but spamware able 
> to defeat it by resending spam are now available. We need a way of adressing 
> the new problem 

yes, that was to be expected.

>  
> 2) The idea 
> The sender machines does not stay idle between the two resends. It tries to 
> send spam to other locations. If we have spam honeypots (aka spam traps) 
> everywhere on the Internet, we have good chances that the sender machine 
> will send a spam to a honeypot before the resend. If we have a Distributed 
> Spam Trap (DST) netwowrk, one site can catch a sender machine and inform 
> all the sites participating to the DSTnet that it found a spammer.

Isn't that very similar to existing blacklists, either RBLs for sending hosts, 
URIBLs for advertised URLs in emails or message hash comparing systems like 
Pyzor and Razor (which also take automated submissions from other systems after 
these identified messages as spam)? So, there's already quite a few distributed 
systems out there. I'm not sure if we need another although a bit different 
one.
Also, spamtraps have a high potential of false positives. As you said spammers 
might send over a legitimate ISP and that mailer then gets blacklisted simply 
because spam by a customer was sent thru it. It's almost impossible to stop 
this unless you are actively scanning all outgoing messages against spam. It's 
by far the no. 1 reason for false positives in the existing RBLs. F.i. I use 
SORBS-SAFE instead of SORBS because SORBS contains SORBS-SPAM which is a 
spamtrap filled list. The FP-rate isn't high, but even 0,1% is too much if you 
block at MTA level based on this. Spam-trap lists are only usable if you 
maintain huge whitelists at the same time.

I'd rather build on top of the existing greylist method and try to expand on 
it. There are two possible ways I can think of: distribute it globally or 
combine it with other local feedback methods.

-- distribute it
- use milter-greylist as the base of your Distributed Network. I only have a 
vague idea how to do that. Maybe one could distribute all hosts that do not 
retry after one hour to the network as "probably spammer". This could be 
weighted with scores. So, after one hour the weight could start with 1 and 
increase hour by hour or so and make the host more "spammy" over time. The 
other machines in the network could then start greylisting this host with a far 
higher retry period than usual and also exchange results again.

-- local integration
- check if the user exists and if it doesn't exist set the greylisting time for 
this host much higher
- allow feedback from spamassassin and other spam-content-checkers and if a 
message was spam wipe the host from the whitelist and set the greylisting time 
for this host much higher

-- maybe distribute these findings to the network as well?

In general I think it is a good idea to try combining methods and integrating 
feedback. Fighting all spam with one method is bound to fail. Greylisting works 
marvelously at the moment, but it can't catch all spam and other methods won't 
do this single-handedly as well.
F.i. I use (in this order):

-milter-greylist
-sendmail with access.db and three good RBLs
-MailScanner which tests for phishing, bad content and spam checks with the 
help of Spamassassin. Spamassassin is filled with good rulesets from SARE in 
addition to the packaged ones. Only that makes it really effective.

In the end almost nothing spammy or nasty gets thru. I used to kill 50-70% at 
the sendmail level with RBLs and access.db before I used greylisting. Now 
greylisting accounts for somethign like 70-80%, sendmail for another 15-20% and 
only 5% or less is left for MailScanner/Spamassassin. On my main machine my 
Spamassassin wouldn't barely get a spam if I did't get some mail from a 
forwarder which is of course whitelisted. I already fear that SA gets so few to 
do that the bayes database cannot be trained.
I could just go with Spamassassin and don't use any other method and it would 
catch all spam that is now caught before it as well, so the end result of what 
I get is the same. But putting sendmail and greylisting before it saves me 
ample ressources.


Kai

-- 
Kai Sch\ufffdtzl, Berlin, Germany
Get your web at Conactive Internet Services: http://www.conactive.com