Yahoo Groups archive

Milter-greylist

Index last updated: 2026-04-28 23:32 UTC

Thread

Can milter-greylist run after sendmail checks users?

Can milter-greylist run after sendmail checks users?

2011-02-17 by Les Mikesell

I am running sendmail/MimeDefang and recently added milter-greylist as 
an additional spam check. The domains in question are regularly hit with 
dictionary-attack type spam where most of the target users don't exist. 
  Sendmail is very quick at rejecting local addresses that are not in 
the aliases or virtuser tables so that is normally not a problem, but 
when milter-greylist is active it wants to greylist even the 
undeliverable addresses.  For the moment I am working around it by 
tracking the 'real' users, including them in the milter-greylist config, 
and restricting greylisting to the specified addresses.  However, it 
would be nicer if this could be handled automatically by letting 
sendmail reject addresses it can't deliver first.  Is there any way to 
do that?

Also, the extra log line about 'skipping greylist because this is the 
default action' for the unprocessed addresses is filling my disks up to 
the point that I had to change the log rotation.  Is there any way to 
turn that off?  When it doesn't do anything, I don't need to know about it.

-- 
   Les Mikesell
    lesmikesell@...

Re: [milter-greylist] Can milter-greylist run after sendmail checks users?

2011-02-18 by manu@netbsd.org

Les Mikesell <lesmikesell@...> wrote:

>   Sendmail is very quick at rejecting local addresses that are not in
> the aliases or virtuser tables so that is normally not a problem, but
> when milter-greylist is active it wants to greylist even the 
> undeliverable addresses.

milter-greylist runs before sendmail checks the address is valid. I do
not think you can change that.

> For the moment I am working around it by 
> tracking the 'real' users, including them in the milter-greylist config,
> and restricting greylisting to the specified addresses.  However, it 
> would be nicer if this could be handled automatically by letting 
> sendmail reject addresses it can't deliver first.  Is there any way to
> do that?

I solve this problem by having milter-greylist performing LDAP queries
for the recipient address. If there is no match, the mail is rejected.

> Also, the extra log line about 'skipping greylist because this is the
> default action' for the unprocessed addresses is filling my disks up to
> the point that I had to change the log rotation.  Is there any way to
> turn that off?  When it doesn't do anything, I don't need to know about it.

Patch the sources?

-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu@...

RE: [milter-greylist] Can milter-greylist run after sendmail checks users?

2011-02-18 by attila.bruncsak@itu.int

> I am running sendmail/MimeDefang and recently added milter-greylist as
> an additional spam check. The domains in question are regularly hit
with
> dictionary-attack type spam where most of the target users don't
exist.
>   Sendmail is very quick at rejecting local addresses that are not in
> the aliases or virtuser tables so that is normally not a problem, but
> when milter-greylist is active it wants to greylist even the
> undeliverable addresses.  For the moment I am working around it by
> tracking the 'real' users, including them in the milter-greylist
config,
> and restricting greylisting to the specified addresses.  However, it
> would be nicer if this could be handled automatically by letting
> sendmail reject addresses it can't deliver first.  Is there any way to
> do that?
 
I am using the FEATURE(`access_db') in the
sendmail configuration file.
All the e-mails are rejected per default in the access
file for my inbound e-mail domains and only the known
addresses are listed explicitly as allowed to relay to.
This configuration does the job. No entries in the
greylist.db file for e-mail addresses which are not
listed in the access file.

Re: [milter-greylist] Can milter-greylist run after sendmail checks users?

2011-02-18 by Johann Klasek

On Fri, Feb 18, 2011 at 02:44:04AM +0100, manu@... wrote:
> Les Mikesell <lesmikesell@...> wrote:
> 
> >   Sendmail is very quick at rejecting local addresses that are not in
> > the aliases or virtuser tables so that is normally not a problem, but
> > when milter-greylist is active it wants to greylist even the 
> > undeliverable addresses.

What does this look like in maillog?

> 
> milter-greylist runs before sendmail checks the address is valid. I do
> not think you can change that.

As far as I understand sendmail internal processing including milter
callbacks this is not true at least in my sendmail environment. 

In general: in state envelope recipient processing sendmail runs already
rule 0 on this address to see if this address is valid. This result is
later used to route the mail to the recpient (don't know actually
whether rule 0 is called again or the result is recalled from the call
before). If an address is not routeable (invalid) sendmail responds
immediatly with an error. If this check shows that this address could be
valid the recipient address is passed to all active milters call back
hooks (probably in order the milters are defined in sendmail.cf).

I have feature access_db and delayed checks enabled on some servers, on
some not. In both groups I have noticed the same behaviour.

A case from my logs:

Feb 18 00:00:01 mri1 sendmail[3994]: p1HMxsQB003994: <joachim.fabinir@MYDOMAIN>... Generic address unknown
Feb 18 00:00:02 mri1 milter-greylist: p1HMxsQB003994: addr [151.59.218.228][151.59.218.228] from <infod@...> to <joachim.fabini@MYDOMAIN> delayed for 00:03:00 (ACL 566)
Feb 18 00:00:02 mri1 sendmail[3994]: p1HMxsQB003994: Milter: to=<joachim.fabini@MYDOMAIN>, reject=451 4.7.1 Greylisting in action, please come back later
Feb 18 00:00:03 mri1 sendmail[3994]: p1HMxsQB003994: <jso1209jens@MYDOMAIN>... Generic address unknown
Feb 18 00:00:03 mri1 miltrassassin[4445]: p1HMxsQB003994: Message aborted (abort by milter)
Feb 18 00:00:03 mri1 sendmail[3994]: p1HMxsQB003994: lost input channel from [151.59.218.228] to MTA after data
Feb 18 00:00:03 mri1 sendmail[3994]: p1HMxsQB003994: from=<infod@...>, size=1016, class=0, nrcpts=0, bodytype=7BIT, proto=ESMTP, daemon=MTA, relay=[151.59.218.228]


A mail for 3 recipients @MYDOMAIN arrives. One of this recipients is an
existing address (checked via an equivalent mechanism to virtuser
table). The others are invalid. Miltergreylist adds only the entry for
joachim.fabini@MYDOMAIN (the valid address), the other address are
rejected. The sending source gives up and terminates the session because
no recipient can be reached. Another milter (miltrassassin) which is
still active recognizes the termination of the session (which is logged)
- miltergreylist is already done at this state and keeps quiet (or does
not log it anyway).


> > For the moment I am working around it by 
> > tracking the 'real' users, including them in the milter-greylist config,
> > and restricting greylisting to the specified addresses.  However, it 
> > would be nicer if this could be handled automatically by letting 
> > sendmail reject addresses it can't deliver first.  Is there any way to
> > do that?
> 
> I solve this problem by having milter-greylist performing LDAP queries
> for the recipient address. If there is no match, the mail is rejected.

Any other milter (look ahead milters) which determines a invalid address
should able to terminate a recipient. Maybe it the order of milters is
substantial.

> 
> > Also, the extra log line about 'skipping greylist because this is the
> > default action' for the unprocessed addresses is filling my disks up to
> > the point that I had to change the log rotation.  Is there any way to
> > turn that off?  When it doesn't do anything, I don't need to know about it.
> 
> Patch the sources?

Or ...
Insert a filter statement/script for logrotate (removing the all
lines "skipping greylist because this is the default action").
Alternatively (in case you have syslog-ng) you may set up a filter rule
removing this kind of messages on entrance ;)


Johann Klasek

Re: [milter-greylist] Can milter-greylist run after sendmail checks users?

2011-02-18 by Les Mikesell

On 2/18/2011 6:36 AM, Johann Klasek wrote:
>
>  >
>  > > Sendmail is very quick at rejecting local addresses that are not in
>  > > the aliases or virtuser tables so that is normally not a problem, but
>  > > when milter-greylist is active it wants to greylist even the
>  > > undeliverable addresses.
>
> What does this look like in maillog?

If I have 'acl greylist default ...." set, I get entries like:

Feb 18 09:57:43 mailmx milter-greylist: p1IFvfPf026150: addr 
[118.96.199.46][118.96.199.46] from <rosfoewi00leobla@...> to 
<pikepikephiru@...> delayed for :02:00 (ACL 119)
Feb 18 09:57:43 mailmx sendmail[26150]: p1IFvfPf026150: Milter: 
to=<pikepikephiru@...>, reject=451 4.7.1 Greylisting in action, 
please come back later

for every received message, even though the users don't exist. I'd 
rather not do the extra processing.

>  > milter-greylist runs before sendmail checks the address is valid. I do
>  > not think you can change that.
>
> As far as I understand sendmail internal processing including milter
> callbacks this is not true at least in my sendmail environment.
>
> In general: in state envelope recipient processing sendmail runs already
> rule 0 on this address to see if this address is valid. This result is
> later used to route the mail to the recpient (don't know actually
> whether rule 0 is called again or the result is recalled from the call
> before). If an address is not routeable (invalid) sendmail responds
> immediatly with an error. If this check shows that this address could be
> valid the recipient address is passed to all active milters call back
> hooks (probably in order the milters are defined in sendmail.cf).

The target domains are considered local in sendmail.  I need the lookups 
in virtusertable and aliases.

> A mail for 3 recipients @MYDOMAIN arrives. One of this recipients is an
> existing address (checked via an equivalent mechanism to virtuser
> table). The others are invalid. Miltergreylist adds only the entry for
> joachim.fabini@MYDOMAIN (the valid address), the other address are
> rejected.

What about aliases?

>  > > For the moment I am working around it by
>  > > tracking the 'real' users, including them in the milter-greylist
> config,
>  > > and restricting greylisting to the specified addresses. However, it
>  > > would be nicer if this could be handled automatically by letting
>  > > sendmail reject addresses it can't deliver first. Is there any way to
>  > > do that?
>  >
>  > I solve this problem by having milter-greylist performing LDAP queries
>  > for the recipient address. If there is no match, the mail is rejected.
>
> Any other milter (look ahead milters) which determines a invalid address
> should able to terminate a recipient. Maybe it the order of milters is
> substantial.

Sendmail seems to be very fast at checking virtusertable and aliases. 
I'd rather not have to duplicate that elsewhere.   Or to have to 
maintain the same set of acceptable addresses separately in the greylist 
config file.  It doesn't seem to matter whether MimeDefang or 
milter-greylist is first in sendmail.mc - it still wants to process all 
local-looking targets.  Are there any options as to when it runs?

>  > > Also, the extra log line about 'skipping greylist because this is the
>  > > default action' for the unprocessed addresses is filling my disks up to
>  > > the point that I had to change the log rotation. Is there any way to
>  > > turn that off? When it doesn't do anything, I don't need to know
> about it.
>  >
>  > Patch the sources?
>
> Or ...
> Insert a filter statement/script for logrotate (removing the all
> lines "skipping greylist because this is the default action").
> Alternatively (in case you have syslog-ng) you may set up a filter rule
> removing this kind of messages on entrance ;)

For the moment I just reduced the number of old files that logrotate 
keeps around, but maybe when Centos 6 is out I'll get a better syslog 
version.

-- 
   Les Mikesell
    lesmikesell@...

Re: [milter-greylist] Can milter-greylist run after sendmail checks users?

2011-02-22 by Johann Klasek

On Fri, Feb 18, 2011 at 11:06:47AM -0600, Les Mikesell wrote:
> On 2/18/2011 6:36 AM, Johann Klasek wrote:
> >
> >  >
> >  > > Sendmail is very quick at rejecting local addresses that are not in
> >  > > the aliases or virtuser tables so that is normally not a problem, but
> >  > > when milter-greylist is active it wants to greylist even the
> >  > > undeliverable addresses.
> >
> > What does this look like in maillog?
> 
> If I have 'acl greylist default ...." set, I get entries like:
> 
> Feb 18 09:57:43 mailmx milter-greylist: p1IFvfPf026150: addr 
> [118.96.199.46][118.96.199.46] from <rosfoewi00leobla@...> to 
> <pikepikephiru@...> delayed for :02:00 (ACL 119)
> Feb 18 09:57:43 mailmx sendmail[26150]: p1IFvfPf026150: Milter: 
> to=<pikepikephiru@...>, reject=451 4.7.1 Greylisting in action, 
> please come back later

No other entries referencing p1IFvfPf026150 do exist?
Strange ...

What does it look like if milter-greylist is *not* active?
Is something like
Feb 18 09:57:43 sendmail[26150]: p1IFvfPf026150: <pikepikephiru@...>... User unknown
in the log?

> 
> for every received message, even though the users don't exist. I'd 
> rather not do the extra processing.

Maybe I did not got the problem: What means "undeliverable address" when
it is neither local (local user or local alias) nor in the virusertable?

[..]
> 
> The target domains are considered local in sendmail.  I need the lookups 
> in virtusertable and aliases.
> 
> > A mail for 3 recipients @MYDOMAIN arrives. One of this recipients is an
> > existing address (checked via an equivalent mechanism to virtuser
> > table). The others are invalid. Miltergreylist adds only the entry for
> > joachim.fabini@MYDOMAIN (the valid address), the other address are
> > rejected.
> 
> What about aliases?

Of course, aliases and other "local users" which do not exist in
the host's maildomain handled in the same way.

> 
> >  > > For the moment I am working around it by
> >  > > tracking the 'real' users, including them in the milter-greylist
> > config,
> >  > > and restricting greylisting to the specified addresses. However, it
> >  > > would be nicer if this could be handled automatically by letting
> >  > > sendmail reject addresses it can't deliver first. Is there any way to
> >  > > do that?
[..]
> Sendmail seems to be very fast at checking virtusertable and aliases. 
> I'd rather not have to duplicate that elsewhere.   Or to have to 
> maintain the same set of acceptable addresses separately in the greylist 
> config file.  It doesn't seem to matter whether MimeDefang or 
> milter-greylist is first in sendmail.mc - it still wants to process all 
> local-looking targets.  Are there any options as to when it runs?

You mean local targets directly mapped to user accounts and aliases
are treated differently? What happens when an alias is mapped to a local
user?

Is there any difference if you check working/non working addresses
with
/usr/lib/sendmail -bv
# or more verbose
/usr/lib/sendmail -d21.15 -bv
?

Can you tell us how is the "local" mailer defined in sendmail.cf/sendmail.mc
(Mlocal line/Mailer(local) line)?


Johann K.

RE: [milter-greylist] Can milter-greylist run after sendmail checks users?

2011-02-24 by Steger Wolfgang

> From: milter-greylist@yahoogroups.com
[mailto:milter-greylist@yahoogroups.com] On Behalf Of Les Mikesell
> Sent: Thursday, February 17, 2011 6:54 PM
> To: milter-greylist@yahoogroups.com
> Subject: [milter-greylist] Can milter-greylist run after sendmail checks
users?


  

> I am running sendmail/MimeDefang and recently added milter-greylist as 
> an additional spam check. The domains in question are regularly hit with 
> dictionary-attack type spam where most of the target users don't exist. 
> Sendmail is very quick at rejecting local addresses that are not in 
> the aliases or virtuser tables so that is normally not a problem, but 
> when milter-greylist is active it wants to greylist even the 
> undeliverable addresses. For the moment I am working around it by 
> tracking the 'real' users, including them in the milter-greylist config, 
> and restricting greylisting to the specified addresses. However, it 
> would be nicer if this could be handled automatically by letting 
> sendmail reject addresses it can't deliver first. Is there any way to 
> do that?

> Also, the extra log line about 'skipping greylist because this is the 
> default action' for the unprocessed addresses is filling my disks up to 
> the point that I had to change the log rotation. Is there any way to 
> turn that off? When it doesn't do anything, I don't need to know about it.

> -- 
> Les Mikesell
> lesmikesell@... <mailto:lesmikesell%40gmail.com> 

Ok., it's a week since you asked, but here are some ideas 
about the way greylisting works. Hopefully I did not get 
it all wrong.

If you run greylisting *before* user check, it won't make any 
difference, as the trick is, that the same combination of
sender IP, sender & recipient address "never" comes again in 
a SPAM message.

Also, if you are sending out non-deliverable-messages for 
nonexistant recipients: on SPAM they will either be bounced
(because the sender does not exist) or reach somebody innocent.

So I believe it is better to block SPAM with greylisting before
checking valid users. The only "problem" may be you are blowing 
up the greylisting database.

Just my 2cc, Wolfgang Steger
 
-- 
  Dipl.Ing. Wolfgang Steger        first facility GmbH
  IT Server Management             a member of EVN Group
  wolfgang.steger@... 1120 Wien, Technologiestr. 8
  Tel: +43(1) 61424-242            Mobil: +43(676) 88614 242

Re: [milter-greylist] Can milter-greylist run after sendmail checks users?

2011-02-24 by Oliver Fromme

Steger Wolfgang wrote:
 > Ok., it's a week since you asked, but here are some ideas 
 > about the way greylisting works. Hopefully I did not get 
 > it all wrong.
 > 
 > If you run greylisting *before* user check, it won't make any 
 > difference, as the trick is, that the same combination of
 > sender IP, sender & recipient address "never" comes again in 
 > a SPAM message.

How would that change if the recipient check comes first?
The recipient check doesn't change the IP/sender/recipient
tuple, so greylisting would work just as well afterwards.
In fact it would work *better* because the tuple database
will be much smaller, since it won't be "spammed" with
invalid users that won't receive any mail anyway.

In a similar way, sendmail's "greet_pause" feature rejects
connections that violate the SMTP protocol at the very
beginning (also *before* greylisting comes into play, and
before the recipient check).  This also reduces the grey-
listing load.

As a rule of thumb, the various rejection and anti-spam
measures should be executed in the order of complexity.
That is, features that cost almost zero CPU time should
come first, such as greet_pause.  Then check the IP for
various blacklists (doesn't cost much CPU either).  The
next thing should be to check for valid recipients,
because the recipients have to be checked anyway, sooner
or later.  The next thing is greylisting.  And finally,
messages that still haven't been rejected will be given
to more complex antispam software such as spamassassin
or dspam, or pass it through www.surbl.org and similar
services.

Of course, there might be cases where it's different.
For example, if your server is *not* targeted by a large
amount of spam to non-existent users *and* your recipient
checks are expensive (because of a slow LDAP server or
whatever), then it might make sense to greylist before
the recipient check.

 > Also, if you are sending out non-deliverable-messages for 
 > nonexistant recipients: on SPAM they will either be bounced
 > (because the sender does not exist) or reach somebody innocent.

No, you're not sending out any non-deliverable messages
during the envelope phase (before DATA phase), which is
where the recipient checks happen, because the message
won't be accepted in the first place.

Best regards
   Oliver

-- 
Oliver Fromme, secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing b. M.
Handelsregister: Registergericht Muenchen, HRA 74606,  Gesch\ufffdftsfuehrung:
secnetix Verwaltungsgesellsch. mbH, Handelsregister: Registergericht M\ufffdn-
chen, HRB 125758,  Gesch\ufffdftsf\ufffdhrer: Maik Bachmann, Olaf Erb, Ralf Gebhart

FreeBSD-Dienstleistungen, -Produkte und mehr:  http://www.secnetix.de/bsd

"The most important decision in [programming] language design
concerns what is to be left out."  --  Niklaus Wirth

Re: [milter-greylist] Can milter-greylist run after sendmail checks users?

2011-02-24 by Les Mikesell

On 2/22/2011 5:30 AM, Johann Klasek wrote:

>>>   >  >  Sendmail is very quick at rejecting local addresses that are not in
>>>   >  >  the aliases or virtuser tables so that is normally not a problem, but
>>>   >  >  when milter-greylist is active it wants to greylist even the
>>>   >  >  undeliverable addresses.
>>>
>>> What does this look like in maillog?
>>
>> If I have 'acl greylist default ...." set, I get entries like:
>>
>> Feb 18 09:57:43 mailmx milter-greylist: p1IFvfPf026150: addr
>> [118.96.199.46][118.96.199.46] from<rosfoewi00leobla@...>  to
>> <pikepikephiru@...>  delayed for :02:00 (ACL 119)
>> Feb 18 09:57:43 mailmx sendmail[26150]: p1IFvfPf026150: Milter:
>> to=<pikepikephiru@...>, reject=451 4.7.1 Greylisting in action,
>> please come back later
>
> No other entries referencing p1IFvfPf026150 do exist?
> Strange ...
>
> What does it look like if milter-greylist is *not* active?
> Is something like
> Feb 18 09:57:43 sendmail[26150]: p1IFvfPf026150:<pikepikephiru@...>... User unknown
> in the log?

Yes, most of the domains are mapped by the virtuser table with a default 
where I have specified the user unknown error like:
@... error:nouser No such user here.
For the domains that don't have a default in virtusertable, sendmail 
looks at aliases and rejects anyone else with the standard error.

>> for every received message, even though the users don't exist. I'd
>> rather not do the extra processing.
>
> Maybe I did not got the problem: What means "undeliverable address" when
> it is neither local (local user or local alias) nor in the virusertable?

You could be relaying to domains not in DNS, or you might be configured 
to not deliver if the From: domain doesn't resolve.

> You mean local targets directly mapped to user accounts and aliases
> are treated differently? What happens when an alias is mapped to a local
> user?

This machine doesn't do any local deliveries.  Everything it accepts is 
either in virtusertable or aliases with a delivery address on a 
different internal machine.

> Is there any difference if you check working/non working addresses
> with
> /usr/lib/sendmail -bv
> # or more verbose
> /usr/lib/sendmail -d21.15 -bv
> ?

The ones with virtuser/alias matches show deliverable with the 
expansions, ones that don't match give a user unknown error.

>
> Can you tell us how is the "local" mailer defined in sendmail.cf/sendmail.mc
> (Mlocal line/Mailer(local) line)?

That goes to procmail, but no address should ever go there anyway.

-- 
   Les Mikesell
    lesmikesell@...

Re: [milter-greylist] Can milter-greylist run after sendmail checks users?

2011-02-24 by Les Mikesell

On 2/24/2011 1:38 AM, Steger Wolfgang wrote:
>
>  > I am running sendmail/MimeDefang and recently added milter-greylist as
>  > an additional spam check. The domains in question are regularly hit with
>  > dictionary-attack type spam where most of the target users don't exist.
>  > Sendmail is very quick at rejecting local addresses that are not in
>  > the aliases or virtuser tables so that is normally not a problem, but
>  > when milter-greylist is active it wants to greylist even the
>  > undeliverable addresses. For the moment I am working around it by
>  > tracking the 'real' users, including them in the milter-greylist config,
>  > and restricting greylisting to the specified addresses. However, it
>  > would be nicer if this could be handled automatically by letting
>  > sendmail reject addresses it can't deliver first. Is there any way to
>  > do that?
>
>  > Also, the extra log line about 'skipping greylist because this is the
>  > default action' for the unprocessed addresses is filling my disks up to
>  > the point that I had to change the log rotation. Is there any way to
>  > turn that off? When it doesn't do anything, I don't need to know
> about it.
>
> Ok., it's a week since you asked, but here are some ideas
> about the way greylisting works. Hopefully I did not get
> it all wrong.
>
> If you run greylisting *before* user check, it won't make any
> difference, as the trick is, that the same combination of
> sender IP, sender & recipient address "never" comes again in
> a SPAM message.

The difference is that the server has to check a db entry, write a db 
entry that will have to be cleaned up later, and write another log line 
or two.  At many such messages per second, I'd prefer that the server 
didn't do the extra work.  Sendmail's virtuser/alias lookups are very 
fast and less of a problem.

> Also, if you are sending out non-deliverable-messages for
> nonexistant recipients: on SPAM they will either be bounced
> (because the sender does not exist) or reach somebody innocent.

No, this is a rejection at the SMTP level (which might make the sending 
server generate a bounce if it is a real mail server but doesn't do it 
directly).  But in case anyone is interested, I believe that the reason 
these domains are targeted with what appears to be continual dictionary 
attacks is that they were once handled by a qmail setup that accepted 
everything, then tried to generate bounces.  Apparently that got the 
accepted addresses into a database that is widely used to target spam.

> So I believe it is better to block SPAM with greylisting before
> checking valid users. The only "problem" may be you are blowing
> up the greylisting database.

Yes, that and the log files.  Why do that extra work when you know it 
can't possibly be delivered?

-- 
   Les Mikesell
    lesmikesell@...

Move to quarantaine

This moves the raw source file on disk only. The archive index is not changed automatically, so you still need to run a manual refresh afterward.