Milter-greylist

milter-greylist community,

Is there some way to pass more information to spamd, so that it would log some additional information?

Currently, spamd logs only the msgid as mid=:

May  9 09:07:07 host spamd[14983]: spamd: checking message <87ed0-f50-4a058d20@...> for (unknown):98
May  9 09:07:09 host spamd[14983]: spamd: result: . 0 - HTML_MESSAGE,SPF_SOFTFAIL scantime=2.0,size=11466,user=(unknown),uid=98,required_score=5.0,rhost=localhost.localdomain,raddr=127.0.0.1,rport=46827,mid=<87ed0-f50-4a058d20@...>,autolearn=no

Ideally, I'd like for spamd to log the message number, like n49E76U8015189

May  9 09:07:07 host sm-acceptingconnections[15189]: n49E76U8015189: from=<credo-bounce-3920-556752-vstdnrx@...>, size=10810, class=0, nrcpts=1, msgid=<87ed0-f50-4a058d20@...>, proto=ESMTP, daemon=MTA, relay=qmta12.westchester.pa.mail.comcast.net [76.96.59.227]

However, I don't see that option.  Could we pass the recipient email address as the user or maybe have milter-greylist log the information returned from spamd?

Thanks.

Jim

On Sun, May 10, 2009 at 04:02:05PM -0000, Jim Hermann wrote:
> milter-greylist community,
> 
> Is there some way to pass more information to spamd, so that it would
> log some additional information?
> 
> Currently, spamd logs only the msgid as mid=:
> 
> May  9 09:07:07 host spamd[14983]: spamd: checking message <87ed0-...
> May  9 09:07:09 host spamd[14983]: spamd: result: . 0 - HTML_MESSA...
> 
> Ideally, I'd like for spamd to log the message number, like
> n49E76U8015189

I assume that's a queue-id.


> May  9 09:07:07 host sm-acceptingconnections[15189]: n49E76U801518...
> 
> However, I don't see that option.  Could we pass the recipient email
> address as the user or maybe have milter-greylist log the information
> returned from spamd?

We could abuse the `User:' spamd protocol header.  But I'm not sure
about the benefit of such a change.  What do you want to achieve?
Uniqueness for a log-parser?  If yes: What's wrong with the message-id?



   Petar Bogdanovic

--- In milter-greylist@yahoogroups.com, Petar Bogdanovic <petar@...> wrote:
>
> On Sun, May 10, 2009 at 04:02:05PM -0000, Jim Hermann wrote:
> > milter-greylist community,
> > 
> > Is there some way to pass more information to spamd, so that it would
> > log some additional information?
> > 
> > Ideally, I'd like for spamd to log the message number, like
> > n49E76U8015189
> 
> I assume that's a queue-id.
> 
> We could abuse the `User:' spamd protocol header.  But I'm not sure
> about the benefit of such a change.  What do you want to achieve?
> Uniqueness for a log-parser?  If yes: What's wrong with the message-id?

All the maillog entries contain the queue-id.  Only a few contain the message-id.  It's a two-step process to find the recipient from the spamd report or vice-versa.

Thanks.

Jim

Jim Hermann <hostmaster@...> wrote:

> All the maillog entries contain the queue-id.  Only a few contain the
> message-id.  It's a two-step process to find the recipient from the
> spamd report or vice-versa.

The message-id is not known before the DATA stage, this why you cannot
have it everywhere.

-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu@...

--- In milter-greylist@yahoogroups.com, manu@... wrote:
>
> Jim Hermann <hostmaster@...> wrote:
> 
> > All the maillog entries contain the queue-id.  Only a few contain the
> > message-id.  It's a two-step process to find the recipient from the
> > spamd report or vice-versa.
> 
> The message-id is not known before the DATA stage, this why you cannot
> have it everywhere.

That's the only time I want the queue-id, after the DATA stage, when I run spamd.

Couldn't we get some FORMAT STRINGS to allow logging some additional information returned by spamd to miter-greylist?  like the score and the list of failed rules?

Thanks.

Jim

Jim Hermann <hostmaster@...> wrote:

> Couldn't we get some FORMAT STRINGS to allow logging some additional
> information returned by spamd to miter-greylist?  like the score and the
> list of failed rules?

Sure we can, the most difficult thing is to find an available % letter
:-)

-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu@...

... Pushing the queue-id to spamd requires less work and doesn t make some maillog entries redundant.  Could you please test the attached patch? Petar

On Mon, May 11, 2009 at 09:13:53AM +0200, Petar Bogdanovic wrote:
>  	snprintf(buffer, SPAMD_BUFLEN,
> -	  "CHECK SPAMC/1.2\r\nContent-length: %d\r\n\r\n",
> -	  (unsigned int)(priv->priv_msgcount + strlen(rcvhdr)));
> +	  "CHECK SPAMC/1.2\r\n"
> +	  "Content-length: %d\r\n"
> +	  "User: %s\r\n\r\n",
> +	  (unsigned int)(priv->priv_msgcount + strlen(rcvhdr)),
> +	   priv->priv_queueid);

Is it reasonable to abuse the User field?


-- 
Emmanuel Dreyfus
manu@...

On Mon, May 11, 2009 at 07:17:39AM +0000, Emmanuel Dreyfus wrote:
> On Mon, May 11, 2009 at 09:13:53AM +0200, Petar Bogdanovic wrote:
> >  	snprintf(buffer, SPAMD_BUFLEN,
> > -	  "CHECK SPAMC/1.2\r\nContent-length: %d\r\n\r\n",
> > -	  (unsigned int)(priv->priv_msgcount + strlen(rcvhdr)));
> > +	  "CHECK SPAMC/1.2\r\n"
> > +	  "Content-length: %d\r\n"
> > +	  "User: %s\r\n\r\n",
> > +	  (unsigned int)(priv->priv_msgcount + strlen(rcvhdr)),
> > +	   priv->priv_queueid);
> 
> Is it reasonable to abuse the User field?

I'm not entirely sure but this (optional) header seems to be pretty
useless otherwise.  We could add a prefix in order to make it
distinguishable from ordinary users.

But OTOH, we could also do nothing since all you need in order to group
your maillog by smtp-sessions is already there (queueid/msgid).



   Petar Bogdanovic

On Mon, May 11, 2009 at 09:35:20AM +0200, Petar Bogdanovic wrote:
> On Mon, May 11, 2009 at 07:17:39AM +0000, Emmanuel Dreyfus wrote:
> > On Mon, May 11, 2009 at 09:13:53AM +0200, Petar Bogdanovic wrote:
> > >  	snprintf(buffer, SPAMD_BUFLEN,
> > > -	  "CHECK SPAMC/1.2\r\nContent-length: %d\r\n\r\n",
> > > -	  (unsigned int)(priv->priv_msgcount + strlen(rcvhdr)));
> > > +	  "CHECK SPAMC/1.2\r\n"
> > > +	  "Content-length: %d\r\n"
> > > +	  "User: %s\r\n\r\n",
> > > +	  (unsigned int)(priv->priv_msgcount + strlen(rcvhdr)),
> > > +	   priv->priv_queueid);
> > 
> > Is it reasonable to abuse the User field?
> 
> I'm not entirely sure (...)

At least you can't rely on the value of this header since the only thing
which prevents you from providing a bogus value is a correctly compiled
spamc or a strict suid-environment which is not an option if spamd is
listening to an internet socket.



   Petar Bogdanovic

--- In milter-greylist@yahoogroups.com, Petar Bogdanovic <petar@...> wrote:
>
> Pushing the queue-id to spamd requires less work and doesn't make some
> maillog entries redundant.  Could you please test the attached patch?

I don't know much about running patch.  This is the response I got:

# patch -p0 < spamd_patch
patching file spamd.c
patch: **** malformed patch at line 9: spamd_rcvhdr(priv, rcvhdr, SPAMD_BUFLEN);

Was I not supposed to include the first 4 lines?

Index: spamd.c
===================================================================
RCS file: /milter-greylist/milter-greylist/spamd.c,v
retrieving revision 1.10

--- In milter-greylist@yahoogroups.com, "Jim Hermann" <hostmaster@...> wrote:
>
> --- In milter-greylist@yahoogroups.com, Petar Bogdanovic <petar@> wrote:
> >
> > Pushing the queue-id to spamd requires less work and doesn't make some
> > maillog entries redundant.  Could you please test the attached patch?
> 
> I don't know much about running patch.  This is the response I got:

I made the changes manually.  It works fine.

#cat maillog | grep n4C3DdF2019852

[snip]
May 11 22:13:40 host spamd[19237]: spamd: handle_user unable to find user: 'n4C3DdF2019852'
May 11 22:13:40 host spamd[19237]: spamd: checking message <380-2200952123939593@...> for n4C3DdF2019852:98
May 11 22:13:40 host spamd[19237]: spamd: clean message (-3.4/5.0) for n4C3DdF2019852:98 in 0.5 seconds, 52335 bytes.
May 11 22:13:40 host sm-acceptingconnections[19852]: n4C3DdF2019852: May 11 22:13:40 host spamd[19237]: spamd: result: . -3 - BAYES_00,HTML_MESSAGE scantime=0.5,size=52335,user=n4C3DdF2019852,uid=98,required_score=5.0,rhost=localhost.localdomain,raddr=127.0.0.1,rport=57070,mid=<380-2200952123939593@earthlink.net>,bayes=0.000000,autolearn=ham
[snip]

Thanks.

Jim

On Tue, May 12, 2009 at 03:14:04AM -0000, Jim Hermann wrote:
> --- In milter-greylist@yahoogroups.com, "Jim Hermann" <hostmaster@...> wrote:
> >
> > --- In milter-greylist@yahoogroups.com, Petar Bogdanovic <petar@> wrote:
> > >
> > > Pushing the queue-id to spamd requires less work and doesn't make some
> > > maillog entries redundant.  Could you please test the attached patch?
> > 
> > I don't know much about running patch.  This is the response I got:
> >
> > # patch -p0 < spamd_patch
> > patching file spamd.c
> > patch: **** malformed patch at line 9: spamd_rcvhdr(priv, rcvhdr, SPAMD_BUFLEN);

Seems to me like the file is corrupt.


> [snip]
> May 11 22:13:40 host spamd[19237]: spamd: handle_user unable to find user: 'n4C3DdF2019852'

Right.. I guess you're running spamd without -x.


> May 11 22:13:40 host spamd[19237]: spamd: clean message (-3.4/5.0) for n4C3DdF2019852:98 in 0.5 seconds, 52335 bytes.

Nice.  I had always troubles catching this line since it has no msgid..


As it is right now, this won't work on postfix.  Maybe later I'll post
something better.



   Petar Bogdanovic

Petar Bogdanovic <petar@...> wrote:

> As it is right now, this won't work on postfix.  Maybe later I'll post
> something better.

Shall I commit your change, or hold my breath, after all?

-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu@...

On Tue, May 12, 2009 at 02:09:51PM +0200, manu@... wrote:
> Petar Bogdanovic <petar@...> wrote:
> 
> > As it is right now, this won't work on postfix.  Maybe later I'll post
> > something better.
> 
> Shall I commit your change, or hold my breath, after all?

If you refer to the small patch in this thread, please no.  Since it
doesn't work with postfix, it's not good yet.

The other one I posted in an other thread (Mon, 11 May 2009 23:56:00
+0200, spamd-RDNS_NONE problem...) is fine so feel free to commit if
you think it's ok.



   Petar Bogdanovic