Milter-greylist

Hello,

I am trying to use the spamd feature
but I cannot make it to work for even a simple test e-mail.
I understand that the 4.3.8 is a development snapshot
but is that so trivially broken?
(For example the queue-id is sent as a user name for spamd.)
Is there a minimum version of spamassassin
needed to make it work?
I am using what comes with CentOS 5 by default.
 

Apr  7 16:42:01 mail sendmail[31895]: p37EfGkS031895:
from=<xxx@...>, size=22, class=0, nrcpts=1,
msgid=<201104071441.p37EfGkS031895@mail.mydomain.org>, proto=SMTP,
daemon=MTA, relay=[10.0.0.1]
Apr  7 16:42:01 mail spamd[11116]: spamd: connection from
mail.mydomain.org [127.0.0.1] at port 46440 
Apr  7 16:42:01 mail spamd[11116]: spamd: handle_user unable to find
user: 'p37EfGkS031895' 
Apr  7 16:42:01 mail spamd[11116]: spamd: still running as root: user
not specified with -u, not found, or set to root, falling back to nobody
Apr  7 16:47:01 mail sendmail[31895]: p37EfGkS031895: Milter
(milter-greylist): timeout before data read
Apr  7 16:47:01 mail sendmail[31895]: p37EfGkS031895: Milter
(milter-greylist): to error state
Apr  7 16:47:01 mail sendmail[31895]: p37EfGkS031895: Milter: data,
reject=451 4.3.2 Please try again later
Apr  7 16:47:01 mail sendmail[31895]: p37EfGkS031895:
to=<yyy@...>, delay=00:05:10, pri=30022, stat=Please try again
later
Apr  7 16:47:01 mail milter-greylist: spamd protocol version mismatch
Apr  7 16:47:01 mail milter-greylist: spamd returned non-ok
Apr  7 16:47:01 mail milter-greylist: SPAMD/1.0 79 Timeout: (300 second
timeout while trying to CHECK)^M 
Apr  7 16:47:01 mail milter-greylist: ACL evaluation failure
Apr  7 16:47:01 mail spamd[11116]: spamd: timeout: (300 second timeout
while trying to CHECK) at /usr/bin/spamd line 2016, <GEN14> line 11.

Bests,
Attila

> I understand that the 4.3.8 is a development snapshot
> but is that so trivially broken?

So I found the bug.
I would be surprised that even one people could use the spamd feature
of the current development version (4.3.8) without fixing that issue.
There was a double accounting in the variable priv->priv_msgcount
which has leaded to incorrect Content-length: field value provided to
spamd.

The patch which fixes that issue is attached.

<attila.bruncsak@...> wrote:

> I am trying to use the spamd feature
> but I cannot make it to work for even a simple test e-mail.
> I understand that the 4.3.8 is a development snapshot
> but is that so trivially broken?

I do not use it. Manuel Badzong contributed it, perhaps he can comment?
Enrico Scholz and Petar Bogdanovic did send some fixes, therefore they
may have a word to say.

> (For example the queue-id is sent as a user name for spamd.)

Here is the offending code. Indeed the queud-id is sent as User:
        snprintf(buffer, SPAMD_BUFLEN,
          "CHECK SPAMC/1.2\r\n" 
          "Content-length: %d\r\n"
          "User: %s\r\n\r\n",
          (unsigned int)(priv->priv_msgcount + strlen(rcvhdr)),
           priv->priv_queueid);

I can change that, but what should be in the User field? Or is it the
field name that must be changed? I am clueless.


-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu@...

<attila.bruncsak@...> wrote:

> So I found the bug.
> I would be surprised that even one people could use the spamd feature
> of the current development version (4.3.8) without fixing that issue.
> There was a double accounting in the variable priv->priv_msgcount
> which has leaded to incorrect Content-length: field value provided to
> spamd.

I checked it in. That is strange, the bug did not exist on the 4.2
branch.

In real_body(), we also have
                priv->priv_msgcount += strlen(crlf);

That should remain, right?
-- 
Emmanuel Dreyfus
http://hcpnet.free.fr/pubz
manu@...

On Fri, Apr 08, 2011 at 06:20:41PM +0200, manu@... wrote:
> 
> > (For example the queue-id is sent as a user name for spamd.)
> 
> Here is the offending code. Indeed the queud-id is sent as User:
>         snprintf(buffer, SPAMD_BUFLEN,
>           "CHECK SPAMC/1.2\r\n" 
>           "Content-length: %d\r\n"
>           "User: %s\r\n\r\n",
>           (unsigned int)(priv->priv_msgcount + strlen(rcvhdr)),
>            priv->priv_queueid);
> 
> I can change that, but what should be in the User field? Or is it the
> field name that must be changed? I am clueless.


We did that on purpose:

On Mon, May 11, 2009 at 09:55:38AM +0200, Petar Bogdanovic wrote:
> On Mon, May 11, 2009 at 09:35:20AM +0200, Petar Bogdanovic wrote:
> > On Mon, May 11, 2009 at 07:17:39AM +0000, Emmanuel Dreyfus wrote:
> > > On Mon, May 11, 2009 at 09:13:53AM +0200, Petar Bogdanovic wrote:
> > > >         snprintf(buffer, SPAMD_BUFLEN,
> > > > -         "CHECK SPAMC/1.2\r\nContent-length: %d\r\n\r\n",
> > > > -         (unsigned int)(priv->priv_msgcount + strlen(rcvhdr)));
> > > > +         "CHECK SPAMC/1.2\r\n"
> > > > +         "Content-length: %d\r\n"
> > > > +         "User: %s\r\n\r\n",
> > > > +         (unsigned int)(priv->priv_msgcount + strlen(rcvhdr)),
> > > > +          priv->priv_queueid);
> > > 
> > > Is it reasonable to abuse the User field?
> > 
> > I'm not entirely sure (...)
> 
> At least you can't rely on the value of this header since the only thing
> which prevents you from providing a bogus value is a correctly compiled
> spamc or a strict suid-environment which is not an option if spamd is
> listening to an internet socket.

On Fri, Apr 08, 2011 at 06:32:28PM +0200, manu@... wrote:
> <attila.bruncsak@...> wrote:
> 
> > So I found the bug.
> > I would be surprised that even one people could use the spamd feature
> > of the current development version (4.3.8) without fixing that issue.
> > There was a double accounting in the variable priv->priv_msgcount
> > which has leaded to incorrect Content-length: field value provided to
> > spamd.
> 
> I checked it in. That is strange, the bug did not exist on the 4.2
> branch.

That line is unchanged since 2006:

	1.220 (manu 29-Jun-09):              memcpy(b->b_lines + priv->priv_buflen, chunk, i);
	1.175 (manu 04-Mar-07):              b->b_lines[linelen] = '\0';
	1.235 (manu 12-Jul-10):              b->b_len = linelen;
	1.175 (manu 04-Mar-07):              priv->priv_buflen = 0;
	1.174 (manu 04-Mar-07): 
	1.154 (manu 16-Jan-07):              TAILQ_INSERT_TAIL(&priv->priv_body, b, b_list);
	1.146 (manu 26-Dec-06): 

	1.146 (manu 26-Dec-06):              priv->priv_msgcount += linelen;
	       ^^^^^^^^^^^^^^
	1.214 (manu 06-Nov-08):      }
	1.214 (manu 06-Nov-08): 
	1.214 (manu 06-Nov-08):      if(i < size) { /* keep the remains for later */
	1.146 (manu 26-Dec-06):              if ((priv->priv_buf = realloc(priv->priv_buf, 

..years before spamd support was added.  Hence, all spamd users have
been running milter-greylist with that line since spamd was imported.

Attila, what SA version are you running?  What does tcpdump say?

On Fri, Apr 08, 2011 at 06:32:28PM +0200, manu@... wrote:
> In real_body(), we also have
>                 priv->priv_msgcount += strlen(crlf);
> 
> That should remain, right?

Right. This is the lenght of the added crlf between header an body.


On line 892 the lenght of the current junk is added to msgcount. This line
seems new, probably added cleaning up real_body(). I guess line 955 was
overlooked and triggered a message size bug passed on to spamd.c.


Cheers, Manuel

> Attila, what SA version are you running?  What does tcpdump say?
> 

I am using standard SA which comes with CentOS 5.
The name of the rpm kit: spamassassin-3.2.5-1.el5.
I just installed last Thursday with "yum install spamassassin"
Zero customization for the time being.

The "tcpdump -i lo -A -s0 port 783" command gave me the following
information:
For a test e-mail with length 240 bytes the content-length field had a
value of 247.

That was a clue where to search further and with simple code reading I
quickly found the bug.
It is really possible that the message size value was incorrect since
long time but seemingly
no any extension was dependent on the precise value till spamd came into
the picture.

With that fix in mg now the spamd is in production at my site
(for my own e-mail address only for the time being)
and it has happily assassinated many spam e-mails already.