Milter-greylist

I know, 3.0 final is out and I should upgrade to it, it's on my todo.

However, this morning I came in to find my milter-greylist running with a vsize
of 975MB. That's substantially larger than I've ever seen it grow to before.
Upon reducing my timeout from 5d to 3d and restarting, it shrank to 8mb. Right
now, 4 hours later, it's up to 17mb.

Looking at the "dumping records" in my maillog, the largest number of records
all week is 40,445. That was at 2:30 this morning.

Right before reload it dumped 34007 records. After reloading due to the shorter
timeout it reduced itself to 23,280 records.

Currently it's got 23,253 records. Which is almost half of what the peak was,
but the memory usage is a factor of 65 smaller than it was earlier. This leads
me to suspect the size problem is something more than just a lot of records.

2:30am	40445 records, unknown size
11am	34007 records, 975m process size (according to top)
11:15am 23280 records 8m process size
3pm	23253 records  17m process size

Currently (3pm) ps reports a vsz of 44692 and a RSS of 5280
		top reports a size 17460, rss of 5264  and share of 3224
I don't have ps info for the other points in time.


Looking at the ChangeLog, I don't see anything that would suggest any leaking or
memory overflows was fixed in 3.0.

3.0
        Fix crashes during dump reloads (AIDA Shinra)
        Fix DoS in MX sync protocol (AIDA Shinra)

Neither of those sounds like they'd cause my excessive memory use problem.

Anyone else seeing similar problems?

FWIW I am using the new DNSBL code, and am using 14 DNSBLs, but those are
actually boiled down to 3 unique lists. (so milter-greylist makes 14 queries,
but 11 of them will be cached because the query is redundant. The difference
being what result-code the test checks for.)

I just had this happen again over the weekend. A simple restart with no change
in parameters dropped the "size" (as reported by top) of milter-greylist from
over 1 gig to 11,872k.

I'm upgrading to 3.0 now, however, it concerns me that I seem to be suffering
from memory leaks and there's no mention of any memory leak problems being fixed
in the ChangeLog.


Is anyone else seeing explosively-huge milter-greylists recently?

This doesn't look like dictionary-attack overload. My MTA died on Sunday at
14:08, but grepping my logs for milter-greylist create and removed messages
there's nothing unusual around this timeframe.

Number created/removed by hour:
"Dec 10 08:"    268/263
"Dec 10 09:"    216/314
"Dec 10 10:"    265/303
"Dec 10 11:"    208/273
"Dec 10 12:"    253/282
"Dec 10 13:"     208/255
"Dec 10 14:"      41/22

Looking at MRTG, memory usage on the server stayed normal from 11/21 until
Thursday 12/7. Starting on 12/7, memory usage on the server crept up slowly and
steadily until the MTA died on 12/10.


Looking back at last Sunday, for comparison, the load was lower, but not by that
much:

"Dec  3 11:"	190/542
"Dec  3 12:"	217/199
"Dec  3 13:"	166/193


However, looking at what happened when I restarted my server at 10:56 today:
"Dec 11 10:56"    14497/8858

But between 12/10 15:00 and 12/11 10:50, pretty much no create or removes happened.

"Dec 10 2.:"	0/0
"Dec 11 0.:"	0/0

Does MG only do management when sendmail does a connection? That explains all
the removes.. but what about the 14k creates the same minute I restarted? Is
that it just reloading the DB?

I reloaded it again, an hour later..
"Dec 11 11:56"	10060/6

So, my DB isn't a whole lot smaller than it was at the 10:56 reload. The removes
come first, so I suppose at reload time my DB was about 23,355 entries, and now
it's about 10,000. But that's a factor of 2 DB size, but memory is a factor of
100 less...

Any ideas?


on 11/22/2006 Matt Kettler wrote:

> I know, 3.0 final is out and I should upgrade to it, it's on my todo.
> 
> However, this morning I came in to find my milter-greylist running with a vsize
> of 975MB. That's substantially larger than I've ever seen it grow to before.
> Upon reducing my timeout from 5d to 3d and restarting, it shrank to 8mb. Right
> now, 4 hours later, it's up to 17mb.
> 
> Looking at the "dumping records" in my maillog, the largest number of records
> all week is 40,445. That was at 2:30 this morning.
> 
> Right before reload it dumped 34007 records. After reloading due to the shorter
> timeout it reduced itself to 23,280 records.
> 
> Currently it's got 23,253 records. Which is almost half of what the peak was,
> but the memory usage is a factor of 65 smaller than it was earlier. This leads
> me to suspect the size problem is something more than just a lot of records.
> 
> 2:30am	40445 records, unknown size
> 11am	34007 records, 975m process size (according to top)
> 11:15am 23280 records 8m process size
> 3pm	23253 records  17m process size
> 
> Currently (3pm) ps reports a vsz of 44692 and a RSS of 5280
> 		top reports a size 17460, rss of 5264  and share of 3224
> I don't have ps info for the other points in time.
> 
> 
> Looking at the ChangeLog, I don't see anything that would suggest any leaking or
> memory overflows was fixed in 3.0.
> 
> 3.0
>         Fix crashes during dump reloads (AIDA Shinra)
>         Fix DoS in MX sync protocol (AIDA Shinra)
> 
> Neither of those sounds like they'd cause my excessive memory use problem.
> 
> Anyone else seeing similar problems?
> 
> FWIW I am using the new DNSBL code, and am using 14 DNSBLs, but those are
> actually boiled down to 3 unique lists. (so milter-greylist makes 14 queries,
> but 11 of them will be cached because the query is redundant. The difference
> being what result-code the test checks for.)
>