Milter-greylist

Matt Kettler wrote:
> manu@... wrote:
>> Matt Kettler <mkettler@...> wrote:
>>
>>> Wait.. you think the *regex* is too resource intensive, but advocate using
>>> RBLs instead?
>>>
>>> Are you completely out of your MIND???!!!
>>>
>>>
>>> An RBL is a NETWORK TEST. You have to create a UDP socket, send a request,
>>> wait for a reply, parse the reply..
>> That's it: you wait. That means the thread is sleeping and the CPU works
>> somewhere else. Perhaps in another milter-greylist thread, perhaps in
>> another process. 
> 
> Yes, I know that no CPU is being used during this time. I addressed both time
> AND real CPU clock cycles.
> 
>> Indeed a DNS lookup increases lattency, but it does not load the CPU as
>> regexp computation does.
>>
> 
> I'd argue it does load the CPU more than a regex, although to a lesser degree
> than it increases latency. (ie: yes, it's obvious there's a massive increase in
> latency, but I say there's also a smaller increase in CPU load).

As a side note, I made a quick test with queryperf of 100 PTR queries to
sbl-xbl.spamhaus.org and ran it through time. The records were just incrementing
 querries typical of an IP.. ie:
	1.1.1.1.sbl-xbl.spamhaus.org PTR
	1.1.1.2.sbl-xbl.spamhaus.org PTR

This would cause each lookup to be unique, although the basic NS records for the
sbl-xbl.spamhaus.org would be cached. This was directed at a resolving ( Yes,
resolving, ie: no forwarders and a root.hint zone) bind nameserver running on
localhost.

cat querytest.sbl-xbl | time queryperf

1.48user 0.85system 0:05.11elapsed 45%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (191major+26minor)pagefaults 0swaps

2.33 seconds of real CPU (1.48 user + 0.85 system =  was used to complete 100
those queries. 5.11 seconds of wall time elapsed

So a DNS query, with recursion, burns about 2ms of actual CPU. Slightly less due
to queryperf having to parse text input and generate text output.

Even with the whole batch cached in the local resolver, re-running it results in
0.05s = 5ms of real CPU time being used and 0.35s clock-time passing, so even a
cached query is 0.05ms = 50us.


Of course, this is all dependent on CPU, but I ran this on a single CPU 2ghz
intel box. Not the fastest but not too shabby.

The numbers are also quite crude, and probably have significant errors in them.

That said, do you really think your regex execution is THAT slow that it will
take longer than 50us of real CPU time to evaluate one on a 2ghz processor?