Milter-greylist

manu@... wrote:
> Matt Kettler <mkettler@...> wrote:
> 
>> Wait.. you think the *regex* is too resource intensive, but advocate using
>> RBLs instead?
>>
>> Are you completely out of your MIND???!!!
>>
>>
>> An RBL is a NETWORK TEST. You have to create a UDP socket, send a request,
>> wait for a reply, parse the reply..
> 
> That's it: you wait. That means the thread is sleeping and the CPU works
> somewhere else. Perhaps in another milter-greylist thread, perhaps in
> another process. 

Yes, I know that no CPU is being used during this time. I addressed both time
AND real CPU clock cycles.

> 
> Indeed a DNS lookup increases lattency, but it does not load the CPU as
> regexp computation does.
> 

I'd argue it does load the CPU more than a regex, although to a lesser degree
than it increases latency. (ie: yes, it's obvious there's a massive increase in
latency, but I say there's also a smaller increase in CPU load).

Yes, you do wait while waiting for the response to come back, and while doing
that you are consuming no CPU cycles.

However, the number of actual CPU cycles burned building the query, sending it,
receiving the reply and parsing that is by far higher than the regex is.

Remember, you don't just count the code that milter-greylist is running.
Consider all the code in the resolver library, OS IP stack, and NIC driver. That
all has to run too. And that all takes CPU time as well.

Sure, regexes are expensive compared to a binary compare, but they're not *THAT*
expensive.

Parsers are expensive too, and that's exactly what the resolver is going to have
to do with the DNS reply it gets.

Think about it. Picture in your head all the things your computer does to create
a DNS querry in a UDP packet, send it, receive a response, and process the response.

No, really. think about all of it.

Sending the query:
buffer allocation, DNS query formatting, context switch to kernel, IP/UDP header
addition, ethernet header addition, NIC programming.

<sleep for free>

Receiving the response:

interrupt handler, kernel thread wake, (possible memcpy depending on NIC and
kernel behaviors), ethernet header parsing, IP header parsing, UDP header
parsing, match against existing socket handles, wake user thread , context
switch to user space, buffer allocation, context switch to kernel, memcpy data
to user app buffer, context switch to user space, DNS response format parsing,
buffer deallocation.

There's a LOT of work going on under the covers here. That's not cheaper than a
pair of short regexes. If you think it is, you're ignoring a large number of
these steps which are all wrapped up in a library for you.

All the context switches alone are likely on-par in clock-cycles burned with the
regex evaluation. Those are not at all inexpensive because the entire CPU state
has to be saved off into a task descriptor. There's at least 3 context switches
involved here, and that's before you actually do any real work.