Milter-greylist

Hi

I have been able to spot the SIGSEGV on config reload problem on Cyril's
machine. The problem is caused by a low limit on the stack size when a
program is linked with -pthread (it does not happen on non threaded
program). I have a test program to check if it occurs:

# ulimit -s unlimited
# cat > x.c
#include <stdio.h>

void func(void);

int main(void) {
        char buf[5 * 1024 * 1024];      /* 5 MB */

        buf[sizeof(buf) - 1] = '\0';    /* Check we got the memory */
        printf("before func\n");
        func();
        printf("after func\n");

        return;
}


void func(void) {
        printf("function\n");
        return;
}
^D
# cc -pthread -o x x.c
# ./x
Segmentation fault

FreeBSD and Linux break on this. NetBSD is alright. Anyone wants to
check other systems?

Now the problem is spotted, anyone knows the fix? Is there some special
function to call in order to book a bigger stack for a thread?

-- 
Emmanuel Dreyfus
Il y a 10 sortes de personnes dans le monde: ceux qui comprennent 
le binaire et ceux qui ne le comprennent pas.
manu@...

manu@... writes:

> FreeBSD and Linux break on this. NetBSD is alright. Anyone wants to
> check other systems?

I will check on Solaris and maybe HP-UX next week.

> Now the problem is spotted, anyone knows the fix? Is there some special
> function to call in order to book a bigger stack for a thread?

Is pthread_attr_setstacksize() a good candidate ?  I really don't know...

manu@... writes:

> Now the problem is spotted, anyone knows the fix? Is there some special
> function to call in order to book a bigger stack for a thread?

Please ignore my previous silly answer :-[

I do have very good news !!!!

Check this:

$ cat x.c
#include <stdio.h>

#define THREAD_STACK_SIZE 5*1024*1024

void func(void);

int main(void) {
  static char buf[5 * 1024 * 1024];      /* 5 MB */

  buf[sizeof(buf) - 1] = '\0';    /* Check we got the memory */
  printf("before func\n");
  func();
  printf("after func\n");
  
  return;
}


void func(void) {
        printf("function\n");
        return;
}

[cyril@blackbox:~ 18:06:51]
$ cc -pthread -o x x.c

[cyril@blackbox:~ 18:06:55]
$ ./x
before func
function
after func

I will try this on milter-greylist asap. Anyway, do you have a mean to figure
out how large shall be the stack ?

Cyril Guibourg <cg+milter-greylist@...> wrote:

> #define THREAD_STACK_SIZE 5*1024*1024

How portable is that?

pthread_attr_setstacksize() is in the POSIX standards, so it seems the
way to go, but unfortunately, thread creation occurs in libmilter, so
there is no opportunity to use pthread_attr_setstacksize(). Except if I
use a dedicated thread for config reload.  

OTOH, I can't find anything about THREAD_STACK_SIZE. Is it documented in
any standard at all?

> I will try this on milter-greylist asap. Anyway, do you have a mean to figure
> out how large shall be the stack ?

Not really. On your machine, on config file reload, it reaches 4.4MB.
I'd say that 5MB is okay, but it might depends on the config file. It
also depends on the libpthread implementation.

4.4MB is really big. I wonder what consumes that much stack memory. I'd
like to track down stack memory waste, but when running with pthreads,
it can be difficult. Anyone went there already?

-- 
Emmanuel Dreyfus
Il y a 10 sortes de personnes dans le monde: ceux qui comprennent 
le binaire et ceux qui ne le comprennent pas.
manu@...

Cyril Guibourg wrote:
> manu@... writes:
> 
> 
>>Now the problem is spotted, anyone knows the fix? Is there some special
>>function to call in order to book a bigger stack for a thread?
> 
> 
> Please ignore my previous silly answer :-[
> 
> I do have very good news !!!!
> 
> Check this:
> 
> $ cat x.c
> #include <stdio.h>
> 
> #define THREAD_STACK_SIZE 5*1024*1024

I fail to see how this define can affect anyting in you program.
> 
> void func(void);
> 
> int main(void) {
>   static char buf[5 * 1024 * 1024];      /* 5 MB */

'static' variables are allocated in bss, not on the stack.


-- 
					Matthieu

manu@... writes:

> How portable is that?

Don't know. Btw, the freebsd port mechanism could add this during pre-build
stages, it has been designed for that too.

> pthread_attr_setstacksize() is in the POSIX standards, so it seems the
> way to go, but unfortunately, thread creation occurs in libmilter, so
> there is no opportunity to use pthread_attr_setstacksize(). Except if I
> use a dedicated thread for config reload.

Maybe be this is the solution. You are the architect.

> OTOH, I can't find anything about THREAD_STACK_SIZE. Is it documented in
> any standard at all?

Apparently no. I found references to it with google. The links are not
freebsd or linux centric. I would tend to think that it is present in
most implementations but it doesn't answer to your question.

> Not really. On your machine, on config file reload, it reaches 4.4MB.
> I'd say that 5MB is okay, but it might depends on the config file. It
> also depends on the libpthread implementation.

ok.

Matthieu Herrb <matthieu.herrb@...> writes:

> I fail to see how this define can affect anyting in you program.
> 'static' variables are allocated in bss, not on the stack.

You are completely right. I'm going to have some rest. :(

Matthieu Herrb <matthieu.herrb@...> wrote:

> > int main(void) {
> >   static char buf[5 * 1024 * 1024];      /* 5 MB */ 
> 'static' variables are allocated in bss, not on the stack.

I didn't noticed Cyril changed that in the test. The original test does
not use static data.

-- 
Emmanuel Dreyfus
Il y a 10 sortes de personnes dans le monde: ceux qui comprennent 
le binaire et ceux qui ne le comprennent pas.
manu@...

manu@... writes:

> I didn't noticed Cyril changed that in the test.

This is the consequence of trying several things without any though...

> The original test does not use static data.

And of course, using THREAD_STACK_SIZE define in milter-greylist doesn't
change anything while it seems to be the workaround to get Zope to run
under FreeBSD:
http://zope.org/Members/ensane/HowTo.2003-09-24.2929/howto_view

I promise I'll cme back to this list only if I have something clever to
share. 

Apologies.

manu@... writes:

> 4.4MB is really big. I wonder what consumes that much stack memory. I'd
> like to track down stack memory waste, but when running with pthreads,
> it can be difficult.

Do you experience the same stack grow under Net ?

> Anyone went there already?

No, fore sure.

Cyril Guibourg <cg+milter-greylist@...> wrote:

> > 4.4MB is really big. I wonder what consumes that much stack memory. I'd
> > like to track down stack memory waste, but when running with pthreads,
> > it can be difficult. 
> Do you experience the same stack grow under Net ?

Yes, the stack grows up to the same size, but NetBSD's pthreads
implementation seems to have a greater default stack limit, so it does
not screw up.

-- 
Emmanuel Dreyfus
Il y a 10 sortes de personnes dans le monde: ceux qui comprennent 
le binaire et ceux qui ne le comprennent pas.
manu@...