[Python-Dev] Pythreads and BSD descendants

[Andrew MacIntyre]

Martin v. Löwis wrote:
> Cameron Laird schrieb:
>> Folklore that I remember so unreliably I avoid trying to repeat it here
>> held that Python threading had problems on BSD and allied Unixes.  What's
>> the status of this?
> 
> The problem that people run into again and again is the stack size. The
> BSDs allow for so little stack so that even the quite conservative
> estimates of Python as to how many recursions you can have are
> incorrect, and you get an interpreter crash rather than a RuntimeError
> (as you should).

There are 2 aspects to the thread stack size issue:
- the stack size for the primary thread;
- the stack size for created threads.

I haven't done any investigating for FreeBSD 6.x and later, but I know
that FreeBSD 4.x had a hard coded stack size of 1MB for the primary 
thread in a thread enabled application, which is what Martin's comment 
above particularly applies to.  This affects code that doesn't use 
threads at all, and was particularly painful with REs prior to SRE being 
made non-recursive.

If you build the interpreter without thread support, stack space is 
instead controlled by session limits which are usually generous 
(typically 64MB).

I don't recall exactly FreeBSD's default stack size for threads created 
via pthread_create() but it is fairly small (32kB or 64kB comes to 
mind).    Zope is one application known to be affected by this lack of 
stack size in created threads.  At least the stack size for new threads 
can be adjusted at runtime, and a mechanism for doing this was added to 
Python 2.5.

> Furthermore, every time we decrease the that number, the next system
> release somehow manages to make the limit even smaller. This was
> never properly analyzed; I suspect that the stack usage of Python
> increases, either due to compiler changes or due to change to Python
> itself.

I have seen examples of stack consumption increasing with increasing gcc 
version number, sometimes depending on optimisation choices.

Regards,
Andrew.

-- 
-------------------------------------------------------------------------
Andrew I MacIntyre                     "These thoughts are mine alone..."
E-mail: andymac at bullseye.apana.org.au  (pref) | Snail: PO Box 370
        andymac at pcug.org.au             (alt) |        Belconnen ACT 2616
Web:    http://www.andymac.org/               |        Australia