[Python-Dev] Making python C-API thread safe (try 2)

Christian Tismer tismer at tismer.com
Mon Nov 10 21:52:16 EST 2003


Not having read c.l.py for too long, some comments, anyway...

A.M. Kuchling wrote:

> On Fri, 12 Sep 2003 07:56:55 +0300, 
> 	Harri Pesonen <fuerte at sci.fi> wrote:
> 
>>I don't know, I got mail about writing a PEP. It is clear that it would 
>>not be accepted, because it would break the existing API. The change is 
>>so big that I think that it has to be called a different language.
> 
> 
> It would just be a different implementation of the same language.  Jython
> has different garbage collection characteristics from CPython, but they
> still implement the same language; Stackless Python is still Python.

This is only half the truth. Of course, you can run all of your
code in Stackless without change. But as soon as you have become
familiar  with it, your programming sty changes so drastically,
that you never will want to go back.
I realized this late, after my first "eat your own dogfood"
project. Stackless dramatically simplifies your coding style.
This seems to be an irreversible process.
I will provide examples.

>>because this is too important to be ignored. Python *needs* to be 
>>free-threading...

While written so heartily, and I can understand this very much,
it appears to be very, very wrong, since it does not address
general needs.
I admit: TZhere are situations where you need this, and you would
easily pay the extra of an at least 20-30 % overhead for being
free-threaded. But this isn't common-case.

Python's model of object sharing enforces such a costly scheme
at the moment. In most cases, I strongly believe that this is
not necessary, basically. The fact that access to almost any
Python object is possible at almost any time is not a feature,
but an artifact. Having to protect any mutable object at any
time is a consequence of this. This protection currently either
has to be the GIL, or builtin protection for the objects.

I guess, that in most cases, you would want to have almost
completely disjoint object spaces without any implicit sharing
of mutables. You would provide extra communication primitives
in order to share certain objects, instead. This way, most of
the free threading issues would vanish, in favor of a limited
set of controlled, shared objects, while most of the rest would
just run unrestricted.

Playing with such derivatives will be one of the strengths of
the PyPy project, which has ability to try alternatives as one
of its major goals. In CPython, you currently don't have much
more alternatives than to run disjoint processes, which are
communicating by exchanging pickled objects.
(Which is, IMHO, not the worst solution at all!)

> On the other hand, considering that the last free threading packages were
> for 1.4, and no one has bothered to update them, the community doesn't seem
> to find the subject as important as you do. :)

I think it is worthwile to be considered as a special alternative.
Making it *the* requirement is surely not the right goal for a tool
that has to fulfil everybody's needs.

Mysuggestion is to add this as a feature request to PyPy, together
with some effort supporting it. If PyPy is going to be as flexible
as we claimed many times, then it should be possible to derive
a version with the desired properties. But this is meant to be
a challenge for Harri, for instance.

all the best -- chris

-- 
Christian Tismer             :^)   <mailto:tismer at tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  mobile +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/





More information about the Python-Dev mailing list