[Python-Dev] Making python C-API thread safe (try 2)

[Mitch Chapman]

On Friday, September 12, 2003, at 01:57  PM, Skip Montanaro wrote:
>     Mitch> At IPC8 Greg Wilson, then of the Software Carpentry project,
>     Mitch> noted that the GIL made it hard to write multi-threaded 
> Python
>     Mitch> apps which could take advantage of multi-processor systems. 
>  He
>     Mitch> argued that this would limit Python's appeal in some 
> segments of
>     Mitch> the scientific community.
>
> This is a known issue.

For example, it was discussed at IPC8 ;)

> Thusfar, it hasn't seemed to slow down Python's
> acceptance by the scientific community all that much.

Good point.

>     Mitch> Perhaps those who find the subject important have left the
>     Mitch> community?  Perhaps they've adopted kludgey workarounds?
>
> Or perhaps they are happy to have tools like scipy and MayaVi to make 
> their
> jobs easier.

No doubt.  I'm among those happy to have access to scipy.weave, for
example.  But I also wish for better thread scalability to make my job
easier.

Unlike Andrew I don't think the lack of maintenance for 1.4's free
threading packages is due to any perception that threading performance
is unimportant.  It seems more likely that the packages were not updated
because they proved not to solve the performance problems, and that no
alternatives have emerged because the problem is hard to solve.

So, if Harri Pesonen has ideas for achieving better thread scalability,
I want to encourage him to develop them rather than to suggest such
an effort is unimportant to the community.

> Nobody has claimed that it isn't a problem for some people.

I read Andrew's comment as meaning that he believed nobody in the
community thought scalable threading was particularly important. In
order for that to be true, something like Greg Wilson's prediction, that
many who would otherwise have found Python appealing would have turned
to other solutions, would have to have come to pass. ("Perhaps those...
have left the community?")

Or else, as I noted, people might have found kludgey workarounds.  Or as
had happened in my company, they might have deferred some performance
problems which could have been addressed by scalable threading.  Both of
which are consistent with your observation:

> It's maybe less
> of a problem than it appears at first though.

Having said all of that, it looks like this thread originated on python-
dev, rather than on c.l.py where I found it.  So I'm probably missing
some reasons for what appears to be a dismissive attitude toward Harri's
efforts.  I'll go read the python-dev archives.

--
Mitch