[Python-ideas] Exploration PEP : Concurrency for moderately massive (4 to 32 cores) multi-core architectures

[Krishna Sankar]

Stephen,
    Now I see where you are coming from and you are right. Plausible is 
the operative word. It is not that Guido and Sr.Developers will not look 
at it at all, but will not look at it in the absence of a plausible 
implementation. This was evident in the blog discussions with Bruce as 
well. I was encouraged by Guido's reply. He is doing the right thing.

    Anyway, the point is, now we have a good understanding of the 
dynamics. There is a small community who would like to see this happen 
and it is up to us to show it can be done. A few key things need to happen:

    a)       A good set of benchmarks on multi-core systems, extending 
excellent work (for example 
http://blogs.warwick.ac.uk/dwatkins/entry/benchmarking_parallel_python_1_2/)
         I have an 8 core machine (2 X 4 core) and plan to run this 
benchmark as well as create others like the Red Black Tree (Guido had 
suggested that, even before that I was thinking of it)
    b.1)      Plausible POC with GIL - pp, actor pattern/agent paradigm 
et al.
    b.2)      Plausible one, as and when GIL is removed
    c)      Figure out what support is needed from PVM et al (b.1 and/or 
b.2)
    d)      PEP and onwards ...

    Most probably I am stating the obvious. My take is that lots of the 
work is already there. We need to converge and do the rest to work 
towards a PEP.I know that Guido and Sr Developers will involve 
themselves at the appropriate time.

Cheers
<k/>
Stephen J. Turnbull wrote:
> Krishna Sankar writes:
>
>  > > project.  Nothing like it's impossible, stupid, or YAGNI.  But Guido,
>  > > and other senior developers, are saying they're not going to devote
>  > > their resources to it as things currently stand (and one of those
>  > > resources the attention of the folks who review PEPs).
>  > 
>  > <KS>
>  >     I am not sure that is true. I think if we have a well thought out 
>  > PEP that addresses parallelism,
>
> ... and provides a plausible proof-of-concept implementation [Guido
> said that to you explicitly] ...
>
>  > it would be looked into by the folks.
>
> True.  But AIUI without the implementation, it won't become a PEP.
> And at present you don't have a plausible implementation with the GIL.
>
> Now, it looks like Adam has a plausible implementation of removing the
> GIL.  If that holds up under at least some common use-cases, I think
> you'll see enthusiasm from some of the top developers, and acceptance
> from most.  But I really think that has to come first.
>
>
>