[Python-Dev] Exploration PEP : Concurrency for moderately massive (4 to 32 cores) multi-core architectures

Krishna Sankar ksankar at doubleclix.net
Wed Sep 19 23:00:48 CEST 2007


    a)   Yep, SMP for now. Agreed on the need for asymmetric 
architectures like cell-processor. We need to start somewhere and then 
can extend to more exotic realms.
    b)   Yep, need to scale to arbitrary number of cores. But as a 
start, I wanted to differentiate from massive parallelism.
    c)   Yep, we can have message passing semantics at the interface 
level and then underneath share the memory (even optimize with the copy 
on write patter). I was thinking that we would need to cross process 
space; for example federate 8 separate py processes (in an 8 core 
machine) and have a shared data path between them, based on shared 
memory allocated at configuration time.

Steven H. Rogers wrote:
> Krishna Sankar wrote:
>> Folks,
>>    As a follow-up to the py3k discussions started by Bruce and Guido, I 
>> pinged Brett and he suggested I submit an exploratory proposal. Would 
>> appreciate insights, wisdom, the good, the bad and the ugly.
>> A)    Does it make sense ?
>> B)    Which application sets should we consider in designing the 
>> interfaces and implementations
>> C)    In this proposal, parallelism and concurrency are used in an 
>> interchangeable fashion. Thoughts ?
>> D)    Please suggest pertinent links, discussions and insights.
>> E)    I have kept the proposal to a minimum to start the discussions and 
>> to explore if this is the right thing to do. Collaboratively, as we 
>> zero-in on one or two approaches, the idea is to expand it to a crisp 
>> and clear PEP. Need to do some more formatting as well.
>> Cheers
>> <k/>
>> P.S : I had sent this to python-ideas couple of days ago and received 
>> two comments (Thanks Leonardo, Thanks Adam) I haven't incorporated their 
>> comments yet. Folks who are on both lists, pardon me for the spam.
> # Proto-PEP elided.
> Other than number of cores, you don't mention hardware architecture.  I 
> presume that you're thinking of symmetric multiprocessor architectures.  
> If so, this should be explicit.  You should also consider that SMP may 
> not be the predominant multi-core architecture in the future, the Cell 
> processor has one general purpose processor and eight more specialized 
> processors.  You might not want to limit the PEP to 32 cores, I know of 
> startups working on 40 and 64 core chips.
> Shared memory may be necessary for good performance, but it doesn't have 
> to be exposed at the language level.  While Erlang has strictly message 
> passing semantics, I believe that it uses shared memory in the low level 
> implementation.
> # Steve
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/ksankar%40doubleclix.net

More information about the Python-Dev mailing list