[Python-Dev] Exploration PEP : Concurrency for moderately massive (4 to 32 cores) multi-core architectures
ksankar at doubleclix.net
Wed Sep 19 23:00:48 CEST 2007
a) Yep, SMP for now. Agreed on the need for asymmetric
architectures like cell-processor. We need to start somewhere and then
can extend to more exotic realms.
b) Yep, need to scale to arbitrary number of cores. But as a
start, I wanted to differentiate from massive parallelism.
c) Yep, we can have message passing semantics at the interface
level and then underneath share the memory (even optimize with the copy
on write patter). I was thinking that we would need to cross process
space; for example federate 8 separate py processes (in an 8 core
machine) and have a shared data path between them, based on shared
memory allocated at configuration time.
Steven H. Rogers wrote:
> Krishna Sankar wrote:
>> As a follow-up to the py3k discussions started by Bruce and Guido, I
>> pinged Brett and he suggested I submit an exploratory proposal. Would
>> appreciate insights, wisdom, the good, the bad and the ugly.
>> A) Does it make sense ?
>> B) Which application sets should we consider in designing the
>> interfaces and implementations
>> C) In this proposal, parallelism and concurrency are used in an
>> interchangeable fashion. Thoughts ?
>> D) Please suggest pertinent links, discussions and insights.
>> E) I have kept the proposal to a minimum to start the discussions and
>> to explore if this is the right thing to do. Collaboratively, as we
>> zero-in on one or two approaches, the idea is to expand it to a crisp
>> and clear PEP. Need to do some more formatting as well.
>> P.S : I had sent this to python-ideas couple of days ago and received
>> two comments (Thanks Leonardo, Thanks Adam) I haven't incorporated their
>> comments yet. Folks who are on both lists, pardon me for the spam.
> # Proto-PEP elided.
> Other than number of cores, you don't mention hardware architecture. I
> presume that you're thinking of symmetric multiprocessor architectures.
> If so, this should be explicit. You should also consider that SMP may
> not be the predominant multi-core architecture in the future, the Cell
> processor has one general purpose processor and eight more specialized
> processors. You might not want to limit the PEP to 32 cores, I know of
> startups working on 40 and 64 core chips.
> Shared memory may be necessary for good performance, but it doesn't have
> to be exposed at the language level. While Erlang has strictly message
> passing semantics, I believe that it uses shared memory in the low level
> # Steve
> Python-Dev mailing list
> Python-Dev at python.org
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/ksankar%40doubleclix.net
More information about the Python-Dev