Exploration PEP : Concurrency for moderately massive (4 to 32 cores) multi-core architectures
Folks, As a follow-up to the py3k discussions started by Bruce and Guido, I pinged Brett and he suggested I submit an exploratory proposal. Would appreciate insights, wisdom, the good, the bad and the ugly.
A) Does it make sense ? B) Which application sets should we consider in designing the interfaces and implementations C) In this proposal, parallelism and concurrency are used in an interchangeable fashion. Thoughts ? D) Please suggest pertinent links, discussions and insights. E) I have kept the proposal to a minimum to start the discussions and to explore if this is the right thing to do. Collaboratively, as we zero-in on one or two approaches, the idea is to expand it to a crisp and clear PEP. Need to do some more formatting as well. Cheers <k/> P.S : I had sent this to python-ideas couple of days ago and received two comments (Thanks Leonardo, Thanks Adam) I haven't incorporated their comments yet. Folks who are on both lists, pardon me for the spam. ------------------------------------------------------------------------------------------------------------
PEP: xxxxxxxx Title: Concurrency for moderately massive (4 to 32 cores) multi-core architectures Version: $Revision$ Last-Modified: $Date$ Author: Krishna Sankar <ksankar (at) doubleclix.net>, Status: Wandering ! (as in "Not all those who wander are lost ..." -J.R.R.Tolkien) Type: Process Content-Type: text/x-rst Created: 15-Sep-2007
Abstract -------- This proposal aims at leveraging the multi-core capability as an embedded mechanism in python. It is not whether python is slow or fast, but of performance and control of parallelism/concurrency in a moderately massive parallelism world. The aim is 4 to 32 cores. The proposal advocates two mechanisms - one for task parallelism and another for data intensive parallelism. Scientific computing and web 2.0 frameworks are the forefront users for this proposal. Other applications would benefit as well.
Rationale --------- Multicore architectures need no introductions and their ubiquity is evident. It is imperative that Python has one or more standard ways of leveraging multi-core architectures. OTOH, traditional thread based concurrency and lock based exclusions are becoming more and more difficult to program correctly.
First of all, the question is not whether py is slow or fast but performance of a system written in py. Which means, ability to leverage multi-core architectures as well as control. Control in term of things like ability to pin one process/task to a core, ability to pin one or more homogeneous tasks to specific cores et al, as well as not wait for a global lock and similar primitives. (Before anybody jumps into a conclusion, this is not about GIL by any means ;o))
Second, it is clear that we need a good solution (not THE solution) for moderately massive parallelism in multi-core architectures (i.e. 8-32 cores). Share nothing might not be optimal; we need some form of memory sharing, not just copy all data via messages. May be functional programming based on the blackboard pattern would work, who knows.
I have seen systems saturated still having only ~25% of CPU utilization (in a 4 core system!). It is because we didn't leverage multi-cores and parallelism. So while py3k will not be slow, lack of a cohesive multi-core strategy will show up in system performance and byte us later(pun intended!).
At least, in my mind, this is not an exercise about exposing locks and mutexes or threads in Python. I do believe that the GIL will be refactored to more granularity in the coming months (similar to the Global Locks in Linux) and most probably we will get microThreads et al. As we all know, architecture is constraining as well as liberating. The language primitives influence greatly how we think about a problem.
In the discussions, Guido is right in insisting on speed, and Bruce is right in asking for language constructs. Without pragmatic speed, folks won't use it; same is the case without the required constructs. Both are barriers to adoption. We have an opportunity to offer a solution for multi-core architectures and let us seize it - we will rush in where angels fear to tread!
Programming Models ------------------ There are at least 3 possible paradigms
A. conventional threading model B. Functional model, Erlang being the most appropriate C. Some form of limited shared memory model (message passing but pass pointers, blackboard model) D. Others, like Transactional Memory [2]
There is enough literature out there, so do not plan to explain these here. (<KS> Do we need more explanation? </KS>)
Pragmatic proposal ------------------ May I suggest we embed two primitives in Python 3K: A) A functional style share-nothing set of interfaces (and implementations thereof) - provides the task parallelism/concurrency capability, "small messages, big computations" as Joe Armstrong calls it[3] B) A limited shared memory based model for data intensive parallelism
Most probably this would be part of stdlib. While Guido is almost right in saying that this is a (std)library problem, it is not fully so. We would need a few primitives from the underlying PVM substrate. Possibly one reason for Guido's position is the lack of clarity as to what needs to be changed and why. IMHO, just saying take GIL off does not solve the problem either.
The Zen of Python parallelism ----------------------------- I draw inspiration for the very timely article by James Reinders in DDJ [1]. It embodies what we should be doing viz.: 1. Refactor the problem into parallel tasks. We cannot help if the domain is sequential 2. Program to abstraction & program chores not cores. Writing correct program using raw threads et al is difficult. Let the underlying substrate decide how best to optimize 3. Design for scale 4. Have an option to turn concurrency off, for debugging 5. Declarative parallelism based mechanisms (?)
Related Efforts --------------- The good news is there are at least 2 or 3 paradigms with implementations and rough benchmarks. Parallel python http://www.artima.com/weblogs/viewpost.jsp?thread=214303 http://cheeseshop.python.org/pypi/parallel Processing http://cheeseshop.python.org/pypi/processing http://code.google.com/p/papyros/
Discussions ----------- There are at least four thread sets (pardon the pun !) I am aware of: 1. The GIL discussions in python-dev and Guido's blog on GIL http://www.artima.com/weblogs/viewpost.jsp?thread=214235 2. The py3k topics started by Bruce http://www.artima.com/weblogs/viewpost.jsp?thread=214112, response by Guide http://www.artima.com/weblogs/viewpost.jsp?thread=214325 and reply to reply by Bruce http://www.artima.com/weblogs/viewpost.jsp?thread=214480 3. Python and concurrency http://mail.python.org/pipermail/python-ideas/2007-March/000338.html 4. Adam's reply in python-ideas http://mail.python.org/pipermail/python-ideas/2007-September/000972.html
References [1]http://www.ddj.com/architect/201804248 [2]Transaction http://acmqueue.com/modules.php?name=Content&pa=showpage&pid=444 [3]Programming Erlang by Joe Armstrong
On 9/18/07, Krishna Sankar ksankar@doubleclix.net wrote:
Folks, As a follow-up to the py3k discussions started by Bruce and Guido, I pinged Brett and he suggested I submit an exploratory proposal. Would appreciate insights, wisdom, the good, the bad and the ugly.
A) Does it make sense ? B) Which application sets should we consider in designing the interfaces and implementations C) In this proposal, parallelism and concurrency are used in an interchangeable fashion. Thoughts ? D) Please suggest pertinent links, discussions and insights. E) I have kept the proposal to a minimum to start the discussions and to explore if this is the right thing to do. Collaboratively, as we zero-in on one or two approaches, the idea is to expand it to a crisp and clear PEP. Need to do some more formatting as well.
I'd say it is a little light on specific proposals. The only section that actually proposes anything is this:
Pragmatic proposal
May I suggest we embed two primitives in Python 3K: A) A functional style share-nothing set of interfaces (and implementations thereof) - provides the task parallelism/concurrency capability, "small messages, big computations" as Joe Armstrong calls it[3] B) A limited shared memory based model for data intensive parallelism
Most probably this would be part of stdlib. While Guido is almost right in saying that this is a (std)library problem, it is not fully so. We would need a few primitives from the underlying PVM substrate. Possibly one reason for Guido's position is the lack of clarity as to what needs to be changed and why. IMHO, just saying take GIL off does not solve the problem either.
Before I can meaningfully comment I think I'd like to hear more about what specifically you are thinking of.
I don't mind the necessary changes to the PVM. I do like to see how this affects existing C extensions though.
Guido, The vagueness is deliberate, to keep the options open until we have some form o convergence. Parallelism/concurrency is a vast and important domain that I do not want to develop a hasty proposal. But I did want to start thinking in terms of concrete proposals, not pontifying, hence the "pragmatic" section. Happy to hear that you are open to PVM changes. It will not be asked unless and until we all are crisp about it. Cheers <k/>
Guido van Rossum wrote:
On 9/18/07, Krishna Sankar ksankar@doubleclix.net wrote:
Folks, As a follow-up to the py3k discussions started by Bruce and Guido, I pinged Brett and he suggested I submit an exploratory proposal. Would appreciate insights, wisdom, the good, the bad and the ugly.
A) Does it make sense ? B) Which application sets should we consider in designing the interfaces and implementations C) In this proposal, parallelism and concurrency are used in an interchangeable fashion. Thoughts ? D) Please suggest pertinent links, discussions and insights. E) I have kept the proposal to a minimum to start the discussions and to explore if this is the right thing to do. Collaboratively, as we zero-in on one or two approaches, the idea is to expand it to a crisp and clear PEP. Need to do some more formatting as well.
I'd say it is a little light on specific proposals. The only section that actually proposes anything is this:
Pragmatic proposal
May I suggest we embed two primitives in Python 3K: A) A functional style share-nothing set of interfaces (and implementations thereof) - provides the task parallelism/concurrency capability, "small messages, big computations" as Joe Armstrong calls it[3] B) A limited shared memory based model for data intensive parallelism
Most probably this would be part of stdlib. While Guido is almost right in saying that this is a (std)library problem, it is not fully so. We would need a few primitives from the underlying PVM substrate. Possibly one reason for Guido's position is the lack of clarity as to what needs to be changed and why. IMHO, just saying take GIL off does not solve the problem either.
Before I can meaningfully comment I think I'd like to hear more about what specifically you are thinking of.
I don't mind the necessary changes to the PVM. I do like to see how this affects existing C extensions though.
On 9/18/07, Krishna Sankar ksankar@doubleclix.net wrote:
The vagueness is deliberate, to keep the options open until we havesome form o convergence. Parallelism/concurrency is a vast and important domain that I do not want to develop a hasty proposal. But I did want to start thinking in terms of concrete proposals, not pontifying, hence the "pragmatic" section.
As long as it's this vague it doesn't deserve to be called a PEP though. PEPs can't be vague, they must make specific proposals. As long as this is intentionally half-baked it belongs back in python-ideas and there's no point in pretending to be writing a "PEP".
Happy to hear that you are open to PVM changes. It will not be askedunless and until we all are crisp about it. Cheers
<k/>
Guido van Rossum wrote:
On 9/18/07, Krishna Sankar ksankar@doubleclix.net wrote:
Folks, As a follow-up to the py3k discussions started by Bruce and Guido, I pinged Brett and he suggested I submit an exploratory proposal. Would appreciate insights, wisdom, the good, the bad and the ugly.
A) Does it make sense ? B) Which application sets should we consider in designing the interfaces and implementations C) In this proposal, parallelism and concurrency are used in an interchangeable fashion. Thoughts ? D) Please suggest pertinent links, discussions and insights. E) I have kept the proposal to a minimum to start the discussions and to explore if this is the right thing to do. Collaboratively, as we zero-in on one or two approaches, the idea is to expand it to a crisp and clear PEP. Need to do some more formatting as well.
I'd say it is a little light on specific proposals. The only section that actually proposes anything is this:
Pragmatic proposal
May I suggest we embed two primitives in Python 3K: A) A functional style share-nothing set of interfaces (and implementations thereof) - provides the task parallelism/concurrency capability, "small messages, big computations" as Joe Armstrong calls it[3] B) A limited shared memory based model for data intensive parallelism
Most probably this would be part of stdlib. While Guido is almost right in saying that this is a (std)library problem, it is not fully so. We would need a few primitives from the underlying PVM substrate. Possibly one reason for Guido's position is the lack of clarity as to what needs to be changed and why. IMHO, just saying take GIL off does not solve the problem either.
Before I can meaningfully comment I think I'd like to hear more about what specifically you are thinking of.
I don't mind the necessary changes to the PVM. I do like to see how this affects existing C extensions though.
Agreed it is not a PEP yet. Hence the word "Exploration" in front of it. This ia domain which needs some discussions before developing a good PEP. May be I should call it a PEPlet ;o) Cheers <k/> Guido van Rossum wrote:
On 9/18/07, Krishna Sankar ksankar@doubleclix.net wrote:
The vagueness is deliberate, to keep the options open until we havesome form o convergence. Parallelism/concurrency is a vast and important domain that I do not want to develop a hasty proposal. But I did want to start thinking in terms of concrete proposals, not pontifying, hence the "pragmatic" section.
As long as it's this vague it doesn't deserve to be called a PEP though. PEPs can't be vague, they must make specific proposals. As long as this is intentionally half-baked it belongs back in python-ideas and there's no point in pretending to be writing a "PEP".
Happy to hear that you are open to PVM changes. It will not be askedunless and until we all are crisp about it. Cheers
<k/>
Guido van Rossum wrote:
On 9/18/07, Krishna Sankar ksankar@doubleclix.net wrote:
Folks, As a follow-up to the py3k discussions started by Bruce and Guido, I pinged Brett and he suggested I submit an exploratory proposal. Would appreciate insights, wisdom, the good, the bad and the ugly.
A) Does it make sense ? B) Which application sets should we consider in designing the interfaces and implementations C) In this proposal, parallelism and concurrency are used in an interchangeable fashion. Thoughts ? D) Please suggest pertinent links, discussions and insights. E) I have kept the proposal to a minimum to start the discussions and to explore if this is the right thing to do. Collaboratively, as we zero-in on one or two approaches, the idea is to expand it to a crisp and clear PEP. Need to do some more formatting as well.
I'd say it is a little light on specific proposals. The only section that actually proposes anything is this:
Pragmatic proposal
May I suggest we embed two primitives in Python 3K: A) A functional style share-nothing set of interfaces (and implementations thereof) - provides the task parallelism/concurrency capability, "small messages, big computations" as Joe Armstrong calls it[3] B) A limited shared memory based model for data intensive parallelism
Most probably this would be part of stdlib. While Guido is almost right in saying that this is a (std)library problem, it is not fully so. We would need a few primitives from the underlying PVM substrate. Possibly one reason for Guido's position is the lack of clarity as to what needs to be changed and why. IMHO, just saying take GIL off does not solve the problem either.
Before I can meaningfully comment I think I'd like to hear more about what specifically you are thinking of.
I don't mind the necessary changes to the PVM. I do like to see how this affects existing C extensions though.
On Tue, Sep 18, 2007, Krishna Sankar wrote:
As a follow-up to the py3k discussions started by Bruce and Guido, I pinged Brett and he suggested I submit an exploratory proposal. Would appreciate insights, wisdom, the good, the bad and the ugly.
This should probably start in python-ideas.
On 9/18/07, Krishna Sankar ksankar@doubleclix.net wrote:
Folks, As a follow-up to the py3k discussions started by Bruce and Guido, I pinged Brett and he suggested I submit an exploratory proposal. Would appreciate insights, wisdom, the good, the bad and the ugly.
I am currently working on parallelizing python as an undergraduate independent study. I plan on first removing the GIL with as little overall effect as possible and then implementing a task-oriented threading API on top, probably based on Stackless (since they already do a great job with concurrency in a single thread).
If you're interested in all the details, I'd be happy to share. I haven't gotten far yet (the semester just started!), but I feel that actually implementing these things would be the best way to get a PEP through.
Justin
Justin, Yep, trying out an implementation is a good way. Please share your thoughts as and when you are ready. Cheers & good luck <k/> Justin Tulloss wrote:
On 9/18/07, *Krishna Sankar* <ksankar@doubleclix.net mailto:ksankar@doubleclix.net> wrote:
Folks, As a follow-up to the py3k discussions started by Bruce and Guido, I pinged Brett and he suggested I submit an exploratory proposal. Would appreciate insights, wisdom, the good, the bad and the ugly.I am currently working on parallelizing python as an undergraduate independent study. I plan on first removing the GIL with as little overall effect as possible and then implementing a task-oriented threading API on top, probably based on Stackless (since they already do a great job with concurrency in a single thread).
If you're interested in all the details, I'd be happy to share. I haven't gotten far yet (the semester just started!), but I feel that actually implementing these things would be the best way to get a PEP through.
Justin
Krishna Sankar wrote:
Folks, As a follow-up to the py3k discussions started by Bruce and Guido, I pinged Brett and he suggested I submit an exploratory proposal. Would appreciate insights, wisdom, the good, the bad and the ugly.
A) Does it make sense ? B) Which application sets should we consider in designing the interfaces and implementations C) In this proposal, parallelism and concurrency are used in an interchangeable fashion. Thoughts ? D) Please suggest pertinent links, discussions and insights. E) I have kept the proposal to a minimum to start the discussions and to explore if this is the right thing to do. Collaboratively, as we zero-in on one or two approaches, the idea is to expand it to a crisp and clear PEP. Need to do some more formatting as well. Cheers
<k/> P.S : I had sent this to python-ideas couple of days ago and received two comments (Thanks Leonardo, Thanks Adam) I haven't incorporated their comments yet. Folks who are on both lists, pardon me for the spam.
# Proto-PEP elided.
Other than number of cores, you don't mention hardware architecture. I presume that you're thinking of symmetric multiprocessor architectures. If so, this should be explicit. You should also consider that SMP may not be the predominant multi-core architecture in the future, the Cell processor has one general purpose processor and eight more specialized processors. You might not want to limit the PEP to 32 cores, I know of startups working on 40 and 64 core chips.
Shared memory may be necessary for good performance, but it doesn't have to be exposed at the language level. While Erlang has strictly message passing semantics, I believe that it uses shared memory in the low level implementation.
# Steve
Steve,
Thanks. a) Yep, SMP for now. Agreed on the need for asymmetric architectures like cell-processor. We need to start somewhere and then can extend to more exotic realms. b) Yep, need to scale to arbitrary number of cores. But as a start, I wanted to differentiate from massive parallelism. c) Yep, we can have message passing semantics at the interface level and then underneath share the memory (even optimize with the copy on write patter). I was thinking that we would need to cross process space; for example federate 8 separate py processes (in an 8 core machine) and have a shared data path between them, based on shared memory allocated at configuration time.
Cheers <k/> Steven H. Rogers wrote:
Krishna Sankar wrote:
Folks, As a follow-up to the py3k discussions started by Bruce and Guido, I pinged Brett and he suggested I submit an exploratory proposal. Would appreciate insights, wisdom, the good, the bad and the ugly.
A) Does it make sense ? B) Which application sets should we consider in designing the interfaces and implementations C) In this proposal, parallelism and concurrency are used in an interchangeable fashion. Thoughts ? D) Please suggest pertinent links, discussions and insights. E) I have kept the proposal to a minimum to start the discussions and to explore if this is the right thing to do. Collaboratively, as we zero-in on one or two approaches, the idea is to expand it to a crisp and clear PEP. Need to do some more formatting as well. Cheers
<k/> P.S : I had sent this to python-ideas couple of days ago and received two comments (Thanks Leonardo, Thanks Adam) I haven't incorporated their comments yet. Folks who are on both lists, pardon me for the spam.
# Proto-PEP elided.
Other than number of cores, you don't mention hardware architecture. I presume that you're thinking of symmetric multiprocessor architectures. If so, this should be explicit. You should also consider that SMP may not be the predominant multi-core architecture in the future, the Cell processor has one general purpose processor and eight more specialized processors. You might not want to limit the PEP to 32 cores, I know of startups working on 40 and 64 core chips.
Shared memory may be necessary for good performance, but it doesn't have to be exposed at the language level. While Erlang has strictly message passing semantics, I believe that it uses shared memory in the low level implementation.
# Steve
Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/ksankar%40doubleclix.net
-
Aahz -
Guido van Rossum -
Justin Tulloss -
Krishna Sankar -
Steven H. Rogers