[BangPypers] Ideas for Python concurrency...

Vishal vsapre80 at gmail.com
Wed Feb 9 09:46:24 CET 2011


On Wed, Feb 9, 2011 at 11:04 AM, Dhananjay Nene <dhananjay.nene at gmail.com>wrote:

> On Tue, Feb 8, 2011 at 2:15 PM, Vishal <vsapre80 at gmail.com> wrote:
>
> > On Tue, Feb 8, 2011 at 2:10 PM, Baishampayan Ghose <b.ghose at gmail.com
> > >wrote:
> >
> > > > Mutliprocessing means, data copying, talking to each other through
> > PIPES,
> > > > also it has its issues with running on Windows (all function calls
> > should
> > > be pickelable)
> > > >
> > > > Threads seems pretty stable on most platforms where Python runs.
> > >
> > > Threads won't help you much because of the Python GIL.
> > >
> > > Regards,
> > > BG
> > >
> > > --
> > > Baishampayan Ghose
> > > b.ghose at gmail.com
> > > '
> > >
> >
> > The GIL is only limited to one interpreter. GIL comes into picture
> because
> > Threads in python create a new interpreter context, so the same
> interpreter
> > switches to new byte codes. The original idea was to create a new
> > interpreter for each thread, each one with its own GIL !!!. All within
> the
> > same process.
> >
>
> Interesting. But that does raise some curious issues of sharing data across
> interpreters.
> So as per the original thought - would threads be able to access data
> across
> multiple
> interpreters (to implement shared data), or would they have completely
> isolated
> data islands via message passing ?
>
> Dhananjay
>
>
I had forwarded the same question to comp.lang.python. here are the replies
till date:
http://groups.google.com/group/comp.lang.python/browse_thread/thread/dd3938282b64c6ac/4a9d53c02e30cc3b#4a9d53c02e30cc3b

<http://groups.google.com/group/comp.lang.python/browse_thread/thread/dd3938282b64c6ac/4a9d53c02e30cc3b#4a9d53c02e30cc3b>People
have pointed out to the need of very fine grained locking that would be
necessary to make sure data sharing is sane, between the interpreters.
However, reference counting comes in the way of such a fine grained locking.
People have also pointed towards PyPy which does away with reference
counting.

I like your comment about having an internal message passing scheme among
the interpreters, like what processor cores do, when they have to share data
through L2 caches.

Best regards,
Vishal Sapre.


"May we do good and not evil. May we find forgiveness for ourself and
forgive others. May we share freely, never taking more than we give."


More information about the BangPypers mailing list