[pypy-dev] Thinking about the GIL

Bob Ippolito bob at redivi.com
Wed Mar 16 18:30:29 CET 2011


On Wed, Mar 16, 2011 at 9:13 AM, Timothy Baldridge <tbaldridge at gmail.com> wrote:
>> one alternative approach is to have a separate VM in each thread.  Then pass
>> messages between them.  Works well, and no GIL in each VM.  You have to have
>> clean code that allows you to have a separate VMs in a process.  However,
>> it's easier to make your code be able to run in separate VMs, than to recode
>> it to allow concurrent thread access to all data structures.  This way is
>> easier to implement than removing a GIL.
>>
>
> This is what Erlang does, but unfortunately it scales very poorly when
>  you have large datasets (as mentioned above). If you want 4 threads
> pounding on 1GB of data, either they need to ask a 5th thread for each
> data item, or each thread needs its own copy of that 1GB dataset. Even
> embarrassingly parallel things such as raytracing are very hard in
> this model.

If the data is constant you can share it in Erlang by generating a
module for it and putting it in the code server (a hack, but it works)
or simply using a binary. These are zero-copy in Erlang.

-bob



More information about the Pypy-dev mailing list