[pypy-dev] support for 64-bit processors and eliminating global state
fijall at gmail.com
Wed Sep 30 19:22:24 CEST 2009
Hi Leonardo. I think you're not reading this mail in details, let me explain.
On Wed, Sep 30, 2009 at 11:16 AM, Leonardo Santagada
<santagada at gmail.com> wrote:
> On Sep 30, 2009, at 1:27 PM, Jeff Anderson-Lee wrote:
>> I'm new to pypy but would encourage the development folks to apply
>> focus towards two things: support for both 32 and 64-bit processors
>> eliminating global state including the GIL.
>> The near future of mainstream processors is multi-core x86_64. For
>> short-term both 32-bit and 64-bit platforms will be around. Code that
>> makes "naked" assumptions about word size will break and needs to be
>> re-factored to hide the word-size dependencies. Similarly code that
>> assumes a single thread of execution or uses a GIL to protect global
>> state will make efficient use of modern processors. Any language or
>> system that cannot make the transition to 64-bit multi-core will start
>> to loose ground to those that do.
> PyPy does support 32 and 64 bit processors, the jit for x86_64 is not
> ready though but this is just a problem of time, when the 32 bit jit
> is ready doing a 64bit one is simple (but many manhours of work).
It's not that many hours needed to have 64bit JIT as far as I know.
I did a lot of refactoring recently so it should be much easier.
Also we don't have a 64bit buildbot, which means 64bit support might
rot over time,
we don't know and it's not officially supported.
> The GIL in pypy is only there because no one proposed anything to
> change that, pypy already does not depend on reference counting but
> can use a garbage collector so it is probably way easier to change
> than CPython.
It's true that we don't have a good story here and we need one. Something
a'la Jython would work (unlike in CPython), but it's work.
>> At the Parallel Computing Laboratory (UC Berkeley) one of the projects
>> we are working on is called SEJITS which stands for Selective Embedded
>> Just in Time Specialization. The idea is that one can extend a
>> self-introspecting modern scripting language for calling native coded
>> modules (e.g. C) at selected points for handling specialized
>> (e.g. vector/matrix operations using tuned SIMD or CUDA code). You
>> see the abstract of a recent SEJITS paper at
>> http://pmea.ac.upc.edu/program.html (session 1a) but unfortunately the
>> paper is not online yet.
>> Both Python and Ruby are being looked at as potential target languages
>> for SEJITS work. Both have sufficient introspection facilities to
>> support selective JIT operations. Python has an advantage in having
>> been used by the scientific community for longer than Ruby with more
>> established users. I'd love to see this work integrate with pypy. At
>> the moment the folks involved are targeting CPython.
>> In any case, I think the transition to multi-core/multi-threaded 64-
>> machines is a potential watershed of major importance which it would
>> behoove pypy-dev folks to keep in mind.
> I haven't read the paper but pypy does already have a JIT, maybe if
> you are interested in it you can read more on the pypy blog http://morepypy.blogspot.com/
> . Probably someone with more experience with both pypy and the JIT is
> going to answer this email so I will not try to explain it in here.
Note that's not precisely what Jeff wants. General purpose JIT is nice, but
it's rather hard to imagine how it'll generate efficient CUDA code
without hints from the user. Since PyPy actually has a jit-generator, it should
be far easier to implement this in PyPy than somewhere else (you can write
code that is "interpreter" and JIT will be automatically created for
it's still work to get nice paralellizable (or parallelizing?) framework.
More information about the Pypy-dev