[pypy-dev] support for 64-bit processors and eliminating global state

[Gabriel Lavoie]

Hello Maciej,
    actually you have a 64 bits buildbot, mine. It's under FreeBSD but
it's still 64 bits.

Cheers,

Gabriel

2009/9/30 Maciej Fijalkowski <fijall at gmail.com>:
> Hi Leonardo. I think you're not reading this mail in details, let me explain.
>
> On Wed, Sep 30, 2009 at 11:16 AM, Leonardo Santagada
> <santagada at gmail.com> wrote:
>>
>> On Sep 30, 2009, at 1:27 PM, Jeff Anderson-Lee wrote:
>>
>>> I'm new to pypy but would encourage the development folks to apply
>>> some
>>> focus towards two things: support for both 32 and 64-bit processors
>>> and
>>> eliminating global state including the GIL.
>>>
>>> The near future of mainstream processors is multi-core x86_64.  For
>>> the
>>> short-term both 32-bit and 64-bit platforms will be around.  Code that
>>> makes "naked" assumptions about word size will break and needs to be
>>> re-factored to hide the word-size dependencies.   Similarly code that
>>> assumes a single thread of execution or uses a GIL to protect global
>>> state will make efficient use of modern processors.  Any language or
>>> system that cannot make the transition to 64-bit multi-core will start
>>> to loose ground to those that do.
>>
>> PyPy does support 32 and 64 bit processors, the jit for x86_64 is not
>> ready though but this is just a problem of time, when the 32 bit jit
>> is ready doing a 64bit one is simple (but many manhours of work).
>
> It's not that many hours needed to have 64bit JIT as far as I know.
> I did a lot of refactoring recently so it should be much easier.
> Also we don't have a 64bit buildbot, which means 64bit support might
> rot over time,
> we don't know and it's not officially supported.
>
>>
>> The GIL in pypy is only there because no one proposed anything to
>> change that, pypy already does not depend on reference counting but
>> can use a garbage collector so it is probably way easier to change
>> than CPython.
>
> It's true that we don't have a good story here and we need one. Something
> a'la Jython would work (unlike in CPython), but it's work.
>
>>
>>
>>> At the Parallel Computing Laboratory (UC Berkeley) one of the projects
>>> we are working on is called SEJITS which stands for Selective Embedded
>>> Just in Time Specialization.  The idea is that one can extend a
>>> self-introspecting modern scripting language for calling native coded
>>> modules (e.g. C) at selected points for handling specialized
>>> operations
>>> (e.g. vector/matrix operations using tuned SIMD or CUDA code).  You
>>> can
>>> see the abstract of a recent SEJITS paper at
>>> http://pmea.ac.upc.edu/program.html (session 1a) but unfortunately the
>>> paper is not online yet.
>>>
>>> Both Python and Ruby are being looked at as potential target languages
>>> for SEJITS work.  Both have sufficient introspection facilities to
>>> support selective JIT operations.  Python has an advantage in having
>>> been used by the scientific community for longer than Ruby with more
>>> established users.  I'd love to see this work integrate with pypy.  At
>>> the moment the folks involved are targeting CPython.
>>> In any case, I think the transition to multi-core/multi-threaded 64-
>>> bit
>>> machines is a potential watershed of major importance which it would
>>> behoove pypy-dev folks to keep in mind.
>>
>>
>> I haven't read the paper but pypy does already have a JIT, maybe if
>> you are interested in it you can read more on the pypy blog http://morepypy.blogspot.com/
>> . Probably someone with more experience with both pypy and the JIT is
>> going to answer this email so I will not try to explain it in here.
>>
>
> Note that's not precisely what Jeff wants. General purpose JIT is nice, but
> it's rather hard to imagine how it'll generate efficient CUDA code
> automatically,
> without hints from the user. Since PyPy actually has a jit-generator, it should
> be far easier to implement this in PyPy than somewhere else (you can write
> code that is "interpreter" and JIT will be automatically created for
> it), however
> it's still work to get nice paralellizable (or parallelizing?) framework.
>
> Cheers,
> fijal
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev



-- 
Gabriel Lavoie
glavoie at gmail.com