[pypy-dev] array performace?

Maciej Fijalkowski fijall at gmail.com
Fri Jul 2 08:04:26 CEST 2010

On Thu, Jul 1, 2010 at 1:18 PM, Hakan Ardo <Hakan at ardoe.net> wrote:
> OK, so making an interpreter level implementation of array.array seams
> like a good idea. Would it be possible to get the jit to remove the
> wrapping/unwrapping in that case to get better performance than
> _rawffi.Array('d'), which is already an interpreter level
> implementation?

it should work mostly out of the box (you can also try this for
_rawffi.array part of module, if you want to). It's probably enough to
enable module in pypy/module/pypyjit/policy.py so JIT can have a look
there. In case of _rawffi, probably a couple of hints for the jit to
not look inside some functions (which do external calls for example)
should also be needed, since for example JIT as of now does not
support raw mallocs (using C malloc and not our GC). Still, making an
array module interp-level is probably the sanest approach.

> Are there some docs to get me started at writing interpreter level
> objects? I've had a look at _rawffi/array.py and am a bit confused
> about the W_Array.typedef = TypeDef('Array',...)  construction. Maybe
> there is a easier example to start with?

TypeDef is a way to expose interpreter level (RPython) object to
app-level (Python). It tells what methods there are what properties
and what attributes.

> On Thu, Jul 1, 2010 at 5:40 PM, Alex Gaynor <alex.gaynor at gmail.com> wrote:
>> On Thu, Jul 1, 2010 at 10:35 AM, Maciej Fijalkowski <fijall at gmail.com> wrote:
>>> On Thu, Jul 1, 2010 at 9:28 AM, Armin Rigo <arigo at tunes.org> wrote:
>>>> Hi,
>>>> On Thu, Jul 01, 2010 at 04:02:30PM +0200, Hakan Ardo wrote:
>>>>> are there any python construct that the jit will be able to compile
>>>>> into c-type array accesses? Consider the following test:
>>>>>     l=0.0
>>>>>     for i in xrange(640,640*480):
>>>>>         l+=img[i]
>>>>>         intimg[i]=intimg[i-640]+l
>>>> This is still implemented as a list of Python objects (as expected,
>>>> because the JIT cannot prove that we won't suddenly try to put something
>>>> else than a float in the same list).
>>>> Using _rawffi.Array('d') directly is the best option right now.  I'm not
>>>> sure why the array.array module is 400 times slower, but it's definitely
>>>> slower given that it's implemented at app-level using a _rawffi.Array('c')
>>>> and doing the conversion by itself (for some partially stupid reasons like
>>>> doing the right kind of error checking).
>>>> A bientot,
>>>> Armin.
>>> The main reason why _rawffi.Array is slow is that JIT does not look
>>> into that module, so there is wrapping and unwrapping going on.
>>> Relatively easy to fix I suppose, but _rawffi.Array was not meant to
>>> be used like that (array.array looks like a better candidate).
>>> _______________________________________________
>>> pypy-dev at codespeak.net
>>> http://codespeak.net/mailman/listinfo/pypy-dev
>> If array.array performance is important to your work, the array.py
>> module looks like a good target for writing at interp level, and it's
>> not too much code.
>> Alex
>> --
>> "I disapprove of what you say, but I will defend to the death your
>> right to say it." -- Voltaire
>> "The people's good is the highest law." -- Cicero
>> "Code can always be simpler than you think, but never as simple as you
>> want" -- Me
>> _______________________________________________
>> pypy-dev at codespeak.net
>> http://codespeak.net/mailman/listinfo/pypy-dev
> --
> Håkan Ardö

More information about the Pypy-dev mailing list