[pypy-dev] array performace?

Alex Gaynor alex.gaynor at gmail.com
Fri Jul 2 07:40:21 CEST 2010


On Fri, Jul 2, 2010 at 12:24 AM, Hakan Ardo <hakan at debian.org> wrote:
> OK, so making an interpreter level implementation of array.array seams
> like a good idea. Would it be possible to get the jit to remove the
> wrapping/unwrapping in that case to get better performance than
> _rawffi.Array('d'), which is already an interpreter level
> implementation?
>
> Are there some docs to get me started at writing interpreter level
> objects? I've had a look at _rawffi/array.py and am a bit confused
> about the W_Array.typedef = TypeDef('Array',...)  construction. Maybe
> there is a easier example to start with?
>
> On Thu, Jul 1, 2010 at 5:40 PM, Alex Gaynor <alex.gaynor at gmail.com> wrote:
>> On Thu, Jul 1, 2010 at 10:35 AM, Maciej Fijalkowski <fijall at gmail.com> wrote:
>>> On Thu, Jul 1, 2010 at 9:28 AM, Armin Rigo <arigo at tunes.org> wrote:
>>>> Hi,
>>>>
>>>> On Thu, Jul 01, 2010 at 04:02:30PM +0200, Hakan Ardo wrote:
>>>>> are there any python construct that the jit will be able to compile
>>>>> into c-type array accesses? Consider the following test:
>>>>>
>>>>>     l=0.0
>>>>>     for i in xrange(640,640*480):
>>>>>         l+=img[i]
>>>>>         intimg[i]=intimg[i-640]+l
>>>>
>>>> This is still implemented as a list of Python objects (as expected,
>>>> because the JIT cannot prove that we won't suddenly try to put something
>>>> else than a float in the same list).
>>>>
>>>> Using _rawffi.Array('d') directly is the best option right now.  I'm not
>>>> sure why the array.array module is 400 times slower, but it's definitely
>>>> slower given that it's implemented at app-level using a _rawffi.Array('c')
>>>> and doing the conversion by itself (for some partially stupid reasons like
>>>> doing the right kind of error checking).
>>>>
>>>>
>>>> A bientot,
>>>>
>>>> Armin.
>>>
>>> The main reason why _rawffi.Array is slow is that JIT does not look
>>> into that module, so there is wrapping and unwrapping going on.
>>> Relatively easy to fix I suppose, but _rawffi.Array was not meant to
>>> be used like that (array.array looks like a better candidate).
>>> _______________________________________________
>>> pypy-dev at codespeak.net
>>> http://codespeak.net/mailman/listinfo/pypy-dev
>>
>> If array.array performance is important to your work, the array.py
>> module looks like a good target for writing at interp level, and it's
>> not too much code.
>>
>> Alex
>>
>> --
>> "I disapprove of what you say, but I will defend to the death your
>> right to say it." -- Voltaire
>> "The people's good is the highest law." -- Cicero
>> "Code can always be simpler than you think, but never as simple as you
>> want" -- Me
>> _______________________________________________
>> pypy-dev at codespeak.net
>> http://codespeak.net/mailman/listinfo/pypy-dev
>
>
>
> --
> Håkan Ardö
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev
>

I'd take a look at the cStringIO module, it's a decent example of the
APIs (and not too much code).  FWIW one thing to note is that array
uses the struct module, which is also pure python.  I believe it's
possible to still use that with an interp-level module, but it may
just become another bottle neck, just something to consider.

Alex

-- 
"I disapprove of what you say, but I will defend to the death your
right to say it." -- Voltaire
"The people's good is the highest law." -- Cicero
"Code can always be simpler than you think, but never as simple as you
want" -- Me



More information about the Pypy-dev mailing list