[pypy-dev] array performace?

[Maciej Fijalkowski]

On Fri, Jul 2, 2010 at 2:26 AM,  <Ben.Young at sungard.com> wrote:
>> On Fri, Jul 2, 2010 at 1:47 AM, Paolo Giarrusso
> <p.giarrusso at gmail.com>
>> wrote:
>> > On Fri, Jul 2, 2010 at 08:04, Maciej Fijalkowski <fijall at gmail.com>
> wrote:
>> >> On Thu, Jul 1, 2010 at 1:18 PM, Hakan Ardo <Hakan at ardoe.net> wrote:
>> >>> OK, so making an interpreter level implementation of array.array
> seams
>> >>> like a good idea. Would it be possible to get the jit to remove
> the
>> >>> wrapping/unwrapping in that case to get better performance than
>> >>> _rawffi.Array('d'), which is already an interpreter level
>> >>> implementation?
>> >>
>> >> it should work mostly out of the box (you can also try this for
>> >> _rawffi.array part of module, if you want to). It's probably enough
> to
>> >> enable module in pypy/module/pypyjit/policy.py so JIT can have a
> look
>> >> there. In case of _rawffi, probably a couple of hints for the jit
> to
>> >> not look inside some functions (which do external calls for
> example)
>> >> should also be needed, since for example JIT as of now does not
>> >> support raw mallocs (using C malloc and not our GC).
>> >
>> >> Still, making an
>> >> array module interp-level is probably the sanest approach.
>> >
>> > That might be a bad sign.
>> > For CPython, people recommend to write extensions in C for
>> > performance, i.e. to make them less maintainable and understandable
>> > for performance.
>> > A good JIT should make this unnecessary in as many cases as
> possible.
>> > Of course, the array module might be an exception, if it's a single
>> > case.
>> > But performance 20x slower than C, with a JIT, is a big warning,
> since
>> > fast interpreters are documented to be (in general) just 10x slower
>> > than C.
>>
>> There is a lot of unsupported claims in your sentences, however,
>> that's not my point.
>>
>
> That's a little harsh. When the JIT was originally developed it was
> envisaged that it would be faster to re-write code to app level to give
> speed-ups. If that's changed that's fine, but it's not an "unsupported
> claim"
>
> Ben
>

Unsupported claim is for example that fast interpreters are 10x slower
than C. On what exactly? Did he write this particular benchmark in C
and in fast interpreter to compare? Another unsupported claim is that
JIT is 20x slower than C here. Array module is not even JITted,
because it's based on _rawffi which itself operates on low-level
pointers which JIT does not want to deal with. That's exactly the
reason why JIT doesn't look into _rawffi module and making it look
there doesn't sound like a good idea (instead, we're trying to replace
it with something JIT-friendly that knows how to do FFI calls into C,
there is a summer of code project).

All I'm trying to say is that there are valid reasons that array
module should be on interpreter level and none of this has anything to
do with incapabilities of the JIT.

Cheers,
fijal