[pypy-dev] cppyy use case question

[Alex Pyattaev]

I think I've figured it with the missing classes. When supplied with multiple 
headers, genreflex apparently only saves the last one of them...Obviously, 
that can be worked around, but still seems like a bug or just me not finding 
some manual.
BR,
Alex.
sunnuntai 03 kesäkuu 2012 16:44:19 wlavrijsen at lbl.gov kirjoitti:
> Hi Alex,
> 
> >> Performance ... is absolutely deplorable: slower than PyROOT on CPython!
> > 
> > Performance was originally the point, right? Or am I missing something
> > (lol)?
> well, remember that there are two problems here: 1) since there are no
> PyObject's as such in pypy-c (they have different internal structure), one
> has the create them, to have a PyObject* pointer to them; and keep track of
> them to mix the reference counting scheme with the garbage collection scheme
> (this is all done in/by cpyext). And 2) now that the object is created,
> pinning down a variable in the .py code, the JIT can't unbox it.
> 
> The example I used, which was like so:
> 
>        for i in range(N):
>            a.add_data(i)
> 
> and is rather pathological, of course. But then, all micro-benches are.
> 
> Conversely, doing:
> 
>        l = []
>        for i in range(N):
>            l.append(i)
>        a.add_data(l)
> 
> is very speedy, presumably (I don't know, guessing here as it may also be
> that there is another optimization for the list that I can't prevent even
> with the append() calls) because now the cpyext code is in a loop in and
> of itself, so e.g. lookup of the type of the objects in the container (all
> the same here) can become constant.
> 
> > I am giving you my benchmark code
> 
> Which clearly is also pathological. :) I can't manage to run the SWIG test
> from pypy, though? I tried by adding cpyext, but no dice?
> 
> Anyway, you'll be happy to know that:
> 
> CPPYY
> Created container
> No more items
> 
> real	0m2.236s
> user	0m2.156s
> sys	0m0.071s
> 
> Cpython
> Created container
> No more items
> 
> real	0m2.758s
> user	0m2.574s
> sys	0m0.174s
> 
> (I verified with your debug=1 that the objects appear.)
> 
> But that improvement is (I'm sure) only b/c all the other stuff you do
> around it with manipulating the dict on the python side. That does get
> JIT'ed just fine.
> 
> Btw., I had to jack up N by 10x, otherwise all numbers were just noise. :}
> 
> > right now it uses dictionary to map
> > pyobject id into objects when they leave c++ domain. Obviously, you add
> > about O(2*log(N)) overhead to the code, but that should not be too much
> > anyway. If that can be done in rpython, it could probably be made to work
> > massively faster. Like a module that does object caching for the storage
> > of  objects that would be sent to c++ domain. Then in c++ one can do as
> > simple as have an long int index that can be used to resolve back to
> > byobject when the job is done.
> 
> From a quick read of the cpyext code, that is rather similar to what is
> there.
> 
> But as per above, the problem isn't with the extra overhead, the problem is
> with the JIT being prevented to do any work (and with pypy-c being slower
> than CPython in pure interpreted mode).
> 
> > mine is 0.9.0-2009.05.16, but it does crash with gcc 4.6. maybe you have
> > older gcc in a different slot?
> 
> I do, but those should not be accessible from my pypy work environment. I
> did rebuild it. Not sure whether that matters. I'll update the docs showing
> how a different set of headers from another compiler can be selected.
> 
> > The point is that if the header is included, and it confuses the parser,
> > then the patterns have no effect. So it is better not to have extra
> > headers like Qt library=)
> 
> Qt is a story in and of itself, of course. :) Actually, what folks here do,
> is to link any code from external C++ through python to Qt (C++) by making
> use of the fact that both ends can handle opaque pointers, represented as
> long values. (The combination of addressof and bind_object in cppyy.)
> 
> Best regards,
>             Wim
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pypy-dev/attachments/20120604/ce5f5e81/attachment.html>