Exposing rpython based functions to cpython vm.
Hello, folks) I have been tinkering with pypy source tree for a week now and came up with a following question - is there any way to build a rpython program as a shared object which exposes some of its functionality (say a list of functions) for standard cpython vm, in other words I want this functions to be invokable from python script via shared object import. When I try to build my program like so: python rpython --shared -O2 some_program.py libmyprogram-c.so actually gets generated, unimportable from cpython though. Should I use CFFI for cpython next or there is some sort of facility (decorator or something)? -- Kind regards, Aleksandr Koshkin.
Hi, On 19 June 2017 at 13:52, Aleksandr Koshkin <tinysnippets@gmail.com> wrote:
I have been tinkering with pypy source tree for a week now and came up with a following question - is there any way to build a rpython program as a shared object which exposes some of its functionality (say a list of
Answered on https://stackoverflow.com/questions/44629893/invoke-rpython-based-functions-... . A bientôt, Armin.
On Mon, 19 Jun 2017, Armin Rigo wrote:
Answered on https://stackoverflow.com/questions/44629893/invoke-rpython-based-functions-...
I would add to that that in the case that the idea was to take some code out of hot loops, translate it into RPython so that it runs faster, and finally import it into cPython, then (as Armin elaborated) you're missing the point of RPython. However, here are few alternatives to consider: One go-to solution would be to use Cython which lets you easily blend Python / C up to the point of writing restricted C with Python syntax, and import resulting Cython modules into your program running under cPython. A more principled solution would be to port your application to PyPy (if any porting would be required at all), and enjoy the JIT, or else dig around trying to figure out why exactly is that you're not getting the speedups you were hoping for, and make PyPy better in the process! -- Sincerely yours, Yury V. Zaytsev
Thank you guys for your responses, appreciated. So as a study case I am building custom JIT powered regex matcher (somewhat more or less compatible with pcre). It is not a problem to build one as a stand alone application but I want it as a shared library and ultimately as a python compatible module. As a interface between rpython (C, really) and python for input I see a tuple - (program::bytes, input_string::unicode) and for output [(group0::unicode, value0::unicode), ..., ]. 1. Is it even a sane idea to use rpython toolchain to handle such task? I mean specifically embedding problem, because again as a stand alone app it is not a big deal and worth nothing. 2. Why some valid rpython code fails inside a function decorated with an entrypoint_highlevel (e.g. os.write(1, "hello world") would fail with segfault at runtime). And are there any dedicated docs describing such limitations? Cheers, magniff. 2017-06-19 22:55 GMT+03:00 Yury V. Zaytsev <yury@shurup.com>:
On Mon, 19 Jun 2017, Armin Rigo wrote:
Answered on https://stackoverflow.com/questions/44629893/invoke-rpython-
based-functions-from-standard-python-code
I would add to that that in the case that the idea was to take some code out of hot loops, translate it into RPython so that it runs faster, and finally import it into cPython, then (as Armin elaborated) you're missing the point of RPython. However, here are few alternatives to consider:
One go-to solution would be to use Cython which lets you easily blend Python / C up to the point of writing restricted C with Python syntax, and import resulting Cython modules into your program running under cPython.
A more principled solution would be to port your application to PyPy (if any porting would be required at all), and enjoy the JIT, or else dig around trying to figure out why exactly is that you're not getting the speedups you were hoping for, and make PyPy better in the process!
-- Sincerely yours, Yury V. Zaytsev
-- Kind regards, Aleksandr Koshkin.
participants (3)
-
Aleksandr Koshkin -
Armin Rigo -
Yury V. Zaytsev