[pypy-issue] Issue #2810: Embedding PyPy + cpyext API calls (pypy/pypy)

Omer Ben-Amram issues-reply at bitbucket.org
Thu Apr 26 07:19:54 EDT 2018

New issue 2810: Embedding PyPy + cpyext API calls

Omer Ben-Amram:

Hi guys,

I've been fiddling around with adding pypy support for the rust [PyO3](https://github.com/PyO3/pyo3).

I've been quite successful with the original intent of generating PyPy compatible rust extensions! (wrapped as cpython modules) (the modifications are at - [my fork](https://github.com/omerbenamram/pyo3/tree/feature/add-build-pypy-support)) by correctly linking to libpypy, changing the exported names and re implementing some missing macros in rust.

However, the other direction (rust code that embeds pypy) - which would try to call cpyext functions directly from rust works for cpython (since PyO3 handles all cpython initialization correctly), but crashes with PyPy.

I'm linking my rust library against a debug version of libpypy (5.10, haven't downloaded 6 yet) I've tried several combinations of calls to `rpython_startup_code`, `pypy_setup_home` (as referenced [here](http://doc.pypy.org/en/latest/embedding.html)), all of them yielded the following results:

So something like this works (will exit without crashing):
![Screen Shot 2018-04-26 at 13.57.59.png](https://bitbucket.org/repo/R7AbB/images/1079889783-Screen%20Shot%202018-04-26%20at%2013.57.59.png)

But importing via regular API would crash, interestingly after printing the zen:
![Screen Shot 2018-04-26 at 14.00.51.png](https://bitbucket.org/repo/R7AbB/images/3631796291-Screen%20Shot%202018-04-26%20at%2014.00.51.png)
![Screen Shot 2018-04-26 at 14.02.22.png](https://bitbucket.org/repo/R7AbB/images/626902820-Screen%20Shot%202018-04-26%20at%2014.02.22.png)

Interestingly, using an API that doesn't allocate anything, such as `PyBool_FromLong` results in a more verbose crash.
![Screen Shot 2018-04-26 at 14.04.20.png](https://bitbucket.org/repo/R7AbB/images/2586941973-Screen%20Shot%202018-04-26%20at%2014.04.20.png)

I'm suspicious that there is some ref counting/GIL management issue that is causing the crashes.

Am I missing something here? would it be possible to use the embedding API to call cpyext emulated functions from rust? 

A minimal code sample to reproduce the situation can be found [here](https://github.com/omerbenamram/pypy_ffi_playground/tree/pypy-embedding-crash)



More information about the pypy-issue mailing list