[pypy-issue] Issue #2772: Theano working nominally within pypy (pypy/pypy)

garming-sam issues-reply at bitbucket.org
Mon Mar 5 03:28:20 EST 2018

New issue 2772: Theano working nominally within pypy


In trying to find a decent ML library to work on pypy, I figured out some hacks to get Theano working (at least superficially with runtime compiled cpyext modules). 

There seems to be a few problems that need to be resolved before it would work out of the box:

1. A single usage of the ctypes.pythonapi.PyCapsule_Type

2. A single usage of sys.getrefcount (to check that there isn't leaks)

3. Missing build information to runtime load the compiled .so python module
- The correct binary .so filename looks to be able to be constructed using distutils.sysconfig.get_config_var("SO") (or EXT_SUFFIX in python3)
- Expects libpython.so (or equivalent) in the distutils.sysconfig.get_config_var("LIBDIR") location which does not exist
- distutils.sysconfig.get_config_var("LDLIBRARY") does not resolve in pypy currently -- but should in theory point to libpypy-c

If this was to be fixed correctly, there's a few questions that I would expect would be needed to be resolved:
1. Which method is best to runtime determine if the code is running under pypy?
2. Is distutils.sysconfig.get_config_var("LDLIBRARY") meant to return libpypy-c?
3. Is libpypy-c supposed to reside in LIBDIR?

In doing some similar testing around this area, I also noticed that there are some config variables that exist in sysconfig that do not exist in distutils.sysconfig (or vice versa) like INCLUDEDIR which make it difficult to build cpython modules that can work against pypy.

The hacks I've done (for a specific version of pypy2) are in a branch here:

More information about the pypy-issue mailing list