[pypy-dev] pypy callback function pointer become invalid

[Armin Rigo]

Hi,

On 17 August 2015 at 12:59, Yicong Huang <hengha.mao at gmail.com> wrote:
> However, to perform the python function call, we need to parse char*(python
> code), a python code lexer, praser, JIT and etc, in other words do all the
> work that  pypy_execute_source() did.

This is done in a couple of lines in Python.  The following would do all this:

@ffi.function("void(char *)")
def run_function(p_src):
    src = ffi.string(p_src)
    glob = {}
    exec src in glob

In my previous example I used "eval" instead of "exec" only to return a value.

If you want to be more JIT-friendly, and if run_function() is going to
be called with the exact same p_src a lot of times, you could do
instead:

cache = {}
@ffi.function("void(char *)")
def run_function(p_src):
    src = ffi.string(p_src)
    code = cache.get(src)
    if code is None:
        code = compile(src, 'exec', 'exec')
        cache[src] = code
    glob = {}
    exec code in glob

Or maybe:

cache = []
@ffi.function("long(char *)")
def compile_and_cache(p_src):
    code = compile(ffi.string(p_src), 'exec', 'exec')
    index = len(cache)
    cache.append(code)
    return index

@ffi.function("void(long)")
def run_compiled_function(index):
    glob = {}
    exec cache[index] in glob


> So why don't PyPy expose and API to do such job ?

Because PyPy only exposes an *API builder*, not a particular API.  The
pypy_execute_source_ptr() is the bare minimal entry point, which
you're supposed to call once and build your own API as you see fit.
The functions above are examples of such an API.  And as shown above,
even such a simple API has various tweaks that are better in various
cases, like whether caching the code object is a good idea or not; you
can implement any version you like in a few lines of Python.


A bientôt,

Armin.