On Tue, 12 Jan 2016 at 01:59 Victor Stinner <victor.stinner@gmail.com> wrote:
Hi,

2016-01-12 1:47 GMT+01:00 Nick Coghlan <ncoghlan@gmail.com>:
>> This method doesn't make sense at all in PyPy. The method is specific
>> to CPython since it relies on guards which have a pure C API (see
>> below). The PEP must be more explicit about that. IMHO it's perfectly
>> fine that PyPy makes this method a no-op (the method exactly does
>> nothing). It's already the case if a guard "always" fail in
>> first_check().
>
> Perhaps the specialisation call should also move to being a pure C
> API, only exposed through _testcapi for testing purposes?
>
> That would move both this and the dict versioning PEP into the same
> territory as the dynamic memory allocator PEP: low level C plumbing
> that enables interesting CPython specific extensions (like
> tracemalloc, in the dynamic allocator case) without committing other
> implementations to emulating features that aren't useful to them in
> any way.

I really like your idea :-) It solves many issues and technically it's
trivial to only add a C API and then expose it somewhere else at the
Python level (for example in my "fat" module", or as you said in
_testcapi for testing purpose).

Instead of adding func.specialize() and func.get_specialized() at
Python level, we can add *public* functions to the Python C API
(excluded of the stable ABI):

/* Add a specialized function with guards. Result:
 * - return 1 on success
 * - return 0 if the specialization has been ignored
 * - raise an exception and return -1 on error */
PyAPI_DATA(int) PyFunction_Specialize(PyObject *func, PyObject *func2,
                                      PyObject *guards);

/* Get the list of specialized functions as a list of
 * (func, guards) where func is a callable or code object and guards
 * is a list of PyFuncGuard (or subtypes) objects.
 * Raise an exception and return NULL on error. */
PyAPI_FUNC(PyObject*) PyFunction_GetSpecialized(PyObject *func);

/* Get the specialized function of a function. stack is a an array of PyObject*
 * objects: indexed arguments followed by (key, value) objects of keyword
 * arguments. na is the number of indexed arguments, nk is the number of
 * keyword arguments. stack contains na + nk * 2 objects.
 *
 * Return a callable or a code object on success.
 * Raise an exception and return NULL on error. */
PyAPI_FUNC(PyObject*) PyFunction_GetSpecializedFunc(PyObject *func,
                                                    PyObject **stack,
                                                    int na, int nk);

Again, other Python implementations which don't want to implement
function specializations can implement these functions as no-op (it's
fine with the API):

* PyFunction_Specialize() just returns 0
* PyFunction_GetSpecialized() creates an empty list
* PyFunction_GetSpecializedFunc() returns the code object of the
function (which is not something new)

Or not implement these functions at all, since it doesn't make sense for them.

 This is somewhat similar to the JIT API we have been considering through our Pyjion work:

* PyJit_Init()
* PyJit_RegisterCodeObject()
* PyJit_CompileCodeObject()

If both ideas gain traction we may want to talk about whether there is some way to consolidate the APIs so we don't end up with a ton of different ways to optimize code objects.