[Python-Dev] Pre-PEP: Redesigning extension modules

[Nick Coghlan]

On 2 September 2013 18:16, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Le Sun, 1 Sep 2013 02:19:48 +1000,
> Nick Coghlan <ncoghlan at gmail.com> a écrit :
>> On 1 Sep 2013 00:10, "Stefan Behnel" <stefan_ml at behnel.de> wrote:
>> >
>> > *bump*
>> >
>> > Does this sound like a viable solution?
>>
>> This isn't likely to progress until we have Eric's PEP 451 to a point
>> where it's ready for python-dev discussion and pronouncement.
>>
>> However, the revised loader API is being designed to allow for the
>> loader returning arbitrary objects, so something along these lines
>> should work. There will likely be some adjustments to the API
>> signature to allow extension modules to optionally support reloading
>> if they so desire.
>
> I think the biggest challenge here is to propose an API that's simple
> and easy to use (i.e. that doesn't make extension module writing more
> complicated than it currently is).
>
> The basic concept of putting custom module objects in sys.modules is
> sound, IMHO.

The hook API I currently have in mind is a two step initialisation:

    PyImport_PrepareNAME (optional)
    PyImport_ExecNAME

If you don't define prepare, the import system takes care of creating
a module object for you, and passing it in to the exec hook. The
return result from that is just an integer indicating success or
failure (on failure, an exception should be set).

If you *do* define the prepare hook, then it's similar to the existing
init hook, but receives a PEP 451 module spec object with info about
the module being imported (see PEP 451 for the draft details) and is
permitted to return an arbitrary PyObject reference.

The main open questions I have are how to deal with clearly indicating
whether modules support in-place reloading, unloading, loading in
subinterpreters and/or loading a second copy in the same interpreter.
That's actually more a question for PEP 451 though, so I'll post some
more detailed thoughts on that over on import-sig, which may
eventually make their way into a PEP 451 draft.

> As for "extension module as a wrapper", though, it shounds like the
> kind of complication I would personally prefer to stay away from. Also,
> it would make extension modules less like Python modules, rather than
> more.

Yeah, that will be allowed (since we'll probably permit returning
arbitrary objects), but definitely not required.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia