[Python-Dev] PyCapsule_Import semantics, relative imports, module names etc.

[John Dennis]

While porting several existing CPython extension modules that form a 
package to be 2.7 and 3.x compatible the existing PyObject_* API was 
replaced with PyCapsule_*. This introduced some issues the existing 
CPython docs are silent on. I'd like clarification on a few issues and 
wish to raise some questions.

1. Should an extension module name as provided in PyModule_Create (Py3) 
or Py_InitModule3 (Py2) be fully package qualified or just the module 
name? I believe it's just the module name (see item 5 below) Yes/No?

2. PyCapsule_Import does not adhere to the general import semantics. The 
module name must be fully qualified, relative imports are not supported.

3. PyCapsule_Import requires the package (e.g. __init__.py) to import 
*all* of it's submodules which utilize the PyCapsule mechanism 
preventing lazy on demand loading. This is because PyCapsule_Import only 
imports the top level module (e.g. the package). From there it iterates 
over each of the module names in the module path. However the parent 
module (e.g. globals) will not contain an attribute for the submodule 
unless it's already been loaded. If the submodule has not been loaded 
into the parent PyCapsule_Import throws an error instead of trying to 
load the submodule. The only apparent solution is for the package to 
load every possible submodule whether required or not just to avoid a 
loading error. The inability to load modules on demand seems like a 
design flaw and change in semantics from the prior use of 
PyImport_ImportModule in combination with PyObject. [One of the nice 
features with normal import loading is setting the submodule name in the 
parent, the fact this step is omitted is what causes PyCapsule_Import to 
fail unless all submodules are unconditionally loaded). Shouldn't 
PyCapsule_Import utilize PyImport_ImportModule?

4. Relative imports seem much more useful for cooperating submodules in 
a package as opposed to fully qualified package names. Being able to 
import a C_API from the current package (the package I'm a member of) 
seems much more elegant and robust for cooperating modules but this 
semantic isn't supported (in fact the leading dot syntax completely 
confuses PyCapsule_Import, doc should clarify this).

5. The requirement that a module specifies it's name as unqualified when 
it is initializing but then also has to use a fully qualified package 
name for PyCapsule_New, both of which occur inside the same 
initialization function seems like an odd inconsistency (documentation 
clarification would help here). Also, depending on your point of view 
package names could be considered a deployment/packaging decision, a 
module obtains it's fully qualified name by virtue of it's position in 
the filesystem, something at compile time the module will not be aware 
of, another reason why relative imports make sense. Note the identical 
comment regarding _Py_PackageContext in  modsupport.c (Py2) and 
moduleobject.c (Py3) regarding how a module obtains it's fully qualified 
package name (see item 1).

Thanks!

-- 
John