Hi Antoine,

yep, this might be worth discussing here, although the problems I had were mostly related to trying to reproduce CPython's semantics of relative, absolute and cyclic imports in Cython.

Antoine Pitrou schrieb am 22.02.19 um 19:02:

Not sure this is the right forum for this, but on the Cython bug tracker, the problem of managing to use the C API for importing modules came up: https://github.com/cython/cython/issues/2854#issuecomment-466487200

If Stefan Behnel, who is one of the most experimented persons in the world with CPython's C API, has trouble getting it working, then probably it deserves improving. ;-)

(I also realize that the C API is more or less married to the dubious semantics of the standard "__import__" function, which is itself difficult to change, so perhaps a better C API isn't very easy)

Yes, it seems that's a major part of the issue, and also the fact that relative imports were added very late to Python, when most of the C-API import functions already existed.

The main import function, "PyImport_ImportModuleLevelObject()" (link below) mimics the Python semantic difference between (1) "import a.b.c", (2) "import a.b.c as c", and (3) "from a.b import c", where the first only assigns "a" and the second and third assign "c". Thus, its return value changes depending on whether a "fromlist" is passed or not. That is a rather weird interface.

However, none of the other import functions allow relative imports. This is the only one that allows passing a "level", and it's not even consistent with the Python semantics. In Python, a relative import like "from ..pkg import module" assigns to the module name, whereas a relative C-API import of "pgk.module" with level 2 returns the top-level package, which wasn't even mentioned anywhere in the import (there might actually be a reason why I'm trying a relative import here).

Basically, if I want to execute a relative import to get another module, I have to jump through hoops like passing the non-empty fromlist ['*'], just to have it return the module that I wanted to import instead of the top-level module. Or, since there is no simple way to get from the top-level module back to the imported module, I'd have to keep looking up submodule attributes. The functionality for that is implemented in byte code in CPython, not as a C-API function.

And then, I can run into corner cases like circular imports. While one module is initialising, it might not have been assigned to its package as an attribute yet (even though it's already in sys.modules). Looking up the attribute chain in that case risks to fail. So, in order to avoid that AttributeError, I could first execute the relative import, and then throw away the return value and call "PyImport_GetModule()" to get the module from sys.modules.

It really shouldn't be that hard to do these things.

Stefan

[*] https://docs.python.org/3/c-api/import.html#c.PyImport_ImportModuleLevelObje...