[Tutor] import imports

[spir]

Hello,

Here are some ideas around import. I wish to receive comments and to learn whether they fit python. As I am not a highly experimented programmer, there may be some features or practices I do know know or do not know well. The following ideas are not dependant of each other, but as they all concern the import topic, I thought they could be grouped in a single message anyway.



=== import transmission ===

The module attribute __all__ allows defining names to be exported. I always use it for my own modules, and also sometimes define it in external modules, when absent, or adapt it to my needs. Then I'm happy with a secure "import *".
I find nevertheless an issue in the common case of import "transmission" from module to module, where all imported names usually have to repeated to export. This is not only ugly, imo, but also unpracticle when a single change in an name export list has to be corrected anywhere else; there can easily be a whole chain or even a 'tree' of imports.
I have found a way to deal with that, illustrated in below in a case where module M0 imports M1 and M2, which itself imports M3:

*** M3 ***
M3_names = [actual list of relevant names]
__all__ = ["M3_names"] + M3_names

*** M2 ***
from M3 import *
M2_names = [actual list of relevant /local/ names] + M3_names
__all__ = ["M2_names"] + M2_names

*** M1 ***
M1_names = [actual list of relevant names]
__all__ = ["M1_names"] + M1_names

*** M0 ***
from M1 import *
from M2 import *
M0_names = [actual list of relevant /local/ names] + M1_names + M2_names
__all__ = ["M0_names"] + M0_names

This has the advantage to avoid repetition, and allow a change in a name export list with no need of "chain correction". Still, it is only a personal trick. I wonder whether such a practice could be officialised with a kind of __imported_names__ module attribute, then understood and usable by all:

from Mx import *
__all__ = __imported_names__ + [actual list of relevant /local/ names]



=== semantics of "import *" ===

"import *" is not recommanded for good reasons, as a blind and risky practice. Now, when this is done consciously, kwowing that the imported module defines __all__, then the meaning is totally different, if not opposite. As a consequence, I am not happy at all with the present ambiguity. My proposal would be:

* Either a slightly different syntactic form, meaning "import all names defined for export". [I would like "import **", for '*' and '**' are associated in other cases and this is all about named things. But this point is no issue.] As an additional advantage, imo important, trying this kind of import from a module that does not define __all__ would raise an exception; for instance: ImportError("Module <module name> does not define any name list for export.")

* Or change the semantics of "import *" to require __all__ to be defined in the pointed module. Obviously, this raises compatibility issues that may or may not be adressed using warnings (and/or "from __future__ import import" ;-}) in a change path.



=== package module lookup ===

There is a similar issue at the package/application level: I do not know of any practice or feature that simply allows changing a filename, a directory name, or the organisation of the package's directory tree, without having then to correct all dependance information inside the concerned modules.
I would love a kind of import lookup to be based on the following assumption:
"The main/startup module resides in the package's root dir."
Then any import lookup would automatically explore local, relative, sub directories. E basta!
This would also encourage the good practice of never writing absolute imports -- which is a plague.

There are probably many good ways to implement that feature clearly and efficiently. Here is proposed an example that only aims at clarifying the feature's meaning & usefulness; but does not pretend to be good. The purpose is python to know not only which modules exist, or which modules are intended to be imported, but also where actually they are -- or should be. This does not only address import from client code but also intra-module dependances.

A file called __init__.py is used to identify a package as package and provide information about it. I propose to add a __dir__ attribute to a package, defined inside its __init__.py, that would hold a module directory (in the sense of 'index'). This could be implemented as a python dict holding a set of name:path bindings -- while actually it is pure information. It would be either a complement, or an alternative, to a package's __all__ attribute that lists the module names:

__all__ = ["main", "compute", "userinput"]
__dir__ = ["main":"/", "compute":"/tools", "userinput":"/interface/dialogs"]

Note that the pathes are *relative*. The main point is that __dir__'s value would be set by python itself when it is absent from __init__.py or when it needs beeing updated after any change in the package's tree. So that the process of module lookup would be as follows (provided I properly understand it):

* If current module is not part of a package: usual module lookup is performed.
* If it is part of a package which __dir__ is not yet defined, perform a local tree traversal starting from the module's root dir to reference the modules, then write the result into __dir__. Use it to import.
* If __dir__ is defined, try and import using it. If import fails (maybe the package's organisation has changed), while the module is referenced by __all__, update __dir__ as necessary. Then, try import again.
* When intra-package import finally fails, use PYTHONPATH to try external import, as usual.

Obviously, as long as the package's organisation does not change, a single traversal is necessary for all further imports during all further program executions. Intra-package imports may be faster, even if it is not the primary purpose of the proposal.
Actually, this seems so simple that python aware editors may even provide this automatically: meaning maintain __dir__ inside an __init__.py file. (This is close to the notion of "project" that many editors provide.)


Denis


------
la vida e estranya