
Hello, Here are some ideas around import. I wish to receive comments and to learn whether they fit python. As I am not a highly experimented programmer, there may be some features or practices I do know know or do not know well. The following ideas are not dependant of each other, but as they all concern the import topic, I thought they could be grouped in a single message anyway. === import transmission === The module attribute __all__ allows defining names to be exported. I always use it for my own modules, and also sometimes define it in external modules, when absent, or adapt it to my needs. Then I'm happy with a secure "import *". I find nevertheless an issue in the common case of import "transmission" from module to module, where all imported names usually have to repeated to export. This is not only ugly, imo, but also unpracticle when a single change in an name export list has to be corrected anywhere else; there can easily be a whole chain or even a 'tree' of imports. I have found a way to deal with that, illustrated in below in a case where module M0 imports M1 and M2, which itself imports M3: *** M3 *** M3_names = [actual list of relevant names] __all__ = ["M3_names"] + M3_names *** M2 *** from M3 import * M2_names = [actual list of relevant /local/ names] + M3_names __all__ = ["M2_names"] + M2_names *** M1 *** M1_names = [actual list of relevant names] __all__ = ["M1_names"] + M1_names *** M0 *** from M1 import * from M2 import * M0_names = [actual list of relevant /local/ names] + M1_names + M2_names __all__ = ["M0_names"] + M0_names This has the advantage to avoid repetition, and allow a change in a name export list with no need of "chain correction". Still, it is only a personal trick. I wonder whether such a practice could be officialised with a kind of __imported_names__ module attribute, then understood and usable by all: from Mx import * __all__ = __imported_names__ + [actual list of relevant /local/ names] === semantics of "import *" === "import *" is not recommanded for good reasons, as a blind and risky practice. Now, when this is done consciously, kwowing that the imported module defines __all__, then the meaning is totally different, if not opposite. As a consequence, I am not happy at all with the present ambiguity. My proposal would be: * Either a slightly different syntactic form, meaning "import all names defined for export". [I would like "import **", for '*' and '**' are associated in other cases and this is all about named things. But this point is no issue.] As an additional advantage, imo important, trying this kind of import from a module that does not define __all__ would raise an exception; for instance: ImportError("Module <module name> does not define any name list for export.") * Or change the semantics of "import *" to require __all__ to be defined in the pointed module. Obviously, this raises compatibility issues that may or may not be adressed using warnings (and/or "from __future__ import import" ;-}) in a change path. === package module lookup === There is a similar issue at the package/application level: I do not know of any practice or feature that simply allows changing a filename, a directory name, or the organisation of the package's directory tree, without having then to correct all dependance information inside the concerned modules. I would love a kind of import lookup to be based on the following assumption: "The main/startup module resides in the package's root dir." Then any import lookup would automatically explore local, relative, sub directories. E basta! This would also encourage the good practice of never writing absolute imports -- which is a plague. There are probably many good ways to implement that feature clearly and efficiently. Here is proposed an example that only aims at clarifying the feature's meaning & usefulness; but does not pretend to be good. The purpose is python to know not only which modules exist, or which modules are intended to be imported, but also where actually they are -- or should be. This does not only address import from client code but also intra-module dependances. A file called __init__.py is used to identify a package as package and provide information about it. I propose to add a __dir__ attribute to a package, defined inside its __init__.py, that would hold a module directory (in the sense of 'index'). This could be implemented as a python dict holding a set of name:path bindings -- while actually it is pure information. It would be either a complement, or an alternative, to a package's __all__ attribute that lists the module names: __all__ = ["main", "compute", "userinput"] __dir__ = ["main":"/", "compute":"/tools", "userinput":"/interface/dialogs"] Note that the pathes are *relative*. The main point is that __dir__'s value would be set by python itself when it is absent from __init__.py or when it needs beeing updated after any change in the package's tree. So that the process of module lookup would be as follows (provided I properly understand it): * If current module is not part of a package: usual module lookup is performed. * If it is part of a package which __dir__ is not yet defined, perform a local tree traversal starting from the module's root dir to reference the modules, then write the result into __dir__. Use it to import. * If __dir__ is defined, try and import using it. If import fails (maybe the package's organisation has changed), while the module is referenced by __all__, update __dir__ as necessary. Then, try import again. * When intra-package import finally fails, use PYTHONPATH to try external import, as usual. Obviously, as long as the package's organisation does not change, a single traversal is necessary for all further imports during all further program executions. Intra-package imports may be faster, even if it is not the primary purpose of the proposal. Actually, this seems so simple that python aware editors may even provide this automatically: meaning maintain __dir__ inside an __init__.py file. (This is close to the notion of "project" that many editors provide.) Denis ------ la vida e estranya

"spir" <denis.spir@free.fr> wrote
Here are some ideas around import. I wish to receive comments
I've never defined it in any of my modules. It encourages people to use from X import * which is nearly always a bad idea.
Then I'm happy with a secure "import *".
I'm not sure what you mean by a secure import *. The only danger is that of name collisions and limiting the exported names only helps by a tiny amount. There is still a good chance that another module will have the same name exported.
Exactly, thats why its a bad idea IMHO. You can never predict what other modules your module might be used with and to start customising other peoples modules to your own envirohnment leads to madness in my view!
Not at all. Two modules could still have conflicting names. I don;t see how you can avoid it. Thats why using import foo is better. import foo also has the advantage of forcing code to explicitly flag where imported functions and classes are found. This makes debugging and maintence far easier. import * is a "bad idea" for all sorts of reasons. Namespaces are powerful tools - lets use them.
This is a more valid issue although if you always import both package and subpackages directly and especially if you use aliases to protect your code references from change import foo as f means even if foo changes name you only need to change the import statement not all the references to f.x
But most packages don;t have a startup or main module. They are libraries that the application writer uses. Thus the app writer provides the main or startup module not the package writer.
This would also encourage the good practice of never writing absolute imports -- which is a plague.
I'm not sure what you mean by absolute imports. If you mean giving the physical path to the file then You cannot really do that in Python. I don't understand the comment?
I thought it did that already? It looks in sys.path for modules and when it finds them it sets the __file__ attribute if appropriate.. I got lost around here. I assume you are having problems with modules that I've never encountered? -- Alan Gauld Author of the Learn to Program web site http://www.freenetpages.co.uk/hp/alan.gauld

[Sorry, forgot to answer one of the questions in my previous reply.] Le Mon, 26 Jan 2009 22:38:24 -0000, "Alan Gauld" <alan.gauld@btinternet.com> a écrit :
"spir" <denis.spir@free.fr> wrote
=== package module lookup ===
[...]
What I call "absolute import" happens when you give python an absolute path to look for a module (even if its your own). For sure python does not let you write an absolute path in the import instruction itself, but people do that by appending to PYTHONPATH, using 'sys.path'. Some installation instructions even explicitely tell the users to do that. More commonly, application/package authors often rely on absolute pathes for internal imports (meaning that you cannot even chose the dir's name! not to talk about the actual location...) or external ones. sys.path.append("path/to/dir") import foo Actually, for external imports, the only solutions I know are either to use environment variables, or to rely on python standard locations for packages (which are relative to python's root path, itself defined as an environment variable). For intra-package imports, there are now facilities to write relative imports directly: http://www.python.org/dev/peps/pep-0328/#rationale-for-relative-imports Denis ------ la vida e estranya

"spir" <denis.spir@free.fr> wrote
Here are some ideas around import. I wish to receive comments
I've never defined it in any of my modules. It encourages people to use from X import * which is nearly always a bad idea.
Then I'm happy with a secure "import *".
I'm not sure what you mean by a secure import *. The only danger is that of name collisions and limiting the exported names only helps by a tiny amount. There is still a good chance that another module will have the same name exported.
Exactly, thats why its a bad idea IMHO. You can never predict what other modules your module might be used with and to start customising other peoples modules to your own envirohnment leads to madness in my view!
Not at all. Two modules could still have conflicting names. I don;t see how you can avoid it. Thats why using import foo is better. import foo also has the advantage of forcing code to explicitly flag where imported functions and classes are found. This makes debugging and maintence far easier. import * is a "bad idea" for all sorts of reasons. Namespaces are powerful tools - lets use them.
This is a more valid issue although if you always import both package and subpackages directly and especially if you use aliases to protect your code references from change import foo as f means even if foo changes name you only need to change the import statement not all the references to f.x
But most packages don;t have a startup or main module. They are libraries that the application writer uses. Thus the app writer provides the main or startup module not the package writer.
This would also encourage the good practice of never writing absolute imports -- which is a plague.
I'm not sure what you mean by absolute imports. If you mean giving the physical path to the file then You cannot really do that in Python. I don't understand the comment?
I thought it did that already? It looks in sys.path for modules and when it finds them it sets the __file__ attribute if appropriate.. I got lost around here. I assume you are having problems with modules that I've never encountered? -- Alan Gauld Author of the Learn to Program web site http://www.freenetpages.co.uk/hp/alan.gauld

[Sorry, forgot to answer one of the questions in my previous reply.] Le Mon, 26 Jan 2009 22:38:24 -0000, "Alan Gauld" <alan.gauld@btinternet.com> a écrit :
"spir" <denis.spir@free.fr> wrote
=== package module lookup ===
[...]
What I call "absolute import" happens when you give python an absolute path to look for a module (even if its your own). For sure python does not let you write an absolute path in the import instruction itself, but people do that by appending to PYTHONPATH, using 'sys.path'. Some installation instructions even explicitely tell the users to do that. More commonly, application/package authors often rely on absolute pathes for internal imports (meaning that you cannot even chose the dir's name! not to talk about the actual location...) or external ones. sys.path.append("path/to/dir") import foo Actually, for external imports, the only solutions I know are either to use environment variables, or to rely on python standard locations for packages (which are relative to python's root path, itself defined as an environment variable). For intra-package imports, there are now facilities to write relative imports directly: http://www.python.org/dev/peps/pep-0328/#rationale-for-relative-imports Denis ------ la vida e estranya
participants (2)
-
Alan Gauld
-
spir