Relative package imports
Although David has already copy-posted a message regarding this issue to the list, I would like to restate the problem to get a discussion going (and then maybe take it to c.l.p for general flaming ;). The problem we have run into on starship is that some well-known packages have introduced naming conflicts leading to the unfortunate situation that they can't be all installed on the same default path: 1. Zope has a module named DateTime which also is the base name of the package mxDateTime. 2. Both Zope and PIL have a top-level module named ImageFile.py (different ones of course). Now the problem is how to resolve these issues. One possibility is turning Zope and PIL into proper packages altogether. To ease this transition, one would need a way to specify relative intra-package imports and a way to tell pickle where to look for modules/packages. The next problem we'd probably run into sooner or later is that there are quite a few useful top-level modules with generic names that will conflict with package names and other modules with the same name. I guess we'd need at least three things to overcome this situation once and for all ;-): 1. Provide a way to do relative imports, e.g. a single dot could be interpreted as "parent package": modA.py modD.py [A] modA.py modB.py [B] modC.py modD.py In modC.py: from modD import * (works as usual: import A.B.modD) from .modA import * (imports A.modA) from ..modA import * (import the top-level modA) 2. Establish a general vendor based naming scheme much like the one used in the Java world: from org.python.core import time,os,string from org.zope.core import * from com.lemburg import DateTime from com.pythonware import PIL 3. Add a way to prevent double imports of the same file. This is the mayor gripe I have with pickle currently, because intra- package imports often lead to package modules being imported twice leading to many strange problems (e.g. splitting class hierarchies, problems with isinstance() and issubclass(), etc.), e.g. from org.python.core import UserDict u = UserDict.UserDict() import UserDict v = UserDict.UserDict() Now u and v will point to two different classes:
u.__class__
v.__class__
4. Add some kind of redirection or lookup hook to pickle et al. so that imports done during unpickling can be redirected to the correct (possibly renamed) package. -- Marc-Andre Lemburg ______________________________________________________________________ Y2000: 196 days left Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/
2. Both Zope and PIL have a top-level module named ImageFile.py (different ones of course).
Now the problem is how to resolve these issues. One possibility is turning Zope and PIL into proper packages altogether. To ease this transition, one would need a way to specify relative intra-package imports and a way to tell pickle where to look for modules/packages.
fwiw, PIL 1.0b1 can already be used as a package, but you have to explicitly import the file format handlers you need: from PIL import Image import PIL.GifImagePlugin import PIL.PngImagePlugin import PIL.JpegImagePlugin etc. this has been fixed in PIL 1.0 final. </F>
It seems that there is not much interest in the topic... I'll be offline for the next two weeks -- maybe someone could pick the thread up and toss it around a bit while I'm away. Thanks, -- Marc-Andre Lemburg ______________________________________________________________________ Y2000: 193 days left Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/
It seems that there is not much interest in the topic...
I'll be offline for the next two weeks -- maybe someone could pick the thread up and toss it around a bit while I'm away.
OK - here are my 2c on it: Unless I am mistaken, this problem could be solved with 2 steps: * Code moves to Python packages. * The standard Python library move to a package. If all non-trivial Python program used packages, and some agreement on a standard namespace could be met, I think it would be addressed. There was a thread on the newsgroup about the potential naming of the standard library. You did state as much in your proposal - indeed, you state "to ease the transition". Personally, I dont think it is worth it, mainly because we end up with a half-baked scheme purely for the transition, but one that can never be removed. To me, the question is one of: * Why arent Zope/PIL capable of being used as packages. * If they are (as I understand to be the case) why do people choose not to use them as such, or why do the authors not recommend this? * Is there a deficiency in the package scheme that makes it hard to use? Eg, should "__" that ni used for the parent package be reinstated? Mark.
Mark Hammond wrote:
* Why arent Zope/PIL capable of being used as packages.
PIL can be used as a package ("from PIL import Image"), assuming that it's installed under a directory in your path. there's one pro- blem in 1.0b1, though: you have to explicitly import the file format handlers you need: import PIL.JpegImagePlugin import PIL.PngImagePlugin this has been fixed in 1.0 final.
* If they are (as I understand to be the case) why do people choose not to use them as such, or why do the authors not recommend this?
inertia, and compatibility concerns. we've decided that all official material related to PIL 1.0 will use the old syntax (and all 1.X releases will be possible to install using the PIL.pth approach). too many users out there... now, PIL 2.0 is a completely different thing...
* Is there a deficiency in the package scheme that makes it hard to use?
not that I'm aware... </F>
Mark Hammond wrote:
It seems that there is not much interest in the topic...
I'll be offline for the next two weeks -- maybe someone could pick the thread up and toss it around a bit while I'm away.
OK - here are my 2c on it:
Unless I am mistaken, this problem could be solved with 2 steps: * Code moves to Python packages. * The standard Python library move to a package.
If all non-trivial Python program used packages, and some agreement on a standard namespace could be met, I think it would be addressed. There was a thread on the newsgroup about the potential naming of the standard library.
You did state as much in your proposal - indeed, you state "to ease the transition". Personally, I dont think it is worth it, mainly because we end up with a half-baked scheme purely for the transition, but one that can never be removed.
With "easing the transition" I ment introducing a way to do relative package imports: you don't need relative imports if you can be sure that the package name will never change (with a fixed naming scheme, a la com.domain.product.package...). The smarter import mechanism is needed to work-around the pickle problems you face (because pickle uses absolute package names).
To me, the question is one of:
* Why arent Zope/PIL capable of being used as packages. * If they are (as I understand to be the case) why do people choose not to use them as such, or why do the authors not recommend this? * Is there a deficiency in the package scheme that makes it hard to use? Eg, should "__" that ni used for the parent package be reinstated?
I guess this would help a great deal; although I'd personally wouldn't like yet another underscore in the language. Simply leave the name empty as in '.submodule' or '..subpackage.submodule'. Cheers, -- Marc-Andre Lemburg ______________________________________________________________________ Y2000: 193 days left Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/
participants (3)
-
Fredrik Lundh
-
M.-A. Lemburg
-
Mark Hammond