[Python-ideas] Import and '..', '../..' in serach path.

[Ron Adam]

Brett Cannon wrote:
> On 1/30/07, Ron Adam <rrr at ronadam.com> wrote:
>> Brett Cannon wrote:
>> > On 1/30/07, Ron Adam <rrr at ronadam.com> wrote:
>> >>
>> >> In order to resolve a path conflict where I'm working on several
>> >> copies of the
>> >> same package.  I found it useful to add the following near the top of
>> >> modules in
>> >> a package or sub package.
>> >>
>> >>
>> >> Module in package:
>> >>
>> >>      import sys
>> >>      sys.path = ['..'] + sys.path
>> >>
>> >>      import package.module          # Imports module in "this!" 
>> package.
>> >>
>> >>
>> >> Note: There could still be conflicts if a module with the same name is
>> >> in the
>> >> same directory as the package.  But that's much less likely than one
>> >> in the rest
>> >> of the path.
>> >>
>> >>
>> >> Module in sub-package:
>> >>
>> >>      import sys
>> >>      sys.path = ['../..'] + sys.path
>> >>
>> >>      import package.subpackage.module   # finds "self" (subpackage)
>> >> reliably.
>> >>
>> >>
>> >> By explicitly adding the packages parent directory to the *front* of
>> >> sys.path it
>> >> resolves cases where imports using absolute imports, import modules
>> >> from another
>> >> package because they are found first in the search path.
>> >
>> > Why aren't you using relative imports (e.g., ``from . import
>> > module``)?  That should be doing exactly what you want.  That uses
>> > __path__ which is set to the path of the package.
>> >
>> > -Brett
>>
>>
>> But if you try to run a module in a package, vs from a package, with 
>> relative
>> from . imports, you will get:
>>
>>       ValueError: Attempted relative import in non-package
>>
>> And there is no __path__ attribute to look at since no package has 
>> been imported
>> yet.
> 
> Yeah, I have been bitten by that.  You need to have that initial
> package import.  I got nailed by this when using '-m' and a module
> contained within a package for executing tests.

And you can't set __path__ manually, that only changes the Exception to an 
Import Error.


>> So you need to import the package first, and then you may not import 
>> the correct
>> package even then if there is another package or module with the same 
>> name.
>>
> 
> Well, just don't do that.  =)  You really shouldn't have multiple
> things with the same name in sys.path.  But as you noticed, you can
> control it with sys.path if you really need to.
> 
> I just don't see this as a common enough problem to warrant
> documenting it somewhere (and I have no clue where that "somewhere"
> would be).


I expected that someone would say that.

How about adding a note at the bottom of PEP 328. That was one of the places I 
looked for a way to resolve this.  Or possibly posting a message on 
python-ideas.  Consider it done  ;-)

A note on PEP 328 might be a good idea still.  I have no idea what the best 
wording would be.  The import tutorial is another place where a note clarifying 
relative imports can't be used for stand alone scripts is needed.


The nice thing about adjusting sys.path directly is you can then move back and 
forth from different same named versions in development without doing renames. 
If you do a rename, you also have to remember to rename all the submodule 
absolute references in the package as well.  Another alternative is to hide the 
ones you don't want by renaming them.  Then you have to do renames to switch 
back to them.  That makes it more difficult to do side by side comparisons as well.

Altering PYTHONPATH is another way to do it, but you also need to switch it back 
if you switch versions.  But if you forget to edit it back, and use a version 
not on the path, it will import modules from the version on the path.

In this case the package I'm working on may replace pydoc.py module in pythons 
library. (no promises though) The name matters so that site.py can locate the 
help function.  It's convenient for me to be able to easily and dependably 
compare different working versions.


A few things to consider...

-----
According to PEP 328:

You may use relative imports freely. In Python 2.6, any import statement that 
results in an intra-package import will raise DeprecationWarning (this also 
applies to from <> import that fails to use the relative import syntax). In 
Python 2.7, import will always be an absolute import (and the __future__ 
directive will no longer be needed).
-----

So It will probably come up more often as more people start using absolute 
imports and the '.' style relative imports. Or as you put it, more people get 
"bitten".

Take a look at how much __name__ == '__main__' is used in the standard library. 
  Those modules won't be able to use relative imports.  They will need to be 
updated and can use only absolute imports.

Some people will probably want a way to have relative imports work with modules 
meant to be run as scripts (in packages).

An optional function (or property) may be a good way to resolve both of these 
package self reference situations without any changes to the default behavior.


Cheers,
   Ron