[Python-ideas] Import and '..', '../..' in serach path.

[Ron Adam]

Brett Cannon wrote:
> On 1/30/07, Ron Adam <rrr at ronadam.com> wrote:
>> Brett Cannon wrote:
> [SNIP]
>> > I just don't see this as a common enough problem to warrant
>> > documenting it somewhere (and I have no clue where that "somewhere"
>> > would be).
>>
>>
>> I expected that someone would say that.
>>
>> How about adding a note at the bottom of PEP 328. That was one of the 
>> places I
>> looked for a way to resolve this.  Or possibly posting a message on
>> python-ideas.  Consider it done  ;-)
>>
>> A note on PEP 328 might be a good idea still.  I have no idea what the 
>> best
>> wording would be.  The import tutorial is another place where a note 
>> clarifying
>> relative imports can't be used for stand alone scripts is needed.
>>
> 
> Sure, but that is not my PEP.  That is something to bug Thomas Wouters 
> about.

Ok, Maybe later.  I looked into it further and I think it needs more thought in 
any case.


>> The nice thing about adjusting sys.path directly is you can then move 
>> back and
>> forth from different same named versions in development without doing 
>> renames.
>> If you do a rename, you also have to remember to rename all the submodule
>> absolute references in the package as well.  Another alternative is to 
>> hide the
>> ones you don't want by renaming them.  Then you have to do renames to 
>> switch
>> back to them.  That makes it more difficult to do side by side 
>> comparisons as well.

> Right but the point of relative imports is to help prevent renaming.
> It's a catch-22 situation of what matters more to you.

Right, thats what I tried first.  It would have been nice if that worked.


>> Take a look at how much __name__ == '__main__' is used in the standard 
>> library.
>>   Those modules won't be able to use relative imports.  They will need 
>> to be
>> updated and can use only absolute imports.

> It's possible.  Maybe this will finally push forward the idea of
> having a proper 'main' function or something.

<clip>

> Yeah, some built-in or something might do it that lets you decorate
> what function is to be the 'main' function and maybe do some magic to
> make it resolve properly.

It's not the main function that is the problem, it's the imports at the top of 
the file before the main function.  If they import ok, then any main function 
will work also.


I did some experimenting and just adding '../..' in front of the path has some 
problems of its own.  It's relative to the directory you run it from instead of 
the directory the script is in.  So it may or may not allow the script to run.

What is needed is to append the actual root package location to the front os 
sys.path. If absolute imports are used for intra-package imports, then there is 
not chance of shadowing the library by doing this.


This is what I came up with, but there may be an easier way. (and shorter)

def pkg_dir(file):
     import os
     file = os.path.abspath(file)
     path = os.path.dirname(file)
     # Next line should check for (.pyo, .pyc) files too.
     while os.path.exists(os.path.join(path, '__init__.py')):
         path = os.path.abspath(os.path.join(path, os.pardir))
     return path


# Library imports
import sys
sys.path = [pkg_dir(__file__)] + sys.path

# Absolute package imports here
...

tests():
     ...

if __name__ == '__main__':
    tests()


This does what you would expect it to do. It always finds it's own packages 
modules and it's not sensitive to what location you run it from.  But it's bit 
bulky to be putting everywhere.


The real problem isn't that you might get errors, it is that you *may not*.  For 
example, if you decide to modify an existing package, and so copy it to a new 
location for editing.  Then you do your edits, and after everything looks good 
you run the tests in the tests sub module. It passes, Yay!  So then you copy it 
back to it's original location.

Unfortunately the tests may have imported the unedited modules in the live 
package. So the edited package you replaced it with may have errors.  How 
serious that is depends on what the package is used for.  Those mars satellites 
can get pretty expensive after you loose 2 or 3 of them.  ;-)

I admit that's pretty unlikely because the package would probably go though some 
pretty rigorous testing in that case.  But still, there is the chance of 
something like that happening in more casual circumstances.  I don't think this 
is that far fetched.

And the other benefit is it just makes it easier to run modules in different 
circumstances without problems.  On the path, directly, indirectly... while 
editing.  That is really what I want at the moment.  It should *just work*.


Cheers,
    Ron