[Tutor] file accessibility across directories?

[Mats Wichmann]

On 2/28/24 18:45, dn via Tutor wrote:
> On 29/02/24 13:06, Mats Wichmann wrote:
>> On 2/28/24 15:39, James Hartley wrote:
>>> While in the project directory:
>>> $ pwd
>>> /home/me/project
>>> $ cat src/foo.py
>>> class Foo:
>>>      pass
>>> $ cat utils/baz.py
>>> from src.foo import Foo
>>> $ python utils/baz.py
>>> Traceback (most recent call last):
>>>    File "./baz.py", line 1 in <module>
>>>      from src.foo import Foo
>>> ModuleNotFoundError: No module named 'src'
>>
>>
>> When you run a script, the directory of the script is added to 
>> sys.path. Not the directory you were in when you ran it.  You can 
>> print out sys.path in utils/baz.py to convince yourself of this.  
>> There's no src in that directory (utils).
> 
> Whereas pytest works the other way around: run from the 
> project-directory because its (internal) 'process of discovery' will 
> find test* files by searching the entire directory-tree...
> (hence the questions in earlier reply)


Right... pytest takes a common situation and helps you out, in this case 
(a) by discovering tests and (b) fiddling paths so those tests can 
import cleanly. If you don't get that help by using (for example) a test 
framework, you have to take care of it yourself.

This import fails because for an absolute import (src.foo - doesn't 
start with dots), the starting directory has to be in the search path 
(sys.path), but it isn't. You can fix that by adding to sys.path in the 
scripts in utils, or by using PYTHONPATH.  For example, this should work:

PYTHONPATH=. python utils/baz.py

You would think you could use relative paths. However, if you attempt a 
relative import like:

from ..src.foo import Foo

it doesn't work, because the way the script was invoked, Python doesn't 
think it's part of a package (invoked as a script, the name is 
"__main__", and contains no dots, thus you must not be in a package), 
and relative paths are package-hierarchy-relative, not 
filesystem-relative, so you'll get one of Python's famous error messages:

ImportError: attempted relative import with no known parent package

However, you could run baz as a module, like:

python -m utils.baz

because now the name of the running module is "utils.baz", so it looks 
like a package rooted at a level above "utils", and that directory will 
be in sys.path, so "from src/foo import Foo" should work.  It just seems 
a bit ugly to have to invoke a "top level script" as a module to get 
imports to work.

Python imports are... challenging.

(yes, I'm assuming answers to the question @dn asked, so this could be 
wrong if I made the wrong guess)