[Tutor] Best way to get root project directory in module search path

[boB Stepp]

On Sun, Aug 27, 2017 at 6:03 PM, Cameron Simpson <cs at cskk.id.au> wrote:
> On 27Aug2017 14:27, boB Stepp <robertvstepp at gmail.com> wrote:
>>
>> On Sun, Aug 27, 2017 at 2:13 AM, Cameron Simpson <cs at cskk.id.au> wrote:
>>>
>>> On 26Aug2017 21:27, boB Stepp <robertvstepp at gmail.com> wrote:

>>> The trouble with this specific approach is that '..' relies on your
>>> _working_ directory being above ScriptMenuSystem i.e. the directory you
>>> shell's in; '..' is not related to the path to the test.py file. What yu
>>> want for this is the actual path to the code i.e. __file__.  Eg:
>>>
>>>  sys.path.append(dirname(dirname(__file__)))
>>
>>
>> I tried to use this (on Python 2.4/2.6 -- one server has 2.4, the
>> other 2.6) and got the following:
>>
>> Traceback (most recent call last):
>>    File "... /test.py", line 9, in <module>
>>        sys.path.append(dirname(dirname(__file__)))
>> NameError:  name 'dirname' is not defined
>
>
> from os.path import dirname

It did not occur to me to import something from os.path to use sys.path!


>> Anyway, I tried something a bit different that implemented this:
>>
>> pgm_dir = os.path.dirname(os.path.abspath(__file__))
>> pgm_root = pgm_dir.replace('/ScriptMenuSystem', '')
>> sys.path.append(pgm_root)
>
>
> Calling os.path.dirname a second time does what your:
>
>  pgm_dir.replace('/ScriptMenuSystem', '')
>
> does, but reliably. Supposing you have the misfortune to have
> '/ScriptMenuSystem' higher up in the full path also? Supposing your package
> changes its name in the future, or is installed with another name?
> Fragility.
>

>> BTW, I have not used os.walk yet, but would that be better than my bit
>> of string manipulation above?  If yes, an example of walking up and
>> getting the directory exactly one level up from __file__ would be
>> appreciated.

> Given that os.walk walks _down_ the tree from some directory, how do you
> think it would help you?

At https://docs.python.org/release/2.4.4/lib/os-file-dir.html it says:

"walk(top[, topdown=True [, onerror=None]])
walk() generates the file names in a directory tree, by walking the
tree either top down or bottom up."

So I thought I could go in either direction.

>> I have been using the same project structure for some years now, so in
>> that sense it is well-established and not fragile.
>
>
> It is more that embedded specific hardwired knowledge in the code is an
> inherently fragile situation, if only because it must be remembered when
> naming changes occur. And they do. I have a quite large package I've been
> working on for a decade and am very seriously contemplating changing its
> name in the next month or so. Fortunately the installed base is small.

I was working on removing the name dependencies.  So far I had the following:

import os
import sys

def set_project_root():
    pgm_dir = os.path.dirname(os.path.abspath(__filename__))
    dir_to_rmv = pgm_dir.split(os.sep)[-1]
    pgm_root = pgm_dir.replace(dir_to_rmv, '')
    sys.path.append(pgm_root)

But this only works if I only need to go up one level.  If I have
Python files more deeply nested in the project structure than this
won't work.  So I was pondering how I can do this for arbitrarily
deeply nested folders without relying on any specific naming.  I was
also wondering if this sort of code could be put in the __init__.py
files?  I have yet to do this latter experiment.

>> So I am moving to put everything in a centrally accessible
>> location
>
>
> Very sound. But to use it your users should have that location as part of
> their python path, or the code needs to be invoked by some wrapper which can
> insert the needed path. Both these things live _outside_ the package code.

Not being well-versed in Unix, is it possible to change the PYTHONPATH
*only* for a specific group of users and only for python programs in a
certain shared library area?  I will have to research this.  If
possible, this might indeed be the best way.

>> So I think this idea (Or some variation thereof.) is what I need to
>> do.  Or the good experts here show me I'm engaged in foolishness!
>
>
> I think you're better off arranging your users' environments to include the
> shared library area, or providing a wrapper shell script which does that and
> then invokes the real code.
>
> Eg:
>
>  #!/bin/sh
>  PYTHONPATH=/path/to/your/shared/lib:$PYTHONPATH
>  export PYTHONPATH
>  python -m your.module.name ${1+"$@"}
>
> Such a script has the advantage that the $PYTHONPATH change applies only to
> the python running your module, not to your users' wider environment.

This looks doable, though it complicates how I will call my python
programs.  These are always invoked by the Pinnacle HotScript
language.  Everything must start from within the Pinnacle planning
environment.


-- 
boB