[Tutor] What is the easiest way to ensure the current working directory is the same as the directory where the main program is saved?

[Cameron Simpson]

On 26Jun2021 14:14, boB Stepp <robertvstepp at gmail.com> wrote:
>Please forgive the top posting in this instance.  I want to ensure
>Dennis' response gets saved intact into the Tutor archives so I can
>easily find it later as his postings otherwise never show up unless
>someone responds to them.

Aye. I was going to reply to you last night, but Dennis' post is far far 
better.

>But what about during a program's development phase?  It is handy to
>have data files in the same project directory space as everything
>else.

Usually using an environment variable to override the default location.  
For example, I've a little MP3 parser whose associated tests script has 
this line:

    TESTFILE = os.environ.get('TEST_CS_MP3_TESTFILE', 'TEST.mp3')

near the start. That sets the TESTFILE variable to 'TEST.mp3' unless an 
environment variable overrides it; I can set $TEST_CS_MP3_TESTFILE to 
the path of a different test file if I want. Plenty of programmes use 
this approach. The envvar above is weirdly named because it is so 
special purpose, but something like $APPNAME_DATADIR replacing APPNAME 
with the name of your own command would be very normal. Then you can go:

    datadir_path = os.environ.get('MYAPPNAME_DATADIR')
    if not datadir_path:
        datadir_path = expanduser('~/.myappname/data')

to look for an override, and fall back to the "normal" path without one.

When I'm testing in dev, I have a little script which sets the 
"development environment": a little set of environment variables I want 
in place _when in dev_. That way standing in the dev area does not 
change behaviour (I use the "installed" tools), but to run something "in 
dev" I go:

    dev some command ...

where "dev" is my little script to set up the environment _just for that 
command_. For example:

    dev python3 -m my.module.name ...

That would typically:
- set $PYTHONPATH to access the modules in the dev environment, eg a 
  local virtualenv and of course my own code there
- set $PATH to access the bin directories of the development 
  environment, such as the local "bin" and the "bin' from a local 
  virtualenv
- any special variables like $TEST_CS_MP3_TESTFILE or $MYAPPNAME_DATADIR 
  imagined above

I'm in the UNIX world, but I'd expect it should be fairly easy to write 
yourself a little batch file to do this in Windows and achieve exactly 
the same thing.

>I have been doing some online searching and it appears that
>during an installation process that installers can copy data and
>configuration files to those standard locations that you are
>advocating.  But again I know nothing about installing Python programs
>and the software that gets it done.  I merely know how to use pip to
>install applications others have developed, not how to get my code
>ready for such a procedure.

You don't need to use pip for your own code. I've got quite a lot of 
personal modules, and only a subset is published to PyPI for use by pip.

The typical approach is (again, UNIX nomenclature):
- make a virtualenv for development, typically in a "venv" subdirectory 
  of your development tree
- set $PATH and $PYTHONPATH to use it

In UNIX that would go:

    python3 -m venv venv

using the python 'venv" module to make a virtualenv install in "venv".  
Do that once. Install third party modules for your dev env like this:

    ./venv/bin/pip3 install module_names....

Keep your own modules sitting around _outside_ the venv. I use the 
"lib/python" subdirectory myself ("cs" prefixes most of my modules, so 
make that too):

    mkdir -p lib/python/cs

The "dev" environment setup is then (UNIX shell syntax, again):

    PYTHONPATH=$(pwd)/lib/python
    PATH=$(pwd)/venv/bin:$PATH
    export PYTHONPATH PATH

That puts the venv ahead of other stuff via $PATH, and makes python look 
for your modules in the lib/python subdirectory.

My personal "dev" batch script essentially then looks like the above:

    #!/bin/sh
    PYTHONPATH=$(pwd)/lib/python
    PATH=$(pwd)/venv/bin:$PATH
    export PYTHONPATH PATH
    set -x
    exec "$@"

which tweaks the environment, then runs whatever was passed on the 
command line, so that:

    dev python3 -m my_module blah...

runs my in-development module "my_module" with the right environment 
settings.

There's no "install" step for your stuff - you're just tweaking 
$PYTHONPATH to look for it in the right place. "lib/python" is a 
personal foible; whatever works for you will do.

Cheers,
Cameron Simpson <cs at cskk.id.au>