organizing many python scripts, in a large corporate environment.

[bukzor]

On Mar 13, 10:52 pm, "eryksun ()" <eryk... at gmail.com> wrote:
> On Sunday, March 13, 2011 7:27:47 PM UTC-4, bukzor wrote:
> >      e) create custom boilerplate in each script that addresses the
> > issues in a-d. This seems to be the best practice at the moment...
>
> The boilerplate should be pretty simple. For example, if the base path is the parent directory, then the following works for me:
>
> import os.path
> import sys
> base = os.path.dirname(os.path.dirname(__file__))
> sys.path.insert(0, base)
>
> Of course, you need to have the __init__.py in each subdirectory.

I've written this many times. It has issues. In fact, I've created a
library for this purpose, for the following reasons.

What if your script is compiled? You add an rstrip('c') I guess.
What if your script is optimized (python -O). You add an rstrip('o')
probably.
What if someone likes your script enough to simlink it elsewhere? You
add a realpath().
In some debuggers (pudb), __file__ is relative, so you need to add
abspath as well.
Since you're putting this in each of your scripts, it's wise to check
if the directory is already in sys.path before inserting.
We're polluting the global namespace a good bit now, so it would be
good to wrap this in a function.
To make our function more general (especially since we're going to be
copying it everywhere), we'd like to use relative paths.

In total, it looks like below after several years of trial, error, and
debugging. That's a fair amount of code to put in each script, and is
a maintenance headache whenever something needs to be changed. Now the
usual solution to that type of problem is to put it in a library, but
the purpose of this code is to give access to the libraries... What I
do right now is to symlink this library to all script directories to
allow them to bootstrap and gain access to libraries not in the local
directory.

I'd really love to delete this code forever. That's mainly why I'm
here.


#!/not/executable/python
"""
This module helps your code find your libraries more simply, easily,
reliably.

No dependencies apart from the standard library.
Tested in python version 2.3 through 2.6.
"""

DEBUG = False

#this is just enough code to give the module access to the libraries.
#any other shared code should go in a library

def normfile(fname):
    "norm the filename to account for compiled and symlinked scripts"
    from os.path import abspath, islink, realpath
    if fname.endswith(".pyc") or fname.endswith(".pyo"):
        fname = fname[:-1]
    if fname.startswith('./'):
        #hack to get pudb to work in a script that changes directory
        from os import environ
        fname = environ['PWD'] + '/' + fname
    if islink(fname):
        fname = realpath(fname)
    return abspath(fname)

def importer(depth=1):
    "get the importing script's __file__ variable"
    from inspect import getframeinfo
    frame = prev_frame(depth + 1)
    if frame is None:
        return '(none)'
    else:
        return normfile( getframeinfo(frame)[0] )

def prev_frame(depth=1):
    "get the calling stack frame"
    from inspect import currentframe
    frame = currentframe()
    depth += 1
    try:
        while depth:
            frame = frame.f_back
            depth -= 1
    except (KeyError, AttributeError):
        frame = None
    return frame


def here(depth=1):
    "give the path to the current script's directory"
    from os.path import dirname
    return dirname(importer(depth))

def use(mydir, pwd=None):
    """
    add a directory to the Python module search path
    relative paths are relative to the currently executing script,
unless specified otherwise
    """
    from os.path import join, normpath
    from sys import path

    if not pwd:
        pwd = here(2)
    if not mydir.startswith("/"):
        mydir = join(pwd, mydir)
    mydir = normpath(mydir)
    path.insert(1, mydir)
    return mydir