[Distutils] setuptools-0.4a2: Eggs, scripts, and __file__

Phillip J. Eby pje at telecommunity.com
Mon Jun 13 07:15:38 CEST 2005

At 12:16 AM 6/13/2005 -0400, Ryan Tomayko wrote:
> From "[Distutils] A plan for scripts (in EasyInstall)"
>On Mon Jun 6 16:44:19 CEST 2005, Phillip J. Eby wrote:
> > The 'run_main' function would do several things:
> >
> >   * Clear everything but __name__ from the __main__ namespace
> >   * exec the script in __main__, using something like:
> >
> >      maindict['__file__'] = pseudo_filename code =
> >      compile(script_source, pseudo_filename, "exec") exec code in
> >      maindict, maindict
>It seems that bullet one is happening but not bullet two? I have
>scripts that are attempting to use __file__ but failing with:
>     NameError: name '__file__' is not defined

I'll fix this in the next release.

>The script looks like it would work properly if it was given a pseudo
>filename but this has me thinking about what the best way to detect
>development environments in scripts will look like in an eggified

That's the wrong question to ask, IMO.  Think about how to make the script 
work exactly the same in all environments, instead.  :)

>My wrapper scripts generally look something like the
>following to detect whether the script is being run from a
>development location or a deployed location::
>     import sys
>     from os.path import dirname, abspath, join, exists
>     devel_dir = dirname(dirname(abspath(__file__)))
>     if exists(join(devel_dir, 'support')):
>          sys.path.insert(1, join(devel_dir, 'support'))
>          sys.path.insert(1, devel_dir)
>     from package.module import main
>     main(sys.argv)
>Assuming a directory layout of:
>     [devel-dir]/bin (scripts in here)
>     [devel-dir]/support (packages/modules)
>     [devel-dir]/[package-name]

Have you read this:


The complexity you're incurring here is unnecessary; between require() and 
.pth files you should never need to mess with sys.path manually.

>At first I thought I should switch from using path operations on
>__file__ to using `pkg_resources.resource_isdir` and
>`resource_filename` but that doesn't make any sense - if the script
>is running from within a deployed egg, I'm not using it from a
>development environment

Not true; see the link above.  When you do development from your distutils 
package root, your development code *is* in an egg.  However, you still 
shouldn't be checking __file__ or fiddling with sys.path, and there is no 
need anyway.  Here's one idea of what your tree could look like:




     MyPackage.egg-info/   <- this makes devel_dir a "source egg" for MyPackage

         MyPackage-1.1.egg  <- built egg; not actually used for development

You'll notice that everything is just thrown into the same directory; if 
you use this setup for development, everything will "just work".

So, you create a setup.py for your package, and you run setup.py bdist_egg; 
this will dump an egg in dist/, and create MyPackage.egg-info, marking 
devel_dir as a "development egg".

Install any other packages you need to the current directory using 
'easy_install -xd. package_you_need' (the -x excludes their scripts).  Now 
you're ready to party.  Make your scripts use 'require()' to ask for 
'MyPackage'; when you run them (whether you are in the devel_dir or not), 
they will find MyPackage.egg-info, find the dependencies declared, and add 
all the needed .eggs to sys.path automatically.

This is just *one* layout that works.  The dependency eggs don't have to be 
in devel_dir; they can be anywhere that will already be on sys.path, like 
site-packages or a directory named in a .pth file in site-packages.

Does this explain it better?  One side benefit of egg-based installation is 
that you can dump as many libraries in site-packages as you want and not 
worry about version conflicts, so it's definitely how I plan to do most 
development.  The directory where a script is located, however, takes 
precedence over site-packages, which means that even if you have the 
package you're developing already installed in site-packages, your 
development egg will take precedence if the script you're running is in 
that directory.

>  and the resource_*** functions don't make
>sense in __main__ context anyway. So my current thinking is that the
>existing idiom should remain and that a pseudo filename shouldn't
>pose any problems in the scenarios I'm dealing with:
>  * Deployed egg: don't tamper with sys.path
>  * Deployed site-packages: don't tamper with sys.path
>  * Development environment: insert development paths

You should be able to make that last one work the same way; i.e., without 
tampering with sys.path.  If you can't, please explain your situation 
further, because I want pkg_resources to be able to prevent all future 
sys.path munging by anything but EasyInstall itself, and by extensible 
applications that have to manage plugin directories.

More information about the Distutils-SIG mailing list