[Python-Dev] Question about sys.path and sys.argv and how packaging (may) affects default values

[Thomas Wouters]

On Wed, Mar 2, 2016 at 3:50 AM, Michael Felt <aixtools at gmail.com> wrote:

> Hello all,
>
> 1) There are many lists to choose from - if this is the wrong one for
> questions about packaging - please forgive me, and point me in the right
> direction.
>

It's hard to say where this belongs best, but python-list would probably
have done as well.


>
> 2) Normally, I have just packaged python, and then moved on. However,
> recently I have been asked to help with packaging an 'easier to install'
> python by people using cloud-init, and more recently people wanting to use
> salt-stack (on AIX).
>
> FYI: I have been posting about my complete failure to build 2.7.11 (
> http://bugs.python.org/issue26466) - so, what I am testing is based on
> 2.7.10 - which built easily for me.
>
> Going through the 'base documentation' I saw a reference to both sys.argv
> and sys.path. atm, I am looking for an easy way to get the program name
> (e.g., /opt/bin/python, versus ./python).
> I have my reasons (basically, looking for a compiled-in library search
> path to help with http://bugs.python.org/issue26439)
>

I think the only way to get at the compiled-in search path is to recreate
it based on the compiled-in prefix, which you can get through distutils.
Python purposely only uses the compiled-in path as the last resort.
Instead, it searches for its home relative to the executable and adds a set
of directories relative to its home (if they exist).

It's not clear to me why you're focusing on these differences, as (as I
describe below) they are immaterial.


> Looking on two platforms (AIX, my build, and debian for power) I am
> surprised that sys.argv is empty in both cases, and sys.path returns
> /opt/lib/python27.zip with AIX, but not with debian.
>

When you run python interactively, sys.argv[0] will be '', yes. Since
you're not launching a program, there's nothing else to set it to. 'python'
(or the path to the executable) wouldn't be the right thing to set it to,
because python itself isn't a Python program :)

The actual python executable is sys.executable, not sys.argv[0], but you
shouldn't usually care about that, either. If you want to know where to
install things, distutils is the thing to use. If you want to know where
Python thinks it's installed (for debugging purposes only, really),
sys.prefix will tell you.


>
> root at x064:[/data/prj/aixtools/python/python-2.7.10]/opt/bin/python
> Python 2.7.10 (default, Nov  3 2015, 14:36:51) [C] on aix5
> Type "help", "copyright", "credits" or "license" for more information.
> >>> import sys
> >>> sys.argv
> ['']
> >>> sys.path
> ['', '/opt/lib/python27.zip', '/opt/lib/python2.7',
> '/opt/lib/python2.7/plat-aix5', '/opt/lib/python2.7/lib-tk',
> '/opt/lib/python2.7/lib-old', '/opt/lib/python2.7/lib-dynload',
> '/opt/lib/python2.7/site-packages']
>
> michael at ipv4:~$ python
> Python 2.7.9 (default, Mar  1 2015, 13:01:00)
> [GCC 4.9.2] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
> >>> import sys
> >>> sys.argv
> ['']
> >>> sys.path
> ['', '/usr/lib/python2.7', '/usr/lib/python2.7/plat-powerpc-linux-gnu',
> '/usr/lib/python2.7/lib-tk', '/usr/lib/python2.7/lib-old',
> '/usr/lib/python2.7/lib-dynload', '/usr/local/lib/python2.7/dist-packages',
> '/usr/lib/python2.7/dist-packages',
> '/usr/lib/python2.7/dist-packages/PILcompat',
> '/usr/lib/python2.7/dist-packages/gtk-2.0', '/usr/lib/pymodules/python2.7']
>

In sys.path, you're seeing the difference between a vanilla Python and
Debian's patched Python. Vanilla Python adds $prefix/lib/python27.zip to
sys.path unconditionally, whereas Debian removes it when it doesn't exist.
Likewise, the dist-packages directory is a local modification by Debian; in
vanilla Python it's called 'site-packages' instead. The subdirectories in
dist-packages that you see in the Debian case are added by .pth files
installed in $prefix -- third-party packages, in other words, adding their
own directories to the module search path.


>
> And I guess I would be interested in getting
> '/opt/lib/python2.7/dist-packages' in there as well (or learn a way to
> later add it for pre-compiled packages such as cloud-init AND that those
> would also look 'first' in /opt/lib/python2.7/dist-packages/cloud-init for
> modules added to support cloud-init - should I so choose (mainly in case of
> compatibility issues between say cloud-init and salt-stack that have common
> modules BUT may have conflicts) - Hopefully never needed for that reason,
> but it might also simplify packaging applications that depend on python.
>

A vanilla Python (or non-Debian-built python, even) has no business looking
in dist-packages. It should just use site-packages. Third-party packages
shouldn't care whether they're installed in site-packages or dist-packages,
and instead should use distutils one way or another (if not by having an
actual setup.py that uses distutils or setuptools, then at least by
querying distutils for the installation directory the way python-config
does).


>
> Many thanks for your time and pointers into the documentation, It is a bit
> daunting :)
>
> Michael
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/thomas%40python.org
>



-- 
Thomas Wouters <thomas at python.org>

Hi! I'm an email virus! Think twice before sending your email to help me
spread!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20160302/121437a3/attachment.html>