[Python-Dev] Question about sys.path and sys.argv and how packaging (may) affects default values

[Michael Felt]

On 2016-03-02 18:45, Thomas Wouters wrote:
> On Wed, Mar 2, 2016 at 3:50 AM, Michael Felt<aixtools at gmail.com>  wrote:
>
>> Hello all,
>>
>> 1) There are many lists to choose from - if this is the wrong one for
>> questions about packaging - please forgive me, and point me in the right
>> direction.
>>
> It's hard to say where this belongs best, but python-list would probably
> have done as well.
>
>
>> 2) Normally, I have just packaged python, and then moved on. However,
>> recently I have been asked to help with packaging an 'easier to install'
>> python by people using cloud-init, and more recently people wanting to use
>> salt-stack (on AIX).
>>
>> FYI: I have been posting about my complete failure to build 2.7.11 (
>> http://bugs.python.org/issue26466) - so, what I am testing is based on
>> 2.7.10 - which built easily for me.
>>
>> Going through the 'base documentation' I saw a reference to both sys.argv
>> and sys.path. atm, I am looking for an easy way to get the program name
>> (e.g., /opt/bin/python, versus ./python).
>> I have my reasons (basically, looking for a compiled-in library search
>> path to help with http://bugs.python.org/issue26439)
>>
> I think the only way to get at the compiled-in search path is to recreate
> it based on the compiled-in prefix, which you can get through distutils.
> Python purposely only uses the compiled-in path as the last resort.
> Instead, it searches for its home relative to the executable and adds a set
> of directories relative to its home (if they exist).
>
> It's not clear to me why you're focusing on these differences, as (as I
> describe below) they are immaterial.
>
>
>> Looking on two platforms (AIX, my build, and debian for power) I am
>> surprised that sys.argv is empty in both cases, and sys.path returns
>> /opt/lib/python27.zip with AIX, but not with debian.
>>
> When you run python interactively, sys.argv[0] will be '', yes. Since
> you're not launching a program, there's nothing else to set it to. 'python'
> (or the path to the executable) wouldn't be the right thing to set it to,
> because python itself isn't a Python program :)
>
> The actual python executable is sys.executable, not sys.argv[0], but you
> shouldn't usually care about that, either. If you want to know where to
> install things, distutils is the thing to use. If you want to know where
> Python thinks it's installed (for debugging purposes only, really),
> sys.prefix will tell you.
>
>
>> root at x064:[/data/prj/aixtools/python/python-2.7.10]/opt/bin/python
>> Python 2.7.10 (default, Nov  3 2015, 14:36:51) [C] on aix5
>> Type "help", "copyright", "credits" or "license" for more information.
>>>>> import sys
>>>>> sys.argv
>> ['']
>>>>> sys.path
>> ['', '/opt/lib/python27.zip', '/opt/lib/python2.7',
>> '/opt/lib/python2.7/plat-aix5', '/opt/lib/python2.7/lib-tk',
>> '/opt/lib/python2.7/lib-old', '/opt/lib/python2.7/lib-dynload',
>> '/opt/lib/python2.7/site-packages']
>>
>> michael at ipv4:~$ python
>> Python 2.7.9 (default, Mar  1 2015, 13:01:00)
>> [GCC 4.9.2] on linux2
>> Type "help", "copyright", "credits" or "license" for more information.
>>>>> import sys
>>>>> sys.argv
>> ['']
>>>>> sys.path
>> ['', '/usr/lib/python2.7', '/usr/lib/python2.7/plat-powerpc-linux-gnu',
>> '/usr/lib/python2.7/lib-tk', '/usr/lib/python2.7/lib-old',
>> '/usr/lib/python2.7/lib-dynload', '/usr/local/lib/python2.7/dist-packages',
>> '/usr/lib/python2.7/dist-packages',
>> '/usr/lib/python2.7/dist-packages/PILcompat',
>> '/usr/lib/python2.7/dist-packages/gtk-2.0', '/usr/lib/pymodules/python2.7']
>>
> In sys.path, you're seeing the difference between a vanilla Python and
> Debian's patched Python. Vanilla Python adds $prefix/lib/python27.zip to
> sys.path unconditionally, whereas Debian removes it when it doesn't exist.
> Likewise, the dist-packages directory is a local modification by Debian; in
> vanilla Python it's called 'site-packages' instead. The subdirectories in
> dist-packages that you see in the Debian case are added by .pth files
> installed in $prefix -- third-party packages, in other words, adding their
> own directories to the module search path.
>
>
>> And I guess I would be interested in getting
>> '/opt/lib/python2.7/dist-packages' in there as well (or learn a way to
>> later add it for pre-compiled packages such as cloud-init AND that those
>> would also look 'first' in /opt/lib/python2.7/dist-packages/cloud-init for
>> modules added to support cloud-init - should I so choose (mainly in case of
>> compatibility issues between say cloud-init and salt-stack that have common
>> modules BUT may have conflicts) - Hopefully never needed for that reason,
>> but it might also simplify packaging applications that depend on python.
>>
> A vanilla Python (or non-Debian-built python, even) has no business looking
> in dist-packages. It should just use site-packages. Third-party packages
> shouldn't care whether they're installed in site-packages or dist-packages,
> and instead should use distutils one way or another (if not by having an
> actual setup.py that uses distutils or setuptools, then at least by
> querying distutils for the installation directory the way python-config
> does).
>
>
>> Many thanks for your time and pointers into the documentation, It is a bit
>> daunting :)
>>
>> Michael
>> _______________________________________________
>> Python-Dev mailing list
>> Python-Dev at python.org
>> https://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe:
>> https://mail.python.org/mailman/options/python-dev/thomas%40python.org
>>
>
>
Many Thanks Thomas for the extensive answer.

My question is to help me understand where to look for default libraries 
in order to work on a patch.
That has come a long way, but it is stuck now with/by something else I 
need to learn to debug (to find where the non-zero exit status comes 
from during a build).

As a packager I hope to be as 'vanilla' as possible from the python 
perspective.

Another time (another list!) I shall ask about what goes into python.zip

Michael