[Python-Dev] Use ptyhon -s as default shbang for system python executables/daemons

[Nick Coghlan]

On 24 Mar 2015 07:59, "Gregory P. Smith" <greg at krypto.org> wrote:
>
>
> On Wed, Mar 18, 2015 at 9:48 AM Barry Warsaw <barry at python.org> wrote:
>>
>> On Mar 18, 2015, at 05:31 PM, Victor Stinner wrote:
>>
>> >Does it work to pass command line options to Python in the shebang?
>>
>> Yes, but only one "word", thus -Es or -I.
>>
>> We've often mused about whether it makes sense to have two Pythons on the
>> system.  One for system scripts and another for users.  System Python
>> ('/usr/bin/spython' perhaps) would be locked down and only extensible by
>> system packages.  On Debuntu that might mean no /usr/local on sys.path.
It
>> would also have a much more limited set of installed packages, i.e. only
those
>> needed to support system functionality.
>
>
> I recommend this. It'd be nice to see a common OS distro actually do it.
>
> For a system-only Python lockdown you should remove distutils and pip to
prevent anyone from easily installing anything that isn't a distro package
into it. Possibly even removing its site-packages directory and sys.path
entry all together (things your distro needs to install could mix directly
with the stdlib packages)

The hard part of this is the complexity in getpath.c and the current
difficulty of testing the interaction of all the different configuration
settings properly. Completely overriding the standard path setup isn't too
bad, it's *partially* overriding it that's hard.

That's really one of the main goals of the proposed startup refactoring in
PEP 432 - separating things into distinct phases so the path configuration
is easier to customise and so the current behaviour becomes easier to test.

That's a fairly big project with high barriers to entry though, as even
just wrapping your head around the current accumulated complexity in the
way the default sys.path gets constructed is a time consuming task.

That approach has the virtue of not being a "system Python" specific
solution, it's a "make embedding the CPython runtime much easier" solution,
with "system Python with different default settings" as the reference use
case.

>> /usr/bin/python2 and /usr/bin/python3 then would be user tools, with all
the
>> goodness they currently have.
>>
>> It's never gotten much farther than musings, but protecting the system
against
>> the weird things people install would be a good thing.  OTOH, this feels
a lot
>> like virtual environments so maybe there's something useful to be mined
there.

For distro level work it would be closer to the environment modules used in
high performance computing work (which are also the basis of
softwarecollections.org)

Where things get really messy is on the package management side. Either we
end up with yet another copy of modules that are often already duplicated
between the Python 2 & 3 stacks, or else the relevant user Python needs to
be customised to also be able to see the system level libraries for things
like Kerberos, interfacing with the package manager, dbus, etc.

The former approach = "ugh, no, just no" as far as I'm concerned, while the
latter would be a messy patch to try to carry given the current difficulty
of customising and testing the startup sequence.

>
>
> While people sometimes suggest virtualenv as a solution for this. It
isn't really the same thing. It isn't a hermetic clone of the original
interpreter.  It copies the main binary but symlinks back to much of the
stdlib.  So any existing virtualenv can be an out of date unsupported mix
of both after the original interpreter is updated. This has caused users
problems in the past with some minor version updates where an internal
non-public API used between some binary and a stdlib module was updated as
part of a bugfix. Suddenly they didn't match up for existing virtualenvs.
>
> virtualenv is an amazing hack that I promote to most anyone for their own
applications use with the occasional known caveats (solvable by regurly
rebuilding your virtualenvs)...  But I wouldn't want to see it used for the
core OS itself even though it sounds better at first glance.

Right, this is why Fedora/RHEL/CentOS have software collections instead,
and those have their own "gotchas" that currently make them unsuitable for
use as the main system Python (for example: the Fedora installer & package
manager are both written in Python, so having the main system Python be an
SCL would bring SCL support into the minimal package set, which we wouldn't
want to do. The typical aim is to find ways to get things *out* of that
set, not add new ones)

Cheers,
Nick.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150325/4afaf6c4/attachment-0001.html>