[Distutils] Can't count on -S to get a clean Python (at least on mac)

[Jim Fulton]

On Sat, Apr 7, 2012 at 6:42 PM, Zvezdan Petkovic <zvezdan at computer.org> wrote:
> I know what you mean when you say clean Python.
>
> However, for vendors, clean Python is the distribution that supports
  all their features and does not break their promises. Â They do not
  promise Python with no additional extra packages. Â For example,
  Apple has the support for the scripting bridge towards Objective-C
  libraries, py2app builder, and probably some other features that are
  obviously important to them.

Yup.  In some ways, a vendor-supplied (aka "system") Python is like an
application or application platform with lots of specific components
that the OS may depend on.

>
> On Apr 7, 2012, at 4:24 PM, Jim Fulton wrote:
>
>> On my mac (Snow Leopard):
>>
>> $ which python
>> /usr/bin/python
>>
>> $ python -S
>> Python 2.6.1 (r261:67515, Jun 24 2010, 21:47:49)
>> [GCC 4.2.1 (Apple Inc. build 5646)] on darwin
>>>>> import setuptools
>>>>> setuptools.__file__
>> '/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/setuptools/__init__.pyc'
>>>>
>
>
> The only thing this shows is that setuptools are in the Extras and
  that Extras are obviously always in sys.path.

Yup.

>
> This doesn't mean that site.py was loaded.

I never said it was.

> Do notice that site-packages are not included with python -S:

But, as you and I pointed out, Extras is and contains lots of
non-standard stuff that you can't avoid using a platform-independent
mechanism.

...

>
> The python2.6 man page says this:
>
> -S   Disable the import of the module site and the site-dependent
>    manipulations of sys.path that it entails.
>
> According to the tests above that promise is fulfilled.
> Nothing site dependent was loaded.

My point was that using -S didn't yield a (even a relatively) clean
Python. (Check the subject of this thread.)

For years, some people have been confused and annoyed at my assertion
that applications shouldn't be built on system Pythons because the
results are too unpredictable.

At the Python language summit in Atlanta last year, I suggested that
it would be useful to define a standard for a baseline Python that
applications could depend on. System packagers could provide this
alongside their loaded Python installations. At the time, it was
suggested that all that was needed was to to use -S to avoid
system-supplied custom packages. There was even a change made in 3.3
to make the -S option more reliable by allowing useful utilities
previously available in site.py available without importing it.
(In most versions of Python, even if -S is used, site.py may get
imported by something trying to get at one of these utilities.)

Of course, there were other issues like unicode sizes (seemingly
addresses by recent unicode enhancements) and other build options, but
it was thought that extra libraries were the biggest issue.

>> whimper
>
> Your expectations of the -S option might be too high. :-)

My expectations weren't too high, but I suspect other's are.

>> I haven't been able to reproduce this lamosity on any linux systems.
>
>
> Probably because they are not trying to support a scripting bridge
  or an app builder. Perhaps they do, but don't care if it breaks when
  -S is used. :-) I'm pretty sure I've seen python installations on
  Linux that had their own warts.

I think it's more likely because they used the correct mechanism and
add extras to the path in site.py.  I would call the Apple packaging a
bug.

> I know you are trying to do something with this (probably for
> buildout) and that not being able to rely on -S is frustrating, to
> put it mildly.  I'm not arguing that, I just tried to point out what
> I thought was behind this behavior.

Buildout 1.5 tries hard to leverage -S to provide isolation.  The new
buildout will as well, although it won't be very effective.

The position of the new (simpler and more maintainable buildout) is
that buildout will provide facilities for working with system Pythons,
but it will make few if any promises and it won't work nearly as hard
as zc.buildout 1.5 does.  The position will be that if you're
building non-trivial applications, you'd better use a clean Python.

I still think it would be useful for the Python community/PSF to
define a "clean" or baseline Python for applications and work out how
such a thing could live alongside a loaded/extra-batteries-included
installation.

Jim

--
Jim Fulton
http://www.linkedin.com/in/jimfulton
Jerky is better than bacon! http://www.dublinstore.com/