[Pythonmac-SIG] PyObjC and macOS 10.12 (Sierra)

[Andrew Jaffe]

On 06/10/2016 20:26, Glyph Lefkowitz wrote:
>> On Oct 6, 2016, at 2:56 AM, Andrew Jaffe  wrote:
>> On 17/09/2016 18:59, Glyph Lefkowitz wrote:
>>>> On Sep 17, 2016, at 9:27 AM, Ned Deily  wrote:
>>>> On 2016-09-13 19:33, Glyph Lefkowitz wrote:
>>>>>> On Sep 13, 2016, at 3:35 PM, Andrew Jaffe wrote:
>>>>>>
>>>>>> $ ls -lt /Library/Python/2.7/site-packages/
>>>>>> total 0
>>>>>> -rwxr-xr-x  1 root  wheel  157 31 Jul 02:36 Extras.pth*
>>>>>> -rw-r--r--  1 root  wheel  119 31 Jul 02:36 README
>>>>>> $ more /Library/Python/2.7/site-packages/Extras.pth
>>>>>> /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python
>>>>>> /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/PyObjC
>>>>>>
>>>>> Hah!  Thanks for sharing.  Very satisfying to actually make a *correct*
>>>>> prediction about setuptools' behavior :)
>>>>
>>>> This seems to be Apple's doing.  AFAICT, 10.12 is shipping with this
>>>> Extras.pth file in /Library/Python/2.7; it's something new.  And,
>>>> unfortunately, due to https://bugs.python.org/issue4865, the
>>>> site-packages directory for the system Python 2.7 is included in
>>>> sys.path along with the non-system framework Python site-packages.
>>>
>> So, a little more data:
>>
>> If you rename or remove /Library/Python/2.7/site-packages/Extras.pth
>> then pip2 works.
>
> What do you mean by "works"?  Your original error is pip refusing to
> upgrade pyObjC because to upgrade pyObjC it has to delete the old
> version, and pyObjC is part of the operating system, and it can't delete
> the installed version.  This is correct; the error reporting could be
> nicer, but pip is not broken.  Don't mess with files in /System.
>
> The suggestion to use virtualenv isn't really optional.  If you really,
> really want things to be importable by a bare 'python', not inside a
> venv, `pip install --user` is another option you might have.
>
>> *However*, lots of other stuff breaks -- anything that uses Apple's
>> python and relies on access to pyobjc and the frameworks (e.g.,
>> TextMate's latex package).
>
> Yep, that's expected behavior.  This is exactly why Apple is making it
> increasingly difficult to screw up /System.  Apps can, should, and do
> rely upon the libraries installed on the system.
>
>> What I don't understand is: what changed from Yosemite? This file did
>> not exist before Sierra, but there were no problems with (Apple)
>> python accessing these packages.
>
> Do you mean from El Capitan?

Yes, sorry.

> This file previously existed in a different location, and (while the
> particulars stubbornly refuse to stick to memory for me, for some
> reason) this new location is the one generally preferred by the python
> packaging maintainers.
>
>> (Or is there something unique in my setup that is causing this? I kind
>> of doubt it, but it's possible...)
>>
>> Does anyone have any insight?
>
> If you really, really, really want to do this in such a way that /System
> stuff is only present for /System's python and not for python.org
> <http://python.org>'s, you can take advantage of the 'import' hack
> <https://docs.python.org/2/library/site.html>, and
> rewrite /Library/Python/2.7/site-packages/Extras.pth to _conditionally_
> add those entries to sys.path, checking sys.executable or some other
> fingerprint.
>
> But: don't.  Given that system python and python.org <http://python.org>
> python share /Library and ~/Library, they're not /really/ distinct
> environments, and things installed into one will show up in the other,
> so excluding the /System directories on one but not the other will just
> cause other, more confusing issues.
>
> In conclusion: just use virtualenv.  This is not a problem that should
> be fixed.

I agree that this would solve all of the problems, at the cost of some 
minor startup pain.

But one nice thing about the python.org builds is that, for the last few 
releases, they just worked out of the box, where by "worked" I mean that 
(as far as I can tell) the system Python saw its own packages, and the 
python.org build saw its own packages, and (python.org) pip didn't seem 
to care about the system files. And I suppose I still don't understand 
exactly why that changed with Sierra, and whether the change is actually 
beneficial in any way whatever! (And whether it could only be fixed by 
Apple, or whether a change in the python.org build could do something 
about it.)

Sorry to keep harping on about it, but I find it hard to believe this is 
not a fairly widespread problem! (Well, I found at least one other 
complaint: 
https://jasonkratz.com/2016/09/25/python-2-7-on-the-mac-site-package-weirdness/)

Andrew