Mailman 3 March 2014 - Distutils-SIG

OS X and PEP 425 / wheels
by MinRK 08 Mar '14

08 Mar '14

Hello, I proposed a patch <https://github.com/pypa/pip/pull/1465> to pip, with respect to treatment of the platform tag on OS X, and Chris Barker proposed that I bring the discussion here. The situation: PEP 425 describes the platform tag as: The platform tag is simply distutils.util.get*platform() with all hyphens - and periods . replaced with underscore *. but the PEP makes no mention of what should be done on OS X. On OS X, get_platform() has the form: macosx_10_6_intel where 10_6 indicates that the OS X deployment target (of Python) is 10.6, and intel indicates that it is a fat binary with both x86_64 and i386. Other values for the arch include ‘ppc’, ‘ppc64’, and ‘universal’, another multi-arch tag indicating a fat binary with all four of 32/64bit intel and ppc. Where PEP 425 causes problems (or pip’s implementation thereof), is in testing strict equality of this value, not taking into account the meaning of its contents. I proposed two changes: 1. Use >= comparison for the deployment target From the OS X platform documentation<https://developer.apple.com/library/mac/documentation/DeveloperTools/Concep…> : Choose a deployment target. This identifies the earliest OS version on which your software can run. so a binary with deployment target = 10.6 should be installable on OS X >= 10.6. This change restores the longstanding behavior of easy_install, where 10.6 eggs are installable on >= 10.6. 2. support multi-arch names (intel, universal) on their respective components - intel is valid on {x86_64, i386} - universal is valid on {intel, x86_64, i386, ppc64, ppc} easy_install, like pip, also does strict comparison here, so this would be new behavior. My actual use case: I have a wheel (pyzmq), which works on any intel-based Python targeting OS X >= 10.6. To express this with pip-1.5, the filename has to be: pyzmq-14.1.0-cp27-none-macosx_10_6_intel.macosx_10_6_x86_64.macosx_10_6_i386.macosx_10_7_intel.macosx_10_7_x86_64.macosx_10_7_i386.macosx_10_8_intel.macosx_10_8_x86_64.macosx_10_8_i386.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_9_i386.whl and it has to grow every time there is a new OS release. With both proposed changes, I could communicate the same information with just one tag: pyzmq-14.1.0-cp27-none-macosx_10_6_intel.whl With just the first change, it would need three: pyzmq-14.1.0-cp27-none-macosx_10_6_intel.macosx_10_6_x86_64.macosx_10_6_i386.whl Any feedback from experts, especially if my understanding of deployment targets or fat binaries is incorrect, would be much appreciated. Thanks, -MinRK

7 8

Warehouse XMLRPC
by Donald Stufft 07 Mar '14

07 Mar '14

Just a quick FYI that the last missing piece (search) for XMLRPC was merged in Warehouse today. If you have an application that depends on XMLRPC on PyPI and you’d like to test it, please try against the url: https://pypi-preview.a.ssl.fastly.net/pypi Let me know if you find any issues, or you can file a bug at: https://github.com/pypa/warehouse ----------------- Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

4 10

The PEP 426 defined metadata version will be metadata 3.0
by Nick Coghlan 04 Mar '14

04 Mar '14

Vinay brought to my attention that bdist_wheel currently publishes pydist.json files that claim to contain content in the metadata 2.0 format. I was aware that Daniel had been experimenting with generating pydist.json files, but I didn't realise that bdist_wheel did it by default, nor that it used the real file name from the PEP rather than a separate implementation specific file. That's broken: it means that metadata format 2.0 can now never be well defined, because we already have these experimental (and ultimately non-conformant, since there have been backwards incompatible changes to the draft spec) files floating around. I will accordingly be updating the defined metadata version in PEP 426 to 3.0, and including an explicit admonition to *never* include experimental metadata (whether in the base format or as part of an experimental extension) in the main pydist.json file. Experimental metadata should only ever appear in tool-specific files (which don't need to guarantee any kind of interoperability with other tools). In the meantime: please don't publish pydist.json files, use some other filename if you want to experiment with JSON based metadata in advance of the acceptance of PEP 426. Regards, Nick. -- Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia

6 34

Remaining tasks before metadata 2.0 can be accepted (was Re: The PEP 426 defined metadata version will be metadata 3.0)
by Nick Coghlan 03 Mar '14

03 Mar '14

On 27 February 2014 10:46, Marcus Smith <qwcode(a)gmail.com> wrote: > that would be good. If you did, I would link to the tasks from the PUG > future page. OK, these are the things I consider blockers for an accepted metadata 2.0 spec: https://bitbucket.org/pypa/pypi-metadata-formats/issues?priority=blocker&st… Finalising PEP 426/440/459 is on me. At this point, I think that consists of *taking away* things that aren't yet settled (specifically metadata hooks), so we can see how far this next iteration actually gets us before trying to solve the remaining problems that need some kind of trigger support. A required preliminary task is to create a revision of PEP 425 that expands its scope to also handle the parts of the file/directory naming scheme that are common across sdist, wheel and the installation database (with compatibility tags becoming a subsection), as well as fixing the definition of the compatibility tags to better handle Windows and Python 2.x binary extensions. (There's a separate non-blocker issue for better Linux/POSIX support - building from source is far more common there, and both conda and Linux distro packages remain available as a near-term workaround for the lack of upstream binary packages) The other blockers are then sdist 2.0, wheel 1.1 and a second revision of the installation database format. There are more issues in that repo, but they're ones that I don't consider *essential* as part of a usable metadata 2.0 spec - they're about things like Linux binary support, distributing full applications in addition to libraries and handling things that may require the ability to run code at installation time. Once metadata 2.0 itself is published, we can likely explore several of them as extensions before committing to anything in the core PEPs. Cheers, Nick. -- Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia

2 4

Deferring metadata hooks
by Nick Coghlan 02 Mar '14

02 Mar '14

I've just posted updated versions of PEP 426 and 459 that defer the "metadata hooks" feature. The design and behaviour of that extension is still way too speculative for me to approve in its current form, but I also don't want to hold up the rest of the changes in metadata 2.0 while we thrash out the details of a hook system. That said, I still don't want us to get into a situation where someone later publishes a wheel file that expects metadata hook support and older tools silently install it without running the hooks. Accordingly, the revised PEP 426 adds a single simpler feature to the extensions system: the idea of a "required extension". If a project sets that flag for an extension (by including "required_extension": true in the extension metadata), and an installation tool doesn't understand it, then the tool is required to either fail the installation attempt entirely or else fall back to installing from source. That way, project authors will be able to distinguish between "these metadata hooks are just an optimisation, things will still work if you don't run them" and "if you don't run these hooks, your installation will be broken". I think this approach may also encourage a design where projects do something sensible *by default* (e.g. NumPy defaulting to SSE2) and then use the (not yet defined) post-installation hooks to potentially *change away* from the default to something more optimised for that particular system (e.g. NumPy overwriting itself with an SSE3 version), while still *allowing* developers to refuse to let the software install if the metadata hooks won't be run. Regards, Nick. P.S. The draft PEP for metadata hooks is still available at https://bitbucket.org/pypa/pypi-metadata-formats/src/default/metadata-hooks… -- Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia

2 3

[issue158] Python not recognized in registry, PATH not modified
by mrx23dot 02 Mar '14

02 Mar '14

New submission from mrx23dot: setuptools Windows installer doesn't look for HKEY_LOCAL_MACHINE: only looks in: HKEY_CURRENT_USER for Python: [HKEY_LOCAL_MACHINE\Software\Python\PythonCore\2.7] (problem on Win7 x64 Python 2.7 installed for All users) Also it doesn't add Python\Scripts to PATH. So cmd> installed_prg_name is not recognized We have to add manually. ---------- messages: 745 nosy: mrx23dot priority: critical status: unread title: Python not recognized in registry, PATH not modified _______________________________________________ Setuptools tracker <setuptools(a)bugs.python.org> <http://bugs.python.org/setuptools/issue158> _______________________________________________

1 0

Is there any sense to put setuptools as a requirement?
by Piotr Dobrogost 02 Mar '14

02 Mar '14

Hi! I've seen people putting 'setuptools' in 'install_requires' in setup.py starting with import from setuptools like this: from setuptools import setup, find_packages Does it make any sense? In what circumstances should 'setuptools' be listed in 'install_requires' or 'setup_requires'? Regards, Piotr Dobrogost

5 4