Mailman 3 August 2013 - Distutils-SIG

Retiring mirror g.pypi.python.org
by Ralph Bean 30 Aug '13

30 Aug '13

The admins at g.pypi.python.org/mirror.rit.edu have decided they no longer have the resources to maintain their mirror. They've already taken down the content and would like to be removed from any indexes out there (such as www.pypi-mirrors.org). -Ralph

2 1

Rejecting PEP 439: implicit pip bootstrapping
by Nick Coghlan 30 Aug '13

30 Aug '13

Donald's PEP 453 for explicit pip bootstrapping (by inclusion of a "getpip" module in the standard library, automatically executed during installation by the CPython binary installers) is now available on python.org. Accordingly, I am now formally rejecting the implicit bootstrapping proposal that Richard put together for PEP 439. The explicit bootstrapping proposal in PEP 453 should provide a more reliable experience, both on initial installation and when upgrading both pip and CPython. Discussions on PEP 453 will likely be split - here on distutils-sig to ensure the proposal is acceptable to the pip developers, and on python-dev to ensure it is acceptable to the CPython team. As a proposal with a direct impact on CPython, final pronouncement on PEP 453 will take place on python-dev rather than here. Cheers, Nick. -- Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia

1 0

PEP 449 -- Removal of the PyPI Mirror Auto Discovery and Naming Scheme
by Richard Jones 30 Aug '13

30 Aug '13

Hi all, Given that I believe all outstanding issues with PEP 449 < http://python.org/dev/peps/pep-0449/> have been resolved I will accept it in its current form (last modified August 16) so the immediate changes may be made and publicity of the change can be started. My thanks to everyone involved in trying to make mirroring work, and also for everyone who has contributed to getting this PEP to an acceptable state. Richard

2 1

Multi-version import support for wheel files
by Nick Coghlan 28 Aug '13

28 Aug '13

I'm currently working on the docs for the __main__.__requires__ feature of pkg_resources, and have been generally poking around inside pkg_resources before I started on that. It gave me an idea for a question that has come up a few times: how should we support parallel installation of multiple versions of the same distribution in the same Python installation, *without* requiring complete isolation in the form of virtual environments. The current solution (at least in Fedora), is to use the multi-version support in pkg_resources by installing unpacked egg files. To use CherryPy as an example, Fedora currently provides RPMs for both CherryPy 2 and 3. CherryPy 3 is the default and installed directly into site-packages, with an appropriate .egg-info directory CherryPy 2 is *not* the default, and is instead installed as an unpacked "CherryPy-2.3.0-py2.7.egg" directory. You can force this directory to be added to sys.path by doing the following in your __main__ module: __requires__ = ["CherryPy < 3"] import pkg_resources (__main__.__requires__ *has* to be set before importing pkg_resources or the default CherryPy 3 module will be activated automatically, and conflict with a later call to pkg_resources.requires that asks for CherryPy 2) While I'm not a fan (to put it mildly) of non-trivial side effects when importing a module, this approach to multi-version imports *does* work well (and, as noted, I'm currently working on improving the docs for it), and I think the approach to the filesystem layout in particular makes sense - the alternative versions are installed to the usual location, but pushed down a level in a subdirectory or zip archive. So, it seems to me that only two new pieces are needed to gain multi-version import support for wheel files: 1. An option (or options) to pip, telling it to just drop a wheel file (or the unpacked contents of the wheel as a directory) into site-packages instead of installing the distribution directly as the default version. The "root_is_purelib" setting in the wheel metadata would indicate whether the destination was purelib or platlib. A wheel installed this way wouldn't have script wrappers generated, etc - it would only allow the contents to be used as an importable library. 2. Support for the ".whl" filename format and internal layout in pkg_resources, modelling after the existing support for the ".egg" filename format. For wheels that include both purelib and platlib, this would involved adding both directories to the path. That means there wouldn't be an major design work to be done - just replicating what easy_install and pkg_resources already support for the egg format using the wheel format instead. Regardless of my personal feelings about the current *front end* API for multi-version imports (and PJE has put up with my ranting about that with remarkably good grace!), the current egg-based back end design looks solid to me and worth keeping rather than trying to invent something new for the sake of being different. Cheers, Nick. -- Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia

11 44

Re: [Distutils] buildout bootstrap.py doesn't work on Sabayon Linux with system python
by Benedek Zoltan 27 Aug '13

27 Aug '13

@Marius Gedminas @Ralf Schmitt Thanks for explanation and the tipp. Cheers Zoltan Benedek

1 0

Re: [Distutils] Multi-version import support for wheel files
by Vinay Sajip 26 Aug '13

26 Aug '13

> From: Donald Stufft <donald(a)stufft.io> > I think I am against this. > > Part of the beauty of Wheel is that it is simply a package format. This means > it does not need to concern itself with situations that Egg had to which bloat > the spec and make it harder to implement. I feel like tacking too much onto > the Wheel format is going to end us up in the exact same place we are at with > Eggs today. The more use cases we force onto Wheel the more we have to > consider anytime we make a change. For instance with the proposed change > now we have to worry about the importability of a Wheel file if an unpacked > Wheel is added directly to sys.path which is something we currently do not need > to worry about. > > As far as I can tell there's nothing preventing Installing a Wheel *into* > and .egg > directory which will give you exactly the same situation as you have today > without needing to do *anything* except make a tool that will install a Wheel > into an .egg directory. This solves your immediate desire without hanging > more things onto Wheel. > > Additionally I think the way it munges sys.path is completely backwards. I > believe, > as you know, that the current order of the sys.path is somewhat nonsensical by > default it goes ".", then standard library, then user site packages, > then regular > site packages. I believe that the order should be ".", user-packages, > site-packages, > and then standard library to provide a consistent ordering and allowing people > to shadow packages using a hierarchy of specificity (the packages installed by > a particular are more specific than the packages installed globally). However > when > I brought this up to you you were insistent that the fact that user installed > code could > not shadow the standard library was a feature and allowing it would be a bad > move. > However that is exactly what this system does. The sys.path looks like (after > acting > the cherrypy2 egg) ["cherryp2.egg", ".", stdlib, > user-packages, site-packages]. This > is wrong by both your definition and mine. > > So yea given that Wheels are not an on disk format, that you can install a Wheel > into > and egg if you want, and that the sys.path munging required is several broken > I'm > -1 on this. I do think we need a solution to multi version imports but I > don't think this > is it. However I think it's perfectly valid to tell people that the new > tooling doesn't support > this yet and they should continue to use the old. distutils2's problem was > they tried to > solve everything at once let's not make our problem that we rushed to find > an answer > for every problem and thus didn't fully flesh out all the options. +1 Regards, Vinay Sajip

1 0

Changing the way packaging related PEPs are drafted
by Nick Coghlan 24 Aug '13

24 Aug '13

In order to provide an issue tracker and easy pull requests for the metadata PEPs, I have set up a dedicated repo for them on BitBucket: https://bitbucket.org/pypa/pypi-metadata-formats/src The current contents are the PEP 426 (core-metadata.rst) and 440 (versioning.rst) drafts, along with copies of the previously accepted PEPs for 376 (installation-db.rst), 425 (compatibility-tags.rst) and 427 (wheel-format.rst). Cheers, Nick. -- Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia

2 2

buildout bootstrap.py doesn't work on Sabayon Linux with system python
by Benedek Zoltan 23 Aug '13

23 Aug '13

Hi, I've downloaded bootstrap.py and tried to initialize with system python: sabd1@sab /home/buildout $ wget http://svn.zope.org/*checkout*/zc.buildout/trunk/bootstrap/bootstrap.py Warning: wildcards not supported in HTTP. --2013-08-23 06:40:52-- http://svn.zope.org/*checkout*/zc.buildout/trunk/bootstrap/bootstrap.py Resolving svn.zope.org... 74.84.203.155 Connecting to svn.zope.org|74.84.203.155|:80... connected. HTTP request sent, awaiting response... 200 OK Length: unspecified [text/x-python] Saving to: ‘bootstrap.py’ [ <=> ] 10,107 41.2KB/s in 0.2s 2013-08-23 06:40:53 (41.2 KB/s) - ‘bootstrap.py’ saved [10107] sabd1@sab /home/buildout $ python bootstrap.py Downloading http://pypi.python.org/packages/2.7/s/setuptools/setuptools-0.6c11-py2.7.egg Traceback (most recent call last): File "bootstrap.py", line 258, in <module> ws.require(requirement) File "/tmp/tmpzJN6Tt/setuptools-0.6c11-py2.7.egg/pkg_resources.py", line 666, in require File "/tmp/tmpzJN6Tt/setuptools-0.6c11-py2.7.egg/pkg_resources.py", line 569, in resolve pkg_resources.VersionConflict: (setuptools 0.6c11 (/tmp/tmpzJN6Tt/setuptools-0.6c11-py2.7.egg), Requirement.parse('setuptools>=0.7')) I know, it works with virtualenv, but with system python is this expected behavior? Thanks Zoltan Benedek

3 2

Re: [Distutils] What does it mean for Python to "bundle pip"?
by Oscar Benjamin 23 Aug '13

23 Aug '13

Paul wrote: > Given that the installer includes the py.exe launcher, if you leave the defaults, then at a command prompt "python" doesn't work. But that's fine, because "py" does. And if you have multiple versions of Python installed, you don't *want* python on PATH, because then you have to manage your PATH. Why bother when "py -2.7" or "py -3.3" does what you want with no path management? Once you want any *other* executables, though, you have to deal with PATH (especially in the multiple Pythons case). That is a new issue, and one that hasn't been thought through yet, and we don't have a good solution. >From a user perspective I think that 'py -3.4 -m pip ...' is an improvement as it means I can easily install or upgrade for a particular python installation (I tend to have a few). There's no need to put Scripts on PATH just to run pip. I think this should be the recommended invocation for Windows users. Oscar

10 42

How to handle launcher script importability?
by Jason R. Coombs 22 Aug '13

22 Aug '13

In Setuptools 1.0 (currently in beta), I've added an experimental, opt-in feature to install pure Python launcher scripts on Windows instead of installing a launcher executable for each script, with the intention that these scripts will be launched by pylauncher or Python directly, eventually obviating the need for a launcher executable in setuptools at all. This means that instead of installing, for example: Scripts\my-command.exe Scripts\my-command-script.py Scripts\my-command.exe.manifest Instead Setuptools just installs: Scripts\my-command.py This technique is much like the scripts that get installed to bin/ on Unix, except that due to the nature of launching commands on Windows, the .py extension is essentially required. One problem with this technique is that if the script is also a valid module name, it can be imported, and because Python puts the script's directory at the top of sys.path, it _will_ be imported if that name is imported. This happens, for example, after installing Cython. Cython provides a command, 'cython', and a (extension) module called 'cython'. If one launches cython using the script launcher, the 'cython' module will not be importable (because "import cython" will import the launcher script). Presumably, this is why '-script' was added to the launcher scripts in previous versions. This is a rather unfortunate situation, and I'd like to solicit comments for a way to avoid this situation. I see a few options: 1. Have the setuptools-generated launcher scripts del sys.path[0] before launching. 2. Accept the implementation and file bugs with the offending projects like Cython to have them either rename their script or rename their internal module. 3. Continue to generate the script names with '-script.py' appended, requiring invocation to always include -script.py on Windows. 4. Continue to generate executables, duplicating the effort of pylauncher, and dealing with the maintenance burden of that functionality. I don't see (2), (3), or (4) as really viable, so my proposal is to move forward with (1) if there aren't any better suggestions. If we move forward with (1), there are a few concerns that come to mind. First, this approach would not apply to package-supplied scripts. What should be done about those (if anything)? Second, this would apply to Unix and Windows scripts (I'd rather not make the distinction if possible) - are there any concerns about removing sys.path[0] in the launch scripts on Unix? Third, is it possible some users are depending on the presence of sys.path[0] or the assumed positions of other items in sys.path that should be protected by this change? Would it make sense to replace sys.path[0] with a non-existent directory instead? My instinct is that these concerns are not of sufficient importance to account for them, and that we should just implement (1) as simply as possible. Your comments and suggestions are appreciated.

16 110