Mailman 3 August 2013 - Distutils-SIG

Two fixes (at last)
by Greg Ward 29 Apr '18

29 Apr '18

Hi all -- at long last, I have fixed two problems that a couple people noticed a while ago: * I folded in Amos Latteier's NT patches almost verbatim -- just changed an `os.path.sep == "/"' to `os.name == "posix"' and added some comments bitching about the inadequacy of the current library installation model (I think this is Python's fault, but for now Distutils is slavishly aping the situation in Python 1.5.x) * I fixed the problem whereby running "setup.py install" without doing anything else caused a crash (because 'build' hadn't yet been run). Now, the 'install' command automatically runs 'build' before doing anything; to make this bearable, I added a 'have_run' dictionary to the Distribution class to keep track of which commands have been run. So now not only are command classes singletons, but their 'run' method can only be invoked once -- both restrictions enforced by Distribution. The code is checked into CVS, or you can download a snapshot at http://www.python.org/sigs/distutils-sig/distutils-19990607.tar.gz Hope someone (Amos?) can try the new version under NT. Any takers for Mac OS? BTW, all parties involved in the Great "Where Do We Install Stuff?" Debate should take a good, hard look at the 'set_final_options()' method of the Install class in distutils/install.py; this is where all the policy decisions about where to install files are made. Currently it apes the Python 1.5 situation as closely as I could figure it out. Obviously, this is subject to change -- I just don't know to *what* it will change! Greg -- Greg Ward - software developer gward(a)cnri.reston.va.us Corporation for National Research Initiatives 1895 Preston White Drive voice: +1-703-620-8990 Reston, Virginia, USA 20191-5434 fax: +1-703-620-0913

2 1

Package versioning
by Timothy Docker 29 Apr '18

29 Apr '18

Hi all, I've been aware that the distutils sig has been simmerring away, but until recently it has not been directly relevant to what I do. I like the look of the proposed api, but have one question. Will this support an installed system that has multiple versions of the same package installed simultaneously? If not, then this would seem to be a significant limitation, especially when dependencies between packages are considered. Assuming it does, then how will this be achieved? I am presently managing this with a messy arrangement of symlinks. A package is installed with its version number in it's name, and a separate directory is created for an application with links from the unversioned package name to the versioned one. Then I just set the pythonpath to this directory. A sample of what the directory looks like is shown below. I'm sure there is a better solution that this, and I'm not sure that this would work under windows anyway (does windows have symlinks?). So, has this SIG considered such versioning issues yet? Cheers, Tim -------------------------------------------------------------- Tim Docker timd(a)macquarie.com.au Quantative Applications Division Macquarie Bank -------------------------------------------------------------- qad16:qad $ ls -l lib/python/ total 110 drwxr-xr-x 2 mts mts 512 Nov 11 11:23 1.1 -r--r----- 1 root mts 45172 Sep 1 1998 cdrmodule_0_7_1.so drwxr-xr-x 2 mts mts 512 Sep 1 1998 chart_1_1 drwxr-xr-x 3 mts mts 512 Sep 1 1998 Fnorb_0_7_1 dr-xr-x--- 3 mts mts 512 Nov 11 11:21 Fnorb_0_8 drwxr-xr-x 3 mts mts 1536 Mar 3 12:45 mts_1_1 dr-xr-x--- 7 mts mts 512 Nov 11 11:22 OpenGL_1_5_1 dr-xr-x--- 2 mts mts 1024 Nov 11 11:23 PIL_0_3 drwxr-xr-x 3 mts mts 512 Sep 1 1998 Pmw_0_7 dr-xr-x--- 2 mts mts 512 Nov 11 11:21 v3d_1_1 qad16:qad $ ls -l lib/python/1.1 total 30 lrwxrwxrwx 1 root other 29 Apr 10 10:43 _glumodule.so -> ../OpenGL_1_5_1/_glumodule.so lrwxrwxrwx 1 root other 30 Apr 10 10:43 _glutmodule.so -> ../OpenGL_1_5_1/_glutmodule.so lrwxrwxrwx 1 root other 22 Apr 10 10:43 _imaging.so -> ../PIL_0_3/_imaging.so lrwxrwxrwx 1 root other 36 Apr 10 10:43 _opengl_nummodule.so -> ../OpenGL_1_5_1/_opengl_nummodule.so lrwxrwxrwx 1 root other 27 Apr 10 10:43 _tkinter.so -> ../OpenGL_1_5_1/_tkinter.so lrwxrwxrwx 1 mts mts 21 Apr 10 10:43 cdrmodule.so -> ../cdrmodule_0_7_1.so lrwxrwxrwx 1 mts mts 12 Apr 10 10:43 chart -> ../chart_1_1 lrwxrwxrwx 1 root other 12 Apr 10 10:43 Fnorb -> ../Fnorb_0_8 lrwxrwxrwx 1 mts mts 12 Apr 10 10:43 mts -> ../mts_1_1 lrwxrwxrwx 1 root other 15 Apr 10 10:43 OpenGL -> ../OpenGL_1_5_1 lrwxrwxrwx 1 root other 33 Apr 10 10:43 opengltrmodule.so -> ../OpenGL_1_5_1/opengltrmodule.so lrwxrwxrwx 1 root other 33 Apr 10 10:43 openglutil_num.so -> ../OpenGL_1_5_1/openglutil_num.so lrwxrwxrwx 1 root other 10 Apr 10 10:43 PIL -> ../PIL_0_3 lrwxrwxrwx 1 mts mts 10 Apr 10 10:43 Pmw -> ../Pmw_0_7 lrwxrwxrwx 1 root other 10 Apr 10 10:43 v3d -> ../v3d_1_1

3 2

[Distutils-sig] distutils charter and interscript
by John Skaller 29 Apr '18

29 Apr '18

3 2

error in windows 7 installation
by Manuel López 13 May '14

13 May '14

Hello Everybody: I am trying to install distribute-0.6.25 in a windows 7 machine. I have python 2.7.3 in 32 bits, although the machine is 64 bits. I installed the 32 bit python version because I want to install ipython and there is no 64 bit builds of the Windows installer for ipython. In any case, upon installing disuitils with python setup.py install I get the following error error: C:\manuel\prog_red\python\distribute-0.6.25\distribute-0.6.25\setuptools\cli-32.exe: No such file or directory Any help will be greatly appeciated. Thanks, Manuel -- Manuel López Mariscal Depto. de Oceanografía Física/CICESE malope(a)cicese.mx

2 1

503 errors from PyPI
by Paul Moore 23 Feb '14

23 Feb '14

Whenever I do a PyPI search at the moment, I get Error 503 backend read error backend read error Guru Meditation: XID: 477806858 ________________________________ Varnish cache server Paul

3 6

PEP449 - Removal of the PyPI Mirror Auto Discovery and Naming Scheme
by Donald Stufft 12 Sep '13

12 Sep '13

Ok round two of this PEP. I've made a number of modifications to it: * Changed the title to be more relevant to the actual proposal. * Cleared up language to make it obvious that this was explicitly the naming and auto discovery being removed and not the mirrors in general. * Removed pointless distinctions between "Official" and "Unofficial" mirrors * Removed comments about staleness and points about the CDN which were not directly related to this PEP as the PEP doesn't attempt to solve issues with the mirroring protocol, just the discovery and naming scheme. * Incorporate feedback from the thread and simplify the proposal as well as extend the time frame. * Respond to some of the feedback from the thread that wasn't incorporated. I've explicitly CC'd Christian to this email so he can hopefully see it easier. Abstract ======== This PEP provides a path to deprecate and ultimately remove the auto discovery of PyPI mirrors as well as the hard coded naming scheme which requires delegating a domain name under pypi.python.org to a third party. Rationale ========= The PyPI mirroring infrastructure (defined in `PEP381`_) provides a means to mirror the content of PyPI used by the automatic installers. It also provides a method for auto discovery of mirrors and a consistent naming scheme. There are a number of problems with the auto discovery protocol and the naming scheme: * They give control over a \*.python.org domain name to a third party, allowing that third party to set or read cookies on the pypi.python.org and python.org domain name. * The use of a sub domain of pypi.python.org means that the mirror operators will never be able to get a SSL certificate of their own, and giving them one for a python.org domain name is unlikely to happen. * The auto discovery uses an unauthenticated protocol (DNS). * The lack of a TLS certificate on these domains means that clients can not be sure that they have not been a victim of DNS poisoning or a MITM attack. * The auto discovery protocol was designed to enable a client to automatically select a mirror for use. This is no longer a requirement because the CDN that PyPI is now using a globally distributed network of servers which will automatically select one close to the client without any effort on the clients part. * The auto discovery protocol and use of the consistent naming scheme has only ever been implemented by one installer (pip), and its implementation, besides being insecure, has serious issues with performance and is slated for removal with it's next release (1.5). * While there are provisions in `PEP381`_ that would solve *some* of these issues for a dedicated client it would not solve the issues that affect a users browser. Additionally these provisions have not been implemented by any installer to date. Due to the number of issues, some of them very serious, and the CDN which provides most of the benefit of the auto discovery and consistent naming scheme this PEP proposes to first deprecate and then remove the [a..z].pypi.python.org names for mirrors and the last.pypi.python.org name for the auto discovery protocol. The ability to mirror and the method of mirror will not be affected and will continue to exist as written in `PEP381`_. Operators of existing mirrors are encouraged to acquire their own domains and certificates to use for their mirrors if they wish to continue hosting them. Plan for Deprecation & Removal ============================== Immediately upon acceptance of this PEP documentation on PyPI will be updated to reflect the deprecated nature of the official public mirrors and will direct users to external resources like http://www.pypi-mirrors.org/ to discover unofficial public mirrors if they wish to use one. Mirror operators, if they wish to continue operating their mirror, should acquire a domain name to represent their mirror and, if they are able, a TLS certificate. Once they have acquired a domain they should redirect their assigned N.pypi.python.org domain name to their new domain. On Feb 15th, 2014 the DNS entries for [a..z].pypi.python.org and last.pypi.python.org will be removed. At any time prior to Feb 15th, 2014 a mirror operator may request that their domain name be reclaimed by PyPI and pointed back at the master. Why Feb 15th, 2014 ------------------ The most critical decision of this PEP is the final cut off date. If the date is too soon then it needlessly punishes people by forcing them to drop everything to update their deployment scripts. If the date is too far away then the extended period of time does not help with the migration effort and merely puts off the migration until a later date. The date of Feb 15th, 2014 has been chosen because it is roughly 6 months from the date of the PEP. This should ensure a lengthy period of time to enable people to update their deployment procedures to point to the new domains names without merely padding the cut off date. Why the DNS entries must be removed ----------------------------------- While it would be possible to simply reclaim the domain names used in mirror and direct them back at PyPI in order to prevent users from needing to update configurations to point away from those domains this has a number of issues. * Anyone who currently has these names hard coded in their configuration has them hard coded as HTTP. This means that by allowing these names to continue resolving we make it simple for a MITM operator to attack users by rewriting the redirect to HTTPS prior to giving it to the client. * The overhead of maintaining several domains pointing at PyPI has proved troublesome for the small number of N.pypi.python.org domains that have already been reclaimed. They often times get mis-configured when things change on the service which often leaves them broken for months at a time until somebody notices. By leaving them in we leave users of these domains open to random breakages which are less likely to get caught or noticed. * People using these domains have explicitly chosen to use them for one reason or another. One such reason may be because they do not wish to deploy from a host located in a particular country. If these domains continue to resolve but do not point at their existing locations we have silently removed this choice from the existing users of those domains. That being said, removing the entries *will* require users who have modified their configuration to either point back at the master (PyPI) or select a new mirror name to point at. This is regarded as a regrettable requirement to protect PyPI itself and the users of the mirrors from the attacks outlined above or, at the very least, require them to make an informed decision about the insecurity. Public or Private Mirrors ========================= The mirroring protocol will continue to exist as defined in `PEP381`_ and people are encouraged to to host public and private mirrors if they so desire. The recommended mirroring client is `Bandersnatch`_. .. _PyPI: https://pypi.python.org/ .. _PEP381: http://www.python.org/dev/peps/pep-0381/ .. _Bandersnatch: https://pypi.python.org/pypi/bandersnatch ----------------- Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

5 18

Comments on PEP 426
by Antoine Pitrou 08 Sep '13

08 Sep '13

Hello, Some comments about PEP 426: > The information defined in this PEP is serialised to pydist.json files for some > use cases. These are files containing UTF-8 encoded JSON metadata. Perhaps add that on-disk pydist.json files may/should be generated in printed form with sorted keys, to ease direct inspection by users and developers? > Source labels MUST be unique within each project and MUST NOT match any > defined version for the project. Is there a motivation for the "not matching any defined version"? AFAICT it makes it necessary to have two different representation schemes, e.g. "X.Y.Z" for source labels and "vX.Y.Z" for versions. > For source archive references, an expected hash value may be specified by > including a ``<hash-algorithm>=<expected-hash>`` entry as part of the URL > fragment. Why only source archive references (and not e.g. binary)? > "project_urls": { > "Documentation": "https://distlib.readthedocs.org" > "Home": "https://bitbucket.org/pypa/distlib" > "Repository": "https://bitbucket.org/pypa/distlib/src" > "Tracker": "https://bitbucket.org/pypa/distlib/issues" > } This example lacks commas. > An abbreviation of "metadistribution requires". This is a list of > subdistributions that can easily be installed and used together by > depending on this metadistribution. I don't understand what it means :-) Care to explain and/or clarify the purpose? (for me, "meta-requires" sounds like something that setup.py depends on for its own operation, but that the installed software doesn't need) (edit: I now see this is clarified in Appendix C. The section ordering in the PEP makes it look like "meta_requires" are the primary type of requires, though, while according to that appendix they're a rather exotic use case. Would be nice to spell that out *before* the appendices :-)). > * MAY allow direct references What is a direct reference? > Automated tools MUST NOT allow strict version matching clauses or direct > references in this field - if permitted at all, such clauses should appear > in ``meta_requires`` instead. Why so? [test requires] > Public index servers SHOULD NOT allow strict version matching clauses or > direct references in this field. Again, why? Is it important for public index servers that test dependencies be not pinned? > Note that while these are build dependencies for the distribution being > built, the installation is a *deployment* scenario for the dependencies. But there are no deployment requires, right? :) (or is what "meta requires" are for?) > For example, multiple projects might supply > PostgreSQL bindings for use with SQL Alchemy: each project might declare > that it provides ``sqlalchemy-postgresql-bindings``, allowing other > projects to depend only on having at least one of them installed. But the automated installer wouldn't be able to suggest the various packages providing ``sqlalchemy-postgresql-bindings`` if none is installed, which should IMO discourage such a scheme. > To handle this case in a way that doesn't allow for name hijacking, the > authors of the distribution that first defines the virtual dependency > should > create a project on the public index server with the corresponding name, > and > depend on the specific distribution that should be used if no other > provider > is already installed. This also has the benefit of publishing the default > provider in a way that automated tools will understand. But then the alternatives needn't provide the "virtual dependency". They can just provide the "default provider", which saves the time and hassle of defining a well-known virtual dependency for all similar projects. > A string that indicates that this project is no longer being developed. > The > named project provides a substitute or replacement. How about a project that is no longer being developed but has no direct substitution? :) Can it use an empty string (or null / None perhaps?) > Examples indicating supported operating systems:: > > # Windows only > "supports_environments": ["sys_platform == 'win32'"] Hmm, which syntax is it exactly? In a previous section, you used the following example: > "environment": "sys.platform == 'win32'" (note dot vs. underscore) > "modules": ["chair", "chair.cushions", (...)] The example is a bit intriguing. Is it expected that both "chair" and "chair.cushions" be specified there, or is "chair" sufficient? > When installing from an sdist, source archive or VCS checkout, > installation > tools SHOULD create a binary archive using ``setup.py bdist_wheel`` and > then install binary archive normally (including invocation of any install > hooks). Installation tools SHOULD NOT invoke ``setup.py install`` > directly. Interesting. Is "setup.py install" meant to die, or will it be redefined as "bdist_wheel + install_wheel"? (also, why is this mentioned in the postinstall hooks section, or even in a metadata-related PEP?) > Installation tools SHOULD treat an exception thrown by a preuninstall > hook as an indication the removal of the distribution should be aborted. I hope a "--force" option will be provided by such tools. Failure to uninstall because of buggy uninstall tools is a frustrating experience. > Extras are additional dependencies that enable an optional aspect > of the distribution I am confused. To me, extras look like additional provides, not additional dependencies. I.e. in: "requires": ["ComfyChair[warmup]"] -> requires ``ComfyChair`` and ``SoftCushions`` "warmup" is an additional provide of ComfyChair, and it depends on SoftCushions. > "requires": ["ComfyChair[*]"] > -> requires ``ComfyChair`` and ``SoftCushions``, but will also > pick up any new extras defined in later versions This one confuses me (again :-)). What does "pick up" mean? Accept? Require? > pip install ComfyChair[-,:*:,*] > -> installs the full set of development dependencies, but avoids > installing ComfyChair itself Are all these possibilities ("-", ":*:", "*") useful in real life? > Environment markers > =================== In this section, there still are inconsistencies in the format examples ("sys.platform" vs. "sys_platform"). > * ``platform_python_implementation``: ``platform.python_implementation()`` > * ``implementation_name````: ``sys.implementation.name`` Why two different ways to spell nearly the same thing: >>> platform.python_implementation() 'CPython' >>> sys.implementation.name 'cpython' (also, look at how platform.python_implementation() is implemented :-)) Also, do the ordering operators ("<=", ">=", etc.) operate logically or lexicographically on version values? > Build labels > ------------ > > See PEP 440 for the rationale behind the addition of this field. I can't see anything named "Build label" in PEP 426. Did you mean "source label"? > This version of the metadata specification continues to use ``setup.py`` > and the distutils command syntax to invoke build and test related > operations on a source archive or VCS checkout. I don't really understand how Metadata 2.0 is dependent on the distutils command scheme. Can you elaborate? Regards Antoine.

16 88

PEP453 - Explicit bootstrapping of pip in Python installations
by Donald Stufft 04 Sep '13

04 Sep '13

Abstract ======== This PEP proposes the inclusion of a method for explicitly bootstrapping `pip`_ as the default package manager for Python. It also proposes that the distributions of Python available via Python.org will automatically run this explicit bootstrapping method and a recommendation to third party redistributors of Python to also provide pip by default (in a way reasonable for their distributions). This PEP does *not* propose the inclusion of pip itself in the standard library. Proposal ======== This PEP proposes the inclusion of a ``getpip`` bootstrapping module in Python 3.4, as well as in the upcoming maintenance releases of Python 2.7 and Python 3.3. Rationale ========= Installing a third party package into a freshly installed Python requires first installing the package manager. This requires users ahead of time to know what the package manager is, where to get them from, and how to install them. The effect of this is that these external projects are required to either blindly assume the user already has the package manager installed, needs to duplicate the instructions and tell their users how to install the package manager, or completely forgo the use of dependencies to ease installation concerns for their users. All of the available options have their own drawbacks. If a project simply assumes a user already has the tooling then they get a confusing error message when the installation command doesn't work. Some operating may ease this pain by providing a global hook that looks for commands that don't exist and suggest an OS package they can install to make the command work. If a project chooses to duplicate the installation instructions and tell their users how to install the package manager before telling them how to install their own project then whenever these instructions need updates they need updating by every project that has duplicated them. This will inevitably not happen in every case leaving many different instructions on how to install it many of them broken or less than optimal. These additional instructions might also confuse users who try to install the package manager a second time thinking that it's part of the instructions of installing the project. The problem of stale instructions can be alleviated by referencing `pip's own bootstrapping instructions <http://www.pip-installer.org/en/latest/installing.html>`__, but the user experience involved still isn't good (especially on Windows, where downloading and running a Python script with the default OS configuration is significantly more painful than downloading and running a binary executable or installer). The projects that have decided to forgo dependencies all together are forced to either duplicate the efforts of other projects by inventing their own solutions to problems or are required to simply include the other projects in their own source trees. Both of these options present their own problems either in duplicating maintenance work across the ecosystem or potentially leaving users vulnerable to security issues because the included code or duplicated efforts are not automatically updated when upstream releases a new version. By providing the package manager by default it will be easier for users trying to install these third party packages as well as easier for the people distributing them as they no longer need to pick the lesser evil. This will become more important in the future as the Wheel_ package format does not have a built in "installer" in the form of ``setup.py`` so users wishing to install a Wheel package will need an installer even in the simple case. Reducing the burden of actually installing a third party package should also decrease the pressure to add every useful module to the standard library. This will allow additions to the standard library to focus more on why Python should have a particular tool out of the box instead of needing to use the difficulty in installing a package as justification for inclusion. Explicit Bootstrapping ====================== An additional module called ``getpip`` will be added to the standard library whose purpose is to install pip and any of its dependencies into the appropriate location (most commonly site-packages). It will expose a single callable named ``bootstrap()`` as well as offer direct execution via ``python -m getpip``. Options for installing it such as index server, installation location (``--user``, ``--root``, etc) will also be available to enable different installation schemes. It is believed that users will want the most recent versions available to be installed so that they can take advantage of the new advances in packaging. Since any particular version of Python has a much longer staying power than a version of pip in order to satisfy a user's desire to have the most recent version the bootstrap will contact PyPI, find the latest version, download it, and then install it. This process is security sensitive, difficult to get right, and evolves along with the rest of packaging. Instead of attempting to maintain a "mini pip" for the sole purpose of installing pip the ``getpip`` module will, as an implementation detail, include a private copy of pip which will be used to discover and install pip from PyPI. It is important to stress that this private copy of pip is *only* an implementation detail and it should *not* be relied on or assumed to exist. Not all users will have network access to PyPI whenever they run the bootstrap. In order to ensure that these users will still be able to bootstrap pip the bootstrap will fallback to simply installing the included copy of pip. This presents a balance between giving users the latest version of pip, saving them from needing to immediately upgrade pip after bootstrapping it, and allowing the bootstrap to work offline in situations where users might already have packages downloaded that they wish to install. Updating the Bundled pip ------------------------ In order to keep up with evolutions in packaging as well as providing users who are using the offline installation method with as recent version as possible the ``getpip`` module should be updates to the latest versions of everything it bootstraps. During the preparation for any release of Python, a script, provided as part of this PEP, should be run to update the bundled packages to the latest versions. This means that maintenance releases of the CPython installers will include an updated version of the ``getpip`` bootstrap module. Pre-installation ================ During the installation of Python from Python.org ``python -m getpip`` should be executed. Leaving people using the Windows or OSX installers with a working copy of pip once the installation has completed. The exact method of this is left up to the maintainers of the installers however if the bootstrapping is optional it should be opt out rather than opt in. The Windows and OSX installers distributed by Python.org will automatically attempt to run ``python -m getpip`` by default however the ``make install`` and ``make altinstall`` commands of the source distribution will not. Keeping the pip bootstrapping as a separate step for make based installations should minimize the changes CPython redistributors need to make to their build processes. Avoiding the layer of indirection through make for the getpip invocation also ensures those installing from a custom source build can easily force an offline installation of pip, install it from a private index server, or skip installing pip entirely. Python Virtual Environments =========================== Python 3.3 included a standard library approach to virtual Python environments through the ``venv`` module. Since it's release it has become clear that very few users have been willing to use this feature in part due to the lack of an installer present by default inside of the virtual environment. They have instead opted to continue using the ``virtualenv`` package which *does* include pip installed by default. To make the ``venv`` more useful to users it will be modified to issue the pip bootstrap by default inside of the new environment while creating it. This will allow people the same convenience inside of the virtual environment as this PEP provides outside of it as well as bringing the ``venv`` module closer to feature parity with the external ``virtualenv`` package making it a more suitable replacement. Recommendations for Downstream Distributors =========================================== A common source of Python installations are through downstream distributors such as the various Linux Distributions [#ubuntu]_ [#debian]_ [#fedora]_, OSX package managers [#homebrew]_, or python specific tools [#conda]_. In order to provide a consistent, user friendly experience to all users of Python regardless of how they attained Python this PEP recommends and asks that downstream distributors: * Ensure that whenever Python is installed pip is also installed. * This may take the form of separate with dependencies on each either so that installing the python package installs the pip package and installing the pip package installs the Python package. * Do not remove the bundled copy of pip. * This is required for offline installation of pip into a virtual environment. * This is similar to the existing ``virtualenv`` package for which many downstream distributors have already made exception to the common "debundling" policy. * This does mean that if ``pip`` needs to be updated due to a security issue, so does the bundled version in the ``getpip`` bootstrap module * Migrate build systems to utilize `pip`_ and `Wheel`_ instead of directly using ``setup.py``. * This will ensure that downstream packages can utilize the new formats which will not have a ``setup.py`` easier. * Ensure that all features of this PEP continue to work with any modifications made. * Online installation of the latest version of pip into a global or virtual python environment using ``python -m getpip``. * Offline installation of the bundled version of pip into a global or virtual python environment using ``python -m getpip``. * ``pip install --upgrade pip`` in a global installation should not affect any already created virtual environments. * ``pip install --upgrade pip`` in a virtual environment should not affect the global installation. Policies & Governance ===================== The maintainers of the bundled software and the CPython core team will work together in order to address the needs of both. The bundled software will still remain external to CPython and this PEP does not include CPython subsuming the responsibilities or decisions of the bundled software. This PEP aims to decrease the burden on end users wanting to use third party packages and the decisions inside it are pragmatic ones that represent the trust that the Python community has placed in the authors and maintainers of the bundled software. Backwards Compatibility ----------------------- The public API of the ``getpip`` module itself will fall under the typical backwards compatibility policy of Python for its standard library. The externally developed software that this PEP bundles does not. Security Releases ----------------- Any security update that affects the ``getpip`` module will be shared prior to release with the PSRT. The PSRT will then decide if the issue inside warrants a security release of Python. Appendix: Rejected Proposals ============================ Implicit Bootstrap ------------------ `PEP439`_, the predecessor for this PEP, proposes it's own solution. Its solution involves shipping a fake ``pip`` command that when executed would implicitly bootstrap and install pip if it does not already exist. This has been rejected because it is too "magical". It hides from the end user when exactly the pip command will be installed or that it is being installed at all. It also does not provide any recommendations or considerations towards downstream packagers who wish to manage the globally installed pip through the mechanisms typical for their system. Including pip In the Standard Library ------------------------------------- Similar to this PEP is the proposal of just including pip in the standard library. This would ensure that Python always includes pip and fixes all of the end user facing problems with not having pip present by default. This has been rejected because we've learned through the inclusion and history of ``distutils`` in the standard library that losing the ability to update the packaging tools independently can leave the tooling in a state of constant limbo. Making it unable to ever reasonably evolve in a timeframe that actually affects users as any new features will not be available to the general population for *years*. Allowing the packaging tools to progress separately from the Python release and adoption schedules allows the improvements to be used by *all* members of the Python community and not just those able to live on the bleeding edge of Python releases. .. _Wheel: http://www.python.org/dev/peps/pep-0427/ .. _pip: http://www.pip-installer.org .. _setuptools: https://pypi.python.org/pypi/setuptools .. _PEP439: http://www.python.org/dev/peps/pep-0439/ References ========== .. [#ubuntu] `Ubuntu <http://www.ubuntu.com/>` .. [#debian] `Debian <http://www.debian.org>` .. [#fedora] `Fedora <https://fedoraproject.org/>` .. [#homebrew] `Homebrew <http://brew.sh/>` .. [#conda] `Conda <http://www.continuum.io/blog/conda>` ----------------- Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

7 18

Distributable binary with dependencies
by bharath ravi kumar 02 Sep '13

02 Sep '13

Hi, I'm looking to package an application with all its dependencies for deployment on multiple hosts. I'd like to ensure that there is no compilation or setup step before starting the application in production. An nice to have ability would be to isolate base library dependencies per application (like virtualenv does). Ideally, the development -> deployment lifecycle would involve: (a) Build an application archive with all its dependencies baked in (b) Copy archive to a host in production. (c) Unwrap archive (d) Start services. (Note that the build host & production hosts are identical in architecture, OS patch level and python version). Having looked at various tools (e.g. distutils, setup tools, pip + virtualenv, egg, wheel, pyinstaller, etc.) available to address specific aspects/stages of the development lifecycle, I'm undecided on the tool/methodology to adopt in order to comprehensively solve the above problem. Any recommendation in this regard would be very helpful. Thanks for for your time. -Bharath

3 5

How to detect a namespace packages?
by Hartmut Goebel 02 Sep '13

02 Sep '13

Hi, I'm one of the developers of www.pyinstaller.org, a tool for creating stand-alone executables. We need to reliable detect if a package is a namespace package (nspkg). For each namespace, we need to add an empty fake-module into our executable to keep the import mechanism working. This has to work in all versions of Python starting with 2.4. nspkgs set up via a nspkg.pth-file are detected by being in sys.modules, but imp.find_module() files. For nspkgs using __init__.py-files (which use pkg_resources.declare_namespace() or pkgutil.extend_path()) I have no clue how to detect them. I tried to query meta-information using pkgresources, but I did not find a solution. Any help? -- Regards Hartmut Goebel | Hartmut Goebel | h.goebel(a)crazy-compilers.com | | www.crazy-compilers.com | compilers which you thought are impossible |

2 2