maybe this was covered before, but I'm looking for clarification on how pip upgrades are to work under PEP439. I see 3 relevant bits right now: 1) "attempts to import pip machinery. If it can then the pip command proceeds as normal" i.e., once it has a pip installed, it doesn't automatically upgrade on subsequent imports. 2) "A bootstrap is used in the place of a the full pip code so that we don't have to bundle pip and also the install tool is upgradeable outside of the regular Python upgrade timeframe and processes" to be clear, what is the "install tool" here? pip? or the bootstrap? 3) "Manual installation of the pip implementation will be supported through the manual download of the wheel file and "pip3 install <downloaded wheel file>. This installation will not perform standard pip installation steps of saving the file to a cache directory or updating any local database of installed files."" I guess this means "manual upgrades" as well? this special logic is to be inside pip itself? or an override of "install" in the bootstrap (that detects when it's a pip wheel)? Marcus
On 10 June 2013 06:43, Marcus Smith <qwcode@gmail.com> wrote:
maybe this was covered before, but I'm looking for clarification on how pip upgrades are to work under PEP439.
I see 3 relevant bits right now:
1) "attempts to import pip machinery. If it can then the pip command proceeds as normal"
i.e., once it has a pip installed, it doesn't automatically upgrade on subsequent imports.
That's the intention. I've modified that paragraph: """ The Python installation includes an executable called "pip3" (see PEP 394 for naming rationale etc.) that attempts to import pip machinery. If it can then the pip command proceeds as normal. If it cannot it will bootstrap pip by downloading the pip implementation wheel file. Once installed, the pip command proceeds as normal. Once the bootstrap process is complete the "pip3" command is no longer the bootstrap but rather the full pip command. """
2) "A bootstrap is used in the place of a the full pip code so that we don't have to bundle pip and also the install tool is upgradeable outside of the regular Python upgrade timeframe and processes"
to be clear, what is the "install tool" here? pip? or the bootstrap?
I will edit the PEP to replace "install tool" with "pip": """ A boostrap is used in the place of a the full pip code so that we don't have to bundle pip and also pip is upgradeable outside of the regular Python upgrade timeframe and processes. """ 3) "Manual installation of the pip implementation will be supported through
the manual download of the wheel file and "pip3 install <downloaded wheel file>. This installation will not perform standard pip installation steps of saving the file to a cache directory or updating any local database of installed files.""
I guess this means "manual upgrades" as well? this special logic is to be inside pip itself? or an override of "install" in the bootstrap (that detects when it's a pip wheel)?
I've modified this section to read: """ Some users may have no Internet access suitable for fetching the pip implementation file. Manual installation of the pip implementation will be supported through the manual download of the wheel file and "pip3 install <downloaded wheel file>". This installation - since it uses only the bootstrap code - will not perform standard pip installation steps of saving the file to a cache directory or updating any local database of installed files. """ These changes will be online soon. Richard
On 14 June 2013 11:05, Richard Jones <richard@python.org> wrote:
On 10 June 2013 06:43, Marcus Smith <qwcode@gmail.com> wrote:
maybe this was covered before, but I'm looking for clarification on how pip upgrades are to work under PEP439.
I see 3 relevant bits right now:
1) "attempts to import pip machinery. If it can then the pip command proceeds as normal"
i.e., once it has a pip installed, it doesn't automatically upgrade on subsequent imports.
That's the intention. I've modified that paragraph:
""" The Python installation includes an executable called "pip3" (see PEP 394 for naming rationale etc.) that attempts to import pip machinery. If it can then the pip command proceeds as normal. If it cannot it will bootstrap pip by downloading the pip implementation wheel file. Once installed, the pip command proceeds as normal. Once the bootstrap process is complete the "pip3" command is no longer the bootstrap but rather the full pip command. """
The fact that pip just ships a setuptools entry point wrapper as /usr/bin/pip makes this easier. The difference between the stdlib bootstrap script and the ordinary script is that the bootstrap one will: 1. Import pip directly rather than using the pkg_resources entry point mechanism 2. Fallback to retrieving pip from PyPI if that fails 3. Invoke pip.main directly rather than using the pkg_resources entry point mechanism
3) "Manual installation of the pip implementation will be supported through the manual download of the wheel file and "pip3 install <downloaded wheel file>. This installation will not perform standard pip installation steps of saving the file to a cache directory or updating any local database of installed files.""
I guess this means "manual upgrades" as well? this special logic is to be inside pip itself? or an override of "install" in the bootstrap (that detects when it's a pip wheel)?
I've modified this section to read:
""" Some users may have no Internet access suitable for fetching the pip implementation file. Manual installation of the pip implementation will be supported through the manual download of the wheel file and "pip3 install <downloaded wheel file>". This installation - since it uses only the bootstrap code - will not perform standard pip installation steps of saving the file to a cache directory or updating any local database of installed files. """
These changes will be online soon.
I've come to doubt the wisdom of omitting the metadata or using a default install location that differs from pip's default location. Once we download the pip wheel file, we should be able to unpack it to a secure temporary directory and use it to install itself. That means we should be able to do a full install of pip by default, rather than anything that hides the metadata. Updating it later then becomes the same as updating any other pip installed distribution. The Linux distros can deal with it by either preinstalling pip as part of the python packages, or just leave the bootstrap script out of the Python packages and provide a distinct python-pip package as they do now. "sudo yum install python-pip" and "sudo apt-get install python-pip" are already pretty easy ways to bootstrap pip - it's Windows that really needs the help, and a "it's just like any other pip maintained package" approach is highly desirable there. The only trick would be ensuring the pip wheel console script doesn't collide with the bootstrap script, but worst case, we just special case that directly in pip. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
I've come to doubt the wisdom of omitting the metadata or using a default install location that differs from pip's default location. Once we download the pip wheel file, we should be able to unpack it to a secure temporary directory and use it to install itself. That means we should be able to do a full install of pip by default, rather than anything that hides the metadata. Updating it later then becomes the same as updating any other pip installed distribution.
ok, sounds good. once the pip bootstrap is done, you'll just have a standard install of pip. and after that, it's pip's job to support upgrading itself properly for users.
The Linux distros can deal with it by either preinstalling pip as part of the python packages, or just leave the bootstrap script out of the Python packages and provide a distinct python-pip package as they do now. "sudo yum install python-pip" and "sudo apt-get install python-pip" are already pretty easy ways to bootstrap pip - it's Windows that really needs the help, and a "it's just like any other pip maintained package" approach is highly desirable there.
The only trick would be ensuring the pip wheel console script doesn't collide with the bootstrap script, but worst case, we just special case that directly in pip.
Cheers, Nick.
-- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On 15 June 2013 09:58, Marcus Smith <qwcode@gmail.com> wrote:
I've come to doubt the wisdom of omitting the metadata or using a default install location that differs from pip's default location. Once we download the pip wheel file, we should be able to unpack it to a secure temporary directory and use it to install itself. That means we should be able to do a full install of pip by default, rather than anything that hides the metadata. Updating it later then becomes the same as updating any other pip installed distribution.
ok, sounds good.
once the pip bootstrap is done, you'll just have a standard install of pip. and after that, it's pip's job to support upgrading itself properly for users.
Yep. There are a couple of wrinkles from a user experience/sanity perspective point of view though. 1. We should write the bootstrapped pip to the same location that we're installing the package to (so system global, user local or current venv). 2. We should check we can actually write to that location before downloading the pip wheel. If we can't write to that destination, we should emit the same error message pip itself would in that situation. 3. If we can write to it, we download the wheel, add it directly to the front of sys.path (as Paul suggested), import pip.main from there and then use that to install pip itself before continuing with the rest of the installation (still using the copy from the wheel for that initial invocation). Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
1. We should write the bootstrapped pip to the same location that we're installing the package to (so system global, user local or current venv).
this makes sense for global and user installs, but not venv? when you're in a venv, you're using the pip installed in that specific venv, not the bootstrap. and just to confirm about user install bootstrapping, it would work like this? - user1 happens to run "pip3 install --user django" as his first pip3 command - the bootstrap installs pip to /home/user1/.local - thereafter, user1's executions of pip3 detects the importable pip in user local, and does no more bootstrapping - user2 runs "pip3 install --user django" as his first pip3 command - the bootstrap installs pip to /home/user2/.local ... and if instead user1 ran "sudo pip install django" as his first command, pip get's installed globally. and thereafter, all users would be using a global installed pip, no matter what they're first command was. Marcus
On 15 June 2013 16:02, Marcus Smith <qwcode@gmail.com> wrote:
1. We should write the bootstrapped pip to the same location that we're installing the package to (so system global, user local or current venv).
this makes sense for global and user installs, but not venv? when you're in a venv, you're using the pip installed in that specific venv, not the bootstrap.
True for virtualenv, but not for pyvenv - that doesn't install anything into the virtual environment, it just creates it. I expect the 3.4 version will add a link to the appropriate bootstrapping script though, so we need to define the expected behaviour in that case.
and just to confirm about user install bootstrapping, it would work like this? - user1 happens to run "pip3 install --user django" as his first pip3 command - the bootstrap installs pip to /home/user1/.local - thereafter, user1's executions of pip3 detects the importable pip in user local, and does no more bootstrapping - user2 runs "pip3 install --user django" as his first pip3 command - the bootstrap installs pip to /home/user2/.local ...
and if instead user1 ran "sudo pip install django" as his first command, pip get's installed globally. and thereafter, all users would be using a global installed pip, no matter what they're first command was.
That is what I was saying, but with it spelled out like that, I don't like it. The destination of the current install request isn't adequate justification for defining the location of the installed pip. Instead, I'm starting to think that with the right error message, "Explicit is better than implicit" and "In the face of ambiguity, refuse the temptation to guess" should rule the day here. For example: $pip3 install Django pip is not currently installed. Run 'pip3 bootstrap' or 'pip3 bootstrap --system' to install it. $pip3 bootstrap --help Usage: pip3 bootstrap pip3 bootstrap --system Description: Downloads and installs the latest version of pip from the Python Package Index (pypi.python.org). Installs into the active virtual environment (if any) or the current user's site-packages directory. Has no effect if pip is already installed (run "pip install --upgrade pip" to upgrade to a later version). Options: --system Install into the system site-packages instead of the venv or user site-packages. --wheel <file> Installs from the given wheel file instead of downloading from PyPI Linux distros could then patch the 'pip3 bootstrap --system' variant to invoke the appropriate platform specific installation command. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
Instead, I'm starting to think that with the right error message,
"Explicit is better than implicit" and "In the face of ambiguity, refuse the temptation to guess" should rule the day here.
For example:
$pip3 install Django pip is not currently installed. Run 'pip3 bootstrap' or 'pip3 bootstrap --system' to install it. $pip3 bootstrap --help Usage: pip3 bootstrap pip3 bootstrap --system
sounds reasonable to me.
Linux distros could then patch the 'pip3 bootstrap --system' variant to invoke the appropriate platform specific installation command.
well, e.g. debian patches distutils.command.install with it's /usr/local layer and dist-packages patches. I imagine, those kind of patches would also occur in distlib as well now? (which will be handling the bootstrap installs) Marcus
participants (3)
-
Marcus Smith -
Nick Coghlan -
Richard Jones