[Python-ideas] from __pip__ import

[Paul Moore]

On 19 September 2016 at 23:46, אלעזר <elazarg at gmail.com> wrote:
>> import pip
>> pip.install('attrs')
>> import attr
>
> Please forgive me for my ignorance, but it doesn't work as written - what's
> the actual method?

As David Mertz said, pip.main(['install', 'attrs']) works right now,
but it is NOT a supported use of pip[1]. To be 100% explicit, the only
supported way of doing this is

import sys
from subprocess import run
run([sys.executable, '-m', 'pip', 'install', 'attrs'])

I suggested a hypothetical "pip.install" method as there is currently
some discussion on the pip tracker about providing a supported install
method. But it doesn't exist yet. Sorry for being confusing.

While on the whole subject of this, I should also point out that there
are a lot of potential issues with installing new packages while a
Python program is running. They are all low-probability, and easy to
avoid if you're not doing weird things, but for a generally-promoted
mechanism, we need to explain the corner cases[2], and an approach
with a list of caveats longer than the main documentation is
problematic.

1. If the install fails, you need to catch that and report it to the
user, in a more friendly manner than pip's output. For example if the
user has no C compiler and you need a C extension to be built.
2. You quite possibly want to suppress pip's output if it's *not* a
failure, as it'll clutter up the program's real output.
3. If the code has already imported foo.bar, then you install a new
version of foo (there are discussions as to whether pip install foo
should automatically imply --upgrade, so even if it won't do that by
default now, it might in the future), and maybe that new version
doesn't have a bar submodule. So now you have a weird mix of old and
new code in your process.
4. The install mechanism sometimes (I can't recall the details) caches
the fact that it couldn't import a module. If it does that and then
later you pip install that module, imports will still fail because the
information is cached.

I'm still not at all clear why any of this is so much better than a
comment at the top of the script

# To run this script, you need to "pip install attrs" first

Paul.

[1] We've had people report issues where pip breaks their logging
config, for example, because pip uses logging but doesn't expect to be
run from user code that also does so.
[2] That "run([sys.executable, ...])" invocation doesn't work in an
embedded program, for example, where sys.executable isn't "python".