Hi Antoine, 2017-10-29 20:31 GMT+01:00 Antoine Rozo <antoine.rozo@gmail.com>:

Hi,

What would be the difference with current pip module? pip.main(['install', 'some_package'])

My understanding is that direct use of the `pip` module is explicitly not recommended. Stephan

2017-10-29 20:26 GMT+01:00 Alex Walters <tritium-list@sdamon.com>:

...

I have a somewhat better, imo, implementation of a pip object to be loaded into the repl.

class pip:

def __call__(self, *a, **kw):

sys.stderr.write(str(self))

def __repr__(self):

return str(self)

def __str__(self):

return “Please run pip from your system command prompt”

*From:* Python-ideas [mailto:python-ideas-bounces+tritium-list= sdamon.com@python.org] *On Behalf Of *Stephan Houben *Sent:* Sunday, October 29, 2017 3:19 PM *To:* Python-Ideas <python-ideas@python.org> *Subject:* [Python-ideas] install pip packages from Python prompt

Hi all,

Here is in somewhat more detail my earlier proposal for

having in the interactive Python interpreter a `pip` function to

install packages from Pypi.

Motivation: it appears to me that there is a category of newbies

for which "open a shell and do `pip whatever`" is a bit too much.

It would, in my opinion, simplify things a bit if they could just

copy-and-paste some text into the Python interpreter and have

some packages from pip installed.

That would simplify instructions on how to install package xyz,

without going into the vagaries of how to open a shell on various

platforms, and how to get to the right pip executable.

I think this could be as simple as:

def pip(args): import sys import subprocess subprocess.check_call([sys.executable, "-m", "pip"] + args.split())

print("Please re-start Python now to use installed or upgraded packages.")

Note that I added the final message about restarting the interpreter

as a low-tech solution to the problem of packages being already

imported in the current Python session.

I would imagine that the author of package xyz would then put on

their webpage something like:

To use, enter in your Python interpreter:

pip("install xyz --user")

As another example, consider prof. Baldwin from Woolamaloo university

who teaches a course "Introductory Python programming for Sheep Shavers".

In his course material, he instructs his students to execute the

following line in their Python interpreter.

pip("install woolamaloo-sheepshavers-goodies --user")

which will install a package which will in turn, as dependencies,

pull in a number of packages which are relevant for sheep shaving but

which have nevertheless irresponsibly been left outside the stdlib.

Stephan

_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

-- Antoine Rozo

_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

If you are calling pip.main, you know what you are doing, and you know when you have to restart the interpreter or not. An object that displays an error any time you try to use it, instructing a user of the proper way to use pip outside of the interpreter, is intended to catch the newbie mistake of trying to run pip from inside the repl, without encouraging potentially problematic behavior related to calling pip from inside the interpreter. Don’t encourage the wrong way because it’s a common newbie mistake, give them better documentation and error messages. Pip is a tool to be run outside of the repl, let’s keep it that way. My method avoids the entire problem by not attempting to run pip, while still providing a meaningful and useful error message to new users. If there is to be a solution other than SyntaxError or NameError (my solution only solves NameError), it should be to tell users how to use pip properly, not support improper use. From: Python-ideas [mailto:python-ideas-bounces+tritium-list=sdamon.com@python.org] On Behalf Of Antoine Rozo Sent: Sunday, October 29, 2017 3:32 PM To: Python-Ideas <python-ideas@python.org> Subject: Re: [Python-ideas] install pip packages from Python prompt Hi, What would be the difference with current pip module? pip.main(['install', 'some_package']) 2017-10-29 20:26 GMT+01:00 Alex Walters <tritium-list@sdamon.com <mailto:tritium-list@sdamon.com> >: I have a somewhat better, imo, implementation of a pip object to be loaded into the repl. class pip: def __call__(self, *a, **kw): sys.stderr.write(str(self)) def __repr__(self): return str(self) def __str__(self): return “Please run pip from your system command prompt” From: Python-ideas [mailto:python-ideas-bounces+tritium-list <mailto:python-ideas-bounces%2Btritium-list> =sdamon.com@python.org <mailto:sdamon.com@python.org> ] On Behalf Of Stephan Houben Sent: Sunday, October 29, 2017 3:19 PM To: Python-Ideas <python-ideas@python.org <mailto:python-ideas@python.org> > Subject: [Python-ideas] install pip packages from Python prompt Hi all, Here is in somewhat more detail my earlier proposal for having in the interactive Python interpreter a `pip` function to install packages from Pypi. Motivation: it appears to me that there is a category of newbies for which "open a shell and do `pip whatever`" is a bit too much. It would, in my opinion, simplify things a bit if they could just copy-and-paste some text into the Python interpreter and have some packages from pip installed. That would simplify instructions on how to install package xyz, without going into the vagaries of how to open a shell on various platforms, and how to get to the right pip executable. I think this could be as simple as: def pip(args): import sys import subprocess subprocess.check_call([sys.executable, "-m", "pip"] + args.split()) print("Please re-start Python now to use installed or upgraded packages.") Note that I added the final message about restarting the interpreter as a low-tech solution to the problem of packages being already imported in the current Python session. I would imagine that the author of package xyz would then put on their webpage something like: To use, enter in your Python interpreter: pip("install xyz --user") As another example, consider prof. Baldwin from Woolamaloo university who teaches a course "Introductory Python programming for Sheep Shavers". In his course material, he instructs his students to execute the following line in their Python interpreter. pip("install woolamaloo-sheepshavers-goodies --user") which will install a package which will in turn, as dependencies, pull in a number of packages which are relevant for sheep shaving but which have nevertheless irresponsibly been left outside the stdlib. Stephan _______________________________________________ Python-ideas mailing list Python-ideas@python.org <mailto:Python-ideas@python.org> https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/ -- Antoine Rozo

Then those users have more fundamental problems. There is a minimum level of computer knowledge needed to be successful in programming. Insulating users from the reality of the situation is not preparing them to be successful. Pretending that there is no system command prompt, or shell, or whatever platform specific term applies, only hurts new programmers. Give users an error message they can google, and they will be better off in the long run than they would be if we just ran pip for them. From: Stephan Houben [mailto:stephanh42@gmail.com] Sent: Sunday, October 29, 2017 3:43 PM To: Alex Walters <tritium-list@sdamon.com> Cc: Python-Ideas <python-ideas@python.org> Subject: Re: [Python-ideas] install pip packages from Python prompt Hi Alex, 2017-10-29 20:26 GMT+01:00 Alex Walters <tritium-list@sdamon.com <mailto:tritium-list@sdamon.com> >: return “Please run pip from your system command prompt” The target audience for my proposal are people who do not know which part of the sheep the "system command prompt" is. Stephan From: Python-ideas [mailto:python-ideas-bounces+tritium-list <mailto:python-ideas-bounces%2Btritium-list> =sdamon.com@python.org <mailto:sdamon.com@python.org> ] On Behalf Of Stephan Houben Sent: Sunday, October 29, 2017 3:19 PM To: Python-Ideas <python-ideas@python.org <mailto:python-ideas@python.org> > Subject: [Python-ideas] install pip packages from Python prompt Hi all, Here is in somewhat more detail my earlier proposal for having in the interactive Python interpreter a `pip` function to install packages from Pypi. Motivation: it appears to me that there is a category of newbies for which "open a shell and do `pip whatever`" is a bit too much. It would, in my opinion, simplify things a bit if they could just copy-and-paste some text into the Python interpreter and have some packages from pip installed. That would simplify instructions on how to install package xyz, without going into the vagaries of how to open a shell on various platforms, and how to get to the right pip executable. I think this could be as simple as: def pip(args): import sys import subprocess subprocess.check_call([sys.executable, "-m", "pip"] + args.split()) print("Please re-start Python now to use installed or upgraded packages.") Note that I added the final message about restarting the interpreter as a low-tech solution to the problem of packages being already imported in the current Python session. I would imagine that the author of package xyz would then put on their webpage something like: To use, enter in your Python interpreter: pip("install xyz --user") As another example, consider prof. Baldwin from Woolamaloo university who teaches a course "Introductory Python programming for Sheep Shavers". In his course material, he instructs his students to execute the following line in their Python interpreter. pip("install woolamaloo-sheepshavers-goodies --user") which will install a package which will in turn, as dependencies, pull in a number of packages which are relevant for sheep shaving but which have nevertheless irresponsibly been left outside the stdlib. Stephan

On Mon, Oct 30, 2017 at 11:27 AM, Erik Bray <erik.m.bray@gmail.com> wrote:

On Sun, Oct 29, 2017 at 8:45 PM, Alex Walters <tritium-list@sdamon.com> wrote:

...

Then those users have more fundamental problems. There is a minimum level of computer knowledge needed to be successful in programming. Insulating users from the reality of the situation is not preparing them to be successful. Pretending that there is no system command prompt, or shell, or whatever platform specific term applies, only hurts new programmers. Give users an error message they can google, and they will be better off in the long run than they would be if we just ran pip for them.

While I completely agree with this in principle, I think you overestimate the average beginner. Many beginners I've taught or helped, even if they can manage to get to the correct command prompt, often don't even know how to run the correct Python. They might often have multiple Pythons installed on their system--maybe they have Anaconda, maybe Python installed by homebrew, or a Python that came with an IDE like Spyder. If they're on OSX often running "python" from the command prompt gives the system's crippled Python 2.6 and they don't know the difference.

I should add--another case that is becoming extremely common is beginners learning Python for the first time inside the Jupyter/IPython Notebook. And in my experience it can be very difficult for beginners to understand the connection between what's happening in the notebook ("it's in the web-browser--what does that have to do with anything on my computer??") and the underlying Python interpreter, file system, etc. Being able to pip install from within the Notebook would be a big win. This is already possible since IPython allows running system commands and it is possible to run the pip executable from the notebook, then manually restart the Jupyter kernel. It's not 100% clear to me how my proposal below would work within a Jupyter Notebook, so that would also be an angle worth looking into. Best, Erik

One thing that has been a step in the right direction is moving more documentation toward preferring running `python -m pip` over just `pip`, since this often has a better guarantee of running `pip` in the Python interpreter you intended. But that still requires one to know how to run the correct Python interpreter from the command-line (which the newbie double-clicking on IDLE may not even have a concept of...).

While I agree this is something that is important for beginners to learn (e.g. print(sys.executable) if in doubt), it *is* a high bar for many newbies just to install one or two packages from pip, which they often might need/want to do for whatever educational pursuit they're following (heck, it's pretty common even just to want to install the `requests` module, as I would never throw `urllib` at a beginner).

So while I don't think anything proposed here will work technically, I am in favor of an in-interpreter pip install functionality. Perhaps it could work something like this:

a) Allow it *only* in interactive mode: running `pip(...)` (or whatever this looks like) outside of interactive mode raises a `RuntimeError` with the appropriate documentation b) When running `pip(...)` the user is supplied with an interactive prompt explaining that since installing packages with `pip()` can result in changes to the interpreter, it is necessary to restart the interpreter after installation--give them an opportunity to cancel the action in case they have any work they need to save. If they proceed, install the new package then restart the interpreter for them. This avoids any ambiguity as to states of loaded modules before/after pip install.

...

From: Stephan Houben [mailto:stephanh42@gmail.com] Sent: Sunday, October 29, 2017 3:43 PM To: Alex Walters <tritium-list@sdamon.com> Cc: Python-Ideas <python-ideas@python.org> Subject: Re: [Python-ideas] install pip packages from Python prompt

Hi Alex,

2017-10-29 20:26 GMT+01:00 Alex Walters <tritium-list@sdamon.com>:

return “Please run pip from your system command prompt”

The target audience for my proposal are people who do not know

which part of the sheep the "system command prompt" is.

Stephan

From: Python-ideas [mailto:python-ideas-bounces+tritium-list=sdamon.com@python.org] On Behalf Of Stephan Houben Sent: Sunday, October 29, 2017 3:19 PM To: Python-Ideas <python-ideas@python.org> Subject: [Python-ideas] install pip packages from Python prompt

Hi all,

Here is in somewhat more detail my earlier proposal for

having in the interactive Python interpreter a `pip` function to

install packages from Pypi.

Motivation: it appears to me that there is a category of newbies

for which "open a shell and do `pip whatever`" is a bit too much.

It would, in my opinion, simplify things a bit if they could just

copy-and-paste some text into the Python interpreter and have

some packages from pip installed.

That would simplify instructions on how to install package xyz,

without going into the vagaries of how to open a shell on various

platforms, and how to get to the right pip executable.

I think this could be as simple as:

def pip(args): import sys import subprocess subprocess.check_call([sys.executable, "-m", "pip"] + args.split())

print("Please re-start Python now to use installed or upgraded packages.")

Note that I added the final message about restarting the interpreter

as a low-tech solution to the problem of packages being already

imported in the current Python session.

I would imagine that the author of package xyz would then put on

their webpage something like:

To use, enter in your Python interpreter:

pip("install xyz --user")

As another example, consider prof. Baldwin from Woolamaloo university

who teaches a course "Introductory Python programming for Sheep Shavers".

In his course material, he instructs his students to execute the

following line in their Python interpreter.

pip("install woolamaloo-sheepshavers-goodies --user")

which will install a package which will in turn, as dependencies,

pull in a number of packages which are relevant for sheep shaving but

which have nevertheless irresponsibly been left outside the stdlib.

Stephan

_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

On Tue, 31 Oct 2017 01:44:10 +1000 Nick Coghlan <ncoghlan@gmail.com> wrote:

A few specific notes here:

1. As you say, this sort of already works in notebooks, since instructors can say to run "!pip install requests" and then restart the language kernel. 2. We could probably replicate that style in IDLE, since that runs user code in a subprocess, similar to the way Jupyter language kernels are separate from the frontend client 3. We can't replicate it as readily in the regular REPL, since that runs Python code directly in the current process, but even there I believe we could potentially trigger a full process restart via execve (or the C++ style _execve on Windows)

(We'd want a real process restart, rather than emulating it by calling Py_Initialize & Py_Finalize multiple times, as not every module properly supports multiple initialise/finalise cycles within a single process, and module-specific quirks are exactly what we'd be trying to avoid by forcing an interpreter restart)

The main difference, though, is that a notebook will reload and replay all your session, while restarting the regular REPL will simply lose all current work. I think that makes the idea much less appealing. Regards Antoine.

On 31 October 2017 at 02:06, Paul Moore <p.f.moore@gmail.com> wrote:

On 30 October 2017 at 15:53, Antoine Pitrou <solipsis@pitrou.net> wrote:

...

On Tue, 31 Oct 2017 01:44:10 +1000 Nick Coghlan <ncoghlan@gmail.com> wrote:

...

(We'd want a real process restart, rather than emulating it by calling Py_Initialize & Py_Finalize multiple times, as not every module properly supports multiple initialise/finalise cycles within a single process, and module-specific quirks are exactly what we'd be trying to avoid by forcing an interpreter restart)

The main difference, though, is that a notebook will reload and replay all your session, while restarting the regular REPL will simply lose all current work. I think that makes the idea much less appealing.

Right, but if you want an installation to work reliably, you're going to lose that state anyway. Erik's original comment included the suggestion to "give them an opportunity to cancel the action in case they have any work they need to save", and I think some kind of warning's going to be necessary no matter how we handle the restart.

Also, on Windows, I believe that any emulation of execve either leaves the original process in memory, or has problems getting console inheritance right. It's been a long time since I worked at that level, and things may be better now, but getting a robust "restart this process" interface in Windows would need some care (that's one of the reasons the py launcher runs Python as a subprocess rather than doing any sort of exec equivalent).

As long as the standard streams are passed along correctly, whatever the py launcher does would presumably be adequate for a REPL restart as well, assuming we decided to go down that path. It would also be reasonable to say that the regular REPL just issues a warning that a restart might be needed, and it's only REPLs with a separate client process that offer a way to restart the subprocess where code actually executes. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

On 30 October 2017 at 16:22, Nick Coghlan <ncoghlan@gmail.com> wrote:

...

Also, on Windows, I believe that any emulation of execve either leaves the original process in memory, or has problems getting console inheritance right. It's been a long time since I worked at that level, and things may be better now, but getting a robust "restart this process" interface in Windows would need some care (that's one of the reasons the py launcher runs Python as a subprocess rather than doing any sort of exec equivalent).

As long as the standard streams are passed along correctly, whatever the py launcher does would presumably be adequate for a REPL restart as well, assuming we decided to go down that path.

The py launcher starts a subprocess for python.exe and waits on it. I wouldn't have thought that's going to work for installing mods in a REPL - imagine a long working session where I install 10 mods as I explore options for a particular problem (I don't know how likely that is in practice...) - there'd be a chain of 10+ Python processes, only the last of which is still useful. It's probably not a massive problem (I assume everything but the last process is paged out) but it's not exactly friendly. OTOH, if you lose the command history and the interpreter state after each install, you'll probably get fed up long before the number of processes is an issue...

It would also be reasonable to say that the regular REPL just issues a warning that a restart might be needed, and it's only REPLs with a separate client process that offer a way to restart the subprocess where code actually executes.

This feels awfully like the traditional Windows "your mouse has moved - please reboot to have your changes take effect" behaviour. I don't think we're going to impress many people emulating that :-( Paul

On Mon, Oct 30, 2017 at 11:44 AM, Nick Coghlan <ncoghlan@gmail.com> wrote: ..

3. We can't replicate it as readily in the regular REPL, since that runs Python code directly in the current process, but even there I believe we could potentially trigger a full process restart via execve (or the C++ style _execve on Windows)

This exact problem is solved rather elegantly in Julia. When you upgrade a package that is already loaded in the REPL, it prints a warning: "The following packages have been updated but were already imported: ... Restart Julia to use the updated versions." listing the affected packages. See <https://github.com/JuliaLang/julia/blob/19921aa00468edf11f3d61f8c9b8d639ec71...>.

On Mon, 30 Oct 2017 at 03:36 Erik Bray <erik.m.bray@gmail.com> wrote:

...

On Sun, Oct 29, 2017 at 8:45 PM, Alex Walters <tritium-list@sdamon.com> wrote:

...

Then those users have more fundamental problems. There is a minimum level of computer knowledge needed to be successful in programming. Insulating users from the reality of the situation is not preparing them to be successful. Pretending that there is no system command prompt, or shell, or whatever platform specific term applies, only hurts new programmers. Give users an error message they can google, and they will be better off in

On Mon, Oct 30, 2017 at 11:27 AM, Erik Bray <erik.m.bray@gmail.com> wrote: the

...

...

long run than they would be if we just ran pip for them.

While I completely agree with this in principle, I think you overestimate the average beginner. Many beginners I've taught or helped, even if they can manage to get to the correct command prompt, often don't even know how to run the correct Python. They might often have multiple Pythons installed on their system--maybe they have Anaconda, maybe Python installed by homebrew, or a Python that came with an IDE like Spyder. If they're on OSX often running "python" from the command prompt gives the system's crippled Python 2.6 and they don't know the difference.

I should add--another case that is becoming extremely common is beginners learning Python for the first time inside the Jupyter/IPython Notebook. And in my experience it can be very difficult for beginners to understand the connection between what's happening in the notebook ("it's in the web-browser--what does that have to do with anything on my computer??") and the underlying Python interpreter, file system, etc. Being able to pip install from within the Notebook would be a big win. This is already possible since IPython allows running system commands and it is possible to run the pip executable from the notebook, then manually restart the Jupyter kernel.

It's not 100% clear to me how my proposal below would work within a Jupyter Notebook, so that would also be an angle worth looking into.

I'm -1 on this as I view it as a tooling issue, not a language issue. If you're teaching in an environment where you don't want to instruct on the differences between the REPL and a command prompt, then that suggests to me that the environment you're presenting students needs to be different rather than asking Python to work around your teaching environment (remember, any time you ask for a change in Python you're asking the change to be applied to literally millions of developers). I also wouldn't want to tie Python-the-language to pip-the-tool so tightly. While we may make sure that pip is available for convenience, Python as a language is not dependent on pip being installed in order to function. Adding something that shells out to pip wouldn't suddenly change that relationship between language and tool. -Brett

Best, Erik

...

One thing that has been a step in the right direction is moving more documentation toward preferring running `python -m pip` over just `pip`, since this often has a better guarantee of running `pip` in the Python interpreter you intended. But that still requires one to know how to run the correct Python interpreter from the command-line (which the newbie double-clicking on IDLE may not even have a concept of...).

While I agree this is something that is important for beginners to learn (e.g. print(sys.executable) if in doubt), it *is* a high bar for many newbies just to install one or two packages from pip, which they often might need/want to do for whatever educational pursuit they're following (heck, it's pretty common even just to want to install the `requests` module, as I would never throw `urllib` at a beginner).

So while I don't think anything proposed here will work technically, I am in favor of an in-interpreter pip install functionality. Perhaps it could work something like this:

a) Allow it *only* in interactive mode: running `pip(...)` (or whatever this looks like) outside of interactive mode raises a `RuntimeError` with the appropriate documentation b) When running `pip(...)` the user is supplied with an interactive prompt explaining that since installing packages with `pip()` can result in changes to the interpreter, it is necessary to restart the interpreter after installation--give them an opportunity to cancel the action in case they have any work they need to save. If they proceed, install the new package then restart the interpreter for them. This avoids any ambiguity as to states of loaded modules before/after pip install.

...

From: Stephan Houben [mailto:stephanh42@gmail.com] Sent: Sunday, October 29, 2017 3:43 PM To: Alex Walters <tritium-list@sdamon.com> Cc: Python-Ideas <python-ideas@python.org> Subject: Re: [Python-ideas] install pip packages from Python prompt

Hi Alex,

2017-10-29 20:26 GMT+01:00 Alex Walters <tritium-list@sdamon.com>:

return “Please run pip from your system command prompt”

The target audience for my proposal are people who do not know

which part of the sheep the "system command prompt" is.

Stephan

From: Python-ideas [mailto:python-ideas-bounces+tritium-list=sdamon.com@python.org] On Behalf Of Stephan Houben Sent: Sunday, October 29, 2017 3:19 PM To: Python-Ideas <python-ideas@python.org> Subject: [Python-ideas] install pip packages from Python prompt

Hi all,

Here is in somewhat more detail my earlier proposal for

having in the interactive Python interpreter a `pip` function to

install packages from Pypi.

Motivation: it appears to me that there is a category of newbies

for which "open a shell and do `pip whatever`" is a bit too much.

It would, in my opinion, simplify things a bit if they could just

copy-and-paste some text into the Python interpreter and have

some packages from pip installed.

That would simplify instructions on how to install package xyz,

without going into the vagaries of how to open a shell on various

platforms, and how to get to the right pip executable.

I think this could be as simple as:

def pip(args): import sys import subprocess subprocess.check_call([sys.executable, "-m", "pip"] + args.split())

print("Please re-start Python now to use installed or upgraded packages.")

Note that I added the final message about restarting the interpreter

as a low-tech solution to the problem of packages being already

imported in the current Python session.

I would imagine that the author of package xyz would then put on

their webpage something like:

To use, enter in your Python interpreter:

pip("install xyz --user")

As another example, consider prof. Baldwin from Woolamaloo university

who teaches a course "Introductory Python programming for Sheep Shavers".

In his course material, he instructs his students to execute the

following line in their Python interpreter.

pip("install woolamaloo-sheepshavers-goodies --user")

which will install a package which will in turn, as dependencies,

pull in a number of packages which are relevant for sheep shaving but

which have nevertheless irresponsibly been left outside the stdlib.

Stephan

_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

---

Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

-----Original Message----- From: Python-ideas [mailto:python-ideas-bounces+tritium- list=sdamon.com@python.org] On Behalf Of Erik Bray Sent: Monday, October 30, 2017 6:28 AM To: Python-Ideas <python-ideas@python.org> Subject: Re: [Python-ideas] install pip packages from Python prompt

On Sun, Oct 29, 2017 at 8:45 PM, Alex Walters <tritium-list@sdamon.com> wrote:

...

Then those users have more fundamental problems. There is a minimum level of computer knowledge needed to be successful in programming. Insulating users from the reality of the situation is not preparing them to be successful. Pretending that there is no system command prompt, or shell, or whatever platform specific term applies, only hurts new programmers. Give users an error message they can google, and they will be better off in the long run than they would be if we just ran pip for them.

While I completely agree with this in principle, I think you overestimate the average beginner.

Nope. I totally get that they don’t know what a shell or command prompt is. THEY. NEED. TO. LEARN. Hiding it is not a good idea for anyone. If this is an insurmountable problem for the newbie, maybe they really shouldn’t be attempting to program. This field is not for everyone.

Many beginners I've taught or helped, even if they can manage to get to the correct command prompt, often don't even know how to run the correct Python. They might often have multiple Pythons installed on their system--maybe they have Anaconda, maybe Python installed by homebrew, or a Python that came with an IDE like Spyder. If they're on OSX often running "python" from the command prompt gives the system's crippled Python 2.6 and they don't know the difference.

One thing that has been a step in the right direction is moving more documentation toward preferring running `python -m pip` over just `pip`, since this often has a better guarantee of running `pip` in the Python interpreter you intended. But that still requires one to know how to run the correct Python interpreter from the command-line (which the newbie double-clicking on IDLE may not even have a concept of...).

While I agree this is something that is important for beginners to learn (e.g. print(sys.executable) if in doubt), it *is* a high bar for many newbies just to install one or two packages from pip, which they often might need/want to do for whatever educational pursuit they're following (heck, it's pretty common even just to want to install the `requests` module, as I would never throw `urllib` at a beginner).

So while I don't think anything proposed here will work technically, I am in favor of an in-interpreter pip install functionality. Perhaps it could work something like this:

a) Allow it *only* in interactive mode: running `pip(...)` (or whatever this looks like) outside of interactive mode raises a `RuntimeError` with the appropriate documentation b) When running `pip(...)` the user is supplied with an interactive prompt explaining that since installing packages with `pip()` can result in changes to the interpreter, it is necessary to restart the interpreter after installation--give them an opportunity to cancel the action in case they have any work they need to save. If they proceed, install the new package then restart the interpreter for them. This avoids any ambiguity as to states of loaded modules before/after pip install.

...

From: Stephan Houben [mailto:stephanh42@gmail.com] Sent: Sunday, October 29, 2017 3:43 PM To: Alex Walters <tritium-list@sdamon.com> Cc: Python-Ideas <python-ideas@python.org> Subject: Re: [Python-ideas] install pip packages from Python prompt

Hi Alex,

2017-10-29 20:26 GMT+01:00 Alex Walters <tritium-list@sdamon.com>:

return “Please run pip from your system command prompt”

The target audience for my proposal are people who do not know

which part of the sheep the "system command prompt" is.

Stephan

From: Python-ideas [mailto:python-ideas-bounces+tritium-list=sdamon.com@python.org] On Behalf Of Stephan Houben Sent: Sunday, October 29, 2017 3:19 PM To: Python-Ideas <python-ideas@python.org> Subject: [Python-ideas] install pip packages from Python prompt

Hi all,

Here is in somewhat more detail my earlier proposal for

having in the interactive Python interpreter a `pip` function to

install packages from Pypi.

Motivation: it appears to me that there is a category of newbies

for which "open a shell and do `pip whatever`" is a bit too much.

It would, in my opinion, simplify things a bit if they could just

copy-and-paste some text into the Python interpreter and have

some packages from pip installed.

That would simplify instructions on how to install package xyz,

without going into the vagaries of how to open a shell on various

platforms, and how to get to the right pip executable.

I think this could be as simple as:

def pip(args): import sys import subprocess subprocess.check_call([sys.executable, "-m", "pip"] + args.split())

print("Please re-start Python now to use installed or upgraded packages.")

Note that I added the final message about restarting the interpreter

as a low-tech solution to the problem of packages being already

imported in the current Python session.

I would imagine that the author of package xyz would then put on

their webpage something like:

To use, enter in your Python interpreter:

pip("install xyz --user")

As another example, consider prof. Baldwin from Woolamaloo university

who teaches a course "Introductory Python programming for Sheep Shavers".

In his course material, he instructs his students to execute the

following line in their Python interpreter.

pip("install woolamaloo-sheepshavers-goodies --user")

which will install a package which will in turn, as dependencies,

pull in a number of packages which are relevant for sheep shaving but

which have nevertheless irresponsibly been left outside the stdlib.

Stephan

_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

Nope. I totally get that they don’t know what a shell or command prompt is. THEY. NEED. TO. LEARN. Hiding it is not a good idea for anyone.

I actually take this approach myself in my classes. However, I also have as prerequisites for my classes: Some Experience in some programming language And Basic familiarity with the command line. I then let them use whatever dev. Environment they want, while supporting and recommending a good editor and the command line. However, If people want to learn python that don’t have those prerequisites, then we suggest a different class designed for total newbies. In THAT class, we use a more proscribed dev environment so that everyone is doing the same thing in the same way. It was IDLE, and has lately been PyCharm. And the intro to data analytics class uses Anaconda and the Jupyter notebook. My point? We're not in the business of making judgements about who should and shouldn't become Python programmers - we're in the business of making sure that Python is accessible to as many people as possible by removing irrelevant barriers to adoption, Sure, but who is “we”? I think “we” is the python community, not the cPython developers. So providing an environment that makes it easy and obvious to install packages is a great idea, but I think it’s the job of IDEs and other higher level tools, not the REPL. If we need to add a feature to Python itself to make it easier for IDEs and the like to implement dynamic package adding, then by all means, let’s do it. Final note: I DO see a lot of questions ( on mailing lists, etc) from folks that try to type “pip install something” at the python command line. whether that's translating documentation so folks can start learning with instructions in their native language, or making it possible for them to defer learning the idiosyncrasies of the Windows, Linux, and Mac OS X command line environments. On the latter front, the details of the development interfaces offered by traditional desktop operating systems may *never* be relevant to the new generation of folks coming through that are learning to program by manipulating remote coding environments on tablets and other app-centric devices, just as most developers nowadays are able to get by without even learning C, let alone any flavour of assembly language. Our role in this process isn't to create future generations that think and work in exactly the same ways we do, it's to enable them to discover new ways of working that build on top of whatever we create. Jupyter notebooks are a decent example of this, where the difference between a Python statement and a "command line statement" is just an exclamation mark at the beginning of the line - exactly where the backing environment lives is mostly a hidden implementation detail from the user's perspective. Eclipse Che and other online coding environments are another case - there, the "command line" is a different window inside the editor app (this is also going to be a familiar option for heavy IDE users on traditional desktop operating systems). And putting it in those terms makes me think that we should explicitly exclude the default REPL from consideration here, as we've long taken the position that that *isn't* a good teaching environment, and you certainly can't access it remotely without some kind of other service in front of it to manage the network connection (even if that service is just ssh). That means I now see a few potential RFEs from this thread: 1. An import system feature that allows a running Python program to report a timestamp (with the same granularity as pyc file timestamp headers) for *when* the currently loaded modules were last modified. This could be as simple as a new `__mtime__` attribute in each module to store that number. 2. A new importlib.util API to check for potentially out of date modules in sys.modules (those where a freshly calculated module mtime doesn't match the stored __mtime__ attribute) 3. Support in IDLE for Jupyter-style "!" commands 4. Having IDLE call that importlib API and warn about any stale modules after each command line operation The first two features would be about enabling learning environments to more easily detect when the currently loaded modules may not match what's actually on disk (hot reloaders already do this by watching for filesystem changes, but we're currently missing a simpler polling based alternative that will also pick up package updates). The second two would be about enhancing IDLE's capabilities in this area, as we *do* suggest that as a reasonable initial learning environment, even though there are also plenty of alternatives out there now. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

I think it was proposed several times before, but I just wanted to revive the idea that we could add a GUI interface to install/update packages from IDLE (maybe even with some package browser). -- Ivan

You could teach them subprocess and os command injection safety from the start: ```python import subprocess import sys cmd = [sys.executable, -m', 'pip', 'install', '-r', 'psfblessed-requirements.txt']) retcode = subprocess.check_call(cmd) assert retcode == 0 ``` (Because shell=True is dangerous) On Tuesday, October 31, 2017, Terry Reedy <tjreedy@udel.edu> wrote:

On 10/31/2017 12:21 PM, Ivan Levkivskyi wrote:

...

I think it was proposed several times before, but I just wanted to revive the idea that we could add a GUI interface to install/update packages from IDLE (maybe even with some package browser).

https://bugs.python.org/issue23551. I agreed with and still agree with Raymond's opening message in Feb 2015: "In teaching Python, I find that many Windows users are command-line challenged and have difficulties using and accessing PIP. ... I would love to be able to start a class with a fresh Python download from python.org and effortlessly install requests and other tools without users having to fire-up a terminal window and wrestle with the various parts."

The one change I made in Raymond's proposal is that instead of having multiple IDLE menu entries tied to multiple IDLE functions invoking multiple pip functions, there would be one IDLE menu entry, perhaps 'Help => Install packages' (plural intentional), that would invoke a standalone tkinter based gui front-end to pip. 'Standalone' means no dependency on IDLE code. I don't think every IDE or app should *have to* write its own gui. Plus, a standalone tkinter module could be invoked from a command line with 'python -m pipgui' or invoked from interactive python with 'import pipgui; pipgui.main()'.

In April 2016, after posting the idea to pydev list and getting 'go ahead's from Nick Coughlin and someone else, with no negatives, I approved Upendra Kumar's GSOC proposal to write a pip gui. This was https://bugs.python.org/issue27051. On June 20, Ned Deily and Nick Coughlin vetoed adding a pip gui anywhere in the stdlib since it depended on something not in the stdlib, and perhaps for other reasons I don't fully understand.

Looking back, I can see that I made two mistakes.

The first was proposing to use the public-looking pip.main after importing pip. It is actually intended to be private (and should have been named '_main' to make that clearer). As it turns out, the extra work of accessing pip through the intended command line interface (via subprocess) is necessary anyway since running pip makes changes to the in-memory modules that are not reset when .main is called again. So it might as well be used for every access.

The second was not requiring an approved PEP before proceeding to actual coding.

-- Terry Jan Reedy

_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

Suggestions to help to minimize unnecessary logged bandwidth use and even work with a closed loop LAN: This reads from the filesystem: import requests This would read from the PyPi service over the network bandwidth: #!pip install -U requests #%run pip install -U requests #pip('install -U requests') This doesn't work because you SHOULD restart the interpreter after running pip (because imports are cached): import requests !pip install -U requests import requests Some tips on running educational environments for beginners (optionally in a lab): - IPython - requests? # docstring - requests?? # source - !pydoc ipython - %run pydoc ipython - Spyder includes an IPython terminal (``conda install spyder`` even installs Qt) - Just use conda (and docker and binder) - Aggressively cache network accesses (and/or just block internet access entirely) - PIP_INDEX="https://pip.local/simple/" - DevPi can transparently proxy cache packages from pypi (and then work without internet access) - a local httpbin is easy to host within the LAN - Binder can provision Docker containers with conda and everything already installed (With Kubernetes and JupyterHub (and OAuth)) - Each person gets their own container - Jupyter has a web terminal built in - You can also just run the Docker container locally #!pip install ipython scipy devpi certbot # Some back of the napkin calculations: # 30 users run a script which runs 2 pip install requests at least 5 times. # (These should be at the top) These Docker containers are ready to go with everything you need for a lesson: https://github.com/jupyter/docker-stacks You don't need this bash shell lesson if all you need to do is ``!pip install`` with IPython: https://swcarpentry.github.io/shell-novice/ A Tk GUI for pip would need to frustratingly duplicate ~/.bash_history and up-arrow to get the previous command. (Qt and GTK are not consistently installed; though ``conda install spyder`` does install Qt (which is very accessible)) IDLE sucks. https://devpi.net/docs/devpi/devpi/stable/%2Bd/index.html https://github.com/kennethreitz/httpbin/blob/master/Dockerfile These should pretty much get you started: https://github.com/jrjohansson/scientific-python-lectures https://github.com/jrjohansson/scientific-python-lectures/blob/master/Lectur... https://github.com/jrjohansson/scientific-python-lectures/blob/master/Lectur... https://github.com/jrjohansson/scientific-python-lectures/blob/master/Lectur... GitLab and Mattermost work with an offline LAN: https://docs.gitlab.com/omnibus/README.html https://docs.gitlab.com/omnibus/docker/README.html#run-the-image Binder can pull images from a GitLab Docker Container Registry: https://docs.gitlab.com/ce/user/project/container_registry.html These suggestions should help to minimize unnecessary logged bandwidth use and even work with a closed loop LAN. See: "#!pip install ipython scipy devpi certbot" in the middle of this email. On Tuesday, October 31, 2017, Wes Turner <wes.turner@gmail.com> wrote:

You could teach them subprocess and os command injection safety from the start:

```python import subprocess import sys cmd = [sys.executable, -m', 'pip', 'install', '-r', 'psfblessed-requirements.txt']) retcode = subprocess.check_call(cmd) assert retcode == 0 ```

(Because shell=True is dangerous and str.split is dangerous):

```python filename = "'/etc/ passwd' ; shutdown -r now" cmd = ("cat '%s'" % filename) cmd # "cat ''/etc/ passwd'' ; shutdown -r now"

cmd.split() # ["'", '/etc', 'passwd', ';', 'shutdown', '-r', 'now']

shlex.split(cmd) # ['cmd', '', '/etc', 'passwd', ';', 'shutdown', '-r', 'now'] ```

(Sarge solves for a number of additional cases beyond shlex.split (empty string should be '', 'already-quoted commands')

https://sarge.readthedocs.io/en/latest/overview.html#why- not-just-use-subprocess

https://en.wikipedia.org/wiki/Code_injection#Shell_injection

https://sarge.readthedocs.io/en/latest/internals.html#how- shell-quoting-works

Of course, we're programmers and our input is not untrusted, so shell=True without any string operations is not as dangerous.

On Tuesday, October 31, 2017, Wes Turner <wes.turner@gmail.com <javascript:_e(%7B%7D,'cvml','wes.turner@gmail.com');>> wrote:

...

You could teach them subprocess and os command injection safety from the start:

```python import subprocess import sys cmd = [sys.executable, -m', 'pip', 'install', '-r', 'psfblessed-requirements.txt']) retcode = subprocess.check_call(cmd) assert retcode == 0 ```

(Because shell=True is dangerous)

On Tuesday, October 31, 2017, Terry Reedy <tjreedy@udel.edu> wrote:

...

On 10/31/2017 12:21 PM, Ivan Levkivskyi wrote:

...

I think it was proposed several times before, but I just wanted to revive the idea that we could add a GUI interface to install/update packages from IDLE (maybe even with some package browser).

https://bugs.python.org/issue23551. I agreed with and still agree with Raymond's opening message in Feb 2015: "In teaching Python, I find that many Windows users are command-line challenged and have difficulties using and accessing PIP. ... I would love to be able to start a class with a fresh Python download from python.org and effortlessly install requests and other tools without users having to fire-up a terminal window and wrestle with the various parts."

The one change I made in Raymond's proposal is that instead of having multiple IDLE menu entries tied to multiple IDLE functions invoking multiple pip functions, there would be one IDLE menu entry, perhaps 'Help => Install packages' (plural intentional), that would invoke a standalone tkinter based gui front-end to pip. 'Standalone' means no dependency on IDLE code. I don't think every IDE or app should *have to* write its own gui. Plus, a standalone tkinter module could be invoked from a command line with 'python -m pipgui' or invoked from interactive python with 'import pipgui; pipgui.main()'.

In April 2016, after posting the idea to pydev list and getting 'go ahead's from Nick Coughlin and someone else, with no negatives, I approved Upendra Kumar's GSOC proposal to write a pip gui. This was https://bugs.python.org/issue27051. On June 20, Ned Deily and Nick Coughlin vetoed adding a pip gui anywhere in the stdlib since it depended on something not in the stdlib, and perhaps for other reasons I don't fully understand.

Looking back, I can see that I made two mistakes.

The first was proposing to use the public-looking pip.main after importing pip. It is actually intended to be private (and should have been named '_main' to make that clearer). As it turns out, the extra work of accessing pip through the intended command line interface (via subprocess) is necessary anyway since running pip makes changes to the in-memory modules that are not reset when .main is called again. So it might as well be used for every access.

The second was not requiring an approved PEP before proceeding to actual coding.

-- Terry Jan Reedy

_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

On Wednesday, November 1, 2017, Terry Reedy <tjreedy@udel.edu> wrote:

On 11/1/2017 2:54 AM, Wes Turner wrote:

A Tk GUI for pip would need to frustratingly duplicate ~/.bash_history and

...

up-arrow to get the previous command.

No it wouldn't, as the user would not be directly issuing pip commands. In any case, it would be trivial to keep a list of the pip commands submitted behind the gui facade and even to have an optional window to display them.

Now that there's a ``pip freeze`` and a ``conda env export``, it seems like it's much easier to determine what one just did to their {system, --user, virtualenv, conda root, or condaenv} without having a list of commands to review (ala .bash_history) or filesystem diff's. Would such a GUI handle these cases and also virtualenv create-and-restart? Bash can be installed in Windows.

IDLE sucks.

...

Why pollute your post with stupid flame bait? In many respects, IDLE's Shell is unequivocally better than interactive python in a line-oriented shell like Command Prompt. Ironically, given your comment above, one of its improvements is keeping a history of *statements* rather than a history of lines. This makes it possible to retrieve and edit a whole command (statement) at once, rather than in pieces (lines).

Spyder is an IDE with a command prompt (shell window) and a python prompt (python window, ipython window).

-- Terry Jan Reedy

_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

On Wednesday, November 1, 2017, Steve Barnes <gadgetsteve@live.co.uk> wrote:

On 01/11/2017 06:54, Wes Turner wrote:

...

Suggestions to help to minimize unnecessary logged bandwidth use and even work with a closed loop LAN:

This reads from the filesystem:

import requests

This would read from the PyPi service over the network bandwidth:

#!pip install -U requests #%run pip install -U requests

#pip('install -U requests')

This doesn't work because you SHOULD restart the interpreter after running pip (because imports are cached):

import requests !pip install -U requests import requests

Some tips on running educational environments for beginners (optionally in a lab):

<Snipped a lot of good stuff>

One tip that I have used when teaching python in a closed, (sometimes internet free environment), was to pre-prepare by, in an on-line environment:

1. Create a virtual environment with the version of Python that I am going to be teaching on the target platform 2. Activate that environment 3. Ensure that I am On-line 4. Download the pip install packages that I know I will need by using `pip download` to download but not install the packages, ideally using the -r requirements.txt syntax, (plus any windows specific builds from Christoph Gohlke's site). 5. Go Off-line and run pip install with the downloaded package - if I hit any errors due to packages having unspecified dependencies add those to the requirements list and repeat from 3. (While I am at it I often log an issue with the package maintainer). 6. A fast, personal, run through my lesson plan to ensure that I haven't missed anything.

I normally also download a few goodies that might not be essential to the lesson but that can act as a teaser for the more interested students.

At the start of the first lesson I give the students the downloaded packages directory, usually on a USB key, and get them to pip install them while explaining the difference between local and on-line installation.

That works. You could also host the packages with devpi or just a static HTTP server. As a transparent cache, if you test your ``requirements.txt`` with each OS/CPU_architecture combination, DevPi will store and serve each package to everyone at once. So do you do (in a virtualenv): pip install --download ./dir -r requirements.txt And then pip install --no-index --find-links ./dir -r requirements.text Like in the pip docs? https://pip.pypa.io/en/stable/user_guide/#installing-from-local-packages DevPi can host package documentation. https://devpi.net/docs/devpi/devpi/stable/+doc/userman/devpi_packages.html#u... * GitLab Pages can host package documentation and any static HTML pages (e.g. the CPython docs) alongside **the source** https://docs.gitlab.com/ce/user/project/pages/index.html https://github.com/python/cpython/tree/3.6/Doc ReadTheDocs can host Sphinx docs and can be run in a Docker container for an internet-free LAN: https://github.com/rtfd/readthedocs-docker-images

I know that I could save having to get the students to run pip by packaging up the virtual environment as a portable, or using by pyInstaller, but having them run pip on the local downloads gives me a chance to explain how to do it in the wild.

AFAIU, YMMV with ``virtualenv --relocatable``; and it certainly doesn't do all combinations of OS and processor architecture: https://virtualenv.pypa.io/en/stable/userguide/#making-environments-relocata... itertools.combinations( ['win', 'mac', 'lin', '', '686', 'x86_64', 'ARM'], 2)

BTW while Docker is great for this it is a whole other learning experience, (plus getting it running with some corporate security & anti-virus can be quite a challenge).

Yeah, hosting Docker containers with Kubernetes across a couple extra workstations is initially more work than creating a few USB keys and waiting for everyone's virtualenvs to converge (with ``$ pip install``, ``!PIP_INDEX='' pip install``, and now a new pip GUI button press sequence).

These USB keys are often re-used by other co-workers as a getting started or after my computer got changed/re-imaged starting point.

Wouldn't it be create if, just like oldschool floppies, there was a way to break off the sliding write protect tab of a USB stick? https://en.wikipedia.org/wiki/USB_flash_drive_security#Malware_infections CD-ROM and DVD-ROM drives seem to be more and more scare these days.

-- Steve (Gadget) Barnes Any opinions in this message are my personal opinions and do not reflect those of my employer.

--- This email has been checked for viruses by AVG. http://www.avg.com

_______________________________________________ Python-ideas mailing list Python-ideas@python.org <javascript:;> https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

On 11/1/2017 1:25 AM, Nick Coghlan wrote:

On 1 November 2017 at 08:50, Terry Reedy <tjreedy@udel.edu <mailto:tjreedy@udel.edu>> wrote:

In April 2016, after posting the idea to pydev list and getting 'go ahead's from Nick Coughlin and someone else, with no negatives, I approved Upendra Kumar's GSOC proposal to write a pip gui.  This was https://bugs.python.org/issue27051 <https://bugs.python.org/issue27051>.  On June 20, Ned Deily and Nick Coughlin vetoed adding a pip gui anywhere in the stdlib since it depended on something not in the stdlib, and perhaps for other reasons I don't fully understand.

Clarifying the objection here (since the linked issue is a fairly long one): what I'm against is tightly coupling the pip-gui development & release process to the CPython development & release process when we don't have any compelling reason to do so.

Thank you for clarifying. This is a reason that I did not understand ;-).

Given an independently released pip-gui on PyPI (with its own version numbering and release cadence), then I'd be +1 on bundling that as an optional IDLE addon, ensurepip style.

I already agree in my second response on this thread that pip gui should be on pypi. A pip gui would be a pip add-on, not an IDLE add-on, just as turtledemo is a turtle add-on even though one can invoke it from the IDLE menu -- as long as turtle and turtledemo are installed. Issue 27051, write pip gui, is properly a separate issue from https://bugs.python.org/issue23551, start a pip gui, if and when available, from IDLE menu. I am against 'tightly coupling' something to IDLE when there is 'no compelling reason to do so'. -- Terry Jan Reedy

On 11/1/2017 5:47 PM, Terry Reedy wrote:

When pip installs a package into site_packages, does it at any point run  package-specific installation code?  setup.py?  More specifically, can pip install an IDLE extension. If so, I think installing pipgui should add 'x_pipgui.py' to idlelib, if it exists, and add a section to idlelib/config-extension.def.  Using the existing extension mechanism would be an alternative to patching IDLE to conditionally add pipgui to some menu.

I meant to add that using the existing extension mechanism would allow pipgui to be run from 3.5 IDLE or even earlier, depending on how it is coded and the tcl/tk installed on the machine. Pipgui really needs ttk, which requires tcl/tk >= 8.5, but that has been true on Python Windows installs at least since 2.7. -- Terry Jan Reedy

On 2 November 2017 at 18:46, Nathaniel Smith <njs@pobox.com> wrote:

But unfortuately this hasn't been standardized, and there's currently no way to do the lookup from the stdlib, so maybe this is not so helpful for IDLE...

The entry point file format was recently promoted to a PyPA interoperability spec (without a PEP, as we documented it as-is, rather than changing anything): https://packaging.python.org/specifications/entry-points/ While the point about the standard library lacking the ability to read the metadata for installed packages still stands, it's also not too hard to implement a rudimentary version that just iterates over sys.path looking for `entry_points.txt` files in `*.dist-info` subdirectories. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

On 3 November 2017 at 02:22, Ivan Levkivskyi <levkivskyi@gmail.com> wrote:

Just another random idea: What about simply having two menu items in IDLE:

* Install/update package manager * Open package manager

The first one will install the pipgui from PyPI (and pip if not already installed). The second one will run the GUI.

This way it looks like pipgui can be simply published on PyPI without special-casing at all, or am I missing something?

This would also deal with the case where "ensurepip" hadn't been run at install time for some reason. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

On 31/10/2017 18:05, Chris Angelico wrote:

On Wed, Nov 1, 2017 at 2:49 AM, Chris Barker <chris.barker@noaa.gov> wrote:

...

And sure, some of those are completly clueless about what a command line is and how to use it, but others DO have an idea about the command line, but dont know that:

...

...

...

pip install something File "<stdin>", line 1 pip install something ^ SyntaxError: invalid syntax

means: "this was supposed to be run at the command prompt"

So I think defining a "pip" builtin that simply gave a helpful message would be a good start.

(hmm, it's a syntax error, so not as simple as a builtin -- but it could be caught somehow to give a better message)

This sounds like the job for an enhanced REPL like Jupyter/ipython. In fact, it might already exist - I haven't looked.

ChrisA

Of course those in environments where they need to avoid the use of the shell could use ipython and simply start the session with: !pip install -U needed_package before doing any imports. -- Steve (Gadget) Barnes Any opinions in this message are my personal opinions and do not reflect those of my employer. --- This email has been checked for viruses by AVG. http://www.avg.com

On 10/31/2017 10:41 AM, Nick Coghlan wrote:

On 31 October 2017 at 05:57, Alex Walters <tritium-list@sdamon.com <mailto:tritium-list@sdamon.com>> wrote:

> While I completely agree with this in principle, I think you > overestimate the average beginner.

The average Python beginner may have a decade or more experience interacting with computers by voice, gestures, fingers on a screen, mouse clicks and movements, joystick controller clicks and movements, and single key presses. This includes using a word processor. They may or may not have consciously learned (or remember) anything about natural language grammar. Python may well be their first experience with a formal language grammar and a picky interpreter.

Nope.  I totally get that they don’t know what a shell or command prompt is.

A shell is a program in which users issue instructions one at a time as sentences in a formal language and wait for the response before issuing another.

  THEY. NEED. TO. LEARN.

Yes. But what shell should people who want to learn Python learn first? Interactive python, which comes with a 'using' manual besides a tutorial and two reference manuals? Or, on Windows, Command Prompt, which runs a sparsely documented unnamed language, has no tutorial, and has mostly been replaced by File Explorer? One can just as well or better learn the idea of a REPL with formal language input from Python first. Once one has learned interactive Python, learning to use other shells should be straightforward. It should not be too hard to convey that a different language with a different syntax and different built-in commands requires a different interpreter.

Hiding it is not a good idea for anyone.

Tell that to Microsoft, which in Win 10 removed command prompt and other 'expert' tools from the regular start menu.

We're not in the business of making judgements about who should and shouldn't become Python programmers - we're in the business of making sure that Python is accessible to as many people as possible by removing irrelevant barriers to adoption, whether that's translating documentation so folks can start learning with instructions in their native language, or making it possible for them to defer learning the idiosyncrasies of the Windows, Linux, and Mac OS X command line environments.

My daughter learned Python and IDLE (from me) about 5 years before she had to learn about Command Prompt. (In between, she had courses that used the Racket and Jave/Eclipse IDEs.) The notion that she should have been forced to learn an inferior and useless (to her) shell first is to me absurd.

On the latter front, the details of the development interfaces offered by traditional desktop operating systems may *never* be relevant to the new generation of folks coming through that are learning to program by manipulating remote coding environments on tablets and other app-centric devices, just as most developers nowadays are able to get by without even learning C, let alone any flavour of assembly language. Our role in this process isn't to create future generations that think and work in exactly the same ways we do, it's to enable them to discover new ways of working that build on top of whatever we create.

Amen. At least 99+% of my interaction with the hg repository and my clones thereof was through TortoiseHg and its Workbench gui.

That means I now see a few potential RFEs from this thread:

1. An import system feature that allows a running Python program to report a timestamp (with the same granularity as pyc file timestamp headers) for *when* the currently loaded modules were last modified. This could be as simple as a new `__mtime__`  attribute in each module to store that number. 2. A new importlib.util API to check for potentially out of date modules in sys.modules (those where a freshly calculated module mtime doesn't match the stored __mtime__ attribute)

3. Support in IDLE for Jupyter-style "!" commands

I believe that this was suggested on idle-dev and rejected by Guido as an expert feature, or perhaps as abbreviating and thereby encouraging an expert usage. I will let him say what he thinks now. I have considered adding 'Run OS Command' to the Run menu, but I mostly do not see the point for a Python shell or REPL. If one knows what to type after '!', one can just as well click the appropriate icon and enter the command there. Or use os.system or subprocess. What am I missing?

4. Having IDLE call that importlib API and warn about any stale modules after each command line operation

This seems heavy handed, usually unnecessary, but also inadequate. One can now manipulate site-packages while running a Python shell in ways a shell will not know about. One should then either start the shell manually or take the consequences. As it is, IDLE restarts the user process each time one runs code from an editor. Or one can explicitly restart the user process, without exiting IDLE, after installing or updating modules. One objection to the latter has been that restarting, even on IDLE, clears more than one might want. I am working on a separate proposal that would make it easier to regenerate a workspace. ***** Hardly mentioned in this thread is that command line apps in general and pip in particular are not easily usable by many people. One needs to use a particular app regularly or have an unusually good memory for fiddy details or a high tolerance for reading and ability to understand reading -h output. That is why I think a pip gui would greatly help. If nothing that directly goes in the stdlib, other than ensurepip, can depend on pip and its api, a helper module could be installed from pypi along with pip (by ensurepip!). A pip wrapper, gui or not, could alleviate the following problem using pip in Windows. There are up to date Windows binaries on Christopher Gohlke's site, https://www.lfd.uci.edu/~gohlke/pythonlibs/. It currently has 32- and 64-bit builds for 2.7, 3.4, 3.5, and 3.6, as appropriate, for perhaps 450 packages. At least some are source-only on pypi. The site is not a pip package index, so the wheels must be downloaded first before being installed by pip. As for the subject of this thread: I intended that the pip gui that you and Ned Deily vetoed should, unlike pip itself, be runnable from python with "import pipgui; pip.main()". An import module can detect whether is it being run directly with Python or IDLE and, if sys.modules is modified, adjust its exit message or action accordingly. The main problem with pip that I have seen on Stackoverflow is that people run pip on a command line with a binary different from the one that they run IDLE with, and then post 'X imports Y, IDLE does not'. The easiest way to ensure that something is installed for the binary running IDLE is for IDLE to run something with its sys.executable. -- Terry Jan Reedy

-----Original Message----- From: Python-ideas [mailto:python-ideas-bounces+tritium- list=sdamon.com@python.org] On Behalf Of Erik Bray Sent: Monday, October 30, 2017 6:28 AM To: Python-Ideas <python-ideas@python.org> Subject: Re: [Python-ideas] install pip packages from Python prompt

On Sun, Oct 29, 2017 at 8:45 PM, Alex Walters <tritium-list@sdamon.com> wrote:

...

Then those users have more fundamental problems. There is a minimum level of computer knowledge needed to be successful in programming. Insulating users from the reality of the situation is not preparing them to be successful. Pretending that there is no system command prompt, or shell, or whatever platform specific term applies, only hurts new programmers. Give users an error message they can google, and they will be better off in

On Oct 30, 2017 8:57 PM, "Alex Walters" <tritium-list@sdamon.com> wrote: the

...

long run than they would be if we just ran pip for them.

While I completely agree with this in principle, I think you overestimate the average beginner.

Nope. I totally get that they don’t know what a shell or command prompt is. THEY. NEED. TO. LEARN. Hiding it is not a good idea for anyone. If this is an insurmountable problem for the newbie, maybe they really shouldn’t be attempting to program. This field is not for everyone. Reading this I get the impression, and correct me if I'm wrong, that you've never taught beginners programming. Of course long term (heck in fact fairly early on) they need to learn these nitty-gritty and sometimes frustrating lessons, but not in a 2 hour intro to programming for total beginners. And I beg to differ--this field is for everyone, and increasingly moreso every day. Doesn't mean it's easy, but it is and can be for everyone. Whether this specific proposal is technically feasible in a cross-platform manner with the state of the Python interpreter and import system is another question. But that's a discussion worth having. "Some people aren't cut out for programming" isn't.

Many beginners I've taught or helped, even if they can manage to get to the correct command prompt, often don't even know how to run the correct Python. They might often have multiple Pythons installed on their system--maybe they have Anaconda, maybe Python installed by homebrew, or a Python that came with an IDE like Spyder. If they're on OSX often running "python" from the command prompt gives the system's crippled Python 2.6 and they don't know the difference.

One thing that has been a step in the right direction is moving more documentation toward preferring running `python -m pip` over just `pip`, since this often has a better guarantee of running `pip` in the Python interpreter you intended. But that still requires one to know how to run the correct Python interpreter from the command-line (which the newbie double-clicking on IDLE may not even have a concept of...).

While I agree this is something that is important for beginners to learn (e.g. print(sys.executable) if in doubt), it *is* a high bar for many newbies just to install one or two packages from pip, which they often might need/want to do for whatever educational pursuit they're following (heck, it's pretty common even just to want to install the `requests` module, as I would never throw `urllib` at a beginner).

So while I don't think anything proposed here will work technically, I am in favor of an in-interpreter pip install functionality. Perhaps it could work something like this:

a) Allow it *only* in interactive mode: running `pip(...)` (or whatever this looks like) outside of interactive mode raises a `RuntimeError` with the appropriate documentation b) When running `pip(...)` the user is supplied with an interactive prompt explaining that since installing packages with `pip()` can result in changes to the interpreter, it is necessary to restart the interpreter after installation--give them an opportunity to cancel the action in case they have any work they need to save. If they proceed, install the new package then restart the interpreter for them. This avoids any ambiguity as to states of loaded modules before/after pip install.

...

From: Stephan Houben [mailto:stephanh42@gmail.com] Sent: Sunday, October 29, 2017 3:43 PM To: Alex Walters <tritium-list@sdamon.com> Cc: Python-Ideas <python-ideas@python.org> Subject: Re: [Python-ideas] install pip packages from Python prompt

Hi Alex,

2017-10-29 20:26 GMT+01:00 Alex Walters <tritium-list@sdamon.com>:

return “Please run pip from your system command prompt”

The target audience for my proposal are people who do not know

which part of the sheep the "system command prompt" is.

Stephan

From: Python-ideas [mailto:python-ideas-bounces+tritium-list=sdamon.com@python.org] On Behalf Of Stephan Houben Sent: Sunday, October 29, 2017 3:19 PM To: Python-Ideas <python-ideas@python.org> Subject: [Python-ideas] install pip packages from Python prompt

Hi all,

Here is in somewhat more detail my earlier proposal for

having in the interactive Python interpreter a `pip` function to

install packages from Pypi.

Motivation: it appears to me that there is a category of newbies

for which "open a shell and do `pip whatever`" is a bit too much.

It would, in my opinion, simplify things a bit if they could just

copy-and-paste some text into the Python interpreter and have

some packages from pip installed.

That would simplify instructions on how to install package xyz,

without going into the vagaries of how to open a shell on various

platforms, and how to get to the right pip executable.

I think this could be as simple as:

def pip(args): import sys import subprocess subprocess.check_call([sys.executable, "-m", "pip"] + args.split())

print("Please re-start Python now to use installed or upgraded packages.")

Note that I added the final message about restarting the interpreter

as a low-tech solution to the problem of packages being already

imported in the current Python session.

I would imagine that the author of package xyz would then put on

their webpage something like:

To use, enter in your Python interpreter:

pip("install xyz --user")

As another example, consider prof. Baldwin from Woolamaloo university

who teaches a course "Introductory Python programming for Sheep Shavers".

In his course material, he instructs his students to execute the

following line in their Python interpreter.

pip("install woolamaloo-sheepshavers-goodies --user")

which will install a package which will in turn, as dependencies,

pull in a number of packages which are relevant for sheep shaving but

which have nevertheless irresponsibly been left outside the stdlib.

Stephan

_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

Erik Bray writes:

Nope. I totally get that they don’t know what a shell or command prompt is. THEY. NEED. TO. LEARN.

I don't want to take a position on the proposal, and I agree that we should *strongly* encourage everyone to learn. But "THEY. NEED. TO. LEARN." is not obvious to me. Anecdotally, my students are doing remarkably (to me, as a teacher) complex modeling with graphical interfaces to statistical and simulation packages (SPSS/AMOS, Artisoc, respectively), and collection of large textual databases from SNS with cargo-culted Python programs. For the past twenty years teaching social scientists, these accidental barriers (as Fred Brooks would have called them) have dropped dramatically, to the point where it's possible to do superficially good-looking (= complex) but entirely meaningless :-/ empirical research. (In some ways I think this lowered cost has been horribly detrimental to my work as an educator in applied social science. ;-) The point being that "user-friendly" UI in many fields where (fairly) advanced computing is used is more than keeping up with the perceived needs of most computer users, while the essential (in the sense of Brooks) non-computing modeling difficulties of their jobs remain. By "perceived" I mean I want my students using TeX, but it's hard to force them when all their professors (except me and a couple mathematicians) use Word (speaking of irreproducible results). It's good enough for government work, and that's in fact where many of them end up (and the great majority are either in government or in equivalent corporate bureaucrat positions). Yes, I meant the deprecatory connotations of "perceived", but realistically, I admit that maybe they *don't* *need* the more polished tech that I could teach them.

Hiding it is not a good idea for anyone.

Agreed. Command lines and REPLs teach humility, to me as well as my students. :-) Steve -- Associate Professor Division of Policy and Planning Science http://turnbull/sk.tsukuba.ac.jp/ Faculty of Systems and Information Email: turnbull@sk.tsukuba.ac.jp University of Tsukuba Tel: 029-853-5175 Tennodai 1-1-1, Tsukuba 305-8573 JAPAN

On Saturday, November 4, 2017, Erik Bray <erik.m.bray@gmail.com> wrote:

On Nov 4, 2017 08:31, "Stephen J. Turnbull" <turnbull.stephen.fw@u. tsukuba.ac.jp <javascript:_e(%7B%7D,'cvml','turnbull.stephen.fw@u.tsukuba.ac.jp');>> wrote:

Erik Bray writes:

...

Nope. I totally get that they don’t know what a shell or command prompt is. THEY. NEED. TO. LEARN.

Just to be clear I did not write this. Someone replying to me did.

I'm going to go over all the different proposals in this thread and see if I can synthesize a list of options.

Here's are some of the challenges (both for a pip() function and a GUI): - Create reproducible software environments - Log the pip CLI commands - Log whether a version was specified - Be able to replay that log: - Which inputs need to be escaped before passing them through to a shell or exec*? (see: shlex.split()) - Please don't flood PyPi with unnecessary requests - (Turns out there are 5 invocations of `!pip install' buried somewhere in a script that's run 5 times in a hour by 25 students: even with pip caching package downloads, how many HTTP requests is that? Who pays for PyPi?) - There are a number of GUIs for pip: web-based, GUI based, CLI wrappers: - web: - CLI wrappers: pipenv, pipsi; ``conda install pip`` - GUI: PyCharm - Tcl/Tk is in the standard library. - ``conda install spyder`` installs At - pip does not have a stable API. - not even pip.main() - what about the shell CLI? - sarge is still safer than shlex.split()

I think, even if it's not a solution that winds up in the stdlib, it would be good to have some user stories about how package installation from within an interactive prompt might work (even if not from the standard REPL, which it should be noted has had small improvements made to it over the years).

This works today but is inadvisable because it risks wasting bandwidth if the %run (!) instructions are frequently re-run unnecessarily: $ ipython

...

...

!pip install -U ipython exit() $ ipython

I also have my doubts about whether this *shouldn't* be possible. I mean, to a lot of beginners starting out the basic REPL *is* Python. They're so new to the scene they don't even know what IPython or Jupyter is or why they might want that. They aren't experienced enough to even know what they're missing out on. In classrooms we can resolve that easily by pointing our students to whatever tools we think will work best for them, but not everyone has that privilege.

https://conda.io/docs/user-guide/install/index.html $ conda install -y pip ipython notebook spyder Or, just: $ conda install -y ipython notebook

Best, Erik

I don't want to take a position on the proposal, and I agree that we should *strongly* encourage everyone to learn. But "THEY. NEED. TO. LEARN." is not obvious to me.

Anecdotally, my students are doing remarkably (to me, as a teacher) complex modeling with graphical interfaces to statistical and simulation packages (SPSS/AMOS, Artisoc, respectively), and collection of large textual databases from SNS with cargo-culted Python programs. For the past twenty years teaching social scientists, these accidental barriers (as Fred Brooks would have called them) have dropped dramatically, to the point where it's possible to do superficially good-looking (= complex) but entirely meaningless :-/ empirical research. (In some ways I think this lowered cost has been horribly detrimental to my work as an educator in applied social science. ;-)

The point being that "user-friendly" UI in many fields where (fairly) advanced computing is used is more than keeping up with the perceived needs of most computer users, while the essential (in the sense of Brooks) non-computing modeling difficulties of their jobs remain.

By "perceived" I mean I want my students using TeX, but it's hard to force them when all their professors (except me and a couple mathematicians) use Word (speaking of irreproducible results). It's good enough for government work, and that's in fact where many of them end up (and the great majority are either in government or in equivalent corporate bureaucrat positions). Yes, I meant the deprecatory connotations of "perceived", but realistically, I admit that maybe they *don't* *need* the more polished tech that I could teach them.

I remember when I first started out teaching Software Carpentry I made the embarrassing mistake (coming from Physics) of assuming that LaTex is de-facto in most other academic fields :)

...

Hiding it is not a good idea for anyone.

Agreed. Command lines and REPLs teach humility, to me as well as my students. :-)

Steve

-- Associate Professor Division of Policy and Planning Science http://turnbull/sk.tsukuba.ac.jp/ Faculty of Systems and Information Email: turnbull@sk.tsukuba.ac.jp <javascript:_e(%7B%7D,'cvml','turnbull@sk.tsukuba.ac.jp');> University of Tsukuba Tel: 029-853-5175 Tennodai 1-1-1, Tsukuba 305-8573 JAPAN