[Python-ideas] install pip packages from Python prompt

[Wes Turner]

Suggestions to help to minimize unnecessary logged bandwidth use and even
work with a closed loop LAN:

This reads from the filesystem:

    import requests

This would read from the PyPi service over the network bandwidth:

   #!pip install -U requests
   #%run pip install -U requests

   #pip('install -U requests')

This doesn't work because you SHOULD restart the interpreter after running
pip (because imports are cached):

    import requests
    !pip install -U requests
    import requests

Some tips on running educational environments for beginners (optionally in
a lab):

- IPython
  - requests?  # docstring
  - requests?? # source
  - !pydoc ipython
  - %run pydoc ipython
- Spyder includes an IPython terminal
  (``conda install spyder`` even installs Qt)
- Just use conda (and docker and binder)
- Aggressively cache network accesses (and/or just block internet access
entirely)
  - PIP_INDEX="https://pip.local/simple/"
  - DevPi can transparently proxy cache packages from pypi (and then work
without internet access)
  - a local httpbin is easy to host within the LAN

- Binder can provision Docker containers with conda and everything already
installed (With Kubernetes and JupyterHub (and OAuth))
  - Each person gets their own container
  - Jupyter has a web terminal built in
  - You can also just run the Docker container locally

#!pip install ipython scipy devpi certbot
# Some back of the napkin calculations:
# 30 users run a script which runs 2 pip install requests at least 5 times.
# (These should be at the top)

These Docker containers are ready to go with everything you need for a
lesson:
https://github.com/jupyter/docker-stacks

You don't need this bash shell lesson if all you need to do is ``!pip
install`` with IPython:
https://swcarpentry.github.io/shell-novice/

A Tk GUI for pip would need to frustratingly duplicate ~/.bash_history and
up-arrow to get the previous command.
(Qt and GTK are not consistently installed; though ``conda install spyder``
does install Qt (which is very accessible))

IDLE sucks.

https://devpi.net/docs/devpi/devpi/stable/%2Bd/index.html

https://github.com/kennethreitz/httpbin/blob/master/Dockerfile

These should pretty much get you started:

https://github.com/jrjohansson/scientific-python-lectures

https://github.com/jrjohansson/scientific-python-lectures/blob/master/Lecture-0-Scientific-Computing-with-Python.ipynb

https://github.com/jrjohansson/scientific-python-lectures/blob/master/Lecture-1-Introduction-to-Python-Programming.ipynb

https://github.com/jrjohansson/scientific-python-lectures/blob/master/Lecture-7-Revision-Control-Software.ipynb


GitLab and Mattermost work with an offline LAN:

https://docs.gitlab.com/omnibus/README.html
https://docs.gitlab.com/omnibus/docker/README.html#run-the-image

Binder can pull images from a GitLab Docker Container Registry:
https://docs.gitlab.com/ce/user/project/container_registry.html

These suggestions should help to minimize unnecessary logged bandwidth use
and even work with a closed loop LAN.

See: "#!pip install ipython scipy devpi certbot" in the middle of this
email.

On Tuesday, October 31, 2017, Wes Turner <wes.turner at gmail.com> wrote:

> You could teach them subprocess and os command injection safety from the
> start:
>
> ```python
> import subprocess
> import sys
> cmd = [sys.executable, -m', 'pip', 'install', '-r',
> 'psfblessed-requirements.txt'])
> retcode = subprocess.check_call(cmd)
> assert retcode == 0
> ```
>
> (Because shell=True is dangerous and str.split is dangerous):
>
> ```python
> filename = "'/etc/ passwd' ; shutdown -r now"
> cmd = ("cat '%s'" % filename)
> cmd
> # "cat ''/etc/ passwd'' ; shutdown -r now"
>
> cmd.split()
> # ["'", '/etc', 'passwd', ';', 'shutdown', '-r', 'now']
>
> shlex.split(cmd)
> # ['cmd', '', '/etc', 'passwd', ';', 'shutdown', '-r', 'now']
> ```
>
> (Sarge solves for a number of additional cases beyond shlex.split (empty
> string should be '', 'already-quoted commands')
>
> https://sarge.readthedocs.io/en/latest/overview.html#why-
> not-just-use-subprocess
>
>  https://en.wikipedia.org/wiki/Code_injection#Shell_injection
>
> https://sarge.readthedocs.io/en/latest/internals.html#how-
> shell-quoting-works
>
> Of course, we're programmers and our input is not untrusted, so shell=True
> without any string operations is not as dangerous.
>
>
> On Tuesday, October 31, 2017, Wes Turner <wes.turner at gmail.com
> <javascript:_e(%7B%7D,'cvml','wes.turner at gmail.com');>> wrote:
>
>> You could teach them subprocess and os command injection safety from the
>> start:
>>
>> ```python
>> import subprocess
>> import sys
>> cmd = [sys.executable, -m', 'pip', 'install', '-r',
>> 'psfblessed-requirements.txt'])
>> retcode = subprocess.check_call(cmd)
>> assert retcode == 0
>> ```
>>
>> (Because shell=True is dangerous)
>>
>> On Tuesday, October 31, 2017, Terry Reedy <tjreedy at udel.edu> wrote:
>>
>>> On 10/31/2017 12:21 PM, Ivan Levkivskyi wrote:
>>>
>>>> I think it was proposed several times before, but I just wanted to
>>>> revive the idea that we could add
>>>> a GUI interface to install/update packages from IDLE (maybe even with
>>>> some package browser).
>>>>
>>>
>>> https://bugs.python.org/issue23551.  I agreed with and still agree with
>>> Raymond's opening message in Feb 2015:
>>> "In teaching Python, I find that many Windows users are command-line
>>> challenged and have difficulties using and accessing PIP. ... I would love
>>> to be able to start a class with a fresh Python download from python.org
>>> and effortlessly install requests and other tools without users having to
>>> fire-up a terminal window and wrestle with the various parts."
>>>
>>> The one change I made in Raymond's proposal is that instead of having
>>> multiple IDLE menu entries tied to multiple IDLE functions invoking
>>> multiple pip functions, there would be one IDLE menu entry, perhaps 'Help
>>> => Install packages' (plural intentional), that would invoke a standalone
>>> tkinter based gui front-end to pip.  'Standalone' means no dependency on
>>> IDLE code.  I don't think every IDE or app should *have to* write its own
>>> gui.  Plus, a standalone tkinter module could be invoked from a command
>>> line with 'python -m pipgui' or invoked from interactive python with
>>> 'import pipgui; pipgui.main()'.
>>>
>>> In April 2016, after posting the idea to pydev list and getting 'go
>>> ahead's from Nick Coughlin and someone else, with no negatives, I approved
>>> Upendra Kumar's GSOC proposal to write a pip gui.  This was
>>> https://bugs.python.org/issue27051.  On June 20, Ned Deily and Nick
>>> Coughlin vetoed adding a pip gui anywhere in the stdlib since it depended
>>> on something not in the stdlib, and perhaps for other reasons I don't fully
>>> understand.
>>>
>>> Looking back, I can see that I made two mistakes.
>>>
>>> The first was proposing to use the public-looking pip.main after
>>> importing pip.  It is actually intended to be private (and should have been
>>> named '_main' to make that clearer).  As it turns out, the extra work of
>>> accessing pip through the intended command line interface (via
>>> subprocess) is necessary anyway since running pip makes changes to the
>>> in-memory modules that are not reset when .main is called again.  So it
>>> might as well be used for every access.
>>>
>>> The second was not requiring an approved PEP before proceeding to actual
>>> coding.
>>>
>>> --
>>> Terry Jan Reedy
>>>
>>> _______________________________________________
>>> Python-ideas mailing list
>>> Python-ideas at python.org
>>> https://mail.python.org/mailman/listinfo/python-ideas
>>> Code of Conduct: http://python.org/psf/codeofconduct/
>>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20171101/43a032be/attachment-0001.html>