Mailman 3 July 2018 - Distutils-SIG

Issue with using pip install
by Singhal, Monika July 30, 2018

July 30, 2018

I am setting issue installing any package using pip. I am getting below error. Do you have any idea on what could be causing the issue? You should consider upgrading via the 'pip install --upgrade pip' command. (/home/ab63996/test) [ab63996@dwbddisc1r2e ~]$ pip install --trusted-host=files.pythonhosted.org spyder --cert /opt/cloudera/security/x509/cert.pem Collecting spyder Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ProtocolError('Connection aborted.', error(104, 'Connection reset by peer'))': /packages/db/96/59727db154f253ae81a1491481fb53bff0ebf1a6a03d2258e6eb8d57cb66/spyder-3.3.0-py2-none-any.whl Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ProtocolError('Connection aborted.', error(104, 'Connection reset by peer'))': /packages/db/96/59727db154f253ae81a1491481fb53bff0ebf1a6a03d2258e6eb8d57cb66/spyder-3.3.0-py2-none-any.whl Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ProtocolError('Connection aborted.', error(104, 'Connection reset by peer'))': /packages/db/96/59727db154f253ae81a1491481fb53bff0ebf1a6a03d2258e6eb8d57cb66/spyder-3.3.0-py2-none-any.whl Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ProtocolError('Connection aborted.', error(104, 'Connection reset by peer'))': /packages/db/96/59727db154f253ae81a1491481fb53bff0ebf1a6a03d2258e6eb8d57cb66/spyder-3.3.0-py2-none-any.whl Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ProtocolError('Connection aborted.', error(104, 'Connection reset by peer'))': /packages/db/96/59727db154f253ae81a1491481fb53bff0ebf1a6a03d2258e6eb8d57cb66/spyder-3.3.0-py2-none-any.whl Could not install packages due to an EnvironmentError: HTTPSConnectionPool(host='files.pythonhosted.org', port=443): Max retries exceeded with url: /packages/db/96/59727db154f253ae81a1491481fb53bff0ebf1a6a03d2258e6eb8d57cb66/spyder-3.3.0-py2-none-any.whl (Caused by ProtocolError('Connection aborted.', error(104, 'Connection reset by peer'))) You are using pip version 10.0.1, however version 18.0 is available. You should consider upgrading via the 'pip install --upgrade pip' command. (/home/ab63996/test) [ab63996@dwbddisc1r2e ~]$ conda list # packages in environment at /home/ab63996/test: # ca-certificates 2018.03.07 0 certifi 2018.4.16 py27_0 libgcc-ng 7.2.0 hdf63c60_3 openssl 1.0.2o h20670df_0 pip 10.0.1 py27_0 python 2.7.12 1 readline 6.2 2 setuptools 39.2.0 py27_0 sqlite 3.13.0 0 tk 8.5.18 0 wheel 0.31.1 py27_0 zlib 1.2.11 ha838bed_2 CONFIDENTIALITY NOTICE: This e-mail message, including any attachments, is for the sole use of the intended recipient(s) and may contain confidential and privileged information or may otherwise be protected by law. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies of the original message and any attachment thereto.

1 0

Issue with installing python packages using pip
by Monika.singhal＠anthem.com July 30, 2018

July 30, 2018

Hi, I am seeing issue installing python package using pip in my virtual environment. I am getting below error. Could you please assist? I have python version as 2.7.12. # packages in environment at /home/ab63996/test: # ca-certificates 2018.03.07 0 certifi 2018.4.16 py27_0 libgcc-ng 7.2.0 hdf63c60_3 openssl 1.0.2o h20670df_0 pip 10.0.1 py27_0 python 2.7.12 1 readline 6.2 2 setuptools 39.2.0 py27_0 sqlite 3.13.0 0 tk 8.5.18 0 wheel 0.31.1 py27_0 zlib 1.2.11 ha838bed_2 (/home/ab63996/test) [$ pip install --trusted-host=files.pythonhosted.org spyder --cert /opt/cloudera/security/x509/cert.pem Collecting spyder Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ProtocolError('Connection aborted.', error(104, 'Connection reset by peer'))': /packages/db/96/59727db154f253ae81a1491481fb53bff0ebf1a6a03d2258e6eb8d57cb66/spyder-3.3.0-py2-none-any.whl Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ProtocolError('Connection aborted.', error(104, 'Connection reset by peer'))': /packages/db/96/59727db154f253ae81a1491481fb53bff0ebf1a6a03d2258e6eb8d57cb66/spyder-3.3.0-py2-none-any.whl Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ProtocolError('Connection aborted.', error(104, 'Connection reset by peer'))': /packages/db/96/59727db154f253ae81a1491481fb53bff0ebf1a6a03d2258e6eb8d57cb66/spyder-3.3.0-py2-none-any.whl Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ProtocolError('Connection aborted.', error(104, 'Connection reset by peer'))': /packages/db/96/59727db154f253ae81a1491481fb53bff0ebf1a6a03d2258e6eb8d57cb66/spyder-3.3.0-py2-none-any.whl Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ProtocolError('Connection aborted.', error(104, 'Connection reset by peer'))': /packages/db/96/59727db154f253ae81a1491481fb53bff0ebf1a6a03d2258e6eb8d57cb66/spyder-3.3.0-py2-none-any.whl Could not install packages due to an EnvironmentError: HTTPSConnectionPool(host='files.pythonhosted.org', port=443): Max retries exceeded with url: /packages/db/96/59727db154f253ae81a1491481fb53bff0ebf1a6a03d2258e6eb8d57cb66/spyder-3.3.0-py2-none-any.whl (Caused by ProtocolError('Connection aborted.', error(104, 'Connection reset by peer'))) You are using pip version 10.0.1, however version 18.0 is available. You should consider upgrading via the 'pip install --upgrade pip' command.

1 0

Re: Packaging Advice for EFF's Certbot
by Brad Warren July 28, 2018

July 28, 2018

Wow! Thanks for all the feedback everyone. > On Jul 23, 2018, at 6:52 PM, Chris Jerdonek <chris.jerdonek(a)gmail.com> wrote: > > 1) Can you list in the document the non-Python dependencies and what > they're used for to give people an idea? Done! I added this to the “Certbot Background” section at the top of the document. > 2) I don't know how Certbot is architected, but would it be possible > to put the "meat" of Certbot inside a Docker container (ideally > including most of the non-Python dependencies), and then have a much > lighter-weight Python application (with fewer dependencies) running > outside of Docker, and that calls the application inside the Docker > container when it needs to do stuff? Maybe. Currently each method of performing the domain validation required to get a certificate and the support for installing the certificate in different servers is implemented by a plugin to Certbot. These plugins implement a class with a certain interface which it registers as a setuptools entry_point. At runtime, Certbot picks the class type desired by the user, creates an instance of it, and calls its methods as necessary to obtain a certificate, install it, etc. Most of these plugins make use of code in Certbot’s modules that implement common functionality to simplify plugin development. This architecture seems like the opposite of what you’re proposing as Certbot is calling the plugin as needed rather than the other way around, but we’re certainly open to the possibility of changing things to make it easier to distribute Certbot. Are you aware of another significantly sized project that is structured in this way? > On Jul 23, 2018, at 7:04 PM, Eli Ribble via Distutils-SIG <distutils-sig(a)python.org> wrote: > > I think I'd separate out certbot installed by the package manager which is just a simple bootstrapper and the certbot installed in the virtualenv which does real work. This is an interesting idea. By making our own .deb and .rpm packages, we can handle things like installing non-Python dependencies and adding cron jobs or systemd timers in the conventional, system specific way rather than trying to use our single certbot-auto script that tries to work everywhere. It also gives us the option to distribute this Certbot bootstrapper separately on less popular OSes with the caveat that those users will have to install dependencies and set up automating certificate renewal themselves. Thanks for the suggestion! > On Jul 23, 2018, at 7:54 PM, Michael Sarahan <msarahan(a)anaconda.com> wrote: > > > How will separately distributed plugins work? > > Having that layout makes plugins easy, because the paths are both standardized and relocatable (assume relative paths most of the time). Do you know if our approach to using setuptools entry_points to find plugins will work? This is described at https://setuptools.readthedocs.io/en/latest/setuptools.html#dynamic-discove… <https://setuptools.readthedocs.io/en/latest/setuptools.html#dynamic-discove…>. > > I work for Anaconda, and am also a core member of Conda-forge. If you have any questions or just want to talk about how conda/conda-forge may or may not meet your needs, please feel free to reach out. Thanks a lot on this offer. I may take you up on that as I continue to dig into our options and experiment. > On Jul 23, 2018, at 11:10 PM, James Bennett <ubernostrum(a)gmail.com> wrote: > > On Mon, Jul 23, 2018 at 8:17 PM, Alex Walters <tritium-list(a)sdamon.com <mailto:tritium-list@sdamon.com>> wrote: > As a user of certbot, docker, conda, nix, and guix are non-starters. I'm not depending on those tools for my production server (and while docker may be a dependency for some people, that is hardly universal). Adding heavyweight technical dependencies are problematic if your goal is to get everyone using your software. You're better off with cx_freeze or pyinstaller binaries downloaded from a website or a PPA-like-system to add to system package managers, which are not perfect solutions either. > > I would emphasize this point. Not wanting to install a lot of extra software to use Certbot is certainly fair and we’d obviously prefer our packaging solution to be as lightweight as possible. Thanks for bringing this up as a consideration. With that said, our current approaches aren’t working for us. We’re a small development team and continuing to maintain our custom certbot-auto installer which installs dependencies through your OS package manager and creates a virtual environment tucked away in /opt is a significant drain on our resources. If there’s existing tools or projects which can make this easier for us, we’d like to consider them so we can focus our efforts on Certbot and its features rather than packaging and distribution. Our concerns with cx_freeze/pyinstaller and maintaining our own native packaging repos are listed in the Google Doc. If you have information which may help here, please share! Barring someone suggesting something we haven’t considered, maintaining our own native package repos is one of the three options we’re still considering and we haven’t heard many complaints about certbot-auto (when it works). If we go with something that requires significantly more resources though, we’ll definitely take the fact it does this into consideration and would likely offer a lighter weight (but more manual) installation process for sysadmins who prefer it. > The real question here is what use case you're targeting. Our main use case has been the long tail of individuals or small teams of sysadmins who maintain servers and need or want help and automation around maintaining a secure TLS configuration. Making Certbot available to anyone else who wants to use it is just icing on the cake. > On Jul 24, 2018, at 4:36 AM, Nick Coghlan <ncoghlan(a)gmail.com> wrote: > > However, there *are* folks that have been working on allowing > applications to be defined primarily as Python projects, and then have > the creation of wrapper native installers be a pushbutton exercise, > rather than requiring careful human handholding. This is definitely our preferred approach to building native packages right now. To be honest, no one on my team has any experience building .debs and .rpms and we’re happy to learn what we need to if we go with this approach, but the more reliable automation around the process we can use the better. > Regardless, given the design constraints and the target audience, it's > unlikely to be possible to avoid offering a two-level solution, where > the lower platform dependent layer sets up any services and cron jobs > required, as well as establishing a Python virtual environment in > /opt, and then the bulk of certbot, as well as any installed plugins, > run in that venv (plugins would still be managed with pip, just > installed into the certbot venv rather than globally). Outside of FPM’s support for packaging a virtual environment which Freddy Rietdijk mentioned, we really hadn’t considered this approach. I think it’s an interesting idea and could work well for us. I gave a couple more thoughts on this approach in my reply to Eli Ribble who suggested the same thing. > For actually building and publishing those platform dependent pieces, > openSUSE's Open Build Service is the most complete offering I'm aware > of: https://openbuildservice.org/ (Fedora's COPR is good for targeting > RPM based distros, and Ubuntu offer their PPA system, but OBS allows > creation of multiple respository formats). Thanks a lot for this and the other pointers in your post Nick. It’s very helpful!

10 12

pip Install for Python 2.7.6
by Ayesha Anwar July 26, 2018

July 26, 2018

Hi I am Ayesha Anwar student of MS in Remote Sensing and GIS, National University of Sciences and Technology. I am working on my final year project and I am using Python 2.7.6 version on windows 7 (32). I could not use any other python version due to my model requirements. I need to install pip for this python version, I tried both pip files(available on net) to install pip for this version (see attachment) but its not working for this version Kindly send me pip.py file compatible for this version I would be greatfull to you for this kind help. Kind Regards -- Ayesha Anwar MS-15 (RS-GIS) National University of Science and Technology Islamabad, Pakistan Linkedin: Ayesha Anwar https://pk.linkedin.com/in/ayeshaanwar

3 2

Summary of PyPI overhaul in new LWN article
by Sumana Harihareswara July 26, 2018

July 26, 2018

Today, LWN published my new article "A new package index for Python". https://lwn.net/Articles/751458/ In it, I discuss security, policy, UX and developer experience changes in the 15+ years since PyPI's founding, new features (and deprecated old features) in Warehouse, and future plans. Plus: screenshots! If you aren't already an LWN subscriber, you can use this subscriber link for the next week to read the article despite the LWN paywall. https://lwn.net/SubscriberLink/751458/81b2759e7025d6b9/ This summary should help occasional Python programmers -- and frequent Pythonists who don't follow packaging/distro discussions closely -- understand why a new application is necessary, what's new, what features are going away, and what to expect in the near future. I also hope it catches the attention of downstreams that ought to migrate. -- Sumana Harihareswara Warehouse project manager Changeset Consulting https://changeset.nyc

4 9

Export status for setuptools (18.0.1)
by Contact.CCT July 26, 2018

July 26, 2018

Hi Thanks for maintaining setuptools. Do you know whether it is Not controlled, or if it has an Export control classification number (ECCN) in compliance with US export control laws ? (Bureau of Industry and Security website : http://www.bis.doc.gov/) Thanks for your time. Best regards. THALES Group

2 1

Packaging Advice for EFF's Certbot
by Brad Warren July 24, 2018

July 24, 2018

Hi! I work at the Electronic Frontier Foundation on Certbot which is the most popular end user application for obtaining and installing SSL/TLS certificates from Let’s Encrypt. Over the past few years, distributing Certbot has been one of our development team's biggest challenges and we’re currently rethinking how we do so. It was suggested to me that I post to this list to see if anyone was interested in offering advice for how we should approach this. Of course, Certbot is written entirely in Python. If you’re interested, I wrote up a bit of background and what we’re currently thinking at https://docs.google.com/document/d/1y2tc65yWnGuYsun9wsXu7ZRLCVT9eih9eu0pj7A… <https://docs.google.com/document/d/1y2tc65yWnGuYsun9wsXu7ZRLCVT9eih9eu0pj7A…>. Feel free to reach out to me on or off list or on IRC at bmw on Freenode. Thanks so much for any help. Best, Brad Warren Senior Staff Technologist Electronic Frontier Foundation

10 9

Re: Make an ordered list of sdists to be installed?
by Tzu-ping Chung July 23, 2018

July 23, 2018

I just happened to be looking into this area, and may be able to offer some help. Some background information (optional reading): https://github.com/pypa/pipenv/issues/2596#issuecomment-405656806 Sébastien (@sdispater) is the author of Poetry, including the resolver, called Mixology. As he mentioned in the issue, he recently rewrote the resolver, and had yet to decide when and how to extract if from other inner workings of Poetry. The old version of Mixology (before the rewrite), however, is still available on PyPI, and is good enough for this particular use with some additional work. I made a quick, very inefficient implementation, available as a GitHub Gist: https://gist.github.com/uranusjr/a7a9f20c6e43810bd19f0c73e9617182 Hopefully this would help with your need 😊 TP From: distutils-sig-request(a)python.org Sent: 24 July 2018 00:00 To: distutils-sig(a)python.org I don’t know the details, but I did read that Poetry has a sophisticated dependency resolver. https://github.com/sdispater/poetry I don’t know if there is a way to access the resolver independently of the tool, but perhaps it would provide a handy reference. 2018년 7월 23일 (월) 오전 5:49, Thomas Kluyver <thomas(a)kluyver.me.uk>님이 작성: Hi all, Do we know of any tool that can, given the name of one or more packages, follow dependency chains and produce a list of packages in the order they need to be installed, assuming every package needed will be built from source? Running "pip download --no-binary :all: ipython" gets me a set of sdists to be installed, but I lose any information about the order. I assume some packages will fail to build if their dependencies are not installed first, so the order is significant. Pip appears to keep track of the ordering internally: if I run "pip install --no-binary :all: ipython", all the dependencies are downloaded, and then the collected packages are installed starting from those with no dependencies and finishing with the package I requested. But I don't know of any way to get this information out of pip. Is there an option that I'm overlooking? Or some other tool that can do this? The use case I'm thinking about is to automatically generate instructions for a build system which separates the downloading and installing steps, so for each step it expects one or more URLs to download, along with instructions for how to install that piece. The installation steps shouldn't download further data. I could work around the issue by telling it to download all the sdists in a single step and then install in one shot with --no-index and --find-links. But it's more elegant - and better for caching - if we can install each package as a single step. Thanks, Thomas -- Distutils-SIG mailing list -- distutils-sig(a)python.org To unsubscribe send an email to distutils-sig-leave(a)python.org https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/ Message archived at https://mail.python.org/mm3/archives/list/distutils-sig@python.org/message/…

1 0

Make an ordered list of sdists to be installed?
by Thomas Kluyver July 23, 2018

July 23, 2018

Hi all, Do we know of any tool that can, given the name of one or more packages, follow dependency chains and produce a list of packages in the order they need to be installed, assuming every package needed will be built from source? Running "pip download --no-binary :all: ipython" gets me a set of sdists to be installed, but I lose any information about the order. I assume some packages will fail to build if their dependencies are not installed first, so the order is significant. Pip appears to keep track of the ordering internally: if I run "pip install --no-binary :all: ipython", all the dependencies are downloaded, and then the collected packages are installed starting from those with no dependencies and finishing with the package I requested. But I don't know of any way to get this information out of pip. Is there an option that I'm overlooking? Or some other tool that can do this? The use case I'm thinking about is to automatically generate instructions for a build system which separates the downloading and installing steps, so for each step it expects one or more URLs to download, along with instructions for how to install that piece. The installation steps shouldn't download further data. I could work around the issue by telling it to download all the sdists in a single step and then install in one shot with --no-index and --find-links. But it's more elegant - and better for caching - if we can install each package as a single step. Thanks, Thomas

4 3

pip 18.0 has been released!
by Pradyun Gedam July 22, 2018

July 22, 2018

On behalf of the PyPA, I am pleased to announce that pip 18.0 has just been released. To install pip 18.0, you can run python -m pip install --upgrade pip or use get-pip as described in https://pip.pypa.io/en/latest/installing. Note that if you are using a version of pip supplied by your distribution vendor, vendor-supplied upgrades will be available in due course (or you can use pip 18 in a virtual environment). This is the first pip release since adopting 3 month release cadence and a Calendar based versioning scheme (also known as CalVer). In simpler words, there will be a new pip release every 3 months unless there are no changes since the previous release. More details such as release months can be found in pip's development documentation. The highlights of this release are: - Python 3.3 is no longer supported - if you need pip on Python 3.3, you should stay on pip 10, which is the last version to support Python 3.3. - Complete PEP 518 support - includes support for installation of build dependencies from source and Unicode support on Python 2 and Windows - New --prefer-binary flag, to prefer older wheels over newer sdists - Many bug fixes and minor improvements Thanks to everyone who put so much effort into the new release. Many of the contributions came from community members, whether in the form of code, participation in design discussions and/or bug reports. The pip development team is extremely grateful to everyone in the community for their contributions. Thanks, Pradyun

4 4