Distutils-SIG May 2018

distutils-sig@python.org

31 participants
19 discussions

appveyor updateb broken by changes to pip

by Robin Becker

Not exactly sure if this is the issue, but my appveyor test script contains testenv\scripts\pip.exe install --cache-dir=dist --find-links file:dist reportlab where in turn reportlab setup.py contained install_requires=['pillow>=2.4.0']. After an upgrade sometime between Apr 26 and May 18 this started to fail because pip could no longer find a pillow>=2.4. > "=============================== Python27 TEST START" > 941c:\Python%PYTHON%\python.exe -mvirtualenv testenv > 942New python executable in C:\projects\reportlab\testenv\Scripts\python.exe > 943Installing setuptools, pip, wheel...done. > 944testenv\scripts\pip.exe install --cache-dir=dist --find-links file:dist reportlab > 945Looking in links: file:dist > 946Collecting reportlab > 947Collecting pillow>=2.4.0 (from reportlab) > 948 Could not find a version that satisfies the requirement pillow>=2.4.0 (from reportlab) (from versions: ) > 949No matching distribution found for pillow>=2.4.0 (from reportlab) > 950Command exited with code 1 > 951myenv\scripts\python.exe -mrl_ci_tools email --subject="!!!!! APPVEYOR %APPVEYOR_PROJECT_NAME% build failed" --faddr=rl-wheelbuilder --body="see https://ci.appveyor.com/project/MrRLBitBucket/%PROJECT_NAME%/history" I had assumed that pip would fail to find pillow in the dist folder and then move on to somewhere more useful eg pypi, but with pip 10.0.1 this seems not to happen. It's easy to fix with a separate install of the, but what is the correct way to indicate dependencies which will allow installation from my freshly built wheel? -- Robin Becker

3 years, 4 months

Manual PIP upgrade for TLS 1.2 support fails.

by edlipsongm＠gmail.com

I'm trying to install python 3.5.5 on RHEL without root. I used the referenced --prefix to identify where the install is (per webpage). This worked in 3.5.4. PIP installs, but fails to install other modules. Searching for a solution I found a reference to an external way to upgrade PIP to a version where this problem is supposed to be resolved. I obtainde get_pip.py and ran it, but it too fails (below) for TLS issues. Policy says I have to obscure anything related to my company, hence the paths have been edited in this email. Are there any other solutions? bash-3.2$ which python <installpathObscuredbyPolicy>/Python-3.5.5/python -bash-3.2$ python <homepathObscuredbyPolicy >/python/get-pip.py Collecting pip /tmp/tmp9qehrweu/pip.zip/pip/_vendor/urllib3/util/ssl_.py:339: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLError(1, '[SSL: TLSV1_ALERT_PROTOCOL_VERSION] tlsv1 alert protocol version (_ssl.c:719)'),)': /simple/pip/ Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLError(1, '[SSL: TLSV1_ALERT_PROTOCOL_VERSION] tlsv1 alert protocol version (_ssl.c:719)'),)': /simple/pip/ Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLError(1, '[SSL: TLSV1_ALERT_PROTOCOL_VERSION] tlsv1 alert protocol version (_ssl.c:719)'),)': /simple/pip/ Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLError(1, '[SSL: TLSV1_ALERT_PROTOCOL_VERSION] tlsv1 alert protocol version (_ssl.c:719)'),)': /simple/pip/ Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLError(1, '[SSL: TLSV1_ALERT_PROTOCOL_VERSION] tlsv1 alert protocol version (_ssl.c:719)'),)': /simple/pip/ Could not fetch URL https://pypi.org/simple/pip/: There was a problem confirming the ssl certificate: HTTPSConnectionPool(host='pypi.org', port=443): Max retries exceeded with url: /simple/pip/ (Caused by SSLError(SSLError(1, '[SSL: TLSV1_ALERT_PROTOCOL_VERSION] tlsv1 alert protocol version (_ssl.c:719)'),)) - skipping Could not find a version that satisfies the requirement pip (from versions: ) No matching distribution found for pip Could not fetch URL https://pypi.org/simple/pip/: There was a problem confirming the ssl certificate: HTTPSConnectionPool(host='pypi.org', port=443): Max retries exceeded with url: /simple/pip/ (Caused by SSLError(SSLError(1, '[SSL: TLSV1_ALERT_PROTOCOL_VERSION] tlsv1 alert protocol version (_ssl.c:719)'),)) - skipping

3 years, 4 months

requirements.txt or not requirements.txt?

by Carles Sala Cladellas

Hello here! TL;DR: Should I use requirements.txt, or should I have my dependencies only listed inside setup.py? Long version: The two most common ways to have dependencies listed in a python package are either having them listed both in setup.py and requirements.txt files, or having them only listed in requirements.txt and then import them using some `parse_requirements` methods. In the projects which I am currently working on there are five lists of requirements: - requirements.txt - requirements_test.txt - requirements_dev.txt - install_requires inside setup.py - tests_require inside setup.py I wanted to remove duplicities, so I thought about removing the setup.py lists and parse the dependencies from the requirements*.txt files. But then I came across this, https://stackoverflow.com/a/28842733, which recommends using a `dev` entry in `extras_require` and then using `pip install -e .[dev]` to install them, which could be easily extended to also have a `pip install -e .[test]`. I find this a very clean option, since it reduces the amount of files the repository and makes setup.py and makes setup.py self-contained. Are any known drawbacks to this approach? Is it a good way to go? Regards, -- Carles Sala Cladellas carles(a)pythiac.com / +34 666 278 679 https://www.pythiac.com Pythia Consulting - Data Science as a Service

3 years, 4 months

Python 3.7 binary wheels

by Lele Gaifax

Hi all, Python 3.7 is steadily approaching its final release and I start wondering a) when will be the right time to start uploading 3.7 binary wheels on PyPI? b) if/when manylinux2010 happens, does that mean that I should build both manylinux1 and manylinux2010 variants? Thank you in advance, ciao, lele. -- nickname: Lele Gaifax | Quando vivrò di quello che ho pensato ieri real: Emanuele Gaifas | comincerò ad aver paura di chi mi copia. lele(a)metapensiero.it | -- Fortunato Depero, 1929.

3 years, 4 months

Build a module of SWIG interfaces in a single invocation

by Clifford Simon

Dear Distutils SIG, I am asking about the best way to build a module, when the sources are SWIG interface files (*.i). SWIG is supposed to generate the Python scripts. I can build the module by running build_ext and then build_py, but I cannot build the module with build because it runs build_py and build_ext in the wrong order, since the Python scripts don't exist until build_ext has run. I have asked this on StackOverflow, here: https://stackoverflow.com/questions/50239473/building-a-module-with-setupto…, but my question attracted very little interest there. Thanks for the help, Clifford Simon

3 years, 4 months

This list will soon be migrating to Mailman 3

by Nick Coghlan

Hi folks, The recent discussion of communication workflows prompted me to investigate exactly what would be involved in migrating the list to Mailman 3 (with its native web gateway and other account management improvements), and I'm happy to report that there don't appear to be any blockers to our initiating the migration. I'm currently planning to send the migration request to postmaster(a)python.org later this week (probably Thursday evening), but don't have an ETA for how long the actual migration may take after that (as I believe distutils-sig will be one of the larger python.org list migrations undertaken so far). Accordingly, this is an initial heads up as to exactly what the change will mean: 1. If you only ever access the mailing list via email, and never tinker with your subscription settings or look up threads in the list archives, then you shouldn't notice any real changes other than some of the headers on mails from the list changing a bit (such as the new Archived-At header appearing on each message). 2. If you do use the website to change your subscription settings or look up threads in the list archives, or have wished you had a more web-forum-like interface for accessing the list, then the rest of this email is likely to be of interest :) == Changes to subscription management (and list moderation) == Mailman 3 relies on a more conventional user account management model than the historically list-centric model in Mailman 2. This means that either before or after the migration, folks that want to modify their subscription settings will need to go to https://mail.python.org/mm3/ and register for an account. If you use GitLab, GitHub, Google, or Facebook, then you can go straight to https://mail.python.org/mm3/accounts/login/, select one of those options, and grant the required access to look up your email address. (At least for GitHub, the request will come from Mark Sapiro's developer key, and I believe that's the case for the other services as well) If your address on the linked service matches your subscription address, then you're done. If it doesn't match, then you can head to https://mail.python.org/mm3/accounts/email/ to register more addresses (and hence link any related subscriptions to your account). If you don't use any of those services, or simply don't want to use them with mail.python.org, then head to https://mail.python.org/mm3/accounts/signup/ to create a conventional username-and-password based account. Regardless of how you sign up, the primary authentication mechanism is access to the relevant email address - the old MM2 plain text email password isn't used at all. After the migration, the current https://mail.python.org/mailman/listinfo/distutils-sig URL will automatically redirect to https://mail.python.org/mm3/mailman3/lists/distutils-sig.python.org/ For a working example of what that will look like, see https://mail.python.org/mm3/mailman3/lists/core-workflow.python.org/ == Changes to list archiving (and the native web gateway) == After the migration, the current list archive at https://mail.python.org/pipermail/distutils-sig/ will remain in place in order to preserve existing links, but will no longer be updated with new messages. That page will also be updated with a link to the new archiver/web gateway page. That page will be at https://mail.python.org/mm3/archives/list/distutils-sig@python.org/, and not only features stable and predictable URLs for each post, but also includes a native web gateway, allowing folks to both create new threads and reply to existing ones using the web page, without needing to explicitly subscribe to the list first. Again, core-workflow provides an example of what that will look like in practice, with the new archive at https://mail.python.org/mm3/archives/list/core-workflow@python.org/, and the post-migration legacy archive at https://mail.python.org/pipermail/core-workflow/. Regards, Nick. -- Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia

3 years, 4 months

org-mode README file formats

by Diane Trout

Hi, I was building a package where I had a README.org file, which setuptools couldn't find. It listed .md as a valid format, so I was wondering if org-mode was sufficiently plain text to be added to the list of accepted README file formats? For what it's worth, github-markup can handle .org files https://github.com/github/markup Diane

3 years, 4 months

Packaging/Warehouse sprint at PyCon 2018

by Sumana Harihareswara

In case you're planning your PyCon Cleveland travel: we are planning to hold a Warehouse/packaging sprint at PyCon (the sprints are Monday, May 14th - Thursday, May 17th 2018). We welcome package maintainers, backend and frontend web developers, infrastructure administrators, technical writers, and testers to help us make the new PyPI, and the packaging ecosystem more generally, as usable and robust as possible. I took the liberty of updating https://us.pycon.org/2018/community/sprints/ to say so. Once we're closer to the sprints I'll work on a more detailed list of things we'll work on in Cleveland. -- Sumana Harihareswara Changeset Consulting https://changeset.nyc

3 years, 4 months

(Final) PyPI/Warehouse weekly report: legacy is shut down

by Sumana Harihareswara

As I announced yesterday[1], here and on the pypi-announce[2] and general Python announcement[3] lists, we have shut down legacy.pypi.org, on schedule. (See the notes from our final weekly call[4], a screenshot of all our closed milestones[5], a screenshot of the hit ratio for Legacy going to 0%[6], and Ernest pouring out a toast to the old codebase[7].) This is the last weekly report you'll get from me on this project, as the MOSS funding has nearly run out (we set aside a little for me to run the PyCon sprint and for Nicole to run the EuroPython sprint). Thanks so much to Mozilla's Open Source Support program for the award[8] that enabled this work[9]. And thanks to the PSF and its Packaging Working Group[10] for facilitating it. Highlights from the last week: The podcast Talk Python To Me released an episode interviewing Dustin Ingram, Nicole Harris, and Ernest W. Durbin III about Warehouse -- you can listen[11] or read the transcript[12]. And the Python Bytes podcast had a short chat about Warehouse[13] as well. Ernest sunset Legacy[14], fixed a subsequent outage[15] (my fault for putting a hostname in the title of a blog post!), updated a cabotage setting[16], updated CDN configuration[17], and fixed another service disruption[18]. And he improved search for XML-RPC endpoint users[19]. Since we got 1700+ responses to the "buy a feature" survey[20], we took down the banner[21] -- Nicole notes that the data is really useful and will really help with redesigning the project detail page! She also fixed modal alignment[22] and table alignment[23] in IE11. Dustin replaced our Twisted usage with gunicorn[24] and fixed an edge case concerning identical canonical versions of a release[25], and Dustin and Ernest made old pypi.python.org links for files, display actions[26], search and browse actions[27] redirect appropriately. And Dustin merged "Support XML-RPC multicall"[28] and then "Skip tweens for XML-RPC multicall subrequests"[29] then "Deprecate XML-RPC MultiCall"[30] and I think we've all had sequences like that in our lives. Laura Hampton and I ran a Warehouse sprint night[31] in New York City (giving participants several tasks at varying difficulty levels[32]), where Corey Girard helped us make profile pages display "you" versus a username more logically[33] -- thanks, Corey! -- and Kshitij Chawla found a setup issue[34]. And the team found some more developer experience snags and got to fixing them: PyPUG instructions[35], the README[36], Docker instructions[37], discoverability for the architecture overview[38]. We are slowing down a bit on pull request review and issue response as our dedicated time on Warehouse comes to a close, but we still did a lot of review and replying. Thanks to the volunteers who got pull requests merged in the past week: * nixjdm, who added description_content_type to the JSON API[39] * cheungnj, who improved how we display the "last released" date on a project[40] * aalmazan, who fixed how we handle tab cycling inside active modals[41] * alex, who fixed a pytest argument[42] * kpayson64, who updated wheel types Warehouse supports[43] (see the followup conversation, on whether PyPI should allow Linux wheel uploads for ARM[44]) Special shoutout to GitHub user jdufresne[45] who has submitted a bunch of pull requests to various projects, including setuptools[46], updating their URLs from pypi.python.org to pypi.org (example[47]). I've done some similar issue-opening (example[48]). And thanks to Donald Stufft for helping with the infrastructure changeover[49]! You can help by: * updating the distutils docs[50] to reflect how PyPI currently works * giving yeraydiazdiaz feedback on this approach to automated frontend testing[51] * keeping an eye on Warehouse pull requests and reviewing[52] them * telling hiring managers you know to consider hiring Ernest[53] and giving him paid time to work on PyPI * finding us at PyCon North America[54] and giving us friendly feedback Dustin, Ernest, Laura, Nicole and I will continue volunteering a few hours per week around here, just as many of us did before the project. We're all grateful we got to work together and make this happen, and hope to have further paid opportunities to dedicate time to this infrastructure and its symbiotic community. -- Sumana Harihareswara (basically my last note as) Warehouse/PyPI project manager PyPA member Packaging Working Group member Changeset Consulting -- open to new client engagements starting in June/Julysh(a)changeset.nyc Links: 1. https://mail.python.org/mm3/archives/list/pypi-announce@python.org/thread/2… 2. https://mail.python.org/mm3/archives/list/pypi-announce@python.org/thread/2… 3. https://mail.python.org/pipermail/python-announce-list/2018-April/011916.ht… 4. https://wiki.python.org/psf/PackagingWG/2018-04-30-Warehouse 5. https://twitter.com/di_codes/status/990982434572439554 6. https://twitter.com/EWDurbin/status/990959524768112640 7. https://twitter.com/EWDurbin/status/990966616803422208 8. https://pyfound.blogspot.com/2017/11/the-psf-awarded-moss-grant-pypi.html 9. https://blog.mozilla.org/blog/2018/01/23/moss-q4-supporting-python-ecosyste… 10. https://wiki.python.org/psf/PackagingWG 11. https://talkpython.fm/episodes/show/159/inside-the-new-pypi-launch 12. https://talkpython.fm/episodes/transcript/159/inside-the-new-pypi-launch 13. https://pythonbytes.fm/episodes/show/75/pypi.org-officially-launches 14. https://status.python.org/incidents/ptvp1wnn0jmq 15. https://status.python.org/incidents/ttdz2bj310xw 16. https://github.com/cabotage/cabotage-app/commit/6e7eafce737726a04bd6ce08968… 17. https://status.python.org/incidents/w9zkk6k75xrc 18. https://status.python.org/incidents/0gyxs265qp6b 19. https://github.com/pypa/warehouse/pull/3827 20. https://twitter.com/nlhkabu/status/988856279526465537 21. https://github.com/pypa/warehouse/pull/3854 22. https://github.com/pypa/warehouse/pull/3841 23. https://github.com/pypa/warehouse/pull/3839 24. https://github.com/pypa/warehouse/pull/3865 25. https://github.com/pypa/warehouse/pull/3822 26. https://github.com/pypa/warehouse/pull/3856 27. https://github.com/pypa/warehouse/pull/3800 28. https://github.com/pypa/warehouse/pull/3778/files 29. https://github.com/pypa/warehouse/pull/3811/files 30. https://github.com/pypa/warehouse/pull/3821/files 31. https://www.meetup.com/nycpython/events/249722192/ 32. https://docs.google.com/presentation/d/1JHTvs58DaHJzha8aN8oz_IYaiyZlupgioAB… 33. https://github.com/pypa/warehouse/pull/3837 34. https://github.com/pypa/warehouse/issues/3836 35. https://github.com/pypa/python-packaging-user-guide/pull/471 36. https://github.com/pypa/warehouse/pull/3863 37. https://github.com/pypa/warehouse/pull/3871 38. https://github.com/pypa/warehouse/pull/3864 39. https://github.com/pypa/warehouse/pull/3820 40. https://github.com/pypa/warehouse/pull/3481 41. https://github.com/pypa/warehouse/pull/3619 42. https://github.com/pypa/warehouse/pull/3805 43. https://github.com/pypa/warehouse/pull/2010 44. https://github.com/pypa/warehouse/issues/3668 45. https://github.com/jdufresne 46. https://github.com/pypa/setuptools/pull/1340 47. https://github.com/joke2k/faker/pull/750 48. https://github.com/ralic/Lein_Reagent/issues/1 49. https://github.com/python/pypi-infra/pull/24 50. https://docs.python.org/3.6/distutils/packageindex.html#package-index 51. https://github.com/pypa/warehouse/pull/3450 52. https://warehouse.readthedocs.io/development/reviewing-patches/ 53. https://ernest.ly/blog/working 54. https://wiki.python.org/psf/PackagingSprints

3 years, 4 months

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000

1999

1998

Distutils-SIG May 2018