Mailman 3 March 2019 - Distutils-SIG

proposing Python package index upload API spec (potential PEP)

by Sumana Harihareswara

As a new Twine maintainer I've been running into questions like: * Now that Warehouse doesn't use "register" anymore, can we deprecate it from distutils, setuptools, and twine? Are any other package indexes or upload tools using it? https://github.com/pypa/twine/issues/311 * It would be nice if Twine could depend on a package index providing an HTTP 201 response in response to a successful upload, and fail on 200 (a response some non-package-index servers will give to an arbitrary POST request). I do not see specifications to guide me here, e.g., in the official guidance on hosting one's own package index https://packaging.python.org/guides/hosting-your-own-index/ . PEP 301 was long enough ago that it's due an update, and PEP 503 only concerns browsing and download, not upload. I suggest that I write a PEP specifying an API for uploading to a Python package index. This PEP would partially supersede PEP 301 and would document the Warehouse reference implementation. I would write it in collaboration with the Warehouse maintainers who will develop the reference implementation per pypa/warehouse/issues/284 and maybe add a header referring to compliance with this new standard. And I would consult with the maintainers of packaging and distribution tools such as zest.releaser, flit, poetry, devpi, pypiserver, etc. Per Nick Coghlan's formulation, my specific goal here would be close to: > Documenting what the current upload API between twine & warehouse actually is, similar to the way PEP 503 focused on describing the status quo, without making any changes to it. That way, other servers (like devpi) and other upload clients have the info they need to help ensure interoperability. Since Warehouse is trying to redo its various APIs in the next several months, I think it might be more useful to document and work with the new upload API, but I'm open to feedback on this. After a little conversation here on distutils-sig, I believe my steps would be: 1. start a very early PEP draft with lots of To Be Determined blanks, submit as a PR to the python/peps repo, and share it with distutils-sig 2. ping maintainers of related tools 3. discuss with others at the packaging sprints https://wiki.python.org/psf/PackagingSprints next week 4. revise and get consensus, preferably mostly on this list 5. finalize PEP and get PEP accepted by BDFL-Delegate 6. coordinate with PyPA, maintainers of `distutils`, maintainers of packaging and distribution tools, and documentation maintainers to implement PEP compliance Thoughts are welcome. I originally posted this at https://github.com/pypa/packaging-problems/issues/128 . -- Sumana Harihareswara Changeset Consulting https://changeset.nyc

2 weeks, 3 days

4
4
0 0

PyPI security work: multifactor auth progress & help needed

by Sumana Harihareswara

Work has started on the Open Technology Fund-supported project to improve Warehouse security, accessibility, and internationalization. More details in today's progress report: https://discuss.python.org/t/pypi-security-work-multifactor-auth-progress... best, Sumana Harihareswara Warehouse project manager Changeset Consulting

4 months

2
6
0 0

PyPI will no longer accept compromised passwords!

by Donald Stufft

I just wanted to give everyone a heads up that PyPI will no longer accept passwords that have been published in data breaches. For background you can take a look at https://github.com/pypa/warehouse/pull/4541 <https://github.com/pypa/warehouse/pull/4541>. For high level overview see https://pypi.org/help/#compromised-password <https://pypi.org/help/#compromised-password>. Finally if you have any trouble, please file an issue at https://github.com/pypa/warehouse/issues/new/choose <https://github.com/pypa/warehouse/issues/new/choose>.

5 months, 2 weeks

2
2
0 0

Idea: perennial manylinux tag

by Nathaniel Smith

Hi all, The manylinux1 -> manylinux2010 transition has turned out to be very difficult. Timeline so far: March 2017: CentOS 5 went EOL April 2018: PEP 517 accepted May 2018: support for manylinux2010 lands in warehouse November 2018: support lands in auditwheel, and pip master December 2018: 21 months after CentOS 5 EOL, wwee still don't have an official build environment, or support in a pip release We'll get through this, but it's been super painful and maybe we can change things somehow so it will suck less next time. We don't have anything like this pain on Windows or macOS. We never have to update pip, warehouse, etc., after those OSes hit EOLs. Why not? On Windows, we have just two tags: "win32" and "win_amd64". These are defined to mean something like "this wheel will run on any recent-ish Windows system". So the meaning of the tag actually changes over time: it used to be that if a wheel said it ran on win32, then that meant it would work on winxp, but since winxp hit EOL people started uploading "win32" wheels that don't work on winxp, and that's worked fine. On macOS, the tags look like "macosx_10_9_x86_64". So here we have the OS version embedded in the tag. This means that we do occasionally switch which tags we're using, kind of like how manylinux1 -> manylinux2010 is intended to work. But, unlike for the manylinux tags, defining a new macosx tag is totally trivial: every time a new OS version is released, the tag springs into existence without any human intervention. Warehouse already accepts uploads with this tag; pip already knows which systems can install wheels with this tag, etc. Can we take any inspiration from this for manylinux? We could do the Windows thing, and have a plain "manylinux" tag that means "any recent-ish glibc-based Linux". Today it would be defined to be "any distro newer than CentOS 6". When CentOS 6 goes out of service, we could tweak the definition to be "any distro newer than CentOS 7". Most parts of the toolchain wouldn't need to be updated, though, because the tag wouldn't change, and by assumption, enforcement wouldn't really be needed, because the only people who could break would be ones running on unsupported platforms. Just like happens on Windows. We could do the macOS thing, and have a "manylinux_${glibc version}" tag that means "this package works on any Linux using glibc newer than ${glibc version}". We're already using this as our heuristic to handle the current manylinux profiles, so e.g. manylinux1 is effectively equivalent to manylinux_2_5, and manylinux2010 will be equivalent to manylinux_2_12. That way we'd define the manylinux tags once, get support into pip and warehouse and auditwheel once, and then in the future the only thing that would have to change to support new distro releases or new architectures would be to set up a proper build environment. What do y'all think? -n

8 months, 2 weeks

14
26
0 0

wrong SHA256 on setuptools

by Martin Baker

The SHA256 listed for setuptools-40.8.0.zip is 6e4eec90337e849ade7103723b9a99631c1f0d19990d6e8412dc42f5ae8b304d But the actual SHA sum is 3547552b1009283f7ae31fded32ad33ed160e671 Martin -- Martin Baker Head of Product, FastComp 415.845.0729

8 months, 2 weeks

4
5
0 0

Claim abandoned name: dockenv

by Pat H

Hi, Apologies if this isn't the right distro, lookingat https://www.python.org/dev/peps/pep-0541/#abandoned-projects I wasn't sure: I'd like to apply to take ownership of the project name "dockenv", for use as a different tool. Following the pep: - The project hasn't had an update since 2017 - The owner left no email in any of the releases or account - The homepage url goes to a page that 404s - The owner has had no activity since 2017 on pypi - Download status appear to be around 44 for the last month The tool I plan on releasing is one I am presenting this week at a security conference. The tools uses Docker to create and maintain virtual environments, hence the name dockenv. Hopefully all of this is what you require, but let me know if I can help further. - Pat

8 months, 3 weeks

1
1
0 0

Re: Update PEP 508 to allow version specifiers

by Tzu-ping Chung

> On 15 Mar 2019, at 21:47, Simon Ruggier <simon80(a)gmail.com> wrote: > > The packages have to be available online for dependency links to work, yes, but they're not public: one needs to authenticate with an SSH key to clone each repository. Both --find-links and --index-url (or --extra-index-url) provide the possibility to secure the download with authentication (HTTP Basic Auth). > >> On March 13, 2019 2:09:03 PM UTC, Tzu-ping Chung <uranusjr(a)gmail.com> wrote: >> >>> On 6 Mar 2019, at 03:53, Simon <simon80(a)gmail.com> wrote: >>> >>> I hope it's not an issue that I'm replying to a month-old thread. I reviewed the previous discussion to try to avoid duplicating any of it. >>> >>> When using pip with PyPI, calling pip a second time is much quicker than the first time, because it verifies that the requirements, including version constraints, are satisfied in the target environment and doesn't needlessly reinstall stuff. >>> >>> Dependency links allowed the same behaviour to be implemented for private packages with dependencies on other private repositories: given a requirement B >= 3 and a dependency link that B was available from, pip could check if the environment already includes a package B with a new enough version, and only use the dependency link as a fallback if the requirement isn't already satisfied. >>> >>> URL specifiers aren't useful for providing a fallback location to get a package from, because using one prevents the package from specifying a version constraint in the same way that was possible with dependency links, or with normal requirements available from PyPI. Curiously, discussion of version constraints in this thread has focused on how nonsensical it would be to compare them to the specifying URL, ignoring the possibility of comparing the constraint with the target environment. >>> >>> The loss of this functionality means that anyone who was previously using pip to automatically install private packages with private dependencies now has to either forgo automatic dependency management (a large part of why one would use a package manager to begin with) in favour of recursively specified requirements files, publish their private packages somewhere so that pip can find them, or stick with pip 18.1 for now. >> >> Wouldn’t you still need to “publish the private packages somewhere” for dependency links to work? `setup.py sdist` with `pip --find-links` can get you very far; the only differences IMO is you have to provide a proper package, and write a simple HTML file to point to it. >> >> >>> -- >>> Distutils-SIG mailing list -- distutils-sig(a)python.org >>> To unsubscribe send an email to distutils-sig-leave(a)python.org >>> https://mail.python.org/mailman3/lists/distutils-sig.python.org/ >>> Message archived at https://mail.python.org/archives/list/distutils-sig@python.org/message/BA...

8 months, 3 weeks

2
1
0 0

Great Python... Great History Now, Social Media Internal in Python

by pedroguilhermealvino＠gmail.com

It is very common for us to communicate via Github, Telegram or Whatsapp. Why not make Python a fully self-manageable web media? This would be my contribution to making Python the greatest language ever written on the world. What if we use Python with internal media? For example, if we need help - no servers ... we all connect ... with Twiiter, Telegram inside Python. In this case, it is client to client.

8 months, 3 weeks

1
0
0 0

Re: Update PEP 508 to allow version specifiers

by Simon

I hope it's not an issue that I'm replying to a month-old thread. I reviewed the previous discussion to try to avoid duplicating any of it. When using pip with PyPI, calling pip a second time is much quicker than the first time, because it verifies that the requirements, including version constraints, are satisfied in the target environment and doesn't needlessly reinstall stuff. Dependency links allowed the same behaviour to be implemented for private packages with dependencies on other private repositories: given a requirement B >= 3 and a dependency link that B was available from, pip could check if the environment already includes a package B with a new enough version, and only use the dependency link as a fallback if the requirement isn't already satisfied. URL specifiers aren't useful for providing a fallback location to get a package from, because using one prevents the package from specifying a version constraint in the same way that was possible with dependency links, or with normal requirements available from PyPI. Curiously, discussion of version constraints in this thread has focused on how nonsensical it would be to compare them to the specifying URL, ignoring the possibility of comparing the constraint with the target environment. The loss of this functionality means that anyone who was previously using pip to automatically install private packages with private dependencies now has to either forgo automatic dependency management (a large part of why one would use a package manager to begin with) in favour of recursively specified requirements files, publish their private packages somewhere so that pip can find them, or stick with pip 18.1 for now.

8 months, 3 weeks

2
1
1 0

Automatic and internal Pip package management system

by pedroguilhermealvino＠gmail.com

Hi, how are you? I'm trying to create a thing, an idea, a thought. This is about the Pip package management system used to install and manage software packages written in the Python programming language. So I wanted it to be automatic. Instead of always typing ... For example pip install example.py Solution What if Python itself had this automatic feature internally? For example impot example1 # runs pip install example.py Case A. If you do not have the module, the module will be inserted. B. If you have the module, check the installed version. Summary It would be a 'package.json, similar to the Nodejs' within Python. Note I expect an answer Thank you for your attention and respect, I'm starting with Python and I have this idea If the community helps and if I can help, it would be an honor for this idea, implementation ^^ Regards, Pedro Guilherme About this Note 1 I tried to get an answer on Github and asked to check it out here. If the discourse is good, we can include it in the Github repository for this new feature or implementation. I'm going to thank Mariatta for this tip. Written message, reply " Thanks for your interest in improving Python. However the devguide is not the right place to discuss the proposed idea. From the sound of it, perhaps distutils-sig? " Here I am... Note 2 The reference at the end is a link about it. It was closed in the python devguide and peps repositories in the issues area. I want more clarification about my thinking. I like to create and think. My little visual projects were thanks to what I found, vi, li, I researched in Stackoverflow, Github, Codepen.io. Note 3 I'm trying to translate the Arduino into Portuguese. Library Brasilino ... ^^ And I'm adapting or suggesting the next words ... to be a bit like English .... It's that Portuguese has a graphic accent and English does not. For example, consider the word 'Sr não = If no' associated with this 'else'. So I'm reading dictionaries to find a word next to English ... without a graphic accent on what is considered in English. Resume Here https://github.com/OtacilioN/Brasilino and see in content 'issues' ^^ Regards, Pedro Guilherme Reference https://github.com/python/devguide/issues/469

8 months, 4 weeks

3
5
1 0

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000

1999

1998

Distutils-SIG March 2019