Distutils-SIG August 2020

distutils-sig@python.org

18 participants
9 discussions

by Pradyun Gedam

On behalf of the PyPA, I am pleased to announce that we have just released pip 20.2, a new version of pip. You can install it by running python -m pip install --upgrade pip. The highlights for this release are: - The beta of the next-generation dependency resolver is available - Faster installations from wheel files - Improved handling of wheels containing non-ASCII file contents - Faster pip list using parallelized network operations - Installed packages now contain metadata about whether they were directly requested by the user (PEP 376’s REQUESTED file) The new dependency resolver is *off by default* because it is *not yet ready for everyday use*. The new dependency resolver is significantly stricter and more consistent when it receives incompatible instructions, and reduces support for certain kinds of constraints files, so some workarounds and workflows may break. Please test it with the --use-feature=2020-resolver flag. Please see our guide on how to test and migrate, and how to report issues <https://pip.pypa.io/en/latest/user_guide/#changes-to-the-pip-dependency-res…>. We are preparing to change the default dependency resolution behavior and make the new resolver the default in pip 20.3 (in October 2020). This release also partially optimizes pip’s network usage during installation (as part of a Google Summer of Code project by McSinyx <https://summerofcode.withgoogle.com/projects/#5428041779511296>). Please test it with pip install --use-feature=fast-deps ... and report bugs to the issue tracker <https://github.com/pypa/pip/issues/new?template=bug-report.md>. This functionality is *still experimental* and *not ready for everyday use*. You can find more details (including deprecations and removals) in the changelog <https://pip.pypa.io/en/stable/news/>. As with all pip releases, a significant amount of the work was contributed by pip’s user community. Huge thanks to all who have contributed, whether through code, documentation, issue reports and/or discussion. Your help keeps pip improving, and is hugely appreciated. Specific thanks go to Mozilla (through its Mozilla Open Source Support <https://www.mozilla.org/en-US/moss/> Awards) and to the Chan Zuckerberg Initiative <https://chanzuckerberg.com/eoss/> DAF, an advised fund of Silicon Valley Community Foundation, for their funding that enabled substantial work on the new resolver.

1 month

Failure: Error

by oliviajohn123123＠hotmail.com

facing same NOt found issue the issue code is "C:\Users\jaraco\projects\distutils_namespace\myns.projB\test\testBasic.py", can you guide me for this ? https://thelifefoundry.com/hotmail/

2 months

Kasauti Zindagi Ki

by Kazmi Bro

I like your post i got some good information thanks https://yehrishtakyakehlatahaiepisode.com/category/kasauti-zindagi-ki/

3 months

Yeh Rishta Kya Kehlata Hai

by Kazmi Bro

Hi, I really like you post and I was looking such information finally I got it many thanks for writing. <a href="https://yehrishtakyakehlatahaiepisode.com/"><strong>Yeh Rishta Kya Kehlata Hai</strong></a>

3 months

Thursday Aug 20th: short talk on PyPI & sustainability

by Sumana Harihareswara

Two researchers studied PyPI's sustainability to understand problems that affect FOSS infrastructure sustainability in general. On Thursday 20 Aug (2:30 - 3:45 PM EDT), one of the study's authors will speak in a Ford Foundation event about FOSS sustainability: https://www.eventbrite.com/e/digital-infrastructure-labor-roles-and-incenti… You can read the researchers' report at https://github.com/FOSSRIT/mismatches/blob/master/Mismatches%20Final%20repo… . Key section of the summary: > These results suggest that different strategies are needed for addressing the non-technical capacities of digital FOSS infrastructure projects. While capacity for things such as documentation, outreach, project management, design, legal work, etc. are often acknowledged as needs within FOSS communities at-large, they are rarely addressed proactively. Rather, they are often addressed only when a project is in crisis. I also draw your attention to the table of tensions between "FOSS culture" and "Infrastructure culture" on page 7, and the misperception that capacity-building may seem to be slowing a project down (p 6-7). (Disclosure: I was a consultant on this analysis.) -- Sumana Harihareswara Changeset Consulting https://changeset.nyc

3 months, 2 weeks

pip and missing shared system system library

by David Mathog

pip install package often results in compiling (using gcc, g++, whatever) to produce a binary. Usually that proceeds without issue. However, there seems to be no checking that the libraries required to link that binary are already on the system. Or at least the message which results when they are not is not at all clear about what is missing. I discovered that today by wasting several hours figuring out why scanpy-scripts was failing trying to build dependency "louvain", which would not install into a venv with pip. It had something to do with "igraph", but pip had downloaded python-igraph before it got to louvain. When louvain tried to build there was a mysterious message about pkgconfig and igraph Cannot find the C core of igraph on this system using pkg-config. (Note that when python-igraph installs it places an igraph directory in site-packages, so which it is referring to is fairly ambiguous.) Then it tried to install a different version number of igraph, failed, and the install failed. This was very confusing because the second igraph install was not (it turned out) a different version of python-igraph but a system level igraph library, which it could not install either because the process was not privileged and could not write to the target directories. Yet it tried to install anyway. This is discussed in the louvain documentation here (it turns out): https://github.com/vtraag/louvain-igraph but since I was actually trying to install a different package, of course I had not read the louvain documentation. In short form the problem was "cannot build a binary because required library libigraph.so is not present in the operating system" but that was less than obvious in the barrage of warnings and error messages. Is it possible to tell pip or setup.py to fail immediately when a required system library like this is not found, here presumably after that "C core" message, rather than confusing the matter further with a failed partial build and install of the same component? More generally, is there anything in the python installation methods which could list system libraries as dependencies and give a more informative error message when they are missing? Thanks, David Mathog

3 months, 2 weeks

twine upload & network robustness

by Robin Becker

Because of covid-19 I am working a lot from home so my network is likely less robust. I was wondering about what happens if I upload with twine to pypi and halfway through my network fails. Is the uploaded filename remembered so I cannot retry when my network comes back? Or is the upload somehow atomic so that it has to complete(be checked) before the file is entered into pypi? -- Robin Becker

3 months, 3 weeks

Archive this list & redirect conversation elsewhere?

by Pradyun Gedam

TL;DR: OK to archive this mailing list? Reply by Aug 30th. Below: Context, Proposal, Reasoning, and timeline. *Context*: For multiple years now, we've been advising users to use setuptools and to move away from using distutils. There has been a long standing plan [^1] to vendor distutils into setuptools to allow it to evolve independently of the Python standard library and to allow for removal of distutils from the Python standard library. Recently, setuptools adopted the distutils [^2] which, as far as I can tell, starts the long process of removing distutils from the Python standard library. PEP 517 [^3] also removed the special status of distutils/setuptools as the only pipeline that can be used for generating distributions for Python projects. For some time now, "distutils" has not been a "primary" tool in the broader Python Packaging ecosystem, with setuptools being an overall superior tool that also has a good interoperability story with the other Python packaging tooling. This is acknowledged in the description of this mailing list as: > Now, it's better described as the "packaging interoperability SIG", > where issues that cut across different parts of the Python packaging > ecosystem get discussed and resolved. However, this mailing list is no longer serving this stated role either, with the Packaging category on discuss.python.org becoming the primary location for packaging tool interoperability discussions. Over the last year, the Packaging category on discuss.python.org had 841 active topics, with only 40 topics with 3 or fewer responses. [^5] In the last 100 days, the Packaging category on discuss.python.org has had 91 active topics. More than 10 PEPs have been discussed in the Packaging category on discuss.python.org in the last 100 days. Over the last year, distutils-sig had ~109 active threads, with (based on a quick skim) most having 3 or fewer responses/posters. [^4] In the last 100 days, distutils-sig has had 32 active threads (at least 7 of these have the same subject as another thread with Re:/Fwd: added). There has been only 1 PEP-related feedback discussion on distutils-sig in the last year. Most of the other threads are user support requests or announcements. *Proposal*: I suggest that, one month from now, we stop posting to this list (distutils-sig(a)python.org) and archive it. *Reasoning*: I think we do not use this mailing list for its dedicated purpose, and it does not serve any secondary function that isn't better served by a different communication channel already. (1) this mailing list is no longer the primary location for interoperability discussions (2) we have better channels for user support requests (such as issue trackers of various projects, packaging-problems etc.) (3) we have other channels for making announcements (such as the pypi-announce mailing list, the PyPA twitter account, discuss.python.org etc.) *Timeline*: Here's what I suggest, and what I will carry out if there is no objection. In one month, on August 30th, I would verify that no one has argued here for why this mailing list should not be closed/archived. Or, even if a few people have objected to closing the list, I would check for rough consensus, especially of people who are doing SOMETHING productive having to do with Python Packaging (such as maintainers of Python packages, maintainers of Python Packaging tooling, folks running key infrastructure, etc.). Then, I would request the list administrators to post a final message to this list (marking its close and suggesting that people use discuss.python.org instead) and, to archive this mailing list. This would leave archives available at their current URLs, so links, browsing and search would work. And finally, I would look through relevant documentation within PyPA repositories to see what needs updating (READMEs and so on pointing to the old list), and submit pull requests. I appreciate the work folks here have done to carry forward Python packaging over the past 21 years of this mailing list. I don't mean to diminish that or to insult anyone here. I want to help us out, and I think closing this list will help focus our energy better. But I am open to hearing that I am wrong. Realizing-that-distutils-sig-is-older-than-me-ly, Pradyun Gedam [^1]: https://www.pypa.io/en/latest/roadmap/#vendor-distutils-into-setuptools [^2]: https://github.com/pypa/setuptools/pull/2143 [^3]: https://www.python.org/dev/peps/pep-0517/ [^4]: https://mail.python.org/archives/list/distutils-sig@python.org/latest?count… [^5]: https://discuss.python.org/c/packaging/14/l/top/yearly?order=posts

3 months, 3 weeks

Best way for a project to provide an optional C module

by Daniele Varrazzo

Hello, packaging psycopg3, I'm wondering what is the best way to provide an optional optimisation module. 1) provide a separate psycopg3-c distribution 2) provide an extra psycopg3[c] 3) try building the extension and fail quietly. 1) seems the cleanest approach: the psycopg3 distribution would have no build-time external dependency (Cython, -dev packages, a compiler) and psycopg3-c can fail hard if some of these dependencies are missing. I am currently trying this approach, finding some problems in working out a good files layout to have two setup.py in the same git repository. 2) would be nice but I don't see a way to identify the extra requested at build time to implement a build_ext command such that if "c" is not the extra then don't do anything. it seems that extra are thought for a different use case, not for optional build-time parts 3) would give me endless headaches to work out why something failed, differentiate failures for missing dependencies from real errors, and dealing with user reports. What would be your advice? Press on with 1 or a different approach? Examples are welcome (the only one I have in mind is PyYAML doing 3). -- Daniele

3 months, 3 weeks

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000

1999

1998

Distutils-SIG August 2020