Distutils-SIG December 2020

distutils-sig@python.org

14 participants
7 discussions

Making setup.py run external command to build

by Julian Smith

Hello I have a project with a fairly involved build process where we generate C++ via a python-clang-based code analyser, generates Python bindings for the resulting C++ API with SWIG, and finally compile and link to create various .so's and .py files. I'd like to package things up with a setup.py script, but from what i have read, it looks like distutils expects to compile, link and run swig itself, using distutils.core.setup()'s ext_modules arg? Is there a way to write setup.py so that it instead runs an external command (e.g. 'cd foo && make all') when it is asked to build? Thanks for any help, - Jules -- http://op59.net

2 weeks, 1 day

Twine 3.3.0 released

by Brian Rutledge

https://pypi.org/project/twine/3.3.0/ Changelog (now via towncrier): https://twine.readthedocs.io/en/latest/changelog.html Notable improvements include more `twine upload --verbose` output, and a `--strict` option for `twine check`.

3 months, 3 weeks

Python-Setuptools EOL Versions?

by Symphoni Bush - NOAA Affiliate

Good afternoon, I am trying to find out if there are any end-of-life versions for Python-Setuptools, and if so, when do these versions typically become EOL/unsupported? If this information is publicly stored anywhere please let me know. Thanks.

4 months

AWS Glue: Easy_Install usecase

by simon.jones＠axtria.com

Hi everyone, https://setuptools.readthedocs.io/en/latest/deprecated/easy_install.html instructed me that the mailing list would like to know any use cases where Easy_install is felt needed and I believe I have one. and I do not see any posts about this specific use case. AWS Glue is a serverless runtime environment offered by AWS, usually it runs Spark jobs but Jan 22, 2019 they released Python Shells to run quick snippet codes. they have a fairly comprehensive list of preinstalled libraries and allow you to specify wheels and eggs for anything additional stored on an S3 that you would need, with one limitation - no C-extensions, Cython wheels cannot be installed. for my use-case I needed to access some SFTPs but Paramiko depends on cryptography for SSH shenanigans and cryptography (along with its dependencies cffi, PyNaCL, and bcrypt) are CPython. Turns out if you install at runtime there is no issue - it's just with initial setup that has the limitation. of course you then have to install every time that a process runs during paid runtime, rather than startup but better that then not at all. I tried 2 approaches to install at runtime: subprocess.call(['pip' ...]) and setuptools.command.easy_install.main() subprocess.call to pip installs pure python very nicely, but every time it hit a CPython wheel/tarball - CFFI, PyNaCl, bcrypt, cryptography, and paramiko - took 6-7 minutes EACH just to install (files were provided locally, downloaded from the S3 from previous attempt seeing what would happen if you just gave it CPython ) In a separate run setuptools...easy_install processed everything in 2 seconds. Somewhere between pkg_resources.getdistribution('paramiko').activate(), importing them, and setting host keys lead to Main process starting a little less than 2 minutes later. Required use-case? Not technically Fringe use-case? Fairly Appreciated that this is still around? Very much, 2-minute startup runtime is appreciated over upwards of half an hour on every single run.

4 months

pip 20.3 release (new resolver as default)

by Sumana Harihareswara

On behalf of the PyPA and the pip team, I am pleased to announce that we have just released pip 20.3, a new version of pip. You can install it by running `python -m pip install --upgrade pip`. [Cross-posted to https://discuss.python.org/t/announcement-pip-20-3-release/5948 which will be easier to read in a web browser and to link to.] This is an important and disruptive release -- we [explained why in a blog post last year](https://pyfound.blogspot.com/2019/12/moss-czi-support-pip.html). We even made [a video about it](https://www.youtube.com/watch?v=B4GQCBBsuNU). ## Highlights * **DISRUPTION**: Switch to the new dependency resolver by default. (#9019) Watch out for changes in handling editable installs, constraints files, and more: https://pip.pypa.io/en/latest/user_guide/#changes-to-the-pip-dependency-res… * **DEPRECATION**: Deprecate support for Python 3.5 (to be removed in pip 21.0) (#8181) * **DEPRECATION**: pip freeze will stop filtering the pip, setuptools, distribute and wheel packages from pip freeze output in a future version. To keep the previous behavior, users should use the new `--exclude` option. (#4256) * Substantial improvements in new resolver for performance, output and error messages, avoiding infinite loops, and support for constraints files. * Support for PEP 600: Future ‘manylinux’ Platform Tags for Portable Linux Built Distributions. (#9077) * Documentation improvements: Resolver migration guide, quickstart guide, and new documentation theme. * Add support for MacOS Big Sur compatibility tags. (#9138) The new resolver is now *on by default*. It is significantly stricter and more consistent when it receives incompatible instructions, and reduces support for certain kinds of constraints files, so some workarounds and workflows may break. Please see [our guide on how to test and migrate, and how to report issues](https://pip.pypa.io/en/latest/user_guide/#changes-to-the-pip-dependency-resolver-in-20-3-2020). You can use the deprecated (old) resolver, using the flag `--use-deprecated=legacy-resolver`, until we remove it in the pip 21.0 release in January 2021. You can find more details (including deprecations and removals) [in the changelog](https://pip.pypa.io/en/stable/news/). ## User experience Command-line output for this version of pip, and documentation to help with errors, is significantly better, because you worked with our experts to test and improve it. [Contribute to our user experience work: sign up to become a member of the UX Studies group](https://bit.ly/pip-ux-studies) (after you join, we'll notify you about future UX surveys and interviews). ## What to expect in 20.1 We aim to release pip 20.1 in January 2021, per our [usual release cadence](https://pip.pypa.io/en/latest/development/release-process/#release-cadence). You can expect: * Removal of [Python 2.7](https://pip.pypa.io/en/latest/development/release-process/#python-2-support) and 3.5 support * Further improvements in the new resolver * Removal of legacy resolver support ## Thanks As with all pip releases, a significant amount of the work was contributed by pip's user community. Huge thanks to all who have contributed, whether through code, documentation, issue reports and/or discussion. Your help keeps pip improving, and is hugely appreciated. Specific thanks go to Mozilla (through its [Mozilla Open Source Support](https://www.mozilla.org/en-US/moss/) Awards) and to the [Chan Zuckerberg Initiative](https://chanzuckerberg.com/eoss/) DAF, an advised fund of Silicon Valley Community Foundation, for their funding that enabled substantial work on the new resolver. That funding went to [Simply Secure](https://simplysecure.org/) (specifically Georgia Bullen, Bernard Tyers, Nicole Harris, Ngọc Triệu, and Karissa McKelvey), [Changeset Consulting](https://changeset.nyc/) (Sumana Harihareswara), [Atos](https://www.atos.net) (Paul F. Moore), [Tzu-ping Chung](https://uranusjr.com), [Pradyun Gedam](https://pradyunsg.me/), and Ilan Schnell. Thanks also to Ernest W. Durbin III at the Python Software Foundation for liaising with the project. -- Sumana Harihareswara, pip project manager Changeset Consulting https://changeset.nyc

4 months

Installation Instructions for Python?

by Cort, Brian

Hello, We were trying to create an automated installation process. As part of this, our software deployment team has requested the installation instructions, which I was unable to locate. Can you help by pointing me towards them? Thank you, Brian Brian Cort | Application Analyst Kroger Technology: Point of Sale | The Kroger Co. office: 503.797.3229 cell: 971.293.6244 [cid:image001.png@01D6CD4D.01E06F70] For more POS related information, please feel free to see our publicly accessible Wiki/Confluence Site: https://confluence.kroger.com/confluence/x/BySXBQ ________________________________ This e-mail message, including any attachments, is for the sole use of the intended recipient(s) and may contain information that is confidential and protected by law from unauthorized disclosure. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies of the original message.

4 months, 1 week

Critical problem in PyCharm caused by the removal of "--build-dir" in 2020.3

by Mikhail Golubev

Hi there! I'm a member of the PyCharm team, and we faced a rather critical issue with the release of pip 2020.3 (https://youtrack.jetbrains.com/issue/PY-45712). It turned out that we actually still used the "--build-dir" option in our internal scripts for installing packages through pip, and we completely missed the fact that it was deprecated. To make the matter worse, a few releases back we implemented automatic upgrading of packaging tools in all virtual environments created in the IDE, so now it's impossible to install anything in a new project unless pip is explicitly downgraded first. Judging by how pip now operates regarding building of packages, this option can be safely dropped in our wrappers, but we still need some time to prepare and test patches for a few previous releases and upcoming 2020.3. Is it possible to give us some time to soften the blow for the users? The second question is about the behavior of this option. It appears that we initially started using it because in the past packages were not built in a temporary directory by default. Could you please point me to the exact version of pip where it changed? -- I couldn't find it in a changelog. It would help us decide whether we need to keep some compatibility layer for interpreters with an old version of pip installed. Thanks a lot!

4 months, 2 weeks

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000

1999

1998

Distutils-SIG December 2020