Mailman 3 January 2018 - Distutils-SIG

pipenv best practices?
by Chris Withers Jan. 16, 2018

Jan. 16, 2018

Hi All, Couldn't find an obviously better discussion forum than this for pipenv stuff, but please point me to where the rest of the discussions are happening... So, with pipenv, what files do we version control for a project? both Pipfile and Pipfile.lock? Hopefully one I missed from the docs: with the correct files source controlled, how do I reproduce the environment on another machine? Continuing on the deployment theme: I'm used to being able to put /path/to/some/ve/bin/some_script in crontabs and the like, where some_script comes form a setuptools entry point. What's the canonical way of doing this with pipenv? Last up, how should pipenv be used for library development? (ie: where dependencies and minimal constraints are expressed in setup.py and where you want to test against as many python versions and library combinations as you support). Other than sadness around capitalisation of the config file names, it looks really exciting :-) cheers, Chris

4 7

(no subject)
by bige12655 . Jan. 16, 2018

Jan. 16, 2018

Thanks

1 0

Fwd: Warehouse update: recent progress
by Sumana Harihareswara Jan. 16, 2018

Jan. 16, 2018

Forwarding from pypa-dev. -------- Forwarded Message -------- Subject: Warehouse update: recent progress Date: Mon, 15 Jan 2018 22:32:45 -0500 From: Sumana Harihareswara <sh(a)changeset.nyc> To: pypa-dev(a)googlegroups.com Happy new year! After the late-December holiday season, the Warehouse team had a meeting and followup chats last week to discuss the first milestone, the MVP for package maintainers to try Warehouse and give feedback.[0] Full notes from a January 10th meeting and its followups are on the PSF wiki.[1] Highlights from that meeting and related work in the last week and change: * A number of items were blocked on Nicole Harris's availability -- she's the designer working on our front-end views. We sorted out things she needs and are making better progress on those now. * Donald wrote to distutils-sig[2] about removing OpenID/Google login support for PyPI.[3] * Ernest and Nicole are working on a maintainer views meta-issue to "Classify, describe, and share the *current* functionality implemented by PyPI to assist contributors while developing warehouse's maintainer UI."[4] * The Warehouse roadmap, which previously only lived in GitHub milestone descriptions, is now all in one wiki page for easier reading and linking.[5] * Dustin has been working on adding the ability for a project owner to add/remove roles for their projects.[6] * Ernest has been continuing to work on deployment infrastructure, which is getting more robust and resilient. * We've created, reviewed, improved, and/or merged several other pull requests to Warehouse and related projects. In particular, development is now getting easier, as we have an SMTP service for development[7] and a better Makefile[8], and are getting better testing instructions.[9] We are incrementally getting a clearer picture of what work is left in this milestone so I can get you an estimate of when the Maintainer MVP will be. And Warehouse continues to get easier to hack on; feel free to join us on Freenode in case you want any help as you poke around the Warehouse code![10] [0] https://github.com/pypa/warehouse/milestone/8 [1] https://wiki.python.org/psf/PackagingWG/2018-01-10-Warehouse [2] https://mail.python.org/pipermail/distutils-sig/2018-January/031855.html [3] https://github.com/pypa/warehouse/issues/61 [4] https://github.com/pypa/warehouse/issues/2734 [5] https://wiki.python.org/psf/WarehouseRoadmap [6] https://github.com/pypa/warehouse/pull/2705 [7] https://github.com/pypa/warehouse/pull/2709 [8] https://github.com/pypa/warehouse/pull/2728 [9] https://github.com/pypa/warehouse/pull/2758 [10] https://webchat.freenode.net/?channels=%23pypa-dev -- Sumana Harihareswara Changeset Consulting https://changeset.nyc

1 0

Re: [Distutils] Should abi3 tag be usable on Windows?
by Cosimo Lupo Jan. 13, 2018

Jan. 13, 2018

Thank you, Steve. Ok so, setuptools can successfully build extension modules with Py_LIMITED_API on Windows, with a generic *.pyd suffix and no extra tags. However, there are still two problems before I can distribute them: 1) it's not clear how wheels containing such extension modules should be tagged on Windows, given that `abi3` tag is not recognized for that platform. The function imp.get_suffixes() used by both pip and wheel to get the list of supported tags does not return any abi3 suffixes on Windows, and for this reason the bdist_wheel command assumes from this that Windows does not support extensions built with Py_LIMITED_API, and gives an error. Some have resorted to tagging their Windows wheels with abi "none" to work around this, however "none" seems to imply that extension does not use the Python ABI which is not true. Alex Grönholm (current maintainer of pypa/wheel package) pointed me to this other related discussion on distutils-sig (from 2013) where Paul Moore explains what the problem is: https://mail.python.org/pipermail/distutils-sig/2013-February/020022.html Has anything happened since the to address this either in the PEPs and/or the tooling? 2) the current virtualenv module (not the built-in venv) does not copy over the required PYTHON3.DLL and thus importing such extension modules from within the virtual environment fails. The latter issue was already fixed in virtualenv's master branch, but virtualenv package hasn't seen a release since then (the last release 15.10.0 was in November 2016). https://github.com/pypa/virtualenv/pull/1083 I would love to take advantage of CPython limited API feature, and be able to distribute a one/two wheels per platform instead of four (multiply that by two for 32 vs 64 bit arch), and having to rebuild one for each new Python 3.x release. It works fine for Linux and macOS; I hope that Windows python users will also be able to enjoy this soon. I am willing to contribute my time to help solving this issue. Thanks, Cosimo On Mon, Jan 8, 2018 at 1:31 AM Steve Dower <steve.dower(a)python.org> wrote: > There is a solution to that problem on the linked issue. Basically, you > need to declare Py_LIMITED_API in your code or as an extra preprocessor > variable. > > > > Windows doesn’t use a filename suffix for python3.dll linked extensions as > it will be handled at load time. The tag for the wheel is outside of my > area, so hopefully someone can chime in on that for you. > > > > Cheers, > > Steve > > > > Top-posted from my Windows phone > > > > *From: *Cosimo Lupo <cosimo(a)anthrotype.com> > *Sent: *Monday, January 8, 2018 10:32 > *To: *distutils-sig(a)python.org > *Subject: *Re: [Distutils] Should abi3 tag be usable on Windows? > > > > It turns out setuptools is still linking to PYTHON36.DLL instead of > PYTHNO3.DLL even when py_limited_api=True > > https://github.com/pypa/setuptools/issues/1248 > > > > This is a different, though somewhat related, problem from the one > concering wheel/pip pep425tags disallowing `abi3` tag on Windows. > > > > > > On Sun, Jan 7, 2018 at 12:39 PM Cosimo Lupo <cosimo(a)anthrotype.com> wrote: > > I jut found this related pip issue: > https://github.com/pypa/pip/issues/4445 > > > > (I myself as @anthrotype commented on that thread back in April last year > but then completely forgot...) > > > > After re-reading it now, it's still not clear to me what the resolution on > that issue was. > > > > On Sun, Jan 7, 2018 at 10:12 AM Cosimo Lupo <cosimo(a)anthrotype.com> wrote: > > Hello, > > CFFI has recently added support for Py_LIMITED_API extension modules built > for CPython 3. > The wheel module since version 0.30.0 also supports passing > —py-limited-api cp3X to bdist_wheel command to allow the generated .whl to > be installed on all CPython versions equal or greater than the one > specified. > > Yesterday I was trying to apply this on a cffi-built extension module, and > it worked for Linux and macOS but failed for Windows: > > https://github.com/hynek/argon2_cffi/pull/32 > <https://github.com/hynek/argon2_cffi/pull/32#issuecomment-355771832> > The AssertionError from wheel.pep425tags complains that a tag with abi3 > would be unsupported for the target platform. > > Alex Gronholm commented > > > imp.get_suffixes() does not seem to contain any ABI3 suffixes, but I'm > not sure if this is even applicable on Windows. > > Incidentally, I noticed one specific package, PyQt5, that distributes both > abi3-tagged wheels for Mac and manylinux and Windows wheels for a range of > cp35.cp36.cp37 but with abi tag set as “none”, and they do seem to work. > > So, can one make such py_limited_api wheels work on Windows with the > current state of the tooling, and if so how? > > Thank you in advance > > > > Cosimo Lupo > > > > -- > > Cosimo Lupo > > > > -- > > Cosimo Lupo > > > -- Cosimo Lupo

1 0

Allow to debug extension modules
by Ivan Pozdeev Jan. 9, 2018

Jan. 9, 2018

In the https://mail.python.org/pipermail/python-ideas/2018-January/048558.html thread, it was discovered that, stumpingly enough, it seems not possible to debug extension modules! * For a release Python, it's not even possible to build an extension without optimizations * For a debug Python, it's possible to build but not possible to install it, or any other package that depends on it Moreover, the feedback says that it makes no sense to build an extension with _DEBUG against a release Python and vice versa. But distutils' build_ext logic does (or at least trying to do since that breaks) just this, as it looks at the --debug option to `build' instead of checking the type of the running Python. Am I missing something crucial here, or do I need to fix distutils now to be able to debug a problem in my extension module? -- Regards, Ivan

1 1

Add processor generation, Windows version (and possibly other things) to wheel platform tag
by Ivan Pozdeev Jan. 9, 2018

Jan. 9, 2018

This is a follow-up to this thread (the 2nd link is continuation): https://mail.python.org/pipermail/distutils-sig/2017-October/031750.html https://mail.python.org/pipermail/distutils-sig/2017-November/031759.html After discovering https://www.python.org/dev/peps/pep-0425/ , I see that what I was asking for is add more data to the wheel's platform tag. Specifically: * Processor generation if it's higher than what Python officially supports * Windows version if it's higher than what Python officially supports (like it's already done for MacOS) In fact, references to other libraries that the wheel expects on the system that Nick Coghlan proposed in https://www.python.org/dev/peps/pep-0459/ <https://www.python.org/dev/peps/pep-0459/#the-python-constraints-extension> can also be implemented in the platform tag. They belong there with the current infrastructure because: * Anything external to a Python installation that is expected on the system, combined, _is_ the platform. * A wheel's name has to be unique and descriptive (it needs to include everything required for pip to pick the best download). Sure, this can make the name very long, but the more to describe, the longer the description. <https://www.python.org/dev/peps/pep-0459/#the-python-constraints-extension> -- Regards, Ivan

1 0

Should abi3 tag be usable on Windows?
by Cosimo Lupo Jan. 7, 2018

Jan. 7, 2018

Hello, CFFI has recently added support for Py_LIMITED_API extension modules built for CPython 3. The wheel module since version 0.30.0 also supports passing —py-limited-api cp3X to bdist_wheel command to allow the generated .whl to be installed on all CPython versions equal or greater than the one specified. Yesterday I was trying to apply this on a cffi-built extension module, and it worked for Linux and macOS but failed for Windows: https://github.com/hynek/argon2_cffi/pull/32 The AssertionError from wheel.pep425tags complains that a tag with abi3 would be unsupported for the target platform. Alex Gronholm commented > imp.get_suffixes() does not seem to contain any ABI3 suffixes, but I'm not sure if this is even applicable on Windows. Incidentally, I noticed one specific package, PyQt5, that distributes both abi3-tagged wheels for Mac and manylinux and Windows wheels for a range of cp35.cp36.cp37 but with abi tag set as “none”, and they do seem to work. So, can one make such py_limited_api wheels work on Windows with the current state of the tooling, and if so how? Thank you in advance Cosimo Lupo

1 2