Mailman 3 - Distutils-SIG

Trouble combining namespace packages and package_dir option in setup
by Alex Adamson 30 Apr '19

30 Apr '19

Hello, I'm trying to create a package that has a namespace package inside a directory src below the root directory of the project. I was trying to follow the instructions laid out in the documentation here <https://setuptools.readthedocs.io/en/latest/setuptools.html#find-namespace-…>, but could not get the example described there to work using setuptools 41.0.0: [adamson@home] {build-test-37} ~/test-pkg$ tree --charset unicode ..|-- setup.py`-- src `-- namespace `-- mypackage |-- __init__.py `-- mod1.py 3 directories, 3 files [adamson@home] {build-test-37} ~/test-pkg$ cat setup.py from setuptools import setup, find_namespace_packages setup(name="namespace.mypackage", version="0.1", package_dir={'': 'src'}, packages=find_namespace_packages(where='src')) [adamson@home] {build-test-37} ~/test-pkg$ conda list -e# This file may be used to create an environment using:# $ conda create --name <env> --file <this file># platform: linux-64 ca-certificates=2019.1.23=0 certifi=2019.3.9=py37_0 libedit=3.1.20181209=hc058e9b_0 libffi=3.2.1=hd88cf55_4 libgcc-ng=8.2.0=hdf63c60_1 libstdcxx-ng=8.2.0=hdf63c60_1 ncurses=6.1=he6710b0_1 openssl=1.0.2r=h7b6447c_0 pip=19.1=py37_0 python=3.7.0=h6e4f718_3 readline=7.0=h7b6447c_5 setuptools=41.0.0=py37_0 sqlite=3.28.0=h7b6447c_0 tk=8.6.8=hbc83047_0 wheel=0.33.1=py37_0 xz=5.2.4=h14c3975_4 zlib=1.2.11=h7b6447c_3 [adamson@home] {build-test-37} ~/test-pkg$ python setup.py install --record files.txt running install running bdist_egg running egg_info creating src/namespace.mypackage.egg-info writing src/namespace.mypackage.egg-info/PKG-INFO writing dependency_links to src/namespace.mypackage.egg-info/dependency_links.txt writing top-level names to src/namespace.mypackage.egg-info/top_level.txt writing manifest file 'src/namespace.mypackage.egg-info/SOURCES.txt' package init file 'src/namespace/__init__.py' not found (or not a regular file) reading manifest file 'src/namespace.mypackage.egg-info/SOURCES.txt' writing manifest file 'src/namespace.mypackage.egg-info/SOURCES.txt' installing library code to build/bdist.linux-x86_64/egg running install_lib running build_py creating build creating build/lib creating build/lib/namespace creating build/lib/namespace/mypackage copying src/namespace/mypackage/mod1.py -> build/lib/namespace/mypackage copying src/namespace/mypackage/__init__.py -> build/lib/namespace/mypackage creating build/bdist.linux-x86_64 creating build/bdist.linux-x86_64/egg creating build/bdist.linux-x86_64/egg/namespace creating build/bdist.linux-x86_64/egg/namespace/mypackage copying build/lib/namespace/mypackage/mod1.py -> build/bdist.linux-x86_64/egg/namespace/mypackage copying build/lib/namespace/mypackage/__init__.py -> build/bdist.linux-x86_64/egg/namespace/mypackage byte-compiling build/bdist.linux-x86_64/egg/namespace/mypackage/mod1.py to mod1.cpython-37.pyc byte-compiling build/bdist.linux-x86_64/egg/namespace/mypackage/__init__.py to __init__.cpython-37.pyc creating build/bdist.linux-x86_64/egg/EGG-INFO copying src/namespace.mypackage.egg-info/PKG-INFO -> build/bdist.linux-x86_64/egg/EGG-INFO copying src/namespace.mypackage.egg-info/SOURCES.txt -> build/bdist.linux-x86_64/egg/EGG-INFO copying src/namespace.mypackage.egg-info/dependency_links.txt -> build/bdist.linux-x86_64/egg/EGG-INFO copying src/namespace.mypackage.egg-info/top_level.txt -> build/bdist.linux-x86_64/egg/EGG-INFO zip_safe flag not set; analyzing archive contents... creating dist creating 'dist/namespace.mypackage-0.1-py3.7.egg' and adding 'build/bdist.linux-x86_64/egg' to it removing 'build/bdist.linux-x86_64/egg' (and everything under it) Processing namespace.mypackage-0.1-py3.7.egg Copying namespace.mypackage-0.1-py3.7.egg to /nas/dft/ire/adamson/.conda/envs/build-test-37/lib/python3.7/site-packages Adding namespace.mypackage 0.1 to easy-install.pth file Installed /nas/dft/ire/adamson/.conda/envs/build-test-37/lib/python3.7/site-packages/namespace.mypackage-0.1-py3.7.egg Processing dependencies for namespace.mypackage==0.1 Finished processing dependencies for namespace.mypackage==0.1 writing list of installed files to 'files.txt' [adamson@home] {build-test-37} ~/test-pkg$ python -c 'import namespace.mypackage' Traceback (most recent call last): File "<string>", line 1, in <module> ModuleNotFoundError: No module named 'namespace' Is there a known issue preventing package_dir to work with namespace packages? Is there a known good version of setuptools that I could use to package/install a project using a namespace package and package_dir? Thank you! - Alex Adamson

1 0

Quiet IRC channel?
by Derek Kozel 26 Apr '19

26 Apr '19

Hello, I've been on the Distutils Freenode IRC channel listed on the Python.org website for a bit over two weeks now and several people have come by asking questions but it seems that the channel is disused as there's been no response from anyone. Is that true? If so, could someone change the channel message and maybe note that on the Python website? https://www.python.org/community/irc/ I had joined to ask a question about using distutils to install a module on Ubuntu but having it go to site-packages rather than the OS specific dist-packages. I don't think this list is a good place to ask a question like that (but code and log below if anyone can help). Can you point me at a better place to ask questions? Code: https://github.com/dkozel/linux-gpib_debian/tree/upstream/linux-gpib-user/l… Log: https://gist.github.com/dkozel/1c1ff3357ea71bf21c69e8beeaacfa78 Thanks, Derek

2 1

PyCon US Packaging Mini Summit
by Pradyun Gedam 25 Apr '19

25 Apr '19

A lot of people interested in Python Packaging will be in one place, at PyCon US this year, and we're planning to utilize this opportunity to discuss various topics and make progress on them in a Packaging Mini-Summit. If you'll be at PyCon US 2019 and want to take part in this, please provide some details about your PyCon attendance at [1]. The exact structure and flow of the Mini-Summit are still under discussion at [2]. Though not finalized, it will likely include (at least) one open space and one meeting, likely during the sprints. Suggestions for topics to be discussed at the Mini-Summit are being collected at [3], one per post. If you have any suggestions for topics that should be discussed, please post them there. [1]: https://docs.google.com/spreadsheets/d/1DOcLSW2BlH-rebmPNnGIaOpfcEzKVFeu_Bh… [2]: https://discuss.python.org/t/pycon-us-packaging-mini-summit-2019/833 [3]: https://discuss.python.org/t/packaging-mini-summit-pycon-us-2019-topic-sugg… Pradyun

1 0

Draft PEP: Recording Source URL of installed distributions
by Stéphane Bidoul (ACSONE) 23 Apr '19

23 Apr '19

Hi, I'm posting here to raise awareness about a new Draft packaging interoperability PEP I am proposing [1]. This document originated from the need to implement pip issue #609 [2], ie improving the behavior of pip freeze in presence of distributions installed from direct URL references. In a thread [3] on discuss.python.org it was suggested a new PEP was needed to standardize the recording of the required information in .dist-info. After investigation, I found out PEP426 (now withdrawn) had a concept of source_url metadata, which almost satisfied the requirement. [1] is therefore a revival of source_url, with a different storage format to allow for additional use cases. Looking forward to reading your comments. Best regards, -sbi [1] https://github.com/python/peps/pull/1009 [2] https://github.com/pypa/pip/issues/609 [3] https://discuss.python.org/t/pip-freeze-vcs-urls-and-pep-517-feat-editable-…

2 1

PEP 376 & pkgutil - Quick question
by Eric Peterson 16 Apr '19

16 Apr '19

Hi all, PEP 376 mentions several new functions that were anticipated to be implemented in pkgutil to read from the dist-info directories that are created when a wheel is installed: https://www.python.org/dev/peps/pep-0376/#new-functions-and-classes-in-pkgu… But, I don't see them implemented at least as of Python 3.7: https://docs.python.org/3.7/library/pkgutil.html Are those functions implemented somewhere, perhaps outside the standard library? --Eric

3 3

psycopg2/psycopg2-binary
by Bert JW Regeer 08 Apr '19

08 Apr '19

Hey all, You may have seen some hub hub around psycopg2 and no longer shipping binary wheels by default [1][2] (and in fact using psycopg2-binary if you want wheels), and I wanted to bring it up here because it demonstrates a problem area with the current state of packaging in Python: There is no good way for a new package to specify that it provides what another package would provide, and setuptools currently checks that all distributions are found before running the console scripts (so if a console script has a setup.py that depends on psycopg2 and you install psycopg2-binary it fails) [3] So currently if you pip install psycopg2-binary and then install a project that uses psycopg2 as a dependency it will install psycopg2 over top of psycopg2-binary. The author of psycopg2 stopped distributing binaries for psycopg2 because of issues with the version of SSL that Python was compiled for/what was used by psycopg2 and it causing all kinds of issues. I don't have a proposal or a fix, but this is going to be an issue not just for psycopg2 but also for other projects that potentially distribute wheels that are built against a different version of OpenSSL. I see two things that should get some thought: 1. How to have a package provide for another package (there are keywords but AFAIK they are currently ignored by pip) 2. How to handle/deal with shared libraries that are not versioned Thanks, Bert [1]: https://github.com/psycopg/psycopg2/issues/674 <https://github.com/psycopg/psycopg2/issues/674> [2]: https://github.com/psycopg/psycopg2/issues/883 <https://github.com/psycopg/psycopg2/issues/883> [3]: https://github.com/zalando/patroni/issues/1021#issuecomment-480202590 <https://github.com/zalando/patroni/issues/1021#issuecomment-480202590>

4 4

Fwd: BitBounc BS Was: psycopg2/psycopg2-binary
by Bert JW Regeer 07 Apr '19

07 Apr '19

Is there a contact for the list owner so that people like this can get removed? > Begin forwarded message: > > From: eysz7x(a)python-org.link > Subject: Re: [Distutils] psycopg2/psycopg2-binary > Date: April 6, 2019 at 15:34:29 MDT > To: xistence(a)0x58.com > > > > > Thanks for emailing me! No, I haven’t been hacked :) > > I signed up for a spam filtering service called BitBounce. To deliver your email to my inbox, please click the link below and pay the small Bitcoin fee. Thanks! > > $0.05 to deliver your email. > > We’ve never met — I’ll pay your fee. > > <https://bitbounce.com/pay_bitbounce_fee/3742583551> > I know you — Add me to your whitelist. > > <https://bitbounce.com/whitelist/request_access/3014729?email_address=xisten…> > > > BitBounce <https://bitbounce.com/?ref=bitbounce> is powered by > the Credo <https://bitbounce.com/credo> cryptocurrency > > I’m from a business —  > what are my delivery options <https://business.bitbounce.com/> > BitBounce and Credo are > transacted through CredoEx <https://credoex.com/?ref=bitbounce> > Made by Turing Technology Inc. in San Mateo, California Sign Up for BitBounce <https://bitbounce.com/?ref=bitbounce>

3 2

PEP 440: restriction of numeric version components to ASCII digits made explicit
by Nick Coghlan 07 Apr '19

07 Apr '19

Hi folks, Frazer McLean noticed that even though the version parsing regex in PEP 440 Appendix B was explicitly restricted to ASCII digits, the `is_canonical` regex was more permissive and nominally allowed arbitrary Unicode [Nd] characters. The restrictive interpretation was the correct one (and reflects what tools actually support), so I've merged Frazer's change at https://github.com/python/peps/commit/1dd991e9001ffea6b41e9a2cd3a6549dd984b… to fix the omission and update the `is_canonical` regex to be consistent with the version parsing one. Cheers, Nick. -- Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia

1 0

Fwd: Re: psycopg2/psycopg2-binary
by Nathaniel Smith 07 Apr '19

07 Apr '19

Can a mod please disable this person's subscription to distutils-sig...? ---------- Forwarded message --------- From: <eysz7x(a)python-org.link> Date: Sat, Apr 6, 2019 at 3:16 PM Subject: Re: [Distutils] Re: psycopg2/psycopg2-binary To: <njs(a)pobox.com> [image: BitBounce] Thanks for emailing me! No, I haven’t been hacked :) I signed up for a spam filtering service called BitBounce. To deliver your email to my inbox, please click the link below and pay the small Bitcoin fee. Thanks! *$0.05* to deliver your email. We’ve never met — I’ll pay your fee. <https://bitbounce.com/pay_bitbounce_fee/1052592896> I know you — Add me to your whitelist. <https://bitbounce.com/whitelist/request_access/3014729?email_address=njs@po…> *BitBounce* <https://bitbounce.com?ref=bitbounce> is powered by the *Credo* <https://bitbounce.com/credo> cryptocurrency *I’m from a business* — what are my *delivery options* <https://business.bitbounce.com> BitBounce and Credo are transacted through *CredoEx* <https://credoex.com?ref=bitbounce> Made by Turing Technology Inc. in San Mateo, California *Sign Up for BitBounce* <https://bitbounce.com?ref=bitbounce> -- Nathaniel J. Smith -- https://vorpus.org <http://vorpus.org>

2 1

Idea: perennial manylinux tag
by Nathaniel Smith 22 Mar '19

22 Mar '19

Hi all, The manylinux1 -> manylinux2010 transition has turned out to be very difficult. Timeline so far: March 2017: CentOS 5 went EOL April 2018: PEP 517 accepted May 2018: support for manylinux2010 lands in warehouse November 2018: support lands in auditwheel, and pip master December 2018: 21 months after CentOS 5 EOL, wwee still don't have an official build environment, or support in a pip release We'll get through this, but it's been super painful and maybe we can change things somehow so it will suck less next time. We don't have anything like this pain on Windows or macOS. We never have to update pip, warehouse, etc., after those OSes hit EOLs. Why not? On Windows, we have just two tags: "win32" and "win_amd64". These are defined to mean something like "this wheel will run on any recent-ish Windows system". So the meaning of the tag actually changes over time: it used to be that if a wheel said it ran on win32, then that meant it would work on winxp, but since winxp hit EOL people started uploading "win32" wheels that don't work on winxp, and that's worked fine. On macOS, the tags look like "macosx_10_9_x86_64". So here we have the OS version embedded in the tag. This means that we do occasionally switch which tags we're using, kind of like how manylinux1 -> manylinux2010 is intended to work. But, unlike for the manylinux tags, defining a new macosx tag is totally trivial: every time a new OS version is released, the tag springs into existence without any human intervention. Warehouse already accepts uploads with this tag; pip already knows which systems can install wheels with this tag, etc. Can we take any inspiration from this for manylinux? We could do the Windows thing, and have a plain "manylinux" tag that means "any recent-ish glibc-based Linux". Today it would be defined to be "any distro newer than CentOS 6". When CentOS 6 goes out of service, we could tweak the definition to be "any distro newer than CentOS 7". Most parts of the toolchain wouldn't need to be updated, though, because the tag wouldn't change, and by assumption, enforcement wouldn't really be needed, because the only people who could break would be ones running on unsupported platforms. Just like happens on Windows. We could do the macOS thing, and have a "manylinux_${glibc version}" tag that means "this package works on any Linux using glibc newer than ${glibc version}". We're already using this as our heuristic to handle the current manylinux profiles, so e.g. manylinux1 is effectively equivalent to manylinux_2_5, and manylinux2010 will be equivalent to manylinux_2_12. That way we'd define the manylinux tags once, get support into pip and warehouse and auditwheel once, and then in the future the only thing that would have to change to support new distro releases or new architectures would be to set up a proper build environment. What do y'all think? -n

14 26