Mailman 3 - Distutils-SIG

Not at the sprints -- editable installs?
by Bert JW Regeer 08 May '19

08 May '19

Hey all, I am not at the sprints and I am not sure where I can follow the discussion and or see what people are coming up with. Is there any feedback or information on editable installs using PEP517/PEP518 projects? Bert

1 0

Re: Adding namespace support to PyPi (continuation from PyPA Summit/Sprint)
by Lukas Puehringer 08 May '19

08 May '19

> Will the TUF implementation need any changes to support namespaces? > In the minimum security model (PEP 458), namespaces should not affect TUF integration at all, since all target metadata (i.e. metadata about packages uploaded to PyPI) are signed with keys owned by PyPI. In the maximum security model (PEP 480), PyPI delegates trust about packages to the packagers, i.e. the packagers sign TUF target metadata with their keys and upload it to PyPI along with the corresponding packages. The delegation is then **verified using package names**. So in the latter case, namespace ownership and delegation management are indeed related, as in, both deal with package name prefixes. However, I think this is more an organizational matter than something that needs implementation changes. Besides, from what I gather from the "Namespace support in pypi" discussion [1], it's not really clear yet, what namespace support actually means. [1] https://discuss.python.org/t/namespace-support-in-pypi/1609/17, -- lukas.puehringer(a)nyu.edu PGP fingerprint: 8BA6 9B87 D43B E294 F23E 8120 89A2 AD3C 07D9 62E8

1 0

Docker builds, ~/.pip/cache, and O(1) or O(n) bandwidth
by Wes Turner 07 May '19

07 May '19

What is the best way to build docker images without constantly re-downloading packages from PyPI (to use ~O(1) bandwidth instead of O(n) for every build) (AFAIK, nobody has any issue with the amount of bandwidth PyPI uses) Thus far, Docker doesn't want to support a build-time -v option (that could be used to bind-mount .pip/cache in at build time): "build time only -v option" https://github.com/moby/moby/issues/14080 Buildah *does* support a build-time -v option (and can also do rootless builds without a docker socket)

4 6

API for Adding Contributors to a Package?
by Philip James 07 May '19

07 May '19

Hi there! I took a look at the API docs that I could find ( https://warehouse.readthedocs.io/api-reference/) and I couldn't see if it was possible to programmatically edit the contributors who have access to a project on PyPI. My use case is for the BeeWare project being able to dynamically add and remove permissions to upload new versions of packages. Does this API exist? I'm also at PyCon until Thursday if someone wants to talk about my use case directly and is also here. Cheers, Philip

2 2

Adding namespace support to PyPi (continuation from PyPA Summit/Sprint)
by Dave Ashby 07 May '19

07 May '19

Good evening distutils-sig- TL;DR: today during the Python Packaging Summit/Sprint I broached the topic of introducing namespaces on PyPi with several of the luminaries in the PyPA ecosystem. After much discussion I think we concluded that there's general support for the idea, but quite a number of devils in the details that would need worked out. I've tried to capture much of the relevant discussion on in the associated thread on discuss.python.org <https://discuss.python.org/t/namespace-support-in-pypi/1609> (including a first-pass at a draft PEP <https://gist.github.com/vadave/f1f2d07f5e355c6263fc111aae634ea5> ) Please join in the conversation about whether/how namespaces should be implemented in PyPi. Detail: The namespaces discussion had actually come up before the transition to warehouse[1], but at the time it was deferred due to the effort associated with that migration. Now that Warehouse is in a better place, it seems like it's time to reinvigorate this discussion. Most other programming languages have some type of namespace construct that can be used to help better manage the supply chain integrity. As I see it namespaces have the potential to provide significant value to *both* package maintainers and package consumers. For package maintainers it potentially offers a higher-level of control over packages they are responsible for maintaining. For package consumers it potentially offers a cleaner mechanism for "at a glance" recognition of the source of a particular package (e.g. whether the package is vendor-developed vs. community-developed or has a formal relationship with some larger group like PyData). With all that said, my personal goal is for this PEP to serve kind of a foundational role. I *don't want* to try to exhaustively define all the different scenarios that namespaces can/should be used to enable. I *do want* namespaces to become a clear security-boundary that can be used in "high compliance" environments to make security-related risk decisions. And lastly, I *definitely don't want* to replace the current global namespace...I view this as a complement, not a replacement to the current situation. Finally, as this is my first post to this list I'll offer the obligatory "new guy" introduction. I started using Python about 4 years ago, so I still consider myself a relative newbie. For my day job I'm an independent consultant working with what I call "high compliance" customers...folks that really, really care about security and have lots of process baked around how they manage security risk within their organization. I'm based in Fairfax, VA. Most of what I do from day-to-day *is not* writing python code, but rather working issues around IT architecture and governance (with a general focus around enabling infrastructure such as cloud services and devops tooling). Thanks to all the folks that offered their time and opinions on this topic today. I look forward to continued dialogue on this, and welcome any ideas or cautions folks may have about making this a reality. And remember: "Namespaces are one honking great idea - let's do more of those!" -d [1] Original feature request for adding namepsaces support: https://github.com/pypa/warehouse/issues/2589

2 2

Adding namespace support to PyPi (continuation from PyPA Summit/Sprint)
by Dave Ashby 06 May '19

06 May '19

Good evening distutils-sig- TL;DR: today during the Python Packaging Summit/Sprint I broached the topic of introducing namespaces on PyPi with several of the luminaries in the PyPA ecosystem. After much discussion I think we concluded that there's general support for the idea, but quite a number of devils in the details that would need worked out. I've tried to capture much of the relevant discussion on in the associated thread on discuss.python.org <https://discuss.python.org/t/namespace-support-in-pypi/1609> (including a first-pass at a draft PEP <https://gist.github.com/vadave/f1f2d07f5e355c6263fc111aae634ea5> ) Please join in the conversation about whether/how namespaces should be implemented in PyPi. Detail: The namespaces discussion had actually come up before the transition to warehouse[1], but at the time it was deferred due to the effort associated with that migration. Now that Warehouse is in a better place, it seems like it's time to reinvigorate this discussion. Most other programming languages have some type of namespace construct that can be used to help better manage the supply chain integrity. As I see it namespaces have the potential to provide significant value to *both* package maintainers and package consumers. For package maintainers it potentially offers a higher-level of control over packages they are responsible for maintaining. For package consumers it potentially offers a cleaner mechanism for "at a glance" recognition of the source of a particular package (e.g. whether the package is vendor-developed vs. community-developed or has a formal relationship with some larger group like PyData). With all that said, my personal goal is for this PEP to serve kind of a foundational role. I *don't want* to try to exhaustively define all the different scenarios that namespaces can/should be used to enable. I *do want* namespaces to become a clear security-boundary that can be used in "high compliance" environments to make security-related risk decisions. And lastly, I *definitely don't want* to replace the current global namespace...I view this as a complement, not a replacement to the current situation. Finally, as this is my first post to this list I'll offer the obligatory "new guy" introduction. I started using Python about 4 years ago, so I still consider myself a relative newbie. For my day job I'm an independent consultant working with what I call "high compliance" customers...folks that really, really care about security and have lots of process baked around how they manage security risk within their organization. I'm based in Fairfax, VA. Most of what I do from day-to-day *is not* writing python code, but rather working issues around IT architecture and governance (with a general focus around enabling infrastructure such as cloud services and devops tooling). Thanks to all the folks that offered their time and opinions on this topic today. I look forward to continued dialogue on this, and welcome any ideas or cautions folks may have about making this a reality. And remember: "Namespaces are one honking great idea - let's do more of those!" -d [1] Original feature request for adding namepsaces support: https://github.com/pypa/warehouse/issues/2589

1 0

Sprints have started at PyCon NA 2019
by Sumana Harihareswara 06 May '19

06 May '19

https://wiki.python.org/psf/PackagingSprints Sprints have started and Packaging is in room 26C. We're starting a shared editable document of what people are working on at https://docs.google.com/document/d/1Wz2-ECkicJgAmQDxMFivWmU2ZunKvPZ2UfQ59zD… Shortlink: http://bit.ly/pypa2019 (Thanks Chris Wilcox for setting that up!) -- Sumana Harihareswara Changeset Consulting https://changeset.nyc

1 0

Trouble combining namespace packages and package_dir option in setup
by Alex Adamson 02 May '19

02 May '19

Hello, I'm trying to create a package that has a namespace package inside a directory src below the root directory of the project. I was trying to follow the instructions laid out in the documentation here <https://setuptools.readthedocs.io/en/latest/setuptools.html#find-namespace-…>, but could not get the example described there to work using setuptools 41.0.0: [adamson@home] {build-test-37} ~/test-pkg$ tree --charset unicode ..|-- setup.py`-- src `-- namespace `-- mypackage |-- __init__.py `-- mod1.py 3 directories, 3 files [adamson@home] {build-test-37} ~/test-pkg$ cat setup.py from setuptools import setup, find_namespace_packages setup(name="namespace.mypackage", version="0.1", package_dir={'': 'src'}, packages=find_namespace_packages(where='src')) [adamson@home] {build-test-37} ~/test-pkg$ conda list -e# This file may be used to create an environment using:# $ conda create --name <env> --file <this file># platform: linux-64 ca-certificates=2019.1.23=0 certifi=2019.3.9=py37_0 libedit=3.1.20181209=hc058e9b_0 libffi=3.2.1=hd88cf55_4 libgcc-ng=8.2.0=hdf63c60_1 libstdcxx-ng=8.2.0=hdf63c60_1 ncurses=6.1=he6710b0_1 openssl=1.0.2r=h7b6447c_0 pip=19.1=py37_0 python=3.7.0=h6e4f718_3 readline=7.0=h7b6447c_5 setuptools=41.0.0=py37_0 sqlite=3.28.0=h7b6447c_0 tk=8.6.8=hbc83047_0 wheel=0.33.1=py37_0 xz=5.2.4=h14c3975_4 zlib=1.2.11=h7b6447c_3 [adamson@home] {build-test-37} ~/test-pkg$ python setup.py install --record files.txt running install running bdist_egg running egg_info creating src/namespace.mypackage.egg-info writing src/namespace.mypackage.egg-info/PKG-INFO writing dependency_links to src/namespace.mypackage.egg-info/dependency_links.txt writing top-level names to src/namespace.mypackage.egg-info/top_level.txt writing manifest file 'src/namespace.mypackage.egg-info/SOURCES.txt' package init file 'src/namespace/__init__.py' not found (or not a regular file) reading manifest file 'src/namespace.mypackage.egg-info/SOURCES.txt' writing manifest file 'src/namespace.mypackage.egg-info/SOURCES.txt' installing library code to build/bdist.linux-x86_64/egg running install_lib running build_py creating build creating build/lib creating build/lib/namespace creating build/lib/namespace/mypackage copying src/namespace/mypackage/mod1.py -> build/lib/namespace/mypackage copying src/namespace/mypackage/__init__.py -> build/lib/namespace/mypackage creating build/bdist.linux-x86_64 creating build/bdist.linux-x86_64/egg creating build/bdist.linux-x86_64/egg/namespace creating build/bdist.linux-x86_64/egg/namespace/mypackage copying build/lib/namespace/mypackage/mod1.py -> build/bdist.linux-x86_64/egg/namespace/mypackage copying build/lib/namespace/mypackage/__init__.py -> build/bdist.linux-x86_64/egg/namespace/mypackage byte-compiling build/bdist.linux-x86_64/egg/namespace/mypackage/mod1.py to mod1.cpython-37.pyc byte-compiling build/bdist.linux-x86_64/egg/namespace/mypackage/__init__.py to __init__.cpython-37.pyc creating build/bdist.linux-x86_64/egg/EGG-INFO copying src/namespace.mypackage.egg-info/PKG-INFO -> build/bdist.linux-x86_64/egg/EGG-INFO copying src/namespace.mypackage.egg-info/SOURCES.txt -> build/bdist.linux-x86_64/egg/EGG-INFO copying src/namespace.mypackage.egg-info/dependency_links.txt -> build/bdist.linux-x86_64/egg/EGG-INFO copying src/namespace.mypackage.egg-info/top_level.txt -> build/bdist.linux-x86_64/egg/EGG-INFO zip_safe flag not set; analyzing archive contents... creating dist creating 'dist/namespace.mypackage-0.1-py3.7.egg' and adding 'build/bdist.linux-x86_64/egg' to it removing 'build/bdist.linux-x86_64/egg' (and everything under it) Processing namespace.mypackage-0.1-py3.7.egg Copying namespace.mypackage-0.1-py3.7.egg to /nas/dft/ire/adamson/.conda/envs/build-test-37/lib/python3.7/site-packages Adding namespace.mypackage 0.1 to easy-install.pth file Installed /nas/dft/ire/adamson/.conda/envs/build-test-37/lib/python3.7/site-packages/namespace.mypackage-0.1-py3.7.egg Processing dependencies for namespace.mypackage==0.1 Finished processing dependencies for namespace.mypackage==0.1 writing list of installed files to 'files.txt' [adamson@home] {build-test-37} ~/test-pkg$ python -c 'import namespace.mypackage' Traceback (most recent call last): File "<string>", line 1, in <module> ModuleNotFoundError: No module named 'namespace' Is there a known issue preventing package_dir to work with namespace packages? Is there a known good version of setuptools that I could use to package/install a project using a namespace package and package_dir? Thank you! - Alex

2 2

Re: [pypi-announce] PyPI two-factor auth (2FA) trial May 3-20
by Cooper Lees 02 May '19

02 May '19

Hi, What does 3-20 May mean in context to "the private beta" date + time. Thanks, Cooper > On May 2, 2019, at 10:58 AM, Sumana Harihareswara <sh(a)changeset.nyc> wrote: > > Dear PyPI users: > > To increase the security of PyPI downloads, we're beginning to introduce two-factor authentication (2FA) as a login security option, and want project maintainers and owners to start testing it. > > Starting this Friday, May 3rd, you'll be able to use 2FA on Test PyPI http://test.pypi.org/ . And if you'd like to try 2FA on official PyPI https://pypi.org , please fill out this Google form https://docs.google.com/forms/d/e/1FAIpQLSfRmXhkfAL-LgLfcMdzTG7iIaSwPo-pyzk… so we can invite you to the private beta, which we plan to hold 3-20 May. > > More details at https://wiki.python.org/psf/WarehousePackageMaintainerTesting . > > We expect to end this testing period on May 20th, then enable the optional 2FA feature for all PyPI users, and move on to working on WebAuthn support. > > Thanks to the Open Technology Fund for funding this work. More progress reports at https://wiki.python.org/psf/PackagingWG . > > -Sumana Harihareswara on behalf of the PyPI team > _______________________________________________ > pypi-announce mailing list -- pypi-announce(a)python.org > To unsubscribe send an email to pypi-announce-leave(a)python.org > https://mail.python.org/mailman3/lists/pypi-announce.python.org/

1 0

PyPI two-factor auth (2FA) trial May 3-20
by Sumana Harihareswara 02 May '19

02 May '19

PyPI users: To increase the security of PyPI downloads, we're beginning to introduce two-factor authentication (2FA) as a login security option, and want project maintainers and owners to start testing it. Starting this Friday, May 3rd, you'll be able to use 2FA on [Test PyPI](http://test.pypi.org/). And if you'd like to try 2FA on [official PyPI](https://pypi.org), please fill out [this Google form](https://docs.google.com/forms/d/e/1FAIpQLSfRmXhkfAL-LgLfcMdzTG7iIaSwP… so we can invite you to the private beta, which we plan to hold 3-20 May. PyPI currently supports a single 2FA method: generating a code through a Time-based One-time Password (TOTP) application. After you set up 2FA on your PyPI account, then you must provide a TOTP (along with your username and password) to log in. Therefore, to use 2FA on PyPI, you'll need to provision an application (usually a mobile phone app) in order to generate authentication codes; our [our testing wiki page](https://wiki.python.org/psf/WarehousePackageMaintainerTesting) gives you suggestions and pointers. This change only applies to the login step, not package uploads. More details at [our testing wiki page](https://wiki.python.org/psf/WarehousePackageMaintainerTesting). During this testing period, if things go awry, there's a chance we will need to wipe tokens from users' accounts, so if you choose to try it, please be forewarned. We strongly suggest you make sure you have a PyPI-verified email address on your user account before trying the feature, to make potential account recovery smoother. And please [let us know](https://github.com/pypa/warehouse/issues/new) if you run into glitches. We expect to end this testing period on May 20th, then enable the optional 2FA feature for all PyPI users, and move on to working on WebAuthn support. Thanks to the Open Technology Fund for funding this work. More progress reports at [the Packaging Working Group's wiki page](https://wiki.python.org/psf/PackagingWG). -Sumana on behalf of the PyPI team (cross-posted to https://discuss.python.org/t/pypi-two-factor-auth-2fa-trial-may-3-20/1590 ) -- Sumana Harihareswara Changeset Consulting https://changeset.nyc

2 1