Mailman 3 March 2018 - Distutils-SIG

IRC/Twitter livechat hours March 27-April 5
by Sumana Harihareswara 05 Apr '18

05 Apr '18

Warehouse developers will be in IRC, in #pypa-dev on Freenode, and on Twitter (hashtag: #newpypi), available to talk about problems you run into, or about how to hack on Warehouse, for four livechats over the next few weeks: 1. Tuesday, March 27th, 9am-10am PDT, noon-1pm EDT, 18:00-19:00 CEST, 9:30pm-10:30pm India, 16:00-17:00 UTC https://www.timeanddate.com/worldclock/fixedtime.html?msg=Warehouse/PyPI+be… 2. Friday, March 30th, 10-11am EDT, 16:00-17:00 CEST, 7:30pm-8:30pm India, 14:00-15:00 UTC https://www.timeanddate.com/worldclock/fixedtime.html?msg=Warehouse/PyPI+be… 3. Tuesday, April 3rd, 8am-9am PDT, 11am-noon EDT, 17:00-18:00 CEST, 8:30pm-9:30pm India, 15:00-16:00 UTC https://www.timeanddate.com/worldclock/fixedtime.html?msg=Warehouse/PyPI+be… 4. Thursday, April 5th, 5pm-6pm PDT, 8pm-9pm EDT, (April 5th) 8am-9am Manila, (April 5th) 10am-11am Melbourne, (April 5th) 0:00-1:00 UTC https://www.timeanddate.com/worldclock/fixedtime.html?p1=24&iso=20180405T19… Feel free to drop in! (By participating, you agree to abide by the PyPA Code of Conduct: https://www.pypa.io/en/latest/code-of-conduct/ .) -- Sumana Harihareswara Changeset Consulting https://changeset.nyc

1 2

Beta release of pip version 10
by Paul Moore 31 Mar '18

31 Mar '18

On behalf of the PyPA, I am pleased to announce that a beta release 10.0.0b1 of pip has just been released for testing by the community. We're planning on a final release in 2 weeks' time, over the weekend of 14/15 April. To install pip 10.0.0.b1, you can run python -m pip install --upgrade --pre pip (obviously, you should not do this in a production environment!) We would be grateful for all testing that users could do, to ensure that when pip 10 is released it's as solid as we can make it. Highlights of the new release: * Python 2.6 is no longer supported - if you need pip on Python 2.6, you should stay on pip 9, which is the last version to support Python 2.6. * Support for PEP 518, which allows projects to specify what packages they require in order to build from source. (PEP 518 support is currently limited, with full support coming in future versions - see the documentation for details). * Significant improvements in Unicode handling for non-ASCII locales on Windows. * A new "pip config" command. * The default upgrade strategy has become "only-if-needed" * Many bug fixes and minor improvements. In addition, the previously announced reorganisation of pip's internals has now taken place. Unless you are the author of code that imports the pip module (or a user of such code), this change will not affect you. If you are, please report the issue to the author of the affected code (refer them to https://mail.python.org/pipermail/distutils-sig/2017-October/031642.html for the details of the announcement). Please note that there is a minor issue with the NEWS file for this release - the new features in 10.0.0b1 are reported as being for "9.0.3 (2018-03-31)". If you discover any bugs while testing the new release, please report them at https://github.com/pypa/pip/issues. Thanks to everyone who put so much effort into the new release. Many of the contributions came from community members, whether in the form of code, participation in design discussions, or bug reports. The pip development team is extremely grateful to everyone in the community for their contributions. Thanks, Paul

1 0

Need how to install 3rd party python modules
by ?? ?? 29 Mar '18

29 Mar '18

I would like to install openpyxl, and I installed Python shown as follows: Python 3.6.4 (v3.6.4:d48eceb, Dec 19 2017, 06:04:45) [MSC v.1900 32 bit (Intel)] on win32 But as the following situation shows, I cannot succeed installing openpyxl. >>> pip install openpyxl File "<stdin>", line 1 pip install openpyxl ^ SyntaxError: invalid syntax How can I succeed installing openpyxl ? Best Regards, --------------------- Naoki Morihira TEL: 01181-90-6460-6265 ---------------------

5 7

PyPI/Warehouse (short) weekly report: beta, pythonhosted docs, PEP 541
by Sumana Harihareswara 29 Mar '18

29 Mar '18

(A short report this week that doesn't include as much detail about merged PRs, infrastructure work, and so on, because I've been doing stuff like writing to python-list/comp.lang.python to ask for testing[1], but next week's will have more.) The new PyPI went to beta this week![2] Now that PEP 541 is accepted, we are deciding on logistical stuff to implement it and help deal with name transfer requests[3]. Cool things from the notes from the weekly Warehouse core developers' meeting[4] include: if your PyPI project has docs already uploaded to pythonhosted.org, you can now delete those docs[5]. (Further work on docs transition is forthcoming.) And this week and next week some of our team are busy with other projects, so code review and issue resolution will be a little slower. We have livechats coming up in Twitter & IRC[6] (next one is Friday the 30th), and tomorrow I hope to speak on the OpenNews community call[7] at noon Eastern Time. You can help this week by spreading the word[8] about the new PyPI beta and testing it, by opining in our "needs discussion" issues[9], and by just being your wonderful selves. Thank you for your help and kind words through this whole process. It means a lot. More soon! -- Sumana Harihareswara Warehouse project manager Changeset Consulting sh(a)changeset.nyc Links: 1. https://mail.python.org/pipermail/python-list/2018-March/thread.html#732228 2. https://pyfound.blogspot.com/2018/03/warehouse-all-new-pypi-is-now-in-beta.… 3. https://github.com/pypa/warehouse/issues/1506#issuecomment-374626455 4. https://wiki.python.org/psf/PackagingWG/2018-03-26-Warehouse 5. https://github.com/pypa/warehouse/pull/3413 6. https://pyfound.blogspot.com/2018/03/warehouse-all-new-pypi-is-now-in-beta.… 7. https://opennews.org/what/community/calls/ 8. https://pyfound.blogspot.com/2018/03/warehouse-all-new-pypi-is-now-in-beta.… 9. https://github.com/pypa/warehouse/labels/needs%20discussion

1 0

Warehouse is in beta
by Sumana Harihareswara 26 Mar '18

26 Mar '18

As of today, the new Python Package Index at pypi.org, powered by Warehouse, is in beta. What does that mean? >From https://pypi.org/help/#beta : The site is robust, but not *fully* tested and 'production ready'. This means: * The UI may return unusual or erroneous results * Performance may not yet be optimised * Some features that you used in the old interface (pypi.python.org) may be missing * We need your feedback to get the site production ready While we are in beta, the pypi.org infrastructure cannot yet support all of the API traffic generated by our users running pip install, so don't explicitly point to it in automated production setups. But we want you to try it and test it, and we're occasionally redirecting portions of that traffic in load tests we announce on our status page. Uploads, search, and release management are working well, and project maintainers and owners should use this version of PyPI over the legacy site, which no longer supports uploading releases. More at https://pyfound.blogspot.com/2018/03/warehouse-all-new-pypi-is-now-in-beta.… . Congrats and thanks to the team, and onwards to the next milestone (see https://wiki.python.org/psf/WarehouseRoadmap ), the launch and redirect! -- Sumana Harihareswara Changeset Consulting https://changeset.nyc

1 0

quickly installing packages in "editable" mode
by Chris Jerdonek 25 Mar '18

25 Mar '18

Hi, I'm interested in finding ways to speed up a development workflow that involves installing several local packages in "editable" mode in a Docker image. For example, if I use `pip install -e project1 -e project2 ...` in a certain case, it takes on the order of 3-4 seconds. But if I create the .egg-link files, .egg-info directories, etc. "by hand," it only takes on the order of 0.07 seconds. I'm okay with using a hacky solution because these are simple, pure Python projects and it's in a development environment, but if I can rely on existing third-party libraries (via Python, not subprocess), that would be ideal. Do people know of libraries that can help with this sort of thing? As another example, it seems like one of the steps is to update "site-packages/easy-install.pth", but setuptools doesn't seem to expose this as a stand-alone function. I also briefly glanced at things like distlib, the "site" module, and pkg_resources, but I didn't seem to see an API for updating a .pth file. Or maybe I can bypass having to create / update a ".pth" by using symlinks. Thanks, --Chris

1 0

PEP 541 - Accepted
by Mark Mangoba 23 Mar '18

23 Mar '18

Hi All, As the BDFL-Delegate, I’m happy to announce PEP 541 has been accepted. PEP 541 has been voted by the packaging-wg (https://wiki.python.org/psf/P ackagingWG/Charter): - Donald Stufft - Dustin Ingram - Ernest W. Durbin III - Ewa Jodlowska - Kenneth Reitz - Mark Mangoba - Nathaniel J. Smith - Nick Coghlan - Nicole Harris - Sumana Harihareswara Thank you to the packaging-wg and to everyone that has contributed to PEP 541. Best regards, Mark -- Mark Mangoba | PSF IT Manager | Python Software Foundation | mmangoba(a)python.org | python.org | Infrastructure Staff: infrastructure-staff(a)python.org | GPG: 2DE4 D92B 739C 649B EBB8 CCF6 DC05 E024 5F4C A0D1

1 0

Removing wheel signing features from the wheel library
by alex.gronholm＠nextday.fi 23 Mar '18

23 Mar '18

After spending quite some time thinking about this, I've decided to cut out the wheel signature related features from the wheel codebase, unless there is significant resistance among the readers of this mailing list. For those not involved in the previous discussion, the reasoning is that the codebase can be significantly simplified by removing this rarely used feature whose practical value is questionable at best, given the lack of infrastructure for public key distribution. If, as a result, there is a public outcry (which I sincerely doubt), I can always add it back to the next release.

8 16

TUF, Warehouse, Pip, PyPA, ld-signatures, ed25519
by Wes Turner 22 Mar '18

22 Mar '18

TUF, Warehouse, Pip, PyPA, ld-signatures, ed "PEP 480 -- Surviving a Compromise of PyPI" https://www.python.org/dev/peps/pep-0458/ "PEP 480 -- Surviving a Compromise of PyPI: The Maximum Security Model" https://www.python.org/dev/peps/pep-0480/ I need to spend time reviewing these PEPs. Backseat driving here; sorry: Are there pypa/warehouse github issues for implementing the TUF trust root support in warehouse; and client support in pip (or a module that pip and other tools can use)? Warehouse is already a SPOF. That's a hefty responsibility that contributions should support. Would [offline] package mirrors and the CDN still work for/with TUF keys? ld-signatures has some normative language that could be useful. ld-signatures uses URIs for signature suites (a canonicalization algorithm, a message digest algorithm, and a signature algorithm) and JSONLD. That should be pretty future proof in regards to the NIST post-quantum algorithms call that's under review at this time. Blockcerts builds upon ld-signatures. There's a compact form of JSONLD. JSON[LD] can also be serialized as BSON (and RDFHDT). "Linked Data Signatures 1.0" (draft) https://w3c-dvcg.github.io/ld-signatures/ "Ed25519 Signature 2018" (draft) https://w3c-dvcg.github.io/lds-ed25519-2018/ - canonicalizationAlgorithm: https://w3id.org/security#URDNA2015 - digestAlgorithm: http://w3id.org/digests#sha512 - signatureAlgorithm: http://w3id.org/security#ed25519 https://theupdateframework.github.io/ https://github.com/theupdateframework/specification/blob/master/tuf-spec.md… On Thursday, March 22, 2018, Trishank Kuppusamy < trishank.kuppusamy(a)datadoghq.com> wrote: > Hi Wes, > > On Thu, Mar 22, 2018 at 4:40 PM, Wes Turner <wes.turner(a)gmail.com> wrote: > >> >> The hashes serve as file integrity check but provide no assurance that >> they are what the author intended to distribute because there is no >> cryptographic signature. >> >> File hashes help detect bit flips -- due to solar flares -- in storage or >> transit, but do not mitigate against malicious package modification to >> packages in storage or transit. >> >> AFAIU, TUF (The Update Framework) has a mechanism for limiting which >> signing keys are valid for which package? Are pre-shared keys then still >> necessary, or do we then rely on a PKI where one compromised CA cert can >> then forge any other cert? >> > > Yes, you are right, the hashes need to be signed: otherwise you have > integrity, but no authenticity. > > We wrote PEPs 458 <https://www.python.org/dev/peps/pep-0458/> and 480 > <https://www.python.org/dev/peps/pep-0480/> to discuss how TUF might be > deployed on PyPI / Warehouse. The PEPs go into details about public key > distribution. The TLDR is that is that clients (i.e., pip) need to be > shipped with one self-signed root metadata file, and the rest of the PKI is > bootstrapped from there. PyPI would act as an authority that distributes, > revokes, and replaces public keys for packages. > > More details on security vs usability also available in our Diplomat > <https://www.usenix.org/conference/nsdi16/technical-sessions/presentation/ku…> > paper. > > If the community is interested, we'd love to discuss how we could help > make this happen. > > Thanks, > Trishank > >

3 2

PyPI support for linux_ppc64le
by Matt McCormick 22 Mar '18

22 Mar '18

Hi, I created a wheel for Linux / ppc64le (built on CentOS 7). When I try to upload to PyPI via twine, the error message results: unsupported platform tag 'linux_ppc64le' What is the status of the ppc64le wheel support? Is pip / PyPI support expected soon? Thanks, Matt

5 6