Mailman 3 December 2017 - Distutils-SIG

Warehouse update: highest priority tasks
by Sumana Harihareswara Dec. 20, 2017

Dec. 20, 2017

Cross-posting from pypa-dev https://groups.google.com/forum/#!topic/pypa-dev/2hulwNQFhRk . I'm realizing that pypa-dev might be a better place to send updates like this than distutils-sig and would appreciate feedback on that question (probably off-list is better). -Sumana Harihareswara -------- Forwarded Message -------- Subject: Warehouse update: highest priority tasks Date: Tue, 19 Dec 2017 18:59:01 -0500 From: Sumana Harihareswara <sh(a)changeset.nyc> To: pypa-dev(a)googlegroups.com Today the Warehouse project team met to discuss what to deliver first, in the Maintainer MVP milestone[0] (full notes on the Python wiki[1]). The point of this first milestone is to give package maintainers a solid chance to try out Warehouse and report critical bugs early. We'll include several features to let users change and update your accounts & project files: * Enable pages to delete, hide, unhide a project/release/file[2] * Enable users to modify their own account[3] * UI for adding maintainers[4] * Implement "reset lost password" feature[5] Due to changes in hosting documentation on pythonhosted, we will implement a button for users to delete or redirect the documentation they uploaded.[6] We'll make and maintain a roadmap and publicize it on pypi.org, then move it to our GitHub repo when work probably slows down after the launch.[7] We've slotted in some additional issues, and we made some decisions about release metadata immutability[8] and a couple other issues you’ll hear more about on distutils-sig in the next couple days. So now that we've made this first pass and are fairly confident what’s left between us and this first milestone, we're figuring out how long it'll be till we can deliver that. Laura Hampton and I will keep you posted. And we welcome help in GitHub[9] and feedback on GitHub issues or here on the list! -Sumana Harihareswara [0] https://github.com/pypa/warehouse/milestone/8 [1] https://wiki.python.org/psf/PackagingWG/2017-12-19-Warehouse [2] https://github.com/pypa/warehouse/issues/424 [3] https://github.com/pypa/warehouse/issues/423 [4] https://github.com/pypa/warehouse/issues/956 [5] https://github.com/pypa/warehouse/issues/1228 [6] https://github.com/pypa/warehouse/issues/582 [7] https://github.com/pypa/warehouse/issues/1322 [8] https://mail.python.org/pipermail/distutils-sig/2017-December/031826.html [9] https://github.com/pypa/warehouse/issues?q=is%3Aissue+is%3Aopen+label%3A%22… P.S. Thanks to Laura Hampton for help with prep, meeting-running, and followup. -- Sumana Harihareswara Changeset Consulting https://changeset.nyc

1 0

(no subject)
by Suchitrarani Ojha Dec. 20, 2017

Dec. 20, 2017

Sir, I have recently downloaded python 3.6 version and I am very new to python.I tried to open excel files in python using openpyxl but its shows"Module not found error".So I tried to install it using pip install openpyxl and it was showing Syntax error:'Invalid syntax'.Can you help me in resolving the issue. Regards suchitra

2 1

Policy for deleting packages, releases, and artifacts
by Jimmy Jia Dec. 19, 2017

Dec. 19, 2017

Hi everyone, This is moved over from https://github.com/pypa/packaging-problems/issues/112. Currently, PyPI has no limitations around deleting packages, releases, or artifacts. This can be problematic for users, as user builds can break in an unsolicited manner if a dependency is removed from PyPI. In the Node ecosystem, a similar lack of limitations there caused significant problems about a year and a half ago, when a widely-used package was deleted following a dispute: http://blog.npmjs.org/post/141577284765/kik-left-pad-and-npm, http://blog.npmjs.org/post/141905368000/changes-to-npms-unpublish-policy. At the time, the scope of the impact was characterized as "breaking the world". To resolve this, npm adopted a policy where package deletions (there's no distinction between a release and an artifact there) could only be done for the first 24 hours after a release was published. Deletions after the 24 hour mark require contacting npm support, and are contingent on the absence of dependents for the deleted release. Of course, npm is a venture-backed for-profit enterprise that has a paid support team – that's not the case here, so the "support" half of the solution above doesn't make sense here. However, as a starting point, it might still be a good idea to restrict package deletion after that 24-hour window. At least, in the examples given in the links above, the deletion of old packages is strictly a nice-to-have for the package maintainers, balanced against potentially breaking impacts for users. Thanks in advance for any feedback. Thanks, Jimmy Jia

1 1

Re: [Distutils] PEP 345, 566, 508 version specifiers and OR clauses
by Nick Coghlan Dec. 16, 2017

Dec. 16, 2017

On 14 Dec. 2017 1:55 pm, "Donald Stufft" <donald(a)stufft.io> wrote: Overall I’m +1, not sure if it would be a new PEP or an amendment to the current PEPs but I think it’s a good idea. It would be a new PEP bumping the metadata version to 1.4 (that will be less tedious than it used to be though, since it will only have to cover the enhancement, not duplicate all the old field descriptions). Somwehat related though, we should probably have a summary table in PyPUG mapping metadata versions to minimum required tooling versions & their respective release dates. That way folks can make informed decisions about their installability choices before deciding which version to use in their published metadata. Cheers, Nick. _______________________________________________ Distutils-SIG maillist - Distutils-SIG(a)python.org https://mail.python.org/mailman/listinfo/distutils-sig

1 0

Or in version spec...
by Chris Barker - NOAA Federal Dec. 15, 2017

Dec. 15, 2017

Sorry to lose the thread — lousy iPhone mail app... Conda supports or in its meta.yaml format: https://conda.io/docs/user-guide/tasks/build-packages/package-spec.html#bui… Maybe look to that for prior art? And it would be mildly less confusing to have consistency between the systems. -Chris Sent from my iPhone

2 1

PEP 345, 566, 508 version specifiers and OR clauses
by Barry Warsaw Dec. 14, 2017

Dec. 14, 2017

I'm about to release a new version of importlib_resources, so I want to get my flit.ini's require-python clause right. We support Python 2.7, and 3.4 and beyond. This makes me sad: requires-python = '>=2.7,!=3.0,!=3.1,!=3.2,!=3.3' Of course, I'd like to write this like: requires-python = '(>=2.7 and <3) or >= 3.4' I understand that OR clauses aren't supported under any syntax currently, but as PEPs 566 and 508 are still open/active, wouldn't it be reasonable to support something like this explicitly? It seems like wanting to support 2.7 and some versions of Python 3 (but not all) is a fairly common need. Cheers, -Barry

5 7

RFC: PEP 566 - Metadata for Python Software Packages 1.3
by Dustin Ingram Dec. 12, 2017

Dec. 12, 2017

Hi all, I'd appreciate your feedback on the following Metadata 1.3 PEP. The goal here is not to provide a full specification for all fields as in previous PEPs, but to: * Motivate and describe the addition of the new fields or changes to existing fields; * Point to the Core Metadata Specifications reference document as the canonical source for the specification. Previous discussions: * https://mail.python.org/pipermail/distutils-sig/2017-September/031465.html * https://github.com/python/peps/issues/388 Thanks, D. PEP: 566 Title: Metadata for Python Software Packages 1.3 Version: $Revision$ Last-Modified: $Date$ Author: Dustin Ingram <di(a)di.codes> Discussions-To: distutils-sig <distutils-sig at python.org> Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 1-Dec-2017 Python-Version: 3.x Post-History: Replaces: 345 Abstract ======== This PEP describes the changes between versions 1.2 and 1.3 of the core metadata specification for Python packages. Version 1.2 is specified in PEP 345. It also changes to the canonical source for field specifications to the `Core Metadata Specification`_ reference document, which includes specifics of the field names, and their semantics and usage. Fields ====== The canonical source for the names and semantics of each of the supported metadata fields is the `Core Metadata Specification`_ document. Fields marked with "(Multiple use)" may be specified multiple times in a single PKG-INFO file. Other fields may only occur once in a PKG-INFO file. Fields marked with "(optional)" are not required to appear in a valid PKG-INFO file; all other fields must be present. New in Version 1.3 ------------------ Description-Content-Type (optional) ::::::::::::::::::::::::::::::::::: A string stating the markup syntax (if any) used in the distribution's description, so that tools can intelligently render the description. Historically, tools like PyPI assume that a package's description is formatted in `reStructuredText (reST) <http://docutils.sourceforge.net/docs/ref/rst/restructuredtext.html>`_, and fall back on plain text if the description is not valid reST. The introduction of this field allows PyPI to support additional types of markup syntax, and not need to make this assumption. The full specification for this field is defined in the `Core Metadata Specification`_. Provides-Extra (optional, multiple use) ::::::::::::::::::::::::::::::::::::::: A string containing the name of an optional feature. Must be a valid Python identifier. May be used to make a dependency conditional on whether the optional feature has been requested. This introduction of this field allows packge installation tools (such as ``pip``) to determine which extras are provided by a given package, and so that package publication tools (such as ``twine``) can check for issues with environment markers which use extras. The full specification for this field is defined in the `Core Metadata Specification`_. Changed in Version 1.3 ---------------------- Name :::: The specification for the format of this field is now identical to the distribution name specification defined in PEP 508. Version Specifiers ================== Version numbering requirements and the semantics for specifying comparisons between versions are defined in PEP 440. Environment markers =================== An **environment marker** is a marker that can be added at the end of a field after a semi-colon (";"), to add a condition about the execution environment. The environment marker format used to declare such a condition is defined in the environment markers section of PEP 508. JSON-compatible Metadata ======================== It may be necessary to store metadata in a data structure which does not allow for multiple repeated keys, such as JSON. The canonical method to transform metadata fields into such a data structure is as follows: #. The original key-value format should be read with ``email.parser.HeaderParser``; #. All transformed keys should be reduced to lower case, but otherwise should retain all other characters; #. The transformed value for any field marked with "(Multiple-use") should be a single list containing all the original values for the given key; #. The ``Keywords`` field should be converted to a list by splitting the original value on whitespace characters; #. The result should be stored as a string-keyed dictionary. Summary of Differences From PEP 345 =================================== * Metadata-Version is now 1.3. * Fields are now specified via the `Core Metadata Specification`_. * Added two new fields: ``Description-Content-Type`` and ``Provides-Extra`` * Acceptable values for the ``Name`` field are now specified as per PEP 508. * Added canonical method of transformation into JSON-compatible data structure. References ========== This document specifies version 1.3 of the metadata format. Version 1.0 is specified in PEP 241. Version 1.1 is specified in PEP 314. Version 1.2 is specified in PEP 345. .. _`Core Metadata Specification`: https://packaging.python.org/specifications/core-metadata/ Copyright ========= This document has been placed in the public domain. Acknowledgements ================ Thanks to Nick Colgan for contributing to this PEP. .. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 80 End:

8 13

Multiple package authors
by Barry Warsaw Dec. 7, 2017

Dec. 7, 2017

I think I implicitly knew this, but as I've just released a package (to be announced soon) that actually has multiple authors, I found out first hand that PyPI rejects uploads where the author-email field isn't a completely valid email address, and that there is no support for multiple author emails. As it turns out, you can kludge this into your pyproject.toml or setup.py file. flit for example separates multiple emails with a newline, but you could also separate them with commas. You don't notice the problem until PyPI rejects the upload (with a 400 IIRC). I filed this issue with flit: https://github.com/takluyver/flit/issues/153 It looks like Thomas agrees that at least flit will eventually validate its fields so you error early. It was a bit of a PITA to do my upload because I didn't notice the problem until after I'd tagged the repo. Multiple package authors doesn't seem like that fringe of a use case; are there any plans, documents, PEPs, musings, grumbles about supporting multiple package authors explicitly? Cheers, -Barry

2 2

Outstanding questions for PEP 541: Package Index Name Retention
by Sumana Harihareswara Dec. 7, 2017

Dec. 7, 2017

While getting up to speed on Warehouse I saw how many package transfer requests are waiting for PEP 541[0] to be accepted, so I thought it might be helpful to round up what I see as the outstanding questions. 1) Usage criteria for abandoned projects. > The tricky part there is that "being used" is a tough concept to > define. Over what time period? What amount of downloading counts as > "used"?-- Chris Rose[1] Perhaps a month? suggests Matthias Bussonnier.[2] The ensuing discussion includes thoughts on locking old versions of the project[3]; as I see it, that's a potential feature request for Warehouse, but not something to build into this PEP. (Similarly, Nick Timkovich's idea of "salting" hot-button names on PyPI so it isn't possible to register projects like "android"[4] is a feature idea I like but I think this PEP does not need to wait for it.) 2) A few copyedits from Chris Barker.[5] 3) "How would I, for example, start the process of flagging a project as abandoned?" -- Nick Timkovich[6] It seems to me that the PEP's wording in "Removal of an abandoned project" says we'll allow transfer of abandoned projects but will not remove them only for being abandoned. However, the PEP currently doesn't include a "where to file a ticket" line in that section (as it does in "Invalid projects"). Also, if there's some other reason we should be willing to remove abandoned projects even without a transfer, e.g., the project has a critical security flaw, we should say that somewhere in this PEP or in a different policy document. There is one item Łukasz mentioned this in the "I decided to not otherwise touch on:" list regarding the revision on January 14th[7] that I think deserves one more chance for discussion. :) A few people brought up making the reachability criteria and instructions crisper. > Regarding reachability: contact attempts should also include the > relevant project's issue tracker if attempts at private contact have > failed. > > This step is important as it allows a project's *user* community to > respond, even if the person that actually pushes the buttons to > upload new releases to PyPI is out of contact for some reason. -- Nick Coghlan[8] Nick Timkovich also suggested some specific instructions[9] that would help set expectations to, among other things, reassure maintainers about the length of offline vacations they can take without worrying about having their packages usurped. :) I'm totally fine with folks saying to me: no, these are all addressed, or not important enough to slow adoption of this PEP. In which case, yay, I hope Donald can accept it and we can start processing the backlog. [0] https://www.python.org/dev/peps/pep-0541/ [1] https://mail.python.org/pipermail/distutils-sig/2017-January/030017.html [2] https://mail.python.org/pipermail/distutils-sig/2017-January/030020.html [3] https://mail.python.org/pipermail/distutils-sig/2017-January/030034.html [4] https://mail.python.org/pipermail/distutils-sig/2017-January/030006.html [5] https://mail.python.org/pipermail/distutils-sig/2017-September/031517.html [6] https://mail.python.org/pipermail/distutils-sig/2017-January/030005.html [7] https://mail.python.org/pipermail/distutils-sig/2017-January/030009.html [8] https://mail.python.org/pipermail/distutils-sig/2017-January/030008.html [9] https://mail.python.org/pipermail/distutils-sig/2017-January/030005.html -- Sumana Harihareswara Changeset Consulting https://changeset.nyc

3 2

Update on Warehouse kickoff and milestones
by Sumana Harihareswara Dec. 7, 2017

Dec. 7, 2017

As Donald and Nick mentioned last month,[0] several folks are -- thanks to Mozilla Open Source Support -- working on deploying Warehouse and making PyPI more sustainable. I'm the main project manager on this effort and am putting notes on wiki.python.org as we go.[1] Ernest W. Durbin III gave the closing talk at North Bay Python a few days ago: "Running Vintage Software: PyPI's Aging Codebase."[2] As he mentioned (video[3]), he's one of the people who'll be working on this project, funded by MOSS, to get Warehouse across the finish line! The full cast is: * Donald Stufft, backend * Dustin Ingram, backend * Ernest W. Durbin III, backend * Laura Hampton, project management, testing, documentation * Nicole Harris, frontend and design * Sumana Harihareswara, project management, product management, testing, documentation None of us are working 40 hours/week on this but the six people I listed above are dedicating committed time to the work. And we're grateful for help from Mark Mangoba, Eric Holscher, and of course several other folks who are involved in the PSF, PyPA, or MOSS. We had a kickoff call on Monday[4] and today a few of us spoke in #pypa-dev and fleshed out our milestones[5] a bit. This may still change, but right now, the sequence is: 1. Maintainer Minimum Viable Product (for package owners) -- ask some maintainers to use, test, and report back 2. End User MVP -- ask some Python users who aren't package owners to use Warehouse, test, and report back 3. Publicize beta more broadly (invite more testing, redirect some `pip install` traffic) 4. Launch/redirect `pip` and browsers to Warehouse 5. Shut down legacy PyPI 6. Cool feature ideas that are not on critical path We'll be putting issues about infrastructure work (e.g., Kubernetes) in https://github.com/python/pypi-infra . We've started filing, updating, and arranging issues but that isn't yet complete enough for me to even start making schedule estimates. I aim to have a clearer picture next week. (I won't spam this list with updates when there's nothing new to say, but at least as we start I figure you might want news a few times a month to learn how things are shaping up; please let me know privately if my estimation there is way off.) [0] https://mail.python.org/pipermail/distutils-sig/2017-November/031782.html [1] https://wiki.python.org/psf/PackagingWG [2] https://2017.northbaypython.org/schedule/presentation/5/ [3] https://www.youtube.com/watch?v=xyPd4WOR_ng [4] https://wiki.python.org/psf/PackagingWG/2017-12-04-Warehouse [5] https://github.com/pypa/warehouse/milestones -- Sumana Harihareswara Changeset Consulting https://changeset.nyc

1 0