<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body text="#000000" bgcolor="#FFFFFF">
Issues created:<br>
<ul>
<li>Ignore dup uploads instead of error:
<a class="moz-txt-link-freetext" href="https://github.com/pypa/warehouse/issues/2284">https://github.com/pypa/warehouse/issues/2284</a></li>
<li>Rename "/legacy" API URL:
<a class="moz-txt-link-freetext" href="https://github.com/pypa/warehouse/issues/2285">https://github.com/pypa/warehouse/issues/2285</a></li>
<li>Propose "packaging announce" mailing list:
<a class="moz-txt-link-freetext" href="https://github.com/pypa/packaging-problems/issues/98">https://github.com/pypa/packaging-problems/issues/98</a></li>
<ul>
<li>Previous issue noting challenges in communicating on
packaging, many of the same concerns still seem relevant:
<a class="moz-txt-link-freetext" href="https://github.com/pypa/packaging-problems/issues/21">https://github.com/pypa/packaging-problems/issues/21</a></li>
</ul>
<li>Explain what "pre-production" means on Warehouse:
<a class="moz-txt-link-freetext" href="https://github.com/pypa/warehouse/issues/2287">https://github.com/pypa/warehouse/issues/2287</a></li>
<li>Deprecation notice added to:
<a class="moz-txt-link-freetext" href="https://wiki.python.org/moin/TestPyPI">https://wiki.python.org/moin/TestPyPI</a></li>
<ul>
<li>Issue created to enhance docs for TestPyPI:
<a class="moz-txt-link-freetext" href="https://github.com/pypa/python-packaging-user-guide/issues/362">https://github.com/pypa/python-packaging-user-guide/issues/362</a></li>
</ul>
<li>Better error message for PEP 527 accepted file types:
<a class="moz-txt-link-freetext" href="https://github.com/pypa/packaging-problems/issues/99">https://github.com/pypa/packaging-problems/issues/99</a></li>
</ul>
<p>I hope I got all the right places for those issues.<br>
</p>
<p>Thanks Donald for the clear reply from which it was easy to
create these separate issues. I didn't realize you had so little
paid time to work on packaging issues. I'm surprised with all the
large companies that use Python that the PSF can't secure more
donations to invest in fixing these issues. Or that someone like
Redhat can't get a couple engineers to devote time for a couple
years like they do to other OSS projects. (No criticism intended,
just musing).<br>
</p>
<div class="moz-signature"><br>
<b>Randy Syring</b><br>
<small>Husband | Father | Redeemed Sinner</small><br>
<br>
<i><small>"For what does it profit a man to gain the whole world<br>
and forfeit his soul?" (Mark 8:36 ESV)</small></i>
<br>
<br>
</div>
<div class="moz-cite-prefix">On 08/04/2017 05:51 PM, Donald Stufft
wrote:<br>
</div>
<blockquote type="cite"
cite="mid:B461550D-F256-4A07-A1A8-B272862E7DD8@stufft.io">
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<br class="">
<div>
<blockquote type="cite" class="">
<div class="">On Aug 4, 2017, at 4:43 PM, Lucas Boppre Niehues
<<a href="mailto:lucasboppre@gmail.com" class=""
moz-do-not-send="true">lucasboppre@gmail.com</a>>
wrote:</div>
<br class="Apple-interchange-newline">
<div class="">
<div dir="ltr" class="">
<div class="">I just had a dreadful day dealing with PyPI
and the migrations that are happening. I'm posting this
rant here in hopes of reaching the people who can fix it
(apologies if I'm in the wrong list), or at least offer
an explanation of the events that happened.<br class="">
<br class="">
</div>
<div class="">Disclaimers:<br class="">
</div>
- Any and all feedback is welcome, but please be
charitable.<br class="">
- I'm angry *at* things, I hope not to offend anybody.<br
class="">
- I'm posting this rant here and not reddit or hn to
ensure the discussion stays between the interested parties
and not the whole internet.<br class="">
<div class="">
<div class=""> <br class="">
For formatting reasons the text is here:<br class="">
<br class="">
<a
href="https://gist.github.com/boppreh/ac7522b3a4ac46b4f6010eecddc57f21"
class="" moz-do-not-send="true">https://gist.github.com/boppreh/ac7522b3a4ac46b4f6010eecddc57f21</a><br
class="">
</div>
<div class=""><br class="">
<br class="">
</div>
<div class="">Thank you for your attention<br class="">
</div>
</div>
</div>
_______________________________________________<br class="">
Distutils-SIG maillist - <a
href="mailto:Distutils-SIG@python.org" class=""
moz-do-not-send="true">Distutils-SIG@python.org</a><br
class="">
<a
href="https://mail.python.org/mailman/listinfo/distutils-sig"
class="" moz-do-not-send="true">https://mail.python.org/mailman/listinfo/distutils-sig</a><br
class="">
</div>
</blockquote>
</div>
<br class="">
<div class=""><br class="webkit-block-placeholder">
</div>
<div class="">Going through your text, there’s a mixture of things
that sort of combined here, I’m kind of just going to stream of
conscious reply to them here, because I can’t reply inline.</div>
<div class=""><br class="">
</div>
<div class=""><br class="">
</div>
<div class="">> error: No dist file created in earlier command</div>
<div class=""><br class="">
</div>
<div class="">Yes, ``setup.py upload`` *requires* you to create
the files it is uploading within the same command during the
upload. This is one of the reasons that twine as a tool exists
and is generally recommended.</div>
<div class=""><br class="">
</div>
<div class=""><br class="">
</div>
<div class="">> It's the exact same file. You could add a check
for it and ignore, but no, it had to be an error.</div>
<div class=""><br class="">
</div>
<div class="">This seems like a reasonable feature request, please
file an issue with Warehouse asking for it so I don’t forget it.</div>
<div class=""><br class="">
</div>
<div class=""><br class="">
</div>
<div class="">> It seems that long_description in setup.py was
missed entirely.</div>
<div class=""><br class="">
</div>
<div class="">I’m not sure what happened here. If you share the
package I can probably figure it out though. I’m not currently
aware of any bugs with how long_description is handled.</div>
<div class=""><br class="">
</div>
<div class=""><br class="">
</div>
<div class="">> By the way, why was this feature included in
the register command?</div>
<div class=""><br class="">
</div>
<div class="">This was not an explicit feature of the register
command, and more of an accident in the way things got
implemented in PyPI’s decade+ history. </div>
<div class=""><br class="">
</div>
<div class=""><br class="">
</div>
<div class="">> [Stuff about editing the description]</div>
<div class=""><br class="">
</div>
<div class="">You located the issue on Warehouse. The fundamental
issue here is trying to find the right balance between
constraining authors so that end users can have a consistent
behavior between packages and giving authors power to best
manage their own projects. Discussion about editing the
description is probably best handled on the warehouse issue
though (Which I forgot to update with my thoughts!).</div>
<div class=""><br class="">
</div>
<div class=""><br class="">
</div>
<div class="">> My setup.py has been communicating with <a
href="https://upload.pypi.org/legacy/" class=""
moz-do-not-send="true">https://upload.pypi.org/legacy/</a> by
default. Note the legacy part.</div>
<div class=""><br class="">
</div>
<div class="">This is confusing, and it is essentially due to the
fact there are two things here which are considered “legacy”.
One is the legacy codebase/deployment that currently powers <a
href="http://pypi.python.org" class="" moz-do-not-send="true">pypi.python.org</a>
which are are slowly replacing and migrating things over to the
new, modern code base that powers <a href="http://pypi.org"
class="" moz-do-not-send="true">pypi.org</a>. The other thing
is the actual upload API itself, which has stayed the same
currently, but which I plan to replace at some point in the
future. This API is also considered “legacy” (it just doesn’t
have a replacement yet). So the legacy in <a
href="https://upload.pypi.org/legacy/" class=""
moz-do-not-send="true">https://upload.pypi.org/legacy/</a> has
to do with the API rather than the location/deployment.</div>
<div class=""><br class="">
</div>
<div class="">I probably should have named it something other than
/legacy/, my goal was mostly that changing URLs is hard (it
requires a bunch of documentation/updates in different packages
and N years for that to percolate out) so since we were forcing
people to change the URLs with the move from <a
href="http://pypi.python.org" class="" moz-do-not-send="true">pypi.python.org</a> to
<a href="http://pypi.org" class="" moz-do-not-send="true">pypi.org</a>,
might as well get it all done at once. It might be reasonable to
name it something else now, and just keep the /legacy/ around as
an alias. I’m not sure if that adds or subtracts from the
confusion, but if you think that would have helped you, please
open a new issue on Warehouse.</div>
<div class=""><br class="">
</div>
<div class=""><br class="">
</div>
<div class="">> The "new APIs" seem to be missing at last one
important feature</div>
<div class=""><br class="">
</div>
<div class="">Some of this is purposeful as we attempt to rein in
some of the more “random” features that PyPI has grown over
time. Some of it is accidental as there’s not really any
documentation about what PyPI provides, and people have, over
the decade of it’s existence, depended on all sorts of random
bits and pieces of PyPI which makes it hard to tell what people
are even using.</div>
<div class=""><br class="">
</div>
<div class=""><br class="">
</div>
<div class="">> I must have missed the deprecation period that
preceded the complete removal of the old APIs. A friendly email
would have been nice if you expect me to migrate to a new
solution.</div>
<div class=""><br class="">
</div>
<div class="">This is a hard balance to strike. When I have
generated emails in the past I tend to get a slew of people
berating me for sending them emails about things. Particularly
for changes like this where there belief is that for most
people, the impact should be minimal, if there is any at all. I
do try to at least tweet and post to distutils-sig about
possibly disruptive changes like this. For this specific change,
that thread was at <a
href="https://mail.python.org/pipermail/distutils-sig/2017-June/030766.html"
class="" moz-do-not-send="true">https://mail.python.org/pipermail/distutils-sig/2017-June/030766.html</a>.</div>
<div class=""><br class="">
</div>
<div class=""><br class="">
</div>
<div class="">> The new website is "pre-production”.</div>
<div class=""><br class="">
</div>
<div class="">This is mostly just a warning at this point that the
UI might return funky results, some things might not be great
yet, and to not start throwing the full force of people’s
production traffic at it. The current hosting arrangement can’t
handle the full amount of traffic we get from ``pip install …``
(but uploads are fine!) and we’re working on building on the new
infrastructure with our future hosting arrangement that can
handle the full production load.</div>
<div class=""><br class="">
</div>
<div class=""><br class="">
</div>
<div class="">> HTTPError: 400 Client Error: Invalid file
extension. for url: <a href="https://upload.pypi.org/legacy/"
class="" moz-do-not-send="true">https://upload.pypi.org/legacy/</a></div>
<div class=""><br class="">
</div>
<div class="">The ability to upload anything besides sdists,
wheels, and eggs was deprecated and removed. You can read the
PEP that removed them at <a
href="https://www.python.org/dev/peps/pep-0527/" class=""
moz-do-not-send="true">https://www.python.org/dev/peps/pep-0527/</a>.</div>
<div class=""><br class="">
</div>
<div class=""><br class="">
</div>
<div class="">> Maybe the tutorial is outdated, and TestPyPI
supports auto-registration now?</div>
<div class=""><br class="">
</div>
<div class="">I don’t think anyone has kept the tutorial on <a
href="http://wiki.python.org" class="" moz-do-not-send="true">wiki.python.org</a> up
to date. To be frank, I don’t even know how to update <a
href="http://wiki.python.org" class="" moz-do-not-send="true">wiki.python.org</a>.
Generally we’ve been trying to consolidate packaging
documentation onto <a href="http://packaging.python.org"
class="" moz-do-not-send="true">packaging.python.org</a> —
although I don’t think we have one for TestPyPI yet.</div>
<div class=""><br class="">
</div>
<div class="">TestPyPI itself is also severely under maintained. I
don’t use it so I rarely even think about it and my time is
stretched thin to an extreme degree. I plan on getting rid of
TestPyPI at some point and replacing it with something better,
You can see discussion about that at <a
href="https://github.com/pypa/warehouse/issues/726" class=""
moz-do-not-send="true">https://github.com/pypa/warehouse/issues/726</a>.</div>
<div class=""><br class="">
</div>
<div class=""><br class="">
</div>
<div class="">> I try python setup.py bdist upload</div>
<div class=""><br class="">
</div>
<div class="">As I mentioned above, we deprecated/removed
everything but sdist, wheels, and eggs as per PEP 527. We could
probably do better error messages here though.</div>
<div class=""><br class="">
</div>
<div class=""><br class="">
</div>
<div class="">> I can't even specify the description of a
package, not even during registration or upload.</div>
<div class=""><br class="">
</div>
<div class="">Tell me your package name privately or publicly and
I’ll figure out what went wrong.</div>
<div class=""><br class="">
</div>
<div class=""><br class="">
</div>
<div class="">> I cannot even report the issues. [ ... ]
because there's no clear location to report them.</div>
<div class=""><br class="">
</div>
<div class="">This is a problem, and we don’t really have a good
solution for them. I think generally what happens is people just
open them on whichever tool they think is the best fit, and the
authors of these tools all know each other, and if it ends up
getting filed in the incorrect place, we just redirect people
tot he correct place.</div>
<div class=""><br class="">
</div>
<div class=""><br class="">
</div>
<div class="">> I am trying to donate my precious free time to
the Python community by offering free software, receiving
absolutely no compensation whatsoever, and every step of the
experience has been shitty.</div>
<div class=""><br class="">
</div>
<div class="">I/we understand that, and we’re generally trying to
do the best we can with what limited resources we have. The flip
side of this is that we’re *also* (for the most part) donating
our precious free time to the Python community to run PyPI (and
the other PSF services too!). For the record, in July PyPI used
something just under 1 Petabyte of bandwidth and served 38
billion HTTP requests.</div>
<div class=""><br class="">
</div>
<div class="">People working on that include myself with 16h/week
of paid time (split between *all* of the packaging related tools
I work on) plus something like 20-30h/week of volunteer time (on
top of the additional 24h/week of time I spent working on my
employers projects). On the ops side there is another individual
(who I won’t name here to prevent him from getting drug into
this [1]) who donates his spare time as well (though I don’t
know offhand how much that is).</div>
<div class=""><br class="">
</div>
<div class="">As far as keeping PyPI running as well as getting
the new code base developed and deployed… that’s about it [2].
This is a service used by ~everyone in the Python community
without even a single full time person on it.</div>
<div class=""><br class="">
</div>
<div class="">We’re trying to slowly migrate to a new code base
which is (A) easier to maintain/scale taking up fewer hours (B)
easier to actually work on to develop new features, which will
hopefully help attract and retain new contributors and (C) has
removed decades of cruft that has built up and either explicitly
made something supported or explicit decided it is no longer
supported, instead of a quasi supported “well it’s what PyPI
does” scenario we have now.</div>
<div class=""><br class="">
</div>
<div class="">This probably won’t be the last less than perfect
phase of the migration to the new code base. My best
recommendation if you don’t want to discover these things JIT
(even though we *do* try to minimize them) is to subscribe to
and follow distutils-sig. Alternatively if you want to get more
involved, one thing we’re severely lacking is a better mechanism
for communicating these things to users, and if you wanted to
propose/get sign off/implement/etc something for that, then that
is another possibility. Finally if you wanted you could also get
involved in actually shaping the direction of these tools,
either by discussing on distutils-sig and in their respective
issue trackers, or by becoming an active contributor. That’s the
best way to ensure that your relevant use cases are considered
during the design or execution of things.</div>
<div class=""><br class="">
</div>
<div class=""><br class="">
</div>
<div class="">[1] Folks can yell at me if they want, don’t go
yelling at other contributors. Generally the buck stops with me
anyways.</div>
<div class="">[2] There is a designer working on the new code
base, and there have been some other contributors, but since
this is about the APIs and maintenance I left them out.</div>
<div class=""><br class="">
</div>
<div class=""><span style="orphans: 2; text-align: -webkit-auto;
widows: 2;" class="">—</span></div>
<div class="">
<div style="color: rgb(0, 0, 0); font-family: Helvetica;
font-style: normal; font-variant-ligatures: normal;
font-variant-position: normal; font-variant-caps: normal;
font-variant-numeric: normal; font-variant-alternates: normal;
font-variant-east-asian: normal; font-weight: normal;
letter-spacing: normal; line-height: normal; orphans: 2;
text-align: -webkit-auto; text-indent: 0px; text-transform:
none; white-space: normal; widows: 2; word-spacing: 0px;
-webkit-text-stroke-width: 0px; word-wrap: break-word;
-webkit-nbsp-mode: space; -webkit-line-break:
after-white-space;" class="">Donald Stufft<br class="">
</div>
<div style="color: rgb(0, 0, 0); font-family: Helvetica;
font-style: normal; font-variant-ligatures: normal;
font-variant-position: normal; font-variant-caps: normal;
font-variant-numeric: normal; font-variant-alternates: normal;
font-variant-east-asian: normal; font-weight: normal;
letter-spacing: normal; line-height: normal; orphans: 2;
text-align: -webkit-auto; text-indent: 0px; text-transform:
none; white-space: normal; widows: 2; word-spacing: 0px;
-webkit-text-stroke-width: 0px; word-wrap: break-word;
-webkit-nbsp-mode: space; -webkit-line-break:
after-white-space;" class=""><br class="">
</div>
<br class="Apple-interchange-newline">
</div>
<br class="">
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
Distutils-SIG maillist - <a class="moz-txt-link-abbreviated" href="mailto:Distutils-SIG@python.org">Distutils-SIG@python.org</a>
<a class="moz-txt-link-freetext" href="https://mail.python.org/mailman/listinfo/distutils-sig">https://mail.python.org/mailman/listinfo/distutils-sig</a>
</pre>
</blockquote>
<br>
</body>
</html>