[Distutils] PyPI/Warehouse update: new advice & launch, shutdown dates

Sumana Harihareswara sh at changeset.nyc
Tue Apr 3 22:29:30 EDT 2018

Big news: we have a schedule for April, and we're advising less-
intensive API users to switch to hitting pypi.org.
April plans[1] (decided this in yesterday's Warehouse core developers'
1. Sunday April 8th: TLS 1.0/1.1 removal[3]
2. Monday April 9th: publicize pypi.org launch date (especially to
   package maintainers and PyPI sponsors)3. Monday April 16: launch/redirect (API and browser traffic redirected
   to Warehouse)4. Monday April 30: shut down legacy PyPI

And today in IRC we decided: If your site/service used to link or
upload to pypi.python.org, you should possibly start using pypi.org
instead[4]. Now.

If you have an automated tool that hits pypi.python.org less often than
once a second, and/or you are fine with small service interruptions
during the beta, then you should go ahead and start using pypi.org
now[5], and subscribe to the PyPI announcement list (low-traffic) for
further announcements[6].
If you run a mission-critical automated production setup, or an
automated tool that hits pypi.python.org with 1+ requests/second, then
you should not explicitly point to pypi.org yet, but you should test and
prepare to switch[7]. You should also watch our status page[8], since
we’re occasionally redirecting portions of that traffic in load tests
that we announce on the status page. Subscribe to the PyPI announcement
list (low-traffic)[9] to find out when you should switch permanently.

Now the more general work update. It's huge because we've been doing so
much, our volunteers are prolific, and I truncated last week's report so
this is a double issue and I will understand if many of you say "TL;DR".
We have 13 open issues till we can launch and redirect pypi.python.org
traffic to Warehouse, and then we have 9 open issues till we can shut
down legacy PyPI (overview[10]). We've been whittling away at that pile,
and fixing other issues as they come up, while responding to new bug
reports (and praise) over the last few weeks (a bit slower because of
some (planned) team absences).
Some improvements from the last two weeks from the MOSS-funded team:

 * our API docs are more accurate[11] and include policies[12]
 *  our "browse" page has default text and suggested searches[13], and
    adjusts to page width[14]
 * prep in case we ever need to briefly turn the site read-only during
 * project pages point you to sources for project stats[16] and release
 *  notifications (banners) are dismissable[18]
 * performance and indexing/caching-related improvements, e.g., purging
   projects & user pages when someone's role changes[19], updating the
   trending projects[20], and updating the search index more often[21]
 * activity journal item order is more consistent and deterministic[22]
 * a project's last serial ID is available via Simple & JSON APIs[23]
Also, Nicole did a bunch of cross-browser testing[24], Ernest worked on
a docs proxy[25] and updated parts of legacy PyPI to point to
Warehouse[26], Dustin fixed issues affecting twine --skip-existing[27]
and a classifier error[28], and I improved developer docs[29] and
classifier order[30]. And Warehouse founder and continuing contributor
Donald Stufft moved email handling into a service[31], improved
performance[32], and removed TLS 1.0/1.1 support[33].
It's deeply important to me to spread the word about the beta, so as
many of our friends, neighbors, and colleagues as possible can try
Warehouse before the big switch, and so we can know about potential
showstoppers as early as possible. So we've held multiple virtual office
hours (IRC logs from today's chat[34]), and we've posted to Python-
specific spaces like the Python Insider blog[35] and python-announce[36]
and debian-python[37], but we've also done some work to get the word out
to programmers who don't particularly pay attention to English-language
Python-specific news. We found volunteers to translate and post the
announcement a little (Brazilian Portugese[38], Polish[39]), and reached
out to (for instance) Software & Data Carpentry[40], NumFOCUS[41], The
Changelog[42], FLOSS Weekly[43] (who'll interview Ernest and Dustin on
May 2nd), and Public Lab[44]. I have a giant checklist and I have been
crossing many things off. It's very satisfying.
I tried out combining our IRC livechats with a Twitter hashtag,
#newpypi, and have gotten approximately zero questions through that so
far. Regardless: enjoy some screenshots!  [45]
And overlapping with the beta rollout, we've been deprecating TLS
1.0/1.1, which leads to issues on some Macs[46], which leads to a
question from Guido that isn't answered yet[47].
And other volunteers wrote, by my count, 21 pull requests that we merged
over the last 2 weeks[48]. jonparrott, nitinprakash96, venthur, pgadige,
saxenanurag, cheungnj, yeraydiazdiaz, AyumuKasuga, jMuzsik, and
aalmazan, thank you so much for all your work! I am particularly
grateful that our volunteers are helping review each other's work, which
helps everyone learn and improve PRs faster.

How you can help:

 * forward the beta announcement[49] to downstreams
 * tell people on Macs to upgrade pip[50], and answer Guido's
   question[51] about which users are potentially affected
 * test[52] Warehouse pull requests, and consider making one[53]
 * talk with Nicole about being a subject or interviewer for user
 * improve the official Python packaging guide[55]
 * remind well-off companies/foundations you know that further Warehouse
   work is more likely if they give the PSF donations[56],
   sponsorship[57], or grants
Thanks again to the Mozilla Open Source Support grant[58] that makes
this work possible.
Sumana Harihareswara
Warehouse project manager
Changeset Consulting
sh at changeset.nyc


   1. https://wiki.python.org/psf/WarehouseRoadmap
   2. https://wiki.python.org/psf/PackagingWG/2018-04-02-Warehouse
   3. https://github.com/pypa/warehouse/issues/3411
   4. https://warehouse.readthedocs.io/api-reference/integration-guide/#migrating-to-the-new-pypi
   5. https://warehouse.readthedocs.io/api-reference/integration-guide/#migrating-to-the-new-pypi
   6. https://mail.python.org/mm3/mailman3/lists/pypi-announce.python.org/
   7. https://pyfound.blogspot.com/2018/03/warehouse-all-new-pypi-is-now-in-beta.html
   8. http://status.python.org/
   9. https://mail.python.org/mm3/mailman3/lists/pypi-announce.python.org/
  10. https://github.com/pypa/warehouse/milestones
  11. https://github.com/pypa/warehouse/pull/3503
  12. https://github.com/pypa/warehouse/pull/3333
  13. https://github.com/pypa/warehouse/pull/3327
  14. https://github.com/pypa/warehouse/pull/3477
  15. https://github.com/pypa/warehouse/pull/3393
  16. https://github.com/pypa/warehouse/pull/3434
  17. https://github.com/pypa/warehouse/pull/3418
  18. https://github.com/pypa/warehouse/pull/3372
  19. https://github.com/pypa/warehouse/pull/3396
  20. https://github.com/pypa/warehouse/pull/3457
  21. https://github.com/pypa/warehouse/pull/3459
  22. https://github.com/pypa/warehouse/pull/3475
  23. https://github.com/pypa/warehouse/pull/3429
  24. https://github.com/pypa/warehouse/labels/cross%20browser%20bug%20%3Abug%3A
  25. https://github.com/pypa/conveyor/pull/3
  26. https://github.com/pypa/pypi-legacy/commits?author=ewdurbin&since=2018-03-01T05:00:00Z&until=2018-04-01T04:00:00Z
  27. https://github.com/pypa/warehouse/pull/3522
  28. https://github.com/pypa/warehouse/pull/3498
  29. https://github.com/pypa/warehouse/pull/3320
  30. https://github.com/pypa/warehouse/pull/3466
  31. https://github.com/pypa/warehouse/pull/3493
  32. https://github.com/pypa/warehouse/pull/3403
  33. https://github.com/pypa/warehouse/pull/3354
  34. http://kafka.dcpython.org/day/pypa-dev/2018-04-03
  35. https://blog.python.org/2018/03/the-all-new-python-package-index-is-now.html
  36. https://mail.python.org/pipermail/python-announce-list/2018-March/011883.html
  37. https://lists.debian.org/debian-python/2018/04/msg00000.html
  38. https://groups.google.com/forum/#!topic/python-brasil/Synj27Fczww
  39. https://www.facebook.com/groups/pythonpl/permalink/1680880335336289/
  40. http://lists.software-carpentry.org/pipermail/discuss/2018-March/005891.html
  41. https://groups.google.com/forum/#!topic/numfocus/uu8aGRmQ-oc
  42. https://changelog.com/news/the-new-pypi-is-finally-in-beta-l66G
  43. https://twit.tv/shows/floss-weekly
  44. https://www.google.com/calendar/event?eid=cTNzdDByZWxmOGRsaXRiMWo3ZXJvY2lwaW9fMjAxODAzMjdUMTkwMDAwWiA1dm90czZraGxlNm02dnNzdWFsdDJvZjg3MEBn&ctz=America/New_York
  45. https://twitter.com/hashtag/newpypi?src=hash
  46. https://mail.python.org/pipermail/python-announce-list/2018-April/011885.html
  47. https://github.com/pypa/warehouse/issues/3293#issuecomment-378416605
  48. https://github.com/pypa/warehouse/pulls?utf8=%E2%9C%93&q=3410+3448+3467+3322+3495+3412+3405+3485+3243+3535+2163+3533+3500+3415+3407+3314+3328+3202+3377+3388+3409+
  49. https://mail.python.org/pipermail/python-announce-list/2018-March/011883.html
  50. https://mail.python.org/pipermail/python-announce-list/2018-April/011885.html
  51. https://github.com/pypa/warehouse/issues/3293#issuecomment-378416605
  52. https://warehouse.readthedocs.io/development/reviewing-patches/#testing-branches-on-your-local-machine
  53. https://warehouse.readthedocs.io/development/getting-started/
  54. http://whoisnicoleharris.com/2018/03/13/user-testing-warehouse.html
  55. https://github.com/pypa/python-packaging-user-guide/issues?q=is%3Aissue+is%3Aopen+label%3A%22help+wanted%22
  56. https://donate.pypi.org/
  57. https://www.python.org/psf/sponsorship/
  58. https://pyfound.blogspot.com/2017/11/the-psf-awarded-moss-grant-pypi.html
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20180403/0b34c37f/attachment.html>

More information about the Distutils-SIG mailing list