[Distutils] PyPI/Warehouse: infrastructure hardening & the CAPTCHA conundrum
Sumana Harihareswara
sh at changeset.nyc
Tue Mar 20 22:12:48 EDT 2018
So we aren't quite at beta yet, but we'll be shouting about pypi.org
*really soon*. We have nearly all the Warehouse improvements we need for
beta, and nearly all the infrastructure improvements we believe we'll
need for the switchover.
I'll tell you how you can help, then talk about the current state
of things.
* The big blocker keeping us from beta: China & CAPTCHAs. Help
advise us.[1]
* Comment on a "needs discussion" issue[2].
* Help us with large-scope JavaScript issues[3], like our frontend
testing approach.
* Please talk with Nicole about being a subject or interviewer for
user tests[4].
* Tell me if you're planning to join us at sprints at PyCon or
EuroPython[5].
* Check out our open good first Warehouse issues[6] (we usually have
10+ open) and get started[7].
If you follow https://status.python.org/ you saw we did some load
testing last week and learned from it! We redirected some traffic, for a
few periods, for `pip install`, from the old server to Warehouse, and
learned from it. For instance, people running Ubuntu 14.04 LTS (long
term service release)[8] are usually using a pretty old version of pip,
and people on some versions of the Mac OS[9] have older versions of
Python and old versions of security-related libraries that don't support
the version of TLS that we want them to use. Ernest, Donald and Dustin
did a bunch of work addressing this, including Donald putting out pip
9.0.2[10].
(A thing to understand about Ernest's continuing work on PyPI and
distribution infrastructure is that it's in a lot of places. It's
cabotage[11] & a test cabotage app[12], configuration with salt[13],
conveyor[14], pip[15] & get-pip[16], and he filed a bug in
Kubernetes[17] which I personally find particularly impressive. And it's
in user-facing communication in IRC and GitHub comments and on our
statuspage and Twitter, plus a lot of internal discussion with
infrastructure colleagues. I have a harder time gathering links for
Ernest's work for these emails than for my other teammates; regrets.)
As usual, a summary of the past week's work is in our meeting notes[18].
We have new features like letting PyPI administrators add new trove
classifiers easily[19], infrastructure improvements like this complexity
reduction[20], ton of polish and bug fixing around layout, description
content types (Markdown!), a FAQ restructuring[21], a more useful
collaboration page[22], etc. And we reviewed and merged a lot of
volunteers' pull requests!
Thanks to our prolific volunteers:
* pgadige making sure an error message reflects whether you're on PyPI
or Test PyPI[23] * waseem18 providing an error message for the password reset[24]
* cryvate fixing form requirements for password reset[25]
* waseem18 fixing disabled button CSS[26]
* yeraydiazdiaz fixing modal window behavior[27], then refixing[28]
* berkerpeksag adding a "public profile" link to the user dropdown[29] * Mariatta sending notification email when a project
collaborator's added[30] * berkerpeksag hiding the "view project" button for no-release-yet
projects in maintainers' project lists[31] * alexwlchan renaming a CSS class for consistency[32]
* jMuzsik improving documentation of owners' and maintainers'
privileges[33] * yeraydiazdiaz adding JavaScript validation to show the user if "new
password" and "confirm new password" don't match[34] * alexwlchan documenting all the modifiers in our SASS directory[35]
* alanbato and yeraydiazdiaz adding a check to stop someone
from uploading a file whose blake2 hash matches an already-
uploaded file[36] * cryvate improving sorting of package versions in our /simple/
API[37] * jMuzsik improving how PyPI links look on Twitter, adding an image to
our Twitter cards[38]
* 9999years updating the Python Packaging User Guide[39] and sample
project[40] for Markdown/PEP 566
And thanks to our many bug reporters, especially those who helped us
learn from our load tests.
Also, check out discussion on API key support/macaroons[41], supporting
GitHub-flavored Markdown as Description-Content-Type[42], and project
rating/ranking/stars[43].
And finally, we are ever closer to accepting PEP 541 (and planning
followup tasks[44]) and are testing our PEP 566 compliance[45]. And I
may start a PEP for a Python package index upload API specification[46].
More next week, as usual.
*Thanks to Mozilla for their support[47] for the PyPI & Warehouse
work[48]!*
--
Sumana Harihareswara
Warehouse project manager
Changeset Consulting
sh at changeset.nyc
Links:
1. https://github.com/pypa/warehouse/issues/3174
2. https://github.com/pypa/warehouse/issues?q=is%3Aissue+is%3Aopen+sort%3Aupdated-desc+label%3A%22needs+discussion%22
3. https://github.com/pypa/warehouse/issues/1297
4. http://whoisnicoleharris.com/2018/03/13/user-testing-warehouse.html
5. https://wiki.python.org/psf/PackagingSprints
6. https://github.com/pypa/warehouse/issues?q=is%3Aissue+is%3Aopen+label%3A%22good+first+issue%22
7. https://warehouse.readthedocs.io/development/getting-started/
8. https://github.com/pypa/warehouse/issues/3280
9. https://github.com/pypa/warehouse/issues/3293
10. https://pypi.org/project/pip/9.0.2/
11. https://github.com/cabotage/cabotage-app
12. https://github.com/cabotage/test-app/commits?author=ewdurbin&since=2018-03-01T05:00:00Z&until=2018-03-21T04:00:00Z
13. https://github.com/python/pypi-salt/commit/1a20cd53ffce0fd3d018d989199d30e11d35ad83
14. https://github.com/pypa/conveyor/commits?author=ewdurbin&since=2018-03-13T05:00:00Z&until=2018-03-21T04:00:00Z
15. https://github.com/pypa/pip/pull/5076
16. https://github.com/pypa/get-pip/commits?author=ewdurbin&since=2018-03-01T05:00:00Z&until=2018-03-21T04:00:00Z
17. https://github.com/kubernetes/kubectl/issues/335
18. https://wiki.python.org/psf/PackagingWG/2018-03-19-Warehouse
19. https://github.com/pypa/warehouse/issues/2649
20. https://github.com/pypa/warehouse/pull/3289
21. https://github.com/pypa/warehouse/pull/3190
22. https://github.com/pypa/warehouse/pull/3047
23. https://github.com/pypa/warehouse/pull/3314
24. https://github.com/pypa/warehouse/pull/3220
25. https://github.com/pypa/warehouse/pull/3230
26. https://github.com/pypa/warehouse/pull/3254
27. https://github.com/pypa/warehouse/pull/3251
28. https://github.com/pypa/warehouse/pull/3291
29. https://github.com/pypa/warehouse/pull/3255
30. https://github.com/pypa/warehouse/pull/3155
31. https://github.com/pypa/warehouse/pull/3257
32. https://github.com/pypa/warehouse/pull/3261
33. https://github.com/pypa/warehouse/pull/3313
34. https://github.com/pypa/warehouse/pull/3219
35. https://github.com/pypa/warehouse/pull/3262
36. https://github.com/pypa/warehouse/pull/3310
37. https://github.com/pypa/warehouse/pull/2574
38. https://github.com/pypa/warehouse/pull/3304
39. https://github.com/pypa/python-packaging-user-guide/pull/457
40. https://github.com/pypa/sampleproject/pull/66
41. https://github.com/pypa/warehouse/issues/994
42. https://github.com/pypa/packaging-problems/issues/126
43. https://github.com/pypa/warehouse/issues/991#issuecomment-374665356
44. https://github.com/pypa/warehouse/issues/1506#issuecomment-374626455
45. https://github.com/pypa/warehouse/issues/3299
46. https://github.com/pypa/packaging-problems/issues/128
47. https://blog.mozilla.org/blog/2018/01/23/moss-q4-supporting-python-ecosystem/
48. https://pyfound.blogspot.com/2017/11/the-psf-awarded-moss-grant-pypi.html
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20180320/0f75f4d9/attachment.html>
More information about the Distutils-SIG
mailing list