[Distutils] Buildout can't talk to PyPI any more?
Donald Stufft
donald at stufft.io
Wed May 29 11:39:08 CEST 2013
On May 29, 2013, at 5:18 AM, Marius Gedminas <marius at pov.lt> wrote:
> On Wed, May 29, 2013 at 11:03:39AM +0200, Lennart Regebro wrote:
>> On Wed, May 29, 2013 at 10:19 AM, Marius Gedminas <marius at pov.lt> wrote:
>>> Could it be that recent PyPI CDN changes broke zc.buildout somehow?
>>> Witness this:
>>>
>>> $ wget http://downloads.buildout.org/2/bootstrap.py
>>>
>>> $ cat buildout.cfg
>>> [buildout]
>>> parts = zodbbrowser
>>> [zodbbrowser]
>>> recipe = zc.recipe.egg
>>>
>>> $ python bootstrap.py
>>>
>>> $ bin/buildout
>>> Error: Couldn't find a distribution for 'zodbbrowser'.
>>
>> Works for me.
>
> Works for me too, now.
>
>
> Interesting detail: when buildout was failing, and I tried
>
> curl http://pypi.python.org/simple/zodbbrowser/
>
> all I got was a bunch of binary garbage. To be fair, curl -I showed
> Content-encoding: gzip, and piping curl to zcat showed the HTML just
> fine. But the interesting thing is that when I try this now, I get raw
> HTML.
>
> Can it be that buildout doesn't handle Content-Encoding: gzip?
>
> I was also looking at raw HTTP requests with ngrep, and saw that
> buildout issued something like
>
> GET /simple/zodbbrowser HTTP/forgotwhichversionsadly
> Host: ...
> Accept-Encoding: identity
> User-Agent: blah blah distribute 0.6.44 blah blah Python/urllib
>
> That "Accept-Encoding" header kind of hints that distribute can't handle
> gzipped index pages. So maybe Varnish or nginx or something on the CDN
> side was misconfigured and ignored the Accept-Encoding? But why would
> pip/easy_install work then -- they use distribute too!? (Or was I
> accidentally using a virtualenv that had setuptools instead of
> distribute? Python packaging aargh.)
>
>
> Another possibly interesting detail: I made a release of zodbbrowser
> 0.11.0 this morning. Buildout was unable to get it right after that,
> and this condition lasted for several hours (from 07:40 UTC until some
> point between 08:20 and 09:00 UTC).
>
> The Zope Windows buildbot had a couple of similar intermittent errors
> (buildout was unable to find a distribution of some package) that went
> away when the build job was retried after 24 hours. Could it be that
> something about the PyPI CDN causes new package releases to be gzipped
> for an hour or so, and then revert to plain HTML?
>
> I'm tempted to do a test, something like
>
> curl -I http://pypi.python.org/simple/somepackage/
> cd ~/src/somepackage && fullrelease # zest.releaser FTW
> curl -I http://pypi.python.org/simple/somepackage/
>
> if I can find some package ready for a new release.
>
> Marius Gedminas
> --
> We'll show a small example here with one user calling another. (Under
> international treaties describing technical papers these users must be called
> "Alice" and "Bob".)
> -- Anthony Baxter
> _______________________________________________
> Distutils-SIG maillist - Distutils-SIG at python.org
> http://mail.python.org/mailman/listinfo/distutils-sig
The cache timeout is an hour unless modifications are made to the package which triggers a cache invalidation.
I bet there is a missing Vary for the encoding. I'll check and get back to you.
-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20130529/21b0fde7/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 841 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20130529/21b0fde7/attachment-0001.pgp>
More information about the Distutils-SIG
mailing list