Error downloading pythonutils from pypi: Can't process plain .py files
Downloading pythonutils from pypi fails even though it is downloading a perfectly good .zip source file.
easy_install pythonutils==0.3.0 Searching for pythonutils==0.3.0 Reading http://pypi.python.org/simple/pythonutils/ Couldn't find index page for 'pythonutils' (maybe misspelled?) Scanning index of all packages (this may take a while) Reading http://pypi.python.org/simple/ Reading http://pypi.python.org/simple/Pythonutils/ Reading http://www.voidspace.org.uk/python/modules.shtml#pythonutils Reading http://www.voidspace.org.uk/cgi-bin/voidspace/downman.py?file=pythonutils-0. 2.3.zip Reading http://www.voidspace.org.uk/cgi-bin/voidspace/downman.py?file=pythonutils-0. 2.5.zip Reading http://www.voidspace.org.uk/cgi-bin/voidspace/downman.py?file=pythonutils-0. 3.0.zip No local packages or download links found for pythonutils==0.3.0 error: Could not find suitable distribution for Requirement.parse('pythonutils==0.3.0')
If I download the .zip file manually and easy_install the zip file, the install goes great. I think the problem is related to the .py name appearing in the filename.
easy_install http://www.voidspace.org.uk/cgi-bin/voidspace/downman.py?file=pythonutils-0. 3.0.zip Downloading http://www.voidspace.org.uk/cgi-bin/voidspace/downman.py?file=pythonutils-0. 3.0.zip error: Can't process plain .py files without an '#egg=name-version' suffix to enable automatic setup script generation.
I'm guessing that whatever easy_install uses for downloading the file ends up naming it downman.py (or something similar). I checked the headers on the response from voidspace, and I see voidspace is in fact supplying the content-disposition header: content-disposition: attachment; filename= "pythonutils-0.3.0.zip" So it appears that whatever downloader easy_install is using isn't honoring the content-disposition header. or the check for python-ness is otherwise getting a false positive. This problem was observed in Windows Vista 64-bit with Python 2.5.2 32-bit and setuptools 0.6c9. Should I post a bug in bugtracker on this issue? Regards, Jason
At 03:41 PM 12/18/2008 -0700, Jason R. Coombs wrote:
Should I post a bug in bugtracker on this issue?
No. easy_install requires that a URL path ends with the target filename. It *must* be able to determine what it's downloading BEFORE it actually downloads it, since it would otherwise have to download every possible file for a package in order to determine which one was a suitable version, python version, platform, etc. And it's not reasonable to expect it to grow special cases for every site that wants to hide the filename in the query string attached to a CGI script. If someone needs to have their files processed by a script, they could always make the path something like: downman.py/pythonutils-0.2.3.zip Using either PATH_INFO or mod_rewrite or something similar, at which point easy_install can tell it's looking at a source zipfile, rather than a .py script.
Phillip, Thanks for your response.
-----Original Message----- From: Phillip J. Eby [mailto:pje@telecommunity.com] Sent: Thursday, 18 December, 2008 21:59
... And it's not reasonable to expect it to grow special cases for every site that wants to hide the filename in the query string attached to a CGI script.
I absolutely agree it's not reasonable to grow special cases... but the RFCs do provide a standard for describing the content disposition. Perhaps it would be worthwhile to handle that general case. Specifically, one could request the headers for a given URL to determine if a content-disposition filename is supplied. I've put together some sample code which is short, robust, and clean (I hope). This code demonstrates how one might go about querying the content-disposition. See it at: http://paste.turbogears.org/paste/19813 With that code, the filename_test code could go something like this: filename = get_content_disposition_filename(url) or get_filename_the_old_way(url) test_valid_egg_distro(filename) ... or alternatively, one could only perform the content-disposition retrieval when the filename _appears_ to be invalid.
If someone needs to have their files processed by a script, they could always make the path something like:
downman.py/pythonutils-0.2.3.zip
Using either PATH_INFO or mod_rewrite or something similar, at which point easy_install can tell it's looking at a source zipfile, rather than a .py script.
While I agree it's unreasonable for setuptools to have to grow to handle nonstandard cases, I would say the same argument applies to the content providers. They shouldn't have to alter their deployment architecture to accommodate limitations in setuptools, particularly when they're following the web standards. The above solution handles the general case where files can be served through many different content management systems and not just as static file references in the URL. This allows for special urls like http://www.mycompany.com/myproject/latest_distro.jsp, which might generate the content dynamically. Note that the voidspace author is not attempting to "hide" the filename in the get parameters. He's probably using a sort of content management system. Would you consider a patch that incorporates this more robust behavior? Regards, Jason
At 07:41 AM 12/19/2008 -0700, Jason R. Coombs wrote:
Phillip, Thanks for your response.
-----Original Message----- From: Phillip J. Eby [mailto:pje@telecommunity.com] Sent: Thursday, 18 December, 2008 21:59
... And it's not reasonable to expect it to grow special cases for every site that wants to hide the filename in the query string attached to a CGI script.
I absolutely agree it's not reasonable to grow special cases... but the RFCs do provide a standard for describing the content disposition. Perhaps it would be worthwhile to handle that general case. Specifically, one could request the headers for a given URL to determine if a content-disposition filename is supplied.
Please re-read my original message; easy_install can't retrieve every URL in order to determine which URL to retrieve in the first place.
... or alternatively, one could only perform the content-disposition retrieval when the filename _appears_ to be invalid.
...which would mean it would then have to follow every NON-download link on the page.
Would you consider a patch that incorporates this more robust behavior?
If you can come up with a way to do it without retrieving any non-download URLs, and that doesn't need a special case for every kind of CMS, yes. I just don't think that those requirements can be met. Now, for the specific case of the apparent filename not matching the actual filename, it's possible you could do something about that. But it would only work for voidspace and anybody else using a .py file as their wrapping script, because the only reason easy_install's even *considering* that URL a download candidate is the .py extension. Without it, it wouldn't be considered a download URL to start with, let alone actually retrieved. (So, it won't address Java, PHP, or Zope CMS programs without more special cases.)
participants (2)
-
Jason R. Coombs -
Phillip J. Eby