At 08:47 PM 5/30/2005 -0500, Ian Bicking wrote:
Getting the data without screenscraping won't instantly give us all the necessary information. But it does contain good information about available versions, what the active version is, and per-version download URLs (which, if nothing else, could be compared against each other to detect non-version-specific URLs).
Right; but none of that helps with the real problem (from EasyInstall's perspective), which is that the current incarnation of PyPI doesn't list multiple download URLs for a single release of a specific package. For example, when I release PEAK or PyProtocols, I've been releasing sdists (in two formats) plus a bdist_wininst -- and in the future I'll probably drop the bdist_wininst in favor of eggs. But I can't put any of that info on PyPI, so I just link to my downloads directory - as do 25% of the packages I surveyed in a random sampling last week.
In order to get at packages like those, a flexible screen scraper is a must. I agree that PyPI should have better handling of download URLs, but I'm in a lot better position to improve EasyInstall than PyPI.
Hm. You know, what if as an interim step we had the command-line tool just launch a webbrowser pointing you to PyPI? Getting to a page for a suitable version is easy, so we could then let the user find the right download URL and then go back to paste it on the command line. That could be a nice interim addition, although it isn't much of a solution for packages with a lot of un-installed dependencies. You'd keep getting kicked back to the web browser a lot, and more to the point you'd have to keep restarting the tool. So, ultimately we really need a way to actually find the URLs.
That's not a very satisfying experience -- the person might as well just download the file at that point.
Tastes differ, I suppose. I'd just right-click the link to copy it, and then alt-tab, ^R, space, ^K, space, shift-insert, ENTER. But then, I've been downloading a lot of packages this weekend, so that sequence is already in my muscle memory. :)
Hm. Maybe somebody could create a Firefox extension that runs EasyInstall on a selected link. :)
Even with accurate data from PyPI, it's still likely there will be multiple possible URLs. At that point, at least if you are going through the command line, displaying all the URLs (numbered) and asking the user would probably give the user enough information to choose.
In which case you might as well be back in the web browser. I'm fine with options being available to fine-tune the selection process, but the criteria can and should be mechanically processed.
After all unusable versions, platforms, and archive types are eliminated, the prioritization should be in descending version order, with same version archives sorted by archive type, eggs first, everything else second. (Since the eggs don't need to be built.)
By the way, in all of this there's been no discussion about MD5 signatures or code signing. That's probably because I don't know a whole lot about that subject. :) But I'm certainly interested in hearing from those who do.