We're revamping the web interface to match the new pdo style. We've also cleaned a bit of the interface up.
Phillip J. Eby <pje(a)telecommunity.com> wrote:
> 1. It assumes that baseURL/projectname will get to the current version
> of projectname, or a page with a list of projectname's active versions
> 2. It assumes that links within PyPI of the form
> baseURL/something1/something2 are links to version 'something2' of a
> project named 'something1'
> 4. It assumes that if baseURL/projectname returns a page containing the
> text "Index of Packages</title>", it is a list of links of the form
> described in #2.
> 5. It looks for and follows the first links following the strings
> "<th>Home Page" and "<th>Download URL" in a project page.
These remain unchanged.
> 3. It assumes that going to baseURL directly will result in a page with
> links to all available packages in the form described in #2.
This has been removed as it seems completely unnecessary (a flat listing of all 1400+ packages, that is). The XML-RPC interface provides the functionality you require here.
> Also note that even with an XML-RPC interface, easy_install will *still*
> need to read an HTML page to gather links, because it's valid for people
> to provide links in their long_description using reStructuredText. It's
> just that assumptions 1, 3, and 4 (and maybe 5) would not be necessary.
You couldn't just call release_data and parse URLs out of the description text with a simple re search?
> Hm. I just tried to make multiple versions of PEAK active, and it seems
> like you can't get the page that lists multiple versions any more. No
> wonder some people have been having problems downloading older versions
> of certain packages. :(
Whoopsie. This has been fixed. Not sure when it was changed or why.
> What do you mean? You can run "easy_install -u setuptools" to upgrade
> to the latest release at any time. But it doesn't go out looking for
> updates on its own.
Automatically updating existing users to use the xml-rpc would be nice, I suppose.
Phillip J. Eby <pje(a)telecommunity.com> wrote:
> Patches welcome. :) Note that there should still be a fallback to the
> screen scraping code in case of a problem with the XML-RPC, to allow
> to continue using static mirrors of PyPI or imitation PyPIs without
> to support XML-RPC.
I'm trying to install picard (http://musicbrainz.org/wd/PicardDownload) with
python -c "import setuptools;execfile('setup.py')" install --single-version-externally-managed --root=/path/to/root
but it breaks with:
usage: -c [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: -c --help [cmd1 cmd2 ...]
or: -c --help-commands
or: -c cmd --help
error: option --single-version-externally-managed not recognized
Is that ok and i should use ./setup.py without "import setuptools;execfile('setup.py')" hack, or if this a bug in setuptools that should be fixed?
PS: I use setuptools 0.6b3
At 04:04 AM 7/5/2006 -0400, Jim Fulton wrote:
>On Jun 23, 2006, at 4:51 PM, Jim Fulton wrote:
>>That's a lot of screen scraping. :)
>>It would be good to capture this as part of the documentation IMO
>>>I'm considering adding XML-RPC support to easy_install in 0.7,
>>>though. PyPI now has a nice XML-RPC API that is more responsive
>>>than the web UI, and it supports case-insensitive partial match
>>>searches, making it suitable for easy_install to query when a typed-
>>>in name doesn't exactly match the spelling of a PyPI entry.
>>I think that would be much better.
>I just wanted to emphasize that I think this would be a good
Patches welcome. :) Note that there should still be a fallback to the
screen scraping code in case of a problem with the XML-RPC, to allow people
to continue using static mirrors of PyPI or imitation PyPIs without needing
to support XML-RPC.
> I was just talking to Richard, and he pointed out that the
>current approach is a problem for him, because it means he can't
>evolve the pypi UI without risking breaking setuptools.
What I would suggest is creating a "microformat" for marking up web pages
with sniffable information. For example, adding rel="homepage" and
rel="download" to the links that go to those URLs.
In other words, invisible hints on the page to supplement the visible
information. Then, I could change easy_install to start using the
invisible hints, and drop the visible ones, freeing PyPI to evolve the UI
While the XML-RPC API would be great, I still want easy_install to be able
to use a package index that's made from static files, and that requires
some kind of screen scraping. So, let's make it invisible scraping of a
documented format, so that anybody can use it, with whatever visual formats
Currently, easy_install gets most of its information from URLs; the only
actual scraping of visible data is of the title, the download MD5's, and
the table cells that identify links as being to the home page or download
URL (since it needs to specifically identify these in order to spider them).
The MD5 information dependency could be removed if PyPI included "#md5=..."
at the end of the download URLs; easy_install can see that information and
use it. The table cell checking could be removed by adding
'rel="easy_install"' or something like that to the spiderable links.
The title checking is used to distinguish pages that list multiple packages
from pages that list single packages. I don't have any ready ideas as to
how that could or should be represented in a semantic (as opposed to
visual) way. Your thoughts?
At 04:21 PM 6/29/2006 +0100, Paul Moore wrote:
> File "D:\Apps\Python25\Lib\msilib\__init__.py", line 115, in add_data
> raise MSIError("Could not insert "+repr(values)+" into "+table)
>_msi.MSIError: Could not insert [(None, 'site_packages',
>'TEST-1~1.EGG|test-1.0-py2.5.egg-info', 186L, None, None, 512, 1)]
This line is masking whatever the actual error is. If you look at the
source, you'll see it's doing "except Exception,e:" and trapping whatever
the real problem is. I'd suggest commenting out the try/except and see
what error comes up instead. That should tell us more about the problem.
I believe Martin v. Loewis is the author of bdist_msi; I'm not sure if he
reads this list or not. Perhaps he will comment.
At 04:13 PM 6/29/2006 +0100, Paul Moore wrote:
>Agreed. But in the absence of a standard, supporting package authors'
>existing approaches, which work with other distutils mechanisms, seems
>like a reasonable requirement.
Anything that the package author installs as package data, or using the
data_files option to setup(), is included in the egg. And if you install
unzipped, you should be able to browse the included docs.