Mailman 3 July 2006 - Distutils-SIG

Re: [Distutils] [Catalog-sig] Specification for package indexes?
by richardjones＠optusnet.com.au July 7, 2006

July 7, 2006

We're revamping the web interface to match the new pdo style. We've also cleaned a bit of the interface up. Phillip J. Eby <pje(a)telecommunity.com> wrote: > 1. It assumes that baseURL/projectname will get to the current version > of projectname, or a page with a list of projectname's active versions > > 2. It assumes that links within PyPI of the form > baseURL/something1/something2 are links to version 'something2' of a > project named 'something1' > > 4. It assumes that if baseURL/projectname returns a page containing the > text "Index of Packages</title>", it is a list of links of the form > described in #2. > > 5. It looks for and follows the first links following the strings > "<th>Home Page" and "<th>Download URL" in a project page. These remain unchanged. > 3. It assumes that going to baseURL directly will result in a page with > links to all available packages in the form described in #2. This has been removed as it seems completely unnecessary (a flat listing of all 1400+ packages, that is). The XML-RPC interface provides the functionality you require here. > Also note that even with an XML-RPC interface, easy_install will *still* > need to read an HTML page to gather links, because it's valid for people > to provide links in their long_description using reStructuredText. It's > just that assumptions 1, 3, and 4 (and maybe 5) would not be necessary. You couldn't just call release_data and parse URLs out of the description text with a simple re search? > Hm. I just tried to make multiple versions of PEAK active, and it seems > like you can't get the page that lists multiple versions any more. No > wonder some people have been having problems downloading older versions > of certain packages. :( Whoopsie. This has been fixed. Not sure when it was changed or why. > What do you mean? You can run "easy_install -u setuptools" to upgrade > to the latest release at any time. But it doesn't go out looking for > updates on its own. Automatically updating existing users to use the xml-rpc would be nice, I suppose. Richard

2 1

f2py and setuptools
by Flavio Codeco Coelho July 6, 2006

July 6, 2006

Hi, is there a way to handle the compilation of f2py extension modules from setup tools? Or do I have to chose between numpy.distutils.core and setuptools? thanks Flávio

3 3

Re: [Distutils] [Catalog-sig] Specification for package indexes?
by richardjones＠optusnet.com.au July 6, 2006

July 6, 2006

Phillip J. Eby <pje(a)telecommunity.com> wrote: > Patches welcome. :) Note that there should still be a fallback to the > screen scraping code in case of a problem with the XML-RPC, to allow > people > to continue using static mirrors of PyPI or imitation PyPIs without > needing > to support XML-RPC. Why? Richard

3 2

setuptools and picard
by Pavel Volkovitskiy July 6, 2006

July 6, 2006

Hello I'm trying to install picard (http://musicbrainz.org/wd/PicardDownload) with python -c "import setuptools;execfile('setup.py')" install --single-version-externally-managed --root=/path/to/root but it breaks with: usage: -c [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...] or: -c --help [cmd1 cmd2 ...] or: -c --help-commands or: -c cmd --help error: option --single-version-externally-managed not recognized Is that ok and i should use ./setup.py without "import setuptools;execfile('setup.py')" hack, or if this a bug in setuptools that should be fixed? PS: I use setuptools 0.6b3 -- Pavel

2 3

Re: [Distutils] Specification for package indexes?
by Phillip J. Eby July 5, 2006

July 5, 2006

At 04:04 AM 7/5/2006 -0400, Jim Fulton wrote: >On Jun 23, 2006, at 4:51 PM, Jim Fulton wrote: >... >> >>That's a lot of screen scraping. :) >> >>It would be good to capture this as part of the documentation IMO >> >>>I'm considering adding XML-RPC support to easy_install in 0.7, >>>though. PyPI now has a nice XML-RPC API that is more responsive >>>than the web UI, and it supports case-insensitive partial match >>>searches, making it suitable for easy_install to query when a typed- >>>in name doesn't exactly match the spelling of a PyPI entry. >> >>I think that would be much better. > >I just wanted to emphasize that I think this would be a good >idea. Patches welcome. :) Note that there should still be a fallback to the screen scraping code in case of a problem with the XML-RPC, to allow people to continue using static mirrors of PyPI or imitation PyPIs without needing to support XML-RPC. > I was just talking to Richard, and he pointed out that the >current approach is a problem for him, because it means he can't >evolve the pypi UI without risking breaking setuptools. What I would suggest is creating a "microformat" for marking up web pages with sniffable information. For example, adding rel="homepage" and rel="download" to the links that go to those URLs. In other words, invisible hints on the page to supplement the visible information. Then, I could change easy_install to start using the invisible hints, and drop the visible ones, freeing PyPI to evolve the UI again. While the XML-RPC API would be great, I still want easy_install to be able to use a package index that's made from static files, and that requires some kind of screen scraping. So, let's make it invisible scraping of a documented format, so that anybody can use it, with whatever visual formats they like. Currently, easy_install gets most of its information from URLs; the only actual scraping of visible data is of the title, the download MD5's, and the table cells that identify links as being to the home page or download URL (since it needs to specifically identify these in order to spider them). The MD5 information dependency could be removed if PyPI included "#md5=..." at the end of the download URLs; easy_install can see that information and use it. The table cell checking could be removed by adding 'rel="easy_install"' or something like that to the spiderable links. The title checking is used to distinguish pages that list multiple packages from pages that list single packages. I don't have any ready ideas as to how that could or should be represented in a semantic (as opposed to visual) way. Your thoughts?

1 0

Re: [Distutils] Problem between bdist_msi and egg-info (Python 2.5b1)
by Phillip J. Eby July 2, 2006

July 2, 2006

At 04:21 PM 6/29/2006 +0100, Paul Moore wrote: > File "D:\Apps\Python25\Lib\msilib\__init__.py", line 115, in add_data > raise MSIError("Could not insert "+repr(values)+" into "+table) >_msi.MSIError: Could not insert [(None, 'site_packages', >'TEST-1~1.EGG|test-1.0-py2.5.egg-info', 186L, None, None, 512, 1)] >into File This line is masking whatever the actual error is. If you look at the source, you'll see it's doing "except Exception,e:" and trapping whatever the real problem is. I'd suggest commenting out the try/except and see what error comes up instead. That should tell us more about the problem. I believe Martin v. Loewis is the author of bdist_msi; I'm not sure if he reads this list or not. Perhaps he will comment.

2 2

Re: [Distutils] Best practices for creating eggs?
by Phillip J. Eby July 2, 2006

July 2, 2006

At 04:13 PM 6/29/2006 +0100, Paul Moore wrote: >Agreed. But in the absence of a standard, supporting package authors' >existing approaches, which work with other distutils mechanisms, seems >like a reasonable requirement. Anything that the package author installs as package data, or using the data_files option to setup(), is included in the egg. And if you install unzipped, you should be able to browse the included docs.

3 2