-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1

Reinout van Rees wrote:

Hi all,

I'd like to do a bit of experimenting with a custom index with only the right packages in just the right versions. Some possible advantages I'm hoping to see for my usecase:

• Isolation from one of various servers being down (pypi, sourceforge).

• Guaranteed versions even when not using buildout. (Stuff needs "python

setup.py install" in the system python, apparently, for arcgis/arcview on windows to be able to use it).

Personally, I'd just let everyone use buildout, but I think I won't be able to get that working with that "install it in site-packages" requirement. (Ideas welcome, of course).

Question: how to maintain such a custom index?

iirc chrism has a script to build an apache-served folder structure from a bunch of eggs/tarballs, so just having the tarballs is enough. Probably just putting those tarballs in one location is enough anyway.

You need the 'simple' bookkeeping files: the attached script builds them from that directory full of tarballs. Run it via:

$ cd /path/to/directory/full/of/tarballs $ /path/to/python /path/to/makeindex.py *.{gz,tgz,zip,egg} # move aside previous 'simple' first $ mv index simple

But how to get the right collection of tarballs in there? Is that a by-hand approach? Can you automate it?

You might look at compoze: it is designed to allow fetching sources and building an index based on the current workingset:

http://svn.repoze.org/compoze/trunk/

Question 2: I thought you could only have one index. So installing something extra from pypi seems out. How do people handle that?

Fetch it from pypi and put it in your index.

I'm totally used to buildout and its [versions] list and buildout.dumppickedversions and KGS lists, so I need to reset my thinking on this a little. I need some input :-)

Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com

At 09:44 PM 2/2/2010 +0100, Reinout van Rees wrote:

Question: how to maintain such a custom index?

Create an Apache-served folder structure such that for each project you want to serve, indexroot/project/ is a directory listing of downloadables. Alternately, make a indexroot/project/index.html for that project, that that links directly to your download targets (which you need not directly serve).

But how to get the right collection of tarballs in there? Is that a by-hand approach? Can you automate it?

Presumably, if you have something particular in mind, you could write a script to do it. Perhaps you could even use buildout to do it in some fashion. ;-)

Question 2: I thought you could only have one index. So installing something extra from pypi seems out.

You can always add a mod_rewrite rule in your index root, that checks for the nonexistence of a projectname, and then automatically redirects to the PyPI index for that page. easy_install will not be confused by this - it will treat the PyPI page as if it had been served by your index. (But of course it will correctly interpret any relative links found as relative to the PyPI page, rather than as relative to the URL it tried to fetch from your index. If you don't understand any of this, don't worry... it Just Works.)

On 02/02/2010 10:25 PM, Tres Seaver wrote:

...

But how to get the right collection of tarballs in there? Is that a by-hand approach? Can you automate it?

You might look at compoze: it is designed to allow fetching sources and building an index based on the current workingset:

http://svn.repoze.org/compoze/trunk/

Thanks, I'll try that one out. He he, a README in structuredtext (not reST) format if I'm not mistaken. I still like that format :-)

...

Question 2: I thought you could only have one index. So installing something extra from pypi seems out. How do people handle that?

Fetch it from pypi and put it in your index.

My fear is that I end up with quite a lot of project-specific custom indexes if I want to keep it rebuildable. Multiply that with a quite a number of project releases (so: new indexes) and you're looking at a big bucket of bits.

How's that working out in practice for you?

If you have a base "product/framework/whatever" (a bunch of libraries, partly your own and partly from pypi), I can see it is useful to have that in such a custom index. But multiple in-house websites will use that *and* add their own extra needed libraries to the mix.

=> So you need to add a new index for that, duplicating the "product/framework/whatever" release and adding your own extra eggs?

Reinout

On 2010-02-02, at 1:38 PM, Carl Meyer wrote:

...

Question 2: I thought you could only have one index. So installing something extra from pypi seems out. How do people handle that?

I use pip (and pip requirements files) for installation, and pip has both the -i/--index-url option (for replacing PyPI entirely) and the

• --extra-index-url option (for layering another index in addition to

PyPI). I don't know how one would do that with other tools.

You can pass multiple `-i' arguments to pip.

$ pip -i http://internal/packages/ -i http://pypi.python.org/simple install internalpackage

Not sure what the `--extra-index-url' is for, then.

-srid