[Distutils] [ANN] pypiserver 1.1.1 - minimal private pypi server

[holger krekel]

On Sat, Jun 01, 2013 at 23:26 +0200, Ralf Schmitt wrote:
> holger krekel <holger at merlinux.eu> writes:
> 
> >> How does it compare to http://pypi.python.org/pypi/devpi-server?
> >
> > the main differences as i see them (note i am the devpi-server author, though):
> >
> > - pypiserver supports uploading to private indexes, 
> >   devpi-server not yet (but next week / on trunk already :)
> 
> I still consider that a bug. scp works much better. I'm not sure if
> devpi-server maintains some kind of database, but I consider it a big
> plus to be able to just copy files and directories to the package
> directory and serve them instantly.

(devpi-server maintains a database, indeed).

If you scp a large file and someone else is about the pip-install that
package, could you get partial downloads which fail on the user side?

I like the ability of transfering things via scp but i guess in devpi-server
i'd make them go to a "clearing area" where they are screened before put
up for serving.

> ...
> > - pypiserver redirects the lookup of pypi packages to pypi.python.org,
> 
> that's related to a weak point of pypiserver: you currently have to
> parse the log files and look for HTTP redirects if you like to know what
> packages are missing in the local package directory.

It's a recursive problem because those packages might have pulled in
yet more dependencies.

> you also can't install a specific version of a package if the package
> directory doesn't have that specific version but a different version.
> this may or may not be good depending on your use case.
> 
> redirects can also be completely disabled, so pip/easy_install only see
> the packages in the package directory.

FYI devpi-server supports multiple indexes and you can define inheritance 
semantics between indexes.  If an index A inherits from another index B
(which can be the full pypi-mirroring index) then queries on A will
see all release files from index B merged in.  By default, there also
is a "root/prod" index which does not inherit anything and contains
only the packages explicitely pushed to it.  If you install from such
a no-bases index, only its own list of release files are considered.

This is designed such that users can start off with "play areas" to prepare
and test their packages including third party ones before "staging" them 
to a more constrained no-bases index.

> >   devpi-server caches them and serves everything (including #egg-links)
> >   through itself, allowing complete offline operations (including
> >   caching/serving of 3rd party site's packages) using the 
> >   extended PEP381 mirror protocol
> >
> > - pypiserver has a good and simple implementation, devpi-server is
> >   little more complex mostly due to its caching/crawling logic.
> >
> > - both are very well tested and maintained but pypiserver is out there
> >   for a longer time already, so has seen more RL-testing.
> >
> > Ralph, please add/comment as you see fit.
> 
> pypiserver exhibits it's WSGI application and might be easier to deploy
> using different WSGI servers.

In principle, A wsgi-app is also available with devpi-server but there is
one issue i am unsure about: the wsgi-app needs to have async threads/greenlets
running to keep the pypi-mirror cache up-to-date.  Starting those threads
at wsgi-app creation seems too early so it currently starts them before
bottle.run() is invoked.  I guess it could also happen at "first-request"
time or so which should make the creation of the WSGI-app become a
no-sideeffect operation.

cheers,
holger


> -- 
> Cheers
> Ralf
>