[ANN] pypiserver 1.1.1 - minimal private pypi server

Hi, I've just uploaded pypiserver 1.1.1 to the python package index. pypiserver is a minimal PyPI compatible server. It can be used to serve a set of packages and eggs to easy_install or pip. pypiserver is easy to install (i.e. just 'pip install pypiserver'). It doesn't have any external dependencies. https://pypi.python.org/pypi/pypiserver/ should contain enough information to easily get you started running your own PyPI server in a few minutes. The code is available on github: https://github.com/schmir/pypiserver Changes in this version ----------------------- - add 'overwrite' option to allow overwriting existing package files (default: false) - show names with hyphens instead of underscores on the "/simple" listing - make the standalone version work with jython 2.5.3 - upgrade waitress to 0.8.5 in the standalone version - workaround broken xmlrpc api on pypi.python.org by using HTTPS -- Cheers Ralf

Hello Ralf, I already started working on your code and using it a s a base for my pypiserver project. Thanks again, great piece of minimal work! Jonas. On 29 May 2013, at 01:32, Ralf Schmitt wrote:
Hi,
I've just uploaded pypiserver 1.1.1 to the python package index.
pypiserver is a minimal PyPI compatible server. It can be used to serve a set of packages and eggs to easy_install or pip.
pypiserver is easy to install (i.e. just 'pip install pypiserver'). It doesn't have any external dependencies.
https://pypi.python.org/pypi/pypiserver/ should contain enough information to easily get you started running your own PyPI server in a few minutes.
The code is available on github: https://github.com/schmir/pypiserver
Changes in this version ----------------------- - add 'overwrite' option to allow overwriting existing package files (default: false) - show names with hyphens instead of underscores on the "/simple" listing - make the standalone version work with jython 2.5.3 - upgrade waitress to 0.8.5 in the standalone version - workaround broken xmlrpc api on pypi.python.org by using HTTPS
-- Cheers Ralf _______________________________________________ Distutils-SIG maillist - Distutils-SIG@python.org http://mail.python.org/mailman/listinfo/distutils-sig

Jonas Geiregat <jonas@geiregat.org> writes:
Hello Ralf,
I already started working on your code and using it a s a base for my pypiserver project.
Thanks again, great piece of minimal work!
Thanks! What features do you plan to implement? Maybe some of that functionality is already available in other implementations. http://bitofcheese.blogspot.de/2013/05/local-pypi-options.html contains a nice rundown of alternative implementations. I've tested proxypypi and it's quite nice if you need a caching proxy. I assume devpi-server is also quite good given the fact that it's author produces really nice work. -- Cheers Ralf

On 29 May 2013, at 10:33, Ralf Schmitt wrote:
Jonas Geiregat <jonas@geiregat.org> writes:
Hello Ralf,
I already started working on your code and using it a s a base for my pypiserver project.
Thanks again, great piece of minimal work!
Thanks! What features do you plan to implement?
I will be using your code to transform it into a Nexus alike system. First I added jinja2 and sqlalchemy , that's almost finished. Next I want to add a decent web interface, probably based on bootstrap.

On 29/05/2013 00:32, Ralf Schmitt wrote:
I've just uploaded pypiserver 1.1.1 to the python package index.
pypiserver is a minimal PyPI compatible server. It can be used to serve a set of packages and eggs to easy_install or pip.
pypiserver is easy to install (i.e. just 'pip install pypiserver'). It doesn't have any external dependencies.
https://pypi.python.org/pypi/pypiserver/ should contain enough information to easily get you started running your own PyPI server in a
Looks very interesting! How does it compare to http://pypi.python.org/pypi/devpi-server? cheers, Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk

On Wed, May 29, 2013 at 16:40 +0100, Chris Withers wrote:
On 29/05/2013 00:32, Ralf Schmitt wrote:
I've just uploaded pypiserver 1.1.1 to the python package index.
pypiserver is a minimal PyPI compatible server. It can be used to serve a set of packages and eggs to easy_install or pip.
pypiserver is easy to install (i.e. just 'pip install pypiserver'). It doesn't have any external dependencies.
https://pypi.python.org/pypi/pypiserver/ should contain enough information to easily get you started running your own PyPI server in a
Looks very interesting!
How does it compare to http://pypi.python.org/pypi/devpi-server?
the main differences as i see them (note i am the devpi-server author, though): - pypiserver supports uploading to private indexes, devpi-server not yet (but next week / on trunk already :) - pypiserver has no dependencies (ships bottle inline), devpi-server depends on redis (to be dropped next week for no-dep fs-storage) and bottle, requests, py, all proven libraries. - pypiserver redirects the lookup of pypi packages to pypi.python.org, devpi-server caches them and serves everything (including #egg-links) through itself, allowing complete offline operations (including caching/serving of 3rd party site's packages) using the extended PEP381 mirror protocol - pypiserver has a good and simple implementation, devpi-server is little more complex mostly due to its caching/crawling logic. - both are very well tested and maintained but pypiserver is out there for a longer time already, so has seen more RL-testing. Ralph, please add/comment as you see fit. best, holger
cheers,
Chris
-- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk

holger krekel <holger@merlinux.eu> writes:
How does it compare to http://pypi.python.org/pypi/devpi-server?
the main differences as i see them (note i am the devpi-server author, though):
- pypiserver supports uploading to private indexes, devpi-server not yet (but next week / on trunk already :)
I still consider that a bug. scp works much better. I'm not sure if devpi-server maintains some kind of database, but I consider it a big plus to be able to just copy files and directories to the package directory and serve them instantly.
- pypiserver has no dependencies (ships bottle inline), devpi-server depends on redis (to be dropped next week for no-dep fs-storage) and bottle, requests, py, all proven libraries.
nice, I'll try it if the redis dependency is gone.
- pypiserver redirects the lookup of pypi packages to pypi.python.org,
that's related to a weak point of pypiserver: you currently have to parse the log files and look for HTTP redirects if you like to know what packages are missing in the local package directory. you also can't install a specific version of a package if the package directory doesn't have that specific version but a different version. this may or may not be good depending on your use case. redirects can also be completely disabled, so pip/easy_install only see the packages in the package directory.
devpi-server caches them and serves everything (including #egg-links) through itself, allowing complete offline operations (including caching/serving of 3rd party site's packages) using the extended PEP381 mirror protocol
- pypiserver has a good and simple implementation, devpi-server is little more complex mostly due to its caching/crawling logic.
- both are very well tested and maintained but pypiserver is out there for a longer time already, so has seen more RL-testing.
Ralph, please add/comment as you see fit.
pypiserver exhibits it's WSGI application and might be easier to deploy using different WSGI servers. -- Cheers Ralf

On Sat, Jun 01, 2013 at 23:26 +0200, Ralf Schmitt wrote:
holger krekel <holger@merlinux.eu> writes:
How does it compare to http://pypi.python.org/pypi/devpi-server?
the main differences as i see them (note i am the devpi-server author, though):
- pypiserver supports uploading to private indexes, devpi-server not yet (but next week / on trunk already :)
I still consider that a bug. scp works much better. I'm not sure if devpi-server maintains some kind of database, but I consider it a big plus to be able to just copy files and directories to the package directory and serve them instantly.
(devpi-server maintains a database, indeed). If you scp a large file and someone else is about the pip-install that package, could you get partial downloads which fail on the user side? I like the ability of transfering things via scp but i guess in devpi-server i'd make them go to a "clearing area" where they are screened before put up for serving.
...
- pypiserver redirects the lookup of pypi packages to pypi.python.org,
that's related to a weak point of pypiserver: you currently have to parse the log files and look for HTTP redirects if you like to know what packages are missing in the local package directory.
It's a recursive problem because those packages might have pulled in yet more dependencies.
you also can't install a specific version of a package if the package directory doesn't have that specific version but a different version. this may or may not be good depending on your use case.
redirects can also be completely disabled, so pip/easy_install only see the packages in the package directory.
FYI devpi-server supports multiple indexes and you can define inheritance semantics between indexes. If an index A inherits from another index B (which can be the full pypi-mirroring index) then queries on A will see all release files from index B merged in. By default, there also is a "root/prod" index which does not inherit anything and contains only the packages explicitely pushed to it. If you install from such a no-bases index, only its own list of release files are considered. This is designed such that users can start off with "play areas" to prepare and test their packages including third party ones before "staging" them to a more constrained no-bases index.
devpi-server caches them and serves everything (including #egg-links) through itself, allowing complete offline operations (including caching/serving of 3rd party site's packages) using the extended PEP381 mirror protocol
- pypiserver has a good and simple implementation, devpi-server is little more complex mostly due to its caching/crawling logic.
- both are very well tested and maintained but pypiserver is out there for a longer time already, so has seen more RL-testing.
Ralph, please add/comment as you see fit.
pypiserver exhibits it's WSGI application and might be easier to deploy using different WSGI servers.
In principle, A wsgi-app is also available with devpi-server but there is one issue i am unsure about: the wsgi-app needs to have async threads/greenlets running to keep the pypi-mirror cache up-to-date. Starting those threads at wsgi-app creation seems too early so it currently starts them before bottle.run() is invoked. I guess it could also happen at "first-request" time or so which should make the creation of the WSGI-app become a no-sideeffect operation. cheers, holger
-- Cheers Ralf
participants (4)
-
Chris Withers
-
holger krekel
-
Jonas Geiregat
-
Ralf Schmitt