Difference between index type stage and mirror
Hi, I was wondering what the difference between type=mirror and type=stage is. Is it that: - a mirror gets its list of package exclusively from its 'bases' and you can't upload to it - a stage allows you to upload to it and never looks at its list of bases for packages Or is it something else? I'm seeing the following behavior: $ devpi index --create one type=stage bases=root/pypi http://localhost/pypi/root/one: type=stage bases=root/pypi volatile=True uploadtrigger_jenkins=None acl_upload=root $ devpi index --create one type=stage bases=root/pypi http://localhost/pypi/root/one: type=stage bases=root/pypi volatile=True uploadtrigger_jenkins=None acl_upload=root $ devpi list $ devpi index --create two type=mirror bases= http://localhost/pypi/root/two: type=mirror bases= volatile=False Traceback (most recent call last): File "/usr/local/bin/devpi", line 9, in <module> load_entry_point('devpi-common==1.2', 'console_scripts', 'devpi')() File "/usr/local/lib/python2.7/site-packages/devpi/main.py", line 29, in main return method(hub, hub.args) File "/usr/local/lib/python2.7/site-packages/devpi/index.py", line 63, in main return index_create(hub, url, kvdict) File "/usr/local/lib/python2.7/site-packages/devpi/index.py", line 5, in index_create index_show(hub, url) File "/usr/local/lib/python2.7/site-packages/devpi/index.py", line 36, in index_show ixconfig["uploadtrigger_jenkins"],)) KeyError: 'uploadtrigger_jenkins' $ devpi use root/two current devpi index: http://localhost/pypi/root/two/ (logged in as root) ~/.pydistutils.cfg : no config file exists ~/.pip/pip.conf : no config file exists ~/.buildout/default.cfg: no config file exists always-set-cfg: no $ devpi list 0x10c-asm 1009558_nester 131228_pytest_1 17MonIP 18-e 1ee 1pass 1to001 2013007_pyh 2C.py ... [Removed huge list a pacakges presumably from root/pypi] $ devpi use -l root/pypi bases= volatile=False root/two bases= volatile=True root/one bases=root/pypi volatile=True Why didn't 'root/one' list any packages when it had 'root/pypi' as a base. And, why did 'root/two' list a whole bunch of packages when didn't have any bases? Is a 'mirror' always a mirror of the central pypi server? Can I not have 'root/one' and 'root/two' be stage indexes and 'root/three' be a mirror of just those two and not the central public pypi server? And why did I get that 'uploadtrigger_jenkins' error?
Hi Steven, On Tue, Jul 08, 2014 at 13:43 -0700, Steven Willis wrote:
Hi, I was wondering what the difference between type=mirror and type=stage is.
Is it that:
- a mirror gets its list of package exclusively from its 'bases' and you can't upload to it - a stage allows you to upload to it and never looks at its list of bases for packages
Or is it something else? I'm seeing the following behavior:
No, it is something else. A Mirror is a pypi mirror which virtually is serving what https://pypi.python.org/simple/ serves. It's a self-updating cache. You cannot upload packages to the mirror, you have to upload them to the public site. A "stage" is a devpi private stage where you can upload packages privately. Each stage can inherit projects/links from other stages or from a pypi mirror.
$ devpi index --create one type=stage bases=root/pypi http://localhost/pypi/root/one: type=stage bases=root/pypi volatile=True uploadtrigger_jenkins=None acl_upload=root
This is a stage inheriting from the pypi mirror /root/pypi.
$ devpi index --create two type=mirror bases= http://localhost/pypi/root/two: type=mirror bases= volatile=False Traceback (most recent call last): File "/usr/local/bin/devpi", line 9, in <module> load_entry_point('devpi-common==1.2', 'console_scripts', 'devpi')() File "/usr/local/lib/python2.7/site-packages/devpi/main.py", line 29, in main return method(hub, hub.args) File "/usr/local/lib/python2.7/site-packages/devpi/index.py", line 63, in main return index_create(hub, url, kvdict) File "/usr/local/lib/python2.7/site-packages/devpi/index.py", line 5, in index_create index_show(hub, url) File "/usr/local/lib/python2.7/site-packages/devpi/index.py", line 36, in index_show ixconfig["uploadtrigger_jenkins"],)) KeyError: 'uploadtrigger_jenkins'
Yes, you cannot create new mirror stages as there (currently) is only one mirror stage: root/pypi Basically at the moment you don't have to think about anything else then "normal" "stage type" indexes. best holger
$ devpi use root/two current devpi index: http://localhost/pypi/root/two/ (logged in as root) ~/.pydistutils.cfg : no config file exists ~/.pip/pip.conf : no config file exists ~/.buildout/default.cfg: no config file exists always-set-cfg: no $ devpi list 0x10c-asm 1009558_nester 131228_pytest_1 17MonIP 18-e 1ee 1pass 1to001 2013007_pyh 2C.py ... [Removed huge list a pacakges presumably from root/pypi] $ devpi use -l root/pypi bases= volatile=False root/two bases= volatile=True root/one bases=root/pypi volatile=True
Why didn't 'root/one' list any packages when it had 'root/pypi' as a base.
The list command does not show projects from root/pypi because that would mean >45K entries.
And, why did 'root/two' list a whole bunch of packages when didn't have any bases? Is a 'mirror' always a mirror of the central pypi server? Can I not have 'root/one' and 'root/two' be stage indexes and 'root/three' be a mirror of just those two and not the central public pypi server? And why did I get that 'uploadtrigger_jenkins' error?
We should prevent "type=mirror" stage creation. sorry for the confusion. Sometime after the imminent devpi-2.0 release i hope to find the time to prevent bad setting and give proper error messages. best, holger
-- You received this message because you are subscribed to the Google Groups "devpi-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to devpi-dev+...@googlegroups.com. To post to this group, send email to devp...@googlegroups.com. Visit this group at http://groups.google.com/group/devpi-dev. For more options, visit https://groups.google.com/d/optout.
No, it is something else. A Mirror is a pypi mirror which virtually is serving what https://pypi.python.org/simple/ serves. It's a self-updating cache. You cannot upload packages to the mirror, you have to upload them to the public site.
So, should there be exactly one 'mirror' index which is /root/pypi ?
Yes, you cannot create new mirror stages as there (currently) is only one mirror stage: root/pypi
Why didn't 'root/one' list any packages when it had 'root/pypi' as a
What is a 'mirror stage', I'm assuming typo and you mean 'mirror index' ? base.
The list command does not show projects from root/pypi because that would mean >45K entries.
But this shows all 45K entries: $ devpi use /root/pypi $ devpi list | wc -l 45858 However: $ devpi use /root/one $ devpi list | wc -l 0
On Wed, Jul 09, 2014 at 07:30 -0700, Steven Willis wrote:
No, it is something else. A Mirror is a pypi mirror which virtually is serving what https://pypi.python.org/simple/ serves. It's a self-updating cache. You cannot upload packages to the mirror, you have to upload them to the public site.
So, should there be exactly one 'mirror' index which is /root/pypi ?
Yes, you cannot create new mirror stages as there (currently) is only one mirror stage: root/pypi
What is a 'mirror stage', I'm assuming typo and you mean 'mirror index' ?
sorry for sloppyness, yes "mirror index" is probably better.
Why didn't 'root/one' list any packages when it had 'root/pypi' as a base.
The list command does not show projects from root/pypi because that would mean >45K entries.
But this shows all 45K entries:
$ devpi use /root/pypi $ devpi list | wc -l
45858
However: $ devpi use /root/one $ devpi list | wc -l
0
Yes, it's inconcistent. But i don't want to list the 45K everytime you ask "list" on an index that inherits root/pypi. The command should probably say that it omits root/pypi listing to give a hint unless you explicitely ask for it. Note that when you do "devpi list NAME" for listing release files for NAME then all base indexes will be considered, including root/pypi, because there is no 45k-style explosion of entries from it. best, holger
-- You received this message because you are subscribed to the Google Groups "devpi-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to devpi-dev+...@googlegroups.com. To post to this group, send email to devp...@googlegroups.com. Visit this group at http://groups.google.com/group/devpi-dev. For more options, visit https://groups.google.com/d/optout.
participants (2)
-
holger krekel -
Steven Willis