Looking for guidance on how to organize multiple devpi indices
I am curious how people are using (or not using) devpi's capability of having multiple indices.
When initially setting up devpi, I saw this cool new feature that was discussed and length in the docs and I felt compelled to use it. I set up 6 indexes to divide stuff up by OS and by company-internal vs. public.
Now I wonder if we are getting much from value from this separation or if is complicating things without adding much value.
I have different indices for different OSs -- e.g.:
$ devpi index -l wheels/pure_python wheels/ubuntu_12.04 wheels/ubuntu_14.04
The latter two inherit from the first.
The reasoning for this is that we have multiple platforms use at our company and wheels don't really have enough metadata in their filename to express what Linux and distro they were built for, so I need to maintain separate indexes.
The other distinction I have is that all of these indices are for third-party stuff from PyPI and I have a separate index that has company-internal stuff It seemed useful to keep 3rd party and internal stuff separate, because the 3rd party is more critical to keep safe.
How are other folks using indexes?
Marc
See https://github.com/jhermann/devpi-enterprisey#example-for-an-index-per-team-... for what we're doing.
The aspect of wheels and binary distribution is solved by not having wheels – packages for production use (which includes tooling in the dev stage) are DEBs and go into Artifactory. See https://github.com/jhermann/devpi-enterprisey/tree/master/debianized-devpi for a show-case of a piece of software you might know. ;)
On Wed, Sep 10, 2014 at 13:37 -0700, Marc Abramowitz wrote:
I am curious how people are using (or not using) devpi's capability of having multiple indices.
When initially setting up devpi, I saw this cool new feature that was discussed and length in the docs and I felt compelled to use it. I set up 6 indexes to divide stuff up by OS and by company-internal vs. public.
Now I wonder if we are getting much from value from this separation or if is complicating things without adding much value.
I have different indices for different OSs -- e.g.:
$ devpi index -l wheels/pure_python wheels/ubuntu_12.04 wheels/ubuntu_14.04
The latter two inherit from the first.
The reasoning for this is that we have multiple platforms use at our company and wheels don't really have enough metadata in their filename to express what Linux and distro they were built for, so I need to maintain separate indexes.
ASFAIK using a two-index setup (platform specific inherits from platform neutral) is a reasonable way to deal with the wheel/filename issue.
FWIW I have a plan for a PEP that would enhance the wheel platform/filename issue and would avoid the need for this two index setup. Don't hold your breath, though.
The other distinction I have is that all of these indices are for third-party stuff from PyPI and I have a separate index that has company-internal stuff It seemed useful to keep 3rd party and internal stuff separate, because the 3rd party is more critical to keep safe.
Also a valid use case. I talked a bit about this at EP2014 in Berlin a few weeks ago:
http://youtu.be/s_YjaODzM1E?t=4m13s
How are other folks using indexes?
I recommend to keep the number of indexes minimal and especially don't use inherited indexes unless they provide more convenience (that's also true for programming in general IMO).
As to myself personally and for preparing my open source releases i typically have a "hpk/dev" index which inherits from root/pypi and upload releases there and run "devpi test" against the projects i want to release.
best, holger
Marc
-- You received this message because you are subscribed to the Google Groups "devpi-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to devpi-dev+...@googlegroups.com. To post to this group, send email to devp...@googlegroups.com. Visit this group at http://groups.google.com/group/devpi-dev. For more options, visit https://groups.google.com/d/optout.
participants (3)
-
holger krekel -
Jürgen Hermann -
Marc Abramowitz