[Distutils] The sad and insecure state of commercial private package indexes

[Robert Collins]

On 24 April 2017 at 17:10, Nick Coghlan <ncoghlan at gmail.com> wrote:
> On 22 April 2017 at 21:05, Donald Stufft <donald at stufft.io> wrote:
>> I think the biggest barrier to doing it in pip is simply the UX of it. We’re
>> currently constrained by the fact that *all* of our options are available as
>> CLI flags, environment variables, and of course, a config file. This works
>> great for simple key, value configuration but it breaks down with more
>> complex situations like trying to assign a priority to different
>> repositories or selecting which repository a particular package *should*
>> come from (and other more complex situations).
>>
>> Thus far we’ve more or less stuck our fingers in our ears and focused on
>> other problems, but I think we’re going to end up needing to refactor the
>> way pip handles configuration to really make this sort of thing sane.
>
> As much as it annoys me in other ways, the `/etc/yum.repos.d/*`
> approach to managing named repositories seems hard to beat in terms of
> allowing for per-repo configuration settings while limiting command
> line complexity (since the command line mainly deals with repo names
> rather than the individual settings). Debian offers something similar
> now in the form of `/etc/apt/sources.list.d/`.
>
> It doesn't automatically solve the complexity problem around pulling
> components from multiple repositories, but it helps makes the
> complexity more manageable since:
>
> - remote repositories are explicitly modeled as named entities with
> various configurable characteristics
> - other commands then work with the local entity names rather than
> directly with remote URLs
>
> Something I *don't* think either yum/dnf or apt handle well is the
> notion of *relative* priorities for a single command, since they're
> built around the notion of associating static numeric priorities with
> the repository definitions, whereas for development purposes, you
> usually want to express relative priorities like:
>
>     --indices=project-specific,org-common,pypi # Continuous deployment
>     --indices=dev,staging,stable,pypi # Phased deployment, dev
>     --indices=staging,stable # Phased deployment, CI/staging
>     --indices=stable # Phased deployment, production

Well apt and similar tools are working in a system global context. So
pipeline stage based rules are a poor fit for the use of system
packages. Of course if you use a system-in-a-container approach you
can have both worlds, which a lot of folk do.

That said apt pinning is pretty capable and can pick arbitrary
packages in arbitrary order across arbitrary repositories.

-Rob