[Distutils] [proposal] module dependency specification and overriding for packages

[Ronny Pfannschmidt]

Hi,


On 10.09.2016 12:37, Nick Coghlan wrote:
> On 9 September 2016 at 16:46, Ronny Pfannschmidt
> <opensource at ronnypfannschmidt.de> wrote:
>> Hi,
>>
>> while trying to vendor packages that depend on six, i noticed a problematic
>> implication
>>
>> it is impossible to do this without patching those modules, as such adding a
>> fragile maintenance burden.
> It's already possible to do this without patching by writing an
> "install aliases" function that looks something like:
>
>      import sys
>      import myapp._vendor.six
>
>      def install_module_aliases():
>          sys.modules["six"] = myapp._vendor.six
that specifically breaks the very purpose the vendoring happened
which is to decouple from the global installed version

for example if we vendor six in that way in pytest,
we literally break testing anything that depends on a different version 
of six

the same goes for other libraries
> Run that early in your app startup (before you import anything else),
> and "import six" will get your vendored version from the module cache
> rather than looking for the normal one.
>
> For app level bundling, though, it's really better to find a way to
> deploy a full venv if possible, rather than vendoring things (for
> library level work, aim to avoid vendoring in general and use
> dependency version specifiers instead).
for my use-case (which is making a infrastructure package robust and 
having it reuse well tested code)
its basically the worst solution to just pin global dependencies, since 
testing code that depends on different versions will be impossible to test

its seems impossible to avoid vendoring when one wants to use 
library-specific versions of dependencies without literally poisoning 
pythonpath.

the python module system and sys.modules are the main crippling factors 
in that area,
and my understanding so far is, that python-dev will not ever remove 
that limitation.

(and its not a good idea to remove such a limitation without having 
actual working encapsulation)

-- Ronny
> Cheers,
> Nick.
>