[Distutils] distlib.mount API design (was: wheels on sys.path clarification (reboot))

[Nick Coghlan]

On 30 January 2014 21:57, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
>> My one technical issue is with going beyond zipimport
>> behaviour to the point of extracting DLLs to the filesystem.
>> I remain -1 on that feature, and I believe I have explained why I think
>> there are issues (and why I think that any solution should be part of
>> zipimport and not added on in library or user code). But I'm happy to go
>> through the details again, if you like - or just to accept that I don't
>
> Yes please, let's get into some details. Of course I understand that you
> might not want to use the feature, but I don't understand the -1 on the feature
> per se - whether it is in distlib or in zipimport is a secondary consideration. I
> agree that zipimport is the logical place for it, but ISTM the reason why it can't
> go in there just yet is also the reason why one might have some reservations
> about the feature: binary compatibility. I accept that this not yet a fully resolved
> issue in general (cf. the parallel discussion about numpy), but if we can
> isolate these issues, we can perhaps tackle them. But for me, that's the main
> reason why this part of the distlib API is experimental.

I actually think this is a useful thing to experiment with, I'm just
not sure distlib is the best place for that experiment. With
appropriately secure tempfile handling and the right sys.path (and
module __path__) manipulation it's not obviously *impossible* to
handle C extensions at arbitrary positions in the module namespace
this way, just difficult. zipimport itself is a bad place to
experiment though, since not only is it currently a complex ball of C
code, but adding such a feature without clear evidence of robust
support in a third party project would be irresponsible.

In the case of distlib, the potential complexity of ensuring that such
a scheme works consistently across multiple platforms and as part of
various complex package layouts is enough to make me nervous about
having it in the same library as the metadata 2.0 reference
implementation :)

Now, if you were to split that functionality out from distlib into a
separate "wheeltab" project (or a name of your choice), I'd be
substantially less nervous, because endorsing distlib as the metadata
2.0 reference implementation wouldn't carry any implications of
endorsing a feature I consider "potentially interesting but rather
challenging to implement in a robust manner". mount() would become
something I could explore when I had some additional free time (hah!),
rather than something I felt obliged to help get to a more robust
state before releasing metadata 2.0.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia