[Distutils] Optional C extensions in packages

[Jim Fulton]

Phillip J. Eby wrote:
> At 08:11 AM 2/2/2007 -0500, Jim Fulton wrote:
>> Phillip J. Eby wrote:
>>> At 05:32 PM 2/1/2007 -0500, Jim Fulton wrote:
>>>> I'm still worried about the ambiguous case when there are both
>>>> platform-dependent and platform-independent eggs installed.
>>> How would this happen?
>>
>> At least in a couple of ways.
>>
>> 1. As I mentioned in my previous note, when a package has optional
>>    extensions, one will often want to disable the extensions for
>>    debugging purposes.  It is easier debugging Python code than
>>    C code, especially in combination with other Python code.  In the
>>    past, this was typically done by removing .so (or .pyd) files.
>>    This can still be done with eggs, but I thik it will be attractive
>>    to do this by selecting diffeent eggs.
>>
>> 2. Consider the following scenario: Someone has a mac without a 
>> development
>>    environment installed.  They install some eggs and get versions 
>> without
>>    extensions.  Later, they install the development tools that came on 
>> the CD
>>    with their mac.  How do they reinstall the eggs with extensions?  
>> If they
>>    install in multi-version mode, won't they have a mix of eggs with and
>>    without extensions?
> 
> Well, they can always "rm -rf *.egg" and reinstall.  :)  Otherwise, 
> they'll get them by attrition when packages are upgraded to newer 
> versions.  In fact, using -U might be sufficient, although I think 
> EasyInstall actually has some quirks with respect to determining whether 
> -U will end up in a reinstall or not.

I'd be surprised if upgrade would reinstall if there weren't a later version
available.

...

>> I see your point.  This arises from the way that easy_install 
>> incrementally
>> installs distributions.  This potentially wouldn't be a problem for 
>> buildout,
>> but I wouldn't want to break easy_install (or workingenv).
> 
> Yes, so these features would have to wait until 0.7, and a possible 
> redesign of EasyInstall to be based on buildout (or something like it, 
> anyway), instead of the other way around.  ;)

I didn't mean to imply that buildout was better than easy_install,
merely noting that they were different.

>>> Sigh.  I guess at this point I don't really see a way to do optional 
>>> extensions that doesn't turn into a crazy madhouse of support later.  
>>> It seems to me that at least the problems with my approach would at 
>>> most boil down to, "how come this thing is so slow"?  :)
>>
>> OK, so based on this discussion, I'm in favor of your original proposal
>> as a start.  I think there should be a way to cause building/installation
>> of a platform-dependent egg even if there is a platform-independent egg
>> with the same installed already, and the other way around, to deal with
>> the use cases I described  earlier.  Even in multiple-version mode,
>> this is not a problem, because the eggs will have different file names.
>> I'd really *like* to be able to reflect the selection of these somehow
>> in requirement specifications,  but, if need be, this can be dealt with
>> at the tool (e.g. buildout)  level.
> 
> EasyInstall probably just needs to grow an option to force 
> reinstallation of a package, as that's a generally useful feature.  
> I.e., sort of a "don't allow the requirement to be satisfied with an egg 
> that's already on sys.path" option.

That seems like a rather big stick and a round-about way to do it.

Jim

-- 
Jim Fulton           mailto:jim at zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org