[Distutils] Optional C extensions in packages

[Jim Fulton]

On Feb 1, 2007, at 1:03 PM, Phillip J. Eby wrote:

> At 07:11 AM 2/1/2007 -0500, Jim Fulton wrote:
>> I like the goal, since most of our extensions are in this category,
>> however, I have a feeling that this use case might be better served
>> by packaging the extensions as separate distributions.
>
> True, but without automation, it's more work for the package author  
> to "do the right thing".

Yes. It also makes it easier for the consumer to be explicit about  
what they want.
Although maybe there are better ways.


>   If your extension is in a package, you have to make it a  
> namespace package.  You have to create another setup.py and  
> directory structure, and manage another set of releases.  This is a  
> lot more work than just marking the extension as "optional".

Yes.  See below...


>
>
>> Once concern I have is that I think I'd want an egg with the  
>> extensions
>> and the egg without the extensions to have different names, and  
>> perhaps
>> different requirements.  Of course, the names would be different
>> because an egg with a C extension would be platform dependent and an
>> egg without would be platform independent.  What if both a
>> platform-independent egg and a platform-dependent egg were available?
>> Which would take precedence? I think they have the same precedence  
>> now.
>>
>> I think I'd be +1 if platform-dependent eggs were prefered over
>> platform-independent ones.
>
> ...of the same version?

Yes


>   That's easy enough to do by changing the Distribution.hashcmp  
> property to put the 'platform' part earlier in the comparison.  If  
> they're different versions, however, it's a whole 'nother kettle of  
> fish.

Yup.

Of course, that points up that the ambiguity remains, on some level.

So I retract my precedence idea,

> The solution I was thinking of, however, has the additional benefit  
> of working right now.  If you're generating eggs for a package with  
> optional extensions, you would simply *not* generate a cross- 
> platform egg, but instead just ship an sdist tarball, plus platform- 
> specific eggs for the platforms you care about.  That way, anybody  
> on an unsupported platform will end up building a local egg from  
> the source, and either producing the extensions or skipping them  
> locally.  (And if you don't have access to a compiler, it seems  
> unlikely that you will be then *distributing* the eggs you produced.)
>
> Notice that this setup is a natural side-effect of the way most  
> people would produce and upload packages to the Cheeseshop anyway -  
> you would have to make an intentional effort to suppress extension- 
> building and upload a cross-platform egg; otherwise you will just  
> end up with a source distribution and a platform-specific egg,  
> which then works as intended.

I'm still worried about the ambiguous case when there are both  
platform-dependent and platform-independent eggs installed.

I think you were proposing an easy_install option.  This helps when  
someone installs a distribution directly, but doesn't help when a  
distribution is installed as a dependency.  It also doesn't help with  
controlling selection of eggs after installation.  And I think it  
doesn't make it easy to change one's mind.  For example, one might  
install an egg with extensions and then install one without  
extensions to debug a problem using the Python debugger.  Would the  
option let them do that?

Is it possible to control this as part of the requirement specification?
Perhaps this could be some kind of standard extra?

I'd strongly prefer to be able to control this via the requirements  
mechanism.  I'd like to be able to say that I want or don't want  
extensions as part of a requirement string.

Jim

--
Jim Fulton			mailto:jim at zope.com		Python Powered!
CTO 				(540) 361-1714			http://www.python.org
Zope Corporation	http://www.zope.com		http://www.zope.org