[Pythonmac-SIG] Mac Python and eggs...

[Bob Ippolito]

On Jan 23, 2006, at 4:35 PM, Christopher Barker wrote:

> Bob Ippolito wrote:
>> Yes, the egg itself gets hosted on Cheese Shop, as well as the source
>> download (hence the "upload").
>>
>> The packages you often use are not yet using setuptools,
>
> Do you know of a package I can find on cheeseshop that is using
> setuptools? I'd like to get a sense of what features that offers.

Paste/PasteScript/PasteDeploy use just about all of the features that  
setuptools has, but you're unlikely to make any sense out of it.

 From a developer's perspective, setuptools gives you:

- Fixes to distutils to make it suck less (e.g. Cheese Shop register/ 
upload even in Python 2.3, find_packages(), etc.)
- Subversion integration (no more MANIFEST.in junk, allows tagging  
releases with versions)
- Entry points (allows other eggs to provide plugins for your egg, or  
for your egg to provide plugins to others)
- develop and test commands (develop is *awesome*)
- Allows you to declare dependencies and optional dependencies (as  
extras)
- Allows you to require a specific version of a package.. which even  
works if there are multiple versions of that package installed  
(though only one can be loaded by a given Python interpreter process,  
of course).  This is particularly relevant for things like wxPython,  
where fragile hacks are used to try and make this happen.
- Provides for in-package data files and in-egg metadata, in all  
situations (even if the egg is zipped, if you use the pkg_resources  
APIs your code will still work)

 From a user's perspective, setuptools gives you:

- Easy installation, including the download of dependencies
- Easy upgrades

And there are network effects.. because tools like py2app and py2exe  
will take advantage of eggs to more reliably produce packages with no  
hacking.  Libraries like PIL could be extended with additional image  
plugins without doing something dumb like scanning sys.path for  
Python files named "*ImagePlugin.py".

You really ought to read the docs:
http://peak.telecommunity.com/DevCenter/setuptools
http://peak.telecommunity.com/DevCenter/EasyInstall

>> so it's no
>> surprise that they don't take advantage of these new features that
>> aren't in distutils.
>
> Well, matplotlib has an egg on the sourceforge download site, and  
> NumPy
> is putting its include files in a odd place specifically to  
> accommodate
> ssetuptools, and neither of those have eggs on cheeseshop. I guess
> setuptools really hasn't caught on yet. but it does seem to be  
> gaining a
> lot of momentum, so I'm hopeful.

Well if they can build an egg, they can upload it to Cheese Shop with  
one command.  They probably just haven't invested the five minutes to  
read over the setuptools documentation to see that it has an upload  
feature.

>> Someone needs to convince the maintainers of  the
>> packages that you use that they should be using setuptools, at  least
>> for distribution (but not necessarily as a requirement for   
>> installation).
>>
>>> Frankly, that doesn't do any more for me than Google, at least for
>>> packages I already know the name of.
>>
>> That's orthogonal.
>
> That was based on the fact that I just got pointed back to the  
> project's
> primary download site anyway, that's usually not hard to find (for a
> project I know about anyway). If the eggs were available directly on
> Cheeseshop, then it would be better.

They are, but only if the maintainer goes through the trouble to type  
"python setup.py bdist_egg upload" and press enter.  For most  
projects, easy_install will automatically download the correct egg  
from the download page -- but only if the maintainer (or sourceforge)  
doesn't fuck up the HTML.  It sounds like that happened when you  
tried it.

>>> What that tells me is that there is a need for an OS-X package  
>>> (or  egg)
>>> repository on pythonmac.
>>
>> I'm not at all convinced on that point.
>
> The reason we've ever needed it hasn't changed: most package  
> maintainers
> don't build packages for OS-X. The Mac community has always had to do
> that itself for most packages. The only exception for the packages  
> I use
> is wxPython. (and Robin is looking into eggs)
>
> If we can't get package maintainers to build OS-X compatible eggs,  
> what
> good does cheese shop do?

That can change.  Cheese Shop allows package owners to easily add  
additional maintainers.  They could delegate the authority to you to  
upload Mac OS X eggs, for example.  It takes four clicks and a user  
name (click login, click somepackage, click role, type the user name,  
click add role).

> It's also really nice to have one-stop shopping for OS-X Python  
> packages.

That can easily be Cheese Shop.  pythonmac.org could contain some  
metadata for packages that are definitely known to work on Mac OS X,  
but I don't think it should be independent from Cheese Shop.

> Another issue I wonder about is all the possible combinations of OS-X
> and Python versions that we're dealing with. Do they all need a  
> separate
> egg?

Yes.  That's not really different than now, with the exception that  
easy_install/setuptools knows which egg to download.  This is big,  
because it seems that many users have trouble deciding which pkg is  
appropriate to download from pythonmac.org/packages/

Eggs are probably also the solution to distributing universal  
applications.  Using separate eggs for PPC and x86, py2app could  
collect all eggs for both architectures for and stick them in the  
bundle.  Currently, it's basically infeasible for PPC users to  
reliably build x86 eggs (and impossible to test them), so this is big  
for anyone still on PPC.

-bob