[Distutils] [tg-trunk] Re: [ANN] EggFreezer

[Alberto Valverde]

Ian Bicking wrote:
> Alberto Valverde wrote:
>   
>> Ian Bicking wrote:
>>     
>>> Alberto Valverde wrote:
>>>   
>>>       
>>>> The usage is pretty straightforward:
>>>>
>>>> eggfreezer -o AllTurbogears2 -f
>>>> http://turbogears.org/2.0/downloads/current TurboGears2 tg.devtools
>>>>
>>>> That command will try to satisfy all dependencies for TurboGears2 and
>>>> tg.devtools (fetching them from local packages if available), using that
>>>> url to find links, and bundle them into a file called
>>>> AllTurboGears2-${py_version}-${platform}.py.
>>>>     
>>>>         
>>> As long as you are doing platforms, it might be nice to get them right. 
>>>   Specifically the UCS2/UCS4 distinction, though there might be more 
>>> that I'm forgetting.  (If there's actually platform-dependent files in 
>>> there, if not it'd be nice to leave out the platform entirely.)
>>>   
>>>       
>> I'm using pkg_resources.Distribution's 'platform' attribute to get this,
>> the algorithm basically iterates over all dependencies and as soon as
>> one has it set to not None it'll use that. If all are set to None then
>> no ${platform} is added to the filename. I'm assuming of course that all
>> distributions have the same string as platform, which I guess it isn't
>> not too far-fetched, but haven't really checked so I'm sure there's
>> something I might have overlooked.
>>
>> BTW, does pkg_resources populate it properly with he UCS2/UCS4
>> distinction you mention?
>>     
>
> No... which makes binary eggs unusable on Linux.  I feel like there was 
> something else that made binary packages on a Mac unreliable, but I 
> can't remember.  Windows binary eggs generally work fine.  This is 
> discussed some here: 
> http://philikon.wordpress.com/2008/06/26/is-there-a-point-to-distributing-egg-files/
>   

I remember reading that... Perhaps a solution could be to"freeze" the
tar.gzs too, beside the compiled eggs in case we're lucky,  and hope
that a compiler + libs are there when the install script is thawed. If
the compiled egg won't cut it, then try to compile form source. I'm not
sure how to tell easy_install to download the source distributions though.
> One nuisance is that people don't generally know how their Python was 
> built (UCS2 or UCS4).  I was thinking about making something very 
> similar to eggfreezer (which I'm unlikely to do now that eggfreezer 
> exists ;),
You're patches are welcomed, In fact, If you want to include it inside
virtualenv I would be most happy :).

>  and generating an "install" .py file that determines the 
> platform and downloads the appropriate platform bundle.
>   

Hmm, this "download" is precisely what I'm trying to avoid. My main use
case is: A machine has gone down and I want to quickly put back
everything together in another machine with a backup and something that
contains *all* needed software. Sort of like an apt reposisory inside a
dvd which lets you install debian on a machine with absolutely not net
access. The "no-net" condition is just there to guarantee that all deps
will be available no matter how old and discontinued they are (which if
I think about it is rather ambitious... well, at least make it more
likely than with the current situation)

I think that this multi-platform issue could be solved by bundling all
the different binary versions of all binary packages. However, I'm not
sure if pkg_resources could deal with the UCS2/UCS4 issue given that it
doesn't distinguish it in the platform id. Maybe by hacking in an extra
placeholder, before the .egg and after the ${platform}, that the script
uses to distinguish and then remove it before giving it to easy_install?
Though this smells like the root of the problem comes from setuptools
and should be fixed there...

Alberto