[Distutils] extends-cache

[Rafael Monnerat]

Hi,

I posted the bugs (I think are 2 diferent) into launchpad with my 
patches which I made on friday:

   - https://bugs.launchpad.net/zc.buildout/+bug/717730

   - https://bugs.launchpad.net/zc.buildout/+bug/717802

You are right about fall-back, I used the "ugly" way to not change 
download utility API.

Feel free to comment, I can update my patch.

Regards,
Rafael Monnerat



On 13-02-2011 08:05, Thomas Lotze wrote:
> Rafael Monnerat wrote:
>
>> I found that the file http://url/to/sample1.cfg is downloaded twice in the
>> same run when I use buildout or bootstrap.py, even if the file is already
>> present into the extends-cache.
> This is probably because the code that downloads that config file uses the
> cache in fall-back mode, i.e. it always tries to get a fresh version of
> the file (and update the cache with it) and uses a cached copy only when
> offline.
>
>> 1) Is there a way to avoid download twice?
> Since the download utility doesn't know about the needs of the code
> calling it (which is a good thing in terms of orthogonality), the part of
> the buildout code which recursively evaluates the config files that extend
> each other would have to be changed to remember which files it already
> downloaded. It could then either reuse them instead of calling the
> download utility again, or use the download utility's regular cache mode
> for any further downloads of the same file.
>
>> PS.: If it looks like a bug, I'm volunteer to fix it.
> If you implement the above (or something better) on a branch, I'd be happy
> to review it.
>
> BTW, this raises the question whether the fall-back mode of the cache
> should be made selectable with each call to a download utility. Currently,
> fall-back mode can only be configured when first instantiating a download
> utility (or by overriding its "fallback" attribute later, which would be
> ugly as a usage pattern).