[Distutils] extends-cache

Thomas Lotze thomas at thomas-lotze.de
Sun Feb 13 11:05:56 CET 2011


Rafael Monnerat wrote:

> I found that the file http://url/to/sample1.cfg is downloaded twice in the
> same run when I use buildout or bootstrap.py, even if the file is already
> present into the extends-cache.

This is probably because the code that downloads that config file uses the
cache in fall-back mode, i.e. it always tries to get a fresh version of
the file (and update the cache with it) and uses a cached copy only when
offline.

> 1) Is there a way to avoid download twice?

Since the download utility doesn't know about the needs of the code
calling it (which is a good thing in terms of orthogonality), the part of
the buildout code which recursively evaluates the config files that extend
each other would have to be changed to remember which files it already
downloaded. It could then either reuse them instead of calling the
download utility again, or use the download utility's regular cache mode
for any further downloads of the same file.

> PS.: If it looks like a bug, I'm volunteer to fix it.

If you implement the above (or something better) on a branch, I'd be happy
to review it.

BTW, this raises the question whether the fall-back mode of the cache
should be made selectable with each call to a download utility. Currently,
fall-back mode can only be configured when first instantiating a download
utility (or by overriding its "fallback" attribute later, which would be
ugly as a usage pattern).

-- 
Thomas





More information about the Distutils-SIG mailing list