[Distutils] buildout in offline mode - install receipts also from cache?

[Jim Fulton]

On Sat, Jan 12, 2013 at 7:11 PM, Philippe Ombredanne
<pombredanne at nexb.com> wrote:
>
> On Mon Dec 17 15:35:47 CET 2012, Anton Koval' psihonavt at gmail.com wrote:
>> in our project we've decided to completely move
>> buildout process to offline mode.
>> according to documentation
>> (http://pypi.python.org/pypi/zc.buildout/2.0.0a5)
>> I made updates in our main buildout configuration file, like:
>> download-cache = ...
>> install-from-cache = true
>> offline = true
>> But when buildout tries to install any part with recipe specified,
>> it is (obviously) failing, e.g.:
>> Error: Couldn't find a distribution for 'iw.recipe.cmd'.
>> So, my question is: are there ways to fetch recipes also from
>> some cache (or whatever) on local disk?
>
> I have experienced the same issue
> See https://github.com/buildout/buildout/issues/41
> This is a bug and also per Jim Fulton a lack of specification/documentation
> of what offline really means.
>
> IMHO it is not possible today to bootstrap and buildout offline. Anyone with
> a better story there?

Try:

install-from-cache = true

without specifying offline mode.

install-from-cache was intended to support exactly this use case.

>
> That said the fixes are rather easy ... And I have an ugly but nicely
> working monkey patching extension that I will publish on pypi in a couple
> days... Which ironically enough needs to be installed with pip before a
> boostrap.
>
> Now Jim suggested I should bring the discussion here.
>
> In general installing packages without a network connection should be
> straight forward, simple and easy. That would be the common understanding of
> what offline means.
>
> It works fine with setuptools/distribute and pip albeit a tad circumvoluted
> and under documented.
>
> It does not with buildout and fails because with the offline flag/config,
> buildout calls its install function with a None dest argument making
> setuptools/distribute fail downstream with a weird exception as you
> reported.
>
> This affects all current and recent versions of buildout.

Again, I think this works by setting install-from-cache to true.

The original semantics of offline mode didn't fit it's name. It origibally
meant "don't install anything".  That's how it was implemented.  The original
goal was both not to download anything and not to **install** anything.
Note that this was added before there was a cache.

Later, offline mode was extended to avoid network access. In particular
offline mode was changed to disallow extending remote configuration
files.  This change,
not made by me, broke some of my companies buildouts.  I didn't argue with it at
the time as the change was consistent with the option name, if not the original
intent.

Offline mode is largely a relic of buildout's early evolution.  I almost
never use it and, when I do, I use it in a way that depend on it's
current semantics.  In particular, I use buildout to create configurations
for installed software.  In this context, buildout is running as root and it
is not OK to install new software.

I believe that install-from-cache (without offline mode) would address
Philippe's use case.  (It's used by zc.sourcerelease.)

That leaves the ambiguous meaning of offline mode.  In particular,
buildout 2 potentially provides an opportunity to fix things in a backward-
incompatible way.

OTOH, I'd like to avoid any major changes in 2.0 at this point
because it really needs to get finished as soon as possible.

Among the options I see at the present:

- Do nothing.  And just document the somewhat weird semantics
  of the name.

- Add a new option: install, such that install=false has the same
meaning as offline=true.
  Deprecate the offline option and accompanying command-line option.

Jim

-- 
Jim Fulton
http://www.linkedin.com/in/jimfulton
Jerky is better than bacon! http://zo.pe/Kqm