[borgbackup] initial 0.26 cache re-sync

[Leo Famulari]

I'm really glad to see this work being done. Am I right to assume that
the motivation behind this is to make it easier to backup multiple
systems to the same repository, achieving deduplication of "system"
files like /etc, /usr, /lib, et cetera?

On Sun, Sep 20, 2015, at 08:37, Thomas Waldmann wrote:
> Hi, just wanted to post a note about this:
> 
> When you use 0.26.0 the first time, you might see a rather long
> "synchronizing chunks cache".
> 
> This is due to the way the synchronisation works in 0.26+ and will only
> take that long ONCE. Subsequent re-syncs will only fetch the new
> archives it discovers in the repo (not all of them).
> 
> I changed the way the single-archive indexes are kept locally, they are
> now single files in ~/.cache/borg/REPOID/chunks.archive.d/.
> 
> 0.25 used a compressed tar archive "chunks.archive" - but dealing with
> tar and (re-)compression took way too much time and cpu and the
> compression did not work as great in practice as in my experiments.
> 
> If you have a slow connection to the repository and/or or a huge number
> of archives, you can save some time by manually extracting your pre-0.26
> chunks.archive to that location.
> 
> This manual procedure is OPTIONAL, if you do not do it, borg will kill
> the compressed tar automatically and then fetch all single-archive
> indexes from the (remote?) repo.
> 
> Make sure you have lots of disk space free in .cache/borg:
> 
> cd ~/.cache/borg/REPOID
> mkdir chunks.archive.d
> cd chunks.archive.d
> tar xJvf ../chunks.archive
> # if you have a older python, it might be also xjvf or xzvf.
> 
> # at the end, check that permissions/mode are as you see
> # for the other files in .cache/borg:
> cd ..
> chown -R borg.borg chunks.archive.d
> chmod -R go-rwX chunks.archive.d
> 
> # after successfully extracting the chunks.archive. remove it
> # 0.26 does not use it any more (and would also remove it the
> # first time a cache resync happens):
> rm chunks.archive
> 
> In case something does not work, you can still kill chunks.archive.d and
> let borg do it (slowly).
> 
> Cheers,
> 
> Thomas
> 
> -- 
> 
> GPG ID: FAF7B393
> GPG FP: 6D5B EF9A DD20 7580 5747 B70F 9F88 FB52 FAF7 B393
>