[Borgbackup] tag A or U when files are updated ?

[Ernest CHIARELLO]

Hello Thorsten,

thank you for taking the time to answer me ! 😁

I want to clarify my request. we are typically in a case of libvirt/kvm 
virtualization.

for the moment, i was using borgbackup in order to save VM-images and 
snaphots from a local backups folder to a repository on a remote 
server... it works perfectly.

actually, VM-images are managed by fi-backup 
<https://github.com/dguerri/LibVirtKvm-scripts/blob/master/fi-backup.sh>, 
a bash script which produces one snaphot per hour of each VM, and which 
consolidates them at the end of the day into a single qcow2 file ... 
then starts again the next day.

after each snaphot creation, fi-backup copies the file to the backups 
folder.when it consolidates all the snapshots (small files), it also 
copies the new image (big file) to the backups folder.it uses the 
command "cp -u", which is expensive in time...

my question is: would it be possible to replace the "cp -u" by "borg 
create" (from /var/lib/libvirt/images to /var/lib/libvirt/backups), in 
order to benefit from the deduplication and to shorten the copy times ?

the creation of dummy files would then also be managed by fi-backup.

What do you think ?

Thanks in advance,

Ernest.


Le 02/09/2021 à 15:48, Thorsten Schöning a écrit :
> Guten Tag Ernest Chiarello,
> am Donnerstag, 2. September 2021 um 15:08 schrieben Sie:
>
>> but I don't see how to avoid this type of problem when saving large
>> files from time to time (qcow2 images)... 😕
> Things depend on if you ONLY store the large failes additionally and
> never more current smaller ones as well. For most backup purposes this
> is not the case, as some smaller files will most likely be available,
> not only large images. If it's different in your case, you simply need
> to make sure by creating some small dummy file to "trick" BorgBackup.
>
> I did exactly that e.g. for VMs which I archive and don't change
> anymore: VM-images are at timestamp X, dummy file is at timestamp X+1,
> so no problem ever. If VM-images would change at some point again, one
> would simply touch the dummy file again so that it's newer and things
> work as before. My dummy fille even simply contains the content of the
> linked FAQ, so that I always know why it's there. :-)
>
> If you want to make things easier to use, I can suggest BorgMatic.
> That allows you to have everythign in yome YAML-config and
> additionally provides hooks being executed before the backup. Those
> hooks can simply be used to touch your dummy file always.
>
>> hooks:
>>      # Custom preparation scripts to run.
>>      before_backup:
>>          - prepare-for-backup.sh
> https://torsion.org/borgmatic/
>
> Mit freundlichen Grüßen
>
> Thorsten Schöning
>

-- 
Ernest CHIARELLO - Ernest.Chiarello at univ-fcomte.fr

UMR6049 ThéMA - CNRS / Université de Franche-Comté
Service de gestion et production de l'information
32 rue Mégevand - F 25030 BESANCON Cedex
Tel : 03 81 66 53 66           Mob : 07 82 99 11 08

UAR3124 MSHE Ledoux -  CNRS / UFC
Plateforme technologique SHERPA
1 rue Charles Nodier - F 25030 BESANCON Cedex
Tel : 03 81 66 53 66           Mob : 07 82 99 11 08

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.python.org/pipermail/borgbackup/attachments/20210903/2159761a/attachment.html>