[Borgbackup] performance problem

[Giuseppe Arvati]

Hello,

again on performance

the 1st archive is about 195GB with 462.000 files and take
about 11  hours to complete

the 2nd archive is about 377GB with 1.020.000 files and take
less then 2 hours

I do not understand what can slow down the performance of the first
archive.

Which tests can I do to debug the first archive ?

Thank you


Archive name: utenti-2017-08-30-19
Time (start): Wed, 2017-08-30 19:30:18
Time (end):   Thu, 2017-08-31 06:40:27
Duration: 11 hours 10 minutes 8.99 seconds
Number of files: 462310
------------------------------------------------------------------------------
                        Original size      Compressed size 
Deduplicated size
This archive:              194.81 GB            194.83 GB 
35.63 MB
All archives:              584.40 GB            584.46 GB 
156.92 GB

                        Unique chunks         Total chunks
Chunk index:                  378370              1481376
------------------------------------------------------------------------------
------------------------------------------------------------------------------
Archive name: gruppi-2017-08-30-19
Time (start): Thu, 2017-08-31 06:40:36
Time (end):   Thu, 2017-08-31 08:19:50
Duration: 1 hours 39 minutes 13.62 seconds
Number of files: 1021125
------------------------------------------------------------------------------
                        Original size      Compressed size 
Deduplicated size
This archive:              376.39 GB            376.44 GB 
161.53 MB
All archives:                1.13 TB              1.13 TB 
315.65 GB

                        Unique chunks         Total chunks
Chunk index:                  948040              3423105



Il 25/08/2017 14:46, Thomas Waldmann ha scritto:
>> I’m trying to use borg as backup tool for a file server but I’m not able
>> to finish the backup during the night.
> 
> No problem, just finish it somehow, even if it runs over the day.
> 
> Maybe your first backup will not be super consistent (as at daytime,
> people will change files). But it will transfer most data chunks to the
> repository and create a files cache, so it can be faster next time.
> 
> Keep the first backup at least until you finished the 2nd backup of the
> same data.
> 
>> I would to backup all three directory every day between 23:00 and
>> 4:00 ( but it can be 22:00 – 5:00 ).
> 
> It might take longer the first time, but (assuming that most of your
> data is not changing and also the files cache of borg is working
> normally), your subsequent backups will be way faster than the first one.
> 
>> The first directory one take about 11 hours to backup 184GB so is not
>> possible to complete all 3 directory during the night.
> 
> It is, just not the first time.
> 
>> Where are my mistakes ? Missing options, error during archive init
>> phase, server performance, …
>> What can I test ?
> 
> Unless your 2nd backup is also unexpectedly slow, there is no big problem.
> 
> 
>