Why doesn’t BIT make an incremental backup?
I created a ‘full’ backup against my most important folder tree – about 250 GiB, containing about 50K items. It took far longer than I expected, but did complete. I did not use checksum. I did specify that the folder tree was to be checked for changes every 12 minutes. I then made a single file change as a test – an important change to an important file. 4 hours later BIT had still not made an incremental backup, but rsync has been running, consuming about 1% of my reasonably fast CPU throughout that time. A simple running of FreeFileSync comparing the current folder tree with the full backup made 4 hours ago immediately shows the one file I changed, but BIT has not seen it – meaning that this change might be lost if I restore my ‘original’ folder tree now. This loss of data is not what I expected. Why does it happen? I also notice that BIT did not backup a file of type .kra~ (a Krita file). This too appears to be an example of data lost by BIT. Why did that happen?
On Thu, 2023-11-30 at 22:26 +0000, LateJunction wrote:
I created a ‘full’ backup against my most important folder tree – about 250 GiB, containing about 50K items. It took far longer than I expected, but did complete. I did not use checksum. I did specify that the folder tree was to be checked for changes every 12 minutes. I then made a single file change as a test – an important change to an important file. 4 hours later BIT had still not made an incremental backup, but rsync has been running, consuming about 1% of my reasonably fast CPU throughout that time. A simple running of FreeFileSync comparing the current folder tree with the full backup made 4 hours ago immediately shows the one file I changed, but BIT has not seen it – meaning that this change might be lost if I restore my ‘original’ folder tree now.
This loss of data is not what I expected. Why does it happen?
I also notice that BIT did not backup a file of type .kra~ (a Krita file). This too appears to be an example of data lost by BIT. Why did that happen?
I think at this point it is better to open a new issue https://github.com/bit-team/backintime/issues and provide us all information to make this reproducible, mainly: - the configuration (screen shots) - screen shot of the taken snapshots .kra~ files are not backed-up because there is a default exclusion list for temp files in the "exclude" tab of the configuration dialog: *~
On 11/30/23 17:26, LateJunction wrote:
I created a ‘full’ backup against my most important folder tree – about 250 GiB, containing about 50K items. It took far longer than I expected, but did complete. I did not use checksum. I did specify that the folder tree was to be checked for changes every 12 minutes. I then made a single file change as a test – an important change to an important file. 4 hours later BIT had still not made an incremental backup, but rsync has been running, consuming about 1% of my reasonably fast CPU throughout that time. A simple running of FreeFileSync comparing the current folder tree with the full backup made 4 hours ago immediately shows the one file I changed, but BIT has not seen it – meaning that this change might be lost if I restore my ‘original’ folder tree now.
This loss of data is not what I expected. Why does it happen?
What file system type are you using on your host and backup medium files? You can reply with the output of the "df -T" command, which will show this information for all mounted file systems. Is your backup medium directly connected to your computer, or is it accessed via a network connection? To make use of the "hard link" features BIT expects, you must be using a file system that supports this feature. All native Linux file systems do, but non-native file systems may not. This could result in BIT copying all 250 GiB every time.
Firstly, my apology for seeming to have ignored your helpful reply to my original post. This was not intentional: I had no idea that there had been any responses to any of my post to the mailing list, until earlier this morning,, because I can see no indication that replies have been made. Indeed I will say that I find this mailing list implementation to be difficult to use: even now the only way I can find the list of threads, and potential responses, is via the 'posting activity' of my account profile. There should be a simpler way. In the interim between your reply and this acknowledgement, I have concluded that the situation I reported has one explanation: operator ineptitude. Having deleted that profile and its associated snap shots, BIT is now delivering the results I expected. To answer your questions: 1. File system type: Ext4 on both host and backup medium, under Linux Mint 21.2 2. output of 'df -T' command: Filesystem Type 1K-blocks Used Available Use% Mounted on tmpfs tmpfs 3266288 2068 3264220 1% /run /dev/nvme0n1p2 ext4 155572568 17273612 130323484 12% / tmpfs tmpfs 16331428 8 16331420 1% /dev/shm tmpfs tmpfs 5120 4 5116 1% /run/lock /dev/nvme0n1p3 ext4 322826564 158398048 147956760 52% /media/tony/7D-xtra /dev/nvme0n1p1 vfat 94759 6186 88574 7% /boot/efi /dev/nvme1n1p2 ext4 921853660 284315764 590636548 33% /home /dev/sda1 ext4 960302096 371353232 540094440 41% /media/tony/7D-Backup tmpfs tmpfs 3266284 1708 3264576 1% /run/user/1000 3. Backup medium (HDD) is directly connected to my system (and is manually replicated, for resiliency, using FreeFileSync, to multiple USB connected HDDs) 4. I believe my system, as far as BIT use is concerned, supports hardlinks throughout.
participants (4)
-
BiT dev -
LateJunction -
Reece R. Pollack -
Tony Hamilton