From storkeen at gmail.com Wed Jan 5 10:13:49 2022 From: storkeen at gmail.com (F Wessels) Date: Wed, 5 Jan 2022 16:13:49 +0100 Subject: [Borgbackup] Very large delta's when pruning a static backup Message-ID: Hi all, best wishes for 2022. I have a backup of a not changing source directory. The files within are static aka not written to. This directory remains in the backup script so it get's processed daily and therefore also pruned. This all works as expected however the directory containing the repo has very large delta's (many bytes are changed daily) therefore I need to replicate very much data offsite while all that's hapening is pruning old data. The source directory contains about 40 large files some 500GB in total. borg create ended with a deduplicated size of 2.65kB That's to be expected because everything is static (for months) borg prune ends with -24.60GB deduplicated size. But in the end I needed to replicate 32.764GB to my offsite mirror location. My expectation was that since no new data was written to the source files, no extra data was added to the repo. That behaviour seems to happen. However I also expected that when pruning this repo data would be deleted and that I only would be removing data in my offsite location, not transfering so much. I'm still using borgbackup 1.1.11 and upgrading involves red tape. Could anybody please comment on this? Thank you for your interest and time. Kind regards, Stor Keen -------------- next part -------------- An HTML attachment was scrubbed... URL: From tw at waldmann-edv.de Wed Jan 5 10:58:20 2022 From: tw at waldmann-edv.de (Thomas Waldmann) Date: Wed, 5 Jan 2022 16:58:20 +0100 Subject: [Borgbackup] Very large delta's when pruning a static backup In-Reply-To: References: Message-ID: > best wishes for 2022. 2u2! > borg create ended with a deduplicated size of 2.65kB Sounds like borg is deduplicating perfectly: - it has deduped all the files contents - it has deduped even the metadata stream (containing all file metadata, like filenames, owner/group/mode, etc.) - the few "new" kB remaining after this are some archive metadata > borg prune ends with -24.60GB deduplicated size. Likely because it removes 1 old archive after creating the new one to stay at the same archive count. > But in the end I needed to replicate 32.764GB to my offsite mirror location. That's way more than expected. Ideas: - offsite syncing broken somehow (not borg) - can you get a list of files it transfers? what tool do you use? does it work with mtime or ctime or based on what precisely? - borg's compact_segments() internal code runs after every write operation (create, delete, ...), shuffling data from old non-compact segment files into new compact ones. in your case, borg prune deletes 1 old archive, removing that old archive metadata stream from the segment files (punching a hole, making it non-compact) and also creating 1 new archive (writing the metadata stream to a new segment file). if you don't have too many files (thus not dealing with large md streams), that should only touch a few segment files, though. But maybe it is worth changing the max segment file size (default 500MB, for remote syncing it is better to go a bit smaller, like 25..50MB). But this is rather about optimizing from 1GB to 0.1GB (not from 32GB to ...). - if that does not help, create full ls -lR listing of the repo before borg operations and afterwards (directly before the big sync) and compare them. > My expectation was that since no new data was written to the source > files, no extra data was added to the repo. That behaviour seems to happen. > However I also expected that when pruning this repo data would be > deleted and that I only would be removing data in my offsite location, > not transfering so much. Some overhead comes from compact_segments. Without that, space usage would grow forever (like in append-only mode). In borg 1.2, there will be a separate borg compact command and compact_segments() won't be called implicitly any more, so there is better control about when and how often that happens. > I'm still using borgbackup 1.1.11 and upgrading involves red tape. New stable releases within the same series (like 1.1.x) usually work quite good, if that is the problem. From storkeen at gmail.com Wed Jan 5 11:38:44 2022 From: storkeen at gmail.com (F Wessels) Date: Wed, 5 Jan 2022 17:38:44 +0100 Subject: [Borgbackup] Very large delta's when pruning a static backup Message-ID: Hi mr. Waldmann, Unfortunately the ZFS filesystem where the backup repo is stored isn't snapshotted. So calculating the diff using that method is unfortunately unavailable but I'll make a request to include that as well. The program for the remote sync is rclone. We use it to store the repo in two different offsite locations. One is a public cloud object store the other is a private object store. I'll produce some more data regarding this issue. I did see that rclone was transfering many files. thank you very much for looking into this. -------------- next part -------------- An HTML attachment was scrubbed... URL: From tw at waldmann-edv.de Sun Jan 23 16:44:01 2022 From: tw at waldmann-edv.de (Thomas Waldmann) Date: Sun, 23 Jan 2022 22:44:01 +0100 Subject: [Borgbackup] borgbackup 1.2.0 beta 4 released Message-ID: <400e7bcf-f7a5-33b3-0ab8-82d75c8182e1@waldmann-edv.de> https://github.com/borgbackup/borg/releases/tag/1.2.0b4 More details: see URL above. Cheers, Thomas From jeffbrown.the at gmail.com Wed Jan 26 09:00:22 2022 From: jeffbrown.the at gmail.com (Jeffrey Brown) Date: Wed, 26 Jan 2022 09:00:22 -0500 Subject: [Borgbackup] Copy a repo with different encryption options? Message-ID: I'd like to recreate my borg repo with a different security mode. Can it be done? The documentation for `borg init`[1] states that encryption settings are fixed at creation time, and I don't see options for initializing a repo that will continue where some other one left off. Thank you. [1] https://borgbackup.readthedocs.io/en/stable/usage/init.html#borg-init -- Jeff Brown | Jeffrey Benjamin Brown LinkedIn | Github | Twitter | Facebook -------------- next part -------------- An HTML attachment was scrubbed... URL: From tw at waldmann-edv.de Sat Feb 5 17:01:28 2022 From: tw at waldmann-edv.de (Thomas Waldmann) Date: Sat, 5 Feb 2022 23:01:28 +0100 Subject: [Borgbackup] borgbackup 1.2.0rc1 released! Message-ID: <4946cebf-35b3-4345-68b4-e3d4be0e3654@waldmann-edv.de> borgbackup 1.2.0rc1 released for testing (beta quality, do not use for production, but please stress test). https://github.com/borgbackup/borg/releases/tag/1.2.0rc1 -- GPG ID: 9F88FB52FAF7B393 GPG FP: 6D5B EF9A DD20 7580 5747 B70F 9F88 FB52 FAF7 B393 From romeo.disca at gmail.com Tue Feb 15 15:54:08 2022 From: romeo.disca at gmail.com (romeo.disca at gmail.com) Date: Tue, 15 Feb 2022 21:54:08 +0100 Subject: [Borgbackup] I want to drive the windows port from release 1.2.0 on Message-ID: <005201d822ae$2d34d280$879e7780$@gmail.com> Hi all, I want to drive the windows port development and need reviewers for this deed. You can also help accomplish tasks. If you are willing to do so, just add your request to this thread, with a comment of what you want to do. In the first place, because there is no one else at the moment for the windows port, I need reviewers. I've just started and want to get the first commits merged in the major branch of the 1.2.0 release from February, 22th 2022 on. Thank you in advance. Cheers, Romeo -------------- next part -------------- An HTML attachment was scrubbed... URL: From simplerezo at gmail.com Wed Feb 16 13:36:11 2022 From: simplerezo at gmail.com (Support SimpleRezo) Date: Wed, 16 Feb 2022 19:36:11 +0100 Subject: [Borgbackup] Disk usage of repository Message-ID: Hi We are using borg for some years now, and we really like it ! But on a server backed up using borg 1.1.17, the repository has grown to 6 TB, as reported by borg info: Original size Compressed size Deduplicated size All archives: 85.24 TB 85.02 TB 6.29 TB Because I need to retrieve some space, I'm trying to figure out which archive is using space. I ran "borg info -P ''" to have a look at "Deduplicated size" of each archive. But something is confusing me: if I make the "deduplicated size" sum of all archives, the result is less than 800GB... I don't think this make sense, and it does not help me to clean some space :/ Regards -- Clement SimpleRezo From tw at waldmann-edv.de Thu Feb 17 15:43:39 2022 From: tw at waldmann-edv.de (Thomas Waldmann) Date: Thu, 17 Feb 2022 21:43:39 +0100 Subject: [Borgbackup] Disk usage of repository In-Reply-To: References: Message-ID: <55d2bfca-d199-f362-4234-cd506e8d8b37@waldmann-edv.de> > We are using borg for some years now, and we really like it ! You're welcome! > But on a server backed up using borg 1.1.17, the repository has grown > to 6 TB, as reported by borg info: > Original size Compressed size Deduplicated size > All archives: 85.24 TB 85.02 TB 6.29 TB > > Because I need to retrieve some space, I'm trying to figure out which > archive is using space. Due to deduplication, that is not as easy as it sounds. > I ran "borg info -P ''" to have a look at "Deduplicated size" of each archive. > > But something is confusing me: if I make the "deduplicated size" sum > of all archives, the result is less than 800GB... I don't think this > make sense, and it does not help me to clean some space :/ It makes sense: the amount it shows as deduped size of an archive is the repo space allocated by chunks ONLY referenced by that archive. So, if you just delete this single archive and keep all others, it will free that amount. If you always backup the same system, a lot of files that do seldomly change will be contained in many (N) archives, so if you delete a single one of them, there will be still N-1 archives referencing the same chunks (and thus, that space will only get freed after you delete the LAST archive referencing these chunks). So "summing up" these values is an invalid way to calculate something interesting. From lazyvirus at gmx.com Fri Feb 18 16:06:09 2022 From: lazyvirus at gmx.com (Bzzzz) Date: Fri, 18 Feb 2022 22:06:09 +0100 Subject: [Borgbackup] Disk usage of repository In-Reply-To: References: Message-ID: <20220218220609.4b1c4349@msi.defcon1.lan> On Wed, 16 Feb 2022 19:36:11 +0100 Support SimpleRezo wrote: > Hi Hi Simple ;-), > Because I need to retrieve some space, I'm trying to figure out which > archive is using space. I ran "borg info -P ''" to have a look at > "Deduplicated size" of each archive. To achieve that, as you have many common parts to all backups, the best is to keep a logbook, either a manual one or better: use the comment possibility for each backup. For example, when I migrated some big data from one server to another among several days, I changed my (manual) script to ask for a backup comment where I wrote down the main changes - this way, when I was absolutely sure there was no data corruption on the new server and several days had passed, I was able to destroy all former backups from the original server up to the day after the migration, recovering a lot of place. My ??0.017658753 Jean-Yves From tw at waldmann-edv.de Tue Feb 22 16:59:46 2022 From: tw at waldmann-edv.de (Thomas Waldmann) Date: Tue, 22 Feb 2022 22:59:46 +0100 Subject: [Borgbackup] borgbackup 1.2.0 released! Message-ID: <151d0e4f-f34d-05b3-d36d-1bb8a65443ee@waldmann-edv.de> see there: https://github.com/borgbackup/borg/releases/tag/1.2.0 From blackbirdralph at gmail.com Thu Feb 24 03:01:51 2022 From: blackbirdralph at gmail.com (ralph strebbing) Date: Thu, 24 Feb 2022 03:01:51 -0500 Subject: [Borgbackup] ACLs not preserved Message-ID: Morning All, I've been working on setting Borg up as our new backup solution, and I've run into a snag. When following the Pull mode instructions using the ssh-agent method, the files backed up don't seem to have any ACLs included, and aren't shown when mounted or extracted on the backup server. The owner and group UIDs are the only thing that seem to be preserved. I'm running both systems on an ext4 filesystem with ACLs enabled, and the ACL utils installed. Running borg 1.2.0 via the PPA on Ubuntu 20.04 on both the client and backup server. Is there an option somewhere that I'm missing? Or an additional step maybe? Any help is appreciated! Regards, Ralph -------------- next part -------------- An HTML attachment was scrubbed... URL: From blackbirdralph at gmail.com Fri Feb 25 10:37:43 2022 From: blackbirdralph at gmail.com (ralph strebbing) Date: Fri, 25 Feb 2022 10:37:43 -0500 Subject: [Borgbackup] ACLs not preserved In-Reply-To: References: Message-ID: On Thu, Feb 24, 2022 at 2:03 PM Thomas Waldmann wrote: > Please link to the docs you followed. https://borgbackup.readthedocs.io/en/stable/deployment/pull-backup.html#ssh-agent > Also give the path of the files you back up (and you have issues with) > and show the "mount" output at the time when borg create is running. I'm backing up `/storage/netfiles/testfolder` and the repo directory is located in `/root/repo` I'm mounting to `/tmp/testRepo` using the following command `borg mount ./repo::filesrv1_2-23-2022 /tmp/testRepo` > As far as I am aware, borg by default will back up ACLs presented by the > file system you back up. You can disable that by using --no-acls (not > sure about --no-xattrs). I want all xattrs and acls backed up, so none of these options are included in my backup command (listed below) ( eval $(ssh-agent) > /dev/null; ssh-add -q ~/.ssh/borg-client_key; ssh -A -o StrictHostKeyChecking=no root at filesrv1 "borg --rsh 'ssh -o StrictHostKeyChecking=no' create --list ssh://root at borg-test/~/repo::filesrv1 _2-23-2022 /storage/netfiles/testfolder"; kill "${SSH_AGENT_PID}"; ) Note the above command is not exactly what's on the docs linked, as that is from my most recent attempt, being without encryption. > If the file system you back up does not support ACLs, borg won't see them. All systems we're backing up have acl support, they are all running Ubuntu and if it's not explicitly set in the mounting options, it's mounted with ACLs by default as modern Ubuntu does now. > Is that a regression / did that work with 1.1.x? No, the first try I did was using the default version in the ubuntu repos, which was 1.1.11, and I was seeing the same issue. > Also, it would be good if you could reproduce with the normal push mode. I just gave it a shot on push mode, and both mount and extract show the same issue. From tw at waldmann-edv.de Fri Feb 25 15:23:24 2022 From: tw at waldmann-edv.de (Thomas Waldmann) Date: Fri, 25 Feb 2022 21:23:24 +0100 Subject: [Borgbackup] ACLs not preserved In-Reply-To: References: Message-ID: >> it would be good if you could reproduce with the normal push mode. > I just gave it a shot on push mode, and both mount and extract show > the same issue. OK, guess it makes sense to open an issue on github about this. Do NOT refer to special pull mode, but post a minimal push-mode reproducer script that shows the issue when executing it. If you can reproduce even with a local repo, use a local repo. From blackbirdralph at gmail.com Fri Feb 25 16:14:18 2022 From: blackbirdralph at gmail.com (ralph strebbing) Date: Fri, 25 Feb 2022 16:14:18 -0500 Subject: [Borgbackup] ACLs not preserved In-Reply-To: References: Message-ID: On Fri, Feb 25, 2022 at 3:23 PM Thomas Waldmann wrote: > OK, guess it makes sense to open an issue on github about this. > > Do NOT refer to special pull mode, but post a minimal push-mode > reproducer script that shows the issue when executing it. > > If you can reproduce even with a local repo, use a local repo. Will make an issue, and I did test with a local repo, and THAT preserves the ACLs, so it's something that happens when running the command to a remote machine. From tw at waldmann-edv.de Fri Feb 25 17:21:14 2022 From: tw at waldmann-edv.de (Thomas Waldmann) Date: Fri, 25 Feb 2022 23:21:14 +0100 Subject: [Borgbackup] ACLs not preserved In-Reply-To: References: Message-ID: <0486adf9-d801-e070-e16a-7417823f0832@waldmann-edv.de> >> If you can reproduce even with a local repo, use a local repo. > Will make an issue, and I did test with a local repo, and THAT > preserves the ACLs, so it's something that happens when running the > command to a remote machine. It now seems unlikely that this is a borg issue. borg almost works in the same way when used with local or remote repos. Especially all actions that happen BEFORE storing stuff into the repo are exactly the same (like input reading (meta+data), chunking, compression, encryption, authentication). Even storing into the repo is the same code (just executed at another place, connected via ssh pipes). So, it might be caused by sth else in your setup. From public at enkore.de Fri Feb 25 17:43:47 2022 From: public at enkore.de (d0) Date: Fri, 25 Feb 2022 23:43:47 +0100 Subject: [Borgbackup] ACLs not preserved In-Reply-To: <0486adf9-d801-e070-e16a-7417823f0832@waldmann-edv.de> References: <0486adf9-d801-e070-e16a-7417823f0832@waldmann-edv.de> Message-ID: I'm pretty sure sshfs supports neither ACLs nor extended attributes because sshfs is based on SFTP, which doesn't support any of that (afaik). Note that OpenSSH implements SFTP v3 and some extensions, not the later drafts. Am Fr., 25. Feb. 2022 um 23:21 Uhr schrieb Thomas Waldmann < tw at waldmann-edv.de>: > >> If you can reproduce even with a local repo, use a local repo. > > > Will make an issue, and I did test with a local repo, and THAT > > preserves the ACLs, so it's something that happens when running the > > command to a remote machine. > > It now seems unlikely that this is a borg issue. > > borg almost works in the same way when used with local or remote repos. > > Especially all actions that happen BEFORE storing stuff into the repo > are exactly the same (like input reading (meta+data), chunking, > compression, encryption, authentication). Even storing into the repo is > the same code (just executed at another place, connected via ssh pipes). > > So, it might be caused by sth else in your setup. > _______________________________________________ > Borgbackup mailing list > Borgbackup at python.org > https://mail.python.org/mailman/listinfo/borgbackup > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lazyvirus at gmx.com Fri Feb 25 20:44:21 2022 From: lazyvirus at gmx.com (Bzzzz) Date: Sat, 26 Feb 2022 02:44:21 +0100 Subject: [Borgbackup] ACLs not preserved In-Reply-To: References: <0486adf9-d801-e070-e16a-7417823f0832@waldmann-edv.de> Message-ID: <20220226024421.41c87458@msi.defcon1.lan> On Fri, 25 Feb 2022 23:43:47 +0100 d0 wrote: > I'm pretty sure sshfs supports neither ACLs nor extended attributes > because sshfs is based on SFTP, which doesn't support any of that > (afaik). Note that OpenSSH implements SFTP v3 and some extensions, > not the later drafts. It is only a part of the truth, xattrs can't be read nor set the normal way on a NFS(v3|4) share, however there are special programs to do so : nfs4_getfacl nfs4_setfacl nfs4_editfacl Note that, of course, xattrs do not work the other way for borg (when the client is mounting a remote server repo through NFS) and that sendfile and scp do not retain the xattrs either. Jean-Yves From blackbirdralph at gmail.com Fri Feb 25 20:51:58 2022 From: blackbirdralph at gmail.com (ralph strebbing) Date: Fri, 25 Feb 2022 20:51:58 -0500 Subject: [Borgbackup] ACLs not preserved In-Reply-To: <20220226024421.41c87458@msi.defcon1.lan> References: <0486adf9-d801-e070-e16a-7417823f0832@waldmann-edv.de> <20220226024421.41c87458@msi.defcon1.lan> Message-ID: On Fri, Feb 25, 2022, 8:44 PM Bzzzz wrote: > Note that, of course, xattrs do not work the other way for borg (when > the client is mounting a remote server repo through NFS) and that > sendfile and scp do not retain the xattrs either. > I wonder then, since this issue seems to be hard to fix given most of the ways borg handles remote hosts right now, is there any way rsync can potentially be leveraged? Or am I maybe misunderstanding the issue. Ralph > -------------- next part -------------- An HTML attachment was scrubbed... URL: From lazyvirus at gmx.com Fri Feb 25 21:28:57 2022 From: lazyvirus at gmx.com (Bzzzz) Date: Sat, 26 Feb 2022 03:28:57 +0100 Subject: [Borgbackup] ACLs not preserved In-Reply-To: References: <0486adf9-d801-e070-e16a-7417823f0832@waldmann-edv.de> <20220226024421.41c87458@msi.defcon1.lan> Message-ID: <20220226032857.17749a76@msi.defcon1.lan> On Fri, 25 Feb 2022 20:51:58 -0500 ralph strebbing wrote: > On Fri, Feb 25, 2022, 8:44 PM Bzzzz wrote: > > > Note that, of course, xattrs do not work the other way for borg > > (when the client is mounting a remote server repo through NFS) and > > that sendfile and scp do not retain the xattrs either. > > > I wonder then, since this issue seems to be hard to fix given most of > the ways borg handles remote hosts right now, is there any way rsync > can potentially be leveraged? Or am I maybe misunderstanding the > issue. Oops, sorry for the PM msg. Mouarf, just tested, doesn't work either and more than surely, doesn't work when using an SSH tunnel. Switches used: -av (-a including -X which is supposed to keep extended attributes). Oh, cp -a too ! In fact, I wonder if there is any viable solution at all. Jean-Yves From tschoening at am-soft.de Sat Feb 26 04:13:54 2022 From: tschoening at am-soft.de (=?utf-8?Q?Thorsten_Sch=C3=B6ning?=) Date: Sat, 26 Feb 2022 10:13:54 +0100 Subject: [Borgbackup] ACLs not preserved In-Reply-To: References: <0486adf9-d801-e070-e16a-7417823f0832@waldmann-edv.de> <20220226024421.41c87458@msi.defcon1.lan> Message-ID: <19410155776.20220226101354@am-soft.de> Guten Tag ralph strebbing, am Samstag, 26. Februar 2022 um 02:51 schrieben Sie: > I wonder then, since this issue seems to be hard to fix given most of the > ways borg handles remote hosts right now, is there any way rsync can > potentially be leveraged? Or am I maybe misunderstanding the issue. You should debug further, RSYNC isn't any different than BORG instances with ssh-agent. In both cases two instances of some reading app with full access to the filesystem are executed on the to-be-backed-up client. That is the difference with SSHFS, which is an additional layer with additional limitations like unstable INODES, missing ACLs etc. Readers using that layer don't have access to all data, compared to readers natively running on the client. But that shouldn't be used in your setup. Where does the following backup source comes from, what are the mount options exactly? Should be some EXT4 file system at filesrv1. You should provide the mount options and some example output showing that ACLs are really available. > /storage/netfiles/testfolder Your problem reads to me like there's some hiding layer between BORG and the filesystem. While that might be SSHFS, it shouldn't be, because SSH-AGENT doesn't use that. What happens when you run your backups multiple times after another? Check the output for modified files, as if all of those are modified over and over again, that's another hint for some layer like SSHFS with unstable INODES between backups. https://borgbackup.readthedocs.io/en/stable/faq.html#why-is-backup-slow-for-me https://borgbackup.readthedocs.io/en/stable/faq.html#it-always-chunks-all-my-files-even-unchanged-ones Mit freundlichen Gr??en Thorsten Sch?ning -- AM-SoFT IT-Service - Bitstore Hameln GmbH Mitglied der Bitstore Gruppe - Ihr Full-Service-Dienstleister f?r IT und TK E-Mail: Thorsten.Schoening at AM-SoFT.de Web: http://www.AM-SoFT.de/ Tel: 05151- 9468- 0 Tel: 05151- 9468-55 Fax: 05151- 9468-88 Mobil: 0178-8 9468-04 AM-SoFT IT-Service - Bitstore Hameln GmbH, Brandenburger Str. 7c, 31789 Hameln AG Hannover HRB 221853 - Gesch?ftsf?hrer: Janine Galonska F?r R?ckfragen stehe ich Ihnen jederzeit zur Verf?gung. Mit freundlichen Gr??en, Thorsten Sch?ning Telefon: +49 (0)515 94 68 - 0 Fax: E-Mail: TSchoening at am-soft.de AM-Soft IT-Service - Bitstore Hameln GmbH Brandenburger Stra?e 7c 31789 Hameln Diese E-Mail enth?lt vertrauliche und/oder rechtlich gesch?tzte Informationen und ist ausschliesslich f?r den Adressaten bestimmt. Jeglicher Zugriff auf diese E-Mail durch andere Personen als den Adressaten ist untersagt. Wenn Sie nicht der richtige Adressat sind oder diese E-Mail irrt?mlich erhalten haben, informieren Sie bitte sofort den Absender und vernichten Sie diese E-Mail. Sollten Sie nicht der f?r diese E-Mail bestimmte Adressat sein, ist Ihnen jede Ver?ffentlichung, Vervielf?ltigung oder Weitergabe wie auch das Ergreifen oder Unterlassen von Massnahmen im Vertrauen auf erlangte Information untersagt. This e-mail may contain confidential and/or privileged information and is intended solely for the addressee. Access to this email by anyone else is unauthorized. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and destroy this e-mail. If you are not the intended recipient, any disclosure, copying, distribution or any action taken or omitted to be taken in reliance on it, is prohibited and may be unlawful. Hinweise zum Datenschutz: bitstore.group/datenschutz From tschoening at am-soft.de Sat Feb 26 04:04:24 2022 From: tschoening at am-soft.de (=?utf-8?Q?Thorsten_Sch=C3=B6ning?=) Date: Sat, 26 Feb 2022 10:04:24 +0100 Subject: [Borgbackup] ACLs not preserved In-Reply-To: References: <0486adf9-d801-e070-e16a-7417823f0832@waldmann-edv.de> Message-ID: <1659015970.20220226100424@am-soft.de> Guten Tag d0, am Freitag, 25. Februar 2022 um 23:43 schrieben Sie: > I'm pretty sure sshfs supports neither ACLs nor extended attributes because > sshfs is based on SFTP, which doesn't support any of that (afaik). Note > that OpenSSH implements SFTP v3 and some extensions, not the later drafts. SSHFS isn't used in the setup with ssh-agent, but natively running BORG instances with full access to the client filesystem instead. Mit freundlichen Gr??en Thorsten Sch?ning -- AM-SoFT IT-Service - Bitstore Hameln GmbH Mitglied der Bitstore Gruppe - Ihr Full-Service-Dienstleister f?r IT und TK E-Mail: Thorsten.Schoening at AM-SoFT.de Web: http://www.AM-SoFT.de/ Tel: 05151- 9468- 0 Tel: 05151- 9468-55 Fax: 05151- 9468-88 Mobil: 0178-8 9468-04 AM-SoFT IT-Service - Bitstore Hameln GmbH, Brandenburger Str. 7c, 31789 Hameln AG Hannover HRB 221853 - Gesch?ftsf?hrer: Janine Galonska F?r R?ckfragen stehe ich Ihnen jederzeit zur Verf?gung. Mit freundlichen Gr??en, Thorsten Sch?ning Telefon: +49 (0)515 94 68 - 0 Fax: E-Mail: TSchoening at am-soft.de AM-Soft IT-Service - Bitstore Hameln GmbH Brandenburger Stra?e 7c 31789 Hameln Diese E-Mail enth?lt vertrauliche und/oder rechtlich gesch?tzte Informationen und ist ausschliesslich f?r den Adressaten bestimmt. Jeglicher Zugriff auf diese E-Mail durch andere Personen als den Adressaten ist untersagt. Wenn Sie nicht der richtige Adressat sind oder diese E-Mail irrt?mlich erhalten haben, informieren Sie bitte sofort den Absender und vernichten Sie diese E-Mail. Sollten Sie nicht der f?r diese E-Mail bestimmte Adressat sein, ist Ihnen jede Ver?ffentlichung, Vervielf?ltigung oder Weitergabe wie auch das Ergreifen oder Unterlassen von Massnahmen im Vertrauen auf erlangte Information untersagt. This e-mail may contain confidential and/or privileged information and is intended solely for the addressee. Access to this email by anyone else is unauthorized. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and destroy this e-mail. If you are not the intended recipient, any disclosure, copying, distribution or any action taken or omitted to be taken in reliance on it, is prohibited and may be unlawful. Hinweise zum Datenschutz: bitstore.group/datenschutz From KevinBowrin at Cunet.Carleton.Ca Sat Feb 26 12:27:28 2022 From: KevinBowrin at Cunet.Carleton.Ca (Kevin Bowrin) Date: Sat, 26 Feb 2022 17:27:28 +0000 Subject: [Borgbackup] Congratulations and thank you to the Borg Collective Message-ID: Hello all, Congratulations to the Borg devs for the 1.2.0 release! We're testing out Borg as a backup solution for our systems, where we need long term storage of large files which don't change much. The deduplication feature is saving us a lot of headaches. The streaming tar extract pairs very well with gpg and rclone for off site backups. The append only option for clients is a nice feature for ransomware protection. Installation, config, and deployment were a breeze, and the docs are very well done. Thank you for a great piece of software. Best, Kevin Bowrin Carleton University Ottawa -------------- next part -------------- An HTML attachment was scrubbed... URL: From blackbirdralph at gmail.com Sat Feb 26 21:53:19 2022 From: blackbirdralph at gmail.com (ralph strebbing) Date: Sat, 26 Feb 2022 21:53:19 -0500 Subject: [Borgbackup] ACLs not preserved In-Reply-To: <19410155776.20220226101354@am-soft.de> References: <0486adf9-d801-e070-e16a-7417823f0832@waldmann-edv.de> <20220226024421.41c87458@msi.defcon1.lan> <19410155776.20220226101354@am-soft.de> Message-ID: On Sat, Feb 26, 2022 at 4:22 AM Thorsten Sch?ning wrote: > You should debug further, RSYNC isn't any different than BORG > instances with ssh-agent. In both cases two instances of some reading > app with full access to the filesystem are executed on the > to-be-backed-up client. If you have any specific things I can try, I'm open to further testing and debugging. Borg is a new system for me, but I'm fairly well versed in my systems as a general statement. > That is the difference with SSHFS, which is an additional layer with > additional limitations like unstable INODES, missing ACLs etc. Readers > using that layer don't have access to all data, compared to readers > natively running on the client. But that shouldn't be used in your > setup. If possible, the SSH-Agent process is what I need to have working, as our backups would be started from the remote backup server via cron. > Where does the following backup source comes from, what are the mount > options exactly? Should be some EXT4 file system at filesrv1. You > should provide the mount options and some example output showing that > ACLs are really available. The backup source comes from our file server (filesrv1), running Ubuntu 20.04 that exists as a Proxmox KVM, with 2 Virtual HDDs, one for the OS, the other for data (/storage mount). Both of those however are running simple ext4, which by default Ubuntu mounts with ACL support, but here is the output of `mount`: /dev/sdb1 on /storage type ext4 (rw,relatime) Output of fstab: /dev/sdb1 /storage ext4 defaults 0 0 Output of tune2fs -l /dev/sdb1: http://haste.thegamingcorner.net/odymexyjil.apache The above is proof that my system supports ACLs. Similar output is given on the backup server, but the simple fact is: Just rsyncing the data preserves the ACLs, only when using Borg to the remote server, either via push or pull am I having the issue. > > /storage/netfiles/testfolder > > Your problem reads to me like there's some hiding layer between BORG > and the filesystem. While that might be SSHFS, it shouldn't be, > because SSH-AGENT doesn't use that. > > What happens when you run your backups multiple times after another? > Check the output for modified files, as if all of those are modified > over and over again, that's another hint for some layer like SSHFS > with unstable INODES between backups. The files are marked as unmodified if they are as such when running backups multiple times on different archives. From tschoening at am-soft.de Sun Feb 27 16:05:04 2022 From: tschoening at am-soft.de (=?utf-8?Q?Thorsten_Sch=C3=B6ning?=) Date: Sun, 27 Feb 2022 22:05:04 +0100 Subject: [Borgbackup] ACLs not preserved In-Reply-To: References: <0486adf9-d801-e070-e16a-7417823f0832@waldmann-edv.de> <20220226024421.41c87458@msi.defcon1.lan> <19410155776.20220226101354@am-soft.de> Message-ID: <937906403.20220227220504@am-soft.de> Guten Tag ralph strebbing, am Sonntag, 27. Februar 2022 um 03:53 schrieben Sie: > http://haste.thegamingcorner.net/odymexyjil.apache > The above is proof that my system supports ACLs. Agreed. > Similar output is > given on the backup server, but the simple fact is: Just rsyncing the > data preserves the ACLs, only when using Borg to the remote server, > either via push or pull am I having the issue. You were talking about testing with a local BORG repo preserving ACLs in some former mail. How exactly did you test that? At filesrv1, so with the same version of BORG etc. like in PUSH/PULL, only with a local repo and one running BORG instance in that case? Or somewhere else only to see if BORG keeps ACLs at all? The thing is that the BORG instance on filesrv1 needs to be able to read the ACLs. When that works with a local repo on filesrv1, there should be something really strange going on. From my understanding of how BORG works, the instance in the client is doing all the heavy stuff like knowing about metadata, ACLs etc. and only forwards encrypted data to the BORG instance "hosting" the remote repo using "borg serve". How should that be able to not store ACLs, when it only gets encrypted data of something? AFAIK metadata and file contents are even handled the same in the end, regarding de-duplication, compression and stuff. "borg serve" shouldn't be able to distinguish any of that. Mit freundlichen Gr??en Thorsten Sch?ning -- AM-SoFT IT-Service - Bitstore Hameln GmbH Mitglied der Bitstore Gruppe - Ihr Full-Service-Dienstleister f?r IT und TK E-Mail: Thorsten.Schoening at AM-SoFT.de Web: http://www.AM-SoFT.de/ Tel: 05151- 9468- 0 Tel: 05151- 9468-55 Fax: 05151- 9468-88 Mobil: 0178-8 9468-04 AM-SoFT IT-Service - Bitstore Hameln GmbH, Brandenburger Str. 7c, 31789 Hameln AG Hannover HRB 221853 - Gesch?ftsf?hrer: Janine Galonska F?r R?ckfragen stehe ich Ihnen jederzeit zur Verf?gung. Mit freundlichen Gr??en, Thorsten Sch?ning Telefon: +49 (0)515 94 68 - 0 Fax: E-Mail: TSchoening at am-soft.de AM-Soft IT-Service - Bitstore Hameln GmbH Brandenburger Stra?e 7c 31789 Hameln Diese E-Mail enth?lt vertrauliche und/oder rechtlich gesch?tzte Informationen und ist ausschliesslich f?r den Adressaten bestimmt. Jeglicher Zugriff auf diese E-Mail durch andere Personen als den Adressaten ist untersagt. Wenn Sie nicht der richtige Adressat sind oder diese E-Mail irrt?mlich erhalten haben, informieren Sie bitte sofort den Absender und vernichten Sie diese E-Mail. Sollten Sie nicht der f?r diese E-Mail bestimmte Adressat sein, ist Ihnen jede Ver?ffentlichung, Vervielf?ltigung oder Weitergabe wie auch das Ergreifen oder Unterlassen von Massnahmen im Vertrauen auf erlangte Information untersagt. This e-mail may contain confidential and/or privileged information and is intended solely for the addressee. Access to this email by anyone else is unauthorized. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and destroy this e-mail. If you are not the intended recipient, any disclosure, copying, distribution or any action taken or omitted to be taken in reliance on it, is prohibited and may be unlawful. Hinweise zum Datenschutz: bitstore.group/datenschutz From alexander_goetzenstein at web.de Mon Feb 28 06:16:02 2022 From: alexander_goetzenstein at web.de (Alexander Goetzenstein) Date: Mon, 28 Feb 2022 12:16:02 +0100 Subject: [Borgbackup] Borgmatic reports failure Message-ID: <165191f1-bfea-79c0-0ec5-fbf72b29299b@alexander-goetzenstein.my-fqdn.de> Hi, since the last automatic update of borgbackup (1.1.17-5), borgmatic (1.5.17-2.4) and vorta (0.8.3-1.1) via YAST, some day (not every day) I get a popup saying > Repository data check for repo /home/alex/Dokumente/temp/BorgBackup/ failed. Error code 2 > /home/alex/Dokumente/temp/BorgBackup is not a valid repository. Check repo config. > Consider repairing or recreating the repository soon to avoid missing data. I have no clue what I have to do. Could someone please give me a hint? -- Gru? Alex From blackbirdralph at gmail.com Mon Feb 28 09:44:09 2022 From: blackbirdralph at gmail.com (ralph strebbing) Date: Mon, 28 Feb 2022 09:44:09 -0500 Subject: [Borgbackup] ACLs not preserved In-Reply-To: <937906403.20220227220504@am-soft.de> References: <0486adf9-d801-e070-e16a-7417823f0832@waldmann-edv.de> <20220226024421.41c87458@msi.defcon1.lan> <19410155776.20220226101354@am-soft.de> <937906403.20220227220504@am-soft.de> Message-ID: On Sun, Feb 27, 2022 at 4:05 PM Thorsten Sch?ning wrote: > You were talking about testing with a local BORG repo preserving ACLs > in some former mail. How exactly did you test that? At filesrv1, so > with the same version of BORG etc. like in PUSH/PULL, only with a > local repo and one running BORG instance in that case? Or somewhere > else only to see if BORG keeps ACLs at all? Correct, I tested with a local repo on filesrv1 in another directory. > The thing is that the BORG instance on filesrv1 needs to be able to > read the ACLs. When that works with a local repo on filesrv1, there > should be something really strange going on. From my understanding of > how BORG works, the instance in the client is doing all the heavy > stuff like knowing about metadata, ACLs etc. and only forwards > encrypted data to the BORG instance "hosting" the remote repo using > "borg serve". > > How should that be able to not store ACLs, when it only gets encrypted > data of something? AFAIK metadata and file contents are even handled > the same in the end, regarding de-duplication, compression and stuff. > "borg serve" shouldn't be able to distinguish any of that. Well that's where I'm hung up honestly. We currently only rsync the data from the systems to our backup server, and that preserves our ACLs just fine, but ZFS has some issues, so we're looking to move back to straight ext4 and use borg to enable encryption among some of its other features. Ralph From jdc at uwo.ca Mon Feb 28 10:24:29 2022 From: jdc at uwo.ca (Dan Christensen) Date: Mon, 28 Feb 2022 15:24:29 +0000 Subject: [Borgbackup] ACLs not preserved In-Reply-To: (ralph strebbing's message of "Mon, 28 Feb 2022 09:44:09 -0500") References: <0486adf9-d801-e070-e16a-7417823f0832@waldmann-edv.de> <20220226024421.41c87458@msi.defcon1.lan> <19410155776.20220226101354@am-soft.de> <937906403.20220227220504@am-soft.de> Message-ID: <87wnhfxbde.fsf@uwo.ca> This is almost certainly something happening at the "borg extract" step not the "borg create" step. I believe the OP said that the problem occurs when extracting on the remote system from the remote repository. I'm guessing that if you extract on the local system but from the remote repository, you ACLs will be present, but the OP should test this. When the issue occurs (extracting on the remote system), does the filesystem support ACLs, and do the right users exist on that system? Dan On Feb 28, 2022, ralph strebbing wrote: > On Sun, Feb 27, 2022 at 4:05 PM Thorsten Sch?ning wrote: >> You were talking about testing with a local BORG repo preserving ACLs >> in some former mail. How exactly did you test that? At filesrv1, so >> with the same version of BORG etc. like in PUSH/PULL, only with a >> local repo and one running BORG instance in that case? Or somewhere >> else only to see if BORG keeps ACLs at all? > Correct, I tested with a local repo on filesrv1 in another directory. > >> The thing is that the BORG instance on filesrv1 needs to be able to >> read the ACLs. When that works with a local repo on filesrv1, there >> should be something really strange going on. From my understanding of >> how BORG works, the instance in the client is doing all the heavy >> stuff like knowing about metadata, ACLs etc. and only forwards >> encrypted data to the BORG instance "hosting" the remote repo using >> "borg serve". >> >> How should that be able to not store ACLs, when it only gets encrypted >> data of something? AFAIK metadata and file contents are even handled >> the same in the end, regarding de-duplication, compression and stuff. >> "borg serve" shouldn't be able to distinguish any of that. > Well that's where I'm hung up honestly. We currently only rsync the > data from the systems to our backup server, and that preserves our > ACLs just fine, but ZFS has some issues, so we're looking to move back > to straight ext4 and use borg to enable encryption among some of its > other features. > > Ralph > _______________________________________________ > Borgbackup mailing list > Borgbackup at python.org > https://mail.python.org/mailman/listinfo/borgbackup From tkpapp at gmail.com Sat Mar 5 09:35:48 2022 From: tkpapp at gmail.com (Tamas Papp) Date: Sat, 05 Mar 2022 15:35:48 +0100 Subject: [Borgbackup] best practices for repo access hardcopy Message-ID: <87sfrwfo2y.fsf@gmail.com> Hi, I am wondering what others do to back up repository access information as hardcopy. I am using a borgbase remote repository. Currently I print 1. the borgmatic .yaml file and 2. the ssh key on paper, expecting to retype it in case a disaster happens to electronic copies (which I keep on a USB stick). I thought of 1. stripping the YAML comments, 2. making a tar.gz archive of the above, 3. printing it as a large QR code (with as much redundancy as allowed by the limits of QR). Suggestions appreciated. best, Tamas From lazyvirus at gmx.com Sat Mar 5 10:35:35 2022 From: lazyvirus at gmx.com (Bzzzz) Date: Sat, 5 Mar 2022 16:35:35 +0100 Subject: [Borgbackup] best practices for repo access hardcopy In-Reply-To: <87sfrwfo2y.fsf@gmail.com> References: <87sfrwfo2y.fsf@gmail.com> Message-ID: <20220305163535.0c27324e@msi.defcon1.lan> On Sat, 05 Mar 2022 15:35:48 +0100 Tamas Papp wrote: > Hi, Hi, > I am wondering what others do to back up repository access information > as hardcopy. > > I am using a borgbase remote repository. Currently I print > > 1. the borgmatic .yaml file and > 2. the ssh key > > on paper, expecting to retype it in case a disaster happens to > electronic copies (which I keep on a USB stick). > > I thought of > > 1. stripping the YAML comments, > > 2. making a tar.gz archive of the above, > > 3. printing it as a large QR code (with as much redundancy as allowed > by the limits of QR). > > Suggestions appreciated. I made bash scripts for almost every Borg order I need and embedded all of that into each script (repo key, password and SSH key) - of course all belonging to root, *.sh in /usr/local/sbin, with 700 permissions. These scripts are replicated on a USB stick that stays in my pocket and kept up to date when modified, the originals too are on this stick (key file, SSH files and password as a text file, just in case). The redundancy is enough to my eyes but this is for a house setup, for a company, this would need 3 USB sticks carried by 3 different people. Jean-Yves From tschoening at am-soft.de Sat Mar 5 11:15:37 2022 From: tschoening at am-soft.de (=?windows-1250?Q?Thorsten_Sch=F6ning?=) Date: Sat, 5 Mar 2022 17:15:37 +0100 Subject: [Borgbackup] best practices for repo access hardcopy In-Reply-To: <87sfrwfo2y.fsf@gmail.com> References: <87sfrwfo2y.fsf@gmail.com> Message-ID: <509959680.20220305171537@am-soft.de> Guten Tag Tamas Papp, am Samstag, 5. M?rz 2022 um 15:35 schrieben Sie: > 1. the borgmatic .yaml file and Configs belong into SCMs like GIT or SVN in my opinion, which easily allows to have additional backups and/or clones/working copies at admin/developer PCs. My data is (mostly) backed up offsite using RSYNC to some private NAS as well. > 2. the ssh key I additionally store those in KeePass, which is 1. stored on a company share and using triggers additionally privately at the PC of whoever updates that database. > on paper, expecting to retype it in case a disaster happens to > electronic copies (which I keep on a USB stick). Paper is not very reliable, can't exist easily multiple times at multiple places etc. I wouldn't rely on that and spread files instead. Mit freundlichen Gr??en Thorsten Sch?ning -- AM-SoFT IT-Service - Bitstore Hameln GmbH Mitglied der Bitstore Gruppe - Ihr Full-Service-Dienstleister f?r IT und TK E-Mail: Thorsten.Schoening at AM-SoFT.de Web: http://www.AM-SoFT.de/ Tel: 05151- 9468- 0 Tel: 05151- 9468-55 Fax: 05151- 9468-88 Mobil: 0178-8 9468-04 AM-SoFT IT-Service - Bitstore Hameln GmbH, Brandenburger Str. 7c, 31789 Hameln AG Hannover HRB 221853 - Gesch?ftsf?hrer: Janine Galonska F?r R?ckfragen stehe ich Ihnen jederzeit zur Verf?gung. Mit freundlichen Gr??en, Thorsten Sch?ning Telefon: +49 (0)515 94 68 - 0 Fax: E-Mail: TSchoening at am-soft.de AM-Soft IT-Service - Bitstore Hameln GmbH Brandenburger Stra?e 7c 31789 Hameln Diese E-Mail enth?lt vertrauliche und/oder rechtlich gesch?tzte Informationen und ist ausschliesslich f?r den Adressaten bestimmt. Jeglicher Zugriff auf diese E-Mail durch andere Personen als den Adressaten ist untersagt. Wenn Sie nicht der richtige Adressat sind oder diese E-Mail irrt?mlich erhalten haben, informieren Sie bitte sofort den Absender und vernichten Sie diese E-Mail. Sollten Sie nicht der f?r diese E-Mail bestimmte Adressat sein, ist Ihnen jede Ver?ffentlichung, Vervielf?ltigung oder Weitergabe wie auch das Ergreifen oder Unterlassen von Massnahmen im Vertrauen auf erlangte Information untersagt. This e-mail may contain confidential and/or privileged information and is intended solely for the addressee. Access to this email by anyone else is unauthorized. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and destroy this e-mail. If you are not the intended recipient, any disclosure, copying, distribution or any action taken or omitted to be taken in reliance on it, is prohibited and may be unlawful. Hinweise zum Datenschutz: bitstore.group/datenschutz From mail at tgries.de Sat Mar 5 11:21:55 2022 From: mail at tgries.de (Thomas Gries) Date: Sat, 5 Mar 2022 17:21:55 +0100 Subject: [Borgbackup] best practices for repo access hardcopy In-Reply-To: <87sfrwfo2y.fsf@gmail.com> References: <87sfrwfo2y.fsf@gmail.com> Message-ID: Am 05.03.22 um 15:35 schrieb Tamas Papp: > Hi, > > I am wondering what others do to back up repository access information > as hardcopy. > > I am using a borgbase remote repository. Currently I print > > 1. the borgmatic .yaml file and > 2. the ssh key > > on paper, expecting to retype it in case a disaster happens to > electronic copies (which I keep on a USB stick). > > I thought of > > 1. stripping the YAML comments, > > 2. making a tar.gz archive of the above, > > 3. printing it as a large QR code (with as much redundancy as allowed by > the limits of QR). > > Suggestions appreciated. > > best, > > Tamas > _______________________________________________ > Borgbackup mailing list > Borgbackup at python.org > https://mail.python.org/mailman/listinfo/borgbackup See https://borgbackup.readthedocs.io/en/stable/paperkey.html To create a printable key, either paste the contents of your keyfile or a key export in the text field below, or select a key export file. To create a key export use borg key export /path/to/repository exportfile.txt If you are using keyfile mode, keyfiles are usually stored in $HOME/.config/borg/keys/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From mail at tgries.de Sat Mar 5 11:35:37 2022 From: mail at tgries.de (Thomas Gries) Date: Sat, 5 Mar 2022 17:35:37 +0100 Subject: [Borgbackup] best practices for repo access hardcopy In-Reply-To: <87sfrwfo2y.fsf@gmail.com> References: <87sfrwfo2y.fsf@gmail.com> Message-ID: Am 05.03.22 um 15:35 schrieb Tamas Papp: > Hi, > > I am wondering what others do to back up repository access information > as hardcopy. > > I am using a borgbase remote repository. Currently I print > > 1. the borgmatic .yaml file and > 2. the ssh key > > on paper, expecting to retype it in case a disaster happens to > electronic copies (which I keep on a USB stick). > > I thought of > > 1. stripping the YAML comments, > > 2. making a tar.gz archive of the above, > > 3. printing it as a large QR code (with as much redundancy as allowed by > the limits of QR). > > Suggestions appreciated. > > best, > > Tamas > _______________________________________________ > Borgbackup mailing list > Borgbackup at python.org > https://mail.python.org/mailman/listinfo/borgbackup https://borgbackup.readthedocs.io/en/stable/usage/key.html also lists the --qr-html option There are three backup formats. The normal backup format is suitable for digital storage as a file. The |--paper| backup format is optimized for printing and typing in while importing, with per line checks to reduce problems with manual input. The |--qr-html| creates a printable HTML template with a QR code and a copy of the |--paper|-formatted key. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: 5DZXTVEpLjY7DJMs.jpeg Type: image/jpeg Size: 25320 bytes Desc: not available URL: From clickwir at gmail.com Sat Mar 5 16:12:39 2022 From: clickwir at gmail.com (Zack Coffey) Date: Sat, 5 Mar 2022 14:12:39 -0700 Subject: [Borgbackup] best practices for repo access hardcopy In-Reply-To: References: <87sfrwfo2y.fsf@gmail.com> Message-ID: You can print it, sure. Retyping works, but is less than ideal. What I'd prefer, 7zip/zip it with a password. You've now got a small file that's encrypted. Much safer, more portable and much easier to restore from. You can email it to yourself, maybe across multiple services. You can put it on a USB drive or a few. Leave one at work or a families house. Upload it on a cloud storage, Google Drive for example. On Sat, Mar 5, 2022 at 9:35 AM Thomas Gries wrote: > Am 05.03.22 um 15:35 schrieb Tamas Papp: > > Hi, > > I am wondering what others do to back up repository access information > as hardcopy. > > I am using a borgbase remote repository. Currently I print > > 1. the borgmatic .yaml file and > 2. the ssh key > > on paper, expecting to retype it in case a disaster happens to > electronic copies (which I keep on a USB stick). > > I thought of > > 1. stripping the YAML comments, > > 2. making a tar.gz archive of the above, > > 3. printing it as a large QR code (with as much redundancy as allowed by > the limits of QR). > > Suggestions appreciated. > > best, > > Tamas > _______________________________________________ > Borgbackup mailing listBorgbackup at python.orghttps://mail.python.org/mailman/listinfo/borgbackup > > > https://borgbackup.readthedocs.io/en/stable/usage/key.html > > also lists the --qr-html option > > There are three backup formats. > > The normal backup format is suitable for digital storage as a file. > The --paper backup format is optimized for printing and typing in while > importing, with per line checks to reduce problems with manual input. > The --qr-html creates a printable HTML template with a QR code and a copy > of the --paper-formatted key. > _______________________________________________ > Borgbackup mailing list > Borgbackup at python.org > https://mail.python.org/mailman/listinfo/borgbackup > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: 5DZXTVEpLjY7DJMs.jpeg Type: image/jpeg Size: 25320 bytes Desc: not available URL: From philippe.milink at gmx.fr Sun Mar 13 14:14:58 2022 From: philippe.milink at gmx.fr (Philippe MILINK) Date: Sun, 13 Mar 2022 19:14:58 +0100 Subject: [Borgbackup] Prune encrypted repository without the key Message-ID: Hello, Is it possible to prune an encrypted repository without having its key (and/or passphrase: the credentials) ? Here is my configuration: - a server A backups its data with borg to servers owned by third-party members. To prevent third-party members reading the data in the backup, the borg repository is encrypted (borg init -e repokey) and members do not know the passphrase. - servers of third-party members are configured as described in the documentation (https://borgbackup.readthedocs.io/en/stable/deployment/hosting-repositories.html), with the --append-only options added to borg serve (thus if an attacker has access to the server A, he cannot delete backups). However, I would like to let the "storage policy" to the third-party members: if a member has only few storage space, he will only keep the backups of the 7 last days; if a member has lot of storage space, he can keep the last two months of backups. This supposes to let the third-party members run the borg prune command from their servers, on the repository which is encrypted. I just discovered this isn't possible, because borg prune asks for the passphrase... (and it makes sense: to prevent malicious removing from the repository host...) I cannot run the borg prune commands from the server A, because of the --append-only option. Is there a way to do this ? Thanks, Philippe. From tw at waldmann-edv.de Sun Mar 13 16:49:43 2022 From: tw at waldmann-edv.de (Thomas Waldmann) Date: Sun, 13 Mar 2022 21:49:43 +0100 Subject: [Borgbackup] Prune encrypted repository without the key In-Reply-To: References: Message-ID: > Is it possible to prune an encrypted repository without having its key > (and/or passphrase: the credentials) ? borg prune: no borg compact: yes starting with borg 1.2, the compaction is separate from the pruning. for borg < 1.2, compaction is done at the end of borg prune. > with the --append-only options added to borg serve (thus if an attacker > has access to the server A, he cannot delete backups). > > However, I would like to let the "storage policy" to the third-party > members: if a member has only few storage space, he will only keep the > backups of the 7 last days; if a member has lot of storage space, he can > keep the last two months of backups. This supposes to let the > third-party members run the borg prune command from their servers, on > the repository which is encrypted. If you can't read the repo, how would you be able to decide what to delete? It's all encrypted. Also, if you use append only, borg prune will only logically delete archives, but not free space. borg compact will just not do anything.