[Borgbackup] 550GB rsync-tree will not dump

Tue Aug 9 04:06:45 EDT 2016

Op 08-08-16 om 16:41 schreef Adrian Klaver:

> What is the error when you use no compression?

When not using compression I get a Data integrity error and a unusable repository:

553.53 GB O 553.75 GB C 319.15 GB D 4585967 N data/backup/.zfs/snapshot/borg/rsnapshot/1daily.0/hosts_ssh/...
Data integrity error

Traceback (most recent call last):
   File "/usr/local/lib/python3.4/site-packages/borg/archiver.py", line 1609, in main
     exit_code = archiver.run(args)
   File "/usr/local/lib/python3.4/site-packages/borg/archiver.py", line 1546, in run
     return args.func(args)
   File "/usr/local/lib/python3.4/site-packages/borg/archiver.py", line 81, in wrapper
     return method(self, args, repository=repository, **kwargs)
   File "/usr/local/lib/python3.4/site-packages/borg/archiver.py", line 247, in do_create
     create_inner(archive, cache)
   File "/usr/local/lib/python3.4/site-packages/borg/archiver.py", line 219, in create_inner
     read_special=args.read_special, dry_run=dry_run)
   File "/usr/local/lib/python3.4/site-packages/borg/archiver.py", line 301, in _process
     read_special=read_special, dry_run=dry_run)
   File "/usr/local/lib/python3.4/site-packages/borg/archiver.py", line 301, in _process
     read_special=read_special, dry_run=dry_run)
   File "/usr/local/lib/python3.4/site-packages/borg/archiver.py", line 301, in _process
     read_special=read_special, dry_run=dry_run)
   File "/usr/local/lib/python3.4/site-packages/borg/archiver.py", line 301, in _process
     read_special=read_special, dry_run=dry_run)
   File "/usr/local/lib/python3.4/site-packages/borg/archiver.py", line 301, in _process
     read_special=read_special, dry_run=dry_run)
   File "/usr/local/lib/python3.4/site-packages/borg/archiver.py", line 301, in _process
     read_special=read_special, dry_run=dry_run)
   File "/usr/local/lib/python3.4/site-packages/borg/archiver.py", line 301, in _process
     read_special=read_special, dry_run=dry_run)
   File "/usr/local/lib/python3.4/site-packages/borg/archiver.py", line 301, in _process
     read_special=read_special, dry_run=dry_run)
   File "/usr/local/lib/python3.4/site-packages/borg/archiver.py", line 301, in _process
     read_special=read_special, dry_run=dry_run)
   File "/usr/local/lib/python3.4/site-packages/borg/archiver.py", line 301, in _process
     read_special=read_special, dry_run=dry_run)
   File "/usr/local/lib/python3.4/site-packages/borg/archiver.py", line 301, in _process
     read_special=read_special, dry_run=dry_run)
   File "/usr/local/lib/python3.4/site-packages/borg/archiver.py", line 301, in _process
     read_special=read_special, dry_run=dry_run)
   File "/usr/local/lib/python3.4/site-packages/borg/archiver.py", line 301, in _process
     read_special=read_special, dry_run=dry_run)
   File "/usr/local/lib/python3.4/site-packages/borg/archiver.py", line 301, in _process
     read_special=read_special, dry_run=dry_run)
   File "/usr/local/lib/python3.4/site-packages/borg/archiver.py", line 301, in _process
     read_special=read_special, dry_run=dry_run)
   File "/usr/local/lib/python3.4/site-packages/borg/archiver.py", line 275, in _process
     status = archive.process_file(path, st, cache, self.ignore_inode)
   File "/usr/local/lib/python3.4/site-packages/borg/archive.py", line 638, in process_file
     self.add_item(item)
   File "/usr/local/lib/python3.4/site-packages/borg/archive.py", line 281, in add_item
     self.write_checkpoint()
   File "/usr/local/lib/python3.4/site-packages/borg/archive.py", line 285, in write_checkpoint
     self.save(self.checkpoint_name)
   File "/usr/local/lib/python3.4/site-packages/borg/archive.py", line 317, in save
     self.repository.commit()
   File "/usr/local/lib/python3.4/site-packages/borg/repository.py", line 199, in commit
     self.compact_segments(save_space=save_space)
   File "/usr/local/lib/python3.4/site-packages/borg/repository.py", line 287, in compact_segments
     for tag, key, offset, data in self.io.iter_objects(segment, include_data=True):
   File "/usr/local/lib/python3.4/site-packages/borg/repository.py", line 648, in iter_objects
     (TAG_PUT, TAG_DELETE, TAG_COMMIT))
   File "/usr/local/lib/python3.4/site-packages/borg/repository.py", line 704, in _read
     segment, offset))
borg.helpers.IntegrityError: Invalid segment entry size [segment 48441, offset 4210392]

Platform: FreeBSD sanger 10.3-RELEASE-p2 FreeBSD 10.3-RELEASE-p2 #0: Wed May  4 06:03:51 UTC 2016     root at amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC amd64 amd64
Borg: 1.0.6  Python: CPython 3.4.5
PID: 44736  CWD: /usr/home/sysman
sys.argv: ['/usr/local/bin/borg', 'create', '--debug', '--exclude', '*/borg/rsnapshot*/?monthly.*', '--progress', '--stats', '--verbose', '::data_backup at _data_backup_.zfs_snapshot_borg', '/data/backup/.zfs/snapshot/borg']
SSH_ORIGINAL_COMMAND: None

zfsborg_create :: list archive(s)
using builtin fallback logging configuration
Replaying segments   0%
Replaying segments   5%
Replaying segments  10%
Replaying segments  15%
Replaying segments  20%
Replaying segments  25%
Replaying segments  30%
Replaying segments  35%
Replaying segments  40%
Replaying segments  45%
Replaying segments  50%
Replaying segments  55%
Replaying segments  60%
Replaying segments  65%
Replaying segments  70%
Replaying segments  75%
Replaying segments  80%
Replaying segments  85%
Replaying segments  90%
Replaying segments  95%
Data integrity error

Traceback (most recent call last):
   File "/usr/local/lib/python3.4/site-packages/borg/archiver.py", line 1609, in main
     exit_code = archiver.run(args)
   File "/usr/local/lib/python3.4/site-packages/borg/archiver.py", line 1546, in run
     return args.func(args)
   File "/usr/local/lib/python3.4/site-packages/borg/archiver.py", line 75, in wrapper
     kwargs['manifest'], kwargs['key'] = Manifest.load(repository)
   File "/usr/local/lib/python3.4/site-packages/borg/helpers.py", line 106, in load
     cdata = repository.get(cls.MANIFEST_ID)
   File "/usr/local/lib/python3.4/site-packages/borg/repository.py", line 458, in get
     self.index = self.open_index(self.get_transaction_id())
   File "/usr/local/lib/python3.4/site-packages/borg/repository.py", line 160, in get_transaction_id
     self.replay_segments(replay_from, segments_transaction_id)
   File "/usr/local/lib/python3.4/site-packages/borg/repository.py", line 322, in replay_segments
     self._update_index(segment, objects)
   File "/usr/local/lib/python3.4/site-packages/borg/repository.py", line 331, in _update_index
     for tag, key, offset in objects:
   File "/usr/local/lib/python3.4/site-packages/borg/repository.py", line 648, in iter_objects
     (TAG_PUT, TAG_DELETE, TAG_COMMIT))
   File "/usr/local/lib/python3.4/site-packages/borg/repository.py", line 704, in _read
     segment, offset))
borg.helpers.IntegrityError: Invalid segment entry size [segment 48570, offset 316616]

Platform: FreeBSD sanger 10.3-RELEASE-p2 FreeBSD 10.3-RELEASE-p2 #0: Wed May  4 06:03:51 UTC 2016     root at amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC amd64 amd64
Borg: 1.0.6  Python: CPython 3.4.5
PID: 58353  CWD: /usr/home/sysman
sys.argv: ['/usr/local/bin/borg', 'list', '--debug', '::']
SSH_ORIGINAL_COMMAND: None