hashtar is a utility designed for encrypted archiving to media vulnerable to corruption (eg, CDR, DVDR).

http://nitace.bsd.uchicago.edu:8080/hashtar

Comments, bug reports, suggestions for improvement all welcome.

John Hunter

OVERVIEW

hashtar: an encrypted archive utility designed for secure archiving to media vulnerable to corruption.

Recursively encrypt the files and directories passed as arguments. Rather than preserving the directory structure, or archiving to a single file as in tar, the files are encrypted to a single dir and named with the hash of their relative path. The file information (filename, hash, permission mode, uid, gid) is encrypted and stored in the header of the file itself, and can be used to restore the original file with dir structure from the archive file.

For example, the command

> hashtar.py -cvf tmp.htar finance/

prompts for a password and generates an encrypted recursive archive of the finance dir in the tmp.htar dir, with filenames mapped like

finance/irs/98/f1040.pdf -> tmp.htar/e5/e5ed546c0bc0191d80d791bc2f73c890 finance/sale_house/notes -> tmp.htar/58/580e89bad7563ae76c295f75aecea030 finance/online/accounts.gz.mcr -> tmp.htar/bb/bbf12f06dc3fcee04067d40b9781f4a8 finance/phone/prepaid1242.doc -> tmp.htar/c1/c1fe52a9d8cbef55eff8840d379d972a

The encrypted files are placed in subdirs based on the first two characters in their hash name because if too many files are placed in one dir, it may not be possible to pass all of them as command line arguments to the restore command. The entire finance dir structure can later be restored with

> hashtar.py -xvf tmp.htar

The advantage of this method of encrypted archiving, as opposed to archiving to a single tar file and encrypting it, is that this method is not sensitive to single byte corruption, which becomes important especially on externally stored archives, such as on CDR, or DVDR. Any individual file contains all the information needed to restore itself, with directory structure, permission bits, etc. So only the specific files that are corrupted on the media will be lost.

The alternative strategy, encrypting all the files in place and then archiving to external media, doesn't suffer from single byte corruption but affords less privacy since the filenames, dir structure, and permission bits are available, and less security since a filename may indicate contents and thus expose the archive to a known plaintext attack.

A match string allows you to only extract files matching a given pattern. Eg, to only extract pdf and xls files, do

hashtar.py -m pdf,xls -xvf tmp.htar

Because the filenames are stored in the header, only a small portion of the file needs to be decrypted to determine the match, so this is quite fast.

Data can be encrypted and decrypted across platforms (tested between linux and win32 and vice-versa) but of course some information may be lost, such as uid, gid for platforms that don't support it.

USAGE:

hashtar.py [OPTIONS] files

OPTIONS

-h, --help Show help message and exit -fDIR, --arcdir=DIR Write hashed filenames to archive dir -pFILE, --passwdfile=FILE Get passwd from FILE, otherwise prompt -mPATTERN, --match=PATTERN Only extract files that match PATTERN. PATTERN is a comma separated list of strings, one of which must match the filename -u, --unlink Delete files after archiving them -c, --create Create archive dir -x, --extract Extract files recursively from archive dir -v, --verbose Decrypt files recursively

WARNING:

I think this software is suitable to protect your data from your sister, your boss, and even the nosy computer hacker next door, but not the NSA.

REQUIREMENTS:

python2.3 - python.org yawPyCrypto and Flatten - http://yawpycrypto.sourceforge.net/ pycrypto - http://www.amk.ca/python/code/crypto.html

The python dependencies are very easy to install; just do the usual

python setup.py install

PLATFORMS:

Tested on linux and win32

AUTHOR:

John D. Hunter jdhunter@ace.bsd.uchicago.edu

LICENSE:

same as python2.3

KNOWN BUGS:

Ignores symbolic links

DEDICATION:

For Erik Curiel, who's life's work I lost when I volunteered to backup the only copy of his home dir on a CD containing a single encrypted gzipped tar file, which was subsequently corrupted.