ANN hashtar 0.1: archival encryption to corruptible media

[John Hunter]

hashtar is a utility designed for encrypted archiving to media
vulnerable to corruption (eg, CDR, DVDR).

  http://nitace.bsd.uchicago.edu:8080/hashtar

Comments, bug reports, suggestions for improvement all welcome.

John Hunter

OVERVIEW

  hashtar: an encrypted archive utility designed for secure archiving
  to media vulnerable to corruption.

  Recursively encrypt the files and directories passed as arguments.
  Rather than preserving the directory structure, or archiving to a
  single file as in tar, the files are encrypted to a single dir and
  named with the hash of their relative path.  The file information
  (filename, hash, permission mode, uid, gid) is encrypted and stored
  in the header of the file itself, and can be used to restore the
  original file with dir structure from the archive file.  

  For example, the command

    > hashtar.py -cvf tmp.htar finance/

  prompts for a password and generates an encrypted recursive archive
  of the finance dir in the tmp.htar dir, with filenames mapped like

    finance/irs/98/f1040.pdf -> tmp.htar/e5/e5ed546c0bc0191d80d791bc2f73c890
    finance/sale_house/notes -> tmp.htar/58/580e89bad7563ae76c295f75aecea030
    finance/online/accounts.gz.mcr -> tmp.htar/bb/bbf12f06dc3fcee04067d40b9781f4a8
    finance/phone/prepaid1242.doc -> tmp.htar/c1/c1fe52a9d8cbef55eff8840d379d972a

  The encrypted files are placed in subdirs based on the first two
  characters in their hash name because if too many files are placed
  in one dir, it may not be possible to pass all of them as command
  line arguments to the restore command.  The entire finance dir
  structure can later be restored with
 
    > hashtar.py -xvf tmp.htar

  The advantage of this method of encrypted archiving, as opposed to
  archiving to a single tar file and encrypting it, is that this
  method is not sensitive to single byte corruption, which becomes
  important especially on externally stored archives, such as on CDR,
  or DVDR.  Any individual file contains all the information needed to
  restore itself, with directory structure, permission bits, etc.  So
  only the specific files that are corrupted on the media will be
  lost.

  The alternative strategy, encrypting all the files in place and then
  archiving to external media, doesn't suffer from single byte
  corruption but affords less privacy since the filenames, dir
  structure, and permission bits are available, and less security
  since a filename may indicate contents and thus expose the archive
  to a known plaintext attack.

  A match string allows you to only extract files matching a given
  pattern.  Eg, to only extract pdf and xls files, do

   > hashtar.py -m pdf,xls -xvf tmp.htar

  Because the filenames are stored in the header, only a small portion
  of the file needs to be decrypted to determine the match, so this is
  quite fast.

  Data can be encrypted and decrypted across platforms (tested between
  linux and win32 and vice-versa) but of course some information may
  be lost, such as uid, gid for platforms that don't support it.
  
USAGE:

  > hashtar.py [OPTIONS] files

  OPTIONS

  -h, --help            Show  help message and exit
  -fDIR, --arcdir=DIR   Write hashed filenames to archive dir
  -pFILE, --passwdfile=FILE
                        Get passwd from FILE, otherwise prompt
  -mPATTERN, --match=PATTERN
                        Only extract files that match PATTERN.
                        PATTERN is a comma separated list of strings,
                        one of which must match the filename
  -u, --unlink          Delete files after archiving them
  -c, --create          Create archive dir
  -x, --extract         Extract files recursively from archive dir
  -v, --verbose         Decrypt files recursively

WARNING:

  I think this software is suitable to protect your data from your
  sister, your boss, and even the nosy computer hacker next door, but
  not the NSA.

REQUIREMENTS:

  python2.3               - python.org
  yawPyCrypto and Flatten - http://yawpycrypto.sourceforge.net/
  pycrypto                - http://www.amk.ca/python/code/crypto.html

  The python dependencies are very easy to install; just do the usual
  > python setup.py install

  
PLATFORMS:

  Tested on linux and win32

AUTHOR:

  John D. Hunter <jdhunter at ace.bsd.uchicago.edu>

LICENSE:

  same as python2.3

KNOWN BUGS:

  Ignores symbolic links

DEDICATION:

  For Erik Curiel, who's life's work I lost when I volunteered to
  backup the only copy of his home dir on a CD containing a single
  encrypted gzipped tar file, which was subsequently corrupted.