[ python-Bugs-1543303 ] tarfile in mode w|gz adds padding that annoys gunzip

[SourceForge.net]

Bugs item #1543303, was opened at 2006-08-19 22:48
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1543303&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Python Library
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Submitted By: alexis (asak)
Assigned to: Nobody/Anonymous (nobody)
Summary: tarfile in mode w|gz adds padding that annoys gunzip

Initial Comment:
In mode w|gz tarfile pads the final block with NULs,
until its size reaches the bufsize value passed to
tarfile.open.  This makes gunzip complain about
"invalid compressed data" because of CRC and length errors.

To reproduce it, put this fragment in a file archive.py


import sys
import tarfile

tar = tarfile.open(mode='w|gz', fileobj=sys.stdout)
tar.close()


and then:
$ python2.5 archive.py | gunzip -c

gunzip: stdin: invalid compressed data--crc error

gunzip: stdin: invalid compressed data--length error

Everything works fine with python 2.3.5 and 2.4.1 on
Debian sarge.

The padding is added by the following lines in
_Stream.close:

blocks, remainder = divmod(len(self.buf), self.bufsize)
if remainder > 0:
    self.buf += NUL * (self.bufsize - remainder)

They were added in revision 38581, but I'm not sure why
- at first sight, "Add tarfile open mode r|*" shouldn't
have to change this write path.

Removing them makes gunzip happy again, but I have no
idea if it breaks something else (test_tarfile doesn't
complain).

A similar problem happens if you use mode w|bz2 and
feed the output to bunzip - it complains about
"trailing garbage after EOF ignored".

Problems found while running the test suite from the
Mercurial SCM.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1543303&group_id=5470