tarfile's tar.extractfile() file-like object incompatible with pickle.load()?

[Matt Doucleff]

"Tom B." <sbabbitt at commspeed.net> wrote in message news:<1093572564.733113 at news.commspeed.net>...
> "Matt Doucleff" <mattduke at ugcs.caltech.edu> wrote in message
> news:3e6d6c21.0408261043.264c3a2d at posting.google.com...
> > Hi everyone!  I must be doing something wrong here :)  I have a
> > tarball that contains a single file whose contents are a pickled
> > object.  I would like to unpickle the object directly from the tarball
> > using the file-like object provided by extractfile().  Attempts to do
> > this result in EOFError.  However if I first extract to a temporary
> > file, then unpickle from there, it works.  The below code reproduces
> > the problem (on my machine at least).  I'm running Python 2.3.4,
> > manually installed on Debian Woody (original python removed).  Thanks!
> >
> > This sample code creates (and then removes) files in the tmp directory
> > and in the current working directory.
> >
> > # demonstrates extractfile/unpickle failure (bug?)
> >
> > # pickle a dict to a temp file
> > # create tar file, add temp file to it, close tar file
> > # open tar file for reading
> > # obtain file-like object for pickled file using extractfile()
> > # attempt to unpickle dict from file-like object
> > # fails with EOFError exception
> >
> > import tarfile
> > import pickle
> > import tempfile
> > import os
> >
> > if __name__ == '__main__':
> >     try:
> >         hashtopickle = { 'a' : 1, 'b' : 2 }
> >
> >         # pickle to temp file
> >         (fd, tmpfilename) = tempfile.mkstemp()
> >         tmpfile = os.fdopen(fd, 'w')
> >         pickle.dump(hashtopickle, tmpfile)
> >         tmpfile.close()
> >
> >         # create tar; add temp file
> >         tar = tarfile.open('tarpickle.tar', 'w')
> >         tar.add(tmpfilename, 'pickledhash')
> >         tar.close()
> >
> >         # remove temp file
> >         os.remove(tmpfilename)
> >
> >         # open tarfile for reading, get filelike
> >         tar = tarfile.open('tarpickle.tar', 'r')

> >         filelike = tar.extractfile('pickledhash')
> >
> >         # fails
> >         hashcopy = pickle.load(filelike)
> >
> >     finally:
> >         # cleanup
> >         os.remove('tarpickle.tar')
> 
> It occurs to me that you need to do,
> 
>  hashcopy = pickle.loads(filelike)
> 
> if filelike is a string.
> 
> Tom
> P.S. have a look at pickle.dumps()

The tarfile.extractfile() method does not read the contents
of the encapsulated file into a string, but constructs a new
object that implements file operations (it is like a file)
and is intended to be used as if you had simply opened the
tar-encapsulated file directly.

    Matt