unzip problem - solved

[Ahmed, Shakir]

-----Original Message-----
From: Ethan Furman [mailto:ethan at stoneleaf.us] 
Sent: Friday, June 24, 2011 3:47 PM
To: Ahmed, Shakir
Cc: Python
Subject: Re: unzip problem

Ahmed, Shakir wrote:
> Thanks once again and you are right I am trying to unzip in the
network
> share drive. here is the script now: If I am doing any wrong. :
> ## code start here
> 
> import zipfile
> import os
> import time
> dir1 = "T:\\applications\\tst\\py\\Zip_Process"
> test = '%s/shp'%dir1
> os.chdir(test)
> cwd = os.getcwd()
> print cwd
> 
> 
> CHUNK_SIZE = 10 * 1024 * 1024
> 
> 
> fn = open('T:\\applications\\tst\\py\\Zip_Process\\Zip\\myzip.zip',
> 'rb')
> z = zipfile.ZipFile(fn)
> for name in z.namelist():
>     ptr = 0
>     data = z.read(name)
>     size = len(data)
>     print size
>     print ptr
>     while ptr < size:
>         
>         fn.write(data[ptr:ptr+CHUNK_SIZE])
>         ptr += CHUNK_SIZE
> 
> fn.close()
> 
> #Code done here.
>  
> But got error as follows:
> 
>>>> T:\applications\tst\py\Zip_Process\shp
> 59160
> 0
> Traceback (most recent call last):
>   File
>
"C:\Python25\Lib\site-packages\pythonwin\pywin\framework\scriptutils.py"
> , line 325, in RunScript
>     exec codeObject in __main__.__dict__
>   File "t:\scratch\shahmed\test2_new.py", line 24, in <module>
>     fn.write(data[ptr:ptr+CHUNK_SIZE])
> IOError: [Errno 9] Bad file descriptor

I didn't notice this in your last post, but you are using fn as both the

zipfile name and the name of the file you are writing to.  You'll need 
to create the file you want to write before you write to it (plus any 
intermediate directories).


Here's the (slightly stripped) version I actually use:

8<--------------------------------------------------------------------
import os
from zipfile import  ZipFile

def retrieve_files(zipped, destination, files=None):
     "retrieves files from <zipped>"
     CHUNK_SIZE = 10 * 1024 * 1024
     try:
         os.makedirs(destination)
     except WindowsError:
         pass
     target = ZipFile(zipped, 'r')
     stored = dict([(k.filename.lower(), k.filename) for k in 
target.infolist()])
     if files is None:
         files = [member.filename.lower() for member in
target.infolist()]
     elif isinstance(files, (str, unicode)):
         files = [files.lower()]
     else:
         files = [f.lower() for f in files]
     for compressed_file in files:
         uncompressed_file = os.path.join(destination, compressed_file)
         path, filename = os.path.split(uncompressed_file)
         if not os.path.exists(path):
             os.makedirs(path)
         if filename == '__empty__':
             continue
         data = target.read(stored[compressed_file])
         fn = open(uncompressed_file, 'wb')
         ptr = 0
         size = len(data)
         while ptr < size:
             fn.write(data[ptr:ptr+CHUNK_SIZE])
             ptr += CHUNK_SIZE
         fn.close()
     target.close()
8<--------------------------------------------------------------------

I convert all filenames to lower case since MS Windows doesn't care, but

Python does.

The test for filename == '__empty__': when I create the zipfile, if the 
directory is empty I store a 0-length file named '__empty__' in that 
subdirectory (actually, it only happens in the zipfile) so that I can 
get the directory back later.

Hope this helps.

~Ethan~

Thanks a lot to Ethan who really helped to find out the clue in the
code.
Here is the final code that worked to unzip a large file in the network
drive.

CHUNK_SIZE = 10 * 1024 * 1024


fh = open('T:\\applications\\tst\\py\\Zip_Process\\Zip\\myzip.zip',
'rb')
z = zipfile.ZipFile(fh)
for name in z.namelist():
    fn = open(name, 'wb')
    ptr = 0
    data = z.read(name)
    size = len(name)  
    print size
    print ptr
    while ptr < size:
        #fn.write(data)
        fn.write(data[ptr:ptr+CHUNK_SIZE])
        ptr += CHUNK_SIZE
    fn.close()
fh.close()