[Tutor] Unzipping a Zip of folders that have zips within them that I'd like to unzip all at once.
Gregory Lund
gnj091405 at gmail.com
Wed Sep 19 17:10:53 CEST 2012
>
> Your lack of response in the previous thread
>
> http://mail.python.org/pipermail/tutor/2012-August/090742.html
>
> is not a big motivation to answer this one.
I didn't have any response to post, I got the basics to work using a
hint from a colleague in and was able to grade the assignment, however
it seems as though the way the files are NOW saved in the Learning
Management System are not as I prepared my testing situation.
I wasn't able to use what Peter provided because I didn't understand
it (again, rookie/neophyte).
To an intermediate user, you most likely answered my questions, but I
didn't understand what you were saying/writing/typing.
It was full of code that I didn't (don't) understand (sys.argv, glob,
$ tree, etc.)
I had to go with what I could understand or at least work with. I
tried studying up on sys.argv, glob, etc. and got lost in the details)
Again, a colleague was able to help enough to get it to work using my
simplified code. (or at least code I could read).
>>
>> originalzip.zip
>> --originalfolder
>> --folder1 (w/ two zip folders)
>> --internalzip1_a.zip
>> --internalfolder1_a
>> --files
>> --folders
>> --internalzip1_b.zip
>> --internalfolder1_b
>> --files
>> --folders
>> --folder2 (w/1 zip folder)
>> --internalzip2.zip
>> --internalfolder2
>> --files
>> --folders
>> --etc....
I attempted to replicate Peter's 'Folder Structure' diagram above, he
did a better job 'drawing it' but the one above is my data situation.
Using my sample data, this is what the folder structure resembles below.
|__ Student_Work_Sample_use (a folder I create before running the
script, the original zip is placed in this folder before running the
script)
|__originalzip.zip (entire class all zipped up into one) (THIS
NEEDS TO BE UNZIPPED IN THE CODE)
|__Lab_2 (a folder that is the contents of originalzip.zip unzipped)
|__aforker (unzipped folder that comes in Lab_2)
|__aforker_lab_2.zip (student username 'aforker''s' lab)
(THIS NEEDS TO BE UNZIPPED IN THE CODE)
|__aforker_lab_writeup.docx (a docx file that may or may
not come in with the lab submission (it may or may not be zipped, if
it's not zipped, no need to deal with it)
|__aforker_lab_2 (a folder that is the contents of
aforker_lab_2.zip, unzipped)
|__ datafolder (unzipped folders within aforkers
lab folder) (no need to worry about, just here for reference)
|__ mapsfolder (unzipped folders within aforkers
lab folder) (no need to worry about, just here for reference)
|__awilliams (unzipped folder that comes in Lab_2)
|__awilliams_lab_2.zip (student username 'awilliams''s'
lab) (THIS NEEDS TO BE UNZIPPED IN THE CODE)
|__awilliams_lab_2 (a folder that is the contents of
awilliams_lab_2.zip, unzipped)
|__ datafolder (unzipped folders within awilliams
lab folder) (no need to worry about, just here for reference)
|__ mapsfolder (unzipped folders within awilliams
lab folder) (no need to worry about, just here for reference)
|__awilliams_lab_2_RESUB.zip (student username
'awilliams''s' lab) (THIS NEEDS TO BE UNZIPPED IN THE CODE)
|__awilliams_lab_2_RESUB (a folder that is the contents
of awilliams_lab_2_RESUB.zip, unzipped)
|__ datafolder (unzipped folders within awilliams
lab folder) (no need to worry about, just here for reference)
|__ mapsfolder (unzipped folders within awilliams
lab folder) (no need to worry about, just here for reference)
|__jsmith (unzipped folder that comes in Lab_2)
|__jsmith_lab_2.zip (student username 'jsmith''s' lab)
(THIS NEEDS TO BE UNZIPPED IN THE CODE)
|__jsmith_lab_2 (a folder that is the contents of
jsmith_lab_2.zip, unzipped)
|__ datafolder (unzipped folders within jsmith lab
folder) (no need to worry about, just here for reference)
|__ mapsfolder (unzipped folders within jsmith lab
folder) (no need to worry about, just here for reference)
|__ etc. etc. etc. up to 40 students.....
>>
>> My goal is to:
>> a) Unzip the 'originalzip.zip'
>> b) go to the 'originalfolder' (the unzipped version of the
>> originalzip.zip) c) go into the first folder (folder1) in the original
>> folder and unzip any and all zipfiles within it
>> d) go to the second folder (folder2) in the original folder and unzip
>> any and all zipfiles within it
>> e) continue until all folders within originalfolders have been checked
>> for internalzips
>>
>>
>> I have some code that works to extract the 'originalzip.zip', to an
>> 'originalfolder' but it won't go to the folders (folder1, folder2,
>> etc.) to unzip the zipfiles within them.
>> It gets hung up on access to the first student folder and won't unzip it.
>
> Hm, I would have expeced an exception. Perhaps you should omit the ArcGIS
> integration until everything else works.
Fair enough, although the ArcGIS integration is simple. Setting it up,
is all in an Arc GUI.
>
> Excessive comments impair readability. Comments stating the obvious are
> particularly bad.
Point taken, however 'obvious to neophytes' is quite different than
'obvious to experts' . (I still need comments- :-0)
#super short Code below, without comments: (and using the two fixes
suggested thus far by Peter (no os.sep and two names for one value
issue)
import os, os.path, zipfile, arcpy
in_Zip = arcpy.GetParameterAsText(0)
outDir = os.getcwd()
z = zipfile.ZipFile(in_Zip)
z.extractall(outDir)
zipContents = z.namelist()
for item in zipContents:
itemLoc = outDir + os.sep + item
z = zipfile.ZipFile(itemLoc)
z.extractall(os.path.split(itemLoc)[0])
student_lab = z.namelist()
>> #2.1 in_zip is a variable for "What zipfile (LAB) do you want to extract?"
>> in_Zip = arcpy.GetParameterAsText(0)
>> cZ = in_Zip
>
> Why two names for one value?
Rookie Mistake. I thought I'd need the raw in_Zip in some instances
and then cZ in others. (Fixed Now, above)
> You make no attempt to filter out the contents that are not zipfiles. That
> will cause an exception further down where you try to unzip.
True, and that's what happened.
My code does not show it above.
I tried... but it messed things up further (worse) so reverted back to
what was 'working' the most.
I tried some 'While' statements and some 'If' statements, both gave me
alphabet soup in return.
Again, I'm a neophyte.
>
>> #6.2 Get the location (note the location is the 'outDir' plus what
>> namelist() gave me)
>> #(have never used 'os.sep', had to look it up, when someone suggested
>> #it)
>> itemLoc = outDir + os.sep + item
>
> The standard way (which is also more robust) is to use os.path.join():
>
> itemLoc = os.path.join(outDir, item)
>
Thank you, I changed that above
>> #6.3 Opens the first (2nd, 3rd, etc files) of the internal zip file
>> #(*.zip)
>> z = zipfile.ZipFile(itemLoc)
>>
>> #6.4 Extract all files in *.zip in the same folder as the zipfile
>> z.extractall(os.path.split(itemLoc)[0])
>
> The zip files's contents will probably end up in the same folder as the
> zipfile. Is that what you want?
Yes, that is exactly what I want.
my desire is to get them to end up in their respective 'username' folders.
folder1, folder2 etc are all 'usernames' I need to keep all of the
submissions separated by username
Which is the way they come from the Learning Management System.
That's how I keep the individual assignment straight.
Originally, doing this all manually: I had to unzip the
originalzip.zip (relatively simple and not time consuming)
Open the resulting folder (Lab_2 in this example)
Open each non-zipped folder (a folder for each student) within Lab_2
(this was a time consuming task (if you figure 9 labs throughout the
quarter and ... 40 students (40 folders)!)
THEN unzip each of the zipfiles in each of those non-zipped folders
that the students submitted (very, very time consuming!)
>>
>> #6.5 determining the list of items in each students' folder
>> student_lab = z.namelist()
>
> Unused variable alert.
Whoops, this was part of my attempt to unzip the second round of
folders, putting them into a list.
>
>>
>> Thank you for any and all suggestions/ fixes, constructive criticism
>> and assistance with my beginner code!
>
> Have you considered the simpler code I gave in
>
> http://mail.python.org/pipermail/tutor/2012-August/090743.html
>
> before prodding on?
Yes, I did consider it, but I didn't understand it enough to 'run with it'.
If it worked perfectly I still wouldn't of understood it, and it I
needed to tweak it, there would have been no way for me to figure out
what to do to make it fit my scenario.
While it may be 'simpler' for the experienced python coder, Not being
familiar with Python, it wasn't simpler for me, I could hardly read
any of it.
I even printed it out and saved, but I couldn't understand it and
didn't want to bother you with it any more (felt like an idiot, to be
honest) (you would of had to explain everything, as I didn't
understand hardly any of it)
A Colleague gave me some tips (for example: "itemLoc = outDir + os.sep
+ item" and that worked at the time... so I went with it.
Absolutely no disrespect intended Peter! I just couldn't decipher what
you meant, whereas my colleague's help fixed what I had, so I went
with it.
And again, since I didn't understand what you wrote, I didn't want to
bother you with it anymore.
For reference, this is the error I get:
<type 'exceptions.IOError'>: [Errno 13] Permission denied:
'D:\\D_Drive_Documents\\Student_Work_Sample_use\\Lab_2/aforker/'
Failed to execute (unzipsZipOfZips)
'Student_Work_Sample_use' is a folder that housed my small test data
set (originalzip.zip).
'Lab_2' is the correct 'answer' for the first unzip process (I used my
Spring 2012 students Lab_2 as the sample dataset) This part worked and
still works.
'aforker' is an actual student from the class (his/her username)
I see that there is an issue with the slashes in the folder path,
despite numerous attempts, I can't resolve that.
Thank you again Peter and the rest of this maillist!
Greg
> _______________________________________________
> Tutor maillist - Tutor at python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/mailman/listinfo/tutor
More information about the Tutor
mailing list