Mailman 3 January 2013 - Python-checkins

cpython (merge 3.3 -> default): Added test to ensure localized calendar methods return strings and not bytes.
by serhiy.storchaka Jan. 31, 2013

Jan. 31, 2013

http://hg.python.org/cpython/rev/e460dedf8633 changeset: 81854:e460dedf8633 parent: 81850:e406b8bd7b38 parent: 81853:5bad2c422315 user: Serhiy Storchaka <storchaka(a)gmail.com> date: Thu Jan 31 16:01:46 2013 +0200 summary: Added test to ensure localized calendar methods return strings and not bytes. files: Lib/test/test_calendar.py | 16 +++++++++++++--- 1 files changed, 13 insertions(+), 3 deletions(-) diff --git a/Lib/test/test_calendar.py b/Lib/test/test_calendar.py --- a/Lib/test/test_calendar.py +++ b/Lib/test/test_calendar.py @@ -457,11 +457,21 @@ # (it is still not thread-safe though) old_october = calendar.TextCalendar().formatmonthname(2010, 10, 10) try: - calendar.LocaleTextCalendar(locale='').formatmonthname(2010, 10, 10) + cal = calendar.LocaleTextCalendar(locale='') + local_weekday = cal.formatweekday(1, 10) + local_month = cal.formatmonthname(2010, 10, 10) except locale.Error: # cannot set the system default locale -- skip rest of test - return - calendar.LocaleHTMLCalendar(locale='').formatmonthname(2010, 10) + raise unittest.SkipTest('cannot set the system default locale') + self.assertIsInstance(local_weekday, str) + self.assertIsInstance(local_month, str) + self.assertEqual(len(local_weekday), 10) + self.assertGreaterEqual(len(local_month), 10) + cal = calendar.LocaleHTMLCalendar(locale='') + local_weekday = cal.formatweekday(1) + local_month = cal.formatmonthname(2010, 10) + self.assertIsInstance(local_weekday, str) + self.assertIsInstance(local_month, str) new_october = calendar.TextCalendar().formatmonthname(2010, 10, 10) self.assertEqual(old_october, new_october) -- Repository URL: http://hg.python.org/cpython

1 0

cpython (merge 3.2 -> 3.3): Added test to ensure localized calendar methods return strings and not bytes.
by serhiy.storchaka Jan. 31, 2013

Jan. 31, 2013

http://hg.python.org/cpython/rev/5bad2c422315 changeset: 81853:5bad2c422315 branch: 3.3 parent: 81849:46f24a18a4ab parent: 81852:b45da1761567 user: Serhiy Storchaka <storchaka(a)gmail.com> date: Thu Jan 31 16:01:21 2013 +0200 summary: Added test to ensure localized calendar methods return strings and not bytes. files: Lib/test/test_calendar.py | 16 +++++++++++++--- 1 files changed, 13 insertions(+), 3 deletions(-) diff --git a/Lib/test/test_calendar.py b/Lib/test/test_calendar.py --- a/Lib/test/test_calendar.py +++ b/Lib/test/test_calendar.py @@ -457,11 +457,21 @@ # (it is still not thread-safe though) old_october = calendar.TextCalendar().formatmonthname(2010, 10, 10) try: - calendar.LocaleTextCalendar(locale='').formatmonthname(2010, 10, 10) + cal = calendar.LocaleTextCalendar(locale='') + local_weekday = cal.formatweekday(1, 10) + local_month = cal.formatmonthname(2010, 10, 10) except locale.Error: # cannot set the system default locale -- skip rest of test - return - calendar.LocaleHTMLCalendar(locale='').formatmonthname(2010, 10) + raise unittest.SkipTest('cannot set the system default locale') + self.assertIsInstance(local_weekday, str) + self.assertIsInstance(local_month, str) + self.assertEqual(len(local_weekday), 10) + self.assertGreaterEqual(len(local_month), 10) + cal = calendar.LocaleHTMLCalendar(locale='') + local_weekday = cal.formatweekday(1) + local_month = cal.formatmonthname(2010, 10) + self.assertIsInstance(local_weekday, str) + self.assertIsInstance(local_month, str) new_october = calendar.TextCalendar().formatmonthname(2010, 10, 10) self.assertEqual(old_october, new_october) -- Repository URL: http://hg.python.org/cpython

1 0

cpython (3.2): Added test to ensure localized calendar methods return strings and not bytes.
by serhiy.storchaka Jan. 31, 2013

Jan. 31, 2013

http://hg.python.org/cpython/rev/b45da1761567 changeset: 81852:b45da1761567 branch: 3.2 parent: 81848:01147e468c8c user: Serhiy Storchaka <storchaka(a)gmail.com> date: Thu Jan 31 16:00:42 2013 +0200 summary: Added test to ensure localized calendar methods return strings and not bytes. files: Lib/test/test_calendar.py | 16 +++++++++++++--- 1 files changed, 13 insertions(+), 3 deletions(-) diff --git a/Lib/test/test_calendar.py b/Lib/test/test_calendar.py --- a/Lib/test/test_calendar.py +++ b/Lib/test/test_calendar.py @@ -258,11 +258,21 @@ # (it is still not thread-safe though) old_october = calendar.TextCalendar().formatmonthname(2010, 10, 10) try: - calendar.LocaleTextCalendar(locale='').formatmonthname(2010, 10, 10) + cal = calendar.LocaleTextCalendar(locale='') + local_weekday = cal.formatweekday(1, 10) + local_month = cal.formatmonthname(2010, 10, 10) except locale.Error: # cannot set the system default locale -- skip rest of test - return - calendar.LocaleHTMLCalendar(locale='').formatmonthname(2010, 10) + raise unittest.SkipTest('cannot set the system default locale') + self.assertIsInstance(local_weekday, str) + self.assertIsInstance(local_month, str) + self.assertEqual(len(local_weekday), 10) + self.assertGreaterEqual(len(local_month), 10) + cal = calendar.LocaleHTMLCalendar(locale='') + local_weekday = cal.formatweekday(1) + local_month = cal.formatmonthname(2010, 10) + self.assertIsInstance(local_weekday, str) + self.assertIsInstance(local_month, str) new_october = calendar.TextCalendar().formatmonthname(2010, 10, 10) self.assertEqual(old_october, new_october) -- Repository URL: http://hg.python.org/cpython

1 0

cpython (2.7): Issue #17049: Localized calendar methods now return unicode if a locale
by serhiy.storchaka Jan. 31, 2013

Jan. 31, 2013

http://hg.python.org/cpython/rev/af41eca1959e changeset: 81851:af41eca1959e branch: 2.7 parent: 81847:32de35f0f877 user: Serhiy Storchaka <storchaka(a)gmail.com> date: Thu Jan 31 15:57:51 2013 +0200 summary: Issue #17049: Localized calendar methods now return unicode if a locale includes an encoding and the result string contains month or weekday (was regression from Python 2.6). files: Lib/calendar.py | 1 + Lib/test/test_calendar.py | 18 +++++++++++++++--- Misc/NEWS | 4 ++++ 3 files changed, 20 insertions(+), 3 deletions(-) diff --git a/Lib/calendar.py b/Lib/calendar.py --- a/Lib/calendar.py +++ b/Lib/calendar.py @@ -492,6 +492,7 @@ def __enter__(self): self.oldlocale = _locale.getlocale(_locale.LC_TIME) _locale.setlocale(_locale.LC_TIME, self.locale) + return _locale.getlocale(_locale.LC_TIME)[1] def __exit__(self, *args): _locale.setlocale(_locale.LC_TIME, self.oldlocale) diff --git a/Lib/test/test_calendar.py b/Lib/test/test_calendar.py --- a/Lib/test/test_calendar.py +++ b/Lib/test/test_calendar.py @@ -255,11 +255,23 @@ # (it is still not thread-safe though) old_october = calendar.TextCalendar().formatmonthname(2010, 10, 10) try: - calendar.LocaleTextCalendar(locale='').formatmonthname(2010, 10, 10) + cal = calendar.LocaleTextCalendar(locale='') + local_weekday = cal.formatweekday(1, 10) + local_month = cal.formatmonthname(2010, 10, 10) except locale.Error: # cannot set the system default locale -- skip rest of test - return - calendar.LocaleHTMLCalendar(locale='').formatmonthname(2010, 10) + raise unittest.SkipTest('cannot set the system default locale') + # should be encodable + local_weekday.encode('utf-8') + local_month.encode('utf-8') + self.assertEqual(len(local_weekday), 10) + self.assertGreaterEqual(len(local_month), 10) + cal = calendar.LocaleHTMLCalendar(locale='') + local_weekday = cal.formatweekday(1) + local_month = cal.formatmonthname(2010, 10) + # should be encodable + local_weekday.encode('utf-8') + local_month.encode('utf-8') new_october = calendar.TextCalendar().formatmonthname(2010, 10, 10) self.assertEqual(old_october, new_october) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -202,6 +202,10 @@ Library ------- +- Issue #17049: Localized calendar methods now return unicode if a locale + includes an encoding and the result string contains month or weekday (was + regression from Python 2.6). + - Issue #4844: ZipFile now raises BadZipfile when opens a ZIP file with an incomplete "End of Central Directory" record. Original patch by Guilherme Polo and Alan McIntyre. -- Repository URL: http://hg.python.org/cpython

1 0

cpython (merge 3.3 -> default): Issue #4844: ZipFile now raises BadZipFile when opens a ZIP file with an
by serhiy.storchaka Jan. 31, 2013

Jan. 31, 2013

http://hg.python.org/cpython/rev/e406b8bd7b38 changeset: 81850:e406b8bd7b38 parent: 81846:022477b09dc3 parent: 81849:46f24a18a4ab user: Serhiy Storchaka <storchaka(a)gmail.com> date: Thu Jan 31 15:30:36 2013 +0200 summary: Issue #4844: ZipFile now raises BadZipFile when opens a ZIP file with an incomplete "End of Central Directory" record. Original patch by Guilherme Polo and Alan McIntyre. files: Lib/test/test_zipfile.py | 14 ++++++++++++++ Lib/zipfile.py | 25 +++++++++++++++++++------ Misc/NEWS | 4 ++++ 3 files changed, 37 insertions(+), 6 deletions(-) diff --git a/Lib/test/test_zipfile.py b/Lib/test/test_zipfile.py --- a/Lib/test/test_zipfile.py +++ b/Lib/test/test_zipfile.py @@ -996,6 +996,20 @@ chk = zipfile.is_zipfile(fp) self.assertTrue(not chk) + def test_damaged_zipfile(self): + """Check that zipfiles with missing bytes at the end raise BadZipFile.""" + # - Create a valid zip file + fp = io.BytesIO() + with zipfile.ZipFile(fp, mode="w") as zipf: + zipf.writestr("foo.txt", b"O, for a Muse of Fire!") + zipfiledata = fp.getvalue() + + # - Now create copies of it missing the last N bytes and make sure + # a BadZipFile exception is raised when we try to open it + for N in range(len(zipfiledata)): + fp = io.BytesIO(zipfiledata[:N]) + self.assertRaises(zipfile.BadZipFile, zipfile.ZipFile, fp) + def test_is_zip_valid_file(self): """Check that is_zipfile() correctly identifies zip files.""" # - passing a filename diff --git a/Lib/zipfile.py b/Lib/zipfile.py --- a/Lib/zipfile.py +++ b/Lib/zipfile.py @@ -196,6 +196,8 @@ return endrec data = fpin.read(sizeEndCentDir64Locator) + if len(data) != sizeEndCentDir64Locator: + return endrec sig, diskno, reloff, disks = struct.unpack(structEndArchive64Locator, data) if sig != stringEndArchive64Locator: return endrec @@ -206,6 +208,8 @@ # Assume no 'zip64 extensible data' fpin.seek(offset - sizeEndCentDir64Locator - sizeEndCentDir64, 2) data = fpin.read(sizeEndCentDir64) + if len(data) != sizeEndCentDir64: + return endrec sig, sz, create_version, read_version, disk_num, disk_dir, \ dircount, dircount2, dirsize, diroffset = \ struct.unpack(structEndArchive64, data) @@ -241,7 +245,9 @@ except OSError: return None data = fpin.read() - if data[0:4] == stringEndArchive and data[-2:] == b"\000\000": + if (len(data) == sizeEndCentDir and + data[0:4] == stringEndArchive and + data[-2:] == b"\000\000"): # the signature is correct and there's no comment, unpack structure endrec = struct.unpack(structEndArchive, data) endrec=list(endrec) @@ -265,6 +271,9 @@ if start >= 0: # found the magic number; attempt to unpack and interpret recData = data[start:start+sizeEndCentDir] + if len(recData) != sizeEndCentDir: + # Zip file is corrupted. + return None endrec = list(struct.unpack(structEndArchive, recData)) commentSize = endrec[_ECD_COMMENT_SIZE] #as claimed by the zip file comment = data[start+sizeEndCentDir:start+sizeEndCentDir+commentSize] @@ -276,7 +285,7 @@ endrec) # Unable to find a valid end of central directory structure - return + return None class ZipInfo (object): @@ -978,9 +987,11 @@ total = 0 while total < size_cd: centdir = fp.read(sizeCentralDir) - if centdir[0:4] != stringCentralDir: + if len(centdir) != sizeCentralDir: + raise BadZipFile("Truncated central directory") + centdir = struct.unpack(structCentralDir, centdir) + if centdir[_CD_SIGNATURE] != stringCentralDir: raise BadZipFile("Bad magic number for central directory") - centdir = struct.unpack(structCentralDir, centdir) if self.debug > 2: print(centdir) filename = fp.read(centdir[_CD_FILENAME_LENGTH]) @@ -1123,10 +1134,12 @@ # Skip the file header: fheader = zef_file.read(sizeFileHeader) - if fheader[0:4] != stringFileHeader: + if len(fheader) != sizeFileHeader: + raise BadZipFile("Truncated file header") + fheader = struct.unpack(structFileHeader, fheader) + if fheader[_FH_SIGNATURE] != stringFileHeader: raise BadZipFile("Bad magic number for file header") - fheader = struct.unpack(structFileHeader, fheader) fname = zef_file.read(fheader[_FH_FILENAME_LENGTH]) if fheader[_FH_EXTRA_FIELD_LENGTH]: zef_file.read(fheader[_FH_EXTRA_FIELD_LENGTH]) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -236,6 +236,10 @@ Library ------- +- Issue #4844: ZipFile now raises BadZipFile when opens a ZIP file with an + incomplete "End of Central Directory" record. Original patch by Guilherme + Polo and Alan McIntyre. + - Issue #17071: Signature.bind() now works when one of the keyword arguments is named ``self``. -- Repository URL: http://hg.python.org/cpython

1 0

cpython (merge 3.2 -> 3.3): Issue #4844: ZipFile now raises BadZipFile when opens a ZIP file with an
by serhiy.storchaka Jan. 31, 2013

Jan. 31, 2013

http://hg.python.org/cpython/rev/46f24a18a4ab changeset: 81849:46f24a18a4ab branch: 3.3 parent: 81845:d218ecb7392d parent: 81848:01147e468c8c user: Serhiy Storchaka <storchaka(a)gmail.com> date: Thu Jan 31 15:29:20 2013 +0200 summary: Issue #4844: ZipFile now raises BadZipFile when opens a ZIP file with an incomplete "End of Central Directory" record. Original patch by Guilherme Polo and Alan McIntyre. files: Lib/test/test_zipfile.py | 14 ++++++++++++++ Lib/zipfile.py | 25 +++++++++++++++++++------ Misc/NEWS | 4 ++++ 3 files changed, 37 insertions(+), 6 deletions(-) diff --git a/Lib/test/test_zipfile.py b/Lib/test/test_zipfile.py --- a/Lib/test/test_zipfile.py +++ b/Lib/test/test_zipfile.py @@ -996,6 +996,20 @@ chk = zipfile.is_zipfile(fp) self.assertTrue(not chk) + def test_damaged_zipfile(self): + """Check that zipfiles with missing bytes at the end raise BadZipFile.""" + # - Create a valid zip file + fp = io.BytesIO() + with zipfile.ZipFile(fp, mode="w") as zipf: + zipf.writestr("foo.txt", b"O, for a Muse of Fire!") + zipfiledata = fp.getvalue() + + # - Now create copies of it missing the last N bytes and make sure + # a BadZipFile exception is raised when we try to open it + for N in range(len(zipfiledata)): + fp = io.BytesIO(zipfiledata[:N]) + self.assertRaises(zipfile.BadZipFile, zipfile.ZipFile, fp) + def test_is_zip_valid_file(self): """Check that is_zipfile() correctly identifies zip files.""" # - passing a filename diff --git a/Lib/zipfile.py b/Lib/zipfile.py --- a/Lib/zipfile.py +++ b/Lib/zipfile.py @@ -196,6 +196,8 @@ return endrec data = fpin.read(sizeEndCentDir64Locator) + if len(data) != sizeEndCentDir64Locator: + return endrec sig, diskno, reloff, disks = struct.unpack(structEndArchive64Locator, data) if sig != stringEndArchive64Locator: return endrec @@ -206,6 +208,8 @@ # Assume no 'zip64 extensible data' fpin.seek(offset - sizeEndCentDir64Locator - sizeEndCentDir64, 2) data = fpin.read(sizeEndCentDir64) + if len(data) != sizeEndCentDir64: + return endrec sig, sz, create_version, read_version, disk_num, disk_dir, \ dircount, dircount2, dirsize, diroffset = \ struct.unpack(structEndArchive64, data) @@ -241,7 +245,9 @@ except IOError: return None data = fpin.read() - if data[0:4] == stringEndArchive and data[-2:] == b"\000\000": + if (len(data) == sizeEndCentDir and + data[0:4] == stringEndArchive and + data[-2:] == b"\000\000"): # the signature is correct and there's no comment, unpack structure endrec = struct.unpack(structEndArchive, data) endrec=list(endrec) @@ -265,6 +271,9 @@ if start >= 0: # found the magic number; attempt to unpack and interpret recData = data[start:start+sizeEndCentDir] + if len(recData) != sizeEndCentDir: + # Zip file is corrupted. + return None endrec = list(struct.unpack(structEndArchive, recData)) commentSize = endrec[_ECD_COMMENT_SIZE] #as claimed by the zip file comment = data[start+sizeEndCentDir:start+sizeEndCentDir+commentSize] @@ -276,7 +285,7 @@ endrec) # Unable to find a valid end of central directory structure - return + return None class ZipInfo (object): @@ -978,9 +987,11 @@ total = 0 while total < size_cd: centdir = fp.read(sizeCentralDir) - if centdir[0:4] != stringCentralDir: + if len(centdir) != sizeCentralDir: + raise BadZipFile("Truncated central directory") + centdir = struct.unpack(structCentralDir, centdir) + if centdir[_CD_SIGNATURE] != stringCentralDir: raise BadZipFile("Bad magic number for central directory") - centdir = struct.unpack(structCentralDir, centdir) if self.debug > 2: print(centdir) filename = fp.read(centdir[_CD_FILENAME_LENGTH]) @@ -1123,10 +1134,12 @@ # Skip the file header: fheader = zef_file.read(sizeFileHeader) - if fheader[0:4] != stringFileHeader: + if len(fheader) != sizeFileHeader: + raise BadZipFile("Truncated file header") + fheader = struct.unpack(structFileHeader, fheader) + if fheader[_FH_SIGNATURE] != stringFileHeader: raise BadZipFile("Bad magic number for file header") - fheader = struct.unpack(structFileHeader, fheader) fname = zef_file.read(fheader[_FH_FILENAME_LENGTH]) if fheader[_FH_EXTRA_FIELD_LENGTH]: zef_file.read(fheader[_FH_EXTRA_FIELD_LENGTH]) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -164,6 +164,10 @@ Library ------- +- Issue #4844: ZipFile now raises BadZipFile when opens a ZIP file with an + incomplete "End of Central Directory" record. Original patch by Guilherme + Polo and Alan McIntyre. + - Issue #17071: Signature.bind() now works when one of the keyword arguments is named ``self``. -- Repository URL: http://hg.python.org/cpython

1 0

cpython (3.2): Issue #4844: ZipFile now raises BadZipFile when opens a ZIP file with an
by serhiy.storchaka Jan. 31, 2013

Jan. 31, 2013

http://hg.python.org/cpython/rev/01147e468c8c changeset: 81848:01147e468c8c branch: 3.2 parent: 81844:502b139b3b19 user: Serhiy Storchaka <storchaka(a)gmail.com> date: Thu Jan 31 15:27:07 2013 +0200 summary: Issue #4844: ZipFile now raises BadZipFile when opens a ZIP file with an incomplete "End of Central Directory" record. Original patch by Guilherme Polo and Alan McIntyre. files: Lib/test/test_zipfile.py | 14 ++++++++++++++ Lib/zipfile.py | 25 +++++++++++++++++++------ Misc/NEWS | 4 ++++ 3 files changed, 37 insertions(+), 6 deletions(-) diff --git a/Lib/test/test_zipfile.py b/Lib/test/test_zipfile.py --- a/Lib/test/test_zipfile.py +++ b/Lib/test/test_zipfile.py @@ -844,6 +844,20 @@ chk = zipfile.is_zipfile(fp) self.assertTrue(not chk) + def test_damaged_zipfile(self): + """Check that zipfiles with missing bytes at the end raise BadZipFile.""" + # - Create a valid zip file + fp = io.BytesIO() + with zipfile.ZipFile(fp, mode="w") as zipf: + zipf.writestr("foo.txt", b"O, for a Muse of Fire!") + zipfiledata = fp.getvalue() + + # - Now create copies of it missing the last N bytes and make sure + # a BadZipFile exception is raised when we try to open it + for N in range(len(zipfiledata)): + fp = io.BytesIO(zipfiledata[:N]) + self.assertRaises(zipfile.BadZipFile, zipfile.ZipFile, fp) + def test_is_zip_valid_file(self): """Check that is_zipfile() correctly identifies zip files.""" # - passing a filename diff --git a/Lib/zipfile.py b/Lib/zipfile.py --- a/Lib/zipfile.py +++ b/Lib/zipfile.py @@ -176,6 +176,8 @@ return endrec data = fpin.read(sizeEndCentDir64Locator) + if len(data) != sizeEndCentDir64Locator: + return endrec sig, diskno, reloff, disks = struct.unpack(structEndArchive64Locator, data) if sig != stringEndArchive64Locator: return endrec @@ -186,6 +188,8 @@ # Assume no 'zip64 extensible data' fpin.seek(offset - sizeEndCentDir64Locator - sizeEndCentDir64, 2) data = fpin.read(sizeEndCentDir64) + if len(data) != sizeEndCentDir64: + return endrec sig, sz, create_version, read_version, disk_num, disk_dir, \ dircount, dircount2, dirsize, diroffset = \ struct.unpack(structEndArchive64, data) @@ -221,7 +225,9 @@ except IOError: return None data = fpin.read() - if data[0:4] == stringEndArchive and data[-2:] == b"\000\000": + if (len(data) == sizeEndCentDir and + data[0:4] == stringEndArchive and + data[-2:] == b"\000\000"): # the signature is correct and there's no comment, unpack structure endrec = struct.unpack(structEndArchive, data) endrec=list(endrec) @@ -245,6 +251,9 @@ if start >= 0: # found the magic number; attempt to unpack and interpret recData = data[start:start+sizeEndCentDir] + if len(recData) != sizeEndCentDir: + # Zip file is corrupted. + return None endrec = list(struct.unpack(structEndArchive, recData)) commentSize = endrec[_ECD_COMMENT_SIZE] #as claimed by the zip file comment = data[start+sizeEndCentDir:start+sizeEndCentDir+commentSize] @@ -256,7 +265,7 @@ endrec) # Unable to find a valid end of central directory structure - return + return None class ZipInfo (object): @@ -819,9 +828,11 @@ total = 0 while total < size_cd: centdir = fp.read(sizeCentralDir) - if centdir[0:4] != stringCentralDir: + if len(centdir) != sizeCentralDir: + raise BadZipFile("Truncated central directory") + centdir = struct.unpack(structCentralDir, centdir) + if centdir[_CD_SIGNATURE] != stringCentralDir: raise BadZipFile("Bad magic number for central directory") - centdir = struct.unpack(structCentralDir, centdir) if self.debug > 2: print(centdir) filename = fp.read(centdir[_CD_FILENAME_LENGTH]) @@ -964,10 +975,12 @@ # Skip the file header: fheader = zef_file.read(sizeFileHeader) - if fheader[0:4] != stringFileHeader: + if len(fheader) != sizeFileHeader: + raise BadZipFile("Truncated file header") + fheader = struct.unpack(structFileHeader, fheader) + if fheader[_FH_SIGNATURE] != stringFileHeader: raise BadZipFile("Bad magic number for file header") - fheader = struct.unpack(structFileHeader, fheader) fname = zef_file.read(fheader[_FH_FILENAME_LENGTH]) if fheader[_FH_EXTRA_FIELD_LENGTH]: zef_file.read(fheader[_FH_EXTRA_FIELD_LENGTH]) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -216,6 +216,10 @@ Library ------- +- Issue #4844: ZipFile now raises BadZipFile when opens a ZIP file with an + incomplete "End of Central Directory" record. Original patch by Guilherme + Polo and Alan McIntyre. + - Issue #15505: `unittest.installHandler` no longer assumes SIGINT handler is set to a callable object. -- Repository URL: http://hg.python.org/cpython

1 0

cpython (2.7): Issue #4844: ZipFile now raises BadZipfile when opens a ZIP file with an
by serhiy.storchaka Jan. 31, 2013

Jan. 31, 2013

http://hg.python.org/cpython/rev/32de35f0f877 changeset: 81847:32de35f0f877 branch: 2.7 parent: 81843:1d061f83a6bf user: Serhiy Storchaka <storchaka(a)gmail.com> date: Thu Jan 31 15:26:55 2013 +0200 summary: Issue #4844: ZipFile now raises BadZipfile when opens a ZIP file with an incomplete "End of Central Directory" record. Original patch by Guilherme Polo and Alan McIntyre. files: Lib/test/test_zipfile.py | 14 ++++++++++++++ Lib/zipfile.py | 27 ++++++++++++++++++++------- Misc/NEWS | 4 ++++ 3 files changed, 38 insertions(+), 7 deletions(-) diff --git a/Lib/test/test_zipfile.py b/Lib/test/test_zipfile.py --- a/Lib/test/test_zipfile.py +++ b/Lib/test/test_zipfile.py @@ -760,6 +760,20 @@ chk = zipfile.is_zipfile(fp) self.assertTrue(not chk) + def test_damaged_zipfile(self): + """Check that zipfiles with missing bytes at the end raise BadZipFile.""" + # - Create a valid zip file + fp = io.BytesIO() + with zipfile.ZipFile(fp, mode="w") as zipf: + zipf.writestr("foo.txt", b"O, for a Muse of Fire!") + zipfiledata = fp.getvalue() + + # - Now create copies of it missing the last N bytes and make sure + # a BadZipFile exception is raised when we try to open it + for N in range(len(zipfiledata)): + fp = io.BytesIO(zipfiledata[:N]) + self.assertRaises(zipfile.BadZipfile, zipfile.ZipFile, fp) + def test_is_zip_valid_file(self): """Check that is_zipfile() correctly identifies zip files.""" # - passing a filename diff --git a/Lib/zipfile.py b/Lib/zipfile.py --- a/Lib/zipfile.py +++ b/Lib/zipfile.py @@ -166,6 +166,8 @@ return endrec data = fpin.read(sizeEndCentDir64Locator) + if len(data) != sizeEndCentDir64Locator: + return endrec sig, diskno, reloff, disks = struct.unpack(structEndArchive64Locator, data) if sig != stringEndArchive64Locator: return endrec @@ -176,6 +178,8 @@ # Assume no 'zip64 extensible data' fpin.seek(offset - sizeEndCentDir64Locator - sizeEndCentDir64, 2) data = fpin.read(sizeEndCentDir64) + if len(data) != sizeEndCentDir64: + return endrec sig, sz, create_version, read_version, disk_num, disk_dir, \ dircount, dircount2, dirsize, diroffset = \ struct.unpack(structEndArchive64, data) @@ -211,7 +215,9 @@ except IOError: return None data = fpin.read() - if data[0:4] == stringEndArchive and data[-2:] == "\000\000": + if (len(data) == sizeEndCentDir and + data[0:4] == stringEndArchive and + data[-2:] == b"\000\000"): # the signature is correct and there's no comment, unpack structure endrec = struct.unpack(structEndArchive, data) endrec=list(endrec) @@ -235,6 +241,9 @@ if start >= 0: # found the magic number; attempt to unpack and interpret recData = data[start:start+sizeEndCentDir] + if len(recData) != sizeEndCentDir: + # Zip file is corrupted. + return None endrec = list(struct.unpack(structEndArchive, recData)) commentSize = endrec[_ECD_COMMENT_SIZE] #as claimed by the zip file comment = data[start+sizeEndCentDir:start+sizeEndCentDir+commentSize] @@ -246,7 +255,7 @@ endrec) # Unable to find a valid end of central directory structure - return + return None class ZipInfo (object): @@ -818,9 +827,11 @@ total = 0 while total < size_cd: centdir = fp.read(sizeCentralDir) - if centdir[0:4] != stringCentralDir: - raise BadZipfile, "Bad magic number for central directory" + if len(centdir) != sizeCentralDir: + raise BadZipfile("Truncated central directory") centdir = struct.unpack(structCentralDir, centdir) + if centdir[_CD_SIGNATURE] != stringCentralDir: + raise BadZipfile("Bad magic number for central directory") if self.debug > 2: print centdir filename = fp.read(centdir[_CD_FILENAME_LENGTH]) @@ -948,10 +959,12 @@ # Skip the file header: fheader = zef_file.read(sizeFileHeader) - if fheader[0:4] != stringFileHeader: - raise BadZipfile, "Bad magic number for file header" + if len(fheader) != sizeFileHeader: + raise BadZipfile("Truncated file header") + fheader = struct.unpack(structFileHeader, fheader) + if fheader[_FH_SIGNATURE] != stringFileHeader: + raise BadZipfile("Bad magic number for file header") - fheader = struct.unpack(structFileHeader, fheader) fname = zef_file.read(fheader[_FH_FILENAME_LENGTH]) if fheader[_FH_EXTRA_FIELD_LENGTH]: zef_file.read(fheader[_FH_EXTRA_FIELD_LENGTH]) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -202,6 +202,10 @@ Library ------- +- Issue #4844: ZipFile now raises BadZipfile when opens a ZIP file with an + incomplete "End of Central Directory" record. Original patch by Guilherme + Polo and Alan McIntyre. + - Issue #15505: `unittest.installHandler` no longer assumes SIGINT handler is set to a callable object. -- Repository URL: http://hg.python.org/cpython

1 0

cpython (merge 3.3 -> default): null merge
by ned.deily Jan. 31, 2013

Jan. 31, 2013

http://hg.python.org/cpython/rev/022477b09dc3 changeset: 81846:022477b09dc3 parent: 81842:58e72cb89848 parent: 81845:d218ecb7392d user: Ned Deily <nad(a)acm.org> date: Thu Jan 31 01:36:22 2013 -0800 summary: null merge files: -- Repository URL: http://hg.python.org/cpython

1 0

cpython (merge 3.2 -> 3.3): null merge
by ned.deily Jan. 31, 2013

Jan. 31, 2013

http://hg.python.org/cpython/rev/d218ecb7392d changeset: 81845:d218ecb7392d branch: 3.3 parent: 81840:91435b4dc8d4 parent: 81844:502b139b3b19 user: Ned Deily <nad(a)acm.org> date: Thu Jan 31 01:34:02 2013 -0800 summary: null merge files: -- Repository URL: http://hg.python.org/cpython

1 0