From python-checkins at python.org Sun Jul 1 10:11:36 2007 From: python-checkins at python.org (georg.brandl) Date: Sun, 1 Jul 2007 10:11:36 +0200 (CEST) Subject: [Python-checkins] r56137 - python/trunk/Lib/webbrowser.py Message-ID: <20070701081136.11BD51E4005@bag.python.org> Author: georg.brandl Date: Sun Jul 1 10:11:35 2007 New Revision: 56137 Modified: python/trunk/Lib/webbrowser.py Log: Fix a few webbrowser.py problems. Modified: python/trunk/Lib/webbrowser.py ============================================================================== --- python/trunk/Lib/webbrowser.py (original) +++ python/trunk/Lib/webbrowser.py Sun Jul 1 10:11:35 2007 @@ -1,5 +1,6 @@ #! /usr/bin/env python """Interfaces for launching and remotely controlling Web browsers.""" +# Maintained by Georg Brandl. import os import shlex @@ -160,6 +161,7 @@ def __init__(self, name): if isinstance(name, basestring): self.name = name + self.args = ["%s"] else: # name should be a list with arguments self.name = name[0] From python-checkins at python.org Sun Jul 1 10:11:40 2007 From: python-checkins at python.org (georg.brandl) Date: Sun, 1 Jul 2007 10:11:40 +0200 (CEST) Subject: [Python-checkins] r56138 - python/branches/release25-maint/Lib/webbrowser.py Message-ID: <20070701081140.7BD0B1E400B@bag.python.org> Author: georg.brandl Date: Sun Jul 1 10:11:40 2007 New Revision: 56138 Modified: python/branches/release25-maint/Lib/webbrowser.py Log: Fix a few webbrowser.py problems. (backport from rev. 56137) Modified: python/branches/release25-maint/Lib/webbrowser.py ============================================================================== --- python/branches/release25-maint/Lib/webbrowser.py (original) +++ python/branches/release25-maint/Lib/webbrowser.py Sun Jul 1 10:11:40 2007 @@ -1,5 +1,6 @@ #! /usr/bin/env python """Interfaces for launching and remotely controlling Web browsers.""" +# Maintained by Georg Brandl. import os import shlex @@ -160,6 +161,7 @@ def __init__(self, name): if isinstance(name, basestring): self.name = name + self.args = ["%s"] else: # name should be a list with arguments self.name = name[0] @@ -452,7 +454,7 @@ # if successful, register it if retncode is None and commd: - register("gnome", None, BackgroundBrowser(commd.split())) + register("gnome", None, BackgroundBrowser(shlex.split(commd))) # First, the Mozilla/Netscape browsers for browser in ("mozilla-firefox", "firefox", From buildbot at python.org Sun Jul 1 10:27:35 2007 From: buildbot at python.org (buildbot at python.org) Date: Sun, 01 Jul 2007 08:27:35 +0000 Subject: [Python-checkins] buildbot warnings in PPC64 Debian trunk Message-ID: <20070701082735.5431F1E4005@bag.python.org> The Buildbot has detected a new failure of PPC64 Debian trunk. Full details are available at: http://www.python.org/dev/buildbot/all/PPC64%2520Debian%2520trunk/builds/35 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: georg.brandl Build had warnings: warnings test Excerpt from the test logfile: make: *** [buildbottest] Segmentation fault sincerely, -The Buildbot From buildbot at python.org Sun Jul 1 10:51:41 2007 From: buildbot at python.org (buildbot at python.org) Date: Sun, 01 Jul 2007 08:51:41 +0000 Subject: [Python-checkins] buildbot warnings in ppc Debian unstable trunk Message-ID: <20070701085141.84CAB1E4005@bag.python.org> The Buildbot has detected a new failure of ppc Debian unstable trunk. Full details are available at: http://www.python.org/dev/buildbot/all/ppc%2520Debian%2520unstable%2520trunk/builds/35 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: georg.brandl Build had warnings: warnings test Excerpt from the test logfile: 1 test failed: test_bsddb3 make: *** [buildbottest] Error 1 sincerely, -The Buildbot From buildbot at python.org Sun Jul 1 11:37:24 2007 From: buildbot at python.org (buildbot at python.org) Date: Sun, 01 Jul 2007 09:37:24 +0000 Subject: [Python-checkins] buildbot warnings in hppa Ubuntu dapper trunk Message-ID: <20070701093724.371D11E4005@bag.python.org> The Buildbot has detected a new failure of hppa Ubuntu dapper trunk. Full details are available at: http://www.python.org/dev/buildbot/all/hppa%2520Ubuntu%2520dapper%2520trunk/builds/0 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: georg.brandl Build had warnings: warnings test Excerpt from the test logfile: sincerely, -The Buildbot From buildbot at python.org Sun Jul 1 12:56:36 2007 From: buildbot at python.org (buildbot at python.org) Date: Sun, 01 Jul 2007 10:56:36 +0000 Subject: [Python-checkins] buildbot warnings in hppa Ubuntu dapper 2.5 Message-ID: <20070701105636.480051E4005@bag.python.org> The Buildbot has detected a new failure of hppa Ubuntu dapper 2.5. Full details are available at: http://www.python.org/dev/buildbot/all/hppa%2520Ubuntu%2520dapper%25202.5/builds/0 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch branches/release25-maint] HEAD Blamelist: georg.brandl Build had warnings: warnings test Excerpt from the test logfile: sincerely, -The Buildbot From buildbot at python.org Sun Jul 1 14:22:35 2007 From: buildbot at python.org (buildbot at python.org) Date: Sun, 01 Jul 2007 12:22:35 +0000 Subject: [Python-checkins] buildbot warnings in MIPS Debian trunk Message-ID: <20070701122235.7B89C1E4005@bag.python.org> The Buildbot has detected a new failure of MIPS Debian trunk. Full details are available at: http://www.python.org/dev/buildbot/all/MIPS%2520Debian%2520trunk/builds/0 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: georg.brandl Build had warnings: warnings test Excerpt from the test logfile: 1 test failed: test_urllib2net ====================================================================== ERROR: test_ftp_NoneNodefault (test.test_urllib2net.TimeoutTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/pybot/buildarea/trunk.klose-debian-mips/build/Lib/test/test_urllib2net.py", line 306, in test_ftp_NoneNodefault u = urllib2.urlopen("ftp://ftp.mirror.nl/pub/mirror/gnu/", timeout=None) File "/home/pybot/buildarea/trunk.klose-debian-mips/build/Lib/urllib2.py", line 124, in urlopen return _opener.open(url, data, timeout) File "/home/pybot/buildarea/trunk.klose-debian-mips/build/Lib/urllib2.py", line 376, in open response = self._open(req, data) File "/home/pybot/buildarea/trunk.klose-debian-mips/build/Lib/urllib2.py", line 394, in _open '_open', req) File "/home/pybot/buildarea/trunk.klose-debian-mips/build/Lib/urllib2.py", line 354, in _call_chain result = func(*args) File "/home/pybot/buildarea/trunk.klose-debian-mips/build/Lib/urllib2.py", line 1273, in ftp_open fw = self.connect_ftp(user, passwd, host, port, dirs, req.timeout) File "/home/pybot/buildarea/trunk.klose-debian-mips/build/Lib/urllib2.py", line 1319, in connect_ftp self.cache[key] = ftpwrapper(user, passwd, host, port, dirs, timeout) File "/home/pybot/buildarea/trunk.klose-debian-mips/build/Lib/urllib.py", line 829, in __init__ self.init() File "/home/pybot/buildarea/trunk.klose-debian-mips/build/Lib/urllib.py", line 835, in init self.ftp.connect(self.host, self.port, self.timeout) File "/home/pybot/buildarea/trunk.klose-debian-mips/build/Lib/ftplib.py", line 129, in connect self.sock = socket.create_connection((self.host, self.port), self.timeout) File "/home/pybot/buildarea/trunk.klose-debian-mips/build/Lib/socket.py", line 443, in create_connection raise error, msg IOError: [Errno ftp error] (145, 'Connection timed out') ====================================================================== ERROR: test_ftp_NoneWithdefault (test.test_urllib2net.TimeoutTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/pybot/buildarea/trunk.klose-debian-mips/build/Lib/test/test_urllib2net.py", line 300, in test_ftp_NoneWithdefault u = urllib2.urlopen("ftp://ftp.mirror.nl/pub/mirror/gnu/", timeout=None) File "/home/pybot/buildarea/trunk.klose-debian-mips/build/Lib/urllib2.py", line 124, in urlopen return _opener.open(url, data, timeout) File "/home/pybot/buildarea/trunk.klose-debian-mips/build/Lib/urllib2.py", line 376, in open response = self._open(req, data) File "/home/pybot/buildarea/trunk.klose-debian-mips/build/Lib/urllib2.py", line 394, in _open '_open', req) File "/home/pybot/buildarea/trunk.klose-debian-mips/build/Lib/urllib2.py", line 354, in _call_chain result = func(*args) File "/home/pybot/buildarea/trunk.klose-debian-mips/build/Lib/urllib2.py", line 1273, in ftp_open fw = self.connect_ftp(user, passwd, host, port, dirs, req.timeout) File "/home/pybot/buildarea/trunk.klose-debian-mips/build/Lib/urllib2.py", line 1319, in connect_ftp self.cache[key] = ftpwrapper(user, passwd, host, port, dirs, timeout) File "/home/pybot/buildarea/trunk.klose-debian-mips/build/Lib/urllib.py", line 829, in __init__ self.init() File "/home/pybot/buildarea/trunk.klose-debian-mips/build/Lib/urllib.py", line 835, in init self.ftp.connect(self.host, self.port, self.timeout) File "/home/pybot/buildarea/trunk.klose-debian-mips/build/Lib/ftplib.py", line 129, in connect self.sock = socket.create_connection((self.host, self.port), self.timeout) File "/home/pybot/buildarea/trunk.klose-debian-mips/build/Lib/socket.py", line 443, in create_connection raise error, msg IOError: [Errno ftp error] timed out ====================================================================== ERROR: test_ftp_Value (test.test_urllib2net.TimeoutTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/pybot/buildarea/trunk.klose-debian-mips/build/Lib/test/test_urllib2net.py", line 310, in test_ftp_Value u = urllib2.urlopen("ftp://ftp.mirror.nl/pub/mirror/gnu/", timeout=60) File "/home/pybot/buildarea/trunk.klose-debian-mips/build/Lib/urllib2.py", line 124, in urlopen return _opener.open(url, data, timeout) File "/home/pybot/buildarea/trunk.klose-debian-mips/build/Lib/urllib2.py", line 376, in open response = self._open(req, data) File "/home/pybot/buildarea/trunk.klose-debian-mips/build/Lib/urllib2.py", line 394, in _open '_open', req) File "/home/pybot/buildarea/trunk.klose-debian-mips/build/Lib/urllib2.py", line 354, in _call_chain result = func(*args) File "/home/pybot/buildarea/trunk.klose-debian-mips/build/Lib/urllib2.py", line 1273, in ftp_open fw = self.connect_ftp(user, passwd, host, port, dirs, req.timeout) File "/home/pybot/buildarea/trunk.klose-debian-mips/build/Lib/urllib2.py", line 1319, in connect_ftp self.cache[key] = ftpwrapper(user, passwd, host, port, dirs, timeout) File "/home/pybot/buildarea/trunk.klose-debian-mips/build/Lib/urllib.py", line 829, in __init__ self.init() File "/home/pybot/buildarea/trunk.klose-debian-mips/build/Lib/urllib.py", line 835, in init self.ftp.connect(self.host, self.port, self.timeout) File "/home/pybot/buildarea/trunk.klose-debian-mips/build/Lib/ftplib.py", line 129, in connect self.sock = socket.create_connection((self.host, self.port), self.timeout) File "/home/pybot/buildarea/trunk.klose-debian-mips/build/Lib/socket.py", line 443, in create_connection raise error, msg IOError: [Errno ftp error] timed out ====================================================================== ERROR: test_ftp_basic (test.test_urllib2net.TimeoutTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/pybot/buildarea/trunk.klose-debian-mips/build/Lib/test/test_urllib2net.py", line 293, in test_ftp_basic u = urllib2.urlopen("ftp://ftp.mirror.nl/pub/mirror/gnu/") File "/home/pybot/buildarea/trunk.klose-debian-mips/build/Lib/urllib2.py", line 124, in urlopen return _opener.open(url, data, timeout) File "/home/pybot/buildarea/trunk.klose-debian-mips/build/Lib/urllib2.py", line 376, in open response = self._open(req, data) File "/home/pybot/buildarea/trunk.klose-debian-mips/build/Lib/urllib2.py", line 394, in _open '_open', req) File "/home/pybot/buildarea/trunk.klose-debian-mips/build/Lib/urllib2.py", line 354, in _call_chain result = func(*args) File "/home/pybot/buildarea/trunk.klose-debian-mips/build/Lib/urllib2.py", line 1273, in ftp_open fw = self.connect_ftp(user, passwd, host, port, dirs, req.timeout) File "/home/pybot/buildarea/trunk.klose-debian-mips/build/Lib/urllib2.py", line 1319, in connect_ftp self.cache[key] = ftpwrapper(user, passwd, host, port, dirs, timeout) File "/home/pybot/buildarea/trunk.klose-debian-mips/build/Lib/urllib.py", line 829, in __init__ self.init() File "/home/pybot/buildarea/trunk.klose-debian-mips/build/Lib/urllib.py", line 835, in init self.ftp.connect(self.host, self.port, self.timeout) File "/home/pybot/buildarea/trunk.klose-debian-mips/build/Lib/ftplib.py", line 129, in connect self.sock = socket.create_connection((self.host, self.port), self.timeout) File "/home/pybot/buildarea/trunk.klose-debian-mips/build/Lib/socket.py", line 443, in create_connection raise error, msg IOError: [Errno ftp error] (145, 'Connection timed out') make: *** [buildbottest] Error 1 sincerely, -The Buildbot From python-checkins at python.org Mon Jul 2 05:01:48 2007 From: python-checkins at python.org (brett.cannon) Date: Mon, 2 Jul 2007 05:01:48 +0200 (CEST) Subject: [Python-checkins] r56141 - python/branches/bcannon-objcap/build_secure_py.sh Message-ID: <20070702030148.226951E4007@bag.python.org> Author: brett.cannon Date: Mon Jul 2 05:01:47 2007 New Revision: 56141 Modified: python/branches/bcannon-objcap/build_secure_py.sh Log: Add a comment about what someone might need to change to build. Modified: python/branches/bcannon-objcap/build_secure_py.sh ============================================================================== --- python/branches/bcannon-objcap/build_secure_py.sh (original) +++ python/branches/bcannon-objcap/build_secure_py.sh Mon Jul 2 05:01:47 2007 @@ -1,4 +1,5 @@ echo "Build secure_python.o ..." gcc -Wall -g -c -IInclude -I. secure_python.c echo "Build executable ..." +# If the command below fails, might need to add -lutil or -Wl,-E . gcc -L. -lpthread -lm -ldl -o secure_python.exe secure_python.o libpython2.6.a From nnorwitz at gmail.com Mon Jul 2 11:43:35 2007 From: nnorwitz at gmail.com (Neal Norwitz) Date: Mon, 2 Jul 2007 05:43:35 -0400 Subject: [Python-checkins] Python Regression Test Failures refleak (1) Message-ID: <20070702094335.GA22051@python.psfb.org> test_popen2 leaked [-26, 26, -26] references, sum=-26 test_sys leaked [-132, 0, 0] references, sum=-132 test_urllib2_localnet leaked [3, 3, 3] references, sum=9 From python-checkins at python.org Mon Jul 2 13:54:28 2007 From: python-checkins at python.org (georg.brandl) Date: Mon, 2 Jul 2007 13:54:28 +0200 (CEST) Subject: [Python-checkins] r56143 - python/trunk/Doc/lib/libsignal.tex Message-ID: <20070702115428.8BE731E4007@bag.python.org> Author: georg.brandl Date: Mon Jul 2 13:54:28 2007 New Revision: 56143 Modified: python/trunk/Doc/lib/libsignal.tex Log: Remove duplicate sentence from alarm() doc. Modified: python/trunk/Doc/lib/libsignal.tex ============================================================================== --- python/trunk/Doc/lib/libsignal.tex (original) +++ python/trunk/Doc/lib/libsignal.tex Mon Jul 2 13:54:28 2007 @@ -101,8 +101,7 @@ be scheduled at any time). The returned value is then the number of seconds before any previously set alarm was to have been delivered. If \var{time} is zero, no alarm is scheduled, and any scheduled - alarm is canceled. The return value is the number of seconds - remaining before a previously scheduled alarm. If the return value + alarm is canceled. If the return value is zero, no alarm is currently scheduled. (See the \UNIX{} man page \manpage{alarm}{2}.) Availability: \UNIX. From python-checkins at python.org Mon Jul 2 13:54:31 2007 From: python-checkins at python.org (georg.brandl) Date: Mon, 2 Jul 2007 13:54:31 +0200 (CEST) Subject: [Python-checkins] r56144 - python/branches/release25-maint/Doc/lib/libsignal.tex Message-ID: <20070702115431.3DBBC1E400D@bag.python.org> Author: georg.brandl Date: Mon Jul 2 13:54:30 2007 New Revision: 56144 Modified: python/branches/release25-maint/Doc/lib/libsignal.tex Log: Remove duplicate sentence from alarm() doc. (backport from rev. 56143) Modified: python/branches/release25-maint/Doc/lib/libsignal.tex ============================================================================== --- python/branches/release25-maint/Doc/lib/libsignal.tex (original) +++ python/branches/release25-maint/Doc/lib/libsignal.tex Mon Jul 2 13:54:30 2007 @@ -101,8 +101,7 @@ be scheduled at any time). The returned value is then the number of seconds before any previously set alarm was to have been delivered. If \var{time} is zero, no alarm is scheduled, and any scheduled - alarm is canceled. The return value is the number of seconds - remaining before a previously scheduled alarm. If the return value + alarm is canceled. If the return value is zero, no alarm is currently scheduled. (See the \UNIX{} man page \manpage{alarm}{2}.) Availability: \UNIX. From python-checkins at python.org Mon Jul 2 15:19:58 2007 From: python-checkins at python.org (guido.van.rossum) Date: Mon, 2 Jul 2007 15:19:58 +0200 (CEST) Subject: [Python-checkins] r56145 - sandbox/trunk/2to3/fixes/fix_dict.py sandbox/trunk/2to3/fixes/fix_xrange.py Message-ID: <20070702131958.9ADC31E4007@bag.python.org> Author: guido.van.rossum Date: Mon Jul 2 15:19:58 2007 New Revision: 56145 Modified: sandbox/trunk/2to3/fixes/fix_dict.py sandbox/trunk/2to3/fixes/fix_xrange.py Log: Complete copyright assignment. Modified: sandbox/trunk/2to3/fixes/fix_dict.py ============================================================================== --- sandbox/trunk/2to3/fixes/fix_dict.py (original) +++ sandbox/trunk/2to3/fixes/fix_dict.py Mon Jul 2 15:19:58 2007 @@ -1,4 +1,5 @@ # Copyright 2007 Google, Inc. All Rights Reserved. +# Licensed to PSF under a Contributor Agreement. """Fixer for dict methods. Modified: sandbox/trunk/2to3/fixes/fix_xrange.py ============================================================================== --- sandbox/trunk/2to3/fixes/fix_xrange.py (original) +++ sandbox/trunk/2to3/fixes/fix_xrange.py Mon Jul 2 15:19:58 2007 @@ -1,4 +1,5 @@ # Copyright 2007 Google, Inc. All Rights Reserved. +# Licensed to PSF under a Contributor Agreement. """Fixer that changes xrange(...) into range(...).""" From python-checkins at python.org Mon Jul 2 15:20:43 2007 From: python-checkins at python.org (guido.van.rossum) Date: Mon, 2 Jul 2007 15:20:43 +0200 (CEST) Subject: [Python-checkins] r56146 - in sandbox/trunk/2to3: example.py fixes/fix_filter.py fixes/fix_map.py fixes/util.py tests/test_fixers.py Message-ID: <20070702132043.CBCAC1E4007@bag.python.org> Author: guido.van.rossum Date: Mon Jul 2 15:20:43 2007 New Revision: 56146 Added: sandbox/trunk/2to3/fixes/fix_filter.py (contents, props changed) sandbox/trunk/2to3/fixes/fix_map.py (contents, props changed) Modified: sandbox/trunk/2to3/example.py sandbox/trunk/2to3/fixes/util.py sandbox/trunk/2to3/tests/test_fixers.py Log: Add fixers for map and filter. Not perfect, but a start. Modified: sandbox/trunk/2to3/example.py ============================================================================== --- sandbox/trunk/2to3/example.py (original) +++ sandbox/trunk/2to3/example.py Mon Jul 2 15:20:43 2007 @@ -344,4 +344,17 @@ a = raw_input() b = raw_input(a.rstrip()) +def filter_examples(): + filter(os.unlink, filenames) + filter(None, "whatever") + filter(lambda x: not x, range(4)) + +def map_examples(): + map(None, foo.bar) + map(None, foo.bar,) + map(None, foo, bar) + map(f, foo.bar) + map(lambda x: x+1, range(10)) + + # This is the last line. Added: sandbox/trunk/2to3/fixes/fix_filter.py ============================================================================== --- (empty file) +++ sandbox/trunk/2to3/fixes/fix_filter.py Mon Jul 2 15:20:43 2007 @@ -0,0 +1,107 @@ +# Copyright 2007 Google, Inc. All Rights Reserved. +# Licensed to PSF under a Contributor Agreement. + +"""Fixer that changes filter(F, X) into list(filter(F, X)). + +We avoid the transformation if the filter() call is directly contained +in iter(<>), list(<>), tuple(<>), sorted(<>), ...join(<>), or +for V in <>:. + +NOTE: This is still not correct if the original code was depending on +filter(F, X) to return a string if X is a string and a tuple if X is a +tuple. That would require type inference, which we don't do. Let +Python 2.6 figure it out. +""" + +# Local imports +import pytree +import patcomp +from pgen2 import token +from fixes import basefix +from fixes.util import Name, Call, ListComp + +class FixFilter(basefix.BaseFix): + + PATTERN = """ + filter_lambda=power< + 'filter' + trailer< + '(' + arglist< + lambdef< 'lambda' fp=NAME ':' xp=any > + ',' + it=any + > + ')' + > + > + | + power< + 'filter' + args=trailer< '(' [any] ')' > + > + """ + + def transform(self, node): + results = self.match(node) + if "filter_lambda" in results: + new = ListComp(results.get("fp").clone(), + results.get("fp").clone(), + results.get("it").clone(), + results.get("xp").clone()) + else: + if self.in_special_context(node): + return None + new = node.clone() + new.set_prefix("") + new = Call(Name("list"), [new]) + new.set_prefix(node.get_prefix()) + return new + + P0 = """for_stmt< 'for' any 'in' node=any ':' any* > + | list_for< 'for' any 'in' node=any any* > + | gen_for< 'for' any 'in' node=any any* > + """ + p0 = patcomp.PatternCompiler().compile_pattern(P0) + + P1 = """ + power< + NAME< 'iter' | 'list' | 'tuple' | 'sorted' > + trailer< '(' node=any ')' > + any* + > + """ + p1 = patcomp.PatternCompiler().compile_pattern(P1) + + P2 = """ + power< + 'sorted' + trailer< '(' arglist ')' > + any* + > + """ + p2 = patcomp.PatternCompiler().compile_pattern(P2) + + def in_special_context(self, node): + p = node.parent + if p is None: + return False + results = {} + if self.p0.match(p, results) and results["node"] is node: + return True + + pp = p.parent + if pp is None: + return False + results = {} + if self.p1.match(pp, results) and results["node"] is node: + return True + + ppp = pp.parent + if ppp is None: + return False + results = {} + if self.p2.match(ppp, results) and results["node"] is node: + return True + + return False Added: sandbox/trunk/2to3/fixes/fix_map.py ============================================================================== --- (empty file) +++ sandbox/trunk/2to3/fixes/fix_map.py Mon Jul 2 15:20:43 2007 @@ -0,0 +1,119 @@ +# Copyright 2007 Google, Inc. All Rights Reserved. +# Licensed to PSF under a Contributor Agreement. + +"""Fixer that changes map(F, ...) into list(map(F, ...)). + +As a special case, map(None, X) is changed into list(X). (This is +necessary because the semantics are changed in this case -- the new +map(None, X) is equivalent to [(x,) for x in X].) + +We avoid the transformation (except for the special case mentioned +above) if the map() call is directly contained in iter(<>), list(<>), +tuple(<>), sorted(<>), ...join(<>), or for V in <>:. + +NOTE: This is still not correct if the original code was depending on +map(F, X, Y, ...) to go on until the longest argument is exhausted, +substituting None for missing values -- like zip(), it now stops as +soon as the shortest argument is exhausted. +""" + +# Local imports +import pytree +import patcomp +from pgen2 import token +from fixes import basefix +from fixes.util import Name, Call, ListComp +from pygram import python_symbols as syms + +class FixMap(basefix.BaseFix): + + PATTERN = """ + map_none=power< + 'map' + trailer< '(' arglist< 'None' ',' arg=any [','] > ')' > + > + | + map_lambda=power< + 'map' + trailer< + '(' + arglist< + lambdef< 'lambda' fp=NAME ':' xp=any > + ',' + it=any + > + ')' + > + > + | + power< + 'map' + args=trailer< '(' [any] ')' > + > + """ + + def transform(self, node): + results = self.match(node) + if "map_lambda" in results: + new = ListComp(results.get("xp").clone(), + results.get("fp").clone(), + results.get("it").clone()) + else: + if "map_none" in results: + new = results["arg"].clone() + else: + if self.in_special_context(node): + return None + new = node.clone() + new.set_prefix("") + new = Call(Name("list"), [new]) + new.set_prefix(node.get_prefix()) + return new + + P0 = """for_stmt< 'for' any 'in' node=any ':' any* > + | list_for< 'for' any 'in' node=any any* > + | gen_for< 'for' any 'in' node=any any* > + """ + p0 = patcomp.PatternCompiler().compile_pattern(P0) + + P1 = """ + power< + NAME< 'iter' | 'list' | 'tuple' | 'sorted' > + trailer< '(' node=any ')' > + any* + > + """ + p1 = patcomp.PatternCompiler().compile_pattern(P1) + + P2 = """ + power< + 'sorted' + trailer< '(' arglist ')' > + any* + > + """ + p2 = patcomp.PatternCompiler().compile_pattern(P2) + + def in_special_context(self, node): + p = node.parent + if p is None: + return False + results = {} + if self.p0.match(p, results) and results["node"] is node: + return True + + pp = p.parent + if pp is None: + return False + results = {} + if self.p1.match(pp, results) and results["node"] is node: + return True + + ppp = pp.parent + if ppp is None: + return False + results = {} + if self.p2.match(ppp, results) and results["node"] is node: + return True + + return False Modified: sandbox/trunk/2to3/fixes/util.py ============================================================================== --- sandbox/trunk/2to3/fixes/util.py (original) +++ sandbox/trunk/2to3/fixes/util.py Mon Jul 2 15:20:43 2007 @@ -76,6 +76,30 @@ """A string leaf""" return Leaf(token.STRING, string, prefix=prefix) +def ListComp(xp, fp, it, test=None): + """A list comprehension of the form [xp for fp in it if test]. + + If test is None, the "if test" part is omitted. + """ + xp.set_prefix("") + fp.set_prefix(" ") + it.set_prefix(" ") + for_leaf = Leaf(token.NAME, "for") + for_leaf.set_prefix(" ") + in_leaf = Leaf(token.NAME, "in") + in_leaf.set_prefix(" ") + inner_args = [for_leaf, fp, in_leaf, it] + if test: + test.set_prefix(" ") + if_leaf = Leaf(token.NAME, "if") + if_leaf.set_prefix(" ") + inner_args.append(Node(syms.list_if, [if_leaf, test])) + inner = Node(syms.listmaker, [xp, Node(syms.list_for, inner_args)]) + return Node(syms.atom, + [Leaf(token.LBRACE, "["), + inner, + Leaf(token.RBRACE, "]")]) + ########################################################### ### Determine whether a node represents a given literal ########################################################### Modified: sandbox/trunk/2to3/tests/test_fixers.py ============================================================================== --- sandbox/trunk/2to3/tests/test_fixers.py (original) +++ sandbox/trunk/2to3/tests/test_fixers.py Mon Jul 2 15:20:43 2007 @@ -1858,6 +1858,100 @@ a = """callable(x, kw=y)""" self.check(a, a) +class Test_filter(FixerTestCase): + fixer = "filter" + + def test_filter_basic(self): + b = """x = filter(None, 'abc')""" + a = """x = list(filter(None, 'abc'))""" + self.check(b, a) + + b = """x = filter(f, 'abc')""" + a = """x = list(filter(f, 'abc'))""" + self.check(b, a) + + b = """x = filter(lambda x: x%2 == 0, range(10))""" + a = """x = [x for x in range(10) if x%2 == 0]""" + self.check(b, a) + + # XXX This (rare) case is not supported +## b = """x = filter(f, 'abc')[0]""" +## a = """x = list(filter(f, 'abc'))[0]""" +## self.check(b, a) + + def test_filter_nochange(self): + a = """iter(filter(f, 'abc'))""" + self.check(a, a) + a = """list(filter(f, 'abc'))""" + self.check(a, a) + a = """list(filter(f, 'abc'))[0]""" + self.check(a, a) + a = """tuple(filter(f, 'abc'))""" + self.check(a, a) + a = """sorted(filter(f, 'abc'))""" + self.check(a, a) + a = """sorted(filter(f, 'abc'), key=blah)""" + self.check(a, a) + a = """sorted(filter(f, 'abc'), key=blah)[0]""" + self.check(a, a) + a = """for i in filter(f, 'abc'): pass""" + self.check(a, a) + a = """[x for x in filter(f, 'abc')]""" + self.check(a, a) + a = """(x for x in filter(f, 'abc'))""" + self.check(a, a) + +class Test_map(FixerTestCase): + fixer = "map" + + def test_map_basic(self): + b = """x = map(f, 'abc')""" + a = """x = list(map(f, 'abc'))""" + self.check(b, a) + + b = """x = map(f, 'abc', 'def')""" + a = """x = list(map(f, 'abc', 'def'))""" + self.check(b, a) + + b = """x = map(None, 'abc')""" + a = """x = list('abc')""" + self.check(b, a) + + b = """x = map(None, 'abc', 'def')""" + a = """x = list(map(None, 'abc', 'def'))""" + self.check(b, a) + + b = """x = map(lambda x: x+1, range(4))""" + a = """x = [x+1 for x in range(4)]""" + self.check(b, a) + + # XXX This (rare) case is not supported +## b = """x = map(f, 'abc')[0]""" +## a = """x = list(map(f, 'abc'))[0]""" +## self.check(b, a) + + def test_map_nochange(self): + a = """iter(map(f, 'abc'))""" + self.check(a, a) + a = """list(map(f, 'abc'))""" + self.check(a, a) + a = """list(map(f, 'abc'))[0]""" + self.check(a, a) + a = """tuple(map(f, 'abc'))""" + self.check(a, a) + a = """sorted(map(f, 'abc'))""" + self.check(a, a) + a = """sorted(map(f, 'abc'), key=blah)""" + self.check(a, a) + a = """sorted(map(f, 'abc'), key=blah)[0]""" + self.check(a, a) + a = """for i in map(f, 'abc'): pass""" + self.check(a, a) + a = """[x for x in map(f, 'abc')]""" + self.check(a, a) + a = """(x for x in map(f, 'abc'))""" + self.check(a, a) + if __name__ == "__main__": import __main__ From python-checkins at python.org Mon Jul 2 20:13:30 2007 From: python-checkins at python.org (alexandre.vassalotti) Date: Mon, 2 Jul 2007 20:13:30 +0200 (CEST) Subject: [Python-checkins] r56148 - python/branches/cpy_merge/Modules/_bytes_iomodule.c Message-ID: <20070702181330.1B8711E4007@bag.python.org> Author: alexandre.vassalotti Date: Mon Jul 2 20:13:29 2007 New Revision: 56148 Modified: python/branches/cpy_merge/Modules/_bytes_iomodule.c Log: Use the buffer protocol API for writelines and setvalue. Modified: python/branches/cpy_merge/Modules/_bytes_iomodule.c ============================================================================== --- python/branches/cpy_merge/Modules/_bytes_iomodule.c (original) +++ python/branches/cpy_merge/Modules/_bytes_iomodule.c Mon Jul 2 20:13:29 2007 @@ -146,6 +146,9 @@ static int bytes_io_setvalue(BytesIOObject *self, PyObject *value) { + const char *bytes; + Py_ssize_t len; + if (self->buf == NULL) { err_closed(); return -1; @@ -156,13 +159,11 @@ if (value == NULL) return 0; - - if (!PyString_Check(value)) { - PyErr_SetString(PyExc_TypeError, "need a string"); + + if (PyObject_AsCharBuffer(value, &bytes, &len) == -1) return -1; - } - if ((write_bytes(self, PyString_AsString(value), - PyString_Size(value))) < 0) { + + if (write_bytes(self, bytes, len) < 0) { return -1; /* out of memory */ } /* Reset the position back to beginning-of-file, since @@ -385,6 +386,8 @@ bytes_io_writelines(BytesIOObject *self, PyObject *v) { PyObject *it, *item; + const char *bytes; + Py_ssize_t len; if (self->buf == NULL) return err_closed(); @@ -394,16 +397,14 @@ return NULL; while ((item = PyIter_Next(it)) != NULL) { - Py_ssize_t n; - char *c; - if (PyString_AsStringAndSize(item, &c, &n) == -1) { + if (PyObject_AsCharBuffer(item, &bytes, &len) == -1) { Py_DECREF(it); Py_DECREF(item); return NULL; } Py_DECREF(item); - if (write_bytes(self, c, n) == -1) { + if (write_bytes(self, bytes, len) == -1) { Py_DECREF(it); return NULL; } From python-checkins at python.org Mon Jul 2 20:20:18 2007 From: python-checkins at python.org (alexandre.vassalotti) Date: Mon, 2 Jul 2007 20:20:18 +0200 (CEST) Subject: [Python-checkins] r56149 - python/branches/cpy_merge/Lib/test/test_memoryio.py Message-ID: <20070702182018.6143E1E4011@bag.python.org> Author: alexandre.vassalotti Date: Mon Jul 2 20:20:18 2007 New Revision: 56149 Added: python/branches/cpy_merge/Lib/test/test_memoryio.py Log: Add unit tests for memory-based IO. Not perfect, but it's a start. Added: python/branches/cpy_merge/Lib/test/test_memoryio.py ============================================================================== --- (empty file) +++ python/branches/cpy_merge/Lib/test/test_memoryio.py Mon Jul 2 20:20:18 2007 @@ -0,0 +1,248 @@ +"""Unit tests for memory-based file-like objects. +StringIO -- for unicode strings +BytesIO -- for bytes +""" + +import unittest +from test import test_support + +import io + +try: + import _string_io, _bytes_io + has_c_implementation = True +except ImportError: + has_c_implementation = False + + +class MemoryTestMixin: + + def write_ops(self, f): + t = self.buftype + self.assertEqual(f.write(t("blah.")), 5) + self.assertEqual(f.seek(0), 0) + self.assertEqual(f.write(t("Hello.")), 6) + self.assertEqual(f.tell(), 6) + self.assertEqual(f.seek(-1, 1), 5) + self.assertEqual(f.tell(), 5) + self.assertEqual(f.write(t(" world\n\n\n")), 9) + self.assertEqual(f.seek(0), 0) + self.assertEqual(f.write(t("h")), 1) + self.assertEqual(f.seek(-1, 2), 13) + self.assertEqual(f.tell(), 13) + self.assertEqual(f.truncate(12), 12) + self.assertEqual(f.tell(), 13) + + def test_write(self): + buf = self.buftype("hello world\n") + memio = self.ioclass(buf) + + self.write_ops(memio) + + def test_write_noinit(self): + memio = self.ioclass() + + self.write_ops(memio) + + def test_writelines(self): + buf = self.buftype("1234567890") + memio = self.ioclass() + + memio.writelines([buf] * 100) + self.assertEqual(memio.getvalue(), buf * 100) + + def test_writelines_error(self): + memio = self.ioclass() + def error_gen(): + yield self.buftype('spam') + raise KeyboardInterrupt + + self.assertRaises(KeyboardInterrupt, memio.writelines, error_gen()) + + def test_truncate(self): + buf = self.buftype("1234567890") + memio = self.ioclass(buf) + + memio.seek(6) + self.assertEqual(memio.truncate(), 6) + self.assertEqual(memio.getvalue(), buf[:6]) + self.assertEqual(memio.truncate(4), 4) + self.assertEqual(memio.getvalue(), buf[:4]) + self.assertEqual(memio.tell(), 4) + memio.write(buf) + self.assertEqual(memio.getvalue(), buf[:4] + buf) + self.assertRaises(IOError, memio.truncate, -1) + + def test_close(self): + buf = self.buftype("1234567890") + memio = self.ioclass(buf) + + self.assertEqual(memio.closed, False) + memio.close() + self.assertRaises(ValueError, memio.read) + self.assertRaises(ValueError, memio.write, buf) + self.assertRaises(ValueError, memio.writelines, buf) + self.assertRaises(ValueError, memio.seek, 3) + self.assertRaises(ValueError, memio.truncate) + self.assertRaises(ValueError, memio.getvalue) + self.assertRaises(ValueError, memio.tell) + self.assertEqual(memio.closed, True) + + def test_init(self): + buf = self.buftype("1234567890") + memio = self.ioclass(buf) + + def read_ops(self, f, data): + t = self.buftype + data = f.read(5) + self.assertEqual(data, t("hello")) + self.assertEqual(f.readinto(data), 5) + self.assertEqual(data, t(" worl")) + self.assertEqual(f.readinto(data), 2) + self.assertEqual(len(data), 5) + self.assertEqual(data[:2], t("d\n")) + self.assertEqual(f.seek(0), 0) + self.assertEqual(f.read(20), t("hello world\n")) + self.assertEqual(f.read(1), t("")) + self.assertEqual(f.readinto(t("x")), 0) + self.assertEqual(f.seek(-6, 2), 6) + self.assertEqual(f.read(5), t("world")) + self.assertEqual(f.read(0), t("")) + self.assertEqual(f.readinto(t("")), 0) + self.assertEqual(f.seek(-6, 1), 5) + self.assertEqual(f.read(5), t(" worl")) + self.assertEqual(f.tell(), 10) + + def test_read(self): + buf = self.buftype("1234567890") + memio = self.ioclass(buf) + + self.assertEqual(memio.read(1), buf[:1]) + self.assertEqual(memio.read(4), buf[1:5]) + self.assertEqual(memio.read(900), buf[5:]) + self.assertEqual(memio.read(), self.EOF) + + buf = self.buftype("hello world\n") + memio = self.ioclass(buf) + + self.read_ops(memio, buf) + + def test_read_noargs(self): + buf = self.buftype("1234567890") + memio = self.ioclass(buf) + + self.assertEqual(buf, memio.read()) + self.assertEqual(self.EOF, memio.read()) + + def test_readline(self): + pass + + def test_readlines(self): + pass + + def test_iterator(self): + buf = self.buftype("1234567890\n") + memio = self.ioclass(buf * 10) + + self.assertEqual(iter(memio), memio) + self.failUnless(hasattr(memio, '__iter__')) + self.failUnless(hasattr(memio, '__next__')) + i = 0 + for line in memio: + self.assertEqual(line, buf) + i += 1 + self.assertEqual(i, 10) + + memio.seek(0) + i = 0 + for line in memio: + self.assertEqual(line, buf) + i += 1 + self.assertEqual(i, 10) + + def test_getvalue(self): + buf = self.buftype("1234567890") + memio = self.ioclass(buf) + + self.assertEqual(memio.getvalue(), buf) + memio = self.ioclass(buf * 1000) + self.assertEqual(memio.getvalue()[-3:], "890") + + def test_seek(self): + buf = self.buftype("1234567890") + memio = self.ioclass(buf) + + memio.read(5) + memio.seek(0) + self.assertEqual(buf, memio.read()) + + memio.seek(3) + self.assertEqual(buf[3:], memio.read()) + + def test_tell(self): + buf = self.buftype("1234567890") + memio = self.ioclass(buf) + + self.assertEqual(0, memio.tell()) + memio.seek(5) + self.assertEqual(5, memio.tell()) + memio.seek(10000) + self.assertEqual(10000, memio.tell()) + + def test_flags(self): + memio = self.ioclass() + + self.assertEqual(memio.writable(), True) + self.assertEqual(memio.readable(), True) + self.assertEqual(memio.seekable(), True) + self.assertEqual(memio.isatty(), False) + memio.close() + self.assertEqual(memio.writable(), True) + self.assertEqual(memio.readable(), True) + self.assertEqual(memio.seekable(), True) + + +class PythonBytesIOTest(MemoryTestMixin, unittest.TestCase): + """ + Test the Python implementation of BytesIO. + """ + buftype = bytes + ioclass = io._BytesIO + EOF = b"" + + +class PythonStringIOTest(MemoryTestMixin, unittest.TestCase): + """ + Test the Python implementation of StringIO. + """ + buftype = str + ioclass = io._StringIO + EOF = "" + +if has_c_implementation: + class CBytesIOTest(MemoryTestMixin, unittest.TestCase): + """ + Test the C implementation of BytesIO if available. + """ + buftype = bytes + ioclass = _bytes_io.BytesIO + EOF = b"" + + + class CStringIOTest(MemoryTestMixin, unittest.TestCase): + """ + Test the C implementation of StringIO if available. + """ + buftype = unicode + ioclass = _string_io.StringIO + EOF = u"" + +def test_main(): + if has_c_implementation: + test_support.run_unittest(PythonBytesIOTest, PythonStringIOTest, + CBytesIOTest, CStringIOTest) + else: + test_support.run_unittest(PythonBytesIOTest, PythonStringIOTest) + +if __name__ == '__main__': + test_main() From python-checkins at python.org Mon Jul 2 22:08:49 2007 From: python-checkins at python.org (alexandre.vassalotti) Date: Mon, 2 Jul 2007 22:08:49 +0200 (CEST) Subject: [Python-checkins] r56150 - python/branches/cpy_merge/Modules/_bytes_iomodule.c Message-ID: <20070702200849.DEF6C1E4007@bag.python.org> Author: alexandre.vassalotti Date: Mon Jul 2 22:08:49 2007 New Revision: 56150 Modified: python/branches/cpy_merge/Modules/_bytes_iomodule.c Log: Add the readinto method. Modified: python/branches/cpy_merge/Modules/_bytes_iomodule.c ============================================================================== --- python/branches/cpy_merge/Modules/_bytes_iomodule.c (original) +++ python/branches/cpy_merge/Modules/_bytes_iomodule.c Mon Jul 2 22:08:49 2007 @@ -278,6 +278,24 @@ } static PyObject * +bytes_io_readinto(BytesIOObject *self, PyObject *buffer) +{ + void *raw_buffer; + Py_ssize_t len; + + if (PyObject_AsWriteBuffer(buffer, &raw_buffer, &len) == -1) + return NULL; + + if (len > self->string_size) + len = self->string_size; + + memcpy(raw_buffer, self->buf + self->pos, len); + self->pos += len; + + return PyInt_FromSsize_t(len); +} + +static PyObject * bytes_io_truncate(BytesIOObject *self, PyObject *args) { Py_ssize_t size; @@ -517,6 +535,12 @@ "The optional size argument, if given, is an approximate bound on the\n" "total number of bytes in the lines returned.\n"); +PyDoc_STRVAR(BytesIO_readinto_doc, +"readinto(bytes) -> int. Read up to len(b) bytes into b.\n" +"\n" +"Returns number of bytes read (0 for EOF), or None if the object\n" +"is set not to block as has no data to read." + PyDoc_STRVAR(BytesIO_tell_doc, "tell() -> current file position, an integer\n"); @@ -579,6 +603,8 @@ BytesIO_readline_doc}, {"readlines", (PyCFunction) bytes_io_readlines, METH_VARARGS, BytesIO_readlines_doc}, + {"readinto", (PyCFunction) bytes_io_readinto, METH_O, + BytesIO_readinto_doc}, {"tell", (PyCFunction) bytes_io_tell, METH_NOARGS, BytesIO_tell_doc}, {"truncate", (PyCFunction) bytes_io_truncate, METH_VARARGS, From python-checkins at python.org Mon Jul 2 22:39:49 2007 From: python-checkins at python.org (alexandre.vassalotti) Date: Mon, 2 Jul 2007 22:39:49 +0200 (CEST) Subject: [Python-checkins] r56151 - python/branches/cpy_merge/Modules/_bytes_iomodule.c Message-ID: <20070702203949.67CB11E4007@bag.python.org> Author: alexandre.vassalotti Date: Mon Jul 2 22:39:49 2007 New Revision: 56151 Modified: python/branches/cpy_merge/Modules/_bytes_iomodule.c Log: Make readinto set the buffer position correctly. Fix a minor error with the docstring of readinto. Modified: python/branches/cpy_merge/Modules/_bytes_iomodule.c ============================================================================== --- python/branches/cpy_merge/Modules/_bytes_iomodule.c (original) +++ python/branches/cpy_merge/Modules/_bytes_iomodule.c Mon Jul 2 22:39:49 2007 @@ -286,8 +286,8 @@ if (PyObject_AsWriteBuffer(buffer, &raw_buffer, &len) == -1) return NULL; - if (len > self->string_size) - len = self->string_size; + if (self->pos + len > self->string_size) + len = self->string_size - self->pos; memcpy(raw_buffer, self->buf + self->pos, len); self->pos += len; @@ -539,7 +539,7 @@ "readinto(bytes) -> int. Read up to len(b) bytes into b.\n" "\n" "Returns number of bytes read (0 for EOF), or None if the object\n" -"is set not to block as has no data to read." +"is set not to block as has no data to read."); PyDoc_STRVAR(BytesIO_tell_doc, "tell() -> current file position, an integer\n"); From python-checkins at python.org Mon Jul 2 22:53:11 2007 From: python-checkins at python.org (alexandre.vassalotti) Date: Mon, 2 Jul 2007 22:53:11 +0200 (CEST) Subject: [Python-checkins] r56152 - python/branches/cpy_merge/Lib/test/test_memoryio.py Message-ID: <20070702205311.81BA11E4007@bag.python.org> Author: alexandre.vassalotti Date: Mon Jul 2 22:53:11 2007 New Revision: 56152 Modified: python/branches/cpy_merge/Lib/test/test_memoryio.py Log: Add a unit test for BytesIO.readinto Clean up the main MemoryIOTest. Modified: python/branches/cpy_merge/Lib/test/test_memoryio.py ============================================================================== --- python/branches/cpy_merge/Lib/test/test_memoryio.py (original) +++ python/branches/cpy_merge/Lib/test/test_memoryio.py Mon Jul 2 22:53:11 2007 @@ -31,7 +31,7 @@ self.assertEqual(f.seek(-1, 2), 13) self.assertEqual(f.tell(), 13) self.assertEqual(f.truncate(12), 12) - self.assertEqual(f.tell(), 13) + self.assertEqual(f.tell(), 12) def test_write(self): buf = self.buftype("hello world\n") @@ -92,27 +92,6 @@ buf = self.buftype("1234567890") memio = self.ioclass(buf) - def read_ops(self, f, data): - t = self.buftype - data = f.read(5) - self.assertEqual(data, t("hello")) - self.assertEqual(f.readinto(data), 5) - self.assertEqual(data, t(" worl")) - self.assertEqual(f.readinto(data), 2) - self.assertEqual(len(data), 5) - self.assertEqual(data[:2], t("d\n")) - self.assertEqual(f.seek(0), 0) - self.assertEqual(f.read(20), t("hello world\n")) - self.assertEqual(f.read(1), t("")) - self.assertEqual(f.readinto(t("x")), 0) - self.assertEqual(f.seek(-6, 2), 6) - self.assertEqual(f.read(5), t("world")) - self.assertEqual(f.read(0), t("")) - self.assertEqual(f.readinto(t("")), 0) - self.assertEqual(f.seek(-6, 1), 5) - self.assertEqual(f.read(5), t(" worl")) - self.assertEqual(f.tell(), 10) - def test_read(self): buf = self.buftype("1234567890") memio = self.ioclass(buf) @@ -121,11 +100,9 @@ self.assertEqual(memio.read(4), buf[1:5]) self.assertEqual(memio.read(900), buf[5:]) self.assertEqual(memio.read(), self.EOF) - - buf = self.buftype("hello world\n") - memio = self.ioclass(buf) - - self.read_ops(memio, buf) + self.assertEqual(memio.seek(0), 0) + self.assertEqual(memio.read(), buf) + self.assertEqual(memio.tell(), 10) def test_read_noargs(self): buf = self.buftype("1234567890") @@ -202,7 +179,7 @@ self.assertEqual(memio.seekable(), True) -class PythonBytesIOTest(MemoryTestMixin, unittest.TestCase): +class PyBytesIOTest(MemoryTestMixin, unittest.TestCase): """ Test the Python implementation of BytesIO. """ @@ -210,36 +187,50 @@ ioclass = io._BytesIO EOF = b"" + def test_readinto(self): + buf = self.buftype("1234567890") + memio = self.ioclass(buf) + + b = bytes("hello") + self.assertEqual(memio.readinto(b), 5) + self.assertEqual(b, b"12345") + self.assertEqual(memio.readinto(b), 5) + self.assertEqual(b, b"67890") + self.assertEqual(memio.readinto(b), 0) + self.assertEqual(b, b"67890") -class PythonStringIOTest(MemoryTestMixin, unittest.TestCase): + b = bytes("hello world") + self.assertEqual(memio.seek(0), 0) + self.assertEqual(memio.readinto(b), 10) + self.assertEqual(b, "1234567890d") + + +class PyStringIOTest(MemoryTestMixin, unittest.TestCase): """ Test the Python implementation of StringIO. """ - buftype = str + buftype = unicode ioclass = io._StringIO EOF = "" if has_c_implementation: - class CBytesIOTest(MemoryTestMixin, unittest.TestCase): + class CBytesIOTest(PyBytesIOTest): """ Test the C implementation of BytesIO if available. """ - buftype = bytes ioclass = _bytes_io.BytesIO - EOF = b"" - class CStringIOTest(MemoryTestMixin, unittest.TestCase): + class CStringIOTest(PyStringIOTest): """ Test the C implementation of StringIO if available. """ - buftype = unicode ioclass = _string_io.StringIO - EOF = u"" + def test_main(): if has_c_implementation: - test_support.run_unittest(PythonBytesIOTest, PythonStringIOTest, + test_support.run_unittest(PyBytesIOTest, PyStringIOTest, CBytesIOTest, CStringIOTest) else: test_support.run_unittest(PythonBytesIOTest, PythonStringIOTest) From python-checkins at python.org Mon Jul 2 23:23:39 2007 From: python-checkins at python.org (alexandre.vassalotti) Date: Mon, 2 Jul 2007 23:23:39 +0200 (CEST) Subject: [Python-checkins] r56153 - python/branches/cpy_merge/Lib/test/test_memoryio.py Message-ID: <20070702212339.574671E4007@bag.python.org> Author: alexandre.vassalotti Date: Mon Jul 2 23:23:39 2007 New Revision: 56153 Modified: python/branches/cpy_merge/Lib/test/test_memoryio.py Log: Add test for readline. Add test for readlines. Don't check the return value of seek in other tests. Fix a typo, PythonBytesIOTest -> PyBytesIOTest Modified: python/branches/cpy_merge/Lib/test/test_memoryio.py ============================================================================== --- python/branches/cpy_merge/Lib/test/test_memoryio.py (original) +++ python/branches/cpy_merge/Lib/test/test_memoryio.py Mon Jul 2 23:23:39 2007 @@ -100,7 +100,7 @@ self.assertEqual(memio.read(4), buf[1:5]) self.assertEqual(memio.read(900), buf[5:]) self.assertEqual(memio.read(), self.EOF) - self.assertEqual(memio.seek(0), 0) + memio.seek(0) self.assertEqual(memio.read(), buf) self.assertEqual(memio.tell(), 10) @@ -112,10 +112,26 @@ self.assertEqual(self.EOF, memio.read()) def test_readline(self): - pass + buf = self.buftype("1234567890\n") + memio = self.ioclass(buf * 2) + + self.assertEqual(memio.readline(), buf) + self.assertEqual(memio.readline(), buf) + self.assertEqual(memio.readline(), self.EOF) + memio.seek(0) + self.assertEqual(memio.readline(5), "12345") + self.assertEqual(memio.readline(5), "67890") + self.assertEqual(memio.readline(5), '\n') def test_readlines(self): - pass + buf = self.buftype("1234567890\n") + memio = self.ioclass(buf * 10) + + self.assertEqual(memio.readlines(), [buf] * 10) + memio.seek(5) + self.assertEqual(memio.readlines(), ['67890\n'] + [buf] * 9) + memio.seek(0) + self.assertEqual(memio.readlines(15), [buf] * 2) def test_iterator(self): buf = self.buftype("1234567890\n") @@ -200,7 +216,7 @@ self.assertEqual(b, b"67890") b = bytes("hello world") - self.assertEqual(memio.seek(0), 0) + memio.seek(0) self.assertEqual(memio.readinto(b), 10) self.assertEqual(b, "1234567890d") @@ -233,7 +249,7 @@ test_support.run_unittest(PyBytesIOTest, PyStringIOTest, CBytesIOTest, CStringIOTest) else: - test_support.run_unittest(PythonBytesIOTest, PythonStringIOTest) + test_support.run_unittest(PyBytesIOTest, PyStringIOTest) if __name__ == '__main__': test_main() From python-checkins at python.org Mon Jul 2 23:46:52 2007 From: python-checkins at python.org (alexandre.vassalotti) Date: Mon, 2 Jul 2007 23:46:52 +0200 (CEST) Subject: [Python-checkins] r56154 - python/branches/cpy_merge/Modules/_bytes_iomodule.c python/branches/cpy_merge/Modules/_string_iomodule.c Message-ID: <20070702214652.C95871E4007@bag.python.org> Author: alexandre.vassalotti Date: Mon Jul 2 23:46:52 2007 New Revision: 56154 Modified: python/branches/cpy_merge/Modules/_bytes_iomodule.c python/branches/cpy_merge/Modules/_string_iomodule.c Log: Fix a slight inefficiency in the seek method. Modified: python/branches/cpy_merge/Modules/_bytes_iomodule.c ============================================================================== --- python/branches/cpy_merge/Modules/_bytes_iomodule.c (original) +++ python/branches/cpy_merge/Modules/_bytes_iomodule.c Mon Jul 2 23:46:52 2007 @@ -342,7 +342,7 @@ static PyObject * bytes_io_seek(BytesIOObject *self, PyObject *args) { - Py_ssize_t newpos; + Py_ssize_t newpos, prevpos; int mode = 0; if (self->buf == NULL) @@ -371,12 +371,15 @@ if (resize_buffer(self, newpos) < 0) return NULL; /* out of memory */ + prevpos = self->pos; self->pos = newpos; - /* Pad with zeros the buffer region larger than the string size. - XXX This is inefficient for multiple seeks. */ - while (--newpos >= self->string_size) + /* Pad with zeros the buffer region larger than the string size and + not previously padded with zeros. */ + while (newpos >= self->string_size && newpos >= prevpos) { self->buf[newpos] = 0; + newpos--; + } return PyInt_FromSsize_t(self->pos); } Modified: python/branches/cpy_merge/Modules/_string_iomodule.c ============================================================================== --- python/branches/cpy_merge/Modules/_string_iomodule.c (original) +++ python/branches/cpy_merge/Modules/_string_iomodule.c Mon Jul 2 23:46:52 2007 @@ -322,7 +322,7 @@ static PyObject * string_io_seek(StringIOObject *self, PyObject *args) { - Py_ssize_t newpos; + Py_ssize_t newpos, prevpos; int mode = 0; if (self->buf == NULL) @@ -351,12 +351,15 @@ if (resize_buffer(self, newpos) < 0) return NULL; /* out of memory */ + prevpos = self->pos; self->pos = newpos; - /* Pad with zeros the buffer region larger than the string size. - XXX This is inefficient for multiple seeks. */ - while (--newpos >= self->string_size) + /* Pad with zeros the buffer region larger than the string size and + not previously padded with zeros. */ + while (newpos >= self->string_size && newpos >= prevpos) { self->buf[newpos] = 0; + newpos--; + } return PyInt_FromSsize_t(self->pos); } From python-checkins at python.org Tue Jul 3 08:05:09 2007 From: python-checkins at python.org (neal.norwitz) Date: Tue, 3 Jul 2007 08:05:09 +0200 (CEST) Subject: [Python-checkins] r56156 - peps/trunk/pep-3100.txt Message-ID: <20070703060509.695641E4003@bag.python.org> Author: neal.norwitz Date: Tue Jul 3 08:05:09 2007 New Revision: 56156 Modified: peps/trunk/pep-3100.txt Log: Now map and filter also return an iterator Modified: peps/trunk/pep-3100.txt ============================================================================== --- peps/trunk/pep-3100.txt (original) +++ peps/trunk/pep-3100.txt Tue Jul 3 08:05:09 2007 @@ -168,7 +168,7 @@ ================== * Make built-ins return an iterator where appropriate (e.g. ``range()``, - ``zip()``, ``map()``, ``filter()``, etc.) [zip and range: done] + ``zip()``, ``map()``, ``filter()``, etc.) [done] * Relevant functions should consume iterators (e.g. ``min()``, ``max()``) [They already do, since 2.2.] * Remove ``input()`` and rename ``raw_input()`` to ``input()``. From python-checkins at python.org Tue Jul 3 17:41:23 2007 From: python-checkins at python.org (alexandre.vassalotti) Date: Tue, 3 Jul 2007 17:41:23 +0200 (CEST) Subject: [Python-checkins] r56160 - python/branches/cpy_merge/Modules/_bytes_iomodule.c Message-ID: <20070703154123.37C661E4007@bag.python.org> Author: alexandre.vassalotti Date: Tue Jul 3 17:41:22 2007 New Revision: 56160 Modified: python/branches/cpy_merge/Modules/_bytes_iomodule.c Log: Check if the object is closed in the readinto method. Modified: python/branches/cpy_merge/Modules/_bytes_iomodule.c ============================================================================== --- python/branches/cpy_merge/Modules/_bytes_iomodule.c (original) +++ python/branches/cpy_merge/Modules/_bytes_iomodule.c Tue Jul 3 17:41:22 2007 @@ -283,6 +283,9 @@ void *raw_buffer; Py_ssize_t len; + if (self->buf == NULL) + return err_closed(); + if (PyObject_AsWriteBuffer(buffer, &raw_buffer, &len) == -1) return NULL; From python-checkins at python.org Tue Jul 3 18:43:37 2007 From: python-checkins at python.org (guido.van.rossum) Date: Tue, 3 Jul 2007 18:43:37 +0200 (CEST) Subject: [Python-checkins] r56163 - in sandbox/trunk/2to3: fixes/fix_filter.py fixes/fix_map.py tests/test_fixers.py Message-ID: <20070703164337.4BB401E4003@bag.python.org> Author: guido.van.rossum Date: Tue Jul 3 18:43:36 2007 New Revision: 56163 Modified: sandbox/trunk/2to3/fixes/fix_filter.py sandbox/trunk/2to3/fixes/fix_map.py sandbox/trunk/2to3/tests/test_fixers.py Log: Fix a problem where a filter() or map() call inside another call was always ignored (instead of only when the called function was iter(), list(), tuple() or sorted()). Modified: sandbox/trunk/2to3/fixes/fix_filter.py ============================================================================== --- sandbox/trunk/2to3/fixes/fix_filter.py (original) +++ sandbox/trunk/2to3/fixes/fix_filter.py Tue Jul 3 18:43:36 2007 @@ -66,7 +66,7 @@ P1 = """ power< - NAME< 'iter' | 'list' | 'tuple' | 'sorted' > + ( 'iter' | 'list' | 'tuple' | 'sorted' ) trailer< '(' node=any ')' > any* > Modified: sandbox/trunk/2to3/fixes/fix_map.py ============================================================================== --- sandbox/trunk/2to3/fixes/fix_map.py (original) +++ sandbox/trunk/2to3/fixes/fix_map.py Tue Jul 3 18:43:36 2007 @@ -78,7 +78,7 @@ P1 = """ power< - NAME< 'iter' | 'list' | 'tuple' | 'sorted' > + ( 'iter' | 'list' | 'tuple' | 'sorted' ) trailer< '(' node=any ')' > any* > Modified: sandbox/trunk/2to3/tests/test_fixers.py ============================================================================== --- sandbox/trunk/2to3/tests/test_fixers.py (original) +++ sandbox/trunk/2to3/tests/test_fixers.py Tue Jul 3 18:43:36 2007 @@ -1866,8 +1866,8 @@ a = """x = list(filter(None, 'abc'))""" self.check(b, a) - b = """x = filter(f, 'abc')""" - a = """x = list(filter(f, 'abc'))""" + b = """x = len(filter(f, 'abc'))""" + a = """x = len(list(filter(f, 'abc')))""" self.check(b, a) b = """x = filter(lambda x: x%2 == 0, range(10))""" @@ -1909,8 +1909,8 @@ a = """x = list(map(f, 'abc'))""" self.check(b, a) - b = """x = map(f, 'abc', 'def')""" - a = """x = list(map(f, 'abc', 'def'))""" + b = """x = len(map(f, 'abc', 'def'))""" + a = """x = len(list(map(f, 'abc', 'def')))""" self.check(b, a) b = """x = map(None, 'abc')""" From python-checkins at python.org Wed Jul 4 01:23:59 2007 From: python-checkins at python.org (alexandre.vassalotti) Date: Wed, 4 Jul 2007 01:23:59 +0200 (CEST) Subject: [Python-checkins] r56169 - python/branches/cpy_merge/Lib/test/test_memoryio.py Message-ID: <20070703232359.EFA051E4003@bag.python.org> Author: alexandre.vassalotti Date: Wed Jul 4 01:23:59 2007 New Revision: 56169 Modified: python/branches/cpy_merge/Lib/test/test_memoryio.py Log: Clean up the unit tests. Modified: python/branches/cpy_merge/Lib/test/test_memoryio.py ============================================================================== --- python/branches/cpy_merge/Lib/test/test_memoryio.py (original) +++ python/branches/cpy_merge/Lib/test/test_memoryio.py Wed Jul 4 01:23:59 2007 @@ -38,11 +38,10 @@ memio = self.ioclass(buf) self.write_ops(memio) - - def test_write_noinit(self): memio = self.ioclass() - self.write_ops(memio) + memio.close() + self.assertRaises(ValueError, memio.write, buf) def test_writelines(self): buf = self.buftype("1234567890") @@ -50,6 +49,8 @@ memio.writelines([buf] * 100) self.assertEqual(memio.getvalue(), buf * 100) + memio.close() + self.assertRaises(ValueError, memio.writelines, buf) def test_writelines_error(self): memio = self.ioclass() @@ -72,21 +73,8 @@ memio.write(buf) self.assertEqual(memio.getvalue(), buf[:4] + buf) self.assertRaises(IOError, memio.truncate, -1) - - def test_close(self): - buf = self.buftype("1234567890") - memio = self.ioclass(buf) - - self.assertEqual(memio.closed, False) memio.close() - self.assertRaises(ValueError, memio.read) - self.assertRaises(ValueError, memio.write, buf) - self.assertRaises(ValueError, memio.writelines, buf) - self.assertRaises(ValueError, memio.seek, 3) self.assertRaises(ValueError, memio.truncate) - self.assertRaises(ValueError, memio.getvalue) - self.assertRaises(ValueError, memio.tell) - self.assertEqual(memio.closed, True) def test_init(self): buf = self.buftype("1234567890") @@ -102,14 +90,10 @@ self.assertEqual(memio.read(), self.EOF) memio.seek(0) self.assertEqual(memio.read(), buf) + self.assertEqual(memio.read(), self.EOF) self.assertEqual(memio.tell(), 10) - - def test_read_noargs(self): - buf = self.buftype("1234567890") - memio = self.ioclass(buf) - - self.assertEqual(buf, memio.read()) - self.assertEqual(self.EOF, memio.read()) + memio.close() + self.assertRaises(ValueError, memio.read) def test_readline(self): buf = self.buftype("1234567890\n") @@ -122,6 +106,8 @@ self.assertEqual(memio.readline(5), "12345") self.assertEqual(memio.readline(5), "67890") self.assertEqual(memio.readline(5), '\n') + memio.close() + self.assertRaises(ValueError, memio.readline) def test_readlines(self): buf = self.buftype("1234567890\n") @@ -132,6 +118,8 @@ self.assertEqual(memio.readlines(), ['67890\n'] + [buf] * 9) memio.seek(0) self.assertEqual(memio.readlines(15), [buf] * 2) + memio.close() + self.assertRaises(ValueError, memio.readlines) def test_iterator(self): buf = self.buftype("1234567890\n") @@ -145,13 +133,14 @@ self.assertEqual(line, buf) i += 1 self.assertEqual(i, 10) - memio.seek(0) i = 0 for line in memio: self.assertEqual(line, buf) i += 1 self.assertEqual(i, 10) + memio.close() + self.assertRaises(ValueError, memio.__next__) def test_getvalue(self): buf = self.buftype("1234567890") @@ -160,6 +149,8 @@ self.assertEqual(memio.getvalue(), buf) memio = self.ioclass(buf * 1000) self.assertEqual(memio.getvalue()[-3:], "890") + memio.close() + self.assertRaises(ValueError, memio.getvalue) def test_seek(self): buf = self.buftype("1234567890") @@ -171,6 +162,8 @@ memio.seek(3) self.assertEqual(buf[3:], memio.read()) + memio.close() + self.assertRaises(ValueError, memio.seek, 3) def test_tell(self): buf = self.buftype("1234567890") @@ -181,6 +174,8 @@ self.assertEqual(5, memio.tell()) memio.seek(10000) self.assertEqual(10000, memio.tell()) + memio.close() + self.assertRaises(ValueError, memio.tell) def test_flags(self): memio = self.ioclass() @@ -189,16 +184,16 @@ self.assertEqual(memio.readable(), True) self.assertEqual(memio.seekable(), True) self.assertEqual(memio.isatty(), False) + self.assertEqual(memio.closed, False) memio.close() self.assertEqual(memio.writable(), True) self.assertEqual(memio.readable(), True) self.assertEqual(memio.seekable(), True) + self.assertRaises(ValueError, memio.isatty) + self.assertEqual(memio.closed, True) class PyBytesIOTest(MemoryTestMixin, unittest.TestCase): - """ - Test the Python implementation of BytesIO. - """ buftype = bytes ioclass = io._BytesIO EOF = b"" @@ -214,42 +209,36 @@ self.assertEqual(b, b"67890") self.assertEqual(memio.readinto(b), 0) self.assertEqual(b, b"67890") - b = bytes("hello world") memio.seek(0) self.assertEqual(memio.readinto(b), 10) self.assertEqual(b, "1234567890d") + b = bytes() + memio.seek(0) + self.assertEqual(memio.readinto(b), 0) + self.assertEqual(b, b"") + memio.close() + self.assertRaises(ValueError, memio.readinto, b) class PyStringIOTest(MemoryTestMixin, unittest.TestCase): - """ - Test the Python implementation of StringIO. - """ buftype = unicode ioclass = io._StringIO EOF = "" if has_c_implementation: class CBytesIOTest(PyBytesIOTest): - """ - Test the C implementation of BytesIO if available. - """ ioclass = _bytes_io.BytesIO - class CStringIOTest(PyStringIOTest): - """ - Test the C implementation of StringIO if available. - """ ioclass = _string_io.StringIO def test_main(): + tests = [PyBytesIOTest, PyStringIOTest] if has_c_implementation: - test_support.run_unittest(PyBytesIOTest, PyStringIOTest, - CBytesIOTest, CStringIOTest) - else: - test_support.run_unittest(PyBytesIOTest, PyStringIOTest) + tests.extend([CBytesIOTest, CStringIOTest]) + test_support.run_unittest(*tests) if __name__ == '__main__': test_main() From python-checkins at python.org Wed Jul 4 04:03:10 2007 From: python-checkins at python.org (mark.hammond) Date: Wed, 4 Jul 2007 04:03:10 +0200 (CEST) Subject: [Python-checkins] r56170 - python/trunk/PCbuild8/build.bat Message-ID: <20070704020310.C9C6D1E4003@bag.python.org> Author: mark.hammond Date: Wed Jul 4 04:03:10 2007 New Revision: 56170 Modified: python/trunk/PCbuild8/build.bat Log: copy built files to the PCBuild directory, where tools like distutils or external build processes can find them. Modified: python/trunk/PCbuild8/build.bat ============================================================================== --- python/trunk/PCbuild8/build.bat (original) +++ python/trunk/PCbuild8/build.bat Wed Jul 4 04:03:10 2007 @@ -14,4 +14,15 @@ set cmd=devenv pcbuild.sln %build% "%conf%|%platf%" echo %cmd% -%cmd% \ No newline at end of file +%cmd% + +rem Copy whatever was built to the canonical 'PCBuild' directory. +rem This helps extensions which use distutils etc. +rem (Don't check if the build was successful - we expect a few failures +rem due to missing libs) +echo Copying built files to ..\PCBuild +if not exist %platf%%conf%\. (echo %platf%%conf% does not exist - nothing copied & goto xit) +if not exist ..\PCBuild\. (echo ..\PCBuild does not exist - nothing copied & goto xit) +xcopy /q/y %platf%%conf%\* ..\PCBuild\. + +:xit From python-checkins at python.org Wed Jul 4 14:26:46 2007 From: python-checkins at python.org (nick.coghlan) Date: Wed, 4 Jul 2007 14:26:46 +0200 (CEST) Subject: [Python-checkins] r56171 - peps/trunk/pep-0366.txt Message-ID: <20070704122646.E8C061E4018@bag.python.org> Author: nick.coghlan Date: Wed Jul 4 14:26:46 2007 New Revision: 56171 Modified: peps/trunk/pep-0366.txt Log: Add reference to c.l.p discussion of bundling scripts as part of a package Modified: peps/trunk/pep-0366.txt ============================================================================== --- peps/trunk/pep-0366.txt (original) +++ peps/trunk/pep-0366.txt Wed Jul 4 14:26:46 2007 @@ -18,7 +18,9 @@ the use of explicit relative imports from executable modules within packages. Such imports currently fail due to an awkward interaction between PEP 328 and PEP 338 - this behaviour is the subject of at -least one open SF bug report (#1510172)[1]. +least one open SF bug report (#1510172)[1], and has lead to at least +a few queries on comp.lang.python (such as Alan Isaac's question in +[2]). With the proposed mechanism, relative imports will work automatically if the module is executed using the ``-m`` switch. A small amount of @@ -173,6 +175,9 @@ .. [2] Guido's rejection of PEP 3122 (http://mail.python.org/pipermail/python-3000/2007-April/006793.html) +.. [3] c.l.p. question about modules and relative imports + (http://groups.google.com/group/comp.lang.python/browse_thread/thread/c44c769a72ca69fa/) + Copyright ========= From python-checkins at python.org Wed Jul 4 14:47:13 2007 From: python-checkins at python.org (nick.coghlan) Date: Wed, 4 Jul 2007 14:47:13 +0200 (CEST) Subject: [Python-checkins] r56172 - peps/trunk/pep-0366.txt Message-ID: <20070704124713.EDB551E4003@bag.python.org> Author: nick.coghlan Date: Wed Jul 4 14:47:13 2007 New Revision: 56172 Modified: peps/trunk/pep-0366.txt Log: Typo fixes, note limitation of direct execution boilerplate Modified: peps/trunk/pep-0366.txt ============================================================================== --- peps/trunk/pep-0366.txt (original) +++ peps/trunk/pep-0366.txt Wed Jul 4 14:47:13 2007 @@ -18,9 +18,9 @@ the use of explicit relative imports from executable modules within packages. Such imports currently fail due to an awkward interaction between PEP 328 and PEP 338 - this behaviour is the subject of at -least one open SF bug report (#1510172)[1], and has lead to at least -a few queries on comp.lang.python (such as Alan Isaac's question in -[2]). +least one open SF bug report (#1510172)[1], and has most likely +been a factor in at least a few queries on comp.lang.python (such +as Alan Isaac's question in [2]). With the proposed mechanism, relative imports will work automatically if the module is executed using the ``-m`` switch. A small amount of @@ -55,8 +55,8 @@ For the 2.5 release, the recommendation is to always use absolute imports in any module that is intended to be used as a main module. The ``-m`` switch already provides a benefit here, as it inserts the -current directory into ``sys.path``, instead of the directory contain the -main module. This means that it is possible to run a module from +current directory into ``sys.path``, instead of the directory containing +the main module. This means that it is possible to run a module from inside a package using ``-m`` so long as the current directory contains the top level directory for the package. Absolute imports will work correctly even if the package isn't installed anywhere else on @@ -135,10 +135,15 @@ if __name__ == "__main__" and not __package_name__: __package_name__ = "" -Note that this boilerplate has the same disadvantage as the use of -absolute imports of sibling modules - if the script is moved to a -different package or subpackage, the boilerplate will need to be -updated manually. +Note that this boilerplate is sufficient only if the top level package +is already accessible via sys.path. Additional code that manipulates +sys.path would be needed in order for direct execution to work +without the top level package already being on sys.path. + +This approach also has the same disadvantage as the use of absolute +imports of sibling modules - if the script is moved to a different +package or subpackage, the boilerplate will need to be updated +manually. With this feature in place, the test scripts in the package above would be able to change their import lines to something along the From python-checkins at python.org Wed Jul 4 19:13:04 2007 From: python-checkins at python.org (alexandre.vassalotti) Date: Wed, 4 Jul 2007 19:13:04 +0200 (CEST) Subject: [Python-checkins] r56173 - python/branches/cpy_merge/Modules/_bytes_iomodule.c python/branches/cpy_merge/Modules/_string_iomodule.c Message-ID: <20070704171304.306231E4012@bag.python.org> Author: alexandre.vassalotti Date: Wed Jul 4 19:13:03 2007 New Revision: 56173 Modified: python/branches/cpy_merge/Modules/_bytes_iomodule.c python/branches/cpy_merge/Modules/_string_iomodule.c Log: Made the object struct slightly clearer. Add assert check to readinto. Modified: python/branches/cpy_merge/Modules/_bytes_iomodule.c ============================================================================== --- python/branches/cpy_merge/Modules/_bytes_iomodule.c (original) +++ python/branches/cpy_merge/Modules/_bytes_iomodule.c Wed Jul 4 19:13:03 2007 @@ -6,12 +6,11 @@ typedef struct { PyObject_HEAD char *buf; - Py_ssize_t pos, string_size; + Py_ssize_t pos; + Py_ssize_t string_size; Py_ssize_t buf_size; } BytesIOObject; -static PyTypeObject BytesIO_Type; - static PyObject * err_closed(void) @@ -293,6 +292,8 @@ len = self->string_size - self->pos; memcpy(raw_buffer, self->buf + self->pos, len); + assert(self->pos + len < PY_SSIZE_T_MAX); + assert(len >= 0); self->pos += len; return PyInt_FromSsize_t(len); @@ -633,7 +634,6 @@ "_bytes_io.BytesIO", /*tp_name*/ sizeof(BytesIOObject), /*tp_basicsize*/ 0, /*tp_itemsize*/ - /* methods */ (destructor)BytesIO_dealloc, /*tp_dealloc*/ 0, /*tp_print*/ 0, /*tp_getattr*/ Modified: python/branches/cpy_merge/Modules/_string_iomodule.c ============================================================================== --- python/branches/cpy_merge/Modules/_string_iomodule.c (original) +++ python/branches/cpy_merge/Modules/_string_iomodule.c Wed Jul 4 19:13:03 2007 @@ -6,11 +6,11 @@ typedef struct { PyObject_HEAD Py_UNICODE *buf; - Py_ssize_t pos, string_size; + Py_ssize_t pos; + Py_ssize_t string_size; Py_ssize_t buf_size; } StringIOObject; -static PyTypeObject StringIO_Type; static PyObject * err_closed(void) @@ -609,7 +609,6 @@ "_string_io.StringIO", /*tp_name*/ sizeof(StringIOObject), /*tp_basicsize*/ 0, /*tp_itemsize*/ - /* methods */ (destructor)StringIO_dealloc, /*tp_dealloc*/ 0, /*tp_print*/ 0, /*tp_getattr*/ From python-checkins at python.org Wed Jul 4 21:12:03 2007 From: python-checkins at python.org (alexandre.vassalotti) Date: Wed, 4 Jul 2007 21:12:03 +0200 (CEST) Subject: [Python-checkins] r56174 - python/branches/cpy_merge/Modules/_bytes_iomodule.c python/branches/cpy_merge/Modules/_string_iomodule.c Message-ID: <20070704191203.E084C1E4004@bag.python.org> Author: alexandre.vassalotti Date: Wed Jul 4 21:12:03 2007 New Revision: 56174 Modified: python/branches/cpy_merge/Modules/_bytes_iomodule.c python/branches/cpy_merge/Modules/_string_iomodule.c Log: Remove useless asserts. Py_ssize_t is an unsigned type, so it can't be negative. Modified: python/branches/cpy_merge/Modules/_bytes_iomodule.c ============================================================================== --- python/branches/cpy_merge/Modules/_bytes_iomodule.c (original) +++ python/branches/cpy_merge/Modules/_bytes_iomodule.c Wed Jul 4 21:12:03 2007 @@ -47,7 +47,6 @@ *output = self->buf + self->pos; assert(self->pos + len < PY_SSIZE_T_MAX); - assert(len >= 0); self->pos += len; return len; @@ -293,7 +292,6 @@ memcpy(raw_buffer, self->buf + self->pos, len); assert(self->pos + len < PY_SSIZE_T_MAX); - assert(len >= 0); self->pos += len; return PyInt_FromSsize_t(len); Modified: python/branches/cpy_merge/Modules/_string_iomodule.c ============================================================================== --- python/branches/cpy_merge/Modules/_string_iomodule.c (original) +++ python/branches/cpy_merge/Modules/_string_iomodule.c Wed Jul 4 21:12:03 2007 @@ -47,7 +47,6 @@ *output = self->buf + self->pos; assert(self->pos + len < PY_SSIZE_T_MAX); - assert(len >= 0); self->pos += len; return len; From martin at v.loewis.de Wed Jul 4 21:27:50 2007 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 04 Jul 2007 21:27:50 +0200 Subject: [Python-checkins] r56174 - python/branches/cpy_merge/Modules/_bytes_iomodule.c python/branches/cpy_merge/Modules/_string_iomodule.c In-Reply-To: <20070704191203.E084C1E4004@bag.python.org> References: <20070704191203.E084C1E4004@bag.python.org> Message-ID: <468BF4B6.50607@v.loewis.de> > Py_ssize_t is an unsigned type, so it can't be negative. That's not true. *s*size_t is meant to indicate that it is a *signed* version of size_t. Regards, Martin From alexandre at peadrop.com Wed Jul 4 22:36:16 2007 From: alexandre at peadrop.com (Alexandre Vassalotti) Date: Wed, 4 Jul 2007 16:36:16 -0400 Subject: [Python-checkins] r56174 - python/branches/cpy_merge/Modules/_bytes_iomodule.c python/branches/cpy_merge/Modules/_string_iomodule.c In-Reply-To: <468BF4B6.50607@v.loewis.de> References: <20070704191203.E084C1E4004@bag.python.org> <468BF4B6.50607@v.loewis.de> Message-ID: On 7/4/07, "Martin v. L?wis" wrote: > > Py_ssize_t is an unsigned type, so it can't be negative. > > That's not true. *s*size_t is meant to indicate that it > is a *signed* version of size_t. Ah! I thought Py_ssize_t was just the Python analogue of size_t, which is unsigned. Thanks for the note! From martin at v.loewis.de Wed Jul 4 22:55:05 2007 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Wed, 04 Jul 2007 22:55:05 +0200 Subject: [Python-checkins] r56174 - python/branches/cpy_merge/Modules/_bytes_iomodule.c python/branches/cpy_merge/Modules/_string_iomodule.c In-Reply-To: References: <20070704191203.E084C1E4004@bag.python.org> <468BF4B6.50607@v.loewis.de> Message-ID: <468C0929.5080606@v.loewis.de> Alexandre Vassalotti schrieb: > On 7/4/07, "Martin v. L?wis" wrote: >> > Py_ssize_t is an unsigned type, so it can't be negative. >> >> That's not true. *s*size_t is meant to indicate that it >> is a *signed* version of size_t. > > Ah! I thought Py_ssize_t was just the Python analogue of size_t, which > is unsigned. Thanks for the note! Actually, it is the analogue of ssize_t, which doesn't exist on all platforms (hence the Py_ prefix). Regards, Martin From python-checkins at python.org Wed Jul 4 22:57:09 2007 From: python-checkins at python.org (alexandre.vassalotti) Date: Wed, 4 Jul 2007 22:57:09 +0200 (CEST) Subject: [Python-checkins] r56175 - python/branches/cpy_merge/Modules/_bytes_iomodule.c python/branches/cpy_merge/Modules/_string_iomodule.c Message-ID: <20070704205709.D40E51E4013@bag.python.org> Author: alexandre.vassalotti Date: Wed Jul 4 22:57:09 2007 New Revision: 56175 Modified: python/branches/cpy_merge/Modules/_bytes_iomodule.c python/branches/cpy_merge/Modules/_string_iomodule.c Log: Revert to r56173, as per recommendation of Martin v. L?wis. Modified: python/branches/cpy_merge/Modules/_bytes_iomodule.c ============================================================================== --- python/branches/cpy_merge/Modules/_bytes_iomodule.c (original) +++ python/branches/cpy_merge/Modules/_bytes_iomodule.c Wed Jul 4 22:57:09 2007 @@ -47,6 +47,7 @@ *output = self->buf + self->pos; assert(self->pos + len < PY_SSIZE_T_MAX); + assert(len >= 0); self->pos += len; return len; @@ -292,6 +293,7 @@ memcpy(raw_buffer, self->buf + self->pos, len); assert(self->pos + len < PY_SSIZE_T_MAX); + assert(len >= 0); self->pos += len; return PyInt_FromSsize_t(len); Modified: python/branches/cpy_merge/Modules/_string_iomodule.c ============================================================================== --- python/branches/cpy_merge/Modules/_string_iomodule.c (original) +++ python/branches/cpy_merge/Modules/_string_iomodule.c Wed Jul 4 22:57:09 2007 @@ -47,6 +47,7 @@ *output = self->buf + self->pos; assert(self->pos + len < PY_SSIZE_T_MAX); + assert(len >= 0); self->pos += len; return len; From alexandre at peadrop.com Wed Jul 4 23:02:21 2007 From: alexandre at peadrop.com (Alexandre Vassalotti) Date: Wed, 4 Jul 2007 17:02:21 -0400 Subject: [Python-checkins] r56174 - python/branches/cpy_merge/Modules/_bytes_iomodule.c python/branches/cpy_merge/Modules/_string_iomodule.c In-Reply-To: <468C0929.5080606@v.loewis.de> References: <20070704191203.E084C1E4004@bag.python.org> <468BF4B6.50607@v.loewis.de> <468C0929.5080606@v.loewis.de> Message-ID: On 7/4/07, "Martin v. L?wis" wrote: > Actually, it is the analogue of ssize_t, which doesn't exist on all > platforms (hence the Py_ prefix). Just read PEP-353 that explained me everything I wanted to know. Thanks again, -- Alexandre From nnorwitz at gmail.com Thu Jul 5 11:43:46 2007 From: nnorwitz at gmail.com (Neal Norwitz) Date: Thu, 5 Jul 2007 05:43:46 -0400 Subject: [Python-checkins] Python Regression Test Failures refleak (1) Message-ID: <20070705094346.GA1019@python.psfb.org> test_popen2 leaked [-26, 26, -26] references, sum=-26 test_sys leaked [-132, 0, 0] references, sum=-132 test_urllib2_localnet leaked [1, 5, 3] references, sum=9 From python-checkins at python.org Fri Jul 6 00:03:39 2007 From: python-checkins at python.org (kurt.kaiser) Date: Fri, 6 Jul 2007 00:03:39 +0200 (CEST) Subject: [Python-checkins] r56176 - python/trunk/Modules/_tkinter.c Message-ID: <20070705220339.A1AF41E4004@bag.python.org> Author: kurt.kaiser Date: Fri Jul 6 00:03:39 2007 New Revision: 56176 Modified: python/trunk/Modules/_tkinter.c Log: Many calls to tk.call involve an arglist containing a single tuple. Calls using METH_OLDARGS unpack this tuple; calls using METH_VARARG don't. Tcl's concatenation of args was affected; IDLE doesn't start. Modify Tkapp_Call() to unpack single tuple arglists. Bug 1733943 Ref http://mail.python.org/pipermail/python-checkins/2007-May/060454.html Modified: python/trunk/Modules/_tkinter.c ============================================================================== --- python/trunk/Modules/_tkinter.c (original) +++ python/trunk/Modules/_tkinter.c Fri Jul 6 00:03:39 2007 @@ -1285,6 +1285,12 @@ /* Could add TCL_EVAL_GLOBAL if wrapped by GlobalCall... */ int flags = TCL_EVAL_DIRECT; + /* If args is a single tuple, replace with contents of tuple */ + if (1 == PyTuple_Size(args)){ + PyObject* item = PyTuple_GetItem(args, 0); + if (PyTuple_Check(item)) + args = item; + } #ifdef WITH_THREAD if (self->threaded && self->thread_id != Tcl_GetCurrentThread()) { /* We cannot call the command directly. Instead, we must From buildbot at python.org Fri Jul 6 00:30:39 2007 From: buildbot at python.org (buildbot at python.org) Date: Thu, 05 Jul 2007 22:30:39 +0000 Subject: [Python-checkins] buildbot warnings in x86 mvlgcc trunk Message-ID: <20070705223040.04DD01E4004@bag.python.org> The Buildbot has detected a new failure of x86 mvlgcc trunk. Full details are available at: http://www.python.org/dev/buildbot/all/x86%2520mvlgcc%2520trunk/builds/635 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: kurt.kaiser Build had warnings: warnings test Excerpt from the test logfile: Traceback (most recent call last): File "/home2/buildbot/slave/trunk.loewis-linux/build/Lib/threading.py", line 465, in __bootstrap self.run() File "/home2/buildbot/slave/trunk.loewis-linux/build/Lib/test/test_urllib2_localnet.py", line 66, in run self._RequestHandlerClass) File "/home2/buildbot/slave/trunk.loewis-linux/build/Lib/test/test_urllib2_localnet.py", line 22, in __init__ RequestHandlerClass) File "/home2/buildbot/slave/trunk.loewis-linux/build/Lib/SocketServer.py", line 331, in __init__ self.server_bind() File "/home2/buildbot/slave/trunk.loewis-linux/build/Lib/BaseHTTPServer.py", line 101, in server_bind SocketServer.TCPServer.server_bind(self) File "/home2/buildbot/slave/trunk.loewis-linux/build/Lib/SocketServer.py", line 342, in server_bind self.socket.bind(self.server_address) File "", line 1, in bind error: (98, 'Address already in use') Traceback (most recent call last): File "/home2/buildbot/slave/trunk.loewis-linux/build/Lib/threading.py", line 465, in __bootstrap self.run() File "/home2/buildbot/slave/trunk.loewis-linux/build/Lib/test/test_urllib2_localnet.py", line 66, in run self._RequestHandlerClass) File "/home2/buildbot/slave/trunk.loewis-linux/build/Lib/test/test_urllib2_localnet.py", line 22, in __init__ RequestHandlerClass) File "/home2/buildbot/slave/trunk.loewis-linux/build/Lib/SocketServer.py", line 331, in __init__ self.server_bind() File "/home2/buildbot/slave/trunk.loewis-linux/build/Lib/BaseHTTPServer.py", line 101, in server_bind SocketServer.TCPServer.server_bind(self) File "/home2/buildbot/slave/trunk.loewis-linux/build/Lib/SocketServer.py", line 342, in server_bind self.socket.bind(self.server_address) File "", line 1, in bind error: (98, 'Address already in use') Traceback (most recent call last): File "/home2/buildbot/slave/trunk.loewis-linux/build/Lib/threading.py", line 465, in __bootstrap self.run() File "/home2/buildbot/slave/trunk.loewis-linux/build/Lib/test/test_urllib2_localnet.py", line 66, in run self._RequestHandlerClass) File "/home2/buildbot/slave/trunk.loewis-linux/build/Lib/test/test_urllib2_localnet.py", line 22, in __init__ RequestHandlerClass) File "/home2/buildbot/slave/trunk.loewis-linux/build/Lib/SocketServer.py", line 331, in __init__ self.server_bind() File "/home2/buildbot/slave/trunk.loewis-linux/build/Lib/BaseHTTPServer.py", line 101, in server_bind SocketServer.TCPServer.server_bind(self) File "/home2/buildbot/slave/trunk.loewis-linux/build/Lib/SocketServer.py", line 342, in server_bind self.socket.bind(self.server_address) File "", line 1, in bind error: (98, 'Address already in use') Traceback (most recent call last): File "/home2/buildbot/slave/trunk.loewis-linux/build/Lib/threading.py", line 465, in __bootstrap self.run() File "/home2/buildbot/slave/trunk.loewis-linux/build/Lib/test/test_urllib2_localnet.py", line 66, in run self._RequestHandlerClass) File "/home2/buildbot/slave/trunk.loewis-linux/build/Lib/test/test_urllib2_localnet.py", line 22, in __init__ RequestHandlerClass) File "/home2/buildbot/slave/trunk.loewis-linux/build/Lib/SocketServer.py", line 331, in __init__ self.server_bind() File "/home2/buildbot/slave/trunk.loewis-linux/build/Lib/BaseHTTPServer.py", line 101, in server_bind SocketServer.TCPServer.server_bind(self) File "/home2/buildbot/slave/trunk.loewis-linux/build/Lib/SocketServer.py", line 342, in server_bind self.socket.bind(self.server_address) File "", line 1, in bind error: (98, 'Address already in use') 1 test failed: test_urllib2_localnet Traceback (most recent call last): File "/home2/buildbot/slave/trunk.loewis-linux/build/Lib/threading.py", line 465, in __bootstrap self.run() File "/home2/buildbot/slave/trunk.loewis-linux/build/Lib/test/test_urllib2_localnet.py", line 66, in run self._RequestHandlerClass) File "/home2/buildbot/slave/trunk.loewis-linux/build/Lib/test/test_urllib2_localnet.py", line 22, in __init__ RequestHandlerClass) File "/home2/buildbot/slave/trunk.loewis-linux/build/Lib/SocketServer.py", line 331, in __init__ self.server_bind() File "/home2/buildbot/slave/trunk.loewis-linux/build/Lib/BaseHTTPServer.py", line 101, in server_bind SocketServer.TCPServer.server_bind(self) File "/home2/buildbot/slave/trunk.loewis-linux/build/Lib/SocketServer.py", line 342, in server_bind self.socket.bind(self.server_address) File "", line 1, in bind error: (98, 'Address already in use') Traceback (most recent call last): File "/home2/buildbot/slave/trunk.loewis-linux/build/Lib/threading.py", line 465, in __bootstrap self.run() File "/home2/buildbot/slave/trunk.loewis-linux/build/Lib/test/test_urllib2_localnet.py", line 66, in run self._RequestHandlerClass) File "/home2/buildbot/slave/trunk.loewis-linux/build/Lib/test/test_urllib2_localnet.py", line 22, in __init__ RequestHandlerClass) File "/home2/buildbot/slave/trunk.loewis-linux/build/Lib/SocketServer.py", line 331, in __init__ self.server_bind() File "/home2/buildbot/slave/trunk.loewis-linux/build/Lib/BaseHTTPServer.py", line 101, in server_bind SocketServer.TCPServer.server_bind(self) File "/home2/buildbot/slave/trunk.loewis-linux/build/Lib/SocketServer.py", line 342, in server_bind self.socket.bind(self.server_address) File "", line 1, in bind error: (98, 'Address already in use') Traceback (most recent call last): File "/home2/buildbot/slave/trunk.loewis-linux/build/Lib/threading.py", line 465, in __bootstrap self.run() File "/home2/buildbot/slave/trunk.loewis-linux/build/Lib/test/test_urllib2_localnet.py", line 66, in run self._RequestHandlerClass) File "/home2/buildbot/slave/trunk.loewis-linux/build/Lib/test/test_urllib2_localnet.py", line 22, in __init__ RequestHandlerClass) File "/home2/buildbot/slave/trunk.loewis-linux/build/Lib/SocketServer.py", line 331, in __init__ self.server_bind() File "/home2/buildbot/slave/trunk.loewis-linux/build/Lib/BaseHTTPServer.py", line 101, in server_bind SocketServer.TCPServer.server_bind(self) File "/home2/buildbot/slave/trunk.loewis-linux/build/Lib/SocketServer.py", line 342, in server_bind self.socket.bind(self.server_address) File "", line 1, in bind error: (98, 'Address already in use') ====================================================================== ERROR: test_proxy_qop_auth_int_works_or_throws_urlerror (test.test_urllib2_localnet.ProxyAuthTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home2/buildbot/slave/trunk.loewis-linux/build/Lib/test/test_urllib2_localnet.py", line 252, in setUp raise self.server.error error: (98, 'Address already in use') ====================================================================== ERROR: test_proxy_qop_auth_works (test.test_urllib2_localnet.ProxyAuthTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home2/buildbot/slave/trunk.loewis-linux/build/Lib/test/test_urllib2_localnet.py", line 252, in setUp raise self.server.error error: (98, 'Address already in use') ====================================================================== ERROR: test_proxy_with_bad_password_raises_httperror (test.test_urllib2_localnet.ProxyAuthTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home2/buildbot/slave/trunk.loewis-linux/build/Lib/test/test_urllib2_localnet.py", line 252, in setUp raise self.server.error error: (98, 'Address already in use') ====================================================================== ERROR: test_proxy_with_no_password_raises_httperror (test.test_urllib2_localnet.ProxyAuthTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home2/buildbot/slave/trunk.loewis-linux/build/Lib/test/test_urllib2_localnet.py", line 252, in setUp raise self.server.error error: (98, 'Address already in use') Traceback (most recent call last): File "/home2/buildbot/slave/trunk.loewis-linux/build/Lib/threading.py", line 465, in __bootstrap self.run() File "/home2/buildbot/slave/trunk.loewis-linux/build/Lib/test/test_urllib2_localnet.py", line 66, in run self._RequestHandlerClass) File "/home2/buildbot/slave/trunk.loewis-linux/build/Lib/test/test_urllib2_localnet.py", line 22, in __init__ RequestHandlerClass) File "/home2/buildbot/slave/trunk.loewis-linux/build/Lib/SocketServer.py", line 331, in __init__ self.server_bind() File "/home2/buildbot/slave/trunk.loewis-linux/build/Lib/BaseHTTPServer.py", line 101, in server_bind SocketServer.TCPServer.server_bind(self) File "/home2/buildbot/slave/trunk.loewis-linux/build/Lib/SocketServer.py", line 342, in server_bind self.socket.bind(self.server_address) File "", line 1, in bind error: (98, 'Address already in use') make: *** [buildbottest] Error 1 sincerely, -The Buildbot From buildbot at python.org Fri Jul 6 00:37:28 2007 From: buildbot at python.org (buildbot at python.org) Date: Thu, 05 Jul 2007 22:37:28 +0000 Subject: [Python-checkins] buildbot warnings in x86 XP trunk Message-ID: <20070705223728.DD64C1E4004@bag.python.org> The Buildbot has detected a new failure of x86 XP trunk. Full details are available at: http://www.python.org/dev/buildbot/all/x86%2520XP%2520trunk/builds/500 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: kurt.kaiser Build had warnings: warnings failed slave lost sincerely, -The Buildbot From buildbot at python.org Fri Jul 6 00:48:08 2007 From: buildbot at python.org (buildbot at python.org) Date: Thu, 05 Jul 2007 22:48:08 +0000 Subject: [Python-checkins] buildbot warnings in sparc solaris10 gcc trunk Message-ID: <20070705224808.ED5A61E4007@bag.python.org> The Buildbot has detected a new failure of sparc solaris10 gcc trunk. Full details are available at: http://www.python.org/dev/buildbot/all/sparc%2520solaris10%2520gcc%2520trunk/builds/2113 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: kurt.kaiser Build had warnings: warnings test Excerpt from the test logfile: Traceback (most recent call last): File "/opt/users/buildbot/slave/trunk.loewis-sun/build/Lib/threading.py", line 465, in __bootstrap self.run() File "/opt/users/buildbot/slave/trunk.loewis-sun/build/Lib/test/test_socketserver.py", line 93, in run svr.serve_a_few() File "/opt/users/buildbot/slave/trunk.loewis-sun/build/Lib/test/test_socketserver.py", line 35, in serve_a_few self.handle_request() File "/opt/users/buildbot/slave/trunk.loewis-sun/build/Lib/SocketServer.py", line 224, in handle_request self.handle_error(request, client_address) File "/opt/users/buildbot/slave/trunk.loewis-sun/build/Lib/SocketServer.py", line 222, in handle_request self.process_request(request, client_address) File "/opt/users/buildbot/slave/trunk.loewis-sun/build/Lib/SocketServer.py", line 429, in process_request self.collect_children() File "/opt/users/buildbot/slave/trunk.loewis-sun/build/Lib/SocketServer.py", line 425, in collect_children self.active_children.remove(pid) ValueError: list.remove(x): x not in list 1 test failed: test_socketserver sincerely, -The Buildbot From buildbot at python.org Fri Jul 6 01:55:47 2007 From: buildbot at python.org (buildbot at python.org) Date: Thu, 05 Jul 2007 23:55:47 +0000 Subject: [Python-checkins] buildbot warnings in alpha Debian trunk Message-ID: <20070705235547.CCA4D1E4004@bag.python.org> The Buildbot has detected a new failure of alpha Debian trunk. Full details are available at: http://www.python.org/dev/buildbot/all/alpha%2520Debian%2520trunk/builds/33 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: kurt.kaiser Build had warnings: warnings test Excerpt from the test logfile: sincerely, -The Buildbot From python-checkins at python.org Fri Jul 6 06:13:39 2007 From: python-checkins at python.org (neal.norwitz) Date: Fri, 6 Jul 2007 06:13:39 +0200 (CEST) Subject: [Python-checkins] r56177 - python/trunk/Lib/sre_compile.py Message-ID: <20070706041339.CD7131E4004@bag.python.org> Author: neal.norwitz Date: Fri Jul 6 06:13:39 2007 New Revision: 56177 Modified: python/trunk/Lib/sre_compile.py Log: Fix typo in comment Modified: python/trunk/Lib/sre_compile.py ============================================================================== --- python/trunk/Lib/sre_compile.py (original) +++ python/trunk/Lib/sre_compile.py Fri Jul 6 06:13:39 2007 @@ -280,7 +280,7 @@ # To represent a big charset, first a bitmap of all characters in the # set is constructed. Then, this bitmap is sliced into chunks of 256 -# characters, duplicate chunks are eliminitated, and each chunk is +# characters, duplicate chunks are eliminated, and each chunk is # given a number. In the compiled expression, the charset is # represented by a 16-bit word sequence, consisting of one word for # the number of different chunks, a sequence of 256 bytes (128 words) From buildbot at python.org Fri Jul 6 07:02:35 2007 From: buildbot at python.org (buildbot at python.org) Date: Fri, 06 Jul 2007 05:02:35 +0000 Subject: [Python-checkins] buildbot warnings in g4 osx.4 trunk Message-ID: <20070706050235.E0B0A1E4004@bag.python.org> The Buildbot has detected a new failure of g4 osx.4 trunk. Full details are available at: http://www.python.org/dev/buildbot/all/g4%2520osx.4%2520trunk/builds/2123 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: neal.norwitz Build had warnings: warnings test Excerpt from the test logfile: 1 test failed: test_urllibnet make: *** [buildbottest] Error 1 sincerely, -The Buildbot From python-checkins at python.org Fri Jul 6 15:22:40 2007 From: python-checkins at python.org (collin.winter) Date: Fri, 6 Jul 2007 15:22:40 +0200 (CEST) Subject: [Python-checkins] r56178 - sandbox/trunk/2to3/fixes/fix_filter.py sandbox/trunk/2to3/fixes/fix_map.py Message-ID: <20070706132240.85F361E4007@bag.python.org> Author: collin.winter Date: Fri Jul 6 15:22:40 2007 New Revision: 56178 Modified: sandbox/trunk/2to3/fixes/fix_filter.py sandbox/trunk/2to3/fixes/fix_map.py Log: Whitespace fixes. Modified: sandbox/trunk/2to3/fixes/fix_filter.py ============================================================================== --- sandbox/trunk/2to3/fixes/fix_filter.py (original) +++ sandbox/trunk/2to3/fixes/fix_filter.py Fri Jul 6 15:22:40 2007 @@ -103,5 +103,5 @@ results = {} if self.p2.match(ppp, results) and results["node"] is node: return True - + return False Modified: sandbox/trunk/2to3/fixes/fix_map.py ============================================================================== --- sandbox/trunk/2to3/fixes/fix_map.py (original) +++ sandbox/trunk/2to3/fixes/fix_map.py Fri Jul 6 15:22:40 2007 @@ -115,5 +115,5 @@ results = {} if self.p2.match(ppp, results) and results["node"] is node: return True - + return False From python-checkins at python.org Fri Jul 6 15:30:54 2007 From: python-checkins at python.org (collin.winter) Date: Fri, 6 Jul 2007 15:30:54 +0200 (CEST) Subject: [Python-checkins] r56179 - in sandbox/trunk/2to3: README fixes/fix_sysexcattrs.py fixes/fix_sysexcinfo.py tests/test_fixers.py Message-ID: <20070706133054.02BCE1E400E@bag.python.org> Author: collin.winter Date: Fri Jul 6 15:30:53 2007 New Revision: 56179 Added: sandbox/trunk/2to3/fixes/fix_sysexcattrs.py - copied, changed from r56177, sandbox/trunk/2to3/fixes/fix_sysexcinfo.py Removed: sandbox/trunk/2to3/fixes/fix_sysexcinfo.py Modified: sandbox/trunk/2to3/README sandbox/trunk/2to3/tests/test_fixers.py Log: Rename the sysexcinfo fixer to fix_sysexcattrs to reflect that it no longer warns on sys.exc_info() (per PEP 3100). Modified: sandbox/trunk/2to3/README ============================================================================== --- sandbox/trunk/2to3/README (original) +++ sandbox/trunk/2to3/README Fri Jul 6 15:30:53 2007 @@ -64,7 +64,7 @@ * **fix_repr** - swap backticks for repr() calls. -* **fix_sysexcinfo** - warn on usage of sys.value, sys.type and +* **fix_sysexcattrs** - warn on usage of sys.value, sys.type and sys.traceback. * **fix_throw** - fix generator.throw() calls to be 3.0-compliant (PEP 3109). Copied: sandbox/trunk/2to3/fixes/fix_sysexcattrs.py (from r56177, sandbox/trunk/2to3/fixes/fix_sysexcinfo.py) ============================================================================== --- sandbox/trunk/2to3/fixes/fix_sysexcinfo.py (original) +++ sandbox/trunk/2to3/fixes/fix_sysexcattrs.py Fri Jul 6 15:30:53 2007 @@ -1,4 +1,4 @@ -"""Fixer/warner for sys.exc_{info,value,type,traceback}""" +"""Fixer/warner for sys.exc_{value,type,traceback}""" # Author: Collin Winter # Local imports @@ -6,24 +6,13 @@ from fixes import basefix -class FixSysexcinfo(basefix.BaseFix): +class FixSysexcattrs(basefix.BaseFix): PATTERN = """ - power< 'sys' trailer< '.' attr='exc_info'> any* > - | power< 'sys' - trailer< '.' attr=('exc_value' | 'exc_traceback' | 'exc_type')> + trailer< '.' ('exc_value' | 'exc_traceback' | 'exc_type')> any* > """ def transform(self, node): - results = self.match(node) - assert results - attr = results['attr'] - - if isinstance(attr, Leaf) and attr.value == 'exc_info': - self.cannot_convert(node, - "This function is going away in Python 3") - else: - self.cannot_convert(node, - "This attribute is going away in Python 3") + self.cannot_convert(node, "This attribute is going away in Python 3") Deleted: /sandbox/trunk/2to3/fixes/fix_sysexcinfo.py ============================================================================== --- /sandbox/trunk/2to3/fixes/fix_sysexcinfo.py Fri Jul 6 15:30:53 2007 +++ (empty file) @@ -1,29 +0,0 @@ -"""Fixer/warner for sys.exc_{info,value,type,traceback}""" -# Author: Collin Winter - -# Local imports -from pytree import Leaf -from fixes import basefix - - -class FixSysexcinfo(basefix.BaseFix): - - PATTERN = """ - power< 'sys' trailer< '.' attr='exc_info'> any* > - | - power< 'sys' - trailer< '.' attr=('exc_value' | 'exc_traceback' | 'exc_type')> - any* > - """ - - def transform(self, node): - results = self.match(node) - assert results - attr = results['attr'] - - if isinstance(attr, Leaf) and attr.value == 'exc_info': - self.cannot_convert(node, - "This function is going away in Python 3") - else: - self.cannot_convert(node, - "This attribute is going away in Python 3") Modified: sandbox/trunk/2to3/tests/test_fixers.py ============================================================================== --- sandbox/trunk/2to3/tests/test_fixers.py (original) +++ sandbox/trunk/2to3/tests/test_fixers.py Fri Jul 6 15:30:53 2007 @@ -877,25 +877,19 @@ self.check(b, a) -class Test_sysexcinfo(FixerTestCase): - fixer = "sysexcinfo" +class Test_sysexcattrs(FixerTestCase): + fixer = "sysexcattrs" def test_1(self): - s = """sys.exc_info()""" - self.warns(s, s, "This function is going away") + s = """f = sys.exc_type""" + self.warns(s, s, "This attribute is going away") def test_2(self): - s = """if sys.exc_info()[1] == 1: - pass""" - - self.warns(s, s, "This function is going away") + s = """f = sys.exc_value""" + self.warns(s, s, "This attribute is going away") def test_3(self): - s = """f = sys.exc_info""" - self.warns(s, s, "This function is going away") - - def test_4(self): - s = """f = sys.exc_type + ":" + sys.exc_value""" + s = """f = sys.exc_traceback""" self.warns(s, s, "This attribute is going away") From python-checkins at python.org Fri Jul 6 15:42:13 2007 From: python-checkins at python.org (collin.winter) Date: Fri, 6 Jul 2007 15:42:13 +0200 (CEST) Subject: [Python-checkins] r56180 - in sandbox/trunk/2to3: README examples examples/fix_ws_comma.py fixes/fix_ws_comma.py Message-ID: <20070706134213.714C11E4007@bag.python.org> Author: collin.winter Date: Fri Jul 6 15:42:13 2007 New Revision: 56180 Added: sandbox/trunk/2to3/examples/ sandbox/trunk/2to3/examples/fix_ws_comma.py - copied unchanged from r56177, sandbox/trunk/2to3/fixes/fix_ws_comma.py Removed: sandbox/trunk/2to3/fixes/fix_ws_comma.py Modified: sandbox/trunk/2to3/README Log: Move fix_ws_comma out of fixes (we don't want it run by -f all). Modified: sandbox/trunk/2to3/README ============================================================================== --- sandbox/trunk/2to3/README (original) +++ sandbox/trunk/2to3/README Fri Jul 6 15:42:13 2007 @@ -24,6 +24,7 @@ pgen2/ - Parser generator and driver ([1]_, [2]_) fixes/ - Individual transformations tests/ - Test files for pytree, fixers, grammar, etc +examples/ - Fixers that while neat, we don't want run by refactor's -f all Capabilities Deleted: /sandbox/trunk/2to3/fixes/fix_ws_comma.py ============================================================================== --- /sandbox/trunk/2to3/fixes/fix_ws_comma.py Fri Jul 6 15:42:13 2007 +++ (empty file) @@ -1,37 +0,0 @@ -"""Fixer that changes 'a ,b' into 'a, b'. - -This also changes '{a :b}' into '{a: b}', but does not touch other -uses of colons. It does not touch other uses of whitespace. - -""" - -import pytree -from pgen2 import token -from fixes import basefix - -class FixWsComma(basefix.BaseFix): - - PATTERN = """ - any<(not(',') any)+ ',' ((not(',') any)+ ',')* [not(',') any]> - """ - - COMMA = pytree.Leaf(token.COMMA, ",") - COLON = pytree.Leaf(token.COLON, ":") - SEPS = (COMMA, COLON) - - def transform(self, node): - new = node.clone() - comma = False - for child in new.children: - if child in self.SEPS: - prefix = child.get_prefix() - if prefix.isspace() and "\n" not in prefix: - child.set_prefix("") - comma = True - else: - if comma: - prefix = child.get_prefix() - if not prefix: - child.set_prefix(" ") - comma = False - return new From python-checkins at python.org Sat Jul 7 19:03:03 2007 From: python-checkins at python.org (alexandre.vassalotti) Date: Sat, 7 Jul 2007 19:03:03 +0200 (CEST) Subject: [Python-checkins] r56181 - python/branches/cpy_merge/Modules/_picklemodule.c Message-ID: <20070707170303.A9B391E4007@bag.python.org> Author: alexandre.vassalotti Date: Sat Jul 7 19:03:03 2007 New Revision: 56181 Added: python/branches/cpy_merge/Modules/_picklemodule.c - copied unchanged from r56174, python/branches/cpy_merge/Modules/cPickle.c Log: Copy cPickle.c to _picklemodule.c From python-checkins at python.org Sat Jul 7 19:47:04 2007 From: python-checkins at python.org (alexandre.vassalotti) Date: Sat, 7 Jul 2007 19:47:04 +0200 (CEST) Subject: [Python-checkins] r56182 - python/branches/cpy_merge/Modules/_picklemodule.c Message-ID: <20070707174704.C4C7C1E4002@bag.python.org> Author: alexandre.vassalotti Date: Sat Jul 7 19:47:03 2007 New Revision: 56182 Modified: python/branches/cpy_merge/Modules/_picklemodule.c Log: Quick mecanical clean up with: indent -i4 -l79 -bap -nbad -ncdb \ -sc -br -nce -cdw -cli0 \ -nss -npcs -saf -sai -saw \ -nbc -di1 -brs -psl -lp \ -nlps -nbbo Modified: python/branches/cpy_merge/Modules/_picklemodule.c ============================================================================== --- python/branches/cpy_merge/Modules/_picklemodule.c (original) +++ python/branches/cpy_merge/Modules/_picklemodule.c Sat Jul 7 19:47:03 2007 @@ -3,7 +3,7 @@ #include "structmember.h" PyDoc_STRVAR(cPickle_module_documentation, -"C implementation and optimization of the Python pickle module."); + "C implementation and optimization of the Python pickle module."); #ifndef Py_eval_input #include @@ -64,18 +64,18 @@ #define SETITEMS 'u' /* Protocol 2. */ -#define PROTO '\x80' /* identify pickle protocol */ -#define NEWOBJ '\x81' /* build object by applying cls.__new__ to argtuple */ -#define EXT1 '\x82' /* push object from extension registry; 1-byte index */ -#define EXT2 '\x83' /* ditto, but 2-byte index */ -#define EXT4 '\x84' /* ditto, but 4-byte index */ -#define TUPLE1 '\x85' /* build 1-tuple from stack top */ -#define TUPLE2 '\x86' /* build 2-tuple from two topmost stack items */ -#define TUPLE3 '\x87' /* build 3-tuple from three topmost stack items */ -#define NEWTRUE '\x88' /* push True */ -#define NEWFALSE '\x89' /* push False */ -#define LONG1 '\x8a' /* push long from < 256 bytes */ -#define LONG4 '\x8b' /* push really big long */ +#define PROTO '\x80' /* identify pickle protocol */ +#define NEWOBJ '\x81' /* build object by applying cls.__new__ to argtuple */ +#define EXT1 '\x82' /* push object from extension registry; 1-byte index */ +#define EXT2 '\x83' /* ditto, but 2-byte index */ +#define EXT4 '\x84' /* ditto, but 4-byte index */ +#define TUPLE1 '\x85' /* build 1-tuple from stack top */ +#define TUPLE2 '\x86' /* build 2-tuple from two topmost stack items */ +#define TUPLE3 '\x87' /* build 3-tuple from three topmost stack items */ +#define NEWTRUE '\x88' /* push True */ +#define NEWFALSE '\x89' /* push False */ +#define LONG1 '\x8a' /* push long from < 256 bytes */ +#define LONG4 '\x8b' /* push really big long */ /* There aren't opcodes -- they're ways to pickle bools before protocol 2, * so that unpicklers written before bools were introduced unpickle them @@ -120,40 +120,39 @@ static PyObject *two_tuple; static PyObject *__class___str, *__getinitargs___str, *__dict___str, - *__getstate___str, *__setstate___str, *__name___str, *__reduce___str, - *__reduce_ex___str, - *write_str, *append_str, - *read_str, *readline_str, *__main___str, - *copy_reg_str, *dispatch_table_str; + *__getstate___str, *__setstate___str, *__name___str, *__reduce___str, + *__reduce_ex___str, + *write_str, *append_str, + *read_str, *readline_str, *__main___str, + *copy_reg_str, *dispatch_table_str; /************************************************************************* Internal Data type for pickle data. */ typedef struct { - PyObject_HEAD - int length; /* number of initial slots in data currently used */ - int size; /* number of slots in data allocated */ - PyObject **data; + PyObject_HEAD int length; /* number of initial slots in data currently used */ + int size; /* number of slots in data allocated */ + PyObject **data; } Pdata; static void -Pdata_dealloc(Pdata *self) +Pdata_dealloc(Pdata * self) { - int i; - PyObject **p; + int i; + PyObject **p; - for (i = self->length, p = self->data; --i >= 0; p++) { - Py_DECREF(*p); - } - if (self->data) - free(self->data); - PyObject_Del(self); + for (i = self->length, p = self->data; --i >= 0; p++) { + Py_DECREF(*p); + } + if (self->data) + free(self->data); + PyObject_Del(self); } static PyTypeObject PdataType = { - PyObject_HEAD_INIT(NULL) 0, "cPickle.Pdata", sizeof(Pdata), 0, - (destructor)Pdata_dealloc, - 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0L,0L,0L,0L, "" + PyObject_HEAD_INIT(NULL) 0, "cPickle.Pdata", sizeof(Pdata), 0, + (destructor) Pdata_dealloc, + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0L, 0L, 0L, 0L, "" }; #define Pdata_Check(O) ((O)->ob_type == &PdataType) @@ -161,73 +160,73 @@ static PyObject * Pdata_New(void) { - Pdata *self; + Pdata *self; - if (!(self = PyObject_New(Pdata, &PdataType))) - return NULL; - self->size = 8; - self->length = 0; - self->data = malloc(self->size * sizeof(PyObject*)); - if (self->data) - return (PyObject*)self; - Py_DECREF(self); - return PyErr_NoMemory(); + if (!(self = PyObject_New(Pdata, &PdataType))) + return NULL; + self->size = 8; + self->length = 0; + self->data = malloc(self->size * sizeof(PyObject *)); + if (self->data) + return (PyObject *) self; + Py_DECREF(self); + return PyErr_NoMemory(); } static int stackUnderflow(void) { - PyErr_SetString(UnpicklingError, "unpickling stack underflow"); - return -1; + PyErr_SetString(UnpicklingError, "unpickling stack underflow"); + return -1; } /* Retain only the initial clearto items. If clearto >= the current * number of items, this is a (non-erroneous) NOP. */ static int -Pdata_clear(Pdata *self, int clearto) +Pdata_clear(Pdata * self, int clearto) { - int i; - PyObject **p; + int i; + PyObject **p; - if (clearto < 0) return stackUnderflow(); - if (clearto >= self->length) return 0; + if (clearto < 0) + return stackUnderflow(); + if (clearto >= self->length) + return 0; - for (i = self->length, p = self->data + clearto; - --i >= clearto; - p++) { - Py_CLEAR(*p); - } - self->length = clearto; + for (i = self->length, p = self->data + clearto; --i >= clearto; p++) { + Py_CLEAR(*p); + } + self->length = clearto; - return 0; + return 0; } static int -Pdata_grow(Pdata *self) +Pdata_grow(Pdata * self) { - int bigger; - size_t nbytes; - PyObject **tmp; + int bigger; + size_t nbytes; + PyObject **tmp; - bigger = self->size << 1; - if (bigger <= 0) /* was 0, or new value overflows */ - goto nomemory; - if ((int)(size_t)bigger != bigger) - goto nomemory; - nbytes = (size_t)bigger * sizeof(PyObject *); - if (nbytes / sizeof(PyObject *) != (size_t)bigger) - goto nomemory; - tmp = realloc(self->data, nbytes); - if (tmp == NULL) - goto nomemory; - self->data = tmp; - self->size = bigger; - return 0; + bigger = self->size << 1; + if (bigger <= 0) /* was 0, or new value overflows */ + goto nomemory; + if ((int) (size_t) bigger != bigger) + goto nomemory; + nbytes = (size_t) bigger *sizeof(PyObject *); + if (nbytes / sizeof(PyObject *) != (size_t) bigger) + goto nomemory; + tmp = realloc(self->data, nbytes); + if (tmp == NULL) + goto nomemory; + self->data = tmp; + self->size = bigger; + return 0; nomemory: - PyErr_NoMemory(); - return -1; + PyErr_NoMemory(); + return -1; } /* D is a Pdata*. Pop the topmost element and store it into V, which @@ -272,35 +271,36 @@ static PyObject * -Pdata_popTuple(Pdata *self, int start) +Pdata_popTuple(Pdata * self, int start) { - PyObject *r; - int i, j, l; + PyObject *r; + int i, j, l; - l = self->length-start; - r = PyTuple_New(l); - if (r == NULL) - return NULL; - for (i = start, j = 0 ; j < l; i++, j++) - PyTuple_SET_ITEM(r, j, self->data[i]); + l = self->length - start; + r = PyTuple_New(l); + if (r == NULL) + return NULL; + for (i = start, j = 0; j < l; i++, j++) + PyTuple_SET_ITEM(r, j, self->data[i]); - self->length = start; - return r; + self->length = start; + return r; } static PyObject * -Pdata_popList(Pdata *self, int start) +Pdata_popList(Pdata * self, int start) { - PyObject *r; - int i, j, l; + PyObject *r; + int i, j, l; - l=self->length-start; - if (!( r=PyList_New(l))) return NULL; - for (i=start, j=0 ; j < l; i++, j++) - PyList_SET_ITEM(r, j, self->data[i]); + l = self->length - start; + if (!(r = PyList_New(l))) + return NULL; + for (i = start, j = 0; j < l; i++, j++) + PyList_SET_ITEM(r, j, self->data[i]); - self->length=start; - return r; + self->length = start; + return r; } /*************************************************************************/ @@ -323,29 +323,28 @@ } typedef struct Picklerobject { - PyObject_HEAD - FILE *fp; - PyObject *write; - PyObject *file; - PyObject *memo; - PyObject *arg; - PyObject *pers_func; - PyObject *inst_pers_func; - - /* pickle protocol number, >= 0 */ - int proto; - - /* bool, true if proto > 0 */ - int bin; - - int fast; /* Fast mode doesn't save in memo, don't use if circ ref */ - int nesting; - int (*write_func)(struct Picklerobject *, const char *, Py_ssize_t); - char *write_buf; - int buf_size; - PyObject *dispatch_table; - int fast_container; /* count nested container dumps */ - PyObject *fast_memo; + PyObject_HEAD FILE * fp; + PyObject *write; + PyObject *file; + PyObject *memo; + PyObject *arg; + PyObject *pers_func; + PyObject *inst_pers_func; + + /* pickle protocol number, >= 0 */ + int proto; + + /* bool, true if proto > 0 */ + int bin; + + int fast; /* Fast mode doesn't save in memo, don't use if circ ref */ + int nesting; + int (*write_func) (struct Picklerobject *, const char *, Py_ssize_t); + char *write_buf; + int buf_size; + PyObject *dispatch_table; + int fast_container; /* count nested container dumps */ + PyObject *fast_memo; } Picklerobject; #ifndef PY_CPICKLE_FAST_LIMIT @@ -355,25 +354,24 @@ static PyTypeObject Picklertype; typedef struct Unpicklerobject { - PyObject_HEAD - FILE *fp; - PyObject *file; - PyObject *readline; - PyObject *read; - PyObject *memo; - PyObject *arg; - Pdata *stack; - PyObject *mark; - PyObject *pers_func; - PyObject *last_string; - int *marks; - int num_marks; - int marks_size; - Py_ssize_t (*read_func)(struct Unpicklerobject *, char **, Py_ssize_t); - Py_ssize_t (*readline_func)(struct Unpicklerobject *, char **); - int buf_size; - char *buf; - PyObject *find_class; + PyObject_HEAD FILE * fp; + PyObject *file; + PyObject *readline; + PyObject *read; + PyObject *memo; + PyObject *arg; + Pdata *stack; + PyObject *mark; + PyObject *pers_func; + PyObject *last_string; + int *marks; + int num_marks; + int marks_size; + Py_ssize_t(*read_func) (struct Unpicklerobject *, char **, Py_ssize_t); + Py_ssize_t(*readline_func) (struct Unpicklerobject *, char **); + int buf_size; + char *buf; + PyObject *find_class; } Unpicklerobject; static PyTypeObject Unpicklertype; @@ -384,298 +382,304 @@ static PyObject * -cPickle_ErrFormat(PyObject *ErrType, char *stringformat, char *format, ...) +cPickle_ErrFormat(PyObject * ErrType, char *stringformat, char *format, ...) { - va_list va; - PyObject *args=0, *retval=0; - va_start(va, format); - - if (format) args = Py_VaBuildValue(format, va); - va_end(va); - if (format && ! args) return NULL; - if (stringformat && !(retval=PyString_FromString(stringformat))) - return NULL; + va_list va; + PyObject *args = 0, *retval = 0; + va_start(va, format); + + if (format) + args = Py_VaBuildValue(format, va); + va_end(va); + if (format && !args) + return NULL; + if (stringformat && !(retval = PyString_FromString(stringformat))) + return NULL; - if (retval) { - if (args) { - PyObject *v; - v=PyString_Format(retval, args); - Py_DECREF(retval); - Py_DECREF(args); - if (! v) return NULL; - retval=v; - } + if (retval) { + if (args) { + PyObject *v; + v = PyString_Format(retval, args); + Py_DECREF(retval); + Py_DECREF(args); + if (!v) + return NULL; + retval = v; } - else - if (args) retval=args; - else { - PyErr_SetObject(ErrType,Py_None); - return NULL; - } - PyErr_SetObject(ErrType,retval); - Py_DECREF(retval); + } + else if (args) + retval = args; + else { + PyErr_SetObject(ErrType, Py_None); return NULL; + } + PyErr_SetObject(ErrType, retval); + Py_DECREF(retval); + return NULL; } static int -write_file(Picklerobject *self, const char *s, Py_ssize_t n) +write_file(Picklerobject * self, const char *s, Py_ssize_t n) { - size_t nbyteswritten; + size_t nbyteswritten; - if (s == NULL) { - return 0; - } + if (s == NULL) { + return 0; + } - if (n > INT_MAX) { - /* String too large */ - return -1; - } + if (n > INT_MAX) { + /* String too large */ + return -1; + } - Py_BEGIN_ALLOW_THREADS + Py_BEGIN_ALLOW_THREADS nbyteswritten = fwrite(s, sizeof(char), n, self->fp); - Py_END_ALLOW_THREADS - if (nbyteswritten != (size_t)n) { - PyErr_SetFromErrno(PyExc_IOError); - return -1; - } + Py_END_ALLOW_THREADS if (nbyteswritten != (size_t) n) { + PyErr_SetFromErrno(PyExc_IOError); + return -1; + } - return (int)n; + return (int) n; } static int -write_cStringIO(Picklerobject *self, const char *s, Py_ssize_t n) +write_cStringIO(Picklerobject * self, const char *s, Py_ssize_t n) { - if (s == NULL) { - return 0; - } + if (s == NULL) { + return 0; + } - if (PycStringIO->cwrite((PyObject *)self->file, s, n) != n) { - return -1; - } + if (PycStringIO->cwrite((PyObject *) self->file, s, n) != n) { + return -1; + } - return (int)n; + return (int) n; } static int -write_none(Picklerobject *self, const char *s, Py_ssize_t n) +write_none(Picklerobject * self, const char *s, Py_ssize_t n) { - if (s == NULL) return 0; - if (n > INT_MAX) return -1; - return (int)n; + if (s == NULL) + return 0; + if (n > INT_MAX) + return -1; + return (int) n; } static int -write_other(Picklerobject *self, const char *s, Py_ssize_t _n) +write_other(Picklerobject * self, const char *s, Py_ssize_t _n) { - PyObject *py_str = 0, *junk = 0; - int n; + PyObject *py_str = 0, *junk = 0; + int n; - if (_n > INT_MAX) + if (_n > INT_MAX) + return -1; + n = (int) _n; + if (s == NULL) { + if (!(self->buf_size)) + return 0; + py_str = PyString_FromStringAndSize(self->write_buf, self->buf_size); + if (!py_str) + return -1; + } + else { + if (self->buf_size && (n + self->buf_size) > WRITE_BUF_SIZE) { + if (write_other(self, NULL, 0) < 0) return -1; - n = (int)_n; - if (s == NULL) { - if (!( self->buf_size )) return 0; - py_str = PyString_FromStringAndSize(self->write_buf, - self->buf_size); - if (!py_str) - return -1; } - else { - if (self->buf_size && (n + self->buf_size) > WRITE_BUF_SIZE) { - if (write_other(self, NULL, 0) < 0) - return -1; - } - if (n > WRITE_BUF_SIZE) { - if (!( py_str = - PyString_FromStringAndSize(s, n))) - return -1; - } - else { - memcpy(self->write_buf + self->buf_size, s, n); - self->buf_size += n; - return n; - } + if (n > WRITE_BUF_SIZE) { + if (!(py_str = PyString_FromStringAndSize(s, n))) + return -1; } - - if (self->write) { - /* object with write method */ - ARG_TUP(self, py_str); - if (self->arg) { - junk = PyObject_Call(self->write, self->arg, NULL); - FREE_ARG_TUP(self); - } - if (junk) Py_DECREF(junk); - else return -1; + else { + memcpy(self->write_buf + self->buf_size, s, n); + self->buf_size += n; + return n; + } + } + + if (self->write) { + /* object with write method */ + ARG_TUP(self, py_str); + if (self->arg) { + junk = PyObject_Call(self->write, self->arg, NULL); + FREE_ARG_TUP(self); } + if (junk) + Py_DECREF(junk); else - PDATA_PUSH(self->file, py_str, -1); + return -1; + } + else + PDATA_PUSH(self->file, py_str, -1); - self->buf_size = 0; - return n; + self->buf_size = 0; + return n; } static Py_ssize_t -read_file(Unpicklerobject *self, char **s, Py_ssize_t n) +read_file(Unpicklerobject * self, char **s, Py_ssize_t n) { - size_t nbytesread; + size_t nbytesread; - if (self->buf_size == 0) { - int size; - - size = ((n < 32) ? 32 : n); - if (!( self->buf = (char *)malloc(size))) { - PyErr_NoMemory(); - return -1; - } + if (self->buf_size == 0) { + int size; - self->buf_size = size; + size = ((n < 32) ? 32 : n); + if (!(self->buf = (char *) malloc(size))) { + PyErr_NoMemory(); + return -1; } - else if (n > self->buf_size) { - char *newbuf = (char *)realloc(self->buf, n); - if (!newbuf) { - PyErr_NoMemory(); - return -1; - } - self->buf = newbuf; - self->buf_size = n; + + self->buf_size = size; + } + else if (n > self->buf_size) { + char *newbuf = (char *) realloc(self->buf, n); + if (!newbuf) { + PyErr_NoMemory(); + return -1; } + self->buf = newbuf; + self->buf_size = n; + } - Py_BEGIN_ALLOW_THREADS + Py_BEGIN_ALLOW_THREADS nbytesread = fread(self->buf, sizeof(char), n, self->fp); - Py_END_ALLOW_THREADS - if (nbytesread != (size_t)n) { - if (feof(self->fp)) { - PyErr_SetNone(PyExc_EOFError); - return -1; - } - - PyErr_SetFromErrno(PyExc_IOError); - return -1; + Py_END_ALLOW_THREADS if (nbytesread != (size_t) n) { + if (feof(self->fp)) { + PyErr_SetNone(PyExc_EOFError); + return -1; } - *s = self->buf; + PyErr_SetFromErrno(PyExc_IOError); + return -1; + } + + *s = self->buf; - return n; + return n; } static Py_ssize_t -readline_file(Unpicklerobject *self, char **s) +readline_file(Unpicklerobject * self, char **s) { - int i; + int i; - if (self->buf_size == 0) { - if (!( self->buf = (char *)malloc(40))) { - PyErr_NoMemory(); - return -1; - } - self->buf_size = 40; + if (self->buf_size == 0) { + if (!(self->buf = (char *) malloc(40))) { + PyErr_NoMemory(); + return -1; } + self->buf_size = 40; + } - i = 0; - while (1) { - int bigger; - char *newbuf; - for (; i < (self->buf_size - 1); i++) { - if (feof(self->fp) || - (self->buf[i] = getc(self->fp)) == '\n') { - self->buf[i + 1] = '\0'; - *s = self->buf; - return i + 1; - } - } - bigger = self->buf_size << 1; - if (bigger <= 0) { /* overflow */ - PyErr_NoMemory(); - return -1; - } - newbuf = (char *)realloc(self->buf, bigger); - if (!newbuf) { - PyErr_NoMemory(); - return -1; - } - self->buf = newbuf; - self->buf_size = bigger; - } + i = 0; + while (1) { + int bigger; + char *newbuf; + for (; i < (self->buf_size - 1); i++) { + if (feof(self->fp) || (self->buf[i] = getc(self->fp)) == '\n') { + self->buf[i + 1] = '\0'; + *s = self->buf; + return i + 1; + } + } + bigger = self->buf_size << 1; + if (bigger <= 0) { /* overflow */ + PyErr_NoMemory(); + return -1; + } + newbuf = (char *) realloc(self->buf, bigger); + if (!newbuf) { + PyErr_NoMemory(); + return -1; + } + self->buf = newbuf; + self->buf_size = bigger; + } } static Py_ssize_t -read_cStringIO(Unpicklerobject *self, char **s, Py_ssize_t n) +read_cStringIO(Unpicklerobject * self, char **s, Py_ssize_t n) { - char *ptr; + char *ptr; - if (PycStringIO->cread((PyObject *)self->file, &ptr, n) != n) { - PyErr_SetNone(PyExc_EOFError); - return -1; - } + if (PycStringIO->cread((PyObject *) self->file, &ptr, n) != n) { + PyErr_SetNone(PyExc_EOFError); + return -1; + } - *s = ptr; + *s = ptr; - return n; + return n; } static Py_ssize_t -readline_cStringIO(Unpicklerobject *self, char **s) +readline_cStringIO(Unpicklerobject * self, char **s) { - Py_ssize_t n; - char *ptr; + Py_ssize_t n; + char *ptr; - if ((n = PycStringIO->creadline((PyObject *)self->file, &ptr)) < 0) { - return -1; - } + if ((n = PycStringIO->creadline((PyObject *) self->file, &ptr)) < 0) { + return -1; + } - *s = ptr; + *s = ptr; - return n; + return n; } static Py_ssize_t -read_other(Unpicklerobject *self, char **s, Py_ssize_t n) +read_other(Unpicklerobject * self, char **s, Py_ssize_t n) { - PyObject *bytes, *str=0; + PyObject *bytes, *str = 0; - if (!( bytes = PyInt_FromSsize_t(n))) return -1; + if (!(bytes = PyInt_FromSsize_t(n))) + return -1; - ARG_TUP(self, bytes); - if (self->arg) { - str = PyObject_Call(self->read, self->arg, NULL); - FREE_ARG_TUP(self); - } - if (! str) return -1; + ARG_TUP(self, bytes); + if (self->arg) { + str = PyObject_Call(self->read, self->arg, NULL); + FREE_ARG_TUP(self); + } + if (!str) + return -1; - Py_XDECREF(self->last_string); - self->last_string = str; + Py_XDECREF(self->last_string); + self->last_string = str; - if (! (*s = PyString_AsString(str))) return -1; - return n; + if (!(*s = PyString_AsString(str))) + return -1; + return n; } static Py_ssize_t -readline_other(Unpicklerobject *self, char **s) +readline_other(Unpicklerobject * self, char **s) { - PyObject *str; - Py_ssize_t str_size; + PyObject *str; + Py_ssize_t str_size; - if (!( str = PyObject_CallObject(self->readline, empty_tuple))) { - return -1; - } + if (!(str = PyObject_CallObject(self->readline, empty_tuple))) { + return -1; + } - if ((str_size = PyString_Size(str)) < 0) - return -1; + if ((str_size = PyString_Size(str)) < 0) + return -1; - Py_XDECREF(self->last_string); - self->last_string = str; + Py_XDECREF(self->last_string); + self->last_string = str; - if (! (*s = PyString_AsString(str))) - return -1; + if (!(*s = PyString_AsString(str))) + return -1; - return str_size; + return str_size; } /* Copy the first n bytes from s into newly malloc'ed memory, plus a @@ -685,574 +689,580 @@ static char * pystrndup(const char *s, int n) { - char *r = (char *)malloc(n+1); - if (r == NULL) - return (char*)PyErr_NoMemory(); - memcpy(r, s, n); - r[n] = 0; - return r; + char *r = (char *) malloc(n + 1); + if (r == NULL) + return (char *) PyErr_NoMemory(); + memcpy(r, s, n); + r[n] = 0; + return r; } static int -get(Picklerobject *self, PyObject *id) +get(Picklerobject * self, PyObject * id) { - PyObject *value, *mv; - long c_value; - char s[30]; - size_t len; + PyObject *value, *mv; + long c_value; + char s[30]; + size_t len; - if (!( mv = PyDict_GetItem(self->memo, id))) { - PyErr_SetObject(PyExc_KeyError, id); - return -1; - } + if (!(mv = PyDict_GetItem(self->memo, id))) { + PyErr_SetObject(PyExc_KeyError, id); + return -1; + } - if (!( value = PyTuple_GetItem(mv, 0))) - return -1; + if (!(value = PyTuple_GetItem(mv, 0))) + return -1; - if (!( PyInt_Check(value))) { - PyErr_SetString(PicklingError, "no int where int expected in memo"); - return -1; - } - c_value = PyInt_AsLong(value); - if (c_value == -1 && PyErr_Occurred()) - return -1; + if (!(PyInt_Check(value))) { + PyErr_SetString(PicklingError, "no int where int expected in memo"); + return -1; + } + c_value = PyInt_AsLong(value); + if (c_value == -1 && PyErr_Occurred()) + return -1; - if (!self->bin) { - s[0] = GET; - PyOS_snprintf(s + 1, sizeof(s) - 1, "%ld\n", c_value); - len = strlen(s); - } - else if (Pdata_Check(self->file)) { - if (write_other(self, NULL, 0) < 0) return -1; - PDATA_APPEND(self->file, mv, -1); - return 0; + if (!self->bin) { + s[0] = GET; + PyOS_snprintf(s + 1, sizeof(s) - 1, "%ld\n", c_value); + len = strlen(s); + } + else if (Pdata_Check(self->file)) { + if (write_other(self, NULL, 0) < 0) + return -1; + PDATA_APPEND(self->file, mv, -1); + return 0; + } + else { + if (c_value < 256) { + s[0] = BINGET; + s[1] = (int) (c_value & 0xff); + len = 2; } else { - if (c_value < 256) { - s[0] = BINGET; - s[1] = (int)(c_value & 0xff); - len = 2; - } - else { - s[0] = LONG_BINGET; - s[1] = (int)(c_value & 0xff); - s[2] = (int)((c_value >> 8) & 0xff); - s[3] = (int)((c_value >> 16) & 0xff); - s[4] = (int)((c_value >> 24) & 0xff); - len = 5; - } + s[0] = LONG_BINGET; + s[1] = (int) (c_value & 0xff); + s[2] = (int) ((c_value >> 8) & 0xff); + s[3] = (int) ((c_value >> 16) & 0xff); + s[4] = (int) ((c_value >> 24) & 0xff); + len = 5; } + } - if (self->write_func(self, s, len) < 0) - return -1; + if (self->write_func(self, s, len) < 0) + return -1; - return 0; + return 0; } static int -put(Picklerobject *self, PyObject *ob) +put(Picklerobject * self, PyObject * ob) { - if (ob->ob_refcnt < 2 || self->fast) - return 0; + if (ob->ob_refcnt < 2 || self->fast) + return 0; - return put2(self, ob); + return put2(self, ob); } static int -put2(Picklerobject *self, PyObject *ob) +put2(Picklerobject * self, PyObject * ob) { - char c_str[30]; - int p; - size_t len; - int res = -1; - PyObject *py_ob_id = 0, *memo_len = 0, *t = 0; + char c_str[30]; + int p; + size_t len; + int res = -1; + PyObject *py_ob_id = 0, *memo_len = 0, *t = 0; - if (self->fast) - return 0; + if (self->fast) + return 0; - if ((p = PyDict_Size(self->memo)) < 0) - goto finally; + if ((p = PyDict_Size(self->memo)) < 0) + goto finally; - /* Make sure memo keys are positive! */ - /* XXX Why? - * XXX And does "positive" really mean non-negative? - * XXX pickle.py starts with PUT index 0, not 1. This makes for - * XXX gratuitous differences between the pickling modules. - */ - p++; + /* Make sure memo keys are positive! */ + /* XXX Why? + * XXX And does "positive" really mean non-negative? + * XXX pickle.py starts with PUT index 0, not 1. This makes for + * XXX gratuitous differences between the pickling modules. + */ + p++; - if (!( py_ob_id = PyLong_FromVoidPtr(ob))) - goto finally; + if (!(py_ob_id = PyLong_FromVoidPtr(ob))) + goto finally; - if (!( memo_len = PyInt_FromLong(p))) - goto finally; + if (!(memo_len = PyInt_FromLong(p))) + goto finally; - if (!( t = PyTuple_New(2))) - goto finally; + if (!(t = PyTuple_New(2))) + goto finally; - PyTuple_SET_ITEM(t, 0, memo_len); - Py_INCREF(memo_len); - PyTuple_SET_ITEM(t, 1, ob); - Py_INCREF(ob); + PyTuple_SET_ITEM(t, 0, memo_len); + Py_INCREF(memo_len); + PyTuple_SET_ITEM(t, 1, ob); + Py_INCREF(ob); - if (PyDict_SetItem(self->memo, py_ob_id, t) < 0) - goto finally; + if (PyDict_SetItem(self->memo, py_ob_id, t) < 0) + goto finally; - if (!self->bin) { - c_str[0] = PUT; - PyOS_snprintf(c_str + 1, sizeof(c_str) - 1, "%d\n", p); - len = strlen(c_str); - } - else if (Pdata_Check(self->file)) { - if (write_other(self, NULL, 0) < 0) return -1; - PDATA_APPEND(self->file, memo_len, -1); - res=0; /* Job well done ;) */ - goto finally; + if (!self->bin) { + c_str[0] = PUT; + PyOS_snprintf(c_str + 1, sizeof(c_str) - 1, "%d\n", p); + len = strlen(c_str); + } + else if (Pdata_Check(self->file)) { + if (write_other(self, NULL, 0) < 0) + return -1; + PDATA_APPEND(self->file, memo_len, -1); + res = 0; /* Job well done ;) */ + goto finally; + } + else { + if (p >= 256) { + c_str[0] = LONG_BINPUT; + c_str[1] = (int) (p & 0xff); + c_str[2] = (int) ((p >> 8) & 0xff); + c_str[3] = (int) ((p >> 16) & 0xff); + c_str[4] = (int) ((p >> 24) & 0xff); + len = 5; } else { - if (p >= 256) { - c_str[0] = LONG_BINPUT; - c_str[1] = (int)(p & 0xff); - c_str[2] = (int)((p >> 8) & 0xff); - c_str[3] = (int)((p >> 16) & 0xff); - c_str[4] = (int)((p >> 24) & 0xff); - len = 5; - } - else { - c_str[0] = BINPUT; - c_str[1] = p; - len = 2; - } + c_str[0] = BINPUT; + c_str[1] = p; + len = 2; } + } - if (self->write_func(self, c_str, len) < 0) - goto finally; + if (self->write_func(self, c_str, len) < 0) + goto finally; - res = 0; + res = 0; finally: - Py_XDECREF(py_ob_id); - Py_XDECREF(memo_len); - Py_XDECREF(t); + Py_XDECREF(py_ob_id); + Py_XDECREF(memo_len); + Py_XDECREF(t); - return res; + return res; } static PyObject * -whichmodule(PyObject *global, PyObject *global_name) +whichmodule(PyObject * global, PyObject * global_name) { - Py_ssize_t i, j; - PyObject *module = 0, *modules_dict = 0, - *global_name_attr = 0, *name = 0; - - module = PyObject_GetAttrString(global, "__module__"); - if (module) - return module; - if (PyErr_ExceptionMatches(PyExc_AttributeError)) - PyErr_Clear(); - else - return NULL; - - if (!( modules_dict = PySys_GetObject("modules"))) - return NULL; + Py_ssize_t i, j; + PyObject *module = 0, *modules_dict = 0, *global_name_attr = 0, *name = 0; - i = 0; - while ((j = PyDict_Next(modules_dict, &i, &name, &module))) { - - if (PyObject_Compare(name, __main___str)==0) continue; + module = PyObject_GetAttrString(global, "__module__"); + if (module) + return module; + if (PyErr_ExceptionMatches(PyExc_AttributeError)) + PyErr_Clear(); + else + return NULL; - global_name_attr = PyObject_GetAttr(module, global_name); - if (!global_name_attr) { - if (PyErr_ExceptionMatches(PyExc_AttributeError)) - PyErr_Clear(); - else - return NULL; - continue; - } + if (!(modules_dict = PySys_GetObject("modules"))) + return NULL; - if (global_name_attr != global) { - Py_DECREF(global_name_attr); - continue; - } + i = 0; + while ((j = PyDict_Next(modules_dict, &i, &name, &module))) { - Py_DECREF(global_name_attr); + if (PyObject_Compare(name, __main___str) == 0) + continue; - break; + global_name_attr = PyObject_GetAttr(module, global_name); + if (!global_name_attr) { + if (PyErr_ExceptionMatches(PyExc_AttributeError)) + PyErr_Clear(); + else + return NULL; + continue; } - /* The following implements the rule in pickle.py added in 1.5 - that used __main__ if no module is found. I don't actually - like this rule. jlf - */ - if (!j) { - j=1; - name=__main___str; + if (global_name_attr != global) { + Py_DECREF(global_name_attr); + continue; } - Py_INCREF(name); - return name; + Py_DECREF(global_name_attr); + + break; + } + + /* The following implements the rule in pickle.py added in 1.5 + * that used __main__ if no module is found. I don't actually + * like this rule. jlf + */ + if (!j) { + j = 1; + name = __main___str; + } + + Py_INCREF(name); + return name; } static int -fast_save_enter(Picklerobject *self, PyObject *obj) +fast_save_enter(Picklerobject * self, PyObject * obj) { - /* if fast_container < 0, we're doing an error exit. */ - if (++self->fast_container >= PY_CPICKLE_FAST_LIMIT) { - PyObject *key = NULL; - if (self->fast_memo == NULL) { - self->fast_memo = PyDict_New(); - if (self->fast_memo == NULL) { - self->fast_container = -1; - return 0; - } - } - key = PyLong_FromVoidPtr(obj); - if (key == NULL) - return 0; - if (PyDict_GetItem(self->fast_memo, key)) { - Py_DECREF(key); - PyErr_Format(PyExc_ValueError, - "fast mode: can't pickle cyclic objects " - "including object type %s at %p", - obj->ob_type->tp_name, obj); - self->fast_container = -1; - return 0; - } - if (PyDict_SetItem(self->fast_memo, key, Py_None) < 0) { - Py_DECREF(key); - self->fast_container = -1; - return 0; - } - Py_DECREF(key); + /* if fast_container < 0, we're doing an error exit. */ + if (++self->fast_container >= PY_CPICKLE_FAST_LIMIT) { + PyObject *key = NULL; + if (self->fast_memo == NULL) { + self->fast_memo = PyDict_New(); + if (self->fast_memo == NULL) { + self->fast_container = -1; + return 0; + } } - return 1; + key = PyLong_FromVoidPtr(obj); + if (key == NULL) + return 0; + if (PyDict_GetItem(self->fast_memo, key)) { + Py_DECREF(key); + PyErr_Format(PyExc_ValueError, + "fast mode: can't pickle cyclic objects " + "including object type %s at %p", + obj->ob_type->tp_name, obj); + self->fast_container = -1; + return 0; + } + if (PyDict_SetItem(self->fast_memo, key, Py_None) < 0) { + Py_DECREF(key); + self->fast_container = -1; + return 0; + } + Py_DECREF(key); + } + return 1; } int -fast_save_leave(Picklerobject *self, PyObject *obj) +fast_save_leave(Picklerobject * self, PyObject * obj) { - if (self->fast_container-- >= PY_CPICKLE_FAST_LIMIT) { - PyObject *key = PyLong_FromVoidPtr(obj); - if (key == NULL) - return 0; - if (PyDict_DelItem(self->fast_memo, key) < 0) { - Py_DECREF(key); - return 0; - } - Py_DECREF(key); + if (self->fast_container-- >= PY_CPICKLE_FAST_LIMIT) { + PyObject *key = PyLong_FromVoidPtr(obj); + if (key == NULL) + return 0; + if (PyDict_DelItem(self->fast_memo, key) < 0) { + Py_DECREF(key); + return 0; } - return 1; + Py_DECREF(key); + } + return 1; } static int -save_none(Picklerobject *self, PyObject *args) +save_none(Picklerobject * self, PyObject * args) { - static char none = NONE; - if (self->write_func(self, &none, 1) < 0) - return -1; + static char none = NONE; + if (self->write_func(self, &none, 1) < 0) + return -1; - return 0; + return 0; } static int -save_bool(Picklerobject *self, PyObject *args) +save_bool(Picklerobject * self, PyObject * args) { - static const char *buf[2] = {FALSE, TRUE}; - static char len[2] = {sizeof(FALSE)-1, sizeof(TRUE)-1}; - long l = args == Py_True; - - if (self->proto >= 2) { - char opcode = l ? NEWTRUE : NEWFALSE; - if (self->write_func(self, &opcode, 1) < 0) - return -1; - } - else if (self->write_func(self, buf[l], len[l]) < 0) - return -1; - return 0; + static const char *buf[2] = { FALSE, TRUE }; + static char len[2] = { sizeof(FALSE) - 1, sizeof(TRUE) - 1 }; + long l = args == Py_True; + + if (self->proto >= 2) { + char opcode = l ? NEWTRUE : NEWFALSE; + if (self->write_func(self, &opcode, 1) < 0) + return -1; + } + else if (self->write_func(self, buf[l], len[l]) < 0) + return -1; + return 0; } static int -save_int(Picklerobject *self, long l) +save_int(Picklerobject * self, long l) { - char c_str[32]; - int len = 0; + char c_str[32]; + int len = 0; - if (!self->bin + if (!self->bin #if SIZEOF_LONG > 4 - || l > 0x7fffffffL - || l < -0x80000000L + || l > 0x7fffffffL || l < -0x80000000L #endif - ) { - /* Text-mode pickle, or long too big to fit in the 4-byte - * signed BININT format: store as a string. - */ - c_str[0] = INT; - PyOS_snprintf(c_str + 1, sizeof(c_str) - 1, "%ld\n", l); - if (self->write_func(self, c_str, strlen(c_str)) < 0) - return -1; + ) { + /* Text-mode pickle, or long too big to fit in the 4-byte + * signed BININT format: store as a string. + */ + c_str[0] = INT; + PyOS_snprintf(c_str + 1, sizeof(c_str) - 1, "%ld\n", l); + if (self->write_func(self, c_str, strlen(c_str)) < 0) + return -1; + } + else { + /* Binary pickle and l fits in a signed 4-byte int. */ + c_str[1] = (int) (l & 0xff); + c_str[2] = (int) ((l >> 8) & 0xff); + c_str[3] = (int) ((l >> 16) & 0xff); + c_str[4] = (int) ((l >> 24) & 0xff); + + if ((c_str[4] == 0) && (c_str[3] == 0)) { + if (c_str[2] == 0) { + c_str[0] = BININT1; + len = 2; + } + else { + c_str[0] = BININT2; + len = 3; + } } else { - /* Binary pickle and l fits in a signed 4-byte int. */ - c_str[1] = (int)( l & 0xff); - c_str[2] = (int)((l >> 8) & 0xff); - c_str[3] = (int)((l >> 16) & 0xff); - c_str[4] = (int)((l >> 24) & 0xff); - - if ((c_str[4] == 0) && (c_str[3] == 0)) { - if (c_str[2] == 0) { - c_str[0] = BININT1; - len = 2; - } - else { - c_str[0] = BININT2; - len = 3; - } - } - else { - c_str[0] = BININT; - len = 5; - } - - if (self->write_func(self, c_str, len) < 0) - return -1; + c_str[0] = BININT; + len = 5; } - return 0; + if (self->write_func(self, c_str, len) < 0) + return -1; + } + + return 0; } static int -save_long(Picklerobject *self, PyObject *args) +save_long(Picklerobject * self, PyObject * args) { - Py_ssize_t size; - int res = -1; - PyObject *repr = NULL; - long val = PyInt_AsLong(args); - static char l = LONG; + Py_ssize_t size; + int res = -1; + PyObject *repr = NULL; + long val = PyInt_AsLong(args); + static char l = LONG; - if (val == -1 && PyErr_Occurred()) { - /* out of range for int pickling */ - PyErr_Clear(); - } - else - return save_int(self, val); + if (val == -1 && PyErr_Occurred()) { + /* out of range for int pickling */ + PyErr_Clear(); + } + else + return save_int(self, val); + + if (self->proto >= 2) { + /* Linear-time pickling. */ + size_t nbits; + size_t nbytes; + unsigned char *pdata; + char c_str[5]; + int i; + int sign = _PyLong_Sign(args); - if (self->proto >= 2) { - /* Linear-time pickling. */ - size_t nbits; - size_t nbytes; - unsigned char *pdata; - char c_str[5]; - int i; - int sign = _PyLong_Sign(args); - - if (sign == 0) { - /* It's 0 -- an empty bytestring. */ - c_str[0] = LONG1; - c_str[1] = 0; - i = self->write_func(self, c_str, 2); - if (i < 0) goto finally; - res = 0; - goto finally; - } - nbits = _PyLong_NumBits(args); - if (nbits == (size_t)-1 && PyErr_Occurred()) - goto finally; - /* How many bytes do we need? There are nbits >> 3 full - * bytes of data, and nbits & 7 leftover bits. If there - * are any leftover bits, then we clearly need another - * byte. Wnat's not so obvious is that we *probably* - * need another byte even if there aren't any leftovers: - * the most-significant bit of the most-significant byte - * acts like a sign bit, and it's usually got a sense - * opposite of the one we need. The exception is longs - * of the form -(2**(8*j-1)) for j > 0. Such a long is - * its own 256's-complement, so has the right sign bit - * even without the extra byte. That's a pain to check - * for in advance, though, so we always grab an extra - * byte at the start, and cut it back later if possible. - */ - nbytes = (nbits >> 3) + 1; - if (nbytes > INT_MAX) { - PyErr_SetString(PyExc_OverflowError, "long too large " - "to pickle"); - goto finally; - } - repr = PyString_FromStringAndSize(NULL, (int)nbytes); - if (repr == NULL) goto finally; - pdata = (unsigned char *)PyString_AS_STRING(repr); - i = _PyLong_AsByteArray((PyLongObject *)args, - pdata, nbytes, - 1 /* little endian */, 1 /* signed */); - if (i < 0) goto finally; - /* If the long is negative, this may be a byte more than - * needed. This is so iff the MSB is all redundant sign - * bits. - */ - if (sign < 0 && nbytes > 1 && pdata[nbytes - 1] == 0xff && - (pdata[nbytes - 2] & 0x80) != 0) - --nbytes; - - if (nbytes < 256) { - c_str[0] = LONG1; - c_str[1] = (char)nbytes; - size = 2; - } - else { - c_str[0] = LONG4; - size = (int)nbytes; - for (i = 1; i < 5; i++) { - c_str[i] = (char)(size & 0xff); - size >>= 8; - } - size = 5; - } - i = self->write_func(self, c_str, size); - if (i < 0) goto finally; - i = self->write_func(self, (char *)pdata, (int)nbytes); - if (i < 0) goto finally; - res = 0; - goto finally; + if (sign == 0) { + /* It's 0 -- an empty bytestring. */ + c_str[0] = LONG1; + c_str[1] = 0; + i = self->write_func(self, c_str, 2); + if (i < 0) + goto finally; + res = 0; + goto finally; + } + nbits = _PyLong_NumBits(args); + if (nbits == (size_t) - 1 && PyErr_Occurred()) + goto finally; + /* How many bytes do we need? There are nbits >> 3 full + * bytes of data, and nbits & 7 leftover bits. If there + * are any leftover bits, then we clearly need another + * byte. Wnat's not so obvious is that we *probably* + * need another byte even if there aren't any leftovers: + * the most-significant bit of the most-significant byte + * acts like a sign bit, and it's usually got a sense + * opposite of the one we need. The exception is longs + * of the form -(2**(8*j-1)) for j > 0. Such a long is + * its own 256's-complement, so has the right sign bit + * even without the extra byte. That's a pain to check + * for in advance, though, so we always grab an extra + * byte at the start, and cut it back later if possible. + */ + nbytes = (nbits >> 3) + 1; + if (nbytes > INT_MAX) { + PyErr_SetString(PyExc_OverflowError, "long too large " + "to pickle"); + goto finally; } - - /* proto < 2: write the repr and newline. This is quadratic-time - * (in the number of digits), in both directions. + repr = PyString_FromStringAndSize(NULL, (int) nbytes); + if (repr == NULL) + goto finally; + pdata = (unsigned char *) PyString_AS_STRING(repr); + i = _PyLong_AsByteArray((PyLongObject *) args, + pdata, nbytes, + 1 /* little endian */ , 1 /* signed */ ); + if (i < 0) + goto finally; + /* If the long is negative, this may be a byte more than + * needed. This is so iff the MSB is all redundant sign + * bits. */ - if (!( repr = PyObject_Repr(args))) - goto finally; - - if ((size = PyString_Size(repr)) < 0) - goto finally; - - if (self->write_func(self, &l, 1) < 0) - goto finally; + if (sign < 0 && nbytes > 1 && pdata[nbytes - 1] == 0xff && + (pdata[nbytes - 2] & 0x80) != 0) + --nbytes; + + if (nbytes < 256) { + c_str[0] = LONG1; + c_str[1] = (char) nbytes; + size = 2; + } + else { + c_str[0] = LONG4; + size = (int) nbytes; + for (i = 1; i < 5; i++) { + c_str[i] = (char) (size & 0xff); + size >>= 8; + } + size = 5; + } + i = self->write_func(self, c_str, size); + if (i < 0) + goto finally; + i = self->write_func(self, (char *) pdata, (int) nbytes); + if (i < 0) + goto finally; + res = 0; + goto finally; + } - if (self->write_func(self, - PyString_AS_STRING((PyStringObject *)repr), - size) < 0) - goto finally; + /* proto < 2: write the repr and newline. This is quadratic-time + * (in the number of digits), in both directions. + */ + if (!(repr = PyObject_Repr(args))) + goto finally; + + if ((size = PyString_Size(repr)) < 0) + goto finally; + + if (self->write_func(self, &l, 1) < 0) + goto finally; + + if (self->write_func(self, + PyString_AS_STRING((PyStringObject *) repr), + size) < 0) + goto finally; - if (self->write_func(self, "\n", 1) < 0) - goto finally; + if (self->write_func(self, "\n", 1) < 0) + goto finally; - res = 0; + res = 0; finally: - Py_XDECREF(repr); - return res; + Py_XDECREF(repr); + return res; } static int -save_float(Picklerobject *self, PyObject *args) +save_float(Picklerobject * self, PyObject * args) { - double x = PyFloat_AS_DOUBLE((PyFloatObject *)args); + double x = PyFloat_AS_DOUBLE((PyFloatObject *) args); - if (self->bin) { - char str[9]; - str[0] = BINFLOAT; - if (_PyFloat_Pack8(x, (unsigned char *)&str[1], 0) < 0) - return -1; - if (self->write_func(self, str, 9) < 0) - return -1; - } - else { - char c_str[250]; - c_str[0] = FLOAT; - PyOS_ascii_formatd(c_str + 1, sizeof(c_str) - 2, "%.17g", x); - /* Extend the formatted string with a newline character */ - strcat(c_str, "\n"); + if (self->bin) { + char str[9]; + str[0] = BINFLOAT; + if (_PyFloat_Pack8(x, (unsigned char *) &str[1], 0) < 0) + return -1; + if (self->write_func(self, str, 9) < 0) + return -1; + } + else { + char c_str[250]; + c_str[0] = FLOAT; + PyOS_ascii_formatd(c_str + 1, sizeof(c_str) - 2, "%.17g", x); + /* Extend the formatted string with a newline character */ + strcat(c_str, "\n"); - if (self->write_func(self, c_str, strlen(c_str)) < 0) - return -1; - } + if (self->write_func(self, c_str, strlen(c_str)) < 0) + return -1; + } - return 0; + return 0; } static int -save_string(Picklerobject *self, PyObject *args, int doput) +save_string(Picklerobject * self, PyObject * args, int doput) { - int size, len; - PyObject *repr=0; + int size, len; + PyObject *repr = 0; - if ((size = PyString_Size(args)) < 0) - return -1; + if ((size = PyString_Size(args)) < 0) + return -1; - if (!self->bin) { - char *repr_str; + if (!self->bin) { + char *repr_str; - static char string = STRING; + static char string = STRING; - if (!( repr = PyObject_Repr(args))) - return -1; + if (!(repr = PyObject_Repr(args))) + return -1; - if ((len = PyString_Size(repr)) < 0) - goto err; - repr_str = PyString_AS_STRING((PyStringObject *)repr); + if ((len = PyString_Size(repr)) < 0) + goto err; + repr_str = PyString_AS_STRING((PyStringObject *) repr); - if (self->write_func(self, &string, 1) < 0) - goto err; + if (self->write_func(self, &string, 1) < 0) + goto err; - if (self->write_func(self, repr_str, len) < 0) - goto err; + if (self->write_func(self, repr_str, len) < 0) + goto err; - if (self->write_func(self, "\n", 1) < 0) - goto err; + if (self->write_func(self, "\n", 1) < 0) + goto err; - Py_XDECREF(repr); - } - else { - int i; - char c_str[5]; + Py_XDECREF(repr); + } + else { + int i; + char c_str[5]; - if ((size = PyString_Size(args)) < 0) - return -1; + if ((size = PyString_Size(args)) < 0) + return -1; - if (size < 256) { - c_str[0] = SHORT_BINSTRING; - c_str[1] = size; - len = 2; - } - else if (size <= INT_MAX) { - c_str[0] = BINSTRING; - for (i = 1; i < 5; i++) - c_str[i] = (int)(size >> ((i - 1) * 8)); - len = 5; - } - else - return -1; /* string too large */ + if (size < 256) { + c_str[0] = SHORT_BINSTRING; + c_str[1] = size; + len = 2; + } + else if (size <= INT_MAX) { + c_str[0] = BINSTRING; + for (i = 1; i < 5; i++) + c_str[i] = (int) (size >> ((i - 1) * 8)); + len = 5; + } + else + return -1; /* string too large */ - if (self->write_func(self, c_str, len) < 0) - return -1; + if (self->write_func(self, c_str, len) < 0) + return -1; - if (size > 128 && Pdata_Check(self->file)) { - if (write_other(self, NULL, 0) < 0) return -1; - PDATA_APPEND(self->file, args, -1); - } - else { - if (self->write_func(self, - PyString_AS_STRING( - (PyStringObject *)args), - size) < 0) - return -1; - } + if (size > 128 && Pdata_Check(self->file)) { + if (write_other(self, NULL, 0) < 0) + return -1; + PDATA_APPEND(self->file, args, -1); + } + else { + if (self->write_func(self, + PyString_AS_STRING((PyStringObject *) args), + size) < 0) + return -1; } + } - if (doput) - if (put(self, args) < 0) - return -1; + if (doput) + if (put(self, args) < 0) + return -1; - return 0; + return 0; err: - Py_XDECREF(repr); - return -1; + Py_XDECREF(repr); + return -1; } @@ -1260,142 +1270,142 @@ /* A copy of PyUnicode_EncodeRawUnicodeEscape() that also translates backslash and newline characters to \uXXXX escapes. */ static PyObject * -modified_EncodeRawUnicodeEscape(const Py_UNICODE *s, int size) +modified_EncodeRawUnicodeEscape(const Py_UNICODE * s, int size) { - PyObject *repr; - char *p; - char *q; + PyObject *repr; + char *p; + char *q; - static const char *hexdigit = "0123456789ABCDEF"; + static const char *hexdigit = "0123456789ABCDEF"; - repr = PyString_FromStringAndSize(NULL, 6 * size); - if (repr == NULL) - return NULL; - if (size == 0) - return repr; + repr = PyString_FromStringAndSize(NULL, 6 * size); + if (repr == NULL) + return NULL; + if (size == 0) + return repr; - p = q = PyString_AS_STRING(repr); - while (size-- > 0) { - Py_UNICODE ch = *s++; - /* Map 16-bit characters to '\uxxxx' */ - if (ch >= 256 || ch == '\\' || ch == '\n') { - *p++ = '\\'; - *p++ = 'u'; - *p++ = hexdigit[(ch >> 12) & 0xf]; - *p++ = hexdigit[(ch >> 8) & 0xf]; - *p++ = hexdigit[(ch >> 4) & 0xf]; - *p++ = hexdigit[ch & 15]; - } - /* Copy everything else as-is */ - else - *p++ = (char) ch; + p = q = PyString_AS_STRING(repr); + while (size-- > 0) { + Py_UNICODE ch = *s++; + /* Map 16-bit characters to '\uxxxx' */ + if (ch >= 256 || ch == '\\' || ch == '\n') { + *p++ = '\\'; + *p++ = 'u'; + *p++ = hexdigit[(ch >> 12) & 0xf]; + *p++ = hexdigit[(ch >> 8) & 0xf]; + *p++ = hexdigit[(ch >> 4) & 0xf]; + *p++ = hexdigit[ch & 15]; } - *p = '\0'; - _PyString_Resize(&repr, p - q); - return repr; + /* Copy everything else as-is */ + else + *p++ = (char) ch; + } + *p = '\0'; + _PyString_Resize(&repr, p - q); + return repr; } static int -save_unicode(Picklerobject *self, PyObject *args, int doput) +save_unicode(Picklerobject * self, PyObject * args, int doput) { - Py_ssize_t size, len; - PyObject *repr=0; + Py_ssize_t size, len; + PyObject *repr = 0; - if (!PyUnicode_Check(args)) - return -1; + if (!PyUnicode_Check(args)) + return -1; - if (!self->bin) { - char *repr_str; - static char string = UNICODE; + if (!self->bin) { + char *repr_str; + static char string = UNICODE; + + repr = + modified_EncodeRawUnicodeEscape(PyUnicode_AS_UNICODE(args), + PyUnicode_GET_SIZE(args)); + if (!repr) + return -1; + + if ((len = PyString_Size(repr)) < 0) + goto err; + repr_str = PyString_AS_STRING((PyStringObject *) repr); - repr = modified_EncodeRawUnicodeEscape( - PyUnicode_AS_UNICODE(args), PyUnicode_GET_SIZE(args)); - if (!repr) - return -1; + if (self->write_func(self, &string, 1) < 0) + goto err; - if ((len = PyString_Size(repr)) < 0) - goto err; - repr_str = PyString_AS_STRING((PyStringObject *)repr); + if (self->write_func(self, repr_str, len) < 0) + goto err; - if (self->write_func(self, &string, 1) < 0) - goto err; + if (self->write_func(self, "\n", 1) < 0) + goto err; - if (self->write_func(self, repr_str, len) < 0) - goto err; + Py_XDECREF(repr); + } + else { + int i; + char c_str[5]; - if (self->write_func(self, "\n", 1) < 0) - goto err; + if (!(repr = PyUnicode_AsUTF8String(args))) + return -1; - Py_XDECREF(repr); - } - else { - int i; - char c_str[5]; + if ((size = PyString_Size(repr)) < 0) + goto err; + if (size > INT_MAX) + return -1; /* string too large */ + + c_str[0] = BINUNICODE; + for (i = 1; i < 5; i++) + c_str[i] = (int) (size >> ((i - 1) * 8)); + len = 5; - if (!( repr = PyUnicode_AsUTF8String(args))) - return -1; + if (self->write_func(self, c_str, len) < 0) + goto err; - if ((size = PyString_Size(repr)) < 0) - goto err; - if (size > INT_MAX) - return -1; /* string too large */ - - c_str[0] = BINUNICODE; - for (i = 1; i < 5; i++) - c_str[i] = (int)(size >> ((i - 1) * 8)); - len = 5; - - if (self->write_func(self, c_str, len) < 0) - goto err; - - if (size > 128 && Pdata_Check(self->file)) { - if (write_other(self, NULL, 0) < 0) - goto err; - PDATA_APPEND(self->file, repr, -1); - } - else { - if (self->write_func(self, PyString_AS_STRING(repr), - size) < 0) - goto err; - } - - Py_DECREF(repr); + if (size > 128 && Pdata_Check(self->file)) { + if (write_other(self, NULL, 0) < 0) + goto err; + PDATA_APPEND(self->file, repr, -1); + } + else { + if (self->write_func(self, PyString_AS_STRING(repr), size) < 0) + goto err; } - if (doput) - if (put(self, args) < 0) - return -1; + Py_DECREF(repr); + } - return 0; + if (doput) + if (put(self, args) < 0) + return -1; + + return 0; err: - Py_XDECREF(repr); - return -1; + Py_XDECREF(repr); + return -1; } #endif /* A helper for save_tuple. Push the len elements in tuple t on the stack. */ static int -store_tuple_elements(Picklerobject *self, PyObject *t, int len) +store_tuple_elements(Picklerobject * self, PyObject * t, int len) { - int i; - int res = -1; /* guilty until proved innocent */ - - assert(PyTuple_Size(t) == len); + int i; + int res = -1; /* guilty until proved innocent */ - for (i = 0; i < len; i++) { - PyObject *element = PyTuple_GET_ITEM(t, i); + assert(PyTuple_Size(t) == len); - if (element == NULL) - goto finally; - if (save(self, element, 0) < 0) - goto finally; - } - res = 0; + for (i = 0; i < len; i++) { + PyObject *element = PyTuple_GET_ITEM(t, i); + + if (element == NULL) + goto finally; + if (save(self, element, 0) < 0) + goto finally; + } + res = 0; finally: - return res; + return res; } /* Tuples are ubiquitous in the pickle protocols, so many techniques are @@ -1405,110 +1415,110 @@ * magic so that it works in all cases. IOW, this is a long routine. */ static int -save_tuple(Picklerobject *self, PyObject *args) +save_tuple(Picklerobject * self, PyObject * args) { - PyObject *py_tuple_id = NULL; - int len, i; - int res = -1; - - static char tuple = TUPLE; - static char pop = POP; - static char pop_mark = POP_MARK; - static char len2opcode[] = {EMPTY_TUPLE, TUPLE1, TUPLE2, TUPLE3}; - - if ((len = PyTuple_Size(args)) < 0) - goto finally; - - if (len == 0) { - char c_str[2]; - - if (self->proto) { - c_str[0] = EMPTY_TUPLE; - len = 1; - } - else { - c_str[0] = MARK; - c_str[1] = TUPLE; - len = 2; - } - if (self->write_func(self, c_str, len) >= 0) - res = 0; - /* Don't memoize an empty tuple. */ - goto finally; + PyObject *py_tuple_id = NULL; + int len, i; + int res = -1; + + static char tuple = TUPLE; + static char pop = POP; + static char pop_mark = POP_MARK; + static char len2opcode[] = { EMPTY_TUPLE, TUPLE1, TUPLE2, TUPLE3 }; + + if ((len = PyTuple_Size(args)) < 0) + goto finally; + + if (len == 0) { + char c_str[2]; + + if (self->proto) { + c_str[0] = EMPTY_TUPLE; + len = 1; } + else { + c_str[0] = MARK; + c_str[1] = TUPLE; + len = 2; + } + if (self->write_func(self, c_str, len) >= 0) + res = 0; + /* Don't memoize an empty tuple. */ + goto finally; + } + + /* A non-empty tuple. */ + + /* id(tuple) isn't in the memo now. If it shows up there after + * saving the tuple elements, the tuple must be recursive, in + * which case we'll pop everything we put on the stack, and fetch + * its value from the memo. + */ + py_tuple_id = PyLong_FromVoidPtr(args); + if (py_tuple_id == NULL) + goto finally; - /* A non-empty tuple. */ - - /* id(tuple) isn't in the memo now. If it shows up there after - * saving the tuple elements, the tuple must be recursive, in - * which case we'll pop everything we put on the stack, and fetch - * its value from the memo. - */ - py_tuple_id = PyLong_FromVoidPtr(args); - if (py_tuple_id == NULL) - goto finally; - - if (len <= 3 && self->proto >= 2) { - /* Use TUPLE{1,2,3} opcodes. */ - if (store_tuple_elements(self, args, len) < 0) - goto finally; - if (PyDict_GetItem(self->memo, py_tuple_id)) { - /* pop the len elements */ - for (i = 0; i < len; ++i) - if (self->write_func(self, &pop, 1) < 0) - goto finally; - /* fetch from memo */ - if (get(self, py_tuple_id) < 0) - goto finally; - res = 0; - goto finally; - } - /* Not recursive. */ - if (self->write_func(self, len2opcode + len, 1) < 0) - goto finally; - goto memoize; - } - - /* proto < 2 and len > 0, or proto >= 2 and len > 3. - * Generate MARK elt1 elt2 ... TUPLE - */ - if (self->write_func(self, &MARKv, 1) < 0) - goto finally; - + if (len <= 3 && self->proto >= 2) { + /* Use TUPLE{1,2,3} opcodes. */ if (store_tuple_elements(self, args, len) < 0) - goto finally; - + goto finally; if (PyDict_GetItem(self->memo, py_tuple_id)) { - /* pop the stack stuff we pushed */ - if (self->bin) { - if (self->write_func(self, &pop_mark, 1) < 0) - goto finally; - } - else { - /* Note that we pop one more than len, to remove - * the MARK too. - */ - for (i = 0; i <= len; i++) - if (self->write_func(self, &pop, 1) < 0) - goto finally; - } - /* fetch from memo */ - if (get(self, py_tuple_id) >= 0) - res = 0; + /* pop the len elements */ + for (i = 0; i < len; ++i) + if (self->write_func(self, &pop, 1) < 0) + goto finally; + /* fetch from memo */ + if (get(self, py_tuple_id) < 0) goto finally; + res = 0; + goto finally; } - /* Not recursive. */ - if (self->write_func(self, &tuple, 1) < 0) + if (self->write_func(self, len2opcode + len, 1) < 0) + goto finally; + goto memoize; + } + + /* proto < 2 and len > 0, or proto >= 2 and len > 3. + * Generate MARK elt1 elt2 ... TUPLE + */ + if (self->write_func(self, &MARKv, 1) < 0) + goto finally; + + if (store_tuple_elements(self, args, len) < 0) + goto finally; + + if (PyDict_GetItem(self->memo, py_tuple_id)) { + /* pop the stack stuff we pushed */ + if (self->bin) { + if (self->write_func(self, &pop_mark, 1) < 0) goto finally; + } + else { + /* Note that we pop one more than len, to remove + * the MARK too. + */ + for (i = 0; i <= len; i++) + if (self->write_func(self, &pop, 1) < 0) + goto finally; + } + /* fetch from memo */ + if (get(self, py_tuple_id) >= 0) + res = 0; + goto finally; + } + + /* Not recursive. */ + if (self->write_func(self, &tuple, 1) < 0) + goto finally; memoize: - if (put(self, args) >= 0) - res = 0; + if (put(self, args) >= 0) + res = 0; finally: - Py_XDECREF(py_tuple_id); - return res; + Py_XDECREF(py_tuple_id); + return res; } /* iter is an iterator giving items, and we batch up chunks of @@ -1518,130 +1528,130 @@ * Returns 0 on success, <0 on error. */ static int -batch_list(Picklerobject *self, PyObject *iter) +batch_list(Picklerobject * self, PyObject * iter) { - PyObject *obj; - PyObject *slice[BATCHSIZE]; - int i, n; - - static char append = APPEND; - static char appends = APPENDS; - - assert(iter != NULL); - - if (self->proto == 0) { - /* APPENDS isn't available; do one at a time. */ - for (;;) { - obj = PyIter_Next(iter); - if (obj == NULL) { - if (PyErr_Occurred()) - return -1; - break; - } - i = save(self, obj, 0); - Py_DECREF(obj); - if (i < 0) - return -1; - if (self->write_func(self, &append, 1) < 0) - return -1; - } - return 0; + PyObject *obj; + PyObject *slice[BATCHSIZE]; + int i, n; + + static char append = APPEND; + static char appends = APPENDS; + + assert(iter != NULL); + + if (self->proto == 0) { + /* APPENDS isn't available; do one at a time. */ + for (;;) { + obj = PyIter_Next(iter); + if (obj == NULL) { + if (PyErr_Occurred()) + return -1; + break; + } + i = save(self, obj, 0); + Py_DECREF(obj); + if (i < 0) + return -1; + if (self->write_func(self, &append, 1) < 0) + return -1; } - - /* proto > 0: write in batches of BATCHSIZE. */ - do { - /* Get next group of (no more than) BATCHSIZE elements. */ - for (n = 0; n < BATCHSIZE; ++n) { - obj = PyIter_Next(iter); - if (obj == NULL) { - if (PyErr_Occurred()) - goto BatchFailed; - break; - } - slice[n] = obj; - } - - if (n > 1) { - /* Pump out MARK, slice[0:n], APPENDS. */ - if (self->write_func(self, &MARKv, 1) < 0) - goto BatchFailed; - for (i = 0; i < n; ++i) { - if (save(self, slice[i], 0) < 0) - goto BatchFailed; - } - if (self->write_func(self, &appends, 1) < 0) - goto BatchFailed; - } - else if (n == 1) { - if (save(self, slice[0], 0) < 0) - goto BatchFailed; - if (self->write_func(self, &append, 1) < 0) - goto BatchFailed; - } - - for (i = 0; i < n; ++i) { - Py_DECREF(slice[i]); - } - } while (n == BATCHSIZE); return 0; + } -BatchFailed: - while (--n >= 0) { - Py_DECREF(slice[n]); - } - return -1; -} - -static int -save_list(Picklerobject *self, PyObject *args) -{ - int res = -1; - char s[3]; - int len; - PyObject *iter; - - if (self->fast && !fast_save_enter(self, args)) - goto finally; - - /* Create an empty list. */ - if (self->bin) { - s[0] = EMPTY_LIST; - len = 1; - } - else { - s[0] = MARK; - s[1] = LIST; - len = 2; + /* proto > 0: write in batches of BATCHSIZE. */ + do { + /* Get next group of (no more than) BATCHSIZE elements. */ + for (n = 0; n < BATCHSIZE; ++n) { + obj = PyIter_Next(iter); + if (obj == NULL) { + if (PyErr_Occurred()) + goto BatchFailed; + break; + } + slice[n] = obj; } - if (self->write_func(self, s, len) < 0) - goto finally; - - /* Get list length, and bow out early if empty. */ - if ((len = PyList_Size(args)) < 0) - goto finally; - - /* Memoize. */ - if (len == 0) { - if (put(self, args) >= 0) - res = 0; - goto finally; - } - if (put2(self, args) < 0) - goto finally; + if (n > 1) { + /* Pump out MARK, slice[0:n], APPENDS. */ + if (self->write_func(self, &MARKv, 1) < 0) + goto BatchFailed; + for (i = 0; i < n; ++i) { + if (save(self, slice[i], 0) < 0) + goto BatchFailed; + } + if (self->write_func(self, &appends, 1) < 0) + goto BatchFailed; + } + else if (n == 1) { + if (save(self, slice[0], 0) < 0) + goto BatchFailed; + if (self->write_func(self, &append, 1) < 0) + goto BatchFailed; + } + + for (i = 0; i < n; ++i) { + Py_DECREF(slice[i]); + } + } while (n == BATCHSIZE); + return 0; + + BatchFailed: + while (--n >= 0) { + Py_DECREF(slice[n]); + } + return -1; +} + +static int +save_list(Picklerobject * self, PyObject * args) +{ + int res = -1; + char s[3]; + int len; + PyObject *iter; + + if (self->fast && !fast_save_enter(self, args)) + goto finally; + + /* Create an empty list. */ + if (self->bin) { + s[0] = EMPTY_LIST; + len = 1; + } + else { + s[0] = MARK; + s[1] = LIST; + len = 2; + } + + if (self->write_func(self, s, len) < 0) + goto finally; + + /* Get list length, and bow out early if empty. */ + if ((len = PyList_Size(args)) < 0) + goto finally; - /* Materialize the list elements. */ - iter = PyObject_GetIter(args); - if (iter == NULL) - goto finally; - res = batch_list(self, iter); - Py_DECREF(iter); + /* Memoize. */ + if (len == 0) { + if (put(self, args) >= 0) + res = 0; + goto finally; + } + if (put2(self, args) < 0) + goto finally; + + /* Materialize the list elements. */ + iter = PyObject_GetIter(args); + if (iter == NULL) + goto finally; + res = batch_list(self, iter); + Py_DECREF(iter); finally: - if (self->fast && !fast_save_leave(self, args)) - res = -1; + if (self->fast && !fast_save_leave(self, args)) + res = -1; - return res; + return res; } @@ -1657,1213 +1667,1196 @@ * ugly to bear. */ static int -batch_dict(Picklerobject *self, PyObject *iter) +batch_dict(Picklerobject * self, PyObject * iter) { - PyObject *p; - PyObject *slice[BATCHSIZE]; - int i, n; - - static char setitem = SETITEM; - static char setitems = SETITEMS; - - assert(iter != NULL); - - if (self->proto == 0) { - /* SETITEMS isn't available; do one at a time. */ - for (;;) { - p = PyIter_Next(iter); - if (p == NULL) { - if (PyErr_Occurred()) - return -1; - break; - } - if (!PyTuple_Check(p) || PyTuple_Size(p) != 2) { - PyErr_SetString(PyExc_TypeError, "dict items " - "iterator must return 2-tuples"); - return -1; - } - i = save(self, PyTuple_GET_ITEM(p, 0), 0); - if (i >= 0) - i = save(self, PyTuple_GET_ITEM(p, 1), 0); - Py_DECREF(p); - if (i < 0) - return -1; - if (self->write_func(self, &setitem, 1) < 0) - return -1; - } - return 0; + PyObject *p; + PyObject *slice[BATCHSIZE]; + int i, n; + + static char setitem = SETITEM; + static char setitems = SETITEMS; + + assert(iter != NULL); + + if (self->proto == 0) { + /* SETITEMS isn't available; do one at a time. */ + for (;;) { + p = PyIter_Next(iter); + if (p == NULL) { + if (PyErr_Occurred()) + return -1; + break; + } + if (!PyTuple_Check(p) || PyTuple_Size(p) != 2) { + PyErr_SetString(PyExc_TypeError, "dict items " + "iterator must return 2-tuples"); + return -1; + } + i = save(self, PyTuple_GET_ITEM(p, 0), 0); + if (i >= 0) + i = save(self, PyTuple_GET_ITEM(p, 1), 0); + Py_DECREF(p); + if (i < 0) + return -1; + if (self->write_func(self, &setitem, 1) < 0) + return -1; } - - /* proto > 0: write in batches of BATCHSIZE. */ - do { - /* Get next group of (no more than) BATCHSIZE elements. */ - for (n = 0; n < BATCHSIZE; ++n) { - p = PyIter_Next(iter); - if (p == NULL) { - if (PyErr_Occurred()) - goto BatchFailed; - break; - } - if (!PyTuple_Check(p) || PyTuple_Size(p) != 2) { - PyErr_SetString(PyExc_TypeError, "dict items " - "iterator must return 2-tuples"); - goto BatchFailed; - } - slice[n] = p; - } - - if (n > 1) { - /* Pump out MARK, slice[0:n], SETITEMS. */ - if (self->write_func(self, &MARKv, 1) < 0) - goto BatchFailed; - for (i = 0; i < n; ++i) { - p = slice[i]; - if (save(self, PyTuple_GET_ITEM(p, 0), 0) < 0) - goto BatchFailed; - if (save(self, PyTuple_GET_ITEM(p, 1), 0) < 0) - goto BatchFailed; - } - if (self->write_func(self, &setitems, 1) < 0) - goto BatchFailed; - } - else if (n == 1) { - p = slice[0]; - if (save(self, PyTuple_GET_ITEM(p, 0), 0) < 0) - goto BatchFailed; - if (save(self, PyTuple_GET_ITEM(p, 1), 0) < 0) - goto BatchFailed; - if (self->write_func(self, &setitem, 1) < 0) - goto BatchFailed; - } - - for (i = 0; i < n; ++i) { - Py_DECREF(slice[i]); - } - } while (n == BATCHSIZE); return 0; + } + + /* proto > 0: write in batches of BATCHSIZE. */ + do { + /* Get next group of (no more than) BATCHSIZE elements. */ + for (n = 0; n < BATCHSIZE; ++n) { + p = PyIter_Next(iter); + if (p == NULL) { + if (PyErr_Occurred()) + goto BatchFailed; + break; + } + if (!PyTuple_Check(p) || PyTuple_Size(p) != 2) { + PyErr_SetString(PyExc_TypeError, "dict items " + "iterator must return 2-tuples"); + goto BatchFailed; + } + slice[n] = p; + } + + if (n > 1) { + /* Pump out MARK, slice[0:n], SETITEMS. */ + if (self->write_func(self, &MARKv, 1) < 0) + goto BatchFailed; + for (i = 0; i < n; ++i) { + p = slice[i]; + if (save(self, PyTuple_GET_ITEM(p, 0), 0) < 0) + goto BatchFailed; + if (save(self, PyTuple_GET_ITEM(p, 1), 0) < 0) + goto BatchFailed; + } + if (self->write_func(self, &setitems, 1) < 0) + goto BatchFailed; + } + else if (n == 1) { + p = slice[0]; + if (save(self, PyTuple_GET_ITEM(p, 0), 0) < 0) + goto BatchFailed; + if (save(self, PyTuple_GET_ITEM(p, 1), 0) < 0) + goto BatchFailed; + if (self->write_func(self, &setitem, 1) < 0) + goto BatchFailed; + } + + for (i = 0; i < n; ++i) { + Py_DECREF(slice[i]); + } + } while (n == BATCHSIZE); + return 0; + + BatchFailed: + while (--n >= 0) { + Py_DECREF(slice[n]); + } + return -1; +} + +static int +save_dict(Picklerobject * self, PyObject * args) +{ + int res = -1; + char s[3]; + int len; + PyObject *items, *iter; + + if (self->fast && !fast_save_enter(self, args)) + goto finally; + + /* Create an empty dict. */ + if (self->bin) { + s[0] = EMPTY_DICT; + len = 1; + } + else { + s[0] = MARK; + s[1] = DICT; + len = 2; + } + + if (self->write_func(self, s, len) < 0) + goto finally; + + /* Get dict size, and bow out early if empty. */ + if ((len = PyDict_Size(args)) < 0) + goto finally; -BatchFailed: - while (--n >= 0) { - Py_DECREF(slice[n]); - } - return -1; -} - -static int -save_dict(Picklerobject *self, PyObject *args) -{ - int res = -1; - char s[3]; - int len; - PyObject *items, *iter; - - if (self->fast && !fast_save_enter(self, args)) - goto finally; - - /* Create an empty dict. */ - if (self->bin) { - s[0] = EMPTY_DICT; - len = 1; - } - else { - s[0] = MARK; - s[1] = DICT; - len = 2; - } - - if (self->write_func(self, s, len) < 0) - goto finally; - - /* Get dict size, and bow out early if empty. */ - if ((len = PyDict_Size(args)) < 0) - goto finally; - - if (len == 0) { - if (put(self, args) >= 0) - res = 0; - goto finally; - } - if (put2(self, args) < 0) - goto finally; - - /* Materialize the dict items. */ - items = PyObject_CallMethod(args, "items", "()"); - if (items == NULL) - goto finally; - iter = PyObject_GetIter(items); - Py_DECREF(items); - if (iter == NULL) - goto finally; - res = batch_dict(self, iter); - Py_DECREF(iter); + if (len == 0) { + if (put(self, args) >= 0) + res = 0; + goto finally; + } + if (put2(self, args) < 0) + goto finally; + + /* Materialize the dict items. */ + items = PyObject_CallMethod(args, "items", "()"); + if (items == NULL) + goto finally; + iter = PyObject_GetIter(items); + Py_DECREF(items); + if (iter == NULL) + goto finally; + res = batch_dict(self, iter); + Py_DECREF(iter); finally: - if (self->fast && !fast_save_leave(self, args)) - res = -1; + if (self->fast && !fast_save_leave(self, args)) + res = -1; - return res; + return res; } static int -save_global(Picklerobject *self, PyObject *args, PyObject *name) -{ - PyObject *global_name = 0, *module = 0, *mod = 0, *klass = 0; - char *name_str, *module_str; - int module_size, name_size, res = -1; - - static char global = GLOBAL; +save_global(Picklerobject * self, PyObject * args, PyObject * name) +{ + PyObject *global_name = 0, *module = 0, *mod = 0, *klass = 0; + char *name_str, *module_str; + int module_size, name_size, res = -1; + + static char global = GLOBAL; + + if (name) { + global_name = name; + Py_INCREF(global_name); + } + else { + if (!(global_name = PyObject_GetAttr(args, __name___str))) + goto finally; + } + + if (!(module = whichmodule(args, global_name))) + goto finally; + + if ((module_size = PyString_Size(module)) < 0 || + (name_size = PyString_Size(global_name)) < 0) + goto finally; + + module_str = PyString_AS_STRING((PyStringObject *) module); + name_str = PyString_AS_STRING((PyStringObject *) global_name); + + /* XXX This can be doing a relative import. Clearly it shouldn't, + * but I don't know how to stop it. :-( */ + mod = PyImport_ImportModule(module_str); + if (mod == NULL) { + cPickle_ErrFormat(PicklingError, + "Can't pickle %s: import of module %s " + "failed", "OS", args, module); + goto finally; + } + klass = PyObject_GetAttrString(mod, name_str); + if (klass == NULL) { + cPickle_ErrFormat(PicklingError, + "Can't pickle %s: attribute lookup %s.%s " + "failed", "OSS", args, module, global_name); + goto finally; + } + if (klass != args) { + Py_DECREF(klass); + cPickle_ErrFormat(PicklingError, + "Can't pickle %s: it's not the same object " + "as %s.%s", "OSS", args, module, global_name); + goto finally; + } + Py_DECREF(klass); + + if (self->proto >= 2) { + /* See whether this is in the extension registry, and if + * so generate an EXT opcode. + */ + PyObject *py_code; /* extension code as Python object */ + long code; /* extension code as C value */ + char c_str[5]; + int n; - if (name) { - global_name = name; - Py_INCREF(global_name); + PyTuple_SET_ITEM(two_tuple, 0, module); + PyTuple_SET_ITEM(two_tuple, 1, global_name); + py_code = PyDict_GetItem(extension_registry, two_tuple); + if (py_code == NULL) + goto gen_global; /* not registered */ + + /* Verify py_code has the right type and value. */ + if (!PyInt_Check(py_code)) { + cPickle_ErrFormat(PicklingError, "Can't pickle %s: " + "extension code %s isn't an integer", + "OO", args, py_code); + goto finally; + } + code = PyInt_AS_LONG(py_code); + if (code <= 0 || code > 0x7fffffffL) { + cPickle_ErrFormat(PicklingError, "Can't pickle %s: " + "extension code %ld is out of range", + "Ol", args, code); + goto finally; + } + + /* Generate an EXT opcode. */ + if (code <= 0xff) { + c_str[0] = EXT1; + c_str[1] = (char) code; + n = 2; + } + else if (code <= 0xffff) { + c_str[0] = EXT2; + c_str[1] = (char) (code & 0xff); + c_str[2] = (char) ((code >> 8) & 0xff); + n = 3; } else { - if (!( global_name = PyObject_GetAttr(args, __name___str))) - goto finally; - } - - if (!( module = whichmodule(args, global_name))) - goto finally; - - if ((module_size = PyString_Size(module)) < 0 || - (name_size = PyString_Size(global_name)) < 0) - goto finally; - - module_str = PyString_AS_STRING((PyStringObject *)module); - name_str = PyString_AS_STRING((PyStringObject *)global_name); - - /* XXX This can be doing a relative import. Clearly it shouldn't, - but I don't know how to stop it. :-( */ - mod = PyImport_ImportModule(module_str); - if (mod == NULL) { - cPickle_ErrFormat(PicklingError, - "Can't pickle %s: import of module %s " - "failed", - "OS", args, module); - goto finally; - } - klass = PyObject_GetAttrString(mod, name_str); - if (klass == NULL) { - cPickle_ErrFormat(PicklingError, - "Can't pickle %s: attribute lookup %s.%s " - "failed", - "OSS", args, module, global_name); - goto finally; + c_str[0] = EXT4; + c_str[1] = (char) (code & 0xff); + c_str[2] = (char) ((code >> 8) & 0xff); + c_str[3] = (char) ((code >> 16) & 0xff); + c_str[4] = (char) ((code >> 24) & 0xff); + n = 5; } - if (klass != args) { - Py_DECREF(klass); - cPickle_ErrFormat(PicklingError, - "Can't pickle %s: it's not the same object " - "as %s.%s", - "OSS", args, module, global_name); - goto finally; - } - Py_DECREF(klass); - - if (self->proto >= 2) { - /* See whether this is in the extension registry, and if - * so generate an EXT opcode. - */ - PyObject *py_code; /* extension code as Python object */ - long code; /* extension code as C value */ - char c_str[5]; - int n; - - PyTuple_SET_ITEM(two_tuple, 0, module); - PyTuple_SET_ITEM(two_tuple, 1, global_name); - py_code = PyDict_GetItem(extension_registry, two_tuple); - if (py_code == NULL) - goto gen_global; /* not registered */ - - /* Verify py_code has the right type and value. */ - if (!PyInt_Check(py_code)) { - cPickle_ErrFormat(PicklingError, "Can't pickle %s: " - "extension code %s isn't an integer", - "OO", args, py_code); - goto finally; - } - code = PyInt_AS_LONG(py_code); - if (code <= 0 || code > 0x7fffffffL) { - cPickle_ErrFormat(PicklingError, "Can't pickle %s: " - "extension code %ld is out of range", - "Ol", args, code); - goto finally; - } - - /* Generate an EXT opcode. */ - if (code <= 0xff) { - c_str[0] = EXT1; - c_str[1] = (char)code; - n = 2; - } - else if (code <= 0xffff) { - c_str[0] = EXT2; - c_str[1] = (char)(code & 0xff); - c_str[2] = (char)((code >> 8) & 0xff); - n = 3; - } - else { - c_str[0] = EXT4; - c_str[1] = (char)(code & 0xff); - c_str[2] = (char)((code >> 8) & 0xff); - c_str[3] = (char)((code >> 16) & 0xff); - c_str[4] = (char)((code >> 24) & 0xff); - n = 5; - } - if (self->write_func(self, c_str, n) >= 0) - res = 0; - goto finally; /* and don't memoize */ - } + if (self->write_func(self, c_str, n) >= 0) + res = 0; + goto finally; /* and don't memoize */ + } gen_global: - if (self->write_func(self, &global, 1) < 0) - goto finally; + if (self->write_func(self, &global, 1) < 0) + goto finally; - if (self->write_func(self, module_str, module_size) < 0) - goto finally; + if (self->write_func(self, module_str, module_size) < 0) + goto finally; - if (self->write_func(self, "\n", 1) < 0) - goto finally; + if (self->write_func(self, "\n", 1) < 0) + goto finally; - if (self->write_func(self, name_str, name_size) < 0) - goto finally; + if (self->write_func(self, name_str, name_size) < 0) + goto finally; - if (self->write_func(self, "\n", 1) < 0) - goto finally; + if (self->write_func(self, "\n", 1) < 0) + goto finally; - if (put(self, args) < 0) - goto finally; + if (put(self, args) < 0) + goto finally; - res = 0; + res = 0; finally: - Py_XDECREF(module); - Py_XDECREF(global_name); - Py_XDECREF(mod); + Py_XDECREF(module); + Py_XDECREF(global_name); + Py_XDECREF(mod); - return res; + return res; } static int -save_pers(Picklerobject *self, PyObject *args, PyObject *f) +save_pers(Picklerobject * self, PyObject * args, PyObject * f) { - PyObject *pid = 0; - int size, res = -1; + PyObject *pid = 0; + int size, res = -1; - static char persid = PERSID, binpersid = BINPERSID; + static char persid = PERSID, binpersid = BINPERSID; - Py_INCREF(args); - ARG_TUP(self, args); - if (self->arg) { - pid = PyObject_Call(f, self->arg, NULL); - FREE_ARG_TUP(self); - } - if (! pid) return -1; + Py_INCREF(args); + ARG_TUP(self, args); + if (self->arg) { + pid = PyObject_Call(f, self->arg, NULL); + FREE_ARG_TUP(self); + } + if (!pid) + return -1; - if (pid != Py_None) { - if (!self->bin) { - if (!PyString_Check(pid)) { - PyErr_SetString(PicklingError, - "persistent id must be string"); - goto finally; - } - - if (self->write_func(self, &persid, 1) < 0) - goto finally; - - if ((size = PyString_Size(pid)) < 0) - goto finally; - - if (self->write_func(self, - PyString_AS_STRING( - (PyStringObject *)pid), - size) < 0) - goto finally; + if (pid != Py_None) { + if (!self->bin) { + if (!PyString_Check(pid)) { + PyErr_SetString(PicklingError, "persistent id must be string"); + goto finally; + } - if (self->write_func(self, "\n", 1) < 0) - goto finally; + if (self->write_func(self, &persid, 1) < 0) + goto finally; - res = 1; - goto finally; - } - else if (save(self, pid, 1) >= 0) { - if (self->write_func(self, &binpersid, 1) < 0) - res = -1; - else - res = 1; - } + if ((size = PyString_Size(pid)) < 0) + goto finally; + + if (self->write_func(self, + PyString_AS_STRING((PyStringObject *) pid), + size) < 0) + goto finally; + if (self->write_func(self, "\n", 1) < 0) goto finally; + + res = 1; + goto finally; + } + else if (save(self, pid, 1) >= 0) { + if (self->write_func(self, &binpersid, 1) < 0) + res = -1; + else + res = 1; } - res = 0; + goto finally; + } + + res = 0; finally: - Py_XDECREF(pid); + Py_XDECREF(pid); - return res; + return res; } /* We're saving ob, and args is the 2-thru-5 tuple returned by the * appropriate __reduce__ method for ob. */ static int -save_reduce(Picklerobject *self, PyObject *args, PyObject *ob) +save_reduce(Picklerobject * self, PyObject * args, PyObject * ob) { - PyObject *callable; - PyObject *argtup; - PyObject *state = NULL; - PyObject *listitems = NULL; - PyObject *dictitems = NULL; - - int use_newobj = self->proto >= 2; - - static char reduce = REDUCE; - static char build = BUILD; - static char newobj = NEWOBJ; - - if (! PyArg_UnpackTuple(args, "save_reduce", 2, 5, - &callable, - &argtup, - &state, - &listitems, - &dictitems)) - return -1; + PyObject *callable; + PyObject *argtup; + PyObject *state = NULL; + PyObject *listitems = NULL; + PyObject *dictitems = NULL; + + int use_newobj = self->proto >= 2; + + static char reduce = REDUCE; + static char build = BUILD; + static char newobj = NEWOBJ; - if (!PyTuple_Check(argtup)) { - PyErr_SetString(PicklingError, - "args from reduce() should be a tuple"); - return -1; - } - - if (state == Py_None) - state = NULL; - if (listitems == Py_None) - listitems = NULL; - if (dictitems == Py_None) - dictitems = NULL; - - /* Protocol 2 special case: if callable's name is __newobj__, use - * NEWOBJ. This consumes a lot of code. - */ - if (use_newobj) { - PyObject *temp = PyObject_GetAttr(callable, __name___str); - - if (temp == NULL) { - if (PyErr_ExceptionMatches(PyExc_AttributeError)) - PyErr_Clear(); - else - return -1; - use_newobj = 0; - } - else { - use_newobj = PyString_Check(temp) && - strcmp(PyString_AS_STRING(temp), - "__newobj__") == 0; - Py_DECREF(temp); - } - } - if (use_newobj) { - PyObject *cls; - PyObject *newargtup; - int n, i; - - /* Sanity checks. */ - n = PyTuple_Size(argtup); - if (n < 1) { - PyErr_SetString(PicklingError, "__newobj__ arglist " - "is empty"); - return -1; - } + if (!PyArg_UnpackTuple(args, "save_reduce", 2, 5, + &callable, &argtup, &state, &listitems, &dictitems)) + return -1; - cls = PyTuple_GET_ITEM(argtup, 0); - if (! PyObject_HasAttrString(cls, "__new__")) { - PyErr_SetString(PicklingError, "args[0] from " - "__newobj__ args has no __new__"); - return -1; - } + if (!PyTuple_Check(argtup)) { + PyErr_SetString(PicklingError, "args from reduce() should be a tuple"); + return -1; + } - /* XXX How could ob be NULL? */ - if (ob != NULL) { - PyObject *ob_dot_class; - - ob_dot_class = PyObject_GetAttr(ob, __class___str); - if (ob_dot_class == NULL) { - if (PyErr_ExceptionMatches( - PyExc_AttributeError)) - PyErr_Clear(); - else - return -1; - } - i = ob_dot_class != cls; /* true iff a problem */ - Py_XDECREF(ob_dot_class); - if (i) { - PyErr_SetString(PicklingError, "args[0] from " - "__newobj__ args has the wrong class"); - return -1; - } - } + if (state == Py_None) + state = NULL; + if (listitems == Py_None) + listitems = NULL; + if (dictitems == Py_None) + dictitems = NULL; + + /* Protocol 2 special case: if callable's name is __newobj__, use + * NEWOBJ. This consumes a lot of code. + */ + if (use_newobj) { + PyObject *temp = PyObject_GetAttr(callable, __name___str); - /* Save the class and its __new__ arguments. */ - if (save(self, cls, 0) < 0) - return -1; - - newargtup = PyTuple_New(n-1); /* argtup[1:] */ - if (newargtup == NULL) - return -1; - for (i = 1; i < n; ++i) { - PyObject *temp = PyTuple_GET_ITEM(argtup, i); - Py_INCREF(temp); - PyTuple_SET_ITEM(newargtup, i-1, temp); - } - i = save(self, newargtup, 0) < 0; - Py_DECREF(newargtup); - if (i < 0) - return -1; - - /* Add NEWOBJ opcode. */ - if (self->write_func(self, &newobj, 1) < 0) - return -1; + if (temp == NULL) { + if (PyErr_ExceptionMatches(PyExc_AttributeError)) + PyErr_Clear(); + else + return -1; + use_newobj = 0; } else { - /* Not using NEWOBJ. */ - if (save(self, callable, 0) < 0 || - save(self, argtup, 0) < 0 || - self->write_func(self, &reduce, 1) < 0) - return -1; + use_newobj = PyString_Check(temp) && + strcmp(PyString_AS_STRING(temp), "__newobj__") == 0; + Py_DECREF(temp); + } + } + if (use_newobj) { + PyObject *cls; + PyObject *newargtup; + int n, i; + + /* Sanity checks. */ + n = PyTuple_Size(argtup); + if (n < 1) { + PyErr_SetString(PicklingError, "__newobj__ arglist " "is empty"); + return -1; + } + + cls = PyTuple_GET_ITEM(argtup, 0); + if (!PyObject_HasAttrString(cls, "__new__")) { + PyErr_SetString(PicklingError, "args[0] from " + "__newobj__ args has no __new__"); + return -1; } - /* Memoize. */ - /* XXX How can ob be NULL? */ + /* XXX How could ob be NULL? */ if (ob != NULL) { - if (state && !PyDict_Check(state)) { - if (put2(self, ob) < 0) - return -1; - } - else if (put(self, ob) < 0) - return -1; + PyObject *ob_dot_class; + + ob_dot_class = PyObject_GetAttr(ob, __class___str); + if (ob_dot_class == NULL) { + if (PyErr_ExceptionMatches(PyExc_AttributeError)) + PyErr_Clear(); + else + return -1; + } + i = ob_dot_class != cls; /* true iff a problem */ + Py_XDECREF(ob_dot_class); + if (i) { + PyErr_SetString(PicklingError, "args[0] from " + "__newobj__ args has the wrong class"); + return -1; + } + } + + /* Save the class and its __new__ arguments. */ + if (save(self, cls, 0) < 0) + return -1; + + newargtup = PyTuple_New(n - 1); /* argtup[1:] */ + if (newargtup == NULL) + return -1; + for (i = 1; i < n; ++i) { + PyObject *temp = PyTuple_GET_ITEM(argtup, i); + Py_INCREF(temp); + PyTuple_SET_ITEM(newargtup, i - 1, temp); } + i = save(self, newargtup, 0) < 0; + Py_DECREF(newargtup); + if (i < 0) + return -1; + /* Add NEWOBJ opcode. */ + if (self->write_func(self, &newobj, 1) < 0) + return -1; + } + else { + /* Not using NEWOBJ. */ + if (save(self, callable, 0) < 0 || + save(self, argtup, 0) < 0 || + self->write_func(self, &reduce, 1) < 0) + return -1; + } + + /* Memoize. */ + /* XXX How can ob be NULL? */ + if (ob != NULL) { + if (state && !PyDict_Check(state)) { + if (put2(self, ob) < 0) + return -1; + } + else if (put(self, ob) < 0) + return -1; + } - if (listitems && batch_list(self, listitems) < 0) - return -1; - if (dictitems && batch_dict(self, dictitems) < 0) - return -1; + if (listitems && batch_list(self, listitems) < 0) + return -1; - if (state) { - if (save(self, state, 0) < 0 || - self->write_func(self, &build, 1) < 0) - return -1; - } + if (dictitems && batch_dict(self, dictitems) < 0) + return -1; - return 0; + if (state) { + if (save(self, state, 0) < 0 || self->write_func(self, &build, 1) < 0) + return -1; + } + + return 0; } static int -save(Picklerobject *self, PyObject *args, int pers_save) +save(Picklerobject * self, PyObject * args, int pers_save) { - PyTypeObject *type; - PyObject *py_ob_id = 0, *__reduce__ = 0, *t = 0; - PyObject *arg_tup; - int res = -1; - int tmp, size; - - if (self->nesting++ > Py_GetRecursionLimit()){ - PyErr_SetString(PyExc_RuntimeError, - "maximum recursion depth exceeded"); - goto finally; - } + PyTypeObject *type; + PyObject *py_ob_id = 0, *__reduce__ = 0, *t = 0; + PyObject *arg_tup; + int res = -1; + int tmp, size; - if (!pers_save && self->pers_func) { - if ((tmp = save_pers(self, args, self->pers_func)) != 0) { - res = tmp; - goto finally; - } - } + if (self->nesting++ > Py_GetRecursionLimit()) { + PyErr_SetString(PyExc_RuntimeError, + "maximum recursion depth exceeded"); + goto finally; + } - if (args == Py_None) { - res = save_none(self, args); - goto finally; + if (!pers_save && self->pers_func) { + if ((tmp = save_pers(self, args, self->pers_func)) != 0) { + res = tmp; + goto finally; } + } - type = args->ob_type; - - switch (type->tp_name[0]) { - case 'b': - if (args == Py_False || args == Py_True) { - res = save_bool(self, args); - goto finally; - } - break; - case 'i': - if (type == &PyLong_Type) { - res = save_long(self, args); - goto finally; - } - break; - - case 'f': - if (type == &PyFloat_Type) { - res = save_float(self, args); - goto finally; - } - break; - - case 't': - if (type == &PyTuple_Type && PyTuple_Size(args) == 0) { - res = save_tuple(self, args); - goto finally; - } - break; + if (args == Py_None) { + res = save_none(self, args); + goto finally; + } - case 's': - if ((type == &PyString_Type) && (PyString_GET_SIZE(args) < 2)) { - res = save_string(self, args, 0); - goto finally; - } + type = args->ob_type; -#ifdef Py_USING_UNICODE - case 'u': - if ((type == &PyUnicode_Type) && (PyString_GET_SIZE(args) < 2)) { - res = save_unicode(self, args, 0); - goto finally; - } -#endif + switch (type->tp_name[0]) { + case 'b': + if (args == Py_False || args == Py_True) { + res = save_bool(self, args); + goto finally; } + break; + case 'i': + if (type == &PyLong_Type) { + res = save_long(self, args); + goto finally; + } + break; - if (args->ob_refcnt > 1) { - if (!( py_ob_id = PyLong_FromVoidPtr(args))) - goto finally; - - if (PyDict_GetItem(self->memo, py_ob_id)) { - if (get(self, py_ob_id) < 0) - goto finally; + case 'f': + if (type == &PyFloat_Type) { + res = save_float(self, args); + goto finally; + } + break; - res = 0; - goto finally; - } + case 't': + if (type == &PyTuple_Type && PyTuple_Size(args) == 0) { + res = save_tuple(self, args); + goto finally; } + break; - switch (type->tp_name[0]) { - case 's': - if (type == &PyString_Type) { - res = save_string(self, args, 1); - goto finally; - } - break; + case 's': + if ((type == &PyString_Type) && (PyString_GET_SIZE(args) < 2)) { + res = save_string(self, args, 0); + goto finally; + } #ifdef Py_USING_UNICODE - case 'u': - if (type == &PyUnicode_Type) { - res = save_unicode(self, args, 1); - goto finally; - } - break; + case 'u': + if ((type == &PyUnicode_Type) && (PyString_GET_SIZE(args) < 2)) { + res = save_unicode(self, args, 0); + goto finally; + } #endif + } - case 't': - if (type == &PyTuple_Type) { - res = save_tuple(self, args); - goto finally; - } - if (type == &PyType_Type) { - res = save_global(self, args, NULL); - goto finally; - } - break; + if (args->ob_refcnt > 1) { + if (!(py_ob_id = PyLong_FromVoidPtr(args))) + goto finally; - case 'l': - if (type == &PyList_Type) { - res = save_list(self, args); - goto finally; - } - break; - - case 'd': - if (type == &PyDict_Type) { - res = save_dict(self, args); - goto finally; - } - break; - - case 'i': - break; - - case 'c': - break; + if (PyDict_GetItem(self->memo, py_ob_id)) { + if (get(self, py_ob_id) < 0) + goto finally; - case 'f': - if (type == &PyFunction_Type) { - res = save_global(self, args, NULL); - if (res && PyErr_ExceptionMatches(PickleError)) { - /* fall back to reduce */ - PyErr_Clear(); - break; - } - goto finally; - } - break; + res = 0; + goto finally; + } + } - case 'b': - if (type == &PyCFunction_Type) { - res = save_global(self, args, NULL); - goto finally; - } + switch (type->tp_name[0]) { + case 's': + if (type == &PyString_Type) { + res = save_string(self, args, 1); + goto finally; } + break; - if (!pers_save && self->inst_pers_func) { - if ((tmp = save_pers(self, args, self->inst_pers_func)) != 0) { - res = tmp; - goto finally; - } +#ifdef Py_USING_UNICODE + case 'u': + if (type == &PyUnicode_Type) { + res = save_unicode(self, args, 1); + goto finally; } + break; +#endif - if (PyType_IsSubtype(type, &PyType_Type)) { - res = save_global(self, args, NULL); - goto finally; + case 't': + if (type == &PyTuple_Type) { + res = save_tuple(self, args); + goto finally; + } + if (type == &PyType_Type) { + res = save_global(self, args, NULL); + goto finally; + } + break; + + case 'l': + if (type == &PyList_Type) { + res = save_list(self, args); + goto finally; + } + break; + + case 'd': + if (type == &PyDict_Type) { + res = save_dict(self, args); + goto finally; + } + break; + + case 'i': + break; + + case 'c': + break; + + case 'f': + if (type == &PyFunction_Type) { + res = save_global(self, args, NULL); + if (res && PyErr_ExceptionMatches(PickleError)) { + /* fall back to reduce */ + PyErr_Clear(); + break; + } + goto finally; } + break; - /* Get a reduction callable, and call it. This may come from - * copy_reg.dispatch_table, the object's __reduce_ex__ method, - * or the object's __reduce__ method. - */ - __reduce__ = PyDict_GetItem(dispatch_table, (PyObject *)type); + case 'b': + if (type == &PyCFunction_Type) { + res = save_global(self, args, NULL); + goto finally; + } + } + + if (!pers_save && self->inst_pers_func) { + if ((tmp = save_pers(self, args, self->inst_pers_func)) != 0) { + res = tmp; + goto finally; + } + } + + if (PyType_IsSubtype(type, &PyType_Type)) { + res = save_global(self, args, NULL); + goto finally; + } + + /* Get a reduction callable, and call it. This may come from + * copy_reg.dispatch_table, the object's __reduce_ex__ method, + * or the object's __reduce__ method. + */ + __reduce__ = PyDict_GetItem(dispatch_table, (PyObject *) type); + if (__reduce__ != NULL) { + Py_INCREF(__reduce__); + Py_INCREF(args); + ARG_TUP(self, args); + if (self->arg) { + t = PyObject_Call(__reduce__, self->arg, NULL); + FREE_ARG_TUP(self); + } + } + else { + /* Check for a __reduce_ex__ method. */ + __reduce__ = PyObject_GetAttr(args, __reduce_ex___str); if (__reduce__ != NULL) { - Py_INCREF(__reduce__); - Py_INCREF(args); - ARG_TUP(self, args); + t = PyInt_FromLong(self->proto); + if (t != NULL) { + ARG_TUP(self, t); + t = NULL; if (self->arg) { - t = PyObject_Call(__reduce__, self->arg, NULL); - FREE_ARG_TUP(self); + t = PyObject_Call(__reduce__, self->arg, NULL); + FREE_ARG_TUP(self); } + } } else { - /* Check for a __reduce_ex__ method. */ - __reduce__ = PyObject_GetAttr(args, __reduce_ex___str); - if (__reduce__ != NULL) { - t = PyInt_FromLong(self->proto); - if (t != NULL) { - ARG_TUP(self, t); - t = NULL; - if (self->arg) { - t = PyObject_Call(__reduce__, - self->arg, NULL); - FREE_ARG_TUP(self); - } - } - } - else { - if (PyErr_ExceptionMatches(PyExc_AttributeError)) - PyErr_Clear(); - else - goto finally; - /* Check for a __reduce__ method. */ - __reduce__ = PyObject_GetAttr(args, __reduce___str); - if (__reduce__ != NULL) { - t = PyObject_Call(__reduce__, - empty_tuple, NULL); - } - else { - PyErr_SetObject(UnpickleableError, args); - goto finally; - } - } - } - - if (t == NULL) - goto finally; - - if (PyString_Check(t)) { - res = save_global(self, args, t); - goto finally; - } - - if (! PyTuple_Check(t)) { - cPickle_ErrFormat(PicklingError, "Value returned by " - "%s must be string or tuple", - "O", __reduce__); - goto finally; - } - - size = PyTuple_Size(t); - if (size < 2 || size > 5) { - cPickle_ErrFormat(PicklingError, "tuple returned by " - "%s must contain 2 through 5 elements", - "O", __reduce__); - goto finally; - } - - arg_tup = PyTuple_GET_ITEM(t, 1); - if (!(PyTuple_Check(arg_tup) || arg_tup == Py_None)) { - cPickle_ErrFormat(PicklingError, "Second element of " - "tuple returned by %s must be a tuple", - "O", __reduce__); + if (PyErr_ExceptionMatches(PyExc_AttributeError)) + PyErr_Clear(); + else goto finally; - } + /* Check for a __reduce__ method. */ + __reduce__ = PyObject_GetAttr(args, __reduce___str); + if (__reduce__ != NULL) { + t = PyObject_Call(__reduce__, empty_tuple, NULL); + } + else { + PyErr_SetObject(UnpickleableError, args); + goto finally; + } + } + } + + if (t == NULL) + goto finally; + + if (PyString_Check(t)) { + res = save_global(self, args, t); + goto finally; + } + + if (!PyTuple_Check(t)) { + cPickle_ErrFormat(PicklingError, "Value returned by " + "%s must be string or tuple", "O", __reduce__); + goto finally; + } + + size = PyTuple_Size(t); + if (size < 2 || size > 5) { + cPickle_ErrFormat(PicklingError, "tuple returned by " + "%s must contain 2 through 5 elements", + "O", __reduce__); + goto finally; + } + + arg_tup = PyTuple_GET_ITEM(t, 1); + if (!(PyTuple_Check(arg_tup) || arg_tup == Py_None)) { + cPickle_ErrFormat(PicklingError, "Second element of " + "tuple returned by %s must be a tuple", + "O", __reduce__); + goto finally; + } - res = save_reduce(self, t, args); + res = save_reduce(self, t, args); finally: - self->nesting--; - Py_XDECREF(py_ob_id); - Py_XDECREF(__reduce__); - Py_XDECREF(t); + self->nesting--; + Py_XDECREF(py_ob_id); + Py_XDECREF(__reduce__); + Py_XDECREF(t); - return res; + return res; } static int -dump(Picklerobject *self, PyObject *args) +dump(Picklerobject * self, PyObject * args) { - static char stop = STOP; + static char stop = STOP; - if (self->proto >= 2) { - char bytes[2]; + if (self->proto >= 2) { + char bytes[2]; - bytes[0] = PROTO; - assert(self->proto >= 0 && self->proto < 256); - bytes[1] = (char)self->proto; - if (self->write_func(self, bytes, 2) < 0) - return -1; - } + bytes[0] = PROTO; + assert(self->proto >= 0 && self->proto < 256); + bytes[1] = (char) self->proto; + if (self->write_func(self, bytes, 2) < 0) + return -1; + } - if (save(self, args, 0) < 0) - return -1; + if (save(self, args, 0) < 0) + return -1; - if (self->write_func(self, &stop, 1) < 0) - return -1; + if (self->write_func(self, &stop, 1) < 0) + return -1; - if (self->write_func(self, NULL, 0) < 0) - return -1; + if (self->write_func(self, NULL, 0) < 0) + return -1; - return 0; + return 0; } static PyObject * -Pickle_clear_memo(Picklerobject *self, PyObject *args) +Pickle_clear_memo(Picklerobject * self, PyObject * args) { - if (self->memo) - PyDict_Clear(self->memo); - Py_INCREF(Py_None); - return Py_None; + if (self->memo) + PyDict_Clear(self->memo); + Py_INCREF(Py_None); + return Py_None; } static PyObject * -Pickle_getvalue(Picklerobject *self, PyObject *args) +Pickle_getvalue(Picklerobject * self, PyObject * args) { - int l, i, rsize, ssize, clear=1, lm; - long ik; - PyObject *k, *r; - char *s, *p, *have_get; - Pdata *data; - - /* Can be called by Python code or C code */ - if (args && !PyArg_ParseTuple(args, "|i:getvalue", &clear)) - return NULL; - - /* Check to make sure we are based on a list */ - if (! Pdata_Check(self->file)) { - PyErr_SetString(PicklingError, - "Attempt to getvalue() a non-list-based pickler"); - return NULL; - } + int l, i, rsize, ssize, clear = 1, lm; + long ik; + PyObject *k, *r; + char *s, *p, *have_get; + Pdata *data; - /* flush write buffer */ - if (write_other(self, NULL, 0) < 0) return NULL; + /* Can be called by Python code or C code */ + if (args && !PyArg_ParseTuple(args, "|i:getvalue", &clear)) + return NULL; - data=(Pdata*)self->file; - l=data->length; + /* Check to make sure we are based on a list */ + if (!Pdata_Check(self->file)) { + PyErr_SetString(PicklingError, + "Attempt to getvalue() a non-list-based pickler"); + return NULL; + } - /* set up an array to hold get/put status */ - lm = PyDict_Size(self->memo); - if (lm < 0) return NULL; - lm++; - have_get = malloc(lm); - if (have_get == NULL) return PyErr_NoMemory(); - memset(have_get, 0, lm); - - /* Scan for gets. */ - for (rsize = 0, i = l; --i >= 0; ) { - k = data->data[i]; - - if (PyString_Check(k)) - rsize += PyString_GET_SIZE(k); - - else if (PyInt_Check(k)) { /* put */ - ik = PyInt_AsLong(k); - if (ik == -1 && PyErr_Occurred()) - goto err; - if (ik >= lm || ik == 0) { - PyErr_SetString(PicklingError, - "Invalid get data"); - goto err; - } - if (have_get[ik]) /* with matching get */ - rsize += ik < 256 ? 2 : 5; - } + /* flush write buffer */ + if (write_other(self, NULL, 0) < 0) + return NULL; - else if (! (PyTuple_Check(k) && - PyTuple_GET_SIZE(k) == 2 && - PyInt_Check((k = PyTuple_GET_ITEM(k, 0)))) - ) { - PyErr_SetString(PicklingError, - "Unexpected data in internal list"); - goto err; - } + data = (Pdata *) self->file; + l = data->length; - else { /* put */ - ik = PyInt_AsLong(k); - if (ik == -1 && PyErr_Occurred()) - goto err; - if (ik >= lm || ik == 0) { - PyErr_SetString(PicklingError, - "Invalid get data"); - return NULL; - } - have_get[ik] = 1; - rsize += ik < 256 ? 2 : 5; - } - } - - /* Now generate the result */ - r = PyString_FromStringAndSize(NULL, rsize); - if (r == NULL) goto err; - s = PyString_AS_STRING((PyStringObject *)r); - - for (i = 0; i < l; i++) { - k = data->data[i]; - - if (PyString_Check(k)) { - ssize = PyString_GET_SIZE(k); - if (ssize) { - p=PyString_AS_STRING((PyStringObject *)k); - while (--ssize >= 0) - *s++ = *p++; - } - } + /* set up an array to hold get/put status */ + lm = PyDict_Size(self->memo); + if (lm < 0) + return NULL; + lm++; + have_get = malloc(lm); + if (have_get == NULL) + return PyErr_NoMemory(); + memset(have_get, 0, lm); - else if (PyTuple_Check(k)) { /* get */ - ik = PyLong_AsLong(PyTuple_GET_ITEM(k, 0)); - if (ik == -1 && PyErr_Occurred()) - goto err; - if (ik < 256) { - *s++ = BINGET; - *s++ = (int)(ik & 0xff); - } - else { - *s++ = LONG_BINGET; - *s++ = (int)(ik & 0xff); - *s++ = (int)((ik >> 8) & 0xff); - *s++ = (int)((ik >> 16) & 0xff); - *s++ = (int)((ik >> 24) & 0xff); - } - } + /* Scan for gets. */ + for (rsize = 0, i = l; --i >= 0;) { + k = data->data[i]; + + if (PyString_Check(k)) + rsize += PyString_GET_SIZE(k); + + else if (PyInt_Check(k)) { /* put */ + ik = PyInt_AsLong(k); + if (ik == -1 && PyErr_Occurred()) + goto err; + if (ik >= lm || ik == 0) { + PyErr_SetString(PicklingError, "Invalid get data"); + goto err; + } + if (have_get[ik]) /* with matching get */ + rsize += ik < 256 ? 2 : 5; + } + + else if (!(PyTuple_Check(k) && + PyTuple_GET_SIZE(k) == 2 && + PyInt_Check((k = PyTuple_GET_ITEM(k, 0)))) + ) { + PyErr_SetString(PicklingError, "Unexpected data in internal list"); + goto err; + } + + else { /* put */ + ik = PyInt_AsLong(k); + if (ik == -1 && PyErr_Occurred()) + goto err; + if (ik >= lm || ik == 0) { + PyErr_SetString(PicklingError, "Invalid get data"); + return NULL; + } + have_get[ik] = 1; + rsize += ik < 256 ? 2 : 5; + } + } + + /* Now generate the result */ + r = PyString_FromStringAndSize(NULL, rsize); + if (r == NULL) + goto err; + s = PyString_AS_STRING((PyStringObject *) r); + + for (i = 0; i < l; i++) { + k = data->data[i]; + + if (PyString_Check(k)) { + ssize = PyString_GET_SIZE(k); + if (ssize) { + p = PyString_AS_STRING((PyStringObject *) k); + while (--ssize >= 0) + *s++ = *p++; + } + } + + else if (PyTuple_Check(k)) { /* get */ + ik = PyLong_AsLong(PyTuple_GET_ITEM(k, 0)); + if (ik == -1 && PyErr_Occurred()) + goto err; + if (ik < 256) { + *s++ = BINGET; + *s++ = (int) (ik & 0xff); + } + else { + *s++ = LONG_BINGET; + *s++ = (int) (ik & 0xff); + *s++ = (int) ((ik >> 8) & 0xff); + *s++ = (int) ((ik >> 16) & 0xff); + *s++ = (int) ((ik >> 24) & 0xff); + } + } + + else { /* put */ + ik = PyLong_AsLong(k); + if (ik == -1 && PyErr_Occurred()) + goto err; - else { /* put */ - ik = PyLong_AsLong(k); - if (ik == -1 && PyErr_Occurred()) - goto err; - - if (have_get[ik]) { /* with matching get */ - if (ik < 256) { - *s++ = BINPUT; - *s++ = (int)(ik & 0xff); - } - else { - *s++ = LONG_BINPUT; - *s++ = (int)(ik & 0xff); - *s++ = (int)((ik >> 8) & 0xff); - *s++ = (int)((ik >> 16) & 0xff); - *s++ = (int)((ik >> 24) & 0xff); - } - } + if (have_get[ik]) { /* with matching get */ + if (ik < 256) { + *s++ = BINPUT; + *s++ = (int) (ik & 0xff); } - } - - if (clear) { - PyDict_Clear(self->memo); - Pdata_clear(data, 0); - } + else { + *s++ = LONG_BINPUT; + *s++ = (int) (ik & 0xff); + *s++ = (int) ((ik >> 8) & 0xff); + *s++ = (int) ((ik >> 16) & 0xff); + *s++ = (int) ((ik >> 24) & 0xff); + } + } + } + } + + if (clear) { + PyDict_Clear(self->memo); + Pdata_clear(data, 0); + } - free(have_get); - return r; + free(have_get); + return r; err: - free(have_get); - return NULL; + free(have_get); + return NULL; } static PyObject * -Pickler_dump(Picklerobject *self, PyObject *args) +Pickler_dump(Picklerobject * self, PyObject * args) { - PyObject *ob; - int get=0; + PyObject *ob; + int get = 0; - if (!( PyArg_ParseTuple(args, "O|i:dump", &ob, &get))) - return NULL; + if (!(PyArg_ParseTuple(args, "O|i:dump", &ob, &get))) + return NULL; - if (dump(self, ob) < 0) - return NULL; + if (dump(self, ob) < 0) + return NULL; - if (get) return Pickle_getvalue(self, NULL); + if (get) + return Pickle_getvalue(self, NULL); - /* XXX Why does dump() return self? */ - Py_INCREF(self); - return (PyObject*)self; + /* XXX Why does dump() return self? */ + Py_INCREF(self); + return (PyObject *) self; } -static struct PyMethodDef Pickler_methods[] = -{ - {"dump", (PyCFunction)Pickler_dump, METH_VARARGS, - PyDoc_STR("dump(object) -- " - "Write an object in pickle format to the object's pickle stream")}, - {"clear_memo", (PyCFunction)Pickle_clear_memo, METH_NOARGS, - PyDoc_STR("clear_memo() -- Clear the picklers memo")}, - {"getvalue", (PyCFunction)Pickle_getvalue, METH_VARARGS, - PyDoc_STR("getvalue() -- Finish picking a list-based pickle")}, - {NULL, NULL} /* sentinel */ +static struct PyMethodDef Pickler_methods[] = { + {"dump", (PyCFunction) Pickler_dump, METH_VARARGS, + PyDoc_STR("dump(object) -- " + "Write an object in pickle format to the object's pickle stream")}, + {"clear_memo", (PyCFunction) Pickle_clear_memo, METH_NOARGS, + PyDoc_STR("clear_memo() -- Clear the picklers memo")}, + {"getvalue", (PyCFunction) Pickle_getvalue, METH_VARARGS, + PyDoc_STR("getvalue() -- Finish picking a list-based pickle")}, + {NULL, NULL} /* sentinel */ }; static Picklerobject * -newPicklerobject(PyObject *file, int proto) +newPicklerobject(PyObject * file, int proto) { - Picklerobject *self; - - if (proto < 0) - proto = HIGHEST_PROTOCOL; - if (proto > HIGHEST_PROTOCOL) { - PyErr_Format(PyExc_ValueError, "pickle protocol %d asked for; " - "the highest available protocol is %d", - proto, HIGHEST_PROTOCOL); - return NULL; - } + Picklerobject *self; - self = PyObject_GC_New(Picklerobject, &Picklertype); - if (self == NULL) - return NULL; - self->proto = proto; - self->bin = proto > 0; - self->fp = NULL; - self->write = NULL; - self->memo = NULL; - self->arg = NULL; - self->pers_func = NULL; - self->inst_pers_func = NULL; - self->write_buf = NULL; - self->fast = 0; - self->nesting = 0; - self->fast_container = 0; - self->fast_memo = NULL; - self->buf_size = 0; - self->dispatch_table = NULL; - - self->file = NULL; - if (file) - Py_INCREF(file); - else { - file = Pdata_New(); - if (file == NULL) - goto err; - } - self->file = file; + if (proto < 0) + proto = HIGHEST_PROTOCOL; + if (proto > HIGHEST_PROTOCOL) { + PyErr_Format(PyExc_ValueError, "pickle protocol %d asked for; " + "the highest available protocol is %d", + proto, HIGHEST_PROTOCOL); + return NULL; + } - if (!( self->memo = PyDict_New())) + self = PyObject_GC_New(Picklerobject, &Picklertype); + if (self == NULL) + return NULL; + self->proto = proto; + self->bin = proto > 0; + self->fp = NULL; + self->write = NULL; + self->memo = NULL; + self->arg = NULL; + self->pers_func = NULL; + self->inst_pers_func = NULL; + self->write_buf = NULL; + self->fast = 0; + self->nesting = 0; + self->fast_container = 0; + self->fast_memo = NULL; + self->buf_size = 0; + self->dispatch_table = NULL; + + self->file = NULL; + if (file) + Py_INCREF(file); + else { + file = Pdata_New(); + if (file == NULL) + goto err; + } + self->file = file; + + if (!(self->memo = PyDict_New())) + goto err; + + if (PyFile_Check(file)) { + self->fp = PyFile_AsFile(file); + if (self->fp == NULL) { + PyErr_SetString(PyExc_ValueError, "I/O operation on closed file"); + goto err; + } + self->write_func = write_file; + } + else if (PycStringIO_OutputCheck(file)) { + self->write_func = write_cStringIO; + } + else if (file == Py_None) { + self->write_func = write_none; + } + else { + self->write_func = write_other; + + if (!Pdata_Check(file)) { + self->write = PyObject_GetAttr(file, write_str); + if (!self->write) { + PyErr_Clear(); + PyErr_SetString(PyExc_TypeError, + "argument must have 'write' " "attribute"); goto err; - - if (PyFile_Check(file)) { - self->fp = PyFile_AsFile(file); - if (self->fp == NULL) { - PyErr_SetString(PyExc_ValueError, - "I/O operation on closed file"); - goto err; - } - self->write_func = write_file; + } } - else if (PycStringIO_OutputCheck(file)) { - self->write_func = write_cStringIO; - } - else if (file == Py_None) { - self->write_func = write_none; - } - else { - self->write_func = write_other; - if (! Pdata_Check(file)) { - self->write = PyObject_GetAttr(file, write_str); - if (!self->write) { - PyErr_Clear(); - PyErr_SetString(PyExc_TypeError, - "argument must have 'write' " - "attribute"); - goto err; - } - } - - self->write_buf = (char *)PyMem_Malloc(WRITE_BUF_SIZE); - if (self->write_buf == NULL) { - PyErr_NoMemory(); - goto err; - } + self->write_buf = (char *) PyMem_Malloc(WRITE_BUF_SIZE); + if (self->write_buf == NULL) { + PyErr_NoMemory(); + goto err; } + } - self->dispatch_table = dispatch_table; - Py_INCREF(dispatch_table); - PyObject_GC_Track(self); + self->dispatch_table = dispatch_table; + Py_INCREF(dispatch_table); + PyObject_GC_Track(self); - return self; + return self; err: - Py_DECREF(self); - return NULL; + Py_DECREF(self); + return NULL; } static PyObject * -get_Pickler(PyObject *self, PyObject *args, PyObject *kwds) +get_Pickler(PyObject * self, PyObject * args, PyObject * kwds) { - static char *kwlist[] = {"file", "protocol", NULL}; - PyObject *file = NULL; - int proto = 0; - - /* XXX - * The documented signature is Pickler(file, protocol=0), but this - * accepts Pickler() and Pickler(integer) too. The meaning then - * is clear as mud, undocumented, and not supported by pickle.py. - * I'm told Zope uses this, but I haven't traced into this code - * far enough to figure out what it means. - */ - if (!PyArg_ParseTuple(args, "|i:Pickler", &proto)) { - PyErr_Clear(); - proto = 0; - if (!PyArg_ParseTupleAndKeywords(args, kwds, "O|i:Pickler", - kwlist, &file, &proto)) - return NULL; - } - return (PyObject *)newPicklerobject(file, proto); + static char *kwlist[] = { "file", "protocol", NULL }; + PyObject *file = NULL; + int proto = 0; + + /* XXX + * The documented signature is Pickler(file, protocol=0), but this + * accepts Pickler() and Pickler(integer) too. The meaning then + * is clear as mud, undocumented, and not supported by pickle.py. + * I'm told Zope uses this, but I haven't traced into this code + * far enough to figure out what it means. + */ + if (!PyArg_ParseTuple(args, "|i:Pickler", &proto)) { + PyErr_Clear(); + proto = 0; + if (!PyArg_ParseTupleAndKeywords(args, kwds, "O|i:Pickler", + kwlist, &file, &proto)) + return NULL; + } + return (PyObject *) newPicklerobject(file, proto); } static void -Pickler_dealloc(Picklerobject *self) +Pickler_dealloc(Picklerobject * self) { - PyObject_GC_UnTrack(self); - Py_XDECREF(self->write); - Py_XDECREF(self->memo); - Py_XDECREF(self->fast_memo); - Py_XDECREF(self->arg); - Py_XDECREF(self->file); - Py_XDECREF(self->pers_func); - Py_XDECREF(self->inst_pers_func); - Py_XDECREF(self->dispatch_table); - PyMem_Free(self->write_buf); - self->ob_type->tp_free((PyObject *)self); + PyObject_GC_UnTrack(self); + Py_XDECREF(self->write); + Py_XDECREF(self->memo); + Py_XDECREF(self->fast_memo); + Py_XDECREF(self->arg); + Py_XDECREF(self->file); + Py_XDECREF(self->pers_func); + Py_XDECREF(self->inst_pers_func); + Py_XDECREF(self->dispatch_table); + PyMem_Free(self->write_buf); + self->ob_type->tp_free((PyObject *) self); } static int -Pickler_traverse(Picklerobject *self, visitproc visit, void *arg) -{ - Py_VISIT(self->write); - Py_VISIT(self->memo); - Py_VISIT(self->fast_memo); - Py_VISIT(self->arg); - Py_VISIT(self->file); - Py_VISIT(self->pers_func); - Py_VISIT(self->inst_pers_func); - Py_VISIT(self->dispatch_table); - return 0; +Pickler_traverse(Picklerobject * self, visitproc visit, void *arg) +{ + Py_VISIT(self->write); + Py_VISIT(self->memo); + Py_VISIT(self->fast_memo); + Py_VISIT(self->arg); + Py_VISIT(self->file); + Py_VISIT(self->pers_func); + Py_VISIT(self->inst_pers_func); + Py_VISIT(self->dispatch_table); + return 0; } static int -Pickler_clear(Picklerobject *self) -{ - Py_CLEAR(self->write); - Py_CLEAR(self->memo); - Py_CLEAR(self->fast_memo); - Py_CLEAR(self->arg); - Py_CLEAR(self->file); - Py_CLEAR(self->pers_func); - Py_CLEAR(self->inst_pers_func); - Py_CLEAR(self->dispatch_table); - return 0; +Pickler_clear(Picklerobject * self) +{ + Py_CLEAR(self->write); + Py_CLEAR(self->memo); + Py_CLEAR(self->fast_memo); + Py_CLEAR(self->arg); + Py_CLEAR(self->file); + Py_CLEAR(self->pers_func); + Py_CLEAR(self->inst_pers_func); + Py_CLEAR(self->dispatch_table); + return 0; } static PyObject * -Pickler_get_pers_func(Picklerobject *p) +Pickler_get_pers_func(Picklerobject * p) { - if (p->pers_func == NULL) - PyErr_SetString(PyExc_AttributeError, "persistent_id"); - else - Py_INCREF(p->pers_func); - return p->pers_func; + if (p->pers_func == NULL) + PyErr_SetString(PyExc_AttributeError, "persistent_id"); + else + Py_INCREF(p->pers_func); + return p->pers_func; } static int -Pickler_set_pers_func(Picklerobject *p, PyObject *v) +Pickler_set_pers_func(Picklerobject * p, PyObject * v) { - if (v == NULL) { - PyErr_SetString(PyExc_TypeError, - "attribute deletion is not supported"); - return -1; - } - Py_XDECREF(p->pers_func); - Py_INCREF(v); - p->pers_func = v; - return 0; + if (v == NULL) { + PyErr_SetString(PyExc_TypeError, + "attribute deletion is not supported"); + return -1; + } + Py_XDECREF(p->pers_func); + Py_INCREF(v); + p->pers_func = v; + return 0; } static int -Pickler_set_inst_pers_func(Picklerobject *p, PyObject *v) +Pickler_set_inst_pers_func(Picklerobject * p, PyObject * v) { - if (v == NULL) { - PyErr_SetString(PyExc_TypeError, - "attribute deletion is not supported"); - return -1; - } - Py_XDECREF(p->inst_pers_func); - Py_INCREF(v); - p->inst_pers_func = v; - return 0; + if (v == NULL) { + PyErr_SetString(PyExc_TypeError, + "attribute deletion is not supported"); + return -1; + } + Py_XDECREF(p->inst_pers_func); + Py_INCREF(v); + p->inst_pers_func = v; + return 0; } static PyObject * -Pickler_get_memo(Picklerobject *p) +Pickler_get_memo(Picklerobject * p) { - if (p->memo == NULL) - PyErr_SetString(PyExc_AttributeError, "memo"); - else - Py_INCREF(p->memo); - return p->memo; + if (p->memo == NULL) + PyErr_SetString(PyExc_AttributeError, "memo"); + else + Py_INCREF(p->memo); + return p->memo; } static int -Pickler_set_memo(Picklerobject *p, PyObject *v) +Pickler_set_memo(Picklerobject * p, PyObject * v) { - if (v == NULL) { - PyErr_SetString(PyExc_TypeError, - "attribute deletion is not supported"); - return -1; - } - if (!PyDict_Check(v)) { - PyErr_SetString(PyExc_TypeError, "memo must be a dictionary"); - return -1; - } - Py_XDECREF(p->memo); - Py_INCREF(v); - p->memo = v; - return 0; + if (v == NULL) { + PyErr_SetString(PyExc_TypeError, + "attribute deletion is not supported"); + return -1; + } + if (!PyDict_Check(v)) { + PyErr_SetString(PyExc_TypeError, "memo must be a dictionary"); + return -1; + } + Py_XDECREF(p->memo); + Py_INCREF(v); + p->memo = v; + return 0; } static PyObject * -Pickler_get_error(Picklerobject *p) +Pickler_get_error(Picklerobject * p) { - /* why is this an attribute on the Pickler? */ - Py_INCREF(PicklingError); - return PicklingError; + /* why is this an attribute on the Pickler? */ + Py_INCREF(PicklingError); + return PicklingError; } static PyMemberDef Pickler_members[] = { @@ -2873,160 +2866,164 @@ }; static PyGetSetDef Pickler_getsets[] = { - {"persistent_id", (getter)Pickler_get_pers_func, - (setter)Pickler_set_pers_func}, - {"inst_persistent_id", NULL, (setter)Pickler_set_inst_pers_func}, - {"memo", (getter)Pickler_get_memo, (setter)Pickler_set_memo}, - {"PicklingError", (getter)Pickler_get_error, NULL}, + {"persistent_id", (getter) Pickler_get_pers_func, + (setter) Pickler_set_pers_func}, + {"inst_persistent_id", NULL, (setter) Pickler_set_inst_pers_func}, + {"memo", (getter) Pickler_get_memo, (setter) Pickler_set_memo}, + {"PicklingError", (getter) Pickler_get_error, NULL}, {NULL} }; -PyDoc_STRVAR(Picklertype__doc__, -"Objects that know how to pickle objects\n"); +PyDoc_STRVAR(Picklertype__doc__, "Objects that know how to pickle objects\n"); static PyTypeObject Picklertype = { PyObject_HEAD_INIT(NULL) - 0, /*ob_size*/ - "cPickle.Pickler", /*tp_name*/ - sizeof(Picklerobject), /*tp_basicsize*/ + 0, /*ob_size */ + "cPickle.Pickler", /*tp_name */ + sizeof(Picklerobject), /*tp_basicsize */ 0, - (destructor)Pickler_dealloc, /* tp_dealloc */ - 0, /* tp_print */ - 0, /* tp_getattr */ - 0, /* tp_setattr */ - 0, /* tp_compare */ - 0, /* tp_repr */ - 0, /* tp_as_number */ - 0, /* tp_as_sequence */ - 0, /* tp_as_mapping */ - 0, /* tp_hash */ - 0, /* tp_call */ - 0, /* tp_str */ - PyObject_GenericGetAttr, /* tp_getattro */ - PyObject_GenericSetAttr, /* tp_setattro */ - 0, /* tp_as_buffer */ + (destructor) Pickler_dealloc, /* tp_dealloc */ + 0, /* tp_print */ + 0, /* tp_getattr */ + 0, /* tp_setattr */ + 0, /* tp_compare */ + 0, /* tp_repr */ + 0, /* tp_as_number */ + 0, /* tp_as_sequence */ + 0, /* tp_as_mapping */ + 0, /* tp_hash */ + 0, /* tp_call */ + 0, /* tp_str */ + PyObject_GenericGetAttr, /* tp_getattro */ + PyObject_GenericSetAttr, /* tp_setattro */ + 0, /* tp_as_buffer */ Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE | Py_TPFLAGS_HAVE_GC, - Picklertype__doc__, /* tp_doc */ - (traverseproc)Pickler_traverse, /* tp_traverse */ - (inquiry)Pickler_clear, /* tp_clear */ - 0, /* tp_richcompare */ - 0, /* tp_weaklistoffset */ - 0, /* tp_iter */ - 0, /* tp_iternext */ - Pickler_methods, /* tp_methods */ - Pickler_members, /* tp_members */ - Pickler_getsets, /* tp_getset */ + Picklertype__doc__, /* tp_doc */ + (traverseproc) Pickler_traverse, /* tp_traverse */ + (inquiry) Pickler_clear, /* tp_clear */ + 0, /* tp_richcompare */ + 0, /* tp_weaklistoffset */ + 0, /* tp_iter */ + 0, /* tp_iternext */ + Pickler_methods, /* tp_methods */ + Pickler_members, /* tp_members */ + Pickler_getsets, /* tp_getset */ }; static PyObject * -find_class(PyObject *py_module_name, PyObject *py_global_name, PyObject *fc) +find_class(PyObject * py_module_name, PyObject * py_global_name, PyObject * fc) { - PyObject *global = 0, *module; + PyObject *global = 0, *module; - if (fc) { - if (fc==Py_None) { - PyErr_SetString(UnpicklingError, "Global and instance " - "pickles are not supported."); - return NULL; - } - return PyObject_CallFunctionObjArgs(fc, py_module_name, - py_global_name, NULL); - } + if (fc) { + if (fc == Py_None) { + PyErr_SetString(UnpicklingError, "Global and instance " + "pickles are not supported."); + return NULL; + } + return PyObject_CallFunctionObjArgs(fc, py_module_name, + py_global_name, NULL); + } - module = PySys_GetObject("modules"); - if (module == NULL) - return NULL; + module = PySys_GetObject("modules"); + if (module == NULL) + return NULL; - module = PyDict_GetItem(module, py_module_name); - if (module == NULL) { - module = PyImport_Import(py_module_name); - if (!module) - return NULL; - global = PyObject_GetAttr(module, py_global_name); - Py_DECREF(module); - } - else - global = PyObject_GetAttr(module, py_global_name); - return global; + module = PyDict_GetItem(module, py_module_name); + if (module == NULL) { + module = PyImport_Import(py_module_name); + if (!module) + return NULL; + global = PyObject_GetAttr(module, py_global_name); + Py_DECREF(module); + } + else + global = PyObject_GetAttr(module, py_global_name); + return global; } static int -marker(Unpicklerobject *self) +marker(Unpicklerobject * self) { - if (self->num_marks < 1) { - PyErr_SetString(UnpicklingError, "could not find MARK"); - return -1; - } + if (self->num_marks < 1) { + PyErr_SetString(UnpicklingError, "could not find MARK"); + return -1; + } - return self->marks[--self->num_marks]; + return self->marks[--self->num_marks]; } static int -load_none(Unpicklerobject *self) +load_none(Unpicklerobject * self) { - PDATA_APPEND(self->stack, Py_None, -1); - return 0; + PDATA_APPEND(self->stack, Py_None, -1); + return 0; } static int bad_readline(void) { - PyErr_SetString(UnpicklingError, "pickle data was truncated"); - return -1; + PyErr_SetString(UnpicklingError, "pickle data was truncated"); + return -1; } static int -load_int(Unpicklerobject *self) +load_int(Unpicklerobject * self) { - PyObject *py_int = 0; - char *endptr, *s; - int len, res = -1; - long l; + PyObject *py_int = 0; + char *endptr, *s; + int len, res = -1; + long l; - if ((len = self->readline_func(self, &s)) < 0) return -1; - if (len < 2) return bad_readline(); - if (!( s=pystrndup(s,len))) return -1; + if ((len = self->readline_func(self, &s)) < 0) + return -1; + if (len < 2) + return bad_readline(); + if (!(s = pystrndup(s, len))) + return -1; - errno = 0; - l = strtol(s, &endptr, 0); + errno = 0; + l = strtol(s, &endptr, 0); - if (errno || (*endptr != '\n') || (endptr[1] != '\0')) { - /* Hm, maybe we've got something long. Let's try reading - it as a Python long object. */ - errno = 0; - py_int = PyLong_FromString(s, NULL, 0); - if (py_int == NULL) { - PyErr_SetString(PyExc_ValueError, - "could not convert string to int"); - goto finally; - } + if (errno || (*endptr != '\n') || (endptr[1] != '\0')) { + /* Hm, maybe we've got something long. Let's try reading + * it as a Python long object. */ + errno = 0; + py_int = PyLong_FromString(s, NULL, 0); + if (py_int == NULL) { + PyErr_SetString(PyExc_ValueError, + "could not convert string to int"); + goto finally; + } + } + else { + if (len == 3 && (l == 0 || l == 1)) { + if (!(py_int = PyBool_FromLong(l))) + goto finally; } else { - if (len == 3 && (l == 0 || l == 1)) { - if (!( py_int = PyBool_FromLong(l))) goto finally; - } - else { - if (!( py_int = PyInt_FromLong(l))) goto finally; - } + if (!(py_int = PyInt_FromLong(l))) + goto finally; } + } - free(s); - PDATA_PUSH(self->stack, py_int, -1); - return 0; + free(s); + PDATA_PUSH(self->stack, py_int, -1); + return 0; finally: - free(s); + free(s); - return res; + return res; } static int -load_bool(Unpicklerobject *self, PyObject *boolean) +load_bool(Unpicklerobject * self, PyObject * boolean) { - assert(boolean == Py_True || boolean == Py_False); - PDATA_APPEND(self->stack, boolean, -1); - return 0; + assert(boolean == Py_True || boolean == Py_False); + PDATA_APPEND(self->stack, boolean, -1); + return 0; } /* s contains x bytes of a little-endian integer. Return its value as a @@ -3037,1526 +3034,1594 @@ static long calc_binint(char *s, int x) { - unsigned char c; - int i; - long l; - - for (i = 0, l = 0L; i < x; i++) { - c = (unsigned char)s[i]; - l |= (long)c << (i * 8); - } + unsigned char c; + int i; + long l; + + for (i = 0, l = 0L; i < x; i++) { + c = (unsigned char) s[i]; + l |= (long) c << (i * 8); + } #if SIZEOF_LONG > 4 - /* Unlike BININT1 and BININT2, BININT (more accurately BININT4) - * is signed, so on a box with longs bigger than 4 bytes we need - * to extend a BININT's sign bit to the full width. - */ - if (x == 4 && l & (1L << 31)) - l |= (~0L) << 32; + /* Unlike BININT1 and BININT2, BININT (more accurately BININT4) + * is signed, so on a box with longs bigger than 4 bytes we need + * to extend a BININT's sign bit to the full width. + */ + if (x == 4 && l & (1L << 31)) + l |= (~0L) << 32; #endif - return l; + return l; } static int -load_binintx(Unpicklerobject *self, char *s, int x) +load_binintx(Unpicklerobject * self, char *s, int x) { - PyObject *py_int = 0; - long l; + PyObject *py_int = 0; + long l; - l = calc_binint(s, x); + l = calc_binint(s, x); - if (!( py_int = PyInt_FromLong(l))) - return -1; + if (!(py_int = PyInt_FromLong(l))) + return -1; - PDATA_PUSH(self->stack, py_int, -1); - return 0; + PDATA_PUSH(self->stack, py_int, -1); + return 0; } static int -load_binint(Unpicklerobject *self) +load_binint(Unpicklerobject * self) { - char *s; + char *s; - if (self->read_func(self, &s, 4) < 0) - return -1; + if (self->read_func(self, &s, 4) < 0) + return -1; - return load_binintx(self, s, 4); + return load_binintx(self, s, 4); } static int -load_binint1(Unpicklerobject *self) +load_binint1(Unpicklerobject * self) { - char *s; + char *s; - if (self->read_func(self, &s, 1) < 0) - return -1; + if (self->read_func(self, &s, 1) < 0) + return -1; - return load_binintx(self, s, 1); + return load_binintx(self, s, 1); } static int -load_binint2(Unpicklerobject *self) +load_binint2(Unpicklerobject * self) { - char *s; + char *s; - if (self->read_func(self, &s, 2) < 0) - return -1; + if (self->read_func(self, &s, 2) < 0) + return -1; - return load_binintx(self, s, 2); + return load_binintx(self, s, 2); } static int -load_long(Unpicklerobject *self) +load_long(Unpicklerobject * self) { - PyObject *l = 0; - char *end, *s; - int len, res = -1; + PyObject *l = 0; + char *end, *s; + int len, res = -1; - if ((len = self->readline_func(self, &s)) < 0) return -1; - if (len < 2) return bad_readline(); - if (!( s=pystrndup(s,len))) return -1; + if ((len = self->readline_func(self, &s)) < 0) + return -1; + if (len < 2) + return bad_readline(); + if (!(s = pystrndup(s, len))) + return -1; - if (!( l = PyLong_FromString(s, &end, 0))) - goto finally; + if (!(l = PyLong_FromString(s, &end, 0))) + goto finally; - free(s); - PDATA_PUSH(self->stack, l, -1); - return 0; + free(s); + PDATA_PUSH(self->stack, l, -1); + return 0; finally: - free(s); + free(s); - return res; + return res; } /* 'size' bytes contain the # of bytes of little-endian 256's-complement * data following. */ static int -load_counted_long(Unpicklerobject *self, int size) +load_counted_long(Unpicklerobject * self, int size) { - Py_ssize_t i; - char *nbytes; - unsigned char *pdata; - PyObject *along; + Py_ssize_t i; + char *nbytes; + unsigned char *pdata; + PyObject *along; + + assert(size == 1 || size == 4); + i = self->read_func(self, &nbytes, size); + if (i < 0) + return -1; - assert(size == 1 || size == 4); - i = self->read_func(self, &nbytes, size); - if (i < 0) return -1; - - size = calc_binint(nbytes, size); - if (size < 0) { - /* Corrupt or hostile pickle -- we never write one like - * this. - */ - PyErr_SetString(UnpicklingError, "LONG pickle has negative " - "byte count"); - return -1; - } + size = calc_binint(nbytes, size); + if (size < 0) { + /* Corrupt or hostile pickle -- we never write one like + * this. + */ + PyErr_SetString(UnpicklingError, "LONG pickle has negative " + "byte count"); + return -1; + } - if (size == 0) - along = PyLong_FromLong(0L); - else { - /* Read the raw little-endian bytes & convert. */ - i = self->read_func(self, (char **)&pdata, size); - if (i < 0) return -1; - along = _PyLong_FromByteArray(pdata, (size_t)size, - 1 /* little endian */, 1 /* signed */); - } - if (along == NULL) - return -1; - PDATA_PUSH(self->stack, along, -1); - return 0; + if (size == 0) + along = PyLong_FromLong(0L); + else { + /* Read the raw little-endian bytes & convert. */ + i = self->read_func(self, (char **) &pdata, size); + if (i < 0) + return -1; + along = _PyLong_FromByteArray(pdata, (size_t) size, + 1 /* little endian */ , 1 /* signed */ ); + } + if (along == NULL) + return -1; + PDATA_PUSH(self->stack, along, -1); + return 0; } static int -load_float(Unpicklerobject *self) +load_float(Unpicklerobject * self) { - PyObject *py_float = 0; - char *endptr, *s; - int len, res = -1; - double d; - - if ((len = self->readline_func(self, &s)) < 0) return -1; - if (len < 2) return bad_readline(); - if (!( s=pystrndup(s,len))) return -1; + PyObject *py_float = 0; + char *endptr, *s; + int len, res = -1; + double d; - errno = 0; - d = PyOS_ascii_strtod(s, &endptr); - - if (errno || (endptr[0] != '\n') || (endptr[1] != '\0')) { - PyErr_SetString(PyExc_ValueError, - "could not convert string to float"); - goto finally; - } + if ((len = self->readline_func(self, &s)) < 0) + return -1; + if (len < 2) + return bad_readline(); + if (!(s = pystrndup(s, len))) + return -1; - if (!( py_float = PyFloat_FromDouble(d))) - goto finally; + errno = 0; + d = PyOS_ascii_strtod(s, &endptr); - free(s); - PDATA_PUSH(self->stack, py_float, -1); - return 0; + if (errno || (endptr[0] != '\n') || (endptr[1] != '\0')) { + PyErr_SetString(PyExc_ValueError, "could not convert string to float"); + goto finally; + } + + if (!(py_float = PyFloat_FromDouble(d))) + goto finally; + + free(s); + PDATA_PUSH(self->stack, py_float, -1); + return 0; finally: - free(s); + free(s); - return res; + return res; } static int -load_binfloat(Unpicklerobject *self) +load_binfloat(Unpicklerobject * self) { - PyObject *py_float; - double x; - char *p; + PyObject *py_float; + double x; + char *p; - if (self->read_func(self, &p, 8) < 0) - return -1; + if (self->read_func(self, &p, 8) < 0) + return -1; - x = _PyFloat_Unpack8((unsigned char *)p, 0); - if (x == -1.0 && PyErr_Occurred()) - return -1; + x = _PyFloat_Unpack8((unsigned char *) p, 0); + if (x == -1.0 && PyErr_Occurred()) + return -1; - py_float = PyFloat_FromDouble(x); - if (py_float == NULL) - return -1; + py_float = PyFloat_FromDouble(x); + if (py_float == NULL) + return -1; - PDATA_PUSH(self->stack, py_float, -1); - return 0; + PDATA_PUSH(self->stack, py_float, -1); + return 0; } static int -load_string(Unpicklerobject *self) +load_string(Unpicklerobject * self) { - PyObject *str = 0; - int len, res = -1; - char *s, *p; - - if ((len = self->readline_func(self, &s)) < 0) return -1; - if (len < 2) return bad_readline(); - if (!( s=pystrndup(s,len))) return -1; - - - /* Strip outermost quotes */ - while (s[len-1] <= ' ') - len--; - if(s[0]=='"' && s[len-1]=='"'){ - s[len-1] = '\0'; - p = s + 1 ; - len -= 2; - } else if(s[0]=='\'' && s[len-1]=='\''){ - s[len-1] = '\0'; - p = s + 1 ; - len -= 2; - } else - goto insecure; + PyObject *str = 0; + int len, res = -1; + char *s, *p; + + if ((len = self->readline_func(self, &s)) < 0) + return -1; + if (len < 2) + return bad_readline(); + if (!(s = pystrndup(s, len))) + return -1; + + + /* Strip outermost quotes */ + while (s[len - 1] <= ' ') + len--; + if (s[0] == '"' && s[len - 1] == '"') { + s[len - 1] = '\0'; + p = s + 1; + len -= 2; + } + else if (s[0] == '\'' && s[len - 1] == '\'') { + s[len - 1] = '\0'; + p = s + 1; + len -= 2; + } + else + goto insecure; /********************************************/ - str = PyString_DecodeEscape(p, len, NULL, 0, NULL); - free(s); - if (str) { - PDATA_PUSH(self->stack, str, -1); - res = 0; - } - return res; + str = PyString_DecodeEscape(p, len, NULL, 0, NULL); + free(s); + if (str) { + PDATA_PUSH(self->stack, str, -1); + res = 0; + } + return res; insecure: - free(s); - PyErr_SetString(PyExc_ValueError,"insecure string pickle"); - return -1; + free(s); + PyErr_SetString(PyExc_ValueError, "insecure string pickle"); + return -1; } static int -load_binstring(Unpicklerobject *self) +load_binstring(Unpicklerobject * self) { - PyObject *py_string = 0; - long l; - char *s; + PyObject *py_string = 0; + long l; + char *s; - if (self->read_func(self, &s, 4) < 0) return -1; + if (self->read_func(self, &s, 4) < 0) + return -1; - l = calc_binint(s, 4); + l = calc_binint(s, 4); - if (self->read_func(self, &s, l) < 0) - return -1; + if (self->read_func(self, &s, l) < 0) + return -1; - if (!( py_string = PyString_FromStringAndSize(s, l))) - return -1; + if (!(py_string = PyString_FromStringAndSize(s, l))) + return -1; - PDATA_PUSH(self->stack, py_string, -1); - return 0; + PDATA_PUSH(self->stack, py_string, -1); + return 0; } static int -load_short_binstring(Unpicklerobject *self) +load_short_binstring(Unpicklerobject * self) { - PyObject *py_string = 0; - unsigned char l; - char *s; + PyObject *py_string = 0; + unsigned char l; + char *s; - if (self->read_func(self, &s, 1) < 0) - return -1; + if (self->read_func(self, &s, 1) < 0) + return -1; - l = (unsigned char)s[0]; + l = (unsigned char) s[0]; - if (self->read_func(self, &s, l) < 0) return -1; + if (self->read_func(self, &s, l) < 0) + return -1; - if (!( py_string = PyString_FromStringAndSize(s, l))) return -1; + if (!(py_string = PyString_FromStringAndSize(s, l))) + return -1; - PDATA_PUSH(self->stack, py_string, -1); - return 0; + PDATA_PUSH(self->stack, py_string, -1); + return 0; } #ifdef Py_USING_UNICODE static int -load_unicode(Unpicklerobject *self) +load_unicode(Unpicklerobject * self) { - PyObject *str = 0; - int len, res = -1; - char *s; + PyObject *str = 0; + int len, res = -1; + char *s; - if ((len = self->readline_func(self, &s)) < 0) return -1; - if (len < 1) return bad_readline(); + if ((len = self->readline_func(self, &s)) < 0) + return -1; + if (len < 1) + return bad_readline(); - if (!( str = PyUnicode_DecodeRawUnicodeEscape(s, len - 1, NULL))) - goto finally; + if (!(str = PyUnicode_DecodeRawUnicodeEscape(s, len - 1, NULL))) + goto finally; - PDATA_PUSH(self->stack, str, -1); - return 0; + PDATA_PUSH(self->stack, str, -1); + return 0; finally: - return res; + return res; } #endif #ifdef Py_USING_UNICODE static int -load_binunicode(Unpicklerobject *self) +load_binunicode(Unpicklerobject * self) { - PyObject *unicode; - long l; - char *s; + PyObject *unicode; + long l; + char *s; - if (self->read_func(self, &s, 4) < 0) return -1; + if (self->read_func(self, &s, 4) < 0) + return -1; - l = calc_binint(s, 4); + l = calc_binint(s, 4); - if (self->read_func(self, &s, l) < 0) - return -1; + if (self->read_func(self, &s, l) < 0) + return -1; - if (!( unicode = PyUnicode_DecodeUTF8(s, l, NULL))) - return -1; + if (!(unicode = PyUnicode_DecodeUTF8(s, l, NULL))) + return -1; - PDATA_PUSH(self->stack, unicode, -1); - return 0; + PDATA_PUSH(self->stack, unicode, -1); + return 0; } #endif static int -load_tuple(Unpicklerobject *self) +load_tuple(Unpicklerobject * self) { - PyObject *tup; - int i; + PyObject *tup; + int i; - if ((i = marker(self)) < 0) return -1; - if (!( tup=Pdata_popTuple(self->stack, i))) return -1; - PDATA_PUSH(self->stack, tup, -1); - return 0; + if ((i = marker(self)) < 0) + return -1; + if (!(tup = Pdata_popTuple(self->stack, i))) + return -1; + PDATA_PUSH(self->stack, tup, -1); + return 0; } static int -load_counted_tuple(Unpicklerobject *self, int len) +load_counted_tuple(Unpicklerobject * self, int len) { - PyObject *tup = PyTuple_New(len); + PyObject *tup = PyTuple_New(len); - if (tup == NULL) - return -1; + if (tup == NULL) + return -1; - while (--len >= 0) { - PyObject *element; + while (--len >= 0) { + PyObject *element; - PDATA_POP(self->stack, element); - if (element == NULL) - return -1; - PyTuple_SET_ITEM(tup, len, element); - } - PDATA_PUSH(self->stack, tup, -1); - return 0; + PDATA_POP(self->stack, element); + if (element == NULL) + return -1; + PyTuple_SET_ITEM(tup, len, element); + } + PDATA_PUSH(self->stack, tup, -1); + return 0; } static int -load_empty_list(Unpicklerobject *self) +load_empty_list(Unpicklerobject * self) { - PyObject *list; + PyObject *list; - if (!( list=PyList_New(0))) return -1; - PDATA_PUSH(self->stack, list, -1); - return 0; + if (!(list = PyList_New(0))) + return -1; + PDATA_PUSH(self->stack, list, -1); + return 0; } static int -load_empty_dict(Unpicklerobject *self) +load_empty_dict(Unpicklerobject * self) { - PyObject *dict; + PyObject *dict; - if (!( dict=PyDict_New())) return -1; - PDATA_PUSH(self->stack, dict, -1); - return 0; + if (!(dict = PyDict_New())) + return -1; + PDATA_PUSH(self->stack, dict, -1); + return 0; } static int -load_list(Unpicklerobject *self) +load_list(Unpicklerobject * self) { - PyObject *list = 0; - int i; + PyObject *list = 0; + int i; - if ((i = marker(self)) < 0) return -1; - if (!( list=Pdata_popList(self->stack, i))) return -1; - PDATA_PUSH(self->stack, list, -1); - return 0; + if ((i = marker(self)) < 0) + return -1; + if (!(list = Pdata_popList(self->stack, i))) + return -1; + PDATA_PUSH(self->stack, list, -1); + return 0; } static int -load_dict(Unpicklerobject *self) +load_dict(Unpicklerobject * self) { - PyObject *dict, *key, *value; - int i, j, k; + PyObject *dict, *key, *value; + int i, j, k; - if ((i = marker(self)) < 0) return -1; - j=self->stack->length; + if ((i = marker(self)) < 0) + return -1; + j = self->stack->length; - if (!( dict = PyDict_New())) return -1; - - for (k = i+1; k < j; k += 2) { - key =self->stack->data[k-1]; - value=self->stack->data[k ]; - if (PyDict_SetItem(dict, key, value) < 0) { - Py_DECREF(dict); - return -1; - } - } - Pdata_clear(self->stack, i); - PDATA_PUSH(self->stack, dict, -1); - return 0; + if (!(dict = PyDict_New())) + return -1; + + for (k = i + 1; k < j; k += 2) { + key = self->stack->data[k - 1]; + value = self->stack->data[k]; + if (PyDict_SetItem(dict, key, value) < 0) { + Py_DECREF(dict); + return -1; + } + } + Pdata_clear(self->stack, i); + PDATA_PUSH(self->stack, dict, -1); + return 0; } static PyObject * -Instance_New(PyObject *cls, PyObject *args) +Instance_New(PyObject * cls, PyObject * args) { - PyObject *r = 0; + PyObject *r = 0; - if ((r=PyObject_CallObject(cls, args))) return r; + if ((r = PyObject_CallObject(cls, args))) + return r; - { - PyObject *tp, *v, *tb, *tmp_value; + { + PyObject *tp, *v, *tb, *tmp_value; - PyErr_Fetch(&tp, &v, &tb); - tmp_value = v; - /* NULL occurs when there was a KeyboardInterrupt */ - if (tmp_value == NULL) - tmp_value = Py_None; - if ((r = PyTuple_Pack(3, tmp_value, cls, args))) { - Py_XDECREF(v); - v=r; - } - PyErr_Restore(tp,v,tb); + PyErr_Fetch(&tp, &v, &tb); + tmp_value = v; + /* NULL occurs when there was a KeyboardInterrupt */ + if (tmp_value == NULL) + tmp_value = Py_None; + if ((r = PyTuple_Pack(3, tmp_value, cls, args))) { + Py_XDECREF(v); + v = r; } - return NULL; + PyErr_Restore(tp, v, tb); + } + return NULL; } static int -load_obj(Unpicklerobject *self) +load_obj(Unpicklerobject * self) { - PyObject *class, *tup, *obj=0; - int i; + PyObject *class, *tup, *obj = 0; + int i; - if ((i = marker(self)) < 0) return -1; - if (!( tup=Pdata_popTuple(self->stack, i+1))) return -1; - PDATA_POP(self->stack, class); - if (class) { - obj = Instance_New(class, tup); - Py_DECREF(class); - } - Py_DECREF(tup); + if ((i = marker(self)) < 0) + return -1; + if (!(tup = Pdata_popTuple(self->stack, i + 1))) + return -1; + PDATA_POP(self->stack, class); + if (class) { + obj = Instance_New(class, tup); + Py_DECREF(class); + } + Py_DECREF(tup); - if (! obj) return -1; - PDATA_PUSH(self->stack, obj, -1); - return 0; + if (!obj) + return -1; + PDATA_PUSH(self->stack, obj, -1); + return 0; } static int -load_inst(Unpicklerobject *self) +load_inst(Unpicklerobject * self) { - PyObject *tup, *class=0, *obj=0, *module_name, *class_name; - int i, len; - char *s; - - if ((i = marker(self)) < 0) return -1; - - if ((len = self->readline_func(self, &s)) < 0) return -1; - if (len < 2) return bad_readline(); - module_name = PyString_FromStringAndSize(s, len - 1); - if (!module_name) return -1; - - if ((len = self->readline_func(self, &s)) >= 0) { - if (len < 2) return bad_readline(); - if ((class_name = PyString_FromStringAndSize(s, len - 1))) { - class = find_class(module_name, class_name, - self->find_class); - Py_DECREF(class_name); - } - } - Py_DECREF(module_name); + PyObject *tup, *class = 0, *obj = 0, *module_name, *class_name; + int i, len; + char *s; - if (! class) return -1; + if ((i = marker(self)) < 0) + return -1; + + if ((len = self->readline_func(self, &s)) < 0) + return -1; + if (len < 2) + return bad_readline(); + module_name = PyString_FromStringAndSize(s, len - 1); + if (!module_name) + return -1; - if ((tup=Pdata_popTuple(self->stack, i))) { - obj = Instance_New(class, tup); - Py_DECREF(tup); + if ((len = self->readline_func(self, &s)) >= 0) { + if (len < 2) + return bad_readline(); + if ((class_name = PyString_FromStringAndSize(s, len - 1))) { + class = find_class(module_name, class_name, self->find_class); + Py_DECREF(class_name); } - Py_DECREF(class); + } + Py_DECREF(module_name); - if (! obj) return -1; + if (!class) + return -1; - PDATA_PUSH(self->stack, obj, -1); - return 0; + if ((tup = Pdata_popTuple(self->stack, i))) { + obj = Instance_New(class, tup); + Py_DECREF(tup); + } + Py_DECREF(class); + + if (!obj) + return -1; + + PDATA_PUSH(self->stack, obj, -1); + return 0; } static int -load_newobj(Unpicklerobject *self) +load_newobj(Unpicklerobject * self) { - PyObject *args = NULL; - PyObject *clsraw = NULL; - PyTypeObject *cls; /* clsraw cast to its true type */ - PyObject *obj; - - /* Stack is ... cls argtuple, and we want to call - * cls.__new__(cls, *argtuple). - */ - PDATA_POP(self->stack, args); - if (args == NULL) goto Fail; - if (! PyTuple_Check(args)) { - PyErr_SetString(UnpicklingError, "NEWOBJ expected an arg " - "tuple."); - goto Fail; - } - - PDATA_POP(self->stack, clsraw); - cls = (PyTypeObject *)clsraw; - if (cls == NULL) goto Fail; - if (! PyType_Check(cls)) { - PyErr_SetString(UnpicklingError, "NEWOBJ class argument " - "isn't a type object"); - goto Fail; - } - if (cls->tp_new == NULL) { - PyErr_SetString(UnpicklingError, "NEWOBJ class argument " - "has NULL tp_new"); - goto Fail; - } - - /* Call __new__. */ - obj = cls->tp_new(cls, args, NULL); - if (obj == NULL) goto Fail; - - Py_DECREF(args); - Py_DECREF(clsraw); - PDATA_PUSH(self->stack, obj, -1); - return 0; - - Fail: - Py_XDECREF(args); - Py_XDECREF(clsraw); - return -1; + PyObject *args = NULL; + PyObject *clsraw = NULL; + PyTypeObject *cls; /* clsraw cast to its true type */ + PyObject *obj; + + /* Stack is ... cls argtuple, and we want to call + * cls.__new__(cls, *argtuple). + */ + PDATA_POP(self->stack, args); + if (args == NULL) + goto Fail; + if (!PyTuple_Check(args)) { + PyErr_SetString(UnpicklingError, "NEWOBJ expected an arg " "tuple."); + goto Fail; + } + + PDATA_POP(self->stack, clsraw); + cls = (PyTypeObject *) clsraw; + if (cls == NULL) + goto Fail; + if (!PyType_Check(cls)) { + PyErr_SetString(UnpicklingError, "NEWOBJ class argument " + "isn't a type object"); + goto Fail; + } + if (cls->tp_new == NULL) { + PyErr_SetString(UnpicklingError, "NEWOBJ class argument " + "has NULL tp_new"); + goto Fail; + } + + /* Call __new__. */ + obj = cls->tp_new(cls, args, NULL); + if (obj == NULL) + goto Fail; + + Py_DECREF(args); + Py_DECREF(clsraw); + PDATA_PUSH(self->stack, obj, -1); + return 0; + + Fail: + Py_XDECREF(args); + Py_XDECREF(clsraw); + return -1; } static int -load_global(Unpicklerobject *self) -{ - PyObject *class = 0, *module_name = 0, *class_name = 0; - int len; - char *s; - - if ((len = self->readline_func(self, &s)) < 0) return -1; - if (len < 2) return bad_readline(); - module_name = PyString_FromStringAndSize(s, len - 1); - if (!module_name) return -1; - - if ((len = self->readline_func(self, &s)) >= 0) { - if (len < 2) { - Py_DECREF(module_name); - return bad_readline(); - } - if ((class_name = PyString_FromStringAndSize(s, len - 1))) { - class = find_class(module_name, class_name, - self->find_class); - Py_DECREF(class_name); - } +load_global(Unpicklerobject * self) +{ + PyObject *class = 0, *module_name = 0, *class_name = 0; + int len; + char *s; + + if ((len = self->readline_func(self, &s)) < 0) + return -1; + if (len < 2) + return bad_readline(); + module_name = PyString_FromStringAndSize(s, len - 1); + if (!module_name) + return -1; + + if ((len = self->readline_func(self, &s)) >= 0) { + if (len < 2) { + Py_DECREF(module_name); + return bad_readline(); + } + if ((class_name = PyString_FromStringAndSize(s, len - 1))) { + class = find_class(module_name, class_name, self->find_class); + Py_DECREF(class_name); } - Py_DECREF(module_name); + } + Py_DECREF(module_name); - if (! class) return -1; - PDATA_PUSH(self->stack, class, -1); - return 0; + if (!class) + return -1; + PDATA_PUSH(self->stack, class, -1); + return 0; } static int -load_persid(Unpicklerobject *self) +load_persid(Unpicklerobject * self) { - PyObject *pid = 0; - int len; - char *s; - - if (self->pers_func) { - if ((len = self->readline_func(self, &s)) < 0) return -1; - if (len < 2) return bad_readline(); - - pid = PyString_FromStringAndSize(s, len - 1); - if (!pid) return -1; - - if (PyList_Check(self->pers_func)) { - if (PyList_Append(self->pers_func, pid) < 0) { - Py_DECREF(pid); - return -1; - } - } - else { - ARG_TUP(self, pid); - if (self->arg) { - pid = PyObject_Call(self->pers_func, self->arg, - NULL); - FREE_ARG_TUP(self); - } - } + PyObject *pid = 0; + int len; + char *s; - if (! pid) return -1; + if (self->pers_func) { + if ((len = self->readline_func(self, &s)) < 0) + return -1; + if (len < 2) + return bad_readline(); - PDATA_PUSH(self->stack, pid, -1); - return 0; + pid = PyString_FromStringAndSize(s, len - 1); + if (!pid) + return -1; + + if (PyList_Check(self->pers_func)) { + if (PyList_Append(self->pers_func, pid) < 0) { + Py_DECREF(pid); + return -1; + } } else { - PyErr_SetString(UnpicklingError, - "A load persistent id instruction was encountered,\n" - "but no persistent_load function was specified."); - return -1; + ARG_TUP(self, pid); + if (self->arg) { + pid = PyObject_Call(self->pers_func, self->arg, NULL); + FREE_ARG_TUP(self); + } } + + if (!pid) + return -1; + + PDATA_PUSH(self->stack, pid, -1); + return 0; + } + else { + PyErr_SetString(UnpicklingError, + "A load persistent id instruction was encountered,\n" + "but no persistent_load function was specified."); + return -1; + } } static int -load_binpersid(Unpicklerobject *self) +load_binpersid(Unpicklerobject * self) { - PyObject *pid = 0; + PyObject *pid = 0; - if (self->pers_func) { - PDATA_POP(self->stack, pid); - if (! pid) return -1; - - if (PyList_Check(self->pers_func)) { - if (PyList_Append(self->pers_func, pid) < 0) { - Py_DECREF(pid); - return -1; - } - } - else { - ARG_TUP(self, pid); - if (self->arg) { - pid = PyObject_Call(self->pers_func, self->arg, - NULL); - FREE_ARG_TUP(self); - } - if (! pid) return -1; - } + if (self->pers_func) { + PDATA_POP(self->stack, pid); + if (!pid) + return -1; - PDATA_PUSH(self->stack, pid, -1); - return 0; + if (PyList_Check(self->pers_func)) { + if (PyList_Append(self->pers_func, pid) < 0) { + Py_DECREF(pid); + return -1; + } } else { - PyErr_SetString(UnpicklingError, - "A load persistent id instruction was encountered,\n" - "but no persistent_load function was specified."); + ARG_TUP(self, pid); + if (self->arg) { + pid = PyObject_Call(self->pers_func, self->arg, NULL); + FREE_ARG_TUP(self); + } + if (!pid) return -1; } + + PDATA_PUSH(self->stack, pid, -1); + return 0; + } + else { + PyErr_SetString(UnpicklingError, + "A load persistent id instruction was encountered,\n" + "but no persistent_load function was specified."); + return -1; + } } static int -load_pop(Unpicklerobject *self) +load_pop(Unpicklerobject * self) { - int len; + int len; - if (!( (len=self->stack->length) > 0 )) return stackUnderflow(); + if (!((len = self->stack->length) > 0)) + return stackUnderflow(); - /* Note that we split the (pickle.py) stack into two stacks, - an object stack and a mark stack. We have to be clever and - pop the right one. We do this by looking at the top of the - mark stack. - */ + /* Note that we split the (pickle.py) stack into two stacks, + * an object stack and a mark stack. We have to be clever and + * pop the right one. We do this by looking at the top of the + * mark stack. + */ - if ((self->num_marks > 0) && - (self->marks[self->num_marks - 1] == len)) - self->num_marks--; - else { - len--; - Py_DECREF(self->stack->data[len]); - self->stack->length=len; - } + if ((self->num_marks > 0) && (self->marks[self->num_marks - 1] == len)) + self->num_marks--; + else { + len--; + Py_DECREF(self->stack->data[len]); + self->stack->length = len; + } - return 0; + return 0; } static int -load_pop_mark(Unpicklerobject *self) +load_pop_mark(Unpicklerobject * self) { - int i; + int i; - if ((i = marker(self)) < 0) - return -1; + if ((i = marker(self)) < 0) + return -1; - Pdata_clear(self->stack, i); + Pdata_clear(self->stack, i); - return 0; + return 0; } static int -load_dup(Unpicklerobject *self) +load_dup(Unpicklerobject * self) { - PyObject *last; - int len; + PyObject *last; + int len; - if ((len = self->stack->length) <= 0) return stackUnderflow(); - last=self->stack->data[len-1]; - Py_INCREF(last); - PDATA_PUSH(self->stack, last, -1); - return 0; + if ((len = self->stack->length) <= 0) + return stackUnderflow(); + last = self->stack->data[len - 1]; + Py_INCREF(last); + PDATA_PUSH(self->stack, last, -1); + return 0; } static int -load_get(Unpicklerobject *self) +load_get(Unpicklerobject * self) { - PyObject *py_str = 0, *value = 0; - int len; - char *s; - int rc; - - if ((len = self->readline_func(self, &s)) < 0) return -1; - if (len < 2) return bad_readline(); - - if (!( py_str = PyString_FromStringAndSize(s, len - 1))) return -1; - - value = PyDict_GetItem(self->memo, py_str); - if (! value) { - PyErr_SetObject(BadPickleGet, py_str); - rc = -1; - } - else { - PDATA_APPEND(self->stack, value, -1); - rc = 0; - } + PyObject *py_str = 0, *value = 0; + int len; + char *s; + int rc; + + if ((len = self->readline_func(self, &s)) < 0) + return -1; + if (len < 2) + return bad_readline(); + + if (!(py_str = PyString_FromStringAndSize(s, len - 1))) + return -1; + + value = PyDict_GetItem(self->memo, py_str); + if (!value) { + PyErr_SetObject(BadPickleGet, py_str); + rc = -1; + } + else { + PDATA_APPEND(self->stack, value, -1); + rc = 0; + } - Py_DECREF(py_str); - return rc; + Py_DECREF(py_str); + return rc; } static int -load_binget(Unpicklerobject *self) +load_binget(Unpicklerobject * self) { - PyObject *py_key = 0, *value = 0; - unsigned char key; - char *s; - int rc; + PyObject *py_key = 0, *value = 0; + unsigned char key; + char *s; + int rc; - if (self->read_func(self, &s, 1) < 0) return -1; + if (self->read_func(self, &s, 1) < 0) + return -1; - key = (unsigned char)s[0]; - if (!( py_key = PyInt_FromLong((long)key))) return -1; + key = (unsigned char) s[0]; + if (!(py_key = PyInt_FromLong((long) key))) + return -1; - value = PyDict_GetItem(self->memo, py_key); - if (! value) { - PyErr_SetObject(BadPickleGet, py_key); - rc = -1; - } - else { - PDATA_APPEND(self->stack, value, -1); - rc = 0; - } + value = PyDict_GetItem(self->memo, py_key); + if (!value) { + PyErr_SetObject(BadPickleGet, py_key); + rc = -1; + } + else { + PDATA_APPEND(self->stack, value, -1); + rc = 0; + } - Py_DECREF(py_key); - return rc; + Py_DECREF(py_key); + return rc; } static int -load_long_binget(Unpicklerobject *self) +load_long_binget(Unpicklerobject * self) { - PyObject *py_key = 0, *value = 0; - unsigned char c; - char *s; - long key; - int rc; + PyObject *py_key = 0, *value = 0; + unsigned char c; + char *s; + long key; + int rc; - if (self->read_func(self, &s, 4) < 0) return -1; + if (self->read_func(self, &s, 4) < 0) + return -1; - c = (unsigned char)s[0]; - key = (long)c; - c = (unsigned char)s[1]; - key |= (long)c << 8; - c = (unsigned char)s[2]; - key |= (long)c << 16; - c = (unsigned char)s[3]; - key |= (long)c << 24; + c = (unsigned char) s[0]; + key = (long) c; + c = (unsigned char) s[1]; + key |= (long) c << 8; + c = (unsigned char) s[2]; + key |= (long) c << 16; + c = (unsigned char) s[3]; + key |= (long) c << 24; - if (!( py_key = PyInt_FromLong((long)key))) return -1; + if (!(py_key = PyInt_FromLong((long) key))) + return -1; - value = PyDict_GetItem(self->memo, py_key); - if (! value) { - PyErr_SetObject(BadPickleGet, py_key); - rc = -1; - } - else { - PDATA_APPEND(self->stack, value, -1); - rc = 0; - } + value = PyDict_GetItem(self->memo, py_key); + if (!value) { + PyErr_SetObject(BadPickleGet, py_key); + rc = -1; + } + else { + PDATA_APPEND(self->stack, value, -1); + rc = 0; + } - Py_DECREF(py_key); - return rc; + Py_DECREF(py_key); + return rc; } /* Push an object from the extension registry (EXT[124]). nbytes is * the number of bytes following the opcode, holding the index (code) value. */ static int -load_extension(Unpicklerobject *self, int nbytes) +load_extension(Unpicklerobject * self, int nbytes) { - char *codebytes; /* the nbytes bytes after the opcode */ - long code; /* calc_binint returns long */ - PyObject *py_code; /* code as a Python int */ - PyObject *obj; /* the object to push */ - PyObject *pair; /* (module_name, class_name) */ - PyObject *module_name, *class_name; - - assert(nbytes == 1 || nbytes == 2 || nbytes == 4); - if (self->read_func(self, &codebytes, nbytes) < 0) return -1; - code = calc_binint(codebytes, nbytes); - if (code <= 0) { /* note that 0 is forbidden */ - /* Corrupt or hostile pickle. */ - PyErr_SetString(UnpicklingError, "EXT specifies code <= 0"); - return -1; - } + char *codebytes; /* the nbytes bytes after the opcode */ + long code; /* calc_binint returns long */ + PyObject *py_code; /* code as a Python int */ + PyObject *obj; /* the object to push */ + PyObject *pair; /* (module_name, class_name) */ + PyObject *module_name, *class_name; - /* Look for the code in the cache. */ - py_code = PyInt_FromLong(code); - if (py_code == NULL) return -1; - obj = PyDict_GetItem(extension_cache, py_code); - if (obj != NULL) { - /* Bingo. */ - Py_DECREF(py_code); - PDATA_APPEND(self->stack, obj, -1); - return 0; - } + assert(nbytes == 1 || nbytes == 2 || nbytes == 4); + if (self->read_func(self, &codebytes, nbytes) < 0) + return -1; + code = calc_binint(codebytes, nbytes); + if (code <= 0) { /* note that 0 is forbidden */ + /* Corrupt or hostile pickle. */ + PyErr_SetString(UnpicklingError, "EXT specifies code <= 0"); + return -1; + } - /* Look up the (module_name, class_name) pair. */ - pair = PyDict_GetItem(inverted_registry, py_code); - if (pair == NULL) { - Py_DECREF(py_code); - PyErr_Format(PyExc_ValueError, "unregistered extension " - "code %ld", code); - return -1; - } - /* Since the extension registry is manipulable via Python code, - * confirm that pair is really a 2-tuple of strings. - */ - if (!PyTuple_Check(pair) || PyTuple_Size(pair) != 2 || - !PyString_Check(module_name = PyTuple_GET_ITEM(pair, 0)) || - !PyString_Check(class_name = PyTuple_GET_ITEM(pair, 1))) { - Py_DECREF(py_code); - PyErr_Format(PyExc_ValueError, "_inverted_registry[%ld] " - "isn't a 2-tuple of strings", code); - return -1; - } - /* Load the object. */ - obj = find_class(module_name, class_name, self->find_class); - if (obj == NULL) { - Py_DECREF(py_code); - return -1; - } - /* Cache code -> obj. */ - code = PyDict_SetItem(extension_cache, py_code, obj); + /* Look for the code in the cache. */ + py_code = PyInt_FromLong(code); + if (py_code == NULL) + return -1; + obj = PyDict_GetItem(extension_cache, py_code); + if (obj != NULL) { + /* Bingo. */ Py_DECREF(py_code); - if (code < 0) { - Py_DECREF(obj); - return -1; - } - PDATA_PUSH(self->stack, obj, -1); + PDATA_APPEND(self->stack, obj, -1); return 0; + } + + /* Look up the (module_name, class_name) pair. */ + pair = PyDict_GetItem(inverted_registry, py_code); + if (pair == NULL) { + Py_DECREF(py_code); + PyErr_Format(PyExc_ValueError, "unregistered extension " + "code %ld", code); + return -1; + } + /* Since the extension registry is manipulable via Python code, + * confirm that pair is really a 2-tuple of strings. + */ + if (!PyTuple_Check(pair) || PyTuple_Size(pair) != 2 || + !PyString_Check(module_name = PyTuple_GET_ITEM(pair, 0)) || + !PyString_Check(class_name = PyTuple_GET_ITEM(pair, 1))) { + Py_DECREF(py_code); + PyErr_Format(PyExc_ValueError, "_inverted_registry[%ld] " + "isn't a 2-tuple of strings", code); + return -1; + } + /* Load the object. */ + obj = find_class(module_name, class_name, self->find_class); + if (obj == NULL) { + Py_DECREF(py_code); + return -1; + } + /* Cache code -> obj. */ + code = PyDict_SetItem(extension_cache, py_code, obj); + Py_DECREF(py_code); + if (code < 0) { + Py_DECREF(obj); + return -1; + } + PDATA_PUSH(self->stack, obj, -1); + return 0; } static int -load_put(Unpicklerobject *self) +load_put(Unpicklerobject * self) { - PyObject *py_str = 0, *value = 0; - int len, l; - char *s; + PyObject *py_str = 0, *value = 0; + int len, l; + char *s; - if ((l = self->readline_func(self, &s)) < 0) return -1; - if (l < 2) return bad_readline(); - if (!( len=self->stack->length )) return stackUnderflow(); - if (!( py_str = PyString_FromStringAndSize(s, l - 1))) return -1; - value=self->stack->data[len-1]; - l=PyDict_SetItem(self->memo, py_str, value); - Py_DECREF(py_str); - return l; + if ((l = self->readline_func(self, &s)) < 0) + return -1; + if (l < 2) + return bad_readline(); + if (!(len = self->stack->length)) + return stackUnderflow(); + if (!(py_str = PyString_FromStringAndSize(s, l - 1))) + return -1; + value = self->stack->data[len - 1]; + l = PyDict_SetItem(self->memo, py_str, value); + Py_DECREF(py_str); + return l; } static int -load_binput(Unpicklerobject *self) +load_binput(Unpicklerobject * self) { - PyObject *py_key = 0, *value = 0; - unsigned char key; - char *s; - int len; + PyObject *py_key = 0, *value = 0; + unsigned char key; + char *s; + int len; - if (self->read_func(self, &s, 1) < 0) return -1; - if (!( (len=self->stack->length) > 0 )) return stackUnderflow(); + if (self->read_func(self, &s, 1) < 0) + return -1; + if (!((len = self->stack->length) > 0)) + return stackUnderflow(); - key = (unsigned char)s[0]; + key = (unsigned char) s[0]; - if (!( py_key = PyInt_FromLong((long)key))) return -1; - value=self->stack->data[len-1]; - len=PyDict_SetItem(self->memo, py_key, value); - Py_DECREF(py_key); - return len; + if (!(py_key = PyInt_FromLong((long) key))) + return -1; + value = self->stack->data[len - 1]; + len = PyDict_SetItem(self->memo, py_key, value); + Py_DECREF(py_key); + return len; } static int -load_long_binput(Unpicklerobject *self) +load_long_binput(Unpicklerobject * self) { - PyObject *py_key = 0, *value = 0; - long key; - unsigned char c; - char *s; - int len; + PyObject *py_key = 0, *value = 0; + long key; + unsigned char c; + char *s; + int len; - if (self->read_func(self, &s, 4) < 0) return -1; - if (!( len=self->stack->length )) return stackUnderflow(); + if (self->read_func(self, &s, 4) < 0) + return -1; + if (!(len = self->stack->length)) + return stackUnderflow(); - c = (unsigned char)s[0]; - key = (long)c; - c = (unsigned char)s[1]; - key |= (long)c << 8; - c = (unsigned char)s[2]; - key |= (long)c << 16; - c = (unsigned char)s[3]; - key |= (long)c << 24; + c = (unsigned char) s[0]; + key = (long) c; + c = (unsigned char) s[1]; + key |= (long) c << 8; + c = (unsigned char) s[2]; + key |= (long) c << 16; + c = (unsigned char) s[3]; + key |= (long) c << 24; - if (!( py_key = PyInt_FromLong(key))) return -1; - value=self->stack->data[len-1]; - len=PyDict_SetItem(self->memo, py_key, value); - Py_DECREF(py_key); - return len; + if (!(py_key = PyInt_FromLong(key))) + return -1; + value = self->stack->data[len - 1]; + len = PyDict_SetItem(self->memo, py_key, value); + Py_DECREF(py_key); + return len; } static int -do_append(Unpicklerobject *self, int x) +do_append(Unpicklerobject * self, int x) { - PyObject *value = 0, *list = 0, *append_method = 0; - int len, i; + PyObject *value = 0, *list = 0, *append_method = 0; + int len, i; - len=self->stack->length; - if (!( len >= x && x > 0 )) return stackUnderflow(); - /* nothing to do */ - if (len==x) return 0; + len = self->stack->length; + if (!(len >= x && x > 0)) + return stackUnderflow(); + /* nothing to do */ + if (len == x) + return 0; - list=self->stack->data[x-1]; + list = self->stack->data[x - 1]; - if (PyList_Check(list)) { - PyObject *slice; - int list_len; + if (PyList_Check(list)) { + PyObject *slice; + int list_len; - slice=Pdata_popList(self->stack, x); - if (! slice) return -1; - list_len = PyList_GET_SIZE(list); - i=PyList_SetSlice(list, list_len, list_len, slice); - Py_DECREF(slice); - return i; - } - else { + slice = Pdata_popList(self->stack, x); + if (!slice) + return -1; + list_len = PyList_GET_SIZE(list); + i = PyList_SetSlice(list, list_len, list_len, slice); + Py_DECREF(slice); + return i; + } + else { - if (!( append_method = PyObject_GetAttr(list, append_str))) - return -1; + if (!(append_method = PyObject_GetAttr(list, append_str))) + return -1; - for (i = x; i < len; i++) { - PyObject *junk; + for (i = x; i < len; i++) { + PyObject *junk; - value=self->stack->data[i]; - junk=0; - ARG_TUP(self, value); - if (self->arg) { - junk = PyObject_Call(append_method, self->arg, - NULL); - FREE_ARG_TUP(self); - } - if (! junk) { - Pdata_clear(self->stack, i+1); - self->stack->length=x; - Py_DECREF(append_method); - return -1; - } - Py_DECREF(junk); - } - self->stack->length=x; + value = self->stack->data[i]; + junk = 0; + ARG_TUP(self, value); + if (self->arg) { + junk = PyObject_Call(append_method, self->arg, NULL); + FREE_ARG_TUP(self); + } + if (!junk) { + Pdata_clear(self->stack, i + 1); + self->stack->length = x; Py_DECREF(append_method); + return -1; + } + Py_DECREF(junk); } + self->stack->length = x; + Py_DECREF(append_method); + } - return 0; + return 0; } static int -load_append(Unpicklerobject *self) +load_append(Unpicklerobject * self) { - return do_append(self, self->stack->length - 1); + return do_append(self, self->stack->length - 1); } static int -load_appends(Unpicklerobject *self) +load_appends(Unpicklerobject * self) { - return do_append(self, marker(self)); + return do_append(self, marker(self)); } static int -do_setitems(Unpicklerobject *self, int x) +do_setitems(Unpicklerobject * self, int x) { - PyObject *value = 0, *key = 0, *dict = 0; - int len, i, r=0; + PyObject *value = 0, *key = 0, *dict = 0; + int len, i, r = 0; - if (!( (len=self->stack->length) >= x - && x > 0 )) return stackUnderflow(); + if (!((len = self->stack->length) >= x && x > 0)) + return stackUnderflow(); - dict=self->stack->data[x-1]; + dict = self->stack->data[x - 1]; - for (i = x+1; i < len; i += 2) { - key =self->stack->data[i-1]; - value=self->stack->data[i ]; - if (PyObject_SetItem(dict, key, value) < 0) { - r=-1; - break; - } + for (i = x + 1; i < len; i += 2) { + key = self->stack->data[i - 1]; + value = self->stack->data[i]; + if (PyObject_SetItem(dict, key, value) < 0) { + r = -1; + break; } + } - Pdata_clear(self->stack, x); + Pdata_clear(self->stack, x); - return r; + return r; } static int -load_setitem(Unpicklerobject *self) +load_setitem(Unpicklerobject * self) { - return do_setitems(self, self->stack->length - 2); + return do_setitems(self, self->stack->length - 2); } static int -load_setitems(Unpicklerobject *self) +load_setitems(Unpicklerobject * self) { - return do_setitems(self, marker(self)); + return do_setitems(self, marker(self)); } static int -load_build(Unpicklerobject *self) +load_build(Unpicklerobject * self) { - PyObject *state, *inst, *slotstate; - PyObject *__setstate__; - PyObject *d_key, *d_value; - Py_ssize_t i; - int res = -1; + PyObject *state, *inst, *slotstate; + PyObject *__setstate__; + PyObject *d_key, *d_value; + Py_ssize_t i; + int res = -1; - /* Stack is ... instance, state. We want to leave instance at - * the stack top, possibly mutated via instance.__setstate__(state). - */ - if (self->stack->length < 2) - return stackUnderflow(); - PDATA_POP(self->stack, state); - if (state == NULL) - return -1; - inst = self->stack->data[self->stack->length - 1]; + /* Stack is ... instance, state. We want to leave instance at + * the stack top, possibly mutated via instance.__setstate__(state). + */ + if (self->stack->length < 2) + return stackUnderflow(); + PDATA_POP(self->stack, state); + if (state == NULL) + return -1; + inst = self->stack->data[self->stack->length - 1]; - __setstate__ = PyObject_GetAttr(inst, __setstate___str); - if (__setstate__ != NULL) { - PyObject *junk = NULL; + __setstate__ = PyObject_GetAttr(inst, __setstate___str); + if (__setstate__ != NULL) { + PyObject *junk = NULL; - /* The explicit __setstate__ is responsible for everything. */ - ARG_TUP(self, state); - if (self->arg) { - junk = PyObject_Call(__setstate__, self->arg, NULL); - FREE_ARG_TUP(self); - } - Py_DECREF(__setstate__); - if (junk == NULL) - return -1; - Py_DECREF(junk); - return 0; + /* The explicit __setstate__ is responsible for everything. */ + ARG_TUP(self, state); + if (self->arg) { + junk = PyObject_Call(__setstate__, self->arg, NULL); + FREE_ARG_TUP(self); } - if (!PyErr_ExceptionMatches(PyExc_AttributeError)) - return -1; - PyErr_Clear(); + Py_DECREF(__setstate__); + if (junk == NULL) + return -1; + Py_DECREF(junk); + return 0; + } + if (!PyErr_ExceptionMatches(PyExc_AttributeError)) + return -1; + PyErr_Clear(); - /* A default __setstate__. First see whether state embeds a - * slot state dict too (a proto 2 addition). - */ - if (PyTuple_Check(state) && PyTuple_Size(state) == 2) { - PyObject *temp = state; - state = PyTuple_GET_ITEM(temp, 0); - slotstate = PyTuple_GET_ITEM(temp, 1); - Py_INCREF(state); - Py_INCREF(slotstate); - Py_DECREF(temp); - } - else - slotstate = NULL; + /* A default __setstate__. First see whether state embeds a + * slot state dict too (a proto 2 addition). + */ + if (PyTuple_Check(state) && PyTuple_Size(state) == 2) { + PyObject *temp = state; + state = PyTuple_GET_ITEM(temp, 0); + slotstate = PyTuple_GET_ITEM(temp, 1); + Py_INCREF(state); + Py_INCREF(slotstate); + Py_DECREF(temp); + } + else + slotstate = NULL; - /* Set inst.__dict__ from the state dict (if any). */ - if (state != Py_None) { - PyObject *dict; - if (! PyDict_Check(state)) { - PyErr_SetString(UnpicklingError, "state is not a " - "dictionary"); - goto finally; - } - dict = PyObject_GetAttr(inst, __dict___str); - if (dict == NULL) - goto finally; - - i = 0; - while (PyDict_Next(state, &i, &d_key, &d_value)) { - if (PyObject_SetItem(dict, d_key, d_value) < 0) - goto finally; - } - Py_DECREF(dict); + /* Set inst.__dict__ from the state dict (if any). */ + if (state != Py_None) { + PyObject *dict; + if (!PyDict_Check(state)) { + PyErr_SetString(UnpicklingError, "state is not a " "dictionary"); + goto finally; + } + dict = PyObject_GetAttr(inst, __dict___str); + if (dict == NULL) + goto finally; + + i = 0; + while (PyDict_Next(state, &i, &d_key, &d_value)) { + if (PyObject_SetItem(dict, d_key, d_value) < 0) + goto finally; } + Py_DECREF(dict); + } - /* Also set instance attributes from the slotstate dict (if any). */ - if (slotstate != NULL) { - if (! PyDict_Check(slotstate)) { - PyErr_SetString(UnpicklingError, "slot state is not " - "a dictionary"); - goto finally; - } - i = 0; - while (PyDict_Next(slotstate, &i, &d_key, &d_value)) { - if (PyObject_SetAttr(inst, d_key, d_value) < 0) - goto finally; - } + /* Also set instance attributes from the slotstate dict (if any). */ + if (slotstate != NULL) { + if (!PyDict_Check(slotstate)) { + PyErr_SetString(UnpicklingError, "slot state is not " + "a dictionary"); + goto finally; } - res = 0; + i = 0; + while (PyDict_Next(slotstate, &i, &d_key, &d_value)) { + if (PyObject_SetAttr(inst, d_key, d_value) < 0) + goto finally; + } + } + res = 0; finally: - Py_DECREF(state); - Py_XDECREF(slotstate); - return res; + Py_DECREF(state); + Py_XDECREF(slotstate); + return res; } static int -load_mark(Unpicklerobject *self) +load_mark(Unpicklerobject * self) { - int s; + int s; - /* Note that we split the (pickle.py) stack into two stacks, an - object stack and a mark stack. Here we push a mark onto the - mark stack. - */ + /* Note that we split the (pickle.py) stack into two stacks, an + * object stack and a mark stack. Here we push a mark onto the + * mark stack. + */ - if ((self->num_marks + 1) >= self->marks_size) { - int *marks; - s=self->marks_size+20; - if (s <= self->num_marks) s=self->num_marks + 1; - if (self->marks == NULL) - marks=(int *)malloc(s * sizeof(int)); - else - marks=(int *)realloc(self->marks, - s * sizeof(int)); - if (!marks) { - PyErr_NoMemory(); - return -1; - } - self->marks = marks; - self->marks_size = s; + if ((self->num_marks + 1) >= self->marks_size) { + int *marks; + s = self->marks_size + 20; + if (s <= self->num_marks) + s = self->num_marks + 1; + if (self->marks == NULL) + marks = (int *) malloc(s * sizeof(int)); + else + marks = (int *) realloc(self->marks, s * sizeof(int)); + if (!marks) { + PyErr_NoMemory(); + return -1; } + self->marks = marks; + self->marks_size = s; + } - self->marks[self->num_marks++] = self->stack->length; + self->marks[self->num_marks++] = self->stack->length; - return 0; + return 0; } static int -load_reduce(Unpicklerobject *self) +load_reduce(Unpicklerobject * self) { - PyObject *callable = 0, *arg_tup = 0, *ob = 0; + PyObject *callable = 0, *arg_tup = 0, *ob = 0; - PDATA_POP(self->stack, arg_tup); - if (! arg_tup) return -1; - PDATA_POP(self->stack, callable); - if (callable) { - ob = Instance_New(callable, arg_tup); - Py_DECREF(callable); - } - Py_DECREF(arg_tup); + PDATA_POP(self->stack, arg_tup); + if (!arg_tup) + return -1; + PDATA_POP(self->stack, callable); + if (callable) { + ob = Instance_New(callable, arg_tup); + Py_DECREF(callable); + } + Py_DECREF(arg_tup); - if (! ob) return -1; + if (!ob) + return -1; - PDATA_PUSH(self->stack, ob, -1); - return 0; + PDATA_PUSH(self->stack, ob, -1); + return 0; } /* Just raises an error if we don't know the protocol specified. PROTO * is the first opcode for protocols >= 2. */ static int -load_proto(Unpicklerobject *self) +load_proto(Unpicklerobject * self) { - int i; - char *protobyte; + int i; + char *protobyte; - i = self->read_func(self, &protobyte, 1); - if (i < 0) - return -1; + i = self->read_func(self, &protobyte, 1); + if (i < 0) + return -1; - i = calc_binint(protobyte, 1); - /* No point checking for < 0, since calc_binint returns an unsigned - * int when chewing on 1 byte. - */ - assert(i >= 0); - if (i <= HIGHEST_PROTOCOL) - return 0; + i = calc_binint(protobyte, 1); + /* No point checking for < 0, since calc_binint returns an unsigned + * int when chewing on 1 byte. + */ + assert(i >= 0); + if (i <= HIGHEST_PROTOCOL) + return 0; + + PyErr_Format(PyExc_ValueError, "unsupported pickle protocol: %d", i); + return -1; +} + +static PyObject * +load(Unpicklerobject * self) +{ + PyObject *err = 0, *val = 0; + char *s; + + self->num_marks = 0; + if (self->stack->length) + Pdata_clear(self->stack, 0); + + while (1) { + if (self->read_func(self, &s, 1) < 0) + break; + + switch (s[0]) { + case NONE: + if (load_none(self) < 0) + break; + continue; + + case BININT: + if (load_binint(self) < 0) + break; + continue; + + case BININT1: + if (load_binint1(self) < 0) + break; + continue; + + case BININT2: + if (load_binint2(self) < 0) + break; + continue; + + case INT: + if (load_int(self) < 0) + break; + continue; + + case LONG: + if (load_long(self) < 0) + break; + continue; + + case LONG1: + if (load_counted_long(self, 1) < 0) + break; + continue; + + case LONG4: + if (load_counted_long(self, 4) < 0) + break; + continue; + + case FLOAT: + if (load_float(self) < 0) + break; + continue; + + case BINFLOAT: + if (load_binfloat(self) < 0) + break; + continue; + + case BINSTRING: + if (load_binstring(self) < 0) + break; + continue; + + case SHORT_BINSTRING: + if (load_short_binstring(self) < 0) + break; + continue; + + case STRING: + if (load_string(self) < 0) + break; + continue; + +#ifdef Py_USING_UNICODE + case UNICODE: + if (load_unicode(self) < 0) + break; + continue; + + case BINUNICODE: + if (load_binunicode(self) < 0) + break; + continue; +#endif + + case EMPTY_TUPLE: + if (load_counted_tuple(self, 0) < 0) + break; + continue; + + case TUPLE1: + if (load_counted_tuple(self, 1) < 0) + break; + continue; + + case TUPLE2: + if (load_counted_tuple(self, 2) < 0) + break; + continue; + + case TUPLE3: + if (load_counted_tuple(self, 3) < 0) + break; + continue; + + case TUPLE: + if (load_tuple(self) < 0) + break; + continue; + + case EMPTY_LIST: + if (load_empty_list(self) < 0) + break; + continue; + + case LIST: + if (load_list(self) < 0) + break; + continue; + + case EMPTY_DICT: + if (load_empty_dict(self) < 0) + break; + continue; + + case DICT: + if (load_dict(self) < 0) + break; + continue; + + case OBJ: + if (load_obj(self) < 0) + break; + continue; + + case INST: + if (load_inst(self) < 0) + break; + continue; + + case NEWOBJ: + if (load_newobj(self) < 0) + break; + continue; + + case GLOBAL: + if (load_global(self) < 0) + break; + continue; + + case APPEND: + if (load_append(self) < 0) + break; + continue; + + case APPENDS: + if (load_appends(self) < 0) + break; + continue; + + case BUILD: + if (load_build(self) < 0) + break; + continue; + + case DUP: + if (load_dup(self) < 0) + break; + continue; + + case BINGET: + if (load_binget(self) < 0) + break; + continue; + + case LONG_BINGET: + if (load_long_binget(self) < 0) + break; + continue; + + case GET: + if (load_get(self) < 0) + break; + continue; + + case EXT1: + if (load_extension(self, 1) < 0) + break; + continue; + + case EXT2: + if (load_extension(self, 2) < 0) + break; + continue; + + case EXT4: + if (load_extension(self, 4) < 0) + break; + continue; + case MARK: + if (load_mark(self) < 0) + break; + continue; + + case BINPUT: + if (load_binput(self) < 0) + break; + continue; + + case LONG_BINPUT: + if (load_long_binput(self) < 0) + break; + continue; + + case PUT: + if (load_put(self) < 0) + break; + continue; + + case POP: + if (load_pop(self) < 0) + break; + continue; + + case POP_MARK: + if (load_pop_mark(self) < 0) + break; + continue; + + case SETITEM: + if (load_setitem(self) < 0) + break; + continue; + + case SETITEMS: + if (load_setitems(self) < 0) + break; + continue; - PyErr_Format(PyExc_ValueError, "unsupported pickle protocol: %d", i); - return -1; -} + case STOP: + break; -static PyObject * -load(Unpicklerobject *self) -{ - PyObject *err = 0, *val = 0; - char *s; + case PERSID: + if (load_persid(self) < 0) + break; + continue; - self->num_marks = 0; - if (self->stack->length) Pdata_clear(self->stack, 0); + case BINPERSID: + if (load_binpersid(self) < 0) + break; + continue; - while (1) { - if (self->read_func(self, &s, 1) < 0) - break; - - switch (s[0]) { - case NONE: - if (load_none(self) < 0) - break; - continue; - - case BININT: - if (load_binint(self) < 0) - break; - continue; - - case BININT1: - if (load_binint1(self) < 0) - break; - continue; - - case BININT2: - if (load_binint2(self) < 0) - break; - continue; - - case INT: - if (load_int(self) < 0) - break; - continue; - - case LONG: - if (load_long(self) < 0) - break; - continue; - - case LONG1: - if (load_counted_long(self, 1) < 0) - break; - continue; - - case LONG4: - if (load_counted_long(self, 4) < 0) - break; - continue; - - case FLOAT: - if (load_float(self) < 0) - break; - continue; - - case BINFLOAT: - if (load_binfloat(self) < 0) - break; - continue; - - case BINSTRING: - if (load_binstring(self) < 0) - break; - continue; - - case SHORT_BINSTRING: - if (load_short_binstring(self) < 0) - break; - continue; - - case STRING: - if (load_string(self) < 0) - break; - continue; + case REDUCE: + if (load_reduce(self) < 0) + break; + continue; -#ifdef Py_USING_UNICODE - case UNICODE: - if (load_unicode(self) < 0) - break; - continue; - - case BINUNICODE: - if (load_binunicode(self) < 0) - break; - continue; -#endif + case PROTO: + if (load_proto(self) < 0) + break; + continue; - case EMPTY_TUPLE: - if (load_counted_tuple(self, 0) < 0) - break; - continue; - - case TUPLE1: - if (load_counted_tuple(self, 1) < 0) - break; - continue; - - case TUPLE2: - if (load_counted_tuple(self, 2) < 0) - break; - continue; - - case TUPLE3: - if (load_counted_tuple(self, 3) < 0) - break; - continue; - - case TUPLE: - if (load_tuple(self) < 0) - break; - continue; - - case EMPTY_LIST: - if (load_empty_list(self) < 0) - break; - continue; - - case LIST: - if (load_list(self) < 0) - break; - continue; - - case EMPTY_DICT: - if (load_empty_dict(self) < 0) - break; - continue; - - case DICT: - if (load_dict(self) < 0) - break; - continue; - - case OBJ: - if (load_obj(self) < 0) - break; - continue; - - case INST: - if (load_inst(self) < 0) - break; - continue; - - case NEWOBJ: - if (load_newobj(self) < 0) - break; - continue; - - case GLOBAL: - if (load_global(self) < 0) - break; - continue; - - case APPEND: - if (load_append(self) < 0) - break; - continue; - - case APPENDS: - if (load_appends(self) < 0) - break; - continue; - - case BUILD: - if (load_build(self) < 0) - break; - continue; - - case DUP: - if (load_dup(self) < 0) - break; - continue; - - case BINGET: - if (load_binget(self) < 0) - break; - continue; - - case LONG_BINGET: - if (load_long_binget(self) < 0) - break; - continue; - - case GET: - if (load_get(self) < 0) - break; - continue; - - case EXT1: - if (load_extension(self, 1) < 0) - break; - continue; - - case EXT2: - if (load_extension(self, 2) < 0) - break; - continue; - - case EXT4: - if (load_extension(self, 4) < 0) - break; - continue; - case MARK: - if (load_mark(self) < 0) - break; - continue; - - case BINPUT: - if (load_binput(self) < 0) - break; - continue; - - case LONG_BINPUT: - if (load_long_binput(self) < 0) - break; - continue; - - case PUT: - if (load_put(self) < 0) - break; - continue; - - case POP: - if (load_pop(self) < 0) - break; - continue; - - case POP_MARK: - if (load_pop_mark(self) < 0) - break; - continue; - - case SETITEM: - if (load_setitem(self) < 0) - break; - continue; - - case SETITEMS: - if (load_setitems(self) < 0) - break; - continue; - - case STOP: - break; - - case PERSID: - if (load_persid(self) < 0) - break; - continue; - - case BINPERSID: - if (load_binpersid(self) < 0) - break; - continue; - - case REDUCE: - if (load_reduce(self) < 0) - break; - continue; - - case PROTO: - if (load_proto(self) < 0) - break; - continue; - - case NEWTRUE: - if (load_bool(self, Py_True) < 0) - break; - continue; - - case NEWFALSE: - if (load_bool(self, Py_False) < 0) - break; - continue; - - case '\0': - /* end of file */ - PyErr_SetNone(PyExc_EOFError); - break; - - default: - cPickle_ErrFormat(UnpicklingError, - "invalid load key, '%s'.", - "c", s[0]); - return NULL; - } + case NEWTRUE: + if (load_bool(self, Py_True) < 0) + break; + continue; + case NEWFALSE: + if (load_bool(self, Py_False) < 0) break; + continue; + + case '\0': + /* end of file */ + PyErr_SetNone(PyExc_EOFError); + break; + + default: + cPickle_ErrFormat(UnpicklingError, + "invalid load key, '%s'.", "c", s[0]); + return NULL; } - if ((err = PyErr_Occurred())) { - if (err == PyExc_EOFError) { - PyErr_SetNone(PyExc_EOFError); - } - return NULL; + break; + } + + if ((err = PyErr_Occurred())) { + if (err == PyExc_EOFError) { + PyErr_SetNone(PyExc_EOFError); } + return NULL; + } - PDATA_POP(self->stack, val); - return val; + PDATA_POP(self->stack, val); + return val; } @@ -4564,630 +4629,639 @@ find persistent references. */ static int -noload_obj(Unpicklerobject *self) +noload_obj(Unpicklerobject * self) { - int i; + int i; - if ((i = marker(self)) < 0) return -1; - return Pdata_clear(self->stack, i+1); + if ((i = marker(self)) < 0) + return -1; + return Pdata_clear(self->stack, i + 1); } static int -noload_inst(Unpicklerobject *self) +noload_inst(Unpicklerobject * self) { - int i; - char *s; + int i; + char *s; - if ((i = marker(self)) < 0) return -1; - Pdata_clear(self->stack, i); - if (self->readline_func(self, &s) < 0) return -1; - if (self->readline_func(self, &s) < 0) return -1; - PDATA_APPEND(self->stack, Py_None, -1); - return 0; + if ((i = marker(self)) < 0) + return -1; + Pdata_clear(self->stack, i); + if (self->readline_func(self, &s) < 0) + return -1; + if (self->readline_func(self, &s) < 0) + return -1; + PDATA_APPEND(self->stack, Py_None, -1); + return 0; } static int -noload_newobj(Unpicklerobject *self) +noload_newobj(Unpicklerobject * self) { - PyObject *obj; + PyObject *obj; - PDATA_POP(self->stack, obj); /* pop argtuple */ - if (obj == NULL) return -1; - Py_DECREF(obj); + PDATA_POP(self->stack, obj); /* pop argtuple */ + if (obj == NULL) + return -1; + Py_DECREF(obj); - PDATA_POP(self->stack, obj); /* pop cls */ - if (obj == NULL) return -1; - Py_DECREF(obj); + PDATA_POP(self->stack, obj); /* pop cls */ + if (obj == NULL) + return -1; + Py_DECREF(obj); - PDATA_APPEND(self->stack, Py_None, -1); - return 0; + PDATA_APPEND(self->stack, Py_None, -1); + return 0; } static int -noload_global(Unpicklerobject *self) +noload_global(Unpicklerobject * self) { - char *s; + char *s; - if (self->readline_func(self, &s) < 0) return -1; - if (self->readline_func(self, &s) < 0) return -1; - PDATA_APPEND(self->stack, Py_None,-1); - return 0; + if (self->readline_func(self, &s) < 0) + return -1; + if (self->readline_func(self, &s) < 0) + return -1; + PDATA_APPEND(self->stack, Py_None, -1); + return 0; } static int -noload_reduce(Unpicklerobject *self) +noload_reduce(Unpicklerobject * self) { - if (self->stack->length < 2) return stackUnderflow(); - Pdata_clear(self->stack, self->stack->length-2); - PDATA_APPEND(self->stack, Py_None,-1); - return 0; + if (self->stack->length < 2) + return stackUnderflow(); + Pdata_clear(self->stack, self->stack->length - 2); + PDATA_APPEND(self->stack, Py_None, -1); + return 0; } static int -noload_build(Unpicklerobject *self) { +noload_build(Unpicklerobject * self) +{ - if (self->stack->length < 1) return stackUnderflow(); - Pdata_clear(self->stack, self->stack->length-1); - return 0; + if (self->stack->length < 1) + return stackUnderflow(); + Pdata_clear(self->stack, self->stack->length - 1); + return 0; } static int -noload_extension(Unpicklerobject *self, int nbytes) +noload_extension(Unpicklerobject * self, int nbytes) { - char *codebytes; + char *codebytes; - assert(nbytes == 1 || nbytes == 2 || nbytes == 4); - if (self->read_func(self, &codebytes, nbytes) < 0) return -1; - PDATA_APPEND(self->stack, Py_None, -1); - return 0; + assert(nbytes == 1 || nbytes == 2 || nbytes == 4); + if (self->read_func(self, &codebytes, nbytes) < 0) + return -1; + PDATA_APPEND(self->stack, Py_None, -1); + return 0; } static PyObject * -noload(Unpicklerobject *self) +noload(Unpicklerobject * self) { - PyObject *err = 0, *val = 0; - char *s; + PyObject *err = 0, *val = 0; + char *s; - self->num_marks = 0; - Pdata_clear(self->stack, 0); + self->num_marks = 0; + Pdata_clear(self->stack, 0); + + while (1) { + if (self->read_func(self, &s, 1) < 0) + break; + + switch (s[0]) { + case NONE: + if (load_none(self) < 0) + break; + continue; + + case BININT: + if (load_binint(self) < 0) + break; + continue; + + case BININT1: + if (load_binint1(self) < 0) + break; + continue; + + case BININT2: + if (load_binint2(self) < 0) + break; + continue; + + case INT: + if (load_int(self) < 0) + break; + continue; + + case LONG: + if (load_long(self) < 0) + break; + continue; + + case LONG1: + if (load_counted_long(self, 1) < 0) + break; + continue; + + case LONG4: + if (load_counted_long(self, 4) < 0) + break; + continue; + + case FLOAT: + if (load_float(self) < 0) + break; + continue; + + case BINFLOAT: + if (load_binfloat(self) < 0) + break; + continue; + + case BINSTRING: + if (load_binstring(self) < 0) + break; + continue; + + case SHORT_BINSTRING: + if (load_short_binstring(self) < 0) + break; + continue; - while (1) { - if (self->read_func(self, &s, 1) < 0) - break; - - switch (s[0]) { - case NONE: - if (load_none(self) < 0) - break; - continue; - - case BININT: - if (load_binint(self) < 0) - break; - continue; - - case BININT1: - if (load_binint1(self) < 0) - break; - continue; - - case BININT2: - if (load_binint2(self) < 0) - break; - continue; - - case INT: - if (load_int(self) < 0) - break; - continue; - - case LONG: - if (load_long(self) < 0) - break; - continue; - - case LONG1: - if (load_counted_long(self, 1) < 0) - break; - continue; - - case LONG4: - if (load_counted_long(self, 4) < 0) - break; - continue; - - case FLOAT: - if (load_float(self) < 0) - break; - continue; - - case BINFLOAT: - if (load_binfloat(self) < 0) - break; - continue; - - case BINSTRING: - if (load_binstring(self) < 0) - break; - continue; - - case SHORT_BINSTRING: - if (load_short_binstring(self) < 0) - break; - continue; - - case STRING: - if (load_string(self) < 0) - break; - continue; + case STRING: + if (load_string(self) < 0) + break; + continue; #ifdef Py_USING_UNICODE - case UNICODE: - if (load_unicode(self) < 0) - break; - continue; - - case BINUNICODE: - if (load_binunicode(self) < 0) - break; - continue; + case UNICODE: + if (load_unicode(self) < 0) + break; + continue; + + case BINUNICODE: + if (load_binunicode(self) < 0) + break; + continue; #endif - case EMPTY_TUPLE: - if (load_counted_tuple(self, 0) < 0) - break; - continue; - - case TUPLE1: - if (load_counted_tuple(self, 1) < 0) - break; - continue; - - case TUPLE2: - if (load_counted_tuple(self, 2) < 0) - break; - continue; - - case TUPLE3: - if (load_counted_tuple(self, 3) < 0) - break; - continue; - - case TUPLE: - if (load_tuple(self) < 0) - break; - continue; - - case EMPTY_LIST: - if (load_empty_list(self) < 0) - break; - continue; - - case LIST: - if (load_list(self) < 0) - break; - continue; - - case EMPTY_DICT: - if (load_empty_dict(self) < 0) - break; - continue; - - case DICT: - if (load_dict(self) < 0) - break; - continue; - - case OBJ: - if (noload_obj(self) < 0) - break; - continue; - - case INST: - if (noload_inst(self) < 0) - break; - continue; - - case NEWOBJ: - if (noload_newobj(self) < 0) - break; - continue; - - case GLOBAL: - if (noload_global(self) < 0) - break; - continue; - - case APPEND: - if (load_append(self) < 0) - break; - continue; - - case APPENDS: - if (load_appends(self) < 0) - break; - continue; - - case BUILD: - if (noload_build(self) < 0) - break; - continue; - - case DUP: - if (load_dup(self) < 0) - break; - continue; - - case BINGET: - if (load_binget(self) < 0) - break; - continue; - - case LONG_BINGET: - if (load_long_binget(self) < 0) - break; - continue; - - case GET: - if (load_get(self) < 0) - break; - continue; - - case EXT1: - if (noload_extension(self, 1) < 0) - break; - continue; - - case EXT2: - if (noload_extension(self, 2) < 0) - break; - continue; - - case EXT4: - if (noload_extension(self, 4) < 0) - break; - continue; - - case MARK: - if (load_mark(self) < 0) - break; - continue; - - case BINPUT: - if (load_binput(self) < 0) - break; - continue; - - case LONG_BINPUT: - if (load_long_binput(self) < 0) - break; - continue; - - case PUT: - if (load_put(self) < 0) - break; - continue; - - case POP: - if (load_pop(self) < 0) - break; - continue; - - case POP_MARK: - if (load_pop_mark(self) < 0) - break; - continue; - - case SETITEM: - if (load_setitem(self) < 0) - break; - continue; - - case SETITEMS: - if (load_setitems(self) < 0) - break; - continue; - - case STOP: - break; - - case PERSID: - if (load_persid(self) < 0) - break; - continue; - - case BINPERSID: - if (load_binpersid(self) < 0) - break; - continue; - - case REDUCE: - if (noload_reduce(self) < 0) - break; - continue; - - case PROTO: - if (load_proto(self) < 0) - break; - continue; - - case NEWTRUE: - if (load_bool(self, Py_True) < 0) - break; - continue; - - case NEWFALSE: - if (load_bool(self, Py_False) < 0) - break; - continue; - default: - cPickle_ErrFormat(UnpicklingError, - "invalid load key, '%s'.", - "c", s[0]); - return NULL; - } + case EMPTY_TUPLE: + if (load_counted_tuple(self, 0) < 0) + break; + continue; + + case TUPLE1: + if (load_counted_tuple(self, 1) < 0) + break; + continue; + + case TUPLE2: + if (load_counted_tuple(self, 2) < 0) + break; + continue; + + case TUPLE3: + if (load_counted_tuple(self, 3) < 0) + break; + continue; + + case TUPLE: + if (load_tuple(self) < 0) + break; + continue; + + case EMPTY_LIST: + if (load_empty_list(self) < 0) + break; + continue; + + case LIST: + if (load_list(self) < 0) + break; + continue; + + case EMPTY_DICT: + if (load_empty_dict(self) < 0) + break; + continue; + + case DICT: + if (load_dict(self) < 0) + break; + continue; + + case OBJ: + if (noload_obj(self) < 0) + break; + continue; + + case INST: + if (noload_inst(self) < 0) + break; + continue; + + case NEWOBJ: + if (noload_newobj(self) < 0) + break; + continue; + + case GLOBAL: + if (noload_global(self) < 0) + break; + continue; + + case APPEND: + if (load_append(self) < 0) + break; + continue; + + case APPENDS: + if (load_appends(self) < 0) + break; + continue; + + case BUILD: + if (noload_build(self) < 0) + break; + continue; + + case DUP: + if (load_dup(self) < 0) + break; + continue; + + case BINGET: + if (load_binget(self) < 0) + break; + continue; + + case LONG_BINGET: + if (load_long_binget(self) < 0) + break; + continue; + + case GET: + if (load_get(self) < 0) + break; + continue; + + case EXT1: + if (noload_extension(self, 1) < 0) + break; + continue; + + case EXT2: + if (noload_extension(self, 2) < 0) + break; + continue; + + case EXT4: + if (noload_extension(self, 4) < 0) + break; + continue; + + case MARK: + if (load_mark(self) < 0) + break; + continue; + + case BINPUT: + if (load_binput(self) < 0) + break; + continue; + + case LONG_BINPUT: + if (load_long_binput(self) < 0) + break; + continue; + + case PUT: + if (load_put(self) < 0) + break; + continue; + + case POP: + if (load_pop(self) < 0) + break; + continue; + + case POP_MARK: + if (load_pop_mark(self) < 0) + break; + continue; + + case SETITEM: + if (load_setitem(self) < 0) + break; + continue; + + case SETITEMS: + if (load_setitems(self) < 0) + break; + continue; + + case STOP: + break; + + case PERSID: + if (load_persid(self) < 0) + break; + continue; + + case BINPERSID: + if (load_binpersid(self) < 0) + break; + continue; + + case REDUCE: + if (noload_reduce(self) < 0) + break; + continue; + case PROTO: + if (load_proto(self) < 0) break; + continue; + + case NEWTRUE: + if (load_bool(self, Py_True) < 0) + break; + continue; + + case NEWFALSE: + if (load_bool(self, Py_False) < 0) + break; + continue; + default: + cPickle_ErrFormat(UnpicklingError, + "invalid load key, '%s'.", "c", s[0]); + return NULL; } - if ((err = PyErr_Occurred())) { - if (err == PyExc_EOFError) { - PyErr_SetNone(PyExc_EOFError); - } - return NULL; + break; + } + + if ((err = PyErr_Occurred())) { + if (err == PyExc_EOFError) { + PyErr_SetNone(PyExc_EOFError); } + return NULL; + } - PDATA_POP(self->stack, val); - return val; + PDATA_POP(self->stack, val); + return val; } static PyObject * -Unpickler_load(Unpicklerobject *self, PyObject *unused) +Unpickler_load(Unpicklerobject * self, PyObject * unused) { - return load(self); + return load(self); } static PyObject * -Unpickler_noload(Unpicklerobject *self, PyObject *unused) +Unpickler_noload(Unpicklerobject * self, PyObject * unused) { - return noload(self); + return noload(self); } static struct PyMethodDef Unpickler_methods[] = { - {"load", (PyCFunction)Unpickler_load, METH_NOARGS, - PyDoc_STR("load() -- Load a pickle") - }, - {"noload", (PyCFunction)Unpickler_noload, METH_NOARGS, - PyDoc_STR( - "noload() -- not load a pickle, but go through most of the motions\n" - "\n" - "This function can be used to read past a pickle without instantiating\n" - "any objects or importing any modules. It can also be used to find all\n" - "persistent references without instantiating any objects or importing\n" - "any modules.\n") - }, - {NULL, NULL} /* sentinel */ + {"load", (PyCFunction) Unpickler_load, METH_NOARGS, + PyDoc_STR("load() -- Load a pickle") + }, + {"noload", (PyCFunction) Unpickler_noload, METH_NOARGS, + PyDoc_STR + ("noload() -- not load a pickle, but go through most of the motions\n" + "\n" + "This function can be used to read past a pickle without instantiating\n" + "any objects or importing any modules. It can also be used to find all\n" + "persistent references without instantiating any objects or importing\n" + "any modules.\n") + }, + {NULL, NULL} /* sentinel */ }; static Unpicklerobject * -newUnpicklerobject(PyObject *f) +newUnpicklerobject(PyObject * f) { - Unpicklerobject *self; - - if (!( self = PyObject_GC_New(Unpicklerobject, &Unpicklertype))) - return NULL; - - self->file = NULL; - self->arg = NULL; - self->stack = (Pdata*)Pdata_New(); - self->pers_func = NULL; - self->last_string = NULL; - self->marks = NULL; - self->num_marks = 0; - self->marks_size = 0; - self->buf_size = 0; - self->read = NULL; - self->readline = NULL; - self->find_class = NULL; - - if (!( self->memo = PyDict_New())) - goto err; + Unpicklerobject *self; - if (!self->stack) - goto err; + if (!(self = PyObject_GC_New(Unpicklerobject, &Unpicklertype))) + return NULL; - Py_INCREF(f); - self->file = f; + self->file = NULL; + self->arg = NULL; + self->stack = (Pdata *) Pdata_New(); + self->pers_func = NULL; + self->last_string = NULL; + self->marks = NULL; + self->num_marks = 0; + self->marks_size = 0; + self->buf_size = 0; + self->read = NULL; + self->readline = NULL; + self->find_class = NULL; + + if (!(self->memo = PyDict_New())) + goto err; + + if (!self->stack) + goto err; + + Py_INCREF(f); + self->file = f; + + /* Set read, readline based on type of f */ + if (PyFile_Check(f)) { + self->fp = PyFile_AsFile(f); + if (self->fp == NULL) { + PyErr_SetString(PyExc_ValueError, "I/O operation on closed file"); + goto err; + } + self->read_func = read_file; + self->readline_func = readline_file; + } + else if (PycStringIO_InputCheck(f)) { + self->fp = NULL; + self->read_func = read_cStringIO; + self->readline_func = readline_cStringIO; + } + else { - /* Set read, readline based on type of f */ - if (PyFile_Check(f)) { - self->fp = PyFile_AsFile(f); - if (self->fp == NULL) { - PyErr_SetString(PyExc_ValueError, - "I/O operation on closed file"); - goto err; - } - self->read_func = read_file; - self->readline_func = readline_file; - } - else if (PycStringIO_InputCheck(f)) { - self->fp = NULL; - self->read_func = read_cStringIO; - self->readline_func = readline_cStringIO; - } - else { + self->fp = NULL; + self->read_func = read_other; + self->readline_func = readline_other; - self->fp = NULL; - self->read_func = read_other; - self->readline_func = readline_other; - - if (!( (self->readline = PyObject_GetAttr(f, readline_str)) && - (self->read = PyObject_GetAttr(f, read_str)))) { - PyErr_Clear(); - PyErr_SetString( PyExc_TypeError, - "argument must have 'read' and " - "'readline' attributes" ); - goto err; - } + if (!((self->readline = PyObject_GetAttr(f, readline_str)) && + (self->read = PyObject_GetAttr(f, read_str)))) { + PyErr_Clear(); + PyErr_SetString(PyExc_TypeError, + "argument must have 'read' and " + "'readline' attributes"); + goto err; } - PyObject_GC_Track(self); + } + PyObject_GC_Track(self); - return self; + return self; err: - Py_DECREF((PyObject *)self); - return NULL; + Py_DECREF((PyObject *) self); + return NULL; } static PyObject * -get_Unpickler(PyObject *self, PyObject *file) +get_Unpickler(PyObject * self, PyObject * file) { - return (PyObject *)newUnpicklerobject(file); + return (PyObject *) newUnpicklerobject(file); } static void -Unpickler_dealloc(Unpicklerobject *self) +Unpickler_dealloc(Unpicklerobject * self) { - PyObject_GC_UnTrack((PyObject *)self); - Py_XDECREF(self->readline); - Py_XDECREF(self->read); - Py_XDECREF(self->file); - Py_XDECREF(self->memo); - Py_XDECREF(self->stack); - Py_XDECREF(self->pers_func); - Py_XDECREF(self->arg); - Py_XDECREF(self->last_string); - Py_XDECREF(self->find_class); - - if (self->marks) { - free(self->marks); - } - - if (self->buf_size) { - free(self->buf); - } - - self->ob_type->tp_free((PyObject *)self); + PyObject_GC_UnTrack((PyObject *) self); + Py_XDECREF(self->readline); + Py_XDECREF(self->read); + Py_XDECREF(self->file); + Py_XDECREF(self->memo); + Py_XDECREF(self->stack); + Py_XDECREF(self->pers_func); + Py_XDECREF(self->arg); + Py_XDECREF(self->last_string); + Py_XDECREF(self->find_class); + + if (self->marks) { + free(self->marks); + } + + if (self->buf_size) { + free(self->buf); + } + + self->ob_type->tp_free((PyObject *) self); } static int -Unpickler_traverse(Unpicklerobject *self, visitproc visit, void *arg) -{ - Py_VISIT(self->readline); - Py_VISIT(self->read); - Py_VISIT(self->file); - Py_VISIT(self->memo); - Py_VISIT(self->stack); - Py_VISIT(self->pers_func); - Py_VISIT(self->arg); - Py_VISIT(self->last_string); - Py_VISIT(self->find_class); - return 0; +Unpickler_traverse(Unpicklerobject * self, visitproc visit, void *arg) +{ + Py_VISIT(self->readline); + Py_VISIT(self->read); + Py_VISIT(self->file); + Py_VISIT(self->memo); + Py_VISIT(self->stack); + Py_VISIT(self->pers_func); + Py_VISIT(self->arg); + Py_VISIT(self->last_string); + Py_VISIT(self->find_class); + return 0; } static int -Unpickler_clear(Unpicklerobject *self) -{ - Py_CLEAR(self->readline); - Py_CLEAR(self->read); - Py_CLEAR(self->file); - Py_CLEAR(self->memo); - Py_CLEAR(self->stack); - Py_CLEAR(self->pers_func); - Py_CLEAR(self->arg); - Py_CLEAR(self->last_string); - Py_CLEAR(self->find_class); - return 0; +Unpickler_clear(Unpicklerobject * self) +{ + Py_CLEAR(self->readline); + Py_CLEAR(self->read); + Py_CLEAR(self->file); + Py_CLEAR(self->memo); + Py_CLEAR(self->stack); + Py_CLEAR(self->pers_func); + Py_CLEAR(self->arg); + Py_CLEAR(self->last_string); + Py_CLEAR(self->find_class); + return 0; } static PyObject * -Unpickler_getattr(Unpicklerobject *self, char *name) +Unpickler_getattr(Unpicklerobject * self, char *name) { - if (!strcmp(name, "persistent_load")) { - if (!self->pers_func) { - PyErr_SetString(PyExc_AttributeError, name); - return NULL; - } - - Py_INCREF(self->pers_func); - return self->pers_func; + if (!strcmp(name, "persistent_load")) { + if (!self->pers_func) { + PyErr_SetString(PyExc_AttributeError, name); + return NULL; } - if (!strcmp(name, "find_global")) { - if (!self->find_class) { - PyErr_SetString(PyExc_AttributeError, name); - return NULL; - } + Py_INCREF(self->pers_func); + return self->pers_func; + } - Py_INCREF(self->find_class); - return self->find_class; + if (!strcmp(name, "find_global")) { + if (!self->find_class) { + PyErr_SetString(PyExc_AttributeError, name); + return NULL; } - if (!strcmp(name, "memo")) { - if (!self->memo) { - PyErr_SetString(PyExc_AttributeError, name); - return NULL; - } + Py_INCREF(self->find_class); + return self->find_class; + } - Py_INCREF(self->memo); - return self->memo; + if (!strcmp(name, "memo")) { + if (!self->memo) { + PyErr_SetString(PyExc_AttributeError, name); + return NULL; } - if (!strcmp(name, "UnpicklingError")) { - Py_INCREF(UnpicklingError); - return UnpicklingError; - } + Py_INCREF(self->memo); + return self->memo; + } + + if (!strcmp(name, "UnpicklingError")) { + Py_INCREF(UnpicklingError); + return UnpicklingError; + } - return Py_FindMethod(Unpickler_methods, (PyObject *)self, name); + return Py_FindMethod(Unpickler_methods, (PyObject *) self, name); } static int -Unpickler_setattr(Unpicklerobject *self, char *name, PyObject *value) +Unpickler_setattr(Unpicklerobject * self, char *name, PyObject * value) { - if (!strcmp(name, "persistent_load")) { - Py_XDECREF(self->pers_func); - self->pers_func = value; - Py_XINCREF(value); - return 0; - } + if (!strcmp(name, "persistent_load")) { + Py_XDECREF(self->pers_func); + self->pers_func = value; + Py_XINCREF(value); + return 0; + } - if (!strcmp(name, "find_global")) { - Py_XDECREF(self->find_class); - self->find_class = value; - Py_XINCREF(value); - return 0; - } + if (!strcmp(name, "find_global")) { + Py_XDECREF(self->find_class); + self->find_class = value; + Py_XINCREF(value); + return 0; + } - if (! value) { - PyErr_SetString(PyExc_TypeError, - "attribute deletion is not supported"); - return -1; - } + if (!value) { + PyErr_SetString(PyExc_TypeError, + "attribute deletion is not supported"); + return -1; + } - if (strcmp(name, "memo") == 0) { - if (!PyDict_Check(value)) { - PyErr_SetString(PyExc_TypeError, - "memo must be a dictionary"); - return -1; - } - Py_XDECREF(self->memo); - self->memo = value; - Py_INCREF(value); - return 0; + if (strcmp(name, "memo") == 0) { + if (!PyDict_Check(value)) { + PyErr_SetString(PyExc_TypeError, "memo must be a dictionary"); + return -1; } + Py_XDECREF(self->memo); + self->memo = value; + Py_INCREF(value); + return 0; + } - PyErr_SetString(PyExc_AttributeError, name); - return -1; + PyErr_SetString(PyExc_AttributeError, name); + return -1; } /* --------------------------------------------------------------------------- @@ -5196,388 +5270,389 @@ /* dump(obj, file, protocol=0). */ static PyObject * -cpm_dump(PyObject *self, PyObject *args, PyObject *kwds) +cpm_dump(PyObject * self, PyObject * args, PyObject * kwds) { - static char *kwlist[] = {"obj", "file", "protocol", NULL}; - PyObject *ob, *file, *res = NULL; - Picklerobject *pickler = 0; - int proto = 0; + static char *kwlist[] = { "obj", "file", "protocol", NULL }; + PyObject *ob, *file, *res = NULL; + Picklerobject *pickler = 0; + int proto = 0; - if (!( PyArg_ParseTupleAndKeywords(args, kwds, "OO|i", kwlist, - &ob, &file, &proto))) - goto finally; + if (!(PyArg_ParseTupleAndKeywords(args, kwds, "OO|i", kwlist, + &ob, &file, &proto))) + goto finally; - if (!( pickler = newPicklerobject(file, proto))) - goto finally; + if (!(pickler = newPicklerobject(file, proto))) + goto finally; - if (dump(pickler, ob) < 0) - goto finally; + if (dump(pickler, ob) < 0) + goto finally; - Py_INCREF(Py_None); - res = Py_None; + Py_INCREF(Py_None); + res = Py_None; finally: - Py_XDECREF(pickler); + Py_XDECREF(pickler); - return res; + return res; } /* dumps(obj, protocol=0). */ static PyObject * -cpm_dumps(PyObject *self, PyObject *args, PyObject *kwds) +cpm_dumps(PyObject * self, PyObject * args, PyObject * kwds) { - static char *kwlist[] = {"obj", "protocol", NULL}; - PyObject *ob, *file = 0, *res = NULL; - Picklerobject *pickler = 0; - int proto = 0; + static char *kwlist[] = { "obj", "protocol", NULL }; + PyObject *ob, *file = 0, *res = NULL; + Picklerobject *pickler = 0; + int proto = 0; - if (!( PyArg_ParseTupleAndKeywords(args, kwds, "O|i:dumps", kwlist, - &ob, &proto))) - goto finally; + if (!(PyArg_ParseTupleAndKeywords(args, kwds, "O|i:dumps", kwlist, + &ob, &proto))) + goto finally; - if (!( file = PycStringIO->NewOutput(128))) - goto finally; + if (!(file = PycStringIO->NewOutput(128))) + goto finally; - if (!( pickler = newPicklerobject(file, proto))) - goto finally; + if (!(pickler = newPicklerobject(file, proto))) + goto finally; - if (dump(pickler, ob) < 0) - goto finally; + if (dump(pickler, ob) < 0) + goto finally; - res = PycStringIO->cgetvalue(file); + res = PycStringIO->cgetvalue(file); finally: - Py_XDECREF(pickler); - Py_XDECREF(file); + Py_XDECREF(pickler); + Py_XDECREF(file); - return res; + return res; } /* load(fileobj). */ static PyObject * -cpm_load(PyObject *self, PyObject *ob) +cpm_load(PyObject * self, PyObject * ob) { - Unpicklerobject *unpickler = 0; - PyObject *res = NULL; + Unpicklerobject *unpickler = 0; + PyObject *res = NULL; - if (!( unpickler = newUnpicklerobject(ob))) - goto finally; + if (!(unpickler = newUnpicklerobject(ob))) + goto finally; - res = load(unpickler); + res = load(unpickler); finally: - Py_XDECREF(unpickler); + Py_XDECREF(unpickler); - return res; + return res; } /* loads(string) */ static PyObject * -cpm_loads(PyObject *self, PyObject *args) +cpm_loads(PyObject * self, PyObject * args) { - PyObject *ob, *file = 0, *res = NULL; - Unpicklerobject *unpickler = 0; + PyObject *ob, *file = 0, *res = NULL; + Unpicklerobject *unpickler = 0; - if (!( PyArg_ParseTuple(args, "S:loads", &ob))) - goto finally; + if (!(PyArg_ParseTuple(args, "S:loads", &ob))) + goto finally; - if (!( file = PycStringIO->NewInput(ob))) - goto finally; + if (!(file = PycStringIO->NewInput(ob))) + goto finally; - if (!( unpickler = newUnpicklerobject(file))) - goto finally; + if (!(unpickler = newUnpicklerobject(file))) + goto finally; - res = load(unpickler); + res = load(unpickler); finally: - Py_XDECREF(file); - Py_XDECREF(unpickler); + Py_XDECREF(file); + Py_XDECREF(unpickler); - return res; + return res; } -PyDoc_STRVAR(Unpicklertype__doc__, -"Objects that know how to unpickle"); +PyDoc_STRVAR(Unpicklertype__doc__, "Objects that know how to unpickle"); static PyTypeObject Unpicklertype = { PyObject_HEAD_INIT(NULL) - 0, /*ob_size*/ - "cPickle.Unpickler", /*tp_name*/ - sizeof(Unpicklerobject), /*tp_basicsize*/ + 0, /*ob_size */ + "cPickle.Unpickler", /*tp_name */ + sizeof(Unpicklerobject), /*tp_basicsize */ 0, - (destructor)Unpickler_dealloc, /* tp_dealloc */ - 0, /* tp_print */ - (getattrfunc)Unpickler_getattr, /* tp_getattr */ - (setattrfunc)Unpickler_setattr, /* tp_setattr */ - 0, /* tp_compare */ - 0, /* tp_repr */ - 0, /* tp_as_number */ - 0, /* tp_as_sequence */ - 0, /* tp_as_mapping */ - 0, /* tp_hash */ - 0, /* tp_call */ - 0, /* tp_str */ - 0, /* tp_getattro */ - 0, /* tp_setattro */ - 0, /* tp_as_buffer */ + (destructor) Unpickler_dealloc, /* tp_dealloc */ + 0, /* tp_print */ + (getattrfunc) Unpickler_getattr, /* tp_getattr */ + (setattrfunc) Unpickler_setattr, /* tp_setattr */ + 0, /* tp_compare */ + 0, /* tp_repr */ + 0, /* tp_as_number */ + 0, /* tp_as_sequence */ + 0, /* tp_as_mapping */ + 0, /* tp_hash */ + 0, /* tp_call */ + 0, /* tp_str */ + 0, /* tp_getattro */ + 0, /* tp_setattro */ + 0, /* tp_as_buffer */ Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE | Py_TPFLAGS_HAVE_GC, - Unpicklertype__doc__, /* tp_doc */ - (traverseproc)Unpickler_traverse, /* tp_traverse */ - (inquiry)Unpickler_clear, /* tp_clear */ + Unpicklertype__doc__, /* tp_doc */ + (traverseproc) Unpickler_traverse, /* tp_traverse */ + (inquiry) Unpickler_clear, /* tp_clear */ }; static struct PyMethodDef cPickle_methods[] = { - {"dump", (PyCFunction)cpm_dump, METH_VARARGS | METH_KEYWORDS, - PyDoc_STR("dump(obj, file, protocol=0) -- " - "Write an object in pickle format to the given file.\n" - "\n" - "See the Pickler docstring for the meaning of optional argument proto.") - }, - - {"dumps", (PyCFunction)cpm_dumps, METH_VARARGS | METH_KEYWORDS, - PyDoc_STR("dumps(obj, protocol=0) -- " - "Return a string containing an object in pickle format.\n" - "\n" - "See the Pickler docstring for the meaning of optional argument proto.") - }, - - {"load", (PyCFunction)cpm_load, METH_O, - PyDoc_STR("load(file) -- Load a pickle from the given file")}, - - {"loads", (PyCFunction)cpm_loads, METH_VARARGS, - PyDoc_STR("loads(string) -- Load a pickle from the given string")}, - - {"Pickler", (PyCFunction)get_Pickler, METH_VARARGS | METH_KEYWORDS, - PyDoc_STR("Pickler(file, protocol=0) -- Create a pickler.\n" - "\n" - "This takes a file-like object for writing a pickle data stream.\n" - "The optional proto argument tells the pickler to use the given\n" - "protocol; supported protocols are 0, 1, 2. The default\n" - "protocol is 0, to be backwards compatible. (Protocol 0 is the\n" - "only protocol that can be written to a file opened in text\n" - "mode and read back successfully. When using a protocol higher\n" - "than 0, make sure the file is opened in binary mode, both when\n" - "pickling and unpickling.)\n" - "\n" - "Protocol 1 is more efficient than protocol 0; protocol 2 is\n" - "more efficient than protocol 1.\n" - "\n" - "Specifying a negative protocol version selects the highest\n" - "protocol version supported. The higher the protocol used, the\n" - "more recent the version of Python needed to read the pickle\n" - "produced.\n" - "\n" - "The file parameter must have a write() method that accepts a single\n" - "string argument. It can thus be an open file object, a StringIO\n" - "object, or any other custom object that meets this interface.\n") - }, + {"dump", (PyCFunction) cpm_dump, METH_VARARGS | METH_KEYWORDS, + PyDoc_STR("dump(obj, file, protocol=0) -- " + "Write an object in pickle format to the given file.\n" + "\n" + "See the Pickler docstring for the meaning of optional argument proto.") + }, + + {"dumps", (PyCFunction) cpm_dumps, METH_VARARGS | METH_KEYWORDS, + PyDoc_STR("dumps(obj, protocol=0) -- " + "Return a string containing an object in pickle format.\n" + "\n" + "See the Pickler docstring for the meaning of optional argument proto.") + }, + + {"load", (PyCFunction) cpm_load, METH_O, + PyDoc_STR("load(file) -- Load a pickle from the given file")}, + + {"loads", (PyCFunction) cpm_loads, METH_VARARGS, + PyDoc_STR("loads(string) -- Load a pickle from the given string")}, + + {"Pickler", (PyCFunction) get_Pickler, METH_VARARGS | METH_KEYWORDS, + PyDoc_STR("Pickler(file, protocol=0) -- Create a pickler.\n" + "\n" + "This takes a file-like object for writing a pickle data stream.\n" + "The optional proto argument tells the pickler to use the given\n" + "protocol; supported protocols are 0, 1, 2. The default\n" + "protocol is 0, to be backwards compatible. (Protocol 0 is the\n" + "only protocol that can be written to a file opened in text\n" + "mode and read back successfully. When using a protocol higher\n" + "than 0, make sure the file is opened in binary mode, both when\n" + "pickling and unpickling.)\n" + "\n" + "Protocol 1 is more efficient than protocol 0; protocol 2 is\n" + "more efficient than protocol 1.\n" + "\n" + "Specifying a negative protocol version selects the highest\n" + "protocol version supported. The higher the protocol used, the\n" + "more recent the version of Python needed to read the pickle\n" + "produced.\n" + "\n" + "The file parameter must have a write() method that accepts a single\n" + "string argument. It can thus be an open file object, a StringIO\n" + "object, or any other custom object that meets this interface.\n") + }, - {"Unpickler", (PyCFunction)get_Unpickler, METH_O, - PyDoc_STR("Unpickler(file) -- Create an unpickler.")}, + {"Unpickler", (PyCFunction) get_Unpickler, METH_O, + PyDoc_STR("Unpickler(file) -- Create an unpickler.")}, - { NULL, NULL } + {NULL, NULL} }; static int -init_stuff(PyObject *module_dict) +init_stuff(PyObject * module_dict) { - PyObject *copy_reg, *t, *r; + PyObject *copy_reg, *t, *r; #define INIT_STR(S) if (!( S ## _str=PyString_InternFromString(#S))) return -1; - if (PyType_Ready(&Unpicklertype) < 0) - return -1; - if (PyType_Ready(&Picklertype) < 0) - return -1; + if (PyType_Ready(&Unpicklertype) < 0) + return -1; + if (PyType_Ready(&Picklertype) < 0) + return -1; - INIT_STR(__class__); - INIT_STR(__getinitargs__); - INIT_STR(__dict__); - INIT_STR(__getstate__); - INIT_STR(__setstate__); - INIT_STR(__name__); - INIT_STR(__main__); - INIT_STR(__reduce__); - INIT_STR(__reduce_ex__); - INIT_STR(write); - INIT_STR(append); - INIT_STR(read); - INIT_STR(readline); - INIT_STR(copy_reg); - INIT_STR(dispatch_table); + INIT_STR(__class__); + INIT_STR(__getinitargs__); + INIT_STR(__dict__); + INIT_STR(__getstate__); + INIT_STR(__setstate__); + INIT_STR(__name__); + INIT_STR(__main__); + INIT_STR(__reduce__); + INIT_STR(__reduce_ex__); + INIT_STR(write); + INIT_STR(append); + INIT_STR(read); + INIT_STR(readline); + INIT_STR(copy_reg); + INIT_STR(dispatch_table); - if (!( copy_reg = PyImport_ImportModule("copy_reg"))) - return -1; + if (!(copy_reg = PyImport_ImportModule("copy_reg"))) + return -1; - /* This is special because we want to use a different - one in restricted mode. */ - dispatch_table = PyObject_GetAttr(copy_reg, dispatch_table_str); - if (!dispatch_table) return -1; - - extension_registry = PyObject_GetAttrString(copy_reg, - "_extension_registry"); - if (!extension_registry) return -1; - - inverted_registry = PyObject_GetAttrString(copy_reg, - "_inverted_registry"); - if (!inverted_registry) return -1; - - extension_cache = PyObject_GetAttrString(copy_reg, - "_extension_cache"); - if (!extension_cache) return -1; + /* This is special because we want to use a different + * one in restricted mode. */ + dispatch_table = PyObject_GetAttr(copy_reg, dispatch_table_str); + if (!dispatch_table) + return -1; - Py_DECREF(copy_reg); + extension_registry = PyObject_GetAttrString(copy_reg, + "_extension_registry"); + if (!extension_registry) + return -1; - if (!(empty_tuple = PyTuple_New(0))) - return -1; + inverted_registry = PyObject_GetAttrString(copy_reg, "_inverted_registry"); + if (!inverted_registry) + return -1; - two_tuple = PyTuple_New(2); - if (two_tuple == NULL) - return -1; - /* We use this temp container with no regard to refcounts, or to - * keeping containees alive. Exempt from GC, because we don't - * want anything looking at two_tuple() by magic. - */ - PyObject_GC_UnTrack(two_tuple); + extension_cache = PyObject_GetAttrString(copy_reg, "_extension_cache"); + if (!extension_cache) + return -1; - /* Ugh */ - if (!( t=PyImport_ImportModule("__builtin__"))) return -1; - if (PyDict_SetItemString(module_dict, "__builtins__", t) < 0) - return -1; + Py_DECREF(copy_reg); - if (!( t=PyDict_New())) return -1; - if (!( r=PyRun_String( - "def __str__(self):\n" - " return self.args and ('%s' % self.args[0]) or '(what)'\n", - Py_file_input, - module_dict, t) )) return -1; - Py_DECREF(r); + if (!(empty_tuple = PyTuple_New(0))) + return -1; - PickleError = PyErr_NewException("cPickle.PickleError", NULL, t); - if (!PickleError) - return -1; + two_tuple = PyTuple_New(2); + if (two_tuple == NULL) + return -1; + /* We use this temp container with no regard to refcounts, or to + * keeping containees alive. Exempt from GC, because we don't + * want anything looking at two_tuple() by magic. + */ + PyObject_GC_UnTrack(two_tuple); - Py_DECREF(t); + /* Ugh */ + if (!(t = PyImport_ImportModule("__builtin__"))) + return -1; + if (PyDict_SetItemString(module_dict, "__builtins__", t) < 0) + return -1; - PicklingError = PyErr_NewException("cPickle.PicklingError", - PickleError, NULL); - if (!PicklingError) - return -1; + if (!(t = PyDict_New())) + return -1; + if (!(r = PyRun_String("def __str__(self):\n" + " return self.args and ('%s' % self.args[0]) or '(what)'\n", + Py_file_input, module_dict, t))) + return -1; + Py_DECREF(r); - if (!( t=PyDict_New())) return -1; - if (!( r=PyRun_String( - "def __str__(self):\n" - " a=self.args\n" - " a=a and type(a[0]) or '(what)'\n" - " return 'Cannot pickle %s objects' % a\n" - , Py_file_input, - module_dict, t) )) return -1; - Py_DECREF(r); + PickleError = PyErr_NewException("cPickle.PickleError", NULL, t); + if (!PickleError) + return -1; - if (!( UnpickleableError = PyErr_NewException( - "cPickle.UnpickleableError", PicklingError, t))) - return -1; + Py_DECREF(t); - Py_DECREF(t); + PicklingError = PyErr_NewException("cPickle.PicklingError", + PickleError, NULL); + if (!PicklingError) + return -1; - if (!( UnpicklingError = PyErr_NewException("cPickle.UnpicklingError", - PickleError, NULL))) - return -1; + if (!(t = PyDict_New())) + return -1; + if (!(r = PyRun_String("def __str__(self):\n" + " a=self.args\n" + " a=a and type(a[0]) or '(what)'\n" + " return 'Cannot pickle %s objects' % a\n", + Py_file_input, module_dict, t))) + return -1; + Py_DECREF(r); - if (!( BadPickleGet = PyErr_NewException("cPickle.BadPickleGet", - UnpicklingError, NULL))) - return -1; + if (! + (UnpickleableError = + PyErr_NewException("cPickle.UnpickleableError", PicklingError, t))) + return -1; - if (PyDict_SetItemString(module_dict, "PickleError", - PickleError) < 0) - return -1; + Py_DECREF(t); - if (PyDict_SetItemString(module_dict, "PicklingError", - PicklingError) < 0) - return -1; + if (!(UnpicklingError = PyErr_NewException("cPickle.UnpicklingError", + PickleError, NULL))) + return -1; - if (PyDict_SetItemString(module_dict, "UnpicklingError", - UnpicklingError) < 0) - return -1; + if (!(BadPickleGet = PyErr_NewException("cPickle.BadPickleGet", + UnpicklingError, NULL))) + return -1; - if (PyDict_SetItemString(module_dict, "UnpickleableError", - UnpickleableError) < 0) - return -1; + if (PyDict_SetItemString(module_dict, "PickleError", PickleError) < 0) + return -1; - if (PyDict_SetItemString(module_dict, "BadPickleGet", - BadPickleGet) < 0) - return -1; + if (PyDict_SetItemString(module_dict, "PicklingError", PicklingError) < 0) + return -1; - PycString_IMPORT; + if (PyDict_SetItemString(module_dict, "UnpicklingError", + UnpicklingError) < 0) + return -1; - return 0; + if (PyDict_SetItemString(module_dict, "UnpickleableError", + UnpickleableError) < 0) + return -1; + + if (PyDict_SetItemString(module_dict, "BadPickleGet", BadPickleGet) < 0) + return -1; + + PycString_IMPORT; + + return 0; } -#ifndef PyMODINIT_FUNC /* declarations for DLL import/export */ +#ifndef PyMODINIT_FUNC /* declarations for DLL import/export */ #define PyMODINIT_FUNC void #endif PyMODINIT_FUNC initcPickle(void) { - PyObject *m, *d, *di, *v, *k; - Py_ssize_t i; - char *rev = "1.71"; /* XXX when does this change? */ - PyObject *format_version; - PyObject *compatible_formats; - - Picklertype.ob_type = &PyType_Type; - Unpicklertype.ob_type = &PyType_Type; - PdataType.ob_type = &PyType_Type; - - /* Initialize some pieces. We need to do this before module creation, - * so we're forced to use a temporary dictionary. :( - */ - di = PyDict_New(); - if (!di) return; - if (init_stuff(di) < 0) return; - - /* Create the module and add the functions */ - m = Py_InitModule4("cPickle", cPickle_methods, - cPickle_module_documentation, - (PyObject*)NULL,PYTHON_API_VERSION); - if (m == NULL) - return; - - /* Add some symbolic constants to the module */ - d = PyModule_GetDict(m); - v = PyString_FromString(rev); - PyDict_SetItemString(d, "__version__", v); - Py_XDECREF(v); - - /* Copy data from di. Waaa. */ - for (i=0; PyDict_Next(di, &i, &k, &v); ) { - if (PyObject_SetItem(d, k, v) < 0) { - Py_DECREF(di); - return; - } - } - Py_DECREF(di); - - i = PyModule_AddIntConstant(m, "HIGHEST_PROTOCOL", HIGHEST_PROTOCOL); - if (i < 0) - return; - - /* These are purely informational; no code uses them. */ - /* File format version we write. */ - format_version = PyString_FromString("2.0"); - /* Format versions we can read. */ - compatible_formats = Py_BuildValue("[sssss]", - "1.0", /* Original protocol 0 */ - "1.1", /* Protocol 0 + INST */ - "1.2", /* Original protocol 1 */ - "1.3", /* Protocol 1 + BINFLOAT */ - "2.0"); /* Original protocol 2 */ - PyDict_SetItemString(d, "format_version", format_version); - PyDict_SetItemString(d, "compatible_formats", compatible_formats); - Py_XDECREF(format_version); - Py_XDECREF(compatible_formats); + PyObject *m, *d, *di, *v, *k; + Py_ssize_t i; + char *rev = "1.71"; /* XXX when does this change? */ + PyObject *format_version; + PyObject *compatible_formats; + + Picklertype.ob_type = &PyType_Type; + Unpicklertype.ob_type = &PyType_Type; + PdataType.ob_type = &PyType_Type; + + /* Initialize some pieces. We need to do this before module creation, + * so we're forced to use a temporary dictionary. :( + */ + di = PyDict_New(); + if (!di) + return; + if (init_stuff(di) < 0) + return; + + /* Create the module and add the functions */ + m = Py_InitModule4("cPickle", cPickle_methods, + cPickle_module_documentation, + (PyObject *) NULL, PYTHON_API_VERSION); + if (m == NULL) + return; + + /* Add some symbolic constants to the module */ + d = PyModule_GetDict(m); + v = PyString_FromString(rev); + PyDict_SetItemString(d, "__version__", v); + Py_XDECREF(v); + + /* Copy data from di. Waaa. */ + for (i = 0; PyDict_Next(di, &i, &k, &v);) { + if (PyObject_SetItem(d, k, v) < 0) { + Py_DECREF(di); + return; + } + } + Py_DECREF(di); + + i = PyModule_AddIntConstant(m, "HIGHEST_PROTOCOL", HIGHEST_PROTOCOL); + if (i < 0) + return; + + /* These are purely informational; no code uses them. */ + /* File format version we write. */ + format_version = PyString_FromString("2.0"); + /* Format versions we can read. */ + compatible_formats = Py_BuildValue("[sssss]", "1.0", /* Original protocol 0 */ + "1.1", /* Protocol 0 + INST */ + "1.2", /* Original protocol 1 */ + "1.3", /* Protocol 1 + BINFLOAT */ + "2.0"); /* Original protocol 2 */ + PyDict_SetItemString(d, "format_version", format_version); + PyDict_SetItemString(d, "compatible_formats", compatible_formats); + Py_XDECREF(format_version); + Py_XDECREF(compatible_formats); } From python-checkins at python.org Sat Jul 7 20:02:40 2007 From: python-checkins at python.org (alexandre.vassalotti) Date: Sat, 7 Jul 2007 20:02:40 +0200 (CEST) Subject: [Python-checkins] r56183 - python/branches/cpy_merge/Modules/_picklemodule.c Message-ID: <20070707180240.9954E1E4007@bag.python.org> Author: alexandre.vassalotti Date: Sat Jul 7 20:02:39 2007 New Revision: 56183 Modified: python/branches/cpy_merge/Modules/_picklemodule.c Log: Expanded tabs. Modified: python/branches/cpy_merge/Modules/_picklemodule.c ============================================================================== --- python/branches/cpy_merge/Modules/_picklemodule.c (original) +++ python/branches/cpy_merge/Modules/_picklemodule.c Sat Jul 7 20:02:39 2007 @@ -3,7 +3,7 @@ #include "structmember.h" PyDoc_STRVAR(cPickle_module_documentation, - "C implementation and optimization of the Python pickle module."); +"C implementation and optimization of the Python pickle module."); #ifndef Py_eval_input #include @@ -64,18 +64,18 @@ #define SETITEMS 'u' /* Protocol 2. */ -#define PROTO '\x80' /* identify pickle protocol */ -#define NEWOBJ '\x81' /* build object by applying cls.__new__ to argtuple */ -#define EXT1 '\x82' /* push object from extension registry; 1-byte index */ -#define EXT2 '\x83' /* ditto, but 2-byte index */ -#define EXT4 '\x84' /* ditto, but 4-byte index */ -#define TUPLE1 '\x85' /* build 1-tuple from stack top */ -#define TUPLE2 '\x86' /* build 2-tuple from two topmost stack items */ -#define TUPLE3 '\x87' /* build 3-tuple from three topmost stack items */ -#define NEWTRUE '\x88' /* push True */ -#define NEWFALSE '\x89' /* push False */ -#define LONG1 '\x8a' /* push long from < 256 bytes */ -#define LONG4 '\x8b' /* push really big long */ +#define PROTO '\x80' /* identify pickle protocol */ +#define NEWOBJ '\x81' /* build object by applying cls.__new__ to argtuple */ +#define EXT1 '\x82' /* push object from extension registry; 1-byte index */ +#define EXT2 '\x83' /* ditto, but 2-byte index */ +#define EXT4 '\x84' /* ditto, but 4-byte index */ +#define TUPLE1 '\x85' /* build 1-tuple from stack top */ +#define TUPLE2 '\x86' /* build 2-tuple from two topmost stack items */ +#define TUPLE3 '\x87' /* build 3-tuple from three topmost stack items */ +#define NEWTRUE '\x88' /* push True */ +#define NEWFALSE '\x89' /* push False */ +#define LONG1 '\x8a' /* push long from < 256 bytes */ +#define LONG4 '\x8b' /* push really big long */ /* There aren't opcodes -- they're ways to pickle bools before protocol 2, * so that unpicklers written before bools were introduced unpickle them @@ -130,8 +130,8 @@ Internal Data type for pickle data. */ typedef struct { - PyObject_HEAD int length; /* number of initial slots in data currently used */ - int size; /* number of slots in data allocated */ + PyObject_HEAD int length; /* number of initial slots in data currently used */ + int size; /* number of slots in data allocated */ PyObject **data; } Pdata; @@ -142,10 +142,10 @@ PyObject **p; for (i = self->length, p = self->data; --i >= 0; p++) { - Py_DECREF(*p); + Py_DECREF(*p); } if (self->data) - free(self->data); + free(self->data); PyObject_Del(self); } @@ -163,12 +163,12 @@ Pdata *self; if (!(self = PyObject_New(Pdata, &PdataType))) - return NULL; + return NULL; self->size = 8; self->length = 0; self->data = malloc(self->size * sizeof(PyObject *)); if (self->data) - return (PyObject *) self; + return (PyObject *) self; Py_DECREF(self); return PyErr_NoMemory(); } @@ -190,12 +190,12 @@ PyObject **p; if (clearto < 0) - return stackUnderflow(); + return stackUnderflow(); if (clearto >= self->length) - return 0; + return 0; for (i = self->length, p = self->data + clearto; --i >= clearto; p++) { - Py_CLEAR(*p); + Py_CLEAR(*p); } self->length = clearto; @@ -210,16 +210,16 @@ PyObject **tmp; bigger = self->size << 1; - if (bigger <= 0) /* was 0, or new value overflows */ - goto nomemory; + if (bigger <= 0) /* was 0, or new value overflows */ + goto nomemory; if ((int) (size_t) bigger != bigger) - goto nomemory; + goto nomemory; nbytes = (size_t) bigger *sizeof(PyObject *); if (nbytes / sizeof(PyObject *) != (size_t) bigger) - goto nomemory; + goto nomemory; tmp = realloc(self->data, nbytes); if (tmp == NULL) - goto nomemory; + goto nomemory; self->data = tmp; self->size = bigger; return 0; @@ -233,13 +233,13 @@ * must be an lvalue holding PyObject*. On stack underflow, UnpicklingError * is raised and V is set to NULL. D and V may be evaluated several times. */ -#define PDATA_POP(D, V) { \ - if ((D)->length) \ - (V) = (D)->data[--((D)->length)]; \ - else { \ - PyErr_SetString(UnpicklingError, "bad pickle data"); \ - (V) = NULL; \ - } \ +#define PDATA_POP(D, V) { \ + if ((D)->length) \ + (V) = (D)->data[--((D)->length)]; \ + else { \ + PyErr_SetString(UnpicklingError, "bad pickle data"); \ + (V) = NULL; \ + } \ } /* PDATA_PUSH and PDATA_APPEND both push rvalue PyObject* O on to Pdata* @@ -251,22 +251,22 @@ */ /* Push O on stack D, giving ownership of O to the stack. */ -#define PDATA_PUSH(D, O, ER) { \ - if (((Pdata*)(D))->length == ((Pdata*)(D))->size && \ - Pdata_grow((Pdata*)(D)) < 0) { \ - Py_DECREF(O); \ - return ER; \ - } \ - ((Pdata*)(D))->data[((Pdata*)(D))->length++] = (O); \ +#define PDATA_PUSH(D, O, ER) { \ + if (((Pdata*)(D))->length == ((Pdata*)(D))->size && \ + Pdata_grow((Pdata*)(D)) < 0) { \ + Py_DECREF(O); \ + return ER; \ + } \ + ((Pdata*)(D))->data[((Pdata*)(D))->length++] = (O); \ } /* Push O on stack D, pushing a new reference. */ -#define PDATA_APPEND(D, O, ER) { \ - if (((Pdata*)(D))->length == ((Pdata*)(D))->size && \ - Pdata_grow((Pdata*)(D)) < 0) \ - return ER; \ - Py_INCREF(O); \ - ((Pdata*)(D))->data[((Pdata*)(D))->length++] = (O); \ +#define PDATA_APPEND(D, O, ER) { \ + if (((Pdata*)(D))->length == ((Pdata*)(D))->size && \ + Pdata_grow((Pdata*)(D)) < 0) \ + return ER; \ + Py_INCREF(O); \ + ((Pdata*)(D))->data[((Pdata*)(D))->length++] = (O); \ } @@ -279,9 +279,9 @@ l = self->length - start; r = PyTuple_New(l); if (r == NULL) - return NULL; + return NULL; for (i = start, j = 0; j < l; i++, j++) - PyTuple_SET_ITEM(r, j, self->data[i]); + PyTuple_SET_ITEM(r, j, self->data[i]); self->length = start; return r; @@ -295,9 +295,9 @@ l = self->length - start; if (!(r = PyList_New(l))) - return NULL; + return NULL; for (i = start, j = 0; j < l; i++, j++) - PyList_SET_ITEM(r, j, self->data[i]); + PyList_SET_ITEM(r, j, self->data[i]); self->length = start; return r; @@ -337,13 +337,13 @@ /* bool, true if proto > 0 */ int bin; - int fast; /* Fast mode doesn't save in memo, don't use if circ ref */ + int fast; /* Fast mode doesn't save in memo, don't use if circ ref */ int nesting; int (*write_func) (struct Picklerobject *, const char *, Py_ssize_t); char *write_buf; int buf_size; PyObject *dispatch_table; - int fast_container; /* count nested container dumps */ + int fast_container; /* count nested container dumps */ PyObject *fast_memo; } Picklerobject; @@ -389,29 +389,29 @@ va_start(va, format); if (format) - args = Py_VaBuildValue(format, va); + args = Py_VaBuildValue(format, va); va_end(va); if (format && !args) - return NULL; + return NULL; if (stringformat && !(retval = PyString_FromString(stringformat))) - return NULL; + return NULL; if (retval) { - if (args) { - PyObject *v; - v = PyString_Format(retval, args); - Py_DECREF(retval); - Py_DECREF(args); - if (!v) - return NULL; - retval = v; - } + if (args) { + PyObject *v; + v = PyString_Format(retval, args); + Py_DECREF(retval); + Py_DECREF(args); + if (!v) + return NULL; + retval = v; + } } else if (args) - retval = args; + retval = args; else { - PyErr_SetObject(ErrType, Py_None); - return NULL; + PyErr_SetObject(ErrType, Py_None); + return NULL; } PyErr_SetObject(ErrType, retval); Py_DECREF(retval); @@ -424,19 +424,19 @@ size_t nbyteswritten; if (s == NULL) { - return 0; + return 0; } if (n > INT_MAX) { - /* String too large */ - return -1; + /* String too large */ + return -1; } Py_BEGIN_ALLOW_THREADS - nbyteswritten = fwrite(s, sizeof(char), n, self->fp); + nbyteswritten = fwrite(s, sizeof(char), n, self->fp); Py_END_ALLOW_THREADS if (nbyteswritten != (size_t) n) { - PyErr_SetFromErrno(PyExc_IOError); - return -1; + PyErr_SetFromErrno(PyExc_IOError); + return -1; } return (int) n; @@ -446,11 +446,11 @@ write_cStringIO(Picklerobject * self, const char *s, Py_ssize_t n) { if (s == NULL) { - return 0; + return 0; } if (PycStringIO->cwrite((PyObject *) self->file, s, n) != n) { - return -1; + return -1; } return (int) n; @@ -460,9 +460,9 @@ write_none(Picklerobject * self, const char *s, Py_ssize_t n) { if (s == NULL) - return 0; + return 0; if (n > INT_MAX) - return -1; + return -1; return (int) n; } @@ -473,46 +473,46 @@ int n; if (_n > INT_MAX) - return -1; + return -1; n = (int) _n; if (s == NULL) { - if (!(self->buf_size)) - return 0; - py_str = PyString_FromStringAndSize(self->write_buf, self->buf_size); - if (!py_str) - return -1; + if (!(self->buf_size)) + return 0; + py_str = PyString_FromStringAndSize(self->write_buf, self->buf_size); + if (!py_str) + return -1; } else { - if (self->buf_size && (n + self->buf_size) > WRITE_BUF_SIZE) { - if (write_other(self, NULL, 0) < 0) - return -1; - } - - if (n > WRITE_BUF_SIZE) { - if (!(py_str = PyString_FromStringAndSize(s, n))) - return -1; - } - else { - memcpy(self->write_buf + self->buf_size, s, n); - self->buf_size += n; - return n; - } + if (self->buf_size && (n + self->buf_size) > WRITE_BUF_SIZE) { + if (write_other(self, NULL, 0) < 0) + return -1; + } + + if (n > WRITE_BUF_SIZE) { + if (!(py_str = PyString_FromStringAndSize(s, n))) + return -1; + } + else { + memcpy(self->write_buf + self->buf_size, s, n); + self->buf_size += n; + return n; + } } if (self->write) { - /* object with write method */ - ARG_TUP(self, py_str); - if (self->arg) { - junk = PyObject_Call(self->write, self->arg, NULL); - FREE_ARG_TUP(self); - } - if (junk) - Py_DECREF(junk); - else - return -1; + /* object with write method */ + ARG_TUP(self, py_str); + if (self->arg) { + junk = PyObject_Call(self->write, self->arg, NULL); + FREE_ARG_TUP(self); + } + if (junk) + Py_DECREF(junk); + else + return -1; } else - PDATA_PUSH(self->file, py_str, -1); + PDATA_PUSH(self->file, py_str, -1); self->buf_size = 0; return n; @@ -525,36 +525,36 @@ size_t nbytesread; if (self->buf_size == 0) { - int size; + int size; - size = ((n < 32) ? 32 : n); - if (!(self->buf = (char *) malloc(size))) { - PyErr_NoMemory(); - return -1; - } + size = ((n < 32) ? 32 : n); + if (!(self->buf = (char *) malloc(size))) { + PyErr_NoMemory(); + return -1; + } - self->buf_size = size; + self->buf_size = size; } else if (n > self->buf_size) { - char *newbuf = (char *) realloc(self->buf, n); - if (!newbuf) { - PyErr_NoMemory(); - return -1; - } - self->buf = newbuf; - self->buf_size = n; + char *newbuf = (char *) realloc(self->buf, n); + if (!newbuf) { + PyErr_NoMemory(); + return -1; + } + self->buf = newbuf; + self->buf_size = n; } Py_BEGIN_ALLOW_THREADS - nbytesread = fread(self->buf, sizeof(char), n, self->fp); + nbytesread = fread(self->buf, sizeof(char), n, self->fp); Py_END_ALLOW_THREADS if (nbytesread != (size_t) n) { - if (feof(self->fp)) { - PyErr_SetNone(PyExc_EOFError); - return -1; - } + if (feof(self->fp)) { + PyErr_SetNone(PyExc_EOFError); + return -1; + } - PyErr_SetFromErrno(PyExc_IOError); - return -1; + PyErr_SetFromErrno(PyExc_IOError); + return -1; } *s = self->buf; @@ -569,36 +569,36 @@ int i; if (self->buf_size == 0) { - if (!(self->buf = (char *) malloc(40))) { - PyErr_NoMemory(); - return -1; - } - self->buf_size = 40; + if (!(self->buf = (char *) malloc(40))) { + PyErr_NoMemory(); + return -1; + } + self->buf_size = 40; } i = 0; while (1) { - int bigger; - char *newbuf; - for (; i < (self->buf_size - 1); i++) { - if (feof(self->fp) || (self->buf[i] = getc(self->fp)) == '\n') { - self->buf[i + 1] = '\0'; - *s = self->buf; - return i + 1; - } - } - bigger = self->buf_size << 1; - if (bigger <= 0) { /* overflow */ - PyErr_NoMemory(); - return -1; - } - newbuf = (char *) realloc(self->buf, bigger); - if (!newbuf) { - PyErr_NoMemory(); - return -1; - } - self->buf = newbuf; - self->buf_size = bigger; + int bigger; + char *newbuf; + for (; i < (self->buf_size - 1); i++) { + if (feof(self->fp) || (self->buf[i] = getc(self->fp)) == '\n') { + self->buf[i + 1] = '\0'; + *s = self->buf; + return i + 1; + } + } + bigger = self->buf_size << 1; + if (bigger <= 0) { /* overflow */ + PyErr_NoMemory(); + return -1; + } + newbuf = (char *) realloc(self->buf, bigger); + if (!newbuf) { + PyErr_NoMemory(); + return -1; + } + self->buf = newbuf; + self->buf_size = bigger; } } @@ -609,8 +609,8 @@ char *ptr; if (PycStringIO->cread((PyObject *) self->file, &ptr, n) != n) { - PyErr_SetNone(PyExc_EOFError); - return -1; + PyErr_SetNone(PyExc_EOFError); + return -1; } *s = ptr; @@ -626,7 +626,7 @@ char *ptr; if ((n = PycStringIO->creadline((PyObject *) self->file, &ptr)) < 0) { - return -1; + return -1; } *s = ptr; @@ -641,21 +641,21 @@ PyObject *bytes, *str = 0; if (!(bytes = PyInt_FromSsize_t(n))) - return -1; + return -1; ARG_TUP(self, bytes); if (self->arg) { - str = PyObject_Call(self->read, self->arg, NULL); - FREE_ARG_TUP(self); + str = PyObject_Call(self->read, self->arg, NULL); + FREE_ARG_TUP(self); } if (!str) - return -1; + return -1; Py_XDECREF(self->last_string); self->last_string = str; if (!(*s = PyString_AsString(str))) - return -1; + return -1; return n; } @@ -667,17 +667,17 @@ Py_ssize_t str_size; if (!(str = PyObject_CallObject(self->readline, empty_tuple))) { - return -1; + return -1; } if ((str_size = PyString_Size(str)) < 0) - return -1; + return -1; Py_XDECREF(self->last_string); self->last_string = str; if (!(*s = PyString_AsString(str))) - return -1; + return -1; return str_size; } @@ -691,7 +691,7 @@ { char *r = (char *) malloc(n + 1); if (r == NULL) - return (char *) PyErr_NoMemory(); + return (char *) PyErr_NoMemory(); memcpy(r, s, n); r[n] = 0; return r; @@ -707,50 +707,50 @@ size_t len; if (!(mv = PyDict_GetItem(self->memo, id))) { - PyErr_SetObject(PyExc_KeyError, id); - return -1; + PyErr_SetObject(PyExc_KeyError, id); + return -1; } if (!(value = PyTuple_GetItem(mv, 0))) - return -1; + return -1; if (!(PyInt_Check(value))) { - PyErr_SetString(PicklingError, "no int where int expected in memo"); - return -1; + PyErr_SetString(PicklingError, "no int where int expected in memo"); + return -1; } c_value = PyInt_AsLong(value); if (c_value == -1 && PyErr_Occurred()) - return -1; + return -1; if (!self->bin) { - s[0] = GET; - PyOS_snprintf(s + 1, sizeof(s) - 1, "%ld\n", c_value); - len = strlen(s); + s[0] = GET; + PyOS_snprintf(s + 1, sizeof(s) - 1, "%ld\n", c_value); + len = strlen(s); } else if (Pdata_Check(self->file)) { - if (write_other(self, NULL, 0) < 0) - return -1; - PDATA_APPEND(self->file, mv, -1); - return 0; + if (write_other(self, NULL, 0) < 0) + return -1; + PDATA_APPEND(self->file, mv, -1); + return 0; } else { - if (c_value < 256) { - s[0] = BINGET; - s[1] = (int) (c_value & 0xff); - len = 2; - } - else { - s[0] = LONG_BINGET; - s[1] = (int) (c_value & 0xff); - s[2] = (int) ((c_value >> 8) & 0xff); - s[3] = (int) ((c_value >> 16) & 0xff); - s[4] = (int) ((c_value >> 24) & 0xff); - len = 5; - } + if (c_value < 256) { + s[0] = BINGET; + s[1] = (int) (c_value & 0xff); + len = 2; + } + else { + s[0] = LONG_BINGET; + s[1] = (int) (c_value & 0xff); + s[2] = (int) ((c_value >> 8) & 0xff); + s[3] = (int) ((c_value >> 16) & 0xff); + s[4] = (int) ((c_value >> 24) & 0xff); + len = 5; + } } if (self->write_func(self, s, len) < 0) - return -1; + return -1; return 0; } @@ -760,7 +760,7 @@ put(Picklerobject * self, PyObject * ob) { if (ob->ob_refcnt < 2 || self->fast) - return 0; + return 0; return put2(self, ob); } @@ -776,10 +776,10 @@ PyObject *py_ob_id = 0, *memo_len = 0, *t = 0; if (self->fast) - return 0; + return 0; if ((p = PyDict_Size(self->memo)) < 0) - goto finally; + goto finally; /* Make sure memo keys are positive! */ /* XXX Why? @@ -790,13 +790,13 @@ p++; if (!(py_ob_id = PyLong_FromVoidPtr(ob))) - goto finally; + goto finally; if (!(memo_len = PyInt_FromLong(p))) - goto finally; + goto finally; if (!(t = PyTuple_New(2))) - goto finally; + goto finally; PyTuple_SET_ITEM(t, 0, memo_len); Py_INCREF(memo_len); @@ -804,38 +804,38 @@ Py_INCREF(ob); if (PyDict_SetItem(self->memo, py_ob_id, t) < 0) - goto finally; + goto finally; if (!self->bin) { - c_str[0] = PUT; - PyOS_snprintf(c_str + 1, sizeof(c_str) - 1, "%d\n", p); - len = strlen(c_str); + c_str[0] = PUT; + PyOS_snprintf(c_str + 1, sizeof(c_str) - 1, "%d\n", p); + len = strlen(c_str); } else if (Pdata_Check(self->file)) { - if (write_other(self, NULL, 0) < 0) - return -1; - PDATA_APPEND(self->file, memo_len, -1); - res = 0; /* Job well done ;) */ - goto finally; + if (write_other(self, NULL, 0) < 0) + return -1; + PDATA_APPEND(self->file, memo_len, -1); + res = 0; /* Job well done ;) */ + goto finally; } else { - if (p >= 256) { - c_str[0] = LONG_BINPUT; - c_str[1] = (int) (p & 0xff); - c_str[2] = (int) ((p >> 8) & 0xff); - c_str[3] = (int) ((p >> 16) & 0xff); - c_str[4] = (int) ((p >> 24) & 0xff); - len = 5; - } - else { - c_str[0] = BINPUT; - c_str[1] = p; - len = 2; - } + if (p >= 256) { + c_str[0] = LONG_BINPUT; + c_str[1] = (int) (p & 0xff); + c_str[2] = (int) ((p >> 8) & 0xff); + c_str[3] = (int) ((p >> 16) & 0xff); + c_str[4] = (int) ((p >> 24) & 0xff); + len = 5; + } + else { + c_str[0] = BINPUT; + c_str[1] = p; + len = 2; + } } if (self->write_func(self, c_str, len) < 0) - goto finally; + goto finally; res = 0; @@ -855,38 +855,38 @@ module = PyObject_GetAttrString(global, "__module__"); if (module) - return module; + return module; if (PyErr_ExceptionMatches(PyExc_AttributeError)) - PyErr_Clear(); + PyErr_Clear(); else - return NULL; + return NULL; if (!(modules_dict = PySys_GetObject("modules"))) - return NULL; + return NULL; i = 0; while ((j = PyDict_Next(modules_dict, &i, &name, &module))) { - if (PyObject_Compare(name, __main___str) == 0) - continue; + if (PyObject_Compare(name, __main___str) == 0) + continue; - global_name_attr = PyObject_GetAttr(module, global_name); - if (!global_name_attr) { - if (PyErr_ExceptionMatches(PyExc_AttributeError)) - PyErr_Clear(); - else - return NULL; - continue; - } - - if (global_name_attr != global) { - Py_DECREF(global_name_attr); - continue; - } + global_name_attr = PyObject_GetAttr(module, global_name); + if (!global_name_attr) { + if (PyErr_ExceptionMatches(PyExc_AttributeError)) + PyErr_Clear(); + else + return NULL; + continue; + } + + if (global_name_attr != global) { + Py_DECREF(global_name_attr); + continue; + } - Py_DECREF(global_name_attr); + Py_DECREF(global_name_attr); - break; + break; } /* The following implements the rule in pickle.py added in 1.5 @@ -894,8 +894,8 @@ * like this rule. jlf */ if (!j) { - j = 1; - name = __main___str; + j = 1; + name = __main___str; } Py_INCREF(name); @@ -908,32 +908,32 @@ { /* if fast_container < 0, we're doing an error exit. */ if (++self->fast_container >= PY_CPICKLE_FAST_LIMIT) { - PyObject *key = NULL; - if (self->fast_memo == NULL) { - self->fast_memo = PyDict_New(); - if (self->fast_memo == NULL) { - self->fast_container = -1; - return 0; - } - } - key = PyLong_FromVoidPtr(obj); - if (key == NULL) - return 0; - if (PyDict_GetItem(self->fast_memo, key)) { - Py_DECREF(key); - PyErr_Format(PyExc_ValueError, - "fast mode: can't pickle cyclic objects " - "including object type %s at %p", - obj->ob_type->tp_name, obj); - self->fast_container = -1; - return 0; - } - if (PyDict_SetItem(self->fast_memo, key, Py_None) < 0) { - Py_DECREF(key); - self->fast_container = -1; - return 0; - } - Py_DECREF(key); + PyObject *key = NULL; + if (self->fast_memo == NULL) { + self->fast_memo = PyDict_New(); + if (self->fast_memo == NULL) { + self->fast_container = -1; + return 0; + } + } + key = PyLong_FromVoidPtr(obj); + if (key == NULL) + return 0; + if (PyDict_GetItem(self->fast_memo, key)) { + Py_DECREF(key); + PyErr_Format(PyExc_ValueError, + "fast mode: can't pickle cyclic objects " + "including object type %s at %p", + obj->ob_type->tp_name, obj); + self->fast_container = -1; + return 0; + } + if (PyDict_SetItem(self->fast_memo, key, Py_None) < 0) { + Py_DECREF(key); + self->fast_container = -1; + return 0; + } + Py_DECREF(key); } return 1; } @@ -942,14 +942,14 @@ fast_save_leave(Picklerobject * self, PyObject * obj) { if (self->fast_container-- >= PY_CPICKLE_FAST_LIMIT) { - PyObject *key = PyLong_FromVoidPtr(obj); - if (key == NULL) - return 0; - if (PyDict_DelItem(self->fast_memo, key) < 0) { - Py_DECREF(key); - return 0; - } - Py_DECREF(key); + PyObject *key = PyLong_FromVoidPtr(obj); + if (key == NULL) + return 0; + if (PyDict_DelItem(self->fast_memo, key) < 0) { + Py_DECREF(key); + return 0; + } + Py_DECREF(key); } return 1; } @@ -959,7 +959,7 @@ { static char none = NONE; if (self->write_func(self, &none, 1) < 0) - return -1; + return -1; return 0; } @@ -972,12 +972,12 @@ long l = args == Py_True; if (self->proto >= 2) { - char opcode = l ? NEWTRUE : NEWFALSE; - if (self->write_func(self, &opcode, 1) < 0) - return -1; + char opcode = l ? NEWTRUE : NEWFALSE; + if (self->write_func(self, &opcode, 1) < 0) + return -1; } else if (self->write_func(self, buf[l], len[l]) < 0) - return -1; + return -1; return 0; } @@ -989,41 +989,41 @@ if (!self->bin #if SIZEOF_LONG > 4 - || l > 0x7fffffffL || l < -0x80000000L + || l > 0x7fffffffL || l < -0x80000000L #endif - ) { - /* Text-mode pickle, or long too big to fit in the 4-byte - * signed BININT format: store as a string. - */ - c_str[0] = INT; - PyOS_snprintf(c_str + 1, sizeof(c_str) - 1, "%ld\n", l); - if (self->write_func(self, c_str, strlen(c_str)) < 0) - return -1; + ) { + /* Text-mode pickle, or long too big to fit in the 4-byte + * signed BININT format: store as a string. + */ + c_str[0] = INT; + PyOS_snprintf(c_str + 1, sizeof(c_str) - 1, "%ld\n", l); + if (self->write_func(self, c_str, strlen(c_str)) < 0) + return -1; } else { - /* Binary pickle and l fits in a signed 4-byte int. */ - c_str[1] = (int) (l & 0xff); - c_str[2] = (int) ((l >> 8) & 0xff); - c_str[3] = (int) ((l >> 16) & 0xff); - c_str[4] = (int) ((l >> 24) & 0xff); - - if ((c_str[4] == 0) && (c_str[3] == 0)) { - if (c_str[2] == 0) { - c_str[0] = BININT1; - len = 2; - } - else { - c_str[0] = BININT2; - len = 3; - } - } - else { - c_str[0] = BININT; - len = 5; - } + /* Binary pickle and l fits in a signed 4-byte int. */ + c_str[1] = (int) (l & 0xff); + c_str[2] = (int) ((l >> 8) & 0xff); + c_str[3] = (int) ((l >> 16) & 0xff); + c_str[4] = (int) ((l >> 24) & 0xff); + + if ((c_str[4] == 0) && (c_str[3] == 0)) { + if (c_str[2] == 0) { + c_str[0] = BININT1; + len = 2; + } + else { + c_str[0] = BININT2; + len = 3; + } + } + else { + c_str[0] = BININT; + len = 5; + } - if (self->write_func(self, c_str, len) < 0) - return -1; + if (self->write_func(self, c_str, len) < 0) + return -1; } return 0; @@ -1040,114 +1040,114 @@ static char l = LONG; if (val == -1 && PyErr_Occurred()) { - /* out of range for int pickling */ - PyErr_Clear(); + /* out of range for int pickling */ + PyErr_Clear(); } else - return save_int(self, val); + return save_int(self, val); if (self->proto >= 2) { - /* Linear-time pickling. */ - size_t nbits; - size_t nbytes; - unsigned char *pdata; - char c_str[5]; - int i; - int sign = _PyLong_Sign(args); - - if (sign == 0) { - /* It's 0 -- an empty bytestring. */ - c_str[0] = LONG1; - c_str[1] = 0; - i = self->write_func(self, c_str, 2); - if (i < 0) - goto finally; - res = 0; - goto finally; - } - nbits = _PyLong_NumBits(args); - if (nbits == (size_t) - 1 && PyErr_Occurred()) - goto finally; - /* How many bytes do we need? There are nbits >> 3 full - * bytes of data, and nbits & 7 leftover bits. If there - * are any leftover bits, then we clearly need another - * byte. Wnat's not so obvious is that we *probably* - * need another byte even if there aren't any leftovers: - * the most-significant bit of the most-significant byte - * acts like a sign bit, and it's usually got a sense - * opposite of the one we need. The exception is longs - * of the form -(2**(8*j-1)) for j > 0. Such a long is - * its own 256's-complement, so has the right sign bit - * even without the extra byte. That's a pain to check - * for in advance, though, so we always grab an extra - * byte at the start, and cut it back later if possible. - */ - nbytes = (nbits >> 3) + 1; - if (nbytes > INT_MAX) { - PyErr_SetString(PyExc_OverflowError, "long too large " - "to pickle"); - goto finally; - } - repr = PyString_FromStringAndSize(NULL, (int) nbytes); - if (repr == NULL) - goto finally; - pdata = (unsigned char *) PyString_AS_STRING(repr); - i = _PyLong_AsByteArray((PyLongObject *) args, - pdata, nbytes, - 1 /* little endian */ , 1 /* signed */ ); - if (i < 0) - goto finally; - /* If the long is negative, this may be a byte more than - * needed. This is so iff the MSB is all redundant sign - * bits. - */ - if (sign < 0 && nbytes > 1 && pdata[nbytes - 1] == 0xff && - (pdata[nbytes - 2] & 0x80) != 0) - --nbytes; - - if (nbytes < 256) { - c_str[0] = LONG1; - c_str[1] = (char) nbytes; - size = 2; - } - else { - c_str[0] = LONG4; - size = (int) nbytes; - for (i = 1; i < 5; i++) { - c_str[i] = (char) (size & 0xff); - size >>= 8; - } - size = 5; - } - i = self->write_func(self, c_str, size); - if (i < 0) - goto finally; - i = self->write_func(self, (char *) pdata, (int) nbytes); - if (i < 0) - goto finally; - res = 0; - goto finally; + /* Linear-time pickling. */ + size_t nbits; + size_t nbytes; + unsigned char *pdata; + char c_str[5]; + int i; + int sign = _PyLong_Sign(args); + + if (sign == 0) { + /* It's 0 -- an empty bytestring. */ + c_str[0] = LONG1; + c_str[1] = 0; + i = self->write_func(self, c_str, 2); + if (i < 0) + goto finally; + res = 0; + goto finally; + } + nbits = _PyLong_NumBits(args); + if (nbits == (size_t) - 1 && PyErr_Occurred()) + goto finally; + /* How many bytes do we need? There are nbits >> 3 full + * bytes of data, and nbits & 7 leftover bits. If there + * are any leftover bits, then we clearly need another + * byte. Wnat's not so obvious is that we *probably* + * need another byte even if there aren't any leftovers: + * the most-significant bit of the most-significant byte + * acts like a sign bit, and it's usually got a sense + * opposite of the one we need. The exception is longs + * of the form -(2**(8*j-1)) for j > 0. Such a long is + * its own 256's-complement, so has the right sign bit + * even without the extra byte. That's a pain to check + * for in advance, though, so we always grab an extra + * byte at the start, and cut it back later if possible. + */ + nbytes = (nbits >> 3) + 1; + if (nbytes > INT_MAX) { + PyErr_SetString(PyExc_OverflowError, "long too large " + "to pickle"); + goto finally; + } + repr = PyString_FromStringAndSize(NULL, (int) nbytes); + if (repr == NULL) + goto finally; + pdata = (unsigned char *) PyString_AS_STRING(repr); + i = _PyLong_AsByteArray((PyLongObject *) args, + pdata, nbytes, + 1 /* little endian */ , 1 /* signed */ ); + if (i < 0) + goto finally; + /* If the long is negative, this may be a byte more than + * needed. This is so iff the MSB is all redundant sign + * bits. + */ + if (sign < 0 && nbytes > 1 && pdata[nbytes - 1] == 0xff && + (pdata[nbytes - 2] & 0x80) != 0) + --nbytes; + + if (nbytes < 256) { + c_str[0] = LONG1; + c_str[1] = (char) nbytes; + size = 2; + } + else { + c_str[0] = LONG4; + size = (int) nbytes; + for (i = 1; i < 5; i++) { + c_str[i] = (char) (size & 0xff); + size >>= 8; + } + size = 5; + } + i = self->write_func(self, c_str, size); + if (i < 0) + goto finally; + i = self->write_func(self, (char *) pdata, (int) nbytes); + if (i < 0) + goto finally; + res = 0; + goto finally; } /* proto < 2: write the repr and newline. This is quadratic-time * (in the number of digits), in both directions. */ if (!(repr = PyObject_Repr(args))) - goto finally; + goto finally; if ((size = PyString_Size(repr)) < 0) - goto finally; + goto finally; if (self->write_func(self, &l, 1) < 0) - goto finally; + goto finally; if (self->write_func(self, - PyString_AS_STRING((PyStringObject *) repr), - size) < 0) - goto finally; + PyString_AS_STRING((PyStringObject *) repr), + size) < 0) + goto finally; if (self->write_func(self, "\n", 1) < 0) - goto finally; + goto finally; res = 0; @@ -1163,22 +1163,22 @@ double x = PyFloat_AS_DOUBLE((PyFloatObject *) args); if (self->bin) { - char str[9]; - str[0] = BINFLOAT; - if (_PyFloat_Pack8(x, (unsigned char *) &str[1], 0) < 0) - return -1; - if (self->write_func(self, str, 9) < 0) - return -1; + char str[9]; + str[0] = BINFLOAT; + if (_PyFloat_Pack8(x, (unsigned char *) &str[1], 0) < 0) + return -1; + if (self->write_func(self, str, 9) < 0) + return -1; } else { - char c_str[250]; - c_str[0] = FLOAT; - PyOS_ascii_formatd(c_str + 1, sizeof(c_str) - 2, "%.17g", x); - /* Extend the formatted string with a newline character */ - strcat(c_str, "\n"); + char c_str[250]; + c_str[0] = FLOAT; + PyOS_ascii_formatd(c_str + 1, sizeof(c_str) - 2, "%.17g", x); + /* Extend the formatted string with a newline character */ + strcat(c_str, "\n"); - if (self->write_func(self, c_str, strlen(c_str)) < 0) - return -1; + if (self->write_func(self, c_str, strlen(c_str)) < 0) + return -1; } return 0; @@ -1192,71 +1192,71 @@ PyObject *repr = 0; if ((size = PyString_Size(args)) < 0) - return -1; + return -1; if (!self->bin) { - char *repr_str; + char *repr_str; - static char string = STRING; + static char string = STRING; - if (!(repr = PyObject_Repr(args))) - return -1; + if (!(repr = PyObject_Repr(args))) + return -1; - if ((len = PyString_Size(repr)) < 0) - goto err; - repr_str = PyString_AS_STRING((PyStringObject *) repr); + if ((len = PyString_Size(repr)) < 0) + goto err; + repr_str = PyString_AS_STRING((PyStringObject *) repr); - if (self->write_func(self, &string, 1) < 0) - goto err; + if (self->write_func(self, &string, 1) < 0) + goto err; - if (self->write_func(self, repr_str, len) < 0) - goto err; + if (self->write_func(self, repr_str, len) < 0) + goto err; - if (self->write_func(self, "\n", 1) < 0) - goto err; + if (self->write_func(self, "\n", 1) < 0) + goto err; - Py_XDECREF(repr); + Py_XDECREF(repr); } else { - int i; - char c_str[5]; + int i; + char c_str[5]; - if ((size = PyString_Size(args)) < 0) - return -1; + if ((size = PyString_Size(args)) < 0) + return -1; - if (size < 256) { - c_str[0] = SHORT_BINSTRING; - c_str[1] = size; - len = 2; - } - else if (size <= INT_MAX) { - c_str[0] = BINSTRING; - for (i = 1; i < 5; i++) - c_str[i] = (int) (size >> ((i - 1) * 8)); - len = 5; - } - else - return -1; /* string too large */ - - if (self->write_func(self, c_str, len) < 0) - return -1; - - if (size > 128 && Pdata_Check(self->file)) { - if (write_other(self, NULL, 0) < 0) - return -1; - PDATA_APPEND(self->file, args, -1); - } - else { - if (self->write_func(self, - PyString_AS_STRING((PyStringObject *) args), - size) < 0) - return -1; - } + if (size < 256) { + c_str[0] = SHORT_BINSTRING; + c_str[1] = size; + len = 2; + } + else if (size <= INT_MAX) { + c_str[0] = BINSTRING; + for (i = 1; i < 5; i++) + c_str[i] = (int) (size >> ((i - 1) * 8)); + len = 5; + } + else + return -1; /* string too large */ + + if (self->write_func(self, c_str, len) < 0) + return -1; + + if (size > 128 && Pdata_Check(self->file)) { + if (write_other(self, NULL, 0) < 0) + return -1; + PDATA_APPEND(self->file, args, -1); + } + else { + if (self->write_func(self, + PyString_AS_STRING((PyStringObject *) args), + size) < 0) + return -1; + } } if (doput) - if (put(self, args) < 0) - return -1; + if (put(self, args) < 0) + return -1; return 0; @@ -1280,25 +1280,25 @@ repr = PyString_FromStringAndSize(NULL, 6 * size); if (repr == NULL) - return NULL; + return NULL; if (size == 0) - return repr; + return repr; p = q = PyString_AS_STRING(repr); while (size-- > 0) { - Py_UNICODE ch = *s++; - /* Map 16-bit characters to '\uxxxx' */ - if (ch >= 256 || ch == '\\' || ch == '\n') { - *p++ = '\\'; - *p++ = 'u'; - *p++ = hexdigit[(ch >> 12) & 0xf]; - *p++ = hexdigit[(ch >> 8) & 0xf]; - *p++ = hexdigit[(ch >> 4) & 0xf]; - *p++ = hexdigit[ch & 15]; - } - /* Copy everything else as-is */ - else - *p++ = (char) ch; + Py_UNICODE ch = *s++; + /* Map 16-bit characters to '\uxxxx' */ + if (ch >= 256 || ch == '\\' || ch == '\n') { + *p++ = '\\'; + *p++ = 'u'; + *p++ = hexdigit[(ch >> 12) & 0xf]; + *p++ = hexdigit[(ch >> 8) & 0xf]; + *p++ = hexdigit[(ch >> 4) & 0xf]; + *p++ = hexdigit[ch & 15]; + } + /* Copy everything else as-is */ + else + *p++ = (char) ch; } *p = '\0'; _PyString_Resize(&repr, p - q); @@ -1313,69 +1313,69 @@ PyObject *repr = 0; if (!PyUnicode_Check(args)) - return -1; + return -1; if (!self->bin) { - char *repr_str; - static char string = UNICODE; + char *repr_str; + static char string = UNICODE; - repr = - modified_EncodeRawUnicodeEscape(PyUnicode_AS_UNICODE(args), - PyUnicode_GET_SIZE(args)); - if (!repr) - return -1; + repr = + modified_EncodeRawUnicodeEscape(PyUnicode_AS_UNICODE(args), + PyUnicode_GET_SIZE(args)); + if (!repr) + return -1; - if ((len = PyString_Size(repr)) < 0) - goto err; - repr_str = PyString_AS_STRING((PyStringObject *) repr); + if ((len = PyString_Size(repr)) < 0) + goto err; + repr_str = PyString_AS_STRING((PyStringObject *) repr); - if (self->write_func(self, &string, 1) < 0) - goto err; + if (self->write_func(self, &string, 1) < 0) + goto err; - if (self->write_func(self, repr_str, len) < 0) - goto err; + if (self->write_func(self, repr_str, len) < 0) + goto err; - if (self->write_func(self, "\n", 1) < 0) - goto err; + if (self->write_func(self, "\n", 1) < 0) + goto err; - Py_XDECREF(repr); + Py_XDECREF(repr); } else { - int i; - char c_str[5]; + int i; + char c_str[5]; - if (!(repr = PyUnicode_AsUTF8String(args))) - return -1; + if (!(repr = PyUnicode_AsUTF8String(args))) + return -1; - if ((size = PyString_Size(repr)) < 0) - goto err; - if (size > INT_MAX) - return -1; /* string too large */ - - c_str[0] = BINUNICODE; - for (i = 1; i < 5; i++) - c_str[i] = (int) (size >> ((i - 1) * 8)); - len = 5; - - if (self->write_func(self, c_str, len) < 0) - goto err; - - if (size > 128 && Pdata_Check(self->file)) { - if (write_other(self, NULL, 0) < 0) - goto err; - PDATA_APPEND(self->file, repr, -1); - } - else { - if (self->write_func(self, PyString_AS_STRING(repr), size) < 0) - goto err; - } + if ((size = PyString_Size(repr)) < 0) + goto err; + if (size > INT_MAX) + return -1; /* string too large */ + + c_str[0] = BINUNICODE; + for (i = 1; i < 5; i++) + c_str[i] = (int) (size >> ((i - 1) * 8)); + len = 5; + + if (self->write_func(self, c_str, len) < 0) + goto err; + + if (size > 128 && Pdata_Check(self->file)) { + if (write_other(self, NULL, 0) < 0) + goto err; + PDATA_APPEND(self->file, repr, -1); + } + else { + if (self->write_func(self, PyString_AS_STRING(repr), size) < 0) + goto err; + } - Py_DECREF(repr); + Py_DECREF(repr); } if (doput) - if (put(self, args) < 0) - return -1; + if (put(self, args) < 0) + return -1; return 0; @@ -1390,17 +1390,17 @@ store_tuple_elements(Picklerobject * self, PyObject * t, int len) { int i; - int res = -1; /* guilty until proved innocent */ + int res = -1; /* guilty until proved innocent */ assert(PyTuple_Size(t) == len); for (i = 0; i < len; i++) { - PyObject *element = PyTuple_GET_ITEM(t, i); + PyObject *element = PyTuple_GET_ITEM(t, i); - if (element == NULL) - goto finally; - if (save(self, element, 0) < 0) - goto finally; + if (element == NULL) + goto finally; + if (save(self, element, 0) < 0) + goto finally; } res = 0; @@ -1427,24 +1427,24 @@ static char len2opcode[] = { EMPTY_TUPLE, TUPLE1, TUPLE2, TUPLE3 }; if ((len = PyTuple_Size(args)) < 0) - goto finally; + goto finally; if (len == 0) { - char c_str[2]; + char c_str[2]; - if (self->proto) { - c_str[0] = EMPTY_TUPLE; - len = 1; - } - else { - c_str[0] = MARK; - c_str[1] = TUPLE; - len = 2; - } - if (self->write_func(self, c_str, len) >= 0) - res = 0; - /* Don't memoize an empty tuple. */ - goto finally; + if (self->proto) { + c_str[0] = EMPTY_TUPLE; + len = 1; + } + else { + c_str[0] = MARK; + c_str[1] = TUPLE; + len = 2; + } + if (self->write_func(self, c_str, len) >= 0) + res = 0; + /* Don't memoize an empty tuple. */ + goto finally; } /* A non-empty tuple. */ @@ -1456,65 +1456,65 @@ */ py_tuple_id = PyLong_FromVoidPtr(args); if (py_tuple_id == NULL) - goto finally; + goto finally; if (len <= 3 && self->proto >= 2) { - /* Use TUPLE{1,2,3} opcodes. */ - if (store_tuple_elements(self, args, len) < 0) - goto finally; - if (PyDict_GetItem(self->memo, py_tuple_id)) { - /* pop the len elements */ - for (i = 0; i < len; ++i) - if (self->write_func(self, &pop, 1) < 0) - goto finally; - /* fetch from memo */ - if (get(self, py_tuple_id) < 0) - goto finally; - res = 0; - goto finally; - } - /* Not recursive. */ - if (self->write_func(self, len2opcode + len, 1) < 0) - goto finally; - goto memoize; + /* Use TUPLE{1,2,3} opcodes. */ + if (store_tuple_elements(self, args, len) < 0) + goto finally; + if (PyDict_GetItem(self->memo, py_tuple_id)) { + /* pop the len elements */ + for (i = 0; i < len; ++i) + if (self->write_func(self, &pop, 1) < 0) + goto finally; + /* fetch from memo */ + if (get(self, py_tuple_id) < 0) + goto finally; + res = 0; + goto finally; + } + /* Not recursive. */ + if (self->write_func(self, len2opcode + len, 1) < 0) + goto finally; + goto memoize; } /* proto < 2 and len > 0, or proto >= 2 and len > 3. * Generate MARK elt1 elt2 ... TUPLE */ if (self->write_func(self, &MARKv, 1) < 0) - goto finally; + goto finally; if (store_tuple_elements(self, args, len) < 0) - goto finally; + goto finally; if (PyDict_GetItem(self->memo, py_tuple_id)) { - /* pop the stack stuff we pushed */ - if (self->bin) { - if (self->write_func(self, &pop_mark, 1) < 0) - goto finally; - } - else { - /* Note that we pop one more than len, to remove - * the MARK too. - */ - for (i = 0; i <= len; i++) - if (self->write_func(self, &pop, 1) < 0) - goto finally; - } - /* fetch from memo */ - if (get(self, py_tuple_id) >= 0) - res = 0; - goto finally; + /* pop the stack stuff we pushed */ + if (self->bin) { + if (self->write_func(self, &pop_mark, 1) < 0) + goto finally; + } + else { + /* Note that we pop one more than len, to remove + * the MARK too. + */ + for (i = 0; i <= len; i++) + if (self->write_func(self, &pop, 1) < 0) + goto finally; + } + /* fetch from memo */ + if (get(self, py_tuple_id) >= 0) + res = 0; + goto finally; } /* Not recursive. */ if (self->write_func(self, &tuple, 1) < 0) - goto finally; + goto finally; memoize: if (put(self, args) >= 0) - res = 0; + res = 0; finally: Py_XDECREF(py_tuple_id); @@ -1540,64 +1540,64 @@ assert(iter != NULL); if (self->proto == 0) { - /* APPENDS isn't available; do one at a time. */ - for (;;) { - obj = PyIter_Next(iter); - if (obj == NULL) { - if (PyErr_Occurred()) - return -1; - break; - } - i = save(self, obj, 0); - Py_DECREF(obj); - if (i < 0) - return -1; - if (self->write_func(self, &append, 1) < 0) - return -1; - } - return 0; + /* APPENDS isn't available; do one at a time. */ + for (;;) { + obj = PyIter_Next(iter); + if (obj == NULL) { + if (PyErr_Occurred()) + return -1; + break; + } + i = save(self, obj, 0); + Py_DECREF(obj); + if (i < 0) + return -1; + if (self->write_func(self, &append, 1) < 0) + return -1; + } + return 0; } /* proto > 0: write in batches of BATCHSIZE. */ do { - /* Get next group of (no more than) BATCHSIZE elements. */ - for (n = 0; n < BATCHSIZE; ++n) { - obj = PyIter_Next(iter); - if (obj == NULL) { - if (PyErr_Occurred()) - goto BatchFailed; - break; - } - slice[n] = obj; - } - - if (n > 1) { - /* Pump out MARK, slice[0:n], APPENDS. */ - if (self->write_func(self, &MARKv, 1) < 0) - goto BatchFailed; - for (i = 0; i < n; ++i) { - if (save(self, slice[i], 0) < 0) - goto BatchFailed; - } - if (self->write_func(self, &appends, 1) < 0) - goto BatchFailed; - } - else if (n == 1) { - if (save(self, slice[0], 0) < 0) - goto BatchFailed; - if (self->write_func(self, &append, 1) < 0) - goto BatchFailed; - } - - for (i = 0; i < n; ++i) { - Py_DECREF(slice[i]); - } + /* Get next group of (no more than) BATCHSIZE elements. */ + for (n = 0; n < BATCHSIZE; ++n) { + obj = PyIter_Next(iter); + if (obj == NULL) { + if (PyErr_Occurred()) + goto BatchFailed; + break; + } + slice[n] = obj; + } + + if (n > 1) { + /* Pump out MARK, slice[0:n], APPENDS. */ + if (self->write_func(self, &MARKv, 1) < 0) + goto BatchFailed; + for (i = 0; i < n; ++i) { + if (save(self, slice[i], 0) < 0) + goto BatchFailed; + } + if (self->write_func(self, &appends, 1) < 0) + goto BatchFailed; + } + else if (n == 1) { + if (save(self, slice[0], 0) < 0) + goto BatchFailed; + if (self->write_func(self, &append, 1) < 0) + goto BatchFailed; + } + + for (i = 0; i < n; ++i) { + Py_DECREF(slice[i]); + } } while (n == BATCHSIZE); return 0; BatchFailed: while (--n >= 0) { - Py_DECREF(slice[n]); + Py_DECREF(slice[n]); } return -1; } @@ -1611,45 +1611,45 @@ PyObject *iter; if (self->fast && !fast_save_enter(self, args)) - goto finally; + goto finally; /* Create an empty list. */ if (self->bin) { - s[0] = EMPTY_LIST; - len = 1; + s[0] = EMPTY_LIST; + len = 1; } else { - s[0] = MARK; - s[1] = LIST; - len = 2; + s[0] = MARK; + s[1] = LIST; + len = 2; } if (self->write_func(self, s, len) < 0) - goto finally; + goto finally; /* Get list length, and bow out early if empty. */ if ((len = PyList_Size(args)) < 0) - goto finally; + goto finally; /* Memoize. */ if (len == 0) { - if (put(self, args) >= 0) - res = 0; - goto finally; + if (put(self, args) >= 0) + res = 0; + goto finally; } if (put2(self, args) < 0) - goto finally; + goto finally; /* Materialize the list elements. */ iter = PyObject_GetIter(args); if (iter == NULL) - goto finally; + goto finally; res = batch_list(self, iter); Py_DECREF(iter); finally: if (self->fast && !fast_save_leave(self, args)) - res = -1; + res = -1; return res; } @@ -1679,82 +1679,82 @@ assert(iter != NULL); if (self->proto == 0) { - /* SETITEMS isn't available; do one at a time. */ - for (;;) { - p = PyIter_Next(iter); - if (p == NULL) { - if (PyErr_Occurred()) - return -1; - break; - } - if (!PyTuple_Check(p) || PyTuple_Size(p) != 2) { - PyErr_SetString(PyExc_TypeError, "dict items " - "iterator must return 2-tuples"); - return -1; - } - i = save(self, PyTuple_GET_ITEM(p, 0), 0); - if (i >= 0) - i = save(self, PyTuple_GET_ITEM(p, 1), 0); - Py_DECREF(p); - if (i < 0) - return -1; - if (self->write_func(self, &setitem, 1) < 0) - return -1; - } - return 0; + /* SETITEMS isn't available; do one at a time. */ + for (;;) { + p = PyIter_Next(iter); + if (p == NULL) { + if (PyErr_Occurred()) + return -1; + break; + } + if (!PyTuple_Check(p) || PyTuple_Size(p) != 2) { + PyErr_SetString(PyExc_TypeError, "dict items " + "iterator must return 2-tuples"); + return -1; + } + i = save(self, PyTuple_GET_ITEM(p, 0), 0); + if (i >= 0) + i = save(self, PyTuple_GET_ITEM(p, 1), 0); + Py_DECREF(p); + if (i < 0) + return -1; + if (self->write_func(self, &setitem, 1) < 0) + return -1; + } + return 0; } /* proto > 0: write in batches of BATCHSIZE. */ do { - /* Get next group of (no more than) BATCHSIZE elements. */ - for (n = 0; n < BATCHSIZE; ++n) { - p = PyIter_Next(iter); - if (p == NULL) { - if (PyErr_Occurred()) - goto BatchFailed; - break; - } - if (!PyTuple_Check(p) || PyTuple_Size(p) != 2) { - PyErr_SetString(PyExc_TypeError, "dict items " - "iterator must return 2-tuples"); - goto BatchFailed; - } - slice[n] = p; - } - - if (n > 1) { - /* Pump out MARK, slice[0:n], SETITEMS. */ - if (self->write_func(self, &MARKv, 1) < 0) - goto BatchFailed; - for (i = 0; i < n; ++i) { - p = slice[i]; - if (save(self, PyTuple_GET_ITEM(p, 0), 0) < 0) - goto BatchFailed; - if (save(self, PyTuple_GET_ITEM(p, 1), 0) < 0) - goto BatchFailed; - } - if (self->write_func(self, &setitems, 1) < 0) - goto BatchFailed; - } - else if (n == 1) { - p = slice[0]; - if (save(self, PyTuple_GET_ITEM(p, 0), 0) < 0) - goto BatchFailed; - if (save(self, PyTuple_GET_ITEM(p, 1), 0) < 0) - goto BatchFailed; - if (self->write_func(self, &setitem, 1) < 0) - goto BatchFailed; - } - - for (i = 0; i < n; ++i) { - Py_DECREF(slice[i]); - } + /* Get next group of (no more than) BATCHSIZE elements. */ + for (n = 0; n < BATCHSIZE; ++n) { + p = PyIter_Next(iter); + if (p == NULL) { + if (PyErr_Occurred()) + goto BatchFailed; + break; + } + if (!PyTuple_Check(p) || PyTuple_Size(p) != 2) { + PyErr_SetString(PyExc_TypeError, "dict items " + "iterator must return 2-tuples"); + goto BatchFailed; + } + slice[n] = p; + } + + if (n > 1) { + /* Pump out MARK, slice[0:n], SETITEMS. */ + if (self->write_func(self, &MARKv, 1) < 0) + goto BatchFailed; + for (i = 0; i < n; ++i) { + p = slice[i]; + if (save(self, PyTuple_GET_ITEM(p, 0), 0) < 0) + goto BatchFailed; + if (save(self, PyTuple_GET_ITEM(p, 1), 0) < 0) + goto BatchFailed; + } + if (self->write_func(self, &setitems, 1) < 0) + goto BatchFailed; + } + else if (n == 1) { + p = slice[0]; + if (save(self, PyTuple_GET_ITEM(p, 0), 0) < 0) + goto BatchFailed; + if (save(self, PyTuple_GET_ITEM(p, 1), 0) < 0) + goto BatchFailed; + if (self->write_func(self, &setitem, 1) < 0) + goto BatchFailed; + } + + for (i = 0; i < n; ++i) { + Py_DECREF(slice[i]); + } } while (n == BATCHSIZE); return 0; BatchFailed: while (--n >= 0) { - Py_DECREF(slice[n]); + Py_DECREF(slice[n]); } return -1; } @@ -1768,48 +1768,48 @@ PyObject *items, *iter; if (self->fast && !fast_save_enter(self, args)) - goto finally; + goto finally; /* Create an empty dict. */ if (self->bin) { - s[0] = EMPTY_DICT; - len = 1; + s[0] = EMPTY_DICT; + len = 1; } else { - s[0] = MARK; - s[1] = DICT; - len = 2; + s[0] = MARK; + s[1] = DICT; + len = 2; } if (self->write_func(self, s, len) < 0) - goto finally; + goto finally; /* Get dict size, and bow out early if empty. */ if ((len = PyDict_Size(args)) < 0) - goto finally; + goto finally; if (len == 0) { - if (put(self, args) >= 0) - res = 0; - goto finally; + if (put(self, args) >= 0) + res = 0; + goto finally; } if (put2(self, args) < 0) - goto finally; + goto finally; /* Materialize the dict items. */ items = PyObject_CallMethod(args, "items", "()"); if (items == NULL) - goto finally; + goto finally; iter = PyObject_GetIter(items); Py_DECREF(items); if (iter == NULL) - goto finally; + goto finally; res = batch_dict(self, iter); Py_DECREF(iter); finally: if (self->fast && !fast_save_leave(self, args)) - res = -1; + res = -1; return res; } @@ -1825,20 +1825,20 @@ static char global = GLOBAL; if (name) { - global_name = name; - Py_INCREF(global_name); + global_name = name; + Py_INCREF(global_name); } else { - if (!(global_name = PyObject_GetAttr(args, __name___str))) - goto finally; + if (!(global_name = PyObject_GetAttr(args, __name___str))) + goto finally; } if (!(module = whichmodule(args, global_name))) - goto finally; + goto finally; if ((module_size = PyString_Size(module)) < 0 || - (name_size = PyString_Size(global_name)) < 0) - goto finally; + (name_size = PyString_Size(global_name)) < 0) + goto finally; module_str = PyString_AS_STRING((PyStringObject *) module); name_str = PyString_AS_STRING((PyStringObject *) global_name); @@ -1847,101 +1847,101 @@ * but I don't know how to stop it. :-( */ mod = PyImport_ImportModule(module_str); if (mod == NULL) { - cPickle_ErrFormat(PicklingError, - "Can't pickle %s: import of module %s " - "failed", "OS", args, module); - goto finally; + cPickle_ErrFormat(PicklingError, + "Can't pickle %s: import of module %s " + "failed", "OS", args, module); + goto finally; } klass = PyObject_GetAttrString(mod, name_str); if (klass == NULL) { - cPickle_ErrFormat(PicklingError, - "Can't pickle %s: attribute lookup %s.%s " - "failed", "OSS", args, module, global_name); - goto finally; + cPickle_ErrFormat(PicklingError, + "Can't pickle %s: attribute lookup %s.%s " + "failed", "OSS", args, module, global_name); + goto finally; } if (klass != args) { - Py_DECREF(klass); - cPickle_ErrFormat(PicklingError, - "Can't pickle %s: it's not the same object " - "as %s.%s", "OSS", args, module, global_name); - goto finally; + Py_DECREF(klass); + cPickle_ErrFormat(PicklingError, + "Can't pickle %s: it's not the same object " + "as %s.%s", "OSS", args, module, global_name); + goto finally; } Py_DECREF(klass); if (self->proto >= 2) { - /* See whether this is in the extension registry, and if - * so generate an EXT opcode. - */ - PyObject *py_code; /* extension code as Python object */ - long code; /* extension code as C value */ - char c_str[5]; - int n; - - PyTuple_SET_ITEM(two_tuple, 0, module); - PyTuple_SET_ITEM(two_tuple, 1, global_name); - py_code = PyDict_GetItem(extension_registry, two_tuple); - if (py_code == NULL) - goto gen_global; /* not registered */ - - /* Verify py_code has the right type and value. */ - if (!PyInt_Check(py_code)) { - cPickle_ErrFormat(PicklingError, "Can't pickle %s: " - "extension code %s isn't an integer", - "OO", args, py_code); - goto finally; - } - code = PyInt_AS_LONG(py_code); - if (code <= 0 || code > 0x7fffffffL) { - cPickle_ErrFormat(PicklingError, "Can't pickle %s: " - "extension code %ld is out of range", - "Ol", args, code); - goto finally; - } - - /* Generate an EXT opcode. */ - if (code <= 0xff) { - c_str[0] = EXT1; - c_str[1] = (char) code; - n = 2; - } - else if (code <= 0xffff) { - c_str[0] = EXT2; - c_str[1] = (char) (code & 0xff); - c_str[2] = (char) ((code >> 8) & 0xff); - n = 3; - } - else { - c_str[0] = EXT4; - c_str[1] = (char) (code & 0xff); - c_str[2] = (char) ((code >> 8) & 0xff); - c_str[3] = (char) ((code >> 16) & 0xff); - c_str[4] = (char) ((code >> 24) & 0xff); - n = 5; - } - - if (self->write_func(self, c_str, n) >= 0) - res = 0; - goto finally; /* and don't memoize */ + /* See whether this is in the extension registry, and if + * so generate an EXT opcode. + */ + PyObject *py_code; /* extension code as Python object */ + long code; /* extension code as C value */ + char c_str[5]; + int n; + + PyTuple_SET_ITEM(two_tuple, 0, module); + PyTuple_SET_ITEM(two_tuple, 1, global_name); + py_code = PyDict_GetItem(extension_registry, two_tuple); + if (py_code == NULL) + goto gen_global; /* not registered */ + + /* Verify py_code has the right type and value. */ + if (!PyInt_Check(py_code)) { + cPickle_ErrFormat(PicklingError, "Can't pickle %s: " + "extension code %s isn't an integer", + "OO", args, py_code); + goto finally; + } + code = PyInt_AS_LONG(py_code); + if (code <= 0 || code > 0x7fffffffL) { + cPickle_ErrFormat(PicklingError, "Can't pickle %s: " + "extension code %ld is out of range", + "Ol", args, code); + goto finally; + } + + /* Generate an EXT opcode. */ + if (code <= 0xff) { + c_str[0] = EXT1; + c_str[1] = (char) code; + n = 2; + } + else if (code <= 0xffff) { + c_str[0] = EXT2; + c_str[1] = (char) (code & 0xff); + c_str[2] = (char) ((code >> 8) & 0xff); + n = 3; + } + else { + c_str[0] = EXT4; + c_str[1] = (char) (code & 0xff); + c_str[2] = (char) ((code >> 8) & 0xff); + c_str[3] = (char) ((code >> 16) & 0xff); + c_str[4] = (char) ((code >> 24) & 0xff); + n = 5; + } + + if (self->write_func(self, c_str, n) >= 0) + res = 0; + goto finally; /* and don't memoize */ } gen_global: if (self->write_func(self, &global, 1) < 0) - goto finally; + goto finally; if (self->write_func(self, module_str, module_size) < 0) - goto finally; + goto finally; if (self->write_func(self, "\n", 1) < 0) - goto finally; + goto finally; if (self->write_func(self, name_str, name_size) < 0) - goto finally; + goto finally; if (self->write_func(self, "\n", 1) < 0) - goto finally; + goto finally; if (put(self, args) < 0) - goto finally; + goto finally; res = 0; @@ -1964,44 +1964,44 @@ Py_INCREF(args); ARG_TUP(self, args); if (self->arg) { - pid = PyObject_Call(f, self->arg, NULL); - FREE_ARG_TUP(self); + pid = PyObject_Call(f, self->arg, NULL); + FREE_ARG_TUP(self); } if (!pid) - return -1; + return -1; if (pid != Py_None) { - if (!self->bin) { - if (!PyString_Check(pid)) { - PyErr_SetString(PicklingError, "persistent id must be string"); - goto finally; - } - - if (self->write_func(self, &persid, 1) < 0) - goto finally; - - if ((size = PyString_Size(pid)) < 0) - goto finally; - - if (self->write_func(self, - PyString_AS_STRING((PyStringObject *) pid), - size) < 0) - goto finally; - - if (self->write_func(self, "\n", 1) < 0) - goto finally; - - res = 1; - goto finally; - } - else if (save(self, pid, 1) >= 0) { - if (self->write_func(self, &binpersid, 1) < 0) - res = -1; - else - res = 1; - } + if (!self->bin) { + if (!PyString_Check(pid)) { + PyErr_SetString(PicklingError, "persistent id must be string"); + goto finally; + } + + if (self->write_func(self, &persid, 1) < 0) + goto finally; + + if ((size = PyString_Size(pid)) < 0) + goto finally; + + if (self->write_func(self, + PyString_AS_STRING((PyStringObject *) pid), + size) < 0) + goto finally; + + if (self->write_func(self, "\n", 1) < 0) + goto finally; + + res = 1; + goto finally; + } + else if (save(self, pid, 1) >= 0) { + if (self->write_func(self, &binpersid, 1) < 0) + res = -1; + else + res = 1; + } - goto finally; + goto finally; } res = 0; @@ -2031,129 +2031,129 @@ static char newobj = NEWOBJ; if (!PyArg_UnpackTuple(args, "save_reduce", 2, 5, - &callable, &argtup, &state, &listitems, &dictitems)) - return -1; + &callable, &argtup, &state, &listitems, &dictitems)) + return -1; if (!PyTuple_Check(argtup)) { - PyErr_SetString(PicklingError, "args from reduce() should be a tuple"); - return -1; + PyErr_SetString(PicklingError, "args from reduce() should be a tuple"); + return -1; } if (state == Py_None) - state = NULL; + state = NULL; if (listitems == Py_None) - listitems = NULL; + listitems = NULL; if (dictitems == Py_None) - dictitems = NULL; + dictitems = NULL; /* Protocol 2 special case: if callable's name is __newobj__, use * NEWOBJ. This consumes a lot of code. */ if (use_newobj) { - PyObject *temp = PyObject_GetAttr(callable, __name___str); + PyObject *temp = PyObject_GetAttr(callable, __name___str); - if (temp == NULL) { - if (PyErr_ExceptionMatches(PyExc_AttributeError)) - PyErr_Clear(); - else - return -1; - use_newobj = 0; - } - else { - use_newobj = PyString_Check(temp) && - strcmp(PyString_AS_STRING(temp), "__newobj__") == 0; - Py_DECREF(temp); - } + if (temp == NULL) { + if (PyErr_ExceptionMatches(PyExc_AttributeError)) + PyErr_Clear(); + else + return -1; + use_newobj = 0; + } + else { + use_newobj = PyString_Check(temp) && + strcmp(PyString_AS_STRING(temp), "__newobj__") == 0; + Py_DECREF(temp); + } } if (use_newobj) { - PyObject *cls; - PyObject *newargtup; - int n, i; - - /* Sanity checks. */ - n = PyTuple_Size(argtup); - if (n < 1) { - PyErr_SetString(PicklingError, "__newobj__ arglist " "is empty"); - return -1; - } - - cls = PyTuple_GET_ITEM(argtup, 0); - if (!PyObject_HasAttrString(cls, "__new__")) { - PyErr_SetString(PicklingError, "args[0] from " - "__newobj__ args has no __new__"); - return -1; - } - - /* XXX How could ob be NULL? */ - if (ob != NULL) { - PyObject *ob_dot_class; - - ob_dot_class = PyObject_GetAttr(ob, __class___str); - if (ob_dot_class == NULL) { - if (PyErr_ExceptionMatches(PyExc_AttributeError)) - PyErr_Clear(); - else - return -1; - } - i = ob_dot_class != cls; /* true iff a problem */ - Py_XDECREF(ob_dot_class); - if (i) { - PyErr_SetString(PicklingError, "args[0] from " - "__newobj__ args has the wrong class"); - return -1; - } - } - - /* Save the class and its __new__ arguments. */ - if (save(self, cls, 0) < 0) - return -1; - - newargtup = PyTuple_New(n - 1); /* argtup[1:] */ - if (newargtup == NULL) - return -1; - for (i = 1; i < n; ++i) { - PyObject *temp = PyTuple_GET_ITEM(argtup, i); - Py_INCREF(temp); - PyTuple_SET_ITEM(newargtup, i - 1, temp); - } - i = save(self, newargtup, 0) < 0; - Py_DECREF(newargtup); - if (i < 0) - return -1; - - /* Add NEWOBJ opcode. */ - if (self->write_func(self, &newobj, 1) < 0) - return -1; + PyObject *cls; + PyObject *newargtup; + int n, i; + + /* Sanity checks. */ + n = PyTuple_Size(argtup); + if (n < 1) { + PyErr_SetString(PicklingError, "__newobj__ arglist " "is empty"); + return -1; + } + + cls = PyTuple_GET_ITEM(argtup, 0); + if (!PyObject_HasAttrString(cls, "__new__")) { + PyErr_SetString(PicklingError, "args[0] from " + "__newobj__ args has no __new__"); + return -1; + } + + /* XXX How could ob be NULL? */ + if (ob != NULL) { + PyObject *ob_dot_class; + + ob_dot_class = PyObject_GetAttr(ob, __class___str); + if (ob_dot_class == NULL) { + if (PyErr_ExceptionMatches(PyExc_AttributeError)) + PyErr_Clear(); + else + return -1; + } + i = ob_dot_class != cls; /* true iff a problem */ + Py_XDECREF(ob_dot_class); + if (i) { + PyErr_SetString(PicklingError, "args[0] from " + "__newobj__ args has the wrong class"); + return -1; + } + } + + /* Save the class and its __new__ arguments. */ + if (save(self, cls, 0) < 0) + return -1; + + newargtup = PyTuple_New(n - 1); /* argtup[1:] */ + if (newargtup == NULL) + return -1; + for (i = 1; i < n; ++i) { + PyObject *temp = PyTuple_GET_ITEM(argtup, i); + Py_INCREF(temp); + PyTuple_SET_ITEM(newargtup, i - 1, temp); + } + i = save(self, newargtup, 0) < 0; + Py_DECREF(newargtup); + if (i < 0) + return -1; + + /* Add NEWOBJ opcode. */ + if (self->write_func(self, &newobj, 1) < 0) + return -1; } else { - /* Not using NEWOBJ. */ - if (save(self, callable, 0) < 0 || - save(self, argtup, 0) < 0 || - self->write_func(self, &reduce, 1) < 0) - return -1; + /* Not using NEWOBJ. */ + if (save(self, callable, 0) < 0 || + save(self, argtup, 0) < 0 || + self->write_func(self, &reduce, 1) < 0) + return -1; } /* Memoize. */ /* XXX How can ob be NULL? */ if (ob != NULL) { - if (state && !PyDict_Check(state)) { - if (put2(self, ob) < 0) - return -1; - } - else if (put(self, ob) < 0) - return -1; + if (state && !PyDict_Check(state)) { + if (put2(self, ob) < 0) + return -1; + } + else if (put(self, ob) < 0) + return -1; } if (listitems && batch_list(self, listitems) < 0) - return -1; + return -1; if (dictitems && batch_dict(self, dictitems) < 0) - return -1; + return -1; if (state) { - if (save(self, state, 0) < 0 || self->write_func(self, &build, 1) < 0) - return -1; + if (save(self, state, 0) < 0 || self->write_func(self, &build, 1) < 0) + return -1; } return 0; @@ -2169,158 +2169,158 @@ int tmp, size; if (self->nesting++ > Py_GetRecursionLimit()) { - PyErr_SetString(PyExc_RuntimeError, - "maximum recursion depth exceeded"); - goto finally; + PyErr_SetString(PyExc_RuntimeError, + "maximum recursion depth exceeded"); + goto finally; } if (!pers_save && self->pers_func) { - if ((tmp = save_pers(self, args, self->pers_func)) != 0) { - res = tmp; - goto finally; - } + if ((tmp = save_pers(self, args, self->pers_func)) != 0) { + res = tmp; + goto finally; + } } if (args == Py_None) { - res = save_none(self, args); - goto finally; + res = save_none(self, args); + goto finally; } type = args->ob_type; switch (type->tp_name[0]) { case 'b': - if (args == Py_False || args == Py_True) { - res = save_bool(self, args); - goto finally; - } - break; + if (args == Py_False || args == Py_True) { + res = save_bool(self, args); + goto finally; + } + break; case 'i': - if (type == &PyLong_Type) { - res = save_long(self, args); - goto finally; - } - break; + if (type == &PyLong_Type) { + res = save_long(self, args); + goto finally; + } + break; case 'f': - if (type == &PyFloat_Type) { - res = save_float(self, args); - goto finally; - } - break; + if (type == &PyFloat_Type) { + res = save_float(self, args); + goto finally; + } + break; case 't': - if (type == &PyTuple_Type && PyTuple_Size(args) == 0) { - res = save_tuple(self, args); - goto finally; - } - break; + if (type == &PyTuple_Type && PyTuple_Size(args) == 0) { + res = save_tuple(self, args); + goto finally; + } + break; case 's': - if ((type == &PyString_Type) && (PyString_GET_SIZE(args) < 2)) { - res = save_string(self, args, 0); - goto finally; - } + if ((type == &PyString_Type) && (PyString_GET_SIZE(args) < 2)) { + res = save_string(self, args, 0); + goto finally; + } #ifdef Py_USING_UNICODE case 'u': - if ((type == &PyUnicode_Type) && (PyString_GET_SIZE(args) < 2)) { - res = save_unicode(self, args, 0); - goto finally; - } + if ((type == &PyUnicode_Type) && (PyString_GET_SIZE(args) < 2)) { + res = save_unicode(self, args, 0); + goto finally; + } #endif } if (args->ob_refcnt > 1) { - if (!(py_ob_id = PyLong_FromVoidPtr(args))) - goto finally; + if (!(py_ob_id = PyLong_FromVoidPtr(args))) + goto finally; - if (PyDict_GetItem(self->memo, py_ob_id)) { - if (get(self, py_ob_id) < 0) - goto finally; - - res = 0; - goto finally; - } + if (PyDict_GetItem(self->memo, py_ob_id)) { + if (get(self, py_ob_id) < 0) + goto finally; + + res = 0; + goto finally; + } } switch (type->tp_name[0]) { case 's': - if (type == &PyString_Type) { - res = save_string(self, args, 1); - goto finally; - } - break; + if (type == &PyString_Type) { + res = save_string(self, args, 1); + goto finally; + } + break; #ifdef Py_USING_UNICODE case 'u': - if (type == &PyUnicode_Type) { - res = save_unicode(self, args, 1); - goto finally; - } - break; + if (type == &PyUnicode_Type) { + res = save_unicode(self, args, 1); + goto finally; + } + break; #endif case 't': - if (type == &PyTuple_Type) { - res = save_tuple(self, args); - goto finally; - } - if (type == &PyType_Type) { - res = save_global(self, args, NULL); - goto finally; - } - break; + if (type == &PyTuple_Type) { + res = save_tuple(self, args); + goto finally; + } + if (type == &PyType_Type) { + res = save_global(self, args, NULL); + goto finally; + } + break; case 'l': - if (type == &PyList_Type) { - res = save_list(self, args); - goto finally; - } - break; + if (type == &PyList_Type) { + res = save_list(self, args); + goto finally; + } + break; case 'd': - if (type == &PyDict_Type) { - res = save_dict(self, args); - goto finally; - } - break; + if (type == &PyDict_Type) { + res = save_dict(self, args); + goto finally; + } + break; case 'i': - break; + break; case 'c': - break; + break; case 'f': - if (type == &PyFunction_Type) { - res = save_global(self, args, NULL); - if (res && PyErr_ExceptionMatches(PickleError)) { - /* fall back to reduce */ - PyErr_Clear(); - break; - } - goto finally; - } - break; + if (type == &PyFunction_Type) { + res = save_global(self, args, NULL); + if (res && PyErr_ExceptionMatches(PickleError)) { + /* fall back to reduce */ + PyErr_Clear(); + break; + } + goto finally; + } + break; case 'b': - if (type == &PyCFunction_Type) { - res = save_global(self, args, NULL); - goto finally; - } + if (type == &PyCFunction_Type) { + res = save_global(self, args, NULL); + goto finally; + } } if (!pers_save && self->inst_pers_func) { - if ((tmp = save_pers(self, args, self->inst_pers_func)) != 0) { - res = tmp; - goto finally; - } + if ((tmp = save_pers(self, args, self->inst_pers_func)) != 0) { + res = tmp; + goto finally; + } } if (PyType_IsSubtype(type, &PyType_Type)) { - res = save_global(self, args, NULL); - goto finally; + res = save_global(self, args, NULL); + goto finally; } /* Get a reduction callable, and call it. This may come from @@ -2329,73 +2329,73 @@ */ __reduce__ = PyDict_GetItem(dispatch_table, (PyObject *) type); if (__reduce__ != NULL) { - Py_INCREF(__reduce__); - Py_INCREF(args); - ARG_TUP(self, args); - if (self->arg) { - t = PyObject_Call(__reduce__, self->arg, NULL); - FREE_ARG_TUP(self); - } + Py_INCREF(__reduce__); + Py_INCREF(args); + ARG_TUP(self, args); + if (self->arg) { + t = PyObject_Call(__reduce__, self->arg, NULL); + FREE_ARG_TUP(self); + } } else { - /* Check for a __reduce_ex__ method. */ - __reduce__ = PyObject_GetAttr(args, __reduce_ex___str); - if (__reduce__ != NULL) { - t = PyInt_FromLong(self->proto); - if (t != NULL) { - ARG_TUP(self, t); - t = NULL; - if (self->arg) { - t = PyObject_Call(__reduce__, self->arg, NULL); - FREE_ARG_TUP(self); - } - } - } - else { - if (PyErr_ExceptionMatches(PyExc_AttributeError)) - PyErr_Clear(); - else - goto finally; - /* Check for a __reduce__ method. */ - __reduce__ = PyObject_GetAttr(args, __reduce___str); - if (__reduce__ != NULL) { - t = PyObject_Call(__reduce__, empty_tuple, NULL); - } - else { - PyErr_SetObject(UnpickleableError, args); - goto finally; - } - } + /* Check for a __reduce_ex__ method. */ + __reduce__ = PyObject_GetAttr(args, __reduce_ex___str); + if (__reduce__ != NULL) { + t = PyInt_FromLong(self->proto); + if (t != NULL) { + ARG_TUP(self, t); + t = NULL; + if (self->arg) { + t = PyObject_Call(__reduce__, self->arg, NULL); + FREE_ARG_TUP(self); + } + } + } + else { + if (PyErr_ExceptionMatches(PyExc_AttributeError)) + PyErr_Clear(); + else + goto finally; + /* Check for a __reduce__ method. */ + __reduce__ = PyObject_GetAttr(args, __reduce___str); + if (__reduce__ != NULL) { + t = PyObject_Call(__reduce__, empty_tuple, NULL); + } + else { + PyErr_SetObject(UnpickleableError, args); + goto finally; + } + } } if (t == NULL) - goto finally; + goto finally; if (PyString_Check(t)) { - res = save_global(self, args, t); - goto finally; + res = save_global(self, args, t); + goto finally; } if (!PyTuple_Check(t)) { - cPickle_ErrFormat(PicklingError, "Value returned by " - "%s must be string or tuple", "O", __reduce__); - goto finally; + cPickle_ErrFormat(PicklingError, "Value returned by " + "%s must be string or tuple", "O", __reduce__); + goto finally; } size = PyTuple_Size(t); if (size < 2 || size > 5) { - cPickle_ErrFormat(PicklingError, "tuple returned by " - "%s must contain 2 through 5 elements", - "O", __reduce__); - goto finally; + cPickle_ErrFormat(PicklingError, "tuple returned by " + "%s must contain 2 through 5 elements", + "O", __reduce__); + goto finally; } arg_tup = PyTuple_GET_ITEM(t, 1); if (!(PyTuple_Check(arg_tup) || arg_tup == Py_None)) { - cPickle_ErrFormat(PicklingError, "Second element of " - "tuple returned by %s must be a tuple", - "O", __reduce__); - goto finally; + cPickle_ErrFormat(PicklingError, "Second element of " + "tuple returned by %s must be a tuple", + "O", __reduce__); + goto finally; } res = save_reduce(self, t, args); @@ -2416,23 +2416,23 @@ static char stop = STOP; if (self->proto >= 2) { - char bytes[2]; + char bytes[2]; - bytes[0] = PROTO; - assert(self->proto >= 0 && self->proto < 256); - bytes[1] = (char) self->proto; - if (self->write_func(self, bytes, 2) < 0) - return -1; + bytes[0] = PROTO; + assert(self->proto >= 0 && self->proto < 256); + bytes[1] = (char) self->proto; + if (self->write_func(self, bytes, 2) < 0) + return -1; } if (save(self, args, 0) < 0) - return -1; + return -1; if (self->write_func(self, &stop, 1) < 0) - return -1; + return -1; if (self->write_func(self, NULL, 0) < 0) - return -1; + return -1; return 0; } @@ -2441,7 +2441,7 @@ Pickle_clear_memo(Picklerobject * self, PyObject * args) { if (self->memo) - PyDict_Clear(self->memo); + PyDict_Clear(self->memo); Py_INCREF(Py_None); return Py_None; } @@ -2457,18 +2457,18 @@ /* Can be called by Python code or C code */ if (args && !PyArg_ParseTuple(args, "|i:getvalue", &clear)) - return NULL; + return NULL; /* Check to make sure we are based on a list */ if (!Pdata_Check(self->file)) { - PyErr_SetString(PicklingError, - "Attempt to getvalue() a non-list-based pickler"); - return NULL; + PyErr_SetString(PicklingError, + "Attempt to getvalue() a non-list-based pickler"); + return NULL; } /* flush write buffer */ if (write_other(self, NULL, 0) < 0) - return NULL; + return NULL; data = (Pdata *) self->file; l = data->length; @@ -2476,112 +2476,112 @@ /* set up an array to hold get/put status */ lm = PyDict_Size(self->memo); if (lm < 0) - return NULL; + return NULL; lm++; have_get = malloc(lm); if (have_get == NULL) - return PyErr_NoMemory(); + return PyErr_NoMemory(); memset(have_get, 0, lm); /* Scan for gets. */ for (rsize = 0, i = l; --i >= 0;) { - k = data->data[i]; + k = data->data[i]; - if (PyString_Check(k)) - rsize += PyString_GET_SIZE(k); + if (PyString_Check(k)) + rsize += PyString_GET_SIZE(k); - else if (PyInt_Check(k)) { /* put */ - ik = PyInt_AsLong(k); - if (ik == -1 && PyErr_Occurred()) - goto err; - if (ik >= lm || ik == 0) { - PyErr_SetString(PicklingError, "Invalid get data"); - goto err; - } - if (have_get[ik]) /* with matching get */ - rsize += ik < 256 ? 2 : 5; - } - - else if (!(PyTuple_Check(k) && - PyTuple_GET_SIZE(k) == 2 && - PyInt_Check((k = PyTuple_GET_ITEM(k, 0)))) - ) { - PyErr_SetString(PicklingError, "Unexpected data in internal list"); - goto err; - } - - else { /* put */ - ik = PyInt_AsLong(k); - if (ik == -1 && PyErr_Occurred()) - goto err; - if (ik >= lm || ik == 0) { - PyErr_SetString(PicklingError, "Invalid get data"); - return NULL; - } - have_get[ik] = 1; - rsize += ik < 256 ? 2 : 5; - } + else if (PyInt_Check(k)) { /* put */ + ik = PyInt_AsLong(k); + if (ik == -1 && PyErr_Occurred()) + goto err; + if (ik >= lm || ik == 0) { + PyErr_SetString(PicklingError, "Invalid get data"); + goto err; + } + if (have_get[ik]) /* with matching get */ + rsize += ik < 256 ? 2 : 5; + } + + else if (!(PyTuple_Check(k) && + PyTuple_GET_SIZE(k) == 2 && + PyInt_Check((k = PyTuple_GET_ITEM(k, 0)))) + ) { + PyErr_SetString(PicklingError, "Unexpected data in internal list"); + goto err; + } + + else { /* put */ + ik = PyInt_AsLong(k); + if (ik == -1 && PyErr_Occurred()) + goto err; + if (ik >= lm || ik == 0) { + PyErr_SetString(PicklingError, "Invalid get data"); + return NULL; + } + have_get[ik] = 1; + rsize += ik < 256 ? 2 : 5; + } } /* Now generate the result */ r = PyString_FromStringAndSize(NULL, rsize); if (r == NULL) - goto err; + goto err; s = PyString_AS_STRING((PyStringObject *) r); for (i = 0; i < l; i++) { - k = data->data[i]; + k = data->data[i]; - if (PyString_Check(k)) { - ssize = PyString_GET_SIZE(k); - if (ssize) { - p = PyString_AS_STRING((PyStringObject *) k); - while (--ssize >= 0) - *s++ = *p++; - } - } - - else if (PyTuple_Check(k)) { /* get */ - ik = PyLong_AsLong(PyTuple_GET_ITEM(k, 0)); - if (ik == -1 && PyErr_Occurred()) - goto err; - if (ik < 256) { - *s++ = BINGET; - *s++ = (int) (ik & 0xff); - } - else { - *s++ = LONG_BINGET; - *s++ = (int) (ik & 0xff); - *s++ = (int) ((ik >> 8) & 0xff); - *s++ = (int) ((ik >> 16) & 0xff); - *s++ = (int) ((ik >> 24) & 0xff); - } - } - - else { /* put */ - ik = PyLong_AsLong(k); - if (ik == -1 && PyErr_Occurred()) - goto err; - - if (have_get[ik]) { /* with matching get */ - if (ik < 256) { - *s++ = BINPUT; - *s++ = (int) (ik & 0xff); - } - else { - *s++ = LONG_BINPUT; - *s++ = (int) (ik & 0xff); - *s++ = (int) ((ik >> 8) & 0xff); - *s++ = (int) ((ik >> 16) & 0xff); - *s++ = (int) ((ik >> 24) & 0xff); - } - } - } + if (PyString_Check(k)) { + ssize = PyString_GET_SIZE(k); + if (ssize) { + p = PyString_AS_STRING((PyStringObject *) k); + while (--ssize >= 0) + *s++ = *p++; + } + } + + else if (PyTuple_Check(k)) { /* get */ + ik = PyLong_AsLong(PyTuple_GET_ITEM(k, 0)); + if (ik == -1 && PyErr_Occurred()) + goto err; + if (ik < 256) { + *s++ = BINGET; + *s++ = (int) (ik & 0xff); + } + else { + *s++ = LONG_BINGET; + *s++ = (int) (ik & 0xff); + *s++ = (int) ((ik >> 8) & 0xff); + *s++ = (int) ((ik >> 16) & 0xff); + *s++ = (int) ((ik >> 24) & 0xff); + } + } + + else { /* put */ + ik = PyLong_AsLong(k); + if (ik == -1 && PyErr_Occurred()) + goto err; + + if (have_get[ik]) { /* with matching get */ + if (ik < 256) { + *s++ = BINPUT; + *s++ = (int) (ik & 0xff); + } + else { + *s++ = LONG_BINPUT; + *s++ = (int) (ik & 0xff); + *s++ = (int) ((ik >> 8) & 0xff); + *s++ = (int) ((ik >> 16) & 0xff); + *s++ = (int) ((ik >> 24) & 0xff); + } + } + } } if (clear) { - PyDict_Clear(self->memo); - Pdata_clear(data, 0); + PyDict_Clear(self->memo); + Pdata_clear(data, 0); } free(have_get); @@ -2598,13 +2598,13 @@ int get = 0; if (!(PyArg_ParseTuple(args, "O|i:dump", &ob, &get))) - return NULL; + return NULL; if (dump(self, ob) < 0) - return NULL; + return NULL; if (get) - return Pickle_getvalue(self, NULL); + return Pickle_getvalue(self, NULL); /* XXX Why does dump() return self? */ Py_INCREF(self); @@ -2615,12 +2615,12 @@ static struct PyMethodDef Pickler_methods[] = { {"dump", (PyCFunction) Pickler_dump, METH_VARARGS, PyDoc_STR("dump(object) -- " - "Write an object in pickle format to the object's pickle stream")}, + "Write an object in pickle format to the object's pickle stream")}, {"clear_memo", (PyCFunction) Pickle_clear_memo, METH_NOARGS, PyDoc_STR("clear_memo() -- Clear the picklers memo")}, {"getvalue", (PyCFunction) Pickle_getvalue, METH_VARARGS, PyDoc_STR("getvalue() -- Finish picking a list-based pickle")}, - {NULL, NULL} /* sentinel */ + {NULL, NULL} /* sentinel */ }; @@ -2630,17 +2630,17 @@ Picklerobject *self; if (proto < 0) - proto = HIGHEST_PROTOCOL; + proto = HIGHEST_PROTOCOL; if (proto > HIGHEST_PROTOCOL) { - PyErr_Format(PyExc_ValueError, "pickle protocol %d asked for; " - "the highest available protocol is %d", - proto, HIGHEST_PROTOCOL); - return NULL; + PyErr_Format(PyExc_ValueError, "pickle protocol %d asked for; " + "the highest available protocol is %d", + proto, HIGHEST_PROTOCOL); + return NULL; } self = PyObject_GC_New(Picklerobject, &Picklertype); if (self == NULL) - return NULL; + return NULL; self->proto = proto; self->bin = proto > 0; self->fp = NULL; @@ -2659,49 +2659,49 @@ self->file = NULL; if (file) - Py_INCREF(file); + Py_INCREF(file); else { - file = Pdata_New(); - if (file == NULL) - goto err; + file = Pdata_New(); + if (file == NULL) + goto err; } self->file = file; if (!(self->memo = PyDict_New())) - goto err; + goto err; if (PyFile_Check(file)) { - self->fp = PyFile_AsFile(file); - if (self->fp == NULL) { - PyErr_SetString(PyExc_ValueError, "I/O operation on closed file"); - goto err; - } - self->write_func = write_file; + self->fp = PyFile_AsFile(file); + if (self->fp == NULL) { + PyErr_SetString(PyExc_ValueError, "I/O operation on closed file"); + goto err; + } + self->write_func = write_file; } else if (PycStringIO_OutputCheck(file)) { - self->write_func = write_cStringIO; + self->write_func = write_cStringIO; } else if (file == Py_None) { - self->write_func = write_none; + self->write_func = write_none; } else { - self->write_func = write_other; + self->write_func = write_other; - if (!Pdata_Check(file)) { - self->write = PyObject_GetAttr(file, write_str); - if (!self->write) { - PyErr_Clear(); - PyErr_SetString(PyExc_TypeError, - "argument must have 'write' " "attribute"); - goto err; - } - } - - self->write_buf = (char *) PyMem_Malloc(WRITE_BUF_SIZE); - if (self->write_buf == NULL) { - PyErr_NoMemory(); - goto err; - } + if (!Pdata_Check(file)) { + self->write = PyObject_GetAttr(file, write_str); + if (!self->write) { + PyErr_Clear(); + PyErr_SetString(PyExc_TypeError, + "argument must have 'write' " "attribute"); + goto err; + } + } + + self->write_buf = (char *) PyMem_Malloc(WRITE_BUF_SIZE); + if (self->write_buf == NULL) { + PyErr_NoMemory(); + goto err; + } } self->dispatch_table = dispatch_table; @@ -2731,11 +2731,11 @@ * far enough to figure out what it means. */ if (!PyArg_ParseTuple(args, "|i:Pickler", &proto)) { - PyErr_Clear(); - proto = 0; - if (!PyArg_ParseTupleAndKeywords(args, kwds, "O|i:Pickler", - kwlist, &file, &proto)) - return NULL; + PyErr_Clear(); + proto = 0; + if (!PyArg_ParseTupleAndKeywords(args, kwds, "O|i:Pickler", + kwlist, &file, &proto)) + return NULL; } return (PyObject *) newPicklerobject(file, proto); } @@ -2789,9 +2789,9 @@ Pickler_get_pers_func(Picklerobject * p) { if (p->pers_func == NULL) - PyErr_SetString(PyExc_AttributeError, "persistent_id"); + PyErr_SetString(PyExc_AttributeError, "persistent_id"); else - Py_INCREF(p->pers_func); + Py_INCREF(p->pers_func); return p->pers_func; } @@ -2799,9 +2799,9 @@ Pickler_set_pers_func(Picklerobject * p, PyObject * v) { if (v == NULL) { - PyErr_SetString(PyExc_TypeError, - "attribute deletion is not supported"); - return -1; + PyErr_SetString(PyExc_TypeError, + "attribute deletion is not supported"); + return -1; } Py_XDECREF(p->pers_func); Py_INCREF(v); @@ -2813,9 +2813,9 @@ Pickler_set_inst_pers_func(Picklerobject * p, PyObject * v) { if (v == NULL) { - PyErr_SetString(PyExc_TypeError, - "attribute deletion is not supported"); - return -1; + PyErr_SetString(PyExc_TypeError, + "attribute deletion is not supported"); + return -1; } Py_XDECREF(p->inst_pers_func); Py_INCREF(v); @@ -2827,9 +2827,9 @@ Pickler_get_memo(Picklerobject * p) { if (p->memo == NULL) - PyErr_SetString(PyExc_AttributeError, "memo"); + PyErr_SetString(PyExc_AttributeError, "memo"); else - Py_INCREF(p->memo); + Py_INCREF(p->memo); return p->memo; } @@ -2837,13 +2837,13 @@ Pickler_set_memo(Picklerobject * p, PyObject * v) { if (v == NULL) { - PyErr_SetString(PyExc_TypeError, - "attribute deletion is not supported"); - return -1; + PyErr_SetString(PyExc_TypeError, + "attribute deletion is not supported"); + return -1; } if (!PyDict_Check(v)) { - PyErr_SetString(PyExc_TypeError, "memo must be a dictionary"); - return -1; + PyErr_SetString(PyExc_TypeError, "memo must be a dictionary"); + return -1; } Py_XDECREF(p->memo); Py_INCREF(v); @@ -2878,36 +2878,36 @@ static PyTypeObject Picklertype = { PyObject_HEAD_INIT(NULL) - 0, /*ob_size */ - "cPickle.Pickler", /*tp_name */ - sizeof(Picklerobject), /*tp_basicsize */ + 0, /*ob_size */ + "cPickle.Pickler", /*tp_name */ + sizeof(Picklerobject), /*tp_basicsize */ 0, - (destructor) Pickler_dealloc, /* tp_dealloc */ - 0, /* tp_print */ - 0, /* tp_getattr */ - 0, /* tp_setattr */ - 0, /* tp_compare */ - 0, /* tp_repr */ - 0, /* tp_as_number */ - 0, /* tp_as_sequence */ - 0, /* tp_as_mapping */ - 0, /* tp_hash */ - 0, /* tp_call */ - 0, /* tp_str */ - PyObject_GenericGetAttr, /* tp_getattro */ - PyObject_GenericSetAttr, /* tp_setattro */ - 0, /* tp_as_buffer */ + (destructor) Pickler_dealloc, /* tp_dealloc */ + 0, /* tp_print */ + 0, /* tp_getattr */ + 0, /* tp_setattr */ + 0, /* tp_compare */ + 0, /* tp_repr */ + 0, /* tp_as_number */ + 0, /* tp_as_sequence */ + 0, /* tp_as_mapping */ + 0, /* tp_hash */ + 0, /* tp_call */ + 0, /* tp_str */ + PyObject_GenericGetAttr, /* tp_getattro */ + PyObject_GenericSetAttr, /* tp_setattro */ + 0, /* tp_as_buffer */ Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE | Py_TPFLAGS_HAVE_GC, - Picklertype__doc__, /* tp_doc */ - (traverseproc) Pickler_traverse, /* tp_traverse */ - (inquiry) Pickler_clear, /* tp_clear */ - 0, /* tp_richcompare */ - 0, /* tp_weaklistoffset */ - 0, /* tp_iter */ - 0, /* tp_iternext */ - Pickler_methods, /* tp_methods */ - Pickler_members, /* tp_members */ - Pickler_getsets, /* tp_getset */ + Picklertype__doc__, /* tp_doc */ + (traverseproc) Pickler_traverse, /* tp_traverse */ + (inquiry) Pickler_clear, /* tp_clear */ + 0, /* tp_richcompare */ + 0, /* tp_weaklistoffset */ + 0, /* tp_iter */ + 0, /* tp_iternext */ + Pickler_methods, /* tp_methods */ + Pickler_members, /* tp_members */ + Pickler_getsets, /* tp_getset */ }; static PyObject * @@ -2916,29 +2916,29 @@ PyObject *global = 0, *module; if (fc) { - if (fc == Py_None) { - PyErr_SetString(UnpicklingError, "Global and instance " - "pickles are not supported."); - return NULL; - } - return PyObject_CallFunctionObjArgs(fc, py_module_name, - py_global_name, NULL); + if (fc == Py_None) { + PyErr_SetString(UnpicklingError, "Global and instance " + "pickles are not supported."); + return NULL; + } + return PyObject_CallFunctionObjArgs(fc, py_module_name, + py_global_name, NULL); } module = PySys_GetObject("modules"); if (module == NULL) - return NULL; + return NULL; module = PyDict_GetItem(module, py_module_name); if (module == NULL) { - module = PyImport_Import(py_module_name); - if (!module) - return NULL; - global = PyObject_GetAttr(module, py_global_name); - Py_DECREF(module); + module = PyImport_Import(py_module_name); + if (!module) + return NULL; + global = PyObject_GetAttr(module, py_global_name); + Py_DECREF(module); } else - global = PyObject_GetAttr(module, py_global_name); + global = PyObject_GetAttr(module, py_global_name); return global; } @@ -2946,8 +2946,8 @@ marker(Unpicklerobject * self) { if (self->num_marks < 1) { - PyErr_SetString(UnpicklingError, "could not find MARK"); - return -1; + PyErr_SetString(UnpicklingError, "could not find MARK"); + return -1; } return self->marks[--self->num_marks]; @@ -2977,35 +2977,35 @@ long l; if ((len = self->readline_func(self, &s)) < 0) - return -1; + return -1; if (len < 2) - return bad_readline(); + return bad_readline(); if (!(s = pystrndup(s, len))) - return -1; + return -1; errno = 0; l = strtol(s, &endptr, 0); if (errno || (*endptr != '\n') || (endptr[1] != '\0')) { - /* Hm, maybe we've got something long. Let's try reading - * it as a Python long object. */ - errno = 0; - py_int = PyLong_FromString(s, NULL, 0); - if (py_int == NULL) { - PyErr_SetString(PyExc_ValueError, - "could not convert string to int"); - goto finally; - } + /* Hm, maybe we've got something long. Let's try reading + * it as a Python long object. */ + errno = 0; + py_int = PyLong_FromString(s, NULL, 0); + if (py_int == NULL) { + PyErr_SetString(PyExc_ValueError, + "could not convert string to int"); + goto finally; + } } else { - if (len == 3 && (l == 0 || l == 1)) { - if (!(py_int = PyBool_FromLong(l))) - goto finally; - } - else { - if (!(py_int = PyInt_FromLong(l))) - goto finally; - } + if (len == 3 && (l == 0 || l == 1)) { + if (!(py_int = PyBool_FromLong(l))) + goto finally; + } + else { + if (!(py_int = PyInt_FromLong(l))) + goto finally; + } } free(s); @@ -3039,8 +3039,8 @@ long l; for (i = 0, l = 0L; i < x; i++) { - c = (unsigned char) s[i]; - l |= (long) c << (i * 8); + c = (unsigned char) s[i]; + l |= (long) c << (i * 8); } #if SIZEOF_LONG > 4 /* Unlike BININT1 and BININT2, BININT (more accurately BININT4) @@ -3048,7 +3048,7 @@ * to extend a BININT's sign bit to the full width. */ if (x == 4 && l & (1L << 31)) - l |= (~0L) << 32; + l |= (~0L) << 32; #endif return l; } @@ -3063,7 +3063,7 @@ l = calc_binint(s, x); if (!(py_int = PyInt_FromLong(l))) - return -1; + return -1; PDATA_PUSH(self->stack, py_int, -1); return 0; @@ -3076,7 +3076,7 @@ char *s; if (self->read_func(self, &s, 4) < 0) - return -1; + return -1; return load_binintx(self, s, 4); } @@ -3088,7 +3088,7 @@ char *s; if (self->read_func(self, &s, 1) < 0) - return -1; + return -1; return load_binintx(self, s, 1); } @@ -3100,7 +3100,7 @@ char *s; if (self->read_func(self, &s, 2) < 0) - return -1; + return -1; return load_binintx(self, s, 2); } @@ -3113,14 +3113,14 @@ int len, res = -1; if ((len = self->readline_func(self, &s)) < 0) - return -1; + return -1; if (len < 2) - return bad_readline(); + return bad_readline(); if (!(s = pystrndup(s, len))) - return -1; + return -1; if (!(l = PyLong_FromString(s, &end, 0))) - goto finally; + goto finally; free(s); PDATA_PUSH(self->stack, l, -1); @@ -3146,30 +3146,30 @@ assert(size == 1 || size == 4); i = self->read_func(self, &nbytes, size); if (i < 0) - return -1; + return -1; size = calc_binint(nbytes, size); if (size < 0) { - /* Corrupt or hostile pickle -- we never write one like - * this. - */ - PyErr_SetString(UnpicklingError, "LONG pickle has negative " - "byte count"); - return -1; + /* Corrupt or hostile pickle -- we never write one like + * this. + */ + PyErr_SetString(UnpicklingError, "LONG pickle has negative " + "byte count"); + return -1; } if (size == 0) - along = PyLong_FromLong(0L); + along = PyLong_FromLong(0L); else { - /* Read the raw little-endian bytes & convert. */ - i = self->read_func(self, (char **) &pdata, size); - if (i < 0) - return -1; - along = _PyLong_FromByteArray(pdata, (size_t) size, - 1 /* little endian */ , 1 /* signed */ ); + /* Read the raw little-endian bytes & convert. */ + i = self->read_func(self, (char **) &pdata, size); + if (i < 0) + return -1; + along = _PyLong_FromByteArray(pdata, (size_t) size, + 1 /* little endian */ , 1 /* signed */ ); } if (along == NULL) - return -1; + return -1; PDATA_PUSH(self->stack, along, -1); return 0; } @@ -3183,22 +3183,22 @@ double d; if ((len = self->readline_func(self, &s)) < 0) - return -1; + return -1; if (len < 2) - return bad_readline(); + return bad_readline(); if (!(s = pystrndup(s, len))) - return -1; + return -1; errno = 0; d = PyOS_ascii_strtod(s, &endptr); if (errno || (endptr[0] != '\n') || (endptr[1] != '\0')) { - PyErr_SetString(PyExc_ValueError, "could not convert string to float"); - goto finally; + PyErr_SetString(PyExc_ValueError, "could not convert string to float"); + goto finally; } if (!(py_float = PyFloat_FromDouble(d))) - goto finally; + goto finally; free(s); PDATA_PUSH(self->stack, py_float, -1); @@ -3218,15 +3218,15 @@ char *p; if (self->read_func(self, &p, 8) < 0) - return -1; + return -1; x = _PyFloat_Unpack8((unsigned char *) p, 0); if (x == -1.0 && PyErr_Occurred()) - return -1; + return -1; py_float = PyFloat_FromDouble(x); if (py_float == NULL) - return -1; + return -1; PDATA_PUSH(self->stack, py_float, -1); return 0; @@ -3240,35 +3240,35 @@ char *s, *p; if ((len = self->readline_func(self, &s)) < 0) - return -1; + return -1; if (len < 2) - return bad_readline(); + return bad_readline(); if (!(s = pystrndup(s, len))) - return -1; + return -1; /* Strip outermost quotes */ while (s[len - 1] <= ' ') - len--; + len--; if (s[0] == '"' && s[len - 1] == '"') { - s[len - 1] = '\0'; - p = s + 1; - len -= 2; + s[len - 1] = '\0'; + p = s + 1; + len -= 2; } else if (s[0] == '\'' && s[len - 1] == '\'') { - s[len - 1] = '\0'; - p = s + 1; - len -= 2; + s[len - 1] = '\0'; + p = s + 1; + len -= 2; } else - goto insecure; - /********************************************/ + goto insecure; + /********************************************/ str = PyString_DecodeEscape(p, len, NULL, 0, NULL); free(s); if (str) { - PDATA_PUSH(self->stack, str, -1); - res = 0; + PDATA_PUSH(self->stack, str, -1); + res = 0; } return res; @@ -3287,15 +3287,15 @@ char *s; if (self->read_func(self, &s, 4) < 0) - return -1; + return -1; l = calc_binint(s, 4); if (self->read_func(self, &s, l) < 0) - return -1; + return -1; if (!(py_string = PyString_FromStringAndSize(s, l))) - return -1; + return -1; PDATA_PUSH(self->stack, py_string, -1); return 0; @@ -3310,15 +3310,15 @@ char *s; if (self->read_func(self, &s, 1) < 0) - return -1; + return -1; l = (unsigned char) s[0]; if (self->read_func(self, &s, l) < 0) - return -1; + return -1; if (!(py_string = PyString_FromStringAndSize(s, l))) - return -1; + return -1; PDATA_PUSH(self->stack, py_string, -1); return 0; @@ -3334,12 +3334,12 @@ char *s; if ((len = self->readline_func(self, &s)) < 0) - return -1; + return -1; if (len < 1) - return bad_readline(); + return bad_readline(); if (!(str = PyUnicode_DecodeRawUnicodeEscape(s, len - 1, NULL))) - goto finally; + goto finally; PDATA_PUSH(self->stack, str, -1); return 0; @@ -3359,15 +3359,15 @@ char *s; if (self->read_func(self, &s, 4) < 0) - return -1; + return -1; l = calc_binint(s, 4); if (self->read_func(self, &s, l) < 0) - return -1; + return -1; if (!(unicode = PyUnicode_DecodeUTF8(s, l, NULL))) - return -1; + return -1; PDATA_PUSH(self->stack, unicode, -1); return 0; @@ -3382,9 +3382,9 @@ int i; if ((i = marker(self)) < 0) - return -1; + return -1; if (!(tup = Pdata_popTuple(self->stack, i))) - return -1; + return -1; PDATA_PUSH(self->stack, tup, -1); return 0; } @@ -3395,15 +3395,15 @@ PyObject *tup = PyTuple_New(len); if (tup == NULL) - return -1; + return -1; while (--len >= 0) { - PyObject *element; + PyObject *element; - PDATA_POP(self->stack, element); - if (element == NULL) - return -1; - PyTuple_SET_ITEM(tup, len, element); + PDATA_POP(self->stack, element); + if (element == NULL) + return -1; + PyTuple_SET_ITEM(tup, len, element); } PDATA_PUSH(self->stack, tup, -1); return 0; @@ -3415,7 +3415,7 @@ PyObject *list; if (!(list = PyList_New(0))) - return -1; + return -1; PDATA_PUSH(self->stack, list, -1); return 0; } @@ -3426,7 +3426,7 @@ PyObject *dict; if (!(dict = PyDict_New())) - return -1; + return -1; PDATA_PUSH(self->stack, dict, -1); return 0; } @@ -3439,9 +3439,9 @@ int i; if ((i = marker(self)) < 0) - return -1; + return -1; if (!(list = Pdata_popList(self->stack, i))) - return -1; + return -1; PDATA_PUSH(self->stack, list, -1); return 0; } @@ -3453,19 +3453,19 @@ int i, j, k; if ((i = marker(self)) < 0) - return -1; + return -1; j = self->stack->length; if (!(dict = PyDict_New())) - return -1; + return -1; for (k = i + 1; k < j; k += 2) { - key = self->stack->data[k - 1]; - value = self->stack->data[k]; - if (PyDict_SetItem(dict, key, value) < 0) { - Py_DECREF(dict); - return -1; - } + key = self->stack->data[k - 1]; + value = self->stack->data[k]; + if (PyDict_SetItem(dict, key, value) < 0) { + Py_DECREF(dict); + return -1; + } } Pdata_clear(self->stack, i); PDATA_PUSH(self->stack, dict, -1); @@ -3478,21 +3478,21 @@ PyObject *r = 0; if ((r = PyObject_CallObject(cls, args))) - return r; + return r; { - PyObject *tp, *v, *tb, *tmp_value; + PyObject *tp, *v, *tb, *tmp_value; - PyErr_Fetch(&tp, &v, &tb); - tmp_value = v; - /* NULL occurs when there was a KeyboardInterrupt */ - if (tmp_value == NULL) - tmp_value = Py_None; - if ((r = PyTuple_Pack(3, tmp_value, cls, args))) { - Py_XDECREF(v); - v = r; - } - PyErr_Restore(tp, v, tb); + PyErr_Fetch(&tp, &v, &tb); + tmp_value = v; + /* NULL occurs when there was a KeyboardInterrupt */ + if (tmp_value == NULL) + tmp_value = Py_None; + if ((r = PyTuple_Pack(3, tmp_value, cls, args))) { + Py_XDECREF(v); + v = r; + } + PyErr_Restore(tp, v, tb); } return NULL; } @@ -3505,18 +3505,18 @@ int i; if ((i = marker(self)) < 0) - return -1; + return -1; if (!(tup = Pdata_popTuple(self->stack, i + 1))) - return -1; + return -1; PDATA_POP(self->stack, class); if (class) { - obj = Instance_New(class, tup); - Py_DECREF(class); + obj = Instance_New(class, tup); + Py_DECREF(class); } Py_DECREF(tup); if (!obj) - return -1; + return -1; PDATA_PUSH(self->stack, obj, -1); return 0; } @@ -3530,37 +3530,37 @@ char *s; if ((i = marker(self)) < 0) - return -1; + return -1; if ((len = self->readline_func(self, &s)) < 0) - return -1; + return -1; if (len < 2) - return bad_readline(); + return bad_readline(); module_name = PyString_FromStringAndSize(s, len - 1); if (!module_name) - return -1; + return -1; if ((len = self->readline_func(self, &s)) >= 0) { - if (len < 2) - return bad_readline(); - if ((class_name = PyString_FromStringAndSize(s, len - 1))) { - class = find_class(module_name, class_name, self->find_class); - Py_DECREF(class_name); - } + if (len < 2) + return bad_readline(); + if ((class_name = PyString_FromStringAndSize(s, len - 1))) { + class = find_class(module_name, class_name, self->find_class); + Py_DECREF(class_name); + } } Py_DECREF(module_name); if (!class) - return -1; + return -1; if ((tup = Pdata_popTuple(self->stack, i))) { - obj = Instance_New(class, tup); - Py_DECREF(tup); + obj = Instance_New(class, tup); + Py_DECREF(tup); } Py_DECREF(class); if (!obj) - return -1; + return -1; PDATA_PUSH(self->stack, obj, -1); return 0; @@ -3571,7 +3571,7 @@ { PyObject *args = NULL; PyObject *clsraw = NULL; - PyTypeObject *cls; /* clsraw cast to its true type */ + PyTypeObject *cls; /* clsraw cast to its true type */ PyObject *obj; /* Stack is ... cls argtuple, and we want to call @@ -3579,31 +3579,31 @@ */ PDATA_POP(self->stack, args); if (args == NULL) - goto Fail; + goto Fail; if (!PyTuple_Check(args)) { - PyErr_SetString(UnpicklingError, "NEWOBJ expected an arg " "tuple."); - goto Fail; + PyErr_SetString(UnpicklingError, "NEWOBJ expected an arg " "tuple."); + goto Fail; } PDATA_POP(self->stack, clsraw); cls = (PyTypeObject *) clsraw; if (cls == NULL) - goto Fail; + goto Fail; if (!PyType_Check(cls)) { - PyErr_SetString(UnpicklingError, "NEWOBJ class argument " - "isn't a type object"); - goto Fail; + PyErr_SetString(UnpicklingError, "NEWOBJ class argument " + "isn't a type object"); + goto Fail; } if (cls->tp_new == NULL) { - PyErr_SetString(UnpicklingError, "NEWOBJ class argument " - "has NULL tp_new"); - goto Fail; + PyErr_SetString(UnpicklingError, "NEWOBJ class argument " + "has NULL tp_new"); + goto Fail; } /* Call __new__. */ obj = cls->tp_new(cls, args, NULL); if (obj == NULL) - goto Fail; + goto Fail; Py_DECREF(args); Py_DECREF(clsraw); @@ -3624,27 +3624,27 @@ char *s; if ((len = self->readline_func(self, &s)) < 0) - return -1; + return -1; if (len < 2) - return bad_readline(); + return bad_readline(); module_name = PyString_FromStringAndSize(s, len - 1); if (!module_name) - return -1; + return -1; if ((len = self->readline_func(self, &s)) >= 0) { - if (len < 2) { - Py_DECREF(module_name); - return bad_readline(); - } - if ((class_name = PyString_FromStringAndSize(s, len - 1))) { - class = find_class(module_name, class_name, self->find_class); - Py_DECREF(class_name); - } + if (len < 2) { + Py_DECREF(module_name); + return bad_readline(); + } + if ((class_name = PyString_FromStringAndSize(s, len - 1))) { + class = find_class(module_name, class_name, self->find_class); + Py_DECREF(class_name); + } } Py_DECREF(module_name); if (!class) - return -1; + return -1; PDATA_PUSH(self->stack, class, -1); return 0; } @@ -3658,40 +3658,40 @@ char *s; if (self->pers_func) { - if ((len = self->readline_func(self, &s)) < 0) - return -1; - if (len < 2) - return bad_readline(); - - pid = PyString_FromStringAndSize(s, len - 1); - if (!pid) - return -1; - - if (PyList_Check(self->pers_func)) { - if (PyList_Append(self->pers_func, pid) < 0) { - Py_DECREF(pid); - return -1; - } - } - else { - ARG_TUP(self, pid); - if (self->arg) { - pid = PyObject_Call(self->pers_func, self->arg, NULL); - FREE_ARG_TUP(self); - } - } + if ((len = self->readline_func(self, &s)) < 0) + return -1; + if (len < 2) + return bad_readline(); + + pid = PyString_FromStringAndSize(s, len - 1); + if (!pid) + return -1; + + if (PyList_Check(self->pers_func)) { + if (PyList_Append(self->pers_func, pid) < 0) { + Py_DECREF(pid); + return -1; + } + } + else { + ARG_TUP(self, pid); + if (self->arg) { + pid = PyObject_Call(self->pers_func, self->arg, NULL); + FREE_ARG_TUP(self); + } + } - if (!pid) - return -1; + if (!pid) + return -1; - PDATA_PUSH(self->stack, pid, -1); - return 0; + PDATA_PUSH(self->stack, pid, -1); + return 0; } else { - PyErr_SetString(UnpicklingError, - "A load persistent id instruction was encountered,\n" - "but no persistent_load function was specified."); - return -1; + PyErr_SetString(UnpicklingError, + "A load persistent id instruction was encountered,\n" + "but no persistent_load function was specified."); + return -1; } } @@ -3701,34 +3701,34 @@ PyObject *pid = 0; if (self->pers_func) { - PDATA_POP(self->stack, pid); - if (!pid) - return -1; - - if (PyList_Check(self->pers_func)) { - if (PyList_Append(self->pers_func, pid) < 0) { - Py_DECREF(pid); - return -1; - } - } - else { - ARG_TUP(self, pid); - if (self->arg) { - pid = PyObject_Call(self->pers_func, self->arg, NULL); - FREE_ARG_TUP(self); - } - if (!pid) - return -1; - } + PDATA_POP(self->stack, pid); + if (!pid) + return -1; + + if (PyList_Check(self->pers_func)) { + if (PyList_Append(self->pers_func, pid) < 0) { + Py_DECREF(pid); + return -1; + } + } + else { + ARG_TUP(self, pid); + if (self->arg) { + pid = PyObject_Call(self->pers_func, self->arg, NULL); + FREE_ARG_TUP(self); + } + if (!pid) + return -1; + } - PDATA_PUSH(self->stack, pid, -1); - return 0; + PDATA_PUSH(self->stack, pid, -1); + return 0; } else { - PyErr_SetString(UnpicklingError, - "A load persistent id instruction was encountered,\n" - "but no persistent_load function was specified."); - return -1; + PyErr_SetString(UnpicklingError, + "A load persistent id instruction was encountered,\n" + "but no persistent_load function was specified."); + return -1; } } @@ -3739,7 +3739,7 @@ int len; if (!((len = self->stack->length) > 0)) - return stackUnderflow(); + return stackUnderflow(); /* Note that we split the (pickle.py) stack into two stacks, * an object stack and a mark stack. We have to be clever and @@ -3748,11 +3748,11 @@ */ if ((self->num_marks > 0) && (self->marks[self->num_marks - 1] == len)) - self->num_marks--; + self->num_marks--; else { - len--; - Py_DECREF(self->stack->data[len]); - self->stack->length = len; + len--; + Py_DECREF(self->stack->data[len]); + self->stack->length = len; } return 0; @@ -3765,7 +3765,7 @@ int i; if ((i = marker(self)) < 0) - return -1; + return -1; Pdata_clear(self->stack, i); @@ -3780,7 +3780,7 @@ int len; if ((len = self->stack->length) <= 0) - return stackUnderflow(); + return stackUnderflow(); last = self->stack->data[len - 1]; Py_INCREF(last); PDATA_PUSH(self->stack, last, -1); @@ -3797,21 +3797,21 @@ int rc; if ((len = self->readline_func(self, &s)) < 0) - return -1; + return -1; if (len < 2) - return bad_readline(); + return bad_readline(); if (!(py_str = PyString_FromStringAndSize(s, len - 1))) - return -1; + return -1; value = PyDict_GetItem(self->memo, py_str); if (!value) { - PyErr_SetObject(BadPickleGet, py_str); - rc = -1; + PyErr_SetObject(BadPickleGet, py_str); + rc = -1; } else { - PDATA_APPEND(self->stack, value, -1); - rc = 0; + PDATA_APPEND(self->stack, value, -1); + rc = 0; } Py_DECREF(py_str); @@ -3828,20 +3828,20 @@ int rc; if (self->read_func(self, &s, 1) < 0) - return -1; + return -1; key = (unsigned char) s[0]; if (!(py_key = PyInt_FromLong((long) key))) - return -1; + return -1; value = PyDict_GetItem(self->memo, py_key); if (!value) { - PyErr_SetObject(BadPickleGet, py_key); - rc = -1; + PyErr_SetObject(BadPickleGet, py_key); + rc = -1; } else { - PDATA_APPEND(self->stack, value, -1); - rc = 0; + PDATA_APPEND(self->stack, value, -1); + rc = 0; } Py_DECREF(py_key); @@ -3859,7 +3859,7 @@ int rc; if (self->read_func(self, &s, 4) < 0) - return -1; + return -1; c = (unsigned char) s[0]; key = (long) c; @@ -3871,16 +3871,16 @@ key |= (long) c << 24; if (!(py_key = PyInt_FromLong((long) key))) - return -1; + return -1; value = PyDict_GetItem(self->memo, py_key); if (!value) { - PyErr_SetObject(BadPickleGet, py_key); - rc = -1; + PyErr_SetObject(BadPickleGet, py_key); + rc = -1; } else { - PDATA_APPEND(self->stack, value, -1); - rc = 0; + PDATA_APPEND(self->stack, value, -1); + rc = 0; } Py_DECREF(py_key); @@ -3893,66 +3893,66 @@ static int load_extension(Unpicklerobject * self, int nbytes) { - char *codebytes; /* the nbytes bytes after the opcode */ - long code; /* calc_binint returns long */ - PyObject *py_code; /* code as a Python int */ - PyObject *obj; /* the object to push */ - PyObject *pair; /* (module_name, class_name) */ + char *codebytes; /* the nbytes bytes after the opcode */ + long code; /* calc_binint returns long */ + PyObject *py_code; /* code as a Python int */ + PyObject *obj; /* the object to push */ + PyObject *pair; /* (module_name, class_name) */ PyObject *module_name, *class_name; assert(nbytes == 1 || nbytes == 2 || nbytes == 4); if (self->read_func(self, &codebytes, nbytes) < 0) - return -1; + return -1; code = calc_binint(codebytes, nbytes); - if (code <= 0) { /* note that 0 is forbidden */ - /* Corrupt or hostile pickle. */ - PyErr_SetString(UnpicklingError, "EXT specifies code <= 0"); - return -1; + if (code <= 0) { /* note that 0 is forbidden */ + /* Corrupt or hostile pickle. */ + PyErr_SetString(UnpicklingError, "EXT specifies code <= 0"); + return -1; } /* Look for the code in the cache. */ py_code = PyInt_FromLong(code); if (py_code == NULL) - return -1; + return -1; obj = PyDict_GetItem(extension_cache, py_code); if (obj != NULL) { - /* Bingo. */ - Py_DECREF(py_code); - PDATA_APPEND(self->stack, obj, -1); - return 0; + /* Bingo. */ + Py_DECREF(py_code); + PDATA_APPEND(self->stack, obj, -1); + return 0; } /* Look up the (module_name, class_name) pair. */ pair = PyDict_GetItem(inverted_registry, py_code); if (pair == NULL) { - Py_DECREF(py_code); - PyErr_Format(PyExc_ValueError, "unregistered extension " - "code %ld", code); - return -1; + Py_DECREF(py_code); + PyErr_Format(PyExc_ValueError, "unregistered extension " + "code %ld", code); + return -1; } /* Since the extension registry is manipulable via Python code, * confirm that pair is really a 2-tuple of strings. */ if (!PyTuple_Check(pair) || PyTuple_Size(pair) != 2 || - !PyString_Check(module_name = PyTuple_GET_ITEM(pair, 0)) || - !PyString_Check(class_name = PyTuple_GET_ITEM(pair, 1))) { - Py_DECREF(py_code); - PyErr_Format(PyExc_ValueError, "_inverted_registry[%ld] " - "isn't a 2-tuple of strings", code); - return -1; + !PyString_Check(module_name = PyTuple_GET_ITEM(pair, 0)) || + !PyString_Check(class_name = PyTuple_GET_ITEM(pair, 1))) { + Py_DECREF(py_code); + PyErr_Format(PyExc_ValueError, "_inverted_registry[%ld] " + "isn't a 2-tuple of strings", code); + return -1; } /* Load the object. */ obj = find_class(module_name, class_name, self->find_class); if (obj == NULL) { - Py_DECREF(py_code); - return -1; + Py_DECREF(py_code); + return -1; } /* Cache code -> obj. */ code = PyDict_SetItem(extension_cache, py_code, obj); Py_DECREF(py_code); if (code < 0) { - Py_DECREF(obj); - return -1; + Py_DECREF(obj); + return -1; } PDATA_PUSH(self->stack, obj, -1); return 0; @@ -3966,13 +3966,13 @@ char *s; if ((l = self->readline_func(self, &s)) < 0) - return -1; + return -1; if (l < 2) - return bad_readline(); + return bad_readline(); if (!(len = self->stack->length)) - return stackUnderflow(); + return stackUnderflow(); if (!(py_str = PyString_FromStringAndSize(s, l - 1))) - return -1; + return -1; value = self->stack->data[len - 1]; l = PyDict_SetItem(self->memo, py_str, value); Py_DECREF(py_str); @@ -3989,14 +3989,14 @@ int len; if (self->read_func(self, &s, 1) < 0) - return -1; + return -1; if (!((len = self->stack->length) > 0)) - return stackUnderflow(); + return stackUnderflow(); key = (unsigned char) s[0]; if (!(py_key = PyInt_FromLong((long) key))) - return -1; + return -1; value = self->stack->data[len - 1]; len = PyDict_SetItem(self->memo, py_key, value); Py_DECREF(py_key); @@ -4014,9 +4014,9 @@ int len; if (self->read_func(self, &s, 4) < 0) - return -1; + return -1; if (!(len = self->stack->length)) - return stackUnderflow(); + return stackUnderflow(); c = (unsigned char) s[0]; key = (long) c; @@ -4028,7 +4028,7 @@ key |= (long) c << 24; if (!(py_key = PyInt_FromLong(key))) - return -1; + return -1; value = self->stack->data[len - 1]; len = PyDict_SetItem(self->memo, py_key, value); Py_DECREF(py_key); @@ -4044,50 +4044,50 @@ len = self->stack->length; if (!(len >= x && x > 0)) - return stackUnderflow(); + return stackUnderflow(); /* nothing to do */ if (len == x) - return 0; + return 0; list = self->stack->data[x - 1]; if (PyList_Check(list)) { - PyObject *slice; - int list_len; + PyObject *slice; + int list_len; - slice = Pdata_popList(self->stack, x); - if (!slice) - return -1; - list_len = PyList_GET_SIZE(list); - i = PyList_SetSlice(list, list_len, list_len, slice); - Py_DECREF(slice); - return i; + slice = Pdata_popList(self->stack, x); + if (!slice) + return -1; + list_len = PyList_GET_SIZE(list); + i = PyList_SetSlice(list, list_len, list_len, slice); + Py_DECREF(slice); + return i; } else { - if (!(append_method = PyObject_GetAttr(list, append_str))) - return -1; + if (!(append_method = PyObject_GetAttr(list, append_str))) + return -1; - for (i = x; i < len; i++) { - PyObject *junk; + for (i = x; i < len; i++) { + PyObject *junk; - value = self->stack->data[i]; - junk = 0; - ARG_TUP(self, value); - if (self->arg) { - junk = PyObject_Call(append_method, self->arg, NULL); - FREE_ARG_TUP(self); - } - if (!junk) { - Pdata_clear(self->stack, i + 1); - self->stack->length = x; - Py_DECREF(append_method); - return -1; - } - Py_DECREF(junk); - } - self->stack->length = x; - Py_DECREF(append_method); + value = self->stack->data[i]; + junk = 0; + ARG_TUP(self, value); + if (self->arg) { + junk = PyObject_Call(append_method, self->arg, NULL); + FREE_ARG_TUP(self); + } + if (!junk) { + Pdata_clear(self->stack, i + 1); + self->stack->length = x; + Py_DECREF(append_method); + return -1; + } + Py_DECREF(junk); + } + self->stack->length = x; + Py_DECREF(append_method); } return 0; @@ -4115,17 +4115,17 @@ int len, i, r = 0; if (!((len = self->stack->length) >= x && x > 0)) - return stackUnderflow(); + return stackUnderflow(); dict = self->stack->data[x - 1]; for (i = x + 1; i < len; i += 2) { - key = self->stack->data[i - 1]; - value = self->stack->data[i]; - if (PyObject_SetItem(dict, key, value) < 0) { - r = -1; - break; - } + key = self->stack->data[i - 1]; + value = self->stack->data[i]; + if (PyObject_SetItem(dict, key, value) < 0) { + r = -1; + break; + } } Pdata_clear(self->stack, x); @@ -4160,77 +4160,77 @@ * the stack top, possibly mutated via instance.__setstate__(state). */ if (self->stack->length < 2) - return stackUnderflow(); + return stackUnderflow(); PDATA_POP(self->stack, state); if (state == NULL) - return -1; + return -1; inst = self->stack->data[self->stack->length - 1]; __setstate__ = PyObject_GetAttr(inst, __setstate___str); if (__setstate__ != NULL) { - PyObject *junk = NULL; + PyObject *junk = NULL; - /* The explicit __setstate__ is responsible for everything. */ - ARG_TUP(self, state); - if (self->arg) { - junk = PyObject_Call(__setstate__, self->arg, NULL); - FREE_ARG_TUP(self); - } - Py_DECREF(__setstate__); - if (junk == NULL) - return -1; - Py_DECREF(junk); - return 0; + /* The explicit __setstate__ is responsible for everything. */ + ARG_TUP(self, state); + if (self->arg) { + junk = PyObject_Call(__setstate__, self->arg, NULL); + FREE_ARG_TUP(self); + } + Py_DECREF(__setstate__); + if (junk == NULL) + return -1; + Py_DECREF(junk); + return 0; } if (!PyErr_ExceptionMatches(PyExc_AttributeError)) - return -1; + return -1; PyErr_Clear(); /* A default __setstate__. First see whether state embeds a * slot state dict too (a proto 2 addition). */ if (PyTuple_Check(state) && PyTuple_Size(state) == 2) { - PyObject *temp = state; - state = PyTuple_GET_ITEM(temp, 0); - slotstate = PyTuple_GET_ITEM(temp, 1); - Py_INCREF(state); - Py_INCREF(slotstate); - Py_DECREF(temp); + PyObject *temp = state; + state = PyTuple_GET_ITEM(temp, 0); + slotstate = PyTuple_GET_ITEM(temp, 1); + Py_INCREF(state); + Py_INCREF(slotstate); + Py_DECREF(temp); } else - slotstate = NULL; + slotstate = NULL; /* Set inst.__dict__ from the state dict (if any). */ if (state != Py_None) { - PyObject *dict; - if (!PyDict_Check(state)) { - PyErr_SetString(UnpicklingError, "state is not a " "dictionary"); - goto finally; - } - dict = PyObject_GetAttr(inst, __dict___str); - if (dict == NULL) - goto finally; - - i = 0; - while (PyDict_Next(state, &i, &d_key, &d_value)) { - if (PyObject_SetItem(dict, d_key, d_value) < 0) - goto finally; - } - Py_DECREF(dict); + PyObject *dict; + if (!PyDict_Check(state)) { + PyErr_SetString(UnpicklingError, "state is not a " "dictionary"); + goto finally; + } + dict = PyObject_GetAttr(inst, __dict___str); + if (dict == NULL) + goto finally; + + i = 0; + while (PyDict_Next(state, &i, &d_key, &d_value)) { + if (PyObject_SetItem(dict, d_key, d_value) < 0) + goto finally; + } + Py_DECREF(dict); } /* Also set instance attributes from the slotstate dict (if any). */ if (slotstate != NULL) { - if (!PyDict_Check(slotstate)) { - PyErr_SetString(UnpicklingError, "slot state is not " - "a dictionary"); - goto finally; - } - i = 0; - while (PyDict_Next(slotstate, &i, &d_key, &d_value)) { - if (PyObject_SetAttr(inst, d_key, d_value) < 0) - goto finally; - } + if (!PyDict_Check(slotstate)) { + PyErr_SetString(UnpicklingError, "slot state is not " + "a dictionary"); + goto finally; + } + i = 0; + while (PyDict_Next(slotstate, &i, &d_key, &d_value)) { + if (PyObject_SetAttr(inst, d_key, d_value) < 0) + goto finally; + } } res = 0; @@ -4252,20 +4252,20 @@ */ if ((self->num_marks + 1) >= self->marks_size) { - int *marks; - s = self->marks_size + 20; - if (s <= self->num_marks) - s = self->num_marks + 1; - if (self->marks == NULL) - marks = (int *) malloc(s * sizeof(int)); - else - marks = (int *) realloc(self->marks, s * sizeof(int)); - if (!marks) { - PyErr_NoMemory(); - return -1; - } - self->marks = marks; - self->marks_size = s; + int *marks; + s = self->marks_size + 20; + if (s <= self->num_marks) + s = self->num_marks + 1; + if (self->marks == NULL) + marks = (int *) malloc(s * sizeof(int)); + else + marks = (int *) realloc(self->marks, s * sizeof(int)); + if (!marks) { + PyErr_NoMemory(); + return -1; + } + self->marks = marks; + self->marks_size = s; } self->marks[self->num_marks++] = self->stack->length; @@ -4280,16 +4280,16 @@ PDATA_POP(self->stack, arg_tup); if (!arg_tup) - return -1; + return -1; PDATA_POP(self->stack, callable); if (callable) { - ob = Instance_New(callable, arg_tup); - Py_DECREF(callable); + ob = Instance_New(callable, arg_tup); + Py_DECREF(callable); } Py_DECREF(arg_tup); if (!ob) - return -1; + return -1; PDATA_PUSH(self->stack, ob, -1); return 0; @@ -4306,7 +4306,7 @@ i = self->read_func(self, &protobyte, 1); if (i < 0) - return -1; + return -1; i = calc_binint(protobyte, 1); /* No point checking for < 0, since calc_binint returns an unsigned @@ -4314,7 +4314,7 @@ */ assert(i >= 0); if (i <= HIGHEST_PROTOCOL) - return 0; + return 0; PyErr_Format(PyExc_ValueError, "unsupported pickle protocol: %d", i); return -1; @@ -4328,296 +4328,296 @@ self->num_marks = 0; if (self->stack->length) - Pdata_clear(self->stack, 0); + Pdata_clear(self->stack, 0); while (1) { - if (self->read_func(self, &s, 1) < 0) - break; + if (self->read_func(self, &s, 1) < 0) + break; - switch (s[0]) { - case NONE: - if (load_none(self) < 0) - break; - continue; - - case BININT: - if (load_binint(self) < 0) - break; - continue; - - case BININT1: - if (load_binint1(self) < 0) - break; - continue; - - case BININT2: - if (load_binint2(self) < 0) - break; - continue; - - case INT: - if (load_int(self) < 0) - break; - continue; - - case LONG: - if (load_long(self) < 0) - break; - continue; - - case LONG1: - if (load_counted_long(self, 1) < 0) - break; - continue; - - case LONG4: - if (load_counted_long(self, 4) < 0) - break; - continue; - - case FLOAT: - if (load_float(self) < 0) - break; - continue; - - case BINFLOAT: - if (load_binfloat(self) < 0) - break; - continue; - - case BINSTRING: - if (load_binstring(self) < 0) - break; - continue; - - case SHORT_BINSTRING: - if (load_short_binstring(self) < 0) - break; - continue; - - case STRING: - if (load_string(self) < 0) - break; - continue; + switch (s[0]) { + case NONE: + if (load_none(self) < 0) + break; + continue; + + case BININT: + if (load_binint(self) < 0) + break; + continue; + + case BININT1: + if (load_binint1(self) < 0) + break; + continue; + + case BININT2: + if (load_binint2(self) < 0) + break; + continue; + + case INT: + if (load_int(self) < 0) + break; + continue; + + case LONG: + if (load_long(self) < 0) + break; + continue; + + case LONG1: + if (load_counted_long(self, 1) < 0) + break; + continue; + + case LONG4: + if (load_counted_long(self, 4) < 0) + break; + continue; + + case FLOAT: + if (load_float(self) < 0) + break; + continue; + + case BINFLOAT: + if (load_binfloat(self) < 0) + break; + continue; + + case BINSTRING: + if (load_binstring(self) < 0) + break; + continue; + + case SHORT_BINSTRING: + if (load_short_binstring(self) < 0) + break; + continue; + + case STRING: + if (load_string(self) < 0) + break; + continue; #ifdef Py_USING_UNICODE - case UNICODE: - if (load_unicode(self) < 0) - break; - continue; - - case BINUNICODE: - if (load_binunicode(self) < 0) - break; - continue; + case UNICODE: + if (load_unicode(self) < 0) + break; + continue; + + case BINUNICODE: + if (load_binunicode(self) < 0) + break; + continue; #endif - case EMPTY_TUPLE: - if (load_counted_tuple(self, 0) < 0) - break; - continue; - - case TUPLE1: - if (load_counted_tuple(self, 1) < 0) - break; - continue; - - case TUPLE2: - if (load_counted_tuple(self, 2) < 0) - break; - continue; - - case TUPLE3: - if (load_counted_tuple(self, 3) < 0) - break; - continue; - - case TUPLE: - if (load_tuple(self) < 0) - break; - continue; - - case EMPTY_LIST: - if (load_empty_list(self) < 0) - break; - continue; - - case LIST: - if (load_list(self) < 0) - break; - continue; - - case EMPTY_DICT: - if (load_empty_dict(self) < 0) - break; - continue; - - case DICT: - if (load_dict(self) < 0) - break; - continue; - - case OBJ: - if (load_obj(self) < 0) - break; - continue; - - case INST: - if (load_inst(self) < 0) - break; - continue; - - case NEWOBJ: - if (load_newobj(self) < 0) - break; - continue; - - case GLOBAL: - if (load_global(self) < 0) - break; - continue; - - case APPEND: - if (load_append(self) < 0) - break; - continue; - - case APPENDS: - if (load_appends(self) < 0) - break; - continue; - - case BUILD: - if (load_build(self) < 0) - break; - continue; - - case DUP: - if (load_dup(self) < 0) - break; - continue; - - case BINGET: - if (load_binget(self) < 0) - break; - continue; - - case LONG_BINGET: - if (load_long_binget(self) < 0) - break; - continue; - - case GET: - if (load_get(self) < 0) - break; - continue; - - case EXT1: - if (load_extension(self, 1) < 0) - break; - continue; - - case EXT2: - if (load_extension(self, 2) < 0) - break; - continue; - - case EXT4: - if (load_extension(self, 4) < 0) - break; - continue; - case MARK: - if (load_mark(self) < 0) - break; - continue; - - case BINPUT: - if (load_binput(self) < 0) - break; - continue; - - case LONG_BINPUT: - if (load_long_binput(self) < 0) - break; - continue; - - case PUT: - if (load_put(self) < 0) - break; - continue; - - case POP: - if (load_pop(self) < 0) - break; - continue; - - case POP_MARK: - if (load_pop_mark(self) < 0) - break; - continue; - - case SETITEM: - if (load_setitem(self) < 0) - break; - continue; - - case SETITEMS: - if (load_setitems(self) < 0) - break; - continue; - - case STOP: - break; - - case PERSID: - if (load_persid(self) < 0) - break; - continue; - - case BINPERSID: - if (load_binpersid(self) < 0) - break; - continue; - - case REDUCE: - if (load_reduce(self) < 0) - break; - continue; - - case PROTO: - if (load_proto(self) < 0) - break; - continue; - - case NEWTRUE: - if (load_bool(self, Py_True) < 0) - break; - continue; - - case NEWFALSE: - if (load_bool(self, Py_False) < 0) - break; - continue; - - case '\0': - /* end of file */ - PyErr_SetNone(PyExc_EOFError); - break; - - default: - cPickle_ErrFormat(UnpicklingError, - "invalid load key, '%s'.", "c", s[0]); - return NULL; - } + case EMPTY_TUPLE: + if (load_counted_tuple(self, 0) < 0) + break; + continue; + + case TUPLE1: + if (load_counted_tuple(self, 1) < 0) + break; + continue; + + case TUPLE2: + if (load_counted_tuple(self, 2) < 0) + break; + continue; + + case TUPLE3: + if (load_counted_tuple(self, 3) < 0) + break; + continue; + + case TUPLE: + if (load_tuple(self) < 0) + break; + continue; + + case EMPTY_LIST: + if (load_empty_list(self) < 0) + break; + continue; + + case LIST: + if (load_list(self) < 0) + break; + continue; + + case EMPTY_DICT: + if (load_empty_dict(self) < 0) + break; + continue; + + case DICT: + if (load_dict(self) < 0) + break; + continue; + + case OBJ: + if (load_obj(self) < 0) + break; + continue; + + case INST: + if (load_inst(self) < 0) + break; + continue; + + case NEWOBJ: + if (load_newobj(self) < 0) + break; + continue; + + case GLOBAL: + if (load_global(self) < 0) + break; + continue; + + case APPEND: + if (load_append(self) < 0) + break; + continue; + + case APPENDS: + if (load_appends(self) < 0) + break; + continue; + + case BUILD: + if (load_build(self) < 0) + break; + continue; + + case DUP: + if (load_dup(self) < 0) + break; + continue; + + case BINGET: + if (load_binget(self) < 0) + break; + continue; + + case LONG_BINGET: + if (load_long_binget(self) < 0) + break; + continue; + + case GET: + if (load_get(self) < 0) + break; + continue; + + case EXT1: + if (load_extension(self, 1) < 0) + break; + continue; + + case EXT2: + if (load_extension(self, 2) < 0) + break; + continue; + + case EXT4: + if (load_extension(self, 4) < 0) + break; + continue; + case MARK: + if (load_mark(self) < 0) + break; + continue; + + case BINPUT: + if (load_binput(self) < 0) + break; + continue; + + case LONG_BINPUT: + if (load_long_binput(self) < 0) + break; + continue; + + case PUT: + if (load_put(self) < 0) + break; + continue; + + case POP: + if (load_pop(self) < 0) + break; + continue; + + case POP_MARK: + if (load_pop_mark(self) < 0) + break; + continue; + + case SETITEM: + if (load_setitem(self) < 0) + break; + continue; + + case SETITEMS: + if (load_setitems(self) < 0) + break; + continue; + + case STOP: + break; + + case PERSID: + if (load_persid(self) < 0) + break; + continue; + + case BINPERSID: + if (load_binpersid(self) < 0) + break; + continue; + + case REDUCE: + if (load_reduce(self) < 0) + break; + continue; + + case PROTO: + if (load_proto(self) < 0) + break; + continue; + + case NEWTRUE: + if (load_bool(self, Py_True) < 0) + break; + continue; + + case NEWFALSE: + if (load_bool(self, Py_False) < 0) + break; + continue; + + case '\0': + /* end of file */ + PyErr_SetNone(PyExc_EOFError); + break; + + default: + cPickle_ErrFormat(UnpicklingError, + "invalid load key, '%s'.", "c", s[0]); + return NULL; + } - break; + break; } if ((err = PyErr_Occurred())) { - if (err == PyExc_EOFError) { - PyErr_SetNone(PyExc_EOFError); - } - return NULL; + if (err == PyExc_EOFError) { + PyErr_SetNone(PyExc_EOFError); + } + return NULL; } PDATA_POP(self->stack, val); @@ -4634,7 +4634,7 @@ int i; if ((i = marker(self)) < 0) - return -1; + return -1; return Pdata_clear(self->stack, i + 1); } @@ -4646,12 +4646,12 @@ char *s; if ((i = marker(self)) < 0) - return -1; + return -1; Pdata_clear(self->stack, i); if (self->readline_func(self, &s) < 0) - return -1; + return -1; if (self->readline_func(self, &s) < 0) - return -1; + return -1; PDATA_APPEND(self->stack, Py_None, -1); return 0; } @@ -4661,14 +4661,14 @@ { PyObject *obj; - PDATA_POP(self->stack, obj); /* pop argtuple */ + PDATA_POP(self->stack, obj); /* pop argtuple */ if (obj == NULL) - return -1; + return -1; Py_DECREF(obj); - PDATA_POP(self->stack, obj); /* pop cls */ + PDATA_POP(self->stack, obj); /* pop cls */ if (obj == NULL) - return -1; + return -1; Py_DECREF(obj); PDATA_APPEND(self->stack, Py_None, -1); @@ -4681,9 +4681,9 @@ char *s; if (self->readline_func(self, &s) < 0) - return -1; + return -1; if (self->readline_func(self, &s) < 0) - return -1; + return -1; PDATA_APPEND(self->stack, Py_None, -1); return 0; } @@ -4693,7 +4693,7 @@ { if (self->stack->length < 2) - return stackUnderflow(); + return stackUnderflow(); Pdata_clear(self->stack, self->stack->length - 2); PDATA_APPEND(self->stack, Py_None, -1); return 0; @@ -4704,7 +4704,7 @@ { if (self->stack->length < 1) - return stackUnderflow(); + return stackUnderflow(); Pdata_clear(self->stack, self->stack->length - 1); return 0; } @@ -4716,7 +4716,7 @@ assert(nbytes == 1 || nbytes == 2 || nbytes == 4); if (self->read_func(self, &codebytes, nbytes) < 0) - return -1; + return -1; PDATA_APPEND(self->stack, Py_None, -1); return 0; } @@ -4732,288 +4732,288 @@ Pdata_clear(self->stack, 0); while (1) { - if (self->read_func(self, &s, 1) < 0) - break; + if (self->read_func(self, &s, 1) < 0) + break; - switch (s[0]) { - case NONE: - if (load_none(self) < 0) - break; - continue; - - case BININT: - if (load_binint(self) < 0) - break; - continue; - - case BININT1: - if (load_binint1(self) < 0) - break; - continue; - - case BININT2: - if (load_binint2(self) < 0) - break; - continue; - - case INT: - if (load_int(self) < 0) - break; - continue; - - case LONG: - if (load_long(self) < 0) - break; - continue; - - case LONG1: - if (load_counted_long(self, 1) < 0) - break; - continue; - - case LONG4: - if (load_counted_long(self, 4) < 0) - break; - continue; - - case FLOAT: - if (load_float(self) < 0) - break; - continue; - - case BINFLOAT: - if (load_binfloat(self) < 0) - break; - continue; - - case BINSTRING: - if (load_binstring(self) < 0) - break; - continue; - - case SHORT_BINSTRING: - if (load_short_binstring(self) < 0) - break; - continue; - - case STRING: - if (load_string(self) < 0) - break; - continue; + switch (s[0]) { + case NONE: + if (load_none(self) < 0) + break; + continue; + + case BININT: + if (load_binint(self) < 0) + break; + continue; + + case BININT1: + if (load_binint1(self) < 0) + break; + continue; + + case BININT2: + if (load_binint2(self) < 0) + break; + continue; + + case INT: + if (load_int(self) < 0) + break; + continue; + + case LONG: + if (load_long(self) < 0) + break; + continue; + + case LONG1: + if (load_counted_long(self, 1) < 0) + break; + continue; + + case LONG4: + if (load_counted_long(self, 4) < 0) + break; + continue; + + case FLOAT: + if (load_float(self) < 0) + break; + continue; + + case BINFLOAT: + if (load_binfloat(self) < 0) + break; + continue; + + case BINSTRING: + if (load_binstring(self) < 0) + break; + continue; + + case SHORT_BINSTRING: + if (load_short_binstring(self) < 0) + break; + continue; + + case STRING: + if (load_string(self) < 0) + break; + continue; #ifdef Py_USING_UNICODE - case UNICODE: - if (load_unicode(self) < 0) - break; - continue; - - case BINUNICODE: - if (load_binunicode(self) < 0) - break; - continue; + case UNICODE: + if (load_unicode(self) < 0) + break; + continue; + + case BINUNICODE: + if (load_binunicode(self) < 0) + break; + continue; #endif - case EMPTY_TUPLE: - if (load_counted_tuple(self, 0) < 0) - break; - continue; - - case TUPLE1: - if (load_counted_tuple(self, 1) < 0) - break; - continue; - - case TUPLE2: - if (load_counted_tuple(self, 2) < 0) - break; - continue; - - case TUPLE3: - if (load_counted_tuple(self, 3) < 0) - break; - continue; - - case TUPLE: - if (load_tuple(self) < 0) - break; - continue; - - case EMPTY_LIST: - if (load_empty_list(self) < 0) - break; - continue; - - case LIST: - if (load_list(self) < 0) - break; - continue; - - case EMPTY_DICT: - if (load_empty_dict(self) < 0) - break; - continue; - - case DICT: - if (load_dict(self) < 0) - break; - continue; - - case OBJ: - if (noload_obj(self) < 0) - break; - continue; - - case INST: - if (noload_inst(self) < 0) - break; - continue; - - case NEWOBJ: - if (noload_newobj(self) < 0) - break; - continue; - - case GLOBAL: - if (noload_global(self) < 0) - break; - continue; - - case APPEND: - if (load_append(self) < 0) - break; - continue; - - case APPENDS: - if (load_appends(self) < 0) - break; - continue; - - case BUILD: - if (noload_build(self) < 0) - break; - continue; - - case DUP: - if (load_dup(self) < 0) - break; - continue; - - case BINGET: - if (load_binget(self) < 0) - break; - continue; - - case LONG_BINGET: - if (load_long_binget(self) < 0) - break; - continue; - - case GET: - if (load_get(self) < 0) - break; - continue; - - case EXT1: - if (noload_extension(self, 1) < 0) - break; - continue; - - case EXT2: - if (noload_extension(self, 2) < 0) - break; - continue; - - case EXT4: - if (noload_extension(self, 4) < 0) - break; - continue; - - case MARK: - if (load_mark(self) < 0) - break; - continue; - - case BINPUT: - if (load_binput(self) < 0) - break; - continue; - - case LONG_BINPUT: - if (load_long_binput(self) < 0) - break; - continue; - - case PUT: - if (load_put(self) < 0) - break; - continue; - - case POP: - if (load_pop(self) < 0) - break; - continue; - - case POP_MARK: - if (load_pop_mark(self) < 0) - break; - continue; - - case SETITEM: - if (load_setitem(self) < 0) - break; - continue; - - case SETITEMS: - if (load_setitems(self) < 0) - break; - continue; - - case STOP: - break; - - case PERSID: - if (load_persid(self) < 0) - break; - continue; - - case BINPERSID: - if (load_binpersid(self) < 0) - break; - continue; - - case REDUCE: - if (noload_reduce(self) < 0) - break; - continue; - - case PROTO: - if (load_proto(self) < 0) - break; - continue; - - case NEWTRUE: - if (load_bool(self, Py_True) < 0) - break; - continue; - - case NEWFALSE: - if (load_bool(self, Py_False) < 0) - break; - continue; - default: - cPickle_ErrFormat(UnpicklingError, - "invalid load key, '%s'.", "c", s[0]); - return NULL; - } + case EMPTY_TUPLE: + if (load_counted_tuple(self, 0) < 0) + break; + continue; + + case TUPLE1: + if (load_counted_tuple(self, 1) < 0) + break; + continue; + + case TUPLE2: + if (load_counted_tuple(self, 2) < 0) + break; + continue; + + case TUPLE3: + if (load_counted_tuple(self, 3) < 0) + break; + continue; + + case TUPLE: + if (load_tuple(self) < 0) + break; + continue; + + case EMPTY_LIST: + if (load_empty_list(self) < 0) + break; + continue; + + case LIST: + if (load_list(self) < 0) + break; + continue; + + case EMPTY_DICT: + if (load_empty_dict(self) < 0) + break; + continue; + + case DICT: + if (load_dict(self) < 0) + break; + continue; + + case OBJ: + if (noload_obj(self) < 0) + break; + continue; + + case INST: + if (noload_inst(self) < 0) + break; + continue; + + case NEWOBJ: + if (noload_newobj(self) < 0) + break; + continue; + + case GLOBAL: + if (noload_global(self) < 0) + break; + continue; + + case APPEND: + if (load_append(self) < 0) + break; + continue; + + case APPENDS: + if (load_appends(self) < 0) + break; + continue; + + case BUILD: + if (noload_build(self) < 0) + break; + continue; + + case DUP: + if (load_dup(self) < 0) + break; + continue; + + case BINGET: + if (load_binget(self) < 0) + break; + continue; + + case LONG_BINGET: + if (load_long_binget(self) < 0) + break; + continue; + + case GET: + if (load_get(self) < 0) + break; + continue; + + case EXT1: + if (noload_extension(self, 1) < 0) + break; + continue; + + case EXT2: + if (noload_extension(self, 2) < 0) + break; + continue; + + case EXT4: + if (noload_extension(self, 4) < 0) + break; + continue; + + case MARK: + if (load_mark(self) < 0) + break; + continue; + + case BINPUT: + if (load_binput(self) < 0) + break; + continue; + + case LONG_BINPUT: + if (load_long_binput(self) < 0) + break; + continue; + + case PUT: + if (load_put(self) < 0) + break; + continue; + + case POP: + if (load_pop(self) < 0) + break; + continue; + + case POP_MARK: + if (load_pop_mark(self) < 0) + break; + continue; + + case SETITEM: + if (load_setitem(self) < 0) + break; + continue; + + case SETITEMS: + if (load_setitems(self) < 0) + break; + continue; + + case STOP: + break; + + case PERSID: + if (load_persid(self) < 0) + break; + continue; + + case BINPERSID: + if (load_binpersid(self) < 0) + break; + continue; + + case REDUCE: + if (noload_reduce(self) < 0) + break; + continue; + + case PROTO: + if (load_proto(self) < 0) + break; + continue; + + case NEWTRUE: + if (load_bool(self, Py_True) < 0) + break; + continue; + + case NEWFALSE: + if (load_bool(self, Py_False) < 0) + break; + continue; + default: + cPickle_ErrFormat(UnpicklingError, + "invalid load key, '%s'.", "c", s[0]); + return NULL; + } - break; + break; } if ((err = PyErr_Occurred())) { - if (err == PyExc_EOFError) { - PyErr_SetNone(PyExc_EOFError); - } - return NULL; + if (err == PyExc_EOFError) { + PyErr_SetNone(PyExc_EOFError); + } + return NULL; } PDATA_POP(self->stack, val); @@ -5047,7 +5047,7 @@ "persistent references without instantiating any objects or importing\n" "any modules.\n") }, - {NULL, NULL} /* sentinel */ + {NULL, NULL} /* sentinel */ }; @@ -5057,7 +5057,7 @@ Unpicklerobject *self; if (!(self = PyObject_GC_New(Unpicklerobject, &Unpicklertype))) - return NULL; + return NULL; self->file = NULL; self->arg = NULL; @@ -5073,43 +5073,43 @@ self->find_class = NULL; if (!(self->memo = PyDict_New())) - goto err; + goto err; if (!self->stack) - goto err; + goto err; Py_INCREF(f); self->file = f; /* Set read, readline based on type of f */ if (PyFile_Check(f)) { - self->fp = PyFile_AsFile(f); - if (self->fp == NULL) { - PyErr_SetString(PyExc_ValueError, "I/O operation on closed file"); - goto err; - } - self->read_func = read_file; - self->readline_func = readline_file; + self->fp = PyFile_AsFile(f); + if (self->fp == NULL) { + PyErr_SetString(PyExc_ValueError, "I/O operation on closed file"); + goto err; + } + self->read_func = read_file; + self->readline_func = readline_file; } else if (PycStringIO_InputCheck(f)) { - self->fp = NULL; - self->read_func = read_cStringIO; - self->readline_func = readline_cStringIO; + self->fp = NULL; + self->read_func = read_cStringIO; + self->readline_func = readline_cStringIO; } else { - self->fp = NULL; - self->read_func = read_other; - self->readline_func = readline_other; - - if (!((self->readline = PyObject_GetAttr(f, readline_str)) && - (self->read = PyObject_GetAttr(f, read_str)))) { - PyErr_Clear(); - PyErr_SetString(PyExc_TypeError, - "argument must have 'read' and " - "'readline' attributes"); - goto err; - } + self->fp = NULL; + self->read_func = read_other; + self->readline_func = readline_other; + + if (!((self->readline = PyObject_GetAttr(f, readline_str)) && + (self->read = PyObject_GetAttr(f, read_str)))) { + PyErr_Clear(); + PyErr_SetString(PyExc_TypeError, + "argument must have 'read' and " + "'readline' attributes"); + goto err; + } } PyObject_GC_Track(self); @@ -5143,11 +5143,11 @@ Py_XDECREF(self->find_class); if (self->marks) { - free(self->marks); + free(self->marks); } if (self->buf_size) { - free(self->buf); + free(self->buf); } self->ob_type->tp_free((PyObject *) self); @@ -5187,38 +5187,38 @@ Unpickler_getattr(Unpicklerobject * self, char *name) { if (!strcmp(name, "persistent_load")) { - if (!self->pers_func) { - PyErr_SetString(PyExc_AttributeError, name); - return NULL; - } + if (!self->pers_func) { + PyErr_SetString(PyExc_AttributeError, name); + return NULL; + } - Py_INCREF(self->pers_func); - return self->pers_func; + Py_INCREF(self->pers_func); + return self->pers_func; } if (!strcmp(name, "find_global")) { - if (!self->find_class) { - PyErr_SetString(PyExc_AttributeError, name); - return NULL; - } + if (!self->find_class) { + PyErr_SetString(PyExc_AttributeError, name); + return NULL; + } - Py_INCREF(self->find_class); - return self->find_class; + Py_INCREF(self->find_class); + return self->find_class; } if (!strcmp(name, "memo")) { - if (!self->memo) { - PyErr_SetString(PyExc_AttributeError, name); - return NULL; - } + if (!self->memo) { + PyErr_SetString(PyExc_AttributeError, name); + return NULL; + } - Py_INCREF(self->memo); - return self->memo; + Py_INCREF(self->memo); + return self->memo; } if (!strcmp(name, "UnpicklingError")) { - Py_INCREF(UnpicklingError); - return UnpicklingError; + Py_INCREF(UnpicklingError); + return UnpicklingError; } return Py_FindMethod(Unpickler_methods, (PyObject *) self, name); @@ -5230,34 +5230,34 @@ { if (!strcmp(name, "persistent_load")) { - Py_XDECREF(self->pers_func); - self->pers_func = value; - Py_XINCREF(value); - return 0; + Py_XDECREF(self->pers_func); + self->pers_func = value; + Py_XINCREF(value); + return 0; } if (!strcmp(name, "find_global")) { - Py_XDECREF(self->find_class); - self->find_class = value; - Py_XINCREF(value); - return 0; + Py_XDECREF(self->find_class); + self->find_class = value; + Py_XINCREF(value); + return 0; } if (!value) { - PyErr_SetString(PyExc_TypeError, - "attribute deletion is not supported"); - return -1; + PyErr_SetString(PyExc_TypeError, + "attribute deletion is not supported"); + return -1; } if (strcmp(name, "memo") == 0) { - if (!PyDict_Check(value)) { - PyErr_SetString(PyExc_TypeError, "memo must be a dictionary"); - return -1; - } - Py_XDECREF(self->memo); - self->memo = value; - Py_INCREF(value); - return 0; + if (!PyDict_Check(value)) { + PyErr_SetString(PyExc_TypeError, "memo must be a dictionary"); + return -1; + } + Py_XDECREF(self->memo); + self->memo = value; + Py_INCREF(value); + return 0; } PyErr_SetString(PyExc_AttributeError, name); @@ -5278,14 +5278,14 @@ int proto = 0; if (!(PyArg_ParseTupleAndKeywords(args, kwds, "OO|i", kwlist, - &ob, &file, &proto))) - goto finally; + &ob, &file, &proto))) + goto finally; if (!(pickler = newPicklerobject(file, proto))) - goto finally; + goto finally; if (dump(pickler, ob) < 0) - goto finally; + goto finally; Py_INCREF(Py_None); res = Py_None; @@ -5307,17 +5307,17 @@ int proto = 0; if (!(PyArg_ParseTupleAndKeywords(args, kwds, "O|i:dumps", kwlist, - &ob, &proto))) - goto finally; + &ob, &proto))) + goto finally; if (!(file = PycStringIO->NewOutput(128))) - goto finally; + goto finally; if (!(pickler = newPicklerobject(file, proto))) - goto finally; + goto finally; if (dump(pickler, ob) < 0) - goto finally; + goto finally; res = PycStringIO->cgetvalue(file); @@ -5337,7 +5337,7 @@ PyObject *res = NULL; if (!(unpickler = newUnpicklerobject(ob))) - goto finally; + goto finally; res = load(unpickler); @@ -5356,13 +5356,13 @@ Unpicklerobject *unpickler = 0; if (!(PyArg_ParseTuple(args, "S:loads", &ob))) - goto finally; + goto finally; if (!(file = PycStringIO->NewInput(ob))) - goto finally; + goto finally; if (!(unpickler = newUnpicklerobject(file))) - goto finally; + goto finally; res = load(unpickler); @@ -5378,44 +5378,44 @@ static PyTypeObject Unpicklertype = { PyObject_HEAD_INIT(NULL) - 0, /*ob_size */ - "cPickle.Unpickler", /*tp_name */ - sizeof(Unpicklerobject), /*tp_basicsize */ + 0, /*ob_size */ + "cPickle.Unpickler", /*tp_name */ + sizeof(Unpicklerobject), /*tp_basicsize */ 0, - (destructor) Unpickler_dealloc, /* tp_dealloc */ - 0, /* tp_print */ - (getattrfunc) Unpickler_getattr, /* tp_getattr */ - (setattrfunc) Unpickler_setattr, /* tp_setattr */ - 0, /* tp_compare */ - 0, /* tp_repr */ - 0, /* tp_as_number */ - 0, /* tp_as_sequence */ - 0, /* tp_as_mapping */ - 0, /* tp_hash */ - 0, /* tp_call */ - 0, /* tp_str */ - 0, /* tp_getattro */ - 0, /* tp_setattro */ - 0, /* tp_as_buffer */ + (destructor) Unpickler_dealloc, /* tp_dealloc */ + 0, /* tp_print */ + (getattrfunc) Unpickler_getattr, /* tp_getattr */ + (setattrfunc) Unpickler_setattr, /* tp_setattr */ + 0, /* tp_compare */ + 0, /* tp_repr */ + 0, /* tp_as_number */ + 0, /* tp_as_sequence */ + 0, /* tp_as_mapping */ + 0, /* tp_hash */ + 0, /* tp_call */ + 0, /* tp_str */ + 0, /* tp_getattro */ + 0, /* tp_setattro */ + 0, /* tp_as_buffer */ Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE | Py_TPFLAGS_HAVE_GC, - Unpicklertype__doc__, /* tp_doc */ - (traverseproc) Unpickler_traverse, /* tp_traverse */ - (inquiry) Unpickler_clear, /* tp_clear */ + Unpicklertype__doc__, /* tp_doc */ + (traverseproc) Unpickler_traverse, /* tp_traverse */ + (inquiry) Unpickler_clear, /* tp_clear */ }; static struct PyMethodDef cPickle_methods[] = { {"dump", (PyCFunction) cpm_dump, METH_VARARGS | METH_KEYWORDS, PyDoc_STR("dump(obj, file, protocol=0) -- " - "Write an object in pickle format to the given file.\n" - "\n" - "See the Pickler docstring for the meaning of optional argument proto.") + "Write an object in pickle format to the given file.\n" + "\n" + "See the Pickler docstring for the meaning of optional argument proto.") }, {"dumps", (PyCFunction) cpm_dumps, METH_VARARGS | METH_KEYWORDS, PyDoc_STR("dumps(obj, protocol=0) -- " - "Return a string containing an object in pickle format.\n" - "\n" - "See the Pickler docstring for the meaning of optional argument proto.") + "Return a string containing an object in pickle format.\n" + "\n" + "See the Pickler docstring for the meaning of optional argument proto.") }, {"load", (PyCFunction) cpm_load, METH_O, @@ -5426,27 +5426,27 @@ {"Pickler", (PyCFunction) get_Pickler, METH_VARARGS | METH_KEYWORDS, PyDoc_STR("Pickler(file, protocol=0) -- Create a pickler.\n" - "\n" - "This takes a file-like object for writing a pickle data stream.\n" - "The optional proto argument tells the pickler to use the given\n" - "protocol; supported protocols are 0, 1, 2. The default\n" - "protocol is 0, to be backwards compatible. (Protocol 0 is the\n" - "only protocol that can be written to a file opened in text\n" - "mode and read back successfully. When using a protocol higher\n" - "than 0, make sure the file is opened in binary mode, both when\n" - "pickling and unpickling.)\n" - "\n" - "Protocol 1 is more efficient than protocol 0; protocol 2 is\n" - "more efficient than protocol 1.\n" - "\n" - "Specifying a negative protocol version selects the highest\n" - "protocol version supported. The higher the protocol used, the\n" - "more recent the version of Python needed to read the pickle\n" - "produced.\n" - "\n" - "The file parameter must have a write() method that accepts a single\n" - "string argument. It can thus be an open file object, a StringIO\n" - "object, or any other custom object that meets this interface.\n") + "\n" + "This takes a file-like object for writing a pickle data stream.\n" + "The optional proto argument tells the pickler to use the given\n" + "protocol; supported protocols are 0, 1, 2. The default\n" + "protocol is 0, to be backwards compatible. (Protocol 0 is the\n" + "only protocol that can be written to a file opened in text\n" + "mode and read back successfully. When using a protocol higher\n" + "than 0, make sure the file is opened in binary mode, both when\n" + "pickling and unpickling.)\n" + "\n" + "Protocol 1 is more efficient than protocol 0; protocol 2 is\n" + "more efficient than protocol 1.\n" + "\n" + "Specifying a negative protocol version selects the highest\n" + "protocol version supported. The higher the protocol used, the\n" + "more recent the version of Python needed to read the pickle\n" + "produced.\n" + "\n" + "The file parameter must have a write() method that accepts a single\n" + "string argument. It can thus be an open file object, a StringIO\n" + "object, or any other custom object that meets this interface.\n") }, {"Unpickler", (PyCFunction) get_Unpickler, METH_O, @@ -5463,9 +5463,9 @@ #define INIT_STR(S) if (!( S ## _str=PyString_InternFromString(#S))) return -1; if (PyType_Ready(&Unpicklertype) < 0) - return -1; + return -1; if (PyType_Ready(&Picklertype) < 0) - return -1; + return -1; INIT_STR(__class__); INIT_STR(__getinitargs__); @@ -5484,35 +5484,35 @@ INIT_STR(dispatch_table); if (!(copy_reg = PyImport_ImportModule("copy_reg"))) - return -1; + return -1; /* This is special because we want to use a different * one in restricted mode. */ dispatch_table = PyObject_GetAttr(copy_reg, dispatch_table_str); if (!dispatch_table) - return -1; + return -1; extension_registry = PyObject_GetAttrString(copy_reg, - "_extension_registry"); + "_extension_registry"); if (!extension_registry) - return -1; + return -1; inverted_registry = PyObject_GetAttrString(copy_reg, "_inverted_registry"); if (!inverted_registry) - return -1; + return -1; extension_cache = PyObject_GetAttrString(copy_reg, "_extension_cache"); if (!extension_cache) - return -1; + return -1; Py_DECREF(copy_reg); if (!(empty_tuple = PyTuple_New(0))) - return -1; + return -1; two_tuple = PyTuple_New(2); if (two_tuple == NULL) - return -1; + return -1; /* We use this temp container with no regard to refcounts, or to * keeping containees alive. Exempt from GC, because we don't * want anything looking at two_tuple() by magic. @@ -5521,77 +5521,77 @@ /* Ugh */ if (!(t = PyImport_ImportModule("__builtin__"))) - return -1; + return -1; if (PyDict_SetItemString(module_dict, "__builtins__", t) < 0) - return -1; + return -1; if (!(t = PyDict_New())) - return -1; + return -1; if (!(r = PyRun_String("def __str__(self):\n" - " return self.args and ('%s' % self.args[0]) or '(what)'\n", - Py_file_input, module_dict, t))) - return -1; + " return self.args and ('%s' % self.args[0]) or '(what)'\n", + Py_file_input, module_dict, t))) + return -1; Py_DECREF(r); PickleError = PyErr_NewException("cPickle.PickleError", NULL, t); if (!PickleError) - return -1; + return -1; Py_DECREF(t); PicklingError = PyErr_NewException("cPickle.PicklingError", - PickleError, NULL); + PickleError, NULL); if (!PicklingError) - return -1; + return -1; if (!(t = PyDict_New())) - return -1; + return -1; if (!(r = PyRun_String("def __str__(self):\n" - " a=self.args\n" - " a=a and type(a[0]) or '(what)'\n" - " return 'Cannot pickle %s objects' % a\n", - Py_file_input, module_dict, t))) - return -1; + " a=self.args\n" + " a=a and type(a[0]) or '(what)'\n" + " return 'Cannot pickle %s objects' % a\n", + Py_file_input, module_dict, t))) + return -1; Py_DECREF(r); if (! - (UnpickleableError = - PyErr_NewException("cPickle.UnpickleableError", PicklingError, t))) - return -1; + (UnpickleableError = + PyErr_NewException("cPickle.UnpickleableError", PicklingError, t))) + return -1; Py_DECREF(t); if (!(UnpicklingError = PyErr_NewException("cPickle.UnpicklingError", - PickleError, NULL))) - return -1; + PickleError, NULL))) + return -1; if (!(BadPickleGet = PyErr_NewException("cPickle.BadPickleGet", - UnpicklingError, NULL))) - return -1; + UnpicklingError, NULL))) + return -1; if (PyDict_SetItemString(module_dict, "PickleError", PickleError) < 0) - return -1; + return -1; if (PyDict_SetItemString(module_dict, "PicklingError", PicklingError) < 0) - return -1; + return -1; if (PyDict_SetItemString(module_dict, "UnpicklingError", - UnpicklingError) < 0) - return -1; + UnpicklingError) < 0) + return -1; if (PyDict_SetItemString(module_dict, "UnpickleableError", - UnpickleableError) < 0) - return -1; + UnpickleableError) < 0) + return -1; if (PyDict_SetItemString(module_dict, "BadPickleGet", BadPickleGet) < 0) - return -1; + return -1; PycString_IMPORT; return 0; } -#ifndef PyMODINIT_FUNC /* declarations for DLL import/export */ +#ifndef PyMODINIT_FUNC /* declarations for DLL import/export */ #define PyMODINIT_FUNC void #endif PyMODINIT_FUNC @@ -5599,7 +5599,7 @@ { PyObject *m, *d, *di, *v, *k; Py_ssize_t i; - char *rev = "1.71"; /* XXX when does this change? */ + char *rev = "1.71"; /* XXX when does this change? */ PyObject *format_version; PyObject *compatible_formats; @@ -5612,16 +5612,16 @@ */ di = PyDict_New(); if (!di) - return; + return; if (init_stuff(di) < 0) - return; + return; /* Create the module and add the functions */ m = Py_InitModule4("cPickle", cPickle_methods, - cPickle_module_documentation, - (PyObject *) NULL, PYTHON_API_VERSION); + cPickle_module_documentation, + (PyObject *) NULL, PYTHON_API_VERSION); if (m == NULL) - return; + return; /* Add some symbolic constants to the module */ d = PyModule_GetDict(m); @@ -5631,26 +5631,26 @@ /* Copy data from di. Waaa. */ for (i = 0; PyDict_Next(di, &i, &k, &v);) { - if (PyObject_SetItem(d, k, v) < 0) { - Py_DECREF(di); - return; - } + if (PyObject_SetItem(d, k, v) < 0) { + Py_DECREF(di); + return; + } } Py_DECREF(di); i = PyModule_AddIntConstant(m, "HIGHEST_PROTOCOL", HIGHEST_PROTOCOL); if (i < 0) - return; + return; /* These are purely informational; no code uses them. */ /* File format version we write. */ format_version = PyString_FromString("2.0"); /* Format versions we can read. */ - compatible_formats = Py_BuildValue("[sssss]", "1.0", /* Original protocol 0 */ - "1.1", /* Protocol 0 + INST */ - "1.2", /* Original protocol 1 */ - "1.3", /* Protocol 1 + BINFLOAT */ - "2.0"); /* Original protocol 2 */ + compatible_formats = Py_BuildValue("[sssss]", "1.0", /* Original protocol 0 */ + "1.1", /* Protocol 0 + INST */ + "1.2", /* Original protocol 1 */ + "1.3", /* Protocol 1 + BINFLOAT */ + "2.0"); /* Original protocol 2 */ PyDict_SetItemString(d, "format_version", format_version); PyDict_SetItemString(d, "compatible_formats", compatible_formats); Py_XDECREF(format_version); From python-checkins at python.org Sat Jul 7 20:22:41 2007 From: python-checkins at python.org (alexandre.vassalotti) Date: Sat, 7 Jul 2007 20:22:41 +0200 (CEST) Subject: [Python-checkins] r56184 - in python/branches/cpy_merge: Modules/_picklemodule.c setup.py Message-ID: <20070707182241.D38C41E4002@bag.python.org> Author: alexandre.vassalotti Date: Sat Jul 7 20:22:41 2007 New Revision: 56184 Modified: python/branches/cpy_merge/Modules/_picklemodule.c python/branches/cpy_merge/setup.py Log: Finish renaming cPickle to _pickle. Add build instruction to setup.py. Modified: python/branches/cpy_merge/Modules/_picklemodule.c ============================================================================== --- python/branches/cpy_merge/Modules/_picklemodule.c (original) +++ python/branches/cpy_merge/Modules/_picklemodule.c Sat Jul 7 20:22:41 2007 @@ -2,7 +2,7 @@ #include "cStringIO.h" #include "structmember.h" -PyDoc_STRVAR(cPickle_module_documentation, +PyDoc_STRVAR(pickle_module_documentation, "C implementation and optimization of the Python pickle module."); #ifndef Py_eval_input @@ -150,7 +150,7 @@ } static PyTypeObject PdataType = { - PyObject_HEAD_INIT(NULL) 0, "cPickle.Pdata", sizeof(Pdata), 0, + PyObject_HEAD_INIT(NULL) 0, "pickle.Pdata", sizeof(Pdata), 0, (destructor) Pdata_dealloc, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0L, 0L, 0L, 0L, "" }; @@ -382,7 +382,7 @@ static PyObject * -cPickle_ErrFormat(PyObject * ErrType, char *stringformat, char *format, ...) +pickle_ErrFormat(PyObject * ErrType, char *stringformat, char *format, ...) { va_list va; PyObject *args = 0, *retval = 0; @@ -1847,21 +1847,21 @@ * but I don't know how to stop it. :-( */ mod = PyImport_ImportModule(module_str); if (mod == NULL) { - cPickle_ErrFormat(PicklingError, + pickle_ErrFormat(PicklingError, "Can't pickle %s: import of module %s " "failed", "OS", args, module); goto finally; } klass = PyObject_GetAttrString(mod, name_str); if (klass == NULL) { - cPickle_ErrFormat(PicklingError, + pickle_ErrFormat(PicklingError, "Can't pickle %s: attribute lookup %s.%s " "failed", "OSS", args, module, global_name); goto finally; } if (klass != args) { Py_DECREF(klass); - cPickle_ErrFormat(PicklingError, + pickle_ErrFormat(PicklingError, "Can't pickle %s: it's not the same object " "as %s.%s", "OSS", args, module, global_name); goto finally; @@ -1885,14 +1885,14 @@ /* Verify py_code has the right type and value. */ if (!PyInt_Check(py_code)) { - cPickle_ErrFormat(PicklingError, "Can't pickle %s: " + pickle_ErrFormat(PicklingError, "Can't pickle %s: " "extension code %s isn't an integer", "OO", args, py_code); goto finally; } code = PyInt_AS_LONG(py_code); if (code <= 0 || code > 0x7fffffffL) { - cPickle_ErrFormat(PicklingError, "Can't pickle %s: " + pickle_ErrFormat(PicklingError, "Can't pickle %s: " "extension code %ld is out of range", "Ol", args, code); goto finally; @@ -2377,14 +2377,14 @@ } if (!PyTuple_Check(t)) { - cPickle_ErrFormat(PicklingError, "Value returned by " + pickle_ErrFormat(PicklingError, "Value returned by " "%s must be string or tuple", "O", __reduce__); goto finally; } size = PyTuple_Size(t); if (size < 2 || size > 5) { - cPickle_ErrFormat(PicklingError, "tuple returned by " + pickle_ErrFormat(PicklingError, "tuple returned by " "%s must contain 2 through 5 elements", "O", __reduce__); goto finally; @@ -2392,7 +2392,7 @@ arg_tup = PyTuple_GET_ITEM(t, 1); if (!(PyTuple_Check(arg_tup) || arg_tup == Py_None)) { - cPickle_ErrFormat(PicklingError, "Second element of " + pickle_ErrFormat(PicklingError, "Second element of " "tuple returned by %s must be a tuple", "O", __reduce__); goto finally; @@ -2879,7 +2879,7 @@ static PyTypeObject Picklertype = { PyObject_HEAD_INIT(NULL) 0, /*ob_size */ - "cPickle.Pickler", /*tp_name */ + "pickle.Pickler", /*tp_name */ sizeof(Picklerobject), /*tp_basicsize */ 0, (destructor) Pickler_dealloc, /* tp_dealloc */ @@ -4605,7 +4605,7 @@ break; default: - cPickle_ErrFormat(UnpicklingError, + pickle_ErrFormat(UnpicklingError, "invalid load key, '%s'.", "c", s[0]); return NULL; } @@ -5001,7 +5001,7 @@ break; continue; default: - cPickle_ErrFormat(UnpicklingError, + pickle_ErrFormat(UnpicklingError, "invalid load key, '%s'.", "c", s[0]); return NULL; } @@ -5379,7 +5379,7 @@ static PyTypeObject Unpicklertype = { PyObject_HEAD_INIT(NULL) 0, /*ob_size */ - "cPickle.Unpickler", /*tp_name */ + "pickle.Unpickler", /*tp_name */ sizeof(Unpicklerobject), /*tp_basicsize */ 0, (destructor) Unpickler_dealloc, /* tp_dealloc */ @@ -5403,7 +5403,7 @@ (inquiry) Unpickler_clear, /* tp_clear */ }; -static struct PyMethodDef cPickle_methods[] = { +static struct PyMethodDef pickle_methods[] = { {"dump", (PyCFunction) cpm_dump, METH_VARARGS | METH_KEYWORDS, PyDoc_STR("dump(obj, file, protocol=0) -- " "Write an object in pickle format to the given file.\n" @@ -5533,13 +5533,13 @@ return -1; Py_DECREF(r); - PickleError = PyErr_NewException("cPickle.PickleError", NULL, t); + PickleError = PyErr_NewException("pickle.PickleError", NULL, t); if (!PickleError) return -1; Py_DECREF(t); - PicklingError = PyErr_NewException("cPickle.PicklingError", + PicklingError = PyErr_NewException("pickle.PicklingError", PickleError, NULL); if (!PicklingError) return -1; @@ -5556,16 +5556,16 @@ if (! (UnpickleableError = - PyErr_NewException("cPickle.UnpickleableError", PicklingError, t))) + PyErr_NewException("pickle.UnpickleableError", PicklingError, t))) return -1; Py_DECREF(t); - if (!(UnpicklingError = PyErr_NewException("cPickle.UnpicklingError", + if (!(UnpicklingError = PyErr_NewException("pickle.UnpicklingError", PickleError, NULL))) return -1; - if (!(BadPickleGet = PyErr_NewException("cPickle.BadPickleGet", + if (!(BadPickleGet = PyErr_NewException("pickle.BadPickleGet", UnpicklingError, NULL))) return -1; @@ -5595,7 +5595,7 @@ #define PyMODINIT_FUNC void #endif PyMODINIT_FUNC -initcPickle(void) +init_pickle(void) { PyObject *m, *d, *di, *v, *k; Py_ssize_t i; @@ -5617,8 +5617,8 @@ return; /* Create the module and add the functions */ - m = Py_InitModule4("cPickle", cPickle_methods, - cPickle_module_documentation, + m = Py_InitModule4("_pickle", pickle_methods, + pickle_module_documentation, (PyObject *) NULL, PYTHON_API_VERSION); if (m == NULL) return; Modified: python/branches/cpy_merge/setup.py ============================================================================== --- python/branches/cpy_merge/setup.py (original) +++ python/branches/cpy_merge/setup.py Sat Jul 7 20:22:41 2007 @@ -472,6 +472,9 @@ exts.append( Extension('cStringIO', ['cStringIO.c']) ) exts.append( Extension('cPickle', ['cPickle.c']) ) + # Optimization for pickle + exts.append( Extension('_pickle', ['_picklemodule.c']) ) + # Fast implementation of BytesIO and StringIO exts.append( Extension('_bytes_io', ['_bytes_iomodule.c']) ) exts.append( Extension('_string_io', ['_string_iomodule.c']) ) From python-checkins at python.org Sat Jul 7 20:36:09 2007 From: python-checkins at python.org (alexandre.vassalotti) Date: Sat, 7 Jul 2007 20:36:09 +0200 (CEST) Subject: [Python-checkins] r56185 - python/branches/cpy_merge/Modules/_picklemodule.c Message-ID: <20070707183609.774DC1E4017@bag.python.org> Author: alexandre.vassalotti Date: Sat Jul 7 20:36:09 2007 New Revision: 56185 Modified: python/branches/cpy_merge/Modules/_picklemodule.c Log: Remove dependency on cStringIO. Remove the dumps and loads methods. Modified: python/branches/cpy_merge/Modules/_picklemodule.c ============================================================================== --- python/branches/cpy_merge/Modules/_picklemodule.c (original) +++ python/branches/cpy_merge/Modules/_picklemodule.c Sat Jul 7 20:36:09 2007 @@ -1,5 +1,4 @@ #include "Python.h" -#include "cStringIO.h" #include "structmember.h" PyDoc_STRVAR(pickle_module_documentation, @@ -443,20 +442,6 @@ } static int -write_cStringIO(Picklerobject * self, const char *s, Py_ssize_t n) -{ - if (s == NULL) { - return 0; - } - - if (PycStringIO->cwrite((PyObject *) self->file, s, n) != n) { - return -1; - } - - return (int) n; -} - -static int write_none(Picklerobject * self, const char *s, Py_ssize_t n) { if (s == NULL) @@ -602,39 +587,6 @@ } } - -static Py_ssize_t -read_cStringIO(Unpicklerobject * self, char **s, Py_ssize_t n) -{ - char *ptr; - - if (PycStringIO->cread((PyObject *) self->file, &ptr, n) != n) { - PyErr_SetNone(PyExc_EOFError); - return -1; - } - - *s = ptr; - - return n; -} - - -static Py_ssize_t -readline_cStringIO(Unpicklerobject * self, char **s) -{ - Py_ssize_t n; - char *ptr; - - if ((n = PycStringIO->creadline((PyObject *) self->file, &ptr)) < 0) { - return -1; - } - - *s = ptr; - - return n; -} - - static Py_ssize_t read_other(Unpicklerobject * self, char **s, Py_ssize_t n) { @@ -2678,9 +2630,6 @@ } self->write_func = write_file; } - else if (PycStringIO_OutputCheck(file)) { - self->write_func = write_cStringIO; - } else if (file == Py_None) { self->write_func = write_none; } @@ -5091,11 +5040,6 @@ self->read_func = read_file; self->readline_func = readline_file; } - else if (PycStringIO_InputCheck(f)) { - self->fp = NULL; - self->read_func = read_cStringIO; - self->readline_func = readline_cStringIO; - } else { self->fp = NULL; @@ -5296,39 +5240,6 @@ return res; } - -/* dumps(obj, protocol=0). */ -static PyObject * -cpm_dumps(PyObject * self, PyObject * args, PyObject * kwds) -{ - static char *kwlist[] = { "obj", "protocol", NULL }; - PyObject *ob, *file = 0, *res = NULL; - Picklerobject *pickler = 0; - int proto = 0; - - if (!(PyArg_ParseTupleAndKeywords(args, kwds, "O|i:dumps", kwlist, - &ob, &proto))) - goto finally; - - if (!(file = PycStringIO->NewOutput(128))) - goto finally; - - if (!(pickler = newPicklerobject(file, proto))) - goto finally; - - if (dump(pickler, ob) < 0) - goto finally; - - res = PycStringIO->cgetvalue(file); - - finally: - Py_XDECREF(pickler); - Py_XDECREF(file); - - return res; -} - - /* load(fileobj). */ static PyObject * cpm_load(PyObject * self, PyObject * ob) @@ -5347,33 +5258,6 @@ return res; } - -/* loads(string) */ -static PyObject * -cpm_loads(PyObject * self, PyObject * args) -{ - PyObject *ob, *file = 0, *res = NULL; - Unpicklerobject *unpickler = 0; - - if (!(PyArg_ParseTuple(args, "S:loads", &ob))) - goto finally; - - if (!(file = PycStringIO->NewInput(ob))) - goto finally; - - if (!(unpickler = newUnpicklerobject(file))) - goto finally; - - res = load(unpickler); - - finally: - Py_XDECREF(file); - Py_XDECREF(unpickler); - - return res; -} - - PyDoc_STRVAR(Unpicklertype__doc__, "Objects that know how to unpickle"); static PyTypeObject Unpicklertype = { @@ -5411,19 +5295,9 @@ "See the Pickler docstring for the meaning of optional argument proto.") }, - {"dumps", (PyCFunction) cpm_dumps, METH_VARARGS | METH_KEYWORDS, - PyDoc_STR("dumps(obj, protocol=0) -- " - "Return a string containing an object in pickle format.\n" - "\n" - "See the Pickler docstring for the meaning of optional argument proto.") - }, - {"load", (PyCFunction) cpm_load, METH_O, PyDoc_STR("load(file) -- Load a pickle from the given file")}, - {"loads", (PyCFunction) cpm_loads, METH_VARARGS, - PyDoc_STR("loads(string) -- Load a pickle from the given string")}, - {"Pickler", (PyCFunction) get_Pickler, METH_VARARGS | METH_KEYWORDS, PyDoc_STR("Pickler(file, protocol=0) -- Create a pickler.\n" "\n" @@ -5586,8 +5460,6 @@ if (PyDict_SetItemString(module_dict, "BadPickleGet", BadPickleGet) < 0) return -1; - PycString_IMPORT; - return 0; } From python-checkins at python.org Sat Jul 7 21:04:38 2007 From: python-checkins at python.org (alexandre.vassalotti) Date: Sat, 7 Jul 2007 21:04:38 +0200 (CEST) Subject: [Python-checkins] r56186 - python/branches/cpy_merge/Modules/_picklemodule.c Message-ID: <20070707190438.489AB1E4002@bag.python.org> Author: alexandre.vassalotti Date: Sat Jul 7 21:04:37 2007 New Revision: 56186 Modified: python/branches/cpy_merge/Modules/_picklemodule.c Log: Fix remaining formatting problems with typedefs. Modified: python/branches/cpy_merge/Modules/_picklemodule.c ============================================================================== --- python/branches/cpy_merge/Modules/_picklemodule.c (original) +++ python/branches/cpy_merge/Modules/_picklemodule.c Sat Jul 7 21:04:37 2007 @@ -129,13 +129,14 @@ Internal Data type for pickle data. */ typedef struct { - PyObject_HEAD int length; /* number of initial slots in data currently used */ + PyObject_HEAD + int length; /* number of initial slots in data currently used */ int size; /* number of slots in data allocated */ PyObject **data; } Pdata; static void -Pdata_dealloc(Pdata * self) +Pdata_dealloc(Pdata *self) { int i; PyObject **p; @@ -183,7 +184,7 @@ * number of items, this is a (non-erroneous) NOP. */ static int -Pdata_clear(Pdata * self, int clearto) +Pdata_clear(Pdata *self, int clearto) { int i; PyObject **p; @@ -202,7 +203,7 @@ } static int -Pdata_grow(Pdata * self) +Pdata_grow(Pdata *self) { int bigger; size_t nbytes; @@ -270,7 +271,7 @@ static PyObject * -Pdata_popTuple(Pdata * self, int start) +Pdata_popTuple(Pdata *self, int start) { PyObject *r; int i, j, l; @@ -287,7 +288,7 @@ } static PyObject * -Pdata_popList(Pdata * self, int start) +Pdata_popList(Pdata *self, int start) { PyObject *r; int i, j, l; @@ -322,7 +323,8 @@ } typedef struct Picklerobject { - PyObject_HEAD FILE * fp; + PyObject_HEAD + FILE *fp; PyObject *write; PyObject *file; PyObject *memo; @@ -353,7 +355,8 @@ static PyTypeObject Picklertype; typedef struct Unpicklerobject { - PyObject_HEAD FILE * fp; + PyObject_HEAD + FILE *fp; PyObject *file; PyObject *readline; PyObject *read; @@ -366,8 +369,8 @@ int *marks; int num_marks; int marks_size; - Py_ssize_t(*read_func) (struct Unpicklerobject *, char **, Py_ssize_t); - Py_ssize_t(*readline_func) (struct Unpicklerobject *, char **); + Py_ssize_t(*read_func) (struct Unpicklerobject *, char **, Py_ssize_t); + Py_ssize_t(*readline_func) (struct Unpicklerobject *, char **); int buf_size; char *buf; PyObject *find_class; @@ -381,7 +384,7 @@ static PyObject * -pickle_ErrFormat(PyObject * ErrType, char *stringformat, char *format, ...) +pickle_ErrFormat(PyObject *ErrType, char *stringformat, char *format, ...) { va_list va; PyObject *args = 0, *retval = 0; @@ -418,7 +421,7 @@ } static int -write_file(Picklerobject * self, const char *s, Py_ssize_t n) +write_file(Picklerobject *self, const char *s, Py_ssize_t n) { size_t nbyteswritten; @@ -433,7 +436,9 @@ Py_BEGIN_ALLOW_THREADS nbyteswritten = fwrite(s, sizeof(char), n, self->fp); - Py_END_ALLOW_THREADS if (nbyteswritten != (size_t) n) { + Py_END_ALLOW_THREADS + + if (nbyteswritten != (size_t) n) { PyErr_SetFromErrno(PyExc_IOError); return -1; } @@ -442,7 +447,7 @@ } static int -write_none(Picklerobject * self, const char *s, Py_ssize_t n) +write_none(Picklerobject *self, const char *s, Py_ssize_t n) { if (s == NULL) return 0; @@ -452,7 +457,7 @@ } static int -write_other(Picklerobject * self, const char *s, Py_ssize_t _n) +write_other(Picklerobject *self, const char *s, Py_ssize_t _n) { PyObject *py_str = 0, *junk = 0; int n; @@ -505,7 +510,7 @@ static Py_ssize_t -read_file(Unpicklerobject * self, char **s, Py_ssize_t n) +read_file(Unpicklerobject *self, char **s, Py_ssize_t n) { size_t nbytesread; @@ -532,7 +537,8 @@ Py_BEGIN_ALLOW_THREADS nbytesread = fread(self->buf, sizeof(char), n, self->fp); - Py_END_ALLOW_THREADS if (nbytesread != (size_t) n) { + Py_END_ALLOW_THREADS + if (nbytesread != (size_t) n) { if (feof(self->fp)) { PyErr_SetNone(PyExc_EOFError); return -1; @@ -549,7 +555,7 @@ static Py_ssize_t -readline_file(Unpicklerobject * self, char **s) +readline_file(Unpicklerobject *self, char **s) { int i; @@ -588,7 +594,7 @@ } static Py_ssize_t -read_other(Unpicklerobject * self, char **s, Py_ssize_t n) +read_other(Unpicklerobject *self, char **s, Py_ssize_t n) { PyObject *bytes, *str = 0; @@ -613,7 +619,7 @@ static Py_ssize_t -readline_other(Unpicklerobject * self, char **s) +readline_other(Unpicklerobject *self, char **s) { PyObject *str; Py_ssize_t str_size; @@ -651,7 +657,7 @@ static int -get(Picklerobject * self, PyObject * id) +get(Picklerobject *self, PyObject * id) { PyObject *value, *mv; long c_value; @@ -709,7 +715,7 @@ static int -put(Picklerobject * self, PyObject * ob) +put(Picklerobject *self, PyObject *ob) { if (ob->ob_refcnt < 2 || self->fast) return 0; @@ -719,7 +725,7 @@ static int -put2(Picklerobject * self, PyObject * ob) +put2(Picklerobject *self, PyObject *ob) { char c_str[30]; int p; @@ -800,7 +806,7 @@ } static PyObject * -whichmodule(PyObject * global, PyObject * global_name) +whichmodule(PyObject *global, PyObject *global_name) { Py_ssize_t i, j; PyObject *module = 0, *modules_dict = 0, *global_name_attr = 0, *name = 0; @@ -856,7 +862,7 @@ static int -fast_save_enter(Picklerobject * self, PyObject * obj) +fast_save_enter(Picklerobject *self, PyObject *obj) { /* if fast_container < 0, we're doing an error exit. */ if (++self->fast_container >= PY_CPICKLE_FAST_LIMIT) { @@ -891,7 +897,7 @@ } int -fast_save_leave(Picklerobject * self, PyObject * obj) +fast_save_leave(Picklerobject *self, PyObject *obj) { if (self->fast_container-- >= PY_CPICKLE_FAST_LIMIT) { PyObject *key = PyLong_FromVoidPtr(obj); @@ -907,7 +913,7 @@ } static int -save_none(Picklerobject * self, PyObject * args) +save_none(Picklerobject *self, PyObject *args) { static char none = NONE; if (self->write_func(self, &none, 1) < 0) @@ -917,7 +923,7 @@ } static int -save_bool(Picklerobject * self, PyObject * args) +save_bool(Picklerobject *self, PyObject *args) { static const char *buf[2] = { FALSE, TRUE }; static char len[2] = { sizeof(FALSE) - 1, sizeof(TRUE) - 1 }; @@ -934,7 +940,7 @@ } static int -save_int(Picklerobject * self, long l) +save_int(Picklerobject *self, long l) { char c_str[32]; int len = 0; @@ -983,7 +989,7 @@ static int -save_long(Picklerobject * self, PyObject * args) +save_long(Picklerobject *self, PyObject *args) { Py_ssize_t size; int res = -1; @@ -1110,7 +1116,7 @@ static int -save_float(Picklerobject * self, PyObject * args) +save_float(Picklerobject *self, PyObject *args) { double x = PyFloat_AS_DOUBLE((PyFloatObject *) args); @@ -1138,7 +1144,7 @@ static int -save_string(Picklerobject * self, PyObject * args, int doput) +save_string(Picklerobject *self, PyObject *args, int doput) { int size, len; PyObject *repr = 0; @@ -1222,7 +1228,7 @@ /* A copy of PyUnicode_EncodeRawUnicodeEscape() that also translates backslash and newline characters to \uXXXX escapes. */ static PyObject * -modified_EncodeRawUnicodeEscape(const Py_UNICODE * s, int size) +modified_EncodeRawUnicodeEscape(const Py_UNICODE *s, int size) { PyObject *repr; char *p; @@ -1259,7 +1265,7 @@ static int -save_unicode(Picklerobject * self, PyObject * args, int doput) +save_unicode(Picklerobject *self, PyObject *args, int doput) { Py_ssize_t size, len; PyObject *repr = 0; @@ -1339,7 +1345,7 @@ /* A helper for save_tuple. Push the len elements in tuple t on the stack. */ static int -store_tuple_elements(Picklerobject * self, PyObject * t, int len) +store_tuple_elements(Picklerobject *self, PyObject *t, int len) { int i; int res = -1; /* guilty until proved innocent */ @@ -1367,7 +1373,7 @@ * magic so that it works in all cases. IOW, this is a long routine. */ static int -save_tuple(Picklerobject * self, PyObject * args) +save_tuple(Picklerobject *self, PyObject *args) { PyObject *py_tuple_id = NULL; int len, i; @@ -1480,7 +1486,7 @@ * Returns 0 on success, <0 on error. */ static int -batch_list(Picklerobject * self, PyObject * iter) +batch_list(Picklerobject *self, PyObject *iter) { PyObject *obj; PyObject *slice[BATCHSIZE]; @@ -1555,7 +1561,7 @@ } static int -save_list(Picklerobject * self, PyObject * args) +save_list(Picklerobject *self, PyObject *args) { int res = -1; char s[3]; @@ -1619,7 +1625,7 @@ * ugly to bear. */ static int -batch_dict(Picklerobject * self, PyObject * iter) +batch_dict(Picklerobject *self, PyObject *iter) { PyObject *p; PyObject *slice[BATCHSIZE]; @@ -1712,7 +1718,7 @@ } static int -save_dict(Picklerobject * self, PyObject * args) +save_dict(Picklerobject *self, PyObject *args) { int res = -1; char s[3]; @@ -1768,7 +1774,7 @@ static int -save_global(Picklerobject * self, PyObject * args, PyObject * name) +save_global(Picklerobject *self, PyObject *args, PyObject *name) { PyObject *global_name = 0, *module = 0, *mod = 0, *klass = 0; char *name_str, *module_str; @@ -1906,7 +1912,7 @@ } static int -save_pers(Picklerobject * self, PyObject * args, PyObject * f) +save_pers(Picklerobject *self, PyObject *args, PyObject *f) { PyObject *pid = 0; int size, res = -1; @@ -1968,7 +1974,7 @@ * appropriate __reduce__ method for ob. */ static int -save_reduce(Picklerobject * self, PyObject * args, PyObject * ob) +save_reduce(Picklerobject *self, PyObject *args, PyObject *ob) { PyObject *callable; PyObject *argtup; @@ -2112,7 +2118,7 @@ } static int -save(Picklerobject * self, PyObject * args, int pers_save) +save(Picklerobject *self, PyObject *args, int pers_save) { PyTypeObject *type; PyObject *py_ob_id = 0, *__reduce__ = 0, *t = 0; @@ -2363,7 +2369,7 @@ static int -dump(Picklerobject * self, PyObject * args) +dump(Picklerobject *self, PyObject *args) { static char stop = STOP; @@ -2390,7 +2396,7 @@ } static PyObject * -Pickle_clear_memo(Picklerobject * self, PyObject * args) +Pickle_clear_memo(Picklerobject *self, PyObject *args) { if (self->memo) PyDict_Clear(self->memo); @@ -2399,7 +2405,7 @@ } static PyObject * -Pickle_getvalue(Picklerobject * self, PyObject * args) +Pickle_getvalue(Picklerobject *self, PyObject *args) { int l, i, rsize, ssize, clear = 1, lm; long ik; @@ -2544,7 +2550,7 @@ } static PyObject * -Pickler_dump(Picklerobject * self, PyObject * args) +Pickler_dump(Picklerobject *self, PyObject *args) { PyObject *ob; int get = 0; @@ -2577,7 +2583,7 @@ static Picklerobject * -newPicklerobject(PyObject * file, int proto) +newPicklerobject(PyObject *file, int proto) { Picklerobject *self; @@ -2666,7 +2672,7 @@ static PyObject * -get_Pickler(PyObject * self, PyObject * args, PyObject * kwds) +get_Pickler(PyObject *self, PyObject *args, PyObject *kwds) { static char *kwlist[] = { "file", "protocol", NULL }; PyObject *file = NULL; @@ -2691,7 +2697,7 @@ static void -Pickler_dealloc(Picklerobject * self) +Pickler_dealloc(Picklerobject *self) { PyObject_GC_UnTrack(self); Py_XDECREF(self->write); @@ -2707,7 +2713,7 @@ } static int -Pickler_traverse(Picklerobject * self, visitproc visit, void *arg) +Pickler_traverse(Picklerobject *self, visitproc visit, void *arg) { Py_VISIT(self->write); Py_VISIT(self->memo); @@ -2721,7 +2727,7 @@ } static int -Pickler_clear(Picklerobject * self) +Pickler_clear(Picklerobject *self) { Py_CLEAR(self->write); Py_CLEAR(self->memo); @@ -2735,7 +2741,7 @@ } static PyObject * -Pickler_get_pers_func(Picklerobject * p) +Pickler_get_pers_func(Picklerobject *p) { if (p->pers_func == NULL) PyErr_SetString(PyExc_AttributeError, "persistent_id"); @@ -2745,7 +2751,7 @@ } static int -Pickler_set_pers_func(Picklerobject * p, PyObject * v) +Pickler_set_pers_func(Picklerobject *p, PyObject *v) { if (v == NULL) { PyErr_SetString(PyExc_TypeError, @@ -2759,7 +2765,7 @@ } static int -Pickler_set_inst_pers_func(Picklerobject * p, PyObject * v) +Pickler_set_inst_pers_func(Picklerobject *p, PyObject *v) { if (v == NULL) { PyErr_SetString(PyExc_TypeError, @@ -2773,7 +2779,7 @@ } static PyObject * -Pickler_get_memo(Picklerobject * p) +Pickler_get_memo(Picklerobject *p) { if (p->memo == NULL) PyErr_SetString(PyExc_AttributeError, "memo"); @@ -2783,7 +2789,7 @@ } static int -Pickler_set_memo(Picklerobject * p, PyObject * v) +Pickler_set_memo(Picklerobject *p, PyObject *v) { if (v == NULL) { PyErr_SetString(PyExc_TypeError, @@ -2801,7 +2807,7 @@ } static PyObject * -Pickler_get_error(Picklerobject * p) +Pickler_get_error(Picklerobject *p) { /* why is this an attribute on the Pickler? */ Py_INCREF(PicklingError); @@ -2860,7 +2866,7 @@ }; static PyObject * -find_class(PyObject * py_module_name, PyObject * py_global_name, PyObject * fc) +find_class(PyObject *py_module_name, PyObject *py_global_name, PyObject *fc) { PyObject *global = 0, *module; @@ -2892,7 +2898,7 @@ } static int -marker(Unpicklerobject * self) +marker(Unpicklerobject *self) { if (self->num_marks < 1) { PyErr_SetString(UnpicklingError, "could not find MARK"); @@ -2904,7 +2910,7 @@ static int -load_none(Unpicklerobject * self) +load_none(Unpicklerobject *self) { PDATA_APPEND(self->stack, Py_None, -1); return 0; @@ -2918,7 +2924,7 @@ } static int -load_int(Unpicklerobject * self) +load_int(Unpicklerobject *self) { PyObject *py_int = 0; char *endptr, *s; @@ -2968,7 +2974,7 @@ } static int -load_bool(Unpicklerobject * self, PyObject * boolean) +load_bool(Unpicklerobject *self, PyObject *boolean) { assert(boolean == Py_True || boolean == Py_False); PDATA_APPEND(self->stack, boolean, -1); @@ -3004,7 +3010,7 @@ static int -load_binintx(Unpicklerobject * self, char *s, int x) +load_binintx(Unpicklerobject *self, char *s, int x) { PyObject *py_int = 0; long l; @@ -3020,7 +3026,7 @@ static int -load_binint(Unpicklerobject * self) +load_binint(Unpicklerobject *self) { char *s; @@ -3032,7 +3038,7 @@ static int -load_binint1(Unpicklerobject * self) +load_binint1(Unpicklerobject *self) { char *s; @@ -3044,7 +3050,7 @@ static int -load_binint2(Unpicklerobject * self) +load_binint2(Unpicklerobject *self) { char *s; @@ -3055,7 +3061,7 @@ } static int -load_long(Unpicklerobject * self) +load_long(Unpicklerobject *self) { PyObject *l = 0; char *end, *s; @@ -3085,7 +3091,7 @@ * data following. */ static int -load_counted_long(Unpicklerobject * self, int size) +load_counted_long(Unpicklerobject *self, int size) { Py_ssize_t i; char *nbytes; @@ -3124,7 +3130,7 @@ } static int -load_float(Unpicklerobject * self) +load_float(Unpicklerobject *self) { PyObject *py_float = 0; char *endptr, *s; @@ -3160,7 +3166,7 @@ } static int -load_binfloat(Unpicklerobject * self) +load_binfloat(Unpicklerobject *self) { PyObject *py_float; double x; @@ -3182,7 +3188,7 @@ } static int -load_string(Unpicklerobject * self) +load_string(Unpicklerobject *self) { PyObject *str = 0; int len, res = -1; @@ -3229,7 +3235,7 @@ static int -load_binstring(Unpicklerobject * self) +load_binstring(Unpicklerobject *self) { PyObject *py_string = 0; long l; @@ -3252,7 +3258,7 @@ static int -load_short_binstring(Unpicklerobject * self) +load_short_binstring(Unpicklerobject *self) { PyObject *py_string = 0; unsigned char l; @@ -3276,7 +3282,7 @@ #ifdef Py_USING_UNICODE static int -load_unicode(Unpicklerobject * self) +load_unicode(Unpicklerobject *self) { PyObject *str = 0; int len, res = -1; @@ -3301,7 +3307,7 @@ #ifdef Py_USING_UNICODE static int -load_binunicode(Unpicklerobject * self) +load_binunicode(Unpicklerobject *self) { PyObject *unicode; long l; @@ -3325,7 +3331,7 @@ static int -load_tuple(Unpicklerobject * self) +load_tuple(Unpicklerobject *self) { PyObject *tup; int i; @@ -3339,7 +3345,7 @@ } static int -load_counted_tuple(Unpicklerobject * self, int len) +load_counted_tuple(Unpicklerobject *self, int len) { PyObject *tup = PyTuple_New(len); @@ -3359,7 +3365,7 @@ } static int -load_empty_list(Unpicklerobject * self) +load_empty_list(Unpicklerobject *self) { PyObject *list; @@ -3370,7 +3376,7 @@ } static int -load_empty_dict(Unpicklerobject * self) +load_empty_dict(Unpicklerobject *self) { PyObject *dict; @@ -3382,7 +3388,7 @@ static int -load_list(Unpicklerobject * self) +load_list(Unpicklerobject *self) { PyObject *list = 0; int i; @@ -3396,7 +3402,7 @@ } static int -load_dict(Unpicklerobject * self) +load_dict(Unpicklerobject *self) { PyObject *dict, *key, *value; int i, j, k; @@ -3422,7 +3428,7 @@ } static PyObject * -Instance_New(PyObject * cls, PyObject * args) +Instance_New(PyObject *cls, PyObject *args) { PyObject *r = 0; @@ -3448,7 +3454,7 @@ static int -load_obj(Unpicklerobject * self) +load_obj(Unpicklerobject *self) { PyObject *class, *tup, *obj = 0; int i; @@ -3472,7 +3478,7 @@ static int -load_inst(Unpicklerobject * self) +load_inst(Unpicklerobject *self) { PyObject *tup, *class = 0, *obj = 0, *module_name, *class_name; int i, len; @@ -3516,7 +3522,7 @@ } static int -load_newobj(Unpicklerobject * self) +load_newobj(Unpicklerobject *self) { PyObject *args = NULL; PyObject *clsraw = NULL; @@ -3566,7 +3572,7 @@ } static int -load_global(Unpicklerobject * self) +load_global(Unpicklerobject *self) { PyObject *class = 0, *module_name = 0, *class_name = 0; int len; @@ -3600,7 +3606,7 @@ static int -load_persid(Unpicklerobject * self) +load_persid(Unpicklerobject *self) { PyObject *pid = 0; int len; @@ -3645,7 +3651,7 @@ } static int -load_binpersid(Unpicklerobject * self) +load_binpersid(Unpicklerobject *self) { PyObject *pid = 0; @@ -3683,7 +3689,7 @@ static int -load_pop(Unpicklerobject * self) +load_pop(Unpicklerobject *self) { int len; @@ -3709,7 +3715,7 @@ static int -load_pop_mark(Unpicklerobject * self) +load_pop_mark(Unpicklerobject *self) { int i; @@ -3723,7 +3729,7 @@ static int -load_dup(Unpicklerobject * self) +load_dup(Unpicklerobject *self) { PyObject *last; int len; @@ -3738,7 +3744,7 @@ static int -load_get(Unpicklerobject * self) +load_get(Unpicklerobject *self) { PyObject *py_str = 0, *value = 0; int len; @@ -3769,7 +3775,7 @@ static int -load_binget(Unpicklerobject * self) +load_binget(Unpicklerobject *self) { PyObject *py_key = 0, *value = 0; unsigned char key; @@ -3799,7 +3805,7 @@ static int -load_long_binget(Unpicklerobject * self) +load_long_binget(Unpicklerobject *self) { PyObject *py_key = 0, *value = 0; unsigned char c; @@ -3840,7 +3846,7 @@ * the number of bytes following the opcode, holding the index (code) value. */ static int -load_extension(Unpicklerobject * self, int nbytes) +load_extension(Unpicklerobject *self, int nbytes) { char *codebytes; /* the nbytes bytes after the opcode */ long code; /* calc_binint returns long */ @@ -3908,7 +3914,7 @@ } static int -load_put(Unpicklerobject * self) +load_put(Unpicklerobject *self) { PyObject *py_str = 0, *value = 0; int len, l; @@ -3930,7 +3936,7 @@ static int -load_binput(Unpicklerobject * self) +load_binput(Unpicklerobject *self) { PyObject *py_key = 0, *value = 0; unsigned char key; @@ -3954,7 +3960,7 @@ static int -load_long_binput(Unpicklerobject * self) +load_long_binput(Unpicklerobject *self) { PyObject *py_key = 0, *value = 0; long key; @@ -3986,7 +3992,7 @@ static int -do_append(Unpicklerobject * self, int x) +do_append(Unpicklerobject *self, int x) { PyObject *value = 0, *list = 0, *append_method = 0; int len, i; @@ -4044,21 +4050,21 @@ static int -load_append(Unpicklerobject * self) +load_append(Unpicklerobject *self) { return do_append(self, self->stack->length - 1); } static int -load_appends(Unpicklerobject * self) +load_appends(Unpicklerobject *self) { return do_append(self, marker(self)); } static int -do_setitems(Unpicklerobject * self, int x) +do_setitems(Unpicklerobject *self, int x) { PyObject *value = 0, *key = 0, *dict = 0; int len, i, r = 0; @@ -4084,20 +4090,20 @@ static int -load_setitem(Unpicklerobject * self) +load_setitem(Unpicklerobject *self) { return do_setitems(self, self->stack->length - 2); } static int -load_setitems(Unpicklerobject * self) +load_setitems(Unpicklerobject *self) { return do_setitems(self, marker(self)); } static int -load_build(Unpicklerobject * self) +load_build(Unpicklerobject *self) { PyObject *state, *inst, *slotstate; PyObject *__setstate__; @@ -4191,7 +4197,7 @@ static int -load_mark(Unpicklerobject * self) +load_mark(Unpicklerobject *self) { int s; @@ -4223,7 +4229,7 @@ } static int -load_reduce(Unpicklerobject * self) +load_reduce(Unpicklerobject *self) { PyObject *callable = 0, *arg_tup = 0, *ob = 0; @@ -4248,7 +4254,7 @@ * is the first opcode for protocols >= 2. */ static int -load_proto(Unpicklerobject * self) +load_proto(Unpicklerobject *self) { int i; char *protobyte; @@ -4270,7 +4276,7 @@ } static PyObject * -load(Unpicklerobject * self) +load(Unpicklerobject *self) { PyObject *err = 0, *val = 0; char *s; @@ -4625,7 +4631,7 @@ } static int -noload_global(Unpicklerobject * self) +noload_global(Unpicklerobject *self) { char *s; @@ -4638,7 +4644,7 @@ } static int -noload_reduce(Unpicklerobject * self) +noload_reduce(Unpicklerobject *self) { if (self->stack->length < 2) @@ -4649,7 +4655,7 @@ } static int -noload_build(Unpicklerobject * self) +noload_build(Unpicklerobject *self) { if (self->stack->length < 1) @@ -4659,7 +4665,7 @@ } static int -noload_extension(Unpicklerobject * self, int nbytes) +noload_extension(Unpicklerobject *self, int nbytes) { char *codebytes; @@ -4672,7 +4678,7 @@ static PyObject * -noload(Unpicklerobject * self) +noload(Unpicklerobject *self) { PyObject *err = 0, *val = 0; char *s; @@ -4977,7 +4983,7 @@ } static PyObject * -Unpickler_noload(Unpicklerobject * self, PyObject * unused) +Unpickler_noload(Unpicklerobject *self, PyObject *unused) { return noload(self); } @@ -5001,7 +5007,7 @@ static Unpicklerobject * -newUnpicklerobject(PyObject * f) +newUnpicklerobject(PyObject *f) { Unpicklerobject *self; @@ -5066,14 +5072,14 @@ static PyObject * -get_Unpickler(PyObject * self, PyObject * file) +get_Unpickler(PyObject *self, PyObject *file) { return (PyObject *) newUnpicklerobject(file); } static void -Unpickler_dealloc(Unpicklerobject * self) +Unpickler_dealloc(Unpicklerobject *self) { PyObject_GC_UnTrack((PyObject *) self); Py_XDECREF(self->readline); @@ -5098,7 +5104,7 @@ } static int -Unpickler_traverse(Unpicklerobject * self, visitproc visit, void *arg) +Unpickler_traverse(Unpicklerobject *self, visitproc visit, void *arg) { Py_VISIT(self->readline); Py_VISIT(self->read); @@ -5113,7 +5119,7 @@ } static int -Unpickler_clear(Unpicklerobject * self) +Unpickler_clear(Unpicklerobject *self) { Py_CLEAR(self->readline); Py_CLEAR(self->read); @@ -5128,7 +5134,7 @@ } static PyObject * -Unpickler_getattr(Unpicklerobject * self, char *name) +Unpickler_getattr(Unpicklerobject *self, char *name) { if (!strcmp(name, "persistent_load")) { if (!self->pers_func) { @@ -5170,7 +5176,7 @@ static int -Unpickler_setattr(Unpicklerobject * self, char *name, PyObject * value) +Unpickler_setattr(Unpicklerobject *self, char *name, PyObject *value) { if (!strcmp(name, "persistent_load")) { @@ -5214,7 +5220,7 @@ /* dump(obj, file, protocol=0). */ static PyObject * -cpm_dump(PyObject * self, PyObject * args, PyObject * kwds) +cpm_dump(PyObject *self, PyObject *args, PyObject *kwds) { static char *kwlist[] = { "obj", "file", "protocol", NULL }; PyObject *ob, *file, *res = NULL; @@ -5242,7 +5248,7 @@ /* load(fileobj). */ static PyObject * -cpm_load(PyObject * self, PyObject * ob) +cpm_load(PyObject *self, PyObject *ob) { Unpicklerobject *unpickler = 0; PyObject *res = NULL; @@ -5262,29 +5268,29 @@ static PyTypeObject Unpicklertype = { PyObject_HEAD_INIT(NULL) - 0, /*ob_size */ - "pickle.Unpickler", /*tp_name */ - sizeof(Unpicklerobject), /*tp_basicsize */ + 0, /*ob_size */ + "pickle.Unpickler", /*tp_name */ + sizeof(Unpicklerobject), /*tp_basicsize */ 0, (destructor) Unpickler_dealloc, /* tp_dealloc */ - 0, /* tp_print */ + 0, /* tp_print */ (getattrfunc) Unpickler_getattr, /* tp_getattr */ (setattrfunc) Unpickler_setattr, /* tp_setattr */ - 0, /* tp_compare */ - 0, /* tp_repr */ - 0, /* tp_as_number */ - 0, /* tp_as_sequence */ - 0, /* tp_as_mapping */ - 0, /* tp_hash */ - 0, /* tp_call */ - 0, /* tp_str */ - 0, /* tp_getattro */ - 0, /* tp_setattro */ - 0, /* tp_as_buffer */ + 0, /* tp_compare */ + 0, /* tp_repr */ + 0, /* tp_as_number */ + 0, /* tp_as_sequence */ + 0, /* tp_as_mapping */ + 0, /* tp_hash */ + 0, /* tp_call */ + 0, /* tp_str */ + 0, /* tp_getattro */ + 0, /* tp_setattro */ + 0, /* tp_as_buffer */ Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE | Py_TPFLAGS_HAVE_GC, - Unpicklertype__doc__, /* tp_doc */ + Unpicklertype__doc__, /* tp_doc */ (traverseproc) Unpickler_traverse, /* tp_traverse */ - (inquiry) Unpickler_clear, /* tp_clear */ + (inquiry) Unpickler_clear, /* tp_clear */ }; static struct PyMethodDef pickle_methods[] = { @@ -5325,12 +5331,11 @@ {"Unpickler", (PyCFunction) get_Unpickler, METH_O, PyDoc_STR("Unpickler(file) -- Create an unpickler.")}, - {NULL, NULL} }; static int -init_stuff(PyObject * module_dict) +init_stuff(PyObject *module_dict) { PyObject *copy_reg, *t, *r; @@ -5463,9 +5468,6 @@ return 0; } -#ifndef PyMODINIT_FUNC /* declarations for DLL import/export */ -#define PyMODINIT_FUNC void -#endif PyMODINIT_FUNC init_pickle(void) { From python-checkins at python.org Sun Jul 8 03:01:55 2007 From: python-checkins at python.org (brett.cannon) Date: Sun, 8 Jul 2007 03:01:55 +0200 (CEST) Subject: [Python-checkins] r56187 - sandbox/trunk/import_in_py/importlib.py Message-ID: <20070708010155.27D781E4002@bag.python.org> Author: brett.cannon Date: Sun Jul 8 03:01:54 2007 New Revision: 56187 Modified: sandbox/trunk/import_in_py/importlib.py Log: Remove an unneeded import. Modified: sandbox/trunk/import_in_py/importlib.py ============================================================================== --- sandbox/trunk/import_in_py/importlib.py (original) +++ sandbox/trunk/import_in_py/importlib.py Sun Jul 8 03:01:54 2007 @@ -38,12 +38,10 @@ # (Neal). import errno import os -import contextlib import warnings _importlib.errno = errno _importlib.os = os -_importlib.contextlib = contextlib _importlib.warnings = warnings del _importlib From python-checkins at python.org Sun Jul 8 03:02:13 2007 From: python-checkins at python.org (brett.cannon) Date: Sun, 8 Jul 2007 03:02:13 +0200 (CEST) Subject: [Python-checkins] r56188 - sandbox/trunk/import_in_py/tests/__init__.py Message-ID: <20070708010213.7FBA91E4002@bag.python.org> Author: brett.cannon Date: Sun Jul 8 03:02:13 2007 New Revision: 56188 Modified: sandbox/trunk/import_in_py/tests/__init__.py Log: Add a visual separator between tests. Modified: sandbox/trunk/import_in_py/tests/__init__.py ============================================================================== --- sandbox/trunk/import_in_py/tests/__init__.py (original) +++ sandbox/trunk/import_in_py/tests/__init__.py Sun Jul 8 03:02:13 2007 @@ -10,3 +10,4 @@ module_name = os.path.splitext(filename)[0] module = __import__('tests.' + module_name) getattr(module, module_name).test_main() + print '~' * 20 From python-checkins at python.org Sun Jul 8 03:13:21 2007 From: python-checkins at python.org (brett.cannon) Date: Sun, 8 Jul 2007 03:13:21 +0200 (CEST) Subject: [Python-checkins] r56189 - sandbox/trunk/import_in_py/_importlib.py Message-ID: <20070708011321.9DFE01E4002@bag.python.org> Author: brett.cannon Date: Sun Jul 8 03:13:21 2007 New Revision: 56189 Modified: sandbox/trunk/import_in_py/_importlib.py Log: Remove use of os.path.splitext. Modified: sandbox/trunk/import_in_py/_importlib.py ============================================================================== --- sandbox/trunk/import_in_py/_importlib.py (original) +++ sandbox/trunk/import_in_py/_importlib.py Sun Jul 8 03:13:21 2007 @@ -329,7 +329,8 @@ This method is required for PyPycHandler. """ - return os.path.splitext(path) + splits = path.rpartition('.') + return splits[0], splits[1] + splits[2] def create_path(self, base_path, type_, must_exist=False): """Create a new path based on a base path and requested path type. From python-checkins at python.org Sun Jul 8 09:45:47 2007 From: python-checkins at python.org (nick.coghlan) Date: Sun, 8 Jul 2007 09:45:47 +0200 (CEST) Subject: [Python-checkins] r56190 - peps/trunk/pep-0366.txt Message-ID: <20070708074547.2284B1E4002@bag.python.org> Author: nick.coghlan Date: Sun Jul 8 09:45:46 2007 New Revision: 56190 Modified: peps/trunk/pep-0366.txt Log: Major rewrite to simplify things and to incorporate python-dev feedback Modified: peps/trunk/pep-0366.txt ============================================================================== --- peps/trunk/pep-0366.txt (original) +++ peps/trunk/pep-0366.txt Sun Jul 8 09:45:46 2007 @@ -7,8 +7,8 @@ Type: Standards Track Content-Type: text/x-rst Created: 1-May-2007 -Python-Version: 2.6 -Post-History: 1-May-2007 +Python-Version: 2.6, 3.0 +Post-History: 1-May-2007, 4-Jul-2007, 7-Jul-2007 Abstract @@ -17,158 +17,95 @@ This PEP proposes a backwards compatible mechanism that permits the use of explicit relative imports from executable modules within packages. Such imports currently fail due to an awkward interaction -between PEP 328 and PEP 338 - this behaviour is the subject of at -least one open SF bug report (#1510172)[1], and has most likely -been a factor in at least a few queries on comp.lang.python (such -as Alan Isaac's question in [2]). - -With the proposed mechanism, relative imports will work automatically -if the module is executed using the ``-m`` switch. A small amount of -boilerplate will be needed in the module itself to allow the relative +between PEP 328 and PEP 338. + +By adding a new module level attribute, this PEP allows relative imports +to work automatically if the module is executed using the ``-m``switch. +A small amount of boilerplate in the module itself will allow the relative imports to work when the file is executed by name. -Import Statements and the Main Module -===================================== +Proposed Change +=============== + +The major proposed change is the introduction of a new module level +attribute, ``__package__``. When it is present, relative imports will +be based on this attribute rather than the module ``__name__`` +attribute. + +As with the current ``__name__`` attribute, setting ``__package__`` will +be the responsibility of the PEP 302 loader used to import a module. +Loaders which use ``imp.new_module()`` to create the module object will +have the new attribute set automatically to +``__name__.rpartition('.')[0]``. + +``runpy.run_module`` will also set the new attribute, basing it off the +``mod_name`` argument, rather than the ``run_name`` argument. This will +allow relative imports to work correctly from main modules executed with +the ``-m`` switch. + +When the main module is specified by its filename, then the +``__package__`` attribute will be set to the empty string. To allow +relative imports when the module is executed directly, boilerplate +similar to the following would be needed before the first relative +import statement: + + if __name__ == "__main__" and not __package_name__: + __package_name__ = "" -(This section is taken from the final revision of PEP 338) +Note that this boilerplate is sufficient only if the top level package +is already accessible via ``sys.path``. Additional code that manipulates +``sys.path`` would be needed in order for direct execution to work +without the top level package already being importable. -The release of 2.5b1 showed a surprising (although obvious in -retrospect) interaction between PEP 338 and PEP 328 - explicit -relative imports don't work from a main module. This is due to -the fact that relative imports rely on ``__name__`` to determine -the current module's position in the package hierarchy. In a main -module, the value of ``__name__`` is always ``'__main__'``, so -explicit relative imports will always fail (as they only work for -a module inside a package). - -Investigation into why implicit relative imports *appear* to work when -a main module is executed directly but fail when executed using ``-m`` -showed that such imports are actually always treated as absolute -imports. Because of the way direct execution works, the package -containing the executed module is added to sys.path, so its sibling -modules are actually imported as top level modules. This can easily -lead to multiple copies of the sibling modules in the application if -implicit relative imports are used in modules that may be directly -executed (e.g. test modules or utility scripts). - -For the 2.5 release, the recommendation is to always use absolute -imports in any module that is intended to be used as a main module. -The ``-m`` switch already provides a benefit here, as it inserts the -current directory into ``sys.path``, instead of the directory containing -the main module. This means that it is possible to run a module from -inside a package using ``-m`` so long as the current directory contains -the top level directory for the package. Absolute imports will work -correctly even if the package isn't installed anywhere else on -sys.path. If the module is executed directly and uses absolute imports -to retrieve its sibling modules, then the top level package directory -needs to be installed somewhere on sys.path (since the current directory -won't be added automatically). - -Here's an example file layout:: - - devel/ - pkg/ - __init__.py - moduleA.py - moduleB.py - test/ - __init__.py - test_A.py - test_B.py - -So long as the current directory is ``devel``, or ``devel`` is already -on ``sys.path`` and the test modules use absolute imports (such as -``import pkg.moduleA`` to retrieve the module under test, PEP 338 -allows the tests to be run as:: +This approach also has the same disadvantage as the use of absolute +imports of sibling modules - if the script is moved to a different +package or subpackage, the boilerplate will need to be updated +manually. - python -m pkg.test.test_A - python -m pkg.test.test_B Rationale for Change ==================== -In rejecting PEP 3122 (which proposed a higher impact solution to this -problem), Guido has indicated that he still isn't particularly keen on -the idea of executing modules inside packages as scripts [2]. Despite -these misgivings he has previously approved the addition of the ``-m`` -switch in Python 2.4, and the ``runpy`` module based enhancements -described in PEP 338 for Python 2.5. - -The philosophy that motivated those previous additions (i.e. access to -utility or testing scripts without needing to worry about name clashes in -either the OS executable namespace or the top level Python namespace) is -also the motivation behind fixing what I see as a bug in the current -implementation. +The current inability to use explicit relative imports from the main +module is the subject of at least one open SF bug report (#1510172)[1], +and has most likely been a factor in at least a few queries on +comp.lang.python (such as Alan Isaac's question in [2]). This PEP is intended to provide a solution which permits explicit relative imports from main modules, without incurring any significant costs during interpreter startup or normal module import. +The section in PEP 338 on relative imports and the main module provides +further details and background on this problem. -Proposed Solution -================= -The heart of the proposed solution is a new module attribute -``__package_name__``. This attribute will be defined only in -the main module (i.e. modules where ``__name__ == "__main__"``). - -For a directly executed main module, this attribute will be set -to the empty string. For a module executed using -``runpy.run_module()`` with the ``run_name`` parameter set to -``"__main__"``, the attribute will be set to -``mod_name.rpartition('.')[0]`` (i.e., everything up to -but not including the last period). - -In the import machinery there is an error handling path which -deals with the case where an explicit relative reference attempts -to go higher than the top level in the package hierarchy. This -error path would be changed to fall back on the ``__package_name__`` -attribute for explicit relative imports when the importing module -is called ``"__main__"``. - -With this change, explicit relative imports will work automatically -from a script executed with the ``-m`` switch. To allow direct -execution of the module, the following boilerplate would be needed at -the top of the script:: +Reference Implementation +======================== - if __name__ == "__main__" and not __package_name__: - __package_name__ = "" - -Note that this boilerplate is sufficient only if the top level package -is already accessible via sys.path. Additional code that manipulates -sys.path would be needed in order for direct execution to work -without the top level package already being on sys.path. - -This approach also has the same disadvantage as the use of absolute -imports of sibling modules - if the script is moved to a different -package or subpackage, the boilerplate will need to be updated -manually. - -With this feature in place, the test scripts in the package above -would be able to change their import lines to something along the -lines of ``import ..moduleA``. The scripts could then be -executed unmodified even if the name of the package was changed. - -(Rev 47142 in SVN implemented an early variant of this proposal +Rev 47142 in SVN implemented an early variant of this proposal which stored the main module's real module name in the '__module_name__' attribute. It was reverted due to the fact -that 2.5 was already in beta by that time.) +that 2.5 was already in beta by that time. + +A new patch will be developed for 2.6, and forward ported to +Py3k via svnmerge. Alternative Proposals ===================== PEP 3122 proposed addressing this problem by changing the way -the main module is identified. That's a huge compatibility cost +the main module is identified. That's a significant compatibility cost to incur to fix something that is a pretty minor bug in the overall -scheme of things. +scheme of things, and the PEP was rejected [3]. The advantage of the proposal in this PEP is that its only impact on -normal code is the tiny amount of time needed at startup to set the extra -attribute in the main module. The changes to the import machinery are all -in an existing error handling path, so normal imports don't incur any -performance penalty at all. +normal code is the small amount of time needed to set the extra +attribute when importing a module. Relative imports themselves should +be sped up fractionally, as the package name is stored in the module +globals, rather than having to be worked out again for each relative +import. References @@ -177,11 +114,12 @@ .. [1] Absolute/relative import not working? (http://www.python.org/sf/1510172) -.. [2] Guido's rejection of PEP 3122 +.. [2] c.l.p. question about modules and relative imports + (http://groups.google.com/group/comp.lang.python/browse_thread/thread/c44c769a72ca69fa/) + +.. [3] Guido's rejection of PEP 3122 (http://mail.python.org/pipermail/python-3000/2007-April/006793.html) -.. [3] c.l.p. question about modules and relative imports - (http://groups.google.com/group/comp.lang.python/browse_thread/thread/c44c769a72ca69fa/) Copyright ========= From python-checkins at python.org Sun Jul 8 10:49:55 2007 From: python-checkins at python.org (martin.v.loewis) Date: Sun, 8 Jul 2007 10:49:55 +0200 (CEST) Subject: [Python-checkins] r56191 - peps/trunk/PyRSS2Gen.py peps/trunk/pep2rss.py Message-ID: <20070708084955.353BC1E4002@bag.python.org> Author: martin.v.loewis Date: Sun Jul 8 10:49:54 2007 New Revision: 56191 Added: peps/trunk/PyRSS2Gen.py (contents, props changed) peps/trunk/pep2rss.py (contents, props changed) Log: Add pep2rss, contributed by Jonathan Ellis. Added: peps/trunk/PyRSS2Gen.py ============================================================================== --- (empty file) +++ peps/trunk/PyRSS2Gen.py Sun Jul 8 10:49:54 2007 @@ -0,0 +1,443 @@ +"""PyRSS2Gen - A Python library for generating RSS 2.0 feeds.""" + +__name__ = "PyRSS2Gen" +__version__ = (1, 0, 0) +__author__ = "Andrew Dalke " + +_generator_name = __name__ + "-" + ".".join(map(str, __version__)) + +import datetime + +# Could make this the base class; will need to add 'publish' +class WriteXmlMixin: + def write_xml(self, outfile, encoding = "iso-8859-1"): + from xml.sax import saxutils + handler = saxutils.XMLGenerator(outfile, encoding) + handler.startDocument() + self.publish(handler) + handler.endDocument() + + def to_xml(self, encoding = "iso-8859-1"): + try: + import cStringIO as StringIO + except ImportError: + import StringIO + f = StringIO.StringIO() + self.write_xml(f, encoding) + return f.getvalue() + + +def _element(handler, name, obj, d = {}): + if isinstance(obj, basestring) or obj is None: + # special-case handling to make the API easier + # to use for the common case. + handler.startElement(name, d) + if obj is not None: + handler.characters(obj) + handler.endElement(name) + else: + # It better know how to emit the correct XML. + obj.publish(handler) + +def _opt_element(handler, name, obj): + if obj is None: + return + _element(handler, name, obj) + + +def _format_date(dt): + """convert a datetime into an RFC 822 formatted date + + Input date must be in GMT. + """ + # Looks like: + # Sat, 07 Sep 2002 00:00:01 GMT + # Can't use strftime because that's locale dependent + # + # Isn't there a standard way to do this for Python? The + # rfc822 and email.Utils modules assume a timestamp. The + # following is based on the rfc822 module. + return "%s, %02d %s %04d %02d:%02d:%02d GMT" % ( + ["Mon", "Tue", "Wed", "Thu", "Fri", "Sat", "Sun"][dt.weekday()], + dt.day, + ["Jan", "Feb", "Mar", "Apr", "May", "Jun", + "Jul", "Aug", "Sep", "Oct", "Nov", "Dec"][dt.month-1], + dt.year, dt.hour, dt.minute, dt.second) + + +## +# A couple simple wrapper objects for the fields which +# take a simple value other than a string. +class IntElement: + """implements the 'publish' API for integers + + Takes the tag name and the integer value to publish. + + (Could be used for anything which uses str() to be published + to text for XML.) + """ + element_attrs = {} + def __init__(self, name, val): + self.name = name + self.val = val + def publish(self, handler): + handler.startElement(self.name, self.element_attrs) + handler.characters(str(self.val)) + handler.endElement(self.name) + +class DateElement: + """implements the 'publish' API for a datetime.datetime + + Takes the tag name and the datetime to publish. + + Converts the datetime to RFC 2822 timestamp (4-digit year). + """ + def __init__(self, name, dt): + self.name = name + self.dt = dt + def publish(self, handler): + _element(handler, self.name, _format_date(self.dt)) +#### + +class Category: + """Publish a category element""" + def __init__(self, category, domain = None): + self.category = category + self.domain = domain + def publish(self, handler): + d = {} + if self.domain is not None: + d["domain"] = self.domain + _element(handler, "category", self.category, d) + +class Cloud: + """Publish a cloud""" + def __init__(self, domain, port, path, + registerProcedure, protocol): + self.domain = domain + self.port = port + self.path = path + self.registerProcedure = registerProcedure + self.protocol = protocol + def publish(self, handler): + _element(handler, "cloud", None, { + "domain": self.domain, + "port": str(self.port), + "path": self.path, + "registerProcedure": self.registerProcedure, + "protocol": self.protocol}) + +class Image: + """Publish a channel Image""" + element_attrs = {} + def __init__(self, url, title, link, + width = None, height = None, description = None): + self.url = url + self.title = title + self.link = link + self.width = width + self.height = height + self.description = description + + def publish(self, handler): + handler.startElement("image", self.element_attrs) + + _element(handler, "url", self.url) + _element(handler, "title", self.title) + _element(handler, "link", self.link) + + width = self.width + if isinstance(width, int): + width = IntElement("width", width) + _opt_element(handler, "width", width) + + height = self.height + if isinstance(height, int): + height = IntElement("height", height) + _opt_element(handler, "height", height) + + _opt_element(handler, "description", self.description) + + handler.endElement("image") + +class Guid: + """Publish a guid + + Defaults to being a permalink, which is the assumption if it's + omitted. Hence strings are always permalinks. + """ + def __init__(self, guid, isPermaLink = 1): + self.guid = guid + self.isPermaLink = isPermaLink + def publish(self, handler): + d = {} + if self.isPermaLink: + d["isPermaLink"] = "true" + else: + d["isPermaLink"] = "false" + _element(handler, "guid", self.guid, d) + +class TextInput: + """Publish a textInput + + Apparently this is rarely used. + """ + element_attrs = {} + def __init__(self, title, description, name, link): + self.title = title + self.description = description + self.name = name + self.link = link + + def publish(self, handler): + handler.startElement("textInput", self.element_attrs) + _element(handler, "title", self.title) + _element(handler, "description", self.description) + _element(handler, "name", self.name) + _element(handler, "link", self.link) + handler.endElement("textInput") + + +class Enclosure: + """Publish an enclosure""" + def __init__(self, url, length, type): + self.url = url + self.length = length + self.type = type + def publish(self, handler): + _element(handler, "enclosure", None, + {"url": self.url, + "length": str(self.length), + "type": self.type, + }) + +class Source: + """Publish the item's original source, used by aggregators""" + def __init__(self, name, url): + self.name = name + self.url = url + def publish(self, handler): + _element(handler, "source", self.name, {"url": self.url}) + +class SkipHours: + """Publish the skipHours + + This takes a list of hours, as integers. + """ + element_attrs = {} + def __init__(self, hours): + self.hours = hours + def publish(self, handler): + if self.hours: + handler.startElement("skipHours", self.element_attrs) + for hour in self.hours: + _element(handler, "hour", str(hour)) + handler.endElement("skipHours") + +class SkipDays: + """Publish the skipDays + + This takes a list of days as strings. + """ + element_attrs = {} + def __init__(self, days): + self.days = days + def publish(self, handler): + if self.days: + handler.startElement("skipDays", self.element_attrs) + for day in self.days: + _element(handler, "day", day) + handler.endElement("skipDays") + +class RSS2(WriteXmlMixin): + """The main RSS class. + + Stores the channel attributes, with the "category" elements under + ".categories" and the RSS items under ".items". + """ + + rss_attrs = {"version": "2.0"} + element_attrs = {} + def __init__(self, + title, + link, + description, + + language = None, + copyright = None, + managingEditor = None, + webMaster = None, + pubDate = None, # a datetime, *in* *GMT* + lastBuildDate = None, # a datetime + + categories = None, # list of strings or Category + generator = _generator_name, + docs = "http://blogs.law.harvard.edu/tech/rss", + cloud = None, # a Cloud + ttl = None, # integer number of minutes + + image = None, # an Image + rating = None, # a string; I don't know how it's used + textInput = None, # a TextInput + skipHours = None, # a SkipHours with a list of integers + skipDays = None, # a SkipDays with a list of strings + + items = None, # list of RSSItems + ): + self.title = title + self.link = link + self.description = description + self.language = language + self.copyright = copyright + self.managingEditor = managingEditor + + self.webMaster = webMaster + self.pubDate = pubDate + self.lastBuildDate = lastBuildDate + + if categories is None: + categories = [] + self.categories = categories + self.generator = generator + self.docs = docs + self.cloud = cloud + self.ttl = ttl + self.image = image + self.rating = rating + self.textInput = textInput + self.skipHours = skipHours + self.skipDays = skipDays + + if items is None: + items = [] + self.items = items + + def publish(self, handler): + handler.startElement("rss", self.rss_attrs) + handler.startElement("channel", self.element_attrs) + _element(handler, "title", self.title) + _element(handler, "link", self.link) + _element(handler, "description", self.description) + + self.publish_extensions(handler) + + _opt_element(handler, "language", self.language) + _opt_element(handler, "copyright", self.copyright) + _opt_element(handler, "managingEditor", self.managingEditor) + _opt_element(handler, "webMaster", self.webMaster) + + pubDate = self.pubDate + if isinstance(pubDate, datetime.datetime): + pubDate = DateElement("pubDate", pubDate) + _opt_element(handler, "pubDate", pubDate) + + lastBuildDate = self.lastBuildDate + if isinstance(lastBuildDate, datetime.datetime): + lastBuildDate = DateElement("lastBuildDate", lastBuildDate) + _opt_element(handler, "lastBuildDate", lastBuildDate) + + for category in self.categories: + if isinstance(category, basestring): + category = Category(category) + category.publish(handler) + + _opt_element(handler, "generator", self.generator) + _opt_element(handler, "docs", self.docs) + + if self.cloud is not None: + self.cloud.publish(handler) + + ttl = self.ttl + if isinstance(self.ttl, int): + ttl = IntElement("ttl", ttl) + _opt_element(handler, "tt", ttl) + + if self.image is not None: + self.image.publish(handler) + + _opt_element(handler, "rating", self.rating) + if self.textInput is not None: + self.textInput.publish(handler) + if self.skipHours is not None: + self.skipHours.publish(handler) + if self.skipDays is not None: + self.skipDays.publish(handler) + + for item in self.items: + item.publish(handler) + + handler.endElement("channel") + handler.endElement("rss") + + def publish_extensions(self, handler): + # Derived classes can hook into this to insert + # output after the three required fields. + pass + + + +class RSSItem(WriteXmlMixin): + """Publish an RSS Item""" + element_attrs = {} + def __init__(self, + title = None, # string + link = None, # url as string + description = None, # string + author = None, # email address as string + categories = None, # list of string or Category + comments = None, # url as string + enclosure = None, # an Enclosure + guid = None, # a unique string + pubDate = None, # a datetime + source = None, # a Source + ): + + if title is None and description is None: + raise TypeError( + "must define at least one of 'title' or 'description'") + self.title = title + self.link = link + self.description = description + self.author = author + if categories is None: + categories = [] + self.categories = categories + self.comments = comments + self.enclosure = enclosure + self.guid = guid + self.pubDate = pubDate + self.source = source + # It sure does get tedious typing these names three times... + + def publish(self, handler): + handler.startElement("item", self.element_attrs) + _opt_element(handler, "title", self.title) + _opt_element(handler, "link", self.link) + self.publish_extensions(handler) + _opt_element(handler, "description", self.description) + _opt_element(handler, "author", self.author) + + for category in self.categories: + if isinstance(category, basestring): + category = Category(category) + category.publish(handler) + + _opt_element(handler, "comments", self.comments) + if self.enclosure is not None: + self.enclosure.publish(handler) + _opt_element(handler, "guid", self.guid) + + pubDate = self.pubDate + if isinstance(pubDate, datetime.datetime): + pubDate = DateElement("pubDate", pubDate) + _opt_element(handler, "pubDate", pubDate) + + if self.source is not None: + self.source.publish(handler) + + handler.endElement("item") + + def publish_extensions(self, handler): + # Derived classes can hook into this to insert + # output after the title and link elements + pass Added: peps/trunk/pep2rss.py ============================================================================== --- (empty file) +++ peps/trunk/pep2rss.py Sun Jul 8 10:49:54 2007 @@ -0,0 +1,71 @@ +#!/usr/bin/env python + +# usage: pep-hook.py $REPOS $REV +# (standard post-commit args) + +import os, glob, time, datetime, stat, re, sys +from subprocess import Popen, PIPE +import PyRSS2Gen as rssgen + +RSS_PATH = os.path.join(sys.argv[1], 'peps.rss') + +def firstline_startingwith(full_path, text): + for line in file(full_path): + if line.startswith(text): + return line[len(text):].strip() + return None + +# get list of peps with creation time (from "Created:" string in pep .txt) +peps = glob.glob('pep-*.txt') +def pep_creation_dt(full_path): + created_str = firstline_startingwith(full_path, 'Created:') + # bleh, I was hoping to avoid re but some PEPs editorialize + # on the Created line + m = re.search(r'''(\d+-\w+-\d{4})''', created_str) + if not m: + # some older ones have an empty line, that's okay, if it's old + # we ipso facto don't care about it. + # "return None" would make the most sense but datetime objects + # refuse to compare with that. :-| + return datetime.datetime(*time.localtime(0)[:6]) + created_str = m.group(1) + try: + t = time.strptime(created_str, '%d-%b-%Y') + except ValueError: + t = time.strptime(created_str, '%d-%B-%Y') + return datetime.datetime(*t[:6]) +peps_with_dt = [(pep_creation_dt(full_path), full_path) for full_path in peps] +peps_with_dt.sort() + +# generate rss items for 10 most recent peps +items = [] +for dt, full_path in peps_with_dt[-10:]: + try: + n = int(full_path.split('-')[-1].split('.')[0]) + except ValueError: + pass + title = firstline_startingwith(full_path, 'Title:') + author = firstline_startingwith(full_path, 'Author:') + url = 'http://www.python.org/dev/peps/pep-%d' % n + item = rssgen.RSSItem( + title = 'PEP %d: %s' % (n, title), + link = url, + description = 'Author: %s' % author, + guid = rssgen.Guid(url), + pubDate = dt) + items.append(item) + +# the rss envelope +desc = """ +Newest Python Enhancement Proposals (PEPs) - Information on new +language features, and some meta-information like release +procedure and schedules +""".strip() +rss = rssgen.RSS2( + title = 'Newest Python PEPs', + link = 'http://www.python.org/dev/peps', + description = desc, + lastBuildDate = datetime.datetime.now(), + items = items) + +file(RSS_PATH, 'w').write(rss.to_xml()) From python-checkins at python.org Sun Jul 8 10:59:16 2007 From: python-checkins at python.org (martin.v.loewis) Date: Sun, 8 Jul 2007 10:59:16 +0200 (CEST) Subject: [Python-checkins] r56192 - peps/trunk/pep-0353.txt Message-ID: <20070708085916.B3DBE1E4002@bag.python.org> Author: martin.v.loewis Date: Sun Jul 8 10:59:16 2007 New Revision: 56192 Modified: peps/trunk/pep-0353.txt Log: Fix typo reported by Lino Mastrodomenico. Modified: peps/trunk/pep-0353.txt ============================================================================== --- peps/trunk/pep-0353.txt (original) +++ peps/trunk/pep-0353.txt Sun Jul 8 10:59:16 2007 @@ -167,7 +167,7 @@ Particular care must be taken for PyArg_ParseTuple calls: they need all be checked for s# and t# converters, and -PY_SIZE_T_CLEAN must be defined before including Python.h +PY_SSIZE_T_CLEAN must be defined before including Python.h if the calls have been updated accordingly. Fredrik Lundh has written a scanner_ which checks the code From python-checkins at python.org Sun Jul 8 20:53:53 2007 From: python-checkins at python.org (alexandre.vassalotti) Date: Sun, 8 Jul 2007 20:53:53 +0200 (CEST) Subject: [Python-checkins] r56193 - python/branches/cpy_merge/Modules/_picklemodule.c Message-ID: <20070708185353.AAAA41E4002@bag.python.org> Author: alexandre.vassalotti Date: Sun Jul 8 20:53:53 2007 New Revision: 56193 Modified: python/branches/cpy_merge/Modules/_picklemodule.c Log: Fix the indentation of macros. Annotate PdataType. Move the INIT_STR to the top of the module. Modified: python/branches/cpy_merge/Modules/_picklemodule.c ============================================================================== --- python/branches/cpy_merge/Modules/_picklemodule.c (original) +++ python/branches/cpy_merge/Modules/_picklemodule.c Sun Jul 8 20:53:53 2007 @@ -118,12 +118,26 @@ /* For looking up name pairs in copy_reg._extension_registry. */ static PyObject *two_tuple; -static PyObject *__class___str, *__getinitargs___str, *__dict___str, - *__getstate___str, *__setstate___str, *__name___str, *__reduce___str, +#define INIT_STR(S) \ + if (!(S##_str = PyString_InternFromString(#S))) \ + return -1; + +static PyObject \ + *__class___str, + *__getinitargs___str, + *__dict___str, + *__getstate___str, + *__setstate___str, + *__name___str, + *__reduce___str, *__reduce_ex___str, - *write_str, *append_str, - *read_str, *readline_str, *__main___str, - *copy_reg_str, *dispatch_table_str; + *write_str, + *append_str, + *read_str, + *readline_str, + *__main___str, + *copy_reg_str, + *dispatch_table_str; /************************************************************************* Internal Data type for pickle data. */ @@ -131,7 +145,7 @@ typedef struct { PyObject_HEAD int length; /* number of initial slots in data currently used */ - int size; /* number of slots in data allocated */ + int size; /* number of slots in data allocated */ PyObject **data; } Pdata; @@ -150,9 +164,12 @@ } static PyTypeObject PdataType = { - PyObject_HEAD_INIT(NULL) 0, "pickle.Pdata", sizeof(Pdata), 0, - (destructor) Pdata_dealloc, - 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0L, 0L, 0L, 0L, "" + PyObject_HEAD_INIT(NULL) + 0, /*ob_size*/ + "_pickle.Pdata", /*tp_name*/ + sizeof(Pdata), /*tp_basicsize*/ + 0, /*tp_itemsize*/ + (destructor) Pdata_dealloc, /*tp_dealloc*/ }; #define Pdata_Check(O) ((O)->ob_type == &PdataType) @@ -233,14 +250,14 @@ * must be an lvalue holding PyObject*. On stack underflow, UnpicklingError * is raised and V is set to NULL. D and V may be evaluated several times. */ -#define PDATA_POP(D, V) { \ - if ((D)->length) \ - (V) = (D)->data[--((D)->length)]; \ - else { \ - PyErr_SetString(UnpicklingError, "bad pickle data"); \ - (V) = NULL; \ - } \ -} +#define PDATA_POP(D, V) { \ + if ((D)->length) \ + (V) = (D)->data[--((D)->length)]; \ + else { \ + PyErr_SetString(UnpicklingError, "bad pickle data"); \ + (V) = NULL; \ + } \ + } /* PDATA_PUSH and PDATA_APPEND both push rvalue PyObject* O on to Pdata* * D. If the Pdata stack can't be grown to hold the new value, both @@ -254,17 +271,17 @@ #define PDATA_PUSH(D, O, ER) { \ if (((Pdata*)(D))->length == ((Pdata*)(D))->size && \ Pdata_grow((Pdata*)(D)) < 0) { \ - Py_DECREF(O); \ - return ER; \ + Py_DECREF(O); \ + return ER; \ } \ ((Pdata*)(D))->data[((Pdata*)(D))->length++] = (O); \ -} + } /* Push O on stack D, pushing a new reference. */ #define PDATA_APPEND(D, O, ER) { \ if (((Pdata*)(D))->length == ((Pdata*)(D))->size && \ Pdata_grow((Pdata*)(D)) < 0) \ - return ER; \ + return ER; \ Py_INCREF(O); \ ((Pdata*)(D))->data[((Pdata*)(D))->length++] = (O); \ } @@ -305,22 +322,22 @@ /*************************************************************************/ -#define ARG_TUP(self, o) { \ - if (self->arg || (self->arg=PyTuple_New(1))) { \ - Py_XDECREF(PyTuple_GET_ITEM(self->arg,0)); \ - PyTuple_SET_ITEM(self->arg,0,o); \ - } \ - else { \ - Py_DECREF(o); \ - } \ -} +#define ARG_TUP(self, o) { \ + if (self->arg || (self->arg=PyTuple_New(1))) { \ + Py_XDECREF(PyTuple_GET_ITEM(self->arg,0)); \ + PyTuple_SET_ITEM(self->arg,0,o); \ + } \ + else { \ + Py_DECREF(o); \ + } \ + } #define FREE_ARG_TUP(self) { \ - if (self->arg->ob_refcnt > 1) { \ - Py_DECREF(self->arg); \ - self->arg=NULL; \ - } \ - } + if (self->arg->ob_refcnt > 1) { \ + Py_DECREF(self->arg); \ + self->arg=NULL; \ + } \ + } typedef struct Picklerobject { PyObject_HEAD @@ -338,7 +355,7 @@ /* bool, true if proto > 0 */ int bin; - int fast; /* Fast mode doesn't save in memo, don't use if circ ref */ + int fast; /* Fast mode doesn't save in memo, don't use if circ ref */ int nesting; int (*write_func) (struct Picklerobject *, const char *, Py_ssize_t); char *write_buf; @@ -434,9 +451,9 @@ return -1; } - Py_BEGIN_ALLOW_THREADS - nbyteswritten = fwrite(s, sizeof(char), n, self->fp); - Py_END_ALLOW_THREADS + Py_BEGIN_ALLOW_THREADS; + nbyteswritten = fwrite(s, sizeof(char), n, self->fp); + Py_END_ALLOW_THREADS; if (nbyteswritten != (size_t) n) { PyErr_SetFromErrno(PyExc_IOError); @@ -5269,7 +5286,7 @@ static PyTypeObject Unpicklertype = { PyObject_HEAD_INIT(NULL) 0, /*ob_size */ - "pickle.Unpickler", /*tp_name */ + "_pickle.Unpickler", /*tp_name */ sizeof(Unpicklerobject), /*tp_basicsize */ 0, (destructor) Unpickler_dealloc, /* tp_dealloc */ @@ -5339,8 +5356,6 @@ { PyObject *copy_reg, *t, *r; -#define INIT_STR(S) if (!( S ## _str=PyString_InternFromString(#S))) return -1; - if (PyType_Ready(&Unpicklertype) < 0) return -1; if (PyType_Ready(&Picklertype) < 0) @@ -5433,9 +5448,8 @@ return -1; Py_DECREF(r); - if (! - (UnpickleableError = - PyErr_NewException("pickle.UnpickleableError", PicklingError, t))) + if (!(UnpickleableError = PyErr_NewException("pickle.UnpickleableError", + PicklingError, t))) return -1; Py_DECREF(t); From python-checkins at python.org Sun Jul 8 21:29:28 2007 From: python-checkins at python.org (alexandre.vassalotti) Date: Sun, 8 Jul 2007 21:29:28 +0200 (CEST) Subject: [Python-checkins] r56194 - python/branches/cpy_merge/Modules/_picklemodule.c Message-ID: <20070708192928.122B51E4002@bag.python.org> Author: alexandre.vassalotti Date: Sun Jul 8 21:29:27 2007 New Revision: 56194 Modified: python/branches/cpy_merge/Modules/_picklemodule.c Log: Make fast_save_leave() static. Modified: python/branches/cpy_merge/Modules/_picklemodule.c ============================================================================== --- python/branches/cpy_merge/Modules/_picklemodule.c (original) +++ python/branches/cpy_merge/Modules/_picklemodule.c Sun Jul 8 21:29:27 2007 @@ -913,7 +913,7 @@ return 1; } -int +static int fast_save_leave(Picklerobject *self, PyObject *obj) { if (self->fast_container-- >= PY_CPICKLE_FAST_LIMIT) { From python-checkins at python.org Sun Jul 8 21:33:11 2007 From: python-checkins at python.org (alexandre.vassalotti) Date: Sun, 8 Jul 2007 21:33:11 +0200 (CEST) Subject: [Python-checkins] r56195 - python/branches/cpy_merge/Modules/_picklemodule.c Message-ID: <20070708193311.7BE5B1E4002@bag.python.org> Author: alexandre.vassalotti Date: Sun Jul 8 21:33:11 2007 New Revision: 56195 Modified: python/branches/cpy_merge/Modules/_picklemodule.c Log: Remove the unused DEL_LIST_SLICE macro. Modified: python/branches/cpy_merge/Modules/_picklemodule.c ============================================================================== --- python/branches/cpy_merge/Modules/_picklemodule.c (original) +++ python/branches/cpy_merge/Modules/_picklemodule.c Sun Jul 8 21:33:11 2007 @@ -9,8 +9,6 @@ #define Py_eval_input eval_input #endif /* Py_eval_input */ -#define DEL_LIST_SLICE(list, from, to) (PyList_SetSlice(list, from, to, NULL)) - #define WRITE_BUF_SIZE 256 /* Bump this when new opcodes are added to the pickle protocol. */ From python-checkins at python.org Sun Jul 8 21:53:06 2007 From: python-checkins at python.org (alexandre.vassalotti) Date: Sun, 8 Jul 2007 21:53:06 +0200 (CEST) Subject: [Python-checkins] r56196 - python/branches/cpy_merge/Modules/_picklemodule.c Message-ID: <20070708195306.53DD81E400F@bag.python.org> Author: alexandre.vassalotti Date: Sun Jul 8 21:53:06 2007 New Revision: 56196 Modified: python/branches/cpy_merge/Modules/_picklemodule.c Log: Remove write_none. pickle.py doesn't allow to use None as the output file, anyway. Modified: python/branches/cpy_merge/Modules/_picklemodule.c ============================================================================== --- python/branches/cpy_merge/Modules/_picklemodule.c (original) +++ python/branches/cpy_merge/Modules/_picklemodule.c Sun Jul 8 21:53:06 2007 @@ -462,16 +462,6 @@ } static int -write_none(Picklerobject *self, const char *s, Py_ssize_t n) -{ - if (s == NULL) - return 0; - if (n > INT_MAX) - return -1; - return (int) n; -} - -static int write_other(Picklerobject *self, const char *s, Py_ssize_t _n) { PyObject *py_str = 0, *junk = 0; @@ -2651,9 +2641,6 @@ } self->write_func = write_file; } - else if (file == Py_None) { - self->write_func = write_none; - } else { self->write_func = write_other; From python-checkins at python.org Sun Jul 8 22:17:58 2007 From: python-checkins at python.org (collin.winter) Date: Sun, 8 Jul 2007 22:17:58 +0200 (CEST) Subject: [Python-checkins] r56197 - sandbox/trunk/2to3/tests/test_grammar.py sandbox/trunk/2to3/tests/test_pytree.py Message-ID: <20070708201758.C4E371E4002@bag.python.org> Author: collin.winter Date: Sun Jul 8 22:17:58 2007 New Revision: 56197 Modified: sandbox/trunk/2to3/tests/test_grammar.py sandbox/trunk/2to3/tests/test_pytree.py Log: Whitespace cleanup. Modified: sandbox/trunk/2to3/tests/test_grammar.py ============================================================================== --- sandbox/trunk/2to3/tests/test_grammar.py (original) +++ sandbox/trunk/2to3/tests/test_grammar.py Sun Jul 8 22:17:58 2007 @@ -21,7 +21,7 @@ class GrammarTest(support.TestCase): def validate(self, code): support.parse_string(code) - + def invalid_syntax(self, code): try: self.validate(code) @@ -46,16 +46,16 @@ def test_3x_style(self): self.validate("raise E1 from E2") - + def test_3x_style_invalid_1(self): self.invalid_syntax("raise E, V from E1") - + def test_3x_style_invalid_2(self): self.invalid_syntax("raise E from E1, E2") - + def test_3x_style_invalid_3(self): self.invalid_syntax("raise from E1, E2") - + def test_3x_style_invalid_4(self): self.invalid_syntax("raise E from") @@ -64,25 +64,25 @@ class TestFunctionAnnotations(GrammarTest): def test_1(self): self.validate("""def f(x) -> list: pass""") - + def test_2(self): self.validate("""def f(x:int): pass""") - + def test_3(self): self.validate("""def f(*x:str): pass""") - + def test_4(self): self.validate("""def f(**x:float): pass""") - + def test_5(self): self.validate("""def f(x, y:1+2): pass""") - + def test_6(self): self.validate("""def f(a, (b:1, c:2, d)): pass""") - + def test_7(self): self.validate("""def f(a, (b:1, c:2, d), e:3=4, f=5, *g:6): pass""") - + def test_8(self): s = """def f(a, (b:1, c:2, d), e:3=4, f=5, *g:6, h:7, i=8, j:9=10, **k:11) -> 12: pass""" @@ -95,15 +95,15 @@ try: x except E as N: - y""" + y""" self.validate(s) - + def test_old(self): s = """ try: x except E, N: - y""" + y""" self.validate(s) @@ -111,13 +111,13 @@ class TestSetLiteral(GrammarTest): def test_1(self): self.validate("""x = {'one'}""") - + def test_2(self): self.validate("""x = {'one', 1,}""") - + def test_3(self): self.validate("""x = {'one', 'two', 'three'}""") - + def test_4(self): self.validate("""x = {2, 3, 4,}""") @@ -126,12 +126,12 @@ def test_new_octal_notation(self): self.validate("""0o7777777777777""") self.invalid_syntax("""0o7324528887""") - + def test_new_binary_notation(self): self.validate("""0b101010""") self.invalid_syntax("""0b0101021""") - + class TestClassDef(GrammarTest): def test_new_syntax(self): self.validate("class B(t=7): pass") @@ -145,7 +145,7 @@ def test_python2(self): f = os.path.join(test_dir, "data", "py2_test_grammar.py") driver.parse_file(f) - + def test_python3(self): f = os.path.join(test_dir, "data", "py3_test_grammar.py") driver.parse_file(f) Modified: sandbox/trunk/2to3/tests/test_pytree.py ============================================================================== --- sandbox/trunk/2to3/tests/test_pytree.py (original) +++ sandbox/trunk/2to3/tests/test_pytree.py Sun Jul 8 22:17:58 2007 @@ -47,7 +47,7 @@ self.assertEqual(str(l1), "foo") l2 = pytree.Leaf(100, "foo", context=(" ", (10, 1))) self.assertEqual(str(l2), " foo") - + def testLeafStrNumericValue(self): # Make sure that the Leaf's value is stringified. Failing to # do this can cause a TypeError in certain situations. @@ -103,11 +103,11 @@ l1 = pytree.Leaf(100, "foo", prefix="a") l2 = pytree.Leaf(100, "bar", prefix="b") n1 = pytree.Node(1000, [l1, l2]) - + self.assertEqual(l1.get_suffix(), l2.get_prefix()) self.assertEqual(l2.get_suffix(), "") self.assertEqual(n1.get_suffix(), "") - + l3 = pytree.Leaf(100, "bar", prefix="c") n2 = pytree.Node(1000, [n1, l3]) @@ -147,22 +147,22 @@ def testConvert(self): # XXX pass - + def testChangedLeaf(self): l1 = pytree.Leaf(100, "f") self.failIf(l1.was_changed) - + l1.changed() self.failUnless(l1.was_changed) - + def testChangedNode(self): l1 = pytree.Leaf(100, "f") n1 = pytree.Node(1000, [l1]) self.failIf(n1.was_changed) - + n1.changed() self.failUnless(n1.was_changed) - + def testChangedRecursive(self): l1 = pytree.Leaf(100, "foo") l2 = pytree.Leaf(100, "+") @@ -172,18 +172,18 @@ self.failIf(l1.was_changed) self.failIf(n1.was_changed) self.failIf(n2.was_changed) - + n1.changed() self.failUnless(n1.was_changed) self.failUnless(n2.was_changed) self.failIf(l1.was_changed) - + def testLeafConstructorPrefix(self): for prefix in ("xyz_", ""): l1 = pytree.Leaf(100, "self", prefix=prefix) self.failUnless(str(l1), prefix + "self") self.assertEqual(l1.get_prefix(), prefix) - + def testNodeConstructorPrefix(self): for prefix in ("xyz_", ""): l1 = pytree.Leaf(100, "self") @@ -193,7 +193,7 @@ self.assertEqual(n1.get_prefix(), prefix) self.assertEqual(l1.get_prefix(), prefix) self.assertEqual(l2.get_prefix(), "_") - + def testRemove(self): l1 = pytree.Leaf(100, "foo") l2 = pytree.Leaf(100, "foo") @@ -216,83 +216,83 @@ self.assertEqual(n2.parent, None) self.failUnless(n1.was_changed) self.failUnless(n2.was_changed) - + def testRemoveParentless(self): n1 = pytree.Node(1000, []) n1.remove() self.assertEqual(n1.parent, None) - + l1 = pytree.Leaf(100, "foo") l1.remove() self.assertEqual(l1.parent, None) - + def testNodeSetChild(self): l1 = pytree.Leaf(100, "foo") n1 = pytree.Node(1000, [l1]) - + l2 = pytree.Leaf(100, "bar") n1.set_child(0, l2) self.assertEqual(l1.parent, None) self.assertEqual(l2.parent, n1) self.assertEqual(n1.children, [l2]) - + n2 = pytree.Node(1000, [l1]) n2.set_child(0, n1) self.assertEqual(l1.parent, None) self.assertEqual(n1.parent, n2) self.assertEqual(n2.parent, None) self.assertEqual(n2.children, [n1]) - + self.assertRaises(IndexError, n1.set_child, 4, l2) # I don't care what it raises, so long as it's an exception self.assertRaises(Exception, n1.set_child, 0, list) - + def testNodeInsertChild(self): l1 = pytree.Leaf(100, "foo") n1 = pytree.Node(1000, [l1]) - + l2 = pytree.Leaf(100, "bar") n1.insert_child(0, l2) self.assertEqual(l2.parent, n1) self.assertEqual(n1.children, [l2, l1]) - + l3 = pytree.Leaf(100, "abc") n1.insert_child(2, l3) self.assertEqual(n1.children, [l2, l1, l3]) - + # I don't care what it raises, so long as it's an exception self.assertRaises(Exception, n1.insert_child, 0, list) - + def testNodeAppendChild(self): n1 = pytree.Node(1000, []) - + l1 = pytree.Leaf(100, "foo") n1.append_child(l1) self.assertEqual(l1.parent, n1) self.assertEqual(n1.children, [l1]) - + l2 = pytree.Leaf(100, "bar") n1.append_child(l2) self.assertEqual(l2.parent, n1) self.assertEqual(n1.children, [l1, l2]) - + # I don't care what it raises, so long as it's an exception self.assertRaises(Exception, n1.append_child, list) - + def testNodeNextSibling(self): n1 = pytree.Node(1000, []) n2 = pytree.Node(1000, []) p1 = pytree.Node(1000, [n1, n2]) - + self.failUnless(n1.get_next_sibling() is n2) self.assertEqual(n2.get_next_sibling(), None) self.assertEqual(p1.get_next_sibling(), None) - + def testLeafNextSibling(self): l1 = pytree.Leaf(100, "a") l2 = pytree.Leaf(100, "b") p1 = pytree.Node(1000, [l1, l2]) - + self.failUnless(l1.get_next_sibling() is l2) self.assertEqual(l2.get_next_sibling(), None) self.assertEqual(p1.get_next_sibling(), None) From python-checkins at python.org Sun Jul 8 22:35:52 2007 From: python-checkins at python.org (brett.cannon) Date: Sun, 8 Jul 2007 22:35:52 +0200 (CEST) Subject: [Python-checkins] r56198 - sandbox/trunk/import_in_py/_importlib.py Message-ID: <20070708203552.516B21E400C@bag.python.org> Author: brett.cannon Date: Sun Jul 8 22:35:52 2007 New Revision: 56198 Modified: sandbox/trunk/import_in_py/_importlib.py Log: Minor whitespace cleanup. Modified: sandbox/trunk/import_in_py/_importlib.py ============================================================================== --- sandbox/trunk/import_in_py/_importlib.py (original) +++ sandbox/trunk/import_in_py/_importlib.py Sun Jul 8 22:35:52 2007 @@ -69,11 +69,13 @@ original__import__ = __import__ __builtins__['__import__'] = Import() + def _reset__import__(): """Set __import__ back to the original implementation (assumes _set__import__ was called previously).""" __builtins__['__import__'] = original__import__ + def _w_long(x): """Convert a 32-bit integer to little-endian. @@ -87,6 +89,7 @@ bytes.append((x >> 24) & 0xFF) return ''.join(chr(x) for x in bytes) + def _r_long(bytes): """Convert 4 bytes in little-endian to an integer. @@ -99,6 +102,7 @@ x |= ord(bytes[3]) << 24 return x + def _case_ok(directory, file_name): """Verify that file_name (as found in 'directory') has the proper case. From python-checkins at python.org Sun Jul 8 22:45:19 2007 From: python-checkins at python.org (brett.cannon) Date: Sun, 8 Jul 2007 22:45:19 +0200 (CEST) Subject: [Python-checkins] r56199 - sandbox/trunk/import_in_py/_importlib.py sandbox/trunk/import_in_py/importlib.py Message-ID: <20070708204519.BC7811E4002@bag.python.org> Author: brett.cannon Date: Sun Jul 8 22:45:19 2007 New Revision: 56199 Modified: sandbox/trunk/import_in_py/_importlib.py sandbox/trunk/import_in_py/importlib.py Log: Create an os.path.join replacement. Modified: sandbox/trunk/import_in_py/_importlib.py ============================================================================== --- sandbox/trunk/import_in_py/_importlib.py (original) +++ sandbox/trunk/import_in_py/_importlib.py Sun Jul 8 22:45:19 2007 @@ -103,6 +103,11 @@ return x +def _path_join(*args): + """Replacement for os.path.join so as to remove dependency on os module.""" + return path_sep.join(args) + + def _case_ok(directory, file_name): """Verify that file_name (as found in 'directory') has the proper case. @@ -251,7 +256,7 @@ """ tail_module = fullname.rsplit('.', 1)[-1] - package_directory = os.path.join(self.path_entry, tail_module) + package_directory = _path_join(self.path_entry, tail_module) for handler in self.handlers: if handler.cannot_handle(fullname): continue @@ -259,7 +264,7 @@ # XXX Backwards-incompatible to use anything but .py/.pyc # files for __init__? init_filename = '__init__' + file_ext - package_init = os.path.join(package_directory, init_filename) + package_init = _path_join(package_directory, init_filename) # Check if it is a package with an __init__ file. if (os.path.isfile(package_init) and _case_ok(self.path_entry, tail_module) and @@ -267,7 +272,7 @@ return self.loader(package_init, handler, package_directory) # See if it is a module. file_name = tail_module + file_ext - file_path = os.path.join(self.path_entry, file_name) + file_path = _path_join(self.path_entry, file_name) if (os.path.isfile(file_path) and _case_ok(self.path_entry, file_name)): return self.loader(file_path, handler) Modified: sandbox/trunk/import_in_py/importlib.py ============================================================================== --- sandbox/trunk/import_in_py/importlib.py (original) +++ sandbox/trunk/import_in_py/importlib.py Sun Jul 8 22:45:19 2007 @@ -30,6 +30,7 @@ """ import _importlib +# Import needed built-in modules. for builtin_name in _importlib._required_builtins: module = __import__(builtin_name) _importlib.__dict__[builtin_name] = module @@ -42,6 +43,7 @@ _importlib.errno = errno _importlib.os = os +_importlib.path_sep = os.sep # For os.path.join replacement. _importlib.warnings = warnings del _importlib From python-checkins at python.org Sun Jul 8 22:53:11 2007 From: python-checkins at python.org (brett.cannon) Date: Sun, 8 Jul 2007 22:53:11 +0200 (CEST) Subject: [Python-checkins] r56200 - in sandbox/trunk/import_in_py: _importlib.py importlib.py tests/mock_importlib.py tests/test_py_handler.py tests/test_regression.py Message-ID: <20070708205311.A182B1E4002@bag.python.org> Author: brett.cannon Date: Sun Jul 8 22:53:11 2007 New Revision: 56200 Modified: sandbox/trunk/import_in_py/_importlib.py sandbox/trunk/import_in_py/importlib.py sandbox/trunk/import_in_py/tests/mock_importlib.py sandbox/trunk/import_in_py/tests/test_py_handler.py sandbox/trunk/import_in_py/tests/test_regression.py Log: Move hand-coded functions that should really come from C version out to importlib. Modified: sandbox/trunk/import_in_py/_importlib.py ============================================================================== --- sandbox/trunk/import_in_py/_importlib.py (original) +++ sandbox/trunk/import_in_py/_importlib.py Sun Jul 8 22:53:11 2007 @@ -63,76 +63,11 @@ _required_builtins = ['imp', 'sys', 'marshal'] -def _set__import__(): - """Set __import__ to an instance of Import.""" - global original__import__ - original__import__ = __import__ - __builtins__['__import__'] = Import() - - -def _reset__import__(): - """Set __import__ back to the original implementation (assumes - _set__import__ was called previously).""" - __builtins__['__import__'] = original__import__ - - -def _w_long(x): - """Convert a 32-bit integer to little-endian. - - XXX Temporary until marshal's long functions are exposed. - - """ - bytes = [] - bytes.append(x & 0xFF) - bytes.append((x >> 8) & 0xFF) - bytes.append((x >> 16) & 0xFF) - bytes.append((x >> 24) & 0xFF) - return ''.join(chr(x) for x in bytes) - - -def _r_long(bytes): - """Convert 4 bytes in little-endian to an integer. - - XXX Temporary until marshal's long function are exposed. - - """ - x = ord(bytes[0]) - x |= ord(bytes[1]) << 8 - x |= ord(bytes[2]) << 16 - x |= ord(bytes[3]) << 24 - return x - - def _path_join(*args): """Replacement for os.path.join so as to remove dependency on os module.""" return path_sep.join(args) -def _case_ok(directory, file_name): - """Verify that file_name (as found in 'directory') has the proper case. - - The path is assumed to already exist. - - XXX Temporary until imp's case_ok function can be exposed. - - XXX Better to roll this into a single function some how so that existence - check can be part of case check and thus cut down on stat calls? - - """ - # If platform is not case-sensitive *or* the environment variable - # PYTHONCASEOK is defined, then os.path.exists already handled the case by - # either doing a case-sensitive check or from the user saying he does not - # want case-sensitivity, respectively. - if sys.platform not in ('win32', 'mac', 'riscos', 'darwin', 'cygwin', - 'os2emx') or os.environ.get('PYTHONCASEOK'): - return True - directory_contents = os.listdir(directory) - if file_name in directory_contents: - return True - else: - return False - - class _BuiltinFrozenBaseImporter(object): """Base class for meta_path importers for built-in and frozen modules. Modified: sandbox/trunk/import_in_py/importlib.py ============================================================================== --- sandbox/trunk/import_in_py/importlib.py (original) +++ sandbox/trunk/import_in_py/importlib.py Sun Jul 8 22:53:11 2007 @@ -30,6 +30,72 @@ """ import _importlib +#XXX Temporary functions that should eventually be removed. +def _set__import__(): + """Set __import__ to an instance of Import.""" + global original__import__ + original__import__ = __import__ + __builtins__['__import__'] = Import() + + +def _reset__import__(): + """Set __import__ back to the original implementation (assumes + _set__import__ was called previously).""" + __builtins__['__import__'] = original__import__ + + +def _w_long(x): + """Convert a 32-bit integer to little-endian. + + XXX Temporary until marshal's long functions are exposed. + + """ + bytes = [] + bytes.append(x & 0xFF) + bytes.append((x >> 8) & 0xFF) + bytes.append((x >> 16) & 0xFF) + bytes.append((x >> 24) & 0xFF) + return ''.join(chr(x) for x in bytes) + + +def _r_long(bytes): + """Convert 4 bytes in little-endian to an integer. + + XXX Temporary until marshal's long function are exposed. + + """ + x = ord(bytes[0]) + x |= ord(bytes[1]) << 8 + x |= ord(bytes[2]) << 16 + x |= ord(bytes[3]) << 24 + return x + + +def _case_ok(directory, file_name): + """Verify that file_name (as found in 'directory') has the proper case. + + The path is assumed to already exist. + + XXX Temporary until imp's case_ok function can be exposed. + + XXX Better to roll this into a single function some how so that existence + check can be part of case check and thus cut down on stat calls? + + """ + # If platform is not case-sensitive *or* the environment variable + # PYTHONCASEOK is defined, then os.path.exists already handled the case by + # either doing a case-sensitive check or from the user saying he does not + # want case-sensitivity, respectively. + if sys.platform not in ('win32', 'mac', 'riscos', 'darwin', 'cygwin', + 'os2emx') or os.environ.get('PYTHONCASEOK'): + return True + directory_contents = os.listdir(directory) + if file_name in directory_contents: + return True + else: + return False + + # Import needed built-in modules. for builtin_name in _importlib._required_builtins: module = __import__(builtin_name) @@ -41,9 +107,13 @@ import os import warnings +_importlib._r_long = _r_long #XXX Expose original from marshal. +_importlib._w_long = _w_long #XXX Expose original from marshal. +_importlib._case_ok = _case_ok #XXX Expose original from imp. +_importlib.path_sep = os.sep # For os.path.join replacement. + _importlib.errno = errno _importlib.os = os -_importlib.path_sep = os.sep # For os.path.join replacement. _importlib.warnings = warnings del _importlib Modified: sandbox/trunk/import_in_py/tests/mock_importlib.py ============================================================================== --- sandbox/trunk/import_in_py/tests/mock_importlib.py (original) +++ sandbox/trunk/import_in_py/tests/mock_importlib.py Sun Jul 8 22:53:11 2007 @@ -2,7 +2,7 @@ import marshal import imp from test import test_support -from _importlib import _w_long +from importlib import _w_long def log_call(method): """Log method calls to self.log.""" Modified: sandbox/trunk/import_in_py/tests/test_py_handler.py ============================================================================== --- sandbox/trunk/import_in_py/tests/test_py_handler.py (original) +++ sandbox/trunk/import_in_py/tests/test_py_handler.py Sun Jul 8 22:53:11 2007 @@ -4,7 +4,7 @@ from tests import mock_importlib from tests.py_help import TestPyPycFiles -from _importlib import _r_long +from importlib import _r_long import imp import marshal Modified: sandbox/trunk/import_in_py/tests/test_regression.py ============================================================================== --- sandbox/trunk/import_in_py/tests/test_regression.py (original) +++ sandbox/trunk/import_in_py/tests/test_regression.py Sun Jul 8 22:53:11 2007 @@ -2,7 +2,7 @@ from tests import mock_importlib from tests.py_help import TestPyPycPackages -from _importlib import _r_long +from importlib import _r_long import imp import marshal From python-checkins at python.org Sun Jul 8 23:21:49 2007 From: python-checkins at python.org (collin.winter) Date: Sun, 8 Jul 2007 23:21:49 +0200 (CEST) Subject: [Python-checkins] r56201 - in sandbox/trunk/2to3: fixes/fix_dict.py fixes/fix_raise.py tests/test_fixers.py Message-ID: <20070708212149.762E61E4007@bag.python.org> Author: collin.winter Date: Sun Jul 8 23:21:49 2007 New Revision: 56201 Modified: sandbox/trunk/2to3/fixes/fix_dict.py sandbox/trunk/2to3/fixes/fix_raise.py sandbox/trunk/2to3/tests/test_fixers.py Log: Add a whole slew of prefix-related tests; fix two prefix-related bugs in the dict and raise fixers; minor test suite cleanup. Modified: sandbox/trunk/2to3/fixes/fix_dict.py ============================================================================== --- sandbox/trunk/2to3/fixes/fix_dict.py (original) +++ sandbox/trunk/2to3/fixes/fix_dict.py Sun Jul 8 23:21:49 2007 @@ -27,14 +27,14 @@ import patcomp from pgen2 import token from fixes import basefix -from fixes.util import Name, Call, LParen, RParen +from fixes.util import Name, Call, LParen, RParen, ArgList class FixDict(basefix.BaseFix): PATTERN = """ power< head=any+ trailer< '.' method=('keys'|'items'|'values'| 'iterkeys'|'iteritems'|'itervalues') > - trailer< '(' ')' > + parens=trailer< '(' ')' > tail=any* > """ @@ -42,20 +42,22 @@ def transform(self, node): results = self.match(node) head = results["head"] - method = results["method"][0].value # Extract method name + method = results["method"][0] # Extract node for method name tail = results["tail"] syms = self.syms - isiter = method.startswith("iter") + method_name = method.value + isiter = method_name.startswith("iter") if isiter: - method = method[4:] - assert method in ("keys", "items", "values"), repr(method) + method_name = method_name[4:] + assert method_name in ("keys", "items", "values"), repr(method) head = [n.clone() for n in head] tail = [n.clone() for n in tail] special = not tail and self.in_special_context(node, isiter) args = head + [pytree.Node(syms.trailer, [pytree.Leaf(token.DOT, '.'), - Name(method)]), - pytree.Node(syms.trailer, [LParen(), RParen()])] + Name(method_name, + prefix=method.get_prefix())]), + results["parens"].clone()] new = pytree.Node(syms.power, args) if not special: new.set_prefix("") Modified: sandbox/trunk/2to3/fixes/fix_raise.py ============================================================================== --- sandbox/trunk/2to3/fixes/fix_raise.py (original) +++ sandbox/trunk/2to3/fixes/fix_raise.py Sun Jul 8 23:21:49 2007 @@ -37,7 +37,7 @@ syms = self.syms results = self.match(node) assert results - + exc = results["exc"].clone() if exc.type is token.STRING: self.cannot_convert(node, "Python 3 does not support string exceptions") @@ -49,18 +49,19 @@ # raise E1, V # Since Python 3 will not support this, we recurse down any tuple # literals, always taking the first element. - while is_tuple(exc): - # exc.children[1:-1] is the unparenthesized tuple - # exc.children[1].children[0] is the first element of the tuple - exc = exc.children[1].children[0].clone() - exc.set_prefix(" ") + if is_tuple(exc): + while is_tuple(exc): + # exc.children[1:-1] is the unparenthesized tuple + # exc.children[1].children[0] is the first element of the tuple + exc = exc.children[1].children[0].clone() + exc.set_prefix(" ") if "val" not in results: # One-argument raise new = pytree.Node(syms.raise_stmt, [Name("raise"), exc]) new.set_prefix(node.get_prefix()) return new - + val = results["val"].clone() if is_tuple(val): args = [c.clone() for c in val.children[1:-1]] @@ -71,7 +72,7 @@ if "tb" in results: tb = results["tb"].clone() tb.set_prefix("") - + e = Call(exc, args) with_tb = Attr(e, Name('with_traceback')) + [ArgList([tb])] new = pytree.Node(syms.simple_stmt, [Name("raise")] + with_tb) Modified: sandbox/trunk/2to3/tests/test_fixers.py ============================================================================== --- sandbox/trunk/2to3/tests/test_fixers.py (original) +++ sandbox/trunk/2to3/tests/test_fixers.py Sun Jul 8 23:21:49 2007 @@ -277,49 +277,60 @@ class Test_intern(FixerTestCase): fixer = "intern" - def test_1(self): - b = """x = intern(a)""" - a = """x = sys.intern(a)""" + def test_prefix_preservation(self): + b = """x = intern( a )""" + a = """x = sys.intern( a )""" self.check(b, a) - def test_2(self): b = """y = intern("b" # test )""" a = """y = sys.intern("b" # test )""" self.check(b, a) - def test_3(self): + b = """z = intern(a+b+c.d, )""" + a = """z = sys.intern(a+b+c.d, )""" + self.check(b, a) + + def test(self): + b = """x = intern(a)""" + a = """x = sys.intern(a)""" + self.check(b, a) + b = """z = intern(a+b+c.d,)""" a = """z = sys.intern(a+b+c.d,)""" self.check(b, a) - def test_4(self): b = """intern("y%s" % 5).replace("y", "")""" a = """sys.intern("y%s" % 5).replace("y", "")""" self.check(b, a) # These should not be refactored - def test_unchanged_1(self): + def test_unchanged(self): s = """intern(a=1)""" self.check(s, s) - def test_unchanged_2(self): s = """intern(f, g)""" self.check(s, s) - def test_unchanged_3(self): s = """intern(*h)""" self.check(s, s) - def test_unchanged_4(self): s = """intern(**i)""" self.check(s, s) + s = """intern()""" + self.check(s, s) + class Test_print(FixerTestCase): fixer = "print" + def test_prefix_preservation(self): + b = """print 1, 1+1, 1+1+1""" + a = """print(1, 1+1, 1+1+1)""" + self.check(b, a) + def test_1(self): b = """print 1, 1+1, 1+1+1""" a = """print(1, 1+1, 1+1+1)""" @@ -378,6 +389,11 @@ class Test_exec(FixerTestCase): fixer = "exec" + def test_prefix_preservation(self): + b = """ exec code in ns1, ns2""" + a = """ exec(code, ns1, ns2)""" + self.check(b, a) + def test_basic(self): b = """exec code""" a = """exec(code)""" @@ -425,6 +441,11 @@ class Test_repr(FixerTestCase): fixer = "repr" + def test_prefix_preservation(self): + b = """x = `1 + 2`""" + a = """x = repr(1 + 2)""" + self.check(b, a) + def test_simple_1(self): b = """x = `1 + 2`""" a = """x = repr(1 + 2)""" @@ -458,6 +479,19 @@ class Test_except(FixerTestCase): fixer = "except" + def test_prefix_preservation(self): + b = """ + try: + pass + except (RuntimeError, ImportError), e: + pass""" + a = """ + try: + pass + except (RuntimeError, ImportError) as e: + pass""" + self.check(b, a) + def test_tuple_unpack(self): b = """ def foo(): @@ -606,11 +640,30 @@ a = """raise Exception(5)""" self.check(b, a) - def test_prefix(self): + def test_prefix_preservation(self): b = """raise Exception,5""" a = """raise Exception(5)""" self.check(b, a) + b = """raise Exception, 5""" + a = """raise Exception(5)""" + self.check(b, a) + + def test_with_comments(self): + b = """raise Exception, 5 # foo""" + a = """raise Exception(5) # foo""" + self.check(b, a) + + b = """raise E, (5, 6) % (a, b) # foo""" + a = """raise E((5, 6) % (a, b)) # foo""" + self.check(b, a) + + b = """def foo(): + raise Exception, 5, 6 # foo""" + a = """def foo(): + raise Exception(5).with_traceback(6) # foo""" + self.check(b, a) + def test_tuple_value(self): b = """raise Exception, (5, 6, 7)""" a = """raise Exception(5, 6, 7)""" @@ -859,36 +912,34 @@ a = """b = 0x12""" self.check(b, a) - # These should not be touched - - def test_6(self): + def test_unchanged_1(self): b = """a = 12""" - a = """a = 12""" - self.check(b, a) + self.check(b, b) - def test_7(self): + def test_unchanged_2(self): b = """b = 0x12""" - a = """b = 0x12""" - self.check(b, a) + self.check(b, b) - def test_8(self): + def test_unchanged_3(self): b = """c = 3.14""" - a = """c = 3.14""" + self.check(b, b) + + def test_prefix_preservation(self): + b = """x = long( x )""" + a = """x = int( x )""" self.check(b, a) class Test_sysexcattrs(FixerTestCase): fixer = "sysexcattrs" - def test_1(self): + def test(self): s = """f = sys.exc_type""" self.warns(s, s, "This attribute is going away") - def test_2(self): s = """f = sys.exc_value""" self.warns(s, s, "This attribute is going away") - def test_3(self): s = """f = sys.exc_traceback""" self.warns(s, s, "This attribute is going away") @@ -896,12 +947,47 @@ class Test_dict(FixerTestCase): fixer = "dict" + def test_prefix_preservation(self): + b = "if d. keys ( ) : pass" + a = "if list(d. keys ( )) : pass" + self.check(b, a) + + b = "if d. items ( ) : pass" + a = "if list(d. items ( )) : pass" + self.check(b, a) + + b = "if d. iterkeys ( ) : pass" + a = "if iter(d. keys ( )) : pass" + self.check(b, a) + + b = "[i for i in d. iterkeys( ) ]" + a = "[i for i in d. keys( ) ]" + self.check(b, a) + + def test_trailing_comment(self): + b = "d.keys() # foo" + a = "list(d.keys()) # foo" + self.check(b, a) + + b = "d.items() # foo" + a = "list(d.items()) # foo" + self.check(b, a) + + b = "d.iterkeys() # foo" + a = "iter(d.keys()) # foo" + self.check(b, a) + + b = """[i for i in d.iterkeys() # foo + ]""" + a = """[i for i in d.keys() # foo + ]""" + self.check(b, a) + def test_01(self): b = "d.keys()" a = "list(d.keys())" self.check(b, a) - def test_01a(self): b = "a[0].foo().keys()" a = "list(a[0].foo().keys())" self.check(b, a) @@ -933,13 +1019,11 @@ def test_07(self): b = "list(d.keys())" - a = b - self.check(b, a) + self.check(b, b) def test_08(self): b = "sorted(d.keys())" - a = b - self.check(b, a) + self.check(b, b) def test_09(self): b = "iter(d.keys())" @@ -1024,6 +1108,19 @@ class Test_xrange(FixerTestCase): fixer = "xrange" + def test_prefix_preservation(self): + b = """x = xrange( 10 )""" + a = """x = range( 10 )""" + self.check(b, a) + + b = """x = xrange( 1 , 10 )""" + a = """x = range( 1 , 10 )""" + self.check(b, a) + + b = """x = xrange( 0 , 10 , 2 )""" + a = """x = range( 0 , 10 , 2 )""" + self.check(b, a) + def test_1(self): b = """x = xrange(10)""" a = """x = range(10)""" @@ -1048,6 +1145,15 @@ class Test_raw_input(FixerTestCase): fixer = "raw_input" + def test_prefix_preservation(self): + b = """x = raw_input( )""" + a = """x = input( )""" + self.check(b, a) + + b = """x = raw_input( '' )""" + a = """x = input( '' )""" + self.check(b, a) + def test_1(self): b = """x = raw_input()""" a = """x = input()""" @@ -1067,6 +1173,20 @@ class Test_input(FixerTestCase): fixer = "input" + def test_prefix_preservation(self): + b = """x = input( )""" + a = """x = eval(input( ))""" + self.check(b, a) + + b = """x = input( '' )""" + a = """x = eval(input( '' ))""" + self.check(b, a) + + def test_trailing_comment(self): + b = """x = input() # foo""" + a = """x = eval(input()) # foo""" + self.check(b, a) + def test_1(self): b = """x = input()""" a = """x = eval(input())""" @@ -1837,6 +1957,15 @@ class Test_callable(FixerTestCase): fixer = "callable" + def test_prefix_preservation(self): + b = """callable( x)""" + a = """hasattr( x, '__call__')""" + self.check(b, a) + + b = """if callable(x): pass""" + a = """if hasattr(x, '__call__'): pass""" + self.check(b, a) + def test_callable_call(self): b = """callable(x)""" a = """hasattr(x, '__call__')""" @@ -1852,9 +1981,17 @@ a = """callable(x, kw=y)""" self.check(a, a) + a = """callable()""" + self.check(a, a) + class Test_filter(FixerTestCase): fixer = "filter" + def test_prefix_preservation(self): + b = """x = filter( None, 'abc' )""" + a = """x = list(filter( None, 'abc' ))""" + self.check(b, a) + def test_filter_basic(self): b = """x = filter(None, 'abc')""" a = """x = list(filter(None, 'abc'))""" @@ -1898,6 +2035,16 @@ class Test_map(FixerTestCase): fixer = "map" + def test_prefix_preservation(self): + b = """x = map( f, 'abc' )""" + a = """x = list(map( f, 'abc' ))""" + self.check(b, a) + + def test_trailing_comment(self): + b = """x = map(f, 'abc') # foo""" + a = """x = list(map(f, 'abc')) # foo""" + self.check(b, a) + def test_map_basic(self): b = """x = map(f, 'abc')""" a = """x = list(map(f, 'abc'))""" From python-checkins at python.org Mon Jul 9 07:46:04 2007 From: python-checkins at python.org (ronald.oussoren) Date: Mon, 9 Jul 2007 07:46:04 +0200 (CEST) Subject: [Python-checkins] r56203 - python/branches/release25-maint/Lib/idlelib/macosxSupport.py Message-ID: <20070709054604.F36411E4002@bag.python.org> Author: ronald.oussoren Date: Mon Jul 9 07:46:04 2007 New Revision: 56203 Modified: python/branches/release25-maint/Lib/idlelib/macosxSupport.py Log: Fixes IDLE crash on OSX: some versions of Tcl/Tk on OSX don't have a console object, avoid crashing in that case. Modified: python/branches/release25-maint/Lib/idlelib/macosxSupport.py ============================================================================== --- python/branches/release25-maint/Lib/idlelib/macosxSupport.py (original) +++ python/branches/release25-maint/Lib/idlelib/macosxSupport.py Mon Jul 9 07:46:04 2007 @@ -3,6 +3,7 @@ GUI application (as opposed to an X11 application). """ import sys +import Tkinter def runningAsOSXApp(): """ Returns True iff running from the IDLE.app bundle on OSX """ @@ -23,7 +24,11 @@ root.createcommand("::tk::mac::OpenDocument", doOpenFile) def hideTkConsole(root): - root.tk.call('console', 'hide') + try: + root.tk.call('console', 'hide') + except Tkinter.TclError: + # Some versions of the Tk framework don't have a console object + pass def overrideRootMenu(root, flist): """ From buildbot at python.org Mon Jul 9 07:53:23 2007 From: buildbot at python.org (buildbot at python.org) Date: Mon, 09 Jul 2007 05:53:23 +0000 Subject: [Python-checkins] buildbot failure in amd64 XP 2.5 Message-ID: <20070709055323.50EB21E4002@bag.python.org> The Buildbot has detected a new failure of amd64 XP 2.5. Full details are available at: http://www.python.org/dev/buildbot/all/amd64%2520XP%25202.5/builds/7 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch branches/release25-maint] HEAD Blamelist: ronald.oussoren BUILD FAILED: failed compile sincerely, -The Buildbot From python-checkins at python.org Mon Jul 9 08:02:21 2007 From: python-checkins at python.org (ronald.oussoren) Date: Mon, 9 Jul 2007 08:02:21 +0200 (CEST) Subject: [Python-checkins] r56204 - python/trunk/Lib/idlelib/macosxSupport.py Message-ID: <20070709060221.69BAA1E4016@bag.python.org> Author: ronald.oussoren Date: Mon Jul 9 08:02:21 2007 New Revision: 56204 Modified: python/trunk/Lib/idlelib/macosxSupport.py Log: Patch 1693258: Fix for duplicate "preferences" menu-OS X Modified: python/trunk/Lib/idlelib/macosxSupport.py ============================================================================== --- python/trunk/Lib/idlelib/macosxSupport.py (original) +++ python/trunk/Lib/idlelib/macosxSupport.py Mon Jul 9 08:02:21 2007 @@ -3,6 +3,7 @@ GUI application (as opposed to an X11 application). """ import sys +import Tkinter def runningAsOSXApp(): """ Returns True iff running from the IDLE.app bundle on OSX """ @@ -23,7 +24,11 @@ root.createcommand("::tk::mac::OpenDocument", doOpenFile) def hideTkConsole(root): - root.tk.call('console', 'hide') + try: + root.tk.call('console', 'hide') + except Tkinter.TclError: + # Some versions of the Tk framework don't have a console object + pass def overrideRootMenu(root, flist): """ @@ -75,32 +80,40 @@ import configDialog configDialog.ConfigDialog(root, 'Settings') + root.bind('<>', about_dialog) root.bind('<>', config_dialog) if flist: root.bind('<>', flist.close_all_callback) - for mname, entrylist in Bindings.menudefs: - menu = menudict.get(mname) - if not menu: - continue - for entry in entrylist: - if not entry: - menu.add_separator() + + ###check if Tk version >= 8.4.14; if so, use hard-coded showprefs binding + tkversion = root.tk.eval('info patchlevel') + if tkversion >= '8.4.14': + Bindings.menudefs[0] = ('application', [ + ('About IDLE', '<>'), + None, + ]) + root.createcommand('::tk::mac::ShowPreferences', config_dialog) + else: + for mname, entrylist in Bindings.menudefs: + menu = menudict.get(mname) + if not menu: + continue else: - label, eventname = entry - underline, label = prepstr(label) - accelerator = get_accelerator(Bindings.default_keydefs, + for entry in entrylist: + if not entry: + menu.add_separator() + else: + label, eventname = entry + underline, label = prepstr(label) + accelerator = get_accelerator(Bindings.default_keydefs, eventname) - def command(text=root, eventname=eventname): - text.event_generate(eventname) - menu.add_command(label=label, underline=underline, + def command(text=root, eventname=eventname): + text.event_generate(eventname) + menu.add_command(label=label, underline=underline, command=command, accelerator=accelerator) - - - - def setupApp(root, flist): """ Perform setup for the OSX application bundle. From python-checkins at python.org Mon Jul 9 08:03:48 2007 From: python-checkins at python.org (ronald.oussoren) Date: Mon, 9 Jul 2007 08:03:48 +0200 (CEST) Subject: [Python-checkins] r56205 - python/branches/release25-maint/Lib/idlelib/macosxSupport.py Message-ID: <20070709060348.18CC01E4002@bag.python.org> Author: ronald.oussoren Date: Mon Jul 9 08:03:47 2007 New Revision: 56205 Modified: python/branches/release25-maint/Lib/idlelib/macosxSupport.py Log: Patch 1693258: Fix for duplicate "preferences" menu-OS X Backport of 56204. Modified: python/branches/release25-maint/Lib/idlelib/macosxSupport.py ============================================================================== --- python/branches/release25-maint/Lib/idlelib/macosxSupport.py (original) +++ python/branches/release25-maint/Lib/idlelib/macosxSupport.py Mon Jul 9 08:03:47 2007 @@ -80,32 +80,40 @@ import configDialog configDialog.ConfigDialog(root, 'Settings') + root.bind('<>', about_dialog) root.bind('<>', config_dialog) if flist: root.bind('<>', flist.close_all_callback) - for mname, entrylist in Bindings.menudefs: - menu = menudict.get(mname) - if not menu: - continue - for entry in entrylist: - if not entry: - menu.add_separator() + + ###check if Tk version >= 8.4.14; if so, use hard-coded showprefs binding + tkversion = root.tk.eval('info patchlevel') + if tkversion >= '8.4.14': + Bindings.menudefs[0] = ('application', [ + ('About IDLE', '<>'), + None, + ]) + root.createcommand('::tk::mac::ShowPreferences', config_dialog) + else: + for mname, entrylist in Bindings.menudefs: + menu = menudict.get(mname) + if not menu: + continue else: - label, eventname = entry - underline, label = prepstr(label) - accelerator = get_accelerator(Bindings.default_keydefs, + for entry in entrylist: + if not entry: + menu.add_separator() + else: + label, eventname = entry + underline, label = prepstr(label) + accelerator = get_accelerator(Bindings.default_keydefs, eventname) - def command(text=root, eventname=eventname): - text.event_generate(eventname) - menu.add_command(label=label, underline=underline, + def command(text=root, eventname=eventname): + text.event_generate(eventname) + menu.add_command(label=label, underline=underline, command=command, accelerator=accelerator) - - - - def setupApp(root, flist): """ Perform setup for the OSX application bundle. From buildbot at python.org Mon Jul 9 08:27:33 2007 From: buildbot at python.org (buildbot at python.org) Date: Mon, 09 Jul 2007 06:27:33 +0000 Subject: [Python-checkins] buildbot warnings in x86 gentoo trunk Message-ID: <20070709062733.D51611E4002@bag.python.org> The Buildbot has detected a new failure of x86 gentoo trunk. Full details are available at: http://www.python.org/dev/buildbot/all/x86%2520gentoo%2520trunk/builds/2294 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: ronald.oussoren Build had warnings: warnings test Excerpt from the test logfile: 2 tests failed: test_urllib2 test_urllib2net ====================================================================== ERROR: test_trivial (test.test_urllib2.TrivialTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/test/test_urllib2.py", line 19, in test_trivial self.assertRaises(ValueError, urllib2.urlopen, 'bogus url') File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/unittest.py", line 329, in failUnlessRaises callableObj(*args, **kwargs) File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/urllib2.py", line 123, in urlopen _opener = build_opener() File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/urllib2.py", line 457, in build_opener opener.add_handler(klass()) File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/urllib2.py", line 666, in __init__ proxies = getproxies() File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/urllib.py", line 1267, in getproxies_environment for name, value in os.environ.items(): AttributeError: 'NoneType' object has no attribute 'environ' ====================================================================== ERROR: test_file (test.test_urllib2.HandlerTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/test/test_urllib2.py", line 613, in test_file r = h.file_open(Request(url)) File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/urllib2.py", line 1200, in file_open return self.open_local_file(req) File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/urllib2.py", line 1219, in open_local_file localfile = url2pathname(file) File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/urllib.py", line 55, in url2pathname return unquote(pathname) TypeError: 'NoneType' object is not callable ====================================================================== ERROR: test_http (test.test_urllib2.HandlerTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/test/test_urllib2.py", line 719, in test_http r.read; r.readline # wrapped MockFile methods AttributeError: addinfourl instance has no attribute 'read' ====================================================================== ERROR: test_build_opener (test.test_urllib2.MiscTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/test/test_urllib2.py", line 1025, in test_build_opener o = build_opener(FooHandler, BarHandler) File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/urllib2.py", line 457, in build_opener opener.add_handler(klass()) File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/urllib2.py", line 666, in __init__ proxies = getproxies() File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/urllib.py", line 1267, in getproxies_environment for name, value in os.environ.items(): AttributeError: 'NoneType' object has no attribute 'environ' ====================================================================== ERROR: testURLread (test.test_urllib2net.URLTimeoutTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/test/test_urllib2net.py", line 24, in testURLread f = urllib2.urlopen("http://www.python.org/") File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/urllib2.py", line 123, in urlopen _opener = build_opener() File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/urllib2.py", line 457, in build_opener opener.add_handler(klass()) File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/urllib2.py", line 666, in __init__ proxies = getproxies() File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/urllib.py", line 1267, in getproxies_environment for name, value in os.environ.items(): AttributeError: 'NoneType' object has no attribute 'environ' ====================================================================== ERROR: test_bad_address (test.test_urllib2net.urlopenNetworkTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/test/test_urllib2net.py", line 147, in test_bad_address urllib2.urlopen, "http://www.python.invalid./") File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/unittest.py", line 329, in failUnlessRaises callableObj(*args, **kwargs) File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/urllib2.py", line 123, in urlopen _opener = build_opener() File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/urllib2.py", line 457, in build_opener opener.add_handler(klass()) File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/urllib2.py", line 666, in __init__ proxies = getproxies() File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/urllib.py", line 1267, in getproxies_environment for name, value in os.environ.items(): AttributeError: 'NoneType' object has no attribute 'environ' ====================================================================== ERROR: test_basic (test.test_urllib2net.urlopenNetworkTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/test/test_urllib2net.py", line 105, in test_basic open_url = urllib2.urlopen("http://www.python.org/") File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/urllib2.py", line 123, in urlopen _opener = build_opener() File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/urllib2.py", line 457, in build_opener opener.add_handler(klass()) File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/urllib2.py", line 666, in __init__ proxies = getproxies() File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/urllib.py", line 1267, in getproxies_environment for name, value in os.environ.items(): AttributeError: 'NoneType' object has no attribute 'environ' ====================================================================== ERROR: test_geturl (test.test_urllib2net.urlopenNetworkTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/test/test_urllib2net.py", line 129, in test_geturl open_url = urllib2.urlopen(URL) File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/urllib2.py", line 123, in urlopen _opener = build_opener() File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/urllib2.py", line 457, in build_opener opener.add_handler(klass()) File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/urllib2.py", line 666, in __init__ proxies = getproxies() File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/urllib.py", line 1267, in getproxies_environment for name, value in os.environ.items(): AttributeError: 'NoneType' object has no attribute 'environ' ====================================================================== ERROR: test_info (test.test_urllib2net.urlopenNetworkTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/test/test_urllib2net.py", line 116, in test_info open_url = urllib2.urlopen("http://www.python.org/") File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/urllib2.py", line 123, in urlopen _opener = build_opener() File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/urllib2.py", line 457, in build_opener opener.add_handler(klass()) File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/urllib2.py", line 666, in __init__ proxies = getproxies() File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/urllib.py", line 1267, in getproxies_environment for name, value in os.environ.items(): AttributeError: 'NoneType' object has no attribute 'environ' ====================================================================== ERROR: test_file (test.test_urllib2net.OtherNetworkTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/test/test_urllib2net.py", line 189, in test_file self._test_urls(urls, self._extra_handlers()) File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/test/test_urllib2net.py", line 237, in _test_urls urllib2.install_opener(urllib2.build_opener(*handlers)) File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/urllib2.py", line 457, in build_opener opener.add_handler(klass()) File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/urllib2.py", line 666, in __init__ proxies = getproxies() File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/urllib.py", line 1267, in getproxies_environment for name, value in os.environ.items(): AttributeError: 'NoneType' object has no attribute 'environ' ====================================================================== ERROR: test_ftp (test.test_urllib2net.OtherNetworkTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/test/test_urllib2net.py", line 174, in test_ftp self._test_urls(urls, self._extra_handlers()) File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/test/test_urllib2net.py", line 237, in _test_urls urllib2.install_opener(urllib2.build_opener(*handlers)) File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/urllib2.py", line 457, in build_opener opener.add_handler(klass()) File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/urllib2.py", line 666, in __init__ proxies = getproxies() File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/urllib.py", line 1267, in getproxies_environment for name, value in os.environ.items(): AttributeError: 'NoneType' object has no attribute 'environ' ====================================================================== ERROR: test_http (test.test_urllib2net.OtherNetworkTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/test/test_urllib2net.py", line 201, in test_http self._test_urls(urls, self._extra_handlers()) File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/test/test_urllib2net.py", line 237, in _test_urls urllib2.install_opener(urllib2.build_opener(*handlers)) File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/urllib2.py", line 457, in build_opener opener.add_handler(klass()) File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/urllib2.py", line 666, in __init__ proxies = getproxies() File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/urllib.py", line 1267, in getproxies_environment for name, value in os.environ.items(): AttributeError: 'NoneType' object has no attribute 'environ' ====================================================================== ERROR: test_range (test.test_urllib2net.OtherNetworkTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/test/test_urllib2net.py", line 160, in test_range result = urllib2.urlopen(req) File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/urllib2.py", line 123, in urlopen _opener = build_opener() File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/urllib2.py", line 457, in build_opener opener.add_handler(klass()) File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/urllib2.py", line 666, in __init__ proxies = getproxies() File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/urllib.py", line 1267, in getproxies_environment for name, value in os.environ.items(): AttributeError: 'NoneType' object has no attribute 'environ' ====================================================================== ERROR: test_close (test.test_urllib2net.CloseSocketTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/test/test_urllib2net.py", line 76, in test_close response = urllib2.urlopen("http://www.python.org/") File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/urllib2.py", line 123, in urlopen _opener = build_opener() File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/urllib2.py", line 457, in build_opener opener.add_handler(klass()) File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/urllib2.py", line 666, in __init__ proxies = getproxies() File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/urllib.py", line 1267, in getproxies_environment for name, value in os.environ.items(): AttributeError: 'NoneType' object has no attribute 'environ' ====================================================================== ERROR: test_ftp_NoneNodefault (test.test_urllib2net.TimeoutTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/test/test_urllib2net.py", line 306, in test_ftp_NoneNodefault u = urllib2.urlopen("ftp://ftp.mirror.nl/pub/mirror/gnu/", timeout=None) File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/urllib2.py", line 123, in urlopen _opener = build_opener() File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/urllib2.py", line 457, in build_opener opener.add_handler(klass()) File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/urllib2.py", line 666, in __init__ proxies = getproxies() File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/urllib.py", line 1267, in getproxies_environment for name, value in os.environ.items(): AttributeError: 'NoneType' object has no attribute 'environ' ====================================================================== ERROR: test_ftp_NoneWithdefault (test.test_urllib2net.TimeoutTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/test/test_urllib2net.py", line 300, in test_ftp_NoneWithdefault u = urllib2.urlopen("ftp://ftp.mirror.nl/pub/mirror/gnu/", timeout=None) File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/urllib2.py", line 123, in urlopen _opener = build_opener() File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/urllib2.py", line 457, in build_opener opener.add_handler(klass()) File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/urllib2.py", line 666, in __init__ proxies = getproxies() File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/urllib.py", line 1267, in getproxies_environment for name, value in os.environ.items(): AttributeError: 'NoneType' object has no attribute 'environ' ====================================================================== ERROR: test_ftp_Value (test.test_urllib2net.TimeoutTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/test/test_urllib2net.py", line 310, in test_ftp_Value u = urllib2.urlopen("ftp://ftp.mirror.nl/pub/mirror/gnu/", timeout=60) File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/urllib2.py", line 123, in urlopen _opener = build_opener() File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/urllib2.py", line 457, in build_opener opener.add_handler(klass()) File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/urllib2.py", line 666, in __init__ proxies = getproxies() File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/urllib.py", line 1267, in getproxies_environment for name, value in os.environ.items(): AttributeError: 'NoneType' object has no attribute 'environ' ====================================================================== ERROR: test_ftp_basic (test.test_urllib2net.TimeoutTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/test/test_urllib2net.py", line 293, in test_ftp_basic u = urllib2.urlopen("ftp://ftp.mirror.nl/pub/mirror/gnu/") File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/urllib2.py", line 123, in urlopen _opener = build_opener() File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/urllib2.py", line 457, in build_opener opener.add_handler(klass()) File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/urllib2.py", line 666, in __init__ proxies = getproxies() File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/urllib.py", line 1267, in getproxies_environment for name, value in os.environ.items(): AttributeError: 'NoneType' object has no attribute 'environ' ====================================================================== ERROR: test_http_NoneNodefault (test.test_urllib2net.TimeoutTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/test/test_urllib2net.py", line 289, in test_http_NoneNodefault u = urllib2.urlopen("http://www.python.org", timeout=None) File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/urllib2.py", line 123, in urlopen _opener = build_opener() File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/urllib2.py", line 457, in build_opener opener.add_handler(klass()) File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/urllib2.py", line 666, in __init__ proxies = getproxies() File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/urllib.py", line 1267, in getproxies_environment for name, value in os.environ.items(): AttributeError: 'NoneType' object has no attribute 'environ' ====================================================================== ERROR: test_http_NoneWithdefault (test.test_urllib2net.TimeoutTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/test/test_urllib2net.py", line 279, in test_http_NoneWithdefault u = urllib2.urlopen("http://www.python.org", timeout=None) File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/urllib2.py", line 123, in urlopen _opener = build_opener() File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/urllib2.py", line 457, in build_opener opener.add_handler(klass()) File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/urllib2.py", line 666, in __init__ proxies = getproxies() File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/urllib.py", line 1267, in getproxies_environment for name, value in os.environ.items(): AttributeError: 'NoneType' object has no attribute 'environ' ====================================================================== ERROR: test_http_Value (test.test_urllib2net.TimeoutTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/test/test_urllib2net.py", line 285, in test_http_Value u = urllib2.urlopen("http://www.python.org", timeout=120) File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/urllib2.py", line 123, in urlopen _opener = build_opener() File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/urllib2.py", line 457, in build_opener opener.add_handler(klass()) File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/urllib2.py", line 666, in __init__ proxies = getproxies() File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/urllib.py", line 1267, in getproxies_environment for name, value in os.environ.items(): AttributeError: 'NoneType' object has no attribute 'environ' ====================================================================== ERROR: test_http_basic (test.test_urllib2net.TimeoutTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/test/test_urllib2net.py", line 272, in test_http_basic u = urllib2.urlopen("http://www.python.org") File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/urllib2.py", line 123, in urlopen _opener = build_opener() File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/urllib2.py", line 457, in build_opener opener.add_handler(klass()) File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/urllib2.py", line 666, in __init__ proxies = getproxies() File "/home/buildslave/python-trunk/trunk.norwitz-x86/build/Lib/urllib.py", line 1267, in getproxies_environment for name, value in os.environ.items(): AttributeError: 'NoneType' object has no attribute 'environ' make: *** [buildbottest] Error 1 sincerely, -The Buildbot From buildbot at python.org Mon Jul 9 08:29:21 2007 From: buildbot at python.org (buildbot at python.org) Date: Mon, 09 Jul 2007 06:29:21 +0000 Subject: [Python-checkins] buildbot warnings in x86 mvlgcc 2.5 Message-ID: <20070709062921.56CEF1E4002@bag.python.org> The Buildbot has detected a new failure of x86 mvlgcc 2.5. Full details are available at: http://www.python.org/dev/buildbot/all/x86%2520mvlgcc%25202.5/builds/257 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch branches/release25-maint] HEAD Blamelist: ronald.oussoren Build had warnings: warnings test Excerpt from the test logfile: Traceback (most recent call last): File "/home2/buildbot/slave/2.5.loewis-linux/build/Lib/threading.py", line 460, in __bootstrap self.run() File "/home2/buildbot/slave/2.5.loewis-linux/build/Lib/test/test_urllib2_localnet.py", line 64, in run self._RequestHandlerClass) File "/home2/buildbot/slave/2.5.loewis-linux/build/Lib/test/test_urllib2_localnet.py", line 22, in __init__ RequestHandlerClass) File "/home2/buildbot/slave/2.5.loewis-linux/build/Lib/SocketServer.py", line 330, in __init__ self.server_bind() File "/home2/buildbot/slave/2.5.loewis-linux/build/Lib/BaseHTTPServer.py", line 101, in server_bind SocketServer.TCPServer.server_bind(self) File "/home2/buildbot/slave/2.5.loewis-linux/build/Lib/SocketServer.py", line 341, in server_bind self.socket.bind(self.server_address) File "", line 1, in bind error: (98, 'Address already in use') sincerely, -The Buildbot From python-checkins at python.org Mon Jul 9 10:40:34 2007 From: python-checkins at python.org (ronald.oussoren) Date: Mon, 9 Jul 2007 10:40:34 +0200 (CEST) Subject: [Python-checkins] r56206 - in python/branches/release25-maint: Makefile.pre.in Misc/NEWS Message-ID: <20070709084034.6DECB1E4013@bag.python.org> Author: ronald.oussoren Date: Mon Jul 9 10:40:34 2007 New Revision: 56206 Modified: python/branches/release25-maint/Makefile.pre.in python/branches/release25-maint/Misc/NEWS Log: Patch 1673122: be explicit about which libtool to use, to avoid name clashes when a users install GNU libtool early in his PATH Modified: python/branches/release25-maint/Makefile.pre.in ============================================================================== --- python/branches/release25-maint/Makefile.pre.in (original) +++ python/branches/release25-maint/Makefile.pre.in Mon Jul 9 10:40:34 2007 @@ -391,7 +391,7 @@ -compatibility_version $(VERSION) \ -current_version $(VERSION); \ else \ - libtool -o $(LDLIBRARY) -dynamic $(OTHER_LIBTOOL_OPT) $(LIBRARY) \ + /usr/bin/libtool -o $(LDLIBRARY) -dynamic $(OTHER_LIBTOOL_OPT) $(LIBRARY) \ @LIBTOOL_CRUFT@ ;\ fi $(INSTALL) -d -m $(DIRMODE) \ Modified: python/branches/release25-maint/Misc/NEWS ============================================================================== --- python/branches/release25-maint/Misc/NEWS (original) +++ python/branches/release25-maint/Misc/NEWS Mon Jul 9 10:40:34 2007 @@ -57,6 +57,9 @@ - Fix test_pty.py to not hang on OS X (and theoretically other *nixes) when run in verbose mode. +- Bug #1693258: IDLE would show two "Preferences" menu's with some versions + of Tcl/Tk + Extension Modules ----------------- @@ -74,6 +77,12 @@ - Bug #1569057: Document that calling file.next() on a file open for writing has undefined behaviour. Backport of r54712. +Build +----- + +- Patch #1673122: Use an explicit path to libtool when building a framework. + This avoids picking up GNU libtool from a users PATH. + What's New in Python 2.5.1? ============================= From python-checkins at python.org Mon Jul 9 10:41:15 2007 From: python-checkins at python.org (ronald.oussoren) Date: Mon, 9 Jul 2007 10:41:15 +0200 (CEST) Subject: [Python-checkins] r56207 - python/trunk/Makefile.pre.in Message-ID: <20070709084115.70B201E4017@bag.python.org> Author: ronald.oussoren Date: Mon Jul 9 10:41:15 2007 New Revision: 56207 Modified: python/trunk/Makefile.pre.in Log: Patch 1673122: be explicit about which libtool to use, to avoid name clashes when a users install GNU libtool early in his PATH Modified: python/trunk/Makefile.pre.in ============================================================================== --- python/trunk/Makefile.pre.in (original) +++ python/trunk/Makefile.pre.in Mon Jul 9 10:41:15 2007 @@ -392,7 +392,7 @@ -compatibility_version $(VERSION) \ -current_version $(VERSION); \ else \ - libtool -o $(LDLIBRARY) -dynamic $(OTHER_LIBTOOL_OPT) $(LIBRARY) \ + /usr/bin/libtool -o $(LDLIBRARY) -dynamic $(OTHER_LIBTOOL_OPT) $(LIBRARY) \ @LIBTOOL_CRUFT@ ;\ fi $(INSTALL) -d -m $(DIRMODE) \ From buildbot at python.org Mon Jul 9 11:20:46 2007 From: buildbot at python.org (buildbot at python.org) Date: Mon, 09 Jul 2007 09:20:46 +0000 Subject: [Python-checkins] buildbot warnings in alpha Tru64 5.1 trunk Message-ID: <20070709092046.EF4281E4013@bag.python.org> The Buildbot has detected a new failure of alpha Tru64 5.1 trunk. Full details are available at: http://www.python.org/dev/buildbot/all/alpha%2520Tru64%25205.1%2520trunk/builds/1702 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: ronald.oussoren Build had warnings: warnings test Excerpt from the test logfile: sincerely, -The Buildbot From buildbot at python.org Mon Jul 9 12:32:55 2007 From: buildbot at python.org (buildbot at python.org) Date: Mon, 09 Jul 2007 10:32:55 +0000 Subject: [Python-checkins] buildbot warnings in S-390 Debian trunk Message-ID: <20070709103255.51F901E4002@bag.python.org> The Buildbot has detected a new failure of S-390 Debian trunk. Full details are available at: http://www.python.org/dev/buildbot/all/S-390%2520Debian%2520trunk/builds/1035 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: ronald.oussoren Build had warnings: warnings test Excerpt from the test logfile: 1 test failed: test_socket_ssl make: *** [buildbottest] Error 1 sincerely, -The Buildbot From python-checkins at python.org Mon Jul 9 13:33:39 2007 From: python-checkins at python.org (collin.winter) Date: Mon, 9 Jul 2007 13:33:39 +0200 (CEST) Subject: [Python-checkins] r56212 - sandbox/trunk/2to3/README Message-ID: <20070709113339.F0F311E4002@bag.python.org> Author: collin.winter Date: Mon Jul 9 13:33:39 2007 New Revision: 56212 Modified: sandbox/trunk/2to3/README Log: Add another caveat to the README description of fix_raise. Modified: sandbox/trunk/2to3/README ============================================================================== --- sandbox/trunk/2to3/README (original) +++ sandbox/trunk/2to3/README Mon Jul 9 13:33:39 2007 @@ -139,6 +139,14 @@ but since we can't detect instance-hood by syntax alone and since any client code would have to be changed as well, we don't automate this. +Another translation problem is this: :: + + t = ((E, E2), E3) + raise t + +2to3 has no way of knowing that t is a tuple, and so this code will raise an +exception at runtime since the ability to raise tuples is going away. + Notes ===== From buildbot at python.org Mon Jul 9 15:35:14 2007 From: buildbot at python.org (buildbot at python.org) Date: Mon, 09 Jul 2007 13:35:14 +0000 Subject: [Python-checkins] buildbot warnings in alpha Debian trunk Message-ID: <20070709133514.2A9C71E4002@bag.python.org> The Buildbot has detected a new failure of alpha Debian trunk. Full details are available at: http://www.python.org/dev/buildbot/all/alpha%2520Debian%2520trunk/builds/36 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: ronald.oussoren Build had warnings: warnings test Excerpt from the test logfile: 2 tests failed: test_pow test_resource ====================================================================== ERROR: test_bug705231 (test.test_pow.PowTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/pybot/buildarea/trunk.klose-debian-alpha/build/Lib/test/test_pow.py", line 109, in test_bug705231 eq(pow(a, 1.23e167), 1.0) ValueError: negative number cannot be raised to a fractional power ====================================================================== ERROR: test_fsize_enforced (test.test_resource.ResourceTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/pybot/buildarea/trunk.klose-debian-alpha/build/Lib/test/test_resource.py", line 59, in test_fsize_enforced f.close() IOError: [Errno 27] File too large make: *** [buildbottest] Error 1 sincerely, -The Buildbot From python-checkins at python.org Mon Jul 9 16:17:01 2007 From: python-checkins at python.org (collin.winter) Date: Mon, 9 Jul 2007 16:17:01 +0200 (CEST) Subject: [Python-checkins] r56214 - sandbox/trunk/2to3/README Message-ID: <20070709141701.925D51E4002@bag.python.org> Author: collin.winter Date: Mon Jul 9 16:17:01 2007 New Revision: 56214 Modified: sandbox/trunk/2to3/README Log: Update the README with more recent fixers. Modified: sandbox/trunk/2to3/README ============================================================================== --- sandbox/trunk/2to3/README (original) +++ sandbox/trunk/2to3/README Mon Jul 9 16:17:01 2007 @@ -34,6 +34,8 @@ * **fix_apply** - convert apply() calls to real function calls. +* **fix_callable** - converts callable(obj) into hasattr(obj, '__call__'). + * **fix_dict** - fix up dict.keys(), .values(), .items() and their iterator versions. @@ -41,6 +43,8 @@ * **fix_exec** - convert "exec" statements to exec() function calls. +* **fix_filter** - changes filter(F, X) into list(filter(F, X)). + * **fix_has_key** - "d.has_key(x)" -> "x in d". * **fix_input** - "input()" -> "eval(input())" (PEP 3111). @@ -49,6 +53,8 @@ * **fix_long** - remove all usage of explicit longs in favor of ints. +* **fix_map** - generally changes map(F, ...) into list(map(F, ...)). + * **fix_ne** - convert the "<>" operator to "!=". * **fix_next** - fixer for it.next() -> next(it) (PEP 3114). @@ -73,6 +79,8 @@ * **fix_tuple_params** - remove tuple parameters from function, method and lambda declarations (PEP 3113). +* **fix_unicode** - convert, e.g., u"..." to "...", unicode(x) to str(x), etc. + * **fix_xrange** - "xrange()" -> "range()". @@ -119,6 +127,15 @@ This is seen frequently when dealing with OSError. +fix_filter +'''''''''' + +The transformation is not correct if the original code depended on +filter(F, X) returning a string if X is a string (or a tuple if X is a +tuple, etc). That would require type inference, which we don't do. Python +2.6's Python 3 compatibility mode should be used to detect such cases. + + fix_has_key ''''''''''' @@ -128,6 +145,15 @@ advised to pay close attention when using this fixer. +fix_map +''''''' + +The transformation is not correct if the original code was depending on +map(F, X, Y, ...) to go on until the longest argument is exhausted, +substituting None for missing values -- like zip(), it now stops as +soon as the shortest argument is exhausted. + + fix_raise ''''''''' From python-checkins at python.org Mon Jul 9 19:44:47 2007 From: python-checkins at python.org (alexandre.vassalotti) Date: Mon, 9 Jul 2007 19:44:47 +0200 (CEST) Subject: [Python-checkins] r56218 - python/branches/cpy_merge/Modules/_picklemodule.c Message-ID: <20070709174447.DE95D1E4003@bag.python.org> Author: alexandre.vassalotti Date: Mon Jul 9 19:44:47 2007 New Revision: 56218 Modified: python/branches/cpy_merge/Modules/_picklemodule.c Log: Remove the undocumented "list-based" pickler. Start PUT index at 0. Modified: python/branches/cpy_merge/Modules/_picklemodule.c ============================================================================== --- python/branches/cpy_merge/Modules/_picklemodule.c (original) +++ python/branches/cpy_merge/Modules/_picklemodule.c Mon Jul 9 19:44:47 2007 @@ -170,8 +170,6 @@ (destructor) Pdata_dealloc, /*tp_dealloc*/ }; -#define Pdata_Check(O) ((O)->ob_type == &PdataType) - static PyObject * Pdata_New(void) { @@ -397,8 +395,7 @@ static int save(Picklerobject *, PyObject *, int); static int put2(Picklerobject *, PyObject *); -static -PyObject * +static PyObject * pickle_ErrFormat(PyObject *ErrType, char *stringformat, char *format, ...) { va_list va; @@ -540,9 +537,9 @@ self->buf_size = n; } - Py_BEGIN_ALLOW_THREADS - nbytesread = fread(self->buf, sizeof(char), n, self->fp); - Py_END_ALLOW_THREADS + Py_BEGIN_ALLOW_THREADS; + nbytesread = fread(self->buf, sizeof(char), n, self->fp); + Py_END_ALLOW_THREADS; if (nbytesread != (size_t) n) { if (feof(self->fp)) { PyErr_SetNone(PyExc_EOFError); @@ -690,12 +687,6 @@ PyOS_snprintf(s + 1, sizeof(s) - 1, "%ld\n", c_value); len = strlen(s); } - else if (Pdata_Check(self->file)) { - if (write_other(self, NULL, 0) < 0) - return -1; - PDATA_APPEND(self->file, mv, -1); - return 0; - } else { if (c_value < 256) { s[0] = BINGET; @@ -744,14 +735,6 @@ if ((p = PyDict_Size(self->memo)) < 0) goto finally; - /* Make sure memo keys are positive! */ - /* XXX Why? - * XXX And does "positive" really mean non-negative? - * XXX pickle.py starts with PUT index 0, not 1. This makes for - * XXX gratuitous differences between the pickling modules. - */ - p++; - if (!(py_ob_id = PyLong_FromVoidPtr(ob))) goto finally; @@ -774,13 +757,6 @@ PyOS_snprintf(c_str + 1, sizeof(c_str) - 1, "%d\n", p); len = strlen(c_str); } - else if (Pdata_Check(self->file)) { - if (write_other(self, NULL, 0) < 0) - return -1; - PDATA_APPEND(self->file, memo_len, -1); - res = 0; /* Job well done ;) */ - goto finally; - } else { if (p >= 256) { c_str[0] = LONG_BINPUT; @@ -1204,17 +1180,10 @@ if (self->write_func(self, c_str, len) < 0) return -1; - if (size > 128 && Pdata_Check(self->file)) { - if (write_other(self, NULL, 0) < 0) - return -1; - PDATA_APPEND(self->file, args, -1); - } - else { - if (self->write_func(self, - PyString_AS_STRING((PyStringObject *) args), - size) < 0) - return -1; - } + if (self->write_func(self, + PyString_AS_STRING((PyStringObject *) args), + size) < 0) + return -1; } if (doput) @@ -1323,15 +1292,8 @@ if (self->write_func(self, c_str, len) < 0) goto err; - if (size > 128 && Pdata_Check(self->file)) { - if (write_other(self, NULL, 0) < 0) - goto err; - PDATA_APPEND(self->file, repr, -1); - } - else { - if (self->write_func(self, PyString_AS_STRING(repr), size) < 0) - goto err; - } + if (self->write_func(self, PyString_AS_STRING(repr), size) < 0) + goto err; Py_DECREF(repr); } @@ -2405,188 +2367,33 @@ { if (self->memo) PyDict_Clear(self->memo); - Py_INCREF(Py_None); - return Py_None; -} - -static PyObject * -Pickle_getvalue(Picklerobject *self, PyObject *args) -{ - int l, i, rsize, ssize, clear = 1, lm; - long ik; - PyObject *k, *r; - char *s, *p, *have_get; - Pdata *data; - - /* Can be called by Python code or C code */ - if (args && !PyArg_ParseTuple(args, "|i:getvalue", &clear)) - return NULL; - - /* Check to make sure we are based on a list */ - if (!Pdata_Check(self->file)) { - PyErr_SetString(PicklingError, - "Attempt to getvalue() a non-list-based pickler"); - return NULL; - } - /* flush write buffer */ - if (write_other(self, NULL, 0) < 0) - return NULL; - - data = (Pdata *) self->file; - l = data->length; - - /* set up an array to hold get/put status */ - lm = PyDict_Size(self->memo); - if (lm < 0) - return NULL; - lm++; - have_get = malloc(lm); - if (have_get == NULL) - return PyErr_NoMemory(); - memset(have_get, 0, lm); - - /* Scan for gets. */ - for (rsize = 0, i = l; --i >= 0;) { - k = data->data[i]; - - if (PyString_Check(k)) - rsize += PyString_GET_SIZE(k); - - else if (PyInt_Check(k)) { /* put */ - ik = PyInt_AsLong(k); - if (ik == -1 && PyErr_Occurred()) - goto err; - if (ik >= lm || ik == 0) { - PyErr_SetString(PicklingError, "Invalid get data"); - goto err; - } - if (have_get[ik]) /* with matching get */ - rsize += ik < 256 ? 2 : 5; - } - - else if (!(PyTuple_Check(k) && - PyTuple_GET_SIZE(k) == 2 && - PyInt_Check((k = PyTuple_GET_ITEM(k, 0)))) - ) { - PyErr_SetString(PicklingError, "Unexpected data in internal list"); - goto err; - } - - else { /* put */ - ik = PyInt_AsLong(k); - if (ik == -1 && PyErr_Occurred()) - goto err; - if (ik >= lm || ik == 0) { - PyErr_SetString(PicklingError, "Invalid get data"); - return NULL; - } - have_get[ik] = 1; - rsize += ik < 256 ? 2 : 5; - } - } - - /* Now generate the result */ - r = PyString_FromStringAndSize(NULL, rsize); - if (r == NULL) - goto err; - s = PyString_AS_STRING((PyStringObject *) r); - - for (i = 0; i < l; i++) { - k = data->data[i]; - - if (PyString_Check(k)) { - ssize = PyString_GET_SIZE(k); - if (ssize) { - p = PyString_AS_STRING((PyStringObject *) k); - while (--ssize >= 0) - *s++ = *p++; - } - } - - else if (PyTuple_Check(k)) { /* get */ - ik = PyLong_AsLong(PyTuple_GET_ITEM(k, 0)); - if (ik == -1 && PyErr_Occurred()) - goto err; - if (ik < 256) { - *s++ = BINGET; - *s++ = (int) (ik & 0xff); - } - else { - *s++ = LONG_BINGET; - *s++ = (int) (ik & 0xff); - *s++ = (int) ((ik >> 8) & 0xff); - *s++ = (int) ((ik >> 16) & 0xff); - *s++ = (int) ((ik >> 24) & 0xff); - } - } - - else { /* put */ - ik = PyLong_AsLong(k); - if (ik == -1 && PyErr_Occurred()) - goto err; - - if (have_get[ik]) { /* with matching get */ - if (ik < 256) { - *s++ = BINPUT; - *s++ = (int) (ik & 0xff); - } - else { - *s++ = LONG_BINPUT; - *s++ = (int) (ik & 0xff); - *s++ = (int) ((ik >> 8) & 0xff); - *s++ = (int) ((ik >> 16) & 0xff); - *s++ = (int) ((ik >> 24) & 0xff); - } - } - } - } - - if (clear) { - PyDict_Clear(self->memo); - Pdata_clear(data, 0); - } - - free(have_get); - return r; - err: - free(have_get); - return NULL; + Py_RETURN_NONE; } static PyObject * Pickler_dump(Picklerobject *self, PyObject *args) { PyObject *ob; - int get = 0; - if (!(PyArg_ParseTuple(args, "O|i:dump", &ob, &get))) + if (!(PyArg_ParseTuple(args, "O:dump", &ob))) return NULL; if (dump(self, ob) < 0) return NULL; - if (get) - return Pickle_getvalue(self, NULL); - - /* XXX Why does dump() return self? */ - Py_INCREF(self); - return (PyObject *) self; + Py_RETURN_NONE; } - static struct PyMethodDef Pickler_methods[] = { {"dump", (PyCFunction) Pickler_dump, METH_VARARGS, PyDoc_STR("dump(object) -- " "Write an object in pickle format to the object's pickle stream")}, {"clear_memo", (PyCFunction) Pickle_clear_memo, METH_NOARGS, PyDoc_STR("clear_memo() -- Clear the picklers memo")}, - {"getvalue", (PyCFunction) Pickle_getvalue, METH_VARARGS, - PyDoc_STR("getvalue() -- Finish picking a list-based pickle")}, {NULL, NULL} /* sentinel */ }; - static Picklerobject * newPicklerobject(PyObject *file, int proto) { @@ -2623,11 +2430,9 @@ self->file = NULL; if (file) Py_INCREF(file); - else { - file = Pdata_New(); - if (file == NULL) - goto err; - } + else + goto err; + self->file = file; if (!(self->memo = PyDict_New())) @@ -2644,14 +2449,12 @@ else { self->write_func = write_other; - if (!Pdata_Check(file)) { - self->write = PyObject_GetAttr(file, write_str); - if (!self->write) { - PyErr_Clear(); - PyErr_SetString(PyExc_TypeError, - "argument must have 'write' " "attribute"); - goto err; - } + self->write = PyObject_GetAttr(file, write_str); + if (!self->write) { + PyErr_Clear(); + PyErr_SetString(PyExc_TypeError, + "argument must have 'write' " "attribute"); + goto err; } self->write_buf = (char *) PyMem_Malloc(WRITE_BUF_SIZE); @@ -2680,20 +2483,10 @@ PyObject *file = NULL; int proto = 0; - /* XXX - * The documented signature is Pickler(file, protocol=0), but this - * accepts Pickler() and Pickler(integer) too. The meaning then - * is clear as mud, undocumented, and not supported by pickle.py. - * I'm told Zope uses this, but I haven't traced into this code - * far enough to figure out what it means. - */ - if (!PyArg_ParseTuple(args, "|i:Pickler", &proto)) { - PyErr_Clear(); - proto = 0; - if (!PyArg_ParseTupleAndKeywords(args, kwds, "O|i:Pickler", - kwlist, &file, &proto)) - return NULL; - } + if (!PyArg_ParseTupleAndKeywords(args, kwds, "O|i:Pickler", + kwlist, &file, &proto)) + return NULL; + return (PyObject *) newPicklerobject(file, proto); } @@ -3219,7 +3012,6 @@ } else goto insecure; - /********************************************/ str = PyString_DecodeEscape(p, len, NULL, 0, NULL); free(s); From python-checkins at python.org Mon Jul 9 21:36:01 2007 From: python-checkins at python.org (alexandre.vassalotti) Date: Mon, 9 Jul 2007 21:36:01 +0200 (CEST) Subject: [Python-checkins] r56219 - python/branches/cpy_merge/Modules/_picklemodule.c Message-ID: <20070709193601.9ACE31E4002@bag.python.org> Author: alexandre.vassalotti Date: Mon Jul 9 21:36:01 2007 New Revision: 56219 Modified: python/branches/cpy_merge/Modules/_picklemodule.c Log: Add basic subclassing support to Pickler. Remove all module-level function. Remove a left-over from the "list-based" pickle in write_other(). Clean up Picklertype. Modified: python/branches/cpy_merge/Modules/_picklemodule.c ============================================================================== --- python/branches/cpy_merge/Modules/_picklemodule.c (original) +++ python/branches/cpy_merge/Modules/_picklemodule.c Mon Jul 9 21:36:01 2007 @@ -461,7 +461,7 @@ static int write_other(Picklerobject *self, const char *s, Py_ssize_t _n) { - PyObject *py_str = 0, *junk = 0; + PyObject *py_str, *result; int n; if (_n > INT_MAX) @@ -491,21 +491,16 @@ } } - if (self->write) { - /* object with write method */ - ARG_TUP(self, py_str); - if (self->arg) { - junk = PyObject_Call(self->write, self->arg, NULL); - FREE_ARG_TUP(self); - } - if (junk) - Py_DECREF(junk); - else - return -1; + /* object with write method */ + ARG_TUP(self, py_str); + if (self->arg) { + result = PyObject_Call(self->write, self->arg, NULL); + FREE_ARG_TUP(self); } - else - PDATA_PUSH(self->file, py_str, -1); + if (result == NULL) + return -1; + Py_DECREF(result); self->buf_size = 0; return n; } @@ -2394,10 +2389,17 @@ {NULL, NULL} /* sentinel */ }; -static Picklerobject * -newPicklerobject(PyObject *file, int proto) +static PyObject * +Pickler_new(PyTypeObject *type, PyObject *args, PyObject *kwds) { + static char *kwlist[] = { "file", "protocol", NULL }; Picklerobject *self; + PyObject *file; + int proto = 0; + + if (!PyArg_ParseTupleAndKeywords(args, kwds, "O|i:Pickler", + kwlist, &file, &proto)) + return NULL; if (proto < 0) proto = HIGHEST_PROTOCOL; @@ -2411,86 +2413,66 @@ self = PyObject_GC_New(Picklerobject, &Picklertype); if (self == NULL) return NULL; - self->proto = proto; - self->bin = proto > 0; - self->fp = NULL; - self->write = NULL; - self->memo = NULL; - self->arg = NULL; - self->pers_func = NULL; - self->inst_pers_func = NULL; - self->write_buf = NULL; - self->fast = 0; - self->nesting = 0; - self->fast_container = 0; - self->fast_memo = NULL; - self->buf_size = 0; - self->dispatch_table = NULL; - - self->file = NULL; - if (file) - Py_INCREF(file); - else - goto err; - self->file = file; + self->proto = proto; + self->bin = proto > 0; + self->fp = NULL; + self->write = NULL; + self->memo = NULL; + self->arg = NULL; + self->pers_func = NULL; + self->inst_pers_func = NULL; + self->write_buf = NULL; + self->fast = 0; + self->nesting = 0; + self->fast_container = 0; + self->fast_memo = NULL; + self->buf_size = 0; + self->dispatch_table = NULL; + + self->memo = PyDict_New(); + if (self->memo == NULL) + goto error; - if (!(self->memo = PyDict_New())) - goto err; + Py_INCREF(dispatch_table); + self->dispatch_table = dispatch_table; + Py_INCREF(file); if (PyFile_Check(file)) { self->fp = PyFile_AsFile(file); if (self->fp == NULL) { PyErr_SetString(PyExc_ValueError, "I/O operation on closed file"); - goto err; + goto io_error; } self->write_func = write_file; } else { - self->write_func = write_other; - self->write = PyObject_GetAttr(file, write_str); - if (!self->write) { + if (self->write == NULL) { PyErr_Clear(); PyErr_SetString(PyExc_TypeError, "argument must have 'write' " "attribute"); - goto err; + goto io_error; } self->write_buf = (char *) PyMem_Malloc(WRITE_BUF_SIZE); if (self->write_buf == NULL) { PyErr_NoMemory(); - goto err; + goto io_error; } + self->write_func = write_other; } - self->dispatch_table = dispatch_table; - Py_INCREF(dispatch_table); PyObject_GC_Track(self); + return (PyObject *) self; - return self; - - err: + io_error: + Py_DECREF(file); + error: Py_DECREF(self); return NULL; } - -static PyObject * -get_Pickler(PyObject *self, PyObject *args, PyObject *kwds) -{ - static char *kwlist[] = { "file", "protocol", NULL }; - PyObject *file = NULL; - int proto = 0; - - if (!PyArg_ParseTupleAndKeywords(args, kwds, "O|i:Pickler", - kwlist, &file, &proto)) - return NULL; - - return (PyObject *) newPicklerobject(file, proto); -} - - static void Pickler_dealloc(Picklerobject *self) { @@ -2499,7 +2481,6 @@ Py_XDECREF(self->memo); Py_XDECREF(self->fast_memo); Py_XDECREF(self->arg); - Py_XDECREF(self->file); Py_XDECREF(self->pers_func); Py_XDECREF(self->inst_pers_func); Py_XDECREF(self->dispatch_table); @@ -2514,7 +2495,6 @@ Py_VISIT(self->memo); Py_VISIT(self->fast_memo); Py_VISIT(self->arg); - Py_VISIT(self->file); Py_VISIT(self->pers_func); Py_VISIT(self->inst_pers_func); Py_VISIT(self->dispatch_table); @@ -2528,7 +2508,6 @@ Py_CLEAR(self->memo); Py_CLEAR(self->fast_memo); Py_CLEAR(self->arg); - Py_CLEAR(self->file); Py_CLEAR(self->pers_func); Py_CLEAR(self->inst_pers_func); Py_CLEAR(self->dispatch_table); @@ -2624,40 +2603,73 @@ {NULL} }; -PyDoc_STRVAR(Picklertype__doc__, "Objects that know how to pickle objects\n"); +PyDoc_STRVAR(Pickler_doc, +"Pickler(file, protocol=0) -- Create a pickler.\n" +"\n" +"This takes a file-like object for writing a pickle data stream.\n" +"The optional proto argument tells the pickler to use the given\n" +"protocol; supported protocols are 0, 1, 2. The default\n" +"protocol is 0, to be backwards compatible. (Protocol 0 is the\n" +"only protocol that can be written to a file opened in text\n" +"mode and read back successfully. When using a protocol higher\n" +"than 0, make sure the file is opened in binary mode, both when\n" +"pickling and unpickling.)\n" +"\n" +"Protocol 1 is more efficient than protocol 0; protocol 2 is\n" +"more efficient than protocol 1.\n" +"\n" +"Specifying a negative protocol version selects the highest\n" +"protocol version supported. The higher the protocol used, the\n" +"more recent the version of Python needed to read the pickle\n" +"produced.\n" +"\n" +"The file parameter must have a write() method that accepts a single\n" +"string argument. It can thus be an open file object, a StringIO\n" +"object, or any other custom object that meets this interface.\n"); + static PyTypeObject Picklertype = { PyObject_HEAD_INIT(NULL) - 0, /*ob_size */ - "pickle.Pickler", /*tp_name */ - sizeof(Picklerobject), /*tp_basicsize */ - 0, - (destructor) Pickler_dealloc, /* tp_dealloc */ - 0, /* tp_print */ - 0, /* tp_getattr */ - 0, /* tp_setattr */ - 0, /* tp_compare */ - 0, /* tp_repr */ - 0, /* tp_as_number */ - 0, /* tp_as_sequence */ - 0, /* tp_as_mapping */ - 0, /* tp_hash */ - 0, /* tp_call */ - 0, /* tp_str */ - PyObject_GenericGetAttr, /* tp_getattro */ - PyObject_GenericSetAttr, /* tp_setattro */ - 0, /* tp_as_buffer */ + 0, /*ob_size*/ + "_pickle.Pickler" , /*tp_name*/ + sizeof(Picklerobject), /*tp_basicsize*/ + 0, /*tp_itemsize*/ + (destructor)Pickler_dealloc, /*tp_dealloc*/ + 0, /*tp_print*/ + 0, /*tp_getattr*/ + 0, /*tp_setattr*/ + 0, /*tp_compare*/ + 0, /*tp_repr*/ + 0, /*tp_as_number*/ + 0, /*tp_as_sequence*/ + 0, /*tp_as_mapping*/ + 0, /*tp_hash*/ + 0, /*tp_call*/ + 0, /*tp_str*/ + PyObject_GenericGetAttr, /*tp_getattro*/ + PyObject_GenericSetAttr, /*tp_setattro*/ + 0, /*tp_as_buffer*/ Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE | Py_TPFLAGS_HAVE_GC, - Picklertype__doc__, /* tp_doc */ - (traverseproc) Pickler_traverse, /* tp_traverse */ - (inquiry) Pickler_clear, /* tp_clear */ - 0, /* tp_richcompare */ - 0, /* tp_weaklistoffset */ - 0, /* tp_iter */ - 0, /* tp_iternext */ - Pickler_methods, /* tp_methods */ - Pickler_members, /* tp_members */ - Pickler_getsets, /* tp_getset */ + Pickler_doc, /*tp_doc*/ + (traverseproc)Pickler_traverse, /*tp_traverse*/ + (inquiry)Pickler_clear, /*tp_clear*/ + 0, /*tp_richcompare*/ + 0, /*tp_weaklistoffset*/ + 0, /*tp_iter*/ + 0, /*tp_iternext*/ + Pickler_methods, /*tp_methods*/ + Pickler_members, /*tp_members*/ + Pickler_getsets, /*tp_getset*/ + 0, /*tp_base*/ + 0, /*tp_dict*/ + 0, /*tp_descr_get*/ + 0, /*tp_descr_set*/ + 0, /*tp_dictoffset*/ + 0, /*tp_init*/ + 0, /*tp_alloc*/ + Pickler_new, /*tp_new*/ + 0, /*tp_free*/ + 0, /*tp_is_gc*/ }; static PyObject * @@ -5008,56 +5020,6 @@ return -1; } -/* --------------------------------------------------------------------------- - * Module-level functions. - */ - -/* dump(obj, file, protocol=0). */ -static PyObject * -cpm_dump(PyObject *self, PyObject *args, PyObject *kwds) -{ - static char *kwlist[] = { "obj", "file", "protocol", NULL }; - PyObject *ob, *file, *res = NULL; - Picklerobject *pickler = 0; - int proto = 0; - - if (!(PyArg_ParseTupleAndKeywords(args, kwds, "OO|i", kwlist, - &ob, &file, &proto))) - goto finally; - - if (!(pickler = newPicklerobject(file, proto))) - goto finally; - - if (dump(pickler, ob) < 0) - goto finally; - - Py_INCREF(Py_None); - res = Py_None; - - finally: - Py_XDECREF(pickler); - - return res; -} - -/* load(fileobj). */ -static PyObject * -cpm_load(PyObject *self, PyObject *ob) -{ - Unpicklerobject *unpickler = 0; - PyObject *res = NULL; - - if (!(unpickler = newUnpicklerobject(ob))) - goto finally; - - res = load(unpickler); - - finally: - Py_XDECREF(unpickler); - - return res; -} - PyDoc_STRVAR(Unpicklertype__doc__, "Objects that know how to unpickle"); static PyTypeObject Unpicklertype = { @@ -5087,47 +5049,6 @@ (inquiry) Unpickler_clear, /* tp_clear */ }; -static struct PyMethodDef pickle_methods[] = { - {"dump", (PyCFunction) cpm_dump, METH_VARARGS | METH_KEYWORDS, - PyDoc_STR("dump(obj, file, protocol=0) -- " - "Write an object in pickle format to the given file.\n" - "\n" - "See the Pickler docstring for the meaning of optional argument proto.") - }, - - {"load", (PyCFunction) cpm_load, METH_O, - PyDoc_STR("load(file) -- Load a pickle from the given file")}, - - {"Pickler", (PyCFunction) get_Pickler, METH_VARARGS | METH_KEYWORDS, - PyDoc_STR("Pickler(file, protocol=0) -- Create a pickler.\n" - "\n" - "This takes a file-like object for writing a pickle data stream.\n" - "The optional proto argument tells the pickler to use the given\n" - "protocol; supported protocols are 0, 1, 2. The default\n" - "protocol is 0, to be backwards compatible. (Protocol 0 is the\n" - "only protocol that can be written to a file opened in text\n" - "mode and read back successfully. When using a protocol higher\n" - "than 0, make sure the file is opened in binary mode, both when\n" - "pickling and unpickling.)\n" - "\n" - "Protocol 1 is more efficient than protocol 0; protocol 2 is\n" - "more efficient than protocol 1.\n" - "\n" - "Specifying a negative protocol version selects the highest\n" - "protocol version supported. The higher the protocol used, the\n" - "more recent the version of Python needed to read the pickle\n" - "produced.\n" - "\n" - "The file parameter must have a write() method that accepts a single\n" - "string argument. It can thus be an open file object, a StringIO\n" - "object, or any other custom object that meets this interface.\n") - }, - - {"Unpickler", (PyCFunction) get_Unpickler, METH_O, - PyDoc_STR("Unpickler(file) -- Create an unpickler.")}, - {NULL, NULL} -}; - static int init_stuff(PyObject *module_dict) { @@ -5282,12 +5203,12 @@ return; /* Create the module and add the functions */ - m = Py_InitModule4("_pickle", pickle_methods, - pickle_module_documentation, - (PyObject *) NULL, PYTHON_API_VERSION); + m = Py_InitModule3("_pickle", NULL, pickle_module_documentation); if (m == NULL) return; + PyModule_AddObject(m, "Pickler", (PyObject *)&Picklertype); + /* Add some symbolic constants to the module */ d = PyModule_GetDict(m); v = PyString_FromString(rev); From python-checkins at python.org Mon Jul 9 21:37:19 2007 From: python-checkins at python.org (alexandre.vassalotti) Date: Mon, 9 Jul 2007 21:37:19 +0200 (CEST) Subject: [Python-checkins] r56220 - python/branches/cpy_merge/Modules/_picklemodule.c Message-ID: <20070709193719.E07C91E4002@bag.python.org> Author: alexandre.vassalotti Date: Mon Jul 9 21:37:19 2007 New Revision: 56220 Modified: python/branches/cpy_merge/Modules/_picklemodule.c Log: Rename Picklertype to Pickler_Type. Rename Picklerobject to PicklerObject. Modified: python/branches/cpy_merge/Modules/_picklemodule.c ============================================================================== --- python/branches/cpy_merge/Modules/_picklemodule.c (original) +++ python/branches/cpy_merge/Modules/_picklemodule.c Mon Jul 9 21:37:19 2007 @@ -335,7 +335,7 @@ } \ } -typedef struct Picklerobject { +typedef struct PicklerObject { PyObject_HEAD FILE *fp; PyObject *write; @@ -353,19 +353,19 @@ int fast; /* Fast mode doesn't save in memo, don't use if circ ref */ int nesting; - int (*write_func) (struct Picklerobject *, const char *, Py_ssize_t); + int (*write_func) (struct PicklerObject *, const char *, Py_ssize_t); char *write_buf; int buf_size; PyObject *dispatch_table; int fast_container; /* count nested container dumps */ PyObject *fast_memo; -} Picklerobject; +} PicklerObject; #ifndef PY_CPICKLE_FAST_LIMIT #define PY_CPICKLE_FAST_LIMIT 50 #endif -static PyTypeObject Picklertype; +static PyTypeObject Pickler_Type; typedef struct Unpicklerobject { PyObject_HEAD @@ -392,8 +392,8 @@ static PyTypeObject Unpicklertype; /* Forward decls that need the above structs */ -static int save(Picklerobject *, PyObject *, int); -static int put2(Picklerobject *, PyObject *); +static int save(PicklerObject *, PyObject *, int); +static int put2(PicklerObject *, PyObject *); static PyObject * pickle_ErrFormat(PyObject *ErrType, char *stringformat, char *format, ...) @@ -433,7 +433,7 @@ } static int -write_file(Picklerobject *self, const char *s, Py_ssize_t n) +write_file(PicklerObject *self, const char *s, Py_ssize_t n) { size_t nbyteswritten; @@ -459,7 +459,7 @@ } static int -write_other(Picklerobject *self, const char *s, Py_ssize_t _n) +write_other(PicklerObject *self, const char *s, Py_ssize_t _n) { PyObject *py_str, *result; int n; @@ -654,7 +654,7 @@ static int -get(Picklerobject *self, PyObject * id) +get(PicklerObject *self, PyObject * id) { PyObject *value, *mv; long c_value; @@ -706,7 +706,7 @@ static int -put(Picklerobject *self, PyObject *ob) +put(PicklerObject *self, PyObject *ob) { if (ob->ob_refcnt < 2 || self->fast) return 0; @@ -716,7 +716,7 @@ static int -put2(Picklerobject *self, PyObject *ob) +put2(PicklerObject *self, PyObject *ob) { char c_str[30]; int p; @@ -838,7 +838,7 @@ static int -fast_save_enter(Picklerobject *self, PyObject *obj) +fast_save_enter(PicklerObject *self, PyObject *obj) { /* if fast_container < 0, we're doing an error exit. */ if (++self->fast_container >= PY_CPICKLE_FAST_LIMIT) { @@ -873,7 +873,7 @@ } static int -fast_save_leave(Picklerobject *self, PyObject *obj) +fast_save_leave(PicklerObject *self, PyObject *obj) { if (self->fast_container-- >= PY_CPICKLE_FAST_LIMIT) { PyObject *key = PyLong_FromVoidPtr(obj); @@ -889,7 +889,7 @@ } static int -save_none(Picklerobject *self, PyObject *args) +save_none(PicklerObject *self, PyObject *args) { static char none = NONE; if (self->write_func(self, &none, 1) < 0) @@ -899,7 +899,7 @@ } static int -save_bool(Picklerobject *self, PyObject *args) +save_bool(PicklerObject *self, PyObject *args) { static const char *buf[2] = { FALSE, TRUE }; static char len[2] = { sizeof(FALSE) - 1, sizeof(TRUE) - 1 }; @@ -916,7 +916,7 @@ } static int -save_int(Picklerobject *self, long l) +save_int(PicklerObject *self, long l) { char c_str[32]; int len = 0; @@ -965,7 +965,7 @@ static int -save_long(Picklerobject *self, PyObject *args) +save_long(PicklerObject *self, PyObject *args) { Py_ssize_t size; int res = -1; @@ -1092,7 +1092,7 @@ static int -save_float(Picklerobject *self, PyObject *args) +save_float(PicklerObject *self, PyObject *args) { double x = PyFloat_AS_DOUBLE((PyFloatObject *) args); @@ -1120,7 +1120,7 @@ static int -save_string(Picklerobject *self, PyObject *args, int doput) +save_string(PicklerObject *self, PyObject *args, int doput) { int size, len; PyObject *repr = 0; @@ -1234,7 +1234,7 @@ static int -save_unicode(Picklerobject *self, PyObject *args, int doput) +save_unicode(PicklerObject *self, PyObject *args, int doput) { Py_ssize_t size, len; PyObject *repr = 0; @@ -1307,7 +1307,7 @@ /* A helper for save_tuple. Push the len elements in tuple t on the stack. */ static int -store_tuple_elements(Picklerobject *self, PyObject *t, int len) +store_tuple_elements(PicklerObject *self, PyObject *t, int len) { int i; int res = -1; /* guilty until proved innocent */ @@ -1335,7 +1335,7 @@ * magic so that it works in all cases. IOW, this is a long routine. */ static int -save_tuple(Picklerobject *self, PyObject *args) +save_tuple(PicklerObject *self, PyObject *args) { PyObject *py_tuple_id = NULL; int len, i; @@ -1448,7 +1448,7 @@ * Returns 0 on success, <0 on error. */ static int -batch_list(Picklerobject *self, PyObject *iter) +batch_list(PicklerObject *self, PyObject *iter) { PyObject *obj; PyObject *slice[BATCHSIZE]; @@ -1523,7 +1523,7 @@ } static int -save_list(Picklerobject *self, PyObject *args) +save_list(PicklerObject *self, PyObject *args) { int res = -1; char s[3]; @@ -1587,7 +1587,7 @@ * ugly to bear. */ static int -batch_dict(Picklerobject *self, PyObject *iter) +batch_dict(PicklerObject *self, PyObject *iter) { PyObject *p; PyObject *slice[BATCHSIZE]; @@ -1680,7 +1680,7 @@ } static int -save_dict(Picklerobject *self, PyObject *args) +save_dict(PicklerObject *self, PyObject *args) { int res = -1; char s[3]; @@ -1736,7 +1736,7 @@ static int -save_global(Picklerobject *self, PyObject *args, PyObject *name) +save_global(PicklerObject *self, PyObject *args, PyObject *name) { PyObject *global_name = 0, *module = 0, *mod = 0, *klass = 0; char *name_str, *module_str; @@ -1874,7 +1874,7 @@ } static int -save_pers(Picklerobject *self, PyObject *args, PyObject *f) +save_pers(PicklerObject *self, PyObject *args, PyObject *f) { PyObject *pid = 0; int size, res = -1; @@ -1936,7 +1936,7 @@ * appropriate __reduce__ method for ob. */ static int -save_reduce(Picklerobject *self, PyObject *args, PyObject *ob) +save_reduce(PicklerObject *self, PyObject *args, PyObject *ob) { PyObject *callable; PyObject *argtup; @@ -2080,7 +2080,7 @@ } static int -save(Picklerobject *self, PyObject *args, int pers_save) +save(PicklerObject *self, PyObject *args, int pers_save) { PyTypeObject *type; PyObject *py_ob_id = 0, *__reduce__ = 0, *t = 0; @@ -2331,7 +2331,7 @@ static int -dump(Picklerobject *self, PyObject *args) +dump(PicklerObject *self, PyObject *args) { static char stop = STOP; @@ -2358,7 +2358,7 @@ } static PyObject * -Pickle_clear_memo(Picklerobject *self, PyObject *args) +Pickle_clear_memo(PicklerObject *self, PyObject *args) { if (self->memo) PyDict_Clear(self->memo); @@ -2367,7 +2367,7 @@ } static PyObject * -Pickler_dump(Picklerobject *self, PyObject *args) +Pickler_dump(PicklerObject *self, PyObject *args) { PyObject *ob; @@ -2393,7 +2393,7 @@ Pickler_new(PyTypeObject *type, PyObject *args, PyObject *kwds) { static char *kwlist[] = { "file", "protocol", NULL }; - Picklerobject *self; + PicklerObject *self; PyObject *file; int proto = 0; @@ -2410,7 +2410,7 @@ return NULL; } - self = PyObject_GC_New(Picklerobject, &Picklertype); + self = PyObject_GC_New(PicklerObject, &Pickler_Type); if (self == NULL) return NULL; @@ -2474,7 +2474,7 @@ } static void -Pickler_dealloc(Picklerobject *self) +Pickler_dealloc(PicklerObject *self) { PyObject_GC_UnTrack(self); Py_XDECREF(self->write); @@ -2489,7 +2489,7 @@ } static int -Pickler_traverse(Picklerobject *self, visitproc visit, void *arg) +Pickler_traverse(PicklerObject *self, visitproc visit, void *arg) { Py_VISIT(self->write); Py_VISIT(self->memo); @@ -2502,7 +2502,7 @@ } static int -Pickler_clear(Picklerobject *self) +Pickler_clear(PicklerObject *self) { Py_CLEAR(self->write); Py_CLEAR(self->memo); @@ -2515,7 +2515,7 @@ } static PyObject * -Pickler_get_pers_func(Picklerobject *p) +Pickler_get_pers_func(PicklerObject *p) { if (p->pers_func == NULL) PyErr_SetString(PyExc_AttributeError, "persistent_id"); @@ -2525,7 +2525,7 @@ } static int -Pickler_set_pers_func(Picklerobject *p, PyObject *v) +Pickler_set_pers_func(PicklerObject *p, PyObject *v) { if (v == NULL) { PyErr_SetString(PyExc_TypeError, @@ -2539,7 +2539,7 @@ } static int -Pickler_set_inst_pers_func(Picklerobject *p, PyObject *v) +Pickler_set_inst_pers_func(PicklerObject *p, PyObject *v) { if (v == NULL) { PyErr_SetString(PyExc_TypeError, @@ -2553,7 +2553,7 @@ } static PyObject * -Pickler_get_memo(Picklerobject *p) +Pickler_get_memo(PicklerObject *p) { if (p->memo == NULL) PyErr_SetString(PyExc_AttributeError, "memo"); @@ -2563,7 +2563,7 @@ } static int -Pickler_set_memo(Picklerobject *p, PyObject *v) +Pickler_set_memo(PicklerObject *p, PyObject *v) { if (v == NULL) { PyErr_SetString(PyExc_TypeError, @@ -2581,7 +2581,7 @@ } static PyObject * -Pickler_get_error(Picklerobject *p) +Pickler_get_error(PicklerObject *p) { /* why is this an attribute on the Pickler? */ Py_INCREF(PicklingError); @@ -2589,8 +2589,8 @@ } static PyMemberDef Pickler_members[] = { - {"binary", T_INT, offsetof(Picklerobject, bin)}, - {"fast", T_INT, offsetof(Picklerobject, fast)}, + {"binary", T_INT, offsetof(PicklerObject, bin)}, + {"fast", T_INT, offsetof(PicklerObject, fast)}, {NULL} }; @@ -2628,11 +2628,11 @@ "object, or any other custom object that meets this interface.\n"); -static PyTypeObject Picklertype = { +static PyTypeObject Pickler_Type = { PyObject_HEAD_INIT(NULL) 0, /*ob_size*/ "_pickle.Pickler" , /*tp_name*/ - sizeof(Picklerobject), /*tp_basicsize*/ + sizeof(PicklerObject), /*tp_basicsize*/ 0, /*tp_itemsize*/ (destructor)Pickler_dealloc, /*tp_dealloc*/ 0, /*tp_print*/ @@ -5056,7 +5056,7 @@ if (PyType_Ready(&Unpicklertype) < 0) return -1; - if (PyType_Ready(&Picklertype) < 0) + if (PyType_Ready(&Pickler_Type) < 0) return -1; INIT_STR(__class__); @@ -5189,7 +5189,7 @@ PyObject *format_version; PyObject *compatible_formats; - Picklertype.ob_type = &PyType_Type; + Pickler_Type.ob_type = &PyType_Type; Unpicklertype.ob_type = &PyType_Type; PdataType.ob_type = &PyType_Type; @@ -5207,7 +5207,7 @@ if (m == NULL) return; - PyModule_AddObject(m, "Pickler", (PyObject *)&Picklertype); + PyModule_AddObject(m, "Pickler", (PyObject *)&Pickler_Type); /* Add some symbolic constants to the module */ d = PyModule_GetDict(m); From python-checkins at python.org Mon Jul 9 21:56:36 2007 From: python-checkins at python.org (alexandre.vassalotti) Date: Mon, 9 Jul 2007 21:56:36 +0200 (CEST) Subject: [Python-checkins] r56221 - python/branches/cpy_merge/Modules/_picklemodule.c Message-ID: <20070709195636.3C5331E4007@bag.python.org> Author: alexandre.vassalotti Date: Mon Jul 9 21:56:35 2007 New Revision: 56221 Modified: python/branches/cpy_merge/Modules/_picklemodule.c Log: Rename goto labels "err" to "error". Modified: python/branches/cpy_merge/Modules/_picklemodule.c ============================================================================== --- python/branches/cpy_merge/Modules/_picklemodule.c (original) +++ python/branches/cpy_merge/Modules/_picklemodule.c Mon Jul 9 21:56:35 2007 @@ -1137,17 +1137,17 @@ return -1; if ((len = PyString_Size(repr)) < 0) - goto err; + goto error; repr_str = PyString_AS_STRING((PyStringObject *) repr); if (self->write_func(self, &string, 1) < 0) - goto err; + goto error; if (self->write_func(self, repr_str, len) < 0) - goto err; + goto error; if (self->write_func(self, "\n", 1) < 0) - goto err; + goto error; Py_XDECREF(repr); } @@ -1187,7 +1187,7 @@ return 0; - err: + error: Py_XDECREF(repr); return -1; } @@ -1253,17 +1253,17 @@ return -1; if ((len = PyString_Size(repr)) < 0) - goto err; + goto error; repr_str = PyString_AS_STRING((PyStringObject *) repr); if (self->write_func(self, &string, 1) < 0) - goto err; + goto error; if (self->write_func(self, repr_str, len) < 0) - goto err; + goto error; if (self->write_func(self, "\n", 1) < 0) - goto err; + goto error; Py_XDECREF(repr); } @@ -1275,7 +1275,7 @@ return -1; if ((size = PyString_Size(repr)) < 0) - goto err; + goto error; if (size > INT_MAX) return -1; /* string too large */ @@ -1285,10 +1285,10 @@ len = 5; if (self->write_func(self, c_str, len) < 0) - goto err; + goto error; if (self->write_func(self, PyString_AS_STRING(repr), size) < 0) - goto err; + goto error; Py_DECREF(repr); } @@ -1299,7 +1299,7 @@ return 0; - err: + error: Py_XDECREF(repr); return -1; } @@ -4834,10 +4834,10 @@ self->find_class = NULL; if (!(self->memo = PyDict_New())) - goto err; + goto error; if (!self->stack) - goto err; + goto error; Py_INCREF(f); self->file = f; @@ -4847,7 +4847,7 @@ self->fp = PyFile_AsFile(f); if (self->fp == NULL) { PyErr_SetString(PyExc_ValueError, "I/O operation on closed file"); - goto err; + goto error; } self->read_func = read_file; self->readline_func = readline_file; @@ -4864,14 +4864,14 @@ PyErr_SetString(PyExc_TypeError, "argument must have 'read' and " "'readline' attributes"); - goto err; + goto error; } } PyObject_GC_Track(self); return self; - err: + error: Py_DECREF((PyObject *) self); return NULL; } From python-checkins at python.org Mon Jul 9 22:00:09 2007 From: python-checkins at python.org (alexandre.vassalotti) Date: Mon, 9 Jul 2007 22:00:09 +0200 (CEST) Subject: [Python-checkins] r56222 - python/branches/cpy_merge/Modules/_picklemodule.c Message-ID: <20070709200009.BCED41E400C@bag.python.org> Author: alexandre.vassalotti Date: Mon Jul 9 22:00:09 2007 New Revision: 56222 Modified: python/branches/cpy_merge/Modules/_picklemodule.c Log: Rename Unpicklerobject to UnpicklerObject. Rename Unpicklertype to Unpickler_Type. Modified: python/branches/cpy_merge/Modules/_picklemodule.c ============================================================================== --- python/branches/cpy_merge/Modules/_picklemodule.c (original) +++ python/branches/cpy_merge/Modules/_picklemodule.c Mon Jul 9 22:00:09 2007 @@ -367,7 +367,7 @@ static PyTypeObject Pickler_Type; -typedef struct Unpicklerobject { +typedef struct UnpicklerObject { PyObject_HEAD FILE *fp; PyObject *file; @@ -382,14 +382,14 @@ int *marks; int num_marks; int marks_size; - Py_ssize_t(*read_func) (struct Unpicklerobject *, char **, Py_ssize_t); - Py_ssize_t(*readline_func) (struct Unpicklerobject *, char **); + Py_ssize_t(*read_func) (struct UnpicklerObject *, char **, Py_ssize_t); + Py_ssize_t(*readline_func) (struct UnpicklerObject *, char **); int buf_size; char *buf; PyObject *find_class; -} Unpicklerobject; +} UnpicklerObject; -static PyTypeObject Unpicklertype; +static PyTypeObject Unpickler_Type; /* Forward decls that need the above structs */ static int save(PicklerObject *, PyObject *, int); @@ -507,7 +507,7 @@ static Py_ssize_t -read_file(Unpicklerobject *self, char **s, Py_ssize_t n) +read_file(UnpicklerObject *self, char **s, Py_ssize_t n) { size_t nbytesread; @@ -552,7 +552,7 @@ static Py_ssize_t -readline_file(Unpicklerobject *self, char **s) +readline_file(UnpicklerObject *self, char **s) { int i; @@ -591,7 +591,7 @@ } static Py_ssize_t -read_other(Unpicklerobject *self, char **s, Py_ssize_t n) +read_other(UnpicklerObject *self, char **s, Py_ssize_t n) { PyObject *bytes, *str = 0; @@ -616,7 +616,7 @@ static Py_ssize_t -readline_other(Unpicklerobject *self, char **s) +readline_other(UnpicklerObject *self, char **s) { PyObject *str; Py_ssize_t str_size; @@ -2705,7 +2705,7 @@ } static int -marker(Unpicklerobject *self) +marker(UnpicklerObject *self) { if (self->num_marks < 1) { PyErr_SetString(UnpicklingError, "could not find MARK"); @@ -2717,7 +2717,7 @@ static int -load_none(Unpicklerobject *self) +load_none(UnpicklerObject *self) { PDATA_APPEND(self->stack, Py_None, -1); return 0; @@ -2731,7 +2731,7 @@ } static int -load_int(Unpicklerobject *self) +load_int(UnpicklerObject *self) { PyObject *py_int = 0; char *endptr, *s; @@ -2781,7 +2781,7 @@ } static int -load_bool(Unpicklerobject *self, PyObject *boolean) +load_bool(UnpicklerObject *self, PyObject *boolean) { assert(boolean == Py_True || boolean == Py_False); PDATA_APPEND(self->stack, boolean, -1); @@ -2817,7 +2817,7 @@ static int -load_binintx(Unpicklerobject *self, char *s, int x) +load_binintx(UnpicklerObject *self, char *s, int x) { PyObject *py_int = 0; long l; @@ -2833,7 +2833,7 @@ static int -load_binint(Unpicklerobject *self) +load_binint(UnpicklerObject *self) { char *s; @@ -2845,7 +2845,7 @@ static int -load_binint1(Unpicklerobject *self) +load_binint1(UnpicklerObject *self) { char *s; @@ -2857,7 +2857,7 @@ static int -load_binint2(Unpicklerobject *self) +load_binint2(UnpicklerObject *self) { char *s; @@ -2868,7 +2868,7 @@ } static int -load_long(Unpicklerobject *self) +load_long(UnpicklerObject *self) { PyObject *l = 0; char *end, *s; @@ -2898,7 +2898,7 @@ * data following. */ static int -load_counted_long(Unpicklerobject *self, int size) +load_counted_long(UnpicklerObject *self, int size) { Py_ssize_t i; char *nbytes; @@ -2937,7 +2937,7 @@ } static int -load_float(Unpicklerobject *self) +load_float(UnpicklerObject *self) { PyObject *py_float = 0; char *endptr, *s; @@ -2973,7 +2973,7 @@ } static int -load_binfloat(Unpicklerobject *self) +load_binfloat(UnpicklerObject *self) { PyObject *py_float; double x; @@ -2995,7 +2995,7 @@ } static int -load_string(Unpicklerobject *self) +load_string(UnpicklerObject *self) { PyObject *str = 0; int len, res = -1; @@ -3041,7 +3041,7 @@ static int -load_binstring(Unpicklerobject *self) +load_binstring(UnpicklerObject *self) { PyObject *py_string = 0; long l; @@ -3064,7 +3064,7 @@ static int -load_short_binstring(Unpicklerobject *self) +load_short_binstring(UnpicklerObject *self) { PyObject *py_string = 0; unsigned char l; @@ -3088,7 +3088,7 @@ #ifdef Py_USING_UNICODE static int -load_unicode(Unpicklerobject *self) +load_unicode(UnpicklerObject *self) { PyObject *str = 0; int len, res = -1; @@ -3113,7 +3113,7 @@ #ifdef Py_USING_UNICODE static int -load_binunicode(Unpicklerobject *self) +load_binunicode(UnpicklerObject *self) { PyObject *unicode; long l; @@ -3137,7 +3137,7 @@ static int -load_tuple(Unpicklerobject *self) +load_tuple(UnpicklerObject *self) { PyObject *tup; int i; @@ -3151,7 +3151,7 @@ } static int -load_counted_tuple(Unpicklerobject *self, int len) +load_counted_tuple(UnpicklerObject *self, int len) { PyObject *tup = PyTuple_New(len); @@ -3171,7 +3171,7 @@ } static int -load_empty_list(Unpicklerobject *self) +load_empty_list(UnpicklerObject *self) { PyObject *list; @@ -3182,7 +3182,7 @@ } static int -load_empty_dict(Unpicklerobject *self) +load_empty_dict(UnpicklerObject *self) { PyObject *dict; @@ -3194,7 +3194,7 @@ static int -load_list(Unpicklerobject *self) +load_list(UnpicklerObject *self) { PyObject *list = 0; int i; @@ -3208,7 +3208,7 @@ } static int -load_dict(Unpicklerobject *self) +load_dict(UnpicklerObject *self) { PyObject *dict, *key, *value; int i, j, k; @@ -3260,7 +3260,7 @@ static int -load_obj(Unpicklerobject *self) +load_obj(UnpicklerObject *self) { PyObject *class, *tup, *obj = 0; int i; @@ -3284,7 +3284,7 @@ static int -load_inst(Unpicklerobject *self) +load_inst(UnpicklerObject *self) { PyObject *tup, *class = 0, *obj = 0, *module_name, *class_name; int i, len; @@ -3328,7 +3328,7 @@ } static int -load_newobj(Unpicklerobject *self) +load_newobj(UnpicklerObject *self) { PyObject *args = NULL; PyObject *clsraw = NULL; @@ -3378,7 +3378,7 @@ } static int -load_global(Unpicklerobject *self) +load_global(UnpicklerObject *self) { PyObject *class = 0, *module_name = 0, *class_name = 0; int len; @@ -3412,7 +3412,7 @@ static int -load_persid(Unpicklerobject *self) +load_persid(UnpicklerObject *self) { PyObject *pid = 0; int len; @@ -3457,7 +3457,7 @@ } static int -load_binpersid(Unpicklerobject *self) +load_binpersid(UnpicklerObject *self) { PyObject *pid = 0; @@ -3495,7 +3495,7 @@ static int -load_pop(Unpicklerobject *self) +load_pop(UnpicklerObject *self) { int len; @@ -3521,7 +3521,7 @@ static int -load_pop_mark(Unpicklerobject *self) +load_pop_mark(UnpicklerObject *self) { int i; @@ -3535,7 +3535,7 @@ static int -load_dup(Unpicklerobject *self) +load_dup(UnpicklerObject *self) { PyObject *last; int len; @@ -3550,7 +3550,7 @@ static int -load_get(Unpicklerobject *self) +load_get(UnpicklerObject *self) { PyObject *py_str = 0, *value = 0; int len; @@ -3581,7 +3581,7 @@ static int -load_binget(Unpicklerobject *self) +load_binget(UnpicklerObject *self) { PyObject *py_key = 0, *value = 0; unsigned char key; @@ -3611,7 +3611,7 @@ static int -load_long_binget(Unpicklerobject *self) +load_long_binget(UnpicklerObject *self) { PyObject *py_key = 0, *value = 0; unsigned char c; @@ -3652,7 +3652,7 @@ * the number of bytes following the opcode, holding the index (code) value. */ static int -load_extension(Unpicklerobject *self, int nbytes) +load_extension(UnpicklerObject *self, int nbytes) { char *codebytes; /* the nbytes bytes after the opcode */ long code; /* calc_binint returns long */ @@ -3720,7 +3720,7 @@ } static int -load_put(Unpicklerobject *self) +load_put(UnpicklerObject *self) { PyObject *py_str = 0, *value = 0; int len, l; @@ -3742,7 +3742,7 @@ static int -load_binput(Unpicklerobject *self) +load_binput(UnpicklerObject *self) { PyObject *py_key = 0, *value = 0; unsigned char key; @@ -3766,7 +3766,7 @@ static int -load_long_binput(Unpicklerobject *self) +load_long_binput(UnpicklerObject *self) { PyObject *py_key = 0, *value = 0; long key; @@ -3798,7 +3798,7 @@ static int -do_append(Unpicklerobject *self, int x) +do_append(UnpicklerObject *self, int x) { PyObject *value = 0, *list = 0, *append_method = 0; int len, i; @@ -3856,21 +3856,21 @@ static int -load_append(Unpicklerobject *self) +load_append(UnpicklerObject *self) { return do_append(self, self->stack->length - 1); } static int -load_appends(Unpicklerobject *self) +load_appends(UnpicklerObject *self) { return do_append(self, marker(self)); } static int -do_setitems(Unpicklerobject *self, int x) +do_setitems(UnpicklerObject *self, int x) { PyObject *value = 0, *key = 0, *dict = 0; int len, i, r = 0; @@ -3896,20 +3896,20 @@ static int -load_setitem(Unpicklerobject *self) +load_setitem(UnpicklerObject *self) { return do_setitems(self, self->stack->length - 2); } static int -load_setitems(Unpicklerobject *self) +load_setitems(UnpicklerObject *self) { return do_setitems(self, marker(self)); } static int -load_build(Unpicklerobject *self) +load_build(UnpicklerObject *self) { PyObject *state, *inst, *slotstate; PyObject *__setstate__; @@ -4003,7 +4003,7 @@ static int -load_mark(Unpicklerobject *self) +load_mark(UnpicklerObject *self) { int s; @@ -4035,7 +4035,7 @@ } static int -load_reduce(Unpicklerobject *self) +load_reduce(UnpicklerObject *self) { PyObject *callable = 0, *arg_tup = 0, *ob = 0; @@ -4060,7 +4060,7 @@ * is the first opcode for protocols >= 2. */ static int -load_proto(Unpicklerobject *self) +load_proto(UnpicklerObject *self) { int i; char *protobyte; @@ -4082,7 +4082,7 @@ } static PyObject * -load(Unpicklerobject *self) +load(UnpicklerObject *self) { PyObject *err = 0, *val = 0; char *s; @@ -4390,7 +4390,7 @@ find persistent references. */ static int -noload_obj(Unpicklerobject * self) +noload_obj(UnpicklerObject * self) { int i; @@ -4401,7 +4401,7 @@ static int -noload_inst(Unpicklerobject * self) +noload_inst(UnpicklerObject * self) { int i; char *s; @@ -4418,7 +4418,7 @@ } static int -noload_newobj(Unpicklerobject * self) +noload_newobj(UnpicklerObject * self) { PyObject *obj; @@ -4437,7 +4437,7 @@ } static int -noload_global(Unpicklerobject *self) +noload_global(UnpicklerObject *self) { char *s; @@ -4450,7 +4450,7 @@ } static int -noload_reduce(Unpicklerobject *self) +noload_reduce(UnpicklerObject *self) { if (self->stack->length < 2) @@ -4461,7 +4461,7 @@ } static int -noload_build(Unpicklerobject *self) +noload_build(UnpicklerObject *self) { if (self->stack->length < 1) @@ -4471,7 +4471,7 @@ } static int -noload_extension(Unpicklerobject *self, int nbytes) +noload_extension(UnpicklerObject *self, int nbytes) { char *codebytes; @@ -4484,7 +4484,7 @@ static PyObject * -noload(Unpicklerobject *self) +noload(UnpicklerObject *self) { PyObject *err = 0, *val = 0; char *s; @@ -4783,13 +4783,13 @@ static PyObject * -Unpickler_load(Unpicklerobject * self, PyObject * unused) +Unpickler_load(UnpicklerObject * self, PyObject * unused) { return load(self); } static PyObject * -Unpickler_noload(Unpicklerobject *self, PyObject *unused) +Unpickler_noload(UnpicklerObject *self, PyObject *unused) { return noload(self); } @@ -4812,12 +4812,12 @@ }; -static Unpicklerobject * -newUnpicklerobject(PyObject *f) +static UnpicklerObject * +newUnpicklerObject(PyObject *f) { - Unpicklerobject *self; + UnpicklerObject *self; - if (!(self = PyObject_GC_New(Unpicklerobject, &Unpicklertype))) + if (!(self = PyObject_GC_New(UnpicklerObject, &Unpickler_Type))) return NULL; self->file = NULL; @@ -4880,12 +4880,12 @@ static PyObject * get_Unpickler(PyObject *self, PyObject *file) { - return (PyObject *) newUnpicklerobject(file); + return (PyObject *) newUnpicklerObject(file); } static void -Unpickler_dealloc(Unpicklerobject *self) +Unpickler_dealloc(UnpicklerObject *self) { PyObject_GC_UnTrack((PyObject *) self); Py_XDECREF(self->readline); @@ -4910,7 +4910,7 @@ } static int -Unpickler_traverse(Unpicklerobject *self, visitproc visit, void *arg) +Unpickler_traverse(UnpicklerObject *self, visitproc visit, void *arg) { Py_VISIT(self->readline); Py_VISIT(self->read); @@ -4925,7 +4925,7 @@ } static int -Unpickler_clear(Unpicklerobject *self) +Unpickler_clear(UnpicklerObject *self) { Py_CLEAR(self->readline); Py_CLEAR(self->read); @@ -4940,7 +4940,7 @@ } static PyObject * -Unpickler_getattr(Unpicklerobject *self, char *name) +Unpickler_getattr(UnpicklerObject *self, char *name) { if (!strcmp(name, "persistent_load")) { if (!self->pers_func) { @@ -4982,7 +4982,7 @@ static int -Unpickler_setattr(Unpicklerobject *self, char *name, PyObject *value) +Unpickler_setattr(UnpicklerObject *self, char *name, PyObject *value) { if (!strcmp(name, "persistent_load")) { @@ -5020,13 +5020,13 @@ return -1; } -PyDoc_STRVAR(Unpicklertype__doc__, "Objects that know how to unpickle"); +PyDoc_STRVAR(Unpickler_Type__doc__, "Objects that know how to unpickle"); -static PyTypeObject Unpicklertype = { +static PyTypeObject Unpickler_Type = { PyObject_HEAD_INIT(NULL) 0, /*ob_size */ "_pickle.Unpickler", /*tp_name */ - sizeof(Unpicklerobject), /*tp_basicsize */ + sizeof(UnpicklerObject), /*tp_basicsize */ 0, (destructor) Unpickler_dealloc, /* tp_dealloc */ 0, /* tp_print */ @@ -5044,7 +5044,7 @@ 0, /* tp_setattro */ 0, /* tp_as_buffer */ Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE | Py_TPFLAGS_HAVE_GC, - Unpicklertype__doc__, /* tp_doc */ + Unpickler_Type__doc__, /* tp_doc */ (traverseproc) Unpickler_traverse, /* tp_traverse */ (inquiry) Unpickler_clear, /* tp_clear */ }; @@ -5054,7 +5054,7 @@ { PyObject *copy_reg, *t, *r; - if (PyType_Ready(&Unpicklertype) < 0) + if (PyType_Ready(&Unpickler_Type) < 0) return -1; if (PyType_Ready(&Pickler_Type) < 0) return -1; @@ -5190,7 +5190,7 @@ PyObject *compatible_formats; Pickler_Type.ob_type = &PyType_Type; - Unpicklertype.ob_type = &PyType_Type; + Unpickler_Type.ob_type = &PyType_Type; PdataType.ob_type = &PyType_Type; /* Initialize some pieces. We need to do this before module creation, From python-checkins at python.org Mon Jul 9 23:10:34 2007 From: python-checkins at python.org (brett.cannon) Date: Mon, 9 Jul 2007 23:10:34 +0200 (CEST) Subject: [Python-checkins] r56223 - sandbox/trunk/import_in_py/_importlib.py Message-ID: <20070709211034.A5C6C1E4003@bag.python.org> Author: brett.cannon Date: Mon Jul 9 23:10:34 2007 New Revision: 56223 Modified: sandbox/trunk/import_in_py/_importlib.py Log: Remove os.path.exists dependency. Modified: sandbox/trunk/import_in_py/_importlib.py ============================================================================== --- sandbox/trunk/import_in_py/_importlib.py (original) +++ sandbox/trunk/import_in_py/_importlib.py Mon Jul 9 23:10:34 2007 @@ -60,13 +60,22 @@ from __future__ import with_statement # Built-in modules required by this module to work. -_required_builtins = ['imp', 'sys', 'marshal'] +_required_builtins = ['imp', 'sys', 'marshal', 'posix'] def _path_join(*args): """Replacement for os.path.join so as to remove dependency on os module.""" return path_sep.join(args) +def _path_exists(path): + """Replacement for os.path.exists to help remove dependency on 'os'.""" + try: + posix.stat(path) + except OSError: + return False + else: + return True + class _BuiltinFrozenBaseImporter(object): @@ -287,7 +296,7 @@ """ path = base_path + type_ if must_exist: - path = path if os.path.exists(path) else None + path = path if _path_exists(path) else None return path def read_data(self, path, binary=False): From python-checkins at python.org Mon Jul 9 23:16:00 2007 From: python-checkins at python.org (brett.cannon) Date: Mon, 9 Jul 2007 23:16:00 +0200 (CEST) Subject: [Python-checkins] r56224 - sandbox/trunk/import_in_py/_importlib.py Message-ID: <20070709211600.000EE1E4002@bag.python.org> Author: brett.cannon Date: Mon Jul 9 23:16:00 2007 New Revision: 56224 Modified: sandbox/trunk/import_in_py/_importlib.py Log: Remove os.stat dependency. Assumes posix is the name of the built-in module. This is not the case on Windows or OS/2. Also don't know how it works out with other platforms such as RISC. Modified: sandbox/trunk/import_in_py/_importlib.py ============================================================================== --- sandbox/trunk/import_in_py/_importlib.py (original) +++ sandbox/trunk/import_in_py/_importlib.py Mon Jul 9 23:16:00 2007 @@ -60,6 +60,8 @@ from __future__ import with_statement # Built-in modules required by this module to work. +# XXX posix can be named os2 or nt. +# XXX posix guaranteed on all platforms (especially posix.stat)? _required_builtins = ['imp', 'sys', 'marshal', 'posix'] @@ -273,7 +275,7 @@ PyPycHandler. """ - return int(os.stat(path).st_mtime) + return int(posix.stat(path).st_mtime) def split_path(self, path): """Split the specified path into a base path and the type of the From python-checkins at python.org Tue Jul 10 09:33:58 2007 From: python-checkins at python.org (collin.winter) Date: Tue, 10 Jul 2007 09:33:58 +0200 (CEST) Subject: [Python-checkins] r56227 - sandbox/trunk/2to3/HACKING Message-ID: <20070710073358.0977C1E4003@bag.python.org> Author: collin.winter Date: Tue Jul 10 09:33:57 2007 New Revision: 56227 Modified: sandbox/trunk/2to3/HACKING Log: Add the examples/ directory to HACKING, minor style changes. Modified: sandbox/trunk/2to3/HACKING ============================================================================== --- sandbox/trunk/2to3/HACKING (original) +++ sandbox/trunk/2to3/HACKING Tue Jul 10 09:33:57 2007 @@ -13,7 +13,6 @@ just see what the parser does. - Putting 2to3 to work somewhere else: * By default, 2to3 uses a merger of Python 2.x and Python 3's grammars. @@ -28,6 +27,8 @@ could leverage 90% of the existing infrastructure with primarily cosmetic changes (e.g., fixes/fix_*.py -> styles/style_*.py). + * The examples/ directory contains fixers that show off 2to3's flexibility, + such as a fixer for whitespace. TODO @@ -35,7 +36,7 @@ Simple: ####### - * Refactor common code out of fixes/fix_*.py into fixes.util (ongoing). + * Refactor common code out of fixes/fix_*.py into fixes.util (on-going). Complex: From python-checkins at python.org Tue Jul 10 09:51:46 2007 From: python-checkins at python.org (collin.winter) Date: Tue, 10 Jul 2007 09:51:46 +0200 (CEST) Subject: [Python-checkins] r56228 - sandbox/trunk/2to3/tests/test_all_fixers.py Message-ID: <20070710075146.25A9C1E4017@bag.python.org> Author: collin.winter Date: Tue Jul 10 09:51:45 2007 New Revision: 56228 Modified: sandbox/trunk/2to3/tests/test_all_fixers.py Log: Tweak test_all_fixers's status report messages. Modified: sandbox/trunk/2to3/tests/test_all_fixers.py ============================================================================== --- sandbox/trunk/2to3/tests/test_all_fixers.py (original) +++ sandbox/trunk/2to3/tests/test_all_fixers.py Tue Jul 10 09:51:45 2007 @@ -51,9 +51,8 @@ fixerdir = os.path.join(basedir, "fixes") for filename in os.listdir(fixerdir): if not filename.endswith(".py"): - print "Skipping %s" % filename continue - print "Testing fixer %s..." % filename + print "Fixing %s..." % filename fixer = os.path.join(fixerdir, filename) self.refactor_stream(fixer, open(fixer)) From python-checkins at python.org Tue Jul 10 09:52:18 2007 From: python-checkins at python.org (collin.winter) Date: Tue, 10 Jul 2007 09:52:18 +0200 (CEST) Subject: [Python-checkins] r56229 - sandbox/trunk/2to3/tests/test_parser.py Message-ID: <20070710075218.A89F11E400D@bag.python.org> Author: collin.winter Date: Tue Jul 10 09:52:18 2007 New Revision: 56229 Added: sandbox/trunk/2to3/tests/test_parser.py Log: Add a cut-down version of pytree_idempotency.py as test_parser.py. Added: sandbox/trunk/2to3/tests/test_parser.py ============================================================================== --- (empty file) +++ sandbox/trunk/2to3/tests/test_parser.py Tue Jul 10 09:52:18 2007 @@ -0,0 +1,48 @@ +#!/usr/bin/env python2.5 +"""Test suite for 2to3's parser.""" +# Author: Collin Winter + +# Testing imports +import support +from support import driver, test_dir + +# Python imports +import os +import os.path + +# Local imports +from pgen2.parse import ParseError + + +class TestParserIdempotency(support.TestCase): + + """A cut-down version of pytree_idempotency.py.""" + + def test_2to3_files(self): + proj_dir = os.path.join(test_dir, "..") + + for dirpath, dirnames, filenames in os.walk(proj_dir): + for filename in filenames: + if filename.endswith(".py"): + filepath = os.path.join(dirpath, filename) + print "Parsing %s..." % os.path.normpath(filepath) + tree = driver.parse_file(filepath, debug=True) + if diff(filepath, tree): + self.fail("Idempotency failed: %s" % filename) + + +def diff(fn, tree): + f = open("@", "w") + try: + f.write(str(tree)) + finally: + f.close() + try: + return os.system("diff -u %s @" % fn) + finally: + os.remove("@") + + +if __name__ == "__main__": + import __main__ + support.run_all_tests(__main__) From python-checkins at python.org Tue Jul 10 16:13:31 2007 From: python-checkins at python.org (alexandre.vassalotti) Date: Tue, 10 Jul 2007 16:13:31 +0200 (CEST) Subject: [Python-checkins] r56240 - python/branches/cpy_merge/Modules/_picklemodule.c Message-ID: <20070710141331.DA7DD1E4003@bag.python.org> Author: alexandre.vassalotti Date: Tue Jul 10 16:13:31 2007 New Revision: 56240 Modified: python/branches/cpy_merge/Modules/_picklemodule.c Log: Add basic subclassing support to Unpickler. Fix docstring of Pickler and Unpickler. Fix a reference leak in Pickler. Modified: python/branches/cpy_merge/Modules/_picklemodule.c ============================================================================== --- python/branches/cpy_merge/Modules/_picklemodule.c (original) +++ python/branches/cpy_merge/Modules/_picklemodule.c Tue Jul 10 16:13:31 2007 @@ -2438,11 +2438,12 @@ self->dispatch_table = dispatch_table; Py_INCREF(file); + self->file = file; if (PyFile_Check(file)) { self->fp = PyFile_AsFile(file); if (self->fp == NULL) { PyErr_SetString(PyExc_ValueError, "I/O operation on closed file"); - goto io_error; + goto error; } self->write_func = write_file; } @@ -2452,13 +2453,13 @@ PyErr_Clear(); PyErr_SetString(PyExc_TypeError, "argument must have 'write' " "attribute"); - goto io_error; + goto error; } self->write_buf = (char *) PyMem_Malloc(WRITE_BUF_SIZE); if (self->write_buf == NULL) { PyErr_NoMemory(); - goto io_error; + goto error; } self->write_func = write_other; } @@ -2466,8 +2467,6 @@ PyObject_GC_Track(self); return (PyObject *) self; - io_error: - Py_DECREF(file); error: Py_DECREF(self); return NULL; @@ -2477,6 +2476,7 @@ Pickler_dealloc(PicklerObject *self) { PyObject_GC_UnTrack(self); + Py_XDECREF(self->file); Py_XDECREF(self->write); Py_XDECREF(self->memo); Py_XDECREF(self->fast_memo); @@ -2604,11 +2604,12 @@ }; PyDoc_STRVAR(Pickler_doc, -"Pickler(file, protocol=0) -- Create a pickler.\n" +"Pickler(file, protocol=0) -> new pickler object" "\n" "This takes a file-like object for writing a pickle data stream.\n" -"The optional proto argument tells the pickler to use the given\n" -"protocol; supported protocols are 0, 1, 2. The default\n" +"\n" +"The optional protocol argument tells the pickler to use the\n" +"given protocol; supported protocols are 0, 1, 2. The default\n" "protocol is 0, to be backwards compatible. (Protocol 0 is the\n" "only protocol that can be written to a file opened in text\n" "mode and read back successfully. When using a protocol higher\n" @@ -4812,17 +4813,23 @@ }; -static UnpicklerObject * -newUnpicklerObject(PyObject *f) +static PyObject * +Unpickler_new(PyTypeObject *type, PyObject *args, PyObject *kwds) { + static char *kwlist[] = { "file", NULL }; UnpicklerObject *self; + PyObject *file; + + if (!PyArg_ParseTupleAndKeywords(args, kwds, "O:Unpickler", + kwlist, &file)) + return NULL; - if (!(self = PyObject_GC_New(UnpicklerObject, &Unpickler_Type))) + self = PyObject_GC_New(UnpicklerObject, &Unpickler_Type); + if (self == NULL) return NULL; - self->file = NULL; + self->fp = NULL; self->arg = NULL; - self->stack = (Pdata *) Pdata_New(); self->pers_func = NULL; self->last_string = NULL; self->marks = NULL; @@ -4833,18 +4840,19 @@ self->readline = NULL; self->find_class = NULL; - if (!(self->memo = PyDict_New())) + self->memo = PyDict_New(); + if (self->memo == NULL) goto error; - if (!self->stack) + self->stack = (Pdata *) Pdata_New(); + if (self->stack == NULL) goto error; - Py_INCREF(f); - self->file = f; - - /* Set read, readline based on type of f */ - if (PyFile_Check(f)) { - self->fp = PyFile_AsFile(f); + Py_INCREF(file); + self->file = file; + /* Set read, readline based on type of file */ + if (PyFile_Check(file)) { + self->fp = PyFile_AsFile(file); if (self->fp == NULL) { PyErr_SetString(PyExc_ValueError, "I/O operation on closed file"); goto error; @@ -4853,37 +4861,27 @@ self->readline_func = readline_file; } else { - - self->fp = NULL; - self->read_func = read_other; - self->readline_func = readline_other; - - if (!((self->readline = PyObject_GetAttr(f, readline_str)) && - (self->read = PyObject_GetAttr(f, read_str)))) { + self->readline = PyObject_GetAttr(file, readline_str); + self->read = PyObject_GetAttr(file, read_str); + if (self->readline == NULL || self->read == NULL) { PyErr_Clear(); PyErr_SetString(PyExc_TypeError, "argument must have 'read' and " "'readline' attributes"); goto error; } + self->read_func = read_other; + self->readline_func = readline_other; } - PyObject_GC_Track(self); - return self; + PyObject_GC_Track(self); + return (PyObject *)self; error: - Py_DECREF((PyObject *) self); + Py_DECREF(self); return NULL; } - -static PyObject * -get_Unpickler(PyObject *self, PyObject *file) -{ - return (PyObject *) newUnpicklerObject(file); -} - - static void Unpickler_dealloc(UnpicklerObject *self) { @@ -5020,33 +5018,62 @@ return -1; } -PyDoc_STRVAR(Unpickler_Type__doc__, "Objects that know how to unpickle"); +PyDoc_STRVAR(Unpickler_doc, +"Unpickler(file) -> new unpickler object" +"\n" +"This takes a file-like object for reading a pickle data stream.\n" +"\n" +"The protocol version of the pickle is detected automatically, so no\n" +"proto argument is needed.\n" +"\n" +"The file-like object must have two methods, a read() method that\n" +"takes an integer argument, and a readline() method that requires no\n" +"arguments. Both methods should return a string. Thus file-like\n" +"object can be a file object opened for reading, a StringIO object,\n" +"or any other custom object that meets this interface.\n"); static PyTypeObject Unpickler_Type = { PyObject_HEAD_INIT(NULL) - 0, /*ob_size */ - "_pickle.Unpickler", /*tp_name */ - sizeof(UnpicklerObject), /*tp_basicsize */ - 0, - (destructor) Unpickler_dealloc, /* tp_dealloc */ - 0, /* tp_print */ - (getattrfunc) Unpickler_getattr, /* tp_getattr */ - (setattrfunc) Unpickler_setattr, /* tp_setattr */ - 0, /* tp_compare */ - 0, /* tp_repr */ - 0, /* tp_as_number */ - 0, /* tp_as_sequence */ - 0, /* tp_as_mapping */ - 0, /* tp_hash */ - 0, /* tp_call */ - 0, /* tp_str */ - 0, /* tp_getattro */ - 0, /* tp_setattro */ - 0, /* tp_as_buffer */ + 0, /*ob_size*/ + "_pickle.Unpickler", /*tp_name*/ + sizeof(UnpicklerObject), /*tp_basicsize*/ + 0, /*tp_itemsize*/ + (destructor)Unpickler_dealloc, /*tp_dealloc*/ + 0, /*tp_print*/ + (getattrfunc)Unpickler_getattr, /*tp_getattr*/ + (setattrfunc)Unpickler_setattr, /*tp_setattr*/ + 0, /*tp_compare*/ + 0, /*tp_repr*/ + 0, /*tp_as_number*/ + 0, /*tp_as_sequence*/ + 0, /*tp_as_mapping*/ + 0, /*tp_hash*/ + 0, /*tp_call*/ + 0, /*tp_str*/ + 0, /*tp_getattro*/ + 0, /*tp_setattro*/ + 0, /*tp_as_buffer*/ Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE | Py_TPFLAGS_HAVE_GC, - Unpickler_Type__doc__, /* tp_doc */ - (traverseproc) Unpickler_traverse, /* tp_traverse */ - (inquiry) Unpickler_clear, /* tp_clear */ + Unpickler_doc, /*tp_doc*/ + (traverseproc)Unpickler_traverse, /*tp_traverse*/ + (inquiry)Unpickler_clear, /*tp_clear*/ + 0, /*tp_richcompare*/ + 0, /*tp_weaklistoffset*/ + 0, /*tp_iter*/ + 0, /*tp_iternext*/ + 0, /*tp_methods*/ + 0, /*tp_members*/ + 0, /*tp_getset*/ + 0, /*tp_base*/ + 0, /*tp_dict*/ + 0, /*tp_descr_get*/ + 0, /*tp_descr_set*/ + 0, /*tp_dictoffset*/ + 0, /*tp_init*/ + 0, /*tp_alloc*/ + Unpickler_new, /*tp_new*/ + 0, /*tp_free*/ + 0, /*tp_is_gc*/ }; static int @@ -5208,6 +5235,7 @@ return; PyModule_AddObject(m, "Pickler", (PyObject *)&Pickler_Type); + PyModule_AddObject(m, "Unpickler", (PyObject *)&Unpickler_Type); /* Add some symbolic constants to the module */ d = PyModule_GetDict(m); From python-checkins at python.org Tue Jul 10 17:32:19 2007 From: python-checkins at python.org (collin.winter) Date: Tue, 10 Jul 2007 17:32:19 +0200 (CEST) Subject: [Python-checkins] r56241 - sandbox/trunk/2to3/pytree.py Message-ID: <20070710153219.5B30E1E4003@bag.python.org> Author: collin.winter Date: Tue Jul 10 17:32:19 2007 New Revision: 56241 Modified: sandbox/trunk/2to3/pytree.py Log: Whitespace cleanup. Modified: sandbox/trunk/2to3/pytree.py ============================================================================== --- sandbox/trunk/2to3/pytree.py (original) +++ sandbox/trunk/2to3/pytree.py Tue Jul 10 17:32:19 2007 @@ -126,12 +126,12 @@ return node = node.children[0] return node.lineno - + def changed(self): if self.parent: self.parent.changed() self.was_changed = True - + def remove(self): """Remove the node from the tree. Returns the position of the node in its parent's children before it was removed.""" @@ -149,7 +149,7 @@ sibling, return None.""" if self.parent is None: return None - + # Can't use index(); we need to test by identity for i, sibling in enumerate(self.parent.children): if sibling is self: @@ -157,7 +157,7 @@ return self.parent.children[i+1] except IndexError: return None - + def get_suffix(self): """Return the string immediately following the invocant node. This is effectively equivalent to node.get_next_sibling().get_prefix()""" @@ -232,20 +232,20 @@ if not self.children: return "" return self.children[0].get_prefix() - + def set_child(self, i, child): """Equivalent to 'node.children[i] = child'. This method also sets the child's parent attribute appropriately.""" child.parent = self self.children[i].parent = None self.children[i] = child - + def insert_child(self, i, child): """Equivalent to 'node.children.insert(i, child)'. This method also sets the child's parent attribute appropriately.""" child.parent = self self.children.insert(i, child) - + def append_child(self, child): """Equivalent to 'node.children.append(child)'. This method also sets the child's parent attribute appropriately.""" From python-checkins at python.org Tue Jul 10 17:34:18 2007 From: python-checkins at python.org (collin.winter) Date: Tue, 10 Jul 2007 17:34:18 +0200 (CEST) Subject: [Python-checkins] r56242 - in sandbox/trunk/2to3: pytree.py tests/test_pytree.py Message-ID: <20070710153418.93AAC1E4003@bag.python.org> Author: collin.winter Date: Tue Jul 10 17:34:18 2007 New Revision: 56242 Modified: sandbox/trunk/2to3/pytree.py sandbox/trunk/2to3/tests/test_pytree.py Log: Fix a bug where set_prefix() didn't register that the node had changed. Modified: sandbox/trunk/2to3/pytree.py ============================================================================== --- sandbox/trunk/2to3/pytree.py (original) +++ sandbox/trunk/2to3/pytree.py Tue Jul 10 17:34:18 2007 @@ -304,6 +304,7 @@ def set_prefix(self, prefix): """Sets the prefix for the node.""" + self.changed() self.prefix = prefix def get_prefix(self): Modified: sandbox/trunk/2to3/tests/test_pytree.py ============================================================================== --- sandbox/trunk/2to3/tests/test_pytree.py (original) +++ sandbox/trunk/2to3/tests/test_pytree.py Tue Jul 10 17:34:18 2007 @@ -67,8 +67,10 @@ def testLeafPrefix(self): l1 = pytree.Leaf(100, "foo") self.assertEqual(l1.get_prefix(), "") + self.failIf(l1.was_changed) l1.set_prefix(" ##\n\n") self.assertEqual(l1.get_prefix(), " ##\n\n") + self.failUnless(l1.was_changed) def testNode(self): l1 = pytree.Leaf(100, "foo") From python-checkins at python.org Wed Jul 11 01:48:48 2007 From: python-checkins at python.org (brett.cannon) Date: Wed, 11 Jul 2007 01:48:48 +0200 (CEST) Subject: [Python-checkins] r56246 - sandbox/trunk/import_in_py/_importlib.py sandbox/trunk/import_in_py/importlib.py Message-ID: <20070710234848.88A5A1E4007@bag.python.org> Author: brett.cannon Date: Wed Jul 11 01:48:48 2007 New Revision: 56246 Modified: sandbox/trunk/import_in_py/_importlib.py sandbox/trunk/import_in_py/importlib.py Log: Rework importing of built-in modules. Modified: sandbox/trunk/import_in_py/_importlib.py ============================================================================== --- sandbox/trunk/import_in_py/_importlib.py (original) +++ sandbox/trunk/import_in_py/_importlib.py Wed Jul 11 01:48:48 2007 @@ -58,12 +58,7 @@ """ from __future__ import with_statement - -# Built-in modules required by this module to work. -# XXX posix can be named os2 or nt. -# XXX posix guaranteed on all platforms (especially posix.stat)? -_required_builtins = ['imp', 'sys', 'marshal', 'posix'] - +# The injected modules are 'imp', 'sys', 'marshal', 'posix' (aka 'nt' & 'os2'). def _path_join(*args): """Replacement for os.path.join so as to remove dependency on os module.""" Modified: sandbox/trunk/import_in_py/importlib.py ============================================================================== --- sandbox/trunk/import_in_py/importlib.py (original) +++ sandbox/trunk/import_in_py/importlib.py Wed Jul 11 01:48:48 2007 @@ -96,10 +96,22 @@ return False -# Import needed built-in modules. -for builtin_name in _importlib._required_builtins: - module = __import__(builtin_name) - _importlib.__dict__[builtin_name] = module +# Required built-in modules. +import imp, sys, marshal +_importlib.imp = imp +_importlib.sys = sys +_importlib.marshal = marshal +try: + import posix +except ImportError: + try: + import nt as posix + except ImportError: + try: + import os2 as posix + except ImportError: + raise ImportError('posix, nt, or os2 required for importlib') +_importlib.posix = posix # XXX These all need to either go away or become built-in modules # (Neal). From python-checkins at python.org Wed Jul 11 01:59:30 2007 From: python-checkins at python.org (brett.cannon) Date: Wed, 11 Jul 2007 01:59:30 +0200 (CEST) Subject: [Python-checkins] r56247 - sandbox/trunk/import_in_py/_importlib.py sandbox/trunk/import_in_py/importlib.py Message-ID: <20070710235930.9794D1E4011@bag.python.org> Author: brett.cannon Date: Wed Jul 11 01:59:30 2007 New Revision: 56247 Modified: sandbox/trunk/import_in_py/_importlib.py sandbox/trunk/import_in_py/importlib.py Log: Remove os.path.(isfile | isdir) dependency. Modified: sandbox/trunk/import_in_py/_importlib.py ============================================================================== --- sandbox/trunk/import_in_py/_importlib.py (original) +++ sandbox/trunk/import_in_py/_importlib.py Wed Jul 11 01:59:30 2007 @@ -74,6 +74,23 @@ return True +def _path_is_mode_type(path, mode): + """Test whether the path is the specified mode type.""" + try: + stat_info = posix.stat(path) + except OSError: + return False + return (stat_info.st_mode & 0170000) == mode + +def _path_isfile(path): + """Replacement for os.path.isfile.""" + return _path_is_mode_type(path, 0100000) + +def _path_isdir(path): + """Replacement for os.path.isdir.""" + return _path_is_mode_type(path, 0040000) + + class _BuiltinFrozenBaseImporter(object): """Base class for meta_path importers for built-in and frozen modules. @@ -164,7 +181,7 @@ """ absolute_path = os.path.abspath(path_entry) - if os.path.isdir(absolute_path): + if _path_isdir(absolute_path): return FileSystemImporter(absolute_path, *self.handlers) else: raise ImportError("can only handle directories") @@ -207,19 +224,19 @@ init_filename = '__init__' + file_ext package_init = _path_join(package_directory, init_filename) # Check if it is a package with an __init__ file. - if (os.path.isfile(package_init) and + if (_path_isfile(package_init) and _case_ok(self.path_entry, tail_module) and _case_ok(package_directory, init_filename)): return self.loader(package_init, handler, package_directory) # See if it is a module. file_name = tail_module + file_ext file_path = _path_join(self.path_entry, file_name) - if (os.path.isfile(file_path) and + if (_path_isfile(file_path) and _case_ok(self.path_entry, file_name)): return self.loader(file_path, handler) else: # Raise a warning if it matches a directory w/o an __init__ file. - if (os.path.isdir(package_directory) and + if (_path_isdir(package_directory) and _case_ok(self.path_entry, tail_module)): warnings.warn("Not importing directory %s: missing __init__.py" % package_directory, Modified: sandbox/trunk/import_in_py/importlib.py ============================================================================== --- sandbox/trunk/import_in_py/importlib.py (original) +++ sandbox/trunk/import_in_py/importlib.py Wed Jul 11 01:59:30 2007 @@ -96,6 +96,7 @@ return False + # Required built-in modules. import imp, sys, marshal _importlib.imp = imp From python-checkins at python.org Wed Jul 11 02:03:48 2007 From: python-checkins at python.org (brett.cannon) Date: Wed, 11 Jul 2007 02:03:48 +0200 (CEST) Subject: [Python-checkins] r56248 - sandbox/trunk/import_in_py/_importlib.py sandbox/trunk/import_in_py/importlib.py Message-ID: <20070711000348.2A9271E4007@bag.python.org> Author: brett.cannon Date: Wed Jul 11 02:03:47 2007 New Revision: 56248 Modified: sandbox/trunk/import_in_py/_importlib.py sandbox/trunk/import_in_py/importlib.py Log: Isolate attribute required from errno. Modified: sandbox/trunk/import_in_py/_importlib.py ============================================================================== --- sandbox/trunk/import_in_py/_importlib.py (original) +++ sandbox/trunk/import_in_py/_importlib.py Wed Jul 11 02:03:47 2007 @@ -336,7 +336,7 @@ with open(path, 'wb' if binary else 'w') as data_file: data_file.write(data) except IOError as exc: - if exc.errno == errno.EACCES: + if exc.errno == EACCES: pass else: raise Modified: sandbox/trunk/import_in_py/importlib.py ============================================================================== --- sandbox/trunk/import_in_py/importlib.py (original) +++ sandbox/trunk/import_in_py/importlib.py Wed Jul 11 02:03:47 2007 @@ -116,7 +116,7 @@ # XXX These all need to either go away or become built-in modules # (Neal). -import errno +from errno import EACCES import os import warnings @@ -124,8 +124,9 @@ _importlib._w_long = _w_long #XXX Expose original from marshal. _importlib._case_ok = _case_ok #XXX Expose original from imp. _importlib.path_sep = os.sep # For os.path.join replacement. +# For allowing silent failure of .pyc creation when permission is denied. +_importlib.EACCES = EACCES -_importlib.errno = errno _importlib.os = os _importlib.warnings = warnings From python-checkins at python.org Wed Jul 11 02:27:42 2007 From: python-checkins at python.org (brett.cannon) Date: Wed, 11 Jul 2007 02:27:42 +0200 (CEST) Subject: [Python-checkins] r56249 - sandbox/trunk/import_in_py/_importlib.py sandbox/trunk/import_in_py/importlib.py Message-ID: <20070711002742.D22821E4007@bag.python.org> Author: brett.cannon Date: Wed Jul 11 02:27:42 2007 New Revision: 56249 Modified: sandbox/trunk/import_in_py/_importlib.py sandbox/trunk/import_in_py/importlib.py Log: Remove os.path.abspath dependency. That should remove os module dependency entirely from _importlib (still have one in _case_ok, but that will be exposed through imp). Modified: sandbox/trunk/import_in_py/_importlib.py ============================================================================== --- sandbox/trunk/import_in_py/_importlib.py (original) +++ sandbox/trunk/import_in_py/_importlib.py Wed Jul 11 02:27:42 2007 @@ -60,6 +60,7 @@ from __future__ import with_statement # The injected modules are 'imp', 'sys', 'marshal', 'posix' (aka 'nt' & 'os2'). +# XXX Could also expose Modules/getpath.c:joinpath() def _path_join(*args): """Replacement for os.path.join so as to remove dependency on os module.""" return path_sep.join(args) @@ -82,14 +83,26 @@ return False return (stat_info.st_mode & 0170000) == mode +# XXX Could also expose Modules/getpath.c:isfile() def _path_isfile(path): """Replacement for os.path.isfile.""" return _path_is_mode_type(path, 0100000) +# XXX Could also expose Modules/getpath.c:isdir() def _path_isdir(path): """Replacement for os.path.isdir.""" return _path_is_mode_type(path, 0040000) +def _path_absolute(path): + """Replacement for os.path.abspath.""" + try: + return posix._getfullpathname(path) + except AttributeError: + if path.startswith('/'): + return path + else: + return _path_join(posix.getcwd(), path) + class _BuiltinFrozenBaseImporter(object): @@ -180,7 +193,7 @@ interactive interpreter usage). """ - absolute_path = os.path.abspath(path_entry) + absolute_path = _path_absolute(path_entry) if _path_isdir(absolute_path): return FileSystemImporter(absolute_path, *self.handlers) else: Modified: sandbox/trunk/import_in_py/importlib.py ============================================================================== --- sandbox/trunk/import_in_py/importlib.py (original) +++ sandbox/trunk/import_in_py/importlib.py Wed Jul 11 02:27:42 2007 @@ -30,7 +30,9 @@ """ import _importlib -#XXX Temporary functions that should eventually be removed. +# XXX Temporary functions that should eventually be removed. +import os + def _set__import__(): """Set __import__ to an instance of Import.""" global original__import__ @@ -117,17 +119,16 @@ # XXX These all need to either go away or become built-in modules # (Neal). from errno import EACCES -import os +from os import sep import warnings _importlib._r_long = _r_long #XXX Expose original from marshal. _importlib._w_long = _w_long #XXX Expose original from marshal. _importlib._case_ok = _case_ok #XXX Expose original from imp. -_importlib.path_sep = os.sep # For os.path.join replacement. +_importlib.path_sep = sep # For os.path.join replacement. # For allowing silent failure of .pyc creation when permission is denied. _importlib.EACCES = EACCES -_importlib.os = os _importlib.warnings = warnings del _importlib From python-checkins at python.org Wed Jul 11 14:35:47 2007 From: python-checkins at python.org (thomas.wouters) Date: Wed, 11 Jul 2007 14:35:47 +0200 (CEST) Subject: [Python-checkins] r56262 - peps/trunk/pep-3100.txt Message-ID: <20070711123547.74AE41E4005@bag.python.org> Author: thomas.wouters Date: Wed Jul 11 14:35:47 2007 New Revision: 56262 Modified: peps/trunk/pep-3100.txt Log: Update for dict comprehensions. Modified: peps/trunk/pep-3100.txt ============================================================================== --- peps/trunk/pep-3100.txt (original) +++ peps/trunk/pep-3100.txt Wed Jul 11 14:35:47 2007 @@ -112,6 +112,8 @@ * The ``__nonzero__`` special method will be renamed to ``__bool__`` and have to return a bool. The typeobject slot will be called ``tp_bool`` [23]_ [done] +* Dict comprehensions, as first proposed in [#pep274]_ [done] + {K(x): V(x) for x in S if P(x)} means dict((K(x), V(x)) for x in S if P(x)). To be removed: @@ -371,6 +373,9 @@ .. [#pep238] PEP 238 (Changing the Division Operator) http://www.python.org/dev/peps/pep-0238 +.. [#pep274] PEP 274 (Dict Comprehensions) + http://www.python.org/dev/peps/pep-0274 + .. [#pep289] PEP 289 ("Generator Expressions") http://www.python.org/dev/peps/pep-0289 From python-checkins at python.org Wed Jul 11 15:00:58 2007 From: python-checkins at python.org (collin.winter) Date: Wed, 11 Jul 2007 15:00:58 +0200 (CEST) Subject: [Python-checkins] r56265 - sandbox/trunk/2to3/fixes/fix_next.py Message-ID: <20070711130058.C4F7C1E4017@bag.python.org> Author: collin.winter Date: Wed Jul 11 15:00:58 2007 New Revision: 56265 Modified: sandbox/trunk/2to3/fixes/fix_next.py Log: Whitespace cleanup in fix_next. Modified: sandbox/trunk/2to3/fixes/fix_next.py ============================================================================== --- sandbox/trunk/2to3/fixes/fix_next.py (original) +++ sandbox/trunk/2to3/fixes/fix_next.py Wed Jul 11 15:00:58 2007 @@ -107,7 +107,7 @@ if n: self.warning(n, bind_warning) self.shadowed_next = True - + def finish_tree(self, tree, filename): super(FixNext, self).finish_tree(tree, filename) if self.shadowed_next: @@ -119,17 +119,17 @@ ### target. def is_assign_target(node): - assign = find_assign(node) + assign = find_assign(node) if assign is None: return False - + for child in assign.children: if child.type == token.EQUAL: return False elif is_subtree(child, node): return True return False - + def find_assign(node): if node.type == syms.expr_stmt: return node From python-checkins at python.org Wed Jul 11 17:11:45 2007 From: python-checkins at python.org (collin.winter) Date: Wed, 11 Jul 2007 17:11:45 +0200 (CEST) Subject: [Python-checkins] r56275 - sandbox/trunk/2to3/fixes/fix_dict.py sandbox/trunk/2to3/fixes/util.py Message-ID: <20070711151145.981FB1E4005@bag.python.org> Author: collin.winter Date: Wed Jul 11 17:11:45 2007 New Revision: 56275 Modified: sandbox/trunk/2to3/fixes/fix_dict.py sandbox/trunk/2to3/fixes/util.py Log: Add a Dot() node macro. Modified: sandbox/trunk/2to3/fixes/fix_dict.py ============================================================================== --- sandbox/trunk/2to3/fixes/fix_dict.py (original) +++ sandbox/trunk/2to3/fixes/fix_dict.py Wed Jul 11 17:11:45 2007 @@ -27,7 +27,7 @@ import patcomp from pgen2 import token from fixes import basefix -from fixes.util import Name, Call, LParen, RParen, ArgList +from fixes.util import Name, Call, LParen, RParen, ArgList, Dot class FixDict(basefix.BaseFix): PATTERN = """ @@ -54,7 +54,7 @@ tail = [n.clone() for n in tail] special = not tail and self.in_special_context(node, isiter) args = head + [pytree.Node(syms.trailer, - [pytree.Leaf(token.DOT, '.'), + [Dot(), Name(method_name, prefix=method.get_prefix())]), results["parens"].clone()] Modified: sandbox/trunk/2to3/fixes/util.py ============================================================================== --- sandbox/trunk/2to3/fixes/util.py (original) +++ sandbox/trunk/2to3/fixes/util.py Wed Jul 11 17:11:45 2007 @@ -41,13 +41,16 @@ def Attr(obj, attr): """A node tuple for obj.attr""" - return [obj, - Node(syms.trailer, [Leaf(token.DOT, '.'), attr])] + return [obj, Node(syms.trailer, [Dot(), attr])] def Comma(): """A comma leaf""" return Leaf(token.COMMA, ",") +def Dot(): + """A period (.) leaf""" + return Leaf(token.DOT, ".") + def ArgList(args, lparen=LParen(), rparen=RParen()): """A parenthesised argument list, used by Call()""" return Node(syms.trailer, From python-checkins at python.org Wed Jul 11 19:34:03 2007 From: python-checkins at python.org (phillip.eby) Date: Wed, 11 Jul 2007 19:34:03 +0200 (CEST) Subject: [Python-checkins] r56276 - in sandbox/trunk/setuptools/setuptools: __init__.py command/egg_info.py Message-ID: <20070711173403.132BA1E400A@bag.python.org> Author: phillip.eby Date: Wed Jul 11 19:34:02 2007 New Revision: 56276 Modified: sandbox/trunk/setuptools/setuptools/__init__.py sandbox/trunk/setuptools/setuptools/command/egg_info.py Log: Fix distutils.filelist.findall() crashing on broken symlinks. Fix egg_info failures on new, uncommitted SVN directories. Modified: sandbox/trunk/setuptools/setuptools/__init__.py ============================================================================== --- sandbox/trunk/setuptools/setuptools/__init__.py (original) +++ sandbox/trunk/setuptools/setuptools/__init__.py Wed Jul 11 19:34:02 2007 @@ -62,3 +62,21 @@ import distutils.core distutils.core.Command = Command # we can't patch distutils.cmd, alas + +def findall(dir = os.curdir): + """Find all files under 'dir' and return the list of full filenames + (relative to 'dir'). + """ + all_files = [] + for base, dirs, files in os.walk(dir): + if base!=os.curdir: + files = [os.path.join(base, f) for f in files] + all_files.extend(filter(os.path.isfile, files)) + return all_files + +import distutils.filelist +distutils.filelist.findall = findall # fix findall bug in distutils. + + + + Modified: sandbox/trunk/setuptools/setuptools/command/egg_info.py ============================================================================== --- sandbox/trunk/setuptools/setuptools/command/egg_info.py (original) +++ sandbox/trunk/setuptools/setuptools/command/egg_info.py Wed Jul 11 19:34:02 2007 @@ -221,10 +221,10 @@ data = map(str.splitlines,data.split('\n\x0c\n')) del data[0][0] # get rid of the '8' dirurl = data[0][3] - localrev = max([int(d[9]) for d in data if len(d)>9 and d[9]]) + localrev = max([int(d[9]) for d in data if len(d)>9 and d[9]]+[0]) elif data.startswith(' Author: phillip.eby Date: Wed Jul 11 19:37:17 2007 New Revision: 56277 Modified: sandbox/branches/setuptools-0.6/setuptools.egg-info/entry_points.txt sandbox/branches/setuptools-0.6/setuptools/__init__.py sandbox/branches/setuptools-0.6/setuptools/command/egg_info.py Log: Fix distutils.filelist.findall() crashing on broken symlinks. Fix egg_info failures on new, uncommitted SVN directories. Modified: sandbox/branches/setuptools-0.6/setuptools.egg-info/entry_points.txt ============================================================================== --- sandbox/branches/setuptools-0.6/setuptools.egg-info/entry_points.txt (original) +++ sandbox/branches/setuptools-0.6/setuptools.egg-info/entry_points.txt Wed Jul 11 19:37:17 2007 @@ -1,38 +1,3 @@ -[distutils.setup_keywords] -dependency_links = setuptools.dist:assert_string_list -entry_points = setuptools.dist:check_entry_points -extras_require = setuptools.dist:check_extras -package_data = setuptools.dist:check_package_data -install_requires = setuptools.dist:check_requirements -include_package_data = setuptools.dist:assert_bool -exclude_package_data = setuptools.dist:check_package_data -namespace_packages = setuptools.dist:check_nsp -test_suite = setuptools.dist:check_test_suite -eager_resources = setuptools.dist:assert_string_list -zip_safe = setuptools.dist:assert_bool -test_loader = setuptools.dist:check_importable -tests_require = setuptools.dist:check_requirements - -[setuptools.file_finders] -svn_cvs = setuptools.command.sdist:_default_revctrl - -[egg_info.writers] -dependency_links.txt = setuptools.command.egg_info:overwrite_arg -requires.txt = setuptools.command.egg_info:write_requirements -PKG-INFO = setuptools.command.egg_info:write_pkg_info -eager_resources.txt = setuptools.command.egg_info:overwrite_arg -top_level.txt = setuptools.command.egg_info:write_toplevel_names -namespace_packages.txt = setuptools.command.egg_info:overwrite_arg -entry_points.txt = setuptools.command.egg_info:write_entries -depends.txt = setuptools.command.egg_info:warn_depends_obsolete - -[console_scripts] -easy_install = setuptools.command.easy_install:main -easy_install-2.4 = setuptools.command.easy_install:main - -[setuptools.installation] -eggsecutable = setuptools.command.easy_install:bootstrap - [distutils.commands] bdist_rpm = setuptools.command.bdist_rpm:bdist_rpm rotate = setuptools.command.rotate:rotate @@ -42,7 +7,6 @@ saveopts = setuptools.command.saveopts:saveopts egg_info = setuptools.command.egg_info:egg_info register = setuptools.command.register:register -upload = setuptools.command.upload:upload install_egg_info = setuptools.command.install_egg_info:install_egg_info alias = setuptools.command.alias:alias easy_install = setuptools.command.easy_install:easy_install @@ -54,3 +18,39 @@ install_lib = setuptools.command.install_lib:install_lib build_ext = setuptools.command.build_ext:build_ext sdist = setuptools.command.sdist:sdist + +[egg_info.writers] +dependency_links.txt = setuptools.command.egg_info:overwrite_arg +requires.txt = setuptools.command.egg_info:write_requirements +PKG-INFO = setuptools.command.egg_info:write_pkg_info +eager_resources.txt = setuptools.command.egg_info:overwrite_arg +top_level.txt = setuptools.command.egg_info:write_toplevel_names +namespace_packages.txt = setuptools.command.egg_info:overwrite_arg +entry_points.txt = setuptools.command.egg_info:write_entries +depends.txt = setuptools.command.egg_info:warn_depends_obsolete + +[console_scripts] +easy_install = setuptools.command.easy_install:main +easy_install-2.5 = setuptools.command.easy_install:main + +[setuptools.file_finders] +svn_cvs = setuptools.command.sdist:_default_revctrl + +[distutils.setup_keywords] +dependency_links = setuptools.dist:assert_string_list +entry_points = setuptools.dist:check_entry_points +extras_require = setuptools.dist:check_extras +package_data = setuptools.dist:check_package_data +install_requires = setuptools.dist:check_requirements +include_package_data = setuptools.dist:assert_bool +exclude_package_data = setuptools.dist:check_package_data +namespace_packages = setuptools.dist:check_nsp +test_suite = setuptools.dist:check_test_suite +eager_resources = setuptools.dist:assert_string_list +zip_safe = setuptools.dist:assert_bool +test_loader = setuptools.dist:check_importable +tests_require = setuptools.dist:check_requirements + +[setuptools.installation] +eggsecutable = setuptools.command.easy_install:bootstrap + Modified: sandbox/branches/setuptools-0.6/setuptools/__init__.py ============================================================================== --- sandbox/branches/setuptools-0.6/setuptools/__init__.py (original) +++ sandbox/branches/setuptools-0.6/setuptools/__init__.py Wed Jul 11 19:37:17 2007 @@ -63,19 +63,19 @@ import distutils.core distutils.core.Command = Command # we can't patch distutils.cmd, alas +def findall(dir = os.curdir): + """Find all files under 'dir' and return the list of full filenames + (relative to 'dir'). + """ + all_files = [] + for base, dirs, files in os.walk(dir): + if base!=os.curdir: + files = [os.path.join(base, f) for f in files] + all_files.extend(filter(os.path.isfile, files)) + return all_files - - - - - - - - - - - - +import distutils.filelist +distutils.filelist.findall = findall # fix findall bug in distutils. Modified: sandbox/branches/setuptools-0.6/setuptools/command/egg_info.py ============================================================================== --- sandbox/branches/setuptools-0.6/setuptools/command/egg_info.py (original) +++ sandbox/branches/setuptools-0.6/setuptools/command/egg_info.py Wed Jul 11 19:37:17 2007 @@ -221,10 +221,10 @@ data = map(str.splitlines,data.split('\n\x0c\n')) del data[0][0] # get rid of the '8' dirurl = data[0][3] - localrev = max([int(d[9]) for d in data if len(d)>9 and d[9]]) + localrev = max([int(d[9]) for d in data if len(d)>9 and d[9]]+[0]) elif data.startswith(' Author: phillip.eby Date: Wed Jul 11 19:38:18 2007 New Revision: 56278 Modified: sandbox/branches/setuptools-0.6/setuptools.txt Log: Update release notes w/fixes. Modified: sandbox/branches/setuptools-0.6/setuptools.txt ============================================================================== --- sandbox/branches/setuptools-0.6/setuptools.txt (original) +++ sandbox/branches/setuptools-0.6/setuptools.txt Wed Jul 11 19:38:18 2007 @@ -2611,6 +2611,11 @@ Release Notes/Change History ---------------------------- +0.6c7 + + * Fixed ``distutils.filelist.findall()`` crashing on broken symlinks, and + ``egg_info`` command failing on new, uncommitted SVN directories. + 0.6c6 * Added ``--egg-path`` option to ``develop`` command, allowing you to force ``.egg-link`` files to use relative paths (allowing them to be shared across From python-checkins at python.org Wed Jul 11 21:41:49 2007 From: python-checkins at python.org (georg.brandl) Date: Wed, 11 Jul 2007 21:41:49 +0200 (CEST) Subject: [Python-checkins] r56280 - python/trunk/Include/opcode.h Message-ID: <20070711194149.78A201E400E@bag.python.org> Author: georg.brandl Date: Wed Jul 11 21:41:49 2007 New Revision: 56280 Modified: python/trunk/Include/opcode.h Log: Fix #1752132: wrong comment in opcode description. Modified: python/trunk/Include/opcode.h ============================================================================== --- python/trunk/Include/opcode.h (original) +++ python/trunk/Include/opcode.h Wed Jul 11 21:41:49 2007 @@ -112,7 +112,7 @@ #define LOAD_GLOBAL 116 /* Index in name list */ #define CONTINUE_LOOP 119 /* Start of loop (absolute) */ -#define SETUP_LOOP 120 /* Target address (absolute) */ +#define SETUP_LOOP 120 /* Target address (relative) */ #define SETUP_EXCEPT 121 /* "" */ #define SETUP_FINALLY 122 /* "" */ From python-checkins at python.org Wed Jul 11 21:41:53 2007 From: python-checkins at python.org (georg.brandl) Date: Wed, 11 Jul 2007 21:41:53 +0200 (CEST) Subject: [Python-checkins] r56281 - python/branches/release25-maint/Include/opcode.h Message-ID: <20070711194153.5A0AB1E400C@bag.python.org> Author: georg.brandl Date: Wed Jul 11 21:41:53 2007 New Revision: 56281 Modified: python/branches/release25-maint/Include/opcode.h Log: Fix #1752132: wrong comment in opcode description. (backport from rev. 56280) Modified: python/branches/release25-maint/Include/opcode.h ============================================================================== --- python/branches/release25-maint/Include/opcode.h (original) +++ python/branches/release25-maint/Include/opcode.h Wed Jul 11 21:41:53 2007 @@ -112,7 +112,7 @@ #define LOAD_GLOBAL 116 /* Index in name list */ #define CONTINUE_LOOP 119 /* Start of loop (absolute) */ -#define SETUP_LOOP 120 /* Target address (absolute) */ +#define SETUP_LOOP 120 /* Target address (relative) */ #define SETUP_EXCEPT 121 /* "" */ #define SETUP_FINALLY 122 /* "" */ From buildbot at python.org Wed Jul 11 22:00:19 2007 From: buildbot at python.org (buildbot at python.org) Date: Wed, 11 Jul 2007 20:00:19 +0000 Subject: [Python-checkins] buildbot warnings in x86 W2k trunk Message-ID: <20070711200032.E8F5C1E4005@bag.python.org> The Buildbot has detected a new failure of x86 W2k trunk. Full details are available at: http://www.python.org/dev/buildbot/all/x86%2520W2k%2520trunk/builds/388 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: georg.brandl Build had warnings: warnings failed slave lost sincerely, -The Buildbot From buildbot at python.org Wed Jul 11 22:20:22 2007 From: buildbot at python.org (buildbot at python.org) Date: Wed, 11 Jul 2007 20:20:22 +0000 Subject: [Python-checkins] buildbot warnings in x86 XP trunk Message-ID: <20070711202023.108641E400D@bag.python.org> The Buildbot has detected a new failure of x86 XP trunk. Full details are available at: http://www.python.org/dev/buildbot/all/x86%2520XP%2520trunk/builds/504 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: georg.brandl Build had warnings: warnings failed slave lost sincerely, -The Buildbot From buildbot at python.org Wed Jul 11 22:30:31 2007 From: buildbot at python.org (buildbot at python.org) Date: Wed, 11 Jul 2007 20:30:31 +0000 Subject: [Python-checkins] buildbot warnings in ppc Debian unstable trunk Message-ID: <20070711203031.71D0C1E4005@bag.python.org> The Buildbot has detected a new failure of ppc Debian unstable trunk. Full details are available at: http://www.python.org/dev/buildbot/all/ppc%2520Debian%2520unstable%2520trunk/builds/41 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: georg.brandl Build had warnings: warnings test Excerpt from the test logfile: Traceback (most recent call last): File "/home/pybot/buildarea/trunk.klose-debian-ppc/build/Lib/threading.py", line 465, in __bootstrap self.run() File "/home/pybot/buildarea/trunk.klose-debian-ppc/build/Lib/test/test_socketserver.py", line 93, in run svr.serve_a_few() File "/home/pybot/buildarea/trunk.klose-debian-ppc/build/Lib/test/test_socketserver.py", line 35, in serve_a_few self.handle_request() File "/home/pybot/buildarea/trunk.klose-debian-ppc/build/Lib/SocketServer.py", line 224, in handle_request self.handle_error(request, client_address) File "/home/pybot/buildarea/trunk.klose-debian-ppc/build/Lib/SocketServer.py", line 222, in handle_request self.process_request(request, client_address) File "/home/pybot/buildarea/trunk.klose-debian-ppc/build/Lib/SocketServer.py", line 429, in process_request self.collect_children() File "/home/pybot/buildarea/trunk.klose-debian-ppc/build/Lib/SocketServer.py", line 425, in collect_children self.active_children.remove(pid) ValueError: list.remove(x): x not in list 1 test failed: test_socketserver Traceback (most recent call last): File "./Lib/test/regrtest.py", line 557, in runtest_inner indirect_test() File "/home/pybot/buildarea/trunk.klose-debian-ppc/build/Lib/test/test_socketserver.py", line 216, in test_main testall() File "/home/pybot/buildarea/trunk.klose-debian-ppc/build/Lib/test/test_socketserver.py", line 199, in testall testloop(socket.AF_INET, tcpservers, MyStreamHandler, teststream) File "/home/pybot/buildarea/trunk.klose-debian-ppc/build/Lib/test/test_socketserver.py", line 148, in testloop testfunc(proto, addr) File "/home/pybot/buildarea/trunk.klose-debian-ppc/build/Lib/test/test_socketserver.py", line 62, in teststream s.connect(addr) File "", line 1, in connect error: (111, 'Connection refused') make: *** [buildbottest] Error 1 sincerely, -The Buildbot From python-checkins at python.org Thu Jul 12 10:05:45 2007 From: python-checkins at python.org (georg.brandl) Date: Thu, 12 Jul 2007 10:05:45 +0200 (CEST) Subject: [Python-checkins] r56293 - in python/trunk/Lib: test/test_urllib2.py urllib2.py Message-ID: <20070712080545.AB6061E400A@bag.python.org> Author: georg.brandl Date: Thu Jul 12 10:05:45 2007 New Revision: 56293 Modified: python/trunk/Lib/test/test_urllib2.py python/trunk/Lib/urllib2.py Log: Patch #1752270, #1750931: complain if urllib2 add_handler called without handler. Modified: python/trunk/Lib/test/test_urllib2.py ============================================================================== --- python/trunk/Lib/test/test_urllib2.py (original) +++ python/trunk/Lib/test/test_urllib2.py Thu Jul 12 10:05:45 2007 @@ -381,6 +381,12 @@ class OpenerDirectorTests(unittest.TestCase): + def test_add_non_handler(self): + class NonHandler(object): + pass + self.assertRaises(TypeError, + OpenerDirector().add_handler, NonHandler()) + def test_badly_named_methods(self): # test work-around for three methods that accidentally follow the # naming conventions for handler methods Modified: python/trunk/Lib/urllib2.py ============================================================================== --- python/trunk/Lib/urllib2.py (original) +++ python/trunk/Lib/urllib2.py Thu Jul 12 10:05:45 2007 @@ -295,6 +295,10 @@ self.process_request = {} def add_handler(self, handler): + if not hasattr(handler, "add_parent"): + raise TypeError("expected BaseHandler instance, got %r" % + type(handler)) + added = False for meth in dir(handler): if meth in ["redirect_request", "do_open", "proxy_open"]: From python-checkins at python.org Thu Jul 12 10:05:48 2007 From: python-checkins at python.org (georg.brandl) Date: Thu, 12 Jul 2007 10:05:48 +0200 (CEST) Subject: [Python-checkins] r56294 - in python/branches/release25-maint/Lib: test/test_urllib2.py urllib2.py Message-ID: <20070712080548.ADDAE1E400E@bag.python.org> Author: georg.brandl Date: Thu Jul 12 10:05:48 2007 New Revision: 56294 Modified: python/branches/release25-maint/Lib/test/test_urllib2.py python/branches/release25-maint/Lib/urllib2.py Log: Patch #1752270, #1750931: complain if urllib2 add_handler called without handler. (backport from rev. 56293) Modified: python/branches/release25-maint/Lib/test/test_urllib2.py ============================================================================== --- python/branches/release25-maint/Lib/test/test_urllib2.py (original) +++ python/branches/release25-maint/Lib/test/test_urllib2.py Thu Jul 12 10:05:48 2007 @@ -381,6 +381,12 @@ class OpenerDirectorTests(unittest.TestCase): + def test_add_non_handler(self): + class NonHandler(object): + pass + self.assertRaises(TypeError, + OpenerDirector().add_handler, NonHandler()) + def test_badly_named_methods(self): # test work-around for three methods that accidentally follow the # naming conventions for handler methods Modified: python/branches/release25-maint/Lib/urllib2.py ============================================================================== --- python/branches/release25-maint/Lib/urllib2.py (original) +++ python/branches/release25-maint/Lib/urllib2.py Thu Jul 12 10:05:48 2007 @@ -298,6 +298,10 @@ self.process_request = {} def add_handler(self, handler): + if not hasattr(handler, "add_parent"): + raise TypeError("expected BaseHandler instance, got %r" % + type(handler)) + added = False for meth in dir(handler): if meth in ["redirect_request", "do_open", "proxy_open"]: From python-checkins at python.org Thu Jul 12 10:11:30 2007 From: python-checkins at python.org (georg.brandl) Date: Thu, 12 Jul 2007 10:11:30 +0200 (CEST) Subject: [Python-checkins] r56296 - python/trunk/Lib/inspect.py Message-ID: <20070712081130.3DF991E400B@bag.python.org> Author: georg.brandl Date: Thu Jul 12 10:11:29 2007 New Revision: 56296 Modified: python/trunk/Lib/inspect.py Log: Patch #1739696: use code.co_code only if really necessary Modified: python/trunk/Lib/inspect.py ============================================================================== --- python/trunk/Lib/inspect.py (original) +++ python/trunk/Lib/inspect.py Thu Jul 12 10:11:29 2007 @@ -679,7 +679,6 @@ if not iscode(co): raise TypeError('arg is not a code object') - code = co.co_code nargs = co.co_argcount names = co.co_varnames args = list(names[:nargs]) @@ -689,12 +688,12 @@ for i in range(nargs): if args[i][:1] in ('', '.'): stack, remain, count = [], [], [] - while step < len(code): - op = ord(code[step]) + while step < len(co.co_code): + op = ord(co.co_code[step]) step = step + 1 if op >= dis.HAVE_ARGUMENT: opname = dis.opname[op] - value = ord(code[step]) + ord(code[step+1])*256 + value = ord(co.co_code[step]) + ord(co.co_code[step+1])*256 step = step + 2 if opname in ('UNPACK_TUPLE', 'UNPACK_SEQUENCE'): remain.append(value) From python-checkins at python.org Thu Jul 12 10:11:33 2007 From: python-checkins at python.org (georg.brandl) Date: Thu, 12 Jul 2007 10:11:33 +0200 (CEST) Subject: [Python-checkins] r56297 - python/branches/release25-maint/Lib/inspect.py Message-ID: <20070712081133.437E51E400B@bag.python.org> Author: georg.brandl Date: Thu Jul 12 10:11:32 2007 New Revision: 56297 Modified: python/branches/release25-maint/Lib/inspect.py Log: Patch #1739696: use code.co_code only if really necessary (backport from rev. 56296) Modified: python/branches/release25-maint/Lib/inspect.py ============================================================================== --- python/branches/release25-maint/Lib/inspect.py (original) +++ python/branches/release25-maint/Lib/inspect.py Thu Jul 12 10:11:32 2007 @@ -679,7 +679,6 @@ if not iscode(co): raise TypeError('arg is not a code object') - code = co.co_code nargs = co.co_argcount names = co.co_varnames args = list(names[:nargs]) @@ -689,12 +688,12 @@ for i in range(nargs): if args[i][:1] in ('', '.'): stack, remain, count = [], [], [] - while step < len(code): - op = ord(code[step]) + while step < len(co.co_code): + op = ord(co.co_code[step]) step = step + 1 if op >= dis.HAVE_ARGUMENT: opname = dis.opname[op] - value = ord(code[step]) + ord(code[step+1])*256 + value = ord(co.co_code[step]) + ord(co.co_code[step+1])*256 step = step + 2 if opname in ('UNPACK_TUPLE', 'UNPACK_SEQUENCE'): remain.append(value) From python-checkins at python.org Thu Jul 12 10:38:01 2007 From: python-checkins at python.org (georg.brandl) Date: Thu, 12 Jul 2007 10:38:01 +0200 (CEST) Subject: [Python-checkins] r56298 - in python/trunk: Lib/test/test_format.py Misc/NEWS Objects/stringobject.c Objects/unicodeobject.c Message-ID: <20070712083801.A686A1E400B@bag.python.org> Author: georg.brandl Date: Thu Jul 12 10:38:00 2007 New Revision: 56298 Modified: python/trunk/Lib/test/test_format.py python/trunk/Misc/NEWS python/trunk/Objects/stringobject.c python/trunk/Objects/unicodeobject.c Log: Patch #1673759: add a missing overflow check when formatting floats with %G. Modified: python/trunk/Lib/test/test_format.py ============================================================================== --- python/trunk/Lib/test/test_format.py (original) +++ python/trunk/Lib/test/test_format.py Thu Jul 12 10:38:00 2007 @@ -9,6 +9,7 @@ # test on unicode strings as well overflowok = 1 +overflowrequired = 0 def testformat(formatstr, args, output=None): if verbose: @@ -25,11 +26,16 @@ if verbose: print 'overflow (this is fine)' else: - if output and result != output: + if overflowrequired: if verbose: print 'no' - print "%s %% %s == %s != %s" %\ - (repr(formatstr), repr(args), repr(result), repr(output)) + print "overflow expected on %s %% %s" % \ + (repr(formatstr), repr(args)) + elif output and result != output: + if verbose: + print 'no' + print "%s %% %s == %s != %s" % \ + (repr(formatstr), repr(args), repr(result), repr(output)) else: if verbose: print 'yes' @@ -57,6 +63,14 @@ # test some ridiculously large precision, expect overflow testboth('%12.*f', (123456, 1.0)) +# check for internal overflow validation on length of precision +overflowrequired = 1 +testboth("%#.*g", (110, -1.e+100/3.)) +testboth("%#.*G", (110, -1.e+100/3.)) +testboth("%#.*f", (110, -1.e+100/3.)) +testboth("%#.*F", (110, -1.e+100/3.)) +overflowrequired = 0 + # Formatting of long integers. Overflow is not ok overflowok = 0 testboth("%x", 10L, "a") Modified: python/trunk/Misc/NEWS ============================================================================== --- python/trunk/Misc/NEWS (original) +++ python/trunk/Misc/NEWS Thu Jul 12 10:38:00 2007 @@ -12,6 +12,9 @@ Core and builtins ----------------- +- Patch #1673759: add a missing overflow check when formatting floats + with %G. + - Patch #1733960: Allow T_LONGLONG to accept ints. - T_PYSSIZET can now be used in PyMemberDef lists for Py_ssize_t members. Modified: python/trunk/Objects/stringobject.c ============================================================================== --- python/trunk/Objects/stringobject.c (original) +++ python/trunk/Objects/stringobject.c Thu Jul 12 10:38:00 2007 @@ -4198,7 +4198,8 @@ always given), therefore increase the length by one. */ - if ((type == 'g' && buflen <= (size_t)10 + (size_t)prec) || + if (((type == 'g' || type == 'G') && + buflen <= (size_t)10 + (size_t)prec) || (type == 'f' && buflen <= (size_t)53 + (size_t)prec)) { PyErr_SetString(PyExc_OverflowError, "formatted float is too long (precision too large?)"); Modified: python/trunk/Objects/unicodeobject.c ============================================================================== --- python/trunk/Objects/unicodeobject.c (original) +++ python/trunk/Objects/unicodeobject.c Thu Jul 12 10:38:00 2007 @@ -7294,7 +7294,8 @@ always given), therefore increase the length by one. */ - if ((type == 'g' && buflen <= (size_t)10 + (size_t)prec) || + if (((type == 'g' || type == 'G') && + buflen <= (size_t)10 + (size_t)prec) || (type == 'f' && buflen <= (size_t)53 + (size_t)prec)) { PyErr_SetString(PyExc_OverflowError, "formatted float is too long (precision too large?)"); From python-checkins at python.org Thu Jul 12 10:38:05 2007 From: python-checkins at python.org (georg.brandl) Date: Thu, 12 Jul 2007 10:38:05 +0200 (CEST) Subject: [Python-checkins] r56299 - in python/branches/release25-maint: Lib/test/test_format.py Misc/NEWS Objects/stringobject.c Objects/unicodeobject.c Message-ID: <20070712083805.28FEB1E4010@bag.python.org> Author: georg.brandl Date: Thu Jul 12 10:38:04 2007 New Revision: 56299 Modified: python/branches/release25-maint/Lib/test/test_format.py python/branches/release25-maint/Misc/NEWS python/branches/release25-maint/Objects/stringobject.c python/branches/release25-maint/Objects/unicodeobject.c Log: Patch #1673759: add a missing overflow check when formatting floats with %G. (backport from rev. 56298) Modified: python/branches/release25-maint/Lib/test/test_format.py ============================================================================== --- python/branches/release25-maint/Lib/test/test_format.py (original) +++ python/branches/release25-maint/Lib/test/test_format.py Thu Jul 12 10:38:04 2007 @@ -9,6 +9,7 @@ # test on unicode strings as well overflowok = 1 +overflowrequired = 0 def testformat(formatstr, args, output=None): if verbose: @@ -25,11 +26,16 @@ if verbose: print 'overflow (this is fine)' else: - if output and result != output: + if overflowrequired: if verbose: print 'no' - print "%s %% %s == %s != %s" %\ - (repr(formatstr), repr(args), repr(result), repr(output)) + print "overflow expected on %s %% %s" % \ + (repr(formatstr), repr(args)) + elif output and result != output: + if verbose: + print 'no' + print "%s %% %s == %s != %s" % \ + (repr(formatstr), repr(args), repr(result), repr(output)) else: if verbose: print 'yes' @@ -57,6 +63,14 @@ # test some ridiculously large precision, expect overflow testboth('%12.*f', (123456, 1.0)) +# check for internal overflow validation on length of precision +overflowrequired = 1 +testboth("%#.*g", (110, -1.e+100/3.)) +testboth("%#.*G", (110, -1.e+100/3.)) +testboth("%#.*f", (110, -1.e+100/3.)) +testboth("%#.*F", (110, -1.e+100/3.)) +overflowrequired = 0 + # Formatting of long integers. Overflow is not ok overflowok = 0 testboth("%x", 10L, "a") Modified: python/branches/release25-maint/Misc/NEWS ============================================================================== --- python/branches/release25-maint/Misc/NEWS (original) +++ python/branches/release25-maint/Misc/NEWS Thu Jul 12 10:38:04 2007 @@ -12,6 +12,9 @@ Core and builtins ----------------- +- Patch #1673759: add a missing overflow check when formatting floats + with %G. + - Patch #1733960: Allow T_LONGLONG to accept ints. - Prevent expandtabs() on string and unicode objects from causing a segfault Modified: python/branches/release25-maint/Objects/stringobject.c ============================================================================== --- python/branches/release25-maint/Objects/stringobject.c (original) +++ python/branches/release25-maint/Objects/stringobject.c Thu Jul 12 10:38:04 2007 @@ -4188,7 +4188,8 @@ always given), therefore increase the length by one. */ - if ((type == 'g' && buflen <= (size_t)10 + (size_t)prec) || + if (((type == 'g' || type == 'G') && + buflen <= (size_t)10 + (size_t)prec) || (type == 'f' && buflen <= (size_t)53 + (size_t)prec)) { PyErr_SetString(PyExc_OverflowError, "formatted float is too long (precision too large?)"); Modified: python/branches/release25-maint/Objects/unicodeobject.c ============================================================================== --- python/branches/release25-maint/Objects/unicodeobject.c (original) +++ python/branches/release25-maint/Objects/unicodeobject.c Thu Jul 12 10:38:04 2007 @@ -7290,7 +7290,8 @@ always given), therefore increase the length by one. */ - if ((type == 'g' && buflen <= (size_t)10 + (size_t)prec) || + if (((type == 'g' || type == 'G') && + buflen <= (size_t)10 + (size_t)prec) || (type == 'f' && buflen <= (size_t)53 + (size_t)prec)) { PyErr_SetString(PyExc_OverflowError, "formatted float is too long (precision too large?)"); From buildbot at python.org Thu Jul 12 10:42:48 2007 From: buildbot at python.org (buildbot at python.org) Date: Thu, 12 Jul 2007 08:42:48 +0000 Subject: [Python-checkins] buildbot warnings in amd64 XP trunk Message-ID: <20070712084248.A35BA1E400A@bag.python.org> The Buildbot has detected a new failure of amd64 XP trunk. Full details are available at: http://www.python.org/dev/buildbot/all/amd64%2520XP%2520trunk/builds/39 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: georg.brandl Build had warnings: warnings test Excerpt from the test logfile: 2 tests failed: test_ctypes test_winsound ====================================================================== ERROR: test_extremes (test.test_winsound.BeepTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "C:\buildbot\trunk.heller-windows-amd64\build\lib\test\test_winsound.py", line 18, in test_extremes winsound.Beep(37, 75) RuntimeError: Failed to beep ====================================================================== ERROR: test_increasingfrequency (test.test_winsound.BeepTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "C:\buildbot\trunk.heller-windows-amd64\build\lib\test\test_winsound.py", line 23, in test_increasingfrequency winsound.Beep(i, 75) RuntimeError: Failed to beep ====================================================================== ERROR: test_alias_asterisk (test.test_winsound.PlaySoundTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "C:\buildbot\trunk.heller-windows-amd64\build\lib\test\test_winsound.py", line 64, in test_alias_asterisk winsound.PlaySound('SystemAsterisk', winsound.SND_ALIAS) RuntimeError: Failed to play sound ====================================================================== ERROR: test_alias_exclamation (test.test_winsound.PlaySoundTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "C:\buildbot\trunk.heller-windows-amd64\build\lib\test\test_winsound.py", line 74, in test_alias_exclamation winsound.PlaySound('SystemExclamation', winsound.SND_ALIAS) RuntimeError: Failed to play sound ====================================================================== ERROR: test_alias_exit (test.test_winsound.PlaySoundTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "C:\buildbot\trunk.heller-windows-amd64\build\lib\test\test_winsound.py", line 84, in test_alias_exit winsound.PlaySound('SystemExit', winsound.SND_ALIAS) RuntimeError: Failed to play sound ====================================================================== ERROR: test_alias_hand (test.test_winsound.PlaySoundTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "C:\buildbot\trunk.heller-windows-amd64\build\lib\test\test_winsound.py", line 94, in test_alias_hand winsound.PlaySound('SystemHand', winsound.SND_ALIAS) RuntimeError: Failed to play sound ====================================================================== ERROR: test_alias_question (test.test_winsound.PlaySoundTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "C:\buildbot\trunk.heller-windows-amd64\build\lib\test\test_winsound.py", line 104, in test_alias_question winsound.PlaySound('SystemQuestion', winsound.SND_ALIAS) RuntimeError: Failed to play sound sincerely, -The Buildbot From python-checkins at python.org Thu Jul 12 11:06:41 2007 From: python-checkins at python.org (georg.brandl) Date: Thu, 12 Jul 2007 11:06:41 +0200 (CEST) Subject: [Python-checkins] r56302 - python/trunk/Doc/lib/libtime.tex Message-ID: <20070712090641.C8B1E1E400A@bag.python.org> Author: georg.brandl Date: Thu Jul 12 11:06:41 2007 New Revision: 56302 Modified: python/trunk/Doc/lib/libtime.tex Log: Patch #1731659: improve time.strptime docs. Modified: python/trunk/Doc/lib/libtime.tex ============================================================================== --- python/trunk/Doc/lib/libtime.tex (original) +++ python/trunk/Doc/lib/libtime.tex Thu Jul 12 11:06:41 2007 @@ -309,15 +309,23 @@ \begin{funcdesc}{strptime}{string\optional{, format}} Parse a string representing a time according to a format. The return value is a \class{struct_time} as returned by \function{gmtime()} or -\function{localtime()}. The \var{format} parameter uses the same -directives as those used by \function{strftime()}; it defaults to -\code{"\%a \%b \%d \%H:\%M:\%S \%Y"} which matches the formatting -returned by \function{ctime()}. If \var{string} cannot be parsed -according to \var{format}, \exception{ValueError} is raised. If the -string to be parsed has excess data after parsing, -\exception{ValueError} is raised. The default values used to fill in -any missing data when more accurate values cannot be inferred are -\code{(1900, 1, 1, 0, 0, 0, 0, 1, -1)} . +\function{localtime()}. + +The \var{format} parameter uses the same directives as those used by +\function{strftime()}; it defaults to \code{"\%a \%b \%d \%H:\%M:\%S + \%Y"} which matches the formatting returned by \function{ctime()}. +If \var{string} cannot be parsed according to \var{format}, or if it +has excess data after parsing, \exception{ValueError} is raised. The +default values used to fill in any missing data when more accurate +values cannot be inferred are \code{(1900, 1, 1, 0, 0, 0, 0, 1, -1)}. + +For example: + +\begin{verbatim} +>>> import time +>>> time.strptime("30 Nov 00", "%d %b %y") +(2000, 11, 30, 0, 0, 0, 3, 335, -1) +\end{verbatim} Support for the \code{\%Z} directive is based on the values contained in \code{tzname} and whether \code{daylight} is true. Because of this, From python-checkins at python.org Thu Jul 12 11:06:44 2007 From: python-checkins at python.org (georg.brandl) Date: Thu, 12 Jul 2007 11:06:44 +0200 (CEST) Subject: [Python-checkins] r56303 - python/branches/release25-maint/Doc/lib/libtime.tex Message-ID: <20070712090644.110091E400A@bag.python.org> Author: georg.brandl Date: Thu Jul 12 11:06:43 2007 New Revision: 56303 Modified: python/branches/release25-maint/Doc/lib/libtime.tex Log: Patch #1731659: improve time.strptime docs. (backport from rev. 56302) Modified: python/branches/release25-maint/Doc/lib/libtime.tex ============================================================================== --- python/branches/release25-maint/Doc/lib/libtime.tex (original) +++ python/branches/release25-maint/Doc/lib/libtime.tex Thu Jul 12 11:06:43 2007 @@ -309,15 +309,23 @@ \begin{funcdesc}{strptime}{string\optional{, format}} Parse a string representing a time according to a format. The return value is a \class{struct_time} as returned by \function{gmtime()} or -\function{localtime()}. The \var{format} parameter uses the same -directives as those used by \function{strftime()}; it defaults to -\code{"\%a \%b \%d \%H:\%M:\%S \%Y"} which matches the formatting -returned by \function{ctime()}. If \var{string} cannot be parsed -according to \var{format}, \exception{ValueError} is raised. If the -string to be parsed has excess data after parsing, -\exception{ValueError} is raised. The default values used to fill in -any missing data when more accurate values cannot be inferred are -\code{(1900, 1, 1, 0, 0, 0, 0, 1, -1)} . +\function{localtime()}. + +The \var{format} parameter uses the same directives as those used by +\function{strftime()}; it defaults to \code{"\%a \%b \%d \%H:\%M:\%S + \%Y"} which matches the formatting returned by \function{ctime()}. +If \var{string} cannot be parsed according to \var{format}, or if it +has excess data after parsing, \exception{ValueError} is raised. The +default values used to fill in any missing data when more accurate +values cannot be inferred are \code{(1900, 1, 1, 0, 0, 0, 0, 1, -1)}. + +For example: + +\begin{verbatim} +>>> import time +>>> time.strptime("30 Nov 00", "%d %b %y") +(2000, 11, 30, 0, 0, 0, 3, 335, -1) +\end{verbatim} Support for the \code{\%Z} directive is based on the values contained in \code{tzname} and whether \code{daylight} is true. Because of this, From buildbot at python.org Thu Jul 12 11:16:48 2007 From: buildbot at python.org (buildbot at python.org) Date: Thu, 12 Jul 2007 09:16:48 +0000 Subject: [Python-checkins] buildbot warnings in alpha Tru64 5.1 2.5 Message-ID: <20070712091648.509A61E400A@bag.python.org> The Buildbot has detected a new failure of alpha Tru64 5.1 2.5. Full details are available at: http://www.python.org/dev/buildbot/all/alpha%2520Tru64%25205.1%25202.5/builds/278 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch branches/release25-maint] HEAD Blamelist: georg.brandl Build had warnings: warnings test Excerpt from the test logfile: 1 test failed: test_socket ====================================================================== FAIL: testInterruptedTimeout (test.test_socket.TCPTimeoutTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/net/taipan/scratch1/nnorwitz/python/2.5.norwitz-tru64/build/Lib/test/test_socket.py", line 879, in testInterruptedTimeout self.fail("got Alarm in wrong place") AssertionError: got Alarm in wrong place sincerely, -The Buildbot From python-checkins at python.org Thu Jul 12 11:24:05 2007 From: python-checkins at python.org (georg.brandl) Date: Thu, 12 Jul 2007 11:24:05 +0200 (CEST) Subject: [Python-checkins] r56304 - python/trunk/Lib/test/regrtest.py Message-ID: <20070712092405.452B61E400A@bag.python.org> Author: georg.brandl Date: Thu Jul 12 11:24:04 2007 New Revision: 56304 Modified: python/trunk/Lib/test/regrtest.py Log: Patch #1731169: clean up expected skips list. Modified: python/trunk/Lib/test/regrtest.py ============================================================================== --- python/trunk/Lib/test/regrtest.py (original) +++ python/trunk/Lib/test/regrtest.py Thu Jul 12 11:24:04 2007 @@ -806,17 +806,16 @@ # test_timeout # Controlled by test_timeout.skip_expected. Requires the network # resource and a socket module. +# +# Tests that are expected to be skipped everywhere except on one platform +# are also handled separately. _expectations = { 'win32': """ test__locale - test_applesingle - test_al test_bsddb185 test_bsddb3 - test_cd - test_cl test_commands test_crypt test_curses @@ -825,14 +824,10 @@ test_fcntl test_fork1 test_gdbm - test_gl test_grp - test_imgfile test_ioctl test_largefile - test_linuxaudiodev test_mhlib - test_nis test_openpty test_ossaudiodev test_poll @@ -841,7 +836,6 @@ test_pwd test_resource test_signal - test_sunaudiodev test_threadsignals test_timing test_wait3 @@ -849,34 +843,19 @@ """, 'linux2': """ - test_al - test_applesingle test_bsddb185 - test_cd - test_cl test_curses test_dl - test_gl - test_imgfile test_largefile - test_linuxaudiodev - test_nis - test_ntpath test_ossaudiodev - test_sqlite - test_startfile - test_sunaudiodev """, 'mac': """ - test_al test_atexit test_bsddb test_bsddb185 test_bsddb3 test_bz2 - test_cd - test_cl test_commands test_crypt test_curses @@ -884,16 +863,11 @@ test_dl test_fcntl test_fork1 - test_gl test_grp test_ioctl - test_imgfile test_largefile - test_linuxaudiodev test_locale test_mmap - test_nis - test_ntpath test_openpty test_ossaudiodev test_poll @@ -904,88 +878,49 @@ test_pwd test_resource test_signal - test_sqlite - test_startfile - test_sunaudiodev test_sundry test_tarfile test_timing """, 'unixware7': """ - test_al - test_applesingle test_bsddb test_bsddb185 - test_cd - test_cl test_dl - test_gl - test_imgfile test_largefile - test_linuxaudiodev test_minidom - test_nis - test_ntpath test_openpty test_pyexpat test_sax - test_startfile - test_sqlite - test_sunaudiodev test_sundry """, 'openunix8': """ - test_al - test_applesingle test_bsddb test_bsddb185 - test_cd - test_cl test_dl - test_gl - test_imgfile test_largefile - test_linuxaudiodev test_minidom - test_nis - test_ntpath test_openpty test_pyexpat test_sax - test_sqlite - test_startfile - test_sunaudiodev test_sundry """, 'sco_sv3': """ - test_al - test_applesingle test_asynchat test_bsddb test_bsddb185 - test_cd - test_cl test_dl test_fork1 test_gettext - test_gl - test_imgfile test_largefile - test_linuxaudiodev test_locale test_minidom - test_nis - test_ntpath test_openpty test_pyexpat test_queue test_sax - test_sqlite - test_startfile - test_sunaudiodev test_sundry test_thread test_threaded_import @@ -994,15 +929,11 @@ """, 'riscos': """ - test_al - test_applesingle test_asynchat test_atexit test_bsddb test_bsddb185 test_bsddb3 - test_cd - test_cl test_commands test_crypt test_dbm @@ -1010,24 +941,16 @@ test_fcntl test_fork1 test_gdbm - test_gl test_grp - test_imgfile test_largefile - test_linuxaudiodev test_locale test_mmap - test_nis - test_ntpath test_openpty test_poll test_popen2 test_pty test_pwd test_strop - test_sqlite - test_startfile - test_sunaudiodev test_sundry test_thread test_threaded_import @@ -1038,274 +961,143 @@ 'darwin': """ test__locale - test_al test_bsddb test_bsddb3 - test_cd - test_cl test_curses test_gdbm - test_gl - test_imgfile test_largefile - test_linuxaudiodev test_locale test_minidom - test_nis - test_ntpath test_ossaudiodev test_poll - test_sqlite - test_startfile - test_sunaudiodev """, 'sunos5': """ - test_al - test_applesingle test_bsddb test_bsddb185 - test_cd - test_cl test_curses test_dbm test_gdbm - test_gl test_gzip - test_imgfile - test_linuxaudiodev test_openpty - test_sqlite - test_startfile test_zipfile test_zlib """, 'hp-ux11': """ - test_al - test_applesingle test_bsddb test_bsddb185 - test_cd - test_cl test_curses test_dl test_gdbm - test_gl test_gzip - test_imgfile test_largefile - test_linuxaudiodev test_locale test_minidom - test_nis - test_ntpath test_openpty test_pyexpat test_sax - test_sqlite - test_startfile - test_sunaudiodev test_zipfile test_zlib """, 'atheos': """ - test_al - test_applesingle test_bsddb185 - test_cd - test_cl test_curses test_dl test_gdbm - test_gl - test_imgfile test_largefile - test_linuxaudiodev test_locale test_mhlib test_mmap - test_nis test_poll test_popen2 test_resource - test_sqlite - test_startfile - test_sunaudiodev """, 'cygwin': """ - test_al - test_applesingle test_bsddb185 test_bsddb3 - test_cd - test_cl test_curses test_dbm - test_gl - test_imgfile test_ioctl test_largefile - test_linuxaudiodev test_locale - test_nis test_ossaudiodev test_socketserver - test_sqlite - test_sunaudiodev """, 'os2emx': """ - test_al - test_applesingle test_audioop test_bsddb185 test_bsddb3 - test_cd - test_cl test_commands test_curses test_dl - test_gl - test_imgfile test_largefile - test_linuxaudiodev test_mhlib test_mmap - test_nis test_openpty test_ossaudiodev test_pty test_resource test_signal - test_sqlite - test_startfile - test_sunaudiodev """, 'freebsd4': """ - test_aepack - test_al - test_applesingle test_bsddb test_bsddb3 - test_cd - test_cl test_gdbm - test_gl - test_imgfile - test_linuxaudiodev test_locale - test_macostools - test_nis test_ossaudiodev test_pep277 - test_plistlib test_pty - test_scriptpackages test_socket_ssl test_socketserver - test_sqlite - test_startfile - test_sunaudiodev test_tcl test_timeout - test_unicode_file test_urllibnet - test_winreg - test_winsound """, 'aix5': """ - test_aepack - test_al - test_applesingle test_bsddb test_bsddb185 test_bsddb3 test_bz2 - test_cd - test_cl test_dl test_gdbm - test_gl test_gzip - test_imgfile - test_linuxaudiodev - test_macostools - test_nis test_ossaudiodev - test_sqlite - test_startfile - test_sunaudiodev test_tcl - test_winreg - test_winsound test_zipimport test_zlib """, 'openbsd3': """ - test_aepack - test_al - test_applesingle test_bsddb test_bsddb3 - test_cd - test_cl test_ctypes test_dl test_gdbm - test_gl - test_imgfile - test_linuxaudiodev test_locale - test_macostools - test_nis test_normalization test_ossaudiodev test_pep277 - test_plistlib - test_scriptpackages test_tcl - test_sqlite - test_startfile - test_sunaudiodev - test_unicode_file - test_winreg - test_winsound """, 'netbsd3': """ - test_aepack - test_al - test_applesingle test_bsddb test_bsddb185 test_bsddb3 - test_cd - test_cl test_ctypes test_curses test_dl test_gdbm - test_gl - test_imgfile - test_linuxaudiodev test_locale - test_macostools - test_nis test_ossaudiodev test_pep277 - test_sqlite - test_startfile - test_sunaudiodev test_tcl - test_unicode_file - test_winreg - test_winsound """, } _expectations['freebsd5'] = _expectations['freebsd4'] @@ -1323,6 +1115,9 @@ s = _expectations[sys.platform] self.expected = set(s.split()) + # expected to be skipped on every platform, even Linux + self.expected.add('test_linuxaudiodev') + if not os.path.supports_unicode_filenames: self.expected.add('test_pep277') @@ -1337,21 +1132,30 @@ if not sys.platform in ("mac", "darwin"): MAC_ONLY = ["test_macostools", "test_aepack", - "test_plistlib", "test_scriptpackages"] + "test_plistlib", "test_scriptpackages", + "test_applesingle"] for skip in MAC_ONLY: self.expected.add(skip) if sys.platform != "win32": + # test_sqlite is only reliable on Windows where the library + # is distributed with Python WIN_ONLY = ["test_unicode_file", "test_winreg", - "test_winsound"] + "test_winsound", "test_startfile", + "test_sqlite"] for skip in WIN_ONLY: self.expected.add(skip) if sys.platform != 'irix': - IRIX_ONLY =["test_imageop"] + IRIX_ONLY = ["test_imageop", "test_al", "test_cd", "test_cl", + "test_gl", "test_imgfile"] for skip in IRIX_ONLY: self.expected.add(skip) + if sys.platform != 'sunos5': + self.expected.add('test_sunaudiodev') + self.expected.add('test_nis') + self.valid = True def isvalid(self): From python-checkins at python.org Thu Jul 12 11:37:49 2007 From: python-checkins at python.org (georg.brandl) Date: Thu, 12 Jul 2007 11:37:49 +0200 (CEST) Subject: [Python-checkins] r56306 - in python/trunk: Doc/tut/tut.tex Misc/NEWS Message-ID: <20070712093749.CB1201E401C@bag.python.org> Author: georg.brandl Date: Thu Jul 12 11:37:49 2007 New Revision: 56306 Modified: python/trunk/Doc/tut/tut.tex python/trunk/Misc/NEWS Log: Bug #1637365: add subsection about "__name__ == __main__" to the Python tutorial. Modified: python/trunk/Doc/tut/tut.tex ============================================================================== --- python/trunk/Doc/tut/tut.tex (original) +++ python/trunk/Doc/tut/tut.tex Thu Jul 12 11:37:49 2007 @@ -2522,6 +2522,44 @@ This imports all names except those beginning with an underscore (\code{_}). +\subsection{Executing modules as scripts \label{modulesAsScripts}} + +When you run a Python module with + +\begin{verbatim} +python fibo.py +\end{verbatim} + +the code in the module will be executed, just as if you imported it, but +with the \code{__name__} set to \code{"__main__"}. That means that by +adding this code at the end of your module: + +\begin{verbatim} +if __name__ == "__main__": + import sys + fib(int(sys.argv[1])) +\end{verbatim} + +you can make the file usable as a script as well as an importable module, +because the code that parses the command line only runs if the module is +executed as the ``main'' file: + +\begin{verbatim} +$ python fibo.py 50 +1 1 2 3 5 8 13 21 34 +\end{verbatim} + +If the module is imported, the code is not run: + +\begin{verbatim} +>>> import fibo +>>> +\end{verbatim} + +This is often used either to provide a convenient user interface to a +module, or for testing purposes (running the module as a script executes +a test suite). + \subsection{The Module Search Path \label{searchPath}} Modified: python/trunk/Misc/NEWS ============================================================================== --- python/trunk/Misc/NEWS (original) +++ python/trunk/Misc/NEWS Thu Jul 12 11:37:49 2007 @@ -851,6 +851,9 @@ Documentation ------------- +- Bug #1637365: add subsection about "__name__ == __main__" to the + Python tutorial. + - Patch #1698768: updated the "using Python on the Mac" intro. - Bug #1569057: Document that calling file.next() when the file is open for From python-checkins at python.org Thu Jul 12 11:37:53 2007 From: python-checkins at python.org (georg.brandl) Date: Thu, 12 Jul 2007 11:37:53 +0200 (CEST) Subject: [Python-checkins] r56307 - in python/branches/release25-maint: Doc/tut/tut.tex Misc/NEWS Message-ID: <20070712093753.C6C001E400F@bag.python.org> Author: georg.brandl Date: Thu Jul 12 11:37:53 2007 New Revision: 56307 Modified: python/branches/release25-maint/Doc/tut/tut.tex python/branches/release25-maint/Misc/NEWS Log: Bug #1637365: add subsection about "__name__ == __main__" to the Python tutorial. (backport from rev. 56306) Modified: python/branches/release25-maint/Doc/tut/tut.tex ============================================================================== --- python/branches/release25-maint/Doc/tut/tut.tex (original) +++ python/branches/release25-maint/Doc/tut/tut.tex Thu Jul 12 11:37:53 2007 @@ -2522,6 +2522,44 @@ This imports all names except those beginning with an underscore (\code{_}). +\subsection{Executing modules as scripts \label{modulesAsScripts}} + +When you run a Python module with + +\begin{verbatim} +python fibo.py +\end{verbatim} + +the code in the module will be executed, just as if you imported it, but +with the \code{__name__} set to \code{"__main__"}. That means that by +adding this code at the end of your module: + +\begin{verbatim} +if __name__ == "__main__": + import sys + fib(int(sys.argv[1])) +\end{verbatim} + +you can make the file usable as a script as well as an importable module, +because the code that parses the command line only runs if the module is +executed as the ``main'' file: + +\begin{verbatim} +$ python fibo.py 50 +1 1 2 3 5 8 13 21 34 +\end{verbatim} + +If the module is imported, the code is not run: + +\begin{verbatim} +>>> import fibo +>>> +\end{verbatim} + +This is often used either to provide a convenient user interface to a +module, or for testing purposes (running the module as a script executes +a test suite). + \subsection{The Module Search Path \label{searchPath}} Modified: python/branches/release25-maint/Misc/NEWS ============================================================================== --- python/branches/release25-maint/Misc/NEWS (original) +++ python/branches/release25-maint/Misc/NEWS Thu Jul 12 11:37:53 2007 @@ -77,6 +77,9 @@ Documentation ------------- +- Bug #1637365: add subsection about "__name__ == __main__" to the + Python tutorial. + - Bug #1569057: Document that calling file.next() on a file open for writing has undefined behaviour. Backport of r54712. From buildbot at python.org Thu Jul 12 11:44:19 2007 From: buildbot at python.org (buildbot at python.org) Date: Thu, 12 Jul 2007 09:44:19 +0000 Subject: [Python-checkins] buildbot warnings in ppc Debian unstable trunk Message-ID: <20070712094419.D156A1E4011@bag.python.org> The Buildbot has detected a new failure of ppc Debian unstable trunk. Full details are available at: http://www.python.org/dev/buildbot/all/ppc%2520Debian%2520unstable%2520trunk/builds/43 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: georg.brandl Build had warnings: warnings test Excerpt from the test logfile: Traceback (most recent call last): File "/home/pybot/buildarea/trunk.klose-debian-ppc/build/Lib/threading.py", line 465, in __bootstrap self.run() File "/home/pybot/buildarea/trunk.klose-debian-ppc/build/Lib/test/test_socketserver.py", line 93, in run svr.serve_a_few() File "/home/pybot/buildarea/trunk.klose-debian-ppc/build/Lib/test/test_socketserver.py", line 35, in serve_a_few self.handle_request() File "/home/pybot/buildarea/trunk.klose-debian-ppc/build/Lib/SocketServer.py", line 224, in handle_request self.handle_error(request, client_address) File "/home/pybot/buildarea/trunk.klose-debian-ppc/build/Lib/SocketServer.py", line 222, in handle_request self.process_request(request, client_address) File "/home/pybot/buildarea/trunk.klose-debian-ppc/build/Lib/SocketServer.py", line 429, in process_request self.collect_children() File "/home/pybot/buildarea/trunk.klose-debian-ppc/build/Lib/SocketServer.py", line 425, in collect_children self.active_children.remove(pid) ValueError: list.remove(x): x not in list 1 test failed: test_socketserver Traceback (most recent call last): File "./Lib/test/regrtest.py", line 557, in runtest_inner indirect_test() File "/home/pybot/buildarea/trunk.klose-debian-ppc/build/Lib/test/test_socketserver.py", line 216, in test_main testall() File "/home/pybot/buildarea/trunk.klose-debian-ppc/build/Lib/test/test_socketserver.py", line 199, in testall testloop(socket.AF_INET, tcpservers, MyStreamHandler, teststream) File "/home/pybot/buildarea/trunk.klose-debian-ppc/build/Lib/test/test_socketserver.py", line 148, in testloop testfunc(proto, addr) File "/home/pybot/buildarea/trunk.klose-debian-ppc/build/Lib/test/test_socketserver.py", line 62, in teststream s.connect(addr) File "", line 1, in connect error: (111, 'Connection refused') make: *** [buildbottest] Error 1 sincerely, -The Buildbot From python-checkins at python.org Thu Jul 12 11:59:22 2007 From: python-checkins at python.org (georg.brandl) Date: Thu, 12 Jul 2007 11:59:22 +0200 (CEST) Subject: [Python-checkins] r56308 - in python/trunk: Doc/lib/libzipfile.tex Lib/test/test_zipfile.py Lib/zipfile.py Misc/NEWS Message-ID: <20070712095922.8E6961E400A@bag.python.org> Author: georg.brandl Date: Thu Jul 12 11:59:22 2007 New Revision: 56308 Modified: python/trunk/Doc/lib/libzipfile.tex python/trunk/Lib/test/test_zipfile.py python/trunk/Lib/zipfile.py python/trunk/Misc/NEWS Log: Patch #1675424: Added tests for uncovered code in the zipfile module. The KeyError raised by Zipfile.getinfo for nonexistent names now has a descriptive message. Modified: python/trunk/Doc/lib/libzipfile.tex ============================================================================== --- python/trunk/Doc/lib/libzipfile.tex (original) +++ python/trunk/Doc/lib/libzipfile.tex Thu Jul 12 11:59:22 2007 @@ -24,8 +24,8 @@ The available attributes of this module are: -\begin{excdesc}{error} - The error raised for bad ZIP files. +\begin{excdesc}{BadZipfile} + The error raised for bad ZIP files (old name: \code{zipfile.error}). \end{excdesc} \begin{excdesc}{LargeZipFile} @@ -90,7 +90,7 @@ (a string) or a file-like object. The \var{mode} parameter should be \code{'r'} to read an existing file, \code{'w'} to truncate and write a new file, or \code{'a'} to append to an - existing file. For \var{mode} is \code{'a'} and \var{file} + existing file. If \var{mode} is \code{'a'} and \var{file} refers to an existing ZIP file, then additional files are added to it. If \var{file} does not refer to a ZIP file, then a new ZIP archive is appended to the file. This is meant for adding a ZIP @@ -128,7 +128,8 @@ \begin{methoddesc}{getinfo}{name} Return a \class{ZipInfo} object with information about the archive - member \var{name}. + member \var{name}. Calling \method{getinfo()} for a name not currently + contained in the archive will raise a \exception{KeyError}. \end{methoddesc} \begin{methoddesc}{infolist}{} @@ -147,7 +148,9 @@ parameter, if included, must be one of the following: \code{'r'} (the default), \code{'U'}, or \code{'rU'}. Choosing \code{'U'} or \code{'rU'} will enable universal newline support in the read-only - object. \var{pwd} is the password used for encrypted files. + object. \var{pwd} is the password used for encrypted files. Calling + \method{open()} on a closed ZipFile will raise a + \exception{RuntimeError}. \begin{notice} The file-like object is read-only and provides the following methods: \method{read()}, \method{readline()}, \method{readlines()}, @@ -182,7 +185,8 @@ Return the bytes of the file in the archive. The archive must be open for read or append. \var{pwd} is the password used for encrypted files and, if specified, it will override the default password set with - \method{setpassword()}. + \method{setpassword()}. Calling \method{read()} on a closed ZipFile + will raise a \exception{RuntimeError}. \versionchanged[\var{pwd} was added]{2.6} \end{methoddesc} @@ -190,6 +194,8 @@ \begin{methoddesc}{testzip}{} Read all the files in the archive and check their CRC's and file headers. Return the name of the first bad file, or else return \code{None}. + Calling \method{testzip()} on a closed ZipFile will raise a + \exception{RuntimeError}. \end{methoddesc} \begin{methoddesc}{write}{filename\optional{, arcname\optional{, @@ -200,7 +206,10 @@ separators removed). If given, \var{compress_type} overrides the value given for the \var{compression} parameter to the constructor for the new entry. The archive must be open with mode \code{'w'} - or \code{'a'}. + or \code{'a'} -- calling \method{write()} on a ZipFile created with + mode \code{'r'} will raise a \exception{RuntimeError}. Calling + \method{write()} on a closed ZipFile will raise a + \exception{RuntimeError}. \note{There is no official file name encoding for ZIP files. If you have unicode file names, please convert them to byte strings @@ -210,6 +219,11 @@ \note{Archive names should be relative to the archive root, that is, they should not start with a path separator.} + + \note{If \code{arcname} (or \code{filename}, if \code{arcname} is + not given) contains a null byte, the name of the file in the archive will + be truncated at the null byte.} + \end{methoddesc} \begin{methoddesc}{writestr}{zinfo_or_arcname, bytes} @@ -218,7 +232,10 @@ \class{ZipInfo} instance. If it's an instance, at least the filename, date, and time must be given. If it's a name, the date and time is set to the current date and time. The archive must be - opened with mode \code{'w'} or \code{'a'}. + opened with mode \code{'w'} or \code{'a'} -- calling + \method{writestr()} on a ZipFile created with mode \code{'r'} + will raise a \exception{RuntimeError}. Calling \method{writestr()} + on a closed ZipFile will raise a \exception{RuntimeError}. \end{methoddesc} @@ -243,12 +260,13 @@ available, else a \file{*.pyc} file, compiling if necessary. If the pathname is a file, the filename must end with \file{.py}, and just the (corresponding \file{*.py[co]}) file is added at the top level - (no path information). If it is a directory, and the directory is - not a package directory, then all the files \file{*.py[co]} are - added at the top level. If the directory is a package directory, - then all \file{*.py[oc]} are added under the package name as a file - path, and if any subdirectories are package directories, all of - these are added recursively. \var{basename} is intended for + (no path information). If the pathname is a file that does not end with + \file{.py}, a \exception{RuntimeError} will be raised. If it is a + directory, and the directory is not a package directory, then all the + files \file{*.py[co]} are added at the top level. If the directory is + a package directory, then all \file{*.py[co]} are added under the package + name as a file path, and if any subdirectories are package directories, all + of these are added recursively. \var{basename} is intended for internal use only. The \method{writepy()} method makes archives with file names like this: Modified: python/trunk/Lib/test/test_zipfile.py ============================================================================== --- python/trunk/Lib/test/test_zipfile.py (original) +++ python/trunk/Lib/test/test_zipfile.py Thu Jul 12 11:59:22 2007 @@ -14,12 +14,12 @@ from test.test_support import TESTFN, run_unittest TESTFN2 = TESTFN + "2" -FIXEDTEST_SIZE = 10 +FIXEDTEST_SIZE = 1000 class TestsWithSourceFile(unittest.TestCase): def setUp(self): - self.line_gen = ("Zipfile test line %d. random float: %f" % (i, random()) - for i in xrange(FIXEDTEST_SIZE)) + self.line_gen = ["Zipfile test line %d. random float: %f" % (i, random()) + for i in xrange(FIXEDTEST_SIZE)] self.data = '\n'.join(self.line_gen) + '\n' # Make a source file with some lines @@ -239,6 +239,63 @@ self.assertEqual(zipfp.namelist(), ["absolute"]) zipfp.close() + def testAppendToZipFile(self): + # Test appending to an existing zipfile + zipfp = zipfile.ZipFile(TESTFN2, "w", zipfile.ZIP_STORED) + zipfp.write(TESTFN, TESTFN) + zipfp.close() + zipfp = zipfile.ZipFile(TESTFN2, "a", zipfile.ZIP_STORED) + zipfp.writestr("strfile", self.data) + self.assertEqual(zipfp.namelist(), [TESTFN, "strfile"]) + zipfp.close() + + def testAppendToNonZipFile(self): + # Test appending to an existing file that is not a zipfile + # NOTE: this test fails if len(d) < 22 because of the first + # line "fpin.seek(-22, 2)" in _EndRecData + d = 'I am not a ZipFile!'*10 + f = file(TESTFN2, 'wb') + f.write(d) + f.close() + zipfp = zipfile.ZipFile(TESTFN2, "a", zipfile.ZIP_STORED) + zipfp.write(TESTFN, TESTFN) + zipfp.close() + + f = file(TESTFN2, 'rb') + f.seek(len(d)) + zipfp = zipfile.ZipFile(f, "r") + self.assertEqual(zipfp.namelist(), [TESTFN]) + zipfp.close() + f.close() + + def test_WriteDefaultName(self): + # Check that calling ZipFile.write without arcname specified produces the expected result + zipfp = zipfile.ZipFile(TESTFN2, "w") + zipfp.write(TESTFN) + self.assertEqual(zipfp.read(TESTFN), file(TESTFN).read()) + zipfp.close() + + def test_PerFileCompression(self): + # Check that files within a Zip archive can have different compression options + zipfp = zipfile.ZipFile(TESTFN2, "w") + zipfp.write(TESTFN, 'storeme', zipfile.ZIP_STORED) + zipfp.write(TESTFN, 'deflateme', zipfile.ZIP_DEFLATED) + sinfo = zipfp.getinfo('storeme') + dinfo = zipfp.getinfo('deflateme') + self.assertEqual(sinfo.compress_type, zipfile.ZIP_STORED) + self.assertEqual(dinfo.compress_type, zipfile.ZIP_DEFLATED) + zipfp.close() + + def test_WriteToReadonly(self): + # Check that trying to call write() on a readonly ZipFile object + # raises a RuntimeError + zipf = zipfile.ZipFile(TESTFN2, mode="w") + zipf.writestr("somefile.txt", "bogus") + zipf.close() + zipf = zipfile.ZipFile(TESTFN2, mode="r") + self.assertRaises(RuntimeError, zipf.write, TESTFN) + zipf.close() + def tearDown(self): os.remove(TESTFN) os.remove(TESTFN2) @@ -361,7 +418,6 @@ self.assertEqual(zipfp.namelist(), ["absolute"]) zipfp.close() - def tearDown(self): zipfile.ZIP64_LIMIT = self._limit os.remove(TESTFN) @@ -432,6 +488,11 @@ finally: shutil.rmtree(TESTFN2) + def testWriteNonPyfile(self): + zipfp = zipfile.PyZipFile(TemporaryFile(), "w") + file(TESTFN, 'w').write('most definitely not a python file') + self.assertRaises(RuntimeError, zipfp.writepy, TESTFN) + os.remove(TESTFN) class OtherTests(unittest.TestCase): @@ -513,7 +574,56 @@ # a RuntimeError, and so should calling .testzip. An earlier # version of .testzip would swallow this exception (and any other) # and report that the first file in the archive was corrupt. + self.assertRaises(RuntimeError, zipf.read, "foo.txt") + self.assertRaises(RuntimeError, zipf.open, "foo.txt") self.assertRaises(RuntimeError, zipf.testzip) + self.assertRaises(RuntimeError, zipf.writestr, "bogus.txt", "bogus") + file(TESTFN, 'w').write('zipfile test data') + self.assertRaises(RuntimeError, zipf.write, TESTFN) + + def test_BadConstructorMode(self): + # Check that bad modes passed to ZipFile constructor are caught + self.assertRaises(RuntimeError, zipfile.ZipFile, TESTFN, "q") + + def test_BadOpenMode(self): + # Check that bad modes passed to ZipFile.open are caught + zipf = zipfile.ZipFile(TESTFN, mode="w") + zipf.writestr("foo.txt", "O, for a Muse of Fire!") + zipf.close() + zipf = zipfile.ZipFile(TESTFN, mode="r") + # read the data to make sure the file is there + zipf.read("foo.txt") + self.assertRaises(RuntimeError, zipf.open, "foo.txt", "q") + zipf.close() + + def test_Read0(self): + # Check that calling read(0) on a ZipExtFile object returns an empty + # string and doesn't advance file pointer + zipf = zipfile.ZipFile(TESTFN, mode="w") + zipf.writestr("foo.txt", "O, for a Muse of Fire!") + # read the data to make sure the file is there + f = zipf.open("foo.txt") + for i in xrange(FIXEDTEST_SIZE): + self.assertEqual(f.read(0), '') + + self.assertEqual(f.read(), "O, for a Muse of Fire!") + zipf.close() + + def test_OpenNonexistentItem(self): + # Check that attempting to call open() for an item that doesn't + # exist in the archive raises a RuntimeError + zipf = zipfile.ZipFile(TESTFN, mode="w") + self.assertRaises(KeyError, zipf.open, "foo.txt", "r") + + def test_BadCompressionMode(self): + # Check that bad compression methods passed to ZipFile.open are caught + self.assertRaises(RuntimeError, zipfile.ZipFile, TESTFN, "w", -1) + + def test_NullByteInFilename(self): + # Check that a filename containing a null byte is properly terminated + zipf = zipfile.ZipFile(TESTFN, mode="w") + zipf.writestr("foo.txt\x00qqq", "O, for a Muse of Fire!") + self.assertEqual(zipf.namelist(), ['foo.txt']) def tearDown(self): support.unlink(TESTFN) Modified: python/trunk/Lib/zipfile.py ============================================================================== --- python/trunk/Lib/zipfile.py (original) +++ python/trunk/Lib/zipfile.py Thu Jul 12 11:59:22 2007 @@ -568,8 +568,9 @@ def __init__(self, file, mode="r", compression=ZIP_STORED, allowZip64=False): """Open the ZIP file with mode read "r", write "w" or append "a".""" - self._allowZip64 = allowZip64 - self._didModify = False + if mode not in ("r", "w", "a"): + raise RuntimeError('ZipFile() requires mode "r", "w", or "a"') + if compression == ZIP_STORED: pass elif compression == ZIP_DEFLATED: @@ -578,6 +579,9 @@ "Compression requires the (missing) zlib module" else: raise RuntimeError, "That compression method is not supported" + + self._allowZip64 = allowZip64 + self._didModify = False self.debug = 0 # Level of printing: 0 through 3 self.NameToInfo = {} # Find file info given name self.filelist = [] # List of ZipInfo instances for archive @@ -720,7 +724,12 @@ def getinfo(self, name): """Return the instance of ZipInfo given 'name'.""" - return self.NameToInfo[name] + info = self.NameToInfo.get(name) + if info is None: + raise KeyError( + 'There is no item named %r in the archive' % name) + + return info def setpassword(self, pwd): """Set default password for encrypted files.""" @@ -824,6 +833,10 @@ def write(self, filename, arcname=None, compress_type=None): """Put the bytes from filename into the archive under the name arcname.""" + if not self.fp: + raise RuntimeError( + "Attempt to write to ZIP archive that was already closed") + st = os.stat(filename) mtime = time.localtime(st.st_mtime) date_time = mtime[0:6] @@ -896,6 +909,11 @@ zinfo.compress_type = self.compression else: zinfo = zinfo_or_arcname + + if not self.fp: + raise RuntimeError( + "Attempt to write to ZIP archive that was already closed") + zinfo.file_size = len(bytes) # Uncompressed size zinfo.header_offset = self.fp.tell() # Start of header bytes self._writecheck(zinfo) Modified: python/trunk/Misc/NEWS ============================================================================== --- python/trunk/Misc/NEWS (original) +++ python/trunk/Misc/NEWS Thu Jul 12 11:59:22 2007 @@ -420,6 +420,10 @@ - Patch #1481079: add support for HTTP_REFERER to CGIHTTPServer. +- Patch #1675424: Added tests for uncovered code in the zipfile module. + The KeyError raised by Zipfile.getinfo for nonexistent names now has + a descriptive message. + - Bug #1115886: os.path.splitext('.cshrc') gives now ('.cshrc', ''). - unittest now verifies more of its assumptions. In particular, TestCase From python-checkins at python.org Thu Jul 12 19:31:06 2007 From: python-checkins at python.org (phillip.eby) Date: Thu, 12 Jul 2007 19:31:06 +0200 (CEST) Subject: [Python-checkins] r56320 - sandbox/trunk/setuptools/setuptools/__init__.py Message-ID: <20070712173106.C4C401E400F@bag.python.org> Author: phillip.eby Date: Thu Jul 12 19:31:06 2007 New Revision: 56320 Modified: sandbox/trunk/setuptools/setuptools/__init__.py Log: Fix a problem with the findall() fix. :( Modified: sandbox/trunk/setuptools/setuptools/__init__.py ============================================================================== --- sandbox/trunk/setuptools/setuptools/__init__.py (original) +++ sandbox/trunk/setuptools/setuptools/__init__.py Thu Jul 12 19:31:06 2007 @@ -69,7 +69,9 @@ """ all_files = [] for base, dirs, files in os.walk(dir): - if base!=os.curdir: + if base==os.curdir or base.startswith(os.curdir+os.sep): + base = base[2:] + if base: files = [os.path.join(base, f) for f in files] all_files.extend(filter(os.path.isfile, files)) return all_files From python-checkins at python.org Thu Jul 12 19:32:02 2007 From: python-checkins at python.org (phillip.eby) Date: Thu, 12 Jul 2007 19:32:02 +0200 (CEST) Subject: [Python-checkins] r56321 - sandbox/branches/setuptools-0.6/setuptools/__init__.py Message-ID: <20070712173202.AA3381E400A@bag.python.org> Author: phillip.eby Date: Thu Jul 12 19:32:02 2007 New Revision: 56321 Modified: sandbox/branches/setuptools-0.6/setuptools/__init__.py Log: Backport findall() fix fix from trunk. Modified: sandbox/branches/setuptools-0.6/setuptools/__init__.py ============================================================================== --- sandbox/branches/setuptools-0.6/setuptools/__init__.py (original) +++ sandbox/branches/setuptools-0.6/setuptools/__init__.py Thu Jul 12 19:32:02 2007 @@ -69,7 +69,9 @@ """ all_files = [] for base, dirs, files in os.walk(dir): - if base!=os.curdir: + if base==os.curdir or base.startswith(os.curdir+os.sep): + base = base[2:] + if base: files = [os.path.join(base, f) for f in files] all_files.extend(filter(os.path.isfile, files)) return all_files From python-checkins at python.org Fri Jul 13 12:43:44 2007 From: python-checkins at python.org (facundo.batista) Date: Fri, 13 Jul 2007 12:43:44 +0200 (CEST) Subject: [Python-checkins] r56340 - python/trunk/Lib/test/test_xmlrpc.py Message-ID: <20070713104344.E65E21E4013@bag.python.org> Author: facundo.batista Date: Fri Jul 13 12:43:44 2007 New Revision: 56340 Modified: python/trunk/Lib/test/test_xmlrpc.py Log: Added tests for basic behavior of DateTime, Binary, and Fault classes and the escape function. Check that marshalling recursive sequences & dicts raises TypeError. Check that marshalling out-of-range ints raises OverflowError [Alan McIntyre - GSoC] Modified: python/trunk/Lib/test/test_xmlrpc.py ============================================================================== --- python/trunk/Lib/test/test_xmlrpc.py (original) +++ python/trunk/Lib/test/test_xmlrpc.py Fri Jul 13 12:43:44 2007 @@ -1,5 +1,7 @@ +import base64 import datetime import sys +import time import unittest import xmlrpclib from test import test_support @@ -25,6 +27,10 @@ (2005, 02, 10, 11, 41, 23, 0, 1, -1)), 'datetime3': xmlrpclib.DateTime( datetime.datetime(2005, 02, 10, 11, 41, 23)), + 'datetime4': xmlrpclib.DateTime( + datetime.date(2005, 02, 10)), + 'datetime5': xmlrpclib.DateTime( + datetime.time(11, 41, 23)), }] class XMLRPCTestCase(unittest.TestCase): @@ -101,11 +107,37 @@ def test_dump_bad_dict(self): self.assertRaises(TypeError, xmlrpclib.dumps, ({(1,2,3): 1},)) + def test_dump_recursive_seq(self): + l = [1,2,3] + t = [3,4,5,l] + l.append(t) + self.assertRaises(TypeError, xmlrpclib.dumps, (l,)) + + def test_dump_recursive_dict(self): + d = {'1':1, '2':1} + t = {'3':3, 'd':d} + d['t'] = t + self.assertRaises(TypeError, xmlrpclib.dumps, (d,)) + def test_dump_big_int(self): if sys.maxint > 2L**31-1: self.assertRaises(OverflowError, xmlrpclib.dumps, (int(2L**34),)) + xmlrpclib.dumps((xmlrpclib.MAXINT, xmlrpclib.MININT)) + self.assertRaises(OverflowError, xmlrpclib.dumps, (xmlrpclib.MAXINT+1,)) + self.assertRaises(OverflowError, xmlrpclib.dumps, (xmlrpclib.MININT-1,)) + + def dummy_write(s): + pass + + m = xmlrpclib.Marshaller() + m.dump_int(xmlrpclib.MAXINT, dummy_write) + m.dump_int(xmlrpclib.MININT, dummy_write) + self.assertRaises(OverflowError, m.dump_int, xmlrpclib.MAXINT+1, dummy_write) + self.assertRaises(OverflowError, m.dump_int, xmlrpclib.MININT-1, dummy_write) + + def test_dump_none(self): value = alist + [None] arg1 = (alist + [None],) @@ -156,8 +188,109 @@ self.assertEquals(s, "abc \xc2\x95") self.assertEquals(items, [("def \xc2\x96", "ghi \xc2\x97")]) + +class HelperTestCase(unittest.TestCase): + def test_escape(self): + self.assertEqual(xmlrpclib.escape("a&b"), "a&b") + self.assertEqual(xmlrpclib.escape("ab"), "a>b") + +class FaultTestCase(unittest.TestCase): + def test_repr(self): + f = xmlrpclib.Fault(42, 'Test Fault') + self.assertEqual(repr(f), "") + self.assertEqual(repr(f), str(f)) + + def test_dump_fault(self): + f = xmlrpclib.Fault(42, 'Test Fault') + s = xmlrpclib.dumps((f,)) + (newf,), m = xmlrpclib.loads(s) + self.assertEquals(newf, {'faultCode': 42, 'faultString': 'Test Fault'}) + self.assertEquals(m, None) + + s = xmlrpclib.Marshaller().dumps(f) + self.assertRaises(xmlrpclib.Fault, xmlrpclib.loads, s) + + +class DateTimeTestCase(unittest.TestCase): + def test_default(self): + t = xmlrpclib.DateTime() + + def test_time(self): + d = 1181399930.036952 + t = xmlrpclib.DateTime(d) + self.assertEqual(str(t), time.strftime("%Y%m%dT%H:%M:%S", time.localtime(d))) + + def test_time_tuple(self): + d = (2007,6,9,10,38,50,5,160,0) + t = xmlrpclib.DateTime(d) + self.assertEqual(str(t), '20070609T10:38:50') + + def test_time_struct(self): + d = time.localtime(1181399930.036952) + t = xmlrpclib.DateTime(d) + self.assertEqual(str(t), time.strftime("%Y%m%dT%H:%M:%S", d)) + + def test_datetime_datetime(self): + d = datetime.datetime(2007,1,2,3,4,5) + t = xmlrpclib.DateTime(d) + self.assertEqual(str(t), '20070102T03:04:05') + + def test_datetime_date(self): + d = datetime.date(2007,9,8) + t = xmlrpclib.DateTime(d) + self.assertEqual(str(t), '20070908T00:00:00') + + def test_datetime_time(self): + d = datetime.time(13,17,19) + # allow for date rollover by checking today's or tomorrow's dates + dd1 = datetime.datetime.now().date() + dd2 = dd1 + datetime.timedelta(days=1) + vals = (dd1.strftime('%Y%m%dT13:17:19'), + dd2.strftime('%Y%m%dT13:17:19')) + t = xmlrpclib.DateTime(d) + self.assertEqual(str(t) in vals, True) + + def test_repr(self): + d = datetime.datetime(2007,1,2,3,4,5) + t = xmlrpclib.DateTime(d) + val ="" % id(t) + self.assertEqual(repr(t), val) + + def test_decode(self): + d = ' 20070908T07:11:13 ' + t1 = xmlrpclib.DateTime() + t1.decode(d) + tref = xmlrpclib.DateTime(datetime.datetime(2007,9,8,7,11,13)) + self.assertEqual(t1, tref) + + t2 = xmlrpclib._datetime(d) + self.assertEqual(t1, tref) + +class BinaryTestCase(unittest.TestCase): + def test_default(self): + t = xmlrpclib.Binary() + self.assertEqual(str(t), '') + + def test_string(self): + d = '\x01\x02\x03abc123\xff\xfe' + t = xmlrpclib.Binary(d) + self.assertEqual(str(t), d) + + def test_decode(self): + d = '\x01\x02\x03abc123\xff\xfe' + de = base64.encodestring(d) + t1 = xmlrpclib.Binary() + t1.decode(de) + self.assertEqual(str(t1), d) + + t2 = xmlrpclib._binary(de) + self.assertEqual(str(t2), d) + + def test_main(): - test_support.run_unittest(XMLRPCTestCase) + test_support.run_unittest(XMLRPCTestCase, HelperTestCase, + DateTimeTestCase, BinaryTestCase, FaultTestCase) if __name__ == "__main__": From buildbot at python.org Fri Jul 13 12:48:50 2007 From: buildbot at python.org (buildbot at python.org) Date: Fri, 13 Jul 2007 10:48:50 +0000 Subject: [Python-checkins] buildbot failure in x86 gentoo trunk Message-ID: <20070713104850.02F621E4010@bag.python.org> The Buildbot has detected a new failure of x86 gentoo trunk. Full details are available at: http://www.python.org/dev/buildbot/all/x86%2520gentoo%2520trunk/builds/2301 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: facundo.batista BUILD FAILED: failed svn sincerely, -The Buildbot From buildbot at python.org Fri Jul 13 12:48:56 2007 From: buildbot at python.org (buildbot at python.org) Date: Fri, 13 Jul 2007 10:48:56 +0000 Subject: [Python-checkins] buildbot failure in x86 mvlgcc trunk Message-ID: <20070713104857.0C1A81E4022@bag.python.org> The Buildbot has detected a new failure of x86 mvlgcc trunk. Full details are available at: http://www.python.org/dev/buildbot/all/x86%2520mvlgcc%2520trunk/builds/644 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: facundo.batista BUILD FAILED: failed svn sincerely, -The Buildbot From buildbot at python.org Fri Jul 13 12:48:57 2007 From: buildbot at python.org (buildbot at python.org) Date: Fri, 13 Jul 2007 10:48:57 +0000 Subject: [Python-checkins] buildbot failure in S-390 Debian trunk Message-ID: <20070713104857.6A3911E401D@bag.python.org> The Buildbot has detected a new failure of S-390 Debian trunk. Full details are available at: http://www.python.org/dev/buildbot/all/S-390%2520Debian%2520trunk/builds/1037 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: facundo.batista BUILD FAILED: failed svn sincerely, -The Buildbot From buildbot at python.org Fri Jul 13 12:49:12 2007 From: buildbot at python.org (buildbot at python.org) Date: Fri, 13 Jul 2007 10:49:12 +0000 Subject: [Python-checkins] buildbot failure in x86 W2k trunk Message-ID: <20070713104912.8BE991E400E@bag.python.org> The Buildbot has detected a new failure of x86 W2k trunk. Full details are available at: http://www.python.org/dev/buildbot/all/x86%2520W2k%2520trunk/builds/393 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: facundo.batista BUILD FAILED: failed svn sincerely, -The Buildbot From buildbot at python.org Fri Jul 13 12:49:12 2007 From: buildbot at python.org (buildbot at python.org) Date: Fri, 13 Jul 2007 10:49:12 +0000 Subject: [Python-checkins] buildbot failure in PPC64 Debian trunk Message-ID: <20070713104912.E88DE1E400E@bag.python.org> The Buildbot has detected a new failure of PPC64 Debian trunk. Full details are available at: http://www.python.org/dev/buildbot/all/PPC64%2520Debian%2520trunk/builds/46 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: facundo.batista BUILD FAILED: failed svn sincerely, -The Buildbot From buildbot at python.org Fri Jul 13 12:49:14 2007 From: buildbot at python.org (buildbot at python.org) Date: Fri, 13 Jul 2007 10:49:14 +0000 Subject: [Python-checkins] buildbot failure in ppc Debian unstable trunk Message-ID: <20070713104914.6277D1E400E@bag.python.org> The Buildbot has detected a new failure of ppc Debian unstable trunk. Full details are available at: http://www.python.org/dev/buildbot/all/ppc%2520Debian%2520unstable%2520trunk/builds/46 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: facundo.batista BUILD FAILED: failed svn sincerely, -The Buildbot From buildbot at python.org Fri Jul 13 12:49:17 2007 From: buildbot at python.org (buildbot at python.org) Date: Fri, 13 Jul 2007 10:49:17 +0000 Subject: [Python-checkins] buildbot failure in g4 osx.4 trunk Message-ID: <20070713104917.527A01E400A@bag.python.org> The Buildbot has detected a new failure of g4 osx.4 trunk. Full details are available at: http://www.python.org/dev/buildbot/all/g4%2520osx.4%2520trunk/builds/2130 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: facundo.batista BUILD FAILED: failed svn sincerely, -The Buildbot From buildbot at python.org Fri Jul 13 12:49:18 2007 From: buildbot at python.org (buildbot at python.org) Date: Fri, 13 Jul 2007 10:49:18 +0000 Subject: [Python-checkins] buildbot failure in alpha Debian trunk Message-ID: <20070713104918.D12971E400A@bag.python.org> The Buildbot has detected a new failure of alpha Debian trunk. Full details are available at: http://www.python.org/dev/buildbot/all/alpha%2520Debian%2520trunk/builds/41 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: facundo.batista BUILD FAILED: failed svn sincerely, -The Buildbot From buildbot at python.org Fri Jul 13 12:49:25 2007 From: buildbot at python.org (buildbot at python.org) Date: Fri, 13 Jul 2007 10:49:25 +0000 Subject: [Python-checkins] buildbot failure in sparc solaris10 gcc trunk Message-ID: <20070713104925.5B66A1E400A@bag.python.org> The Buildbot has detected a new failure of sparc solaris10 gcc trunk. Full details are available at: http://www.python.org/dev/buildbot/all/sparc%2520solaris10%2520gcc%2520trunk/builds/2121 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: facundo.batista BUILD FAILED: failed svn sincerely, -The Buildbot From buildbot at python.org Fri Jul 13 12:50:49 2007 From: buildbot at python.org (buildbot at python.org) Date: Fri, 13 Jul 2007 10:50:49 +0000 Subject: [Python-checkins] buildbot failure in alpha Tru64 5.1 trunk Message-ID: <20070713105049.E42F51E400A@bag.python.org> The Buildbot has detected a new failure of alpha Tru64 5.1 trunk. Full details are available at: http://www.python.org/dev/buildbot/all/alpha%2520Tru64%25205.1%2520trunk/builds/1708 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: facundo.batista BUILD FAILED: failed svn sincerely, -The Buildbot From buildbot at python.org Fri Jul 13 12:51:31 2007 From: buildbot at python.org (buildbot at python.org) Date: Fri, 13 Jul 2007 10:51:31 +0000 Subject: [Python-checkins] buildbot failure in x86 XP trunk Message-ID: <20070713105131.D26571E4006@bag.python.org> The Buildbot has detected a new failure of x86 XP trunk. Full details are available at: http://www.python.org/dev/buildbot/all/x86%2520XP%2520trunk/builds/509 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: facundo.batista BUILD FAILED: failed svn sincerely, -The Buildbot From buildbot at python.org Fri Jul 13 12:53:21 2007 From: buildbot at python.org (buildbot at python.org) Date: Fri, 13 Jul 2007 10:53:21 +0000 Subject: [Python-checkins] buildbot failure in x86 XP-3 trunk Message-ID: <20070713105322.02A7E1E4006@bag.python.org> The Buildbot has detected a new failure of x86 XP-3 trunk. Full details are available at: http://www.python.org/dev/buildbot/all/x86%2520XP-3%2520trunk/builds/76 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: facundo.batista BUILD FAILED: failed svn sincerely, -The Buildbot From buildbot at python.org Fri Jul 13 12:54:16 2007 From: buildbot at python.org (buildbot at python.org) Date: Fri, 13 Jul 2007 10:54:16 +0000 Subject: [Python-checkins] buildbot failure in amd64 XP trunk Message-ID: <20070713105417.15FF81E4006@bag.python.org> The Buildbot has detected a new failure of amd64 XP trunk. Full details are available at: http://www.python.org/dev/buildbot/all/amd64%2520XP%2520trunk/builds/44 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: facundo.batista BUILD FAILED: failed svn sincerely, -The Buildbot From buildbot at python.org Fri Jul 13 12:55:07 2007 From: buildbot at python.org (buildbot at python.org) Date: Fri, 13 Jul 2007 10:55:07 +0000 Subject: [Python-checkins] buildbot failure in ia64 Ubuntu trunk trunk Message-ID: <20070713105507.A4AB01E4006@bag.python.org> The Buildbot has detected a new failure of ia64 Ubuntu trunk trunk. Full details are available at: http://www.python.org/dev/buildbot/all/ia64%2520Ubuntu%2520trunk%2520trunk/builds/745 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: facundo.batista BUILD FAILED: failed svn sincerely, -The Buildbot From buildbot at python.org Fri Jul 13 12:55:09 2007 From: buildbot at python.org (buildbot at python.org) Date: Fri, 13 Jul 2007 10:55:09 +0000 Subject: [Python-checkins] buildbot failure in hppa Ubuntu dapper trunk Message-ID: <20070713105509.B1CD91E400D@bag.python.org> The Buildbot has detected a new failure of hppa Ubuntu dapper trunk. Full details are available at: http://www.python.org/dev/buildbot/all/hppa%2520Ubuntu%2520dapper%2520trunk/builds/10 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: facundo.batista BUILD FAILED: failed svn sincerely, -The Buildbot From python-checkins at python.org Fri Jul 13 14:09:42 2007 From: python-checkins at python.org (raymond.hettinger) Date: Fri, 13 Jul 2007 14:09:42 +0200 (CEST) Subject: [Python-checkins] r56345 - python/trunk/Doc/lib/libitertools.tex Message-ID: <20070713120942.35CA61E4011@bag.python.org> Author: raymond.hettinger Date: Fri Jul 13 14:09:41 2007 New Revision: 56345 Modified: python/trunk/Doc/lib/libitertools.tex Log: Correct the docs for takewhile(). Improve the recipe for nth(). Should be backported Modified: python/trunk/Doc/lib/libitertools.tex ============================================================================== --- python/trunk/Doc/lib/libitertools.tex (original) +++ python/trunk/Doc/lib/libitertools.tex Fri Jul 13 14:09:41 2007 @@ -117,7 +117,7 @@ Make an iterator that drops elements from the iterable as long as the predicate is true; afterwards, returns every element. Note, the iterator does not produce \emph{any} output until the predicate - is true, so it may have a lengthy start-up time. Equivalent to: + first becomes false, so it may have a lengthy start-up time. Equivalent to: \begin{verbatim} def dropwhile(predicate, iterable): @@ -509,8 +509,8 @@ return izip(mapping.iterkeys(), mapping.itervalues()) def nth(iterable, n): - "Returns the nth item or raise IndexError" - return list(islice(iterable, n, n+1))[0] + "Returns the nth item or raise StopIteration" + return islice(iterable, n, None).next() def all(seq, pred=None): "Returns True if pred(x) is true for every element in the iterable" From python-checkins at python.org Fri Jul 13 15:59:40 2007 From: python-checkins at python.org (thomas.heller) Date: Fri, 13 Jul 2007 15:59:40 +0200 (CEST) Subject: [Python-checkins] r56348 - python/trunk/Modules/_ctypes/_ctypes.c Message-ID: <20070713135940.2054A1E400A@bag.python.org> Author: thomas.heller Date: Fri Jul 13 15:59:39 2007 New Revision: 56348 Modified: python/trunk/Modules/_ctypes/_ctypes.c Log: Repair COMError. Since exceptions are new style classes now, setting the methods and docstring after the type creation does not work, they must be in the dictionary before creating the type. Modified: python/trunk/Modules/_ctypes/_ctypes.c ============================================================================== --- python/trunk/Modules/_ctypes/_ctypes.c (original) +++ python/trunk/Modules/_ctypes/_ctypes.c Fri Jul 13 15:59:39 2007 @@ -4520,11 +4520,6 @@ PyObject *s; int status; - ComError = PyErr_NewException("_ctypes.COMError", - NULL, - dict); - if (ComError == NULL) - return -1; while (methods->ml_name) { /* get a wrapper for the built-in function */ PyObject *func = PyCFunction_New(methods, NULL); @@ -4539,13 +4534,24 @@ Py_DECREF(meth); ++methods; } - Py_INCREF(ComError); + s = PyString_FromString(comerror_doc); if (s == NULL) return -1; status = PyDict_SetItemString(dict, "__doc__", s); Py_DECREF(s); - return status; + if (status == -1) { + Py_DECREF(dict); + return -1; + } + + ComError = PyErr_NewException("_ctypes.COMError", + NULL, + dict); + if (ComError == NULL) + return -1; + + return 0; } #endif From python-checkins at python.org Fri Jul 13 16:18:06 2007 From: python-checkins at python.org (thomas.heller) Date: Fri, 13 Jul 2007 16:18:06 +0200 (CEST) Subject: [Python-checkins] r56349 - python/trunk/Lib/ctypes/test/test_win32.py Message-ID: <20070713141806.5F2AF1E400A@bag.python.org> Author: thomas.heller Date: Fri Jul 13 16:18:06 2007 New Revision: 56349 Modified: python/trunk/Lib/ctypes/test/test_win32.py Log: Add tests for _ctypes.COMError. Modified: python/trunk/Lib/ctypes/test/test_win32.py ============================================================================== --- python/trunk/Lib/ctypes/test/test_win32.py (original) +++ python/trunk/Lib/ctypes/test/test_win32.py Fri Jul 13 16:18:06 2007 @@ -58,6 +58,15 @@ self.failUnlessEqual(sizeof(wintypes.LPARAM), sizeof(c_void_p)) + def test_COMError(self): + from _ctypes import COMError + self.assertEqual(COMError.__doc__, "Raised when a COM method call failed.") + + ex = COMError(-1, "text", ("details",)) + self.assertEqual(ex.hresult, -1) + self.assertEqual(ex.text, "text") + self.assertEqual(ex.details, ("details",)) + class Structures(unittest.TestCase): def test_struct_by_value(self): From python-checkins at python.org Fri Jul 13 18:50:44 2007 From: python-checkins at python.org (thomas.heller) Date: Fri, 13 Jul 2007 18:50:44 +0200 (CEST) Subject: [Python-checkins] r56350 - python/trunk/Lib/ctypes/test/test_find.py Message-ID: <20070713165044.7BFFC1E4014@bag.python.org> Author: thomas.heller Date: Fri Jul 13 18:50:43 2007 New Revision: 56350 Modified: python/trunk/Lib/ctypes/test/test_find.py Log: Do not try to load the GLUT library in the ctypes tests. This test adds little value, but has a large problem on OS X, as explained in SF# 1581906. Modified: python/trunk/Lib/ctypes/test/test_find.py ============================================================================== --- python/trunk/Lib/ctypes/test/test_find.py (original) +++ python/trunk/Lib/ctypes/test/test_find.py Fri Jul 13 18:50:43 2007 @@ -7,25 +7,21 @@ if sys.platform == "win32": lib_gl = find_library("OpenGL32") lib_glu = find_library("Glu32") - lib_glut = find_library("glut32") lib_gle = None elif sys.platform == "darwin": lib_gl = lib_glu = find_library("OpenGL") - lib_glut = find_library("GLUT") lib_gle = None else: lib_gl = find_library("GL") lib_glu = find_library("GLU") - lib_glut = find_library("glut") lib_gle = find_library("gle") ## print, for debugging if is_resource_enabled("printing"): - if lib_gl or lib_glu or lib_glut or lib_gle: + if lib_gl or lib_glu or lib_gle: print "OpenGL libraries:" for item in (("GL", lib_gl), ("GLU", lib_glu), - ("glut", lib_glut), ("gle", lib_gle)): print "\t", item @@ -33,24 +29,11 @@ # On some systems, loading the OpenGL libraries needs the RTLD_GLOBAL mode. class Test_OpenGL_libs(unittest.TestCase): def setUp(self): - self.gl = self.glu = self.gle = self.glut = None + self.gl = self.glu = self.gle = None if lib_gl: self.gl = CDLL(lib_gl, mode=RTLD_GLOBAL) if lib_glu: self.glu = CDLL(lib_glu, RTLD_GLOBAL) - if lib_glut: - # On some systems, additional libraries seem to be - # required, loading glut fails with - # "OSError: /usr/lib/libglut.so.3: undefined symbol: XGetExtensionVersion" - # I cannot figure out how to repair the test on these - # systems (red hat), so we ignore it when the glut or gle - # libraries cannot be loaded. See also: - # https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1478253&group_id=5470 - # http://mail.python.org/pipermail/python-dev/2006-May/064789.html - try: - self.glut = CDLL(lib_glut) - except OSError: - pass if lib_gle: try: self.gle = CDLL(lib_gle) @@ -67,11 +50,6 @@ if self.glu: self.glu.gluBeginCurve - if lib_glut: - def test_glut(self): - if self.glut: - self.glut.glutWireTetrahedron - if lib_gle: def test_gle(self): if self.gle: From python-checkins at python.org Fri Jul 13 19:07:55 2007 From: python-checkins at python.org (thomas.heller) Date: Fri, 13 Jul 2007 19:07:55 +0200 (CEST) Subject: [Python-checkins] r56351 - in python/branches/release25-maint: Lib/ctypes/__init__.py Misc/NEWS Message-ID: <20070713170755.B4A5B1E400A@bag.python.org> Author: thomas.heller Date: Fri Jul 13 19:07:55 2007 New Revision: 56351 Modified: python/branches/release25-maint/Lib/ctypes/__init__.py python/branches/release25-maint/Misc/NEWS Log: Fix for SF# 1701409: segfault in c_char_p of ctypes. The repr output of c_char_p and c_wchar_p has changed as a sideeffect. Modified: python/branches/release25-maint/Lib/ctypes/__init__.py ============================================================================== --- python/branches/release25-maint/Lib/ctypes/__init__.py (original) +++ python/branches/release25-maint/Lib/ctypes/__init__.py Fri Jul 13 19:07:55 2007 @@ -226,6 +226,14 @@ class c_char_p(_SimpleCData): _type_ = "z" + if _os.name == "nt": + def __repr__(self): + if not windll.kernel32.IsBadStringPtrA(self, -1): + return "%s(%r)" % (self.__class__.__name__, self.value) + return "%s(%s)" % (self.__class__.__name__, cast(self, c_void_p).value) + else: + def __repr__(self): + return "%s(%s)" % (self.__class__.__name__, cast(self, c_void_p).value) _check_size(c_char_p, "P") class c_void_p(_SimpleCData): Modified: python/branches/release25-maint/Misc/NEWS ============================================================================== --- python/branches/release25-maint/Misc/NEWS (original) +++ python/branches/release25-maint/Misc/NEWS Fri Jul 13 19:07:55 2007 @@ -26,6 +26,10 @@ Library ------- +- Bug #1701409: Fix a segfault in printing ctypes.c_char_p and + ctypes.c_wchar_p when they point to an invalid location. As a + sideeffect the representation of these instances has changed. + - Bug #1734723: Fix repr.Repr() so it doesn't ignore the maxtuple attribute. - Bug #1728403: Fix a bug that CJKCodecs StreamReader hangs when it From buildbot at python.org Fri Jul 13 19:08:30 2007 From: buildbot at python.org (buildbot at python.org) Date: Fri, 13 Jul 2007 17:08:30 +0000 Subject: [Python-checkins] buildbot warnings in alpha Tru64 5.1 trunk Message-ID: <20070713170830.813A61E400A@bag.python.org> The Buildbot has detected a new failure of alpha Tru64 5.1 trunk. Full details are available at: http://www.python.org/dev/buildbot/all/alpha%2520Tru64%25205.1%2520trunk/builds/1711 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: thomas.heller Build had warnings: warnings test Excerpt from the test logfile: sincerely, -The Buildbot From python-checkins at python.org Fri Jul 13 19:12:24 2007 From: python-checkins at python.org (thomas.heller) Date: Fri, 13 Jul 2007 19:12:24 +0200 (CEST) Subject: [Python-checkins] r56352 - in python/trunk: Lib/ctypes/__init__.py Misc/NEWS Message-ID: <20070713171224.25DC11E400A@bag.python.org> Author: thomas.heller Date: Fri Jul 13 19:12:23 2007 New Revision: 56352 Modified: python/trunk/Lib/ctypes/__init__.py python/trunk/Misc/NEWS Log: Fix for SF# 1701409: segfault in c_char_p of ctypes. The repr output of c_char_p and c_wchar_p has changed as a sideeffect. Modified: python/trunk/Lib/ctypes/__init__.py ============================================================================== --- python/trunk/Lib/ctypes/__init__.py (original) +++ python/trunk/Lib/ctypes/__init__.py Fri Jul 13 19:12:23 2007 @@ -226,6 +226,14 @@ class c_char_p(_SimpleCData): _type_ = "z" + if _os.name == "nt": + def __repr__(self): + if not windll.kernel32.IsBadStringPtrA(self, -1): + return "%s(%r)" % (self.__class__.__name__, self.value) + return "%s(%s)" % (self.__class__.__name__, cast(self, c_void_p).value) + else: + def __repr__(self): + return "%s(%s)" % (self.__class__.__name__, cast(self, c_void_p).value) _check_size(c_char_p, "P") class c_void_p(_SimpleCData): Modified: python/trunk/Misc/NEWS ============================================================================== --- python/trunk/Misc/NEWS (original) +++ python/trunk/Misc/NEWS Fri Jul 13 19:12:23 2007 @@ -234,6 +234,10 @@ Library ------- +- Bug #1701409: Fix a segfault in printing ctypes.c_char_p and + ctypes.c_wchar_p when they point to an invalid location. As a + sideeffect the representation of these instances has changed. + - tarfile.py: Added "exclude" keyword argument to TarFile.add(). - Bug #1734723: Fix repr.Repr() so it doesn't ignore the maxtuple attribute. From buildbot at python.org Fri Jul 13 19:18:06 2007 From: buildbot at python.org (buildbot at python.org) Date: Fri, 13 Jul 2007 17:18:06 +0000 Subject: [Python-checkins] buildbot warnings in x86 mvlgcc trunk Message-ID: <20070713171806.9336E1E4012@bag.python.org> The Buildbot has detected a new failure of x86 mvlgcc trunk. Full details are available at: http://www.python.org/dev/buildbot/all/x86%2520mvlgcc%2520trunk/builds/647 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: thomas.heller Build had warnings: warnings test Excerpt from the test logfile: Traceback (most recent call last): File "/home2/buildbot/slave/trunk.loewis-linux/build/Lib/threading.py", line 465, in __bootstrap self.run() File "/home2/buildbot/slave/trunk.loewis-linux/build/Lib/test/test_urllib2_localnet.py", line 66, in run self._RequestHandlerClass) File "/home2/buildbot/slave/trunk.loewis-linux/build/Lib/test/test_urllib2_localnet.py", line 22, in __init__ RequestHandlerClass) File "/home2/buildbot/slave/trunk.loewis-linux/build/Lib/SocketServer.py", line 331, in __init__ self.server_bind() File "/home2/buildbot/slave/trunk.loewis-linux/build/Lib/BaseHTTPServer.py", line 101, in server_bind SocketServer.TCPServer.server_bind(self) File "/home2/buildbot/slave/trunk.loewis-linux/build/Lib/SocketServer.py", line 342, in server_bind self.socket.bind(self.server_address) File "", line 1, in bind error: (98, 'Address already in use') Traceback (most recent call last): File "/home2/buildbot/slave/trunk.loewis-linux/build/Lib/threading.py", line 465, in __bootstrap self.run() File "/home2/buildbot/slave/trunk.loewis-linux/build/Lib/test/test_urllib2_localnet.py", line 66, in run self._RequestHandlerClass) File "/home2/buildbot/slave/trunk.loewis-linux/build/Lib/test/test_urllib2_localnet.py", line 22, in __init__ RequestHandlerClass) File "/home2/buildbot/slave/trunk.loewis-linux/build/Lib/SocketServer.py", line 331, in __init__ self.server_bind() File "/home2/buildbot/slave/trunk.loewis-linux/build/Lib/BaseHTTPServer.py", line 101, in server_bind SocketServer.TCPServer.server_bind(self) File "/home2/buildbot/slave/trunk.loewis-linux/build/Lib/SocketServer.py", line 342, in server_bind self.socket.bind(self.server_address) File "", line 1, in bind error: (98, 'Address already in use') Traceback (most recent call last): File "/home2/buildbot/slave/trunk.loewis-linux/build/Lib/threading.py", line 465, in __bootstrap self.run() File "/home2/buildbot/slave/trunk.loewis-linux/build/Lib/test/test_urllib2_localnet.py", line 66, in run self._RequestHandlerClass) File "/home2/buildbot/slave/trunk.loewis-linux/build/Lib/test/test_urllib2_localnet.py", line 22, in __init__ RequestHandlerClass) File "/home2/buildbot/slave/trunk.loewis-linux/build/Lib/SocketServer.py", line 331, in __init__ self.server_bind() File "/home2/buildbot/slave/trunk.loewis-linux/build/Lib/BaseHTTPServer.py", line 101, in server_bind SocketServer.TCPServer.server_bind(self) File "/home2/buildbot/slave/trunk.loewis-linux/build/Lib/SocketServer.py", line 342, in server_bind self.socket.bind(self.server_address) File "", line 1, in bind error: (98, 'Address already in use') Traceback (most recent call last): File "/home2/buildbot/slave/trunk.loewis-linux/build/Lib/threading.py", line 465, in __bootstrap self.run() File "/home2/buildbot/slave/trunk.loewis-linux/build/Lib/test/test_urllib2_localnet.py", line 66, in run self._RequestHandlerClass) File "/home2/buildbot/slave/trunk.loewis-linux/build/Lib/test/test_urllib2_localnet.py", line 22, in __init__ RequestHandlerClass) File "/home2/buildbot/slave/trunk.loewis-linux/build/Lib/SocketServer.py", line 331, in __init__ self.server_bind() File "/home2/buildbot/slave/trunk.loewis-linux/build/Lib/BaseHTTPServer.py", line 101, in server_bind SocketServer.TCPServer.server_bind(self) File "/home2/buildbot/slave/trunk.loewis-linux/build/Lib/SocketServer.py", line 342, in server_bind self.socket.bind(self.server_address) File "", line 1, in bind error: (98, 'Address already in use') 1 test failed: test_urllib2_localnet ====================================================================== ERROR: test_proxy_qop_auth_int_works_or_throws_urlerror (test.test_urllib2_localnet.ProxyAuthTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home2/buildbot/slave/trunk.loewis-linux/build/Lib/test/test_urllib2_localnet.py", line 252, in setUp raise self.server.error error: (98, 'Address already in use') ====================================================================== ERROR: test_proxy_qop_auth_works (test.test_urllib2_localnet.ProxyAuthTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home2/buildbot/slave/trunk.loewis-linux/build/Lib/test/test_urllib2_localnet.py", line 252, in setUp raise self.server.error error: (98, 'Address already in use') ====================================================================== ERROR: test_proxy_with_bad_password_raises_httperror (test.test_urllib2_localnet.ProxyAuthTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home2/buildbot/slave/trunk.loewis-linux/build/Lib/test/test_urllib2_localnet.py", line 252, in setUp raise self.server.error error: (98, 'Address already in use') ====================================================================== ERROR: test_proxy_with_no_password_raises_httperror (test.test_urllib2_localnet.ProxyAuthTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home2/buildbot/slave/trunk.loewis-linux/build/Lib/test/test_urlException in thread Thread-119: Traceback (most recent call last): Traceback (most recent call last): File "/home2/buildbot/slave/trunk.loewis-linux/build/Lib/threading.py", line 478, in __bootstrap (self.getName(), _format_exc())) File "/home2/buildbot/slave/trunk.loewis-linux/build/Lib/traceback.py", line 236, in format_exc return ''.join(format_exception(etype, value, tb, limit)) File "/home2/buildbot/slave/trunk.loewis-linux/build/Lib/traceback.py", line 142, in format_exception list = list + format_tb(tb, limit) File "/home2/buildbot/slave/trunk.loewis-linux/build/Lib/traceback.py", line 76, in format_tb return format_list(extract_tb(tb, limit)) File "/home2/buildbot/slave/trunk.loewis-linux/build/Lib/traceback.py", line 101, in extract_tb line = linecache.getline(filename, lineno, f.f_globals) AttributeError: 'NoneType' object has no attribute 'getline' Traceback (most recent call last): File "/home2/buildbot/slave/trunk.loewis-linux/build/Lib/threading.py", line 478, in __bootstrap (self.getName(), _format_exc())) File "/home2/buildbot/slave/trunk.loewis-linux/build/Lib/traceback.py", line 236, in format_exc return ''.join(format_exception(etype, value, tb, limit)) File "/home2/buildbot/slave/trunk.loewis-linux/build/Lib/traceback.py", line 142, in format_exception list = list + format_tb(tb, limit) File "/home2/buildbot/slave/trunk.loewis-linux/build/Lib/traceback.py", line 76, in format_tb return format_list(extract_tb(tb, limit)) File "/home2/buildbot/slave/trunk.loewis-linux/build/Lib/traceback.py", line 101, in extract_tb line = linecache.getline(filename, lineno, f.f_globals) AttributeError: 'NoneType' object has no attribute 'getline' Traceback (most recent call last): File "/home2/buildbot/slave/trunk.loewis-linux/build/Lib/threading.py", line 478, in __bootstrap (self.getName(), _format_exc())) File "/home2/buildbot/slave/trunk.loewis-linux/build/Lib/traceback.py", line 236, in format_exc return ''.join(format_exception(etype, value, tb, limit)) File "/home2/buildbot/slave/trunk.loewis-linux/build/Lib/traceback.py", line 142, in format_exception list = list + format_tb(tb, limit) File "/home2/buildbot/slave/trunk.loewis-linux/build/Lib/traceback.py", line 76, in format_tb return format_list(extract_tb(tb, limit)) File "/home2/buildbot/slave/trunk.loewis-linux/build/Lib/traceback.py", line 101, in extract_tb line = linecache.getline(filename, lineno, f.f_globals) AttributeError: 'NoneType' object has no attribute 'getline' make: *** [buildbottest] Error 1 sincerely, -The Buildbot From python-checkins at python.org Fri Jul 13 19:33:13 2007 From: python-checkins at python.org (thomas.heller) Date: Fri, 13 Jul 2007 19:33:13 +0200 (CEST) Subject: [Python-checkins] r56353 - python/branches/release25-maint/Modules/_ctypes/callbacks.c python/branches/release25-maint/Modules/_ctypes/ctypes.h Message-ID: <20070713173313.CAA161E400A@bag.python.org> Author: thomas.heller Date: Fri Jul 13 19:33:13 2007 New Revision: 56353 Modified: python/branches/release25-maint/Modules/_ctypes/callbacks.c python/branches/release25-maint/Modules/_ctypes/ctypes.h Log: Fix for SF# 1649098: avoid zero-sized array declaration in structure. Modified: python/branches/release25-maint/Modules/_ctypes/callbacks.c ============================================================================== --- python/branches/release25-maint/Modules/_ctypes/callbacks.c (original) +++ python/branches/release25-maint/Modules/_ctypes/callbacks.c Fri Jul 13 19:33:13 2007 @@ -268,7 +268,7 @@ ffi_abi cc; nArgs = PySequence_Size(converters); - p = (ffi_info *)PyMem_Malloc(sizeof(ffi_info) + sizeof(ffi_type) * (nArgs + 1)); + p = (ffi_info *)PyMem_Malloc(sizeof(ffi_info) + sizeof(ffi_type) * (nArgs)); if (p == NULL) { PyErr_NoMemory(); return NULL; Modified: python/branches/release25-maint/Modules/_ctypes/ctypes.h ============================================================================== --- python/branches/release25-maint/Modules/_ctypes/ctypes.h (original) +++ python/branches/release25-maint/Modules/_ctypes/ctypes.h Fri Jul 13 19:33:13 2007 @@ -74,7 +74,7 @@ PyObject *callable; SETFUNC setfunc; ffi_type *restype; - ffi_type *atypes[0]; + ffi_type *atypes[1]; } ffi_info; typedef struct { From buildbot at python.org Fri Jul 13 19:34:43 2007 From: buildbot at python.org (buildbot at python.org) Date: Fri, 13 Jul 2007 17:34:43 +0000 Subject: [Python-checkins] buildbot warnings in ppc Debian unstable trunk Message-ID: <20070713173443.D6BC91E4012@bag.python.org> The Buildbot has detected a new failure of ppc Debian unstable trunk. Full details are available at: http://www.python.org/dev/buildbot/all/ppc%2520Debian%2520unstable%2520trunk/builds/49 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: thomas.heller Build had warnings: warnings test Excerpt from the test logfile: 1 test failed: test_resource ====================================================================== ERROR: test_fsize_enforced (test.test_resource.ResourceTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/pybot/buildarea/trunk.klose-debian-ppc/build/Lib/test/test_resource.py", line 59, in test_fsize_enforced f.close() IOError: [Errno 27] File too large make: *** [buildbottest] Error 1 sincerely, -The Buildbot From python-checkins at python.org Fri Jul 13 19:36:01 2007 From: python-checkins at python.org (thomas.heller) Date: Fri, 13 Jul 2007 19:36:01 +0200 (CEST) Subject: [Python-checkins] r56354 - python/branches/release25-maint/Misc/NEWS Message-ID: <20070713173601.B1CD81E400C@bag.python.org> Author: thomas.heller Date: Fri Jul 13 19:36:01 2007 New Revision: 56354 Modified: python/branches/release25-maint/Misc/NEWS Log: News entry for Fix for SF# 1649098: avoid zero-sized array declaration in structure. Modified: python/branches/release25-maint/Misc/NEWS ============================================================================== --- python/branches/release25-maint/Misc/NEWS (original) +++ python/branches/release25-maint/Misc/NEWS Fri Jul 13 19:36:01 2007 @@ -71,6 +71,9 @@ Extension Modules ----------------- +- Bug #1649098: Avoid declaration of zero-sized array declaration in + structure. + - Bug #1703286: ctypes no longer truncates 64-bit pointers. - Bug #1721309: prevent bsddb module from freeing random memory. From python-checkins at python.org Fri Jul 13 19:46:55 2007 From: python-checkins at python.org (thomas.heller) Date: Fri, 13 Jul 2007 19:46:55 +0200 (CEST) Subject: [Python-checkins] r56355 - in python/trunk: Misc/NEWS Modules/_ctypes/callbacks.c Modules/_ctypes/ctypes.h Message-ID: <20070713174655.4E69A1E4015@bag.python.org> Author: thomas.heller Date: Fri Jul 13 19:46:54 2007 New Revision: 56355 Modified: python/trunk/Misc/NEWS python/trunk/Modules/_ctypes/callbacks.c python/trunk/Modules/_ctypes/ctypes.h Log: Fix for SF# 1649098: avoid zero-sized array declaration in structure. Modified: python/trunk/Misc/NEWS ============================================================================== --- python/trunk/Misc/NEWS (original) +++ python/trunk/Misc/NEWS Fri Jul 13 19:46:54 2007 @@ -708,6 +708,9 @@ Extension Modules ----------------- +- Bug #1649098: Avoid declaration of zero-sized array declaration in + structure. + - Removed the rgbimg module; been deprecated since Python 2.5. - Bug #1721309: prevent bsddb module from freeing random memory. Modified: python/trunk/Modules/_ctypes/callbacks.c ============================================================================== --- python/trunk/Modules/_ctypes/callbacks.c (original) +++ python/trunk/Modules/_ctypes/callbacks.c Fri Jul 13 19:46:54 2007 @@ -268,7 +268,7 @@ ffi_abi cc; nArgs = PySequence_Size(converters); - p = (ffi_info *)PyMem_Malloc(sizeof(ffi_info) + sizeof(ffi_type) * (nArgs + 1)); + p = (ffi_info *)PyMem_Malloc(sizeof(ffi_info) + sizeof(ffi_type) * (nArgs)); if (p == NULL) { PyErr_NoMemory(); return NULL; Modified: python/trunk/Modules/_ctypes/ctypes.h ============================================================================== --- python/trunk/Modules/_ctypes/ctypes.h (original) +++ python/trunk/Modules/_ctypes/ctypes.h Fri Jul 13 19:46:54 2007 @@ -75,7 +75,7 @@ PyObject *callable; SETFUNC setfunc; ffi_type *restype; - ffi_type *atypes[0]; + ffi_type *atypes[1]; } ffi_info; typedef struct { From buildbot at python.org Fri Jul 13 20:09:05 2007 From: buildbot at python.org (buildbot at python.org) Date: Fri, 13 Jul 2007 18:09:05 +0000 Subject: [Python-checkins] buildbot warnings in ppc Debian unstable 2.5 Message-ID: <20070713180905.59CED1E400F@bag.python.org> The Buildbot has detected a new failure of ppc Debian unstable 2.5. Full details are available at: http://www.python.org/dev/buildbot/all/ppc%2520Debian%2520unstable%25202.5/builds/18 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch branches/release25-maint] HEAD Blamelist: georg.brandl,thomas.heller Build had warnings: warnings test Excerpt from the test logfile: 1 test failed: test_resource Traceback (most recent call last): File "./Lib/test/regrtest.py", line 549, in runtest_inner the_package = __import__(abstest, globals(), locals(), []) File "/home/pybot/buildarea/2.5.klose-debian-ppc/build/Lib/test/test_resource.py", line 42, in f.close() IOError: [Errno 27] File too large make: *** [buildbottest] Error 1 sincerely, -The Buildbot From buildbot at python.org Fri Jul 13 20:24:03 2007 From: buildbot at python.org (buildbot at python.org) Date: Fri, 13 Jul 2007 18:24:03 +0000 Subject: [Python-checkins] buildbot warnings in hppa Ubuntu dapper trunk Message-ID: <20070713182403.EF1001E400A@bag.python.org> The Buildbot has detected a new failure of hppa Ubuntu dapper trunk. Full details are available at: http://www.python.org/dev/buildbot/all/hppa%2520Ubuntu%2520dapper%2520trunk/builds/13 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: thomas.heller Build had warnings: warnings test Excerpt from the test logfile: make: *** [buildbottest] Segmentation fault sincerely, -The Buildbot From buildbot at python.org Fri Jul 13 21:25:44 2007 From: buildbot at python.org (buildbot at python.org) Date: Fri, 13 Jul 2007 19:25:44 +0000 Subject: [Python-checkins] buildbot warnings in alpha Tru64 5.1 2.5 Message-ID: <20070713192544.D79001E4014@bag.python.org> The Buildbot has detected a new failure of alpha Tru64 5.1 2.5. Full details are available at: http://www.python.org/dev/buildbot/all/alpha%2520Tru64%25205.1%25202.5/builds/281 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch branches/release25-maint] HEAD Blamelist: thomas.heller Build had warnings: warnings test Excerpt from the test logfile: 2 tests failed: test_signal test_socket Traceback (most recent call last): File "./Lib/test/regrtest.py", line 549, in runtest_inner the_package = __import__(abstest, globals(), locals(), []) File "/net/taipan/scratch1/nnorwitz/python/2.5.norwitz-tru64/build/Lib/test/test_signal.py", line 143, in print "KeyboardInterrupt (the alarm() went off)" File "/net/taipan/scratch1/nnorwitz/python/2.5.norwitz-tru64/build/Lib/test/test_signal.py", line 49, in handlerB raise HandlerBCalled, args HandlerBCalled: (30, ) sincerely, -The Buildbot From buildbot at python.org Fri Jul 13 21:33:30 2007 From: buildbot at python.org (buildbot at python.org) Date: Fri, 13 Jul 2007 19:33:30 +0000 Subject: [Python-checkins] buildbot warnings in alpha Debian trunk Message-ID: <20070713193330.8E0201E4010@bag.python.org> The Buildbot has detected a new failure of alpha Debian trunk. Full details are available at: http://www.python.org/dev/buildbot/all/alpha%2520Debian%2520trunk/builds/44 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: thomas.heller Build had warnings: warnings test Excerpt from the test logfile: 2 tests failed: test_pow test_resource ====================================================================== ERROR: test_bug705231 (test.test_pow.PowTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/pybot/buildarea/trunk.klose-debian-alpha/build/Lib/test/test_pow.py", line 109, in test_bug705231 eq(pow(a, 1.23e167), 1.0) ValueError: negative number cannot be raised to a fractional power ====================================================================== ERROR: test_fsize_enforced (test.test_resource.ResourceTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/pybot/buildarea/trunk.klose-debian-alpha/build/Lib/test/test_resource.py", line 59, in test_fsize_enforced f.close() IOError: [Errno 27] File too large sincerely, -The Buildbot From python-checkins at python.org Fri Jul 13 21:51:55 2007 From: python-checkins at python.org (thomas.heller) Date: Fri, 13 Jul 2007 21:51:55 +0200 (CEST) Subject: [Python-checkins] r56357 - python/trunk/Modules/_ctypes/stgdict.c Message-ID: <20070713195155.596481E400A@bag.python.org> Author: thomas.heller Date: Fri Jul 13 21:51:55 2007 New Revision: 56357 Modified: python/trunk/Modules/_ctypes/stgdict.c Log: PyType_stgdict() returns a borrowed reference which must not be Py_DECREF'd. Modified: python/trunk/Modules/_ctypes/stgdict.c ============================================================================== --- python/trunk/Modules/_ctypes/stgdict.c (original) +++ python/trunk/Modules/_ctypes/stgdict.c Fri Jul 13 21:51:55 2007 @@ -469,13 +469,11 @@ if (!prop) { Py_DECREF(pair); - Py_DECREF((PyObject *)stgdict); return -1; } if (-1 == PyDict_SetItem(realdict, name, prop)) { Py_DECREF(prop); Py_DECREF(pair); - Py_DECREF((PyObject *)stgdict); return -1; } Py_DECREF(pair); From python-checkins at python.org Fri Jul 13 21:52:42 2007 From: python-checkins at python.org (thomas.heller) Date: Fri, 13 Jul 2007 21:52:42 +0200 (CEST) Subject: [Python-checkins] r56358 - python/branches/release25-maint/Modules/_ctypes/stgdict.c Message-ID: <20070713195242.10E431E400A@bag.python.org> Author: thomas.heller Date: Fri Jul 13 21:52:41 2007 New Revision: 56358 Modified: python/branches/release25-maint/Modules/_ctypes/stgdict.c Log: PyType_stgdict() returns a borrowed reference which must not be Py_DECREF'd. Modified: python/branches/release25-maint/Modules/_ctypes/stgdict.c ============================================================================== --- python/branches/release25-maint/Modules/_ctypes/stgdict.c (original) +++ python/branches/release25-maint/Modules/_ctypes/stgdict.c Fri Jul 13 21:52:41 2007 @@ -465,13 +465,11 @@ if (!prop) { Py_DECREF(pair); - Py_DECREF((PyObject *)stgdict); return -1; } if (-1 == PyDict_SetItem(realdict, name, prop)) { Py_DECREF(prop); Py_DECREF(pair); - Py_DECREF((PyObject *)stgdict); return -1; } Py_DECREF(pair); From buildbot at python.org Fri Jul 13 22:57:12 2007 From: buildbot at python.org (buildbot at python.org) Date: Fri, 13 Jul 2007 20:57:12 +0000 Subject: [Python-checkins] buildbot warnings in amd64 XP trunk Message-ID: <20070713205712.748051E400A@bag.python.org> The Buildbot has detected a new failure of amd64 XP trunk. Full details are available at: http://www.python.org/dev/buildbot/all/amd64%2520XP%2520trunk/builds/47 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: thomas.heller Build had warnings: warnings test Excerpt from the test logfile: 2 tests failed: test_ctypes test_winsound ====================================================================== ERROR: test_extremes (test.test_winsound.BeepTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "C:\buildbot\trunk.heller-windows-amd64\build\lib\test\test_winsound.py", line 18, in test_extremes winsound.Beep(37, 75) RuntimeError: Failed to beep ====================================================================== ERROR: test_increasingfrequency (test.test_winsound.BeepTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "C:\buildbot\trunk.heller-windows-amd64\build\lib\test\test_winsound.py", line 23, in test_increasingfrequency winsound.Beep(i, 75) RuntimeError: Failed to beep ====================================================================== ERROR: test_alias_asterisk (test.test_winsound.PlaySoundTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "C:\buildbot\trunk.heller-windows-amd64\build\lib\test\test_winsound.py", line 64, in test_alias_asterisk winsound.PlaySound('SystemAsterisk', winsound.SND_ALIAS) RuntimeError: Failed to play sound ====================================================================== ERROR: test_alias_exclamation (test.test_winsound.PlaySoundTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "C:\buildbot\trunk.heller-windows-amd64\build\lib\test\test_winsound.py", line 74, in test_alias_exclamation winsound.PlaySound('SystemExclamation', winsound.SND_ALIAS) RuntimeError: Failed to play sound ====================================================================== ERROR: test_alias_exit (test.test_winsound.PlaySoundTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "C:\buildbot\trunk.heller-windows-amd64\build\lib\test\test_winsound.py", line 84, in test_alias_exit winsound.PlaySound('SystemExit', winsound.SND_ALIAS) RuntimeError: Failed to play sound ====================================================================== ERROR: test_alias_hand (test.test_winsound.PlaySoundTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "C:\buildbot\trunk.heller-windows-amd64\build\lib\test\test_winsound.py", line 94, in test_alias_hand winsound.PlaySound('SystemHand', winsound.SND_ALIAS) RuntimeError: Failed to play sound ====================================================================== ERROR: test_alias_question (test.test_winsound.PlaySoundTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "C:\buildbot\trunk.heller-windows-amd64\build\lib\test\test_winsound.py", line 104, in test_alias_question winsound.PlaySound('SystemQuestion', winsound.SND_ALIAS) RuntimeError: Failed to play sound sincerely, -The Buildbot From python-checkins at python.org Fri Jul 13 23:20:38 2007 From: python-checkins at python.org (barry.warsaw) Date: Fri, 13 Jul 2007 23:20:38 +0200 (CEST) Subject: [Python-checkins] r56359 - python/branches/release25-maint/Lib/email/test/test_email.py python/branches/release25-maint/Lib/email/test/test_email_renamed.py Message-ID: <20070713212038.CEEC21E4010@bag.python.org> Author: barry.warsaw Date: Fri Jul 13 23:20:38 2007 New Revision: 56359 Modified: python/branches/release25-maint/Lib/email/test/test_email.py python/branches/release25-maint/Lib/email/test/test_email_renamed.py Log: In response to this SF bug: [ 1752723 ] email.message_from_string: initial line gets discarded I added a test to assert that when the first line of text passed to message_from_string() contains a leading space, the message ends up with the appropriate FirstHeaderLineIsContinuationDefect on its defects list. The bug is invalid. Modified: python/branches/release25-maint/Lib/email/test/test_email.py ============================================================================== --- python/branches/release25-maint/Lib/email/test/test_email.py (original) +++ python/branches/release25-maint/Lib/email/test/test_email.py Fri Jul 13 23:20:38 2007 @@ -1492,6 +1492,18 @@ self.failUnless(isinstance(bad.defects[0], Errors.StartBoundaryNotFoundDefect)) + def test_first_line_is_continuation_header(self): + eq = self.assertEqual + m = ' Line 1\nLine 2\nLine 3' + msg = email.message_from_string(m) + eq(msg.keys(), []) + eq(msg.get_payload(), 'Line 2\nLine 3') + eq(len(msg.defects), 1) + self.failUnless(isinstance(msg.defects[0], + Errors.FirstHeaderLineIsContinuationDefect)) + eq(msg.defects[0].line, ' Line 1\n') + + # Test RFC 2047 header encoding and decoding Modified: python/branches/release25-maint/Lib/email/test/test_email_renamed.py ============================================================================== --- python/branches/release25-maint/Lib/email/test/test_email_renamed.py (original) +++ python/branches/release25-maint/Lib/email/test/test_email_renamed.py Fri Jul 13 23:20:38 2007 @@ -1490,6 +1490,17 @@ self.failUnless(isinstance(bad.defects[0], errors.StartBoundaryNotFoundDefect)) + def test_first_line_is_continuation_header(self): + eq = self.assertEqual + m = ' Line 1\nLine 2\nLine 3' + msg = email.message_from_string(m) + eq(msg.keys(), []) + eq(msg.get_payload(), 'Line 2\nLine 3') + eq(len(msg.defects), 1) + self.failUnless(isinstance(msg.defects[0], + errors.FirstHeaderLineIsContinuationDefect)) + eq(msg.defects[0].line, ' Line 1\n') + # Test RFC 2047 header encoding and decoding From python-checkins at python.org Sat Jul 14 00:12:58 2007 From: python-checkins at python.org (barry.warsaw) Date: Sat, 14 Jul 2007 00:12:58 +0200 (CEST) Subject: [Python-checkins] r56360 - python/trunk/Lib/email/test/test_email.py python/trunk/Lib/email/test/test_email_renamed.py Message-ID: <20070713221258.B60D91E4011@bag.python.org> Author: barry.warsaw Date: Sat Jul 14 00:12:58 2007 New Revision: 56360 Modified: python/trunk/Lib/email/test/test_email.py python/trunk/Lib/email/test/test_email_renamed.py Log: In response to this SF bug: [ 1752723 ] email.message_from_string: initial line gets discarded I added a test to assert that when the first line of text passed to message_from_string() contains a leading space, the message ends up with the appropriate FirstHeaderLineIsContinuationDefect on its defects list. The bug is invalid. Modified: python/trunk/Lib/email/test/test_email.py ============================================================================== --- python/trunk/Lib/email/test/test_email.py (original) +++ python/trunk/Lib/email/test/test_email.py Sat Jul 14 00:12:58 2007 @@ -1492,6 +1492,18 @@ self.failUnless(isinstance(bad.defects[0], Errors.StartBoundaryNotFoundDefect)) + def test_first_line_is_continuation_header(self): + eq = self.assertEqual + m = ' Line 1\nLine 2\nLine 3' + msg = email.message_from_string(m) + eq(msg.keys(), []) + eq(msg.get_payload(), 'Line 2\nLine 3') + eq(len(msg.defects), 1) + self.failUnless(isinstance(msg.defects[0], + Errors.FirstHeaderLineIsContinuationDefect)) + eq(msg.defects[0].line, ' Line 1\n') + + # Test RFC 2047 header encoding and decoding Modified: python/trunk/Lib/email/test/test_email_renamed.py ============================================================================== --- python/trunk/Lib/email/test/test_email_renamed.py (original) +++ python/trunk/Lib/email/test/test_email_renamed.py Sat Jul 14 00:12:58 2007 @@ -1490,6 +1490,17 @@ self.failUnless(isinstance(bad.defects[0], errors.StartBoundaryNotFoundDefect)) + def test_first_line_is_continuation_header(self): + eq = self.assertEqual + m = ' Line 1\nLine 2\nLine 3' + msg = email.message_from_string(m) + eq(msg.keys(), []) + eq(msg.get_payload(), 'Line 2\nLine 3') + eq(len(msg.defects), 1) + self.failUnless(isinstance(msg.defects[0], + errors.FirstHeaderLineIsContinuationDefect)) + eq(msg.defects[0].line, ' Line 1\n') + # Test RFC 2047 header encoding and decoding From buildbot at python.org Sat Jul 14 00:59:30 2007 From: buildbot at python.org (buildbot at python.org) Date: Fri, 13 Jul 2007 22:59:30 +0000 Subject: [Python-checkins] buildbot warnings in hppa Ubuntu dapper 2.5 Message-ID: <20070713225931.0D60F1E4066@bag.python.org> The Buildbot has detected a new failure of hppa Ubuntu dapper 2.5. Full details are available at: http://www.python.org/dev/buildbot/all/hppa%2520Ubuntu%2520dapper%25202.5/builds/9 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch branches/release25-maint] HEAD Blamelist: thomas.heller Build had warnings: warnings test Excerpt from the test logfile: sincerely, -The Buildbot From python-checkins at python.org Sat Jul 14 13:31:35 2007 From: python-checkins at python.org (raymond.hettinger) Date: Sat, 14 Jul 2007 13:31:35 +0200 (CEST) Subject: [Python-checkins] r56363 - python/branches/release25-maint/Doc/lib/libitertools.tex Message-ID: <20070714113135.64BA31E4015@bag.python.org> Author: raymond.hettinger Date: Sat Jul 14 13:31:35 2007 New Revision: 56363 Modified: python/branches/release25-maint/Doc/lib/libitertools.tex Log: Backport 56345 Modified: python/branches/release25-maint/Doc/lib/libitertools.tex ============================================================================== --- python/branches/release25-maint/Doc/lib/libitertools.tex (original) +++ python/branches/release25-maint/Doc/lib/libitertools.tex Sat Jul 14 13:31:35 2007 @@ -117,7 +117,7 @@ Make an iterator that drops elements from the iterable as long as the predicate is true; afterwards, returns every element. Note, the iterator does not produce \emph{any} output until the predicate - is true, so it may have a lengthy start-up time. Equivalent to: + first becomes false, so it may have a lengthy start-up time. Equivalent to: \begin{verbatim} def dropwhile(predicate, iterable): @@ -474,8 +474,8 @@ return izip(mapping.iterkeys(), mapping.itervalues()) def nth(iterable, n): - "Returns the nth item or raise IndexError" - return list(islice(iterable, n, n+1))[0] + "Returns the nth item or raise StopIteration" + return islice(iterable, n, None).next() def all(seq, pred=None): "Returns True if pred(x) is true for every element in the iterable" From python-checkins at python.org Sat Jul 14 19:12:23 2007 From: python-checkins at python.org (georg.brandl) Date: Sat, 14 Jul 2007 19:12:23 +0200 (CEST) Subject: [Python-checkins] r56364 - python/trunk/Doc/lib/libsubprocess.tex Message-ID: <20070714171223.552201E400B@bag.python.org> Author: georg.brandl Date: Sat Jul 14 19:12:23 2007 New Revision: 56364 Modified: python/trunk/Doc/lib/libsubprocess.tex Log: Bug #1753406: missing \versionadded for subprocess.check_call. Modified: python/trunk/Doc/lib/libsubprocess.tex ============================================================================== --- python/trunk/Doc/lib/libsubprocess.tex (original) +++ python/trunk/Doc/lib/libsubprocess.tex Sat Jul 14 19:12:23 2007 @@ -147,6 +147,8 @@ \begin{verbatim} check_call(["ls", "-l"]) \end{verbatim} + +\versionadded{2.5} \end{funcdesc} \subsubsection{Exceptions} From python-checkins at python.org Sat Jul 14 19:12:27 2007 From: python-checkins at python.org (georg.brandl) Date: Sat, 14 Jul 2007 19:12:27 +0200 (CEST) Subject: [Python-checkins] r56365 - python/branches/release25-maint/Doc/lib/libsubprocess.tex Message-ID: <20070714171227.79B831E4010@bag.python.org> Author: georg.brandl Date: Sat Jul 14 19:12:27 2007 New Revision: 56365 Modified: python/branches/release25-maint/Doc/lib/libsubprocess.tex Log: Bug #1753406: missing \versionadded for subprocess.check_call. (backport from rev. 56364) Modified: python/branches/release25-maint/Doc/lib/libsubprocess.tex ============================================================================== --- python/branches/release25-maint/Doc/lib/libsubprocess.tex (original) +++ python/branches/release25-maint/Doc/lib/libsubprocess.tex Sat Jul 14 19:12:27 2007 @@ -144,6 +144,8 @@ \begin{verbatim} check_call(["ls", "-l"]) \end{verbatim} + +\versionadded{2.5} \end{funcdesc} \subsubsection{Exceptions} From python-checkins at python.org Sat Jul 14 19:32:42 2007 From: python-checkins at python.org (georg.brandl) Date: Sat, 14 Jul 2007 19:32:42 +0200 (CEST) Subject: [Python-checkins] r56366 - python/trunk/Doc/lib/libwebbrowser.tex Message-ID: <20070714173242.118C81E400B@bag.python.org> Author: georg.brandl Date: Sat Jul 14 19:32:41 2007 New Revision: 56366 Modified: python/trunk/Doc/lib/libwebbrowser.tex Log: Clarify webbrowser.open description. Modified: python/trunk/Doc/lib/libwebbrowser.tex ============================================================================== --- python/trunk/Doc/lib/libwebbrowser.tex (original) +++ python/trunk/Doc/lib/libwebbrowser.tex Sat Jul 14 19:32:41 2007 @@ -47,7 +47,7 @@ \begin{funcdesc}{open}{url\optional{, new=0\optional{, autoraise=1}}} Display \var{url} using the default browser. If \var{new} is 0, the - \var{url} is opened in the same browser window. If \var{new} is 1, + \var{url} is opened in the same browser window if possible. If \var{new} is 1, a new browser window is opened if possible. If \var{new} is 2, a new browser page ("tab") is opened if possible. If \var{autoraise} is true, the window is raised if possible (note that under many window From python-checkins at python.org Sat Jul 14 19:32:47 2007 From: python-checkins at python.org (georg.brandl) Date: Sat, 14 Jul 2007 19:32:47 +0200 (CEST) Subject: [Python-checkins] r56367 - python/branches/release25-maint/Doc/lib/libwebbrowser.tex Message-ID: <20070714173247.561F31E400B@bag.python.org> Author: georg.brandl Date: Sat Jul 14 19:32:47 2007 New Revision: 56367 Modified: python/branches/release25-maint/Doc/lib/libwebbrowser.tex Log: Clarify webbrowser.open description. (backport from rev. 56366) Modified: python/branches/release25-maint/Doc/lib/libwebbrowser.tex ============================================================================== --- python/branches/release25-maint/Doc/lib/libwebbrowser.tex (original) +++ python/branches/release25-maint/Doc/lib/libwebbrowser.tex Sat Jul 14 19:32:47 2007 @@ -47,7 +47,7 @@ \begin{funcdesc}{open}{url\optional{, new=0\optional{, autoraise=1}}} Display \var{url} using the default browser. If \var{new} is 0, the - \var{url} is opened in the same browser window. If \var{new} is 1, + \var{url} is opened in the same browser window if possible. If \var{new} is 1, a new browser window is opened if possible. If \var{new} is 2, a new browser page ("tab") is opened if possible. If \var{autoraise} is true, the window is raised if possible (note that under many window From python-checkins at python.org Sat Jul 14 20:22:12 2007 From: python-checkins at python.org (collin.winter) Date: Sat, 14 Jul 2007 20:22:12 +0200 (CEST) Subject: [Python-checkins] r56368 - in sandbox/trunk/2to3: fixes/fix_dict.py fixes/fix_filter.py fixes/fix_map.py patcomp.py Message-ID: <20070714182212.94A461E400C@bag.python.org> Author: collin.winter Date: Sat Jul 14 20:22:12 2007 New Revision: 56368 Modified: sandbox/trunk/2to3/ (props changed) sandbox/trunk/2to3/fixes/fix_dict.py sandbox/trunk/2to3/fixes/fix_filter.py sandbox/trunk/2to3/fixes/fix_map.py sandbox/trunk/2to3/patcomp.py Log: Add a compile_pattern() shortcut function to patcomp.py (and use it in several fixers). Modified: sandbox/trunk/2to3/fixes/fix_dict.py ============================================================================== --- sandbox/trunk/2to3/fixes/fix_dict.py (original) +++ sandbox/trunk/2to3/fixes/fix_dict.py Sat Jul 14 20:22:12 2007 @@ -68,13 +68,13 @@ return new P1 = "power< func=NAME trailer< '(' node=any ')' > any* >" - p1 = patcomp.PatternCompiler().compile_pattern(P1) + p1 = patcomp.compile_pattern(P1) P2 = """for_stmt< 'for' any 'in' node=any ':' any* > | list_for< 'for' any 'in' node=any any* > | gen_for< 'for' any 'in' node=any any* > """ - p2 = patcomp.PatternCompiler().compile_pattern(P2) + p2 = patcomp.compile_pattern(P2) def in_special_context(self, node, isiter): if node.parent is None: Modified: sandbox/trunk/2to3/fixes/fix_filter.py ============================================================================== --- sandbox/trunk/2to3/fixes/fix_filter.py (original) +++ sandbox/trunk/2to3/fixes/fix_filter.py Sat Jul 14 20:22:12 2007 @@ -62,7 +62,7 @@ | list_for< 'for' any 'in' node=any any* > | gen_for< 'for' any 'in' node=any any* > """ - p0 = patcomp.PatternCompiler().compile_pattern(P0) + p0 = patcomp.compile_pattern(P0) P1 = """ power< @@ -71,7 +71,7 @@ any* > """ - p1 = patcomp.PatternCompiler().compile_pattern(P1) + p1 = patcomp.compile_pattern(P1) P2 = """ power< @@ -80,7 +80,7 @@ any* > """ - p2 = patcomp.PatternCompiler().compile_pattern(P2) + p2 = patcomp.compile_pattern(P2) def in_special_context(self, node): p = node.parent Modified: sandbox/trunk/2to3/fixes/fix_map.py ============================================================================== --- sandbox/trunk/2to3/fixes/fix_map.py (original) +++ sandbox/trunk/2to3/fixes/fix_map.py Sat Jul 14 20:22:12 2007 @@ -74,7 +74,7 @@ | list_for< 'for' any 'in' node=any any* > | gen_for< 'for' any 'in' node=any any* > """ - p0 = patcomp.PatternCompiler().compile_pattern(P0) + p0 = patcomp.compile_pattern(P0) P1 = """ power< @@ -83,7 +83,7 @@ any* > """ - p1 = patcomp.PatternCompiler().compile_pattern(P1) + p1 = patcomp.compile_pattern(P1) P2 = """ power< @@ -92,7 +92,7 @@ any* > """ - p2 = patcomp.PatternCompiler().compile_pattern(P2) + p2 = patcomp.compile_pattern(P2) def in_special_context(self, node): p = node.parent Modified: sandbox/trunk/2to3/patcomp.py ============================================================================== --- sandbox/trunk/2to3/patcomp.py (original) +++ sandbox/trunk/2to3/patcomp.py Sat Jul 14 20:22:12 2007 @@ -182,15 +182,5 @@ return pytree.Leaf(type, value, context=context) -_SAMPLE = """(a=(power< ('apply' trailer<'(' b=(not STRING) ')'> ) >){1}) -{1,1}""" - - -def _test(): - pc = PatternCompiler() - pat = pc.compile_pattern(_SAMPLE) - print pat - - -if __name__ == "__main__": - _test() +def compile_pattern(pattern): + return PatternCompiler().compile_pattern(pattern) From python-checkins at python.org Sat Jul 14 20:22:28 2007 From: python-checkins at python.org (collin.winter) Date: Sat, 14 Jul 2007 20:22:28 +0200 (CEST) Subject: [Python-checkins] r56369 - in sandbox/trunk/2to3: tests/test_fixers.py Message-ID: <20070714182228.1593E1E400C@bag.python.org> Author: collin.winter Date: Sat Jul 14 20:22:27 2007 New Revision: 56369 Modified: sandbox/trunk/2to3/ (props changed) sandbox/trunk/2to3/tests/test_fixers.py Log: Tighten down the fixer tests to make sure we're tracking all warnings. Modified: sandbox/trunk/2to3/tests/test_fixers.py ============================================================================== --- sandbox/trunk/2to3/tests/test_fixers.py (original) +++ sandbox/trunk/2to3/tests/test_fixers.py Sat Jul 14 20:22:27 2007 @@ -49,15 +49,22 @@ sh.setFormatter(logging.Formatter("%(message)s")) self.refactor.fixers = [Fixer(f, sh) for f in self.refactor.fixers] - def check(self, before, after): + def tearDown(self): + self.logging_stream = None + + def _check(self, before, after): before = support.reformat(before) after = support.reformat(after) refactored = self.refactor_stream("", StringIO(before)) self.failUnlessEqual(after, refactored) - def warns(self, before, after, message): - self.check(before, after) + def check(self, before, after, ignore_warnings=False): + self._check(before, after) + if not ignore_warnings: + self.failUnlessEqual(self.logging_stream.getvalue(), "") + def warns(self, before, after, message): + self._check(before, after) self.failUnless(message in self.logging_stream.getvalue()) def refactor_stream(self, stream_name, stream): @@ -1422,7 +1429,7 @@ foo(a) a.__next__() """ - self.check(b, a) + self.check(b, a, ignore_warnings=True) def test_prefix_preservation_4(self): b = """ @@ -1439,7 +1446,7 @@ # def a.__next__() """ - self.check(b, a) + self.check(b, a, ignore_warnings=True) def test_prefix_preservation_5(self): b = """ @@ -1454,7 +1461,7 @@ foo(foo(a), # abc a.__next__()) """ - self.check(b, a) + self.check(b, a, ignore_warnings=True) def test_prefix_preservation_6(self): b = """ From python-checkins at python.org Sat Jul 14 20:22:44 2007 From: python-checkins at python.org (collin.winter) Date: Sat, 14 Jul 2007 20:22:44 +0200 (CEST) Subject: [Python-checkins] r56370 - in sandbox/trunk/2to3: fixes/fix_apply.py fixes/fix_callable.py fixes/fix_dict.py fixes/fix_dummy.py fixes/fix_except.py fixes/fix_exec.py fixes/fix_filter.py fixes/fix_has_key.py fixes/fix_input.py fixes/fix_intern.py fixes/fix_long.py fixes/fix_map.py fixes/fix_ne.py fixes/fix_next.py fixes/fix_nonzero.py fixes/fix_numliterals.py fixes/fix_print.py fixes/fix_raise.py fixes/fix_raw_input.py fixes/fix_repr.py fixes/fix_sysexcattrs.py fixes/fix_throw.py fixes/fix_tuple_params.py fixes/fix_unicode.py fixes/fix_xrange.py refactor.py Message-ID: <20070714182244.7F3381E400C@bag.python.org> Author: collin.winter Date: Sat Jul 14 20:22:43 2007 New Revision: 56370 Modified: sandbox/trunk/2to3/ (props changed) sandbox/trunk/2to3/fixes/fix_apply.py sandbox/trunk/2to3/fixes/fix_callable.py sandbox/trunk/2to3/fixes/fix_dict.py sandbox/trunk/2to3/fixes/fix_dummy.py sandbox/trunk/2to3/fixes/fix_except.py sandbox/trunk/2to3/fixes/fix_exec.py sandbox/trunk/2to3/fixes/fix_filter.py sandbox/trunk/2to3/fixes/fix_has_key.py sandbox/trunk/2to3/fixes/fix_input.py sandbox/trunk/2to3/fixes/fix_intern.py sandbox/trunk/2to3/fixes/fix_long.py sandbox/trunk/2to3/fixes/fix_map.py sandbox/trunk/2to3/fixes/fix_ne.py sandbox/trunk/2to3/fixes/fix_next.py sandbox/trunk/2to3/fixes/fix_nonzero.py sandbox/trunk/2to3/fixes/fix_numliterals.py sandbox/trunk/2to3/fixes/fix_print.py sandbox/trunk/2to3/fixes/fix_raise.py sandbox/trunk/2to3/fixes/fix_raw_input.py sandbox/trunk/2to3/fixes/fix_repr.py sandbox/trunk/2to3/fixes/fix_sysexcattrs.py sandbox/trunk/2to3/fixes/fix_throw.py sandbox/trunk/2to3/fixes/fix_tuple_params.py sandbox/trunk/2to3/fixes/fix_unicode.py sandbox/trunk/2to3/fixes/fix_xrange.py sandbox/trunk/2to3/refactor.py Log: Pass the results dict from the fixer's match() method to transform() via refactor.py; this saves having to do the match twice. Modified: sandbox/trunk/2to3/fixes/fix_apply.py ============================================================================== --- sandbox/trunk/2to3/fixes/fix_apply.py (original) +++ sandbox/trunk/2to3/fixes/fix_apply.py Sat Jul 14 20:22:43 2007 @@ -27,9 +27,8 @@ > """ - def transform(self, node): + def transform(self, node, results): syms = self.syms - results = self.match(node) assert results func = results["func"] args = results["args"] Modified: sandbox/trunk/2to3/fixes/fix_callable.py ============================================================================== --- sandbox/trunk/2to3/fixes/fix_callable.py (original) +++ sandbox/trunk/2to3/fixes/fix_callable.py Sat Jul 14 20:22:43 2007 @@ -24,8 +24,7 @@ > """ - def transform(self, node): - results = self.match(node) + def transform(self, node, results): func = results["func"] args = [func.clone(), String(', '), String("'__call__'")] Modified: sandbox/trunk/2to3/fixes/fix_dict.py ============================================================================== --- sandbox/trunk/2to3/fixes/fix_dict.py (original) +++ sandbox/trunk/2to3/fixes/fix_dict.py Sat Jul 14 20:22:43 2007 @@ -39,8 +39,7 @@ > """ - def transform(self, node): - results = self.match(node) + def transform(self, node, results): head = results["head"] method = results["method"][0] # Extract node for method name tail = results["tail"] Modified: sandbox/trunk/2to3/fixes/fix_dummy.py ============================================================================== --- sandbox/trunk/2to3/fixes/fix_dummy.py (original) +++ sandbox/trunk/2to3/fixes/fix_dummy.py Sat Jul 14 20:22:43 2007 @@ -12,5 +12,5 @@ def match(self, node): return True - def transform(self, node): + def transform(self, node, results): node.changed() Modified: sandbox/trunk/2to3/fixes/fix_except.py ============================================================================== --- sandbox/trunk/2to3/fixes/fix_except.py (original) +++ sandbox/trunk/2to3/fixes/fix_except.py Sat Jul 14 20:22:43 2007 @@ -42,10 +42,9 @@ | 'finally' ':' suite) > """ - def transform(self, node): - syms = self.syms - results = self.match(node) + def transform(self, node, results): assert results + syms = self.syms try_cleanup = [ch.clone() for ch in results['cleanup']] for except_clause, e_suite in find_excepts(try_cleanup): Modified: sandbox/trunk/2to3/fixes/fix_exec.py ============================================================================== --- sandbox/trunk/2to3/fixes/fix_exec.py (original) +++ sandbox/trunk/2to3/fixes/fix_exec.py Sat Jul 14 20:22:43 2007 @@ -23,10 +23,9 @@ exec_stmt< 'exec' (not atom<'(' [any] ')'>) a=any > """ - def transform(self, node): - syms = self.syms - results = self.match(node) + def transform(self, node, results): assert results + syms = self.syms a = results["a"] b = results.get("b") c = results.get("c") Modified: sandbox/trunk/2to3/fixes/fix_filter.py ============================================================================== --- sandbox/trunk/2to3/fixes/fix_filter.py (original) +++ sandbox/trunk/2to3/fixes/fix_filter.py Sat Jul 14 20:22:43 2007 @@ -42,8 +42,7 @@ > """ - def transform(self, node): - results = self.match(node) + def transform(self, node, results): if "filter_lambda" in results: new = ListComp(results.get("fp").clone(), results.get("fp").clone(), Modified: sandbox/trunk/2to3/fixes/fix_has_key.py ============================================================================== --- sandbox/trunk/2to3/fixes/fix_has_key.py (original) +++ sandbox/trunk/2to3/fixes/fix_has_key.py Sat Jul 14 20:22:43 2007 @@ -68,10 +68,9 @@ > """ - def transform(self, node): - syms = self.syms - results = self.match(node) + def transform(self, node, results): assert results + syms = self.syms if (node.parent.type == syms.not_test and self.pattern.match(node.parent)): # Don't transform a node matching the first alternative of the Modified: sandbox/trunk/2to3/fixes/fix_input.py ============================================================================== --- sandbox/trunk/2to3/fixes/fix_input.py (original) +++ sandbox/trunk/2to3/fixes/fix_input.py Sat Jul 14 20:22:43 2007 @@ -14,7 +14,7 @@ > """ - def transform(self, node): + def transform(self, node, results): new = node.clone() new.set_prefix("") new = Call(Name("eval"), [new]) Modified: sandbox/trunk/2to3/fixes/fix_intern.py ============================================================================== --- sandbox/trunk/2to3/fixes/fix_intern.py (original) +++ sandbox/trunk/2to3/fixes/fix_intern.py Sat Jul 14 20:22:43 2007 @@ -23,10 +23,9 @@ > """ - def transform(self, node): - syms = self.syms - results = self.match(node) + def transform(self, node, results): assert results + syms = self.syms obj = results["obj"].clone() if obj.type == syms.arglist: newarglist = obj.clone() Modified: sandbox/trunk/2to3/fixes/fix_long.py ============================================================================== --- sandbox/trunk/2to3/fixes/fix_long.py (original) +++ sandbox/trunk/2to3/fixes/fix_long.py Sat Jul 14 20:22:43 2007 @@ -21,8 +21,7 @@ static_long = Name("long") static_int = Name("int") - def transform(self, node): - results = self.match(node) + def transform(self, node, results): long_type = results.get("long_type") number = results.get("number") new = None Modified: sandbox/trunk/2to3/fixes/fix_map.py ============================================================================== --- sandbox/trunk/2to3/fixes/fix_map.py (original) +++ sandbox/trunk/2to3/fixes/fix_map.py Sat Jul 14 20:22:43 2007 @@ -52,8 +52,7 @@ > """ - def transform(self, node): - results = self.match(node) + def transform(self, node, results): if "map_lambda" in results: new = ListComp(results.get("xp").clone(), results.get("fp").clone(), Modified: sandbox/trunk/2to3/fixes/fix_ne.py ============================================================================== --- sandbox/trunk/2to3/fixes/fix_ne.py (original) +++ sandbox/trunk/2to3/fixes/fix_ne.py Sat Jul 14 20:22:43 2007 @@ -16,7 +16,7 @@ # Override return node.type == token.NOTEQUAL and node.value == "<>" - def transform(self, node): + def transform(self, node, results): new = pytree.Leaf(token.NOTEQUAL, "!=") new.set_prefix(node.get_prefix()) return new Modified: sandbox/trunk/2to3/fixes/fix_next.py ============================================================================== --- sandbox/trunk/2to3/fixes/fix_next.py (original) +++ sandbox/trunk/2to3/fixes/fix_next.py Sat Jul 14 20:22:43 2007 @@ -73,8 +73,7 @@ self.shadowed_next = False self.delayed = [] - def transform(self, node): - results = self.match(node) + def transform(self, node, results): assert results base = results.get("base") Modified: sandbox/trunk/2to3/fixes/fix_nonzero.py ============================================================================== --- sandbox/trunk/2to3/fixes/fix_nonzero.py (original) +++ sandbox/trunk/2to3/fixes/fix_nonzero.py Sat Jul 14 20:22:43 2007 @@ -13,9 +13,8 @@ parameters< '(' NAME ')' > any+ > any* > > """ - - def transform(self, node): - results = self.match(node) + + def transform(self, node, results): assert results name = results["name"] Modified: sandbox/trunk/2to3/fixes/fix_numliterals.py ============================================================================== --- sandbox/trunk/2to3/fixes/fix_numliterals.py (original) +++ sandbox/trunk/2to3/fixes/fix_numliterals.py Sat Jul 14 20:22:43 2007 @@ -17,7 +17,7 @@ return (node.type == token.NUMBER and (node.value.startswith("0") or node.value[-1] in "Ll")) - def transform(self, node): + def transform(self, node, results): val = node.value if val[-1] in 'Ll': val = val[:-1] Modified: sandbox/trunk/2to3/fixes/fix_print.py ============================================================================== --- sandbox/trunk/2to3/fixes/fix_print.py (original) +++ sandbox/trunk/2to3/fixes/fix_print.py Sat Jul 14 20:22:43 2007 @@ -30,10 +30,9 @@ return None return self.pattern.match(node) - def transform(self, node): - syms = self.syms - results = self.match(node) + def transform(self, node, results): assert results + syms = self.syms if node == Name("print"): # Special-case print all by itself Modified: sandbox/trunk/2to3/fixes/fix_raise.py ============================================================================== --- sandbox/trunk/2to3/fixes/fix_raise.py (original) +++ sandbox/trunk/2to3/fixes/fix_raise.py Sat Jul 14 20:22:43 2007 @@ -33,10 +33,9 @@ raise_stmt< 'raise' exc=any [',' val=any [',' tb=any]] > """ - def transform(self, node): - syms = self.syms - results = self.match(node) + def transform(self, node, results): assert results + syms = self.syms exc = results["exc"].clone() if exc.type is token.STRING: Modified: sandbox/trunk/2to3/fixes/fix_raw_input.py ============================================================================== --- sandbox/trunk/2to3/fixes/fix_raw_input.py (original) +++ sandbox/trunk/2to3/fixes/fix_raw_input.py Sat Jul 14 20:22:43 2007 @@ -12,8 +12,7 @@ power< 'raw_input' args=trailer< '(' [any] ')' > > """ - def transform(self, node): - results = self.match(node) + def transform(self, node, results): args = results["args"] new = pytree.Node(self.syms.power, [Name("input"), args.clone()]) Modified: sandbox/trunk/2to3/fixes/fix_repr.py ============================================================================== --- sandbox/trunk/2to3/fixes/fix_repr.py (original) +++ sandbox/trunk/2to3/fixes/fix_repr.py Sat Jul 14 20:22:43 2007 @@ -14,9 +14,7 @@ atom < '`' expr=any '`' > """ - def transform(self, node): - results = self.match(node) - assert results + def transform(self, node, results): expr = results["expr"].clone() if expr.type == self.syms.testlist1: Modified: sandbox/trunk/2to3/fixes/fix_sysexcattrs.py ============================================================================== --- sandbox/trunk/2to3/fixes/fix_sysexcattrs.py (original) +++ sandbox/trunk/2to3/fixes/fix_sysexcattrs.py Sat Jul 14 20:22:43 2007 @@ -14,5 +14,5 @@ any* > """ - def transform(self, node): + def transform(self, node, results): self.cannot_convert(node, "This attribute is going away in Python 3") Modified: sandbox/trunk/2to3/fixes/fix_throw.py ============================================================================== --- sandbox/trunk/2to3/fixes/fix_throw.py (original) +++ sandbox/trunk/2to3/fixes/fix_throw.py Sat Jul 14 20:22:43 2007 @@ -23,10 +23,8 @@ power< any trailer< '.' 'throw' > trailer< '(' exc=any ')' > > """ - def transform(self, node): + def transform(self, node, results): syms = self.syms - results = self.match(node) - assert results exc = results["exc"].clone() if exc.type is token.STRING: Modified: sandbox/trunk/2to3/fixes/fix_tuple_params.py ============================================================================== --- sandbox/trunk/2to3/fixes/fix_tuple_params.py (original) +++ sandbox/trunk/2to3/fixes/fix_tuple_params.py Sat Jul 14 20:22:43 2007 @@ -32,8 +32,7 @@ | lambda=lambdef< 'lambda' args=vfpdef< any+ > ':' body=any >""" - def transform(self, node): - results = self.match(node) + def transform(self, node, results): assert results if "lambda" in results: Modified: sandbox/trunk/2to3/fixes/fix_unicode.py ============================================================================== --- sandbox/trunk/2to3/fixes/fix_unicode.py (original) +++ sandbox/trunk/2to3/fixes/fix_unicode.py Sat Jul 14 20:22:43 2007 @@ -11,7 +11,7 @@ PATTERN = "STRING | NAME<'unicode' | 'unichr'>" - def transform(self, node): + def transform(self, node, results): if node.type == token.NAME: if node.value == "unicode": new = node.clone() Modified: sandbox/trunk/2to3/fixes/fix_xrange.py ============================================================================== --- sandbox/trunk/2to3/fixes/fix_xrange.py (original) +++ sandbox/trunk/2to3/fixes/fix_xrange.py Sat Jul 14 20:22:43 2007 @@ -18,8 +18,7 @@ > """ - def transform(self, node): - results = self.match(node) + def transform(self, node, results): args = results["args"] new = pytree.Node(self.syms.power, [Name("range"), args.clone()]) Modified: sandbox/trunk/2to3/refactor.py ============================================================================== --- sandbox/trunk/2to3/refactor.py (original) +++ sandbox/trunk/2to3/refactor.py Sat Jul 14 20:22:43 2007 @@ -254,8 +254,9 @@ changes = 0 for node in tree.post_order(): for fixer in self.fixers: - if fixer.match(node): - new = fixer.transform(node) + results = fixer.match(node) + if results: + new = fixer.transform(node, results) if new is not None and (new != node or str(new) != str(node)): node.replace(new) From python-checkins at python.org Sat Jul 14 20:23:00 2007 From: python-checkins at python.org (collin.winter) Date: Sat, 14 Jul 2007 20:23:00 +0200 (CEST) Subject: [Python-checkins] r56371 - in sandbox/trunk/2to3: fixes/util.py Message-ID: <20070714182300.82C411E400C@bag.python.org> Author: collin.winter Date: Sat Jul 14 20:23:00 2007 New Revision: 56371 Modified: sandbox/trunk/2to3/ (props changed) sandbox/trunk/2to3/fixes/util.py Log: Add an attr_chain() utility function. Modified: sandbox/trunk/2to3/fixes/util.py ============================================================================== --- sandbox/trunk/2to3/fixes/util.py (original) +++ sandbox/trunk/2to3/fixes/util.py Sat Jul 14 20:23:00 2007 @@ -152,6 +152,29 @@ return l[::-1] ########################################################### +### Misc +########################################################### + +def attr_chain(obj, attr): + """Follow an attribute chain. + + If you have a chain of objects where a.foo -> b, b.foo-> c, etc, + use this to iterate over all objects in the chain. Iteration is + terminated by getattr(x, attr) is None. + + Args: + obj: the starting object + attr: the name of the chaining attribute + + Yields: + Each successive object in the chain. + """ + next = getattr(obj, attr) + while next: + yield next + next = getattr(next, attr) + +########################################################### ### The following functions are to find bindings in a suite ########################################################### From python-checkins at python.org Sat Jul 14 20:23:16 2007 From: python-checkins at python.org (collin.winter) Date: Sat, 14 Jul 2007 20:23:16 +0200 (CEST) Subject: [Python-checkins] r56372 - in sandbox/trunk/2to3: README fixes/fix_stringio.py tests/test_fixers.py Message-ID: <20070714182316.A70F51E400C@bag.python.org> Author: collin.winter Date: Sat Jul 14 20:23:16 2007 New Revision: 56372 Added: sandbox/trunk/2to3/fixes/fix_stringio.py Modified: sandbox/trunk/2to3/ (props changed) sandbox/trunk/2to3/README sandbox/trunk/2to3/tests/test_fixers.py Log: First draft of StringIO fixer. Modified: sandbox/trunk/2to3/README ============================================================================== --- sandbox/trunk/2to3/README (original) +++ sandbox/trunk/2to3/README Sat Jul 14 20:23:16 2007 @@ -71,6 +71,8 @@ * **fix_repr** - swap backticks for repr() calls. +* **fix_stringio** - StringIO.StringIO -> io.StringIO (imports, too). + * **fix_sysexcattrs** - warn on usage of sys.value, sys.type and sys.traceback. Added: sandbox/trunk/2to3/fixes/fix_stringio.py ============================================================================== --- (empty file) +++ sandbox/trunk/2to3/fixes/fix_stringio.py Sat Jul 14 20:23:16 2007 @@ -0,0 +1,81 @@ +"""StringIO.StringIO -> io.StringIO (imports, too). + +Imports this fixer picks up on: +* "import StringIO" -> "import io" +* "from StringIO import StringIO" -> "from io import StringIO" +* "import StringIO as foo" -> "import io as foo" + +If the fixer finds "import StringIO", all "StringIO.StringIO" attribute +lookups will be translated to "io.StringIO" and all "StringIO" names +will be translated to "io". +""" +# Author: Collin Winter + +# Local imports +import patcomp +from fixes import basefix +from fixes.util import Name, attr_chain, any + + +class DelayedStrLeaf(object): + def __init__(self, fixer, leaf): + self.fixer = fixer + self.leaf = leaf + self.parent = None + + def __getattr__(self, attr): + return getattr(self.leaf, attr) + + def __str__(self): + if self.fixer.module_import: + return self.leaf.get_prefix() + "io" + else: + return str(self.leaf) + + def clone(self): + return DelayedStrLeaf(self.fixer, self.leaf) + + +class FixStringio(basefix.BaseFix): + PATTERN = """ + import_name< 'import' (module='StringIO' + | dotted_as_names< any* module='StringIO' any* >) > + | + import_from< 'from' module_name='StringIO' 'import' + ( 'StringIO' | import_as_name< 'StringIO' 'as' any >) > + | + import_name< 'import' dotted_as_name< module_name='StringIO' 'as' any > > + | + power< module_name='StringIO' trailer< '.' 'StringIO' > any* > + | + bare_name='StringIO' + """ + + # Don't match 'StringIO' if it's within another match + def match(self, node): + match = super(FixStringio, self).match + results = match(node) + if results: + if any([match(obj) for obj in attr_chain(node, "parent")]): + return False + return results + return False + + def start_tree(self, tree, filename): + super(FixStringio, self).start_tree(tree, filename) + self.module_import = False + + def transform(self, node, results): + import_mod = results.get("module") + module_name = results.get("module_name") + bare_name = results.get("bare_name") + + if import_mod: + self.module_import = True + import_mod.replace(Name("io", prefix=import_mod.get_prefix())) + elif module_name: + module_name.replace(Name("io", prefix=module_name.get_prefix())) + elif bare_name: + bare_name.replace(DelayedStrLeaf(self, bare_name)) + else: + raise RuntimeError("Hmm, shouldn't have gotten here") Modified: sandbox/trunk/2to3/tests/test_fixers.py ============================================================================== --- sandbox/trunk/2to3/tests/test_fixers.py (original) +++ sandbox/trunk/2to3/tests/test_fixers.py Sat Jul 14 20:23:16 2007 @@ -1177,6 +1177,63 @@ self.check(b, a) +class Test_stringio(FixerTestCase): + fixer = "stringio" + + def test_import_module(self): + b = "import StringIO" + a = "import io" + self.check(b, a) + + b = "import foo, StringIO, bar" + a = "import foo, io, bar" + self.check(b, a) + + def test_import_from(self): + b = "from StringIO import StringIO" + a = "from io import StringIO" + self.check(b, a) + + s = "from foo import StringIO" + self.check(s, s) + + def test_import_module_as(self): + b = "import StringIO as foo_bar" + a = "import io as foo_bar" + self.check(b, a) + + b = "import StringIO as foo_bar" + a = "import io as foo_bar" + self.check(b, a) + + def test_import_from_as(self): + b = "from StringIO import StringIO as foo_bar" + a = "from io import StringIO as foo_bar" + self.check(b, a) + + def test_import_module_usage(self): + b = """ + import StringIO + foo(StringIO, StringIO.StringIO) + """ + a = """ + import io + foo(io, io.StringIO) + """ + self.check(b, a) + + def test_from_import_usage(self): + b = """ + from StringIO import StringIO + foo(StringIO, StringIO()) + """ + a = """ + from io import StringIO + foo(StringIO, StringIO()) + """ + self.check(b, a) + + class Test_input(FixerTestCase): fixer = "input" From python-checkins at python.org Sat Jul 14 20:23:32 2007 From: python-checkins at python.org (collin.winter) Date: Sat, 14 Jul 2007 20:23:32 +0200 (CEST) Subject: [Python-checkins] r56373 - in sandbox/trunk/2to3: fixes/basefix.py fixes/fix_next.py fixes/fix_stringio.py pytree.py refactor.py tests/test_fixers.py tests/test_pytree.py Message-ID: <20070714182332.97CFA1E400F@bag.python.org> Author: collin.winter Date: Sat Jul 14 20:23:32 2007 New Revision: 56373 Modified: sandbox/trunk/2to3/ (props changed) sandbox/trunk/2to3/fixes/basefix.py sandbox/trunk/2to3/fixes/fix_next.py sandbox/trunk/2to3/fixes/fix_stringio.py sandbox/trunk/2to3/pytree.py sandbox/trunk/2to3/refactor.py sandbox/trunk/2to3/tests/test_fixers.py sandbox/trunk/2to3/tests/test_pytree.py Log: Add the ability for fixers to indicate whether they want a pre- or post-order traversal of the AST; change the StringIO and next fixers to take advantage of this (all the delayed-stringification crap is gone). Modified: sandbox/trunk/2to3/fixes/basefix.py ============================================================================== --- sandbox/trunk/2to3/fixes/basefix.py (original) +++ sandbox/trunk/2to3/fixes/basefix.py Sat Jul 14 20:23:32 2007 @@ -34,6 +34,7 @@ logger = None # A logger (set by set_filename) numbers = itertools.count(1) # For new_name() used_names = set() # A set of all used NAMEs + order = "post" # Does the fixer prefer pre- or post-order traversal # Shortcut for access to Python grammar symbols syms = pygram.python_symbols Modified: sandbox/trunk/2to3/fixes/fix_next.py ============================================================================== --- sandbox/trunk/2to3/fixes/fix_next.py (original) +++ sandbox/trunk/2to3/fixes/fix_next.py Sat Jul 14 20:23:32 2007 @@ -6,7 +6,6 @@ # - "with" statement targets aren't checked # Local imports -import pytree from pgen2 import token from pygram import python_symbols as syms from fixes import basefix @@ -15,44 +14,9 @@ bind_warning = "Calls to builtin next() possibly shadowed by global binding" -class DelayedStrNode(object): - - def __init__(self, type, base): - self.parent = None - self.shadowed_next = False - self.base = base - self.type = type - self.value = "" - self.prefix = "" - - def __str__(self): - if self.shadowed_next: - b = "".join([str(n) for n in self.base]) - return self.prefix + "%s.__next__()" % b - else: - b_prefix = prefix = self.base[0].get_prefix() - self.base[0].set_prefix("") - b = "".join([str(n) for n in self.base]) - self.base[0].set_prefix(b_prefix) - return self.prefix + prefix + "next(%s)" % b - - def clone(self): - node = DelayedStrNode(self.type, self.base) - node.shadowed_next = self.shadowed_next - node.value = self.value - node.prefix = self.prefix - return node - - def set_prefix(self, prefix): - self.prefix = prefix - - def get_prefix(self): - return self.prefix - - class FixNext(basefix.BaseFix): PATTERN = """ - power< base=any+ trailer< '.' 'next' > trailer< '(' ')' > > + power< base=any+ trailer< '.' attr='next' > trailer< '(' ')' > > | power< head=any+ trailer< '.' attr='next' > not trailer< '(' ')' > > | @@ -68,10 +32,11 @@ mod=file_input< any+ > """ + order = "pre" # Pre-order tree traversal + def start_tree(self, tree, filename): super(FixNext, self).start_tree(tree, filename) self.shadowed_next = False - self.delayed = [] def transform(self, node, results): assert results @@ -82,9 +47,12 @@ mod = results.get("mod") if base: - n = DelayedStrNode(syms.power, base) - node.replace(n) - self.delayed.append(n) + if self.shadowed_next: + attr.replace(Name("__next__", prefix=attr.get_prefix())) + else: + base = [n.clone() for n in base] + base[0].set_prefix("") + node.replace(Call(Name("next", prefix=node.get_prefix()), base)) elif name: n = Name("__next__", prefix=name.get_prefix()) name.replace(n) @@ -107,12 +75,6 @@ self.warning(n, bind_warning) self.shadowed_next = True - def finish_tree(self, tree, filename): - super(FixNext, self).finish_tree(tree, filename) - if self.shadowed_next: - for node in self.delayed: - node.shadowed_next = True - ### The following functions help test if node is part of an assignment ### target. Modified: sandbox/trunk/2to3/fixes/fix_stringio.py ============================================================================== --- sandbox/trunk/2to3/fixes/fix_stringio.py (original) +++ sandbox/trunk/2to3/fixes/fix_stringio.py Sat Jul 14 20:23:32 2007 @@ -12,30 +12,10 @@ # Author: Collin Winter # Local imports -import patcomp from fixes import basefix from fixes.util import Name, attr_chain, any -class DelayedStrLeaf(object): - def __init__(self, fixer, leaf): - self.fixer = fixer - self.leaf = leaf - self.parent = None - - def __getattr__(self, attr): - return getattr(self.leaf, attr) - - def __str__(self): - if self.fixer.module_import: - return self.leaf.get_prefix() + "io" - else: - return str(self.leaf) - - def clone(self): - return DelayedStrLeaf(self.fixer, self.leaf) - - class FixStringio(basefix.BaseFix): PATTERN = """ import_name< 'import' (module='StringIO' @@ -51,6 +31,8 @@ bare_name='StringIO' """ + order = "pre" # Pre-order tree traversal + # Don't match 'StringIO' if it's within another match def match(self, node): match = super(FixStringio, self).match @@ -75,7 +57,5 @@ import_mod.replace(Name("io", prefix=import_mod.get_prefix())) elif module_name: module_name.replace(Name("io", prefix=module_name.get_prefix())) - elif bare_name: - bare_name.replace(DelayedStrLeaf(self, bare_name)) - else: - raise RuntimeError("Hmm, shouldn't have gotten here") + elif bare_name and self.module_import: + bare_name.replace(Name("io", prefix=bare_name.get_prefix())) Modified: sandbox/trunk/2to3/pytree.py ============================================================================== --- sandbox/trunk/2to3/pytree.py (original) +++ sandbox/trunk/2to3/pytree.py Sat Jul 14 20:23:32 2007 @@ -79,6 +79,13 @@ """ raise NotImplementedError + def pre_order(self): + """Returns a pre-order iterator for the tree. + + This must be implemented by the concrete subclass. + """ + raise NotImplementedError + def set_prefix(self, prefix): """Sets the prefix for the node (see Leaf class). @@ -216,6 +223,13 @@ yield node yield self + def pre_order(self): + """Returns a pre-order iterator for the tree.""" + yield self + for child in self.children: + for node in child.post_order(): + yield node + def set_prefix(self, prefix): """Sets the prefix for the node. @@ -302,6 +316,10 @@ """Returns a post-order iterator for the tree.""" yield self + def pre_order(self): + """Returns a pre-order iterator for the tree.""" + yield self + def set_prefix(self, prefix): """Sets the prefix for the node.""" self.changed() Modified: sandbox/trunk/2to3/refactor.py ============================================================================== --- sandbox/trunk/2to3/refactor.py (original) +++ sandbox/trunk/2to3/refactor.py Sat Jul 14 20:23:32 2007 @@ -111,12 +111,19 @@ self.driver = driver.Driver(pygram.python_grammar, convert=pytree.convert, logger=self.logger) - self.fixers = self.get_fixers() + self.pre_order, self.post_order = self.get_fixers() self.files = [] # List of files that were or should be modified def get_fixers(self): - """Inspects the options to load the requested patterns and handlers.""" - fixers = [] + """Inspects the options to load the requested patterns and handlers. + + Returns: + (pre_order, post_order), where pre_order is the list of fixers that + want a pre-order AST traversal, and post_order is the list that want + post-order traversal. + """ + pre_order_fixers = [] + post_order_fixers = [] fix_names = self.options.fix if not fix_names or "all" in fix_names: fix_names = get_all_fix_names() @@ -142,8 +149,14 @@ continue if self.options.verbose: self.log_message("Adding transformation: %s", fix_name) - fixers.append(fixer) - return fixers + + if fixer.order == "pre": + pre_order_fixers.append(fixer) + elif fixer.order == "post": + post_order_fixers.append(fixer) + else: + raise ValueError("Illegal fixer order: %r" % fixer.order) + return (pre_order_fixers, post_order_fixers) def log_error(self, msg, *args, **kwds): """Increments error count and log a message.""" @@ -249,11 +262,26 @@ def refactor_tree(self, tree, filename): """Refactors a parse tree (modifying the tree in place).""" - for fixer in self.fixers: + changed = False + all_fixers = self.pre_order + self.post_order + for fixer in all_fixers: fixer.start_tree(tree, filename) - changes = 0 - for node in tree.post_order(): - for fixer in self.fixers: + + changed |= self.traverse_by(self.pre_order, tree.pre_order()) + changed |= self.traverse_by(self.post_order, tree.post_order()) + if tree.was_changed: + changes = True + + for fixer in all_fixers: + fixer.finish_tree(tree, filename) + return changed + + def traverse_by(self, fixers, traversal): + changed = False + if not fixers: + return changed + for node in traversal: + for fixer in fixers: results = fixer.match(node) if results: new = fixer.transform(node, results) @@ -261,12 +289,8 @@ str(new) != str(node)): node.replace(new) node = new - changes += 1 - elif tree.was_changed: - changes += 1 - for fixer in self.fixers: - fixer.finish_tree(tree, filename) - return changes + changed = True + return changed def write_file(self, new_text, filename, old_text=None): """Writes a string to a file. Modified: sandbox/trunk/2to3/tests/test_fixers.py ============================================================================== --- sandbox/trunk/2to3/tests/test_fixers.py (original) +++ sandbox/trunk/2to3/tests/test_fixers.py Sat Jul 14 20:23:32 2007 @@ -47,7 +47,10 @@ self.logging_stream = StringIO() sh = logging.StreamHandler(self.logging_stream) sh.setFormatter(logging.Formatter("%(message)s")) - self.refactor.fixers = [Fixer(f, sh) for f in self.refactor.fixers] + self.refactor.pre_order = [Fixer(f, sh) for f + in self.refactor.pre_order] + self.refactor.post_order = [Fixer(f, sh) for f + in self.refactor.post_order] def tearDown(self): self.logging_stream = None Modified: sandbox/trunk/2to3/tests/test_pytree.py ============================================================================== --- sandbox/trunk/2to3/tests/test_pytree.py (original) +++ sandbox/trunk/2to3/tests/test_pytree.py Sat Jul 14 20:23:32 2007 @@ -150,6 +150,18 @@ # XXX pass + def testPostOrder(self): + l1 = pytree.Leaf(100, "foo") + l2 = pytree.Leaf(100, "bar") + n1 = pytree.Node(1000, [l1, l2]) + self.assertEqual(list(n1.post_order()), [l1, l2, n1]) + + def testPreOrder(self): + l1 = pytree.Leaf(100, "foo") + l2 = pytree.Leaf(100, "bar") + n1 = pytree.Node(1000, [l1, l2]) + self.assertEqual(list(n1.pre_order()), [n1, l1, l2]) + def testChangedLeaf(self): l1 = pytree.Leaf(100, "f") self.failIf(l1.was_changed) From python-checkins at python.org Sat Jul 14 20:23:48 2007 From: python-checkins at python.org (collin.winter) Date: Sat, 14 Jul 2007 20:23:48 +0200 (CEST) Subject: [Python-checkins] r56374 - in sandbox/trunk/2to3: tests/support.py tests/test_all_fixers.py tests/test_parser.py Message-ID: <20070714182348.A19AA1E400C@bag.python.org> Author: collin.winter Date: Sat Jul 14 20:23:48 2007 New Revision: 56374 Modified: sandbox/trunk/2to3/ (props changed) sandbox/trunk/2to3/tests/support.py sandbox/trunk/2to3/tests/test_all_fixers.py sandbox/trunk/2to3/tests/test_parser.py Log: Create a helper function for tests that use the project's files as input data (fixer and parser tests). Modified: sandbox/trunk/2to3/tests/support.py ============================================================================== --- sandbox/trunk/2to3/tests/support.py (original) +++ sandbox/trunk/2to3/tests/support.py Sat Jul 14 20:23:48 2007 @@ -4,6 +4,7 @@ # Python imports import unittest import sys +import os import os.path import re from textwrap import dedent @@ -15,6 +16,7 @@ from pgen2 import driver test_dir = os.path.dirname(__file__) +proj_dir = os.path.normpath(os.path.join(test_dir, "..")) grammar_path = os.path.join(test_dir, "..", "Grammar.txt") grammar = driver.load_grammar(grammar_path) driver = driver.Driver(grammar, convert=pytree.convert) @@ -36,4 +38,10 @@ def reformat(string): return dedent(string) + "\n\n" +def all_project_files(): + for dirpath, dirnames, filenames in os.walk(proj_dir): + for filename in filenames: + if filename.endswith(".py"): + yield os.path.join(dirpath, filename) + TestCase = unittest.TestCase Modified: sandbox/trunk/2to3/tests/test_all_fixers.py ============================================================================== --- sandbox/trunk/2to3/tests/test_all_fixers.py (original) +++ sandbox/trunk/2to3/tests/test_all_fixers.py Sat Jul 14 20:23:48 2007 @@ -39,22 +39,10 @@ self.refactor.refactor_tree(tree, stream_name) return str(tree) - def test_examples_file(self): - # Just test that we can parse examples.py without failing - basedir = os.path.dirname(refactor.__file__) - example = os.path.join(basedir, "example.py") - self.refactor_stream("example.py", open(example)) - - def test_fixers(self): - # Just test that we can parse all the fixers without failing - basedir = os.path.dirname(refactor.__file__) - fixerdir = os.path.join(basedir, "fixes") - for filename in os.listdir(fixerdir): - if not filename.endswith(".py"): - continue - print "Fixing %s..." % filename - fixer = os.path.join(fixerdir, filename) - self.refactor_stream(fixer, open(fixer)) + def test_all_project_files(self): + for filepath in support.all_project_files(): + print "Fixing %s..." % filepath + self.refactor_stream(filepath, open(filepath)) if __name__ == "__main__": Modified: sandbox/trunk/2to3/tests/test_parser.py ============================================================================== --- sandbox/trunk/2to3/tests/test_parser.py (original) +++ sandbox/trunk/2to3/tests/test_parser.py Sat Jul 14 20:23:48 2007 @@ -18,17 +18,12 @@ """A cut-down version of pytree_idempotency.py.""" - def test_2to3_files(self): - proj_dir = os.path.join(test_dir, "..") - - for dirpath, dirnames, filenames in os.walk(proj_dir): - for filename in filenames: - if filename.endswith(".py"): - filepath = os.path.join(dirpath, filename) - print "Parsing %s..." % os.path.normpath(filepath) - tree = driver.parse_file(filepath, debug=True) - if diff(filepath, tree): - self.fail("Idempotency failed: %s" % filename) + def test_all_project_files(self): + for filepath in support.all_project_files(): + print "Parsing %s..." % filepath + tree = driver.parse_file(filepath, debug=True) + if diff(filepath, tree): + self.fail("Idempotency failed: %s" % filepath) def diff(fn, tree): From python-checkins at python.org Sat Jul 14 21:23:08 2007 From: python-checkins at python.org (alexandre.vassalotti) Date: Sat, 14 Jul 2007 21:23:08 +0200 (CEST) Subject: [Python-checkins] r56378 - python/branches/cpy_merge/Modules/_picklemodule.c Message-ID: <20070714192308.25BF31E4008@bag.python.org> Author: alexandre.vassalotti Date: Sat Jul 14 21:23:07 2007 New Revision: 56378 Modified: python/branches/cpy_merge/Modules/_picklemodule.c Log: Remove the no-load functions. Modified: python/branches/cpy_merge/Modules/_picklemodule.c ============================================================================== --- python/branches/cpy_merge/Modules/_picklemodule.c (original) +++ python/branches/cpy_merge/Modules/_picklemodule.c Sat Jul 14 21:23:07 2007 @@ -2382,10 +2382,10 @@ static struct PyMethodDef Pickler_methods[] = { {"dump", (PyCFunction) Pickler_dump, METH_VARARGS, - PyDoc_STR("dump(object) -- " + PyDoc_STR("dump(object) -> None.\n\n" "Write an object in pickle format to the object's pickle stream")}, {"clear_memo", (PyCFunction) Pickle_clear_memo, METH_NOARGS, - PyDoc_STR("clear_memo() -- Clear the picklers memo")}, + PyDoc_STR("clear_memo() -> None. Clear the picklers memo.")}, {NULL, NULL} /* sentinel */ }; @@ -4387,428 +4387,16 @@ } -/* No-load functions to support noload, which is used to - find persistent references. */ - -static int -noload_obj(UnpicklerObject * self) -{ - int i; - - if ((i = marker(self)) < 0) - return -1; - return Pdata_clear(self->stack, i + 1); -} - - -static int -noload_inst(UnpicklerObject * self) -{ - int i; - char *s; - - if ((i = marker(self)) < 0) - return -1; - Pdata_clear(self->stack, i); - if (self->readline_func(self, &s) < 0) - return -1; - if (self->readline_func(self, &s) < 0) - return -1; - PDATA_APPEND(self->stack, Py_None, -1); - return 0; -} - -static int -noload_newobj(UnpicklerObject * self) -{ - PyObject *obj; - - PDATA_POP(self->stack, obj); /* pop argtuple */ - if (obj == NULL) - return -1; - Py_DECREF(obj); - - PDATA_POP(self->stack, obj); /* pop cls */ - if (obj == NULL) - return -1; - Py_DECREF(obj); - - PDATA_APPEND(self->stack, Py_None, -1); - return 0; -} - -static int -noload_global(UnpicklerObject *self) -{ - char *s; - - if (self->readline_func(self, &s) < 0) - return -1; - if (self->readline_func(self, &s) < 0) - return -1; - PDATA_APPEND(self->stack, Py_None, -1); - return 0; -} - -static int -noload_reduce(UnpicklerObject *self) -{ - - if (self->stack->length < 2) - return stackUnderflow(); - Pdata_clear(self->stack, self->stack->length - 2); - PDATA_APPEND(self->stack, Py_None, -1); - return 0; -} - -static int -noload_build(UnpicklerObject *self) -{ - - if (self->stack->length < 1) - return stackUnderflow(); - Pdata_clear(self->stack, self->stack->length - 1); - return 0; -} - -static int -noload_extension(UnpicklerObject *self, int nbytes) -{ - char *codebytes; - - assert(nbytes == 1 || nbytes == 2 || nbytes == 4); - if (self->read_func(self, &codebytes, nbytes) < 0) - return -1; - PDATA_APPEND(self->stack, Py_None, -1); - return 0; -} - - static PyObject * -noload(UnpicklerObject *self) -{ - PyObject *err = 0, *val = 0; - char *s; - - self->num_marks = 0; - Pdata_clear(self->stack, 0); - - while (1) { - if (self->read_func(self, &s, 1) < 0) - break; - - switch (s[0]) { - case NONE: - if (load_none(self) < 0) - break; - continue; - - case BININT: - if (load_binint(self) < 0) - break; - continue; - - case BININT1: - if (load_binint1(self) < 0) - break; - continue; - - case BININT2: - if (load_binint2(self) < 0) - break; - continue; - - case INT: - if (load_int(self) < 0) - break; - continue; - - case LONG: - if (load_long(self) < 0) - break; - continue; - - case LONG1: - if (load_counted_long(self, 1) < 0) - break; - continue; - - case LONG4: - if (load_counted_long(self, 4) < 0) - break; - continue; - - case FLOAT: - if (load_float(self) < 0) - break; - continue; - - case BINFLOAT: - if (load_binfloat(self) < 0) - break; - continue; - - case BINSTRING: - if (load_binstring(self) < 0) - break; - continue; - - case SHORT_BINSTRING: - if (load_short_binstring(self) < 0) - break; - continue; - - case STRING: - if (load_string(self) < 0) - break; - continue; - -#ifdef Py_USING_UNICODE - case UNICODE: - if (load_unicode(self) < 0) - break; - continue; - - case BINUNICODE: - if (load_binunicode(self) < 0) - break; - continue; -#endif - - case EMPTY_TUPLE: - if (load_counted_tuple(self, 0) < 0) - break; - continue; - - case TUPLE1: - if (load_counted_tuple(self, 1) < 0) - break; - continue; - - case TUPLE2: - if (load_counted_tuple(self, 2) < 0) - break; - continue; - - case TUPLE3: - if (load_counted_tuple(self, 3) < 0) - break; - continue; - - case TUPLE: - if (load_tuple(self) < 0) - break; - continue; - - case EMPTY_LIST: - if (load_empty_list(self) < 0) - break; - continue; - - case LIST: - if (load_list(self) < 0) - break; - continue; - - case EMPTY_DICT: - if (load_empty_dict(self) < 0) - break; - continue; - - case DICT: - if (load_dict(self) < 0) - break; - continue; - - case OBJ: - if (noload_obj(self) < 0) - break; - continue; - - case INST: - if (noload_inst(self) < 0) - break; - continue; - - case NEWOBJ: - if (noload_newobj(self) < 0) - break; - continue; - - case GLOBAL: - if (noload_global(self) < 0) - break; - continue; - - case APPEND: - if (load_append(self) < 0) - break; - continue; - - case APPENDS: - if (load_appends(self) < 0) - break; - continue; - - case BUILD: - if (noload_build(self) < 0) - break; - continue; - - case DUP: - if (load_dup(self) < 0) - break; - continue; - - case BINGET: - if (load_binget(self) < 0) - break; - continue; - - case LONG_BINGET: - if (load_long_binget(self) < 0) - break; - continue; - - case GET: - if (load_get(self) < 0) - break; - continue; - - case EXT1: - if (noload_extension(self, 1) < 0) - break; - continue; - - case EXT2: - if (noload_extension(self, 2) < 0) - break; - continue; - - case EXT4: - if (noload_extension(self, 4) < 0) - break; - continue; - - case MARK: - if (load_mark(self) < 0) - break; - continue; - - case BINPUT: - if (load_binput(self) < 0) - break; - continue; - - case LONG_BINPUT: - if (load_long_binput(self) < 0) - break; - continue; - - case PUT: - if (load_put(self) < 0) - break; - continue; - - case POP: - if (load_pop(self) < 0) - break; - continue; - - case POP_MARK: - if (load_pop_mark(self) < 0) - break; - continue; - - case SETITEM: - if (load_setitem(self) < 0) - break; - continue; - - case SETITEMS: - if (load_setitems(self) < 0) - break; - continue; - - case STOP: - break; - - case PERSID: - if (load_persid(self) < 0) - break; - continue; - - case BINPERSID: - if (load_binpersid(self) < 0) - break; - continue; - - case REDUCE: - if (noload_reduce(self) < 0) - break; - continue; - - case PROTO: - if (load_proto(self) < 0) - break; - continue; - - case NEWTRUE: - if (load_bool(self, Py_True) < 0) - break; - continue; - - case NEWFALSE: - if (load_bool(self, Py_False) < 0) - break; - continue; - default: - pickle_ErrFormat(UnpicklingError, - "invalid load key, '%s'.", "c", s[0]); - return NULL; - } - - break; - } - - if ((err = PyErr_Occurred())) { - if (err == PyExc_EOFError) { - PyErr_SetNone(PyExc_EOFError); - } - return NULL; - } - - PDATA_POP(self->stack, val); - return val; -} - - -static PyObject * -Unpickler_load(UnpicklerObject * self, PyObject * unused) +Unpickler_load(UnpicklerObject *self) { return load(self); } -static PyObject * -Unpickler_noload(UnpicklerObject *self, PyObject *unused) -{ - return noload(self); -} - static struct PyMethodDef Unpickler_methods[] = { - {"load", (PyCFunction) Unpickler_load, METH_NOARGS, - PyDoc_STR("load() -- Load a pickle") - }, - {"noload", (PyCFunction) Unpickler_noload, METH_NOARGS, - PyDoc_STR - ("noload() -- not load a pickle, but go through most of the motions\n" - "\n" - "This function can be used to read past a pickle without instantiating\n" - "any objects or importing any modules. It can also be used to find all\n" - "persistent references without instantiating any objects or importing\n" - "any modules.\n") - }, + {"load", (PyCFunction)Unpickler_load, METH_NOARGS, + PyDoc_STR("load() -> None. Load a pickle")}, {NULL, NULL} /* sentinel */ }; From python-checkins at python.org Sat Jul 14 21:37:21 2007 From: python-checkins at python.org (alexandre.vassalotti) Date: Sat, 14 Jul 2007 21:37:21 +0200 (CEST) Subject: [Python-checkins] r56379 - python/branches/cpy_merge/Modules/_bytes_iomodule.c python/branches/cpy_merge/Modules/_string_iomodule.c Message-ID: <20070714193721.7F2781E4008@bag.python.org> Author: alexandre.vassalotti Date: Sat Jul 14 21:37:21 2007 New Revision: 56379 Modified: python/branches/cpy_merge/Modules/_bytes_iomodule.c python/branches/cpy_merge/Modules/_string_iomodule.c Log: Fix a minor reference leak. Modified: python/branches/cpy_merge/Modules/_bytes_iomodule.c ============================================================================== --- python/branches/cpy_merge/Modules/_bytes_iomodule.c (original) +++ python/branches/cpy_merge/Modules/_bytes_iomodule.c Sat Jul 14 21:37:21 2007 @@ -491,8 +491,10 @@ self->buf_size = size + 1; if (size > 0) { - if (write_bytes(self, buf, size) == -1) + if (write_bytes(self, buf, size) == -1) { + Py_DECREF(self); return NULL; + } self->pos = 0; } if (self->buf == NULL) { Modified: python/branches/cpy_merge/Modules/_string_iomodule.c ============================================================================== --- python/branches/cpy_merge/Modules/_string_iomodule.c (original) +++ python/branches/cpy_merge/Modules/_string_iomodule.c Sat Jul 14 21:37:21 2007 @@ -474,8 +474,10 @@ self->buf_size = size + 1; if (size> 0) { - if (write_str(self, buf, size) == -1) + if (write_str(self, buf, size) == -1) { + Py_DECREF(self); return NULL; + } self->pos = 0; } if (self->buf == NULL) { From python-checkins at python.org Sat Jul 14 22:58:21 2007 From: python-checkins at python.org (andrew.kuchling) Date: Sat, 14 Jul 2007 22:58:21 +0200 (CEST) Subject: [Python-checkins] r56380 - python/trunk/Doc/lib/email.tex Message-ID: <20070714205821.C29AA1E4008@bag.python.org> Author: andrew.kuchling Date: Sat Jul 14 22:58:21 2007 New Revision: 56380 Modified: python/trunk/Doc/lib/email.tex Log: Typo fix Modified: python/trunk/Doc/lib/email.tex ============================================================================== --- python/trunk/Doc/lib/email.tex (original) +++ python/trunk/Doc/lib/email.tex Sat Jul 14 22:58:21 2007 @@ -94,7 +94,7 @@ This table describes the release history of the email package, corresponding to the version of Python that the package was released with. For purposes of this document, when you see a note about change or added versions, these refer -to the Python version the change was made it, \emph{not} the email package +to the Python version the change was made in, \emph{not} the email package version. This table also describes the Python compatibility of each version of the package. From python-checkins at python.org Sat Jul 14 22:58:24 2007 From: python-checkins at python.org (andrew.kuchling) Date: Sat, 14 Jul 2007 22:58:24 +0200 (CEST) Subject: [Python-checkins] r56381 - python/branches/release25-maint/Doc/lib/email.tex Message-ID: <20070714205824.162731E4008@bag.python.org> Author: andrew.kuchling Date: Sat Jul 14 22:58:23 2007 New Revision: 56381 Modified: python/branches/release25-maint/Doc/lib/email.tex Log: Typo fix Modified: python/branches/release25-maint/Doc/lib/email.tex ============================================================================== --- python/branches/release25-maint/Doc/lib/email.tex (original) +++ python/branches/release25-maint/Doc/lib/email.tex Sat Jul 14 22:58:23 2007 @@ -94,7 +94,7 @@ This table describes the release history of the email package, corresponding to the version of Python that the package was released with. For purposes of this document, when you see a note about change or added versions, these refer -to the Python version the change was made it, \emph{not} the email package +to the Python version the change was made in, \emph{not} the email package version. This table also describes the Python compatibility of each version of the package. From python-checkins at python.org Sat Jul 14 23:56:19 2007 From: python-checkins at python.org (andrew.kuchling) Date: Sat, 14 Jul 2007 23:56:19 +0200 (CEST) Subject: [Python-checkins] r56382 - in python/trunk/Lib: mailbox.py test/test_mailbox.py Message-ID: <20070714215619.773881E4008@bag.python.org> Author: andrew.kuchling Date: Sat Jul 14 23:56:19 2007 New Revision: 56382 Modified: python/trunk/Lib/mailbox.py python/trunk/Lib/test/test_mailbox.py Log: Avoid exception if there's a stray directory inside a Maildir folder. The Maildir specification doesn't seem to say anything about this situation, and it can happen if you're keeping a Maildir mailbox in Subversion (.svn directories) or some similar system. The patch just ignores directories in the cur/, new/, tmp/ folders. Modified: python/trunk/Lib/mailbox.py ============================================================================== --- python/trunk/Lib/mailbox.py (original) +++ python/trunk/Lib/mailbox.py Sat Jul 14 23:56:19 2007 @@ -459,7 +459,11 @@ """Update table of contents mapping.""" self._toc = {} for subdir in ('new', 'cur'): - for entry in os.listdir(os.path.join(self._path, subdir)): + subdir_path = os.path.join(self._path, subdir) + for entry in os.listdir(subdir_path): + p = os.path.join(subdir_path, entry) + if os.path.isdir(p): + continue uniq = entry.split(self.colon)[0] self._toc[uniq] = os.path.join(subdir, entry) Modified: python/trunk/Lib/test/test_mailbox.py ============================================================================== --- python/trunk/Lib/test/test_mailbox.py (original) +++ python/trunk/Lib/test/test_mailbox.py Sat Jul 14 23:56:19 2007 @@ -686,7 +686,18 @@ folder1_alias = box.get_folder('folder1') self.assert_(folder1_alias._factory is dummy_factory) - + def test_directory_in_folder (self): + # Test that mailboxes still work if there's a stray extra directory + # in a folder. + for i in range(10): + self._box.add(mailbox.Message(_sample_message)) + + # Create a stray directory + os.mkdir(os.path.join(self._path, 'cur', 'stray-dir')) + + # Check that looping still works with the directory present. + for msg in self._box: + pass class _TestMboxMMDF(TestMailbox): From python-checkins at python.org Sun Jul 15 00:06:32 2007 From: python-checkins at python.org (andrew.kuchling) Date: Sun, 15 Jul 2007 00:06:32 +0200 (CEST) Subject: [Python-checkins] r56383 - in python/branches/release25-maint: Lib/mailbox.py Lib/test/test_mailbox.py Misc/NEWS Message-ID: <20070714220632.131081E4008@bag.python.org> Author: andrew.kuchling Date: Sun Jul 15 00:06:30 2007 New Revision: 56383 Modified: python/branches/release25-maint/Lib/mailbox.py python/branches/release25-maint/Lib/test/test_mailbox.py python/branches/release25-maint/Misc/NEWS Log: [Backport of r56382] Avoid exception if there's a stray directory inside a Maildir folder. The Maildir specification doesn't seem to say anything about this situation, and it can happen if you're keeping a Maildir mailbox in Subversion (.svn directories) or some similar system. The patch just ignores directories in the cur/, new/, tmp/ folders. Modified: python/branches/release25-maint/Lib/mailbox.py ============================================================================== --- python/branches/release25-maint/Lib/mailbox.py (original) +++ python/branches/release25-maint/Lib/mailbox.py Sun Jul 15 00:06:30 2007 @@ -459,7 +459,11 @@ """Update table of contents mapping.""" self._toc = {} for subdir in ('new', 'cur'): - for entry in os.listdir(os.path.join(self._path, subdir)): + subdir_path = os.path.join(self._path, subdir) + for entry in os.listdir(subdir_path): + p = os.path.join(subdir_path, entry) + if os.path.isdir(p): + continue uniq = entry.split(self.colon)[0] self._toc[uniq] = os.path.join(subdir, entry) Modified: python/branches/release25-maint/Lib/test/test_mailbox.py ============================================================================== --- python/branches/release25-maint/Lib/test/test_mailbox.py (original) +++ python/branches/release25-maint/Lib/test/test_mailbox.py Sun Jul 15 00:06:30 2007 @@ -685,7 +685,18 @@ folder1_alias = box.get_folder('folder1') self.assert_(folder1_alias._factory is dummy_factory) - + def test_directory_in_folder (self): + # Test that mailboxes still work if there's a stray extra directory + # in a folder. + for i in range(10): + self._box.add(mailbox.Message(_sample_message)) + + # Create a stray directory + os.mkdir(os.path.join(self._path, 'cur', 'stray-dir')) + + # Check that looping still works with the directory present. + for msg in self._box: + pass class _TestMboxMMDF(TestMailbox): Modified: python/branches/release25-maint/Misc/NEWS ============================================================================== --- python/branches/release25-maint/Misc/NEWS (original) +++ python/branches/release25-maint/Misc/NEWS Sun Jul 15 00:06:30 2007 @@ -45,6 +45,9 @@ - Fix bug in marshal where bad data would cause a segfault due to lack of an infinite recursion check. +- mailbox.py: Ignore stray directories found in Maildir's cur/new/tmp + subdirectories. + - HTML-escape the plain traceback in cgitb's HTML output, to prevent the traceback inadvertently or maliciously closing the comment and injecting HTML into the error page. From python-checkins at python.org Sun Jul 15 00:33:12 2007 From: python-checkins at python.org (collin.winter) Date: Sun, 15 Jul 2007 00:33:12 +0200 (CEST) Subject: [Python-checkins] r56384 - in sandbox/trunk/2to3: fixes/fix_stringio.py tests/test_fixers.py Message-ID: <20070714223312.185B91E4008@bag.python.org> Author: collin.winter Date: Sun Jul 15 00:33:11 2007 New Revision: 56384 Modified: sandbox/trunk/2to3/ (props changed) sandbox/trunk/2to3/fixes/fix_stringio.py sandbox/trunk/2to3/tests/test_fixers.py Log: Add support for 'from StringIO import *' to StringIO fixer. Modified: sandbox/trunk/2to3/fixes/fix_stringio.py ============================================================================== --- sandbox/trunk/2to3/fixes/fix_stringio.py (original) +++ sandbox/trunk/2to3/fixes/fix_stringio.py Sun Jul 15 00:33:11 2007 @@ -24,6 +24,8 @@ import_from< 'from' module_name='StringIO' 'import' ( 'StringIO' | import_as_name< 'StringIO' 'as' any >) > | + import_from< 'from' module_name='StringIO' 'import' star='*' > + | import_name< 'import' dotted_as_name< module_name='StringIO' 'as' any > > | power< module_name='StringIO' trailer< '.' 'StringIO' > any* > @@ -51,11 +53,14 @@ import_mod = results.get("module") module_name = results.get("module_name") bare_name = results.get("bare_name") + star = results.get("star") if import_mod: self.module_import = True import_mod.replace(Name("io", prefix=import_mod.get_prefix())) elif module_name: module_name.replace(Name("io", prefix=module_name.get_prefix())) + if star: + star.replace(Name("StringIO", prefix=star.get_prefix())) elif bare_name and self.module_import: bare_name.replace(Name("io", prefix=bare_name.get_prefix())) Modified: sandbox/trunk/2to3/tests/test_fixers.py ============================================================================== --- sandbox/trunk/2to3/tests/test_fixers.py (original) +++ sandbox/trunk/2to3/tests/test_fixers.py Sun Jul 15 00:33:11 2007 @@ -1197,6 +1197,10 @@ a = "from io import StringIO" self.check(b, a) + b = "from StringIO import *" + a = "from io import StringIO" + self.check(b, a) + s = "from foo import StringIO" self.check(s, s) From python-checkins at python.org Sun Jul 15 00:33:31 2007 From: python-checkins at python.org (collin.winter) Date: Sun, 15 Jul 2007 00:33:31 +0200 (CEST) Subject: [Python-checkins] r56385 - in sandbox/trunk/2to3: tests/test_grammar.py tests/test_parser.py Message-ID: <20070714223331.B86EE1E400C@bag.python.org> Author: collin.winter Date: Sun Jul 15 00:33:29 2007 New Revision: 56385 Removed: sandbox/trunk/2to3/tests/test_grammar.py Modified: sandbox/trunk/2to3/ (props changed) sandbox/trunk/2to3/tests/test_parser.py Log: Merge test_grammar into test_parser. Deleted: /sandbox/trunk/2to3/tests/test_grammar.py ============================================================================== --- /sandbox/trunk/2to3/tests/test_grammar.py Sun Jul 15 00:33:29 2007 +++ (empty file) @@ -1,156 +0,0 @@ -#!/usr/bin/env python2.5 -""" Test suite for Grammar.txt. This is the place to add tests for -changes to 2to3's grammar, such as those merging the grammars for -Python 2 and 3. - -In addition to specific tests for parts of the grammar we've changed, -TestGrammarFiles also attempts to process the test_grammar.py files -from Python 2 and Python 3. """ -# Author: Collin Winter - -# Testing imports -import support -from support import driver, test_dir - -# Python imports -import os.path - -# Local imports -from pgen2.parse import ParseError - -class GrammarTest(support.TestCase): - def validate(self, code): - support.parse_string(code) - - def invalid_syntax(self, code): - try: - self.validate(code) - except ParseError: - pass - else: - raise AssertionError("Syntax shouldn't have been valid") - - -class TestRaiseChanges(GrammarTest): - def test_2x_style_1(self): - self.validate("raise") - - def test_2x_style_2(self): - self.validate("raise E, V") - - def test_2x_style_3(self): - self.validate("raise E, V, T") - - def test_2x_style_invalid_1(self): - self.invalid_syntax("raise E, V, T, Z") - - def test_3x_style(self): - self.validate("raise E1 from E2") - - def test_3x_style_invalid_1(self): - self.invalid_syntax("raise E, V from E1") - - def test_3x_style_invalid_2(self): - self.invalid_syntax("raise E from E1, E2") - - def test_3x_style_invalid_3(self): - self.invalid_syntax("raise from E1, E2") - - def test_3x_style_invalid_4(self): - self.invalid_syntax("raise E from") - - -# Adapated from Python 3's Lib/test/test_grammar.py:GrammarTests.testFuncdef -class TestFunctionAnnotations(GrammarTest): - def test_1(self): - self.validate("""def f(x) -> list: pass""") - - def test_2(self): - self.validate("""def f(x:int): pass""") - - def test_3(self): - self.validate("""def f(*x:str): pass""") - - def test_4(self): - self.validate("""def f(**x:float): pass""") - - def test_5(self): - self.validate("""def f(x, y:1+2): pass""") - - def test_6(self): - self.validate("""def f(a, (b:1, c:2, d)): pass""") - - def test_7(self): - self.validate("""def f(a, (b:1, c:2, d), e:3=4, f=5, *g:6): pass""") - - def test_8(self): - s = """def f(a, (b:1, c:2, d), e:3=4, f=5, - *g:6, h:7, i=8, j:9=10, **k:11) -> 12: pass""" - self.validate(s) - - -class TestExcept(GrammarTest): - def test_new(self): - s = """ - try: - x - except E as N: - y""" - self.validate(s) - - def test_old(self): - s = """ - try: - x - except E, N: - y""" - self.validate(s) - - -# Adapted from Python 3's Lib/test/test_grammar.py:GrammarTests.testAtoms -class TestSetLiteral(GrammarTest): - def test_1(self): - self.validate("""x = {'one'}""") - - def test_2(self): - self.validate("""x = {'one', 1,}""") - - def test_3(self): - self.validate("""x = {'one', 'two', 'three'}""") - - def test_4(self): - self.validate("""x = {2, 3, 4,}""") - - -class TestNumericLiterals(GrammarTest): - def test_new_octal_notation(self): - self.validate("""0o7777777777777""") - self.invalid_syntax("""0o7324528887""") - - def test_new_binary_notation(self): - self.validate("""0b101010""") - self.invalid_syntax("""0b0101021""") - - -class TestClassDef(GrammarTest): - def test_new_syntax(self): - self.validate("class B(t=7): pass") - self.validate("class B(t, *args): pass") - self.validate("class B(t, **kwargs): pass") - self.validate("class B(t, *args, **kwargs): pass") - self.validate("class B(t, y=9, *args, **kwargs): pass") - - -class TestGrammarFiles(GrammarTest): - def test_python2(self): - f = os.path.join(test_dir, "data", "py2_test_grammar.py") - driver.parse_file(f) - - def test_python3(self): - f = os.path.join(test_dir, "data", "py3_test_grammar.py") - driver.parse_file(f) - - -if __name__ == "__main__": - import __main__ - support.run_all_tests(__main__) Modified: sandbox/trunk/2to3/tests/test_parser.py ============================================================================== --- sandbox/trunk/2to3/tests/test_parser.py (original) +++ sandbox/trunk/2to3/tests/test_parser.py Sun Jul 15 00:33:29 2007 @@ -1,5 +1,11 @@ #!/usr/bin/env python2.5 -"""Test suite for 2to3's parser.""" +"""Test suite for 2to3's parser and grammar files. + +This is the place to add tests for changes to 2to3's grammar, such as those +merging the grammars for Python 2 and 3. In addition to specific tests for +parts of the grammar we've changed, we also make sure we can parse the +test_grammar.py files from both Python 2 and Python 3. +""" # Author: Collin Winter # Testing imports @@ -14,6 +20,129 @@ from pgen2.parse import ParseError +class GrammarTest(support.TestCase): + def validate(self, code): + support.parse_string(code) + + def invalid_syntax(self, code): + try: + self.validate(code) + except ParseError: + pass + else: + raise AssertionError("Syntax shouldn't have been valid") + + +class TestRaiseChanges(GrammarTest): + def test_2x_style_1(self): + self.validate("raise") + + def test_2x_style_2(self): + self.validate("raise E, V") + + def test_2x_style_3(self): + self.validate("raise E, V, T") + + def test_2x_style_invalid_1(self): + self.invalid_syntax("raise E, V, T, Z") + + def test_3x_style(self): + self.validate("raise E1 from E2") + + def test_3x_style_invalid_1(self): + self.invalid_syntax("raise E, V from E1") + + def test_3x_style_invalid_2(self): + self.invalid_syntax("raise E from E1, E2") + + def test_3x_style_invalid_3(self): + self.invalid_syntax("raise from E1, E2") + + def test_3x_style_invalid_4(self): + self.invalid_syntax("raise E from") + + +# Adapated from Python 3's Lib/test/test_grammar.py:GrammarTests.testFuncdef +class TestFunctionAnnotations(GrammarTest): + def test_1(self): + self.validate("""def f(x) -> list: pass""") + + def test_2(self): + self.validate("""def f(x:int): pass""") + + def test_3(self): + self.validate("""def f(*x:str): pass""") + + def test_4(self): + self.validate("""def f(**x:float): pass""") + + def test_5(self): + self.validate("""def f(x, y:1+2): pass""") + + def test_6(self): + self.validate("""def f(a, (b:1, c:2, d)): pass""") + + def test_7(self): + self.validate("""def f(a, (b:1, c:2, d), e:3=4, f=5, *g:6): pass""") + + def test_8(self): + s = """def f(a, (b:1, c:2, d), e:3=4, f=5, + *g:6, h:7, i=8, j:9=10, **k:11) -> 12: pass""" + self.validate(s) + + +class TestExcept(GrammarTest): + def test_new(self): + s = """ + try: + x + except E as N: + y""" + self.validate(s) + + def test_old(self): + s = """ + try: + x + except E, N: + y""" + self.validate(s) + + +# Adapted from Python 3's Lib/test/test_grammar.py:GrammarTests.testAtoms +class TestSetLiteral(GrammarTest): + def test_1(self): + self.validate("""x = {'one'}""") + + def test_2(self): + self.validate("""x = {'one', 1,}""") + + def test_3(self): + self.validate("""x = {'one', 'two', 'three'}""") + + def test_4(self): + self.validate("""x = {2, 3, 4,}""") + + +class TestNumericLiterals(GrammarTest): + def test_new_octal_notation(self): + self.validate("""0o7777777777777""") + self.invalid_syntax("""0o7324528887""") + + def test_new_binary_notation(self): + self.validate("""0b101010""") + self.invalid_syntax("""0b0101021""") + + +class TestClassDef(GrammarTest): + def test_new_syntax(self): + self.validate("class B(t=7): pass") + self.validate("class B(t, *args): pass") + self.validate("class B(t, **kwargs): pass") + self.validate("class B(t, *args, **kwargs): pass") + self.validate("class B(t, y=9, *args, **kwargs): pass") + + class TestParserIdempotency(support.TestCase): """A cut-down version of pytree_idempotency.py.""" From buildbot at python.org Sun Jul 15 00:33:36 2007 From: buildbot at python.org (buildbot at python.org) Date: Sat, 14 Jul 2007 22:33:36 +0000 Subject: [Python-checkins] buildbot warnings in x86 XP-3 trunk Message-ID: <20070714223337.1E4181E4008@bag.python.org> The Buildbot has detected a new failure of x86 XP-3 trunk. Full details are available at: http://www.python.org/dev/buildbot/all/x86%2520XP-3%2520trunk/builds/81 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: andrew.kuchling,georg.brandl Build had warnings: warnings test Excerpt from the test logfile: 1 test failed: test_timeout sincerely, -The Buildbot From python-checkins at python.org Sun Jul 15 00:33:47 2007 From: python-checkins at python.org (collin.winter) Date: Sun, 15 Jul 2007 00:33:47 +0200 (CEST) Subject: [Python-checkins] r56386 - in sandbox/trunk/2to3: Grammar.txt fixes/fix_dict.py fixes/fix_filter.py fixes/fix_map.py fixes/util.py tests/data/py3_test_grammar.py Message-ID: <20070714223347.8EC361E4008@bag.python.org> Author: collin.winter Date: Sun Jul 15 00:33:47 2007 New Revision: 56386 Modified: sandbox/trunk/2to3/ (props changed) sandbox/trunk/2to3/Grammar.txt sandbox/trunk/2to3/fixes/fix_dict.py sandbox/trunk/2to3/fixes/fix_filter.py sandbox/trunk/2to3/fixes/fix_map.py sandbox/trunk/2to3/fixes/util.py sandbox/trunk/2to3/tests/data/py3_test_grammar.py Log: Sync grammar with p3yk branch, update some fixers accordingly. Modified: sandbox/trunk/2to3/Grammar.txt ============================================================================== --- sandbox/trunk/2to3/Grammar.txt (original) +++ sandbox/trunk/2to3/Grammar.txt Sun Jul 15 00:33:47 2007 @@ -124,8 +124,8 @@ '{' [dictsetmaker] '}' | '`' testlist1 '`' | NAME | NUMBER | STRING+ | '.' '.' '.') -listmaker: test ( list_for | (',' test)* [','] ) -testlist_gexp: test ( gen_for | (',' test)* [','] ) +listmaker: test ( comp_for | (',' test)* [','] ) +testlist_gexp: test ( comp_for | (',' test)* [','] ) lambdef: 'lambda' [varargslist] ':' test trailer: '(' [arglist] ')' | '[' subscriptlist ']' | '.' NAME subscriptlist: subscript (',' subscript)* [','] @@ -133,20 +133,17 @@ sliceop: ':' [test] exprlist: expr (',' expr)* [','] testlist: test (',' test)* [','] -dictsetmaker: (test ':' test (',' test ':' test)* [',']) | (test (',' test)* [',']) +dictsetmaker: ( (test ':' test (comp_for | (',' test ':' test)* [','])) | + (test (comp_for | (',' test)* [','])) ) classdef: 'class' NAME ['(' [arglist] ')'] ':' suite arglist: (argument ',')* (argument [',']| '*' test [',' '**' test] | '**' test) -argument: test [gen_for] | test '=' test # Really [keyword '='] test +argument: test [comp_for] | test '=' test # Really [keyword '='] test -list_iter: list_for | list_if -list_for: 'for' exprlist 'in' testlist_safe [list_iter] -list_if: 'if' old_test [list_iter] - -gen_iter: gen_for | gen_if -gen_for: 'for' exprlist 'in' or_test [gen_iter] -gen_if: 'if' old_test [gen_iter] +comp_iter: comp_for | comp_if +comp_for: 'for' exprlist 'in' testlist_safe [comp_iter] +comp_if: 'if' old_test [comp_iter] testlist1: test (',' test)* Modified: sandbox/trunk/2to3/fixes/fix_dict.py ============================================================================== --- sandbox/trunk/2to3/fixes/fix_dict.py (original) +++ sandbox/trunk/2to3/fixes/fix_dict.py Sun Jul 15 00:33:47 2007 @@ -70,8 +70,7 @@ p1 = patcomp.compile_pattern(P1) P2 = """for_stmt< 'for' any 'in' node=any ':' any* > - | list_for< 'for' any 'in' node=any any* > - | gen_for< 'for' any 'in' node=any any* > + | comp_for< 'for' any 'in' node=any any* > """ p2 = patcomp.compile_pattern(P2) Modified: sandbox/trunk/2to3/fixes/fix_filter.py ============================================================================== --- sandbox/trunk/2to3/fixes/fix_filter.py (original) +++ sandbox/trunk/2to3/fixes/fix_filter.py Sun Jul 15 00:33:47 2007 @@ -58,8 +58,7 @@ return new P0 = """for_stmt< 'for' any 'in' node=any ':' any* > - | list_for< 'for' any 'in' node=any any* > - | gen_for< 'for' any 'in' node=any any* > + | comp_for< 'for' any 'in' node=any any* > """ p0 = patcomp.compile_pattern(P0) Modified: sandbox/trunk/2to3/fixes/fix_map.py ============================================================================== --- sandbox/trunk/2to3/fixes/fix_map.py (original) +++ sandbox/trunk/2to3/fixes/fix_map.py Sun Jul 15 00:33:47 2007 @@ -70,8 +70,7 @@ return new P0 = """for_stmt< 'for' any 'in' node=any ':' any* > - | list_for< 'for' any 'in' node=any any* > - | gen_for< 'for' any 'in' node=any any* > + | comp_for< 'for' any 'in' node=any any* > """ p0 = patcomp.compile_pattern(P0) Modified: sandbox/trunk/2to3/fixes/util.py ============================================================================== --- sandbox/trunk/2to3/fixes/util.py (original) +++ sandbox/trunk/2to3/fixes/util.py Sun Jul 15 00:33:47 2007 @@ -96,8 +96,8 @@ test.set_prefix(" ") if_leaf = Leaf(token.NAME, "if") if_leaf.set_prefix(" ") - inner_args.append(Node(syms.list_if, [if_leaf, test])) - inner = Node(syms.listmaker, [xp, Node(syms.list_for, inner_args)]) + inner_args.append(Node(syms.comp_if, [if_leaf, test])) + inner = Node(syms.listmaker, [xp, Node(syms.comp_for, inner_args)]) return Node(syms.atom, [Leaf(token.LBRACE, "["), inner, Modified: sandbox/trunk/2to3/tests/data/py3_test_grammar.py ============================================================================== --- sandbox/trunk/2to3/tests/data/py3_test_grammar.py (original) +++ sandbox/trunk/2to3/tests/data/py3_test_grammar.py Sun Jul 15 00:33:47 2007 @@ -1,5 +1,3 @@ -# Python 3's Lib/test/test_grammar.py (r53781) - # Python test set -- part 1, grammar. # This just tests whether the parser accepts them all. @@ -16,7 +14,6 @@ # testing import * from sys import * - at class_decorator class TokenTests(unittest.TestCase): def testBackslash(self): @@ -30,26 +27,32 @@ self.assertEquals(x, 0, 'backslash ending comment') def testPlainIntegers(self): + self.assertEquals(type(000), type(0)) self.assertEquals(0xff, 255) - self.assertEquals(0377, 255) - self.assertEquals(2147483647, 017777777777) + self.assertEquals(0o377, 255) + self.assertEquals(2147483647, 0o17777777777) + self.assertEquals(0b1001, 9) from sys import maxint if maxint == 2147483647: - self.assertEquals(-2147483647-1, -020000000000) + self.assertEquals(-2147483647-1, -0o20000000000) # XXX -2147483648 - self.assert_(037777777777 > 0) + self.assert_(0o37777777777 > 0) self.assert_(0xffffffff > 0) - for s in '2147483648', '040000000000', '0x100000000': + self.assert_(0b1111111111111111111111111111111 > 0) + for s in ('2147483648', '0o40000000000', '0x100000000', + '0b10000000000000000000000000000000'): try: x = eval(s) except OverflowError: self.fail("OverflowError on huge integer literal %r" % s) elif maxint == 9223372036854775807: - self.assertEquals(-9223372036854775807-1, -01000000000000000000000) - self.assert_(01777777777777777777777 > 0) + self.assertEquals(-9223372036854775807-1, -0o1000000000000000000000) + self.assert_(0o1777777777777777777777 > 0) self.assert_(0xffffffffffffffff > 0) - for s in '9223372036854775808', '02000000000000000000000', \ - '0x10000000000000000': + self.assert_(0b11111111111111111111111111111111111111111111111111111111111111 > 0) + for s in '9223372036854775808', '0o2000000000000000000000', \ + '0x10000000000000000', \ + '0b100000000000000000000000000000000000000000000000000000000000000': try: x = eval(s) except OverflowError: @@ -59,13 +62,13 @@ def testLongIntegers(self): x = 0 - x = 0 - x = 0xffffffffffffffff x = 0xffffffffffffffff - x = 077777777777777777 - x = 077777777777777777 - x = 123456789012345678901234567890 + x = 0Xffffffffffffffff + x = 0o77777777777777777 + x = 0O77777777777777777 x = 123456789012345678901234567890 + x = 0b100000000000000000000000000000000000000000000000000000000000000000000 + x = 0B111111111111111111111111111111111111111111111111111111111111111111111 def testFloats(self): x = 3.14 @@ -124,6 +127,7 @@ def testEllipsis(self): x = ... self.assert_(x is Ellipsis) + self.assertRaises(SyntaxError, eval, ".. .") class GrammarTests(unittest.TestCase): @@ -146,51 +150,32 @@ ### decorators: decorator+ ### parameters: '(' [typedargslist] ')' ### typedargslist: ((tfpdef ['=' test] ',')* - ### ('*' [tname] (',' tname ['=' test])* [',' '**' tname] | '**' tname) + ### ('*' [tfpdef] (',' tfpdef ['=' test])* [',' '**' tfpdef] | '**' tfpdef) ### | tfpdef ['=' test] (',' tfpdef ['=' test])* [',']) - ### tname: NAME [':' test] - ### tfpdef: tname | '(' tfplist ')' - ### tfplist: tfpdef (',' tfpdef)* [','] + ### tfpdef: NAME [':' test] ### varargslist: ((vfpdef ['=' test] ',')* - ### ('*' [vname] (',' vname ['=' test])* [',' '**' vname] | '**' vname) + ### ('*' [vfpdef] (',' vfpdef ['=' test])* [',' '**' vfpdef] | '**' vfpdef) ### | vfpdef ['=' test] (',' vfpdef ['=' test])* [',']) - ### vname: NAME - ### vfpdef: vname | '(' vfplist ')' - ### vfplist: vfpdef (',' vfpdef)* [','] + ### vfpdef: NAME def f1(): pass f1() f1(*()) f1(*(), **{}) def f2(one_argument): pass def f3(two, arguments): pass - def f4(two, (compound, (argument, list))): pass - def f5((compound, first), two): pass - self.assertEquals(f2.func_code.co_varnames, ('one_argument',)) - self.assertEquals(f3.func_code.co_varnames, ('two', 'arguments')) - if sys.platform.startswith('java'): - self.assertEquals(f4.func_code.co_varnames, - ('two', '(compound, (argument, list))', 'compound', 'argument', - 'list',)) - self.assertEquals(f5.func_code.co_varnames, - ('(compound, first)', 'two', 'compound', 'first')) - else: - self.assertEquals(f4.func_code.co_varnames, - ('two', '.1', 'compound', 'argument', 'list')) - self.assertEquals(f5.func_code.co_varnames, - ('.0', 'two', 'compound', 'first')) + self.assertEquals(f2.__code__.co_varnames, ('one_argument',)) + self.assertEquals(f3.__code__.co_varnames, ('two', 'arguments')) def a1(one_arg,): pass def a2(two, args,): pass def v0(*rest): pass def v1(a, *rest): pass def v2(a, b, *rest): pass - def v3(a, (b, c), *rest): return a, b, c, rest f1() f2(1) f2(1,) f3(1, 2) f3(1, 2,) - f4(1, (2, (3, 4))) v0() v0(1) v0(1,) @@ -205,17 +190,7 @@ v2(1,2,3) v2(1,2,3,4) v2(1,2,3,4,5,6,7,8,9,0) - v3(1,(2,3)) - v3(1,(2,3),4) - v3(1,(2,3),4,5,6,7,8,9,0) - - # ceval unpacks the formal arguments into the first argcount names; - # thus, the names nested inside tuples must appear after these names. - if sys.platform.startswith('java'): - self.assertEquals(v3.func_code.co_varnames, ('a', '(b, c)', 'rest', 'b', 'c')) - else: - self.assertEquals(v3.func_code.co_varnames, ('a', '.1', 'rest', 'b', 'c')) - self.assertEquals(v3(1, (2, 3), 4), (1, 2, 3, (4,))) + def d01(a=1): pass d01() d01(1) @@ -288,10 +263,6 @@ d22v(*(1, 2, 3, 4)) d22v(1, 2, *(3, 4, 5)) d22v(1, *(2, 3), **{'d': 4}) - def d31v((x)): pass - d31v(1) - def d32v((x,)): pass - d32v((1,)) # keyword only argument tests def pos0key1(*, key): return key pos0key1(key=100) @@ -305,25 +276,37 @@ # argument annotation tests def f(x) -> list: pass - self.assertEquals(f.func_annotations, {'return': list}) + self.assertEquals(f.__annotations__, {'return': list}) def f(x:int): pass - self.assertEquals(f.func_annotations, {'x': int}) + self.assertEquals(f.__annotations__, {'x': int}) def f(*x:str): pass - self.assertEquals(f.func_annotations, {'x': str}) + self.assertEquals(f.__annotations__, {'x': str}) def f(**x:float): pass - self.assertEquals(f.func_annotations, {'x': float}) + self.assertEquals(f.__annotations__, {'x': float}) def f(x, y:1+2): pass - self.assertEquals(f.func_annotations, {'y': 3}) - def f(a, (b:1, c:2, d)): pass - self.assertEquals(f.func_annotations, {'b': 1, 'c': 2}) - def f(a, (b:1, c:2, d), e:3=4, f=5, *g:6): pass - self.assertEquals(f.func_annotations, + self.assertEquals(f.__annotations__, {'y': 3}) + def f(a, b:1, c:2, d): pass + self.assertEquals(f.__annotations__, {'b': 1, 'c': 2}) + def f(a, b:1, c:2, d, e:3=4, f=5, *g:6): pass + self.assertEquals(f.__annotations__, {'b': 1, 'c': 2, 'e': 3, 'g': 6}) - def f(a, (b:1, c:2, d), e:3=4, f=5, *g:6, h:7, i=8, j:9=10, + def f(a, b:1, c:2, d, e:3=4, f=5, *g:6, h:7, i=8, j:9=10, **k:11) -> 12: pass - self.assertEquals(f.func_annotations, + self.assertEquals(f.__annotations__, {'b': 1, 'c': 2, 'e': 3, 'g': 6, 'h': 7, 'j': 9, 'k': 11, 'return': 12}) + # Check for SF Bug #1697248 - mixing decorators and a return annotation + def null(x): return x + @null + def f(x) -> list: pass + self.assertEquals(f.__annotations__, {'return': list}) + + # test MAKE_CLOSURE with a variety of oparg's + closure = 1 + def f(): return closure + def f(x=1): return closure + def f(*, k=1): return closure + def f() -> int: return closure def testLambdef(self): ### lambdef: 'lambda' [varargslist] ':' test @@ -354,7 +337,7 @@ x = 1; pass; del x; foo() - ### small_stmt: expr_stmt | pass_stmt | del_stmt | flow_stmt | import_stmt | global_stmt | access_stmt + ### small_stmt: expr_stmt | pass_stmt | del_stmt | flow_stmt | import_stmt | global_stmt | access_stmt # Tested below def testExprStmt(self): @@ -714,6 +697,20 @@ def meth2(self, arg): pass def meth3(self, a1, a2): pass + # decorator: '@' dotted_name [ '(' [arglist] ')' ] NEWLINE + # decorators: decorator+ + # decorated: decorators (classdef | funcdef) + def class_decorator(x): return x + @class_decorator + class G: pass + + def testDictcomps(self): + # dictorsetmaker: ( (test ':' test (comp_for | + # (',' test ':' test)* [','])) | + # (test (comp_for | (',' test)* [','])) ) + nums = [1, 2, 3] + self.assertEqual({i:i+1 for i in nums}, {1: 2, 2: 3, 3: 4}) + def testListcomps(self): # list comprehension tests nums = [1, 2, 3, 4, 5] @@ -780,9 +777,9 @@ def testGenexps(self): # generator expression tests g = ([x for x in range(10)] for x in range(1)) - self.assertEqual(g.next(), [x for x in range(10)]) + self.assertEqual(next(g), [x for x in range(10)]) try: - g.next() + next(g) self.fail('should produce StopIteration exception') except StopIteration: pass @@ -790,7 +787,7 @@ a = 1 try: g = (a for d in a) - g.next() + next(g) self.fail('should produce TypeError') except TypeError: pass @@ -838,7 +835,8 @@ print(x) return ret - self.assertEqual([ x() for x in lambda: True, lambda: False if x() ], [True]) + # the next line is not allowed anymore + #self.assertEqual([ x() for x in lambda: True, lambda: False if x() ], [True]) self.assertEqual([ x() for x in (lambda: True, lambda: False) if x() ], [True]) self.assertEqual([ x(False) for x in (lambda x: False if x else True, lambda x: True if x else False) if x(False) ], [True]) self.assertEqual((5 if 1 else _checkeval("check 1", 0)), 5) From python-checkins at python.org Sun Jul 15 00:34:03 2007 From: python-checkins at python.org (collin.winter) Date: Sun, 15 Jul 2007 00:34:03 +0200 (CEST) Subject: [Python-checkins] r56387 - in sandbox/trunk/2to3: README fixes/fix_funcattrs.py tests/test_fixers.py Message-ID: <20070714223403.44DBE1E4008@bag.python.org> Author: collin.winter Date: Sun Jul 15 00:34:02 2007 New Revision: 56387 Added: sandbox/trunk/2to3/fixes/fix_funcattrs.py Modified: sandbox/trunk/2to3/ (props changed) sandbox/trunk/2to3/README sandbox/trunk/2to3/tests/test_fixers.py Log: Add a fixer for function attributes (f.func_x -> f.__x__). Modified: sandbox/trunk/2to3/README ============================================================================== --- sandbox/trunk/2to3/README (original) +++ sandbox/trunk/2to3/README Sun Jul 15 00:34:02 2007 @@ -45,6 +45,8 @@ * **fix_filter** - changes filter(F, X) into list(filter(F, X)). +* **fix_funcattrs** - fix function attribute names (f.func_x -> f.__x__). + * **fix_has_key** - "d.has_key(x)" -> "x in d". * **fix_input** - "input()" -> "eval(input())" (PEP 3111). Added: sandbox/trunk/2to3/fixes/fix_funcattrs.py ============================================================================== --- (empty file) +++ sandbox/trunk/2to3/fixes/fix_funcattrs.py Sun Jul 15 00:34:02 2007 @@ -0,0 +1,19 @@ +"""Fix function attribute names (f.func_x -> f.__x__).""" +# Author: Collin Winter + +# Local imports +from fixes import basefix +from fixes.util import Name + + +class FixFuncattrs(basefix.BaseFix): + PATTERN = """ + power< any trailer< '.' attr=('func_closure' | 'func_doc' | 'func_globals' + | 'func_name' | 'func_defaults' | 'func_code' + | 'func_dict') > any* > + """ + + def transform(self, node, results): + attr = results["attr"][0] + attr.replace(Name(("__%s__" % attr.value[5:]), + prefix=attr.get_prefix())) Modified: sandbox/trunk/2to3/tests/test_fixers.py ============================================================================== --- sandbox/trunk/2to3/tests/test_fixers.py (original) +++ sandbox/trunk/2to3/tests/test_fixers.py Sun Jul 15 00:34:02 2007 @@ -1180,6 +1180,26 @@ self.check(b, a) +class Test_funcattrs(FixerTestCase): + fixer = "funcattrs" + + attrs = ["closure", "doc", "name", "defaults", "code", "globals", "dict"] + + def test(self): + for attr in self.attrs: + b = "a.func_%s" % attr + a = "a.__%s__" % attr + self.check(b, a) + + def test_unchanged(self): + for attr in self.attrs: + s = "foo(func_%s + 5)" % attr + self.check(s, s) + + s = "f(foo.__%s__)" % attr + self.check(s, s) + + class Test_stringio(FixerTestCase): fixer = "stringio" From python-checkins at python.org Sun Jul 15 00:34:18 2007 From: python-checkins at python.org (collin.winter) Date: Sun, 15 Jul 2007 00:34:18 +0200 (CEST) Subject: [Python-checkins] r56388 - in sandbox/trunk/2to3: pytree.py tests/test_pytree.py Message-ID: <20070714223418.7B1201E4008@bag.python.org> Author: collin.winter Date: Sun Jul 15 00:34:18 2007 New Revision: 56388 Modified: sandbox/trunk/2to3/ (props changed) sandbox/trunk/2to3/pytree.py sandbox/trunk/2to3/tests/test_pytree.py Log: Remove the ability to remove a node by passing None to replace(); add the ability to replace a node with a list of nodes. Modified: sandbox/trunk/2to3/pytree.py ============================================================================== --- sandbox/trunk/2to3/pytree.py (original) +++ sandbox/trunk/2to3/pytree.py Sun Jul 15 00:34:18 2007 @@ -101,28 +101,26 @@ raise NotImplementedError def replace(self, new): - """Replaces this node with a new one in the parent. - - This can also be used to remove this node from the parent by - passing None. - """ + """Replaces this node with a new one in the parent.""" assert self.parent is not None, str(self) - assert new is None or new.parent is None, str(new) + assert new is not None + if not isinstance(new, list): + new = [new] l_children = [] found = False for ch in self.parent.children: if ch is self: assert not found, (self.parent.children, self, new) if new is not None: - l_children.append(new) + l_children.extend(new) found = True else: l_children.append(ch) assert found, (self.children, self, new) - self.changed() - self.parent.children = tuple(l_children) - if new is not None: - new.parent = self.parent + self.parent.changed() + self.parent.children = l_children + for x in new: + x.parent = self.parent self.parent = None def get_lineno(self): Modified: sandbox/trunk/2to3/tests/test_pytree.py ============================================================================== --- sandbox/trunk/2to3/tests/test_pytree.py (original) +++ sandbox/trunk/2to3/tests/test_pytree.py Sun Jul 15 00:34:18 2007 @@ -140,12 +140,24 @@ l3 = pytree.Leaf(100, "bar") n1 = pytree.Node(1000, [l1, l2, l3]) self.assertEqual(n1.children, [l1, l2, l3]) + self.failUnless(isinstance(n1.children, list)) self.failIf(n1.was_changed) l2new = pytree.Leaf(100, "-") l2.replace(l2new) - self.assertEqual(n1.children, (l1, l2new, l3)) + self.assertEqual(n1.children, [l1, l2new, l3]) + self.failUnless(isinstance(n1.children, list)) self.failUnless(n1.was_changed) + def testReplaceWithList(self): + l1 = pytree.Leaf(100, "foo") + l2 = pytree.Leaf(100, "+") + l3 = pytree.Leaf(100, "bar") + n1 = pytree.Node(1000, [l1, l2, l3]) + + l2.replace([pytree.Leaf(100, "*"), pytree.Leaf(100, "*")]) + self.assertEqual(str(n1), "foo**bar") + self.failUnless(isinstance(n1.children, list)) + def testConvert(self): # XXX pass From python-checkins at python.org Sun Jul 15 00:34:33 2007 From: python-checkins at python.org (collin.winter) Date: Sun, 15 Jul 2007 00:34:33 +0200 (CEST) Subject: [Python-checkins] r56389 - in sandbox/trunk/2to3: tests/test_pytree.py Message-ID: <20070714223433.6113D1E4008@bag.python.org> Author: collin.winter Date: Sun Jul 15 00:34:33 2007 New Revision: 56389 Modified: sandbox/trunk/2to3/ (props changed) sandbox/trunk/2to3/tests/test_pytree.py Log: Remove the stub for a test that's never going to get filled in. Modified: sandbox/trunk/2to3/tests/test_pytree.py ============================================================================== --- sandbox/trunk/2to3/tests/test_pytree.py (original) +++ sandbox/trunk/2to3/tests/test_pytree.py Sun Jul 15 00:34:33 2007 @@ -158,10 +158,6 @@ self.assertEqual(str(n1), "foo**bar") self.failUnless(isinstance(n1.children, list)) - def testConvert(self): - # XXX - pass - def testPostOrder(self): l1 = pytree.Leaf(100, "foo") l2 = pytree.Leaf(100, "bar") From python-checkins at python.org Sun Jul 15 00:34:48 2007 From: python-checkins at python.org (collin.winter) Date: Sun, 15 Jul 2007 00:34:48 +0200 (CEST) Subject: [Python-checkins] r56390 - in sandbox/trunk/2to3: README fixes/fix_xreadlines.py tests/test_fixers.py Message-ID: <20070714223448.808921E4016@bag.python.org> Author: collin.winter Date: Sun Jul 15 00:34:48 2007 New Revision: 56390 Added: sandbox/trunk/2to3/fixes/fix_xreadlines.py Modified: sandbox/trunk/2to3/ (props changed) sandbox/trunk/2to3/README sandbox/trunk/2to3/tests/test_fixers.py Log: Add a fixer to convert 'for x in f.xreadlines():' into 'for x in f:'. Modified: sandbox/trunk/2to3/README ============================================================================== --- sandbox/trunk/2to3/README (original) +++ sandbox/trunk/2to3/README Sun Jul 15 00:34:48 2007 @@ -87,6 +87,9 @@ * **fix_xrange** - "xrange()" -> "range()". +* **fix_xreadlines** - "for x in f.xreadlines():" -> "for x in f:". Also, + "g(f.xreadlines)" -> "g(f.__iter__)". + Limitations =========== Added: sandbox/trunk/2to3/fixes/fix_xreadlines.py ============================================================================== --- (empty file) +++ sandbox/trunk/2to3/fixes/fix_xreadlines.py Sun Jul 15 00:34:48 2007 @@ -0,0 +1,24 @@ +"""Fix "for x in f.xreadlines()" -> "for x in f". + +This fixer will also convert g(f.xreadlines) into g(f.__iter__).""" +# Author: Collin Winter + +# Local imports +from fixes import basefix +from fixes.util import Name + + +class FixXreadlines(basefix.BaseFix): + PATTERN = """ + power< call=any+ trailer< '.' 'xreadlines' > trailer< '(' ')' > > + | + power< any+ trailer< '.' no_call='xreadlines' > > + """ + + def transform(self, node, results): + no_call = results.get("no_call") + + if no_call: + no_call.replace(Name("__iter__", prefix=no_call.get_prefix())) + else: + node.replace([x.clone() for x in results["call"]]) Modified: sandbox/trunk/2to3/tests/test_fixers.py ============================================================================== --- sandbox/trunk/2to3/tests/test_fixers.py (original) +++ sandbox/trunk/2to3/tests/test_fixers.py Sun Jul 15 00:34:48 2007 @@ -1200,6 +1200,49 @@ self.check(s, s) +class Test_xreadlines(FixerTestCase): + fixer = "xreadlines" + + def test_call(self): + b = "for x in f.xreadlines(): pass" + a = "for x in f: pass" + self.check(b, a) + + b = "for x in foo().xreadlines(): pass" + a = "for x in foo(): pass" + self.check(b, a) + + b = "for x in (5 + foo()).xreadlines(): pass" + a = "for x in (5 + foo()): pass" + self.check(b, a) + + def test_attr_ref(self): + b = "foo(f.xreadlines + 5)" + a = "foo(f.__iter__ + 5)" + self.check(b, a) + + b = "foo(f().xreadlines + 5)" + a = "foo(f().__iter__ + 5)" + self.check(b, a) + + b = "foo((5 + f()).xreadlines + 5)" + a = "foo((5 + f()).__iter__ + 5)" + self.check(b, a) + + def test_unchanged(self): + s = "for x in f.xreadlines(5): pass" + self.check(s, s) + + s = "for x in f.xreadlines(k=5): pass" + self.check(s, s) + + s = "for x in f.xreadlines(*k, **v): pass" + self.check(s, s) + + s = "foo(xreadlines)" + self.check(s, s) + + class Test_stringio(FixerTestCase): fixer = "stringio" From python-checkins at python.org Sun Jul 15 00:35:04 2007 From: python-checkins at python.org (collin.winter) Date: Sun, 15 Jul 2007 00:35:04 +0200 (CEST) Subject: [Python-checkins] r56391 - in sandbox/trunk/2to3: README fixes/fix_sysexcattrs.py tests/test_fixers.py Message-ID: <20070714223504.1D8D71E401C@bag.python.org> Author: collin.winter Date: Sun Jul 15 00:35:03 2007 New Revision: 56391 Removed: sandbox/trunk/2to3/fixes/fix_sysexcattrs.py Modified: sandbox/trunk/2to3/ (props changed) sandbox/trunk/2to3/README sandbox/trunk/2to3/tests/test_fixers.py Log: Remove the sysexcattrs fixer; 2.6's warning mode will take care of this. Modified: sandbox/trunk/2to3/README ============================================================================== --- sandbox/trunk/2to3/README (original) +++ sandbox/trunk/2to3/README Sun Jul 15 00:35:03 2007 @@ -75,9 +75,6 @@ * **fix_stringio** - StringIO.StringIO -> io.StringIO (imports, too). -* **fix_sysexcattrs** - warn on usage of sys.value, sys.type and - sys.traceback. - * **fix_throw** - fix generator.throw() calls to be 3.0-compliant (PEP 3109). * **fix_tuple_params** - remove tuple parameters from function, method and Deleted: /sandbox/trunk/2to3/fixes/fix_sysexcattrs.py ============================================================================== --- /sandbox/trunk/2to3/fixes/fix_sysexcattrs.py Sun Jul 15 00:35:03 2007 +++ (empty file) @@ -1,18 +0,0 @@ -"""Fixer/warner for sys.exc_{value,type,traceback}""" -# Author: Collin Winter - -# Local imports -from pytree import Leaf -from fixes import basefix - - -class FixSysexcattrs(basefix.BaseFix): - - PATTERN = """ - power< 'sys' - trailer< '.' ('exc_value' | 'exc_traceback' | 'exc_type')> - any* > - """ - - def transform(self, node, results): - self.cannot_convert(node, "This attribute is going away in Python 3") Modified: sandbox/trunk/2to3/tests/test_fixers.py ============================================================================== --- sandbox/trunk/2to3/tests/test_fixers.py (original) +++ sandbox/trunk/2to3/tests/test_fixers.py Sun Jul 15 00:35:03 2007 @@ -940,20 +940,6 @@ self.check(b, a) -class Test_sysexcattrs(FixerTestCase): - fixer = "sysexcattrs" - - def test(self): - s = """f = sys.exc_type""" - self.warns(s, s, "This attribute is going away") - - s = """f = sys.exc_value""" - self.warns(s, s, "This attribute is going away") - - s = """f = sys.exc_traceback""" - self.warns(s, s, "This attribute is going away") - - class Test_dict(FixerTestCase): fixer = "dict" From python-checkins at python.org Sun Jul 15 00:41:45 2007 From: python-checkins at python.org (facundo.batista) Date: Sun, 15 Jul 2007 00:41:45 +0200 (CEST) Subject: [Python-checkins] r56392 - python/trunk/Lib/test/test_asyncore.py Message-ID: <20070714224145.9C1A31E4008@bag.python.org> Author: facundo.batista Date: Sun Jul 15 00:41:45 2007 New Revision: 56392 Added: python/trunk/Lib/test/test_asyncore.py Log: First version. Includes tests for helper functions: read, write, _exception, readwrite, closeall, compact_traceback; and for classes dispatcher, dispatcher_with_send, and file_wrapper. [Alan McIntyre - GSoC] Added: python/trunk/Lib/test/test_asyncore.py ============================================================================== --- (empty file) +++ python/trunk/Lib/test/test_asyncore.py Sun Jul 15 00:41:45 2007 @@ -0,0 +1,370 @@ +import asyncore +import unittest +import select +import os +import socket +import threading +import sys +import time + +from test import test_support +from test.test_support import TESTFN, run_unittest, unlink +from StringIO import StringIO + +HOST = "127.0.0.1" +PORT = 54322 + +class dummysocket: + def __init__(self): + self.closed = False + + def close(self): + self.closed = True + + def fileno(self): + return 42 + +class dummychannel: + def __init__(self): + self.socket = dummysocket() + +class exitingdummy: + def __init__(self): + pass + + def handle_read_event(self): + raise asyncore.ExitNow() + + handle_write_event = handle_read_event + handle_expt_event = handle_read_event + +class crashingdummy: + def __init__(self): + self.error_handled = False + + def handle_read_event(self): + raise Exception() + + handle_write_event = handle_read_event + handle_expt_event = handle_read_event + + def handle_error(self): + self.error_handled = True + +# used when testing senders; just collects what it gets until newline is sent +class capture_server(threading.Thread): + def run(self): + sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) + sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) + global PORT + PORT = test_support.bind_port(sock, HOST, PORT) + sock.listen(1) + conn, client = sock.accept() + self.captured = "" + while 1: + data = conn.recv(10) + if '\n' in data: + break + self.captured = self.captured + data + + conn.close() + sock.close() + + +class HelperFunctionTests(unittest.TestCase): + def test_readwriteexc(self): + # Check exception handling behavior of read, write and _exception + + # check that ExitNow exceptions in the object handler method + # bubbles all the way up through asyncore read/write/_exception calls + tr1 = exitingdummy() + self.assertRaises(asyncore.ExitNow, asyncore.read, tr1) + self.assertRaises(asyncore.ExitNow, asyncore.write, tr1) + self.assertRaises(asyncore.ExitNow, asyncore._exception, tr1) + + # check that an exception other than ExitNow in the object handler + # method causes the handle_error method to get called + tr2 = crashingdummy() + asyncore.read(tr2) + self.assertEqual(tr2.error_handled, True) + + tr2 = crashingdummy() + asyncore.write(tr2) + self.assertEqual(tr2.error_handled, True) + + tr2 = crashingdummy() + asyncore._exception(tr2) + self.assertEqual(tr2.error_handled, True) + + def test_readwrite(self): + # Check that correct methods are called by readwrite() + + class testobj: + def __init__(self): + self.read = False + self.write = False + self.expt = False + + def handle_read_event(self): + self.read = True + + def handle_write_event(self): + self.write = True + + def handle_expt_event(self): + self.expt = True + + def handle_error(self): + self.error_handled = True + + for flag in (select.POLLIN, select.POLLPRI): + tobj = testobj() + self.assertEqual(tobj.read, False) + asyncore.readwrite(tobj, flag) + self.assertEqual(tobj.read, True) + + # check that ExitNow exceptions in the object handler method + # bubbles all the way up through asyncore readwrite call + tr1 = exitingdummy() + self.assertRaises(asyncore.ExitNow, asyncore.readwrite, tr1, flag) + + # check that an exception other than ExitNow in the object handler + # method causes the handle_error method to get called + tr2 = crashingdummy() + asyncore.readwrite(tr2, flag) + self.assertEqual(tr2.error_handled, True) + + tobj = testobj() + self.assertEqual(tobj.write, False) + asyncore.readwrite(tobj, select.POLLOUT) + self.assertEqual(tobj.write, True) + + # check that ExitNow exceptions in the object handler method + # bubbles all the way up through asyncore readwrite call + tr1 = exitingdummy() + self.assertRaises(asyncore.ExitNow, asyncore.readwrite, tr1, + select.POLLOUT) + + # check that an exception other than ExitNow in the object handler + # method causes the handle_error method to get called + tr2 = crashingdummy() + asyncore.readwrite(tr2, select.POLLOUT) + self.assertEqual(tr2.error_handled, True) + + for flag in (select.POLLERR, select.POLLHUP, select.POLLNVAL): + tobj = testobj() + self.assertEqual(tobj.expt, False) + asyncore.readwrite(tobj, flag) + self.assertEqual(tobj.expt, True) + + # check that ExitNow exceptions in the object handler method + # bubbles all the way up through asyncore readwrite calls + tr1 = exitingdummy() + self.assertRaises(asyncore.ExitNow, asyncore.readwrite, tr1, flag) + + # check that an exception other than ExitNow in the object handler + # method causes the handle_error method to get called + tr2 = crashingdummy() + asyncore.readwrite(tr2, flag) + self.assertEqual(tr2.error_handled, True) + + def test_closeall(self): + self.closeall_check(False) + + def test_closeall_default(self): + self.closeall_check(True) + + def closeall_check(self, usedefault): + # Check that close_all() closes everything in a given map + + l = [] + testmap = {} + for i in range(10): + c = dummychannel() + l.append(c) + self.assertEqual(c.socket.closed, False) + testmap[i] = c + + if usedefault: + socketmap = asyncore.socket_map + try: + asyncore.socket_map = testmap + asyncore.close_all() + finally: + testmap, asyncore.socket_map = asyncore.socket_map, socketmap + else: + asyncore.close_all(testmap) + + self.assertEqual(len(testmap), 0) + + for c in l: + self.assertEqual(c.socket.closed, True) + + def test_compact_traceback(self): + try: + raise Exception("I don't like spam!") + except: + real_t, real_v, real_tb = sys.exc_info() + r = asyncore.compact_traceback() + else: + self.fail("Expected exception") + + (f, function, line), t, v, info = r + self.assertEqual(os.path.split(f)[-1], 'test_asyncore.py') + self.assertEqual(function, 'test_compact_traceback') + self.assertEqual(t, real_t) + self.assertEqual(v, real_v) + self.assertEqual(info, '[%s|%s|%s]' % (f, function, line)) + + +class DispatcherTests(unittest.TestCase): + def test_basic(self): + d = asyncore.dispatcher() + self.assertEqual(d.readable(), True) + self.assertEqual(d.writable(), True) + + def test_repr(self): + d = asyncore.dispatcher() + self.assertEqual(repr(d), '' % id(d)) + + def test_log(self): + d = asyncore.dispatcher() + + # capture output of dispatcher.log() (to stderr) + fp = StringIO() + stderr = sys.stderr + l1 = "Lovely spam! Wonderful spam!" + l2 = "I don't like spam!" + try: + sys.stderr = fp + d.log(l1) + d.log(l2) + finally: + sys.stderr = stderr + + lines = fp.getvalue().splitlines() + self.assertEquals(lines, ['log: %s' % l1, 'log: %s' % l2]) + + def test_log_info(self): + d = asyncore.dispatcher() + + # capture output of dispatcher.log_info() (to stdout via print) + fp = StringIO() + stdout = sys.stdout + l1 = "Have you got anything without spam?" + l2 = "Why can't she have egg bacon spam and sausage?" + l3 = "THAT'S got spam in it!" + try: + sys.stdout = fp + d.log_info(l1, 'EGGS') + d.log_info(l2) + d.log_info(l3, 'SPAM') + finally: + sys.stdout = stdout + + lines = fp.getvalue().splitlines() + expected = ['EGGS: %s' % l1, 'info: %s' % l2, 'SPAM: %s' % l3] + self.assertEquals(lines, expected) + + def test_unhandled(self): + d = asyncore.dispatcher() + + # capture output of dispatcher.log_info() (to stdout via print) + fp = StringIO() + stdout = sys.stdout + try: + sys.stdout = fp + d.handle_expt() + d.handle_read() + d.handle_write() + d.handle_connect() + d.handle_accept() + finally: + sys.stdout = stdout + + lines = fp.getvalue().splitlines() + expected = ['warning: unhandled exception', + 'warning: unhandled read event', + 'warning: unhandled write event', + 'warning: unhandled connect event', + 'warning: unhandled accept event'] + self.assertEquals(lines, expected) + + + +class dispatcherwithse