Mailman 3 April 2013 - Python-checkins

peps: Small touches on framing
by antoine.pitrou April 26, 2013

April 26, 2013

http://hg.python.org/peps/rev/881901606fbe changeset: 4859:881901606fbe user: Antoine Pitrou <solipsis(a)pitrou.net> date: Fri Apr 26 23:10:16 2013 +0200 summary: Small touches on framing files: pep-3154.txt | 11 +++++++---- 1 files changed, 7 insertions(+), 4 deletions(-) diff --git a/pep-3154.txt b/pep-3154.txt --- a/pep-3154.txt +++ b/pep-3154.txt @@ -71,11 +71,14 @@ To keep the implementation simple, it is forbidden for a pickle opcode to overlap frame boundaries. The pickler takes care not to produce such -pickles, and the unpickler refuses them. +pickles, and the unpickler refuses them. Also, there is no "last frame" +marker. The last frame is simply the one which ends with a STOP opcode. -How the pickler decides frame sizes is an implementation detail. -A simple heuristic committing the current frame as soon as it reaches -64 KiB seems sufficient. +.. note:: + + How the pickler partitions the pickle stream into frames is an + implementation detail. "Closing" a frame as soon as it reaches + ~64 KiB should be ok for both performance and pickle size overhead. Binary encoding for all opcodes ------------------------------- -- Repository URL: http://hg.python.org/peps

1 0

peps: Pickle 4 changes:
by antoine.pitrou April 26, 2013

April 26, 2013

http://hg.python.org/peps/rev/6086bd69599e changeset: 4858:6086bd69599e user: Antoine Pitrou <solipsis(a)pitrou.net> date: Fri Apr 26 22:57:06 2013 +0200 summary: Pickle 4 changes: - add framing - change BINGLOBAL to Alexandre Vassalotti's GLOBAL_STACK files: pep-3154.txt | 91 ++++++++++++++++++++++++++------------- 1 files changed, 60 insertions(+), 31 deletions(-) diff --git a/pep-3154.txt b/pep-3154.txt --- a/pep-3154.txt +++ b/pep-3154.txt @@ -42,11 +42,67 @@ introduction of a new protocol version should be a rare occurrence. -Improvements in discussion -========================== +Proposed changes +================ -64-bit compatibility for large objects --------------------------------------- +Framing +------- + +Traditionally, when unpickling an object from a stream (by calling +``load()`` rather than ``loads()``), many small ``read()`` +calls can be issued on the file-like object, with a potentially huge +performance impact. + +Protocol 4, by contrast, features binary framing. The general structure +of a pickle is thus the following:: + + +------+------+ + | 0x80 | 0x03 | protocol header (2 bytes) + +------+------+-----------+ + | AA BB CC DD EE FF GG HH | frame size (8 bytes, little-endian) + +------+------------------+ + | .... | first frame contents (N bytes) + +------+------+-----------+ + | AA BB CC DD EE FF GG HH | frame size (8 bytes, little-endian) + +------+------------------+ + | .... | second frame contents (N bytes) + +------+ + etc. + +To keep the implementation simple, it is forbidden for a pickle opcode +to overlap frame boundaries. The pickler takes care not to produce such +pickles, and the unpickler refuses them. + +How the pickler decides frame sizes is an implementation detail. +A simple heuristic committing the current frame as soon as it reaches +64 KiB seems sufficient. + +Binary encoding for all opcodes +------------------------------- + +The GLOBAL opcode, which is still used in protocol 3, uses the +so-called "text" mode of the pickle protocol, which involves looking +for newlines in the pickle stream. It also complicates the implementation +of binary framing. + +Protocol 4 forbids use of the GLOBAL opcode and replaces it with +GLOBAL_STACK, a new opcode which takes its operand from the stack. + +Serializing more "lookupable" objects +------------------------------------- + +By default, pickle is only able to serialize module-global functions and +classes. Supporting other kinds of objects, such as unbound methods [4]_, +is a common request. Actually, third-party support for some of them, such +as bound methods, is implemented in the multiprocessing module [5]_. + +The ``__qualname__`` attribute from :pep:`3155` makes it possible to +lookup many more objects by name. Making the GLOBAL_STACK opcode accept +dot-separated names, or adding a special GETATTR opcode, would allow the +standard pickle implementation to support all those kinds of objects. + +64-bit opcodes for large objects +-------------------------------- Current protocol versions export object sizes for various built-in types (str, bytes) as 32-bit ints. This forbids serialization of @@ -71,33 +127,6 @@ special method (``__getnewargs_ex__`` ?) and a new opcode (NEWOBJEX ?) are needed. -Serializing more "lookupable" objects -------------------------------------- - -For some kinds of objects, it only makes sense to serialize them by name -(for example classes and functions). By default, pickle is only able to -serialize module-global functions and classes by name. Supporting other -kinds of objects, such as unbound methods [4]_, is a common request. -Actually, third-party support for some of them, such as bound methods, -is implemented in the multiprocessing module [5]_. - -:pep:`3155` now makes it possible to lookup many more objects by name. -Generalizing the GLOBAL opcode to accept dot-separated names, or adding -a special GETATTR opcode, would allow the standard pickle implementation -to support, in an efficient way, all those kinds of objects. - -Binary encoding for all opcodes -------------------------------- - -The GLOBAL opcode, which is still used in protocol 3, uses the -so-called "text" mode of the pickle protocol, which involves looking -for newlines in the pickle stream. Looking for newlines is difficult -to optimize on a non-seekable stream, and therefore a new version of -GLOBAL (BINGLOBAL?) could use a binary encoding instead. - -It seems that all other opcodes emitted when using protocol 3 already -use binary encoding. - Better string encoding ---------------------- -- Repository URL: http://hg.python.org/peps

1 0

cpython (merge default -> default): merge heads
by giampaolo.rodola April 26, 2013

April 26, 2013

http://hg.python.org/cpython/rev/1a9c9861e56a changeset: 83523:1a9c9861e56a parent: 83522:8f7bdfe81b76 parent: 83521:58b0e301b78a user: Giampaolo Rodola' <g.rodola(a)gmail.com> date: Fri Apr 26 15:24:42 2013 +0200 summary: merge heads files: Lib/keyword.py | 22 +++++---- Lib/test/test_keyword.py | 58 ++++++++++++++------------- Lib/test/test_urllib2.py | 24 +++++++++++ Lib/urllib/request.py | 27 +++++++++--- Misc/ACKS | 1 + Misc/NEWS | 6 ++ 6 files changed, 93 insertions(+), 45 deletions(-) diff --git a/Lib/keyword.py b/Lib/keyword.py --- a/Lib/keyword.py +++ b/Lib/keyword.py @@ -60,6 +60,12 @@ if len(args) > 1: optfile = args[1] else: optfile = "Lib/keyword.py" + # load the output skeleton from the target, taking care to preserve its + # newline convention. + with open(optfile, newline='') as fp: + format = fp.readlines() + nl = format[0][len(format[0].strip()):] if format else '\n' + # scan the source file for keywords with open(iptfile) as fp: strprog = re.compile('"([^"]+)"') @@ -68,25 +74,21 @@ if '{1, "' in line: match = strprog.search(line) if match: - lines.append(" '" + match.group(1) + "',\n") + lines.append(" '" + match.group(1) + "'," + nl) lines.sort() - # load the output skeleton from the target - with open(optfile) as fp: - format = fp.readlines() - - # insert the lines of keywords + # insert the lines of keywords into the skeleton try: - start = format.index("#--start keywords--\n") + 1 - end = format.index("#--end keywords--\n") + start = format.index("#--start keywords--" + nl) + 1 + end = format.index("#--end keywords--" + nl) format[start:end] = lines except ValueError: sys.stderr.write("target does not contain format markers\n") sys.exit(1) # write the output file - with open(optfile, 'w') as fp: - fp.write(''.join(format)) + with open(optfile, 'w', newline='') as fp: + fp.writelines(format) if __name__ == "__main__": main() diff --git a/Lib/test/test_keyword.py b/Lib/test/test_keyword.py --- a/Lib/test/test_keyword.py +++ b/Lib/test/test_keyword.py @@ -9,7 +9,8 @@ import textwrap KEYWORD_FILE = support.findfile('keyword.py') -GRAMMAR_FILE = os.path.join('..', '..', 'Python', 'graminit.c') +GRAMMAR_FILE = os.path.join(os.path.split(__file__)[0], + '..', '..', 'Python', 'graminit.c') TEST_PY_FILE = 'keyword_test.py' GRAMMAR_TEST_FILE = 'graminit_test.c' PY_FILE_WITHOUT_KEYWORDS = 'minimal_keyword.py' @@ -30,7 +31,7 @@ # preserved for backward compatibility. def test_changing_the_kwlist_does_not_affect_iskeyword(self): oldlist = keyword.kwlist - self.addCleanup(lambda: setattr(keyword, 'kwlist', oldlist)) + self.addCleanup(setattr, keyword, 'kwlist', oldlist) keyword.kwlist = ['its', 'all', 'eggs', 'beans', 'and', 'a', 'slice'] self.assertFalse(keyword.iskeyword('eggs')) @@ -38,11 +39,12 @@ class TestKeywordGeneration(unittest.TestCase): def _copy_file_without_generated_keywords(self, source_file, dest_file): - with open(source_file) as fp: + with open(source_file, 'rb') as fp: lines = fp.readlines() - with open(dest_file, 'w') as fp: - fp.writelines(lines[:lines.index("#--start keywords--\n") + 1]) - fp.writelines(lines[lines.index("#--end keywords--\n"):]) + nl = lines[0][len(lines[0].strip()):] + with open(dest_file, 'wb') as fp: + fp.writelines(lines[:lines.index(b"#--start keywords--" + nl) + 1]) + fp.writelines(lines[lines.index(b"#--end keywords--" + nl):]) def _generate_keywords(self, grammar_file, target_keyword_py_file): proc = subprocess.Popen([sys.executable, @@ -56,15 +58,15 @@ 'test only works from source build directory') def test_real_grammar_and_keyword_file(self): self._copy_file_without_generated_keywords(KEYWORD_FILE, TEST_PY_FILE) - self.addCleanup(lambda: support.unlink(TEST_PY_FILE)) + self.addCleanup(support.unlink, TEST_PY_FILE) self.assertFalse(filecmp.cmp(KEYWORD_FILE, TEST_PY_FILE)) - self.assertEqual(0, self._generate_keywords(GRAMMAR_FILE, - TEST_PY_FILE)[0]) + self.assertEqual((0, b''), self._generate_keywords(GRAMMAR_FILE, + TEST_PY_FILE)) self.assertTrue(filecmp.cmp(KEYWORD_FILE, TEST_PY_FILE)) def test_grammar(self): self._copy_file_without_generated_keywords(KEYWORD_FILE, TEST_PY_FILE) - self.addCleanup(lambda: support.unlink(TEST_PY_FILE)) + self.addCleanup(support.unlink, TEST_PY_FILE) with open(GRAMMAR_TEST_FILE, 'w') as fp: # Some of these are probably implementation accidents. fp.writelines(textwrap.dedent("""\ @@ -86,40 +88,40 @@ {1, 'no good'} {283, 0}, {1, "too many spaces"}""")) - self.addCleanup(lambda: support.unlink(GRAMMAR_TEST_FILE)) + self.addCleanup(support.unlink, GRAMMAR_TEST_FILE) self._generate_keywords(GRAMMAR_TEST_FILE, TEST_PY_FILE) expected = [ - " 'This one is tab indented',\n", - " 'also legal',\n", - " 'continue',\n", - " 'crazy but legal',\n", - " 'jello',\n", - " 'lemon',\n", - " 'tomato',\n", - " 'turnip',\n", - " 'wigii',\n", + " 'This one is tab indented',", + " 'also legal',", + " 'continue',", + " 'crazy but legal',", + " 'jello',", + " 'lemon',", + " 'tomato',", + " 'turnip',", + " 'wigii',", ] with open(TEST_PY_FILE) as fp: - lines = fp.readlines() - start = lines.index("#--start keywords--\n") + 1 - end = lines.index("#--end keywords--\n") + lines = fp.read().splitlines() + start = lines.index("#--start keywords--") + 1 + end = lines.index("#--end keywords--") actual = lines[start:end] self.assertEqual(actual, expected) def test_empty_grammar_results_in_no_keywords(self): self._copy_file_without_generated_keywords(KEYWORD_FILE, PY_FILE_WITHOUT_KEYWORDS) - self.addCleanup(lambda: support.unlink(PY_FILE_WITHOUT_KEYWORDS)) + self.addCleanup(support.unlink, PY_FILE_WITHOUT_KEYWORDS) shutil.copyfile(KEYWORD_FILE, TEST_PY_FILE) - self.addCleanup(lambda: support.unlink(TEST_PY_FILE)) - self.assertEqual(0, self._generate_keywords(os.devnull, - TEST_PY_FILE)[0]) + self.addCleanup(support.unlink, TEST_PY_FILE) + self.assertEqual((0, b''), self._generate_keywords(os.devnull, + TEST_PY_FILE)) self.assertTrue(filecmp.cmp(TEST_PY_FILE, PY_FILE_WITHOUT_KEYWORDS)) def test_keywords_py_without_markers_produces_error(self): rc, stderr = self._generate_keywords(os.devnull, os.devnull) self.assertNotEqual(rc, 0) - self.assertEqual(stderr, b'target does not contain format markers\n') + self.assertRegex(stderr, b'does not contain format markers') def test_missing_grammar_file_produces_error(self): rc, stderr = self._generate_keywords(NONEXISTENT_FILE, KEYWORD_FILE) diff --git a/Lib/test/test_urllib2.py b/Lib/test/test_urllib2.py --- a/Lib/test/test_urllib2.py +++ b/Lib/test/test_urllib2.py @@ -904,6 +904,30 @@ p_ds_req = h.do_request_(ds_req) self.assertEqual(p_ds_req.unredirected_hdrs["Host"],"example.com") + def test_full_url_setter(self): + # Checks to ensure that components are set correctly after setting the + # full_url of a Request object + + urls = [ + 'http://example.com?foo=bar#baz', + 'http://example.com?foo=bar&spam=eggs#bash', + 'http://example.com', + ] + + # testing a reusable request instance, but the url parameter is + # required, so just use a dummy one to instantiate + r = Request('http://example.com') + for url in urls: + r.full_url = url + self.assertEqual(r.get_full_url(), url) + + def test_full_url_deleter(self): + r = Request('http://www.example.com') + del r.full_url + self.assertIsNone(r.full_url) + self.assertIsNone(r.fragment) + self.assertEqual(r.selector, '') + def test_fixpath_in_weirdurls(self): # Issue4493: urllib2 to supply '/' when to urls where path does not # start with'/' diff --git a/Lib/urllib/request.py b/Lib/urllib/request.py --- a/Lib/urllib/request.py +++ b/Lib/urllib/request.py @@ -259,9 +259,7 @@ def __init__(self, url, data=None, headers={}, origin_req_host=None, unverifiable=False, method=None): - # unwrap('<URL:type://host/path>') --> 'type://host/path' - self.full_url = unwrap(url) - self.full_url, self.fragment = splittag(self.full_url) + self.full_url = url self.headers = {} self.unredirected_hdrs = {} self._data = None @@ -274,8 +272,24 @@ self.origin_req_host = origin_req_host self.unverifiable = unverifiable self.method = method + + @property + def full_url(self): + return self._full_url + + @full_url.setter + def full_url(self, url): + # unwrap('<URL:type://host/path>') --> 'type://host/path' + self._full_url = unwrap(url) + self._full_url, self.fragment = splittag(self._full_url) self._parse() + @full_url.deleter + def full_url(self): + self._full_url = None + self.fragment = None + self.selector = '' + @property def data(self): return self._data @@ -295,7 +309,7 @@ self.data = None def _parse(self): - self.type, rest = splittype(self.full_url) + self.type, rest = splittype(self._full_url) if self.type is None: raise ValueError("unknown url type: %r" % self.full_url) self.host, self.selector = splithost(rest) @@ -313,9 +327,8 @@ def get_full_url(self): if self.fragment: - return '%s#%s' % (self.full_url, self.fragment) - else: - return self.full_url + return '{}#{}'.format(self.full_url, self.fragment) + return self.full_url def set_proxy(self, host, type): if self.type == 'https' and not self._tunnel_host: diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -153,6 +153,7 @@ Sven Brauch Erik Bray Brian Brazil +Demian Brecht Dave Brennan Tom Bridgman Anthony Briggs diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -49,6 +49,12 @@ Library ------- +- Issue #17830: When keyword.py is used to update a keyword file, it now + preserves the line endings of the original file. + +- Issue #17272: Making the urllib.request's Request.full_url a descriptor. + Fixes bugs with assignment to full_url. Patch by Demian Brecht. + - Issue #17353: Plistlib emitted empty data tags with deeply nested datastructures - Issue #11714: Use 'with' statements to assure a Semaphore releases a -- Repository URL: http://hg.python.org/cpython

1 0

cpython: ftplib: update connect() docstring by mentioning 'timeout' argument
by giampaolo.rodola April 26, 2013

April 26, 2013

http://hg.python.org/cpython/rev/8f7bdfe81b76 changeset: 83522:8f7bdfe81b76 parent: 83518:de35eae9048a user: Giampaolo Rodola' <g.rodola(a)gmail.com> date: Fri Apr 26 15:23:43 2013 +0200 summary: ftplib: update connect() docstring by mentioning 'timeout' argument files: Lib/ftplib.py | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/Lib/ftplib.py b/Lib/ftplib.py --- a/Lib/ftplib.py +++ b/Lib/ftplib.py @@ -134,6 +134,7 @@ '''Connect to host. Arguments are: - host: hostname to connect to (string, default previous host) - port: port to connect to (integer, default previous port) + - timeout: the timeout to set against the ftp socket(s) - source_address: a 2-tuple (host, port) for the socket to bind to as its source address before connecting. ''' -- Repository URL: http://hg.python.org/cpython

1 0

peps: Added missing right paren.
by eric.smith April 26, 2013

April 26, 2013

http://hg.python.org/peps/rev/2e2e7178886e changeset: 4857:2e2e7178886e user: Eric V. Smith <eric(a)trueblade.com> date: Fri Apr 26 06:31:02 2013 -0400 summary: Added missing right paren. files: pep-3156.txt | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/pep-3156.txt b/pep-3156.txt --- a/pep-3156.txt +++ b/pep-3156.txt @@ -213,7 +213,7 @@ this PEP, where ``run()`` had different behavior, in part because there are already too many APIs that have a method named ``run()``, and in part because there shouldn't be many places where this is - called anyway. + called anyway.) - ``run_until_complete(future, timeout=None)``. Runs the event loop until the Future is done. If a timeout is given, it waits at most -- Repository URL: http://hg.python.org/peps

1 0

Daily reference leaks (58b0e301b78a): sum=2
by solipsis＠pitrou.net April 26, 2013

April 26, 2013

results for 58b0e301b78a on branch "default" -------------------------------------------- test_unittest leaked [-1, 1, 2] memory blocks, sum=2 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflogZIRPcK', '-x']

1 0

Cron <docs@dinsdale> /home/docs/build-devguide
by root＠python.org April 25, 2013

April 25, 2013

abort: error: Connection timed out

1 0

cpython: 17830: preserve line endings of original file when updating keywords.
by r.david.murray April 25, 2013

April 25, 2013

http://hg.python.org/cpython/rev/58b0e301b78a changeset: 83521:58b0e301b78a user: R David Murray <rdmurray(a)bitdance.com> date: Thu Apr 25 12:01:36 2013 -0400 summary: 17830: preserve line endings of original file when updating keywords. This fixes the test failures on Windows from the new tests, and includes test fixes as well as the module fix. files: Lib/keyword.py | 22 +++++---- Lib/test/test_keyword.py | 58 ++++++++++++++------------- Misc/NEWS | 3 + 3 files changed, 45 insertions(+), 38 deletions(-) diff --git a/Lib/keyword.py b/Lib/keyword.py --- a/Lib/keyword.py +++ b/Lib/keyword.py @@ -60,6 +60,12 @@ if len(args) > 1: optfile = args[1] else: optfile = "Lib/keyword.py" + # load the output skeleton from the target, taking care to preserve its + # newline convention. + with open(optfile, newline='') as fp: + format = fp.readlines() + nl = format[0][len(format[0].strip()):] if format else '\n' + # scan the source file for keywords with open(iptfile) as fp: strprog = re.compile('"([^"]+)"') @@ -68,25 +74,21 @@ if '{1, "' in line: match = strprog.search(line) if match: - lines.append(" '" + match.group(1) + "',\n") + lines.append(" '" + match.group(1) + "'," + nl) lines.sort() - # load the output skeleton from the target - with open(optfile) as fp: - format = fp.readlines() - - # insert the lines of keywords + # insert the lines of keywords into the skeleton try: - start = format.index("#--start keywords--\n") + 1 - end = format.index("#--end keywords--\n") + start = format.index("#--start keywords--" + nl) + 1 + end = format.index("#--end keywords--" + nl) format[start:end] = lines except ValueError: sys.stderr.write("target does not contain format markers\n") sys.exit(1) # write the output file - with open(optfile, 'w') as fp: - fp.write(''.join(format)) + with open(optfile, 'w', newline='') as fp: + fp.writelines(format) if __name__ == "__main__": main() diff --git a/Lib/test/test_keyword.py b/Lib/test/test_keyword.py --- a/Lib/test/test_keyword.py +++ b/Lib/test/test_keyword.py @@ -9,7 +9,8 @@ import textwrap KEYWORD_FILE = support.findfile('keyword.py') -GRAMMAR_FILE = os.path.join('..', '..', 'Python', 'graminit.c') +GRAMMAR_FILE = os.path.join(os.path.split(__file__)[0], + '..', '..', 'Python', 'graminit.c') TEST_PY_FILE = 'keyword_test.py' GRAMMAR_TEST_FILE = 'graminit_test.c' PY_FILE_WITHOUT_KEYWORDS = 'minimal_keyword.py' @@ -30,7 +31,7 @@ # preserved for backward compatibility. def test_changing_the_kwlist_does_not_affect_iskeyword(self): oldlist = keyword.kwlist - self.addCleanup(lambda: setattr(keyword, 'kwlist', oldlist)) + self.addCleanup(setattr, keyword, 'kwlist', oldlist) keyword.kwlist = ['its', 'all', 'eggs', 'beans', 'and', 'a', 'slice'] self.assertFalse(keyword.iskeyword('eggs')) @@ -38,11 +39,12 @@ class TestKeywordGeneration(unittest.TestCase): def _copy_file_without_generated_keywords(self, source_file, dest_file): - with open(source_file) as fp: + with open(source_file, 'rb') as fp: lines = fp.readlines() - with open(dest_file, 'w') as fp: - fp.writelines(lines[:lines.index("#--start keywords--\n") + 1]) - fp.writelines(lines[lines.index("#--end keywords--\n"):]) + nl = lines[0][len(lines[0].strip()):] + with open(dest_file, 'wb') as fp: + fp.writelines(lines[:lines.index(b"#--start keywords--" + nl) + 1]) + fp.writelines(lines[lines.index(b"#--end keywords--" + nl):]) def _generate_keywords(self, grammar_file, target_keyword_py_file): proc = subprocess.Popen([sys.executable, @@ -56,15 +58,15 @@ 'test only works from source build directory') def test_real_grammar_and_keyword_file(self): self._copy_file_without_generated_keywords(KEYWORD_FILE, TEST_PY_FILE) - self.addCleanup(lambda: support.unlink(TEST_PY_FILE)) + self.addCleanup(support.unlink, TEST_PY_FILE) self.assertFalse(filecmp.cmp(KEYWORD_FILE, TEST_PY_FILE)) - self.assertEqual(0, self._generate_keywords(GRAMMAR_FILE, - TEST_PY_FILE)[0]) + self.assertEqual((0, b''), self._generate_keywords(GRAMMAR_FILE, + TEST_PY_FILE)) self.assertTrue(filecmp.cmp(KEYWORD_FILE, TEST_PY_FILE)) def test_grammar(self): self._copy_file_without_generated_keywords(KEYWORD_FILE, TEST_PY_FILE) - self.addCleanup(lambda: support.unlink(TEST_PY_FILE)) + self.addCleanup(support.unlink, TEST_PY_FILE) with open(GRAMMAR_TEST_FILE, 'w') as fp: # Some of these are probably implementation accidents. fp.writelines(textwrap.dedent("""\ @@ -86,40 +88,40 @@ {1, 'no good'} {283, 0}, {1, "too many spaces"}""")) - self.addCleanup(lambda: support.unlink(GRAMMAR_TEST_FILE)) + self.addCleanup(support.unlink, GRAMMAR_TEST_FILE) self._generate_keywords(GRAMMAR_TEST_FILE, TEST_PY_FILE) expected = [ - " 'This one is tab indented',\n", - " 'also legal',\n", - " 'continue',\n", - " 'crazy but legal',\n", - " 'jello',\n", - " 'lemon',\n", - " 'tomato',\n", - " 'turnip',\n", - " 'wigii',\n", + " 'This one is tab indented',", + " 'also legal',", + " 'continue',", + " 'crazy but legal',", + " 'jello',", + " 'lemon',", + " 'tomato',", + " 'turnip',", + " 'wigii',", ] with open(TEST_PY_FILE) as fp: - lines = fp.readlines() - start = lines.index("#--start keywords--\n") + 1 - end = lines.index("#--end keywords--\n") + lines = fp.read().splitlines() + start = lines.index("#--start keywords--") + 1 + end = lines.index("#--end keywords--") actual = lines[start:end] self.assertEqual(actual, expected) def test_empty_grammar_results_in_no_keywords(self): self._copy_file_without_generated_keywords(KEYWORD_FILE, PY_FILE_WITHOUT_KEYWORDS) - self.addCleanup(lambda: support.unlink(PY_FILE_WITHOUT_KEYWORDS)) + self.addCleanup(support.unlink, PY_FILE_WITHOUT_KEYWORDS) shutil.copyfile(KEYWORD_FILE, TEST_PY_FILE) - self.addCleanup(lambda: support.unlink(TEST_PY_FILE)) - self.assertEqual(0, self._generate_keywords(os.devnull, - TEST_PY_FILE)[0]) + self.addCleanup(support.unlink, TEST_PY_FILE) + self.assertEqual((0, b''), self._generate_keywords(os.devnull, + TEST_PY_FILE)) self.assertTrue(filecmp.cmp(TEST_PY_FILE, PY_FILE_WITHOUT_KEYWORDS)) def test_keywords_py_without_markers_produces_error(self): rc, stderr = self._generate_keywords(os.devnull, os.devnull) self.assertNotEqual(rc, 0) - self.assertEqual(stderr, b'target does not contain format markers\n') + self.assertRegex(stderr, b'does not contain format markers') def test_missing_grammar_file_produces_error(self): rc, stderr = self._generate_keywords(NONEXISTENT_FILE, KEYWORD_FILE) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -49,6 +49,9 @@ Library ------- +- Issue #17830: When keyword.py is used to update a keyword file, it now + preserves the line endings of the original file. + - Issue #17272: Making the urllib.request's Request.full_url a descriptor. Fixes bugs with assignment to full_url. Patch by Demian Brecht. -- Repository URL: http://hg.python.org/cpython

1 0

cpython: Issue #17272: Making the urllib.request's Request.full_url a descriptor. Fixes
by senthil.kumaran April 25, 2013

April 25, 2013

http://hg.python.org/cpython/rev/2d4189e9bbe8 changeset: 83520:2d4189e9bbe8 parent: 83518:de35eae9048a user: Senthil Kumaran <senthil(a)uthcode.com> date: Thu Apr 25 05:45:48 2013 -0700 summary: Issue #17272: Making the urllib.request's Request.full_url a descriptor. Fixes bugs with assignment to full_url. Patch by Demian Brecht. files: Lib/test/test_urllib2.py | 24 ++++++++++++++++++++++++ Lib/urllib/request.py | 27 ++++++++++++++++++++------- Misc/ACKS | 1 + Misc/NEWS | 3 +++ 4 files changed, 48 insertions(+), 7 deletions(-) diff --git a/Lib/test/test_urllib2.py b/Lib/test/test_urllib2.py --- a/Lib/test/test_urllib2.py +++ b/Lib/test/test_urllib2.py @@ -904,6 +904,30 @@ p_ds_req = h.do_request_(ds_req) self.assertEqual(p_ds_req.unredirected_hdrs["Host"],"example.com") + def test_full_url_setter(self): + # Checks to ensure that components are set correctly after setting the + # full_url of a Request object + + urls = [ + 'http://example.com?foo=bar#baz', + 'http://example.com?foo=bar&spam=eggs#bash', + 'http://example.com', + ] + + # testing a reusable request instance, but the url parameter is + # required, so just use a dummy one to instantiate + r = Request('http://example.com') + for url in urls: + r.full_url = url + self.assertEqual(r.get_full_url(), url) + + def test_full_url_deleter(self): + r = Request('http://www.example.com') + del r.full_url + self.assertIsNone(r.full_url) + self.assertIsNone(r.fragment) + self.assertEqual(r.selector, '') + def test_fixpath_in_weirdurls(self): # Issue4493: urllib2 to supply '/' when to urls where path does not # start with'/' diff --git a/Lib/urllib/request.py b/Lib/urllib/request.py --- a/Lib/urllib/request.py +++ b/Lib/urllib/request.py @@ -259,9 +259,7 @@ def __init__(self, url, data=None, headers={}, origin_req_host=None, unverifiable=False, method=None): - # unwrap('<URL:type://host/path>') --> 'type://host/path' - self.full_url = unwrap(url) - self.full_url, self.fragment = splittag(self.full_url) + self.full_url = url self.headers = {} self.unredirected_hdrs = {} self._data = None @@ -274,8 +272,24 @@ self.origin_req_host = origin_req_host self.unverifiable = unverifiable self.method = method + + @property + def full_url(self): + return self._full_url + + @full_url.setter + def full_url(self, url): + # unwrap('<URL:type://host/path>') --> 'type://host/path' + self._full_url = unwrap(url) + self._full_url, self.fragment = splittag(self._full_url) self._parse() + @full_url.deleter + def full_url(self): + self._full_url = None + self.fragment = None + self.selector = '' + @property def data(self): return self._data @@ -295,7 +309,7 @@ self.data = None def _parse(self): - self.type, rest = splittype(self.full_url) + self.type, rest = splittype(self._full_url) if self.type is None: raise ValueError("unknown url type: %r" % self.full_url) self.host, self.selector = splithost(rest) @@ -313,9 +327,8 @@ def get_full_url(self): if self.fragment: - return '%s#%s' % (self.full_url, self.fragment) - else: - return self.full_url + return '{}#{}'.format(self.full_url, self.fragment) + return self.full_url def set_proxy(self, host, type): if self.type == 'https' and not self._tunnel_host: diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -153,6 +153,7 @@ Sven Brauch Erik Bray Brian Brazil +Demian Brecht Dave Brennan Tom Bridgman Anthony Briggs diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -49,6 +49,9 @@ Library ------- +- Issue #17272: Making the urllib.request's Request.full_url a descriptor. + Fixes bugs with assignment to full_url. Patch by Demian Brecht. + - Issue #17353: Plistlib emitted empty data tags with deeply nested datastructures - Issue #11714: Use 'with' statements to assure a Semaphore releases a -- Repository URL: http://hg.python.org/cpython

1 0

Daily reference leaks (de35eae9048a): sum=0
by solipsis＠pitrou.net April 25, 2013

April 25, 2013

results for de35eae9048a on branch "default" -------------------------------------------- Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflogFTtPNJ', '-x']

1 0