Mailman 3 November 2010 - Python-checkins

r86817 - python/branches/py3k-stat-on-windows/Lib/test/test_shutil.py
by hirokazu.yamamoto 02 Dec '10

02 Dec '10

Author: hirokazu.yamamoto Date: Fri Nov 26 19:44:28 2010 New Revision: 86817 Log: Now can reproduce the error on AMD64 Windows Server 2008 even where os.symlink is not supported. Modified: python/branches/py3k-stat-on-windows/Lib/test/test_shutil.py Modified: python/branches/py3k-stat-on-windows/Lib/test/test_shutil.py ============================================================================== --- python/branches/py3k-stat-on-windows/Lib/test/test_shutil.py (original) +++ python/branches/py3k-stat-on-windows/Lib/test/test_shutil.py Fri Nov 26 19:44:28 2010 @@ -271,24 +271,32 @@ shutil.rmtree(src_dir) shutil.rmtree(os.path.dirname(dst_dir)) - @support.skip_unless_symlink + @unittest.skipUnless(hasattr(os, 'link'), 'requires os.link') def test_dont_copy_file_onto_link_to_itself(self): # bug 851123. os.mkdir(TESTFN) src = os.path.join(TESTFN, 'cheese') dst = os.path.join(TESTFN, 'shop') try: - f = open(src, 'w') - f.write('cheddar') - f.close() - - if hasattr(os, "link"): - os.link(src, dst) - self.assertRaises(shutil.Error, shutil.copyfile, src, dst) - with open(src, 'r') as f: - self.assertEqual(f.read(), 'cheddar') - os.remove(dst) + with open(src, 'w') as f: + f.write('cheddar') + os.link(src, dst) + self.assertRaises(shutil.Error, shutil.copyfile, src, dst) + with open(src, 'r') as f: + self.assertEqual(f.read(), 'cheddar') + os.remove(dst) + finally: + shutil.rmtree(TESTFN, ignore_errors=True) + @support.skip_unless_symlink + def test_dont_copy_file_onto_symlink_to_itself(self): + # bug 851123. + os.mkdir(TESTFN) + src = os.path.join(TESTFN, 'cheese') + dst = os.path.join(TESTFN, 'shop') + try: + with open(src, 'w') as f: + f.write('cheddar') # Using `src` here would mean we end up with a symlink pointing # to TESTFN/TESTFN/cheese, while it should point at # TESTFN/cheese. @@ -298,10 +306,7 @@ self.assertEqual(f.read(), 'cheddar') os.remove(dst) finally: - try: - shutil.rmtree(TESTFN) - except OSError: - pass + shutil.rmtree(TESTFN, ignore_errors=True) @support.skip_unless_symlink def test_rmtree_on_symlink(self):

3 6

r86906 - python/branches/py3k/Lib/test/test_os.py
by brian.curtin 30 Nov '10

30 Nov '10

Author: brian.curtin Date: Wed Dec 1 00:46:54 2010 New Revision: 86906 Log: Fix #10591. Fix test_os for refleak runs. Split a common setUp/tearDown into the appropriate parts. Modified: python/branches/py3k/Lib/test/test_os.py Modified: python/branches/py3k/Lib/test/test_os.py ============================================================================== --- python/branches/py3k/Lib/test/test_os.py (original) +++ python/branches/py3k/Lib/test/test_os.py Wed Dec 1 00:46:54 2010 @@ -866,12 +866,11 @@ self.file1 = support.TESTFN self.file2 = os.path.join(support.TESTFN + "2") + def tearDown(self): for file in (self.file1, self.file2): if os.path.exists(file): os.unlink(file) - tearDown = setUp - def _test_link(self, file1, file2): with open(file1, "w") as f1: f1.write("test")

1 0

r86905 - in python/branches/py3k: Misc/NEWS Objects/setobject.c
by antoine.pitrou 30 Nov '10

30 Nov '10

Author: antoine.pitrou Date: Tue Nov 30 23:23:20 2010 New Revision: 86905 Log: Issue #8685: Speed up set difference `a - b` when source set `a` is much larger than operand `b`. Patch by Andrew Bennetts. Modified: python/branches/py3k/Misc/NEWS python/branches/py3k/Objects/setobject.c Modified: python/branches/py3k/Misc/NEWS ============================================================================== --- python/branches/py3k/Misc/NEWS (original) +++ python/branches/py3k/Misc/NEWS Tue Nov 30 23:23:20 2010 @@ -10,6 +10,9 @@ Core and Builtins ----------------- +- Issue #8685: Speed up set difference ``a - b`` when source set ``a`` is + much larger than operand ``b``. Patch by Andrew Bennetts. + - Issue #10518: Bring back the callable() builtin. - Issue #8879. Add os.link support for Windows. Modified: python/branches/py3k/Objects/setobject.c ============================================================================== --- python/branches/py3k/Objects/setobject.c (original) +++ python/branches/py3k/Objects/setobject.c Tue Nov 30 23:23:20 2010 @@ -1525,6 +1525,20 @@ "Remove all elements of another set from this set."); static PyObject * +set_copy_and_difference(PySetObject *so, PyObject *other) +{ + PyObject *result; + + result = set_copy(so); + if (result == NULL) + return NULL; + if (set_difference_update_internal((PySetObject *) result, other) != -1) + return result; + Py_DECREF(result); + return NULL; +} + +static PyObject * set_difference(PySetObject *so, PyObject *other) { PyObject *result; @@ -1532,13 +1546,13 @@ Py_ssize_t pos = 0; if (!PyAnySet_Check(other) && !PyDict_CheckExact(other)) { - result = set_copy(so); - if (result == NULL) - return NULL; - if (set_difference_update_internal((PySetObject *)result, other) != -1) - return result; - Py_DECREF(result); - return NULL; + return set_copy_and_difference(so, other); + } + + /* If len(so) much more than len(other), it's more efficient to simply copy + * so and then iterate other looking for common elements. */ + if ((PySet_GET_SIZE(so) >> 2) > PyObject_Size(other)) { + return set_copy_and_difference(so, other); } result = make_new_set_basetype(Py_TYPE(so), NULL); @@ -1560,6 +1574,7 @@ return result; } + /* Iterate over so, checking for common elements in other. */ while (set_next(so, &pos, &entry)) { int rv = set_contains_entry((PySetObject *)other, entry); if (rv == -1) {

1 0

r86904 - peps/trunk/pep-0291.txt
by barry.warsaw 30 Nov '10

30 Nov '10

Author: barry.warsaw Date: Tue Nov 30 22:05:26 2010 New Revision: 86904 Log: Make it clear that PEP 291 covers only Python 2, and mark it final (now that Python 2.7 - the last in the Python 2 line - has been released). Modified: peps/trunk/pep-0291.txt Modified: peps/trunk/pep-0291.txt ============================================================================== --- peps/trunk/pep-0291.txt (original) +++ peps/trunk/pep-0291.txt Tue Nov 30 22:05:26 2010 @@ -1,9 +1,9 @@ PEP: 291 -Title: Backward Compatibility for Standard Library +Title: Backward Compatibility for the Python 2 Standard Library Version: $Revision$ Last-Modified: $Date$ Author: nnorwitz(a)gmail.com (Neal Norwitz) -Status: Active +Status: Final Type: Informational Created: 06-Jun-2002 Python-Version: 2.3 @@ -12,11 +12,13 @@ Abstract - This PEP describes the packages and modules in the standard - library which should remain backward compatible with previous - versions of Python. If a package is not listed here, then it need - only remain compatible with the version of Python it is - distributed with. + This PEP describes the packages and modules in the Python 2 + standard library which should remain backward compatible with + previous versions of Python. If a package is not listed here, + then it need only remain compatible with the version of Python it + is distributed with. + + This PEP has no bearing on the Python 3 standard library. Rationale

1 0

r86903 - python/branches/py3k/Lib/decimal.py
by raymond.hettinger 30 Nov '10

30 Nov '10

Author: raymond.hettinger Date: Tue Nov 30 21:32:59 2010 New Revision: 86903 Log: Add link to specification. Modified: python/branches/py3k/Lib/decimal.py Modified: python/branches/py3k/Lib/decimal.py ============================================================================== --- python/branches/py3k/Lib/decimal.py (original) +++ python/branches/py3k/Lib/decimal.py Tue Nov 30 21:32:59 2010 @@ -132,6 +132,7 @@ ] __version__ = '1.70' # Highest version of the spec this complies with + # See http://speleotrove.com/decimal/ import copy as _copy import math as _math

1 0

r86902 - python/branches/py3k/Doc/library/functools.rst
by raymond.hettinger 30 Nov '10

30 Nov '10

Author: raymond.hettinger Date: Tue Nov 30 21:02:57 2010 New Revision: 86902 Log: Documentation nits. Modified: python/branches/py3k/Doc/library/functools.rst Modified: python/branches/py3k/Doc/library/functools.rst ============================================================================== --- python/branches/py3k/Doc/library/functools.rst (original) +++ python/branches/py3k/Doc/library/functools.rst Tue Nov 30 21:02:57 2010 @@ -63,16 +63,16 @@ :attr:`__wrapped__` attribute. This is useful for introspection, for bypassing the cache, or for rewrapping the function with a different cache. - A `LRU (least recently used) cache + An `LRU (least recently used) cache <http://en.wikipedia.org/wiki/Cache_algorithms#Least_Recently_Used>`_ works best when more recent calls are the best predictors of upcoming calls (for example, the most popular articles on a news server tend to change daily). The cache's size limit assures that the cache does not grow without bound on long-running processes such as web servers. - Example -- Caching static web content:: + Example of an LRU cache for static web content:: - @functools.lru_cache(maxsize=20) + @lru_cache(maxsize=20) def get_pep(num): 'Retrieve text of a Python Enhancement Proposal' resource = 'http://www.python.org/dev/peps/pep-%04d/' % num

1 0

r86901 - in python/branches/py3k: Doc/library/functools.rst Doc/whatsnew/3.2.rst Lib/functools.py Lib/test/test_functools.py
by raymond.hettinger 30 Nov '10

30 Nov '10

Author: raymond.hettinger Date: Tue Nov 30 20:15:45 2010 New Revision: 86901 Log: Add example, tighten text, and minor clean-ups. Modified: python/branches/py3k/Doc/library/functools.rst python/branches/py3k/Doc/whatsnew/3.2.rst python/branches/py3k/Lib/functools.py python/branches/py3k/Lib/test/test_functools.py Modified: python/branches/py3k/Doc/library/functools.rst ============================================================================== --- python/branches/py3k/Doc/library/functools.rst (original) +++ python/branches/py3k/Doc/library/functools.rst Tue Nov 30 20:15:45 2010 @@ -42,40 +42,52 @@ .. versionadded:: 3.2 -.. decorator:: lru_cache(maxsize) +.. decorator:: lru_cache(maxsize=100) Decorator to wrap a function with a memoizing callable that saves up to the *maxsize* most recent calls. It can save time when an expensive or I/O bound function is periodically called with the same arguments. - The *maxsize* parameter defaults to 100. Since a dictionary is used to cache - results, the positional and keyword arguments to the function must be - hashable. - - The wrapped function is instrumented with a :attr:`cache_info` attribute that - can be called to retrieve a named tuple with the following fields: - - - :attr:`maxsize`: maximum cache size (as set by the *maxsize* parameter) - - :attr:`size`: current number of entries in the cache - - :attr:`hits`: number of successful cache lookups - - :attr:`misses`: number of unsuccessful cache lookups. + Since a dictionary is used to cache results, the positional and keyword + arguments to the function must be hashable. - These statistics are helpful for tuning the *maxsize* parameter and for measuring - the effectiveness of the cache. + To help measure the effectiveness of the cache and tune the *maxsize* + parameter, the wrapped function is instrumented with a :func:`cache_info` + function that returns a :term:`named tuple` showing *hits*, *misses*, + *maxsize* and *currsize*. - The wrapped function also has a :attr:`cache_clear` attribute which can be - called (with no arguments) to clear the cache. + The decorator also provides a :func:`cache_clear` function for clearing or + invalidating the cache. The original underlying function is accessible through the - :attr:`__wrapped__` attribute. This allows introspection, bypassing - the cache, or rewrapping the function with a different caching tool. + :attr:`__wrapped__` attribute. This is useful for introspection, for + bypassing the cache, or for rewrapping the function with a different cache. A `LRU (least recently used) cache - <http://en.wikipedia.org/wiki/Cache_algorithms#Least_Recently_Used>`_ - works best when more recent calls are the best predictors of upcoming calls - (for example, the most popular articles on a news server tend to - change each day). The cache's size limit assurs that caching does not - grow without bound on long-running processes such as web servers. + <http://en.wikipedia.org/wiki/Cache_algorithms#Least_Recently_Used>`_ works + best when more recent calls are the best predictors of upcoming calls (for + example, the most popular articles on a news server tend to change daily). + The cache's size limit assures that the cache does not grow without bound on + long-running processes such as web servers. + + Example -- Caching static web content:: + + @functools.lru_cache(maxsize=20) + def get_pep(num): + 'Retrieve text of a Python Enhancement Proposal' + resource = 'http://www.python.org/dev/peps/pep-%04d/' % num + try: + with urllib.request.urlopen(resource) as s: + return s.read() + except urllib.error.HTTPError: + return 'Not Found' + + >>> for n in 8, 290, 308, 320, 8, 218, 320, 279, 289, 320, 9991: + ... pep = get_pep(n) + ... print(n, len(pep)) + + >>> print(get_pep.cache_info()) + CacheInfo(hits=3, misses=8, maxsize=20, currsize=8) .. versionadded:: 3.2 Modified: python/branches/py3k/Doc/whatsnew/3.2.rst ============================================================================== --- python/branches/py3k/Doc/whatsnew/3.2.rst (original) +++ python/branches/py3k/Doc/whatsnew/3.2.rst Tue Nov 30 20:15:45 2010 @@ -332,13 +332,14 @@ c.execute('SELECT phonenumber FROM phonelist WHERE name=?', (name,)) return c.fetchone()[0] + >>> for name in user_requests: + ... get_phone_number(name) # cached lookup + To help with choosing an effective cache size, the wrapped function is - instrumented with info function: + instrumented for tracking cache statistics: - >>> for name in user_requests: - ... get_phone_number(name) >>> get_phone_number.cache_info() - CacheInfo(maxsize=300, size=300, hits=4805, misses=980) + CacheInfo(hits=4805, misses=980, maxsize=300, currsize=300) If the phonelist table gets updated, the outdated contents of the cache can be cleared with: Modified: python/branches/py3k/Lib/functools.py ============================================================================== --- python/branches/py3k/Lib/functools.py (original) +++ python/branches/py3k/Lib/functools.py Tue Nov 30 20:15:45 2010 @@ -114,7 +114,7 @@ raise TypeError('hash not implemented') return K -_CacheInfo = namedtuple("CacheInfo", "maxsize, size, hits, misses") +_CacheInfo = namedtuple("CacheInfo", "hits misses maxsize currsize") def lru_cache(maxsize=100): """Least-recently-used cache decorator. @@ -166,7 +166,7 @@ def cache_info(): """Report cache statistics""" with lock: - return _CacheInfo(maxsize, len(cache), hits, misses) + return _CacheInfo(hits, misses, maxsize, len(cache)) def cache_clear(): """Clear the cache and cache statistics""" Modified: python/branches/py3k/Lib/test/test_functools.py ============================================================================== --- python/branches/py3k/Lib/test/test_functools.py (original) +++ python/branches/py3k/Lib/test/test_functools.py Tue Nov 30 20:15:45 2010 @@ -501,7 +501,7 @@ def orig(x, y): return 3*x+y f = functools.lru_cache(maxsize=20)(orig) - maxsize, currsize, hits, misses = f.cache_info() + hits, misses, maxsize, currsize = f.cache_info() self.assertEqual(maxsize, 20) self.assertEqual(currsize, 0) self.assertEqual(hits, 0) @@ -513,18 +513,18 @@ actual = f(x, y) expected = orig(x, y) self.assertEqual(actual, expected) - maxsize, currsize, hits, misses = f.cache_info() + hits, misses, maxsize, currsize = f.cache_info() self.assertTrue(hits > misses) self.assertEqual(hits + misses, 1000) self.assertEqual(currsize, 20) f.cache_clear() # test clearing - maxsize, currsize, hits, misses = f.cache_info() + hits, misses, maxsize, currsize = f.cache_info() self.assertEqual(hits, 0) self.assertEqual(misses, 0) self.assertEqual(currsize, 0) f(x, y) - maxsize, currsize, hits, misses = f.cache_info() + hits, misses, maxsize, currsize = f.cache_info() self.assertEqual(hits, 0) self.assertEqual(misses, 1) self.assertEqual(currsize, 1) @@ -532,7 +532,7 @@ # Test bypassing the cache self.assertIs(f.__wrapped__, orig) f.__wrapped__(x, y) - maxsize, currsize, hits, misses = f.cache_info() + hits, misses, maxsize, currsize = f.cache_info() self.assertEqual(hits, 0) self.assertEqual(misses, 1) self.assertEqual(currsize, 1) @@ -548,7 +548,7 @@ for i in range(5): self.assertEqual(f(), 20) self.assertEqual(f_cnt, 5) - maxsize, currsize, hits, misses = f.cache_info() + hits, misses, maxsize, currsize = f.cache_info() self.assertEqual(hits, 0) self.assertEqual(misses, 5) self.assertEqual(currsize, 0) @@ -564,7 +564,7 @@ for i in range(5): self.assertEqual(f(), 20) self.assertEqual(f_cnt, 1) - maxsize, currsize, hits, misses = f.cache_info() + hits, misses, maxsize, currsize = f.cache_info() self.assertEqual(hits, 4) self.assertEqual(misses, 1) self.assertEqual(currsize, 1) @@ -581,7 +581,7 @@ # * * * * self.assertEqual(f(x), x*10) self.assertEqual(f_cnt, 4) - maxsize, currsize, hits, misses = f.cache_info() + hits, misses, maxsize, currsize = f.cache_info() self.assertEqual(hits, 12) self.assertEqual(misses, 4) self.assertEqual(currsize, 2)

1 0

r86900 - in python/branches/release27-maint: Doc/library/io.rst
by daniel.stutzbach 30 Nov '10

30 Nov '10

Author: daniel.stutzbach Date: Tue Nov 30 18:58:13 2010 New Revision: 86900 Log: Merged revisions 86896 via svnmerge from svn+ssh://pythondev@svn.python.org/python/branches/py3k ........ r86896 | daniel.stutzbach | 2010-11-30 09:49:53 -0800 (Tue, 30 Nov 2010) | 1 line Fix typo: "ofbytes" should be "of bytes" ........ Modified: python/branches/release27-maint/ (props changed) python/branches/release27-maint/Doc/library/io.rst Modified: python/branches/release27-maint/Doc/library/io.rst ============================================================================== --- python/branches/release27-maint/Doc/library/io.rst (original) +++ python/branches/release27-maint/Doc/library/io.rst Tue Nov 30 18:58:13 2010 @@ -371,9 +371,9 @@ .. method:: readinto(b) - Read up to len(b) bytes into bytearray *b* and return the number ofbytes - read. If the object is in non-blocking mode and no bytes are available, - ``None`` is returned. + Read up to len(b) bytes into bytearray *b* and return the number + of bytes read. If the object is in non-blocking mode and no + bytes are available, ``None`` is returned. .. method:: write(b)

1 0

r86899 - in python/branches/release31-maint: Doc/library/io.rst
by daniel.stutzbach 30 Nov '10

30 Nov '10

Author: daniel.stutzbach Date: Tue Nov 30 18:54:31 2010 New Revision: 86899 Log: Merged revisions 86896 via svnmerge from svn+ssh://pythondev@svn.python.org/python/branches/py3k ........ r86896 | daniel.stutzbach | 2010-11-30 09:49:53 -0800 (Tue, 30 Nov 2010) | 1 line Fix typo: "ofbytes" should be "of bytes" ........ Modified: python/branches/release31-maint/ (props changed) python/branches/release31-maint/Doc/library/io.rst Modified: python/branches/release31-maint/Doc/library/io.rst ============================================================================== --- python/branches/release31-maint/Doc/library/io.rst (original) +++ python/branches/release31-maint/Doc/library/io.rst Tue Nov 30 18:54:31 2010 @@ -361,9 +361,9 @@ .. method:: readinto(b) - Read up to len(b) bytes into bytearray *b* and return the number ofbytes - read. If the object is in non-blocking mode and no bytes are available, - ``None`` is returned. + Read up to len(b) bytes into bytearray *b* and return the number + of bytes read. If the object is in non-blocking mode and no + bytes are available, ``None`` is returned. .. method:: write(b)

1 0

r86898 - in python/branches/release27-maint: Doc/library/os.rst Doc/library/parser.rst
by eric.araujo 30 Nov '10

30 Nov '10

Author: eric.araujo Date: Tue Nov 30 18:53:45 2010 New Revision: 86898 Log: Merged revisions 86892 via svnmerge from svn+ssh://pythondev@svn.python.org/python/branches/py3k ........ r86892 | eric.araujo | 2010-11-30 18:20:31 +0100 (mar., 30 nov. 2010) | 2 lines Let’s keep “throw” for the generator method and use “raise” elsewhere. ........ Modified: python/branches/release27-maint/ (props changed) python/branches/release27-maint/Doc/library/os.rst python/branches/release27-maint/Doc/library/parser.rst Modified: python/branches/release27-maint/Doc/library/os.rst ============================================================================== --- python/branches/release27-maint/Doc/library/os.rst (original) +++ python/branches/release27-maint/Doc/library/os.rst Tue Nov 30 18:53:45 2010 @@ -1191,7 +1191,7 @@ single: UNC paths; and os.makedirs() Recursive directory creation function. Like :func:`mkdir`, but makes all - intermediate-level directories needed to contain the leaf directory. Throws an + intermediate-level directories needed to contain the leaf directory. Raises an :exc:`error` exception if the leaf directory already exists or cannot be created. The default *mode* is ``0777`` (octal). On some systems, *mode* is ignored. Where it is used, the current umask value is first masked out. Modified: python/branches/release27-maint/Doc/library/parser.rst ============================================================================== --- python/branches/release27-maint/Doc/library/parser.rst (original) +++ python/branches/release27-maint/Doc/library/parser.rst Tue Nov 30 18:53:45 2010 @@ -273,7 +273,7 @@ will only need to be aware of the simple string values. Note that the functions :func:`compilest`, :func:`expr`, and :func:`suite` may -raise exceptions which are normally thrown by the parsing and compilation +raise exceptions which are normally raised by the parsing and compilation process. These include the built in exceptions :exc:`MemoryError`, :exc:`OverflowError`, :exc:`SyntaxError`, and :exc:`SystemError`. In these cases, these exceptions carry all the meaning normally associated with them.

1 0