Mailman 3 July 2011 - Python-Dev

Re: [Python-Dev] cpython (3.2): Fix closes Issue12576 - fix urlopen behavior on sites which do not send (or
by Antoine Pitrou 27 Jul '11

27 Jul '11

On Wed, 27 Jul 2011 02:25:56 +0200 senthil.kumaran <python-checkins(a)python.org> wrote: > > + def test_sites_no_connection_close(self): > + # Some sites do not send Connection: close header. > + # Verify that those work properly. (#issue12576) > + > + try: > + with urllib.request.urlopen('http://www.imdb.com') as res: > + pass Can you please at least use support.transient_internet() as in other tests in this file? > + except ValueError as e: > + self.fail("urlopen failed for sites not sending Connection:close") > + else: > + self.assertTrue(res) > + > + req = urllib.request.urlopen('http://www.imdb.com') Why a second time? > + res = req.read() > + self.assertTrue(res) Also, when does "req" get closed? Right now I get resource warnings: test_sites_no_connection_close (test.test_urllib2net.OtherNetworkTests) ... /home/antoine/cpython/default/Lib/socket.py:342: ResourceWarning: unclosed <socket.socket object, fd=3, family=2, type=1, proto=6> self._sock = None /home/antoine/cpython/default/Lib/socket.py:342: ResourceWarning: unclosed <socket.socket object, fd=3, family=2, type=1, proto=6> self._sock = None ok Regards Antoine.

2 3

Re: [Python-Dev] [Python-checkins] cpython: Issue #11049: adding some tests to test.support
by Éric Araujo 27 Jul '11

27 Jul '11

Hi, > changeset: 71465:be558ad15789 > user: Eli Bendersky <eliben(a)gmail.com> > summary: > Issue #11049: adding some tests to test.support > Based on original patch by Giampaolo Rodola with contributions from R. David Murray > > files: > Lib/test/support.py | 21 +- > Lib/test/test_support.py | 178 +++++++++++++++++++++++++++ > 2 files changed, 189 insertions(+), 10 deletions(-) > > diff --git a/Lib/test/support.py b/Lib/test/support.py > --- a/Lib/test/support.py > +++ b/Lib/test/support.py > @@ -170,7 +170,7 @@ > attribute = getattr(obj, name) > except AttributeError: > raise unittest.SkipTest("module %s has no attribute %s" % ( > - obj.__name__, name)) > + repr(obj), name)) I would use %r instead of %s for both fields here. Non-ASCII characters and unseen whitespace are at least two reasons to overuse %r in debug/error messages instead of %s. Regards

6 10

Comments of the PEP 3151
by Victor Stinner 27 Jul '11

27 Jul '11

Arguments in favor of specific errors ------------------------------------- Using specific errors avoids the need of "import errno". The import is sometimes done in the except block, which may raise a new error (and may be slow). I like specific exceptions because it avoids the need of re-raise unwanted exception. It makes the code more readable: we ignore explicitly specific errors instead of re-raised unwanted exceptions and implicitly ignore some errors. For example: try: os.mkdir(dir) except OSError as err: if err.errno != errno.EEXIST: raise becomes try: os.mkdir(dir) except FileExistsError: pass By the way, is it faster to not handle and than re-raise unwanted exceptions? (I don't like tests like "err.errno != errno.EEXIST", it's confusing to me. It uses errno identifier twice: once as an attribute, once as a module name.) Make specific errors builtins ----------------------------- I agree that it's better to make specific errors builtins. If we move exceptions to more specific modules, like io and socket, you have to load these modules to catch an exception and remember in which module the exception is. An advantage of builtin specific exceptions is to avoid an import (import errno). Making them builtin may also avoid a bootstrap issue (catch a specific error at startup). Choice of the specific errors ----------------------------- I don't understand how Antoine decided which errno should have an exception or not. For example, EINTR and EPIPE are handled in many modules of the standard library but don't have their exception, whereas EISDIR (IsADirectoryError) and ENOTDIR (NotADirectoryError) are very rare (EISDIR is never handled in the standard library) but have their exception. If we add EINTR, I don't know if it's better to add it to BlockingIOError or to create a new exception (InterruptError?). Another good candidate is EINVAL. Would it be possible to have an (exhaustive?) list of errno with their popularity? At least, their popularity in the Python standard library? Raise a specific error ---------------------- Does raising a specific error (e.g. raise FileNotFound()) set errno and message attributes (to something different than None)? If we provide an error message error: should it be localized? The description of FileDescriptorError tells about the "default error message". It we use a localized message, it's not possible to preallocate or cache instances. I don't see how we could choose a default errno value, because some exceptions handle multiple errno. For example, PermissionError.errno can be EACCES or EPERM. Do specific errors slows down raising exceptions? Do raising an IOError (or a "legacy" error) use a O(1) lookup to choose the exception class? PermissionError --------------- EACCES and EPERM have a different meaning. Even that they are very similar, we might provide two different exceptions. EPERM is only used once in the Python stdlib, so we might only provide AccesError. On Linux, EPERM usually indicates an operation requiring root priviledge. EACCES can be related to filesystem permissions (read-only, user is not allowed to write, etc.) or can be an mmap error. Deprecation ----------- Because IOError, OSError, select.error, etc. are well known exceptions, I'm not in favor of removing them from Python 3. It would make porting from Python 2 worse. If we don't remove them, they should not be deprecated. I'm in favor of adding a note in the documentation of all legacy exceptions to advice to use IOError or specific exceptions instead. I suppose that these notes will have to indicate a Python version. Test the implementation ----------------------- Did someone test Blender, Django or another major applications on the implementation of the PEP 3151? -1 on FileSystemError --------------------- I'm not sure that we need FileSystemError or ConnectionError. Careless code will use IOError, whereas careful code will use an explicit list like (ConnectionAbortedError, ConnectionRefusedError, ConnectionResetError). If we remove IsADirectoryError and NotADirectoryError, FileSystemError only contains FileExistsError and FileNotFoundError. I don't think that these errors can occur on a same function. For example, rmdir() only raises ENOTDIR. I only see one advantage of FileSystemError: it doesn't handle FileDescriptorError. Advice usage of FileSystemError would avoid to hide real bugs like FileDescriptorError. I don't really care of ConnectionError. Anyway, FileSystemError and ConnectionError can be added later if needed. WindowsError and winerror attribute ----------------------------------- I don't like the idea of having a winerror attribute platforms other than Windows. Why not using hasattr(err, "winerror") instead? WindowsError is already specific to Windows. I don't see any usage of the winerror attribute in the Python stdlib. shutil module uses: try: WindowsError except NameError: WindowsError = None (...) try: copystat(src, dst) except OSError as why: if WindowsError is not None and isinstance(why, WindowsError): # Copying file access times may fail on Windows pass else: errors.extend((src, dst, str(why))) Misc ---- Lock.acquire() doesn't raise an exception on timeout: just remove "(for example in Lock.acquire())". Should FileNotFound handle ENODEV? (see test_ossaudiodev) Victor

9 19

Re: [Python-Dev] [Python-checkins] cpython (merge 3.2 -> default): Issue #12102: Merge with 3.2.
by Éric Araujo 27 Jul '11

27 Jul '11

Hi, > changeset: 71497:4898b14dcd69 > user: Ross Lagerwall <rosslagerwall(a)gmail.com> > summary: > Issue #12102: Merge with 3.2. > > files: > Doc/ACKS.txt | 1 + > Doc/library/mmap.rst | 6 ++++++ > Misc/NEWS | 3 +++ > diff --git a/Misc/NEWS b/Misc/NEWS > --- a/Misc/NEWS > +++ b/Misc/NEWS > @@ -237,6 +237,9 @@ > Library > ------- > > +- Issue #12102: Document that buffered files must be flushed before being used > + with mmap. Patch by Steffen Daode Nurpmeso. We don’t add NEWS entries for each and every doc fix, otherwise it would be very huge :) We rather document large changes to the documentation, like adding links to the source files, using “python3” instead of “python” in all examples, etc. In addition, Library is the wrong section, it should be Documentation. I’m doing commits this afternoon, so I’ll take the occasion to remove this entry. Cheers

4 5

Re: [Python-Dev] [Python-checkins] cpython (2.7): Issue #8746: Correct faulty configure checks so that os.chflags() and
by Éric Araujo 27 Jul '11

27 Jul '11

Hi, > changeset: 71030:abfe28e7e5cd > branch: 2.7 > user: Ned Deily <nad(a)acm.org> > diff --git a/Lib/test/test_posix.py b/Lib/test/test_posix.py > --- a/Lib/test/test_posix.py > +++ b/Lib/test/test_posix.py > @@ -11,10 +11,12 @@ > import os > import pwd > import shutil > +import stat > import sys > import unittest > import warnings > > +_DUMMY_SYMLINK = '%s/dummy-symlink' % os.getenv('TMPDIR', '/tmp') Why not let the tempfile module do this for you? Regards

2 1

Re: [Python-Dev] [Python-checkins] cpython: Issue #11784: Improve multiprocessing.Process.join() documentation. Patch by
by Éric Araujo 26 Jul '11

26 Jul '11

> changeset: 71499:8d67fd820627 > parent: 71497:4898b14dcd69 > user: Charles-François Natali <neologix(a)free.fr> > date: Mon Jul 25 18:35:49 2011 +0200 > summary: > Issue #11784: Improve multiprocessing.Process.join() documentation. Patch by > Patrick Sabin. > > files: > Doc/library/multiprocessing.rst | 7 +++---- > Misc/ACKS | 1 + There’s a dedicated file to thank doc contributors: Doc/ACKS.rst Regards

2 1

is sys.modules not meant to be replaced?
by Eric Snow 25 Jul '11

25 Jul '11

The documentation[1] doesn't say, but the implementation of the imp module makes me wonder if sys.modules was not meant to be replaceable. No doubt this has to do with its tie to the interpreter's modules dict. I ran into this doing "sys.modules = sys.modules.copy()" to protect the actual sys.modules dict during some import related test cases. If the modules I imported were extension modules it broke. So, is sys.modules not meant to be open to re-binding? -eric p.s. I tried opening a tracker ticket on this, but it wouldn't go through. I'll try again later. [1] http://docs.python.org/library/sys.html#sys.modules

5 8

3.2.1 encoding surprise
by Glenn Linderman 24 Jul '11

24 Jul '11

Attached reduced test case works fine with Python 3.1, fails with Python3.2: SyntaxError: Non-ASCII character '\xc3' in file D:\my\py\t32enc.py on line 1, but no encoding declared; see http://www.python.org/peps/pep-0263.html for details I'm familiar with the PEP, but thought 3.x cured that. Python 3.1 produces no error; I'm not sure that 3.2 did or didn't report such problems, or if the programs I ran with 3.2 simply didn't contain non-ASCII characters. The file is UTF-8 encoded with no pseudo-BOM. Is this intentional, or a regression (in which case I will open a ticket)? If a regression, does that mean we have no tests for it, possibly because most of the tests contain the PEP 263 encoding directive, because most people using 3.x are still also using 2.x (I am not)?

11 41

tuple[int] optimization
by Andrew Svetlov 24 Jul '11

24 Jul '11

tuple[int] is 30% slower than list[int]. Please let me explain the problem. (1, 2, 3)[1] has compiled as BINARY_SUBSCR operation. ceval.c has optimization for list: case BINARY_SUBSCR: w = POP(); v = TOP(); if (PyList_CheckExact(v) && PyInt_CheckExact(w)) { /* INLINE: list[int] */ Py_ssize_t i = PyInt_AsSsize_t(w); if (i < 0) i += PyList_GET_SIZE(v); if (i >= 0 && i < PyList_GET_SIZE(v)) { x = PyList_GET_ITEM(v, i); Py_INCREF(x); } else goto slow_get; } else slow_get: x = PyObject_GetItem(v, w); Py_DECREF(v); Py_DECREF(w); SET_TOP(x); if (x != NULL) continue; break; For tuple code follows slow way via tp_as_mapping slot and calls tuplesubscript from Objects/tupleobject.c Looks like tuple is goot applicant to be optimized as list does. tuples is used in python programs very often and getting element from tuple by index as common operation as list[int]. Is there reasons to prevent special case for tuples in BINARY_SUBSCR?

4 6

Re: [Python-Dev] cpython: Merge - Issue #12592: Make Python build on OpenBSD 5 (and future major
by Georg Brandl 23 Jul '11

23 Jul '11

Am 22.07.2011 23:51, schrieb charles-francois.natali: > http://hg.python.org/cpython/rev/63de97ae832e > changeset: 71462:63de97ae832e > parent: 71459:d3f0f72c31f8 > user: Charles-François Natali <neologix(a)free.fr> > date: Fri Jul 22 23:52:02 2011 +0200 > summary: > Merge - Issue #12592: Make Python build on OpenBSD 5 (and future major > releases). > > files: > Misc/NEWS | 2 + > configure | 597 ++++++++++++++++++-------------------- > configure.in | 4 +- > 3 files changed, 291 insertions(+), 312 deletions(-) Note that this commit wasn't actually a merge -- you'll have to use the "hg merge" command for that. Georg

3 5