Mailman 3 March 2013 - Python-Dev

IDLE in the stdlib
by Kurt B. Kaiser March 20, 2013

March 20, 2013

I apologize for that formatting mess - the Barracuda rejected my original email for some reason (squirrelmail directly from shore.net) and I had to resend it from rejects - it got re-wrapped on transmission. -- KBK

1 0

Recent changes to TextIOWrapper and its tests
by Jeff Allen March 20, 2013

March 20, 2013

I'm pulling recent changes in the io module across to Jython. I am looking for help understanding the changes in http://hg.python.org/cpython/rev/19a33ef3821d That change set is about what should happen if the underlying buffer does not return bytes when read, but instead, for example, unicode characters. The test test_read_nonbytes() constructs a pathological text stream reader t where the usual BytesIO or BufferedReader is replaced with a StringIO. It then checks that r.read(1) and t.… [View More]

2 3

PyCon Sprints - Thank you
by Daniel Wozniak March 20, 2013

March 20, 2013

David and Senthil, I won't make it to the sprints today because my ride wants to go into San Francisco to do touristy things. I'll be flying back to Arizona this evening. I still have a fair amount of code that has not been submitted to the issue tracker. I will sprint from my house tomorrow and will submit whatever else I have ready tomorrow night. In the future, I'd also like to continue contributing by continuing to improve test coverage since that is helping me get familiar with both … [View More]

3 2

Re: [Python-Dev] cpython: Closes issue 17467. Add readline and readlines support to
by Antoine Pitrou March 20, 2013

March 20, 2013

On Wed, 20 Mar 2013 01:22:58 +0100 (CET) michael.foord <python-checkins(a)python.org> wrote: > http://hg.python.org/cpython/rev/684b75600fa9 > changeset: 82811:684b75600fa9 > user: Michael Foord <michael(a)voidspace.org.uk> > date: Tue Mar 19 17:22:51 2013 -0700 > summary: > Closes issue 17467. Add readline and readlines support to unittest.mock.mock_open Wasn't it possible to re-use an existing implementation (such as TextIOBase or StringIO) … [View More]

3 7

Re: [Python-Dev] [Python-checkins] peps: update 2.7.4 release dates
by Andrew Svetlov March 20, 2013

March 20, 2013

Are you sure about 2.7.4 2012-04-06? I mean 2012 year. On Tue, Mar 19, 2013 at 9:15 PM, benjamin.peterson <python-checkins(a)python.org> wrote: > http://hg.python.org/peps/rev/ce17779c395c > changeset: 4810:ce17779c395c > user: Benjamin Peterson <benjamin(a)python.org> > date: Tue Mar 19 23:15:23 2013 -0500 > summary: > update 2.7.4 release dates > > files: > pep-0373.txt | 4 ++-- > 1 files changed, 2 insertions(+), 2 deletions(-)… [View More]

2 1

Re: [Python-Dev] [Python-checkins] cpython: #15927: Fix cvs.reader parsing of escaped \r\n with quoting off.
by Kristján Valur Jónsson March 20, 2013

March 20, 2013

The compiler complains about this line: if (c == '\n' | c=='\r') { Perhaps you wanted a Boolean operator? -----Original Message----- From: Python-checkins [mailto:python-checkins-bounces+kristjan=ccpgames.com@python.org] On Behalf Of r.david.murray Sent: 19. mars 2013 19:42 To: python-checkins(a)python.org Subject: [Python-checkins] cpython: #15927: Fix cvs.reader parsing of escaped \r\n with quoting off. http://hg.python.org/cpython/rev/940748853712 changeset: 82815:940748853712 parent: … [View More] 82811:684b75600fa9 user: R David Murray <rdmurray(a)bitdance.com> date: Tue Mar 19 22:41:47 2013 -0400 summary: #15927: Fix cvs.reader parsing of escaped \r\n with quoting off. This fix means that such values are correctly roundtripped, since cvs.writer already does the correct escaping. Patch by Michael Johnson. files: Lib/test/test_csv.py | 9 +++++++++ Misc/ACKS | 1 + Misc/NEWS | 3 +++ Modules/_csv.c | 13 ++++++++++++- 4 files changed, 25 insertions(+), 1 deletions(-) diff --git a/Lib/test/test_csv.py b/Lib/test/test_csv.py --- a/Lib/test/test_csv.py +++ b/Lib/test/test_csv.py @@ -308,6 +308,15 @@ for i, row in enumerate(csv.reader(fileobj)): self.assertEqual(row, rows[i]) + def test_roundtrip_escaped_unquoted_newlines(self): + with TemporaryFile("w+", newline='') as fileobj: + writer = csv.writer(fileobj,quoting=csv.QUOTE_NONE,escapechar="\\") + rows = [['a\nb','b'],['c','x\r\nd']] + writer.writerows(rows) + fileobj.seek(0) + for i, row in enumerate(csv.reader(fileobj,quoting=csv.QUOTE_NONE,escapechar="\\")): + self.assertEqual(row,rows[i]) + class TestDialectRegistry(unittest.TestCase): def test_registry_badargs(self): self.assertRaises(TypeError, csv.list_dialects, None) diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -591,6 +591,7 @@ Fredrik Johansson Gregory K. Johnson Kent Johnson +Michael Johnson Simon Johnston Matt Joiner Thomas Jollans diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -289,6 +289,9 @@ Library ------- +- Issue #15927: CVS now correctly parses escaped newlines and carriage + when parsing with quoting turned off. + - Issue #17467: add readline and readlines support to mock_open in unittest.mock. diff --git a/Modules/_csv.c b/Modules/_csv.c --- a/Modules/_csv.c +++ b/Modules/_csv.c @@ -51,7 +51,7 @@ typedef enum { START_RECORD, START_FIELD, ESCAPED_CHAR, IN_FIELD, IN_QUOTED_FIELD, ESCAPE_IN_QUOTED_FIELD, QUOTE_IN_QUOTED_FIELD, - EAT_CRNL + EAT_CRNL,AFTER_ESCAPED_CRNL } ParserState; typedef enum { @@ -644,6 +644,12 @@ break; case ESCAPED_CHAR: + if (c == '\n' | c=='\r') { + if (parse_add_char(self, c) < 0) + return -1; + self->state = AFTER_ESCAPED_CRNL; + break; + } if (c == '\0') c = '\n'; if (parse_add_char(self, c) < 0) @@ -651,6 +657,11 @@ self->state = IN_FIELD; break; + case AFTER_ESCAPED_CRNL: + if (c == '\0') + break; + /*fallthru*/ + case IN_FIELD: /* in unquoted field */ if (c == '\n' || c == '\r' || c == '\0') { -- Repository URL: http://hg.python.org/cpython [View Less]

2 1

Re: [Python-Dev] [Python-checkins] cpython: Issue #17385: Fix quadratic behavior in threading.Condition
by Ezio Melotti March 20, 2013

March 20, 2013

Hi, On Mon, Mar 11, 2013 at 2:58 AM, raymond.hettinger <python-checkins(a)python.org> wrote: > http://hg.python.org/cpython/rev/0f86b51f8f8b > changeset: 82592:0f86b51f8f8b > user: Raymond Hettinger <python(a)rcn.com> > date: Sun Mar 10 17:57:28 2013 -0700 > summary: > Issue #17385: Fix quadratic behavior in threading.Condition > > files: > Lib/threading.py | 10 ++++++++-- > Misc/NEWS | 3 +++ > 2 files changed, 11 … [View More]

1 1

Early results from Argument Clinic automation discussion
by Larry Hastings March 20, 2013

March 20, 2013

Mark Shannon, Dmitry Jemerov (from PyCharm) and I sat down to talk about rearchitecting the Argument Clinic prototype to make it easily to interact with. We came up with the following. The DSL will now produce an intermediate representation. The output will consume this intermediate representation. The two defined outputs from the IR so far: 1. The inplace generated C code (what it has right now), and 2. Argument Clinic DSL code itself. The intermediate representation will be … [View More]

1 0

Re: [Python-Dev] [Python-checkins] cpython (2.7): Issue #9090 : Error code 10035 calling socket.recv() on a socket with a timeout
by Guido van Rossum March 19, 2013

March 19, 2013

On Tue, Mar 19, 2013 at 11:08 AM, kristjan.jonsson < python-checkins(a)python.org> wrote: > http://hg.python.org/cpython/rev/8ec39bfd1f01 > changeset: 82764:8ec39bfd1f01 > branch: 2.7 > parent: 82740:b10ec5083a53 > user: Kristján Valur Jónsson <sweskman(a)gmail.com> > date: Tue Mar 19 10:58:59 2013 -0700 > summary: > Issue #9090 : Error code 10035 calling socket.recv() on a socket with a > timeout > (WSAEWOULDBLOCK - A non-… [View More]

1 0

Re: [Python-Dev] [Python-checkins] cpython (2.7): Issue #9090 : Error code 10035 calling socket.recv() on a socket with a timeout
by Kristján Valur Jónsson March 19, 2013

March 19, 2013

Yes, it is a symbol problem on unix. Working on it. -----Original Message----- From: Python-checkins [mailto:python-checkins-bounces+kristjan=ccpgames.com@python.org] On Behalf Of Senthil Kumaran Sent: 19. mars 2013 12:28 To: sweskman(a)gmail.com Cc: python-checkins(a)python.org Subject: Re: [Python-checkins] cpython (2.7): Issue #9090 : Error code 10035 calling socket.recv() on a socket with a timeout Looks like RHEL 2.7 buildbots are unhappy with this change. -- Senthil On Tue, Mar 19, … [View More]2013 at 11:08 AM, kristjan.jonsson <python-checkins(a)python.org> wrote: > http://hg.python.org/cpython/rev/8ec39bfd1f01 > changeset: 82764:8ec39bfd1f01 > branch: 2.7 > parent: 82740:b10ec5083a53 > user: Kristján Valur Jónsson <sweskman(a)gmail.com> > date: Tue Mar 19 10:58:59 2013 -0700 > summary: > Issue #9090 : Error code 10035 calling socket.recv() on a socket > with a timeout (WSAEWOULDBLOCK - A non-blocking socket operation > could not be completed > immediately) > > files: > Misc/NEWS | 5 + > Modules/socketmodule.c | 104 ++++++++++++++++++++++++---- > Modules/timemodule.c | 7 + > 3 files changed, 101 insertions(+), 15 deletions(-) > > > diff --git a/Misc/NEWS b/Misc/NEWS > --- a/Misc/NEWS > +++ b/Misc/NEWS > @@ -214,6 +214,11 @@ > Library > ------- > > +- Issue #9090: When a socket with a timeout fails with EWOULDBLOCK or > +EAGAIN, > + retry the select() loop instead of bailing out. This is because > +select() > + can incorrectly report a socket as ready for reading (for example, > +if it > + received some data with an invalid checksum). > + > - Issue #1285086: Get rid of the refcounting hack and speed up urllib.unquote(). > > - Issue #17368: Fix an off-by-one error in the Python JSON decoder > that caused diff --git a/Modules/socketmodule.c > b/Modules/socketmodule.c > --- a/Modules/socketmodule.c > +++ b/Modules/socketmodule.c > @@ -473,6 +473,17 @@ > return NULL; > } > > +#ifdef MS_WINDOWS > +#ifndef WSAEAGAIN > +#define WSAEAGAIN WSAEWOULDBLOCK > +#endif > +#define CHECK_ERRNO(expected) \ > + (WSAGetLastError() == WSA ## expected) #else #define > +CHECK_ERRNO(expected) \ > + (errno == expected) > +#endif > + > /* Convenience function to raise an error according to errno > and return a NULL pointer from a function. */ > > @@ -661,7 +672,7 @@ > after they've reacquired the interpreter lock. > Returns 1 on timeout, -1 on error, 0 otherwise. */ static int > -internal_select(PySocketSockObject *s, int writing) > +internal_select_ex(PySocketSockObject *s, int writing, double > +interval) > { > int n; > > @@ -673,6 +684,10 @@ > if (s->sock_fd < 0) > return 0; > > + /* Handling this condition here simplifies the select loops */ > + if (interval < 0.0) > + return 1; > + > /* Prefer poll, if available, since you can poll() any fd > * which can't be done with select(). */ #ifdef HAVE_POLL @@ > -684,7 +699,7 @@ > pollfd.events = writing ? POLLOUT : POLLIN; > > /* s->sock_timeout is in seconds, timeout in ms */ > - timeout = (int)(s->sock_timeout * 1000 + 0.5); > + timeout = (int)(interval * 1000 + 0.5); > n = poll(&pollfd, 1, timeout); > } > #else > @@ -692,8 +707,8 @@ > /* Construct the arguments to select */ > fd_set fds; > struct timeval tv; > - tv.tv_sec = (int)s->sock_timeout; > - tv.tv_usec = (int)((s->sock_timeout - tv.tv_sec) * 1e6); > + tv.tv_sec = (int)interval; > + tv.tv_usec = (int)((interval - tv.tv_sec) * 1e6); > FD_ZERO(&fds); > FD_SET(s->sock_fd, &fds); > > @@ -712,6 +727,49 @@ > return 0; > } > > +static int > +internal_select(PySocketSockObject *s, int writing) { > + return internal_select_ex(s, writing, s->sock_timeout); } > + > +/* > + Two macros for automatic retry of select() in case of false positives > + (for example, select() could indicate a socket is ready for reading > + but the data then discarded by the OS because of a wrong checksum). > + Here is an example of use: > + > + BEGIN_SELECT_LOOP(s) > + Py_BEGIN_ALLOW_THREADS > + timeout = internal_select_ex(s, 0, interval); > + if (!timeout) > + outlen = recv(s->sock_fd, cbuf, len, flags); > + Py_END_ALLOW_THREADS > + if (timeout == 1) { > + PyErr_SetString(socket_timeout, "timed out"); > + return -1; > + } > + END_SELECT_LOOP(s) > +*/ > +PyAPI_FUNC(double) _PyTime_floattime(void); /* defined in timemodule.c */ > +#define BEGIN_SELECT_LOOP(s) \ > + { \ > + double deadline, interval = s->sock_timeout; \ > + int has_timeout = s->sock_timeout > 0.0; \ > + if (has_timeout) { \ > + deadline = _PyTime_floattime() + s->sock_timeout; \ > + } \ > + while (1) { \ > + errno = 0; \ > + > +#define END_SELECT_LOOP(s) \ > + if (!has_timeout || \ > + (!CHECK_ERRNO(EWOULDBLOCK) && !CHECK_ERRNO(EAGAIN))) \ > + break; \ > + interval = deadline - _PyTime_floattime(); \ > + } \ > + } \ > + > /* Initialize a new socket object. */ > > static double defaulttimeout = -1.0; /* Default timeout for new sockets */ > @@ -1656,8 +1714,9 @@ > if (!IS_SELECTABLE(s)) > return select_error(); > > + BEGIN_SELECT_LOOP(s) > Py_BEGIN_ALLOW_THREADS > - timeout = internal_select(s, 0); > + timeout = internal_select_ex(s, 0, interval); > if (!timeout) > newfd = accept(s->sock_fd, SAS2SA(&addrbuf), &addrlen); > Py_END_ALLOW_THREADS > @@ -1666,6 +1725,7 @@ > PyErr_SetString(socket_timeout, "timed out"); > return NULL; > } > + END_SELECT_LOOP(s) > > #ifdef MS_WINDOWS > if (newfd == INVALID_SOCKET) > @@ -2355,8 +2415,9 @@ > } > > #ifndef __VMS > + BEGIN_SELECT_LOOP(s) > Py_BEGIN_ALLOW_THREADS > - timeout = internal_select(s, 0); > + timeout = internal_select_ex(s, 0, interval); > if (!timeout) > outlen = recv(s->sock_fd, cbuf, len, flags); > Py_END_ALLOW_THREADS > @@ -2365,6 +2426,7 @@ > PyErr_SetString(socket_timeout, "timed out"); > return -1; > } > + END_SELECT_LOOP(s) > if (outlen < 0) { > /* Note: the call to errorhandler() ALWAYS indirectly returned > NULL, so ignore its return value */ > @@ -2386,8 +2448,9 @@ > segment = remaining; > } > > + BEGIN_SELECT_LOOP(s) > Py_BEGIN_ALLOW_THREADS > - timeout = internal_select(s, 0); > + timeout = internal_select_ex(s, 0, interval); > if (!timeout) > nread = recv(s->sock_fd, read_buf, segment, flags); > Py_END_ALLOW_THREADS > @@ -2396,6 +2459,8 @@ > PyErr_SetString(socket_timeout, "timed out"); > return -1; > } > + END_SELECT_LOOP(s) > + > if (nread < 0) { > s->errorhandler(); > return -1; > @@ -2559,9 +2624,10 @@ > return -1; > } > > + BEGIN_SELECT_LOOP(s) > Py_BEGIN_ALLOW_THREADS > memset(&addrbuf, 0, addrlen); > - timeout = internal_select(s, 0); > + timeout = internal_select_ex(s, 0, interval); > if (!timeout) { > #ifndef MS_WINDOWS > #if defined(PYOS_OS2) && !defined(PYCC_GCC) > @@ -2582,6 +2648,7 @@ > PyErr_SetString(socket_timeout, "timed out"); > return -1; > } > + END_SELECT_LOOP(s) > if (n < 0) { > s->errorhandler(); > return -1; > @@ -2719,8 +2786,9 @@ > buf = pbuf.buf; > len = pbuf.len; > > + BEGIN_SELECT_LOOP(s) > Py_BEGIN_ALLOW_THREADS > - timeout = internal_select(s, 1); > + timeout = internal_select_ex(s, 1, interval); > if (!timeout) > #ifdef __VMS > n = sendsegmented(s->sock_fd, buf, len, flags); > @@ -2728,13 +2796,14 @@ > n = send(s->sock_fd, buf, len, flags); > #endif > Py_END_ALLOW_THREADS > - > - PyBuffer_Release(&pbuf); > - > if (timeout == 1) { > + PyBuffer_Release(&pbuf); > PyErr_SetString(socket_timeout, "timed out"); > return NULL; > } > + END_SELECT_LOOP(s) > + > + PyBuffer_Release(&pbuf); > if (n < 0) > return s->errorhandler(); > return PyInt_FromLong((long)n); > @@ -2768,8 +2837,9 @@ > } > > do { > + BEGIN_SELECT_LOOP(s) > Py_BEGIN_ALLOW_THREADS > - timeout = internal_select(s, 1); > + timeout = internal_select_ex(s, 1, interval); > n = -1; > if (!timeout) { > #ifdef __VMS > @@ -2784,6 +2854,7 @@ > PyErr_SetString(socket_timeout, "timed out"); > return NULL; > } > + END_SELECT_LOOP(s) > /* PyErr_CheckSignals() might change errno */ > saved_errno = errno; > /* We must run our signal handlers before looping again. > @@ -2863,17 +2934,20 @@ > return NULL; > } > > + BEGIN_SELECT_LOOP(s) > Py_BEGIN_ALLOW_THREADS > - timeout = internal_select(s, 1); > + timeout = internal_select_ex(s, 1, interval); > if (!timeout) > n = sendto(s->sock_fd, buf, len, flags, SAS2SA(&addrbuf), addrlen); > Py_END_ALLOW_THREADS > > - PyBuffer_Release(&pbuf); > if (timeout == 1) { > + PyBuffer_Release(&pbuf); > PyErr_SetString(socket_timeout, "timed out"); > return NULL; > } > + END_SELECT_LOOP(s) > + PyBuffer_Release(&pbuf); > if (n < 0) > return s->errorhandler(); > return PyInt_FromLong((long)n); > diff --git a/Modules/timemodule.c b/Modules/timemodule.c > --- a/Modules/timemodule.c > +++ b/Modules/timemodule.c > @@ -1055,3 +1055,10 @@ > > return 0; > } > + > +/* export floattime to socketmodule.c */ > +PyAPI_FUNC(double) > +_PyTime_floattime(void) > +{ > + return floattime(); > +} > > -- > Repository URL: http://hg.python.org/cpython > > _______________________________________________ > Python-checkins mailing list > Python-checkins(a)python.org > http://mail.python.org/mailman/listinfo/python-checkins > _______________________________________________ Python-checkins mailing list Python-checkins(a)python.org http://mail.python.org/mailman/listinfo/python-checkins [View Less]

1 1