Mailman 3 August 2014 - Python-checkins

cpython (merge 3.4 -> default): Issue #22185: Fix an occasional RuntimeError in threading.Condition.wait()
by antoine.pitrou 29 Aug '14

29 Aug '14

http://hg.python.org/cpython/rev/78a38f8bd5d9 changeset: 92269:78a38f8bd5d9 parent: 92267:ab81b4cdc33c parent: 92268:4cce39cfe46c user: Antoine Pitrou <solipsis(a)pitrou.net> date: Fri Aug 29 23:27:33 2014 +0200 summary: Issue #22185: Fix an occasional RuntimeError in threading.Condition.wait() caused by mutation of the waiters queue without holding the lock. Patch by Doug Zongker. files: Lib/threading.py | 11 ++++++----- Misc/ACKS | 1 + Misc/NEWS | 4 ++++ 3 files changed, 11 insertions(+), 5 deletions(-) diff --git a/Lib/threading.py b/Lib/threading.py --- a/Lib/threading.py +++ b/Lib/threading.py @@ -290,6 +290,7 @@ waiter.acquire() self._waiters.append(waiter) saved_state = self._release_save() + gotit = False try: # restore state no matter what (e.g., KeyboardInterrupt) if timeout is None: waiter.acquire() @@ -299,14 +300,14 @@ gotit = waiter.acquire(True, timeout) else: gotit = waiter.acquire(False) - if not gotit: - try: - self._waiters.remove(waiter) - except ValueError: - pass return gotit finally: self._acquire_restore(saved_state) + if not gotit: + try: + self._waiters.remove(waiter) + except ValueError: + pass def wait_for(self, predicate, timeout=None): """Wait until a condition evaluates to True. diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -1518,4 +1518,5 @@ Kai Zhu Tarek Ziadé Gennadiy Zlobin +Doug Zongker Peter Åstrand diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -124,6 +124,10 @@ Library ------- +- Issue #22185: Fix an occasional RuntimeError in threading.Condition.wait() + caused by mutation of the waiters queue without holding the lock. Patch + by Doug Zongker. + - Issue #22287: On UNIX, _PyTime_gettimeofday() now uses clock_gettime(CLOCK_REALTIME) if available. As a side effect, Python now depends on the librt library on Solaris and on Linux (only with glibc older -- Repository URL: http://hg.python.org/cpython

1 0

cpython (3.4): Issue #22185: Fix an occasional RuntimeError in threading.Condition.wait()
by antoine.pitrou 29 Aug '14

29 Aug '14

http://hg.python.org/cpython/rev/4cce39cfe46c changeset: 92268:4cce39cfe46c branch: 3.4 parent: 92262:a3452677a386 user: Antoine Pitrou <solipsis(a)pitrou.net> date: Fri Aug 29 23:26:36 2014 +0200 summary: Issue #22185: Fix an occasional RuntimeError in threading.Condition.wait() caused by mutation of the waiters queue without holding the lock. Patch by Doug Zongker. files: Lib/threading.py | 11 ++++++----- Misc/ACKS | 1 + Misc/NEWS | 4 ++++ 3 files changed, 11 insertions(+), 5 deletions(-) diff --git a/Lib/threading.py b/Lib/threading.py --- a/Lib/threading.py +++ b/Lib/threading.py @@ -284,6 +284,7 @@ waiter.acquire() self._waiters.append(waiter) saved_state = self._release_save() + gotit = False try: # restore state no matter what (e.g., KeyboardInterrupt) if timeout is None: waiter.acquire() @@ -293,14 +294,14 @@ gotit = waiter.acquire(True, timeout) else: gotit = waiter.acquire(False) - if not gotit: - try: - self._waiters.remove(waiter) - except ValueError: - pass return gotit finally: self._acquire_restore(saved_state) + if not gotit: + try: + self._waiters.remove(waiter) + except ValueError: + pass def wait_for(self, predicate, timeout=None): """Wait until a condition evaluates to True. diff --git a/Misc/ACKS b/Misc/ACKS --- a/Misc/ACKS +++ b/Misc/ACKS @@ -1500,4 +1500,5 @@ Kai Zhu Tarek Ziadé Gennadiy Zlobin +Doug Zongker Peter Åstrand diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -27,6 +27,10 @@ Library ------- +- Issue #22185: Fix an occasional RuntimeError in threading.Condition.wait() + caused by mutation of the waiters queue without holding the lock. Patch + by Doug Zongker. + - Issue #22182: Use e.args to unpack exceptions correctly in distutils.file_util.move_file. Patch by Claudiu Popa. -- Repository URL: http://hg.python.org/cpython

1 0

peps: Add PEP 476 - Enabling certificate verification by default for stdlib http
by donald.stufft 29 Aug '14

29 Aug '14

http://hg.python.org/peps/rev/b128992118a8 changeset: 5530:b128992118a8 user: Donald Stufft <donald(a)stufft.io> date: Fri Aug 29 15:44:32 2014 -0400 summary: Add PEP 476 - Enabling certificate verification by default for stdlib http clients files: pep-0476.txt | 145 +++++++++++++++++++++++++++++++++++++++ 1 files changed, 145 insertions(+), 0 deletions(-) diff --git a/pep-0476.txt b/pep-0476.txt new file mode 100644 --- /dev/null +++ b/pep-0476.txt @@ -0,0 +1,145 @@ +PEP: 476 +Title: Enabling certificate verification by default for stdlib http clients +Version: $Revision$ +Last-Modified: $Date$ +Author: Alex Gaynor <alex.gaynor(a)gmail.com> +Status: Draft +Type: Standards Track +Content-Type: text/x-rst +Created: 28-August-2014 + +Abstract +======== + +Currently when a standard library http client (the ``urllib`` and ``http`` +modules) encounters an ``https://`` URL it will wrap the network HTTP traffic +in a TLS stream, as is necessary to communicate with such a server. However, +during the TLS handshake it will not actually check that the server has an X509 +certificate is signed by a CA in any trust root, nor will it verify that the +Common Name (or Subject Alternate Name) on the presented certificate matches +the requested host. + +The failure to do these checks means that anyone with a privileged network +position is able to trivially execute a man in the middle attack against a +Python application using either of these HTTP clients, and change traffic at +will. + +This PEP proposes to enable verification of X509 certificate signatures, as +well as hostname verification for Python's HTTP clients by default, subject to +opt-out on a per-call basis. + +Rationale +========= + +The "S" in "HTTPS" stands for secure. When Python's users type "HTTPS" they are +expecting a secure connection, and Python should adhere to a reasonable +standard of care in delivering this. Currently we are failing at this, and in +doing so, APIs which appear simple are misleading users. + +When asked, many Python users state that they were not aware that Python failed +to perform these validations, and are shocked. + +The popularity of ``requests`` (which enables these checks by default) +demonstrates that these checks are not overly burdensome in any way, and the +fact that it is widely recommended as a major security improvement over the +standard library clients demonstrates that many expect a higher standard for +"security by default" from their tools. + +The failure of various applications to note Python's negligence in this matter +is a source of *regular* CVE assignment [#]_ [#]_ [#]_ [#]_ [#]_ [#]_ [#]_ [#]_ +[#]_ [#]_ [#]_. + +.. [#] https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2010-4340 +.. [#] https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2012-3533 +.. [#] https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2012-5822 +.. [#] https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2012-5825 +.. [#] https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2013-1909 +.. [#] https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2013-2037 +.. [#] https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2013-2073 +.. [#] https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2013-2191 +.. [#] https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2013-4111 +.. [#] https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2013-6396 +.. [#] https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2013-6444 + +Technical Details +================= + +Python would use the system provided certificate database on all platforms. +Failure to locate such a database would be an error, and users would need to +explicitly specify a location to fix it. + +This can be achieved by simply replacing the use of +``ssl._create_stdlib_context`` with ``ssl.create_default_context`` in +``http.client``. + +Trust database +-------------- + +This PEP proposes using the system-provided certificate database. Previous +discussions have suggested bundling Mozilla's certificate database and using +that by default. This was decided against for several reasons: + +* Using the platform trust database imposes a lower maintenance burden on the + Python developers -- shipping our own trust database would require doing a + release every time a certificate was revoked. +* Linux vendors, and other downstreams, would unbundle the Mozilla + certificates, resulting in a more fragmented set of behaviors. +* Using the platform stores makes it easier to handle situations such as + corporate internal CAs. + +Backwards compatibility +----------------------- + +This change will have the appearance of causing some HTTPS connections to +"break", because they will now raise an Exception during handshake. + +This is misleading however, in fact these connections are presently failing +silently, an HTTPS URL indicates an expectation of confidentiality and +authentication. The fact that Python does not actually verify that the user's +request has been made is a bug, further: "Errors should never pass silently." + +Nevertheless, users who have a need to access servers with self-signed or +incorrect certificates would be able to do so by providing a context with +custom trust roots or which disables validation (documentation should strongly +recommend the former where possible). Users will also be able to add necessary +certificates to system trust stores in order to trust them globally. + +Twisted's 14.0 release made this same change, and it has been met with almost +no opposition. + +Other protocols +=============== + +This PEP only proposes requiring this level of validation for HTTP clients, not +for other protocols such as SMTP. + +This is because while a high percentage of HTTPS servers have correct +certificates, as a result of the validation performed by browsers, for other +protocols self-signed or otherwise incorrect certificates are far more common. +Note that for SMTP at least, this appears to be changing and should be reviewed +for a potential similar PEP in the future: + +* https://www.facebook.com/notes/protect-the-graph/the-current-state-of-smtp-… +* https://www.facebook.com/notes/protect-the-graph/massive-growth-in-smtp-sta… + +Python Versions +=============== + +This PEP proposes making these changes to ``default`` (Python 3) branch. I +strongly believe these changes also belong in Python 2, but doing them in a +patch-release isn't reasonable, and there is strong opposition to doing a 2.8 +release. + +Copyright +========= + +This document has been placed into the public domain. + + +.. + Local Variables: + mode: indented-text + indent-tabs-mode: nil + sentence-end-double-space: t + fill-column: 70 + coding: utf-8 -- Repository URL: http://hg.python.org/peps

1 0

cpython: Issue #22043: Oops, fix perf_counter() on UNIX if no monotonic clock is
by victor.stinner 29 Aug '14

29 Aug '14

http://hg.python.org/cpython/rev/ab81b4cdc33c changeset: 92267:ab81b4cdc33c user: Victor Stinner <victor.stinner(a)gmail.com> date: Fri Aug 29 17:00:17 2014 +0200 summary: Issue #22043: Oops, fix perf_counter() on UNIX if no monotonic clock is available (unlikely) files: Modules/timemodule.c | 3 +-- 1 files changed, 1 insertions(+), 2 deletions(-) diff --git a/Modules/timemodule.c b/Modules/timemodule.c --- a/Modules/timemodule.c +++ b/Modules/timemodule.c @@ -1046,9 +1046,8 @@ use_monotonic = 0; PyErr_Clear(); } -#else +#endif return floattime(info); -#endif #endif } -- Repository URL: http://hg.python.org/cpython

1 0

cpython: Issue #22043: Simplify time.perf_counter() on Windows
by victor.stinner 29 Aug '14

29 Aug '14

http://hg.python.org/cpython/rev/76bc15c918b1 changeset: 92266:76bc15c918b1 user: Victor Stinner <victor.stinner(a)gmail.com> date: Fri Aug 29 16:51:33 2014 +0200 summary: Issue #22043: Simplify time.perf_counter() on Windows QueryPerformanceFrequency() cannot fail on Windows XP and later according to its documentation: raise an exception on error and drop the fallback to the system clock. files: Modules/timemodule.c | 46 +++++++++++-------------------- 1 files changed, 16 insertions(+), 30 deletions(-) diff --git a/Modules/timemodule.c b/Modules/timemodule.c --- a/Modules/timemodule.c +++ b/Modules/timemodule.c @@ -96,8 +96,8 @@ #define WIN32_PERF_COUNTER /* Win32 has better clock replacement; we have our own version, due to Mark Hammond and Tim Peters */ -static int -win_perf_counter(_Py_clock_info_t *info, PyObject **result) +static PyObject* +win_perf_counter(_Py_clock_info_t *info) { static LONGLONG cpu_frequency = 0; static LONGLONG ctrStart; @@ -109,10 +109,8 @@ QueryPerformanceCounter(&now); ctrStart = now.QuadPart; if (!QueryPerformanceFrequency(&freq) || freq.QuadPart == 0) { - /* Unlikely to happen - this works on all intel - machines at least! Revert to clock() */ - *result = NULL; - return -1; + PyErr_SetFromWindowsErr(0); + return NULL; } cpu_frequency = freq.QuadPart; } @@ -124,8 +122,7 @@ info->monotonic = 1; info->adjustable = 0; } - *result = PyFloat_FromDouble(diff / (double)cpu_frequency); - return 0; + return PyFloat_FromDouble(diff / (double)cpu_frequency); } #endif @@ -135,11 +132,10 @@ pyclock(_Py_clock_info_t *info) { #ifdef WIN32_PERF_COUNTER - PyObject *res; - if (win_perf_counter(info, &res) == 0) - return res; + return win_perf_counter(info); +#else + return floatclock(info); #endif - return floatclock(info); } static PyObject * @@ -1036,35 +1032,25 @@ static PyObject* perf_counter(_Py_clock_info_t *info) { -#if defined(WIN32_PERF_COUNTER) || defined(PYMONOTONIC) - PyObject *res; -#endif -#if defined(WIN32_PERF_COUNTER) - static int use_perf_counter = 1; -#endif +#ifdef WIN32_PERF_COUNTER + return win_perf_counter(info); +#else + #ifdef PYMONOTONIC static int use_monotonic = 1; -#endif -#ifdef WIN32_PERF_COUNTER - if (use_perf_counter) { - if (win_perf_counter(info, &res) == 0) - return res; - use_perf_counter = 0; - } -#endif - -#ifdef PYMONOTONIC if (use_monotonic) { - res = pymonotonic(info); + PyObject *res = pymonotonic(info); if (res != NULL) return res; use_monotonic = 0; PyErr_Clear(); } +#else + return floattime(info); #endif - return floattime(info); +#endif } static PyObject * -- Repository URL: http://hg.python.org/cpython

1 0

cpython: Issue #22043: _PyTime_Init() now checks if the system clock works.
by victor.stinner 29 Aug '14

29 Aug '14

http://hg.python.org/cpython/rev/668e0bf30042 changeset: 92265:668e0bf30042 user: Victor Stinner <victor.stinner(a)gmail.com> date: Fri Aug 29 16:31:59 2014 +0200 summary: Issue #22043: _PyTime_Init() now checks if the system clock works. Other changes: * The whole _PyTime API is private (not defined if Py_LIMITED_API is set) * _PyTime_gettimeofday_info() also returns -1 on error * Simplify PyTime_gettimeofday(): only use clock_gettime(CLOCK_REALTIME) or gettimeofday() on UNIX. Don't fallback to ftime() or time() anymore. files: Include/pytime.h | 13 +- Modules/timemodule.c | 5 +- Python/pythonrun.c | 3 +- Python/pytime.c | 145 ++++++++++++------------------ 4 files changed, 73 insertions(+), 93 deletions(-) diff --git a/Include/pytime.h b/Include/pytime.h --- a/Include/pytime.h +++ b/Include/pytime.h @@ -13,6 +13,8 @@ extern "C" { #endif +#ifndef Py_LIMITED_API + #ifdef HAVE_GETTIMEOFDAY typedef struct timeval _PyTime_timeval; #else @@ -37,7 +39,7 @@ /* Similar to _PyTime_gettimeofday() but retrieve also information on the * clock used to get the current time. */ -PyAPI_FUNC(void) _PyTime_gettimeofday_info( +PyAPI_FUNC(int) _PyTime_gettimeofday_info( _PyTime_timeval *tp, _Py_clock_info_t *info); @@ -52,8 +54,6 @@ ((tv_end.tv_sec - tv_start.tv_sec) + \ (tv_end.tv_usec - tv_start.tv_usec) * 0.000001) -#ifndef Py_LIMITED_API - typedef enum { /* Round towards zero. */ _PyTime_ROUND_DOWN=0, @@ -92,10 +92,11 @@ time_t *sec, long *nsec, _PyTime_round_t); -#endif -/* Dummy to force linking. */ -PyAPI_FUNC(void) _PyTime_Init(void); +/* Initialize time. + Return 0 on success, raise an exception and return -1 on error. */ +PyAPI_FUNC(int) _PyTime_Init(void); +#endif /* Py_LIMITED_API */ #ifdef __cplusplus } diff --git a/Modules/timemodule.c b/Modules/timemodule.c --- a/Modules/timemodule.c +++ b/Modules/timemodule.c @@ -1535,7 +1535,10 @@ floattime(_Py_clock_info_t *info) { _PyTime_timeval t; - _PyTime_gettimeofday_info(&t, info); + if (_PyTime_gettimeofday_info(&t, info) < 0) { + assert(info != NULL); + return NULL; + } return PyFloat_FromDouble((double)t.tv_sec + t.tv_usec * 1e-6); } diff --git a/Python/pythonrun.c b/Python/pythonrun.c --- a/Python/pythonrun.c +++ b/Python/pythonrun.c @@ -452,7 +452,8 @@ if (_PyFaulthandler_Init()) Py_FatalError("Py_Initialize: can't initialize faulthandler"); - _PyTime_Init(); + if (_PyTime_Init() < 0) + Py_FatalError("Py_Initialize: can't initialize time"); if (initfsencoding(interp) < 0) Py_FatalError("Py_Initialize: unable to load the file system codec"); diff --git a/Python/pytime.c b/Python/pytime.c --- a/Python/pytime.c +++ b/Python/pytime.c @@ -3,29 +3,16 @@ #include <windows.h> #endif -#if defined(__APPLE__) && defined(HAVE_GETTIMEOFDAY) && defined(HAVE_FTIME) - /* - * _PyTime_gettimeofday falls back to ftime when getttimeofday fails because the latter - * might fail on some platforms. This fallback is unwanted on MacOSX because - * that makes it impossible to use a binary build on OSX 10.4 on earlier - * releases of the OS. Therefore claim we don't support ftime. - */ -# undef HAVE_FTIME -#endif - -#if defined(HAVE_FTIME) && !defined(MS_WINDOWS) -#include <sys/timeb.h> -extern int ftime(struct timeb *); -#endif - -static void -pygettimeofday(_PyTime_timeval *tp, _Py_clock_info_t *info) +static int +pygettimeofday(_PyTime_timeval *tp, _Py_clock_info_t *info, int raise) { #ifdef MS_WINDOWS FILETIME system_time; ULARGE_INTEGER large; ULONGLONG microseconds; + assert(info == NULL || raise); + GetSystemTimeAsFileTime(&system_time); large.u.LowPart = system_time.dwLowDateTime; large.u.HighPart = system_time.dwHighDateTime; @@ -37,55 +24,51 @@ tp->tv_usec = microseconds % 1000000; if (info) { DWORD timeAdjustment, timeIncrement; - BOOL isTimeAdjustmentDisabled; + BOOL isTimeAdjustmentDisabled, ok; info->implementation = "GetSystemTimeAsFileTime()"; info->monotonic = 0; - (void) GetSystemTimeAdjustment(&timeAdjustment, &timeIncrement, - &isTimeAdjustmentDisabled); + ok = GetSystemTimeAdjustment(&timeAdjustment, &timeIncrement, + &isTimeAdjustmentDisabled); + if (!ok) { + PyErr_SetFromWindowsErr(0); + return -1; + } info->resolution = timeIncrement * 1e-7; info->adjustable = 1; } -#else - /* There are three ways to get the time: - (1) gettimeofday() -- resolution in microseconds - (2) ftime() -- resolution in milliseconds - (3) time() -- resolution in seconds - In all cases the return value in a timeval struct. - Since on some systems (e.g. SCO ODT 3.0) gettimeofday() may - fail, so we fall back on ftime() or time(). - Note: clock resolution does not imply clock accuracy! */ + return 0; -#if (defined(HAVE_CLOCK_GETTIME) || defined(HAVE_GETTIMEOFDAY) \ - || defined(HAVE_FTIME)) +#else /* MS_WINDOWS */ int err; -#endif #ifdef HAVE_CLOCK_GETTIME struct timespec ts; #endif -#ifdef HAVE_FTIME - struct timeb t; -#endif - /* test clock_gettime(CLOCK_REALTIME) */ + assert(info == NULL || raise); + #ifdef HAVE_CLOCK_GETTIME - err = clock_gettime(CLOCK_REALTIME, &ts); - if (err == 0) { - if (info) { - struct timespec res; - info->implementation = "clock_gettime(CLOCK_REALTIME)"; - info->monotonic = 0; - info->adjustable = 1; - if (clock_getres(CLOCK_REALTIME, &res) == 0) - info->resolution = res.tv_sec + res.tv_nsec * 1e-9; - else - info->resolution = 1e-9; - } - tp->tv_sec = ts.tv_sec; - tp->tv_usec = ts.tv_nsec / 1000; - return; + err = clock_gettime(CLOCK_REALTIME, &ts); + if (err) { + if (raise) + PyErr_SetFromErrno(PyExc_OSError); + return -1; } -#endif + tp->tv_sec = ts.tv_sec; + tp->tv_usec = ts.tv_nsec / 1000; + + if (info) { + struct timespec res; + info->implementation = "clock_gettime(CLOCK_REALTIME)"; + info->monotonic = 0; + info->adjustable = 1; + if (clock_getres(CLOCK_REALTIME, &res) == 0) + info->resolution = res.tv_sec + res.tv_nsec * 1e-9; + else + info->resolution = 1e-9; + } + return 0; +#else /* HAVE_CLOCK_GETTIME */ /* test gettimeofday() */ #ifdef HAVE_GETTIMEOFDAY @@ -94,51 +77,39 @@ #else err = gettimeofday(tp, (struct timezone *)NULL); #endif - if (err == 0) { - if (info) { - info->implementation = "gettimeofday()"; - info->resolution = 1e-6; - info->monotonic = 0; - info->adjustable = 1; - } - return; + if (err) { + if (raise) + PyErr_SetFromErrno(PyExc_OSError); + return -1; } -#endif /* HAVE_GETTIMEOFDAY */ -#ifdef HAVE_FTIME - ftime(&t); - tp->tv_sec = t.time; - tp->tv_usec = t.millitm * 1000; if (info) { - info->implementation = "ftime()"; - info->resolution = 1e-3; + info->implementation = "gettimeofday()"; + info->resolution = 1e-6; info->monotonic = 0; info->adjustable = 1; } -#else /* !HAVE_FTIME */ - tp->tv_sec = time(NULL); - tp->tv_usec = 0; - if (info) { - info->implementation = "time()"; - info->resolution = 1.0; - info->monotonic = 0; - info->adjustable = 1; - } -#endif /* !HAVE_FTIME */ - -#endif /* MS_WINDOWS */ + return 0; +#endif /* HAVE_GETTIMEOFDAY */ +#endif /* !HAVE_CLOCK_GETTIME */ +#endif /* !MS_WINDOWS */ } void _PyTime_gettimeofday(_PyTime_timeval *tp) { - pygettimeofday(tp, NULL); + if (pygettimeofday(tp, NULL, 0) < 0) { + /* cannot happen, _PyTime_Init() checks that pygettimeofday() works */ + assert(0); + tp->tv_sec = 0; + tp->tv_usec = 0; + } } -void +int _PyTime_gettimeofday_info(_PyTime_timeval *tp, _Py_clock_info_t *info) { - pygettimeofday(tp, info); + return pygettimeofday(tp, info, 1); } static void @@ -273,8 +244,12 @@ return _PyTime_ObjectToDenominator(obj, sec, usec, 1e6, round); } -void -_PyTime_Init() +int +_PyTime_Init(void) { - /* Do nothing. Needed to force linking. */ + _PyTime_timeval tv; + /* ensure that the system clock works */ + if (_PyTime_gettimeofday_info(&tv, NULL) < 0) + return -1; + return 0; } -- Repository URL: http://hg.python.org/cpython

1 0

cpython: Issue #22287: On UNIX, _PyTime_gettimeofday() now uses
by victor.stinner 29 Aug '14

29 Aug '14

http://hg.python.org/cpython/rev/8e13ad4e5ae6 changeset: 92264:8e13ad4e5ae6 user: Victor Stinner <victor.stinner(a)gmail.com> date: Fri Aug 29 15:41:08 2014 +0200 summary: Issue #22287: On UNIX, _PyTime_gettimeofday() now uses clock_gettime(CLOCK_REALTIME) if available. As a side effect, Python now depends on the librt library on Solaris and on Linux (only with glibc older than 2.17). files: Misc/NEWS | 5 ++ Modules/timemodule.c | 22 ------------- Python/pytime.c | 54 ++++++++++++++++++++++++------- configure | 1 + configure.ac | 1 + 5 files changed, 48 insertions(+), 35 deletions(-) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -124,6 +124,11 @@ Library ------- +- Issue #22287: On UNIX, _PyTime_gettimeofday() now uses + clock_gettime(CLOCK_REALTIME) if available. As a side effect, Python now + depends on the librt library on Solaris and on Linux (only with glibc older + than 2.17). + - Issue #22182: Use e.args to unpack exceptions correctly in distutils.file_util.move_file. Patch by Claudiu Popa. diff --git a/Modules/timemodule.c b/Modules/timemodule.c --- a/Modules/timemodule.c +++ b/Modules/timemodule.c @@ -1535,28 +1535,6 @@ floattime(_Py_clock_info_t *info) { _PyTime_timeval t; -#ifdef HAVE_CLOCK_GETTIME - struct timespec tp; - int ret; - - /* _PyTime_gettimeofday() does not use clock_gettime() - because it would require to link Python to the rt (real-time) - library, at least on Linux */ - ret = clock_gettime(CLOCK_REALTIME, &tp); - if (ret == 0) { - if (info) { - struct timespec res; - info->implementation = "clock_gettime(CLOCK_REALTIME)"; - info->monotonic = 0; - info->adjustable = 1; - if (clock_getres(CLOCK_REALTIME, &res) == 0) - info->resolution = res.tv_sec + res.tv_nsec * 1e-9; - else - info->resolution = 1e-9; - } - return PyFloat_FromDouble(tp.tv_sec + tp.tv_nsec * 1e-9); - } -#endif _PyTime_gettimeofday_info(&t, info); return PyFloat_FromDouble((double)t.tv_sec + t.tv_usec * 1e-6); } diff --git a/Python/pytime.c b/Python/pytime.c --- a/Python/pytime.c +++ b/Python/pytime.c @@ -56,8 +56,39 @@ fail, so we fall back on ftime() or time(). Note: clock resolution does not imply clock accuracy! */ +#if (defined(HAVE_CLOCK_GETTIME) || defined(HAVE_GETTIMEOFDAY) \ + || defined(HAVE_FTIME)) + int err; +#endif +#ifdef HAVE_CLOCK_GETTIME + struct timespec ts; +#endif +#ifdef HAVE_FTIME + struct timeb t; +#endif + + /* test clock_gettime(CLOCK_REALTIME) */ +#ifdef HAVE_CLOCK_GETTIME + err = clock_gettime(CLOCK_REALTIME, &ts); + if (err == 0) { + if (info) { + struct timespec res; + info->implementation = "clock_gettime(CLOCK_REALTIME)"; + info->monotonic = 0; + info->adjustable = 1; + if (clock_getres(CLOCK_REALTIME, &res) == 0) + info->resolution = res.tv_sec + res.tv_nsec * 1e-9; + else + info->resolution = 1e-9; + } + tp->tv_sec = ts.tv_sec; + tp->tv_usec = ts.tv_nsec / 1000; + return; + } +#endif + + /* test gettimeofday() */ #ifdef HAVE_GETTIMEOFDAY - int err; #ifdef GETTIMEOFDAY_NO_TZ err = gettimeofday(tp); #else @@ -74,18 +105,15 @@ } #endif /* HAVE_GETTIMEOFDAY */ -#if defined(HAVE_FTIME) - { - struct timeb t; - ftime(&t); - tp->tv_sec = t.time; - tp->tv_usec = t.millitm * 1000; - if (info) { - info->implementation = "ftime()"; - info->resolution = 1e-3; - info->monotonic = 0; - info->adjustable = 1; - } +#ifdef HAVE_FTIME + ftime(&t); + tp->tv_sec = t.time; + tp->tv_usec = t.millitm * 1000; + if (info) { + info->implementation = "ftime()"; + info->resolution = 1e-3; + info->monotonic = 0; + info->adjustable = 1; } #else /* !HAVE_FTIME */ tp->tv_sec = time(NULL); diff --git a/configure b/configure --- a/configure +++ b/configure @@ -11904,6 +11904,7 @@ $as_echo "$ac_cv_lib_rt_clock_gettime" >&6; } if test "x$ac_cv_lib_rt_clock_gettime" = xyes; then : + LIBS="$LIBS -lrt" $as_echo "#define HAVE_CLOCK_GETTIME 1" >>confdefs.h diff --git a/configure.ac b/configure.ac --- a/configure.ac +++ b/configure.ac @@ -3319,6 +3319,7 @@ AC_CHECK_FUNCS(clock_gettime, [], [ AC_CHECK_LIB(rt, clock_gettime, [ + LIBS="$LIBS -lrt" AC_DEFINE(HAVE_CLOCK_GETTIME, 1) AC_DEFINE(TIMEMODULE_LIB, [rt], [Library needed by timemodule.c: librt may be needed for clock_gettime()]) -- Repository URL: http://hg.python.org/cpython

1 0

Daily reference leaks (4fc575d55e2b): sum=21000
by solipsis＠pitrou.net 29 Aug '14

29 Aug '14

results for 4fc575d55e2b on branch "default" -------------------------------------------- test_codecs leaked [5825, 5825, 5825] references, sum=17475 test_codecs leaked [1172, 1174, 1174] memory blocks, sum=3520 test_collections leaked [-2, 0, 0] references, sum=-2 test_functools leaked [0, 0, 3] memory blocks, sum=3 test_site leaked [0, 2, 0] references, sum=2 test_site leaked [0, 2, 0] memory blocks, sum=2 Command line was: ['./python', '-m', 'test.regrtest', '-uall', '-R', '3:3:/home/antoine/cpython/refleaks/reflogRlIWdr', '-x']

1 0

cpython (merge 3.4 -> default): Issue #22182: Use e.args to unpack exceptions correctly in
by berker.peksag 29 Aug '14

29 Aug '14

http://hg.python.org/cpython/rev/f01413758114 changeset: 92263:f01413758114 parent: 92258:4fc575d55e2b parent: 92262:a3452677a386 user: Berker Peksag <berker.peksag(a)gmail.com> date: Fri Aug 29 07:09:25 2014 +0300 summary: Issue #22182: Use e.args to unpack exceptions correctly in distutils.file_util.move_file. Patch by Claudiu Popa. files: Lib/distutils/file_util.py | 4 +- Lib/distutils/tests/test_file_util.py | 20 +++++++++++++++ Misc/NEWS | 3 ++ 3 files changed, 25 insertions(+), 2 deletions(-) diff --git a/Lib/distutils/file_util.py b/Lib/distutils/file_util.py --- a/Lib/distutils/file_util.py +++ b/Lib/distutils/file_util.py @@ -194,7 +194,7 @@ try: os.rename(src, dst) except OSError as e: - (num, msg) = e + (num, msg) = e.args if num == errno.EXDEV: copy_it = True else: @@ -206,7 +206,7 @@ try: os.unlink(src) except OSError as e: - (num, msg) = e + (num, msg) = e.args try: os.unlink(dst) except OSError: diff --git a/Lib/distutils/tests/test_file_util.py b/Lib/distutils/tests/test_file_util.py --- a/Lib/distutils/tests/test_file_util.py +++ b/Lib/distutils/tests/test_file_util.py @@ -2,10 +2,13 @@ import unittest import os import shutil +import errno +from unittest.mock import patch from distutils.file_util import move_file from distutils import log from distutils.tests import support +from distutils.errors import DistutilsFileError from test.support import run_unittest class FileUtilTestCase(support.TempdirManager, unittest.TestCase): @@ -58,6 +61,23 @@ wanted = ['moving %s -> %s' % (self.source, self.target_dir)] self.assertEqual(self._logs, wanted) + @patch('os.rename', side_effect=OSError('wrong', 1)) + def test_move_file_exception_unpacking_rename(self, _): + # see issue 22182 + with self.assertRaises(DistutilsFileError): + with open(self.source, 'w') as fobj: + fobj.write('spam eggs') + move_file(self.source, self.target, verbose=0) + + @patch('os.rename', side_effect=OSError(errno.EXDEV, 'wrong')) + @patch('os.unlink', side_effect=OSError('wrong', 1)) + def test_move_file_exception_unpacking_unlink(self, rename, unlink): + # see issue 22182 + with self.assertRaises(DistutilsFileError): + with open(self.source, 'w') as fobj: + fobj.write('spam eggs') + move_file(self.source, self.target, verbose=0) + def test_suite(): return unittest.makeSuite(FileUtilTestCase) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -124,6 +124,9 @@ Library ------- +- Issue #22182: Use e.args to unpack exceptions correctly in + distutils.file_util.move_file. Patch by Claudiu Popa. + - The webbrowser module now uses subprocess's start_new_session=True rather than a potentially risky preexec_fn=os.setsid call. -- Repository URL: http://hg.python.org/cpython

1 0

cpython (3.4): Issue #22182: Use e.args to unpack exceptions correctly in
by berker.peksag 29 Aug '14

29 Aug '14

http://hg.python.org/cpython/rev/a3452677a386 changeset: 92262:a3452677a386 branch: 3.4 parent: 92257:20244eda2946 user: Berker Peksag <berker.peksag(a)gmail.com> date: Fri Aug 29 07:07:35 2014 +0300 summary: Issue #22182: Use e.args to unpack exceptions correctly in distutils.file_util.move_file. Patch by Claudiu Popa. files: Lib/distutils/file_util.py | 4 +- Lib/distutils/tests/test_file_util.py | 20 +++++++++++++++ Misc/NEWS | 3 ++ 3 files changed, 25 insertions(+), 2 deletions(-) diff --git a/Lib/distutils/file_util.py b/Lib/distutils/file_util.py --- a/Lib/distutils/file_util.py +++ b/Lib/distutils/file_util.py @@ -194,7 +194,7 @@ try: os.rename(src, dst) except OSError as e: - (num, msg) = e + (num, msg) = e.args if num == errno.EXDEV: copy_it = True else: @@ -206,7 +206,7 @@ try: os.unlink(src) except OSError as e: - (num, msg) = e + (num, msg) = e.args try: os.unlink(dst) except OSError: diff --git a/Lib/distutils/tests/test_file_util.py b/Lib/distutils/tests/test_file_util.py --- a/Lib/distutils/tests/test_file_util.py +++ b/Lib/distutils/tests/test_file_util.py @@ -2,10 +2,13 @@ import unittest import os import shutil +import errno +from unittest.mock import patch from distutils.file_util import move_file from distutils import log from distutils.tests import support +from distutils.errors import DistutilsFileError from test.support import run_unittest class FileUtilTestCase(support.TempdirManager, unittest.TestCase): @@ -58,6 +61,23 @@ wanted = ['moving %s -> %s' % (self.source, self.target_dir)] self.assertEqual(self._logs, wanted) + @patch('os.rename', side_effect=OSError('wrong', 1)) + def test_move_file_exception_unpacking_rename(self, _): + # see issue 22182 + with self.assertRaises(DistutilsFileError): + with open(self.source, 'w') as fobj: + fobj.write('spam eggs') + move_file(self.source, self.target, verbose=0) + + @patch('os.rename', side_effect=OSError(errno.EXDEV, 'wrong')) + @patch('os.unlink', side_effect=OSError('wrong', 1)) + def test_move_file_exception_unpacking_unlink(self, rename, unlink): + # see issue 22182 + with self.assertRaises(DistutilsFileError): + with open(self.source, 'w') as fobj: + fobj.write('spam eggs') + move_file(self.source, self.target, verbose=0) + def test_suite(): return unittest.makeSuite(FileUtilTestCase) diff --git a/Misc/NEWS b/Misc/NEWS --- a/Misc/NEWS +++ b/Misc/NEWS @@ -27,6 +27,9 @@ Library ------- +- Issue #22182: Use e.args to unpack exceptions correctly in + distutils.file_util.move_file. Patch by Claudiu Popa. + - The webbrowser module now uses subprocess's start_new_session=True rather than a potentially risky preexec_fn=os.setsid call. -- Repository URL: http://hg.python.org/cpython

1 0