Mailman 3 April 2014 - Python-Dev

Python "2migr8"
by Steve Dower April 16, 2014

April 16, 2014

Just in case there's anyone out there who isn't yet sick of discussing how to proceed with Python 2.7, I have some more inputs to contribute. To put it up front, I'm totally against "CPython 2.8" ever becoming a real thing. Anything that comes out should be seen as a migration path, not an upgrade path. I'll also admit I'm not heavily invested in working on it myself, but I had a number of conversations during PyCon (as well as being at the language summit) that puts me in a position to share … [View More]the ideas and concerns that have been raised. The main trigger was a conversation I had with two employees of a very large bank that has about 3000 Python users (not developers - mostly financial analysts) and 16 million lines of code running on 2.7. They are keen to migrate to 3.x but cannot afford to stop work entirely while their code is updated. (There was much more to the conversation than I'm relating here - I'm keeping to the directly relevant bits.) In describing the approach they'd like to take, they made me realise that there is definitely a place for a Python that is different but mostly compatible with 2.7, in a way that 2.7.x could not be. For the sake of having a name, I'll refer to this as "Python 2migr8" (pronounced "to migrate" :) ). The two important components of Python 2migr8 would be the ability to disable 2.7-only features, and to do so on a module-by-module basis. My best idea so far would be to have a magic comment (to ensure 2.7 compatibility better than a "from __future__ ...") near the top of the file that marks that file as "must straddle 2.7 and 3.3". Adding this comment causes (for example) the parser to treat "except x, y" as a syntax error in this file, forces "from __future__ import ...", hides "dict.iterkeys", undefines "basestring", etc., but only for this file. (I haven't thought through all the possibilities or implications - Eric Snow said he was going to sprint on this today/tomorrow, so he'll soon have a better idea just what can be done.) In effect, 2migr8 would be the version that *only* supports "single-source" files. This allows large code bases to progressively migrate modules from 2.x to single-source while continuing to run against Python 2.7. As files are updated, they are marked as being single-source. When all files have this marker, it should be possible to flip the switch and run with Python 3.3 or later. You could also think of this as enabling "-3 --warnings-as-errors" for individual modules, though since the user has already opted in to 2migr8, it isn't unreasonable to make more significant changes, like having dict.keys returning a list that warns if it is mutated. This sort of warning can only really be done by changing the interpreter - static analysis just can't catch everything - and only when users accept a potential performance hit and low probability of breakage when they move to 2migr8 (followed by a not-quite-as-low probability of breaking when they eventually move from 2migr8 to 3.x, but it's still better than guaranteed breakage). As a fork, it would also be possible to bundle the modules that have been backported, and possibly also to disallow importing deprecated stdlib modules when 2.7 functionality is disabled. As I said, I haven't thought through all the possibilities, but the general idea is to take 2.7 and *remove* features so it becomes easier to migrate. Where does python-dev come in? Obviously this is where a fork like this would have to start - there has been such strong and public opposition to any significant changes like this that you'd be hard pressed to find someone willing to start and promote it from outside. There is also a good opportunity to make a start and directly invite those using it to contribute the rules or warnings that they need - the 3000 Python "users" I mentioned earlier are backed by a team of true developers who are more than capable of contributing, and this would be a great opportunity to directly invite them. However unfair and incorrect it may be, there is a perception in some businesses that open-source projects do not want contributions from them. I invited more than one business to have someone join python-dev and get involved during PyCon, and I heard that others did the same - it may not be at the level of employing a core developer full time, but it's the starting point that some companies will need to be able to become comfortable with employing a core dev. I'm not pretending to have a full plan on how this will work. I was privileged to have some private conversations during PyCon that are directly relevant, so I'm bringing it here to promote the discussion. Thanks to everyone I had a chance to chat to, and to everyone generally for a great PyCon. Cheers, Steve [View Less]

14 23

Appeal for reviews
by Nikolaus Rath April 15, 2014

April 15, 2014

Hello, I've accumulated a number of patches in the issue tracker that are waiting for someone to review/commit/reject them. I'm eager to make corrections as necessary, I just need someone to look the work that I've done so far: * http://bugs.python.org/issue20951 (SSLSocket.send() returns 0 for non-blocking socket) * http://bugs.python.org/issue1738 (filecmp.dircmp does exact match only) * http://bugs.python.org/issue7776 (http.client.HTTPConnection tunneling is broken) * http://bugs.… [View More]

12 26

Mercurial sluggishness (was: this is what happens if you freeze all the modules required for startup)
by Skip Montanaro April 15, 2014

April 15, 2014

Eric wrote: > Perhaps not so much "a very real advantage" as "less of a > distraction". It's still significantly slower than 2.7. :) I'm still confused. I peeked in /usr/bin/hg. The only "system" modules it imports directly are os and system (maybe I'm using an ancient version?). After that, it imports its own lazy import module. This suggests to me that Mercurial's import slowness is mostly in its own code (I counted 104 Python modules and six shared objects in its mercurial package, … [View More]which isn't going to be affected (much?) by freezing the Python standard modules. I'm not trying to be difficult here. I thought that way back in the day a huge amount of work was done to remove needless filesystem activity, and zip packaging has been around for quite awhile. As an experiment, I ran hg pull as /usr/bin/python -v /usr/bin/hg pull in my cpython repo then looked at the -v output. Summarizing the output I saw the following: 30 imports (0 dlopens) Python banner printed 86 imports (18 dlopens) adding changesets message 5 imports (2 dlopens) adding manifests message 1 import (0 dlopens) adding file changes message 7 imports (3 dlopens) added ... changesets message 4 imports (0 dlopens) run 'hg update' message (I missed a "searching" message in there somewhere.) That's a total of 133 imports, 23 of which were satisfied by loading an extension module. The imports break down as follows: 51 imports (4 dlopens) from the mercurial package 5 imports from the hgext package 7 imports from the encodings package Everything else is imported from the top level, and at a glance appears to be all Python stdlib stuff. The key period of execution looks to be between the printing of the Python banner and the printing of the adding changesets message. I see 46 imports (2 dlopens) from the mercurial or hgext packages. That leaves 40 stdlib imports, of which 16 were satisfied by dlopen. As a final check, I saved all the stdlib import statements from the -v output (77 statements) to a file and timed its execution: % time /usr/bin/python ~/tmp/pyimp.py real 0m0.162s user 0m0.034s sys 0m0.010s It doesn't take much time to import every stdlib module that Mercurial needs. I don't know how much slower all this import machinery is in 3.x (and can't test at work, as we don't have a copy laying about). It would probably have to be 3x or more slower for it to have much visible impact on even simple hg commands. I find it hard to believe that freezing the stdlib is going to lower the barrier enough for the Mercurial folks, if, in fact, import slowness is their main reason for not moving to 3.x. Skip [View Less]

8 14

[numpy wishlist] PyMem_*Calloc
by Nathaniel Smith April 15, 2014

April 15, 2014

Hi all, The new tracemalloc infrastructure in python 3.4 is super-interesting to numerical folks, because we really like memory profiling. Numerical programs allocate a lot of memory, and sometimes it's not clear which operations allocate memory (some numpy operations return views of the original array without allocating anything; others return copies). So people actually use memory tracking tools[1], even though traditionally these have been pretty hacky (i.e., just checking RSS before and … [View More]after each line is executed), and numpy has even grown its own little tracemalloc-like infrastructure [2], but it only works for numpy data. BUT, we also really like calloc(). One of the basic array creation routines in numpy is numpy.zeros(), which returns an array full of -- you guessed it -- zeros. For pretty much all the data types numpy supports, the value zero is represented by the bytestring consisting of all zeros. So numpy.zeros() usually uses calloc() to allocate its memory. calloc() is more awesome than malloc()+memset() for two reasons. First, calloc() for larger allocations is usually implemented using clever VM tricks, so that it doesn't actually allocate any memory up front, it just creates a COW mapping of the system zero page and then does the actual allocation one page at a time as different entries are written to. This means that in the somewhat common case where you allocate a large array full of zeros, and then only set a few scattered entries to non-zero values, you can end up using much much less memory than otherwise. It's entirely possible for this to make the difference between being able to run an analysis versus not. memset() forces the whole amount of RAM to be committed immediately. Secondly, even if you *are* going to touch all the memory, then calloc() is still faster than malloc()+memset(). The reason is that for large allocations, malloc() usually does a calloc() no matter what -- when you get a new page from the kernel, the kernel has to make sure you can't see random bits of other processes's memory, so it unconditionally zeros out the page before you get to see it. calloc() knows this, so it doesn't bother zeroing it again. malloc()+memset(), by contrast, zeros the page twice, producing twice as much memory traffic, which is huge. SO, we'd like to route our allocations through PyMem_* in order to let tracemalloc "see" them, but because there is no PyMem_*Calloc, doing this would force us to give up on the calloc() optimizations. The obvious solution is to add a PyMem_*Calloc to the API. Would this be possible? Unfortunately it would require adding a new field to the PyMemAllocator struct, which would be an ABI/API break; PyMemAllocator is exposed directly in the C API and passed by value: https://docs.python.org/3.5/c-api/memory.html#c.PyMemAllocator (Too bad we didn't notice this a few months ago before 3.4 was released :-(.) I guess we could just rename the struct in 3.5, to force people to update their code. (I guess there aren't too many people who would have to update their code.) Thoughts? -n [1] http://scikit-learn.org/stable/developers/performance.html#memory-usage-pro… [2] https://github.com/numpy/numpy/pull/309 -- Nathaniel J. Smith Postdoctoral researcher - Informatics - University of Edinburgh http://vorpus.org [View Less]

5 5

Re: [Python-Dev] [Python-checkins] cpython (3.1): disallow a negative idx parameter
by Zachary Ware April 14, 2014

April 14, 2014

On Mon, Apr 14, 2014 at 10:52 AM, benjamin.peterson <python-checkins(a)python.org> wrote: > http://hg.python.org/cpython/rev/4bd1fb0f4f44 > changeset: 90256:4bd1fb0f4f44 > branch: 3.1 > parent: 90235:a8facac493ef > user: Benjamin Peterson <benjamin(a)python.org> > date: Mon Apr 14 11:45:21 2014 -0400 > summary: > disallow a negative idx parameter > > files: > Modules/_json.c | 9 +++++---- > 1 files changed, 5 … [View More]

2 1

death to 2.7; long live 2.7
by Benjamin Peterson April 14, 2014

April 14, 2014

This email is to share idea that has been bouncing around in my head for a while about 2.7 releases. Guido's last email containing notes from the language summit made me think it's time to propose it. We'll keep doing what we're currently doing for another year, making normal bug fix releases with installers. After that, we _won't_ close 2.7 to normal bug fixes as is currently implied by the release schedule. Instead dealing 2.7 will just be completely optional for core developers. (The much … [View More]

12 21

your next regular installment of python 2.7
by Benjamin Peterson April 14, 2014

April 14, 2014

Yep, it's almost that time of the year again. My proposed schedule for 2.7.7 is May 12th, 2.7.7rc1 May 26th, 2.7.7 final That means if you want your 2.7 improvement to see the light of day before November, the next few weeks are the time to do it. (PEP 446, I'm looking at you.) Also, I doubt anyone cares, but I'm going to do the last security release of python 3.1 concurrently. That will, of course, be a source-only release. Regards, Benjamin

1 0

Fwd: Jython Report
by Michael Foord April 14, 2014

April 14, 2014

Below is the Jython "status update" report on Jython I received from Jim Baker and summarised in the Language Summit. It comes with one addendum from Frank: Jim's list is fantastic - the one bit I'd like to add to the list: Jython now supports a buffer protocol that parallels CPython's C API buffer protocol. This provided the basis for support of buffer() and memoryview(). The work was done with Jython3 in mind and will be a huge boost to that eventual effort. Begin forwarded message: > … [View More]From: Jim Baker <jbaker(a)zyasoft.com> > Subject: Re: Jython Report > Date: 7 April 2014 06:42:51 BST > To: Michael Foord <michael(a)voidspace.org.uk> > Cc: Frank Wierzbicki <fwierzbicki(a)gmail.com> > > Recent changes to trunk (last 6 months) > > * Recently tagged a soft beta 2! > * Java 7 JVM is now the minimum version, which gives a larger base of functionality to work with (such as using Java 7's AutoCloseable to imply corresponding context manager support in using Python code) > * Enable mixing Python and Java types in the bases of a class when using a metaclass > * Added support for buffer, memoryview, although not complete yet with respect to Java integration > * Console and encoding support, such as unicodedata/idna updates > * Many, many small fixes > > About to be in trunk, to support beta 3 > > * socket-reboot reimplements socket/select/ssl on top of Netty 4, a popular event loop networking framework for the JVM (used by a large number of performant projects in Java space and originally part of JBoss). There was no ssl support before, but now socket and especially select semantics are much closer to CPython as well (basically close to the Windows socket model). > * socket-reboot in turn enables requests and thereby pip. A branch of pip currently works, actually modifying an upstream vendor lib (html5lib) so that it doesn't use isolated UTF-16 surrogates in literals, since this is not actually legal unicode, nor does it work in Jython's UTF-16 based representation. Ironically this usage is to detect such illegal use in input streams. > * Relative star imports, which seems to impact a number of interesting projects. > * Performance tuning of sre. Jython has a port of CPython's sre, however our use of UTF-16 requires expansion into an array of codepoints. Currently this is done on demand, which can potentially add another O(n) factor in evaluating regexes. A pull request we will apply memoizes. In the future, we will rewrite the logic in sre so that it does next/prev, much like JRuby currently does for similar encoding issues. > > Related work > > * Other PyPA tooling including virtualenv and wheel needs more diagnosis to see why they currently fail on Jython, but our hope is that this is minor. > * New project jythontools by a number of Jython developers (including Frank and Jim). This includes a number of projects that will help evolve Jython, but outside the usual release schedule and the usual problem of being in core (such as eventual deprecation): > - Clamp - precise integration with Java, enabling such capabilities as Java directly importing Python modules without explicitly initializing the Jython runtime or using object factories. Future work will enable Java annotation integration, as decorators. Integrates with setuptools; future integration as well with Maven via Aether. > - Jiffy - provide a CFFI backend for Jython. Right now it is pure vaporware, but cursory examination of cffi.backend_ctypes suggests that it should be straightforward and of modest effort to provide a similar backend by using JFFI, which Jython and JRuby both use to access native runtime services (such as Posix API) as part of the Java native runtime project. > * The Patois project has been started to collect examples for cross-implementation support, as seen in surrogate support, but it will be a good question to get that really going, vs just talking about it. > * JyNI - simply adding this jar to the classpath enables C extension API support. Note that this project has been licensed by its developer (not a Jython committer) under an LGPL license. > > Release schedule > > * Complete beta 2 > * Beta 3 is forthcoming, likely in 2 weeks > * For beta 4, need to perform a comprehensive bug triage - what will be in, not in for 2.7.0 > * EuroPython sprint to finalize a release candidate for 2.7.0? > > Future > > * Mostly around performance, Java integration, and of course the usual bug fixes > * Python bytecode compiler remains important, including for support targeting Android and removing restriction on getting too large a method for the JVM > * More hooks for Java integration, including managing generated bytecode > * Integrating Zippy could provide for PyPy-like performance, but requires Graal JVM > * Supporting invokedynamic is a more realistic solution, but without the use of annotations (eg turn off Python frames) is going to be limited (maybe 2x?) based on earlier analysis > > Jython 3.x? > > * This comes up periodically, and it would be super nice for us to complete this support. At the very least it would make unicode strings and bytestrings correspond directly to how they are represented in Java, so that will be a nice cleanup. > * Release schedule: we will get there at some point! > > > On Sun, Apr 6, 2014 at 5:20 PM, Jim Baker <jbaker(a)zyasoft.com> wrote: > Michael, > > I was thinking about this very topic this morning! Will send you the latest status in the next 24h, specifically our work to support pypa (setuptools, pip, virtualenv, wheel) and related tooling. > > - Jim > > > On Sun, Apr 6, 2014 at 11:30 AM, Michael Foord <michael(a)voidspace.org.uk> wrote: > Hey guys, > > Would you be able to write up a brief report on the current and future status of Jython, for me to read out at the Python language summit on Wednesday? (Unless someone who works on Jython will be there - but as far as I know that isn't the case.) > > All the best, > > Michael Foord > > -- > http://www.voidspace.org.uk/ > > > May you do good and not evil > May you find forgiveness for yourself and forgive others > May you share freely, never taking more than you give. > -- the sqlite blessing > http://www.sqlite.org/different.html > > > > > > > > -- http://www.voidspace.org.uk/ May you do good and not evil May you find forgiveness for yourself and forgive others May you share freely, never taking more than you give. -- the sqlite blessing http://www.sqlite.org/different.html [View Less]

2 1

Windows installers and OpenSSL
by Paul Moore April 13, 2014

April 13, 2014

Given the OpenSSL vulnerability and the fact that we bundle OpenSSL with the Windows installers (1.0.1e in Python 3.4.0) should we be releasing updated installers? Paul

8 8

python-3.4.0
by Greg Mildenstein April 13, 2014

April 13, 2014

Hi, I'm running windows 8.1 64 bit. I was using 'python-3.3.1.amd64' but have uninstalled it and installed the above version. However, when I try running 'PyScripter', I get an error stating, 'ERROR - unable to open python, it will now exit'. Is there something I'm doing wrong? Thanks Greg Mildenstein

2 1

flock in Python 3
by Jakub QB Dorňák April 13, 2014

April 13, 2014

Hi all, writing a threaded application I've been surprised that there is no object api for fcntl.flock implementing __enter__() and __exit__() methods to be used with 'with' statement. I have written one (https://pypi.python.org/pypi), but I wonder whether this could get into the Python Standard Library. Regards, QB

3 3

PEP 461: Adding % formatting to bytes and bytearray -- Final, Take 3
by Ethan Furman April 12, 2014

April 12, 2014

Okay, I included that last round of comments (from late February). Barring typos, this should be the final version. Final comments? ----------------------------------------------------------------------------- PEP: 461 Title: Adding % formatting to bytes and bytearray Version: $Revision$ Last-Modified: $Date$ Author: Ethan Furman <ethan(a)stoneleaf.us> Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 2014-01-13 Python-Version: 3.5 Post-History: 2014-01-14, 2014-01-… [View More]15, 2014-01-17, 2014-02-22, 2014-03-25 Resolution: Abstract ======== This PEP proposes adding % formatting operations similar to Python 2's ``str`` type to ``bytes`` and ``bytearray`` [1]_ [2]_. Rationale ========= While interpolation is usually thought of as a string operation, there are cases where interpolation on ``bytes`` or ``bytearrays`` make sense, and the work needed to make up for this missing functionality detracts from the overall readability of the code. Motivation ========== With Python 3 and the split between ``str`` and ``bytes``, one small but important area of programming became slightly more difficult, and much more painful -- wire format protocols [3]_. This area of programming is characterized by a mixture of binary data and ASCII compatible segments of text (aka ASCII-encoded text). Bringing back a restricted %-interpolation for ``bytes`` and ``bytearray`` will aid both in writing new wire format code, and in porting Python 2 wire format code. Common use-cases include ``dbf`` and ``pdf`` file formats, ``email`` formats, and ``FTP`` and ``HTTP`` communications, among many others. Proposed semantics for ``bytes`` and ``bytearray`` formatting ============================================================= %-interpolation --------------- All the numeric formatting codes (such as ``%x``, ``%o``, ``%e``, ``%f``, ``%g``, etc.) will be supported, and will work as they do for str, including the padding, justification and other related modifiers. The only difference will be that the results from these codes will be ASCII-encoded text, not unicode. In other words, for any numeric formatting code `%x`:: b"%x" % val is equivalent to ("%x" % val).encode("ascii") Examples:: >>> b'%4x' % 10 b' a' >>> b'%#4x' % 10 ' 0xa' >>> b'%04X' % 10 '000A' ``%c`` will insert a single byte, either from an ``int`` in range(256), or from a ``bytes`` argument of length 1, not from a ``str``. Examples:: >>> b'%c' % 48 b'0' >>> b'%c' % b'a' b'a' ``%s`` is included for two reasons: 1) `b` is already a format code for ``format`` numerics (binary), and 2) it will make 2/3 code easier as Python 2.x code uses ``%s``; however, it is restricted in what it will accept:: - input type supports ``Py_buffer`` [6]_? use it to collect the necessary bytes - input type is something else? use its ``__bytes__`` method [7]_ ; if there isn't one, raise a ``TypeError`` In particular, ``%s`` will not accept numbers (use a numeric format code for that), nor ``str`` (encode it to ``bytes``). Examples:: >>> b'%s' % b'abc' b'abc' >>> b'%s' % 'some string'.encode('utf8') b'some string' >>> b'%s' % 3.14 Traceback (most recent call last): ... TypeError: b'%s' does not accept numbers, use a numeric code instead >>> b'%s' % 'hello world!' Traceback (most recent call last): ... TypeError: b'%s' does not accept 'str', it must be encoded to `bytes` ``%a`` will call ``ascii()`` on the interpolated value. This is intended as a debugging aid, rather than something that should be used in production. Non-ASCII values will be encoded to either ``\xnn`` or ``\unnnn`` representation. Use cases include developing a new protocol and writing landmarks into the stream; debugging data going into an existing protocol to see if the problem is the protocol itself or bad data; a fall-back for a serialization format; or even a rudimentary serialization format when defining ``__bytes__`` would not be appropriate [8]. .. note:: If a ``str`` is passed into ``%a``, it will be surrounded by quotes. Unsupported codes ----------------- ``%r`` (which calls ``__repr__`` and returns a ``str``) is not supported. Proposed variations =================== It was suggested to let ``%s`` accept numbers, but since numbers have their own format codes this idea was discarded. It has been suggested to use ``%b`` for bytes as well as ``%s``. This was rejected as not adding any value either in clarity or simplicity. It has been proposed to automatically use ``.encode('ascii','strict')`` for ``str`` arguments to ``%s``. - Rejected as this would lead to intermittent failures. Better to have the operation always fail so the trouble-spot can be correctly fixed. It has been proposed to have ``%s`` return the ascii-encoded repr when the value is a ``str`` (b'%s' % 'abc' --> b"'abc'"). - Rejected as this would lead to hard to debug failures far from the problem site. Better to have the operation always fail so the trouble-spot can be easily fixed. Originally this PEP also proposed adding format-style formatting, but it was decided that format and its related machinery were all strictly text (aka ``str``) based, and it was dropped. Various new special methods were proposed, such as ``__ascii__``, ``__format_bytes__``, etc.; such methods are not needed at this time, but can be visited again later if real-world use shows deficiencies with this solution. Objections ========== The objections raised against this PEP were mainly variations on two themes:: - the ``bytes`` and ``bytearray`` types are for pure binary data, with no assumptions about encodings - offering %-interpolation that assumes an ASCII encoding will be an attractive nuisance and lead us back to the problems of the Python 2 ``str``/``unicode`` text model As was seen during the discussion, ``bytes`` and ``bytearray`` are also used for mixed binary data and ASCII-compatible segments: file formats such as ``dbf`` and ``pdf``, network protocols such as ``ftp`` and ``email``, etc. ``bytes`` and ``bytearray`` already have several methods which assume an ASCII compatible encoding. ``upper()``, ``isalpha()``, and ``expandtabs()`` to name just a few. %-interpolation, with its very restricted mini-language, will not be any more of a nuisance than the already existing methods. Some have objected to allowing the full range of numeric formatting codes with the claim that decimal alone would be sufficient. However, at least two formats (dbf and pdf) make use of non-decimal numbers. Footnotes ========= .. [1] http://docs.python.org/2/library/stdtypes.html#string-formatting .. [2] neither string.Template, format, nor str.format are under consideration .. [3] https://mail.python.org/pipermail/python-dev/2014-January/131518.html .. [4] to use a str object in a bytes interpolation, encode it first .. [5] %c is not an exception as neither of its possible arguments are str .. [6] http://docs.python.org/3/c-api/buffer.html examples: ``memoryview``, ``array.array``, ``bytearray``, ``bytes`` .. [7] http://docs.python.org/3/reference/datamodel.html#object.__bytes__ .. [8] https://mail.python.org/pipermail/python-dev/2014-February/132750.html Copyright ========= This document has been placed in the public domain. .. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 coding: utf-8 End: [View Less]

15 51

arguments policy: **kwargs.pop()
by Christian Tismer April 12, 2014

April 12, 2014

Hi guys, I tried to find advice for hours, but failed so fer, so here is my question: Whenever I think to adopt a new module that does a good job, I always can't stand the temptation to look at the coding style and certain principles. Then I rather often see things like this: class someclass(object): # note that there is no comment about argument destruction... def __init__(self, **kwargs): first_arg = kwargs.pop('option_1', somedefault) ... nth_arg = kwargs.… [View More]

6 9

Receiving email updates for the bug tracker
by Antony Lee April 11, 2014

April 11, 2014

Hi, Sorry for the slightly off-topic(?) question but I would like to know how to receive email notifications for activity on bugs I've opened on the bugs.python.org -- so far I don't receive anything even though I'm on the nosy list. Thanks, Antony

3 3

Re: [Python-Dev] Receiving email updates for the bug tracker
by Antony Lee April 11, 2014

April 11, 2014

Nope, the email is correct... 2014-04-11 12:12 GMT-07:00 Brett Cannon <brett(a)python.org>: > > > > On Fri, Apr 11, 2014 at 1:47 PM, Antony Lee <antony.lee(a)berkeley.edu>wrote: > >> Hi, >> Sorry for the slightly off-topic(?) question but I would like to know how >> to receive email notifications for activity on bugs I've opened on the >> bugs.python.org -- so far I don't receive anything even though I'm on >> the nosy list. >> &… [View More]

2 1

Summary of Python tracker Issues
by Python tracker April 11, 2014

April 11, 2014

ACTIVITY SUMMARY (2014-04-04 - 2014-04-11) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open 4567 (+17) closed 28405 (+29) total 32972 (+46) Open issues with patches: 2096 Issues opened (32) ================== #21015: support SSL_CTX_set_ecdh_auto on newer OpenSSLs http://bugs.python.org/issue21015 reopened by ned.deily #21156: Consider moving … [View More]

1 0

PEP 465: A dedicated infix operator for matrix multiplication
by Nathaniel Smith April 10, 2014

April 10, 2014

Hi all, I've just finished updating PEP 465 with resolutions to the various issues that were raised during the python-ideas thread about it. (The main changes since that thread are that @@ has been removed, and we now definitely propose that @ have the same precedence and associativity as *.) I believe this leaves only one open question, which is where exactly to stick the new matmul slots into PyTypeObject. This is the kind of fiddly detail that can easily be settled later if the PEP is … [View More]

23 59

A Friendly IDLE
by adnanumer95＠gmail.com April 10, 2014

April 10, 2014

Greeting Everyone. First of all I want to introduce my self Adnan Umer as a student of bachelors in Information Technology. I’ve few suggestions on improving IDLE. Here are few: On windows we can open any python file from context menu because IDLE is not a application. I recommends to create a simple executable that just calls ‘idle.pyw’ module in lib\idlelib. On executing python script with IDLE we can’t determine which file is executed. I recommends to print file name before executing. … [View More]

4 3

issue with itertools leads the crash
by Kirill April 10, 2014

April 10, 2014

issue with itertools leads the crash The following code leads to system failure and crash on Ubuntu 12.04.3 with Python 2.7.6. import itertools r = range (1,10) r[1:9:2] = itertools.cycle([0]) Kirill .

10 11

Windows buildbots are red: test_idlelib
by Victor Stinner April 8, 2014

April 8, 2014

Hi, Unit tests are failing on Windows because of this issue: http://bugs.python.org/issue21059 It looks like a regression in test_idlelib introduced with this issue: http://bugs.python.org/issue15968 Zachary Ware wrote a fix: http://bugs.python.org/issue20035 Can someone please review Zachary's patch? If not, I suggest to revert changes of issue #15968 to have working Windows buildbots. Victor

2 2

Re: [Python-Dev] [Python-checkins] cpython (3.4): docs: Document __objclass__. Closes #19281.
by Ethan Furman April 8, 2014

April 8, 2014

Sorry, posted to wrong list the first time. On 04/08/2014 09:33 AM, Ethan Furman wrote: > On 04/08/2014 09:07 AM, yury.selivanov wrote: >> http://hg.python.org/cpython/rev/0973d45197cc > > >> +The :attr:`__objclass__` is interpreted by the :mod:`inspect` module as >> +specifying the class where this object was defined (setting this >> appropriately >> +can assist in runtime introspection of dynamic class attributes). For >> callables, >> +it … [View More]

2 1

Re: [Python-Dev] [Python-checkins] cpython (3.4): asyncio: Document Task.cancel() properly.
by Terry Reedy April 7, 2014

April 7, 2014

On 4/7/2014 5:22 AM, victor.stinner wrote: > def cancel(self): > + """Request that a task to cancel itself. For proper English, this should be one of these: "Request that a task cancel itself." "Request a task to cancel itself." I think the first is slightly better. TJR

1 0

Re: [Python-Dev] Python-Dev Digest, Vol 129, Issue 6
by Kells Pablo April 7, 2014

April 7, 2014

HELLO... !thank you for all the cooperation and emails send. i would like that you now stop sending them.. thank you in advance On Fri, Apr 4, 2014 at 4:22 PM, <python-dev-request(a)python.org> wrote: > Send Python-Dev mailing list submissions to > python-dev(a)python.org > > To subscribe or unsubscribe via the World Wide Web, visit > https://mail.python.org/mailman/listinfo/python-dev > or, via email, send a message with subject or body 'help' to &… [View More]gt; python-dev-request(a)python.org > > You can reach the person managing the list at > python-dev-owner(a)python.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of Python-Dev digest..." > > > Today's Topics: > > 1. Re: Pyston: a Python JIT on LLVM (Dag Sverre Seljebotn) > 2. Re: [Python-checkins] cpython: fix #21076: turn signal module > constants into enums (Brett Cannon) > 3. Re: [Python-checkins] cpython: fix #21076: turn signal module > constants into enums (Brett Cannon) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Fri, 04 Apr 2014 12:13:53 +0200 > From: Dag Sverre Seljebotn <d.s.seljebotn(a)astro.uio.no> > To: python-dev(a)python.org, kmod(a)dropbox.com > Subject: Re: [Python-Dev] Pyston: a Python JIT on LLVM > Message-ID: <533E85E1.7050001(a)astro.uio.no> > Content-Type: text/plain; charset=ISO-8859-1; format=flowed > > On 04/04/2014 12:42 AM, Sturla Molden wrote: > > Kevin Modzelewski <kmod(a)dropbox.com> wrote: > > > >> Since it's the question that I think most people will inevitably (and > >> rightly) ask, why do we think there's a place for Pyston when there's > PyPy > >> and (previously) Unladen Swallow? > > > > Have you seen Numba, the Python JIT that integrates with NumPy? > > > > http://numba.pydata.org > > Specifically, Numba compiles to LLVM too, and tries to be somewhat > general-purpose although it's tuned to numerical code. And their reason > for not using PyPy is the same: C extensions. So while their "market > segment" is different from yours, the technology may not be. > > Dag Sverre > > > > > > It uses LLVM to compile Python bytecode. When I have tried it I tend to > get > > speed comparable to -O2 in C for numerical and algorithmic code. > > > > Here is an example, giving a 150 times speed boost to Python: > > > > > http://stackoverflow.com/questions/21811381/how-to-shove-this-loop-into-num… > > > > > > Sturla > > > > _______________________________________________ > > Python-Dev mailing list > > Python-Dev(a)python.org > > https://mail.python.org/mailman/listinfo/python-dev > > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/d.s.seljebotn%40astro.ui… > > > > > > ------------------------------ > > Message: 2 > Date: Fri, 4 Apr 2014 10:12:23 -0400 > From: Brett Cannon <brett(a)python.org> > To: python-dev <python-dev(a)python.org> > Cc: python-checkins <python-checkins(a)python.org> > Subject: Re: [Python-Dev] [Python-checkins] cpython: fix #21076: turn > signal module constants into enums > Message-ID: > <CAP1=2W7Qif5qhND= > VA+rOnt1+fGFM75Rp-K9aN0T3oGZweXMUA(a)mail.gmail.com> > Content-Type: text/plain; charset="utf-8" > > This broke compilation on at least OS X, but I'm willing to bet for all > UNIX-based systems. I have a fix in the works. > > > On Fri, Apr 4, 2014 at 9:34 AM, giampaolo.rodola < > python-checkins(a)python.org > > wrote: > > > http://hg.python.org/cpython/rev/c9239171e429 > > changeset: 90128:c9239171e429 > > user: Giampaolo Rodola' <g.rodola(a)gmail.com> > > date: Fri Apr 04 15:34:17 2014 +0200 > > summary: > > fix #21076: turn signal module constants into enums > > > > files: > > Doc/library/signal.rst | 10 +++ > > Doc/whatsnew/3.5.rst | 5 + > > Lib/signal.py | 84 ++++++++++++++++++++++++++++ > > Lib/test/test_doctest.py | 2 +- > > Lib/test/test_signal.py | 39 +++++++++++- > > Modules/signalmodule.c | 4 +- > > PC/config.c | 2 +- > > 7 files changed, 138 insertions(+), 8 deletions(-) > > > > > > diff --git a/Doc/library/signal.rst b/Doc/library/signal.rst > > --- a/Doc/library/signal.rst > > +++ b/Doc/library/signal.rst > > @@ -65,6 +65,16 @@ > > Module contents > > --------------- > > > > +.. versionchanged:: 3.5 > > + signal (SIG*), handler (:const:`SIG_DFL`, :const:`SIG_IGN`) and > sigmask > > + (:const:`SIG_BLOCK`, :const:`SIG_UNBLOCK`, :const:`SIG_SETMASK`) > > + related constants listed below were turned into > > + :class:`enums <enum.IntEnum>`. > > + :func:`getsignal`, :func:`pthread_sigmask`, :func:`sigpending` and > > + :func:`sigwait` functions return human-readable > > + :class:`enums <enum.IntEnum>`. > > + > > + > > The variables defined in the :mod:`signal` module are: > > > > > > diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst > > --- a/Doc/whatsnew/3.5.rst > > +++ b/Doc/whatsnew/3.5.rst > > @@ -134,6 +134,11 @@ > > Improved Modules > > ================ > > > > +* Different constants of :mod:`signal` module are now enumeration values > > using > > + the :mod:`enum` module. This allows meaningful names to be printed > > during > > + debugging, instead of integer ?magic numbers?. (contribute by > Giampaolo > > + Rodola' in :issue:`21076`) > > + > > * :class:`xmlrpc.client.ServerProxy` is now a :term:`context manager` > > (contributed by Claudiu Popa in :issue:`20627`). > > > > diff --git a/Lib/signal.py b/Lib/signal.py > > new file mode 100644 > > --- /dev/null > > +++ b/Lib/signal.py > > @@ -0,0 +1,84 @@ > > +import _signal > > +from _signal import * > > +from functools import wraps as _wraps > > +from enum import IntEnum as _IntEnum > > + > > +_globals = globals() > > + > > +Signals = _IntEnum( > > + 'Signals', > > + {name: value for name, value in _globals.items() > > + if name.isupper() > > + and (name.startswith('SIG') and not name.startswith('SIG_')) > > + or name.startswith('CTRL_')}) > > + > > +class Handlers(_IntEnum): > > + SIG_DFL = _signal.SIG_DFL > > + SIG_IGN = _signal.SIG_IGN > > + > > +_globals.update(Signals.__members__) > > +_globals.update(Handlers.__members__) > > + > > +if 'pthread_sigmask' in _globals: > > + class Sigmasks(_IntEnum): > > + SIG_BLOCK = _signal.SIG_BLOCK > > + SIG_UNBLOCK = _signal.SIG_UNBLOCK > > + SIG_SETMASK = _signal.SIG_SETMASK > > + > > + _globals.update(Sigmasks.__members__) > > + > > + > > +def _int_to_enum(value, enum_klass): > > + """Convert a numeric value to an IntEnum member. > > + If it's not a known member, return the numeric value itself. > > + """ > > + try: > > + return enum_klass(value) > > + except ValueError: > > + return value > > + > > + > > +def _enum_to_int(value): > > + """Convert an IntEnum member to a numeric value. > > + If it's not a IntEnum member return the value itself. > > + """ > > + try: > > + return int(value) > > + except (ValueError, TypeError): > > + return value > > + > > + > > +(a)_wraps(_signal.signal) > > +def signal(signalnum, handler): > > + handler = _signal.signal(_enum_to_int(signalnum), > > _enum_to_int(handler)) > > + return _int_to_enum(handler, Handlers) > > + > > + > > +(a)_wraps(_signal.getsignal) > > +def getsignal(signalnum): > > + handler = _signal.getsignal(signalnum) > > + return _int_to_enum(handler, Handlers) > > + > > + > > +if 'pthread_sigmask' in _globals: > > + @_wraps(_signal.pthread_sigmask) > > + def pthread_sigmask(how, mask): > > + sigs_set = _signal.pthread_sigmask(how, mask) > > + return set(_int_to_enum(x, Signals) for x in sigs_set) > > + pthread_sigmask.__doc__ = _signal.pthread_sigmask.__doc__ > > + > > + > > +(a)_wraps(_signal.sigpending) > > +def sigpending(): > > + sigs = _signal.sigpending() > > + return set(_int_to_enum(x, Signals) for x in sigs) > > + > > + > > +if 'sigwait' in _globals: > > + @_wraps(_signal.sigwait) > > + def sigwait(sigset): > > + retsig = _signal.sigwait(sigset) > > + return _int_to_enum(retsig, Signals) > > + sigwait.__doc__ = _signal.sigwait > > + > > +del _globals, _wraps > > diff --git a/Lib/test/test_doctest.py b/Lib/test/test_doctest.py > > --- a/Lib/test/test_doctest.py > > +++ b/Lib/test/test_doctest.py > > @@ -2897,7 +2897,7 @@ > > > > def test_main(): > > # Check the doctest cases in doctest itself: > > - support.run_doctest(doctest, verbosity=True) > > + ret = support.run_doctest(doctest, verbosity=True) > > # Check the doctest cases defined here: > > from test import test_doctest > > support.run_doctest(test_doctest, verbosity=True) > > diff --git a/Lib/test/test_signal.py b/Lib/test/test_signal.py > > --- a/Lib/test/test_signal.py > > +++ b/Lib/test/test_signal.py > > @@ -1,6 +1,7 @@ > > import unittest > > from test import support > > from contextlib import closing > > +import enum > > import gc > > import pickle > > import select > > @@ -39,6 +40,22 @@ > > return None > > > > > > +class GenericTests(unittest.TestCase): > > + > > + def test_enums(self): > > + for name in dir(signal): > > + sig = getattr(signal, name) > > + if name in {'SIG_DFL', 'SIG_IGN'}: > > + self.assertIsInstance(sig, signal.Handlers) > > + elif name in {'SIG_BLOCK', 'SIG_UNBLOCK', 'SIG_SETMASK'}: > > + self.assertIsInstance(sig, signal.Sigmasks) > > + elif name.startswith('SIG') and not name.startswith('SIG_'): > > + self.assertIsInstance(sig, signal.Signals) > > + elif name.startswith('CTRL_'): > > + self.assertIsInstance(sig, signal.Signals) > > + self.assertEqual(sys.platform, "win32") > > + > > + > > @unittest.skipIf(sys.platform == "win32", "Not valid on Windows") > > class InterProcessSignalTests(unittest.TestCase): > > MAX_DURATION = 20 # Entire test should last at most 20 sec. > > @@ -195,6 +212,7 @@ > > > > def test_getsignal(self): > > hup = signal.signal(signal.SIGHUP, self.trivial_signal_handler) > > + self.assertIsInstance(hup, signal.Handlers) > > self.assertEqual(signal.getsignal(signal.SIGHUP), > > self.trivial_signal_handler) > > signal.signal(signal.SIGHUP, hup) > > @@ -271,7 +289,7 @@ > > > > os.close(read) > > os.close(write) > > - """.format(signals, ordered, test_body) > > + """.format(tuple(map(int, signals)), ordered, test_body) > > > > assert_python_ok('-c', code) > > > > @@ -604,6 +622,8 @@ > > signal.pthread_sigmask(signal.SIG_BLOCK, [signum]) > > os.kill(os.getpid(), signum) > > pending = signal.sigpending() > > + for sig in pending: > > + assert isinstance(sig, signal.Signals), repr(pending) > > if pending != {signum}: > > raise Exception('%s != {%s}' % (pending, signum)) > > try: > > @@ -660,6 +680,7 @@ > > code = '''if 1: > > import signal > > import sys > > + from signal import Signals > > > > def handler(signum, frame): > > 1/0 > > @@ -702,6 +723,7 @@ > > def test(signum): > > signal.alarm(1) > > received = signal.sigwait([signum]) > > + assert isinstance(received, signal.Signals), received > > if received != signum: > > raise Exception('received %s, not %s' % (received, > > signum)) > > ''') > > @@ -842,8 +864,14 @@ > > def kill(signum): > > os.kill(os.getpid(), signum) > > > > + def check_mask(mask): > > + for sig in mask: > > + assert isinstance(sig, signal.Signals), repr(sig) > > + > > def read_sigmask(): > > - return signal.pthread_sigmask(signal.SIG_BLOCK, []) > > + sigmask = signal.pthread_sigmask(signal.SIG_BLOCK, []) > > + check_mask(sigmask) > > + return sigmask > > > > signum = signal.SIGUSR1 > > > > @@ -852,6 +880,7 @@ > > > > # Unblock SIGUSR1 (and copy the old mask) to test our signal > > handler > > old_mask = signal.pthread_sigmask(signal.SIG_UNBLOCK, [signum]) > > + check_mask(old_mask) > > try: > > kill(signum) > > except ZeroDivisionError: > > @@ -861,11 +890,13 @@ > > > > # Block and then raise SIGUSR1. The signal is blocked: the > signal > > # handler is not called, and the signal is now pending > > - signal.pthread_sigmask(signal.SIG_BLOCK, [signum]) > > + mask = signal.pthread_sigmask(signal.SIG_BLOCK, [signum]) > > + check_mask(mask) > > kill(signum) > > > > # Check the new mask > > blocked = read_sigmask() > > + check_mask(blocked) > > if signum not in blocked: > > raise Exception("%s not in %s" % (signum, blocked)) > > if old_mask ^ blocked != {signum}: > > @@ -928,7 +959,7 @@ > > > > def test_main(): > > try: > > - support.run_unittest(PosixTests, InterProcessSignalTests, > > + support.run_unittest(GenericTests, PosixTests, > > InterProcessSignalTests, > > WakeupFDTests, WakeupSignalTests, > > SiginterruptTest, ItimerTest, > > WindowsSignalTests, > > PendingSignalsTests) > > diff --git a/Modules/signalmodule.c b/Modules/signalmodule.c > > --- a/Modules/signalmodule.c > > +++ b/Modules/signalmodule.c > > @@ -967,7 +967,7 @@ > > }; > > > > PyMODINIT_FUNC > > -PyInit_signal(void) > > +PyInit__signal(void) > > { > > PyObject *m, *d, *x; > > int i; > > @@ -1380,7 +1380,7 @@ > > void > > PyOS_InitInterrupts(void) > > { > > - PyObject *m = PyImport_ImportModule("signal"); > > + PyObject *m = PyImport_ImportModule("_signal"); > > if (m) { > > Py_DECREF(m); > > } > > diff --git a/PC/config.c b/PC/config.c > > --- a/PC/config.c > > +++ b/PC/config.c > > @@ -19,7 +19,7 @@ > > extern PyObject* PyInit__md5(void); > > extern PyObject* PyInit_nt(void); > > extern PyObject* PyInit__operator(void); > > -extern PyObject* PyInit_signal(void); > > +extern PyObject* PyInit__signal(void); > > extern PyObject* PyInit__sha1(void); > > extern PyObject* PyInit__sha256(void); > > extern PyObject* PyInit__sha512(void); > > > > -- > > Repository URL: http://hg.python.org/cpython > > > > _______________________________________________ > > Python-checkins mailing list > > Python-checkins(a)python.org > > https://mail.python.org/mailman/listinfo/python-checkins > > > > > [View Less]

2 1

Re: [Python-Dev] [Python-checkins] cpython (3.3): minor docfix (reported at docs@python.org) trace.py argument is --count not
by Benjamin Peterson April 6, 2014

April 6, 2014

On Sun, Apr 6, 2014, at 11:01, senthil.kumaran wrote: > http://hg.python.org/cpython/rev/b49d990aaa9d > changeset: 90160:b49d990aaa9d > branch: 3.3 > parent: 90130:63e6afd3ff1f > user: Senthil Kumaran <senthil(a)uthcode.com> > date: Sun Apr 06 10:59:47 2014 -0700 > summary: > minor docfix (reported at docs(a)python.org) trace.py argument is --count > not --counts This change is of course not harmful, but remember that 3.3 is in security-fix only mode.

1 0

Pyston: a Python JIT on LLVM
by Kevin Modzelewski April 5, 2014

April 5, 2014

Hi all, I'm excited to announce Pyston, a Python JIT under development at Dropbox, built on top of LLVM. You can read more about it at the introductory blog post<https://tech.dropbox.com/2014/04/introducing-pyston-an-upcoming-jit-based-p…>, or check out the code on github <https://github.com/dropbox/pyston>. Since it's the question that I think most people will inevitably (and rightly) ask, why do we think there's a place for Pyston when there's PyPy and (previously) Unladen … [View More]

4 6